https://arxiv.org/api/ZyxFQq/TC3ap+MclYu5p5bOygJM 2026-07-17T21:43:11Z 12022 30 15 http://arxiv.org/abs/2603.16450v2 2026-07-12T16:19:08Z

Apache Spark SQL is a cornerstone of modern big data analytics.However,optimizing Spark SQL performance is challenging due to its vast configuration space and the prohibitive cost of evaluating massive workloads. Existing tuning methods predominantly rely on full-fidelity evaluations, which are extremely time-consuming,often leading to suboptimal performance within practical budgets.While multi-fidelity optimization offers a potential solution, directly applying standard techniques-such as data volume reduction or early stopping-proves ineffective for Spark SQL as they fail to preserve performance correlations or represent true system bottlenecks. To address these challenges, we propose MFTune, an efficient multi-fidelity framework that introduces a query-based fidelity partitioning strategy, utilizing representative SQL subsets to provide accurate, low-cost proxies. To navigate the huge search space, MFTune incorporates a density-based optimization mechanism for automated knob and range compression, alongside an adapted transfer learning approach and a two-phase warm start to further accelerate the tuning process. Experimental results on TPC-H and TPC-DS benchmarks demonstrate that MFTune significantly outperforms five state-of-the-art tuning methods, identifying superior configurations within practical time constraints.

2026-03-17T12:31:13Z Accepted at VLDB 2026 Beicheng Xu Lingching Tung Yuchen Wang Yupeng Lu Bin Cui http://arxiv.org/abs/2607.10771v1 2026-07-12T13:59:54Z

Matching dependency is a generalization of the functional dependency concept, which allows users to apply custom similarity functions for matching individual attributes. Matching dependencies have a wide range of applications for solving various data quality problems, such as entity resolution, data deduplication, data integration, schema matching, and many more. However, their discovery is a very computationally intensive problem, which limits their practical application. In this paper, we describe a number of optimization techniques for HyMD - currently the state-of-the-art algorithm for the discovery of matching dependencies. These optimizations belong to both technical and scientific domains. The most important of them are: 1) a new sampling technique, 2) a faster generalization lookup technique, and 3) an improved representation of a dependency. The first one aims to raise the efficiency of inference from record pairs, while the last two are designed to speed up lattice-related operations. To evaluate our optimizations, we implemented our version of HyMD in Desbordante, an open-source high-performance data profiler. Experiments demonstrated that they allow for a speedup of more than 40x over the state-of-the-art implementation on average, reaching a speedup greater than 170x in some cases. Finally, the improved version of HyMD is ready to use by anyone. It comes with bidirectional Python integration, which allows calling the C++ algorithm implementation from Python programs while allowing users to supply their custom matching functions.

2026-07-12T13:59:54Z 2024 36th Conference of Open Innovations Association (FRUCT), Lappeenranta, Finland, 2024, pp. 729-740 Alexey Shlyonskikh Michael Sinelnikov Daniil Nikolaev Yurii Litvinov George Chernishev 10.23919/FRUCT64283.2024.10749955 http://arxiv.org/abs/2607.10508v1 2026-07-11T23:46:01Z

LLM agents that conduct research (proposing ideas, writing and running code, analyzing results) can already carry a study from research question to figures, yet cannot be fully trusted. The same question asked twice in a row returns different answers; the agent announces a number that no execution produced, and tool use does not prevent this, because nothing binds what the agent reports to what its tools returned; a small upstream change leaves downstream results silently stale, with no way to list which ones; and the agent re-runs preprocessing and rewrites code it has already produced. We argue these failures share one root: every step of today's agent loop is a stochastic LLM call whose internal state nobody, including the agent, can check. Rather than trying to see inside the LLM, we take a lesson from databases, which earn trust without being watched, because deterministic operators over well-defined state make their guarantees hold by construction. We propose organizing a research project the same way. The project lives in a deterministic, versioned dataflow engine (in effect, a query plan over materialized views), and the LLM, together with the user, is a stochastic compiler that may only edit that plan. The executor never calls the LLM; LLM output enters only as versioned code and data that the executor then runs, and any asserted result enters the record only with an execution behind it. Five design rules at this boundary turn familiar database machinery, from versioning and provenance to incremental maintenance and cost-based scheduling, into guarantees that make research reliable, non-wasteful, transparent, and collaborative. This report presents the diagnosis, the requirements, and the design; the guarantee walkthrough, a prototype, and the research agenda appear in the full version, in preparation. The LLM, we argue, should be the query compiler, never the executor.

2026-07-11T23:46:01Z Kyoungmin Kim Anastasia Ailamaki http://arxiv.org/abs/2607.10444v1 2026-07-11T19:02:59Z

A data management system can be separated in typical data processing systems. Unfortunately, relational data management systems are not efficient enough to handle the on-line signal processing task in a monitoring system. The main current in research into database management system model for the needs of monitoring systems is connected with a data stream model. However, these systems are non-deterministic. This paper presents the developed methods of data stream processing for signal processing tasks in medical database management systems, as well as the developed theorems of data sequences (stream) algebra with formal proofs. A direct link between some introduced operators and Beatty and Fraenkel theorems has been proved

2026-07-11T19:02:59Z 18 pages, 1 figure. Author-initiated deposit of the version published in Annales UMCS Sectio AI Informatica, Vol. 4 (2006), pp. 314-331, under a CC BY 4.0 license. Establishes the stream algebra and the Beatty/Fraenkel foundations later developed in arXiv:2607.07730 Annales UMCS Sectio AI Informatica, Vol. 4, No. 1, pp. 314-331, 2006 Michal Widera http://arxiv.org/abs/2606.20853v2 2026-07-11T16:12:37Z

Heuristic query rewriting has long complemented cost-based optimization to improve performance. Such rewrites transform SQL queries into semantically equivalent forms that are easier or faster to execute. Examples are standardizing expressions, eliminating redundancy, propagating constants, pushing down selections and projections, unnesting queries, and utilizing constraints. Modern DBMSs implement hundreds to thousands of such rules, but maintaining them is notoriously difficult. The interactions among rules are complex, and their static nature and application order prevent adaptation to specific query and database characteristics. Recent approaches that use large language models (LLMs) for query rewriting show promise but face challenges regarding the large search space, reliable query verification, and exploitation of metadata. We present ReSequel, an outer optimization layer on top of existing DBMSs to rewrite SQL queries using LLMs. ReSequel leverages catalog and statistical metadata to infer template-specific rules that guide the LLM toward effective query transformations. We generate, verify, and rank rewritten query variants on sampled data to ensure result correctness and runtime improvements. Our experiments cover eight benchmarks: JOB, TPC-H, Stats(-CEB), Public BI, IMDB, DSB, and StackOverflow; multiple DBMSs: PostgreSQL, MySQL, and DuckDB; as well as LLM-based query rewriting baselines. ReSequel yields workload-level speedups of up to 16x over native DBMSs and 22x over LLM-based systems, with individual queries exceeding 600x, across eight benchmarks and three DBMSs.

2026-06-18T18:37:05Z Proc. VLDB Endow., 19(10):2880-2893, 2026 Saeed Fathollahzadeh Essam Mansour Matthias Boehm 10.14778/3828612.3828639 http://arxiv.org/abs/2602.23342v4 2026-07-11T13:41:03Z

On-disk graph-based approximate nearest neighbor search (ANNS) is essential for large-scale, high-dimensional vector retrieval, yet its performance is widely recognized to be limited by the prohibitive I/O costs. Interestingly, we observed that the performance of on-disk graph-based index systems is compute-bound, not I/O-bound, with the rising of the vector data dimensionality (e.g., hundreds or thousands). This insight uncovers a significant optimization opportunity: existing on-disk graph-based index systems universally target I/O reduction and largely overlook computational overhead, which leaves a substantial performance improvement space. In this work, we propose AlayaLaser, an efficient on-disk graph-based index system for large-scale high-dimensional vector similarity search. In particular, we first conduct performance analysis on existing on-disk graph-based index systems via the adapted roofline model, then we devise a novel on-disk data layout in AlayaLaser to effectively alleviate the compute-bound, which is revealed by the above roofline model analysis, by exploiting SIMD instructions on modern CPUs. We next design a suite of optimization techniques (e.g., degree-based node cache, cluster-based entry point selection, and early dispatch strategy) to further improve the performance of AlayaLaser. We last conduct extensive experimental studies on a wide range of large-scale high-dimensional vector datasets to verify the superiority of AlayaLaser. Specifically, AlayaLaser not only surpasses existing on-disk graph-based index systems but also matches or even exceeds the performance of in-memory index systems.

2026-02-26T18:48:29Z The paper has been accepted by SIGMOD 2026 Weijian Chen Haotian Liu Yangshen Deng Long Xiang Liang Huang Bo Tang http://arxiv.org/abs/2607.10265v1 2026-07-11T11:47:26Z

Temporal graph questions require reliable handling of time, identifiers, and arithmetic. Large language model (LLM) agents often fail on these tasks, especially when a graph records both ordinary evolution and later corrections. We present TGMS, a bi-temporal property graph management system that exposes thirteen verified temporal operators as agent tools. Each operator is typed, deterministic, bounded, cost-guarded, and bi-temporal by default. The LLM plans operator calls and writes the final response, while the system performs all graph computation. Numeric, entity, ordering, and pattern claims are checked against the content-addressed execution trace. TGMS separates valid time from transaction time. It can therefore answer belief-state questions such as ``as of transaction time $T$, what did the system believe?'' Standard latest-state snapshots and retrieval pipelines do not preserve enough information to answer such questions. On a development benchmark built from a real communication network, TGMS with a 14B open-source model reaches 0.409 exact match. Vector-RAG, static-graph RAG, and text-to-Cypher reach 0.045--0.182 under the same serving setup. TGMS reaches 0.67 exact match on correction probes, while the three 14B baselines score zero. The claim verifier detects all 500 injected count and entity errors with no false positives on the clean answers. Two implementation findings were especially important. First, operator output contracts prevent plans from referring to fields that do not exist. Second, verification must track whether the cited evidence is complete, because correct arithmetic over a truncated result is still misleading. The code, benchmark, and trace viewer are open source under Apache-2.0.

2026-07-11T11:47:26Z Xiaofei Zhang http://arxiv.org/abs/2406.06761v7 2026-07-10T19:41:55Z

We present Wally, a batched private nearest-neighbor search protocol that uses differential privacy to break the linear computation barrier of fully-oblivious schemes. In Tiptoe, the server must process the entire database per query to hide the access pattern, resulting in low throughput (909 QPS) and high communication (17.4 MB) on a 3.2M-entry database. Sublinear alternatives like Pacmann require 614 MB of client storage and an offline streaming phase. Wally's key insight is that fully-oblivious schemes are prohibitively expensive at scale, but the same scale also provides an opportunity. Large-scale systems naturally have many concurrent clients. Wally batches queries from non-coordinating clients, each independently adding fake queries to hide which clusters it accesses. The fake query counts follow a negative binomial distribution, which is non-negative and infinitely divisible, allowing independent sampling without coordination. Clients send queries at random times through an existing anonymization service, avoiding a centralized shuffler. The server sees only an anonymized, noisy stream of cluster accesses that is provably (epsilon, delta)-differentially private, computing over only the relevant clusters. The client encrypts its query under SHE so the server returns only encrypted similarity scores. On a 3.2M-entry database with 500K-query batches, Wally achieves 7-29x higher throughput and 6.7-31x lower communication than Tiptoe, and 15,000x lower client storage than Pacmann, with strong (epsilon=0.1, delta=2^{-26})-DP and comparable accuracy. We also propose optimizations to SHE and keyword PIR yielding 2-3x improvements in PIR and 20-25% in BFV operations, and release an open-source BFV library in Swift.

2024-06-10T19:41:25Z Hilal Asi Fabian Boemer Nicholas Genise Muhammad Haris Mughees Tabitha Ogilvie Rehan Rishi Kunal Talwar Karl Tarbe Akshay Wadia Ruiyu Zhu Marco Zuliani http://arxiv.org/abs/2602.21247v3 2026-07-10T19:02:20Z

The fastest indexes for Approximate Nearest Neighbor Search today are also the slowest to build: graph-based methods like HNSW and Vamana achieve state-of-the-art query performance but have large construction times due to relying on random-access-heavy beam searches. We introduce PiPNN (Pick-in-Partitions Nearest Neighbors), an ultra-scalable graph construction algorithm that avoids this ``search bottleneck'' that existing graph-based methods suffer from. PiPNN's core innovation is HashPrune, a novel online pruning algorithm which dynamically maintains sparse collections of edges. HashPrune enables PiPNN to partition the dataset into overlapping sub-problems, efficiently perform bulk distance comparisons via dense matrix multiplication kernels, and stream a subset of the edges into HashPrune. HashPrune guarantees bounded memory during index construction which permits PiPNN to build higher quality indices without the use of extra intermediate memory. PiPNN builds state-of-the-art indexes up to 11.6x faster than Vamana (DiskANN) and up to 12.9x faster than HNSW. PiPNN is significantly more scalable than recent algorithms for fast graph construction. PiPNN builds indexes at least 19.1x faster than MIRAGE and 17.3x than FastKCNA while producing indexes that achieve higher query throughput. PiPNN enables us to build, for the first time, high-quality ANN indexes on billion-scale datasets in under 20 minutes using a single multicore machine.

2026-02-17T02:18:17Z To appear at KDD'26 Tobias Rubel Richard Wen Laxman Dhulipala Lars Gottesbüren Rajesh Jayaram Jakub Łącki http://arxiv.org/abs/2607.09528v1 2026-07-10T15:36:13Z

The emergence of metaverse platforms has created virtual economies that introduce new challenges related to fraud, bot activity, and illicit financial behavior. Despite growing interest in trustworthy metaverse analytics, existing datasets typically focus on user behavior, authentication, or financial transactions in isolation, limiting the development and reproducible evaluation of multimodal fraud detection methods. To address this gap, we present TSAI-MetaFraud, a multimodal, multi-task benchmark dataset for fraud analytics in virtual economies. TSAI-MetaFraud integrates behavioral, transactional, and graph-structured information while incorporating realistic fraud and automated bot scenarios. We define benchmark tasks including transaction fraud detection, cross-modal node classification, temporal link prediction, and weakly supervised fraud detection, and provide baseline evaluations using machine learning models and graph neural networks. By jointly capturing behavioral activity, financial interactions, and relational structure within a unified virtual economy, TSAI-MetaFraud provides a benchmark for advancing multimodal learning, graph mining, fraud analytics, and trustworthy AI in emerging metaverse ecosystems.

2026-07-10T15:36:13Z Refat Ishrak Hemel Ehsan Hallaji Roozbeh Razavi-Far http://arxiv.org/abs/2607.09251v1 2026-07-10T10:04:18Z

Statement-level SQL rewriting can improve query performance and maintainability without changing the DBMS kernel, but existing benchmarks do not evaluate rewrite methods as deployable systems. They typically focus on DBMS performance, rule regression, query equivalence, or dialect translation, while missing the full path from accepting an input query to producing an executable, result-consistent, and operationally useful rewrite. We present SQL-RewriteBench, a benchmark for statement-level SQL rewriting that applies correctness gating and full-denominator accounting. Its metric suite explicitly separates Source Acceptance, Generation Rate, Execution Coverage, Result Consistency, UnsafeRewrite Rate, and speedup distribution. It also defines SCS, a deterministic index of static SQL structure, and CGOQ, a correctness-gated optimization-quality score that gives optimization credit only after the case-specific Checker Contract is satisfied. CGOQ combines runtime improvement with structural simplification through a continuous scoring function, making it suitable for deployment-oriented rewrite assessment. As an artifact, SQL-RewriteBench provides 180 executable Benchmark Instances organized into EQUIV, PERF, ROBUST, and DIALECT pools, each packaged with SQL, schema metadata, provenance, evidence, and rewrite-opportunity documentation. Across seven representative academic and LLM-based methods, every full-benchmark CGOQ is negative. Existing methods often fail before rewriting, fail result checks, or return correct rewrites that are slower or no better than the input. These results show that deployable SQL rewrite requires broader input handling, result validation, and benefit-aware rewrite decisions.

2026-07-10T10:04:18Z Jiang Long Zhejiang University, Hangzhou, China Huawei Company, China Tianci Gao Huawei Company, China Shiyuan Hao Huawei Company, China Haochen Zhang Huawei Company, China Shuncheng Liu Huawei Company, China Jiang Zhang Huawei Company, China http://arxiv.org/abs/2607.01182v2 2026-07-10T07:22:08Z

Filter-and-refine spatial joins have always avoided touching exact geometry for certified candidate pairs, but the field never modeled the decompression cost of the pairs that survive the filter. When geometry is stored in a compressed, progressively-decodable multiresolution codec, the join's true cost is bytes decoded. We study provably-exact polygon intersection joins over a Douglas-Peucker level-of-detail (LOD) ladder, certified by a two-sided Hausdorff-margin test, and make two contributions. First, a reproducible mechanism and harness: on real U.S. Census TIGER water polygons, our progressive certificate join returns the exact join result while decoding 3.4-16.8x (median 5.9x) fewer vertices than naive decompress-then-refine, and about 4.9x fewer than the single-approximation multi-step baseline of Brinkhoff et al. (1994), with zero correctness violations (set-equality against a full-precision oracle) across 31 workloads. Second, a characterization we call the decode-work law: decode work is governed by each pair's signed-clearance margin -- how close it is to the predicate-flip boundary -- independent of object size, because the certificate descends the ladder only until its resolution beats the margin. The law is clean on controlled geometry (held-out R2=0.87, size-independent) and directional on real data (R2 ~= 0.55). We are explicit about what does not hold: a near-boundary-vertex predictor is the wrong model (we pre-registered one and rejected it), a selectivity regime forecaster did not materialize, and the worst case is the trivial Omega(v) read bound on adversarially interleaved boundaries. We contribute the mechanism, budget-honest decode accounting, and an open harness; we do not claim a new index.

2026-07-01T17:14:08Z 8 pages, 2 figures. v2: headline unchanged, better supported. v1 claimed to count LOD header bytes and did not; now charged to all methods (byte advantage 2.02x -> 1.91x; progressive loses at 0.99x on the adversarial fixture). Five pre-registered controls, none reported in v1, are now reported; the relabel null confirms the law. Code: https://github.com/samyama-ai/spatial-join-on-compressed Madhulatha Mandarapu Sandeep Kunkunuru http://arxiv.org/abs/2607.00728v2 2026-07-10T07:18:36Z

Graph approximate-nearest-neighbor (ANN) indexes (HNSW, DiskANN/Vamana) lose recall under insert/delete churn, because deletions orphan the greedy-search paths that route through removed nodes. Production systems restore navigability by repairing the graph on a fixed schedule (consolidate every X operations). We asked whether triggering local edge repair on a measured navigability-degradation signal, rather than a blind clock, spends a fixed repair budget better. At matched repair budget, it does not. On two real ANN datasets (SIFT-128 and Fashion-MNIST-784) under a bursty churn stream, compared against a fixed-cadence baseline actually run at the triggered policy's realized consolidation count, the tail-recall advantage is indistinguishable from zero at every operating point, graph degree, and index scale; at several points the clock is better. We trace our earlier positive result to an interpolated baseline: recall is sharply concave in repair budget -- one consolidation captures over half of all achievable gain -- so reading the baseline off a straight line between zero and four passes understates it by more than the effect claimed. Evaluated by the statistic we pre-registered -- correlation with the subsequent recall drop rather than with the concurrent recall level -- the probe signal is also not a leading indicator. What remains is useful: an exact live-set recall oracle, a reproducible churn harness, a drift-severity regime map, and a budget-parity protocol that makes this error detectable. We report the negative result and the trap, because the trap generalizes: any "at matched budget X" comparison whose baseline is read off an interpolated curve, at the scarce end of a concave response, will manufacture an effect favouring the proposal.

2026-07-01T10:13:20Z 10 pages, 2 figures. v2 is a substantial correction: v1's main result and mechanism claim are withdrawn. v1 compared against an interpolated, never-executed fixed-cadence baseline; running it erases the effect. Four pre-registered controls, asserted in v1 as passing, were unimplemented; all now run, and budget parity fails. Code: https://github.com/samyama-ai/updatable-graph-index Madhulatha Mandarapu Sandeep Kunkunuru http://arxiv.org/abs/2607.09149v1 2026-07-10T07:03:08Z

The rapid growth of scientific publications makes scholarly taxonomies quickly obsolete. We study taxonomy maintenance in the wild, a new problem that moves beyond static construction by continuously adapting taxonomies to evolving scholarly repositories, such as arXiv, for a given research topic. We propose GIST, a robust framework for maintaining evolving taxonomies. Unlike purely LLM-centric approaches, GIST grounds structure induction in expert-curated evidence by extracting partial hierarchies from the "Related Work" sections of papers. It integrates these partial taxonomies into a unified global taxonomy in a geometric box-embedding space, where box containment encodes the inductive bias of is-a relations. To connect semantics with geometric structure, GIST learns a bidirectional mapping between word embeddings and box embeddings. For efficient incremental updates, GIST uses novelty-aware coreset selection to update the model with representative historical signals and new evidence, avoiding costly full retraining. To handle high-velocity paper streams under user-specific token budgets, GIST further combines a hypothesized concept generator with a cost-effective evidence retrieval module. Experiments on real-world arXiv datasets show that GIST outperforms state-of-the-art baselines, improving Node F1 and Edge F1 by 11.0% and 13.1% over the strongest baseline while requiring only 9.6% of its runtime and 12.7% of its monetary cost.

2026-07-10T07:03:08Z The paper has been accepted by SIGMOD 2027 Daomin Ji Hui Luo Zhifeng Bao Junhao Gan Zi Huang http://arxiv.org/abs/2607.08956v1 2026-07-09T21:29:08Z

Traditional vector-based point-in-polygon queries rely on computationally expensive geometric predicates that scale poorly for massive datasets, even when accelerated by spatial indices. Discrete Global Grid Systems (DGGS) offer a scalable alternative by discretizing geometries into hierarchical cells, transforming complex spatial relations into constant-time relational hash-joins. However, adopting a DGGS introduces an overhead to encode data, and current grid implementations exhibit a significant performance ``tooling gap.'' In this demonstration, we present an interactive dashboard that empirically evaluates these computational tradeoffs across four DGGS implementations (H3, S2, A5, and ISEA4H) using DuckDB. Through progressive scenarios, the platform visualizes the overhead of on-the-fly encoding and demonstrates how pre-indexing spatial datasets eliminates this overhead. Ultimately, the demo proves that when data is pre-indexed, all DGGS regardless of their mathematical complexity or tooling converge to sub-second join latencies, unlocking the throughput of modern vectorized execution engines.

2026-07-09T21:29:08Z Levente Juhasz