https://arxiv.org/api/Xc4y03TuWIs6Wx3y3OsBXn8tXWg 2026-10-02T18:45:12Z 12468 165 15 http://arxiv.org/abs/2609.11251v1 2026-09-10T08:46:09Z

Blockchains usually pay for authenticated reads by maintaining a structure that spans the entire state. We show how validators can support historical membership and exclusion proofs by authenticating each block's writes instead. A Block Update Digest (BUD) commits a write log whose predecessor pointers link successive modifications of each key. A SuperBUD summarizes last writes over a window; an exponential hierarchy turns long unchanged intervals into short proofs. The digest count is logarithmic in the gap within the hierarchy's range, with one additional digest per top-level window beyond it. We prove soundness against adversarial provers and up to f Byzantine validators, and completeness for queries anchored by a post-deployment modification, assuming archive, attestation, and committee evidence is available. Across a 50x increase in state size, the measured base-BUD path rises by 1.24x, compared with 3.1x and 69.5x for in-memory and cache-bounded disk-backed Merkle Patricia tries. On the synthetic trace, two-digest read-layer payloads stay below 800 bytes, and warm hash-path verification takes at most 146 microseconds at p99.

2026-09-10T08:46:09Z Alejandro Ranchal-Pedrosa Cody Littley Ben Marsh http://arxiv.org/abs/2609.11148v1 2026-09-10T06:52:15Z

Data curation is a critical bottleneck in industrial-grade LLM development, where petabyte-scale unstructured corpora are scattered across hundreds of physical tables, feature engineering relies on manual, table-centric pipeline orchestration, and data lineage is largely absent. We present OmniTable as an architecture blueprint for a unified wide-table layer built on Logical Unification, Physical Separation, targeting petabyte-scale LLM data curation and exploration. OmniTable makes four contributions: (1) a unified wide-table abstraction that consolidates multi-source heterogeneous data and thousands of derived features under a single logical schema via logical-physical mapping; (2) declarative feature lifecycle management that automates dependency resolution, execution planning, operator fusion, and lineage tracking, replacing manual pipeline orchestration with a "declare-and-execute" paradigm; (3) an adaptive execution engine with autonomous governance that achieves stable PB-scale feature backfill through heterogeneous compute routing (CPU/GPU), adaptive tuning, UDF-level fault tolerance, and automated storage layout optimization; and (4) hybrid-accelerated data exploration combining a global ID index, transparent OLAP offloading, and background materialized views to deliver second-level point lookups and filtered exports exceeding 20 TB/hour. In production, OmniTable manages over 35 PB of training data across web, code, PDF, and SFT domains, reducing the human-in-the-loop curation cycle from approximately 14 days to approximately 2.5 days (5.6x over the pre-OmniTable production workflow), with consistent feature versioning, auditable lineage, and minimal manual intervention.

2026-09-10T06:52:15Z VLDB 2026 Best Industry Paper Proceedings of the VLDB Endowment 19(12):4276-4289, 2026 Yuzhuo Fu Xiangchun Wang Chao Huang Liyi Wang Binwei Zeng Yuhan Wang Taotao Nie Dongke Hu Wang Hong Jiayi Wang Wenwen Cui Zhuyan Zhou Yushun Guo Yuhan Xing Jiaxin Lian Peng Lin Qing Cui Wenhui Shi Jun Zhou 10.14778/3827998.3828032 http://arxiv.org/abs/2605.31475v5 2026-09-09T12:28:44Z

DBLog is a change-data-capture (CDC) mechanism for copying a table or selected keys from a source database while continuing to stream new changes from its commit log. It reads a table in primary-key ordered chunks and brackets each read with low and high watermarks in the source log. A chunk row is discarded if a log event for the same key appears anywhere in that window. The remaining rows are emitted as refresh events after the high watermark. DBLog allows backfills to run at any time during normal operation, rather than only as an initial bootstrap, while writes and live capture continue. It was introduced in the 2019 Netflix Tech Blog post and further discussed in the 2020 DBLog paper. The mechanism has since been adopted by open-source projects, including Debezium and Apache Flink CDC. The original blog post and paper explained this mechanism but did not provide a formal correctness proof or a precise description of its replay result. In this paper, we formalize how chunk reads and log events are combined and prove that their replay reconstructs the source state up to a specific log position for the keys being copied. We define this as a virtual cut, representing a snapshot-equivalent replay without requiring a physical snapshot. DBLog emits the events needed to reconstruct the source state downstream, without storing that state itself. To verify executions independently, we provide a certificate that validates this equivalence from recorded log evidence and chunk observations. Once chunk processing completes and covers all keys, this guarantee extends to the whole table. Furthermore, appending subsequent change-stream events advances the cut forward, formalizing the seamless transition from historical backfill to live streaming. All definitions and proofs are mechanized in Isabelle/HOL.

2026-05-29T16:04:34Z 28 pages, 5 figures. Machine-checked Isabelle/HOL formal development included as an ancillary file and archived at https://doi.org/10.5281/zenodo.21732790 Andreas Andreakis http://arxiv.org/abs/2609.09978v1 2026-09-09T10:06:44Z

Bounded SQL equivalence checkers search for a counter-example database of bounded size, and a search that comes back empty proves nothing. We supply missing theory: computable bounds $B$ such that agreement on all databases with at most $B$ tuples per relation implies equivalence. We work in the combined-semantics framework, which captures SQL's mix of duplicate-eliminating (DISTINCT) and duplicate-preserving computation over set-valued relations. For conjunctive queries we prove a bound linear in the query size for fixed multiset width: inequivalent queries already disagree on a database with at most $2^w |Q|$ tuples, where the width $w$ counts only the columns the queries actually read, independently of the total number of multiset variables. Declared keys shrink the bound to $2^{kw} |Q|$ for the smaller key-width $kw$, acyclic foreign keys leave it unchanged, and the result extends to several classes of queries with comparisons, for which equivalence had not previously been characterized. For these fragments, bounded search becomes a terminating, complete decision procedure.

2026-09-09T10:06:44Z 27 pages, including 10 pages of appendices; no figures Sara Cohen http://arxiv.org/abs/2609.09867v1 2026-09-09T08:20:19Z

Post-training quantizers select finite code changes using reconstruction proxies or local loss approximations, but the utility of a quantization move depends on the state through which it is executed. We identify two sources of this contextual dependence. First, the displacement of the move matters: evaluating the gradient at the move midpoint captures curvature accumulated along the move that a current-state linearization omits. Across frozen two-bit moves from Llama-3.2 models, midpoint evaluation predicts the direction of exact endpoint loss changes substantially more accurately than current-state gradients. Second, moves interact: exhaustive lattices of legal quantized states are well approximated by quadratic pseudo-Boolean functions, yet their small pairwise components can determine Pareto fronts and cause different evaluation functionals to prefer opposite directions. These effects explain failures of reconstruction-optimal code re-selection and additive composition. Reading each move at its own midpoint repairs the local selection step and improves downstream accuracy and held-out perplexity, while larger supports require evaluating exact endpoints from the state actually reached. Exact-endpoint beam search finds sparse changes that dominate much larger one-shot updates, and repricing the same moves after intervening changes produces widespread sign reversals. These results show that quantization utility is contextual at the granularity of a few moves: reliable construction must evaluate finite changes along their own paths and compose them from the evolving quantized state.

2026-09-09T08:20:19Z Preprint. 5 figures. Includes appendices Wenxuan Xiao Xu Cao http://arxiv.org/abs/2609.09854v1 2026-09-09T08:06:50Z

Low-bit quantization can achieve high recall on some vector representations and fail sharply on others, while average distortion and global rank correlation do not explain the difference. We study quantized vector search at the level of the comparisons consumed by ranking and graph-pruning algorithms. Our first result is a distribution-free decomposition: the probability that a comparison flips is bounded by the probability mass of exact margins near zero plus the tail probability of the calibrated residual. We then account for dependence between residuals that share a query or graph node, and derive covariance-aware second-moment identities and tail bounds under a joint MGF proxy. For a frozen candidate permutation, we prove a deterministic coupling theorem for Vamana neighbour selection: the approximate replay returns the exact neighbour list exactly when all candidate-level pruning actions agree on the frozen exact states. We connect these results to representation geometry through an exact Gaussian oracle, establish a strict correlation gain from a deterministic magnitude bit in an aligned bilinear model, and give a rare-contamination construction showing why marginal Gaussian diagnostics do not imply the required residual tails. When analytical assumptions are unavailable, a held-out block certificate bounds the selective failure risk of a frozen quantized rule. Across learned, classical, and synthetic embeddings, standardized exact margins predict held-out ranking and pruning flip rates substantially better than global rank correlation. The framework applies to coordinate binary codes, RaBitQ, Lucene BBQ, and product quantizers through a common decision interface.

2026-09-09T08:06:50Z JMLR-style preprint with theoretical and experimental appendices Wenxuan Xiao Xu Cao http://arxiv.org/abs/2609.09671v1 2026-09-09T03:38:38Z

When an agent writes code, the development framework becomes the control system for a non-deterministic worker. Spec-first, agent-driven frameworks have gained rapid traction since 2025; the installable ones, GitHub Spec Kit, obra/superpowers, BMAD, and GSD, and our own, all capture intent through a specification or durable planning artifacts. Since they agree on capturing intent up front, what separates them is how each enforces the engineering discipline that keeps agent-written code clean, correct, and maintainable. Every framework enforces that discipline somehow; they differ in how. We characterize three modes: enforcement by persuasion (prompt discipline the model may ignore), by front-loaded structure (strong specs, then a trusted build), and through controls the agent cannot edit (a deterministic orchestrator, human-approved gates, immutable tests, and a green result that must pass against a live, branched database). We introduce Consort, a spec-first, test-driven agent framework built on the third, enforcing that discipline through controls the agent runs inside but cannot bypass, in which a deterministic orchestrator drives separate role agents through a spec-first design lane and a test-driven build lane on a live database branch. We argue that enforcing the tests and gates in code keeps agent-written code honest and verifiable, while its specialized roles, like the human roles before them, are what make it maintainable, claims we frame as a pre-registered, testable hypothesis.

2026-09-09T03:38:38Z 9 pages, 2 figures, 3 tables, for associated framework, see https://github.com/databricks-solutions/consort Kevin Hartman http://arxiv.org/abs/2608.11889v2 2026-09-08T22:58:19Z

Prompting-based (i.e., non-fine-tuning) Text-to-SQL methods, where underlying large language model parameters are not changed for the task, face three problems: (i) relying on coarse-grained schema information that may not reveal the fine-grained relationships needed to distinguish ambiguous columns, (ii) failing to capture recurring SQL-generation failures, and (iii) suffering from omission or hallucination of components in complex questions. This paper develops DexterSQL, a prompting/non-fine-tuning-based Text-to-SQL system that improves SQL generation with three novel components: (i) deep schema explorator that identifies ambiguous columns, analyzes their individual and joint data distributions to uncover their relationships and the distinct role of each, (ii) database-agnostic rule creator that mines mismatches between generated and gold SQL only on the training database and converts them into database-agnostic corrective rules that capture recurring LLM failure patterns; and (iii) multi-path SQL generation that introduces a dependency-tree-based intermediate representation that uses the question's sentence structure to guide its decomposition into an SQL skeleton for final SQL generation. DexterSQL achieves a higher accuracy compared to the state-of-the-art using both open-source/weight and closed-source/weight models. Particularly, DexterSQL shows a high improvement of at least 5.5% using an open-weight model (GPT-OSS-120B) on BIRDDev, with total accuracy 70.4%. DexterSQL also shows better improvement of at least 1.4% using closed-weight models, with total accuracy 72.1% and 72.9% on BIRD-Dev with GPT-4o and GPT-5.2.

2026-08-12T10:17:08Z This version of the paper improved the SQL generation algorithm, increasing the system's overall accuracy. For details, please see the paper Anik Pramanik Murat Kantarcioglu Vincent Oria Shantanu Sharma http://arxiv.org/abs/2609.09002v1 2026-09-08T16:40:36Z

Many-to-many joins are central to analytical and semantic workloads such as fraud detection, network analysis, and recommendation, where insights arise from relationships between entities. These workloads often suffer from an explosion of intermediate results, sometimes orders of magnitude larger than the inputs. Factorized representations address this problem by exploiting conditional independence among attributes to encode intermediates more compactly. In some cases, they can reduce the output size asymptotically below the worst-case output size. However, adopting factorization in modern vectorized query processors remains challenging: factorized representations are hierarchical, whereas vectorized execution is built around flat, block-oriented processing. Prior approaches either rely on full materialization or support only restricted factorization layouts, sacrificing much of the benefits of both factorization and vectorization. We present FFX, a novel engine for Fast Factorized eXecution. FFX is the first pipelined engine to support arbitrary factorization schemes while preserving full vectorization. The engine introduces packed factorized vectors and operators that maintain cache-friendly, contiguous layouts. Beyond analytics, FFX also co-optimizes semantic operators by serializing factorized intermediates into compact prompts for large language models (LLMs), substantially reducing token usage and inference cost while maintaining output quality and, in some cases, improving it. Together, these contributions enable efficient execution of join-heavy analytical queries, including queries augmented with semantic operators.

2026-09-08T16:40:36Z Proceedings of the ACM on Management of Data, 4(3), Article 178, 2026 Sunny Yasser Anas Dorbani Amine Mhedhbi 10.1145/3802055 http://arxiv.org/abs/2609.08950v1 2026-09-08T16:08:43Z

Text-to-SQL systems translate natural language queries into executable SQL, democratizing access to structured data. Despite recent advances driven by large language models (LLMs), evaluation remains a major bottleneck: public benchmarks fail to capture the complexity of enterprise schema, while building private evaluation sets is costly and nondeterministic, making evaluation results difficult to reproduce. To address this issue, we present SQLMorph, a framework for Text-to-SQL evaluation via query mutation. SQLMorph introduces two techniques to automatically generate and expand evaluation sets: Join Query Expansion (JQE), which systematically increases structural complexity through valid join additions, and Textual Query Augmentation (TQA), which generates controlled natural language perturbations to assess robustness to linguistic variation. JQE and TQA create targeted choke points to challenge specific system components. When applied to state-of-the-art systems, JQE increases query coverage and reveals accuracy degradation as the number of joins grows. Meanwhile, TQA shows that linguistic brittleness induced by heavy abbreviation can reduce accuracy by up to 17%. Beyond evaluation sets, SQLMorph introduces a family of execution-level metrics that address the limitations of current binary measures, such as Execution Accuracy. We define Execution Precision (EXP) and Execution Recall (EXR) to quantify the fraction of correct and recovered results, respectively, and combine them via F1 for unified scoring. Our experiments show that these relaxed metrics enable fine-grained analysis of over- and under-prediction, revealing differences across systems that binary metrics obscure. Together, SQLMorph's query mutation and fine-grained metrics support debugging and better align Text-to-SQL evaluation practices with real-world deployments.

2026-09-08T16:08:43Z 2026 IEEE 42nd International Conference on Data Engineering (ICDE), pp. 2628-2640 Mohammadhossein Malekpour Mohamed Riahi Maxime Lamothe Amine Mhedhbi 10.1109/ICDE65706.2026.00196 http://arxiv.org/abs/2607.01605v2 2026-09-08T15:33:07Z

CityGML, the OGC standard for modeling, storage, and exchange of semantic 3D city models, describes urban objects with detailed semantics, geometry, and topology. Yet this richness is difficult to query directly: CityGML's XML encoding is designed for exchange rather than analysis, and relational mappings expose it through schemas requiring expert knowledge. We present pykci (Python Knowledge Graph for Cities), an open-source system that transforms CityGML 2.0 datasets into a compact urban knowledge graph in Neo4j and makes it queryable in natural language. The graph schema covers all thematic feature modules of CityGML 2.0 across all levels of detail and is spatially indexed with an R-tree for efficient geometric retrieval. A complete end-to-end Python pipeline ingests CityGML datasets into the knowledge graph, exports them to OGC 3D Tiles for interactive visualization, and supports lossless round-trip export of all content back to CityGML. For querying, the graph is paired with a large language model through a model-agnostic text-to-Cypher mechanism: the graph schema is supplied as context, and the model translates natural-language questions into Cypher queries executed against the graph. We evaluate both a locally running open-weight model, which keeps sensitive city data on-premise, and a state-of-the-art commercial model for the most demanding spatial and semantic queries. Answers are grounded in exact city data rather than the model's parametric memory, reducing hallucination and providing auditable provenance for every response. We demonstrate the system on open-government CityGML LoD2 datasets from Hamburg, Germany, including complex semantic and spatial queries such as identifying roof surfaces suitable for greening. pykci enables urban planners, GIS practitioners, and citizens to interact with semantic 3D city models without expertise in query languages and database schemas.

2026-07-02T02:07:54Z Huynh Duc An Son Nguyen Lukas Arzoumanidis Youness Dehbi 10.1145/3841645.3843314 http://arxiv.org/abs/2609.08869v1 2026-09-08T15:11:38Z

Analysts in emerging equity markets keep answering the same questions. Did fundamentals match the market's response? How does the local currency co-move with returns? Which firms outperform sector and benchmark, and which disclosures coincide with abnormal trading? These answers come from ad-hoc spreadsheets that are hard to reproduce, audit, or trust. We present OntoKG-EQ, a knowledge-based system that makes such queries reproducible, evidence-linked, temporally explicit, valid, and inspectable. It couples a bounded, competency-question-governed core ontology with a provenance-aware knowledge graph in which every class, property, shape, and metric is justified by one of five frozen questions. The system materialises market data into the graph, computes the metrics, validates its structure against declarative shape constraints, answers each competency question with a graph query, derives typed findings, and generates an explanation tracing each result to its observations, evidence, sources, and provenance. We evaluate on curated datasets from three emerging markets (Pakistan, Malaysia, Indonesia). Once each market's data is mapped into the common schema, the ontology, shapes, queries, and rules are reused unchanged. A relational-database baseline shows the graph changes no analytics. Its value is governance, provenance, and self-explaining structure. Because answers are rendered deterministically from the validated graph, their consistency with it is guaranteed by construction. Used as a reference, the system measures how consistently eight open language models transcribe the same evidence (provenance coverage 0.00 to 1.00). A study with a 17-participant convenience panel finds the evidence bundle significantly increased perceived trust and completeness. Code and data are openly released.

2026-09-08T15:11:38Z 36 pages, 2 figures Furqan Nasir Muhammad Atif Saeed Muhammad Ehsan Sher Jeel Ahmad Abdul Moiz Altaf http://arxiv.org/abs/2609.08556v1 2026-09-08T10:38:26Z

With the rapid development of aerospace technology and the large-scale deployment of low Earth orbit (LEO) constellations, the risk of orbital collisions has increased, creating a growing demand for reliable observations of satellite maneuvers. However, public datasets containing real maneuver records remain scarce. We present MAD-LEO, a Maneuver-Annotated orbital Dataset for LEO satellites. The mission-reported subset contains 1,134 maneuver events from eleven geodetic and altimetry satellites spanning 1992 to 2026, with labels taken directly from mission-published maneuver histories. Each event is checked against two-line element (TLE) data, precise orbit products, and satellite laser ranging (SLR) observations, with evidence tiers assigned according to data availability. The operational subset pairs operator-published ephemerides for 6,785 Starlink satellites with cataloged TLE records over a continuous 107-hour period. Technical validation across seven machine-readable experiment suites confirms the cross-source consistency of the labels and the evidence products.

2026-09-08T10:38:26Z Zhixin Guo Qi Shi Xiaofan Xu Linqiang Ge Hua Zhu Liyan Ben Bendian Nie Yuanrui Zhao Xiaohan Li http://arxiv.org/abs/2609.08508v1 2026-09-08T09:55:10Z

Knowledge graphs have become a key resource for integrating heterogeneous data and powering downstream tasks such as question answering, entity linking, and semantic search. They are built and maintained incrementally, either (i) fully automated, e.g., YAGO, (ii) semiautomatically with community oversight, e.g., DBpedia, or (iii) manually through collaborative editing, e.g., Wikidata. Understanding the evolution of knowledge graphs is essential as changes may reflect real-world updates, error corrections, or noise introduced by vandalism, all of which affect the reliability of downstream applications. Among openly available knowledge graphs, Wikidata is the most challenging case to study evolution, with over 120 million entities edited by humans and bots and an edit history spanning more than a decade. Although Wikidata exposes change data in various formats (e.g., periodic dumps and real-time event streams), none support analytical queries over the complete edit history. Therefore, we present WiDiff, a tool that extracts changes from Wikidata's complete edit history and provides a unified interface for large-scale analytical queries over it.

2026-09-08T09:55:10Z Carolina Cortés Lisa Ehrlinger Lorena Etcheverry Felix Naumann http://arxiv.org/abs/2609.08279v1 2026-09-08T05:48:10Z

Agent memory systems must discard stored information when their history exceeds a fixed token budget. Existing budget-accuracy frontiers quantify the resulting loss in accuracy, but do not distinguish irreversible losses caused by eviction from recoverable retrieval failures. We introduce the restore counterfactual, a per-question paired intervention that reinstates the question's gold evidence in the read-time context and reruns the same reader. Combining the change in correctness with whether the evidence was retained after eviction classifies each oracle-answerable error as recoverable, irreversible, or residual; in the residual case, the answer remains incorrect after restoration. We evaluate FIFO, random, redundancy-aware, and LLM-importance eviction on LongMemEval-S at three budgets and under two retrieval regimes, using GPT-4o-mini as the primary reader and judge and GPT-5.4-mini as a robustness reader. Under top-k retrieval at an 80k-token budget, the irreversible share among errors corrected by restoration is 0.67-0.73 for FIFO, random, and redundancy-aware eviction, compared with 0.60 for LLM-importance. At 8k tokens, it reaches 1.00 for all four policies. Recoverable errors occur under top-k retrieval at 80k tokens but are absent under forced-gold injection by construction, so budget-accuracy results are not directly comparable unless the retrieval regime is reported. An exploratory matched-accuracy analysis detects no difference in irreversible rate among accuracy-matched policy pairs at a resolution of 1.2-6 percentage points. The same analysis detects the deliberately destructive control. To our knowledge, this is the first per-item, per-question restore-counterfactual audit of eviction for external agent-memory stores on a standard conversational benchmark.

2026-09-08T05:48:10Z Chen Shen