https://arxiv.org/api/O2QJ8CSDAprpc/BAud5eFyB8YWU 2026-09-11T20:04:50Z 118753 30 15 http://arxiv.org/abs/2604.08485v2 2026-09-10T15:24:21Z

The purpose of this paper is two-fold. First, we show that, after a specified form isometry, the two-coordinate reduction in the binary Hilbert-symbol realization of Chinburg and Zhang is inverse to Kim's building-up construction, up to permutation equivalence. Second, for $q\equiv1\pmod4$, we develop a $q$-ary analogue of this reduction-and-extension mechanism. The identity $c^2=-1$ yields the isotropic line governing the split construction. For every fixed ordered pairing of the coordinates, we obtain a universal rank-$r$ boxed normal form, where $r$ is the dimension of the intersection with the product of these isotropic lines. Applications include optimal self-dual $[6,3,4]$ and $[8,4,4]$ codes over $\mathbb F_{5}$, optimal self-dual $[8,4,5]$ and $[10,5,6]$ codes over $\mathbb F_{13}$, and a self-dual $[12,6,6]$ code over $\mathbb F_{13}$. We also give an exact repeated boxed realization of self-dual $[18,9,8]$ and $[20,10,10]$ codes over $\mathbb F_{13}$, in which the split-boxed parent and its building-up child occur in one complete generator matrix. The algebraic core is formalized in Lean 4.

2026-04-09T17:27:46Z 31 pages Jae-Hyun Baek Jon-Lark Kim http://arxiv.org/abs/2609.11699v1 2026-09-10T15:24:17Z

On-Policy Self-Distillation (OPSD) has emerged as a popular paradigm for large language model (LLM) self-improvement, allowing models to act as their own teachers by leveraging privileged information such as ground-truth solutions. However, recent findings indicate that OPSD can severely degrade the performance of LLMs on complex reasoning tasks: By forcing the student to imitate an artificially confident reasoning trace conditioned on privileged information, OPSD inadvertently suppresses expressions of uncertainty and penalizes the exploratory, self-corrective behaviors required to solve challenging problems. To address this, we introduce Negative Self-Distillation (NSD), a new framework that optimizes LLMs by diverging from flawed reasoning rather than imitating privileged solutions. Instead of relying on ground-truth answers or external supervision, NSD uses the model itself to generate a question-specific negative condition (eg, acting as a ``careless reasoner'') and pushes the student's distribution away from this self-generated negative teacher. Naively applying unlearning objectives to achieve this divergence is problematic, as flawed reasoning tokens are confounded with basic linguistic tokens; indiscriminately penalizing both risks catastrophically degrading the model's foundational language capabilities. We resolve this by designing a dynamic gating mechanism that automatically identifies and isolates reasoning-critical tokens, ensuring gradient updates target only behavioral flaws while preserving the model's linguistic priors. Empirically, NSD consistently outperforms OPSD and other label-free, self-bootstrapping reinforcement learning (RL) baselines.

2026-09-10T15:24:17Z 23 pages, 7 figures Rongcan Pei Zhepei Wei Shuyao Xu Xinyu Zhu Wei-Lin Chen Yu Meng http://arxiv.org/abs/2609.11687v1 2026-09-10T15:15:50Z

We revisit Kashin-decomposition-based weight quantization for large language models and propose an improved algorithm with stronger convergence properties and structured, efficient orthogonal transforms. The method retains the core factorization of each weight into two components -- one with bounded infinity norm and the other with bounded infinity norm after an orthogonal transformation -- but replaces the dense random orthogonal matrix with a sign-randomized Discrete Cosine Transform (DCT), reducing the per-iteration cost from $\mathcal{O}(N^2)$ to $\mathcal{O}(N \log N)$. The proposed greedy algorithm with alternating updates guarantees the four-peak distribution required for stable 2-bit clustering of each factor and admits closed-form initialization of cluster centers, removing the multi-restart k-means bottleneck of prior work. Composed with OPTQ-style sequential error compensation and QuIP-style incoherence preprocessing, the resulting JAX pipeline is competitive with OPTQ, QuIP, QuIP-RG and a fine-tuning- and vector-quantization-free variant of QuIP# at 4-bit per channel on OPT, Llama-2 and Pythia, with favorable wall-clock scaling. The bounded-$\ell_\infty$ factorization is also notably robust: on stress configurations where QuIP variants diverge to four-digit perplexity (Pythia-6.9B) or abort with NaNs in LDL back-substitution (Mistral-7B), Kashin-DCT remains numerically stable and stays close to FP16 baseline. At inference time, each weight decomposes into two 2-bit factor codes per channel that are structurally suited to native-2-bit hardware.

2026-09-10T15:15:50Z Daria Cherniuk Alexander Rudikov Boris Kashin Ivan Oseledets http://arxiv.org/abs/2609.09425v2 2026-09-10T15:06:26Z

Educational data filters have become a practical way to improve language-model pre-training, but most filters treat educational value as a single scalar property. This may be too broad for some applications, especially if the data set already features a high density of educational material. Useful learning material needs to be accurate, engaging, well structured, and appropriate for the intended audience and application (e.g. learner- vs teacher-facing). Following QuRating (Wettig et al. 2024), we introduce Edu-QuRating: a pipeline for multi-dimensional educational data scoring and curation. Edu-QuRating defines education-specific rubrics, uses an LLM judge to label sampled document pairs and distills those pairwise preferences into reusable Edu-QuRaters, which can score individual text chunks on a set of educational criteria. Across two sequence-classification base models and six educational criteria, the best Edu-QuRater recovers held-out GPT-4.1-mini pairwise judgements with mean accuracy 0.917. We then apply the resulting scorers in two applications. First, we investigate the potential of Edu-QuRaters for corpus filtering to improve pretraining of small language models. We scored 322.25M FineWeb-Edu-Fortified documents to obtain a filtered pre-training mixture. In matched single-run pre-training comparisons, models trained with Edu-QuRating-based mixtures reached higher observed aggregate accuracy across nine benchmarks than the FineWeb-Edu baseline, with gains concentrated in particular tasks. Second, we used Edu-QuRater scores as reward terms for GRPO post-training. In held-out pairwise judge evaluations, combining Edu-QuRater and answer-structure rewards produced responses preferred to the Qwen3-4B base model on both pedagogical quality and instruction following.

2026-09-08T20:24:54Z Oliver G. B. Garrod Robin A. A. Ince Meng Liu Mohamed Huti Moritz Boos Amy Waldock Dominic Andrews Romana Alonso-Kropil Paul Atherton http://arxiv.org/abs/2608.08477v3 2026-09-10T14:59:27Z

25 pages, 1 figure, 10 tables. v3: transplanting a natively-trained visual tower (Qwen2-VL) onto the same frozen decoder takes the failing 8-nibble address field from 0.00 to 0.81 exact, at a coarser token budget than 2x2 tiling, refuting resolution as the operative variable. Second pre-registered field found 63% contaminated, demoted. B6/B7 tool-id remains at floor. Code/checkpoints on HF.

2026-08-09T04:46:58Z 20 pages, 1 figure, 9 tables. v2: retracted the B6 tool-id score of 0.08 (v1) after finding 3 benchmark harness bugs; under the fixed harness every B6/B7 metric is 0.0. Fixed a LoRA LR bug and a checkpoint-load-order bug. Added a 9-field gate + linear probe. Reframed the NoPE ablation as open, confound quantified. Code/checkpoints on HF Juan S. Santillana http://arxiv.org/abs/2604.25053v2 2026-09-10T14:55:40Z

While large language models (LLMs) are increasingly being explored for mental health applications, recent studies reveal that they can exhibit stigma toward individuals with psychological conditions. Existing evaluations of this stigma primarily rely on multiple-choice questions (MCQs), which fail to capture the biases embedded within the models' underlying logic. In this paper, we analyze the intermediate reasoning steps of LLMs to uncover hidden stigmatizing language and the internal rationales driving it. We leverage clinical expertise to categorize common patterns of stigmatizing language directed at individuals with psychological conditions and use this framework to identify and tag problematic statements in LLM reasoning. Furthermore, we rate the severity of these statements, distinguishing between overt prejudice and more subtle, less immediately harmful biases. To broaden the reasoning domain and capture a wider array of patterns, we also extend an existing mental health stigma benchmark by incorporating additional psychological conditions. Our findings demonstrate that evaluating model reasoning not only exposes substantially more stigma than traditional MCQ-based methods but also helps identify the flaws in the LLMs' logic and their understanding of mental health conditions.

2026-04-27T23:08:39Z Sreehari Sankar Aliakbar Nafar Mona Barman Hannah K. Heitz Ashwin Kumar Pouria Tohidi Dailun Li Danish Hussain Russell DuBois Hamed Hasheminia Farshad Majzoubi http://arxiv.org/abs/2607.07669v3 2026-09-10T14:54:04Z

Large language models increasingly understand dialectal English, yet still produce only standard, US-leaning English, leaving dialectal generation, the harder half of the problem, largely unaddressed. We introduce DiaLLM, which continually pretrains three open-weight language model families on the International Corpus of English and applies implicit and explicit post-training paradigms, each combined with three model alignment strategies, giving the first controlled comparison of these components across Australian, Indian, and Northern British English. Our results reveal a robustness-generation gap: benchmarks are shaped by continual pretraining and SFT, while alignment visibly reshapes generation in ways benchmarks do not capture. Explicit variety-targeted adaptation produces output reliably recognised as dialectal and judged more dialectal than broad alignment, yet where human judgement was directly assessed, the method that most aggressively optimises the dialectal reward is not the one judged most dialectal. Independent linguistic analysis corroborates this reward-quality gap, most clearly on two of the three families. No single alignment method dominates, and closing the gap will require richer reward designs and continued investment in dialectal resources. We release all code, checkpoints, and preference datasets.

2026-07-08T17:24:27Z Jordan Painter Dipankar Srirag Adarsh Kappiyath Diptesh Kanojia Aditya Joshi Lu Yin http://arxiv.org/abs/2609.11620v1 2026-09-10T14:32:35Z

High-dimensional dense text embeddings and large language models face real obstacles in financial-disclosure analysis: context-window limits, hallucination risk, high computational cost, and the arbitrary rotation of vector spaces across independently trained models. We present a training-free, alignment-free framework for corporate intelligence built on deterministic sparse seed vectors. Hashing word strings into a fixed high-dimensional basis places all documents and all temporal epochs in a common coordinate system by construction, removing any need for training or alignment. Accumulating these seed vectors across sentence contexts yields corpus-specific semantic signatures that compose linearly, supporting sub-second document comparison, issuer fingerprinting, tracking of how an issuer's vocabulary shifts between filings, and thematic sentence extraction, all on ordinary CPU hardware. Demonstrating the approach on a multi-year corpus of SEC filings (10-K, 10-Q, 8-K), we show how material corporate events, among them Boeing's 737 MAX crisis, Intel's supply-chain disruptions, and Bunge's acquisition of Viterra, emerge as distinct, interpretable semantic profiles, each traceable to the exact source sentences that produced it, with no domain-specific training and no LLM inference.

2026-09-10T14:32:35Z 26 pages, 2 figures Jean-François Delpech http://arxiv.org/abs/2604.04036v2 2026-09-10T14:02:30Z

Novice math teachers often encounter students' mistakes that are difficult to diagnose and remediate. Misconceptions are especially challenging because teachers must explain what went wrong and how to solve them. Although many existing large language model (LLM) platforms can assist in generating instructional feedback, these LLMs loosely connect pedagogical knowledge and student mistakes, which might make the guidance less actionable for teachers. To address this gap, we propose MisEdu-RAG, a dual-hypergraph-based retrieval-augmented generation (RAG) framework that organizes pedagogical knowledge as a concept hypergraph and real student mistake cases as an instance hypergraph. Given a query, MisEdu-RAG performs a two-stage retrieval to gather connected evidence from both layers and generates a response grounded in the retrieved cases and pedagogical principles. We evaluate on \textit{MisstepMath}, a dataset of math mistakes paired with teacher solutions, as a benchmark for misconception-aware retrieval and response generation across topics and error types. Evaluation results on \textit{MisstepMath} show that, compared with baseline models, MisEdu-RAG improves token-F1 by 10.95\% and yields up to 15.3\% higher five-dimension response quality, with the largest gains on \textit{Diversity} and \textit{Empowerment}. To verify its applicability in practical use, we further conduct a pilot study through a questionnaire survey of 221 teachers and interviews with 6 novices. The findings suggest that MisEdu-RAG provides diagnosis results and concrete teaching moves for high-demand misconception scenarios. Overall, MisEdu-RAG demonstrates strong potential for scalable teacher training and AI-assisted instruction for misconception handling. Our code is available on GitHub: https://github.com/GEMLab-HKU/MisEdu-RAG.

2026-04-05T09:31:47Z Zhihan Guo Yuting Lu Jionghao Lin http://arxiv.org/abs/2303.09136v2 2026-09-10T13:50:46Z

Large language models (LLMs) have transformed many fields, including natural language processing, computer vision, and reinforcement learning. These models have also made a significant impact in the field of law, where they are being increasingly utilized to automate various legal tasks, such as legal judgement prediction, legal document analysis, and legal document writing. However, the integration of LLMs into the legal field has also raised several legal problems, including privacy concerns, bias, and explainability. In this survey, we explore the integration of LLMs into the field of law. We discuss the various applications of LLMs in legal tasks, examine the legal challenges that arise from their use, and explore the data resources that can be used to specialize LLMs in the legal domain. Finally, we discuss several promising directions and conclude this paper. By doing so, we hope to provide an overview of the current state of LLMs in law and highlight the potential benefits and challenges of their integration.

2023-03-16T08:01:22Z 8 pages Zhongxiang Sun http://arxiv.org/abs/2510.15685v2 2026-09-10T13:48:07Z

This paper investigates the use of an LLM to generate auxiliary background context for social media posts, and explores four methods to incorporate this context into the input of an SBERT-based Hate Speech Detection (HSD) classifier. These are: text concatenation, embedding concatenation, a hierarchical transformer-based fusion, and LLM-driven text enhancement. We evaluate the impact of our context generation and incorporation strategies in a textual setting on the Latent Hatred dataset of implicitly hateful tweets and a multimodal setting on the MAMI dataset of misogynous internet memes. Results are evaluated against a zero-context baseline, two previous approaches based on entity linking, and a zero-shot LLM classifier. Findings indicate that incorporating generated context improves HSD performance by up to 3 and 6 F1 points on textual and multimodal settings respectively, from a zero-context baseline to the highest-performing system, based on embedding concatenation.

2025-10-17T14:28:57Z 8 pages, 9 figures, accepted for publication with the 10th Workshop on Online Abuse and Harms (WOAH) at EMNLP 2026 Joshua Wolfe Brook Ilia Markov http://arxiv.org/abs/2605.21049v2 2026-09-10T13:46:25Z

Brain-language model alignment is often interpreted as evidence that transformer models implement computations similar to those of the human brain. This assumes that neural predictivity reflects internal computational properties of large language models (LLMs), such as hierarchical contextual processing, predictive coding, or representational compression. An alternative possibility is that brain scores primarily reflect stable lexical-semantic correspondences shared by language models and the brain. Here we tested these interpretations using whole-brain encoding models across Mandarin, English, and French. Across all three languages, transformer representations significantly predicted activity in a distributed network spanning classical language regions, transmodal cortical systems, and subcortical structures. These spatial patterns showed substantial cross-linguistic overlap and remained remarkably stable across layers, providing little evidence that model depth systematically maps onto cortical processing hierarchies. Likewise, contextual transformer embeddings did not consistently outperform static lexical embeddings, despite providing some unique predictive variance. Finally, neither surprisal nor intrinsic dimensionality reproduced the layer-wise profile of brain scores, arguing against prediction and information compression as primary explanations for brain-LLM alignment. Together, these findings suggest that brain-LLM alignment is more robust across languages, transformer depth, and model architectures than previously appreciated, but less informative about shared computational mechanisms. Our results are more consistent with neural predictivity reflecting stable representational structure preserved across model transformations than with a one-to-one correspondence between their underlying computations.

2026-05-20T11:34:05Z Ni Yang Rui He Philipp Homan Iris Sommer Davide Staub Wolfram Hinzen http://arxiv.org/abs/2608.22230v3 2026-09-10T13:42:49Z

Large language models (LLMs) are increasingly used for hate speech moderation, often within human--AI workflows in which reviewers provide feedback before a final decision. Such feedback introduces two manipulation directions: whitewashing hateful content as normal and smearing normal content as hateful. This study examines the susceptibility of initially correct model judgments to annotator-style rebuttals and analyzes whether attack effectiveness differs across manipulation directions. We introduce a rejudge protocol that extends direct contradiction with decision-boundary perturbations and adversarial rationales. Experiments with multiple LLMs on two hate speech datasets show that annotator-style rebuttals substantially degrade moderation performance, with stronger effects in multi-turn settings. The results further reveal stable, model-specific asymmetries between whitewashing and smearing across attack configurations, indicating distinct directional vulnerability patterns. Explicit reasoning prompts and defensive instructions reduce these effects but do not eliminate them. These findings highlight the need for direction-aware safeguards and dedicated feedback-robustness evaluation in human--AI moderation workflows.

2026-08-23T05:59:25Z We identified errors in the experimental setup and analysis that affect several key results and conclusions. As substantial re-analysis is required and the conclusions may change, we respectfully request withdrawal of the current version Junyu Lu Kaiyuan Liu Kaichun Wang Jingyi Kang Deyi Ji Hailong Zhang Lanyun Zhu Qi Zhu Bo Xu Liang Yang Hongfei Lin http://arxiv.org/abs/2609.11545v1 2026-09-10T13:41:22Z

Low-resource multilingual text-to-speech (TTS) systems have expanded language coverage, but their robustness under complex text inputs remains insufficiently diagnosed. Existing evaluations mainly focus on naturalness, speaker similarity, and content consistency using regular test sentences, while providing limited insight into how multilingual TTS systems fail when handling challenging inputs such as numbers, dates, named entities, long sentences, code-switched expressions, and punctuation-related structures. This paper proposes a complex-text robustness diagnosis framework for low-resource multilingual TTS. We evaluate robustness from three dimensions: content consistency, language consistency, and generation stability. A multilingual robustness testing scheme is designed for Thai, Vietnamese, Swahili, and Indonesian, covering ordinary sentences and multiple types of complex text inputs. We further introduce automatic diagnostic metrics, including character error rate, language identification accuracy, and duration abnormal rate. To support input-level risk analysis before speech generation, we propose a lightweight Text Risk Score (TRS), which estimates synthesis risk from interpretable text features without manual annotation or model training. Experiments on three representative multilingual TTS systems, including OmniVoice, VoxCPM2, and MMS-TTS, show that complex text inputs expose systematic failure patterns that are not fully reflected by ordinary short-sentence evaluation. Different systems exhibit distinct vulnerabilities in number normalization, named entity handling, long-text generation, and code-switched input processing. Furthermore, TRS shows a positive correlation with content errors and duration abnormalities, demonstrating its usefulness as a low-cost pre-synthesis indicator for complex-text risk diagnosis in low-resource multilingual TTS.

2026-09-10T13:41:22Z NCMMSC 2026 accepted Tianlun Zuo Ziyu Zhang Tingzhi Mao Zhonghua Fu Lei Xie http://arxiv.org/abs/2609.11505v1 2026-09-10T13:12:48Z

Efficient language learning requires methods to reduce the reliance on large data and computational resources. We investigate structural transfer: First training models on non-language data to induce useful priors for natural language. This approach is a form of weight initialization for multilingual language modeling. We evaluate transfer via next-token-prediction loss, weight shifts in the model, and downstream linguistic benchmarks. Several symbolic data types - notably music, probabilistic grammars, and cellular automata - yield lower language-modeling loss than random initialization. These gains coincide with smaller weight shifts during subsequent language training, suggesting that structural transfer positions models in a more favorable region of the parameter space. However, a lower loss does not translate consistently into better downstream linguistic performance, and transfer from non-language data is less efficient than additional language data. We conclude that non-language data can serve as a partial substitute for language data for the training objective of next-token prediction but does not reliably support broader linguistic generalization.

2026-09-10T13:12:48Z EMNLP 2026, BabyLM Challenge; 18 pages, 11 figures Yana Veitsman Jonas Mayer Martins Jonathan Lautenschlager Lisa Beinborn