https://arxiv.org/api/WYMRCjG3gme2Rys9Y+pdpikOhCE 2026-11-12T03:27:33Z 23159 0 15 http://arxiv.org/abs/2611.14443v1 2026-11-10T18:13:47Z

Automatic Music Transcription (AMT) relies on paired audio--MIDI datasets, implicitly assuming near-perfect temporal alignment. However, widely used piano datasets have been shown to contain alignment errors that may bias model evaluation and training. We propose a framework for evaluating audio--MIDI alignment scoring methods through controlled temporal, pitch, and structural MIDI corruptions. We also introduce STI, a scoring method built on AMT models, and benchmark it against existing alignment scoring approaches. Our results show that STI, together with methods based on CENS and spectrograms, achieves the strongest overall performance, with STI exhibiting the highest sensitivity to fine-grained temporal misalignments, the main alignment error in AMT datasets. We then apply the best-performing methods to widely used piano AMT datasets and identify trends in their relative alignment quality. These results support more informed dataset curation, model training, and AMT evaluation.

2026-11-10T18:13:47Z 6 pages Lucas de Souza Brandão Lucas Gris Joaquim Cavalcante Chukwuemeka Nkama Andrea Poltronieri http://arxiv.org/abs/2611.14421v1 2026-11-10T17:59:28Z

Streaming speech recognizers output a word only after the speaker has finished it, typically 90-400 ms later. For a voice agent every millisecond counts: the sooner the words are outputted, the sooner the agent can start a search or plan its reply. We present Earlybird, a 29M-parameter streaming recognizer (16.8 MB at 4 bits). Earlybird learns from audio cut off at random points, including partway through a word, and is asked to predict every word that is due by then in full, plus a guess of the next two words. Its raw token probabilities give a confidence for every word including the one still being spoken. On clean audiobook speech (LibriSpeech test-clean), the median word is predicted correctly 90 ms before the speaker finishes it, while released streaming models get it 90 to 400 ms after it ends. Earlybird is the earliest on all six test sets we time, covering audiobooks, TED talks, public speech, earnings calls, and synthetic speech. On the same audiobooks, 69% of words are predicted correctly before they end, against 3-25% for the other models. For words still being spoken, confidence is calibrated, with an expected calibration error (ECE) of 3.7 to 5.5. On the eight English Open ASR Leaderboard sets, it is more accurate than Moonshine Tiny on every set and roughly ties Whistle, a model of the same size.

2026-11-10T17:59:28Z 9 pages, 3 figures, 6 tables. Model: https://huggingface.co/sahilmahendrakar/Earlybird-29M Sahil Mahendrakar http://arxiv.org/abs/2611.14043v1 2026-11-10T13:12:34Z

Despite recent advances in neural speech codecs, achieving highfidelity speech reconstruction at low bitrates remains a formidable challenge. To address this limitation, we propose ESDCodec, a speech codec that integrates a thoroughly enhanced semantic quantizer and a conditioned decoder network. Specifically, we employ a randomly initialized and frozen codebook, followed by a lightweight projector, to encode semantic details entirely within a linear space while enhancing codebook utilization. To further improve perceptual quality, we design a condition network that explicitly injects prior subband knowledge into the upsampling decoder. Taking the de-quantized feature as input, this network predicts subband signals, thereby providing fine-grained guidance for waveform reconstruction. Extensive experimental results show that ESDCodec achieves superior reconstruction performance at an ultra-low bitrate of 0.85kbps. For LLM-based speech generation tasks, ESDCodec also consistently outperforms existing codec models.

2026-11-10T13:12:34Z 5 pages, 2 figures Nan Xu Kecan Mao Xin Gu Yi Zhao Zhi Gong http://arxiv.org/abs/2611.14016v1 2026-11-10T12:52:00Z

A loudspeaker in a closed box is modeled as a rigid sphere with an axially moving cap. For an axisymmetric cone-velocity profile, its radiation impedance is written as a network. Each spherical-harmonic mode has an impedance rational in frequency, realized exactly as a finite passive ladder of masses and compliances terminated in a resistance. Ideal transformers set by Legendre coefficients couple these ladders; truncated modes reduce to a residual mass. Despite the different geometry, the impedance resembles that of a baffled piston but, at equal volume velocity, has half its low-frequency resistance and about three quarters of its radiation mass.

2026-11-10T12:52:00Z 13 pages (preprint format), 3 figures. Submitted to JASA Express Letters Ronald M. Aarts http://arxiv.org/abs/2611.14003v1 2026-11-10T12:44:52Z

Microphones let an autonomous vehicle sense into areas its cameras and LiDAR cannot see, by picking up the sound that other traffic participants emit. Locating those sources and fusing them with the visual perception requires the microphone array to be calibrated and registered relative to the vehicle's other sensors. Existing calibration methods assume small, unobstructed indoor rigs, whereas a vehicle body casts an acoustic shadow that straight-line propagation cannot represent, and to our knowledge no calibration of an array distributed around a vehicle has been reported. We propose a pipeline that uses a balloon burst as a single cross-modal target, which the LiDAR locates in metric coordinates, every microphone hears as a sharp onset and a camera times. Each observation is weighted by its own timing variance, which discounts shadowed arrivals instead of discarding them and needs no model of the vehicle. On a full-size vehicle with eight microphones over a 4.9 m aperture, the held-out range residual of 6.37 cm approaches the 6.07 cm floor set by the onset timing noise, and the calibrated array tracks a moving source to within a few degrees of the vehicle's own LiDAR. A dozen bursts spread around the vehicle suffice, and spreading them matters more than adding more.

2026-11-10T12:44:52Z Submitted to IEEE ICRA 2027 Marvin Seegert Richard Taupitz Korbinian Moller Johannes Betz http://arxiv.org/abs/2611.13990v1 2026-11-10T12:33:12Z

Which linguistic units minimise the bitrate required to encode speech content: phones, syllables, words, maybe larger units, or something in between? This question has implications for the development of speech and language systems and theories of human language processing. Using a transcribed corpus of 960 hours of English audiobooks, I compare the information rates of phones, syllables, words, multi-word combinations and intermediate tokenisations. While all units are equivalent under a perfect language model, it makes sense to compare these under limited modelling budgets; I use $n$-gram and transformer language models. The lowest bitrates are achieved using words, with slightly longer units sometimes performing best. The results show that, when modelling capacity is limited, units larger than syllables but smaller than long multi-word phrases provide the most efficient representation in terms of information rate.

2026-11-10T12:33:12Z 5 pages, 1 table, 2 figures Herman Kamper http://arxiv.org/abs/2611.13875v1 2026-11-10T10:35:48Z

Existing instruction-based TTS systems primarily support one-shot, utterance-level control, where a single instruction determines the speaking style of an entire utterance. As a result, even minor refinements require rewriting the instruction and regenerating the full utterance, limiting fine-grained interactive control. We propose DialInstruct-TTS, an interactive multi-turn expressive TTS framework that enables users to iteratively refine generated speech through natural-language dialogue until the desired style is achieved. At each dialogue turn, a Qwen-based planner integrates the dialogue history and latest user instruction into a complete TemporalPlan, while a temporal acoustic control module provides fine-grained control over emotion, speaking rate, volume, and phonation. Reference-latent revision preserves non-target regions across turns, allowing users to modify only the requested attributes or temporal spans while retaining previously satisfactory speech. Subjective evaluations demonstrate improved global instruction following, accurate local style realization, and effective non-target preservation. Compared with conventional one-shot interaction requiring full instruction restatement and utterance regeneration, DialInstruct-TTS substantially improves perceived convenience and interaction comfort. Audio samples are available at https://anonymous.4open.science/w/eacl-demo-17C5/.

2026-11-10T10:35:48Z Wangzixi Zhou Bagus Tris Atmaja Sakriani Sakti http://arxiv.org/abs/2610.29164v2 2026-11-10T09:07:45Z

This paper presents XingChen-UniFE, a unified large audio language model (LALM) for general speech restoration (GSR) and target speaker extraction (TSE). The proposed model employs complementary semantic and acoustic representations through a dual-stream encoder, where a self-supervised learning model provides hierarchical semantic features and a learnable acoustic branch captures fine-grained structures from complex spectra. A bidirectional cross-attention module further facilitates information exchange between the two representations. The fused representation conditions the LALM for discrete speech token generation with task and speaker information. Experiments demonstrate strong performance on both GSR and TSE and robust generalization to unseen degradation conditions.

2026-10-21T03:55:18Z Submitted to ICASSP 2027. Added an appendix with additional evaluation results for speech restoration and target speaker extraction Kelan Kuang Qihan Hu Jian Kang Jie Li Yongxiang Li Yi Liu http://arxiv.org/abs/2611.13689v1 2026-11-10T07:37:38Z

We present Walrus, a unified 8B speech-and-audio foundation model handling speech recognition (including code-switching and multi-talker speech), speech-to-text and speech-to-speech translation, text-to-speech, emotion recognition, audio captioning, and general audio understanding. Our main contribution is not a new architecture but a data-centric training recipe; we use a standard decoder-only backbone initialized from Gemma 4, so that gains can be traced to the data. For speech generation, Walrus predicts low-bitrate discrete speech tokens, and a lightweight token-to-waveform decoder converts them into high-fidelity audio. The recipe is a three-stage data curriculum that moves from broad multilingual data to multi-talker, code-switching, and domain-specific data. At each stage, task-aware and dataset-aware weighting balances tasks with very different token budgets and protects core skills from forgetting. Trained on over 600M samples covering more than 20 tasks and 100 languages, Walrus leads on most multilingual, multi-talker, and code-switching ASR, speech translation, and audio understanding benchmarks, outperforming both open speech language models of comparable size and dedicated task-specific models. On most benchmarks, it even matches or surpasses the $4\times$ larger Qwen3-Omni's performance. Results suggest that data engineering is a primary driver of quality in multi-task speech and audio modeling.

2026-11-10T07:37:38Z 30 pages, 3 figures, technical report Alkis Koudounas Osamu Take Hayato Futami Quentin Jodelet Emiru Tsunoo http://arxiv.org/abs/2611.13614v1 2026-11-10T05:45:36Z

Generating long-form speech requires models to keep content, speaker identity, turn-taking, and expressive style consistent over time. Yet most open training corpora contain examples lasting only tens of seconds and open long-form benchmarks cover at most two speakers per recording. We introduce Emilia2, a 229K-hour multilingual dataset for controllable long-form speech generation. From in-the-wild data, Emilia2 constructs short utterances, single-speaker long-form speech, and multi-speaker dialogues, preserving transcripts, speaker identities, pauses, and acoustic transitions. Examples longer than five minutes account for 50.0% of single-speaker long-form hours and 66.8% of dialogue hours. To support expressive control, Emilia2 provides natural language instructions for 4.77M examples and nonverbal-vocalization tags for 933K examples. We also introduce Emilia2-testset, 23 hours of bilingual recordings of up to 30 minutes and four speakers, each speaker paired with reference audio. We train autoregressive (AR) and non-autoregressive (NAR) models on Emilia2. Our 0.6B-backbone NAR model achieves the lowest Chinese CER and English WER on TTSD-Eval (5.27%/9.93%) among compared systems, while our AR model is on par with Qwen3-TTS-12Hz-VD on InstructTTSEval (zh/en: 76.2/78.3 vs. 77.1/77.9). These results validate Emilia2 as a comprehensive data foundation for long-form speech generation and minute-scale expressive control. Demo: https://charlesnii.github.io/Emilia2-demo/.

2026-11-10T05:45:36Z Qinke Ni Dekun Chen Junwen Qiu Huan Liao Xu Li Xingwen Han Wan Lin Kunyu Feng Yuxiang Wang Yusheng Mu Chaoren Wang Yingda Shen Mingjie Chen Haolong Zheng Xulin Fan Mark A. Hasegawa-Johnson Zhizheng Wu http://arxiv.org/abs/2610.35581v3 2026-11-10T02:51:25Z

When a user talks over a full-duplex dialogue model, it must decide whether the user is interrupting (stop) or backchannelling (carry on). Post-training can improve this discrimination or change how readily it stops. In our GRPO runs the second dominates: no configuration lowers both error rates on more than one model, and two released checkpoints discriminate no better than their bases. We trace this partly to how the reward is estimated. The probability of stopping inside a short window is a known function of a few stop probabilities the policy emits, yet GRPO estimates it by sampling. A group of rollouts that all agree gives no gradient, even when all fail; the push on the stop threshold is strongest where the model is unsure, whichever outcome is right; and whole-rollout credit reaches decisions the reward never judges. Confining the credit to the window removes only that leakage. Window likelihood (WL) samples the words but integrates over the stop decisions, so the probability of the correct outcome is computed exactly and its logarithm maximised. On HumDial-FDBench, across four open models, WL gives the best F1 and lowers both error rates by more than their confidence intervals, without talking into silence.

2026-10-26T15:14:03Z Guoqiang Hu Lichao Zhang Yizhou Peng Yuhao Luo Haifeng Hu Zhongqian Sun Wei Yang Eng Siong Chng http://arxiv.org/abs/2611.13430v1 2026-11-10T01:15:31Z

Spoken Language Diarization (SLD) models provide frame-level language information of a speech recording. SLD modeling pipeline includes feature extraction from pre-trained speech models followed by clustering. Fine-tuning pre-trained speech models has shown performance gains for SLD with its magnitude being dataset dependent. Off-target Posterior Mass (OTPM), computed based on posterior language identification probabilities is proposed as a diagnostic metric to forecast performance gains from SLD fine-tuning. OTPM obtains positive correlation with probe gains, 0.56 on unseen synthetic bilingual data and 0.23 on real-world corpora. These results indicate promising potential of OTPM as a diagnostic metric for SLD.

2026-11-10T01:15:31Z 10 pages, 2 figures, 3 tables. Submitted to JASA Express Letters Zhuoyan Tao Aditya Kommineni Shrikanth Narayanan http://arxiv.org/abs/2611.13243v1 2026-11-09T21:14:56Z

We present GigaEdge Spench, an open-source SDK for streaming speech enhancement on CPU servers, mobile devices, and in browsers. Applications can build enhancement channels from neural denoising, normalization and dynamics blocks, sample-rate conversion, and acoustic echo cancellation where supported. These stages can be combined, while normalization can also run without a neural model. The release includes an 8 kHz INT8 denoiser and discriminative and hybrid 48 kHz FP32 models. A shared C++ core supports ONNX Runtime and TensorFlow Lite through native interfaces, Android and iOS SDKs, and WebAssembly integration. Independent stream contexts share model resources, and consecutive frames from one stream can be processed in a single inference call. We evaluate the released models on EARS-WHAM v2 and compare their quality and processing costs with RNNoise and DPDFNet2. All tested native and Web configurations process files faster than real time, although larger chunks require more time to collect input. We also measure how many streams a shared model can process on a CPU. Our contribution is a configurable deployment system and an experimental study of how model choice, chunk size, and execution environment affect its use. All reported SDK timing uses ONNX Runtime. Index Terms: speech enhancement, streaming inference, cross-platform deployment, temporal frame grouping, computational efficiency

2026-11-09T21:14:56Z full version. draft Artem Sokolov Artem Khaidarov Yana Lysova Alexander Gladkikh Vitaliy Yulis http://arxiv.org/abs/2611.12742v1 2026-11-09T17:25:31Z

Echolocating bats shape their calls and the returning echoes with their noseleaf or mouth, head and pinnae. Understanding these spectrospatial directivity patterns is central to bat echolocation. We describe a dataset of simulated far-field emission and reception directivities of seven echolocating bat species from three families (Phyllostomidae, Rhinolophidae and Vespertilionidae). For each species, the head of one preserved specimen was scanned with micro-computed tomography and converted into a closed surface mesh, on which the responses of both ears and of the emitter were computed with a boundary element method or, for the two mouth-emitting species, with an analytical piston model. The complex-valued responses cover the full sphere at 2.5 deg resolution from 20 to 160 kHz in 0.5 kHz steps. They come with the meshes, the landmarks of ears, nostrils and mouth, alternative source positions and skeletons for posing the pinnae and noseleaves, all in one frame, and with Python and MATLAB code. The technical validation covers solver verification, mesh convergence, comparisons with earlier simulations and measurements, and sensitivity analyses.

2026-11-09T17:25:31Z 34 pages, 8 figures, supplementary information appended. Data: https://doi.org/10.5281/zenodo.23729269. Code: https://github.com/Cosys-Lab/bat_ertf_dataset Jan Steckel http://arxiv.org/abs/2609.02165v2 2026-11-09T16:36:14Z

Clear capture of the wearer's own voice (OV) is essential when using earbuds for mobile communication. However, OV capture remains challenging in noisy environments. Bone-conducted (BC) speech, which can be sensed as vibrations of the earbud housing, can be used to improve OV capture. However, neither bandwidth nor spatial characteristics of OV-induced earbud vibrations have been analyzed in detail, despite both characteristics being relevant, e.g., for sensor choice and placement. This study investigates both characteristics, based on measurements with two earbud models. Spectrally, results indicate that OV-induced earbud vibrations exhibit a low-pass characteristic, with a steep roll-off of -93 dB per decade above 400 Hz. Thus, sensors with comparatively low noise floors are required to sense the vibrations above \SI{1}{\kilo\hertz}. Spatially, results indicate that the earbuds mainly vibrate in and out of the ear canal entrance, with high consistency between subjects and fits. Simulations confirm that this enables capture of the high-power vibrations below 400 Hz by a single-axis sensor with less than 1.5 dB mean attenuation.

2026-09-02T06:25:10Z 14 pages, 12 figures, journal J. Audio Eng. Soc., vol. 74, no. 11, pp. 796-810, 2026 Christoph Weyer Peter Jax 10.17743/jaes.2026.0301