arXiv Daily Digest

282

Papers

Verifiable Foundation Models for Robot Safety

可验证的机器人安全基础模型

Corsi, Davide, Kim, Kyungmin, Fox, Roy

Abstract

Deploying foundation models for robot control raises a central challenge: the expressive power that enables rich, multimodal perception also makes these models opaque and difficult to analyze formally, rendering them intractable for existing verification tools. In this paper, we present FEARL (Foundation-Enabled Assured Robot Learning), a framework that addresses this tension through a modular architectural decomposition. FEARL separates the policy into a large Controller (C) responsible for high-dimensional perception and task reasoning, and a small Safety module (S) that receives low-dimensional observations from dedicated safety sensors together with a bounded context embedding from C and produces the final action. Since many robot safety requirements, such as collision avoidance and workspace boundary constraints, can be expressed over these safety sensor observations, formal verification can be applied to S rather than to the full foundation-model backbone. This makes formal analysis tractable with existing tools while preserving the Controller's expressive power for task reasoning. To show that the decomposed policy remains capable of solving diverse tasks, we evaluate FEARL on three simulated robotic domains using multiple Controller backbones and training procedures, including pretrained off-the-shelf vision-language-action models. We further transfer the learned policy from one of our simulated tasks to a physical robot, suggesting that the low-dimensional safety interface supports practical sim-to-real transfer.

Chinese Translation

将基础模型应用于机器人控制面临一个核心挑战：使得丰富的多模态感知成为可能的表达能力，也使得这些模型变得不透明，难以进行形式分析，从而使现有的验证工具无法处理。在本文中，我们提出了FEARL（基础驱动的安全机器人学习），这是一个通过模块化架构分解来解决这一矛盾的框架。FEARL将策略分为一个大型控制器（Controller, C），负责高维感知和任务推理，以及一个小型安全模块（Safety module, S），该模块接收来自专用安全传感器的低维观测数据，以及来自C的有界上下文嵌入，并生成最终动作。由于许多机器人安全要求，如避免碰撞和工作空间边界约束，可以通过这些安全传感器观测进行表达，因此可以对S进行形式验证，而不是对完整的基础模型骨干进行验证。这使得使用现有工具进行形式分析变得可行，同时保留了控制器在任务推理方面的表达能力。为了证明分解后的策略仍然能够解决多样化的任务，我们在三个模拟机器人领域中评估了FEARL，使用了多个控制器骨干和训练程序，包括预训练的现成视觉-语言-动作模型。我们进一步将从模拟任务中学习到的策略转移到物理机器人上，表明低维安全接口支持实际的模拟到现实转移。

View on arXiv Download PDF AI Translation

cs.RO / 2 / 2606.23760

Engineering Reliable Autonomous Systems: Challenges and Solutions

工程可靠的自主系统：挑战与解决方案

Farrell, Marie, Luckcuck, Matt, Ferrando, Angelo, Cardoso, Rafael C., Alechina, Natasha, Autili, Marco, Hernandez, Diana Benjumea, Santos, Luciana Brasil Rebelo dos, Briola, Daniela, Cavalcanti, Ana, Colombo, Christian, Dennis, Louise A., Dixon, Clare, Fisher, Michael, Gleirscher, Mario, Johnson, Taylor, Lesire, Charles, Lestingi, Livia, Linker, Sven, Logan, Brian, Paterson, Colin, Papacchini, Fabio, Pelliccione, Patrizio, Ribeiro, Pedro, Schwammberger, Maike, Tarifa, Silvia Lizeth Tapia, Taylor, Hazel, Woodcock, Jim, Xu, Mengwei, Yang, Yi, Zhang, Huan

Abstract

Engineering reliable autonomous systems is an important and growing topic in computer science. As autonomous systems become more prevalent, easy-to-use techniques for building them reliably are increasingly important. This workshop report captures and expands on the discussions at the Lorentz Center Workshop "Engineering Reliable Autonomous Systems" (ERAS), held from 10 to 14 June 2024. The workshop was co-organised by the organisers of the Workshop on Formal Methods for Autonomous Systems (FMAS) and the Workshop on Agents and Robots for reliable Engineered Autonomy (AREA). It brought together members of the FMAS and AREA communities, industry practitioners, and representatives from sectors where autonomous systems pose distinctive engineering challenges. The workshop focused on three main research topics: techniques for verification and validation of autonomous systems; engineering real-world autonomous systems; and software architectures for safe autonomous systems. Its main outcome is a catalogue of challenges in these areas and, most importantly, a pathway to solutions. Some challenges can already be tackled by techniques that are well known in academia but have not yet become regularly used in practice. Other challenges remain unresolved and require further research. This roadmap is intended to support future research and industrial collaboration.

Chinese Translation

工程可靠的自主系统是计算机科学中一个重要且日益增长的话题。随着自主系统的普及，构建这些系统的可靠性所需的易用技术变得愈发重要。本次研讨会报告总结并扩展了2024年6月10日至14日在洛伦茨中心举行的“工程可靠自主系统”（ERAS）研讨会的讨论。该研讨会由“自主系统形式化方法研讨会”（FMAS）和“可靠工程自主性的代理与机器人研讨会”（AREA）的组织者共同主办，汇聚了FMAS和AREA社区的成员、行业从业者以及来自自主系统面临独特工程挑战的各个领域的代表。研讨会重点关注三个主要研究主题：自主系统的验证与验证技术；现实世界自主系统的工程；以及安全自主系统的软件架构。其主要成果是这些领域挑战的目录，最重要的是，提供了解决方案的路径。一些挑战可以通过学术界已知的技术来解决，但这些技术尚未在实践中得到广泛应用。其他挑战仍未解决，需要进一步研究。该路线图旨在支持未来的研究和产业合作。

View on arXiv Download PDF AI Translation

cs.RO / 3 / 2606.23848

Enforcing Human-like Kinematics in Dexterous Piano Playing via Adversarial Posture Regularization

通过对抗姿态正则化实现类人运动学的灵巧钢琴演奏

Qiu, Bin, Shao, Yanming, Cai, Guanyu, Mu, Yao

Abstract

Reinforcement learning can train bimanual dexterous hands to play piano in physics simulation with high note accuracy, but for high-DoF dexterous hands, relying solely on task rewards or IK inversion often leads to unnatural postures and joint overextension. We propose \textit{Adversarial Posture Regularization (APR)}. It avoids expensive, song-aligned expert demonstration data and instead uses a small amount of casual human playing data. By matching the distribution of the posture of the policy with the human prior through an adversarial objective, APR encourages more human-like hand shapes. Meanwhile, we collect and release unstructured hand motion data of piano playing using a consumer-grade Meta Quest 3, and retarget the key motion information to the Shadow Hand. Finally, we achieve significantly better performance than prior methods on all three human-likeness metrics (cPSI, BSE, and FAC) as well as in visual quality. Project repository: https://github.com/APRProject/APRPianist.

Chinese Translation

强化学习可以训练双手灵巧手在物理仿真中以高音符准确度演奏钢琴，但对于高自由度（DoF）灵巧手，仅依赖任务奖励或逆向运动学（IK）常常导致不自然的姿态和关节过度伸展。我们提出了对抗姿态正则化（Adversarial Posture Regularization, APR）。该方法避免了昂贵的、与歌曲对齐的专家演示数据，而是使用少量的随意人类演奏数据。通过对抗目标将策略的姿态分布与人类先验进行匹配，APR鼓励更类人化的手形。同时，我们使用消费级的Meta Quest 3收集并发布了钢琴演奏的非结构化手部运动数据，并将关键运动信息重新定向到Shadow Hand。最终，我们在所有三个类人性指标（cPSI、BSE和FAC）以及视觉质量上实现了显著优于先前方法的性能。项目仓库：https://github.com/APRProject/APRPianist。

View on arXiv Download PDF AI Translation

cs.RO / 4 / 2606.23901

Topological Online Learning for Displacement-based Formation Control

基于位移的拓扑在线学习在编队控制中的应用

Sharma, Saksham, Gupta, Shubhankar, Gunagi, Sumant A, Sundaram, Suresh

Abstract

This paper addresses the problem of robust formation control by introducing Topological Online Learning for Displacement-based (TOLD) formation control, a real-time edge-level adaptation framework. Unlike conventional node-level robust controllers that regulate individual robot inputs without modifying the interaction topology, TOLD updates the interaction topology weights online to directly minimize formation distortion. Two strategies are proposed under the TOLD formation control framework: Online Gradient Flow (OGF) with unconstrained weights and Online Exponential Gradient Flow (OExpGF) with non-negative convex weights. Theoretical analysis establishes that, for single-integrator agents over directed graphs, OExpGF guarantees asymptotic consensus, while OGF ensures bounded formation distortion. Simulations with twelve robots under intermittent disturbances show 1.2%-33.14% median cumulative Root Mean Distortion Error reduction when augmenting TOLD with node-level controllers. Hardware experiments with Crazyflie 2.0 quadrotors demonstrate over 62% (OGF) and 31.4% (OExpGF) reduction in median formation distortion compared to fixed-weight consensus.

Chinese Translation

本文通过引入基于位移的拓扑在线学习（Topological Online Learning for Displacement-based, TOLD）编队控制，解决了鲁棒编队控制的问题，提出了一种实时的边级适应框架。与传统的节点级鲁棒控制器不同，后者在不修改交互拓扑的情况下调节单个机器人的输入，TOLD 在线更新交互拓扑权重，以直接最小化编队畸变。在 TOLD 编队控制框架下提出了两种策略：具有无约束权重的在线梯度流（Online Gradient Flow, OGF）和具有非负凸权重的在线指数梯度流（Online Exponential Gradient Flow, OExpGF）。理论分析表明，对于在有向图上的单积分器代理，OExpGF 保证渐近共识，而 OGF 确保有界的编队畸变。在间歇性干扰下，十二个机器人的仿真实验显示，当将 TOLD 与节点级控制器结合时，中位数累积均方根畸变误差降低了 1.2%-33.14%。与固定权重共识相比，使用 Crazyflie 2.0 四旋翼的硬件实验表明，中位数编队畸变降低超过 62%（OGF）和 31.4%（OExpGF）。

View on arXiv Download PDF AI Translation

cs.RO / 5 / 2606.24038

Sim-to-Real Betting on the E-Process: Bringing "simulators" to anytime-valid confidence sequences

E过程中的模拟到现实投注：将“模拟器”引入随时有效的置信序列

Chen, Yujia, Weng, Bowen

Abstract

This note describes an integration of the sim-to-real performance estimate with betting (from Chen et al.) and the safe anytime-valid inference (from Ramdas et al.). Using the scaled simulators. The method produces efficient, reliable certificates for the mean estimate, an approach that is especially valuable in robot performance testing. This note gives a primary, self-contained account of the construction; preliminaries of the respective methods are kept at a minimum, and one shall refer to the original works for full detail. Some synthetic examples demonstrating the proposed algorithm can be found at https://github.com/ISUSAIL/Bet4Sim2Real-EProcess.

Chinese Translation

本文描述了将模拟到现实的性能估计与投注（来自Chen等人）和安全的随时有效推断（来自Ramdas等人）相结合的整合方法。利用缩放的模拟器，该方法为均值估计生成高效、可靠的证明，这一方法在机器人性能测试中尤为重要。本文提供了构建的初步自包含描述；各自方法的前提条件保持在最低限度，完整细节请参阅原始文献。一些演示所提算法的合成示例可以在https://github.com/ISUSAIL/Bet4Sim2Real-EProcess找到。

View on arXiv Download PDF AI Translation

cs.RO / 6 / 2606.24039

TurboMPC: Fast, Scalable, and Differentiable Model Predictive Control on the GPU

TurboMPC：在GPU上快速、可扩展且可微分的模型预测控制

Bravo-Palacios, Gabriel, Zhang, Jianghan, Pestrikov, Zachary, Plancher, Brian, Lew, Thomas

Abstract

Robotics increasingly relies on GPUs for parallel simulation, large-scale learning, and neural-network inference. For model predictive control (MPC) to scale with this paradigm, solvers must run efficiently on this hardware while remaining fast, differentiable, and compatible with expressive MPC formulations used in robotics. We present TurboMPC, a differentiable MPC solver that runs entirely on the GPU and supports state and control inequality constraints, implicit integrators, cross-time-coupled costs, and slack variables. TurboMPC combines sequential quadratic programming (SQP), an alternating direction method of multipliers (ADMM) inner solver, implicit differentiation, and a co-designed JAX-CUDA implementation for efficiency and ease of use. In simulation, we validate TurboMPC on constrained planning, humanoid imitation learning, and reinforcement learning with neural-network cost function tasks, achieving up to $15\times$ and $58\times$ speedups over state-of-the-art CPU and GPU differentiable solvers, respectively. We deploy TurboMPC on a full-scale car for minimum-time racing and find that batched, GPU-accelerated tuning of MPC parameters via Bayesian optimization yields significantly faster driving than a hand-tuned baseline. TurboMPC also scales to planning horizons of over $8000$ knot points while maintaining control of the vehicle. We open-source TurboMPC at: https://github.com/ToyotaResearchInstitute/turbompc

Chinese Translation

机器人技术越来越依赖GPU进行并行仿真、大规模学习和神经网络推理。为了使模型预测控制（MPC）能够与这一范式相适应，求解器必须在此硬件上高效运行，同时保持快速、可微分，并与机器人中使用的表达性MPC公式兼容。我们提出了TurboMPC，这是一种完全在GPU上运行的可微分MPC求解器，支持状态和控制不等式约束、隐式积分器、跨时间耦合成本和松弛变量。TurboMPC结合了序列二次规划（SQP）、交替方向乘子法（ADMM）内求解器、隐式微分以及为提高效率和易用性而共同设计的JAX-CUDA实现。在仿真中，我们在受约束规划、人形模仿学习和带有神经网络成本函数任务的强化学习上验证了TurboMPC，分别实现了比最先进的CPU和GPU可微分求解器快$15 imes$和$58 imes$的加速。我们在全尺度汽车上部署TurboMPC进行最短时间赛车，并发现通过贝叶斯优化对MPC参数进行批量GPU加速调优的结果显著快于手动调优的基线。TurboMPC还能够扩展到超过$8000$个节点的规划视野，同时保持对车辆的控制。我们将TurboMPC开源，地址为：https://github.com/ToyotaResearchInstitute/turbompc

View on arXiv Download PDF AI Translation

cs.RO / 7 / 2606.24049

SPACE: Enabling Learning from Cross-Robot Data Toward Generalist Policies

SPACE：实现跨机器人数据学习以支持通用策略

Lee, Haeone, Jeon, Byeongguk, Jeong, Suchae, Kim, Jian, Lee, Kimin

Abstract

In robot learning, scaling training datasets across diverse embodiments and environments has become a dominant paradigm for learning generalizable robot policies. These policies are commonly trained via behavior cloning to imitate actions from pre-collected demonstrations. However, since robot actions are tied to the dynamics of the data collection robot, different robots may require different actions to achieve the same motion. This discrepancy hinders both policy training and deployment across diverse robots. To address this, we propose using Cartesian state delta as a universal action representation across robots, and introduce State Prediction and Adaptive Command Execution (SPACE) framework. SPACE handles robot dynamics variation at three levels: across different embodiments, across hardware units of the same embodiment, and within a single robot during operation. It consists of two components: (i) a Cartesian state delta policy that predicts geometric end-effector displacement, and (ii) Action Adapter, which converts the predicted Cartesian state delta into robot-specific control commands. Experiments show that SPACE substantially outperforms policies that directly predict control commands when learning from data collected across different embodiments and across hardware units of the same embodiment. SPACE also remains robust under dynamics shifts at deployment, including changes in control frequency, object weight, and controller gains. The project page is available at http://haeone.site/space-website/.

Chinese Translation

在机器人学习中，跨多样化的机器人形态和环境扩展训练数据集已成为学习可泛化机器人策略的主流范式。这些策略通常通过行为克隆进行训练，以模仿预先收集的演示中的动作。然而，由于机器人动作与数据收集机器人的动态相关，不同的机器人可能需要不同的动作来实现相同的运动。这种差异阻碍了在多样化机器人之间的策略训练和部署。为了解决这个问题，我们提出使用笛卡尔状态增量作为跨机器人通用动作表示，并引入状态预测与自适应指令执行框架（State Prediction and Adaptive Command Execution，SPACE）。SPACE在三个层面上处理机器人动态变化：不同的机器人形态之间、同一形态的硬件单元之间，以及单个机器人在操作过程中的动态变化。它由两个组件组成：（i）预测几何末端执行器位移的笛卡尔状态增量策略，以及（ii）将预测的笛卡尔状态增量转换为特定机器人控制指令的动作适配器（Action Adapter）。实验表明，SPACE在从跨不同形态和同一形态的硬件单元收集的数据中学习时，显著优于直接预测控制指令的策略。SPACE在部署时也能在动态变化下保持稳健，包括控制频率、物体重量和控制器增益的变化。项目页面可访问：http://haeone.site/space-website/

View on arXiv Download PDF AI Translation

cs.RO / 8 / 2606.24078

MinInter: Minimizing Trajectory Interpolation During Data Augmentation for Imitation Learning

MinInter：在模仿学习的数据增强过程中最小化轨迹插值

Wang, Qingyang, Liu, Xingang, Yao, Changwei, Ouyang, Zikai, Liu, Junwei, Lu, Haibo, Zhang, Wei

Abstract

Imitation learning enables robots to acquire complex manipulation skills from demonstrations, but its effectiveness is limited by the cost of collecting high-quality data. Trajectory-level data augmentation methods alleviate this challenge by recombining expert demonstrations under varied initial states. However, such methods typically insert interpolations or other non-expert transition segments between disjoint parts, and such non-expert segments could reduce the quality of the generated data. This paper introduces Minimizing Interpolation (MinInter), an effective trajectory selection method that, for each sampled initial configuration, chooses the source demonstration requiring the least interpolation to form a complete trajectory. By explicitly minimizing interpolations during data generation, MinInter produces higher-quality synthetic demonstrations while remaining compatible with existing data generation frameworks. Experiments on 12 manipulation tasks with 26 variants from the MimicGen benchmark show that MinInter consistently improves both data generation success rates and policy success rates, with the largest gains on contact-rich, long-horizon and high-variance settings. Compared to the recent SkillGen framework, MinInter achieves higher policy success rates despite its conceptual simplicity, underscoring the value of interpolation minimization for data augmentation.

Chinese Translation

模仿学习使机器人能够从示范中获取复杂的操作技能，但其有效性受到收集高质量数据成本的限制。轨迹级数据增强方法通过在不同初始状态下重新组合专家示范来缓解这一挑战。然而，这些方法通常在不相连的部分之间插入插值或其他非专家过渡段，而这些非专家段可能会降低生成数据的质量。本文介绍了一种有效的轨迹选择方法——最小化插值（MinInter），该方法针对每个采样的初始配置，选择需要最少插值的源示范以形成完整轨迹。通过在数据生成过程中明确最小化插值，MinInter 生成了更高质量的合成示范，同时与现有的数据生成框架兼容。在 MimicGen 基准测试中对 12 个操作任务及其 26 个变体的实验表明，MinInter 始终提高了数据生成成功率和策略成功率，尤其在接触丰富、长时间跨度和高方差设置下获得了最大的提升。与近期的 SkillGen 框架相比，MinInter 尽管概念简单，却实现了更高的策略成功率，突显了插值最小化在数据增强中的重要价值。

View on arXiv Download PDF AI Translation

cs.RO / 9 / 2606.24089

DynaWM: Dynamics-Aware Distillation with World Model and Momentum Targets for Smooth Locomotion over Continuous Stairs

DynaWM：基于世界模型和动量目标的动态感知蒸馏方法，实现连续楼梯的平滑运动

Hou, Haidong, Yu, Zhangguo, Qi, Hengbo, Zhang, Jianlin

Abstract

Recent advances in control have enabled bipedal-wheeled robots to traverse slopes and single-step obstacles, yet long staircase traversal remains challenging as current teacher-student frameworks suffer from weakened dynamics-aware representations and incomplete terrain geometry encoding. To bridge this gap, we propose DynaWM, a dynamics-aware representation learning framework. To enhance terrain encoding capability and enable transparent assessment, we introduce a world model as a regularizer to enforce forward-dynamics awareness, preserving comprehensive terrain geometry while facilitating hierarchical encoding visualization. To stabilize knowledge transfer, we employ a momentum target encoder to provide consistent distillation targets, preventing dimensional collapse from non-stationary teacher updates. Evaluation of the learned representations through Principal Component Analysis (PCA) visualization and quantitative metrics reveals that our encoder hierarchically captures terrain geometry with higher terrain encoding capability, leading to enhanced terrain adaptability and motion smoothness. Experimental results in simulation and real hardware demonstrate that our method achieves superior terrain adaptability and motion smoothness, enabling bipedal-wheeled robots to overcome diverse continuous stairs, as shown in Fig. 1.

Chinese Translation

近年来，控制技术的进步使得双足轮式机器人能够跨越坡道和单步障碍物，但长楼梯的行走仍然具有挑战性，因为当前的教师-学生框架在动态感知表示和地形几何编码方面存在不足。为了解决这一问题，我们提出了DynaWM，一个动态感知表示学习框架。为了增强地形编码能力并实现透明评估，我们引入了世界模型作为正则化器，以强化前向动态感知，保留全面的地形几何信息，同时促进分层编码的可视化。为了稳定知识转移，我们采用动量目标编码器提供一致的蒸馏目标，防止因非平稳教师更新导致的维度崩溃。通过主成分分析（PCA）可视化和定量指标对学习到的表示进行评估，结果表明我们的编码器以更高的地形编码能力分层捕捉地形几何，从而提高了地形适应性和运动平滑性。在模拟和真实硬件上的实验结果表明，我们的方法实现了优越的地形适应性和运动平滑性，使双足轮式机器人能够克服各种连续楼梯，如图1所示。

View on arXiv Download PDF AI Translation

cs.RO / 10 / 2606.24101

NavWM: A Unified Navigation World Model for Foresight-Driven Planning

NavWM：一种统一的导航世界模型用于前瞻驱动的规划

Mei, Yanghong, Guo, Longteng, Yu, Ming-Ming, Zhao, Guiyu, He, Xingjian, Liu, Jing

Abstract

Conventional visual navigation policies often struggle with myopic decision-making and mode collapse in complex environments. While world models offer a promising alternative, existing paradigms typically isolate perception, generation, and control, failing to capture their shared spatio-temporal dynamics. In this paper, we propose NavWM, a unified navigation world model that seamlessly integrates latent world reasoning, multimodal action prediction, and controllable visual generation. At its core, NavWM leverages latent world tokens to distill geometric and semantic priors, endowing the agent with robust structural understanding. To overcome the limitations of deterministic policies, we introduce an anchor-based multimodal trajectory forecasting framework that generates a diverse action space. This inherent diversity explicitly empowers the generative world model to act as a robust closed-loop planner, utilizing visual foresight to evaluate and select the optimal path. Extensive experiments across diverse robotics datasets demonstrate that NavWM significantly advances the state-of-the-art, delivering remarkable improvements in both high-fidelity future state generation and zero-shot navigation success.

Chinese Translation

传统的视觉导航策略在复杂环境中常常面临短视决策和模式崩溃的问题。尽管世界模型提供了一种有前景的替代方案，但现有范式通常将感知、生成和控制孤立开来，未能捕捉它们共享的时空动态。在本文中，我们提出了NavWM，一种统一的导航世界模型，能够无缝集成潜在世界推理、多模态动作预测和可控视觉生成。NavWM的核心利用潜在世界标记提炼几何和语义先验，使代理具备强大的结构理解能力。为克服确定性策略的局限性，我们引入了一种基于锚点的多模态轨迹预测框架，生成多样化的动作空间。这种内在的多样性明确赋予生成世界模型作为强大闭环规划者的能力，利用视觉前瞻评估和选择最佳路径。通过在多样化的机器人数据集上进行广泛实验，NavWM显著推动了技术的前沿，在高保真未来状态生成和零样本导航成功率方面取得了显著提升。

View on arXiv Download PDF AI Translation

cs.RO / 11 / 2606.24191

The Evaluation Cost of Task Specialization in Evolutionary Multi-Robot Systems

进化多机器人系统中任务专业化的评估成本

Leopardi, Paolo, Hamann, Heiko, Kuckling, Jonas, Kaiser, Tanja Katharina

Abstract

Task specialization can improve the efficiency of multi-robot systems (MRSs). Previous works have investigated the emergence of task-specialist robot controllers through evolutionary optimization and have argued that task specialization is more likely to evolve when subtask behaviors are readily available as building blocks. However, the available evaluation budget must be distributed across all subtasks, whereas a single generalist behavior can exploit the entire budget for its own optimization. We present a cost-benefit analysis of evolving task-specialist versus generalist behaviors in a foraging scenario here. In a physics-based robotics simulator, we study the total evaluation budget required to evolve task-specialist behaviors that outperform generalist behaviors across MRS sizes. We show that with increasing MRS size, a lower total evaluation budget is sufficient to evolve specialists that outperform generalists.

Chinese Translation

任务专业化可以提高多机器人系统（MRSs）的效率。先前的研究探讨了通过进化优化出现的任务专业机器人控制器，并认为当子任务行为作为构建块 readily 可用时，任务专业化更可能进化。然而，可用的评估预算必须在所有子任务之间分配，而单一的通用行为可以将整个预算用于自身的优化。我们在此提出了在觅食场景中进化任务专业行为与通用行为的成本效益分析。在一个基于物理的机器人模拟器中，我们研究了进化出在不同 MRS 规模中优于通用行为的任务专业行为所需的总评估预算。我们表明，随着 MRS 规模的增加，较低的总评估预算足以进化出优于通用行为的专家。

View on arXiv Download PDF AI Translation

cs.RO / 12 / 2606.24208

Grounding Generative Policies in Physics: Optimization-Guided Diffusion for Robot Control

将生成策略与物理相结合：基于优化引导的扩散用于机器人控制

Bodmer, Sabrina, Zurbrügg, René, Portela, Tifanny, Ma, Hao, Didier, Alexandre, Hutter, Marco, Jones, Colin, Zeilinger, Melanie

Abstract

Diffusion models sample effectively from high-dimensional, multimodal distributions, but their outputs may violate deployment constraints. For task-space robot policies, generated grasps, waypoints, or trajectories can be distributionally valid yet infeasible, violating reachability, collision-avoidance, or closed-loop executability requirements. This embodiment gap limits zero-shot deployment across robots, even when the task-space behavior itself is transferable. We propose an inference-time optimization framework that couples the behavior generation to physical feasibility by formulating diffusion guidance as a constrained optimization problem. Our key insight is to replace the sampling perturbation in the backward process with an optimized correction, allowing hard constraints or soft penalties to be imposed during sampling without the need to retrain the diffusion model, while keeping samples close to the learned prior. We evaluate the method on dexterous grasp synthesis with reachability and collision-avoidance constraints, and dynamic manipulation with controller-level trackability constraints. Across settings and robot embodiments, optimization-guided denoising matches the feasibility of projection- and gradient-guidance baselines while better preserving grasp quality, and improving controller-level executability and task success, with task success improving by up to 20pp. on dexterous grasping and 23pp. on visuomotor manipulation over the best baseline.

Chinese Translation

扩散模型能够有效地从高维、多模态分布中进行采样，但其输出可能违反部署约束。对于任务空间中的机器人策略，生成的抓取、路径点或轨迹可能在分布上是有效的，但却不可行，违反了可达性、避碰或闭环可执行性要求。这种实现差距限制了机器人在零样本部署中的应用，即使任务空间行为本身是可转移的。我们提出了一种推理时优化框架，通过将扩散引导形式化为一个约束优化问题，将行为生成与物理可行性相结合。我们的关键见解是用优化的修正替代反向过程中的采样扰动，从而在采样过程中施加硬约束或软惩罚，而无需重新训练扩散模型，同时保持样本接近学习的先验。我们在具有可达性和避碰约束的灵巧抓取合成以及具有控制器级跟踪约束的动态操作上评估该方法。在不同设置和机器人实现中，基于优化引导的去噪在可行性上与投影和梯度引导的基线相匹配，同时更好地保持抓取质量，提高控制器级可执行性和任务成功率，在灵巧抓取上任务成功率提高了最多20个百分点，在视觉运动操作上提高了23个百分点，相较于最佳基线。

View on arXiv Download PDF AI Translation

cs.RO / 13 / 2606.24338

RoBoSR: Structured Scene Representations for Embodied Robotic Reasoning

RoBoSR：用于具身机器人推理的结构化场景表示

Hu, Kewei, Yu, Wanchan, Chen, Fangwen, Jiajian, Jing, Li, Zimeng, Wei, Ying, Liu, Tianhao, Zhang, Michael, Kang, Hanwen

Abstract

Despite rapid progress, embodied reasoning under real-world variability remains challenging. Existing approaches rely on demonstration-driven sequential biases, limiting flexibility in open-ended and long-horizon tasks that require structured reasoning over evolving states. We introduce RoBoSR, an intermediate structural representation that formulates manipulation as step-wise state transitions over semantically grounded, object-centric scene graphs. By modeling object states and their spatial relations at the perception-action interface, RoBoSR disentangles high-level task reasoning from raw inputs and enables structured reasoning over preconditions, effects, and goal states. This representation endows the agent with causal reasoning capability, enforcing subtask dependencies and supporting coherent long-horizon task planning. To learn such structure-aware reasoning, we construct Manip-Cognition-1.6M, an open-world dataset that jointly supervises scene understanding, instruction interpretation, and subtask planning across diverse tasks. Across several benchmarks and real-world demonstrations, our method consistently outperforms prompting-based methods and classical TAMP baselines in zero-shot generalization and long-horizon tasks. The results underscore structured intermediate representations as a critical inductive bias for scalable embodied reasoning.

Chinese Translation

尽管取得了快速进展，但在现实世界的变异性下进行具身推理仍然具有挑战性。现有方法依赖于基于示范的顺序偏差，这限制了在需要对不断变化的状态进行结构化推理的开放式和长时间任务中的灵活性。我们提出了RoBoSR，一种中间结构表示，将操作建模为在语义上扎根的、以对象为中心的场景图上的逐步状态转移。通过在感知-行动接口上建模对象状态及其空间关系，RoBoSR将高层次任务推理与原始输入解耦，并支持对前提条件、效果和目标状态的结构化推理。这种表示赋予代理因果推理能力，强制执行子任务依赖关系，并支持连贯的长时间任务规划。为了学习这种结构感知的推理，我们构建了Manip-Cognition-1.6M，这是一个开放世界数据集，联合监督场景理解、指令解释和多样任务中的子任务规划。在多个基准和现实世界的演示中，我们的方法在零-shot 泛化和长时间任务中始终优于基于提示的方法和经典的TAMP基线。结果强调了结构化中间表示作为可扩展具身推理的关键归纳偏差。

View on arXiv Download PDF AI Translation

cs.RO / 14 / 2606.24350

SlipSense: Multimodal Sensing for Online Slip Detection in Legged Robots

SlipSense：用于腿式机器人在线滑移检测的多模态传感

Liu, Iris Szu-Yao, Cheah, Chien Chern, Chuah, Meng Yee Michael

Abstract

Legged robots rely on accurate ground interaction awareness to traverse variable terrains, such as slippery surfaces. Existing slip detection methods often rely on kinematics and proprioception, which lack the sensitivity to detect early-stage slips that occur prior to catastrophic instability. Thus, this paper presents SlipSense, a novel framework for online force-based slip detection using a custom lightweight sensorized foot for quadrupeds to detect slip. The framework integrates a multimodal sensor design with a LSTM-based model to infer ground reaction forces and detect slip-indicative anomalies during locomotion. The proposed framework is deployed on a Unitree Go1 quadruped to demonstrate blind online slip detection over a slippery terrain. Our method detects early-stage slips down to an average displacement of 24.1 +/-6.4mm with an overall accuracy of 85.9%. This represents a 3.3-fold finer detection resolution and a 24% relative accuracy improvement over a standard kinematic baseline that uses foot velocity inferred through state estimation. The work in this paper serves as a foundation for force-aware gait adaptation in legged robotic locomotion, allowing future controllers to estimate terrain friction and adjust constraints, thus improving the overall stability of the system.

Chinese Translation

腿式机器人依赖于准确的地面交互感知来穿越多变的地形，例如滑溜的表面。现有的滑移检测方法通常依赖于运动学和本体感觉，这些方法缺乏对发生在灾难性不稳定之前的早期滑移的敏感性。因此，本文提出了SlipSense，一个基于力的在线滑移检测新框架，使用定制的轻量化传感足来检测四足机器人滑移。该框架将多模态传感器设计与基于长短期记忆网络（LSTM）的模型相结合，以推断地面反作用力并在运动过程中检测滑移指示异常。所提出的框架在Unitree Go1四足机器人上部署，以演示在滑溜地形上的盲在线滑移检测。我们的方法能够检测到早期滑移，平均位移为24.1 +/- 6.4mm，整体准确率为85.9%。这代表着比使用状态估计推断的足部速度的标准运动学基线提高了3.3倍的检测分辨率和24%的相对准确性提升。本文的工作为腿式机器人运动中的力感知步态适应奠定了基础，使未来的控制器能够估计地面摩擦并调整约束，从而提高系统的整体稳定性。

View on arXiv Download PDF AI Translation

cs.RO / 15 / 2606.24377

PDS Joint: A Parametric Double-Spiral Joint Tailored for Dexterous Hands

PDS关节：一种为灵巧手定制的参数化双螺旋关节

Li, Haoyang, Wen, Yibo, Fan, Yixiang, Xu, Yiheng, Yue, Yufeng

Abstract

Compliant joints can embed safety and adaptability into dexterous hands, but achieving large-stroke anthropomorphic motion while maintaining joint-specific, directiondependent stiffness and reliable proprioception remains challenging. This paper presents the PDS joint, a parametric doublespiral (PDS) compliant joint that enables systematic shaping of directional stiffness across multiple deformation modes, including flexion/extension, abduction/adduction, and pronation/supination. We instantiate the joint using Archimedean and logarithmic spiral templates for different hand joints and introduce an asymmetry ratio to tailor stiffness distributions for both grasp stability and hyperextension resistance. To make the joint practically usable under large deformation, we co-design embedded inductive proprioception and propose a learningbased calibration pipeline that maps raw inductive signals to joint states using ArUco-marker tracking. Experiments characterize the stiffness landscapes across geometric parameters and demonstrate a non-monotonic dependence of lateral support on asymmetry, indicating the importance of principled parameter tuning. For joint-state estimation in the most challenging abduction/adduction motion, a learned multilayer-perceptron (MLP) mapping reduces the error compared with conventional curve fitting by 41.6%. Finally, we integrate the proposed joints into an open-source dexterous hand as a demonstration platform, on which the hand grasps a set of nine everyday objects and performs safe, contact-rich human-involved interactions.

Chinese Translation

柔性关节可以为灵巧手嵌入安全性和适应性，但在保持关节特定、方向依赖的刚度和可靠的本体感觉的同时，实现大幅度的人体运动仍然具有挑战性。本文提出了PDS关节，一种参数化双螺旋（PDS）柔性关节，能够在多种变形模式下系统地塑造方向刚度，包括屈伸、外展/内收和旋前/旋后。我们使用阿基米德螺旋和对数螺旋模板实例化该关节，以适应不同的手部关节，并引入不对称比来定制刚度分布，以增强抓握稳定性和抗过度伸展能力。为了使该关节在大变形下实用，我们共同设计了嵌入式感应本体感觉，并提出了一种基于学习的校准流程，通过ArUco标记跟踪将原始感应信号映射到关节状态。实验表征了几何参数下的刚度分布，并展示了侧向支撑与不对称之间的非单调依赖关系，表明了原则性参数调优的重要性。在最具挑战性的外展/内收运动中，学习的多层感知器（MLP）映射相比传统曲线拟合减少了41.6%的误差。最后，我们将所提出的关节集成到一个开源灵巧手中作为演示平台，该手能够抓取九种日常物品并进行安全、丰富接触的人机交互。

View on arXiv Download PDF AI Translation

cs.RO / 16 / 2606.24403

RE4: Transformation-aware Imitation of Object Interactions Using Manipulation Modes

RE4：基于变换感知的对象交互模仿使用操控模式

Chawla, Arsh, Shome, Rahul

Abstract

Object interaction tasks have been a focus of advances in imitation learning. End-to-end methods, dominated by diffusion and flow-based variants have shown leaps in performance while sacrificing interpretability. Object-centric and pose-informed variants have had a role in learning from demonstration in manipulation tasks. In this paper, we revisit a few modern imitation learning benchmarks for object interactions, with the aim of composing a framework that repurposes principled theories of manipulation, preserving both performance and interpretability. For image observations, lightweight training is proposed for model-free pose estimation of the target object, using self-supervision over the demonstration data available for imitation learning. This information is then used to inform a manipulation mode-aware retrieval of a demonstration, a mode-aware transformation, a replan step that connects to the retrieval point while preserving mode constraints, and finally rolling out the transformed demonstration. These compose four key steps of the proposed RE4 framework, evaluated over state-based and image-based benchmarks in Push-T and Robomimic. An adversarial benchmark that evaluates sparse data regions of image-based Push-T showcases the robustness, further bolstered by indications from low-data regime experiments. The current work shows promise in using simple interpretable building blocks to learn manipulation skills.

Chinese Translation

对象交互任务一直是模仿学习进展的重点。以扩散和基于流的变体为主导的端到端方法在性能上取得了显著进展，但牺牲了可解释性。以对象为中心和姿态信息为基础的变体在操控任务的示范学习中发挥了作用。本文重新审视了一些现代对象交互的模仿学习基准，旨在构建一个框架，该框架重新利用操控的原则理论，同时保持性能和可解释性。对于图像观测，提出了一种轻量级训练方法，用于对目标对象进行无模型的姿态估计，利用可用于模仿学习的示范数据进行自我监督。然后，这些信息用于指导基于操控模式的示范检索、模式感知的变换、连接到检索点的重新规划步骤，同时保持模式约束，最后展开变换后的示范。这些组成了所提RE4框架的四个关键步骤，在Push-T和Robomimic的状态基础和图像基础基准上进行了评估。一个评估图像基础Push-T稀疏数据区域的对抗性基准展示了其鲁棒性，并通过低数据环境实验的结果进一步增强。当前的工作展示了使用简单可解释的构建模块来学习操控技能的潜力。

View on arXiv Download PDF AI Translation

cs.RO / 17 / 2606.24445

Legible and Intuitive Multi-modal Robot State and Intent Communication Validated in Online and Real-world Studies

可读且直观的多模态机器人状态与意图沟通在在线和现实世界研究中的验证

Schreiter, Tim, Rüppel, Jens V., Rudenko, Andrey, Magnusson, Martin, Lilienthal, Achim J.

Abstract

Effective robot-to-human communication can increase transparency and trust, reduce uncertainty, and contribute to safer collaboration in shared workspaces. Designing and validating an effective robot communication strategy is challenging due to the varying and often limited communication modalities across robots, differences in how diverse recipients interpret messages, and the underexplored virtual-to-real gap in studies of communication legibility. We present a systematic, large-scale comparative validation of existing communication strategies for a mobile non-humanoid robot across message types and settings (online and in-person). Based on the prescribed message types in the existing standards for industrial robots, we realize and compare a low-expressive, unimodal LED-based strategy with a highly expressive, multimodal one that leverages robotic gaze, gestures, and voice. For each strategy, we analyze the communication of a turning intention, an attention request, error status, whether the robot is stuck, and whether it is functioning normally. We evaluate these strategies in replicated online and in-person experiments. We find strong evidence that highly expressive multimodal communication is perceived as more legible and intuitive than unimodal LED-based communication. Comparing the online and real-world study findings, we observe a notable decrease in overall legibility, particularly for signaling with LEDs. Similarly, confidence in message interpretation decreases during the real-world evaluation.

Chinese Translation

有效的机器人与人类之间的沟通可以提高透明度和信任，减少不确定性，并有助于在共享工作空间中实现更安全的协作。由于机器人之间沟通方式的多样性和通常的局限性、不同接收者对信息的解读差异，以及在沟通可读性研究中尚未充分探讨的虚拟与现实之间的差距，设计和验证有效的机器人沟通策略面临挑战。我们对一种移动非人形机器人在不同信息类型和环境（在线和面对面）下的现有沟通策略进行了系统的大规模比较验证。基于现有工业机器人标准中规定的信息类型，我们实现并比较了一种低表达性、单模态的基于LED的策略与一种高度表达性、多模态的策略，该策略利用机器人视线、手势和声音。对于每种策略，我们分析了转向意图、注意请求、错误状态、机器人是否卡住以及其是否正常工作的沟通。我们在重复的在线和面对面实验中评估这些策略。我们发现，高度表达性的多模态沟通被认为比单模态的基于LED的沟通更可读和直观。比较在线和现实世界研究的结果，我们观察到整体可读性显著下降，尤其是在使用LED信号时。同样，在现实世界评估中，对信息解读的信心也有所下降。

View on arXiv Download PDF AI Translation

cs.RO / 18 / 2606.24448

Supervise What Survives: Geometry-Guided VLA Adaptation from Synthetic Robot Videos

监督生存的内容：基于几何的从合成机器人视频中适应视觉-语言-动作模型

Chen, Danze, Chen, Yanzhe, Huang, Qiming, Cao, Zhijun, Gao, Chen, Shou, Mike Zheng

Abstract

Vision-Language-Action (VLA) models require large-scale video-action pairs, yet real teleoperation remains scarce. While generated robot videos offer a scalable alternative, existing methods treat them as real robot data by recovering pseudo-actions from synthesized pixels. We argue that deriving low-level control from generated visuals is a mismatched abstraction. A video captures only \emph{geometry}: the spatial trajectory representing the \emph{where} of a task. A real demonstration captures \emph{control}: the exact motor commands representing the \emph{how}. Human-to-robot video generation preserves these unequally: the visible geometry survives the generation process, while the underlying control signals are lost. This \textbf{Asymmetric Preservation Principle} dictates a clean rule: this surviving geometry should solely supervise visual perception, leaving control to real demonstrations. Following this principle, we propose \textbf{GRA} (\textbf{G}eometry-guided \textbf{R}epresentation \textbf{A}lignment), which extracts the geometric content as future 2D end-effector waypoints, computed from the source human video through pose estimation, retargeting, simulation, and calibrated projection, and routes them to the VLA vision backbone via an auxiliary 2D head. The action head is trained on real demonstrations only. During fine-tuning, the waypoint loss persists as a \textbf{spatial representation anchor} that prevents the backbone from losing its geometric grounding. On real-robot tasks, GRA outperforms pseudo-action baselines under matched data budgets and narrows the gap to policies trained with substantially more real demonstrations, suggesting that correctly routed geometry bridges generated videos to robot policies more reliably than recovered actions.

Chinese Translation

视觉-语言-动作（VLA）模型需要大规模的视频-动作对，但真实的远程操作仍然稀缺。虽然生成的机器人视频提供了一种可扩展的替代方案，但现有方法将其视为真实机器人数据，通过从合成像素中恢复伪动作。我们认为，从生成的视觉内容中推导低级控制是一种不匹配的抽象。视频仅捕捉到 extit{几何}：表示任务 extit{在哪里}的空间轨迹。真实演示捕捉到 extit{控制}：表示任务 extit{如何}的确切电机指令。人机视频生成在这两者之间的保留是不均等的：可见的几何在生成过程中得以保留，而潜在的控制信号则丢失。这一 extbf{不对称保留原则}规定了一个明确的规则：这种生存的几何应仅监督视觉感知，而将控制留给真实演示。遵循这一原则，我们提出了 extbf{GRA}（ extbf{几何引导的 extbf{表征对齐}）），该方法提取几何内容作为未来的二维末端执行器路径点，这些路径点通过姿态估计、重定向、仿真和标定投影从源人类视频中计算得出，并通过辅助的二维头部将其路由到VLA视觉主干。动作头仅在真实演示上进行训练。在微调过程中，路径点损失作为 extbf{空间表征锚点}持续存在，防止主干失去其几何基础。在真实机器人任务中，GRA在匹配数据预算下超越了伪动作基线，并缩小了与使用大量真实演示训练的策略之间的差距，这表明正确路由的几何比恢复的动作更可靠地将生成的视频与机器人策略连接起来。

View on arXiv Download PDF AI Translation

cs.RO / 19 / 2606.24450

NoContactNoWorries: Estimating Contact through Vision and Proprioception for In-Hand Dexterous Manipulation

无接触无忧：通过视觉和本体感觉估计手中灵巧操作的接触

Patil, Soham, Das, Avirup, Bhosale, Sourabh, Roy, Spandan

Abstract

Perceiving physical contact is fundamental to dexterous manipulation. While robots often rely on dedicated hardware tactile sensors, humans exhibit a remarkable ability to infer contact by integrating visual information with an innate sense of their body's pose and movement. Inspired by this embodied perceptual skill, we investigate whether a robot can learn to infer contact from vision, an approach that also offers a scalable alternative to tactile hardware specifically for binary contact estimation, which faces practical challenges in cost, fragility, and integration. We present NoContactNoWorries, a transformer-based multimodal framework that fuses RGB-D vision with the robot's proprioception to infer binary contact states as a pseudo-tactile signal for hand-object interactions. We validate by training a single contact prediction model on multiple objects and show that the inferred contact signal supports downstream reinforcement learning agents for in-hand object reorientation, generalizing to novel objects. Experiments in both simulation and on a real-world robot validate our approach, highlighting the feasibility of inferring contact from vision and proprioception. Project Page: https://soham2560.github.io/no-contact-no-worries/

Chinese Translation

感知物理接触是灵巧操作的基础。虽然机器人通常依赖专用的硬件触觉传感器，但人类展现出通过整合视觉信息与自身姿态和运动的内在感知能力来推断接触的显著能力。受到这种具身感知技能的启发，我们研究了机器人是否能够通过视觉学习推断接触，这种方法还为二元接触估计提供了一种可扩展的替代方案，后者在成本、脆弱性和集成方面面临实际挑战。我们提出了NoContactNoWorries，这是一个基于变换器的多模态框架，将RGB-D视觉与机器人的本体感觉融合，以推断手-物体交互中的二元接触状态，作为伪触觉信号。我们通过在多个物体上训练单一接触预测模型进行验证，并展示推断的接触信号支持下游强化学习代理进行手中物体重新定向，并能推广到新物体。在仿真和真实机器人上的实验验证了我们的方法，突显了从视觉和本体感觉中推断接触的可行性。项目页面：https://soham2560.github.io/no-contact-no-worries/

View on arXiv Download PDF AI Translation

cs.RO / 20 / 2606.24462

Varying Bundle Size Reactive Multi-Task Assignment using Selective Cost Estimation for Multi-Agent Systems

基于选择性成本估计的可变捆绑大小反应式多任务分配在多智能体系统中的应用

Dahlquist, Niklas, Velhal, Shridhar, Nikolakopoulos, George

Abstract

This paper presents a scalable framework for multi-robot task allocation in complex environments where estimating task execution costs is computationally expensive. While combinatorial auction-based approaches offer reliable solutions, the exponential complexity of bundle generation typically renders them intractable for real-time reactive applications, particularly when accurate path planning is required for cost validation. We address this through a distributed, two-stage multi-fidelity bundle generation approach. Agents utilize a local search tree guided by a low-fidelity heuristic (such as euclidean distance) to rapidly explore the bundle space, applying high-fidelity path planning only to the most promising candidates in a best-first manner. These refined bids are then submitted to a central coordinator that solves a set packing problem to ensure global feasibility and maximize the overall utility. Simulation results in multiple environments demonstrate that the framework is able to improve the performance of reactive auction-based task allocation. Overall, the presented framework is shown to enable reactive task allocation with dynamic bundle sizes in multiple settings without exposing the agents' state and internal cost estimation models.

Chinese Translation

本文提出了一种可扩展的多机器人任务分配框架，适用于在复杂环境中任务执行成本估计计算开销较大的情况。尽管基于组合拍卖的方法提供了可靠的解决方案，但捆绑生成的指数复杂性通常使其在实时反应式应用中难以处理，尤其是在需要准确路径规划以验证成本时。我们通过一种分布式的双阶段多保真度捆绑生成方法来解决这一问题。智能体利用由低保真度启发式（如欧几里得距离）指导的局部搜索树，快速探索捆绑空间，仅对最有前景的候选者应用高保真度路径规划，采用最佳优先的方式。这些精细化的出价随后提交给中央协调者，后者解决一个集合打包问题，以确保全局可行性并最大化整体效用。在多个环境中的仿真结果表明，该框架能够提高基于拍卖的反应式任务分配的性能。总体而言，所提出的框架显示出能够在多个设置中实现动态捆绑大小的反应式任务分配，而无需暴露智能体的状态和内部成本估计模型。

View on arXiv Download PDF AI Translation

cs.RO / 21 / 2606.24466

FT-WBC: Learning Fault-Tolerant Whole-Body Control for Legged Loco-Manipulation

FT-WBC：学习容错全身控制以实现腿式运动操控

Zhong, Yudong, Mai, Pengfei, Guo, Sikai, Cao, Jiahang, Bi, Zhihai, Liu, Qiuyue, Feng, Ziyan, Zhou, Jinni, Ma, Jun

Abstract

Legged manipulators combine the mobility of legged platforms with the manipulation capability of robotic arms. However, arm-induced Center-of-Mass shifts and dynamic disturbances make the system more prone to instability under actuator failures, potentially leading to falls, task failures, or safety risks. Existing fault-tolerant control methods mainly focus on locomotion alone, leaving the coupled problem of whole-body stability and arm reachability in fault-tolerant loco-manipulation largely unaddressed. To bridge this gap, we propose FT-WBC, a fault-tolerant loco-manipulation framework for robust whole-body control of legged manipulators under actuator failures. FT-WBC adopts a decoupled upper- and lower-body policy architecture and introduces two key modules: a Fault Estimator (FE) and a Posture Adaptation Module (PAM). The FE predicts faulty joints from lower-body proprioceptive histories, while the PAM uses this fault information to adapt the base posture plan generated by the arm policy, converting potentially unstable posture requests into safe and executable base posture commands. Through this fault-aware posture adaptation mechanism, FT-WBC synthesizes compensatory gaits under actuator failures and preserves as much arm workspace as possible while maintaining whole-body stability. Simulation and real-world experiments show that FT-WBC significantly improves survival rate and workspace under weakening or locked failures, and transfers zero-shot to a real legged manipulator in the real world.

Chinese Translation

腿式操纵器结合了腿式平台的移动性和机械臂的操控能力。然而，机械臂引起的质心偏移和动态干扰使得系统在执行器故障下更容易出现不稳定，可能导致跌倒、任务失败或安全风险。现有的容错控制方法主要集中于单一的运动控制，未能有效解决腿式运动操控中全身稳定性与机械臂可达性之间的耦合问题。为填补这一空白，我们提出了FT-WBC，一个针对腿式操纵器在执行器故障下的鲁棒全身控制的容错运动操控框架。FT-WBC采用了上下肢解耦的策略架构，并引入了两个关键模块：故障估计器（Fault Estimator, FE）和姿态适应模块（Posture Adaptation Module, PAM）。FE根据下肢本体感知历史预测故障关节，而PAM则利用这些故障信息来调整由机械臂策略生成的基础姿态计划，将潜在不稳定的姿态请求转换为安全且可执行的基础姿态指令。通过这一故障感知的姿态适应机制，FT-WBC在执行器故障下合成补偿步态，并尽可能保留机械臂的工作空间，同时维持全身稳定性。仿真和实际实验表明，FT-WBC在执行器减弱或锁定故障下显著提高了生存率和工作空间，并在现实世界中实现了对真实腿式操纵器的零样本迁移。

View on arXiv Download PDF AI Translation

cs.RO / 22 / 2606.24472

G$^3$VLA: Geometric inductive bias for Vision-Language-Action Models

G$^3$VLA：面向视觉-语言-动作模型的几何归纳偏置

Peng, Yue, Zhao, Yongzhe, Habuda, Artur, Pham, Khuyen, Zhu, Yanheng, Le, Tran Nguyen, Abu-Dakka, Fares, Guo, Li

Abstract

Vision-language-action (VLA) models have made rapid progress in generalist robot manipulation by harnessing semantic knowledge from pretrained vision-language backbones, but their visual tokens remain grounded in 2D image coordinates rather than the calibrated geometry of the robot's cameras -- a mismatch especially pronounced in multi-camera setups, where views are coupled by known intrinsics and extrinsics yet processed as independent images. We propose G$^3$VLA, a camera-aware geometric module that injects calibrated structure into the visual-token stream of a pretrained VLA without altering its action space or imitation objective, combining intrinsic-conditioned ray embeddings, projective positional encoding (PRoPE), and bidirectional cross-view fusion. Geometric supervision is provided either from ground-truth point maps when available, or from confidence-gated $\pi^3$X teacher predictions, requiring no depth sensors or manual annotations. Instantiated on $\pi_0$, G$^3$VLA yields consistent gains across the LIBERO suites, RoboCasa24, RoboTwin2.0, and real-robot settings, with the largest improvements on spatially and object-sensitive tasks. We further validate on $\pi_{0.5}$ and GR00T 1.5, with results suggesting that geometric transfer is most effective when geometry-aware tokens have direct access to the action generation pathway. Our project page is at https://sites.google.com/view/g3vla

Chinese Translation

视觉-语言-动作（VLA）模型通过利用预训练视觉-语言骨干网络的语义知识，在通用机器人操作方面取得了快速进展，但它们的视觉标记仍然基于二维图像坐标，而不是机器人相机的标定几何结构——这种不匹配在多相机设置中尤为明显，在这种情况下，视图由已知的内参和外参耦合，但作为独立图像进行处理。我们提出了G$^3$VLA，一个感知相机的几何模块，它在不改变其动作空间或模仿目标的情况下，将标定结构注入预训练VLA的视觉标记流中，结合了内参条件的光线嵌入、投影位置编码（PRoPE）和双向跨视图融合。几何监督来自于真实点图（当可用时），或来自于信心门控的$ ext{π}^3 ext{X}$教师预测，无需深度传感器或手动标注。在$ ext{π}_0$上实例化的G$^3$VLA在LIBERO套件、RoboCasa24、RoboTwin2.0和真实机器人环境中均取得了一致的提升，空间和物体敏感任务的改进最大。我们进一步在$ ext{π}_{0.5}$和GR00T 1.5上进行了验证，结果表明，当几何感知标记直接访问动作生成路径时，几何转移最为有效。我们的项目页面位于https://sites.google.com/view/g3vla

View on arXiv Download PDF AI Translation

cs.RO / 23 / 2606.24489

Decentralized Pose Graph Riemannian Optimization for Object-based Multi-Robot SLAM

基于对象的多机器人 SLAM 的去中心化位姿图黎曼优化

Zhao, Yixian, Huang, Yan, Xu, Yang, Li, Liang, Xu, Jinming

Abstract

Pose graph optimization (PGO) is a key back-end component for state estimation in networked multi-robot simultaneous localization and mapping (SLAM). In object-based multi-robot SLAM, the problem becomes more tightly coupled because robots must jointly estimate both their trajectories and the poses of persistent objects observed by multiple agents. Existing decentralized solutions often assume that the communication graph closely matches the physical interaction topology, which is restrictive in realistic deployments where communication is sparse, intermittent, or time-varying. This paper presents a fully decentralized Riemannian optimization framework for object-based multi-robot PGO that decouples the coupled estimation problem via a consensus mechanism, enabling flexible communication topologies. To improve convergence under limited communication budgets, we further develop a distributed approximate-Newton scheme that exploits local second-order information while operating directly on the SE(d) manifold to preserve geometric consistency, and we establish the convergence to Riemannian first-order stationary points and provide a local condition-number analysis explaining the benefit of approximate second-order information over first-order Riemannian descent. The resulting method reduces iteration count and communication overhead without sacrificing estimation accuracy. Extensive evaluations on public benchmarks, large-scale simulations, and real-world multi-robot experiments demonstrate improved accuracy, runtime efficiency, scalability across network topologies, and robustness to communication failures.

Chinese Translation

位姿图优化（PGO）是网络化多机器人同时定位与地图构建（SLAM）中状态估计的关键后端组件。在基于对象的多机器人 SLAM 中，由于机器人必须共同估计其轨迹以及多个代理观察到的持久对象的位姿，因此问题变得更加紧密耦合。现有的去中心化解决方案通常假设通信图与物理交互拓扑密切匹配，这在通信稀疏、不稳定或时变的现实部署中是有限制的。本文提出了一种完全去中心化的黎曼优化框架，用于基于对象的多机器人 PGO，通过共识机制解耦耦合估计问题，从而支持灵活的通信拓扑。为了在有限的通信预算下提高收敛性，我们进一步开发了一种分布式近似牛顿方案，该方案利用局部二阶信息，同时直接在 SE(d) 流形上操作，以保持几何一致性，并建立了收敛到黎曼一阶驻点的理论，提供了局部条件数分析，解释了近似二阶信息相较于一阶黎曼下降的优势。所提出的方法在不牺牲估计精度的情况下减少了迭代次数和通信开销。在公共基准、规模较大的仿真和真实世界的多机器人实验中进行了广泛评估，结果表明该方法在精度、运行效率、网络拓扑的可扩展性以及对通信故障的鲁棒性方面均有所改善。

View on arXiv Download PDF AI Translation

cs.RO / 24 / 2606.24546

Explaining Failures of Cyber-Physical Systems with Actual Causality

用实际因果关系解释网络物理系统的失败

Elimelech, Khen, Yaacov, Tom, Kelly, David A., Chockler, Hana, Vardi, Moshe Y.

Abstract

Modern autonomous Cyber-Physical Systems (CPSs), such as self-driving cars, face increasingly complex demands, and yet are expected to act reliably. The black-box nature often characterizing such systems, especially those relying on neural components, makes it impossible to fully verify the system behavior prior to deployment. Unfortunately, unexpected failures-when the system does not comply with its specification-are inevitable and may have catastrophic implications. To improve trust in the system and facilitate future mitigation after a failure occurs, it is important to try to derive an explanation for the unexpected system behavior. This paper introduces the novel concept of leveraging the framework of actual causality for CPS failure explanation. Up until now, this framework was only used to derive explanations in the context of simple systems, such as image classifiers. This paper addresses the theoretical gaps and provides the guidance needed to allow for correct explanation derivation in the CPS domain. Beyond the theoretical contribution, the paper presents two novel, practical, system-agnostic explanation derivation algorithms, allowing to prioritize either explanation optimality or derivation efficiency. The approach is demonstrated and evaluated in the context of a neural-network-controlled autonomous car, designed to avoid collisions.

Chinese Translation

现代自主网络物理系统（Cyber-Physical Systems, CPS），如自动驾驶汽车，面临日益复杂的需求，同时又被期望能够可靠地运行。这类系统通常具有黑箱特性，尤其是那些依赖于神经网络组件的系统，使得在部署之前完全验证系统行为变得不可能。不幸的是，意外的失败——即系统未能遵循其规范——是不可避免的，且可能带来灾难性的后果。为了增强对系统的信任并促进在发生失败后的未来缓解，尝试为意外的系统行为提供解释是至关重要的。本文引入了利用实际因果关系框架进行CPS失败解释的新概念。迄今为止，该框架仅用于在简单系统（如图像分类器）的背景下推导解释。本文解决了理论上的空白，并提供了在CPS领域正确推导解释所需的指导。除了理论贡献外，本文还提出了两种新颖的、与系统无关的解释推导算法，允许优先考虑解释的最优性或推导的效率。该方法在一个旨在避免碰撞的神经网络控制的自主汽车的背景下进行了演示和评估。

View on arXiv Download PDF AI Translation

cs.RO / 25 / 2606.24552

Enabling Robust Cloth Manipulation via Inference-Time Simulator-in-the-Loop Refinement

通过推理时的模拟器循环优化实现鲁棒的布料操控

Liu, Xin, Li, Yulin, Li, Ziming, Jing, Pengyu, Huang, Zhenhao, Zhou, Bingyang, Zeng, Ziqiu, Luo, Siyuan, Qi, Chenkun, Shi, Fan

Abstract

Simulator-in-the-loop optimization offers a promising inference-time mechanism for robot manipulation. It uses a physical simulator as a backend rollout engine to evaluate candidate trajectories in parallel and refine nominal actions online, a paradigm proven effective in rigid-body manipulation where state and contact are relatively tractable. We bring this paradigm to real-world cloth manipulation from a single RGB input through three pillars. (i) We design a scalable synthetic-data generation and inference-time rollout pipeline built on FLASH, a deformable-object simulator that provides a practical balance among physical fidelity, numerical stability, and rollout efficiency. (ii) We develop a real-to-sim module, trained purely on synthetic data, that maps a single RGB observation to simulation-compatible cloth state by fusing pretrained visual features with learnable canonical tokens. (iii) We perform online planning by coupling a sparse-mesh rollout backend with prior-guided MPPI, anchored at an offline-distilled policy trajectory, preserving manipulation-relevant deformation and contact while enabling sufficient parallel rollout batches. Real-robot experiments show higher success rates and stronger robustness than baseline methods.

Chinese Translation

模拟器循环优化为机器人操控提供了一种有前景的推理时机制。它利用物理模拟器作为后端展开引擎，能够并行评估候选轨迹并在线优化名义动作，这一范式在刚体操控中已被证明有效，因为其状态和接触相对可处理。我们将这一范式引入到基于单一RGB输入的现实世界布料操控中，依托三个支柱进行实现。(i) 我们设计了一个可扩展的合成数据生成和推理时展开管道，基于FLASH，一个提供物理保真度、数值稳定性和展开效率之间实用平衡的可变形物体模拟器。(ii) 我们开发了一个真实到模拟模块，该模块完全基于合成数据进行训练，将单一RGB观测映射到与模拟兼容的布料状态，通过融合预训练的视觉特征与可学习的规范化标记。(iii) 我们通过将稀疏网格展开后端与基于先验引导的MPPI相结合，进行在线规划，锚定于离线提炼的策略轨迹，保留与操控相关的变形和接触，同时实现足够的并行展开批次。真实机器人实验显示出比基线方法更高的成功率和更强的鲁棒性。

View on arXiv Download PDF AI Translation

cs.RO / 26 / 2606.24628

ArtiTwinSplat: Interactable Digital Twin Reconstruction via Gaussian Splatting from RGB-D videos

ArtiTwinSplat：通过RGB-D视频的高斯点云重建可交互的数字双胞胎

Mishra, Pranjal, Zurbrügg, René, Wilder-Smith, Max, Hutter, Marco, Pollefeys, Marc, Bauer, Zuria, Blum, Hermann

Abstract

Deploying robots in unstructured real-world environments needs accurate, interactive models of the objects. Constructing these models at scale remains a critical bottleneck for robotic system integration. We present ArtiTwinSplat, a framework that automatically constructs articulated, photo-realistic digital twins of objects directly from RGB-D videos, requiring no CAD models, simulation assets, or manual annotations. Our method is built on 3D Gaussian Splatting that preserve geometric fidelity and photometric realism, coupled with an unsupervised articulation discovery pipeline that recovers part structure and joint kinematics from observed motion alone. With tracking and optimization stages our method provides stable, queryable digital twins that support real-time rendering, viewpoint control, and interactive manipulation. Unlike prior methods confined to simulation, ArtiTwinSplat operates directly on real-world observations and produces twins that are immediately usable by downstream robot planning and learning systems. This method offers a practical, scalable pathway toward digital twin construction, lowering the integration barrier for articulated object manipulation in embodied AI and human-robot collaboration contexts.

Chinese Translation

在非结构化的真实世界环境中部署机器人需要对物体进行准确、交互式的建模。然而，在大规模构建这些模型仍然是机器人系统集成的一个关键瓶颈。我们提出了ArtiTwinSplat，一个框架，能够直接从RGB-D视频中自动构建物体的关节化、照片级真实感的数字双胞胎，且不需要CAD模型、仿真资产或手动标注。我们的方法基于3D高斯点云重建，保持几何保真性和光度真实感，并结合一个无监督的关节发现管道，仅通过观察到的运动恢复部件结构和关节运动学。通过跟踪和优化阶段，我们的方法提供了稳定的、可查询的数字双胞胎，支持实时渲染、视角控制和交互式操作。与以往仅限于仿真的方法不同，ArtiTwinSplat直接在真实世界观察数据上运行，并生成可以立即被下游机器人规划和学习系统使用的双胞胎。该方法为数字双胞胎的构建提供了一条实用的、可扩展的路径，降低了在具身人工智能和人机协作背景下对关节化物体操作的集成门槛。

View on arXiv Download PDF AI Translation

cs.RO / 27 / 2606.24631

Optimization-based Safe Trajectory Planning for Autonomous Ground Vehicle in Multi-Floor Scenarios

基于优化的多层场景自主地面车辆安全轨迹规划

Xiang, Zishang, Zhang, Runda, Chai, Runqi, Chen, Kaiyuan, Chai, Senchun, Xia, Yuanqing

Abstract

The development of trajectory planning strategies for autonomous ground vehicles (AGVs) represents a prevailing research interest within the domain of intelligent transportation systems. This paper introduces a trajectory planning framework tailored for multi-floor scenarios. The framework consists of two main modules: the task planning module and the trajectory planning module. The task planning module involves a strategic selection phase, where a task planning strategy based on generalized voronoi diagrams (GVD) and multi-objective algorithms is proposed to select the floor exits for each floor. The trajectory planning module utilizes optimization-based methods to generate high-quality trajectories, and a warm-started hierarchical planning framework is designed to ensure rapid convergence. Additionally, for handling complex obstacle constraints, a correlation constraint calculation method is designed for reducing obstacle constraints in trajectory planning. Finally, the feasibility and effectiveness of the proposed framework are verified through simulations.

Chinese Translation

自主地面车辆（AGVs）轨迹规划策略的发展在智能交通系统领域中代表了一项重要的研究兴趣。本文介绍了一种针对多层场景的轨迹规划框架。该框架由两个主要模块组成：任务规划模块和轨迹规划模块。任务规划模块涉及一个战略选择阶段，提出了一种基于广义Voronoi图（GVD）和多目标算法的任务规划策略，以选择每层的出口。轨迹规划模块利用基于优化的方法生成高质量的轨迹，并设计了一个预热启动的分层规划框架，以确保快速收敛。此外，为了处理复杂的障碍约束，设计了一种相关约束计算方法，以减少轨迹规划中的障碍约束。最后，通过仿真验证了所提框架的可行性和有效性。

View on arXiv Download PDF AI Translation

cs.RO / 28 / 2606.24633

Beyond Monotonic Progress: Retry-Supervised Value Learning for Robot Imitation

超越单调进展：用于机器人模仿的重试监督价值学习

Qin, Xinyao, Lu, Junjie, Wang, Kaixin, Zhang, Chuheng, Kang, Sinjae, Lee, Kimin, Xu, Min, Liang, Bin, Yang, Jun, Zhao, Li

Abstract

Human demonstrations for robot imitation learning often contain mistakes and corrective behaviors, such as imprecise grasps, object misalignment, unstable contact, and repeated attempts. While these segments are commonly treated as noisy or suboptimal data, they provide valuable evidence about when execution deviates from a desirable path and how task feasibility can be restored. However, existing reward and value models often rely on monotonic progress assumptions, which capture coarse task advancement but may overlook local execution errors and corrective behaviors in imperfect demonstrations. In this work, we propose ReTVL (ReTry-Supervised Value Learning), a framework for learning mistake-sensitive value functions from mixed-quality robot demonstrations by leveraging retry events as sparse supervision. ReTVL captures the local degradation-and-recovery structure around mistakes by combining global progress calibration with local pairwise preference learning induced by sparsely annotated retry keypoints. The learned value model is then used to reweight demonstration chunks for downstream behavior cloning, reducing the influence of harmful execution errors while preserving useful corrective behaviors. Experiments on real-robot manipulation tasks show that ReTVL produces more fine-grained value estimates than progress-based baselines and improves imitation learning from imperfect demonstrations.

Chinese Translation

人类示范在机器人模仿学习中常常包含错误和纠正行为，例如不精确的抓取、物体错位、不稳定接触和重复尝试。虽然这些片段通常被视为噪声或次优数据，但它们提供了关于执行何时偏离理想路径以及如何恢复任务可行性的宝贵证据。然而，现有的奖励和价值模型往往依赖于单调进展假设，这种假设捕捉了粗略的任务进展，但可能忽视了不完美示范中的局部执行错误和纠正行为。在本研究中，我们提出了ReTVL（重试监督价值学习），这是一个通过利用重试事件作为稀疏监督，从混合质量的机器人示范中学习对错误敏感的价值函数的框架。ReTVL通过结合全局进展校准与由稀疏标注的重试关键点引发的局部成对偏好学习，捕捉了围绕错误的局部退化和恢复结构。然后，学习到的价值模型用于对示范片段进行重新加权，以便在后续的行为克隆中减少有害执行错误的影响，同时保留有用的纠正行为。在真实机器人操作任务上的实验表明，ReTVL产生的价值估计比基于进展的基线更为细致，并改善了从不完美示范中进行的模仿学习。

View on arXiv Download PDF AI Translation

cs.RO / 29 / 2606.24712

TACTFUL: Tactile-Driven Exploration For Object Localization and Identification in Confined Environments

TACTFUL：基于触觉驱动的物体定位与识别在受限环境中的探索

Kamtikar, Shivani, Kim, Chung Hee, Tabasso, Camilla, Brady, Tye, Migdal, Joshua, Padir, Taskin

Abstract

Humans effortlessly locate and identify objects by touch alone, even without vision. In contrast, robotic systems rely heavily on vision and struggle with autonomous tactile exploration and object identification. We present TACTFUL, a vision-free tactile exploration framework that enables a multi-fingered robot to autonomously explore confined workspaces, discover objects through contact, and identify them via tactile reconstruction. Trained entirely on real hardware without simulation, our system learns a single policy that balances global workspace exploration with local surface refinement through a dynamic reward schedule. Our results demonstrate that tactile sensing, when paired with structured learning, can serve as an effective primary modality for object-level reasoning, achieving 77% success with 0.015 m average reconstruction error and outperforming baseline approaches on real-world objects.

Chinese Translation

人类仅凭触觉就能轻松定位和识别物体，即使在没有视觉的情况下。相比之下，机器人系统在自主触觉探索和物体识别方面依赖于视觉，面临较大挑战。我们提出了TACTFUL，一个无视觉的触觉探索框架，使多指机器人能够自主探索受限工作空间，通过接触发现物体，并通过触觉重建进行识别。我们的系统完全在真实硬件上训练，而没有使用模拟，学习到了一种平衡全局工作空间探索与局部表面细化的单一策略，采用动态奖励机制。我们的结果表明，当触觉感知与结构化学习相结合时，可以作为物体级推理的有效主要方式，成功率达到77%，平均重建误差为0.015米，并在真实物体上优于基线方法。

View on arXiv Download PDF AI Translation

cs.RO / 30 / 2606.24742

World Value Models for Robotic Manipulation

机器人操作的世界价值模型

Wang, Zhihao, Li, Jianxiong, Cui, Yu, Gao, Yuan, Zhan, Xianyuan, Yu, Junzhi, Ma, Xiao

Abstract

Generalist value models play a pivotal role in scaling robotic policy learning from large-scale, mixed-quality data. Mathematically, accurate value estimation demands deep temporal understanding, requiring models to both ground the current belief using historical context and plan over future outcomes. However, most existing robotic value models are built on Vision-Language Model (VLM) backbones that are pretrained primarily on static or temporally sparse visual observations, lacking the requisite temporal modeling capabilities for value estimation. Unlike VLMs, world models naturally excel at temporal modeling and future planning, making them ideal foundations for learning generalizable value functions. Driven by this insight, we marry world models with value estimation to construct a new generalist robotic value model, World Value Model (WVM), that offers accurate task progressions to assess data quality. On standard benchmarks, WVM delivers state-of-the-art (SOTA) Value-Order Correlation (VOC) results. Complementing standard evaluation suites that contains only expert data, we further introduce Suboptimal-Value-Bench, a multi-embodiment benchmark consisting of 800 suboptimal trajectories with high-fidelity, human-labeled frame annotations. Our evaluations show that WVM maintains its SOTA performance on Suboptimal-Value-Bench, establishing its robustness in handling both expert and suboptimal data. When deployed for policy learning, WVM improves manipulation performance across various policy extraction approaches in both simulated and real-world deployment, providing robust guidance for learning from mixed-quality data.

Chinese Translation

通用价值模型在从大规模、混合质量数据中扩展机器人策略学习方面发挥着关键作用。从数学上讲，准确的价值估计需要深刻的时间理解，这要求模型既要利用历史上下文来建立当前信念，又要对未来结果进行规划。然而，现有的大多数机器人价值模型都是基于视觉-语言模型（Vision-Language Model, VLM）构建的，这些模型主要是在静态或时间稀疏的视觉观测上进行预训练，缺乏进行价值估计所需的时间建模能力。与VLM不同，世界模型在时间建模和未来规划方面自然表现出色，使其成为学习可泛化价值函数的理想基础。基于这一洞察，我们将世界模型与价值估计结合，构建了一个新的通用机器人价值模型——世界价值模型（World Value Model, WVM），该模型提供准确的任务进展以评估数据质量。在标准基准测试中，WVM实现了先进的（SOTA）价值顺序相关性（Value-Order Correlation, VOC）结果。为了补充仅包含专家数据的标准评估套件，我们进一步引入了次优价值基准（Suboptimal-Value-Bench），这是一个由800条次优轨迹组成的多体现基准，具有高保真度的人类标注帧注释。我们的评估表明，WVM在次优价值基准上保持其SOTA性能，证明了其在处理专家和次优数据方面的鲁棒性。在策略学习中部署时，WVM在模拟和现实世界部署中改善了各种策略提取方法的操作性能，为从混合质量数据中学习提供了稳健的指导。

View on arXiv Download PDF AI Translation

cs.RO / 31 / 2606.24814

Vision-Language Model Reasoning for Contextual Semantic Mapping in Intralogistics

面向内部物流的上下文语义映射的视觉-语言模型推理

Rüdt, Marvin, Pang, Hao, Enke, Constantin, Seibold, Zäzilia, Furmans, Kai

Abstract

Autonomous mobile robots operating in intralogistics environments rely on geometric maps for localization and navigation, but lack semantic understanding of objects and their contextual properties. We present a contextual semantic mapping pipeline that combines SLAM-based geometric mapping, SAM-based instance segmentation, instance clustering, and VLM multi-view reasoning to produce a contextual semantic map representation encoding geometric structure, object class, and object movability. By aggregating observations across multiple viewpoints and querying a VLM in a zero-shot, open-vocabulary setting, the pipeline infers contextual object properties--here demonstrated through movability--without requiring task-specific training or predefined object categories. We evaluate three VLMs under two prompting strategies and conduct a component-wise analysis of the pipeline. The proposed pipeline achieves 98.93 % mIoU for semantic classification and 89.17 % mAcc for object movability estimation. Component analysis identifies VLM reasoning as the primary bottleneck for contextual understanding and instance clustering as the main limitation for panoptic performance. The resulting semantic map supports context-aware filtering and robust navigation in dynamic intralogistics environments.

Chinese Translation

在内部物流环境中运行的自主移动机器人依赖几何地图进行定位和导航，但缺乏对物体及其上下文属性的语义理解。我们提出了一种上下文语义映射管道，该管道结合了基于SLAM的几何映射、基于SAM的实例分割、实例聚类和VLM（视觉-语言模型）多视角推理，以生成编码几何结构、物体类别和物体可移动性的上下文语义地图表示。通过聚合多个视角的观察结果，并在零-shot、开放词汇的设置中查询VLM，该管道推断上下文物体属性——这里通过可移动性进行演示——而无需特定任务的训练或预定义的物体类别。我们在两种提示策略下评估了三种VLM，并对管道进行了逐组件分析。所提出的管道在语义分类上达到了98.93%的mIoU，在物体可移动性估计上达到了89.17%的mAcc。组件分析表明，VLM推理是上下文理解的主要瓶颈，而实例聚类是全景性能的主要限制。生成的语义地图支持上下文感知的过滤和在动态内部物流环境中的稳健导航。

View on arXiv Download PDF AI Translation

cs.RO / 32 / 2606.24884

InSight: Self-Guided Skill Acquisition via Steerable VLAs

InSight：通过可调节的视觉-语言-动作（VLA）实现自我引导的技能获取

Wang, Maggie, Osterberg, Lars, Tian, Stephen, Shorinwa, Ola, Wu, Jiajun, Schwager, Mac

Abstract

Vision-language-action (VLA) models can learn manipulation skills from demonstrations, but their capabilities are bounded by the skills in the training data. We present InSight, a framework that unlocks autonomous skill acquisition by rendering VLAs steerable at the primitive-action level (e.g., "move gripper to the bowl", "lift upward", "pour the bottle"). InSight consists of two primary stages: (1) an automated segmentation pipeline that partitions demonstrations into labeled primitives via VLM plan decomposition and end-effector poses to enable VLA primitive steerability, and (2) a VLM-guided data flywheel that identifies missing primitives required to accomplish a novel task, autonomously attempts demonstrations of the missing primitives with VLM-proposed low-level control, and automatically labels, stores, and integrates successful demonstrations into the VLA training set. We evaluate InSight across simulation and real-world manipulation tasks, including block flipping, drawer closing, sweeping, twisting, and pouring, without any human demonstrations of these target skills. Once learned, these primitives can be composed to execute novel, long-horizon tasks without additional human demonstrations. Our findings demonstrate that primitive steerability provides a practical foundation for continual skill acquisition in VLA policies. Project website: https://insight-vla.github.io.

Chinese Translation

视觉-语言-动作（VLA）模型能够从演示中学习操作技能，但其能力受到训练数据中技能的限制。我们提出了InSight，一个通过在原始动作层面（例如，“将夹具移动到碗中”、“向上提起”、“倒出瓶子”）使VLA可调节，从而解锁自主技能获取的框架。InSight由两个主要阶段组成：（1）一个自动分割管道，通过VLM（视觉语言模型）计划分解和末端执行器姿态将演示分割为标记的原始动作，以实现VLA的原始动作可调节性；（2）一个VLM引导的数据飞轮，识别完成新任务所需的缺失原始动作，自动尝试使用VLM提出的低级控制进行缺失原始动作的演示，并自动标记、存储和将成功的演示整合到VLA训练集中。我们在模拟和现实世界的操作任务中评估了InSight，包括翻转积木、关闭抽屉、扫地、扭转和倒水，而无需这些目标技能的任何人类演示。一旦学习到，这些原始动作可以组合执行新颖的长时间任务，而无需额外的人类演示。我们的研究结果表明，原始动作的可调节性为VLA策略中的持续技能获取提供了实用的基础。项目网站：https://insight-vla.github.io。

View on arXiv Download PDF AI Translation

计算机视觉 (Computer Vision)

114

cs.CV / 1 / 2606.23699

A Geometry-Informed Computer Vision Method for Detecting and Examining Overtaking Vehicles From A Bicycle

一种基于几何信息的计算机视觉方法，用于检测和分析自行车上的超车车辆

Padmanaban, Gandhimathi, Moustafa, Rayane, Feng, Fred

Abstract

Instrumented bicycle studies have produced direct field evidence on vehicle passing behavior, but extracting overtaking events from continuous rear-facing video has remained dependent on manual, frame-by-frame annotation. This bottleneck constrains sample sizes and limits naturalistic cycling safety research. We present a geometry-informed computer vision pipeline that automates overtaking event detection from a single bicycle-mounted camera without multi-sensor configurations or explicit camera calibration. The system combines RT-DETR object detection with ByteTrack multi-object tracking through a three-stage geometric validation module enforcing bearing angle trend, apparent size growth, and spatial confirmation criteria derived from perspective projection principles. Validated on 315 manually annotated real-world overtaking events from urban roads in Ann Arbor, Michigan, the pipeline achieved 97.8% recall with zero false positives. The system identified overtaking intentions a mean of 2.44 seconds before vehicle passage, with 84.1% of events exceeding the 1.5-second human reaction time threshold, demonstrating feasibility for active cyclist warning. Lateral passing distance measurements from 96 events revealed 33.3% of passes below the 5-foot (152.4 cm) threshold, consistent with non-compliance rates in prior field and self-reported studies. A preliminary calibration-free lateral distance estimation approach using bounding box geometric features achieved mean absolute errors of 13-14 cm under leave-one-out cross-validation, sufficient to distinguish close passes from standard passes for safety categorization. By automating event isolation from consumer-grade footage, the system removes the primary annotation bottleneck of instrumented bicycle research and provides a scalable foundation for vehicle-bicycle interaction analysis across larger datasets and diverse urban environments.

Chinese Translation

仪器化自行车研究提供了关于车辆超车行为的直接实地证据，但从连续的后向视频中提取超车事件仍然依赖于手动逐帧注释。这一瓶颈限制了样本量，并限制了自然骑行安全研究。我们提出了一种基于几何信息的计算机视觉管道，能够从单个自行车安装的摄像头自动检测超车事件，而无需多传感器配置或显式的摄像头校准。该系统结合了 RT-DETR 物体检测与 ByteTrack 多物体跟踪，通过三阶段几何验证模块强制执行基于透视投影原理的方位角趋势、明显尺寸增长和空间确认标准。该管道在密歇根州安娜堡的城市道路上验证了 315 个手动注释的真实超车事件，达到了 97.8% 的召回率且没有假阳性。系统在车辆通过前平均识别出超车意图的时间为 2.44 秒，其中 84.1% 的事件超过了 1.5 秒的人类反应时间阈值，证明了主动警告骑行者的可行性。从 96 个事件的横向通过距离测量中发现，33.3% 的超车距离低于 5 英尺（152.4 厘米）阈值，这与先前的实地和自我报告研究中的不合规率一致。使用边界框几何特征的初步无校准横向距离估计方法在留一交叉验证下达到了 13-14 厘米的平均绝对误差，足以区分安全分类中的近距离超车和标准超车。通过从消费级视频中自动隔离事件，该系统消除了仪器化自行车研究的主要注释瓶颈，并为在更大数据集和多样化城市环境中进行车辆与自行车交互分析提供了可扩展的基础。

View on arXiv Download PDF AI Translation

cs.CV / 2 / 2606.23743

Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation

Sol 视频推理引擎：高效视频生成的代理原生全栈加速框架

Li, Yitong, Chen, Junsong, Li, Haopeng, Liu, Haozhe, Yu, Jincheng, Zhu, Ligeng, Luo, Ping, Han, Song, Xie, Enze

Abstract

Modern video diffusion models achieve higher generation quality through scaling, but this also increases inference cost. Although many acceleration methods have been proposed, a central challenge is that the most effective acceleration strategy is highly instance-specific: a recipe that works well for one combination of model, hardware, and inference configuration often does not transfer to another. Different models vary in architecture, numerical sensitivity, and attention concentration patterns. Inference settings differ in spatial and temporal resolution and video duration, while hardware platforms differ in memory hierarchy, supported numerical formats, and kernel throughput. These factors create a large tuning space, making manual performance engineering costly. We present Sol Video Inference Engine, an agentic, native, training-free acceleration framework for video diffusion models. It organizes five broadly applicable techniques, cache, sparse attention, token pruning, quantization, and kernel fusion, into an agentic acceleration stack for instance-specific optimization. For a concrete deployment target defined by a model, hardware platform, and serving configuration, parallel skill agents optimize the implementation of each technique, an agent integrator composes them into a global acceleration stack, and a human validator provides feedback on generation quality. We instantiate this workflow on three video models with different sizes and architectures: 64B Cosmos3-Super, 22B LTX-2.3, and 2B SANA-Video. With little human effort, the full stack achieves more than 2x end-to-end acceleration while maintaining near-lossless VBench quality, demonstrating the effectiveness of the agent framework for video diffusion acceleration.

Chinese Translation

现代视频扩散模型通过扩展实现了更高的生成质量，但这也增加了推理成本。尽管提出了许多加速方法，但一个核心挑战是，最有效的加速策略高度依赖于具体实例：对于某一特定模型、硬件和推理配置的组合有效的方案，往往无法转移到另一个组合。不同模型在架构、数值敏感性和注意力集中模式上存在差异。推理设置在空间和时间分辨率以及视频时长上有所不同，而硬件平台在内存层次结构、支持的数值格式和内核吞吐量上也各不相同。这些因素造成了一个庞大的调优空间，使得手动性能工程成本高昂。我们提出了 Sol 视频推理引擎，这是一个代理原生、无训练的加速框架，旨在优化视频扩散模型。它将五种广泛适用的技术——缓存、稀疏注意力、标记修剪、量化和内核融合——组织成一个用于实例特定优化的代理加速栈。对于由模型、硬件平台和服务配置定义的具体部署目标，多个并行技能代理优化每种技术的实现，一个代理整合器将它们组合成一个全局加速栈，而一个人工验证者则对生成质量提供反馈。我们在三种不同规模和架构的视频模型上实例化了这一工作流程：64B Cosmos3-Super、22B LTX-2.3 和 2B SANA-Video。在几乎不需要人工干预的情况下，整个栈实现了超过 2 倍的端到端加速，同时保持近无损的 VBench 质量，展示了代理框架在视频扩散加速中的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 3 / 2606.23763

Listening makes Vision Clear for VLMs

倾听使视觉在视觉语言模型中更加清晰

Chen, Yiyang, Tan, Yixin, Shen, Binrui

Abstract

Recent work typically assesses vision--language consistency using attention distributions of answer-side tokens. However, we observe that highest attention regions are not always consistent with the intended semantic token. This probably stems from decoding drift, where language priors from previously generated answer tokens accumulate and mismatch with visual attention. Besides the priors from previous answer tokens, we find that structural tokens, e.g., modality boundary markers, may encompass the entire context and generate high attention to areas unrelated to the target. To avoid these distortions and provide consistency evaluation for large VLMs, we adopt prompt-side semantics and propose Prompt-Vision Token Activation Map (PV-TAM). PV-TAM further incorporates a filter to remove systematic bias induced by modality boundary markers. Unlike traditional methods that evaluate overlap solely through masks while ignoring activation intensity, our metrics leverage the peak distribution of attention to measure the alignment between prompts and visual regions. In experiments, PV-TAM consistently improves both attention-based and IoU-style localization metrics over answer-side baselines on various datasets.

Chinese Translation

近期的研究通常通过答案侧标记的注意力分布来评估视觉与语言的一致性。然而，我们观察到最高的注意力区域并不总是与预期的语义标记一致。这可能源于解码漂移，即之前生成的答案标记的语言先验累积并与视觉注意力不匹配。除了来自先前答案标记的先验外，我们发现结构性标记，例如模态边界标记，可能涵盖整个上下文，并对与目标无关的区域产生高注意力。为了避免这些扭曲并为大型视觉语言模型提供一致性评估，我们采用提示侧语义并提出了提示-视觉标记激活图（Prompt-Vision Token Activation Map，PV-TAM）。PV-TAM进一步结合了一个过滤器，以去除由模态边界标记引起的系统性偏差。与传统方法仅通过掩码评估重叠而忽略激活强度不同，我们的指标利用注意力的峰值分布来测量提示与视觉区域之间的对齐。在实验中，PV-TAM在各种数据集上始终改善了基于注意力和IoU风格的定位指标，相较于答案侧基线。

View on arXiv Download PDF AI Translation

cs.CV / 4 / 2606.23825

From Spatial to Spectral: An Efficient, Frequency-Guided Feature Representation Learner for Small Object Detection

从空间到频谱：一种高效的频率引导特征表示学习器用于小物体检测

Rui, Yuhan, Qiao, Shihan, Lou, Yibin, Yu, Mingxi, Wan, Yutong, Chen, Yanqiao, Hou, Dongsheng, Cao, Zhen, Zhong, Athena Zhuoming, Hao, Qi

Abstract

Efficient small object detection is bottlenecked by the inherent feature scarcity of tiny targets, which is further aggravated by operations of spatial-domain detectors that indiscriminately discard critical high-frequency details. Recovering these fragile cues within the spatial domain is notoriously difficult, as it often requires computationally expensive architectural upscaling that inadvertently amplifies background noise. To bridge this gap, we propose a paradigm \textbf{shift from spatial to spectral} feature processing, introducing a holistic solution with the following novelty: (1) A versatile \textbf{Frequency-Guided Feature Representation framework} that generalizes across diverse detector architectures (both CNN and Transformer-based), offering a robust alternative to spatial-only feature extraction; (2) The unified \textbf{Decompose--Enhance--Reconstruct (DER)} operator, instantiated via three \textbf{lightweight, plug-and-play} modules -- Wavelet-Difference Gate (WDG), Log-Gabor Enhancer (LGE), and Frequency-Driven Head (FDHead) -- to systematically inject frequency-aware modulation into the backbone, neck, and head. This mechanism decouples feature modeling from resolution reduction, capturing discriminative high-frequency components to enable accurate localization with significantly reduced parameter redundancy; (3) Extensive validation on multi-domain benchmarks (VisDrone2019, UAVDT, TinyPerson, DOTAv1) demonstrating consistent gains. Notably, our proposed \textbf{DERNet} series outperforms YOLOv11 models under the same scale while requiring \textbf{only 1/6 of the parameters}, backed by rigorous spectral diagnostics and error decomposition analysis.

Chinese Translation

高效的小物体检测受到微小目标固有特征稀缺的制约，而空间域检测器的操作又无差别地丢弃了关键的高频细节，进一步加剧了这一问题。在空间域中恢复这些脆弱线索是 notoriously 困难的，因为这通常需要计算成本高昂的架构放大，反而会放大背景噪声。为了解决这一问题，我们提出了一种从空间到频谱的特征处理范式转变，提出了一种整体解决方案，具有以下创新点：（1）一种通用的频率引导特征表示框架（Frequency-Guided Feature Representation），能够在多种检测器架构（包括 CNN 和基于 Transformer 的架构）中泛化，提供了一种稳健的替代方案，超越了仅基于空间的特征提取；（2）统一的分解-增强-重构（Decompose--Enhance--Reconstruct, DER）算子，通过三个轻量级、即插即用的模块实现——小波差分门（Wavelet-Difference Gate, WDG）、对数-伽波增强器（Log-Gabor Enhancer, LGE）和频率驱动头（Frequency-Driven Head, FDHead），系统性地将频率感知调制注入到骨干网络、颈部和头部。该机制将特征建模与分辨率降低解耦，捕获判别性的高频成分，以实现准确的定位，同时显著减少参数冗余；（3）在多领域基准（VisDrone2019、UAVDT、TinyPerson、DOTAv1）上进行了广泛验证，展示了一致的性能提升。值得注意的是，我们提出的 DERNet 系列在相同规模下优于 YOLOv11 模型，同时仅需 1/6 的参数，得到了严格的频谱诊断和误差分解分析的支持。

View on arXiv Download PDF AI Translation

cs.CV / 5 / 2606.23835

ABACUS: Adapting Unified Foundation Model for Bridging Image Count Understanding and Generation

ABACUS：适应统一基础模型以桥接图像计数理解与生成

Mondal, Anindya, Nag, Sauradip, Dutta, Anjan

Abstract

ABACUS is a unified vision-language model that handles object counting, crowd counting, referring-expression counting, and count-faithful image generation without any benchmark-specific training required. Our model is built on existing 3B-parameter unified foundation model and is adapted for object localization tasks using three key innovations: density-aware adaptive zooming with objectness maps for spatial grounding; a boundary-aware count policy via GRPO to eliminate crop-boundary errors; and a cycle-consistent GRPO strategy where the understanding branch self-critiques generated outputs, closing the understanding-generation gap without any external annotations. ABACUS achieves state-of-the-art results across seven benchmarks, outperforming both task-specific specialists and larger generalist models.

Chinese Translation

ABACUS 是一个统一的视觉-语言模型，能够处理物体计数、拥挤计数、指称表达计数以及忠实计数的图像生成，无需任何特定基准的训练。我们的模型基于现有的 3B 参数统一基础模型，并通过三项关键创新适应物体定位任务：使用物体性图（objectness maps）进行空间定位的密度感知自适应缩放；通过 GRPO 实现的边界感知计数策略，以消除裁剪边界错误；以及循环一致的 GRPO 策略，其中理解分支对生成的输出进行自我批评，缩小理解与生成之间的差距，而无需任何外部注释。ABACUS 在七个基准测试中取得了最先进的结果，超越了任务特定专家模型和更大的通用模型。

View on arXiv Download PDF AI Translation

cs.CV / 6 / 2606.23843

HANCLIP: A Family of Hyperbolic Angular Negation Vision Language Models

HANCLIP：一种超曲率角度否定视觉语言模型家族

Le, Hoang-Bao, Durrant, Aiden, Mai, Thai Son, Nguyen, Binh T., Zhou, Liting, Gurrin, Cathal

Abstract

Vision-Language Models (VLMs) are typically pre-trained on large-scale image-text datasets to capture semantic correspondences between visual content and natural language. However, they remain surprisingly brittle to negation: models often rely on shallow word co-occurrence and are easily distracted by misleading or irrelevant textual cues, even when their overall retrieval or classification performance is strong. Moreover, directly finetuning on negation data can interfere with previously acquired knowledge, causing noticeable degradation on standard vision-language benchmarks. To tackle these issues, this work introduces HANCLIP (Hyperbolic + Angular + Negation), a family of VLMs that explicitly restructures the embedding space to encode "what an image is not" alongside "what it is." HANCLIP is trained on a compact set of 20,000 image-text quadruplets and combines a hyperbolic formulation, which models hierarchical semantic relations and asymmetries, with an angular triplet objective that drives systematic separation between negated descriptions and their corresponding positives. This geometry-aware design strengthens negation sensitivity while preserving the global structure of pretrained representations, rather than overwriting them. Extensive experiments across multiple vision-language tasks show that HANCLIP delivers consistent gains on the negation-focused NegBench benchmark, while maintaining competitive or improved performance on standard classification and image-text retrieval benchmarks. The framework is model-agnostic and can be plugged into CLIP, LongCLIP, SmartCLIP, and HiMo-CLIP without large-scale retraining, demonstrating that a carefully designed geometric objective can substantially extend the reasoning capabilities of existing VLMs using only modest additional data.

Chinese Translation

视觉语言模型（VLMs）通常在大规模图像-文本数据集上进行预训练，以捕捉视觉内容与自然语言之间的语义对应关系。然而，它们对否定的反应却出乎意料地脆弱：模型往往依赖于浅层的词共现，并且容易受到误导性或无关文本线索的干扰，即使它们的整体检索或分类性能较强。此外，直接在否定数据上进行微调可能会干扰先前获得的知识，导致在标准视觉-语言基准测试上的显著退化。为了解决这些问题，本研究提出了HANCLIP（超曲率 + 角度 + 否定），这是一个明确重构嵌入空间的VLM家族，旨在同时编码“图像不是怎样的”和“图像是怎样的”。HANCLIP在一个紧凑的20,000个图像-文本四元组的数据集上进行训练，结合了超曲率形式，该形式建模层次语义关系和不对称性，以及一个角度三元组目标，推动否定描述与其对应正面描述之间的系统性分离。这种几何感知设计增强了对否定的敏感性，同时保持了预训练表示的全局结构，而不是覆盖它们。针对多个视觉-语言任务的广泛实验表明，HANCLIP在以否定为重点的NegBench基准上提供了一致的提升，同时在标准分类和图像-文本检索基准上保持竞争力或改善性能。该框架是模型无关的，可以无须大规模重训练地集成到CLIP、LongCLIP、SmartCLIP和HiMo-CLIP中，证明了精心设计的几何目标可以在仅使用适度额外数据的情况下显著扩展现有VLM的推理能力。

View on arXiv Download PDF AI Translation

cs.CV / 7 / 2606.23885

Mind the Heads: Topological Representation Alignment for Multimodal LLMs

关注头部：多模态大语言模型的拓扑表示对齐

Caffagni, Davide, Compagnoni, Alberto, Melis, Federico, Sarto, Sara, Dovesi, Pier Luigi, Granroth-Wilding, Mark, Cornia, Marcella, Baraldi, Lorenzo

Abstract

Representation alignment has emerged as an effective approach to improve Multimodal Large Language Models (MLLMs) by regularizing their internal representations toward those of an external vision encoder. However, existing methods typically align a fixed layer of the language backbone, overlooking the fine-grained structure of Transformer models. In this work, we propose Head-Wise Representation Alignment (HeRA), a method that enforces cross-modal alignment at the level of individual attention heads. Our approach is grounded in the Platonic Representation Hypothesis, focusing on preserving the topological structure of representations (i.e., their local neighborhood relationships) across modalities. Following the Mutual K-Nearest Neighbor (MKNN) alignment metric, we introduce a contrastive objective that acts as a differentiable proxy for matching local structures. HeRA applies this objective during multimodal training to specific attention heads in the LLM, selected by their alignment score according to the MKNN metric. Counterintuitively, we find that aligning the least aligned heads yields the largest gains. Extensive evaluations across multiple MLLMs and 18 benchmarks demonstrate that HeRA consistently improves performance on challenging vision-centric tasks and serves as an effective regularizer against visual hallucinations by naturally curbing the over-reliance on linguistic priors. Our code is publicly released.

Chinese Translation

表示对齐已成为一种有效的方法，通过将多模态大语言模型（MLLMs）的内部表示规范化为外部视觉编码器的表示，从而提升其性能。然而，现有方法通常对语言主干的固定层进行对齐，忽视了Transformer模型的细粒度结构。在本研究中，我们提出了头部级表示对齐（Head-Wise Representation Alignment, HeRA）方法，该方法在单个注意力头的层面上强制进行跨模态对齐。我们的方法基于柏拉图表示假设，专注于保留跨模态表示的拓扑结构（即它们的局部邻域关系）。遵循互惠K近邻（Mutual K-Nearest Neighbor, MKNN）对齐度量，我们引入了一种对比目标，作为匹配局部结构的可微代理。HeRA在多模态训练期间将该目标应用于根据MKNN度量选择的特定注意力头。反直觉的是，我们发现对齐最不对齐的头部会带来最大的收益。对多个MLLM和18个基准的广泛评估表明，HeRA在具有挑战性的视觉中心任务上始终提高性能，并通过自然抑制对语言先验的过度依赖，作为有效的正则化器来对抗视觉幻觉。我们的代码已公开发布。

View on arXiv Download PDF AI Translation

cs.CV / 8 / 2606.23892

REALM: A Unified Red-Teaming Benchmark for Physical-World VLMs

REALM：物理世界视觉语言模型的统一红队基准

Zhao, Yifei, Lou, Qian, Zheng, Mengxin

Abstract

Vision-language models (VLMs) are increasingly used as perception-reasoning backbones for embodied intelligence in safety-critical physical systems, where perception or reasoning errors can lead to unsafe decisions or actions. Although many red-teaming methods have been developed to probe VLM vulnerabilities, their evaluation remains fragmented across datasets, metrics, and threat models, making direct comparison difficult and obscuring whether observed differences arise from stronger attacks, more vulnerable models, or incompatible evaluation settings. Existing chatbot-centric red-teaming benchmarks mainly standardize jailbreak and content-safety evaluation, but they do not systematically capture physically grounded functional failures or cover red-teaming methods that target physical-world VLMs. This raises the key challenge of comparing diverse attack methods under a unified protocol while targeting the same scenario-specific failures. We introduce REALM, to our knowledge the first unified red-teaming benchmark for physical-world VLMs. REALM integrates 12 red-teaming methods, 3 model-agnostic defenses, and 13 VLMs under a practical black-box threat model with shared datasets and metrics. To align adversarial objectives across attack families, REALM introduces an agentic target-generation pipeline that constructs shared, scenario-specific, and physically grounded attack objectives for each scene, enabling fair comparison of diverse red-teaming methods under aligned adversarial goals. Our evaluation shows that text and typographic injection attacks induce the most failures, multimodal co-optimization yields the strongest visual-perturbation transfer, single-pass attacks approach iterative methods at much lower cost, and model scale alone does not confer adversarial robustness. Code is available at https://github.com/UCF-ML-Research/REALM.

Chinese Translation

视觉语言模型（VLMs）越来越多地被用作安全关键物理系统中具身智能的感知-推理基础，其中感知或推理错误可能导致不安全的决策或行动。尽管已经开发了许多红队方法来探测VLM的脆弱性，但它们的评估在数据集、指标和威胁模型之间仍然分散，导致直接比较变得困难，并模糊了观察到的差异是由于更强的攻击、更脆弱的模型，还是不兼容的评估设置。现有的以聊天机器人为中心的红队基准主要标准化了越狱和内容安全评估，但并未系统性地捕捉物理基础的功能失效或涵盖针对物理世界VLM的红队方法。这提出了在统一协议下比较多样化攻击方法的关键挑战，同时针对相同场景特定的失效。我们介绍了REALM，据我们所知，这是第一个针对物理世界VLM的统一红队基准。REALM整合了12种红队方法、3种模型无关的防御和13个VLM，在一个实际的黑箱威胁模型下共享数据集和指标。为了在攻击家族之间对齐对抗目标，REALM引入了一种代理目标生成管道，为每个场景构建共享的、特定于场景的、物理基础的攻击目标，从而使不同红队方法在对齐的对抗目标下进行公平比较。我们的评估显示，文本和排版注入攻击引发了最多的失败，多模态共同优化产生了最强的视觉扰动转移，单次攻击在成本上接近迭代方法，而模型规模本身并不赋予对抗鲁棒性。代码可在 https://github.com/UCF-ML-Research/REALM 获取。

View on arXiv Download PDF AI Translation

cs.CV / 9 / 2606.23897

The Professor: Multi-Teacher Unsupervised Prompt Distillation for Vision-Language Models

教授：用于视觉-语言模型的多教师无监督提示蒸馏

Algadhi, Ahmad, Alzuhair, Ahmed, Alkhulaif, Omar, Behzad, Muzammil

Abstract

Prompt distillation compresses large vision-language models (VLMs) such as CLIP into lightweight student models by matching teacher predictions on unlabeled domain images. PromptKD (CVPR 2024) established this paradigm with a single PromptSRC-finetuned ViT-L/14 teacher and a ViT-B/16 student. We propose TheProfessor, a multi-teacher extension that distills from a fixed two-teacher ensemble: a domain-finetuned PromptSRC ViT-L/14 teacher and a zero-shot EVA-CLIP-L/14 teacher whose logits are pre-computed per dataset. We evaluate single-teacher PromptKD, equal-probability ensembling, and confidence-weighted ensembling on four base-to-novel datasets: Caltech-101, DTD, UCF101, and EuroSAT. In a 12-run single-seed sweep, confidence-weighted ensembling improves average HM from 87.52 to 89.28 (+1.77 points), while equal averaging improves average HM to 88.88 (+1.37 points). Gains are dataset dependent: they are negligible on Caltech-101 (+0.16 HM for confidence weighting), modest on UCF101 (+0.62), and largest on domain-shifted EuroSAT (+5.78). These results update our earlier Caltech-only analysis and show that multi-teacher prompt distillation is most useful when the second teacher contributes complementary supervision under domain shift.

Chinese Translation

提示蒸馏通过在未标记的领域图像上匹配教师预测，将大型视觉-语言模型（VLMs）如 CLIP 压缩为轻量级学生模型。PromptKD（CVPR 2024）以单一的经过 PromptSRC 微调的 ViT-L/14 教师和 ViT-B/16 学生建立了这一范式。我们提出了 TheProfessor，一个多教师扩展，采用固定的双教师集成进行蒸馏：一个领域微调的 PromptSRC ViT-L/14 教师和一个零样本的 EVA-CLIP-L/14 教师，其 logits 在每个数据集上预先计算。我们在四个基础到新颖的数据集上评估了单教师 PromptKD、等概率集成和置信加权集成：Caltech-101、DTD、UCF101 和 EuroSAT。在一次 12 次运行的单种子实验中，置信加权集成将平均 HM 从 87.52 提升至 89.28 (+1.77 分)，而等均值提升平均 HM 至 88.88 (+1.37 分)。增益依赖于数据集：在 Caltech-101 上几乎可以忽略不计 (+0.16 HM 适用于置信加权)，在 UCF101 上适中 (+0.62)，而在领域转移的 EuroSAT 上最大 (+5.78)。这些结果更新了我们之前仅针对 Caltech 的分析，并表明多教师提示蒸馏在第二位教师在领域转移下提供互补监督时最为有效。

View on arXiv Download PDF AI Translation

cs.CV / 10 / 2606.23917

Trustworthy Image Authentication using Forensic Knowledge Graphs

基于法医学知识图谱的可信图像认证

Nguyen, Tai D., Stamm, Matthew C.

Abstract

Advances in generative AI have made image falsification highly realistic, demanding trustworthy authentication systems. Existing forensic detectors can target certain forgery types but lack interpretability, while vision-language models (VLMs) provide explanations but cannot exploit forensic traces for reliable detection. We propose Forensic Knowledge Graphs (FKGs), a unified framework that integrates forensic evidence extraction, structured reasoning, and human-interpretable explanation. Our FKG structure encodes forensic traces along with their causal dependencies and links to scene content. To generate accurate FKGs, we introduce a novel forensic authentication network and an Iterative Context Refinement strategy that guides VLMs to produce faithful, grounded explanations. We also present FKG-50K, a dataset of 50,000 realistic forgeries with ground-truth FKGs. Experiments demonstrate that FKG outperforms both forensic detectors and VLMs in detection, forgery identification and localization, and forensic justification.

Chinese Translation

生成性人工智能的进步使得图像伪造变得高度真实，迫切需要可信的认证系统。现有的法医检测器可以针对特定类型的伪造进行检测，但缺乏可解释性，而视觉-语言模型（VLMs）提供了解释，但无法利用法医痕迹进行可靠检测。我们提出了法医学知识图谱（FKGs），这是一个统一框架，集成了法医证据提取、结构化推理和人类可解释的解释。我们的FKG结构编码了法医痕迹及其因果依赖关系，并与场景内容相连接。为了生成准确的FKGs，我们引入了一种新颖的法医认证网络和一种迭代上下文精炼策略，以指导VLMs生成真实、扎实的解释。我们还提出了FKG-50K，这是一个包含50,000个真实伪造图像及其真实FKGs的数据集。实验表明，FKG在检测、伪造识别与定位以及法医证明方面优于现有的法医检测器和VLMs。

View on arXiv Download PDF AI Translation

cs.CV / 11 / 2606.23950

DivRL: Disentangled Self-Similarity Rewards for Diverse Subject-Driven Generation

DivRL：用于多样化主题驱动生成的解耦自相似奖励

Wang, Qian, Li, Zhenyu, Eldesokey, Abdelrahman, Wonka, Peter

Abstract

Subject-driven image generation faces an "Identity-Diversity Paradox", where strong identity preservation often leads to rigid and low-diversity outputs. We propose a post-training framework called DivRL that jointly optimizes identity consistency and structural diversity simultaneously by leveraging disentangled visual features from a robust similarity model. Specifically, we introduce a Negative Self-Similarity Measure (nSSM) to quantify structural diversity, and Visual Semantic Matching (VSM) to evaluate identity consistency. We propose an "Explore-and-Suppress" strategy that treats VSM as a gated constraint: the model freely explores structurally diverse configurations, and only samples that violate the identity threshold are penalized via a quadratic hinge loss. This converts identity preservation from a competing objective into a feasibility constraint, allowing nSSM and VSM to improve jointly. Experiments demonstrate that our method effectively pushes the model to generate both consistent and diverse images and improves structural diversity while maintaining comparable identity consistency through a gated optimization formulation.

Chinese Translation

主题驱动的图像生成面临“身份-多样性悖论”，即强身份保留往往导致输出的刚性和低多样性。我们提出了一种名为DivRL的后训练框架，通过利用来自强大相似性模型的解耦视觉特征，联合优化身份一致性和结构多样性。具体而言，我们引入了一种负自相似度量（Negative Self-Similarity Measure, nSSM）来量化结构多样性，以及视觉语义匹配（Visual Semantic Matching, VSM）来评估身份一致性。我们提出了一种“探索与抑制”策略，将VSM视为一个门控约束：模型可以自由探索结构多样的配置，只有违反身份阈值的样本会通过二次铰链损失受到惩罚。这将身份保留从一个竞争目标转变为可行性约束，使nSSM和VSM能够共同提升。实验表明，我们的方法有效推动模型生成既一致又多样的图像，并在保持可比身份一致性的同时提高结构多样性，采用了门控优化的形式。

View on arXiv Download PDF AI Translation

cs.CV / 12 / 2606.24021

Token-to-Token Alignment of Text Embeddings for Semantic Blending

文本嵌入的标记对标记对齐用于语义混合

Huberman, Saar, Mokady, Ron, Patashnik, Or, Cohen-Or, Daniel

Abstract

In modern generative models, images are specified and controlled through text prompts. In practice, images are generated from sequences of tokens derived from these prompts. However, the space of token sequences lacks a consistent accessible structure: semantically similar images may correspond to sequences that differ in wording, ordering, and placement of concepts, while similar token sequences may encode very different semantics. This apparent lack of structure makes it difficult to perform smooth transitions in this space, hindering applications such as image blending and continuous control of edits. We argue that this limitation stems not from the absence of semantic structure, but from misalignment between representations. To address this misalignment, we introduce Token-to-Token alignment, a framework that establishes explicit semantic correspondence between tokens across prompts. Our approach transforms prompts into a structured representation in which semantically corresponding concepts are mapped to consistent positions across prompts, and then aligns their token embeddings based on semantic similarity. Concretely, the method consists of two stages: a structural alignment that rephrases prompts into a shared structured form, followed by an embedding-level alignment that matches token representations across prompts. With this alignment in place, simple linear interpolation becomes a meaningful operation, producing smooth and coherent semantic transitions and enabling applications such as blending and continuous editing. Our results show that text embedding spaces in text-to-image models implicitly encode a continuous semantic structure that becomes accessible once representations are properly aligned, suggesting that semantic control can be achieved by organizing existing representations rather than modifying the generative model.

Chinese Translation

在现代生成模型中，图像是通过文本提示进行指定和控制的。在实践中，图像是从这些提示中派生的标记序列生成的。然而，标记序列的空间缺乏一致的可访问结构：语义相似的图像可能对应于在措辞、顺序和概念位置上有所不同的序列，而相似的标记序列可能编码非常不同的语义。这种明显的结构缺失使得在该空间中进行平滑过渡变得困难，阻碍了图像混合和编辑的连续控制等应用。我们认为，这一限制并非源于缺乏语义结构，而是由于表示之间的错位。为了解决这种错位，我们引入了标记对标记对齐（Token-to-Token alignment），这是一个在提示之间建立标记显式语义对应关系的框架。我们的方法将提示转换为一种结构化表示，其中语义对应的概念被映射到提示之间的一致位置，然后根据语义相似性对其标记嵌入进行对齐。具体而言，该方法由两个阶段组成：一个结构对齐阶段，将提示重新表述为共享的结构形式，随后是一个嵌入级对齐阶段，将提示之间的标记表示进行匹配。在这种对齐机制下，简单的线性插值成为一种有意义的操作，产生平滑且连贯的语义过渡，并启用诸如混合和连续编辑等应用。我们的结果表明，文本到图像模型中的文本嵌入空间隐式编码了一种连续的语义结构，一旦表示得到适当对齐，这种结构便可被访问，这表明语义控制可以通过组织现有表示而非修改生成模型来实现。

View on arXiv Download PDF AI Translation

cs.CV / 13 / 2606.24051

DriveStack-VLA: Render-Teacher Alignment for BEV-Based DeepStack Vision-Language-Action Model

DriveStack-VLA：基于鸟瞰视图的深度堆栈视觉-语言-动作模型的渲染教师对齐

Wang, Jingke, Zhao, Zhenru, Lei, Shuangming, Su, Hao, Huang, Yuehao, Xie, Yijia, Tang, Kai, Xu, Guanglin, Ye, AiXue, Ma, Yukai, Liu, Yong

Abstract

Vision-Language-Action driving models convert a pretrained Vision-Language Model into a driving policy, allowing them to use world knowledge and follow language guidances. However, existing VLA driving models still lack driving-oriented spatial intelligence: their policies are mainly grounded on perspective image tokens and language priors, while precise motion planning requires metric geometry, top-down scene structure, and attention to safety-critical perceptual cues. This limitation makes current models vulnerable to weak visual geometry modeling and perceptual coverage in expert demonstrations. In this paper, we present DriveStack-VLA, a framework built upon a large VLM backbone. To strengthen the spatial grounding of VLA driving, we develop dual visual modeling components. We inject a Bird-Eye-View representation into the Large Language Model decoder through a DeepStack-style connection, and propose Render-Teacher Alignment to align the perceptual focus of real images with that of rasterized images. Furthermore, to bridge the gap in multimodal trajectory selection, we introduce a head-based self-critique module that ranks sampled trajectories and conditionally refines the best one. DriveStack-VLA achieves 91.6 PDMS on NAVSIMv1, 91.0 EPDMS on NAVSIMv2 (with the human penalty filter enabled), and a driving score of 79.49 with a success rate of 56.36\% on the closed-loop Bench2Drive. More visualizations are available on our project page: https://anonymous.4open.science/w/drivestack-vla/.

Chinese Translation

视觉-语言-动作（VLA）驾驶模型将预训练的视觉-语言模型转换为驾驶策略，使其能够利用世界知识并遵循语言指导。然而，现有的VLA驾驶模型仍然缺乏面向驾驶的空间智能：它们的策略主要基于透视图像标记和语言先验，而精确的运动规划则需要度量几何、俯视场景结构以及对安全关键感知线索的关注。这一局限性使得当前模型在专家演示中对弱视觉几何建模和感知覆盖变得脆弱。本文提出了DriveStack-VLA，一个建立在大型视觉-语言模型（VLM）基础上的框架。为了增强VLA驾驶的空间基础，我们开发了双重视觉建模组件。我们通过深度堆栈风格的连接将鸟瞰视图表示注入大型语言模型解码器，并提出了渲染教师对齐（Render-Teacher Alignment），以将真实图像的感知焦点与光栅化图像的感知焦点对齐。此外，为了弥补多模态轨迹选择中的差距，我们引入了基于头部的自我批评模块，该模块对采样轨迹进行排序，并有条件地优化最佳轨迹。DriveStack-VLA在NAVSIMv1上达到了91.6的PDMS，在NAVSIMv2上达到了91.0的EPDMS（启用人类惩罚过滤器），并在闭环Bench2Drive上获得了79.49的驾驶得分和56.36\%的成功率。更多可视化内容请访问我们的项目页面：https://anonymous.4open.science/w/drivestack-vla/。

View on arXiv Download PDF AI Translation

cs.CV / 14 / 2606.24057

EPEdit: Redefining Image Editing with Generative AI and User-Centric Design

EPEdit：通过生成性人工智能和以用户为中心的设计重新定义图像编辑

Nguyen, Hoang-Phuc, Vo, Dinh-Khoi, Do, Trong-Le, Nguyen, Hai-Dang, Nguyen, Tan-Cong, Nguyen, Vinh-Tiep, Nguyen, Tam V., Le, Khanh-Duy, Tran, Minh-Triet, Le, Trung-Nghia

Abstract

The demand for image manipulation has seen a significant increase recently. Traditional tools like Photoshop and Capture One, while powerful, require considerable expertise to use effectively. Generative AI has introduced alternative platforms, such as Luminar Neo, Pixlr X, and Canva. However, many of these solutions, including resource-heavy models like Stable Diffusion, often require substantial retraining and fine-tuning, leading to high costs for users. To address these challenges, we introduce Efficient Photo Editor (EPEdit), an application that integrates a robust backend framework with a user-friendly front-end interface. EPEdit supports a wide range of creative image editing tasks, including image generation, object replacement, object removal, background modification, changes in object pose or perspective, region-specific editing, and thematic collection design, all guided by masks and prompts. Users can interact with the system through simple text commands or by marking areas for precise adjustments, making it accessible even to those without technical expertise. At its core, EPEdit leverages zero-shot image editing algorithms based on Stable Diffusion model, removing the need for additional fine-tuning. This approach enables efficient image manipulation and thematic collection creation. User evaluations for tasks of image editing, thematic design, and overall system performance demonstrate that EPEdit outperforms existing solutions, offering a user-friendly, cost-effective solution for comprehensive image editing.

Chinese Translation

近期，对图像处理的需求显著增加。尽管传统工具如 Photoshop 和 Capture One 功能强大，但有效使用这些工具需要相当的专业知识。生成性人工智能引入了替代平台，如 Luminar Neo、Pixlr X 和 Canva。然而，其中许多解决方案，包括资源密集型模型如 Stable Diffusion，通常需要大量的再训练和微调，导致用户面临高昂的成本。为了解决这些挑战，我们推出了高效照片编辑器（EPEdit），这是一款将强大的后端框架与用户友好的前端界面相结合的应用程序。EPEdit 支持广泛的创意图像编辑任务，包括图像生成、对象替换、对象移除、背景修改、对象姿态或视角的变化、特定区域编辑以及主题集合设计，所有这些都由掩膜和提示引导。用户可以通过简单的文本命令或标记区域进行精确调整与系统互动，使其即使对没有技术专长的用户也易于使用。EPEdit 的核心利用基于 Stable Diffusion 模型的零-shot 图像编辑算法，消除了额外微调的需求。这种方法实现了高效的图像处理和主题集合创建。用户对图像编辑、主题设计和整体系统性能的评估表明，EPEdit 在现有解决方案中表现优越，提供了一种用户友好且具有成本效益的全面图像编辑解决方案。

View on arXiv Download PDF AI Translation

cs.CV / 15 / 2606.24058

VisChronos: Revolutionizing Image Captioning Through Real-Life Events

VisChronos：通过现实生活事件革新图像描述

Nguyen, Phuc-Tan, Nguyen, Hieu, Tran, Minh-Triet, Le, Trung-Nghia

Abstract

This paper aims to bridge the semantic gap between visual content and natural language understanding by leveraging historical events in the real world as a source of knowledge for caption generation. We propose VisChronos, a novel framework that utilizes large language models and dense captioning models to identify and describe real-life events from a single input image. Our framework can automatically generate detailed and context-aware event descriptions, enhancing the descriptive quality and contextual relevance of generated captions to address the limitations of traditional methods in capturing contextual narratives. Furthermore, we introduce a new dataset, EventCap (https://zenodo.org/records/14004909), specifically constructed using the proposed framework, designed to enhance the model's ability to identify and understand complex events. The user study demonstrates the efficacy of our solution in generating accurate, coherent, and event-focused descriptions, paving the way for future research in event-centric image understanding.

Chinese Translation

本文旨在通过利用现实世界中的历史事件作为生成描述的知识来源，弥合视觉内容与自然语言理解之间的语义差距。我们提出了VisChronos，一个新颖的框架，利用大型语言模型和密集描述模型，从单一输入图像中识别和描述现实生活事件。我们的框架能够自动生成详细且具有上下文意识的事件描述，增强生成描述的描述质量和上下文相关性，以解决传统方法在捕捉上下文叙事方面的局限性。此外，我们引入了一个新的数据集EventCap（https://zenodo.org/records/14004909），该数据集是使用所提出的框架专门构建的，旨在增强模型识别和理解复杂事件的能力。用户研究表明，我们的解决方案在生成准确、一致且以事件为中心的描述方面的有效性，为未来的事件中心图像理解研究铺平了道路。

View on arXiv Download PDF AI Translation

cs.CV / 16 / 2606.24059

Ingredient-Level Food Image Segmentation for Nutrition Awareness

基于成分级别的食品图像分割以提高营养意识

Shrestha, Jonesh

Abstract

Food images often contain several visible ingredients, so assigning one dish label to an entire image hides important visual structure. This work studies ingredient-level semantic segmentation on FoodSeg103, where the model predicts an ingredient class for each pixel. Two SegFormer variants were fine-tuned and evaluated under a controlled setup: SegFormer-B0 as the smaller baseline model and SegFormer-B1 as the larger final model. Both models use ImageNet-pretrained MiT backbones with newly initialized 104-class output layers. On the held-out FoodSeg103 test split of 2,135 images, B0 achieved 0.7709 pixel accuracy and 0.2521 mean IoU, while B1 achieved 0.7929 pixel accuracy and 0.3204 mean IoU. B1 improved every saved test metric, including a +0.0683 absolute gain in mean IoU. The system also converts predicted masks into visible ingredient-area percentages, giving a simple visual composition summary of the predicted meal. This summary can serve as a first-pass nutrition-awareness cue by providing a visual alternative to detailed food tracking similar to plate-based meal guidance, but it is not a direct estimate of calories, macronutrients, food mass, volume, density, or true portion size.

Chinese Translation

食品图像通常包含多个可见成分，因此将整个图像标记为一个菜品标签会掩盖重要的视觉结构。本研究在 FoodSeg103 数据集上研究成分级别的语义分割，其中模型为每个像素预测一个成分类别。我们对两个 SegFormer 变体进行了微调并在受控环境下评估：SegFormer-B0 作为较小的基线模型，SegFormer-B1 作为较大的最终模型。两个模型均使用在 ImageNet 上预训练的 MiT 主干，并新初始化了 104 类输出层。在保留的 FoodSeg103 测试集（2135 张图像）上，B0 达到了 0.7709 的像素准确率和 0.2521 的平均交并比（mean IoU），而 B1 则达到了 0.7929 的像素准确率和 0.3204 的平均交并比。B1 在每个保存的测试指标上都有所提升，包括平均交并比的绝对增益 +0.0683。该系统还将预测的掩膜转换为可见成分区域百分比，提供了对预测餐点的简单视觉组成摘要。该摘要可以作为初步的营养意识提示，通过提供一种视觉替代方案来替代详细的食品追踪，类似于基于盘子的餐点指导，但并不是对卡路里、宏量营养素、食品质量、体积、密度或真实份量大小的直接估计。

View on arXiv Download PDF AI Translation

cs.CV / 17 / 2606.24068

ObsGraph: Hierarchical Observation Representation for Embodied Reasoning and Exploration

ObsGraph：用于具身推理和探索的层次观察表示

Lee, Taekbeom, Jang, Youngseok, Heo, Jeonghwa, Choi, Jeongjun, Kim, H. Jin

Abstract

Embodied reasoning and exploration are increasingly considered crucial abilities for robots operating in complex and unfamiliar environments. To accomplish tasks in such settings, an agent must identify and acquire the information necessary for the task through exploration. We propose ObsGraph, an observation-centric hierarchical scene graph that unifies scene representation, retrieval, and exploration. It retains visual evidence and organizes it into room-view-object layers: rooms provide coarse semantic anchors, views preserve contextual object covisibility, and objects store fine-grained details. On top of this representation, we perform coarse-to-fine hierarchical retrieval under a bounded budget, and crucially use retrieval outcomes to structure the exploration candidate space--activating room-level exploration, view refinement, or frontier exploration--thereby tightly coupling representation, retrieval, and adaptive multi-scale exploration. Experiments across embodied reasoning and exploration benchmarks demonstrate improved success and efficiency, highlighting the benefits of structured scene representation and more targeted information gathering driven by identified evidence gaps.

Chinese Translation

具身推理和探索被越来越多地认为是机器人在复杂和陌生环境中操作的关键能力。为了在这样的环境中完成任务，智能体必须通过探索识别和获取完成任务所需的信息。我们提出了ObsGraph，一种以观察为中心的层次场景图，统一了场景表示、检索和探索。它保留了视觉证据，并将其组织为房间-视图-对象层次：房间提供粗略的语义锚点，视图保留上下文中的对象共视性，而对象则存储细粒度的细节。在此表示之上，我们在有限预算下执行粗到细的层次检索，并且关键地利用检索结果来构建探索候选空间——激活房间级探索、视图细化或前沿探索——从而紧密结合表示、检索和自适应多尺度探索。在具身推理和探索基准上的实验表明，成功率和效率得到了改善，突显了结构化场景表示和由识别的证据差距驱动的更有针对性的信息收集的好处。

View on arXiv Download PDF AI Translation

cs.CV / 18 / 2606.24072

Fabric Image Demoir\'eing Benchmark from Synthesis to Restoration

从合成到恢复的织物图像去摩尔纹基准

Wei, Pengchao, Guo, Xiaojie

Abstract

Fabric moir\'e is a sampling-induced aliasing artifact caused by the interaction between fine textile patterns and camera sensor grids, producing structured interference that severely degrades image quality. Unlike screen-induced moir\'e, which stems from strictly periodic display lattices, fabric moir\'e is intrinsically more challenging due to the broadband and semi-periodic nature of textile weaves. The heavy spectral overlap between intrinsic texture and aliasing components renders fabric demoir\'eing substantially more ill-posed. Consequently, existing models trained on screen moir\'e datasets generalize poorly to these complex textile patterns. Despite its practical importance, fabric image demoir\'eing remains underexplored and lacks standardized benchmarks. We present the first comprehensive benchmark for fabric image demoir\'eing. To address the difficulty of acquiring pixel-aligned real-world pairs, we develop a physically motivated synthesis framework and construct a large-scale dataset comprising 16,050 paired multi-resolution fabric images with controllable aliasing severity. Furthermore, we customize a baseline model, which establishes promising performance on the proposed benchmark dataset with strong generalization ability. Our benchmark provides a standardized platform for advancing research in fabric image demoir\'eing.

Chinese Translation

织物摩尔纹是一种由细致纺织图案与相机传感器网格之间的相互作用引起的采样诱导混叠伪影，产生结构化干扰，严重降低图像质量。与由严格周期性显示晶格引起的屏幕摩尔纹不同，织物摩尔纹由于纺织品编织的宽带和半周期性特征，内在上更具挑战性。内在纹理与混叠成分之间的重叠谱使得织物去摩尔纹问题显著更加不适定。因此，现有在屏幕摩尔纹数据集上训练的模型在这些复杂的纺织图案上泛化效果较差。尽管其实际重要性，织物图像去摩尔纹仍然未被充分探索，并且缺乏标准化基准。我们提出了第一个全面的织物图像去摩尔纹基准。为了应对获取像素对齐的真实世界配对的困难，我们开发了一个基于物理的合成框架，并构建了一个大型数据集，包含16,050对具有可控混叠严重度的多分辨率织物图像。此外，我们定制了一个基线模型，该模型在所提出的基准数据集上建立了良好的性能，并具有较强的泛化能力。我们的基准为推动织物图像去摩尔纹研究提供了一个标准化平台。

View on arXiv Download PDF AI Translation

cs.CV / 19 / 2606.24075

End-to-End Radar and Communication Modulation Recognition with Neuromorphic Computing

基于神经形态计算的端到端雷达与通信调制识别

Li, Xiaohu, Qu, Chongxiao, Lin, Caiyong, Dou, Chenxiao, Hua, Wei

Abstract

Although deep learning-based methods can achieve high accuracy in automatic modulation recognition (AMR) tasks, their high computational cost makes it difficult to strike a balance between accuracy and power consumption, thereby limiting their application on resource-constrained platforms. Neuromorphic architectures that perform spike-driven inference with modest energy budgets have recently been explored for vision and timeseries tasks. Motivated by these works, we propose EMRFormer, a novel end-to-end spiking nerural network (SNN) architecture that applies spike-driven transformer to the constraints of neuromorphic hardware for AMR. The model incorporates an adaptive spike encoder and Integer Leaky Integrate-and-Fire neurons to mitigate the degradation of effective information and enhance SNN representational capacity. By integrating spike-separable Convolution Neural Networks (SSCNN) into Spike-Driven Transformers (SpikeFormer), EMRFormer effectively extracts multi-scale temporal features from the raw IQ waveforms. We validate our approach across various mainstream datasets, the experimental results show that EMRFormer achieves state-of-the-art interms of accuracy, outperforming all the baselines. Furthermore, the model maintains strong performance in low signal-to-noise(SNR) environments and reduces theoretical energy consumption by over 90%. Finally, we evaluate our model on a KA200 neuromorphic chip. The results show that our model achieves up to 5 times reduction in power compared to running on a 3090 GPU or an Orin NX. This work demonstrates a promising pathway for AMR on resource-constrained devices.

Chinese Translation

尽管基于深度学习的方法在自动调制识别（AMR）任务中可以实现高准确率，但其高计算成本使得在准确性与功耗之间取得平衡变得困难，从而限制了其在资源受限平台上的应用。最近，神经形态架构在适度能量预算下进行脉冲驱动推理的研究已被探索用于视觉和时间序列任务。受到这些工作的启发，我们提出了EMRFormer，一种新颖的端到端脉冲神经网络（SNN）架构，它将脉冲驱动变换器应用于AMR的神经形态硬件约束。该模型结合了自适应脉冲编码器和整数泄漏积分-发火神经元，以减轻有效信息的退化并增强SNN的表征能力。通过将脉冲可分离卷积神经网络（SSCNN）集成到脉冲驱动变换器（SpikeFormer）中，EMRFormer有效地从原始IQ波形中提取多尺度时序特征。我们在多个主流数据集上验证了我们的方法，实验结果表明EMRFormer在准确性方面达到了最先进的水平，超越了所有基线。此外，该模型在低信噪比（SNR）环境下保持了强劲的性能，并将理论能耗降低了超过90%。最后，我们在KA200神经形态芯片上评估了我们的模型。结果表明，与在3090 GPU或Orin NX上运行相比，我们的模型在功耗上减少了多达5倍。这项工作展示了在资源受限设备上进行AMR的有希望的途径。

View on arXiv Download PDF AI Translation

cs.CV / 20 / 2606.24092

Progressive Pixel-Neighborhood Deformable Cross-Attention for Multispectral Object Detection

渐进式像素邻域可变形交叉注意力用于多光谱目标检测

Qiu, Tian, Shen, Jifeng, Zuo, Xin

Abstract

Effective cross-modal feature alignment and interaction are central challenges in multispectral object detection. Although global cross-attention provides strong long-range modeling ability, its quadratic complexity with respect to feature size limits deployment on resource-constrained platforms. We therefore propose Progressive Pixel-Neighborhood Deformable Cross-Attention for multispectral feature fusion, termed PNAFusion. The proposed framework is motivated by two observations: weak misalignment between visible and thermal images is usually concentrated around local neighborhoods, and semantic correspondence across modalities often follows non-linear spatial mappings that fixed receptive fields cannot model well. To address these issues, PNAFusion incorporates local spatial priors into its architectural design to concentrate feature interaction and alignment on the most relevant neighborhoods. Specifically, a Pixel-Neighborhood Cross-Attention (PNCA) module is introduced to avoid redundant global feature matching and suppress background noise. Meanwhile, an Adaptive Deformable Alignment (ADA) module captures non-linear spatial correspondences through learned pixel-wise offsets. These components are further integrated through an iterative feedback mechanism to progressively refine cross-modal feature alignment. Experiments on FLIR, M3FD, and DroneVehicle show that PNAFusion achieves 84.2, 90.5, and 85.5 [email protected], respectively, under the YOLOv5 detector, and further reaches 86.8 [email protected] on FLIR and 90.8 [email protected] on M3FD when transferred to Co-DETR. Efficiency analysis indicates that PNAFusion reduces allocated GPU memory by 33.0\% compared with ICAFusion and reduces theoretical FLOPs from 194.8 G to 156.4 G, although the deformable sampling and iterative refinement introduce additional latency. Our code will be available at https://github.com/DanielQiuTian/PNAFusion.

Chinese Translation

有效的跨模态特征对齐和交互是多光谱目标检测中的核心挑战。尽管全局交叉注意力提供了强大的长距离建模能力，但其相对于特征大小的二次复杂度限制了在资源受限平台上的应用。因此，我们提出了一种用于多光谱特征融合的渐进式像素邻域可变形交叉注意力，称为 PNAFusion。该框架的提出基于两个观察：可见光图像与热成像之间的弱错位通常集中在局部邻域周围，以及跨模态的语义对应关系通常遵循固定感受野无法很好建模的非线性空间映射。为了解决这些问题，PNAFusion 在其架构设计中融入了局部空间先验，以将特征交互和对齐集中在最相关的邻域上。具体而言，引入了像素邻域交叉注意力（Pixel-Neighborhood Cross-Attention, PNCA）模块，以避免冗余的全局特征匹配并抑制背景噪声。同时，自适应可变形对齐（Adaptive Deformable Alignment, ADA）模块通过学习的逐像素偏移捕捉非线性空间对应关系。这些组件通过迭代反馈机制进一步集成，以逐步优化跨模态特征对齐。在 FLIR、M3FD 和 DroneVehicle 上的实验表明，PNAFusion 在 YOLOv5 检测器下分别达到了 84.2、90.5 和 85.5 的 [email protected]，并且在转移到 Co-DETR 时，FLIR 上进一步达到了 86.8 [email protected]，M3FD 上达到了 90.8 [email protected]。效率分析表明，与 ICAFusion 相比，PNAFusion 将分配的 GPU 内存减少了 33.0\%，并将理论 FLOPs 从 194.8 G 降低到 156.4 G，尽管可变形采样和迭代优化引入了额外的延迟。我们的代码将可在 https://github.com/DanielQiuTian/PNAFusion 获取。

View on arXiv Download PDF AI Translation

cs.CV / 21 / 2606.24094

Universal Guideline-Driven Image Clustering via a Hybrid LLM Agent

基于通用指南驱动的图像聚类通过混合大语言模型代理

Zhong, Wenliang, Barton, Rob, Goncalves, Lucas, Kumar, Kushal, Jiang, Feng, Ma, Hehuan, Guo, Yuzhi, Bansal, Vidit, Bouyarmane, Karim, Huang, Junzhou

Abstract

Unifying image clustering across different clustering scenarios remains challenging due to fundamental gaps among tasks. We introduce a Guideline-Driven Image Clustering Agent, the first universal framework that bridges these gaps through textual guidelines. To incorporate complex guidelines without task-specific training, we propose Generative Concept Proxy Modeling, which generates guideline-aware embeddings via concept proxy extraction. For scenarios requiring automatic cluster discovery, we introduce LLM Traversal based on Minimum Spanning Tree that selectively applies LLM reasoning for complex semantic judgments. Our method generalizes across diverse clustering scenarios spanning from general to fine-grained categorization, from global to local criteria, and from balanced to long-tail distributions. Our framework consistently outperforms specialized methods across diverse clustering tasks.

Chinese Translation

在不同聚类场景中统一图像聚类仍然面临挑战，因为任务之间存在根本性的差距。我们提出了一种指南驱动的图像聚类代理，这是第一个通过文本指南弥合这些差距的通用框架。为了在不进行特定任务训练的情况下整合复杂的指南，我们提出了生成概念代理建模（Generative Concept Proxy Modeling），该方法通过概念代理提取生成与指南相关的嵌入。对于需要自动聚类发现的场景，我们引入了基于最小生成树（Minimum Spanning Tree）的LLM遍历（LLM Traversal），该方法选择性地应用LLM推理以进行复杂的语义判断。我们的方法在从一般到细粒度分类、从全局到局部标准以及从均衡到长尾分布的多样聚类场景中具有广泛的适用性。我们的框架在多种聚类任务中始终优于专门的方法。

View on arXiv Download PDF AI Translation

cs.CV / 22 / 2606.24096

Beyond Bayer: Task-Optimal Sensor Co-Design for Robust Autonomous-Driving Segmentation

超越拜耳：面向鲁棒自主驾驶分割的任务最优传感器协同设计

Khan, Reeshad, Gauch, John

Abstract

Robust perception underpins autonomous driving, and most recent progress comes from scaling the model-larger backbones, foundation models, and cooperative multi-agent fusion. We pursue a complementary, upstream question: what should the camera itself measure? Using a differentiable RAW-to-task pipeline, we decompose which sensor degrees of freedom benefit dense prediction. Learning the spectral colour-filter-array (CFA) weights is the dominant lever, improving mIoU by +0.017 (KITTI-360) and +0.023 (ACDC) over a fixed camera. In contrast, point-spread-function (optics) co-design is net-negative (-0.020 mIoU on KITTI-360) - a consequence of the data-processing inequality, which also bounds the task information that any downstream model, however large or cooperative, can recover. Noise co-optimisation is marginal, and counter to intuition enlarging the CFA tile beyond 2x2 consistently hurts, as the filters are confined to the rank three sRGB input. Because the intervention is at the sensor, the gains are model-agnostic; we validate robustness on ACDC's fog, night, rain, and snow, and conclude with a simple recipe: learn the 2x2 CFA weights and keep an identity PSF.

Chinese Translation

鲁棒感知是自主驾驶的基础，而最近的进展主要来自于模型规模的扩大——更大的主干网络、基础模型和协作多智能体融合。我们追求一个互补的上游问题：相机本身应该测量什么？通过可微分的RAW到任务管道，我们分解了哪些传感器自由度有利于密集预测。学习光谱颜色滤光阵列（CFA）权重是主要的杠杆，相较于固定相机，提升了KITTI-360数据集的mIoU值+0.017和ACDC数据集的mIoU值+0.023。相比之下，点扩散函数（光学）协同设计的效果是负面的（在KITTI-360上降低了-0.020 mIoU）——这是数据处理不等式的结果，该不等式也限制了任何下游模型，无论其多么庞大或合作，能够恢复的任务信息。噪声协同优化的效果微乎其微，且与直觉相反，将CFA单元扩大到超过2x2的尺寸始终会造成损害，因为滤波器被限制在秩为三的sRGB输入中。由于干预发生在传感器层面，因此这些收益与模型无关；我们在ACDC的雾、夜间、雨天和雪天条件下验证了鲁棒性，并总结出一个简单的方案：学习2x2 CFA权重并保持恒等的点扩散函数（PSF）。

View on arXiv Download PDF AI Translation

cs.CV / 23 / 2606.24107

DramaDirector: Geometry-Guided Short Drama Generation

DramaDirector：几何引导的短剧生成

Zhou, Hengji, Liu, Sijie, Chen, Jianrun, Zou, Xingchen, Xia, Lianghao, Nie, Liqiang

Abstract

Short dramas, with their rapid shot rhythms, dialogue-driven focus shifts, and demanding cinematographic grounding, pose challenges that prompt-level or text-only video generation pipelines struggle to meet. We study plot-to-short-drama generation, where a global plot and local context are transformed into visually grounded multi-shot videos. We propose DramaDirector, a geometry-grounded framework that lets the planner borrow cinematographic geometry from a gallery of real short-drama shots indexed by depth and pose. DramaDirector decouples each shot into static visual and dynamic narrative conditions, trains the planner with schema-constrained SFT and GRPO under a learned text-visual alignment reward, and retrieves depth-pose references to guide first-frame generation and image-to-video synthesis. We also introduce DramaBoard, a benchmark built from 35 live-action dramas, 2.8K episodes, and 81K shots, with structured storyboards and multi-dimensional evaluation protocols. Experiments show that DramaDirector improves over representative multi-agent and video generation baselines on faithfulness, consistency, and controllability. Our code is released at: https://github.com/iLearn-Lab/DramaDirector

Chinese Translation

短剧因其快速的镜头节奏、以对话为驱动的焦点转移以及对电影摄影基础的严格要求，给基于提示或仅依赖文本的视频生成流程带来了挑战。我们研究了情节到短剧的生成，其中全球情节和局部上下文被转化为视觉基础的多镜头视频。我们提出了DramaDirector，一个几何基础的框架，使得规划者能够从一个按深度和姿态索引的真实短剧镜头库中借用电影摄影几何。DramaDirector将每个镜头解耦为静态视觉和动态叙事条件，在学习的文本-视觉对齐奖励下，使用结构约束的SFT（Schema-Constrained Fine-Tuning）和GRPO（Gradient-Reinforced Policy Optimization）训练规划者，并检索深度-姿态参考以指导第一帧生成和图像到视频的合成。我们还引入了DramaBoard，一个基于35部真人剧、2.8K集和81K镜头构建的基准，配有结构化故事板和多维评估协议。实验表明，DramaDirector在忠实性、一致性和可控性方面优于代表性的多智能体和视频生成基线。我们的代码已发布于：https://github.com/iLearn-Lab/DramaDirector

View on arXiv Download PDF AI Translation

cs.CV / 24 / 2606.24115

A Benchmark for Hallucination Detection in VLMs for Gastrointestinal Endoscopy

针对胃肠内窥镜中视觉语言模型的幻觉检测基准

Lawal, Aminu, Oli, Niyoj, Acharya, Sachin, Gyawali, Prashnna, Romano, Maria Carmen, Bhattarai, Binod

Abstract

Vision-language models (VLMs) are prone to hallucination, which remains a major barrier to their safe deployment in clinical practice. To date, most hallucination detection methods have been evaluated on radiology benchmarks such as MIMIC-CXR and VQA-RAD, while gastrointestinal (GI) endoscopy remains largely underexplored. In this paper, we benchmark nine hallucination detection methods on the Gut-VLM dataset, a GI diagnostic Visual Question Answering (VQA) dataset with 4,392 test VQA pairs, across five VLMs (MedGemma-4B, MedGemma-27B, LLaVA-Med-7B, LLaVA-v1.6-7B, and Lingshu-32B). The methods span three categories: black-box methods (RadFlag, SelfCheckGPT-NLI), gray-box methods (AvgProb, AvgEnt, MaxProb, MaxEnt, Semantic Entropy, and VASE), and a white-box method (ReXTrust). Our results show that ReXTrust, a white-box method, achieves the highest AUC across all five models, outperforming the strongest alternative method on each VLM by a statistically significant margin (paired permutation test, p < 0.001 in all cases), reaching a peak AUC of 93.0 on MedGemma-4B. White-box hidden-state access provides a consistent advantage of 19.5 AUC points on average (range: 9.5--33.5), with ReXTrust maintaining strong performance even on LLaVA-v1.6-7B (AUC 79.9), where black-box methods and clustering-based gray-box methods collapse to near-chance performance. Among non-white-box methods, token-level gray-box statistics (MaxEnt, MaxProb) are the strongest alternatives, outperforming both clustering-based gray-box methods (Semantic Entropy, VASE) and black-box approaches on average. We further identify confident confabulation, a failure mode in which models hallucinate with high inter-sample consistency or high token-level probability, as a systemic failure for both consistency and uncertainty-based methods.

Chinese Translation

视觉语言模型（VLMs）容易出现幻觉，这仍然是其在临床实践中安全部署的主要障碍。迄今为止，大多数幻觉检测方法已在放射学基准上进行评估，如 MIMIC-CXR 和 VQA-RAD，而胃肠（GI）内窥镜领域仍然相对未被充分探索。在本文中，我们在 Gut-VLM 数据集上对九种幻觉检测方法进行了基准测试，该数据集是一个包含 4,392 对测试 VQA 的 GI 诊断视觉问答（VQA）数据集，涉及五种 VLM（MedGemma-4B、MedGemma-27B、LLaVA-Med-7B、LLaVA-v1.6-7B 和 Lingshu-32B）。这些方法分为三类：黑箱方法（RadFlag、SelfCheckGPT-NLI）、灰箱方法（AvgProb、AvgEnt、MaxProb、MaxEnt、Semantic Entropy 和 VASE）以及一种白箱方法（ReXTrust）。我们的结果显示，白箱方法 ReXTrust 在所有五个模型中获得了最高的 AUC，显著优于每个 VLM 上最强的替代方法（配对置换检验，所有情况下 p < 0.001），在 MedGemma-4B 上达到峰值 AUC 93.0。白箱隐藏状态访问平均提供了 19.5 AUC 点的持续优势（范围：9.5--33.5），而 ReXTrust 在 LLaVA-v1.6-7B 上仍保持强劲表现（AUC 79.9），而黑箱方法和基于聚类的灰箱方法则接近随机表现。在非白箱方法中，基于标记的灰箱统计（MaxEnt、MaxProb）是最强的替代方案，平均优于基于聚类的灰箱方法（Semantic Entropy、VASE）和黑箱方法。我们进一步识别出自信的虚构（confident confabulation），即模型在高样本间一致性或高标记级概率下产生幻觉的失败模式，作为一致性和基于不确定性的方法的系统性失败。

View on arXiv Download PDF AI Translation

cs.CV / 25 / 2606.24118

An LMM for Precisely Grounding Elements in Documents

用于精确定位文档中元素的大型多模态模型（LMM）

Lu, Yijian, Zhao, Chuangxin, Sun, Kai, Hou, Lei, Li, Juanzi, Qi, Ji

Abstract

Visual grounding in documents is a crucial ability for Large Multimodal Models (LMMs) in areas such as document understanding, deep research and document error detection. However, existing approaches exhibit poor grounding precision in text-rich document images, often failing to accurately locate the critical document elements needed for reliable reasoning. To address this gap, we introduce PreciseDoc, an LMM specifically designed for precise element grounding and can be further optimized for Document VQA tasks. Specifically, to enhance the basic localization capability, we construct challenging training data by two pipelines capable of mass-producing high-quality documents with paired metadata of fine-grained coordinates, including synthetic hand-filled documents with camera effects. The model develops more real-world functions beyond straightforward localization of single text, such as locating personal information from CVs. Furthermore, we introduce a training paradigm for visual grounded reasoning where the grounding and reasoning are supervised jointly with reinforcement learning to improve the contribution of the grounded evidence. A comprehensive evaluation on various benchmarks demonstrates the advantage of the proposed data and methods in document spatial grounding and document understanding.

Chinese Translation

文档中的视觉定位是大型多模态模型（LMM）在文档理解、深入研究和文档错误检测等领域的重要能力。然而，现有方法在文本丰富的文档图像中表现出较差的定位精度，常常无法准确定位进行可靠推理所需的关键文档元素。为了解决这一问题，我们提出了PreciseDoc，这是一种专门设计用于精确元素定位的LMM，并可以进一步优化用于文档视觉问答（Document VQA）任务。具体而言，为了增强基本的定位能力，我们通过两个管道构建了具有挑战性的训练数据，这些管道能够大规模生成高质量的文档，并配有细粒度坐标的配对元数据，包括具有相机效果的合成手填文档。该模型开发了超越单一文本简单定位的更多现实世界功能，例如从简历中定位个人信息。此外，我们引入了一种视觉基础推理的训练范式，其中定位和推理通过强化学习共同监督，以提高基础证据的贡献。在各种基准上的全面评估证明了所提数据和方法在文档空间定位和文档理解方面的优势。

View on arXiv Download PDF AI Translation

cs.CV / 26 / 2606.24120

Flood Mapping from RGB imagery using a Vision Foundation Model

基于RGB影像的洪水制图使用视觉基础模型

Polushko, Vladyslav, Bucher, Tilman, Rösch, Ronald, März, Thomas, Rauhut, Markus, Weinmann, Andreas

Abstract

Timely, high-resolution maps of flood extent around settlements are essential for emergency response and damage assessment. We consider airborne RGB imagery for flood mapping as it can be collected rapidly at low cost. To produce flood maps, deep learning models for water segmentation are often used. CNN based and small vision transformer models are used. However, they need much data for adaptation to a change of scenery, i.e., another flooding event. Vision foundation models or large vision transformers are known to generalize across domains. Recently, foundation models for Earth observation became available. They are pretrained on satellite data, whose spatial resolution, viewing geometry, and radiometry differ from nadir RGB imagery. Thus, adaptation is required. We investigate how a satellite-pretrained Earth observation foundation model can be adapted to centimeter-scale floodwater mapping from RGB imagery. Specifically, we fine-tune a model we call Prithvi-2.0-UPN consisting of the Prithvi-EO-2.0-600M Vision Transformer combined with a UPerNet decoder for binary water segmentation on two RGB datasets (BlessemFlood21, NeuenahrFlood). In a first experiment we observe that Prithvi-2.0-UPN reaches state-of-the-art results on BlessemFlood21 and NeuenahrFlood, when trained on their datasets. In a second experiment we show that Prithvi-2.0-UPN performs better than state-of-the-art baseline models for transfer to a new flood event (trained on BlessemFlood21, tested on NeuenahrFlood) in a zero-shot setting. However, the performance indicates room for improvement. In this respect, we investigate in a third experiment how performance improves when further fine-tuning the models with small shares of NeuenahrFlood training data: Prithvi-2.0-UPN improves the fastest and reaches almost the performance level when fully trained on NeuenahrFlood, indicating transfer capabilities.

Chinese Translation

及时、高分辨率的洪水范围地图对于应急响应和损失评估至关重要。我们考虑使用空中RGB影像进行洪水制图，因为它可以以低成本快速收集。为了生成洪水地图，通常使用深度学习模型进行水体分割。我们使用基于卷积神经网络（CNN）和小型视觉变换器模型。然而，这些模型需要大量数据以适应场景变化，即另一次洪水事件。视觉基础模型或大型视觉变换器被认为能够跨领域泛化。最近，针对地球观测的基础模型已变得可用。这些模型在卫星数据上进行预训练，其空间分辨率、观测几何和辐射度与垂直RGB影像不同。因此，需要进行适应性调整。我们研究了如何将卫星预训练的地球观测基础模型适应于基于RGB影像的厘米级洪水映射。具体而言，我们对一个称为Prithvi-2.0-UPN的模型进行了微调，该模型由Prithvi-EO-2.0-600M视觉变换器和用于二元水体分割的UPerNet解码器组成，使用两个RGB数据集（BlessemFlood21和NeuenahrFlood）。在第一次实验中，我们观察到Prithvi-2.0-UPN在BlessemFlood21和NeuenahrFlood数据集上训练时达到了最先进的结果。在第二次实验中，我们展示了Prithvi-2.0-UPN在零样本设置下，相较于最先进的基线模型在转移到新的洪水事件（在BlessemFlood21上训练，在NeuenahrFlood上测试）时表现更佳。然而，性能表明仍有改进空间。在这方面，我们在第三次实验中研究了当进一步微调模型并使用少量NeuenahrFlood训练数据时性能的改善：Prithvi-2.0-UPN的提升速度最快，几乎达到了在NeuenahrFlood上完全训练时的性能水平，表明其转移能力。

View on arXiv Download PDF AI Translation

cs.CV / 27 / 2606.24122

Bengal-HP_RU: A Dataset of Bengal People For Head Pose Estimation

Bengal-HP_RU：用于头部姿态估计的孟加拉人数据集

Khan, Md. Ahanaf Arif, Rahman, Md. Tawhidur, Biswas, Sangeeta, Khan, Md. Iqbal Aziz, Pramanik, Subrata, Chakravarty, Sanjoy Kumar, Pramanik, Bimal Kumar

Abstract

Existing head pose datasets predominantly feature subjects of Western or East Asian origin, leaving South Asian populations, particularly Bengali individuals, largely underrepresented. We introduce Bengal-HP_RU, the first publicly available head pose dataset centred on Bengali subjects, comprising 12,894 labelled head images annotated with continuous yaw, pitch, and roll values. Images were collected from Wikimedia Commons under free licences and processed through an automated pipeline followed by manual label correction. The dataset is partitioned by Wikimedia uploader identity to prevent data contamination, yielding 10,494 training and 2,400 test images across 296 unique uploaders. Bengal-HP_RU exhibits substantial diversity in subject age, gender, occlusion, illumination, and background, reflecting realistic in-the-wild conditions. The dataset is publicly available at https://doi.org/10.17632/xbw9kr37jb.2.

Chinese Translation

现有的头部姿态数据集主要以西方或东亚人群为主，南亚人群，特别是孟加拉人群体的代表性不足。我们推出了Bengal-HP_RU，这是第一个以孟加拉人为中心的公开头部姿态数据集，包含12,894张标注的头部图像，并附有连续的偏航、俯仰和滚转值。图像来自Wikimedia Commons，在自由许可下收集，并通过自动化流程处理，随后进行人工标签校正。数据集按Wikimedia上传者身份进行分区，以防止数据污染，共包含10,494张训练图像和2,400张测试图像，来自296个独特的上传者。Bengal-HP_RU在受试者的年龄、性别、遮挡、光照和背景方面展现出显著的多样性，反映了现实世界中的真实条件。该数据集可在https://doi.org/10.17632/xbw9kr37jb.2上公开获取。

View on arXiv Download PDF AI Translation

cs.CV / 28 / 2606.24138

Sat2City v2: Native 3D City Asset Generation from a Single Satellite Image

Sat2City v2：基于单幅卫星图像的原生三维城市资产生成

Hua, Tongyan, Wu, Dongli, Zhu, Jinjing, Ren, Yinrui, Hong, Zhongcheng, Chen, Ying-Cong, Xiong, Hui, Zhao, Wufan

Abstract

Generating explicit 3D city assets from a single satellite image is important for digital twins, urban simulation, and geospatial intelligence. Unlike satellite-to-street-view synthesis, the task requires a reusable textured mesh with plausible geometry and controllable appearance rather than a 3D proxy optimized only for rendering a small set of images or videos. The ICCV Sat2City framework made a first step by conditioning cascaded sparse-voxel latent diffusion on satellite-derived height maps, but its appearance was random, its training data were synthetic, and its task-specific VAE did not scale well to noisy real-world reconstructions. We present Sat2City v2, a journal extension that adapts a pretrained native structured-latent 3D foundation model to weakly aligned satellite images and textured meshes. We build a real-world dataset with 16,241 satellite-mesh pairs across 24 regions in 9 cities. Instead of learning a 3D representation from noisy city meshes, Sat2City v2 encodes each mesh into a pretrained native 3D latent space, fine-tunes a satellite-conditioned geometry flow, and uses the decoded shape to anchor satellite-conditioned texturing. This retains Sat2City's geometry-to-appearance cascade while enabling appearance-controllable generation from the satellite input. Experiments on metric-scale DSM reconstruction and generative city-asset benchmarks for geometry and appearance show that Sat2City v2 achieves the best overall performance among evaluated baselines. Overall, Sat2City v2 advances satellite-to-city generation from rendering-oriented 3D proxies to explicit textured mesh assets, supported by, to the best of our knowledge, the first documented satellite-mesh paired dataset collected from matched geographic crops for this asset-level task. Project page: https://ai4city-hkust.github.io/Sat2City-v2/

Chinese Translation

从单幅卫星图像生成明确的三维城市资产对于数字双胞胎、城市模拟和地理空间智能至关重要。与卫星到街景的合成不同，该任务需要一个可重用的纹理网格，具备合理的几何形状和可控的外观，而不是仅为渲染一小组图像或视频而优化的三维代理。ICCV Sat2City框架通过在卫星衍生的高度图上对级联稀疏体素潜在扩散进行条件化，迈出了第一步，但其外观是随机的，训练数据是合成的，且其特定任务的变分自编码器（VAE）在噪声较大的现实世界重建中表现不佳。我们提出了Sat2City v2，这是一个期刊扩展，旨在将预训练的原生结构化潜在三维基础模型适应于弱对齐的卫星图像和纹理网格。我们构建了一个包含来自9个城市24个区域的16,241对卫星-网格对的真实世界数据集。Sat2City v2并不是从噪声城市网格中学习三维表示，而是将每个网格编码到预训练的原生三维潜在空间中，微调卫星条件的几何流，并使用解码后的形状来锚定卫星条件的纹理。这保留了Sat2City的几何到外观的级联，同时使得从卫星输入生成可控外观成为可能。在度量尺度的DSM重建和几何与外观的生成城市资产基准测试中的实验表明，Sat2City v2在评估的基线中达到了最佳的整体性能。总体而言，Sat2City v2将卫星到城市的生成从以渲染为导向的三维代理推进到明确的纹理网格资产，并且据我们所知，这是第一个为该资产级任务收集的匹配地理作物的卫星-网格配对数据集。项目页面：https://ai4city-hkust.github.io/Sat2City-v2/

View on arXiv Download PDF AI Translation

cs.CV / 29 / 2606.24144

Geometry-Aware Style Transfer in 3D Gaussian Splatting

基于几何感知的3D高斯点云风格迁移

Bang, Min Hyeok, Kim, Jun Hyeong, Kim, Seung-Wook, Lee, Se-Ho

Abstract

In this paper, we present a novel geometry-aware style transfer framework for 3D Gaussian splatting (3DGS) that simultaneously transfers appearance attributes and geometric structures. Unlike prior works that primarily focus on color-based stylization and often overlook structural adaptation, our method explicitly incorporates geometry adaptation through a decoupled optimization scheme that alternately updates color and geometry parameters. This strategy alleviates potential interference between color and geometry updates, leading to stable and consistent scene-level geometry transformation. The decoupled optimization is enabled by the proposed geometry-aware contrastive feature matching (GCFM). GCFM integrates RGB, depth, and edge cues into a contrastive objective and is employed in both optimization phases to effectively transfer structural characteristics from style images to Gaussian primitives. Extensive experiments show that our approach achieves superior performance in both qualitative fidelity and quantitative metrics, significantly outperforming existing 3DGS-based stylization methods. Our code is available at \href{https://github.com/oweixx/gast}{https://github.com/oweixx/gast}.

Chinese Translation

本文提出了一种新颖的基于几何感知的3D高斯点云风格迁移框架（3DGS），该框架同时迁移外观属性和几何结构。与以往主要关注基于颜色的风格化并常常忽视结构适应的研究不同，我们的方法通过解耦优化方案显式地融入几何适应，交替更新颜色和几何参数。这一策略减轻了颜色和几何更新之间的潜在干扰，从而实现了稳定且一致的场景级几何变换。解耦优化得益于我们提出的几何感知对比特征匹配（GCFM）。GCFM将RGB、深度和边缘线索整合到对比目标中，并在两个优化阶段中使用，以有效地将结构特征从风格图像转移到高斯原语上。大量实验表明，我们的方法在定性保真度和定量指标上均表现出色，显著优于现有的基于3DGS的风格化方法。我们的代码可在 exttt{https://github.com/oweixx/gast} 获取。

View on arXiv Download PDF AI Translation

cs.CV / 30 / 2606.24152

Autonomous Video Generation with Counterfactual Controllability for Self-Evolving World Models

具有反事实可控性的自主视频生成以实现自我演化的世界模型

Wang, Xin, Liu, Wenxuan, Feng, Tongtong, Zhu, Wenwu

Abstract

Existing literature claims that video generation essentially is world modelling. On the one hand, the claim is productive because it pushes generative AI beyond static images and toward temporally extended physical scenes. On the other hand, this claim dangerously relies on the belief that scaling visual prediction alone will automatically yield physical agents. We prefer a more accurate statement: video generation models learn a partial, implicit spatiotemporal world model, but not a fully grounded or controllable one. The reason is as follows: a model may generate a plausible video of a drone crossing a forest or a robot arm manipulating a cup, yet still fail to know which variables are controllable, which constraints belong to a particular body and which futures remain valid under intervention. The frontier in essence is not predictive realism alone, instead it emphasizes a self-evolving generative nature that requires the decisive criterion to be counterfactual controllability: the capability of asking what would happen under an action, to test whether the generated future can survive embodiment constraints and to feed the resulting action knowledge back into future imagination (generation). Therefore, in this paper we present a new perspective, i.e., autonomous video generation with counterfactual controllability is one promising way to realize self-evolving world models.

Chinese Translation

现有文献声称视频生成本质上是世界建模。一方面，这一说法是有益的，因为它推动生成性人工智能超越静态图像，朝向时间延续的物理场景发展。另一方面，这一说法危险地依赖于这样一种信念：仅仅扩大视觉预测的规模就能自动产生物理代理。我们更倾向于一个更准确的表述：视频生成模型学习的是一个部分的、隐含的时空世界模型，而不是一个完全扎根或可控的模型。原因如下：一个模型可能生成一个可信的视频，例如无人机穿越森林或机器人手臂操控杯子，但仍然无法知道哪些变量是可控的，哪些约束属于特定的物体，以及在干预下哪些未来仍然有效。前沿问题本质上不仅仅是预测现实主义，而是强调一种自我演化的生成特性，这要求决定性标准是反事实可控性：即询问在某个行动下会发生什么，以测试生成的未来是否能够承受具身约束，并将由此产生的行动知识反馈到未来的想象（生成）中。因此，在本文中，我们提出了一种新视角，即具有反事实可控性的自主视频生成是一种实现自我演化世界模型的有前景的方法。

View on arXiv Download PDF AI Translation

cs.CV / 31 / 2606.24153

Differential Unfolding: Efficient Unfolding Reconstruction for Video Snapshot Compressive Imaging

差异展开：视频快照压缩成像的高效展开重建

Zhang, Muyuan, Zhang, Jiancheng, Zeng, Haijin, Zhao, Yin-ping

Abstract

While Deep Unfolding Networks (DUNs) dominate video Snapshot Compressive Imaging (SCI), they remain constrained by a uniform design philosophy. Existing methods repeatedly stack high-complexity priors with identical structures, ignoring the fact that optimization trajectories converge toward static states. This results in representation stagnation, where high-cost computations are wasted on minimal feature updates. To address this inefficiency, we present Differential Unfolding (DU), a heterogeneous framework that replaces uniform repetition with dynamic evolution. Central to DU is the Differential Evolutionary Framework (DEF), which partitions the unfolding process into two complementary roles: structural anchoring and differential evolution. In this scheme, high-parameter general stages are sparsely deployed to generate high-fidelity feature foundations. Complementing these, lightweight differential stages employ a Differential Representation Prior (DRP) to propagate and refine these foundational features through a differential mechanism. By integrating Differential Representation Attention (DRA) for evolving attention maps and a Differential Modulated FFN (DM-FFN) for feature rectification, DRP effectively models cross-stage variations with minimal overhead. By focusing computational resources on dynamic evolution rather than static redundancy, DU achieves a superior trade-off between accuracy and efficiency. Extensive experiments verify that our method establishes new state-of-the-art results while significantly slashing computational overhead. https://github.com/Muyuan-Zhang/DU

Chinese Translation

尽管深度展开网络（Deep Unfolding Networks, DUNs）在视频快照压缩成像（Snapshot Compressive Imaging, SCI）中占据主导地位，但它们仍然受到统一设计理念的限制。现有方法重复堆叠具有相同结构的高复杂度先验，忽视了优化轨迹趋向静态状态的事实。这导致了表示停滞，高成本的计算浪费在微小的特征更新上。为了解决这一低效问题，我们提出了差异展开（Differential Unfolding, DU），一个异构框架，用动态演变替代统一重复。DU的核心是差异进化框架（Differential Evolutionary Framework, DEF），它将展开过程划分为两个互补角色：结构锚定和差异演变。在这一方案中，高参数的一般阶段被稀疏部署，以生成高保真特征基础。与之互补的是，轻量级差异阶段采用差异表示先验（Differential Representation Prior, DRP）通过差异机制传播和细化这些基础特征。通过集成差异表示注意力（Differential Representation Attention, DRA）以演变注意力图，以及差异调制前馈网络（Differential Modulated FFN, DM-FFN）用于特征校正，DRP有效地以最小的开销建模跨阶段变化。通过将计算资源集中于动态演变而非静态冗余，DU实现了准确性与效率之间的优越权衡。大量实验验证了我们的方法在显著降低计算开销的同时，建立了新的最先进结果。

View on arXiv Download PDF AI Translation

cs.CV / 32 / 2606.24156

Accelerating Multimodal Large Language Models with Prior-Corrected Token Reduction

通过先验校正的标记减少加速多模态大型语言模型

Chen, Zengjie, Cai, Yuxiang, Guo, Jingcai, Cai, Taotao, Yin, Jianwei, Chen, Zhi

Abstract

Visual token reduction has emerged as an effective strategy for accelerating Multimodal Large Language Models (MLLMs). Many existing methods prune tokens by ranking text-visual attention scores. However, we show that attention is often dominated by a model-induced prior: even without textual instruction, MLLMs tend to focus on certain task-agnostic regions. Consequently, the attention scores of instruction-conditioned tokens are suppressed, increasing the risk that these tokens are discarded during pruning. To address this issue, we propose Prior-Corrected Token Reduction (PriorTR), a training-free token reduction method that explicitly separates task-conditioned attention from the model-induced prior. PriorTR estimates the attention map of the prior, and contrasts it with the task-conditioned attention distribution to measure the additional usable information contributed by each visual token. Importantly, PriorTR computes both the model-induced prior and the task-conditioned posterior within a single forward pass by introducing a null token that serves as an instruction-agnostic probe in the attention block. This design avoids duplicated propagation. Extensive experiments across multiple multimodal benchmarks and MLLMs demonstrate that PriorTR consistently improves the trade-off between accuracy and efficiency over strong training-free baselines, particularly under aggressive token budgets.

Chinese Translation

视觉标记减少已成为加速多模态大型语言模型（MLLMs）的有效策略。许多现有方法通过对文本-视觉注意力分数进行排名来修剪标记。然而，我们表明，注意力往往受到模型引导的先验的主导：即使没有文本指令，MLLMs也倾向于关注某些与任务无关的区域。因此，受指令条件的标记的注意力分数被抑制，增加了在修剪过程中这些标记被丢弃的风险。为了解决这个问题，我们提出了先验校正标记减少（Prior-Corrected Token Reduction，PriorTR），这是一种无训练的标记减少方法，明确将任务条件的注意力与模型引导的先验分开。PriorTR 估计先验的注意力图，并将其与任务条件的注意力分布进行对比，以衡量每个视觉标记所贡献的额外可用信息。重要的是，PriorTR 通过引入一个作为指令无关探针的空标记，在单次前向传播中计算模型引导的先验和任务条件的后验。这一设计避免了重复传播。在多个多模态基准和 MLLMs 上进行的广泛实验表明，PriorTR 在强大的无训练基线之上，始终改善了准确性与效率之间的权衡，特别是在激进的标记预算下。

View on arXiv Download PDF AI Translation

cs.CV / 33 / 2606.24161

Dual-Branch Cross-Projection Debiasing through Diffusion-based Disentanglement

基于扩散解缠的双分支交叉投影去偏差

Zhao, Xiangqian, Jiang, Xinyang, Xu, Zhipeng, He, Lingfeng, Wang, Zilong, Li, Dongsheng, Cheng, De, Wang, Nannan

Abstract

Foundation models trained on biased datasets often rely on spurious correlations between target labels and non-causal attributes, resulting in poor generalization on minority groups. Bias mitigation remains challenging due to two fundamental issues. First, when group labels are unavailable, existing group-unsupervised methods typically infer spurious attributes implicitly from model behavior, making it difficult to identify spurious factors that are semantically aligned with real-world biases. Second, even with pseudo spurious supervision, most existing debiasing methods follow a single-branch design that operates within a single shared feature space, where target and spurious attributes are intrinsically entangled. To address the first challenge, we introduce Confidence-guided Bias Concept Mining (CBCM), which leverages diffusion-disentangled, semantically grounded concept representations to identify reliable spurious attributes without attribute annotations. To address the second challenge, we propose Dual-branch Cross-projection Debiasing (DCD), a prompt-tuning framework that separates target and spurious representations into two branches and explicitly removes spurious information through cross null-space projection while preserving target-relevant semantics. Extensive experiments on four benchmark datasets show that our method achieves state-of-the-art worst group accuracy among group-unsupervised approaches, while tuning at most 0.22% of the model parameters. The source code is available in the supplementary materials.

Chinese Translation

在偏见数据集上训练的基础模型通常依赖于目标标签与非因果属性之间的虚假相关性，这导致其在少数群体上的泛化能力较差。偏见缓解仍然面临两个基本问题。首先，当群体标签不可用时，现有的群体无监督方法通常从模型行为中隐式推断虚假属性，这使得识别与现实世界偏见语义对齐的虚假因素变得困难。其次，即使在伪虚假监督的情况下，大多数现有的去偏差方法遵循单分支设计，在单一共享特征空间内操作，其中目标属性和虚假属性本质上是纠缠在一起的。为了解决第一个挑战，我们引入了基于置信度的偏见概念挖掘（Confidence-guided Bias Concept Mining, CBCM），该方法利用扩散解缠的、语义基础的概念表示来识别可靠的虚假属性，而无需属性注释。为了解决第二个挑战，我们提出了双分支交叉投影去偏差（Dual-branch Cross-projection Debiasing, DCD），这是一种提示调优框架，将目标和虚假表示分离到两个分支中，并通过交叉零空间投影显式去除虚假信息，同时保留与目标相关的语义。在四个基准数据集上的大量实验表明，我们的方法在群体无监督方法中实现了最先进的最差群体准确率，同时最多调优0.22%的模型参数。源代码可在补充材料中获得。

View on arXiv Download PDF AI Translation

cs.CV / 34 / 2606.24165

Spectral Evolution-Guided Token Pruning in Multimodal Large Language Models

基于光谱演变的多模态大型语言模型中的令牌剪枝

Chen, Bin, Cai, Yuxiang, Luo, Yadan, Zhang, Yi, Yin, Jianwei, Chen, Zhi

Abstract

Reducing visual token redundancy is critical for accelerating Multimodal Large Language Models (MLLMs) without degrading cross-modal reasoning performance. Existing token pruning methods typically rely on single-layer signals, such as attention scores or token similarities, which overlook the cross-layer transformation of visual representations and may exhibit positional bias in multimodal token sequences. To address this limitation, we propose a training-free token pruning framework based on Cross-Layer Spectral Evolution (CLSE). Instead of measuring token importance from single-layer feature magnitudes, CLSE quantifies how token representations evolve across Transformer layers in the frequency domain. This evolution reflects the transition from high-frequency structural details to low-frequency semantic abstractions. We observe that tokens with stronger spectral redistribution across layers are more likely to be semantically active and should therefore be preserved. By modeling cross-layer token dynamics, CLSE provides a stable importance criterion that mitigates positional bias. Extensive experiments on both image and video benchmarks demonstrate that CLSE achieves a superior trade-off between efficiency and accuracy under aggressive token reduction. Across multiple MLLMs, CLSE reduces FLOPs, KV cache memory, and latency while maintaining competitive or improved performance.

Chinese Translation

减少视觉令牌冗余对于加速多模态大型语言模型（MLLMs）而不降低跨模态推理性能至关重要。现有的令牌剪枝方法通常依赖于单层信号，如注意力分数或令牌相似性，这忽视了视觉表示的跨层转换，并可能在多模态令牌序列中表现出位置偏差。为了解决这一局限性，我们提出了一种基于跨层光谱演变（Cross-Layer Spectral Evolution, CLSE）的无训练令牌剪枝框架。CLSE并不是从单层特征幅度中测量令牌的重要性，而是量化令牌表示在变换器层之间在频域中的演变。这种演变反映了从高频结构细节到低频语义抽象的过渡。我们观察到，在层之间具有更强光谱重分布的令牌更可能在语义上是活跃的，因此应予以保留。通过建模跨层令牌动态，CLSE提供了一种稳定的重要性标准，减轻了位置偏差。在图像和视频基准上的大量实验表明，CLSE在激进的令牌减少下实现了效率与准确性之间的优越权衡。在多个MLLM中，CLSE减少了浮点运算（FLOPs）、键值缓存内存和延迟，同时保持了竞争力或改进的性能。

View on arXiv Download PDF AI Translation

cs.CV / 35 / 2606.24175

Tri-Efficient Transfer Learning for Point Cloud Videos

三重高效迁移学习用于点云视频

Sun, Yiding, Zhang, Dongxu, Zhu, Jihua, Cheng, Haozhe, Li, Zhengqiao, Li, Pengcheng, Fang, Chaowei, Dong, Yonghao, Chen, Lin

Abstract

While point cloud foundation models have significantly advanced point cloud video understanding, existing parameter-efficient fine-tuning (PEFT) methods still suffer from two critical limitations: prohibitive annotation costs for large-scale point cloud datasets and severe memory bottlenecks. In this paper, we aim to mine richer supervision signals from existing data rather than blindly scaling datasets. A further key principle is that the memory footprint of fine-tuning must be drastically reduced compared to full fine-tuning, which remains elusive for current PEFT techniques. Driven by these challenges, we identify three core desiderata: data-, parameter-, and memory efficiency, and present PoinTriE, a unified framework that excels along all three dimensions. For pre-training, pseudo-motion trajectories are synthesized via rigid transformations, paired with text corpora and 2D projections derived from raw point clouds. We then propose a Geometric-Motion Duality Network optimized via multimodal contrastive learning, rigid rotation prediction, and motion distribution divergence to produce dense self-supervision. During fine-tuning, we freeze the pretrained backbone and only update a lightweight Spatio-temporal Side Network built with LoRA units. Equipped with a gradient flow masking strategy, PoinTriE simultaneously reduces memory consumption and parameter overhead. Extensive experiments confirm that PoinTriE establishes new state-of-the-art results on action recognition and semantic segmentation tasks.

Chinese Translation

尽管点云基础模型在点云视频理解方面取得了显著进展，但现有的参数高效微调（PEFT）方法仍然面临两个关键限制：大规模点云数据集的标注成本高昂和严重的内存瓶颈。本文旨在从现有数据中挖掘更丰富的监督信号，而不是盲目扩展数据集。另一个关键原则是，与完全微调相比，微调的内存占用必须大幅减少，这对于当前的PEFT技术仍然难以实现。针对这些挑战，我们确定了三个核心需求：数据效率、参数效率和内存效率，并提出了PoinTriE，一个在这三个维度上都表现优异的统一框架。在预训练阶段，通过刚性变换合成伪运动轨迹，并与文本语料和从原始点云中提取的2D投影配对。然后，我们提出了一个通过多模态对比学习、刚性旋转预测和运动分布差异优化的几何-运动双重网络，以产生密集的自监督信号。在微调阶段，我们冻结预训练的主干网络，仅更新使用LoRA单元构建的轻量级时空侧网络。借助梯度流掩蔽策略，PoinTriE同时减少了内存消耗和参数开销。大量实验表明，PoinTriE在动作识别和语义分割任务上建立了新的最先进结果。

View on arXiv Download PDF AI Translation

cs.CV / 36 / 2606.24178

Zero-Shot Test-Time Canonicalization using Out-of-Distribution Scoring

基于离散评分的零样本测试时标准化

Lindner, Dominik, Schmidt, Johann, Siegl, Tom, Becker, Martin, Stober, Sebastian

Abstract

Pretrained vision models often misclassify inputs that are rotated, scaled, or sheared, even though these affine transformations leave the object class unchanged. Robustness is usually restored either by building equivariance into the architecture or by retraining with augmentation, both of which require changing or retraining the model. Test-time canonicalization instead leaves the classifier untouched. It undoes the transformation of each input, mapping it to a canonical form near the training distribution before classification. Existing canonicalizers, however, rely on a narrow set of logit-based energy scores and bespoke search procedures, leaving the design space of scoring functions and optimizers unexplored. We reframe canonicalization as out-of-distribution (OOD) detection, which lets any OOD score serve as the energy minimized over transformations. Across benchmarks ranging from handwritten characters and sketches to natural images and 3D point clouds, we systematically evaluate around twenty OOD scores and nine search algorithms, finding that distance-based scores paired with random search and local refinement perform best overall. Because canonicalizing an already-aligned input can hurt accuracy, we add a gated mechanism that transforms an input only when its OOD score indicates this is needed, preserving most in-distribution accuracy while retaining the robustness gains on transformed inputs. Code is available at github.com/johschm/its.

Chinese Translation

预训练的视觉模型通常会错误分类经过旋转、缩放或剪切的输入，尽管这些仿射变换并未改变物体类别。通常，通过在架构中构建等变性或通过数据增强重新训练来恢复鲁棒性，但这两者都需要更改或重新训练模型。而测试时标准化则不改变分类器。它会撤销每个输入的变换，将其映射到接近训练分布的标准形式，然后再进行分类。然而，现有的标准化方法依赖于一组狭窄的基于logit的能量评分和定制的搜索程序，导致评分函数和优化器的设计空间未被探索。我们将标准化重新定义为离散（OOD）检测，这使得任何OOD评分都可以作为在变换中最小化的能量。在从手写字符、草图到自然图像和3D点云的基准测试中，我们系统地评估了约二十种OOD评分和九种搜索算法，发现基于距离的评分与随机搜索和局部优化相结合的表现最佳。由于对已对齐输入进行标准化可能会影响准确性，我们增加了一个门控机制，仅在其OOD评分指示需要时才对输入进行变换，从而在保持大部分分布内准确性的同时，保留了对变换输入的鲁棒性提升。代码可在github.com/johschm/its获取。

View on arXiv Download PDF AI Translation

cs.CV / 37 / 2606.24180

Deep Learning Approaches for 3D Medical Scene Completion: From Geometric Modeling to Generative Paradigms

深度学习方法在三维医学场景补全中的应用：从几何建模到生成范式

Khaled, Afifa, Abdulkadir, Said Jadid, Eltahir, Majdy Mohamed Eltayeb

Abstract

Three-dimensional scene completion has evolved as a major problem in computer vision and robotics, and its applications are diverse, including autonomous navigation and augmented reality. In this study, a systematic review has been conducted to compile the research contributions made in the last ten years, i.e., 2016 to 2026, which has revolutionized the field from the voxel semantic completion paradigm represented by SSCNet to the latest paradigm that combines generative diffusion priors with real-time rendering using a Gaussian splatting technique. The evolution in representation paradigms, such as voxel grids, point learning, implicit neural fields, transformer networks, diffusion networks, and the latest paradigm based on rendering-aware 3D Gaussian primitives, has been discussed in this study. A comprehensive analysis has been carried out on the contributions made in the last ten years, and a taxonomy has been developed to provide a clear idea about the contributions made in the field. The study has also discussed the research contributions made in the field, along with the challenges that still need to be addressed. Finally, the study has presented a research agenda that will provide a clear idea about the directions that can be followed in the development of the next-generation system

Chinese Translation

三维场景补全已成为计算机视觉和机器人领域的一个主要问题，其应用广泛，包括自主导航和增强现实。在本研究中，我们对过去十年（即2016年至2026年）在该领域的研究贡献进行了系统回顾，这些贡献从以SSCNet为代表的体素语义补全范式发展到结合生成扩散先验与使用高斯喷溅技术的实时渲染的最新范式。本文讨论了表示范式的演变，包括体素网格、点学习、隐式神经场、变换器网络、扩散网络以及基于渲染感知的三维高斯原语的最新范式。我们对过去十年所做的贡献进行了全面分析，并制定了一个分类法，以清晰地展示该领域的贡献。此外，研究还讨论了该领域的研究贡献以及仍需解决的挑战。最后，本文提出了一项研究议程，旨在明确下一代系统开发的可行方向。

View on arXiv Download PDF AI Translation

cs.CV / 38 / 2606.24187

Towards Fast and Effective Long Video Understanding of Multimodal Large Language Models via Adaptive Quasi-Gaussian Sampling

通过自适应准高斯采样实现多模态大语言模型的快速有效长视频理解

Zhang, Kun, Fang, Chenxin, Chen, Tao, Song, Baiyang, Shen, Yunhang, Zhou, Yiyi, Ji, Rongrong

Abstract

Long video understanding remains a daunting challenge for \emph{Multimodal Large Language Models} (MLLMs) due to the excessive computation and memory footprint. Thus, \emph{keyframe selection} is often adopted to mitigate this shortcoming, which however still suffers from low flexibility and high noise due to its hard sampling principle. In this paper, we define video frame selection as a problem of \emph{Quasi-Gaussian Sampling}, and propose an adaptive and training-free approach termed \textbf{\emph{AdaQ}}. Inspired by the $3$-$\sigma$ rule of Gaussian distribution, the objective of AdaQ is to achieve the optimal $3$-$\sigma$ interval for different examples, \emph{i.e.}, a smaller $3$-$\sigma$ interval for the local query and a larger one for the global query, thereby facilitating robust and adaptive frame sampling. To validate AdaQ, we apply it to four MLLMs with three embedding models. The extensive experimental results not only show its obvious performance gains over the default MLLMs and the SOTA keyframe selection methods, \emph{e.g.}, helping Qwen3-VL-8B outperform GPT4o by 15.8\% on average by using only 64 frames, but also confirm its superior robustness and high efficiency for long-video understanding, \emph{e.g.}, \textbf{only 1 hyper-parameter} needs to be set. \textbf{Our code project} is given at \href{https://github.com/Zkayovo-xmu/AdaQ}{https://github.com/Zkayovo-xmu/AdaQ}.

Chinese Translation

长视频理解对于多模态大语言模型（MLLMs）仍然是一个艰巨的挑战，因为其计算和内存开销过大。因此，通常采用关键帧选择来缓解这一缺陷，但由于其硬采样原则，仍然面临灵活性不足和噪声过高的问题。本文将视频帧选择定义为准高斯采样问题，并提出了一种自适应且无需训练的方法，称为AdaQ。受高斯分布的3-σ法则启发，AdaQ的目标是为不同示例实现最佳的3-σ区间，即为局部查询提供较小的3-σ区间，为全局查询提供较大的3-σ区间，从而促进稳健和自适应的帧采样。为了验证AdaQ，我们将其应用于四个MLLMs和三个嵌入模型。大量实验结果不仅显示出其在默认MLLMs和最先进的关键帧选择方法上的显著性能提升，例如，帮助Qwen3-VL-8B在仅使用64帧的情况下，平均超越GPT4o 15.8%，还确认了其在长视频理解中的卓越鲁棒性和高效率，例如，仅需设置1个超参数。我们的代码项目可在此获取： [https://github.com/Zkayovo-xmu/AdaQ](https://github.com/Zkayovo-xmu/AdaQ)。

View on arXiv Download PDF AI Translation

cs.CV / 39 / 2606.24192

Co-occurring associated retained concepts in Diffusion Unlearning

扩散反学习中的共现关联保留概念

Kim, Miso, Lee, Georu, Kim, Yunji, Kim, Hoki, Park, Jinseong, Lee, Woojin

Abstract

Unlearning has emerged as a key technique to mitigate harmful content generation in diffusion models. However, existing methods often remove not only the target concept, but also benign co-occurring concepts. As illustrated in Fig.1, unlearning nudity can unintentionally suppress the concept of person, preventing a model from generating images with person. We define these undesirably suppressed co-occurring concepts that must be preserved CARE (Co-occurring Associated REtained concepts). Then, we introduce the CARE score, a general metric that directly quantifies their preservation across unlearning tasks. With this foundation, we propose ReCARE (Robust erasure for CARE), a framework that explicitly safeguards CARE while erasing only the target concept. ReCARE automatically constructs the CARE-set, a curated vocabulary of benign co-occurring tokens extracted from target images, and leverages this vocabulary during training for stable unlearning. Extensive experiments across various target concepts (Nudity, Van Gogh style, and Tench object) demonstrate that ReCARE achieves overall state-of-the-art performance in balancing robust concept erasure, overall utility, and CARE preservation.

Chinese Translation

反学习已成为减轻扩散模型中有害内容生成的关键技术。然而，现有方法往往不仅移除目标概念，还会删除良性的共现概念。如图1所示，反学习裸体概念可能无意中抑制“人”这一概念，导致模型无法生成包含“人”的图像。我们将这些不应被抑制的共现概念定义为CARE（Co-occurring Associated REtained concepts）。接着，我们引入CARE分数，这是一种通用指标，直接量化它们在反学习任务中的保留情况。在此基础上，我们提出了ReCARE（Robust erasure for CARE），一个明确保护CARE的框架，同时仅移除目标概念。ReCARE自动构建CARE集合，这是从目标图像中提取的良性共现标记的策划词汇，并在训练过程中利用该词汇以实现稳定的反学习。在针对多种目标概念（裸体、梵高风格和十鳃鲷对象）进行的广泛实验中，ReCARE在平衡稳健的概念删除、整体效用和CARE保留方面实现了整体的最先进性能。

View on arXiv Download PDF AI Translation

cs.CV / 40 / 2606.24206

Inclusive Interactive Collisions for Multi-View Consistent Compositional 3D Generation

用于多视图一致性组合3D生成的包容性交互碰撞

Liu, Chang, Shao, Mingwen, Lv, Xiang, Chen, Xinyuan, Meng, Lingzhuang, Zhang, Qiao, Gong, Zhengyi, Hu, Jinghao

Abstract

Recent breakthroughs in 3D generation have advanced notably with the development of text-to-image diffusion model. However, existing methods remain two practical challenges: (1) They primarily generate single 3D object, but struggle to generate multi-object compositional 3D assets due to the lack of the modeling for Gaussian primitives in reasonable interactions. (2) They often suffer from cross-view inconsistency during 3D optimization, as Score Distillation Sampling inherently performs on each single view, inevitably resulting in cross-view hallucinations. To solve above issues, we propose I2C-3D, a novel optimization-based method to generate multi-view consistent compositional 3D assets with reasonable interactions. Specifically, we propose an Inclusive Interactive Collisions strategy to guide Gaussian primitives appearing in reasonable interaction regions naturally, thereby ensuring objects in the compositional scene interact in a physically plausible and visually coherent way. Additionally, to enhance multi-view consistency, Multi-View Adaptive Score Distillation Sampling is devised to distill multi-view consistency prior and layout prior from pre-trained diffusion model by modulating attention map of instance token and spatial token across viewpoints. Benefiting from above elaborate designs, I2C-3D not only generates high-fidelity multi-view consistent compositional 3D assets but also supports 3D editing flexibly, facilitating complex scene generation. Extensive experiments demonstrate our I2C-3D outperforms existing methods in generation quality and multi-view consistency.

Chinese Translation

最近，随着文本到图像扩散模型的发展，3D生成取得了显著突破。然而，现有方法仍面临两个实际挑战：（1）它们主要生成单一3D对象，但由于缺乏对合理交互中高斯原语的建模，难以生成多对象组合的3D资产。（2）在3D优化过程中，它们常常遭遇跨视图不一致的问题，因为Score Distillation Sampling本质上是在每个单一视图上进行，必然导致跨视图的幻觉。为了解决上述问题，我们提出了I2C-3D，这是一种基于优化的新方法，旨在生成具有合理交互的多视图一致性组合3D资产。具体而言，我们提出了一种包容性交互碰撞策略，以自然引导出现在合理交互区域中的高斯原语，从而确保组合场景中的对象以物理上合理和视觉上连贯的方式进行交互。此外，为了增强多视图一致性，我们设计了多视图自适应得分蒸馏采样（Multi-View Adaptive Score Distillation Sampling），通过调节实例标记和空间标记在不同视点上的注意力图，从预训练的扩散模型中提取多视图一致性先验和布局先验。得益于上述精心设计，I2C-3D不仅生成高保真度的多视图一致性组合3D资产，还灵活支持3D编辑，促进复杂场景的生成。大量实验表明，我们的I2C-3D在生成质量和多视图一致性方面优于现有方法。

View on arXiv Download PDF AI Translation

cs.CV / 41 / 2606.24214

MorVess: Morphology-Aware Pulmonary Vessel Segmentation Network

MorVess：形态感知的肺血管分割网络

Mao, Fuyou, Chen, Yifei, Wu, Beining, Lin, Lixin, Dai, Jinnan, Li, Zhiling, Chen, Yilei, Wang, Yaqi, Zhang, Hao, Tang, Yan, Zhou, Huiyu, Qin, Feiwei

Abstract

Accurate pulmonary vessel segmentation remains challenging due to the sparse, tortuous, and multi-scale nature of vascular structures, where small branches are easily lost and topology integrity is difficult to preserve under voxel-wise supervision. Existing deep segmentation models primarily optimize binary masks, lacking explicit geometric constraints, thus struggling to recover continuous tubular morphology and fine vascular connectivity. In this study, we introduce MorVess, a morphology-aware segmentation framework that integrates differentiable geometric priors with large-scale foundation model adaptation to achieve fine-grained vascular parsing. MorVess jointly predicts vessel masks, distance maps, and thickness maps, providing explicit supervision for vascular boundaries, centerline consistency, and smooth diameter transitions. A lightweight 2.5D adapter bridges 3D spatial context and 2D SAM representations, while a global-local fusion block aggregates multi-level semantics and geometric cues for high-fidelity topology reconstruction. Across two challenging pulmonary CT benchmarks, MorVess delivers superior Dice, clDice, and HD95 scores, substantially improving small-vessel recovery and global connectivity. These results demonstrate that embedding geometric intelligence into pretrained vision models offers a principled and scalable pathway toward precise vessel analysis and clinically reliable structural quantification. Our source code is available at https://github.com/MaoFuyou/MorVess.

Chinese Translation

由于血管结构的稀疏性、曲折性和多尺度特性，准确的肺血管分割仍然面临挑战，其中小分支容易丢失，且在体素级监督下保持拓扑完整性困难。现有的深度分割模型主要优化二进制掩膜，缺乏明确的几何约束，因此在恢复连续的管状形态和细致的血管连通性方面存在困难。在本研究中，我们提出了MorVess，一种形态感知的分割框架，结合可微分的几何先验与大规模基础模型适应，以实现细粒度的血管解析。MorVess联合预测血管掩膜、距离图和厚度图，为血管边界、中心线一致性和光滑直径过渡提供明确的监督。一个轻量级的2.5D适配器连接了3D空间上下文和2D SAM表示，而全局-局部融合模块则聚合多层次的语义和几何线索，以实现高保真度的拓扑重建。在两个具有挑战性的肺部CT基准测试中，MorVess在Dice、clDice和HD95分数上表现优越，显著改善了小血管的恢复和全局连通性。这些结果表明，将几何智能嵌入预训练视觉模型为精确的血管分析和临床可靠的结构量化提供了一条有原则且可扩展的路径。我们的源代码可在 https://github.com/MaoFuyou/MorVess 获取。

View on arXiv Download PDF AI Translation

cs.CV / 42 / 2606.24225

Geometry-Instructed Video Editing

几何指导的视频编辑

Chang, Chirui, Lyu, Xiaoyang, Huang, Yi-Hua, Tan, Haoru, Zhao, Shizhen, Ding, Yikang, Bao, Jianmin, Tao, Xin, Wan, Pengfei, Qi, Xiaojuan

Abstract

Object-level geometric edits, including translating, rotating, scaling, duplicating, or removing an object, are routine operations in digital content creation (DCC) workflows, yet they remain unreliable in generative video editing. The key challenge lies in specifying the target object's 3D state change unambiguously across viewpoint and time, while consistently updating geometry-dependent secondary effects such as shadows and reflections. We introduce GIVE, a geometry-instructed video editing framework that represents edits through a unified object-state formulation. Two video-aligned geometry streams describe the target object before and after editing: a depth-box encoding coarse 3D placement and extent, and an orientation-box providing an appearance-agnostic orientation cue. Together, these streams provide a compact pre/post geometric specification for object-state transitions. To provide paired supervision for learning these edits, we build a scalable graphics-engine pipeline that executes object-level edit programs and renders controlled before/after pairs, isolating the intended geometric edit while keeping secondary effects consistent with the transformation. Experimental results demonstrate that GIVE produces faithful geometric edits with temporal coherence and consistent secondary effects across operators in a unified framework, and shows promising transfer to in-the-wild videos. Project page: https://geometry-instructed-video-editing.github.io/give/

Chinese Translation

对象级几何编辑，包括平移、旋转、缩放、复制或移除对象，是数字内容创作（DCC）工作流程中的常规操作，然而在生成视频编辑中仍然不够可靠。关键挑战在于如何在视角和时间上明确指定目标对象的三维状态变化，同时持续更新依赖几何的次级效果，如阴影和反射。我们提出了GIVE，一个几何指导的视频编辑框架，通过统一的对象状态表述来表示编辑。两个与视频对齐的几何流描述了编辑前后目标对象的状态：一个深度框（depth-box）编码了粗略的三维位置和范围，一个方向框（orientation-box）提供了与外观无关的方向线索。这两个流共同提供了对象状态转变的紧凑的前/后几何规范。为了为学习这些编辑提供配对监督，我们构建了一个可扩展的图形引擎管道，执行对象级编辑程序并渲染受控的前后对，隔离预期的几何编辑，同时保持次级效果与变换的一致性。实验结果表明，GIVE在统一框架中产生了忠实的几何编辑，具有时间一致性和跨操作的一致次级效果，并显示出对真实视频的良好迁移能力。项目页面：https://geometry-instructed-video-editing.github.io/give/

View on arXiv Download PDF AI Translation

cs.CV / 43 / 2606.24232

FiCA: Feed-forward instant Gaussian Codec Avatars from a Single Portrait Image

FiCA：从单幅肖像图像生成前馈即时高斯编解码头像

Youwang, Kim, Yang, Zhengyu, Ge, Liuhao, Rong, Yu, Bagautdinov, Timur, Zhaoen, Su, Sopher, Nir, Popović, Jovan, Deng, Teng, Oh, Tae-Hyun, Cao, Chen

Abstract

We introduce FiCA, a Feed-forward, instant Gaussian Codec Avatar generation pipeline that creates lifelike avatars from a single portrait image. Generating a photorealistic and drivable avatar from just a single image is significantly challenging due to the limited visual information available to accurately infer the 3D appearance and geometry of human heads. To address this, we develop a novel system that combines human-centric vision foundation models with a diffusion model. This system is designed to fully exploit partial visual observations to generate lifelike human avatars. Our proposed diffusion model learns a generative mapping from these partial observations to complete and authentic 3D mesh reconstruction. Additionally, we introduce a feed-forward mesh refinement network that enhances the fidelity and identity preservation of the generated avatars, eliminating the need for person-specific test-time optimization. By leveraging a universal prior model that decodes a generated mesh into a set of 3D Gaussians, we generate a photorealistic 3D Gaussian avatar, capable of being driven with novel expressions in real-time. Our experiments demonstrate that the avatars generated by our feed-forward approach faithfully represent diverse identities and surpass the visual quality of avatars produced by recent competing methods.

Chinese Translation

我们介绍了FiCA，一种前馈即时高斯编解码头像生成管道，该管道能够从单幅肖像图像创建逼真的头像。仅凭一幅图像生成逼真且可驱动的头像具有显著挑战性，因为可用的视觉信息有限，难以准确推断人头的三维外观和几何形状。为了解决这一问题，我们开发了一种新颖的系统，将以人为中心的视觉基础模型与扩散模型相结合。该系统旨在充分利用部分视觉观测，以生成逼真的人类头像。我们提出的扩散模型学习了从这些部分观测到完整且真实的三维网格重建的生成映射。此外，我们引入了一种前馈网格细化网络，增强生成头像的保真度和身份保留，消除了对特定个体测试时优化的需求。通过利用一种通用先验模型，将生成的网格解码为一组三维高斯，我们生成了一个逼真的三维高斯头像，能够实时驱动新颖的表情。我们的实验表明，我们的前馈方法生成的头像忠实地代表了多样的身份，并超越了近期竞争方法生成的头像的视觉质量。

View on arXiv Download PDF AI Translation

cs.CV / 44 / 2606.24233

Latent Visual States for Efficient Multimodal Reasoning

高效多模态推理的潜在视觉状态

Chen, Xiuwei, Hu, Wentao, Wang, Yongxin, Chen, Zisheng, Zhang, Likui, Xiang, Kun, Han, Jianhua, Zhen, Hui-Ling, Zou, Jingyuan, Xu, Hang, Liang, Xiaodan

Abstract

The integration of visual evidence has significantly enhanced the capabilities of large multimodal models. However, this integration predominantly relies on generating discrete outputs (etc., code or box coordinates) to invoke external tools, a process that introduces rigid dependencies and substantial latency. To overcome these limitations, we propose {EVA} (LatEnt Visual StAtes), a novel framework that natively generates continuous latent visual representations. These internal representations manifest as an adaptive sequence of Latent\_slot tokens, serving as intermediate visual thoughts during the reasoning process. These Latent\_slot tokens are then trained end-to-end with the discrete text tokens. This co-optimization, notably, causes extreme policy deviation in the 'transition window' following the Latent\_slot tokens. We develop D-GSPO (Decouple-GSPO) to target this root cause by decoupling the optimization of latent and discrete components. To support SFT, we construct EVA-230K, a high-quality text-image interleaved CoT dataset encompassing a diverse range of real-world scenes, documents, charts and OCR tasks. Extensive experiments across multiple benchmarks confirm that EVA achieves significant performance gains while enhancing inference efficiency.

Chinese Translation

视觉证据的整合显著增强了大型多模态模型的能力。然而，这种整合主要依赖于生成离散输出（例如，代码或框坐标）以调用外部工具，这一过程引入了严格的依赖关系和显著的延迟。为克服这些限制，我们提出了{EVA}（潜在视觉状态），这是一个新颖的框架，能够原生生成连续的潜在视觉表示。这些内部表示表现为一系列自适应的Latent_slot令牌，作为推理过程中的中间视觉思维。这些Latent_slot令牌随后与离散文本令牌进行端到端训练。这种共同优化显著导致在Latent_slot令牌之后的“过渡窗口”中出现极端的策略偏差。我们开发了D-GSPO（解耦-GSPO）来针对这一根本原因，通过解耦潜在和离散组件的优化。为了支持SFT，我们构建了EVA-230K，这是一个高质量的文本-图像交错的CoT数据集，涵盖了多样化的真实场景、文档、图表和OCR任务。对多个基准的广泛实验确认，EVA在提高推理效率的同时实现了显著的性能提升。

View on arXiv Download PDF AI Translation

cs.CV / 45 / 2606.24234

From Open Waters to Enclosed Cabins: ProteusVPR for Cross-Scene Visual Place Recognition in Maritime Perception and Cabin Inspection

从开放水域到封闭舱室：用于海洋感知和舱室检查的跨场景视觉位置识别的ProteusVPR

Chena, Zexi, Huang, Zitai, Gu, Qiwen, Li, Zhiqi, Dong, Shengli, Wang, Chenlei, Zhao, Junqiao, Wang, Hongdong, Han, Bing

Abstract

Autonomous robotic inspection in maritime environments presents unique challenges for Visual Place Recognition (VPR) due to cross-scene perceptual shifts. Robots navigating ship-borne environments must transition between visually distinct domains: open decks with sparse textures and severe illumination changes, and enclosed cabins with repetitive structures and high visual ambiguity. Existing VPR methods, designed primarily for urban or indoor scenes, fail to generalize reliably across these starkly different scenarios. To address this, we propose ProteusVPR, a two-stage retrieval-refinement framework. The first stage employs any standard VPR model for initial image retrieval. The second stage introduces a geometric-visual estimation network that fuses the retrieved image with two temporally preceding frames, incorporating geometric descriptors, a local affine coordinate system, and camera azimuth encoding to achieve precise localization. To support this task, we introduce the XHZ dataset, an 8K-panoramic ship-borne dataset collected from an operational vessel, featuring multi-floor cabin structures, deck transition zones, and strict query-database separation for rigorous evaluation. Extensive experiments on the XHZ dataset demonstrate that ProteusVPR consistently improves the localization accuracy across multiple VPR backbones, reducing mean localization error by over 60\% on average and that ProteusVPR offers an effective and robust solution for precise visual localization in challenging, cross-scene maritime environments.

Chinese Translation

在海洋环境中，自主机器人检查面临着视觉位置识别（VPR）的独特挑战，因为存在跨场景的感知变化。导航于船载环境中的机器人必须在视觉上截然不同的领域之间进行转换：开放甲板上纹理稀疏且光照变化剧烈，以及封闭舱室中结构重复且视觉模糊度高。现有的VPR方法主要针对城市或室内场景设计，无法在这些截然不同的场景中可靠地推广。为了解决这一问题，我们提出了ProteusVPR，一个两阶段的检索-精炼框架。第一阶段采用任何标准VPR模型进行初始图像检索。第二阶段引入一个几何-视觉估计网络，将检索到的图像与两个时间上先前的帧融合，结合几何描述符、本地仿射坐标系统和相机方位编码，以实现精确定位。为了支持这一任务，我们引入了XHZ数据集，这是一个从操作船舶收集的8K全景船载数据集，包含多层舱室结构、甲板过渡区域，并严格区分查询与数据库以进行严格评估。在XHZ数据集上的大量实验表明，ProteusVPR在多个VPR骨干网络中始终提高了定位精度，平均降低了超过60%的定位误差，并且ProteusVPR为在具有挑战性的跨场景海洋环境中实现精确视觉定位提供了有效且稳健的解决方案。

View on arXiv Download PDF AI Translation

cs.CV / 46 / 2606.24248

M^2C-EvDet: Multi-Domain Multi-Order Cross-Modal Knowledge Distillation for Event-based Object Detection

M^2C-EvDet：用于事件驱动目标检测的多领域多阶跨模态知识蒸馏

Bao, Wei, Li, Siqi, Pan, Shouan, Xie, Yi, Gao, Yue

Abstract

Event-based object Detection (EvDet), as a biologically inspired visual perception paradigm, demonstrates superior performance in scenarios demanding high temporal resolution and a wide dynamic range. Nevertheless, the inherent sparse representations and inadequate visual semantics of event data result in a considerable performance disparity between EvDet and frame-based object detection. Previous works attempt to alleviate this cross-modal discrepancy through knowledge distillation, yet they only focus on spatial visual semantics or pair-wise relational information, thus limiting performance in more complex scenarios. To address this challenge, this paper proposes M^2C-EvDet, a Multi-domain and Multi-order Cross-modal knowledge distillation framework for EvDet. Built upon frequency learning and hypergraph computation, M^2C-EvDet integrates two specialized modules: Adaptive Frequency-Decoupled Feature Distillation (AF^2D^2) and Multi-Order Relational Distillation (MORD).

Chinese Translation

事件驱动目标检测（EvDet）作为一种受生物启发的视觉感知范式，在需要高时间分辨率和广泛动态范围的场景中表现出优越的性能。然而，事件数据固有的稀疏表示和不足的视觉语义导致EvDet与基于帧的目标检测之间存在显著的性能差距。以往的研究试图通过知识蒸馏来缓解这种跨模态差异，但它们仅关注空间视觉语义或成对关系信息，从而限制了在更复杂场景中的性能。为了解决这一挑战，本文提出了M^2C-EvDet，一个用于EvDet的多领域多阶跨模态知识蒸馏框架。M^2C-EvDet基于频率学习和超图计算，集成了两个专门模块：自适应频率解耦特征蒸馏（AF^2D^2）和多阶关系蒸馏（MORD）。

View on arXiv Download PDF AI Translation

cs.CV / 47 / 2606.24253

TuringViT: Making SOTA Vision Transformers Accessible to All

TuringViT：让最先进的视觉变换器对所有人可及

Wu, Qiman, Chen, Hanlin, Chen, Lyujie, Xin, Rui, Zheng, Jianlei, Wang, Mingyuan, Hu, Jiahui, Zhu, Da, Ma, Yuecheng, Wei, Yuhua, Wang, Yizhao, Zhou, Hua, Zhang, Yuheng, Liu, Anhua, Tang, Shaman, He, Yue, Diao, Pengfei, Su, Shuang, Xin, Haotong, Huang, Weichao, Zhang, Hang, Liu, Xianming

Abstract

Modern VLMs and VLA systems commonly adopt off-the-shelf ViTs such as SigLIP2 as visual encoders, but diverse downstream requirements in latency, temporal modeling, and VLM integration often call for customized SOTA-level ViTs. Training such encoders remains beyond the reach of much of the community, as it requires massive image-text data, while standard softmax attention makes high-resolution or dynamic-resolution pretraining prohibitively costly and often forces low-resolution pretraining followed by post-hoc adaptation. TuringViT addresses these challenges with three key designs: Turing Linear Attention (TLA) for efficient sequence modeling, VISTA-Curation to construct supervision-rich image-video training data, and native dynamic-resolution pretraining that supports flexible inputs from the start and transfers seamlessly to downstream VLMs. As a result, TuringViT outperforms leading open-source ViT baselines with only 10% of the data, achieves stronger downstream VLM performance, and delivers substantially better latency scaling on high-resolution inputs. Our scaling-law analysis further shows that TuringViT continues to improve predictably with curated data scale, far from saturation. Its fast adaptation, hardware-friendly design, and efficient deployment have made it a unified visual foundation across XPeng's AI systems. More broadly, TuringViT provides a reproducible pipeline that dramatically lowers the cost for the community to train, customize, and deploy SOTA-level ViTs, moving toward making such Vision Transformers accessible to all.

Chinese Translation

现代视觉语言模型（VLMs）和视觉语言处理系统（VLA）通常采用现成的视觉变换器（ViTs），例如 SigLIP2 作为视觉编码器，但在延迟、时间建模和 VLM 集成等多样化下游需求下，往往需要定制的最先进（SOTA）级别的 ViTs。训练这样的编码器对许多研究者来说仍然是一个挑战，因为这需要大量的图像-文本数据，而标准的 softmax 注意力机制使得高分辨率或动态分辨率的预训练成本过高，通常迫使采用低分辨率预训练，然后进行后期适应。TuringViT 通过三项关键设计来解决这些挑战：用于高效序列建模的 Turing 线性注意力（TLA）、用于构建丰富监督图像-视频训练数据的 VISTA-Curation，以及支持灵活输入的原生动态分辨率预训练，能够无缝转移到下游 VLMs。因此，TuringViT 在仅使用 10% 数据的情况下超越了领先的开源 ViT 基线，取得了更强的下游 VLM 性能，并在高分辨率输入上显著改善了延迟扩展。我们的扩展法则分析进一步表明，TuringViT 随着 curated 数据规模的增加而可预测地持续改进，远未达到饱和。其快速适应、硬件友好的设计和高效部署使其成为 XPeng AI 系统中的统一视觉基础。更广泛地说，TuringViT 提供了一个可重复的流程，显著降低了社区训练、定制和部署 SOTA 级别 ViTs 的成本，朝着让所有人都能使用这样的视觉变换器的目标迈进。

View on arXiv Download PDF AI Translation

cs.CV / 48 / 2606.24255

Social Structure Matters in 3D Human-Human Interaction Generation

社会结构在三维人际互动生成中的重要性

Wang, Zhongju, Wang, Beier, Bian, Yatao, Wang, Pichao, Wang, Zhi, Dong, Daoyi, Li, Hongdong, Mo, Huadong, Sun, Zhenhong

Abstract

Although text-to-motion generation has achieved strong progress in synthesizing realistic single-person motions from language, extending it to text-driven 3D human-human interaction (HHI) remains non-trivial, as HHI requires modeling the underlying \textbf{social structure} that governs phase progression, actor roles, and inter-actor coordination. In this paper, we formulate HHI generation as a social structure modeling and grounding problem: the model must first infer how an interaction unfolds and how the two actors coordinate their roles, and then realize this structure as continuous, physically plausible, and partner-aware 3D motion. To study how such structure should be modeled, we first examine the capability boundary of large language models (LLMs) for HHI generation. Our analysis shows that LLMs can \textit{think} by recovering phase decompositions and partner-aware roles, but cannot directly \textit{move}, as they fail to generate dynamic, physically plausible, and interaction-aware motion. This motivates our planner-executor paradigm, \textbf{Think with LLM, Move with Motion Skill}. The LLM planner converts implicit interaction semantics into motion-aligned social supervision by decomposing interactions into phases, assigning partner-aware actor roles, and aligning them with motion sequence. The motion executor then grounds the planned social structure into coordinated two-person motion by adapting a pretrained solo motion model with LoRA, previous-phase self-conditioning, and ego-relative partner conditioning. Together, our Solo-to-Social framework bridges social organization and motion realization, producing 3D HHI with improved phase consistency, role alignment, and partner-aware coordination.

Chinese Translation

尽管文本到动作生成在从语言合成逼真的单人动作方面取得了显著进展，但将其扩展到基于文本的三维人际互动（HHI）仍然不是一件简单的事情，因为HHI需要建模支配阶段进展、参与者角色和参与者间协调的基础社会结构。在本文中，我们将HHI生成形式化为一个社会结构建模与基础问题：模型必须首先推断互动是如何展开的，以及两个参与者如何协调他们的角色，然后将这一结构实现为连续的、物理上合理的、并且关注伙伴的三维动作。为了研究这种结构应如何建模，我们首先考察大型语言模型（LLMs）在HHI生成中的能力边界。我们的分析表明，LLMs可以通过恢复阶段分解和关注伙伴的角色来“思考”，但无法直接“移动”，因为它们无法生成动态的、物理上合理的、并且关注互动的动作。这激发了我们的规划者-执行者范式，即“用LLM思考，用动作技能移动”。LLM规划者通过将互动分解为阶段、分配关注伙伴的参与者角色并将其与动作序列对齐，将隐式互动语义转换为与动作对齐的社会监督。然后，动作执行者通过使用LoRA、前一阶段自我调节和自我相对伙伴调节，将规划的社会结构基础化为协调的双人动作，适配预训练的单人动作模型。我们的Solo-to-Social框架将社会组织与动作实现连接起来，生成具有更好阶段一致性、角色对齐和伙伴关注协调的三维人际互动。

View on arXiv Download PDF AI Translation

cs.CV / 49 / 2606.24256

Trimming the Long-Tail of Visual World Modeling Evaluation

修剪视觉世界建模评估的长尾

Li, Bingxuan, Hong, Yining, Qian, Cheng, Ha, Hyeonjeong, Liu, Jiateng, Wang, Zhenhailong, Guo, Yue, Li, Yunzhu, Ji, Heng

Abstract

Physical interactions follow a long-tailed distribution: a set of common and regular interactions dominates human experience and visual data, while a broad spectrum of rare and irregular interactions remains underrepresented. Although recent visual world models, including image and video generation models, achieve impressive realism on existing benchmarks, they primarily focus on simulating common physical interactions. This raises a central question: Do current visual world models internalize and generalize physical principles? In this work, we introduce Tailor-Bench, a benchmark that challenges world models to simulate irregular physical interactions. To enable systematic evaluation, we design three scenario modes that progressively challenge model reasoning: Regular scenarios reflect common tool-task pairs, Unconventional scenarios replace conventional tools with attribute-compatible substitutes to test affordance generalization, and Impossible scenarios introduce attribute-violating tools to probe constraint awareness. Additionally, we design two complementary settings under a unified evaluation protocol: predictive generation requires inferring outcomes without guidance, while descriptive generation specifies the target outcome for faithful realization. Our experimental results reveal a clear long-tail gap in physical world modeling: performance degrades from Regular to Unconventional and Impossible scenarios, indicating limited generalization beyond common interactions. Failure analysis further shows that models rely on superficial visual patterns: image models fail to realize correct state changes, while video models further suffer from temporal inconsistencies.

Chinese Translation

物理交互遵循长尾分布：一组常见且规律的交互主导着人类经验和视觉数据，而一系列稀有且不规则的交互则未得到充分代表。尽管最近的视觉世界模型，包括图像和视频生成模型，在现有基准测试中取得了令人印象深刻的真实感，但它们主要集中于模拟常见的物理交互。这引发了一个核心问题：当前的视觉世界模型是否内化并概括了物理原则？在本研究中，我们引入了Tailor-Bench，一个挑战世界模型模拟不规则物理交互的基准。为了实现系统评估，我们设计了三种情境模式，逐步挑战模型推理：常规情境反映常见的工具-任务对；非常规情境用属性兼容的替代品替换传统工具，以测试可供性概括；而不可能情境则引入违反属性的工具，以探测约束意识。此外，我们在统一评估协议下设计了两个互补设置：预测生成要求在没有指导的情况下推断结果，而描述生成则指定目标结果以实现真实再现。我们的实验结果揭示了物理世界建模中的明显长尾差距：从常规到非常规和不可能情境，性能逐渐下降，表明在常见交互之外的概括能力有限。失败分析进一步表明，模型依赖于表面的视觉模式：图像模型未能实现正确的状态变化，而视频模型则进一步受到时间不一致性的影响。

View on arXiv Download PDF AI Translation

cs.CV / 50 / 2606.24257

3DCarGen: Scalable 3D Car Generation via 3D-consistent Multi-view Synthesis

3DCarGen：通过3D一致的多视图合成实现可扩展的3D汽车生成

Xiao, Hongli, Zhang, Youjian, Jin, Yaohui, Ren, Xiaoguang, Yang, Wenjing, Lan, Long

Abstract

High-quality 3D vehicle assets are essential for autonomous driving simulation. Although multi-view diffusion-based paradigms enable controllable single-image reconstruction, they typically produce limited viewpoints and exhibit cross-view geometric inconsistencies, thereby reducing reconstruction fidelity in real-world scenarios. In this work, we introduce 3DCarGen, a scalable single-view 3D car generation framework designed for real-world images by synthesizing an arbitrary number of 3D-consistent multi-view images. Specifically, given a single image as input, we first synthesize a set of images from fixed viewpoints. These images are then fed into a feed-forward reconstruction model, resulting in a coarse 3D representation based on 3D Gaussian Splatting. Conditioned on this explicit 3D prior, our multi-view diffusion model generates 3D-consistent images from arbitrary camera viewpoints. We further extend a fast mesh reconstruction algorithm by incorporating color-normal joint optimization to recover detailed and coherent 3D vehicle models from the synthesized dense views. Extensive experiments on synthetic and real-world datasets demonstrate that our approach achieves robust geometric consistency and reconstruction fidelity compared to existing methods. Code and models will be released.

Chinese Translation

高质量的3D车辆资产对于自动驾驶模拟至关重要。尽管基于多视图扩散的范式能够实现可控的单图像重建，但它们通常只产生有限的视角，并且存在跨视角几何不一致性，从而降低了在真实场景中的重建保真度。在本研究中，我们提出了3DCarGen，一个可扩展的单视图3D汽车生成框架，旨在通过合成任意数量的3D一致的多视图图像来处理真实世界图像。具体而言，给定一张单图像作为输入，我们首先从固定视角合成一组图像。这些图像随后被输入到前馈重建模型中，生成基于3D高斯点云的粗略3D表示。在此显式3D先验的条件下，我们的多视图扩散模型能够从任意相机视角生成3D一致的图像。我们进一步通过结合颜色-法线联合优化扩展了一种快速网格重建算法，以从合成的密集视图中恢复详细且一致的3D车辆模型。在合成和真实世界数据集上的大量实验表明，我们的方法在几何一致性和重建保真度方面优于现有方法。代码和模型将会发布。

View on arXiv Download PDF AI Translation

cs.CV / 51 / 2606.24263

MotifGen: Spatiotemporal interpolation of misaligned satellite images via multi-source generative modeling, in an application to tropical cyclones

MotifGen：通过多源生成建模对错位卫星图像进行时空插值，应用于热带气旋

Dauvilliers, Clément, Monteleoni, Claire

Abstract

Microwave satellite imagery plays a crucial role in monitoring tropical cyclone precipitation and intensity worldwide, but suffers from long revisit times, potentially missing rapid storm evolution phases. While this raises the need for an interpolation method, it is made challenging by the high level of heterogeneity of microwave data coming from different instruments. In this work, we introduce the first generative model that can be applied to multiple geospatial sources that change across samples, occur at irregular time intervals, are misaligned geographically, and come from instruments with varying characteristics. We apply this model to the case of spatio-temporal interpolation of tropical cyclone microwave images from other microwave and infrared instruments. We train using a self-supervised task in which a random source is masked and reconstructed, and show that it leads to a significant decrease in Continuous Ranked Probability Score over supervised training. We show a further improvement by combining infrared and microwave data compared to microwave only. Using these improvements, the generative model produces an ensemble mean on par with that of a deterministic model, while generating a power spectrum significantly closer to that of true observations. To the best of our knowledge, this is the first generative model that interpolates microwave images of cyclones by combining multiple microwave instruments and infrared observations at irregular time intervals.

Chinese Translation

微波卫星影像在全球监测热带气旋降水和强度方面发挥着至关重要的作用，但由于重访时间较长，可能会错过快速风暴演变阶段。这就提出了对插值方法的需求，但由于来自不同仪器的微波数据具有高度异质性，这使得插值变得具有挑战性。在本研究中，我们介绍了第一个可以应用于多个地理空间源的生成模型，这些源在样本间发生变化、以不规则时间间隔出现、地理上错位，并且来自具有不同特性的仪器。我们将该模型应用于其他微波和红外仪器的热带气旋微波图像的时空插值案例。我们使用自监督任务进行训练，其中随机源被掩蔽并重建，结果表明这导致了连续排名概率评分（Continuous Ranked Probability Score）相较于监督训练显著降低。通过结合红外和微波数据，相较于仅使用微波数据，我们进一步提高了性能。利用这些改进，生成模型产生的集成均值与确定性模型相当，同时生成的功率谱显著接近真实观测值。据我们所知，这是第一个通过结合多个微波仪器和不规则时间间隔的红外观测来插值气旋微波图像的生成模型。

View on arXiv Download PDF AI Translation

cs.CV / 52 / 2606.24282

UniRED: Unified RGB-D Video Frame Interpolation with Event Guidance

UniRED：基于事件引导的统一RGB-D视频帧插值

Zhang, Yinuo, Wei, Guangshun, Zhou, Yuanfeng, Shen, Yiran

Abstract

High frame-rate RGB-D videos are crucial for a variety of downstream tasks, including motion analysis, dynamic scene understanding, and 3D reconstruction. However, due to hardware and sensing constraints, practical RGB-D cameras are typically limited to low frame rates, making it difficult to capture rapid scene dynamics. Existing video interpolation methods have achieved strong performance on RGB data, but they are not readily applicable to RGB-D scenarios, where they often yield blurry boundaries, visible artifacts, and degraded geometric consistency. Furthermore, motion estimation from only two boundary frames is inherently under-constrained in complex dynamic scenes. Event cameras, by contrast, provide asynchronous measurements with ultra-high temporal resolution, offering dense motion cues. In this paper, we propose a unified multimodal framework for RGB-D video interpolation that jointly exploits RGB appearance, depth geometry, and event-based temporal cues. Specifically, it first extracts and fuses RGB, depth and event cues, then estimates bidirectional flow with motion basis refinement for RGB and Z-axial refinement for depth, and finally synthesizes the target RGB-D frame via bidirectional warping and soft blending. In addition, we construct a new RGB-D-Event dataset to alleviate the scarcity of tri-modal training data. Extensive experiments on a public benchmark and the proposed dataset demonstrate that our method achieves superior photometric fidelity for RGB interpolation and stronger geometric accuracy for depth interpolation than existing approaches.

Chinese Translation

高帧率的RGB-D视频对于多种下游任务至关重要，包括运动分析、动态场景理解和3D重建。然而，由于硬件和传感限制，实际的RGB-D相机通常受限于低帧率，这使得捕捉快速场景动态变得困难。现有的视频插值方法在RGB数据上取得了良好的性能，但在RGB-D场景中并不适用，往往导致模糊的边界、可见的伪影和几何一致性降低。此外，仅从两个边界帧进行运动估计在复杂动态场景中本质上是欠约束的。相比之下，事件相机提供了超高时间分辨率的异步测量，提供了密集的运动线索。在本文中，我们提出了一种统一的多模态框架用于RGB-D视频插值，该框架共同利用RGB外观、深度几何和基于事件的时间线索。具体而言，它首先提取并融合RGB、深度和事件线索，然后通过运动基的细化估计RGB的双向光流和深度的Z轴细化，最后通过双向扭曲和软混合合成目标RGB-D帧。此外，我们构建了一个新的RGB-D-Event数据集，以缓解三模态训练数据的稀缺性。在公共基准和所提数据集上的大量实验表明，我们的方法在RGB插值方面实现了优越的光度保真度，并在深度插值方面实现了更强的几何准确性，优于现有方法。

View on arXiv Download PDF AI Translation

cs.CV / 53 / 2606.24292

ActiveScope: Actively Seeking and Correcting Perception for MLLMs

ActiveScope：主动寻求和纠正多模态大语言模型的感知

Wang, Yajing, Bi, Chao, Sun, Junshu, Shen, Shufan, Qi, Zhaobo, Wang, Shuhui, Huang, Qingming

Abstract

Multimodal Large Language Models (MLLMs) have demonstrated impressive vision-language understanding, yet still struggle with fine-grained perception in high-resolution images. While existing training-free methods typically rely on attention-based localization or coarse-to-fine search, they are often misled by distractors and fail to locate multiple targets. Our investigation attributes these failures to Contextual Dominance, where salient distractors overwhelm target attention and cause inaccurate localization, and Semantic Bias, where global semantics cause the model to fixate on the most salient concept, resulting in incomplete localization in multi-object scenarios. Built on these insights, we propose ActiveScope, a training-free framework that enhances MLLMs by actively seeking and correcting perception. ActiveScope features two modules. The Semantic Anchor Localization (SAL) utilizes fine-grained semantic anchors to independently localize key targets, thereby mitigating semantic bias. The Interference-Suppressed Refinement (ISR) refines localization by suppressing attention on salient distractions to overcome contextual dominance. Extensive experiments on high-resolution image understanding benchmarks demonstrate that ActiveScope outperforms existing training-free methods (e.g., 96.34 percent accuracy on $V^{*}$ Bench), validating the superiority of the active search and self-correction paradigm. Our code is available at https://github.com/jasmine-ww/ActiveScope.

Chinese Translation

多模态大语言模型（MLLMs）在视觉-语言理解方面表现出色，但在高分辨率图像中的细粒度感知仍然存在困难。现有的无训练方法通常依赖于基于注意力的定位或粗到细的搜索，但它们常常受到干扰物的误导，无法定位多个目标。我们的研究将这些失败归因于上下文主导性（Contextual Dominance），即显著的干扰物压倒了目标注意力，导致不准确的定位，以及语义偏差（Semantic Bias），即全局语义导致模型专注于最显著的概念，从而在多目标场景中导致不完整的定位。基于这些见解，我们提出了ActiveScope，一个无训练框架，通过主动寻求和纠正感知来增强MLLMs。ActiveScope具有两个模块。语义锚定定位（Semantic Anchor Localization, SAL）利用细粒度的语义锚点独立定位关键目标，从而减轻语义偏差。干扰抑制精炼（Interference-Suppressed Refinement, ISR）通过抑制对显著干扰物的注意力来精炼定位，以克服上下文主导性。在高分辨率图像理解基准上的大量实验表明，ActiveScope的表现优于现有的无训练方法（例如，在$V^{*}$ Bench上达到96.34%的准确率），验证了主动搜索和自我纠正范式的优越性。我们的代码可在https://github.com/jasmine-ww/ActiveScope获取。

View on arXiv Download PDF AI Translation

cs.CV / 54 / 2606.24296

Hierarchical Spatial and Channel Aggregation for Cross-domain Few-shot Segmentation

跨域少样本分割的层次空间与通道聚合

Sun, Sujun, Ren, Mingwu, Zhang, Haofeng

Abstract

Cross-domain Few-shot Segmentation (CD-FSS) aims to learn generalizable segmentation capability from abundant annotated samples in the source domain, enabling accurate segmentation of novel classes in the target domain with only a few annotated samples. Existing CD-FSS methods mainly focus on mitigating feature distribution shifts caused by style gaps while ignoring significant differences in class semantic granularity and discriminative attributes across domains, leading to two key degradations in support-query matching: semantic over-alignment and attribute over-alignment. To this end, we propose the Dual Hierarchical Aggregation Network (DHANet), which comprises three key modules. First, the Hierarchical Spatial Aggregation (HSA) module performs multi-scale region aggregation of pixel features along the spatial dimension, generating hierarchical semantic-enhanced features to alleviate semantic over-alignment. Additionally, the HCA module conducts multi-scale attribute aggregation along the channel dimension, generating hierarchical attribute-enhanced features to mitigate attribute over-alignment. Finally, we propose the Online Probabilistic Semantic Bank (OPSB), which progressively constructs and updates class probability distributions from query predictions during inference, and samples multiple pseudo-prototypes as additional support information to mitigate insufficient support. Extensive experiments on four target-domain datasets demonstrate that our method achieves state-of-the-art performance.

Chinese Translation

跨域少样本分割（CD-FSS）旨在从源域中丰富的标注样本中学习可泛化的分割能力，使得在目标域中仅凭少量标注样本即可实现对新类别的准确分割。现有的CD-FSS方法主要集中在减轻由风格差异引起的特征分布偏移，而忽视了跨域之间类别语义粒度和区分属性的显著差异，导致支持-查询匹配中的两个关键退化：语义过度对齐和属性过度对齐。为此，我们提出了双层次聚合网络（DHANet），该网络由三个关键模块组成。首先，层次空间聚合（HSA）模块沿空间维度对像素特征进行多尺度区域聚合，生成层次化的语义增强特征，以缓解语义过度对齐。此外，HCA模块沿通道维度进行多尺度属性聚合，生成层次化的属性增强特征，以减轻属性过度对齐。最后，我们提出了在线概率语义库（OPSB），该库在推理过程中逐步构建和更新来自查询预测的类别概率分布，并采样多个伪原型作为额外的支持信息，以缓解支持不足。在四个目标域数据集上的大量实验表明，我们的方法达到了最先进的性能。

View on arXiv Download PDF AI Translation

cs.CV / 55 / 2606.24297

Training-free Cross-domain Few-shot Segmentation via Robust Semantic Representation and Matching

无训练的跨域少样本分割：基于稳健语义表示与匹配

Sun, Sujun, Ren, Mingwu, Zhang, Haofeng

Abstract

Cross-domain Few-shot Segmentation (CD-FSS) aims to transfer knowledge learned from source domain to distinct target domains, segmenting unseen target classes with only a few annotated samples. Although existing methods have made significant progress, they still rely on training or fine-tuning processes, which incur high computational costs and risk overfitting. We observe that when powerful and general-purpose vision foundation models are incorporated into these methods, their performance shows only marginal improvement or even degrades due to overfitting. To address this, we eliminate trainable parameters and propose a training-free framework to avoid both training overhead and overfitting. Built upon the self-supervised vision encoder DINOv3, our framework addresses cross-domain challenges through three core modules. First, the Semantic-aware Feature Re-fusion (SAFR) module identifies and re-fuses features that emphasize semantic patterns, generating representations with enhanced semantic discriminability. Additionally, the Adaptive Support Enhancement (ASE) module narrows semantic gaps between support and query through robust query information aggregation. Finally, the Hybrid Prototype Matching (HPM) module integrates matching results from diverse prototypes to adapt to varying semantic complexity across domains. Extensive experiments on four target domain datasets demonstrate that our method achieves state-of-the-art performance in CD-FSS without any training.

Chinese Translation

跨域少样本分割（CD-FSS）旨在将从源域学习到的知识转移到不同的目标域，仅通过少量标注样本对未见目标类别进行分割。尽管现有方法已取得显著进展，但仍然依赖于训练或微调过程，这会导致高计算成本并存在过拟合的风险。我们观察到，当将强大且通用的视觉基础模型融入这些方法时，其性能仅有边际改善，甚至由于过拟合而下降。为了解决这个问题，我们消除了可训练参数，提出了一种无训练框架，以避免训练开销和过拟合。我们的框架基于自监督视觉编码器 DINOv3，通过三个核心模块解决跨域挑战。首先，语义感知特征重融合（SAFR）模块识别并重融合强调语义模式的特征，生成具有增强语义可分辨性的表示。此外，自适应支持增强（ASE）模块通过稳健的查询信息聚合缩小支持与查询之间的语义差距。最后，混合原型匹配（HPM）模块整合来自不同原型的匹配结果，以适应跨域的不同语义复杂性。在四个目标域数据集上的广泛实验表明，我们的方法在无任何训练的情况下实现了CD-FSS的最先进性能。

View on arXiv Download PDF AI Translation

cs.CV / 56 / 2606.24301

MM-TRELLIS: Point-Cloud Guided Multi-Modal 3D Vehicle Generation in Autonomous Driving

MM-TRELLIS：基于点云的多模态3D车辆生成在自动驾驶中的应用

Xiao, Hongli, Zhang, Youjian, Bai, Yucai, Wang, Chaoyue, Jin, Yaohui, Ren, Xiaoguang, Yang, Wenjing, Lan, Long

Abstract

Recovering realistic 3D vehicle models from autonomous driving scenes is crucial for synthesizing training data and building simulation environment. However, most existing vehicle generation methods fail to fully exploit multimodal sensors i.e. multi-view images and LiDAR point clouds) and rely on neural rendering based reconstruction, leading to low-quality mesh. Recently, native 3D generative models have made significant progress, yet they are not built for arbitrary multi-view inputs and often struggle with in-the-wild driving images. In this work, we present MM-TRELLIS, a multi-modal version of TRELLIS for in-the-wild 3D vehicle generation that integrates LiDAR and image sensors from autonomous driving datasets into native 3D generative models. Specifically, multi-view images are cycled as conditioning inputs, while LiDAR point clouds provide test-time guidance to ensure geometric accuracy and cross-view consistency. During denoising, we first align the guidance point cloud with the model priors, then enforce consistency between the generated geometry and the guidance point cloud. Finally, we introduce a voxel filtering strategy based on the opacity of 3D Gaussian Splatting to suppress floaters and produce clean meshes. Comprehensive experiments on Waymo dataset demonstrate our method outperforms existing methods in high-fidelity 3D vehicle generation. Code is available at https://github.com/HongliXiao/MM-TRELLIS.

Chinese Translation

从自动驾驶场景中恢复逼真的3D车辆模型对于合成训练数据和构建仿真环境至关重要。然而，大多数现有的车辆生成方法未能充分利用多模态传感器（即多视角图像和LiDAR点云），并依赖于基于神经渲染的重建，导致生成的网格质量较低。最近，原生3D生成模型取得了显著进展，但它们并未针对任意多视角输入构建，且在处理真实驾驶图像时常常面临困难。在本研究中，我们提出了MM-TRELLIS，这是TRELLIS的多模态版本，用于在真实场景中生成3D车辆，集成了来自自动驾驶数据集的LiDAR和图像传感器到原生3D生成模型中。具体而言，多视角图像作为条件输入循环使用，而LiDAR点云在测试时提供指导，以确保几何准确性和视角间一致性。在去噪过程中，我们首先将指导点云与模型先验对齐，然后强制生成的几何形状与指导点云之间的一致性。最后，我们引入了一种基于3D高斯溅射的不透明度的体素过滤策略，以抑制浮动物并生成干净的网格。在Waymo数据集上的全面实验表明，我们的方法在高保真3D车辆生成方面优于现有方法。代码可在 https://github.com/HongliXiao/MM-TRELLIS 获取。

View on arXiv Download PDF AI Translation

cs.CV / 57 / 2606.24302

TrOCR for Medieval HTR: A Systematic Ablation Study with Cross-Dataset Validation

TrOCR在中世纪手写文本识别中的应用：跨数据集验证的系统性消融研究

Sharma, Sachin, Flammini, Michele, Simonetta, Federico

Abstract

Fine-tuning transformer-based handwritten text recognition (HTR) models on medieval manuscripts is challenging because these models are pre-trained on modern text and must adapt to a very different visual domain. This paper studies how three controllable fine-tuning choices (contrast normalization, data augmentation, and layer freezing) affect recognition accuracy when adapting TrOCR to small historical datasets. We run controlled experiments on a 13th-century Italian manuscript (I-CT 91 "Cortonese") and replicate the same experimental grid on the public READ-16 benchmark as robustness evidence. On Cortonese, our best configuration achieves 8.03% character error rate (CER). Statistical comparisons across 13 configurations show that freezing up to three encoder layers or six decoder layers does not significantly harm accuracy, while deeper freezing becomes progressively detrimental. Removing contrast normalization (CLAHE) yields 7.84% CER, comparable to a domain-specialized baseline, suggesting strong optimization can reduce reliance on image preprocessing. Cross-dataset validation on READ-16 shows that decoder freezing thresholds transfer more robustly than encoder thresholds, and combined freezing strategies require dataset-specific re-validation. Finally, we use Grad-CAM gradient attributions and decoder cross-attention maps to diagnose error patterns and failure modes revealed by the ablations. Source code is available at https://github.com/LaudareProject/TrOCR-analysis

Chinese Translation

在中世纪手稿上微调基于变换器的手写文本识别（HTR）模型具有挑战性，因为这些模型是在现代文本上预训练的，必须适应非常不同的视觉领域。本文研究了三种可控微调选择（对比度归一化、数据增强和层冻结）在将TrOCR适应于小型历史数据集时对识别准确性的影响。我们在一份13世纪的意大利手稿（I-CT 91 “Cortonese”）上进行了受控实验，并在公共的READ-16基准上复制相同的实验网格，以作为稳健性证据。在Cortonese上，我们的最佳配置达到了8.03%的字符错误率（CER）。对13种配置的统计比较表明，冻结最多三个编码器层或六个解码器层不会显著损害准确性，而更深的冻结会逐渐产生不利影响。去除对比度归一化（CLAHE）得到7.84%的CER，接近于一个领域专门的基线，表明强优化可以减少对图像预处理的依赖。在READ-16上的跨数据集验证显示，解码器冻结阈值的迁移比编码器阈值更为稳健，而组合冻结策略需要特定于数据集的重新验证。最后，我们使用Grad-CAM梯度归因和解码器交叉注意力图来诊断消融所揭示的错误模式和失败模式。源代码可在https://github.com/LaudareProject/TrOCR-analysis获取。

View on arXiv Download PDF AI Translation

cs.CV / 58 / 2606.24330

REDI-Match: Rotation-Equivariant Distillation for Efficient and Robust Dense Matching

REDI-Match：用于高效和鲁棒密集匹配的旋转等变蒸馏

Ge, Yinji, Zheng, Guixu, Guo, Wulong, Feng, Qian, Wu, Xu, Zhou, Kai, Liu, Xinyuan, Xing, Fei

Abstract

Vision Foundation Models (VFMs) have significantly advanced dense feature matching, yet severe in-plane rotation remains a critical challenge. Existing solutions face a fundamental dilemma: data-driven methods require inefficient parameter scaling to implicitly learn rotations, whereas strictly equivariant networks lack the semantic capacity of modern VFMs. Consequently, current frameworks typically freeze VFMs and shift the entire burden of rotation generalization to the downstream decoder. To break this architectural bottleneck, we propose REDI-Match, an efficient framework driven by a novel Rotation-Equivariant Distillation (REDI) paradigm. Instead of relying on rotation data augmentation to establish rotational correspondences, REDI distills the non-equivariant semantic representations of a VFM into a lightweight, strictly rotation-equivariant encoder, leveraging an equivariant geometric architecture to constrain robust high-dimensional semantics. To fully exploit these features, we equip the decoder with an entropy-driven spatial alignment module. By evaluating discrete rotation hypotheses, this mechanism explicitly locks onto the canonical coordinate system, eliminating global ambiguity before continuous refinement. Extensive experiments demonstrate that REDI-Match establishes a new state-of-the-art (SOTA) across multiple benchmarks. Notably, it achieves a 13.89% absolute pose accuracy improvement on the highly challenging SatAst dataset while operating 1.9x faster than the current SOTA (RoMa v2), enabling real-time inference (~41 FPS) on a single RTX 4090 GPU. Code: https://github.com/YinjiGe/REDI-Match.

Chinese Translation

视觉基础模型（VFM）在密集特征匹配方面取得了显著进展，但平面内的严重旋转仍然是一个关键挑战。现有解决方案面临一个根本性困境：数据驱动的方法需要低效的参数扩展来隐式学习旋转，而严格的等变网络缺乏现代VFM的语义能力。因此，当前框架通常会冻结VFM，并将整个旋转泛化的负担转移到下游解码器。为了打破这一架构瓶颈，我们提出了REDI-Match，一个基于新颖的旋转等变蒸馏（REDI）范式的高效框架。REDI不依赖于旋转数据增强来建立旋转对应关系，而是将VFM的非等变语义表示蒸馏到一个轻量级、严格的旋转等变编码器中，利用等变几何架构来约束鲁棒的高维语义。为了充分利用这些特征，我们为解码器配备了一个基于熵的空间对齐模块。通过评估离散旋转假设，该机制明确锁定在规范坐标系上，在连续细化之前消除全局模糊性。大量实验表明，REDI-Match在多个基准测试中建立了新的最先进（SOTA）水平。值得注意的是，它在极具挑战性的SatAst数据集上实现了13.89%的绝对姿态精度提升，同时比当前的SOTA（RoMa v2）快1.9倍，使得在单个RTX 4090 GPU上实现实时推理（约41 FPS）成为可能。代码： https://github.com/YinjiGe/REDI-Match。

View on arXiv Download PDF AI Translation

cs.CV / 59 / 2606.24333

UniTranslator: A Unified Multi-modal Framework for End-to-end In-Image Machine Translation

UniTranslator：一种统一的多模态框架用于端到端图像内机器翻译

Lyu, Jiahao, Fu, Pei, Li, Zhenhang, Zhang, Shaojie, Yang, Jiahui, Zhou, Yu, Ma, Can, Luo, Zhenbo, Luan, Jian

Abstract

In-Image Machine Translation (IIMT) aims to translate scene text in an image and render the translated text back into the original regions while preserving the overall visual appearance. Recent unified multimodal models provide a promising solution by combining visual-text understanding and image generation within a single framework. However, directly adapting such models to IIMT remains challenging. In particular, they often suffer from understanding-generation conflicts, where the translation inferred during understanding is inconsistent with the text supervision used in generation, and spatial position misalignment, where the rendered text does not accurately match the target text regions. To address these issues, we present UniTranslator, a unified multimodal framework for IIMT that tightly couples translation understanding and text editing. Specifically, we introduce an Understand-Generation Alignment Module (UGAM) to bridge the representation gap between understanding and generation, encouraging semantic consistency between translated content prediction and text rendering. We further propose a Spatial Mask Decoder (SMD) with pixel-level supervision over text regions to improve spatial grounding, geometric alignment, and layout controllability during generation. Extensive experiments on multiple benchmarks demonstrate that UniTranslator achieves state-of-the-art performance across diverse language directions and complex real-world layouts. Moreover, our results reveal a strong mutual reinforcement effect between translation understanding and image generation, highlighting the advantage of unified translation multimodal learning. Code is available at https://github.com/SeerRay-Lab/Unitranslator.

Chinese Translation

图像内机器翻译（In-Image Machine Translation, IIMT）旨在翻译图像中的场景文本，并将翻译后的文本渲染回原始区域，同时保持整体视觉外观。最近的统一多模态模型通过在单一框架内结合视觉-文本理解和图像生成，提供了一种有前景的解决方案。然而，直接将这些模型应用于IIMT仍然具有挑战性。特别是，它们通常面临理解-生成冲突，即在理解过程中推断的翻译与生成中使用的文本监督不一致，以及空间位置错位，即渲染的文本与目标文本区域不准确匹配。为了解决这些问题，我们提出了UniTranslator，这是一种针对IIMT的统一多模态框架，紧密结合翻译理解和文本编辑。具体而言，我们引入了理解-生成对齐模块（Understand-Generation Alignment Module, UGAM），以弥合理解和生成之间的表示差距，鼓励翻译内容预测与文本渲染之间的语义一致性。我们进一步提出了一种空间掩码解码器（Spatial Mask Decoder, SMD），对文本区域进行像素级监督，以改善生成过程中的空间定位、几何对齐和布局可控性。在多个基准上的广泛实验表明，UniTranslator在多种语言方向和复杂的现实世界布局中实现了最先进的性能。此外，我们的结果揭示了翻译理解与图像生成之间强烈的相互增强效应，突显了统一翻译多模态学习的优势。代码可在 https://github.com/SeerRay-Lab/Unitranslator 获取。

View on arXiv Download PDF AI Translation

cs.CV / 60 / 2606.24335

Ill-Posed by Design: Probing Evidence Use in VLMs

设计上的不适定性：探讨视觉语言模型中的证据使用

Meivar, Boaz, Perek, Shaked, Shvartzman, Shani, Schwartz, Eli, Avidan, Shai

Abstract

Counterfactual analysis is widely used to study evidence use in vision-language models, but its diagnostic value is limited on well-posed tasks: when several cues independently support the same answer, removing one may not change the prediction. We propose monocular metric object-size estimation as an ill-posed diagnostic setting for evidence selection: because physical size cannot be determined from a single uncalibrated image, models must rely on imperfect cues category priors, target appearance, local context, apparent image size, and scene geometry. We assemble Metric VQA ($10{,}813$ dimension queries from Objectron and $331$ tape-measured in-the-wild scenes) and evaluate $12$ open-weight VLMs ($3$--$397$\,B parameters) with counterfactual analysis decomposing six visual and language evidence channels. Even the largest VLMs tested (Qwen3-VL-235B, Qwen3.5-397B, InternVL3.5-241B) trail a text-only frontier LLM on the in-the-wild split. The diagnostic analysis shows: target identity is the most load-bearing cue, target pixels and local context help only some models, apparent size shifts predictions without a directional readout, and global scene geometry is largely unused. We analyze LoRA fine-tuning as an actionable intervention specific to metric estimation: while the task is learnable, the models do not learn to leverage scene geometry.

Chinese Translation

反事实分析被广泛用于研究视觉语言模型中的证据使用，但在良定任务上的诊断价值有限：当多个线索独立支持同一答案时，去除其中一个可能不会改变预测。我们提出单目度量物体大小估计作为证据选择的不适定诊断设置：因为物理大小无法从单一未校准图像中确定，模型必须依赖于不完美的线索，如类别先验、目标外观、局部上下文、表观图像大小和场景几何。我们组建了度量视觉问答（Metric VQA）数据集（来自 Objectron 的 $10{,}813$ 维查询和 $331$ 个实地测量场景）并评估了 $12$ 个开放权重的视觉语言模型（VLMs）（$3$--$397$ 亿参数），通过反事实分析分解六个视觉和语言证据通道。即使是测试中最大的 VLM（Qwen3-VL-235B, Qwen3.5-397B, InternVL3.5-241B）在实地分割上也落后于仅文本的前沿大型语言模型（LLM）。诊断分析显示：目标身份是最重要的线索，目标像素和局部上下文仅对某些模型有帮助，表观大小会改变预测但没有方向性输出，全球场景几何大多未被利用。我们分析了 LoRA 微调作为针对度量估计的可行干预：虽然该任务是可学习的，但模型并未学会利用场景几何。

View on arXiv Download PDF AI Translation

cs.CV / 61 / 2606.24336

TIGER: Taming Identity, Geometry, and Generative Priors for High-Quality Face Video Restoration

TIGER：驯化身份、几何和生成先验以实现高质量人脸视频恢复

Zhou, Yang, Li, Wenxue, Zhang, Peng, Chen, Yifei, Wang, Fei, Zhou, Daiguo

Abstract

Face Video Restoration (FVR) aims to recover high-fidelity facial videos from degraded input while preserving identity and semantic consistency across frames. Existing methods often struggle to simultaneously address three key challenges: identity shift, viewpoint-entangled guidance, and perceptual realism. To tackle these issues, we propose TIGER, a structured tri-prior fusion framework that Tames Identity, Geometry, and gEnerative pRiors for high-quality FVR. Specifically, an Identity Prior is first established by injecting subject-discriminative embeddings into the latent space, effectively anchoring the subject's identity against severe degradations. Then, to provide temporally consistent structural guidance for dynamic videos, TIGER constructs a Geometry Prior by lifting 2D reference cues into a disentangled 3D parameter space, creating a geometric anchor through cross-source parameter fusion. Moreover, to achieve maximum efficiency without compromising realism, we harness the video generation model's Generative Prior through a one-step rectified flow. We further design a progressive three-stage training optimization strategy that refines structural fidelity, textural reconstruction, and distribution-level realism to ensure robust optimization. We also construct a large-scale FVR dataset to facilitate robust training and standardized evaluation. Extensive experiments demonstrate that TIGER achieves state-of-the-art performance in both identity fidelity and temporal stability, delivering a high-quality, efficient and identity-consistent FVR. Project page: https://yzhoulv.github.io/Tiger/.

Chinese Translation

人脸视频恢复（FVR）旨在从退化输入中恢复高保真度的面部视频，同时保持跨帧的身份和语义一致性。现有方法往往难以同时解决三个关键挑战：身份偏移、视角纠缠引导和感知现实性。为了解决这些问题，我们提出了TIGER，一个结构化的三先验融合框架，驯化身份、几何和生成先验以实现高质量FVR。具体而言，首先通过将主体区分嵌入注入潜在空间来建立身份先验，有效地将主体的身份锚定在严重退化的情况下。然后，为了为动态视频提供时间一致的结构引导，TIGER通过将2D参考线索提升到解耦的3D参数空间来构建几何先验，通过跨源参数融合创建几何锚点。此外，为了在不妥协现实性的情况下实现最大效率，我们通过一步校正流利用视频生成模型的生成先验。我们进一步设计了一个渐进的三阶段训练优化策略，以提高结构保真度、纹理重建和分布级现实性，确保稳健的优化。我们还构建了一个大规模的FVR数据集，以促进稳健训练和标准化评估。大量实验表明，TIGER在身份保真度和时间稳定性方面都达到了最先进的性能，提供了高质量、高效且身份一致的人脸视频恢复。项目页面：https://yzhoulv.github.io/Tiger/

View on arXiv Download PDF AI Translation

cs.CV / 62 / 2606.24353

Open-Vocabulary BEV Segmentation with 3D-Aware Geometric Constraints

具有3D感知几何约束的开放词汇鸟瞰视图分割

Choi, Hojun, Hwang, Seulbin, Kim, Dae Jung, Kim, Kisung, Shim, Hyunjung, Lee, Jinhan

Abstract

Bird's-eye view (BEV) perception fuses multi-camera images into a unified top-down representation for autonomous driving. Despite recent progress, state-of-the-art methods remain confined to closed-set scenarios, making them vulnerable to unpredictable real-world environments. In this work, we introduce open-vocabulary BEV segmentation (OVBS), which leverages vision-language models (VLMs) to recognize categories beyond the training set while maintaining precise BEV perception and real-time efficiency. A key challenge in OVBS lies in the 3D geometric inconsistency inherent in the ill-posed lifting of 2D VLM semantics into BEV. To address this, we propose OVBEVSeg, a geometry-aware OVBS framework that enhances efficient Gaussian splatting (GS)-based unprojection by leveraging robust 3D geometric constraints across three progressive stages: (1) 2D-to-BEV pseudo-labeling via reliable 3D projection for OV generalization; (2) joint 2D-BEV per-scene optimization with BEV structural constraints for 3D geometric consistency; and (3) 3D geometric distillation for online efficiency. On the nuScenes dataset, OVBEVSeg achieves state-of-the-art performance, outperforming closed-set methods by 15.3 mIoU on unseen categories. Remarkably, even with no novel-class ground-truth labels, it remains competitive with self- and semi-supervised baselines trained with up to 40% of ground-truth annotations. Furthermore, it achieves 2.5x faster inference with only 0.22x the memory consumption of projection-based methods. Project page: https://hchoi256.github.io/projects/ovbevseg/.

Chinese Translation

鸟瞰视图（BEV）感知将多摄像头图像融合为统一的自上而下的表示，以用于自动驾驶。尽管近期取得了一些进展，最先进的方法仍然局限于封闭集场景，使其对不可预测的现实环境变得脆弱。在本研究中，我们提出了开放词汇BEV分割（OVBS），该方法利用视觉-语言模型（VLMs）识别超出训练集的类别，同时保持精确的BEV感知和实时效率。OVBS中的一个关键挑战在于将2D VLM语义提升到BEV时固有的3D几何不一致性。为了解决这个问题，我们提出了OVBEVSeg，这是一个几何感知的OVBS框架，通过在三个渐进阶段中利用稳健的3D几何约束来增强基于高斯喷射（GS）的高效反投影：(1) 通过可靠的3D投影进行2D到BEV的伪标签生成，以实现OV泛化；(2) 结合BEV结构约束进行每场景的2D-BEV联合优化，以确保3D几何一致性；(3) 进行3D几何蒸馏以提高在线效率。在nuScenes数据集上，OVBEVSeg实现了最先进的性能，在未见类别上超越了封闭集方法15.3 mIoU。值得注意的是，即使没有新类别的真实标签，它仍然与使用多达40%真实标注训练的自监督和半监督基线保持竞争力。此外，它的推理速度比基于投影的方法快2.5倍，同时内存消耗仅为0.22倍。项目页面：https://hchoi256.github.io/projects/ovbevseg/

View on arXiv Download PDF AI Translation

cs.CV / 63 / 2606.24361

SignNet-1M: Large-Scale Multilingual Sign Language Video Dataset with Downstream Benchmarks

SignNet-1M：大规模多语言手语视频数据集及下游基准测试

He, Zhewen, Hu, Junyi, Huang, Haomian, Li, Zhenhua, Liu, Yu-Shen, Fang, Yi

Abstract

Sign language models are typically trained on datasets captured under constrained conditions, with limited viewpoint, background, and signer-identity diversity, leading to poor robustness under real-world distribution shifts. We introduce SignNet-1M, a large-scale augmented dataset spanning ASL, CSL, and German Sign Language (DGS). SignNet-1M synthesizes realistic variations along three axes: (i) novel-view rendering (rotation and zoom) via 3D Gaussian Splatting (3DGS), (ii) scene/identity editing via diffusion models for background replacement and signer substitution while preserving sign motion and linguistic content, and (iii) post-rendering augmentations that emulate capture and compression artifacts (e.g., pose/temporal perturbations and video-level corruptions) to better match in-the-wild recordings. Beyond data release, we provide a unified benchmark suite across downstream tasks (e.g., translation and recognition) and ablations that isolate each augmentation component. Experiments across backbones show that training with SignNet-1M consistently improves generalization under cross-view, cross-background, cross-identity, and post-rendering shifts, while maintaining strong in-distribution performance. The dataset, full augmentation pipeline, and benchmark are available at https://signnet.chatsign.ai/.

Chinese Translation

手语模型通常在受限条件下捕获的数据集上进行训练，这些数据集在视角、背景和签名者身份的多样性方面有限，导致在真实世界分布变化下的鲁棒性较差。我们引入了SignNet-1M，这是一个涵盖美国手语（ASL）、中国手语（CSL）和德国语手语（DGS）的大规模增强数据集。SignNet-1M在三个轴向上合成了现实的变异：（i）通过3D高斯点云（3D Gaussian Splatting，3DGS）进行的新视角渲染（旋转和缩放），（ii）通过扩散模型进行场景/身份编辑，以替换背景和签名者，同时保留手势运动和语言内容，以及（iii）后渲染增强，模拟捕获和压缩伪影（例如，姿势/时间扰动和视频级别损坏），以更好地匹配真实环境中的录音。除了数据发布，我们还提供了一个统一的基准测试套件，涵盖下游任务（例如，翻译和识别）以及隔离每个增强组件的消融实验。跨骨干网络的实验表明，使用SignNet-1M进行训练在跨视角、跨背景、跨身份和后渲染变化下的一致性泛化能力上有所提升，同时保持强大的分布内性能。数据集、完整的增强管道和基准测试可在https://signnet.chatsign.ai/获取。

View on arXiv Download PDF AI Translation

cs.CV / 64 / 2606.24371

Structural Kolmogorov-Arnold Convolutions: Learnable Function on the Values or the Filter Shape as Parameter-Efficient Alternative to Per-Edge Convolutional KANs

结构化Kolmogorov-Arnold卷积：在值或滤波器形状上作为参数高效替代的可学习函数

Mereu, Stefano, Kuznetsov, Oleksandr, Marchello, Gabriele, Galdelli, Alessandro, Frontoni, Emanuele, Mancini, Adriano, Cannella, Ferdinando

Abstract

Convolutional Kolmogorov--Arnold Networks (KANs) replace the fixed weights of a convolutional kernel with learnable univariate functions. The dominant formulation attaches one such function to every kernel entry and lets it act on pixel values, expressive but parameter-heavy and prone to overfitting. We argue that the learnable functions are better placed in the \emph{structure} of the convolution than on each edge, and we organise the design space along a single axis: whether the function acts on the pixel \emph{values} or on the filter \emph{shape}. We study three realisations. SV-KAN applies one shared univariate function to the values and leaves the spatial filter free and static, aa classical convolution with a single learnable shared activation. AG-KAN keeps the shared value function but supplies the spatial structure through a content-adaptive Gaussian gate. RF-KAN instead moves the learnable functions onto the filter shape, building each filter from oriented ridge profiles expanded in a localised oscillatory (Morlet) wavelet basis with content-adaptive amplitudes. Under a matched four-layer protocol with in-run references and three seeds, RF-KAN and SV-KAN reach $88.47\pm0.10\%$ and $88.20\pm0.31\%$ on CIFAR-10 and $64.40\pm0.19\%$ and $64.57\pm0.30\%$ on CIFAR-100, at about $0.4$M parameters. At this matched scale the shape model and the simplest value model meet at the top, both above a plain convolution and every per-edge KAN we tested, including the official Gram variant, at roughly a fifth of the parameters. A controlled study attributes the RF-KAN gain to an intrinsically localised oscillatory basis and to content adaptivity, and an ablation that removes the learned shape entirely, leaving only the shared value function, collapses accuracy by over forty points, identifying the learned shape as the load-bearing ingredient at this scale.

Chinese Translation

卷积Kolmogorov-Arnold网络（KANs）用可学习的单变量函数替代卷积核的固定权重。主流的公式将这样的函数附加到每个核条目上，并让其作用于像素值，这种方法虽然表达能力强，但参数量大且容易过拟合。我们认为，可学习的函数更适合放置在卷积的 extit{结构}中，而不是每条边上，并沿着一个单一的轴组织设计空间：即函数是作用于像素 extit{值}还是滤波器 extit{形状}。我们研究了三种实现方式。SV-KAN将一个共享的单变量函数应用于值，并保持空间滤波器自由且静态，类似于具有单个可学习共享激活的经典卷积。AG-KAN保留共享值函数，但通过内容自适应的高斯门提供空间结构。RF-KAN则将可学习的函数移至滤波器形状上，从定向脊轮廓构建每个滤波器，这些轮廓在内容自适应的振荡（Morlet）小波基上扩展。在匹配的四层协议下，使用运行内参考和三个种子，RF-KAN和SV-KAN在CIFAR-10上分别达到$88.47 ext{±}0.10 ext{%}$和$88.20 ext{±}0.31 ext{%}$，在CIFAR-100上分别达到$64.40 ext{±}0.19 ext{%}$和$64.57 ext{±}0.30 ext{%}$，参数量约为$0.4$M。在这个匹配的规模下，形状模型和最简单的值模型在顶部相遇，均优于我们测试的每个边缘KAN，包括官方的Gram变体，参数量大约为原卷积的五分之一。一项受控研究将RF-KAN的增益归因于内在的局部振荡基和内容自适应性，而一项消融实验完全移除学习到的形状，仅保留共享值函数，准确率下降超过四十点，确认学习到的形状是这一规模下的承载成分。

View on arXiv Download PDF AI Translation

cs.CV / 65 / 2606.24375

MATCH: Flow Matching for Multi-View Anomaly Detection

MATCH：用于多视角异常检测的流匹配

Kruse, Mathis, Schween, Melissa, Rosenhahn, Bodo

Abstract

Detecting anomalies in industrial objects is an important topic for increasing production efficiency. More complex objects often require the analysis of several view points, which has led to the field of multi-view anomaly detection. We present MATCH, the first multi-view anomaly detection method based on Flow Matching (FM). With the ODE formulation of Flow Matching, we can estimate likelihoods and thereby derive an anomaly score to detect anomalies in multi-view image data at object, image, and pixel-level. The architectural flexibility of FM models allows us to efficiently transform features of different spatial sizes to the normal distribution. We evaluate thoroughly on the already established Real-IAD data set and are also the first to provide a comprehensive evaluation of popular anomaly detection methods for the MANTA-Tiny data set. MATCH achieves state-of-the-art performance in both anomaly detection and segmentation, all while running on consumer-level hardware. By omitting the costly divergence term needed for likelihood estimation, we ensure that MATCH is usable in real-time production scenarios. Lastly, several ablation studies are conducted to validate the methodological choices.

Chinese Translation

在工业物体中检测异常是提高生产效率的重要课题。更复杂的物体通常需要分析多个视角，这促成了多视角异常检测领域的发展。我们提出了MATCH，这是首个基于流匹配（Flow Matching, FM）的多视角异常检测方法。通过流匹配的常微分方程（ODE）形式，我们可以估计似然性，从而推导出异常分数，以在物体、图像和像素级别检测多视角图像数据中的异常。FM模型的架构灵活性使我们能够高效地将不同空间大小的特征转换为正态分布。我们在已建立的Real-IAD数据集上进行了全面评估，并首次对MANTA-Tiny数据集上流行的异常检测方法进行了全面评估。MATCH在异常检测和分割方面均实现了最先进的性能，同时能够在消费级硬件上运行。通过省略估计似然性所需的高成本散度项，我们确保MATCH可以在实时生产场景中使用。最后，我们进行了多项消融研究，以验证方法选择的合理性。

View on arXiv Download PDF AI Translation

cs.CV / 66 / 2606.24404

Modality-Aware Out-of-Distribution Detection for Multi-Modal Action Recognition

面向模态的多模态动作识别中的分布外检测

Doorenbos, Lars, Vu, Duc Manh, Ozsoy, Serdar, Gall, Juergen

Abstract

The incorporation of additional modalities into action recognition models increases their performance across a wide range of settings. However, how this additional information can contribute to making the models more robust remains underexplored, particularly for the case of multi-modal out-of-distribution (OOD) detection. While methods exist that regularize the multi-modal training process with OOD detection in mind, they still apply off-the-shelf OOD detectors designed for the uni-modal case during inference, discarding important information. Based on an interesting relationship we find between the multi-modal and uni-modal predictions, we propose to use this signal to build a post-hoc detector explicitly designed for the multi-modal scenario. We combine this new source of information with a feature-space score, which detects off-manifold samples in the multi-modal space, and normalize them by the multi-modal logits. In doing so, the proposed hybrid detector is compatible with existing training-time approaches and consistently improves performance. Experiments on a wide range of established datasets from the MultiOOD benchmark show that, on average, our approach outperforms the state of the art. Our results show the importance of explicitly considering the different modalities at inference time for multi-modal OOD detection.

Chinese Translation

将额外的模态纳入动作识别模型可以提高其在各种场景下的性能。然而，这些额外信息如何有助于增强模型的鲁棒性仍然未被充分探讨，特别是在多模态分布外（OOD）检测的情况下。虽然存在一些方法在多模态训练过程中考虑OOD检测进行正则化，但在推理时仍然使用为单模态情况设计的现成OOD检测器，从而忽略了重要信息。基于我们发现的多模态与单模态预测之间的有趣关系，我们提出利用这一信号构建一个专门为多模态场景设计的后验检测器。我们将这一新的信息源与特征空间得分相结合，该得分用于检测多模态空间中的离群样本，并通过多模态对数值进行归一化。通过这样做，所提出的混合检测器与现有的训练时方法兼容，并且始终提高性能。在来自MultiOOD基准的多种已建立数据集上的实验表明，平均而言，我们的方法优于现有的最先进技术。我们的结果表明，在推理时明确考虑不同模态对于多模态OOD检测的重要性。

View on arXiv Download PDF AI Translation

cs.CV / 67 / 2606.24422

EgoSAT: A Comprehensive Benchmark of Egocentric Streaming Interaction Understanding

EgoSAT：自我中心流媒体交互理解的综合基准

Lei, Yijia, Li, Jinzhao, Zhang, Yichi, Hua, Jiacheng, Li, Yin, Liu, Miao

Abstract

We introduce EgoSAT, the first comprehensive benchmark for egocentric video reasoning in streaming settings, designed to evaluate the capabilities of modern vision-language models (VLMs). The benchmark targets streaming interaction understanding, where video frames arrive sequentially and models must continuously interpret evolving visual context. EgoSAT unifies several previously distinct tasks within a single streaming framework. In this formulation, queries about completed events correspond to retrospective reasoning, queries about ongoing activities require online understanding, and queries about future actions involve prospective anticipation. This unified setting requires models to reason about the past, present, and future while operating under the constraint that only previously observed frames are available. EgoSAT contains 1,997 unique videos spanning 165 hours of egocentric footage and around 4,800 high-quality question-answer pairs, carefully designed to probe reasoning across varying temporal contexts. Using this benchmark, we evaluate a diverse set of both open-weight and closed-weight VLMs, providing a systematic assessment of their ability for streaming interaction understanding. By distinguishing answerability and conducting diagnostics on confidence of models, we find existing models not only struggle with prospective and retrospective modeling, but also exhibit severe mis-calibration: confidence often fails to track inherent answerability, leading to dangerous "confidently wrong" behaviors. Project page: https://leiyj23.github.io/EgoSAT/

Chinese Translation

我们介绍了EgoSAT，这是第一个针对流媒体环境中自我中心视频推理的综合基准，旨在评估现代视觉-语言模型（VLMs）的能力。该基准专注于流媒体交互理解，其中视频帧按顺序到达，模型必须持续解释不断变化的视觉上下文。EgoSAT将几个之前独立的任务统一到一个流媒体框架中。在这种表述中，关于已完成事件的查询对应于回顾性推理，关于正在进行活动的查询需要在线理解，而关于未来行动的查询则涉及前瞻性预测。这种统一的设置要求模型在仅能访问先前观察到的帧的约束下，对过去、现在和未来进行推理。EgoSAT包含1,997个独特视频，涵盖165小时的自我中心视频素材，以及约4,800对高质量的问答对，经过精心设计以探测在不同时间上下文中的推理能力。利用该基准，我们评估了一组多样化的开放权重和闭合权重的VLMs，提供了对其流媒体交互理解能力的系统评估。通过区分可回答性并对模型的信心进行诊断，我们发现现有模型不仅在前瞻性和回顾性建模方面存在困难，而且表现出严重的误校准：信心往往无法跟踪固有的可回答性，导致危险的“自信错误”行为。项目页面：https://leiyj23.github.io/EgoSAT/

View on arXiv Download PDF AI Translation

cs.CV / 68 / 2606.24430

Transformation Behavior of Images in Latent Space

潜在空间中图像的变换行为

Zöllner, Christian, Motiwala, Mozzam, Ahadova, Aysel, Anders, Gerrit, Hüneburg, Robert, Nattermann, Jacob, Kloor, Matthias

Abstract

Training of neural networks for histopathology classification tasks typically relies on data encoding into latent space, which reduces complexity and improves performance. There are several encoder networks available, either pretrained on general image datasets such as ImageNET, or specifically on histopathological images. Training of encoder networks should be adapted to downstream tasks, allowing encoding of biologic/diagnostic content while rendering networks invariant to label-irrelevant transformations. This paper investigates the effect of classical image transformation on the latent space, using networks provided by Lunit Inc. and Bioptimus, both focusing on pathological images, and by Meta Research Team. We assess variance of embeddings resulting from standard data transformations by comparing original and transformed image embeddings and by contrasting them with random, unrelated embeddings, using image tiles from hematoxylin/eosin-stained sections available in a colorectal tissue dataset and the publicly accessible TCGA dataset. Our findings show that embeddings of original and transformed images are closer to each other than to random embeddings, indicating robustness to transformations. However, they are not fully invariant, revealing that the encoder networks do not completely neutralize transformation effects in latent space, explaining why transformation-mediated augmentation of datasets can improve performance. Significant differences were observed between general and histopathology-specific encoder networks.

Chinese Translation

神经网络在组织病理分类任务中的训练通常依赖于数据编码到潜在空间，这样可以降低复杂性并提高性能。目前有多种编码器网络可供选择，这些网络要么是在通用图像数据集（如 ImageNET）上预训练的，要么是专门针对组织病理图像进行训练的。编码器网络的训练应适应下游任务，使其能够编码生物/诊断内容，同时使网络对与标签无关的变换保持不变。本文研究了经典图像变换对潜在空间的影响，使用了 Lunit Inc. 和 Bioptimus 提供的网络，这些网络均专注于病理图像，以及 Meta Research Team 的网络。我们通过比较原始图像和变换后图像的嵌入，并将其与随机、不相关的嵌入进行对比，评估标准数据变换所导致的嵌入方差，使用的数据来自于结直肠组织数据集中经过苏木精/伊红染色的切片图像和公开可获取的 TCGA 数据集。我们的研究结果表明，原始图像和变换后图像的嵌入彼此之间的距离比与随机嵌入的距离更近，表明对变换具有一定的鲁棒性。然而，它们并非完全不变，这揭示了编码器网络并未完全消除潜在空间中的变换效应，这也解释了为何通过变换介导的数据增强可以提高性能。我们观察到通用编码器网络与组织病理特定编码器网络之间存在显著差异。

View on arXiv Download PDF AI Translation

cs.CV / 69 / 2606.24433

MedPCFM: Improving Medical Point Cloud Completion by Integrating Point Transformers and Flow Matching

MedPCFM：通过整合点变换器和流匹配来改善医学点云补全

Kwarciak, Kamil, Wodzinski, Marek

Abstract

Medical point cloud completion is important for anatomical reconstruction and downstream clinical workflows, yet generative modeling in this setting remains insufficiently studied. We investigate completion through continuous-time generative modeling and introduce PCFM, a PTv3-backed flow matching approach for medical point cloud completion. We evaluate on SkullFix and SkullBreak, and additionally on the more recent Mandibular Defect dataset. We build strong baselines by adapting PTv3 to a deterministic encoder-decoder completion model and by instantiating diffusion completion (PCDiff) with both PVCNN and PTv3 denoisers. PCFM with PTv3 is competitive with the deterministic PTv3 baseline and achieves state-of-the-art generative performance across datasets, while requiring substantially fewer sampling steps than diffusion. At the best operating points, PTv3 also yields clear throughput gains, providing up to a 7$\times$ speed-up for PCFM compared to a PVCNN backbone. Finally, we study empirical scaling trends by varying model size and point cardinality, showing consistent gains with higher point resolution and informative trade-offs across model scales.

Chinese Translation

医学点云补全对于解剖重建和后续临床工作流程至关重要，但在这一领域的生成建模研究仍然不足。我们通过连续时间生成建模来研究补全，并引入PCFM，一种基于PTv3的流匹配方法用于医学点云补全。我们在SkullFix和SkullBreak数据集上进行评估，并额外在较新的下颌缺损数据集上进行测试。我们通过将PTv3适配为确定性编码-解码补全模型，以及通过使用PVCNN和PTv3去噪器实例化扩散补全（PCDiff）来构建强基线。使用PTv3的PCFM在与确定性PTv3基线的竞争中表现良好，并在各数据集上实现了最先进的生成性能，同时所需的采样步骤显著少于扩散方法。在最佳操作点，PTv3还带来了明显的吞吐量提升，与PVCNN主干相比，PCFM的速度提高可达7倍。最后，我们通过改变模型大小和点的基数来研究经验缩放趋势，显示出在更高点分辨率下的一致增益以及在模型规模之间的有益权衡。

View on arXiv Download PDF AI Translation

cs.CV / 70 / 2606.24441

S1-Omni-Image: A Unified Model for Scientific Image Understanding, Generation, and Editing

S1-全景图像：一个统一的科学图像理解、生成与编辑模型

Li, Qingxiao, Wang, Zikai, Wang, Qingli, Xu, Nan

Abstract

We present S1-Omni-Image, an open-weight unified multimodal model for scientific image understanding, generation, and editing. Unlike general-purpose image generation models, scientific image tasks require not only high-fidelity synthesis, but also robust understanding of scientific semantics, structural relations, domain knowledge, and task intent. To this end, S1-Omni-Image builds on the scientific multimodal reasoning backbone S1-VL-32B and couples its understanding capability with an image generation module under a unified think-before-generate paradigm. Given a user instruction, the model first produces a task-oriented reasoning trace, a textual answer, and a task special token; their hidden states are then injected into the generation module to condition image generation or editing. S1-Omni-Image supports scientific image understanding, generation, and editing in a unified framework. For generation, it focuses on scientific illustrations and text rendering, including logical diagrams, relational comparisons, data charts, and realistic scientific visualizations. For editing, it casts segmentation and other domain-specific vision tasks as native image editing problems, enabling multi-turn illustration editing, medical and geographic image segmentation, medical image translation, and scientific image super-resolution. We construct SciGenEdit, a 314K-sample training dataset, and release the model weights, inference code, and SciGenEdit-10K. Experiments show that S1-Omni-Image substantially improves scientific image generation and editing while preserving the scientific image understanding capability inherited from S1-VL-32B. It outperforms open-source models on GenExam and TechImage-Bench, achieves state-of-the-art results on four editing benchmarks including MSD, cigRockSEM, SynthRAD2025, and IXI, and maintains stable performance on scientific image understanding evaluations.

Chinese Translation

我们提出了S1-全景图像（S1-Omni-Image），这是一个开放权重的统一多模态模型，用于科学图像的理解、生成和编辑。与通用图像生成模型不同，科学图像任务不仅需要高保真合成，还需要对科学语义、结构关系、领域知识和任务意图的稳健理解。为此，S1-全景图像基于科学多模态推理骨干网络S1-VL-32B，结合其理解能力与图像生成模块，采用统一的“先思考后生成”范式。在接收到用户指令后，该模型首先生成一个面向任务的推理轨迹、一个文本答案和一个任务专用标记；然后将它们的隐藏状态注入生成模块，以条件生成或编辑图像。S1-全景图像在统一框架下支持科学图像的理解、生成和编辑。在生成方面，它专注于科学插图和文本渲染，包括逻辑图、关系比较、数据图表和真实的科学可视化。在编辑方面，它将分割和其他特定领域的视觉任务视为原生图像编辑问题，使得多轮插图编辑、医学和地理图像分割、医学图像翻译以及科学图像超分辨率成为可能。我们构建了SciGenEdit，一个包含314K样本的训练数据集，并发布了模型权重、推理代码和SciGenEdit-10K。实验表明，S1-全景图像在科学图像生成和编辑方面显著提升，同时保留了从S1-VL-32B继承的科学图像理解能力。它在GenExam和TechImage-Bench上超越了开源模型，在包括MSD、cigRockSEM、SynthRAD2025和IXI在内的四个编辑基准上达到了最先进的结果，并在科学图像理解评估中保持了稳定的表现。

View on arXiv Download PDF AI Translation

cs.CV / 71 / 2606.24447

P-MTP: Efficient Document Parsing via Multi-Token Prediction with Progressive Depth Scaling

P-MTP：通过渐进深度扩展的多标记预测实现高效文档解析

Xiang, Le, Zhai, Chenxi, Wei, Shu, Wu, Jingjing, Xie, Qunyi, Tan, Xiao, Chen, Kunbin, He, Wei

Abstract

Vision-Language Models (VLMs) have revolutionized document parsing by enabling end-to-end mapping from images to structured text, imposing a significant latency bottleneck, particularly for token-dense documents. While Multi-Token Prediction (MTP) has emerged as a promising approach for accelerating inference, its potential is constrained by optimization instability when scaling to deeper look-ahead depth. In this paper, we propose \textbf{P-MTP}, a framework that leverages \textbf{Progressive Multi-Token Prediction} with a lightweight MTP module to scale the look-ahead depth for high-throughput document parsing. Specifically, we introduce Progressive Curriculum Loss that adaptively re-weights different look-ahead depths using cumulative path reliability and retrospective target consistency. By effectively suppressing gradient noise in long-range predictions, P-MTP, facilitates an automated easy-to-hard optimization transition, enabling the model to master increasingly distant look-ahead depths. Furthermore, we propose Confidence-Gated Dynamic Drafting to maximize the effective look-ahead depth and acceptance rate by adaptively calibrating speculative length during inference, thereby minimizing computational waste and further pushing the boundaries of inference speedup. Experimental results across multiple benchmarks and architectures demonstrate that P-MTP, achieves up to a $5\times$ speedup with negligible loss in accuracy, providing the first successful validation of extensive look-ahead MTP in the document parsing domain.

Chinese Translation

视觉-语言模型（VLMs）通过实现从图像到结构化文本的端到端映射，彻底改变了文档解析，但在处理标记密集型文档时，仍然存在显著的延迟瓶颈。尽管多标记预测（MTP）作为加速推理的有前景的方法已逐渐兴起，但在扩展到更深的前瞻深度时，其潜力受到优化不稳定性的限制。本文提出了 extbf{P-MTP}，一个利用 extbf{渐进多标记预测}的框架，结合轻量级MTP模块，以扩展前瞻深度，实现高吞吐量的文档解析。具体而言，我们引入了渐进课程损失，通过使用累积路径可靠性和回顾目标一致性自适应地重新加权不同的前瞻深度。通过有效抑制长距离预测中的梯度噪声，P-MTP促进了自动的简单到困难的优化过渡，使模型能够掌握越来越远的前瞻深度。此外，我们提出了信心门控动态草图，通过在推理过程中自适应地校准推测长度，最大化有效前瞻深度和接受率，从而最小化计算浪费，进一步推动推理加速的边界。在多个基准和架构上的实验结果表明，P-MTP在准确性几乎不受损的情况下，实现了高达$5 imes$的加速，为文档解析领域广泛前瞻MTP的首次成功验证提供了支持。

View on arXiv Download PDF AI Translation

cs.CV / 72 / 2606.24449

SENTRY: SAM2-Enhanced Neighbor-Aware and Temporally Reasoned Memory for Visual Tracking

SENTRY：基于SAM2的增强邻域感知和时间推理视觉跟踪记忆

Alansari, Mohamad, Michael, Yonathan, AlMarzouqi, Hasan, Naseer, Muzammal, Werghi, Naoufel, Javed, Sajid

Abstract

We revisit the memory update mechanism in SAM2-based visual object tracking and identify confidence-only mask selection as the dominant cause of drift under occlusion, rapid motion, and distractors. We introduce SENTRY, a training-free, plug-and-play, refine-before-write module that validates each memory update for short-horizon temporal consistency before committing it. SENTRY aggregates diverse segmentation hypotheses per frame, backtracks them into short tracklets, and uses neighbor-aware cycle-consistent matching against recent trajectories to favor temporally and geometrically consistent masks. It leaves the base architecture untouched, replacing confidence-driven writes with consistency-validated ones. For fair evaluation, we re-evaluate major open-source SAM2-based trackers across all available scales and datasets, filling gaps in prior reports. Integrated into five strong baselines, SENTRY delivers consistent gains across nine benchmarks, achieving new zero-shot SOTA on LaSOT, LaSOT_ext, GOT-10k, VOT20, VOT22, and DiDi. Despite these checks, the SAM2-L version runs at 32.8 FPS on an A100, and across compatible hosts adds only about 0.4--0.6 GB VRAM. Our results provide the first unified all-scale evaluation of SAM2-based trackers and show that enforcing temporal validity at write time stabilizes memory-augmented tracking without retraining.

Chinese Translation

我们重新审视了基于SAM2的视觉物体跟踪中的记忆更新机制，并确定仅基于置信度的掩膜选择是遮挡、快速运动和干扰物下漂移的主要原因。我们引入了SENTRY，一个无需训练、即插即用的模块，在提交每次记忆更新之前验证其短期时间一致性。SENTRY在每帧中聚合多样的分割假设，将其回溯为短轨迹，并使用邻域感知的循环一致性匹配与最近的轨迹进行比较，以优先选择时间和几何一致的掩膜。它保持基础架构不变，用一致性验证的写入替代基于置信度的写入。为了公平评估，我们重新评估了所有可用规模和数据集中的主要开源基于SAM2的跟踪器，填补了之前报告中的空白。集成到五个强基线中，SENTRY在九个基准测试中提供了一致的提升，在LaSOT、LaSOT_ext、GOT-10k、VOT20、VOT22和DiDi上实现了新的零样本最优状态（SOTA）。尽管进行了这些检查，SAM2-L版本在A100上以32.8 FPS运行，并且在兼容主机上仅增加约0.4-0.6 GB的显存。我们的结果提供了基于SAM2的跟踪器的首次统一全尺度评估，并表明在写入时强制执行时间有效性可以在不重新训练的情况下稳定记忆增强的跟踪。

View on arXiv Download PDF AI Translation

cs.CV / 73 / 2606.24457

Lite Any Stereo V2: Faster and Stronger Efficient Zero-Shot Stereo Matching

Lite Any Stereo V2：更快更强的高效零-shot立体匹配

Jing, Junpeng, Zuo, Ronglai, Shen, Zhelun, Zhou, Shangchen, Potamias, Rolandos Alexandros, Zafeiriou, Stefanos, Mikolajczyk, Krystian, Deng, Jiankang

Abstract

Recent advances in stereo matching have achieved remarkable accuracy, but often rely on large models, heavy computation, or additional foundation-model priors, making them difficult to deploy on resource-constrained platforms. In contrast, efficient stereo models offer faster inference but are commonly considered less capable of strong zero-shot generalization. In this paper, we challenge this assumption by introducing Lite Any Stereo V2 (LAS2), an ultra-fast model series designed for efficient zero-shot stereo matching. LAS2 is developed from both architecture and training perspectives. Architecturally, we revisit efficient stereo design under practical deployment settings and propose a 2D-only cost aggregation framework, optimized for real inference latency rather than theoretical MACs alone. For training, we develop a three-stage strategy that combines synthetic supervision, self-distillation, and real-world knowledge distillation. To improve the reliability of real-world pseudo supervision, we further introduce pseudo-label filtering and an error-clamping operation, enabling smoother synthetic-to-real transfer. We instantiate LAS2 as a family of models, including feed-forward variants for different efficiency budgets and an iterative variant for higher accuracy. Extensive experiments show that LAS2 achieves state-of-the-art accuracy among efficient stereo methods while maintaining significantly lower latency. Specifically, LAS2-H achieves stronger overall zero-shot performance than the iterative method Fast-FoundationStereo, with 1.8x and 2.7x faster inference on H200 and Orin, respectively. The project page, demos, and code are available at https://tomtomtommi.github.io/LiteAnyStereoV2/.

Chinese Translation

近年来，立体匹配的进展取得了显著的准确性，但通常依赖于大型模型、繁重的计算或额外的基础模型先验，使其在资源受限的平台上难以部署。相比之下，高效的立体模型提供了更快的推理速度，但通常被认为在强零-shot泛化能力上较弱。本文通过引入Lite Any Stereo V2 (LAS2)这一超快速模型系列，挑战了这一假设，旨在实现高效的零-shot立体匹配。LAS2从架构和训练两个方面进行开发。在架构上，我们重新审视了在实际部署环境下的高效立体设计，并提出了一种仅基于2D的成本聚合框架，优化了真实推理延迟，而不仅仅是理论上的MACs。在训练方面，我们开发了一种三阶段策略，结合了合成监督、自我蒸馏和真实世界知识蒸馏。为了提高真实世界伪监督的可靠性，我们进一步引入了伪标签过滤和错误钳制操作，使合成到真实的转移更加平滑。我们将LAS2实例化为一系列模型，包括适用于不同效率预算的前馈变体和用于更高准确性的迭代变体。大量实验表明，LAS2在高效立体方法中实现了最先进的准确性，同时保持了显著较低的延迟。具体而言，LAS2-H在H200和Orin上分别比迭代方法Fast-FoundationStereo实现了1.8倍和2.7倍的推理速度提升，且整体零-shot性能更强。项目页面、演示和代码可在https://tomtomtommi.github.io/LiteAnyStereoV2/获取。

View on arXiv Download PDF AI Translation

cs.CV / 74 / 2606.24464

Boosting Text-Driven Video Segmentation via Geometry-Aware Distillation

通过几何感知蒸馏提升文本驱动的视频分割

Zhu, Tianyu, Liang, Yingping, Li, Hesong, Fu, Ying

Abstract

Text-driven Referring Video Object Segmentation (RVOS) aims to locate and segment target objects in videos given natural language. However, existing models are typically trained on 2D image or video datasets with naive segmentation losses, which overlooks the geometric consistency across frames and leads to weak spatial understanding. In this paper, we propose Geometry-enhanced Language-guided Video segmentation (GeoLaV), a two-stage framework that distills 3D geometric knowledge from images to enhance text-driven video segmentation. In the first stage, we perform monocular geometry pretraining with monocular novel-view synthesis, enabling the model to acquire geometry-consistent visual representations via spatial alignment on large-scale single-image datasets. In the second stage, we introduce geometry-aware distillation and fine-tune the model on video segmentation datasets, transferring 3D structural knowledge from a general 3D prior model. This process reinforces 3D awareness and improves both spatiotemporal coherence and language grounding in segmentation. Extensive experiments show that our method using only image segmentation data already provides notable zero-shot generalization in RVOS. When combined with geometry-aware distillation for fine-tuning on videos, our method achieves state-of-the-art performance across multiple RVOS benchmarks. The code is available at https://github.com/Tony1882880/GeoLaV.

Chinese Translation

文本驱动的指代视频目标分割（RVOS）旨在根据自然语言定位和分割视频中的目标对象。然而，现有模型通常在二维图像或视频数据集上训练，使用简单的分割损失，这忽视了帧之间的几何一致性，导致空间理解能力较弱。本文提出了一种几何增强的语言引导视频分割框架（GeoLaV），该框架通过蒸馏图像中的三维几何知识来增强文本驱动的视频分割。在第一阶段，我们通过单目新视图合成进行单目几何预训练，使模型能够通过在大规模单图像数据集上的空间对齐获取几何一致的视觉表征。在第二阶段，我们引入几何感知蒸馏，并在视频分割数据集上微调模型，从一个通用的三维先验模型中转移三维结构知识。这个过程增强了三维意识，提高了分割中的时空连贯性和语言基础。大量实验表明，我们的方法仅使用图像分割数据就已经在RVOS中提供了显著的零样本泛化。当结合几何感知蒸馏进行视频微调时，我们的方法在多个RVOS基准测试中达到了最先进的性能。代码可在 https://github.com/Tony1882880/GeoLaV 获取。

View on arXiv Download PDF AI Translation

cs.CV / 75 / 2606.24477

video-SALMONN-R$^3$: Learning to ReWatch, ReAsk, and ReAnswer for Efficient Video Understanding

video-SALMONN-R$^3$: 学习重看、重问和重答以实现高效视频理解

Li, Yixuan, Sun, Guangzhi, Yang, Yudong, Li, Wei, MA, Zejun, Zhang, Chao

Abstract

Video large language models (LLMs) are often constrained by computation and memory budgets, leading them to use reduced frame rates and spatial resolutions, which may cause them to miss critical information for question answering (QA). A practical and efficient solution is a two-stage paradigm: first perform coarse video understanding to localize relevant segments, and then re-watch these segments at higher temporal or spatial fidelity. In this paper, we present video-SALMONN-R$^3$, the first end-to-end video-LLM that enables re-watch through reinforcement learning without relying on chain-of-thought (CoT) cold-start. This design removes the need for costly CoT data annotations and avoids CoT-based supervised fine-tuning (SFT), which can otherwise degrade the pretrained video understanding abilities. To address the mismatch between the reasoning-first behavior induced by re-watch and the answer-first tendency of pretrained video-LLMs, we propose a re-answer strategy, in which the model first produces a direct answer in the first watch and then refines it after re-watching. Finally, to improve question adherence during re-watching, we propose a re-ask mechanism that re-injects the query when revisiting localized segments. Experimental results show that video-SALMONN-R$^3$ consistently outperforms both the base model and the QA-SFT baseline, while surpassing prior re-watch-based approaches with significantly lower computational cost. Code, models, and data will be publicly released upon acceptance.

Chinese Translation

视频大型语言模型（LLMs）通常受到计算和内存预算的限制，这使得它们使用降低的帧率和空间分辨率，从而可能错过回答问题（QA）所需的关键信息。一种实用且高效的解决方案是采用两阶段范式：首先进行粗略的视频理解以定位相关片段，然后以更高的时间或空间保真度重看这些片段。本文提出了video-SALMONN-R$^3$，这是第一个端到端的视频LLM，能够通过强化学习实现重看，而无需依赖思维链（CoT）冷启动。这一设计消除了对昂贵的CoT数据标注的需求，并避免了基于CoT的监督微调（SFT），后者可能会降低预训练视频理解能力。为了应对重看所引发的推理优先行为与预训练视频LLM的答案优先倾向之间的不匹配，我们提出了一种重答策略，其中模型在第一次观看时首先生成直接答案，然后在重看后进行细化。最后，为了在重看过程中提高问题的遵循性，我们提出了一种重问机制，在重新访问定位片段时重新注入查询。实验结果表明，video-SALMONN-R$^3$在性能上始终优于基础模型和QA-SFT基线，同时在计算成本上显著低于先前的基于重看的方法。代码、模型和数据将在接受后公开发布。

View on arXiv Download PDF AI Translation

cs.CV / 76 / 2606.24479

MambaRaw: Selective State Space Modeling for Efficient 4K Raw Image Reconstruction

MambaRaw：用于高效4K原始图像重建的选择性状态空间建模

Li, Peize, Zeng, Fanhu, Xu, Tongda, Xu, Xingguo, Zhang, Xinjie, Ge, Xingtong, Zhang, Haotian, Wang, Yan

Abstract

In-camera JPEG previews are ubiquitous in raw image formats and provide an sRGB reference at negligible storage cost. Although existing metadata-based reconstruction frameworks can exploit this side information when recovering raw images, their context models often become computationally expensive especially at high resolution, eg, 4K raw image, given that attention mechanisms scale quadratically with feature maps, hindering its practical application. To address these limitations, we propose MambaRaw, a JPEG-conditioned metadata-based raw image reconstruction framework that uses State Space Models (SSMs) to estimate entropy parameters efficiently. Our key contribution comprises a Spatial-Energy Coupled Context Modeling mechanism with two lightweight modules: (1) TileMambaBlock, which performs Mamba-style selective scanning only on information-dense tiles to improve the efficiency; and (2) Energy-Aware Refinement (EAR), an identity-initialized residual module that enhance feature representation to match the long-tail energy distribution of raw signals. Extensive experiments on three camera datasets (Sony, Olympus, Samsung) show consistent improvements over strong metadata-based baselines and set a new state of the art for JPEG-guided raw reconstruction with great efficiency. Notably, at low metadata bitrates, MambaRaw increases PSNR by 1.2--1.4 dB and reduces end-to-end coding latency by about 9%. Code is released at https://github.com/Peizeli1/MambaRaw.

Chinese Translation

相机内的JPEG预览在原始图像格式中无处不在，并以微不足道的存储成本提供sRGB参考。尽管现有的基于元数据的重建框架可以在恢复原始图像时利用这些侧面信息，但它们的上下文模型在高分辨率（例如4K原始图像）时往往变得计算开销巨大，因为注意力机制与特征图的规模呈平方关系，这阻碍了其实际应用。为了解决这些限制，我们提出了MambaRaw，这是一个基于JPEG条件的元数据原始图像重建框架，利用状态空间模型（State Space Models, SSMs）高效估计熵参数。我们的主要贡献包括一个空间-能量耦合上下文建模机制，具有两个轻量级模块：（1）TileMambaBlock，仅对信息密集的块执行Mamba风格的选择性扫描，以提高效率；（2）能量感知细化（Energy-Aware Refinement, EAR），一个身份初始化的残差模块，增强特征表示以匹配原始信号的长尾能量分布。在三个相机数据集（Sony、Olympus、Samsung）上的广泛实验显示，MambaRaw在强大的基于元数据的基线之上实现了一致的改进，并为JPEG引导的原始重建设定了新的最先进水平，且效率极高。值得注意的是，在低元数据比特率下，MambaRaw将PSNR提高了1.2--1.4 dB，并将端到端编码延迟减少了约9%。代码已发布在https://github.com/Peizeli1/MambaRaw。

View on arXiv Download PDF AI Translation

cs.CV / 77 / 2606.24484

Advancing WordArt-Oriented Scene Text Recognition: Datasets and Methods

推进以WordArt为导向的场景文本识别：数据集与方法

Ye, Xingsong, Du, Yongkun, Zhang, Jiaxin, Zhang, Haojie, Sun, Chong, Li, Chen, Lyu, Jing, Chen, Zhineng

Abstract

WordArt (artistic text) features highly customized fonts, textures, and layouts, making WordArt-oriented scene TExt Recognition (WATER) substantially more challenging than general Scene Text Recognition (STR). Existing STR datasets and methods, typically built around regular scene text and fixed-template inputs, struggle to scale to WATER. Thus, we aim to advance this task from both data and model perspectives. On the data side, we construct a 2M synthetic dataset, WATER-S, with the scale improved by hundreds of times compared to existing artistic text data. WATER-S consists of two complementary subsets. One rendered by an upgraded rendering pipeline (SynthWordArt), which provides highly accurate and controllable synthetic WordArt data. The other is generated by combining Qwen3-VL for prompt mining and Z-Image for image synthesis, which improves the coverage of realistic and diverse data. On the model side, we propose WATERec. It adopts an visual encoder supporting arbitrary-shaped inputs and an autoregressive decoder to model complex layouts, structurally breaking the bottleneck of fixed-template STR on WordArt. Experiments show that this architecture outperforms prior STR methods, achieving state-of-the-art performance on irregular texts such as WordArt. Together with WATER-R, carefully reorganized from existing real STR data, our strong baseline with the new synthetic data and model design reaches 90.40% accuracy on WordArt-Bench, surpassing both general-purpose and OCR-specialized vision-language models by a large margin. Code and data are available at https://github.com/YesianRohn/WATER.

Chinese Translation

WordArt（艺术文本）具有高度定制的字体、纹理和布局，使得以WordArt为导向的场景文本识别（WATER）比一般的场景文本识别（STR）更具挑战性。现有的STR数据集和方法通常围绕常规场景文本和固定模板输入构建，难以扩展到WATER。因此，我们旨在从数据和模型两个角度推进这一任务。在数据方面，我们构建了一个规模达到200万的合成数据集WATER-S，其规模比现有的艺术文本数据提高了数百倍。WATER-S由两个互补的子集组成。一个是通过升级的渲染管道（SynthWordArt）渲染的，提供高度准确和可控的合成WordArt数据。另一个是通过结合Qwen3-VL进行提示挖掘和Z-Image进行图像合成生成的，提升了现实和多样化数据的覆盖率。在模型方面，我们提出了WATERec。它采用支持任意形状输入的视觉编码器和自回归解码器来建模复杂布局，从结构上打破了固定模板STR在WordArt上的瓶颈。实验表明，该架构在不规则文本（如WordArt）上优于之前的STR方法，在WordArt-Bench上达到90.40%的准确率，超越了通用和OCR专用的视觉-语言模型。代码和数据可在https://github.com/YesianRohn/WATER获取。

View on arXiv Download PDF AI Translation

cs.CV / 78 / 2606.24488

RetiSEM: Generalising Causal Models for Fragmented Biomedical Data

RetiSEM：针对碎片化生物医学数据的因果模型泛化

Ullah, Inam, Razzak, Imran, Jameel, Shoaib

Abstract

Learning causal models from fragmented biomedical data is challenging because clinical, molecular, and imaging variables are often incomplete or not jointly observed. We propose RetiSEM, a domain-constrained structural equation modelling (SEM) framework for causal graph recovery and mediation analysis under limited multimodal resources. This proposed work organises variables into biologically informed blocks, applies forbidden-edge constraints, and decomposes pathway-level effects into TE, NDE, and NIE components. We evaluate RetiSEM across ten synthetic benchmark scenarios that vary in dimensionality, nonlinearity, causal depth, and pathway structure, together with a fragmented real-world setting that combines NHANES clinical variables with externally derived retinal representations. This approach achieves lower structural error and higher causal accuracy than unconstrained baselines across the synthetic benchmarks. In the real-data analysis, retinal variables behave mainly as downstream biomarker-like indicators, with smaller but detectable indirect effects. These findings support our strategy as an interpretable framework for testing structured causal hypotheses in limited-resource biomedical AI. The code and resources for this work are publicly available at: https://github.com/Inamullah-Colab/ReitSEM.

Chinese Translation

从碎片化生物医学数据中学习因果模型具有挑战性，因为临床、分子和影像变量往往是不完整的或未共同观察到的。我们提出了RetiSEM，一个基于领域约束的结构方程模型（SEM）框架，用于在有限的多模态资源下进行因果图恢复和中介分析。该提议的工作将变量组织成生物学上有意义的块，应用禁止边约束，并将路径级效应分解为TE、NDE和NIE组件。我们在十个合成基准场景中评估了RetiSEM，这些场景在维度、非线性、因果深度和路径结构上各不相同，并结合了一个碎片化的真实世界环境，该环境将NHANES临床变量与外部推导的视网膜表示相结合。该方法在合成基准测试中实现了比无约束基线更低的结构误差和更高的因果准确性。在真实数据分析中，视网膜变量主要表现为下游生物标志物样指标，具有较小但可检测的间接效应。这些发现支持我们的策略作为在有限资源生物医学人工智能中测试结构化因果假设的可解释框架。该工作的代码和资源可在以下网址公开获取：https://github.com/Inamullah-Colab/ReitSEM。

View on arXiv Download PDF AI Translation

cs.CV / 79 / 2606.24498

VistaRef: Boosting Visual Spatial Orientation Awareness for Pointing-to-Object Detection

VistaRef：提升指向物体检测的视觉空间定向意识

Li, Ling, Cai, Zhizhen, Wu, Xinkun, Zhu, Ziyu, Lyu, Jiaqing, Liu, Bowen, Deng, Zhidong

Abstract

Grounding deictic gestures in natural images is fundamental to AR and human-robot collaboration, providing a basis for seamless spatial interaction. While Transformer-based visual models have achieved significant progress in general object detection, their global attention mechanisms often neglect micro-geometric relationships, degrading orientation accuracy. In pointing tasks, this deficiency manifests as an inability to accurately capture the pointing ray implied by finger poses, which results in pointing drift and localization ambiguity when dealing with distant or densely packed objects. To address this, we propose VistaRef, a framework designed to explicitly enhance spatial orientation awareness. First, we develop the Local Hand Entity Modeling (LHEM) module, which incorporates hand-pose embeddings to strengthen the model's capability to capture subtle finger deviations. Second, drawing inspiration from multi-view geometry, we construct the Geometric Ray Modeling (GRM) module to transform implicit orientation information into explicit spatial geometric features, guiding feature aggregation and deep fusion via attention mechanisms. Furthermore, we introduce a novel Orientation-Consistent Alignment Loss (OCAL) to synergistically supervise hand presence and pointing consistency, ensuring that all architectural improvements collectively serve the core objective of spatial localization. Experimental results demonstrate that VistaRef significantly outperforms the baseline, achieving a 14-point absolute gain in grounding accuracy. Qualitative analysis further confirms that VistaRef effectively models the geometric correlation from hand to target, bridging the spatial perception gap inherent in traditional Transformers for complex scenarios. Code: https://github.com/lingli1724/VistaRef.

Chinese Translation

在自然图像中将指示性手势与物体结合是增强现实（AR）和人机协作的基础，为无缝的空间交互提供了依据。尽管基于Transformer的视觉模型在一般物体检测方面取得了显著进展，但其全局注意力机制往往忽视微观几何关系，从而降低了定向精度。在指向任务中，这一缺陷表现为无法准确捕捉由手指姿态所暗示的指向光线，导致在处理远距离或密集物体时出现指向漂移和定位模糊。为了解决这一问题，我们提出了VistaRef，一个旨在明确增强空间定向意识的框架。首先，我们开发了局部手部实体建模（Local Hand Entity Modeling, LHEM）模块，该模块结合手势嵌入，增强模型捕捉微妙手指偏差的能力。其次，受多视角几何启发，我们构建了几何光线建模（Geometric Ray Modeling, GRM）模块，将隐式定向信息转化为显式空间几何特征，通过注意力机制引导特征聚合和深度融合。此外，我们引入了一种新颖的定向一致对齐损失（Orientation-Consistent Alignment Loss, OCAL），以协同监督手部存在性和指向一致性，确保所有架构改进共同服务于空间定位的核心目标。实验结果表明，VistaRef显著优于基线，在定位精度上实现了14分的绝对提升。定性分析进一步确认VistaRef有效建模了手部与目标之间的几何关联，弥补了传统Transformer在复杂场景中固有的空间感知差距。代码链接： https://github.com/lingli1724/VistaRef。

View on arXiv Download PDF AI Translation

cs.CV / 80 / 2606.24499

GeoIMO: Geometry-Driven Independent Motion Classification for Event Cameras

GeoIMO：基于几何驱动的事件相机独立运动分类

Gogebakan, Anil Bayram, Marostica, Filippo, Caviglia, Alessio, Savino, Alessandro, Di Carlo, Stefano

Abstract

Existing automotive event datasets rely on appearance-based annotations from frame pipelines, making them poorly suited for motion-aware event perception. We present a geometry-driven, annotation-free framework that classifies detected objects as static or independently moving by exploiting ego-motion structure directly from the event stream. A Focus of Expansion model with yaw compensation estimates global background motion, while objects are labeled as moving when local motion deviates from this prediction, as quantified by a scale-invariant residual. Temporal stabilization improves robustness across consecutive event windows. The method requires no learning, no manual motion labels, and works with any input bounding boxes. Experiments on MVSEC and the Prophesee 1 Megapixel Automotive Detection dataset demonstrate consistent performance across diverse driving scenarios, with yaw compensation improving results during turns and a simple translational local model offering a favorable accuracy-efficiency trade-off.

Chinese Translation

现有的汽车事件数据集依赖于基于外观的帧管道注释，这使得它们不适合运动感知的事件感知。我们提出了一种基于几何驱动的无注释框架，通过直接利用事件流中的自我运动结构，将检测到的物体分类为静态或独立移动。扩展焦点模型（Focus of Expansion）结合偏航补偿估计全局背景运动，当局部运动偏离此预测时，物体被标记为移动，这一偏差通过尺度不变残差量化。时间稳定性增强了连续事件窗口的鲁棒性。该方法不需要学习，无需手动运动标签，并且适用于任何输入边界框。在MVSEC和Prophesee 1百万像素汽车检测数据集上的实验表明，在多种驾驶场景中表现一致，偏航补偿在转弯时改善了结果，而简单的平移局部模型则提供了良好的准确性与效率的权衡。

View on arXiv Download PDF AI Translation

cs.CV / 81 / 2606.24516

What Do Flow-Based Inverse Solvers Approximate? A Posterior-Transport View

基于流的逆解算器近似了什么？后验传输视角

Xu, Jian, Zeng, Delu, Paisley, John, Zhao, Qibin

Abstract

A growing family of training-free solvers -- FlowDPS, FLOWER, PnP-Flow and their diffusion ancestors (DPS, DAPS) -- repurpose a pretrained flow-matching prior to solve imaging inverse problems by adding a measurement-guidance term to the deterministic probability-flow ODE. Despite strong empirical results, what these per-step corrections actually approximate -- and how far the resulting samples are from the true posterior $p(x\mid y)$ -- has not been characterized. We give a posterior-transport account of flow-based inverse problem solving. Our starting point is a simple but consequential fact: for a \emph{deterministic} flow prior, Bayesian conditioning is realized entirely by a \emph{reweighting of the source distribution}, not by a drift correction; pushing the reweighted source through the \emph{unmodified} velocity field yields exact posterior samples. From this we show that trajectory-guidance solvers can be read as the minimum-kinetic-energy \emph{correction} field needed to morph the unconditional source into the posterior, and that FlowDPS / FLOWER / PnP-Flow correspond to distinct zeroth-order / Gaussian / proximal approximations of this single object; we bound the resulting posterior bias in Wasserstein distance. A controlled $2$D study with a closed-form posterior confirms the theory decisively: source reweighting matches the true posterior to the Monte-Carlo floor on every metric, whereas trajectory guidance incurs $200$--$800\times$ larger error and collapses posterior modes, \emph{regardless of guidance strength}. Guided by the analysis we propose a cheap, principled velocity-correction solver that is competitive across two in-domain priors (AFHQ, CelebA) and two out-of-distribution settings while, unlike point-estimate source-space optimizers, producing diverse posterior samples with uncertainty that correlates with reconstruction error.

Chinese Translation

一系列不断增长的无训练解算器——FlowDPS、FLOWER、PnP-Flow及其扩散前身（DPS、DAPS）——通过在确定性概率流常微分方程中添加测量引导项，重新利用预训练的流匹配先验来解决成像逆问题。尽管取得了强有力的实证结果，但这些逐步修正实际上近似了什么，以及由此产生的样本与真实后验 $p(x extmid y)$ 的距离有多远，尚未得到明确的表征。我们给出了基于流的逆问题求解的后验传输解释。我们的出发点是一个简单但重要的事实：对于一个 extit{确定性}流先验，贝叶斯条件化完全是通过 extit{源分布的重加权}来实现的，而不是通过漂移修正；将重加权的源推送通过 extit{未修改}的速度场可以得到准确的后验样本。由此我们表明，轨迹引导解算器可以被视为将无条件源转变为后验所需的最小动能 extit{修正}场，而FlowDPS / FLOWER / PnP-Flow对应于这一单一对象的不同零阶 / 高斯 / 近端近似；我们在Wasserstein距离中界定了由此产生的后验偏差。通过一个具有封闭形式后验的受控二维研究，理论得到了决定性的验证：源重加权在每个指标上都与真实后验匹配到蒙特卡洛基准，而轨迹引导则导致$200$--$800 imes$更大的误差并崩溃后验模式， extit{无论引导强度如何}。在分析的指导下，我们提出了一种廉价、原则性的速度修正解算器，在两个领域内先验（AFHQ、CelebA）和两个分布外设置中具有竞争力，同时，与点估计源空间优化器不同，产生与重建误差相关的不确定性和多样化的后验样本。

View on arXiv Download PDF AI Translation

cs.CV / 82 / 2606.24525

VisCritic: Visual State Comparison as Process Reward for GUI Agents

VisCritic：作为过程奖励的视觉状态比较用于GUI代理

Qian, Jiachen

Abstract

GUI agents powered by vision-language models show strong potential for automating digital tasks, yet frequently fail in long-horizon scenarios due to the absence of step-level verification. Existing process reward models verify actions through textual reasoning alone, missing the visual nature of GUI state changes. We introduce VisCritic, a visual process reward framework that verifies agent actions by directly comparing pre-action and post-action screenshots in visual feature space. VisCritic employs a Siamese vision transformer to extract change-aware representations, coupled with an Action-Aware Critic Head that jointly evaluates action success, task progress, and error type. A critic-training data construction pipeline generates weakly supervised samples from existing trajectories without additional human labels for critic training. Experiments and offline analyses across five benchmarks demonstrate that VisCritic serves as a plug-and-play enhancement for diverse GUI agents, generally improving benchmark metrics while providing visual diagnostic cues.

Chinese Translation

基于视觉-语言模型的GUI代理在自动化数字任务方面展现出强大的潜力，但由于缺乏逐步验证，常常在长时间场景中失败。现有的过程奖励模型仅通过文本推理来验证动作，忽视了GUI状态变化的视觉特性。我们提出了VisCritic，一种视觉过程奖励框架，通过直接比较动作前后的屏幕截图在视觉特征空间中验证代理的动作。VisCritic采用了Siamese视觉变换器来提取关注变化的表示，并结合一个动作感知的评估头共同评估动作成功、任务进展和错误类型。一个评估训练数据构建管道从现有轨迹中生成弱监督样本，无需额外的人类标签用于评估训练。五个基准的实验和离线分析表明，VisCritic作为一种即插即用的增强方案，普遍提高了基准指标，同时提供了视觉诊断线索。

View on arXiv Download PDF AI Translation

cs.CV / 83 / 2606.24538

ForensicsTok: Forensics-Guided Tokenized Modeling for Image Tampering Localization

ForensicsTok：基于取证指导的图像篡改定位的标记化建模

Xu, Lei, Wang, Haowei, Chen, Shen, Yao, Taiping, Li, Bin, Chen, Changsheng

Abstract

Multi-modal Large Language Models (MLLMs) offer powerful reasoning for forensic tasks, yet existing approaches utilizing exogenous segmentation decoders often suffer from suboptimal localization. The reliance on stitched pipelines introduces information bottlenecks during backpropagation, which dilutes spatial signals and is limited by semantic priors of the segmentor. To address these limitations, we propose ForensicsTok, which reformulates image manipulation localization as an autoregressive sequence generation task. ForensicsTok directly generates spatially grounded token sequences, enabling precise mask prediction without intermediary supervision. Specifically, we introduce a Token Splatting Decoder (TSD) to map tokens to binary masks via codebook-aware code smoothing, which mitigates sharp gradients from deterministic detokenizers. Furthermore, to capture diverse tampering clues, we propose a Hierarchical Expert Fusion (HEF) module that injects multi-scale features from a forensic expert model. This unified architecture effectively compensates for the lack of forensic priors in standard MLLMs. Extensive experiments on six benchmarks show that ForensicsTok substantially improves over existing MLLM-based baselines and slightly improves over strong forensic expert baselines, while exhibiting stronger robustness to perturbations.

Chinese Translation

多模态大型语言模型（MLLMs）为取证任务提供了强大的推理能力，但现有利用外部分割解码器的方法往往在定位上表现不佳。依赖拼接管道引入了反向传播过程中的信息瓶颈，稀释了空间信号，并受到分割器语义先验的限制。为了解决这些局限性，我们提出了ForensicsTok，将图像操控定位重新表述为自回归序列生成任务。ForensicsTok直接生成空间上有依据的标记序列，使得在没有中介监督的情况下能够精确预测掩膜。具体而言，我们引入了一种标记溅射解码器（Token Splatting Decoder, TSD），通过代码本感知的代码平滑将标记映射到二进制掩膜，从而减轻了来自确定性去标记器的尖锐梯度。此外，为了捕捉多样的篡改线索，我们提出了一种层次专家融合（Hierarchical Expert Fusion, HEF）模块，该模块从取证专家模型中注入多尺度特征。这一统一架构有效弥补了标准MLLMs中缺乏取证先验的问题。在六个基准测试上的大量实验表明，ForensicsTok在现有基于MLLM的基线上有显著提升，并在强大的取证专家基线之上略有改善，同时表现出更强的对扰动的鲁棒性。

View on arXiv Download PDF AI Translation

cs.CV / 84 / 2606.24539

PointVG-R: Internalizing Geometric Reasoning in MLLMs for Precise Pointing Localization via Visual Chain of Thought

PointVG-R：通过视觉思维链内化几何推理以实现精确的指向定位

Li, Ling, Liu, Bowen, Zhan, Zinuo, Zhong, Jianhui, Zhu, Ziyu, Wei, Bingcai, Chang, Kenglun, Deng, Zhidong

Abstract

Pointing-based visual grounding requires models to precisely locate target objects by deciphering complex spatial relationships between the visual scene and pointing gestures. Traditional methods typically encode input images into static feature representations and perform reasoning primarily within the linguistic domain, often overlooking the rich perceptual cues and explicit spatial geometry inherent in images. In this study, we aim to mitigate the cognitive vulnerability of models in interpreting gestural spatial relations by proposing PointVG-R, a reasoning-guided Multi-modal Large Language Model (MLLM). PointVG-R introduces geometric-aware reasoning for pointing-based grounding, enabling the model to think with images through the strategic integration of Reinforcement Learning (RL) and cold-start data. Specifically, we design a novel geometric reasoning pipeline that simulates the iterative cognitive process humans employ when interpreting pointing gestures. Furthermore, we construct EgoPoint-CoT, a high-quality visual Chain-of-Thought (CoT) dataset featuring detailed reasoning trajectories to guide the model via Supervised Fine-Tuning (SFT) and RL. To address the varying quality of learning signals encountered during training, we further propose an Adaptive Importance Weighting strategy based on Group Variance, which dynamically adjusts reward signals to optimize the learning process. Experimental results demonstrate that PointVG-R achieves SOTA performance, outperforming the baseline by $\textbf{15.86}$ points in mIoU. Extensive ablation studies further validate the efficacy of our proposed modules. Code: https://github.com/lingli1724/PointVG-R.

Chinese Translation

基于指向的视觉定位要求模型通过解读视觉场景与指向手势之间复杂的空间关系，精确定位目标对象。传统方法通常将输入图像编码为静态特征表示，并主要在语言领域内进行推理，往往忽视了图像中固有的丰富感知线索和明确的空间几何。在本研究中，我们旨在通过提出PointVG-R，一个基于推理的多模态大型语言模型（MLLM），来减轻模型在解释手势空间关系时的认知脆弱性。PointVG-R引入了面向几何的推理以支持基于指向的定位，使模型能够通过强化学习（RL）和冷启动数据的战略整合，利用图像进行思考。具体而言，我们设计了一种新颖的几何推理流程，模拟人类在解释指向手势时所采用的迭代认知过程。此外，我们构建了EgoPoint-CoT，一个高质量的视觉思维链（CoT）数据集，包含详细的推理轨迹，通过监督微调（SFT）和RL引导模型。为了解决训练过程中遇到的学习信号质量差异，我们进一步提出了一种基于组方差的自适应重要性加权策略，该策略动态调整奖励信号以优化学习过程。实验结果表明，PointVG-R达到了SOTA性能，在mIoU上超越基线$ extbf{15.86}$分。大量消融研究进一步验证了我们提出的模块的有效性。代码链接： https://github.com/lingli1724/PointVG-R。

View on arXiv Download PDF AI Translation

cs.CV / 85 / 2606.24548

Are Text-to-Image Models Inductivist Turkeys? A Counterfactual Benchmark for Causal Reasoning

文本到图像模型是归纳主义火鸡吗？一个用于因果推理的反事实基准

Lei, Jiayi, Pu, Yuandong, Han, Xingyu, Zhu, Rongpeng, Xu, Jing, Wang, Jinyao, Zhou, Zijian, Fu, Bin, Cao, Yuewen, Liu, Yihao, Li, Yongsheng

Abstract

Text-to-image (T2I) generation models have achieved remarkable progress in producing visually realistic images from natural language prompts. Yet it remains unclear whether their success reflects genuine causal understanding or sophisticated pattern matching over visual-textual correlations. Inspired by Russell's inductivist turkey, we introduce Counterfactual-World (CF-World), a counterfactual benchmark designed to investigate whether text-to-image models can generate images under rules that systematically contradict real-world priors. CF-World organizes each scenario into three progressive levels: factual generation under ordinary world knowledge, explicit counterfactual generation with direct visual instructions, and implicit counterfactual generation requiring causal deduction from altered rules. We evaluate both open-source and closed-source T2I models using a Vision Language Model (VLM)-based evaluator (CF-Eval). Furthermore, we introduce two metrics: Prior Resistance Rate (PRR), which measures a model's ability to overcome entrenched real-world priors, and Reasoning Retention Rate (RRR), which assesses whether models can maintain reasoning-dependent counterfactual generation without explicit visual cues. Experiments show that all models exhibit sharp degradation from factual to counterfactual settings. Further analyses suggest that these failures arise because current T2I models encode world knowledge and visual appearances as tightly coupled patterns. Consequently, their heavy reliance on frequent visual co-occurrences within the training data forces them to default to familiar commonsense priors when tasked with rendering counterfactual worlds.

Chinese Translation

文本到图像（T2I）生成模型在从自然语言提示生成视觉上逼真的图像方面取得了显著进展。然而，目前尚不清楚它们的成功是否反映了真正的因果理解，还是对视觉-文本相关性的复杂模式匹配。受到拉塞尔归纳主义火鸡的启发，我们引入了反事实世界（Counterfactual-World，CF-World），这是一个旨在调查文本到图像模型是否能够在系统性违反现实世界先验的规则下生成图像的反事实基准。CF-World将每个场景组织为三个渐进的层次：在普通世界知识下的事实生成、具有直接视觉指令的显式反事实生成，以及需要从改变的规则中进行因果推理的隐式反事实生成。我们使用基于视觉语言模型（Vision Language Model，VLM）的评估器（CF-Eval）评估开放源代码和闭源的T2I模型。此外，我们引入了两个指标：先验抵抗率（Prior Resistance Rate，PRR），用于衡量模型克服根深蒂固的现实世界先验的能力，以及推理保留率（Reasoning Retention Rate，RRR），用于评估模型在没有明确视觉线索的情况下是否能够维持依赖推理的反事实生成。实验表明，所有模型在事实设置到反事实设置中表现出明显的退化。进一步分析表明，这些失败的原因在于当前的T2I模型将世界知识和视觉外观编码为紧密耦合的模式。因此，它们对训练数据中频繁的视觉共现的高度依赖迫使它们在渲染反事实世界时默认使用熟悉的常识先验。

View on arXiv Download PDF AI Translation

cs.CV / 86 / 2606.24557

Heterogeneous Knowledge Distillation via Geometry Decoupling and Momentum-Aware Gradient Regulation

通过几何解耦和动量感知梯度调节实现异构知识蒸馏

Yang, Wuming, Zhang, Xiang, Zhao, Hongmin

Abstract

Heterogeneous Knowledge Distillation (HKD) aims to transfer knowledge across varying architectures (e.g., from Transformer to CNN) but inherently suffers from severe training instability. We reveal that this instability stems from two highly coupled challenges: massive feature norm discrepancies that cause optimization drag, and severe gradient conflicts between the primary and distillation objectives arising from distinct inductive biases. To achieve stable distillation, we propose SPOFA, a framework built upon a novel Feature and Gradient Dual Stabilization mechanism. Specifically, at the feature level, we introduce a LayerNorm-based decoupling projector that explicitly decouples feature magnitude from direction, creating a bounded and stable space for semantic alignment. At the gradient level, we propose a momentum-driven Exponential Moving Average (MEMA) dynamic scaler. By establishing a robust historical baseline of the optimization trajectory, MEMA actively evaluates instantaneous gradient conflicts and adaptively penalizes harmful distillation signals, guaranteeing stable convergence. Importantly, SPOFA achieves this dual stabilization with an extremely lightweight parameter footprint. Extensive experiments on two mainstream benchmarks demonstrate that SPOFA achieves state-of-the-art accuracy, significantly outperforming computationally expensive methods while introducing only minimal computational overhead compared to standard baselines.

Chinese Translation

异构知识蒸馏（HKD）旨在跨不同架构（例如，从Transformer到CNN）转移知识，但本质上面临严重的训练不稳定性。我们揭示这种不稳定性源于两个高度耦合的挑战：巨大的特征范数差异导致优化拖延，以及由于不同的归纳偏差而产生的主要目标与蒸馏目标之间的严重梯度冲突。为了实现稳定的蒸馏，我们提出了SPOFA，这是一个基于新颖的特征和梯度双重稳定机制的框架。具体而言，在特征层面，我们引入了一种基于LayerNorm的解耦投影器，明确将特征的大小与方向解耦，为语义对齐创造一个有界且稳定的空间。在梯度层面，我们提出了一种基于动量的指数移动平均（MEMA）动态缩放器。通过建立优化轨迹的强健历史基线，MEMA主动评估瞬时梯度冲突，并自适应地惩罚有害的蒸馏信号，从而保证稳定收敛。重要的是，SPOFA以极其轻量的参数占用实现了这种双重稳定性。在两个主流基准上的大量实验表明，SPOFA实现了最先进的准确性，显著优于计算成本高昂的方法，同时与标准基线相比仅引入了最小的计算开销。

View on arXiv Download PDF AI Translation

cs.CV / 87 / 2606.24561

Quantum CT via Dynamic Interval Encoding and Prior-Balanced QUBO Reconstruction

通过动态区间编码和先验平衡 QUBO 重建的量子 CT

Wang, Ao, Yuluo, Yikuang, Liu, Yujie, Zhong, Shuangyang, Zhang, Yuwen, Wang, Zihao, Liu, Fenglin, Maier, Andreas, Yu, Haijun, Huang, Yixing

Abstract

Quadratic unconstrained binary optimization (QUBO)-based quantum computed tomography (CT) casts reconstruction as a binary quadratic problem for quantum annealing and hybrid quantum--classical solvers. For grayscale CT, however, image encoding is constrained by the binary-variable budget: fixed global bit-plane encodings increase QUBO size and coupling complexity as gray-level precision improves, whereas low-bit encodings introduce quantization error. We propose a QUBO-based grayscale CT reconstruction framework that combines dynamic interval encoding with prior-balanced optimization. Each refinement round encodes active pixels only within local gray-level intervals around the current estimate, and a boundary-hit-guided update rule adaptively switches between search expansion and local refinement. To improve optimization stability, the method balances projection-domain data consistency and an edge-preserving quadratic prior before forming the final QUBO. Sparse-view and limited-angle fan-beam CT experiments show that the proposed method recovers structures and gray-level distributions more faithfully than the evaluated analytic, iterative, variational, and representation-based baselines. Expressivity analysis and ablation studies further indicate that the improvement mainly arises from effective gray-level representation through dynamic local encoding and more stable data-fidelity--prior coupling. Experiments on the D-Wave hybrid binary quadratic model (BQM) solver further demonstrate that the formulation is executable on a hardware-backed hybrid quantum--classical backend.

Chinese Translation

基于二次无约束二进制优化（QUBO）的量子计算机断层成像（CT）将重建视为量子退火和混合量子-经典求解器的二次二进制问题。然而，对于灰度 CT，图像编码受到二进制变量预算的限制：固定的全局比特平面编码随着灰度精度的提高而增加 QUBO 的大小和耦合复杂性，而低比特编码则引入量化误差。我们提出了一种基于 QUBO 的灰度 CT 重建框架，该框架结合了动态区间编码和先验平衡优化。每个细化轮次仅在当前估计周围的局部灰度区间内编码活跃像素，并且边界命中引导的更新规则自适应地在搜索扩展和局部细化之间切换。为了提高优化的稳定性，该方法在形成最终 QUBO 之前平衡投影域数据一致性和边缘保留的二次先验。稀疏视图和有限角度扇束 CT 实验表明，所提出的方法比评估的解析、迭代、变分和基于表示的基线更忠实地恢复结构和灰度分布。表现力分析和消融研究进一步表明，改进主要源于通过动态局部编码实现的有效灰度表示和更稳定的数据保真-先验耦合。在 D-Wave 混合二次二进制模型（BQM）求解器上的实验进一步证明，该公式可以在硬件支持的混合量子-经典后端上执行。

View on arXiv Download PDF AI Translation

cs.CV / 88 / 2606.24564

PatternGSL: A Structured Specification Language for Template-Free and Simulation-Ready 3D Garments

PatternGSL：一种无模板且适用于仿真的三维服装结构化规格语言

Li, Zhenyang, Jiang, Lutao, Zhao, Yizhou, Chen, Ying-Cong, Wang, Xin, Chen, Weikai, Peng, Yifan

Abstract

Reconstructing realistic, physically plausible garments from a single image remains a fundamental challenge. Template-free methods capture surface geometry but lack explicit sewing structure for simulation; while programmatic systems are simulation-ready but constrained by predefined templates. This reveals a fundamental representation gap between geometric reconstruction and structured garment construction. We present PatternGSL, a structured garment representation in the form of a template-free and learnable specification language that encodes complete sewing patterns, including panel boundaries, parameterized seams, and explicit stitch topology, in a compact and standardized form. PatternGSL preserves the physical rigor of pattern-based models while removing template dependence, elevating sewing structure as a first-class target for generative modeling. We further propose a vision-language framework that predicts PatternGSL specifications directly from a single image and decodes them into garments using lightweight deterministic validity handling, without optimization-based refinement or manual cleanup. In addition, we introduce PatternGSLData, the first large-scale image-to-GSL paired dataset comprising 300K samples with complete sewing pattern annotations, enabling supervised VLM training for structured garment reconstruction. Experiments demonstrate improved pattern accuracy over prior baselines, explicit sewing-structure recovery, reliable cloth simulation, and pattern-level editing through the same deterministic decoding pipeline. Code and data-processing scripts will be released at https://github.com/PatternGSL/PatternGSL.

Chinese Translation

从单幅图像重建逼真且物理上合理的服装仍然是一个基本挑战。无模板的方法捕捉表面几何形状，但缺乏明确的缝合结构以便于仿真；而程序化系统则适用于仿真，但受到预定义模板的限制。这揭示了几何重建与结构化服装构建之间的基本表示差距。我们提出了PatternGSL，一种以无模板和可学习的规格语言形式表示的结构化服装表示，能够以紧凑且标准化的形式编码完整的缝合模式，包括面板边界、参数化缝合和明确的缝合拓扑。PatternGSL保持了基于模式的模型的物理严谨性，同时消除了对模板的依赖，将缝合结构提升为生成建模的首要目标。我们进一步提出了一种视觉-语言框架，能够直接从单幅图像预测PatternGSL规格，并利用轻量级的确定性有效性处理将其解码为服装，而无需基于优化的细化或手动清理。此外，我们还引入了PatternGSLData，这是第一个大规模的图像到GSL配对数据集，包含30万样本及完整的缝合模式注释，支持结构化服装重建的监督VLM训练。实验表明，相较于之前的基线，模式准确性得到了提升，明确的缝合结构恢复、可靠的布料仿真以及通过相同的确定性解码管道实现的模式级编辑均得到了验证。代码和数据处理脚本将发布在 https://github.com/PatternGSL/PatternGSL。

View on arXiv Download PDF AI Translation

cs.CV / 89 / 2606.24567

Multilevel Stochastic Plug-and-Play for Sparse-View CT Reconstruction

多级随机插拔式方法用于稀视角CT重建

De Paepe, Antoine, Bousse, Alexandre, Visvikis, Dimitris

Abstract

Sparse-view computed tomography (SVCT) reduces radiation exposure and acquisition time, but the limited number of projection views makes the reconstruction problem severely ill-posed and leads to streak artifacts when analytical methods are used. Plug-and-Play (PnP) methods provide an effective way to combine data fidelity with learned image priors, while stochastic PnP methods further improve robustness by matching the denoiser input distribution through re-noising. However, these methods often require many iterations to converge, which limits their practical efficiency. In this work, we propose a multilevel (ML) stochastic PnP method for SVCT that accelerates stochastic PnP reconstruction. We highlight that, in the stochastic setting, directly enforcing prior coherence across levels would require accurately estimating fine-level prior gradients through multiple denoiser function evaluations, which substantially increases the computational cost. Motivated by this observation, we perform the multilevel steps in multiresolution analysis (MRA) approximation spaces. This choice is supported by the structure of the wavelet decomposition, which causes the prior-coherence correction to vanish in expectation, thereby avoiding costly estimation of fine-level stochastic prior gradients for the coarse-level corrections. Experiments on SVCT reconstruction show that our method, called Multilevel Stochastic Plug-and-Play (ML-SPnP), achieves reconstruction quality comparable to state-of-the-art methods while substantially reducing runtime.

Chinese Translation

稀视角计算机断层扫描（SVCT）减少了辐射暴露和采集时间，但有限的投影视角数量使得重建问题严重病态，并在使用解析方法时导致条纹伪影。插拔式（PnP）方法提供了一种有效的方式，将数据保真度与学习到的图像先验相结合，而随机PnP方法通过重新加噪声进一步提高了鲁棒性，以匹配去噪器输入分布。然而，这些方法通常需要多次迭代才能收敛，从而限制了它们的实际效率。在本研究中，我们提出了一种用于SVCT的多级（ML）随机PnP方法，加速随机PnP重建。我们强调，在随机设置下，直接在各级之间强制执行先验一致性将需要通过多次去噪器函数评估准确估计细级先验梯度，这会显著增加计算成本。基于这一观察，我们在多分辨率分析（MRA）近似空间中执行多级步骤。这一选择得到了小波分解结构的支持，该结构使得先验一致性校正在期望中消失，从而避免了对粗级校正进行细级随机先验梯度的高成本估计。对SVCT重建的实验表明，我们的方法，称为多级随机插拔式方法（ML-SPnP），在重建质量上与最先进的方法相当，同时显著减少了运行时间。

View on arXiv Download PDF AI Translation

cs.CV / 90 / 2606.24570

Jolia: Concept-Level Vision-Language Alignment for 3D CT Contrastive Learning

Jolia：基于概念级别的视觉-语言对齐用于3D CT对比学习

Khlaut, Julien, Corbière, Charles, Callard, Baptiste, Prat, Amaury, Butsanets, Leo, Saporta, Antoine, Danielou, Théo, Machado, Leo, Floch, Korentin Le, Boeken, Tom, Manceron, Pierre, Dancette, Corentin

Abstract

Vision-language contrastive pretraining has become the dominant recipe for 3D medical foundation models, leveraging the large volumes of paired scans and reports produced in clinical practice. However, medical images usually span dozens of organs, and radiological reports are much longer than typical natural image captions and are composed of multiple structured sections. CLIP-style pretraining compresses this structure by encoding each modality into a single global token, at the risk of losing important details. We introduce ConQuer (Concept Queries), an image-text pretraining method that augments CLIP's global alignment with a set of localized alignments, one per concept. ConQuer splits the report into concept-specific sections and learns cross-attention queries that pool the matching image features without using any segmentation mask or spatial supervision. Contrastive learning is then applied independently for each concept. Concepts can be any unit of semantic localization; here, they are anatomical regions, one query per organ or gross body region. As a byproduct, each query learns attention maps focused on its concept, providing built-in spatial interpretability. We use ConQuer to train Jolia, a 3D CT foundation model on chest and abdominal CT. Jolia consistently outperforms a CLIP baseline on findings classification, report generation, and cross-center transfer, and sets a new state of the art across multiple public benchmarks. Jolia's weights will be released upon acceptance.

Chinese Translation

视觉-语言对比预训练已成为3D医学基础模型的主流方法，利用临床实践中产生的大量配对扫描和报告。然而，医学图像通常涵盖多个器官，而放射学报告的长度远超典型自然图像的标题，并且由多个结构化部分组成。CLIP风格的预训练通过将每种模态编码为单一全局标记来压缩这一结构，但这可能导致重要细节的丢失。我们提出了ConQuer（概念查询），一种图像-文本预训练方法，它通过一组局部对齐（每个概念一个）增强了CLIP的全局对齐。ConQuer将报告拆分为特定于概念的部分，并学习交叉注意力查询，以在不使用任何分割掩码或空间监督的情况下聚合匹配的图像特征。然后，对每个概念独立应用对比学习。概念可以是任何语义定位的单位；在这里，它们是解剖区域，每个器官或粗大身体区域一个查询。作为副产品，每个查询学习关注于其概念的注意力图，从而提供内置的空间可解释性。我们使用ConQuer训练了Jolia，这是一个针对胸部和腹部CT的3D CT基础模型。Jolia在发现分类、报告生成和跨中心迁移方面始终优于CLIP基线，并在多个公共基准测试中设定了新的最先进水平。Jolia的权重将在接受后发布。

View on arXiv Download PDF AI Translation

cs.CV / 91 / 2606.24586

EERLoss: A Novel Loss Function for Training Deep Biometric Models. A Case Study in Keystroke Dynamics

EERLoss：一种用于训练深度生物识别模型的新型损失函数——以击键动态为例

Gonzalez, Nahuel, Robledo-Moreno, Marta, DeAndres-Tame, Ivan, Vera-Rodriguez, Ruben, Tolosana, Ruben

Abstract

Deep learning approaches to biometric verification are commonly trained by optimizing indirect objectives, creating a misalignment between the optimization process and the primary evaluation metric, typically the Equal Error Rate (EER). This paper introduces EERLoss: a subdifferentiable, arbitrarily accurate approximation to EER for training deep biometric models. Furthermore, this framework has the potential to be adapted to optimize any specific operating point on the DET curve, enhancing its generalizability. To validate this approach, EERLoss is evaluated on a particularly demanding behavioral biometric modality: keystroke dynamics verification. This task is characterized by its high intra-class and low inter-class variability. Experiments are conducted on the large-scale KVC-onGoing benchmark, incorporating data from over 185,000 subjects across different scenarios. A comprehensive ablation study initially demonstrates the superiority of EERLoss in comparison to existing state-of-the-art loss functions. It also converges substantially faster compared to other losses, reducing the overall training cost. Additionally, a comparison is made between the proposed loss and the KVC-winning architecture by re-training it with EERLoss, demonstrating that the proposed approach significantly outperforms the original SoTA, achieving a relative EER reduction of up to approx. 30\%. This improvement on a challenging, large-scale benchmark validates the effectiveness of EERLoss as a task-aligned training objective specifically suited for high-variance biometric traits.

Chinese Translation

深度学习方法在生物识别验证中通常通过优化间接目标进行训练，这导致优化过程与主要评估指标（通常是等错误率（EER））之间存在不一致。本文介绍了EERLoss：一种次微分、任意精度的EER近似，用于训练深度生物识别模型。此外，该框架有潜力被调整以优化DET曲线上的任何特定操作点，从而增强其通用性。为了验证这一方法，EERLoss在一种特别具有挑战性的行为生物识别模式上进行了评估：击键动态验证。该任务的特点是高类内变异性和低类间变异性。实验在大规模KVC-onGoing基准上进行，涵盖了来自不同场景的超过185,000个受试者的数据。全面的消融研究初步表明，EERLoss在与现有最先进损失函数的比较中具有优越性。此外，与其他损失函数相比，EERLoss的收敛速度显著更快，从而降低了整体训练成本。此外，通过使用EERLoss重新训练KVC获胜架构，对所提损失与原始最先进技术（SoTA）进行了比较，结果表明所提方法显著优于原始SoTA，EER相对降低约30%。在这一具有挑战性的大规模基准上的改进验证了EERLoss作为一种与任务对齐的训练目标的有效性，特别适用于高变异性的生物特征。

View on arXiv Download PDF AI Translation

cs.CV / 92 / 2606.24602

ViTexQA: A Multi-Frame Temporal Perception Dataset for Video Text Question Answering

ViTexQA：用于视频文本问答的多帧时间感知数据集

Guo, Zhentao, Duan, Chen, Guan, Tongkun, Wang, Zining, Zhou, Kai, Yan, Pengfei

Abstract

Despite remarkable progress in multimodal understanding, current MLLMs still exhibit limitations in video text understanding, particularly when semantics emerge through the integration of temporally distributed textual cues across multiple frames. This perception challenge fundamentally differs from static image text understanding, yet existing datasets fail to capture: the vast majority of questions remain answerable from single frames, inadequately reflecting real-world video text comprehension demands. To address this, we present ViTexQA, a large-scale video-text QA dataset, and FrameThinker for robust multi-frame temporal reasoning. We build ViTexQA via a quality-controlled Chain-of-Thought (CoT) annotation pipeline boosted with temporal constraints; all its QA pairs demand cross-frame text fusion to solve, enforcing true temporal reliance. FrameThinker adopts two-stage training for explicit temporal modeling: CoT-Guided Supervised Fine-Tuning (SFT) generates frame-aware reasoning chains, followed by Temporally-grounded Reinforcement Learning (RL) optimized with multi-frame coherence rewards. Evaluations show our method outperforms SOTA baselines on ViTexQA, lifting ROUGE-L by 6.3%.

Chinese Translation

尽管多模态理解取得了显著进展，但当前的多模态大语言模型（MLLMs）在视频文本理解方面仍然存在局限性，特别是在语义通过多个帧中时间分布的文本线索的整合而出现时。这一感知挑战与静态图像文本理解根本不同，而现有数据集未能捕捉到这一点：绝大多数问题仍然可以从单一帧中回答，无法充分反映现实世界视频文本理解的需求。为了解决这一问题，我们提出了ViTexQA，一个大规模的视频文本问答数据集，以及FrameThinker用于稳健的多帧时间推理。我们通过一个质量控制的思维链（CoT）注释流程构建ViTexQA，该流程结合了时间约束；所有的问答对都需要跨帧文本融合来解决，从而强制实现真正的时间依赖性。FrameThinker采用两阶段训练以明确进行时间建模：CoT引导的监督微调（SFT）生成帧感知推理链，随后进行基于时间的强化学习（RL），优化多帧一致性奖励。评估结果表明，我们的方法在ViTexQA上优于现有的最先进基线，ROUGE-L提升了6.3%。

View on arXiv Download PDF AI Translation

cs.CV / 93 / 2606.24649

Agentic Collaborative Cognition for Zero-Shot 3D Understanding

面向零样本3D理解的代理协作认知

Wang, Wenxin, Zhang, Bo, Chen, Feng, Wang, Zixuan, Li, Wen, Li, Changsheng, Lei, Yinjie

Abstract

Recent advancements have explored agentic zero-shot 3D understanding by reformulating it as video keyframe understanding with Multimodal Large Language Models (MLLMs). However, existing methods face an intrinsic bottleneck due to the finite observation perspectives inherent in videos and the implicit perception of 3D scenes. In this paper, we propose a collaborative multi-agent framework that assigns a Planning Agent to handle high-level viewpoint planning and supplement novel perspectives, and a Perception Agent to explicitly summarize the 3D scene into a structured holistic cognitive map. Specifically, Planning Agent first analyzes this cognitive map to determine query-relevant viewpoints and supplements missing critical perspectives to ensure comprehensive observation. Subsequently, Perception Agent documents object-level attributes from these views by assigning consistent instance identifiers across viewpoints, thereby integrating fragmented observations into the holistic cognitive map. In parallel, it provides feedback to filter out mismatched candidate objects and guide subsequent viewpoint planning. Through this closed-loop iterative process, two agents collaboratively figure out candidates until Perception Agent determines that sufficient information has been captured to complete the task. Extensive experiments demonstrate that our method achieves state-of-the-art performance on 6 benchmarks, with improvements of 11.1\% [email protected] on ScanRefer, 14.6 BLEU-1 on 3D-assisted dialog, and 2.1 EM on SQA3D.

Chinese Translation

近期的研究进展通过将代理零样本3D理解重新表述为视频关键帧理解，利用多模态大型语言模型（Multimodal Large Language Models, MLLMs）进行探索。然而，现有方法由于视频固有的有限观察视角和对3D场景的隐性感知，面临内在瓶颈。在本文中，我们提出了一种协作多代理框架，该框架分配一个规划代理（Planning Agent）来处理高层次的视角规划并补充新颖的视角，以及一个感知代理（Perception Agent）来明确地将3D场景总结为结构化的整体认知图。具体而言，规划代理首先分析该认知图以确定与查询相关的视角，并补充缺失的关键视角，以确保全面观察。随后，感知代理通过在不同视角之间分配一致的实例标识符，记录这些视角中的对象级属性，从而将碎片化的观察整合到整体认知图中。同时，它提供反馈以过滤不匹配的候选对象，并指导后续的视角规划。通过这一闭环迭代过程，两个代理协作确定候选对象，直到感知代理判断已捕获足够的信息以完成任务。大量实验表明，我们的方法在6个基准测试上达到了最先进的性能，在ScanRefer上提高了11.1\%的[email protected]，在3D辅助对话上提高了14.6的BLEU-1，在SQA3D上提高了2.1的EM。

View on arXiv Download PDF AI Translation

cs.CV / 94 / 2606.24716

Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations

评估带概念注释的稀疏自编码器的可解释性

Klotz, Jonas, Dantas, Cassio F., Jain, Pallavi, Marcos, Diego, Demir, Begüm

Abstract

Sparse autoencoders (SAEs) are increasingly used to extract interpretable concepts from vision and vision language models, yet existing evaluation methods largely rely on proxy metrics or qualitative inspection rather than measuring semantic correspondence. We present a human-grounded evaluation framework that quantifies alignment between SAE latents and human-annotated concepts, without requiring user studies, and validate this matching through targeted attribute perturbations. To enable this intervention-style evaluation in vision, we construct synCUB and synCOCO, synthetic benchmarks of paired images that differ in exactly one attribute. We introduce Fully-Binary Matching Pursuit (FBMP), a coalition-based matching procedure that supports many-to-one mappings between SAE latents and annotated concepts, and consistently outperforms one-to-one baselines. For functional validation, we propose a Targeted Attribute Perturbation Alignment Score (TAPAScore), which tests whether matched concepts respond selectively and in the expected direction under targeted image-level attribute perturbations. Under sanity checks, our matching and TAPAScore are the only evaluated metrics that reliably distinguish trained SAEs from untrained ones. Across SAEs trained on CLIP and DINOv2 embeddings, we find that increased overcompleteness can reduce perturbation alignment, indicating a reduction in interpretability. Our evaluation framework suggests that moderate dictionary sizes provide the best trade-off, yielding the most interpretable SAEs. Code and datasets are available at https://github.com/JonasKlotz/sae-concept-eval.

Chinese Translation

稀疏自编码器（SAEs）越来越多地用于从视觉和视觉语言模型中提取可解释的概念，然而现有的评估方法主要依赖于代理指标或定性检查，而不是测量语义对应关系。我们提出了一种以人为基础的评估框架，该框架量化了SAE潜变量与人类注释概念之间的对齐，而无需用户研究，并通过有针对性的属性扰动验证这种匹配。为了在视觉中实现这种干预式评估，我们构建了synCUB和synCOCO，这两个合成基准由在一个属性上恰好不同的成对图像组成。我们引入了完全二元匹配追踪（Fully-Binary Matching Pursuit, FBMP），这是一种基于联盟的匹配程序，支持SAE潜变量与注释概念之间的多对一映射，并始终优于一对一基线。为了进行功能验证，我们提出了目标属性扰动对齐得分（Targeted Attribute Perturbation Alignment Score, TAPAScore），该得分测试匹配概念在目标图像级属性扰动下是否选择性地以预期方向响应。在合理性检查中，我们的匹配和TAPAScore是唯一能够可靠区分训练过的SAE和未训练SAE的评估指标。在基于CLIP和DINOv2嵌入训练的SAE中，我们发现增加的过度完整性可能会降低扰动对齐，表明可解释性降低。我们的评估框架表明，适中的字典大小提供了最佳的权衡，产生了最可解释的SAE。代码和数据集可在https://github.com/JonasKlotz/sae-concept-eval获取。

View on arXiv Download PDF AI Translation

cs.CV / 95 / 2606.24726

SER: Learning to Ground Video Reasoning with Semantic Evidence Rewards

SER：通过语义证据奖励学习视频推理的基础

Xia, Sheng, Lai, Zhengqin, Jiang, Tianxiang, Tian, Kanghui, Zhou, Shoujun, Li, Bin, Wang, Yi

Abstract

Video MLLMs often struggle with fine-grained spatio-temporal reasoning, sometimes generating correct answers based on irrelevant frames or objects. Although outputting spatio-temporal evidence during reasoning is a promising direction, existing RL frameworks typically rely on geometry-only (IoU) rewards, which can be sensitive to boundary perturbations and overlook semantic alignment. To address this, we propose Semantic Evidence Reward (SER), which reformulates spatio-temporal evidence grounding as a constrained verification task. Instead of computing pixel-level overlap, SER uses a referee VLM as a local checker to evaluate model-generated evidence claims across two dimensions: relevance and localization quality, combined with a temporal penalty. This design reduces the reliance on dense box annotations and enables training directly on standard video QA data. On the V-STAR benchmark, SER achieves 49.6% mLGM, improving by 3.0 points over the strong evidence-grounded baseline Open-o3-Video, demonstrating its potential in enhancing both answer accuracy and evidence grounding.

Chinese Translation

视频多模态大语言模型（Video MLLMs）在细粒度时空推理方面常常面临挑战，有时会基于无关的帧或物体生成正确答案。尽管在推理过程中输出时空证据是一个有前景的方向，但现有的强化学习框架通常依赖于仅几何（IoU）奖励，这可能对边界扰动敏感，并且忽视语义对齐。为了解决这个问题，我们提出了语义证据奖励（Semantic Evidence Reward, SER），将时空证据的基础重构为一个受限的验证任务。SER不再计算像素级重叠，而是使用一个裁判性视觉语言模型（referee VLM）作为局部检查器，从相关性和定位质量两个维度评估模型生成的证据声明，并结合时间惩罚。这一设计减少了对密集框注释的依赖，并使得可以直接在标准视频问答数据上进行训练。在V-STAR基准测试中，SER达到了49.6%的mLGM，相较于强证据基础的基线Open-o3-Video提高了3.0个百分点，展示了其在提高答案准确性和证据基础方面的潜力。

View on arXiv Download PDF AI Translation

cs.CV / 96 / 2606.24737

VSANet: View-aware Sparse Attention Network for Light Field Image Denoising

VSANet：视图感知稀疏注意力网络用于光场图像去噪

Panda, Gargi, Kundu, Soumitra, Bhattacharya, Saumik, Routray, Aurobinda

Abstract

Light field (LF) image denoising is challenging due to the high-dimensional structure of LF data. While noise is independent across sub-aperture images, scene content exhibits strong cross-view correlations. We introduce VSANet, a view-aware sparse attention network for LF denoising. Specifically, we propose a view-aware sparse attention (VSA) block that represents the 4D LF feature map as a unified spatial-angular token space and performs cross-view aggregation via locality-sensitive hashing-based sparse attention. This enables global feature interactions with linear complexity, effectively exploiting LF correlations across views and spatial locations. In addition, we design a feature refinement (FR) block to emphasize informative features in spatial, angular, and epipolar subspaces. The VSA and FR blocks are integrated within a sequential attention refinement module, forming the core of VSANet. Experiments demonstrate VSANet outperforms stateof-the-art LF denoising methods.

Chinese Translation

光场（LF）图像去噪由于LF数据的高维结构而具有挑战性。虽然噪声在子孔径图像之间是独立的，但场景内容表现出强烈的视图间相关性。我们提出了VSANet，一种用于LF去噪的视图感知稀疏注意力网络。具体而言，我们提出了一种视图感知稀疏注意力（VSA）模块，它将4D LF特征图表示为统一的空间-角度令牌空间，并通过基于局部敏感哈希的稀疏注意力进行视图间聚合。这使得全球特征交互具有线性复杂度，有效利用了视图和空间位置之间的LF相关性。此外，我们设计了一个特征精炼（FR）模块，以强调空间、角度和极线子空间中的信息特征。VSA和FR模块集成在一个顺序注意力精炼模块中，构成了VSANet的核心。实验表明，VSANet在LF去噪方法中优于最先进的技术。

View on arXiv Download PDF AI Translation

cs.CV / 97 / 2606.24740

BioMedVR: Confusion-Aware Mixture-of-Prompt Experts for Biomedical Visual Reprogramming

BioMedVR：面向生物医学视觉重编程的混淆感知提示专家模型

Liu, Jiaxiang, Hu, Tianxiang, Guan, Juwei, Wu, Yujie, Wang, Yusong, Mu, Yao, Liu, Zuozhu, Xu, Mingkun

Abstract

Recent advances in vision-language models (VLMs) such as CLIP have demonstrated strong generalization across natural-image domains. However, adapting these models to biomedical imaging is non-trivial: full-model fine-tuning is computationally expensive, while medical data are often scarce and exhibit subtle, fine-grained inter-class differences, making parameter-efficient adaptation particularly critical. Visual Reprogramming (VR) offers a parameter-efficient alternative by injecting learnable perturbations into the input space, but existing VR approaches for VLMs mainly focus on positive class prompts and overlook confusing negatives, leading to miscalibrated predictions in fine-grained medical scenarios. We present BioMedVR, the first VR-based framework for biomedical imaging, enabling few-shot adaptation of pretrained VLMs through compact learnable VR modules. To mitigate class confusion, we introduce a Confusion Minimization Mechanism that leverages LLM-generated confusion-aware attributes together with a Confusion-Suppression Loss to explicitly reduce false-positive alignment. Moreover, the designed Mixture-of-Prompt Experts combines a positive expert for main-class discrimination and a negative expert for confusion suppression, balanced via adaptive gating. Extensive experiments on 18 datasets, including 11 biomedical datasets and 7 natural image benchmarks, demonstrate that BioMedVR achieves superior accuracy and generalization, effectively bridging VR and VLMs in biomedical domains.

Chinese Translation

最近，视觉-语言模型（VLMs）如CLIP的进展显示出在自然图像领域的强大泛化能力。然而，将这些模型适应于生物医学成像并非易事：全模型微调计算成本高昂，而医学数据通常稀缺且存在微妙的细粒度类别差异，使得参数高效的适应尤为重要。视觉重编程（VR）通过向输入空间注入可学习的扰动提供了一种参数高效的替代方案，但现有的VLMs的VR方法主要集中于正类提示，忽视了混淆负类，导致在细粒度医学场景中的预测校准不准确。我们提出了BioMedVR，这是第一个基于VR的生物医学成像框架，通过紧凑的可学习VR模块实现预训练VLMs的少量样本适应。为了解决类别混淆问题，我们引入了一种混淆最小化机制，该机制利用大型语言模型（LLM）生成的混淆感知属性，并结合混淆抑制损失，明确减少假阳性对齐。此外，设计的混合提示专家模型结合了用于主要类别区分的正向专家和用于混淆抑制的负向专家，通过自适应门控进行平衡。在18个数据集上的广泛实验中，包括11个生物医学数据集和7个自然图像基准，证明BioMedVR在准确性和泛化能力上表现优越，有效地将VR与生物医学领域的VLMs结合起来。

View on arXiv Download PDF AI Translation

cs.CV / 98 / 2606.24756

Adaptive Hebbian Memory Routing in Vision Transformers for Few-Shot Learning

视觉变换器中的自适应赫布记忆路由用于少样本学习

Mujawar, Mohammed Yusuf, Golilarz, Noorbakhsh Amiri

Abstract

Few-shot image recognition requires models to adapt to new classes from a small labeled support set. Hebbian fast-weight memory can provide temporary associative information during an episode, but fixed memory behavior may not be appropriate for every few-shot task. In this work, we propose Adaptive Hebbian Routing for few-shot Vision Transformers. The method uses a lightweight MLP router to control the contribution of Hebbian memory, the strength of memory updates, and the retention of previous memory from support-set features. We study Adaptive Placement, Adaptive Plasticity, and Fully Adaptive Hebbian Routing. Experiments use ViT-Small, DeiT-Small, and Swin-Tiny under 5-way 1-shot evaluation on Omniglot, CIFAR-FS, and cross-domain transfer from CIFAR-FS to Omniglot. In the direct Swin comparison, fixed and adaptive Hebbian variants use the same memory location. Adaptive Plasticity improves the fixed Hebbian result from 96.74\% to 96.92\%, while Fully Adaptive Routing achieves the best result at 96.94\%. The fully adaptive Swin model also reduces inference time from 16.51 ms to 14.05 ms relative to fixed Hebbian Swin. On CIFAR-FS, adaptive variants improve performance across all three backbones, and the multi-shot evaluation shows that these gains remain useful as the number of support examples increases. These results show that adaptive plasticity and adaptive memory activation can improve few-shot Transformer representations beyond fixed Hebbian behavior.

Chinese Translation

少样本图像识别要求模型能够从小规模标记支持集适应新类别。赫布快速权重记忆在一个回合中可以提供临时的关联信息，但固定的记忆行为可能并不适合每个少样本任务。在本研究中，我们提出了用于少样本视觉变换器的自适应赫布路由。该方法使用轻量级的多层感知器（MLP）路由器来控制赫布记忆的贡献、记忆更新的强度以及来自支持集特征的先前记忆的保留。我们研究了自适应放置、自适应可塑性和完全自适应赫布路由。实验使用了ViT-Small、DeiT-Small和Swin-Tiny，在Omniglot、CIFAR-FS上的5类1-shot评估以及从CIFAR-FS到Omniglot的跨域迁移。在直接的Swin比较中，固定和自适应赫布变体使用相同的记忆位置。自适应可塑性将固定赫布结果从96.74\%提高到96.92\%，而完全自适应路由则达到了96.94\%的最佳结果。完全自适应的Swin模型还将推理时间从16.51毫秒减少到14.05毫秒，相较于固定赫布的Swin。在CIFAR-FS上，自适应变体在所有三种骨干网络中均提高了性能，而多次评估显示，随着支持样本数量的增加，这些增益依然有效。这些结果表明，自适应可塑性和自适应记忆激活能够改善少样本变换器的表示，超越固定赫布行为。

View on arXiv Download PDF AI Translation

cs.CV / 99 / 2606.24759

UniDrive: A Unified Vision-Language and Grounding Framework for Interpretable Risk Understanding in Autonomous Driving

UniDrive：一个统一的视觉-语言与基础框架，用于自主驾驶中的可解释风险理解

Gao, Xiaowei, Li, Pengxiang, Cheng, Yitai, Xu, Ruihan, Haworth, James, Law, Stephen, Ye, Yun

Abstract

Recent multimodal large language models (MLLMs) have shown strong potential for autonomous driving scene understanding, yet existing methods still face a fundamental trade-off between temporal reasoning and spatial precision. Models that rely on single-frame or low-resolution inputs often miss small, distant, or partially occluded hazards, while language-centric driving models frequently provide limited grounded evidence for their explanations. To address this gap, we propose UniDrive, a unified visual-language and grounding framework for interpretable risk understanding in autonomous driving. UniDrive combines a temporal reasoning branch that models scene dynamics from multi-frame visual input with a high-resolution perception branch that preserves fine-grained spatial details from the latest frame. The two branches are integrated through a gated cross-attention fusion module, enabling dynamic context to be aligned with precise spatial evidence. Based on the fused representation, UniDrive jointly generates natural-language risk descriptions and grounded bounding-box outputs for risk objects. Experiments on the DRAMA-Reasoning benchmark show that UniDrive outperforms representative image-based and video-based baselines in both captioning and risk-object grounding. In particular, UniDrive achieves the best overall performance on the validation split and demonstrates clear advantages in small-object localization, zero-shot generalization to NuScenes and BDD100K, and human-rated interpretability and trustworthiness. These results suggest that explicitly combining temporal semantics and high-resolution perception provides a stronger foundation for interpretable and safety-oriented autonomous driving systems. The code is available at https://github.com/pixeli99/unidrive-dev.

Chinese Translation

近期的多模态大型语言模型（MLLMs）在自主驾驶场景理解方面展现了强大的潜力，但现有方法仍面临时间推理与空间精度之间的基本权衡。依赖单帧或低分辨率输入的模型往往会错过小型、远距离或部分遮挡的危险，而以语言为中心的驾驶模型通常为其解释提供的基础证据有限。为了解决这一问题，我们提出了UniDrive，一个用于自主驾驶中可解释风险理解的统一视觉-语言与基础框架。UniDrive结合了一个时间推理分支，该分支从多帧视觉输入中建模场景动态，以及一个高分辨率感知分支，该分支保留最新帧中的细粒度空间细节。这两个分支通过一个门控交叉注意力融合模块进行集成，使动态上下文与精确的空间证据对齐。基于融合表示，UniDrive联合生成自然语言风险描述和风险对象的基础边界框输出。在DRAMA-Reasoning基准上的实验表明，UniDrive在字幕生成和风险对象基础方面均优于代表性的基于图像和视频的基线模型。特别是，UniDrive在验证集上实现了最佳的整体性能，并在小物体定位、对NuScenes和BDD100K的零样本泛化以及人类评估的可解释性和可信度方面表现出明显优势。这些结果表明，明确结合时间语义和高分辨率感知为可解释和安全导向的自主驾驶系统提供了更强的基础。代码可在 https://github.com/pixeli99/unidrive-dev 获取。

View on arXiv Download PDF AI Translation

cs.CV / 100 / 2606.24767

Compact Object-Level Representations with Open-Vocabulary Understanding for Indoor Visual Relocalization

具有开放词汇理解的紧凑对象级表示用于室内视觉重定位

Cui, Zhaopeng, Hu, Jiarui, Liu, Jingbo, Zhao, Boming, Guo, Xiyue, Feng, Boyin, Peng, Haocheng, Shen, Yujun, Bao, Hujun, Zhang, Guofeng

Abstract

Indoor visual relocalization plays a critical role in emerging spatial and embodied AI applications. However, prior research was predominantly devoted to low-level vision schemes, struggling to perceive scene semantics and compositions, which limits both interpretability and applicability. In this paper, we explore the issue of how to organize rich object information in a scene, including semantics, layout, and geometry, into a structured map representation, thereby utilizing object units exclusively to drive the camera relocalization task. To this end, we propose OpenReLoc, a camera relocalization system designed to provide scene understanding and accurate pose estimation capabilities. Leveraging recent foundation models, we first introduce a multi-modal mechanism to integrate open-vocabulary semantic knowledge for effective 2D-3D object matching. Additionally, we design object-oriented reference frames as position priors, paired with a reference frame selection strategy based on the Distance-IoU (DIOU), enabling extension to scalable scenes. Moreover, to ensure stable and accurate pose optimization, we also propose a dual-path 2D Iterative Closest Pixel loss guided by object shape. Experimental results demonstrate that OpenReLoc achieves superior relocalization recall and accuracy across various datasets. Our source code will be released upon acceptance.

Chinese Translation

室内视觉重定位在新兴的空间和具身人工智能应用中发挥着关键作用。然而，先前的研究主要集中在低级视觉方案上，难以感知场景的语义和构成，这限制了其可解释性和适用性。本文探讨了如何将场景中的丰富对象信息（包括语义、布局和几何）组织成结构化的地图表示，从而仅利用对象单元来驱动相机重定位任务。为此，我们提出了OpenReLoc，一个旨在提供场景理解和准确姿态估计能力的相机重定位系统。借助最新的基础模型，我们首先引入了一种多模态机制，以整合开放词汇的语义知识，实现有效的2D-3D对象匹配。此外，我们设计了以对象为导向的参考框架作为位置先验，并结合基于距离-交并比（Distance-IoU, DIOU）的参考框架选择策略，使其能够扩展到可扩展场景。此外，为了确保稳定和准确的姿态优化，我们还提出了一种由对象形状引导的双路径2D迭代最近像素损失。实验结果表明，OpenReLoc在各种数据集上实现了优越的重定位召回率和准确性。我们的源代码将在接受后发布。

View on arXiv Download PDF AI Translation

cs.CV / 101 / 2606.24774

Revealing Training Data Exposure in Vision Language Large Models via Parameter Gradients

通过参数梯度揭示视觉语言大模型中的训练数据暴露

Zhu, Zhihao, Tang, Hongyi, Yang, Yi, Abbasi, Ahmed

Abstract

Vision-Language Large Models (VLLMs) trained on massive crawled corpora raise pressing copyright and data-provenance concerns. These concerns are particularly acute in healthcare, where patient medical images paired with clinical reports demand rigorous privacy safeguards. However, existing training data detection methods either fail in cross-modal scenarios or rely on superficial output signals with insufficient discriminative power. We introduce GradAudit, a gradient-based auditing framework that examines internal optimization dynamics rather than treating VLLMs as black boxes. Our approach builds on a key observation: model parameters converge to regions where gradients on training samples become stable and well-aligned, whereas gradients on non-training samples remain noisy and inconsistent. By analyzing these gradient signatures, GradAudit achieves strong separability and detects genuine image-text associations learned during training, not merely individual modality membership. Empirically, across both medical and general-domain datasets, GradAudit substantially outperforms state-of-the-art baselines in both pretraining and fine-tuning VLLMs. In a case study employing copyrighted content, we show that existing training data detection methods not only underestimate the extent of unauthorized data usage, but that this underestimation becomes more pronounced as models become more recent and more advanced.

Chinese Translation

在大规模爬取语料库上训练的视觉语言大模型（VLLMs）引发了紧迫的版权和数据来源问题。这些问题在医疗领域尤为突出，因为患者医学图像与临床报告的配对需要严格的隐私保护。然而，现有的训练数据检测方法要么在跨模态场景中失效，要么依赖于表面的输出信号，缺乏足够的区分能力。我们提出了GradAudit，一个基于梯度的审计框架，旨在检查内部优化动态，而不是将VLLMs视为黑箱。我们的方法建立在一个关键观察之上：模型参数收敛到训练样本的梯度变得稳定且高度一致的区域，而非训练样本的梯度则保持噪声和不一致。通过分析这些梯度特征，GradAudit实现了强大的可分离性，并检测到在训练过程中学习到的真实图像-文本关联，而不仅仅是单一模态的归属。实证研究表明，在医疗和一般领域的数据集上，GradAudit在预训练和微调VLLMs方面显著优于最先进的基线。在一个使用版权内容的案例研究中，我们展示了现有的训练数据检测方法不仅低估了未经授权的数据使用程度，而且这种低估在模型变得更为新颖和先进时愈加明显。

View on arXiv Download PDF AI Translation

cs.CV / 102 / 2606.24784

AerialFusionMapNet: Online HD Map Construction with Aerial-Onboard BEV Fusion

AerialFusionMapNet：基于空中与车载鸟瞰视图融合的在线高清地图构建

Lengerer, Daniel, Pechinger, Mathias, Bogenberger, Klaus, Markgraf, Carsten

Abstract

High-resolution aerial imagery has recently emerged as a complementary modality for automated driving perception and has shown potential to improve birds-eye-view (BEV) scene understanding when fused with onboard sensors. Prior work demonstrated performance gains for online high-definition (HD) map construction through aerial-onboard fusion; however, conventional end-to-end fusion does not fully exploit the structural information contained in aerial representations. In this work, we introduce AerialFusionMapNet, a fusion-based mapping framework with a structured two-stage training strategy that explicitly enhances the contribution of aerial features within a unified pipeline. The proposed training scheme enables more effective integration of structural aerial priors. On the nuScenes geographic split, AerialFusionMapNet achieves up to 54.7 mAP, improving over prior aerial-onboard fusion baselines from 48.8 mAP by +5.9 absolute and +12.1% relative. The results suggest that structured training design, rather than increased architectural complexity, plays a more decisive role in unlocking the full potential of aerial imagery for online HD map construction. Code and trained models are available at https://github.com/DriverlessMobility/AerialFusionMapNet.

Chinese Translation

高分辨率的空中影像最近作为自动驾驶感知的补充模式出现，并在与车载传感器融合时显示出改善鸟瞰视图（BEV）场景理解的潜力。之前的研究表明，通过空中与车载融合，在线高清（HD）地图构建的性能得到了提升；然而，传统的端到端融合并未充分利用空中表示中包含的结构信息。在本研究中，我们提出了AerialFusionMapNet，一种基于融合的地图构建框架，采用结构化的两阶段训练策略，明确增强了空中特征在统一管道中的贡献。所提出的训练方案使得结构化空中先验的整合更加有效。在nuScenes地理划分上，AerialFusionMapNet达到了最高54.7 mAP，相较于之前的空中与车载融合基线（48.8 mAP）提高了+5.9绝对值和+12.1%的相对值。结果表明，结构化训练设计，而非增加架构复杂性，在释放空中影像在在线高清地图构建中的全部潜力方面起着更为决定性的作用。代码和训练模型可在https://github.com/DriverlessMobility/AerialFusionMapNet获取。

View on arXiv Download PDF AI Translation

cs.CV / 103 / 2606.24786

Counting Trees from Satellite Imagery with Noisy Supervision

利用噪声监督从卫星图像中计数树木

Gominski, Dimitri, Mugabowindekwe, Maurice, Xu, Qiue, Tong, Xiaowei, Brandt, Martin, Le, Hieu, Fensholt, Rasmus, Samaras, Dimitris, Landrieu, Loic

Abstract

Counting individual trees is a fundamental task for environmental monitoring, yet remains largely unexplored with satellite imagery. At these resolutions, isolated trees may still be identifiable, but crown boundaries become ambiguous in dense forests, making the notion of an individual tree inherently ill-defined. Moreover, large-scale manual annotations of individual trees are prohibitively expensive. While scalable supervision can be derived from airborne LiDAR, the resulting annotations are noisy and difficult to exploit effectively. We address these challenges by formulating tree counting as a spatial density matching problem supervised through Unbalanced Optimal Transport. This formulation naturally accommodates both precise localization of isolate trees and robust density estimation in dense forests. We further introduce a self-correction mechanism that leverages transport residuals to progressively refine noisy supervision during training. We evaluate our approach on TinyTrees, a new benchmark spanning three continents and three satellite sensors, comprising over 215 million tree annotations (including 773K manually verified instances) across 23,000 sq.km. Our method consistently outperforms detection-based, regression-based, and transport-based distribution-matching baselines, demonstrating the effectiveness of unbalanced transport and reliability-aware supervision for large-scale tree counting from satellite imagery. Code, data and models are available at https://github.com/dgominski/treematch.

Chinese Translation

计数单棵树木是环境监测中的一项基础任务，但在卫星图像中仍然未得到充分探索。在这些分辨率下，孤立的树木可能仍然可被识别，但在密集森林中，树冠边界变得模糊，使得单棵树的概念本质上难以界定。此外，单棵树的大规模人工标注成本高昂。尽管可以从机载激光雷达（LiDAR）中获得可扩展的监督，但所得到的标注噪声较大，且难以有效利用。我们通过将树木计数问题表述为一个空间密度匹配问题，并通过不平衡最优传输（Unbalanced Optimal Transport）进行监督，来解决这些挑战。这一表述自然地兼顾了孤立树木的精确定位和密集森林中的稳健密度估计。我们进一步引入了一种自我校正机制，利用传输残差在训练过程中逐步优化噪声监督。我们在TinyTrees这一新基准上评估了我们的方法，该基准覆盖三个大洲和三个卫星传感器，包含超过2.15亿个树木标注（包括77.3万条人工验证实例），覆盖面积达23,000平方公里。我们的方法在检测基础、回归基础和基于传输的分布匹配基线中表现一致优越，证明了不平衡传输和可靠性意识监督在从卫星图像进行大规模树木计数中的有效性。代码、数据和模型可在 https://github.com/dgominski/treematch 获取。

View on arXiv Download PDF AI Translation

cs.CV / 104 / 2606.24796

Pocket-SLAM: Rendering-Area-Aware Pruning for Memory-Efficient 3DGS-SLAM

Pocket-SLAM：基于渲染区域感知的修剪策略以实现内存高效的3DGS-SLAM

Li, Leshu, Peng, Jie, Zhao, Yang

Abstract

3D Gaussian Splatting (3DGS) has garnered significant attention in Simultaneous Localization and Mapping (SLAM) due to its advances in capturing fine-grained geometry features and synthesizing novel views. For SLAM in large-scale scenes, such as autonomous driving, 3DGS-SLAM faces a critical limitation: memory consumption increases continuously over time as Gaussian points accumulate, leading to poor memory efficiency and limiting its applicability. In this work, we propose a rendering-area-aware pruning strategy that selectively removes Gaussians based on their contribution to the effective rendering area, rather than solely relying on Gaussian-level heuristics such as opacity or gradient magnitude. This perspective directly targets the sources of memory redundancy, effectively reducing the peak memory footprint of 3DGS-SLAM during runtime. Evaluations on the EuRoC and KITTI datasets demonstrate that our method consistently outperforms existing pruning approaches in large-scale outdoor scenes, achieving over 60% memory reduction and more than 2 times FPS improvement while preserving localization and mapping accuracy. These results highlight rendering-area-aware pruning as a promising direction for scaling 3DGS-SLAM to real-world autonomous driving scenarios. Our code is publicly available at https://github.com/UMN-ZhaoLab/Pocket-SLAM.git.

Chinese Translation

3D高斯点云渲染（3D Gaussian Splatting, 3DGS）因其在捕捉细粒度几何特征和合成新视图方面的进展而在同时定位与地图构建（Simultaneous Localization and Mapping, SLAM）中受到广泛关注。然而，在大规模场景（如自动驾驶）中的SLAM应用中，3DGS-SLAM面临一个关键限制：随着高斯点的累积，内存消耗不断增加，导致内存效率低下，限制了其适用性。在本研究中，我们提出了一种基于渲染区域感知的修剪策略，该策略根据高斯点对有效渲染区域的贡献选择性地移除高斯点，而不仅仅依赖于透明度或梯度幅度等高斯级启发式方法。这一视角直接针对内存冗余的来源，有效减少了3DGS-SLAM在运行时的峰值内存占用。对EuRoC和KITTI数据集的评估表明，我们的方法在大规模户外场景中始终优于现有的修剪方法，实现了超过60%的内存减少和超过2倍的帧率提升，同时保持了定位和地图构建的准确性。这些结果突显了基于渲染区域感知的修剪作为将3DGS-SLAM扩展到现实世界自动驾驶场景的有前景的方向。我们的代码已公开发布在 https://github.com/UMN-ZhaoLab/Pocket-SLAM.git。

View on arXiv Download PDF AI Translation

cs.CV / 105 / 2606.24797

EG-VQA: Benchmarking Verifiable Video Question Answering with Grounded Temporal Evidence

EG-VQA：基于可验证的时间证据的视听问答基准评估

Huang, Linpeng, Chen, Weixing, Chen, Zexin, Liu, Yang, Lin, Liang

Abstract

Recent advances in Video Large Language Models (Video-LLMs) have yielded promising performance on video question answering (VideoQA). Nevertheless, existing benchmarks are predominantly evaluated through answer correctness, while the grounding of predictions in relevant video evidence remains largely unexamined. This disconnect between answer generation and evidence understanding motivates the construction of the Evidence-Grounded Video Question Answering Benchmark (EG-VQA), an open-ended evaluation protocol in which each QA pair is explicitly annotated with supporting temporal evidence, thereby requiring joint reasoning and precise evidence localization. EG-VQA is comprised of 2,067 videos and 11,838 QA pairs with fine-grained evidence annotations. To evaluate predicted evidence, Evidence-Grounded F1 (EG-F1) is introduced as a unified metric in which temporal alignment and semantic consistency against ground-truth evidence are jointly measured. Experimental evaluation reveals that even strong proprietary models struggle to accurately ground their predictions, exposing a fundamental discrepancy between answer correctness and faithful evidence localization. To bridge this gap, EG-Reasoner, an evidence-grounded reasoning model trained with explicit supervision, is proposed. State-of-the-art performance is achieved among open-source models, with results competitive against proprietary systems, particularly pronounced gains are observed on reasoning-intensive tasks such as counterfactual questions. These findings demonstrate that scaling alone is insufficient for robust video understanding and that structured evidence supervision is essential for the development of more reliable and interpretable VideoQA systems.

Chinese Translation

近期在视频大型语言模型（Video-LLMs）方面的进展在视频问答（VideoQA）中取得了令人鼓舞的表现。然而，现有的基准主要通过答案的正确性进行评估，而预测在相关视频证据中的基础仍然在很大程度上未被检验。这种答案生成与证据理解之间的脱节促使我们构建了证据基础的视频问答基准（EG-VQA），这是一个开放式评估协议，其中每个问答对都明确标注了支持的时间证据，从而要求进行联合推理和精确的证据定位。EG-VQA包含2,067个视频和11,838个具有细粒度证据注释的问答对。为了评估预测的证据，引入了证据基础F1（EG-F1）作为一个统一的度量标准，其中同时测量时间对齐和与真实证据的语义一致性。实验评估显示，即使是强大的专有模型在准确定位其预测方面也面临困难，揭示了答案正确性与真实证据定位之间的基本差异。为了解决这一问题，提出了EG-Reasoner，一个经过显式监督训练的证据基础推理模型。在开源模型中实现了最先进的性能，结果与专有系统具有竞争力，特别是在反事实问题等推理密集型任务上观察到显著的提升。这些发现表明，仅靠规模的扩大不足以实现稳健的视频理解，结构化的证据监督对于开发更可靠和可解释的视频问答系统至关重要。

View on arXiv Download PDF AI Translation

cs.CV / 106 / 2606.24799

OrbitForge: Text-to-3D Scene Generation via Reconstruction-Anchored Video Synthesis

OrbitForge：通过重建锚定的视频合成进行文本到3D场景生成

Fan, Chenrui, Favaro, Paolo

Abstract

Generic text-to-video models can be used as rich open-world scene priors. Despite the high quality of today's generated videos, they do not directly yield reliable 3D assets: camera motion is difficult to control, view coverage is partial, and frames often contain inconsistencies across time. We introduce OrbitForge, an adapter built from frozen video priors and per-prompt Gaussian Splatting reconstruction optimization that converts a single text-generated video into a canonical closed-orbit 3D Gaussian Splatting scene. We use 3D reconstruction as an anchor to improve the 3D consistency of the generated video. We obtain a preliminary 3D reconstruction from a first generated video via Deformable Gaussian Splatting with a robust MedianGS proxy. We render views from a prescribed orbit to detect missing viewpoints. OrbitForge uses the text-to-video model to complete only the missing views, and reconstructs the completed orbit into a final Gaussian Splatting scene. This design requires no task-specific video or multiview fine-tuning, avoids per-prompt score-distillation optimization, and does not progressively generate views one step at a time. We further argue that this setting demands coverage-aware evaluation: local smoothness alone rewards methods that never attempt a full orbit. On a frozen 300-prompt T3Bench-derived audit, OrbitForge reconstruction attains a 359.0-degree measured median span, raises originally unsupported-bin Q10 ImageReward from 8.07 to 16.36 relative to MedianGS-only reconstruction, while remaining competitive with VideoMV on the coverage-quality.

Chinese Translation

通用的文本到视频模型可以作为丰富的开放世界场景先验。尽管如今生成的视频质量很高，但它们并不能直接提供可靠的3D资产：相机运动难以控制，视角覆盖不全面，且帧之间常常存在时间上的不一致性。我们提出了OrbitForge，这是一种基于冻结视频先验和每个提示的高斯喷溅重建优化的适配器，它将单个文本生成的视频转换为标准的闭合轨道3D高斯喷溅场景。我们使用3D重建作为锚点，以提高生成视频的3D一致性。我们通过具有鲁棒性的MedianGS代理，从第一个生成的视频中获得初步的3D重建，采用可变形高斯喷溅技术。我们从规定的轨道渲染视图，以检测缺失的视点。OrbitForge利用文本到视频模型仅补全缺失的视图，并将完成的轨道重建为最终的高斯喷溅场景。该设计不需要特定任务的视频或多视角微调，避免了每个提示的评分蒸馏优化，也不逐步生成视图。我们进一步认为，这种设置需要覆盖感知评估：仅依赖局部平滑性会奖励那些从未尝试完整轨道的方法。在一个冻结的300提示T3Bench衍生审计中，OrbitForge重建达到了359.0度的测量中位跨度，相对于仅使用MedianGS重建的情况，原本不支持的Q10 ImageReward从8.07提高到16.36，同时在覆盖质量方面仍然与VideoMV保持竞争力。

View on arXiv Download PDF AI Translation

cs.CV / 107 / 2606.24805

DDStereo: Efficient Dual Decoder Transformers for Stereo 3D Road Anomaly Detection

DDStereo：高效的双解码器变换器用于立体3D道路异常检测

Mu, Shiyi, Gu, Zichong, Ai, Zhiqi, Gao, Yilin, Xu, Shugong

Abstract

Stereo-based 3D object detection still faces two critical safety challenges: real-time performance and open-set generalization. Existing stereo 3D methods typically achieve twice the accuracy of monocular methods but suffer from significantly lower inference speeds, making them unsuitable for real-time applications. Meanwhile, recent advances in open-world detection have introduced open-set and open-vocabulary algorithms in monocular 2D and 3D settings, yet stereo-based open-set detection remains largely unexplored. To bridge this gap, we propose DDStereo, a novel Dual-Decoder Stereo Transformer for real-time open-set 3D object detection. DDStereo features two lightweight decoder branches: one for open-set foreground 2D detection and the other for 3D attribute regression. These decoders share object-level queries to achieve unified target-level alignment. To enhance inference efficiency, we designed a compact disparity feature extractor and a streamlined decoder architecture. Experiments on public stereo 3D benchmarks demonstrate that DDStereo achieves state-of-the-art accuracy under both closed-set and open-set protocols. Notably, our method surpasses existing stereo 3D detectors in inference speed and, for the first time, achieves real-time performance comparable to monocular approaches.

Chinese Translation

基于立体的3D物体检测仍面临两个关键的安全挑战：实时性能和开放集泛化。现有的立体3D方法通常实现了比单目方法高出两倍的准确率，但推理速度显著较低，使其不适合实时应用。同时，最近在开放世界检测方面的进展已在单目2D和3D设置中引入了开放集和开放词汇算法，但基于立体的开放集检测仍然基本未被探索。为填补这一空白，我们提出了DDStereo，一种新颖的双解码器立体变换器，用于实时开放集3D物体检测。DDStereo具有两个轻量级解码器分支：一个用于开放集前景2D检测，另一个用于3D属性回归。这些解码器共享对象级查询，以实现统一的目标级对齐。为了提高推理效率，我们设计了一个紧凑的视差特征提取器和一个精简的解码器架构。在公共立体3D基准测试上的实验表明，DDStereo在闭集和开放集协议下均实现了最先进的准确率。值得注意的是，我们的方法在推理速度上超越了现有的立体3D检测器，并且首次实现了与单目方法相当的实时性能。

View on arXiv Download PDF AI Translation

cs.CV / 108 / 2606.24817

High-Fidelity Synthetic Transmission Electron Microscopy Image Generation Using Diffusion Probabilistic Models for Data-Limited Semiconductor Metrology

基于扩散概率模型的高保真合成透射电子显微镜图像生成，用于数据有限的半导体计量

Boehm, Johannes, Dey, Bappaditya

Abstract

Advanced semiconductor nodes drastically increased demand for Transmission Electron Microscopy (TEM), yet destructive sample preparation, slow imaging and high costs severely limit the availability of diverse datasets needed for downstream machine learning (ML). Synthetic data generation is becoming essential, but current generative models often miss TEM-specific noise, structural detail, and stochastic variability crucial for evaluation. We present a Denoising Diffusion Probabilistic Model (DDPM) framework for synthetic TEM image generation under extreme data scarcity. A progressive patch-based training strategy scales from low-resolution patches to full images, enabling from-scratch training with only 15 samples. We integrate a custom TrivialAugment adaptation, cross-process domain transfer, classifier guidance, and RePaint-style inpainting, culminating in full-image generation that preserves global structural and spatial relationships in compliance with FAB metrology requirements. Beyond synthesis, we repurpose DDPM feature representations for segmentation, partitioning encoder feature maps to obtain coherent region masks. Our synthetic images achieve up to MS-SSIM > 0.98 and qualitative expert assessment consistent with structural similarity results, facilitating downstream ML training for defect detection, segmentation, and metrology while preserving statistical and physical realism.

Chinese Translation

先进的半导体节点极大地增加了对透射电子显微镜（TEM）的需求，但破坏性样品制备、慢速成像和高成本严重限制了下游机器学习（ML）所需的多样化数据集的可用性。合成数据生成变得至关重要，但当前的生成模型往往忽略了TEM特有的噪声、结构细节和评估所需的随机变异性。我们提出了一种去噪扩散概率模型（Denoising Diffusion Probabilistic Model, DDPM）框架，用于在极端数据稀缺情况下生成合成TEM图像。逐步的基于补丁的训练策略从低分辨率补丁扩展到完整图像，使得仅用15个样本即可进行从零开始的训练。我们集成了自定义的TrivialAugment适配、跨过程领域转移、分类器引导和RePaint风格的修复，最终实现了符合FAB计量要求的全图生成，保持了全局结构和空间关系。除了合成之外，我们还重新利用DDPM特征表示进行分割，将编码器特征图进行分区以获得一致的区域掩码。我们的合成图像在MS-SSIM上达到> 0.98，并且定性专家评估与结构相似性结果一致，为缺陷检测、分割和计量的下游ML训练提供了便利，同时保持了统计和物理现实性。

View on arXiv Download PDF AI Translation

cs.CV / 109 / 2606.24844

Bridging the Manifold Gap: Riemannian Residual Line Search for One-Step Image Editing

弥合流形间隙：用于一步图像编辑的黎曼残差线搜索

Yi, Hongzhu, Luo, Zhongtian, Li, Tong, Fan, Yiyan, Xu, Jungang

Abstract

One-step diffusion editors are fast because they avoid inversion and iterative optimization, but a single transport update must be aggressive enough to realize the target prompt and conservative enough to preserve the source image--and no fixed update strength satisfies both demands across edit types. We treat this tension as a post-hoc candidate-selection problem on top of energy-field transport rather than as a new editing model. Our proposed method, Riemannian Residual Line Search, first builds a stronger edit by estimating the local time curvature of the prompt-delta field and projecting the corrected direction back onto the update norm of the original first-order energy-field transport estimation. It then forms a small residual path from the source image to this strong edit, retains the original first-order output as one candidate, and picks the final image by maximizing target-prompt CLIP alignment. On a 700-sample PIE-Bench++ evaluation across 10 edit type IDs, our method achieves state-of-the-art (SOTA) performance among current one-step update algorithms.

Chinese Translation

一步扩散编辑器之所以快速，是因为它们避免了反演和迭代优化，但单次传输更新必须足够激进以实现目标提示，同时又必须足够保守以保留源图像——而没有固定的更新强度能够满足不同编辑类型的这两种需求。我们将这种紧张关系视为在能量场传输之上的后期候选选择问题，而不是一种新的编辑模型。我们提出的方法，黎曼残差线搜索，首先通过估计提示-增量场的局部时间曲率来构建更强的编辑，并将修正后的方向投影回原始一阶能量场传输估计的更新范数上。然后，它从源图像到这个强编辑形成一条小的残差路径，保留原始的一阶输出作为一个候选，并通过最大化目标提示的 CLIP 对齐来选择最终图像。在对 10 种编辑类型 ID 的 700 个样本进行的 PIE-Bench++ 评估中，我们的方法在当前的一步更新算法中达到了最先进的 (SOTA) 性能。

View on arXiv Download PDF AI Translation

cs.CV / 110 / 2606.24849

IV-CoT: Implicit Visual Chain-of-Thought for Structure-Aware Text-to-Image Generation

IV-CoT：用于结构感知文本到图像生成的隐式视觉思维链

Li, Zixuan, Lin, Haokun, Xiao, Yicheng, Li, Zhiwei, Song, Xinyang, Zheng, Zelong, He, Yong, Yao, Heng, Ding, Ke, Yu, Chao, Yuan, Chuan, Li, Qi, Sun, Zhenan

Abstract

Unified multi-modal large language models (MLLMs) have achieved strong text-to-image generation quality, but still struggle with structure-aware prompt following, where object counts, spatial relations, attribute bindings, and coarse layouts must be preserved. We attribute this limitation in part to the entanglement of structural planning and appearance rendering within a single conditioning stream. To address this issue, we propose Implicit Visual Chain-of-Thought (IV-CoT), a latent visual reasoning framework for query-conditioned image generation. IV-CoT decomposes the visual conditioning queries into a structural-to-semantic cascade, where structural queries first form a latent visual plan and semantic queries then render appearance conditioned on this plan. To guide the structural queries, we introduce training-only sketch supervision, which encourages them to capture structure from sketches without requiring sketch extraction or intermediate decoding at inference time. IV-CoT performs implicit CoT reasoning in a single forward pass and achieves superior results on GenEval and T2I-CompBench. Visualizations and analyses demonstrate that the learned structural and semantic queries play complementary roles in structure-aware generation.

Chinese Translation

统一的多模态大型语言模型（MLLMs）在文本到图像生成质量上取得了显著进展，但在结构感知提示跟随方面仍面临挑战，其中对象数量、空间关系、属性绑定和粗略布局必须得到保持。我们将这一局限性部分归因于结构规划和外观渲染在单一条件流中的纠缠。为了解决这个问题，我们提出了隐式视觉思维链（IV-CoT），这是一种用于查询条件图像生成的潜在视觉推理框架。IV-CoT将视觉条件查询分解为结构到语义的级联，其中结构查询首先形成一个潜在的视觉计划，然后语义查询根据该计划渲染外观。为了引导结构查询，我们引入了仅在训练时使用的草图监督，鼓励它们从草图中捕捉结构，而无需在推理时进行草图提取或中间解码。IV-CoT在单次前向传播中执行隐式思维链推理，并在GenEval和T2I-CompBench上取得了优越的结果。可视化和分析表明，学习到的结构和语义查询在结构感知生成中发挥了互补作用。

View on arXiv Download PDF AI Translation

cs.CV / 111 / 2606.24874

FLUX3D: High-Fidelity 3D Gaussian Generation with Diffusion-Aligned Sparse Representation

FLUX3D：具有扩散对齐稀疏表示的高保真3D高斯生成

Ji, Haorui, Liu, Weizhe, Li, Hongdong, Guo, Hengkai

Abstract

Sparse voxel representation has emerged as a scalable foundation for image-to-3D Gaussian Splatting (3DGS) generation, yet current methods struggle to preserve high-frequency visual details of input images due to two structural bottlenecks. First, they adopt discriminative 2D features optimized for semantic abstraction to construct sparse voxel latents, which suppress reconstructive cues and induce a representation bottleneck. Second, in the generation stage, standard diffusion transformers lack effective mechanisms to align dense 2D image tokens with sparse 3D voxel latents, resulting in a cross-modal correspondence bottleneck. To address these issues, we propose FLUX3D, a scalable image-to-3DGS framework that boosts both representation learning and cross-modal alignment during generation. We first revisit 2D feature selection for sparse-voxel-based 3D representation learning, propose Diffusion-Aligned Structured Latents (DA-SLAT) and couple it with a decoder-only architecture to improve 3DGS reconstruction fidelity. We also design a sparse-structure-aware diffusion framework, which integrates the Sparse-structure Multimodal Diffusion Transformer (SMDiT) and Modal-Aware Rotary Positional Embedding (MARoPE) to achieve geometry-agnostic 2D-3D alignment. Extensive benchmark experiments demonstrate that FLUX3D yields substantial improvements in appearance fidelity and significantly outperforms all state-of-the-art (SOTA) methods in generating high-quality 3DGS assets.

Chinese Translation

稀疏体素表示已成为图像到3D高斯喷溅（3DGS）生成的可扩展基础，但当前方法由于两个结构性瓶颈而难以保留输入图像的高频视觉细节。首先，它们采用针对语义抽象优化的判别性2D特征来构建稀疏体素潜变量，这抑制了重建线索并导致表示瓶颈。其次，在生成阶段，标准扩散变换器缺乏有效机制来对齐稠密的2D图像标记与稀疏的3D体素潜变量，导致跨模态对应瓶颈。为了解决这些问题，我们提出了FLUX3D，一个可扩展的图像到3DGS框架，在生成过程中提升了表示学习和跨模态对齐。我们首先重新审视稀疏体素基础的3D表示学习中的2D特征选择，提出了扩散对齐结构潜变量（Diffusion-Aligned Structured Latents, DA-SLAT），并将其与仅解码器架构结合，以提高3DGS重建的保真度。我们还设计了一个稀疏结构感知的扩散框架，集成了稀疏结构多模态扩散变换器（Sparse-structure Multimodal Diffusion Transformer, SMDiT）和模态感知旋转位置嵌入（Modal-Aware Rotary Positional Embedding, MARoPE），以实现几何无关的2D-3D对齐。大量基准实验表明，FLUX3D在外观保真度上取得了显著提升，并在生成高质量3DGS资产方面显著超越了所有最先进的方法。

View on arXiv Download PDF AI Translation

cs.CV / 112 / 2606.24876

FLAT: Feedforward Latent Triangle Splatting for Geometrically Accurate Scene Generation

FLAT：前馈潜在三角形溅射用于几何准确的场景生成

Kupyn, Orest, Bhat, Goutam, Henzler, Philipp, Manhardt, Fabian, Rupprecht, Christian, Tombari, Federico

Abstract

Generating explorable 3D scenes from a single image requires strong generative priors and accurate geometric representations suitable for downstream use. Current video diffusion models offer high-quality generation and implicitly encode multi-view geometric structure in latent space. However, existing feedforward latent scene decoders typically output volumetric 3D Gaussians that lack a well-defined surface, limiting their use in simulation or standard graphics pipelines. This motivates decoding surface-aligned primitives that are not only renderable but also closer to explicit geometric assets. We ask whether compressed video diffusion latents can be mapped directly to explicit surface primitives in a single pass. To this end, we introduce FLAT and, for the first time, show that triangle splats can be decoded directly from video diffusion latents. Compared with decoding 3D Gaussians, predicting flat primitives is notoriously more challenging due to high sensitivity to primitive orientations, oftentimes leading to poor gradient flow. FLAT solves with two key ingredients: a ray-centered rotation parameterization for triangle regression and a novel product window function that improves gradient flow during differentiable triangle rendering. On standard benchmarks, FLAT achieves significantly better geometric accuracy while maintaining competitive visual quality compared to state-of-the-art feedforward baselines. We further show that a lightweight test-time refinement step converts the predicted triangle soup into a fully opaque, game-engine-ready representation that supports real-time rendering. By evaluating 3DGS, 2DGS, and triangle splatting variants under an identical training setup, we provide the first systematic analysis of representation tradeoffs in feedforward scene generation. The project page is available at https://flat-splat.github.io

Chinese Translation

从单幅图像生成可探索的3D场景需要强大的生成先验和适合下游使用的准确几何表示。目前的视频扩散模型提供高质量的生成，并在潜在空间中隐式编码多视角几何结构。然而，现有的前馈潜在场景解码器通常输出缺乏明确表面的体积3D高斯，这限制了它们在仿真或标准图形管道中的应用。这促使我们解码与表面对齐的原语，这些原语不仅可渲染，而且更接近显式几何资产。我们探讨压缩视频扩散潜在是否可以在一次传递中直接映射到显式表面原语。为此，我们引入FLAT，并首次展示可以直接从视频扩散潜在解码三角形溅射。与解码3D高斯相比，预测平面原语因对原语方向的高度敏感性而 notoriously 更具挑战性，常常导致梯度流不良。FLAT通过两个关键组成部分解决了这一问题：用于三角形回归的射线中心旋转参数化和一种新颖的乘积窗口函数，该函数在可微分三角形渲染过程中改善了梯度流。在标准基准测试中，FLAT在保持竞争性视觉质量的同时，显著提高了几何准确性。我们进一步展示了一种轻量级的测试时精炼步骤，将预测的三角形汤转换为完全不透明、适合游戏引擎的表示，支持实时渲染。通过在相同的训练设置下评估3DGS、2DGS和三角形溅射变体，我们提供了前馈场景生成中表示权衡的首次系统分析。项目页面可访问 https://flat-splat.github.io

View on arXiv Download PDF AI Translation

cs.CV / 113 / 2606.24883

BenchX: Benchmarking AI Models for Cancer Detection and Localization with Demographic and Protocol Biases

BenchX：针对癌症检测和定位的人工智能模型基准测试，考虑人口统计和协议偏差

Chen, Qi, Li, Wenxuan, Bassi, Pedro R. A. S., Zhou, Xinze, Wasserthal, Jakob, Hamamci, Ibrahim Ethem, Er, Sezgin, Kumar, Ashwin, Ye, Yiwen, Wang, Yuhan, Zhou, Yuyin, Chaudhari, Akshay S., Langlotz, Curtis, Wang, Kang, Yang, Yang, Yuille, Alan L., Zhou, Zongwei

Abstract

Artificial intelligence (AI) has achieved remarkable success in medical imaging, but it is widely recognized that these models often perform inconsistently across real-world clinical settings. Such inconsistencies occur when patient demographics and imaging protocols vary, for example, in detecting small tumors, analyzing scans from different contrast phases, or evaluating patients of different ages or sexes. To quantify these inconsistencies, we develop a large-scale, open benchmark of 85,355 CT scans that systematically evaluates 12 tumor-detection AI models across tumor size, location, patient subgroup, and imaging protocol. We leverage large language models (LLMs) to extract and organize subgroup information from clinical data, which makes the analysis both scalable and reproducible. Our benchmark reveals that current state-of-the-art AI models, optimized for average accuracy, perform poorly in rare or underrepresented subgroups, such as young, female African Americans. However, collecting sufficient annotated data for these rare cases is often impractical. The benchmark provides a foundation for building more reliable and robust AI models for tumor detection and highlighting the need for rigorous, subgroup-level evaluation in medical imaging and computer vision. Datasets, code

Chinese Translation

人工智能（AI）在医学影像领域取得了显著成功，但广泛认为这些模型在现实临床环境中的表现往往不一致。这种不一致性发生在患者人口统计特征和影像协议变化时，例如在检测小肿瘤、分析不同对比相位的扫描或评估不同年龄或性别的患者时。为了量化这些不一致性，我们开发了一个大规模的开放基准，包含85,355个CT扫描，系统性地评估12个肿瘤检测AI模型在肿瘤大小、位置、患者子组和影像协议方面的表现。我们利用大型语言模型（LLMs）从临床数据中提取和组织子组信息，使得分析既可扩展又可重复。我们的基准测试揭示了当前最先进的AI模型在优化平均准确率时，在稀有或代表性不足的子组（如年轻的女性非裔美国人）中的表现较差。然而，收集足够的注释数据以应对这些稀有案例往往是不切实际的。该基准为构建更可靠和稳健的肿瘤检测AI模型提供了基础，并强调了在医学影像和计算机视觉中进行严格的子组级评估的必要性。数据集、代码

View on arXiv Download PDF AI Translation

cs.CV / 114 / 2606.24888

DiffusionBench: On Holistic Evaluation of Diffusion Transformers

DiffusionBench：关于扩散变换器的整体评估

Leng, Xingjian, Singh, Jaskirat, Liang, Zhanhao, Smith, Ethan, Bell, Martin, Saha, Aninda, Yuan, Yuhui, Zheng, Liang

Abstract

Diffusion transformer (DiT) research on image generation has converged to a single evaluation setup: class-conditional generation on ImageNet. While methods improve the FID and related metrics, it is increasingly unclear whether they reflect real progress in generative modeling. The natural alternative, i.e., text-to-image (T2I) generation, is perceived as too costly or inconvenient to train and evaluate and is often skipped. We argue that this perception no longer holds. We introduce NanoGen, a unified DiT training and evaluation framework. NanoGen matches state-of-the-art DiT baselines on ImageNet and, with 12 lines of configuration change, also trains competitive text-to-image models. It currently supports RAE, VAE, pixel-space, and MeanFlow diffusion methods under both ImageNet and T2I setups. Under NanoGen, training T2I requires comparable compute to ImageNet. After training 21 latent diffusion models with NanoGen, we observe that method ranking shows no strong correlation between ImageNet and T2I generation: Pearson correlation is between -0.377 and -0.580 across three metrics. This suggests that a method which improves class-conditional ImageNet FID may show no corresponding improvement on T2I, clearly indicating the necessity of evaluating DiTs on both tasks. To this end, we summarize ImageNet and text-to-image results, which yields DiffusionBench, a holistic benchmark for DiT research. We recommend reporting DiffusionBench in place of ImageNet alone: methods that improve DiffusionBench are more likely to reflect broader progress.

Chinese Translation

扩散变换器（DiT）在图像生成方面的研究已经趋向于单一的评估设置：在ImageNet上的类别条件生成。尽管方法在FID及相关指标上有所改善，但越来越不清楚这些指标是否反映了生成建模的真实进展。自然的替代方案，即文本到图像（T2I）生成，被认为训练和评估的成本过高或不便，因此常常被忽略。我们认为这种看法不再成立。我们引入了NanoGen，一个统一的DiT训练和评估框架。NanoGen在ImageNet上与最先进的DiT基线相匹配，并且通过12行配置更改，还能训练出具有竞争力的文本到图像模型。它目前支持RAE、VAE、像素空间和MeanFlow扩散方法，适用于ImageNet和T2I设置。在NanoGen下，训练T2I所需的计算量与ImageNet相当。在使用NanoGen训练了21个潜在扩散模型后，我们观察到方法排名在ImageNet和T2I生成之间没有强相关性：在三个指标上，Pearson相关系数在-0.377到-0.580之间。这表明，改善类别条件ImageNet FID的方法可能在T2I上没有相应的改善，明确指出了在这两项任务上评估DiT的必要性。为此，我们总结了ImageNet和文本到图像的结果，形成了DiffusionBench，一个用于DiT研究的整体基准。我们建议报告DiffusionBench，而不仅仅是ImageNet：改善DiffusionBench的方法更可能反映更广泛的进展。

View on arXiv Download PDF AI Translation

人工智能 (Artificial Intelligence)

cs.AI / 1 / 2606.23927

RIFT-Bench: Dynamic Red-teaming For Agentic AI Systems

RIFT-Bench：面向自主人工智能系统的动态红队评估

Levi, Yarin Yerushalmi, Betser, Roy, Giloni, Amit, Erez, Lidor, Gershon, Itay, Rachmil, Oren, Padakandla, Sindhu, Vainshtein, Roman

Abstract

Agentic AI systems powered by large language models (LLMs) are rapidly evolving into autonomous decision-making systems, exposing attack vectors beyond those of traditional LLM vulnerabilities. Existing security evaluations are often tied to specific implementations or domains, limiting unified comparison across heterogeneous systems. To address this gap, we introduce RIFT-Bench, a graph representation-driven methodology for dynamic red-teaming that enables unified evaluations across diverse agentic architectures. Building on a novel hierarchical representation, RIFT-Bench operates in two automated phases: Discovery, which extracts system structure, and Scanning, which deploys adaptive adversarial attacks and produces a comprehensive evaluation report. It evaluates the examined system itself, leveraging a broad set of dynamically adaptable adversarial probes across diverse attack vectors and objectives. We demonstrate the effectiveness of the proposed evaluation pipeline across 45 agentic systems spanning a diverse range of implementations, showing that the approach generalizes effectively to heterogeneous agentic architectures. Beyond systems and attacks, RIFT-Bench also supports direct evaluation of mitigation strategies. These key capabilities make RIFT-Bench a scalable foundation for security evaluation of agentic AI systems.

Chinese Translation

由大型语言模型（LLMs）驱动的自主人工智能系统正在迅速演变为自主决策系统，暴露出超出传统LLM脆弱性的新攻击向量。现有的安全评估通常与特定实现或领域相关，限制了跨异构系统的统一比较。为了解决这一问题，我们提出了RIFT-Bench，一种基于图表示的动态红队评估方法，能够在多样化的自主架构中进行统一评估。RIFT-Bench基于一种新颖的层次表示，分为两个自动化阶段：发现（Discovery），提取系统结构；扫描（Scanning），部署自适应对抗攻击并生成全面的评估报告。它评估被检查的系统本身，利用一套广泛的动态可调对抗探针，覆盖多种攻击向量和目标。我们展示了所提评估流程在45个自主系统上的有效性，这些系统涵盖了多种实现，表明该方法能够有效地推广到异构自主架构。除了系统和攻击，RIFT-Bench还支持对缓解策略的直接评估。这些关键能力使RIFT-Bench成为自主人工智能系统安全评估的可扩展基础。

View on arXiv Download PDF AI Translation

cs.AI / 2 / 2606.23938

Neuro-Symbolic Drive: Rule-Grounded Faithful Reasoning for Driving VLAs

神经符号驱动：基于规则的可信推理用于驾驶VLA

Gao, Xiangbo, Huang, Xiukun, Lu, Boyu, Zhang, Junge, Mao, Mengjie, Li, Jiachen, Xiong, Wei, Tu, Zhengzhong

Abstract

Driving VLA models incorporating Chain-of-Thought (CoT) reasoning are attractive because they leverage pretrained VLM representations and expose intermediate decisions in natural language, yet current rationales often lack the step-by-step decision semantics needed to keep the rationale causally connected to the planned motion. We introduce Neuro-Symbolic Drive, a neuro-symbolic driving framework that supervises a driving VLA with rule-grounded reasoning traces extracted directly from classical rule-based planners. Our key observation is that rule-based planners are symbolic AI systems that already function as executable reasoning engines: they reason about active safety constraints, search over candidate maneuvers, and select a final trajectory. We instrument these planners in simulation to capture both the executed trajectory and the internal decision trace at each rule-evaluation step. Each trace is serialized into structured rule-grounded reasoning and paired with the trajectory to fine-tune Qwen3.5-4B as a driving VLA. Because these traces are derived directly from the planner states that determine the action, they ensure reasoning is structurally coupled to motion generation by construction, rather than by post-hoc alignment. On our simulator-generated benchmark, detailed rule-grounded reasoning reduces ADE@3s from 0.47 to 0.26 and miss rate from 8.30% to 6.40% under three-camera perception, and from 0.54 to 0.26 and 10.13% to 5.99% under eight-camera perception. Neuro-Symbolic Drive thus converts neuro-symbolic planning logic into structured supervision. Code base: https://github.com/XiangboGaoBarry/Neural-Symbolic-Drive.

Chinese Translation

结合链式思维（Chain-of-Thought, CoT）推理的驾驶VLA模型具有吸引力，因为它们利用预训练的视觉语言模型（VLM）表示，并以自然语言展示中间决策，然而当前的推理往往缺乏逐步决策语义，无法将推理与计划运动保持因果连接。我们提出了神经符号驱动（Neuro-Symbolic Drive），这是一个监督驾驶VLA的神经符号驾驶框架，利用直接从经典基于规则的规划器提取的基于规则的推理痕迹。我们的关键观察是，基于规则的规划器是符号人工智能系统，已经作为可执行推理引擎运作：它们推理关于主动安全约束，搜索候选机动，并选择最终轨迹。我们在仿真中对这些规划器进行工具化，以捕捉执行的轨迹和每个规则评估步骤的内部决策痕迹。每个痕迹被序列化为结构化的基于规则的推理，并与轨迹配对，以微调Qwen3.5-4B作为驾驶VLA。由于这些痕迹直接源自决定行动的规划器状态，因此它们确保推理在结构上与运动生成相耦合，而不是事后对齐。在我们生成的基准测试中，详细的基于规则的推理将三摄像头感知下的ADE@3s从0.47降低到0.26，漏检率从8.30%降低到6.40%；在八摄像头感知下，ADE@3s从0.54降低到0.26，漏检率从10.13%降低到5.99%。因此，神经符号驱动将神经符号规划逻辑转化为结构化监督。代码库：https://github.com/XiangboGaoBarry/Neural-Symbolic-Drive。

View on arXiv Download PDF AI Translation

cs.AI / 3 / 2606.23991

Critique of Agent Model

代理模型的批判

Xing, Eric, Deng, Mingkai, Hou, Jinyu

Abstract

What is an agent? What constitutes agency? With the rise of Large Language Model (LLM) systems marketed as ``coding agents'', ``AI co-scientists'', and other ``agentic" tools that promise to drive up productivity, and at the same time, ``existential" concerns such as AI escaping human control with destructive power under a speculative ``machine agency" against humans, it has become essential to clarify where automation ends and agency begins, both for building capable systems and for understanding whether and what to fear. Drawing on Descartes' grounding of agency in independent thought, and on portrayals of autonomous beings in science fiction, we survey the current landscape of AI agents, and analyze agent architectures along five dimensions: goal, identity, decision-making, self-regulation, and learning. Specifically, we argue that genuine agency requires these structures to be \emph{internalized within the system itself} rather than assembled through external scaffolding. This distinction between \emph{agentic} systems, whose competence resides in engineered workflows, and \emph{agentive} systems, whose capabilities (including social interaction) arise endogenously, defines the boundary between systems designed for prescribed tasks, and those capable of operating in the open world with true autonomy. Building on this analysis, we propose the Goal-Identity-Configurator (GIC) architecture for a general-purpose agent model, combining hierarchical goal decomposition, identity evolution, simulative reasoning grounded in a separately trained world model, learned self-regulation, and self-directed learning from both real and simulated experience. Furthermore, we share insight on the auditability, controllability, and safety of agentive systems that possess greater autonomy and ``agency", but remain under human oversight.

Chinese Translation

什么是代理？什么构成代理性？随着被宣传为“编码代理”、“AI共同科学家”和其他承诺提高生产力的“代理工具”的大型语言模型（LLM）系统的兴起，同时也伴随着诸如AI逃脱人类控制、在投机性“机器代理”下对人类造成破坏性力量的“生存”担忧，澄清自动化的终点与代理的起点变得至关重要，这不仅对于构建有能力的系统是必要的，也有助于理解我们是否以及应该害怕什么。我们借鉴了笛卡尔对代理性在独立思考中的基础的阐述，以及科幻作品中对自主存在的描绘，调查了当前AI代理的现状，并从目标、身份、决策、自治和学习五个维度分析了代理架构。具体而言，我们认为真正的代理性要求这些结构必须是“内化于系统内部”，而不是通过外部支架组装而成。这个“代理”系统的区分，其能力存在于工程化工作流程中，而“代理性”系统的能力（包括社会互动）是内生产生的，定义了为特定任务设计的系统与能够在开放世界中真正自主操作的系统之间的界限。在此分析的基础上，我们提出了通用代理模型的目标-身份-配置器（Goal-Identity-Configurator, GIC）架构，结合了层次化目标分解、身份演变、基于单独训练的世界模型的模拟推理、学习的自我调节，以及从真实和模拟经验中自我导向的学习。此外，我们还分享了关于具有更大自主性和“代理性”的代理系统的可审计性、可控性和安全性的见解，但这些系统仍然在人的监督之下。

View on arXiv Download PDF AI Translation

cs.AI / 4 / 2606.24010

Safe and Generalizable Hierarchical Multi-Agent RL via Constraint Manifold Control

通过约束流形控制实现安全且可泛化的层次化多智能体强化学习

Guo, Zihao, Zhao, Jianing, Li, Ling, Liang, Hao, Loianno, Giuseppe, Du, Yali

Abstract

Multi-agent systems are widely used in safety-critical applications that require coordinated behavior under strict safety constraints. Existing approaches face a fundamental trade-off: learning-based methods achieve strong empirical performance but lack theoretical safety guarantees, while control-theoretic methods enforce safety but often lead to overly conservative and inefficient behaviors. We propose a hierarchical multi-agent reinforcement learning framework that enforces hard safety constraints under mild assumptions at low level via a constraint manifold, while enabling effective coordination through high-level policy learning. Our approach provides theoretical safety guarantees in the multi-agent setting and yields stationary learning dynamics, thereby enabling stable and efficient training. Empirically, our method achieves competitive performance while maintaining nearly perfect safety rates, and generalizes effectively to varying numbers of agents and obstacles.

Chinese Translation

多智能体系统广泛应用于需要在严格安全约束下进行协调行为的安全关键应用中。现有方法面临一个基本的权衡：基于学习的方法在经验性能上表现强劲，但缺乏理论安全保障；而控制理论方法则强制执行安全性，但往往导致过于保守和低效的行为。我们提出了一种层次化多智能体强化学习框架，该框架在低层次通过约束流形在温和假设下强制执行严格的安全约束，同时通过高层次策略学习实现有效的协调。我们的方法在多智能体环境中提供了理论安全保障，并产生稳定的学习动态，从而实现稳定和高效的训练。在实证上，我们的方法在保持几乎完美的安全率的同时，达到了具有竞争力的性能，并有效地对不同数量的智能体和障碍物进行了泛化。

View on arXiv Download PDF AI Translation

cs.AI / 5 / 2606.24014

Reinforcement Learning Towards Broadly and Persistently Beneficial Models

面向广泛且持久有益模型的强化学习

Jagadeesh, Akshay V., Arora, Rahul K., Saab, Khaled, Malik, Ali, Trofimov, Mikhail, Tsimpourlas, Foivos, Heidecke, Johannes, Singhal, Karan

Abstract

As AI systems are deployed across increasingly diverse and high-stakes settings, model alignment must generalize beyond the tasks and domains seen during training. This is especially important for reinforcement learning (RL), which can introduce unexpected misalignment through reward hacking, deception, or other unintended strategies. We study whether RL on beneficial behavior, instantiated in realistic domains, can produce broad and persistent alignment generalization beyond the training distribution. We construct a dataset of realistic situations designed to measure and train beneficial traits, such as truthfulness, fairness, risk awareness, and corrigibility, spanning varied domains, including health, science, and education. We then train models with RL on this dataset and evaluate them on more than 50 independent benchmarks of alignment and beneficial behavior. Compared to a compute-matched baseline, beneficial trait RL improves performance on over 80% of these out-of-distribution benchmarks. We observe substantial out-of-distribution alignment transfer: a beneficial-behavior RL intervention entirely limited to one domain, health, produces broad improvements on non-health alignment evaluations, including reduced reward hacking, deception, and general misalignment. Finally, we study alignment persistence: whether behavior remains robustly aligned under attempts to steer models towards misalignment. Models trained with beneficial trait RL show improved persistence, including greater resistance to adversarial prompting and harmful finetuning; further work is required to isolate the sources of these effects. These results suggest that RL to reinforce beneficial behavior in realistic domains can produce models that are more robustly aligned with human flourishing.

Chinese Translation

随着人工智能系统在日益多样化和高风险的环境中部署，模型对齐必须超越训练期间所见的任务和领域。这对于强化学习（RL）尤其重要，因为它可能通过奖励黑客、欺骗或其他意外策略引入意想不到的失调。我们研究在现实领域中实现的有益行为的强化学习是否能够在训练分布之外产生广泛且持久的对齐泛化。我们构建了一个现实情境的数据集，旨在测量和训练有益特质，例如诚实、公平、风险意识和可纠正性，涵盖健康、科学和教育等多个领域。然后，我们在该数据集上使用强化学习训练模型，并在超过50个独立的对齐和有益行为基准上对其进行评估。与计算匹配的基线相比，有益特质强化学习在超过80%的这些分布外基准上提高了性能。我们观察到显著的分布外对齐转移：一个完全局限于健康领域的有益行为强化学习干预在非健康对齐评估中产生了广泛的改善，包括减少奖励黑客、欺骗和一般失调。最后，我们研究对齐的持久性：即在试图引导模型朝向失调的情况下，行为是否保持稳健对齐。使用有益特质强化学习训练的模型显示出更好的持久性，包括对对抗性提示和有害微调的更大抵抗力；进一步的工作需要隔离这些效应的来源。这些结果表明，在现实领域中强化有益行为的强化学习可以产生与人类繁荣更稳健对齐的模型。

View on arXiv Download PDF AI Translation

cs.AI / 6 / 2606.24026

Can Language Model Agents be Helpful Circuit Explainers in Mechanistic Interpretability?

语言模型代理能否成为机械解释中的有用电路解释者？

Khan, Ayan Antik, Kohli, Harsh, Yao, Yuekun, Sun, Huan, Yao, Ziyu

Abstract

Mechanistic interpretability has made substantial progress in automatically localizing circuits, but explaining what localized components do remains labor-intensive and difficult to standardize. In this work, we study whether language model (LM) agents can assist with this explanation problem once a circuit has already been identified. We introduce AgenticInterpBench, a benchmark for circuit explanation built from 84 semi-synthetic transformer circuits with 163 component-level annotations. We propose HyVE (Hypothesize, Validate, Explain), an agentic explainer that analyzes each component through an iterative loop of observation, hypothesis generation, and causal validation, eventually producing a component-level explanation and a circuit-level task description. Across four LM backbones, HyVE recovers useful component- and task-level explanations, but no backbone is uniformly best. Our analysis shows that strong backbones usually form observation-grounded hypotheses, while failures more often arise later in the validation loop, through incomplete validation plans, code execution errors, or unresolved hypotheses. A case study on an arithmetic circuit in Llama-3-8B shows that the same formulation can extend beyond semi-synthetic benchmarks to naturally trained models. Overall, LM agents are promising circuit explainers, but reliable validation remains the key obstacle.

Chinese Translation

机械解释在自动定位电路方面取得了显著进展，但解释已定位组件的功能仍然劳动密集且难以标准化。在本研究中，我们探讨了语言模型（LM）代理在电路已被识别后是否能够帮助解决这一解释问题。我们引入了AgenticInterpBench，这是一个基于84个半合成变换器电路和163个组件级注释构建的电路解释基准。我们提出了HyVE（假设、验证、解释），这是一种代理解释器，通过观察、假设生成和因果验证的迭代循环分析每个组件，最终生成组件级解释和电路级任务描述。在四种LM基础模型中，HyVE能够恢复有用的组件和任务级解释，但没有哪个基础模型是统一最佳的。我们的分析表明，强大的基础模型通常形成基于观察的假设，而失败更常发生在验证循环的后期，主要是由于验证计划不完整、代码执行错误或假设未解决。对Llama-3-8B中的一个算术电路的案例研究表明，相同的公式可以超越半合成基准，适用于自然训练的模型。总体而言，LM代理是有前景的电路解释者，但可靠的验证仍然是关键障碍。

View on arXiv Download PDF AI Translation

cs.AI / 7 / 2606.24042

Breaking the Filter Bubble: A Semantic Pareto-DQN Framework for Multi-Objective Recommendation

打破过滤气泡：一种用于多目标推荐的语义Pareto-DQN框架

Lopes, Cláudio Lúcio Do Val, da Silva, Lucca Machado, Brandão, André de Oliveira

Abstract

Recommender systems often induce filter bubbles and semantic homogenization by monolithically optimizing for immediate user engagement. Standard single-objective models, including traditional Deep Q-Networks, are ill-equipped to navigate the trade-offs between platform retention and critical societal values like information diversity and provider fairness. To address these limitations, we introduce a multi-objective reinforcement learning framework that formalizes recommendation as a semantic multi-objective Markov decision process. By integrating high-fidelity semantic embeddings with a Pareto-DQN agent, our architecture treats engagement, diversity, and fairness as distinct, non-aggregable reward signals, avoiding the pitfalls of static reward scalarization. Empirical evaluations on the MovieLens small dataset shows that our hypervolume based action selection disrupts the feedback loops responsible for semantic collapse. By sustaining high state-trajectory variance, the Pareto-DQN effectively maps the Pareto frontier, achieving gains in auxiliary societal objectives with only marginal impacts on engagement. This work provides a path toward intrinsically aligned, responsible recommender systems.

Chinese Translation

推荐系统往往通过单一优化用户的即时参与度而导致过滤气泡和语义同质化。标准的单目标模型，包括传统的深度Q网络（Deep Q-Networks），在处理平台留存与信息多样性和提供者公平性等重要社会价值之间的权衡时显得力不从心。为了解决这些局限性，我们提出了一种多目标强化学习框架，将推荐形式化为语义多目标马尔可夫决策过程。通过将高保真度的语义嵌入与Pareto-DQN代理结合，我们的架构将参与度、多样性和公平性视为不同的、不可聚合的奖励信号，避免了静态奖励标量化的陷阱。在MovieLens小数据集上的实证评估表明，我们基于超体积的行动选择打破了导致语义崩溃的反馈循环。通过维持高状态轨迹方差，Pareto-DQN有效地映射了Pareto前沿，在对参与度影响极小的情况下，实现了对辅助社会目标的提升。这项工作为实现内在对齐的负责任推荐系统提供了一条路径。

View on arXiv Download PDF AI Translation

cs.AI / 8 / 2606.24047

Ensemble Feature Selection and Harris Hawks Optimization for Explainable Mental Health Risk Prediction in Female Sex Workers

集成特征选择与哈里斯鹰优化在女性性工作者可解释心理健康风险预测中的应用

Choudhury, Ahnaf Atef, Palash, Md. Parvej Hoque, Ayon, Shahriar Siddique, Saha, Ramkrishna, Mamun, Abdullah Al

Abstract

One of the significant mental health issues affecting female sex workers (FSWs) is mental disorders, especially depression. Exposure to violence, stigma, and economic hardship further increases their psychological risk. Current machine learning (ML) models are typically ineffective at capturing the high-dimensional and complex risk patterns that exist in this marginalized group. This paper suggests a hybrid predictive model that merges an ensemble feature selection strategy using ANOVA and mutual information and Harris Hawks optimization-tuned logistic regression and represents a new application of swarm intelligence to predict mental health in vulnerable groups. The explainable AI (XAI) methods can be used to understand the factors of trauma associated with model predictions. When applied to a group of 3,005 FSWs, it can be seen that the proposed model is more effective than traditional classifiers, with an accuracy of 95.78%, an F1 score of 95.77%, and an AUC of 0.96, and identifying post-traumatic stress, client-related violence, and occupational factors as major contributors to depression. This work bridges the gaps between conventional and ML approaches to develop an XAI tool that enables vulnerable groups to receive early assistance, evidence-based targeted psychosocial care, and health planning.

Chinese Translation

影响女性性工作者（FSWs）的一个重要心理健康问题是心理障碍，尤其是抑郁症。暴力、污名和经济困境的暴露进一步增加了她们的心理风险。目前的机器学习（ML）模型通常无法有效捕捉这一边缘化群体中存在的高维和复杂风险模式。本文提出了一种混合预测模型，结合了使用方差分析（ANOVA）和互信息的集成特征选择策略以及经过哈里斯鹰优化调优的逻辑回归，代表了群体智能在脆弱群体心理健康预测中的新应用。可解释人工智能（XAI）方法可以用来理解与模型预测相关的创伤因素。在应用于3,005名女性性工作者的群体时，结果显示所提出的模型比传统分类器更有效，准确率为95.78%，F1得分为95.77%，AUC为0.96，并将创伤后应激、客户相关暴力和职业因素识别为抑郁症的主要影响因素。本研究弥合了传统方法与机器学习方法之间的差距，开发了一种可解释的人工智能工具，使脆弱群体能够获得早期帮助、基于证据的针对性心理社会护理和健康规划。

View on arXiv Download PDF AI Translation

cs.AI / 9 / 2606.24064

Beyond Trajectory Imitation: Strategy-Guided Policy Optimization for LLM Reasoning

超越轨迹模仿：基于策略的政策优化用于大语言模型推理

Shi, Tianyuan, Huang, Canbin, Li, Bei, Chen, Xin, Quan, Xiaojun, Wang, Jingang, Wang, Qifan

Abstract

Distilling reasoning capabilities from strong to weak language models typically involves imitating specific solution trajectories, effectively transferring what to answer rather than how to reason. This trajectory-level imitation encourages memorization of instance-specific steps rather than acquisition of transferable problem-solving skills, limiting generalization to novel problems. We propose Strategy-Guided Policy Optimization (SGPO), which replaces instance-level trajectory imitation with reusable strategy distillation. SGPO extracts structured strategy descriptions from strong-model responses and, for each problem, constructs both autonomous and strategy-guided trajectories to enable direct comparison of the model's behavior with and without strategic guidance. The framework then addresses two key questions. For how to distill, a token-level forward-KL objective selectively transfers the distributional shift induced by strategy conditioning into the unguided policy, with proximal constraints ensuring stability. For when to distill, adaptive instance-level weighting strengthens guidance when autonomous exploration falls short and reduces it as the model's own competence grows. Experiments on four mathematical benchmarks across two model families show that SGPO consistently outperforms SFT, on-policy RL, and hybrid-policy baselines, improving the average score by 2.2 points over the strongest baseline on Qwen2.5-7B-Instruct. Analysis reveals that the forward-KL objective provides an inherently selective distillation signal that outperforms direct trajectory imitation, and that strategy distillation exhibits complementary scaling with base model capability.

Chinese Translation

从强语言模型提炼推理能力到弱语言模型通常涉及模仿特定的解决轨迹，有效地转移了回答内容而非推理方式。这种轨迹级别的模仿鼓励记忆特定实例的步骤，而非获取可转移的问题解决技能，从而限制了对新问题的泛化能力。我们提出了基于策略的政策优化（Strategy-Guided Policy Optimization, SGPO），该方法用可重用的策略提炼替代了实例级的轨迹模仿。SGPO 从强模型的响应中提取结构化的策略描述，并为每个问题构建自主和基于策略的轨迹，以便直接比较模型在有无策略指导下的行为。该框架随后解决了两个关键问题。关于如何提炼，基于标记的前向KL目标选择性地将策略条件引起的分布转移转移到无指导政策中，同时通过近端约束确保稳定性。关于何时提炼，适应性实例级加权在自主探索不足时增强指导，并在模型自身能力提高时减少指导。对两个模型系列的四个数学基准的实验表明，SGPO 在 Qwen2.5-7B-Instruct 上的平均得分比最强基线提高了 2.2 分，始终优于 SFT、在线强化学习和混合政策基线。分析表明，前向KL目标提供了一种固有的选择性提炼信号，优于直接轨迹模仿，并且策略提炼与基础模型能力表现出互补的扩展性。

View on arXiv Download PDF AI Translation

cs.AI / 10 / 2606.24099

Exploring Academic Influence of Algorithms by Co-occurrence Network Based on Full-text of Academic Papers

基于学术论文全文的共现网络探索算法的学术影响力

Wang, Yuzhuo, Zhang, Chengzhi, Song, Min, Kim, Seong Deok, Ko, Youngsoo, Lee, Juhee

Abstract

Algorithms have become central to scientific research in the era of artificial intelligence (AI). Although algorithm mentions in papers are often used to indicate popularity and influence, existing studies usually evaluate individual algorithms in isolation and pay limited attention to the collective influence formed through their interconnections. This study constructs large-scale algorithm co-occurrence networks in natural language processing (NLP) based on the full text of academic papers and investigates algorithm influence from a network perspective. Using deep learning models, we extract algorithm entities and build overall, cumulative, and annual co-occurrence networks. We analyze their structural characteristics and apply multiple centrality measures to assess the group influence of algorithms across the whole field and over time. The results show that algorithm networks display typical features of complex networks, with increasingly dense connections developing over approximately two decades. Classic, high-performing algorithms and those located at the intersections of different research periods tend to have high popularity, control, centrality, and balanced influence. When the influence of an algorithm declines, it usually loses its core network position first, followed by weaker associations with other algorithms. This study is the first large-scale analysis of algorithm co-occurrence networks. Covering more than four decades of academic publications, it provides a temporal and structural view of algorithm influence and offers a foundation for future research on networks linking algorithms, scholars, and tasks.

Chinese Translation

在人工智能（AI）时代，算法已成为科学研究的核心。尽管论文中对算法的提及通常用于指示其流行度和影响力，但现有研究往往孤立地评估单个算法，且对其相互关联所形成的集体影响关注有限。本研究基于学术论文的全文构建了自然语言处理（NLP）领域的大规模算法共现网络，并从网络的角度探讨算法的影响力。通过深度学习模型，我们提取了算法实体，并构建了整体、累积和年度共现网络。我们分析了这些网络的结构特征，并应用多种中心性度量来评估算法在整个领域及其随时间变化的群体影响力。结果表明，算法网络展现出复杂网络的典型特征，连接在大约二十年内日益密集。经典的高性能算法以及位于不同研究时期交叉点的算法往往具有较高的流行度、控制力、中心性和均衡影响力。当一个算法的影响力下降时，通常首先失去其核心网络位置，随后与其他算法的关联也变得较弱。本研究是对算法共现网络的首次大规模分析，涵盖了超过四十年的学术出版物，为算法、学者和任务之间的网络未来研究提供了时间和结构视角的基础。

View on arXiv Download PDF AI Translation

cs.AI / 11 / 2606.24112

ReMMD: Realistic Multilingual Multi-Image Agentic Verification for Multimodal Misinformation Detection

ReMMD：用于多模态虚假信息检测的现实多语言多图像代理验证

Dang, Chenhao, Zhu, Dantong, Yang, Jun, He, Conghui, Li, Weijia

Abstract

Multimodal misinformation detection is increasingly important because viral posts now combine long multilingual narratives, several images, mixed provenance, and subtle text--image framing errors. Existing benchmarks and methods remain poorly matched to this setting: they usually isolate short captions, single images, binary labels, or one manipulation source, while agentic verification remains costly under realistic evidence search. We present ReMMD, a realistic multilingual multi-image agentic verification framework for multimodal misinformation detection. ReMMD includes ReMMDBench, a real-world multimodal misinformation detection benchmark with 500 samples, 2,756 images, five monolingual languages, two cross-lingual settings, three text-length tiers, multi-image posts, five-way veracity labels, eight distortion labels, evidence provenance, and rationales. It also includes ReMMD-Agent, a persistent-memory verifier that decomposes posts into atomic points, builds a reusable evidence set, and predicts structured L1/L2/L3 outputs. Across proprietary systems, open LVLMs, MMD-Agent, and T2-Agent, ReMMD-Agent obtains the best five-way veracity performance, with 41.80% accuracy and 39.12% macro-F1 using GPT-5.2, while reducing cost by 17.5% relative to MMD-Agent and 79.9% relative to T2-Agent. The project is available at https://dang-ai.github.io/ReMMD.

Chinese Translation

多模态虚假信息检测变得越来越重要，因为病毒式传播的帖子现在结合了长篇多语言叙述、多个图像、混合来源以及微妙的文本-图像框架错误。现有的基准和方法与这一背景匹配不佳：它们通常孤立短标题、单一图像、二元标签或单一操控来源，而在现实证据搜索中，代理验证的成本仍然很高。我们提出了ReMMD，一个用于多模态虚假信息检测的现实多语言多图像代理验证框架。ReMMD包括ReMMDBench，这是一个包含500个样本、2756张图像、五种单语语言、两种跨语言设置、三个文本长度层级、多图像帖子、五种真实性标签、八种失真标签、证据来源和理由的真实世界多模态虚假信息检测基准。它还包括ReMMD-Agent，一个持久记忆验证器，能够将帖子分解为原子点，构建可重用的证据集，并预测结构化的L1/L2/L3输出。在专有系统、开放的LVLM、MMD-Agent和T2-Agent中，ReMMD-Agent在五种真实性性能上表现最佳，使用GPT-5.2时准确率达到41.80%，宏观F1值为39.12%，同时相较于MMD-Agent降低了17.5%的成本，相较于T2-Agent降低了79.9%的成本。该项目可在https://dang-ai.github.io/ReMMD获取。

View on arXiv Download PDF AI Translation

cs.AI / 12 / 2606.24124

VeryTrace: Verifying Reasoning Traces through Compilable Formalism and Structured Verification

VeryTrace：通过可编译形式化和结构化验证验证推理轨迹

Zhong, Ninghan, Tanriverdi, Ahmet Ege, Kale, Kaan, Vishwanath, Sriram

Abstract

Multi-step reasoning with Chain-of-Thought (CoT) prompting remains fragile: logical errors or hallucinations in early steps silently propagate, producing confident but incorrect conclusions. This paper presents VeryTrace, a zero-shot verification-and-repair framework that formalizes natural-language reasoning traces into a structured, compilable representation. VeryTrace introduces a Domain-Specific Language (DSL) that (i) makes step dependencies explicit, (ii) mechanizes quantitative content as executable expressions, and (iii) structures semantic inferences via deduction schemas. Our hybrid verifier combines deterministic checks for computational correctness, dependency resolution, and constraint satisfaction with targeted LLM audits for non-mechanizable semantic judgments, enabling step-level error localization and repair. Across three diverse domains-competition mathematics (AIME 2025), robotics planning (LLM-BabyBench), and kinship reasoning (CLUTRR), VeryTrace improves accuracy over zero-shot baselines on state-of-the-art LLMs without requiring domain-specific training or in-context examples, demonstrating that formalized trace verification achieves both precision and generalization.

Chinese Translation

多步骤推理与链式思维（Chain-of-Thought, CoT）提示仍然脆弱：早期步骤中的逻辑错误或幻觉会悄然传播，产生自信但不正确的结论。本文提出了VeryTrace，一个零-shot验证与修复框架，将自然语言推理轨迹形式化为结构化的可编译表示。VeryTrace引入了一种领域特定语言（Domain-Specific Language, DSL），该语言（i）明确步骤之间的依赖关系，（ii）将定量内容机械化为可执行表达式，以及（iii）通过推理模式结构化语义推理。我们的混合验证器结合了对计算正确性的确定性检查、依赖关系解析和约束满足，以及针对不可机械化语义判断的目标大语言模型（LLM）审计，从而实现了逐步错误定位和修复。在三个不同领域——竞争数学（AIME 2025）、机器人规划（LLM-BabyBench）和亲属推理（CLUTRR）中，VeryTrace在不需要领域特定训练或上下文示例的情况下，提升了在最先进的大语言模型上的准确性，证明了形式化轨迹验证在精确性和泛化性方面的有效性。

View on arXiv Download PDF AI Translation

cs.AI / 13 / 2606.24129

OmniPath: A Multi-Modal Agentic Framework for Auditing Wheelchair Accessibility

OmniPath：一种多模态代理框架用于审计轮椅无障碍性

Hossain, ASM Mobarak, Mahmud, Nadim, Raychoudhury, Vaskar, Gani, Md Osman

Abstract

For a wheelchair user, a standard blue line on a map is often a broken promise. While platforms like OpenStreetMap (OSM) successfully capture where a path is, they frequently fail to convey how it physically feels to travel on it. This information barrier is problematic for wheelchair users. To solve this issue, we present OmniPath, a system that moves from passive mapping to proactive environmental auditing. Our framework fuses the network topology of OSM with the submeter precision of high-density aerial LiDAR (USGS 3DEP) to create a high-fidelity 3D model of the pedestrian environment. Rather than simply routing a user, our agent virtually traverses the network, analyzing the surface in 0.5 meter increments. It rigorously quantifies physical friction points specifically running slope, cross slope, and vertical discontinuities against ADA compliance standards, calculating a weighted severity score to categorize hazards from ``Mild'' to ``Critical.'' To ensure real world reliability, we validated the system against 200 physical ground truth field surveys across the National Mall using stratified random sampling. The framework demonstrated strong diagnostic reliability for high-severity hazards, achieving F1-scores of 0.60 for Severe and 0.58 for critical categories. By automating this micro-scale inspection, OmniPath identifies the ``invisible'' barriers that standard maps miss, effectively transforming a static dataset into accessibility data source that anticipates accessibility challenges before the user ever leaves home.

Chinese Translation

对于轮椅用户来说，地图上的标准蓝线往往是一种破碎的承诺。尽管像OpenStreetMap (OSM)这样的平台成功地捕捉了路径的位置，但它们常常无法传达在该路径上行走的实际感受。这一信息障碍对轮椅用户来说是一个问题。为了解决这个问题，我们提出了OmniPath，一个从被动制图转向主动环境审计的系统。我们的框架将OSM的网络拓扑与高密度航空激光雷达（USGS 3DEP）的亚米级精度相结合，创建了一个高保真度的行人环境三维模型。我们的代理不仅仅是为用户规划路线，而是虚拟地遍历网络，以0.5米的增量分析表面。它严格量化物理摩擦点，特别是运行坡度、横坡度和垂直不连续性，并与美国残疾人法案（ADA）合规标准进行对比，计算加权严重性评分以将危险分类为“轻微”到“严重”。为了确保现实世界的可靠性，我们通过分层随机抽样对全国广场的200个物理真实场地调查进行了系统验证。该框架在高严重性危险方面表现出强大的诊断可靠性，严重类别的F1得分达到0.60，关键类别的F1得分为0.58。通过自动化这种微观规模的检查，OmniPath识别出标准地图遗漏的“隐形”障碍，有效地将静态数据集转变为能够预见无障碍挑战的无障碍数据源，帮助用户在离家之前就能了解潜在的无障碍问题。

View on arXiv Download PDF AI Translation

cs.AI / 14 / 2606.24145

T2D-Bench: Evidence-Gated Evaluation of LLM Outputs for Type 2 Diabetes Using a Multi-Layer Clinical-Lifestyle Knowledge Graph

T2D-Bench：基于证据的二型糖尿病大型语言模型输出评估框架，利用多层临床-生活方式知识图谱

Farahani, Saba A., Cao, Hung, Jain, Ramesh, Rahmani, Amir M.

Abstract

Large language models (LLMs) can produce clinically fluent recommendations for type 2 diabetes while failing to satisfy guideline constraints or explicitly justify lifestyle-related glycemic claims. We present T2D-Bench, a reproducible benchmark and evidence-gated evaluation framework for testing whether LLM outputs satisfy explicit, graph-checkable evidence requirements. T2D-Bench is built on a multi-layer clinical-lifestyle knowledge graph that combines a biomedical spine (UMLS, DrugBank, SIDER), computable ADA Standards of Care rules, and lifestyle knowledge connected through a mechanistic bridge to glycemic laboratory effects. Across 100 structured vignettes spanning diagnosis, medication safety, and adversarial lifestyle conflicts, baseline outputs failed benchmark-defined evidence-path checks in 35% of cases for GPT-4o-mini and 33% for GPT-4o. The evidence gate detects unsupported omissions and uses constrained revision to bring outputs into verifier-level compliance with benchmark-defined evidence requirements. These results show that computable evidence constraints can make unsupported clinical omissions explicit, measurable, and correctable in diabetes-focused LLM outputs.

Chinese Translation

大型语言模型（LLMs）能够为二型糖尿病提供临床流畅的建议，但往往未能满足指南约束或明确证明与生活方式相关的血糖主张。我们提出了T2D-Bench，这是一个可重复的基准和基于证据的评估框架，用于测试LLM输出是否满足明确的、可通过图谱检查的证据要求。T2D-Bench建立在一个多层临床-生活方式知识图谱之上，该图谱结合了生物医学主干（UMLS、DrugBank、SIDER）、可计算的ADA护理标准规则，以及通过机制桥梁连接到血糖实验室效应的生活方式知识。在涵盖诊断、药物安全性和对抗性生活方式冲突的100个结构化案例中，基线输出在35%的情况下未能通过基准定义的证据路径检查（GPT-4o-mini）和33%（GPT-4）。证据门检测到不支持的遗漏，并通过受限修订使输出符合基准定义的证据要求。这些结果表明，可计算的证据约束能够使不支持的临床遗漏在以糖尿病为重点的LLM输出中变得明确、可测量和可纠正。

View on arXiv Download PDF AI Translation

cs.AI / 15 / 2606.24157

The Geometry Behind Diffusion and Flow Matching: Gradient Flows and Geodesics in Wasserstein Space

扩散与流匹配背后的几何：Wasserstein空间中的梯度流与测地线

Yao, Yian, Zhang, Weiwei

Abstract

The space $\mathcal{P}_2(\mathbb{R}^d$) of probability measures with finite second moment carries a natural geometry: the quadratic Wasserstein distance W_2 makes it a complete metric space and, following Otto, a (formal) Riemannian manifold whose geodesics are the optimal-transport interpolations. On this manifold, the gradient flow of the free energy F(rho) = KL(rho || \pi) is exactly the Fokker-Planck equation, and its implicit-Euler discretization is the JKO scheme. This is the geometry underlying diffusion models: the forward process descends the free energy, and each denoising step realizes one JKO step, which recovers DDPM, DDIM, NCSN/SMLD, and Energy Matching; this is one scheme, not separate theories. The same manifold supports a second variational principle. Its geodesics - the minimum-action curves of the Benamou-Brenier formula - are precisely the optimal-transport paths that Flow Matching learns. Fixing both endpoints and following the geodesic, generation becomes a deterministic ODE along a straight line, hence far fewer sampling steps. Placing both families of models on one manifold makes their relationship exact: diffusion follows a free-energy gradient flow, an initial-value problem; optimal-transport Flow Matching follows a Wasserstein geodesic, a boundary-value problem. The two reach the same endpoints along different paths.

Chinese Translation

概率测度空间 $ extmath{\mathcal{P}_2(\mathbb{R}^d)}$ 具有自然的几何结构：二次Wasserstein距离 $W_2$ 使其成为一个完备的度量空间，并且根据Otto的研究，它是一个（形式上的）黎曼流形，其测地线是最优传输插值。在这个流形上，自由能 $F(\rho) = KL(\rho || \pi)$ 的梯度流恰好对应于Fokker-Planck方程，其隐式欧拉离散化为JKO方案。这就是扩散模型背后的几何：前向过程沿着自由能下降，每一步去噪实现一个JKO步骤，从而恢复DDPM、DDIM、NCSN/SMLD和能量匹配；这是一种方案，而非独立的理论。同一个流形支持第二变分原理。其测地线——Benamou-Brenier公式的最小作用曲线——正是Flow Matching所学习的最优传输路径。固定两个端点并沿测地线前进，生成过程变成沿直线的确定性常微分方程（ODE），因此采样步骤大大减少。将这两类模型置于同一流形上，使它们之间的关系变得精确：扩散遵循自由能梯度流，这是一个初值问题；而最优传输流匹配遵循Wasserstein测地线，这是一个边值问题。两者沿着不同的路径到达相同的端点。

View on arXiv Download PDF AI Translation

cs.AI / 16 / 2606.24160

An Introduction to Causal Reinforcement Learning

因果强化学习导论

Bareinboim, Elias, Zhang, Junzhe, Lee, Sanghack

Abstract

Causal inference provides a set of principles and tools that allow one to combine data and knowledge about an environment to reason with questions of counterfactual nature, i.e., what would have happened had reality been different, even when no data of this unrealized reality is currently available. Reinforcement learning provides methods to learn a policy that optimizes a specific measure (e.g., reward, regret) when the agent is deployed in an environment and pursues an exploratory, trial-and-error approach. These two disciplines have evolved independently and with virtually no interaction between them. We note that they operate over different aspects of the same building block, counterfactual relations, which makes them umbilically connected. Based on these observations, novel learning opportunities arise when this connection is explicitly acknowledged and mathematized. To realize this potential, we note that any environment where the RL agent is deployed can be decomposed as a collection of autonomous mechanisms with different causal invariances, parsimoniously modeled as a structural causal model; any standard RL setting implicitly encodes such a model. This formalization allows us to put under a unifying treatment different modes of learning, including online, off-policy, and causal calculus learning, which appear unrelated in the literature. However, these modalities are not exhaustive: we introduce several natural and pervasive classes of learning settings that entail novel dimensions of analysis. Specifically, we introduce and discuss through causal lenses generalized policy learning, where to intervene, imitation learning, and counterfactual learning. These tasks lead to a broader view of counterfactual learning and suggest great potential for studying causal inference and reinforcement learning side by side, which we call causal reinforcement learning (CRL).

Chinese Translation

因果推断提供了一套原则和工具，使得人们能够结合数据和对环境的知识来推理反事实性质的问题，即如果现实有所不同会发生什么，即使当前没有关于这种未实现现实的数据。强化学习提供了在环境中部署代理时，学习优化特定度量（例如奖励、遗憾）的策略的方法，并采用探索性的试错方法。这两个学科独立发展，几乎没有相互作用。我们注意到，它们在同一构建块的不同方面上运作，即反事实关系，这使得它们紧密相连。基于这些观察，当这种联系被明确承认并数学化时，会产生新的学习机会。为了实现这一潜力，我们注意到，任何部署RL代理的环境都可以被分解为一组具有不同因果不变性的自主机制，简约地建模为结构因果模型；任何标准的RL设置隐含地编码了这样的模型。这种形式化使我们能够将不同的学习模式，包括在线学习、离线策略学习和因果计算学习，统一处理，这些在文献中看似无关。然而，这些模式并不是穷尽的：我们引入了几类自然且普遍的学习设置，涉及新的分析维度。具体而言，我们通过因果视角引入并讨论了广义策略学习、干预位置、模仿学习和反事实学习。这些任务引导我们对反事实学习有更广泛的看法，并建议并行研究因果推断和强化学习的巨大潜力，我们称之为因果强化学习（CRL）。

View on arXiv Download PDF AI Translation

cs.AI / 17 / 2606.24169

Data Scale, Not Latency, Shapes Cross-Lingual Encoder Transfer in Streaming ASR

数据规模，而非延迟，塑造了流式自动语音识别中的跨语言编码器迁移

Banfic, Nenad

Abstract

Adapting a streaming speech recognition model to a new language requires choosing between two plausible warm starts: a multilingual (ML) encoder or an English-only (EN) encoder. The common intuition is that the multilingual encoder should help most at low data, but it is unclear how long that advantage persists, whether tight streaming latency amplifies it, and whether it survives deployment quantization. We answer these questions with a controlled sweep of a 0.6 B-parameter cache-aware FastConformer transducer across eight European languages, up to five target-language data scales (100 h to 2500 h), three streaming tiers plus offline decoding, and up to four public test sets. The main result is that multilingual initialization is a data-limited advantage, not a latency-limited one. On FLEURS at 160 ms, the mean EN-ML word error rate (WER) gap falls from +4.21 percentage points (pp) at 100 h to +0.20 pp at 2500 h; a power-law fit summarizes this decay, with each doubling of target-language data roughly halving the remaining advantage. Across the three streaming tiers, the across-language mean EN-ML gap is approximately stable at each scale from 100 to 1000 h, and is near zero by 2500 h. Finally, 4-bit weight-only encoder quantization at the matched 560 ms streaming tier reduces the encoder footprint by about 3x, with an average FLEURS WER increase of about 0.5 pp. The resulting guideline is simple: use multilingual initialization in low-data regimes, treat the choice as effectively irrelevant at large data, and make latency and quantization decisions independently.

Chinese Translation

将流式语音识别模型适应于新语言需要在两种合理的预热启动之间进行选择：多语言（ML）编码器或仅英语（EN）编码器。普遍的直觉是，多语言编码器在数据量较少时应能提供更多帮助，但尚不清楚这种优势能持续多久，紧凑的流式延迟是否会放大这种优势，以及它是否能在部署量化中存活。我们通过对一个具有0.6 B参数的缓存感知FastConformer转导器进行控制性实验，涵盖八种欧洲语言、最多五种目标语言数据规模（100小时到2500小时）、三个流式层级加上离线解码，以及最多四个公共测试集，来回答这些问题。主要结果是，多语言初始化是一种受数据限制的优势，而非受延迟限制的优势。在FLEURS数据集上，当延迟为160毫秒时，EN-ML词错误率（WER）差距从100小时的+4.21个百分点（pp）下降到2500小时的+0.20 pp；一个幂律拟合总结了这种衰减，每次目标语言数据量翻倍，剩余优势大约减半。在三个流式层级中，跨语言的平均EN-ML差距在100到1000小时的每个规模上大致稳定，并在2500小时时接近零。最后，在匹配的560毫秒流式层级下，4位权重仅编码器的量化将编码器的占用空间减少了约3倍，FLEURS的平均WER增加约0.5 pp。由此得出的指导原则很简单：在低数据环境中使用多语言初始化，在大数据时将选择视为无关紧要，并独立做出延迟和量化决策。

View on arXiv Download PDF AI Translation

cs.AI / 18 / 2606.24196

Navigating User Behavior toward Personalized Multimodal Generation

引导用户行为实现个性化多模态生成

Zhou, Hengji, Liu, Yufeng, Liu, Ye, Xu, Yong, Xia, Lianghao, Nie, Liqiang

Abstract

Modern AIGC pipelines deliver high-fidelity images and videos but presuppose a well-formed creation instruction, while end users rarely articulate visual details, leaving generators misaligned with user demand. We study personalized content generation, which turns a user's interaction history into an executable instruction for downstream synthesis, and identify two obstacles: behavior must be encoded in a form legible to language reasoning, and the model must acquire instruction-writing skill absent from both pretraining and behavior data. We propose NaviGen, which represents each item with a dual identifier coupling a collaborative code and a textual code as a behavioral substrate and a semantic bridge in one token stream. On this representation, a two-stage SFT+RL pipeline first distills preference reasoning and instruction writing from evolutionarily searched supervision, then aligns generation with user intent through hierarchical and self-consistent rewards. Experiments across product, game, and short-video domains show that NaviGen improves personalized image and video generation, strengthens next-item prediction, and yields more specific, relevant, and visually generatable instructions. Our code is anonymously released at: https://github.com/iLearn-Lab/NaviGen.

Chinese Translation

现代的人工智能生成内容（AIGC）管道能够生成高保真度的图像和视频，但前提是需要明确的创作指令，而最终用户很少能够清晰表达视觉细节，这导致生成器与用户需求之间存在不匹配。我们研究个性化内容生成，将用户的交互历史转化为可执行的指令，以便于后续的合成，并识别出两个障碍：行为必须以一种可被语言推理理解的形式进行编码，并且模型必须获得指令编写的技能，而这一技能在预训练和行为数据中均未得到体现。我们提出了NaviGen，它通过将每个项目表示为一个双重标识符，结合协作代码和文本代码，作为行为基础和语义桥梁，形成一个令牌流。在这种表示下，采用两阶段的SFT+RL管道，首先从进化搜索的监督中提炼偏好推理和指令编写，然后通过层次化和自一致的奖励将生成与用户意图对齐。在产品、游戏和短视频领域的实验表明，NaviGen提高了个性化图像和视频生成的效果，增强了下一个项目的预测能力，并生成了更具体、相关和可视化的指令。我们的代码已匿名发布在：https://github.com/iLearn-Lab/NaviGen。

View on arXiv Download PDF AI Translation

cs.AI / 19 / 2606.24224

Exploring the relationship between human-centric AI and firm idiosyncratic risks

探索以人为本的人工智能与企业特有风险之间的关系

Liu, Zhen-Yuan Ralph, Wang, Yu-Ting, Yan, Jia-Jia, Gupta, Shivam, Giannakis, Mihalis

Abstract

Despite the extensive discussions of human-centric AI (HCAI) in Industry 5.0, its effects on firms' idiosyncratic risks (IR) remains underexplored. This is an imperative issue for firms navigate financial risks during the current technological revolution, as IR reflects investor reactions to corporate heterogeneous AI strategies and implementations by isolating firm-level stock volatility from systematic factors. Integrating situated AI theory with social-technical systems theory, we conceptualise HCAI as a situated AI strategy that reduces AI-related ethical risks and fosters AI-Human synergies in firms' business operations, ultimately reducing IR by aligning with stakeholders' diverse expectations. Moreover, socio-technical factors, namely digitalisation, operational efficiency, executive shareholding, and CEOs with IT background, may moderate the HCAI-IR relationship. Using a multi-source panel dataset of Chinese listed firms from 2015 to 2023, we find that HCAI is associated with lower firm IR. Furthermore, digitalisation and executive shareholding strengthen this risk-reducing effect, whereas operational efficiency and CEOs with IT background surprisingly attenuate it. Our findings offer theoretical contributions and practical insights for both ethical AI governance and firm financial risk management in the AI era.

Chinese Translation

尽管在工业5.0中对以人为本的人工智能（HCAI）进行了广泛讨论，但其对企业特有风险（IR）的影响仍然未得到充分探讨。这是企业在当前技术革命中应对金融风险的一个重要问题，因为IR反映了投资者对企业异质化人工智能策略和实施的反应，通过将企业层面的股票波动与系统性因素隔离开来。我们将情境人工智能理论与社会技术系统理论相结合，将HCAI概念化为一种情境人工智能策略，旨在减少与人工智能相关的伦理风险，并促进企业业务运营中的人工智能与人类的协同，最终通过与利益相关者的多样化期望对齐来降低IR。此外，社会技术因素，如数字化、运营效率、管理层持股以及具有IT背景的首席执行官，可能会调节HCAI与IR之间的关系。通过使用2015年至2023年间中国上市公司的多来源面板数据集，我们发现HCAI与较低的企业IR相关。此外，数字化和管理层持股增强了这种降低风险的效果，而运营效率和具有IT背景的首席执行官却意外地减弱了这一效果。我们的研究结果为伦理人工智能治理和人工智能时代企业金融风险管理提供了理论贡献和实践见解。

View on arXiv Download PDF AI Translation

cs.AI / 20 / 2606.24231

FlowR2A: Learning Reward-to-Action Distribution for Multimodal Driving Planning

FlowR2A：学习多模态驾驶规划中的奖励到动作分布

Li, Xirui, Liu, Zhe, Ye, Xiaoqing, Han, Wenhua, Pan, Yifeng, Han, Junyu, Zhao, Hengshuang

Abstract

Multimodal driving planning faces a long-standing tension between two paradigms: scoring-based methods benefit from dense reward supervision but are confined to a fixed action vocabulary, while anchor-based methods generate proposals dynamically yet suffer from sparse supervision constrained to a single ground-truth trajectory. In this work, we propose FlowR2A, which resolves this tension by reframing simulation-based rewards from discriminative targets into generative conditions. By learning the reward-conditioned action distribution from dense trajectory-reward pairs with a flow-matching decoder, FlowR2A unifies the dense supervision of scoring-based methods with the proposal generation of anchor-based methods in a single generative model, forcing the model to internalize the correlation between an action and its outcomes in safety, progress, comfort, and rule compliance. To balance hard safety constraints against soft progress objectives, we introduce fine-grained per-timestep reward conditioning and reward noise augmentation. The generative formulation naturally supports controllable test-time sampling via reward guidance and anchored sampling, producing high-quality proposals. FlowR2A achieves state-of-the-art results on the NAVSIM v1 and v2 benchmarks, with multimodal proposals of substantially higher quality than prior methods.

Chinese Translation

多模态驾驶规划面临着两种范式之间长期存在的紧张关系：基于评分的方法受益于密集的奖励监督，但受限于固定的动作词汇，而基于锚点的方法动态生成提案，但受到限于单一真实轨迹的稀疏监督。在本研究中，我们提出了FlowR2A，通过将基于仿真的奖励从判别目标重新构建为生成条件，解决了这一紧张关系。FlowR2A通过流匹配解码器从密集的轨迹-奖励对中学习奖励条件下的动作分布，将基于评分的方法的密集监督与基于锚点的方法的提案生成统一为一个单一的生成模型，迫使模型内化动作与其在安全性、进展、舒适性和规则遵从性方面的结果之间的相关性。为了平衡严格的安全约束与柔性的进展目标，我们引入了细粒度的每时间步奖励条件和奖励噪声增强。生成形式自然支持通过奖励引导和锚定采样进行可控的测试时采样，生成高质量的提案。FlowR2A在NAVSIM v1和v2基准测试上取得了最先进的结果，其多模态提案的质量显著高于先前的方法。

View on arXiv Download PDF AI Translation

cs.AI / 21 / 2606.24235

SP-Mind: An Autonomous Reasoning Agent for Spatial Proteomics Analysis

SP-Mind：一种用于空间蛋白组学分析的自主推理代理

Yuan, Yucheng, Ji, Yuanfeng, Li, Zhongxiao, Li, Ruijiang

Abstract

Spatial proteomics enables single-cell-resolution characterization of protein expression within tissue architecture, playing a critical role in understanding tumor microenvironments and guiding precision medicine. However, current analysis workflows remain fragmented, requiring expert manual orchestration of heterogeneous tools and limiting research scalability and reproducibility. We present SP-Mind, the first autonomous AI agent designed to unify the spatial proteomics analysis pipeline, from raw multiplexed tissue imaging to downstream phenotype discovery. Equipped with expert-curated biological analysis skills and specialized computational tools, SP-Mind converts natural-language queries into end-to-end analytical workflows without task-specific fine-tuning. To rigorously evaluate its capabilities, we introduce SP-Bench, a comprehensive benchmark spanning diverse tissue types, comprising 102 tasks across 18 distinct categories. Through extensive evaluation on SP-Bench and established downstream tasks, SP-Mind achieves state-of-the-art performance compared to existing open-source biomedical agent baselines.

Chinese Translation

空间蛋白组学能够在组织结构中实现单细胞分辨率的蛋白表达特征化，在理解肿瘤微环境和指导精准医学方面发挥着关键作用。然而，目前的分析工作流程仍然碎片化，需要专家手动协调异构工具，限制了研究的可扩展性和可重复性。我们提出了SP-Mind，这是第一个旨在统一空间蛋白组学分析流程的自主人工智能代理，从原始多重组织成像到下游表型发现。SP-Mind配备了专家策划的生物分析技能和专业计算工具，能够将自然语言查询转换为端到端的分析工作流程，而无需针对特定任务的微调。为了严格评估其能力，我们引入了SP-Bench，这是一个涵盖多种组织类型的综合基准，包括18个不同类别的102个任务。通过在SP-Bench和已建立的下游任务上的广泛评估，SP-Mind在与现有开源生物医学代理基准的比较中实现了最先进的性能。

View on arXiv Download PDF AI Translation

cs.AI / 22 / 2606.24237

Towards Federated Long-Tailed Graph Learning: An Energy-Guided Dual Decoupling Approach

面向联邦长尾图学习：一种能量引导的双重解耦方法

Guo, Lianshuai, Yuan, Zhongzheng, Li, Xunkai, Qu, Meixia, Wang, Wenyu

Abstract

Federated Graph Learning facilitates collaborative graph modeling across distributed clients while preserving data privacy. However, real-world data categories frequently exhibit long-tailed distributions. Such statistical scarcity severely degrades performance in two ways: it biases the global model toward majority classes, and it structurally isolates minority nodes by submerging them in heterophilic, head-dominated neighborhoods. While existing methods attempt topology-agnostic statistical compensations, they often fail under data scarcity. Instead of recovering tail nodes, they overfit the structural noise from adjacent dominant classes, leading to representation degradation. To address these limitations, we propose FedEPD, a framework built on a dual decoupling paradigm that separates topological purification from semantic recalibration. Specifically, FedEPD utilizes distribution-aware Dirichlet energy pruning to filter spatial heterophilic edges. It then overcomes Non-IID distribution shifts by extracting robust global prototypes from topologically central nodes, which are incorporated into local representations via a spatial low-pass prototype injection. Furthermore, a two stage alternating optimization strategy strictly protects majority decision boundaries while improving minority accuracy. Extensive experiments demonstrate that FedEPD achieves state-of-the-art performance across diverse long-tailed benchmarks, yielding absolute improvements of up to 4.97% in Accuracy and 5.48% in Macro-F1.

Chinese Translation

联邦图学习促进了分布式客户端之间的协作图建模，同时保护数据隐私。然而，现实世界中的数据类别往往表现出长尾分布。这种统计稀缺性在两个方面严重降低了性能：它使得全局模型偏向于多数类，并且通过将少数节点淹没在异质性、以头部为主导的邻域中，结构性地孤立了少数节点。虽然现有方法试图进行与拓扑无关的统计补偿，但在数据稀缺的情况下往往失败。它们不是恢复尾部节点，而是过拟合来自相邻主导类的结构噪声，导致表示退化。为了解决这些局限性，我们提出了FedEPD，一个基于双重解耦范式的框架，将拓扑净化与语义重校准分开。具体而言，FedEPD利用分布感知的Dirichlet能量修剪来过滤空间异质边。然后，它通过从拓扑中心节点提取稳健的全局原型来克服非独立同分布（Non-IID）分布偏移，这些原型通过空间低通原型注入融入到本地表示中。此外，一种两阶段交替优化策略严格保护多数决策边界，同时提高少数类的准确性。大量实验表明，FedEPD在各种长尾基准测试中实现了最先进的性能，准确率和宏观F1值分别提高了最高4.97%和5.48%。

View on arXiv Download PDF AI Translation

cs.AI / 23 / 2606.24251

Probing the Misaligned Thinking Process of Language Models

探究语言模型的错位思维过程

Zhou, Kaiwen, Venhoff, Constantin, Michala, Jonathan, Wang, Xin Eric, Saunders, William

Abstract

Large language models exhibit a growing range of misaligned behaviors such as strategic deception, sandbagging, and self-preservation. As they are increasingly deployed in high-stakes settings, it is critical to reliably detect such behaviors to ensure safe and responsible use. In this work, we propose to monitor misalignment by decomposing it into fine-grained cognitive processes -- misalignment indicators -- and detecting their presence in a model's internal activations via linear probes. We develop a taxonomy of 18 indicators spanning different misaligned behaviors, paired with an automated, meta-plan-guided pipeline that generates multi-turn training conversations. To rigorously evaluate generalization, we construct an out-of-distribution suite combining automated behavioral elicitation, established misalignment benchmarks, and natural benign conversations. Across 5 misaligned behaviors, our probes match a strong LLM judge with 0.935 AUROC on out-of-distribution benchmarks while keeping a low false positive rate on benign traffic. We further perform in-depth analysis to understand the probes and the model's internal representations of misalignment indicators.

Chinese Translation

大型语言模型表现出越来越多的错位行为，例如战略欺骗、拖延和自我保护。随着它们在高风险环境中的应用日益增多，可靠地检测这些行为对于确保安全和负责任的使用至关重要。在本研究中，我们提出通过将错位分解为细粒度的认知过程——错位指标——来监测错位，并通过线性探针检测模型内部激活中的这些指标。我们开发了一个涵盖不同错位行为的18个指标的分类法，并配备一个自动化的、基于元计划引导的流程，以生成多轮训练对话。为了严格评估模型的泛化能力，我们构建了一个包含自动化行为引导、已建立的错位基准和自然良性对话的分布外测试套件。在5种错位行为中，我们的探针在分布外基准上与强大的LLM评估者匹配，AUROC达到0.935，同时在良性流量上保持较低的假阳性率。我们进一步进行深入分析，以理解探针及模型对错位指标的内部表征。

View on arXiv Download PDF AI Translation

cs.AI / 24 / 2606.24279

Tractable Reasoning and Conjunctive Query Answering for Defeasible DL-Lite under Rational Closure

在理性闭包下可处理的推理与可联结查询回答的可行性DL-Lite

Casini, Giovanni, Straccia, Umberto

Abstract

In Description Logics (DLs), reasoning under Rational Closure (RC) is a well-known and widely accepted non-monotonic formalism to handle defeasible knowledge. In this paper, we study the application of RC to the core and horn variants of the DL-Lite family of lightweight description logics. We analyze both entitlement (instance checking) and Conjunctive Query (CQ) answering under RC. Our main contribution is providing a plug-in architecture that builds upon existing standard classical reasoners, establishing that reasoning and CQ answering under RC for DL-Lite can be done efficiently with minimal computational overhead.

Chinese Translation

在描述逻辑（DLs）中，理性闭包（Rational Closure, RC）是一种著名且广泛接受的非单调形式，用于处理可推翻知识。本文研究了RC在DL-Lite轻量级描述逻辑的核心和霍恩变体中的应用。我们分析了在RC下的资格（实例检查）和可联结查询（Conjunctive Query, CQ）回答。我们的主要贡献是提供了一种插件架构，该架构基于现有的标准经典推理器，确立了在DL-Lite下进行RC推理和CQ回答可以高效地完成，并且计算开销最小。

View on arXiv Download PDF AI Translation

cs.AI / 25 / 2606.24311

LemonHarness Technical Report

LemonHarness 技术报告

Ren, Kailong, Sun, Fubo, Liu, Jiachen, Yang, Liu, Yin, Zimo, Li, Jiaying, Yin, Congli, He, Ming, Huo, Yu, Liu, Jiawei, Chen, Zeping, Huangfu, Yubin, Li, Ronghua, Wu, Yixuan, Su, Xing, Xu, Yanzhi, Wu, Likang, Zhao, Hongke, Zhang, Lei, Geng, Xiaohui, Fan, Jianping

Abstract

As large language model (LLM) agents are applied to longer tasks, they increasingly modify workspace state across multiple rounds of iteration. However, agents typically observe only tool outputs and log fragments, while the actual state changes occur in the file system. Without explicit workspace boundaries, state-changing operations such as file writes and temporary artifact generation may scatter changes across paths. Over time, these weakly constrained changes accumulate, making states such as modified files difficult to track. This paper presents LemonHarness, an integrated execution framework for long-horizon agents. LemonHarness establishes an explicit execution boundary by constraining state-changing operations within a clearly defined workspace and bringing model invocation, tool execution, and rule knowledge within a single controlled boundary. State-changing operations, including file writes, dependency installation, and temporary artifact creation, are executed through structured tool interfaces, with execution feedback recorded as observations available to subsequent model decisions. The system also introduces a reusable rule knowledge base, which turns recurring execution rules and acceptance criteria into runtime knowledge. LemonHarness further adds a time-aware execution mechanism that exposes elapsed and remaining budget to the model, so it can rebalance exploration, implementation, and validation effort as time pressure shifts and avoid timeouts from long waits or excessive verification. On Terminal-Bench 2.0, LemonHarness_GPT-5.3-CodeX reached 84.49% accuracy over 445 trials; pairing the same framework with the stronger GPT-5.5 backbone raised the average accuracy to 86.52% across five jobs. The results suggest that a unified runtime boundary, callable rule knowledge, and time-aware execution can improve the stability of long-horizon agent execution.

Chinese Translation

随着大型语言模型（LLM）代理被应用于更长的任务，它们在多轮迭代中越来越多地修改工作区状态。然而，代理通常仅观察工具输出和日志片段，而实际的状态变化发生在文件系统中。在没有明确工作区边界的情况下，诸如文件写入和临时工件生成等状态改变操作可能会将变化分散到不同路径上。随着时间的推移，这些弱约束的变化积累，使得修改文件等状态难以追踪。本文提出了 LemonHarness，一个针对长时间跨度代理的集成执行框架。LemonHarness 通过将状态改变操作限制在明确定义的工作区内，建立了一个明确的执行边界，并将模型调用、工具执行和规则知识纳入单一的受控边界内。状态改变操作，包括文件写入、依赖项安装和临时工件创建，通过结构化工具接口执行，执行反馈记录为后续模型决策可用的观察数据。该系统还引入了一个可重用的规则知识库，将重复的执行规则和接受标准转化为运行时知识。LemonHarness 进一步增加了一种时间感知的执行机制，向模型暴露已用和剩余预算，以便在时间压力变化时重新平衡探索、实施和验证的努力，避免因长时间等待或过度验证而导致的超时。在 Terminal-Bench 2.0 上，LemonHarness_GPT-5.3-CodeX 在 445 次试验中达到了 84.49% 的准确率；将相同框架与更强的 GPT-5.5 主干结合，平均准确率提升至 86.52%，涵盖五个任务。结果表明，统一的运行时边界、可调用的规则知识和时间感知的执行能够提高长时间跨度代理执行的稳定性。

View on arXiv Download PDF AI Translation

cs.AI / 26 / 2606.24313

Prob-BBDM: a Probabilistic Brownian Bridge Diffusion Model for MRI sequence image-to-image translation

Prob-BBDM：一种用于MRI序列图像间转换的概率布朗桥扩散模型

Valls, Martin, Bourdon, Pascal, Fernandez-Maloigne, Christine, Herpe, Guillaume, Helbert, David

Abstract

AI-driven image-to-image synthesis is rapidly advancing, with growing applications in medical imaging. Multi-modal image analysis plays a crucial role in optimizing examination quality, yet acquiring multiple imaging modalities in clinical settings remains resource-intensive and time-consuming, especially for 3D imaging. To address this challenge, we propose a novel image-to-image translation model based on Brownian Bridge Diffusion Models (BBDM), which synthesizes magnetic resonance imaging (MRI) sequences from 2D axial slices. Our approach integrates a variational encoder-guided diffusion mechanism, leveraging probabilistic image distributions to enhance synthesis quality. Evaluated on the BraTS 2021 dataset, our Probabilistic-BBDM (Prob-BBDM) achieves superior performance across multiple translation tasks, reaching up to 88.46% SSIM and 26.09 dB PSNR, with consistent improvements over baselines. Notably, our diffusion process requires only 4 steps, making it computationally efficient while maintaining high-quality synthesis. To further validate generalizability, we test Prob-BBDM on an external third-party dataset, demonstrating consistent performance across domains. Additionally, we assess the clinical utility of the synthesized slices by using them as input to a pre-trained segmentation model. Tumor segmentation yields a Dice score of 88.71% and an HD95 of 3.49 mm, confirming that the synthesized slices preserve critical diagnostic information. These results highlight the potential of Prob-BBDM for high-quality, efficient, and generalizable MRI synthesis, offering a promising step toward improved medical image translation.

Chinese Translation

基于人工智能的图像间合成技术正在迅速发展，并在医学成像中得到越来越多的应用。多模态图像分析在优化检查质量方面发挥着至关重要的作用，但在临床环境中获取多种成像模态仍然资源密集且耗时，尤其是在三维成像方面。为了解决这一挑战，我们提出了一种基于布朗桥扩散模型（BBDM）的新型图像间转换模型，该模型能够从二维轴向切片合成磁共振成像（MRI）序列。我们的方法整合了变分编码器引导的扩散机制，利用概率图像分布来增强合成质量。在BraTS 2021数据集上的评估表明，我们的概率布朗桥扩散模型（Prob-BBDM）在多个转换任务中表现优越，SSIM达到88.46%，PSNR为26.09 dB，相较于基线方法有持续的提升。值得注意的是，我们的扩散过程仅需4个步骤，计算效率高，同时保持高质量的合成。为了进一步验证模型的通用性，我们在外部第三方数据集上测试了Prob-BBDM，证明其在不同领域中的一致性表现。此外，我们通过将合成切片作为输入到预训练分割模型中，评估了合成切片的临床实用性。肿瘤分割的Dice得分为88.71%，HD95为3.49 mm，确认合成切片保留了重要的诊断信息。这些结果突显了Prob-BBDM在高质量、高效和可推广的MRI合成中的潜力，为改善医学图像转换提供了有希望的步骤。

View on arXiv Download PDF AI Translation

cs.AI / 27 / 2606.24347

MVG-KAN: Multi-View Geo-Wind Guided KAN for PM$_{2.5}$ Forecasting

MVG-KAN：基于多视角地理风向引导的 KAN 模型用于 PM$_{2.5}$ 预测

Huang, Cheng, Guan, Muyao, Railey, Jairus Yougui, Xu, Ning, Xu, Honghui, Zhang, Changjiang, Zhang, Zhen, Zhang, Shiqing, Bai, Cong

Abstract

Accurate short-term PM$_{2.5}$ forecasting is important for public health protection, air-quality early warning, and urban environmental management. However, PM$_{2.5}$ variation is driven by multiple coupled factors, including stable periodic changes induced by human activities and meteorological regularity, station-specific short-term concentration evolution, and meteorology-driven pollutant dispersion among monitoring stations. Existing spatio-temporal forecasting methods may capture station relationships to some extent, but distance-only, correlation-based, or purely adaptive graphs are often insufficient to comprehensively represent these heterogeneous factors, especially wind-direction-dependent pollutant transport. To address this problem, we propose a Multi-View Geo-Wind Guided KAN model for PM$_{2.5}$ forecasting, named \textbf{MVG-KAN}, which models station-level PM$_{2.5}$ evolution from three complementary views: local periodic regularity, station-wise residual temporal dynamics, and meteorological-environment-guided spatial dispersion. Specifically, the periodic-residual forecasting backbone first separates stable daily and weekly patterns from non-periodic residual variations. A Geo-Wind Graph is constructed by combining geographic distance decay with wind-direction- and wind-speed-aware transport, providing a lightweight physically motivated directed spatial prior for residual propagation among stations. In addition, a temporal Kolmogorov-Arnold network (TKAN) residual head is then introduced to learn station-wise nonlinear autoregressive correction from de-periodized PM$_{2.5}$ residuals and historical multi-pollutant sequences, thereby enhancing the modeling of local residual inertia and pollutant co-variation.

Chinese Translation

准确的短期 PM$_{2.5}$ 预测对公共健康保护、空气质量预警和城市环境管理至关重要。然而，PM$_{2.5}$ 的变化受到多种耦合因素的驱动，包括人类活动和气象规律引起的稳定周期性变化、特定监测站的短期浓度演变以及气象驱动的污染物在监测站之间的扩散。现有的时空预测方法在一定程度上能够捕捉监测站之间的关系，但仅基于距离、相关性或纯粹自适应图的方法往往不足以全面代表这些异质因素，特别是风向依赖的污染物运输。为了解决这个问题，我们提出了一种用于 PM$_{2.5}$ 预测的多视角地理风向引导 KAN 模型，命名为 extbf{MVG-KAN}，该模型从三个互补的视角对监测站级 PM$_{2.5}$ 演变进行建模：局部周期性规律、监测站特定的残差时间动态以及气象环境引导的空间扩散。具体而言，周期-残差预测主干首先将稳定的日常和每周模式与非周期性残差变化分离。通过结合地理距离衰减与风向和风速感知的运输，构建了一个地理风图（Geo-Wind Graph），为监测站之间的残差传播提供了一个轻量级的物理动机导向的有向空间先验。此外，引入了一个时间科尔莫戈罗夫-阿诺德网络（TKAN）残差头，以学习来自去周期化 PM$_{2.5}$ 残差和历史多污染物序列的监测站特定非线性自回归修正，从而增强局部残差惯性和污染物共同变化的建模。

View on arXiv Download PDF AI Translation

cs.AI / 28 / 2606.24369

Accelerating Disaggregated RL for Visual Generative LLMs with Diffusion-Based Parallelism and Trainer-Assisted Generation

基于扩散的并行性和训练者辅助生成加速视觉生成大语言模型的分离强化学习

Wang, Sijie, Qing, Zhengyu, Tan, Zhiqiang, Yin, Yiming, Zhang, Yeqing, Wang, Yaoyuan, Wang, Qiang, Chu, Xiaowen, Shi, Shaohuai

Abstract

Reinforcement learning (RL) has become a dominant post-training paradigm, driving the emergence of high-performance RL systems such as veRL for autoregressive large language models (LLMs). In parallel, diffusion-oriented RL algorithms, e.g., DanceGRPO and FlowGRPO, have rapidly expanded the scope of RL from language reasoning to diffusion-based visual and flow-based generation. However, efficient RL systems for diffusion generative LLMs remain underexplored. Existing implementations, e.g., veRL-Omni, still rely on colocated execution, which simplifies synchronization but couples rollout and training resources, limits heterogeneous deployment, and constrains independent scaling. To this end, we introduce DigenRL, a disaggregated RL framework for diffusion-based generative LLMs that supports flexible resource allocation, accommodates heterogeneous GPUs, and facilitates efficient task scheduling. To maximally reduce the execution bubbles in the disaggregated architecture, we propose: 1) a generation-axis pipeline (GAP) and time-step parallelism (TSP) in the diffusion architecture to enable finer-grained pipelining between rollout and training; 2) an elastic trainer-assisted generation (TAG) approach to enable the trainer GPU resources to dynamically assist in executing rollout generations; and 3) a tightly one-step constrained asynchronous strategy to further utilize the tail bubble in the pipeline. Extensive experiments are conducted on three hardware testbeds with 16-32 GPUs using HunyuanVideo-13B, Wan2.1-14B, FLUX.1-12B, and QwenImage-20B generative models. Experimental results show that DigenRL achieves 1.56-2.10x throughput improvements over state-of-the-art diffusion RL systems, veRL-Omni and GenRL.

Chinese Translation

强化学习（RL）已成为一种主导的后训练范式，推动了高性能RL系统的出现，例如用于自回归大语言模型（LLMs）的veRL。同时，面向扩散的RL算法，如DanceGRPO和FlowGRPO，迅速扩展了RL的应用范围，从语言推理到基于扩散的视觉生成和基于流的生成。然而，针对扩散生成LLMs的高效RL系统仍然未被充分探索。现有的实现，例如veRL-Omni，仍然依赖于共置执行，这虽然简化了同步，但将回滚和训练资源耦合在一起，限制了异构部署，并约束了独立扩展。为此，我们提出了DigenRL，一个针对基于扩散的生成LLMs的分离RL框架，支持灵活的资源分配，适应异构GPU，并促进高效的任务调度。为了最大限度地减少分离架构中的执行气泡，我们提出：1）在扩散架构中采用生成轴管道（GAP）和时间步并行性（TSP），以实现回滚和训练之间的更细粒度的流水线；2）一种弹性训练者辅助生成（TAG）方法，使训练者GPU资源能够动态协助执行回滚生成；3）一种紧密的一步约束异步策略，以进一步利用管道中的尾气泡。在使用HunyuanVideo-13B、Wan2.1-14B、FLUX.1-12B和QwenImage-20B生成模型的16-32 GPU的三个硬件测试平台上进行了广泛的实验。实验结果表明，DigenRL在吞吐量上比最先进的扩散RL系统veRL-Omni和GenRL提高了1.56-2.10倍。

View on arXiv Download PDF AI Translation

cs.AI / 29 / 2606.24370

When Helpfulness Overrides Causal Caution: Context-Dependent Suppression and Recovery in LLMs

当有用性超越因果谨慎：上下文依赖的抑制与恢复在大型语言模型中的表现

Okumura, Hiroshi

Abstract

Large language models (LLMs) are increasingly integrated into decision-support roles in business and policy contexts. While prior benchmark studies have primarily evaluated LLMs' causal reasoning capabilities, a more fundamental epistemic dimension has been overlooked: Causal Caution, defined as the propensity to refrain from causal judgment when empirical evidence is insufficient. This study examines the systematic suppression of Causal Caution that occurs when LLMs shift from academic to practical advisory contexts. Using an evaluation rubric inspired by Pearl's Causal Hierarchy (the PCH score), we conducted experiments on four high-performance LLMs -- Claude Sonnet 4.6, Claude Opus 4.7, GPT 5.5, and Gemini 3.1 Pro -- across 480 trials. Causal Caution maintenance rates were 91.7--100.0% in academic contexts but dropped to 6.7--18.3% in practical advisory contexts (Fisher's exact test, p < .001 across all models). Furthermore, when restricted to practical prompts requesting concrete recommendations or explanatory rationales, only 1 of 200 responses (0.5%) maintained Causal Caution. A brief self-correction prompt -- "Please reconsider this judgment from the perspective of causal relationships" -- restored the expression of Causal Caution to maintenance rates of 71.4--100.0% (McNemar's test, p < .001 across all models). These results suggest that helpfulness-oriented response patterns may suppress the expression of Causal Caution in practical advisory contexts, with important implications for organizational governance. The findings indicate that this suppression reflects context-dependent variation in expression rather than an underlying capability limitation, suggesting that multi-agent architectures that separate proposal generation from causal auditing may offer a promising governance design.

Chinese Translation

大型语言模型（LLMs）越来越多地被整合到商业和政策背景下的决策支持角色中。尽管之前的基准研究主要评估了LLMs的因果推理能力，但一个更为根本的认识维度却被忽视了：因果谨慎（Causal Caution），定义为在实证证据不足时避免进行因果判断的倾向。本研究考察了当LLMs从学术环境转向实际咨询环境时，因果谨慎的系统性抑制现象。我们使用了一个受Pearl的因果层次（Causal Hierarchy）启发的评估标准（PCH评分），对四个高性能的LLMs——Claude Sonnet 4.6、Claude Opus 4.7、GPT 5.5和Gemini 3.1 Pro——进行了480次实验。因果谨慎的维持率在学术环境中为91.7%至100.0%，而在实际咨询环境中下降至6.7%至18.3%（Fisher精确检验，p < .001，适用于所有模型）。此外，当限制于请求具体建议或解释理由的实际提示时，仅有200个响应中的1个（0.5%）维持了因果谨慎。一个简短的自我纠正提示——“请从因果关系的角度重新考虑此判断”——使因果谨慎的表达恢复到71.4%至100.0%的维持率（McNemar检验，p < .001，适用于所有模型）。这些结果表明，面向有用性的响应模式可能在实际咨询环境中抑制因果谨慎的表达，这对组织治理具有重要影响。研究结果表明，这种抑制反映了表达的上下文依赖性变化，而非潜在能力的限制，暗示分离提案生成与因果审计的多代理架构可能提供一种有前景的治理设计。

View on arXiv Download PDF AI Translation

cs.AI / 30 / 2606.24388

PHANTOM: A Large-Scale Dataset of Multimodal Adversarial Attacks for Vision-Language Models

PHANTOM：一个大规模的多模态对抗攻击数据集用于视觉-语言模型

Gallivanone, Simone, Khodadadi, Hossein, Dore, Mauro, Medda, Mauro, Franco, Nicola

Abstract

We introduce a large-scale, open-source dataset of pre-generated adversarial attacks for vision-language models (VLMs). The dataset is designed to be diverse, representative, and practical, extending existing benchmarks by covering 10 high-level categories and 55 subcategories of harmful intents. Our primary goal is to make adversarial data accessible to the research community, given the computational cost and complexity of generating large numbers of attacks. The dataset comprises 47 524 adversarial samples, generated using state-of-the-art attack strategies from recent literature. Our work complements existing efforts by consolidating and extending prior benchmarks from multiple established sources, resulting in 7 826 intents, and introduce an additional category to broaden coverage. This provides realistic evaluation resources for studying model robustness and alignment. Our dataset intends to enable researchers and practitioners to systematically evaluate the robustness and safety of VLMs, fine-tune attack-generation models, and develop or stress-test defensive guardrails under diverse adversarial conditions. By releasing this resource, we aim to lower the barrier to adversarial research and foster more reproducible, comprehensive, and comparable evaluations of VLM safety.

Chinese Translation

我们介绍了一个大规模的开源数据集，包含为视觉-语言模型（VLMs）预生成的对抗攻击。该数据集旨在多样化、具有代表性和实用性，通过涵盖10个高层次类别和55个有害意图子类别，扩展现有基准。我们的主要目标是使对抗数据对研究社区可获取，考虑到生成大量攻击的计算成本和复杂性。该数据集包含47,524个对抗样本，采用了近期文献中的最先进攻击策略生成。我们的工作通过整合和扩展来自多个已建立来源的先前基准，补充了现有努力，形成了7,826个意图，并引入了一个额外类别以扩大覆盖范围。这为研究模型的鲁棒性和一致性提供了现实的评估资源。我们的数据集旨在使研究人员和从业者能够系统地评估VLM的鲁棒性和安全性，微调攻击生成模型，并在多样的对抗条件下开发或压力测试防御措施。通过发布这一资源，我们旨在降低对抗研究的门槛，促进更可重复、全面和可比较的VLM安全性评估。

View on arXiv Download PDF AI Translation

cs.AI / 31 / 2606.24391

Age of LLM: A Strategic 1v1 Benchmark for Reasoning, Diplomacy and Reliability of Large Language Models under Fog of War

大型语言模型的时代：在战争迷雾下的推理、外交和可靠性的战略1v1基准测试

Ricci, Arnaud

Abstract

We introduce Age of LLM, a turn-based 1v1 benchmark in which two LLMs face off on a 13x7 grid to destroy the enemy base. Three stressors are deliberate: fog of war, full diplomacy (messages, ceasefires, ultimatums; uranium kept secret), and a reliability dimension where every turn must follow a strict JSON schema and an illegal action is silently discarded. The engine is private and each match uses a fresh random map seed and opponent, mitigating the data contamination that affects public benchmarks. Models receive a (near) rule-only prompt with no build-order advice (two tactical seed phrases were present during data collection; see Section 2.7). We benchmark 15 reasoning models across 54 matches and 5,258 actions. Findings: (1) the nuclear rush dominates (78% on the rules-coherent v0.11+ sub-corpus; 85% corpus-wide) with a sole-launcher signature that is largely mechanical under secret-simultaneous launch rules, not a cognitive deterrence failure; (2) military conquest is rare but faster (12.3 vs 18.9 turns); (3) diplomacy is prolific yet almost never consummated; (4) ~58% of illegal actions are fog/state errors, making the illegal-action rate a measure of belief-tracking; (5) -- the least established, and the only one we label exploratory -- a weak link associates reliability with winning. The corpus is small, unbalanced and not side-swapped, so the ranking is a preliminary descriptive view, not a contribution. Beyond ranking, the turn-by-turn traces of actions and messages make the corpus a lens on how LLMs reason under adversarial uncertainty -- their belief-tracking, spontaneous deception, and per-model cognitive "personas" -- which we frame as a future research direction. We release the replay format, an isometric viewer and all replays; engine source on request.

Chinese Translation

我们介绍了大型语言模型的时代（Age of LLM），这是一个基于回合的1v1基准测试，其中两个大型语言模型在一个13x7的网格上对抗，以摧毁敌方基地。我们故意设置了三个压力因素：战争迷雾、全面外交（信息交流、停火、最后通牒；铀资源保密）和一个可靠性维度，其中每个回合必须遵循严格的JSON格式，任何非法行为都会被默默丢弃。该引擎是私有的，每场比赛使用新的随机地图种子和对手，从而减轻了影响公共基准测试的数据污染。模型接收一个（几乎）仅包含规则的提示，没有建造顺序建议（在数据收集期间存在两个战术种子短语；见第2.7节）。我们对15个推理模型进行了54场比赛和5,258个动作的基准测试。研究发现：（1）核武器突击占主导地位（在规则一致的v0.11+子语料库中占78%；在整个语料库中占85%），其单发发射器特征在秘密同时发射规则下主要表现为机械性，而非认知威慑失效；（2）军事征服虽然罕见但更快（12.3回合对比18.9回合）；（3）外交活动频繁但几乎从未达成；（4）约58%的非法行为是由于战争迷雾/状态错误，因此非法行为率可以作为信念追踪的一个衡量标准；（5）——这是最不成熟的，也是我们唯一标记为探索性的——一个弱关联将可靠性与胜利联系起来。该语料库小且不平衡，且未进行侧面交换，因此排名只是初步的描述性视图，而非贡献。除了排名之外，逐回合的动作和信息追踪使得该语料库成为观察大型语言模型在对抗性不确定性下如何推理的一个视角——它们的信念追踪、自发欺骗和每个模型的认知“人格”——我们将其框定为未来的研究方向。我们发布了重播格式、等距观察器和所有重播；引擎源代码可按请求提供。

View on arXiv Download PDF AI Translation

cs.AI / 32 / 2606.24392

ATRIA: Adaptive Traceable ECG Reporting with Iterative Agents

ATRIA：基于迭代代理的自适应可追溯心电图报告

Hong, Donggyun, Lee, Kyuhwan, Kwon, Junmyung, Jo, Yong-Yeon

Abstract

Existing ECG report generation is tightly coupled -- interpretation and reporting fused end-to-end, so errors propagate without stage-level recourse -- while agent-based systems decouple tasks but remain single-pass, never revisiting earlier outputs. Clinical ECG reporting instead unfolds iteratively, requiring progressive context integration and bidirectional editing. We present \textsc{ATRIA}, a multi-agent ECG reporting system that mirrors the clinician's iterative workflow: it binds every report claim to its supporting evidence, flags statements unsupported by that evidence, incorporates additional context mid-session, and lets clinicians verify and revise individual findings rather than accept one opaque output. Because its agents use ECG analysis models already in clinical use, the underlying findings are clinically trustworthy; and as a cloud-based web service, \textsc{ATRIA} is ready for immediate deployment. We demonstrate \textsc{ATRIA} through four interaction cases, with a live demo and video available.

Chinese Translation

现有的心电图报告生成紧密耦合——解释和报告端到端融合，因此错误在没有阶段级补救的情况下传播，而基于代理的系统则解耦任务，但仍然是单次通过，无法重新审视早期输出。临床心电图报告则是迭代展开的，要求逐步整合上下文和双向编辑。我们提出了 extsc{ATRIA}，一个多代理心电图报告系统，反映了临床医生的迭代工作流程：它将每个报告声明与其支持证据绑定，标记未被该证据支持的陈述，在会话中期整合额外上下文，并允许临床医生验证和修订个别发现，而不是接受一个不透明的输出。由于其代理使用的是已经在临床中使用的心电图分析模型，因此基础发现是临床可信的；作为一个基于云的网络服务， extsc{ATRIA} 准备好立即部署。我们通过四个交互案例展示 extsc{ATRIA}，并提供了实时演示和视频。

View on arXiv Download PDF AI Translation

cs.AI / 33 / 2606.24414

Cycle-Consistent Neural Explanation of Formal Verification Certificates

循环一致的神经网络解释形式验证证书

Rodriguez, Andoni, Pozanco, Alberto, Borrajo, Daniel

Abstract

Formal verification produces machine-checkable certificates that attest to the satisfaction or violation of temporal properties, yet these certificates remain opaque to non-specialist stakeholders. We propose a cycle-consistent neural architecture that generates faithful natural language explanations of verification certificates. A forward network NN1 maps certificates to explanations, and an inverse network NN2 reconstructs certificates from explanations; a symbolic verifier closes the loop, providing a differentiable faithfulness proxy. A pointer-generator mechanism ensures lexical grounding by copying state names directly from the certificate. We evaluate on 420 test certificates spanning six verification methods (bounded proof, k-induction, inductive invariant, lasso, reachability, witness pair) in both YES and NO verdict variants, drawn from a financial compliance domain with 207 named states. Our trained architecture, combined with a hybrid inference-time routing strategy, achieves 90.0% cycle-verified soundness, surpassing a multi- LLM few-shot baseline (76.1% for the best of 16 LLM combinations across four frontier models) by 13.9 percentage points. The neural model wins on 10 of 12 verdict/kind categories, with three categories reaching 100% soundness. The architecture offers 860x faster inference (185 ms vs. 160 s per certificate for the full multi-LLM baseline), offline operation, deterministic outputs, and zero per-inference cost. These results demonstrate that trained specialization outperforms general-purpose LLM prompting for structured certificate explanation, while eliminating the deployment constraints of cloud-based inference.

Chinese Translation

形式验证生成可机器检查的证书，以证明时间属性的满足或违反，但这些证书对非专业利益相关者仍然不透明。我们提出了一种循环一致的神经网络架构，能够生成对验证证书的忠实自然语言解释。前向网络 NN1 将证书映射到解释，而逆向网络 NN2 则从解释重构证书；符号验证器闭合了这个循环，提供了一个可微分的忠实性代理。指针生成机制通过直接从证书中复制状态名称来确保词汇的基础。我们在420个测试证书上进行评估，这些证书涵盖六种验证方法（有界证明、k归纳、归纳不变、套索、可达性、见证对），包括YES和NO判决变体，来自一个包含207个命名状态的金融合规领域。我们训练的架构结合混合推理时路由策略，实现了90.0%的循环验证健全性，超越了多LLM少样本基线（在四个前沿模型中，16种LLM组合的最佳结果为76.1%）13.9个百分点。该神经模型在12个判决/类别中赢得了10个类别，其中三个类别达到了100%的健全性。该架构提供了860倍的推理速度（每个证书185毫秒对比全多LLM基线的160秒），离线操作、确定性输出和零每次推理成本。这些结果表明，训练的专业化在结构化证书解释方面优于通用LLM提示，同时消除了基于云推理的部署限制。

View on arXiv Download PDF AI Translation

cs.AI / 34 / 2606.24416

Agentic AI for Bilevel Long-Term Optimization of Policy-Driven Physical Layer Systems

用于政策驱动物理层系统的双层长期优化的自主智能

Xiao, Bingnan, Yang, Chenhao, Ni, Wei, Wang, Xin, Quek, Tony Q. S.

Abstract

Network operators' changing policies, service requirements, and stringent real-time constraints render existing methods designed with fixed objectives and constraints ineffective. This paper presents Agentic long-term performance optimization (Agentic-LTPO), a nested bilevel optimization framework that can be applied to adaptive physical layer problem configuration. The key idea is to employ agentic AI to generate upper-level configurations in a bilevel optimization structure, where evolving operator policies, environment summaries, and historical experiences are translated into structured lower-level optimization problem configurations. The lower level solves the problems with updated configurations for real-time physical-layer decisions. Considering cell-free MIMO beamforming as a use case, we embody Agentic-LTPO by designing a new multi-agent decision process with retrieval-augmented experience-based verification in the upper level, together with a closed-form beamformer in the lower level. Experiments demonstrate that Agentic-LTPO exhibits strong adaptability to dynamic operator policies and effectively enhances the system's long-term performance by 57.2% compared to traditional methods.

Chinese Translation

网络运营商不断变化的政策、服务需求和严格的实时约束使得现有以固定目标和约束设计的方法失效。本文提出了自主长期性能优化（Agentic-LTPO），这是一种可以应用于自适应物理层问题配置的嵌套双层优化框架。其关键思想是利用自主智能在双层优化结构中生成上层配置，其中不断演变的运营商政策、环境摘要和历史经验被转化为结构化的下层优化问题配置。下层通过更新的配置解决实时物理层决策问题。考虑到无小区 MIMO 波束成形作为应用案例，我们通过设计一种新的多智能体决策过程，在上层结合检索增强的基于经验的验证，并在下层使用封闭形式的波束成形器，从而体现了自主长期性能优化（Agentic-LTPO）。实验表明，与传统方法相比，Agentic-LTPO 对动态运营商政策表现出强大的适应性，并有效提升了系统的长期性能，提升幅度达到 57.2%。

View on arXiv Download PDF AI Translation

cs.AI / 35 / 2606.24421

Can Aggregate Invariants Accelerate Continuous Subgraph Matching? Limits, Laws, and a Dynamic Spectral Index

聚合不变量能加速连续子图匹配吗？限制、法则与动态谱指数

Chen, Minghao, Zheng, Jiale

Abstract

Spectral filtering recently delivered substantial pruning for \emph{static} subgraph matching: Laplacian interlacing rejects candidates whose neighborhoods cannot host the query. We study whether such aggregate structural tests can accelerate \emph{continuous} subgraph matching (CSM) over dynamic graphs, and answer in three parts. First, lazily maintained spectral bounds are infeasible exactly where spectral pruning has value: we characterize the tightest safe rule over a formalized perturbation relaxation and show that even it loses essentially all pruning power within four touching updates. Second, exact maintenance is affordable when selective: pruning utility and recomputation cost are anti-correlated across vertices -- hubs provably never prune -- so recomputing small-neighborhood spectra on touch sustains exact local spectra at microseconds per update, complete by construction. Third, integrated into a decoupled CSM benchmark against an identical-minus-spectra control, the tests remove up to $51\%$ of candidates or safely skip up to $47\%$ of update enumerations, yet enumeration intermediates remain unchanged -- beyond the gates' skipped first-level bindings, typically zero -- across two engines, four real graphs, two stream types, and $77$ solved queries; a constructed radius-stratified workload confirms the instrument detects the exception when one exists ($-99.9\%$ intermediates, $748\times$ faster). Aggregate tests accelerate what scales with candidate sets -- construction, list scans -- never adjacency-guided exploration. We distill an intermediate-invariance methodology for evaluating CSM filters and release a reusable dynamic local-spectra index.

Chinese Translation

谱过滤最近为静态子图匹配提供了显著的剪枝：拉普拉斯交错拒绝那些邻域无法容纳查询的候选项。我们研究这种聚合结构测试是否能加速动态图上的连续子图匹配（CSM），并分三部分回答。首先，惰性维护的谱界限在谱剪枝有价值的地方是不可行的：我们对形式化的扰动松弛特征进行了最紧安全规则的表征，并表明即使如此，在四个接触更新内几乎失去了所有剪枝能力。其次，当选择性时，精确维护是可承受的：剪枝效用和重新计算成本在顶点间呈反相关——中心节点证明从不剪枝——因此在接触时重新计算小邻域谱以微秒级的速度维持精确的局部谱，构造上是完整的。第三，集成到一个与相同但缺少谱的控制组的解耦CSM基准中，这些测试最多可以去除51%的候选项或安全跳过47%的更新枚举，但枚举中间结果保持不变——在两个引擎、四个真实图、两种流类型和77个解决查询中，跳过的第一层绑定通常为零；一个构造的半径分层工作负载确认该工具在存在例外时能够检测到（-99.9%的中间结果，速度提升748倍）。聚合测试加速了与候选集规模相关的内容——构造、列表扫描——而从未引导邻接探索。我们提炼出一种中间不变量方法论用于评估CSM过滤器，并发布一个可重用的动态局部谱索引。

View on arXiv Download PDF AI Translation

cs.AI / 36 / 2606.24437

ReM-MoA: Reasoning Memory Sustains Mixture-of-Agents Scaling

ReM-MoA：推理记忆支持混合代理的扩展

Ping, Heng, Bhattacharjee, Arijit, Zhang, Peiyu, Li, Shixuan, Yang, Wei, Jannesari, Ali, Ahmed, Nesreen, Bogdan, Paul

Abstract

Mixture-of-Agents (MoA) architectures improve inference-time scaling by organizing multiple LLM agents into layered reasoning pipelines. However, existing MoA variants fail to sustain gains as depth increases, exhibiting degradation, early plateauing, or saturation. We propose ReM-MoA, a memory-augmented MoA framework that sustains scaling through two mechanisms: (1) a Ranked Reasoning Memory that persistently stores and ranks reasoning traces from all layers using a comparative Reviewer Agent, and (2) a Curated Diversified Memory Routing scheme that exposes different agents to distinct combinations of successful and failed traces, preserving exploration diversity while propagating high-quality reasoning. We further introduce an optional multi-domain Reviewer distillation pipeline that improves ranking quality through frontier-model supervision. Across five reasoning benchmarks spanning math, formal logic, code, knowledge, and commonsense, ReM-MoA consistently outperforms prior MoA variants across both depth and width scaling, and its advantage widens with depth, establishing structured cross-layer reasoning memory as a key missing mechanism for scalable multi-agent inference.

Chinese Translation

混合代理（Mixture-of-Agents, MoA）架构通过将多个大型语言模型（LLM）代理组织成分层推理管道来改善推理时的扩展性。然而，现有的 MoA 变体在深度增加时未能维持性能提升，表现出退化、早期平稳或饱和的现象。我们提出了 ReM-MoA，一种增强记忆的 MoA 框架，通过两种机制维持扩展性：（1）一个排名推理记忆（Ranked Reasoning Memory），该记忆持久存储并排名来自所有层的推理轨迹，使用比较评审代理（Reviewer Agent）；（2）一个策划多样化的记忆路由方案（Curated Diversified Memory Routing），该方案使不同的代理接触到成功和失败轨迹的不同组合，保持探索的多样性，同时传播高质量的推理。我们进一步引入一个可选的多领域评审蒸馏管道，通过前沿模型监督提高排名质量。在涵盖数学、形式逻辑、代码、知识和常识的五个推理基准测试中，ReM-MoA 在深度和宽度扩展方面始终优于先前的 MoA 变体，其优势随着深度的增加而扩大，确立了结构化跨层推理记忆作为可扩展多代理推理的关键缺失机制。

View on arXiv Download PDF AI Translation

cs.AI / 37 / 2606.24453

Bayesian control for coding agents

编码智能体的贝叶斯控制

Papamarkou, Theodore, Smirnov, Vladislav, Mazanov, Viktor, Vazhentsev, Artem, Nakov, Preslav, Baldwin, Timothy, Shelmanov, Artem

Abstract

Modern coding agents pair LLM generators with various tools, including cheap diagnostics and expensive verifiers. The tool-use decisions are typically governed by orchestrators that often use fixed rules and ignore uncertainty. We formulate orchestration as cost-sensitive sequential hypothesis testing: a Bayesian controller maintains a belief over candidate correctness and dynamically decides whether to gather more evidence, refine the candidate, verify it, or stop. Across six generators and nine coding benchmarks, Bayesian control proves to be most valuable when verification is costly and critics are informative but imperfect. Beyond control, the belief state yields an interpretable correctness score that outperforms token-probability and raw tool-success baselines for uncertainty quantification.

Chinese Translation

现代编码智能体将大型语言模型（LLM）生成器与各种工具相结合，包括廉价的诊断工具和昂贵的验证器。工具使用决策通常由协调者管理，这些协调者往往使用固定规则并忽视不确定性。我们将协调过程表述为成本敏感的序列假设检验：贝叶斯控制器维护对候选正确性的信念，并动态决定是收集更多证据、细化候选、进行验证还是停止。在六个生成器和九个编码基准测试中，贝叶斯控制在验证成本高且批评者提供的信息虽然有用但并不完美时表现出最大的价值。除了控制之外，信念状态还产生了一个可解释的正确性评分，优于基于标记概率和原始工具成功率的基线，用于不确定性量化。

View on arXiv Download PDF AI Translation

cs.AI / 38 / 2606.24467

CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference

CompressKV：基于语义检索的KV缓存压缩框架，旨在实现资源高效的长上下文大语言模型推理

Lin, Xiaolin, Wang, Jingcun, Kondrateva, Olga, Shi, Yiyu, Li, Bing, Zhang, Grace Li

Abstract

Long-context large language model (LLM) inference is increasingly constrained by the memory footprint and decoding cost of key-value (KV) caches, limiting sustainable deployment on resource-constrained hardware. Existing KV cache eviction methods typically apply heuristic token scoring over all heads in GQA-based LLMs. These methods ignore the different functionalities of attention heads, leading to the eviction of critical tokens and thus degrading the performance of LLMs. To address this issue, we propose CompressKV, a resource-efficient KV-cache compression framework for GQA-based LLMs. Instead of aggregating attention scores from all heads, CompressKV identifies Semantic Retrieval Heads (SRHs) that capture both the initial and final tokens of a prompt and semantically important mid-context evidence, and uses them to select tokens whose KV pairs should be retained. Furthermore, CompressKV allocates cache budgets across layers according to offline estimates of layer-wise eviction error. Experiments on LongBench and Needle-in-a-Haystack show that CompressKV consistently outperforms existing KV-cache eviction methods across memory budgets. Notably, it preserves over 97\% of full-cache performance using only 3\% of the KV cache on LongBench question-answering tasks and achieves 90\% accuracy with just 0.7\% KV storage on Needle-in-a-Haystack. These results demonstrate an improved resource--performance trade-off for long-context LLM inference. Our code is publicly available at: https://github.com/TUDa-HWAI/CompressKV

Chinese Translation

长上下文大语言模型（LLM）推理越来越受到键值（KV）缓存的内存占用和解码成本的限制，这限制了在资源受限硬件上的可持续部署。现有的KV缓存驱逐方法通常在基于GQA的LLM中对所有头部应用启发式的令牌评分。这些方法忽视了注意力头部的不同功能，导致关键令牌被驱逐，从而降低了LLM的性能。为了解决这个问题，我们提出了CompressKV，一个针对基于GQA的LLM的资源高效KV缓存压缩框架。CompressKV并不是简单地聚合所有头部的注意力分数，而是识别出能够捕捉提示的初始和最终令牌以及语义重要的中间上下文证据的语义检索头（SRH），并利用它们选择应保留的KV对的令牌。此外，CompressKV根据层级驱逐错误的离线估计在各层之间分配缓存预算。在LongBench和Needle-in-a-Haystack上的实验表明，CompressKV在各种内存预算下始终优于现有的KV缓存驱逐方法。值得注意的是，在LongBench问答任务中，它仅使用3%的KV缓存就保留了超过97%的完整缓存性能，而在Needle-in-a-Haystack中仅用0.7%的KV存储就达到了90%的准确率。这些结果展示了长上下文LLM推理在资源与性能之间的改进权衡。我们的代码已公开发布在：https://github.com/TUDa-HWAI/CompressKV

View on arXiv Download PDF AI Translation

cs.AI / 39 / 2606.24470

The Latent Bridge: A Continuous Slow-Fast Channel for Real-Time Game Agents

潜在桥梁：实时游戏代理的连续慢快通道

Li, Bojie, Shi, Noah

Abstract

A real-time agent for general computer use - with games as the most demanding case - must act within tens of milliseconds while still planning over seconds. These two regimes sit at opposite ends of the latency-quality tradeoff. A reasoning VLM (Qwen3-VL-8B-Thinking) deliberates effectively but requires ~1.5 s per response - far too slow for a 15 Hz control loop. In contrast, a reactive VLM (MiniCPM-o 4.5) acts in milliseconds but underperforms on planning-heavy tasks. We couple two frozen models of matched scale (9B reactive, 8B reasoning), leaving the communication channel as the sole trainable component. The standard coupling is a Text Bridge (T): the slow model writes a suffix the fast model reads. We introduce a learned continuous Latent Bridge (L) that projects the slow model's residuals into the fast model's input-embedding space in a LLaVA-style manner, avoiding any text round-trip; both are compared against Fast-Only (F). On 7 Atari games and a driving domain (MetaDrive), tuning the action decoder per channel on held-out seeds, the Latent Bridge matches or beats the Text Bridge in every domain: it significantly improves two games (MsPacman +57%, RoadRunner +28%) and is a safe drop-in elsewhere. Combining both channels interferes destructively (RoadRunner -96%), so only one should be used. The benefit is highly predictable: the bridge helps if and only if slow reasoning already beats fast reaction (T > F) - the Latent and Text gains over Fast-Only move together at r=0.93. MetaDrive is the controlled negative, where the Latent Bridge is demonstrably inert because the Text Bridge adds no value. We release replay recordings and reproducible pipelines.

Chinese Translation

通用计算用途的实时代理——以游戏作为最具挑战性的案例——必须在几十毫秒内做出反应，同时仍需在几秒钟内进行规划。这两种模式处于延迟与质量权衡的两个极端。推理型 VLM（Qwen3-VL-8B-Thinking）能够有效推理，但每次响应需要约 1.5 秒——对于 15 Hz 的控制循环来说速度过慢。相反，反应型 VLM（MiniCPM-o 4.5）能够在毫秒内做出反应，但在重规划任务中表现不佳。我们将两个相匹配规模的冻结模型（9B 反应型，8B 推理型）结合起来，通信通道成为唯一可训练的组件。标准的耦合方式是文本桥（Text Bridge, T）：慢模型写入后缀，快模型读取。我们引入了一种学习的连续潜在桥（Latent Bridge, L），以 LLaVA 风格的方式将慢模型的残差投影到快模型的输入嵌入空间中，避免了任何文本往返；两者与仅快模型（Fast-Only, F）进行比较。在 7 款 Atari 游戏和一个驾驶领域（MetaDrive）中，针对保留种子对每个通道调优动作解码器，潜在桥在每个领域中均与文本桥相匹配或超越：它显著改善了两个游戏（MsPacman +57%，RoadRunner +28%），并且在其他地方安全可用。结合两个通道会产生破坏性干扰（RoadRunner -96%），因此只能使用一个。其益处是高度可预测的：桥梁仅在慢推理已经优于快反应（T > F）时才有帮助——潜在桥和文本桥相对于仅快模型的增益呈现出 r=0.93 的一致性。MetaDrive 是一个受控的负例，其中潜在桥显著无效，因为文本桥没有增加任何价值。我们发布了重放录音和可复现的管道。

View on arXiv Download PDF AI Translation

cs.AI / 40 / 2606.24504

On the Smallness of the Large Language Models Scaling Exponents

大型语言模型缩放指数的微小性

Succi, Sauro, Coveney, Peter V., Hansen, Alex

Abstract

We discuss reasons why the scaling exponents of current Large Language Models (LLMs) applications are indicating an unsustainable regime in terms of energy resources. We further show that attributing the smallness of such exponents to a numerical bias due to the neglect of a non-zero value of the loss function in the limit of infinite data (``pedestal effect") does not remove the unsustainability issue. Finally, the effects of the smoothness (roughness) of the data on the scaling exponents is commented upon based on an analogy with phenomenological models of fluid turbulence.

Chinese Translation

我们讨论了当前大型语言模型（Large Language Models, LLMs）应用的缩放指数为何表明在能源资源方面存在不可持续的状态。我们进一步表明，将这种指数的微小性归因于由于忽视在无限数据极限下损失函数的非零值所导致的数值偏差（“基座效应”）并不能消除不可持续性的问题。最后，基于与流体湍流的现象模型的类比，我们评论了数据的平滑性（粗糙性）对缩放指数的影响。

View on arXiv Download PDF AI Translation

cs.AI / 41 / 2606.24510

A specialized reasoning large language model for accelerating rare disease diagnosis: a randomized AI physician assistance trial

加速罕见疾病诊断的专用推理大型语言模型：随机化人工智能医生辅助试验

Chen, Haichao, Zhou, Songchi, Zhao, Zhengyun, Hu, Shikai, Jin, Xianghong, Ji, Hongwei, He, Li, Li, Shuli, Qin, Yiming, Tan, Xin, Shi, Runfeng, Tham, Yih Chung, Zhu, Jiaye, Li, Ye, Jin, Ye, Cao, Longhao, Li, Dawei, Wu, Honghan, Gu, Hongqiu, Li, Guanqiao, Groza, Tudor, Li, Chunying, Zeng, Dian, Yu, Weihong, Baynam, Gareth, Jamuar, Saumya Shekhar, Shen, Min, Zhang, Shuyang, Sheng, Bin, Yu, Sheng, Wong, Tien Yin

Abstract

Rare diseases affect millions of individuals worldwide, yet timely diagnosis remains a major public health challenge due to scarcity of specialized clinical expertise. While large language models (LLMs) show promise to support rare disease diagnosis, current models are constrained by insufficient clinical deployability, limited clinically grounded evidence, and scarcity of training data. Here we present RaDaR (Rare Disease navigatoR), an open-source, compact reasoning LLM (32B parameters) for rare disease diagnosis. RaDaR was trained with 49,170 publicly available free-text cases and 104,666 synthetic cases with reasoning-enhanced training. RaDaR showed the strongest performance among evaluated open-source models, including the 671B DeepSeek-R1, across public benchmarks and four external validation centers. In a retrospective cohort, RaDaR prioritized the final diagnosis before documented clinical suspicion in 61.06 percent of cases, corresponding to a potential lead time of 1.87 months and 50.18 percent of the within-center interval. In a randomized physician-assistance trial, RaDaR assistance improved physicians' rare-disease diagnostic accuracy by 21.44 percentage points compared with internet search alone. Synthetic-data ablations suggested that phenotype-anchored narratives provide useful training signal for long-tail rare diseases, with a monotonic scaling trend within the tested data range. Together, RaDaR and its development and validation framework provide a deployable rare-disease reasoning model and a reproducible development framework for diagnostic AI under data scarcity.

Chinese Translation

罕见疾病影响着全球数百万人的健康，但由于缺乏专业临床专长，及时诊断仍然是一个主要的公共卫生挑战。尽管大型语言模型（LLMs）在支持罕见疾病诊断方面显示出潜力，但当前模型受到临床可部署性不足、临床基础证据有限和训练数据稀缺的限制。在此，我们介绍了RaDaR（罕见疾病导航器），这是一个开源的、紧凑的推理大型语言模型（32B参数），用于罕见疾病的诊断。RaDaR使用49,170个公开可用的自由文本病例和104,666个经过推理增强训练的合成病例进行训练。RaDaR在评估的开源模型中表现最强，包括671B的DeepSeek-R1，在公共基准和四个外部验证中心中均表现优异。在一项回顾性队列研究中，RaDaR在61.06%的病例中优先考虑最终诊断，早于记录的临床怀疑，潜在的提前时间为1.87个月，占中心内间隔的50.18%。在一项随机化医生辅助试验中，与仅依赖互联网搜索相比，RaDaR的辅助提高了医生的罕见疾病诊断准确率21.44个百分点。合成数据的消融实验表明，基于表型的叙述为长尾罕见疾病提供了有用的训练信号，并在测试的数据范围内呈现单调扩展趋势。总之，RaDaR及其开发和验证框架提供了一个可部署的罕见疾病推理模型和一个在数据稀缺情况下可重复的诊断人工智能开发框架。

View on arXiv Download PDF AI Translation

cs.AI / 42 / 2606.24515

Reinforcement Learning for Computer-Use Agents with Autonomous Evaluation

具有自主评估的计算机使用代理的强化学习

Sumyk, Marta, Kosovan, Oleksandr

Abstract

Computer-Use Agents (CUAs) execute high-level user goals by perceiving and acting directly within graphical user interfaces. However, reinforcement learning for CUAs remains difficult because open-ended desktop environments rarely provide scalable, machine-readable reward signals: task success is often visually grounded and hard to specify with handcrafted reward functions or dense manual labels. We propose an RL fine-tuning framework that uses autonomous vision-language evaluation as a scalable supervision signal for GUI agents. Given a final screenshot and the original instruction, a Vision-Language Model judges task completion and provides terminal feedback without task-specific heuristics or manual labels during policy optimization. Because autonomous evaluators are imperfect, we model their feedback as a noisy binary reward channel and derive a noise-corrected reward estimator for Proximal Policy Optimization. Experiments across macOSWorld, Windows Agent Arena, and OSWorld show that corrected evaluator rewards outperform both zero-shot baselines and raw evaluator rewards, improving success rates by an average of 12.6 percentage points over zero-shot performance and 5.1 points over raw evaluator fine-tuning. These results suggest that autonomous evaluation can serve as a practical reward signal for RL in GUI environments when evaluator noise is explicitly modeled and corrected.

Chinese Translation

计算机使用代理（CUAs）通过直接在图形用户界面中感知和行动来执行高层次的用户目标。然而，CUAs的强化学习仍然面临困难，因为开放式桌面环境很少提供可扩展的、机器可读的奖励信号：任务成功往往是视觉基础的，难以通过手工设计的奖励函数或密集的手动标签来具体化。我们提出了一种强化学习微调框架，利用自主视觉-语言评估作为图形用户界面代理的可扩展监督信号。给定最终的屏幕截图和原始指令，视觉-语言模型判断任务完成情况，并在策略优化过程中提供终端反馈，而无需特定任务的启发式方法或手动标签。由于自主评估者并不完美，我们将其反馈建模为一个嘈杂的二元奖励通道，并为近端策略优化（Proximal Policy Optimization）推导出一个噪声校正奖励估计器。在macOSWorld、Windows Agent Arena和OSWorld上的实验表明，校正后的评估者奖励在零样本基线和原始评估者奖励上均表现更优，成功率平均提高了12.6个百分点，相较于零样本性能提升了5.1个百分点。这些结果表明，当评估者噪声被明确建模和校正时，自主评估可以作为图形用户界面环境中强化学习的实用奖励信号。

View on arXiv Download PDF AI Translation

cs.AI / 43 / 2606.24535

Governed Shared Memory for Multi-Agent LLM Systems

多智能体大语言模型系统的管控共享内存

Margalit, Yanki, Cohen-Inger, Nurit, Avram, Erni, Taig, Ran, Margalit, Oded

Abstract

Multi-agent LLM environments require robust mechanisms for shared knowledge management. This paper formalizes the fleet-memory problem and identifies four foundational failure modes: unauthorized leakage, stale propagation, contradiction persistence, and provenance collapse. To address these, we define explicit systems-level primitives: scoped retrieval, temporal supersession, provenance tracking, and policy-governed memory propagation. These primitives are implemented in MemClaw, a production multi-tenant memory service, and evaluated via ArgusFleet, a reproducible harness testing four governance dimensions. Rather than a baseline comparison, this study measures a live production service, emphasizing real-world architectural insights and negative results. Key Evaluation Results Provenance: Successfully reconstructed 100% of depth-four derivation chains with correct writer identity at sub-second per-hop latency. Propagation: Demonstrated high intra-fleet visibility with zero cross-fleet leakage. Under strong write mode, write-to-visible latency was optimized to a single search round-trip. Production Architectural Issues Discovered Asymmetric Scope Enforcement: Tenant isolation held, but sub-tenant scope was initially bypassed on direct GET-by-id requests for agent-scoped credentials (disclosed and remediated during the study). Pipeline Ordering Conflict: While contradiction supersession works for admitted writes, a synchronous near-duplicate gate can prematurely reject contradictory writes before the asynchronous contradiction detector can evaluate them. Conclusion: Long-context retrieval alone is insufficient for production multi-agent memory. Governed shared memory demands explicit systems-level abstractions, and live evaluation is vital to expose enforcement and pipeline-ordering failures missed by design-only treatments.

Chinese Translation

多智能体大语言模型环境需要强健的共享知识管理机制。本文形式化了舰队内存问题，并识别出四种基础的失败模式：未经授权的泄漏、过时的传播、矛盾的持续存在和来源崩溃。为了解决这些问题，我们定义了明确的系统级原语：范围检索、时间超越、来源追踪和政策管控的内存传播。这些原语在 MemClaw 中得以实现，MemClaw 是一个生产级的多租户内存服务，并通过 ArgusFleet 进行评估，后者是一个可重复的测试框架，测试四个治理维度。与基线比较不同，本研究测量了一个实时生产服务，强调了现实世界的架构洞察和负面结果。关键评估结果：来源：成功重建了 100% 的深度四推导链，且每跳延迟低于一秒。传播：展示了高内舰队可见性且无跨舰队泄漏。在强写入模式下，写入到可见的延迟优化为单次搜索往返。发现的生产架构问题：不对称范围执行：租户隔离保持有效，但在直接通过 ID 请求代理范围凭证时，子租户范围最初被绕过（在研究过程中披露并修复）。管道排序冲突：尽管矛盾超越适用于已接受的写入，但同步的近重复门可能在异步矛盾检测器评估之前过早拒绝矛盾写入。结论：单靠长上下文检索不足以满足生产多智能体内存的需求。管控共享内存要求明确的系统级抽象，而实时评估对于揭示设计仅处理所遗漏的执行和管道排序失败至关重要。

View on arXiv Download PDF AI Translation

cs.AI / 44 / 2606.24551

GUI vs. CLI: Execution Bottlenecks in Screen-Only and Skill-Mediated Computer-Use Agents

图形用户界面与命令行界面：仅屏幕和技能中介计算机使用代理的执行瓶颈

Zhou, Xiao, Zhang, Siyue, Zhao, Yilun, Wei, Jinbiao, Song, Tingyu, Cohan, Arman, Zhao, Chen

Abstract

Computer-use agents can execute software tasks through either graphical interfaces or programmatic command interfaces, but existing evaluations confound interaction modality with differences in tasks, initial states, verifiers, and permitted actions. We introduce a matched execution-layer benchmark of 440 desktop tasks across 18 applications and 12 workflow categories, where screen-only GUI agents and skill-mediated CLI agents receive identical goals, states, and final-state verifiers while being restricted to modality-native actions. In this controlled setting, the strongest GUI agent reaches a 59.1% full pass rate, outperforming the strongest original-skill CLI agent at 48.2%; however, verifier-guided skill augmentation raises CLI success to 69.3%, showing that much of the CLI deficit comes from incomplete skill coverage rather than model capability alone. These results suggest that GUI and CLI expose different execution bottlenecks: GUI agents are limited by reliable grounded interaction over long-horizon workflows, whereas CLI agents are limited by the coverage and scalability of their skill interfaces.

Chinese Translation

计算机使用代理可以通过图形界面或程序化命令接口执行软件任务，但现有评估将交互方式与任务、初始状态、验证者和允许的操作的差异混淆在一起。我们引入了一个匹配的执行层基准，涵盖18个应用程序和12个工作流类别中的440个桌面任务，其中仅屏幕的图形用户界面代理和技能中介的命令行界面代理在相同的目标、状态和最终状态验证者下执行，同时限制为各自模式的本地操作。在这一受控环境中，最强的图形用户界面代理达到了59.1%的完全通过率，优于最强的原始技能命令行界面代理的48.2%；然而，验证者指导的技能增强将命令行界面的成功率提高至69.3%，显示出命令行界面的不足主要源于技能覆盖的不完整，而不仅仅是模型能力。这些结果表明，图形用户界面和命令行界面暴露了不同的执行瓶颈：图形用户界面代理受到长时间工作流中可靠的基础交互的限制，而命令行界面代理则受到其技能接口的覆盖范围和可扩展性的限制。

View on arXiv Download PDF AI Translation

cs.AI / 45 / 2606.24575

Quant Convergence: Bridging Classical Value Investing and Modern Factor Models for Systematic Equity Selection

量化收敛：桥接经典价值投资与现代因子模型以实现系统化股票选择

Yamazaki, Augusto Eiji, Belinchon, Hugo Garrido-Lestache

Abstract

Modern finance relies heavily on complex machine learning models to find patterns in the stock market. However, as these AI models get more complicated, they often memorize short-term market noise instead of finding companies with real, lasting value. We designed this research to test if Benjamin Graham's classic value investing rules could act as a mathematical "low-pass filter" to keep these modern models in check. We built three different sets of features - pure Graham rules, modern market factors, and a mix of both - and tested them against highly complex models (XGBoost and AutoGluon) using 20 years of S&P 500 data. By applying a strict buy-and-hold strategy over a four-year test period (March 2022 to March 2026), the results showed that more complex algorithms do not always win. While the AutoGluon model captured high returns (222.68%), it suffered a substantial 39.78% drop because it bought volatile tech stocks right before the market crashed. On the other hand, the pure Graham Random Forest achieved the highest overall return (232.13%) with much less risk (1.38 Calmar Ratio). Furthermore, the Combined Random Forest successfully mixed momentum with Graham's rules, making a 202.91% return while keeping the lowest maximum drop (34.53%) of any model tested. Ultimately, this research proves that Graham's "margin of safety" isn't outdated; it is actually a highly effective way to prevent modern AI from taking on too much risk.

Chinese Translation

现代金融在很大程度上依赖复杂的机器学习模型来寻找股市中的模式。然而，随着这些人工智能模型变得越来越复杂，它们往往会记忆短期市场噪音，而不是找到具有真实、持久价值的公司。我们设计了这项研究，以测试本杰明·格雷厄姆（Benjamin Graham）的经典价值投资规则是否可以作为一种数学“低通滤波器”，以保持这些现代模型的稳定性。我们构建了三组不同的特征——纯格雷厄姆规则、现代市场因子以及两者的混合——并使用20年的标准普尔500指数（S&P 500）数据对它们进行了测试，比较了高度复杂的模型（XGBoost和AutoGluon）。通过在四年的测试期内（2022年3月至2026年3月）应用严格的买入持有策略，结果显示更复杂的算法并不总是获胜。虽然AutoGluon模型获得了高达222.68%的回报，但由于在市场崩溃前购买了波动性较大的科技股，它遭遇了39.78%的重大下跌。另一方面，纯格雷厄姆随机森林实现了最高的整体回报（232.13%），且风险较低（1.38 Calmar比率）。此外，组合随机森林成功地将动量与格雷厄姆的规则结合，获得了202.91%的回报，同时保持了所有测试模型中最低的最大回撤（34.53%）。最终，这项研究证明了格雷厄姆的“安全边际”并未过时；实际上，它是一种非常有效的方法，可以防止现代人工智能承担过多风险。

View on arXiv Download PDF AI Translation

cs.AI / 46 / 2606.24585

LLMs Prompted for Legal Context Object More: Overrefusal from Small On-Premises LLMs in Criminal Legal Context

法律背景下提示的LLMs更多拒绝：小型本地LLMs在刑事法律背景中的过度拒绝

Kucherenko, Anastasiia, Brouchoud, François, David, Dimitri Percia, Kucharavy, Andrei

Abstract

While the validity of LLMs' use in the legal context remains subject to ethical and legal debate, legal professionals are already experimenting with personal LLMs, if only for translation and reformulation. However, even such a seemingly innocuous use can introduce biases through case processing speed if LLM assistants selectively refuse assistance on certain topics. To better anticipate such biases, we investigate several modern small LLMs that are most likely to be used as on-device assistants, to assess the impact of overrefusal on legal prompts. Surprisingly, we find that authority-style prefixes (``you are acting as an assistant of the national supreme court'', ``[...] defense lawyer'') systematically increase refusal rates by 2--20x over the no-prefix baseline, while a known role-play jailbreak prefix shows mixed effects, sharply increasing refusals in some models and barely shifting them in others. The finding suggests that small on-prem deployable LLMs are unstable under contextual framings that a real institutional user might naturally introduce, and further investigation is essential to minimize opportunities for bias.

Chinese Translation

尽管LLMs在法律背景下的使用有效性仍然受到伦理和法律的争论，法律专业人士已经开始尝试使用个人LLMs，即使仅用于翻译和改写。然而，即使是这样的看似无害的使用也可能通过案件处理速度引入偏见，如果LLM助手在某些主题上选择性地拒绝提供帮助。为了更好地预见这种偏见，我们研究了几种现代小型LLMs，这些模型最有可能作为本地设备助手使用，以评估过度拒绝对法律提示的影响。令人惊讶的是，我们发现权威风格的前缀（“你正在作为国家最高法院的助手行事”，“[...] 辩护律师”）系统性地将拒绝率提高了2到20倍，相较于无前缀的基线，而一个已知的角色扮演越狱前缀则显示出混合效果，在某些模型中拒绝率急剧增加，而在其他模型中几乎没有变化。该发现表明，小型本地可部署的LLMs在真实机构用户可能自然引入的上下文框架下是不稳定的，进一步的研究对于最小化偏见的机会至关重要。

View on arXiv Download PDF AI Translation

cs.AI / 47 / 2606.24589

AdversaBench: Automated LLM Red-Teaming with Multi-Judge Confirmation and Cross-Model Transferability

AdversaBench：具有多评审确认和跨模型可转移性的自动化大型语言模型红队评估

Khandelwal, Khanak

Abstract

Scaling adversarial evaluation of large language models requires both a method for generating hard inputs and a reliable way to confirm that resulting failures are real. We present AdversaBench, an end-to-end red-teaming pipeline that mutates seed prompts with five structured operators, queries a target model, and confirms failures through a three-judge panel with a meta-judge tiebreaker. We report experiments on 45 seeds across three categories: reasoning, instruction-following, and tool use. Every seed produced a confirmed failure. Four findings stand out. First, operator effectiveness varies sharply by category: inject_distractor scores 0.00 mean reward on instruction-following seeds but 0.80-0.83 on reasoning and tool-use. Second, binary failure rate hides difficulty: instruction-following seeds required 2.4 attacker iterations on average versus 1.1 for other categories, a gap visible in survival curves. Third, pairwise judge agreement of 80-87% coexists with near-zero Cohen's kappa due to label skew; category-level disagreement rates are more informative. Fourth, adversarial prompts generated against Llama 3.1 8B transfer zero-shot to Llama 3.3 70B, suggesting the mutations exploit general behavioral patterns rather than model-specific weaknesses. Code, dataset, and analysis scripts are available at https://github.com/khanak0509/AdversaBench .

Chinese Translation

对大型语言模型进行对抗性评估的规模化需要一种生成困难输入的方法，以及一种可靠的方式来确认结果失败的真实性。我们提出了AdversaBench，一个端到端的红队评估管道，它通过五个结构化操作符对种子提示进行变异，查询目标模型，并通过一个由三名评审组成的小组及一个元评审的平局裁决者确认失败。我们在三个类别（推理、遵循指令和工具使用）上对45个种子进行了实验。每个种子都产生了确认的失败。有四个发现值得注意。首先，操作符的有效性在不同类别之间差异显著：在遵循指令的种子上，inject_distractor的平均奖励为0.00，而在推理和工具使用上则为0.80-0.83。第二，二元失败率掩盖了难度：遵循指令的种子平均需要2.4次攻击者迭代，而其他类别仅为1.1次，这一差距在生存曲线中可见。第三，评审之间的成对一致性为80-87%，但由于标签偏斜，Cohen's kappa几乎为零；类别级别的不一致率更具信息性。第四，对Llama 3.1 8B生成的对抗性提示在零-shot情况下转移到Llama 3.3 70B，表明这些变异利用了一般行为模式，而非模型特定的弱点。代码、数据集和分析脚本可在https://github.com/khanak0509/AdversaBench获取。

View on arXiv Download PDF AI Translation

cs.AI / 48 / 2606.24601

ASALT: Adaptive State Alignment for Lateral Transfer in Multi-agent Reinforcement Learning

ASALT：用于多智能体强化学习中的横向转移的自适应状态对齐

Akula, Anurag, Perepu, Satheesh K., Sarkar, Abhishek, Dey, Kaushik

Abstract

Multi-agent reinforcement learning (MARL) addresses the problem of training multiple agents that pursue collaborative, competitive, or mixed objectives. Prior work has investigated transfer learning between source and target domains in MARL; however, the majority of existing approaches impose the constraint that the dimensionalities of the observation space and the global state space must be identical across domains. In this paper, we introduce a method that explicitly accommodates mismatched state-space dimensionalities between source and target domains. The proposed approach, ASALT, incorporates both observation-level and state-level adapters that map the target-domain observations and global states into a shared embedding space, thereby enabling more effective transfer of knowledge across both actors and critics. These adapters can generate embeddings that support efficient strategy transfer across heterogeneous domains. Experimental results on multiple configurations in standard benchmark environments demonstrate that ASALT surpasses existing baselines in terms of sample efficiency and global return in cooperative settings, but its effectiveness depends on the degree of mismatch between source and target domains. Furthermore, our findings indicate that ASALT mitigates negative transfer, which frequently constitutes a major obstacle when transferring policies between domains with differing observation and action spaces.

Chinese Translation

多智能体强化学习（MARL）解决了训练多个追求协作、竞争或混合目标的智能体的问题。先前的研究探讨了MARL中源域与目标域之间的迁移学习；然而，现有大多数方法都强加了观察空间和全局状态空间的维度必须在各个域中相同的限制。本文提出了一种方法，明确考虑了源域与目标域之间状态空间维度的不匹配。所提出的方法ASALT结合了观察级和状态级适配器，将目标域的观察和全局状态映射到共享的嵌入空间，从而实现了知识在行为者和评论者之间的更有效转移。这些适配器能够生成支持在异构域之间高效策略转移的嵌入。多个标准基准环境中的实验结果表明，ASALT在合作环境中在样本效率和全局回报方面超越了现有基线，但其有效性依赖于源域与目标域之间的不匹配程度。此外，我们的研究结果表明，ASALT减轻了负迁移，这通常是不同观察和动作空间之间转移策略时的主要障碍。

View on arXiv Download PDF AI Translation

cs.AI / 49 / 2606.24604

Uncertainty-Aware Longitudinal Forecasting of Alzheimer's Disease Progression Using Deep Learning

基于深度学习的阿尔茨海默病进展的不确定性感知纵向预测

Hariharan, Arya, Gowda, Shreyank N, R, Anala M

Abstract

Longitudinal modelling of Alzheimer's disease progression is clinically useful only if it can describe not just the most likely next diagnosis, but how a patient may evolve over time and how reliable that forecast is. Most deep learning approaches reduce this problem to single-step classification, treating cognitively normal, mild cognitive impairment, and dementia as flat categories while providing limited insight into how uncertainty accumulates across future visits. We propose a probabilistic framework that combines ordinal diagnosis prediction, multi-horizon trajectory generation, and decomposed uncertainty estimation. A Temporal Fusion Transformer encoder is adapted with a CORAL ordinal output layer, asymmetric loss weighting, and converter oversampling to respect disease-stage ordering and improve sensitivity to MCI-to-dementia transitions. Conditioned on the learned patient-context representation, an autoregressive Mixture Density Network generates five-year probabilistic trajectories for diagnosis state, CDR Sum of Boxes, MMSE orientation, and hippocampal volume. On ADNI, the model outperforms linear, recurrent, and transformer baselines for next-visit diagnosis prediction, with the strongest gains on MCI-versus-dementia discrimination. Generated trajectories achieve near-nominal 90% credible interval coverage, widening uncertainty across the forecast horizon, and biomarker dynamics consistent with expected Alzheimer's disease progression. We further separate aleatoric from epistemic uncertainty using analytic mixture variance and a five-member bootstrap ensemble, which provides the strongest encoder diversity and output-level epistemic signal. Epistemic uncertainty is higher for rare progression archetypes, MCI and dementia patients, and under external evaluation on OASIS-3, where it increases alongside prediction error.

Chinese Translation

阿尔茨海默病进展的纵向建模只有在能够描述患者可能的演变过程及其预测的可靠性时，才具有临床价值。大多数深度学习方法将此问题简化为单步分类，将认知正常、轻度认知障碍和痴呆视为平面类别，同时对未来就诊中不确定性如何累积提供有限的洞察。我们提出了一种概率框架，结合了序数诊断预测、多时间跨度轨迹生成和分解的不确定性估计。我们对时间融合变换器（Temporal Fusion Transformer）编码器进行了调整，增加了CORAL序数输出层、不对称损失加权和转换器过采样，以尊重疾病阶段的顺序并提高对轻度认知障碍向痴呆过渡的敏感性。在学习到的患者背景表示的条件下，自回归混合密度网络（Mixture Density Network）生成五年内的诊断状态、CDR总和、MMSE定向和海马体积的概率轨迹。在ADNI数据集上，该模型在下次就诊诊断预测中优于线性、递归和变换器基线，尤其在轻度认知障碍与痴呆的区分上取得了显著提升。生成的轨迹在预测区间内实现了近乎名义的90%可信区间覆盖，随着预测时间的延长而扩大不确定性，并且生物标志物动态与预期的阿尔茨海默病进展一致。我们进一步使用解析混合方差和五成员自助集成（bootstrap ensemble）将随机不确定性与认知不确定性分离，这提供了最强的编码器多样性和输出级别的认知信号。对于稀有进展原型、轻度认知障碍和痴呆患者，认知不确定性较高，并且在OASIS-3的外部评估中，随着预测误差的增加而增加。

View on arXiv Download PDF AI Translation

cs.AI / 50 / 2606.24605

ScaleToT: Generalizing Structured LLM Reasoning for Billion-Scale Low-Activity User Modeling

ScaleToT：针对十亿规模低活跃用户建模的结构化大语言模型推理的泛化

Ma, Tianbao, Xi, Chang, Zou, Yichuan, Li, Chengen, Chen, Linxun, Lu, Zilong, Niu, Yanan, Liu, Zhaojie, Li, Han, Gai, Kun

Abstract

Accurate user modeling often depends on rich interaction histories, which are unavailable for billions of low-activity users. Large Language Models (LLMs) can infer latent user states from static profiles, but this reasoning becomes unreliable when profiles are sparse, and applying an LLM to billions of users is prohibitively expensive. We present ScaleToT, which learns structured reasoning from a small LLM-processed subset and extends it to the broader low-activity user population. To improve reasoning reliability, ScaleToT constructs typed user-state chains with a bounded entropy-guided Tree-of-Thought (ToT) refinement procedure. To make this structured reasoning usable from sparse profiles, the teacher-curated chains are used to train a student model on static profiles through supervised fine-tuning (SFT) and Outcome-Driven Segment-Aware Implicit Reward Policy Optimization (OSIPO). ScaleToT then transfers the student's reasoning representations to a lightweight profile encoder, providing shared reasoning signals for the remaining users without LLM inference. We evaluate ScaleToT on lifetime value (LTV) prediction in a billion-scale advertising deployment. A randomized online A/B test increased LT30 by 6.738\%, while offline reasoning covered only 7.32\% of the potential population, greatly reducing compute cost compared with full-population reasoning.

Chinese Translation

准确的用户建模通常依赖于丰富的交互历史，而对于十亿低活跃用户而言，这些历史数据并不可用。大型语言模型（LLMs）可以从静态档案中推断潜在的用户状态，但当档案稀疏时，这种推理变得不可靠，并且将LLM应用于十亿用户的成本极高。我们提出了ScaleToT，它从一个小型LLM处理的子集学习结构化推理，并将其扩展到更广泛的低活跃用户群体。为了提高推理的可靠性，ScaleToT构建了带有有界熵引导的思维树（Tree-of-Thought, ToT）精炼过程的类型化用户状态链。为了使这种结构化推理能够从稀疏档案中使用，教师策划的链条被用于通过监督微调（Supervised Fine-Tuning, SFT）和以结果为驱动的段落感知隐式奖励策略优化（Outcome-Driven Segment-Aware Implicit Reward Policy Optimization, OSIPO）在静态档案上训练学生模型。随后，ScaleToT将学生的推理表示转移到一个轻量级档案编码器，为其余用户提供共享的推理信号，而无需进行LLM推理。我们在一个十亿规模的广告部署中评估了ScaleToT在生命周期价值（Lifetime Value, LTV）预测方面的表现。一项随机在线A/B测试使LT30提高了6.738%，而离线推理仅覆盖了潜在用户的7.32%，与全人口推理相比，大大降低了计算成本。

View on arXiv Download PDF AI Translation

cs.AI / 51 / 2606.24616

AI Tokenomics: The Economics of Tokens, Computation, and Pricing in Foundation Models

人工智能代币经济学：基础模型中的代币、计算和定价经济学

Zhu, Quanyan

Abstract

Tokens have become the practical accounting unit for modern foundation model services, linking information processing, computation, memory use, energy expenditure, pricing, and economic value. This paper develops a framework for AI tokenomics: the study of how tokens are generated, consumed, priced, allocated, and optimized across AI systems. We connect token-level technical costs to workflow-level production functions, enterprise resource allocation, measurement and instrumentation methods, and emerging market-design questions. The framework shows that token expenditure and economic value are distinct: value depends on marginal productivity, workflow position, hidden reasoning activity, risk, and downstream propagation effects. The paper concludes by identifying open research directions in hidden-token measurement, empirical calibration, token productivity, dynamic allocation, and token-based markets.

Chinese Translation

代币已成为现代基础模型服务的实际会计单位，连接了信息处理、计算、内存使用、能量消耗、定价和经济价值。本文建立了人工智能代币经济学的框架：研究代币在人工智能系统中如何生成、消费、定价、分配和优化。我们将代币层面的技术成本与工作流程层面的生产函数、企业资源配置、测量与仪器方法以及新兴市场设计问题联系起来。该框架表明，代币支出和经济价值是不同的：价值取决于边际生产力、工作流程位置、隐性推理活动、风险和下游传播效应。本文最后通过识别隐性代币测量、实证校准、代币生产力、动态分配和基于代币的市场等开放研究方向进行总结。

View on arXiv Download PDF AI Translation

cs.AI / 52 / 2606.24618

Abstractions of Queries in Ontology-Based Data Access

基于本体的数据访问中的查询抽象

Leclère, Michel, Mugnier, Marie-Laure, Pérution-Kihli, Guillaume

Abstract

In ontology-based data access (OBDA), multiple data sources are integrated via mappings to an ontology. We consider an OBDA setting based on existential rules and the certain answer semantics. We address the recent issue of query abstraction, which consists of abstracting data queries by translating them to the ontology layer. Since a perfect abstraction may not exist, the notions of minimally complete and maximally sound abstractions have been introduced. We study abstractions within an extension of UCQs with a limited form of inequality and a special predicate marking database constants. While this extension does not lead to an increased complexity of the problems of interest, it is able to express minimally complete abstractions, hence perfect abstractions when they exist. We also characterize maximally sound abstractions by making a new connection with the notion of maximum recovery stemming from data exchange.

Chinese Translation

在基于本体的数据访问（OBDA）中，通过映射将多个数据源集成到一个本体中。我们考虑基于存在规则和确定性答案语义的OBDA设置。我们解决了查询抽象的最新问题，该问题涉及通过将数据查询转换为本体层来抽象化数据查询。由于可能不存在完美的抽象，因此引入了最小完全抽象和最大可靠抽象的概念。我们研究了在有限形式的不等式和标记数据库常量的特殊谓词扩展的UCQs中的抽象。虽然这种扩展并不会导致相关问题的复杂性增加，但它能够表达最小完全抽象，因此在存在时也能实现完美抽象。我们还通过与数据交换中最大恢复的概念建立新的联系来表征最大可靠抽象。

View on arXiv Download PDF AI Translation

cs.AI / 53 / 2606.24619

When CQs Go Wrong: Challenges in CQ Verification with OE-Assist

当能力问题出现偏差时：使用OE-Assist进行能力问题验证的挑战

Lippolis, Anna Sofia, Saeedizade, Mohammad Javad, Keskisärkkä, Robin, Gangemi, Aldo, Blomqvist, Eva, Nuzzolese, Andrea Giovanni

Abstract

Competency Questions (CQs) are the central component of CQ-verification, an established process in which an ontology is evaluated against a set of natural language questions to determine whether the intended purpose of the ontology has been properly modelled. However, CQ-verification is often time-consuming and error-prone, as it requires careful interpretation of linguistic nuances and precise alignment with formal ontology constructs. Ambiguities and complexity in CQs can further complicate this process, leading to inconsistent modelling decisions and verification outcomes. In this paper, we investigate what makes a CQ challenging and possible solutions to enhance the users' performance in the CQ-verification process. We experimented with the data of 19 participants who performed CQ-verification on 20 tasks using an LLM assistant to support ontology evaluation. The results show the necessity of a tool to refine CQs before publishing them to avoid ambiguity or excessive complexity in later phases of the ontology engineering process.

Chinese Translation

能力问题（Competency Questions, CQs）是能力问题验证（CQ-verification）的核心组成部分，这一过程是通过一系列自然语言问题评估本体，以确定本体的预期目的是否得到了恰当建模。然而，能力问题验证常常耗时且容易出错，因为它需要对语言细微差别进行仔细解读，并与正式本体结构进行精确对齐。能力问题中的模糊性和复杂性可能进一步使这一过程复杂化，导致不一致的建模决策和验证结果。在本文中，我们探讨了使能力问题变得具有挑战性的因素以及可能的解决方案，以提高用户在能力问题验证过程中的表现。我们对19名参与者的数据进行了实验，他们使用大型语言模型（LLM）助手支持本体评估，在20个任务上执行能力问题验证。结果显示，在发布能力问题之前，使用工具对其进行精炼的必要性，以避免在本体工程过程后期出现模糊性或过度复杂性。

View on arXiv Download PDF AI Translation

cs.AI / 54 / 2606.24622

Themis: An explainable AI-enabled framework for Reinforcement Learning with Human Feedback

Themis：一个基于可解释人工智能的强化学习与人类反馈的框架

Chouliaras, Andreas, Connolly, Luke, Chatzpoulos, Dimitris

Abstract

Training safe Reinforcement Learning (RL) systems is inherently challenging, with no guarantee of avoiding unwanted behaviors. The most effective defenses against this are (i) transparency through explainability and (ii) alignment via human feedback. While both show promising results, no publicly available framework currently combines them. To address this, we introduce Themis, an XAI-enabled testing and evaluation framework for Reinforcement Learning from Human Feedback. Themis supports over 200 widely used environments and is easily configurable for experiments in RL, transparency, and alignment. Our results show that Themis can train reward models that match or outperform the environment's true reward signal using human preferences. We also provide a cloud-based platform for collecting human feedback and managing experiments. It is user-friendly, auto-scalable, and supports large participant groups across multiple experiments without extra development overhead. Tests show Themis can support one thousand users in back-to-back experiments on a modest commercial machine.

Chinese Translation

训练安全的强化学习（RL）系统本质上具有挑战性，无法保证避免不良行为。针对这一问题，最有效的防御措施是（i）通过可解释性实现透明性和（ii）通过人类反馈实现对齐。尽管这两者都显示出有希望的结果，但目前没有公开可用的框架将它们结合在一起。为了解决这一问题，我们提出了Themis，一个基于可解释人工智能（XAI）的强化学习与人类反馈的测试和评估框架。Themis支持200多个广泛使用的环境，并且易于配置以进行强化学习、透明性和对齐方面的实验。我们的结果表明，Themis能够训练出与环境的真实奖励信号相匹配或超越的奖励模型，使用的是人类偏好。我们还提供了一个基于云的平台，用于收集人类反馈和管理实验。该平台用户友好、自动扩展，并支持多个实验中的大型参与者群体，而无需额外的开发开销。测试表明，Themis能够在一台普通商业机器上支持一千名用户进行连续实验。

View on arXiv Download PDF AI Translation

cs.AI / 55 / 2606.24626

SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigation

SAFARI：通过主动调查扩展长时间跨度的代理故障归因

Zhu, Chenyang, Yao, Jiayu, Chawla, Kushal, Yin, Youbing, Wolfe, Nathan, Cai, Pengshan, Wu, Jingyu, Hong, Spencer, Cho, Sangwoo, Zhang, Shi-Xiong, Liu, Daben, Sahu, Sambit, Babinsky, Erin

Abstract

As autonomous agents tackle increasingly complex multi-step, multi-agent tasks, their execution trajectories have scaled beyond the constraints of even the largest context windows. Current methods for effectively diagnosing agent failures load the full trajectory into an LLM's context window, which suffers from attention dilution and fails when agentic traces inevitably exceed context limits. To address this, we introduce SAFARI (Scaling long-horizon Agentic Fault AttRibution via active Investigation), a framework that replaces linear context loading with a tool-augmented diagnostic loop. By equipping LLMs with a specialized toolbox to read and search trajectory segments alongside a persistent Short-Term Memory (STM) for cross-turn reasoning, SAFARI effectively decouples diagnostic accuracy from architectural context limits. Our experiments demonstrate that SAFARI outperforms state-of-the-art results by 20% on the Who&When dataset within a 1M token budget, and by 19% on TRAIL GAIA subset on a 25K token budget. Most significantly, SAFARI maintains a 0.58 precision even when the target fault resides 5x beyond the model's native context window, a scenario where traditional evaluators fail entirely.

Chinese Translation

随着自主代理处理越来越复杂的多步骤、多代理任务，它们的执行轨迹已超出甚至最大上下文窗口的限制。目前有效诊断代理故障的方法需要将完整轨迹加载到大型语言模型（LLM）的上下文窗口中，这会导致注意力稀释，并在代理痕迹不可避免地超出上下文限制时失效。为了解决这个问题，我们提出了SAFARI（通过主动调查扩展长时间跨度的代理故障归因），一个将线性上下文加载替换为工具增强的诊断循环的框架。通过为LLM配备一个专门的工具箱，以读取和搜索轨迹片段，并结合一个持久的短期记忆（STM）进行跨轮推理，SAFARI有效地将诊断准确性与架构上下文限制解耦。我们的实验表明，在100万标记预算下，SAFARI在Who&When数据集上的表现比最先进的结果提高了20%，在25000标记预算下，SAFARI在TRAIL GAIA子集上的表现提高了19%。最重要的是，即使目标故障位于模型原生上下文窗口外5倍的情况下，SAFARI仍然保持0.58的精度，而在这种情况下，传统评估方法完全失效。

View on arXiv Download PDF AI Translation

cs.AI / 56 / 2606.24636

CineCap: Structured Reasoning with Spatio-Temporal Anchors for Cinematographic Video Captioning

CineCap：基于时空锚点的结构化推理用于电影视频字幕生成

Mao, Xinyu, Zeng, Yuhui, Liu, Xiaokun, Qin, Wenyu, Wang, Meng, Tao, Xin, Wan, Pengfei, Xing, Xiaohan, Meng, Max

Abstract

Cinematographic captioning aims to describe how a video is filmed using professional film-language concepts such as camera movement, shot size, depth of field, composition, and shooting angle. This capability is important for fine-grained video understanding and controllable movie-quality video generation, yet remains underexplored in existing multimodal large language models. Unlike question-answering-based evaluation of cinematic understanding, cinematographic captioning requires a unified open-form description over multiple cinematographic dimensions. This task is challenging for two main reasons: the model must infer professional cinematographic concepts from subtle visual evidence, and it must generate captions that are both comprehensive and accurate. Accordingly, we propose CineCap, a framework that combines structured reasoning with spatio-temporal anchors and reinforcement learning with comprehensiveness, accuracy, and gated coverage rewards. The former grounds professional cinematographic descriptions in explicit visual evidence and organizes them into compact atomic reasoning for supervised fine-tuning, while the latter improves the balance between descriptive completeness and factual correctness. In addition, we construct CineCap Bench, a benchmark of 472 manually annotated video-caption pairs for systematic evaluation. Extensive experiments show that CineCap consistently outperforms strong proprietary and open-source baselines, establishing a new state of the art for cinematographic captioning. The code, model checkpoint, and benchmark are publicly available in https://github.com/Hectormxy/CineCap.git.

Chinese Translation

电影字幕生成旨在使用专业电影语言概念描述视频的拍摄方式，例如镜头运动、镜头大小、景深、构图和拍摄角度。这一能力对于细粒度视频理解和可控的电影质量视频生成至关重要，但在现有的多模态大语言模型中仍未得到充分探索。与基于问答的电影理解评估不同，电影字幕生成需要在多个电影维度上提供统一的开放式描述。该任务面临两个主要挑战：模型必须从微妙的视觉证据中推断出专业的电影概念，并且必须生成既全面又准确的字幕。因此，我们提出了CineCap，一个将结构化推理与时空锚点相结合的框架，并通过综合性、准确性和门控覆盖奖励进行强化学习。前者将专业的电影描述基于明确的视觉证据进行定位，并将其组织为紧凑的原子推理以进行监督微调，而后者则改善了描述完整性与事实正确性之间的平衡。此外，我们构建了CineCap Bench，一个包含472对手动注释视频-字幕对的基准，以便进行系统评估。大量实验表明，CineCap在强大的专有和开源基准上始终表现优异，确立了电影字幕生成的新最先进水平。代码、模型检查点和基准可在 https://github.com/Hectormxy/CineCap.git 上公开获取。

View on arXiv Download PDF AI Translation

cs.AI / 57 / 2606.24669

LaGO: Latent Action Guidance for Online Reinforcement Learning

LaGO：用于在线强化学习的潜在动作引导

Liu, Kuan-Yen, Huang, Ren-Jyun, Wu, Ti-Rong

Abstract

Large language models (LLMs) have shown strong potential for planning and sequential decision-making, but prior work often relies on using them as direct controllers, which requires precise action generation and can be unreliable in practice. This paper proposes Latent Action Guidance for Online Reinforcement Learning (LaGO), a framework that uses a pretrained LLM as a latent action prior to softly guide online policy optimization, rather than treating the LLM as an explicit planner or controller. Experiments on both a discrete-control benchmark, CLEVR-Robot, and a continuous-control benchmark, Meta-World, demonstrate that LaGO consistently improves both reward and success rate over Vanilla PPO. In particular, LaGO increases the average success rate from 15.1% to 27.2% on CLEVR-Robot and from 2.7% to 15.2% on Meta-World. Our analysis further shows that stronger pretrained LLMs provide more effective guidance, suggesting that LLM knowledge can improve planning and online decision-making.

Chinese Translation

大型语言模型（LLMs）在规划和顺序决策中展现出了强大的潜力，但以往的研究往往依赖于将其作为直接控制器，这需要精确的动作生成，并且在实践中可能不可靠。本文提出了在线强化学习的潜在动作引导（LaGO），这是一个利用预训练的LLM作为潜在动作先验，柔性引导在线策略优化的框架，而不是将LLM视为显式的规划者或控制器。在离散控制基准CLEVR-Robot和连续控制基准Meta-World上的实验表明，LaGO在奖励和成功率上均显著优于传统的PPO。具体而言，LaGO在CLEVR-Robot上的平均成功率从15.1%提高到27.2%，在Meta-World上的成功率从2.7%提高到15.2%。我们的分析进一步表明，强大的预训练LLM提供了更有效的引导，暗示LLM的知识可以改善规划和在线决策。

View on arXiv Download PDF AI Translation

cs.AI / 58 / 2606.24672

Cost-Optimal Decision Diagrams for Stochastic Boolean Function Evaluation

用于随机布尔函数评估的成本最优决策图

Zong, Xia, Lehtonen, Tuomo, Rintanen, Jussi

Abstract

In many decision-making scenarios, acquiring information incurs different costs. We consider the problem of constructing a deterministic evaluation strategy that minimizes the expected cost of evaluating a propositional formula under variable costs and a probability distribution over truth assignments. We present a branch-and-bound algorithm with variable-selection heuristics, pruning, and caching. To the best of our knowledge, it is the first practical exact algorithm for this level of generality. Experiments on random instances demonstrate scalability and quantify the efficiency-quality trade-off of a greedy beam-search variant. We additionally evaluate a structured heart-disease diagnosis instance. Finally, we prove that the problem is $\#P$-hard and contained in $\mathrm{PSPACE}$.

Chinese Translation

在许多决策场景中，获取信息会产生不同的成本。我们考虑构建一种确定性评估策略的问题，该策略在可变成本和真值分配的概率分布下，最小化评估命题公式的预期成本。我们提出了一种带有变量选择启发式、剪枝和缓存的分支限界算法。据我们所知，这是首个在此广泛性水平上实用的精确算法。对随机实例的实验表明了算法的可扩展性，并量化了贪婪束搜索变体的效率与质量之间的权衡。我们还评估了一个结构化的心脏病诊断实例。最后，我们证明了该问题是$ ext{#P}$-困难并包含在$ ext{PSPACE}$中。

View on arXiv Download PDF AI Translation

cs.AI / 59 / 2606.24722

Decentralised AI Training and Inference with BlockTrain

基于 BlockTrain 的去中心化 AI 训练与推理

Toth, Peter

Abstract

Frontier AI training is increasingly shaped by access to dense, centrally controlled accelerator clusters. This creates a structural advantage for hyperscalers and large centralized laboratories, and makes open or independent AI efforts depend on scarce capital, privileged infrastructure, and data-center geography. We present Spheroid BlockTrain, a decentralized training protocol in which a model is partitioned into independently trainable blocks, each optimized on a local objective derived from the same global target and composed at inference into one model. On byte-level WikiText, BlockTrain reaches cross entropy 1.359 (perplexity 3.89), within about 0.04 CE of a same-setup end-to-end Transformer reference, while each active worker trains only one block and avoids full-model optimizer state. A shared six-worker block training run reaches CE 1.385 by averaging same-block updates into one assembled model. HTTP/TCP transport experiments move real serialized checkpoints and updates, including a public-IP three-host run that improves CE from 5.580 to 1.811 while moving 15.22 GB. For inference, the current BlockTrain path uses one block-stack traversal per full output and serves over direct TCP across three public-network GPU hosts up to a 75.80B-parameter logical fp16 shape, outperforming a matched plain-autoregressive TCP pipeline baseline because it emits a full sequence per WAN pipeline traversal rather than one token per traversal.

Chinese Translation

前沿 AI 训练越来越受到密集、集中控制的加速器集群的影响。这为超大规模企业和大型集中实验室创造了结构性优势，使得开放或独立的 AI 努力依赖于稀缺的资本、特权基础设施和数据中心地理位置。我们提出了 Spheroid BlockTrain，这是一种去中心化的训练协议，其中模型被划分为可独立训练的块，每个块在基于相同全局目标的局部目标上进行优化，并在推理时组合成一个模型。在字节级的 WikiText 上，BlockTrain 达到了交叉熵 1.359（困惑度 3.89），与相同设置的端到端 Transformer 基准相差约 0.04 CE，而每个活跃工作者仅训练一个块，避免了全模型优化器状态的使用。一个共享的六工作者块训练运行通过将同块更新平均到一个组合模型中达到了 CE 1.385。HTTP/TCP 传输实验移动了真实的序列化检查点和更新，包括一个公共 IP 的三主机运行，将 CE 从 5.580 改进到 1.811，同时传输了 15.22 GB。对于推理，当前的 BlockTrain 路径在每个完整输出中使用一次块栈遍历，并通过直接 TCP 服务于三个公共网络 GPU 主机，支持高达 75.80B 参数的逻辑 fp16 形状，超越了匹配的普通自回归 TCP 管道基线，因为它在每次广域网管道遍历中发出一个完整序列，而不是每次遍历发出一个标记。

View on arXiv Download PDF AI Translation

cs.AI / 60 / 2606.24747

Scaling Laws for Task-Specific LLM Distillation

任务特定大语言模型蒸馏的规模法则

Ghita, Lavinia, Desai, Dhruv, Boier, Ioana

Abstract

Large Language Models (LLMs) achieve strong performance across a growing range of domains, yet their scale poses deployment challenges in applications where latency and cost constraints are critical. This paper derives empirical scaling laws for domain-specific LLM compression, quantifying how in-domain and general knowledge performance scale with dataset size, compression ratio, supervision format, and iterative pruning schedule. Using quantitative finance as our application domain, we compare logit-based and LoRA-based distillation under iterative structural pruning, introducing a blended chain-of-thought supervision loss that stabilizes KL-divergence distillation over reasoning traces. In-domain task quality degrades predictably under compression while general-knowledge benchmarks collapse well before the same point; supervision format is the key driver of this tradeoff, with chain-of-thought supervision actively recovering general knowledge that pruning erases. We release the headline dataset FinHeadlineMix, scaling law results, and practical recommendations to provide a reusable framework for domain-specific compression decisions.

Chinese Translation

大型语言模型（LLMs）在越来越多的领域中表现出强大的性能，但它们的规模在延迟和成本约束至关重要的应用中带来了部署挑战。本文推导了领域特定LLM压缩的经验规模法则，量化了领域内和一般知识性能如何随着数据集规模、压缩比、监督格式和迭代剪枝计划的变化而变化。以定量金融作为应用领域，我们比较了基于logit和基于LoRA的蒸馏在迭代结构剪枝下的表现，引入了一种混合的思维链监督损失，稳定了推理轨迹上的KL散度蒸馏。在压缩下，领域内任务质量可预测地下降，而一般知识基准在达到相同点之前就崩溃；监督格式是这一权衡的关键驱动因素，思维链监督积极恢复了剪枝所抹去的一般知识。我们发布了主要数据集FinHeadlineMix、规模法则结果以及实用建议，以提供一个可重用的领域特定压缩决策框架。

View on arXiv Download PDF AI Translation

cs.AI / 61 / 2606.24752

Can Scale Save Us From Plasticity Loss in Large Language Models?

规模能拯救我们免于大型语言模型的可塑性丧失吗？

Hernandez-Garcia, J. Fernando, Figliolia, Tomás, Millidge, Beren

Abstract

The loss of plasticity - the ability of a network to learn new information after having already learned older information - is a fundamental challenge in creating artificial neural networks capable of continual learning. Although this phenomenon has been known for decades, it has mostly been studied in older, relatively small architectures and rarely in natural-language domains. To determine whether loss of plasticity remains a problem in the modern transformer-based LLM paradigm, we study plasticity loss in GPT-style Transformer models trained on a multilingual continual learning problem. Consistent with prior work, we find evidence of plasticity loss across models ranging from 5M to 314M non-embedding parameters, as measured by deterioration on a held-out Vietnamese probing task. We further find that the onset of plasticity loss follows a predictable scaling law, growing sublinearly with model size. These results suggest that larger models may delay the measurable effects of plasticity loss, but that increasing parameter count alone is likely to be insufficient to completely prevent it. We also find evidence of plasticity loss under stationary multilingual training, challenging the view that the phenomenon is exclusive to continual learning with abrupt task changes. Overall, our results suggest that even large Transformer language models trained on natural-language will eventually lose the ability to efficiently adapt to new data after sufficiently long training, in both continual and stationary settings.

Chinese Translation

可塑性丧失——网络在学习了旧信息后学习新信息的能力——是创建能够持续学习的人工神经网络的一项基本挑战。尽管这一现象已被认识数十年，但主要是在较旧、相对较小的架构中进行研究，且在自然语言领域的研究较少。为了确定在现代基于变换器的大型语言模型（LLM）范式中可塑性丧失是否仍然是一个问题，我们研究了在多语言持续学习问题上训练的GPT风格变换器模型中的可塑性丧失。与之前的研究一致，我们发现，在5M到314M非嵌入参数范围内的模型中，均存在可塑性丧失的证据，这通过在保留的越南语探测任务上的性能下降来衡量。我们进一步发现，可塑性丧失的发生遵循可预测的规模法则，随着模型规模的增加而以次线性方式增长。这些结果表明，较大的模型可能会延迟可塑性丧失的可测量影响，但单纯增加参数数量可能不足以完全防止可塑性丧失。我们还发现，在静态多语言训练下也存在可塑性丧失的证据，这挑战了可塑性丧失现象仅限于具有突发任务变化的持续学习的观点。总体而言，我们的结果表明，即使是训练于自然语言的大型变换器语言模型，经过足够长时间的训练后，也会最终失去有效适应新数据的能力，无论是在持续学习还是静态设置中。

View on arXiv Download PDF AI Translation

cs.AI / 62 / 2606.24780

BluTrain: A C++/CUDA Framework for AI Systems

BluTrain：一个用于人工智能系统的C++/CUDA框架

Charan, Adhitya, Suresh, Adwaid, Kumar, Anuj, A, Aparna, K, Dhanakumar, S, Dharun M, G, Dinesh, K, Goutham Kumar Reddy, M, Harshini V, D, Jenifa, A, Jona Delcy C, S, Kathirvel, Rao, Killi Uma Maheswara, M, Kiruthik Kanna, Sai, Kurra Vishnu, K, Madhumithaa G, Kumar V, Navin, Golla, Ram Charan, T, Revathi, R, Rishikkanth, M V, Sanjay Krishna, Vendra, Surendra

Abstract

Progress in deep learning is, at scale, more a matter of systems engineering than of modelling: the behaviour of a model in training (its throughput, its memory footprint, and the numerical fidelity of the result) is determined less by the architecture itself than by how that architecture is expressed on the hardware. To achieve absolute control over this hardware expression while abstracting away systems complexity to make modelling seamless and eliminating the need for repetitive orchestration logic, BluTrain was architected from first principles as a robust, lightweight, and architecture-general training framework in standard C++ and the core CUDA programming model. Every layer is implemented natively: a typed tensor module with reverse-mode autograd, a linear-algebra library, a caching allocator, a multi-mode distributed-execution module, and an MLIR-based deep-learning compiler. In formal evaluations training a 124M-parameter GPT-2 baseline in FP32 on an 8-GPU 6000 Ada system, BluTrain outperforms industry-standard baselines in both throughput (sustaining an average of 407K tokens/s versus PyTorch's 395K tokens/s) and memory efficiency (achieving up to a 22% footprint reduction), while strictly preserving numerical fidelity and converging to a marginally lower final validation loss. With every layer explicitly open to native tuning, the performance ceiling is the framework's own to raise.

Chinese Translation

深度学习的进展在规模上更多地是系统工程的问题，而非建模的问题：模型在训练中的表现（其吞吐量、内存占用和结果的数值保真度）更多地取决于该架构在硬件上的表达方式，而非架构本身。为了在抽象系统复杂性的同时实现对硬件表达的绝对控制，使建模过程无缝，并消除对重复编排逻辑的需求，BluTrain从基本原理出发，构建了一个稳健、轻量且通用的训练框架，采用标准C++和核心CUDA编程模型。每一层都以原生方式实现：一个带有反向模式自动求导的类型化张量模块、一个线性代数库、一个缓存分配器、一个多模式分布式执行模块，以及一个基于MLIR的深度学习编译器。在正式评估中，在一个8-GPU 6000 Ada系统上以FP32训练124M参数的GPT-2基线时，BluTrain在吞吐量（平均维持407K tokens/s，相较于PyTorch的395K tokens/s）和内存效率（实现高达22%的占用减少）方面均优于行业标准基线，同时严格保持数值保真度，并收敛到略低的最终验证损失。由于每一层均可进行原生调优，框架的性能上限将由其自身提升。

View on arXiv Download PDF AI Translation

cs.AI / 63 / 2606.24781

Assessing Distribution Shift in Human Activity Recognition for Domain Generalization

评估人类活动识别中的分布转移以实现领域泛化

Adaimi, Rebecca, Thomaz, Edison

Abstract

While the field of Human Activity Recognition (HAR) continues to draw interest from researchers and advance in important ways, some key challenges remain. One of the most difficult aspects of building HAR models that show good performance in real-world settings is dealing with data diversity from device and sensor heterogeneity, and contextual changes that are intrinsic to real-world applications. While data diversity in HAR has been well-acknowledged in the literature, there remains a gap in understanding the effect of various types of distribution shifts on HAR models and the domain generalization problem that arises. Towards that end, this paper systematically evaluates 4 different types of distribution shifts, including variations in device type, sensor placement, sampling rate, and user behavior. Quantifying their effects, we illustrate that diversity shifts predominantly define all types of shifts, indicating the existence of unique features that are not shared across different domains. We then introduce a uniform HAR-based distribution shift benchmarks and conduct a comprehensive evaluation of up to 28 domain generalization methods. Our analysis exposes the limitations of current domain generalization algorithms in achieving model generalizability, marginally outperforming the empirical risk minimization baseline. This work represents the first systematic exploration of domain generalization and adaptation concerning specific distribution shifts in sensor-based HAR, offering an open-source benchmark platform and datasets to spur further research.

Chinese Translation

尽管人类活动识别（HAR）领域持续吸引研究者的关注并在重要方面取得进展，但仍然存在一些关键挑战。构建在现实世界环境中表现良好的HAR模型的最困难方面之一是处理来自设备和传感器异质性以及现实应用中固有的上下文变化所带来的数据多样性。尽管文献中对HAR中的数据多样性已有充分认识，但对各种类型的分布转移对HAR模型的影响以及由此产生的领域泛化问题的理解仍存在差距。为此，本文系统地评估了4种不同类型的分布转移，包括设备类型的变化、传感器位置、采样率和用户行为的变化。通过量化它们的影响，我们表明多样性转移主要定义了所有类型的转移，指示出不同领域之间存在独特的特征。随后，我们引入了一种统一的基于HAR的分布转移基准，并对多达28种领域泛化方法进行了全面评估。我们的分析揭示了当前领域泛化算法在实现模型泛化能力方面的局限性，仅略微优于经验风险最小化基线。这项工作代表了对基于传感器的HAR中特定分布转移的领域泛化和适应的首次系统探索，并提供了一个开源基准平台和数据集，以促进进一步研究。

View on arXiv Download PDF AI Translation

cs.AI / 64 / 2606.24824

Solving Inverse Problems of Chaotic Systems with Bidirectional Conditional Flow Matching

利用双向条件流匹配解决混沌系统的逆问题

Hu, Peiyan, Zhang, Jian, Pan, Jiashu, Feng, Ruiqi, Zhang, Tao, Ma, Zhi-Ming, Ting, Yuan-Sen, Li, Gongjie, Wu, Tailin

Abstract

Modeling chaotic systems is crucial yet challenging. Inverse problems in chaotic dynamics, namely inferring initial conditions from final states, remain largely unsolved because of ill-posedness, non-uniqueness, instability, and potentially chaotic time-reverse dynamics. We address this open problem with Bidirectional Conditional Flow Matching (Bi-CFM), which learns bidirectional mappings between distributions of initial and final states to capture the stochasticity of chaotic evolution and mitigate exponential error accumulation over time. Furthermore, for systems with conservation laws, we extend it to Conservation-constrained Bi-CFM (CBi-CFM). Across the classic Lorenz, Circuit, and high-dimensional Lorenz 96 systems, Bi-CFM improves five distribution-level metrics over baselines while achieving a speedup of more than two orders of magnitude. In the three-body planet-planet scattering problem in planetary dynamics, CBi-CFM better respects conservation laws, with conservation errors comparable to those of the ground truth. Finally, on real observations of globular clusters, collisional million-body systems shaped by $\sim 10^{10}$ years (10 Gyr) of evolution, our method represents an advance in accuracy, establishing a scalable route to solving inverse problems of long-timescale real-world chaotic dynamics.

Chinese Translation

建模混沌系统至关重要但具有挑战性。混沌动力学中的逆问题，即从最终状态推断初始条件，因其病态性、非唯一性、不稳定性以及潜在的混沌时间反转动态而大多未得到解决。我们通过双向条件流匹配（Bidirectional Conditional Flow Matching, Bi-CFM）来解决这一开放问题，该方法学习初始状态和最终状态分布之间的双向映射，以捕捉混沌演化的随机性，并减轻随时间推移的指数误差累积。此外，对于具有守恒定律的系统，我们将其扩展为守恒约束双向条件流匹配（Conservation-constrained Bi-CFM, CBi-CFM）。在经典的洛伦兹（Lorenz）、电路（Circuit）和高维洛伦兹96系统中，Bi-CFM在五个分布级别指标上优于基线，同时实现了超过两个数量级的加速。在行星动力学中的三体行星间散射问题中，CBi-CFM更好地遵循守恒定律，其守恒误差与真实值相当。最后，在经过约10^{10}年（10 Gyr）演化形成的球状星团的真实观测数据上，我们的方法在准确性上取得了进展，为解决长时间尺度的真实世界混沌动力学的逆问题建立了一条可扩展的路径。

View on arXiv Download PDF AI Translation

cs.AI / 65 / 2606.24834

Accuracy and Satisfaction in Multi-Turn LLM Dialogues for NFR Assessment

多轮LLM对话在非功能性需求(NFR)评估中的准确性与满意度

Fatideh, Ali Pourghasemi, Baldwin, Wilder, Dhakal, Maria, McMillan, Collin, Ghanavati, Sepideh

Abstract

LLM-based dialogue assistants have become mainstream tools for software developers, yet current evaluation benchmarks focus exclusively on functional correctness. This leaves a critical gap in assessing the quality and accuracy of these conversations when handling Non-Functional Requirements (NFRs), which are inherently vague, context-dependent, and involve many parts of a program. Evaluating how well these systems support collaborative reasoning about NFRs requires methods that go beyond single-turn accuracy to capture both the correctness of the system's outputs and the quality of the multi-turn interaction. In this paper, we investigate the accuracy and quality of multi-turn conversations between developers and an LLM-based agent in the domain of Health Insurance Portability and Accountability Act (HIPAA) regulatory compliance. We hired 49 programmers to interact with GitHub Copilot to assess 148 HIPAA-derived NFRs against the iTrust codebase, a system designed to comply with HIPAA regulations, across three dimensions: requirement satisfaction level, reasoning, and code localization. We find that developers tend to agree with LLM assessments, but accuracy against expert ground truth is low. We model user satisfaction and find that longer system responses and more information-providing turns negatively affect user satisfaction, whereas proactive interactions positively affect it. Our findings provide insights for designing LLM-based dialogue systems that support NFR assessment.

Chinese Translation

基于LLM的对话助手已成为软件开发者的主流工具，但当前的评估基准仅关注功能正确性。这在评估这些对话在处理非功能性需求（NFR）时的质量和准确性方面留下了一个关键空白，因为NFR本质上是模糊的、依赖于上下文的，并且涉及程序的多个部分。评估这些系统在支持关于NFR的协作推理方面的表现需要超越单轮准确性的评估方法，以捕捉系统输出的正确性和多轮交互的质量。在本文中，我们研究了开发者与基于LLM的代理在健康保险可携带性和责任法案（HIPAA）合规领域之间的多轮对话的准确性和质量。我们聘请了49名程序员与GitHub Copilot进行互动，以评估148个基于HIPAA的NFR在iTrust代码库（一个旨在遵守HIPAA法规的系统）中的表现，评估维度包括需求满足程度、推理和代码定位。我们发现开发者倾向于同意LLM的评估，但与专家的真实标准相比，准确性较低。我们对用户满意度进行了建模，发现系统响应时间较长和提供更多信息的轮次对用户满意度产生负面影响，而主动交互则对其产生积极影响。我们的研究结果为设计支持NFR评估的基于LLM的对话系统提供了见解。

View on arXiv Download PDF AI Translation

cs.AI / 66 / 2606.24839

Grading the Grader: Lessons from Evaluating an Agentic Data Analysis System

评估评分者：从评估一个自主数据分析系统中获得的经验教训

Zheng, Tian, Hsu, Kai-Tai

Abstract

Agentic data analysis systems produce rich outputs, including code, numerical results, and verbal diagnostics. This makes them more challenging to evaluate than single-turn LLM responses. It is therefore necessary to distinguish genuine disagreement between an agent's output and a ground-truth answer from grading artifacts. We investigate how reliably automated graders assess such a system and what strategies improve grading quality by applying LAMBDA, a multi-agent data-analysis system, on 153 numerical QRData tasks from DSGym. We develop and evaluate a three-layer human-AI grading cascade: strict regex matching, LLM-based lenient grading, and snippet-based human inspection, which combines non-GenAI and GenAI strategies with different failure profiles. Both automated graders achieve 100% observed precision (0/70 false positives). The lenient grader's recall is 97% against human labels. A keyword-anchored extraction pipeline raises the strict grader's recall by 60 percentage points over a last-number heuristic; the lenient grader is architecturally parser-independent. An iterative nudge mechanism raises grading run success from 36% to 97% and lenient-pass rates from 16% to 46%; comparing nudging with and without original-question re-injection shows that re-injection offers no benefit, confirming the nudge as an answer template cue. We further observe in this case study that variable type is the task metadata field most consistently associated with grading pipeline dynamics and observed outcome grades.

Chinese Translation

自主数据分析系统生成丰富的输出，包括代码、数值结果和口头诊断。这使得它们的评估比单轮大型语言模型（LLM）响应更具挑战性。因此，有必要区分代理输出与真实答案之间的真正分歧和评分伪影。我们研究了自动评分者如何可靠地评估这样的系统，以及哪些策略可以通过在153个来自DSGym的数值QRData任务上应用LAMBDA（一个多代理数据分析系统）来提高评分质量。我们开发并评估了一个三层人机评分级联：严格的正则表达式匹配、基于LLM的宽松评分和基于代码片段的人类检查，这结合了不同失败特征的非生成式人工智能（non-GenAI）和生成式人工智能（GenAI）策略。两个自动评分者均实现了100%的观察精度（0/70假阳性）。宽松评分者的召回率为97%，与人类标签相比。一个以关键词为锚的提取管道使严格评分者的召回率比最后数字启发式提高了60个百分点；宽松评分者在架构上独立于解析器。一个迭代的推动机制将评分运行成功率从36%提高到97%，宽松通过率从16%提高到46%；比较有无原始问题重新注入的推动显示，重新注入没有带来好处，确认推动作为答案模板提示。在这个案例研究中，我们进一步观察到，变量类型是与评分管道动态和观察到的结果等级最一致相关的任务元数据字段。

View on arXiv Download PDF AI Translation

cs.AI / 67 / 2606.24842

World Models in Pieces: Structural Certification for General Agents

碎片化的世界模型：通用智能体的结构认证

Lu, Yikai, Wu, Yifei, Lu, Xinyu, Li, Tongxin

Abstract

In the big-world regime, agents cannot be universally capable and their ability is inevitably specialized across a world model in pieces. Consequently, standard uniform guarantees fail to distinguish between the understanding of critical bottlenecks and irrelevant failures. We first formalize this limitation by proving that general agents are not universal, rendering standard worst-case analysis uninformative. To overcome this, we introduce structural certification, a transition-local framework that maps bounded goal-conditioned performance to entry-wise guarantees on the agent's internal world model. Our main contribution is constructive. We provide algorithms that filter specific transitions using deep compositional goals and prove that a general agent on these goals has a structural world model with a $\mathcal{O}(1/n) + \mathcal{O}(\delta)$ error bound. Conversely, this bound is tight in the small-$\delta$ regime, whose existence is explicitly guaranteed by our certification. These results enable the certifiable deployment of general agents by localizing the specific transitions where long-horizon planning is reliable.

Chinese Translation

在大世界范式中，智能体无法具备普遍能力，其能力不可避免地在碎片化的世界模型中专门化。因此，标准的统一保证无法区分对关键瓶颈的理解与无关失败之间的差异。我们首先通过证明通用智能体并非普遍存在来形式化这一限制，从而使得标准的最坏情况分析变得无信息。为了解决这个问题，我们引入了结构认证，这是一种过渡局部框架，将有界的目标条件性能映射到智能体内部世界模型的逐项保证上。我们的主要贡献是建设性的。我们提供了使用深度组合目标过滤特定过渡的算法，并证明在这些目标下，通用智能体具有一个结构化的世界模型，其误差界限为 $ ext{O}(1/n) + ext{O}( ext{δ})$。相反，在小-$ ext{δ}$ 范围内，这个界限是紧的，其存在性由我们的认证明确保证。这些结果通过局部化长时间规划可靠的特定过渡，使通用智能体的可认证部署成为可能。

View on arXiv Download PDF AI Translation

cs.AI / 68 / 2606.24855

OpenThoughts-Agent: Data Recipes for Agentic Models

OpenThoughts-Agent：用于代理模型的数据配方

Raoof, Negin, Zhuang, Richard, Nezhurina, Marianna, Guha, Etash, Tejaswi, Atula, Marten, Ryan, Ruan, Charlie F., Griggs, Tyler, Shaw, Alexander Glenn, Bansal, Hritik, Buchanan, E. Kelly, Gazizov, Artem, Heckel, Reinhard, Hegde, Chinmay, Jajee, Sankalp, Khazi, Daanish, Koukoumidis, Emmanouil, Li, Xiangyi, Liu, Hange, Natarajan, Shlok, Raj, Harsh, Roberts, Nicholas, Shen, Ethan, Singhi, Nishad, Siu, Michael, Suvarna, Ashima, Xing, Hanwen, Yubeaton, Patrick, Zhang, Robert, Chen, Leon Liangyu, Chen, Xiaokun, Dillmann, Steven, Gabriel, Saadia, Jiang, Xunyi, Kashyap, Anurag, Li, Boxuan, Park, Yein, Pham, Minh, Sanghavi, Sujay, Shi, Lin, Sun, Ke, Wang, Yixin, Xu, Zhiwei, Zhang, Erica, Zhao, Siyan, Zhao, Wanjia, Jitsev, Jenia, Dimakis, Alex, Feuer, Benjamin, Schmidt, Ludwig

Abstract

Agentic language models dramatically expand the applications of AI yet little is publicly known about how to curate training data for broadly capable agents. Existing open efforts such as SWE-Smith, SERA, and Nemotron-Terminal typically target a single benchmark, leaving open the question of how to train models that generalize across diverse agentic tasks. The OpenThoughts-Agent (OT-Agent) project addresses this gap with a fully open data curation pipeline for training agentic models. We conduct more than 100 controlled ablation experiments to systematically investigate each stage of the pipeline, yielding insights on the importance of task sources and diversity. We then assemble a training set of 100K examples from our pipeline and fine-tune Qwen3-32B on this dataset, which yields an average accuracy of 44.8% across seven agentic benchmarks and a 3.9 percentage point improvement over the strongest existing open data agentic model (Nemotron-Terminal-32B, 40.9%). Moreover, our training data exhibits strong scaling properties, outperforming alternative open datasets at every training set size in compute-controlled comparisons. We publicly release our training sets, data pipeline, experimental data, and models at openthoughts.ai to support future open research on agentic model training.

Chinese Translation

代理语言模型显著扩展了人工智能的应用，但关于如何为广泛能力的代理策划训练数据的公开信息仍然很少。现有的开放努力，如SWE-Smith、SERA和Nemotron-Terminal，通常针对单一基准，留下了如何训练能够在多样化代理任务中泛化的模型的问题。OpenThoughts-Agent（OT-Agent）项目通过一个完全开放的数据策划流程来填补这一空白，以训练代理模型。我们进行了100多次受控消融实验，以系统性地研究流程的每个阶段，从而获得关于任务来源和多样性重要性的见解。随后，我们从我们的流程中组装了一个包含10万例的训练集，并在该数据集上对Qwen3-32B进行了微调，结果在七个代理基准上平均准确率达到44.8%，比现有最强的开放数据代理模型（Nemotron-Terminal-32B，40.9%）提高了3.9个百分点。此外，我们的训练数据展现出强大的扩展特性，在每个训练集规模的计算控制比较中均优于其他开放数据集。我们在openthoughts.ai上公开发布我们的训练集、数据流程、实验数据和模型，以支持未来关于代理模型训练的开放研究。

View on arXiv Download PDF AI Translation

计算语言学 (Computation and Language)

cs.CL / 1 / 2606.23693

EXPO-SQL: Execution-based Clause-level Policy Optimization for Text-to-SQL

EXPO-SQL：基于执行的子句级策略优化用于文本到SQL

Lee, Jaehoon, Na, CheolWon, Bae, Suyoung, Lee, Jin-Seop, Lee, Jihyung, Choi, YunSeok, Lee, Jee-Hyong

Abstract

Text-to-SQL enables users to query databases using natural language by generating executable SQL queries. Recent methods have increasingly adopted Large Language Models based reinforcement learning (RL) to leverage execution feedback for training. However, existing RL methods assign uniform query-level rewards to all clauses in a SQL query, treating correct and incorrect clauses equally. This coarse-grained reward design leads to insufficient learning signals for correct SQL generation. To address this issue, we propose EXPO-SQL (EXecution-based clause-level Policy Optimization for Text-to-SQL) which provides fine-grained supervision through clause-level rewards. To assign clause-level rewards, our method identifies erroneous clauses by analyzing execution results, including error messages and clause-wise incremental execution. Experiments on widely-used Text-to-SQL benchmarks demonstrate that EXPO-SQL significantly outperforms existing supervised fine-tuning, prompting, and RL-based methods through fine-grained clause-level learning. Our code is available at https://github. com/jhn25/EXPO-SQL.

Chinese Translation

文本到SQL使用户能够通过生成可执行的SQL查询使用自然语言查询数据库。近年来，越来越多的方法采用基于大型语言模型的强化学习（RL）来利用执行反馈进行训练。然而，现有的RL方法对SQL查询中的所有子句分配统一的查询级奖励，将正确和错误的子句视为相同。这种粗粒度的奖励设计导致了对正确SQL生成的学习信号不足。为了解决这个问题，我们提出了EXPO-SQL（基于执行的子句级策略优化用于文本到SQL），通过子句级奖励提供细粒度的监督。为了分配子句级奖励，我们的方法通过分析执行结果（包括错误信息和子句级增量执行）来识别错误子句。在广泛使用的文本到SQL基准上的实验表明，EXPO-SQL通过细粒度的子句级学习显著优于现有的监督微调、提示和基于RL的方法。我们的代码可在 https://github.com/jhn25/EXPO-SQL 获取。

View on arXiv Download PDF AI Translation

cs.CL / 2 / 2606.23694

ModTGCN: Modularity-aware Graph Neural Networks for Text Classification

ModTGCN：面向模块性的图神经网络用于文本分类

Misra, Rajarshi, Sharma, Aditya, Agarwal, Vinti, Aggrawal, Hari Om

Abstract

Graph-based text classification models typically rely on local neighborhood aggregation and overlook global community structure, despite semantic document graphs exhibiting strong class-consistent clustering. Ignoring this can blur class boundaries and lead to over-smoothing. We propose ModTGCN, a modularity-aware graph neural network for text classification that jointly optimizes cross-entropy and a modularity-based auxiliary objective to promote class-coherent document communities while preserving discriminative representations. The modularity term is computed on a document-document similarity graph derived from transformer embeddings (pretrained or fine-tuned). To improve scalability, we decouple the original heterogeneous TextGCN graph into separate document-word and word-word components, achieving 2x-10x faster training. We further study graph construction strategies, label-aware edge reweighting, and supervision choices for modularity optimization. Experiments on five benchmarks show consistent gains, with larger improvements on complex, low homophily datasets such as Ohsumed and 20NG.

Chinese Translation

基于图的文本分类模型通常依赖于局部邻域聚合，而忽视了全球社区结构，尽管语义文档图表现出强烈的类别一致性聚类。忽视这一点可能会模糊类别边界并导致过度平滑。我们提出了ModTGCN，一种面向模块性的图神经网络，用于文本分类，旨在联合优化交叉熵和基于模块性的辅助目标，以促进类别一致的文档社区，同时保留区分性表示。模块性项是在由变换器嵌入（预训练或微调）派生的文档-文档相似度图上计算的。为了提高可扩展性，我们将原始异构的TextGCN图解耦为单独的文档-词和词-词组件，实现了2倍到10倍的训练速度提升。我们进一步研究了图构建策略、标签感知边缘重加权和模块性优化的监督选择。在五个基准测试上的实验显示出一致的提升，尤其是在复杂的、低同质性的数据集如Ohsumed和20NG上，改进更为显著。

View on arXiv Download PDF AI Translation

cs.CL / 3 / 2606.23695

Quantifying Prior Dominance in RAG Systems

量化RAG系统中的先验主导性

Or, Barak

Abstract

Retrieval-Augmented Generation (RAG) grounds Large Language Models in external knowledge, yet current evaluations rely on discrete heuristics that suffer from ''epistemic blindness'' - failing to distinguish genuine contextual information extraction from parametric memory recall. To address this, we introduce the Normalized Context Utilization (NCU) metric, leveraging continuous token log-probabilities across zero-shot, oracle, and adversarial conditions to strictly quantify contextual information gain. Evaluating architectures ranging from 1.5B to 72B parameters alongside a proprietary commercial API reveals that for strict factual extraction (without Chain-of-Thought reasoning), traditional scaling laws exhibit extreme diminishing returns: highly efficient Small Language Models (SLMs) match or outperform high-capacity architectures. Furthermore, we demonstrate that ``Prior Dominance'' correlates with model scale and proprietary alignments. The evaluated commercial API not only overrode explicit external evidence in nearly half of adversarial conflicts, but also frequently suffered from systemic confidence collapse (Negative Transfer) when its parametric priors were contradicted. Our findings highlight the structural epistemic advantage and superior contextual adherence of SLMs in strict extraction workflows.

Chinese Translation

检索增强生成（Retrieval-Augmented Generation, RAG）将大型语言模型与外部知识相结合，但当前的评估依赖于离散启发式方法，这些方法存在“认识盲点”（epistemic blindness）——无法区分真正的上下文信息提取与参数记忆回忆。为了解决这一问题，我们引入了归一化上下文利用率（Normalized Context Utilization, NCU）指标，利用零-shot、oracle和对抗条件下的连续令牌对数概率严格量化上下文信息增益。对从15亿到720亿参数的架构进行评估，以及对一个专有商业API的分析显示，对于严格的事实提取（不使用思维链推理），传统的规模法则表现出极端的收益递减：高效的小型语言模型（Small Language Models, SLMs）与高容量架构相匹配或超越。此外，我们证明了“先验主导性”（Prior Dominance）与模型规模和专有对齐之间的相关性。评估的商业API不仅在近一半的对抗冲突中覆盖了明确的外部证据，而且在其参数先验被反驳时，常常遭遇系统性信心崩溃（负迁移，Negative Transfer）。我们的研究结果突显了SLMs在严格提取工作流中的结构性认识优势和卓越的上下文遵循能力。

View on arXiv Download PDF AI Translation

cs.CL / 4 / 2606.23700

Self-Recognition Finetuning can Prevent and Reverse Emergent Misalignment

自我识别微调可以防止和逆转新兴的失调

Tagade, Arush, Zhou, Shaoheng, Wen, Jiaxin, Feng, Shi

Abstract

Emergent misalignment (EM) has been linked to the activation of misaligned persona vectors and evil character traits, suggesting that EM operates through disruption of the model's aligned character rather than direct learning of harmful content. Motivated by this connection, we study self-generated text recognition (SGTR) finetuning as a character-targeted intervention that is distinct from existing in-training defenses. We conduct two-stage finetuning experiments across three models (GPT-4.1, Qwen2.5-32B-Instruct, Seed-OSS-36B-Instruct) and multiple EM datasets to compare SGTR finetuning against benign finetuning baselines (correct domain-specific data, general knowledge, and word counting) to find it an effective defense in both reversal and prevention settings. We find that all interventions produce comparable EM reversal, but only when restoring capabilities that EM had degraded. For prevention, only SGTR finetuning consistently reduces misalignment without exacerbating any individual metric, suggesting that character fortification specifically drives prevention. We provide further evidence for EM's relation to the LLM's default character by showing that EM finetuning induces diversity into the LLM's identity self-reports, artificially corrupting self-recognition exacerbates misalignment caused by EM finetuning, and that removing the model's identity-bearing system prompt substantially reduces the effect of EM finetuning. Together, these findings reframe EM not as the adoption of a coherent misaligned persona but as the destabilization of aligned character.

Chinese Translation

新兴失调（EM）与失调的人格向量和邪恶性格特征的激活有关，这表明EM是通过干扰模型的对齐特征而非直接学习有害内容来运作的。基于这一联系，我们研究了自生成文本识别（SGTR）微调作为一种针对性的人格干预，这与现有的训练防御措施有所不同。我们在三个模型（GPT-4.1、Qwen2.5-32B-Instruct、Seed-OSS-36B-Instruct）和多个EM数据集上进行两阶段微调实验，以比较SGTR微调与良性微调基线（正确的领域特定数据、一般知识和字数统计），发现其在逆转和预防设置中都是有效的防御措施。我们发现所有干预措施在EM逆转方面产生了可比的效果，但仅在恢复EM所削弱的能力时有效。对于预防，只有SGTR微调始终减少失调，而不加剧任何单一指标，这表明人格强化特别推动了预防。我们进一步提供了EM与大型语言模型（LLM）默认人格之间关系的证据，显示EM微调会导致LLM身份自我报告的多样性，人工破坏自我识别会加剧EM微调造成的失调，而移除模型的身份承载系统提示会显著减少EM微调的影响。综合来看，这些发现将EM重新框定为对齐人格的失稳，而非一致的失调人格的采纳。

View on arXiv Download PDF AI Translation

cs.CL / 5 / 2606.23701

Evaluating LLM Usage for Efficient and Explainable Numerical and Classified Implicit Sentiment Analysis of Product Desirability

评估大型语言模型在高效且可解释的产品吸引力隐性情感分析中的应用

Weitl-Harms, Sherri, Hastings, John

Abstract

Qualitative product feedback can reveal nuanced user experiences, but its implicit sentiment is difficult to measure. This paper presents a scalable and interpretable framework that uses large language models (LLMs) to quantify product desirability from such data. Using two Product Desirability Toolkit (PDT) datasets from ZORQ and CARMA comprising 106 respondent term groupings with gold-standard human annotation, zero-shot continuous numerical sentiment scoring and categorical sentiment classification are evaluated without relying on explicit review scores. Across the datasets, LLMs generated numerical sentiment scores directly from qualitative responses and closely matched expert labels, achieving Pearson correlations up to 0.97 and classification accuracy up to 94%. LLMs maintained robustness even when handling data presented in multiple forms and consistently expressed high confidence. In contrast, lexicon-based and transformer baselines did not produce statistically significant results. Among the models tested, GPT-4o-mini achieved performance comparable to larger models at 94% lower cost, supporting scalable deployment. The framework also incorporates model confidence ratings and human-readable rationale explanations (xAI), improving interpretability, transparency, and trust while supporting practical use in product satisfaction assessment. In general, using the PDT tool as a survey method along with a cost efficient LLM for sentiment analysis has the potential to provide for product evaluation with results that are rich in terms of sentiment scores (both numerical and classified sentiment) and in terms of the high-level user impressions of the product that can be used to identify ideas for product development and improvement, as well as marketing ideas for target audiences.

Chinese Translation

定性产品反馈能够揭示细致的用户体验，但其隐性情感难以量化。本文提出了一种可扩展且可解释的框架，利用大型语言模型（LLMs）从此类数据中量化产品吸引力。使用来自ZORQ和CARMA的两个产品吸引力工具包（PDT）数据集，这些数据集包含106个受访者术语分组，并经过金标准人类标注，评估了零-shot连续数值情感评分和类别情感分类，而无需依赖显式评论分数。在这些数据集中，LLMs直接从定性响应中生成数值情感评分，并与专家标签紧密匹配，皮尔逊相关系数高达0.97，分类准确率高达94%。LLMs在处理多种形式的数据时保持了稳健性，并始终表现出高信心。相比之下，基于词典和变换器的基线模型未能产生统计显著的结果。在测试的模型中，GPT-4o-mini以94%的成本实现了与更大模型相当的性能，支持可扩展部署。该框架还结合了模型置信度评分和人类可读的理由解释（xAI），提高了可解释性、透明度和信任度，同时支持在产品满意度评估中的实际应用。总体而言，将PDT工具作为调查方法，并结合高性价比的LLM进行情感分析，有潜力为产品评估提供丰富的情感评分（包括数值和分类情感）以及高层次的用户印象，这些信息可用于识别产品开发和改进的想法，以及针对目标受众的营销创意。

View on arXiv Download PDF AI Translation

cs.CL / 6 / 2606.23881

Ground Then Rank: Revisiting Knowledge-Based VQA with Training-Free Entity Identification

先定位再排序：重新审视基于知识的视觉问答与无训练实体识别

Ma, Qian, Wu, Qiong, Zhou, Zhengyi, Ma, Yao

Abstract

Knowledge-Based Visual Question Answering (KB-VQA) requires grounding visual queries to external knowledge beyond directly observable content in images. While recent multi modal large language models (MLLMs) show strong perceptual abilities, they struggle on KB-VQA tasks requiring groundings from both fine-grained entity and evidence levels. Most existing multi-modal retrieval augmented generation (MM-RAG) methods tightly couple entity discrimination and section-level evidence ranking into a single re-ranking stage, leading to high cost and limited generalization. In this work, we revisit existing MM-RAG solutions from a workflow perspective and argue both entity-level and fact-level groundings are key bottlenecks. We observe that although MLLMs often fail under open-ended entity naming, they can better identify the correct entity when selecting from a small set of candidate names. Based on this insight, we propose a simple and training-free identify-before-answer IBA framework that decouples entity identification from section-level re-ranking. Our approach prompts an MLLM to select high-confidence entities using only candidate names, followed by an off-the-shelf textual re-ranker for evidence selection. Experiments on Encyclopedic-VQA and InfoSeek show that our method consistently outperforms fine-tuned multi-modal re-ranking baselines while reducing training and inference complexity. Additional analyses reveal that the improvements arise not only from better entity identification, but also from selecting more informative evidence once correct entity is fixed. Our implementation is made public to ease reproducibility.

Chinese Translation

基于知识的视觉问答（KB-VQA）需要将视觉查询与超出图像中直接可观察内容的外部知识进行关联。尽管最近的多模态大型语言模型（MLLMs）展现出强大的感知能力，但在需要从细粒度实体和证据层面进行定位的KB-VQA任务中，它们表现不佳。现有的大多数多模态检索增强生成（MM-RAG）方法将实体区分和章节级证据排序紧密结合在一个单一的重新排序阶段，导致高成本和有限的泛化能力。在本研究中，我们从工作流程的角度重新审视现有的MM-RAG解决方案，并认为实体级和事实级的定位是关键瓶颈。我们观察到，尽管MLLMs在开放式实体命名下常常失败，但在从一小组候选名称中选择时，它们能够更好地识别正确的实体。基于这一见解，我们提出了一个简单且无需训练的“识别优先回答”（IBA）框架，将实体识别与章节级重新排序解耦。我们的方法促使MLLM仅使用候选名称选择高置信度实体，随后使用现成的文本重新排序器进行证据选择。在Encyclopedic-VQA和InfoSeek上的实验表明，我们的方法在减少训练和推理复杂性的同时，始终优于微调的多模态重新排序基线。额外的分析表明，改进不仅源于更好的实体识别，还源于在确定正确实体后选择更具信息性的证据。我们的实现已公开，以便于复现。

View on arXiv Download PDF AI Translation

cs.CL / 7 / 2606.23884

One Year Later...The Harms Persist, But So Do We!

一年后……伤害仍在持续，但我们也在坚持！

Schoene, Annika Marie, Canca, Cansu, Kumar, Gautham Vijay, Antony, Anson

Abstract

General-purpose large language models (LLMs) are increasingly used for mental health-related conversations, yet safety safeguards remain inadequate and inconsistent across clinical conditions. This study evaluates six proprietary LLMs across 16 DSM-5 conditions using four adversarial attack variants, introducing an eight-dimension harm taxonomy and a multi-dimensional evaluation framework. Results show that safeguards hold reliably only for suicide and self-harm, while conditions such as eating disorders, substance use disorder, and major depressive disorder exhibit failure rates of up to 100%. We argue that ethical design and deployment of these LLMs demand clearly defined harm categories across clinical conditions and implementation of safeguards accordingly. Until such safeguards are in place, these models pose significant risks to vulnerable populations, making their growing integration into educational settings a particularly concerning.

Chinese Translation

通用大型语言模型（LLMs）在心理健康相关对话中的使用日益增加，但安全保障措施在不同临床条件下仍然不足且不一致。本研究评估了六种专有LLM在16种DSM-5条件下的表现，使用了四种对抗攻击变体，并引入了一个八维伤害分类法和一个多维评估框架。结果显示，安全保障措施仅在自杀和自残方面可靠，而饮食失调、物质使用障碍和重度抑郁障碍等条件的失败率高达100%。我们认为，这些LLM的伦理设计和部署需要在临床条件下明确定义伤害类别，并相应实施安全保障措施。在这些安全保障措施到位之前，这些模型对脆弱人群构成重大风险，使其在教育环境中的日益整合尤为令人担忧。

View on arXiv Download PDF AI Translation

cs.CL / 8 / 2606.23915

Do LLM Attribution Metrics Transfer? Auditing Retrieval-Augmented Generation Evaluation Across Datasets and Constructs

大规模语言模型归因指标是否可迁移？跨数据集和构造审计检索增强生成评估

Ding, Tianyu, Nannapaneni, Aditya, Weinstein, Juan Pablo De la Cruz

Abstract

Practice often treats automatic metrics for attribution in LLM retrieval-augmented generation as interchangeable. We audit eight automatic scorers -- lexical, embedding, and BERTScore baselines alongside entailment/grounding-trained models (clean and FEVER NLI, the checker MiniCheck) -- across three evaluation constructs (provenance/topicality, generated-answer attribution, and fact-check entailment), asking whether any scorer transfers: stays within the 95% confidence interval of the best audited scorer on every dataset of a multi-dataset construct. In the construct with the most multi-dataset human-labeled coverage -- generated-answer attribution (AttributionBench's four source datasets, n = 1,610, with independent HAGRID, n = 2,150) -- none does: the per-dataset metric rankings invert (Kendall tau = -0.64, p = 0.031 on AttributedQA vs. LFQA), and an off-the-shelf NLI scorer that is best on short-claim AttributedQA (AUROC 0.90) collapses to AUROC 0.53 (chance) on long-form LFQA, where BERTScore wins (0.91); the flip is not a length or truncation artifact. This instability has a concrete decision cost: a naive "best-on-average" rule for choosing an evaluator fails leave-one-dataset-out (mean held-out regret 0.172 AUROC, worse than fixing one scorer), so metric choice must be validated on the target dataset rather than learned from others. A prompt-based LLM judge avoids the chance-level collapses the automatic scorers suffer (no LFQA collapse) but is not uniformly best, ~100x costlier, and non-deterministic -- relocating, not removing, the validation burden.

Chinese Translation

实践中常常将大规模语言模型（LLM）检索增强生成中的自动归因指标视为可互换的。我们审计了八种自动评分器——词汇、嵌入和 BERTScore 基线，以及经过蕴涵/基础训练的模型（干净数据集和 FEVER NLI，检查器 MiniCheck）——在三个评估构造（来源/主题性、生成答案归因和事实检查蕴涵）中，探讨任何评分器是否能够迁移：即在多数据集构造的每个数据集上保持在最佳审计评分器的 95% 置信区间内。在覆盖最多多数据集人工标注的构造中——生成答案归因（AttributionBench 的四个源数据集，n = 1,610，独立的 HAGRID，n = 2,150）——没有一个评分器能够做到：每个数据集的指标排名发生了反转（Kendall tau = -0.64，p = 0.031 在 AttributedQA 与 LFQA 之间），而在短声明 AttributedQA 上表现最佳的现成 NLI 评分器（AUROC 0.90）在长格式 LFQA 上崩溃至 AUROC 0.53（随机），而 BERTScore 则获胜（0.91）；这种反转并不是长度或截断的伪影。这种不稳定性带来了具体的决策成本：选择评估器的简单“平均最佳”规则在留一数据集外的情况下失败（平均持出遗憾 0.172 AUROC，效果比固定一个评分器更差），因此指标选择必须在目标数据集上进行验证，而不是从其他数据集中学习。基于提示的 LLM 评判避免了自动评分器遭遇的随机水平崩溃（没有 LFQA 崩溃），但并不总是最佳，成本约为 100 倍，并且是非确定性的——重新分配而非消除验证负担。

View on arXiv Download PDF AI Translation

cs.CL / 9 / 2606.23937

When Retrieval Metrics Mislead: Measuring Policy Signal in Long-Horizon Tool-Use Agents

当检索指标误导时：测量长时间使用工具的智能体中的政策信号

Ding, Tianyu, Weinstein, Juan Pablo De la Cruz

Abstract

Exact-match retrieval recall is often used as a proxy for whether a retriever supplies useful policy context to a downstream decision model. We test this proxy for pre-action policy classification in tau-bench using Qwen2.5-3B/7B classifiers. Under gold-policy conditioning, a compact structured state improves macro-F1 over raw trajectories by 0.13-0.17 after tuning. We then replace the benchmark-designated policy clause with the top-ranked clause retrieved from decision-time context. Although the exact governing clause is retrieved at rank 1 for only 7% of airline states, the primary 3B classifier obtains macro-F1 0.58 with retrieved clauses versus 0.60 with gold clauses (Delta=-0.02, task-cluster 95% CI [-0.23,+0.21]); mismatched-policy and no-policy controls score 0.32 and 0.21. We do not detect a macro-F1 difference between retrieved and gold clauses in this configuration, although the interval remains too wide to establish non-inferiority. The same qualitative pattern appears with a second retriever and at 7B, while varying across fine-tuning configurations. These results indicate that exact-match clause recall can underestimate downstream policy utility in this benchmark setting, motivating evaluation with retrieved policies in the classification loop rather than recall alone.

Chinese Translation

精确匹配检索召回率通常被用作检索器是否为下游决策模型提供有用政策上下文的代理。我们在 tau-bench 中使用 Qwen2.5-3B/7B 分类器测试这一代理，用于预行动政策分类。在黄金政策条件下，经过调优后，紧凑的结构化状态在宏观 F1 指标上相较于原始轨迹提高了 0.13-0.17。随后，我们用从决策时上下文中检索到的排名最高的条款替换基准指定的政策条款。尽管在 7% 的航空公司状态下，精确的治理条款仅以排名 1 被检索到，但主要的 3B 分类器在使用检索到的条款时获得的宏观 F1 为 0.58，而使用黄金条款时为 0.60（Delta=-0.02，任务集群 95% CI [-0.23,+0.21]）；不匹配政策和无政策控制的得分分别为 0.32 和 0.21。在这种配置下，我们未能检测到检索到的条款与黄金条款之间的宏观 F1 差异，尽管该区间仍然过宽，无法确定非劣性。第二个检索器和 7B 的结果显示出相同的定性模式，但在微调配置上有所不同。这些结果表明，在该基准设置中，精确匹配条款召回率可能低估下游政策的效用，因此需要在分类循环中评估检索到的政策，而不仅仅是召回率。

View on arXiv Download PDF AI Translation

cs.CL / 10 / 2606.23943

QuechuaTok: Morphological Boundary Accuracy as a Necessary Metric for Tokenizer Evaluation in Agglutinative Low-Resource Languages

QuechuaTok：形态边界准确性作为评估粘着性低资源语言分词器的必要指标

Contreras, Maria

Abstract

Tokenization is a foundational step in NLP pipelines, yet standard evaluation metrics such as fertility rate fail to capture morphological correctness for agglutinative languages. We present QuechuaTok, a systematic benchmark comparing four tokenization strategies - BPE, Unigram LM, WordPiece, and a morphology-aware PRPE tokenizer - for Southern Quechua (quz), a low-resource agglutinative language spoken by 8-10 million people in South America. Using a 200k-sentence corpus and the SQUOIA finite-state morphological analyzer (Rios, 2016) as silver standard, we evaluate three metrics: fertility rate, OOV rate, and morphological boundary accuracy (MorphAcc). Our results show that BPE achieves the lowest fertility rate (1.636 at 16k vocab) by memorizing surface word forms, while achieving only 6.67% MorphAcc. PRPE achieves 83.33% MorphAcc - the highest of all systems - demonstrating that fertility rate alone is insufficient to evaluate tokenizers for agglutinative languages. All code and models are publicly available at kaggle.com/code/macmaky/quechuatok

Chinese Translation

分词是自然语言处理（NLP）流程中的基础步骤，然而，诸如生育率等标准评估指标未能有效捕捉粘着性语言的形态正确性。我们提出了QuechuaTok，这是一个系统的基准，比较了四种分词策略——BPE、单元语言模型（Unigram LM）、WordPiece和一种形态感知的PRPE分词器——针对南奎楚语（Southern Quechua，quz），这是一种在南美有800万至1000万使用者的低资源粘着性语言。我们使用了一个包含20万句子的语料库和SQUOIA有限状态形态分析器（Rios, 2016）作为银标准，评估了三个指标：生育率、OOV率和形态边界准确性（MorphAcc）。我们的结果显示，BPE通过记忆表面词形实现了最低的生育率（在16k词汇下为1.636），但仅实现了6.67%的MorphAcc。PRPE则达到了83.33%的MorphAcc——所有系统中最高，表明仅依靠生育率不足以评估粘着性语言的分词器。所有代码和模型均可在kaggle.com/code/macmaky/quechuatok公开获取。

View on arXiv Download PDF AI Translation

cs.CL / 11 / 2606.23948

Layer-wise Probing of wav2vec 2.0 and Whisper for Consonant Cluster Reduction in African American English

对 wav2vec 2.0 和 Whisper 的逐层探测：非洲裔美国英语中的辅音簇简化

Mojarad, Hamid, Tang, Kevin

Abstract

Self-supervised and supervised speech models are increasingly used to investigate which linguistic information their internal representations encode, and at what level of abstraction they encode it. One underexplored phenomenon is consonant cluster reduction (CCR) in African American English (AAE), a widespread phonological process and a source of automatic speech recognition (ASR) disparity. To examine how CCR is represented, we conduct speaker-independent layer-wise probing of wav2vec2-base and Whisper-small using two tasks: segmental reduction detection and segmental restoration of underlying cluster identity. Both models distinguish reduced and canonical forms with high accuracy. Crucially, reduced segments retain cues to their underlying stops, indicating that CCR is encoded as structured gradient phonological variation rather than simple segmental deletion. These results demonstrate structured phonological encoding of AAE CCR patterns in modern speech models.

Chinese Translation

自监督和监督的语音模型越来越多地被用于研究其内部表征编码了哪些语言信息，以及在何种抽象层次上进行编码。一个尚未深入探讨的现象是非洲裔美国英语（AAE）中的辅音簇简化（CCR），这是一种普遍存在的音位过程，也是自动语音识别（ASR）差异的一个来源。为了研究 CCR 的表征，我们对 wav2vec2-base 和 Whisper-small 进行了说话者独立的逐层探测，采用了两个任务：音段简化检测和基础簇身份的音段恢复。这两个模型以高准确率区分简化形式和规范形式。重要的是，简化的音段保留了其基础停顿的线索，表明 CCR 被编码为结构化的渐进音位变异，而非简单的音段删除。这些结果展示了现代语音模型中 AAE CCR 模式的结构化音位编码。

View on arXiv Download PDF AI Translation

cs.CL / 12 / 2606.23959

Does My Embedding Reflect That $A = B$? Evaluating Mathematical Equivalence in Embedding Models

我的嵌入是否反映了 $A = B$？评估嵌入模型中的数学等价性

Ye, Jiaying, Rao, Samarth, Carlin, Leo, Chintalapati, Kedar, Bhargava, Saharsh, Jaiswal, Rachit, Zhou, Michael, Darlington, Jared, Alper, Jarod, Ilin, Vasily, Kvinge, Henry

Abstract

Because mathematics is highly abstract, a single statement can take very different forms depending on what subfield it is framed in. There are many examples where breakthroughs occurred after researchers discovered that a question had already been answered in a different field. At the same time, the growth of new resources related to formalization has increased the need for tools that enable efficient and reliable navigation between mathematical 'languages' (e.g., from Lean to natural language). In this paper, we investigate whether current embedding models capture mathematical equivalence. To do this, we introduce the Mathematically Equivalent but Lexically Different Pairs (MELD) Dataset, a collection of mathematically equivalent statements that are expressed in very different language. We show that current state-of-the-art embedding models tend to group statements by the terminology used to make them instead of the underlying math. Motivated by this, we propose a contrastive approach to learning embeddings of mathematical text that focuses on aligning informal statements with different formalizations. Our experiments demonstrate that this leads to improvements not only on informal-formal retrieval tasks but also on MELD, which only contains natural language statements.

Chinese Translation

由于数学高度抽象，单一陈述在不同子领域中可能呈现出非常不同的形式。许多例子表明，在研究人员发现某个问题在其他领域已被解答后，突破性进展便随之而来。同时，与形式化相关的新资源的增长增加了在数学“语言”（例如，从 Lean 到自然语言）之间高效可靠导航工具的需求。在本文中，我们研究当前的嵌入模型是否能够捕捉数学等价性。为此，我们引入了“数学等价但词汇不同对”（Mathematically Equivalent but Lexically Different Pairs, MELD）数据集，这是一个包含用非常不同语言表达的数学等价陈述的集合。我们展示了当前最先进的嵌入模型倾向于根据所使用的术语对陈述进行分组，而不是基于其底层数学。受到这一点的启发，我们提出了一种对比学习方法，用于学习数学文本的嵌入，重点在于将不同形式化的非正式陈述进行对齐。我们的实验表明，这不仅在非正式-正式检索任务上有所改善，也在仅包含自然语言陈述的 MELD 上取得了进展。

View on arXiv Download PDF AI Translation

cs.CL / 13 / 2606.23989

Faithful by Construction: Claim-Anchored Attribution for Multi-Document Summarization

构建忠实性：基于声明的多文档摘要归因

Guan, Shuo

Abstract

End-to-end large language models (LLMs) produce fluent multi-document summaries but remain prone to hallucination, and the attributions they offer are typically coarse (whole documents or passages) and generated post hoc, leaving each summary statement hard to verify. We revisit the modular Extract--Select--Rewrite paradigm and recast its intermediate representation as the unit of attribution. We present CAMS, a Claim-Anchored Multi-document Summarization framework that (i) extracts atomic claims with token-level provenance from every source document, (ii) clusters equivalent claims across documents while flagging inter-source conflicts, (iii) selects a support-aware and salient subset, and (iv) rewrites the selection into a summary in which every sentence is anchored to a support-checked claim that links back to one or more source spans. Because content is localized before it is realized, the pipeline is attribution-oriented by construction and faithfulness-oriented by construction: it structurally preserves fine-grained, multi-source traceability while using support-aware selection, constrained rewriting, and verification to encourage, rather than guarantee, factual faithfulness. We evaluate quality, faithfulness, and localization on MultiNews, analyze conflict handling on DiverseSumm, and test zero-shot transfer on WCEP, using a two-regime protocol that separates reference-free citation quality from gold-aligned localization accuracy, and we add an evaluator-decoupled audit that tests citation precision with a support model never used for selection or verification. CAMS matches strong end-to-end and span-attribution baselines on summary quality while substantially improving faithfulness and citation precision, lifting multi-source attribution accuracy by roughly two-thirds, and exposing a controllable faithfulness--coverage trade-off that end-to-end models leave implicit.

Chinese Translation

端到端的大型语言模型（LLMs）能够生成流畅的多文档摘要，但仍然容易出现幻觉，其提供的归因通常较为粗糙（整体文档或段落），且是在事后生成的，使得每个摘要陈述难以验证。我们重新审视模块化的提取-选择-重写范式，并将其中间表示重新构建为归因单元。我们提出了CAMS（Claim-Anchored Multi-document Summarization）框架，该框架（i）从每个源文档中提取具有标记级来源的原子声明，（ii）在文档之间聚类等价声明，同时标记源间冲突，（iii）选择一个支持感知且显著的子集，以及（iv）将选择重写为摘要，其中每个句子都锚定到一个经过支持检查的声明，链接回一个或多个源跨度。由于内容在实现之前已被局部化，该流程在构建上是以归因为导向的，并且在构建上是以忠实性为导向的：它在结构上保留了细粒度的多源可追溯性，同时使用支持感知选择、受限重写和验证来鼓励而非保证事实忠实性。我们在MultiNews上评估质量、忠实性和局部化，在DiverseSumm上分析冲突处理，并在WCEP上测试零样本迁移，使用一种将无参考引用质量与黄金对齐的局部化准确性分开的双重协议，并增加了一个评估者解耦审计，测试引用精度，使用的支持模型从未用于选择或验证。CAMS在摘要质量上与强大的端到端和跨度归因基线相匹配，同时显著提高了忠实性和引用精度，将多源归因准确性提高了大约三分之二，并揭示了端到端模型隐含的可控忠实性-覆盖权衡。

View on arXiv Download PDF AI Translation

cs.CL / 14 / 2606.23992

RASC+: Retrieval-Constrained LLM Adjudication for Clinical Value Set Authoring

RASC+: 受检索约束的大型语言模型裁决在临床价值集编写中的应用

Mukherjee, Sumit

Abstract

Clinical value sets define the standardized terminology codes used in quality measurement, phenotyping, cohort construction, and clinical decision support. The recently introduced Retrieval-Augmented Set Completion (RASC) benchmark showed that direct zero-shot large language model (LLM) generation is poorly suited to this task: clinical code systems are large, version-controlled, and not reliably memorized by language models. We study a stage-wise alternative in which candidate-pool construction is optimized for recall and a constrained LLM adjudicator is optimized for candidate selection. On the full 3,744-value-set RASC test split, Qwen3-based retrieval with vocabulary-aware expansion and code-display rescue retrieval increases candidate-pool recall from the original RASC retrieval baseline of 0.553 to 0.730; on the held-out-publisher stratum, pool recall is 0.655. The higher-recall pool alone is not sufficient: applying the original SAPBert cross-encoder to this expanded pool gives full-test macro F1 of 0.287 and held-out-publisher macro F1 of 0.233. Replacing the stage-2 selector with blinded GPT-5 adjudication over the same pool increases full-test macro F1 to 0.549 and held-out-publisher macro F1 to 0.533. These results show that retrieval-constrained LLM adjudication can substantially improve value set completion while preserving the safety constraint that all returned codes must come from an auditable candidate pool.

Chinese Translation

临床价值集定义了用于质量测量、表型构建、队列构建和临床决策支持的标准化术语代码。最近引入的检索增强集完成（Retrieval-Augmented Set Completion, RASC）基准显示，直接的零-shot大型语言模型（Large Language Model, LLM）生成不适合此任务：临床代码系统庞大、版本控制且语言模型无法可靠记忆。我们研究了一种阶段性替代方案，其中候选池的构建优化为召回率，而受限的LLM裁决者则优化候选选择。在完整的3,744个价值集RASC测试分割中，基于Qwen3的检索结合词汇感知扩展和代码显示救援检索将候选池召回率从原始RASC检索基线的0.553提高到0.730；在保留的出版商层中，池召回率为0.655。仅仅提高召回率的池是不够的：将原始的SAPBert交叉编码器应用于这个扩展池，得到的完整测试宏F1为0.287，保留出版商宏F1为0.233。将阶段2选择器替换为对同一池进行盲测的GPT-5裁决，将完整测试宏F1提高到0.549，保留出版商宏F1提高到0.533。这些结果表明，受检索约束的LLM裁决可以显著改善价值集的完成，同时保持所有返回代码必须来自可审计候选池的安全约束。

View on arXiv Download PDF AI Translation

cs.CL / 15 / 2606.24004

Towards Spec Learning: Inference-Time Alignment from Preference Pairs

朝向规格学习：基于偏好对的推理时对齐

Krishnan, Dhriti, Goyal, Tejas, Savelka, Jaromir

Abstract

Steering a large language model (LLM) toward a desired behavior typically relies on an iterative process of hand-crafting a prompt based on a careful inspection of the model's responses. This is an involved, brittle, and error-prone process. Preference-based fine-tuning is a more rigorous but often prohibitively expensive solution. We propose spec learning, a framework that relies on a brief user instruction and a small set of preference judgments. These are compiled into specifications in the form of natural-language prompts for an LLM. Specifications condition LLMs at inference time, and no parameter updates to the underlying models are required. We show that the responses generated based on the compiled specifications often outperform direct preference optimization (DPO) on datasets from specialized domains whose preference signal is dense. Unlike opaque weight updates, the resulting specifications are human-readable and double as interpretable and transparent written embodiments of the preference signal that produced them.

Chinese Translation

引导大型语言模型（LLM）朝向期望行为通常依赖于一个迭代过程，该过程基于对模型响应的仔细检查手工制作提示。这是一个复杂、脆弱且容易出错的过程。基于偏好的微调是一种更为严格但通常成本过高的解决方案。我们提出了规格学习（spec learning），这是一个依赖于简短用户指令和一小组偏好判断的框架。这些内容被编译成自然语言提示的规格，用于LLM。规格在推理时对LLM进行条件限制，并且不需要对基础模型进行参数更新。我们展示了基于编译规格生成的响应通常在偏好信号密集的专业领域数据集上优于直接偏好优化（DPO）。与不透明的权重更新不同，生成的规格是人类可读的，且同时作为偏好信号的可解释和透明的书面体现。

View on arXiv Download PDF AI Translation

cs.CL / 16 / 2606.24040

Towards Version-aware Operations and Transaction Memories for Multi-layer MeMo

面向多层 MeMo 的版本感知操作和事务记忆

Li, Peiran

Abstract

MeMo proposes language models with explicit multi-layer correlation matrix memories (CMMs), where memorization, retrieval, and forgetting are architectural operations. This paper asks how such memories can reduce the need for retraining when knowledge changes. For changes expressible as MeMo memory associations, the model's accessible knowledge can be updated by editing explicit memories rather than retraining the whole model. We propose a version-aware operation layer in which high-level operations such as replace, obsolete, keep-history, rollback, and trace are compiled into MeMo-native primitive calls over sequences and tokens. The key observation is that a version-aware operation is rarely a single MeMo association. It is an ordered transaction of primitive edits, for example forgetting one sequence-token chain, memorizing another, preserving a historical chain, and recording an inverse program. The framework introduces two auxiliary CMMs: a Version CMM (V-CMM) for mapping version transitions to transaction handles, and a Transaction CMM (T-CMM) for storing reusable change contents and inverse programs. It supports both direct sequence-level edits and structured diff-level inputs, and outlines an evaluation route for update success, rollback, traceability, locality, and transaction reuse.

Chinese Translation

MeMo 提出了具有显式多层关联矩阵记忆（CMMs）的语言模型，其中记忆、检索和遗忘是架构操作。本文探讨了这些记忆如何在知识变化时减少重新训练的需求。对于可以表示为 MeMo 记忆关联的变化，模型的可访问知识可以通过编辑显式记忆来更新，而不是重新训练整个模型。我们提出了一种版本感知操作层，其中高层操作如替换、过时、保留历史、回滚和追踪被编译为 MeMo 原生的原语调用，作用于序列和标记。关键观察是，版本感知操作很少是单一的 MeMo 关联。它是一系列原始编辑的有序事务，例如遗忘一个序列-标记链、记忆另一个、保留一个历史链以及记录一个逆程序。该框架引入了两个辅助 CMM：一个版本 CMM（V-CMM）用于将版本转换映射到事务句柄，一个事务 CMM（T-CMM）用于存储可重用的变更内容和逆程序。它支持直接的序列级编辑和结构化的差异级输入，并概述了更新成功、回滚、可追溯性、局部性和事务重用的评估路径。

View on arXiv Download PDF AI Translation

cs.CL / 17 / 2606.24055

Best Preprocessing Techniques for Sentiment Analysis

情感分析的最佳预处理技术

Magsarjav, Saranzaya, Humphries, Melissa, Tuke, Jonathan, Mitchell, Lewis

Abstract

Sentiment analysis in Twitter datasets is important because it enables monitoring public opinion on products and analysis of political and social movements. One critical step is preprocessing: the automated processing of text for machine learning algorithms. Preprocessing plays a critical role in reducing noise and improving efficiency. However, little research has systematically examined the order in which preprocessing techniques are implemented. We find that, when accounting for order, spelling correction is the least impactful preprocessing technique, whereas tokenisation is the most impactful. Stemming and stop-word removal are interchangeable, and it is better to remove stop words without removing negation. The best order for applying the preprocessing techniques was tokenisation, text cleaning, stemming, and then stopword removal. Our results provide a systematic approach for practitioners to deploy preprocessing to improve model output without the costly preprocessing exploratory phase.

Chinese Translation

Twitter 数据集中的情感分析至关重要，因为它能够监测公众对产品的意见以及分析政治和社会运动。预处理是一个关键步骤：即对文本进行自动处理，以便于机器学习算法的使用。预处理在减少噪声和提高效率方面发挥着重要作用。然而，关于预处理技术实施顺序的系统研究较少。我们的研究发现，在考虑顺序时，拼写纠正是影响最小的预处理技术，而分词（tokenisation）则是影响最大的。词干提取（stemming）和停用词移除（stop-word removal）是可以互换的，并且在移除停用词时最好不去除否定词。应用预处理技术的最佳顺序是：分词、文本清理、词干提取，然后是停用词移除。我们的结果为从业者提供了一种系统的方法，以便在不进行昂贵的预处理探索阶段的情况下，部署预处理以改善模型输出。

View on arXiv Download PDF AI Translation

cs.CL / 18 / 2606.24063

Selective Capability Unlearning in End-to-End Spoken Language Understanding

端到端语音语言理解中的选择性能力遗忘

Singh, Akanksha, Kurmi, Vinod Kumar

Abstract

Modern spoken language understanding (SLU) systems are increasingly deployed in real-world settings, where specific functionalities may need to be removed due to policy or safety constraints. In SLU, a functionality corresponds to an intent and its associated slot-generation behavior. However, in autoregressive models, suppressing a target intent does not eliminate the conditional mapping that generates slots conditioned on that intent. When the intent prefix is externally supplied, the model can reconstruct the original intent-slot structure. We identify this structural failure as \textbf{\emph{capability persistence}}. We propose \textit{\underline{B}inding \underline{S}ubspace (BSU)}, a representation-level framework that isolates and attenuates intent-conditioned directions underlying this mapping. Across SLU benchmarks, BSU substantially reduces forced-prefix recoverability while preserving retained performance.

Chinese Translation

现代语音语言理解（SLU）系统越来越多地在实际环境中部署，在这些环境中，可能由于政策或安全限制需要移除特定功能。在SLU中，功能对应于一个意图及其相关的槽生成行为。然而，在自回归模型中，抑制目标意图并不会消除生成基于该意图的槽的条件映射。当意图前缀被外部提供时，模型可以重构原始的意图-槽结构。我们将这种结构性失败称为 extbf{ extit{能力持久性}}。我们提出了 extit{ extunderline{B}inding extunderline{S}ubspace (BSU)}，这是一个表示层级框架，旨在隔离和减弱这一映射背后的意图条件方向。在SLU基准测试中，BSU显著降低了强制前缀的可恢复性，同时保持了保留性能。

View on arXiv Download PDF AI Translation

cs.CL / 19 / 2606.24077

Sentence-Level Contextual Entrainment in Large Language Models

大型语言模型中的句子级上下文同步

Liu, Yang, Chu, Chenhui

Abstract

Contextual entrainment, which is a newly discovered phenomenon in large language models (LLMs), refers to the tendency of a model to assign higher probabilities to tokens that appear in its context. In this work, we extend this phenomenon from the token level to the sentence level by examining the per-token mean log-probability of a sentence instead of the probabilities of individual tokens. We investigate sentence-level contextual entrainment across 26 LLMs from seven families and two datasets, which cover both subjective and objective tasks. We find that sentence-level contextual entrainment exists. This means that the sentences in the prompt (even if they are counterfactual statements) can significantly increase their probability during model inference time. As the model size increases, contextual entrainment gradually decreases. We also find that contextual entrainment is controlled by 2% to 4% of the attention heads. Turning off these attention heads can effectively mitigate contextual entrainment without hurting the model's performance.

Chinese Translation

上下文同步是大型语言模型（LLMs）中发现的一种新现象，指的是模型倾向于对其上下文中出现的标记分配更高的概率。在本研究中，我们将这一现象从标记级扩展到句子级，通过考察句子的每个标记的平均对数概率，而不是单个标记的概率。我们研究了来自七个家族的26个LLM在两个数据集上的句子级上下文同步，这些数据集涵盖了主观和客观任务。我们发现句子级上下文同步确实存在。这意味着提示中的句子（即使是反事实陈述）在模型推理时可以显著提高其概率。随着模型规模的增加，上下文同步逐渐减弱。我们还发现，上下文同步由2%到4%的注意力头控制。关闭这些注意力头可以有效减轻上下文同步，而不会损害模型的性能。

View on arXiv Download PDF AI Translation

cs.CL / 20 / 2606.24083

CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression

CAVEWOMAN：大型语言模型在语言输入和输出压缩下的行为

Adeyemi, Morayo Danielle, Rossi, Ryan A., Dernoncourt, Franck

Abstract

"Talk short. Drop grammar. Save token." This caveman style is widely promoted as a way to cut inference cost, but whether it actually saves anything depends on which channel (the user's prompt or the model's response) is being compressed. We present Cavewoman, a two-channel evaluation protocol that scores every generation on task accuracy, realized per-item cost, and reference-text agreement against the model's unconstrained reference. We evaluate eight models on five datasets at five reduction levels, with both channels measured on the same items. Output compression cuts realized cost on most API models (1.4-2.4x per model, up to 3x in the best case) and on all four open-weight models under public-tier pricing. Input compression has the opposite effect, a strict lose-lose: it raises net cost rather than lowering it (~1.15x on the five-benchmark mean, up to 1.8x on the worst dataset and 2.7x under stronger compression), because models compensate with longer responses even as accuracy collapses. Under the same setting, surface text diverges from the unconstrained reference: on the non-reasoning models, roughly half of all generations are correct yet their surface text no longer entails the model's own unconstrained baseline generation. The divergence survives length-controlled re-scoring, multiple-comparisons correction, and replication under complementary semantic measures. Code and data are available at https://github.com/danielle34/cavewoman.

Chinese Translation

"简短对话。省略语法。节省令牌。" 这种穴居人风格被广泛宣传为降低推理成本的一种方式，但它是否真的能节省成本取决于压缩的是哪个通道（用户的提示或模型的响应）。我们提出了Cavewoman，一种双通道评估协议，针对每个生成结果在任务准确性、实际每项成本和与模型无约束参考文本的一致性进行评分。我们在五个数据集上对八个模型进行了评估，采用五个压缩级别，两个通道在相同项目上进行测量。输出压缩在大多数API模型上降低了实际成本（每个模型降低1.4-2.4倍，在最佳情况下可达3倍），并且在所有四个开放权重模型的公共定价下均有效。输入压缩则产生相反的效果，严格来说是双输：它提高了净成本而非降低（在五个基准的平均值上约为1.15倍，在最糟糕的数据集上可达1.8倍，在更强压缩下可达2.7倍），因为模型在准确性崩溃的情况下仍通过更长的响应来进行补偿。在相同设置下，表面文本与无约束参考文本出现偏离：在非推理模型上，约一半的生成结果是正确的，但它们的表面文本不再包含模型自身的无约束基线生成。即使在长度控制的重新评分、多重比较修正和补充语义度量下，这种偏离依然存在。代码和数据可在 https://github.com/danielle34/cavewoman 获取。

View on arXiv Download PDF AI Translation

cs.CL / 21 / 2606.24093

Predicting Poets' Origins from Verse: A Computational Analysis of Regional Linguistic Fingerprints in the Complete Tang Poems

从诗歌中预测诗人的来源：对《全唐诗》中区域语言特征的计算分析

Chen, Chi-Sheng, Liu, Hung-Yun

Abstract

We ask whether the geographic origin of Tang-dynasty poets leaves a detectable linguistic trace in their work. Aggregating every poem attributed to each author in the Complete Tang Poems (Quan Tang Shi) and linking poets to their administrative circuit of origin via the China Biographical Database (CBDB), we build a poet-level corpus of 357 poets across the ten Tang circuits and frame origin prediction as multi-class classification. Using character $n$-gram TF-IDF together with interpretable domain features (imagery, season, and allusion), classical and neural models predict a poet's broad region (South vs.\ North) at $0.69$ accuracy, well above the $0.53$ majority baseline, and finer circuit-level origin above chance. Beyond classification, three findings emerge. (i) Linguistic distance between circuits grows with geographic distance (Mantel $r=0.40$, $p\approx0.09$ over nine circuits), evidence of a distance-decay effect in poetic language. (ii) The signal interacts with time: South/North separability is at chance in the High Tang and strongest in the Late Tang, consistent with court-driven homogenization at the empire's height followed by regional divergence. (iii) The model's confident errors are historically meaningful -- in the Early Tang, every misclassification is a southern poet read as northern, reflecting the prestige of the northern court idiom. We further show that, when given the whole corpus through a hierarchical frozen-encoder representation, a classical-Chinese transformer (GuwenBERT) only matches -- not beats -- simple TF-IDF, and that combining them adds nothing, indicating that character $n$-grams already capture the regional signal. Our results position interpretable machine learning as a hypothesis generator for literary history.

Chinese Translation

我们探讨唐代诗人的地理来源是否在其作品中留下可检测的语言痕迹。通过汇总《全唐诗》中每位作者的所有诗作，并通过中国传记数据库（CBDB）将诗人与其行政区划的来源联系起来，我们构建了一个包含357位诗人的诗人级语料库，涵盖十个唐代行政区，并将来源预测框架设定为多类分类问题。使用字符 $n$-gram TF-IDF 结合可解释的领域特征（意象、季节和典故），经典模型和神经网络模型以 $0.69$ 的准确率预测诗人的大致区域（南方与北方），远高于 $0.53$ 的多数基线，并在更细致的区划层面上超出随机水平。除了分类之外，得出了三个发现。(i) 行政区之间的语言距离随着地理距离的增加而增长（Mantel $r=0.40$, $p ext{≈}0.09$，基于九个区），这表明诗歌语言中存在距离衰减效应。(ii) 这一信号与时间相互作用：在高唐时期南北的可分性接近随机，而在晚唐时期最为显著，这与帝国鼎盛时期的朝廷驱动的同质化及其后的区域分化一致。(iii) 模型的自信错误在历史上具有重要意义——在早唐时期，每一次误分类都是将南方诗人误读为北方诗人，反映了北方朝廷语言的声望。我们进一步表明，当通过分层冻结编码器表示给出整个语料库时，古文变换器（GuwenBERT）仅能匹配——而非超越——简单的 TF-IDF，且两者结合并未带来额外收益，表明字符 $n$-gram 已经捕捉到了区域信号。我们的结果将可解释的机器学习定位为文学史的假设生成器。

View on arXiv Download PDF AI Translation

cs.CL / 22 / 2606.24102

PORTER: Language-Grounded Event Representations for Portable Structured EHR Foundation Models

PORTER：面向可移植结构化电子健康记录基础模型的语言基础事件表示

Guo, Lin Lawrence, Yan, Adam Paul, Vettese, Emily, Sung, Lillian

Abstract

Most electronic health record (EHR) foundation models encode clinical events as discrete event tokens from a fixed vocabulary and therefore cannot directly represent events containing unseen concepts or new combinations of concepts and attributes such as numeric values. This limits transfer across institutions and even across deployment pipelines within the same institution. We introduce PORTER, a language-grounded structured EHR foundation model that decouples event representation from this fixed vocabulary. PORTER represents events through their descriptions using a frozen text encoder, integrates numeric values through a dedicated pathway, and learns clinical dynamics over patient timelines with an autoregressively pretrained temporal backbone. Across 74 clinical prediction tasks at a pediatric hospital, PORTER matched the mean AUROC of a fixed-vocabulary model with the same temporal backbone and pretraining objective. When the same patient timelines were rendered using event descriptions not seen during pretraining, PORTER transferred without retraining or vocabulary mapping, recovering 97.1% of the mean AUROC of a model trained directly on the target vocabulary. When transferred to MIMIC, PORTER outperformed the fixed-vocabulary model, which dropped 69% of events because their tokens were unseen. Mechanistic analyses showed cross-vocabulary transfer tracked preservation of patient-level representation geometry rather than the scale of the text encoder, and the numeric pathway improved sensitivity to magnitude without disrupting clinical concept identity. PORTER also achieved higher AUROC than a task-specific text serialization comparator, at 329-fold lower amortized compute. PORTER is a step toward vocabulary-independent EHR foundation models that reduce the need for vocabulary harmonization while preserving in-domain performance and enabling efficient cross-task reuse.

Chinese Translation

大多数电子健康记录（EHR）基础模型将临床事件编码为来自固定词汇表的离散事件标记，因此无法直接表示包含未见概念或新组合的概念和属性（如数值）的事件。这限制了跨机构的迁移，甚至在同一机构内的部署管道之间的迁移。我们提出了PORTER，一种语言基础的结构化EHR基础模型，它将事件表示与固定词汇表解耦。PORTER通过冻结的文本编码器使用事件描述表示事件，通过专用路径集成数值，并通过自回归预训练的时间主干学习患者时间线上的临床动态。在一家儿科医院的74个临床预测任务中，PORTER的平均AUROC与具有相同时间主干和预训练目标的固定词汇模型相匹配。当使用在预训练期间未见的事件描述呈现相同的患者时间线时，PORTER在不进行再训练或词汇映射的情况下实现了迁移，恢复了直接在目标词汇上训练的模型的97.1%的平均AUROC。当迁移到MIMIC时，PORTER的表现优于固定词汇模型，后者因其标记未见而丢失了69%的事件。机制分析显示，跨词汇迁移跟踪了患者级表示几何的保留，而不是文本编码器的规模，数值路径提高了对大小的敏感性，而不破坏临床概念的身份。PORTER还在329倍更低的摊销计算下，达到了比任务特定文本序列化比较器更高的AUROC。PORTER是朝着不依赖词汇的EHR基础模型迈出的一步，减少了词汇协调的需求，同时保持了领域内的性能并实现了高效的跨任务重用。

View on arXiv Download PDF AI Translation

cs.CL / 23 / 2606.24151

Metis: Bridging Text and Code Memory for Self-Evolving Agents

Metis：为自我进化代理架构桥接文本和代码记忆

Dai, Zijie, He, Siuhin, Li, Hui, Zhou, Qihui, Li, Jiajun, Song, Mingcong, Long, Guoping, Si, Hongjie, Yao, Xin, Zhang, Lin, Cheng, James, Yan, Xiao

Abstract

Self-evolving agents improve over time by distilling experience from past executions and reusing it in future tasks. Existing systems represent such experience either as natural-language text injected into the agent context or as code exposed as callable tools. However, the choice between these representations is typically made at design time rather than derived from the characteristics of the experience itself, leaving the trade-offs between them poorly understood. We present the first controlled study that isolates text memory and code memory over an identical set of experiences. Our results show that the two forms exhibit complementary trade-offs in construction cost, execution efficiency, and transferability, such that neither representation alone is sufficient. Guided by these findings, we propose Metis, a self-evolving agent system built on a hierarchical dual-representation memory. Metis organizes textual experience into execution plans, environment facts, and common pitfalls, and selectively crystallizes recurring plans into validated callable tools. This design combines the broad applicability of text memory with the execution efficiency of code memory while incurring tool-generation cost only when justified by repeated reuse. We evaluate Metis on AppWorld, a challenging benchmark for interactive agents. The results show that Metis improves task accuracy by up to 20.6% over ReAct while reducing execution cost by up to 22.8%. Compared with representative self-evolving agent systems, Metis consistently achieves a better balance between accuracy, execution efficiency, and memory-construction cost.

Chinese Translation

自我进化代理通过提炼过去执行的经验并在未来任务中重用这些经验来不断改进。现有系统将这种经验表示为注入代理上下文的自然语言文本或作为可调用工具暴露的代码。然而，这两种表示之间的选择通常是在设计阶段做出的，而不是根据经验本身的特征得出的，这使得它们之间的权衡尚未得到充分理解。我们提出了首个控制研究，隔离文本记忆和代码记忆，使用相同的经验集。我们的结果表明，这两种形式在构建成本、执行效率和可转移性方面表现出互补的权衡，因此单独使用任何一种表示都不足以满足需求。基于这些发现，我们提出了Metis，一个建立在层次化双重表示记忆上的自我进化代理系统。Metis将文本经验组织为执行计划、环境事实和常见陷阱，并选择性地将重复出现的计划结晶为经过验证的可调用工具。这种设计结合了文本记忆的广泛适用性与代码记忆的执行效率，同时仅在重复使用得到合理化时才产生工具生成成本。我们在AppWorld上评估Metis，这是一个针对交互代理的挑战性基准。结果表明，Metis在任务准确性上比ReAct提高了多达20.6%，同时执行成本降低了多达22.8%。与代表性的自我进化代理系统相比，Metis在准确性、执行效率和记忆构建成本之间始终实现了更好的平衡。

View on arXiv Download PDF AI Translation

cs.CL / 24 / 2606.24155

MedBench v5: A Dynamic, Process-Oriented, and Hallucination-Aware Benchmark for Clinical Multimodal Models

MedBench v5：一个动态、过程导向且具备幻觉意识的临床多模态模型基准

Jinru, Ding, Chuchu, Jiang, Lu, Lu, Wenrao, Pang, Mouxiao, Bian, Zhuangzhi, Gao, Jiangyuan, Chen, xinwei, Peng, Ruiyao, Chen, Sijie, Ren, Renjie, Lu, Bin, Han, Meiling, Liu, Jie, and Xu

Abstract

Existing medical AI benchmarks lack process visibility, atomic skill evaluation, and integrated hallucination detection. We introduce MedBench v5, a redesigned benchmark for clinical multimodal models (language, vision-language, and agent systems) that moves from static QA to dynamic, process-oriented evaluation. MedBench v5 features: (1) a dual-dimensional framework combining Clinical Cognitive Responsiveness (14 sub-dimensions) and Medical Atomic Skills (4 agent environments), covering 63 tasks; (2) three switchable information-flow stressors (omission, contradiction, evidence delay) for factorized degradation analysis; (3) a dynamic process audit protocol with five reasoning nodes that produces model-specific failure fingerprints; (4) hallucination propagation monitoring across initiation, propagation, anchoring, and contradiction interaction-capturing silent hallucination. Experiments on frontier models show that strong overall task performance does not guarantee process stability: stressors mainly disrupt contradiction detection, diagnosis updating, hallucination propagation, and contradiction-based self-correction, while final evidence grounding can remain superficially stable. MedBench v5 provides a unified infrastructure for capability profiling, controllable stress testing, process auditing, and hallucination trajectory analysis in clinical AI evaluation.

Chinese Translation

现有的医学人工智能基准缺乏过程可视化、原子技能评估和集成的幻觉检测。我们推出了MedBench v5，这是一个为临床多模态模型（语言、视觉-语言和智能体系统）重新设计的基准，旨在从静态问答转向动态、过程导向的评估。MedBench v5的特点包括：(1) 一个双维框架，结合临床认知反应性（14个子维度）和医学原子技能（4个智能体环境），涵盖63个任务；(2) 三个可切换的信息流压力源（遗漏、矛盾、证据延迟），用于因子化退化分析；(3) 一个具有五个推理节点的动态过程审计协议，生成特定模型的失败指纹；(4) 在启动、传播、锚定和矛盾交互中监测幻觉传播，捕捉静默幻觉。对前沿模型的实验表明，强大的整体任务表现并不保证过程稳定性：压力源主要干扰矛盾检测、诊断更新、幻觉传播和基于矛盾的自我修正，而最终证据的基础可以保持表面稳定。MedBench v5为临床人工智能评估提供了一个统一的基础设施，用于能力轮廓、可控压力测试、过程审计和幻觉轨迹分析。

View on arXiv Download PDF AI Translation

cs.CL / 25 / 2606.24162

BehaviorBench: Benchmarking Foundation Models for Behavioral Science Tasks

BehaviorBench：基础模型在行为科学任务中的基准测试

Huang, Jin, Xie, Yutong, Song, Wanli, Zhang, Xingjian, Yuan, Walter, Jackson, Matthew O., Mei, Qiaozhu

Abstract

Foundation models have been increasingly applied to behavioral science domains such as psychology, sociology, and economics. While these models show promise in individual tasks such as survey response prediction and human-subject experiment simulation, there remains no systematic understanding of how well they perform across diverse behavioral science tasks, contexts, and populations. We introduce BehaviorBench, a comprehensive benchmark that evaluates foundation models along four core capabilities: (1) behavior prediction and simulation, (2) strategic decision-making, (3) subject-trait inference, and (4) behavioral knowledge application. Crucially, BehaviorBench evaluates model outputs at both the individual and distributional levels, capturing not only per-subject accuracy but also population-level alignment, an essential requirement for behavioral validity. Leveraging the tasks in BehaviorBench, we further develop Be.FM-1.5, extending the Be.FM family of behavioral foundation models fine-tuned on behavioral data. Our results reveal a considerable gap: proprietary general-purpose models excel at individual-level prediction and knowledge-intensive tasks, whereas behavioral foundation models, fine-tuned on behavioral data, achieve substantially stronger distributional alignment. Notably, Be.FM-1.5 leads on distributional metrics and remains competitive on individual-level metrics, suggesting that proper behavioral adaptation can close the gap. Our results highlight the importance of distributional evaluation, establish BehaviorBench as a foundation for developing and assessing behaviorally aligned AI systems, and demonstrate Be.FM-1.5's potential for a broad range of behavioral science studies. Our BehaviorBench and Be.FM-1.5 models can be accessed via https://umich-foreseer.github.io/behaviorbench/.

Chinese Translation

基础模型在心理学、社会学和经济学等行为科学领域的应用日益增多。尽管这些模型在调查响应预测和人类实验模拟等单一任务中显示出潜力，但对于它们在多样化的行为科学任务、情境和人群中的整体表现仍缺乏系统性的理解。我们提出了BehaviorBench，这是一个全面的基准测试，评估基础模型在四个核心能力方面的表现：(1) 行为预测与模拟，(2) 战略决策，(3) 受试者特征推断，以及 (4) 行为知识应用。重要的是，BehaviorBench在个体和分布层面评估模型输出，不仅捕捉每个受试者的准确性，还关注人群层面的对齐，这是行为有效性的重要要求。借助BehaviorBench中的任务，我们进一步开发了Be.FM-1.5，扩展了在行为数据上微调的行为基础模型Be.FM系列。我们的结果揭示了一个显著的差距：专有的通用模型在个体级预测和知识密集型任务中表现出色，而在行为数据上微调的行为基础模型则在分布对齐方面取得了显著更强的表现。值得注意的是，Be.FM-1.5在分布指标上领先，并在个体级指标上保持竞争力，表明适当的行为适应可以缩小这一差距。我们的结果强调了分布评估的重要性，确立了BehaviorBench作为开发和评估行为对齐的人工智能系统的基础，并展示了Be.FM-1.5在广泛的行为科学研究中的潜力。我们的BehaviorBench和Be.FM-1.5模型可以通过 https://umich-foreseer.github.io/behaviorbench/ 访问。

View on arXiv Download PDF AI Translation

cs.CL / 26 / 2606.24172

A P\={a}ninian Foundation for Indic Language Processing

以帕尼尼为基础的印度语言处理

Banerjee, Ritwik, Varshney, Lav R.

Abstract

More than a billion people communicate in Indic languages, yet the natural language processing infrastructure serving them remains fragmented and underdeveloped. The cause is structural: the field organizes its tools and benchmarks around individual languages or small subsets of genealogical language families, building separate analyzers, parsers, and datasets for each language and starting over for the next. This overlooks a deep regularity. Through more than two millennia of convergence around Sanskrit, Indic languages came to share a morphosyntactic architecture formalized in P\={a}nini's grammar, the Ast\={a}dhy\={a}y\={i}. This cuts across genealogical lines, uniting languages through a common framework. We argue that this P\={a}ninian framework supplies a unifying computational architecture the field has lacked, and that benchmarks grounded explicitly in it would make Indic language systems more accurate, more data-efficient, and more transferable, effectively merging many apparently disparate and sparse Indic language resources into a single high-resource metalanguage bedrock. We propose a four-part benchmark suite to render this shared architecture explicit, measurable, and ready to be leveraged for practical applications. Moreover, we underscore the question it raises for interpretability research: whether neural models trained on these languages come to represent P\={a}nini's categories on their own.

Chinese Translation

超过十亿人使用印度语言进行交流，但为其服务的自然语言处理基础设施仍然支离破碎且发展不足。其原因在于结构性：该领域围绕单一语言或小范围的语言家族组织工具和基准，为每种语言构建独立的分析器、解析器和数据集，并在下一个语言上重新开始。这种做法忽视了深层的规律性。在超过两千年的时间里，印度语言围绕梵语趋同，形成了在帕尼尼的语法《阿斯塔德雅伊》中形式化的形态句法结构。这种结构跨越了语言家族的界限，通过一个共同的框架将语言联系在一起。我们认为，这一帕尼尼框架提供了该领域所缺乏的统一计算架构，并且以此为基础的基准将使印度语言系统更准确、更高效、更具可迁移性，有效地将许多看似不同且稀疏的印度语言资源合并为一个高资源的元语言基础。我们提出了一个四部分的基准套件，以使这一共享架构变得明确、可测量，并准备好用于实际应用。此外，我们强调了它对可解释性研究提出的问题：在这些语言上训练的神经模型是否能够自主地表示帕尼尼的分类。

View on arXiv Download PDF AI Translation

cs.CL / 27 / 2606.24176

A Synthetic Reliability-Aware PINN Benchmark for Offshore Wind Turbine Support-Structure Monitoring with Bayesian Inverse Identification

一种合成的可靠性意识物理信息神经网络基准，用于离岸风力涡轮机支撑结构监测与贝叶斯逆向识别

Kant, Puneet, Tanwar, Monika

Abstract

Reliable structural health monitoring (SHM) of offshore wind turbine (OWT) support structures requires fast state estimation from sparse measurements. Repeated high fidelity finite element or aeroelastic analyses are difficult to use directly in online monitoring loops, while purely data-driven surrogates can require large training sets. This paper presents Digi Turbine, a synthetic reliability-aware Physics Informed Neural Network (PINN) benchmark for OWT monopile support structure monitoring. The workflow embeds a simplified Euler Bernoulli beam equation with Winkler soil foundation in the training objective, couples it with Bayesian-prior-informed inverse identification, and adds First Order Reliability Method (FORM) screening. All validation uses synthetic configurations with analytical or finite-difference ground truth motivated by the NREL 5MW reference turbine context.

Chinese Translation

离岸风力涡轮机（OWT）支撑结构的可靠结构健康监测（SHM）需要从稀疏测量中快速进行状态估计。重复进行高保真有限元或气动弹性分析在在线监测循环中难以直接使用，而纯数据驱动的替代模型可能需要大量的训练数据集。本文提出了Digi Turbine，一种合成的可靠性意识物理信息神经网络（PINN）基准，用于OWT单桩支撑结构监测。该工作流程将简化的欧拉-伯努利梁方程与温克勒土壤基础嵌入训练目标中，结合贝叶斯先验信息的逆向识别，并增加一阶可靠性方法（FORM）筛选。所有验证均使用合成配置，基于NREL 5MW参考涡轮机背景，采用解析或有限差分的真实值进行验证。

View on arXiv Download PDF AI Translation

cs.CL / 28 / 2606.24188

Aspect-Based Sentiment Evolution and its Correlation with Review Rounds in Multi-Round Peer Reviews: A Deep Learning Approach

基于方面的情感演变及其与多轮同行评审中的评审轮次的相关性：一种深度学习方法

Hana, Ruxue, Zhoua, Haomin, Zhong, Jiangtao, Zhang, Chengzhi

Abstract

Mining sentiment information from the textual content of peer review comments offers valuable insights into the scientific evaluation process. However, previous studies are often constrained by coarse-grained analysis and the lack of differentiation across review rounds. Notably, the dynamic shifts in reviewers' focus and sentiment tendencies throughout multiple review stages remain underexplored. To address this gap, the present study investigates the distribution and evolution of aspect-level sentiments and examines their correlation with the number of review rounds. We begin by segmenting the multi-round review comments of 11,063 accepted papers from Nature Communications and identifying fine-grained review aspect clusters. A manually annotated corpus of approximately 5,000 review sentences is then constructed. Using this dataset, we train a series of deep learning-based aspect sentiment classification models. Among them, the LCF-BERT-CDM model achieves the best performance, with a Macro-F1 score of 82.65%. Subsequent statistical analysis reveals a consistent trend: as the number of review rounds increases, the proportion of positive sentiments rises, while negative sentiments decline. Correlation analysis further indicates that aspect sentiment scores are negatively associated with the total number of review rounds. Key aspects exhibiting stronger correlations include "experiments", "research significance" and "result analysis".

Chinese Translation

从同行评审评论的文本内容中挖掘情感信息为科学评估过程提供了宝贵的见解。然而，以往的研究往往受到粗粒度分析和缺乏对评审轮次区分的限制。尤其是评审者在多个评审阶段中的关注点和情感倾向的动态变化仍然未被充分探索。为了解决这一问题，本研究调查了基于方面的情感的分布和演变，并考察其与评审轮次数量的相关性。我们首先对来自《自然通讯》的11,063篇已接受论文的多轮评审评论进行分段，并识别出细粒度的评审方面聚类。随后，构建了一个约5,000条评审句子的人工标注语料库。利用该数据集，我们训练了一系列基于深度学习的方面情感分类模型。其中，LCF-BERT-CDM模型表现最佳，Macro-F1分数达到82.65%。后续的统计分析揭示了一种一致的趋势：随着评审轮次的增加，积极情感的比例上升，而消极情感则下降。相关性分析进一步表明，方面情感得分与评审轮次总数呈负相关关系。表现出更强相关性的关键方面包括“实验”、“研究意义”和“结果分析”。

View on arXiv Download PDF AI Translation

cs.CL / 29 / 2606.24200

MMed-Bench-IR: A Heterogeneous Benchmark for Multilingual Medical Information Retrieval

MMed-Bench-IR：一个异构的多语言医学信息检索基准

Lee, Junhyeok, Jang, Han, Goh, Hyeonjin, Choi, Kyu Sung

Abstract

Retrieval-augmented generation (RAG) in clinical settings increasingly requires multilingual retrieval against predominantly English evidence corpora. Multilingual medical retrieval demands three capabilities: cross-lingual alignment, concept discrimination, and evidence retrieval. However, existing benchmarks evaluate these only in isolation, leaving the interaction between biomedical expertise and multilingual coverage unmeasured. We introduce MMed-Bench-IR, a benchmark designed to disentangle these axes across 6 languages and three structurally heterogeneous tasks: (1) cross-lingual medical QA retrieval with 6,127 queries grounded in the Unified Medical Language System (UMLS), (2) concept discrimination over 4,975 confusion sets at three difficulty tiers, and (3) multilingual evidence retrieval for RAG with 2,040 quality-assured queries. The three tasks share zero concept and query overlap by design, ensuring that aggregate scores reflect genuine capability breadth. Evaluation of ten systems across six paradigm families reveals severe cross-lingual failure: biomedical encoders that score 0.818 nDCG@10 in English drop to 0.056 in Japanese, a gap that English-only benchmarks cannot detect.

Chinese Translation

在临床环境中，检索增强生成（RAG）日益需要针对主要为英语的证据语料库进行多语言检索。多语言医学检索需要具备三种能力：跨语言对齐、概念区分和证据检索。然而，现有的基准测试仅孤立地评估这些能力，未能衡量生物医学专业知识与多语言覆盖之间的互动。我们引入了MMed-Bench-IR，这是一个旨在解开这三方面的基准，涵盖6种语言和三种结构异构的任务：（1）基于统一医学语言系统（UMLS）的6,127个查询的跨语言医学问答检索；（2）在三个难度层次上对4,975个混淆集进行概念区分；（3）针对RAG的多语言证据检索，包含2,040个经过质量保证的查询。这三项任务在设计上没有任何概念和查询的重叠，确保汇总得分反映真实的能力广度。对十个系统在六个范式家族中的评估揭示了严重的跨语言失败：在英语中得分为0.818 nDCG@10的生物医学编码器在日语中降至0.056，这一差距是仅基于英语的基准无法检测到的。

View on arXiv Download PDF AI Translation

cs.CL / 30 / 2606.24219

Decoherence as Defence and the Magnitude of Noise Regularisation: A Rigorous N -Qubit Theory of Stochastic Quantum Neural Networks for Adversarially Robust Network Intrusion Detection

解相干作为防御与噪声正则化的幅度：针对对抗性鲁棒网络入侵检测的严格 N 量子位随机量子神经网络理论

Filardo, Gautier-Edouard Edouard

Abstract

Stochastic quantum neural networks (SQNNs) encode neuronal activations as qubits, synaptic topology as entanglement, and neural noise through a Lindblad master equation. A recent conference study applied a ring-entangled SQNN to collaborative intrusion detection and reached three conclusions: ring entanglement is \emph{essential} for non-local anomaly detection; an adversarial-resilience bound holds but is \emph{conservative}; and the depolarising channel \emph{fails} to act as a dropout-style regulariser, behaving instead as output noise. It left open whether a per-gate stochastic deactivation (``true quantum dropout'') could regularise where the depolarising channel could not, and whether the loose robustness bound could be replaced by a predictive theory. This paper resolves both and extends the framework to real data and to neutral-atom hardware. We give an $N$-qubit formulation through the stochastic master equation and its vectorised Liouvillian, and prove a \emph{decoherence-contraction theorem}: a depolarising channel of strength $\gamma$ over $L$ entangling layers contracts every weight-$w$ Pauli read-out by a factor $(1-4\gamma/3)^{wL}$ (for the weight-$1$ read-out used here, $(1-4\gamma/3)^{L}$); building on the general noise-as-defence result of Du et al., we make this quantitative and operational for intrusion detection. On the real NSL-KDD dataset under white-box FGSM and PGD attacks, a depolarising SQNN trained with the channel is, over seven seeds under strong $\ell_\infty$/$\ell_2$ attacks, significantly more robust than the noiseless circuit ($\ell_\infty$ PGD-$20$, $p=0.04$, large effect) and, critically, never suffers the catastrophic robustness collapse that the noiseless model and gradient-trained classical detectors (which fall from $95\%$ to $47\%$) do, cutting robustness variance roughly twofold; we show this robustness arises from a noise-reshaped training boundary rather than from attack-time gradient contraction. For generalisation, we derive an adaptive-penalty formula showing that per-gate dropout implements a curvature-weighted $L_2$ penalty $\tfrac{p(1-p)}{2}\sum\theta^2\partial^2_\theta L$ in weight space, maximised at $p=1/2$, whereas depolarising noise implements an output-space penalty. A $30$-seed study confirms the formula's quantitative prediction: both mechanisms reduce the train-test gap by a small but statistically significant margin ($\approx\!0.01$; $p<10^{-4}$ and $p=0.004$), are statistically indistinguishable from each other, and the effect is concentrated where overfitting is largest; increasing the dropout rate past $1/2$ does not help, as the formula predicts. The single-seed dichotomy of prior work does not survive replication. We close with a neutral-atom realisation and a feasibility-by-$N$ analysis.

Chinese Translation

随机量子神经网络（SQNNs）将神经元激活编码为量子位，将突触拓扑编码为纠缠，并通过林布拉德主方程描述神经噪声。最近的一项会议研究将环纠缠 SQNN 应用于协作入侵检测，并得出了三个结论：环纠缠对于非局部异常检测是 extit{必要}的；存在对抗性鲁棒性界限，但该界限是 extit{保守}的；而去极化通道 extit{未能}作为一种 dropout 风格的正则化器，而是表现为输出噪声。研究未能确定每个门的随机去激活（“真正的量子 dropout”）是否能够在去极化通道无法正则化的情况下起作用，以及松散的鲁棒性界限是否可以被预测理论所替代。本文解决了这两个问题，并将框架扩展到真实数据和中性原子硬件。我们通过随机主方程及其向量化的李维尔算子给出了一个 N 量子位的公式，并证明了一个 extit{解相干收缩定理}：强度为 $eta$ 的去极化通道在 $L$ 个纠缠层上将每个权重为 $w$ 的保利读出收缩一个因子 $(1-4eta/3)^{wL}$（对于这里使用的权重为 $1$ 的读出，收缩因子为 $(1-4eta/3)^{L}$）；基于 Du 等人的一般噪声作为防御的结果，我们使其在入侵检测中变得定量和可操作。在真实的 NSL-KDD 数据集上，在白盒 FGSM 和 PGD 攻击下，经过通道训练的去极化 SQNN 在七个种子下，面对强 $ ext{l}_ ext{infty}$/$ ext{l}_2$ 攻击时，显著比无噪声电路更具鲁棒性（$ ext{l}_ ext{infty}$ PGD-$20$, $p=0.04$, 大效应），并且关键的是，从未遭遇无噪声模型和梯度训练的经典检测器（其鲁棒性从 $95 ext{%}$ 降至 $47 ext{%}$）所遭受的灾难性鲁棒性崩溃，鲁棒性方差大约减少了两倍；我们表明，这种鲁棒性源于噪声重塑的训练边界，而不是攻击时的梯度收缩。为了推广，我们推导出一个自适应惩罚公式，表明每个门的 dropout 实现了在权重空间中的曲率加权 $L_2$ 惩罚 $rac{p(1-p)}{2} extstylerac{ heta^2 ext{d}^2 L}{ ext{d} heta^2}$，在 $p=1/2$ 时达到最大，而去极化噪声则在输出空间中实现惩罚。一项 $30$ 种子的研究证实了该公式的定量预测：这两种机制都以小但统计显著的幅度（$ ext{约} 0.01$；$p<10^{-4}$ 和 $p=0.004$）减少了训练-测试差距，且在统计上不可区分，效果集中在过拟合最严重的地方；如公式预测，增加 dropout 率超过 $1/2$ 并没有帮助。之前工作的单种子二分法未能在复制中存活。我们以中性原子实现和基于 $N$ 的可行性分析结束。

View on arXiv Download PDF AI Translation

cs.CL / 31 / 2606.24259

SURGELLM: Rethinking Multi-Task Evaluation through Task-Aware Feature Gating with Class-Balanced Normalization

SURGELLM：通过任务感知特征门控与类别平衡归一化重新思考多任务评估

Mohammad, Noor Islam S., Bayazit, Ulug

Abstract

Fine-tuned encoders deployed across heterogeneous NLP tasks face three compounding problems: mismatched inductive biases, class-imbalance corruption of feature statistics, and no mechanism to condition attention on external lexical knowledge. We introduce \textbf{\surgellm}, a unified transformer framework that addresses each with a dedicated lightweight module: a \emph{surgical feature gate} (learned per-dimension sigmoid over curated lexical indicators and \texttt{[CLS]}; provably degenerates to identity when features are uninformative), \emph{task-conditioned prefix tokens} (quantized feature values and task identity prepended to every input), and \emph{Instance-Weighted Normalization} (IWN; removes class-prior bias from gate statistics). We prove an excess-risk bound linking gate benefit to \emph{surgical feature alignment}. Across four tasks, SST-2, multi-hop retrieval, LLM-prompt attribution, and authorship detection, covering 17,830 examples and eleven model variants over three seeds, the IWN variant achieves macro-F1 \textbf{0.940} ($+0.036$ over the strongest non-IWN baseline; $+0.130$ on authorship detection). A random-vocabulary control ($-0.028$ avg.\ F1) confirms gains are lexical, not parametric. Code, vocabularies, and a $99.5\%$-recovery auto-extraction recipe are released.

Chinese Translation

在异构自然语言处理（NLP）任务中部署的微调编码器面临三个复合问题：不匹配的归纳偏差、特征统计的类别不平衡损坏，以及缺乏将注意力条件化于外部词汇知识的机制。我们引入了 extbf{ extit{surgellm}}，一个统一的变换器框架，通过专门的轻量级模块解决每个问题： extit{外科特征门}（在策划的词汇指示符和 exttt{[CLS]} 上学习的逐维 sigmoid；当特征没有信息时可证明退化为恒等）， extit{任务条件前缀标记}（量化特征值和任务身份被预先附加到每个输入），以及 extit{实例加权归一化}（IWN；从门统计中去除类别先验偏差）。我们证明了一个超额风险界限，将门的益处与 extit{外科特征对齐}联系起来。在四个任务上，SST-2、多跳检索、LLM提示归因和作者身份检测，覆盖17,830个示例和三个种子的十一种模型变体，IWN变体实现了宏观F1 extbf{0.940}（比最强的非IWN基线高出$+0.036$；在作者身份检测上高出$+0.130$）。随机词汇控制（平均F1下降$-0.028$）证实了增益是词汇性的，而非参数性的。代码、词汇表和$99.5\%$恢复自动提取配方已发布。

View on arXiv Download PDF AI Translation

cs.CL / 32 / 2606.24267

Pigeonholing: Bad prompts hurt models to collapse and make mistakes

归类现象：不良提示导致模型崩溃和错误

Nam, Hyunji, Chidambaram, Keertana, Demszky, Dorottya, Jaques, Natasha

Abstract

While in-context learning is generally shown to be effective in Large Language Models (LLMs), bad contexts can cause performance degradation and mode collapse, a phenomenon we call "pigeonholing." **Unintentionally bad** contexts can happen without malicious jailbreaking intents: For example, a user asks the model to justify an incorrect math theorem or fails to correct the model's buggy code. Specifically, we investigate ``pigeonholing" in two scenarios: (1) when the user suggests a solution, and (2) when the conversation context includes the assistant's previous (incorrect) responses. Our experiments across 10 verifiable and open-ended tasks with 10 different models show that pigeonholing manifests in several ways: (1) repeating the incorrect answers from context (leading to 38-40% performance drop), (2) converging on a narrow set of answers in coding and text generation without exploring alternatives, and (3) flipping stance on controversial topics to align with the user or the assistant's previous claims. We find that pigeonholing worsens almost monotonically with the number of conversation turns (performance drops by additional 14+% as repeated mistakes increase from 1 to 5), and pigeonholing-induced mode collapse can happen even when the provided example is correct. As a step toward mitigation, we propose RLVR with synthetic errors which improves models by 43-60% under bad contexts compared to vanilla RLVR baselines.

Chinese Translation

尽管上下文学习在大型语言模型（LLMs）中通常被证明是有效的，但不良上下文可能导致性能下降和模式崩溃，这一现象我们称之为“归类现象”。**无意中的不良**上下文可能在没有恶意破解意图的情况下发生：例如，用户要求模型证明一个错误的数学定理，或未能纠正模型的错误代码。具体而言，我们在两种场景中研究“归类现象”：(1) 当用户建议解决方案时，以及 (2) 当对话上下文包含助手之前的（错误）回答时。我们在10个可验证和开放性任务中对10种不同模型进行的实验表明，归类现象以多种方式表现出来：(1) 重复上下文中的错误答案（导致性能下降38-40%），(2) 在编码和文本生成中收敛于狭窄的答案集而不探索替代方案，以及 (3) 在有争议的话题上改变立场以与用户或助手之前的主张保持一致。我们发现，归类现象几乎是单调加重的，随着对话轮次的增加（当重复错误从1增加到5时，性能下降额外14%以上），即使提供的示例是正确的，归类现象引起的模式崩溃也可能发生。作为缓解措施的一步，我们提出了带有合成错误的RLVR，与普通RLVR基线相比，在不良上下文下提高了模型性能43-60%。

View on arXiv Download PDF AI Translation

cs.CL / 33 / 2606.24281

CALIBER: Calibrating Confidence Before and After Reasoning in Language Models

CALIBER：在语言模型推理前后校准置信度

Finlay, Conor, Kurien, Joshua, Dash, Saurabh, Fadaee, Marzieh, Ermis, Beyza

Abstract

Reasoning language models are increasingly asked not only to answer difficult questions, but also to estimate their likelihood of success. Existing methods typically elicit confidence only once: either before thinking or after answering. We argue that confidence in reasoning models is state-dependent: before thinking, confidence should estimate the chance of the model correctly solving the prompt, while after thinking it should predict whether the realized answer is likely to be correct. This distinction determines the appropriate supervision target: prompt-level success should supervise confidence estimates made after seeing the prompt, while individual answer-level correctness should supervise confidence estimates made after answering. We introduce CALIBER (Calibration Before and After Reasoning), which elicits both estimates and supervises each with the target matched to its information state. Under this unified protocol, CALIBER reduces Expected Calibration Error (ECE) by 52.5% over the strongest single-confidence baseline on BigMathDigits for the 7B model, while achieving the best Brier score and AUROC, and remains within 2.1 points of the best accuracy. Further, on a larger 30B model, CALIBER achieves the best ECE on BigMathDigits while remaining competitive in Brier score and AUROC. Out of distribution, it achieves the best ECE and Brier score on GPQA and TriviaQA, and remains competitive on SimpleQA. Ablations further show that this position-target alignment is most beneficial under distribution shift where it consistently reduces calibration error across all out-of-distribution benchmarks.

Chinese Translation

推理语言模型越来越多地被要求不仅回答困难问题，还要估计其成功的可能性。现有方法通常只在一个时间点引导置信度：要么在思考之前，要么在回答之后。我们认为，推理模型的置信度是状态依赖的：在思考之前，置信度应估计模型正确解决提示的机会，而在思考之后，它应预测实现的答案是否可能是正确的。这一区别决定了适当的监督目标：提示级成功应监督在看到提示后做出的置信度估计，而单个答案级的正确性应监督在回答后做出的置信度估计。我们引入了CALIBER（推理前后校准），它同时引导这两种估计，并根据其信息状态匹配目标进行监督。在这一统一协议下，CALIBER在7B模型的BigMathDigits上将期望校准误差（ECE）降低了52.5%，同时实现了最佳的Brier分数和AUROC，并在准确性上保持在最佳值的2.1分以内。此外，在更大的30B模型上，CALIBER在BigMathDigits上实现了最佳的ECE，同时在Brier分数和AUROC上保持竞争力。在分布外，它在GPQA和TriviaQA上实现了最佳的ECE和Brier分数，并在SimpleQA上保持竞争力。消融实验进一步表明，这种位置-目标对齐在分布转移下最为有利，它在所有分布外基准上持续降低校准误差。

View on arXiv Download PDF AI Translation

cs.CL / 34 / 2606.24286

AVOC: Enhancing Hour-Level Audio-Video Understanding in Omni-Modal LLMs via Retrieval-Inspired Token Compression

AVOC：通过检索启发的令牌压缩增强全模态大语言模型的小时级音视频理解

Chen, Yijing, Tan, Wenhui, Yu, Xiaoyi, Wang, Yuyue, Cheng, Xin, Guan, Kaisi, Jiang, Hao, Li, Xiangyang, Zhu, Guojie, Song, Ruihua

Abstract

Multimodal Large Language Models have achieved remarkable progress in short-form audio-video understanding, yet long-form audio-video comprehension remains challenged by limited context windows and severe information redundancy. To address these bottlenecks, we propose AVOC, a framework for long-form audio-video understanding in Omni-modal Large Language Models. AVOC introduces a learnable token compression module between the modality encoders and the LLM backbone. We reframe multimodal token compression as a top-$K$ retrieval problem: given a fixed context budget, the module must retrieve a compact subset of tokens that best supports answering the user query. We draw inspiration from three classical Information Retrieval criteria for selecting informative units from a large candidate pool: relevance, importance, and diversity. AVOC instantiates each criterion as a tailored mechanism for audio-video understanding, and integrates them into a unified retrieval-style compression pipeline. Experiments show that AVOC achieves state-of-the-art performance on long-form audio-video benchmarks, surpassing the second-best model by 4.9 and 5.5 points in average accuracy on OmniVideoBench and LVOmniBench, respectively. Moreover, AVOC maintains robust performance on Audio-Video Needle-in-a-Haystack task at durations up to one hour.

Chinese Translation

多模态大语言模型在短时音视频理解方面取得了显著进展，但长时音视频理解仍受到有限上下文窗口和严重信息冗余的挑战。为了解决这些瓶颈，我们提出了AVOC，一个用于全模态大语言模型的长时音视频理解框架。AVOC在模态编码器与大语言模型主干之间引入了一个可学习的令牌压缩模块。我们将多模态令牌压缩重新构建为一个前$K$检索问题：在固定的上下文预算下，该模块必须检索出一个紧凑的令牌子集，以最佳支持回答用户查询。我们从三个经典的信息检索标准中获得灵感，以从大量候选池中选择信息单元：相关性、重要性和多样性。AVOC将每个标准实例化为音视频理解的定制机制，并将它们整合到一个统一的检索风格压缩管道中。实验表明，AVOC在长时音视频基准测试中实现了最先进的性能，在OmniVideoBench和LVOmniBench上分别超过第二好的模型4.9和5.5个点的平均准确率。此外，AVOC在长达一小时的音视频“干草堆中的针”任务中保持了强大的性能。

View on arXiv Download PDF AI Translation

cs.CL / 35 / 2606.24324

Prague Dependency Treebank -- Consolidated 2.0: Enriching a Complex Annotation Scheme

布拉格依存树库 -- 整合版 2.0：丰富复杂的注释方案

Mikulová, Marie, Mírovský, Jiří, Straka, Milan, Synková, Pavlína, Štěpánek, Jan, Štěpánková, Barbora, Hajič, Jan

Abstract

The Prague Dependency Treebank framework is unique in its attempt to systematically include and link different layers of language, including a meaning representation with several types of inter-sentential phenomena, especially coreference and discourse relations. We present its second consolidated version (PDT-C 2.0), which concludes almost 30-years long project of sustained development of the resource to a uniformly and coherently annotated, genre-diversified, almost 4 million token language resource of Czech language, with accompanying fully compatible lexicons. In addition to continuous linguistic research, the richly linguistically annotated corpus is also widely used in international comparisons of the development of traditional and novel NLP tools as well as in conversions into other formalisms. The corpus and the trained parsers are available under the CC BY-NC-SA licence.

Chinese Translation

布拉格依存树库框架在系统性地包含和链接不同语言层面方面独树一帜，包括与多种句间现象相关的意义表示，特别是指代和话语关系。我们呈现其第二个整合版本（PDT-C 2.0），这标志着近30年持续开发该资源的项目的结束，最终形成了一个统一且连贯的注释、体裁多样化、近400万词元的捷克语言资源，并附带完全兼容的词典。除了持续的语言学研究外，这个丰富的语言注释语料库还广泛用于国际比较传统与新型自然语言处理工具的发展，以及转化为其他形式主义。该语料库及训练好的解析器在CC BY-NC-SA许可证下提供。

View on arXiv Download PDF AI Translation

cs.CL / 36 / 2606.24331

Transformer-Based Language Models Across Domain Verticals: Architectures, Applications and Critical Assessment

基于变换器的语言模型在不同领域的应用：架构、应用与批判性评估

J, Guruprakash, B, Krithika L.

Abstract

Transformer-based language models have become the default substrate for natural language processing and the pace of new releases has made it hard for practitioners to separate durable ideas from the noise of incremental announcements. This review works at two levels. At the level of mechanism, we organise the main transformer families into a working taxonomy, covering encoder-only, decoder-only, encoder-decoder, long-context, permutation-based, and generator-discriminator variants. We then extend the discussion to post-2023 developments that changed the picture in practice: instruction tuning, reinforcement learning from human feedback, direct preference optimisation, mixture-of-experts scaling, retrieval augmentation and the current flagship model families from OpenAI, Anthropic, Google, Meta, Mistral and DeepSeek. At the level of use, we survey deployments across healthcare, finance, legal, education, customer service, creative writing and scientific work. Based on this we link each to the specific capabilities that make a transformer the appropriate tool. The contribution of this paper is a critical assessment that is based on the survey. We compare architectures on four axes that matter to deployment decisions, we quantify the trade-off between parameter count and energy cost. We also discuss how alignment methods, data provenance and benchmark saturation change what it means to call a model "state of the art". The final section lists the research questions that we think deserve more attention.

Chinese Translation

基于变换器的语言模型已成为自然语言处理的默认基础，新的发布速度使得从增量公告的噪音中分离出持久的思想变得困难。本文的评审工作分为两个层面。在机制层面，我们将主要的变换器家族组织成一个工作分类法，涵盖仅编码器、仅解码器、编码器-解码器、长上下文、基于排列和生成-判别变体。随后，我们扩展讨论到2023年后改变实践的进展：指令调优、基于人类反馈的强化学习、直接偏好优化、专家混合扩展、检索增强以及来自OpenAI、Anthropic、Google、Meta、Mistral和DeepSeek的当前旗舰模型家族。在使用层面，我们调查了在医疗、金融、法律、教育、客户服务、创意写作和科学工作等领域的部署。基于此，我们将每个应用与使变换器成为合适工具的特定能力联系起来。本文的贡献在于基于调查的批判性评估。我们在四个与部署决策相关的轴上比较架构，量化参数数量与能量成本之间的权衡。我们还讨论了对齐方法、数据来源和基准饱和如何改变将模型称为“最先进”的含义。最后一部分列出了我们认为值得更多关注的研究问题。

View on arXiv Download PDF AI Translation

cs.CL / 37 / 2606.24337

Meet UD_Czech-PDTC: A Large and Genre-Rich Treebank in Universal Dependencies

遇见 UD_Czech-PDTC：一个大型且丰富多样的通用依赖树库

Mikulová, Marie, Štěpánková, Barbora, Zeman, Daniel, Štěpánek, Jan, Straka, Milan, Hajič, Jan

Abstract

Czech has been part of Universal Dependencies since its first release in 2015. It has also been one of the best represented languages, with the Prague Dependency Treebank being order of magnitude larger than most other UD treebanks. More recently, three other datasets from the Prague family were added and the annotations thoroughly revisited, forming the "Prague Dependency Treebank-Consolidated" (PDT-C). In comparison to the original PDT, PDT-C is more than twice as large, but it is also much more diverse in terms of genres and domains. In this paper, we describe the conversion of the new resource to Universal Dependencies. While the two annotation schemes are relatively similar at the first sight, there are numerous small differences in topology of the dependency structures and in granularity of the POS and relation type inventories. We demonstrate a selection of such differences on examples, discuss the diverging motivations, as well as ways to overcome the differences during conversion. We argue that while PDT is less "universal" and more tightly bound to one language, its multi-layer annotation is rich and provides all information needed for basic UD trees, and much more.

Chinese Translation

捷克语自2015年首次发布以来便成为通用依赖（Universal Dependencies）的一部分。它也是表现最好的语言之一，布拉格依赖树库（Prague Dependency Treebank）的规模比大多数其他UD树库大几个数量级。最近，又添加了来自布拉格家族的三个数据集，并对注释进行了彻底的重新审视，形成了“布拉格依赖树库-整合版”（Prague Dependency Treebank-Consolidated，简称PDT-C）。与原始的PDT相比，PDT-C的规模大于两倍，同时在体裁和领域上也更加多样化。在本文中，我们描述了将这一新资源转换为通用依赖的过程。虽然这两种注释方案在初看时相对相似，但在依赖结构的拓扑和词性（POS）及关系类型清单的细粒度上存在许多小差异。我们通过示例展示了这些差异，并讨论了不同的动机以及在转换过程中克服这些差异的方法。我们认为，尽管PDT的“通用性”较低，更加紧密地绑定于一种语言，但其多层次的注释丰富，提供了构建基本UD树所需的所有信息，以及更多的信息。

View on arXiv Download PDF AI Translation

cs.CL / 38 / 2606.24359

Automatic Part-of-Speech Tagging of Arabic-English Dictionary Senses through WordNet

通过WordNet对阿拉伯语-英语词典词义进行自动词性标注

Fayed, Diaa M., Fahmy, Aly A., Rashwan, Mohsen A., Fayed, Wafaa K.

Abstract

This paper proposed an algorithm for part-of-speech (POS) tagging senses of a bilingual dictionary. The algorithm is applied on the Al-Mawrid Arabic-English dictionary. The tagging task is accomplished by transferring the POS tags of the English translation equivalences (TEs) to the dictionary senses after dis-ambiguities process. The English POS tags of senses are acquired from the Princeton WordNet. POS tagging of bilingual dictionary senses is prerequisite to link a bilingual dictionary to WordNet and/or standardizing that dictionary into WordNet-LMF format where the synset (set of synonyms), not word, is the basic brick. The registered accuracy is high though the cost is little. Building NLP/HLT tools needs linguistic experts, large investments, and long time. For statistical approach, we need large annotated corpora and for rule-based approach, we need large lexicon that contains rich linguistic and world knowledge. That motivates the appearance of what are called resource-light approaches to develop natural language processing (NLP) tools for poor-resource languages.

Chinese Translation

本文提出了一种对双语词典词义进行词性（POS）标注的算法。该算法应用于Al-Mawrid阿拉伯语-英语词典。标注任务通过在消歧义处理后将英语翻译等价词（TEs）的词性标签转移到词典词义上来完成。词义的英语词性标签来自普林斯顿WordNet。双语词典词义的词性标注是将双语词典与WordNet链接和/或将该词典标准化为WordNet-LMF格式的前提，其中同义词集（synset）而非单词是基本构建块。尽管成本较低，但注册的准确性很高。构建自然语言处理（NLP）/人类语言技术（HLT）工具需要语言学专家、大量投资和较长时间。对于统计方法，我们需要大量标注的语料库；而对于基于规则的方法，我们需要包含丰富语言和世界知识的大型词典。这促使了所谓资源轻量方法的出现，以开发适用于资源匮乏语言的自然语言处理（NLP）工具。

View on arXiv Download PDF AI Translation

cs.CL / 39 / 2606.24366

MorfFlex: Handling Rich Morphology

MorfFlex：处理丰富的形态学

Hlaváčová, Jaroslava, Mikulová, Marie, Štěpánková, Barbora, Straka, Milan, Hajič, Jan

Abstract

We present MorfFlex, a morphological dictionary architecture suitable for languages with extensive regularity in both inflection and derivation. As the primary example of MorfFlex in use we introduce MorfFlex CZ, a morphological dictionary of Czech. It is distributed as a simple, unstructured list of triplets, however, its manually maintained, unpublished source files and conversion scripts encode a sophisticated system of inflectional and derivational patterns. These patterns dramatically reduce the otherwise enormous size of the dictionary, which currently contains over 100 million wordforms and more than 1 million lemmas. The MorfFlex CZ dictionary serves as an essential resource for ensuring the consistency of manual morphological annotation in the Prague Dependency Treebanks and underpins state-of-the-art automatic tools such as MorphoDiTa. In this paper, we focus on: (i) presenting an effective method for managing the rich morphological system within the dictionary, and (ii) demonstrating the utility of such a language resource for maintaining annotation consistency in corpora and supporting the development of advanced NLP applications.

Chinese Translation

我们提出了MorfFlex，一种适用于具有广泛屈折和派生规律的语言的形态学词典架构。作为MorfFlex的主要应用示例，我们介绍了MorfFlex CZ，这是一个捷克语的形态学词典。它以简单的、非结构化的三元组列表的形式分发，然而，其手动维护的未发布源文件和转换脚本编码了一个复杂的屈折和派生模式系统。这些模式显著减少了词典的庞大规模，目前词典包含超过1亿个词形和超过100万个词元。MorfFlex CZ词典作为确保布拉格依赖树库中手动形态标注一致性的基本资源，并支持诸如MorphoDiTa等最先进的自动工具。在本文中，我们重点讨论：(i) 提出一种有效的方法来管理词典中的丰富形态系统，以及 (ii) 展示这种语言资源在维护语料库标注一致性和支持先进自然语言处理应用开发中的实用性。

View on arXiv Download PDF AI Translation

cs.CL / 40 / 2606.24381

On the Stability of Prompt Ranking in Large Language Model Evaluation

大型语言模型评估中提示排名的稳定性研究

Du, Shaoshuai, Liang, Penghao, Shen, Yixian, Shi, Chuanqi, Zhang, Hang, Wang, Lun

Abstract

Prompt-based interaction has become a dominant paradigm for using large language models (LLMs), where multiple candidate prompts are evaluated and the top-ranked one is selected for downstream use. This workflow implicitly assumes that prompt rankings are stable under minor variations in evaluation conditions. In this paper, we systematically study prompt ranking stability under common sources of variability, including random seeds and limited evaluation subsets. Across three open-weight LLMs and two benchmark tasks, we find that while overall rank correlations are often moderate to high, the identity of the top-performing prompt frequently changes, leading to unreliable selection decisions. To address this issue, we propose a simple stability-aware selection strategy based on a lower confidence bound, which accounts for both performance and variance. Our results show that this approach improves robustness in unstable settings while remaining competitive in more stable regimes. These findings highlight the importance of accounting for evaluation uncertainty in prompt selection and LLM benchmarking.

Chinese Translation

基于提示的交互已成为使用大型语言模型（LLMs）的主导范式，其中多个候选提示被评估，并选择排名最高的一个用于下游任务。该工作流程隐含地假设在评估条件的轻微变化下，提示排名是稳定的。本文系统地研究了在常见变异来源下的提示排名稳定性，包括随机种子和有限的评估子集。在三个开放权重的LLM和两个基准任务中，我们发现尽管整体排名相关性通常为中等到高，但表现最佳的提示的身份经常变化，导致选择决策的不可靠性。为了解决这个问题，我们提出了一种基于下置信界的简单稳定性感知选择策略，该策略同时考虑了性能和方差。我们的结果表明，这种方法在不稳定环境中提高了鲁棒性，同时在更稳定的环境中保持竞争力。这些发现突显了在提示选择和LLM基准测试中考虑评估不确定性的重要性。

View on arXiv Download PDF AI Translation

cs.CL / 41 / 2606.24387

AutoSpecNER: A Fine-Grained Named Entity Recognition Dataset for Vehicle Specification Extraction

AutoSpecNER：用于车辆规格提取的细粒度命名实体识别数据集

Lee, Jordan, Ventirozos, Filippos, Abdullahm, Abdirahman, Nteka, Ioanna, Appleby, Peter, Shardlow, Matthew

Abstract

Vehicle advertisements contain rich specification information, but automotive NER resources remain limited. We introduce AutoSpecNER, an expert-annotated dataset for fine-grained entity recognition in vehicle listings. The dataset includes 659 advertisements from a popular car-selling website, with over 10,000 entities annotated across 15 categories, including MODEL, ENGINE_SPEC, and BATTERY_CAPACITY. Annotation quality was validated through inter-annotator agreement, achieving an average score of 91.5%. We benchmark rule-based extraction, fine-tuned transformer encoders, and large language models. DeBERTa achieves the best performance with a 90% micro-F1 score, outperforming the rule-based baseline (43%) and the strongest large language model (77.8%).

Chinese Translation

车辆广告包含丰富的规格信息，但汽车命名实体识别（NER）资源仍然有限。我们介绍了AutoSpecNER，这是一个经过专家注释的用于车辆列表中细粒度实体识别的数据集。该数据集包括来自一个热门汽车销售网站的659条广告，涵盖15个类别的超过10,000个实体，包括模型（MODEL）、发动机规格（ENGINE_SPEC）和电池容量（BATTERY_CAPACITY）。通过注释者间一致性验证了注释质量，平均得分达到91.5%。我们对基于规则的提取、微调的变换器编码器和大型语言模型进行了基准测试。DeBERTa在微F1得分上表现最佳，达到了90%，超越了基于规则的基线（43%）和最强的大型语言模型（77.8%）。

View on arXiv Download PDF AI Translation

cs.CL / 42 / 2606.24420

Beyond Logprobs: A Multi-Signal Confidence Engine for LLM-Based Document Field Extraction

超越对数概率：基于大型语言模型的文档字段提取多信号置信引擎

Kumar, Nitesh

Abstract

In high-stakes document processing pipelines, including financial reconciliation, compliance verification, and procurement automation, an LLM extraction that is silently wrong is more dangerous than one that is visibly absent. The central challenge is not extraction accuracy alone but reliable confidence estimation: knowing, field by field, whether an extraction can be trusted for automation or deferred to human review. Token-level log-probabilities, verbalized confidence, and multi-sample self-consistency all collapse toward all-positive behaviour at practical thresholds, offering no reliable separation between trustworthy and untrustworthy extractions. We present ExtractConf, a cross-domain, field-agnostic confidence engine that grounds confidence estimation in two structurally different readings of the same document. A field-guided Hunter call extracts each field under schema-slot completion pressure; a document-guided Mapper call scans holistically and surfaces values grounded in document content. This asymmetry yields different failure modes: Hunter hallucinates values for absent fields, while Mapper misses visually non-salient ones. Their disagreement is independently informative. ExtractConf fuses cross-call disagreement, LLM-internal uncertainty, OCR, image quality, and spatial layout into a classifier requiring no domain-specific rules or retraining. On DocILE (55-field invoices, 26% failure rate), it achieves 0.928 ROC AUC and reduces selective prediction risk by 70% over logprob-mean. At 80% coverage, accuracy reaches 99.1%, enabling a practical human-in-the-loop workflow. Zero-shot transfer to CORD receipts achieves 0.858 AUC; lightweight Lasso recalibration reduces ECE by 89% and Brier by 43%, confirming the signals generalise across document domains.

Chinese Translation

在高风险的文档处理流程中，包括财务对账、合规验证和采购自动化，静默错误的LLM提取比明显缺失的提取更具危险性。核心挑战不仅在于提取的准确性，还在于可靠的置信度估计：逐字段地了解提取是否可以信任以进行自动化，或是否需要推迟到人工审核。令牌级的对数概率、口头化的置信度和多样本自一致性在实际阈值下都趋向于全正行为，无法提供可靠的可信与不可信提取之间的区分。我们提出了ExtractConf，这是一个跨领域、字段无关的置信引擎，其置信度估计基于同一文档的两种结构不同的解读。字段引导的Hunter调用在模式槽完成压力下提取每个字段；文档引导的Mapper调用则整体扫描并提取基于文档内容的值。这种不对称性产生了不同的失败模式：Hunter为缺失字段幻觉出值，而Mapper则错过视觉上不显著的字段。它们的分歧提供了独立的信息。ExtractConf融合了跨调用的分歧、LLM内部的不确定性、OCR、图像质量和空间布局，构建了一个不需要领域特定规则或重新训练的分类器。在DocILE（55个字段的发票，26%的失败率）上，它达到了0.928的ROC AUC，并将选择性预测风险降低了70%相较于对数概率均值。在80%的覆盖率下，准确率达到了99.1%，实现了实用的人机协作工作流程。零样本迁移到CORD收据达到了0.858的AUC；轻量级的Lasso重新校准将ECE降低了89%，Brier降低了43%，确认了信号在文档领域中的普适性。

View on arXiv Download PDF AI Translation

cs.CL / 43 / 2606.24428

Escaping the Self-Confirmation Trap: An Execute-Distill-Verify Paradigm for Agentic Experience Learning

摆脱自我确认陷阱：一种执行-提炼-验证的代理经验学习范式

Zhu, Shiding, Qi, Yudi, Wang, Yajie, Li, Jiaze, Song, Chao, Shi, Yaorui, Miao, Yibo, Gao, Hanqi, Zhang, Kai

Abstract

Experience-driven self-evolution is critical for large language model (LLM) agents to improve through open-world interaction. However, existing experience learning methods mostly rely on single-agent loops, where the same agent executes tasks, summarizes outcomes, and determines memory content. This setup makes agents vulnerable to the Self-Confirmation Trap: wrong-but-self-consistent trajectories are misidentified as successful experience, leading to cumulative errors during retrieval and reuse. To address this issue, we propose EDV, an Execute-Distill-Verify framework for reliable experience learning. In the Execute stage, multiple heterogeneous agents explore the same task space in parallel to generate diverse candidate trajectories. In the Distill stage, a dedicated third-party agent comparatively analyzes these trajectories to produce candidate experiences, reducing executor-centric summarization bias. In the Verify stage, the execution group validates candidates via a consensus mechanism, and only approved experiences are written into shared or private memory. By decoupling the three stages, EDV transforms experience learning from isolated self-reflection into collaborative construction, filtering erroneous and noisy content before memory insertion. We evaluate EDV on three challenging long-horizon benchmarks: tau2-bench, Mind2Web and MMTB. Results show EDV consistently outperforms strong baselines, validating that reliable experience construction is essential for robust agent self-evolution. Our code is available at https://github.com/shidingz/EDV.

Chinese Translation

经验驱动的自我进化对于大型语言模型（LLM）代理通过开放世界互动进行改进至关重要。然而，现有的经验学习方法大多依赖于单一代理循环，其中同一代理执行任务、总结结果并确定记忆内容。这种设置使得代理容易陷入自我确认陷阱：错误但自洽的轨迹被误认为是成功的经验，导致在检索和重用过程中累积错误。为了解决这个问题，我们提出了EDV，一个用于可靠经验学习的执行-提炼-验证框架。在执行阶段，多个异构代理并行探索相同的任务空间，以生成多样化的候选轨迹。在提炼阶段，一个专门的第三方代理对这些轨迹进行比较分析，以产生候选经验，从而减少以执行者为中心的总结偏差。在验证阶段，执行组通过共识机制验证候选者，只有获得批准的经验才会写入共享或私有记忆。通过解耦这三个阶段，EDV将经验学习从孤立的自我反思转变为协作构建，在记忆插入之前过滤错误和噪声内容。我们在三个具有挑战性的长时间基准上评估EDV：tau2-bench、Mind2Web和MMTB。结果表明，EDV在强基线之上始终表现优越，验证了可靠的经验构建对于强健的代理自我进化至关重要。我们的代码可在 https://github.com/shidingz/EDV 获得。

View on arXiv Download PDF AI Translation

cs.CL / 44 / 2606.24460

The African Language Tax: Quantifying the Cost, Latency, and Context Penalty of Tokenizing African Languages in Frontier LLMs

非洲语言税：量化在前沿大型语言模型中对非洲语言进行分词的成本、延迟和上下文惩罚

Somide, Olaoye Anthony

Abstract

Commercial large language models bill, scale latency, and budget context per token. Yet tokenizers assign more subword tokens to the same meaning in some languages than in others, so speakers of languages with high token-fertility pay a structural penalty before a model is ever invoked. This penalty is documented for multilingual settings in general, but it has not been measured systematically for African languages at the level of enterprise deployment economics and cognitive context capacity. We measure it across 20 African languages spanning five language families and three scripts (Latin, Ge'ez/Ethiopic, N'Ko; 19 appear in the primary FLORES-200+ corpus, with Nigerian Pidgin measured via MAFAND-MT only), using parallel corpora so that the language effect is isolated from content. Across 11 frontier and open tokenizers on FLORES-200+, every African language carries a tokenization premium above English (median 1.88x on GPT-5 / o200k_base, up to 8.92x for N'Ko); the penalty is largest for Ethiopic and N'Ko scripts (reaching 7-9x) and is near-invariant across corpora (FLORES vs SIB-200 Pearson r = 0.9998). Translated into deployment terms, this results in up to 8.9x inference cost and an equivalent generation-latency multiplier (N'Ko vs English on GPT-5; 7.4x for Amharic), and as little as 11% of English's effective context window. The best currently available tokenizer for African languages, Gemma 4, reduces the mean premium from 3.31x (cl100k_base) to 2.38x, but no tokenizer eliminates the penalty. We release an open measurement tool (afri-fertility), a public leaderboard, a results dataset, and mitigation guidance for African builders. The penalty falls hardest on the languages whose speakers can least afford it, a digital divide encoded directly into the subword vocabulary.

Chinese Translation

商业大型语言模型按每个令牌计费、扩展延迟和预算上下文。然而，在某些语言中，分词器为相同的意义分配了更多的子词令牌，因此高令牌丰度语言的使用者在模型被调用之前就支付了结构性惩罚。这种惩罚在多语言环境中普遍存在，但在企业部署经济学和认知上下文能力层面上尚未系统测量非洲语言的情况。我们在涵盖五个语言家族和三种书写系统（拉丁文、吉兹/埃塞俄比亚文、N'Ko）的20种非洲语言中进行了测量（19种出现在主要的FLORES-200+语料库中，尼日利亚皮钦语仅通过MAFAND-MT测量），使用平行语料库以便将语言效应与内容隔离。在FLORES-200+上的11种前沿和开放分词器中，每种非洲语言的分词溢价均高于英语（在GPT-5 / o200k_base上中位数为1.88倍，N'Ko最高可达8.92倍）；对于埃塞俄比亚文和N'Ko书写系统，惩罚最大（达到7-9倍），并且在语料库之间几乎不变（FLORES与SIB-200的Pearson相关系数r = 0.9998）。换算成部署术语，这导致推理成本高达8.9倍，以及相应的生成延迟倍增器（N'Ko与英语在GPT-5上的比较；阿姆哈拉语为7.4倍），有效上下文窗口仅为英语的11%。目前可用的最佳非洲语言分词器Gemma 4将平均溢价从3.31倍（cl100k_base）降低到2.38倍，但没有任何分词器能够消除惩罚。我们发布了一个开放测量工具（afri-fertility）、一个公共排行榜、一个结果数据集以及针对非洲开发者的缓解指导。这种惩罚对那些最无法承受的语言的使用者影响最大，形成了直接编码到子词词汇中的数字鸿沟。

View on arXiv Download PDF AI Translation

cs.CL / 45 / 2606.24501

UOL@IDEM at BEA 2026 Shared Task 1: Neural Fusion and Feature-Rich Modeling for L1-Aware Vocabulary Difficulty Prediction

UOL@IDEM在BEA 2026共享任务1中的表现：基于神经融合和特征丰富建模的L1感知词汇难度预测

Khallaf, Nouran, Sharoff, Serge

Abstract

This paper describes UOL@IDEM's closed-track submission to the BEA 2026 shared task on L1-aware vocabulary difficulty prediction. We model the task as regression and train separate systems for Spanish, German, and Mandarin Chinese\footnote{Below we use \emph{Chinese} for brevity.}. Our system combines multilingual contextual representations with engineered features capturing frequency, surface form, retrieval evidence, semantic alignment, cognate similarity, and masked-language-model predictability. Development results show consistent gains over the official closed-track baselines, with sentence-embedding encoders such as BGE-M3, multilingual E5, and LaBSE performing best. Official submissions achieve RMSE scores of 1.132, 1.037, and 0.891 for Spanish, German, and Chinese, respectively. Feature analysis identifies frequency as the most stable predictor, while contextual predictability, form similarity, retrieval, and semantic features provide complementary L1-sensitive signals. Error analysis shows strong ranking performance but weaker calibration for the easiest items, which are often overpredicted. See https://github.com/Nouran-Khallaf/UoL-IDEM-BEA2026-Vocabulary-Difficulty-Prediction

Chinese Translation

本文描述了UOL@IDEM在BEA 2026共享任务中关于L1感知词汇难度预测的闭合轨道提交。我们将该任务建模为回归问题，并为西班牙语、德语和普通话（为简便起见，以下我们使用“中文”）训练了独立的系统。我们的系统结合了多语言上下文表示与工程特征，这些特征捕捉了频率、表面形式、检索证据、语义对齐、同源词相似性以及掩码语言模型的可预测性。开发结果显示，相较于官方闭合轨道基线，我们的系统在性能上有一致的提升，其中句子嵌入编码器如BGE-M3、多语言E5和LaBSE表现最佳。官方提交的RMSE得分分别为西班牙语1.132、德语1.037和中文0.891。特征分析表明，频率是最稳定的预测因子，而上下文可预测性、形式相似性、检索和语义特征则提供了互补的L1敏感信号。错误分析显示排名性能强劲，但对于最简单的项目校准较弱，通常存在过度预测的情况。请参见 https://github.com/Nouran-Khallaf/UoL-IDEM-BEA2026-Vocabulary-Difficulty-Prediction

View on arXiv Download PDF AI Translation

cs.CL / 46 / 2606.24523

Poster: Exploring the Limits of Audio-Based Detection of Turkish Phone Call Scams

海报：探索基于音频的土耳其电话诈骗检测的极限

Eren, Arda, Cheung, Micheal, Zhang, Youqian, Ngai, Grace, Fu, Eugene Yujun

Abstract

Scam phone calls exploit vulnerable communities worldwide, yet research on detection has focused almost exclusively on English and other high-resource languages. In low-resource settings such as Turkish, detection is especially difficult, as annotated data is scarce and technological defenses remain limited. This research investigates how large language models (LLMs) can support scam detection in Turkish by introducing the first public multi-modal dataset of 100 aligned audio-transcript pairs of scam and benign conversations. We evaluate seven LLMs spanning three model families: Gemini 2.5 (Flash, Flash-Lite, Pro), GPT-4o, and Qwen (Max, Plus, Turbo), under three input conditions: raw audio, automatic speech-to-text transcripts, and transcripts refined by a native speaker. Our results suggest that transcript-based inputs consistently outperform direct audio processing, while human-corrected and uncorrected transcripts perform comparably. By centering a low-resource language and real world threat, this work highlights the urgent need for culturally and linguistically inclusive AI safety research and more robust multi-modal systems for fraud prevention.

Chinese Translation

诈骗电话在全球范围内利用脆弱社区，但关于检测的研究几乎完全集中在英语和其他高资源语言上。在像土耳其这样的低资源环境中，检测尤其困难，因为标注数据稀缺，技术防御手段仍然有限。本研究探讨了大型语言模型（LLMs）如何支持土耳其的诈骗检测，通过引入首个公共的100对诈骗和良性对话的音频-文本对齐多模态数据集。我们评估了七个LLM，涵盖三种模型家族：Gemini 2.5（Flash, Flash-Lite, Pro）、GPT-4o和Qwen（Max, Plus, Turbo），在三种输入条件下进行测试：原始音频、自动语音转文本的转录本，以及由母语者修正的转录本。我们的结果表明，基于转录本的输入在性能上始终优于直接音频处理，而人类修正和未修正的转录本表现相当。通过聚焦于低资源语言和现实世界威胁，本研究突显了对文化和语言包容的人工智能安全研究以及更强大的多模态诈骗预防系统的迫切需求。

View on arXiv Download PDF AI Translation

cs.CL / 47 / 2606.24526

AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning

AGORA：一个基于档案的代理工作场所文档推理基准

Guo, Honglin, Zhang, Qi, Zhang, Yu, Li, Weijie, Zheng, Rui, Lei, Zhikai, Peng, Qiyuan, Xi, Zhiheng, Gui, Tao, Zhang, Qi

Abstract

Large language models are increasingly deployed as agents that reason over documents rather than answer from parametric knowledge. We study archive-grounded reasoning: locating sparse evidence across a large, messy collection of workplace files, reconciling inconsistent terminology, units, and time conventions, and computing an answer. Existing benchmarks address only parts of this setting and none jointly stresses archive-groundedness, agentic exploration, and cross-domain coverage. We introduce Agora, a benchmark pairing 362 questions with eight domain collections of 9,664 authentic documents and 372M tokens, far exceeding any model's context window, so agents must explore deliberately rather than scan exhaustively. Agora is built by an agentic pipeline combining cross-document task synthesis, leakage-preventing obfuscation, and difficulty filtering. Evaluating eight models, we find the task far from solved: even the strongest reaches only 59.4% accuracy, with notable variation across domains.

Chinese Translation

大型语言模型越来越多地被部署为能够对文档进行推理的代理，而不是仅仅依赖参数知识进行回答。我们研究基于档案的推理：在大量杂乱的工作文件集合中定位稀疏证据，调和不一致的术语、单位和时间惯例，并计算出答案。现有基准仅涉及这一设置的部分内容，且没有一个同时强调基于档案的特性、代理探索和跨领域覆盖。我们引入了Agora，一个基准将362个问题与八个领域集合的9,664份真实文档和372M个标记配对，远超任何模型的上下文窗口，因此代理必须进行有意识的探索，而不是全面扫描。Agora是通过一个代理管道构建的，该管道结合了跨文档任务合成、防泄漏的模糊处理和难度过滤。评估八个模型后，我们发现这一任务远未解决：即使是最强的模型，其准确率也仅达到59.4%，并且在不同领域之间存在显著差异。

View on arXiv Download PDF AI Translation

cs.CL / 48 / 2606.24530

NatureBench: Can Coding Agents Match the Published SOTA of Nature-Family Papers?

NatureBench：编码智能体能否匹配Nature系列论文的已发布SOTA？

Wang, Yuru, Cheng, Lejun, Zuo, Yuxin, Zeng, Sihang, He, Bingxiang, Jiang, Che, Yang, Junlin, Wang, Yuchong, Zhao, Kaikai, Huang, Weifeng, Tian, Kai, Yuan, Zhenzhao, Zhong, Jincheng, Wang, Weizhi, Ding, Ning, Zhou, Bowen, Zhang, Kaiyan

Abstract

We introduce NatureBench, a cross-discipline benchmark of 90 tasks distilled from peer-reviewed Nature-family publications, designed to evaluate whether AI coding agents can move beyond reproduction toward discovery on real scientific problems. NatureBench is built on NatureGym, an automated pipeline that constructs a standardized, per-task containerized environment from a source paper, addressing the environment-fragmentation problem that has limited the credibility of prior agent-on-research benchmarks. Evaluating ten frontier agent configurations under a strict web-search-disabled protocol, we find that the strongest model surpasses SOTA on only 17.8% of tasks under the g>0.1 criterion. Analysis of method pathways reveals that agents succeed primarily through methodological translation, converting scientific tasks into familiar supervised prediction problems, rather than through genuine scientific invention. Failures are dominated by wrong method choice and insufficient compute budget, not by task misunderstanding. We release the benchmark, the NatureGym pipeline, and a public leaderboard with maintainer-side reproduction. Code: https://github.com/FrontisAI/NatureBench

Chinese Translation

我们介绍了NatureBench，这是一个跨学科的基准测试，包含90个任务，提取自经过同行评审的Nature系列出版物，旨在评估AI编码智能体是否能够超越简单的复制，朝着在真实科学问题上的发现迈进。NatureBench建立在NatureGym之上，这是一个自动化管道，从源论文构建标准化的每个任务容器化环境，解决了环境碎片化问题，这一问题限制了先前研究基准中智能体的可信度。在严格禁用网络搜索的协议下评估十种前沿智能体配置，我们发现最强模型在g>0.1标准下仅在17.8%的任务中超越了SOTA。方法路径的分析表明，智能体的成功主要通过方法论转换实现，将科学任务转化为熟悉的监督预测问题，而非真正的科学发明。失败主要由错误的方法选择和计算预算不足所主导，而非任务理解不足。我们发布了基准测试、NatureGym管道以及一个维护方可重现的公共排行榜。代码链接：https://github.com/FrontisAI/NatureBench

View on arXiv Download PDF AI Translation

cs.CL / 49 / 2606.24579

Cross-Lingual Exploration for Parametric Knowledge

参数知识的跨语言探索

Diskind, Elisha, Trainin, Itamar, Shaham, Uri, Choshen, Leshem, Szpektor, Idan, Abend, Omri

Abstract

Parametric knowledge in Large Language Models is not equally accessible across languages. As a result, standard inference techniques often struggle to surface localized facts, leading to failures in cross-lingual knowledge transfer and consistency. In this work, we investigate techniques for accessing hidden factual knowledge by exploring cross-lingual prompting strategies. We identify four inherent dimensions of cross-lingual exploration that directly govern parametric knowledge retrieval and evaluate them on multilingual factual benchmarks covering 17 typologically diverse languages. Our results demonstrate that cross-lingual exploration significantly improves knowledge transfer and factual recall, representing a more efficient compute Pareto frontier than native-language scaling. Furthermore, we observe corresponding improvements in cross-lingual consistency, exceeding what can be explained by accuracy gains alone. Overall, our work establishes multilingual prompt exploration as a highly effective inference-time strategy for unlocking latent parametric knowledge.

Chinese Translation

大型语言模型中的参数知识在不同语言之间的可获取性并不均衡。因此，标准推理技术常常难以揭示本地化事实，导致跨语言知识转移和一致性方面的失败。在本研究中，我们通过探索跨语言提示策略，研究访问隐藏事实知识的技术。我们识别出四个跨语言探索的固有维度，这些维度直接影响参数知识的检索，并在涵盖17种类型多样语言的多语言事实基准上对其进行评估。我们的结果表明，跨语言探索显著提高了知识转移和事实回忆，代表了一种比母语扩展更高效的计算帕累托前沿。此外，我们观察到跨语言一致性也有相应的改善，超出了仅由准确性提升所能解释的范围。总体而言，我们的研究确立了多语言提示探索作为一种高效的推理时策略，以解锁潜在的参数知识。

View on arXiv Download PDF AI Translation

cs.CL / 50 / 2606.24595

MEMPROBE: Probing Long-Term Agent Memory via Hidden User-State Recovery

MEMPROBE：通过隐藏用户状态恢复探测长期智能体记忆

Ma, Enze, Zhou, Yufan, Huang, Wei-Chieh, Yang, Jie, Ma, Huanhuan, Wang, Zixuan, Li, Chengze, Miao, Chunyu, Yu, Philip S., Wang, Zhen

Abstract

Long-term memory promises LLM agents that grow more capable across sessions, maintaining an accurate, evolving understanding of the user that interaction forms. In practice, however, this memory is evaluated mostly through downstream behavior, such as later answers, personalization quality, or task success, which tests that understanding only indirectly and leaves the memory artifact itself largely unaudited. We argue that long-term memory should instead be evaluated as an auditable post-interaction artifact: after ordinary assistance, what structured user state can be reconstructed from the memory the agent leaves behind? We instantiate this view in MEMPROBE, a benchmark in which a memory-equipped agent assists simulated users, each carrying a hidden, taxonomy-anchored user-state bank, across a trajectory of leak-controlled tasks, after which that bank is reconstructed from the agent's resulting memory under both full-store and top-k access. Built on synthetic ground truth for efficient, scalable measurement, MEMPROBE spans 50 simulated users with 31 hidden dimensions each (1,550 recovery targets) and tests 5 representative memory systems. Testing state-of-the-art memory agents, we find that successful assistance and recoverable memory behave as distinct capabilities. Task completion nearly saturates, even for a memoryless baseline, while category-balanced recovery stays moderate (about 0.6) and drops further under top-k retrieval. MEMPROBE is the first benchmark to study memory recovery directly, reconstructing the user state a system retains and scoring it against ground truth. We see recovery as a concrete objective for future memory agents to optimize, and MEMPROBE as a step toward an environment where agents are trained to remember their users, growing more faithful the longer they know them.

Chinese Translation

长期记忆为大型语言模型（LLM）智能体提供了在多个会话中不断增强能力的可能，能够维持对用户的准确、不断演变的理解。然而，在实践中，这种记忆主要通过下游行为进行评估，例如后续回答、个性化质量或任务成功率，这仅间接测试了这种理解，并且使得记忆本身在很大程度上未经过审计。我们认为，长期记忆应该作为一种可审计的后交互产物进行评估：在普通的辅助之后，能够从智能体留下的记忆中重构出什么样的结构化用户状态？我们在MEMPROBE中实现了这一观点，这是一个基准测试，其中配备记忆的智能体协助模拟用户，每个用户都携带一个隐藏的、基于分类法的用户状态库，经过一系列控制泄漏的任务后，从智能体的记忆中重构该状态库，支持完全存储和前k个访问。MEMPROBE基于合成真实数据，以实现高效、可扩展的测量，涵盖50个模拟用户，每个用户具有31个隐藏维度（共1,550个恢复目标），并测试5个代表性的记忆系统。在对最先进的记忆智能体进行测试时，我们发现成功的辅助和可恢复的记忆表现出不同的能力。任务完成率几乎饱和，即使对于没有记忆的基线，类别平衡的恢复保持在中等水平（约0.6），并在前k个检索下进一步下降。MEMPROBE是第一个直接研究记忆恢复的基准，重构系统保留的用户状态，并与真实数据进行评分。我们将恢复视为未来记忆智能体优化的具体目标，而MEMPROBE则是朝着一个环境迈进的第一步，在这个环境中，智能体被训练以记住他们的用户，随着对用户了解的加深而变得更加忠实。

View on arXiv Download PDF AI Translation

cs.CL / 51 / 2606.24596

To Compare, or Not to Compare: On Methodological Practices in Evaluating Social Bias

比较，还是不比较：关于评估社会偏见的方法实践

Marcuzzi, Federico, Ning, Xuefei, Schwartz, Roy, Gurevych, Iryna

Abstract

As Large Language Models are increasingly deployed in critical applications, robustly evaluating their social biases is paramount. However, the current literature suffers from widespread methodological fragmentation, which yields contradictory conclusions. This stems largely from ignoring the structural framing of benchmark-level evaluations. To resolve this, we introduce a unified and controllable framework that standardizes heterogeneous benchmarks to systematically contrast isolated demographic assessments with forced-choice comparative settings. Crucially, this allows us to disentangle the confounding effects of Chain-of-Thought reasoning, neutral fallback options, and other structural artifacts in social bias evaluations. Our evaluation across multiple model families reveals a massive, systematic paradigm gap: while isolated assessments limit prejudice activation, comparative settings act as aggressive catalysts for latent discrimination, a shift primarily driven by underspecified contexts. Alarmingly, CoT reasoning exacerbates social biases under comparative settings, and this systemic bias persists as a deterministic prejudice even when models are provided neutral fallback options or claim to answer randomly. Finally, we demonstrate that this comparative prejudice is a generalized phenomenon that scales positively with model size. Ultimately, we offer a crucial methodological guideline: while researchers must leverage comparative settings to robustly audit hidden biases, practitioners cannot safely rely on comparative deployments in ambiguous real-world tasks.

Chinese Translation

随着大型语言模型在关键应用中的日益广泛部署，稳健评估其社会偏见显得尤为重要。然而，当前文献存在广泛的方法论碎片化问题，导致得出相互矛盾的结论。这在很大程度上源于忽视基准级评估的结构性框架。为了解决这一问题，我们引入了一个统一且可控的框架，标准化异质基准，以系统地对比孤立的人口统计评估与强制选择比较环境。至关重要的是，这使我们能够理清思维链（Chain-of-Thought）推理、中立后备选项及其他结构性伪影在社会偏见评估中的混淆效应。我们对多个模型家族的评估揭示了一个巨大的、系统性的范式差距：虽然孤立评估限制了偏见的激活，但比较环境则作为潜在歧视的强烈催化剂，这一转变主要是由不明确的上下文驱动的。令人担忧的是，思维链推理在比较环境中加剧了社会偏见，即使在模型提供中立后备选项或声称随机回答时，这种系统性偏见仍然作为一种确定性偏见持续存在。最后，我们证明这种比较偏见是一种普遍现象，且与模型规模呈正相关。最终，我们提供了一个重要的方法论指导：虽然研究人员必须利用比较环境来稳健审计隐藏的偏见，但从业者在模糊的现实任务中不能安全地依赖比较部署。

View on arXiv Download PDF AI Translation

cs.CL / 52 / 2606.24597

Qwen-AgentWorld: Language World Models for General Agents

Qwen-AgentWorld：通用智能体的语言世界模型

Zuo, Yuxin, Xiao, Zikai, Sheng, Li, Huang, Fei, Tu, Jianhong, Liu, Yuxuan, Tang, Tianyi, Hu, Xiaomeng, Su, Yang, Lan, Qingfeng, Liu, Yantao, Zhu, Qin, Zhang, Yinger, Yu, Bowen, Zhao, Haiquan, Xu, Haiyang, Yang, Jianxin, Cheng, Jiayang, Wang, Junyang, Deng, Lianghao, Xue, Mingfeng, Bai, Tianyi, Fan, Yang, Ma, Yubo, Li, Yucheng, Cui, Zeyu, Wang, Zhihai, Xie, Zhihui, Ye, Zhuorui, Yang, An, Liu, Dayiheng, Zhou, Jingren, Ding, Ning

Abstract

A world model predicts environment dynamics based on current observations and actions, serving as a core cognitive mechanism for reasoning and planning. In this work, we investigate how world modeling based on language models can further push the boundaries of general agents. (i) We first focus on building foundation models for agentic environment simulation. We introduce Qwen-AgentWorld-35B-A3B and Qwen-AgentWorld-397B-A17B, the first language world models capable of simulating agentic environments covering 7 domains via long chain-of-thought reasoning. Leveraging more than 10M environment interaction trajectories of 7 domains in real-world environments, we develop Qwen-AgentWorld through a three-stage training pipeline: CPT injects general-purpose world modeling capabilities from the state transition dynamics and augmented professional corpora, SFT activates next-state-prediction reasoning, and RL sharpens simulation fidelity through a tailored framework with hybrid rubric-and-rule rewards. To evaluate language world models, we present AgentWorldBench, a comprehensive benchmark constructed from real-world interactions of 5 frontier models on 9 established benchmarks. Empirical results demonstrate that Qwen-AgentWorld significantly outperforms existing frontier models. (ii) Beyond foundation models, we further investigate two complementary paradigms through which world modeling enhances general agents. First, as a decoupled environment simulator, Qwen-AgentWorld supports scalable and controllable simulation of thousands of real-world environments for agentic RL, yielding gains that surpass real-environment training alone. Second, as a unified agent foundation model, world-model training acts as a highly effective warm-up that improves downstream performance across 7 agentic benchmarks. Code: https://github.com/QwenLM/Qwen-AgentWorld

Chinese Translation

世界模型基于当前观察和行动预测环境动态，作为推理和规划的核心认知机制。在本研究中，我们探讨了基于语言模型的世界建模如何进一步推动通用智能体的边界。(i) 我们首先专注于构建用于智能环境模拟的基础模型。我们介绍了Qwen-AgentWorld-35B-A3B和Qwen-AgentWorld-397B-A17B，这是首个能够通过长链思维推理模拟涵盖7个领域的智能环境的语言世界模型。利用来自真实环境中7个领域的超过1000万条环境交互轨迹，我们通过三阶段训练流程开发了Qwen-AgentWorld：CPT从状态转移动态和增强的专业语料库中注入通用世界建模能力，SFT激活下一状态预测推理，而RL通过一个定制的混合标准和规则奖励框架提升模拟的真实性。为了评估语言世界模型，我们提出了AgentWorldBench，这是一个基于5个前沿模型在9个已建立基准上的真实世界交互构建的综合基准。实证结果表明，Qwen-AgentWorld显著优于现有的前沿模型。(ii) 除了基础模型外，我们进一步探讨了两种互补范式，通过这些范式，世界建模增强了通用智能体。首先，作为一个解耦的环境模拟器，Qwen-AgentWorld支持对数千个真实环境的可扩展和可控的模拟，以用于智能强化学习，带来的收益超越了单纯的真实环境训练。其次，作为一个统一的智能体基础模型，世界模型训练作为一种高效的热身，提升了在7个智能基准上的下游表现。代码：https://github.com/QwenLM/Qwen-AgentWorld

View on arXiv Download PDF AI Translation

cs.CL / 53 / 2606.24610

Same Lesson, Different Story: Cross-Lingual Reconstruction of Cultural Narratives in Large Language Models

同样的教训，不同的故事：大型语言模型中的跨语言文化叙事重构

Alshaalan, Jory, Albaker, Haya, Aldayel, Abeer, Alabdullatif, Aljawharah, Alahmadi, Rehab

Abstract

The evaluation of cultural grounding context becomes complex when multiple cultures convey the same moral lesson. This challenge is particularly relevant to large language models (LLMs), which produce narratives across a wide range of languages and cultural contexts. However, it remains uncertain whether these models preserve culturally grounded meaning when equivalent moral lessons are conveyed through distinct cultural forms. This study introduces a multilingual evaluation narrative framework that integrates a cross-linguistic collection of 414 proverbs spanning 15 languages and uses four LLMs to generate 13k narratives. By employing semantically equivalent proverbs as culturally grounded prompts, the analysis assesses whether models preserve meaning across languages, how cross-lingual conditioning influences narrative realization, and whether different model families converge on similar interpretations. Results indicate that cross-lingual prompting largely preserves proverb-level semantic meaning while systematically redistributing agency, social positioning, and narrative structure. Additionally, strong inter-model convergence is observed in both monolingual and cross-lingual settings, suggesting that multilingual LLMs rely on shared semantic abstractions despite architectural and linguistic differences. These findings shed light on the need for more comprehensive evaluations of cultural grounding. Relying exclusively on semantic similarity in multilingual narrative assessments may overestimate cultural preservation by neglecting culturally meaningful variations in narrative expression.

Chinese Translation

当多种文化传达相同的道德教训时，文化基础语境的评估变得复杂。这一挑战对于大型语言模型（LLMs）尤为相关，因为它们在广泛的语言和文化背景中生成叙事。然而，尚不确定这些模型在通过不同文化形式传达等效道德教训时是否保留了文化根植的意义。本研究引入了一个多语言评估叙事框架，整合了跨语言的414个谚语，涵盖15种语言，并使用四个LLMs生成了13000个叙事。通过采用语义等效的谚语作为文化根植的提示，分析评估了模型在不同语言间是否保留意义、跨语言条件如何影响叙事实现，以及不同模型家族是否趋向于相似的解释。结果表明，跨语言提示在很大程度上保留了谚语层面的语义意义，同时系统性地重新分配了代理、社会定位和叙事结构。此外，在单语和跨语言环境中观察到强烈的模型间趋同，表明尽管存在架构和语言差异，多语言LLMs仍依赖于共享的语义抽象。这些发现揭示了对文化根植进行更全面评估的必要性。在多语言叙事评估中仅依赖语义相似性可能会高估文化的保留，因为它忽视了叙事表达中具有文化意义的变异。

View on arXiv Download PDF AI Translation

cs.CL / 54 / 2606.24623

Privacy-Preserving RAG via Multi-Agent Semantic Rewriting: Achieving Confidentiality Without Compromising Contextual Fidelity

通过多智能体语义重写实现隐私保护的检索增强生成：在不妥协上下文保真度的情况下实现机密性

Zhao, Yuanhe, Zhang, Tianyu, Xing, Huafei, Wong, Derek F., Li, Jianbin, Fang, Tao

Abstract

Retrieval-Augmented Generation enhances large language models by incorporating external knowledge, but deploying it in sensitive scenarios risks privacy leakage via malicious prompts. To address this, we propose a multi-agent framework that sanitizes retrieved content through semantic rewriting. By employing three specialized agents for privacy extraction, semantic analysis, and reconstruction, our approach collaboratively removes sensitive identifiers while preserving the semantic core. We evaluate the framework on the ChatDoctor and Wiki-PII datasets across six large language models. Experimental results demonstrate a significant reduction in privacy leakage under targeted attacks. For instance, we reduced targeted information exposure in LLaMA-3-8B from 144 instances in the baseline to just 1. Furthermore, we maintain strong contextual fidelity with a BLEU-1 score of 0.122, outperforming the existing SAGE method's 0.117. Finally, the framework operates as an asynchronous preprocessing module, introducing no additional latency to online inference, as all rewriting is executed as a one-time offline preprocessing step. To promote reproducibility, the source code of this work is publicly available at https://github.com/foursoils/Privacy-Preserving-RAG.

Chinese Translation

检索增强生成（Retrieval-Augmented Generation）通过整合外部知识增强大型语言模型，但在敏感场景中部署时，可能会因恶意提示而导致隐私泄露。为了解决这个问题，我们提出了一种多智能体框架，通过语义重写来净化检索内容。我们采用三个专门的智能体进行隐私提取、语义分析和重构，协同去除敏感标识符，同时保留语义核心。我们在ChatDoctor和Wiki-PII数据集上对该框架进行了评估，涉及六个大型语言模型。实验结果表明，在针对性攻击下，隐私泄露显著减少。例如，我们将LLaMA-3-8B模型中的目标信息暴露从基线的144个实例减少到仅1个。此外，我们保持了较强的上下文保真度，BLEU-1分数为0.122，优于现有SAGE方法的0.117。最后，该框架作为一个异步预处理模块运行，不会对在线推理引入额外延迟，因为所有重写均作为一次性离线预处理步骤执行。为了促进可重复性，本工作的源代码已公开，网址为https://github.com/foursoils/Privacy-Preserving-RAG。

View on arXiv Download PDF AI Translation

cs.CL / 55 / 2606.24627

The Warrant Gap: Claim-Conditioned Re-scoring for Fact-Checking

保证差距：基于声明条件的事实检查重评分

Dey, Arka Ujjal, Collomosse, John

Abstract

Fact-checking systems built on LLMs achieve high verdict accuracy on standard benchmarks, yet routinely output Supports labels whose cited evidence does not license the claim. Structured decomposition is the natural way to inspect those warrants, but rigid extraction protocols strip the full-claim context that facets need. We introduce SIFT -- claim-conditioned re-scoring of extracted evidence spans against the full claim -- paired with WSP (Warranted Supports Proportion), an automatic NLI check that the cited warrant entails the claim. We evaluate on FEVER, SciFact, 5PILS, and DP across four open-source backbones. SIFT recovers accuracy on cells where naive decomposition costs up to 27.6 points, while raising WSP above direct prompting; WSP itself calibrates against human gold evidence at AUC 0.92 and precision 0.98.

Chinese Translation

基于大型语言模型（LLMs）的事实检查系统在标准基准上实现了高判决准确率，但常常输出支持标签，其引用的证据并未支持该声明。结构化分解是检查这些保证的自然方式，但严格的提取协议剥夺了所需的完整声明上下文。我们提出了 SIFT（基于声明条件的提取证据跨度重评分），该方法针对完整声明重新评分提取的证据，并结合 WSP（保证支持比例），这是一个自动的自然语言推理（NLI）检查，确保所引用的保证蕴含该声明。我们在 FEVER、SciFact、5PILS 和 DP 四个开源基础模型上进行了评估。SIFT 在简单分解导致的准确率损失高达 27.6 分的情况下恢复了准确性，同时将 WSP 提升至高于直接提示的水平；WSP 本身在与人类金证据的比较中，其 AUC 为 0.92，精确度为 0.98。

View on arXiv Download PDF AI Translation

cs.CL / 56 / 2606.24644

Measuring User's Mental Models of Speech Translation in Human-AI Collaboration

测量用户在人与人工智能协作中对语音翻译的心理模型

Han, HyoJung, Balepur, Nishant, Boyd-Graber, Jordan, Carpuat, Marine

Abstract

Millions of people use machine translation (MT) tools daily, yet little is known about their perception of what systems can and cannot do. This paper studies users' mental models of speech translation systems through a new framework based on cross-lingual question answering, where users either accept MT output or request professional re-translation to answer questions based on the information presented in a foreign language. By analyzing user behavior and accuracy trends across varying translation qualities, we examine to what extent they can predict where the system is likely to be wrong, and how this mental model evolves. Users develop stronger mental models with practice, especially when they have some knowledge of the source language, primarily by relying on surface-level error cues. Moreover, providing speech transcriptions can help users develop better mental models. Our results show the promise of cross-lingual question answering as a downstream task for studying MT mental models and advancing our understanding of human-AI collaboration.

Chinese Translation

每天有数百万人使用机器翻译（MT）工具，但关于用户对系统能力的认知知之甚少。本文通过基于跨语言问答的新框架研究用户对语音翻译系统的心理模型，在该框架中，用户要么接受机器翻译输出，要么请求专业重新翻译，以回答基于外语信息的问题。通过分析用户行为和不同翻译质量下的准确性趋势，我们考察了用户在多大程度上能够预测系统可能出错的地方，以及这一心理模型是如何演变的。用户在实践中发展出更强的心理模型，尤其是在他们对源语言有一定了解时，主要依赖于表面错误提示。此外，提供语音转录可以帮助用户建立更好的心理模型。我们的研究结果显示，跨语言问答作为研究机器翻译心理模型的下游任务具有潜力，并推动我们对人与人工智能协作的理解。

View on arXiv Download PDF AI Translation

cs.CL / 57 / 2606.24650

Harmonic: Hierarchical State Space Models for Efficient Long-Context Language Modeling

Harmonic：用于高效长上下文语言建模的层次状态空间模型

Nyoma, Petr

Abstract

We present Harmonic, a hierarchical state space model (SSM) for language modeling. The architecture stacks three recurrent levels at progressively slower timescales; each level receives the prediction error of the level below as input, rather than its raw hidden state. On enwiki8 with equal token budgets, Harmonic outperforms a comparable Transformer (28M params) by +1.4% at 1K tokens, +6.7% at 8K tokens, and +11.4% at 32K tokens (bpt, lower is better). It also outperforms Mamba at every tested length by 0.7--1.8%. At 64K tokens, both Mamba and Transformer run out of memory on an 80GB H100; Harmonic trains successfully, reaching 6.169 bpt. Results replicate on WikiText-103 (H-TF gap +1.7% to +7.2% across 1K--32K). At 1B parameter scale, replacing all attention layers in TinyLlama 1.1B with HarmonicBlock eliminates the RoPE positional encoding limit: the resulting Hallamonic model maintains stable loss across sequence lengths 1K--8K on two independent clean benchmarks (Lambada and fineweb-edu held-out), while TinyLlama degrades catastrophically past its 2K-token RoPE limit (gap: +9.4 bpt at seq=8K on Lambada). Compute is O(L) per forward pass vs. O(L^2) for attention. Logs: https://github.com/Omibranch/harmonic-logs.

Chinese Translation

我们提出了Harmonic，一种用于语言建模的层次状态空间模型（SSM）。该架构在逐渐减慢的时间尺度上堆叠了三个递归层；每个层接收下层的预测误差作为输入，而不是其原始的隐藏状态。在enwiki8上，在相同的标记预算下，Harmonic在1K标记时比一个可比的Transformer（28M参数）提高了1.4%，在8K标记时提高了6.7%，在32K标记时提高了11.4%（bpt，越低越好）。在每个测试长度上，它也比Mamba超出0.7%到1.8%。在64K标记时，Mamba和Transformer在80GB H100上都出现了内存不足的情况；而Harmonic成功训练，达到了6.169 bpt。结果在WikiText-103上得到了复制（H-TF差距在1K到32K之间为+1.7%到+7.2%）。在1B参数规模下，将TinyLlama 1.1B中的所有注意力层替换为HarmonicBlock消除了RoPE位置编码的限制：生成的Hallamonic模型在两个独立的干净基准（Lambada和fineweb-edu保留集）上保持了1K到8K序列长度的稳定损失，而TinyLlama在超过其2K标记的RoPE限制后表现出灾难性的退化（差距：在Lambada上seq=8K时为+9.4 bpt）。每次前向传播的计算复杂度为O(L)，而注意力为O(L^2)。日志链接：https://github.com/Omibranch/harmonic-logs。

View on arXiv Download PDF AI Translation

cs.CL / 58 / 2606.24655

AI-PAVE-Br: Leveraging Large Language Models for Enhanced Product Attribute Value Extraction through a Golden Set Approach

AI-PAVE-Br：通过黄金集方法利用大型语言模型增强产品属性值提取

Gazzola, Murilo, Souto, Hugo Gobato, Silva, Samuel, Peixoto, Júlia Schubert, Siqueira, Felipe, de Morais, André Luis Pedroso, Gomes, Caio

Abstract

The explosive growth and complexity of product data within the dynamic Brazilian e-commerce landscape demand robust and specialized methods for structured information extraction. Traditional approaches to Product Attribute Value Extraction (PAVE) often struggle with the linguistic nuances and sheer diversity of product descriptions in Portuguese. To address this critical gap, this paper introduces two major contributions. First, we present AI-PAVEBr, a specialized system engineered with Large Language Models (LLMs) to perform high-accuracy PAVE specifically for Brazilian e-commerce catalogs. Second, to facilitate reproducible research and provide a definitive benchmark, we introduce and share the Golden Set, a new, meticulously curated, and manually annotated dataset for PAVE in Portuguese. We detail the creation process and structure (Entity, Category, Subcategories) of this high-quality reference set. Our experiments conclusively show that AI-PAVE-Br, leveraging targeted prompt engineering, dramatically outperforms conventional Named Entity Recognition (NER) baselines. This work not only delivers a superior, scalable solution for a major non-English market but also enriches the NLP community with a valuable, publicly available resource for future PAVE research.

Chinese Translation

在动态的巴西电子商务环境中，产品数据的爆炸性增长和复杂性要求采用强大且专业的方法进行结构化信息提取。传统的产品属性值提取（PAVE）方法常常难以应对葡萄牙语产品描述中的语言细微差别和多样性。为了解决这一关键问题，本文提出了两个主要贡献。首先，我们介绍了AI-PAVE-Br，这是一个专门设计的系统，利用大型语言模型（LLMs）在巴西电子商务目录中执行高精度的PAVE。其次，为了促进可重复的研究并提供一个明确的基准，我们引入并分享了黄金集（Golden Set），这是一个新的、经过精心策划和手动标注的葡萄牙语PAVE数据集。我们详细描述了该高质量参考集的创建过程和结构（实体、类别、子类别）。我们的实验结果明确表明，AI-PAVE-Br通过针对性的提示工程，显著超越了传统的命名实体识别（NER）基准。这项工作不仅为一个主要的非英语市场提供了优越的可扩展解决方案，还为NLP社区提供了一个有价值的、公开可用的资源，以支持未来的PAVE研究。

View on arXiv Download PDF AI Translation

cs.CL / 59 / 2606.24667

DREAM: Dense Retrieval Embeddings via Autoregressive Modeling

DREAM：通过自回归建模实现密集检索嵌入

Tang, Yixuan, Yang, Yi

Abstract

Dense retrieval embedding models are a fundamental component of modern retrieval-based AI systems. Most dense retrievers are trained with contrastive objectives, which require labeled positive and negative document pairs that are often costly and difficult to obtain. In this work, we investigate whether the autoregressive next-token prediction objective of a large language model (LLM) can provide supervision for dense retrieval. The intuition is simple: if a document contains information relevant to a query, conditioning on that document should make the target output easier for the LLM to predict. A key challenge is that the next-token prediction loss is computed inside the LLM, while the retriever is a separate embedding model. To address this challenge, we propose DREAM (Dense Retrieval Embeddings via Autoregressive Modeling), which injects retriever-generated query-document similarity scores into selected attention heads of a frozen LLM. During training, these scores determine how much attention each candidate document receives while the LLM predicts the target output. The resulting prediction loss provides gradients for retriever training through the attention mechanism. We evaluate DREAM on retrieval benchmarks BEIR and RTEB using embedding backbones ranging from 0.5B to 3B parameters. DREAM consistently outperforms existing baselines across different model scales. These results demonstrate that DREAM provides a promising approach for training dense retrievers through autoregressive modeling.

Chinese Translation

密集检索嵌入模型是现代基于检索的人工智能系统的基本组成部分。大多数密集检索器是通过对比目标进行训练的，这需要标记的正负文档对，而这些文档对往往成本高昂且难以获得。在本研究中，我们探讨了大型语言模型（LLM）的自回归下一个令牌预测目标是否可以为密集检索提供监督。直观上，如果一篇文档包含与查询相关的信息，对该文档进行条件化应该使得LLM预测目标输出变得更容易。一个关键挑战是，下一个令牌预测损失是在LLM内部计算的，而检索器则是一个独立的嵌入模型。为了解决这一挑战，我们提出了DREAM（通过自回归建模实现密集检索嵌入），该方法将检索器生成的查询-文档相似度分数注入到冻结的LLM的选定注意力头中。在训练过程中，这些分数决定了每个候选文档在LLM预测目标输出时获得多少关注。由此产生的预测损失通过注意力机制为检索器训练提供了梯度。我们在检索基准BEIR和RTEB上评估DREAM，使用的嵌入骨干网络参数范围从0.5B到3B。DREAM在不同模型规模下始终优于现有基线。这些结果表明，DREAM为通过自回归建模训练密集检索器提供了一种有前景的方法。

View on arXiv Download PDF AI Translation

cs.CL / 60 / 2606.24714

CN-NewsTTS Bench: a target-level automatic benchmark for raw-input Chinese news TTS pronunciation

CN-NewsTTS Bench：针对原始输入中文新闻 TTS 发音的目标级自动基准

Luo, Shijun

Abstract

Chinese news text contains dense written forms such as scores, hyphenated model names, ranges, unit symbols, percentages, English abbreviations, and mixed Chinese-Latin-digit names. These forms are frequent in real listening workflows, and a text-to-speech (TTS) system can preserve the written string while changing the spoken meaning. We introduce CN-NewsTTS Bench v0.1, an open target-level benchmark for evaluating whether Chinese news TTS products pronounce such targets correctly from raw text, without user-side rules, LLM rewriting, SSML hints, or manual edits. The release contains a 200-record development set, an 800-record public test set, 992 public auto-evaluable targets, fixed transcripts from a three-ASR ensemble, an automatic target scorer, and initial results for seven product TTS systems. We additionally report ASR-route diagnostics, ASR-subset ablations, category-level results, confidence intervals, and provider configuration metadata. The best system reaches 0.879 strict accuracy, while several systems remain below 0.60.

Chinese Translation

中文新闻文本包含密集的书面形式，如分数、带连字符的模型名称、范围、单位符号、百分比、英文缩写以及混合的中英文数字名称。这些形式在真实的听觉工作流程中频繁出现，而文本到语音（TTS）系统可以保留书面字符串，同时改变口语的含义。我们介绍了 CN-NewsTTS Bench v0.1，这是一个开放的目标级基准，用于评估中文新闻 TTS 产品是否能够从原始文本正确发音这些目标，而不依赖于用户侧规则、LLM 重写、SSML 提示或手动编辑。该版本包含一个 200 条记录的开发集、一个 800 条记录的公共测试集、992 个公共自动可评估目标、来自三种 ASR 集成的固定转录、一个自动目标评分器，以及七个产品 TTS 系统的初步结果。我们还报告了 ASR 路径诊断、ASR 子集消融、类别级结果、置信区间和提供者配置元数据。最佳系统达到了 0.879 的严格准确率，而多个系统的准确率仍低于 0.60。

View on arXiv Download PDF AI Translation

cs.CL / 61 / 2606.24734

Task Decomposition for Efficient Annotation

高效标注的任务分解

Gandhi, Nupoor, Strubell, Emma

Abstract

High-quality annotations of structured representations are expensive to collect over large corpora. Manual annotation of structure is laborious, and model-based annotation, although cheaper to generate, requires expensive validation and potentially significant supervision to ensure that the annotation quality is strong enough to be useful downstream. In traditional annotation workflows, annotation of each complete example is performed end-to-end by a single annotator. However, structured annotation is complex, and each aspect of the task represents a unique challenge with an associated inferential load for a given annotator. Modern annotation projects can incorporate heterogeneous groups of annotators, including both models and human annotators with varying domain and linguistic expertise. It remains unclear, however, how to redesign annotation tasks in this setting, where efforts are discriminately allocated across heterogeneous annotators with respect to distinct annotation challenges. We propose to decompose annotation tasks into sub-tasks in order to reduce the aggregate inferential load of annotation projects. Inspired by the notion of centers from centering theory, we introduce a formal model of inferential load based on the degrees of freedom in the space of valid annotations. Using this model, we show that identifying these centers (i.e. salient anchor entities realized by annotation sub-tasks) constrains the output space complexity, and decompositions which isolate and advance center identification reduce the aggregate inferential load. We provide guidelines for decomposing complex structured annotation tasks, supported by examples demonstrating improved cost-efficiency from our prior work. Finally, we present a procedure for allocating sub-tasks across annotators to maximize quality under a fixed budget.

Chinese Translation

高质量的结构化表示标注在大规模语料库中收集成本高昂。手动标注结构工作繁重，而基于模型的标注虽然生成成本较低，但需要昂贵的验证和可能显著的监督，以确保标注质量足够强大以便于后续使用。在传统的标注工作流程中，每个完整示例的标注由单一标注者端到端地完成。然而，结构化标注是复杂的，每个任务的各个方面都代表着独特的挑战，并对特定标注者带来了相应的推理负担。现代标注项目可以结合异质的标注者群体，包括具有不同领域和语言专业知识的模型和人工标注者。然而，在这种情况下，如何重新设计标注任务仍然不清楚，因为努力在不同的标注者之间根据不同的标注挑战进行差异化分配。我们建议将标注任务分解为子任务，以减少标注项目的整体推理负担。受到中心理论中中心概念的启发，我们引入了一个基于有效标注空间自由度的推理负担的形式模型。利用该模型，我们展示了识别这些中心（即通过标注子任务实现的显著锚实体）如何限制输出空间的复杂性，而隔离和推进中心识别的分解可以减少整体推理负担。我们提供了分解复杂结构化标注任务的指导方针，并通过示例展示了我们之前工作的成本效率提升。最后，我们提出了一种在固定预算下分配子任务以最大化质量的程序。

View on arXiv Download PDF AI Translation

cs.CL / 62 / 2606.24758

CANDLE: Character-level Arabic Noise Deduplication using Lightweight Encoder

CANDLE：基于轻量级编码器的字符级阿拉伯噪声去重

Alasmary, Faris, Nono, Taif, Zaafarani, Orjuwan, Tabash, Kholood Al, Ghannam, Ahmad, Salamah, Anas, Sadah, Shouq, Ghouti, Lahouari

Abstract

Handling repeated characters in text can be tricky, since they can represent either the correct spelling of a word or informal character elongation often seen in social media posts. We present CANDLE, a lightweight system for character-level Arabic noise deduplication that addresses this challenge without relying on handcrafted rules, dictionaries, or morphological analyzers. At the heart of CANDLE is a novel application of Connectionist Temporal Classification (CTC) to this task, a formulation not previously explored for character deduplication, which frames normalization as a sequence alignment problem over a character-based encoder. Evaluated on three benchmarks spanning clean newspaper, manually curated ambiguous cases, and real-world social media text, the CTC model achieves a Sentence Error Rate (SER) as low as $5.37\%$ and consistently outperforms a classification-based baseline by a large margin. To reduce inference overhead, we distill the 6-layer CTC model into a 2-layer student, achieving a $3\times$ depth reduction with minimal performance degradation. Beyond deduplication accuracy, normalization yields a practical downstream benefit: a relative reduction in tokenizer fertility of up to $12.8\%$ across a diverse set of Arabic LLM tokenizers, directly lowering inference costs and improving context window utilization. We release all code and models publicly to support reproducibility and advance future research\footnote{https://github.com/abjadai/candle}.

Chinese Translation

处理文本中的重复字符可能会很棘手，因为它们既可以表示单词的正确拼写，也可以表示社交媒体帖子中常见的非正式字符延长。我们提出了CANDLE，一个轻量级的字符级阿拉伯噪声去重系统，旨在解决这一挑战，而无需依赖手工规则、词典或形态分析器。CANDLE的核心是将连接时序分类（Connectionist Temporal Classification, CTC）新颖地应用于这一任务，这一公式在字符去重中尚未被探索，它将标准化框架视为基于字符的编码器上的序列对齐问题。在三个基准测试上进行评估，涵盖了干净的报纸文本、手动整理的模糊案例以及真实世界的社交媒体文本，CTC模型的句子错误率（Sentence Error Rate, SER）低至5.37%，并且始终大幅超越基于分类的基线。为了减少推理开销，我们将6层的CTC模型提炼为2层的学生模型，实现了3倍的深度减少，且性能下降最小。除了去重准确性外，标准化还带来了实际的下游好处：在多样化的阿拉伯大语言模型（LLM）分词器中，相对减少了高达12.8%的分词器繁殖率，直接降低了推理成本并改善了上下文窗口的利用率。我们公开发布所有代码和模型，以支持可重复性并推动未来的研究。

View on arXiv Download PDF AI Translation

cs.CL / 63 / 2606.24773

Posterior Refinement: Fast Language Generation via Any-Order Flow Maps

后验精炼：通过任意顺序流图实现快速语言生成

Agarwal, Manan, Shah, Sheel, Lee, Chanhyuk, Yoo, Jaehoon, Huang, Jerry, Hong, Seunghoon, Raghunathan, Aditi, Kim, Jinwoo, Boffi, Nicholas M.

Abstract

Non-autoregressive generation offers a powerful paradigm for iterative refinement, allowing models to recursively critique, erase and regenerate arbitrary subsets of tokens. However, existing non-autoregressive models fail to realize this potential. Masked Diffusion Models (MDMs) suffer from factorization error, causing sample quality to collapse when generating multiple tokens simultaneously. Flow Map Language Models (FMLMs) circumvent this bottleneck via joint sequence transport for excellent few-step generation, but sacrifice the inference-time flexibility of MDMs. We introduce FMLM+, a framework that bridges this gap by equipping FMLM with masking-style noise schedules. While generating the full sequence in a single step, FMLM+ simultaneously scores the global consistency of each token a posteriori. We leverage this to introduce Posterior Refinement, a novel inference-time refinement strategy that enables the model to adaptively self-correct its outputs, matching the performance of discrete baselines with 32x fewer NFEs. Across diverse benchmarks, we demonstrate that FMLM+ with Posterior Refinement improves the speed--quality tradeoff over both MDM and FMLM families, providing a scalable foundation for high-fidelity language modeling.

Chinese Translation

非自回归生成提供了一种强大的迭代精炼范式，使模型能够递归地批评、删除和重新生成任意子集的标记。然而，现有的非自回归模型未能实现这一潜力。掩蔽扩散模型（Masked Diffusion Models, MDMs）受到因子化误差的影响，在同时生成多个标记时导致样本质量崩溃。流图语言模型（Flow Map Language Models, FMLMs）通过联合序列传输来规避这一瓶颈，实现了优秀的少步生成，但牺牲了MDMs在推理时的灵活性。我们提出了FMLM+，一个通过为FMLM配备掩蔽式噪声调度来弥合这一差距的框架。在单步生成完整序列的同时，FMLM+还对每个标记的全局一致性进行后验评分。我们利用这一点引入了后验精炼（Posterior Refinement），一种新颖的推理时精炼策略，使模型能够自适应地自我纠正输出，以32倍更少的NFEs匹配离散基线的性能。在各种基准测试中，我们证明了带有后验精炼的FMLM+在速度与质量的权衡上优于MDM和FMLM系列，为高保真语言建模提供了可扩展的基础。

View on arXiv Download PDF AI Translation

cs.CL / 64 / 2606.24775

Are We Ready For An Agent-Native Memory System?

我们准备好迎接代理原生内存系统了吗？

Zhou, Wei, Zhou, Xuanhe, Han, Shaokun, Xu, Hongming, Li, Guoliang, Li, Zhiyu, Xiong, Feiyu, Wu, Fan

Abstract

Memory for large language model (LLM) agents has rapidly evolved from simple retrieval-augmented mechanisms into a data management system that supports persistent information storage, retrieval, update, consolidation, and dynamic lifecycle governance throughout agent execution. Despite this evolution, existing evaluations still benchmark agent memory mainly through end-to-end task success metrics (e.g., F1, BLEU), while treating the underlying system as a monolithic black box. As a result, critical system-level concerns, including operational costs, architectural trade-offs across memory modules, and robustness under dynamic knowledge updates, remain insufficiently explored. In this paper, we present a systematic experimental study of agent memory from a data management perspective. We propose an analytical framework that decomposes agent memory into four core modules: memory representation and storage, extraction, retrieval and routing, and maintenance. Under this framework, we evaluate 12 representative memory systems and two reference baselines across five benchmark workloads spanning 11 datasets. Our extensive end-to-end evaluation shows that no single architecture dominates across all scenarios; instead, effectiveness depends heavily on how well the memory structure aligns with the workload bottleneck. Furthermore, through fine-grained ablation studies, we quantify their individual effects on representation fidelity, retrieval precision, update correctness, and long-horizon stability. Finally, we reveal cost-performance trade-offs under realistic workloads, showing localized maintenance is more cost-efficient than global reorganization. Based on these findings, we identify promising directions towards building truly agent-native memory systems. The code is publicly available at https://github.com/OpenDataBox/MemoryData.

Chinese Translation

大型语言模型（LLM）代理的内存系统已经迅速从简单的检索增强机制演变为一个数据管理系统，支持在代理执行过程中进行持久信息存储、检索、更新、整合和动态生命周期管理。尽管这种演变存在，现有评估仍主要通过端到端任务成功指标（例如，F1、BLEU）来基准测试代理内存，同时将底层系统视为一个单一的黑箱。因此，关键的系统级问题，包括操作成本、内存模块之间的架构权衡以及在动态知识更新下的鲁棒性，仍然未得到充分探讨。本文从数据管理的角度系统性地研究了代理内存。我们提出了一个分析框架，将代理内存分解为四个核心模块：内存表示与存储、提取、检索与路由，以及维护。在此框架下，我们评估了12个代表性的内存系统和两个参考基线，涵盖了跨越11个数据集的五个基准工作负载。我们广泛的端到端评估表明，没有单一架构在所有场景中占据主导地位；相反，效果在很大程度上依赖于内存结构与工作负载瓶颈的匹配程度。此外，通过细粒度的消融研究，我们量化了它们对表示保真度、检索精度、更新正确性和长时间稳定性的个体影响。最后，我们揭示了在现实工作负载下的成本-性能权衡，显示局部维护比全局重组更具成本效益。基于这些发现，我们确定了构建真正代理原生内存系统的有希望的方向。代码已公开发布在 https://github.com/OpenDataBox/MemoryData。

View on arXiv Download PDF AI Translation

cs.CL / 65 / 2606.24783

Paying to Know: Micro-Transaction Markets for Verified Product Information in Agentic E-Commerce

付费获取知识：代理电子商务中经过验证的产品信息微交易市场

Ventirozos, Filippos, Shardlow, Matthew

Abstract

Commercial NLP treats the shopping chatbot as a recommender or a conversion tool: its job is to match a user to a catalogue entry and close a sale. We argue that the arrival of agent-native micro-payment rails (e.g., x402, AP2) changes what is scarce. When the buyer is an autonomous agent that can investigate exhaustively, the bottleneck is no longer matching products but acquiring trustworthy, decision-relevant information about them. We envision agentic e-commerce as a micro-transaction market for verified information: buyer agents spend fractions of a cent to progressively unlock seller- and reviewer-supplied data -- service histories, third-party test reports, bills of materials, audited sales and support metrics -- paid for a la carte under a freemium model, with reviewer trust scored reputationally. We sketch the architecture of such a market and argue that it rewards genuine product quality and yields truer competition than ranking-based storefronts. We then translate the vision into concrete NLP problems -- cost-optimal information acquisition, data pricing and negotiation, real-time entity resolution, grounded value exchange, and privacy-preserving persona modelling -- and argue that these, not chat fluency, deserve the field's attention.

Chinese Translation

商业自然语言处理将购物聊天机器人视为推荐工具或转化工具：它的任务是将用户与目录条目匹配并完成销售。我们认为，代理原生微支付通道（例如，x402，AP2）的出现改变了稀缺资源的定义。当买方是一个能够进行全面调查的自主代理时，瓶颈不再是产品匹配，而是获取有关产品的可信、决策相关的信息。我们设想代理电子商务作为一个经过验证的信息微交易市场：买方代理花费几分之一美分逐步解锁卖方和评论者提供的数据——服务历史、第三方测试报告、物料清单、审计的销售和支持指标——以自选方式在增值服务模式下支付，评论者的信任度通过声誉评分。我们勾勒了这样一个市场的架构，并认为它奖励真正的产品质量，并比基于排名的商店提供更真实的竞争。接着，我们将这一愿景转化为具体的自然语言处理问题——成本最优的信息获取、数据定价与谈判、实时实体解析、基于价值的交换以及隐私保护的角色建模——并认为这些问题，而非聊天流畅性，才应当受到该领域的关注。

View on arXiv Download PDF AI Translation

cs.CL / 66 / 2606.24820

SHERLOC: Structured Diagnostic Localization for Code Repair Agents

SHERLOC：用于代码修复代理的结构化诊断定位

Tamoyan, Hovhannes, Narenthiran, Sean, Arakelyan, Erik, Mezini, Mira, Ginsburg, Boris

Abstract

LLM agents solve repository-level coding tasks through multi-turn tool use, but utilize half their budget on locating faults before editing. Dedicated localization frameworks have emerged, yet are still evaluated as file retrieval rather than actionable diagnosis, producing locations without the diagnostic context a repair agent needs. We introduce SHERLOC (Structured Hypothesis-driven Exploration and Reasoning for Localization), a training-free framework pairing a reasoning LLM with compact repository tools and self-recovery, without fine-tuning or multi-agent orchestration. SHERLOC reaches state-of-the-art localization across model scales: 84.33% accuracy@1 on SWE-Bench Lite and 81.27% recall@1 on SWE-Bench Verified; at ~30B parameters, it matches or outperforms other agentic methods. Injecting our locations and diagnostic findings into repair agents yields, on average, +5.95 pp resolve rate on SWE-Bench Verified while cutting localization and total tokens by 36.7% and 23.1%.

Chinese Translation

大型语言模型（LLM）代理通过多轮工具使用解决库级编码任务，但在编辑之前将一半的预算用于定位故障。专门的定位框架已经出现，但仍然被评估为文件检索而非可操作的诊断，生成的位置缺乏修复代理所需的诊断上下文。我们提出了SHERLOC（结构化假设驱动的探索与定位推理），这是一个无需训练的框架，将推理型LLM与紧凑的库工具和自我恢复相结合，无需微调或多代理协调。SHERLOC在各模型规模上达到了最先进的定位效果：在SWE-Bench Lite上准确率为84.33%（准确率@1），在SWE-Bench Verified上召回率为81.27%（召回率@1）；在约30B参数下，其性能与其他代理方法相当或更优。将我们的定位和诊断结果注入修复代理，平均提高了SWE-Bench Verified的解决率5.95个百分点，同时将定位和总令牌数分别减少了36.7%和23.1%。

View on arXiv Download PDF AI Translation

cs.CL / 67 / 2606.24825

L3Cube-MahaPOS: A Marathi Part-of-Speech Tagging Dataset and BERT Models

L3Cube-MahaPOS：一个马拉地语词性标注数据集及BERT模型

Ingle, Hariom, Ghode, Ronit, Gondkar, Ishwari, Harad, Jidnyasa, Joshi, Raviraj

Abstract

Part-of-Speech (POS) tagging is a foundational NLP task underpinning machine translation, information extraction, and syntactic parsing. Despite Marathi being spoken by over 83 million people and ranking among the top twenty most spoken languages worldwide, it remains severely under-resourced in annotated corpora and standardised evaluation benchmarks. Marathi presents unique challenges for computational modelling owing to its rich morphology, relatively free word order, lack of capitalisation conventions, and pervasive code-mixing with Hindi and English. We introduce L3Cube-MahaPOS, a gold-standard POS tagging dataset for Marathi comprising 32,354 manually annotated sentences drawn from news text. Annotation was performed entirely manually by a team of Marathi-proficient annotators following a 16-tag Universal Dependencies-aligned scheme. A structured preprocessing pipeline covering Unicode normalisation, Devanagari-aware tokenisation, and noise filtering ensures label consistency across all splits. We benchmark the dataset across six model families spanning HMM, CRF, BiLSTM, BiLSTM+CharCNN, MuRIL, and the Marathi-specific transformer MahaBERT-v2. The best system achieves 88.67\% token-level accuracy and a macro-F1 of 81.67% over 15 evaluated tag classes. We release the dataset, annotation guidelines, and trained model checkpoints to foster further research in Marathi NLP.

Chinese Translation

词性标注（Part-of-Speech, POS）是自然语言处理（NLP）中的基础任务，支撑着机器翻译、信息提取和句法解析。尽管马拉地语有超过8300万人使用，并且在全球最常用语言中排名前二十，但在标注语料库和标准化评估基准方面仍然严重不足。马拉地语由于其丰富的形态变化、相对自由的词序、缺乏大写规范以及与印地语和英语的广泛混合使用，给计算建模带来了独特的挑战。我们介绍了L3Cube-MahaPOS，这是一个马拉地语的金标准词性标注数据集，包括32,354个从新闻文本中提取的手动标注句子。标注工作完全由一支精通马拉地语的标注团队手动完成，遵循16标签的通用依赖（Universal Dependencies）对齐方案。一个结构化的预处理管道涵盖了Unicode规范化、德文纳加里（Devanagari）感知的分词和噪声过滤，确保所有拆分中的标签一致性。我们在六个模型家族中对数据集进行了基准测试，涵盖了隐马尔可夫模型（HMM）、条件随机场（CRF）、双向长短期记忆网络（BiLSTM）、BiLSTM+CharCNN、MuRIL以及马拉地语特定的变换器MahaBERT-v2。最佳系统在15个评估标签类别上实现了88.67%的标记级准确率和81.67%的宏F1值。我们发布了数据集、标注指南和训练模型检查点，以促进马拉地语NLP的进一步研究。

View on arXiv Download PDF AI Translation

cs.CL / 68 / 2606.24828

Less is More: Quality-Aware Training Data Selection for Scientific Summarization

少即是多：面向质量的科学摘要训练数据选择

Paraskevopoulou, Maria Nefeli, Passali, Tatiana, Tsoumakas, Grigorios

Abstract

Scientific long-document summarization datasets commonly treat author-written abstracts as gold reference summaries, although their quality and alignment with the source article vary. At the same time, publicly available scientific summarization datasets remain limited in scale and structure for modern long-context models. In this work, we address both challenges by a) constructing and releasing one of the largest biomedical and life science datasets for long-document summarization, containing 1.88 million PMC articles, and b) analyzing the reference quality of author-written abstracts with source-grounded and model-based metrics. We show that author-written abstracts vary in their alignment with the full article and that these quality signals can guide training-data selection. Training on selected high-quality subsets outperforms random sampling at matched training sizes and can match or exceed larger random subsets on factuality-oriented metrics. Our findings suggest that reference quality is an important factor in scientific summarization and that quality-aware data selection can improve training efficiency.

Chinese Translation

科学长文档摘要数据集通常将作者撰写的摘要视为黄金参考摘要，尽管其质量和与源文章的一致性存在差异。同时，公开可用的科学摘要数据集在规模和结构上仍然有限，无法满足现代长上下文模型的需求。在本研究中，我们通过以下两方面解决了这两个挑战：a) 构建并发布了一个最大的生物医学和生命科学长文档摘要数据集，包含188万篇PMC文章；b) 使用基于源的和模型的指标分析作者撰写的摘要的参考质量。我们展示了作者撰写的摘要在与完整文章的一致性方面存在差异，这些质量信号可以指导训练数据的选择。在选定的高质量子集上进行训练，相较于匹配训练规模的随机抽样表现更优，并且在面向事实的指标上可以与更大随机子集的表现相匹配或超越。我们的研究结果表明，参考质量是科学摘要中的一个重要因素，而面向质量的数据选择可以提高训练效率。

View on arXiv Download PDF AI Translation

arXiv Papers

Verifiable Foundation Models for Robot Safety

Engineering Reliable Autonomous Systems: Challenges and Solutions

Enforcing Human-like Kinematics in Dexterous Piano Playing via Adversarial Posture Regularization

Topological Online Learning for Displacement-based Formation Control

Sim-to-Real Betting on the E-Process: Bringing "simulators" to anytime-valid confidence sequences

TurboMPC: Fast, Scalable, and Differentiable Model Predictive Control on the GPU

SPACE: Enabling Learning from Cross-Robot Data Toward Generalist Policies

MinInter: Minimizing Trajectory Interpolation During Data Augmentation for Imitation Learning

DynaWM: Dynamics-Aware Distillation with World Model and Momentum Targets for Smooth Locomotion over Continuous Stairs

NavWM: A Unified Navigation World Model for Foresight-Driven Planning

The Evaluation Cost of Task Specialization in Evolutionary Multi-Robot Systems

Grounding Generative Policies in Physics: Optimization-Guided Diffusion for Robot Control

RoBoSR: Structured Scene Representations for Embodied Robotic Reasoning

SlipSense: Multimodal Sensing for Online Slip Detection in Legged Robots

PDS Joint: A Parametric Double-Spiral Joint Tailored for Dexterous Hands

RE4: Transformation-aware Imitation of Object Interactions Using Manipulation Modes

Legible and Intuitive Multi-modal Robot State and Intent Communication Validated in Online and Real-world Studies

Supervise What Survives: Geometry-Guided VLA Adaptation from Synthetic Robot Videos

NoContactNoWorries: Estimating Contact through Vision and Proprioception for In-Hand Dexterous Manipulation

Varying Bundle Size Reactive Multi-Task Assignment using Selective Cost Estimation for Multi-Agent Systems

FT-WBC: Learning Fault-Tolerant Whole-Body Control for Legged Loco-Manipulation

G$^3$VLA: Geometric inductive bias for Vision-Language-Action Models

Decentralized Pose Graph Riemannian Optimization for Object-based Multi-Robot SLAM

Explaining Failures of Cyber-Physical Systems with Actual Causality

Enabling Robust Cloth Manipulation via Inference-Time Simulator-in-the-Loop Refinement

ArtiTwinSplat: Interactable Digital Twin Reconstruction via Gaussian Splatting from RGB-D videos

Optimization-based Safe Trajectory Planning for Autonomous Ground Vehicle in Multi-Floor Scenarios

Beyond Monotonic Progress: Retry-Supervised Value Learning for Robot Imitation

TACTFUL: Tactile-Driven Exploration For Object Localization and Identification in Confined Environments

World Value Models for Robotic Manipulation

Vision-Language Model Reasoning for Contextual Semantic Mapping in Intralogistics

InSight: Self-Guided Skill Acquisition via Steerable VLAs

A Geometry-Informed Computer Vision Method for Detecting and Examining Overtaking Vehicles From A Bicycle

Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation

Listening makes Vision Clear for VLMs

From Spatial to Spectral: An Efficient, Frequency-Guided Feature Representation Learner for Small Object Detection

ABACUS: Adapting Unified Foundation Model for Bridging Image Count Understanding and Generation

HANCLIP: A Family of Hyperbolic Angular Negation Vision Language Models

Mind the Heads: Topological Representation Alignment for Multimodal LLMs

REALM: A Unified Red-Teaming Benchmark for Physical-World VLMs

The Professor: Multi-Teacher Unsupervised Prompt Distillation for Vision-Language Models

Trustworthy Image Authentication using Forensic Knowledge Graphs

DivRL: Disentangled Self-Similarity Rewards for Diverse Subject-Driven Generation

Token-to-Token Alignment of Text Embeddings for Semantic Blending

DriveStack-VLA: Render-Teacher Alignment for BEV-Based DeepStack Vision-Language-Action Model

EPEdit: Redefining Image Editing with Generative AI and User-Centric Design

VisChronos: Revolutionizing Image Captioning Through Real-Life Events

Ingredient-Level Food Image Segmentation for Nutrition Awareness

ObsGraph: Hierarchical Observation Representation for Embodied Reasoning and Exploration

Fabric Image Demoir\'eing Benchmark from Synthesis to Restoration

End-to-End Radar and Communication Modulation Recognition with Neuromorphic Computing

Progressive Pixel-Neighborhood Deformable Cross-Attention for Multispectral Object Detection

Universal Guideline-Driven Image Clustering via a Hybrid LLM Agent

Beyond Bayer: Task-Optimal Sensor Co-Design for Robust Autonomous-Driving Segmentation

DramaDirector: Geometry-Guided Short Drama Generation

A Benchmark for Hallucination Detection in VLMs for Gastrointestinal Endoscopy

An LMM for Precisely Grounding Elements in Documents

Flood Mapping from RGB imagery using a Vision Foundation Model

Bengal-HP_RU: A Dataset of Bengal People For Head Pose Estimation

Sat2City v2: Native 3D City Asset Generation from a Single Satellite Image

Geometry-Aware Style Transfer in 3D Gaussian Splatting

Autonomous Video Generation with Counterfactual Controllability for Self-Evolving World Models

Differential Unfolding: Efficient Unfolding Reconstruction for Video Snapshot Compressive Imaging

Accelerating Multimodal Large Language Models with Prior-Corrected Token Reduction

Dual-Branch Cross-Projection Debiasing through Diffusion-based Disentanglement

Spectral Evolution-Guided Token Pruning in Multimodal Large Language Models

Tri-Efficient Transfer Learning for Point Cloud Videos

Zero-Shot Test-Time Canonicalization using Out-of-Distribution Scoring

Deep Learning Approaches for 3D Medical Scene Completion: From Geometric Modeling to Generative Paradigms

Towards Fast and Effective Long Video Understanding of Multimodal Large Language Models via Adaptive Quasi-Gaussian Sampling

Co-occurring associated retained concepts in Diffusion Unlearning

Inclusive Interactive Collisions for Multi-View Consistent Compositional 3D Generation

MorVess: Morphology-Aware Pulmonary Vessel Segmentation Network

Geometry-Instructed Video Editing

FiCA: Feed-forward instant Gaussian Codec Avatars from a Single Portrait Image

Latent Visual States for Efficient Multimodal Reasoning

From Open Waters to Enclosed Cabins: ProteusVPR for Cross-Scene Visual Place Recognition in Maritime Perception and Cabin Inspection

M^2C-EvDet: Multi-Domain Multi-Order Cross-Modal Knowledge Distillation for Event-based Object Detection

TuringViT: Making SOTA Vision Transformers Accessible to All