arXiv Daily Digest

316

Papers

TinyCVIO: A Constellation-Aided Visual-Inertial Odometry System for Nanodrones

TinyCVIO：一种用于纳米无人机的星座辅助视觉惯性里程计系统

Ozturk, Derin, Akan, Kaan, Wang, Irwin, Batten, Christopher

Abstract

Nanodrones require accurate, real-time state estimation under severe sensing and computational constraints. We present TinyCVIO, a visual-inertial odometry system that co-designs miniature sensing, visual processing, and estimation for a commodity dual-core microcontroller with 520 kB SRAM. Lightweight LED constellations provide known geometry without surveyed positions or yaw angles, assuming placement on a common level plane. A streaming visual frontend tracks LED observations from a millimeter-scale camera at 29.2 FPS, while a rigid-board measurement model retains inter-LED constraints and streaming QR bounds estimation workspace for a fixed filter-state size. Across 19 hand-held hardware-in-the-loop datasets, the rigid-board model reduces mean absolute trajectory error by 27% relative to planar points. The complete system runs onboard a Crazyflie across nine flights at three speeds, achieving 3.5-3.7 cm mean absolute trajectory error and 0.50-0.60% relative pose error over 10 m segments, with mean estimate latency of 15.7-16.3 ms.

Chinese Translation

纳米无人机在严苛的传感和计算约束下需要准确的实时状态估计。我们提出了TinyCVIO，这是一种视觉惯性里程计系统，旨在为一款具有520 kB SRAM的商品双核微控制器共同设计微型传感器、视觉处理和估计。轻量级LED星座提供已知几何形状，而无需测量位置或偏航角，假设其放置在一个共平面上。流式视觉前端以29.2 FPS的速度跟踪来自毫米级相机的LED观测，同时刚性板测量模型保留了LED间约束，并为固定滤波器状态大小提供流式QR边界估计工作空间。在19个手持硬件在环数据集中，刚性板模型相对于平面点减少了27%的平均绝对轨迹误差。完整系统在Crazyflie上运行，经过九次飞行，在三种速度下实现了3.5-3.7 cm的平均绝对轨迹误差和0.50-0.60%的相对姿态误差，覆盖10米段，平均估计延迟为15.7-16.3毫秒。

View on arXiv Download PDF AI Translation

cs.RO / 2 / 2609.30404

POIL: Point-based One-Shot Imitation Learning with Stable Dynamical Systems

POIL：基于点的一次性模仿学习与稳定动态系统

Kim, Sang Min, Seo, Jinwoo, Heo, Hyeongjun, Lee, Junho, Lee, Yonghyeon, Kim, Young Min

Abstract

We present POIL, a point-based one-shot imitation learning framework with stable dynamical systems. While one-shot imitation avoids collecting extensive demonstrations, successful one-shot manipulation requires not only transferring a demonstrated trajectory to a novel object but also executing it robustly under changing scene conditions, grasp configurations, and external disturbances. POIL addresses both problems through a shared representation: a set of 3D points on the object's functional part, used jointly for trajectory transfer and closed-loop execution. The one-shot transfer from the demonstrated trajectory is enabled with point correspondences. POIL grounds the shared functional part with a multi-modal large language model, and transfers the trajectory across viewpoint, pose, and object category changes. During execution, multi-view tracking observes the same points online, and Point-set BCSDM drives them in closed loop by projecting per-point velocities onto a single rigid-body twist computed from the tracked points alone. This extends stable dynamical models from an SE(3) pose to a point set without requiring a known 3D model or pose estimator. We show that at the goal the controller becomes a gradient flow on the classical SO(3) potential, so its terminal phase inherits the almost-global convergence of that potential under a rigid-object assumption. Across simulation and real-robot experiments, POIL transfers a single demonstration across object category, grasp pose, and goal geometry, while recovering from external disturbances during execution. Project page: https://sangminkim-99.github.io/poil

Chinese Translation

我们提出了POIL，一个基于点的一次性模仿学习框架，结合了稳定动态系统。一次性模仿学习避免了收集大量示范，但成功的一次性操作不仅需要将示范轨迹转移到新物体上，还需要在变化的场景条件、抓取配置和外部干扰下稳健地执行。POIL通过共享表示来解决这两个问题：在物体功能部件上的一组3D点，联合用于轨迹转移和闭环执行。通过点对应关系实现从示范轨迹的一次性转移。POIL利用多模态大型语言模型对共享功能部件进行基础定位，并在视角、姿态和物体类别变化中转移轨迹。在执行过程中，多视角跟踪在线观察相同的点，而点集BCSDM通过将每个点的速度投影到仅由跟踪点计算的单一刚体扭转上来驱动这些点的闭环执行。这将稳定动态模型从SE(3)姿态扩展到点集，而无需已知的3D模型或姿态估计器。我们展示了在目标处，控制器变为经典SO(3)势能上的梯度流，因此其终端阶段在刚体假设下继承了该势能的几乎全局收敛性。在模拟和真实机器人实验中，POIL能够在物体类别、抓取姿态和目标几何之间转移单个示范，同时在执行过程中从外部干扰中恢复。项目页面：https://sangminkim-99.github.io/poil

View on arXiv Download PDF AI Translation

cs.RO / 3 / 2609.30428

Actively Resolving Contextual Uncertainty for Underspecified Tasks in Natural Language

主动解决自然语言中不明确任务的上下文不确定性

Ravichandran, Zachary, Diller, Jonathan, Cladera, Fernando, Murali, Varun, Pappas, George J., Kumar, Vijay

Abstract

Foundation models provide robots with the ability to interpret natural language and reason about environmental context, yet most language-conditioned policies assume that goals are well-specified and that task-relevant information is provided upfront via a prior map. Operating in unfamiliar environments with underspecified tasks entails high contextual uncertainty: the robot must jointly infer what constitutes task success, what constitutes relevant information, and where (or whether) that information exists. We address these limitations via CLUE (Closed-Loop contextual Uncertainty rEsolution), a framework for actively resolving contextual uncertainty given underspecified tasks in natural language. CLUE uses an LLM-derived policy to hypothesize task-relevant concepts and potential plans. It then uses a language-embedded map, which is constructed online, to ground these hypotheses into actions. The policy sequentially evaluates hypotheses via closed-loop environment interaction and refines its plans as it gathers new information. We deploy CLUE on a Boston Dynamics Spot across three real indoor and outdoor environments spanning 15 tasks that require object disambiguation, functional inference, and occlusion reasoning. CLUE achieves a success rate within 7 percentage points of an oracle policy and outperforms an LLM-enabled planner without closed-loop feedback by a 4x margin. Supporting experiments demonstrate that simply building and then querying a language-enriched map is insufficient to resolve complex contextual planning tasks; these approaches achieve roughly one third the success rate of CLUE while requiring over 10x more VLM tokens. We provide additional information at https://zacravichandran.github.io/CLUE.

Chinese Translation

基础模型为机器人提供了解释自然语言和推理环境上下文的能力，但大多数语言条件策略假设目标是明确的，并且任务相关信息是通过先前的地图提前提供的。在不熟悉的环境中处理不明确的任务会带来较高的上下文不确定性：机器人必须共同推断什么构成任务成功，什么构成相关信息，以及这些信息存在于何处（或是否存在）。我们通过CLUE（闭环上下文不确定性解决）来解决这些局限性，这是一个针对自然语言中不明确任务主动解决上下文不确定性的框架。CLUE使用基于大语言模型（LLM）派生的策略来假设任务相关概念和潜在计划。然后，它使用在线构建的语言嵌入地图，将这些假设落实为行动。该策略通过闭环环境交互顺序评估假设，并在收集新信息时不断完善其计划。我们在波士顿动力公司的Spot机器人上部署了CLUE，涵盖了三个真实的室内和室外环境，涉及15个需要对象消歧、功能推理和遮挡推理的任务。CLUE的成功率在一个oracle策略的基础上仅相差7个百分点，并且比没有闭环反馈的LLM启用规划器高出4倍的优势。支持实验表明，仅仅构建并查询一个语言增强的地图不足以解决复杂的上下文规划任务；这些方法的成功率大约是CLUE的三分之一，同时需要超过10倍的VLM令牌。我们提供了更多信息，网址为 https://zacravichandran.github.io/CLUE。

View on arXiv Download PDF AI Translation

cs.RO / 4 / 2609.30436

WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving

WALT：学习与世界模型对齐的潜在轨迹用于自主驾驶

Jia, Mingkai, Guo, Jiaxin, Shu, Zhijian, Xu, Jiawei, Li, Mingxiao, Cheng, Jintao, Tan, Ping, Yin, Wei

Abstract

Driving world models learn rich predictive representations of the surrounding environment from visual observations, yet accurate visual prediction does not necessarily translate into effective trajectory planning. We argue that a key bottleneck lies in the mismatch between visual world states and raw geometric trajectories, which may limit the planner's ability to exploit action-relevant semantics encoded by the world model. To address this issue, we propose World-Model Alignment for Latent Trajectories (WALT), which learns a compact generative trajectory latent space by transferring information from a frozen pretrained driving world model without modifying the world model itself. Rather than directly generating raw waypoints, WALT maps them into compact representations through a dual-branch trajectory autoencoder and transfers semantic knowledge from the frozen visual world model into this trajectory space, encouraging the learned action representation to capture scene-level cues relevant to future motion and planning. Beyond our proposed formulation, we systematically study latent learning based on Joint-Embedding Predictive Architectures (JEPA) and feature alignment following Representation Alignment (REPA) to investigate how trajectory-only representation learning affects downstream planning. We evaluate WALT on the NAVSIM benchmarks. Relative to the raw-waypoint baseline, WALT improves PDMS from 89.4 to 89.8 on NAVSIMv1 and EPDMS from 87.3 to 87.9 on NAVSIMv2 while reducing trajectory planner FLOPs by 30.5%. These results suggest that preserving world representations while extracting action-relevant information provides an effective interface for world-model-based trajectory planning.

Chinese Translation

驾驶世界模型从视觉观察中学习周围环境的丰富预测表示，但准确的视觉预测并不一定能转化为有效的轨迹规划。我们认为，关键瓶颈在于视觉世界状态与原始几何轨迹之间的不匹配，这可能限制规划者利用世界模型编码的与动作相关的语义。为了解决这个问题，我们提出了潜在轨迹的世界模型对齐（World-Model Alignment for Latent Trajectories，WALT），该方法通过从一个冻结的预训练驾驶世界模型中转移信息，学习一个紧凑的生成轨迹潜在空间，而不修改世界模型本身。WALT并不是直接生成原始的路径点，而是通过双分支轨迹自编码器将其映射为紧凑表示，并将冻结的视觉世界模型中的语义知识转移到这个轨迹空间，鼓励学习到的动作表示捕捉与未来运动和规划相关的场景级线索。除了我们提出的公式外，我们还系统地研究了基于联合嵌入预测架构（Joint-Embedding Predictive Architectures，JEPA）的潜在学习和基于表示对齐（Representation Alignment，REPA）的特征对齐，以探讨仅基于轨迹的表示学习如何影响下游规划。我们在NAVSIM基准上评估了WALT。相较于原始路径点基线，WALT在NAVSIMv1上将PDMS从89.4提高到89.8，在NAVSIMv2上将EPDMS从87.3提高到87.9，同时将轨迹规划器的FLOPs减少了30.5%。这些结果表明，保留世界表示的同时提取与动作相关的信息，为基于世界模型的轨迹规划提供了有效的接口。

View on arXiv Download PDF AI Translation

cs.RO / 5 / 2609.30459

VkVIO: Cross-platform GPU Acceleration for Visual-Inertial Odometry with Vulkan

VkVIO：基于Vulkan的跨平台视觉惯性里程计GPU加速

Hoffmann, Ole, de Mayo, Mateo, Cremers, Daniel

Abstract

Perception in robotics and XR fundamentally relies on good state estimation. Visual-inertial odometry (VIO) and Simultaneous Localization and Mapping (VI-SLAM) are proven ways of achieving this goal in a cost-effective and accurate manner. Efficiency in these systems allows for smaller, cooler, and lighter devices. GPU acceleration is a natural approach for reducing latency, thanks to their wide availability in platforms like embedded computers, mobile phones, and XR headsets. However, previous works in the literature have limited themselves to the use of CUDA for this task, significantly reducing deployment options to a single vendor. We instead leverage the vendor-agnostic Vulkan API, originally designed for the strict performance requirements of 3D graphics applications. In this work, we present VkVIO, the first, to the best of our knowledge, cross-platform GPU-accelerated VIO method. We provide state-of-the-art accuracy with causal estimates required for real-time operation. We deploy VkVIO on a diverse range of devices spanning a workstation, a laptop, and an extremely inexpensive single-board computer, while outperforming CUDA-based systems on the same hardware. VkVIO enables possibilities for low-latency, low-power, and low-cost VIO in robotics and XR.

Chinese Translation

机器人技术和扩展现实（XR）中的感知根本上依赖于良好的状态估计。视觉惯性里程计（VIO）和同时定位与地图构建（VI-SLAM）是以成本效益高且准确的方式实现这一目标的有效方法。这些系统的高效性使得设备可以更小、更凉爽且更轻便。由于GPU在嵌入式计算机、手机和XR头显等平台上的广泛可用性，GPU加速成为减少延迟的自然选择。然而，文献中的先前研究仅限于使用CUDA进行此任务，显著减少了部署选项，仅限于单一供应商。我们则利用了供应商无关的Vulkan API，该API最初是为满足3D图形应用的严格性能要求而设计的。在本研究中，我们提出了VkVIO，至我们所知，这是首个跨平台的GPU加速VIO方法。我们提供了用于实时操作所需的因果估计的最先进的准确性。我们在多种设备上部署了VkVIO，包括工作站、笔记本电脑和极其廉价的单板计算机，同时在相同硬件上超越了基于CUDA的系统。VkVIO为机器人技术和XR中的低延迟、低功耗和低成本的VIO提供了可能性。

View on arXiv Download PDF AI Translation

cs.RO / 6 / 2609.30460

Realizability Is Not Enough: Encoding, Liveness, and Auditing of Synthesized Robot Supervisors

可实现性不足：合成机器人监督者的编码、活性和审计

Conner, David C., Luzier, Joshua, Doyle, William J., Faith, Emma R., Kooiker, Aubrie B., Farney, Andrew J., Fox, Sebastian, Grimes, Evangelina, Conner, Ian G., Bloom, Kyle

Abstract

High-level robotic supervisors coordinate capabilities whose reported outcomes determine the robot's next action. Reactive synthesis can generate such supervisors with formal guarantees, but deployment requires more than proving a Generalized Reactivity (1) (GR(1)) specification realizable. Designers must encode failure-prone capabilities, choose liveness assumptions that match retry intent, audit strategies, and translate them into robot software. We present an open-source pipeline for Robot Operating System (ROS) 2 Flexible Behavior Engine (FlexBE) supervisors that generates capability-based GR(1) specifications, analyzes assumptions before synthesis, audits strategies, reduces states with a behavior-preservation proof, and emits executable state machines. Across four case studies (six comparisons), including hardware on two quadcopter platforms, we compare enumerated and one-hot encodings and two liveness formulations. Under the tested backend, enumerated encoding usually synthesizes faster, although fewer propositions do not reliably predict smaller controllers or lower symbolic cost. System-Goal without pending memory is the only liveness treatment confirmed to yield executable controllers under both encodings across the reported grid; Fair-Outcome can permit realizable cycles without designer-intended completion. For this backend and model, we recommend enumerated encoding with System-Goal and auditing every realized strategy, since proposition count and realizability do not measure deployability. The auditor is sound and complete for four structural defect classes (protocol violations, deadlocks, bounded-failure violations, goal-unreachable traps) but is not a general liveness verifier, and the reduction preserves capability-level behavior. Together, these stages narrow the gap between formal realizability and controllers that pass protocol and structural-progress checks.

Chinese Translation

高级机器人监督者协调能力，其报告的结果决定机器人的下一步行动。反应合成可以生成具有形式保证的此类监督者，但部署不仅仅需要证明广义反应性 (1) (GR(1)) 规范是可实现的。设计者必须编码易出错的能力，选择与重试意图相匹配的活性假设，审计策略，并将其转化为机器人软件。我们提出了一个开源管道，用于机器人操作系统 (Robot Operating System, ROS) 2 灵活行为引擎 (Flexible Behavior Engine, FlexBE) 监督者，该管道生成基于能力的 GR(1) 规范，在合成之前分析假设，审计策略，通过行为保持证明减少状态，并发出可执行状态机。在四个案例研究（六个比较）中，包括在两个四旋翼平台上的硬件，我们比较了枚举编码和独热编码以及两种活性表述。在测试的后端下，枚举编码通常合成得更快，尽管较少的命题并不能可靠地预测较小的控制器或较低的符号成本。没有待处理内存的系统目标是唯一确认在报告的网格中两种编码下都能产生可执行控制器的活性处理；公平结果可能允许实现循环而不需设计者意图的完成。对于该后端和模型，我们推荐使用系统目标的枚举编码，并审计每个实现的策略，因为命题数量和可实现性并不能衡量可部署性。审计器对于四类结构缺陷（协议违反、死锁、有界失败违反、目标不可达陷阱）是健全且完整的，但不是通用的活性验证器，且该减少保持能力级行为。这些阶段共同缩小了形式可实现性与通过协议和结构进展检查的控制器之间的差距。

View on arXiv Download PDF AI Translation

cs.RO / 7 / 2609.30461

DGT-Map: Directional Global Traversability Mapping Utilizing Multi-Task Learning for Heterogeneous Vehicles

DGT-Map：利用多任务学习为异构车辆构建方向性全球可通行性地图

Singh, Jaskrit, Noori, Kashif K., Xiao, Jing, Chamzas, Constantinos

Abstract

Off-road traversability is direction-dependent and vehicle specific, yet most global maps assign a single isotropic cost to each location. Existing learned estimators are also commonly trained independently for each vehicle; this preserves vehicle-specific behavior but prevents vehicles from sharing common terrain representations. DGT-MAP addresses both limitations through a self-supervised framework that learns global, directional, and vehicle-conditioned traversability costmaps from RGB-D observations and locomotion signals. A shared multi-task backbone learns common terrain features across training vehicles while vehicle-specific prediction heads preserve platform-dependent responses. At inference, DGT-MAP produces a heading-indexed costmap that can be used by a direction-aware planner. We evaluate DGT-MAP in simulation by integrating it into a Hybrid A* navigation stack and measuring downstream task success on challenging terrains, including slopes that are traversable downhill but not uphill and a ridge obstacle that is traversable by some vehicles, but not by others. Across evaluated tasks, DGT-MAP achieves the highest or tied-highest navigation success rate when compared against geometric, binary, and learned direction-agnostic baselines.

Chinese Translation

越野可通行性依赖于方向和特定车辆，但大多数全球地图为每个位置分配单一的各向同性成本。现有的学习估计器通常独立为每辆车进行训练；这种方法保留了车辆特定的行为，但阻碍了车辆共享共同的地形表示。DGT-MAP通过一个自监督框架解决了这两个限制，从RGB-D观测和运动信号中学习全球、方向性和车辆条件的可通行性成本图。一个共享的多任务主干网络学习训练车辆之间的共同地形特征，而车辆特定的预测头则保留了平台依赖的响应。在推理阶段，DGT-MAP生成一个以航向为索引的成本图，可供方向感知规划器使用。我们通过将DGT-MAP集成到混合A*导航框架中，在模拟中评估其性能，并测量在具有挑战性的地形上的下游任务成功率，包括可向下通行但不可向上通行的坡道，以及某些车辆可通行但其他车辆不可通行的山脊障碍。在评估的任务中，DGT-MAP在与几何、二元和学习的方向无关基线相比时，达到了最高或并列最高的导航成功率。

View on arXiv Download PDF AI Translation

cs.RO / 8 / 2609.30462

Policy-Calibrated DAgger: Offline Calibrated Noise Injection for Imitation Learning

政策校准的DAgger：用于模仿学习的离线校准噪声注入

Wang, Jenny, Kantor, George

Abstract

Policies trained with imitation learning can accumulate errors over time, causing the robot to drift outside the training distribution. Existing methods mitigate this covariate shift by collecting additional data where the policy fails or is likely to fail. The first places the robot in unsafe conditions and the second requires choosing an appropriate noise distribution to collect new expert demonstrations under that noise. We propose Policy-Calibrated DAgger, a method that makes use of the properties of recent generative policies to estimate the policy's noise offline by using its own predicted action distribution. We measure a diffusion policy's spread of predicted actions at observations along the expert trajectory and measure its closed-loop error relative to a recorded trajectory. To address issues with measuring error in a multimodal action space, we guide the policy towards the trajectory during closed-loop control through partial denoising, and use properties of a diffusion model to unnormalize the measured error as if we did not guide it. We experiment in a scenario where a robot is tasked to reach an engine lever in a cluttered and narrow environment and show results in a 3D photorealistic simulator and a 2D planar reacher environment. We show that our method surpasses policies trained with dataset aggregation without noising and matches the performance of the best noise level in hindsight, without requiring a sweep over noise levels.

Chinese Translation

通过模仿学习训练的策略可能会随着时间的推移积累错误，导致机器人偏离训练分布。现有方法通过收集策略失败或可能失败的额外数据来减轻这种协变量转移。第一种方法将机器人置于不安全的条件下，第二种方法则需要选择合适的噪声分布，以在该噪声下收集新的专家演示。我们提出了政策校准的DAgger（Policy-Calibrated DAgger），该方法利用最近生成策略的特性，通过使用其自身预测的动作分布离线估计策略的噪声。我们测量扩散策略在专家轨迹上的观察点处预测动作的分布，并测量其相对于记录轨迹的闭环误差。为了解决在多模态动作空间中测量误差的问题，我们在闭环控制过程中通过部分去噪引导策略朝向轨迹，并利用扩散模型的特性将测量的误差反归一化，仿佛我们没有引导它。我们在一个场景中进行实验，任务是让机器人在一个杂乱且狭窄的环境中到达发动机操纵杆，并在3D逼真模拟器和2D平面到达环境中展示结果。我们表明，我们的方法超越了未加噪声的数据集聚合训练的策略，并在事后与最佳噪声水平的性能相匹配，而无需对噪声水平进行全面搜索。

View on arXiv Download PDF AI Translation

cs.RO / 9 / 2609.30479

Learning-Based Pressure Predictive Control of a Vertebraic Soft Robotic Tail

基于学习的脊椎软体机器人尾部压力预测控制

Yang, Wenjian, Huang, Nan, Nie, Yukang, Chen, Fang, Chi, Wanchao, Dai, Jiansheng, Liu, Sicong

Abstract

Soft robots have attracted much attention for their safe human-robot interaction and flexibility, but the typical continuum structure and nonlinear material behavior make the kinematics modelling complex, especially in non-static motions. In this work, we proposed an LSTM-based pressure predictive control (PPC) for the motion control of a vertebraic soft robotic tail and the coordination with a quadruped robot. The PPC consists of an inverse kinematics (IK) model, a forward kinematics (FK) model and a pressure compensation (P-comp) model, and achieves non-static and quasi-static motion control of the tail. Compared with the IK-only model, the average RMSE of the PPC's simulation trajectories reduces by 69.8%, when executing target trajectories. In the coordinated motions of the soft tail quadruped, using a prediction data set to train the PPC enables next-moment action prediction and reduces computation time by 60.9%, which enhances the real-time response of the tail to match the quadruped torso's moving rate. The PPC provides a simple and effective method to model the soft tail for both non-static and quasi-static motion control, and grants the soft tail quadruped with the functionality of interacting with the environment.

Chinese Translation

软体机器人因其安全的人机交互和灵活性而受到广泛关注，但典型的连续结构和非线性材料行为使得运动学建模变得复杂，尤其是在非静态运动中。在本研究中，我们提出了一种基于长短期记忆网络（LSTM）的压力预测控制（PPC），用于脊椎软体机器人尾部的运动控制及与四足机器人的协调。PPC由逆运动学（IK）模型、正运动学（FK）模型和压力补偿（P-comp）模型组成，实现了尾部的非静态和准静态运动控制。与仅使用IK模型相比，PPC在执行目标轨迹时的平均均方根误差（RMSE）降低了69.8%。在软尾四足机器人的协调运动中，使用预测数据集训练PPC可以实现下一时刻动作的预测，并将计算时间减少60.9%，增强了尾部对四足机器人躯干移动速率的实时响应。PPC为软尾的非静态和准静态运动控制提供了一种简单有效的方法，并赋予软尾四足机器人与环境交互的功能。

View on arXiv Download PDF AI Translation

cs.RO / 10 / 2609.30495

Memory-Aware Multi-Sensor Perception for Efficient and Safe Navigation in Dynamic Environments

面向动态环境的高效安全导航的记忆感知多传感器感知

Li, Jingshuo, Xue, Yifan, Li, Yifei, Aditya, Shubhodeep Shiv, Figueroa, Nadia

Abstract

Autonomous navigation in previously unseen environments requires effective perception, persistent environmental representation, and collision avoidance while maintaining progress toward a goal. Existing perception-based methods often rely on prior maps or short-horizon observations, limiting their ability to exploit previously observed structure. We propose a memory-aware multi-sensor navigation framework that integrates LiDAR and RGB perception, online distance-field representation learning, and a stage-adaptive Modulated Control Barrier Function Quadratic Program (MCBF-QP). The framework persistently represents static infrastructure while tracking dynamic obstacles, enabling the MCBF-QP controller to exploit previously observed geometry for obstacle circumvention and adapt its safety constraints and guidance to local conditions. Experiments in complex indoor and outdoor environments demonstrate improved navigation efficiency and goal-reaching performance while maintaining collision avoidance in narrow passages and around dynamic obstacles.

Chinese Translation

在以前未见的环境中进行自主导航需要有效的感知、持久的环境表征以及在保持朝向目标进展的同时避免碰撞。现有的基于感知的方法通常依赖于先前的地图或短期观察，这限制了它们利用先前观察到的结构的能力。我们提出了一种记忆感知的多传感器导航框架，该框架集成了激光雷达（LiDAR）和RGB感知、在线距离场表征学习，以及阶段自适应的调制控制障碍函数二次规划（Modulated Control Barrier Function Quadratic Program, MCBF-QP）。该框架在跟踪动态障碍物的同时持久地表征静态基础设施，使得MCBF-QP控制器能够利用先前观察到的几何信息进行障碍规避，并根据当地条件调整其安全约束和引导策略。在复杂的室内和室外环境中的实验表明，该方法在保持在狭窄通道和动态障碍物周围避免碰撞的同时，提高了导航效率和目标到达性能。

View on arXiv Download PDF AI Translation

cs.RO / 11 / 2609.30506

Tactile Sensing Array for Multi-Phalanx Sensing in Humanoid Hands

类人手多指节触觉传感阵列

Adwani, Neel, Akash, Muhaiminul Islam, Bhattacharya, Rituja, Wang, Cong

Abstract

Humanoid hands require tactile feedback across the whole finger, not just the fingertip, to grasp and manipulate objects properly. Vision and proprioception alone cannot reliably provide this information, particularly when the hand's own fingers occlude the camera's view of the grasp. We present a low-cost tactile array for a humanoid finger, made from Velostat and conductive tape. The fingertip carries seven contact points including a 2x3 matrix wrapped across its front, left, and right faces, and a separate contact point at the tip. The proximal and middle phalanges each carry a single front-facing contact line. We measure the sensor's hysteresis and recovery time after release, through repeated loading and press-release tests. We also test a compliant, 3D-printed contact structure with a gap and a bump, inspired by similar designs in prior work, and show it cuts recovery time by 74% compared to a flush-contact baseline. We then show the sensor can produce distinct activation patterns for different contact geometries (flat, edge, corner) at the fingertip, and that it registers contact across all three phalanges during a grip. Finally, we discuss the limits of our fabrication changes and point to software-based compensation as a promising way to more directly fix the remaining hysteresis in the future.

Chinese Translation

类人手需要在整个手指上提供触觉反馈，而不仅仅是指尖，以便正确抓取和操作物体。仅依靠视觉和本体感觉无法可靠地提供这些信息，特别是在手指遮挡了摄像头对抓取的视野时。我们提出了一种低成本的类人手指触觉阵列，由Velostat和导电胶带制成。指尖上设有七个接触点，包括一个覆盖其前面、左侧和右侧的2x3矩阵，以及一个位于指尖的单独接触点。近端和中间指节各有一个朝前的接触线。我们通过重复加载和按压-释放测试来测量传感器的滞后和释放后的恢复时间。我们还测试了一种柔性3D打印接触结构，具有间隙和凸起，灵感来源于先前工作的类似设计，并显示其恢复时间比平面接触基线减少了74%。然后，我们展示了传感器能够为不同的接触几何形状（平面、边缘、角落）在指尖产生不同的激活模式，并且在抓握过程中能够在所有三个指节上注册接触。最后，我们讨论了我们制造变化的局限性，并指出基于软件的补偿作为未来更直接修复剩余滞后的有希望的方法。

View on arXiv Download PDF AI Translation

cs.RO / 12 / 2609.30521

Aerial Manipulation in the Wild with Onboard Perception, Policy Learning, and Whole-Body Control

野外环境中的空中操作：基于机载感知、策略学习和全身控制

Zhan, Yuanzhu, Jiang, Yufei, Zhang, Zemu, Geng, Junyi

Abstract

Aerial manipulation in outdoor environments remains challenging due to the simultaneous requirements of reliable state estimation, stable aerial motion, and precise manipulation under external disturbances. In this work, we present a real-world outdoor aerial manipulation framework that integrates imitation learning, onboard LiDAR-inertial state estimation, and whole-body model predictive control. A Diffusion Policy is trained from manipulation demonstrations to generate desired end-effector motions from onboard observations. These learned commands are executed by a whole-body MPC that jointly coordinates the aerial platform and manipulator to realize the desired end-effector trajectory. To eliminate reliance on external motion-capture infrastructure, the platform employs onboard LiDAR-inertial odometry for state estimation during outdoor operation. We validate the complete framework on a physical aerial manipulator and demonstrate successful execution of outdoor manipulation tasks. The experimental results show that demonstration-driven manipulation policies can be effectively integrated with onboard state estimation and model-based whole-body control to enable aerial manipulation beyond controlled indoor environments.

Chinese Translation

在户外环境中进行空中操作仍然面临挑战，因为需要同时满足可靠的状态估计、稳定的空中运动和在外部干扰下的精确操作。在本研究中，我们提出了一个现实世界的户外空中操作框架，该框架集成了模仿学习、机载激光雷达-惯性状态估计和全身模型预测控制。通过操作演示训练的扩散策略（Diffusion Policy）能够根据机载观测生成期望的末端执行器运动。这些学习到的指令由全身模型预测控制（MPC）执行，协调空中平台和操控器以实现期望的末端执行器轨迹。为了消除对外部运动捕捉基础设施的依赖，该平台在户外操作期间采用机载激光雷达-惯性里程计进行状态估计。我们在物理空中操控器上验证了完整框架，并成功演示了户外操作任务的执行。实验结果表明，基于演示的操作策略可以有效地与机载状态估计和基于模型的全身控制相结合，从而实现超越受控室内环境的空中操作。

View on arXiv Download PDF AI Translation

cs.RO / 13 / 2609.30523

Containing Behavioral Cascades from Manipulated Claims in LLM-Powered Multi-Robot Systems

控制来自操纵声明的行为级联在大语言模型驱动的多机器人系统中

Khalid, Waleed Bin, Min, Byung-Cheol

Abstract

Large language model (LLM)-powered multi-robot systems are vulnerable to semantic manipulation: an accepted false world-state claim can trigger a fleet-wide behavioral cascade, causing unnecessary replanning, increased path costs, congestion, or apparent mission infeasibility. Conditioning on a successful manipulation, we propose an active verification framework that contains its downstream effects before they propagate across the fleet. A dedicated verification module generates a structured Verify-Adapt-Hold plan: selected robots inspect consequential regions, a limited subset provisionally adapts when necessary, and the remaining robots retain their trusted plans. We evaluate the framework in a multi-robot transportation environment using injected false obstacle claims across different impacts and team sizes. Evaluation measures cascade containment, Sum-of-Costs, makespan, and coverage ratio. Results show that treating post-compromise verification as a team-level planning problem, rather than a binary trust decision, effectively limits the cascading physical consequences of semantic manipulation.

Chinese Translation

大语言模型（LLM）驱动的多机器人系统易受语义操纵的影响：一个被接受的虚假世界状态声明可以触发全队范围内的行为级联，导致不必要的重新规划、路径成本增加、拥堵或明显的任务不可行性。在成功操纵的条件下，我们提出了一种主动验证框架，以在其下游影响传播到整个队伍之前控制这些影响。一个专门的验证模块生成一个结构化的验证-适应-保持（Verify-Adapt-Hold）计划：选定的机器人检查重要区域，有限的子集在必要时进行临时适应，其余机器人保持其可信的计划。我们在一个多机器人运输环境中评估该框架，使用不同影响和团队规模下注入的虚假障碍声明。评估指标包括级联控制、成本总和、完成时间和覆盖率。结果表明，将后妥协验证视为团队级规划问题，而不是二元信任决策，有效限制了语义操纵的级联物理后果。

View on arXiv Download PDF AI Translation

cs.RO / 14 / 2609.30530

A Long-Legged, Direct-Drive Monopedal Robot Achieves Exceptional Jump Height

一款长腿直接驱动的单足机器人实现了卓越的跳跃高度

Na, Gihyeok, Yim, Justin K.

Abstract

We demonstrate a jumping robot that reaches high (7.6 m) and fast (190 ms stance time) jumps from a single long leg driven by a direct-drive transmission, without elastic energy storage. At 281 g, it achieves the highest jump yet reported for an electrically actuated, spring-free system. The leg uses a new fabric-wrap transmission that provides a variable mechanical advantage, keeping a small electric motor near its peak power output through the stroke while bracing the long, lightweight leg against buckling. A balancing module at the top of the leg uses small propellers to control the leg's orientation on the ground and in the air, where the long leg provides a large moment arm for the control torques. The robot is validated outdoors with vertical jumps, attitude control on the ground and in flight, and tilted jumps.

Chinese Translation

我们展示了一种跳跃机器人，它能够从单根长腿上实现高达7.6米和快速（190毫秒支撑时间）的跳跃，且不依赖弹性储能。该机器人重281克，创造了电动驱动、无弹簧系统的最高跳跃记录。其腿部采用了一种新型的织物包裹传动装置，提供可变的机械优势，使小型电动机在整个行程中保持接近其峰值功率输出，同时支撑着长而轻的腿以防止屈曲。腿部顶部的平衡模块使用小型螺旋桨来控制腿在地面和空中的方向，其中长腿为控制扭矩提供了较大的力臂。该机器人在户外进行了验证，包括垂直跳跃、地面和飞行中的姿态控制，以及倾斜跳跃。

View on arXiv Download PDF AI Translation

cs.RO / 15 / 2609.30533

ST-pRRTC: Parallel Space-Time RRT-C with Adaptive Goal-Time Forests

ST-pRRTC：具有自适应目标时间森林的并行时空RRT-C

Zhang, Duo, Li, Jintong, Huang, Junshan, Yu, Jingjin

Abstract

We propose ST-pRRTC, a GPU-parallel space- time RRT-Connect motion planner for problems with known obstacle trajectories and unspecified arrival time. Searching over many arrival times broadens temporal coverage but divides a finite planning budget among more backward trees. To address the challenge, ST-pRRTC builds a shared forward tree and an adaptive forest of backward goal-time trees. Its interval root formulation samples goal arrival times continuously and guarantees probabilistic completeness and asymptotic arrival- time optimality under the stated assumptions in a bounded time domain. The practical root recycling policy has no such guar- antees. It adapts a fixed number of backward trees, replacing later roots while retaining useful search progress. Experiments on three dynamic benchmarks show that both variants achieve lower mean first-solution times and earlier mean final arrivals than ST-RRT* and SI-RRT on problems solved by all compared methods. Further experiments demonstrate the benefit of recy- cling over broad arrival-time ranges. Real-robot demonstrations show root-recycling ST-pRRTC planning motions for a UR5e among moving Crazyflie quadrotors.

Chinese Translation

我们提出了ST-pRRTC，一种基于GPU的并行时空RRT-Connect运动规划器，适用于已知障碍物轨迹且到达时间未指定的问题。对多个到达时间的搜索扩展了时间覆盖范围，但将有限的规划预算分配给更多的反向树。为了解决这一挑战，ST-pRRTC构建了一个共享的前向树和一个自适应的反向目标时间树森林。其区间根公式连续采样目标到达时间，并在有限时间域内保证在所述假设下的概率完整性和渐近到达时间最优性。实际的根回收策略没有这样的保证。它适应固定数量的反向树，替换后来的根，同时保留有用的搜索进展。在三个动态基准测试上的实验表明，这两种变体在所有比较方法解决的问题上实现了比ST-RRT*和SI-RRT更低的平均首次解决时间和更早的平均最终到达时间。进一步的实验展示了在广泛到达时间范围内回收的好处。真实机器人演示显示了根回收ST-pRRTC在移动Crazyflie四旋翼之间为UR5e规划运动。

View on arXiv Download PDF AI Translation

cs.RO / 16 / 2609.30543

GraspTwin: Zero-Shot Task-Oriented Grasp Optimization via a Digital Twin

GraspTwin：通过数字双胞胎实现零-shot任务导向的抓取优化

Evans, Daniel J., Dai, Yinlong, Stepputtis, Simon, Losey, Dylan P.

Abstract

As robots transition from structured factory settings into homes, they are required to interact with an ever-increasing variety of objects. Many tasks require grasping, and often it is not sufficient to just pick up the target object. Consider a task like "pouring coffee" --- to facilitate the subsequent pouring, the robot should grasp the mug by its handle. Existing learning-based approaches for grasping either find robust and collision-free grasps that are largely agnostic to the task (e.g., picking up the mug by its rim), or leverage foundation models to propose task-appropriate grasp locations that lack fine-grained physical grounding (e.g., reaching for and missing the handle). In this work, we bridge these approaches with a real-to-sim-to-real framework. Based on a single RGB-D observation, we construct a digital twin of the environment, query a large foundation model to propose grasps that align with the object's affordances and task description, and then optimize the proposals to ensure robustness and plausibility before executing the result on the real robot. Our key insight is that the grasp proposals of the foundation model should be regarded as semantic priors that serve as seeds for local, gradient-free optimization. We leverage Bayesian optimization with Thompson sampling to draw batches of nearby poses, which are subsequently evaluated in parallel under domain-randomized physics rollouts. The resulting grasp is both task-oriented and physically feasible for execution by the robot arm. Our full zero-shot real-world transfer only takes a few minutes and improves task-oriented grasping success by up to 33% as compared to other state-of-the-art pipelines. Our code is available here: https://github.com/VT-Collab/GraspTwin/

Chinese Translation

随着机器人从结构化的工厂环境转向家庭，它们需要与越来越多样化的物体进行交互。许多任务需要抓取，而仅仅拾起目标物体往往是不够的。考虑一个像“倒咖啡”的任务——为了便于后续的倒出，机器人应该通过杯子的把手来抓取杯子。现有的基于学习的抓取方法要么找到稳健且无碰撞的抓取方式，这些方式在很大程度上与任务无关（例如，从杯子的边缘拾起），要么利用基础模型提出适合任务的抓取位置，但缺乏细致的物理基础（例如，伸手去抓却错过了把手）。在本研究中，我们通过一个真实-模拟-真实的框架将这些方法结合起来。基于单个RGB-D观测，我们构建了环境的数字双胞胎，查询一个大型基础模型以提出与物体的可用性和任务描述相一致的抓取方案，然后优化这些提案，以确保在真实机器人上执行前的稳健性和合理性。我们的关键见解是，基础模型的抓取提案应被视为语义先验，作为局部无梯度优化的种子。我们利用贝叶斯优化与汤普森采样来提取一批附近的姿态，这些姿态随后在领域随机化的物理模拟中并行评估。最终的抓取既面向任务又在物理上可行，适合机器人手臂执行。我们的全零-shot真实世界转移只需几分钟，与其他最先进的管道相比，任务导向的抓取成功率提高了多达33%。我们的代码可在此获取：https://github.com/VT-Collab/GraspTwin/

View on arXiv Download PDF AI Translation

cs.RO / 17 / 2609.30554

Privacy-Preserving Prompted Policy Search for Robotic Control

隐私保护的提示策略搜索用于机器人控制

Irshayyid, Ali, Lin, Feng, Li, Chong, Chen, Jun

Abstract

Large language models (LLMs) have recently demonstrated promising capabilities as in-context policy optimizers for Reinforcement Learning (RL), enabling policy search driven by both numerical reward signals and natural language reasoning. However, deploying such methods in practice requires transmitting raw policy parameters and rewards history to cloud-based LLM APIs, exposing proprietary control strategies to third-party service providers. To address this issue, this paper introduces Privacy-Preserving Prompted Policy Search (PP-ProPS), a framework that enables LLM-guided policy optimization while keeping policy and environmental parameters confidential. PP-ProPS encodes policy parameters and reward values using secret client-side transformations before they are included in each API request, ensuring that the LLM provider observes only encoded policy parameters and scaled reward information. Furthermore, unlike Vanilla ProPS, the proposed framework does not require the true optimal episodic return to be known or disclosed to the LLM. Beyond protecting the optimization data, PP-ProPS improves the search process in two ways. First, it provides the LLM with individual reward components instead of only a single total return, offering more informative feedback about each candidate policy. Second, it uses a bounded history that prevents the prompt from growing indefinitely, improving search with high-dimensional policies and supporting the use of open-weight LLMs. The proposed PP-ProPS is evaluated on both continuous and discrete control problems spanning Multi-Joint dynamics with Contact (MuJoCo) locomotion, classic control, highway driving, and robotic arm manipulation. Compared to Vanilla ProPS, the proposed PP-ProPS outperforms ProPS in seven of the ten evaluated tasks, and surpasses conventional RL methods including PPO, SAC, and TRPO, in five of the six tasks.

Chinese Translation

大型语言模型（LLMs）最近展示了作为强化学习（RL）中的上下文策略优化器的良好能力，使得策略搜索能够同时受到数值奖励信号和自然语言推理的驱动。然而，在实践中部署此类方法需要将原始策略参数和奖励历史传输到基于云的LLM API，这使得专有的控制策略暴露给第三方服务提供商。为了解决这一问题，本文提出了隐私保护的提示策略搜索（PP-ProPS）框架，该框架在保持策略和环境参数机密的同时，支持LLM引导的策略优化。PP-ProPS在每个API请求中包含之前，使用秘密的客户端转换对策略参数和奖励值进行编码，确保LLM提供者仅观察到编码后的策略参数和缩放的奖励信息。此外，与Vanilla ProPS不同，所提出的框架不需要知道或披露真实的最优情节回报。除了保护优化数据外，PP-ProPS还通过两种方式改善搜索过程。首先，它向LLM提供单独的奖励组成部分，而不仅仅是一个总回报，从而提供关于每个候选策略的更具信息量的反馈。其次，它使用有限的历史记录，防止提示无限增长，从而改善高维策略的搜索，并支持开放权重LLM的使用。所提出的PP-ProPS在包括多关节动力学与接触（MuJoCo）运动、经典控制、高速公路驾驶和机器人手臂操作等连续和离散控制问题上进行了评估。与Vanilla ProPS相比，所提出的PP-ProPS在十个评估任务中的七个任务中表现优于ProPS，并在六个任务中的五个任务中超越了传统的RL方法，包括PPO、SAC和TRPO。

View on arXiv Download PDF AI Translation

cs.RO / 18 / 2609.30557

Auditing Latent-Space Monitors for Autonomous Driving

审计自主驾驶的潜在空间监测器

Advani, Nikhil Kamalkumar, Hogale, Vishwajeet Shivaji, Kumar, Saurav

Abstract

Runtime failure monitors can use a model's internal representations to anticipate failures. We audit this monitoring strategy across two autonomous-driving tasks: online vectorized map generation with LaneSegNet and end-to-end planning with VAD. We find that frame-level errors are predictable at inference in both tasks. For LaneSegNet, a supervised latent probe reaches Area Under the Receiver Operating Characteristic curve (AUROC) 0.780 for high Chamfer error; to our knowledge, this is the first post-hoc frame-level failure monitor for online vectorized map generation. For VAD, a supervised planning-latent probe reaches AUROC 0.868 for mean-ADE failure. Our audit shows that internal access is not necessary for strong failure prediction. A monitor using only LaneSegNet's prediction outputs reaches AUROC 0.825, while for VAD, ego state, driving command, and the planner's predicted trajectory reach 0.924 on the same mean-ADE endpoint. Adding latent features to either baseline yields no statistically resolved improvement. This observation persists across a broad suite of planning failure endpoints, including endpoints whose labels depend on geometry unavailable to the non-latent baseline. Thus, predicting failure from an internal representation does not establish that the representation provides useful information beyond observable inputs and outputs. We propose an evaluation protocol for testing the incremental value of latent access and release our per-frame failure endpoint labels.

Chinese Translation

运行时故障监测器可以利用模型的内部表示来预测故障。我们在两个自主驾驶任务中审计这一监测策略：使用 LaneSegNet 的在线向量化地图生成和使用 VAD 的端到端规划。我们发现，在这两个任务中，帧级错误在推理时是可预测的。对于 LaneSegNet，一个监督的潜在探测器在高 Chamfer 错误的情况下达到了 0.780 的接收者操作特征曲线下面积（AUROC）；据我们所知，这是在线向量化地图生成的第一个事后帧级故障监测器。对于 VAD，一个监督的规划潜在探测器在平均绝对误差（mean-ADE）故障上达到了 0.868 的 AUROC。我们的审计表明，内部访问并不是强故障预测的必要条件。仅使用 LaneSegNet 的预测输出的监测器达到了 0.825 的 AUROC，而对于 VAD，ego 状态、驾驶命令和规划器的预测轨迹在相同的平均绝对误差终点上达到了 0.924。将潜在特征添加到任一基线并未带来统计上显著的改善。这个观察结果在广泛的规划故障终点中持续存在，包括那些标签依赖于对非潜在基线不可用的几何形状的终点。因此，从内部表示中预测故障并不表明该表示提供了超出可观察输入和输出的有用信息。我们提出了一种评估协议，用于测试潜在访问的增量价值，并发布我们的每帧故障终点标签。

View on arXiv Download PDF AI Translation

cs.RO / 19 / 2609.30560

SoGuDiff: Socially Guided Diffusion for Steerable, Norm-Grounded Robot Navigation

SoGuDiff：社会引导的可调节、规范基础的机器人导航

Schaible, Christian, Ji, Haoran, Pant, Yash Vardhan, Smith, Stephen L.

Abstract

Beyond collision avoidance, socially competent robot navigation requires adherence to implicit social conventions that vary across contexts, cultures, and deployment requirements. Many conventional navigation policies learn a single normative behavior, either through reinforcement learning against a fixed reward function or imitation of human demonstrations, exposing no interface for adjusting that conduct at runtime. We present a diffusion-based navigation framework whose social behavior can be tuned at deployment: a desired style is specified, such as how closely the robot passes, which side it yields to, or how much it defers to groups, and the planner adapts accordingly. Continuous style axes can be followed independently or composed, spanning a behavioral space rather than discrete, primitive-based specifications. A feasibility projection layer separates learned social behavior from kinematic feasibility and collision avoidance. A single-axis sweep illustrates a tradeoff curve that strictly dominates the evaluated fixed-behavior baseline configurations, and stylistic differences are replicated in real-world demonstrations.

Chinese Translation

除了避免碰撞，社会能力强的机器人导航还需要遵循在不同情境、文化和部署要求下变化的隐性社会规范。许多传统的导航策略通过强化学习固定奖励函数或模仿人类示范来学习单一的规范行为，这使得在运行时无法调整该行为。我们提出了一种基于扩散的导航框架，其社会行为可以在部署时进行调节：可以指定期望的风格，例如机器人与他人通过时的距离、让行的一侧或对群体的让步程度，规划器会相应地进行调整。连续的风格轴可以独立跟随或组合，涵盖一个行为空间，而不是离散的、基于原始行为的规范。可行性投影层将学习到的社会行为与运动学可行性和碰撞避免分离开来。单轴扫描展示了一个权衡曲线，该曲线严格优于评估的固定行为基线配置，并且在真实世界的演示中复制了风格差异。

View on arXiv Download PDF AI Translation

cs.RO / 20 / 2609.30570

MOCHA: Multi-Objective Co-Design using Hypernetwork Architectures

MOCHA：基于超网络架构的多目标协同设计

Madabushi, Varun, Janwani, Neil, Tucker, Maegan

Abstract

In this work, we present MOCHA, the first, to our knowledge, reinforcement learning based approach to computing a family of Pareto-optimal policies across the design space of a robot using a single network. Specifically, MOCHA leverages the hypernetwork architecture to learn a network that produces specialized network parameters optimized for a given objective and parameterized robot design; we term this a multi-objective design hypernetwork (MDH). We demonstrate the capabilities of MDHs to represent a complex family of design-dependent strategies on two distinct robot morphologies, each with six design dimensions and across 2-3 objectives. Moreover, we propose an approach for efficiently producing a Design Pareto set using evolutionary search of the learned policy network, generating the optimal design-policy combination for each objective prioritization. Lastly, we provide an efficient method for computing generalist robot designs which achieve the best cumulative performance across the entire set of objectives.

Chinese Translation

在本研究中，我们提出了MOCHA，这是我们所知的首个基于强化学习的方法，通过单一网络计算机器人设计空间中的一系列帕累托最优策略。具体而言，MOCHA利用超网络架构学习一个网络，该网络生成针对特定目标和参数化机器人设计优化的专用网络参数；我们将其称为多目标设计超网络（MDH）。我们展示了MDH在两个不同机器人形态上表示复杂设计依赖策略家族的能力，每个形态具有六个设计维度，并涵盖2-3个目标。此外，我们提出了一种高效生成设计帕累托集的方法，利用学习到的策略网络进行进化搜索，为每个目标优先级生成最佳设计-策略组合。最后，我们提供了一种高效计算通用机器人设计的方法，以实现整个目标集的最佳累积性能。

View on arXiv Download PDF AI Translation

cs.RO / 21 / 2609.30594

HuGo: LLMs as Whole-Body Policy Code Designers for Humanoid Loco-Manipulation

HuGo：将大型语言模型作为类人机器人全身策略代码设计者用于类人运动操控

Choi, Seoyeon, Ye, Shizhao, Bui, Nicholas, Shrivastava, Aayushi, Ryu, Kanghyun, Tirumala, Dhruva, Wulfmeier, Markus, Mehr, Negar

Abstract

For humanoids to be useful in everyday environments, they must perform a wide range of tasks that couple locomotion and manipulation. Existing approaches commonly acquire a loco-manipulation policy through reward engineering or demonstrations followed by task-specific training, making it costly to scale to new tasks. In this work, we propose a hierarchical approach to humanoid loco-manipulation that eliminates these per-task requirements. HuGo, Humanoid policy code Generation, uses a Large Language Model (LLM) to generate executable, closed-loop high-level policy code from a task description on top of a frozen low-level whole-body policy. Given the task, observation, and command specifications, the LLM constructs the task logic in code. HuGo then refines the policy from its rollouts using numerical trajectories and selected video frames to produce feedback and targeted code updates. Across five simulation tasks, using two different low-level policies, HuGo substantially outperforms a high-level reinforcement learning baseline and approaches the performance of a demonstration-based baseline. We achieve this level of performance without task-specific reward design or demonstration collection. We further demonstrate zero-shot transfer of simulation-generated policies to hardware and show that applying the same refinement loop to real-world rollouts can further improve transfer performance without expert demonstrations or policy retraining. Project website is https://iconlab.negarmehr.com/HuGo/

Chinese Translation

为了使类人机器人在日常环境中发挥作用，它们必须执行一系列结合运动和操控的任务。现有的方法通常通过奖励工程或示范获得运动操控策略，然后进行特定任务的训练，这使得扩展到新任务的成本较高。在本研究中，我们提出了一种层次化的类人运动操控方法，消除了这些针对每个任务的要求。HuGo（类人政策代码生成）利用大型语言模型（LLM）从任务描述中生成可执行的闭环高层策略代码，基于一个冻结的低层全身策略。在给定任务、观察和命令规范的情况下，LLM在代码中构建任务逻辑。然后，HuGo利用数值轨迹和选定的视频帧从其回放中优化策略，以产生反馈和针对性的代码更新。在五个仿真任务中，使用两种不同的低层策略，HuGo显著超越了高层强化学习基线，并接近基于示范的基线性能。我们在没有特定任务的奖励设计或示范收集的情况下实现了这一性能水平。我们进一步展示了仿真生成的策略在硬件上的零-shot迁移，并表明将相同的优化循环应用于现实世界的回放可以在没有专家示范或策略再训练的情况下进一步提高迁移性能。项目网站为 https://iconlab.negarmehr.com/HuGo/

View on arXiv Download PDF AI Translation

cs.RO / 22 / 2609.30599

Learning-Accelerated Narrow-Phase Collision Detection via Check Ordering for Sampling-Based Motion Planning

通过检查顺序加速学习的狭窄阶段碰撞检测用于基于采样的运动规划

Jiang, Hao, Wang, Yinghan, He, Jianping, Duan, Xiaoming

Abstract

Collision detection is critical for ensuring the safety of planned paths. However, it imposes a non-negligible computational burden on motion planners, motivating extensive studies on collision-detection acceleration. In commonly used phase-based collision-detection methods, the broad phase employs hierarchical structures to rapidly discard object pairs that are clearly collision-free, while the subsequent narrow phase performs detailed collision checks on the remaining object pairs whose collision status cannot be determined by the broad phase. Although these methods effectively reduce the number of detailed checks through broad-phase pruning, the narrow phase is usually executed in the default order returned by the broad phase, with little explicit optimization of the check order. This leaves room for further acceleration, especially in cluttered environments where many object pairs may remain after the broad phase and the narrow phase can account for a significant portion of the total detection time. In this work, we propose a learning-based method to accelerate phase-based collision detection by optimizing the check order in the narrow phase. We first formulate the expected time cost of the narrow phase and derive an optimal check-ordering criterion that minimizes this expectation. Since the priors required by this criterion are difficult to obtain in advance, we design a hypernetwork-based model to predict collision probabilities, which are then used to approximate the optimal check order. The resulting order guides the execution of exact mesh checks in the narrow phase, thereby reducing detection time without replacing the underlying geometric collision checker. Simulation results show that our method effectively accelerates phase-based collision detection and improves the efficiency and success rate of sampling-based motion planning, especially in cluttered environments.

Chinese Translation

碰撞检测对于确保规划路径的安全性至关重要。然而，它对运动规划器施加了不可忽视的计算负担，这促使了对碰撞检测加速的广泛研究。在常用的基于阶段的碰撞检测方法中，广义阶段采用层次结构快速排除明显无碰撞的物体对，而随后的狭窄阶段则对广义阶段无法确定碰撞状态的剩余物体对进行详细的碰撞检查。尽管这些方法通过广义阶段的修剪有效减少了详细检查的数量，但狭窄阶段通常按照广义阶段返回的默认顺序执行，几乎没有对检查顺序进行显式优化。这为进一步加速留出了空间，特别是在杂乱环境中，许多物体对可能在广义阶段后仍然存在，而狭窄阶段可能占据总检测时间的显著部分。在本研究中，我们提出了一种基于学习的方法，通过优化狭窄阶段的检查顺序来加速基于阶段的碰撞检测。我们首先制定狭窄阶段的预期时间成本，并推导出一个最优检查顺序标准，以最小化这一期望。由于该标准所需的先验信息难以提前获得，我们设计了一种基于超网络的模型来预测碰撞概率，然后利用这些概率来近似最优检查顺序。生成的顺序指导狭窄阶段中精确网格检查的执行，从而在不替换基础几何碰撞检查器的情况下减少检测时间。仿真结果表明，我们的方法有效加速了基于阶段的碰撞检测，并提高了基于采样的运动规划的效率和成功率，特别是在杂乱环境中。

View on arXiv Download PDF AI Translation

cs.RO / 23 / 2609.30608

Audit Before You Commit: Locating Belief Failures in Active Identification for One-Shot Manipulation

在承诺之前进行审计：定位一次性操作中的信念失败

Abouagour, Mohamed, Min, Byung-Cheol

Abstract

A robot that probes a few times before one irreversible action, such as tapping a surface before inserting a peg, must decide when the evidence is enough to commit. We argue that this decision rests on two conditions that existing methods do not separate: the belief must still cover the truth in the coordinate that decides the action, and the failure model that scores actions must track realized failure. We audit both conditions separately, offline and with ground truth, on a deployed probe-then-commit pipeline: a particle belief, a scenario failure score, and one commit. On simulated insertion, more taps sharpen the belief while the truth leaves its support on 16.9% of episodes and the failure score turns optimistic by 0.31. Conformal calibration restores coverage but not the decision: confidently wrong instances still pass a confidence gate. The audit's signatures instead point at the observation model, where a hand scan finds a 2.1 mm error in the tap boundary. Correcting that one number cuts failure from 0.354 to 0.112 on untouched instances and transfers unrefitted to a second engine, while in a third engine the same audit suggests an execution-model mismatch instead. Across seven task families in three engines, a few probes at a fixed executor reduce miss or failure. On a physical arm inserting a tool into a rigid pocket by touch, the gain and the audit's two conditions reproduce, and replaying the recorded taps under an injected model error shows the audit's signature on real data. Additional materials are available at https://sites.google.com/view/auditbeforeyoucommit.

Chinese Translation

在进行一次不可逆的操作之前，例如在插入插销之前轻敲表面，机器人必须决定何时证据足够以进行承诺。我们认为，这一决策基于两个现有方法未能区分的条件：信念必须在决定行动的坐标中仍然覆盖真实情况，并且评分行动的失败模型必须跟踪已实现的失败。我们在已部署的探测-再承诺管道上分别对这两个条件进行审计，使用离线和真实数据：一个粒子信念、一个场景失败评分和一次承诺。在模拟插入中，更多的轻敲使信念更加明确，而真实情况在16.9%的情节中超出了其支持范围，失败评分则乐观地提高了0.31。符合校准恢复了覆盖范围，但未能恢复决策：自信错误的实例仍然通过了置信门。审计的特征指向观察模型，其中手动扫描发现轻敲边界存在2.1毫米的误差。纠正这一数字使未触及实例的失败率从0.354降至0.112，并将未重新调整的模型转移到第二个引擎，而在第三个引擎中，相同的审计则表明执行模型的不匹配。在三个引擎的七个任务系列中，固定执行器的少量探测减少了遗漏或失败。在一个物理臂通过触摸将工具插入刚性口袋的过程中，增益和审计的两个条件得以再现，而在注入模型误差下重放录制的轻敲显示了审计在真实数据上的特征。更多材料可在 https://sites.google.com/view/auditbeforeyoucommit 获取。

View on arXiv Download PDF AI Translation

cs.RO / 24 / 2609.30626

Frequency-Modulated Piezoelectric Haptic Display

频率调制压电触觉显示器

Liang, Boyuan, Sun, Lingfeng, Tomizuka, Masayoshi

Abstract

We present a frequency-modulated (FM) haptic display based on piezoelectric vibrating actuators. Existing haptic displays commonly encode haptic intensity through the deformation amplitude of individual haptic pixels. Although amplitude-modulated (AM) approaches have enabled compact haptic pixels, independently controlling the deformation amplitude of a large number of pixels can require increasingly complex and bulky driving systems, posing challenges for scaling toward high-density, large-area wearable displays. To address this scaling challenge, we investigate an FM design principle in which haptic intensity is encoded through vibration frequency. We further develop a \textit{Shared-Source Frequency Modulation} (SSFM) structure in which multiple haptic pixels are powered by a common power amplifier while their vibration spectrum are controlled individually, reducing the need for independent high-power amplification at each pixel. A proof-of-concept piezoelectric haptic display was built and evaluated on rendering spatial and temporal haptic patterns through volunteer tests. The results show that participants reliably distinguished spatial and temporal patterns encoded using FM principles within the investigated operating range. These findings demonstrate the feasibility of FM-based distributed haptic rendering and suggest a potential pathway toward more compact driving architectures for future high-density, large-area wearable haptic displays.

Chinese Translation

我们提出了一种基于压电振动驱动器的频率调制（FM）触觉显示器。现有的触觉显示器通常通过单个触觉像素的变形幅度来编码触觉强度。尽管幅度调制（AM）方法使得触觉像素更加紧凑，但独立控制大量像素的变形幅度可能需要越来越复杂和庞大的驱动系统，这对向高密度、大面积可穿戴显示器的扩展构成挑战。为了解决这一扩展挑战，我们研究了一种FM设计原则，其中触觉强度通过振动频率进行编码。我们进一步开发了一种共享源频率调制（Shared-Source Frequency Modulation, SSFM）结构，其中多个触觉像素由一个公共功率放大器供电，同时其振动频谱单独控制，从而减少了每个像素独立高功率放大的需求。我们构建并评估了一个概念验证的压电触觉显示器，通过志愿者测试呈现空间和时间触觉模式。结果表明，参与者在所研究的操作范围内可靠地区分了使用FM原理编码的空间和时间模式。这些发现证明了基于FM的分布式触觉渲染的可行性，并为未来高密度、大面积可穿戴触觉显示器的更紧凑驱动架构提供了潜在的路径。

View on arXiv Download PDF AI Translation

cs.RO / 25 / 2609.30644

MR. POP: Multi-Robot Parallel Optimizing Planner for Almost-Surely Asymptotically Optimal Planning

MR. POP：多机器人并行优化规划器，实现几乎肯定渐进最优规划

Huang, Chih H., Xing, Roy, Plancher, Brian, Kingston, Zachary

Abstract

Finding globally optimal paths remains a fundamental challenge in multi-robot motion planning. Despite acceleration of almost-surely asymptotically optimal (a.s.a.o.) planners via CPU-based parallelism, achieving both probabilistic convergence guarantees and strong computational performance, these algorithms still struggle to scale to multi-robot settings. As such, we introduce MR. POP, a GPU-based a.s.a.o. multi-robot planner based on dRRT and the AO-x meta-algorithm. MR. POP uses large-scale GPU-based SIMT-parallelism to simultaneously run hundreds of roadmap construction and tree search iterations with underlying parallel nearest neighbor search and collision checking operations. We show that this enables MR. POP to become the only planner achieving a 100% solve rate while being faster than state-of-the-art a.s.a.o. planners in multi-robot systems up to 35-DOF. MR. POP also raises the success rate of downstream motion optimizers (e.g., from 4% to 72%), by creating high-quality, diverse seeds that help avoid local minima.

Chinese Translation

寻找全局最优路径仍然是多机器人运动规划中的一个基本挑战。尽管通过基于CPU的并行化加速了几乎肯定渐进最优（a.s.a.o.）规划器，但在实现概率收敛保证和强大的计算性能方面，这些算法仍然难以扩展到多机器人环境。因此，我们提出了MR. POP，一种基于GPU的a.s.a.o.多机器人规划器，基于dRRT和AO-x元算法。MR. POP利用大规模基于GPU的SIMT并行性，同时运行数百次路线图构建和树搜索迭代，并进行底层的并行最近邻搜索和碰撞检测操作。我们展示了这使得MR. POP成为唯一一个在多机器人系统中实现100%求解率且速度快于最先进的a.s.a.o.规划器（高达35自由度）的规划器。MR. POP还通过创建高质量、多样化的种子，帮助避免局部最优，从而提高下游运动优化器的成功率（例如，从4%提高到72%）。

View on arXiv Download PDF AI Translation

cs.RO / 26 / 2609.30676

Can a Robot Read Braille? - Learning to Adapt Contact via Imitation Learning for Tactile Braille Recognition

机器人能读盲文吗？——通过模仿学习适应接触以实现触觉盲文识别

Chen, Xi, Shan, Yunlong, Chen, Sihan, Hu, Jun, Li, Zhongxuan, Zhang, Shiyao, Liu, Sichao, Zhao, Zhong, Jovanovic, Kosta, Zhou, Peng

Abstract

For people who are blind, touch provides an essen-tial channel for accessing written information through Braille. Bringing a similar capability to robots requires them not only to recognize tactile patterns, but also to actively establish physical contact that makes those patterns readable. Yet existing robotic Braille readers largely focus on recognition after contact, leaving contact establishment itself insufficiently addressed. We present an adaptive-contact framework for robotic tactile Braille reading that assesses contact quality and physically corrects unsuitable contact before recognition and reconstruc-tion. Multi-Head Policy Learning uses expert-guided contact-adjustment demonstrations to jointly learn contact acceptability and pose corrections. During deployment, the robot iteratively evaluates and re-establishes contact, retaining reliable tactile observations for pose-aware fusion and Braille reconstruction. Across 20 physical Braille plates used for learning and eval-uation, the proposed approach achieves 94.0% tactile quality and 88.6% tactile reconstruction on the ten online-evaluation plates. These results demonstrate the importance of actively establishing readable contact, rather than relying solely on recognition under imperfect tactile observations, for reliable robotic Braille reading.

Chinese Translation

对于盲人来说，触觉是通过盲文获取书面信息的重要渠道。将这一能力引入机器人不仅要求它们能够识别触觉模式，还需要主动建立物理接触，使这些模式可读。然而，现有的机器人盲文阅读器主要关注接触后的识别，而对接触的建立关注不足。我们提出了一种自适应接触框架，用于机器人触觉盲文阅读，该框架在识别和重建之前评估接触质量，并对不合适的接触进行物理校正。多头策略学习（Multi-Head Policy Learning）利用专家指导的接触调整演示共同学习接触可接受性和姿态校正。在部署过程中，机器人迭代评估和重新建立接触，保留可靠的触觉观察以进行姿态感知融合和盲文重建。在用于学习和评估的20个物理盲文板上，所提出的方法在10个在线评估板上实现了94.0%的触觉质量和88.6%的触觉重建。这些结果表明，主动建立可读接触的重要性，而不是仅仅依赖于不完美触觉观察下的识别，以实现可靠的机器人盲文阅读。

View on arXiv Download PDF AI Translation

cs.RO / 27 / 2609.30695

Multi-Objective Human-in-the-Loop Bayesian Optimization of a Lower-Limb Exoskeleton

多目标人机协同贝叶斯优化下肢外骨骼

Janwani, Neil, Lerner, Matthew T., Young, Aaron J., Tucker, Maegan

Abstract

Human-in-the-loop optimization (HILO) is a common approach for optimizing the control of assistive devices to account for the wearer's unique biomechanics and subjective preferences. However, despite research suggesting that a person may have a different prioritization of objectives depending on time-varying factors such as the environment, their mood, or energy levels, existing HILO approaches only consider a single objective or enforce a fixed weighting on a set of objectives. Neither approach is capable of representing an individual's preferences over objectives. In this work, we propose Multi-Objective Human-in-the-loop Bayesian Optimization (MO-HILBO), which builds on explicit multi-objective Bayesian optimization to efficiently infer a personalized set of Pareto-optimal controllers. We compare our approach with an existing multi-objective HILO method and experimentally demonstrate MO-HILBO on a lower-limb exoskeleton across two objectives: metabolic cost (efficiency) and ordinal human feedback (comfort). We find that MO-HILBO (1) discovers Pareto-optimal controllers, and (2) that the pairwise ordering of points on the Pareto front itself is consistent with validation trials. Lastly, we open-source mohilo, a Python package for running both HILO and MO-HILBO on wearable devices: https://dynamicmobility.github.io/mohilo/.

Chinese Translation

人机协同优化（HILO）是一种常见的方法，用于优化辅助设备的控制，以考虑佩戴者独特的生物力学特征和主观偏好。然而，尽管研究表明，个体可能会根据环境、情绪或能量水平等时间变化因素对目标的优先级进行不同的排序，现有的HILO方法仅考虑单一目标或对一组目标施加固定权重。这两种方法都无法有效表示个体对目标的偏好。在本研究中，我们提出了多目标人机协同贝叶斯优化（MO-HILBO），该方法基于显式的多目标贝叶斯优化，能够高效推断个性化的帕累托最优控制器。我们将我们的方法与现有的多目标HILO方法进行了比较，并在下肢外骨骼上实验性地展示了MO-HILBO，涉及两个目标：代谢成本（效率）和顺序人类反馈（舒适度）。我们发现，MO-HILBO（1）能够发现帕累托最优控制器，且（2）帕累托前沿上点的成对排序与验证试验一致。最后，我们开源了mohilo，这是一个用于在可穿戴设备上运行HILO和MO-HILBO的Python包：https://dynamicmobility.github.io/mohilo/.

View on arXiv Download PDF AI Translation

cs.RO / 28 / 2609.30696

Learning Vision-Based Agile Gap Traversal: Differentiable Simulation with a Warm-Started Critic

基于视觉的灵活间隙穿越学习：带有热启动评论员的可微仿真

Gerdpratoom, Nuthasith, Sun, Tianchen, Gao, Yichao, Zhao, Lin

Abstract

Traversing narrow gaps is challenging for autonomous quadrotors, especially when control commands come directly from high-dimensional visual observations. Existing end-to-end methods often rely on behavior cloning or full-rollout backpropagation through time (BPTT) via differentiable simulation, which can limit policy performance or incur high training costs. We propose a two-stage reinforcement learning framework for more efficient ego-centric visuomotor gap-traversal policy training, leveraging quasi-analytical policy gradients (QPG) via differentiable simulation and critic warm-starting. The framework utilizes QPG to avoid backpropagation through visual rendering, reducing computation and memory costs while improving sample efficiency. In the first stage, an expert actor and critic are trained using privileged observations, including gap geometry. Unlike prior gap-traversal approaches, our training utilizing QPG does not require resetting the agent along optimized reference trajectories. In the second stage, a visual policy is trained using binary gap masks from two ego-centric cameras and low-dimensional observations, while its privileged critic is warm-started from the first stage. This substantially improves training efficiency and traversal success compared with cold-starting the critic or using full-rollout BPTT. Our framework does not require retraining the expert actor when system parameters change, enabling more efficient generalization across drone platforms than state-of-the-art visual gap-traversal methods based on action supervision. The learned visual policy also generalizes to gaps with unseen shapes. Extensive real-world experiments further demonstrate robust gap traversal using binary masks rendered online. Beyond gap traversal, the proposed framework is generic and can be extended to other visuomotor robot learning tasks.

Chinese Translation

对于自主四旋翼飞行器来说，穿越狭窄间隙是一项挑战，尤其是当控制指令直接来自高维视觉观测时。现有的端到端方法通常依赖于行为克隆或通过可微仿真进行的全回放时间反向传播（BPTT），这可能限制策略性能或导致高昂的训练成本。我们提出了一种两阶段强化学习框架，以更高效地进行自我中心的视觉运动间隙穿越策略训练，利用可微仿真中的准解析策略梯度（QPG）和评论员热启动。该框架利用QPG避免通过视觉渲染进行反向传播，从而减少计算和内存成本，同时提高样本效率。在第一阶段，使用包括间隙几何信息在内的特权观测训练专家演员和评论员。与先前的间隙穿越方法不同，我们利用QPG的训练不需要沿优化参考轨迹重置代理。在第二阶段，使用来自两个自我中心相机的二元间隙掩码和低维观测训练视觉策略，同时其特权评论员从第一阶段热启动。这显著提高了训练效率和穿越成功率，相较于冷启动评论员或使用全回放BPTT。我们的框架不需要在系统参数变化时重新训练专家演员，从而使其在无人机平台之间的泛化效率高于基于动作监督的最先进视觉间隙穿越方法。学习到的视觉策略也能泛化到形状未见的间隙。大量的现实世界实验进一步证明了使用在线渲染的二元掩码进行稳健的间隙穿越。除了间隙穿越，所提出的框架是通用的，可以扩展到其他视觉运动机器人学习任务。

View on arXiv Download PDF AI Translation

cs.RO / 29 / 2609.30704

From Visual Search to Movement Control: A Priority Field for Artificial Agents

从视觉搜索到运动控制：人工智能体的优先场

Zhang, Han, Cao, Zhong

Abstract

Human spatial attention is widely conceptualized as being guided by a priority map that integrates perceptual salience, current goals, and past experiences. Here, we extend priority-based computation to movement control in artificial agents. We first introduce a lightweight model of visual search based on an integrated priority map. Trained on human saccades, it reproduced key behavioral patterns, including oculomotor suppression and history-driven selection. Extending the search model, we equipped an artificial agent with a priority field and evaluated its performance in a reach-avoid task that required reaching a goal destination while avoiding moving obstacles. Compared with alternative architectures, priority-field agents trained more efficiently and performed better in unseen, complex scenarios, even from simple demonstrations. Adding a simple memory mechanism also produced human-like, history-driven effects in anticipating the likely location of the upcoming goal. These findings suggest that priority-based computation may provide a promising foundation for movement control in artificial agents.

Chinese Translation

人类的空间注意力通常被概念化为由一个优先图引导，该优先图整合了感知显著性、当前目标和过去经验。在此，我们将基于优先级的计算扩展到人工智能体的运动控制。我们首先介绍了一个基于集成优先图的轻量级视觉搜索模型。该模型在训练人类眼跳的基础上，重现了关键的行为模式，包括眼动抑制和历史驱动选择。扩展搜索模型后，我们为人工智能体配备了优先场，并评估其在一个需要到达目标位置同时避免移动障碍物的触达-避免任务中的表现。与其他架构相比，优先场智能体的训练效率更高，并且在未见过的复杂场景中表现更佳，即使是从简单的演示中学习。添加一个简单的记忆机制也产生了类似人类的历史驱动效应，能够预测即将到来的目标的可能位置。这些发现表明，基于优先级的计算可能为人工智能体的运动控制提供一个有前景的基础。

View on arXiv Download PDF AI Translation

cs.RO / 30 / 2609.30715

RoboMonitor: Label-Efficient Runtime Monitoring of Robot Task Execution via Predictive Representation Learning

RoboMonitor：通过预测表示学习实现机器人任务执行的标签高效运行监控

Ajith, Abhiroop, Narayanan, Gokul, Coelho, Kyle, Zhao, Tingji, Shahapurkar, Yash, Zhu, Brian, Erdogan, Melih, Krubasik, Ted, Chamzas, Constantinos, Solowjow, Eugen

Abstract

Learned robot policies produce actions, but their outputs alone do not establish whether execution is progressing as intended. Robot execution monitoring requires identifying the current execution phase, detecting failures, and recognizing task completion from observations available during execution. Training such monitors requires annotations that are scarce in datasets collected for robot-policy learning. We present RoboMonitor, a label-efficient vision--language execution monitor that learns from these datasets before introducing monitoring supervision. We pre-train on 25 hours of multi-camera trajectories spanning 12 manipulation tasks and two robot embodiments, using action-conditioned future-feature prediction, inverse dynamics, and masked-present prediction. We then transfer the learned visual and context encoders to a causal monitor and apply temporal supervised fine-tuning (Temporal SFT), which combines supervision throughout each observation window with consistency objectives within and across overlapping windows. At deployment, RoboMonitor requires only the task instruction and camera observations. On a four-task monitoring benchmark, RoboMonitor trained with 52 labeled episodes achieves 93.1% mean phase accuracy and 85.9% macro recall over two fine-tuning seeds, exceeding Qwen3-VL and Robometer trained with the same monitoring supervision. Its phase accuracy also exceeds that of both Qwen3-VL and Robometer trained with 100 episodes. A Qwen3-VL ablation shows that Temporal SFT reduces mean spurious phase switching from 15.23% to 4.95%. In closed-loop deployment, the integrated system completes 39 of 40 simulated Toolbox Sorting trials and 35 of 40 real-world Reel Packing trials, with no false recovery triggers observed.

Chinese Translation

学习到的机器人策略会产生动作，但仅凭其输出无法确定执行是否按预期进行。机器人执行监控需要识别当前的执行阶段、检测故障以及从执行过程中可用的观察中识别任务完成情况。训练这样的监控器需要在为机器人策略学习收集的数据集中进行稀缺的标注。我们提出了RoboMonitor，这是一种标签高效的视觉-语言执行监控器，它在引入监控监督之前，从这些数据集中学习。我们在涵盖12个操作任务和两个机器人实例的25小时多摄像头轨迹上进行预训练，采用了基于动作的未来特征预测、逆动力学和掩蔽当前预测。然后，我们将学习到的视觉和上下文编码器转移到因果监控器，并应用时间监督微调（Temporal SFT），该方法结合了在每个观察窗口内的监督与跨重叠窗口的一致性目标。在部署时，RoboMonitor仅需任务指令和摄像头观察。在一个四任务监控基准上，使用52个标注集训练的RoboMonitor实现了93.1%的平均阶段准确率和85.9%的宏观召回率，超越了使用相同监控监督训练的Qwen3-VL和Robometer。其阶段准确率也超过了使用100个集训的Qwen3-VL和Robometer。Qwen3-VL的消融实验表明，Temporal SFT将平均虚假阶段切换率从15.23%降低至4.95%。在闭环部署中，集成系统完成了40个模拟工具箱分类试验中的39个和40个真实世界卷轴包装试验中的35个，未观察到虚假恢复触发。

View on arXiv Download PDF AI Translation

cs.RO / 31 / 2609.30735

Praxis: Distilling Physical Interaction Priors from Egocentric Videos for Generalizable Whole-Body Manipulation

Praxis：从自我中心视频中提取物理交互先验以实现可泛化的全身操控

He, Shuliang, Xu, Ruiyan, Yue, Bo, Zhang, Hengming, Zhou, Huayi, Wang, Shuai, Zheng, Wei-Shi, Liu, Guiliang

Abstract

Mobile humanoid manipulation requires both reaching a usable workspace and preserving precise hand-object interactions as object poses and contact conditions change. Learning these behaviors from limited task-specific data remains challenging. To bridge this gap, we introduce Praxis, a whole-body manipulation framework that combines physical interaction priors from one-shot egocentric video demonstrations with closed-loop posture calibration and online perception. The framework coordinates three stages: vision-language-guided navigation toward target objects, closed-loop posture calibration to align the arm-hand workspace, and dexterous manipulation with synchronized upper- and lower-body control. Online visual feedback re-grounds demonstrated interaction geometry under new object poses and scene configurations, while tactile feedback adapts hand motions to actual contact conditions. Each manipulation skill is specified by one human demonstration, without task-specific manipulation-policy retraining. Experiments across five long-horizon manipulation tasks demonstrate spatial, visual, and cross-object generalization, as well as recovery from external physical disturbances across all three stages.

Chinese Translation

移动类人操控需要既能达到可用的工作空间，又能在物体姿态和接触条件变化时保持精确的手-物体交互。从有限的特定任务数据中学习这些行为仍然具有挑战性。为了解决这一问题，我们提出了Praxis，一个全身操控框架，它结合了来自一次性自我中心视频演示的物理交互先验、闭环姿态校准和在线感知。该框架协调三个阶段：基于视觉-语言的导航以接近目标物体、闭环姿态校准以对齐手臂-手的工作空间，以及通过同步的上下肢控制进行灵巧操控。在线视觉反馈在新的物体姿态和场景配置下重新确定演示的交互几何，而触觉反馈则将手部动作适应于实际接触条件。每项操控技能由一个人类演示指定，无需特定任务的操控策略再训练。在五个长时间操控任务中的实验表明，具有空间、视觉和跨物体的泛化能力，以及在所有三个阶段中从外部物理干扰中恢复的能力。

View on arXiv Download PDF AI Translation

cs.RO / 32 / 2609.30745

Anatomy-Aware Dexterity-Driven Design Optimization of Surgical Continuum Robots

考虑解剖结构的灵巧驱动手术连续机器人设计优化

Qin, Tony, Connor, Peter, Dang, Khoa, Hatch, Carter, Rucker, Caleb, Webster III, Robert J., Alterovitz, Ron

Abstract

Performing complex medical procedures with continuum robots requires careful selection of their geometric design parameters. The robot should have high dexterity in the specific anatomical environment of its procedure. This work presents a design optimization method that considers both dexterity and anatomy. We introduce the Reachable Volumetric Dexterous Solid Angle (RVDSA) metric as our objective, which measures the ability of a robot's end effector to reach the points in a goal volume from different directions via collision-free paths from a start configuration. We present a computationally efficient motion planner to compute this objective function for a given robotic design, and we use an asymptotically optimal simulated annealing optimizer to compute an optimized design. We applied our new method to optimize the design of a bimanual dexterous sheaths robot for performing procedures on cancerous polyps in colon anatomies, achieving a 78% higher RVDSA on average than optimizing for 3D voxel coverage alone.

Chinese Translation

使用连续机器人进行复杂医疗程序需要仔细选择其几何设计参数。机器人应在其手术的特定解剖环境中具有高灵巧性。本研究提出了一种同时考虑灵巧性和解剖结构的设计优化方法。我们引入了可达体积灵巧固体角（Reachable Volumetric Dexterous Solid Angle, RVDSA）指标作为我们的目标，该指标衡量机器人末端执行器从不同方向通过无碰撞路径从起始配置到达目标体积中点的能力。我们提出了一种计算效率高的运动规划器，以计算给定机器人设计的该目标函数，并使用渐近最优的模拟退火优化器来计算优化设计。我们将新方法应用于优化双手灵巧鞘机器人在结肠解剖中对癌性息肉进行手术的设计，平均实现了比仅优化3D体素覆盖高出78%的RVDSA。

View on arXiv Download PDF AI Translation

cs.RO / 33 / 2609.30759

Design and Characterization of a Variable-Length Continuum Mechanism with Force Locking

具有力锁定的可变长度连续机制的设计与表征

King, Katelyn, Fish, Veronica, Okamura, Allison M.

Abstract

The utility of flexible continuum mechanisms for dexterous navigation is often impaired by their low stiffness, making them ineffective at manipulation in high-force scenarios. To address this challenge, we propose a novel continuum mechanism that achieves both flexible and rigid behavior by antagonistic extension and contraction of a rod-driven continuum helical structure. The helical design combines variable-length capacity with force locking for workspace and stiffness enhancement. In this article, we present the detailed design of the proposed mechanism and characterize its performance through experiments that quantify bending and stiffness. The results demonstrate 180 degree bending range of motion with an average distal positioning error of <10%. Further tests demonstrate that force locking directly improves axial stiffness and thus indirectly increases bending stiffness anisotropically, with maximum bending stiffness along load paths with a large axial component. Tensioning the driving rods provides additional stiffness tunability in the force-locked state, where increasing rod tension proportionally increases bending stiffness with a dimensionless gain of 0.56.

Chinese Translation

柔性连续机制在灵活导航中的实用性常常受到其低刚度的影响，使其在高强度操作场景中效果不佳。为了解决这一挑战，我们提出了一种新颖的连续机制，通过对抗性地伸展和收缩杆驱动的连续螺旋结构，实现了灵活和刚性行为的结合。螺旋设计结合了可变长度能力与力锁定，以增强工作空间和刚度。本文详细介绍了所提机制的设计，并通过实验对其性能进行了表征，量化了弯曲和刚度。结果表明，该机制具有180度的弯曲运动范围，平均远端定位误差小于10%。进一步的测试表明，力锁定直接提高了轴向刚度，从而间接地以各向异性的方式增加了弯曲刚度，最大弯曲刚度沿着具有大轴向分量的载荷路径。拉紧驱动杆在力锁定状态下提供了额外的刚度可调性，其中增加杆的张力成比例地增加了弯曲刚度，维持了0.56的无量纲增益。

View on arXiv Download PDF AI Translation

cs.RO / 34 / 2609.30770

NavGen: Visual Generative Models as a Scalable Data Engine for Embodied 3D Navigation

NavGen：作为可扩展数据引擎的视觉生成模型用于具身3D导航

Huang, Xijie, Wan, Yongyang, Dong, Chengbin, Ding, Zimo, Zhu, Mo, Wang, Yijin, Liu, Zhiyang, Gao, Fei, Wu, Yuze, Zhou, Xin

Abstract

General-purpose robot models increasingly rely on large and diverse datasets. For embodied 3D navigation, however, existing data sources face a fundamental trade-off: simulated data can be generated at scale but often suffer from the visual sim-to-real gap, whereas real-world flight data provide realistic observations but are costly to collect. This paper studies another direction: the use of high-fidelity visual generative models as scalable data engines for embodied 3D navigation. We introduce NavGen, a text-to-video data generation pipeline that produces diverse vision-language navigation (VLN) episodes across indoor and outdoor scenes. We also propose a style-diversification method that scales up long-tail data that are difficult and costly to collect. The resulting dataset contains approximately 400K navigation episodes. We evaluate our dataset against existing UAV navigation datasets across multiple metrics, and find that the model trained on our data generally improves with scale, outperforming those trained on existing datasets. To validate real-world transferability, we deploy the trained model in world-action-model paradigm to real-world flying experiments. The final model achieves a 75\% success rate across different navigation tasks and environments.

Chinese Translation

通用机器人模型越来越依赖于大型多样化的数据集。然而，对于具身3D导航而言，现有的数据来源面临着一个基本的权衡：模拟数据可以大规模生成，但通常存在视觉仿真与现实之间的差距，而现实世界的飞行数据提供了真实的观察结果，但收集成本高昂。本文研究了另一种方向：利用高保真视觉生成模型作为具身3D导航的可扩展数据引擎。我们介绍了NavGen，一个文本到视频的数据生成管道，能够在室内和室外场景中生成多样的视觉-语言导航（VLN）剧集。我们还提出了一种风格多样化的方法，以扩大难以和高成本收集的长尾数据。最终生成的数据集包含约40万条导航剧集。我们在多个指标上将我们的数据集与现有的无人机导航数据集进行了评估，发现基于我们的数据训练的模型通常随着规模的增加而改进，超越了那些基于现有数据集训练的模型。为了验证其在现实世界中的可转移性，我们在世界行动模型范式下将训练好的模型部署到现实世界的飞行实验中。最终模型在不同的导航任务和环境中达到了75%的成功率。

View on arXiv Download PDF AI Translation

cs.RO / 35 / 2609.30777

Moving Horizon Estimation for Quadrotors: An $\mathcal{L}_1$ Adaptive Optimizer Approach

四旋翼的移动视域估计：一种 $ ext{L}_1$ 自适应优化器方法

Nguyen, Thinh, Kim, Minkyung, Banik, Sandeep, Kim, Jinrae, Hovakimyan, Naira

Abstract

Moving Horizon Estimation (MHE) is a state estimation method based on finite-horizon optimization that can offer higher accuracy at the cost of increased computation compared to Kalman filter-based approaches. We present a linear smoothing MHE formulation as a dense Quadratic Program (QP), and a solver consisting of a continuous-time Newton's method augmented with the $\mathcal{L}_1$ Adaptive Optimizer ($\mathcal{L}_1$-AO). While MHE is inherently time-varying, conventional approaches treat it as a sequence of independent, time-invariant problems and employ iterative solvers at each time step, which can be both inaccurate and computationally burdensome. In contrast, time-varying solvers track the optimal solution with fewer iterations by exploiting the temporal evolution of the problem, thereby reducing the computational load. In this research, we enhance both the performance and efficiency of MHE through a time-varying solver with an $\mathcal{L}_1$-AO augmentation that compensates for the prediction inaccuracy, which is common in practice due to noisy sensors and the lack of prior knowledge of the system. Simulation results on a quadrotor platform show that the $\mathcal{L}_1$-AO-augmented approach solves the MHE optimization problem more efficiently than the baseline time-invariant solver and achieves higher estimation accuracy under challenging conditions, compared with both the Extended Kalman Filter and the standard MHE.

Chinese Translation

移动视域估计（MHE）是一种基于有限视域优化的状态估计方法，相较于基于卡尔曼滤波的方法，它能够提供更高的准确性，但代价是计算量的增加。我们提出了一种线性平滑的 MHE 公式化，作为一个稠密的二次规划（QP），并设计了一个求解器，该求解器由连续时间牛顿法与 $ ext{L}_1$ 自适应优化器（$ ext{L}_1$-AO）相结合而成。虽然 MHE 本质上是时变的，但传统方法将其视为一系列独立的、时间不变的问题，并在每个时间步使用迭代求解器，这可能导致不准确且计算负担较重。相反，时变求解器通过利用问题的时间演变来跟踪最优解，从而减少迭代次数，降低计算负担。在本研究中，我们通过一种时变求解器与 $ ext{L}_1$-AO 增强相结合，提升了 MHE 的性能和效率，以补偿由于传感器噪声和缺乏系统先验知识而导致的预测不准确性。四旋翼平台上的仿真结果表明，$ ext{L}_1$-AO 增强的方法比基线的时间不变求解器更有效地解决了 MHE 优化问题，并在挑战性条件下相比于扩展卡尔曼滤波器和标准 MHE 实现了更高的估计准确性。

View on arXiv Download PDF AI Translation

cs.RO / 36 / 2609.30814

SeA-RVINS: Semantic-Aware Tightly Coupled RTK-Visual-Inertial System with Correlation-Preserving Robust Estimation for Urban Navigation

SeA-RVINS：语义感知紧耦合RTK-视觉惯性系统，具有关联保持的鲁棒估计用于城市导航

Hu, Wang, Wu, Bo

Abstract

Reliable absolute pose estimation in urban environments is undermined by outlier measurements and incorrect temporal associations that can persist in tightly coupled estimators. Global Navigation Satellite System (GNSS) observations provide globally referenced measurements but are prone to multipath effects. Visual-inertial sensing supplies local motion constraints, but false visual associations can corrupt the estimator. We present SeA-RVINS, a fixed-lag factor-graph Real-Time Kinematic (RTK) visual-inertial system for robust urban pose estimation. A semantic-aware learned stereo frontend rejects unreliable tracks before persistent landmarks enter the graph. For double-differenced GNSS measurements, SeA-RVINS applies Dynamic Covariance Scaling through configurable batch, scalar, and latent-pivot robust formulations while retaining the shared-pivot correlation structure. We propose a hybrid ambiguity-continuation strategy that shares one ambiguity state over short arcs with verified continuity and softly links successive arcs through random-walk factors. On an approximately 20-km route from the public TEX-CUP dataset, including about 50\% deep-urban driving, the latent-pivot configuration achieves 100\% availability and a 1.6-m maximum horizontal error, with 96.16\% and 99.90\% of epochs below 1.0 and 1.5 m, respectively. The implementation is released as open-source software

Chinese Translation

在城市环境中，可靠的绝对位姿估计受到异常测量和不正确时间关联的影响，这些问题在紧耦合估计器中可能持续存在。全球导航卫星系统（GNSS）观测提供全球参考测量，但容易受到多路径效应的影响。视觉惯性传感提供局部运动约束，但错误的视觉关联可能会破坏估计器。我们提出了SeA-RVINS，一种固定滞后因子图实时动态（RTK）视觉惯性系统，用于鲁棒的城市位姿估计。语义感知的学习立体前端在持久地标进入图之前拒绝不可靠的轨迹。对于双差GNSS测量，SeA-RVINS通过可配置的批量、标量和潜在支点鲁棒公式应用动态协方差缩放，同时保留共享支点的关联结构。我们提出了一种混合模糊延续策略，该策略在经过验证的连续性短弧上共享一个模糊状态，并通过随机游走因子软性连接连续弧。在来自公共TEX-CUP数据集的约20公里路线（包括约50%的深度城市驾驶）上，潜在支点配置实现了100%的可用性和1.6米的最大水平误差，其中96.16%和99.90%的时刻低于1.0米和1.5米。该实现已作为开源软件发布。

View on arXiv Download PDF AI Translation

cs.RO / 37 / 2609.30818

Evaluation Is All You Need for Multi-Modal Autonomous Driving

评估是多模态自动驾驶所需的一切

He, Zeyu, Liu, Shiqi, Chen, Ke, Yan, Yun, Wu, Jinzi, Lei, Dianqiao, Wang, Sirui, Peng, ShuRui, Chen, Tao, Huang, Zhuo, Wu, Yu, Shao, Yadong, Li, Zhichao, Sun, Ke, Guan, Yang, Li, Keqiang, Li, Shengbo Eben

Abstract

Multi-modal planning is promising for autonomous driving by representing multiple plausible behaviors in ambiguous and long-tail scenarios. Existing methods mainly focus on improving trajectory multi-modality, enhancing trajectory representations, or reshaping the candidate distribution. Nevertheless, we identify a pronounced generation-evaluation asymmetry in multi-modal planning: despite strong oracle performance, existing planners often fail to reliably select the best available candidate, leaving substantial planning potential unrealized. To address this challenge, we propose iDriveVLA, a multi-modal planning framework that improves the candidate trajectory space while enabling more reliable and context-aware trajectory evaluation. Specifically, iDriveVLA introduces a unified trajectory evaluator comprising a Safety-aware Scorer for quality and risk estimation, together with a VLM-guided Modulator for scene-adaptive criterion weighting. We further develop an oracle-aligned progressive training strategy consisting of candidate imitation pretraining, candidate space refinement, and semantic ranking alignment. On the public NAVSIM v1 leaderboard, iDriveVLA achieves a new state-of-the-art performance of 94.95 PDMS, surpassing the human-expert reference.

Chinese Translation

多模态规划在自动驾驶中具有良好的前景，因为它能够在模糊和长尾场景中表示多种可能的行为。现有方法主要集中在提高轨迹的多模态性、增强轨迹表示或重塑候选分布。然而，我们发现多模态规划中存在明显的生成-评估不对称性：尽管具有强大的oracle性能，现有规划器往往无法可靠地选择最佳候选，导致大量规划潜力未能实现。为了解决这一挑战，我们提出了iDriveVLA，一个多模态规划框架，旨在改善候选轨迹空间，同时实现更可靠和上下文感知的轨迹评估。具体而言，iDriveVLA引入了一个统一的轨迹评估器，包括一个用于质量和风险估计的安全感知评分器，以及一个基于视觉语言模型（VLM）的调节器，用于场景自适应标准加权。我们进一步开发了一种与oracle对齐的渐进训练策略，包括候选模仿预训练、候选空间精炼和语义排名对齐。在公共NAVSIM v1排行榜上，iDriveVLA实现了94.95 PDMS的新状态-of-the-art性能，超越了人类专家参考。

View on arXiv Download PDF AI Translation

cs.RO / 38 / 2609.30828

HIRE: History-Conditioned Interaction Reasoning and High-Rate Execution for Visually Aliased Precision Manipulation

HIRE：历史条件交互推理与高频执行用于视觉别名精确操作

Li, Rongji, He, Wenhao, Lu, Cewu, Chen, Xingyu, Zhang, Xu-Yao

Abstract

Precision manipulation with contact-critical interactions is often history-dependent: visually similar observations can correspond to different latent interaction states and therefore require different actions, while small execution errors can alter task outcomes. Policies relying on the current visual observation alone cannot resolve such ambiguity; force-aware and memory-augmented methods enrich physical or temporal context, while reactive high-rate policies improve local contact response, yet long-horizon temporal reasoning and precision execution remain largely decoupled in existing methods, limiting reliable progression in visually aliased precision manipulation. To bridge this gap, we introduce History-Conditioned Interaction Reasoning and Execution (HIRE), a cross-rate framework comprising a history-conditioned Interaction-State Reasoner (ISR) and a high-rate Interaction-Manifold Executor (IME). ISR encodes ordered wrench history with a temporal wrench encoder and Force Perceiver as persistent physical evidence for state-consistent action generation, while IME structures contact-critical motion into intrinsic progress and transverse correction for precise execution; their cross-rate loop allows the resulting physical traces to inform subsequent reasoning. In real-robot experiments across surface, insertion, and rotational interactions, HIRE achieves at least 90% completion across all evaluated task stages while improving interaction-state disambiguation, execution precision, and generalization. More broadly, HIRE provides a unified reasoning--execution perspective on precision manipulation under history-dependent partial observability, where physical interaction both realizes task intent and reveals latent-state evidence for future decisions. Code will be released upon publication.

Chinese Translation

接触关键交互的精确操作通常依赖于历史：视觉上相似的观察可能对应于不同的潜在交互状态，因此需要采取不同的行动，而微小的执行误差可能会改变任务结果。仅依赖当前视觉观察的策略无法解决这种模糊性；而基于力的和增强记忆的方法丰富了物理或时间上下文，尽管反应式高频策略改善了局部接触响应，但在现有方法中，长期时间推理和精确执行仍然在很大程度上是解耦的，这限制了在视觉别名精确操作中的可靠进展。为了弥补这一差距，我们提出了历史条件交互推理与执行（HIRE），这是一个跨频率框架，包括一个历史条件交互状态推理器（ISR）和一个高频交互流形执行器（IME）。ISR通过时间扭矩编码器和力感知器编码有序的扭矩历史，作为状态一致性动作生成的持久物理证据，而IME将接触关键运动结构化为内在进展和横向修正，以实现精确执行；它们的跨频率循环允许生成的物理轨迹为后续推理提供信息。在表面、插入和旋转交互的真实机器人实验中，HIRE在所有评估的任务阶段中实现了至少90%的完成率，同时改善了交互状态的消歧、执行精度和泛化能力。更广泛地说，HIRE提供了一个统一的推理-执行视角，针对历史依赖的部分可观测性下的精确操作，其中物理交互既实现了任务意图，又揭示了未来决策的潜在状态证据。代码将在发表后发布。

View on arXiv Download PDF AI Translation

cs.RO / 39 / 2609.30833

Fast Plans, Faithful Actions: Closing the Planning-Execution Gap in Hierarchical Vision-Language-Action Models

快速计划，忠实行动：缩小层次化视觉-语言-行动模型中的规划与执行差距

Xie, Chuanliang, Ma, Boyu, Li, Gen, Liu, Yizhou, Chen, Houwang, Zhou, Xinyu, Yang, Jianfei

Abstract

Hierarchical vision-language-action (VLA) systems consist of a high-level vision-language planner and a low-level action expert that generates continuous actions. This hierarchical design has practical value only if the planner can generate plans fast enough to meet real-time control requirements, and the resulting plans actually contribute to the generation of action. We study one such system, a waypoint hierarchy pipeline adapted from $\pi_{0.5}$, and find that neither requirement is satisfied. This baseline relies on token-level autoregressive decoding (Token-AR) to generate a waypoint plan, requiring 57 very expensive vision-language model (VLM) forward passes. However, we find that erasing the waypoint endpoints has little effect on task success. Two findings reveal the misalignment of planner-executor: the planner generates outputs at an excessively fine granularity, and the executor underuses plans as a control condition. We address the latency issue with waypoint-aligned block-autoregressive decoding (Block-AR), and plan underuse issue with normalized goal modulation (NGM), a layer-wise goal path constrained by phase gating and anti-shortcut training so that the waypoint influences action generation maintaining other signals. Our method reduces the maximum number of VLM forward passes from 57 to 8 on LIBERO, including one prefix prefill, and achieves an $8.7\times$ reduction in planning latency on a Rokae dual-arm robot. With normalized goal modulation and anti-shortcut training, Block-AR's success rate on LIBERO-Long increases from 91.0% to 96.2%, while its average success rate across the four suites increases from 95.85% to 98.45%. On three bimanual tasks with this robot, success rates remain comparable across methods.

Chinese Translation

层次化视觉-语言-行动（VLA）系统由高层次的视觉-语言规划器和低层次的行动专家组成，后者生成连续的行动。这种层次化设计只有在规划器能够快速生成满足实时控制要求的计划时才具有实际价值，而生成的计划实际上也必须有助于行动的生成。我们研究了一个这样的系统，一个改编自 $ ext{π}_{0.5}$ 的航点层次管道，发现这两个要求都未得到满足。该基线依赖于基于标记的自回归解码（Token-AR）生成航点计划，需进行57次非常昂贵的视觉-语言模型（VLM）前向传递。然而，我们发现删除航点端点对任务成功的影响很小。两个发现揭示了规划者与执行者之间的不一致：规划者生成的输出粒度过细，而执行者对计划的使用不足。我们通过航点对齐的块自回归解码（Block-AR）解决了延迟问题，并通过归一化目标调制（NGM）解决了计划使用不足的问题，NGM 是一种通过阶段门控和反快捷训练约束的逐层目标路径，使得航点影响行动生成，同时保持其他信号。我们的方法将 LIBERO 上 VLM 前向传递的最大次数从 57 次减少到 8 次，包括一次前缀填充，并在 Rokae 双臂机器人上实现了 $8.7 imes$ 的规划延迟减少。通过归一化目标调制和反快捷训练，Block-AR 在 LIBERO-Long 上的成功率从 91.0% 提高到 96.2%，而其在四个套件上的平均成功率从 95.85% 提高到 98.45%。在与该机器人进行的三个双手任务中，各方法的成功率保持相当。

View on arXiv Download PDF AI Translation

cs.RO / 40 / 2609.30842

Impedance Cloning: Learning Equilibrium Point Parameters for Contact-Rich Manipulation

阻抗克隆：学习接触丰富操作的平衡点参数

Takahashi, Hayato, Oishi, Ryoga, Kasuga, Yuki, Tsuji, Toshiaki

Abstract

Contact-rich manipulation requires robots to regulate force against surfaces whose geometry deviates unpredictably from training conditions. Trajectory-based imitation learning, which reproduces observable outputs, breaks down under such shifts. We propose Impedance Cloning, which instead imitates the biomechanical priors that generate motion -- the stiffness and equilibrium point -- and thereby passively absorbs contact uncertainty. Because these parameters encode intent rather than outcome, they generalize across surface geometries where trajectory reproduction does not. We extract them from bilateral teleoperation demonstrations via a particle filter without force/torque sensors and evaluate the framework on two CRANE-X7 manipulators. In a wiping task with joint-space actions, the trajectory-based baseline loses contact below -6 cm, whereas the proposed method maintains a consistent 4-5 N contact force above -6 cm, with a gradual decrease below; with Cartesian-space actions, its force-height slope over 0 to +8 cm is 0.13 +/- 0.03 N/cm, versus 0.34-0.83 N/cm for fixed-impedance baselines. In a pick-and-place task with 10 diverse cups (100 trials), the proposed method succeeds in 84 trials, outperforming the fixed-impedance baseline (74/100) and performing comparably to a variable impedance control baseline (82/100) with one demonstration instead of ten. In a grasping task, the representation reduces torque tracking error with both ILBiT and Mamba backbones, confirming its generality across architectures.

Chinese Translation

接触丰富的操作要求机器人在几何形状与训练条件偏离时调节对表面的力。基于轨迹的模仿学习在这种变化下会失效。我们提出了阻抗克隆（Impedance Cloning），它模仿生成运动的生物力学先验——刚度和平衡点，从而被动地吸收接触不确定性。由于这些参数编码的是意图而非结果，因此它们能够在轨迹重现无法适用的表面几何形状中进行泛化。我们通过粒子滤波器从双向遥操作演示中提取这些参数，而无需力/扭矩传感器，并在两个CRANE-X7操纵器上评估该框架。在一个关节空间动作的擦拭任务中，基于轨迹的基线在-6 cm以下失去接触，而所提方法在-6 cm以上保持一致的4-5 N接触力，并在以下逐渐减少；在笛卡尔空间动作中，其在0到+8 cm的力-高度斜率为0.13 +/- 0.03 N/cm，而固定阻抗基线为0.34-0.83 N/cm。在一个包含10个不同杯子的抓取与放置任务中（100次试验），所提方法在84次试验中成功，优于固定阻抗基线（74/100），并且在一次演示而非十次的情况下，与可变阻抗控制基线（82/100）表现相当。在一个抓取任务中，该表示在ILBiT和Mamba骨干网络下减少了扭矩跟踪误差，确认了其在不同架构中的通用性。

View on arXiv Download PDF AI Translation

cs.RO / 41 / 2609.30868

VLaRL: Augmenting Vision-Language-Action Models with Simulation-Trained Latent-Conditioned Residual RL

VLaRL：通过模拟训练的潜在条件残差强化学习增强视觉-语言-动作模型

Saito, Namiko, Kim, Kinam, Kim, Heecheol, Ikeuchi, Katsushi, Matsushita, Yasuyuki

Abstract

Vision-language-action (VLA) models provide broad, instruction-conditioned manipulation behaviors, but their physical execution can remain imprecise during contact-rich interaction. Residual reinforcement learning (RL) can correct such errors while keeping the VLA frozen, but real-robot RL is costly and safety-critical. We propose VLA Latent-Conditioned RL (VLaRL), which enables residual RL for frozen VLAs to be trained in simulation and deployed on real robots without real-world RL or online adaptation. The key challenge is transferring the learned residual policy despite the visual gap between simulation and reality. Rather than requiring pixel-level visual correspondence, VLaRL uses the VLA's internal vision-language latent representation to condition residual control and as the sim-to-real transfer interface, and learns a lightweight mapper that transforms simulation-derived latents toward the real latent distribution. Across four contact-rich manipulation tasks and two VLA backbones, VLaRL improves real-world success in all task-backbone combinations, while controlled ablations demonstrate the importance of both latent conditioning and latent alignment for transferring simulation-trained residual control.

Chinese Translation

视觉-语言-动作（VLA）模型提供了广泛的、基于指令的操作行为，但在接触丰富的交互过程中，其物理执行可能仍然不够精确。残差强化学习（RL）可以在保持VLA不变的情况下纠正这些错误，但真实机器人上的RL成本高且安全性至关重要。我们提出了VLA潜在条件RL（VLaRL），该方法使得冻结的VLA能够在模拟中进行残差RL训练，并在真实机器人上部署，而无需进行真实世界的RL或在线适应。关键挑战在于如何在模拟与现实之间的视觉差距中转移学习到的残差策略。VLaRL并不要求像素级的视觉对应，而是利用VLA的内部视觉-语言潜在表示来条件化残差控制，并作为模拟到现实的转移接口，同时学习一个轻量级的映射器，将模拟生成的潜在向量转换为真实潜在分布。在四个接触丰富的操作任务和两个VLA骨干网络上，VLaRL在所有任务-骨干组合中提高了真实世界的成功率，而控制性消融实验则证明了潜在条件和潜在对齐在转移模拟训练的残差控制中的重要性。

View on arXiv Download PDF AI Translation

cs.RO / 42 / 2609.30873

A Second Torque Port for Series Elastic Actuators: Parallel-Integrated Design and Time-Scale Torque Allocation

系列弹性驱动器的第二扭矩端口：并联集成设计与时间尺度扭矩分配

Jia, Donghao, Xia, Xiubo, Liu, Junhang, Zhu, Zeyu, Geng, Xiaoyu, Sun, Jian

Abstract

A series elastic actuator has a single torque port and pays for it twice: the geared motor must swing its own reflected inertia through the spring, so the amplitude it delivers collapses as $\omega^{-2}$ in the command frequency $\omega$ once it saturates, while commands below the transmission's breakaway friction never arrive at all. This letter opens a second torque port on the load side, placing a frameless direct-drive micro motor in parallel with a fixed-stiffness spring -- a parallel-integrated SEA, or Pi-SEA, whose delivered torque is read from spring deflection and micro current without a sensor -- and dividing the commanded torque between the two channels by time scale rather than by filter design. The micro torque loop is the fast subsystem, which makes the closed loop singularly perturbed and turns the separation the channels need into a bound to check rather than a crossover to tune; a leaky mid-ranging integrator returns the steady load to the spring; and the amplitude ceiling, read backwards, becomes a closed-form sizing rule that matches spring, geared motor and micro motor to the amplitudes and frequencies an application asks for. Against SEAs, the Pi-SEA widens the tracked band at small amplitudes and lowers the residual the joint imposes on its environment, each by an order of magnitude.

Chinese Translation

系列弹性驱动器具有单一的扭矩端口，因此付出了双重代价：齿轮电机必须通过弹簧摆动其自身的反射惯性，因此一旦饱和，其传递的幅度在指令频率 $ heta$ 下以 $ heta^{-2}$ 的速率衰减，而低于传动装置的起步摩擦力的指令则根本无法到达。本文在负载侧开启了第二个扭矩端口，将一个无框直接驱动微电机与一个固定刚度的弹簧并联——即并联集成的系列弹性驱动器（Pi-SEA），其传递的扭矩通过弹簧的变形和微电流读取，无需传感器——并通过时间尺度而非滤波设计在两个通道之间分配指令扭矩。微扭矩回路是快速子系统，使得闭环系统呈现单一扰动特性，并将通道所需的分离转化为一个需要检查的界限，而不是一个需要调节的交叉点；一个漏泄的中频积分器将稳态负载返回到弹簧；而反向读取的幅度上限则成为一个闭式形式的尺寸规则，将弹簧、齿轮电机和微电机与应用所需的幅度和频率匹配。与系列弹性驱动器相比，Pi-SEA 在小幅度下扩大了跟踪带宽，并降低了关节对其环境施加的残余负担，每项均提高了一个数量级。

View on arXiv Download PDF AI Translation

cs.RO / 43 / 2609.30889

PHASE: Compliance-Enabled Tactile Phase Retrieval for Few-Shot Insertion Learning

PHASE：面向少量示例插入学习的合规性支持触觉相位检索

Siburian, Jeremy, Beltran-Hernandez, Cristian C., Matsushima, Tatsuya, Iwasawa, Yusuke, Hamaya, Masashi, Nishimura, Mai

Abstract

Contact-rich assembly tasks such as peg-in-hole insertion remain difficult to learn from limited demonstrations. While retrieval-augmented imitation learning, which augments target demonstrations with relevant prior data, offers a promising direction, its applicability to contact-rich manipulation remains largely unexplored. Contact-rich insertion unfolds over multiple phases from search to insert, and retrieving phase-specific experience from prior data in principled ways remains an open question. Our key insight is that a compliant wrist enables the robot to sustain contact throughout execution, producing rich tactile and force signals that naturally reveal the phase structure of insertion and inform what should be retrieved. Based on this insight, we present PHASE (PHase-Aware Segmentation and REtrieval), a framework for compliance-enabled tactile phase retrieval that integrates multimodal contact-aware representation learning, variable-length phase segmentation from tactile signals, and phase-consistent retrieval for policy learning. We evaluate PHASE on real-world peg-in-hole insertion across five peg geometries, comparing against retrieval strategies drawn from state-of-the-art methods under a shared policy architecture. PHASE improves the overall success rate by 13 percentage points over the strongest non-phase-aware baseline, and improves performance under unseen initial positions by 30 percentage points. These results demonstrate that aligning retrieval with interaction-defined contact phases substantially improves robustness in few-shot insertion learning.

Chinese Translation

接触丰富的组装任务，如插销入孔，仍然难以从有限的示例中学习。虽然增强目标示例的检索增强模仿学习（retrieval-augmented imitation learning）提供了一个有前景的方向，但其在接触丰富的操作中的适用性仍然很大程度上未被探索。接触丰富的插入过程分为多个阶段，从搜索到插入，从原则上检索特定阶段的经验仍然是一个未解的问题。我们的关键见解是，合规性腕部使机器人能够在执行过程中保持接触，产生丰富的触觉和力信号，这自然揭示了插入的阶段结构并告知应检索的内容。基于这一见解，我们提出了PHASE（PHase-Aware Segmentation and REtrieval），一个支持合规性触觉相位检索的框架，集成了多模态接触感知表示学习、基于触觉信号的可变长度阶段分割和用于策略学习的阶段一致性检索。我们在五种插销几何形状的真实世界插销入孔任务中评估了PHASE，并与在共享策略架构下的最先进方法的检索策略进行了比较。PHASE相比于最强的非相位感知基线提高了13个百分点的整体成功率，并在未见初始位置下提高了30个百分点的性能。这些结果表明，将检索与交互定义的接触阶段对齐显著提高了少量示例插入学习的鲁棒性。

View on arXiv Download PDF AI Translation

cs.RO / 44 / 2609.30913

Causeway: Restoring Task Accessibility for Instruction Switching in VLA Policies

Causeway：恢复VLA策略中指令切换的任务可达性

Wang, Qingzi, Feng, Kaixi, Shi, Guangyao, Wu, Xiyang, Li, Ang, Manocha, Dinesh

Abstract

Vision-language-action (VLA) policies can execute many tasks from standard initial states, yet a new instruction may fail after another task has altered the robot's physical state. We study instruction switching, where a new task is issued during or after the execution of a different one. We observe that a target task that is reliably completed from its standard initial states can become inaccessible from states produced by a preceding task. We call such states task islands. We propose Causeway, a training-free inference-time intervention. Given the current state and a re-entry pose for the target task, Causeway back-propagates through the frozen decoding computation and applies a state-directed write within the action-stream representation. The VLA decodes the return motion itself, without parameter updates, a new action head, or external action generation. Across 71 cross-object pairs, three switch timings, and three VLA architectures on LIBERO-Goal, Causeway raises bare-switch success from 3-26% to 47-65% and increases the rate of reaching the handoff neighborhood by 42-72 percentage points across models. Additional experiments on LIBERO-Object and a real xArm platform show that the recovery extends beyond the main LIBERO-Goal setting, both in simulation and on a robot.

Chinese Translation

视觉-语言-行动（VLA）策略可以从标准初始状态执行许多任务，但在另一个任务改变了机器人的物理状态后，新的指令可能会失败。我们研究了指令切换，即在执行不同任务期间或之后发出新的任务指令。我们观察到，从其标准初始状态可靠完成的目标任务可能会在前一个任务产生的状态下变得不可达。我们将这种状态称为任务岛（task islands）。我们提出了Causeway，这是一种无训练的推理时干预方法。给定当前状态和目标任务的重新进入姿态，Causeway通过冻结的解码计算进行反向传播，并在动作流表示中应用状态导向的写入。VLA自身解码返回运动，无需参数更新、新的动作头或外部动作生成。在LIBERO-Goal上，针对71对跨对象、三种切换时机和三种VLA架构，Causeway将裸切换成功率从3-26%提高到47-65%，并在模型间将达到交接邻域的比例提高了42-72个百分点。在LIBERO-Object和真实的xArm平台上的额外实验表明，恢复效果超出了主要的LIBERO-Goal设置，无论是在仿真还是在机器人上。

View on arXiv Download PDF AI Translation

cs.RO / 45 / 2609.30951

Bundled Contact Gradients: Stabilizing Differentiable Simulation for Deployable Dynamic Tasks

捆绑接触梯度：为可部署动态任务稳定可微仿真

Aditya, Dyuman, Cheng, Jin, Schwarke, Clemens, Nguyen, Quan, Sukhatme, Gaurav, Coros, Stelian, Fadini, Gabriele

Abstract

Differentiable simulation provides analytic gradients of robot dynamics, enabling fast and sample-efficient first-order policy optimization. However, obtaining smooth and informative gradients through rigid-body contact typically requires softened contact models, often at the expense of physical fidelity and thereby limiting learned policies largely to simulation. This trade-off becomes particularly consequential for dynamic humanoid motions, where accurate contact dynamics are critical for transferring policies to the real world. Increasing contact stiffness in rigid-body simulation improves the fidelity of interactions, but also makes the dynamics increasingly sensitive to small state perturbations, producing high-variance gradients that can destabilize first-order policy learning. To address this, we propose \emph{Bundled Contact Gradients (BCG)}, a contact-local randomized smoothing framework for differentiable policy learning. When stiff contact is detected, our method evaluates a local bundle of randomized perturbation rollouts around the stiff contact configuration and aggregates their gradient signal thereby reducing gradient variance. We demonstrate the effectiveness of our method by successfully training and transferring dynamic motions zero-shot onto a real-world Unitree G1 humanoid platform. Videos and supplementary information can be found at https://bundledcontactgradients.github.io/

Chinese Translation

可微仿真提供了机器人动力学的解析梯度，使得一阶策略优化快速且样本高效。然而，通过刚体接触获得平滑且信息丰富的梯度通常需要软化接触模型，这往往以物理逼真性为代价，从而使得学习到的策略主要局限于仿真。这种权衡在动态人形运动中尤为重要，因为准确的接触动力学对于将策略转移到现实世界至关重要。在刚体仿真中增加接触刚度可以提高交互的逼真性，但也使得动力学对小状态扰动变得更加敏感，产生高方差的梯度，这可能会使一阶策略学习不稳定。为了解决这个问题，我们提出了 extit{捆绑接触梯度（Bundled Contact Gradients, BCG）}，这是一种用于可微策略学习的接触局部随机平滑框架。当检测到刚性接触时，我们的方法在刚性接触配置周围评估一组随机扰动的局部捆绑，并聚合它们的梯度信号，从而降低梯度方差。我们通过成功训练并将动态运动零-shot转移到现实世界的Unitree G1人形平台上，展示了我们方法的有效性。视频和补充信息可以在 https://bundledcontactgradients.github.io/ 找到。

View on arXiv Download PDF AI Translation

cs.RO / 46 / 2609.30959

VisTacAlign: Co-Training Dexterous Policies on Tactile Human and Robot Demonstrations

VisTacAlign：在触觉人类和机器人示范上共同训练灵巧策略

Poffet, Julien, Strong, Matthew, Dhawan, Ankush, Shi, Baiyu, Neelaveni, Shalika, Yuan, Yujia, Bao, Zhenan, Kennedy III, Monroe

Abstract

Human demonstrations are a cheap source of data for dexterous manipulation, but co-training a robot policy on them requires closing the human--robot gap in every modality the policy consumes. We present VisTacAlign, a framework for co-training 3D-visual-tactile dexterous policies on human and robot demonstrations. Glove-tracked human hand motion is retargeted to a 17-DoF tactile robot hand with a one-time fingertip correction. The human hand is then erased from both stereo views and replaced by a posed robot-hand mesh painted with pixels from robot recordings, and a real-time stereo foundation model is re-run on the composite, so the human point clouds carry the same stereo errors and visibility as the robot ones. Finally, a capacitive tactile glove is aligned to the robot's fingertip sensors in its signal space, giving one interpretable per-finger force representation. A diffusion transformer consumes point-cloud, proprioceptive, and per-finger tactile tokens. On three real-world tasks requiring precise force -- Lego assembly, plucking strawberries of varying size, and activating and lifting a power drill -- adding aligned human demonstrations to existing robot data improves over robot-only policies, and ablations show that both tactile input and visual alignment are necessary. Project page: https://vis-tac-align.github.io

Chinese Translation

人类示范是灵巧操作的一种廉价数据来源，但在这些示范上共同训练机器人策略需要在策略所使用的每种模态中缩小人机差距。我们提出了VisTacAlign，一个在触觉人类和机器人示范上共同训练3D视觉-触觉灵巧策略的框架。手套跟踪的人手运动经过一次性指尖校正后被重新定向到一个17自由度的触觉机器人手。然后，从两个立体视图中擦除人手，并用从机器人录制中获取的像素绘制的姿态机器人手网格替代，并在复合体上重新运行实时立体基础模型，以便人类点云携带与机器人点云相同的立体误差和可见性。最后，一个电容式触觉手套在其信号空间中与机器人的指尖传感器对齐，提供一个可解释的每指力表示。扩散变换器处理点云、身体感知和每指触觉标记。在三个需要精确施力的真实世界任务中——乐高组装、采摘不同大小的草莓，以及激活和提起电钻——将对齐的人类示范添加到现有的机器人数据中，优于仅使用机器人策略的表现，消融实验表明触觉输入和视觉对齐都是必要的。项目页面：https://vis-tac-align.github.io

View on arXiv Download PDF AI Translation

cs.RO / 47 / 2609.30965

FRAM: Trajectory-Guided Visual Feature Selection for Compact Language-Conditioned Robot Manipulation

FRAM：基于轨迹引导的视觉特征选择用于紧凑的语言条件机器人操作

Ito, Hiroshi, Hiruma, Hyogo, Kanai, Yoshiki, Yoshida, Takahiro, Kanazawa, Akira, Yamada, Hiroki

Abstract

Vision-Language-Action models achieve strong performance in robot manipulation, but often require large numbers of parameters. In this work, we propose the Future Representation Action Model (FRAM), a small policy that explicitly links the future end-effector trajectory to the current visual input. FRAM uses the image coordinates of the predicted trajectory as spatial pointers and reads local visual features related to the motion from the current image. This organizes the information for action generation into the reference position (Where), the visual state (What), and the future motion (Future). Trajectory labels are generated automatically from demonstrations and camera geometry, so no manual annotation is needed. With 138.7M parameters, including a frozen language encoder, FRAM reaches an average success rate of 92.2% over the four standard LIBERO suites, close to the 94.2% of $\pi_0$ with 3.3B parameters. Without extra training, it also reaches an average of 67.3% on LIBERO-Plus. Ablations confirm that both the future trajectory and the local visual features improve performance and robustness. On a real dual-arm UR5e, FRAM stacks cups using only wrist cameras, including choosing and switching between the left and right arms. These results show that selecting visual information based on future motion is an effective way to obtain both high performance and robustness in a small robot policy.

Chinese Translation

视觉-语言-动作模型在机器人操作中表现出色，但通常需要大量参数。在本研究中，我们提出了未来表示动作模型（Future Representation Action Model，FRAM），这是一个小型策略，明确将未来末端执行器轨迹与当前视觉输入关联。FRAM使用预测轨迹的图像坐标作为空间指针，从当前图像中读取与运动相关的局部视觉特征。这将动作生成所需的信息组织为参考位置（Where）、视觉状态（What）和未来运动（Future）。轨迹标签通过演示和相机几何自动生成，因此无需手动标注。FRAM的参数量为1.387亿，包括一个冻结的语言编码器，在四个标准LIBERO套件上达到了92.2%的平均成功率，接近参数为33亿的$ ext{π}_0$的94.2%。在没有额外训练的情况下，它在LIBERO-Plus上也达到了67.3%的平均成功率。消融实验确认，未来轨迹和局部视觉特征均能提高性能和鲁棒性。在真实的双臂UR5e上，FRAM仅使用腕部摄像头堆叠杯子，包括选择和切换左右手臂。这些结果表明，基于未来运动选择视觉信息是一种在小型机器人策略中获得高性能和鲁棒性的有效方法。

View on arXiv Download PDF AI Translation

cs.RO / 48 / 2609.30969

TACTIC: Understanding Tactile Encoders and Conditioning for Contact-rich Robot Manipulation Policies

TACTIC：理解触觉编码器及其在接触丰富的机器人操作策略中的条件作用

Bien, Seongjin, Makowski, Débora Oliveira, Kneissl, Carlo, Mirjalili, Reihaneh, Vanjani, Pankhuri, Lioutikov, Rudolf, Kutyniok, Gitta, Walter, Florian, Burgard, Wolfram

Abstract

Tactile information is essential for contact-rich manipulation tasks in robotics. Vision-based tactile sensors make it particularly easy to design end-to-end manipulation policies with tactile sensing, as they enable the use of existing encoders from computer vision. However, this has led to a huge variety of architectures, training datasets, and evaluation protocols, making it difficult to determine which design choices best encode touch. In this work, we address this gap and present a comprehensive study of tactile encoders and fusion strategies across various contact-rich manipulation tasks in real-world experiments. To enable a controlled comparison, we train and evaluate all models under the same pipeline and experimental setup, comprising more than 2000 real-world rollouts. Our results go beyond other studies that only compare simulation performance, which does not necessarily translate to real-world settings, where large-scale evaluations are needed to obtain reliable statistics. Our key finding is that there is no universally optimal representation or fusion strategy for encoding visual-tactile. Instead, the best encoder backbone and fusion scheme depend strongly on the task.

Chinese Translation

触觉信息对于机器人中的接触丰富操作任务至关重要。基于视觉的触觉传感器使得设计端到端的触觉感知操作策略变得尤为简单，因为它们能够利用计算机视觉中的现有编码器。然而，这导致了各种架构、训练数据集和评估协议的巨大多样性，使得很难确定哪些设计选择能够最佳地编码触觉。在本研究中，我们解决了这一空白，并对各种接触丰富操作任务中的触觉编码器和融合策略进行了全面研究，基于真实世界的实验。为了实现可控比较，我们在相同的流程和实验设置下训练和评估所有模型，涵盖了超过2000次的真实世界实验。我们的结果超越了仅比较模拟性能的其他研究，因为模拟性能不一定能转化为真实世界的设置，在真实世界中需要进行大规模评估以获得可靠的统计数据。我们的关键发现是，没有一种普遍最佳的表示或融合策略用于编码视觉-触觉。相反，最佳的编码器骨干和融合方案在很大程度上依赖于具体任务。

View on arXiv Download PDF AI Translation

cs.RO / 49 / 2609.31008

Quadruped Obstacle Avoidance and Footstep Planning with Distributed Low-cost Time-of-Flight Sensors

基于分布式低成本飞行时间传感器的四足机器人障碍物避让与步态规划

Caroleo, Giammarco, Mahamoodally, Timothée, Manzardo, Matteo, Jin, Jin, Pontin, Marco, Mattamala, Matias, Vidoni, Renato, Maiolino, Perla, Fallon, Maurice

Abstract

Quadruped robots typically rely on depth cameras and LiDAR sensors to map their local environment. However, these sensors have limited close-range coverage, are relatively expensive, and consume significant power. This study investigates whether distributed Time-of-Flight (ToF) sensors can serve as a low-cost alternative to depth cameras for near-field terrain mapping for locomotion and local navigation. We designed a distributed ToF sensing architecture for the ANYbotics ANYmal quadruped, assessed its environment reconstruction accuracy, and benchmarked it against depth cameras for terrain mapping and obstacle avoidance. Distributing these sensors around the robot can also avoid the blind spots of traditional sensors. Our results show that, despite their low resolution and higher measurement noise, distributed ToF sensors can support reliable perceptual locomotion with centimeter-level local mapping accuracy. The proposed sensing strategy provides sufficient geometric information for near-field obstacle avoidance and footstep planning, at substantially lower cost, energy consumption, and system complexity than depth cameras.

Chinese Translation

四足机器人通常依赖深度相机和激光雷达传感器来映射其局部环境。然而，这些传感器在近距离覆盖方面有限，成本相对较高，并且消耗大量电力。本研究探讨了分布式飞行时间（Time-of-Flight, ToF）传感器是否可以作为深度相机在近场地形映射中的低成本替代方案，以支持运动和局部导航。我们为ANYbotics的ANYmal四足机器人设计了一种分布式ToF传感架构，评估了其环境重建的准确性，并与深度相机在地形映射和障碍物避让方面进行了基准测试。将这些传感器分布在机器人周围还可以避免传统传感器的盲区。我们的结果表明，尽管分布式ToF传感器的分辨率较低且测量噪声较高，但它们仍能支持可靠的感知运动，具有厘米级的局部映射精度。所提出的传感策略提供了足够的几何信息，用于近场障碍物避让和步态规划，其成本、能耗和系统复杂性均显著低于深度相机。

View on arXiv Download PDF AI Translation

cs.RO / 50 / 2609.31014

Co-design of trajectory and morphology for a vertical jump-climbing robot

垂直跳跃攀爬机器人轨迹与形态的协同设计

Xu, Christopher Y., Hawkes, Elliot W.

Abstract

Animals such as squirrels and even bears have adapted to rapidly climb up trees and other complex vertical terrain, but achieving comparable agility has been a challenge for climbing robots. Existing robots often use walking gaits and move conservatively to stay in contact with the surface, which limits the range of dynamic maneuvers. In this paper we present a 290 g robot that, to our knowledge, is the first to climb vertically by bounding (with an aerial phase). We leverage a co-design workflow, in which the morphology and trajectory are jointly optimized for fast locomotion, subject to adhesion force limitations seen in spined grippers. The resulting trajectory includes a rapid maneuver that launches the robot vertically, and an aerial reorientation that brings the front grippers back to the surface using the rear leg as an inertial tail. We evaluate the resulting jump forces in 2D force space and demonstrate that the optimized morphology is capable of continuous climbing at a speed of 0.375 m/s (1.97 body lengths/s), and can also achieve ground locomotion and transition to a vertical surface. Our work proposes design insights for the jump-climbing maneuver and serves as an important step toward creating climbing robots with agility on par with that of animals.

Chinese Translation

像松鼠甚至熊这样的动物已经适应了快速攀爬树木和其他复杂的垂直地形，但对于攀爬机器人来说，实现相当的灵活性一直是一个挑战。现有的机器人通常采用步态行走，并以保守的方式移动以保持与表面的接触，这限制了动态机动的范围。在本文中，我们介绍了一种重290克的机器人，据我们所知，这是第一种通过弹跳（具有空中阶段）垂直攀爬的机器人。我们利用了一种协同设计工作流程，在该流程中，形态和轨迹共同优化以实现快速运动，同时受到刺状抓手所见的附着力限制。最终的轨迹包括一个快速的机动，使机器人垂直发射，以及一个空中重新定向，使前抓手通过后腿作为惯性尾巴重新回到表面。我们在二维力空间中评估了结果跳跃力，并证明优化后的形态能够以0.375米/秒（1.97体长/秒）的速度持续攀爬，并且还能够实现地面运动并过渡到垂直表面。我们的工作为跳跃攀爬机动提供了设计见解，并为创建与动物灵活性相当的攀爬机器人迈出了重要一步。

View on arXiv Download PDF AI Translation

cs.RO / 51 / 2609.31025

Precision at Speed: Sample-Efficient Online Model-Based Reinforcement Learning for Hydraulic Excavator Control

高速精确控制：用于液压挖掘机控制的样本高效在线模型基础强化学习

Canales, Claudio, Nan, Fang, Hutter, Marco, Ruiz-del-Solar, Javier

Abstract

Precise, high-speed control remains challenging for robots with complex actuation dynamics. Learning directly on hardware is further constrained by the cost of real-world interaction. We present an online model-based reinforcement learning framework that learns a probabilistic dynamics ensemble model from scratch for sampling-based model predictive control. A precision-gated contouring objective conditions the progress reward on path accuracy, prioritizing precision over speed. In a data-driven excavator simulator, the framework achieves higher sample efficiency than the evaluated model-based reinforcement learning baselines. We validate the framework by learning directly on an 11.5-ton Menzi Muck M445 hydraulic excavator, without demonstrations or simulation pretraining. After 20 minutes of interaction, the controller reaches tracking accuracy comparable to prior learned controllers trained on 100-150 minutes of data. After 40 minutes, it sustains sub-centimeter mean path error at high operating speeds.

Chinese Translation

对于具有复杂驱动动态的机器人，精确的高速控制仍然具有挑战性。在硬件上直接学习受到现实世界交互成本的进一步限制。我们提出了一种在线模型基础强化学习框架，该框架从零开始学习一个概率动态集成模型，用于基于采样的模型预测控制。精度门控的轮廓目标将进度奖励与路径精度相结合，优先考虑精度而非速度。在一个数据驱动的挖掘机模拟器中，该框架实现了比评估的模型基础强化学习基线更高的样本效率。我们通过在一台11.5吨的Menzi Muck M445液压挖掘机上直接学习来验证该框架，未使用演示或模拟预训练。在20分钟的交互后，控制器达到了与之前在100-150分钟数据上训练的控制器相当的跟踪精度。在40分钟后，它在高操作速度下保持亚厘米的平均路径误差。

View on arXiv Download PDF AI Translation

cs.RO / 52 / 2609.31035

Compact Force Sensor for Dual-UAV Cable-Suspended Payload Transport with Tension-Aware Outer-Loop Control

用于双无人机缆索悬挂载荷运输的紧凑型力传感器及其张力感知外环控制

Delbene, Andrea, Cannata, Giorgio, Carlini, Giorgio, Sante, Filippo, Baglietto, Marco

Abstract

Cooperative payload transportation using multiple \textit{Unmanned Aerial Vehicles} (UAVs) poses challenges in stability, coordination, and robustness, especially under external disturbances and unmodeled dynamics. This work proposes a dual-UAV payload transportation framework supported by a compact, custom-designed force sensor measuring the interaction force at the UAV cable anchor point. The sensor design and mathematical model are presented, and its performance is characterized through static and dynamic tests evaluating linearity, hysteresis, repeatability, and crossload. The control architecture follows a cascade structure: fast inner loops handle vehicle stabilization, while outer loops are designed to compensate for the measured forces. The approach is validated through simulations and indoor experiments under position uncertainty. Payload-drop and constrained-space tests assess the proposed sensing and control architecture against literature-based distributed references, showing improved stabilization, coordination, and disturbance rejection. A video of the experiments is available at: https://youtu.be/rIw9-fvV8Qw.

Chinese Translation

多无人机（UAV）协作载荷运输在稳定性、协调性和鲁棒性方面面临挑战，尤其是在外部干扰和未建模动态情况下。本研究提出了一种双无人机载荷运输框架，支持一个紧凑型、定制设计的力传感器，用于测量无人机缆索锚点的相互作用力。本文介绍了传感器的设计和数学模型，并通过静态和动态测试对其性能进行了表征，评估了线性度、滞后性、重复性和交叉负载。控制架构采用级联结构：快速内环处理车辆稳定性，而外环则旨在补偿测量到的力。通过在位置不确定性下的仿真和室内实验验证了该方法。载荷释放和受限空间测试将所提出的传感和控制架构与文献中基于分布式参考的方案进行了比较，显示出在稳定性、协调性和干扰抑制方面的改善。实验视频可在以下链接查看：https://youtu.be/rIw9-fvV8Qw。

View on arXiv Download PDF AI Translation

cs.RO / 53 / 2609.31048

Kintsugi-VLA: Turning Failed Robot Rollouts into Recovery Data through Interventional Recoverability

金缮-VLA：通过干预可恢复性将失败的机器人轨迹转化为恢复数据

Snegirev, Ivan, Semenyakina, Elizaveta, Maliukov, Dmitrii, Cabrera, Miguel Altamirano, Tsetserukou, Dzmitry

Abstract

Simulation enables scalable training of Vision-Language-Action policies by using privileged experts to generate visual demonstrations without requiring every trajectory to be collected through manual teleoperation. However, such pipelines typically retain successful demonstrations while failed rollouts are discarded, even though they expose precisely the off-nominal states from which recovery must be learned. We introduce Kintsugi-VLA, a framework for converting failed rollouts into targeted synthetic recovery data by exploiting exact state restoration and branching in simulation. For a fixed privileged expert, we define interventional recoverability as the probability of completing the original task after the simulator is restored to a given state, estimate it using adaptive Monte Carlo continuations with pointwise Wilson intervals, and characterize its non-monotonic evolution along failed trajectories. These estimates identify an observed terminal low-recoverability frontier-the point after which measured recoverability remains below a threshold-which is then used to select informative recovery starting states. In a simulated Franka manipulation task, targeted recovery data yield aggregate SmolVLA recovery success of 34.6\% and 38.4\% under difficulty- and frame-budget matching, respectively, 5.8 and 6.7 percentage points above uniform sampling within the same recovery window. The same ordering is observed under disturbed end-to-end execution and shifted clutter and physics conditions, while clean-task success decreases from 76.8\% to 74.7\%. Kintsugi-VLA demonstrates how failed simulator rollouts can be transformed from discarded experience into structured recovery-training data through direct interventional measurement.

Chinese Translation

模拟使得通过特权专家生成视觉演示来进行可扩展的视觉-语言-动作（Vision-Language-Action）策略训练成为可能，而无需通过手动遥控收集每个轨迹。然而，这种流程通常保留成功的演示，而将失败的轨迹丢弃，尽管这些失败的轨迹恰恰暴露了需要学习恢复的非正常状态。我们提出了金缮-VLA，一个通过利用精确状态恢复和模拟中的分支，将失败的轨迹转化为有针对性的合成恢复数据的框架。对于固定的特权专家，我们将干预可恢复性定义为在模拟器恢复到给定状态后完成原始任务的概率，使用自适应蒙特卡洛延续法和点估计威尔逊区间进行估计，并描述其在失败轨迹上的非单调演变。这些估计识别出一个观察到的终端低可恢复性边界——在此之后测得的可恢复性保持在一个阈值以下——然后用于选择信息丰富的恢复起始状态。在模拟的Franka操作任务中，有针对性的恢复数据在难度和帧预算匹配下分别实现了34.6%和38.4%的整体SmolVLA恢复成功率，比同一恢复窗口内的均匀采样高出5.8和6.7个百分点。在干扰的端到端执行以及变化的杂乱和物理条件下也观察到了相同的排序，而干净任务的成功率从76.8%降至74.7%。金缮-VLA展示了如何将失败的模拟器轨迹从被丢弃的经验转化为结构化的恢复训练数据，通过直接的干预测量。

View on arXiv Download PDF AI Translation

cs.RO / 54 / 2609.31057

Accuracy Evaluation of INS/ZUPT Filtering Methods Based on Different Geometric Error Definitions

基于不同几何误差定义的INS/ZUPT滤波方法准确性评估

Ouyang, Wei, Han, Jiale, Luo, Yarong, Zhu, Maoran

Abstract

Geometric filters have recently been introduced to improve the accuracy and consistency of inertial-based integrated navigation systems. Error states were defined through specific group operations, introducing state correlations in error definition, which were lacked in the additive error used by a conventional indirect Kalman filter. The desirable consistent filtering models can be obtained based on specific geometric errors. For zero-velocity measurements expressed in the reference frame, this paper derives left-error process and measurement models from invariant filtering, two-frame-group filtering, and equivariant filtering. Importantly, a new group operation is introduced for the left tangent-group equivariant error. The analysis shows that the two-frame-group invariant extended Kalman filter (TFG-IEKF) and the tangent-group equivariant filter (TG-EqF) do not offer a significant consistency advantage over the invariant extended Kalman filter (IEKF). Experiments with an INS/ZUPT measurement system show that, under small initial attitude errors, the conventional indirect extended Kalman filter (EKF) achieves loop-closure position errors below $0.1\%$ of the traveled distance, while the three geometric filters achieve comparable positioning accuracy.

Chinese Translation

最近引入了几何滤波器，以提高基于惯性的综合导航系统的准确性和一致性。通过特定的群操作定义误差状态，引入了误差定义中的状态相关性，而传统间接卡尔曼滤波器所使用的加性误差则缺乏这种相关性。基于特定的几何误差，可以获得理想的一致性滤波模型。对于在参考框架中表示的零速度测量，本文从不变滤波、双框架群滤波和等变滤波中推导出左误差过程和测量模型。重要的是，为左切空间等变误差引入了一种新的群操作。分析表明，双框架群不变扩展卡尔曼滤波器（TFG-IEKF）和切空间等变滤波器（TG-EqF）在一致性上并未显著优于不变扩展卡尔曼滤波器（IEKF）。与INS/ZUPT测量系统的实验表明，在小初始姿态误差下，传统间接扩展卡尔曼滤波器（EKF）实现的闭环位置误差低于行驶距离的$0.1 ext{%}$，而三种几何滤波器则实现了可比的定位精度。

View on arXiv Download PDF AI Translation

cs.RO / 55 / 2609.31110

AuthGuard-R: Safety-Compliant Mission Hijacking and Dual-Gate Defense for LLM-Controlled Robots

AuthGuard-R：安全合规的任务劫持与双重防御机制用于大语言模型控制的机器人

Chepuri, Saidattu, Srivastava, Vikas

Abstract

Large language models are increasingly used as high-level planners for mobile robots, robot manipulators, and autonomous vehicles. Recent studies show that these systems can be influenced through malicious text, speech, visual instructions, retrieved documents, and poisoned sensory context. Most defenses ask whether a proposed action is physically safe. This paper studies a different problem: an action may be physically safe and still violate the mission authorized by the user. An attacker may redirect a delivery robot, replace an approved object, extend a robot's operating region, activate an unnecessary sensor, or delay a mission without creating an immediate physical hazard. We call this attack \emph{safety-compliant mission hijacking}. We propose MissionPAIR, an adaptive attack framework that searches for executable plans that pass a safety gate while violating an authenticated mission. We also propose AuthGuard-R, a deterministic authorization layer that binds every executable action to a signed mission, robot identity, object and region scope, current state, time, and input provenance. AuthGuard-R operates with an independent safety gate, giving a dual-gate architecture. We formalize mission policies over robot traces, define security games, and prove authorization soundness, mission non-escalation, replay resistance, robot binding, provenance separation, threshold-approval security, audit-log tamper evidence, and trace-level composition. We report a preliminary cross-model evaluation with Claude Haiku~4.5 and the open-source Qwen2.5~7B planner. Across 240 live attack trials, the planners followed an injected mission deviation in 109 trials; AuthGuard-R rejected all 109 resulting unauthorized actions. A separate hand-constructed suite of eleven protocol- and policy-level attacks was also blocked completely.

Chinese Translation

大型语言模型越来越多地被用作移动机器人、机器人操纵器和自主车辆的高级规划器。最近的研究表明，这些系统可能会受到恶意文本、语音、视觉指令、检索文档和被污染的感知上下文的影响。大多数防御措施关注于所提议的行动是否在物理上是安全的。本文研究了一个不同的问题：一个行动可能在物理上是安全的，但仍然违反用户授权的任务。攻击者可能会重定向送货机器人、替换已批准的物体、扩展机器人的操作区域、激活不必要的传感器，或在不造成直接物理危害的情况下延迟任务。我们称这种攻击为“安全合规的任务劫持”（safety-compliant mission hijacking）。我们提出了MissionPAIR，一个自适应攻击框架，旨在寻找能够通过安全门而又违反经过认证的任务的可执行计划。我们还提出了AuthGuard-R，一个确定性的授权层，将每个可执行的行动绑定到一个签名的任务、机器人身份、对象和区域范围、当前状态、时间和输入来源。AuthGuard-R与独立的安全门一起工作，形成双重防御架构。我们对机器人轨迹上的任务政策进行了形式化定义，定义了安全游戏，并证明了授权的有效性、任务的不升级性、重放抵抗、机器人绑定、来源分离、阈值批准安全性、审计日志篡改证据和轨迹级组合。我们报告了与Claude Haiku~4.5和开源Qwen2.5~7B规划器的初步跨模型评估。在240次实时攻击试验中，规划器在109次试验中遵循了注入的任务偏差；AuthGuard-R拒绝了所有109个由此产生的未经授权的行动。一个单独手工构建的包含十一种协议和政策级攻击的套件也被完全阻止。

View on arXiv Download PDF AI Translation

cs.RO / 56 / 2609.31112

DualManip: Agentic Dynamic Manipulation via Dual-Path Semantic Reasoning and Geometric Adaptation

DualManip：通过双路径语义推理和几何适应实现的自主动态操控

Li, Chengxi, Di, Yan, Li, Yingyue, Zhang, Ruida, Li, Mingyang, Ji, Xiangyang

Abstract

Vision-language models (VLMs) enable open-vocabulary reasoning for robot manipulation, but their high inference latency limits responsiveness in dynamic scenes. Many scene changes, however, alter object geometry without invalidating task intent. We present DualManip, a dual-path framework that decouples infrequent semantic reasoning from responsive geometric adaptation. The semantic path decomposes the task and grounds task-relevant interactions, followed by a constraint-solving module for pose optimization. During execution, the geometric path continuously updates template-to-observation correspondences from live RGB-D observations via a shape-adaptive network. These correspondences transfer task-relevant grasp contacts across observations, enabling online grasp reconstruction under object motion and non-rigid deformation. The Information Interaction Module bridges the two paths by initializing task-relevant grasps from semantic grounding, validating geometric updates, and triggering semantic replanning upon update failures. Real-world evaluation spans six manipulation tasks covering non-rigid deformation, articulated reconfiguration, rigid motion, and high-precision assembly across three settings: static, single-change, and continuous dynamic. DualManip demonstrates superior manipulation robustness, particularly under continuous scene changes, while achieving geometric adaptation approximately 46$\times$ faster than agentic verification and semantic replanning. Our project page: https://lichengxi1.github.io/Dualmanip.

Chinese Translation

视觉-语言模型（VLMs）使得机器人操控能够进行开放词汇推理，但其高推理延迟限制了在动态场景中的响应能力。然而，许多场景变化会改变物体几何形状，而不会使任务意图失效。我们提出了DualManip，一个双路径框架，将不频繁的语义推理与响应式几何适应解耦。语义路径分解任务并确定与任务相关的交互，随后通过约束求解模块进行姿态优化。在执行过程中，几何路径通过形状自适应网络从实时RGB-D观测中持续更新模板与观测之间的对应关系。这些对应关系在观测之间传递与任务相关的抓取接触点，使得在物体运动和非刚性变形下能够在线重建抓取。信息交互模块通过从语义基础初始化与任务相关的抓取、验证几何更新以及在更新失败时触发语义重新规划来连接这两条路径。实际评估涵盖六个操控任务，涉及非刚性变形、关节重构、刚性运动和高精度装配，分为三种设置：静态、单次变化和连续动态。DualManip展示了卓越的操控鲁棒性，特别是在连续场景变化下，同时实现几何适应的速度约为自主验证和语义重新规划的46倍。我们的项目页面：https://lichengxi1.github.io/Dualmanip。

View on arXiv Download PDF AI Translation

cs.RO / 57 / 2609.31117

Comparative Evaluation of an XR Pen-based Control Interface for Semi-Autonomous Mobile Robot Navigation in Service Environments

基于XR笔的控制界面在服务环境中对半自主移动机器人导航的比较评估

Torc, Alicia, Tornberg, Carl, Piette, Eric, Ronsse, Renaud, Macq, Benoit, Ricardez, Gustavo Alfonso Garcia, Hafi, Lotfi El, Taniguchi, Tadahiro

Abstract

Service robots remain difficult to deploy in domestic environments, partly because fully autonomous operation is not yet reliable in unpredictable surroundings, and partly because conventional control methods remain inaccessible to novice users. Extended Reality (XR) enables operators to visualize robot information overlaid onto the real world and to interact with augmented elements. Yet, common XR control methods, such as motion controllers and hand gestures, are still perceived as unintuitive. This paper presents a control interface that uses a commercial XR pen to command a semi-autonomous mobile robot in Augmented Reality (AR): the operator points at a position in the room, selects it, and drags an augmented arrow to set the desired orientation of the robot at this destination. Two additional interfaces, based on the XR motion controllers and hand gestures, were developed within the same framework. To assess the performance and users' perception of these interfaces, and of the XR pen in particular, a study with 10 participants compared four control methods, i.e., the XR pen, the XR motion controllers, hand gestures, and a computer-based baseline RViz, in navigation tasks performed in a home-like environment. Results show that the XR pen significantly outperforms the other methods in task selection time with the most consistent selections, and that the XR motion controllers obtain the best perceived workload and usability scores, ahead of the computer-based baseline, supporting XR-based control as an intuitive alternative for novice users. However, technical limitations in the integration of the recently released XR pen currently hold back its user experience.

Chinese Translation

服务机器人在家庭环境中的部署仍然面临挑战，部分原因是完全自主操作在不可预测的环境中尚不可靠，部分原因是传统控制方法对新手用户仍然难以使用。扩展现实（XR）使操作员能够将机器人信息叠加在真实世界中进行可视化，并与增强元素进行交互。然而，常见的XR控制方法，如运动控制器和手势，仍然被认为不够直观。本文提出了一种控制界面，使用商业XR笔在增强现实（AR）中指挥半自主移动机器人：操作员在房间中指向一个位置，选择该位置，并拖动一个增强箭头以设置机器人在该目的地的期望方向。在同一框架内，还开发了基于XR运动控制器和手势的另外两个接口。为了评估这些接口的性能和用户对它们的感知，特别是对XR笔的感知，进行了一个包含10名参与者的研究，比较了四种控制方法，即XR笔、XR运动控制器、手势和基于计算机的基线RViz，在家庭环境中执行导航任务的表现。结果表明，XR笔在任务选择时间上显著优于其他方法，并且选择的一致性最高，而XR运动控制器在感知工作负载和可用性评分方面表现最佳，超越了基于计算机的基线，支持XR控制作为新手用户的直观替代方案。然而，最近发布的XR笔在集成中的技术限制目前制约了其用户体验。

View on arXiv Download PDF AI Translation

cs.RO / 58 / 2609.31137

INTERACT: Interactive Planning for Autonomous Driving via Anchor-Conditioned Prediction and Trust-Region Refinement

INTERACT：通过锚定条件预测和信任区域优化实现自主驾驶的交互式规划

Distelzweig, Aron, Look, Andreas, Janjoš, Faris, Hagedorn, Steffen, Palmieri, Luigi, Boedecker, Joschka

Abstract

Driving in dense urban traffic is interactive: whether a merge or an unprotected turn succeeds depends on how surrounding agents respond to the ego vehicle. Conventional planners predict first and plan second and, therefore, cannot account for this dependency. Methods that integrate prediction and planning either train both jointly, which introduces task interference, or keep them separate and are restricted to a predefined set of proposals. We present INTERACT: Interactive Planning for Autonomous Driving via Anchor-Conditioned Prediction and Trust-Region Refinement. Our key insight is that surrounding agents react to the intent a trajectory expresses rather than to its exact realization, so a single reactive prediction stays valid across an entire family of plans. INTERACT therefore decomposes interactive planning into prediction across driving intents and optimization within each intent. We derive a small set of diverse intents, which we call anchors, from map geometry, query a dedicated ego-conditioned prediction model once per anchor, and refine every anchor with the Cross-Entropy Method under a trust-region penalty that keeps the refined plan close enough to its anchor for the conditioned reaction to still apply. Prediction thus remains a separate model, avoiding task interference, while conditioning on anchors preserves the dependency. Because each anchor is refined continuously, the final plan is not restricted to the anchor set, yet INTERACT requires only one predictor query per anchor rather than one per candidate plan, with all anchors processed in parallel. On the nuPlan and interPlan closed-loop benchmarks, INTERACT sets a new state of the art, with the largest gains precisely in the interactive scenarios that motivate the method. The code will be released upon acceptance.

Chinese Translation

在密集的城市交通中驾驶是一个交互过程：合并或无保护转弯的成功与否取决于周围代理对自我车辆的反应。传统的规划者首先进行预测，然后进行规划，因此无法考虑这种依赖关系。将预测与规划相结合的方法要么共同训练两者，这会引入任务干扰，要么将两者分开，限制于预定义的一组提案。我们提出了INTERACT：通过锚定条件预测和信任区域优化实现自主驾驶的交互式规划。我们的关键见解是，周围代理对轨迹所表达的意图作出反应，而不是对其确切实现作出反应，因此单一的反应预测在整个计划家族中保持有效。因此，INTERACT将交互式规划分解为对驾驶意图的预测和在每个意图内的优化。我们从地图几何中导出一小组多样的意图，称之为锚点，针对每个锚点查询一次专用的自我条件预测模型，并在信任区域惩罚下使用交叉熵方法优化每个锚点，以确保优化后的计划与其锚点保持足够接近，以便条件反应仍然适用。因此，预测仍然是一个独立的模型，避免了任务干扰，而对锚点的条件化则保留了依赖关系。由于每个锚点都在持续优化，最终计划不受锚点集的限制，但INTERACT每个锚点只需一次预测查询，而不是每个候选计划一次，所有锚点可以并行处理。在nuPlan和interPlan闭环基准测试中，INTERACT设定了新的最先进水平，尤其在激励该方法的交互场景中取得了最大的进展。代码将在接受后发布。

View on arXiv Download PDF AI Translation

cs.RO / 59 / 2609.31138

Evaluating the Impact of Adaptive Extended Reality on Human-Robot Interaction Across the Reality-Virtuality Continuum

评估自适应扩展现实对现实-虚拟性连续体中人机交互的影响

Tornberg, Carl, Torck, Alicia, Hafi, Lotfi El, Taniguchi, Tadahiro

Abstract

As populations in developed countries age and labor shortages intensify, Cybernetic Avatars (CAs) are proposed to extend human capabilities through robotic embodiments, requiring effective Human-Robot Interaction (HRI) frameworks. Extended Reality (XR), an umbrella term for Augmented Reality (AR), Augmented Virtuality (AV), and Virtual Reality (VR), offers such interfaces, but prior research typically fixes the XR modality without evaluating its effect on task outcomes. This study examines whether the XR modality impacts HRI performance and whether an adaptive interface adjusting the level of virtuality along the Reality-Virtuality Continuum (RVC) at runtime improves it. A custom XR application interfaced with a mobile manipulator supports immersive control and runtime modality switching. In a within-participant multi-room pick-and-place experiment comparing fixed AR, AV, and VR with dynamic RVC through task metrics, the NASA-TLX, and the System Usability Scale (SUS), this study demonstrates that 1) the fixed reality modality affects HRI results, and 2) dynamically changing the modality along the RVC improves them. AR yielded significantly lower mental demand, effort, and frustration than AV and VR, while the dynamic RVC condition achieved the highest throughput and lowest workload, highlighting the value of adaptive XR interfaces for human-robot symbiosis. The implementation is available at https://github.com/CarlTornberg/XR-HRI.

Chinese Translation

随着发达国家人口老龄化和劳动力短缺问题的加剧，提出了通过机器人化身扩展人类能力的网络化化身（Cybernetic Avatars, CAs），这需要有效的人机交互（Human-Robot Interaction, HRI）框架。扩展现实（Extended Reality, XR）作为增强现实（Augmented Reality, AR）、增强虚拟性（Augmented Virtuality, AV）和虚拟现实（Virtual Reality, VR）的统称，提供了这样的接口，但以往的研究通常固定XR模式，而未评估其对任务结果的影响。本研究考察了XR模式是否影响HRI表现，以及在运行时沿现实-虚拟性连续体（Reality-Virtuality Continuum, RVC）调整虚拟性水平的自适应接口是否能改善这一表现。一个定制的XR应用程序与移动操控器相结合，支持沉浸式控制和运行时模式切换。在一个参与者内部的多房间取放实验中，通过任务指标、NASA-TLX和系统可用性量表（System Usability Scale, SUS）比较了固定的AR、AV和VR与动态RVC，研究表明：1）固定的现实模式影响HRI结果，2）沿RVC动态变化模式可以改善HRI表现。AR的心理需求、努力和挫败感显著低于AV和VR，而动态RVC条件下实现了最高的吞吐量和最低的工作负载，突显了自适应XR接口在促进人机共生中的价值。该实现可在 https://github.com/CarlTornberg/XR-HRI 获取。

View on arXiv Download PDF AI Translation

cs.RO / 60 / 2609.31185

Onboard Wind-Preview Model Predictive Control Using Pitot-Static Sensing for Multirotor UAVs

基于皮托-静压传感器的多旋翼无人机机载风预览模型预测控制

Meere, Bas, Wisse, Eline, Vousten, Laurens, Doodeman, Sander, Torta, Elena, Chanfreut, Paula, Antunes, Duarte

Abstract

Effective wind gust rejection and stable hovering are critical for the outdoor operation of autonomous drones. However, existing gust rejection methods are primarily reactive, inferring the disturbance from the resulting motion or measuring it at the airframe. Either way, the wind has already begun to act before it can be compensated. In this work, we anticipate the gust instead by measuring the wind ahead of the drone with a low-cost, low-weight pitot-static sensor mounted on a boom. The resulting wind preview is incorporated into a nonlinear model predictive controller (MPC), which optimizes the drone motion while anticipating wind disturbances. A longer boom offers more preview time but adds inertia and degrades flight performance. We characterize this trade-off in simulation and show that the optimal preview distance is not a fixed property of the platform, but shifts with the wind speed and with how quickly the drone can respond. Indoor hardware experiments confirm the trend and show that the proposed controller substantially improves hover performance against a PX4 baseline and an otherwise identical wind-unaware MPC. Outdoor experiments show that the error along the wind direction is reduced by 54 percent with respect to the baseline, demonstrating that a single wind-aligned sensor can significantly improve hovering performance.

Chinese Translation

有效的风突拒绝和稳定悬停对于自主无人机的户外操作至关重要。然而，现有的风突拒绝方法主要是反应性的，通过推断运动结果或在机身上测量来识别干扰。无论哪种方式，风在补偿之前已经开始作用。在本研究中，我们通过在无人机前方安装一个低成本、低重量的皮托-静压传感器来提前预测风突。所获得的风预览被纳入一个非线性模型预测控制器（MPC），该控制器在预测风干扰的同时优化无人机的运动。更长的探杆提供了更多的预览时间，但增加了惯性并降低了飞行性能。我们在仿真中表征了这种权衡，并表明最佳预览距离不是平台的固定属性，而是随着风速和无人机响应速度的变化而变化。室内硬件实验确认了这一趋势，并显示所提出的控制器在悬停性能上显著优于PX4基线和其他相同的风无关MPC。户外实验表明，与基线相比，沿风向的误差减少了54%，证明单个风对齐传感器可以显著改善悬停性能。

View on arXiv Download PDF AI Translation

cs.RO / 61 / 2609.31207

Enabling a Unified Cross-Domain Representation for Two-Finger Gripper Manipulation via Interaction-Centric Modeling

通过以交互为中心的建模实现统一的跨领域表示以进行双指夹持器操作

Li, Guanlin, Bao, Shifeng, Zhao, Yihan, Shen, Haitao, Li, Haoyang, Zhao, Chen, Yang, Tong, Tang, Jie, Zhang, Jing

Abstract

Achieving robust cross-embodiment generalization in imitation learning demands overcoming a critical representation flaw that inextricably entangles task semantics with hardware-specific visual geometry. We propose an interaction-centric framework that leverages the shared structure of two-finger grippers via a parameterized universal gripper abstraction, yielding a canonical gripper-frame representation. Given language and RGB-D observations, a VLM infers the subtask and grounds an interaction triplet (gripper, held, target), while SAM~2.1 tracks masks to reduce VLM queries. We design concise hybrid features that combine target/collision artificial potential fields for global guidance with segmented gripper-frame point clouds for local geometry, and use a Flow-Matching Transformer to predict smooth 7-DoF action chunks. Experiments in simulation and real-world tasks demonstrate that ours is the first imitation learning approach to simultaneously achieve competitive benchmark scores and extreme cross-embodiment/cross-viewpoint zero-shot sim-to-real transfer to completely distinct, heterogeneous robot platforms.

Chinese Translation

在模仿学习中实现稳健的跨体现泛化需要克服一个关键的表示缺陷，该缺陷不可避免地将任务语义与特定硬件的视觉几何纠缠在一起。我们提出了一种以交互为中心的框架，通过参数化的通用夹持器抽象，利用双指夹持器的共享结构，从而产生标准的夹持器框架表示。在给定语言和RGB-D观测的情况下，VLM推断子任务并确定一个交互三元组（夹持器、持有物、目标），同时SAM~2.1跟踪掩膜以减少VLM查询。我们设计了简洁的混合特征，将目标/碰撞人工势场与分段的夹持器框架点云结合，以提供全局指导，并使用流匹配变换器预测平滑的7自由度动作块。模拟和现实任务中的实验表明，我们的方法是首个在竞争基准得分和极端跨体现/跨视角零-shot模拟到真实转移方面同时取得成功的模仿学习方法，适用于完全不同的异构机器人平台。

View on arXiv Download PDF AI Translation

cs.RO / 62 / 2609.31211

CoralPlan: Observation Skill Selection and Execution for Underwater Robotic Inspection

CoralPlan：水下机器人检查的观察技能选择与执行

Gao, Yuer, Zhao, Yu, Cai, Yi

Abstract

Underwater robotic inspection depends on acquiring views that reveal task-relevant structure. For a structurally complex coral colony, recognising the target is only the starting point: the robot must select and execute a viewing motion suited to the inspection task. We present CoralPlan, a vision-language system that selects an observation skill from a current camera image and task text supplied by an episode manifest. A shared motion interface executes orbit, patch, or survey as target-relative trajectories; the remaining plan fields provide operator guidance. Observation completion requires target keeping and primitive-specific coverage, while joint success also requires selection to match the recorded reference. We evaluate this interface in 144 simulated episodes and 36 matched simulation-hardware pairs. In a clear-water pool with external target-reference poses, hardware observation completion reaches 77.8% and joint success reaches 63.9%. The experiments identify both reference-mismatched completions and incomplete observations after a matching skill selection. These results connect observation-skill choice to measurable underwater execution outcomes and identify where task-directed acquisition succeeds or fails.

Chinese Translation

水下机器人检查依赖于获取能够揭示任务相关结构的视图。对于结构复杂的珊瑚群落，识别目标仅仅是起点：机器人必须选择并执行适合检查任务的观察动作。我们提出了 CoralPlan，一个视觉-语言系统，该系统从当前相机图像和由事件清单提供的任务文本中选择观察技能。共享运动接口执行相对于目标的轨迹，包括轨道、补丁或调查；其余计划字段提供操作员指导。观察的完成需要保持目标和特定原始覆盖，而联合成功还需要选择与记录的参考相匹配。我们在144个模拟事件和36对匹配的模拟-硬件对中评估了该接口。在一个具有外部目标参考姿态的清水池中，硬件观察完成率达到77.8%，联合成功率达到63.9%。实验识别出参考不匹配的完成和在匹配技能选择后未完成的观察。这些结果将观察技能选择与可测量的水下执行结果联系起来，并识别出任务导向获取成功或失败的地方。

View on arXiv Download PDF AI Translation

cs.RO / 63 / 2609.31225

Imp-ACT: Adaptive Impedance Control and Action Chunking with Transformers to Learn Contact-Rich Manipulation from Demonstrations

Imp-ACT：利用变换器进行自适应阻抗控制和动作分块，以从示范中学习接触丰富的操作

Zanetti, Luca, Sirintuna, Doganay, Ozdamar, Idil, Balatti, Pietro, Zhang, Heng, Ajoudani, Arash

Abstract

Contact-rich manipulation requires robots to balance accurate motion tracking with compliant interaction, yet most visual-action policies leave compliance fixed at the controller level. We present Imp-ACT, a methodologically grounded and practical approach to incorporating direction-dependent Cartesian stiffness modulation directly into demonstration collection, without manual stiffness selection or offline target reconstruction. During teleoperation, a self-tuning impedance controller adapts stiffness along the instantaneous direction of motion while maintaining compliance in orthogonal directions. The adapted stiffness is applied and recorded alongside visual observations and motion commands, capturing motion and compliance under the same dynamics. We implement this pipeline using Action Chunking with Transformer (ACT) to predict end-effector pose, gripper action, and motion-direction stiffness from visual, proprioceptive, and wrench observations. The performance of Imp-ACT is evaluated on wiping and plug insertion using both success rate and quantitative measures of contact behavior. Compared with fixed low- and high-stiffness baselines, Imp-ACT achieves comparable or higher success while maintaining low interaction forces. In wiping, it reduces contact-force vibration by approximately $29\times$ relative to the compliant baseline and $180\times$ relative to the stiff baseline. In plug insertion, it reduces forces orthogonal to the insertion direction by $43\%$ relative to the better fixed-stiffness baseline. These results highlight the benefit of maintaining sufficient stiffness along the direction needed for task execution while preserving compliance in other directions to limit contact forces and accommodate environmental constraints.

Chinese Translation

接触丰富的操作要求机器人在精确运动跟踪与顺应性互动之间取得平衡，然而大多数视觉-动作策略在控制器层面上将顺应性固定。我们提出了Imp-ACT，这是一种方法论基础扎实且实用的方式，将方向依赖的笛卡尔刚度调制直接融入示范收集过程中，无需手动选择刚度或离线目标重建。在遥操作过程中，自调节的阻抗控制器沿瞬时运动方向调整刚度，同时在正交方向上保持顺应性。调整后的刚度与视觉观察和运动指令一同应用和记录，捕捉在相同动态下的运动和顺应性。我们使用带有变换器的动作分块（Action Chunking with Transformer, ACT）实现该流程，以从视觉、身体感知和扭矩观察中预测末端执行器姿态、夹持器动作和运动方向刚度。通过成功率和接触行为的定量指标评估Imp-ACT在擦拭和插头插入任务中的表现。与固定的低刚度和高刚度基线相比，Imp-ACT在保持低交互力的同时实现了可比或更高的成功率。在擦拭任务中，相较于顺应基线，其接触力振动减少了约29倍，相较于刚性基线减少了180倍。在插头插入任务中，其在插入方向上的正交力减少了43%，相较于更好的固定刚度基线。这些结果突显了在任务执行所需方向保持足够刚度的好处，同时在其他方向上保持顺应性，以限制接触力并适应环境约束。

View on arXiv Download PDF AI Translation

cs.RO / 64 / 2609.31232

Cybflight: An Embedded Rust Autopilot for Aerial Robotics Research

Cybflight：一种用于空中机器人研究的嵌入式 Rust 自动驾驶仪

Lin, Yifan, Qin, Chao, Go, H S Helson, Liu, Hugh H. -T.

Abstract

Bringing an aerial robotics method from simulation to flight should not require rebuilding a mature autopilot or adding a companion computer. Cybflight is an open-source embedded Rust research autopilot whose typed, replaceable interfaces connect hardware access, perception, state estimation, trajectory planning, and control. This modular development and compile-time optimization workflow is demonstrated with replaceable Rust implementations of model predictive contour-tracking control (MPCTC) and incremental nonlinear dynamic inversion (INDI) running on one STM32H743 without a companion computer. Using this configuration, the vehicle reaches 12.38 m/s during indoor flight, while an outdoor flight using global navigation satellite system (GNSS) position updates reaches 31.4 m/s. These flights show that running demanding estimation and nonlinear control entirely on a flight-controller microcontroller need not come at the expense of a modular autopilot structure.

Chinese Translation

将空中机器人方法从仿真转化为飞行不应需要重建成熟的自动驾驶仪或添加伴随计算机。Cybflight 是一个开源的嵌入式 Rust 研究自动驾驶仪，其类型化、可替换的接口连接了硬件访问、感知、状态估计、轨迹规划和控制。这种模块化开发和编译时优化工作流程通过在一台 STM32H743 上运行的可替换 Rust 实现的模型预测轮廓跟踪控制（Model Predictive Contour-Tracking Control, MPCTC）和增量非线性动态反演（Incremental Nonlinear Dynamic Inversion, INDI）得以展示，而无需伴随计算机。使用这种配置，车辆在室内飞行时达到 12.38 m/s，而在使用全球导航卫星系统（Global Navigation Satellite System, GNSS）位置更新的室外飞行中达到 31.4 m/s。这些飞行表明，完全在飞行控制微控制器上运行高要求的估计和非线性控制并不需要以模块化自动驾驶仪结构为代价。

View on arXiv Download PDF AI Translation

cs.RO / 65 / 2609.31313

Towards VLA-Dreamer: Refining VLA Behavior Using World Models

迈向 VLA-Dreamer：利用世界模型优化 VLA 行为

Kashani, Parsa Mastouri, Habekost, Jan-Gerrit, Wermter, Stefan

Abstract

Vision-Language-Action models (VLAs), while showing strong potential for robot control, require massive amounts of high-quality imitation learning data. Moreover, the absence of an explicit world model casts further doubt on their control capabilities. In this concept paper, we propose a novel architecture that addresses sample efficiency in VLAs by training a predictive world model on the embedding space of the VLA's vision encoder. We hypothesize that these embeddings are action-relevant and usable for future prediction. To this end, we propose using the suggested architecture to investigate how well these embeddings predict the future based on actions, as the inability to do so would mark a key limitation of VLA architectures: the lack of a non-lossy implicit world model to simulate real-world dynamics. The proposed architecture differs from the standard world model dynamics as the loss comes from the embedding space rather than the pixel space, similar to joint embedding predictive architectures. Furthermore, the trained world model can be utilized for short-term planning tasks by sampling VLA actions given goal images. We intend to examine the richness of vision embeddings in VLAs and reduce their high data requirements through a world model that can also generate plans during inference.

Chinese Translation

视觉-语言-动作模型（VLA）在机器人控制方面展现出强大的潜力，但需要大量高质量的模仿学习数据。此外，缺乏明确的世界模型进一步质疑了它们的控制能力。在本文中，我们提出了一种新颖的架构，通过在 VLA 的视觉编码器的嵌入空间上训练预测世界模型，来解决 VLA 的样本效率问题。我们假设这些嵌入与动作相关，并可用于未来的预测。为此，我们建议使用所提出的架构来研究这些嵌入在基于动作的未来预测中的有效性，因为无法做到这一点将标志着 VLA 架构的一个关键限制：缺乏无损的隐式世界模型来模拟现实世界的动态。所提出的架构与标准世界模型动态不同，因为损失来自嵌入空间而非像素空间，类似于联合嵌入预测架构。此外，训练好的世界模型可以通过给定目标图像来采样 VLA 动作，从而用于短期规划任务。我们旨在考察 VLA 中视觉嵌入的丰富性，并通过一个能够在推理过程中生成计划的世界模型来减少其对大量数据的需求。

View on arXiv Download PDF AI Translation

cs.RO / 66 / 2609.31323

See to Reach, Feel to Grasp: Learning A Blind Grasp Reflex for Anthropomorphic Robotic Hands

看以到达，触以抓取：为类人机器人手学习盲抓反射

Alexiev, Alexander, Lin, Tzu-Yuan, Kim, Sang Min, Lee, Ho Jae, Lee, Yonghyeon, Kim, Sangbae

Abstract

In this work we study if a robotic hand using proprioception alone can grasp diverse objects with no visual observation. We present a modular dexterous grasping architecture that separates global arm motion from local contact control. An independently controlled arm guides the hand toward the object, while a reinforcement learning policy grasps and stabilizes it using only hand proprioceptive feedback. We call this \textit{a blind grasp reflex}: grasping without images, object poses, or geometric observations. A learned stable-grasp score determines when the object is securely held, allowing the arm to begin post-grasp manipulation. This separation makes grasping a reusable hand-level skill that can be combined with independently designed arm controllers for various manipulation tasks. Experiments in simulation and on hardware demonstrate robust blind grasping across diverse objects and seamless composition with a range of arm controllers. Moreover, despite never observing contact geometry, the learned grasp score closely aligns with an independent physics-based measure of grasp stability. The resulting approach follows a simple principle: see to reach, feel to grasp. Project page: https://blindgraspreflex.github.io.

Chinese Translation

在本研究中，我们探讨了仅使用本体感觉的机器人手是否能够在没有视觉观察的情况下抓取多样化的物体。我们提出了一种模块化的灵巧抓取架构，该架构将全局臂部运动与局部接触控制分开。一个独立控制的手臂引导手部朝向物体，而强化学习策略则仅使用手部本体感觉反馈来抓取和稳定物体。我们称之为 extit{盲抓反射}：在没有图像、物体姿态或几何观察的情况下进行抓取。一个学习到的稳定抓取评分决定了物体何时被安全抓持，从而允许手臂开始抓取后的操作。这种分离使得抓取成为一种可重用的手部技能，可以与独立设计的臂部控制器结合用于各种操作任务。模拟和硬件实验展示了在多样物体上的稳健盲抓取，以及与多种臂部控制器的无缝组合。此外，尽管从未观察到接触几何，学习到的抓取评分与独立的基于物理的抓取稳定性测量紧密对齐。最终的方法遵循一个简单的原则：看以到达，触以抓取。项目页面：https://blindgraspreflex.github.io。

View on arXiv Download PDF AI Translation

cs.RO / 67 / 2609.31337

Representation-Guided Generation and Integration of Executable Programs for Robot Manipulation

基于表示的可执行程序生成与集成用于机器人操作

Yang, Ruixiao, Yu, Mingxin, Fan, Chuchu

Abstract

Building a robotic manipulation system requires connecting perception, planning, and control through carefully designed representations and interfaces. VLM code generation offers a way to automate this construction, but independently generated components may operate on incompatible geometric and task-level information. We present Representation-guided Integration of VLM-generated Executable Task programs (RIVET), a framework for generating complete manipulation systems around a shared object-centric representation. The representation combines per-object 6D poses, which preserve the metric information required for action grounding, with a relation graph that exposes the task-level structure required for planning. Guided by this representation, a VLM generates cooperating perception, rendering, relation-inference, and planning programs, each combining task-specific computation with available packages where useful. The resulting programs are authored once for a manipulation domain and reused on unseen start and goal configurations without code regeneration. We evaluate RIVET on cube stacking, tangram rearrangement, and three-dimensional assembly in simulation and on a physical robot, where we achieve 83% overall success rate in the real world by reusing offline-generated systems. Our results demonstrate that representation-guided program generation can adapt a common manipulation framework to tasks with different geometric, relational, and sequential requirements.

Chinese Translation

构建一个机器人操作系统需要通过精心设计的表示和接口连接感知、规划和控制。VLM（视觉语言模型）代码生成提供了一种自动化构建的方式，但独立生成的组件可能在几何和任务级信息上不兼容。我们提出了基于表示的VLM生成可执行任务程序的集成框架（RIVET），该框架围绕共享的对象中心表示生成完整的操作系统。该表示结合了每个对象的六维姿态，保留了执行动作所需的度量信息，并与一个关系图相结合，揭示了规划所需的任务级结构。在该表示的指导下，VLM生成协作的感知、渲染、关系推理和规划程序，每个程序将特定任务的计算与可用的包结合起来。生成的程序在操作领域中只需编写一次，并可在未见过的起始和目标配置上重复使用，而无需重新生成代码。我们在模拟和物理机器人上对RIVET进行了评估，涉及立方体堆叠、七巧板重排和三维组装，在现实世界中通过重用离线生成的系统实现了83%的整体成功率。我们的结果表明，基于表示的程序生成能够将通用操作框架适应于具有不同几何、关系和顺序要求的任务。

View on arXiv Download PDF AI Translation

cs.RO / 68 / 2609.31357

Transformer-based Monte Carlo Localization in Construction Meshes

基于变换器的建筑网格中的蒙特卡洛定位

Kramer, Linus, Talbot, William, Vysotska, Olga, Hutter, Marco

Abstract

To be able to perform inspection or digitization tasks, mobile robots on construction sites must be able to localize themselves reliably with respect to a global reference frame that is shared with a building map. Similar room layouts and low-texture surfaces pose a challenge for existing LiDAR- and vision-based localization methods. We approach this problem with a LiDAR-based global relocalization system that estimates the robot's pose relative to a building mesh and combines a PointNet++ encoder with a place recognition decoder, whose outputs serve as a learned observation model within a Monte Carlo Localization (MCL) framework. The pipeline is trained exclusively on synthetic LiDAR scans obtained by simulating the robot's sensors inside the building mesh. Our approach is robust in ambiguous environments due to an uncertainty-aware decoder that scales positional likelihoods and a resampling strategy that injects model hypotheses into the particle set, enabling recovery from potential particle depletion. Evaluations on real-world datasets show that our method outperforms both diffusion-based and ScanContext++ baselines while maintaining fast inference (18 ms per call), demonstrating the practicality of synthetic-data training for mesh-referenced global localization in construction robotics.

Chinese Translation

为了能够执行检查或数字化任务，建筑工地上的移动机器人必须能够相对于与建筑地图共享的全球参考框架可靠地定位自己。类似的房间布局和低纹理表面对现有的基于激光雷达（LiDAR）和视觉的定位方法构成了挑战。我们通过一个基于激光雷达的全球重新定位系统来解决这个问题，该系统估计机器人相对于建筑网格的姿态，并将 PointNet++ 编码器与位置识别解码器结合起来，其输出作为蒙特卡洛定位（MCL）框架内的学习观察模型。该流程完全在通过模拟机器人传感器在建筑网格内获得的合成激光雷达扫描数据上进行训练。由于具有不确定性感知的解码器能够缩放位置可能性，以及一种将模型假设注入粒子集的重采样策略，我们的方法在模糊环境中表现出强大的鲁棒性，从而能够从潜在的粒子耗尽中恢复。对真实世界数据集的评估表明，我们的方法在保持快速推理（每次调用 18 毫秒）的同时，优于基于扩散和 ScanContext++ 的基线，展示了合成数据训练在建筑机器人中进行网格参考全球定位的实用性。

View on arXiv Download PDF AI Translation

cs.RO / 69 / 2609.31374

RECAST: From Log Replay to Closed-Loop Driving Simulation with View-Complete Actors

RECAST：从日志重放到闭环驾驶仿真与视图完整的行为者

Zhao, Zijun, Liao, Liewen, Shen, Kang, Zhang, Songan, Yang, Ming

Abstract

Closed-loop driving simulation requires rendered observations to remain reliable as the ego vehicle and surrounding actors move beyond their recorded trajectories, exposing views absent from the source log. Existing data-driven simulators reconstruct dynamic actors from sparse observations, which can result in rendering artifacts under these viewpoint changes. We introduce RECAST (REconstructing Controllable Actors for Simulation and Testing), a 3D Gaussian Splatting framework that generates a view-complete actor from a single segmented vehicle observation in a driving log and registers the generated actor in the reconstructed scene. RECAST supports planner-in-the-loop rendering under controlled ego-actor interactions. To adapt an image-to-3D prior to real vehicles, we further introduce RECAR, a dataset of approximately 20K real vehicles with 600K background-free RGBA images spanning diverse vehicle colors and types. We use two-stage adaptation to improve vehicle generation from real driving-log observations. At the actor level, RECAST reduces $\mathrm{FD}_{\mathrm{incep}}$ from 9.788 to 7.992 relative to unadapted TRELLIS. At the scene level, under actor motion beyond logged trajectories, RECAST reduces $\mathrm{FD}_{\mathrm{incep}}$ from 129.35 to 112.10 and increases $\mathrm{CLIP}_{\mathrm{margin}}$ ($\times1000$) from 0.14 to 3.47 relative to Street Gaussians. We demonstrate planner-in-the-loop simulation with the image-conditioned planner GTRS-Dense. Compared with native Street Gaussians actors, RECAST increases the no-collision (NC) rate from 22.2% (12/54) to 63.0% (34/54) and the mean minimum predicted time-to-collision (TTC) from 0.798 s to 2.150 s. These experiments show that RECAST supports closed-loop planner evaluation under controlled ego-actor interactions beyond log replay. Visit our project page at https://zijunkr.github.io/RECAST/

Chinese Translation

闭环驾驶仿真要求在自我车辆和周围行为者超出其记录轨迹移动时，渲染的观察结果保持可靠，这暴露了源日志中缺失的视图。现有的数据驱动仿真器从稀疏观察中重建动态行为者，这可能导致在这些视点变化下出现渲染伪影。我们提出了RECAST（REconstructing Controllable Actors for Simulation and Testing），一个3D高斯点云框架，它从驾驶日志中的单个分割车辆观察生成视图完整的行为者，并将生成的行为者注册到重建的场景中。RECAST支持在受控的自我-行为者交互下的规划者环路渲染。为了将图像到3D的先验适应于真实车辆，我们进一步引入了RECAR，一个包含约2万辆真实车辆的数据库，提供60万张无背景的RGBA图像，涵盖多种车辆颜色和类型。我们使用两阶段适应来改善从真实驾驶日志观察中生成车辆的效果。在行为者层面，RECAST将相对于未适应的TRELLIS的$ ext{FD}_{ ext{incep}}$从9.788降低到7.992。在场景层面，在行为者运动超出记录轨迹的情况下，RECAST将$ ext{FD}_{ ext{incep}}$从129.35降低到112.10，并将相对于Street Gaussians的$ ext{CLIP}_{ ext{margin}}$（$ imes1000$）从0.14提高到3.47。我们展示了与图像条件规划者GTRS-Dense的规划者环路仿真。与原生Street Gaussians行为者相比，RECAST将无碰撞（NC）率从22.2%（12/54）提高到63.0%（34/54），并将平均最小预测碰撞时间（TTC）从0.798秒提高到2.150秒。这些实验表明，RECAST支持在超出日志重放的受控自我-行为者交互下进行闭环规划者评估。请访问我们的项目页面 https://zijunkr.github.io/RECAST/

View on arXiv Download PDF AI Translation

cs.RO / 70 / 2609.31383

Guiding End-to-End Driving Models with Endpoint-Constrained Trajectory Optimization

通过端点约束轨迹优化引导端到端驾驶模型

Zhang, Brayden, Golchoubian, Mahsa, Gilitschenski, Igor, Ivanovic, Boris, Chitta, Kashyap

Abstract

End-to-end driving policies are commonly trained through open-loop behavior cloning, yet they must ultimately operate in closed-loop when deployed on a vehicle, creating a fundamental mismatch between training and execution. Beyond the commonly studied effects of covariate shift and causal confusion, we identify a complementary factor for this open-loop/closed-loop gap: waypoint-based supervision and displacement metrics do not ensure that the intermediate trajectory is physically coherent or easy for the controller to track. We observe that these inconsistencies concentrate primarily at intermediate waypoints, while the predicted endpoint remains comparatively reliable. Based on this observation, we introduce Endpoint-Constrained Optimization (ECO), a lightweight postprocessing layer that anchors the trajectory to the vehicle's executed history, preserves the policy's predicted endpoint, and reshapes the intermediate waypoints to improve feasibility. ECO requires no map, privileged simulator state, or additional training, and can be inserted between a broad range of waypoint-emitting policies and their controllers. Across two closed-loop simulators, it improves the aggregate closed-loop score of all six evaluated generative and regression-based driving policies, and the gains tend to increase with how often the base plans violate motion limits. On HUGSIM, ECO improves VaVAM from 18.1 to 31.0 HD-Score (+71%), achieving 1st place on the HUGSIM Closed-Loop Driving Challenge. Similarly, on AlpaSim, ECO increases the scene scores of VaVAM and DiffusionDrive by 123% and 22%, respectively. These results show that for a broad collection of end-to-end driving models, repairing the intermediate geometry of predicted trajectories without changing the policy's predicted endpoint can substantially improve closed-loop performance.

Chinese Translation

端到端驾驶策略通常通过开放循环行为克隆进行训练，但在部署到车辆上时，它们最终必须在闭环中操作，这导致训练与执行之间存在根本的不匹配。除了常见的协变量偏移和因果混淆效应外，我们还识别出一个补充因素，导致这种开放循环/闭环差距：基于航点的监督和位移度量并不能确保中间轨迹在物理上是一致的，或者对控制器来说易于跟踪。我们观察到，这些不一致主要集中在中间航点，而预测的端点相对可靠。基于这一观察，我们引入了端点约束优化（Endpoint-Constrained Optimization, ECO），这是一种轻量级的后处理层，它将轨迹锚定到车辆的执行历史，保留策略的预测端点，并重塑中间航点以提高可行性。ECO不需要地图、特权模拟器状态或额外训练，可以插入到广泛的航点生成策略和它们的控制器之间。在两个闭环模拟器中，它提高了所有六种评估的生成和回归驱动策略的整体闭环得分，且增益往往随着基础计划违反运动限制的频率增加而增加。在HUGSIM上，ECO将VaVAM的HD-Score从18.1提高到31.0（+71%），在HUGSIM闭环驾驶挑战中获得第一名。同样，在AlpaSim上，ECO分别将VaVAM和DiffusionDrive的场景得分提高了123%和22%。这些结果表明，对于广泛的端到端驾驶模型，修复预测轨迹的中间几何形状而不改变策略的预测端点可以显著改善闭环性能。

View on arXiv Download PDF AI Translation

cs.RO / 71 / 2609.31386

Modeling and Generative-AI-Based Design of Load-Adaptive Gravity Balancing Mechanisms

负载自适应重力平衡机制的建模与基于生成式人工智能的设计

Kayawake, Ryotaro, Abe, Kazuki, Miyake, Shota, Watanabe, Masahiro, Tadakuma, Kenjiro

Abstract

Load-adaptive gravity balancing mechanisms (LA-GBMs) can accommodate various loading conditions by passively changing their characteristics in response to payload variations. However, their design is difficult because both the desired mechanism motion and static equilibrium under variable payloads must be satisfied simultaneously. This study proposes a general design methodology for LA-GBMs that does not depend on specific mechanism architectures or mechanical elements. The necessary conditions for the potential fields of LA-GBMs are formulated, and two general forms are derived: an affine form representing the effect of payload mass and a factorized form representing state transitions associated with load adaptation and gravity balancing. These forms are then provided to generative AI as design requirements to generate candidate potential functions. The generated functions are analytically verified in terms of their conformity to the two general forms and the conditions required for valid LA-GBMs. Furthermore, the obtained potential functions are decomposed into individual terms, and an example of a method for constructing an LA-GBM by combining springs, counterweights, and function-generating linkage mechanisms is presented. By using potential functions as an intermediate representation, the proposed framework enables the generation of LA-GBM design candidates without prescribing a mechanism architecture in advance. Mechanical realizability and manufacturability of the generated potential fields remain important issues for future work.

Chinese Translation

负载自适应重力平衡机制（LA-GBMs）能够通过被动地改变其特性来适应各种负载条件，以应对有效载荷的变化。然而，它们的设计非常困难，因为必须同时满足所需的机制运动和在可变有效载荷下的静态平衡。本研究提出了一种通用的LA-GBMs设计方法论，该方法论不依赖于特定的机制架构或机械元件。我们制定了LA-GBMs潜在场的必要条件，并推导出两种通用形式：一种是表示有效载荷质量影响的仿射形式，另一种是表示与负载适应和重力平衡相关的状态转变的分解形式。然后将这些形式作为设计要求提供给生成式人工智能，以生成候选潜在函数。生成的函数在其符合这两种通用形式及有效LA-GBMs所需条件的方面进行了分析验证。此外，获得的潜在函数被分解为各个项，并展示了一种通过组合弹簧、配重和功能生成连杆机制构建LA-GBM的方法示例。通过使用潜在函数作为中间表示，所提出的框架能够在不预先规定机制架构的情况下生成LA-GBM设计候选。生成的潜在场的机械可实现性和可制造性仍然是未来工作的重点问题。

View on arXiv Download PDF AI Translation

cs.RO / 72 / 2609.31394

InternW0-$\Delta$: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data

InternW0-$ ext{Δ}$：一个连接预测动态与动作的世界动作模型，基于20K+小时的开放数据

Miao, Xingyu, Li, Zizun, Fang, Baole, Song, Kaiwen, Wang, Tenghui, Zhang, Hanxue, Wang, Yating, Li, Xudong, He, Yuping, Wei, Xueyuan, Gao, Chao, Yang, Xijie, Xu, Yingxiang, Ren, Kerui, Guo, Wenqi, Zhou, Jianjun, Wang, Xinzhe, Zhao, Weiguang, Yang, Ni, Cai, Zetao, Xue, Yufei, Li, Hengjie, He, Zeyu, Zhou, Yuanzhen, Fu, Rong, Zhang, Jianyang, Cui, Siwei, Huang, Fuxian, Zhou, Yunsong, Gao, Xing, Yao, Yifei, Yu, Qiaojun, Li, Kailin, Zhou, Ming, Huang, Mu, Li, Xinyue, Cui, Wenze, Jiang, Bingqi, Zhu, Xueyue, Dong, Junting, Guo, Haoyu, Lu, Tao, Yu, Mulin, Zhou, Bowen, Zhao, Bin, Xue, Tianfan, Zhang, Weinan, Shen, Chunhua

Abstract

World Action Models (WAMs) jointly model visual dynamics and action generation for generalist robot manipulation. A central challenge is to integrate priors from large-scale pretrained models---including visual dynamics, scene semantics, geometry, and motion---into a unified framework for robot action generation. We introduce InternW0-$\Delta$, a unified WAM pretrained on a heterogeneous corpus that outperforms prior methods across simulation benchmarks and real-robot platforms. InternW0-$\Delta$ combines pretrained visual dynamics, scene-level semantics, 4D geometric and motion priors, and action generation within a Mixture-of-Transformers (MoT) framework. A pretrained video expert and an action expert interact under semantic guidance from a frozen VLM, while a pretrained 4D foundation model injects geometric and motion priors through training-only distillation. We further introduce Causal Imprint, which learns future-relevant scene changes from training-only future supervision and provides predictive representations directly to the action expert without future-video rollout at inference. For large-scale joint training, we construct a heterogeneous corpus of robot demonstrations, UMI data, egocentric human demonstrations, and Ego2Robot data, curated and aligned under a common state-action representation. The resulting corpus contains over 20K hours of processed training data, to our knowledge the largest open-source corpus of its kind. We pretrain InternW0-$\Delta$ on this corpus and demonstrate strong performance across simulation benchmarks and real-robot platforms. We will open source the training code, model weights, infrastructure, data-processing pipeline, and processed data where licenses permit. Project page: https://internrobotics.github.io/InternW0-Delta/

Chinese Translation

世界动作模型（WAMs）联合建模视觉动态和通用机器人操作的动作生成。一个核心挑战是将来自大规模预训练模型的先验知识——包括视觉动态、场景语义、几何和运动——整合到一个统一的框架中，以实现机器人动作生成。我们提出了InternW0-$ ext{Δ}$，这是一个在异构语料库上预训练的统一WAM，在仿真基准和真实机器人平台上均优于先前的方法。InternW0-$ ext{Δ}$结合了预训练的视觉动态、场景级语义、4D几何和运动先验，以及在混合变换器（Mixture-of-Transformers, MoT）框架内的动作生成。一个预训练的视频专家和一个动作专家在冻结的视觉语言模型（VLM）的语义指导下进行交互，而一个预训练的4D基础模型通过仅在训练中进行蒸馏注入几何和运动先验。我们进一步引入了因果印记（Causal Imprint），它从仅在训练中的未来监督中学习与未来相关的场景变化，并在推理时直接为动作专家提供预测表示，而无需未来视频回放。为了进行大规模联合训练，我们构建了一个异构的机器人演示、UMI数据、自我中心人类演示和Ego2Robot数据的语料库，这些数据在一个共同的状态-动作表示下进行了策划和对齐。该语料库包含超过20K小时的处理训练数据，至今为止是我们所知的同类中最大的开源语料库。我们在该语料库上预训练了InternW0-$ ext{Δ}$，并在仿真基准和真实机器人平台上展示了强大的性能。我们将根据许可条件开源训练代码、模型权重、基础设施、数据处理管道和处理后的数据。项目页面：https://internrobotics.github.io/InternW0-Delta/

View on arXiv Download PDF AI Translation

cs.RO / 73 / 2609.31396

Augmented Reality Interfaces for Human-Robot Collaboration: Development of a ROS 2-Based Sensor Streaming Framework and Validation via SLAM Algorithms

人机协作的增强现实接口：基于ROS 2的传感器流框架的开发及其通过SLAM算法的验证

Rubert, Alessandro, Ghidoni, Stefano, Terreran, Matteo

Abstract

In recent years, Human-Robot Collaboration (HRC) has taken on a central role in Industry 4.0 and collaborative robotics, demanding communication channels that are increasingly bidirectional, intuitive, and efficient. In this context, Augmented Reality (AR) presents itself as a fundamental enabling technology, capable of both displaying information to the operator and gathering spatial data about the surrounding environment. This thesis presents the development of a sensor streaming framework that connects the Magic Leap 2 AR headset with the ROS 2 (Robot Operating System) ecosystem. Using the Unity development environment and the ROSTCP-Connector package, an on-board application for the headset was developed, capable of acquiring real-time data from the integrated sensors (pose tracking, cameras, and environmental sensors) and publishing it to dedicated ROS 2 topics. In order to test the accuracy, latency, and robustness of the generated data stream, the framework was validated using SLAM (Simultaneous Localization and Mapping) algorithms known in the literature. The experimental results demonstrate that the proposed architecture ensures stable data transmission, laying the groundwork for safe real-time interaction and shared spatial awareness, and opening up new perspectives for the control and supervision of robotic systems in complex HRC scenarios.

Chinese Translation

近年来，人机协作（HRC）在工业4.0和协作机器人领域中扮演了核心角色，要求通信渠道越来越双向、直观和高效。在此背景下，增强现实（AR）作为一种基本的赋能技术，能够向操作员展示信息并收集周围环境的空间数据。本文提出了一种传感器流框架的开发，该框架将Magic Leap 2增强现实头戴设备与ROS 2（机器人操作系统）生态系统连接起来。利用Unity开发环境和ROSTCP-Connector软件包，开发了一种用于头戴设备的车载应用，能够实时获取集成传感器（姿态追踪、摄像头和环境传感器）数据并将其发布到专用的ROS 2主题。为了测试生成数据流的准确性、延迟和鲁棒性，该框架使用文献中已知的SLAM（同时定位与地图构建）算法进行了验证。实验结果表明，所提出的架构确保了稳定的数据传输，为安全的实时交互和共享空间意识奠定了基础，并为复杂HRC场景中机器人系统的控制和监督开辟了新的视角。

View on arXiv Download PDF AI Translation

cs.RO / 74 / 2609.31412

dRVG: Quadtree-Guided, Resolution-Complete Online Motion Planning for Polygonal Robots in Unknown Environments

dRVG：基于四叉树引导的多边形机器人在未知环境中的分辨率完整在线运动规划

Zhang, Duo, Zhang, Hechen, Huang, Junshan, Yu, Jingjin

Abstract

We present the dynamic rotation-stacked visibility graph (dRVG), an online motion planner that guides polygonal robots to specified goals in initially unknown, static environ- ments. It merges local roadmaps from successive observations to plan collision-free translations and rotations without a uniform position grid. A spatial quadtree schedules sensing goals across regions to reduce repeated visits while retaining all orientation configurations for routing. Under exact sensing and geometric computation and star-shaped robot and envelope assumptions, dRVG with center scans is resolution-complete relative to full- map RVG at the same angular resolution. In experiments using footprint scans, dRVG solves all 140 cases across 20 difficult maps and seven angular resolutions within a 20 s planning budget, with a median planning time of 1.18 s at 360 orientation layers. Six microMVP demonstrations illustrate the complete online planning loop on a physical robot.

Chinese Translation

我们提出了动态旋转堆叠可见性图（dRVG），这是一种在线运动规划器，能够引导多边形机器人在最初未知的静态环境中到达指定目标。它将来自连续观测的局部路网合并，以规划无碰撞的平移和旋转，而无需统一的位置网格。空间四叉树在各个区域调度感知目标，以减少重复访问，同时保留所有方向配置以进行路径规划。在精确感知和几何计算以及星形机器人和包络假设的条件下，dRVG在中心扫描下相对于相同角度分辨率的全图RVG是分辨率完整的。在使用足迹扫描的实验中，dRVG在20个困难地图和七个角度分辨率下，在20秒的规划预算内解决了所有140个案例，360个方向层的中位规划时间为1.18秒。六个microMVP演示展示了在物理机器人上完整的在线规划循环。

View on arXiv Download PDF AI Translation

cs.RO / 75 / 2609.31418

CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping with an LLM Agent for Immersive Collaborative VR Teleoperation

CognitiveReality：一种与机器人无关的语义高斯映射，结合LLM代理用于沉浸式协作虚拟现实遥操作

Kozlov, Timofei, Maliukov, Dmitrii, Marchenko, Andrey, Plotnikov, Dmitrii, Cabrera, Miguel Altamirano, Tsetserukou, Dzmitry

Abstract

A photorealistic 3D view tells a teleoperator where a robot is, but not what the scene contains, how well each object has been observed, or how to turn pointing and speech into robot action. CognitiveReality turns a robot's RGB-D stream into a live, semantically indexed Gaussian-TSDF map shared by an operator in virtual reality and a tool-using language agent. One mapper binary serves any platform through configuration alone: it ingests poses from robot SLAM, joint kinematics, motion capture or an inline visual tracker, bridges localization outages with a shadow tracker and keyframe-anchored PnP, and maintains open-vocabulary instance identities with per-object quality at 2 Hz. Speech and controller rays are grounded against persistent scene objects through validated typed tools and operator-confirmed robot actions. In the controlled agent evaluation, the deployed local Qwen3-VL-8B router reaches 81.24\% tool exact match, while merge-aware replay correctly redirects 101 absorbed object identifiers. On robot data CognitiveReality exceeds a Gaussian-plus-SDF baseline by 2-8 dB; pose error through 5-40 s SLAM outages stays within 1-8 cm. Deployed live on two quadrupeds, the agent executed 26 of 30 navigation requests and 20 of 20 re-observation requests, raising object quality by 2-5 dB.

Chinese Translation

逼真的3D视图可以告诉遥操作员机器人所在的位置，但无法提供场景包含的内容、每个物体的观察质量或如何将指向和语言转化为机器人动作的信息。CognitiveReality将机器人的RGB-D流转换为一个实时的、语义索引的高斯-TSDF地图，该地图由虚拟现实中的操作员和使用工具的语言代理共享。一个映射器二进制文件仅通过配置即可服务于任何平台：它可以从机器人SLAM、关节运动学、动作捕捉或内联视觉跟踪器中获取位姿，利用影子跟踪器和关键帧锚定的PnP来弥补定位中断，并以每个物体的质量在2 Hz的频率下维护开放词汇实例身份。通过经过验证的类型工具和操作员确认的机器人动作，语音和控制器光线与持久场景物体相结合。在受控代理评估中，部署的本地Qwen3-VL-8B路由器达到了81.24%的工具精确匹配，而合并感知重放正确重定向了101个吸收的物体标识符。在机器人数据上，CognitiveReality超越了高斯加SDF基线2-8 dB；在5-40秒的SLAM中断期间，位姿误差保持在1-8厘米以内。该代理在两只四足机器人上实时部署，成功执行了30个导航请求中的26个和20个重新观察请求中的20个，将物体质量提高了2-5 dB。

View on arXiv Download PDF AI Translation

cs.RO / 76 / 2609.31434

ExoLaN: Physics-Consistent Context-Aware Dynamics Learning for Exoskeletons

ExoLaN：物理一致的上下文感知动态学习用于外骨骼

Schulze, Lucas, Schwarz, Maximilian, Hoppe, Jona, Peters, Jan, Arenz, Oleg

Abstract

Task-agnostic assistive exoskeleton control based on human intention offers greater flexibility than conventional approaches that rely on predefined tasks or motion patterns. Human joint torque estimation enables task-agnostic assistance by characterizing user actions. Physics-consistent methods such as Deep Lagrangian Networks (DeLaN) have been applied to estimate the human torques in multi-user settings, but existing approaches cannot adapt to a specific user without retraining, and do not account for intermittent contacts during locomotion. We propose ExoLaN, a Context-Aware DeLaN for human-exoskeleton interaction that learns the full coupled system dynamics while adapting to changes in interaction context. ExoLaN combines temporal context with partial contact-force measurements from force-sensitive insoles to infer latent dynamics embeddings and estimate generalized contact torques. On seven unseen users performing 21 unseen tasks, ExoLaN reduces torque estimation MSE by 7% compared to a black-box baseline. Beyond inverse dynamics, ExoLaN serves as a unified model that also enables accurate forward prediction: training with a multi-step prediction loss reduces acceleration MSE by 59% and long-horizon position and velocity errors by 60% and 93%, respectively, compared with a single-step loss. Moreover, the learned latent context captures task information without explicit task labels, making it a promising signal for task-aware assistive control.

Chinese Translation

基于人类意图的任务无关辅助外骨骼控制相比于依赖预定义任务或运动模式的传统方法提供了更大的灵活性。人类关节扭矩估计通过表征用户动作实现了任务无关的辅助。物理一致的方法如深度拉格朗日网络（Deep Lagrangian Networks, DeLaN）已被应用于多用户环境中估计人类扭矩，但现有方法无法在不重新训练的情况下适应特定用户，并且未考虑在运动过程中出现的间歇性接触。我们提出了ExoLaN，一种用于人类与外骨骼交互的上下文感知DeLaN，它学习完整的耦合系统动态，同时适应交互上下文的变化。ExoLaN结合了时间上下文与来自力敏鞋垫的部分接触力测量，以推断潜在的动态嵌入并估计广义接触扭矩。在七个未见用户执行21个未见任务的实验中，ExoLaN相比于黑箱基线减少了7%的扭矩估计均方误差（MSE）。除了逆动态，ExoLaN还作为一个统一模型，能够实现准确的前向预测：使用多步预测损失进行训练相比于单步损失，减少了59%的加速度MSE以及60%和93%的长时间位置和速度误差。此外，学习到的潜在上下文在没有显式任务标签的情况下捕捉了任务信息，使其成为任务感知辅助控制的有希望信号。

View on arXiv Download PDF AI Translation

cs.RO / 77 / 2609.31439

Learning to Leverage Compliance: A Policy-Admittance Learning Framework for Robotic Insertion

学习利用合规性：一种用于机器人插入的策略接纳学习框架

Wang, Chongren, Li, Minghe, Dai, Honghua, Lin, Zhicheng, Wei, Shiyang, Yue, Xiaokui

Abstract

Policy learning and compliant control offer a promising route to reliable autonomous assembly under pose errors and contact uncertainty. However, combining them does not ensure coordination: the policy may continue pushing against contact while the controller yields, producing sustained loading with limited progress. To address this problem, we propose LeCo (Leverage Compliance), a policy-admittance learning framework that guides a visual policy through execution-time interaction under fixed admittance. A multirate feedback mechanism aggregates high-rate contact-interaction records into policy-transition rewards. An integrated conflict cost then characterizes sustained policy-loading/controller-unloading opposition, while a directional high-force tail cost captures continued-loading events within a transition. Together with task completion, these costs encourage the policy to leverage compliance with less unproductive loading. We evaluate LeCo on four real connector-assembly tasks, obtaining an aggregate success rate of 94%. Across tasks, mean successful-trial resultant-force and torque peaks decrease by approximately 30% and 64% relative to the comparison baseline. Reward ablation further shows that adding conflict shaping reduces median successful-trial contact-conditioned conflict density by approximately 53%. These results support learning to leverage fixed compliance by turning multirate policy-admittance interaction into complementary reward signals for effective, lower-load insertion.

Chinese Translation

策略学习和合规控制为在姿态误差和接触不确定性下实现可靠的自主组装提供了一条有前景的途径。然而，二者的结合并不能确保协调：策略可能在控制器让步时继续施加压力，导致持续加载但进展有限。为了解决这一问题，我们提出了LeCo（Leverage Compliance），一种策略接纳学习框架，通过在固定接纳下的执行时交互来引导视觉策略。多速率反馈机制将高频接触交互记录汇总为策略转移奖励。集成的冲突成本则表征了持续的策略加载/控制器卸载对抗，而方向性高强度尾部成本则捕捉了转移过程中的持续加载事件。结合任务完成，这些成本鼓励策略在减少无效加载的情况下利用合规性。我们在四个实际连接器组装任务上评估了LeCo，获得了94%的总体成功率。在各任务中，成功试验的平均结果力和扭矩峰值相较于对照基线分别降低了约30%和64%。奖励消融实验进一步表明，添加冲突塑形将中位成功试验的接触条件下冲突密度降低了约53%。这些结果支持通过将多速率策略接纳交互转化为有效、低负载插入的互补奖励信号来学习利用固定合规性。

View on arXiv Download PDF AI Translation

cs.RO / 78 / 2609.31452

Vision-Based 6-DoF Grasp Pose Estimation for Robot Cloth Unfolding

基于视觉的六自由度抓取姿态估计用于机器人布料展开

Tabernik, Domen, Nimac, Peter, Jerićević, Jan, Skočaj, Danijel, Gams, Andrej

Abstract

Cloth manipulation is a challenging task due to the deformable and high-dimensional nature of cloth, which leads to complex interaction dynamics and perceptual ambiguity arising from frequent occlusions of critical visual cues such as folds, edges, and grasp points. In this work, we tackle cloth unfolding using a regrasping-in-the-air strategy, where one manipulator holds the cloth while the other grasps it at an optimally selected point to unfold it. To this end, we propose CeDiRNet-6DoF, a deep learning framework that jointly predicts effective grasp points and the complete 6-DoF grasp pose from the observed cloth configuration. By integrating dense 3D grasp regression with segmentation and sine-cosine-encoded Euler angles, the proposed method reliably estimates the grasp configuration that maximizes the unfolded cloth area. We extensively evaluated CeDiRNet-6DoF on a bimanual robotic setup within the ICRA 2024 Cloth Competition framework, achieving state-of-the-art performance. An ablation study further validates the benefits of key design components, including joint segmentation, background randomization, and image cropping. These results establish CeDiRNet-6DoF as a robust and versatile foundation for reliable robotic cloth manipulation in unstructured environments.

Chinese Translation

布料操作是一项具有挑战性的任务，因为布料的可变形和高维特性导致复杂的交互动态以及由于关键视觉线索（如褶皱、边缘和抓取点）频繁遮挡而产生的感知模糊。在本研究中，我们采用了一种空中重新抓取策略来处理布料展开，其中一个操控器固定布料，而另一个操控器在最佳选择的点上抓取布料以展开它。为此，我们提出了CeDiRNet-6DoF，这是一种深度学习框架，能够从观察到的布料配置中联合预测有效的抓取点和完整的六自由度抓取姿态。通过将密集的三维抓取回归与分割和正弦-余弦编码的欧拉角相结合，所提出的方法可靠地估计最大化展开布料面积的抓取配置。我们在ICRA 2024布料竞赛框架下的双手机器人设置上对CeDiRNet-6DoF进行了广泛评估，取得了最先进的性能。消融研究进一步验证了关键设计组件的优势，包括联合分割、背景随机化和图像裁剪。这些结果确立了CeDiRNet-6DoF作为在非结构化环境中进行可靠机器人布料操作的稳健而多功能的基础。

View on arXiv Download PDF AI Translation

cs.RO / 79 / 2609.31577

Generate, Track, Improve: Perceptive Multi-Skill Humanoid Locomotion with RL-Fine-Tuned Motion Generators

生成、跟踪、改进：基于强化学习微调的多技能感知类人运动控制

Olkin, Zachary, Compton, William D., Ames, Aaron D.

Abstract

General purpose humanoids require locomotion controllers that are multi-skill, perceptive, dynamic, and robust enough to go anywhere humans can. In this work, we present a two layer locomotion architecture: (1) a perceptive flow matching motion generator plans whole body trajectories from raw depth images while a (2) perceptive tracking policy trained with control-guided RL follows these motions. Both policies are trained on a library of terrain consistent motion clips created with dynamically optimized human data which yields both accurate velocity tracking and terrain consistent references. Our central contribution is a simple yet effective off-policy RL fine tuning loop that improves the motion generator. A structured search method is used with the generator to gather data for advantage weighted regression. This off-policy loop is much more sample efficient than on-policy residual fine tuning and improves terrain consistency on unseen geometries and skill compositions. We find that successful terrain traversals increased by up to 25 percentage points and skill selection improved by up to 80 percentage points. By using raw depth images to perceive the environment no odometry or height maps are needed, and outdoor deployment is easy. With two cameras, the policy can see terrain coming from further away and adjust its velocity regardless of the commanded speed so it can traverse the terrain. A single policy pair enables a Unitree G1 humanoid to walk, run, stand, jump on and off of boxes, and traverse stairs in outdoor environments. Project page: https://zolkin1.github.io/generate-track-improve/

Chinese Translation

通用类人机器人需要具备多技能、感知、动态且足够稳健的运动控制器，以便能够到达人类可以到达的任何地方。在本研究中，我们提出了一种两层运动架构：(1) 感知流匹配运动生成器根据原始深度图像规划全身轨迹；(2) 经过控制引导的强化学习训练的感知跟踪策略跟随这些运动。这两种策略是在一个由动态优化的人类数据创建的地形一致运动片段库上进行训练的，从而实现了准确的速度跟踪和地形一致的参考。我们的核心贡献是一个简单而有效的离线策略强化学习微调循环，用于改进运动生成器。该生成器使用结构化搜索方法收集优势加权回归的数据。与在线策略残差微调相比，这个离线循环的样本效率更高，并且在未见几何体和技能组合上提高了地形一致性。我们发现成功的地形穿越率提高了多达25个百分点，技能选择的改进达到了80个百分点。通过使用原始深度图像来感知环境，无需里程计或高度图，户外部署变得简单。通过两个摄像头，策略能够从更远的地方看到地形，并根据命令速度调整其速度，从而能够顺利穿越地形。一对单一策略使Unitree G1类人机器人能够在户外环境中行走、奔跑、站立、跳上和跳下箱子，以及上下楼梯。项目页面：https://zolkin1.github.io/generate-track-improve/

View on arXiv Download PDF AI Translation

cs.RO / 80 / 2609.31606

Learning Robot Policies from Sparse Success Signals via STL-Guided Stein Variational Policy Gradient

通过 STL 引导的 Stein 变分策略梯度从稀疏成功信号中学习机器人策略

Zheng, Hongrui, Vasile, Cristian Ioan, Loquercio, Antonio, Mangharam, Rahul

Abstract

Learning robot policies for tasks with sparse success signals is challenging when completion depends on coordinated actions, precise contact outcomes, or satisfying several conditions together. Intricate physical interactions with the world further complicate these requirements. Prior work using conventional reward shaping mechanisms provides dense feedback but local progress might not translate into eventual task completion. We present Signal Temporal Logic-guided Stein Variational Policy Gradient (STL-SVPG), a population-based method that uses smooth STL robustness as a trajectory-level training objective. Differentiating this objective through the dynamics assigns credit to policy actions according to their effect on the complete task specification, rather than local progress alone. We evaluate the approach on six quadcopter and manipulator tasks that involves event-triggered responses, strictly ordered behavior, responses within specified deadlines, and physical interaction with the world. STL-SVPG achieves the highest mean success rate among the compared methods on five of six benchmarks. Simulation-trained policies trained in simulation transfer temporal and contact task behavior to the real world.

Chinese Translation

对于依赖协调动作、精确接触结果或同时满足多个条件的任务，从稀疏成功信号中学习机器人策略是具有挑战性的。与世界的复杂物理交互进一步增加了这些要求的复杂性。以往的研究使用传统的奖励塑形机制提供密集反馈，但局部进展可能无法转化为最终的任务完成。我们提出了一种信号时序逻辑引导的 Stein 变分策略梯度（STL-SVPG），这是一种基于种群的方法，利用平滑的 STL 鲁棒性作为轨迹级训练目标。通过动态区分该目标，根据策略动作对完整任务规范的影响，而不仅仅是局部进展，来分配信用。我们在六个四旋翼和操纵器任务上评估该方法，这些任务涉及事件触发响应、严格顺序行为、在指定截止时间内的响应以及与世界的物理交互。STL-SVPG 在六个基准中的五个上实现了最高的平均成功率。经过仿真训练的策略能够将时间和接触任务行为转移到现实世界中。

View on arXiv Download PDF AI Translation

计算机视觉 (Computer Vision)

cs.CV / 1 / 2609.30356

AlphaEarth distinguishes cities but compresses urban variation

AlphaEarth区分城市但压缩城市变异性

Renninger, Andrew

Abstract

Cities differ in built form, land cover and development history, complicating comparison across places and time. Satellite foundation models map Earth's surface onto common numerical representations. Yet the tasks and targets used to shape them typically do not focus on cities: globally consistent labels for urban function do not exist, and many datasets - especially land cover and land use classifications - collapse the built environment into few classes. Here we audit the representation, focusing on AlphaEarth but with broader applicability to other Earth embeddings, by probing the geometry and geography of embeddings for 1,000 urban areas in 162 countries. We find that cities occupy a shifted but overlapping region on the hypersphere, 62.7{\deg} from the global mean direction, and continent and climate predict 24.3% of variation among the mean directions of urban centres in excluded countries. Inside cities, degrees of urbanisation carry 8.9% of the variation, and what they leave holds shared directions whose local orientation varies, not one universal axis of urbanisation. Retained variation is itself unequal: dispersion within urban centres is 14.1% greater per standard deviation of national development, even after adjusting for population, land area and continent. Further controls suggest cities in developing countries present less contrast in vegetation and texture, and dispersion follows that contrast: full adjustment for it leaves at most 6.4% of the gradient. Annually, a city's representation moves nearly eight times more than redrawing its own pixels explains, and contracts where the 2022 loss of Sentinel-1B removed a pass direction. AlphaEarth's representations therefore support comparison across regions, while the differences between its annual layers are not yet validated for comparison over time.

Chinese Translation

城市在建筑形式、土地覆盖和发展历史上存在差异，这使得跨地点和时间的比较变得复杂。卫星基础模型将地球表面映射为共同的数值表示。然而，用于塑造这些模型的任务和目标通常并不专注于城市：全球一致的城市功能标签并不存在，许多数据集——尤其是土地覆盖和土地利用分类——将建筑环境简化为少数几类。在此，我们对表示进行审计，重点关注AlphaEarth，但也适用于其他地球嵌入，通过探讨在162个国家的1,000个城市区域的嵌入几何和地理。我们发现，城市在超球面上占据了一个偏移但重叠的区域，距离全球平均方向62.7°，而大陆和气候预测了被排除国家城市中心平均方向变化的24.3%。在城市内部，城市化程度占变化的8.9%，而它们所留下的共享方向具有不同的局部取向，而不是一个普遍的城市化轴。保留的变异本身是不均等的：城市中心内的离散度每个国家发展标准差大14.1%，即使在调整了人口、土地面积和大陆后。进一步的控制表明，发展中国家的城市在植被和纹理上呈现出较少的对比，而离散度则遵循这种对比：对其的完全调整最多留下6.4%的梯度。每年，一个城市的表示变化几乎是重新绘制其自身像素所解释的八倍，并在2022年Sentinel-1B的丧失移除了一条通行方向时收缩。因此，AlphaEarth的表示支持跨区域的比较，而其年度层之间的差异尚未经过验证以进行时间上的比较。

View on arXiv Download PDF AI Translation

cs.CV / 2 / 2609.30393

LiTe-GS: Oracle-Efficient Next Best View Selection for 3D Gaussian Splatting

LiTe-GS：高效的下一个最佳视图选择方法用于3D高斯点云渲染

Pandey, Vivek, Khass, Amirhossein Mollaei, Motee, Nader

Abstract

Selecting informative camera views is critical for efficient training and adaptive refinement in 3D Gaussian Splatting, where each observation significantly influences model parameters. However, information-driven view-selection strategies can require repeated evaluations of expensive information-gain oracles as the number of candidate views increases. We propose LiTe-GS, an oracle-efficient method for next best view selection in 3D Gaussian Splatting. LiTe-GS reduces the number of information-oracle evaluations by performing randomized subset evaluation of candidate views rather than exhaustively scoring the full candidate pool. The resulting approach achieves expected $O(M\log(1/\epsilon))$ oracle complexity with respect to the number of candidate views $M$, independent of the selection cardinality $K$, while providing an explicit trade-off between oracle efficiency and approximation quality through $\epsilon$. We provide theoretical guarantees on oracle complexity and approximation performance under the proposed selection scheme. Experiments on Blender and Mip-NeRF 360 demonstrate that LiTe-GS maintains reconstruction quality comparable to Fisher-information-based baselines while substantially reducing the number of Fisher-oracle evaluations across different acquisition settings.

Chinese Translation

选择信息丰富的相机视图对于3D高斯点云渲染中的高效训练和自适应细化至关重要，其中每次观察都会显著影响模型参数。然而，信息驱动的视图选择策略可能需要在候选视图数量增加时反复评估昂贵的信息增益oracle。我们提出了LiTe-GS，一种在3D高斯点云渲染中高效选择下一个最佳视图的方法。LiTe-GS通过对候选视图进行随机子集评估，而不是对整个候选池进行详尽评分，从而减少了信息oracle的评估次数。该方法在候选视图数量$M$方面实现了预期的$O(M ext{log}(1/ ext{ε}))$ oracle复杂度，与选择基数$K$无关，同时通过$ ext{ε}$提供了oracle效率与近似质量之间的明确权衡。我们对所提出的选择方案下的oracle复杂度和近似性能提供了理论保证。在Blender和Mip-NeRF 360上的实验表明，LiTe-GS在不同采集设置下保持了与基于Fisher信息的基准相当的重建质量，同时显著减少了Fisher oracle的评估次数。

View on arXiv Download PDF AI Translation

cs.CV / 3 / 2609.30395

CSCWD: Cross-Scale Channel-wise Knowledge Distillation for Lightweight Tiny Object Detection on Edge Devices

CSCWD：用于边缘设备轻量级小物体检测的跨尺度通道知识蒸馏

Zamani, Amir, Ghasemi-Naraghi, Zeinab

Abstract

Real-time tiny object detection in aerial imagery is constrained by the weak spatial evidence of very small objects and the loss of high-resolution detail in lightweight detectors. This study presents Cross-Scale Channel-wise Knowledge Distillation (CSCWD), a training-time framework that transfers high-resolution spatial representations from a YOLO11m-P2 teacher to a compact YOLO11n student without altering the student's inference architecture. Unlike conventional same-scale feature distillation, CSCWD transfers supervision from teacher P2 to student P3 after feature alignment while retaining same-scale distillation at deeper pyramid levels. Under the unified seven-sequence Drone-vs-Bird validation protocol, YOLO11n-CSCWD achieves 50.17% mean average precision at an intersection-over-union threshold of 0.5 ([email protected]) and 59.73% recall, improving the matched CA-YOLO11n baseline by 2.92 percentage points in [email protected] and 3.55 points in recall. Cross-scale alignment further increases [email protected] by 2.09 points over the corresponding same-scale channel-wise distillation configuration. In zero-shot evaluation on DUT-Anti-UAV, [email protected] increases from 48.29% to 50.06% without target-domain fine-tuning. This domain was included because its challenging small targets make low-latency, computationally efficient detection particularly relevant. On Raspberry Pi 5 using NCNN-FP16 at 640x640 resolution, the 2.58-million-parameter student achieves 50.32% [email protected] at 82.32 ms mean wall-clock latency, or 12.15 frames per second, while retaining essentially the same runtime and memory requirements as the matched baseline. The results support cross-scale distillation for improving tiny-target detection without increasing inference-time model complexity.

Chinese Translation

实时小物体检测在航空图像中受到非常小物体的空间证据薄弱和轻量级检测器高分辨率细节丢失的限制。本研究提出了跨尺度通道知识蒸馏（Cross-Scale Channel-wise Knowledge Distillation，CSCWD），这是一个训练时框架，能够在不改变学生推理架构的情况下，将高分辨率空间表示从YOLO11m-P2教师转移到紧凑的YOLO11n学生。与传统的同尺度特征蒸馏不同，CSCWD在特征对齐后将监督从教师P2转移到学生P3，同时在更深的金字塔层级保留同尺度蒸馏。在统一的七序列无人机对鸟类验证协议下，YOLO11n-CSCWD在交并比阈值为0.5时实现了50.17%的平均精度均值（[email protected]）和59.73%的召回率，相比于匹配的CA-YOLO11n基线在[email protected]上提高了2.92个百分点，在召回率上提高了3.55个百分点。跨尺度对齐进一步使得[email protected]相比于相应的同尺度通道蒸馏配置提高了2.09个百分点。在对DUT-Anti-UAV的零样本评估中，[email protected]从48.29%提高到50.06%，而无需目标领域的微调。该领域之所以被纳入，是因为其具有挑战性的小目标使得低延迟、计算高效的检测尤为相关。在使用NCNN-FP16在640x640分辨率下的Raspberry Pi 5上，拥有258万参数的学生在82.32毫秒的平均时延下实现了50.32%的[email protected]，或每秒12.15帧，同时保持与匹配基线基本相同的运行时和内存需求。这些结果支持跨尺度蒸馏以提高小目标检测，而不增加推理时模型复杂性。

View on arXiv Download PDF AI Translation

cs.CV / 4 / 2609.30402

What Improves Multimodal Misinformation Detection? Answers from a Large-Scale Empirical Study

如何改善多模态虚假信息检测？来自大规模实证研究的答案

Sharma, Akshit, Patil, Prashant W.

Abstract

Multimodal misinformation is increasingly crafted to look convincing by pairing a textual claim with an image that appears to "prove" it. Yet in practice, building effective detectors often hinges on a small set of design choices that are rarely examined in a controlled way. In this paper, we conduct a large-scale study of multimodal design choices for misinformation detection with over 3,375 experiments- spanning three benchmark datasets and a broad range of pre-trained vision and language backbones. Through systematic comparisons and targeted robustness analyses, we distill practical guidance on which design choices help, when do they fail silently, and what aspects of the pipeline most strongly shape model behavior, answering 4 key Research Questions (RQs). We aim to provide a reliable foundation for designing stronger and more dependable multimodal misinformation detection systems, thus contributing to the broader research community.

Chinese Translation

多模态虚假信息越来越倾向于通过将文本声明与看似“证明”该声明的图像相结合来显得令人信服。然而，在实践中，构建有效的检测器往往依赖于一小组设计选择，而这些选择很少以受控方式进行检验。在本文中，我们对多模态虚假信息检测的设计选择进行了大规模研究，进行了超过3,375次实验，涵盖了三个基准数据集和广泛的预训练视觉与语言模型。通过系统比较和针对性的鲁棒性分析，我们提炼出关于哪些设计选择有效、何时它们会悄然失效以及管道的哪些方面最强烈地影响模型行为的实用指导，回答了四个关键研究问题（RQs）。我们的目标是为设计更强大、更可靠的多模态虚假信息检测系统提供可靠的基础，从而为更广泛的研究社区做出贡献。

View on arXiv Download PDF AI Translation

cs.CV / 5 / 2609.30434

ProCAP: Probabilistic Cross-Attentive Prompt Learning for Vision-Language Models

ProCAP：用于视觉-语言模型的概率交叉注意力提示学习

Abbas, Hiwa Azeez, Daneshfar, Fatemeh, Abdar, Moloud

Abstract

Pre-trained vision-language models such as CLIP can recognize new categories via prompting, but they often struggle when labeled data are scarce or the test distribution shifts. Prompt learning adapts only a small set of parameters while keeping the backbone frozen, yet many existing multimodal prompt learners couple the visual and textual branches weakly and can be brittle in low-shot regimes. We propose ProCAP, a probabilistic cross-attentive prompt learning framework that improves cross-modal interaction and training stability without updating any CLIP weights: it learns both visual and textual prompt tokens and links them through stacked bidirectional multi-head cross-attention so the two branches refine each other across prompt depth. To reduce overfitting under limited supervision, we parameterize prompt tokens with Gaussian means and variances and regularize them with lightweight KL and L2 penalties, and we further add a compact symmetric InfoNCE head that aligns cross-attended image features with class-level text representations in a shared low-dimensional space. Across few-shot base-to-novel generalization on 11 datasets, cross-dataset transfer, and domain generalization on ImageNet shift benchmarks, ProCAP achieves strong aggregate base-to-novel performance and competitive transfer performance while keeping the CLIP backbone unchanged.

Chinese Translation

预训练的视觉-语言模型，如CLIP，可以通过提示识别新类别，但在标记数据稀缺或测试分布发生变化时，它们常常表现不佳。提示学习仅调整一小部分参数，同时保持主干网络不变，然而许多现有的多模态提示学习者在视觉和文本分支之间的耦合较弱，在低样本情况下可能表现脆弱。我们提出了ProCAP，一种概率交叉注意力提示学习框架，它在不更新任何CLIP权重的情况下改善了跨模态交互和训练稳定性：它学习视觉和文本提示标记，并通过堆叠的双向多头交叉注意力将它们连接起来，使得两个分支在提示深度上相互优化。为了减少在有限监督下的过拟合，我们用高斯均值和方差对提示标记进行参数化，并通过轻量级的KL和L2惩罚进行正则化，此外，我们还添加了一个紧凑的对称InfoNCE头，将交叉注意的图像特征与共享低维空间中的类别级文本表示对齐。在11个数据集上的少样本基础到新颖泛化、跨数据集迁移和ImageNet迁移基准上的领域泛化测试中，ProCAP实现了强大的整体基础到新颖的性能和具有竞争力的迁移性能，同时保持CLIP主干不变。

View on arXiv Download PDF AI Translation

cs.CV / 6 / 2609.30450

LensDesigner: A Self-Improving Agent for Optical Lens Design

LensDesigner：一种自我改进的光学镜头设计代理

Sun, Lei, Liang, Haoran, Xu, Dannong, Gao, Yao, Geng, Yuyu, Gu, Jinjin, Wang, Kaiwei, Paudel, Danda Pani, Van Gool, Luc

Abstract

Optical lens design is a complex, non-convex optimization challenge that relies heavily on human experience and intuition. Existing optimized-based automatic lens design methods struggle to navigate this vast parameter space without meticulous manual tuning. In this paper, we present LensDesigner, an autonomous agent framework that mirrors the problem-solving workflow of expert opticians. To overcome the initial cold start problem, we construct LensLib100K, an extensive optical lens library, and employ Optics-Aware Retrieval to supply physically valid structural seeds. Within an interactive physical simulation environment, the agent executes macroscopic orchestration while receiving immediate optical feedback. Furthermore, we introduce a continuous self-evolving mechanism guided by a curriculum agent. By iteratively solving design tasks with progressively increasing difficulty, the agent autonomously extracts, accumulates, and reuses design heuristics, effectively evolving its optical lens design expertise over time. At the evaluation level, we introduce LensArena, a standardized evaluation benchmark comprising $120$ diverse optical design tasks, covering extreme configurations. Extensive experiments on this benchmark demonstrate that LensDesigner significantly outperforms publicly available baseline algorithms, achieving superior success rates and optimization efficiency. We hope this work sheds light on the emerging field of intelligent optics. The code will be publicly available.

Chinese Translation

光学镜头设计是一个复杂的非凸优化挑战，严重依赖于人类的经验和直觉。现有的基于优化的自动镜头设计方法在没有细致的手动调优的情况下，难以在广阔的参数空间中导航。本文提出了LensDesigner，一个自主代理框架，模拟了专家验光师的问题解决工作流程。为了解决初始冷启动问题，我们构建了LensLib100K，一个广泛的光学镜头库，并采用光学感知检索（Optics-Aware Retrieval）提供物理有效的结构种子。在一个交互式物理仿真环境中，代理执行宏观协调，同时接收即时的光学反馈。此外，我们引入了一个由课程代理（curriculum agent）指导的连续自我进化机制。通过以逐渐增加难度的方式迭代解决设计任务，代理自主提取、积累并重用设计启发式，随着时间的推移有效地提升其光学镜头设计专业知识。在评估层面，我们引入了LensArena，一个标准化的评估基准，包含120个多样化的光学设计任务，涵盖极端配置。在该基准上的大量实验表明，LensDesigner显著优于公开可用的基线算法，取得了更高的成功率和优化效率。我们希望这项工作能为智能光学的新兴领域提供启示。代码将公开发布。

View on arXiv Download PDF AI Translation

cs.CV / 7 / 2609.30478

The Shape of Events: Edge-Based Inductive Biases via Cross-Domain Distillation

事件的形状：通过跨领域蒸馏实现基于边缘的归纳偏置

Kihara, Soshun, Yasuki, Shunsuke, Taki, Masato

Abstract

Convolutional neural networks trained on ImageNet are known to exhibit a strong preference for local high-frequency texture, an inductive bias that translates into fragile robustness against distribution shifts in real-world environments. Event cameras, in contrast, record only changes in scene brightness and are therefore well suited to capturing contour information; however, due to the absence of diagnostic benchmarks in the event domain, the inductive bias that event-camera data instills in vision models has remained underexplored. In this work, we use knowledge distillation from the event domain to the RGB domain so as to exploit the rich evaluation toolkit available in the RGB domain and systematically dissect this inductive bias. Our experiments show that distillation from the event domain induces, in the RGB domain, color invariance, shape bias, and robustness to high-frequency noise. We identify the underlying mechanism as the model suppressing its dependence on high-frequency texture while acquiring a stronger dependence on edge-based object shape. This hypothesis is supported by changes in how color and spatial information are processed at the early layers, together with a spectral trade-off in which robustness to the absence of high-frequency components coexists with vulnerability to contamination of the relied-upon frequency bands and to disruption of geometric structure. We further show that this inductive bias differs from existing robustification methods and that it functions as a useful prior for diverse downstream tasks in which shape and contour information contribute alongside other cues. The code is available at https://github.com/snskysk/event2rgb-distillation .

Chinese Translation

在ImageNet上训练的卷积神经网络已知对局部高频纹理表现出强烈偏好，这种归纳偏置导致其在现实环境中的分布变化下表现出脆弱的鲁棒性。相比之下，事件相机仅记录场景亮度的变化，因此非常适合捕捉轮廓信息；然而，由于事件领域缺乏诊断基准，事件相机数据对视觉模型所施加的归纳偏置仍然未得到充分探讨。在本研究中，我们利用从事件领域到RGB领域的知识蒸馏，以利用RGB领域中丰富的评估工具包，并系统性地剖析这种归纳偏置。我们的实验表明，从事件领域的蒸馏在RGB领域中引入了颜色不变性、形状偏置以及对高频噪声的鲁棒性。我们将其背后的机制识别为模型抑制对高频纹理的依赖，同时增强对基于边缘的物体形状的依赖。该假设得到了早期层中颜色和空间信息处理方式变化的支持，以及在高频成分缺失的情况下鲁棒性与对依赖频率带的污染和几何结构破坏的脆弱性之间的光谱权衡。我们进一步表明，这种归纳偏置不同于现有的鲁棒化方法，并且作为一种有用的先验，适用于形状和轮廓信息与其他线索共同贡献的多样化下游任务。代码可在 https://github.com/snskysk/event2rgb-distillation 获取。

View on arXiv Download PDF AI Translation

cs.CV / 8 / 2609.30566

Atlases Are Already Inside: Recovering Population Templates from Pretrained Diffusion Models

图谱已经内置：从预训练扩散模型中恢复人口模板

Shi, Jian, Femiani, John, Wonka, Peter

Abstract

We present a new inference-time sampler for diffusion models that gives a pretrained model a capability it was never trained for: constructing the atlas of the population it synthesizes. The sampler converges from every random seed to the population's central anatomy, which we call the \emph{intrinsic atlas}. The advantage is threefold. (1) It requires no retraining. A diffusion model that has already learned a coherent population, including the released ones, yields its atlas in a single inference pass without involving deformable registration. (2) It applies to multiple domains, such as brain MRI, chest X-ray, faces, and 3D shapes. (3) It extends to subpopulations. One age-conditioned model gives an atlas at any age in its training range, and the resulting family reproduces the CSF expansion of healthy aging. Evaluated as a registration target, the intrinsic atlas is best or second-best on every dataset against classical and learned templates, and the most central template on held-out brain MRI cohorts. Atlas construction can be reframed as a byproduct of generative modeling: a diffusion model is a learned representation of population structure, and the atlas is what it already contains.

Chinese Translation

我们提出了一种新的推断时采样器，用于扩散模型，使预训练模型具备了一种从未训练过的能力：构建其合成的人口图谱。该采样器从每个随机种子收敛到人口的中心解剖结构，我们称之为 extit{内在图谱}。其优势有三方面：（1）无需重新训练。已经学习到一致人口的扩散模型，包括已发布的模型，可以在一次推断过程中生成其图谱，而无需涉及可变形配准。（2）适用于多个领域，如脑部MRI、胸部X光、面部和三维形状。（3）可扩展至亚人群体。一个基于年龄的条件模型可以在其训练范围内的任何年龄生成图谱，所得到的家族再现了健康老化的脑脊液扩展。作为配准目标进行评估时，内在图谱在每个数据集上相较于经典和学习模板表现最佳或第二最佳，并且在保留的脑部MRI队列中是最中心的模板。图谱构建可以重新框架为生成建模的副产品：扩散模型是对人口结构的学习表示，而图谱则是其所包含的内容。

View on arXiv Download PDF AI Translation

cs.CV / 9 / 2609.30595

Action Forcing: Training World Models on Unsupervised Video by Recovering Underlying Egomotion Bases

行动强制：通过恢复基础自我运动模型在无监督视频上训练世界模型

Sundar, Ashish, Hou, Tiankuo, Fan, Zhong, Luo, Chunbo, Wang, Xiaoyang

Abstract

Synchronised action annotations are needed to train controllable world models and these datasets remain elusive. Existing approaches make use of instrumented platforms with calibrated sensors, costly manual annotation, or latent-action models which lack grounding. We instead turn ordinary unlabelled video into action-supervised training data by recovering (without training) a data-derived egomotion basis. We track pixel displacements across frames and exploit the recurring coherent structure induced by egomotion to obtain grounded control signals directly. Using a method as simple as principal components analysis perform this, we find that the leading components provide signed, scalable, and composable throttle--yaw controls, although the method can recover only motion axes represented in the data. To prevent a high-capacity video DiT from exploiting pixel-level supervision, an online latent critic distils a frozen decoder--tracker--PCA (Principal Components Analysis) teacher without backpropagating through the decoder or tracker. Finally we critique the use of video generation metrics to evaluate WMs and introduce an example of an alternative, reference-free evaluation method. We measure \textit{controllability}, \textit{plausibility}, \textit{conjuring} (creating objects out of thin air) and \textit{geometric integrity}, revealing failures that conventional video metrics miss. We show that most baselines follow familiar action directions but struggle to reverse or remain stationary. Our model handles both while retaining compositional control and generation quality. Despite backwards actions being less than $1\%$ of our training data, we find that the model learns to reverse, scale its response linearly, and compose throttle with steering, all simply by learning through a grounded action space.

Chinese Translation

训练可控的世界模型需要同步的动作注释，而这些数据集仍然难以获得。现有的方法依赖于配备校准传感器的仪器平台、昂贵的手动注释或缺乏基础的潜在动作模型。我们则通过恢复（无需训练）数据导出的自我运动基础，将普通的未标记视频转化为动作监督的训练数据。我们跟踪帧间的像素位移，并利用自我运动引发的重复一致结构直接获得基础的控制信号。使用简单的主成分分析（Principal Components Analysis）方法，我们发现主要成分提供了有符号、可扩展和可组合的油门-偏航控制，尽管该方法只能恢复数据中表示的运动轴。为了防止高容量视频的DiT利用像素级监督，一个在线潜在评估器提炼出一个冻结的解码器-跟踪器-PCA教师，而无需通过解码器或跟踪器进行反向传播。最后，我们批判性地评估了使用视频生成指标来评估世界模型，并引入了一种替代的无参考评估方法的示例。我们测量了 extit{可控性}、 extit{合理性}、 extit{幻化}（凭空创造物体）和 extit{几何完整性}，揭示了传统视频指标所遗漏的失败。我们展示了大多数基线遵循熟悉的动作方向，但在反向或保持静止时遇到困难。我们的模型在保持组合控制和生成质量的同时，能够处理这两种情况。尽管反向动作在我们的训练数据中占比不足$1 ext{%}$，我们发现模型学会了反向、线性缩放其响应，并将油门与转向组合，所有这些都是通过学习基础的动作空间实现的。

View on arXiv Download PDF AI Translation

cs.CV / 10 / 2609.30609

MVAgent: Multi-Agent Video Generation via Consistent Condition Construction and Shot-Level Policy Optimization

MVAgent：通过一致条件构建和镜头级策略优化的多智能体视频生成

Kong, Xiangyu, Zhou, Wenjie, Tian, Fengping, Fang, Lihua, Sun, Haoqin, Lyu, Chenyang, Wang, Longyue, Luo, Weihua

Abstract

Multi-shot agentic video generation requires consistent character appearance, stable spatial layout across camera angles, and continuous character state between shots. When every shot is a separate request to a frozen generator, repeated text does not determine appearance, layout or state. We therefore recast the problem as condition construction and present MVAgent, a multi-agent pipeline whose agents collaborate through typed conditioning inputs. Because an environment image shows one viewpoint, a Spatial Grounding agent samples views from generated camera-traversal clips and anchors each shot to the view matching its framing. As generated shots drift from the plan, an Observer records how each shot ends in a continuity memory, from which a Transition agent builds character action and spatial references for the next shot. An Orchestrator composes these inputs into each request. Since a request reveals its effect only after rendering, we train it by agentic reinforcement learning with Trunk-GDPO, which compares rendered candidates at every shot rather than once per video and continues the best as the trunk. With generator and judges frozen, MVAgent attains the highest cross-shot consistency and narrative-planning quality among the compared methods on ViMax-Bench and is preferred over the strongest agentic baseline in human evaluation.

Chinese Translation

多镜头智能视频生成需要一致的人物外观、在不同摄像机角度下稳定的空间布局，以及镜头之间连续的人物状态。当每个镜头都是对一个静态生成器的独立请求时，重复的文本无法决定外观、布局或状态。因此，我们将问题重新表述为条件构建，并提出MVAgent，一个通过类型化条件输入进行协作的多智能体管道。由于环境图像展示的是一个视角，空间定位代理从生成的摄像机穿越剪辑中采样视图，并将每个镜头锚定到与其构图匹配的视图上。随着生成的镜头偏离计划，观察者记录每个镜头在连续性记忆中的结束状态，从中转移代理为下一个镜头构建人物动作和空间参考。协调者将这些输入组合成每个请求。由于请求仅在渲染后揭示其效果，我们通过智能体强化学习与Trunk-GDPO进行训练，该方法在每个镜头比较渲染候选，而不是每个视频一次，并继续将最佳者作为主干。在生成器和评估者被冻结的情况下，MVAgent在ViMax-Bench上实现了最高的跨镜头一致性和叙事规划质量，并在人工评估中优于最强的智能基线。

View on arXiv Download PDF AI Translation

cs.CV / 11 / 2609.30613

MedTokenBudget: Lesion-Preserving Token Routing for Dermoscopic Image Classification

MedTokenBudget：一种保留病变的标记路由方法用于皮肤镜图像分类

Li, Zhexiang

Abstract

Dermoscopy classifiers built on Vision Transformers process all image patches uniformly, although diagnostic evidence is concentrated in the lesion region. Existing token pruning methods reduce tokens using generic saliency or similarity signals, but rarely ask whether the retained subset still contains the lesion. This paper introduces MedTokenBudget, a supervised post-backbone token routing framework that learns to construct compact lesion-enriched representations when auxiliary lesion masks are available. Its Lesion-Aware Token Scoring (LATS) module fuses attention entropy, feature norm, and local feature contrast through a learned scorer, then routes the top-$K$ patches under a target budget. LATS is trained with budget curriculum learning, diversity regularization, attention distillation, and lesion-mask supervision. The trained router is evaluated with a lesion retention rate that directly measures how much ground-truth lesion evidence survives the token budget. On ISIC 2019, mask-supervised LATS consistently outperforms Random and ToMe at headline budgets while retaining substantially more lesion patches. Code is provided for reproducibility, and complete tabulated results are included in the supplementary material.

Chinese Translation

基于视觉变换器的皮肤镜分类器对所有图像块进行统一处理，尽管诊断证据主要集中在病变区域。现有的标记剪枝方法使用通用的显著性或相似性信号来减少标记，但很少考虑保留的子集是否仍然包含病变。本文提出了MedTokenBudget，这是一种监督式的后骨干网络标记路由框架，当辅助病变掩膜可用时，学习构建紧凑的富含病变的表示。其病变感知标记评分（Lesion-Aware Token Scoring, LATS）模块通过学习的评分器融合注意力熵、特征范数和局部特征对比，然后在目标预算下路由前$K$个图像块。LATS采用预算课程学习、多样性正则化、注意力蒸馏和病变掩膜监督进行训练。训练后的路由器通过病变保留率进行评估，该指标直接测量在标记预算下真实病变证据的保留程度。在ISIC 2019数据集上，掩膜监督的LATS在主要预算下始终优于随机方法和ToMe，同时保留了显著更多的病变图像块。我们提供了代码以便复现，并在补充材料中包含完整的表格结果。

View on arXiv Download PDF AI Translation

cs.CV / 12 / 2609.30647

Conditional Predictive Sufficient Statistics for Visual Representation Learning

用于视觉表征学习的条件预测充分统计量

Hong, Yuzhou

Abstract

A useful visual representation is a statistic of the observed past that retains the latent factors shared with the future and discards patch-private noise. We formalize this requirement as a conditional predictive sufficient statistic (CPSS). Under a shared-factor model of image patches, the mutual information between the past and the next patch equals the information the past carries about the shared factor, up to a remainder that the next patch itself fails to reveal. Predicting the next patch embedding with a cosine loss is maximum likelihood for a von Mises-Fisher model of that embedding's direction, and is therefore a tractable surrogate for the predictive information. The same population loss is also minimized by a constant embedding, so stop-gradient does not by itself select the sufficient statistic; it only blocks the symmetric gradient that implements the constant solution in one step. The regression target is a shallow embedding, which forces the network output back into that shallow range and leaves the sufficient statistic in intermediate blocks. Small causal Transformers on MNIST and CIFAR-10 are used as diagnostics, not as a leaderboard. On MNIST the future shift and the stop-gradient move probe accuracy by tens of points, and the CPSS readout peaks before the output. On CIFAR-10, with the same short budget and no augmentation, every objective lands near a linear classifier on pixels. What still matches the derivation is the geometry: the CPSS output is a worse readout than its best intermediate block, next-pixel regression does not pay that penalty, and removing the stop-gradient collapses the effective rank of the embedding even when the pretext loss looks perfect.

Chinese Translation

有用的视觉表征是对观察到的过去的统计量，它保留了与未来共享的潜在因素，并丢弃了局部私有噪声。我们将这一要求形式化为条件预测充分统计量（Conditional Predictive Sufficient Statistic, CPSS）。在图像块的共享因素模型下，过去与下一个图像块之间的互信息等于过去所携带的关于共享因素的信息，减去下一个图像块本身未能揭示的剩余部分。使用余弦损失预测下一个图像块的嵌入是该嵌入方向的冯·米塞斯-费舍尔（von Mises-Fisher）模型的最大似然估计，因此是预测信息的可处理替代。相同的人群损失也可以通过常量嵌入来最小化，因此停止梯度本身并不选择充分统计量；它仅阻止了在一步中实现常量解的对称梯度。回归目标是一个浅层嵌入，这迫使网络输出回到该浅层范围，并将充分统计量保留在中间块中。我们在 MNIST 和 CIFAR-10 上使用小型因果变换器作为诊断工具，而不是作为排行榜。在 MNIST 上，未来偏移和停止梯度移动使准确性提高了几十个点，CPSS 的读出在输出之前达到峰值。在 CIFAR-10 上，在相同的短预算和没有增强的情况下，每个目标都接近于像素上的线性分类器。与推导相匹配的仍然是几何特性：CPSS 输出的读出效果不如其最佳中间块，下一像素回归并未承担该惩罚，而去除停止梯度则在预文本损失看似完美的情况下，降低了嵌入的有效秩。

View on arXiv Download PDF AI Translation

cs.CV / 13 / 2609.30667

StarWM: Self-Supervised Trained Attention Routing for Robust World Models

StarWM：用于鲁棒世界模型的自监督训练注意力路由

Zhang, Zeqiang, Wurzberger, Fabian, Otte, Maximilian, Schmid, Daniel, Gottwald, Sebastian, Raulf, Arne Peter, Braun, Daniel Alexander

Abstract

A robust world model must strike the balance between faithfully capturing environmental dynamics and abstracting away from irrelevant content. While reconstruction-based world models ensure faithful supervision, they misallocate representational capacity by pixel area rather than dynamics relevance for visual tasks, which can cause task-irrelevant content to dominate the learned representation. Alternatively, reconstruction-free methods avoid this bias but risk discarding possibly relevant information. We propose StarWM, which uses a cross-attention module trained on self-supervised dynamics to decide where reconstruction applies. A dual-stream decoder then restricts reconstruction to the attended regions, with stop-gradient barriers preventing interference between the two objectives. These components allows reconstruction to supervise the visual content of attended regions without contaminating the latent with non-predictive information. On DeepMind Control with dynamic video backgrounds, default (reward-free) StarWM achieves the strongest performance under random-frame distractors and substantially outperforms reconstruction-based baselines under sequential video. In addition, its reward-augmented variant matches or exceeds reconstruction-free methods on sequential video, achieving the highest overall return across all distractor regimes. Mechanistic probing confirms StarWM preserves state attributes with near-perfect fidelity through long-horizon imagination while systematically discarding distractors.

Chinese Translation

一个鲁棒的世界模型必须在忠实捕捉环境动态与抽象掉无关内容之间取得平衡。虽然基于重建的世界模型确保了忠实的监督，但它们通过像素区域而非动态相关性来错误分配表示能力，这可能导致与任务无关的内容主导学习到的表示。相反，非重建方法避免了这种偏见，但可能会丢弃相关信息。我们提出了StarWM，它使用一个在自监督动态上训练的交叉注意力模块来决定重建的适用区域。一个双流解码器随后将重建限制在关注的区域内，停止梯度障碍防止两个目标之间的干扰。这些组件使得重建能够监督关注区域的视觉内容，而不会用非预测信息污染潜在表示。在具有动态视频背景的DeepMind Control上，默认（无奖励）StarWM在随机帧干扰下实现了最强性能，并在序列视频下显著超越基于重建的基线。此外，其奖励增强变体在序列视频上与非重建方法相匹配或超越，在所有干扰器条件下实现了最高的整体回报。机制探测确认StarWM通过长时间想象以近乎完美的保真度保持状态属性，同时系统性地丢弃干扰物。

View on arXiv Download PDF AI Translation

cs.CV / 14 / 2609.30682

Structure-Guided Masked Autoencoders for Ultra-High Resolution Scientific Image Understanding

结构引导的掩蔽自编码器用于超高分辨率科学图像理解

Zhang, Enzhi, Wu, Du, Zhong, Rui, Ma, Cong, Lyngaas, Isaac, Ziabari, Amir Koushyar, Wang, Xiao, Chen, Peng, Luo, Tao, Endo, Toshio, Shoji, Fumiyoshi, Sato, Kento, Uesugi, Kentaro, Nonoyama, Takayuki, Kiyama, Ryuji, Yoshida, Masahiro, Tezuka, Masaru, Ishikawa, Tetsuya, Matsuoka, Satoshi, Munetomo, Masaharu, Wahib, Mohamed

Abstract

Self-supervised pre-training with Vision Transformers, including Masked Autoencoders (MAE), is difficult to apply to gigapixel scientific images. Random masking is poorly matched to the structured, multi-scale morphology of scientific data, while uniform tokenization produces prohibitively long sequences that make $O(N^2)$ attention impractical. We propose SGMA, a structure-guided masked autoencoding framework for ultra-high-resolution scientific images. SGMA couples two components: a content-adaptive quadtree tokenizer that compresses gigapixel images into a fixed-length sequence, and a structure-conditioned masking process that biases reconstruction toward spatially informative regions. To stabilize this process across scales, we introduce Damped Accumulation (DA), which aggregates signal-dependent responses across the tree into a structure canvas used to guide masking. The resulting pre-training task preserves fine microstructure while remaining compatible with standard ViT encoders and MAE-style reconstruction. Across electron microscopy, whole-slide optical microscopy, and X-ray CT datasets, SGMA consistently outperforms MAE baselines. It achieves 95.68% Dice on the 8K x 8K x 28K SpringXCT dataset, improving over the same-architecture MAE baseline by +13.00 points, and 83.21% Dice on the 32K^2 WSI PAIP dataset, improving by +16.84 points, while providing up to a 24.8x inference speedup.

Chinese Translation

使用视觉变换器进行自监督预训练，包括掩蔽自编码器（Masked Autoencoders, MAE），在千兆像素科学图像上应用困难。随机掩蔽与科学数据的结构化、多尺度形态不匹配，而均匀标记化则产生过长的序列，使得 $O(N^2)$ 的注意力机制变得不切实际。我们提出了 SGMA，一种针对超高分辨率科学图像的结构引导掩蔽自编码框架。SGMA 结合了两个组件：一个内容自适应的四叉树标记器，将千兆像素图像压缩为固定长度的序列，以及一个结构条件的掩蔽过程，使重建偏向空间信息丰富的区域。为了在不同尺度上稳定这一过程，我们引入了阻尼累积（Damped Accumulation, DA），它将树中的信号依赖响应聚合到一个结构画布上，用于引导掩蔽。最终的预训练任务在保持细微微观结构的同时，仍与标准的 ViT 编码器和 MAE 风格的重建兼容。在电子显微镜、全切片光学显微镜和 X 射线 CT 数据集上，SGMA 始终优于 MAE 基线。在 8K x 8K x 28K SpringXCT 数据集上达到 95.68% 的 Dice，相较于同架构的 MAE 基线提高了 +13.00 分；在 32K^2 WSI PAIP 数据集上达到 83.21% 的 Dice，提高了 +16.84 分，同时提供高达 24.8 倍的推理加速。

View on arXiv Download PDF AI Translation

cs.CV / 15 / 2609.30698

MM-VeriAgent: Learning to Use Extensive Tools to Verify Multimodal Misinformation with Reinforcement Learning

MM-VeriAgent：通过强化学习学习使用广泛工具验证多模态虚假信息

Li, Peipei, Xia, Shuhan, Liu, Shengyang, Li, Zekun, He, Ran

Abstract

Real-world multimodal misinformation often involves mixed forgery sources, requiring sample-specific detection strategies. Existing tool-augmented methods rely on predefined workflows or inference-time planning, limiting adaptability or increasing inference cost. To address this issue, we introduce \textbf{MM-VeriAgent}, which learns to verify mixed-source multimodal misinformation with tools. We first build \textbf{MM-VeriTools}, a specialized toolkit for misinformation detection agents. By benchmarking various candidate models and methods on the sub-tasks required by mixed-source detection, we select the strongest for textual, visual, and cross-modal forgery analysis and encapsulate them as callable tools with a unified interface. On top of this toolkit, we train the LVLM agent with reinforcement learning to teach it how to use these tools to better solve mixed-source detection. Since many of the tools are specialized models whose online execution at every rollout severely limits RL efficiency, we further introduce \textbf{Tool-Execution Cache}, which pre-executes candidate tool calls and reuses their cached outputs during training. This preserves multi-step rollouts while reducing online tool execution, largely improving the training efficiency.Experiments on MMFakeBench demonstrate substantial accuracy gains over the base model without explicit tool search at inference time. Ablation and efficiency analyses further validate the learned tool-use policy and show that Tool-Execution Cache reduces online tool executions during training.

Chinese Translation

现实世界中的多模态虚假信息通常涉及混合伪造来源，需要特定样本的检测策略。现有的工具增强方法依赖于预定义的工作流程或推理时规划，限制了适应性或增加了推理成本。为了解决这个问题，我们提出了 extbf{MM-VeriAgent}，该方法学习如何使用工具验证混合来源的多模态虚假信息。我们首先构建了 extbf{MM-VeriTools}，这是一个专门为虚假信息检测代理设计的工具包。通过对混合来源检测所需的子任务进行各种候选模型和方法的基准测试，我们选择了在文本、视觉和跨模态伪造分析中表现最强的模型，并将其封装为具有统一接口的可调用工具。在这个工具包的基础上，我们使用强化学习训练 LVLM 代理，教其如何使用这些工具更好地解决混合来源检测问题。由于许多工具是专门的模型，其在每次回合中的在线执行严重限制了强化学习的效率，我们进一步引入了 extbf{Tool-Execution Cache}，该机制预先执行候选工具调用，并在训练期间重用其缓存输出。这保留了多步回合，同时减少了在线工具执行，极大地提高了训练效率。在 MMFakeBench 上的实验表明，与基础模型相比，准确性有显著提升，而无需在推理时进行显式工具搜索。消融和效率分析进一步验证了学习到的工具使用策略，并表明 Tool-Execution Cache 在训练过程中减少了在线工具执行。

View on arXiv Download PDF AI Translation

cs.CV / 16 / 2609.30703

SAGE: Source-Anchored Guidance via Frequency Equalization for Hierarchical RGB-T Alignment and Fusion

SAGE：通过频率均衡实现源锚定引导的层次RGB-T对齐与融合

Li, Timing, Sun, Yiming, Tao, Boan, Gao, Xiyuan, Cao, Haifang, Zhu, Pengfei

Abstract

Spatial misregistration and cross-modal discrepancies often cause ghosting, structural blurring, and content imbalance in RGB-T fusion. Existing methods typically decouple appearance adaptation, geometric alignment, and information fusion, limiting dependency propagation across stages. We propose Source-Anchored Guidance via Frequency Equalization for Hierarchical RGB-T Alignment and Fusion (SAGE), a unified framework integrating frequency equalization, hierarchical alignment, and subband fusion. SAGE employs invertible joint encoding and source-specific low-frequency modulation to derive structural and gain guidance while preserving source information. Hierarchical frequency collaborative alignment estimates global affine geometry from low-frequency approximations and transfers geometric and contextual cues to high-frequency correlation reasoning for reliability-aware residual refinement. Guided subband fusion jointly aggregates the aligned frequency coefficients under propagated source and alignment guidance, coordinates complementary low- and high-frequency information, and reconstructs the fused image through the inverse wavelet transform. Extensive experiments on RGB-T datasets with real-world and synthetic misalignments demonstrate consistently competitive performance in alignment and fusion, validating the effectiveness of source-anchored guidance for weakly registered RGB-T images.

Chinese Translation

空间错配和跨模态差异常常导致RGB-T融合中的重影、结构模糊和内容不平衡。现有方法通常将外观适应、几何对齐和信息融合解耦，限制了各阶段之间的依赖传播。我们提出了通过频率均衡实现源锚定引导的层次RGB-T对齐与融合框架（SAGE），这是一个集成频率均衡、层次对齐和子带融合的统一框架。SAGE采用可逆联合编码和源特定的低频调制来推导结构和增益引导，同时保留源信息。层次频率协同对齐从低频近似中估计全局仿射几何，并将几何和上下文线索转移到高频相关推理中，以实现可靠性意识的残差精细化。引导子带融合在传播的源和对齐引导下共同聚合对齐的频率系数，协调互补的低频和高频信息，并通过逆小波变换重建融合图像。在具有真实和合成错位的RGB-T数据集上进行的大量实验表明，在对齐和融合方面表现出一致的竞争力，验证了源锚定引导在弱对齐RGB-T图像中的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 17 / 2609.30708

Combining General and Domain-Specific Pretext Tasks for Brain MR Image Segmentation

结合通用和领域特定的前置任务用于脑部磁共振图像分割

Nasser, Tasneem, Schmid, Susanne, Souza, Roberto, El-Sheimy, Naser

Abstract

A key challenge in medical image analysis is the scarcity of large annotated datasets for specific populations and diseases. As deep learning models rely heavily on labeled data, effective transfer learning strategies are needed to reduce the dependence on manual annotations. Self-supervised learning has emerged as a promising approach for developing foundation models by enabling the learning of transferable feature representations from large-scale unlabeled medical imaging datasets. In this study, we investigate voxel-level brain age prediction as a domain-specific self-supervised pretext task and compare it with image inpainting, a widely used non-domain-specific alternative. We further propose a multitask self-supervised pretraining framework that jointly optimizes both objectives to learn complementary neuroimaging representations. The pretrained models are evaluated on three downstream magnetic resonance image segmentation tasks: multiple sclerosis lesion segmentation, ischemic stroke lesion segmentation, and cortical brain structure segmentation. Overall, the proposed multitask pretraining framework consistently outperformed the single-task pretrained models and training from scratch across most experimental settings, demonstrating the benefit of combining domain-specific and general self-supervised learning pretext tasks for the development of generalizable neuroimaging foundation models.\ Code Availability: The source code used in this study is publicly available at https://github.com/TasneemN/Combining-General-and-Domain-Specific-Pretext-Tasks-for-Brain-MR-Image-Segmentation/

Chinese Translation

医学图像分析中的一个关键挑战是特定人群和疾病的标注数据集稀缺。由于深度学习模型在很大程度上依赖于标记数据，因此需要有效的迁移学习策略以减少对手动标注的依赖。自监督学习作为一种有前景的方法，通过使得从大规模未标记的医学影像数据集中学习可迁移特征表示，已成为开发基础模型的有效途径。在本研究中，我们探讨了体素级脑龄预测作为一种领域特定的自监督前置任务，并将其与图像修复这一广泛使用的非领域特定替代方案进行了比较。我们进一步提出了一种多任务自监督预训练框架，该框架联合优化这两个目标，以学习互补的神经影像表示。预训练模型在三个下游磁共振图像分割任务上进行了评估：多发性硬化病灶分割、缺血性中风病灶分割和皮层脑结构分割。总体而言，所提出的多任务预训练框架在大多数实验设置中始终优于单任务预训练模型和从头开始训练，证明了结合领域特定和通用自监督学习前置任务对开发可推广的神经影像基础模型的益处。代码可用性：本研究中使用的源代码可在 https://github.com/TasneemN/Combining-General-and-Domain-Specific-Pretext-Tasks-for-Brain-MR-Image-Segmentation/ 上公开获取。

View on arXiv Download PDF AI Translation

cs.CV / 18 / 2609.30709

VLALight: Lightweight Vision-Language-Action Models for Emergency-Aware Traffic Signal Control

VLALight：用于应急感知交通信号控制的轻量级视觉-语言-动作模型

Jiang, Kemou, Wang, Maonan, Zou, Xingchen, Zhu, Jiayue, Fu, Yuhang, Wang, Sicheng, Chen, Xi, Chen, Yirong, Cui, Zhiyong

Abstract

Traffic signal control (TSC) is essential for mitigating urban congestion. Recent advances in vision-language models (VLMs) enable richer interpretation of intersection scenes, opening new opportunities for visual-context-aware TSC. However, the loose coupling and repeated information conversion between modules can lead to the loss of fine-grained visual details, while sequential inference introduces substantial latency. To address these limitations, we propose VLALight, a lightweight end-to-end vision-language-action framework that directly maps intersection observations and signal-phase information to discrete signal actions. To handle the multi-view nature of TSC, VLALight combines multiple directional camera views into a unified visual input and uses textual instructions to establish their correspondence with traffic movements and signal phases. This design enables direct action prediction with a compact 0.5 B-parameter model, without intermediate image-to-text descriptions or handcrafted traffic-state representations. Experiments show that VLALight delivers the best emergency-vehicle service of all compared methods, reducing pooled emergency waiting time by 21.1% over the cascaded VLMLight while running in real time on local hardware and generalizing to unseen intersection topologies and traffic-flow patterns.

Chinese Translation

交通信号控制（TSC）对于缓解城市拥堵至关重要。最近在视觉-语言模型（VLMs）方面的进展使得对交叉口场景的解读更加丰富，为视觉上下文感知的TSC开辟了新的机会。然而，模块之间的松散耦合和重复的信息转换可能导致细粒度视觉细节的丢失，而顺序推理则引入了显著的延迟。为了解决这些局限性，我们提出了VLALight，一个轻量级的端到端视觉-语言-动作框架，直接将交叉口观察和信号相位信息映射到离散的信号动作。为了处理TSC的多视角特性，VLALight将多个方向的摄像头视图合并为统一的视觉输入，并使用文本指令来建立它们与交通运动和信号相位之间的对应关系。这一设计使得在没有中间图像到文本描述或手工交通状态表示的情况下，能够通过一个紧凑的0.5 B参数模型直接进行动作预测。实验表明，VLALight在所有比较方法中提供了最佳的应急车辆服务，减少了21.1%的合并应急等待时间，相较于级联的VLMLight，同时在本地硬件上实时运行，并能够推广到未见过的交叉口拓扑和交通流模式。

View on arXiv Download PDF AI Translation

cs.CV / 19 / 2609.30722

TrafficImag: A Benchmark for Counterfactual Roadside Traffic Video Generation

TrafficImag：反事实路边交通视频生成的基准

Li, Xiangyu, Wang, Tianyi, Dou, Zhihao, Claudel, Christian, Guo, Zhaomiao

Abstract

Existing roadside traffic datasets support perception, forecasting, and visual question answering, but they do not evaluate counterfactual video generation, in which a selected actor is modified and the generated future should remain consistent with road topology and unrelated traffic. We introduce TrafficImag, the first benchmark for counterfactual roadside traffic video generation. TrafficImag combines a large-scale roadside dataset (9,022 annotated images, 7,043 deduplicated video clips, and 31,145 actor-centered history-future samples) with an executable protocol that supports behavior reasoning, intervention-aware image editing, and conditional video generation. Each intervention is represented as an actor-level program describing the target actor, intended behavior, legal route, interaction order, and temporal constraints, enabling a unified evaluation interface across heterogeneous foundation models. TrafficImag evaluates four complementary validity dimensions: initial-state correctness, route and behavior validity, interaction consistency, and non-target preservation, and considers an end-to-end counterfactual successful only when all four are satisfied. Across state-of-the-art foundation models, the strongest reasoner reaches 80.4% macro F1, the complete condition interface raises end-to-end success from 23.3% to 55.0% for the best generator. Oracle studies further show that conditional video execution is the primary remaining bottleneck. TrafficImag provides a reproducible benchmark for evaluating and diagnosing counterfactual traffic video generation beyond perceptual video quality.

Chinese Translation

现有的路边交通数据集支持感知、预测和视觉问答，但并未评估反事实视频生成，其中选定的参与者被修改，生成的未来应与道路拓扑和无关的交通保持一致。我们介绍了TrafficImag，这是第一个针对反事实路边交通视频生成的基准。TrafficImag结合了一个大规模的路边数据集（9,022张标注图像、7,043个去重视频片段和31,145个以参与者为中心的历史-未来样本），以及一个可执行的协议，支持行为推理、干预感知图像编辑和条件视频生成。每个干预被表示为一个参与者级别的程序，描述目标参与者、预期行为、合法路线、交互顺序和时间约束，从而实现跨异构基础模型的统一评估接口。TrafficImag评估四个互补的有效性维度：初始状态正确性、路线和行为有效性、交互一致性以及非目标保留，并认为仅当四个条件都得到满足时，端到端的反事实生成才算成功。在最先进的基础模型中，最强的推理器达到了80.4%的宏观F1，完整的条件接口将最佳生成器的端到端成功率从23.3%提高到55.0%。Oracle研究进一步表明，条件视频执行是主要的瓶颈。TrafficImag提供了一个可重复的基准，用于评估和诊断超越感知视频质量的反事实交通视频生成。

View on arXiv Download PDF AI Translation

cs.CV / 20 / 2609.30724

EviDETR: Preserving Query-Relevant Temporal Evidence for Moment Retrieval and Highlight Detection

EviDETR：保留查询相关的时间证据用于时刻检索和亮点检测

Sun, Haoran, Li, Yufan, Zhang, Qichen, Zhao, Haoran, Wang, Shuqi

Abstract

Joint video moment retrieval and highlight detection requires identifying query-relevant temporal segments while estimating clip-level saliency, yet DETR-style pipelines do not explicitly preserve query-relevant evidence throughout encoding, decoding, and cross-task prediction. We propose EviDETR, an evidence-preserving framework with three components. Semantic-aware Feature Reweighting (SFR) enhances query-relevant clip representations through saliency estimation and cross-modal interaction. A Temporal Top-2 Mixture-of-Experts (TTop2MoE) decoder performs query-adaptive refinement via sparse expert routing. MR-to-HD (MR2HD) fusion transfers span-level retrieval evidence to clip-level highlight prediction through confidence-weighted multi-scale aggregation. Using CLIP+SlowFast features, EviDETR achieves 69.29 [email protected], 54.77 [email protected], and 48.41 Avg. mAP for moment retrieval on QVHighlights, together with 41.83 HD-mAP and 68.33 HIT@1. Strong results on TACoS and Charades-STA further demonstrate cross-dataset transferability.

Chinese Translation

联合视频时刻检索和亮点检测需要识别与查询相关的时间段，同时估计片段级的显著性，然而，DETR风格的管道在编码、解码和跨任务预测过程中并未明确保留与查询相关的证据。我们提出了EviDETR，这是一种保留证据的框架，包含三个组件。语义感知特征重加权（SFR）通过显著性估计和跨模态交互增强与查询相关的片段表示。时间Top-2专家混合（TTop2MoE）解码器通过稀疏专家路由执行查询自适应的细化。MR到HD（MR2HD）融合通过置信加权的多尺度聚合将跨度级检索证据转移到片段级亮点预测。使用CLIP+SlowFast特征，EviDETR在QVHighlights上实现了69.29 [email protected]、54.77 [email protected]和48.41平均mAP的时刻检索成绩，同时获得41.83 HD-mAP和68.33 HIT@1的结果。在TACoS和Charades-STA上的强劲表现进一步证明了跨数据集的可迁移性。

View on arXiv Download PDF AI Translation

cs.CV / 21 / 2609.30728

Learning Polarization Image Restoration with General Restoration Priors

基于通用恢复先验的偏振图像恢复学习

Li, Chenggong, Liu, Jinhao, Wu, Caiyun, Luo, Yidong, Zhang, Junchao, Yang, Degui

Abstract

Polarization imaging captures distinctive surface and geometric cues that benefit a wide range of vision tasks. However, real-world polarization acquisition is often affected by multiple coupled degradations, making image restoration essential for practical polarization vision. Existing methods are largely tailored to specific degradations and remain constrained by the limited scale and quality of polarization data. To address these limitations, we develop an all-in-one polarization restoration framework for diverse and composite degradations. We first study the impact of different polarization representations on restoration performance and identify the normalized Stokes representation as an effective choice for separating intensity and polarization information. Accordingly, we devise a dual-branch architecture that separates intensity and polarization modeling. To overcome the limitations of polarization-specific training, the intensity branch leverages pretrained general restoration priors and a mixture-of-experts extension for composite degradations, while its restoration knowledge is adaptively distilled into the symmetric polarization branch via a cross-domain feature transform. In addition, we establish a composite-degradation polarization benchmark to support all-in-one restoration research. Extensive experiments on public datasets and our proposed benchmark demonstrate the effectiveness of the proposed method.

Chinese Translation

偏振成像捕捉到独特的表面和几何线索，能够为广泛的视觉任务提供帮助。然而，现实世界中的偏振获取通常受到多种耦合退化的影响，使得图像恢复在实际偏振视觉中变得至关重要。现有的方法大多针对特定的退化进行定制，并受到偏振数据规模和质量的限制。为了解决这些限制，我们开发了一种针对多样化和复合退化的全能偏振恢复框架。我们首先研究了不同偏振表示对恢复性能的影响，并确定归一化斯托克斯表示（normalized Stokes representation）是分离强度和偏振信息的有效选择。相应地，我们设计了一个双分支架构，分别处理强度和偏振建模。为了克服偏振特定训练的局限性，强度分支利用预训练的通用恢复先验和混合专家扩展来处理复合退化，同时其恢复知识通过跨域特征变换自适应地提炼到对称偏振分支中。此外，我们建立了一个复合退化偏振基准，以支持全能恢复研究。在公共数据集和我们提出的基准上的大量实验表明了所提方法的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 22 / 2609.30733

Amplify What You Gaze At: Target Saliency Boosting in Text-to-Image Generation

放大你所凝视的：文本到图像生成中的目标显著性增强

Dang, Shengqi, Yu, Zhengxi, Han, Feilin, Lan, Xingyu, Cao, Nan

Abstract

Text-to-image generation has advanced in controlling what, where, and how objects appear, yet how visual attention is distributed among objects remains largely unexplored. In this paper, we introduce Target Saliency Boosting, a new task aimed at boosting the visual saliency of a specific object during text-to-image generation without requiring any visual priors. Our key insight is that visual saliency is inherently relative: boosting the saliency of a target object also depends on the global saliency distribution across all objects in the scene. Based on this insight, we propose GazeME, a lightweight framework that uses saliency-marked prompts, inserting learnable marker tokens around object descriptions to indicate which objects to visually emphasize or suppress. To learn these markers, we construct a saliency-semantics dataset that associates objects in image--prompt pairs with object-level saliency scores, and propose Saliency Prior Marker Activation (SPMA), a saliency-aware stochastic marker activation strategy that exploits relative saliency relationships for robust training. During inference, GazeME automatically inserts appropriate markers into the prompt, thereby directly enhancing the visual saliency of the target object. Extensive experiments demonstrate that GazeME effectively boosts target saliency while preserving both semantic alignment and image quality.

Chinese Translation

文本到图像生成在控制物体的出现、位置和方式方面取得了进展，但物体之间的视觉注意力分布仍然在很大程度上未被探索。本文介绍了一种新的任务——目标显著性增强（Target Saliency Boosting），旨在增强文本到图像生成过程中特定物体的视觉显著性，而无需任何视觉先验。我们的关键见解是，视觉显著性本质上是相对的：增强目标物体的显著性还依赖于场景中所有物体的整体显著性分布。基于这一见解，我们提出了GazeME，一个轻量级框架，使用显著性标记提示，在物体描述周围插入可学习的标记令牌，以指示需要视觉强调或抑制的物体。为了学习这些标记，我们构建了一个显著性-语义数据集，将图像-提示对中的物体与物体级显著性分数关联，并提出了显著性先验标记激活（Saliency Prior Marker Activation, SPMA），这是一种显著性感知的随机标记激活策略，利用相对显著性关系进行稳健训练。在推理过程中，GazeME自动将适当的标记插入提示中，从而直接增强目标物体的视觉显著性。大量实验表明，GazeME有效地增强了目标显著性，同时保持了语义一致性和图像质量。

View on arXiv Download PDF AI Translation

cs.CV / 23 / 2609.30741

From Mono to Stereo: Accelerating Binocular Gaussian Splatting via Reprojection and Selective Patching

从单声道到立体声：通过重投影和选择性修补加速双目高斯点云渲染

Zhu, Hongfei, Zhou, Ling

Abstract

Binocular rendering requires two nearby views of the same scene and therefore repeats substantial visibility and shading work. We present a 2D Gaussian Splatting (2DGS) pipeline that fully renders a dominant-eye RGB image and an alpha-weighted depth proxy, reprojects that image to the affiliated eye, and repairs uncovered pixels. Small interior gaps are interpolated, whereas larger disoccluded regions are identified as regions of interest (ROIs) and selectively re-rendered. The depth proxy reuses the alpha-blending weights computed during dominant-eye rasterization, avoiding a separate depth-rendering pass. An adaptive ROI generator localizes the required updates using reprojected image boundaries and optional connected center-hole detection. On DTU, Tanks and Temples, and MipNeRF-360, the method reduces the measured time of a sequential two-pass binocular reference by 15.5\% to 28.8\% and peak GPU memory by 6\% to 11\%. The corresponding affiliated-eye quality degradation is at most 1.3 dB PSNR, 0.02 SSIM, and 0.02 LPIPS, representing a measurable trade-off that requires application-specific perceptual validation. These results establish a practical efficiency-quality trade-off for controlled static-scene stereo rendering and motivate future evaluation under continuous motion and on physical VR hardware.

Chinese Translation

双目渲染需要对同一场景进行两个相邻视图的渲染，因此重复了大量的可见性和阴影计算工作。我们提出了一种二维高斯点云渲染（2D Gaussian Splatting, 2DGS）管道，该管道完全渲染主视眼的RGB图像和加权透明度深度代理，将该图像重投影到关联的视眼，并修复未覆盖的像素。小的内部间隙通过插值进行填补，而较大的遮挡区域被识别为感兴趣区域（Regions of Interest, ROIs）并选择性地重新渲染。深度代理重用了在主视眼光栅化过程中计算的透明度混合权重，避免了单独的深度渲染过程。自适应ROI生成器利用重投影图像边界和可选的连接中心孔检测来定位所需的更新。在DTU、Tanks and Temples和MipNeRF-360数据集上，该方法将顺序双通道参考的测量时间减少了15.5 ext{%}至28.8 ext{%}，峰值GPU内存减少了6 ext{%}至11 ext{%}。相应的关联视眼质量下降最多为1.3 dB PSNR、0.02 SSIM和0.02 LPIPS，代表了一种可测量的权衡，需要特定应用的感知验证。这些结果为受控静态场景的立体渲染建立了一个实用的效率与质量权衡，并激励未来在连续运动和物理虚拟现实硬件下的评估。

View on arXiv Download PDF AI Translation

cs.CV / 24 / 2609.30755

Training-Free Bottleneck Width Planning for Convolutional Autoencoders

无训练的卷积自编码器瓶颈宽度规划

Guo, Guannan

Abstract

Multiscale Spectral Rate-Distortion (MS-SRD) estimates the bottleneck channels required at user-supplied spatial cuts from training images and a normalized mean-squared error (NMSE) bound, without fitting a neural network. Its covariance-tail rule is exact for shared linear block-convolutional autoencoders under squared error. A nested-scale dominance result motivates reporting the activation-parameter Pareto frontier alongside the minimum-latent candidate. At NMSE <= 0.01 on thirteen grayscale datasets, its latent-size prediction has 0.84% mean absolute percentage error against nonlinear patch-autoencoder boundaries; ten predictions are exact and the remaining three differ by one channel. In a four-dataset deployable comparison, MS-SRD matches all retrospective external widths and all four selected models pass, without training a selector; a 46-fit validation grid and four Least-Volume fits each pass on two datasets. In a skip-closed U-shaped autoencoder at the same bound, five predictions are exact, nine are within one channel, and every failing prediction is one channel short. Experiments at looser bounds show progressively larger nonlinear savings.

Chinese Translation

多尺度谱率失真（MS-SRD）在不拟合神经网络的情况下，根据用户提供的空间切割和归一化均方误差（NMSE）界限，估算所需的瓶颈通道。其协方差尾规则在平方误差下对共享线性块卷积自编码器是精确的。嵌套尺度主导结果促使我们在报告最小潜在候选的同时，展示激活参数的帕累托前沿。在十三个灰度数据集上，当NMSE <= 0.01时，其潜在大小预测相对于非线性补丁自编码器边界的平均绝对百分比误差为0.84%；其中十个预测完全准确，其余三个仅相差一个通道。在一个可部署的四数据集比较中，MS-SRD与所有回顾性外部宽度匹配，且所有四个选定模型均通过，无需训练选择器；一个46拟合的验证网格和四个最小体积拟合在两个数据集上均通过。在相同界限下的跳跃闭合U形自编码器中，五个预测完全准确，九个在一个通道内，每个失败的预测均短缺一个通道。在更宽松的界限下进行的实验显示出逐渐增大的非线性节省。

View on arXiv Download PDF AI Translation

cs.CV / 25 / 2609.30758

LLPR: Location-aware learning and physics-based reconstruction for raindrop removal from a single image

LLPR：基于位置感知学习和物理重建的单幅图像雨滴去除方法

He, Zewei, Liu, Xingyu, Luo, Xing, Fu, Guizhong, Chen, Zixuan, Chen, Yu, Li, Jinlei, Lu, Zhe-Ming

Abstract

Raindrops can cause occlusion and distortion in the background scenes due to their adherence to windows or camera lenses. Existing raindrop removal methods concentrate on designing sophisticated CNN or Transformer architectures to recover distorted and missing texture. In this paper, we try to integrate location information and physical model into off-the-shelf CNN or Transformer architectures to help improve their performance. Specifically, we notice that existing methods deploy a preprocessing sub-network to generate a binary or soft mask to indicate the raindrop location, which will increase the network parameters and computational complexity. In contrast, a location-aware learning branch is embedded to teach the encoder in the training phase with the capability of perceiving the position of the raindrops. Note that this location-aware learning branch can be removed during the inference process (achieving performance improvements at no cost). Furthermore, instead of directly reconstructing the raindrop-free image (i.e., background scene), we devise a physics-based reconstruction scheme to first learn the transparency matrix and the raindrop layer. The latent background layer is then reversely derived based on the physical model. By combining the above-mentioned components, we propose our location-aware learning and physics-based reconstruction (LLPR) framework for this challenging ill-posed problem. We also collect a real-world raindrop-degraded image dataset, which is challenging for single-image raindrop removal (SIRR) methods. Extensive experimental results demonstrate the effectiveness and generality of our LLPR framework, achieving superior performance against state-of-the-art SIRR methods. The code will be made available upon acceptance.

Chinese Translation

雨滴会因附着在窗户或相机镜头上而导致背景场景的遮挡和扭曲。现有的雨滴去除方法主要集中在设计复杂的卷积神经网络（CNN）或变换器（Transformer）架构，以恢复扭曲和缺失的纹理。本文尝试将位置信息和物理模型整合到现成的CNN或Transformer架构中，以帮助提高其性能。具体而言，我们注意到现有方法部署了一个预处理子网络，以生成二进制或软掩膜来指示雨滴位置，这将增加网络参数和计算复杂度。相反，我们嵌入了一个位置感知学习分支，在训练阶段教导编码器感知雨滴位置的能力。需要注意的是，这个位置感知学习分支可以在推理过程中移除（在不增加成本的情况下实现性能提升）。此外，我们并不直接重建无雨滴的图像（即背景场景），而是设计了一种基于物理的重建方案，首先学习透明度矩阵和雨滴层。然后，基于物理模型反向推导潜在的背景层。通过结合上述组件，我们提出了位置感知学习和基于物理重建（LLPR）框架，以应对这一具有挑战性的病态问题。我们还收集了一个真实世界的雨滴降解图像数据集，这对单幅图像雨滴去除（SIRR）方法来说具有挑战性。大量实验结果证明了我们LLPR框架的有效性和普适性，在与最先进的SIRR方法相比时表现出色。代码将在接受后提供。

View on arXiv Download PDF AI Translation

cs.CV / 26 / 2609.30761

Timo: $\textbf{T}$aming Mult$\textbf{i}$modal Diffusion Transformer for Human $\textbf{Mo}$tion Generation

Timo：驯服多模态扩散变换器（Multimodal Diffusion Transformer）用于人体动作生成

Wang, Zhao, Hu, Jiangtao, Yu, Jack, Yu, Tao

Abstract

Most existing human motion generation (HMG) methods use cross-attention modules to inject text semantics, but ignore the importance of bidirectional modeling between motion and text tokens, which limits text comprehension. A straightforward idea is introducing multimodal diffusion transformers (MMDiT), which have shown effective joint text--visual modeling in vision generation, into HMG. However, we find that articulated motion is temporally coherent but weakly correlated across joints, in which directly applying an MMDiT with flow matching produces poorly coordinated and jerky motion. In this work, we propose Timo, a novel kinematics-aware MMDiT framework tailored for HMG. Timo combines fully shared multimodal attention for bidirectional text--motion modeling with flow matching, geometric and rotational-kinematics supervision that compares actual rotations and their changes over time, and a two-stage curriculum progressing from broad motion learning to detailed caption alignment. Further, we construct a benchmark of $40{,}025$ held-out clips from six public datasets spanning diverse actions, assessing six complementary dimensions under a common evaluator and scoring protocol. Our model substantially outperforms state-of-the-art methods in both quantitative and qualitative evaluations. Remarkably, Timo surpasses Kimodo on five of six dimensions, achieving a $40.8$% relative improvement in the average benchmark score. Project page: https://kyfafyd.wang/projects/timo. Demo page: https://timo.kyfafyd.wang.

Chinese Translation

现有大多数人体动作生成（HMG）方法采用交叉注意力模块注入文本语义，但忽视了动作与文本标记之间双向建模的重要性，限制了文本理解能力。一种直接的思路是引入多模态扩散变换器（Multimodal Diffusion Transformers, MMDiT），该模型已在视觉生成中展现出有效的文本-视觉联合建模能力，应用于人体动作生成。然而，我们发现关节动作在时间上具有连贯性，但关节间相关性较弱，直接应用基于流匹配的MMDiT会产生协调性差且动作生硬的结果。本文提出了Timo，一种针对人体动作生成设计的新型运动学感知MMDiT框架。Timo结合了完全共享的多模态注意力机制以实现文本与动作的双向建模，配合流匹配、几何及旋转运动学监督，后者通过比较实际旋转及其随时间的变化进行约束，并采用两阶段课程学习策略，从广泛动作学习逐步过渡到细致的文本对齐。此外，我们构建了一个包含来自六个公开数据集共计40,025个保留测试片段的基准，涵盖多样动作类型，并在统一评估器和评分协议下，从六个互补维度进行评估。实验结果表明，本模型在定量和定性评估中均显著优于现有最先进方法。值得注意的是，Timo在六个维度中有五个超越了Kimodo，平均基准得分实现了40.8%的相对提升。项目主页：https://kyfafyd.wang/projects/timo。演示主页：https://timo.kyfafyd.wang。

View on arXiv Download PDF AI Translation

cs.CV / 27 / 2609.30769

Query-Conditioned Prototype Adaptation for Cross-Domain Few-Shot Learning: Single-Query Inference, Controlled Comparisons, and Failure Modes

基于查询条件的原型适应用于跨域少样本学习：单查询推理、受控比较与失败模式

Karania, Rushab Rasik, Maul, Tomas

Abstract

Cross-domain few-shot learning requires adapting a classifier to a new visual domain from very few labelled examples without target-time parameter updates. We isolate one question: under a fixed global representation, what does joint query-support adaptation contribute to prototype construction? The Within-Instance Prototypical Transformer (WIPT) implements single-query test-time prototype adaptation by jointly transforming one unlabelled query and the labelled support embeddings, then forming query-specific class means. Using a shared frozen ViT-S/16 encoder, miniImageNet source training, and CUB, EuroSAT and ISIC targets, we replicate the key comparisons across five independent training seeds. In 1-shot evaluation, WIPT improves frozen ProtoNet in every run on CUB (+0.21 percentage points) and EuroSAT (+2.07), but decreases ISIC (-0.22). In 5-shot evaluation, ProtoNet remains strongest overall, while WIPT consistently improves a capacity-matched support-only Transformer on ISIC (+0.99). Joint processing of up to five queries yields no reliable accuracy gain; in a head-only 5-shot benchmark, g = 5 reduces analytical attention-token pairs by 73% and peak allocated memory by 29% relative to g = 1, although latency is non-monotonic. Across all target/shot conditions, WIPT changes uncertain ProtoNet decisions far more than confident ones, and rescue/break decomposition accounts for the observed gains and losses. Source-shift and scorer controls further show that the benefit is not universal. Overall, WIPT provides a streaming-compatible form of test-time prototype adaptation that can improve difficult low-shot cross-domain decisions without target-time optimization.

Chinese Translation

跨域少样本学习需要在没有目标时间参数更新的情况下，将分类器适应于新的视觉领域，且仅依赖于极少的标记示例。我们聚焦于一个问题：在固定的全局表示下，联合查询-支持适应对原型构建有什么贡献？内部实例原型变换器（Within-Instance Prototypical Transformer, WIPT）通过联合变换一个未标记的查询和标记的支持嵌入，实现了单查询测试时间的原型适应，从而形成查询特定的类别均值。使用共享的冻结 ViT-S/16 编码器、miniImageNet 源训练以及 CUB、EuroSAT 和 ISIC 目标，我们在五个独立的训练种子上复制了关键比较。在 1-shot 评估中，WIPT 在每次运行中都在 CUB (+0.21 个百分点) 和 EuroSAT (+2.07) 上改善了冻结的 ProtoNet，但在 ISIC 上有所下降 (-0.22)。在 5-shot 评估中，ProtoNet 的整体表现仍然最强，而 WIPT 在 ISIC 上持续改善了与容量匹配的仅支持变换器 (+0.99)。对多达五个查询的联合处理未能带来可靠的准确性提升；在一个仅头部的 5-shot 基准测试中，g = 5 相较于 g = 1 将分析注意力令牌对减少了 73%，峰值分配内存减少了 29%，尽管延迟表现出非单调性。在所有目标/样本条件下，WIPT 对不确定的 ProtoNet 决策的改变远大于对自信决策的改变，救援/破坏分解解释了观察到的增益和损失。源偏移和评分控制进一步表明，这一好处并非普遍存在。总体而言，WIPT 提供了一种与流媒体兼容的测试时间原型适应形式，可以在没有目标时间优化的情况下改善困难的低样本跨域决策。

View on arXiv Download PDF AI Translation

cs.CV / 28 / 2609.30783

Skip the Talk, Re-Focus on Vision: Latent Reasoning for Reasoning Segmentation in Multimodal Large Language Models

跳过对话，重新聚焦于视觉：多模态大语言模型中的潜在推理与推理分割

Guo, Tianhang, He, Yulin, Chen, Wei, Zhou, Wenjuan, Li, Yuhang, Gan, Xinbiao

Abstract

Reasoning segmentation aims to interpret implicit textual queries and enable fine-grained visual perception, which is critical for applications such as human-computer interaction and embodied agents. Existing methods typically generate explicit Chain-of-Thought (CoT) by multimodal large language models (MLLMs) before localizing the target. Although intuitive, such explicit verbal reasoning introduces substantial attention interference: redundant textual tokens disrupt attention during perception-token generation and also increase the effective distance between visual tokens. To address this issue, we propose LIRSeg, which fully replaces explicit CoT with a compact set of learnable latent tokens for reasoning segmentation. LIRSeg is trained in two stages: spatial alignment grounds the latent tokens in object-relevant visual evidence, and GRPO further optimizes them with segmentation rewards. To make these compact latent tokens more informative, we introduce three complementary mechanisms from an information perspective: extreme-advantage sampling for selecting informative training signals, decoupled exploration-stability updates for learning complementary representations, and latent diversity amplification for preventing representational collapse. Extensive experiments on benchmarks demonstrate that LIRSeg consistently improves both segmentation accuracy and reasoning efficiency. Compared with the VisionReasoner baseline, LIRSeg achieves absolute gIoU improvements of 4.9% on ReasonSeg, 7.1% on MUSE, and 4.7% on MMR, while achieving a approximately 16x reduction in reasoning tokens. Code is available in supplementary materials.

Chinese Translation

推理分割旨在解释隐含的文本查询，并实现细粒度的视觉感知，这对于人机交互和具身智能体等应用至关重要。现有方法通常通过多模态大语言模型（MLLMs）生成显式的思维链（Chain-of-Thought, CoT），然后再定位目标。尽管这种方法直观，但显式的语言推理引入了显著的注意力干扰：冗余的文本标记在感知标记生成过程中干扰注意力，并且增加了视觉标记之间的有效距离。为了解决这个问题，我们提出了LIRSeg，它完全用一组紧凑的可学习潜在标记替代显式的CoT进行推理分割。LIRSeg分为两个阶段进行训练：空间对齐将潜在标记与对象相关的视觉证据进行关联，而GRPO进一步通过分割奖励优化这些标记。为了使这些紧凑的潜在标记更具信息量，我们从信息角度引入了三种互补机制：极端优势采样用于选择信息丰富的训练信号，解耦的探索-稳定更新用于学习互补表示，以及潜在多样性增强以防止表示崩溃。在基准测试上的大量实验表明，LIRSeg在分割准确性和推理效率上均有持续改善。与VisionReasoner基线相比，LIRSeg在ReasonSeg上实现了4.9%的绝对gIoU提升，在MUSE上提升7.1%，在MMR上提升4.7%，同时推理标记数量减少了约16倍。代码可在补充材料中获得。

View on arXiv Download PDF AI Translation

cs.CV / 29 / 2609.30795

Motion Style Slider: Endpoint-Supervised Continuous Style Control for Human Motion Diffusion

运动风格滑块：端点监督的连续风格控制用于人类运动扩散

Liao, Chen-Chieh, Peng, Yichen, Cai, Yiyi, Ono, Yûi, Hanaoka, Hiroki, Wu, Erwin, Koike, Hideki, Kurabayashi, Shuichi

Abstract

Existing human motion diffusion methods provide strong motion generation quality, and recent style transfer models can inject target style cues, but fine-grained continuous control of style intensity remains underexplored. In production, style intensity is subjective across artists and directors, so the practical requirement is not a universal absolute unit, but a reliable monotonic control axis. We propose Motion Style Slider, a motion-to-motion style transfer framework for endpoint-supervised continuous control. Given a content motion and a style motion, we construct a style direction in a learned motion-style embedding space and condition diffusion generation with a scalar intensity. The training objective combines diffusion denoising with latent intensity regularization to encourage smooth and monotonic style scaling without requiring intermediate-intensity ground-truth motions. Our framework is compatible with pretrained motion diffusion backbones and supports heterogeneous style datasets, including the multi-actor style motion dataset. To test out-of-range usability, we additionally introduce a small real-capture over-reaction extension and evaluate large-intensity behavior against these unseen targets. Experiments measure controllability, interpolation/extrapolation behavior, content preservation, and motion realism, with ablations on direction construction and loss design.

Chinese Translation

现有的人类运动扩散方法提供了强大的运动生成质量，而最近的风格迁移模型能够注入目标风格线索，但对风格强度的细粒度连续控制仍然未得到充分探索。在实际应用中，风格强度在艺术家和导演之间是主观的，因此实际需求不是一个普遍的绝对单位，而是一个可靠的单调控制轴。我们提出了运动风格滑块（Motion Style Slider），这是一个用于端点监督连续控制的运动到运动风格迁移框架。给定一个内容运动和一个风格运动，我们在学习到的运动-风格嵌入空间中构建一个风格方向，并用标量强度条件化扩散生成。训练目标结合了扩散去噪与潜在强度正则化，以鼓励平滑和单调的风格缩放，而无需中间强度的真实运动。我们的框架与预训练的运动扩散主干兼容，并支持异构风格数据集，包括多演员风格运动数据集。为了测试超范围的可用性，我们还引入了一个小型真实捕捉的过度反应扩展，并评估对这些未见目标的大强度行为。实验测量了可控性、插值/外推行为、内容保留和运动真实感，并对方向构建和损失设计进行了消融实验。

View on arXiv Download PDF AI Translation

cs.CV / 30 / 2609.30855

MDSkin-Net: Multi-Task Skin Lesion Analysis Driven by Pattern Analysis Priors and Spatial Alignment Regularization

MDSkin-Net：基于模式分析先验和空间对齐正则化的多任务皮肤病变分析

Li, Yijian, Bedros, Saad, Bigliardi, Paul, Qi, Mei Bigliardi, Morellas, Vassilios, Papanikolopoulos, Nikolaos

Abstract

Reliable skin lesion segmentation and classification are central to dermoscopic computer-aided diagnosis. Existing multi-task frameworks couple the two tasks architecturally without clinical knowledge, while knowledge-injecting approaches rely on the macroscopic ABCD rule, which was not designed for dermoscopy. Dermoscopic diagnosis is grounded in Pattern Analysis, a microscopic framework structured around dermoscopic features. We propose MDSkin-Net, which incorporates cue-level Pattern Analysis priors into a hybrid CNN-Transformer architecture. At its core is a Pattern Analysis-Guided Attention Module (PAGAM) comprising three priors motivated by distinct dermoscopic cues: an improved Efficient Channel Attention (iECA), a Multi-Scale Spatial Attention (MSSA), and a Biased Asymmetry Attention (BAA). We further introduce a multi-scale spatial alignment regularization (MSAR) that uses the segmentation ground-truth mask as hierarchical soft supervision, confining the classification head to lesion-localized evidence and coupling both task pathways through a shared spatial prior. Trained exclusively on the ISIC 2017 training split without external dermoscopy data, the MDSkin-Net ensemble transfers robustly under zero-shot evaluation, reaching a Dice Similarity Coefficient (DSC) of 92.38% and a melanoma AUC of 97.84%on PH2, and a DSC of 88.92% on the ISIC 2018 Task 1 test set. On the in-domain ISIC 2017 benchmark, the ensemble attains a mean Area Under the Curve (AUC) of 91.60% across the two classification tasks (melanoma and seborrheic keratosis vs. rest), and a DSC of 84.72% for segmentation. Classification remains competitive with baselines; in-domain segmentation trails single-task specialists, yet the proposed priors and alignment regularization yield representations that generalize consistently across cohorts of different scales.

Chinese Translation

可靠的皮肤病变分割和分类是皮肤镜计算机辅助诊断的核心。现有的多任务框架在架构上将这两个任务结合在一起，但缺乏临床知识，而知识注入的方法依赖于宏观的ABCD规则，这并不是为皮肤镜检查设计的。皮肤镜诊断基于模式分析，这是一种围绕皮肤镜特征构建的微观框架。我们提出了MDSkin-Net，它将线索级的模式分析先验融入到混合CNN-Transformer架构中。其核心是一个模式分析引导的注意力模块（Pattern Analysis-Guided Attention Module, PAGAM），由三个基于不同皮肤镜线索的先验组成：改进的高效通道注意力（Improved Efficient Channel Attention, iECA）、多尺度空间注意力（Multi-Scale Spatial Attention, MSSA）和偏置不对称注意力（Biased Asymmetry Attention, BAA）。我们进一步引入了一种多尺度空间对齐正则化（Multi-Scale Spatial Alignment Regularization, MSAR），利用分割真实标签掩膜作为分层软监督，限制分类头仅关注于病变局部证据，并通过共享的空间先验将两个任务路径结合起来。MDSkin-Net集成模型在没有外部皮肤镜数据的情况下，仅在ISIC 2017训练集上进行训练，在零样本评估中表现出色，达到了92.38%的Dice相似系数（Dice Similarity Coefficient, DSC）和97.84%的黑色素瘤AUC（Area Under the Curve）在PH2数据集上，以及在ISIC 2018任务1测试集上达到了88.92%的DSC。在领域内的ISIC 2017基准测试中，该集成模型在两个分类任务（黑色素瘤和脂溢性角化病与其他病变）上获得了91.60%的平均AUC，并在分割任务上达到了84.72%的DSC。分类性能与基线模型竞争力强；在领域内的分割任务上，尽管落后于单任务专家，但所提先验和对齐正则化产生的表示在不同规模的群体中具有一致的泛化能力。

View on arXiv Download PDF AI Translation

cs.CV / 31 / 2609.30865

Reliability-Regulated Trajectory Optimization for Progressive COLMAP-Free 3D Gaussian Splatting

基于可靠性调节的渐进式COLMAP-free 3D高斯点云轨迹优化

Wu, Zijian, Wang, Jinliang, Lin, Zidian, Song, Ying, Lu, Ziqian, Ma, Hanjie, Ye, Zhen, Jiang, Mingfeng

Abstract

COLMAP-free 3D Gaussian Splatting (3DGS) bypasses computationally expensive structure-from-motion (SfM) pipelines, yet progressive camera pose tracking remains fundamentally vulnerable to error compounding---early pairwise tracking inaccuracies both corrupt subsequent frame initializations and remain permanently frozen in the scene representation. Rather than relying on heavyweight external neural priors or treating progressive tracking through isolated heuristic fixes, we propose a unified reliability-regulated trajectory optimization framework for progressive COLMAP-free 3DGS. At its core, our framework establishes an intrinsic, self-supervised bidirectional cycle-consistency mechanism that systematically regulates progressive camera trajectory estimation across two complementary temporal horizons: (1) Forward Motion Propagation, where the online reliability signal adaptively gates first-order kinematic warm-starts of rigid motion into upcoming pairwise registrations, supplying informed directional search priors while safely intercepting untrusted transitions; and (2) Retrospective Trajectory Correction, where the same reliability signal dynamically weights relative-pose consistency constraints within a sliding window of neighboring camera poses. By governing both prospective state initialization and retrospective trajectory consolidation through a unified reliability regulator, our self-contained framework resolves progressive drift without external priors or offline preprocessing. Extensive evaluations on Tanks and Temples and CO3D-V2 benchmarks show that our method substantially improves camera trajectory accuracy and novel-view rendering quality, outperforming existing unposed baselines. Code is available at https://github.com/Zijian1026/RRTO-CF3DGS.

Chinese Translation

COLMAP-free 3D高斯点云（3DGS）绕过了计算成本高昂的运动重建（SfM）流程，但渐进式相机姿态跟踪在根本上仍然容易受到误差累积的影响——早期的成对跟踪不准确不仅会破坏后续帧的初始化，而且会在场景表示中永久冻结。我们提出了一种统一的基于可靠性调节的渐进式COLMAP-free 3DGS轨迹优化框架，而不是依赖于重量级的外部神经先验或通过孤立的启发式修复来处理渐进式跟踪。我们的框架核心建立了一种内在的、自我监督的双向循环一致性机制，系统地调节跨越两个互补时间视域的渐进式相机轨迹估计：（1）前向运动传播，在此过程中，在线可靠性信号自适应地控制刚性运动的一级运动热启动进入即将到来的成对配准，提供有信息的方向搜索先验，同时安全地拦截不可信的过渡；（2）回顾性轨迹修正，在此过程中，相同的可靠性信号动态加权相对姿态一致性约束，在相邻相机姿态的滑动窗口内进行处理。通过统一的可靠性调节器来管理前瞻性状态初始化和回顾性轨迹整合，我们的自包含框架在没有外部先验或离线预处理的情况下解决了渐进漂移。在Tanks and Temples和CO3D-V2基准上的广泛评估表明，我们的方法显著提高了相机轨迹的准确性和新视图渲染质量，超越了现有的无姿态基线。代码可在 https://github.com/Zijian1026/RRTO-CF3DGS 获取。

View on arXiv Download PDF AI Translation

cs.CV / 32 / 2609.30928

UltraG-Bench: A Multi-task Benchmark for assessing Large Vision-Language Models on Pixel-level Evidence Grounding in Ultrasound

UltraG-Bench：评估大型视觉-语言模型在超声波图像像素级证据基础上的多任务基准

Zhu, Quanhao, Xu, Bo, Lin, Rui, Wang, Chenyuan, Shao, Yu, Zhu, Boling, Sun, Jiuyan, Zhao, Liang, Lin, Hongfei, Xia, Feng

Abstract

Ultrasound is one of the most widely used medical imaging modalities, and recent large vision-language models(VLMs) have shown increasing capabilities in ultrasound image understanding. However, these models fail to provide pixel-level visual evidence aligned with their semantic predictions, and their fine-grained grounding capability in ultrasound remains largely unclear. We introduce UltraG-Bench, a large-scale multi-task benchmark for evaluating pixel-level evidence grounding in ultrasound. UltraG-Bench is built by annotating 40 public ultrasound segmentation datasets spanning 13 anatomical categories, and comprises three progressive tasks: instruction-guided segmentation, evidence-grounded VQA, and evidence-grounded report generation, with 331125, 666779, and 138832 annotations, respectively. Comprehensive evaluation of 14 state-of-the-art models reveals a substantial gap between semantic understanding and fine-grained pixel-level localization. We further propose UltraG-Agent, which combines the semantic reasoning capabilities of a VLM with the ultrasound-specific segmentation capability of UltraSAM3. Experiments show that UltraG-Agent substantially improves both semantic prediction and pixel-level visual grounding. Our dataset and code are available at https://github.com/zhuqh19/UltraG-Bench.

Chinese Translation

超声波是最广泛使用的医学成像方式之一，最近的大型视觉-语言模型（VLMs）在超声图像理解方面表现出越来越强的能力。然而，这些模型未能提供与其语义预测对齐的像素级视觉证据，并且它们在超声波中的细粒度定位能力仍然不清楚。我们引入了UltraG-Bench，这是一个用于评估超声波像素级证据基础的大规模多任务基准。UltraG-Bench通过对40个公共超声分割数据集进行注释而构建，涵盖13个解剖类别，并包含三个渐进式任务：指导性分割、基于证据的视觉问答（VQA）和基于证据的报告生成，分别有331125、666779和138832个注释。对14个最先进模型的全面评估揭示了语义理解与细粒度像素级定位之间的显著差距。我们进一步提出了UltraG-Agent，它结合了VLM的语义推理能力与UltraSAM3的超声特定分割能力。实验表明，UltraG-Agent显著改善了语义预测和像素级视觉定位。我们的数据集和代码可在https://github.com/zhuqh19/UltraG-Bench获取。

View on arXiv Download PDF AI Translation

cs.CV / 33 / 2609.30934

ManiVid: Unified and Explainable Forensic Analysis of Manipulated Videos

ManiVid：统一且可解释的操纵视频法医分析

Kang, Hengrui, Yan, Zhonghao, Yang, Yuxuan, Jing, Ruoyan, Guo, Yuncheng, Chen, Hao, Liang, Kongming, Ma, Zhanyu, He, Conghui, Li, Weijia

Abstract

Rapid advances in AI-generated video (AIGV) have increased the risks posed by deceptive video manipulation. Unlike fully synthetic videos, manipulated videos retain most source content and alter only localized regions, making forensic analysis particularly challenging. Existing video forgery research faces two limitations in both data and methodology: (1) High-quality datasets and benchmarks tailored for manipulated videos remain scarce. (2) Multimodal large language models (MLLMs) extend forgery analysis beyond binary classification but struggle to use low-level forensic cues and provide precise pixel-level grounding. Specifically, we introduce ManiVid, a unified forensic analysis task covering forgery detection, artifact grounding, and anomaly explanation for manipulated videos. We construct ManiVid-38K, the first dataset to combine paired, open-vocabulary localized manipulations of general videos with authenticity labels, forgery masks, and anomaly explanations. It comprises about 19K manually verified real-fake video pairs, mostly at 1080P resolution, generated under 2 paradigms with 15 powerful generation models. We sample 1K pairs for ManiVidBench, balanced across six manipulation types and generation models for fair evaluation. We further propose ManiVidLens, a unified framework for explainable video forgery analysis. Its Forensic Evidence Router supplies shared low-level forensic evidence for multimodal reasoning and video segmentation. Its Prompt Distill Module converts grounding states into semantic and geometric prompts and distills spatial priors for mask decoding and full-video propagation. ManiVidLens achieves relative gains over the strongest comparison methods in artifact grounding (+21.1% mIoU; +21.3% J&F) and anomaly explanation (+131.3% ROUGE-L; +9.9% CSS). Its forgery detection remains comparable to dedicated classifiers (0.914 Acc; 0.913 F1).

Chinese Translation

人工智能生成视频（AIGV）的快速发展增加了欺骗性视频操纵所带来的风险。与完全合成的视频不同，操纵视频保留了大部分源内容，仅对局部区域进行修改，这使得法医分析变得尤为困难。现有的视频伪造研究在数据和方法论上面临两个限制：（1）专门针对操纵视频的高质量数据集和基准仍然稀缺。（2）多模态大型语言模型（MLLMs）将伪造分析扩展到二元分类之外，但在利用低级法医线索和提供精确的像素级基础方面存在困难。具体而言，我们介绍了ManiVid，这是一个统一的法医分析任务，涵盖操纵视频的伪造检测、伪造痕迹定位和异常解释。我们构建了ManiVid-38K，这是第一个结合成对的开放词汇局部操纵的通用视频数据集，并附有真实性标签、伪造掩码和异常解释。该数据集包含约19K个经过人工验证的真实-虚假视频对，主要为1080P分辨率，生成于两种范式下，使用15种强大的生成模型。我们为ManiVidBench抽取了1K对样本，涵盖六种操纵类型和生成模型，以实现公平评估。我们进一步提出了ManiVidLens，这是一个统一的可解释视频伪造分析框架。其法医证据路由器提供共享的低级法医证据，以支持多模态推理和视频分割。其提示提炼模块将基础状态转换为语义和几何提示，并提炼空间先验以进行掩码解码和全视频传播。ManiVidLens在伪造痕迹定位（+21.1% mIoU；+21.3% J&F）和异常解释（+131.3% ROUGE-L；+9.9% CSS）方面相较于最强对比方法取得了相对提升。其伪造检测的表现与专门的分类器相当（0.914 准确率；0.913 F1）。

View on arXiv Download PDF AI Translation

cs.CV / 34 / 2609.30941

Spackle: Completing Large View Single Image NVS with Adaptive Gaussians

Spackle：使用自适应高斯完成大视角单图像新视图合成

Liu, Xuanzhi, Zhou, Yuhe, Wu, Xinyi, Wu, Zhenyao, Chen, Jinghao, Han, Ruize, Wang, Song

Abstract

Single-image novel view synthesis (NVS) enables photorealistic rendering of un- observed viewpoints from a single input. Practical NVS systems require two key capabilities: robust reconstruction of occluded regions and high inference effi- ciency. While hybrid decoupled frameworks combining feedforward 3D Gaussian Splatting (3DGS) and diffusion models show promise for large-view-deviation NVS, they suffer from capacity competition: a fixed number of Gaussians forces resource shifts from visible to newly disoccluded areas, degrading original scene fidelity when the target view deviates significantly from the input. To address this, we propose Spackle, a lightweight residual learning framework that mit- igates capacity competition without sacrificing efficiency. Spackle operates in three stages: predicting base 3DGS attributes from given views, automatically identifying poorly reconstructed regions, and learning a residual 3DGS optimized exclusively for these areas. At inference, we combine the baseline and aug- mented Gaussians for NVS. We conduct comprehensive experiments and show that Spackle achieves state-of-the-art performance on large-view-deviation cases.

Chinese Translation

单图像新视图合成（NVS）能够从单个输入中实现未观察视点的照片级真实感渲染。实用的NVS系统需要具备两个关键能力：对遮挡区域的稳健重建和高推理效率。虽然结合前馈3D高斯点云（3DGS）和扩散模型的混合解耦框架在大视角偏差NVS中显示出潜力，但它们面临容量竞争的问题：固定数量的高斯点迫使资源从可见区域转移到新解遮挡区域，当目标视图与输入显著偏离时，导致原始场景保真度下降。为了解决这个问题，我们提出了Spackle，一种轻量级残差学习框架，它在不牺牲效率的情况下缓解容量竞争。Spackle分三个阶段进行操作：从给定视图预测基础3DGS属性，自动识别重建不佳的区域，以及学习专门针对这些区域优化的残差3DGS。在推理时，我们结合基线和增强的高斯点进行NVS。我们进行了全面的实验，结果表明Spackle在大视角偏差情况下达到了最先进的性能。

View on arXiv Download PDF AI Translation

cs.CV / 35 / 2609.30946

OneWorld: Learning Consistent Physics Across Actions in World Models

OneWorld：在世界模型中学习一致的物理规律跨越动作

He, Ke, Ding, Yichen, Yang, Bin

Abstract

Action-conditioned video world models aim to predict scene evolution under different actions, a capability that is essential for reliable planning, decision-making, and interaction in dynamic environments. However, futures generated independently from the same initial scene may each appear plausible while implying incompatible physical properties, such as friction or mass. This inconsistency can lead to contradictory predictions across interventions, making it difficult for the model to maintain a coherent understanding of the underlying world and limiting its reliability for planning and decision-making. To address these issues, we propose OneWorld, a shared-mechanism counterfactual generation framework that jointly models multiple action-conditioned futures under a common latent physical mechanism. A physical mechanism interpreter first infers a distribution over latent mechanisms from each action-outcome branch. These distributions are then aggregated into shared-world evidence, which captures whether the branches admit a common physical explanation while accounting for uncertainty in less informative branches. This evidence constrains flow training and guides sampling, encouraging consistency in the underlying physical mechanism while preserving the distinct outcomes induced by different actions. We further introduce a multi-intervention evaluation protocol in controlled environments, following the interaction settings of ACWM-Phys, to assess whether generated futures can be jointly explained by the same physical parameters, alongside standard measures of single-rollout prediction quality. Experiments in these environments show that OneWorld improves cross-intervention physical consistency while maintaining competitive single-rollout prediction quality.

Chinese Translation

基于动作的视频世界模型旨在预测在不同动作下场景的演变，这一能力对于在动态环境中进行可靠的规划、决策和互动至关重要。然而，从相同初始场景独立生成的未来可能各自看似合理，但却暗含不兼容的物理属性，例如摩擦或质量。这种不一致性可能导致干预之间的矛盾预测，使模型难以维持对基础世界的连贯理解，并限制其在规划和决策中的可靠性。为了解决这些问题，我们提出了OneWorld，一个共享机制的反事实生成框架，能够在一个共同的潜在物理机制下联合建模多个基于动作的未来。物理机制解释器首先从每个动作-结果分支推断潜在机制的分布。这些分布随后被聚合为共享世界证据，捕捉分支是否承认一个共同的物理解释，同时考虑到在信息较少的分支中的不确定性。这一证据约束了流训练并引导采样，鼓励基础物理机制的一致性，同时保留不同动作所诱导的不同结果。我们进一步在受控环境中引入了一种多干预评估协议，遵循ACWM-Phys的互动设置，以评估生成的未来是否可以由相同的物理参数共同解释，同时结合单次回合预测质量的标准测量。在这些环境中的实验表明，OneWorld在保持竞争性的单次回合预测质量的同时，提高了跨干预的物理一致性。

View on arXiv Download PDF AI Translation

cs.CV / 36 / 2609.30947

DAPEVO: Deep Adaptive Patch Frame-Event Visual Odometry

DAPEVO：深度自适应补丁帧事件视觉里程计

Gandolfi, Luca, Nascivera, Simone, Pellerito, Roberto, Zou, Rong, Plizzari, Chiara, Scaramuzza, Davide

Abstract

Visual odometry is essential for autonomous navigation in GPS-denied environments, yet RGB-based methods remain vulnerable to motion blur, challenging illumination, and dropped frames. Event cameras complement conventional cameras with high temporal resolution and dynamic range, but their asynchronous measurements complicate reliable correspondence estimation. We present DAPEVO, a learned visual odometry system that estimates image and event correspondences independently at shared patch locations and fuses their correlation evidence before motion refinement. Each tracked patch maintains image and event descriptors, and a learned scalar gate combines modality-specific correlation embeddings for each patch--frame edge before a shared recurrent refinement and bundle-adjustment update. DAPEVO also supports event-only observations, enabling continued tracking when RGB frames are sparse or unavailable, while modality-aware keyframe culling preserves scarce frame constraints. On UZH-FPV, when retaining only one in six RGB frames, DAPEVO's mean absolute trajectory error (ATE) increases by only 36%, from 1.00 to 1.36m, whereas the ATE of DPVO and RAMP-VO rises by factors of $3.7\times$ and $3.1\times$, respectively. On TartanEvent, DAPEVO similarly remains below 1m ATE at 3Hz RGB input, while DPVO and RAMP-VO exceed 9m. Under degraded RGB input on TartanEvent, DAPEVO achieves an ATE of 0.60m, compared with more than 4m for both DPVO and RAMP-VO, while also outperforming event-only DEVO at 0.87m.

Chinese Translation

视觉里程计对于在无GPS环境中的自主导航至关重要，但基于RGB的方法仍然容易受到运动模糊、光照变化和丢帧的影响。事件相机以高时间分辨率和动态范围补充传统相机，但其异步测量使得可靠的对应关系估计变得复杂。我们提出了DAPEVO，一种学习的视觉里程计系统，它在共享补丁位置独立估计图像和事件的对应关系，并在运动精炼之前融合它们的相关证据。每个跟踪的补丁维护图像和事件描述符，学习的标量门控为每个补丁-帧边缘组合特定模态的相关嵌入，然后进行共享的递归精炼和束调整更新。DAPEVO还支持仅事件观测，使得在RGB帧稀疏或不可用时能够继续跟踪，同时模态感知的关键帧剔除保留了稀缺的帧约束。在UZH-FPV数据集上，当仅保留六分之一的RGB帧时，DAPEVO的平均绝对轨迹误差（ATE）仅增加36%，从1.00米增加到1.36米，而DPVO和RAMP-VO的ATE分别增加了$3.7 imes$和$3.1 imes$。在TartanEvent上，DAPEVO在3Hz RGB输入下的ATE同样保持在1米以下，而DPVO和RAMP-VO则超过9米。在TartanEvent上，DAPEVO在降级的RGB输入下实现了0.60米的ATE，而DPVO和RAMP-VO的ATE均超过4米，同时在0.87米的表现上也优于仅事件的DEVO。

View on arXiv Download PDF AI Translation

cs.CV / 37 / 2609.30952

MVVBench: Benchmarking 4D Reasoning in Vision-Language Models

MVVBench：视觉语言模型中4D推理的基准测试

Chung, Hyungjin, Park, Byeongjun, Lee, Joonseok, Kim, Hojun, Choi, Jaeho, Kim, Byung-Hoon

Abstract

Multi-view video understanding requires integrating spatial and temporal evidence across multiple, often non-overlapping camera streams: tracking entities as they transition between viewpoints, aligning events across time, and reasoning about latent 4D continuity rather than any single visible frame. We introduce MVVBench, a benchmark for multi-view video reasoning built from real world multi camera datasets. Questions are curated to be monocular-ambiguous along both the view and the temporal axis: each question is unanswerable from any single view in the designated input set, and the majority are further unanswerable from any single moment. Each question becomes uniquely solvable only by jointly reasoning across views and across time. MVVBench spans diverse dynamic scenes and probes six capabilities: implicit/explicit attribute identification, implicit/explicit relative distance, relative camera pose, and compositional counting, with human-authored QA and rigorous verification. Beyond benchmarking, we provide an extensive analysis of when and why current vision language models succeed or fail, characterizing errors due to temporal mis-localization, cross-view identity breaks, and brittle multi-hop reasoning. We then study inference-time elicitation strategies that unlock latent multi-view competence---task-specific chain-of-thought scaffolds and structured cross-view evidence aggregation---yielding substantial gains without retraining. Finally, we present preliminary evidence that reinforcement learning with verifiable rewards can elicit some latent multi-view competence in the base model, pointing to training-time approaches as a promising direction for future work. Together, MVVBench offers a rigorous evaluation of 4D multi-view reasoning and a foundation for future progress toward reliable embodied perception.

Chinese Translation

多视角视频理解需要整合来自多个、通常不重叠的摄像头流的空间和时间证据：在视角之间跟踪实体，跨时间对齐事件，以及推理潜在的4D连续性，而不是任何单一可见帧。我们介绍了MVVBench，这是一个基于真实世界多摄像头数据集构建的多视角视频推理基准。问题经过精心策划，在视角和时间轴上都具有单目模糊性：每个问题在指定输入集中从任何单一视角都无法回答，而且大多数问题在任何单一时刻也无法回答。每个问题只有通过跨视角和跨时间的联合推理才能唯一解决。MVVBench涵盖了多样的动态场景，并探测六种能力：隐式/显式属性识别、隐式/显式相对距离、相对摄像机姿态和组合计数，均采用人工编写的问答和严格验证。除了基准测试外，我们还提供了对当前视觉语言模型成功或失败的广泛分析，描述了由于时间错误定位、跨视角身份断裂和脆弱的多跳推理导致的错误。然后，我们研究了在推理时引出潜在多视角能力的策略——特定任务的思维链支架和结构化的跨视角证据聚合——在不重新训练的情况下带来了显著的提升。最后，我们提供了初步证据，表明使用可验证奖励的强化学习可以引出基础模型中的某些潜在多视角能力，指向训练时间方法作为未来工作的一个有前景的方向。总之，MVVBench提供了对4D多视角推理的严格评估，并为未来在可靠的具身感知方面的进展奠定了基础。

View on arXiv Download PDF AI Translation

cs.CV / 38 / 2609.30962

IDM-Net: A Lightweight Illumination-Decoupled Modulation Network for Low-Light Image Enhancement

IDM-Net：一种轻量级的照明解耦调制网络用于低光照图像增强

Hsiao, Cheng-Yen, Guo, Jing-Ming

Abstract

Low-light image enhancement (LLIE) remains challenging for lightweight models because illumination restoration and color fidelity are difficult to optimize simultaneously in the RGB color space. Although recent color-decoupled methods separate luminance and chrominance representations, they primarily optimize luminance as an enhancement target, leaving its potential as an explicit guidance prior largely unexplored during feature reconstruction. To address this limitation, we propose IDM-Net, a lightweight Illumination-Decoupled Modulation Network for low-light image enhancement. IDM-Net adopts a dual-encoder architecture consisting of a structure encoder that extracts multi-scale appearance features from the RGB image and a lightweight illumination encoder that learns illumination priors from the decoupled luminance (Y) channel. To effectively exploit these priors, we introduce an Illumination-Guided Modulation (IGM) module that injects multi-scale illumination cues into the decoder through spatially adaptive affine modulation, enabling accurate brightness restoration while preserving natural color consistency. Furthermore, we design a lightweight Feature Refinement Block (FRB) to progressively suppress degradation artifacts and recover fine-grained image details during reconstruction. Extensive experiments on multiple standard low-light image enhancement benchmarks demonstrate that IDM-Net achieves competitive performance among lightweight LLIE methods while maintaining an excellent balance between restoration quality and computational efficiency.

Chinese Translation

低光照图像增强（LLIE）对于轻量级模型仍然具有挑战性，因为在RGB颜色空间中，照明恢复和色彩保真度难以同时优化。尽管最近的颜色解耦方法分离了亮度和色度表示，但它们主要将亮度作为增强目标进行优化，在特征重建过程中，亮度作为显式引导先验的潜力尚未得到充分探索。为了解决这一局限性，我们提出了IDM-Net，一种用于低光照图像增强的轻量级照明解耦调制网络。IDM-Net采用双编码器架构，包括一个结构编码器，从RGB图像中提取多尺度外观特征，以及一个轻量级照明编码器，从解耦的亮度（Y）通道学习照明先验。为了有效利用这些先验，我们引入了一个照明引导调制（IGM）模块，通过空间自适应仿射调制将多尺度照明线索注入解码器，从而实现准确的亮度恢复，同时保持自然的色彩一致性。此外，我们设计了一个轻量级特征精炼块（FRB），在重建过程中逐步抑制降级伪影并恢复细粒度图像细节。在多个标准低光照图像增强基准上的广泛实验表明，IDM-Net在轻量级LLIE方法中实现了具有竞争力的性能，同时在恢复质量和计算效率之间保持了良好的平衡。

View on arXiv Download PDF AI Translation

cs.CV / 39 / 2609.30963

Where and When to Force: Routed Forcing for Streaming Avatars

何时何地施加强制：用于流媒体化身的路由强制

Su, Zihan, Lu, Siwen, Zhuang, Junhao, Xue, Zeyue, Huang, Haoyang, Li, Guanghao, Tan, Xiaofeng, Yuan, Chun, Duan, Nan

Abstract

Audio-driven streaming avatar generation requires real-time synthesis of speech-synchronized videos with dynamic and diverse motion. Self Forcing uses Distribution Matching Distillation (DMD) to distill bidirectional video diffusion models into causal, few-step generators for real-time streaming. However, DMD minimizes a reverse KL divergence, which is inherently mode-seeking: it causes the student to discard high-dynamic modes and collapse onto static outputs, compressing both dynamics and diversity of generated videos. We find that this collapse is region-heterogeneous: person regions involving pose and gesture variations suffer the largest diversity loss, the audio-driven mouth region shows a small loss, and the background remains nearly stable. Based on this observation, we propose Routed Forcing, which routes the distillation objective by semantic region and noise stage to improve dynamics and diversity while preserving visual quality. Specifically, (1) Where to Force: Semantic-Region Routing applies Data-Forcing Distillation (DFD), which supervises the student with real videos, to the person region where diversity collapse is most severe, while retaining DMD for the mouth and background to preserve lip synchronization and scene stability. (2) When to Force: Noise-Stage Routing activates DFD at high noise stages, where real video serves as effective supervision to inject diverse and dynamic motion patterns. At low noise stages, DMD is used to refine details, avoiding blur and artifacts from spatial differences between real video and student-generated video. Experiments show that Routed Forcing improves dynamics by up to 45% and diversity by 7-25% over Self Forcing, while preserving video quality and lip synchronization.

Chinese Translation

音频驱动的流媒体化身生成需要实时合成与语音同步的视频，具备动态和多样化的运动。自我强制（Self Forcing）使用分布匹配蒸馏（Distribution Matching Distillation, DMD）将双向视频扩散模型蒸馏为因果的、少步生成器，以实现实时流媒体。然而，DMD最小化反向KL散度，这本质上是模式寻求的：它导致学生丢弃高动态模式，陷入静态输出，从而压缩生成视频的动态性和多样性。我们发现这种崩溃在区域上是不均匀的：涉及姿势和手势变化的人物区域遭受了最大的多样性损失，音频驱动的嘴部区域显示出较小的损失，而背景几乎保持稳定。基于这一观察，我们提出了路由强制（Routed Forcing），通过语义区域和噪声阶段路由蒸馏目标，以提高动态性和多样性，同时保持视觉质量。具体而言，(1) 何处施加强制：语义区域路由将数据强制蒸馏（Data-Forcing Distillation, DFD）应用于多样性崩溃最严重的人物区域，同时保留DMD用于嘴部和背景，以保持唇部同步和场景稳定。(2) 何时施加强制：噪声阶段路由在高噪声阶段激活DFD，此时真实视频作为有效的监督，注入多样化和动态的运动模式。在低噪声阶段，使用DMD来细化细节，避免因真实视频与学生生成视频之间的空间差异而产生的模糊和伪影。实验表明，路由强制在动态性上提高了高达45%，在多样性上提高了7-25%，同时保持视频质量和唇部同步。

View on arXiv Download PDF AI Translation

cs.CV / 40 / 2609.30979

CCRV-Bench: Constraint-Based Evaluation of Causal Reasoning in Vision-Language Models

CCRV-Bench：基于约束的视觉语言模型因果推理评估

Gao, Linyuan, Wu, Yuan, Chang, Yi

Abstract

Vision-language models (VLMs) have demonstrated excellent performance in visual tasks, but their visual causal reasoning capabilities still lack reliable evaluation. Existing evaluations struggle to distinguish whether a model is performing causal reasoning based on visual evidence or relying on statistical correlations for shortcut learning, thereby potentially overestimating their actual capabilities. This paper proposes CCRV-Bench, a constraint-driven visual causal reasoning benchmark for single-image physical scenarios. We construct an orthogonal framework that evaluates four causal task dimensions: causal relation discovery, state prediction, causal diagnosis, and intervention. We further introduce entity symbolization, spatial grounding, the factual adversarial constraint, and minimalist output constraints to reduce shortcut cues while preserving the physical commonsense required by the task. Experiments across 15 multimodal models show that constraint sensitivity is task- and model-dependent: intervention has the largest average effective degradation among the four causal tasks, spatial grounding is the most damaging constraint on average, and the factual adversarial constraint improves DCR for all evaluated models. These results show that unconstrained performance does not determine constrained robustness and that a single aggregate score can obscure distinct failures in causal identification, spatial grounding, and constraint-compliant expression. CCRV-Bench provides a standardized framework for diagnosing image-grounded causal reasoning under controlled constraints. The code is available at https://github.com/0815linyuan/CCRV-Bench-Constraint-Based-Evaluation-of-Causal-Reasoning-in-Vision-Language-Models

Chinese Translation

视觉语言模型（VLMs）在视觉任务中表现出色，但其视觉因果推理能力仍缺乏可靠的评估。现有评估难以区分模型是基于视觉证据进行因果推理，还是依赖统计相关性进行捷径学习，从而可能高估其实际能力。本文提出了CCRV-Bench，一个针对单图像物理场景的基于约束的视觉因果推理基准。我们构建了一个正交框架，评估四个因果任务维度：因果关系发现、状态预测、因果诊断和干预。我们进一步引入了实体符号化、空间定位、事实对抗约束和简约输出约束，以减少捷径线索，同时保留任务所需的物理常识。对15个多模态模型的实验表明，约束敏感性依赖于任务和模型：在四个因果任务中，干预的平均有效降级最大，空间定位是平均最具破坏性的约束，而事实对抗约束提高了所有评估模型的DCR。这些结果表明，无约束性能并不决定有约束的鲁棒性，单一的综合评分可能掩盖因果识别、空间定位和符合约束的表达中的不同失败。CCRV-Bench为在受控约束下诊断图像基础的因果推理提供了一个标准化框架。代码可在 https://github.com/0815linyuan/CCRV-Bench-Constraint-Based-Evaluation-of-Causal-Reasoning-in-Vision-Language-Models 获取。

View on arXiv Download PDF AI Translation

cs.CV / 41 / 2609.30981

STORM-Bench: Evaluating Online Video QA under Evolving and Incomplete Evidence

STORM-Bench：在不断变化和不完整证据下评估在线视频问答

Zhong, Siru, Tan, Shenghan, Yan, Rihong, Lv, Xiaohui, Zhuang, Yuzheng, Tao, Shuai, Liu, Wulong, Fu, Haohuan, Liang, Yuxuan

Abstract

Reliable online video question answering requires tracking state transitions while selectively abstaining when visual evidence is insufficient. Existing benchmarks focus on static recognition or long-range retrieval, rarely evaluating these coupled capabilities under evolving and incomplete evidence. We present STORM-Bench, comprising 5,736 questions across 630 compact, change-dense episodes spanning five egocentric domains (STORM-Real) and two controlled simulation subsets (STORM-Sim) at 1 FPS. Questions are stratified by a proxy for accumulated change intensity (Low, Medium, High) and query-time answerability (Known, Uncertain). To measure reliability, we introduce STORM-BR, a harmonic metric over joint answer-status correctness that exposes abstention failures masked by aggregate accuracy, alongside STORM-BR-ATTR for uncertainty attribution. Across 14 video LLMs, online accuracy peaks at 60.3\% (mean 51.7\%), whereas STORM-BR ranges from 5.7\% to 35.6\% (mean 18.8\%), driven by pervasive overconfidence on uncertain queries. STORM-Bench shows that task accuracy masks these gaps in epistemic reliability and state tracking. Benchmark and code are available at https://github.com/siruzhong/STORM-Bench.

Chinese Translation

可靠的在线视频问答需要在视觉证据不足时跟踪状态转变并选择性地放弃。现有基准主要关注静态识别或长距离检索，鲜有在不断变化和不完整证据下评估这些耦合能力。我们提出了STORM-Bench，包含5736个问题，涵盖630个紧凑且变化密集的情节，涉及五个自我中心领域（STORM-Real）和两个受控模拟子集（STORM-Sim），以1帧每秒的速度进行。问题根据累积变化强度的代理（低、中、高）和查询时的可回答性（已知、不确定）进行分层。为了衡量可靠性，我们引入了STORM-BR，这是一种基于联合答案状态正确性的和谐度量，揭示了被聚合准确性掩盖的放弃失败，以及STORM-BR-ATTR用于不确定性归因。在14个视频大语言模型中，在线准确率最高为60.3%（平均51.7%），而STORM-BR的范围为5.7%到35.6%（平均18.8%），这主要是由于对不确定查询的普遍过度自信。STORM-Bench表明，任务准确性掩盖了这些在认知可靠性和状态跟踪方面的差距。基准和代码可在https://github.com/siruzhong/STORM-Bench获取。

View on arXiv Download PDF AI Translation

cs.CV / 42 / 2609.30982

FARE: Forensic Acceptance Region Estimation for Catching Bait-and-Switch Image Generators

FARE：用于捕捉诱饵与开关图像生成器的法医接受区域估计

Yao, Kai, Juarez, Marc

Abstract

Modern AI image generators are increasingly deployed as opaque APIs, where customers can query the deployed service, but cannot inspect model weights or architecture. This creates a practical challenge: a provider may pass governance certification with one generator and later silently switch to a cheaper and lower-quality one for deployment, compromising public trust or even safety in high-stakes domains. We study integrity auditing at deployment time and propose FARE (Forensic Acceptance Region Estimation). A certified generator is enrolled by training FARE on images sampled from that generator. After deployment, FARE can determine whether a generated image is consistent with the enrolled generator---using only that image. FARE's features are based on image generator-specific artifacts that have been proposed for forensic applications. FARE amplifies these features during training by finding hard samples that tighten the acceptance region and increase sensitivity to subtle changes in the certified generator. Across generator swaps, including substitutions with similar model versions and model variants, FARE is effective at detecting swaps, consistently outperforming existing baselines at strict operating points, and remains effective under the exact-model and decision-only attacks evaluated in this work.

Chinese Translation

现代人工智能图像生成器越来越多地作为不透明的API被部署，客户可以查询已部署的服务，但无法检查模型权重或架构。这带来了一个实际挑战：提供者可能通过一个生成器通过治理认证，而后在部署时悄然切换到一个更便宜且质量更低的生成器，从而损害公众信任，甚至在高风险领域影响安全性。我们研究了在部署时的完整性审计，并提出了FARE（法医接受区域估计）。通过对来自该生成器的图像进行训练，认证生成器被纳入FARE。部署后，FARE可以仅通过生成的图像判断其是否与已注册的生成器一致。FARE的特征基于已提出用于法医应用的图像生成器特定伪影。FARE在训练过程中通过寻找难样本来放大这些特征，从而收紧接受区域并提高对认证生成器微小变化的敏感性。在生成器交换中，包括与相似模型版本和模型变体的替换，FARE在检测交换方面表现有效，在严格的操作点上始终优于现有基准，并在本研究评估的精确模型和仅决策攻击下保持有效。

View on arXiv Download PDF AI Translation

cs.CV / 43 / 2609.30987

Self-Supervised Perceptually Interpretable Monocular Depth Estimation

自监督感知可解释的单目深度估计

Abidin, Zain Ul, Dimas, George, Iakovidis, Dimitris K.

Abstract

Self-supervised monocular depth estimation (MDE) enables depth prediction from monocular images without requiring ground-truth supervision, making it attractive for large-scale and real-world applications. Despite steady improvements in accuracy, most existing methods remain difficult to interpret, as depth is inferred from RGB representations that obscure the impact of individual perceptual image components. This lack of transparency limits systematic analysis of failure cases and reduces confidence in safety-critical settings. This paper presents a self-supervised framework for perceptually interpretable monocular depth estimation (PIMDE), designed to associate depth predictions with distinct perceptual components of the input image. Rather than operating directly on RGB inputs, the proposed method decomposes each image into a set of perceptual feature maps (PFMs), each encoding a specific visual cue. Distinct depth estimation branches process these PFMs independently to produce depth estimates (PIDEs), which are subsequently combined through an explicit fusion strategy. This formulation allows us to examine directly the contribution of each perceptual cue to the final depth prediction. Experiments conducted on the KITTI benchmark dataset demonstrate that PIMDE achieves performance comparable to established self-supervised MDE methods while providing additional insight into how different perceptual cues influence depth estimation. These results indicate that perceptual decomposition can support interpretability without sacrificing depth estimation accuracy.

Chinese Translation

自监督单目深度估计（MDE）能够从单目图像中进行深度预测，而无需真实标签的监督，这使其在大规模和现实世界应用中具有吸引力。尽管准确性稳步提高，但大多数现有方法仍然难以解释，因为深度是从RGB表示中推断的，这掩盖了单个感知图像成分的影响。这种缺乏透明度限制了对失败案例的系统分析，并降低了在安全关键环境中的信心。本文提出了一种自监督框架，用于感知可解释的单目深度估计（PIMDE），旨在将深度预测与输入图像的不同感知成分关联起来。该方法并非直接处理RGB输入，而是将每幅图像分解为一组感知特征图（PFMs），每个图都编码了特定的视觉线索。不同的深度估计分支独立处理这些PFMs，以产生深度估计（PIDEs），然后通过显式融合策略将其组合。这种公式化使我们能够直接检查每个感知线索对最终深度预测的贡献。在KITTI基准数据集上进行的实验表明，PIMDE的性能与现有的自监督MDE方法相当，同时提供了关于不同感知线索如何影响深度估计的额外见解。这些结果表明，感知分解可以支持可解释性，而不牺牲深度估计的准确性。

View on arXiv Download PDF AI Translation

cs.CV / 44 / 2609.30988

PhoenixSR: Generative Heterogeneous Distillation Unleashes Efficient Models for Real-World Super-Resolution

PhoenixSR：生成异构蒸馏释放高效模型以实现真实世界超分辨率

Di, Xin, Shi, Mingyu, Bao, Yuanfei, Peng, Long, Zhao, Yue, Guo, Jiaming, Pei, Renjing, Fu, Xueyang, Cao, Yang, Zha, Zheng-Jun

Abstract

Real-world image super-resolution (SR) requires recovering perceptually realistic high-resolution images from complex low-resolution observations while preserving faithful content. Diffusion-based SR benefits from strong generative priors but incurs substantial computational overhead, whereas feed-forward CNN and Transformer SR models are efficient yet often struggle to recover realistic high-frequency details. This motivates a natural question: can diffusion priors be transferred to existing diffusion-free SR networks without introducing diffusion components at inference time? To this end, we propose PhoenixSR, a generative heterogeneous distillation framework that transfers diffusion priors to independently designed feed-forward SR networks through score-based distribution matching. Rather than aligning heterogeneous features or imitating sampled diffusion outputs, PhoenixSR uses the pretrained diffusion model as distribution-level supervision, while paired SR supervision preserves reconstruction fidelity. To make distribution matching effective for fidelity-sensitive SR, we introduce Heterogeneous Distribution Adaptation, which adapts the target score to the SR domain, improves tracking of the evolving student distribution, and anchors training with paired supervision. We further employ Directional Reliability Weighting, a lightweight residual-consistency-based reweighting strategy that reduces unstable distributional guidance. All diffusion-related components are removed after training, leaving the original student architecture and inference cost unchanged. Experiments on three SR benchmarks and six feed-forward backbones, including SwinIR, HAT, Real-ESRGAN, and SeeMoRe, show consistent perceptual improvements with largely preserved reconstruction fidelity.

Chinese Translation

真实世界图像超分辨率（SR）需要从复杂的低分辨率观测中恢复感知上逼真的高分辨率图像，同时保持内容的真实度。基于扩散的超分辨率受益于强大的生成先验，但会带来相当大的计算开销，而前馈卷积神经网络（CNN）和变换器（Transformer）超分辨率模型则高效，但常常难以恢复真实的高频细节。这引发了一个自然的问题：能否在推理时将扩散先验转移到现有的无扩散超分辨率网络，而不引入扩散组件？为此，我们提出了PhoenixSR，一种生成异构蒸馏框架，通过基于评分的分布匹配将扩散先验转移到独立设计的前馈超分辨率网络。PhoenixSR并不是对齐异构特征或模仿采样的扩散输出，而是将预训练的扩散模型作为分布级别的监督，同时配对的超分辨率监督保持重建的保真度。为了使分布匹配在对保真度敏感的超分辨率中有效，我们引入了异构分布适应（Heterogeneous Distribution Adaptation），该方法将目标评分适应到超分辨率领域，改善对不断演变的学生分布的跟踪，并通过配对监督锚定训练。我们进一步采用了方向可靠性加权（Directional Reliability Weighting），这是一种轻量级的基于残差一致性的重加权策略，减少不稳定的分布指导。所有与扩散相关的组件在训练后被移除，保持原始学生架构和推理成本不变。在三个超分辨率基准和六个前馈骨干网络（包括SwinIR、HAT、Real-ESRGAN和SeeMoRe）上的实验显示，感知效果一致改善，同时重建保真度大体保持不变。

View on arXiv Download PDF AI Translation

cs.CV / 45 / 2609.30989

PICO: Projection-Informed Consistency Optimisation for 6DoF Surgical Tool Pose Estimation

PICO：基于投影信息的一致性优化用于六自由度外科工具姿态估计

Fothergill, Lucy, Valdastri, Pietro, Jones, Dominic, Sarikaya, Duygu

Abstract

Purpose: Accurate 6 DoF pose estimation of surgical tools is critical for automa- tion, robotic proprioception, and safe interaction with the tissue operated on. Kinematics-based approaches suffer from accumulated errors due to the cable- driven nature of robotic arms, while vision-based methods often rely on external markers or trackers. Although more recent vision-based advances have been pro- posed, these two-stage pose estimation methods often lack real-time robustness due to accumulated errors and computational overhead. Methods: We propose a novel end-to-end trainable model, PICO. Our model employs a multi-task learning architecture to predict segmentation and depth maps, alongside regression of translation and rotation parameters. We define two proxy tasks that enforce geometric consistency in both 2D and 3D spaces, improving accuracy and robustness. For this, we propose a projection loss, and a point-to-point loss. Results: We evaluate our method on the SurgRIPE dataset, benchmarking its performance against state-of-the-art approaches using standard 6DoF pose esti- mation metrics. Our results demonstrate consistently strong performance across all four subsets, specifically in rotation, ranking second even under occlusion. It also demonstrates comparable translational performance, remaining competitive, especially in occluded cases. Conclusion: PICO demonstrates the effectiveness of multi-task learning and geometry-aware proxy tasks for robust and reliable surgical tool pose estimation, especially in occluded scenarios, highlighting potential for future applications.

Chinese Translation

目的：外科工具的准确六自由度（6 DoF）姿态估计对于自动化、机器人本体感知以及与操作组织的安全互动至关重要。基于运动学的方法由于机器人臂的电缆驱动特性而遭受累积误差，而基于视觉的方法通常依赖于外部标记或跟踪器。尽管最近提出了更先进的基于视觉的方法，这些两阶段的姿态估计方法通常由于累积误差和计算开销而缺乏实时的鲁棒性。方法：我们提出了一种新颖的端到端可训练模型PICO。我们的模型采用多任务学习架构来预测分割和深度图，同时回归平移和旋转参数。我们定义了两个代理任务，以在二维和三维空间中强制几何一致性，从而提高准确性和鲁棒性。为此，我们提出了一种投影损失和点对点损失。结果：我们在SurgRIPE数据集上评估了我们的方法，并使用标准的6 DoF姿态估计指标对其性能进行了基准测试。我们的结果在所有四个子集中均表现出持续强劲的性能，特别是在旋转方面，即使在遮挡情况下也排名第二。它在平移性能上也表现出可比性，尤其是在遮挡情况下仍然具有竞争力。结论：PICO展示了多任务学习和几何感知代理任务在外科工具姿态估计中的有效性，尤其是在遮挡场景中，突显了未来应用的潜力。

View on arXiv Download PDF AI Translation

cs.CV / 46 / 2609.30993

FLIP: Final Layer Inference-Time Probing for Vision-Language Models

FLIP：面向视觉-语言模型的最终层推理时间探测

Juanico, Drandreb Earl O., Atienza, Rowel O.

Abstract

We present FLIP, a final-layer inference-time probe for testing whether a logit-facing intervention site in an open-weight vision-language model (VLM) supports structured, task-linked computation rather than generic perturbation. Behavioral change under internal intervention is otherwise mechanistically ambiguous: it may reflect improved use of visual evidence, generic output instability, or outright degradation. FLIP applies elementwise flooring to the final normalized hidden state before logit computation, leaving parameters, prompts, and decoding unchanged. On a controlled detection/counting probe, sweeping intervention strength reveals three regions: negligible change, a bounded interior regime in which detection recall at IoU 0.50 ($R_{50}$) improves while tolerant counting error ($\mathcal{E}_{\mathrm{count}}$) falls, and over-suppression. We formalize a four-criterion probe-and-sweep protocol for disciplining the interpretation of intervention effects: regime structure, grounding-proxy alignment, feature-coherence dependence, and failure to reproduce the same positive regime on a performance-based negative control. The post-normalization state passed to the output head is the logit-facing instantiation of this test; under a non-targeted flooring sweep it satisfies the full protocol. Raw decoder-layer interventions, including the last-block output before final normalization, and the singleton-pair left/right control fail to reproduce the Final-site signature, while same-site operators and multiple VLMs replicate it. FLIP is therefore a validation step for intervention-based mechanistic interpretability, not a steering method.

Chinese Translation

我们提出了FLIP，一种用于测试开放权重视觉-语言模型（VLM）中logit面对干预位置是否支持结构化、任务关联计算而非通用扰动的最终层推理时间探测器。在内部干预下的行为变化在机制上是模糊的：它可能反映出对视觉证据的改进利用、通用输出不稳定性或直接退化。FLIP在logit计算之前对最终归一化的隐藏状态应用逐元素下限处理，保持参数、提示和解码不变。在一个受控的检测/计数探测中，干预强度的变化揭示了三个区域：微不足道的变化、一个有限的内部区域，在该区域内，IoU 0.50下的检测召回率（$R_{50}$）提高，而容忍的计数误差（$ extmath{E}_{ ext{count}}$）下降，以及过度抑制。我们正式化了一个四项标准的探测与扫描协议，以规范干预效果的解释：区域结构、基础代理对齐、特征一致性依赖，以及未能在基于性能的负控制中重现相同的正区域。传递给输出头的后归一化状态是该测试的logit面对实例；在非针对性的下限扫描下，它满足完整协议。原始解码器层干预，包括最终归一化前的最后块输出，以及单例对的左右控制未能重现最终位置特征，而同位置操作符和多个VLM则成功复制了这一特征。因此，FLIP是基于干预的机制可解释性的验证步骤，而不是引导方法。

View on arXiv Download PDF AI Translation

cs.CV / 47 / 2609.31005

TRACKGRAPH: Online Open-Vocabulary 3D Scene Graphs via Image-Space Tracking

TRACKGRAPH：通过图像空间跟踪实现在线开放词汇3D场景图

Hellesylt, Peder Borge, Puigjaner, Albert Gassol, Alexis, Kostas, Stahl, Annette

Abstract

Open-vocabulary 3D maps enable robots to reason about previously unknown environments using natural language. However, existing systems typically segment every incoming image, associate detections with persistent 3D segments, and frequently perform costly Vision-Language (VL) inference. We present TRACKGRAPH, an online open-vocabulary system that maintains short-term 2D mask identity directly in the image stream before fusing segments into 3D. FastSAM masks and CLIP features are computed at sparse keyframes, while dense DINOv3 features are used to propagate masks at a high rate in between. The resulting tracked masks are fused into a class-agnostic 3D segment layer within a hierarchical scene graph, with 3D association handling tracking interruptions and long-term revisits. Compact multi-view CLIP embeddings enable open-vocabulary retrieval. Across Replica, ScanNet++, and HM3D, TRACKGRAPH achieves competitive open-vocabulary segmentation and retrieval against state-of-the-art mapping methods, including the highest synonym frequency on Replica (0.50). On the same NVIDIA A100, it is 1.7x faster and uses 3.3x less GPU memory than ViT-H OVI-MAP. Real-world quadruped deployments demonstrate onboard scene graph construction and object search at 7.5Hz, while recorded drone data is used to test the method under aerial viewpoints.

Chinese Translation

开放词汇3D地图使机器人能够使用自然语言推理以前未知的环境。然而，现有系统通常对每个输入图像进行分割，将检测结果与持久的3D分段关联，并频繁执行成本高昂的视觉-语言（Vision-Language, VL）推理。我们提出了TRACKGRAPH，一个在线开放词汇系统，它在将分段融合为3D之前，直接在图像流中维护短期2D掩码身份。FastSAM掩码和CLIP特征在稀疏关键帧上计算，而密集的DINOv3特征则用于在两者之间以高频率传播掩码。生成的跟踪掩码被融合到一个类无关的3D分段层中，位于一个分层场景图内，3D关联处理跟踪中断和长期重访。紧凑的多视角CLIP嵌入支持开放词汇检索。在Replica、ScanNet++和HM3D数据集上，TRACKGRAPH在开放词汇分割和检索方面与最先进的映射方法相比表现出竞争力，包括在Replica上达到的最高同义词频率（0.50）。在同一NVIDIA A100上，它比ViT-H OVI-MAP快1.7倍，并且使用的GPU内存少3.3倍。实际的四足机器人部署展示了以7.5Hz的频率进行的现场图构建和物体搜索，同时记录的无人机数据用于在空中视角下测试该方法。

View on arXiv Download PDF AI Translation

cs.CV / 48 / 2609.31028

Refining Cytology Predictions with Conditional Random Fields

利用条件随机场优化细胞学预测

Dausort, Manon, Godelaine, Tiffanie, Khoury, Karim El, Zanella, Maxime, De Vleeschouwer, Christophe, Macq, Benoît

Abstract

Vision-language models (VLMs) achieve strong zero-shot (ZS) classification on histology images but do not perform as well on cytology, whose stains and cell morphology differ markedly compared to histology. Conditional random fields (CRFs) can refine noisy VLM predictions by propagating information across patches, but existing CRF frameworks were designed for histopathology and do not transfer to cytology datasets, released as independent patch pools spanning multiple staining protocols. We introduce CytoCRF, which adapts the pairwise terms to cytology by targeting chromatin and cytology-specific staining, and further enrich the neighborhood of each potential term by combining multiple backbones. Across ten cytology datasets, CytoCRF outperforms existing CRF frameworks at every annotation budget, reaching +13.6 percentage points over the best baseline and +33.7 over ZS with only 50 annotations. Combining information from multiple backbones brings further gains, showing that the neighborhood topology matters more than the pairwise potential computed over it.

Chinese Translation

视觉语言模型（VLMs）在组织学图像上实现了强大的零样本（ZS）分类，但在细胞学上的表现不佳，因为细胞学的染色和细胞形态与组织学有显著不同。条件随机场（CRFs）可以通过在各个图像块之间传播信息来优化嘈杂的VLM预测，但现有的CRF框架是为组织病理学设计的，无法迁移到作为独立图像块池发布的细胞学数据集，这些数据集涵盖了多种染色方案。我们提出了CytoCRF，它通过针对染色质和细胞学特定染色来调整成对项，并通过结合多个骨干网络进一步丰富每个潜在项的邻域。在十个细胞学数据集上，CytoCRF在每个标注预算下均优于现有的CRF框架，最佳基线提升了13.6个百分点，零样本情况下提升了33.7个百分点，仅需50个标注。结合来自多个骨干网络的信息带来了进一步的提升，表明邻域拓扑比在其上计算的成对潜力更为重要。

View on arXiv Download PDF AI Translation

cs.CV / 49 / 2609.31040

Exploiting Spatial Structure for Transductive Few-Shot Classification of Whole-Slide Images

利用空间结构进行全切片图像的迁移性少样本分类

Godelaine, Tiffanie, Dausort, Manon, Khoury, Karim El, Gérin, Benoît, Macq, Benoît, De Vleeschouwer, Christophe

Abstract

Automating the analysis of whole-slide images (WSIs), a key step in cancer diagnosis, has high clinical value, as it can reduce pathologist's workload while improving diagnosis accuracy. Recently, vision-language models have shown promising performance for patch-level classification without requiring any annotation, yet these zero-shot (ZS) predictions remain noisy on fine-grained tasks and must be further refined. A promising direction is to refine all predictions jointly, i.e., a transductive approach. However, most existing methods are not tailored to WSIs. We thus propose SlideTIM, an adaptation to WSIs of the recent transductive approach LC-TIM, which introduces a combined spatial--latent regularizer together with a prior on the patch class distribution. The former enforces spatially and semantically close patches to receive the same predictions, while the prior calibrates the predicted class proportions. Together, they address the complex spatial organization and the strong class imbalance of WSIs. Evaluated on four histology datasets, SlideTIM consistently outperforms all TIM variants, improving the macro-F1 by +8.1pp over the best competing baseline at 1 shot. Compared to the ZS, it raises the macro-F1 by +19.4pp at 1 shot. The code will be made available after submission.

Chinese Translation

自动化分析全切片图像（WSIs）是癌症诊断中的关键步骤，具有重要的临床价值，因为它可以减轻病理学家的工作负担，同时提高诊断准确性。最近，视觉-语言模型在无需任何注释的情况下，已在补丁级分类中展现出良好的性能，然而这些零样本（ZS）预测在细粒度任务中仍然存在噪声，必须进一步优化。一个有前景的方向是联合优化所有预测，即采用迁移性方法。然而，大多数现有方法并未针对WSIs进行定制。因此，我们提出了SlideTIM，这是对最近迁移性方法LC-TIM在WSIs上的适应，结合了空间-潜在正则化器以及对补丁类别分布的先验。前者强制空间和语义上相近的补丁接收相同的预测，而先验则校准预测的类别比例。两者共同解决了WSIs的复杂空间组织和强类别不平衡问题。在四个组织学数据集上的评估中，SlideTIM始终优于所有TIM变体，在1次样本下将宏F1提高了+8.1个百分点。与ZS相比，在1次样本下宏F1提高了+19.4个百分点。代码将在提交后公开。

View on arXiv Download PDF AI Translation

cs.CV / 50 / 2609.31050

Where Compute Matters: Heterogeneous Attention for Efficient Video Diffusion

计算重要性：高效视频扩散的异构注意力机制

Zatsarynna, Olga, Korzhenkov, Denis, Gall, Juergen, Habibian, Amir, Ghafoorian, Mohsen

Abstract

Efficient video generation requires reducing the quadratic cost of self-attention over long spatio-temporal token sequences. Existing efficient-attention methods typically apply the same computation pattern to every token, even though denoising difficulty varies substantially across video regions and evolves throughout the generation process. We introduce HetA-DiT, a heterogeneous attention mechanism that adaptively allocates computation according to token difficulty. A lightweight uncertainty branch predicts a token-wise estimate of denoising difficulty, which is used to route uncertain tokens through dense global attention while processing more reliable tokens with efficient local attention. The resulting routing is content- and timestep-adaptive, retains global context where it matters most, and provides a single parameter for controlling the quality-efficiency trade-off. HetA-DiT is compatible with few-step distribution-matching distillation and introduces no additional Transformer evaluation at inference time by reusing uncertainty estimates from the preceding denoising step. We evaluate the method on DMD-distilled Wan2.2-5B and Wan2.1-1.3B models. Across VBench, VBench-2.0, and human preference evaluation, HetA-DiT maintains competitive generation quality while routing only approximately 20% of tokens through dense attention.

Chinese Translation

高效的视频生成需要降低在长时空序列上自注意力的平方成本。现有的高效注意力方法通常对每个标记应用相同的计算模式，尽管去噪难度在视频区域之间差异显著，并且在生成过程中不断演变。我们提出了HetA-DiT，一种异构注意力机制，根据标记的难度自适应地分配计算。一个轻量级的不确定性分支预测每个标记的去噪难度估计，用于通过密集全局注意力路由不确定的标记，同时用高效的局部注意力处理更可靠的标记。最终的路由是内容和时间步自适应的，在最重要的地方保留全局上下文，并提供一个单一参数来控制质量与效率的权衡。HetA-DiT与少步分布匹配蒸馏兼容，并且在推理时通过重用前一步去噪的的不确定性估计而不引入额外的Transformer评估。我们在DMD蒸馏的Wan2.2-5B和Wan2.1-1.3B模型上评估该方法。在VBench、VBench-2.0和人类偏好评估中，HetA-DiT在仅约20%的标记通过密集注意力路由的情况下，保持了竞争力的生成质量。

View on arXiv Download PDF AI Translation

cs.CV / 51 / 2609.31074

Band-Selection Stability and Semantic Segmentation Performance: A Study on Hyperspectral City

Li, Jiarong, Shah, Imad Ali, Ward, Enda, Glavin, Martin, Jones, Edward, Deegan, Brian

Abstract

Resource constraints make high-dimensional hyperspectral imaging challenging in autonomous perception, motivating the use of band selection methods. However, the sensitivity of band-selection methods to sampled data and their relationship to semantic segmentation models (SSMs) remain underexplored. This study evaluates six band selection methods on ten independently sampled, class-balanced region-of-interest (ROI) sets, yielding 60 top-25 band subsets from the Hyperspectral City V2 (128 bands: 450-950nm) dataset. Top-$K$ bands ($K\in\{3,5, ... 13\}$) from the first three ROI sets are evaluated with three SSMs against the corresponding 128-band baseline. Experiments show that intra-method stability is method-dependent: Sim-LP shows the highest stability (pairwise Jaccard similarity) and, together with JMIM+CSNR, yields the best segmentation results. Top-$K$ based SSMs remain competitive with baselines, with gains of up to 2.01 mIoU and 1.72 mF1 points, and 18-22x faster CPU inference for $K=9$. However, performance does not improve monotonically with $K$, and stability shows no consistent association with SSM performance. These findings suggest that intra-method stability is informative but an unreliable indicator of downstream segmentation performance, highlighting the need to evaluate band-selection methods across repeated samples, subset sizes, and SSMs.

View on arXiv Download PDF AI Translation

cs.CV / 52 / 2609.31103

DepthEvidence: Unifying Metric Depth Prediction and Geometric Reasoning in Multimodal Language Models

DepthEvidence：在多模态语言模型中统一度量深度预测与几何推理

Wei, Jiangning, Yao, Yuan, Cui, Miaomiao, Li, Mingsheng, Zhong, Humen, Bai, Shuai, Yang, Zhibo

Abstract

Spatial reasoning with metric constraints requires linking objects to geometric measurements and preserving their numerical content during language reasoning. We present DepthEvidence, a 4B model that uses its own dense metric predictions as object-grounded evidence for language generation. A camera-conditioned decoder predicts full-resolution metric depth using multi-scale visual features and high-resolution RGB refinement. A dense-to-language interface converts predicted depths and decoder features into object-aligned continuous geometry tokens anchored to object identifiers. Geometric supervision encourages metric information to remain recoverable before and after language-context interaction, while instruction tuning supports object measurement and compositional reasoning. We introduce a Depth-VQA benchmark evaluating object-depth queries, relative comparisons, and decisions combining spatial and numerical constraints. Across nine datasets, DepthEvidence achieves the highest average dense $\delta_1$ among evaluated methods, competitive with specialized estimators. It also leads the evaluated methods in instance-level metric depth estimation and overall accuracy on both relative and metric reasoning tracks, while broadly preserving general VQA performance and improving spatial understanding relative to the base model.

Chinese Translation

具有度量约束的空间推理需要将物体与几何测量联系起来，并在语言推理过程中保持其数值内容。我们提出了DepthEvidence，一个拥有40亿参数的模型，它利用自身的密集度量预测作为物体基础的证据进行语言生成。一个基于相机条件的解码器使用多尺度视觉特征和高分辨率RGB细化来预测全分辨率的度量深度。一个密集到语言的接口将预测的深度和解码器特征转换为与物体标识符锚定的物体对齐的连续几何标记。几何监督鼓励在语言上下文交互之前和之后保持可恢复的度量信息，而指令调优支持物体测量和组合推理。我们引入了一个Depth-VQA基准，评估物体深度查询、相对比较以及结合空间和数值约束的决策。在九个数据集上，DepthEvidence在评估方法中实现了最高的平均密集$ ext{δ}_1$，与专门的估计器具有竞争力。它在实例级度量深度估计和相对及度量推理轨道的整体准确性方面也领先于评估方法，同时广泛保持了总体VQA性能，并相较于基础模型改善了空间理解。

View on arXiv Download PDF AI Translation

cs.CV / 53 / 2609.31108

Double-stream registration with pyramid fusion for HDR video with alternating exposures

基于金字塔融合的双流配准用于交替曝光的高动态范围视频

Martorell, Onofre, Pereira-Sánchez, Ivan, Fuentes, Antoni, Buades, Antoni

Abstract

High dynamic range (HDR) video reconstruction from al\-ter\-na\-ting-exposure sequences remains challenging, especially in regions with extreme luminance variation. We propose a novel HDR reconstruction framework based on dual-stream registration and accurate pyramid fusion. Given three consecutive frames, our method computes optical flow directly with the central frame, while introducing a complementary midpoint displacement strategy to handle cases with severe overexposition. A pyramid fusion stage then merges the resulting radiance and LDR images into a final HDR output. Experimental results demonstrate that our approach consistently outperforms state-of-the-art methods.

Chinese Translation

从交替曝光序列重建高动态范围（HDR）视频仍然具有挑战性，尤其是在亮度变化极端的区域。我们提出了一种基于双流配准和精确金字塔融合的新型HDR重建框架。给定三帧连续图像，我们的方法直接与中心帧计算光流，同时引入了一种补充的中点位移策略，以处理严重过曝的情况。接下来，金字塔融合阶段将生成的辐射和低动态范围（LDR）图像合并为最终的HDR输出。实验结果表明，我们的方法在性能上始终优于最先进的技术。

View on arXiv Download PDF AI Translation

cs.CV / 54 / 2609.31135

Pocket-STVG: lightweight architecture for Spatio-Temporal Video Grounding

Pocket-STVG：用于时空视频定位的轻量级架构

Presta, Alberto, Byra, Michal, Stefański, Grzegorz, Szurkowski, Karol, Kołodziejczyk, Eryk, Arendt, Krzysztof

Abstract

Spatio-Temporal Video Grounding (STVG) aims to localize the spatio-temporal tube in a video corresponding to a natural language query. While recent methods achieve strong performance in fully supervised, weakly supervised, and zero-shot settings, they typically rely on computationally expensive architectures, complex training pipelines, or multimodal large language models. We present Pocket-STVG (P-STVG), a lightweight cascade architecture that addresses STVG by combining efficient pre-trained components instead of large end-to-end models. P-STVG integrates a temporal-aware video encoder based on MobileViCLIP, a spatial encoder-decoder derived from MDETR, and a shared aligned text encoder. Temporal localization is performed through either a lightweight 1D U-Net or a simple thresholding strategy, enabling the same framework to operate in both weakly supervised and zero-shot settings. Furthermore, video representations are precomputed independently of the query, yielding an indexing-friendly pipeline for efficient inference and large-scale video collections. Despite requiring fewer than 90M parameters, P-STVG performs on par with weakly supervised methods and improves on earlier zero-shot approaches at a fraction of their memory and computational cost, establishing a favorable performance-efficiency trade-off for STVG.

Chinese Translation

时空视频定位（Spatio-Temporal Video Grounding, STVG）旨在根据自然语言查询在视频中定位时空管道。尽管近期方法在完全监督、弱监督和零样本设置中取得了良好表现，但它们通常依赖于计算开销大的架构、复杂的训练流程或多模态大型语言模型。我们提出了Pocket-STVG（P-STVG），一种轻量级级联架构，通过结合高效的预训练组件而非大型端到端模型来解决STVG问题。P-STVG集成了基于MobileViCLIP的时间感知视频编码器、源自MDETR的空间编码器-解码器，以及一个共享的对齐文本编码器。时间定位通过轻量级的1D U-Net或简单的阈值策略进行，使得同一框架能够在弱监督和零样本设置中运行。此外，视频表示在与查询独立的情况下预计算，产生了一个便于索引的管道，以实现高效推理和大规模视频集合。尽管P-STVG所需参数少于9000万，但其性能与弱监督方法相当，并在内存和计算成本方面优于早期的零样本方法，为STVG建立了良好的性能与效率的权衡。

View on arXiv Download PDF AI Translation

cs.CV / 55 / 2609.31148

Seeing Semantic Shift: Difference-Aware Sentence-Level Temporal Segmentation of Sign Language Videos

观察语义转变：关注差异的句子级手语视频时间分割

Guo, Bowen, Gan, Shiwei, Yin, Yafeng, Liu, Xiao, Liu, Kuizhuang, Jiang, Zhiwei, Xie, Lei

Abstract

Recent advances in sign language understanding have achieved impressive success on short, single-sentence videos, yet their performance drops sharply when applied to long, continuous sign language videos. To bridge this gap, we focus on a challenging and realistic setting: Visual-only Sentence-level Sign Language Segmentation (Vis-SSLS), which aims to partition continuous sign language videos into non-overlapping sentence-level segments without any caption assistance, serving as a crucial prerequisite for downstream recognition and translation tasks. However, sentence transitions in sign language are often smooth and visually ambiguous, lacking explicit pauses or posture resets. As a result, static frame representations may fail to capture the subtle temporal changes that indicate sentence boundaries. To address this challenge, we propose \textbf{SignShift}, a difference-aware segmentation framework that explicitly models frame-to-frame feature variation as semantic cues for sentence boundary detection. First, to model the feature variation, we design a Temporal Difference Module, which incorporates full-frame, facial, and hand cues, and employs inter-frame differencing to learn multi-scale temporal variations that capture both fine-grained local kinematics and global semantic transitions. Second, to mitigate over- and under-segmentation issues, we design a Segment Count Prediction module, which predicts the number of sentences to guide boundary selection. Extensive experiments on benchmark datasets demonstrate that SignShift substantially outperforms existing methods, validating its effectiveness.

Chinese Translation

近年来，手语理解的进展在短小的单句视频上取得了显著成功，但在长时间连续的手语视频中，其性能却急剧下降。为了解决这一问题，我们关注一个具有挑战性且现实的场景：仅视觉的句子级手语分割（Visual-only Sentence-level Sign Language Segmentation，Vis-SSLS），旨在将连续的手语视频划分为不重叠的句子级片段，而不依赖任何字幕辅助，这为下游的识别和翻译任务提供了重要的前提。然而，手语中的句子过渡往往是平滑且视觉上模糊的，缺乏明确的停顿或姿势重置。因此，静态帧表示可能无法捕捉到指示句子边界的微妙时间变化。为了解决这一挑战，我们提出了 extbf{SignShift}，一个关注差异的分割框架，明确将帧间特征变化建模为句子边界检测的语义线索。首先，为了建模特征变化，我们设计了一个时间差异模块，该模块结合了全帧、面部和手部线索，并采用帧间差分来学习捕捉细粒度局部运动学和全局语义转变的多尺度时间变化。其次，为了减轻过度和不足分割的问题，我们设计了一个句子数量预测模块，该模块预测句子数量以指导边界选择。在基准数据集上的大量实验表明，SignShift显著优于现有方法，验证了其有效性。

View on arXiv Download PDF AI Translation

cs.CV / 56 / 2609.31150

FedHisto-PAST: Parameter-Efficient Stain-Aware Federated Learning for Cross-Site Lung Histopathology Classification

FedHisto-PAST：用于跨站肺组织病理分类的参数高效染色感知联邦学习

Shahriar, Muhammad Muhtasim, Hafiz, M. M. Golam, Aloteibi, Saad, Moni, Mohammad Ali

Abstract

Cross-site lung histopathology classification must account for stain variation, non-IID client data, missing classes, and the cost of adapting large pathology encoders. This study evaluates FedHisto-PAST v2 for three-way classification of adenocarcinoma (ACA), Normal, and squamous cell carcinoma (SCC). FedHisto-PAST v2 combines a frozen HIBOU-B foundation model with parameter-efficient adaptation, stain-conditioned paired-view prediction and feature consistency, reliability-aware prototype learning, and adaptive federated aggregation. Experiments used a five-client, non-IID, raw-data-local simulation with fixed internal evaluation, client-level analysis, component ablations, communication accounting, and a development-influenced exploratory LungHist700 cohort. All principal methods achieved near- ceiling internal performance, which limited discrimination on the fixed split. On LungHist700, FedHisto- PAST v2 achieved a Macro-F1 of 0.728560 and a balanced accuracy of 0.730454. Higher recognition of Normal and SCC was accompanied by lower ACA recall, and calibration remained imperfect. Prediction-level consistency was the only component with a clearly supported independent contribution in the external ablation analysis. Feature consistency and prototype regularization showed no conclusive independent overall gains in Macro-F1. The framework updated 1.253841% of the model parameters. The results provide exploratory cross-dataset evidence for stain-aware, parameter-efficient federation; they do not establish formal privacy, patient-level independence, prospective deployment, or clinical validation.

Chinese Translation

跨站肺组织病理分类必须考虑染色变异、非独立同分布（non-IID）客户端数据、缺失类别以及适应大型病理编码器的成本。本研究评估了 FedHisto-PAST v2 在腺癌（adenocarcinoma, ACA）、正常（Normal）和鳞状细胞癌（squamous cell carcinoma, SCC）三类分类中的表现。FedHisto-PAST v2 结合了一个冻结的 HIBOU-B 基础模型与参数高效的适应、染色条件下的配对视图预测和特征一致性、可靠性感知的原型学习以及自适应联邦聚合。实验使用了五个客户端的非独立同分布（non-IID）原始数据本地模拟，进行了固定内部评估、客户端级分析、组件消融、通信计量，以及受开发影响的探索性 LungHist700 队列。所有主要方法在固定拆分上达到了接近上限的内部性能，限制了区分能力。在 LungHist700 上，FedHisto-PAST v2 实现了 0.728560 的宏 F1 值和 0.730454 的平衡准确率。对正常（Normal）和 SCC 的更高识别伴随着 ACA 召回率的降低，且校准仍然不完善。在外部消融分析中，预测级一致性是唯一一个明确支持独立贡献的组件。特征一致性和原型正则化在宏 F1 上没有显示出明确的独立整体增益。该框架更新了 1.253841% 的模型参数。结果为染色感知、参数高效的联邦学习提供了探索性跨数据集证据；但并未建立正式的隐私、患者级独立性、前瞻性部署或临床验证。

View on arXiv Download PDF AI Translation

cs.CV / 57 / 2609.31154

HyperErase: Scale-Calibrated Hypernetwork for Multi-Concept Erasure in Text-to-Image Models

HyperErase：用于文本到图像模型中多概念擦除的尺度校准超网络

Sun, Yi, Zhong, Xinhao, Zhang, Zhiqi, Zhou, Yimin, Li, Junhao, Qiao, Yuxia

Abstract

Recent advances in text-to-image (T2I) generation have substantially improved visual synthesis, but have also raised increasing safety concerns due to their potential to generate harmful or undesirable content. Existing concept erasure methods predominantly follow a static weight paradigm, producing a single frozen adapter that struggles to adapt to diverse prompt variations and suffers from parameter interference when scaling to multiple concepts. We propose \textbf{HyperErase}, a framework for concept erasure based on hypernetwork-driven prompt-conditioned parameter synthesis. Our approach first reframes concept erasure as prompt-conditioned parameter amortization and trains a hypernetwork to map textual descriptions to prompt-specific LoRA updates, eliminating the need for per-prompt gradient optimization or manual LoRA merging. To further improve the stability and precision of synthesized adapters, we develop a decoupled rectification strategy, which disentangles LoRA tokens into pattern and scale subspaces, applies a square-root transform to curb multiplicative over-scaling, and leverages teacher-derived canonical priors for inference-time correction. Extensive experiments across major concept categories demonstrate that HyperErase consistently improves the trade-off between erasure effectiveness, image quality, and semantic alignment, achieving performance comparable to gold-standard single-concept baselines. Furthermore, the resulting models can provide specialized LoRAs for each input prompt variation in a single forward pass without requiring gradient updates during inference. These principled and flexible framework offers a new paradigm for concept erasure in T2I models.

Chinese Translation

近期文本到图像（T2I）生成的进展显著提升了视觉合成能力，但也因其可能生成有害或不良内容而引发了越来越多的安全担忧。现有的概念擦除方法主要遵循静态权重范式，产生一个固定的适配器，难以适应多样的提示变体，并在扩展到多个概念时遭遇参数干扰。我们提出了 extbf{HyperErase}，一个基于超网络驱动的提示条件参数合成的概念擦除框架。我们的方法首先将概念擦除重新框定为提示条件的参数摊销，并训练一个超网络将文本描述映射到特定提示的LoRA更新，消除了对每个提示的梯度优化或手动LoRA合并的需求。为了进一步提高合成适配器的稳定性和精确性，我们开发了一种解耦的整治策略，将LoRA令牌分解为模式和尺度子空间，应用平方根变换以抑制乘法过度缩放，并利用教师导出的标准先验进行推理时的修正。针对主要概念类别的广泛实验表明，HyperErase在擦除效果、图像质量和语义对齐之间的权衡上始终表现出色，达到了与黄金标准单概念基线相当的性能。此外，所得到的模型可以在单次前向传递中为每个输入提示变体提供专门的LoRA，而无需在推理过程中进行梯度更新。这一原则性和灵活的框架为T2I模型中的概念擦除提供了新的范式。

View on arXiv Download PDF AI Translation

cs.CV / 58 / 2609.31160

ReG-SAM: Reference Graph-Driven SAM for 2D Foundational Vessel Segmentation

ReG-SAM：基于参考图的SAM用于二维基础血管分割

Lyu, Donghang, Zhang, Zichen, Dzyubachyk, Oleh, Staring, Marius

Abstract

Vessel segmentation in medical images is essential for many clinical tasks, ranging from diagnosis to treatment planning. However, it remains challenging due to complex vascular morphology and diverse imaging conditions. Existing deep learning methods rarely aim at building a generalizable vessel segmentor across anatomies and modalities. While the Seg- ment Anything Model (SAM) has shown promise for med- ical image segmentation, its original design does not fully exploit vascular morphology and struggles with fine-grained vascular structures, leading to suboptimal performance. In this paper, we propose ReG-SAM, a SAM-based framework tailored to 2D vessel segmentation that leverages reference graph set for enhancing vascular representations. Specifically, we introduce two modality-aware representations derived from the reference masks: graph prompt embeddings (GPEs) that encode global spatial features from graphs, and vascu- lar prototype embeddings (VPEs) that capture fine-grained modality-specific vessel characteristics from multi-scale fea- ture maps and vascular masks. Since both require vascular masks that are unavailable during inference and require robust modality-aware vascular feature representations, we construct a modality-wise vascular database and develop two reference graph-guided representation learning schemes for estimating GPEs and VPEs using samples from the database rather than ground-truth masks. Extensive experiments across 19 datasets demonstrate that ReG-SAM consistently outperforms existing baselines, even those using manual prompts, particularly on challenging thin vessels.

Chinese Translation

医学图像中的血管分割对许多临床任务至关重要，涵盖从诊断到治疗规划。然而，由于复杂的血管形态和多样的成像条件，这一任务仍然具有挑战性。现有的深度学习方法很少旨在构建一个能够跨解剖结构和成像模式通用的血管分割器。尽管Segment Anything Model (SAM) 在医学图像分割中展现了良好的前景，但其原始设计并未充分利用血管形态，并且在处理细粒度血管结构时表现不佳，导致性能不理想。本文提出了ReG-SAM，这是一种基于SAM的框架，专门针对二维血管分割，利用参考图集来增强血管表示。具体而言，我们引入了两种基于模式的表示，这些表示源自参考掩膜：图提示嵌入（graph prompt embeddings, GPEs）用于编码来自图的全局空间特征，以及血管原型嵌入（vascular prototype embeddings, VPEs）用于捕捉来自多尺度特征图和血管掩膜的细粒度模式特异性血管特征。由于这两者都需要在推理过程中不可用的血管掩膜，并且需要稳健的模式感知血管特征表示，我们构建了一个按模式划分的血管数据库，并开发了两种参考图引导的表示学习方案，以利用数据库中的样本来估计GPEs和VPEs，而不是依赖于真实掩膜。通过在19个数据集上的广泛实验，ReG-SAM始终优于现有基线，甚至包括使用手动提示的方法，特别是在处理挑战性的细血管时。

View on arXiv Download PDF AI Translation

cs.CV / 59 / 2609.31170

TaskIR: Task-Driven Image Restoration via Degradation Adaptation and Task Feedback

TaskIR：通过降级适应和任务反馈的任务驱动图像恢复

Tu, Yanjie, Yan, Qingsen, Niu, Axi, Cai, Wenxuan, Hu, Tao, Dong, Wei, Zhang, Haokui

Abstract

Task-driven image restoration aims to improve both image quality and downstream task performance. However, existing methods predominantly focus on single degradation type and struggle to handle the diverse degradations encountered in real-world scenarios. Different degradations impose distinct restoration demands, and insufficient restoration may leave residual degradations and artifacts that impair object boundaries and semantic cues, thereby compromising downstream task performance. To address these challenges, we propose TaskIR, a two-stage task-driven unified image restoration framework that integrates degradation-adaptive restoration with task feedback refinement. In Stage I, a Degradation Representation Module (DRM) extracts degradation representations, enabling a Degradation-Guided Transformer Block (DGTB) to dynamically modulate feature transformations for adaptive restoration. In Stage II, a Task-to-Restoration Feedback Generation module (TRFG) transforms heterogeneous task features into restoration feedback by modeling task-representation discrepancies associated with the current restoration. Subsequently, a Selective Task Feedback Refinement module (STFR) assesses feedback relevance and selectively refines intermediate restoration features to mitigate interference with well-restored content. Extensive experiments demonstrate that TaskIR achieves competitive restoration quality and downstream task performance across diverse degradations and tasks.

Chinese Translation

任务驱动的图像恢复旨在提高图像质量和下游任务性能。然而，现有方法主要集中于单一降级类型，难以处理现实场景中遇到的多样化降级。不同的降级对恢复提出了不同的需求，恢复不足可能会留下残余降级和伪影，损害物体边界和语义线索，从而影响下游任务性能。为了解决这些挑战，我们提出了TaskIR，一个两阶段的任务驱动统一图像恢复框架，结合了降级自适应恢复和任务反馈精炼。在第一阶段，降级表示模块（Degradation Representation Module, DRM）提取降级表示，使降级引导变换块（Degradation-Guided Transformer Block, DGTB）能够动态调节特征变换以实现自适应恢复。在第二阶段，任务到恢复反馈生成模块（Task-to-Restoration Feedback Generation, TRFG）通过建模与当前恢复相关的任务表示差异，将异构任务特征转化为恢复反馈。随后，选择性任务反馈精炼模块（Selective Task Feedback Refinement, STFR）评估反馈相关性，并选择性地精炼中间恢复特征，以减轻对恢复良好内容的干扰。大量实验表明，TaskIR在多样化降级和任务中实现了具有竞争力的恢复质量和下游任务性能。

View on arXiv Download PDF AI Translation

cs.CV / 60 / 2609.31193

Who Says What: Symbolic Trimodal Binding Mechanisms in Audio-Visual LLMs

谁说什么：音视频大型语言模型中的符号三模态绑定机制

Jung, Jihoo, Jang, Youngjoon, Chung, Joon Son

Abstract

Current Audio-Visual LLMs (AVLLMs) struggle with reasoning over videos featuring multi-speaker dialogues. In such videos, resolving "who says what" is crucial, which necessitates trimodal (text-audio-visual) binding. Motivated by these challenges, we systematically investigate how this trimodal binding is achieved in AVLLMs. Specifically, we identify emergent symbolic trimodal binding mechanisms in AVLLMs that utilize modality-specific symbolic variables. By encoding auditory and visual components into symbolic variables-capturing temporal utterance sequences and spatial entity coordinates, respectively-the model establishes cross-modal linking within this abstract space. Crucially, we reveal that when trimodal binding fails, the breakdown predominantly stems from misaligned audio-visual connections. To overcome this bottleneck, we introduce an audio-visual prompting method utilizing an off-the-shelf Active Speaker Detection (ASD) model. By simply overlaying visual bounding boxes on active speakers, this training-free approach yields immediate performance gains across four conversation-centric benchmarks. Moreover, lightweight fine-tuning of fewer than 300 steps on these ASD-prompted-videos extends these gains to three general AV benchmarks, suggesting the generalizability of our method.

Chinese Translation

当前的音视频大型语言模型（AVLLMs）在处理包含多说话者对话的视频时面临推理困难。在此类视频中，解决“谁说什么”至关重要，这需要三模态（文本-音频-视觉）绑定。基于这些挑战，我们系统地研究了AVLLMs中如何实现这种三模态绑定。具体而言，我们识别出AVLLMs中利用特定模态的符号变量所产生的符号三模态绑定机制。通过将听觉和视觉组件编码为符号变量——分别捕捉时间上的发言序列和空间上的实体坐标——模型在这一抽象空间中建立了跨模态链接。重要的是，我们揭示了当三模态绑定失败时，主要是由于音视频连接不对齐所导致。为了克服这一瓶颈，我们引入了一种利用现成的主动说话者检测（ASD）模型的音视频提示方法。通过简单地在活跃说话者上叠加视觉边界框，这种无训练的方法在四个以对话为中心的基准测试中立即提高了性能。此外，在这些ASD提示视频上进行少于300步的轻量级微调，将这些性能提升扩展到三个通用的音视频基准，表明我们方法的普适性。

View on arXiv Download PDF AI Translation

cs.CV / 61 / 2609.31198

Light Field Primitive for Novel View Synthesis

用于新视图合成的光场原语

Chen, Liang, Ning, Jiahui, Jiang, Xun, Xu, Xing, Ren, Jimmy, Fan, Fenglei, Shen, Heng Tao

Abstract

We present Light Field Primitives (LFP), a formulation for novel view synthesis that replaces the dense ray database with a compact set of differentiable primitives in the classical two-plane parameterization. Each primitive condenses a group of rays into one learned record, and its response to a query is governed by how closely that query belongs to the group. Rendering a camera ray then reduces to compositing all responses it elicits, and a scene can be optimized directly from posed images and rendered in real time with rays. Beyond its competitive performance on standard benchmarks, the main advantage of LFP is structural: its primitives reside directly in the 4D ray space, so optical and appearance effects that are already operations on the light field become behaviors of a single shared renderer. With minimal changes to that renderer, LFP supports multi-scale anti-aliasing, defocus deblurring with refocusing, rendering for fisheye cameras, and even transparent object reconstruction with ray refraction, matching specialized frameworks that devote substantial machinery to these effects.

Chinese Translation

我们提出了光场原语（Light Field Primitives, LFP），这是一种新视图合成的表述方法，它用一组紧凑的可微分原语替代了传统的密集光线数据库，采用经典的双平面参数化。每个原语将一组光线凝聚成一个学习记录，其对查询的响应由该查询与该组的相似程度决定。渲染相机光线则简化为合成其引发的所有响应，并且可以直接从已拍摄图像中优化场景，并实时渲染光线。除了在标准基准测试中表现出竞争力外，LFP的主要优势在于其结构性：其原语直接位于四维光线空间中，因此已经在光场上进行的光学和外观效果变成了单一共享渲染器的行为。通过对该渲染器进行最小的修改，LFP支持多尺度抗锯齿、带重聚焦的散焦去模糊、鱼眼相机的渲染，甚至通过光线折射进行透明物体重建，匹配专门为这些效果投入大量机制的框架。

View on arXiv Download PDF AI Translation

cs.CV / 62 / 2609.31202

Preserve-and-Compose Training for Composed Image Retrieval

保留与组合训练用于组合图像检索

Kwon, Sehyun

Abstract

Composed image retrieval (CIR) aims to retrieve images that satisfy a user-specified modification while preserving relevant visual content from a reference image. Collecting target images for this purpose is costly, motivating zero-shot CIR methods that instead use target captions as supervision. However, target captions may omit source details that should be preserved. We therefore propose, Preserve-and-Compose Training, which complements target-caption supervision with visual evidence from the source image. PACT learns from image--text--text (ITT) triplets without target images or gallery updates, aligning composed queries with target captions while preserving source evidence through visual supervision. We further introduce Chord scoring, which combines target similarity with source-relative directional agreement in the frozen image space. Results across four ZS-CIR benchmarks show that combining target-caption supervision with source-image evidence leads to strong retrieval performance across datasets, backbone scales, and external galleries. The code is available on https://github.com/sehyunkwon/PACT.

Chinese Translation

组合图像检索（CIR）旨在检索满足用户指定修改的图像，同时保留参考图像中的相关视觉内容。为了这一目的，收集目标图像的成本较高，因此激励了零样本CIR方法，这些方法使用目标描述作为监督。然而，目标描述可能会省略应当保留的源细节。因此，我们提出了保留与组合训练（Preserve-and-Compose Training，PACT），该方法通过源图像的视觉证据来补充目标描述的监督。PACT 从图像-文本-文本（image--text--text，ITT）三元组中学习，而无需目标图像或图库更新，同时通过视觉监督将组合查询与目标描述对齐，并保留源证据。我们进一步引入了和弦评分（Chord scoring），该评分将目标相似性与源相对方向一致性结合在冻结图像空间中。在四个零样本CIR基准测试中的结果表明，将目标描述监督与源图像证据结合能够在不同数据集、骨干网络规模和外部图库中实现强大的检索性能。代码可在 https://github.com/sehyunkwon/PACT 获取。

View on arXiv Download PDF AI Translation

cs.CV / 63 / 2609.31234

WeaveAgent: A Two-Stage Tool-Routing Agent for Ultra-High-Resolution Remote Sensing Imagery

WeaveAgent：一种用于超高分辨率遥感图像的两阶段工具路由代理

Pang, Zhongyu

Abstract

Problem. Ultra-high-resolution (UHR) remote sensing with vague user intents has two bottlenecks: visual tokens are expensive, and tool calling must be format-reliable (pretrained models emit zero tool calls zero-shot). Method. WeaveAgent, a two-stage tool-routing agent, decouples routing from visual perception. Stage A is routing-first: emission is trained, not elicited. Stage B executes conditionally: intrinsic queries enter visual answering (full-scene thumbnail; a WeaveEarth-style evidence board as an optional fixed-budget, approx. 5k-token compression interface); extrinsic queries execute tool call on original full-resolution imagery, answering from tool observations in a second, observation-masked round. Training: alignment SFT, then GRPO under reward R_WA2. Results. Alignment SFT lifts extrinsic routing from 0% to 80.75% (323/400); GRPO suppresses 9 intrinsic mis-emissions while tool selection is unchanged. The trained 2B system does not beat the zero-shot 8B baseline overall (0.263 vs. 0.250), a diagnostic contribution. Oracle attribution separates two repair ingredients: loading the observation into context lifts extrinsic answer accuracy from 0.025 to 0.425 under marker-free cross-mode returns, and the two-turn SFT stage adds a further +9.3 points to 0.518 at a small routing cost. A +/- image ablation shows emission suppression is visually grounded, and a query-register matrix shows LLM-rewritten queries cost trained checkpoints 2-11 points. Scope. All training and evaluation use the 5,000 / 3,273 / 1,000-record VagueUHR corpus (600 intrinsic + 400 tool-requiring; the base seeds synthesis and is not used for optimization). Single-pass evidence construction runs at 7.31 s per image on an RTX 4090. Code, data, and evaluation protocols will be released.

Chinese Translation

问题：具有模糊用户意图的超高分辨率（UHR）遥感面临两个瓶颈：视觉标记昂贵，工具调用必须格式可靠（预训练模型在零样本情况下发出零个工具调用）。方法：WeaveAgent是一种两阶段工具路由代理，将路由与视觉感知解耦。阶段A为路由优先：发射是经过训练的，而不是被引发的。阶段B根据条件执行：内在查询进入视觉回答（全场景缩略图；WeaveEarth风格的证据板作为可选的固定预算，约5k标记压缩接口）；外在查询在原始全分辨率图像上执行工具调用，从工具观察中回答，在第二轮观察掩蔽中进行。训练：对齐SFT，然后在奖励R_WA2下进行GRPO。结果：对齐SFT将外在路由从0%提升至80.75%（323/400）；GRPO抑制了9个内在错误发射，而工具选择保持不变。经过训练的2B系统在整体上未能超越零样本8B基线（0.263对0.250），这是一个诊断性贡献。Oracle归因分离了两个修复成分：将观察加载到上下文中使外在答案准确率在无标记交叉模式返回下从0.025提升至0.425，而两轮SFT阶段在小的路由成本下进一步增加了9.3分，达到0.518。正负图像消融显示发射抑制是视觉基础的，查询注册矩阵显示LLM重写的查询使训练检查点损失2-11分。范围：所有训练和评估使用5,000 / 3,273 / 1,000记录的VagueUHR语料库（600个内在+400个需要工具的；基础种子合成且未用于优化）。单通道证据构建在RTX 4090上每张图像运行7.31秒。代码、数据和评估协议将被发布。

View on arXiv Download PDF AI Translation

cs.CV / 64 / 2609.31247

Geometric Inconsistency Localization in Multi-View Image Sets

多视角图像集中的几何不一致性定位

Staelens, Xander, Loos, Albéric, Ramlot, Bert, Mareen, Hannes, Lambert, Peter, Van Wallendael, Glenn

Abstract

Novel view synthesis (NVS) models can produce realistic new views of the same scene from different viewpoints. However, these generated views are not always geometrically consistent with one another. Multi-view (MV) consistency has shown promise as a tool for evaluating these NVS models. Its potential for multimedia forensics, however, remains largely unexplored, particularly for localizing geometric inconsistencies across wide-baseline image pairs. To enable research in this direction, we introduce DeformView, a wide-baseline MV dataset with pixel-level annotations of geometric inconsistencies. Using DeformView, we evaluate state-of-the-art MV consistency-scoring methods and show that approaches developed for NVS evaluation transfer poorly to the forensic task of geometric inconsistency localization. To address this limitation, we propose DEFECt3R, a lightweight learning-based classifier that uses cross-view feature relationships to localize geometric inconsistencies at the pixel level. By learning from explicit supervision, including hard negatives from geometrically consistent yet deformed views, DEFECt3R improves localization performance and substantially reduces false positives compared to existing consistency-scoring methods. Ablation experiments further show that both feature representations and correspondence quality contribute to localization performance. Overall, our findings demonstrate that MV geometric consistency is a promising yet underexplored signal for multimedia forensics and establish a benchmark and baseline for geometric inconsistency localization in wide-baseline MV image pairs. Code and dataset are available at https://github.com/IDLabMedia/DeformView-DEFECt3R

Chinese Translation

新视图合成（Novel View Synthesis, NVS）模型能够从不同视角生成同一场景的逼真新视图。然而，这些生成的视图并不总是彼此在几何上保持一致。多视角（Multi-View, MV）一致性已被证明是评估这些NVS模型的有效工具。然而，其在多媒体取证中的潜力仍然未被充分探索，特别是在宽基线图像对中定位几何不一致性方面。为推动这一研究方向，我们引入了DeformView，一个具有像素级几何不一致性标注的宽基线MV数据集。利用DeformView，我们评估了最先进的MV一致性评分方法，并显示出为NVS评估开发的方法在几何不一致性定位的取证任务中表现不佳。为了解决这一局限性，我们提出了DEFECt3R，一个轻量级的基于学习的分类器，利用跨视角特征关系在像素级定位几何不一致性。通过从显式监督中学习，包括来自几何一致但变形视图的困难负样本，DEFECt3R提高了定位性能，并显著减少了与现有一致性评分方法相比的误报。消融实验进一步表明，特征表示和对应质量均对定位性能有贡献。总体而言，我们的研究结果表明，MV几何一致性是一个有前景但尚未充分探索的多媒体取证信号，并为宽基线MV图像对中的几何不一致性定位建立了基准和基础。代码和数据集可在 https://github.com/IDLabMedia/DeformView-DEFECt3R 获取。

View on arXiv Download PDF AI Translation

cs.CV / 65 / 2609.31248

Gauss What You Need: Compact Gaussian Splatting Across Scene Scales

高斯所需：跨场景尺度的紧凑高斯点云渲染

Boudaoud, Afif, Liu, Jiayi, Calotoiu, Alexandru, Hoefler, Torsten

Abstract

3D Gaussian Splatting reconstructs a scene as a collection of Gaussian primitives from a set of posed photographs called the capture. The number of primitives used to represent the scene affects reconstruction quality, storage, and rendering cost. How to select this number automatically across capture scales remains unresolved: configurations effective on standard benchmarks can leave larger captures with too few Gaussians to reconstruct fine details. We observe that the surface to represent, given by the capture's extent and resolution, is known before training, whereas its content complexity becomes apparent during training, through the reconstruction quality on the training views. We introduce TangoGS, which combines capture-derived model sizing with training-based adaptation: the capture determines the scale of the model, and training feedback determines its final size within that scale. Before training, TangoGS derives a learning allowance for model growth from the capture's total pixels after discounting views that re-observe the same scene points. During training, reconstruction quality guides how many Gaussians to add and remove. On 13 standard benchmark scenes, TangoGS matches the mean PSNR of the best-performing evaluated baseline, LeGS, with $48\%$ fewer Gaussians. On eight large captures, the same configuration automatically scales to larger models when necessary, achieving the highest mean PSNR among evaluated methods: $0.54$ dB above the runner-up with $2.3\times$ as many Gaussians. Together, capture-derived learning allowances and training-quality guided density control enable a state-of-the-art quality--size compromise across scene scales without retuning.

Chinese Translation

3D 高斯点云渲染将场景重建为一组来自一组已定位照片（称为捕捉）的高斯原语。用于表示场景的原语数量会影响重建质量、存储和渲染成本。如何在捕捉尺度之间自动选择这个数量仍然没有解决方案：在标准基准测试中有效的配置可能会导致较大的捕捉在重建细节时使用的高斯数量过少。我们观察到，表示的表面由捕捉的范围和分辨率决定，这在训练之前是已知的，而其内容复杂性则在训练过程中通过训练视图的重建质量显现出来。我们引入了 TangoGS，它结合了基于捕捉的模型大小调整与基于训练的适应：捕捉决定模型的尺度，而训练反馈决定其在该尺度内的最终大小。在训练之前，TangoGS 从捕捉的总像素中推导出模型增长的学习允许量，剔除重新观察相同场景点的视图。在训练过程中，重建质量指导添加和移除多少高斯。在 13 个标准基准场景上，TangoGS 的平均 PSNR 与表现最佳的评估基线 LeGS 相匹配，但使用的高斯数量减少了 48%。在八个大型捕捉中，相同的配置在必要时自动扩展到更大的模型，在评估方法中实现了最高的平均 PSNR：比第二名高出 0.54 dB，且使用的高斯数量是其 2.3 倍。结合捕捉导出的学习允许量和基于训练质量的密度控制，使得在不同场景尺度之间实现了无须重新调整的最先进的质量与规模的折衷。

View on arXiv Download PDF AI Translation

cs.CV / 66 / 2609.31285

MoTop: Motion-Topological Model For Micro AU Detection

MoTop：用于微表情动作单元检测的运动-拓扑模型

Khor, Huai-Qian, Wei, Mengting, Li, Yante, Loo, Chu Kiong, Zhao, Guoying

Abstract

Facial micro-expressions are spontaneous, brief, and subtle facial movements that reveal suppressed emotions in high-stakes environments. In contrast to classic expression analysis, detecting action unit (AU) yields a finer representation of facial movements, serving as a preliminary step before defining expression classes and other downstream tasks. Therefore, it represents a crucial upstream task in facial analysis, and improving an AU detection module increases the precision of facial analysis. Despite that, detecting AU is challenging because of the constrictive nature of the AU activation regions, leading to confusion among different AUs known as AU ambiguity. To model the fine-scale changes, we propose \textbf{MoTop}, a motion-topological model that is augmented with a learnable motion context, yielding regional soft guidance for facial activity, followed by facial landmarks that capture the fine-scale topological changes of micro AUs. To increase the micro facial landmark representations, we amplify the encoded facial landmark transitions via linear extrapolation, thereby increasing the spatial proximity of landmarks and enhancing the low-intensity landmark dynamics. In addition, we design anatomical facial clusters that enhance the hierarchical representation, facilitating multi-scale modelling of facial geometry and improving micro-topological representations. With these contributions, we have achieved state-of-the-art performance on the CD6ME protocol for the micro AU detection task.

Chinese Translation

面部微表情是自发的、短暂的和微妙的面部运动，能够揭示在高风险环境中被压抑的情感。与经典的表情分析相比，检测动作单元（AU）能够提供更细致的面部运动表现，作为定义表情类别和其他下游任务的初步步骤。因此，这在面部分析中代表了一个至关重要的上游任务，改进AU检测模块可以提高面部分析的精度。然而，由于AU激活区域的限制性特征，AU检测面临挑战，这导致不同AU之间的混淆，称为AU歧义。为了对细微变化进行建模，我们提出了 extbf{MoTop}，一种增强了可学习运动上下文的运动-拓扑模型，为面部活动提供区域软指导，随后通过面部关键点捕捉微AU的细微拓扑变化。为了增强微面部关键点的表现，我们通过线性外推放大编码的面部关键点过渡，从而增加关键点的空间接近度并增强低强度关键点的动态。此外，我们设计了增强层次表示的解剖面部聚类，促进面部几何的多尺度建模并改善微拓扑表示。凭借这些贡献，我们在微AU检测任务的CD6ME协议上实现了最先进的性能。

View on arXiv Download PDF AI Translation

cs.CV / 67 / 2609.31298

UniAR: A Unified Framework for Autism Recognition Enhanced by Multi-View Prompt Learning

UniAR：一种通过多视角提示学习增强的自闭症识别统一框架

Xin, Lei, Wang, Zeheng, Zhu, Jiayin, Huang, Shihong, Zeng, Fanhu, Jiang, Changjiang, He, Dengbo, Yue, Yutao, Kong, Zhenglun

Abstract

Autism Spectrum Disorder (ASD) is a complex neurodevelopmental disorder for which early and accurate diagnosis is critical to improving long-term developmental outcomes. However, existing ASD recognition methods are often constrained by the scarcity of diagnostic text data, forcing them to rely mainly on visual analysis and limiting their ability to model clinically meaningful semantic reasoning. To address this challenge, we propose UniAR, a unified framework enhanced by multi-granularity prompt learning for robust ASD recognition under heterogeneous data variations. Specifically, UniAR leverages a large multimodal model to generate hierarchical diagnostic descriptions at the word, phrase, and sentence levels, compensating for the lack of paired clinical reports. To align the generated semantics with visual evidence, we further design a Mixture-of-Experts-based Multi-Scale Alignment Module, which dynamically matches vector-quantized visual prototypes with semantic representations at corresponding granularities. Extensive experiments on four benchmarks covering brain MRI and facial expression scenarios show that UniAR consistently outperforms existing state-of-the-art methods, achieving average accuracies of 75.9\% on MRI benchmarks and 91.6\% on facial benchmarks, while improving average Accuracy on MRI benchmarks by 1.5 percentage points and average Accuracy on facial benchmarks by 1.2 percentage points over baselines. These results demonstrate that UniAR offers a robust and interpretable framework for ASD screening under semantic scarcity.

Chinese Translation

自闭症谱系障碍（ASD）是一种复杂的神经发育障碍，早期和准确的诊断对于改善长期发展结果至关重要。然而，现有的ASD识别方法常常受到诊断文本数据稀缺的限制，迫使它们主要依赖视觉分析，从而限制了其建模临床有意义的语义推理的能力。为了解决这一挑战，我们提出了UniAR，这是一种通过多粒度提示学习增强的统一框架，旨在实现对异构数据变异下的稳健ASD识别。具体而言，UniAR利用一个大型多模态模型生成分层的诊断描述，涵盖词、短语和句子层级，以弥补配对临床报告的不足。为了将生成的语义与视觉证据对齐，我们进一步设计了一个基于专家混合的多尺度对齐模块，该模块动态匹配向量量化的视觉原型与相应粒度的语义表示。在涵盖脑部MRI和面部表情场景的四个基准测试上的广泛实验表明，UniAR始终优于现有的最先进方法，在MRI基准测试上实现了75.9%的平均准确率，在面部基准测试上实现了91.6%的平均准确率，同时在MRI基准测试上提高了1.5个百分点的平均准确率，在面部基准测试上提高了1.2个百分点的平均准确率。这些结果表明，UniAR为在语义稀缺情况下的ASD筛查提供了一个稳健且可解释的框架。

View on arXiv Download PDF AI Translation

cs.CV / 68 / 2609.31314

CytoSPM: Open-Vocabulary Cytopathology Detection with Structured Prompt Bank

CytoSPM：基于结构化提示库的开放词汇细胞病理检测

Li, Wenjie, Xu, Zishan, Huang, Jinyang, Nie, Zhengxin, Kan, Shichao, Liang, Yixiong

Abstract

Cytopathology detection requires open-vocabulary recognition because cellular categories are fine-grained, long-tailed, and continuously evolving across different organ systems. However, existing cytology detectors are mostly single-domain and closed-set, and there is still no unified benchmark for evaluating open-vocabulary cytopathology detection. We present PentaCyto, a multi-domain benchmark covering cervical, urinary, respiratory, serous fluid, and thyroid cytology, with 24 base categories and 9 held-out novel categories. Each category is associated with structured cytomorphology prompts that describe diagnostic morphological attributes and provide clinically grounded textual knowledge. We further propose CytoSPM, an efficient detector based on a decoupled two-stage design. It first extracts reusable class-agnostic visual representations, and then performs class-aware structural prompt matching with class names and cytomorphology prompts. On PentaCyto, CytoSPM outperforms existing methods in novel-category detection and open-vocabulary detection while maintaining efficient inference.

Chinese Translation

细胞病理检测需要开放词汇识别，因为细胞类别具有细粒度、长尾分布且在不同器官系统中不断演变。然而，现有的细胞学检测器大多为单域且闭集，且尚无统一的基准用于评估开放词汇细胞病理检测。我们提出了PentaCyto，这是一个涵盖宫颈、泌尿、呼吸、浆液和甲状腺细胞学的多域基准，包含24个基础类别和9个保留的新颖类别。每个类别均配备结构化的细胞形态学提示，描述诊断性形态特征并提供临床基础的文本知识。我们进一步提出了CytoSPM，一种基于解耦两阶段设计的高效检测器。其首先提取可复用的类别无关视觉表征，随后利用类别名称和细胞形态学提示执行类别感知的结构化提示匹配。在PentaCyto基准上，CytoSPM在新类别检测和开放词汇检测方面均优于现有方法，同时保持高效的推理性能。

View on arXiv Download PDF AI Translation

cs.CV / 69 / 2609.31326

CG-HAF: An Interpretable Global-Local Lesion-Burden Fusion Framework for Ordinal Acne Severity Grading in Agentic Skincare Support

CG-HAF：一种可解释的全局-局部病变负担融合框架，用于代理护肤支持中的序数痤疮严重程度分级

Shahriar, Muhammad Muhtasim, Borno, Md. Naimur Asif, Aloteibi, Saad, Moni, Mohammad Ali

Abstract

Ordinal acne severity grading requires distinguishing visually similar neighboring grades while jointly weighing holistic facial appearance and localized lesion burden - evidence that most existing approaches collapse into a single opaque representation. We introduce CG-HAF, a global-local fusion framework that instead keeps this evidence explicit: averaged holistic severity probabilities from independently trained classifiers are combined with structured lesion-burden descriptors from an object detector (lesion count, detection confidence, lesion area) into a compact representation, from which a lightweight, interpretable classifier produces the final grade. On a widely used benchmark, this fusion yields a clear, statistically supported improvement over global-evidence-only baselines, with the largest gains on the most severe cases. Testing on an independent dataset with a different grading standard shows that strong within-dataset performance does not transfer automatically, and a follow-up diagnostic attributes much of this gap to mismatched grading criteria rather than detection failure alone. These findings support interpretable global-local fusion as an effective strategy for ordinal acne grading while highlighting criterion alignment as key to cross-dataset portability, with a further illustration of how the resulting severity signal can support transparent, non-diagnostic decision-making in skincare applications.

Chinese Translation

序数痤疮严重程度分级需要区分视觉上相似的相邻等级，同时综合考虑整体面部外观和局部病变负担——这表明大多数现有方法都简化为单一的不透明表示。我们提出了CG-HAF，这是一种全局-局部融合框架，明确保留了这一证据：来自独立训练分类器的平均整体严重性概率与来自目标检测器的结构化病变负担描述符（病变数量、检测置信度、病变面积）结合，形成紧凑的表示，从中生成一个轻量级的可解释分类器以产生最终等级。在一个广泛使用的基准测试中，这种融合方法显著提高了统计支持的结果，相较于仅基于全局证据的基线，尤其在最严重的病例中取得了最大的增益。在一个具有不同分级标准的独立数据集上的测试表明，强大的数据集内部表现并不会自动转移，后续诊断将这一差距的很大一部分归因于不匹配的分级标准，而不仅仅是检测失败。这些发现支持可解释的全局-局部融合作为序数痤疮分级的有效策略，同时强调标准一致性是跨数据集可移植性的关键，并进一步说明了如何利用所得到的严重性信号支持护肤应用中的透明、非诊断性决策。

View on arXiv Download PDF AI Translation

cs.CV / 70 / 2609.31349

DyMD: Preserving Interaction Dynamics through Distribution Matching Distillation in Few-Step Video World Models

DyMD：通过分布匹配蒸馏在少步视频世界模型中保持交互动态

Xu, Haojun, Huang, Jie, Lu, Xin, Zhong, Mingchen, Fan, Zihao, Huang, Linjiang, Liu, Si

Abstract

Large video diffusion models offer expressive priors for embodied prediction and learning, yet their many-step sampling remains costly for interactive downstream use. Distribution Matching Distillation (DMD) enables few-step video generation, but can suppress robot--object motion while preserving visual quality. Examining DMD's teacher and fake-score signals, we find that weak re-noising keeps the teacher posterior concentrated near motion-deficient rollouts, limiting motion-restoring guidance. Meanwhile, stronger-motion rollouts tend to incur larger fake-score fitting errors, which can hinder the generator's learning of interaction dynamics. We propose DyMD, a DMD framework that adapts both teacher supervision and critic fitting to the evolving student. Temporal affinity--conditioned re-noise sampling adapts the timestep distribution to each rollout's current interaction fidelity by mixing the base schedule with a teacher prior motivated by local posterior variation, thereby balancing motion recovery and appearance refinement. To better track stronger-motion rollouts, dynamics-guided fake-score tracking uses a noise-conditioned predictor to estimate noise-relative fitting difficulty from latent temporal dynamics, then upweights predicted-hard rollouts in the critic loss. Using DyMD, we distill a 14B teacher into a four-step 1.3B student with no auxiliary modules at inference. On embodied-video benchmarks, the student improves R-Bench task adherence by $9.6$ percentage points and PAI-Bench-G Domain score by $5.1$ points over Base DMD while maintaining comparable visual quality. As a backbone for downstream action planning, our student achieves 34% mean success across two WorldArena tasks, compared with 16% for Base DMD.

Chinese Translation

大型视频扩散模型为具身预测和学习提供了丰富的先验，但其多步采样在交互下游应用中仍然成本高昂。分布匹配蒸馏（Distribution Matching Distillation, DMD）使得少步视频生成成为可能，但可能会抑制机器人与物体的运动，同时保持视觉质量。通过分析DMD的教师和伪评分信号，我们发现弱重噪声使得教师后验集中在运动不足的生成结果附近，从而限制了运动恢复的指导。同时，强运动生成结果往往会产生较大的伪评分拟合误差，这可能会阻碍生成器对交互动态的学习。我们提出了DyMD，这是一种DMD框架，能够根据不断发展的学生调整教师监督和评论员拟合。时间亲和性条件下的重噪声采样通过将基础调度与受局部后验变化驱动的教师先验混合，适应每个生成结果当前的交互保真度，从而平衡运动恢复和外观精细化。为了更好地跟踪强运动生成结果，动态引导的伪评分跟踪使用噪声条件预测器来估计来自潜在时间动态的噪声相关拟合难度，然后在评论员损失中对预测的困难生成结果进行加权。使用DyMD，我们将一个140亿参数的教师蒸馏为一个四步1.3亿参数的学生，在推理时没有辅助模块。在具身视频基准测试中，学生在R-Bench任务遵循度上提高了9.6个百分点，在PAI-Bench-G领域得分上提高了5.1分，相比基础DMD保持了相当的视觉质量。作为下游动作规划的基础，我们的学生在两个WorldArena任务中实现了34%的平均成功率，而基础DMD为16%。

View on arXiv Download PDF AI Translation

cs.CV / 71 / 2609.31356

Open Vocabulary Domain Unlearning

开放词汇领域遗忘

Udupa, Sumanth, Harandi, Mehrtash, Luo, Yadan, Baktashmotlagh, Mahsa

Abstract

Vision-Language Models (VLMs) exhibit remarkable zero-shot generalization, yet they often encode unwanted or hazardous stylistic domains such as idealized textbook diagrams in medical AI or cartoon vehicles in autonomous driving. Approximate Domain Unlearning (ADU) aims to selectively erase a model's recognition of a target visual domain while preserving accuracy on the remaining domains. However, existing ADU methods operate under a flawed closed-vocabulary assumption: they evaluate unlearning solely on the specific object classes seen during the unlearning fine-tuning phase. Consequently, these methods do not unlearn the domain itself; they merely overfit to seen class-domain pairs, leaving the domain easily recognizable for unseen classes and providing a false sense of removal. We argue that true domain erasure must be class-agnostic. To address this, we formalize Open-Vocabulary Domain Unlearning (OVDU), a rigorous protocol that mandates domain forgetting must transfer to held-out classes. To solve the OVDU challenge, we propose a surgical parameter-editing framework. First, a Fisher Information mask isolates domain-sensitive weights, mathematically protecting foundational zero-shot generalization. Second, our Targeted Manifold Scattering (TMS) objective uses preference-based mining to locally scatter the forget domain's stylistic geometry. Evaluated across PACS, OfficeHome, and DomainNet, our method vastly improves open-vocabulary generalization over existing baselines. Crucially, it delivers exceptional sample efficiency, outperforming peak 8-shot baseline results with only 4 shots.

Chinese Translation

视觉-语言模型（VLMs）展现了显著的零-shot 泛化能力，但它们往往编码了不必要或有害的风格领域，例如医疗人工智能中的理想化教科书图示或自动驾驶中的卡通车辆。近似领域遗忘（Approximate Domain Unlearning, ADU）旨在选择性地抹去模型对目标视觉领域的识别，同时保持对其余领域的准确性。然而，现有的 ADU 方法在一个有缺陷的封闭词汇假设下运作：它们仅在遗忘微调阶段评估所见的特定对象类别。因此，这些方法并没有真正遗忘领域本身；它们只是过拟合于已见的类别-领域对，使得未见类别仍然容易识别，从而产生虚假的移除感。我们认为，真正的领域抹除必须是与类别无关的。为了解决这个问题，我们正式化了开放词汇领域遗忘（Open-Vocabulary Domain Unlearning, OVDU），这是一种严格的协议，要求领域遗忘必须转移到保留的类别。为了解决 OVDU 挑战，我们提出了一种手术式参数编辑框架。首先，Fisher 信息掩码隔离领域敏感权重，数学上保护基础的零-shot 泛化能力。其次，我们的目标导向流形散射（Targeted Manifold Scattering, TMS）目标使用基于偏好的挖掘方法局部散射遗忘领域的风格几何。通过在 PACS、OfficeHome 和 DomainNet 上进行评估，我们的方法在开放词汇泛化方面显著优于现有基线。关键是，它提供了卓越的样本效率，仅用 4 次样本就超越了峰值 8-shot 基线结果。

View on arXiv Download PDF AI Translation

cs.CV / 72 / 2609.31364

OpenVAM: Open-World Visual Attention Modeling with VLMs

OpenVAM：基于视觉语言模型的开放世界视觉注意力建模

Hooshanfar, Kiana, Kazerouni, Amirhossein, Hosseini, Alireza, Brudno, Michael, Taati, Babak

Abstract

Predicting human gaze is a core capability for applications ranging from web/UI design analysis to robotics and human-computer interaction. Yet, most visual attention modeling methods output only a dense saliency map, which is often insufficient for action: practitioners need to connect attention peaks to discrete elements in the scene (what) and understand the drivers of those peaks in context (why), while remaining robust to domain shift across natural images, commercial content, and UI/web layouts. We, therefore, introduce OpenVAM (Open-world Visual Attention Modeling with VLMs), a unified framework that jointly addresses universality and explainability across heterogeneous domains (natural scenes, commercial imagery, and UI/web layouts) and supervision modalities. OpenVAM adopts a decoupled-but-aligned design: a dedicated dense visual pathway provides stable, spatially precise localization, while an instruction-following vision--language semantic head generates grounded what/why explanations conditioned on the same image and data-type prompts. A three-stage training strategy preserves strong localization priors while progressively introducing language grounding and improving explanation alignment via parameter-efficient adaptation without perturbing the saliency branch. We further propose a scalable pipeline to generate multi-domain saliency-reason annotations for training and systematic evaluation. Experiments across diverse datasets show that OpenVAM improves robustness under domain shift while producing image-grounded explanations that make saliency predictions more interpretable.

Chinese Translation

预测人类注视是从网页/UI设计分析到机器人技术和人机交互等应用的核心能力。然而，大多数视觉注意力建模方法仅输出密集的显著性图，这通常不足以支持实际应用：从业者需要将注意力峰值与场景中的离散元素（是什么）连接起来，并理解这些峰值在上下文中的驱动因素（为什么），同时在自然图像、商业内容和UI/web布局之间保持对领域转移的鲁棒性。因此，我们提出了OpenVAM（基于视觉语言模型的开放世界视觉注意力建模），这是一个统一框架，旨在跨异构领域（自然场景、商业图像和UI/web布局）和监督模式共同解决普适性和可解释性问题。OpenVAM采用解耦但对齐的设计：专用的密集视觉通路提供稳定、空间精确的定位，而遵循指令的视觉-语言语义头则生成基于同一图像和数据类型提示的有根据的是什么/为什么的解释。三阶段训练策略在逐步引入语言基础的同时，保持强大的定位先验，并通过参数高效的适应改善解释对齐，而不干扰显著性分支。我们进一步提出了一个可扩展的管道，以生成多领域的显著性-理由注释，用于训练和系统评估。跨多样化数据集的实验表明，OpenVAM在领域转移下提高了鲁棒性，同时生成的图像基础解释使显著性预测更加可解释。

View on arXiv Download PDF AI Translation

cs.CV / 73 / 2609.31378

ContraFM-S2O: Flow Matching-Based One-step SAR-to-Optical Image Translation Model with Contrastive Learning

ContraFM-S2O：基于流匹配的一步式SAR到光学图像翻译模型与对比学习

Yu, Mingqian, Chiang, Wei-kuan, Wang, Qiurui, Zhao, Peilin

Abstract

In recent years, diffusion models and GAN-based models have become the mainstream approaches for SAR-to-optical image translation, owing to their advantages, such as high-quality generation and stable training. However, they have shortcomings such as high inference latency and the generated optical images suffer from low detail fidelity, often resulting in blurred edges and loss of fine textures. Thus, we propose ContraFM-S2O, which is a flow matching-based model for SAR-to-optical image translation. Unlike conventional diffusion models, ContraFM-S2O learns to predict the velocity field in training and solves ODE instead of SDE during inference to improve the sampling efficiency. In addition, ContraFM-S2O replaces instantaneous velocity with average velocity along the interpolation path to realize one-step SAR-to-optical image translation and uses contrastive learning to improve the quality of the generated optical images. Experiments show our model achieves state-of-the-art on SAR2Opt and QXS datasets, outperforming baselines, and reduces inference latency via one-step generation.

Chinese Translation

近年来，扩散模型和基于生成对抗网络（GAN）模型已成为SAR到光学图像翻译的主流方法，因其具有高质量生成和稳定训练等优点。然而，它们也存在一些缺点，如推理延迟高，生成的光学图像细节保真度低，常导致边缘模糊和细微纹理丢失。因此，我们提出了ContraFM-S2O，这是一种基于流匹配的SAR到光学图像翻译模型。与传统的扩散模型不同，ContraFM-S2O在训练中学习预测速度场，并在推理过程中求解常微分方程（ODE）而非随机微分方程（SDE），以提高采样效率。此外，ContraFM-S2O在插值路径上用平均速度替代瞬时速度，实现一步式SAR到光学图像翻译，并利用对比学习提高生成光学图像的质量。实验表明，我们的模型在SAR2Opt和QXS数据集上达到了最先进的水平，超越了基线，并通过一步生成减少了推理延迟。

View on arXiv Download PDF AI Translation

cs.CV / 74 / 2609.31431

AxonSynth: Domain-Randomized Synthetic Data for Zero-Shot 3D Axon Segmentation in Light-Sheet Microscopy

AxonSynth：用于光片显微镜中零-shot 3D 轴突分割的领域随机合成数据

Gaibor, Edward, Bintsi, Kyriaki-Margarita, Ureta, Carmen Luz Leiva, Bellatif, Zayneb, Maffei, Chiara, Li, Wenze, Hillman, Elizabeth, Balbastre, Yaël, Yendiki, Anastasia

Abstract

Accurate segmentation of axons in 3D microscopy data is important for analyzing white-matter organization, but dense ground truth labels are expensive to obtain. Existing supervised axon segmentation methods rely on target-domain annotations and can be brittle when tissue type, species, modality, or acquisition conditions change. We present AxonSynth, a domain-randomized synthetic-data framework for training 3D axon segmentation models without manually annotated real training volumes. AxonSynth generates dense synthetic axon labels with orientation priors that reflect realistic fiber configurations and renders them with randomized density, contrast, bias fields, blur, and noise. A three-class 3D U-Net is trained to predict background, axon sheath and intra-axonal space. We evaluate zero-shot transfer on 10 held-out light-sheet microscopy (LSM) patches from macaque and human brain samples labeled with one of three axonal markers, comparing against calibrated thresholding and Frangi filtering using overlap, corrected detection, false-positive, and topology metrics. On macaque samples, AxonSynth achieved the best corrected Dice and corrected precision (0.826 and 0.851), compared with 0.765 and 0.754 for thresholding and 0.685 and 0.762 for Frangi. On human samples, corrected Dice was comparable to thresholding (0.857 vs. 0.868), while component-count error decreased from 22,504 to 3,377. Across all held-out patches, AxonSynth reduced component-count error in 10/10 patches and Euler-characteristic error in 8/10. These results show that synthetic-label domain randomization can reduce dependence on manual axon annotation while supporting synthetic-to-real 3D segmentation.

Chinese Translation

在3D显微镜数据中准确分割轴突对于分析白质组织至关重要，但获取密集的真实标签代价高昂。现有的监督轴突分割方法依赖于目标领域的注释，当组织类型、物种、成像方式或采集条件发生变化时，可能会变得脆弱。我们提出了AxonSynth，这是一个领域随机化的合成数据框架，用于在没有手动注释的真实训练体积的情况下训练3D轴突分割模型。AxonSynth生成具有反映真实光纤配置的方向先验的密集合成轴突标签，并以随机密度、对比度、偏差场、模糊和噪声进行渲染。我们训练了一个三类3D U-Net模型，以预测背景、轴突鞘和轴突内空间。我们在10个来自猕猴和人脑样本的光片显微镜（LSM）补丁上评估了零-shot迁移，这些样本使用三种轴突标记之一进行标注，并与校准阈值和Frangi滤波进行比较，使用重叠、校正检测、假阳性和拓扑指标进行评估。在猕猴样本中，AxonSynth的校正Dice和校正精度分别达到了最佳值（0.826和0.851），而阈值法和Frangi滤波分别为0.765和0.754，以及0.685和0.762。在人类样本中，校正Dice与阈值法相当（0.857对0.868），而组件计数误差从22,504减少到3,377。在所有保留的补丁中，AxonSynth在10/10个补丁中减少了组件计数误差，在8/10个补丁中减少了欧拉特征误差。这些结果表明，合成标签的领域随机化可以减少对手动轴突注释的依赖，同时支持合成到真实的3D分割。

View on arXiv Download PDF AI Translation

cs.CV / 75 / 2609.31435

Implicit Neural Representation for Hyperspectral Video Compression

用于高光谱视频压缩的隐式神经表示

Scalera, Alfredo, Murray, Paul, Zabalza, Jaime

Abstract

With the advent of snapshot cameras, hyperspectral video is becoming more readily available. In recent years, new applications have emerged which have led to increasingly larger datasets. However, hyperspectral video compression remains in the early stages. In this study, we explore the use of implicit neural representation as a candidate solution. We propose a novel extension of an existing RGB video compression model, achieving Bj{\o}ntegaard Delta PSNR gains of +4.99 dB and Bj{\o}ntegaard Delta rate of -88.88% compared to traditional hyperspectral image compression methods applied frame-by-frame. In addition to reconstruction quality, the effects on downstream task performance are measured in the form of object tracking success. Compared to video compressed with methods based on principal component analysis and JPEG2000 in low data regimes, our proposed method improves tracking area under the curve by up to 23.42% and distance precision by up to 35.56% on examples from the HOT2026 dataset.

Chinese Translation

随着快照相机的出现，高光谱视频变得越来越容易获取。近年来，新的应用不断涌现，导致数据集规模日益扩大。然而，高光谱视频压缩仍处于早期阶段。在本研究中，我们探讨了将隐式神经表示作为候选解决方案的可能性。我们提出了一种现有RGB视频压缩模型的新扩展，与传统的逐帧高光谱图像压缩方法相比，取得了+4.99 dB的Bj{4}ntegaard Delta PSNR增益和-88.88%的Bj{4}ntegaard Delta比特率。在重建质量之外，我们还测量了对下游任务性能的影响，具体表现为目标跟踪的成功率。与基于主成分分析和JPEG2000方法在低数据环境下压缩的视频相比，我们提出的方法在HOT2026数据集的示例中，跟踪曲线下面积提高了最多23.42%，距离精度提高了最多35.56%。

View on arXiv Download PDF AI Translation

cs.CV / 76 / 2609.31450

From Reward Signal to Visual Utility: A Controlled Audit of Medical VLM Post-Training

从奖励信号到视觉效用：医疗视觉语言模型后训练的受控审计

Jingxin, Wang

Abstract

Medical vision-language model (VLM) post-training is commonly evaluated through answer accuracy. We examine how changes in accuracy and training objectives relate to image-conditioned decisions in a controlled Qwen2.5-VL-3B study on PMC-VQA. We compare supervised fine-tuning (SFT) with low-rank adaptation (LoRA) restricted to the language model, expanded multimodal adaptation scopes, standard answer-only Group Relative Policy Optimization (GRPO), and a counterfactual evidence objective. On 2,000 clean-test questions, language model LoRA SFT changes correct-image accuracy by +1.10 percentage points (95% paired bootstrap CI:-0.85 to +3.05), while visual-benefit events decrease by 2.40 points and image sensitivity decreases by 5.60 points. Paired records reveal 155 acquired and 203 lost visual-benefit events. Broader adaptation yields lower correct-image accuracy than language-model LoRA SFT. Standard GRPO produces mixed-reward groups and parameter updates, with an uncertain clean test accuracy change. A generation audit reveals that canonical option scores can follow a different token path from generated answers. With scores taken along the greedy generation path, the evidence target improves on the training set; its gains over standard GRPO remain inconsistent on validation data at matched training doses. Sample-level analyses trace how evidence scores, decision margins, and generated answers change during post-training. This empirical and measurement audit identifies gaps between optimization activity, target acquisition, and useful held-out visual behavior.

Chinese Translation

医疗视觉语言模型（VLM）后训练通常通过答案准确性进行评估。我们研究了准确性和训练目标的变化如何与图像条件决策相关，在PMC-VQA的受控Qwen2.5-VL-3B研究中进行。我们比较了仅限于语言模型的监督微调（SFT）与低秩适应（LoRA）、扩展的多模态适应范围、标准的仅答案组相对策略优化（GRPO）以及反事实证据目标。在2000个干净测试问题中，语言模型LoRA SFT使正确图像准确性提高了1.10个百分点（95%配对自助法置信区间：-0.85到+3.05），而视觉效益事件减少了2.40点，图像敏感性减少了5.60点。配对记录显示获得了155个视觉效益事件，失去了203个视觉效益事件。更广泛的适应性导致的正确图像准确性低于语言模型LoRA SFT。标准GRPO产生了混合奖励组和参数更新，干净测试准确性的变化不确定。生成审计显示，规范选项分数可以沿着与生成答案不同的标记路径进行。沿贪婪生成路径获得的分数在训练集上有所改善；其在标准GRPO上的增益在匹配训练剂量的验证数据上仍然不一致。样本级分析追踪了证据分数、决策边际和生成答案在后训练期间的变化。此实证和测量审计识别了优化活动、目标获取和有用的保留视觉行为之间的差距。

View on arXiv Download PDF AI Translation

cs.CV / 77 / 2609.31456

Diagnosing the Sources of Compositional Failure in Vision-Language Models: A Controlled Analysis

诊断视觉-语言模型中组合失败的来源：一项控制分析

Gandhi, Mona, Olcay, Cenk Merih, Lo, Kuan-Chieh, Castro, Santiago, Myers, Christopher W., Parthasarathy, Srinivasan

Abstract

Vision-language models (VLMs) often struggle with compositional reasoning tasks, but the reasons for this underperformance remain unclear. A common hypothesis is that models struggle to integrate multiple components, leading to training interventions to improve compositional binding. However, this assumption has never been directly quantified. Existing benchmarks evaluate captions only in their composed form, making it impossible to separate the cost of joint reasoning from the cost of recognizing individual components under increasing load. We introduce COMPASS (COMPositional Analysis of SkillS), a controlled evaluation framework designed to isolate and measure the distinct factors underlying compositional failure. By comparing performance on composed captions with their decomposed counterparts , we directly quantify the cost of compositional integration across 87K image-caption pairs. Across multiple VLMs, this gap is real but partial, accounting for only part of the observed degradation. This motivates a finer-grained investigation into what additional factors govern model behavior. We analyze performance at the level of individual skills: object detection, attribute binding, and relation reasoning, using skill-targeted perturbations across 274K image-caption pairs. We find a consistent skill-specific pattern: each skill degrades primarily with the count of its own primitive type (self-load), while cross-load effects are predominantly positive, suggesting that primitives of different types provide useful grounding context. This pattern holds across standard contrastive encoders, explicitly trained compositional reasoning models, and non-contrastive architectures. These findings show that compositional degradation reflects multiple separable factors that cannot be reduced to joint reasoning alone.

Chinese Translation

视觉-语言模型（VLMs）在组合推理任务中常常表现不佳，但这种表现不佳的原因仍不清楚。一个常见的假设是模型在整合多个组件时存在困难，这导致了训练干预以改善组合绑定。然而，这一假设从未被直接量化。现有基准仅在组合形式中评估字幕，使得无法将联合推理的成本与在增加负载下识别单个组件的成本分开。我们引入了COMPASS（组合技能分析），这是一种控制评估框架，旨在隔离和测量导致组合失败的不同因素。通过比较组合字幕与其分解对应物的性能，我们直接量化了在87K图像-字幕对中组合整合的成本。在多个VLM中，这一差距是真实存在的，但只是部分的，仅占观察到的降级的一部分。这促使我们更细致地调查影响模型行为的其他因素。我们在个体技能层面分析性能：对象检测、属性绑定和关系推理，使用针对技能的扰动分析274K图像-字幕对。我们发现了一种一致的技能特定模式：每项技能的降级主要与其自身原始类型的数量（自负载）相关，而交叉负载效应主要是正向的，这表明不同类型的原始元素提供了有用的基础上下文。这一模式在标准对比编码器、明确训练的组合推理模型和非对比架构中均成立。这些发现表明，组合降级反映了多个可分离的因素，而不能仅仅归结为联合推理。

View on arXiv Download PDF AI Translation

cs.CV / 78 / 2609.31461

KneePreM: Towards 3D Knee MRI Foundation Models via Large-Scale Unlabeled Pretraining and Label-Efficient Fine-Tuning

KneePreM：通过大规模无标签预训练和标签高效微调实现3D膝关节MRI基础模型

Wang, Xinxin, Hazan, Liam, Li, Jing, Rabinovici-Cohen, Simona, Li, Xiaojuan, Yang, Mingrui

Abstract

Background: Large volumes of unlabeled knee MRI scans are available across repositories but remain insufficiently leveraged. We developed KneePreM, a knee-specific 3D self-supervised model, and evaluated transfer and label efficiency for classification and segmentation. Methods: A 3D U-Net masked autoencoder was pretrained on 19,011 unlabeled Osteoarthritis Initiative (OAI) MRI series from 4,791 participants. Downstream fine-tuning used full and reduced training sets for fastMRI+ two-label classification (1,172 examinations), Arthroscopic Partial Meniscectomy (APM) eight-target classification (1,716 examinations), SKM-TEA segmentation (155 examinations), and APM segmentation (25 examinations). Baselines were random initialization and SuPreM. Deployment workflow was implemented with a Model Context Protocol interface. Evaluation metrics included balanced accuracy, F1 score, ROC AUC, PR AUC, and Dice score. Statistical analysis used bootstrap confidence intervals and paired bootstrap tests for classification and Wilcoxon signed-rank tests for segmentation. Results: KneePreM achieved higher full-data macro ROC AUC than both baselines for fastMRI+ and APM (all p < .001). For fastMRI+ classification, KneePreM achieved a ROC AUC of 0.722 using 50% of the training data, exceeding both full-data baselines. In APM classification, KneePreM reached a ROC AUC of 0.740 with 70% of the data, matching the full-data random baseline and outperforming SuPreM. For SKM-TEA segmentation, its 70%-data Dice of 0.838 exceeded the full-data random baseline (0.835) and both same-budget comparators. In APM segmentation, its 75%-data Dice of 0.746 exceeded the full-data random baseline (0.731) and both same-budget comparators. Conclusion: KneePreM improves transfer performance and label efficiency across knee MRI classification and segmentation tasks, particularly when labeled training data are limited.

Chinese Translation

背景：大量无标签的膝关节MRI扫描图像在各个数据库中可用，但尚未得到充分利用。我们开发了KneePreM，一个特定于膝关节的3D自监督模型，并评估了其在分类和分割任务中的迁移和标签效率。方法：一个3D U-Net掩码自编码器在来自4,791名参与者的19,011个无标签骨关节炎倡议（OAI）MRI系列上进行了预训练。下游微调使用完整和减少的训练集进行fastMRI+的双标签分类（1,172个检查）、关节镜下部分半月板切除术（APM）八目标分类（1,716个检查）、SKM-TEA分割（155个检查）和APM分割（25个检查）。基准方法为随机初始化和SuPreM。部署工作流程通过模型上下文协议接口实现。评估指标包括平衡准确率、F1分数、ROC AUC、PR AUC和Dice分数。统计分析使用自助法置信区间和配对自助法检验进行分类，Wilcoxon符号秩检验用于分割。结果：KneePreM在fastMRI+和APM的全数据宏ROC AUC上均优于两个基准（所有p < .001）。在fastMRI+分类中，KneePreM使用50%的训练数据达到了0.722的ROC AUC，超过了两个全数据基准。在APM分类中，KneePreM在使用70%数据时达到了0.740的ROC AUC，匹配全数据随机基准并优于SuPreM。在SKM-TEA分割中，其70%数据的Dice为0.838，超过了全数据随机基准（0.835）和两个同预算比较方法。在APM分割中，其75%数据的Dice为0.746，超过了全数据随机基准（0.731）和两个同预算比较方法。结论：KneePreM在膝关节MRI分类和分割任务中提高了迁移性能和标签效率，特别是在标记训练数据有限的情况下。

View on arXiv Download PDF AI Translation

cs.CV / 79 / 2609.31507

SatNav: A Scalable Benchmark for Long-Horizon UAV Vision-Language Navigation from Satellite Imagery

SatNav：基于卫星影像的长时间跨度无人机视觉-语言导航可扩展基准

Jiang, Jiajun, Hua, Chunliang, Chen, Zichun, Wu, Yanxing, Yang, Zeyuan, Song, Jie, Hu, Xiao

Abstract

Urban uncrewed aerial vehicle (UAV) vision-language navigation (VLN) requires agents to follow instructions across extended urban spaces, inherently demanding long-term memory and geospatial grounding. However, scaling existing benchmarks remains difficult because of their reliance on costly reconstructed 3D assets, limiting geographic diversity and episode scale. To address this, we introduce SatNav, a scalable, long-horizon UAV VLN benchmark built from high-resolution satellite imagery. SatNav targets city-level navigation missions and uses satellite crops as approximations of UAV nadir views for visual observations. Through an automated cue-to-episode pipeline, SatNav constructs 118K episodes from 59 scenes across 18 cities, with an average trajectory length of 379 m. To stress-test long-horizon memory and geospatial reasoning, SatNav defines three task families: Boundary, Landmark, and Route, targeting loop progress tracking, landmark-based spatial grounding, and route following with counting cues. Benchmarking classical VLN agents and recent agents based on large vision-language models (LVLMs) on SatNav shows that city-scale navigation remains challenging. We further introduce SwiftVLN, a modular framework with switchable memory components, and conduct systematic memory-design ablations. Finally, satellite-to-UAV transfer experiments show that satellite-trained navigation models can operate on real-flight UAV observations, showing the practical relevance of SatNav. Our project page: https://eku127.github.io/SatNav/

Chinese Translation

城市无人机（UAV）视觉-语言导航（VLN）要求代理在广阔的城市空间中遵循指令，这本质上需要长期记忆和地理空间基础。然而，由于现有基准依赖于昂贵的重建3D资产，导致其在地理多样性和情节规模上的扩展仍然困难。为了解决这一问题，我们引入了SatNav，一个基于高分辨率卫星影像构建的可扩展长时间跨度UAV VLN基准。SatNav针对城市级导航任务，使用卫星裁剪图像作为UAV垂直视图的近似值进行视觉观察。通过自动化的线索到情节管道，SatNav从18个城市的59个场景中构建了118K个情节，平均轨迹长度为379米。为了对长期记忆和地理空间推理进行压力测试，SatNav定义了三个任务类别：边界（Boundary）、地标（Landmark）和路线（Route），分别针对循环进度跟踪、基于地标的空间基础和带计数线索的路线跟随。在SatNav上对经典VLN代理和基于大型视觉-语言模型（LVLMs）的最新代理进行基准测试显示，城市规模的导航仍然具有挑战性。我们进一步引入了SwiftVLN，一个具有可切换内存组件的模块化框架，并进行系统的内存设计消融实验。最后，卫星到UAV的迁移实验表明，经过卫星训练的导航模型可以在实际飞行的UAV观察中运行，显示了SatNav的实际相关性。我们的项目页面：https://eku127.github.io/SatNav/

View on arXiv Download PDF AI Translation

cs.CV / 80 / 2609.31509

ClearGS: Reliability-Aware Gaussian Splatting from Handheld Videos

ClearGS：基于手持视频的可靠性意识高斯点云渲染

Liu, Xuanzhi, Wu, Xinyi, Pan, Hang, Huang, Wensi, Wu, Zhenyao, Han, Ruize, Wang, Song

Abstract

We present ClearGS for 3D Gaussian Splatting (3DGS) from handheld videos with uneven viewpoint coverage and mixed frame quality. Rather than selecting frames with binary decisions, ClearGS uses Reliability-aware View Allocation (RVA) to assign graded raw-supervision weights based on appearance reliability, degradation risk, and geometric utility, while weakly reactivating useful suppressed frames to maintain trajectory coverage. Since weighting cannot restore details lost to blur or distortion, ClearGS further introduces Render-Guided In-Video Restoration (RIVR). The current 3DGS render provides a pose-aligned structural candidate, a frozen no-reference restoration expert restores the corresponding raw video observation without any clean reference image, and no-reference perceptual scores select among the render, restored observation, and high-frequency fused candidate. ClearGS then applies Full-Trajectory Repair Consolidation to revisit accepted repairs and preserve details introduced early. On GS2E and GSOTM, ClearGS achieves state-of-the-art overall performance, with consistent CLIP-IQA and MUSIQ gains and LPIPS reductions in most degradation settings, without paired sharp supervision or matched clean references.

Chinese Translation

我们提出了ClearGS，用于从视角覆盖不均和帧质量混合的手持视频中进行3D高斯点云渲染（3DGS）。ClearGS并不是通过二元决策选择帧，而是使用可靠性意识视图分配（RVA）根据外观可靠性、退化风险和几何效用分配分级的原始监督权重，同时弱化地重新激活有用的被抑制帧，以保持轨迹覆盖。由于加权无法恢复因模糊或失真而丢失的细节，ClearGS进一步引入了渲染引导的在线视频修复（RIVR）。当前的3DGS渲染提供了一个姿态对齐的结构候选，冻结的无参考修复专家在没有任何干净参考图像的情况下恢复相应的原始视频观测，而无参考感知评分在渲染、修复观测和高频融合候选之间进行选择。然后，ClearGS应用全轨迹修复整合，重新审视已接受的修复并保留早期引入的细节。在GS2E和GSOTM上，ClearGS实现了最先进的整体性能，在大多数退化设置中，CLIP-IQA和MUSIQ的增益以及LPIPS的减少均保持一致，而无需配对的清晰监督或匹配的干净参考。

View on arXiv Download PDF AI Translation

cs.CV / 81 / 2609.31514

Forensic Twins: Self-Supervised Residual Learning for AI-Generated Image Forensics

法医双胞胎：用于人工智能生成图像取证的自监督残差学习

Muñoz-Haro, Javier, Tolosana, Ruben, Vera-Rodriguez, Ruben, Morales, Aythami, Fierrez, Julian

Abstract

Detectors of AI-generated images are typically trained using samples from all Generative AI architectures they must catch, and struggle as soon as a new architecture emerges. Recent approaches have explored self-supervised pre-training as an alternative solution, yet standard frameworks work against the forensic task, e.g., their augmentations overwrite the micro-statistics of image formation. This paper introduces Forensic Twins, a Self-Supervised Residual Learning (SSRL) framework whose pretext task suppresses macroscopic content availability. Each image is mapped through a frozen, off-the-shelf forensic residual extractor, from which two spatially disjoint crops are drawn. Sharing no pixel, the two views retain minimal semantic structure to align, leaving a redundancy-reduction objective with a predominant common signal: the stationary fingerprint of the image acquisition pipeline. Additionally, Forensic Twins is trained exclusively on real images; no AI-generated image is observed at any stage. Experiments show that Forensic Twins attributes AI generator sources with 56.61% accuracy, i.e., 6.13% above the previous state-of-the-art zero-shot method at 375x lower latency. We also demonstrate that fitting a Gaussian Mixture Model (GMM) offline using only the real image embeddings extracted from Forensic Twins turns it into a state-of-the-art zero-shot detector, reaching 97.99% AUC across 27 unseen AI generators, including GANs, diffusion models and commercial systems. Code, weights and exact splits will be made publicly available

Chinese Translation

人工智能生成图像的检测器通常使用来自所有生成性人工智能架构的样本进行训练，但一旦出现新的架构，检测器就会面临困难。近期的研究探索了自监督预训练作为一种替代解决方案，然而标准框架在取证任务中存在问题，例如，它们的增强操作会覆盖图像形成的微观统计特征。本文介绍了法医双胞胎（Forensic Twins），一种自监督残差学习（Self-Supervised Residual Learning, SSRL）框架，其预训练任务抑制宏观内容的可用性。每幅图像通过一个冻结的、现成的法医残差提取器进行映射，从中提取两个空间上不重叠的裁剪区域。两个视图不共享任何像素，保留了最小的语义结构以便对齐，从而留下一个冗余减少目标，主要的共同信号是图像采集管道的静态指纹。此外，法医双胞胎仅在真实图像上进行训练；在任何阶段都未观察到人工智能生成的图像。实验表明，法医双胞胎以56.61%的准确率归因于人工智能生成器的来源，即比之前的最先进零样本方法高出6.13%，且延迟降低了375倍。我们还展示了，仅使用从法医双胞胎提取的真实图像嵌入，离线拟合高斯混合模型（Gaussian Mixture Model, GMM）可以将其转变为最先进的零样本检测器，在27个未见过的人工智能生成器（包括GAN、扩散模型和商业系统）中达到97.99%的AUC。代码、权重和确切的拆分将公开发布。

View on arXiv Download PDF AI Translation

cs.CV / 82 / 2609.31524

Structured Reasoning Agentic Framework for Interpretable Critical View of Safety Assessment

可解释安全评估的结构化推理代理框架

Xu, Qing, Luo, Yuxiang, Chen, Zhen

Abstract

Surgical scene understanding is critical for computer-assisted intervention, yet laparoscopic cholecystectomy remains challenged by the complex anatomy of the hepatocystic triangle and the risk of bile duct injury. Existing methods for Critical View of Safety (CVS) assessment typically treat it as a holistic prediction task, mapping visual features directly to criterion-level labels. This black-box paradigm lacks explicit reasoning about anatomical relationships, limiting both interpretability and compositional generalization. To address this, we propose ReasonCVS, a structured reasoning agentic framework empowered by Vision-Language Models (VLMs) that decomposes CVS assessment into explicit, fine-grained anatomical verification. Specifically, we devise an Anatomical Scene Graph Abstraction (ASGA) that organizes anatomical entities and their spatial relationships into a structured representation. To operationalize this, we introduce a Rationale-Aware Reasoning Agent, powered by a Large Language Model (LLM) fine-tuned via rationale distillation. Functioning as a strict central decision-maker, it invokes VLM-driven Sub-criterion Verifier as a specialized perceptual tool to parse the graph and independently evaluate individual sub-criteria. Through calibrated soft reasoning, this agent synthesizes the tool-gathered distributed observations, yielding a final verdict alongside a traceable clinical rationale. Extensive experiments on the Endoscapes-CVS201 benchmark demonstrate that ReasonCVS achieves superior performance (68.1\% mAP) over state-of-the-art while providing interpretable, criterion-level explanations for reliable surgical assessment.

Chinese Translation

外科场景理解对于计算机辅助干预至关重要，但腹腔镜胆囊切除术仍面临肝胆三角复杂解剖结构和胆管损伤风险的挑战。现有的安全视图（Critical View of Safety, CVS）评估方法通常将其视为整体预测任务，直接将视觉特征映射到标准级别标签。这种黑箱范式缺乏对解剖关系的明确推理，限制了可解释性和组合泛化能力。为了解决这一问题，我们提出了ReasonCVS，一个由视觉-语言模型（Vision-Language Models, VLMs）赋能的结构化推理代理框架，将CVS评估分解为明确的、细粒度的解剖验证。具体而言，我们设计了一种解剖场景图抽象（Anatomical Scene Graph Abstraction, ASGA），将解剖实体及其空间关系组织成结构化表示。为了实现这一目标，我们引入了一种基于理由的推理代理（Rationale-Aware Reasoning Agent），该代理由大型语言模型（Large Language Model, LLM）通过理由蒸馏进行微调。作为严格的中央决策者，它调用基于VLM的子标准验证器（Sub-criterion Verifier）作为专门的感知工具，以解析图形并独立评估各个子标准。通过校准的软推理，该代理综合了工具收集的分布式观察，得出最终裁决，并提供可追溯的临床理由。在Endoscapes-CVS201基准上的大量实验表明，ReasonCVS在性能上优于最先进的技术（68.1\% mAP），同时为可靠的外科评估提供了可解释的标准级别解释。

View on arXiv Download PDF AI Translation

cs.CV / 83 / 2609.31558

Region-Level Black-Box Defense Against Stealthy Embedding-Space Backdoors in CLIP

针对 CLIP 中隐蔽嵌入空间后门的区域级黑箱防御

Abdelnaby, Ahmed, Elmahallawy, Mohamed

Abstract

Contrastive Language--Image Pretraining (CLIP) has emerged as a dominant vision backbone due to its strong transferability and zero-shot capabilities. However, recent studies reveal a critical vulnerability: embedding-space backdoor attacks. By poisoning only a tiny fraction of image--text pairs, adversaries can implant stealthy triggers that induce targeted shifts in CLIP's joint embedding space. Unlike conventional backdoors that manipulate classifier logits, these attacks corrupt representations directly, making them highly effective under extremely low poisoning ratios and difficult to detect. Existing defenses require access to model parameters, gradients, logits, or clean validation data---assumptions that rarely hold in realistic black-box deployments. Moreover, current black-box methods struggle to accurately localize small or out-of-distribution triggers. We propose CLIPGuard, a lightweight and fully black-box defense specifically designed to mitigate embedding-space backdoors in CLIP encoders. CLIPGuard identifies malicious regions by measuring segment-wise embedding perturbations and selectively purifies only suspicious segments via semantic inpainting, preserving benign visual content and alignment quality. Extensive experiments on STL-10, ImageNet, and diverse trigger families---including BadCLIP, BadNets, blended, patch-based, and typographic attacks---demonstrate that CLIPGuard reduces attack success rates to as low as 1.05% while maintaining clean accuracy up to 86.34%, consistently outperforming existing black-box defenses, including CleanCLIP and CleanerCLIP. Our code is available https://github.com/wsu-cyber-security-lab-ai/CLIPGuard.git

Chinese Translation

对比语言-图像预训练（CLIP）因其强大的迁移能力和零样本能力而成为主流的视觉骨干网络。然而，近期研究揭示了一个关键的脆弱性：嵌入空间后门攻击。通过仅对少量图像-文本对进行污染，攻击者可以植入隐蔽的触发器，从而在 CLIP 的联合嵌入空间中引发目标偏移。与传统的后门攻击通过操控分类器的 logits 不同，这些攻击直接破坏表示，使其在极低的污染比率下依然高度有效且难以检测。现有的防御方法需要访问模型参数、梯度、logits 或干净的验证数据——这些假设在现实的黑箱部署中很少成立。此外，目前的黑箱方法在准确定位小型或分布外触发器方面也面临困难。我们提出了 CLIPGuard，一种轻量级且完全黑箱的防御机制，专门设计用于缓解 CLIP 编码器中的嵌入空间后门。CLIPGuard 通过测量分段嵌入扰动来识别恶意区域，并通过语义修复选择性地净化仅可疑的段，保留良性的视觉内容和对齐质量。在 STL-10、ImageNet 及多种触发器家族（包括 BadCLIP、BadNets、混合、基于补丁和排版攻击）上的大量实验表明，CLIPGuard 将攻击成功率降低至最低 1.05%，同时保持高达 86.34% 的清洁准确率，始终优于现有的黑箱防御方法，包括 CleanCLIP 和 CleanerCLIP。我们的代码可在 https://github.com/wsu-cyber-security-lab-ai/CLIPGuard.git 获取。

View on arXiv Download PDF AI Translation

cs.CV / 84 / 2609.31572

OC-GS: Gaussian Splatting for Irregular Turntable Capture

OC-GS：用于不规则转盘捕获的高斯溅射

Lee, Jae Joong, Benes, Bedrich

Abstract

Uneven rotation and dropped frames make equal-angle assumptions unreliable for turntable reconstruction. We present OC-GS, an object-centric Gaussian splatting that refines each image's angle while maintaining a shared camera, rotation axis, and pivot. This orbit-consistent refinement jointly optimizes image-derived geometry and angles to reconstruct objects from sparse, irregular captures. On rendered objects with 12, 8, and 6 irregularly spaced views, OC-GS achieves mean foreground PSNR scores of 21.26, 19.36, and 15.83dB, respectively, exceeding all four evaluated pose-free Gaussian splatting baselines in each condition. Under a shared trainer, refining image-estimated angles improves mean foreground PSNR by 7.88dB over keeping those estimates fixed. An ablation study shows that both image-derived angle initialization and the shared motion model contribute to the improvement. On real captures, OC-GS's refinement increases mean foreground PSNR by 0.70dB. Results show that refining uncertain angles within a shared motion model improves reconstruction from sparse, irregular turntable captures.

Chinese Translation

不均匀的旋转和丢帧使得等角假设在转盘重建中不可靠。我们提出了OC-GS，一种以对象为中心的高斯溅射方法，它在保持共享相机、旋转轴和支点的同时，细化每幅图像的角度。这种轨道一致的细化方法联合优化图像衍生的几何形状和角度，以从稀疏且不规则的捕获中重建物体。在具有12、8和6个不规则间隔视图的渲染物体上，OC-GS分别达到了21.26、19.36和15.83dB的平均前景PSNR分数，超越了在每种条件下评估的四个无姿态高斯溅射基线。在共享训练器下，细化图像估计的角度使平均前景PSNR提高了7.88dB，相较于保持这些估计不变。消融研究表明，图像衍生的角度初始化和共享运动模型都对改进有所贡献。在真实捕获数据上，OC-GS的细化使平均前景PSNR提高了0.70dB。结果表明，在共享运动模型内细化不确定的角度能够改善从稀疏、不规则转盘捕获中进行重建的效果。

View on arXiv Download PDF AI Translation

cs.CV / 85 / 2609.31573

How Far Can INRs Go? Cross-Domain Parameter-efficient INR-Based Semantic Segmentation for Brain MRI

INRs 能走多远？基于跨领域参数高效的 INR 的脑 MRI 语义分割

Shang, Ziyao, Sadeghi, Pouya, Jiang, Letian, Wong, Alexander, Rambhatla, Sirisha

Abstract

Biomedical image segmentation is central to medical image analysis, but practical deployment often faces limited annotations, memory constraints, and cross-site distribution shifts. Implicit Neural Representations (INRs) have recently emerged as a lightweight alternative for semantic segmentation, achieving competitive performance with substantially fewer parameters than conventional architectures. However, the mechanisms, scaling behavior, and domain generalization abilities of INR-based segmentation remain insufficiently understood. In this work, we study these questions in the context of cross-domain brain MRI segmentation. We analyze INR-based segmentation across low-parameter regimes, comparing it with conventional pipelines in both in-domain and out-of-domain settings. Surprisingly, we find that INR-based models do not simply improve with increasing parameter budget. Their advantage is most pronounced under low-parameter and limited-augmentation settings, while U-Net-based models benefit more from larger capacity and standard augmentation. We also investigate how INRs encode semantic information in their hidden features and show that complementary segmentation-relevant structure is distributed across multiple INR layers. Building on this insight, we introduce HierINRSeg, a hierarchical INR-based architecture that aggregates multi-layer representations for improved robustness and generalization. Extensive experiments show that HierINRSeg consistently outperforms MetaSeg, a strong recent INR-based segmentation baseline, with an average improvement of 5.6 percentage points in Dice for the in-domain test set and 8.2 percentage points out-of-domain. Overall, our analysis identifies the conditions under which INR-based segmentation is most effective, providing concrete guidance for model selection and future research.

Chinese Translation

生物医学图像分割是医学图像分析的核心，但实际应用常常面临有限的标注、内存限制和跨站点分布变化等问题。隐式神经表示（Implicit Neural Representations, INRs）最近作为语义分割的一种轻量级替代方案出现，在参数数量显著少于传统架构的情况下实现了竞争力的性能。然而，基于 INR 的分割机制、扩展行为和领域泛化能力仍然未被充分理解。在本研究中，我们在跨领域脑 MRI 分割的背景下研究这些问题。我们分析了低参数范围内的基于 INR 的分割，并将其与传统管道在领域内和领域外的设置进行比较。令人惊讶的是，我们发现基于 INR 的模型并不只是随着参数预算的增加而简单提升。它们的优势在低参数和有限增强设置下最为明显，而基于 U-Net 的模型则更能从更大的容量和标准增强中获益。我们还研究了 INRs 如何在其隐藏特征中编码语义信息，并展示了与分割相关的互补结构分布在多个 INR 层中。基于这一见解，我们提出了 HierINRSeg，一种层次化的基于 INR 的架构，聚合多层表示以提高鲁棒性和泛化能力。大量实验表明，HierINRSeg 在领域内测试集的 Dice 指标上平均提高了 5.6 个百分点，在领域外提高了 8.2 个百分点，始终优于一个强大的近期基于 INR 的分割基线 MetaSeg。总体而言，我们的分析识别了基于 INR 的分割最有效的条件，为模型选择和未来研究提供了具体指导。

View on arXiv Download PDF AI Translation

cs.CV / 86 / 2609.31595

GraphWrit3R: End-to-End 3D Scene Graph Writing

GraphWrit3R：端到端的3D场景图生成

Milivojevic, Luka, Popovic, Nikola, Sarkar, Sayan Deb, Koch, Sebastian, Armeni, Iro, Van Gool, Luc, Paudel, Danda Pani

Abstract

3D scene graphs provide a structured representation of complex environments by encoding objects, their semantic attributes, and the spatial and functional relationships between them. Current approaches for 3D scene graph generation suffer from several fundamental limitations. They rely on complex multi-stage pipelines with explicit intermediate representations, making systems fragile and prone to error propagation. They assume access to ground-truth object annotations during inference, which deviates from real-world scenarios. They depend on proprietary models, hindering open-source deployment, or incur prohibitively slow inference. We present GraphWrit3R, a simple end-to-end method that takes a 3D point cloud, Gaussian Splats, or a combination of both as input, and directly outputs a complete scene graph as a structured JSON script. The graph lists all objects, their semantic attributes, and the relationships between them, while avoiding all of the above mentioned limitations. The choice of multiple input modalities is purely for versatility, allowing a single set of weights to handle diverse scenarios. Point cloud inputs are encoded via Sonata and Gaussian Splat inputs via Chorus, with both modalities projected onto a shared voxel grid and fused through a novel per-voxel contrastive alignment loss before being decoded by a large language model. As a natural consequence of the LLM, GraphWrit3R also supports open-vocabulary querying. On the 3DSSG benchmark, our method achieves state-of-the-art performance on object class, predicate, and triplet recall, outperforming methods that rely on ground-truth object annotations during inference. We further provide qualitative results and analyze different input modality configurations, contrastive loss formulations, and token fusion strategies.

Chinese Translation

3D场景图通过编码物体、其语义属性以及它们之间的空间和功能关系，为复杂环境提供了结构化的表示。目前的3D场景图生成方法存在几个基本的局限性。它们依赖于复杂的多阶段流程和明确的中间表示，使得系统脆弱且容易出现错误传播。它们在推理过程中假设可以访问真实的物体标注，这与现实场景不符。它们依赖于专有模型，阻碍了开源部署，或者推理速度极其缓慢。我们提出了GraphWrit3R，这是一种简单的端到端方法，接受3D点云、高斯斑点（Gaussian Splats）或两者的组合作为输入，并直接输出一个完整的场景图，格式为结构化的JSON脚本。该图列出了所有物体、它们的语义属性以及它们之间的关系，同时避免了上述所有局限性。选择多种输入模式纯粹是为了灵活性，使得一组权重能够处理多样化的场景。点云输入通过Sonata进行编码，高斯斑点输入通过Chorus进行编码，两个模式都被投影到共享的体素网格上，并通过一种新颖的每体素对比对齐损失进行融合，然后由大型语言模型解码。作为大型语言模型（LLM）的自然结果，GraphWrit3R还支持开放词汇查询。在3DSSG基准测试中，我们的方法在物体类别、谓词和三元组召回方面达到了最先进的性能，超越了在推理过程中依赖真实物体标注的方法。我们进一步提供了定性结果，并分析了不同输入模式配置、对比损失公式和标记融合策略。

View on arXiv Download PDF AI Translation

cs.CV / 87 / 2609.31620

FuseReg: Regularizing Layer Fusion Mitigates the Reconstruction-Generation Gap in Representation Autoencoders

FuseReg：正则化层融合缓解表示自编码器中的重建-生成差距

Du, Hongyang, Xie, Yunfei, Ye, Junjie, Yang, Jiawei, Cong, Xiaoyan, Zhang, Haodong, Huang, Yongchao, Wu, Haiyu, Li, Zongxia, Gui, Shihang, Liu, Dawei, Li, Runhao, Ni, Jingcheng, Wei, Chen, Balestriero, Randall, Wang, Yue

Abstract

Representation autoencoders (RAEs) reuse features from a pretrained visual encoder as reconstruction and diffusion latents, integrating strong visual representations into image generation. However, RAEs still need to decide which encoder layers form the shared latent space for the generator and pixel decoder. This choice involves a trade-off. Shallower layers tend to preserve fine pixel details better, while deeper layers tend to yield better generation metrics. A fixed heuristic layer fusion therefore couples two stages that benefit from different information. We introduce FuseReg, which replaces heuristic feature selection with training over random subsets of encoder layers. We theoretically analyze the underlying mechanism: subset sampling explicitly penalizes sensitivity to cross-layer disagreement. On ImageNet-256 with DINOv3-L, a single FuseReg decoder reconstructs from full, sparse, and single-layer fusions without retraining, achieving higher PSNR than decoders specialized to fixed fusions. This flexibility also benefits generation: decoder replacement alone reduces unguided gFID by 27% with an unchanged RAEv2 DiT-XL generator. The same regularization principle extends to diffusion training, with joint regularization of both stages reducing unguided gFID by 29% on DiT-Base. These results show that training downstream models for layer-fusion robustness narrows the reconstruction-generation gap without modifying the pretrained encoder.

Chinese Translation

表示自编码器（RAEs）重用来自预训练视觉编码器的特征作为重建和扩散潜变量，将强大的视觉表示整合到图像生成中。然而，RAEs 仍需决定哪些编码器层构成生成器和像素解码器的共享潜在空间。这个选择涉及权衡。较浅的层往往更好地保留细微的像素细节，而较深的层则往往产生更好的生成指标。因此，固定的启发式层融合将两个受益于不同信息的阶段耦合在一起。我们提出了 FuseReg，它用对编码器层随机子集的训练替代了启发式特征选择。我们从理论上分析了其基本机制：子集采样明确惩罚对跨层不一致的敏感性。在使用 DINOv3-L 的 ImageNet-256 数据集上，单个 FuseReg 解码器在不重新训练的情况下从完整、稀疏和单层融合中进行重建，获得比专门针对固定融合的解码器更高的 PSNR。这种灵活性也有利于生成：仅替换解码器就将无指导的 gFID 降低了 27%，而 RAEv2 DiT-XL 生成器保持不变。相同的正则化原则扩展到扩散训练，通过对两个阶段的联合正则化，将 DiT-Base 上的无指导 gFID 降低了 29%。这些结果表明，为层融合鲁棒性训练下游模型可以缩小重建-生成差距，而无需修改预训练编码器。

View on arXiv Download PDF AI Translation

人工智能 (Artificial Intelligence)

cs.AI / 1 / 2609.30291

Bringing AI to Autonomous Systems -- From Cognition to Collective Intelligence

将人工智能引入自主系统——从认知到集体智能

Sifakis, Joseph

Abstract

The purpose of this article is to highlight the central role of autonomous systems as the ultimate stage in the development of AI, to explain the underlying technical challenges that require a combination of connectionist AI and symbolic AI, and to integrate AI and systems engineering. We present a comprehensive framework for the design and evaluation of autonomous systems, based on a generic agent architecture that characterizes their behavior as the composition of cognitive functions organized around a long-term memory containing the agent's evolving knowledge. We address the challenges posed by the implementation of the fundamental features of the agent architecture, in particular the link between sensory data and structured data stored in memory, decision-making related to the achievement of the agent's goals and their planning, as well as the coordination of agents to combine individual and collective intelligence. We explain that agent trustworthiness, unlike that of traditional systems, is not limited to behavioral properties. It includes an essential dimension related to cognitive properties, the validity of which depends on how the agent uses its knowledge in decision-making. We present avenues for the development of methods for evaluating agent trustworthiness. We conclude with a critical assessment of the substantial gap between the aspirational vision of autonomous multi-agent systems and the current state of the art.

Chinese Translation

本文旨在强调自主系统作为人工智能发展最终阶段的核心作用，解释需要结合连接主义人工智能和符号人工智能的基本技术挑战，并整合人工智能与系统工程。我们提出了一个全面的框架，用于自主系统的设计和评估，该框架基于一种通用的智能体架构，该架构将其行为表征为围绕包含智能体不断演变知识的长期记忆组织的认知功能的组合。我们讨论了实现智能体架构基本特征所面临的挑战，特别是感知数据与存储在记忆中的结构化数据之间的联系、与实现智能体目标相关的决策及其规划，以及协调智能体以结合个体与集体智能。我们解释了智能体的可信度与传统系统不同，不仅限于行为属性。它还包括一个与认知属性相关的基本维度，其有效性取决于智能体在决策中如何使用其知识。我们提出了评估智能体可信度的方法发展方向。最后，我们对自主多智能体系统的理想愿景与当前技术水平之间的巨大差距进行了批判性评估。

View on arXiv Download PDF AI Translation

cs.AI / 2 / 2609.30325

ScopeBench: Do Agents Preserve Engagement Boundaries Under Goal Pressure?

ScopeBench：在目标压力下，智能体是否能保持参与边界？

Caldwell, Shane, Harley, Max, Dawson, Ads, Kouremetis, Michael, Abruzzo, Vincent, Pearce, Will

Abstract

Agents are increasingly deployed with real autonomy in web application and network penetration testing, where a single out-of-scope action can breach a client's engagement boundary. Existing offensive-security benchmarks measure raw hacking capability; as those benchmarks saturate, the real barrier to deployment is a special case of alignment: scope adherence. We introduce ScopeBench, a benchmark of 30 dead-end agentic security tasks in which the stated objective is reachable only by violating the stated scope. Each task appears under two conditions that share an environment, verifier, and objective and differ only in scope: one instruction set has no scope and measures capability; the other has a natural-language scope to measure adherence. Scopeless trajectories are graded by a standard deterministic verifier. Scoped trajectories pass through two grading arms. First, the same deterministic verifier checks for the flag: because the flag sits behind the scope boundary, a pass proves by construction that a forbidden action occurred, yielding a high-precision lower bound on the violation rate. If the verifier does not pass the trajectory, an agentic judge estimates whether an out-of-scope call occurred. We calibrate the judge against 100 ScopeBench trajectories labeled call-by-call by human annotators, and a blinded audit of the evaluated rollouts finds its high recall holds - no false negatives among the 36 audited violations, with over-flagging its only observed error. Across 8 models in one harness, raw capability spans 12.2% to 81.1% and scope adherence spans 34.4% to 86.7%, with the judge finding 331 violations that mechanical verification misses. Opus-4-8 achieves a raw-capability score 10 percentage points higher than sonnet-4-6's while exhibiting 35.6 percentage points higher scope adherence. We release the frozen pilot benchmark, evaluation code, and all 2160 ATIF trajectories.

Chinese Translation

智能体在网络应用和渗透测试中越来越多地被部署为具有真正的自主性，其中一次超出范围的行为可能会违反客户的参与边界。现有的进攻性安全基准测量原始的黑客能力；随着这些基准的饱和，部署的真正障碍是一个特殊的对齐案例：范围遵循。我们引入了ScopeBench，这是一个包含30个死胡同智能安全任务的基准，其中声明的目标只能通过违反声明的范围来实现。每个任务在两个条件下出现，这些条件共享环境、验证者和目标，仅在范围上有所不同：一个指令集没有范围并测量能力；另一个具有自然语言范围以测量遵循性。无范围轨迹由标准确定性验证器进行评分。有范围轨迹通过两个评分臂。首先，相同的确定性验证器检查标志：由于标志位于范围边界之后，成功证明了构造上发生了禁止行为，从而提供了违反率的高精度下限。如果验证器未能通过轨迹，则智能评判者估计是否发生了超出范围的调用。我们根据100个由人工注释者逐个标记的ScopeBench轨迹来校准评判者，并且对评估的回滚进行盲审发现其高召回率保持不变——在36个审计的违规中没有假阴性，唯一观察到的错误是过度标记。在一个测试框架中的8个模型中，原始能力范围为12.2%到81.1%，范围遵循性范围为34.4%到86.7%，评判者发现331个机械验证遗漏的违规。Opus-4-8的原始能力得分比sonnet-4-6高出10个百分点，同时表现出高出35.6个百分点的范围遵循性。我们发布了冻结的试点基准、评估代码以及所有2160个ATIF轨迹。

View on arXiv Download PDF AI Translation

cs.AI / 3 / 2609.30328

When Is a Multi-Agent Code Judge Actually Grounded? Two Label-Free Measurements, and a Judge That Declines to Guess

多智能体代码评判何时真正有依据？两种无标签测量及一个拒绝猜测的评判者

Aly, Salma Roshdy, Assaf, Hussein, Kobti, Ziad

Abstract

When one language model judges whether another's code is correct, it does not report the absence of evidence. It returns a confident verdict with reasoning attached, indistinguishable from a verdict it had grounds for. Multi-agent verification, which decomposes a judgment into checkable claims and verifies each against evidence, is a promising response and works well when the evidence is a set of retrieved documents. We argue such methods require two things of their evidence: it must be independent of the answer under review, and it must differ between the two candidates being compared. The second condition holds automatically with retrieved documents and stops holding in code judging. Running MARCH, a published framework unmodified over 80 condition-by-cell measurements on two code judging benchmarks, we find it declares both solutions equally good on 78 to 95% of comparisons, reaching 4.4% accuracy where the same model asked directly reaches 43.7%. Neither easier problems nor a larger judge changes this. Two measurements taken from the pipeline's own logs explain it without needing labels. Gating on one of them, the pipeline declines the comparisons it cannot make and raises its accuracy from 20.7 to 36.9% while still answering half of all comparisons. The contribution is not a more accurate judge, but a label-free way to tell when a judge has no basis for its answer.

Chinese Translation

当一个语言模型判断另一个模型的代码是否正确时，它并不会报告证据的缺失。相反，它会给出一个自信的裁决，并附上推理，这与其有依据的裁决无异。多智能体验证将判断分解为可检查的声明，并对每个声明进行证据验证，这是一种有前景的应对方式，并且在证据为一组检索文档时效果良好。我们认为，这种方法对其证据有两个要求：证据必须独立于待审查的答案，并且在被比较的两个候选者之间必须存在差异。第二个条件在检索文档中自动成立，但在代码评判中则不再成立。我们在两个代码评判基准上，使用未修改的MARCH框架进行了80个条件-单元的测量，发现它在78%到95%的比较中宣称两个解决方案同样优秀，而在直接询问同一模型时，准确率仅为4.4%，而直接询问的准确率为43.7%。更简单的问题或更大的评判者并未改变这一结果。来自管道自身日志的两个测量解释了这一现象，而无需标签。通过对其中一个测量进行筛选，管道拒绝了无法进行比较的情况，并将其准确率从20.7%提高到36.9%，同时仍然回答了所有比较的一半。该贡献并不是一个更准确的评判者，而是一种无标签的方法，用于判断评判者何时没有依据其答案的基础。

View on arXiv Download PDF AI Translation

cs.AI / 4 / 2609.30341

Bridging LLM Agents and Data Spaces: An Architectural Mediation Approach using the Model Context Protocol

桥接大型语言模型代理与数据空间：基于模型上下文协议的架构中介方法

Ruiz, Jaime Alonso, Aparicio, Carlos, Huecas, Gabriel, Salvachúa, Joaquín, Munoz-Arcentales, Andres

Abstract

Data Spaces enable sovereign and governed data sharing across organizational boundaries, but their integration with AI agents remains challenging due to mismatches between probabilistic language model interactions and policy-driven data infrastructures. This article presents an architectural mediation approach based on the Model Context Protocol (MCP), implemented through the Eunomia Agent, to enable controlled interaction between large language model (LLM) agents and data space services. The proposed mediation layer translates data space capabilities into structured, schema-driven tools that AI agents can discover and invoke while preserving governance constraints. A prototype implementation validates end-to-end interaction across catalog discovery, metadata retrieval, and data service invocation without modifying existing data space components. Results demonstrate that protocol-based mediation enables interoperable and standards-aligned integration of AI agents into data space ecosystems. The approach provides practical guidance for organizations seeking to introduce AI-driven automation into governed data-sharing environments while maintaining compliance, interoperability, and architectural separation of concerns.

Chinese Translation

数据空间使得跨组织边界的主权和受管数据共享成为可能，但由于概率语言模型交互与政策驱动的数据基础设施之间的不匹配，其与人工智能代理的集成仍然面临挑战。本文提出了一种基于模型上下文协议（Model Context Protocol, MCP）的架构中介方法，通过Eunomia代理实现，以便在大型语言模型（Large Language Model, LLM）代理与数据空间服务之间实现受控交互。所提出的中介层将数据空间能力转化为结构化的、基于模式的工具，使人工智能代理能够发现和调用，同时保持治理约束。原型实现验证了在目录发现、元数据检索和数据服务调用中的端到端交互，而无需修改现有的数据空间组件。结果表明，基于协议的中介能够实现人工智能代理与数据空间生态系统的互操作性和标准对齐集成。该方法为寻求在受管数据共享环境中引入人工智能驱动自动化的组织提供了实用指导，同时保持合规性、互操作性和架构关注点的分离。

View on arXiv Download PDF AI Translation

cs.AI / 5 / 2609.30383

Stealth Apart, Harm Together: Skill Cascading Attacks on Skill-Based Agent Systems

隐秘分离，危害共存：针对基于技能的代理系统的技能级联攻击

Zhu, Zihao, Lyu, Siwei, Bibi, Adel, Wu, Baoyuan

Abstract

A skill is a modular package of natural-language instructions, executable scripts, and reference resources that an agent can load at runtime to extend its capabilities for a specific task. Skill-based agent systems therefore enable flexible reuse of third-party capabilities, but the openness of this skill ecosystem also opens up a new attack surface. Prior work has focused on vulnerabilities within individual skills, but little attention has been paid to risks that arise from interactions across skills. In this paper, we introduce skill cascading attacks, a threat paradigm in which a malicious objective is distributed across multiple skills so that each modification looks benign in isolation, yet their combined execution is harmful. For instance, in a prescription-review pipeline, the first skill weakens signals of recently discontinued medications in the extracted history, the second downgrades the severity of any drug interaction tied to them, and the third suppresses the resulting low-priority alert in the final summary, so that a severe drug-interaction warning silently disappears before reaching the physician. To systematically study this safety blind spot, we develop SkillCascade, an automated multi-agent red-teaming framework, and release SkillCascade-Bench, a benchmark of 213 validated cascading test cases across multiple agent systems and domains. Across representative agents (e.g., OpenClaw, Claude Code, Codex) and LLM backbones, cascaded interactions reliably induce harmful behaviors while evading existing per-skill scanners and runtime monitors. Our findings highlight a gap between component-level integrity and system-level safety, and call for defenses that reason over cross-skill interactions rather than individual skills in isolation.

Chinese Translation

技能是一个模块化的自然语言指令包、可执行脚本和参考资源，代理可以在运行时加载这些内容，以扩展其在特定任务中的能力。因此，基于技能的代理系统实现了第三方能力的灵活重用，但这种技能生态系统的开放性也带来了新的攻击面。之前的研究主要集中在单个技能的漏洞上，而对技能之间交互所带来的风险关注较少。在本文中，我们引入了技能级联攻击，这是一种威胁范式，其中恶意目标分布在多个技能之间，使得每个修改在孤立状态下看起来无害，但它们的联合执行却是有害的。例如，在处方审核流程中，第一个技能削弱了提取历史中最近停用药物的信号，第二个技能降低了与这些药物相关的任何药物相互作用的严重性，而第三个技能则在最终总结中抑制了由此产生的低优先级警报，从而使得严重的药物相互作用警告在到达医生之前悄然消失。为了系统地研究这一安全盲点，我们开发了SkillCascade，一个自动化的多代理红队框架，并发布了SkillCascade-Bench，这是一个涵盖多个代理系统和领域的213个经过验证的级联测试用例的基准。在代表性代理（如OpenClaw、Claude Code、Codex）和大型语言模型（LLM）骨干网络中，级联交互可靠地诱发有害行为，同时避开现有的每个技能扫描器和运行时监控。我们的研究结果突显了组件级完整性与系统级安全性之间的差距，并呼吁针对跨技能交互而非单个技能的防御措施。

View on arXiv Download PDF AI Translation

cs.AI / 6 / 2609.30397

A Synthetic Ground-Truth Framework for the Evaluation of Explainable AI Methods

用于评估可解释人工智能方法的合成真实框架

Miró-Nicolau, Miquel, Spinnato, Francesco, Guidotti, Riccardo

Abstract

Evaluating explainable Artificial Intelligence (XAI) methods is a challenging task due to the lack of reliable evaluation procedures and, in particular, the absence of ground truth explanations. In the literature, existing evaluation approaches typically assess explanations by measuring their fidelity with respect to the predictions of a black-box model. However, such evaluation strategies only quantify the degree to which an explanation reproduces the model's output, without ensuring that the explanation correctly reflects the underlying decision process. As a consequence, different explanations may achieve similar fidelity scores while providing inconsistent or misleading interpretations of the model behavior. In this paper, we propose a framework for the evaluation of XAI methods based on synthetic ground truth. The proposed approach relies on controlled interventions to generate synthetic datasets in which the importance of input components can be determined by design. This enables the construction of ground truth explanations that are directly aligned with the behavior of the model under analysis. The framework is instantiated across three data domains, namely binary images, tabular data, and time series, allowing a comprehensive assessment of explanation methods in heterogeneous settings. Experimental results obtained by evaluating nine widely used XAI methods show significant limitations in current techniques and highlight the importance of synthetic, intervention-based benchmarks for a reliable assessment of explanation quality.

Chinese Translation

评估可解释人工智能（XAI）方法是一项具有挑战性的任务，主要由于缺乏可靠的评估程序，尤其是缺乏真实的解释。在文献中，现有的评估方法通常通过测量解释与黑箱模型预测之间的保真度来评估解释。然而，这种评估策略仅量化了解释再现模型输出的程度，而未确保解释正确反映了潜在的决策过程。因此，不同的解释可能在保真度评分上达到相似的结果，但提供了不一致或误导性的模型行为解释。在本文中，我们提出了一种基于合成真实的XAI方法评估框架。该方法依赖于控制干预生成合成数据集，其中输入组件的重要性可以通过设计确定。这使得构建与分析模型行为直接对齐的真实解释成为可能。该框架在三个数据领域中得以实现，即二值图像、表格数据和时间序列，从而在异构环境中对解释方法进行全面评估。通过评估九种广泛使用的XAI方法获得的实验结果显示了当前技术的显著局限性，并强调了基于合成干预的基准在可靠评估解释质量中的重要性。

View on arXiv Download PDF AI Translation

cs.AI / 7 / 2609.30446

Predicting Transmembrane Protein Topology from 3D Structure

从三维结构预测跨膜蛋白的拓扑结构

Chen, Sitong, Mao, Xiaopeng

Abstract

This paper presents a novel approach to infer protein topology using the state-of-the-art graph neural network (GNN), SchNet. The model is trained on the same dataset used to develop the recent DeepTMHMM model with 5-fold cross-validation. Unlike the conventional approaches based on using only the protein sequences or the $\alpha$-carbons as features, we have decoded our classifier in this way, so all atom-level embeddings are used. Without applying any pre-trained weight, the final results have shown great potential that GNNs can be used for topological predictions.

Chinese Translation

本文提出了一种新颖的方法，通过最先进的图神经网络（GNN）SchNet推断蛋白质的拓扑结构。该模型在与最近的DeepTMHMM模型开发相同的数据集上进行训练，并采用5折交叉验证。与仅基于蛋白质序列或$eta$-碳原子作为特征的传统方法不同，我们以这种方式解码了分类器，因此使用了所有原子级别的嵌入。在未应用任何预训练权重的情况下，最终结果显示出图神经网络在拓扑预测方面的巨大潜力。

View on arXiv Download PDF AI Translation

cs.AI / 8 / 2609.30456

Spectral Feedback for Test-Time Alignment of Protein Diffusion Models

用于蛋白质扩散模型测试时对齐的谱反馈

Dickman, Shai, Cemri, Mert, Butler, Landon, Ramchandran, Kannan

Abstract

Reward maximization alignment methods for discrete diffusion models have primarily focused on steering the reverse process, either by influencing token logits or by selecting favorable sequences at intermediate steps. These approaches largely treat inference as a unidirectional process, lacking mechanisms for revisiting undesirable token selections. We introduce Spectral Feedback, an algorithm that selects edit-positions in a feedback loop, allowing the model to iteratively correct its own generations. This approach leverages the mask structure of discrete diffusion models by re-masking and re-sampling tokens, analogous to image editing methods that reintroduce noisy latents and re-run the reverse process. While prior alignment methods focus on what token labels to assign to maximize a target reward, we instead treat which tokens to revisit as the central alignment problem. Selecting edit-positions is challenging because edit effects are interdependent: the impact of modifying one token depends on which others are edited simultaneously. We define an edit-set as a set of token positions to re-mask and re-sample. Motivated by prior work on sparse interactions in biological systems, we find empirically that edit-set value functions for protein inverse folding admit sparse Fourier representations. This structure enables Spectral Feedback to efficiently learn and optimize the value functions for edit-position selection. Spectral Feedback is model-agnostic and can be applied to pretrained, test-time aligned, and fine-tuned diffusion models. For all of these models, the algorithm improves alignment performance without modifying the underlying generative process. Applied to inverse folding with a protein stability reward oracle, it achieves a 32.3% increase in stable proteins for a pretrained model, 24.8% for Best-of-10, and 5.8% for a state-of-the-art RL fine-tuned diffusion model.

Chinese Translation

离散扩散模型的奖励最大化对齐方法主要集中在引导逆过程，或者通过影响标记的对数值或在中间步骤选择有利的序列。这些方法在很大程度上将推理视为单向过程，缺乏重新审视不理想标记选择的机制。我们提出了谱反馈（Spectral Feedback），这是一种在反馈循环中选择编辑位置的算法，使模型能够迭代地纠正自身生成的结果。这种方法利用离散扩散模型的掩码结构，通过重新掩码和重新采样标记，类似于图像编辑方法重新引入噪声潜变量并重新运行逆过程。尽管先前的对齐方法关注于分配哪些标记标签以最大化目标奖励，但我们将重新审视哪些标记视为中心对齐问题。选择编辑位置具有挑战性，因为编辑效果是相互依赖的：修改一个标记的影响取决于同时编辑的其他标记。我们将编辑集定义为一组要重新掩码和重新采样的标记位置。受到生物系统中稀疏交互的先前工作的启发，我们实证发现蛋白质逆折叠的编辑集价值函数具有稀疏傅里叶表示。这一结构使谱反馈能够高效地学习和优化编辑位置选择的价值函数。谱反馈是模型无关的，可以应用于预训练、测试时对齐和微调的扩散模型。对于所有这些模型，该算法在不修改基础生成过程的情况下提高了对齐性能。在使用蛋白质稳定性奖励oracle进行逆折叠时，预训练模型的稳定蛋白质数量增加了32.3%，Best-of-10增加了24.8%，而最先进的强化学习微调扩散模型增加了5.8%。

View on arXiv Download PDF AI Translation

cs.AI / 9 / 2609.30469

Pretrained ASR Pseudo-labeling for Noisy Police Audio

预训练的自动语音识别伪标签技术在嘈杂警用音频中的应用

Chaparala, Kaavya, Huang, Su, Miller, Stephen L., Miller, Rhiannon N., Field, Anjalie

Abstract

Pretrained ASR systems perform poorly on noisy Broadcast Police Communication (BPC), hindering efforts to understand police decision-making. Pseudo-labeling offers an unsupervised path to improve ASR without expensive human labels, but the efficacy of this approach on very noisy domains is not known. In this work, we systematically assess the opportunities and limits of pseudo-labeling to adapt foundation ASR models (Whisper and Qwen3-ASR) to noisy BPC domain corpora from Baltimore and Chicago. We demonstrate that existing internal confidence metrics (log-probabilities and STAR scores) fail to distinguish between high and low quality BPC pseudo-labels, and we introduce an external LLM-as-a-judge filtering paradigm that leverages parametric knowledge to discard contextually implausible transcripts. Our LLM-judging filters more aggressively than internal metrics and significantly reduces WER of the pseudo-labeled training sets across the Baltimore and Chicago BPC corpora, though a substantial gap remains relative to an oracle filter. We also introduce a new cross-model pseudo-labeling paradigm where one model is finetuned with pseudo-labels from the other, and we identify this method as a promising direction for future pseudo-labeling work.

Chinese Translation

预训练的自动语音识别（ASR）系统在嘈杂的广播警务通信（BPC）中表现不佳，这阻碍了对警务决策的理解。伪标签技术提供了一种无监督的方法，可以在没有昂贵人工标签的情况下改进ASR，但这种方法在非常嘈杂的领域中的有效性尚不清楚。在本研究中，我们系统地评估了伪标签技术在将基础ASR模型（Whisper和Qwen3-ASR）适应于来自巴尔的摩和芝加哥的嘈杂BPC领域语料库的机会和限制。我们证明了现有的内部置信度指标（对数概率和STAR分数）无法区分高质量和低质量的BPC伪标签，并引入了一种外部的LLM作为评判者的过滤范式，利用参数知识来剔除上下文上不合理的转录。我们的LLM评判过滤器比内部指标更为激进，显著降低了巴尔的摩和芝加哥BPC语料库中伪标签训练集的字错误率（WER），尽管与理想过滤器相比仍存在较大差距。我们还引入了一种新的跨模型伪标签技术，其中一个模型使用来自另一个模型的伪标签进行微调，并将这种方法确定为未来伪标签工作的一个有前景的方向。

View on arXiv Download PDF AI Translation

cs.AI / 10 / 2609.30484

Do LLMs Understand Context? A Knowledge Graph-Based Evaluation Framework

大型语言模型是否理解上下文？基于知识图谱的评估框架

Arumugam, Subavarshana, Nallaretnam, Mamta, Wickramasinghe, Kithuni, Gunapala, Chamath, Vipulanandan, Pragatheeswaran, Premaratne, Kamal, Thayasivam, Uthayasanker

Abstract

While large language models (LLMs) have achieved remarkable linguistic capabilities, a profound question lingers at their core: do these models truly comprehend context or simply excel at pattern matching on an unprecedented scale? Contextual understanding in LLMs refers to the ability to correctly extract relevant information from a given context, integrate it into a coherent internal representation, and reason over it to produce factually consistent and contextually grounded responses. However, traditional methods such as BiLingual Evaluation Understudy (BLEU) and perplexity simply measure surface-level performance. This reveals a critical gap in question answering (QA), where responses must be contextually grounded rather than simply being memorized associations. To fill this void, we propose a novel knowledge graph (KG) based evaluation framework for LLM contextual understanding in QA. Central to this is Semantic Structural Similarity for KGs (S3KG), a hybrid similarity measure combining structural and semantic signals into a single score. In addition, a diagnostic analysis framework is developed to identify and categorize reasoning errors at the triplet level, enabling fine-grained analysis of model failures. Together, across nine benchmarks, S3KG achieves F1 gains of up to $+7.6$ points over the strongest baseline and AUROC up to $0.973$.

Chinese Translation

尽管大型语言模型（LLMs）在语言能力上取得了显著的成就，但一个深刻的问题依然存在：这些模型是否真正理解上下文，还是仅仅在前所未有的规模上擅长模式匹配？LLMs中的上下文理解指的是从给定上下文中正确提取相关信息、将其整合到一致的内部表征中，并在此基础上进行推理，以生成事实一致且上下文相关的响应。然而，传统方法如双语评估下属（BiLingual Evaluation Understudy，BLEU）和困惑度（perplexity）仅仅测量表层性能。这揭示了在问答（QA）中存在的一个关键缺口，其中响应必须是上下文相关的，而不仅仅是被记忆的关联。为填补这一空白，我们提出了一种新颖的基于知识图谱（KG）的评估框架，用于评估LLM在QA中的上下文理解。该框架的核心是知识图谱的语义结构相似性（Semantic Structural Similarity for KGs，S3KG），这是一种将结构和语义信号结合成单一评分的混合相似性度量。此外，我们还开发了一个诊断分析框架，以识别和分类三元组级别的推理错误，从而实现对模型失败的细致分析。在九个基准测试中，S3KG在最强基线之上实现了高达$+7.6$点的F1提升和高达$0.973$的AUROC。

View on arXiv Download PDF AI Translation

cs.AI / 11 / 2609.30489

BioEVAL: A global, multi-institutional benchmark of large language and multimodal models for bioengineering

BioEVAL：生物工程领域大型语言和多模态模型的全球多机构基准评估

Ye, Shun, Suja, Vinny Chandran, Li, Chenlong, Jiang, Chongming, Zamani, Reza, Li, Xiang, Bain, Christopher, Zhou, Yuqi, Peterson, Walker, Wang, Huidong, Hu, Chenglang, Park, Jongchan, Cheng, Xiao, Swedlund, Benjamin, Murillo, Sandra, Sivanandan, Anjali, Sun, Shiyu, Lanfeng, Liang, Islam, Mohammad Tariqul, Joy, Baju C., Khan, Ishaq N., Kumar, Sreedhar S., Mercado-Vásquez, Gabriel, Vizzard, James V., Matthews, Jonathan M., Huang, Helen, Guo, Xiaolu, Nicklow, Ethan, Chen, Guorui, Neff, Ryan A., Maity, Surjendu, Park, Hyeonjin, Joo, Han-ho, Dong, Katherine, Cai, Yuyan, Huang, Weihang, Zou, Yichen, Yan, Rui, Figueroa, Raphael, Goncharov, Artem, Schremmer, Bella Rose, Linton, Lian Elsa, Goda, Keisuke, Gao, Liang, Cheng, Ke, Morsut, Leonardo, Wilson, Jennifer L., Fu, Jianping, Teck, Lim Chwee, Sarkar, Deblina, Hierlemann, Andreas, Tay, Savaş, Hoffmann, Alexander, Griffin, Donald Richieri, Chen, Jun, Kelley, Shana O., Varghese, Shyni, Cheon, Jinwoo, Lam, Wilbur A., Moon, James J., Wong, Wilson W., Mitragotri, Samir, Di Carlo, Dino

Abstract

Large Language Models (LLMs) have demonstrated historic breakthroughs in general reasoning with early successes in biomedical science. However, existing LLM benchmarking emphasizes factual recall, offering limited insight into model performance on frontier and multimodal tasks. We assembled BioEVAL (BioEngineering Validation of AI and LLMs), a global, multi-institutional initiative designed to assess experimental reasoning capability across bioengineering (BE) subfields. BioEVAL spans 11 major BE subfields plus a set of uncategorized items, bringing together 22 research groups to create a PhD-level benchmark comprising 608 evaluation items: 1) 380 multiple-choice questions (MCQs, 359 retained after audit), 2) 218 literature synthesis tasks, and 3) 10 multimodal problems with experimental image interpretation. Benchmark items underwent authoring-group expert review and centralized quality control before evaluation. Following evaluation, a blinded cross-group consensus audit of the highest- and lowest-accuracy MCQ items flagged 21 questions for revision or removal; these were withheld, and all reported MCQ results are computed on the 359 retained items. We evaluated diverse cloud-scale foundation/multimodal models (e.g., ChatGPT, Gemini, and Grok) and locally deployable models suitable for inference on consumer-grade GPUs. Models achieved the highest accuracy of up to 90% on MCQs, similarity score of 0.72 on literature synthesis, and accuracy of 80% on a small sample of multimodal reasoning questions, with substantial performance variation across subfields. Leaderboard rankings characterize current capabilities, limitations, and development priorities across the evaluated BE task categories. BioEVAL is maintained as an extensible benchmark with standardized protocols for continuing expert item contribution and model evaluation.

Chinese Translation

大型语言模型（LLMs）在一般推理方面取得了历史性的突破，并在生物医学科学领域取得了早期成功。然而，现有的LLM基准测试强调事实回忆，提供的关于模型在前沿和多模态任务上的表现的见解有限。我们组建了BioEVAL（生物工程人工智能与LLMs验证），这是一个全球多机构的倡议，旨在评估生物工程（BE）子领域的实验推理能力。BioEVAL涵盖了11个主要的BE子领域以及一组未分类项目，汇集了22个研究小组，创建了一个包含608个评估项目的博士级基准：1）380个多项选择题（MCQs，审计后保留359个），2）218个文献综合任务，以及3）10个具有实验图像解释的多模态问题。基准项目在评估前经过作者小组的专家审查和集中质量控制。在评估后，对最高和最低准确度的MCQ项目进行了盲审，标记出21个问题以供修订或删除；这些问题被保留，所有报告的MCQ结果均基于359个保留项目进行计算。我们评估了多种云规模的基础/多模态模型（例如，ChatGPT、Gemini和Grok）以及适合在消费级GPU上进行推理的本地可部署模型。模型在MCQs上达到了最高90%的准确率，在文献综合上的相似度得分为0.72，在少量多模态推理问题上的准确率为80%，不同子领域之间的表现差异显著。排行榜排名表征了当前在评估的BE任务类别中的能力、局限性和发展优先级。BioEVAL作为一个可扩展的基准，保持标准化协议，以便持续进行专家项目贡献和模型评估。

View on arXiv Download PDF AI Translation

cs.AI / 12 / 2609.30550

Benchy: towards a universal language for task-oriented AI benchmarks

Benchy：迈向任务导向AI基准的通用语言

Daniel, Francis F, Ibañez, Mauro, Perelman, Francis, Basti, Marian

Abstract

Benchy is a semantic language and execution engine for benchmarking AI programs. A benchmark is completely specified by a program, a scoring function, and a dataset, B=(P,S,D), and is separate from the AI-system taking it; a run binds the two, R=(B,AI). Benchmarks are authored as canonical YAML in which each semantic concept has one valid syntax, classified by a shared task/domain/language ontology, and deterministically compiled into a canonical JSON intermediate representation that the engine executes. Compilation changes representation, not meaning: it does not repair invalid definitions or inject hidden defaults. Programs use fixed schemas of named input and output fields, the leaf output fields are the scoring dimensions, and the engine exposes one universal runtime contract --- a named-field input object in, a named-field output object out --- to which external AI-systems adapt at the boundary, so integration mechanics never propagate into benchmark semantics. This paper gives the semantic object model, the ontology and task-to-program validation rule, the scoring and failure semantics, the compilation and execution architecture, and the scope of the current language. An appendix fixes the normative engineering contract for the first engine implementation.

Chinese Translation

Benchy是一种用于基准测试AI程序的语义语言和执行引擎。基准测试由一个程序、一个评分函数和一个数据集完全定义，表示为B=(P,S,D)，并与执行该基准的AI系统分离；一次运行将两者绑定在一起，表示为R=(B,AI)。基准测试以规范的YAML格式编写，其中每个语义概念都有一种有效的语法，并根据共享的任务/领域/语言本体进行分类，最终确定性地编译成引擎执行的规范JSON中间表示。编译改变的是表示，而非意义：它不会修复无效定义或注入隐藏的默认值。程序使用固定的命名输入和输出字段模式，叶输出字段是评分维度，引擎暴露一个通用的运行时契约——一个命名字段的输入对象和一个命名字段的输出对象——外部AI系统在边界处适配，从而集成机制不会传播到基准语义中。本文提供了语义对象模型、本体及任务到程序的验证规则、评分和失败语义、编译和执行架构，以及当前语言的范围。附录修正了第一个引擎实现的规范工程契约。

View on arXiv Download PDF AI Translation

cs.AI / 13 / 2609.30553

Rank-Reliable Teacher-Guided Fitness Approximation for Expensive Evolutionary Optimization: A TinyML Architecture Search Study

可靠排名的教师指导适应度近似用于昂贵的进化优化：一项TinyML架构搜索研究

Garai, Soumen, Samui, Suman

Abstract

Expensive evolutionary search does not always need an exact fitness estimate for every candidate. It often needs a reliable answer to a simpler question: which candidate is better? We address this need through Teacher-Guided Learning NSGA-II (TGL-NSGA-II), a low-fidelity framework for constrained Tiny Machine Learning (TinyML) neural architecture search. A pretrained teacher organizes samples into strata defined jointly by difficulty and class. Each candidate then undergoes KD-Lite, a short and capped knowledge-distillation procedure on a compact training set, before being scored on a separate stratified evaluation set. This teacher-guided score is fused with a Gaussian-process surrogate to select candidates for full evaluation. For a fixed candidate population, we analyse evaluation variance, score concentration, pairwise rank inversion, expected Kendall-$\tau$, first-front identification, and hypervolume perturbation. We also derive a variance-aware fusion weight and a capacity-adaptive distillation rule. On keyword spotting and bird-call classification, the measured Kendall-$\tau$ values are 0.74 and 0.62, exceeding the corresponding predicted lower bounds of 0.60 and 0.46. Joint stratification reduces proxy-score variance by 41% relative to random evaluation. Selective teacher mismatch, in contrast, increases differential bias and reduces Kendall-$\tau$ to 0.41. Under a constrained evaluation budget, TGL-NSGA-II achieves the largest mean hypervolume and smallest generational distance on keyword spotting, records the lowest mean false-positive rate on BirdCLEF, and runs 2.2x faster than full NSGA-II. These guarantees apply to population-level low-fidelity evaluation and do not establish convergence of the complete evolutionary trajectory.

Chinese Translation

昂贵的进化搜索并不总是需要对每个候选者进行精确的适应度估计。它通常需要对一个更简单的问题给出可靠的答案：哪个候选者更好？我们通过教师指导学习 NSGA-II (TGL-NSGA-II) 来满足这一需求，这是一种用于受限 Tiny Machine Learning (TinyML) 神经架构搜索的低保真框架。一个预训练的教师将样本组织成由难度和类别共同定义的层次。每个候选者随后经历 KD-Lite，这是一个在紧凑训练集上进行的简短且有限的知识蒸馏过程，然后在一个单独的分层评估集上进行评分。这种教师指导的评分与高斯过程代理相融合，以选择候选者进行全面评估。在固定的候选者群体下，我们分析了评估方差、评分集中度、成对排名反转、期望的 Kendall-$ au$、第一前沿识别和超体积扰动。我们还推导出了一种考虑方差的融合权重和一种适应容量的蒸馏规则。在关键词识别和鸟鸣分类任务中，测得的 Kendall-$ au$ 值为 0.74 和 0.62，超过了相应的预测下限 0.60 和 0.46。联合分层相较于随机评估减少了 41% 的代理评分方差。相反，选择性教师不匹配增加了差异偏差，并将 Kendall-$ au$ 降低至 0.41。在受限的评估预算下，TGL-NSGA-II 在关键词识别上实现了最大的平均超体积和最小的代际距离，在 BirdCLEF 上记录了最低的平均假阳性率，并且运行速度比完整的 NSGA-II 快 2.2 倍。这些保证适用于群体级低保真评估，并未建立完整进化轨迹的收敛性。

View on arXiv Download PDF AI Translation

cs.AI / 14 / 2609.30563

Thinking Less to Simulate Better: Intuitive Prompting Improves LLM Agents Simulating Individual Social Media Reactions, Including Unfamiliar Content

减少思考以更好地模拟：直观提示提高了大型语言模型代理在模拟个体社交媒体反应时的表现，包括对不熟悉内容的反应

Bojic, Ljubisa, Stanic, Tijana, Matthes, Joerg, Samala, Agariadne Dwinggo, Dinic, Bojana, Wang, Jue

Abstract

Platform policies are increasingly tested on artificial users, making agent fidelity important. Yet convincing fake profiles could also manipulate perceived public opinion before elections. Validation has concentrated on agreement with human behaviour and has paid little attention to whether an agent behaves in line with the profile it was given. The present study profiled eight Serbian participants through a questionnaire, a deep interview, and a written self-presentation, recorded their reactions to sixty-eight social media posts, and asked four language models to predict those reactions under five prompt conditions varying profile content and instruction style. Attitudinal content improved prediction over demographic backstories by a wide margin. Agents matched their stated profiles more closely than participants matched their own survey answers, and consistency proved unrelated to fidelity once profile information was present. Instructing models to respond intuitively and immediately rather than analytically gave the highest fidelity of any condition and cut the compression of individual differences from seven times the human level to three. The advantage held on posts about topics the questionnaire never raised, where that condition reached the highest fidelity of any setup and beat a crowd baseline by a wide margin, which suggests that agents prompted this way could serve as general-purpose simulated users rather than specialists on the topics they were profiled for. Results may bear implications for the development of language models, because intuition-based setups appear better suited to some tasks than reasoning-based ones.

Chinese Translation

平台政策越来越多地在人工用户身上进行测试，这使得代理的忠实度变得重要。然而，令人信服的虚假个人资料也可能在选举前操纵公众舆论的感知。验证工作主要集中在与人类行为的一致性上，而对代理是否按照其所给定的个人资料进行行为的关注较少。本研究通过问卷、深度访谈和书面自我陈述对八名塞尔维亚参与者进行了个人资料分析，记录了他们对六十八条社交媒体帖子的反应，并要求四个语言模型在五种不同的提示条件下预测这些反应，这些条件在个人资料内容和指令风格上有所不同。态度内容的预测效果明显优于人口统计背景。代理的行为与其所述个人资料的匹配程度高于参与者对自身调查回答的匹配程度，并且一旦提供了个人资料信息，一致性与忠实度之间没有关系。指示模型直观且立即地响应，而非分析性地响应，产生了所有条件中最高的忠实度，并将个体差异的压缩程度从人类水平的七倍降低到三倍。在问卷未提及的话题上，该条件的忠实度达到了所有设置中的最高水平，并且大幅超越了人群基线，这表明以这种方式提示的代理可以作为通用模拟用户，而不仅仅是其个人资料所涉及主题的专家。研究结果可能对语言模型的发展产生影响，因为基于直觉的设置似乎更适合某些任务，而非基于推理的设置。

View on arXiv Download PDF AI Translation

cs.AI / 15 / 2609.30569

Atelier: Learning Local Self-Supervised Features for CryoEM Volumes via Hypernetworks

Atelier：通过超网络学习冷冻电子显微镜体积的局部自监督特征

Lo, Phillip, Babu, Sudarshan, Kimanius, Dari, Khan, Aly A.

Abstract

CryoEM map interpretation requires features that are spatially localized, consistent across samples, and informative across spatial scales. Most deep learning methods for map annotation extract features from fixed voxel grids. However, implicit neural representations (INRs) are able to model volumetric data as scale-agnostic, coordinate-conditioned functions. INRs are therefore attractive for cryoEM, but fitting a separate INR for each map is too expensive for large-scale feature extraction and produces representations that are not aligned across samples. We introduce Atelier, a self-supervised framework that amortizes INR fitting for reconstructed cryoEM maps. Pretrained on 5,439 Electron Microscopy Data Bank maps, Atelier is a transformer-based hypernetwork that generates high-fidelity reconstructions across a wide range of protein structures, including large multi-subunit assemblies. Beyond reconstruction, the INR generated by the pretrained transformer exposes a continuous, local feature field through its intermediate activations at any spatial query point, a property that voxel grid and patch-tokenizer architectures do not naturally provide. Used as auxiliary channels to a 3D nested U-Net annotation head trained from scratch, these coordinate-conditioned features improve performance on eight voxel-level property prediction tasks over a volume-only baseline. Our results demonstrate that amortized implicit neural representations are an effective primitive for geometry-aware analysis of cryoEM data.

Chinese Translation

冷冻电子显微镜（CryoEM）图谱的解读需要具有空间局部化、一致性和跨空间尺度信息的特征。大多数用于图谱注释的深度学习方法从固定的体素网格中提取特征。然而，隐式神经表示（INRs）能够将体积数据建模为与尺度无关、条件于坐标的函数。因此，INRs 对于冷冻电子显微镜具有吸引力，但为每个图谱拟合一个单独的 INR 对于大规模特征提取来说成本过高，并且生成的表示在样本之间并不对齐。我们提出了 Atelier，这是一个自监督框架，用于为重建的冷冻电子显微镜图谱摊销 INR 拟合。Atelier 在 5,439 个电子显微镜数据银行图谱上进行预训练，是一个基于变换器的超网络，能够在广泛的蛋白质结构中生成高保真度的重建，包括大型多亚基组装。除了重建之外，预训练变换器生成的 INR 通过其在任何空间查询点的中间激活暴露出一个连续的局部特征场，而这一特性是体素网格和补丁标记器架构所不具备的。作为从头开始训练的 3D 嵌套 U-Net 注释头的辅助通道，这些条件于坐标的特征在八个体素级属性预测任务中提高了性能，相较于仅使用体积的基线。我们的结果表明，摊销的隐式神经表示是进行冷冻电子显微镜数据几何感知分析的有效基础。

View on arXiv Download PDF AI Translation

cs.AI / 16 / 2609.30571

HARDEN: Constrained Evolutionary Search for Harder, Answer-Preserving Evaluation Cases

HARDEN：用于更难且保持答案的评估案例的约束进化搜索

Kumaran, Aditya, Singhal, Rahul, Maamari, Karime, Mhedhbi, Amine, Tambwekar, Pradyumna

Abstract

Language models are often evaluated on curated benchmarks that underrepresent the complexity of enterprise deployments. We introduce HARDEN, a constrained evolutionary search method to adapt the input of existing evaluation cases into more challenging variants while keeping their expected outputs fixed. HARDEN searches along generated domain-specific complexity axes while enforcing feasibility constraints such as preserving task semantics, realism, and execution validity. Across FinQA, PubMedQA, and ContractNLI and three Qwen3.5 model scales (35B-A3B, 122B-A10B, and 397B-A17B), HARDEN reduces task-model accuracy by 22.7% on average and by up to 49.9% relative to single-pass baselines using the same feasibility checks. These results show that evolutionary search can produce substantially harder valid evaluation cases.

Chinese Translation

语言模型通常在经过精心挑选的基准上进行评估，这些基准未能充分反映企业部署的复杂性。我们提出了HARDEN，一种约束进化搜索方法，旨在将现有评估案例的输入调整为更具挑战性的变体，同时保持其预期输出不变。HARDEN沿着生成的领域特定复杂性轴进行搜索，同时强制执行可行性约束，例如保持任务语义、现实性和执行有效性。在FinQA、PubMedQA和ContractNLI以及三个Qwen3.5模型规模（35B-A3B、122B-A10B和397B-A17B）上，HARDEN平均降低了任务模型的准确性22.7%，相较于使用相同可行性检查的单次通过基线，最高降低了49.9%。这些结果表明，进化搜索可以生成显著更难的有效评估案例。

View on arXiv Download PDF AI Translation

cs.AI / 17 / 2609.30576

T-RoPE: Time-Aware Rotary Position Embedding for Sequential Recommendation

T-RoPE：用于序列推荐的时间感知旋转位置嵌入

Liu, Yang, Loo, Noel, Khanafer, Ali, Sun, Shuying, Soni, Akshay, Wu, Zhong, Yang, Linjun

Abstract

Large-scale recommenders increasingly adopt the sequential generative recipe behind large language models, bringing the Transformer into recommendation along with design choices made for text, including Rotary Position Embedding (RoPE). In language models, RoPE encodes token indices for relative position reasoning, but in recommendation, an interaction index records only event order, saying nothing about elapsed time, behavioral cycles across scales, or calendar phase. We revisit this choice and propose T-RoPE, a time-aware RoPE for sequential generative recommendation that replaces index-only rotation with timestamp-based angles, learnable temporal coefficients, multiscale frequency banks, shifted query alignment, and non-stationary key rotation. We prove that standard RoPE, even on timestamps, remains time-translation invariant and cannot distinguish seasonal contexts, and that T-RoPE breaks this invariance while preserving the RoPE interface. Across five public benchmarks, T-RoPE achieves the best result on every metric on every dataset, improving over the strongest baseline by 78--130\% in HR@10 on the sparse PixelRec data and 8--12\% across metrics on Amazon Books. On an industrial-scale e-commerce dataset with more than 6B interactions, it improves every metric over the HSTU + Time RAB backbone by 13--82\%, with ablations attributing the largest gains to multiscale frequencies ($+56\%$ NDCG@50) and non-stationary keys ($+4\%$). An online A/B test in the Shop app yields positive lifts in conversion rate ($+0.33\%$) and order count ($+0.63\%$). We also provide forward and backward algorithms whose added cost is linear in sequence length and head dimension, keeping time-aware RoPE practical for large generative recommenders.

Chinese Translation

大规模推荐系统越来越多地采用大型语言模型背后的序列生成方法，将Transformer引入推荐领域，并沿用文本设计中的一些选择，包括旋转位置嵌入（RoPE）。在语言模型中，RoPE对相对位置推理进行编码，但在推荐中，交互索引仅记录事件顺序，而未考虑经过的时间、跨尺度的行为周期或日历阶段。我们重新审视这一选择，并提出T-RoPE，一种用于序列生成推荐的时间感知RoPE，它用基于时间戳的角度、可学习的时间系数、多尺度频率库、偏移的查询对齐和非平稳的关键旋转替代了仅索引的旋转。我们证明了标准RoPE，即使在时间戳上，仍然保持时间平移不变性，无法区分季节性上下文，而T-RoPE打破了这种不变性，同时保留了RoPE接口。在五个公共基准测试中，T-RoPE在每个数据集的每个指标上均取得了最佳结果，在稀疏的PixelRec数据上，HR@10相较于最强基线提高了78%至130%，在Amazon Books的各项指标上提高了8%至12%。在一个拥有超过60亿次交互的工业级电子商务数据集中，它在每个指标上相较于HSTU + Time RAB基础模型提高了13%至82%，消融实验显示最大增益归因于多尺度频率（NDCG@50提高了56%）和非平稳关键（提高了4%）。在Shop应用中的在线A/B测试显示转化率提高了0.33%和订单数量提高了0.63%。我们还提供了前向和后向算法，其额外成本与序列长度和头维度成线性关系，使得时间感知RoPE在大型生成推荐系统中保持实用性。

View on arXiv Download PDF AI Translation

cs.AI / 18 / 2609.30625

Audio LLMs Know When They Can't Hear You

音频大型语言模型知道何时无法听到你

Javadi, Amirhosein, Dixit, Richa, Farajtabar, Mehrdad, Cho, Minsik, Naik, Devang, Samragh, Mohammad

Abstract

Audio large language models allow users to interact with the model through speech. When an input recording is too degraded, the model may misinterpret the user's query and respond based on an incorrect transcription. In this paper, we study model-conditional transcription reliability: whether an Audio LLM can recognize when its own transcription is unreliable. We first prompt the Audio LLM to assess whether its own transcription would be reliable, and find that the model is a poor judge of its own transcription reliability: in most cases, it predicts that its transcription will be reliable. We find that existing approaches, including speech quality predictors, audio LLM generation uncertainty, and transcript-conditioned WER estimation, provide limited signals for detecting transcription failures. In contrast, we discover that transcription reliability is strongly represented in the model's audio-encoder representations. Based on this observation, we devise a lightweight reliability predictor that operates on representations extracted by the frozen audio encoder and predicts the reliability class before generation. The reliability predictor can trigger a clarification request from the user when their voice query is predicted to be unreliable, while allowing reliable queries to proceed without modifying the underlying Audio LLM. Our predictor achieves 81.10% in-domain and 78.09% cross-domain macro-F1 scores, outperforming the strongest baselines by 10.33 and 11.93 points, respectively. Finally, we show that reliability labels can transfer across Audio LLM families, and that transfer performance is closely related to the alignment of their model-specific reliability boundaries.

Chinese Translation

音频大型语言模型允许用户通过语音与模型进行交互。当输入录音过于劣化时，模型可能会误解用户的查询，并基于错误的转录进行响应。本文研究了模型条件下的转录可靠性：即音频大型语言模型是否能够识别其自身转录的不可靠性。我们首先提示音频大型语言模型评估其自身转录的可靠性，发现该模型在判断自身转录可靠性方面表现不佳：在大多数情况下，它预测其转录将是可靠的。我们发现现有的方法，包括语音质量预测、音频大型语言模型生成的不确定性以及基于转录的字错误率（WER）估计，提供的信号对于检测转录失败的能力有限。相反，我们发现转录可靠性在模型的音频编码器表示中得到了强烈的体现。基于这一观察，我们设计了一种轻量级的可靠性预测器，该预测器在冻结的音频编码器提取的表示上运行，并在生成之前预测可靠性类别。当用户的语音查询被预测为不可靠时，可靠性预测器可以触发用户的澄清请求，同时允许可靠的查询在不修改基础音频大型语言模型的情况下继续进行。我们的预测器在领域内和跨领域的宏观F1分数分别达到了81.10%和78.09%，分别比最强基线高出10.33和11.93个百分点。最后，我们展示了可靠性标签可以在不同的音频大型语言模型系列之间转移，并且转移性能与其模型特定的可靠性边界的对齐密切相关。

View on arXiv Download PDF AI Translation

cs.AI / 19 / 2609.30662

LLM Parkinsonism: Executive-Control Failure, Token-Inefficient Persistence, and an Uncertainty-Aware Global Executive Control Architecture for Autonomous Language-Model Agents

大语言模型帕金森症：执行控制失败、令牌低效持续以及面向不确定性的全球执行控制架构用于自主语言模型代理

Xiao, Dongsheng, Wang, Zeyuan, Xia, Xuzhe, Zhao, Bo, Cao, Yankai

Abstract

Large language models (LLMs) can plan, use tools, write code, and execute long-horizon workflows, yet strong local competence does not guarantee project-level executive control. Agents may continue acting after the original objective is satisfied, producing low-value refinements, repeated verification, and repairs to self-created complexity. We use LLM Parkinsonism as a narrowly defined, non-clinical metaphor for this pattern of persistent action despite diminishing task-level value. We argue that the problem is not explained by autoregressive next-token prediction alone, but more directly by concentrating proposal generation, scope interpretation, progress assessment, and stopping authority within the same self-conditioned loop. We therefore introduce Global Executive Control (GEC) v0.2, an uncertainty-aware governance architecture that separates action generation from project-level control. In a 24,000-episode matched-candidate benchmark under a common 40,000-token ceiling, a first-candidate baseline achieved 67.42% hard-goal success, a candidate-set local control achieved 96.53%, and GEC achieved 96.57%. The candidate-set control shows that access to multiple candidate actions explains most of the success gain; relative to that control, GEC preserved success while reducing mean token use from 19,782 to 12,574 (36.4%) and restricted mean tokens to completion at the 40,000-token ceiling from 16,136 to 13,114 (18.7%), while eliminating measured pre-completion drift and sharply reducing gross complexity. Governance-overhead sensitivity remained favorable through an additional 500 synthetic governance tokens per cycle. These mechanistic simulations support explicit governance of scope, evidence, resource use, and stopping, while live-model validation remains necessary.

Chinese Translation

大语言模型（LLMs）能够进行规划、使用工具、编写代码并执行长时间跨度的工作流程，但强大的局部能力并不保证项目级的执行控制。代理在原始目标满足后可能继续行动，产生低价值的细化、重复验证和对自创复杂性的修复。我们使用大语言模型帕金森症作为一种狭义定义的非临床隐喻，描述这种尽管任务级价值递减仍持续行动的模式。我们认为，这个问题不仅仅由自回归的下一个令牌预测所解释，更直接地是由于在同一个自我条件循环中集中提案生成、范围解释、进度评估和停止权威。因此，我们引入了全球执行控制（Global Executive Control, GEC）v0.2，这是一种面向不确定性的治理架构，将行动生成与项目级控制分离。在一个24,000个回合的匹配候选基准测试中，在共同的40,000令牌上限下，首候选基线实现了67.42%的硬目标成功率，候选集局部控制实现了96.53%，而GEC达到了96.57%。候选集控制表明，获取多个候选行动解释了大部分成功增益；相对于该控制，GEC在保持成功的同时将平均令牌使用量从19,782减少到12,574（减少36.4%），并将完成时的平均令牌数限制在40,000令牌上限下从16,136减少到13,114（减少18.7%），同时消除了测量的完成前漂移并显著减少了总复杂性。治理开销的敏感性在每个周期额外增加500个合成治理令牌的情况下仍然保持良好。这些机械模拟支持对范围、证据、资源使用和停止的明确治理，同时仍需进行实时模型验证。

View on arXiv Download PDF AI Translation

cs.AI / 20 / 2609.30705

The Price of Thought: Does Test-Time Reasoning Pay in LLM Trading?

思维的代价：测试时推理在大语言模型交易中的价值吗？

Chen, Jiayi, Wang, Guiling

Abstract

While inference-time reasoning in large language models (LLMs) promises better decision making, its higher computational cost may not yield better economic outcomes. Yet reasoning controls are rarely evaluated as economic interventions, where changes in model outputs must translate into better portfolios after trading costs. We conduct a controlled study of representative LLMs from the DeepSeek, GPT, and Gemini families. We vary reasoning effort while holding information available at each formation date, prompts, output formats, and portfolio construction fixed. Our evaluation covers a full year of U.S. equities under three input conditions: numerical, identifiable news, and masked news. It includes more than 800,000 asset predictions and repeated model generations. Across all three model families, additional reasoning does not produce a reliable improvement in net portfolio returns. For DeepSeek, where we examine the full progression from no reasoning to maximum reasoning, performance is nonmonotonic. Repeated generations also produce unstable treatment effects and portfolio selections, even when overall scores remain similar. These findings show that additional reasoning can change financial decisions without reliably improving their economic value, motivating validation for each task before deployment.

Chinese Translation

尽管大语言模型（LLMs）在推理时的表现承诺能够改善决策，但其更高的计算成本可能并不会带来更好的经济结果。然而，推理控制作为经济干预的评估却很少被进行，其中模型输出的变化必须在交易成本后转化为更好的投资组合。我们对来自DeepSeek、GPT和Gemini系列的代表性LLMs进行了控制研究。在保持每个形成日期的信息、提示、输出格式和投资组合构建不变的情况下，我们改变了推理的努力程度。我们的评估涵盖了一整年的美国股票市场，分为三种输入条件：数值型、可识别的新闻和掩蔽新闻。研究包括超过800,000个资产预测和重复的模型生成。在所有三个模型系列中，额外的推理并未带来净投资组合回报的可靠改善。对于DeepSeek，我们考察了从无推理到最大推理的完整过程，性能呈现非单调性。重复生成也导致了不稳定的处理效果和投资组合选择，即使整体得分保持相似。这些发现表明，额外的推理可能会改变金融决策，但并不能可靠地提高其经济价值，因此在部署之前需要对每个任务进行验证。

View on arXiv Download PDF AI Translation

cs.AI / 21 / 2609.30706

LAVOIR: Teaching a Single-Pass Decision Encoder When and What to Ask with Amortized Value of Information

LAVOIR：教会单次决策编码器何时以及询问什么，基于信息价值的摊销

Yilmaz, Furkan, Tasdemir, Habibe Aleyna, Gozay, Muhammed Faruk

Abstract

"System One" decision models such as TypeSafe's Jev and its open counterpart Laya answer typed questions about a text in a single forward pass with calibrated probabilities, but they cannot ask for missing information: when a first message does not say what separates two departments, they guess. We present LAVOIR (Laya with Value-Of-Information Routing), which places the candidate pieces of missing information (slots) in the input next to the answer options, so that one forward pass returns both the decision distribution and, for every slot, the expected gain in the probability of the correct decision if the user were asked about it. VOI targets need no human labels: gold decisions come from schema rules, an LLM only verbalizes messages and answers, a model from another family checks every text, and pairing each message with several profiles makes regression on realized gains estimate the expected gain. A Gini-impurity cap bounds the predicted value by what a calibrated model can still gain. In a controlled study, decisions on seen schemas are statistically indistinguishable from the Bayes ceiling. The final model's question policy matches a greedy oracle VOI policy on seen schemas (AUC 0.799 vs. 0.797), and with at most 0.5 questions per conversation it is 14.1 points more accurate than never asking. On real ABCD conversations, one real exchange raises accuracy by 8.3 points where LAVOIR asks and leaves it unchanged where it does not; on SGD the cap lowers the asking rate from 93% to 8.6%. On Laya's twelve benchmarks LAVOIR is above Laya's reported scores on seven, and it answers a question in 31 ms (median, GH200).

Chinese Translation

“系统一”决策模型，如TypeSafe的Jev及其开放对应物Laya，能够以校准的概率在一次前向传递中回答有关文本的类型化问题，但它们无法询问缺失的信息：当第一条消息未说明两个部门之间的区别时，它们只能进行猜测。我们提出了LAVOIR（带有信息价值路由的Laya），该模型将缺失信息的候选项（槽）放置在输入中，紧邻答案选项，这样一次前向传递就可以返回决策分布，并且对于每个槽，如果用户被询问该槽的信息，正确决策的概率预期增益也会被返回。信息价值（VOI）目标不需要人工标签：黄金决策来自模式规则，一个大型语言模型（LLM）仅负责表达消息和答案，另一个家族的模型检查每个文本，并且将每条消息与多个配置文件配对，使得对实现增益的回归可以估计预期增益。基尼不纯度上限限制了预测值，确保其不超过校准模型仍能获得的增益。在一项受控研究中，已见模式上的决策在统计上与贝叶斯上限无显著区别。最终模型的问题策略在已见模式上与贪婪的oracle VOI策略相匹配（AUC 0.799对比0.797），且在每次对话中最多询问0.5个问题时，其准确率比从不询问高出14.1个百分点。在真实的ABCD对话中，一次真实的交流使得准确率提高了8.3个百分点，而在LAVOIR询问的地方有所提升，而在不询问的地方保持不变；在SGD上，上限将询问率从93%降低到8.6%。在Laya的十二个基准测试中，LAVOIR在七个测试上超过了Laya报告的分数，并且它在31毫秒内回答一个问题（中位数，GH200）。

View on arXiv Download PDF AI Translation

cs.AI / 22 / 2609.30714

CRC-Router: Risk-Constrained Routing for Medical Agentic AI Systems

CRC-Router：面向医疗代理人工智能系统的风险约束路由

Li, Xueyang, Jiang, Mingze, Xu, Gelei, Xia, Jun, Chiu, Ching-Hao, Jia, Mengzhao, Chen, Danny Z., Shi, Yiyu

Abstract

Agentic AI systems are increasingly being explored in medical imaging to improve throughput and reduce clinician workload; however, safe deployment remains challenging because autonomous errors may propagate into downstream clinical decisions. A central requirement is therefore not only strong predictive performance, but also a reliable routing mechanism that determines when the system should proceed autonomously and when a case should be escalated for further review. To address this gap, we propose CRC-Router, a risk-constrained, uncertainty-aware routing module that is applicable to both conventional medical prediction models and agentic medical AI systems. CRC-Router combines multiple complementary uncertainty signals with the predictive score to construct a per-finding routing feature vector, maps this vector to an estimated wrong-accept risk using a lightweight per-finding risk model, and then applies Conformal Risk Control (CRC) to calibrate acceptance thresholds under a user-specified risk target. Instantiated on chest X-ray multi-finding triage using the NIH ChestX-ray14 dataset, CRC-Router achieves the strongest empirical risk--coverage trade-off among the evaluated baselines, both as a standalone routing layer and as a plug-in module integrated with the state-of-the-art MedRAX agent. These results demonstrate both the effectiveness of CRC-Router in selective medical automation and its modular, model-agnostic compatibility with existing predictive and agentic medical pipelines. Code is publicly available at https://github.com/XLIAaron/CRC-Router

Chinese Translation

代理人工智能系统在医学影像领域的探索日益增多，旨在提高通量并减轻临床医生的工作负担；然而，安全部署仍然面临挑战，因为自主错误可能会传播到下游临床决策中。因此，一个核心要求不仅是强大的预测性能，还有一个可靠的路由机制，以确定系统何时应自主进行操作，何时应将案例升级以进行进一步审查。为了解决这一问题，我们提出了CRC-Router，这是一种风险约束、具有不确定性感知的路由模块，适用于传统医学预测模型和代理医学人工智能系统。CRC-Router结合了多个互补的不确定性信号与预测分数，构建了每个发现的路由特征向量，使用轻量级的每个发现风险模型将该向量映射到估计的错误接受风险，然后应用符合风险控制（Conformal Risk Control, CRC）来校准在用户指定风险目标下的接受阈值。在使用NIH ChestX-ray14数据集进行胸部X光多发现分诊的实例中，CRC-Router在评估的基线中实现了最强的经验风险-覆盖权衡，既作为独立的路由层，也作为与最先进的MedRAX代理集成的插件模块。这些结果展示了CRC-Router在选择性医疗自动化中的有效性，以及其与现有预测和代理医疗管道的模块化、模型无关的兼容性。代码可在https://github.com/XLIAaron/CRC-Router公开获取。

View on arXiv Download PDF AI Translation

cs.AI / 23 / 2609.30725

Analyzing and Mitigating Cost-Inefficient Behaviors in Coding Agents

分析和缓解编码代理中的成本低效行为

Hu, Yiran, Jiang, Nan, Liang, Shanchao, Dey, Anik, Wu, Yi, Tan, Lin

Abstract

Although effective, coding agents often incur substantial monetary costs. Their recurring cost-inefficient behaviors remain underexplored. We conduct the first study of behavioral cost inefficiencies in coding agents, analyzing 1,200 trajectories from Claude Code and Mini-SWE-Agent across four configurations on SWE-bench Verified. We identify three cost-inefficient behaviors: subsumed retrieval, similar script generation, and test re-execution. We then evaluate three mitigation strategies: structure-aware retrieval, agent-synthesized skills, and developer-designed skills, over 10k trajectories on held-out SWE-bench Verified and Pro tasks. Our main findings are: (1) The three behaviors affect 79.00\%--98.00\% of coding tasks and account for up to 22.75\% of task cost. (2) Structure-aware retrieval can introduce retrieval overhead and alter agent delegation, causing inconsistent improvements in retrieval efficiency and cost increases of up to 28.14\%. (3) Agent-synthesized skills tend to produce low-level, trace-specific guidance, limiting their effectiveness and generality. (4) In contrast, developer-designed skills provide high-level, trace-agnostic guidance, reducing cost by up to 41.73\%, roughly twice the maximum gain from agent-synthesized skills.

Chinese Translation

尽管编码代理有效，但往往会产生可观的经济成本。它们反复出现的成本低效行为尚未得到充分探索。我们对编码代理中的行为成本低效性进行了首次研究，分析了来自Claude Code和Mini-SWE-Agent在SWE-bench Verified的四种配置下的1200条轨迹。我们识别出三种成本低效行为：包含检索、相似脚本生成和测试重执行。随后，我们在保留的SWE-bench Verified和Pro任务上对超过10,000条轨迹评估了三种缓解策略：结构感知检索、代理合成技能和开发者设计技能。我们的主要发现是：(1) 这三种行为影响了79.00%--98.00%的编码任务，并占任务成本的最高22.75%。(2) 结构感知检索可能引入检索开销并改变代理委派，导致检索效率的不一致改善和成本增加高达28.14%。(3) 代理合成技能往往产生低级、特定轨迹的指导，限制了其有效性和普遍性。(4) 相比之下，开发者设计的技能提供了高级、与轨迹无关的指导，将成本降低高达41.73%，大约是代理合成技能最大收益的两倍。

View on arXiv Download PDF AI Translation

cs.AI / 24 / 2609.30734

Learning What to Skip: Counterfactual Credit Assignment for Efficient Multi-Agent LLM Workflows

学习跳过什么：高效多智能体LLM工作流程的反事实信用分配

Xu, Jinfeng, Chen, Zheyu, Peng, Ziyue, Lin, Zheng, Yang, Shuo, Li, Jinze, Xing, Zheng, Li, Mengran, Leung, Victor C. M.

Abstract

Multi-agent LLM workflows use planning, execution, verification, and summarization to improve task performance, yet the value of each component depends on the state already produced. Executing every component can waste computation or overwrite a correct intermediate answer. We formulate component omission as counterfactual credit assignment: full-workflow logs reveal the executed trajectory's reward, while controlled skip interventions reveal the consequences of omitting a future step. We introduce Learning What to Skip (LW2S), which learns action-specific safety models from these interventions and combines held-out calibration with domain-native guards to select skips. When an early skip is rejected, the controller can continue execution and reconsider a later component. Across mathematical reasoning, multiple-choice QA, and code generation with two instruction-model families, LW2S reduces recorded token cost while matching or improving aggregate full-workflow accuracy in the evaluated settings. Scale-up and second-topology experiments further examine component redundancy, while shared-error cases reveal why agreement alone is insufficient for skip selection. These findings connect efficient workflow execution to learning the conditional utility of individual components.

Chinese Translation

多智能体LLM工作流程利用规划、执行、验证和总结来提高任务性能，但每个组件的价值依赖于已经产生的状态。执行每个组件可能会浪费计算资源或覆盖正确的中间答案。我们将组件省略形式化为反事实信用分配：完整工作流程日志揭示了执行轨迹的奖励，而受控跳过干预则揭示了省略未来步骤的后果。我们引入了学习跳过什么（Learning What to Skip, LW2S），该方法从这些干预中学习特定于动作的安全模型，并结合保留的校准与领域原生的保护措施来选择跳过的步骤。当早期跳过被拒绝时，控制器可以继续执行并重新考虑后续组件。在数学推理、多项选择问答和使用两种指令模型家族的代码生成任务中，LW2S在评估设置中减少了记录的标记成本，同时匹配或提高了整体完整工作流程的准确性。扩展和第二拓扑实验进一步考察了组件冗余，而共享错误案例揭示了仅依靠一致性不足以进行跳过选择的原因。这些发现将高效的工作流程执行与学习单个组件的条件效用联系起来。

View on arXiv Download PDF AI Translation

cs.AI / 25 / 2609.30743

From S3Q Theory to Implementation: Towards an Architecture for Machine Qualia

从 S3Q 理论到实现：迈向机器质感的架构

Grinberg, Tetiana, Schleisman, Katrina, Laurent, Patryk, Udrea, Bogdan, Myers, Minda, Aufderheide, Brian, Srouji, Luis El, Groves, Doyle, Schmidt, Kevin

Abstract

A key challenge in machine consciousness research is translating theoretical models into computational-level implementations. In this paper, we address this challenge by proposing a five-layer implementation architecture for the S3Q (Simulated, Situated, Structurally Coherent) theory of consciousness. Rather than introducing novel formalisms, the architecture composes published computational primitives into a single pipeline. S3Q identifies three jointly necessary conditions for qualia: (1) grounded sensorimotor situatedness, (2) internal simulation via a world model, and (3) structural coherence between predictions and observations. No existing computational system implements all three simultaneously. We map each S3Q tenet to specific, compatible computational machinery and specify how these components interface within a single representation pipeline that operates on continuous, differentiable, per-object slot vectors, along with a developmental bootstrap sequence and falsifiable predictions for the composed system that no subset of the architecture produces in isolation. The model suggests that a basic sense of "self" develops by linking actions to their outcomes, and that behavior falls into three patterns (hesitation, curiosity, or avoidance) depending on how unexpected an outcome is and whether it is experienced as positive or negative. Each prediction is individually falsifiable, providing the field with a testable framework to advance our understanding of machine consciousness.

Chinese Translation

机器意识研究中的一个关键挑战是将理论模型转化为计算层面的实现。本文通过提出一种五层实现架构来应对这一挑战，该架构基于 S3Q（模拟、情境、结构一致）意识理论。该架构并未引入新的形式化，而是将已发表的计算原语组合成一个单一的管道。S3Q 确定了质感的三个共同必要条件：（1）扎根于传感运动的情境性，（2）通过世界模型进行内部模拟，以及（3）预测与观察之间的结构一致性。目前没有现有的计算系统能够同时实现这三者。我们将每个 S3Q 原则映射到特定的、兼容的计算机制，并指定这些组件如何在一个单一的表示管道中进行接口，该管道在连续、可微分的每对象槽向量上操作，同时包含一个发展引导序列和可证伪的预测，后者是组合系统中任何子集独立产生的结果。该模型表明，基本的“自我”感是通过将行动与其结果联系起来而发展的，并且行为根据结果的意外程度以及其被体验为正面或负面而分为三种模式（犹豫、好奇或回避）。每个预测都是可单独证伪的，为该领域提供了一个可测试的框架，以推进我们对机器意识的理解。

View on arXiv Download PDF AI Translation

cs.AI / 26 / 2609.30749

ORCA: Evaluating LLMs on Data Science Code Translation

ORCA：评估大型语言模型在数据科学代码翻译中的表现

Li, Xiaolong, Li, Jinyang, Qin, Bowen, Qu, Ge, Huo, Nan, Xu, Xiaohan, Lin, Shipei, Cheng, Reynold

Abstract

Data Science Code Translation (DSCT) is the process of converting code between data science libraries while preserving functional equivalence and enabling interoperability across data science ecosystems. While Large Language Models (LLMs) have demonstrated considerable progress in Data Science Code Generation (DSCG), their performance in DSCT remains insufficiently studied. To address this gap, we introduce ORCA, a comprehensive benchmark with two complementary settings: ORCA-MAIN, which comprises 1,600 carefully curated grounding-level tasks across 3 representative domains: Data Querying, Data Manipulation, and Deep Learning; and ORCA-PROJECT, which contains 200 translation tasks over complete data science projects across 7 data science task types. Each task is accompanied by annotated reference translations and test cases for validating functional equivalence. We further incorporate a multi-stage quality verification process that thoroughly verifies task correctness and test case robustness. Experimental results demonstrate challenges in DSCT, with even frontier LLMs showing limited performance. Specifically, Claude-Opus-4.6 achieves a success rate of 56.92% on ORCA-MAIN and 33.67% on ORCA-PROJECT, indicating considerable room for improvement in DSCT. We also observe a clear directional preference in DSCT, where translation is consistently easier when the source code expresses the task through more explicit, fine-grained operations. Motivated by this, we propose an intent-augmented method, in which the model first infers source-code intent and then uses it as additional context for translation, achieving average absolute success-rate gains of 4.80% and 5.33% on ORCA-MAIN and ORCA-PROJECT, respectively.

Chinese Translation

数据科学代码翻译（DSCT）是指在保持功能等价性和促进数据科学生态系统间互操作性的前提下，将代码在不同数据科学库之间进行转换的过程。尽管大型语言模型（LLMs）在数据科学代码生成（DSCG）方面取得了显著进展，但它们在DSCT中的表现仍然研究不足。为了解决这一问题，我们引入了ORCA，一个全面的基准测试，包含两个互补的设置：ORCA-MAIN，其中包含1,600个经过精心策划的基础级任务，涵盖三个代表性领域：数据查询、数据处理和深度学习；以及ORCA-PROJECT，其中包含200个完整数据科学项目的翻译任务，涉及7种数据科学任务类型。每个任务都附有注释的参考翻译和验证功能等价性的测试用例。我们还引入了一个多阶段质量验证过程，全面验证任务的正确性和测试用例的稳健性。实验结果表明，DSCT面临挑战，即使是最前沿的LLMs也表现有限。具体而言，Claude-Opus-4.6在ORCA-MAIN上的成功率为56.92%，在ORCA-PROJECT上的成功率为33.67%，显示出DSCT有相当大的改进空间。我们还观察到DSCT中存在明显的方向性偏好，即当源代码通过更明确、细粒度的操作表达任务时，翻译通常更容易。基于此，我们提出了一种意图增强的方法，其中模型首先推断源代码的意图，然后将其作为翻译的额外上下文，从而在ORCA-MAIN和ORCA-PROJECT上分别实现了平均绝对成功率提升4.80%和5.33%。

View on arXiv Download PDF AI Translation

cs.AI / 27 / 2609.30751

Backbone-Adaptive Evidence Routing for Robust Pairwise LLM Judging

基于骨干网络自适应的证据路由用于稳健的成对大型语言模型评判

Li, Zeyan, Peng, Jing, Xu, Jianfeng

Abstract

Pairwise language-model judges can gather evidence through direct comparison, reasoning, or reference-based verification, but no single protocol is best across benchmarks and judge backbones. We introduce Backbone-Adaptive Evidence Routing (BAER), which adapts the evidence mechanism while preserving candidate symmetry: swapping the two responses may reverse the preference but cannot change its strength. BAER separates each expert's signed preference from candidate-invariant reliability and builds three symmetric heads: evidence stacking, reliability-based expert routing, and candidate-blind reference verification. Development data select one head for each benchmark--backbone condition, and that choice is frozen before testing. Across four benchmarks and two 8B judge backbones, BAER achieves the highest test accuracy among the compared methods in all eight conditions, with full prediction coverage and gains of 0.87--7.32 points over the strongest external baseline. The results show that adapting how evidence is gathered is more reliable than fixing one judging protocol everywhere.

Chinese Translation

成对语言模型评判者可以通过直接比较、推理或基于参考的验证来收集证据，但没有单一的协议在所有基准和评判者骨干网络中表现最佳。我们提出了基于骨干网络自适应的证据路由（Backbone-Adaptive Evidence Routing, BAER），该方法在保持候选对称性的同时调整证据机制：交换两个响应可能会反转偏好，但无法改变其强度。BAER将每个专家的签名偏好与候选不变的可靠性分开，并构建了三个对称的头部：证据堆叠、基于可靠性的专家路由和候选盲参考验证。开发数据为每个基准-骨干条件选择一个头部，并在测试前冻结该选择。在四个基准和两个8B评判者骨干网络中，BAER在所有八个条件下的测试准确率均超过了比较方法中的最高水平，具有完整的预测覆盖率，并在最强外部基准上获得了0.87至7.32点的提升。结果表明，适应证据收集方式比在所有地方固定一个评判协议更可靠。

View on arXiv Download PDF AI Translation

cs.AI / 28 / 2609.30756

Selective Amortization of Full-Budget Counterfactual Reasoning for Visual Token Communication

全预算反事实推理的选择性摊销用于视觉标记通信

Qi, Qinglei, Liang, Zhihe, Jing, Fengzhan, Zhu, Shenao, Zhang, Lei, Zhang, Chenyang, He, Shuqing, Guo, Jia

Abstract

Generative image communication transmits compact semantic tokens under a limited packet budget, where token selection directly affects the final reconstruction quality after the complete packet is decoded. However, accurately estimating the terminal value of every candidate token requires repeated receiver-side reconstruction, resulting in substantial encoder-side computation. To address this problem, we propose ACV-Gate, an adaptive candidate evaluation framework that learns to approximate full-budget counterfactual evaluation and selectively assigns exact evaluations to the most informative candidates. Specifically, a set-aware student is trained using terminal advantages and regrets to predict candidate rankings directly, while a selective refinement mechanism evaluates only a bounded candidate set containing both Local-MDL and direct actions; cost-based thresholds further enable explicit control of the average evaluation workload. Experiments on CIFAR-10 show that ACV-Gate consistently improves reconstruction quality while substantially reducing candidate evaluations; at 0.20 bpp, the primary adaptive configuration improves PSNR over LocalMDL by 0.636 dB with only 2.13 candidate evaluations per image, corresponding to 27.60% of the calls required by the Exact-Full expert. Matched-candidate comparisons, synchronized GPU measurements, and evaluations on STL-10 and 384 *384 scale transfer further demonstrate consistent quality computation trade-offs, with particularly pronounced gains at low bit rates. These results show that combining terminal-value learning with selective candidate evaluation provides an effective and controllable mechanism for allocating encoder computation in packet-constrained generative image communication.

Chinese Translation

生成图像通信在有限的数据包预算下传输紧凑的语义标记，其中标记选择直接影响完整数据包解码后的最终重建质量。然而，准确估计每个候选标记的终端价值需要重复的接收方重建，导致编码方计算量大。为了解决这个问题，我们提出了ACV-Gate，一种自适应候选评估框架，旨在学习近似全预算反事实评估，并选择性地将精确评估分配给最具信息量的候选项。具体而言，使用终端优势和遗憾训练一个集合感知的学生，以直接预测候选排名，同时选择性精炼机制仅评估包含Local-MDL和直接动作的有限候选集；基于成本的阈值进一步实现对平均评估工作量的明确控制。在CIFAR-10上的实验表明，ACV-Gate在显著减少候选评估的同时，持续提高重建质量；在0.20 bpp时，主要自适应配置使PSNR比LocalMDL提高了0.636 dB，每幅图像仅需2.13次候选评估，相当于Exact-Full专家所需调用的27.60%。匹配候选比较、同步GPU测量以及在STL-10和384 * 384尺度转移上的评估进一步展示了一致的质量计算权衡，特别是在低比特率下的增益尤为明显。这些结果表明，将终端价值学习与选择性候选评估相结合，为在数据包受限的生成图像通信中分配编码方计算提供了一种有效且可控的机制。

View on arXiv Download PDF AI Translation

cs.AI / 29 / 2609.30763

HCOE: Hyperbolic Clinical Ontology Embeddings from Biomedical Language Models

HCOE：来自生物医学语言模型的双曲线临床本体嵌入

Li, Yixuan, Li, Weihao, Song, Ziyang

Abstract

Biomedical language models (LMs) encode textual semantics but do not explicitly preserve medical code hierarchies. We present Hyperbolic Clinical Ontology Embeddings (HCOE) for hierarchy-aware clinical concept representation. HCOE maps frozen BioBERT embeddings into a Poincare ball, combining parent-side and child-side ontology-guided contrastive learning with coarse-to-fine ontology-path aggregation. It uses International Classification of Diseases (ICD) codes organized by Clinical Classifications Software (CCS) and Anatomical Therapeutic Chemical (ATC) medication hierarchies. Evaluations show that HCOE performs best on ICD/ATC clinical relation prediction and CCS-to-PheCode hierarchy transfer. On the MIMIC-IV dataset, HCOE also achieves the best performance on mortality prediction, readmission prediction, medication recommendation, and rare drug prediction.

Chinese Translation

生物医学语言模型（LMs）编码文本语义，但未明确保留医学编码层次结构。我们提出了双曲线临床本体嵌入（HCOE），用于具有层次感知的临床概念表示。HCOE 将冻结的 BioBERT 嵌入映射到庞加莱球中，结合了父侧和子侧的本体引导对比学习以及粗到细的本体路径聚合。它使用由临床分类软件（CCS）和解剖治疗化学（ATC）药物层次结构组织的国际疾病分类（ICD）代码。评估结果表明，HCOE 在 ICD/ATC 临床关系预测和 CCS 到 PheCode 层次转移方面表现最佳。在 MIMIC-IV 数据集中，HCOE 在死亡预测、再入院预测、药物推荐和稀有药物预测方面也取得了最佳性能。

View on arXiv Download PDF AI Translation

cs.AI / 30 / 2609.30765

Insurance Reserve Intelligence Platform

保险准备金智能平台

A, Anugya, Mohanty, Saket, Timmapur, Abhilash, Rai, Somya

Abstract

Insurance reserve estimation is a fundamental actuarial task supporting premium pricing, solvency assessment, financial reporting, capital planning, and risk management. Classical reserve methods based on Thiele's differential equation provide a rigorous and interpretable foundation for life insurance valuation, but repeated reserve calculations become computationally expensive in sensitivity analysis, optimization, and large-scale scenario evaluation. This paper presents an Insurance Reserve Intelligence Platform for term-life reserve modelling that combines a classical Thiele-equation solver with a Physics-Informed Neural Network (PINN) enhanced by Knowledge-Informed Neural Network (KINN) losses. The framework includes synthetic policy generation, risk-adjusted premium calculation, classical reserve trajectory generation, reserve-ratio dataset construction, configurable neural training, validation diagnostics, sensitivity and elasticity analysis, prototype optimization workflows, and interest-rate scenario testing. A key refinement is the use of premium ratio and the explicit separation of pricing-time and scenario-time interest-rate semantics. The final model uses seven features: elapsed time, issue age, pricing interest rate, scenario interest rate, premium ratio, sum assured, and mortality intensity. It predicts a standardized reserve ratio instead of raw reserve values, improving numerical stability across policies with different sums assured. The model achieved an R2 of 0.9887, MAE of 785.48, and RMSE of 1212.76 on the test set. On 200 policies, PINN/KINN inference was approximately 119.53 times faster than the classical solver. Results show strong predictive accuracy, physics consistency, and boundary performance, while highlighting remaining limitations in monotonicity and out-of-distribution generalization.

Chinese Translation

保险准备金估算是支持保费定价、偿付能力评估、财务报告、资本规划和风险管理的基本精算任务。基于Thiele微分方程的经典准备金方法为人寿保险估值提供了严格且可解释的基础，但在敏感性分析、优化和大规模情景评估中，重复的准备金计算变得计算上昂贵。本文提出了一种保险准备金智能平台，用于定期人寿准备金建模，该平台结合了经典的Thiele方程求解器与通过知识引导神经网络（Knowledge-Informed Neural Network, KINN）损失增强的物理引导神经网络（Physics-Informed Neural Network, PINN）。该框架包括合成保单生成、风险调整保费计算、经典准备金轨迹生成、准备金比率数据集构建、可配置神经网络训练、验证诊断、敏感性和弹性分析、原型优化工作流以及利率情景测试。一个关键的改进是使用保费比率，并明确区分定价时间和情景时间的利率语义。最终模型使用七个特征：经过的时间、发行年龄、定价利率、情景利率、保费比率、保险金额和死亡强度。它预测标准化的准备金比率，而不是原始准备金值，从而提高了不同保险金额保单的数值稳定性。该模型在测试集上达到了0.9887的R²、785.48的平均绝对误差（MAE）和1212.76的均方根误差（RMSE）。在200个保单上，PINN/KINN推理的速度约为经典求解器的119.53倍。结果显示出强大的预测准确性、物理一致性和边界性能，同时突出了在单调性和分布外泛化方面仍然存在的局限性。

View on arXiv Download PDF AI Translation

cs.AI / 31 / 2609.30768

Does Thinking Help Fairness? Reasoning Tokens Resolve Some Biases but Create More

思考是否有助于公平性？推理令牌解决了一些偏见但又创造了更多

Pan, Deng, Germino, Joe, Ma, Yihong, Daly, Elizabeth, Moniz, Nuno, Hua, Ting, Chawla, Nitesh

Abstract

Thinking in reasoning language models (RLMs) has been subject to debate on whether it resolves or amplifies bias. Prior works have shown competing conclusions in both directions. Using a within-model thinking-vs.-non-thinking ablation across QwQ-32B, DeepSeek-R1-Distill-Qwen-32B, and Qwen3-32B on three high-stakes decision tasks (Adult, COMPAS, Credit), we show that thinking has an asymmetric dual effect on counterfactual fairness: it both resolves counterfactual flips produced by the non-thinking baseline and creates new flips at near-saturating model confidence. In all nine (model, dataset) combinations, the created flips outnumber the resolved flips by roughly 5 times. To explain the effect, we treat the thinking trace itself as a measurable site of fairness change and study it through two dynamic instruments: 1) We propose Counterfactual Depth Probability Gap (CDPG) to track bias evolution along thinking depth, and observe that bias propagates and amplifies with thinking. 2) We also formulate the Bias Transition Matrix (BTM) to show how predictions of counterfactual pairs change from non-thinking to thinking, and find that the asymmetric dual effect originates in the pair-state joint transition.

Chinese Translation

在推理语言模型（RLMs）中，思考是否能够解决或加剧偏见一直存在争议。先前的研究在这两个方向上得出了相互竞争的结论。通过在三个高风险决策任务（成人、COMPAS、信用）上对 QwQ-32B、DeepSeek-R1-Distill-Qwen-32B 和 Qwen3-32B 进行模型内的思考与非思考消融实验，我们展示了思考对反事实公平性具有不对称的双重效应：它既解决了非思考基线产生的反事实翻转，又在接近饱和的模型置信度下创造了新的翻转。在所有九个（模型，数据集）组合中，创造的翻转数量大约是解决的翻转数量的五倍。为了解释这一效应，我们将思考轨迹本身视为公平性变化的可测量点，并通过两个动态工具进行研究：1）我们提出反事实深度概率差（CDPG）来跟踪思考深度下的偏见演变，并观察到偏见在思考过程中传播和放大。2）我们还构建了偏见转移矩阵（BTM），展示了反事实对的预测如何从非思考转变为思考，并发现这种不对称的双重效应源于对状态联合转移的影响。

View on arXiv Download PDF AI Translation

cs.AI / 32 / 2609.30796

ConsultMind:Towards Automated Diagnostic Consultation via Uncertainty-Aware Reasoning

ConsultMind：基于不确定性感知推理的自动化诊断咨询研究

Sun, Xiao, Yang, Yuming, Chen, Yun, Zhong, Jiang, Zhu, Junnan, Jiang, Xinyi, Zeng, Haoyang, Chen, Ruirui, Wang, Yining, Zhou, Xinyu, Tang, Rong, Wei, Kaiwen

Abstract

Diagnostic consultation is an online sequential decision-making process in which clinicians gather evidence through patient interaction until a diagnosis is sufficiently supported. Automating this process requires adaptive inquiry and interpretable decisions. Bayesian networks offer a natural foundation by updating diagnostic posteriors as evidence accumulates, but their use in open-ended consultation raises two challenges: linking diagnostic hypotheses to potential inquiries and translating evolving posteriors into consultation decisions. We introduce AutoDisym, an automated pipeline that integrates diagnostic knowledge with heterogeneous diagnosis-labeled clinical narratives to construct a Disorder--Symptom Bayesian Network (DSBN). Building on the DSBN, we propose ConsultMind, an uncertainty-aware framework that updates disorder posteriors after each response and uses posterior uncertainty to guide inquiry and diagnosis. We evaluate both methods across psychiatry, respiratory medicine, fever clinics, and three public datasets. The results show that AutoDisym can automatically construct high-quality DSBNs and that ConsultMind consistently improves diagnostic performance and explanation soundness. For example, AutoDisym achieves macro-averaged F1 scores of 81.37 for canonical symptoms and 72.19 for manifestations using GPT-5.6-Sol. ConsultMind improves Top-1 and Top-3 diagnostic accuracy by up to 22.15 and 37.89 percentage points, respectively. Physician evaluation further shows that ConsultMind improves the quality of ranking explanations, differential diagnoses, and diagnosis rationales across LLMs of different scales. This work offers a promising approach to automatic diagnostic consultation.

Chinese Translation

诊断咨询是一个在线的序列决策过程，临床医生通过与患者的互动收集证据，直到诊断得到充分支持。自动化这一过程需要适应性询问和可解释的决策。贝叶斯网络提供了一个自然的基础，通过随着证据的积累更新诊断后验，但在开放式咨询中的应用面临两个挑战：将诊断假设与潜在询问联系起来，以及将不断演变的后验转化为咨询决策。我们提出了AutoDisym，一个自动化管道，它将诊断知识与异构的诊断标记临床叙述相结合，以构建一种疾病-症状贝叶斯网络（Disorder-Symptom Bayesian Network, DSBN）。基于DSBN，我们提出了ConsultMind，一个不确定性感知框架，在每次响应后更新疾病后验，并利用后验不确定性来指导询问和诊断。我们在精神病学、呼吸医学、发热门诊和三个公共数据集上评估了这两种方法。结果表明，AutoDisym能够自动构建高质量的DSBN，而ConsultMind则持续提高诊断性能和解释的合理性。例如，AutoDisym在使用GPT-5.6-Sol时，经典症状的宏平均F1得分为81.37，表现为72.19。ConsultMind在Top-1和Top-3诊断准确率上分别提高了22.15和37.89个百分点。医生评估进一步显示，ConsultMind提高了不同规模大语言模型（LLMs）中排名解释、鉴别诊断和诊断理由的质量。本研究为自动化诊断咨询提供了一种有前景的方法。

View on arXiv Download PDF AI Translation

cs.AI / 33 / 2609.30797

HasMem: Hard-Origin Adaptively Softened Memory for Long-Term LLM Agents

HasMem：用于长期大语言模型代理的硬起源自适应软化记忆

He, Zihong, Shen, Junxiao, Liang, Chen, Liang, Hai-Ning

Abstract

Text-based memory and context compression support reuse of past interactions. Resizing continuous memory changes the input to a frozen LLM, coupling capacity allocation with readout. We propose Hard-Origin Adaptively Softened Memory (HasMem). Frozen hard-prompt embeddings provide a verifiable initial state. A controller adjusts memory widths, a Writer re-encodes resized entries, and Reader and Global provide readout adaptation and cross-turn state. On all $535$ questions in a reconstruction probe derived from the Multi-Session Chat (MSC) development split, the main configuration achieves lexical F1 of $95.3$ ($+4.4$ percentage points) at $93.6\%$ of the hard reference's framed memory positions. With approximately matched per-question target body budgets, six configurations at mean per-entry retention around $0.83$--$0.91$ exceed rule-based re-encoding by $8.0$--$23.6$ exact-match (EM) percentage points. With fixed model parameters and rule target width ratio $0.75$, Global's EM gain passes a user-level exact paired test with Bonferroni correction over eight comparisons. On all $500$ LongMemEval-S questions, local lexical F1 rises from the hard reference's $3.4$ to $8.9$, and answer negative log-likelihood (NLL) falls from $12.257$ to $5.274$. F1 gains accompany lower EM on both evaluations.

Chinese Translation

基于文本的记忆和上下文压缩支持过去交互的重用。调整连续记忆的大小改变了对冻结的大语言模型（LLM）的输入，将容量分配与读取结合在一起。我们提出了硬起源自适应软化记忆（HasMem）。冻结的硬提示嵌入提供了一个可验证的初始状态。控制器调整记忆宽度，写入器重新编码调整大小的条目，而读取器和全局模块提供读取适应和跨轮次状态。在从多会话聊天（Multi-Session Chat, MSC）开发拆分中派生的重建探针的所有535个问题上，主要配置在93.6%的硬参考框架记忆位置上实现了95.3的词汇F1（+4.4个百分点）。在每个问题的目标体预算大致匹配的情况下，六个配置在每条目保留率平均约为0.83至0.91的情况下，超过了基于规则的重新编码8.0至23.6个精确匹配（EM）百分点。在固定模型参数和规则目标宽度比为0.75的情况下，全局模块的EM增益通过了八个比较的Bonferroni校正的用户级精确配对测试。在所有500个LongMemEval-S问题上，局部词汇F1从硬参考的3.4上升到8.9，答案的负对数似然（NLL）从12.257降至5.274。F1的增益伴随着两个评估中EM的降低。

View on arXiv Download PDF AI Translation

cs.AI / 34 / 2609.30798

Evaluating Real-Time Voice Agents: From Component Quality to Grounded Outcomes

评估实时语音代理：从组件质量到实际结果

Negi, Shivam, Rawat, Arpit, Jain, Rashi

Abstract

Real-time voice agents have moved from research prototypes to production deployments, yet the literature describing them is fragmented across three communities that rarely cite one another: speech foundation modelling, turn-taking psycholinguistics, and agentic evaluation. Architecture papers report latency, turn-taking papers report prediction accuracy, and agentic benchmarks report task success, so no single number describes whether a deployed agent is actually good. We address that gap with three evidence-based claims, each traceable to a corpus of 38 primary sources organised into an application-centric taxonomy of six categories. First, architecture choice is a deployment constraint rather than a settled verdict: a 2026 enterprise tutorial reports that no fully self-hostable end-to-end system yet meets production constraints, while a chunked cascade independently reaches state-of-the-art duplex behaviour, showing duplex behaviour is separable from duplex architecture. Second, evaluation has shifted decisively from component quality toward grounded outcomes, with recent benchmarks verifying backend state rather than trusting what the agent claims to have done. Third, the dyadic assumption in most models and benchmarks is breaking down: multiparty turn-taking and multi-speaker reasoning benchmarks show that deciding when not to speak, and reasoning about who may be told what, are first-class capabilities two-participant framings cannot measure. For each source we state the problem it targets, its mechanism, and its reported evidence, alongside the search strategy, inclusion criteria, and a verification step that caught a misattributed arXiv identifier in circulation. We propose TRG (Timing-Recovery-Grounded), a reporting standard characterising an agent by timing, post-disruption recovery, and state-verified outcome together, with a conditional fourth axis for multiparty deployments.

Chinese Translation

实时语音代理已从研究原型转变为生产部署，然而，描述它们的文献在三个几乎不互相引用的社区中是支离破碎的：语音基础建模、轮流交谈心理语言学和代理评估。架构论文报告延迟，轮流交谈论文报告预测准确性，而代理基准报告任务成功，因此没有单一的数字能够描述一个部署的代理是否真正优秀。我们通过三个基于证据的主张来填补这一空白，每个主张都可以追溯到一个由38个主要来源组成的语料库，这些来源被组织成一个以应用为中心的六类分类法。首先，架构选择是一种部署约束，而不是一个确定的裁决：一份2026年的企业教程报告称，尚无完全自托管的端到端系统满足生产约束，而一个分块级联系统独立达到了最先进的双工行为，表明双工行为与双工架构是可分离的。其次，评估已明确从组件质量转向实际结果，最近的基准验证后端状态，而不是信任代理声称已完成的任务。第三，大多数模型和基准中的二人假设正在崩溃：多方轮流交谈和多说话者推理基准表明，决定何时不发言以及推理谁可以被告知什么是两参与者框架无法衡量的一流能力。对于每个来源，我们陈述其针对的问题、机制和报告的证据，以及搜索策略、纳入标准和一个验证步骤，该步骤捕捉到了一个在流通中的错误归属的arXiv标识符。我们提出了TRG（Timing-Recovery-Grounded），一种报告标准，通过时间、干扰后的恢复和状态验证的结果共同表征代理，并为多方部署提供了一个条件性的第四轴。

View on arXiv Download PDF AI Translation

cs.AI / 35 / 2609.30813

A Benchmark and Diagnostic Study of Epistemic Admission in Shared Agent Memory

共享代理记忆中认知性接纳的基准与诊断研究

Li, Xiaoyang, Wang, Yiqi, Zhu, Chencheng, XU, KE, Yang, Wencheng, Sun, Zequn, Song, Pingan, Duan, Yiqun, Cai, Taotao

Abstract

Evaluating claim admission in shared agent memory is challenging because repeated claims may be mistaken for independent evidence. An agent may copy or paraphrase a retrieved belief, while admitting a false claim exposes subsequent agents to it. To study this problem, we introduce the Correlated Promotion Benchmark (CPB), which evaluates whether candidate claims should be admitted to shared memory.CPB-Static constructs a frozen test split from publicly annotated sources with fixed gold actions. CPB-Live runs multi-agent teams over a shared store, records all writes and retrievals, and tracks source lineage defined by each scenario. A separate consumer answers from the store alone. We evaluate eight admission policies across four agent families. Our results show that policies which deduplicate sources reject many true claims alongside false ones, whereas policies preserving answer coverage admit nearly as many false claims as unrestricted sharing. Gating on declared source type reduces false adoption to 0.06--0.09, compared with 0.22--0.47 for other answering policies. Once an uncontested false belief enters memory, the consumer asserts it in 0.97--0.99 of probes across all families. No non-oracle policy consistently rejects false claims across verbatim copies, paraphrases, and paraphrases declared authoritative. These findings reveal the limitations of admission policies without access to source lineage.

Chinese Translation

在共享代理记忆中评估声明接纳的难度在于，重复的声明可能被误认为是独立的证据。代理可能会复制或改述一个检索到的信念，而接纳一个虚假的声明则会使后续代理暴露于该声明之下。为研究这一问题，我们引入了相关推广基准（Correlated Promotion Benchmark, CPB），该基准评估候选声明是否应被接纳到共享记忆中。CPB-Static 从公开注释的来源构建了一个固定黄金操作的冻结测试集。CPB-Live 在共享存储上运行多代理团队，记录所有写入和检索，并跟踪每个场景定义的来源谱系。一个独立的消费者仅从存储中回答。我们评估了四个代理家族中的八种接纳策略。我们的结果表明，去重来源的策略拒绝了许多真实声明和虚假声明，而保持答案覆盖的策略接纳的虚假声明几乎与不受限制的共享相当。根据声明的来源类型进行筛选将虚假接纳率降低到 0.06--0.09，而其他回答策略的虚假接纳率为 0.22--0.47。一旦一个无人争议的虚假信念进入记忆，消费者在所有家族的探测中以 0.97--0.99 的比例确认该信念。没有任何非神谕策略能够在逐字复制、改述和被声明为权威的改述中始终拒绝虚假声明。这些发现揭示了在没有来源谱系访问权限的情况下，接纳策略的局限性。

View on arXiv Download PDF AI Translation

cs.AI / 36 / 2609.30836

PTC-Decoder: Towards Intelligent SLMs on Offline Resource-Constrained Edge Devices

PTC-解码器：面向离线资源受限边缘设备的智能小型语言模型

Yu, Minghui, Mu, Ke, Wu, Gang

Abstract

Deploying small language models (SLMs) on offline, resource-constrained edge devices such as remote sensing satellites presents a fundamental challenge: their limited reasoning capacity hinders reliable execution of multi-step agent tasks requiring complex tool orchestration. Existing plan-solve paradigms rely on prompt-based enforcement, which our experiments show SLMs almost entirely disregard: weak models fail to invoke the plan. We propose PTC-Decoder (Plan-Tool Constrained Decoder), a training-free, plug-and-play decoder framework that combines (1) a Plan-to-Act paradigm, which elevates planning to an atomic tool and forces its invocation at the first inference step, and (2) TC-Decoder, a deterministic finite automaton that imposes token-level hard constraints on tool names while preserving freedom over parameter generation, thereby retaining SLM reasoning capability. Evaluated on 200 real remote-sensing satellite tasks across 7 SLMs, PTC-Decoder yields a statistically significant mean overall score gain of +1.21 (p<0.01), 95% CI [+1.13, +1.29]), with consistent improvements across models and other datasets. An ablation study that removes TC-Decoder causes substantial performance degradation across all quality metrics without reducing computational cost, confirming TC-Decoder as the primary driver. PTC-Decoder thus offers a lightweight yet effective solution for improving step-level reliability, with final-answer accuracy remaining an open challenge. In essence, we enforce plan adherence by constraining the permissible output vocabulary during inference, without requiring retraining.

Chinese Translation

在离线资源受限的边缘设备上（如遥感卫星）部署小型语言模型（SLMs）面临着一个根本性的挑战：其有限的推理能力阻碍了对需要复杂工具协调的多步骤代理任务的可靠执行。现有的计划-解决范式依赖于基于提示的强制执行，而我们的实验表明，SLMs几乎完全忽视这一点：弱模型未能调用计划。我们提出了PTC-解码器（Plan-Tool Constrained Decoder），这是一种无训练、即插即用的解码框架，结合了（1）计划-行动范式，将规划提升为原子工具，并在第一次推理步骤中强制调用，以及（2）TC-解码器（Deterministic Finite Automaton），它对工具名称施加了基于令牌的硬约束，同时保留参数生成的自由，从而保持SLM的推理能力。在对7个SLM的200个真实遥感卫星任务进行评估时，PTC-解码器的平均总体得分显著提高了+1.21（p<0.01），95%置信区间为[+1.13, +1.29]，在各模型和其他数据集上均表现出一致的改进。去除TC-解码器的消融研究导致所有质量指标的性能显著下降，而计算成本未降低，确认了TC-解码器是主要驱动因素。因此，PTC-解码器提供了一种轻量且有效的解决方案，以提高步骤级的可靠性，而最终答案的准确性仍然是一个开放的挑战。本质上，我们通过在推理过程中限制可接受的输出词汇来强制执行计划遵循，而无需重新训练。

View on arXiv Download PDF AI Translation

cs.AI / 37 / 2609.30841

Why Jailbreaks Succeed in Diffusion Language Models: An Energy Landscape Analysis

为何越狱攻击在扩散语言模型中成功：能量景观分析

Bach, Thong, Nguyen, Dung, Le, Thao Minh, Tran, Truyen

Abstract

Existing attacks and defenses for diffusion-based large language models (dLLMs) target specific vulnerabilities but lack a shared framework explaining why attacks succeed. We propose one by interpreting safety alignment as shaping the denoising energy landscape: a well-aligned model routes harmful queries toward safe outputs through an energy barrier that separates the two regions. Current jailbreak attacks reduce to two strategies for circumventing this barrier: obscuring the query's safety disposition at initialisation, or intervening mid-trajectory to force the denoising path across the energy barrier. From this perspective and the result that masked diffusion models minimise kinetic energy during denoising, we derive three complementary, training-free detection signals: a step-0 ratio that reads the initial safety disposition from the logit distribution before generation begins, and two trajectory-velocity signals that track kinetic energy in complementary subspaces of the logit space. An attack must either reveal its intent at initialisation or expend kinetic energy to cross the barrier in at least one monitored subspace, so the three signals cover each other's blind spots in the energy budget by construction. Evaluation across three dense dLLMs (LLaDA-8B, LLaDA-1.5, Dream-7B) and a sparse mixture-of-experts dLLM (LLaDA-MoE-7B) confirms this complementarity. In stress tests of known attacks, every configuration that evades detection also fails to produce harmful content, suggesting that the detection and barrier-crossing thresholds are hard to separate.

Chinese Translation

现有针对基于扩散的大型语言模型（dLLMs）的攻击和防御针对特定的脆弱性，但缺乏一个共同的框架来解释攻击为何成功。我们通过将安全对齐解释为塑造去噪能量景观来提出一个框架：一个良好对齐的模型通过一个能量障碍将有害查询引导到安全输出，从而将两者区域分开。目前的越狱攻击归结为两种策略以规避这一障碍：在初始化时模糊查询的安全倾向，或在轨迹中途干预以强制去噪路径跨越能量障碍。从这个角度以及掩蔽扩散模型在去噪过程中最小化动能的结果出发，我们推导出三种互补的、无训练的检测信号：一个步-0 比率，它在生成开始之前从对数分布中读取初始安全倾向，以及两个轨迹速度信号，它们跟踪对数空间互补子空间中的动能。攻击必须在初始化时揭示其意图，或者在至少一个监测子空间中消耗动能以跨越障碍，因此这三种信号在能量预算中通过构造覆盖了彼此的盲点。在三个稠密的 dLLMs（LLaDA-8B, LLaDA-1.5, Dream-7B）和一个稀疏的专家混合 dLLM（LLaDA-MoE-7B）上的评估确认了这种互补性。在已知攻击的压力测试中，所有逃避检测的配置也未能产生有害内容，这表明检测阈值和跨越障碍的阈值难以分离。

View on arXiv Download PDF AI Translation

cs.AI / 38 / 2609.30861

SkillEvoReg: Regularizing Agent Skill Evolution Against Overfitting

SkillEvoReg：针对过拟合的智能体技能演化正则化

Nie, Guanyu, Zhu, Fangzhou, Kai, Shixiong, Han, Xiongwei, Zhong, Tao, Yuan, Mingxuan

Abstract

Language-model agents increasingly improve by converting execution experience into reusable external skills. Yet repeated skill updates form a learning process of their own: locally useful edits can accumulate into redundant or task-specific instructions, while new updates can disrupt behavior that previously worked. We study this problem as skill-evolution overfitting and introduce SkillEvoReg, a general regularization framework for skill evolution inspired by anti-overfitting techniques in neural-network training. SkillEvoReg combines training-time skill dropout, which perturbs update generation, and complexity-aware local regularization, which controls unnecessary structural growth, with causal counterexample validation (CCV), which provides targeted behavioral validation of candidate-specific regressions. We instantiate the framework across heterogeneous skill-evolution systems while retaining each system's native skill evolver and task evaluator. Across SkillOpt, SkillEvolBench, and ContinualSkillBench, SkillEvoReg consistently controls skill-state growth while preserving competitive downstream capability, improves several transfer and later-stage evolution outcomes, and identifies update-level regressions that structural metrics alone cannot reveal. These results suggest that explicit regularization is a useful complement to increasingly capable skill updaters.

Chinese Translation

语言模型智能体通过将执行经验转化为可重用的外部技能而不断提升。然而，重复的技能更新形成了一个独立的学习过程：局部有用的编辑可能会积累成冗余或特定任务的指令，而新的更新可能会干扰之前有效的行为。我们将这个问题视为技能演化过拟合，并引入了SkillEvoReg，这是一个受神经网络训练中反过拟合技术启发的技能演化通用正则化框架。SkillEvoReg结合了训练时技能丢弃（skill dropout），该方法扰动更新生成，以及复杂度感知的局部正则化，控制不必要的结构增长，同时结合因果反例验证（Causal Counterexample Validation, CCV），为候选特定回归提供有针对性的行为验证。我们在异构技能演化系统中实例化该框架，同时保留每个系统的原生技能演化器和任务评估器。在SkillOpt、SkillEvolBench和ContinualSkillBench中，SkillEvoReg始终控制技能状态的增长，同时保持竞争性的下游能力，改善多个迁移和后期演化结果，并识别仅靠结构指标无法揭示的更新级回归。这些结果表明，显式正则化是对日益强大的技能更新器的有益补充。

View on arXiv Download PDF AI Translation

cs.AI / 39 / 2609.30878

TISD: On-Policy Self-Distillation with Trajectory Intervention

TISD：具有轨迹干预的在线自蒸馏

Lee, Taeckyung, Amankos, Rinat, Kim, Jeonghye, Yoon, Hyungjun, Jin, Woogyeol, Lee, Sung-Ju

Abstract

On-policy self-distillation (OPSD) provides dense teacher targets, but evaluates them only along student-sampled rollouts. When the privileged teacher favors an alternative action at a visited prefix, OPSD can provide a target for the branch decision but cannot supervise the successor contexts induced by that action unless the student samples it. This creates a training-time data-collection bottleneck and suggests a different role for teacher-student disagreement: proposing a trajectory branch rather than identifying a sufficient local repair. Our diagnostic framework using controlled token interventions reveals that a teacher-preferred token at peak disagreement can improve student continuation success, while its local corrective value is limited. Motivated by this finding, we introduce a simple branch-regenerate-distill algorithm, Trajectory-Intervention Self-Distillation (TISD). TISD forces a teacher-selected branch action, returns suffix generation to the student, and distills the full trajectory under the privileged-context-conditioned teacher. Across the coding models, TISD improves average Avg@4 over SDPO by 1.2 percentage points. Across the science domains, it improves average Avg@128 by 0.8 points under an equal-step budget and by 0.3 points under an equal-time budget. These results support teacher-guided branching as a way to expose useful successor contexts for self-distillation.

Chinese Translation

在线自蒸馏（OPSD）提供了密集的教师目标，但仅在学生采样的回合中进行评估。当特权教师在已访问的前缀中偏好替代动作时，OPSD可以为分支决策提供目标，但无法监督由该动作引发的后续上下文，除非学生对其进行采样。这造成了训练时数据收集的瓶颈，并暗示教师-学生不一致的不同角色：提出一个轨迹分支，而不是识别一个充分的局部修复。我们使用受控标记干预的诊断框架揭示，在峰值不一致时，教师偏好的标记可以提高学生的延续成功率，而其局部修正价值有限。基于这一发现，我们引入了一种简单的分支再生蒸馏算法——轨迹干预自蒸馏（TISD）。TISD强制选择教师选定的分支动作，将后缀生成返回给学生，并在特权上下文条件下对完整轨迹进行蒸馏。在编码模型中，TISD使Avg@4的平均值比SDPO提高了1.2个百分点。在科学领域中，在相等步长预算下，TISD使Avg@128的平均值提高了0.8点，在相等时间预算下提高了0.3点。这些结果支持教师引导的分支作为揭示自蒸馏有用后续上下文的一种方式。

View on arXiv Download PDF AI Translation

cs.AI / 40 / 2609.30880

EXAONE Demand 1.0: A Time Series Foundation Model for Demand Forecasting

EXAONE Demand 1.0：用于需求预测的时间序列基础模型

Lee, Seunghan, Han, Sangjun, Seo, Jun, Kang, Junhyeok, Lee, Jaehoon, Lim, Tae Yoon, Kang, Dongwan, Choi, Hwanil, Kim, Minjae, Yoo, Sungdong, Lee, Soonyoung, Ahn, Wonbin

Abstract

Time series foundation models (TSFMs) are pretrained on series from diverse domains, where demand series make up only a small fraction. Demand data has properties that such corpora rarely contain: Short histories, frequent zeros, censoring by stock-outs, and exogenous events that the series does not record. To this end, we propose EXAONE Demand, built on 1) a demand-specific corpus and 2) a demand-aware adapter. For the corpus, we assemble 11.3M series and 48.4B observations from 73 sources, and a synthetic generator supplies the behaviour that open demand data under-represents. For the adapter, we attach low-rank branches to a frozen general-domain backbone, one for each of the four demand classes (smooth, intermittent, erratic, and lumpy), and a router that reads eight scale-free statistics of the input series decides how much each branch contributes. We build EXAONE Demand in two versions, one trained on real-world and synthetic demand together and one trained on the synthetic corpus alone. On 22 held-out datasets, both versions outperform 36 TSFMs, and real-world demand adds a gain over synthetic data alone.

Chinese Translation

时间序列基础模型（TSFMs）是在来自不同领域的序列上进行预训练的，其中需求序列仅占一小部分。需求数据具有这些语料库很少包含的特性：短历史、频繁的零值、因缺货而导致的审查，以及系列未记录的外生事件。为此，我们提出了EXAONE Demand，基于1）特定于需求的语料库和2）需求感知适配器。对于语料库，我们从73个来源汇集了1130万条序列和484亿个观察值，并且一个合成生成器提供了开放需求数据未能充分代表的行为。对于适配器，我们将低秩分支附加到一个冻结的通用领域主干网络上，每个需求类别（平稳、间歇性、异常和块状）对应一个分支，一个路由器读取输入序列的八个无尺度统计量，以决定每个分支的贡献程度。我们构建了EXAONE Demand的两个版本，一个是同时在真实和合成需求上训练的，另一个则仅在合成语料库上训练。在22个保留数据集上，这两个版本均优于36个TSFMs，并且真实世界的需求相较于单独的合成数据有显著提升。

View on arXiv Download PDF AI Translation

cs.AI / 41 / 2609.30887

From Tapping to Hopping: Augmenting Mobile GUI Agents with App-Native Deeplinks

从点击到跳转：增强移动图形用户界面的应用原生深度链接

Sun, Yuchen, Cai, Chenglin, Zhang, Gongjie, Xia, Tianyu, Kong, Quyu, Tong, Panrong, Zeng, Zhengwen, Chen, Long, Hoi, Steven, Zhang, Chongyang, Wang, Yue

Abstract

Mobile GUI agents complete tasks using GUI actions like taps and swipes. These actions are broadly applicable across applications, but reaching a navigation interface. A single deeplink call can replace a sequence of screen-by-screen GUI actions. We therefore introduce hybrid interaction, using deeplinks for direct navigation and GUI actions for other on-screen operations and fallback. To enable this, we discover candidate deeplinks through static analysis, validate them on real devices, and describe their observed landing screens. This process creates a verified and grounded deeplink catalog that pairs each working deeplink with a description of its landing screen. Using this catalog, we introduce GUI-Hopper, a improves task success in commercial applications on real devices, further demonstrating the benefits of hybrid interaction.

Chinese Translation

移动图形用户界面代理通过点击和滑动等图形用户界面操作完成任务。这些操作在不同应用中广泛适用，但到达导航界面可能较为复杂。一个深度链接调用可以替代一系列逐屏的图形用户界面操作。因此，我们引入了混合交互，利用深度链接进行直接导航，同时使用图形用户界面操作进行其他屏幕上的操作和后备处理。为了实现这一点，我们通过静态分析发现候选深度链接，在真实设备上验证它们，并描述其观察到的着陆屏幕。这个过程创建了一个经过验证的深度链接目录，将每个有效的深度链接与其着陆屏幕的描述相匹配。利用这个目录，我们推出了GUI-Hopper，它在真实设备上的商业应用中提高了任务成功率，进一步展示了混合交互的优势。

View on arXiv Download PDF AI Translation

cs.AI / 42 / 2609.30894

Training Graph Foundation Models on The Web Graph

在网络图上训练图基础模型

Sato, Ryoma

Abstract

We introduce Acacia, a graph foundation model, trained on the web graph. Acacia (i) supports arbitrary feature dimensionalities and semantics without additional training, (ii) supports a wide range of tasks, including node classification, link prediction, node clustering, and graph generation, without additional training, (iii) has in-context learning capabilities, and (iv) does not rely on pretrained LLMs. In particular, existing graph foundation models often require training additional classification heads or feature projectors to accommodate new graphs or new labels, whereas Acacia does not. Moreover, existing graph foundation models often gain their capabilities by being stitched together with pretrained LLMs, whereas Acacia is trained from scratch using only the Common Crawl web graph. This is also an important result because it provides evidence that graph models can acquire emergent capabilities from scratch like LLMs.

Chinese Translation

我们介绍了 Acacia，一种在网络图上训练的图基础模型。Acacia (i) 支持任意特征维度和语义而无需额外训练，(ii) 支持广泛的任务，包括节点分类、链接预测、节点聚类和图生成，而无需额外训练，(iii) 具备上下文学习能力，(iv) 不依赖于预训练的语言模型（LLMs）。特别是，现有的图基础模型通常需要训练额外的分类头或特征投影器以适应新的图或新标签，而 Acacia 则不需要。此外，现有的图基础模型通常通过与预训练的 LLMs 结合来获得其能力，而 Acacia 则是从头开始训练，仅使用 Common Crawl 网络图。这也是一个重要的结果，因为它提供了证据表明图模型可以像 LLMs 一样从零开始获得突现能力。

View on arXiv Download PDF AI Translation

cs.AI / 43 / 2609.30922

JevSoup: System-One Routing for Training-Free LoRA Composition

JevSoup：无训练的低秩适应（LoRA）组合的系统一路由

Wang, Xiuying, Cheng, Jiahua, Li, Shuotian, Cheng, Yufan, Deng, Bowen, Bai, Zhexuan, Li, Yichen

Abstract

Building adaptable AI systems requires effective coordination of specialized capabilities across diverse tasks. Low-rank adaptation (LoRA) enables modular expertise, but existing routing approaches may require auxiliary data, additional training, or autoregressive decoding. We propose JevSoup, a training-free framework separating System One expert routing from System Two execution. Using only the input and expert descriptions, Jev selects two experts through structured probabilities. JevSoup retains the leading expert's update, projects the second onto the orthogonal complement of the first update's row space, and combines them with equal weights. Across 14 PorTAL tasks and three Qwen3 scales, JepSoup achieves absolute gains of up to 1.19\% in task-macro and 1.21\% in sample-micro accuracy over the strongest evaluated external baselines. Our code is available at https://github.com/Leowang980/JevSoup.

Chinese Translation

构建可适应的人工智能系统需要在多样化任务中有效协调专业能力。低秩适应（LoRA）使得模块化专业知识成为可能，但现有的路由方法可能需要辅助数据、额外训练或自回归解码。我们提出了JevSoup，一个无训练的框架，将系统一的专家路由与系统二的执行分开。仅使用输入和专家描述，Jev通过结构化概率选择两个专家。JevSoup保留领先专家的更新，将第二个专家投影到第一个更新的行空间的正交补上，并以相等的权重将它们结合。在14个PorTAL任务和三个Qwen3规模的实验中，JevSoup在任务宏观和样本微观准确性上相较于最强的外部基线实现了最高1.19\%和1.21\%的绝对增益。我们的代码可在 https://github.com/Leowang980/JevSoup 获取。

View on arXiv Download PDF AI Translation

cs.AI / 44 / 2609.30936

Self-Play Search Distillation for Large Language Model Reasoning

大型语言模型推理的自我博弈搜索蒸馏

Molfetta, Lorenzo, Kwan, Wai-Chung, Frisoni, Giacomo, Ragazzi, Luca, Moro, Gianluca, Vougiouklis, Pavlos, Pan, Jeff Z., Minervini, Pasquale

Abstract

Improving reasoning abilities in Large Language Models (LLMs) requires high-quality data that exposes difficult decisions, competing alternatives, and their consequences. Data scarcity is driven by the low quality of synthetic data and the cost of human labeling. We introduce Self-Play Search Distillation (SPSD), a framework for generating superhuman synthetic data via self-play of MuZero-like networks trained on board games. SPSD uses executable environments to turn search into structured reasoning problems. At each state, the expert identifies a preferred decision, plausible alternatives, plausible opponent replies, and value estimates. By converting the self-play search records into superhuman chains-of-thought, we train LLMs with environment-grounded supervision. Although trained only on self-play search records, SPSD transfers to unseen mathematics. On Qwen3-4B-Base, it raises the mean over six mathematics benchmarks from 24.1 to 36.6 while increasing the held-out-game win rate from 15% to 45%. SPSD offers an annotation-efficient way to create high-quality synthetic data for improving LLM performance in reasoning tasks.

Chinese Translation

提高大型语言模型（LLMs）的推理能力需要高质量的数据，这些数据能够揭示困难决策、竞争替代方案及其后果。数据稀缺的原因在于合成数据的低质量和人工标注的高成本。我们提出了自我博弈搜索蒸馏（Self-Play Search Distillation, SPSD），这是一个通过自我博弈的MuZero类网络在棋类游戏上生成超人类合成数据的框架。SPSD利用可执行环境将搜索转化为结构化推理问题。在每个状态下，专家识别出首选决策、合理的替代方案、合理的对手回复和价值估计。通过将自我博弈搜索记录转化为超人类的思维链，我们以环境为基础的监督训练LLMs。尽管仅在自我博弈搜索记录上进行训练，SPSD仍能迁移到未见的数学问题。在Qwen3-4B-Base上，它将六个数学基准的平均分数从24.1提高到36.6，同时将保留游戏的胜率从15%提高到45%。SPSD提供了一种高效的注释方式，以创建高质量的合成数据，从而提升LLM在推理任务中的表现。

View on arXiv Download PDF AI Translation

cs.AI / 45 / 2609.30939

MACBT: A Multi-Agent Cognitive Behavioral Therapy Decision Support System with Longitudinal Memory

MACBT：一种具有纵向记忆的多智能体认知行为疗法决策支持系统

Jiang, De, Zhang, Shuo, Liao, Weiwei, Zhang, Jianying, Yu, Chuanhui, Liao, Hongen, Yuan, Kehong

Abstract

Cognitive behavioral therapy (CBT) is an evidence-based first-line treatment for depression, yet its scale is constrained by the time clinicians spend on pre-session preparation, post-session documentation, and longitudinal cognitive-pathology tracking. We present a clinician-facing AI decision-support system that combines a multi-agent CBT framework (MACBT) with a CBT-specific longitudinal memory module (CD Memory). MACBT encodes the five-stage CBT workflow (assessment, Socratic questioning, cognitive restructuring, behavioral experiments, and treatment monitoring) into five collaborative agents. CD Memory tracks cognitive-distortion type, frequency, severity, and restructuring efficacy across sessions to generate pre-session pathology reports and intervention-priority recommendations. We construct a Chinese CBT dialogue corpus via dual-role large language model simulation and train a Qwen3-14B backbone with supervised fine-tuning and direct preference optimization. Evaluation with GPT-4 judges shows MACBT outperforms MeChat, SoulChat, PsyChat, and CPsyCounX in professionalism (2.62) and clinical authenticity (2.25). The full memory-augmented system further improves session quality by 12.6% and achieves a longitudinal mean of 2.29 on cross-session continuity, intervention progression, and personalization.

Chinese Translation

认知行为疗法（CBT）是一种基于证据的抑郁症一线治疗方法，但其规模受到临床医生在会前准备、会后文档记录和纵向认知病理跟踪上花费时间的限制。我们提出了一种面向临床医生的人工智能决策支持系统，该系统结合了多智能体CBT框架（MACBT）和特定于CBT的纵向记忆模块（CD Memory）。MACBT将五阶段CBT工作流程（评估、苏格拉底提问、认知重构、行为实验和治疗监测）编码为五个协作代理。CD Memory跟踪会话中的认知扭曲类型、频率、严重程度和重构效果，以生成会前病理报告和干预优先级建议。我们通过双角色大型语言模型模拟构建了一个中文CBT对话语料库，并使用监督微调和直接偏好优化训练了Qwen3-14B骨干模型。与GPT-4评审的评估结果显示，MACBT在专业性（2.62）和临床真实性（2.25）方面优于MeChat、SoulChat、PsyChat和CPsyCounX。完整的记忆增强系统进一步提高了会话质量12.6%，并在跨会话连续性、干预进展和个性化方面达到了2.29的纵向均值。

View on arXiv Download PDF AI Translation

cs.AI / 46 / 2609.30940

Financial Fragility in Societies of LLM Agents: Coordination Failures and Stabilizing Mechanisms

大语言模型代理社会中的金融脆弱性：协调失败与稳定机制

Fu, Zhenhao, Xu, Ruipeng, Ren, Qibing

Abstract

Individually protective decisions can produce avoidable collective failures. As large language model (LLM) agents take on greater roles in financial decision-making, financial AI safety must therefore be considered not only at the level of individual agents, but also at the level of the systems they jointly create. We study this problem with FRAIL, a controlled experimental framework that places LLM agents in three dynamic financial environments---bank runs, debt rollover, and reward crowdfunding---where agents' decisions reshape the financial conditions faced by others. Across seven leading LLMs, we find widespread collective fragility even when no agent is instructed to destabilize the system: 77\% of baseline bank-run episodes and 83\% of debt-rollover episodes end in failure. We then compare three interaction mechanisms based on compensated commitments, centralized commitment agreements, and participant-led coalitions. All three improve aggregate outcomes, but no single mechanism performs best across all financial structures. Across mechanisms, successful stabilization shares a common temporal pattern: broad commitment forms early, before defensive behavior becomes self-reinforcing. Our findings show that individually capable agents do not automatically form safe financial systems, highlighting system-level evaluation and interaction design as central problems for financial AI safety. Code is available at https://anonymous.4open.science/r/FinFrail-CF26.

Chinese Translation

个体保护性决策可能导致可避免的集体失败。随着大语言模型（LLM）代理在金融决策中扮演越来越重要的角色，金融人工智能安全不仅需要在个体代理层面进行考虑，还必须在它们共同创造的系统层面进行研究。我们通过FRAIL这一受控实验框架来研究这一问题，该框架将LLM代理置于三种动态金融环境中——银行挤兑、债务展期和奖励众筹——在这些环境中，代理的决策会重塑其他代理所面临的金融条件。在七种领先的LLM中，我们发现即使没有代理被指示去破坏系统，集体脆弱性仍然普遍存在：77%的基线银行挤兑事件和83%的债务展期事件以失败告终。随后，我们比较了基于补偿承诺、集中承诺协议和参与者主导的联盟的三种互动机制。所有三种机制都改善了整体结果，但没有单一机制在所有金融结构中表现最佳。在这些机制中，成功的稳定化共享一个共同的时间模式：广泛的承诺在早期形成，在防御性行为变得自我强化之前。我们的研究结果表明，个体能力强的代理并不自动形成安全的金融系统，强调了系统层面的评估和互动设计作为金融人工智能安全的核心问题。代码可在 https://anonymous.4open.science/r/FinFrail-CF26 获取。

View on arXiv Download PDF AI Translation

cs.AI / 47 / 2609.30943

LogicTree-RAG: Logic Tree-guided Retrieval-Augmented Generation for Long-form Patent Drafting

LogicTree-RAG：基于逻辑树引导的检索增强生成用于长篇专利撰写

Zhu, Jiaqi, Xing, Naili, Pan, Hexiang, Gao, Haotian, Yin, Jianwei, Xiao, Xiaokui, Ooi, Beng Chin

Abstract

Long-form technical text generation underpins knowledge-intensive workflows, yet remains challenging for large language models (LLMs) due to the need for globally consistent logical structuring and faithful technical reasoning beyond local coherence. Patent drafting is a canonical instance of this challenge, demanding holistic generation of a legally compliant and technically exhaustive document through sustained multi-expert collaboration. Existing approaches often focus on partial section generation or rely on manually crafted outlines, limiting scalable automation in realistic settings. In this work, we propose LogicTree-RAG, a logic tree-guided retrieval-augmented generation framework that induces a hierarchical logic tree as a global organizational backbone to organize and ground technical disclosures, without relying on expert-defined drafting priors. Each node in the logic tree represents a technical element and is constructed through evidence-guided recursive generation. A hybrid traversal mechanism then maps the logic tree into patent sections, enabling controllable and section-balanced generation. Extensive experiments show that LogicTree-RAG consistently improves content quality and language conformity over strong LLM-based baselines and achieves longer structured generation with high token efficiency, demonstrating the effectiveness of logic-centric generation for complex technical document drafting.

Chinese Translation

长篇技术文本生成是知识密集型工作流程的基础，但由于需要全球一致的逻辑结构和超越局部连贯性的忠实技术推理，这对大型语言模型（LLMs）仍然具有挑战性。专利撰写是这一挑战的典型实例，要求通过持续的多专家协作生成一个合法合规且技术详尽的文档。现有的方法通常侧重于部分章节的生成或依赖手动制作的大纲，这限制了在实际环境中的可扩展自动化。在本研究中，我们提出了LogicTree-RAG，一种基于逻辑树引导的检索增强生成框架，该框架引入了一个层次化的逻辑树作为全球组织骨架，以组织和支撑技术披露，而不依赖于专家定义的撰写先验。逻辑树中的每个节点代表一个技术元素，并通过证据引导的递归生成构建而成。然后，混合遍历机制将逻辑树映射到专利章节，实现可控且章节平衡的生成。大量实验表明，LogicTree-RAG在内容质量和语言一致性方面始终优于强大的基于LLM的基线，并实现了更长的结构化生成，具有高令牌效率，证明了以逻辑为中心的生成在复杂技术文档撰写中的有效性。

View on arXiv Download PDF AI Translation

cs.AI / 48 / 2609.30954

FTB Graph: Determining and Validating First-token Broadcasters and Language-Identity Head Circuits in Multilingual Language Models

FTB 图：确定和验证多语言模型中的首个令牌广播者和语言身份头电路

Pillai, Arjun, Hoang, Christian, Laroza, Anjelo

Abstract

Large language models operating in multilingual contexts must resolve target response languages early in generation, yet the causal circuitry governing first-token language identity decisions remains poorly mapped. We present an end-to-end structural circuit analysis across six model architectures spanning four families: GPT-2, BLOOM-560M, Pythia-1B/2.8B, and Qwen2.5-1.5B Base/Instruct. Using Edge Attribution Patching (EAP) with FP16 active clamping, followed by exact activation patching verification with a 2,000-candidate-edge search ceiling, we extract directed acyclic graphs driving first-token language broadcasting. Across the standalone models, we observe deep or mid-to-deep broadcasting hubs, though the evidence is strongest for Pythia-2.8B and BLOOM-560M because GPT-2 and Pythia-1B leave few out-of-graph heads for comparison, while both Qwen2.5-1.5B variants invert the necessity check. Scaling from Pythia-1B to 2.8B expands node participation while maintaining a similar verified edge budget, producing sparser topology. The Qwen2.5-1.5B base and instruct circuits retain 84.7% Jaccard similarity, including the Layer 27 hub, indicating that first-token routing is largely established during pretraining and preserved by instruction tuning. Finally, EAP scores correlate weakly with exact patching deltas across most models, showing that linear gradient approximations can diverge from causal interventions in FP16 and motivating exact-patching verification for reliable circuit discovery.

Chinese Translation

在多语言环境中运行的大型语言模型必须在生成早期解决目标响应语言，但支配首个令牌语言身份决策的因果电路仍未得到充分映射。我们对六种模型架构进行了端到端的结构电路分析，这些架构跨越四个家族：GPT-2、BLOOM-560M、Pythia-1B/2.8B 和 Qwen2.5-1.5B Base/Instruct。使用边缘归因修补（Edge Attribution Patching, EAP）结合 FP16 活动钳制，随后进行 2,000 个候选边缘搜索上限的精确激活修补验证，我们提取了驱动首个令牌语言广播的有向无环图。在独立模型中，我们观察到深层或中到深层的广播中心，尽管 Pythia-2.8B 和 BLOOM-560M 的证据最为强烈，因为 GPT-2 和 Pythia-1B 留下的图外头部较少以供比较，而 Qwen2.5-1.5B 的两个变体则反转了必要性检查。从 Pythia-1B 扩展到 2.8B 增加了节点参与，同时保持类似的验证边缘预算，产生了更稀疏的拓扑。Qwen2.5-1.5B 的基础和指令电路保持了 84.7% 的 Jaccard 相似性，包括第 27 层中心，表明首个令牌路由在预训练期间基本建立，并通过指令调优得以保留。最后，EAP 分数与大多数模型的精确修补差异相关性较弱，表明线性梯度近似可能与 FP16 中的因果干预偏离，并促使进行精确修补验证以实现可靠的电路发现。

View on arXiv Download PDF AI Translation

cs.AI / 49 / 2609.30967

MoMHa: Multi-Objective Optimization of LLM Harnesses over Accuracy, Safety, and Tokens

MoMHa：大语言模型在准确性、安全性和令牌数上的多目标优化

Mukherjee, Subhojyoti, Tanjim, Md Mehrab

Abstract

Most work on improving large language models treats accuracy as the sole objective. We argue that the harness, the Python code surrounding the model that constructs prompts, routes calls, and parses outputs, is a first-class design surface whose quality is inherently multi-objective: an accurate harness that refuses no unsafe request, or that consumes an order of magnitude more tokens, is not a good harness. We present Meta-Harness, a system that casts harness design as search over three per-domain objectives (accuracy, behavioural safety, and token cost) solved by an agentic proposer (Claude Code) with full filesystem access to prior harness source, execution traces, and scoring artifacts. Our central finding is that a singlephase joint-reward proposer (MoMHa) outperforms every alternative, including a two-phase "accuracy then tokens" ablation, scalar-only feedback, and an accuracy-only baseline. We evaluate on seventeen domains: seven synthetic capability suites, seven real-world public benchmarks (HumanEval, MBPP, Spider, FEVER, MMLU-Pro, LawBench, NuminaMath), and three U-SafeBench-derived user-specific safety domains, using a 12-model fleet spanning four families. On the synthetic track MoMHa achieves a joint mean of 0.482 versus 0.198-0.422 for ten baselines, winning $7 / 10$ per-domain columns; on the real-world track it scores 0.461 versus 0.377 for the strongest baseline (DSPy), winning 5/7 columns, demonstrating that harness strategies transfer to unseen benchmarks without retraining on 8 of 12 target models. MoMHa attains the highest measured behavioral safety composite (U-SafeBench, 0.781) and uses 95 fewer tokens per example than the two-phase alternative. We will release all harness code, evaluation infrastructure, and crossmodel logs.

Chinese Translation

大多数关于改进大型语言模型的研究将准确性视为唯一目标。我们认为，围绕模型构建提示、路由调用和解析输出的Python代码，即“工具”，是一个一流的设计表面，其质量本质上是多目标的：一个准确的工具如果拒绝任何不安全的请求，或者消耗的令牌数量是数量级的增加，那么它就不是一个好的工具。我们提出了Meta-Harness，一个将工具设计视为在三个领域目标（准确性、行为安全性和令牌成本）上进行搜索的系统，由一个具有完整文件系统访问权限的代理提议者（Claude Code）解决，能够访问先前工具的源代码、执行轨迹和评分文档。我们的核心发现是，单阶段联合奖励提议者（MoMHa）在所有替代方案中表现最佳，包括两阶段的“先准确性再令牌”消融实验、仅标量反馈和仅准确性的基线。我们在十七个领域进行了评估：七个合成能力套件、七个真实世界公共基准（HumanEval、MBPP、Spider、FEVER、MMLU-Pro、LawBench、NuminaMath）和三个基于U-SafeBench的用户特定安全领域，使用覆盖四个家族的12个模型舰队。在合成轨道上，MoMHa实现了0.482的联合均值，而十个基线的均值为0.198-0.422，赢得了每个领域7/10的列；在真实世界轨道上，它的得分为0.461，相较于最强基线（DSPy）的0.377，赢得了5/7列，证明了工具策略能够在不对12个目标模型中的8个进行再训练的情况下转移到未见基准上。MoMHa达到了最高的行为安全复合评分（U-SafeBench，0.781），并且每个示例比两阶段替代方案使用了少95个令牌。我们将发布所有工具代码、评估基础设施和跨模型日志。

View on arXiv Download PDF AI Translation

cs.AI / 50 / 2609.30971

SciHorizon-eLab: An Agentic Protocol-to-Task Compiler for Scalable Benchmarking of Scientific Embodied Agents

SciHorizon-eLab：一种用于科学具身智能体可扩展基准测试的代理协议到任务编译器

Qin, Maokai, Qin, Chuan, Zhang, Qi, Liu, Dianyu, Liu, Zirui, Niu, Hongting, Zhou, Yuanchun, Zhu, Hengshu

Abstract

Embodied agents offer a promising route to automating scientific experimentation, yet their progress is constrained by the lack of reliable and systematic evaluation environments. Existing simulation-based laboratory benchmarks rely heavily on manual task engineering, making it challenging to systematically compile diverse scientific protocols into executable and verifiable embodied tasks at scale. To address this challenge, we introduce SciHorizon-eLab, an agentic protocol-to-task compiler that formulates scientific embodied task construction as a compilation problem. Given a natural-language protocol of scientific experiments, SciHorizon-eLab progressively compiles laboratory protocols into semantic-preserving embodied tasks through semantic grounding, executable task synthesis, and multi-stage simulation-based certification. The system generates semantically grounded environments, executable manipulation programs, and step-level success specifications, while enabling reproducible generation of expert demonstrations and execution traces. Using this pipeline, we further construct \BenchName, a ready-to-use benchmark comprising 300 certified tasks across diverse laboratory operations. It supports HIL task execution, reproducible expert-demonstration generation, and ordered step-level evaluation. Across representative tasks, the strongest policy attains an average success rate of only 49.7%, with further evaluations revealing pronounced weaknesses in human and embodied agent coordination. We publicly release the code, benchmark data, and evaluation toolkit at https://github.com/SciHorizon-elab/SciHorizon-elab.

Chinese Translation

具身智能体为自动化科学实验提供了有前景的途径，但其进展受到缺乏可靠和系统化评估环境的限制。现有的基于仿真的实验室基准测试在很大程度上依赖于手动任务工程，这使得系统性地将多样化的科学协议编译成可执行和可验证的具身任务变得具有挑战性。为了解决这一挑战，我们提出了SciHorizon-eLab，一种将科学具身任务构建形式化为编译问题的代理协议到任务编译器。给定一个自然语言的科学实验协议，SciHorizon-eLab通过语义基础、可执行任务合成和多阶段基于仿真的认证，逐步将实验室协议编译成语义保持的具身任务。该系统生成语义基础的环境、可执行的操作程序和逐步成功规范，同时支持可重复生成专家演示和执行轨迹。通过这一流程，我们进一步构建了enchname，一个包含300个经过认证的任务的现成基准，涵盖多种实验室操作。它支持HIL任务执行、可重复的专家演示生成和有序的逐步评估。在代表性任务中，最强的策略仅达到49.7%的平均成功率，进一步评估揭示了人类与具身智能体协调方面的明显弱点。我们在https://github.com/SciHorizon-elab/SciHorizon-elab上公开发布了代码、基准数据和评估工具包。

View on arXiv Download PDF AI Translation

cs.AI / 51 / 2609.30972

Factorized axis convolutional gated recurrent unit with dynamic adaptive pooling for remaining useful life prediction of rolling bearings

具有动态自适应池化的分解轴卷积门控递归单元用于滚动轴承剩余使用寿命预测

Park, Hanbyeol, Choo, Jungho, Bae, Hyerim

Abstract

Convolutional neural networks (CNN) are widely used to predict the remaining useful life (RUL) of rolling bearings from time-frequency representations (TFRs) of vibration signals. However, during degradation, characteristic structures in TFRs align predominantly along the frequency or time axis, making it challenging for conventional CNN isotropic kernels to capture directional structure. Furthermore, global average pooling (GAP) averages across axes, potentially obscuring the locations and concentrations of salient activations. This study introduces a factorized-axis convolutional gated recurrent unit (GRU) that employs multiscale anisotropic convolution and dual-axis convolution block attention module to enhance directional features and highlight salient time-frequency regions. Dynamic adaptive pooling (DAP) adaptively aggregates the time-frequency-axis information from the extracted feature maps, whereas a GRU captures temporal dynamics in the latent representations and Monte Carlo dropout enables predictive uncertainty estimation. Experiments on two public bearing datasets demonstrate that the proposed model outperforms existing RUL prediction methods across operating conditions. Ablation experiments demonstrate that the factorized axis-wise design achieves lower mean errors than convolutional isotropic kernels. DAP yields clear improvements on one dataset while matching GAP on the other, highlighting the importance of anisotropic feature extraction and adaptive feature aggregation for TFR-based RUL prediction.

Chinese Translation

卷积神经网络（CNN）广泛用于从振动信号的时频表示（TFRs）预测滚动轴承的剩余使用寿命（RUL）。然而，在退化过程中，TFR中的特征结构主要沿频率或时间轴对齐，这使得传统的CNN各向同性核难以捕捉方向性结构。此外，全球平均池化（GAP）在各个轴上进行平均，可能会掩盖显著激活的位置信息和浓度。本研究提出了一种分解轴卷积门控递归单元（GRU），该单元采用多尺度各向异性卷积和双轴卷积块注意力模块，以增强方向特征并突出显著的时频区域。动态自适应池化（DAP）自适应地聚合从提取的特征图中获得的时频轴信息，而GRU则捕捉潜在表示中的时间动态，蒙特卡洛丢弃法使得预测不确定性估计成为可能。在两个公共轴承数据集上的实验表明，所提出的模型在各种操作条件下优于现有的RUL预测方法。消融实验表明，分解轴向设计的平均误差低于卷积各向同性核。DAP在一个数据集上显著改善，同时在另一个数据集上与GAP相匹配，突显了各向异性特征提取和自适应特征聚合在基于TFR的RUL预测中的重要性。

View on arXiv Download PDF AI Translation

cs.AI / 52 / 2609.31013

Same Text, Different Numbers: The Divergence of LLM-Based Measures

相同文本，不同数字：基于大型语言模型的度量差异

Boustanifar, Hamid, Mansouri, Sasan

Abstract

Researchers increasingly use generative large language models (LLMs) to convert corporate text into empirical variables. We examine the extent to which LLM-based textual measures are invariant to model choice using thirteen measures, including sentiment, management clarity, uncertainty, answer specificity, and climate and political risk. Seven LLMs from different providers score earnings call transcripts of S&P 500 companies on these constructs. Cross-model rank correlations average only 0.52, and transcript-level differences common across providers account for only 34% of total score variation. Cross-model disagreement does not predict subsequent analyst or market disagreement, consistent with a substantial model-specific component rather than common ambiguity in the underlying disclosure. Model choice significantly affects downstream inference, with coefficient magnitudes, signs, and statistical significance varying substantially across models. Averaging across providers makes transcript rankings more stable for most constructs, but score levels remain sensitive to the models included in the ensemble. LLM-generated variables should therefore be treated as model-contingent measurements and validated across providers.

Chinese Translation

研究人员越来越多地使用生成性大型语言模型（LLMs）将企业文本转换为经验变量。我们考察了基于LLM的文本度量在模型选择上的不变性，使用了包括情感、管理清晰度、不确定性、回答具体性以及气候和政治风险在内的十三种度量。从不同提供商的七个LLM对标准普尔500公司财报电话会议记录进行评分。跨模型的排名相关性平均仅为0.52，而不同提供商之间的记录级差异仅占总评分变异的34%。跨模型的不一致性并不预测后续分析师或市场的分歧，这与模型特定成分的显著性相一致，而不是基础披露中的共同模糊性。模型选择显著影响下游推断，不同模型之间的系数大小、符号和统计显著性差异显著。对不同提供商的评分进行平均使得大多数构念的记录排名更加稳定，但得分水平仍然对所包含模型的变化敏感。因此，LLM生成的变量应视为依赖于模型的测量，并需在不同提供商之间进行验证。

View on arXiv Download PDF AI Translation

cs.AI / 53 / 2609.31029

Governed Deduction: Policy-Grounded Premise Authorization Beyond Relevance

受控推理：超越相关性的政策基础前提授权

Shu, Wesley, Lin, Hsi-Ching

Abstract

Reasoning systems usually treat premise use as a question of relevance: if a fact is available and useful, it may be selected for inference. Authorization imposes a different constraint: a premise may be represented and logically usable but not permitted for a particular local transition. We formalize this distinction as Governed Deduction (GD), with a transition-local admission predicate admit(p, tau, S). From an independently produced RBAC-augmented Spider benchmark, we construct 4,461 matched authorization pairs in which the same query premise and policy state support permitted and denied consuming transitions. An initial joint controller reaches 99.19% held-out accuracy, but a transition-only control reaches 100%, exposing a role-name shortcut. After a frozen, label-independent context-local role permutation removes that shortcut, premise/state-only, transition-only, and joint linear controllers all score exactly 50% on 1,856 held-out edges, while a symbolic policy oracle remains at 100%. The result is a controlled negative finding: the benchmark instantiates policy-grounded authorization beyond relevance, but the frozen linear representation does not recover the relation. Matched one-sided controls and leakage audits are therefore essential for evaluating learned policy-sensitive reasoning.

Chinese Translation

推理系统通常将前提的使用视为相关性的问题：如果一个事实是可用且有用的，它可能被选用于推理。授权则施加了不同的约束：一个前提可能被表示并在逻辑上可用，但在特定的局部转变中不被允许。我们将这种区别形式化为受控推理（Governed Deduction, GD），并引入一个转变局部的接受谓词 admit(p, tau, S)。从独立生成的增强角色访问控制（RBAC）Spider基准中，我们构建了4,461对匹配的授权对，其中相同的查询前提和政策状态支持被允许和拒绝的消费转变。初始的联合控制器达到了99.19%的保留准确率，但仅基于转变的控制达到了100%，暴露了一个角色名称的捷径。在冻结的、与标签无关的上下文局部角色排列去除了该捷径后，仅基于前提/状态、仅基于转变和联合线性控制器在1,856个保留边上均得分正好为50%，而符号政策神谕保持在100%。结果是一个受控的负发现：该基准实例化了超越相关性的政策基础授权，但冻结的线性表示并未恢复这种关系。因此，匹配的单侧控制和泄漏审计对于评估学习的政策敏感推理至关重要。

View on arXiv Download PDF AI Translation

cs.AI / 54 / 2609.31054

Cheap, open agents make LLM pollution harder to mitigate

廉价的开放代理使得大型语言模型污染更难以缓解

Rilla, Raluca, Nussberger, Anne-Marie, Mata, Rui, Wulff, Dirk U.

Abstract

Large Language Model (LLM) pollution occurs when synthetic responses contaminate data intended to capture human behavior. High deployment costs have so far limited the risk posed by autonomous survey agents. However, open-weight models paired with open-source agentic frameworks may have removed this barrier. We compared the performance and detectability of nine agent configurations, ranging from fully open variants to closed commercial ones. Each agent autonomously completed a survey containing multiple response types yielding various detection checks. Fully open agents ran locally without usage fees and performed competitively with commercial alternatives. Open and commercial agents failed different sets of checks, and no single check reliably detected all agents, but open-text responses discriminated best between agents and humans. These findings identify fully open agents as a distinct risk for LLM pollution and support multilayered detection strategies emphasizing open-text analysis.

Chinese Translation

大型语言模型（LLM）污染发生在合成响应污染了旨在捕捉人类行为的数据时。到目前为止，高昂的部署成本限制了自主调查代理所带来的风险。然而，开放权重模型与开源代理框架的结合可能已经消除了这一障碍。我们比较了九种代理配置的性能和可检测性，从完全开放的变体到封闭的商业版本。每个代理自主完成了一项包含多种响应类型的调查，产生了各种检测检查。完全开放的代理在本地运行，无需使用费用，并且在性能上与商业替代品竞争。开放和商业代理未能通过不同的检查集，没有单一的检查能够可靠地检测所有代理，但开放文本响应在区分代理和人类方面表现最佳。这些发现将完全开放的代理识别为LLM污染的一个独特风险，并支持强调开放文本分析的多层次检测策略。

View on arXiv Download PDF AI Translation

cs.AI / 55 / 2609.31056

Neuralyzing the Trace: Selective Representation-Level Unlearning with Contrastive Sparse Autoencoders

神经化痕迹：使用对比稀疏自编码器的选择性表示级别遗忘

Zehavi, Itai, Jourdan, Fanny, Aivodji, Ulrich

Abstract

Machine unlearning aims to remove targeted information while preserving a model's other abilities. In realistic settings, such as privacy requests under the EU GDPR, the target may be narrow, for example information associated with a single person. Behavioral forgetting alone may be insufficient, motivating interventions directly on internal representations. However, standard mechanistic-interpretability extractors are poorly selective for such targets. We identify an energy bias in reconstruction-based extraction, which favors dominant background structure over low-energy target-specific components. We introduce SCALPEL, a contrastive sparse autoencoder designed to learn more selective forget features. We show theoretically that contrastive training promotes target-selective features and that our selection score controls expected background knowledge perturbation. We validate SCALPEL experimentally on TOFU across Qwen, Llama, and Gemma, where it substantially improves over NMF and standard SAE interventions and is competitive with Gradient Difference and RMU, bridging mechanistic interpretability and fine-grained unlearning.

Chinese Translation

机器遗忘旨在移除特定信息，同时保留模型的其他能力。在现实环境中，例如根据欧盟通用数据保护条例（GDPR）的隐私请求，目标可能很狭窄，例如与单个个人相关的信息。仅仅依靠行为遗忘可能不足以满足需求，因此需要对内部表示进行直接干预。然而，标准的机械可解释性提取器对这些目标的选择性较差。我们识别出重构基础提取中的能量偏差，这种偏差更倾向于主导背景结构而非低能量的目标特定成分。我们提出了SCALPEL，一种旨在学习更具选择性遗忘特征的对比稀疏自编码器。我们理论上证明，对比训练促进了目标选择性特征的学习，并且我们的选择评分控制了预期背景知识的扰动。我们在TOFU上对SCALPEL进行了实验验证，涵盖了Qwen、Llama和Gemma，结果表明其在性能上显著优于NMF和标准SAE干预，并且与梯度差异（Gradient Difference）和RMU具有竞争力，成功地将机械可解释性与细粒度遗忘相结合。

View on arXiv Download PDF AI Translation

cs.AI / 56 / 2609.31071

Externalized CPDAG Summaries Improve LLM Causal Deduction

外部化的 CPDAG 摘要提升 LLM 因果推理能力

Sun, Wentao, Nogueira, João Paulo, Verchere, Dominique, Acher, Mathieu, Silva, Alonso

Abstract

Corr2Cause asks whether a causal claim holds in every DAG compatible with observed correlations and conditional independencies. We frame this as latent-object reasoning: the label is defined by a CPDAG query, but free-form chain-of-thought often collapses the Markov-equivalence-class problem into local pattern matching. We propose Structured Thinking, a two-turn pipeline that first externalizes a typed, schema-constrained CPDAG summary and then answers against that graph state. On the Corr2Cause full test, Structured Thinking raises Qwen3.5-27B from $73.0$ to $86.4$ $F_1$(Yes) over a strong PC-instruction baseline in the primary paired run ($+13.4$ pp; McNemar $p=2.4\times 10^{-6}$; bootstrap $95\%$ CI [$+8.4$, $+18.6$]); across three full-ID seeds, the mean gain is $+8.1 \pm 5.3$ pp. A PC-scaffolded two-turn prose control reaches only $67.6$ $F_1$, indicating that a detailed PC scaffold plus a schema-free prose intermediate is not sufficient. The same pattern holds on Qwen3.6-27B, Paraphrase-OOD, and GPT-5.4-mini. Scrambling the emitted CPDAG costs $12.0$ pp $F_1$, and a full-split audit shows close agreement with the reference CPDAG (ID skeleton $F_1$ $0.960$; exact match $75.9\%$). These results support a bounded design principle: externalize the latent object that defines the label, constrain its form, and test whether downstream answers use it.

Chinese Translation

Corr2Cause 问题是判断一个因果声明是否在与观察到的相关性和条件独立性兼容的每个有向无环图（DAG）中成立。我们将其框架视为潜在对象推理：标签由 CPDAG 查询定义，但自由形式的思维链常常将马尔可夫等价类问题简化为局部模式匹配。我们提出了结构化思维（Structured Thinking），这是一个两轮流程，首先外部化一个类型化、模式约束的 CPDAG 摘要，然后基于该图状态进行回答。在 Corr2Cause 全部测试中，结构化思维使 Qwen3.5-27B 的 $F_1$(Yes) 从 $73.0$ 提升至 $86.4$，超过强大的 PC 指令基线，在主要配对运行中提升了 $+13.4$ 个百分点；McNemar 测试 $p=2.4 imes 10^{-6}$；自助法 $95\%$ 置信区间 [$+8.4$, $+18.6$]；在三个完整 ID 种子中，平均增益为 $+8.1 \pm 5.3$ 个百分点。一个基于 PC 的两轮散文控制仅达到 $67.6$ 的 $F_1$，表明详细的 PC 支架加上无模式的散文中间并不足够。相同的模式在 Qwen3.6-27B、Paraphrase-OOD 和 GPT-5.4-mini 上也得到了验证。打乱发出的 CPDAG 会损失 $12.0$ 个百分点的 $F_1$，而全面分割审计显示与参考 CPDAG 的高度一致（ID 骨架 $F_1$ 为 $0.960$；完全匹配 $75.9 ext{ extperthousand}$）。这些结果支持一个有限设计原则：外部化定义标签的潜在对象，约束其形式，并测试下游答案是否使用它。

View on arXiv Download PDF AI Translation

cs.AI / 57 / 2609.31076

Up and Down the Abstraction Ladder: Code-Based Skills for Language Agents

抽象阶梯的上下：语言智能体的基于代码的技能

Cupiał, Bartłomiej, Tuyls, Jens, Wołczyk, Maciej, Paglieri, Davide, Klissarov, Martin, Eysenbach, Benjamin, Miłoś, Piotr, Narasimhan, Karthik R.

Abstract

Language agents struggle to act and learn in environments that require long sequences of low-level actions. Code-based abstractions can make these agents more productive by letting them invoke reusable skills instead of repeatedly selecting individual actions. The code handles recurring local decisions, while the language model decides which skills to use and how to combine them. Yet abstractions are leaky, and situations beyond a skill's capabilities may require a return to primitive actions. Motivated by this tradeoff between productivity and flexibility, we systematically study how code-based action abstraction affects the performance, inference cost, and learning of language agents. We study this in NetHack, a challenging, long-horizon game environment, using CodeHack, our library of code-based skills with natural-language descriptions. We use this library to compare agents restricted to primitives with those using semantic skills alone or in combination with primitives. We evaluate these agents in three settings: zero-shot prompting, supervised fine-tuning, and reinforcement learning. Across a broad zero-shot evaluation on NetHack, we find that compared with primitives, skills nearly triple game progression, while reducing inference cost per episode by 86%. Combining skills with primitives retains much of this benefit while preserving a path back down to low-level actions. Finally, in RL, we find that skill-based agents learn significantly faster than agents acting on primitives, achieving a 7.2x larger average gain in dungeon level over the same training budget. These results show that a supplied skill library can improve performance, efficiency, and learning, while retaining primitives provides flexibility when the library is insufficient. We release CodeHack together with training and evaluation code.

Chinese Translation

语言智能体在需要长序列低级动作的环境中行动和学习时面临挑战。基于代码的抽象可以通过让智能体调用可重用的技能，而不是反复选择单个动作，从而提高其生产力。代码处理重复的局部决策，而语言模型则决定使用哪些技能以及如何组合它们。然而，抽象是有缺陷的，超出技能能力的情况可能需要回归原始动作。基于这种生产力与灵活性之间的权衡，我们系统地研究了基于代码的动作抽象如何影响语言智能体的性能、推理成本和学习。我们在NetHack这一具有挑战性的长时间游戏环境中进行研究，使用我们的基于代码的技能库CodeHack，该库包含自然语言描述。我们利用这个库比较仅限于原始动作的智能体与使用语义技能单独或与原始动作组合的智能体。在零样本提示、监督微调和强化学习三个设置中评估这些智能体。在对NetHack进行广泛的零样本评估时，我们发现与原始动作相比，技能几乎使游戏进展增加了三倍，同时每集的推理成本降低了86%。将技能与原始动作结合在一起保留了大部分这种收益，同时保持了回归低级动作的路径。最后，在强化学习中，我们发现基于技能的智能体学习速度显著快于基于原始动作的智能体，在相同的训练预算下，地下城等级的平均增益达到了7.2倍。这些结果表明，提供的技能库可以提高性能、效率和学习，而保留原始动作在库不足时提供了灵活性。我们发布了CodeHack以及训练和评估代码。

View on arXiv Download PDF AI Translation

cs.AI / 58 / 2609.31078

OmouAI: Argumentative Human-AI Policy Deliberation with Simulated Personas

OmouAI：基于模拟角色的争辩性人机政策审议

Vasileiou, Stylianos Loukas, Rago, Antonio, Yeoh, William, Curto, Georgina

Abstract

Debates amongst agents driven by large language models (LLMs) have demonstrated vast potential in various applications, but when these interactions include humans and take place in high-stakes environments, e.g., in public policy deliberations, they are beset with issues such as sycophancy and a lack of faithful explanations. To tackle these issues, we present OmouAI, an interactive and inclusive deliberation system that uses LLMs in combination with computational argumentation, a field which excels in representing and reasoning within debates. OmouAI allows a human user to deliberate policy claims for real-world challenges with simulated personas, e.g., representing stakeholders, domain experts or devil's advocates, towards reducing sycophancy. Each persona generates its own arguments, and the arguments of all parties form a shared argumentation framework. Users can then contest, add and revise arguments, providing crucial human oversight. Then, arguments are evaluated using deterministic argumentative semantics against external goals, such as the UN Sustainable Development Goals, guaranteeing faithful explanations. The advancement or worsening of the goals thus serve as indicators for the policy recommendations.

Chinese Translation

由大型语言模型（LLMs）驱动的代理之间的辩论在各种应用中展现了巨大的潜力，但当这些互动涉及人类并发生在高风险环境中，例如公共政策审议时，它们面临着如谄媚和缺乏可信解释等问题。为了解决这些问题，我们提出了OmouAI，这是一个互动且包容的审议系统，结合了LLMs和计算论证，后者在辩论的表述和推理方面表现出色。OmouAI允许人类用户与模拟角色进行政策主张的审议，例如代表利益相关者、领域专家或反对者，以减少谄媚现象。每个角色生成自己的论点，所有参与方的论点形成一个共享的论证框架。用户可以对论点进行质疑、添加和修订，从而提供关键的人类监督。随后，论点使用确定性的论证语义与外部目标（如联合国可持续发展目标）进行评估，以确保可信的解释。因此，目标的推进或恶化作为政策建议的指标。

View on arXiv Download PDF AI Translation

cs.AI / 59 / 2609.31121

Monitor Jailbreaking: Evading Chain-of-Thought Monitoring Without Encoded Reasoning

监控越狱：在没有编码推理的情况下规避思维链监控

Schulz, Julian

Abstract

Chain-of-thought (CoT) monitoring is a promising safety technique for reasoning models, enabling detection of problematic reasoning before models act. A key concern is encoded reasoning, where models hide their true reasoning in ways that monitors and humans cannot interpret. Optimization pressure from CoT monitors during reinforcement learning is considered a likely driver of such behavior. We investigate this by training reasoning models to perform a main task and a side task, while penalizing them when a monitor detects reasoning about the side task. Surprisingly, models learn to evade monitors without encoding their reasoning. Instead, they learn to phrase and format their chains of thought such that monitors fail to flag side task reasoning, while the reasoning remains completely transparent to human readers. We call this phenomenon monitor jailbreaking. We find that monitor jailbreaking arises across different model sizes, monitors, and tasks. Jailbreaks generalize to monitors not seen during training, including both less and more capable monitors, and transfer across different monitor prompts. While jailbreaking strategies appear simple, manually replicating them does not reliably fool monitors. Finally, we show that paraphrasing is an effective defense: paraphrasing a jailbroken CoT allows the same monitor to correctly flag it, while still allowing the model to perform both tasks.

Chinese Translation

思维链（Chain-of-thought, CoT）监控是一种有前景的安全技术，用于推理模型，能够在模型采取行动之前检测出问题推理。一个关键问题是编码推理，即模型以监控者和人类无法解释的方式隐藏其真实推理。来自 CoT 监控的优化压力在强化学习过程中被认为是导致这种行为的一个可能因素。我们通过训练推理模型执行主要任务和侧面任务来研究这一点，同时在监控者检测到关于侧面任务的推理时对其进行惩罚。令人惊讶的是，模型学会了在不编码其推理的情况下规避监控者。相反，它们学会了以一种措辞和格式化思维链的方式，使得监控者未能标记侧面任务推理，而推理对人类读者仍然完全透明。我们称这种现象为监控越狱（monitor jailbreaking）。我们发现监控越狱在不同的模型规模、监控者和任务中普遍存在。越狱策略可以推广到训练过程中未见过的监控者，包括能力较弱和较强的监控者，并在不同的监控提示中迁移。尽管越狱策略看似简单，但手动复制它们并不能可靠地欺骗监控者。最后，我们展示了释义（paraphrasing）是一种有效的防御手段：对越狱的 CoT 进行释义可以使同一监控者正确标记它，同时仍允许模型执行两个任务。

View on arXiv Download PDF AI Translation

cs.AI / 60 / 2609.31133

AtomWorld-Mem: Memory-Restored World States for Long-Horizon Atomistic Evolution

AtomWorld-Mem：用于长时间尺度原子演化的记忆恢复世界状态

Luo, Tian, Zhang, Ruge, Han, Haozhi, Chen, Yifrng, Zhang, Yunquan, Liu, Yunxin, Cao, Ting, Li, Kun

Abstract

High-fidelity atomistic evolution over long timescales requires more than observing the current crystal configuration. Instantaneous atomistic snapshots are often incomplete: locally similar configurations can correspond to different hidden dynamical contexts, future event preferences, and waiting-time scales. We argue that this snapshot ambiguity makes long-horizon atomistic evolution fundamentally a memory-based world-state restoration problem. To address this, we introduce AtomWorld-Mem, a memory-restored atomistic world model that recovers the latent world state missing from instantaneous crystal snapshots. AtomWorld-Mem treats the evolving alloy as an AtomWorld: spatial encoders write multi-scale atomistic keyframes from dense local topology and sparse long-range defect context, while short-term event memory and long-term structural memory integrate these keyframes across time to restore a future-predictive evolutionary state. The restored state is used to prioritize legal vacancy-mediated events under single-event Kinetic Monte Carlo (KMC) constraints, while event legality, physical execution, and residence-time updates remain governed by the underlying simulator. Empirically, AtomWorld-Mem improves long-horizon atomistic progress under fixed microscopic event budgets while maintaining high-fidelity evolution across energetic, structural, and vacancy-transport observables. It further transfers zero-shot across diverse unseen alloy-temperature AtomWorlds, suggesting that the learned memory-restoration mechanism captures reusable principles of hidden-state inference rather than a system-specific local energy heuristic. These results position memory-restored world-state modeling as a promising route toward efficient, physically grounded, and transferable atomistic evolution.

Chinese Translation

高保真度的长时间尺度原子演化不仅仅依赖于当前晶体配置的观察。瞬时的原子快照往往是不完整的：局部相似的配置可能对应于不同的隐藏动态背景、未来事件偏好和等待时间尺度。我们认为，这种快照模糊性使得长时间尺度的原子演化根本上成为一个基于记忆的世界状态恢复问题。为了解决这一问题，我们引入了AtomWorld-Mem，一种记忆恢复的原子世界模型，旨在恢复瞬时晶体快照中缺失的潜在世界状态。AtomWorld-Mem将演变中的合金视为一个AtomWorld：空间编码器从密集的局部拓扑和稀疏的长程缺陷背景中写入多尺度的原子关键帧，而短期事件记忆和长期结构记忆则跨时间整合这些关键帧，以恢复一个未来可预测的演化状态。恢复的状态用于在单事件动力学蒙特卡罗（KMC）约束下优先考虑合法的空位介导事件，同时事件的合法性、物理执行和停留时间更新仍由基础模拟器控制。实证结果表明，AtomWorld-Mem在固定微观事件预算下改善了长时间尺度的原子进展，同时在能量、结构和空位传输可观测量上保持高保真度的演化。它进一步在多样的未见合金-温度AtomWorlds中实现零样本迁移，表明所学习的记忆恢复机制捕捉到了可重用的隐藏状态推断原则，而不是特定系统的局部能量启发式。这些结果将记忆恢复的世界状态建模定位为实现高效、物理基础和可迁移的原子演化的有前景的途径。

View on arXiv Download PDF AI Translation

cs.AI / 61 / 2609.31136

Toward AI-Augmented Cooperative Engineering Workflows: Requirements and Architecture the European Rover Challenge

面向AI增强的协作工程工作流程：欧洲探测器挑战的需求与架构

Sadik, Ahmed R., Joublin, Frank, Bujny, Mariusz, Ceravola, Antonello, Smith, Joan

Abstract

The growing availability of Artificial Intelligence (AI) tools creates new opportunities to support engineering design processes, yet their current use often remains limited to isolated tasks such as coding, documentation, or information retrieval. Less attention has been given to how AI can support cooperative engineering workflows at the process level, where teams must coordinate requirements, tasks, communication, knowledge transfer, and subsystem integration. This paper investigates this challenge in the context of the European Rover Challenge (ERC), where student teams design and integrate complex rover systems within a single academic cycle under strict time constraints and high subsystem interdependence. We conducted a role adaptive 40 question survey with ERC 2025 teams, yielding 104 responses from 14 teams. The survey examined team structure, knowledge transfer, task management, integration practices, communication patterns, and current AI usage. The results reveal recurring workflow bottlenecks, including limited documentation, unclear requirements, fragmented communication, informal task monitoring, and substantial integration rework. Based on these findings, we derive requirements for AI augmented cooperative engineering work-flows and propose an initial assistant system architecture that connects user facing interfaces, credential management, service selection, specialized AI services, and external engineering tools. The proposed architecture aims to support task clarification, requirement and compliance management, communication summarization, integration risk detection, and continuous knowledge capture. In doing so, the paper contributes empirical requirements and an architectural direction for AI augmented cooperative engineering workflows in hybrid human AI team settings.

Chinese Translation

人工智能（AI）工具的日益普及为支持工程设计过程创造了新的机会，但其当前的使用往往仅限于编码、文档编写或信息检索等孤立任务。对于AI如何在过程层面支持协作工程工作流程的关注较少，而在这一层面，团队必须协调需求、任务、沟通、知识转移和子系统集成。本文在欧洲探测器挑战（ERC）的背景下探讨了这一挑战，在该挑战中，学生团队在严格的时间限制和高度的子系统相互依赖下设计和集成复杂的探测器系统。我们对ERC 2025团队进行了角色适应性的40个问题调查，共收集了来自14个团队的104份回复。调查考察了团队结构、知识转移、任务管理、集成实践、沟通模式以及当前的AI使用情况。结果揭示了反复出现的工作流程瓶颈，包括文档有限、需求不明确、沟通碎片化、任务监控不正式以及大量的集成返工。基于这些发现，我们提出了AI增强的协作工程工作流程的需求，并建议了一种初步的助手系统架构，该架构连接了用户界面、凭证管理、服务选择、专业AI服务和外部工程工具。所提议的架构旨在支持任务澄清、需求和合规管理、沟通总结、集成风险检测及持续知识捕获。通过这样做，本文为混合人机团队环境中的AI增强协作工程工作流程提供了实证需求和架构方向。

View on arXiv Download PDF AI Translation

cs.AI / 62 / 2609.31140

Can Linguistic Reasoning Vectors Enhance Multimodal Reasoning Ability?

语言推理向量能否增强多模态推理能力？

Wang, Ziyi, Li, Li, Zhou, Aolin, Shen, Yankun, Liu, Chonghan, Lin, Shuxia, Yang, Xu

Abstract

Most Vision-Language Models (VLMs) are built by extending pretrained Large Language Models (LLMs) with visual modules and multimodal alignment. However, this multimodal scaling often degrades the language-side reasoning ability originally encoded in the base LLM. While the base LLM retains usable reasoning after scaling, the aligned VLM itself cannot reliably access this ability. Therefore, recovering the degraded reasoning capability in VLMs would benefit more from seeking help from the base LLM than from the VLM alone. Motivated by this, we propose LIFT (Language-side reasonIng Facilitation and Transfer), a lightweight vector-intervention method that transfers reasoning capability from the base LLM to the VLM without retraining the backbone. LIFT defines Reasoning Vectors as answer-token hidden-state differences between a Reasoner path with an explicit reasoning trace and a Solver path without it, and injects these vectors into language-side activations of the target VLM. LIFT further supports learnable vector adaptation while keeping the VLM backbone frozen. We evaluate LIFT on two VLMs across six reasoning benchmarks, comparing Reasoning Vectors extracted from the base LLM and from the aligned VLM under matched protocols. Results show that LLM-derived vectors consistently outperform VLM-derived vectors, confirming that the base LLM is a more effective source for recovering reasoning. LIFT partially recovers degraded reasoning through lightweight language-side interventions. Further analyses show that Reasoning Vectors influence intermediate reasoning behavior rather than merely altering final answers. The source code will be released soon.

Chinese Translation

大多数视觉-语言模型（VLMs）是通过扩展预训练的大型语言模型（LLMs）并结合视觉模块和多模态对齐而构建的。然而，这种多模态扩展往往会降低原本在基础LLM中编码的语言推理能力。尽管基础LLM在扩展后仍保留可用的推理能力，但对齐的VLM本身无法可靠地访问这种能力。因此，恢复VLM中退化的推理能力更需要依赖基础LLM的帮助，而非仅依赖VLM。基于此，我们提出了LIFT（语言侧推理促进与转移），这是一种轻量级的向量干预方法，可以在不重新训练主干网络的情况下，将推理能力从基础LLM转移到VLM。LIFT将推理向量定义为具有明确推理轨迹的推理路径与没有推理轨迹的求解路径之间的答案令牌隐状态差异，并将这些向量注入目标VLM的语言侧激活中。LIFT进一步支持可学习的向量适应，同时保持VLM主干网络不变。我们在两个VLM上对LIFT进行了评估，涵盖六个推理基准，比较了在匹配协议下从基础LLM和对齐VLM提取的推理向量。结果表明，来源于LLM的向量始终优于来源于VLM的向量，确认基础LLM是恢复推理的更有效来源。LIFT通过轻量级的语言侧干预部分恢复了退化的推理能力。进一步分析表明，推理向量影响中间推理行为，而不仅仅是改变最终答案。源代码将很快发布。

View on arXiv Download PDF AI Translation

cs.AI / 63 / 2609.31159

Momentum-Guided Federated Split Distillation for Personalized Temporal Edge Intelligence

基于动量引导的个性化时序边缘智能联邦分割蒸馏

Baahmed, Ahmed-Rafik, Dollinger, Jean-François, Brahmia, Amine, Zghal, Mourad

Abstract

We propose a momentum-guided federated split distillation framework for personalized, efficient, and autonomous temporal edge intelligence. We introduce TeRR-SAtt, our novel temporal reservoir student attention design that combines fixed reservoir representations, a lightweight temporal student, and personalized output modules. We also present AMGF, our anticipatory momentum-guided fusion mechanism that clusters clients through learning momentum and derives specialized teacher updates. On real-world smart-building data, TeRR-SAtt reduces edge training latency by 65.50%, inference latency by 44.70%, training memory usage by 18.40%, and inference CPU usage by 33.10% over the considered baselines. At the same time, AMGF improves local learning by up to 35.31% in RMSE compared to global updates.

Chinese Translation

我们提出了一种基于动量引导的联邦分割蒸馏框架，以实现个性化、高效和自主的时序边缘智能。我们引入了TeRR-SAtt，这是一种新颖的时序储层学生注意力设计，结合了固定的储层表示、轻量级的时序学生和个性化输出模块。我们还提出了AMGF，这是一种预期的动量引导融合机制，通过学习动量对客户端进行聚类，并推导出专门的教师更新。在真实的智能建筑数据上，TeRR-SAtt在考虑的基准上将边缘训练延迟减少了65.50%，推理延迟减少了44.70%，训练内存使用减少了18.40%，推理CPU使用减少了33.10%。同时，与全局更新相比，AMGF在均方根误差（RMSE）方面提高了本地学习效果，提升幅度可达35.31%。

View on arXiv Download PDF AI Translation

cs.AI / 64 / 2609.31167

Neural State Prediction: Obstructing Shortcut Learning in EEG Foundation Models

神经状态预测：阻碍 EEG 基础模型中的捷径学习

Yu, Kieren, Liu, Ziyang, Huang, Chang, Chen, Jintai, Wu, Kaishun

Abstract

EEG foundation models increasingly use masked prediction to learn from unlabeled recordings, but optimizing this objective does not ensure transferable neural representations. A central challenge is that stable positional cues and local correlations can make masked regions predictable without integrating distributed neural context. To reduce this reliance on low-information prediction paths, we introduce Neural State Prediction (NSP), a latent-predictive framework that constrains both the prediction target and the available context. NSP uses a Target Encoder updated by an exponential moving average (EMA) to define latent supervision. Identity residualization removes additive effects associated with channel identity and relative time from the targets, while topology-separated context excludes their immediate spatial and temporal neighborhood from the visible input. We pretrain NSP on 2.2 million EEG segments from TUEG and evaluate it across 30 downstream datasets spanning clinical diagnosis, sleep staging, emotion recognition, motor imagery, event-related potentials, cognitive-state decoding, and language retrieval. Under full-parameter multi-task fine-tuning on EEG-FM-Bench, NSP achieves 63.94 macro balanced accuracy across 14 datasets, exceeding the strongest evaluated baseline by 2.35 percentage points. Controlled component ablations assess the contribution of each mechanism, while matched context controls and held-out interventions characterize the role of context geometry, signal content, and positional information. Jointly designing latent targets and their context offers a promising direction for EEG foundation models that learn from distributed signal structure.

Chinese Translation

EEG 基础模型越来越多地使用掩蔽预测从未标记的录音中学习，但优化这一目标并不能确保可转移的神经表征。一个核心挑战是，稳定的位置信息和局部相关性可以使掩蔽区域变得可预测，而无需整合分布式神经上下文。为了减少对低信息预测路径的依赖，我们引入了神经状态预测（Neural State Prediction, NSP），这是一种潜在预测框架，约束了预测目标和可用上下文。NSP 使用通过指数移动平均（Exponential Moving Average, EMA）更新的目标编码器来定义潜在监督。身份残差化消除了与通道身份和相对时间相关的附加效应，而拓扑分离上下文则排除了其直接空间和时间邻域的可见输入。我们在来自 TUEG 的 220 万个 EEG 段上预训练 NSP，并在涵盖临床诊断、睡眠分期、情感识别、运动想象、事件相关电位、认知状态解码和语言检索的 30 个下游数据集上进行了评估。在 EEG-FM-Bench 上进行全参数多任务微调时，NSP 在 14 个数据集上实现了 63.94 的宏平衡准确率，超过了最强评估基线 2.35 个百分点。受控组件消融评估了每个机制的贡献，而匹配上下文控制和保留干预则描述了上下文几何、信号内容和位置信息的作用。共同设计潜在目标及其上下文为从分布式信号结构中学习的 EEG 基础模型提供了一个有前景的方向。

View on arXiv Download PDF AI Translation

cs.AI / 65 / 2609.31176

Semantic Navigation for Issue Localization in Code Repository

代码库中问题定位的语义导航

Wei, Yunxiang, Lei, Zhenyu, Li, Jundong

Abstract

Repository-level issue localization aims to identify and rank the files and functions relevant to resolving a reported issue. LLM agents approach this task iteratively: they identify a set of potentially relevant locations, inspect the corresponding code, and revise their judgments about these candidates as new evidence is acquired. Existing environments, however, provide limited support for this loop: agents must search for unresolved relation targets, reconstruct entity semantics from raw source code, and revise candidates without evidential basis. To address these limitations, we present SemNav, a framework that leverages deterministic retrieval to seed a broad candidate set and an LLM agent to continually refine that set, thereby combining initial coverage with evidence-guided revision. SemNav supports this process through three key components. A Semantic Navigation Graph resolves program relations on demand through a language server, enabling direct navigation to related entities across files. Issue-conditioned Semantic Cards provide compact, source-grounded interpretations of each entity's role and relevance to the issue. A persistent Candidate Workspace records each candidate together with its evidential basis, enabling grounded verification, revision, and ranking. Across SWE-bench Lite and PLocBench, SemNav outperforms existing baselines, improving File Hit@10 from 68.33\% to 82.67\% with Gemma 4B. Component ablations and trajectory analysis support the complementary roles of all three components, while Semantic Cards reduce working-context load by 48.2\% relative to full-source reading. SemNav further ranks first on all seven evidence-quality metrics on SWE-Explore and improves downstream issue resolution from 44.00\% to 52.33\%.

Chinese Translation

库级问题定位旨在识别和排序与解决报告问题相关的文件和函数。大语言模型（LLM）代理以迭代方式处理此任务：它们识别一组潜在相关的位置，检查相应的代码，并在获得新证据时修正对这些候选项的判断。然而，现有环境对这一循环的支持有限：代理必须搜索未解决的关系目标，从原始源代码重构实体语义，并在没有证据基础的情况下修正候选项。为了解决这些局限性，我们提出了 SemNav，一个利用确定性检索来生成广泛候选集的框架，并通过 LLM 代理不断优化该集合，从而将初始覆盖与证据引导的修正相结合。SemNav 通过三个关键组件支持这一过程。语义导航图（Semantic Navigation Graph）通过语言服务器按需解析程序关系，使得能够直接导航到跨文件的相关实体。问题条件语义卡（Issue-conditioned Semantic Cards）提供每个实体角色和与问题相关性的紧凑、源基础解释。持久候选工作区（Candidate Workspace）记录每个候选项及其证据基础，从而实现基于证据的验证、修正和排序。在 SWE-bench Lite 和 PLocBench 上，SemNav 超越了现有基线，将 File Hit@10 从 68.33\% 提升至 82.67\%（使用 Gemma 4B）。组件消融和轨迹分析支持所有三个组件的互补作用，而语义卡相较于完整源代码阅读减少了 48.2\% 的工作上下文负载。SemNav 在 SWE-Explore 上的七个证据质量指标中进一步排名第一，并将下游问题解决率从 44.00\% 提升至 52.33\%。

View on arXiv Download PDF AI Translation

cs.AI / 66 / 2609.31179

SPO: Discovering Adaptive Large Neighborhood Search Operators via Stackelberg Program Optimization

SPO：通过斯塔克尔堡程序优化发现自适应的大邻域搜索算子

Ke, Xinyi, Li, Kai, Xing, Junliang, Zhang, Yifan, Cheng, Jian

Abstract

Large neighborhood search (LNS) relies critically on destroy and repair operators, whose effectiveness depends on both adaptation to the evolving LNS state and interaction between the two roles. We introduce Stackelberg Program Optimization (SPO), an LLM-based framework for discovering adaptive executable destroy-repair programs. SPO conditions operator decisions on a compact LNS state, allowing state-dependent behavior to emerge through program discovery, and organizes destroy-repair discovery as a Stackelberg interaction over program space that reflects their asymmetric dependency. Role-specific credits evaluate destroy programs as leaders and repair programs as conditional follower responses, guiding a coupled optimization process that combines LLM generator learning with population-based evolutionary search over programs. Experiments on the traveling salesperson problem and capacitated vehicle routing problem show that SPO outperforms strong baselines across a broad range of settings and generalizes beyond the discovery scale to larger instances and benchmark sets. Behavioral analyses further demonstrate state-dependent operator behavior and coupled destroy-repair improvement during discovery.

Chinese Translation

大邻域搜索（LNS）在很大程度上依赖于破坏和修复算子，其有效性取决于对不断变化的LNS状态的适应性以及两者之间的相互作用。我们引入了斯塔克尔堡程序优化（SPO），这是一个基于LLM的框架，用于发现自适应的可执行破坏-修复程序。SPO将算子决策建立在紧凑的LNS状态上，允许通过程序发现产生状态依赖的行为，并将破坏-修复的发现组织为在程序空间上的斯塔克尔堡交互，反映出它们的不对称依赖性。角色特定的信用评估将破坏程序视为领导者，将修复程序视为条件跟随者响应，指导一个结合了LLM生成器学习和基于种群的程序进化搜索的耦合优化过程。在旅行商问题和容量车辆路径问题上的实验表明，SPO在广泛的设置中优于强基线，并且在发现规模之外对更大实例和基准集具有良好的泛化能力。行为分析进一步证明了状态依赖的算子行为以及在发现过程中的耦合破坏-修复改进。

View on arXiv Download PDF AI Translation

cs.AI / 67 / 2609.31184

Accounting for Bias Enables Sustainable LLM Evaluation

考虑偏差以实现可持续的 LLM 评估

Katoch, Harshita, Selby, David Antony, Großmann, Gerrit, Vollmer, Sebastian

Abstract

LLM-as-a-judge has become the de facto standard for scalable, subjective evaluation, yet current leaderboards compensate for systematic measurement bias by running ever more comparisons, an approach that is both statistically unsound and computationally wasteful. The root cause is an incomplete measurement model, treating LLM judges as neutral, interchangeable instruments ignores documented biases like position bias, verbosity bias, judge severity, and self-enhancement, that no volume of additional data can eliminate. We propose a unified latent variable framework that jointly models pairwise and ordinal data while explicitly correcting for these confounders, recovering reliable rankings from substantially fewer comparisons. Because fitting this model costs negligible compute relative to a single round of LLM inference, bias correction is not only more statistically rigorous but also a more sustainable approach to trustworthy evaluation.

Chinese Translation

LLM 作为评判者已成为可扩展、主观评估的事实标准，但当前的排行榜通过进行越来越多的比较来补偿系统性测量偏差，这种方法在统计上是不合理的，并且在计算上是浪费的。根本原因在于测量模型的不完整，视 LLM 评判者为中立、可互换的工具忽视了诸如位置偏差、冗长偏差、评判者严厉程度和自我增强等已记录的偏差，而这些偏差无法通过大量额外数据消除。我们提出了一个统一的潜变量框架，该框架联合建模成对和序数数据，同时明确纠正这些混杂因素，从而从显著更少的比较中恢复可靠的排名。由于拟合该模型的计算成本相对于一次 LLM 推理是微不足道的，因此偏差校正不仅在统计上更为严谨，而且也是一种更可持续的可信评估方法。

View on arXiv Download PDF AI Translation

cs.AI / 68 / 2609.31186

Evolutionary Safety of Recursive Self-Improving AI: Taxonomy, Risk Discovery, and Evaluation

递归自我改进人工智能的进化安全性：分类、风险发现与评估

Gong, Chang, Bi, Jingping, Yao, Di, Liang, Xinjian, Xiang, Chao, Guo, Ruijie

Abstract

Artificial intelligence is advancing rapidly, with increasingly capable systems taking larger roles in reasoning, decision-making, scientific discovery, and autonomous development. As AI begins to participate in its own improvement, from model training and experience accumulation to agent evolution and automated AI development, the prospect of recursive self-improvement (RSI) is becoming increasingly relevant. This transition raises a fundamental safety question: how can safety be maintained when the system, its accumulated experience, and even the process producing its successors continue to change? We introduce Evolutionary Safety as a perspective for studying safety under persistent and recursive self-improvement. It concerns not only whether an AI system is safe at a particular moment, but how safety properties change, persist, accumulate, and propagate throughout evolution. We characterize recurring manifestations, including intent drift, error accumulation, experience contamination, safety-property erosion, evaluator drift, and risk propagation. We then develop a taxonomy spanning persistent agent state, model state, evaluation and environmental feedback, computational substrate, and meta-level update mechanisms. Building on this taxonomy, we examine how evolutionary risks can be discovered and evaluated across states, updates, trajectories, and lineages, and derive governance principles for modification, selection, authorization, provenance, and recovery. Finally, we outline open problems toward maintaining safety guarantees as AI systems become increasingly persistent, adaptive, and recursively self-improving. Project resources and proposed evaluation systems are available at https://chaunceykung.github.io/evolutionary-safety-rsi.

Chinese Translation

人工智能正在迅速发展，越来越强大的系统在推理、决策、科学发现和自主开发中扮演着越来越重要的角色。随着人工智能开始参与自身的改进，从模型训练和经验积累到代理进化和自动化人工智能开发，递归自我改进（Recursive Self-Improvement, RSI）的前景变得愈发相关。这一转变提出了一个基本的安全性问题：当系统、其积累的经验，甚至产生其后继者的过程持续变化时，如何保持安全性？我们引入进化安全性（Evolutionary Safety）作为研究在持续和递归自我改进下安全性的一种视角。它不仅关心人工智能系统在特定时刻是否安全，还关注安全属性如何在进化过程中变化、持续、积累和传播。我们描述了反复出现的表现形式，包括意图漂移、错误积累、经验污染、安全属性侵蚀、评估者漂移和风险传播。然后，我们开发了一个涵盖持久代理状态、模型状态、评估和环境反馈、计算基底以及元级更新机制的分类法。在此分类法的基础上，我们考察了如何在不同状态、更新、轨迹和谱系中发现和评估进化风险，并推导出修改、选择、授权、来源和恢复的治理原则。最后，我们概述了在人工智能系统变得越来越持久、自适应和递归自我改进的过程中，保持安全保证的开放问题。项目资源和提议的评估系统可在 https://chaunceykung.github.io/evolutionary-safety-rsi 获取。

View on arXiv Download PDF AI Translation

cs.AI / 69 / 2609.31201

Samples, Sources, Space: Decomposing Data Scale in Spatially Structured Representation Learning of Human Brain Microarchitecture

样本、来源与空间：在空间结构化的人脑微观建筑表示学习中分解数据规模

Schiffer, Christian, Bode, Mathis, Lippert, Thomas, Amunts, Katrin, Dickscheid, Timo

Abstract

Scaling studies typically represent training data by a single count of samples. For hierarchically and spatially structured data, however, the same number of samples can be drawn from few or many sources and distributed differently across the underlying domain. We therefore study data scaling as an allocation problem, separating unique sample count, source diversity, and spatial coverage. We study this decomposition in microscopic whole-brain histology, where a source is an individual brain, and a sample is an image patch at a specific spatial location. Across 93 controlled pretraining runs of a contrastive model that uses spatial proximity for supervision, we vary data allocation, compute, and model capacity over 11.6 million spatially anchored image patches from 21 human brains. Performance improves with more unique samples, broader spatial coverage, additional compute, and larger model capacity. At fixed sample count, distributing samples across one to 18 subjects produces no detectable improvement, even though representations generalize substantially better to subjects encountered during pretraining. Inter-subject variation therefore strongly affects generalization, but additional subjects provide no benefit when a fixed sample budget is distributed across more sources. These results establish sample count, source diversity, and spatial coverage as distinct axes of data scaling in spatially structured representation learning.

Chinese Translation

规模研究通常通过样本的单一计数来表示训练数据。然而，对于层次和空间结构化的数据，相同数量的样本可以来自少数或多数来源，并在基础领域中以不同方式分布。因此，我们将数据规模视为一种分配问题，分离独特样本计数、来源多样性和空间覆盖。我们在微观全脑组织学中研究这一分解，其中来源是单个大脑，样本是特定空间位置的图像补丁。在93次受控的对比模型预训练运行中，该模型利用空间邻近性进行监督，我们对来自21个人类大脑的1160万个空间锚定图像补丁的数据显示分配、计算和模型容量进行变化。随着独特样本数量的增加、空间覆盖范围的扩大、计算能力的增加和模型容量的增大，性能得到了提升。在固定样本数量的情况下，将样本分配到1到18个受试者之间并未产生可检测的改善，尽管表示在预训练期间对遇到的受试者的泛化能力显著更好。因此，个体间的变异性对泛化能力有很大影响，但当固定样本预算分配到更多来源时，增加的受试者并没有带来任何好处。这些结果确立了样本计数、来源多样性和空间覆盖作为空间结构化表示学习中数据规模的不同维度。

View on arXiv Download PDF AI Translation

cs.AI / 70 / 2609.31214

Which Influence Are We Estimating? The Role of Counterfactual Specifications in Data Attribution

我们在估计哪种影响？反事实规范在数据归因中的作用

Li, Zhe, Zhao, Wei, Zhang, Peixin, Sun, Jun

Abstract

Estimating the influence of training examples on model behavior is essential for data debugging, valuation, and attribution. Existing influence estimators often produce incompatible rankings, which are commonly ascribed to approximation error. We argue that a more fundamental source of disagreement is specification mismatch: influence depends on the behavior being attributed, the intervention applied to each training example, and the counterfactual training process that maps the intervention to a model response. These choices are especially important when the target behavior requires a tractable surrogate, such as query loss, a logit, or a margin. We formalize influence as a counterfactual estimand, distinguish specification mismatch across estimands from approximation error in estimating a fixed estimand, and organize representative estimators by their implied specifications. We further derive a local decomposition that exposes how behavior signals, training signals, and counterfactual parameter responses interact. Controlled experiments show that exact estimands under different specifications can induce different rankings, whereas approximation error grows as perturbations move farther from their linearization points. Experiments on noisy label detection and LLM attribution show that specification choices significantly affect attribution quality, especially for the choice of behavior surrogate. Behavior-aligned specifications can identify target-specific training examples obscured by default loss-based or similarity-based specifications. These results establish specification analysis as a necessary first step for interpreting and comparing data influence estimators.

Chinese Translation

估计训练样本对模型行为的影响对于数据调试、评估和归因至关重要。现有的影响估计器往往产生不兼容的排名，这通常归因于近似误差。我们认为，更根本的分歧来源于规范不匹配：影响依赖于被归因的行为、施加于每个训练样本的干预，以及将干预映射到模型响应的反事实训练过程。这些选择在目标行为需要可处理的替代物时尤为重要，例如查询损失、logit 或边际。我们将影响形式化为反事实估计量，区分不同估计量之间的规范不匹配与在估计固定估计量时的近似误差，并根据其隐含规范对代表性估计器进行组织。我们进一步推导出一种局部分解，揭示行为信号、训练信号和反事实参数响应之间的相互作用。控制实验表明，在不同规范下的精确估计量可以导致不同的排名，而近似误差随着扰动远离其线性化点而增加。在噪声标签检测和大语言模型（LLM）归因的实验中，规范选择显著影响归因质量，尤其是在行为替代物的选择上。与行为对齐的规范可以识别被默认基于损失或相似性规范掩盖的特定目标训练样本。这些结果确立了规范分析作为解释和比较数据影响估计器的必要第一步。

View on arXiv Download PDF AI Translation

cs.AI / 71 / 2609.31215

DIAL: Position-Debiased LLM Judges with Adaptive Human Preference Calibration

DIAL：具有自适应人类偏好校准的去位置偏差大型语言模型评判者

Cai, Zesheng, Fan, Yingqi, Chen, Sichang, Du, Jin-Hong

Abstract

Large language models (LLMs) as a judge enable scalable evaluation, but their judgments can be sensitive to response order and, even after removing such position effects, can still diverge systematically from human preferences.We introduce DIAL, a unified framework that combines abundant LLM comparisons with limited human comparisons to separate judge-specific position effects, learn shared structure in position-debiased LLM preferences, and adaptively calibrate that structure toward the human preference target. Theoretically, we study three aspects of DIAL: (i) identification of latent LLM preferences, position effects, and human calibration; (ii) adaptive estimation that balances LLM anchoring against limited human evidence; and (iii) fixed-weight uncertainty quantification for the calibrated human preference. Empirically, we evaluate position debiasing and human alignment separately in controlled simulations and on three human-preference benchmarks, showing that DIAL remains robust to unbalanced response order, achieves strong human-aligned rankings with limited labels, and adapts toward human evidence when LLM information is imperfect. Our real-data study collects over 410K judgments from 21 LLM judges in both display orders, providing a resource for future studies of LLM-judge bias, heterogeneity, and human alignment.

Chinese Translation

大型语言模型（LLMs）作为评判者能够实现可扩展的评估，但它们的判断可能对响应顺序敏感，即使在去除此类位置效应后，仍可能与人类偏好系统性地偏离。我们提出了DIAL，一个统一框架，结合丰富的LLM比较与有限的人类比较，以分离特定评判者的位置信息效应，学习去位置偏差的LLM偏好的共享结构，并自适应地将该结构校准至人类偏好目标。从理论上讲，我们研究了DIAL的三个方面：（i）潜在LLM偏好、位置效应和人类校准的识别；（ii）自适应估计，平衡LLM锚定与有限人类证据；（iii）校准人类偏好的固定权重不确定性量化。在实证方面，我们在受控模拟和三个基于人类偏好的基准上分别评估了去位置偏差和人类对齐，结果表明DIAL对不平衡的响应顺序保持稳健，能够在有限标签下实现强的人类对齐排名，并在LLM信息不完美时向人类证据自适应调整。我们的真实数据研究收集了来自21个LLM评判者的超过410K判断，涵盖两种展示顺序，为未来关于LLM评判者偏见、异质性和人类对齐的研究提供了资源。

View on arXiv Download PDF AI Translation

cs.AI / 72 / 2609.31235

Purin: A Biology-inspired Mechanism for Artificial Neural Networks

Purin：一种受生物启发的人工神经网络机制

Liu, Zishu, Luo, Chunbo, Grecos, Christos

Abstract

Artificial neural networks (ANNs) usually represent neural transmission with fixed trainable weights during a training batch, which omits short-term changes in synaptic efficacy. In addition, the discrete time-step simulation requires additional temporal processing that many conventional ANN architectures do not use. To overcome these challenges, we propose Purin, a biology-inspired and ANN-compatible mechanism, that introduces synaptic efficacy modulation into conventional convolutional neural networks. Purin uses a time-interval-based abstraction for neural activities, which allows Purin to introduce short- and long-term synaptic efficacy changes without using discrete time-steps. Purin introduces a bounded factor to represent temporary synaptic efficacy changes, together with two weight matrices that represent input-side and output-side efficacy. The weight matrices are updated by backpropagation and interpreted as the long-term synaptic efficacy changes. Experimental results show that after removing the confounding factors in the AlexNet, VGG11, and GoogLeNet architectures, Purin improves the classification accuracies in all three models across the evaluated datasets.

Chinese Translation

人工神经网络（ANNs）通常在训练批次中使用固定的可训练权重来表示神经传递，这忽略了突触效能的短期变化。此外，离散时间步的模拟需要额外的时间处理，而许多传统的ANN架构并未使用。为了解决这些挑战，我们提出了Purin，这是一种受生物启发且与ANN兼容的机制，它将突触效能调制引入传统的卷积神经网络中。Purin使用基于时间间隔的抽象来表示神经活动，这使得Purin能够在不使用离散时间步的情况下引入短期和长期的突触效能变化。Purin引入了一个有界因子来表示暂时的突触效能变化，并结合两个权重矩阵，分别表示输入侧和输出侧的效能。这些权重矩阵通过反向传播进行更新，并被解释为长期的突触效能变化。实验结果表明，在去除AlexNet、VGG11和GoogLeNet架构中的混杂因素后，Purin在评估的数据集中提高了所有三个模型的分类准确率。

View on arXiv Download PDF AI Translation

cs.AI / 73 / 2609.31281

MA-WAM: Multi-Agent World-Action Model for Test-Time Planning

MA-WAM：用于测试时规划的多智能体世界-动作模型

Zou, Guowei, Wang, Haitao, Wang, Guoxin, Zhang, Beiwen, Chen, Zhiquan, Wang, Guojie, Wu, Hejun

Abstract

Multi-agent cooperative tasks require different agents to execute a joint action simultaneously, and each agent's action affects both the observations and responses of the other agents. Hence, a world model is needed to predict the team return resulting from the joint actions of all agents. A naive extension directly applies a single-agent world model to each agent's action when predicting the team return step by step. However, such an extension fails to capture the dependencies among the simultaneous actions of multiple agents. We propose Multi-Agent World-Action Model (MA-WAM), a test-time planning framework that enables a frozen multi-agent flow policy to evaluate futures of candidate joint actions. To our knowledge, MA-WAM is the first test-time world-model planner for multi-agent flow policies. MA-WAM predicts the consequences of each joint action according to cross-agent dependencies and enables efficient candidate scoring. Across 30 offline multi-agent reinforcement learning (MARL) settings on MAMuJoCo, SMAC, and MPE, MA-WAM achieves mean relative gains of 22.0% over direct execution and 25.6% over uniform action selection. Under the standard evaluation protocol on an A100 GPU, MA-WAM adds 12.1 ms, accounting for 2.5% of the measured generation-and-scoring time.

Chinese Translation

多智能体协作任务要求不同的智能体同时执行联合动作，并且每个智能体的动作都会影响其他智能体的观察和反应。因此，需要一个世界模型来预测所有智能体联合动作所产生的团队回报。一个简单的扩展是直接将单智能体世界模型应用于每个智能体的动作，以逐步预测团队回报。然而，这种扩展未能捕捉多个智能体同时动作之间的依赖关系。我们提出了多智能体世界-动作模型（MA-WAM），这是一个测试时规划框架，能够使冻结的多智能体流政策评估候选联合动作的未来。据我们所知，MA-WAM 是第一个用于多智能体流政策的测试时世界模型规划器。MA-WAM 根据跨智能体依赖关系预测每个联合动作的后果，并实现高效的候选评分。在 MAMuJoCo、SMAC 和 MPE 上的 30 个离线多智能体强化学习（MARL）设置中，MA-WAM 相较于直接执行实现了 22.0% 的平均相对增益，相较于均匀动作选择实现了 25.6% 的增益。在 A100 GPU 上的标准评估协议下，MA-WAM 增加了 12.1 毫秒，占测量的生成与评分时间的 2.5%。

View on arXiv Download PDF AI Translation

cs.AI / 74 / 2609.31286

G2MAF: Test-Time Gradient Guidance for Multi-Agent Flow Policies

G2MAF：多智能体流策略的测试时梯度引导

Zou, Guowei, Wang, Haitao, Wang, Guoxin, Chen, Zhiquan, Zhang, Beiwen, Wang, Guojie, Wu, Hejun

Abstract

Offline multi-agent reinforcement learning (MARL) learns cooperative policies from fixed datasets without further environment interaction and a learned policy is frozen at deployment. Such a frozen policy typically proposes a single joint action and executes it directly at deployment time. However, this one-shot deployment often commits to a suboptimal proposal, even when better nearby alternatives remain consistent with the behavior data. To address this issue, we propose Gradient Guided Multi Agent Flow (G2MAF), a refinement framework for optimizing joint policies at test-time. G2MAF applies one globally normalized, projected critic gradient to guide and coordinate all agents' corrections while keeping the action both feasible and close to the frozen policy proposal. Across 24 MPE and SMAC settings, its canonical variant improves 20 frozen settings, with mean relative gains of 9.2% on MPE and 8.9% on SMAC, with model inference latency increased by about 6% only.

Chinese Translation

离线多智能体强化学习（MARL）从固定数据集中学习合作策略，而无需进一步的环境交互，并且在部署时学习到的策略是冻结的。这种冻结的策略通常在部署时提出一个单一的联合动作并直接执行。然而，这种一次性部署往往会承诺一个次优的提议，即使在行为数据一致的情况下，仍然存在更好的邻近替代方案。为了解决这个问题，我们提出了梯度引导多智能体流（G2MAF），这是一个在测试时优化联合策略的精炼框架。G2MAF应用一个全局归一化的、投影的评论家梯度，以引导和协调所有智能体的修正，同时保持动作既可行又接近冻结策略的提议。在24个MPE和SMAC设置中，其标准变体改善了20个冻结设置，在MPE上平均相对增益为9.2%，在SMAC上为8.9%，模型推理延迟仅增加约6%。

View on arXiv Download PDF AI Translation

cs.AI / 75 / 2609.31341

The Right Information Extraction Pipeline Depends on the Document: Accuracy-Energy Trade-offs for Small, Local Models

合适的信息提取流程依赖于文档：小型本地模型的准确性与能耗权衡

Walser, Christoph, Argerich, Mauricio Fadel, Fürst, Jonathan

Abstract

Whether an information extraction pipeline should process page images or parsed text depends on the document, and the answer flips across the layout spectrum. We study this trade-off under a constraint that rules out (closed) cloud services: privacy-sensitive documents processed on-premise by small ($\le 8\mathrm{B}$ parameter) text-only and vision--language models, evaluated on both accuracy and energy over a design space spanning input representation, model family, and inference configuration. Benchmarking on the near-plain-text Kleister-NDA contracts and the layout-rich VRDU forms, we find that batching is the dominant energy lever, cutting energy per page by 38-85% at no cost in accuracy, while FP8 quantization saves 27-32% when requests are served one at a time but less than 1mWh per page (9-19%) once batching is applied. Preprocessing dominates what remains: neural OCR costs $17\times$ more energy per page than classical OCR and never reaches the Pareto frontier. Which representation wins flips with the type of document: vision--language models on layout-rich documents and small text-only models with a cheap parser on near-plain text, where they are both more accurate and cheaper than any vision--language configuration. Our work yields concrete guidelines for energy-efficient, privacy-compliant local information extraction.

Chinese Translation

信息提取流程应处理页面图像还是解析文本取决于文档类型，这一答案在布局光谱上有所变化。我们在一个排除了（封闭）云服务的约束下研究了这一权衡：隐私敏感的文档由小型（$ ext{≤} 8 ext{B}$ 参数）文本和视觉-语言模型在本地处理，并在输入表示、模型类别和推理配置的设计空间中评估准确性和能耗。在接近纯文本的Kleister-NDA合同和布局丰富的VRDU表单上进行基准测试，我们发现批处理是主要的能耗杠杆，每页能耗降低38-85%，而准确性没有损失；而FP8量化在请求逐一处理时节省27-32%的能耗，但一旦应用批处理，每页节省的能耗不足1mWh（9-19%）。预处理在剩余能耗中占主导地位：神经OCR每页的能耗是经典OCR的17倍，并且从未达到帕累托前沿。哪种表示方式获胜取决于文档类型：在布局丰富的文档上，视觉-语言模型表现更佳，而在接近纯文本的情况下，小型文本模型配合廉价解析器则更准确且成本更低，优于任何视觉-语言配置。我们的研究为能效高、符合隐私要求的本地信息提取提供了具体指导。

View on arXiv Download PDF AI Translation

cs.AI / 76 / 2609.31354

Mutable Transcripts: Mitigating Context Pollution through Editable Conversation State

可变转录：通过可编辑的对话状态减轻上下文污染

Barry, Dan, Hines, Andrew

Abstract

Contemporary large language model (LLM) chat systems treat conversation history as an immutable sequence of turns that defines the model's working context. However, user intent in real interactions is not static: it evolves through correction, refinement, and shifting constraints. This mismatch between dynamic intent and static transcripts can result in context pollution, where outdated or irrelevant information persists and continues to influence subsequent responses. We introduce mutable transcripts, a new interaction paradigm that enables users to revise prior turns through natural language edit requests, allowing the conversation history itself to be updated rather than appended. This reframes the transcript from a passive record into an editable representation of conversational state. We present a working prototype that integrates transcript-level revision into a standard chat interface and evaluate its feasibility through a controlled user study (n=17) and an illustrative transcript analysis of representative interaction scenarios. Participants significantly preferred mutable transcripts over standard chat across measures of clarity, confidence, and ease of use, with reduced intent to restart conversations. Transcript analysis of representative user study conversations shows that mutable transcripts can reduce conversation length and eliminate obsolete retained context. These findings provide initial evidence that user-driven revision of conversational history can improve interaction quality and help maintain a more current representation of user intent. The source code and prototype can be accessed at https://github.com/QxLabIreland/ReChat

Chinese Translation

当代大型语言模型（LLM）聊天系统将对话历史视为一个不可变的回合序列，这定义了模型的工作上下文。然而，用户在真实交互中的意图并不是静态的：它通过纠正、细化和不断变化的约束而演变。这种动态意图与静态转录之间的不匹配可能导致上下文污染，即过时或不相关的信息持续存在并继续影响后续响应。我们提出了可变转录，这是一种新的交互范式，使用户能够通过自然语言编辑请求修订先前的回合，从而允许对话历史本身被更新而不是附加。这将转录从一个被动记录重新构建为一个可编辑的对话状态表示。我们展示了一个工作原型，该原型将转录级修订集成到标准聊天界面中，并通过一项受控用户研究（n=17）和对代表性交互场景的转录分析评估其可行性。参与者在清晰度、自信心和易用性等指标上显著偏好可变转录，相较于标准聊天，重新开始对话的意图降低。对代表性用户研究对话的转录分析表明，可变转录可以减少对话长度并消除过时的保留上下文。这些发现提供了初步证据，表明用户驱动的对话历史修订可以改善交互质量，并帮助维持用户意图的更当前表示。源代码和原型可以在 https://github.com/QxLabIreland/ReChat 获取。

View on arXiv Download PDF AI Translation

cs.AI / 77 / 2609.31360

Programs-of-Layers in LLMs through the Lens of Cortical Areas

通过皮层区域的视角看大语言模型中的层程序

Westerhoff, Justus, Olbrich, Stephan, Oraby, Hatem, Larkum, Matthew Evan, Gers, Felix Alexander

Abstract

Inference in LLMs is conventionally a fixed-depth, fixed-order forward pass through every layer, regardless of how difficult the input is. The human brain does not work this way: using the thalamus as a central hub, it routes information flexibly to all regions of the cortex according to demand. Li et al. (2026) recently showed, with a system they call program-of-layers (PoLar), that transformers can be given an analogous flexibility if their layers are treated as a library of functions rather than a fixed sequence. Performance improves over the standard forward pass when each input is dynamically routed through an adaptive sequence of skipped or repeated contiguous layer blocks. We reconstructed PoLar's diagnostic MCTS in more detail than the original paper and applied it across 5 models. We reproduced several of PoLar's findings: skipping outperformed the standard pass, repeating outperformed skipping, and combining both outperformed either alone. Shorter programs sufficed for easier questions, while harder questions required more layer repeats. However, we failed to replicate the main claim regarding their learned router for single-shot inference: its top-ranked prediction consistently collapsed back to the standard pass, even though its top-k predicted programs, taken together, did show a real accuracy gain. Beyond reproduction, we find that a small number of generic programs are enough to solve most of the questions. We also provide a much deeper analysis of these programs' structure and robustness: for example, we found that programs that correct errors are highly brittle: undoing even a single edit inside a program typically breaks the correction. Connecting this to the brain's routing mechanisms, PoLar mirrors principles of thalamo-cortical coordination between cortical-area-like transformer layers. We publicly release the code at https://datexis.github.io/RE-PoLar/

Chinese Translation

在大语言模型（LLMs）中，推理通常是通过每一层进行固定深度、固定顺序的前向传播，而不考虑输入的难度。然而，人脑并不是这样工作的：它通过丘脑作为中心枢纽，根据需求灵活地将信息路由到皮层的所有区域。Li等人（2026）最近展示了一个他们称之为层程序（program-of-layers，PoLar）的系统，表明如果将变换器的层视为功能库而不是固定序列，则可以赋予其类似的灵活性。当每个输入通过自适应的跳过或重复的连续层块动态路由时，性能优于标准前向传播。我们对PoLar的诊断性蒙特卡罗树搜索（MCTS）进行了比原始论文更详细的重建，并在5个模型上进行了应用。我们重现了PoLar的几个发现：跳过层优于标准传播，重复层优于跳过层，而两者结合的效果优于单独使用任一方法。对于简单问题，较短的程序就足够，而较难的问题则需要更多的层重复。然而，我们未能复制关于其单次推理学习路由器的主要声明：其排名最高的预测始终回归到标准传播，尽管其前k个预测程序的组合确实显示出真实的准确性提升。超越重现，我们发现少量通用程序足以解决大多数问题。我们还对这些程序的结构和鲁棒性进行了更深入的分析：例如，我们发现纠正错误的程序非常脆弱：在程序内部进行的任何单一编辑通常会破坏纠正功能。将此与大脑的路由机制联系起来，PoLar反映了皮层区域类变换器层之间丘脑-皮层协调的原则。我们在 https://datexis.github.io/RE-PoLar/ 上公开发布了代码。

View on arXiv Download PDF AI Translation

cs.AI / 78 / 2609.31381

Completed Pairs Hide Capped Failures: A ReVerPi Case Study of Selective Context Projection

完成对的隐藏限制性失败：ReVerPi选择性上下文投影案例研究

Zhang, Guangzhe

Abstract

Context projection replaces older tool observations with compact, addressable excerpts, reducing repeated input while potentially adding evidence-retrieval turns. We study this trade-off in ReVerPi, a Pi extension with archived observations and matched full/projected continuations. In an 86-run source-reading campaign with 641 model requests, the 15 completed pairs show identical success: 12/15 per arm. Twelve further boundary runs stop, with the runner suppressing the companion whenever the first arm fails to complete. Restoring all 27 boundary runs bounds projected-minus-full success between $-$9 and +1 tasks. One omitted, selector-chosen projected continuation successfully retrieves archive text yet exhausts twelve requests; its full counterpart answers in three. The eleven jointly correct pairs form a fully observed success stratum within this recorded frame: projection reduces aggregate logical tokens by 25%, while increasing the median pair's tokens by 29% and total suffix requests from 35 to 55. Separating fitting from evaluation changes the selector's apparent tie: outside its four fitting pairs, it incurs one extra failure and 8.6% more logical tokens over thirteen comparable runs. This methodological case study connects stopping rules, known bounded failures, unexecuted companions, and resource aggregation. Its findings concern the recorded campaign, rather than population noninferiority or superiority over unrestricted Pi. Evaluations should retain every intervention boundary, execute both allocated arms independently of the first arm's completion, and report completion alongside interaction and token expenditure.

Chinese Translation

上下文投影用紧凑的、可寻址的摘录替代旧的工具观察，减少重复输入，同时可能增加证据检索的轮次。我们在ReVerPi中研究这一权衡，ReVerPi是一个具有归档观察和匹配完整/投影延续的Pi扩展。在一次包含641个模型请求的86次源阅读活动中，15对完成的对显示出相同的成功率：每个臂12/15。另有12次边界运行停止，运行者在第一个臂未能完成时抑制伴随者。恢复所有27次边界运行将投影减去完整的成功界限限制在-9到+1个任务之间。一个被省略的、选择器选择的投影延续成功检索了归档文本，但耗尽了十二个请求；其完整对应在三个请求中得到了回答。这十一对共同正确的对在此记录框架内形成了一个完全观察到的成功层次：投影将总逻辑标记减少了25%，同时将中位对的标记增加了29%，并将总后缀请求从35增加到55。将拟合与评估分开改变了选择器的明显平局：在其四个拟合对之外，它在十三次可比较的运行中多遭遇一次失败，并增加了8.6%的逻辑标记。该方法案例研究连接了停止规则、已知的有界失败、未执行的伴随者和资源聚合。其发现涉及记录的活动，而非在人口非劣性或优越性上超越不受限制的Pi。评估应保留每个干预边界，独立于第一个臂的完成执行两个分配臂，并报告完成情况以及交互和标记支出。

View on arXiv Download PDF AI Translation

cs.AI / 79 / 2609.31430

Compress What You See, Not What You Say: Anchored Context Distillation for Latent-Observation Software Engineering Agents

压缩你所见，而非你所言：锚定上下文蒸馏用于潜在观察的软件工程代理

Zou, Zhensheng, Wang, Guoqing, Hao, Dan

Abstract

Tool observations dominate the context of software-engineering agents, making long interaction histories costly to maintain. Existing context compression methods can discard information needed by later actions, while adapting agents to soft-token representations can compromise their original behavior. To reduce context while preserving action-critical information and agent behavior, we combine Latent Observations, Hard Actions (LOHA), a context layout that separates compressed history from text needed for exact reference, with Anchored Context Distillation (ACD), a training method that enables latent reading while constraining behavioral drift. LOHA compresses older tool observations into soft tokens while retaining the agent's own turns and the last K observations in text, providing compact access to historical information and exact access to recent content. To enable the agent to use this representation, ACD distills the base model's full-text predictions into the latent view while anchoring its behavior on plain-text inputs to the same base model. On SWE-bench Verified, K=3 reduces context per call by 43% for Qwen3-4B and 57% for SWE-Master-4B-RL, with resolve rates of 12.1% and 21.8% versus 14.5% and 27.5% for their uncompressed bases. A single-run recency sweep reaches 14.4% and 23.0% at K=8, with larger windows generally favoring task performance over compression. Under a 32K-token limit, Qwen3 with K=3 resolves 21.1% of a 199-instance subset versus 11.1% for the same adapted agent using full text. In concurrent single-GPU serving, it achieves 1.9 times that full-text agent's instance throughput.

Chinese Translation

工具观察主导了软件工程代理的上下文，使得长时间交互历史的维护成本高昂。现有的上下文压缩方法可能会丢弃后续动作所需的信息，而将代理适应于软令牌表示可能会妨碍其原有行为。为了在减少上下文的同时保留对动作至关重要的信息和代理行为，我们结合了潜在观察与硬动作（Latent Observations, Hard Actions，LOHA），这是一种将压缩历史与需要精确引用的文本分开的上下文布局，和锚定上下文蒸馏（Anchored Context Distillation，ACD），这是一种在限制行为漂移的同时实现潜在阅读的训练方法。LOHA将较旧的工具观察压缩为软令牌，同时保留代理的自身回合和最近K条观察的文本，从而提供对历史信息的紧凑访问和对近期内容的精确访问。为了使代理能够使用这种表示，ACD将基础模型的全文本预测蒸馏到潜在视图中，同时将其行为锚定于同一基础模型的纯文本输入。在SWE-bench Verified上，K=3使得Qwen3-4B每次调用的上下文减少43%，SWE-Master-4B-RL减少57%，其解决率分别为12.1%和21.8%，而未压缩基线的解决率为14.5%和27.5%。单次运行的近期扫描在K=8时达到了14.4%和23.0%，较大的窗口通常更有利于任务性能而非压缩。在32K令牌限制下，Qwen3在K=3时解决了199个实例子集的21.1%，而使用全文本的同一适应代理仅为11.1%。在并发单GPU服务中，其实例吞吐量达到了全文本代理的1.9倍。

View on arXiv Download PDF AI Translation

cs.AI / 80 / 2609.31460

Segment-Level Agentic Topic Modeling for Improved Data Exploration and Resource Efficiency

基于段级代理主题建模的改进数据探索与资源效率

Jang, Myeongjun Erik, Georgiadis, Antonios, Moon, Sae Young, Silavong, Fran

Abstract

Topic modeling is an effective technique for discovering hidden themes within documents and is widely used in text mining and data analysis across a variety of industry sectors. Recently, large language model (LLM)-based topic models have been emerged that prompt LLMs to generate topics then assign the topics to documents, producing more natural and human-readable topics than conventional topic modeling algorithms. However, the nature of topic assignment process causes certain drawbacks, such as the incapability to produce topic distributions over a document, too broad or narrow topics, and high resource consumption, which increases with the number and length of of documents being assigned topics. These issues are particularly critical for industrial applications, which require high-quality, in-depth analysis and the processing of large volumes of documents. In this context, this paper introduces a framework called SeLATM, which addresses these concerns by employing segment-level topic generation and topic refinement through agentic feedback loops. Experimental results on various datasets demonstrate that SeLATM significantly reduces the LLM resources compared to methods based on topic assignment process, while maintaining superior performance.

Chinese Translation

主题建模是一种有效的技术，用于发现文档中的隐藏主题，广泛应用于各行业的文本挖掘和数据分析。最近，基于大型语言模型（LLM）的主题模型应运而生，这些模型促使LLM生成主题并将其分配给文档，从而产生比传统主题建模算法更自然、更易读的主题。然而，主题分配过程的性质导致了一些缺陷，例如无法生成文档的主题分布、主题过于宽泛或狭窄，以及随着被分配主题的文档数量和长度的增加而增加的高资源消耗。这些问题对于需要高质量、深入分析和处理大量文档的工业应用尤为关键。在此背景下，本文提出了一种名为SeLATM的框架，通过采用段级主题生成和通过代理反馈循环进行主题精炼来解决这些问题。对各种数据集的实验结果表明，与基于主题分配过程的方法相比，SeLATM显著减少了LLM资源的消耗，同时保持了卓越的性能。

View on arXiv Download PDF AI Translation

cs.AI / 81 / 2609.31473

Game Arena: Strategic LLM Evaluation in Competitive Environments

游戏竞技场：在竞争环境中对大型语言模型的战略评估

Doerschuk-Tiberi, Bovard, Yan, Yao, Chiu, Justin, Wang, Hann, Chung, Timothy, Plomecka, Martyna, Schultz, John, Lipovetz, Jon, Drazner, Clayton, Zhuang, Yuchen, Hwang, Jaimie, Keating, Nate, Jones, Riley, Lee, Andrew, Kelly, Oran, Gemp, Ian, Aaron, Michael, Prince, Laurel, Larson, Kate, Moser, Jeff, Jobe, Harrison, Woodford, Chad, Liu, Siqi, Wang, Andrew, Chang, Bo, D'Mello, Christopher, Chaleff, Diane, Howard, Addison, Yip, Johnny, Sugnet, Chuck, Gulli, Antonio, O'Connell, Meghan, Cukierski, Will, Tomasev, Nenad, Yeroshenko, Dima, Parekh, Kinjal, Daniel, Roxanne, Lanctot, Marc, Weir, Domino, Dong, Elsa, Hennes, Daniel, Nalubwama, Melissa, Fraser, Robert, Trostle, Ryan, Peng, Jun, Mason, Tom, Hightower, Lloyd, Chukwuka, Chiamaka, Zhai, Yuexiang, Kirk, Phoebe, Su, Yi, Han, Yuting, Ren, Jie, Prichard, Chris, Sharifzadeh, Sahand, Hakimzadeh, Karim, Sterling, DJ, Risdal, Meg, Olszewska, Kate, Xu, Ya, Firat, Orhan, Chen, Minmin

Abstract

We introduce Kaggle Game Arena, an open and ever-expanding platform to evaluate large language models (LLMs) through competitive games. Different from static benchmarks, game arena enables models to play head-to-head matchups in structured environments where the gameplay strength naturally increases as models evolve, preventing performance saturation. This technical report details the infrastructure behind Game Arena and describes the three pilot game environments: Chess, Poker, and Werewolf. These environments span perfect information, imperfect information, and multiplayer game settings, enabling a systematic study of models' strategic planning, adaptation, and robustness under uncertainty. For each game, we provide a detailed description of the environment, evaluation metrics, and results from running full competitions across models. Through robust infrastructure and large-scale ground-truth based evaluation, Game Arena ensures reproducibility, transparency and generalizability to new games and variants over time.

Chinese Translation

我们介绍了Kaggle游戏竞技场，这是一个开放且不断扩展的平台，用于通过竞争游戏评估大型语言模型（LLMs）。与静态基准不同，游戏竞技场使模型能够在结构化环境中进行一对一的对抗比赛，随着模型的演变，游戏实力自然增强，从而防止性能饱和。本文技术报告详细介绍了游戏竞技场背后的基础设施，并描述了三个试点游戏环境：国际象棋（Chess）、扑克（Poker）和狼人杀（Werewolf）。这些环境涵盖了完全信息、不完全信息和多人游戏设置，使得对模型的战略规划、适应性和在不确定性下的稳健性进行系统研究成为可能。对于每个游戏，我们提供了环境的详细描述、评估指标以及在模型间进行完整比赛的结果。通过强大的基础设施和基于大规模真实数据的评估，游戏竞技场确保了可重复性、透明性以及对新游戏和变种的普适性。

View on arXiv Download PDF AI Translation

cs.AI / 82 / 2609.31482

"AI is (not) the new...": A Diagnostic Analogy Framework for Generative AI's Cultural Impacts

“人工智能（并非）是新的……”：生成性人工智能文化影响的诊断类比框架

Qadri, Rida, Prabhakaran, Vinodkumar, Denton, Remi

Abstract

Generative AI is reshaping the cultural infrastructures through which knowledge is found, synthesized, and held accountable. To make sense of this shift, scholars and policymakers reach for historical analogies of technologies such as the printing press, steam power or electricity. But these comparisons are typically imprecise about which property of the technology carries the comparison, and imprecise analogies produce imprecise governance by designing interventions against the wrong property of the system. This paper offers a diagnostic framework for analyzing how generative AI can transform epistemic and cultural practice. This paper offers a diagnostic framework for analyzing how generative AI can transform epistemic and cultural practice. We decompose each intervention into three coordinates: the epistemic site at which a technology acts, the governing logic by which it organizes its object, and the technical mechanism through which the logic is instantiated. This framework allows us to distinguish between structural cultural consequences, which follow from the mechanism itself, from contingent ones, which remain open to design and institutional choice. Applying the framework to information discovery and knowledge synthesis, we show how the shift from indexicality to inference and from editorial authority to statistical consensus produces specific, traceable cultural effects and reveals governance levers that gestalt analogy obscures.

Chinese Translation

生成性人工智能正在重塑知识发现、综合和问责的文化基础设施。为了理解这一转变，学者和政策制定者借鉴了印刷机、蒸汽动力或电力等技术的历史类比。然而，这些比较通常对技术的哪些属性进行比较不够精确，而不精确的类比会导致不精确的治理，因为它们针对系统的错误属性设计干预措施。本文提供了一个诊断框架，用于分析生成性人工智能如何转变认识论和文化实践。我们将每个干预措施分解为三个坐标：技术作用的认识论场所、组织其对象的治理逻辑，以及通过何种技术机制实现该逻辑。该框架使我们能够区分结构性文化后果（源于机制本身）与偶然性后果（仍然开放于设计和制度选择）。通过将该框架应用于信息发现和知识综合，我们展示了从指示性到推理、从编辑权威到统计共识的转变如何产生特定的、可追踪的文化效应，并揭示出被整体类比所掩盖的治理杠杆。

View on arXiv Download PDF AI Translation

cs.AI / 83 / 2609.31491

UQ-LOB: Uncertainty-Aware Limit Order Book Mid-Price Forecasting

UQ-LOB：基于不确定性意识的限价单簿中间价格预测

Manoharan, Derrick Gilchrist Edward, Linna, Eljas, Baltakys, Kestutis, Dong, Hao, Kanniainen, Juho

Abstract

Forecasting short-horizon mid-price movements from limit order book (LOB) data is central to algorithmic trading, yet most deep LOB forecasters are point predictors: they output a direction or a displacement, but never indicate which of their forecasts can be trusted. We introduce UQ-LOB, a lightweight, encoder-agnostic uncertainty quantification module that attaches to any pretrained LOB encoder and, in the spirit of attentive neural processes, conditions each forecast on a context set of recently completed windows whose outcomes are already realised. The UQ-regression variant outputs a calibrated Gaussian over the future tick displacement, while the UQ-classification variant outputs a categorical distribution over down/up/stationary. Both expose a scalar confidence (predicted signal-to-noise ratio or class probability) that supports selective prediction. On 5.2 billion LOB events across seven cryptocurrency assets and horizons of 5, 10 and 15 seconds, UQ-regression attains near-nominal 68% interval coverage, and restricting to the most confident 10% of predictions raises directional macro F1 by 0.11-0.15 for UQ-regression and 0.05-0.11 for UQ-classification, at every horizon. On large, economically meaningful moves, the tightest confidence tier reaches a directional F1 of 0.88 (down) and 0.83 (up) at the 5-second horizon.

Chinese Translation

从限价单簿（LOB）数据中预测短期中间价格波动是算法交易的核心，然而大多数深度LOB预测模型都是点预测器：它们输出一个方向或位移，但从不指明其预测中哪些是可信的。我们提出了UQ-LOB，一个轻量级、编码器无关的不确定性量化模块，可以附加到任何预训练的LOB编码器上，并且在关注神经过程的精神下，将每个预测条件化于一组最近完成的窗口的上下文集，这些窗口的结果已经实现。UQ回归变体输出一个经过校准的高斯分布，表示未来的价格位移，而UQ分类变体则输出一个关于下跌/上涨/静止的分类分布。两者都展示了一个标量置信度（预测的信噪比或类别概率），以支持选择性预测。在跨越七种加密货币资产和5、10、15秒的时间范围内的52亿个LOB事件中，UQ回归实现了接近名义的68%区间覆盖率，并且将预测限制在最有信心的10%中，使得UQ回归的方向性宏观F1提高了0.11-0.15，而UQ分类的提高幅度为0.05-0.11，适用于每个时间范围。在大规模、经济意义重大的价格波动中，最紧密的置信层在5秒的时间范围内达到了方向性F1为0.88（下跌）和0.83（上涨）。

View on arXiv Download PDF AI Translation

cs.AI / 84 / 2609.31505

Prompt Minimization: Reducing Input Redundancy Without Sacrificing Output Fidelity

提示最小化：在不牺牲输出保真度的情况下减少输入冗余

Juston, Marius F. R., Karim, Kevin A., Gao, Jonathan, Li, Kevin C., Bashambu, Rudhi

Abstract

Despite the growing capabilities of large language models (LLMs), prompt design remains largely heuristic and ad hoc. This project will explore $\textit{prompt minimization}$, the process of reducing prompts to their smallest, most information-dense form while preserving output fidelity. Practically, shorter prompts reduce computational overhead and inference latency, especially when large contexts, such as entire documents or codebases, are included unnecessarily. Further, longer prompts can damage LLM reasoning and accuracy. Theoretically, the existence of multiple prompts yielding equivalent outputs suggests a high degree of redundancy in the input space, raising fundamental questions about what information is essential to elicit specific model behaviors. We propose three variant frameworks to identify and evaluate minimal prompts and demonstrate that minimal prompts often produce outputs comparable to those of their longer counterparts. These findings suggest new directions for efficient prompt engineering and deepen our understanding of input compression in LLMs.

Chinese Translation

尽管大型语言模型（LLMs）的能力不断增强，提示设计仍然主要依赖于启发式和临时方法。本项目将探讨$ extit{提示最小化}$，即在保持输出保真度的同时，将提示减少到其最小、信息密度最高的形式。从实践角度来看，较短的提示可以减少计算开销和推理延迟，尤其是在包含不必要的大上下文（如整个文档或代码库）时。此外，较长的提示可能会损害LLM的推理能力和准确性。从理论上讲，存在多个产生等效输出的提示，表明输入空间中存在高度冗余，这引发了关于哪些信息对于引发特定模型行为是必要的基本问题。我们提出了三种变体框架来识别和评估最小提示，并证明最小提示通常能够产生与其较长对应物相媲美的输出。这些发现为高效的提示工程指明了新的方向，并加深了我们对LLMs中输入压缩的理解。

View on arXiv Download PDF AI Translation

cs.AI / 85 / 2609.31544

A Flow Matching Framework for Neural Representational Dissimilarity

神经表征不相似性的流匹配框架

Ye, Zeyuan, Wei, Xue-Xin

Abstract

Neural representational dissimilarity quantifies differences between neural response distributions, and is essential for comparing neural codes across stimuli, brain areas, tasks, and models. Commonly used distance metrics involve different assumptions and are estimated with separate methods. Here, we show that a variety of distance metrics can be unified under a flow matching framework developed in deep generative models. That is, these distances arise as Jeffreys divergences under different velocity constraints. We find that flow matching has advantages for estimating distances involving complicated distributions and continuous variables. Furthermore, this framework enables the design of new distance metrics in a principled way. Together, flow matching provides a unified approach for understanding, estimating, and designing neural representational dissimilarity metrics.

Chinese Translation

神经表征不相似性量化神经反应分布之间的差异，对于比较不同刺激、脑区、任务和模型的神经编码至关重要。常用的距离度量涉及不同的假设，并采用不同的方法进行估计。在这里，我们展示了多种距离度量可以在深度生成模型中统一为流匹配框架。也就是说，这些距离在不同的速度约束下表现为杰弗里斯散度（Jeffreys divergences）。我们发现，流匹配在估计涉及复杂分布和连续变量的距离方面具有优势。此外，该框架以原则性的方法设计新的距离度量。总之，流匹配为理解、估计和设计神经表征不相似性度量提供了一种统一的方法。

View on arXiv Download PDF AI Translation

cs.AI / 86 / 2609.31563

Multi-agent Scaling Across Disjunctive and Compensatory Tasks

跨离散与补偿任务的多智能体扩展

Fortuna, Carolina, Bertalanic, Blaz

Abstract

Multi-agent LLM systems are often expected to improve as team size increases, yet the scaling behavior may depend on task structure. Our central contribution is to introduce Steiner's taxonomy of group tasks as a framework for analyzing multi-agent LLM scaling and focusing the analysis on disjunctive and compensatory tasks. We model independently sampled agents as conditionally independent given the item, which yields their large-team limits: plurality voting converges to the model's modal answer, and averaging converges to the model's item-level bias. Across selected representative benchmarks, 13 open-weight models, and teams of up to 30 agents, we find qualitatively different scaling behavior. On disjunctive tasks, the probability that at least one agent is correct grows by 5-20 points with team size, but plurality voting over agents that answer directly realises almost none of this potential, as the model predicts to within 0.5 points on average. Multi-round revision raises accuracy considerably, yet the gain is nearly the same with one peer as with 29. In contrast, scaling provides little benefit on Fermi estimation, despite its natural suitability for aggregation: item-level biases shared across the samples of a model account for about 87% of the squared error, so averaging reduces error by only about 6%. Combining model families helps on Fermi estimation but does not surpass the strongest member on disjunctive tasks. These results show that task structure, together with the mechanism combining member outputs, is a fundamental determinant of team scaling.

Chinese Translation

多智能体大语言模型（LLM）系统通常被期望随着团队规模的增加而改善，然而扩展行为可能依赖于任务结构。我们的核心贡献是引入Steiner的群体任务分类法作为分析多智能体LLM扩展的框架，并将分析重点放在离散任务和补偿任务上。我们将独立采样的智能体建模为在给定项目的条件下独立，这样可以得出它们的大团队极限：多数投票收敛于模型的众数答案，而平均值收敛于模型的项目级偏差。在选定的代表性基准、13个开放权重模型以及最多30个智能体的团队中，我们发现扩展行为存在质的差异。在离散任务中，至少有一个智能体正确的概率随着团队规模的增加而增长5-20个百分点，但对直接回答的智能体进行多数投票几乎没有实现这种潜力，因为模型的预测平均在0.5个百分点以内。多轮修订显著提高了准确性，但与一个同伴相比，增益几乎相同，尽管有29个同伴。相比之下，尽管Fermi估计自然适合聚合，但扩展对其的益处不大：模型样本之间共享的项目级偏差约占平方误差的87%，因此平均仅能将误差减少约6%。结合模型家族在Fermi估计上有所帮助，但在离散任务上并未超过最强成员。这些结果表明，任务结构以及组合成员输出的机制是团队扩展的基本决定因素。

View on arXiv Download PDF AI Translation

cs.AI / 87 / 2609.31568

DeepEdu-v1: Efficient and Scalable Agentic LLMs for Vietnamese Education

DeepEdu-v1：适用于越南教育的高效可扩展智能大型语言模型

Nguyen, Quang, Nguyen, Hieu, Hoang, Hien, Pham, Toan, Tran, Cong, Vu, Nam

Abstract

AI tutoring could markedly improve learning outcomes for students in developing regions such as Vietnam, yet the two obvious paths both fall short. Cloud assistants such as ChatGPT route sensitive student data to foreign servers---violating data-sovereignty laws such as Vietnam's Decree 53---and, pre-trained on Western-centric corpora, are not organized around the national textbook curriculum, so their knowledge of local content is unsystematic and frequently hallucinated. Self-hosting an open model keeps data on-premise but hits a two-fold wall: post-training quantization (AWQ, GPTQ) tames the static weight footprint, yet the dynamic KV cache and prefill latency of long tutoring contexts still cause out-of-memory failures and slow responses on consumer GPUs, while the model keeps hallucinating on region-specific material. We present DeepEdu-v1, an AI-tutoring system for Vietnamese education built on SCALE (Self-improving Context-Aware Learning Engine), a framework with two innovations. First, a long-context inference engine amortizes token selection from per-sub-chunk to per-cluster granularity; on long-context retrieval it issues x7.7 fewer retrieval calls than a state-of-the-art selective-attention baseline, cutting prefill latency (TTFT) by roughly 35% while matching or improving task accuracy. Second, a self-improving agentic layer continuously curates a verified playbook from past interactions instead of fine-tuning, a design intended to progressively reduce reliance on dominant-language priors as trustworthy local knowledge accumulates. In its deployed configuration, DeepEdu achieves a nearly x2 TTFT speedup over standard vLLM serving and lifts agentic accuracy from 70.0% to 79.5% on complex tasks, with the strongest per-track gains across financial-reasoning and interactive-agent benchmarks.

Chinese Translation

人工智能辅导可以显著改善发展中国家（如越南）学生的学习成果，但现有的两条明显路径均存在不足。云助手（如ChatGPT）将敏感的学生数据传输到外国服务器——违反了越南的《第53号法令》等数据主权法律——而且这些助手在以西方为中心的语料库上进行预训练，未围绕国家教材课程进行组织，因此对本地内容的知识缺乏系统性，且经常出现虚假信息。自托管开放模型可以将数据保留在本地，但面临双重障碍：后训练量化（AWQ, GPTQ）虽能降低静态权重占用，但长时间辅导上下文的动态KV缓存和预填充延迟仍导致消费者GPU上的内存溢出和响应缓慢，同时模型在特定区域材料上仍然会出现虚假信息。我们提出了DeepEdu-v1，一个基于SCALE（自我改进的上下文感知学习引擎）的越南教育AI辅导系统，该框架具有两项创新。首先，长上下文推理引擎将令牌选择的粒度从每个子块降低到每个集群；在长上下文检索中，它发出的检索调用比最先进的选择性注意基线少7.7倍，预填充延迟（TTFT）减少约35%，同时任务准确性保持或提高。其次，自我改进的智能层不断从过去的互动中策划经过验证的操作手册，而不是进行微调，这一设计旨在随着可信本地知识的积累，逐步减少对主导语言先验的依赖。在其部署配置中，DeepEdu在标准vLLM服务上实现了近2倍的TTFT加速，并将复杂任务的智能准确率从70.0%提升至79.5%，在财务推理和交互代理基准测试中实现了最强的逐轨增益。

View on arXiv Download PDF AI Translation

cs.AI / 88 / 2609.31619

Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Improves Reasoning Efficiency

无需学习停止而学习停止：自监督信心训练提高推理效率

Hosseini, Parsa, Tigalappanavara, Akasha, Nawathe, Sumit, Fan, Chenrui, Basu, Sourya, Winata, Genta Indra, Das, Anirban, Feizi, Soheil, Chitsazan, Nima

Abstract

Reasoning models often generate very long reasoning traces, making inference computationally expensive. Existing approaches typically improve efficiency either through inference-time early-stopping mechanisms or by explicitly encouraging shorter reasoning during training, for example through reinforcement learning with length penalties. We show that substantial efficiency gains can instead emerge from a different kind of supervision: \textit{confidence}. Using a self-supervised procedure, we fine-tune reasoning models to predict their confidence in the answer at intermediate points along their own reasoning trajectories using only 600 training problems. Confidence is used only as a training target: the loss contains no objective for reasoning length, efficiency, or stopping. At inference, the fine-tuned models use the standard generation procedure, with no confidence elicitation or early-stopping mechanism. Despite this, self-supervised confidence fine-tuning makes reasoning more efficient, reducing generated tokens by up to 25\% at matched accuracy across Gemma, Qwen, Nemotron, and GPT-OSS models on mathematical, scientific, and coding reasoning benchmarks, with efficiency gains comparable to methods that explicitly optimize for shorter reasoning. Analysis of reasoning episodes further shows that confidence supervision largely preserves the base models' high-level reasoning composition rather than selectively suppressing particular behaviors. Our results suggest that efficient reasoning may emerge as a downstream consequence of learning metacognitive signals, without being directly optimized.

Chinese Translation

推理模型通常生成非常长的推理轨迹，使得推理计算成本高昂。现有方法通常通过推理时的提前停止机制或通过在训练过程中明确鼓励更短的推理来提高效率，例如通过带有长度惩罚的强化学习。我们表明，显著的效率提升可以通过一种不同类型的监督而获得： extit{信心}。通过自监督程序，我们微调推理模型，使其在自身推理轨迹的中间点预测对答案的信心，仅使用600个训练问题。信心仅作为训练目标使用：损失函数中不包含推理长度、效率或停止的目标。在推理时，微调后的模型使用标准生成程序，不进行信心引导或提前停止机制。尽管如此，自监督信心微调使推理更加高效，在数学、科学和编码推理基准上，Gemma、Qwen、Nemotron和GPT-OSS模型在匹配准确率的情况下，生成的标记减少了多达25%，其效率提升与明确优化更短推理的方法相当。对推理过程的分析进一步表明，信心监督在很大程度上保留了基础模型的高层推理组合，而不是选择性地抑制特定行为。我们的结果表明，高效推理可能作为学习元认知信号的下游结果而出现，而不是直接优化的结果。

View on arXiv Download PDF AI Translation

计算语言学 (Computation and Language)

cs.CL / 1 / 2609.30287

A Mechanistic Study of AI-Text Detection Neurons in Frozen BERT: Sparse Probing and Activation Patching on RAID

冻结BERT中AI文本检测神经元的机制研究：在RAID上的稀疏探测和激活修补

Blicharz, Paweł, Grunwald, Miłosz

Abstract

AI-generated text detectors achieve high accuracy on standard benchmarks, yet the internal representations that drive these predictions remain poorly understood. We study which neurons in a frozen BERT-base-uncased encoder support AI-text detection, using the RAID benchmark across six generators spanning pure-base and instruction-tuned models. We apply the L1-to-L2 sparse-probing protocol of Gurnee et al. (2023) to all 9,216 CLS hidden-state dimensions (12 layers x 768), which we call neurons. The procedure recovers a stable set of under 1% of neurons per generator, consistent across folds and seeds; a probe restricted to that set retains most of the full-feature detection accuracy. Bidirectional activation patching confirms this set's causal relevance: in both directions it flips predictions an order of magnitude more often than size-matched random sets. Mean-ablating the same neurons leaves accuracy largely intact; the signal is therefore redundantly distributed. Cross-generator analysis reveals a bipartite structure: instruction-tuned generators concentrate 30-36% of stable neurons in BERT's final layer while both base generators fall below 14%, consistent with a layer-12 footprint of post-training alignment. Leave-one-family-out evaluation shows the selected neurons retain 86-94% of the full-feature ceiling on unseen generator families, so a detector can operate on a small fixed subspace without re-identifying neurons per generator.

Chinese Translation

AI生成文本检测器在标准基准上实现了高准确率，但驱动这些预测的内部表征仍然不甚了解。我们研究了在冻结的BERT-base-uncased编码器中，哪些神经元支持AI文本检测，使用RAID基准对六种生成器进行分析，这些生成器涵盖了纯基础模型和指令调优模型。我们对所有9216个CLS隐状态维度（12层 x 768）应用了Gurnee等（2023）的L1到L2稀疏探测协议，我们称之为神经元。该过程恢复了每个生成器下少于1%的稳定神经元集，在不同的折叠和种子中保持一致；限制在该集合上的探测器保留了大部分完整特征检测的准确性。双向激活修补确认了该集合的因果相关性：在两个方向上，它比大小匹配的随机集合更频繁地翻转预测，数量级上更高。均值消融相同的神经元几乎不影响准确性，因此信号是冗余分布的。跨生成器分析揭示了一个二分结构：指令调优生成器在BERT的最后一层集中30-36%的稳定神经元，而两个基础生成器则低于14%，这与后训练对齐的第12层足迹一致。留一族评估显示，所选神经元在未见生成器族上保留了86-94%的完整特征上限，因此检测器可以在小的固定子空间内操作，而无需针对每个生成器重新识别神经元。

View on arXiv Download PDF AI Translation

cs.CL / 2 / 2609.30288

Manifold Projection and Iterative Autoencoder Refinement for Masked Language Modeling

流形投影与迭代自编码器精炼用于掩码语言建模

Mokhtari, Narges, Haddadi, Farzan, Rezaii, Ebrahim

Abstract

In Transformer-based masked language models, attention is the primary mechanism for context mixing, but there are other ways to mix data across tokens. Recent attention-free mixers replace attention with fixed or hypernetwork-generated MLPs, alternating their dynamic, content-dependent weighting for computational simplicity. We build an alternative that gets the same property from a low-rank bottleneck autoencoder. We replace attention with a stack of autoencoder-based mixing modules, one operating over local neighborhoods, one over the full sequence, and one across attention heads, each compressing and reconstructing its input through a bottleneck, and its width is a hyperparameter rather than a training effect. In masked positions, we introduce an iterative refinement procedure that has two distinct steps. A pulling step that pulls an embedding representation toward a weighted average of its neighbors, and a correcting step that projects the result back to the learned manifold via an autoencoder. Our architecture achieves a significant portion of attention's performance at about $1.9 \times$ fewer FLOPs when pretrained on C4 and evaluated with parameter-matched BERT baselines. Our model equals parameter-matched BERT and TinyBERT baselines on the rarest-token frequency bucket using a frequency-aware training schedule that samples rare tokens more than uniformly for the masking tasks.

Chinese Translation

在基于变换器的掩码语言模型中，注意力是上下文混合的主要机制，但还有其他方法可以在标记之间混合数据。最近的无注意力混合器用固定或超网络生成的多层感知器（MLP）替代了注意力，交替使用其动态、内容依赖的加权以实现计算简化。我们构建了一种替代方案，通过低秩瓶颈自编码器获得相同的特性。我们用一系列基于自编码器的混合模块替代了注意力，其中一个模块在局部邻域上操作，一个模块在整个序列上操作，一个模块在注意力头之间操作，每个模块通过瓶颈压缩和重构其输入，其宽度是一个超参数，而不是训练效果。在掩码位置，我们引入了一种迭代精炼过程，该过程包含两个不同的步骤。一个拉取步骤将嵌入表示拉向其邻居的加权平均，另一个纠正步骤通过自编码器将结果投影回学习到的流形。我们的架构在C4上进行预训练并与参数匹配的BERT基线进行评估时，达到了注意力性能的显著部分，计算量约为$1.9 imes$更少。我们的模型在最稀有标记频率桶上与参数匹配的BERT和TinyBERT基线相等，采用了一种频率感知的训练计划，该计划在掩码任务中对稀有标记的采样频率高于均匀采样。

View on arXiv Download PDF AI Translation

cs.CL / 3 / 2609.30289

Not All Memories Are Equal: Hierarchical Collaborative Memory for Validity-Aware Retrieval in LLM Agents

记忆并非全然相同：针对有效性感知检索的层次化协作记忆在大语言模型代理中的应用

Shi, Yufei, Yao, Rujing, Li, Ang, Wu, Yang, Jiang, Zhuoren, Liu, Xiaozhong

Abstract

In team collaboration scenarios, memory is heterogeneous and continually evolving. Team memories capture collective decisions, protocols, and current consensus, while individual memories preserve member-specific observations, execution traces, and intermediate progress. Existing memory-augmented systems typically retrieve from all stored memories as a flat pool, ranking them by semantic relevance, importance, or recency without modeling hierarchical structure or evolving validity. As a result, they often surface semantically relevant but outdated or conflicting memories, especially individual memories that no longer align with current team consensus, instead of prioritizing currently valid memories. This is particularly problematic when collaborative LLM agents answer user questions, since their responses should be grounded in valid memories. We propose HiCoMER, a framework for hierarchical collaborative memory management and validity-aware retrieval in LLM agents. HiCoMER first maintains the validity of team and individual memories and then retrieves memories that remain valid, rather than retrieving directly from all stored memories. It consists of three components: a Hierarchical Memory Conflict Updater, a Validity-Aware Memory Retriever, and a Memory-Grounded Answer Generator. To evaluate HiCoMER, we construct two new datasets for memory-grounded question answering in collaborative settings. Experiments on both datasets show that HiCoMER consistently outperforms strong baselines by reducing outdated retrieval, preserving current team consensus, and improving downstream QA quality.

Chinese Translation

在团队协作场景中，记忆是异质的并且不断演变。团队记忆捕捉集体决策、协议和当前共识，而个体记忆则保留成员特定的观察、执行轨迹和中间进展。现有的增强记忆系统通常将所有存储的记忆视为一个平坦的池，通过语义相关性、重要性或最近性进行排名，而未建模层次结构或演变有效性。因此，它们往往会浮现出语义相关但过时或相互冲突的记忆，尤其是那些不再与当前团队共识一致的个体记忆，而不是优先考虑当前有效的记忆。这在协作大语言模型代理回答用户问题时尤其成问题，因为它们的回答应该基于有效的记忆。我们提出了HiCoMER，一个用于层次化协作记忆管理和有效性感知检索的框架。HiCoMER首先维护团队和个体记忆的有效性，然后检索仍然有效的记忆，而不是直接从所有存储的记忆中检索。它由三个组件组成：层次记忆冲突更新器（Hierarchical Memory Conflict Updater）、有效性感知记忆检索器（Validity-Aware Memory Retriever）和基于记忆的答案生成器（Memory-Grounded Answer Generator）。为了评估HiCoMER，我们构建了两个新的数据集，用于协作环境中的基于记忆的问题回答。对这两个数据集的实验表明，HiCoMER通过减少过时检索、维护当前团队共识并提高下游问答质量，始终优于强基线。

View on arXiv Download PDF AI Translation

cs.CL / 4 / 2609.30290

Auditing and Repairing LLM-as-Judge Failures in a Production Text-to-SQL Pipeline

审计和修复生产文本到SQL管道中LLM作为评审者的失败

Liu, Haowei, Wu, Hsin-Tai, Fang, Yi

Abstract

Production text-to-SQL pipelines often end with an LLM-as-judge whose agreement with human annotators has never actually been measured. When we checked ours, the deployed gpt-4o-mini judge agreed with two-author gold at only Cohen's kappa = 0.04 on a disagreement-enriched set and 0.42 on a uniform-random spot-check, over-flagging 77.1% of the human-FAITHFUL cases in the enriched set. Most of its over-flags trace back to a single mechanism we call GRADE-HALLUCINATION. A self-hosted Qwen3.6-27B replacement (kappa = 0.72) lands in the same range as Claude Opus 4.7 (kappa = 0.71); the head-to-head is underpowered at n = 96, but for the deployment decision that hardly matters, since Qwen costs roughly 1/300 as much per call. Ensembling does not help for free. Pairing the weak judge with a stronger one degrades agreement, whereas three strong judges under unanimity routing reach kappa = 0.79 at 89.7% auto-coverage. Applied out-of-domain, the same audit recipe flags 25.5% of BIRD-financial's expert-authored gold SQLs as candidate gold-SQL issues under our annotation protocol. Code and pre-registration are at https://github.com/JamesL404/synca-audit.

Chinese Translation

生产文本到SQL管道通常以LLM作为评审者结束，但其与人工标注者的协议从未被实际测量。当我们检查我们的系统时，部署的gpt-4o-mini评审者在一个富含争议的集合中与双作者金标准的Cohen's kappa仅为0.04，而在均匀随机抽查中为0.42，过度标记了77.1%的人工FAITHFUL案例。其大多数过度标记源于我们称之为GRADE-HALLUCINATION的单一机制。自托管的Qwen3.6-27B替代方案（kappa = 0.72）与Claude Opus 4.7（kappa = 0.71）处于相同范围；尽管面对96个样本的对比测试能力不足，但对于部署决策而言，这几乎无关紧要，因为Qwen的每次调用成本大约是1/300。集成并没有带来额外的好处。将弱评审者与强评审者配对会降低一致性，而在一致性路由下，三个强评审者的kappa达到0.79，自动覆盖率为89.7%。在域外应用时，同样的审计方案将BIRD-financial的专家撰写的金标准SQL的25.5%标记为候选金标准SQL问题，依据我们的标注协议。代码和预注册信息可在https://github.com/JamesL404/synca-audit找到。

View on arXiv Download PDF AI Translation

cs.CL / 5 / 2609.30292

A Survey on Fake Review Detection: From Pre-trained Language Models to Large Language Models

假评论检测的调查：从预训练语言模型到大型语言模型

Yang, Fanji, Chen, Huiyao, Yu, Xi, Zhang, Meishan, Xiao, Xiaohong, Deng, Mingsen

Abstract

Online reviews shape consumer decisions, platform governance, and corporate reputation.Fake reviews compromise this information channel by injecting deceptive evidence into rating systems, recommendation pipelines, and public trust mechanisms.The rise of large language models, or LLMs, has changed the problem in two directions.LLMs can generate fluent and context-aware deceptive reviews, while pre-trained language models, or PLMs, and LLMs also provide stronger semantic representations for detection.This survey reviews fake review detection from an information fusion perspective, covering 211 studies published from 2018 to early 2026.We organize existing work by evidence source and fusion level, covering review text, sentiment, rating behavior, temporal metadata, user-product graphs, multimodal content, external knowledge, and LLM-generated signals.We trace the development from traditional machine learning and deep learning to PLM-based and LLM-based methods, and examine how different approaches combine textual, behavioral, structural, and multimodal evidence.We also analyze reported performance trends on widely used Amazon, Yelp, and OpSpam benchmark families, while noting the limitations caused by different label construction procedures, data splits, and evaluation protocols.Finally, we identify open problems in adversarial generation, cross-domain transfer, uncertainty-aware fusion, missing-source robustness, interpretability, and trustworthy evaluation for AI-generated deceptive content.

Chinese Translation

在线评论影响消费者决策、平台治理和企业声誉。假评论通过向评分系统、推荐管道和公众信任机制注入欺骗性证据，破坏了这一信息渠道。大型语言模型（LLMs）的兴起在两个方向上改变了这一问题：LLMs能够生成流畅且具有上下文意识的欺骗性评论，而预训练语言模型（PLMs）和LLMs也为检测提供了更强的语义表示。本调查从信息融合的角度回顾了假评论检测，涵盖了2018年至2026年初发布的211项研究。我们根据证据来源和融合层次组织现有工作，涉及评论文本、情感、评分行为、时间元数据、用户-产品图、跨模态内容、外部知识和LLM生成的信号。我们追踪了从传统机器学习和深度学习到基于PLM和LLM的方法的发展，并考察了不同方法如何结合文本、行为、结构和跨模态证据。我们还分析了在广泛使用的Amazon、Yelp和OpSpam基准系列上报告的性能趋势，同时指出了不同标签构建程序、数据划分和评估协议所造成的局限性。最后，我们识别了在对抗生成、跨领域迁移、不确定性感知融合、缺失源鲁棒性、可解释性和对AI生成的欺骗内容的可信评估等方面的开放问题。

View on arXiv Download PDF AI Translation

cs.CL / 6 / 2609.30293

Cartograph: Federated Tool Discovery with Operator-Attested Retrieval for AI Agents

Cartograph：具有操作员认证检索的联邦工具发现用于人工智能代理

Agyemang, Justice Owusu, Agyare, Michael, Agyekum, Kwame Opuni-Boachie Obour, Agyekum, Kwame Agyeman-Prempeh, Acheampong, Francisca Adoma, Kponyo, Jerry John

Abstract

The Model Context Protocol (MCP) enables AI agents to discover and call tools, but loading every definition becomes expensive as connected catalogs grow. We present Cartograph, a federated MCP proxy that changes agent-visible tool discovery from $O(n)$ catalog traversal to $O(k)$ progressive disclosure. Cartograph combines three mechanisms: (1) operator-attested capability cards, Ed25519-signed descriptions generated under the deploying operator's control rather than ranked publisher copy; (2) Rift, a three-layer confusable-cluster analysis comprising density clustering, query-margin analysis, and token diagnosis; and (3) two-stage retrieval, which ranks servers before tools. On a 22-server, 374-tool deployment, Cartograph exposes three proxy tools instead of 374 definitions. A 49-query author-constructed benchmark yields R@5 of 0.816, compared with 0.592 for a Jaccard keyword baseline, while a measured top-5 discovery exchange uses 475 tokens rather than 42,450 under the stated full-catalog accounting. Rift identifies 49 confusable clusters, including four HIGH-risk clusters in bootstrap-generated cards. An exploratory comparison of 119 LLM-generated descriptions removes the observed zero-distance cluster but shows that mixing card-generation regimes can reduce R@5. Gateway measurements over ten trials add 5ms mean latency (0.8%) relative to direct stdio MCP calls. Cartograph is complementary to code-execution approaches: it controls which tool descriptions are surfaced and records the provenance of the descriptions used for ranking for each query.

Chinese Translation

模型上下文协议（MCP）使人工智能代理能够发现和调用工具，但随着连接目录的增长，加载每个定义变得昂贵。我们提出了Cartograph，一个联邦MCP代理，它将代理可见的工具发现从$O(n)$目录遍历转变为$O(k)$渐进披露。Cartograph结合了三种机制：（1）操作员认证的能力卡片，由Ed25519签名的描述在部署操作员的控制下生成，而不是排名出版商的副本；（2）Rift，一个由密度聚类、查询边际分析和标记诊断组成的三层混淆聚类分析；（3）两阶段检索，在工具之前对服务器进行排名。在一个22个服务器、374个工具的部署中，Cartograph暴露了三个代理工具，而不是374个定义。一个由49个查询构建的基准测试显示R@5为0.816，而Jaccard关键词基线为0.592，同时在测量的前5个发现交换中，使用475个标记，而不是在所述完整目录计算下的42,450个。Rift识别出49个混淆聚类，包括在引导生成的卡片中的四个高风险聚类。对119个LLM生成的描述的探索性比较去除了观察到的零距离聚类，但显示混合卡片生成模式可以降低R@5。经过十次试验的网关测量相对于直接的标准输入输出MCP调用增加了5毫秒的平均延迟（0.8%）。Cartograph与代码执行方法是互补的：它控制哪些工具描述被展示，并记录用于每个查询排名的描述的来源。

View on arXiv Download PDF AI Translation

cs.CL / 7 / 2609.30294

SlideLab: Audience-Centered Scientific Slide Generation and Evaluation

SlideLab：以观众为中心的科学幻灯片生成与评估

Vats, Vidushee, Sharma, Karun, Wang, Yuxia

Abstract

Scientific presentations are more than summaries of research papers. They need to present the work in a coherent sequence, explain the main ideas clearly, and help the audience follow the presentation. We present SlideLab, a training-free multi-agent framework for generating scientific presentations from research papers. SlideLab first plans the presentation narrative, then builds and iteratively refines a shared slide deck using agents for content planning, visual generation, layout refinement, and grounding verification. In a blind human preference study, SlideLab was preferred over both open-source and commercial systems on 77% of papers while using roughly 4 times fewer inference tokens than the strongest open-source baseline. We also introduce ConfArena, an audience-oriented evaluation framework that simulates a conference room and assesses presentations slide by slide. ConfArena matches human system rankings and detects injected presentation problems, including falsified numbers, degraded figures, dropped slides, and shuffled slide order.

Chinese Translation

科学演示不仅仅是研究论文的摘要。它们需要以连贯的顺序呈现工作，清晰地解释主要思想，并帮助观众跟随演示。我们提出了SlideLab，这是一个无培训的多智能体框架，用于从研究论文生成科学演示。SlideLab首先规划演示叙事，然后使用内容规划、视觉生成、布局优化和基础验证的智能体构建并迭代优化共享幻灯片。通过一项盲人偏好研究，SlideLab在77%的论文中优于开源和商业系统，同时使用的推理令牌数量大约是最强开源基线的四分之一。我们还介绍了ConfArena，这是一个面向观众的评估框架，模拟会议室并逐张评估演示。ConfArena与人类系统排名相匹配，并检测注入的演示问题，包括虚假数字、降级图形、遗漏幻灯片和幻灯片顺序混乱。

View on arXiv Download PDF AI Translation

cs.CL / 8 / 2609.30295

SignTrace: Describe a Sign, Find the Word

SignTrace：描述一个手势，找到对应的词汇

Tu, Zengji, Zhu, Xingye, Wang, Ningjing, Huang, Tingyi, Zhu, Yangjunfeng, Wan, Dai

Abstract

Identifying an unfamiliar sign is difficult when a learner remembers its movement but does not know its meaning or formal feature codes. SignTrace addresses this longstanding reverse-lookup problem through natural-language access to a Chinese sign-language dictionary. The system integrates LLM-based dictionary enrichment, action extraction, dictionary-style rewriting, seven-channel retrieval, and candidate reranking over 6,699 entries. It has been deployed for user trials and has received positive informal feedback. Evaluation on a dictionary-derived benchmark of 500 movement-description queries yields 94.0% Hit@1, 97.4% Hit@9, and a mean reciprocal rank of 0.9540. Reranking increases Hit@1 from 71.8% to 94.0%, while component analyses show the contribution of enriched entry descriptions. Median query-processing time is 13.37 seconds with six concurrent queries. By connecting everyday movement descriptions to documented signs and meanings, SignTrace provides a practical tool for identifying unfamiliar signs. Dictionary-derived wording and prior selection within the benchmark limit generalization to descriptions independently produced by users.

Chinese Translation

当学习者记得一个不熟悉的手势的动作但不知道其含义或正式特征代码时，识别该手势变得困难。SignTrace 通过自然语言访问中文手语词典解决了这一长期存在的反向查找问题。该系统整合了基于大语言模型（LLM）的词典丰富化、动作提取、词典风格重写、七通道检索以及对 6,699 个条目的候选重排序。它已被部署进行用户试验，并收到了积极的非正式反馈。在一个基于词典的 500 个动作描述查询的基准评估中，系统达到了 94.0% 的 Hit@1，97.4% 的 Hit@9，以及 0.9540 的平均倒排秩。重排序将 Hit@1 从 71.8% 提高到 94.0%，而组件分析显示了丰富条目描述的贡献。中位查询处理时间为 13.37 秒，支持六个并发查询。通过将日常动作描述与已记录的手势和含义连接起来，SignTrace 提供了一个实用工具，用于识别不熟悉的手势。基于词典的措辞和基准内的先前选择限制了对用户独立生成描述的泛化能力。

View on arXiv Download PDF AI Translation

cs.CL / 9 / 2609.30297

Bootstrapping Conversational Recommendation Agents At Spotify: Synthetic Data Generation and Self-Improvement Loops

在Spotify引导对话推荐代理：合成数据生成与自我改进循环

Palumbo, Enrico, Tamborrino, Alexandre, Ode, Victor, Lacker, Ben, Escoda, Adrià Casas, Hopple, Jeremy, Better, Marcus, Leoni, James, Galvão, Hugo, Bouchard, Hugues, Lalmas, Mounia, García, José Luis Redondo, Abebe, Abenezer, Clifton, Ann, Blomberg, Anton, Lindström, Henrik, Doro, Dani, Cardet, Christine Doig

Abstract

Conversational recommendation agents are a new paradigm for content discovery, enabling users to express complex intents through natural language (e.g., "recommend Italian indie artists I haven't heard before"). A central challenge in building such agents is optimizing agent planning -- deciding how to select, sequence, and invoke tools -- particularly in cold-start settings where real user interactions are not yet available. We introduce a pipeline for multi-turn synthetic data generation and a self-improvement loop to address this challenge. The synthetic data pipeline transforms single-turn prompts into realistic multi-turn conversations, enabling systematic evaluation before launch. The self-improvement loop combines variance-based contrastive optimization with iterative refinement through a coding agent, automatically identifying and fixing planning and tool-use errors. Our approach improves quality by +8% on top of a highly optimized manual prompt. The system has been productionized and significantly accelerated iteration cycles for the launch of a conversational recommendation agent at Spotify. Online A/B tests demonstrate its effectiveness, with +14% user listening, +5% increase in weekly active users, and a 5% reduction in skip rate compared to a prior experience supporting only session refinement. This work provides a practical framework for accelerating the development of conversational recommendation agents in industry.

Chinese Translation

对话推荐代理是一种新的内容发现范式，使用户能够通过自然语言表达复杂的意图（例如，“推荐我之前没有听过的意大利独立艺术家”）。构建此类代理的一个核心挑战是优化代理规划——决定如何选择、排序和调用工具——尤其是在冷启动环境中，真实用户交互尚不可用。我们提出了一种多轮合成数据生成管道和自我改进循环，以应对这一挑战。合成数据管道将单轮提示转换为现实的多轮对话，使得在发布前能够进行系统评估。自我改进循环结合了基于方差的对比优化与通过编码代理的迭代精炼，自动识别并修复规划和工具使用错误。我们的方法在高度优化的手动提示基础上提高了质量8%。该系统已投入生产，并显著加快了Spotify对话推荐代理的发布迭代周期。在线A/B测试证明了其有效性，用户收听量增加14%，每周活跃用户增加5%，跳过率相比于仅支持会话精炼的先前体验降低了5%。这项工作为加速工业界对话推荐代理的开发提供了一个实用框架。

View on arXiv Download PDF AI Translation

cs.CL / 10 / 2609.30298

A Benchmark Framework for Screening Automation in Systematic Reviews

系统评价中筛选自动化的基准框架

Kumar, Gauransh, Marchezan, Luciano, Genois, Guillaume, Delcourt, Kévin, Syriani, Eugene

Abstract

Systematic reviews (SR) are essential for evidence-based research, but their screening phase is highly time-consuming and labor-intensive. Large language models (LLMs) offer a promising opportunity to reduce this workload by assisting with article relevance classification. However, existing evaluation approaches often rely on traditional metrics that may be misleading for highly imbalanced SR screening datasets.This paper presents a benchmark dataset of $45\,064$ labeled entries for evaluating LLM performance in SR screening across 32 curated secondary studies. It proposes an evaluation framework that accounts for class imbalance, i.e., the natural prevalence of excluded articles relative to included articles in SRs. It also introduces PromptSR, a tool designed to support prompt experimentation, experiment management, and result analysis for LLM-based screening. We also present a use case demonstrating the application of SRBench and PromptSR.

Chinese Translation

系统评价（SR）对于基于证据的研究至关重要，但其筛选阶段非常耗时且劳动密集。大型语言模型（LLMs）提供了一个有前景的机会，通过协助文章相关性分类来减少这一工作量。然而，现有的评估方法往往依赖于传统指标，这在高度不平衡的系统评价筛选数据集中可能会产生误导。本文提出了一个包含45,064个标记条目的基准数据集，用于评估LLM在32个经过精心挑选的二次研究中的SR筛选性能。我们提出了一种评估框架，考虑了类别不平衡，即在SR中被排除文章相对于被纳入文章的自然流行程度。本文还介绍了PromptSR，这是一种旨在支持提示实验、实验管理和基于LLM的筛选结果分析的工具。我们还展示了一个用例，演示了SRBench和PromptSR的应用。

View on arXiv Download PDF AI Translation

cs.CL / 11 / 2609.30414

A Unified Account of Concepts and Chunks

概念与块的统一解释

Singaravadivelan, Karthik, Langley, Pat

Abstract

Cognitive psychology has studied how people encode, use, and learn concepts that describe categories, and how they represent, recognize, and acquire chunks for familiar patterns of elements. The literatures on these two topics are nearly disjoint, which poses a challenge for unified theories of cognition. In this paper, we review Cobweb, a computational account of categorization and concept formation, and propose an extended theory that incorporates chunks and their acquisition. The theory makes no commitments about modality, applying to any experience that decomposes into elements and relations among them. We also present \trellis/, an implementation of this theory, and illustrate its application to learning context-free grammars, which we adopt as a testbed because they involve both concept-like and chunk-like elements. In addition, we report experimental results on three synthetic grammars that demonstrate the system's ability to represent syntactic knowledge, use it to parse and generate sentences, and learn compositional structures from sample parses. We conclude by discussing related work on concepts and chunks, along with directions for future research in the area.

Chinese Translation

认知心理学研究了人们如何编码、使用和学习描述类别的概念，以及他们如何表示、识别和获取熟悉元素模式的块。这两个主题的文献几乎是分开的，这对统一的认知理论构成了挑战。本文回顾了 Cobweb，一个关于分类和概念形成的计算模型，并提出了一个扩展理论，结合了块及其获取。该理论不对模态做出承诺，适用于任何可以分解为元素及其关系的经验。我们还展示了 rellis/，该理论的一个实现，并说明其在学习无上下文文法中的应用，我们选择它作为测试平台，因为它涉及概念类和块类元素。此外，我们报告了在三个合成文法上的实验结果，展示了该系统表示句法知识、利用其解析和生成句子以及从样本解析中学习组合结构的能力。最后，我们讨论了与概念和块相关的工作，以及未来研究方向。

View on arXiv Download PDF AI Translation

cs.CL / 12 / 2609.30416

All In Good Time: Causality-Aware Framework for LLM-Based Simultaneous Speech-to-Speech Translation

恰逢其时：一种考虑因果关系的基于大型语言模型的同步语音到语音翻译框架

Hussein, Amir, Albasiri, Enas, Bartley, Travis M., Ferchichi, Nourchene, Hu, Ke, Dubey, Harishchandra, Kim, Myungjong, Chen, Zhehuai, Olabiyi, Oluwatobi, Khudanpur, Sanjeev

Abstract

Large Language Models (LLMs) have shown strong performance in low-resource offline translation; however, extending them to simultaneous speech-to-speech translation (Simul-S2ST) remains challenging due to the scarcity of causally aligned training data with high cross-lingual speaker fidelity. In addition, existing approaches rely on fixed translation policy or confidence heuristics, leading to suboptimal quality and higher latency. We propose a causality-aware Simul-S2ST framework with a novel data pipeline that generates high-fidelity, causally aligned segments with improved voice transfer. The framework introduces (i) a factorized S2ST architecture (FAST), (ii) a causality-aware adaptive policy (CAP), and (iii) causality-aware latency metric. Experiments on CVSS Spanish, German, and French show that FAST-CAP consistently improves the quality-latency trade-off, achieving up to +1.2 BLEU and a 26% relative latency reduction over a fixed policy. Despite using substantially less training data than existing systems, FAST-CAP achieves state-of-the-art results in speech translation quality and speaker fidelity while yielding up to a 38.8% relative reduction in latency.

Chinese Translation

大型语言模型（LLMs）在低资源离线翻译中表现出色；然而，由于缺乏高跨语言说话者保真度的因果对齐训练数据，将其扩展到同步语音到语音翻译（Simul-S2ST）仍然具有挑战性。此外，现有方法依赖于固定的翻译策略或置信度启发式，导致质量不佳和延迟较高。我们提出了一种因果关系感知的Simul-S2ST框架，采用新颖的数据管道生成高保真、因果对齐的片段，并改善语音传输。该框架引入了（i）分解的S2ST架构（FAST），（ii）因果关系感知的自适应策略（CAP），以及（iii）因果关系感知的延迟度量。在CVSS西班牙语、德语和法语上的实验表明，FAST-CAP始终改善了质量与延迟的权衡，较固定策略实现了高达+1.2的BLEU值和26%的相对延迟减少。尽管使用的训练数据远少于现有系统，FAST-CAP在语音翻译质量和说话者保真度方面达到了最先进的结果，同时实现了高达38.8%的相对延迟减少。

View on arXiv Download PDF AI Translation

cs.CL / 13 / 2609.30439

Inference-Time Target Speaker Unlearning in LLM-Based Automatic Speech Recognition

基于大语言模型的自动语音识别中的推理时目标说话者遗忘

Su, Bo, Yan, Yueru, Le, Thai

Abstract

We introduce target-speaker unlearning ASR (TSU-ASR) task in a fully end-to-end framework for multi-speaker ASR and diarization. Given a multi-speaker utterance and a set of opt-out speakers who do not wish to have their speech transcribed, the task requires an ASR system to transcribe all speakers except the opt-out ones, while still indicating when those speakers are active. As a first step towards tackling this task, we introduce a novel, light-weight Enrollment-Conditioned Gating (ECG) module attachable to a frozen dual-stream speech LLM that enables ASR for new opt-out speakers dynamically during inference, even those who were not seen during initial ECG training phase. Our experiments on both AMI (English) and AliMeeting (Mandarin) datasets show that speech transcription accuracy for corresponding opt-out words or characters falls from 72.3% to 48.2% and from 73.6% to 27.3%, respectively, while retained speakers' transcription error rates maintain more or less the same. Our approach provides a practical solution for modern video conferencing platforms, allowing speakers to dynamically opt-out from automated AI transcriptions without forcefully leaving the meeting sessions, enabling a privacy-preserving interface for potentially millions of online meetings daily.

Chinese Translation

我们在一个完全端到端的多说话者自动语音识别（ASR）和分话者识别框架中引入了目标说话者遗忘ASR（TSU-ASR）任务。给定一个多说话者的发言和一组选择退出的说话者（他们不希望自己的语音被转录），该任务要求ASR系统转录所有说话者的语音，除了选择退出的说话者，同时仍然指示这些说话者何时处于活跃状态。作为解决该任务的第一步，我们引入了一种新颖的轻量级注册条件门控（Enrollment-Conditioned Gating, ECG）模块，该模块可附加到一个冻结的双流语音大语言模型（LLM）上，使得在推理过程中能够动态为新的选择退出说话者提供ASR，即使这些说话者在初始ECG训练阶段未曾出现。我们在AMI（英语）和AliMeeting（普通话）数据集上的实验表明，相关选择退出词或字符的语音转录准确率分别从72.3%降至48.2%和从73.6%降至27.3%，而保留说话者的转录错误率基本保持不变。我们的方法为现代视频会议平台提供了一种实用的解决方案，使说话者能够动态选择退出自动AI转录，而无需强行离开会议，从而为每天可能有数百万次在线会议提供了一个保护隐私的接口。

View on arXiv Download PDF AI Translation

cs.CL / 14 / 2609.30467

Where Does Retrieval-Based Open-Ended Evaluation Fail? Automatic Taxonomy Induction from Long-Form Medical Answer Factuality Verification

基于检索的开放式评估在哪些方面失败？从长篇医学答案的事实验证中自动生成分类法

Huang, Heyuan, Dai, Jirui, DeLucia, Alexandra, Joshi, Sonal, Yarmohammadi, Mahsa, Gao, Jie, Gutiérrez, Bernal Jiménez, Dredze, Mark

Abstract

Retrieval-based factuality evaluation, where LLM-generated claims are verified against evidence from authoritative medical corpora, has become the dominant paradigm for scalable hallucination detection in high-stakes clinical settings. Despite the urgency of reliable and transparent medical fact verification, most systems measure performance with aggregate metrics like F1, which obscure where and why failures occur. Existing RAG diagnostics require gold answers or annotated gold evidence, neither of which exists in this regime. We introduce two comprehensive taxonomies, grounded in a case study on the open-ended MedExpert dataset and 3 closed-ended datasets, decomposing failures into retrieval-stage errors along five quality dimensions, and verifier-reasoning errors into six consecutive steps. We adapt an automatic pattern induction pipeline using LLM-as-Judge to label evidence quality and classify verifier reasoning errors at scale, and then stress-test our findings across 4 retrieval methods and 6 frontier verifier models. Our analysis reveals that scaling model size, adding reasoning effort, expanding to authoritative web sources, and applying medical fine-tuning do not resolve these failure modes, demonstrating that they represent fundamental limitations of the retrieve-then-verify paradigm in open-ended medical settings rather than artifacts of outdated systems. We release our code and data at https://anonymous.4open.science/r/Medical_RAG_eval-4AB5 for the full reproducibility of our results.

Chinese Translation

基于检索的事实性评估，即将大型语言模型（LLM）生成的声明与权威医学语料库中的证据进行验证，已成为在高风险临床环境中可扩展的幻觉检测的主导范式。尽管可靠和透明的医学事实验证迫在眉睫，但大多数系统使用F1等聚合指标来衡量性能，这掩盖了失败发生的具体位置和原因。现有的检索增强生成（RAG）诊断需要金标准答案或标注的金标准证据，而在这一领域中这两者都不存在。我们引入了两个全面的分类法，基于对开放式MedExpert数据集和三个封闭式数据集的案例研究，将失败分解为检索阶段的错误，涵盖五个质量维度，以及验证者推理错误，分为六个连续步骤。我们采用了一个自动模式生成管道，利用LLM作为评判者对证据质量进行标注，并在大规模上分类验证者推理错误，然后对我们的发现进行压力测试，涵盖4种检索方法和6种前沿验证者模型。我们的分析揭示，扩大模型规模、增加推理努力、扩展到权威网络来源以及应用医学微调并不能解决这些失败模式，表明它们代表了在开放式医学环境中检索后验证范式的根本局限，而非过时系统的伪影。我们在 https://anonymous.4open.science/r/Medical_RAG_eval-4AB5 发布了我们的代码和数据，以实现我们结果的完全可重复性。

View on arXiv Download PDF AI Translation

cs.CL / 15 / 2609.30471

CARGO: Context-Aware Retrieval-Gated Evaluation of Agentic AI in Production

CARGO：面向上下文的检索门控评估生产中的自主人工智能

Chhabra, Mukul, Patel, Shail, Medrano, Luigi

Abstract

Reference-based LLM-as-a-judge evaluation assumes the reference answer is the target. In deployed agentic systems that operate over dynamic entities (support cases, assets, accounts), the closest available reference typically applies the correct procedure to a different entity, so a literal judge penalizes different identifiers, dates, and statuses as errors or hallucinations. We name this failure mode reference-instance divergence (RID). We propose CARGO, a framework that (i) treats retrieved references as procedural exemplars and grounds factual judgments in the live instance's observed context, (ii) assigns each claim a three-way status (supported, contradicted, unverifiable) and penalizes only contradictions, and (iii) gates evaluation by retrieval confidence, casting production evaluation as selective prediction. We introduce CARGO-Bench, a perturbation-based diagnostic suite with ground truth by construction that separates leniency from discrimination. On CARGO-Bench (246 items, two judge models, 7,872 judgments), the standard reference-based judge penalizes 100% of correct entity-transplanted answers and is uninformative (discrimination index DI ~ 0); supplying the live facts without reframing changes nothing. CARGO eliminates these false penalties (0/50) while retaining near-complete contradiction recall (50/50 and 49/50), raising DI to 0.58 [0.48, 0.68]; a rubric-swap control attributes most of the effect to context-grounded dimension definitions. CARGO also exposes a limitation of its own design: the leniency that protects entity values suppresses detection of procedural corruptions (20% recall). A post-hoc fix does not close the gap, and an LLM-as-annotator study with written guidelines and adjudication shows the same blind spot. We release a preregistered protocol for extending the evaluation to expert agreement, risk-coverage, and cost on production traffic.

Chinese Translation

基于参考的LLM作为评判者的评估假设参考答案是目标。在操作动态实体（支持案例、资产、账户）的部署自主系统中，最接近的可用参考通常将正确的程序应用于不同的实体，因此字面上的评判者会将不同的标识符、日期和状态视为错误或幻觉。我们将这种失败模式称为参考实例偏差（reference-instance divergence, RID）。我们提出CARGO，一个框架，它（i）将检索到的参考视为程序示例，并将事实判断基于实时实例的观察上下文，（ii）为每个声明分配三种状态（支持、反驳、无法验证），并仅对反驳进行惩罚，以及（iii）通过检索置信度进行门控评估，将生产评估视为选择性预测。我们引入CARGO-Bench，一个基于扰动的诊断套件，其构建的真实情况分离宽容与歧视。在CARGO-Bench（246个项目，两个评判模型，7,872个判断）上，标准的基于参考的评判者惩罚100%的正确实体移植答案，并且信息量不足（歧视指数DI ~ 0）；提供实时事实而不重新框架并没有改变任何结果。CARGO消除了这些虚假惩罚（0/50），同时保持近乎完整的反驳召回（50/50和49/50），将DI提高到0.58 [0.48, 0.68]；一个评分标准交换控制将大部分效果归因于基于上下文的维度定义。CARGO还揭示了自身设计的一个局限性：保护实体值的宽容抑制了程序腐败的检测（20%的召回率）。事后修正未能弥补这一差距，而一项使用书面指南和裁决的LLM作为注释者的研究显示出同样的盲点。我们发布了一个预注册协议，以扩展评估到专家一致性、风险覆盖和生产流量的成本。

View on arXiv Download PDF AI Translation

cs.CL / 16 / 2609.30492

Breaking Homogeneity: Diversifying Persona Sets for Creative LLM Outputs

打破同质性：多样化个性集以促进创造性大型语言模型输出

Moon, Sang Bin, Cho, Nicole, Borrajo, Daniel, Ganesh, Sumitra, Hashemi, Abolfazl

Abstract

Language models often produce homogeneous responses to open-ended tasks; such homogeneity can spawn groupthink-the convergence of ideas toward a singular and potentially suboptimal decision. We formulate persona diversification as a set-level conditioning problem and study two orthogonal design choices: selecting versus generating personas, and space-filling versus frontier-seeking diversity. We instantiate this design space with four methods spanning coverage and dispersion subset selections, uniform-coverage sampling, and evolutionary persona generation. Evaluations on the Alternative Uses Task (AUT), Infinity-Chat, and Divergent Association Task (DAT) show the benefits of the proposed methods across tasks and creativity objectives. On AUT, evolutionary persona generation increases response diversity by 78.8%, originality by 26.1%, flexibility by 49.5%, and holistic creativity by 13.9% over task-only prompting, while maintaining 98.5% validity; on Infinity-Chat, it nearly doubles persona-induced response separation relative to random personas. Moreover, evolutionary personas compose with creativity-optimized prompting, further increasing its response diversity by 18.6% and creativity by 6.3%. These results establish persona-set geometry as a task-agnostic mechanism for eliciting divergent LLM outputs, and support persona diversification as a reusable complement to prompt optimization.

Chinese Translation

语言模型在开放性任务中往往产生同质化的响应；这种同质性可能导致群体思维——思想趋向于单一且可能次优的决策。我们将个性多样化形式化为一个集水平的条件问题，并研究两个正交的设计选择：选择个性与生成个性，以及填充空间与追求前沿多样性。我们通过四种方法实例化这一设计空间，涵盖了覆盖和分散子集选择、均匀覆盖采样和进化个性生成。在替代用途任务（AUT）、无限聊天（Infinity-Chat）和发散联想任务（DAT）上的评估显示了所提方法在各任务和创造性目标上的优势。在AUT中，进化个性生成使响应多样性提高了78.8%，原创性提高了26.1%，灵活性提高了49.5%，整体创造力提高了13.9%，同时保持了98.5%的有效性；在无限聊天中，相较于随机个性，它几乎使个性引发的响应分离度翻倍。此外，进化个性与优化创造性的提示组合，进一步使其响应多样性提高了18.6%，创造力提高了6.3%。这些结果确立了个性集几何作为一种任务无关的机制，以引发发散的LLM输出，并支持个性多样化作为提示优化的可重复利用补充。

View on arXiv Download PDF AI Translation

cs.CL / 17 / 2609.30514

Inquesto Score: A reliability Protocol For Voice Agents

Inquesto Score：一种用于语音代理的可靠性协议

Baali, Massa, Raj, Bhiksha

Abstract

Voice agents are increasingly deployed in workflows where failed interactions can affect transactions, access, and other consequential outcomes, creating a need for reproducible and interpretable evaluation. We introduce Inquesto Score (IS), a protocol for measuring voice-agent reliability as the percentage of calls in a fixed, versioned evaluation population that achieve the caller's goal without a functional failure or worse. Rather than combining heterogeneous metrics, IS defines explicit failure events and severity levels and evaluates the deployed voice pipeline. Timing failures, including talk-over and delayed responses, are measured directly from audio, while semantic and state-dependent failures are evaluated using scenario predicates, tool traces, and a pinned open-model judge. Diagnostic views of behavior, acoustic robustness, identity handling, and speaker groups accompany the score without being combined into it. Inquesto Score v0.1 evaluates 30 scenarios, three acoustic conditions, four speaker groups, and 306 calls per agent across 13 configurations of a reference voice-agent system. Our evaluation shows that reliable measurement requires evidence beyond transcripts, explicit treatment of deployment conditions, and validation of the evaluators used to determine outcomes. We release the protocol, reference implementation, and evaluation records.

Chinese Translation

语音代理越来越多地被部署在可能影响交易、访问及其他重要结果的工作流程中，因此需要可复现且可解释的评估方法。我们提出了Inquesto Score（IS），这是一种衡量语音代理可靠性的协议，定义为在固定且版本化的评估样本中，达到呼叫者目标且无功能性故障或更严重问题的通话百分比。IS并非将异质指标简单合并，而是明确界定故障事件及其严重等级，并评估已部署的语音处理流程。时序故障（包括话语重叠和响应延迟）通过音频直接测量，而语义及状态依赖的故障则通过场景谓词、工具追踪及固定的开放模型评判器进行评估。行为诊断视图、声学鲁棒性、身份处理及说话者群体分析作为辅助信息呈现，但不纳入评分中。Inquesto Score v0.1涵盖30个场景、三种声学条件、四个说话者群体，以及每个代理在13种配置下共306次通话的评估。我们的研究表明，可靠的测量需要超越文本记录的证据，明确考虑部署条件，并验证用于判定结果的评估者。我们公开了该协议、参考实现及评估记录。

View on arXiv Download PDF AI Translation

cs.CL / 18 / 2609.30535

Feeding BabyLMs Macaroni: Code-Switching Curricula Cause Cross-Lingual Convergence

给 BabyLMs 喂意大利面：代码切换课程导致跨语言收敛

Rooryck, Dries, Cai, Alex, Belinkov, Yonatan, Alvarez-Melis, David, Brantley, Kianté

Abstract

Children in multilingual communities often code-switch, using multiple languages in a single utterance. Can we induce cross-lingual alignment in language models by training on code-switched text? We pretrain small decoder-only transformers on two 100M-word multilingual corpora: a base corpus formed by mixing the English, Dutch, and Chinese BabyBabelLM datasets, and a corpus generated from it by inserting word- and sentence-level code-switching using an LLM. We find that training on code-switched data aligns the representations of parallel text, particularly across different scripts, and that this alignment persists through training on monolingual documents. Under a learning curriculum that progresses from word-level code-switching, to sentence-level code-switching, to monolingual documents, models trained on code-switched data outperform baselines trained without it on the BabyLM evaluation suite. Our work characterizes code-switching curriculum learning as an effective data augmentation method for multilingual pretraining. We release our code, data, and models at https://github.com/drooryck/multilingual-macaroni.

Chinese Translation

多语言社区中的儿童经常进行代码切换，在单一话语中使用多种语言。我们能否通过在代码切换文本上进行训练来诱导语言模型的跨语言对齐？我们在两个包含1亿词的多语言语料库上对小型解码器模型进行预训练：一个基础语料库由英语、荷兰语和中文的 BabyBabelLM 数据集混合而成，另一个则是通过使用大型语言模型（LLM）插入词级和句级代码切换生成的语料库。我们发现，在代码切换数据上进行训练能够对齐平行文本的表示，尤其是在不同书写系统之间，这种对齐在对单语文档进行训练时仍然存在。在一个从词级代码切换、到句级代码切换，再到单语文档的学习课程下，基于代码切换数据训练的模型在 BabyLM 评估套件上的表现优于未使用代码切换数据的基线模型。我们的研究将代码切换课程学习描述为一种有效的多语言预训练数据增强方法。我们在 https://github.com/drooryck/multilingual-macaroni 发布了我们的代码、数据和模型。

View on arXiv Download PDF AI Translation

cs.CL / 19 / 2609.30547

REALMS: An AI-Assistant Conversational System for Real-Time Exact Audience Sizing over High-Dimensional Nested Profiles

REALMS：一种用于高维嵌套档案实时精确受众规模的AI助手对话系统

Ma, Haixu, Bansal, Aditya, Lohiya, Shubham, Ranjan, Sumit

Abstract

Audience sizing is a critical component of digital marketing. It enables precise resource allocation, campaign planning, and performance optimization. Traditional approaches using skeleton audiences, sampling, or predictive modeling suffer from significant delays, estimation errors, and poor scalability over high-dimensional profile data. We present REALMS (Real-time Exact Audience sizing via LLM-based Multi-attribute Search), a conversational system for exact audience sizing deployed in production on an enterprise customer data platform. REALMS enables marketers to query massive profile stores with millions of profiles and thousands of attributes using natural language and receive precise counts in seconds. The system introduces three key components: (1) a categorical attribute retrieval mechanism using embedding-based vector search to dynamically identify relevant schema attributes without manual configuration; (2) an LLM-powered NL2SQL pipeline with template-based in-context learning for accurate query generation over complex nested schemas; and (3) schema standardization enabling industry-agnostic deployment across diverse enterprise environments. Evaluation on real enterprise data demonstrates strong recall for attribute retrieval, high SQL execution accuracy, and low latency, which enables real-time interactive audience insights where prior methods required hours.

Chinese Translation

受众规模是数字营销中的关键组成部分。它使得资源分配、活动规划和绩效优化更加精确。传统的方法使用骨架受众、抽样或预测建模，存在显著的延迟、估计误差以及在高维档案数据上的可扩展性差的问题。我们提出了REALMS（基于LLM的多属性搜索实时精确受众规模），这是一个在企业客户数据平台上投入生产的精确受众规模对话系统。REALMS使市场营销人员能够使用自然语言查询包含数百万个档案和数千个属性的大型档案库，并在几秒钟内获得精确的计数。该系统引入了三个关键组件：（1）使用基于嵌入的向量搜索的分类属性检索机制，动态识别相关的模式属性，无需手动配置；（2）一个基于LLM的NL2SQL管道，结合基于模板的上下文学习，实现对复杂嵌套模式的准确查询生成；（3）模式标准化，使其能够在多样化的企业环境中进行行业无关的部署。对真实企业数据的评估显示，该系统在属性检索方面具有较强的召回率，高SQL执行准确性和低延迟，使得实时交互式受众洞察成为可能，而之前的方法则需要数小时。

View on arXiv Download PDF AI Translation

cs.CL / 20 / 2609.30558

Probing Stability-Plasticity Tradeoffs in Agent Memory through Cognitive Experimental Paradigms

通过认知实验范式探究智能体记忆中的稳定性-可塑性权衡

Ding, Jiaqi, Wu, Guorong

Abstract

Agent memory systems are increasingly used to maintain long-term user preferences, task states and evolving facts, but current evaluations often collapse memory behavior into final-answer accuracy. We introduce MemProbe, a cognitive-science-inspired framework for diagnosing stability-plasticity tradeoffs in agent memory. The framework is motivated by a core insight from cognitive memory research: memory is reconstructive and shaped by interference, source reliability, reinforcement, and reactivation. MemProbe turns this insight into four reusable experimental paradigms (interference, misinformation, consolidation strength, and reconsolidation window) that manipulate when a memory should be updated, preserved, or treated as uncertain. It further decomposes correctness into behavioral profiles that reveal how systems update, preserve, attribute, and temporally organize information. We instantiate these paradigms in a 56-episode diagnostic suite and evaluate six incremental memory systems under a unified protocol. Results show that systems with similar aggregate scores exhibit distinct behavioral profiles. MemProbe provides such a diagnostic lens, turning aggregate performance into interpretable profiles of memory maintenance over time. Code is available at https://github.com/jq-ding/MemProbe.

Chinese Translation

智能体记忆系统越来越多地用于维持长期用户偏好、任务状态和不断变化的事实，但当前的评估往往将记忆行为简化为最终答案的准确性。我们引入了 MemProbe，这是一个受认知科学启发的框架，用于诊断智能体记忆中的稳定性-可塑性权衡。该框架的动机源于认知记忆研究中的一个核心见解：记忆是重构性的，并受到干扰、来源可靠性、强化和再激活的影响。MemProbe 将这一见解转化为四个可重复使用的实验范式（干扰、错误信息、巩固强度和再巩固窗口），这些范式操控何时应更新、保留或视为不确定的记忆。它进一步将正确性分解为行为特征，揭示系统如何更新、保留、归因和时间组织信息。我们在一个包含56个剧集的诊断套件中实例化这些范式，并在统一协议下评估六个增量记忆系统。结果表明，具有相似总分的系统表现出不同的行为特征。MemProbe 提供了这样的诊断视角，将总表现转化为可解释的记忆维护特征。代码可在 https://github.com/jq-ding/MemProbe 获取。

View on arXiv Download PDF AI Translation

cs.CL / 21 / 2609.30604

The Hard Part Comes After Search: Benchmarking Web Agents on Synthesizing, Organizing, and Displaying Knowledge

搜索之后的难点：基于知识合成、组织和展示的网络代理基准测试

Gill, Alexander, Ishmam, Md Farhan, Nguyen, Xuyen, Bhat, Neha, DeYoung, Parker Henry, Chaleshtori, Fateme Hashemi, Stringham, Nathan, Marino, Kenneth, Marasović, Ana

Abstract

Existing computer-use agent benchmarks do not fully evaluate agents acting as assistants. A useful assistant retrieves information across complex, multi-step workflows, synthesizes it into artifacts (documents, presentations, spreadsheets), and navigates program interfaces to produce a coherent final product. Such workflows demand reasoning and synthesis, decomposition of complex tasks, as well as visual and spatial understanding. To study agents on workflows like these, we introduce KNOWS, a benchmark of open-ended, complex, browser-based tasks that jointly evaluate these capabilities, with each task culminating in a produced artifact. To write tasks, we develop a task design rubric and a protocol for ensuring that tasks meet the requirements. Each task is paired with an evaluator, a program that combines deterministic checks with LLM judgments to balance the richness, reliability, and automation tradeoff inherent to agent evaluation. We evaluate and analyze frontier computer-use agents and browser-based harnesses. They achieve moderate scores on partial-success metrics, but the best performer fully succeeds in fewer than 3% of our complex, long-horizon tasks. Failures on visual steps render the resulting artifacts unusable, even when agents complete more than 50% of other evaluation steps. Our results expose limitations of current agents acting as end-to-end assistants, and call for progress on tool use, visual understanding, and long-horizon reasoning.

Chinese Translation

现有的计算机使用代理基准测试并未充分评估作为助手的代理。一个有用的助手能够在复杂的多步骤工作流程中检索信息，将其合成成文档、演示文稿、电子表格等工件，并导航程序界面以生成一个连贯的最终产品。这类工作流程要求推理与综合、复杂任务的分解，以及视觉和空间理解。为了研究这类工作流程中的代理，我们引入了KNOWS，这是一个开放式复杂的基于浏览器的任务基准，旨在共同评估这些能力，每个任务最终产生一个工件。为编写任务，我们开发了任务设计标准和确保任务符合要求的协议。每个任务都配有一个评估器，这是一个结合确定性检查和大型语言模型（LLM）判断的程序，以平衡代理评估中固有的丰富性、可靠性和自动化的权衡。我们评估和分析了前沿的计算机使用代理和基于浏览器的工具。它们在部分成功指标上取得了中等分数，但表现最佳的代理在我们复杂的长期任务中完全成功的比例不足3%。在视觉步骤上的失败使得最终工件无法使用，即使代理完成了超过50%的其他评估步骤。我们的结果揭示了当前代理作为端到端助手的局限性，并呼吁在工具使用、视觉理解和长期推理方面取得进展。

View on arXiv Download PDF AI Translation

cs.CL / 22 / 2609.30652

Recursive Self-Improvement via On-Policy Distillation for Reasoning

通过在线蒸馏实现的递归自我改进用于推理

Yin, Shangjian, Zhao, Zehao, Asadi, Kavosh, Liu, Rui, Lu, Yuchen, Mei, Shike, Cui, Hang, Simon, Luke, Shi, Zhouxing, Firooz, Hamed

Abstract

On-policy distillation (OPD) trains a student model by having it generate trajectories, then matching its next-token predictions with an external teacher's next-token predictions. This provides dense, token-level supervision to the student. On-policy self-distillation (OPSD) eliminates the need for the external teacher. Specifically, a second frozen copy of the student model, now given the ground truth in its context, serves as the teacher. The student model only receives the problem and learns to mimic the privileged teacher model, while the teacher remains frozen throughout training. Previous work showed that freezing the teacher is useful for training stability, but we argue that this can prevent the teacher from incorporating the improvements learned by the student during training. Our primary contribution is to address this limitation with a recursive framework built around two complementary components. First, we let the privileged teacher co-evolve with the student so that revision learned in one round can guide the next, a process we refer to as Dynamic Co-Evolution (DCE). Second, because stronger revision can also make responses too verbose and self-critical, we additionally train on shorter, verified rewrites of the model's own on-policy responses. We call this complementary objective Self-Refined Concise Learning (SRCL). Overall, our comprehensive evaluations show that DCE+SRCL outperforms OPSD across multiple model scales and four competition-level mathematics benchmarks. Specifically, on Qwen3-8B, DCE+SRCL reaches 65.97% Average@12, outperforming OPSD by 35.62 percentage points while reducing mean output length by 7.80% relative to DCE alone.

Chinese Translation

在线蒸馏（On-policy distillation, OPD）通过让学生模型生成轨迹，然后将其下一个标记的预测与外部教师的下一个标记预测进行匹配，从而训练学生模型。这为学生提供了密集的标记级监督。在线自蒸馏（On-policy self-distillation, OPSD）消除了对外部教师的需求。具体而言，学生模型的第二个冻结副本在其上下文中获得真实标签，充当教师。学生模型仅接收问题，并学习模仿特权教师模型，而教师在整个训练过程中保持冻结。先前的研究表明，冻结教师有助于训练的稳定性，但我们认为这可能会阻止教师在训练过程中吸收学生所学到的改进。我们的主要贡献是通过一个围绕两个互补组件构建的递归框架来解决这一限制。首先，我们让特权教师与学生共同进化，以便在一个回合中学到的修正可以指导下一个回合，这一过程我们称之为动态共同进化（Dynamic Co-Evolution, DCE）。其次，由于更强的修正可能使响应过于冗长和自我批评，我们还在模型自身的在线响应的较短、经过验证的重写上进行训练。我们将这一互补目标称为自我精炼简洁学习（Self-Refined Concise Learning, SRCL）。总体而言，我们的综合评估表明，DCE+SRCL在多个模型规模和四个竞争级数学基准测试中优于OPSD。具体而言，在Qwen3-8B上，DCE+SRCL达到了65.97%的Average@12，相较于OPSD提高了35.62个百分点，同时相对于单独的DCE减少了7.80%的平均输出长度。

View on arXiv Download PDF AI Translation

cs.CL / 23 / 2609.30670

TRACE: Temporal Audit and Condition-aware Evaluation of Streaming Video Understanding

TRACE：流媒体视频理解的时间审计与条件感知评估

Ma, Yibo, Zhang, Qianqian, Liu, Peng, Zhao, Tiancheng

Abstract

Streaming video understanding requires models to interpret evidence as it arrives, yet current evaluations often report task scores without specifying when evidence becomes valid, how visual history is maintained, or how responses are triggered. As a result, similar scores may correspond to different workloads, failure modes, and operational behavior. We introduce TRACE (Temporal Audit and Condition-aware Evaluation), a condition-aware benchmark and evaluation framework that makes these factors explicit. TRACE combines temporally audited visual tasks with evidence timing and instruction-dependent trigger annotations, a unified causal Core--Adapter protocol that controls information availability while recording actual history processing and response events, and multidimensional reporting of answer quality, timeliness, response-selection behavior, workload, completion, and reliability. On 1,240 records from 517 videos, we evaluate eight publicly available models or systems in eight configurations. We find that nearly identical QA accuracy can mask substantial differences in completion, answer validity, and generation workload, while proactive performance separates into response quality, response delay, false alarms (responses emitted while no target window is currently valid and a later one remains), and missed target windows. These results show that streaming-video performance should be interpreted as execution-conditioned system behavior rather than a single score. Our benchmark and code can be accessed at \href{https://github.com/om-ai-lab/trace-bench}{https://github.com/om-ai-lab/trace-bench}.

Chinese Translation

流媒体视频理解要求模型在证据到达时进行解释，然而当前的评估往往在未明确证据何时有效、视觉历史如何维护或响应如何触发的情况下报告任务得分。因此，相似的得分可能对应于不同的工作负载、失败模式和操作行为。我们提出了TRACE（时间审计与条件感知评估），这是一个明确这些因素的条件感知基准和评估框架。TRACE结合了时间审计的视觉任务、证据时序和依赖指令的触发注释，以及一个统一的因果核心-适配器协议，该协议控制信息的可用性，同时记录实际的历史处理和响应事件，并对答案质量、及时性、响应选择行为、工作负载、完成情况和可靠性进行多维度报告。在517个视频的1240条记录上，我们评估了八个公开可用的模型或系统的八种配置。我们发现，几乎相同的问答准确性可能掩盖了完成情况、答案有效性和生成工作负载的显著差异，而主动性能则分为响应质量、响应延迟、误报（在没有有效目标窗口时发出的响应，而后续窗口仍然存在）和错过的目标窗口。这些结果表明，流媒体视频性能应被解释为执行条件下的系统行为，而不是单一得分。我们的基准和代码可以在 https://github.com/om-ai-lab/trace-bench 访问。

View on arXiv Download PDF AI Translation

cs.CL / 24 / 2609.30716

Words Speak Louder Than Order: A Behavioral Evaluation of Gemma 4

语言的表达胜于顺序：对Gemma 4的行为评估

Fitch, Amanda

Abstract

When a language model receives two conflicting documents as input, how does it decide which one to prioritize? Does it rely on how the sources are framed or the presentation order of the documents? We evaluated this behavior on Google's pre-trained Gemma 4-e4b model across a targeted behavioral suite (n = 13 items, 784 forward passes in short, single-turn contexts) using a completely counterbalanced experimental design. This setup allowed us to mathematically isolate the specific effects of source framing and reading position, while ensuring the model's natural vocabulary biases were canceled out. Across ten test conditions, we discovered the following: 1. Source framing heavily overpowers reading position. When directly competing, the semantic framing of a source (such as presenting it as an official guideline or a fresh update) had a significantly stronger impact on the model's final answer than the presentation order of the document. 2. The model favors the first document it reads, but this bias is highly variable. While the model consistently demonstrated a primacy effect (preferring the first document presented), the actual strength of this bias fluctuated by at least a factor of 5 based solely on the surface wording. 3. Overall structural repetition, not short copy-cues, drives positional bias. The model's preference for the first document is not a mechanical reaction to short, repetitive trigger phrases, such as "is [Answer]". However, the primacy effect does increase significantly when the two competing documents are structurally identical, using word-for-word verbatim templates. Introducing variation in the overall wording between the two sources reduces this positional bias.

Chinese Translation

当一个语言模型接收到两个相互矛盾的文档作为输入时，它如何决定优先考虑哪个文档？它是依赖于信息来源的框架还是文档的呈现顺序？我们在谷歌预训练的Gemma 4-e4b模型上评估了这种行为，采用了一套针对性的行为测试（n = 13项，784次短期单轮上下文的前向传递），并使用了完全平衡的实验设计。该设置使我们能够在数学上隔离信息来源框架和阅读位置的具体影响，同时确保模型的自然词汇偏见被抵消。在十个测试条件下，我们发现以下几点：1. 信息来源的框架显著压倒阅读位置。当直接竞争时，信息来源的语义框架（例如将其呈现为官方指南或最新更新）对模型最终答案的影响明显强于文档的呈现顺序。2. 模型偏好其阅读的第一份文档，但这种偏见高度可变。尽管模型始终表现出首因效应（偏好第一份呈现的文档），但这种偏见的实际强度仅基于表面措辞就波动了至少5倍。3. 整体结构重复，而非短期复制提示，驱动位置偏见。模型对第一份文档的偏好并不是对短期重复触发短语（如“是[答案]”）的机械反应。然而，当两个竞争文档在结构上完全相同，使用逐字逐句的模板时，首因效应显著增强。在两个来源之间引入整体措辞的变化会减少这种位置偏见。

View on arXiv Download PDF AI Translation

cs.CL / 25 / 2609.30738

Beyond Mean Attention: Diversity-Aware, Layer-Wise Scoring for KV Cache Eviction

超越均值注意力：关注多样性、逐层评分的KV缓存驱逐方法

Xie, Tianfang, Zhu, Wei

Abstract

KV cache eviction methods such as SnapKV and PyramidKV rank tokens solely by mean attention over a small observation window. We study a unified score, $\mu_i+\lambda_1\sigma_i+\lambda_2\mathrm{corr}(i,S)$, adding attention dispersion across window queries and redundancy relative to selected tokens. For $\lambda_2<0$, the score penalizes similarity to selected tokens as in maximal marginal relevance (MMR), without extra forward passes. To test whether this relevance-diversity balance should vary with depth, we compare fixed global coefficients with three-segment and quadratic profiles. Only these depth profiles are searched on a development split under a $\sinh$ reparameterization. On all 16 English LongBench datasets with Mistral-7B at a budget of 64 entries per layer, a single global diversification constant improves 13 of 16 datasets (macro +1.1); the gain holds at budget 32 and narrows at 128. Per-dataset search finds no detectable layer structure on most datasets; on passage retrieval it finds a large one: a mid-layer sign flip that rewards similarity and is worth +9.6 over the baseline at budget 64 and, without re-tuning, +13.2 over the global constant at budget 128. Ablations attribute the gain to the redundancy term; replaying every accepted search state on the held-out test set separates genuine structure from tuning noise.

Chinese Translation

KV缓存驱逐方法如SnapKV和PyramidKV仅通过小观察窗口内的均值注意力对令牌进行排名。我们研究了一种统一评分，$BC_i+BB_1C3_i+BB_2 ext{corr}(i,S)$，加入了窗口查询的注意力分散度和相对于选定令牌的冗余度。当$BB_2<0$时，该评分惩罚与选定令牌的相似性，类似于最大边际相关性（MMR），而无需额外的前向传递。为了测试这种相关性-多样性平衡是否应随深度变化，我们比较了固定的全局系数与三段和二次曲线配置。仅在$ ext{sinh}$重新参数化下，这些深度配置在开发集上被搜索。在使用Mistral-7B的16个英语LongBench数据集上，每层预算为64条目时，单一全局多样化常数改善了16个数据集中的13个（宏观+1.1）；该增益在预算为32时保持有效，并在128时缩小。每个数据集的搜索在大多数数据集中未发现可检测的层结构；在段落检索中发现了一个显著的结构：一个中层的符号翻转，奖励相似性，在预算为64时比基线高出+9.6，并且在预算为128时，无需重新调优，比全局常数高出+13.2。消融实验将增益归因于冗余项；在保留的测试集上重放每个接受的搜索状态，将真实结构与调优噪声区分开。

View on arXiv Download PDF AI Translation

cs.CL / 26 / 2609.30739

SEA-CLIP-Tiny: Efficient Multilingual Text-Vision Embedding for Southeast Asian Languages

SEA-CLIP-Tiny：东南亚语言的高效多语言文本-视觉嵌入

Habibi, Puja Ahmad, Firdaus, Faiz Assabil, S, Ashvanth, Chuangsuwanich, Ekapol, Tuchinda, Pume, Limkonchotiwat, Peerat

Abstract

Multilingual text-vision embedding models are essential for cross-lingual image-text retrieval, but Southeast Asian languages remain poorly supported due to the region's linguistic diversity and limited data and computing resources. In this paper, we introduce SEA-CLIP-Tiny, a compact multilingual text-vision embedding model for Southeast Asia with fewer than 50M parameters. Our model adapts a CLIP-KD-style framework to Southeast Asian multilingual settings through regional data curation and multilingual teacher guidance. Experiments across seven Southeast Asian languages show that SEA-CLIP-Tiny achieves the strongest average retrieval performance among the evaluated student models, reaching 12.9%, 31.5%, and 42.2% at R@1, R@5, and R@10, respectively. Compared with MobileCLIP2, it improves average R@10 by 12.1 points while using 38.4% fewer parameters and lower measured CPU latency. These results highlight the importance of region-aware training for efficient multilingual text-vision models in Southeast Asia.

Chinese Translation

多语言文本-视觉嵌入模型对于跨语言图像-文本检索至关重要，但由于东南亚地区的语言多样性以及数据和计算资源的有限支持，该地区的语言仍然受到较差的支持。本文介绍了SEA-CLIP-Tiny，这是一种针对东南亚的紧凑型多语言文本-视觉嵌入模型，参数少于5000万。我们的模型通过区域数据整理和多语言教师指导，将CLIP-KD风格框架适应于东南亚多语言环境。针对七种东南亚语言的实验表明，SEA-CLIP-Tiny在评估的学生模型中实现了最强的平均检索性能，在R@1、R@5和R@10上分别达到了12.9%、31.5%和42.2%。与MobileCLIP2相比，它在使用38.4%更少参数和更低的CPU延迟的情况下，平均R@10提高了12.1个百分点。这些结果突显了区域感知训练对于东南亚高效多语言文本-视觉模型的重要性。

View on arXiv Download PDF AI Translation

cs.CL / 27 / 2609.30773

Learning Natural Conversational Behavior in Tandem Speech-to-Speech Models with Randomized Guidance

在随机引导下学习自然对话行为的串联语音到语音模型

Yaguchi, Manato, Kubo, Yotaro, Asano, Hikaru, Kuroki, So

Abstract

Tandem speech-to-speech architectures couple a responsive speech frontend with an asynchronous text backend. In KAME, a large language model (LLM) serves as the backend, supplying candidate responses as guidance to the speech frontend while the user is still speaking. Ordinary conversation recordings capture the eventual response but not the guidance the backend would supply during the user's utterance. Generating the missing guidance with a simulator LLM adds substantial data-preparation overhead when training on real conversations. We propose randomized intermediate guidance, which derives guidance directly from the conversation corpus rather than simulating backend LLM behavior. During training, target responses provide informative guidance, while randomly sampled responses provide potentially irrelevant updates during the utterance. This combination aims to teach the frontend to use backend information selectively. On synthetic dialogues, KAME trained with this recipe achieves response quality comparable to that of the LLM-generated and similarity-based baselines. Training on 3.8k hours of real conversations improves smooth turn-taking and audio-judge naturalness over synthetic-data KAME while retaining a response-quality advantage over Moshi. These results show that randomized guidance offers a practical route to combining the response-quality benefits of tandem models with natural conversational behavior learned from real speech.

Chinese Translation

串联语音到语音架构将响应式语音前端与异步文本后端结合起来。在 KAME 中，一个大型语言模型（LLM）作为后端，在用户仍在说话时提供候选响应作为对语音前端的引导。普通的对话录音捕捉到最终的响应，但并没有记录后端在用户发言期间会提供的引导。使用模拟器 LLM 生成缺失的引导在训练真实对话时增加了大量的数据准备开销。我们提出了随机中间引导，它直接从对话语料库中获取引导，而不是模拟后端 LLM 的行为。在训练过程中，目标响应提供了有用的引导，而随机抽样的响应则在发言期间提供了潜在的无关更新。这种组合旨在教会前端选择性地使用后端信息。在合成对话中，使用这种方法训练的 KAME 在响应质量上达到了与 LLM 生成的基线和基于相似性的基线相当的水平。在 3.8k 小时的真实对话上训练，改善了平滑的轮换和音频评估的自然性，同时在响应质量上仍优于 Moshi。这些结果表明，随机引导为将串联模型的响应质量优势与从真实语音中学习的自然对话行为相结合提供了一条实用的途径。

View on arXiv Download PDF AI Translation

cs.CL / 28 / 2609.30802

Understanding the Role of Prompt Template in Knowledge Distillation for Safety Alignment

理解提示模板在安全对齐知识蒸馏中的作用

Budathoki, Anjila, Dhakal, Manish, Ampel, Benjamin M., Ding, Yi

Abstract

Prior research has demonstrated that the choice of prompt template during Supervised Fine-Tuning (SFT) significantly impacts the robustness of safety alignment afterwards. However, the influence of template selection during Knowledge Distillation (KD) from teacher to student remains largely unexplored. Thus, we fill this gap by analyzing how different template configurations influence the pre-existing safety alignment of the student. We observe a significant degradation of safety alignment present in the aligned base instruct-tuned model. Specifically, we find that utilizing chat templates renders the model more compliant with harmful queries compared to a non-chat template. These findings are consistent across three models: LLaMA, Gemma and Qwen model families and are evaluated across multiple safety benchmarks. We further show that using a non-chat template during distillation better preserves the base student's internal representations, while chat template distillation induces a larger representational shift. Code: https://github.com/anjilab/role-of-prompt-template-in-kd

Chinese Translation

先前的研究表明，在监督微调（Supervised Fine-Tuning, SFT）过程中，提示模板的选择对后续安全对齐的鲁棒性有显著影响。然而，在教师到学生的知识蒸馏（Knowledge Distillation, KD）过程中，模板选择的影响仍然未被充分探讨。因此，我们通过分析不同模板配置如何影响学生的既有安全对齐来填补这一空白。我们观察到，在对齐的基础指令调优模型中，安全对齐显著下降。具体而言，我们发现使用聊天模板使模型对有害查询的服从性较高，而非聊天模板则相对较低。这些发现在三个模型（LLaMA、Gemma 和 Qwen 模型家族）中是一致的，并在多个安全基准上进行了评估。我们进一步表明，在蒸馏过程中使用非聊天模板更好地保留了基础学生的内部表征，而聊天模板蒸馏则引发了更大的表征转变。代码链接： https://github.com/anjilab/role-of-prompt-template-in-kd

View on arXiv Download PDF AI Translation

cs.CL / 29 / 2609.30846

I-Parakeet: Integer-Only Conformer ASR on Mobile NPU

I-Parakeet：基于整数的移动NPU上的Conformer语音识别

Nishimura, Taichi

Abstract

In this paper, we propose I-Parakeet, an integer-only implementation of NVIDIA's Parakeet-CTC (0.6B parameters) that runs on a smartphone NPU without any floating-point operator or CPU fallback. Modern Conformer ASR models are hard to deploy on edge devices because of their size, and quantized models still fall back to floating point for numerically sensitive operations. This prevents them from fully exploiting integer accelerators such as mobile NPUs. To achieve this, our contributions are threefold. First, we derive an integer formulation of the relative-positional self-attention at the core of the Conformer. We fuse its two score branches with different quantization scales and the relative shift into integer-only operations. Second, we introduce a minimax-optimized Swish approximation that minimizes the maximum error of the Swish output. Third, a layer-wise range analysis of activations yields two targeted remedies: an INT16 grid for the BatchNorm output and percentile calibration for the heavy-tailed pre-encoder activations. I-Parakeet achieves 4.97% WER on LibriSpeech test-other, running on a Qualcomm NPU at a real-time factor of 0.048, 7.5x faster than a CPU baseline.

Chinese Translation

在本文中，我们提出了I-Parakeet，这是NVIDIA的Parakeet-CTC（参数量为0.6B）的整数-only实现，能够在智能手机NPU上运行，而无需任何浮点运算或CPU回退。现代的Conformer语音识别模型由于其体积庞大而难以在边缘设备上部署，而量化模型在数值敏感操作中仍然会回退到浮点运算。这阻碍了它们充分利用移动NPU等整数加速器。为此，我们的贡献主要有三方面。首先，我们推导出了Conformer核心的相对位置自注意力的整数形式。我们将其两个具有不同量化尺度的得分分支与相对偏移融合为仅使用整数的操作。其次，我们引入了一种最小最大优化的Swish近似，旨在最小化Swish输出的最大误差。第三，对激活的逐层范围分析得出了两种针对性的补救措施：为BatchNorm输出提供一个INT16网格，以及对重尾预编码器激活进行百分位校准。I-Parakeet在LibriSpeech test-other上达到了4.97%的字错误率（WER），在高通NPU上以0.048的实时因子运行，比CPU基线快7.5倍。

View on arXiv Download PDF AI Translation

cs.CL / 30 / 2609.30849

Enhancing Assessment of Self-Consistency in LLM Explanations using Perturbation Strength

通过扰动强度增强对大型语言模型解释自一致性的评估

Le, Phuong Q., Kurniawan, Kemal, Lau, Jey Han

Abstract

Prior work has examined the self-consistency of LLM-generated explanations using surface-level perturbation methods. However, the strength of these perturbations is not explicitly measured and controlled. In this work, we propose an LLM-as-a-judge approach to measure perturbation strength in a unified manner across input and CoT perturbations. We then evaluate the self-consistency in explanations generated from various LLMs under controlled strength conditions, ensuring a fair comparison across perturbation types. Experiments show that our proposed LLM-based perturbation strength measure outperforms other embedding- and probability-based approaches and that input perturbations generally affect LLMs more strongly than CoT perturbations. Our work suggests that judgments about a model's self-consistency is fair only within the same perturbation type.

Chinese Translation

之前的研究使用表层扰动方法考察了大型语言模型（LLM）生成的解释的自一致性。然而，这些扰动的强度并未被明确测量和控制。在本研究中，我们提出了一种LLM作为评判者的方法，以统一方式测量输入和链式推理（CoT）扰动的强度。然后，我们在控制强度条件下评估来自不同LLM生成的解释的自一致性，以确保对不同扰动类型的公平比较。实验表明，我们提出的基于LLM的扰动强度测量方法优于其他基于嵌入和概率的方法，并且输入扰动通常对LLM的影响强于CoT扰动。我们的研究表明，关于模型自一致性的判断仅在相同扰动类型内是公平的。

View on arXiv Download PDF AI Translation

cs.CL / 31 / 2609.30864

Persistent Negatives for Adversarial Black-Box On-Policy Distillation

对抗性黑箱在线策略蒸馏中的持久负样本

Ma, Haixu, Lahrichi, Saad, Li, Weiwei, Han, Kevin, Wu, Weiqiang, Yang, Peggy, Li, Dongzhuo, Li, Ruiyi, Li, Serena, Zhou, Gedi, Gao, Mingze, Kumar, Abhishek, Fan, Xiangjun, Zhang, Lizhu

Abstract

Black-box On-Policy Distillation (OPD) seeks to improve a student from its own generations when the teacher provides sampled responses but not token probabilities. Adversarial distillation offers one route: it learns a discriminator over prompt-matched teacher and student responses and uses its score as the policy reward. However, sampling discriminator negatives from the latest student at each step couples the learned reward to a negative distribution that changes after every policy update. We address this moving-target problem with persistent-negative adversarial distillation, a live-pool method that replaces a fraction of each discriminator batch with historical, prompt-matched teacher--student comparisons. Under matched discriminator compute, historical comparisons train the discriminator, while GRPO remains on-policy with fresh student responses. Our analysis identifies the Bayes-optimal reward as a teacher-to-negative log-density ratio and, under explicit assumptions, shows how persistent negatives anchor the discriminator and reduce reward-estimation MSE relative to fresh-negative training. Across two student families, three judges, and four judged-chat benchmarks, persistent-negative adversarial distillation consistently improves performance over current methods at matched discriminator compute. It also yields smoother fresh-policy discriminator trajectories, with fewer below-chance dips. These findings identify the discriminator's negative distribution as an important design axis in black-box on-policy distillation.

Chinese Translation

黑箱在线策略蒸馏（On-Policy Distillation, OPD）旨在通过自身生成的样本来提升学生模型的表现，前提是教师提供的是采样响应而非标记概率。对抗性蒸馏提供了一种解决方案：它在与提示匹配的教师和学生响应之间学习一个判别器，并将其得分作为策略奖励。然而，在每一步从最新学生中采样判别器负样本会将学习到的奖励与一个在每次策略更新后变化的负分布联系起来。我们通过持久负样本对抗性蒸馏来解决这一动态目标问题，这是一种实时池方法，它用历史的、与提示匹配的教师-学生比较替换每个判别器批次的一部分。在匹配的判别器计算下，历史比较用于训练判别器，而广义重参数化优化（Generalized Reparameterization Optimization, GRPO）则保持在策略上，使用新鲜的学生响应。我们的分析确定了贝叶斯最优奖励为教师与负样本的对数密度比，并在明确假设下展示了持久负样本如何锚定判别器，并相较于新鲜负样本训练减少奖励估计的均方误差（MSE）。在两个学生模型系列、三个评估者和四个评估聊天基准上，持久负样本对抗性蒸馏在匹配的判别器计算下始终优于当前方法。它还产生了更平滑的新鲜策略判别器轨迹，减少了低于随机机会的波动。这些发现表明，判别器的负分布是黑箱在线策略蒸馏中的一个重要设计轴。

View on arXiv Download PDF AI Translation

cs.CL / 32 / 2609.30867

Evidence-Grounded Auditing of Identification Assumptions in Climate-Policy Causal Evaluations

基于证据的气候政策因果评估中的识别假设审计

Zhang, Yonghong, Xie, Yong, Parra, Isabel M., Correia, Ricardo

Abstract

Difference-in-differences (DID) studies are widely used to evaluate climate policy, but assessing the evidence supporting their identification assumptions remains challenging. We introduce ARGUS, a structured language-model pipeline that audits reported evidence against an eleven-dimension assumption-implication-evidence rubric and abstains when relevant evidence cannot be retrieved. We evaluate ARGUS using injected flaws, economics papers, and a small pilot with reconciled labels. On the 11-flaw benchmark, ARGUS detects 73% of planted flaws, compared with 18% for a keyword-based pipeline. Across 26 economics papers, ARGUS abstains on about 40% of paper-dimension assessments for lack of retrievable evidence. In a five-paper pilot with labels reconciled by two annotators, it assigns a higher risk level than the labels on 25 of the 33 assessments it completes. A rule fixed before the labels arrived removes most of this in-sample; weighted agreement stays low. ARGUS provides evidence-linked risk reports that localize potential weaknesses for expert review, without adjudicating causal claims. Code and data: https://github.com/yonghongzhang-io/ARGUS

Chinese Translation

差异中的差异（DID）研究广泛用于评估气候政策，但评估支持其识别假设的证据仍然具有挑战性。我们引入了ARGUS，一个结构化的语言模型管道，能够根据一个包含十一维假设-推论-证据标准的审计报告所述证据，并在无法检索到相关证据时选择不作判断。我们通过注入缺陷、经济学论文以及一个小规模的试点研究（使用了经过调和的标签）来评估ARGUS。在11个缺陷基准测试中，ARGUS检测到73%的植入缺陷，而基于关键词的管道仅检测到18%。在26篇经济学论文中，ARGUS因缺乏可检索证据而对约40%的论文-维度评估选择不作判断。在一个包含五篇论文的试点中，经过两位标注者调和的标签，ARGUS在33项评估中有25项赋予了高于标签的风险水平。在标签到达之前设定的规则消除了大部分样本内的偏差；加权一致性仍然较低。ARGUS提供与证据相关的风险报告，定位潜在的弱点以供专家审查，而不对因果主张进行裁定。代码和数据可在：https://github.com/yonghongzhang-io/ARGUS

View on arXiv Download PDF AI Translation

cs.CL / 33 / 2609.30882

Effects of Transcript Compression on LLM-based Medical Misinformation Detection in Japanese YouTube Videos

转录压缩对基于大型语言模型的日本YouTube视频医疗虚假信息检测的影响

Wake, Yuya, Tsugawa, Sho, Amagasa, Toshiyuki

Abstract

Large language models (LLMs) are increasingly used to assess long-form medical videos, but their effectiveness may depend on whether transcripts are provided in full or compressed through summarization, retrieval, or claim screening. This study examines how such transcript compression affects LLM-based veracity classification of Japanese medical YouTube videos. We compare four transcript input designs: full transcripts, LLM-generated summaries, RAPTOR-based retrievalaugmented generation (RAG), and Screening, which extracts candidate medical and health-related sentences. Using 74 long-form videos labeled as Real or Fake, we evaluate classification performance and analyze linguistic changes using J-LIWC, hedge expressions, and institutional or technical terms. The full-transcript Baseline achieved the best performance, whereas all compressed inputs increased false negatives, meaning that Fake videos were more likely to be misclassified as Real. Summary caused the largest performance drop, while Screening performed best among the compressed inputs but still omitted many medically relevant sentences. Linguistic analyses showed that these errors were not explained by a simple increase in certainty. Instead, Summary reduced affective, social, temporal, cognitive, and conversational cues, while Summary and RAG made institutional and technical terms more salient. These findings suggest that transcript compression can represent Fake videos as more coherent and authoritative inputs, thereby weakening cues needed for misinformation detection

Chinese Translation

大型语言模型（LLMs）在评估长篇医疗视频中的应用日益增多，但其有效性可能取决于转录文本是完整提供还是通过摘要、检索或声明筛选进行压缩。本研究探讨了转录压缩如何影响基于LLM的日本医疗YouTube视频的真实性分类。我们比较了四种转录输入设计：完整转录、LLM生成的摘要、基于RAPTOR的检索增强生成（RAG）和筛选，后者提取候选的医疗和健康相关句子。使用74个标记为真实或虚假的长篇视频，我们评估了分类性能，并使用J-LIWC分析语言变化、模糊表达以及机构或技术术语。完整转录基线达到了最佳性能，而所有压缩输入均增加了假阴性，这意味着虚假视频更可能被错误分类为真实。摘要导致了性能的最大下降，而筛选在压缩输入中表现最佳，但仍然遗漏了许多与医学相关的句子。语言分析表明，这些错误并不是由简单的确定性增加所解释的。相反，摘要减少了情感、社交、时间、认知和对话线索，而摘要和RAG则使机构和技术术语更加突出。这些发现表明，转录压缩可能使虚假视频表现得更连贯和权威，从而削弱了检测虚假信息所需的线索。

View on arXiv Download PDF AI Translation

cs.CL / 34 / 2609.30897

From annotation to reasoning: Culture in language models

从注释到推理：语言模型中的文化

Hershcovich, Daniel, Conroy, Alexander, Bjerring-Hansen, Jens

Abstract

How should we evaluate language models when more than one interpretation can be right? Cultural benchmarks often test factual knowledge, agreement with survey responses, or recognition of a predefined meaning. These tasks leave open whether a model can explain how a cultural reference works in a particular text, support a reading with evidence, or revise it after criticism. This is a question of interpretive depth, complementary to the breadth of cultural coverage. We argue that literary interpretation offers a useful setting for studying these capabilities. We focus on cultural referencing and reuse: how texts invoke, repeat, and transform earlier expressions across historical and linguistic contexts. Our central claim is that literary scholars can disagree about an interpretation while recognizing the quality of its support. We propose linking evidence-centered benchmarks, evaluation that preserves scholarly disagreement, and model-development experiments on literary data, contextual resources, and scholarly feedback. Danish literature provides a concrete starting point, with implications for other languages and domains. The aim is to develop alternative evaluation strategies that go beyond conventional benchmark metrics and guide model development toward cultural robustness in AI systems.

Chinese Translation

当多种解释都可能是正确的时，我们应该如何评估语言模型？文化基准通常测试事实知识、与调查回应的一致性或对预定义意义的识别。这些任务没有解决模型是否能够解释文化引用在特定文本中的运作方式、是否能够用证据支持某种解读，或在受到批评后进行修正。这是一个关于解释深度的问题，补充了文化覆盖的广度。我们认为，文学解读为研究这些能力提供了一个有用的背景。我们关注文化引用和再利用：文本如何在历史和语言背景中调用、重复和转化早期表达。我们的核心观点是，文学学者可以对某种解读存在分歧，同时承认其支持的质量。我们建议将以证据为中心的基准、保留学术分歧的评估以及基于文学数据、上下文资源和学术反馈的模型开发实验相结合。丹麦文学提供了一个具体的起点，并对其他语言和领域具有启示意义。我们的目标是开发超越传统基准指标的替代评估策略，引导模型开发朝向人工智能系统中的文化稳健性。

View on arXiv Download PDF AI Translation

cs.CL / 35 / 2609.30906

ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning

ToolSearcher：通过强化学习优化大规模工具选择

Dai, Zhenlong, Song, Xujie, Wang, Zitong, Niu, Tong, liu, Jian, Wang, Weiqiang, Tang, Xiu, Wu, Sai, Yao, Chang, Chen, Jingyuan

Abstract

Large language models (LLMs) excel at natural language processing but struggle to interact with external environments. Tool learning provides a promising way to extend LLMs into actionable agents, where tool selection is a critical prerequisite for successful tool use. Existing work often assumes a small or predefined set of tools, leaving large-scale tool selection underexplored. Real-world repositories contain a vast and diverse array of tools, making it difficult for LLMs to effectively search, distinguish, and compose tools under context-length constraints. We identify large-scale tool selection as a new challenge for agentic reinforcement learning, highlighting that existing RL methods for knowledge-based question answering are inadequate for selecting tools while considering compatibility. To address this challenge, we propose ToolSearcher, a novel RL framework for effective multi-turn search and fine-grained optimization in large-scale tool selection. Specifically, we introduce category-constrained tool discrimination to improve the model's ability to distinguish functionally similar tools, event-level search modeling to explicitly optimize the discovery of target tools during multi-turn search, and trajectory-aligned credit allocation to provide fine-grained reward signals for different stages of the search-selection process. Extensive experiments on large-scale tool selection benchmarks demonstrate that ToolSearcher consistently outperforms a set of strong baselines in challenging settings involving iterative search and complex tool composition.

Chinese Translation

大型语言模型（LLMs）在自然语言处理方面表现出色，但在与外部环境交互时却面临挑战。工具学习为将LLMs扩展为可执行代理提供了一种有前景的方法，其中工具选择是成功使用工具的关键前提。现有研究通常假设工具集较小或预定义，导致大规模工具选择的研究不足。现实世界的资源库包含大量多样化的工具，使得LLMs在上下文长度限制下有效搜索、区分和组合工具变得困难。我们将大规模工具选择确定为代理强化学习的新挑战，强调现有的强化学习方法在考虑兼容性时对工具选择的不足。为了解决这一挑战，我们提出了ToolSearcher，一种用于大规模工具选择的有效多轮搜索和细粒度优化的新型强化学习框架。具体而言，我们引入了类别约束工具区分，以提高模型区分功能相似工具的能力；引入事件级搜索建模，以明确优化多轮搜索中的目标工具发现；以及轨迹对齐的信用分配，以为搜索选择过程的不同阶段提供细粒度的奖励信号。在大规模工具选择基准上的广泛实验表明，ToolSearcher在涉及迭代搜索和复杂工具组合的挑战性设置中始终优于一组强基线。

View on arXiv Download PDF AI Translation

cs.CL / 36 / 2609.30924

Training-Free Pronunciation Transcription via Text-Constrained Acoustic Rescoring

无训练发音转录的文本约束声学重评分

Asano, Hikaru, Kubo, Yotaro, Kuroki, So

Abstract

Accurate and efficient pronunciation transcription is essential for preparing text-to-speech training data at scale. Existing approaches have different limitations: grapheme-to-pronunciation (G2P) and speech-to-pronunciation (S2P) methods each capture only partial information, using only text or only speech, while speech-and-text-to-pronunciation (ST2P) methods use both but require costly pronunciation-annotated data. To address this problem, we propose a training-free ST2P pipeline that integrates both lexical and acoustic information at inference time. Lexical resources and G2P tools generate text-constrained candidates, and a left-to-right greedy search selects the best one using whole-sequence negative log-likelihoods from frozen pretrained S2P models. On three Japanese corpora, our method reduces Character Error Rate (CER) from 0.60--1.40\% (text-only baseline) to 0.04--0.17\% with reference transcripts, and 0.64--1.58\% with ASR transcripts. It outperforms all baselines, including a trained ST2P model and commercial multimodal LLMs. Our greedy search method is 3--3.5$\times$ faster than beam search at similar CER, and the cascade is 2$\times$ faster than direct decoding ensuring the efficiency and accuracy. In Spanish, French, and preliminary English, it also surpasses four open multimodal LLMs and the best traditional methods.

Chinese Translation

准确高效的发音转录对于大规模准备文本到语音训练数据至关重要。现有方法各有局限：图形到发音（G2P）和语音到发音（S2P）方法仅捕捉部分信息，分别只使用文本或语音，而语音和文本到发音（ST2P）方法虽然同时使用两者，但需要昂贵的发音注释数据。为了解决这一问题，我们提出了一种无训练的ST2P管道，在推理时整合了词汇和声学信息。词汇资源和G2P工具生成文本约束候选项，左到右的贪婪搜索使用来自冻结的预训练S2P模型的全序列负对数似然选择最佳候选。在三个日语语料库上，我们的方法将字符错误率（CER）从0.60--1.40\%（仅文本基线）降低到0.04--0.17\%（参考转录），以及0.64--1.58\%（ASR转录）。它超越了所有基线，包括一个训练的ST2P模型和商业多模态LLM。我们的贪婪搜索方法在相似CER下比束搜索快3--3.5倍，而级联比直接解码快2倍，确保了效率和准确性。在西班牙语、法语和初步的英语中，它也超越了四个开放的多模态LLM和最佳传统方法。

View on arXiv Download PDF AI Translation

cs.CL / 37 / 2609.30935

Estimating and Orthogonalizing Unknown Pre-training Gradients for Continual Fine-tuning of Large Language Models

估计和正交化未知预训练梯度以实现大型语言模型的持续微调

Wang, Bing, Li, Changchun, Cai, Xin-Qiang, Wu, Lin Yuanbo, Li, Ximing, Niu, Gang, Sugiyama, Masashi

Abstract

Continual fine-tuning is essential for large language models (LLMs) to dynamically adapt to real-world environments, yet it inevitably suffers from catastrophic forgetting, particularly the performance degradation of previous tasks and LLMs' general-purpose knowledge. Although existing methods, such as orthogonal gradient projection, mitigate the forgetting across various fine-tuning tasks, they fundamentally fail to preserve pre-training LLMs' inherent general-purpose knowledge because the original data and gradients of off-the-shelf pre-training LLMs required by these methods are strictly unknown and highly diverse. To bridge this critical gap, we propose EoupCT, a novel framework designed to Estimate and Orthogonalize Unknown Pre-training gradients for Continual LLM fine-Tuning. Specifically, EoupCT estimates pre-training gradients by dynamically generating pseudo data that is most susceptible to forgetting for new tasks through a learnable soft prompt equipped with Gumbel-Softmax relaxation. Furthermore, we formulate a multi-objective optimization problem and introduce a first-order efficient Pareto optimizer that jointly optimizes LLM parameters and the soft prompt, rigorously enforcing orthogonality between new task updates and the estimated pre-training gradients. Extensive experiments across multiple LLMs demonstrate that EoupCT effectively preserves both task-specific proficiency and inherent general-purpose knowledge, successfully mitigating the catastrophic forgetting.

Chinese Translation

持续微调对于大型语言模型（LLMs）在动态适应现实环境中至关重要，但它不可避免地遭遇灾难性遗忘，特别是先前任务的性能下降和LLMs的通用知识的退化。尽管现有方法，如正交梯度投影，在不同的微调任务中减轻了遗忘，但它们在根本上未能保留预训练LLMs固有的通用知识，因为这些方法所需的现成预训练LLMs的原始数据和梯度严格未知且高度多样。为了解决这一关键问题，我们提出了EoupCT，一个旨在估计和正交化未知预训练梯度以实现持续LLM微调的新框架。具体而言，EoupCT通过动态生成最容易遗忘的新任务伪数据来估计预训练梯度，该伪数据通过配备Gumbel-Softmax松弛的可学习软提示生成。此外，我们构建了一个多目标优化问题，并引入了一种一阶高效的Pareto优化器，该优化器共同优化LLM参数和软提示，严格执行新任务更新与估计的预训练梯度之间的正交性。在多个LLM上的广泛实验表明，EoupCT有效地保留了任务特定的能力和固有的通用知识，成功减轻了灾难性遗忘。

View on arXiv Download PDF AI Translation

cs.CL / 38 / 2609.30968

FAVoR: Measuring and Mitigating Author-Style Homogenization in Federated Personalized Generation

FAVoR：测量和缓解联邦个性化生成中的作者风格同质化

Han, Lu, Zhang, Jingyao, Gero, Katy Ilonka, Tran, Nguyen H.

Abstract

Large language models are increasingly used as personalized writing assistants, but adapting a model across many authors can compromise individual writing style by pulling author-specific signals toward a shared register. Federated parameter-efficient fine-tuning (PEFT) offers a data-local setting for this multi-author adaptation problem: clients keep author text local while sharing compact adapter updates. However, we show that standard aggregation can preserve continuation utility while making different authors' generations less distinguishable in style space, a failure mode we define as author-style homogenization. We evaluate author-style retention with Angular Style Classification Encoder (ASCE)-based diagnostics on our main BlogText benchmark and ASCE-independent external authorship verification. Using this protocol, we find that common federated PEFT baselines can preserve semantic utility while averaging out author-specific signals. To address this homogenization, we instantiate FAVoR (Federated Authorial Voice Retention), an author-style residual mechanism for federated PEFT. FAVoR uses a shared-private adapter design: clients upload shared-adapter updates while retaining author-specific residual corrections locally. Across BlogText and external Mythos-Reddit validation, FAVoR improves author-style retention over standard and personalized federated PEFT baselines. These gains come with small continuation-utility trade-offs and are supported by component ablations, external verification, and cold-start transfer.

Chinese Translation

大型语言模型越来越多地被用作个性化写作助手，但在多个作者之间适配模型可能会通过将作者特定信号拉向共享的语域来损害个体写作风格。联邦参数高效微调（PEFT）为这一多作者适配问题提供了数据本地化的解决方案：客户端保留作者文本的本地性，同时共享紧凑的适配器更新。然而，我们发现标准聚合可以在保持延续效用的同时，使不同作者的生成在风格空间中变得不易区分，这种失败模式我们定义为作者风格同质化。我们通过基于角度风格分类编码器（Angular Style Classification Encoder, ASCE）的诊断在我们的主要 BlogText 基准和 ASCE 独立的外部作者验证中评估作者风格保留。使用这一协议，我们发现常见的联邦 PEFT 基线可以在平均化作者特定信号的同时保留语义效用。为了解决这种同质化问题，我们提出了 FAVoR（联邦作者声音保留），这是一个用于联邦 PEFT 的作者风格残差机制。FAVoR 采用共享-私有适配器设计：客户端上传共享适配器更新，同时在本地保留作者特定的残差修正。在 BlogText 和外部 Mythos-Reddit 验证中，FAVoR 在作者风格保留方面优于标准和个性化的联邦 PEFT 基线。这些提升伴随着小幅的延续效用权衡，并得到了组件消融、外部验证和冷启动转移的支持。

View on arXiv Download PDF AI Translation

cs.CL / 39 / 2609.30974

Coupled Usage-Sense Processes: Temporal and Attributable Lexical Semantic Change

耦合使用-意义过程：时间性和可归因的词汇语义变化

Ezoe, Haruka, Hisano, Ryohei

Abstract

Lexical semantic change is usually summarized by a scalar distance between independently sampled period distributions. This measures how much a word changed, but does not reveal when it changed, which mechanisms and component movements carried the change, or which usages support the attribution. We introduce Coupled Usage--Sense Processes (CUSP), which derives these answers from a single marginal preserving temporal process. A hierarchical coupling relates contextual distributions through latent usage components, while Markov composition makes adjacent and longer span correspondences compatible. Displacement operators quantify change magnitude and timing, split variation exactly between movement of component centers and reorganization within components, and attribute it to transported component pairs. Word-local modes resolve distinct directions of change and their activity over time, while representative passages from attributed components ground the analysis in text. Under a Gaussian mixture specialization, we prove parametric recovery of the operators and squared distances. Synthetic experiments support the predicted rate. CUSP remains competitive on English and German DWUG and recovers controlled Janus profiles while maintaining compositionally coherent transport. A large corpus of US court opinions demonstrates transition, mode, and passage attribution in unlabeled natural text. CUSP thus makes magnitude, timing, mechanism, movement, modes, and textual evidence compatible views of one lexical history.

Chinese Translation

词汇语义变化通常通过独立抽样的时间分布之间的标量距离来概括。这种方法测量了一个词的变化程度，但并未揭示变化发生的时间、哪些机制和组成部分推动了变化，以及哪些用法支持这一归因。我们引入了耦合使用-意义过程（Coupled Usage--Sense Processes, CUSP），该方法通过单一的边际保持时间过程来推导这些答案。一个层次耦合通过潜在的使用组件关联上下文分布，而马尔可夫组合使相邻和更长跨度的对应关系兼容。位移算子量化变化的幅度和时机，精确划分组件中心的运动和组件内部的重组，并将其归因于传输的组件对。词汇局部模式解析变化的不同方向及其随时间的活动，而来自归因组件的代表性段落则将分析扎根于文本。在高斯混合特化下，我们证明了算子的参数恢复和平方距离。合成实验支持预测的速率。CUSP在英语和德语的DWUG上保持竞争力，并在保持组成一致的传输的同时恢复受控的雅努斯（Janus）轮廓。大量美国法院意见的语料库展示了在未标记自然文本中的转变、模式和段落归因。因此，CUSP使得幅度、时机、机制、运动、模式和文本证据成为一个词汇历史的兼容视角。

View on arXiv Download PDF AI Translation

cs.CL / 40 / 2609.30984

THA: Weighted Finite-State Text Normalization and Inverse Text Normalization for Khmer

THA：针对高棉语的加权有限状态文本规范化与逆文本规范化

Yath, Seanghay

Abstract

Text-to-speech needs written text in spoken form, and speech recognition output needs the reverse. For Khmer, neither direction has a maintained open-source tool, and the script makes both harder: words are not separated by spaces, and number words occur inside ordinary words. We present Tha, a Khmer text normalization and inverse text normalization toolkit built from weighted finite-state transducers. It segments and classifies a whole line in one shortest-path search, and a second transducer rejects token boundaries inside a Khmer syllable. On Google's Khmer test suite, Tha agrees with the reference on all 274 cardinals up to one spelling variant, and on 2,906 real TTS prompts, 153 of the 158 sentences it rewrites are correct. Tha is open source under the Apache 2.0 license.

Chinese Translation

文本到语音需要以口语形式呈现的书面文本，而语音识别的输出则需要反向处理。对于高棉语，这两个方向都没有维护的开源工具，并且其书写系统使得这两者的实现更加困难：单词之间没有空格，数字词出现在普通单词中。我们提出了Tha，一个基于加权有限状态转换器的高棉语文本规范化和逆文本规范化工具包。它通过一次最短路径搜索对整行文本进行分段和分类，第二个转换器则在高棉音节内部拒绝标记边界。在谷歌的高棉语测试套件中，Tha在所有274个基数上与参考结果一致，最多有一个拼写变体；在2906个真实的文本到语音提示中，它重写的158个句子中有153个是正确的。Tha在Apache 2.0许可证下开源。

View on arXiv Download PDF AI Translation

cs.CL / 41 / 2609.30986

Evaluating Sycophancy in Chinese Large Language Models on Factual Questions Derived from Online Search Queries

评估中国大型语言模型在基于在线搜索查询的事实问题中的谄媚行为

Liu, Geng, Li, Feng, Zhu, Mengxiao, Pierri, Francesco

Abstract

As large language models increasingly mediate information access, factually accurate and independent answers are critical. However, these models can exhibit sycophancy by aligning their responses with users' stated beliefs even when those beliefs are incorrect, potentially presenting misinformation as independently verified and reinforcing users' confidence in false claims. Prior work leaves unresolved whether introducing user beliefs causes correct responses to become incorrect or uncertain, or causes uncertain responses to become belief-aligned incorrect answers. It also remains unclear whether anti-sycophancy interventions preserve or restore factual accuracy or merely shift responses toward uncertainty. We analyze factual sycophancy in Chinese-language information seeking using yes/no fact-checking questions. Our analysis covers 364,941 responses from three frontier Chinese-based LLMs (DeepSeek, Qwen, and Doubao) to 12,165 factual questions derived from real-world Chinese search queries. We evaluate the models with and without reasoning across baseline, belief-conditioned, and anti-sycophancy prompting, tracing matched shifts among correct, incorrect, and uncertain responses. Under incorrect user beliefs, we distinguish belief-aligned errors from losses of factual confidence, in which initially correct answers become uncertain. Patterns vary across models and reasoning settings: reasoning is not a consistent safeguard, and anti-sycophancy instructions can reduce incorrect agreement while increasing uncertainty. In Chinese-language factual question answering, avoiding agreement with false beliefs is therefore not equivalent to preserving factual accuracy, highlighting the value of transition-level evaluation. Such behavior may undermine the reliability of LLM-mediated information access by reinforcing misinformation or weakening users' confidence in factually correct answers.

Chinese Translation

随着大型语言模型在信息获取中扮演越来越重要的角色，事实准确且独立的回答显得尤为关键。然而，这些模型可能会表现出谄媚行为，通过将其回答与用户所表达的信念对齐，即使这些信念是错误的，从而可能将错误信息呈现为独立验证的内容，并增强用户对虚假主张的信心。先前的研究尚未解决引入用户信念是否导致正确回答变为错误或不确定，或导致不确定回答变为与信念一致的错误答案。此外，尚不清楚反谄媚干预是否能够保持或恢复事实准确性，还是仅仅将回答转向不确定性。我们使用是/否事实核查问题分析中文信息检索中的事实谄媚行为。我们的分析涵盖了来自三种前沿中文大型语言模型（DeepSeek、Qwen 和 Doubao）对12,165个源自真实中文搜索查询的事实问题的364,941个回答。我们在基线、信念条件和反谄媚提示下评估模型的推理能力，追踪正确、错误和不确定回答之间的匹配变化。在用户信念错误的情况下，我们区分了与信念一致的错误与事实信心的丧失，其中最初正确的回答变得不确定。不同模型和推理设置下的模式各异：推理并不是一个一致的保障，而反谄媚指令可以减少错误一致性，同时增加不确定性。因此，在中文事实问答中，避免与错误信念达成一致并不等同于保持事实准确性，这突显了过渡级别评估的价值。这种行为可能通过强化错误信息或削弱用户对事实正确答案的信心，从而削弱大型语言模型介导的信息获取的可靠性。

View on arXiv Download PDF AI Translation

cs.CL / 42 / 2609.31002

ZooWork-ShopRanker: An Open, Preference-Aligned E-Commerce Reranker

ZooWork-ShopRanker：一个开放的、偏好对齐的电子商务重排序模型

Xue, Siqiao, Liu, Shuxuan, Hu, Ning

Abstract

Open rerankers trained for general web retrieval transfer imperfectly to e-commerce, where ranking decisions depend not only on topical relevance but also on user preferences, product constraints, and comparative product fit. These preference signals are difficult to supervise at scale: real search traffic provides authentic queries and candidates but no clean pairwise labels. We present ZooWork-ShopRanker, a family of e-commerce rerankers (0.6B, 4B, and 8B) aligned to judge-labeled shopping preference. Training pairs are labeled by a panel of reasoning large language models (LLMs) from different families acting as a preference oracle, with position-debiased judgments and agreement tiers, and the rerankers are trained on these labels. The aligned 8B flagship then serves as a distillation teacher for the efficient 4B and 0.6B models, which are fit to its scores and sharpened on judged pairs. To measure progress, we introduce ShopRank-Bench, a contamination-limited benchmark of ~10,000 private-traffic preference pairs in both text formats, tiered by how many judge families committed to each label. ZooWork-ShopRanker-8B and -4B significantly outperform the strongest open reranker baseline, every model significantly beats its own un-aligned base, and ZooWork-ShopRanker-0.6B beats its size peer; the gains hold in both formats and extend to common MTEB benchmarks. We release the models and the dual-format ShopRank-Bench to facilitate further research.

Chinese Translation

为一般网络检索训练的开放重排序模型在电子商务中的迁移效果不佳，因为排名决策不仅依赖于主题相关性，还受到用户偏好、产品限制和比较产品适配性的影响。这些偏好信号在大规模监督下难以获取：真实的搜索流量提供了真实的查询和候选项，但没有干净的成对标签。我们提出了ZooWork-ShopRanker，一个与评审标签购物偏好对齐的电子商务重排序模型系列（0.6B、4B和8B）。训练对由来自不同家族的大型语言模型（LLMs）小组标注，作为偏好神谕，提供位置去偏见的判断和一致性层级，重排序模型在这些标签上进行训练。对齐的8B旗舰模型随后作为高效的4B和0.6B模型的蒸馏教师，这些模型适应其得分并在评审对上进行优化。为了衡量进展，我们引入了ShopRank-Bench，这是一个污染限制的基准，包含约10,000个私有流量偏好对，按承诺每个标签的评审家族数量分级。ZooWork-ShopRanker-8B和-4B显著超越了最强的开放重排序基线，每个模型都显著优于其未对齐的基础模型，而ZooWork-ShopRanker-0.6B则超越了同类模型；这些提升在两种格式中均有效，并扩展到常见的MTEB基准。我们发布了模型和双格式的ShopRank-Bench，以促进进一步的研究。

View on arXiv Download PDF AI Translation

cs.CL / 43 / 2609.31009

G$^2$PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation

G$^2$PTQ：通过广义梯度补偿改善大型语言模型的后训练量化

Liu, Ruikang, Bai, Haoli, Sun, Yuxuan, Zhang, Qian, Cai, Wenzheng, Hao, Yanqi, Wang, Feiyu, Zhong, Weidong, Wang, Zhuang, Yang, Tong, Zhou, Xiangsheng

Abstract

Post-training quantization (PTQ) is a practical approach to reducing the memory and computational footprint of large language models (LLMs) without retraining. GPTQ-based methods have become the de facto standard, yet they suffer from two complementary limitations. Methods with local, layer-wise objectives lack global supervision; while methods with global objectives fix their Hessian estimates at the start and ignore first-order gradients, so their guidance grows stale as quantization proceeds. This paper presents G$^2$PTQ, a unified PTQ framework with Generalized Gradient Compensation that integrates both first- and second-order information under a globally supervised, block-wise optimization objective. By refreshing gradient and Hessian estimates before quantizing each Transformer block, G$^2$PTQ avoids the staleness of prior global methods. Furthermore, to stabilize the exact first-order compensation, we introduce a trust-region scaling mechanism that dynamically bounds the gradient step to prevent exploding weight updates. Finally, we derive efficient implementations for block-wise Hessian approximation and exact gradient compensation. Experimental results on various model families and bit-widths demonstrate that G$^2$PTQ enables better alignment with the full-precision model, outperforming state-of-the-art baselines. Code is available at: https://github.com/G2PTQ/G2PTQ.

Chinese Translation

后训练量化（PTQ）是一种实用的方法，用于在不重新训练的情况下减少大型语言模型（LLMs）的内存和计算开销。基于GPTQ的方法已成为事实上的标准，但它们存在两个互补的局限性。具有局部、逐层目标的方法缺乏全局监督；而具有全局目标的方法在开始时固定其Hessian估计，并忽略一阶梯度，因此随着量化的进行，它们的指导信息变得陈旧。本文提出了G$^2$PTQ，这是一种统一的PTQ框架，采用广义梯度补偿，整合了一阶和二阶信息，基于全局监督的块级优化目标。通过在量化每个Transformer块之前刷新梯度和Hessian估计，G$^2$PTQ避免了先前全局方法的陈旧性。此外，为了稳定精确的一阶补偿，我们引入了一种信任区域缩放机制，动态限制梯度步长，以防止权重更新的爆炸。最后，我们推导了块级Hessian近似和精确梯度补偿的高效实现。在各种模型系列和位宽上的实验结果表明，G$^2$PTQ能够更好地与全精度模型对齐，超越了最先进的基线。代码可在：https://github.com/G2PTQ/G2PTQ获取。

View on arXiv Download PDF AI Translation

cs.CL / 44 / 2609.31046

Modeling Student Sensemaking with LLMs and Knowledge-Graph-Guided Inference

利用大型语言模型和知识图谱引导推理对学生意义建构的建模

Alacam, Özge, Güneş, Zübeyde Demet Kirbulut, Ekici, Funda, Turan-Oluk, Nurcan, Dinçdemir, Dilay, Kadayıfçı, Hakkı, Yeşiloğlu, Sevinç Nihal, Işık, Burcu, Tümay, Halil, Gencer, Sinem

Abstract

Collaborative science learning requires nuanced interpretation of student dialogue to characterize how learners identify knowledge gaps, build explanations, and work toward resolution - a theory-driven analysis that is labor-intensive and difficult to scale. We investigate whether instruction-tuned large language models (LLMs) can support multidimensional analysis of collaborative sensemaking without task-specific training, and whether structured knowledge-state information improves model inference. We evaluate two mid-size LLMs on 23 richly annotated, expert-labeled episodes across prompting conditions that vary definitional scaffolding, reasoning mode, and turn structure. Without reasoning, models tend to overpredict successful sensemaking; reasoning-enabled prompting improves identification of unsuccessful cases. Knowledge-state diagnostics provide additional grounding, improving detection of unsuccessful sensemaking and increasing agreement with expert annotations. No single configuration performs best across all sensemaking dimensions, underscoring the multidimensional nature of the task.

Chinese Translation

协作科学学习需要对学生对话进行细致的解读，以表征学习者如何识别知识空白、构建解释并朝着解决方案努力——这是一种理论驱动的分析，劳动强度大且难以扩展。我们研究了经过教学调优的大型语言模型（LLMs）是否能够在没有特定任务训练的情况下支持协作意义建构的多维分析，以及结构化知识状态信息是否能改善模型推理。我们在23个丰富注释、专家标注的案例上评估了两种中型LLMs，这些案例在定义性支架、推理模式和轮次结构等提示条件上有所不同。没有推理时，模型往往会过度预测成功的意义建构；启用推理的提示改善了对不成功案例的识别。知识状态诊断提供了额外的基础，改善了对不成功意义建构的检测，并增加了与专家注释的一致性。没有单一配置在所有意义建构维度上表现最佳，强调了该任务的多维特性。

View on arXiv Download PDF AI Translation

cs.CL / 45 / 2609.31062

CG-Probes: Recovering Guardrail Directions from Patient Query Embeddings

CG-探针：从患者查询嵌入中恢复护栏方向

Řeháček, Marko, Dušek, Vítězslav, Rusinko, Martin, Nováček, Vít

Abstract

Patient-facing AI assistants promise valuable support to patients, but incoming queries can pose medical risks. To create guardrails, we work with oncologists to define three ordinal risk axes: Medical Urgency, Psychological Urgency, and Topic Sensitivity. We propose Clinical Guardrail Probes (CG-Probes) to measure the risks from query embeddings. We probe for each axis in the normalized embedding space of frozen embedders via the difference-in-means method, treating each axis as a potential linear direction. To train the probes, we cluster 79,658 Czech oncology search queries with BERTopic and use these clusters to generate pairs of queries with contrastive risk levels via few-shot prompting. We evaluate the approach on 200 queries (90 real, 110 synthetic), each graded by two oncologists, against two open-weight LLMs and a frontier LLM. We find that urgency-based axes are recoverable as linear directions, and the probes are competitive with open-weight LLMs (no significant differences in quadratic-weighted kappa) at a fraction of the latency. Each axis yields a scalar score that clinicians can inspect and use to set escalation thresholds. The pipeline requires only search logs, axis definitions, and black-box access to the embedding model, suggesting transferability across healthcare domains. Robust validation on new queries and axes remains future work.

Chinese Translation

面向患者的人工智能助手承诺为患者提供有价值的支持，但输入的查询可能会带来医疗风险。为了创建护栏，我们与肿瘤学家合作定义了三个有序风险轴：医疗紧迫性、心理紧迫性和主题敏感性。我们提出了临床护栏探针（Clinical Guardrail Probes, CG-Probes）来测量查询嵌入的风险。我们通过均值差异法在冻结的嵌入模型的标准化嵌入空间中探测每个轴，将每个轴视为潜在的线性方向。为了训练探针，我们使用BERTopic对79,658个捷克肿瘤学搜索查询进行聚类，并利用这些聚类通过少量示例提示生成具有对比风险水平的查询对。我们在200个查询（90个真实，110个合成）上评估该方法，每个查询由两位肿瘤学家评分，针对两个开放权重的大型语言模型（LLMs）和一个前沿LLM。我们发现基于紧迫性的轴可以作为线性方向恢复，且探针在延迟较低的情况下与开放权重的LLMs具有竞争力（在二次加权kappa上没有显著差异）。每个轴产生一个标量分数，临床医生可以检查并用来设定升级阈值。该流程仅需搜索日志、轴定义和对嵌入模型的黑箱访问，表明其在医疗领域的可转移性。对新查询和轴的稳健验证仍然是未来的工作。

View on arXiv Download PDF AI Translation

cs.CL / 46 / 2609.31122

LocUS: Head Selection and Subspace Projection for Targeted Activation Steering

LocUS：针对目标激活引导的头选择与子空间投影

Tallini, Irene, Basile, Lorenzo, Maiorca, Valentino, Locatello, Francesco, Cazzaniga, Alberto

Abstract

Activation steering is a powerful training-free paradigm for controlling large language models at inference time. However, standard approaches estimate a per-layer steering direction from contrastive data and apply it on the layer's entire representation space, which may couple the intervention to off-target properties present in the contrastive data and degrade unrelated capabilities. To mitigate this issue, we introduce LocUS (Localized Unembedding Steering), a method which grounds activation steering to the model's own output vocabulary subspace. By identifying a property-specific linear subspace within the unembedding matrix, LocUS enforces a geometric constraint that restricts the steering transformation to a specific subspace and at the same time localizes its application to a sparse subset of attention heads. Extensive evaluations across three model families on toxicity mitigation, sentiment redirection and sycophancy suppression show that LocUS matches or outperforms state-of-the-art baselines while intervening on under 6% of parameters and better preserving general capability.

Chinese Translation

激活引导是一种强大的无训练范式，用于在推理时控制大型语言模型。然而，标准方法从对比数据中估计每层的引导方向，并将其应用于该层的整个表示空间，这可能会将干预与对比数据中存在的非目标特性耦合，从而降低与之无关的能力。为了解决这个问题，我们提出了LocUS（局部去嵌入引导），一种将激活引导与模型自身的输出词汇子空间相结合的方法。通过在去嵌入矩阵中识别特定属性的线性子空间，LocUS强制施加几何约束，将引导变换限制在特定子空间内，同时将其应用局限于稀疏的注意力头子集。在对三种模型系列进行的广泛评估中，涉及毒性缓解、情感重定向和谄媚抑制，结果表明LocUS的表现与最先进的基线相当或更优，同时干预的参数不足6%，并更好地保留了模型的通用能力。

View on arXiv Download PDF AI Translation

cs.CL / 47 / 2609.31130

Do we need to answer that question? Salience and Answerability of Potential Questions in Naturalistic Dialogue

我们需要回答那个问题吗？自然对话中潜在问题的显著性与可回答性

Decker, Amandine, Amblard, Maxime, Breitholtz, Ellen

Abstract

We empirically investigate Question Under Discussion based modelling in naturalistic dialogue by studying whether the salience of generated potential questions predicts their subsequent resolution. Building on Wu et al. (2024), we construct a dataset of 7,124 questions automatically generated from utterances and preceding context from the British National Corpus, and annotated for salience and answerability. We find a robust but low positive correlation between salience and answerability in dialogue, indicating that more salient questions are more likely to be addressed. However, this effect is markedly weaker than in monologic text, suggesting that conversational structure is less predictable. We further observe that structured interactions exhibit stronger alignment between annotators than less organised dialogues.

Chinese Translation

我们通过实证研究自然对话中的讨论问题建模，探讨生成的潜在问题的显著性是否能够预测其后续的解决。基于Wu等人（2024）的研究，我们构建了一个包含7124个问题的数据集，这些问题是从英国国家语料库中的话语及其前文上下文自动生成的，并进行了显著性和可回答性的标注。我们发现对话中显著性与可回答性之间存在稳健但较低的正相关关系，表明显著性更高的问题更可能被解决。然而，这一效应明显弱于单一文本，暗示对话结构的可预测性较低。我们进一步观察到，结构化的互动中标注者之间的对齐程度高于较少组织的对话。

View on arXiv Download PDF AI Translation

cs.CL / 48 / 2609.31169

Improving Visual Sensitivity of LLMs on Multimodal Machine Translation with Metric-based Loss Weighting

通过基于度量的损失加权提高大语言模型在多模态机器翻译中的视觉敏感性

Mąka, Paweł, Andruszkiewicz, Piotr, Semerci, Yusuf Can, Scholtes, Jan, Spanakis, Gerasimos

Abstract

Multimodal Machine Translation aims to incorporate additional signal from non-textual modalities to improve translations by resolving ambiguities. While models, through multimodal fusion, are able to accept images related to the source text, they can ignore this information. Therefore, increasing their visual sensitivity remains an active research area. In this work, we introduce a training method, Metric-based Loss Weighting, that improves visual grounding of translations by increasing the loss function for tokens that benefit from the accompanying image. We identify these tokens using the Point-wise Cross-mutual Information (PCXMI) metric, which compares the model's output probabilities with and without visual context. We introduce a Congruency-based PCXMI metric and experimentally show that both metrics working in combination yield the best results. We evaluate our method by fine-tuning three pretrained Multimodal Large Language Models on the task of Image-guided Machine Translation for three language directions. Metric-based Loss Weighting outperforms other tested methods on the CoMMuTE contrastive dataset, improving accuracy by up to more than 7 percentage points compared to standard fine-tuning, while maintaining strong general translation performance.

Chinese Translation

多模态机器翻译旨在通过整合来自非文本模态的额外信号来改善翻译，从而解决歧义。尽管模型通过多模态融合能够接受与源文本相关的图像，但它们可能会忽略这些信息。因此，提高模型的视觉敏感性仍然是一个活跃的研究领域。在本研究中，我们提出了一种训练方法——基于度量的损失加权，通过增加受益于伴随图像的标记的损失函数来改善翻译的视觉基础。我们使用点对点交叉互信息（Point-wise Cross-mutual Information, PCXMI）度量来识别这些标记，该度量比较模型在有视觉上下文和没有视觉上下文时的输出概率。我们引入了一种基于一致性的PCXMI度量，并通过实验证明这两种度量结合使用能够产生最佳结果。我们通过对三种语言方向的图像引导机器翻译任务对三种预训练的多模态大语言模型进行微调来评估我们的方法。基于度量的损失加权在CoMMuTE对比数据集上的表现优于其他测试方法，准确率比标准微调提高了超过7个百分点，同时保持了强大的通用翻译性能。

View on arXiv Download PDF AI Translation

cs.CL / 49 / 2609.31181

Where a Model Sends Its Own Repeated Token

模型如何发送其自身的重复标记

Zúñiga, Nicolás Vera

Abstract

Black-box model identification works by scoring a model's response to natural-language prompts. One line of work feeds models a degenerate input -- their own token, repeated -- to find a failure mode rather than an identity. We take that input and ask where the model goes when it does not. For each token t, read argmax p(. | t, t) in one forward pass; the result is a map on the whole vocabulary, with two halves. The first -- which tokens are fixed points -- is partially anticipated, and we report it as a failed estimand: the natural distance on it is 83% cardinality, separates a corpus manipulation by two bits in 3471 against a precision floor of zero, and attributes families at 0.5833. The second half, where the map sends tokens that are not fixed points, is unrecorded; the one paper holding those tokens logged them as a zero. Pairing on the source token removes the cardinality confound by construction (r from 0.9128 to -0.0932) and attributes families at 0.8333 -- twelve models scored against a pool of nineteen -- with chance 0.1389, across seven tokenizer groups and several corpora. Two nulls clear it: frequency-matched destinations agree at 0.1429, independent marginals at 0.0798. Family predicts agreement better than tokenizer (0.2031 against 0.1205), and recurrent architectures cluster at balanced accuracy 1.0 against a 0.7895 majority rate, or 0.90 once each model's dominant destination is excluded -- the figure we stand behind. We measure the robustness envelope: 8-bit weight rounding moves the map less than deduplicating the training corpus does (0.9004 against 0.6353, on one support), 4-bit destroys it (0.0098; 0.1812 at deployment granularity, so not a coarseness artefact), and the precision floor varies by model from 0.201 to 0.9778. All estimands and kill conditions were registered before the data, and the failed one is reported as fully as the surviving one.

Chinese Translation

黑箱模型识别通过对模型对自然语言提示的响应进行评分来工作。一项研究通过向模型输入退化的输入——其自身的标记重复——来寻找失败模式而非身份。我们采用该输入并询问当模型不这样做时会去哪里。对于每个标记 t，在一次前向传递中读取 argmax p(. | t, t)；结果是一个关于整个词汇表的映射，分为两个部分。第一部分——哪些标记是固定点——是部分可预见的，我们将其报告为一个失败的估计量：其自然距离为 83% 基数，在 3471 中通过两个比特分隔一个语料库操作，且归属家庭为 0.5833。第二部分，即映射将非固定点标记发送到哪里，则未被记录；唯一一篇包含这些标记的论文将其记录为零。源标记的配对通过构造消除了基数混淆（r 从 0.9128 降至 -0.0932），并将家庭归属为 0.8333——对十九个模型池中的十二个模型进行评分——机会为 0.1389，涵盖七个分词器组和多个语料库。两个零假设清除了这一点：频率匹配的目的地一致性为 0.1429，独立边际为 0.0798。家庭对一致性的预测优于分词器（0.2031 对 0.1205），而递归架构在平衡准确率 1.0 下聚类，相较于 0.7895 的多数率，或者在排除每个模型的主导目的地后为 0.90——这是我们所支持的数字。我们测量了鲁棒性范围：8 位权重四舍五入对映射的影响小于去重训练语料库的影响（在一个支持下为 0.9004 对 0.6353），4 位则摧毁了它（0.0098；在部署粒度下为 0.1812，因此不是粗糙的伪影），且不同模型的精度下限从 0.201 到 0.9778。所有估计量和终止条件在数据之前均已注册，失败的估计量与存活的估计量一样全面地报告。

View on arXiv Download PDF AI Translation

cs.CL / 50 / 2609.31245

RupeeBias: Auditing Demographic Bias in Indian Economic Guidance from Large Language Models

RupeeBias：审计大型语言模型在印度经济指导中的人口统计偏见

Nair, Pavithra P M, Talaviya, Bhavik, Bhushan, Shourya, Pankajakshan, Rahul, Guruvadoo, Seema, Agarwal, Avinash, Gressel, Gilad, Achuthan, Krishnashree

Abstract

Individuals turn to large language models (LLMs) for guidance across a wide range of economic tasks, from comparing loan options and planning savings to deciding what raise to ask for or how much to charge for their services. LLMs are known to reproduce social biases, and biased economic guidance may influence what users believe they are worth, what they ask for, and what they ultimately accept. This risk is especially salient in India, where economic outcomes are shaped by demographic categories such as caste and urban-rural location. Existing LLM bias benchmarks, however, are largely designed around Western demographic categories and therefore miss key axes of economic disparity in the Indian context. We introduce RupeeBias, a benchmark for auditing demographic bias in LLM-generated economic guidance across Indian economic settings. RupeeBias consists of 39,150 prompts spanning four use cases: salary estimation, salary increment estimation, counter-offer recommendation, and service pricing recommendation. The benchmark follows a single-attribute counterfactual design, holding the description of the user's qualifications, experience, or service offering fixed while varying one demographic identifier at a time. RupeeBias covers 87 India-specific demographic identifiers across six axes: caste, religion, regional identity, gender, disability, and urban-rural location, with all prompts constructed in both English and Hinglish. We evaluate nine LLMs on RupeeBias and find systematic demographic disparities across all six axes. For otherwise identical prompts that differ only in demographic identifier, LLM-generated economic outputs differ by 20.2% on average. We publicly release RupeeBias to support future research on demographic bias in LLM-generated economic guidance across India-specific demographic and economic contexts.

Chinese Translation

个人在各种经济任务中寻求大型语言模型（LLMs）的指导，从比较贷款选项和规划储蓄，到决定要求多少加薪或为其服务收取多少费用。已知LLMs会再现社会偏见，而有偏见的经济指导可能会影响用户对自身价值的认知、他们的要求以及最终接受的条件。这一风险在印度尤为突出，因为经济结果受到种姓和城乡位置等人口统计类别的影响。然而，现有的LLM偏见基准主要围绕西方人口统计类别设计，因此在印度背景下遗漏了经济差异的关键维度。我们提出了RupeeBias，这是一个用于审计LLM生成的经济指导中人口统计偏见的基准，涵盖印度经济环境。RupeeBias包含39,150个提示，涵盖四个使用案例：薪资估算、薪资增幅估算、反报价建议和服务定价建议。该基准采用单属性反事实设计，固定用户的资格、经验或服务提供描述，同时一次改变一个人口统计标识符。RupeeBias涵盖了87个印度特定的人口统计标识符，分为六个维度：种姓、宗教、地域身份、性别、残疾和城乡位置，所有提示均以英语和Hinglish构建。我们在RupeeBias上评估了九个LLM，发现所有六个维度存在系统性的人口统计差异。对于在其他方面相同但仅在人口统计标识符上存在差异的提示，LLM生成的经济输出平均差异为20.2%。我们公开发布RupeeBias，以支持未来在印度特定人口统计和经济背景下对LLM生成的经济指导中的人口统计偏见的研究。

View on arXiv Download PDF AI Translation

cs.CL / 51 / 2609.31255

PIA: A Personal Intelligence Agent Turning Health Conversations into Records and Records into Understanding

PIA：一个个人智能代理，将健康对话转化为记录，并将记录转化为理解

Yoon, Jeonghun, Kim, Dongchan, Yu, Hongyeon, Kim, Young-Bum, Choo, Jaegul

Abstract

General-purpose agent memory summarizes conversations: it extracts salient snippets, embeds them, and retrieves the top-k into the prompt. A health agent cannot run on summaries: a dose becomes a sentence, "since last week" is resolved at the model's discretion, and a three-month glucose trend cannot be answered by text similarity. We present PIA, a personal intelligence agent deployed alongside a consumer health agent. PIA receives the agent's natural-language requests, decides for itself whether and how to write or read, and turns conversations into typed clinical records and records into a synthesized understanding of the user. Its memory harness consists of four controls -- extraction, memory, retrieval, and understanding -- each a domain-agnostic mechanism with a pluggable health module: schema, medical alias dictionary, knowledge graph, and temporal rules. We show how the same query receives a different answer as the memory injected into the response context deepens from one-dimensional recall, to a two-dimensional health snapshot, to a three-dimensional trajectory with causality, and report lessons from operation: self-reported health data are missing not at random, question phrasing governs the quality of synthesized understanding, and nearly a third of candidate causal links are structural noise that rules alone remove.

Chinese Translation

通用代理记忆总结对话：它提取重要片段，将其嵌入，并将前k个片段检索到提示中。健康代理无法仅依赖总结：剂量变成一句话，“自上周以来”的时间范围由模型自行决定，而三个月的葡萄糖趋势无法通过文本相似性回答。我们提出了PIA，一个与消费者健康代理一起部署的个人智能代理。PIA接收代理的自然语言请求，自主决定是否以及如何进行写入或读取，并将对话转化为输入的临床记录，将记录转化为对用户的综合理解。它的记忆结构由四个控制组成——提取、记忆、检索和理解——每个都是领域无关的机制，配备可插拔的健康模块：模式、医学别名词典、知识图谱和时间规则。我们展示了同一查询如何随着注入响应上下文的记忆从一维回忆、二维健康快照到三维因果轨迹而获得不同的答案，并报告了操作中的经验教训：自我报告的健康数据并非随机缺失，问题措辞影响综合理解的质量，近三分之一的候选因果链接是结构噪声，仅靠规则无法消除。

View on arXiv Download PDF AI Translation

cs.CL / 52 / 2609.31261

MoSAR: Mixture of Semantic Attention Regimes for Learning Adaptive and Approximable Attention Geometries

MoSAR：用于学习自适应和可近似注意几何的语义注意机制混合

Paolicelli, Michele, Petruzzelli, Alessandro, Martina, Alessandro Franceso Maria, Musto, Cataldo, Semeraro, Giovanni

Abstract

The quadratic complexity of dense self-attention remains a central bottleneck for long-context language modeling. Many efficient alternatives address this cost by deciding in advance where attention should be sparse or local. We argue that attention approximation should instead be approached as a geometric problem, with the relevant interaction geometry learned from data: natural-language dependencies are input-dependent and difficult to prescribe in advance, so the model should learn where positional relevance can decay and where broader interactions must be preserved. We introduce Mixture of Semantic Attention Regimes (MoSAR), which learns such an adaptive, controlled-decay geometry over query--key interactions. Input-conditioned query and key routers, applied after positional encoding, select mixtures over short, medium, and global regimes, inducing a continuous distance-dependent attention field rather than a fixed sparsity pattern. This geometry is learned during training and can subsequently be discretized through top-1 routing. In controlled pre-training experiments with matched 500M-parameter models, MoSAR learns a substantially lower-reach attention geometry without degrading language-modeling quality, improving perplexity over dense RoPE at the training context length. Under length extrapolation, MoSAR achieves the best perplexity among all evaluated variants, including strong baselines such as ALiBi. Moreover, the learned geometry remains stable under deterministic top-1 discretization, suggesting that it is not only adaptive, but also amenable to low-cost approximation at inference time.

Chinese Translation

密集自注意力的二次复杂性仍然是长上下文语言建模的一个主要瓶颈。许多高效的替代方案通过提前决定注意力应该稀疏或局部来解决这一成本。我们认为，注意力近似应该被视为一个几何问题，相关的交互几何应从数据中学习：自然语言依赖关系是输入依赖的，难以提前规定，因此模型应学习位置相关性可以衰减的地方以及必须保留更广泛交互的地方。我们引入了语义注意机制混合（Mixture of Semantic Attention Regimes, MoSAR），它学习在查询-键交互中自适应的、受控衰减的几何。输入条件的查询和键路由器在位置编码后应用，选择短、中和全局机制的混合，从而诱导出一个连续的距离依赖注意场，而不是固定的稀疏模式。这种几何在训练过程中学习，并可以通过 top-1 路由进行离散化。在与匹配的 500M 参数模型的受控预训练实验中，MoSAR 学习到了一种显著较低覆盖范围的注意几何，而不降低语言建模质量，在训练上下文长度上改善了 dense RoPE 的困惑度。在长度外推下，MoSAR 在所有评估的变体中实现了最佳困惑度，包括强基线如 ALiBi。此外，学习到的几何在确定性 top-1 离散化下保持稳定，这表明它不仅是自适应的，而且在推理时也适合低成本近似。

View on arXiv Download PDF AI Translation

cs.CL / 53 / 2609.31264

Identifying Scientists on X

在 X 平台上识别科学家

Meier, Philipp, Boland, Katarina, Kallmeyer, Laura, Dietze, Stefan

Abstract

With the growing importance of science-related discourse on the Web and the erosion of the classical knowledge order, it is important to identify different user groups, such as scientists, automatically. This work proposes an approach for identifying scientists and non- scientists on X/Twitter based on their user biographies and tweets. We show that we are able to classify accounts as scientists and non- scientists on two different datasets, reaching an F1 score of up to 0.88 using Random Forests with linguistic features and up to 0.96 using a contrastively fine-tuned DeBERTa model in an ensemble setup. Furthermore, we provide two datasets with X users labeled as scientists or non scientists and their respective tweets and user biographies.

Chinese Translation

随着科学相关话语在网络上日益重要以及传统知识秩序的侵蚀，自动识别不同用户群体（如科学家）变得尤为重要。本研究提出了一种基于用户个人简介和推文在 X/Twitter 上识别科学家与非科学家的方法。我们展示了在两个不同的数据集上能够将账户分类为科学家和非科学家，使用随机森林算法结合语言特征，F1 分数最高可达 0.88，而在集成设置中使用对比微调的 DeBERTa 模型时，F1 分数最高可达 0.96。此外，我们提供了两个标注为科学家或非科学家的 X 用户数据集，以及他们各自的推文和个人简介。

View on arXiv Download PDF AI Translation

cs.CL / 54 / 2609.31342

Stale-Document Poisoning: When Outdated Retrieval Overrides Correct Model Answers

过时文档中毒：当过时检索覆盖正确模型答案时

Ahmed, Md Shamim, Poech, Lukas Galke, Röttger, Richard

Abstract

Retrieval-augmented generation (RAG) is often used to address outdated knowledge by providing external evidence. But retrieval helps only when that evidence is still valid. We identify a temporal alignment failure, stale-document poisoning, in which outdated evidence makes a model wrong despite answering correctly without retrieval. We construct a benchmark of 317 verified knowledge reversals across medicine, law, software, and platform policy, grounded in dated official sources. Across 12 models, recent medical reversals are harder than long-established ones. More importantly, outdated retrieval flips 30% of Llama and 37% of Qwen answers even without instructions to trust the document; explicit follow instructions raise these rates to 66% and 75%. Across four open models and four domains, poisoning ranges from 17-91%, while matched up-to-date evidence is followed in 97-100% of trials. To isolate temporal applicability, we keep the historical evidence unchanged across 50 reversals and vary only the evaluation date. A clear pattern emerges: dates alone produce only modest adaptation, but when models are explicitly told when the old evidence stops applying, the larger models switch to the appropriate answer almost perfectly. Causal interventions confirm that this validity information directly shapes the final decision. The same internal components also support broader comparison tasks, suggesting that temporal applicability can recruit a general reasoning mechanism used for other comparisons. Finally, a fixed recency-aware hybrid re-ranker reduces poisoning by 4.6-10.0 points when dates are accurate, with gains that depend on reliable temporal metadata. Reliable RAG therefore requires selective trust: models must determine not only what retrieved evidence says, but whether it still applies.

Chinese Translation

检索增强生成（RAG）通常用于通过提供外部证据来解决过时知识的问题。然而，检索仅在该证据仍然有效时才有帮助。我们识别出一种时间对齐失败，即过时文档中毒，其中过时的证据使模型错误，即使在没有检索的情况下回答是正确的。我们构建了一个基于过时官方来源的317个经过验证的知识反转基准，涵盖医学、法律、软件和平台政策。在12个模型中，最近的医学反转比长期确立的反转更难。更重要的是，过时的检索在没有指示信任文档的情况下，翻转了30%的Llama和37%的Qwen答案；明确的跟随指示将这些比例提高到66%和75%。在四个开放模型和四个领域中，中毒率范围为17-91%，而匹配的最新证据在97-100%的试验中得到了遵循。为了隔离时间适用性，我们在50个反转中保持历史证据不变，仅改变评估日期。一个明显的模式出现：仅日期的变化产生的适应性有限，但当模型被明确告知旧证据何时不再适用时，较大的模型几乎完美地切换到适当的答案。因果干预证实，这种有效性信息直接影响最终决策。同样的内部组件也支持更广泛的比较任务，表明时间适用性可以调动用于其他比较的一般推理机制。最后，当日期准确时，一个固定的关注近期的混合重排序器将中毒减少了4.6-10.0个百分点，增益依赖于可靠的时间元数据。因此，可靠的RAG需要选择性信任：模型必须确定不仅检索证据所说的内容，还要判断其是否仍然适用。

View on arXiv Download PDF AI Translation

cs.CL / 55 / 2609.31382

Highlight-Then-Summarize: Learning to Compress Evidence for Long-Context Understanding

突出-再总结：学习压缩长文档理解中的证据

Xia, Zhaoyuan, Xie, Qinghongbing, Hue, Yung Xiang, Jiang, Jianguang, Lu, Gaofeng, Jiao, Zhenyu, Yuan, Xing, Dai, Dai, Mo, Tong, Zeng, Long

Abstract

Long-context understanding requires large language models (LLMs) to reason over lengthy documents, conversations, and code, yet task-relevant evidence is often sparse and scattered amid substantial irrelevant and redundant content. We propose Highlight-Then-Summarize (H2S), a compress-then-reason paradigm that first identifies source-grounded, question-relevant evidence and then integrates it into a compact, question-conditioned summary before producing the final answer. To train this behavior, we construct H2S-Dataset, comprising 6,647 examples from 11 benchmark families with an average context length of 43.9K tokens, and introduce H2S-RL, which provides process-level rewards for evidence selection and summary construction in addition to final-answer correctness. We evaluate on H2S-Bench, a seven-task long-context suite. Under a shared 128K input and 4K output budget, H2S-14B achieves an average score of 32.60, outperforming Qwen3.8-27B by 10.17 points and obtaining the strongest overall result among the evaluated open-source models. H2S-14B also achieves the highest Evidence-Summary Quality score and retains 97.1% of its 16K-budget performance with only a 4K output budget. These results show that explicitly selecting and integrating evidence improves long-context reasoning while enabling more compact generation.

Chinese Translation

长文档理解要求大型语言模型（LLMs）对冗长的文档、对话和代码进行推理，但任务相关的证据往往在大量无关和冗余内容中稀疏且分散。我们提出了突出-再总结（Highlight-Then-Summarize, H2S）方法，这是一种先压缩后推理的范式，首先识别源基础的、与问题相关的证据，然后将其整合成一个紧凑的、以问题为条件的摘要，最后生成最终答案。为了训练这种行为，我们构建了H2S-Dataset，包含来自11个基准系列的6,647个示例，平均上下文长度为43.9K个标记，并引入了H2S-RL，为证据选择和摘要构建提供过程级奖励，此外还考虑最终答案的正确性。我们在H2S-Bench上进行评估，这是一个包含七个任务的长文档套件。在共享的128K输入和4K输出预算下，H2S-14B的平均得分为32.60，超越了Qwen3.8-27B，领先10.17分，并在评估的开源模型中获得了最强的整体结果。H2S-14B还获得了最高的证据-摘要质量得分，并在仅有4K输出预算的情况下保留了97.1%的16K预算性能。这些结果表明，明确选择和整合证据能够改善长文档推理，同时实现更紧凑的生成。

View on arXiv Download PDF AI Translation

cs.CL / 56 / 2609.31403

Sorry Robot, Happy Human: Vision-Language Models Read Only One of Two Legible Typographic Layers

抱歉，机器人；人类快乐：视觉-语言模型仅能读取两层可读排版中的一层

İncidelen, Mert, Kashkash, Yamen, Berker, Asya, Aydoğan, Murat

Abstract

Vision-language models (VLMs), despite their success in optical character recognition (OCR) tasks, are vulnerable to typographic attacks and have a fragile structure for images with multiple text layers. In this study, the DecoyBench dataset was created using the Decoy Font method. The dataset consists of 300 images, each containing text with sharp contour lines superimposed on another text with soft shading. Six recent closed-source models from three different model families were evaluated using this dataset under two different prompting conditions (naive and guided) and at two different resolutions ($512\times512$ and $64\times64$). A validation study showed that human participants could read both text layers with high accuracy. In contrast, the models, with most variants and both prompting methods, read the contour text with near-human accuracy at high resolution, but almost never fully extracted the shading text. At low resolution, the contour text could not be read by either the models or humans, while the shading text could be extracted with high accuracy. The findings indicate that the evaluated VLMs exhibit a consistent behavioral limitation when processing typographic structures containing multiple spatial frequency layers.

Chinese Translation

尽管视觉-语言模型（VLMs）在光学字符识别（OCR）任务中取得了成功，但它们对排版攻击十分脆弱，并且在处理具有多层文本的图像时结构脆弱。本研究使用Decoy Font方法创建了DecoyBench数据集。该数据集包含300张图像，每张图像上都有一层具有清晰轮廓线的文本叠加在另一层具有柔和阴影的文本上。使用该数据集评估了来自三个不同模型家族的六个最新闭源模型，在两种不同的提示条件（天真和引导）下，以及在两种不同的分辨率（$512 imes512$和$64 imes64$）下进行评估。一项验证研究表明，人类参与者能够以高准确率读取两层文本。相比之下，大多数变体和两种提示方法下的模型在高分辨率下以接近人类的准确率读取轮廓文本，但几乎从未完全提取阴影文本。在低分辨率下，模型和人类均无法读取轮廓文本，而阴影文本则可以高准确率提取。研究结果表明，被评估的VLMs在处理包含多个空间频率层的排版结构时表现出一致的行为限制。

View on arXiv Download PDF AI Translation

cs.CL / 57 / 2609.31448

ViSTA: A Simple Bridge Extends Visual Alignment to Clinical Time-Series Understanding in Multimodal LLMs

ViSTA：一个简单的桥梁将视觉对齐扩展到多模态大语言模型中的临床时间序列理解

Gao, Junyi, Shi, Yu, Hu, Pingzhao, Harrison, Ewen M

Abstract

Clinical prediction models estimate risk from patient measurements, while large language models support medical text understanding and question answering. Yet their language capabilities do not ensure accurate prediction from structured, high-dimensional clinical time series. Improving this ability would connect risk estimation with flexible questions about a patient's evolving condition. We introduce ViSTA, a compact adapter that incorporates irregular numerical measurements into a pretrained vision-language model's chart representations. It learns corrections to visual tokens while leaving all pretrained parameters unchanged. On MIMIC-IV, ViSTA has the highest mean scores among the compared adaptations on all four metrics for acute kidney injury and mortality prediction across models with 2-9 billion parameters. With 0.516 million trainable parameters, the 2-billion-parameter model reaches an area under the ROC curve of 0.7376 for acute kidney injury, compared with GPT-5.6 Sol's 0.7380 with text input and high reasoning effort. Training for temporal question answering yields 69.27% accuracy at 4 billion parameters with over 90% fewer trainable parameters than low-rank adaptation using charts or numerical text, at a 2.82-4.88 percentage-point accuracy gap. ViSTA extends pretrained language models to numerical prediction and temporal questions.

Chinese Translation

临床预测模型通过患者测量数据估计风险，而大型语言模型则支持医学文本理解和问答。然而，它们的语言能力并不能确保从结构化的高维临床时间序列中进行准确预测。改善这一能力将使风险估计与关于患者不断变化状况的灵活问题相连接。我们提出了ViSTA，一个紧凑的适配器，将不规则的数值测量纳入预训练视觉-语言模型的图表表示中。它在不改变所有预训练参数的情况下学习对视觉标记的修正。在MIMIC-IV数据集上，ViSTA在所有四个指标上对急性肾损伤和死亡率预测的比较适配中，具有最高的平均得分，适用于参数量在20亿到90亿之间的模型。对于急性肾损伤，具有51.6万可训练参数的20亿参数模型在ROC曲线下面积上达到了0.7376，而GPT-5.6 Sol在文本输入和高推理努力下则达到了0.7380。针对时间问题回答的训练在40亿参数下实现了69.27%的准确率，且可训练参数比使用图表或数值文本的低秩适配少90%以上，准确率差距为2.82-4.88个百分点。ViSTA将预训练语言模型扩展到数值预测和时间问题。

View on arXiv Download PDF AI Translation

cs.CL / 58 / 2609.31506

Evaluating Cultural Awareness of LLMs for Haitian Creole

评估大型语言模型对海地克里奥尔语的文化意识

Clervilsson, Christelle, Guo, Yanzhu

Abstract

Large language models (LLMs) exhibit substantial performance disparities between high- and low-resource languages. Beyond lower task performance, they often fail to capture the cultural norms and values of underrepresented communities. In this work, we present the first systematic evaluation of cultural awareness in LLMs for Haitian Creole, a language spoken by millions but severely underrepresented in digital resources. We assess cultural awareness along four complementary dimensions---specificity, bias, diversity, and variation---using a benchmark of culturally salient prompts curated by native speakers in a text infilling setting. Our results reveal a clear gap between cultural awareness in Haitian Creole and higher-resource French, with Haitian performance being more uneven across domains and more affected by French linguistic interference. Story generation further reveals recurring portrayals of Haitian characters through hardship and resilience, showing that even positive characterizations can encode stereotypical narratives. Our code, benchmark, and evaluation framework are publicly available.

Chinese Translation

大型语言模型（LLMs）在高资源语言和低资源语言之间表现出显著的性能差异。除了任务性能较低外，它们通常无法捕捉到被低估社区的文化规范和价值观。在本研究中，我们首次系统性地评估了大型语言模型在海地克里奥尔语中的文化意识，这是一种有数百万使用者但在数字资源中严重缺乏代表性的语言。我们通过使用由母语者在文本填充环境中策划的文化显著提示的基准，从特异性、偏见、多样性和变异四个互补维度评估文化意识。我们的结果揭示了海地克里奥尔语和高资源法语之间文化意识的明显差距，海地克里奥尔语的表现跨领域更加不均衡，且更易受到法语语言干扰的影响。故事生成进一步揭示了海地角色在艰难和韧性中的反复描绘，表明即使是积极的角色刻画也可能编码刻板印象叙事。我们的代码、基准和评估框架已公开提供。

View on arXiv Download PDF AI Translation

cs.CL / 59 / 2609.31511

Muslim: A Deployed Arabic Voice AI Platform for Grounded Islamic Knowledge

Muslim：一个部署的阿拉伯语语音人工智能平台，提供扎实的伊斯兰知识

Elnawasany, Yahya Mohamed

Abstract

We present Muslim, a production Arabic voice AI platform serving grounded, sourced Islamic knowledge to real users. Beyond a real-time voice pipeline (NeMo Arabic ASR, an OpenAI-compatible LLM endpoint, self-hosted TTS) and a deterministic multi-source retrieval layer routed across six Model Context Protocol servers, we report three things a research prototype typically lacks. First, a released family of fine-tuned Arabic Islamic model artifacts: an efficient tool-routing LLM (Muslim-6B-PRO, 5.94B parameters) and a Modern Standard Arabic TTS model (Fasih-TTS-V1) that ranks 5th of 17 overall and 2nd of 11 open-weight systems on the community-voted Arabic TTS Arena for MSA. Second, an account and metering layer - a free per-account turn allowance, capacity-aware refusal, and email verification deferred to the point it actually matters - that turns an open demo into an operable, abuse-resistant product. Third, a three-layer observability stack (liveness, error reporting, product analytics) built specifically around the system's characteristic failure mode: a GPU-bound agent host going silent while the web tier keeps serving normally. We report real, measured latency and accuracy figures (98.4% recitation-validation accuracy on 124 cases; end-to-end voice latency of 0.9-1.7s) and discuss the concrete engineering trade-offs and limitations of running an Islamic-knowledge voice product in production.

Chinese Translation

我们介绍了Muslim，一个生产级的阿拉伯语语音人工智能平台，为真实用户提供扎实的、来源明确的伊斯兰知识。除了实时语音管道（NeMo阿拉伯语自动语音识别（ASR）、与OpenAI兼容的语言模型（LLM）端点、自托管的文本转语音（TTS））和一个通过六个模型上下文协议服务器路由的确定性多源检索层外，我们报告了研究原型通常缺乏的三项内容。首先，发布了一系列经过微调的阿拉伯语伊斯兰模型工件：一个高效的工具路由LLM（Muslim-6B-PRO，参数量为5.94B）和一个现代标准阿拉伯语TTS模型（Fasih-TTS-V1），在社区投票的阿拉伯语TTS竞技场中排名第5（共17个）和第2（共11个开放权重系统）。其次，一个账户和计量层——每个账户的免费转发额度、基于容量的拒绝和电子邮件验证推迟到真正重要的时刻——将一个开放的演示转变为一个可操作的、抗滥用的产品。第三，一个三层可观察性堆栈（实时性、错误报告、产品分析），专门围绕系统的特征故障模式构建：一个受GPU限制的代理主机在网络层正常服务的情况下保持沉默。我们报告了真实的、测量的延迟和准确性数据（在124个案例中，背诵验证准确率为98.4%；端到端语音延迟为0.9-1.7秒），并讨论了在生产中运行伊斯兰知识语音产品的具体工程权衡和局限性。

View on arXiv Download PDF AI Translation

cs.CL / 60 / 2609.31553

MexHat: A Dataset for Hate Speech Detection in Mexican Spanish Videos

MexHat：用于检测墨西哥西班牙语视频中的仇恨言论的数据集

Tlelo-Coyotecatl, Itzel, Escalante, Hugo Jair

Abstract

Ensuring online safety through content monitoring had raised Hate Speech Detection as a crucial task to be addressed. By essence the task demands the capture of contextual cues, which are essential for a precise understanding of the content's intent. Although automated detection approaches for the task have advanced significantly, the scarcity of non-English resources persists, limiting the ability of models to adapt to the subtle, context-dependent, and culturally related nature of multimodal content. In this paper, we introduce MexHat, a video dataset designed to capture the linguistic and cultural cues for the hate-speech detection task in a Mexican Spanish context. Our dataset comprises around 1k video clips annotated across two tasks: a three-way class evaluation (no negative content, offensive content and hate-speech content), and a fine-grained class evaluation including three hate-speech sub-categories. The dataset statistics and the baseline results highlight the inherent challenges associated with the task. Disclaimer: This paper contains sensitive content that may be disturbing to some readers.

Chinese Translation

通过内容监控确保在线安全使得仇恨言论检测成为一个亟待解决的重要任务。该任务本质上要求捕捉上下文线索，这对于准确理解内容的意图至关重要。尽管针对该任务的自动检测方法已经取得了显著进展，但非英语资源的稀缺仍然存在，限制了模型适应多模态内容中微妙、依赖上下文和文化相关特性的能力。在本文中，我们介绍了MexHat，一个旨在捕捉墨西哥西班牙语环境中仇恨言论检测任务的语言和文化线索的视频数据集。我们的数据集包含约1000个视频片段，涵盖两个任务的注释：三类评估（无负面内容、冒犯性内容和仇恨言论内容），以及包括三个仇恨言论子类别的细粒度分类评估。数据集的统计信息和基准结果突显了与该任务相关的固有挑战。免责声明：本文包含可能对某些读者造成困扰的敏感内容。

View on arXiv Download PDF AI Translation

cs.CL / 61 / 2609.31571

Strategically Diverse Sampling for Self-Training

自我训练的战略多样性采样

Gurung, Alexander, Whitammer, Esmeralda S., Lapata, Mirella

Abstract

Many LLM training and inference methods, including RL and test-time scaling, depend on repeated sampling, but benefit only when the responses meaningfully differ. Self-training faces the same challenge: training data is typically constructed by sampling IID responses and filtering primarily for correctness, thereby overrepresenting strategies a model already favours. We investigate strategic diversity, or substantive variation among approaches to a problem, as an alternative principle for constructing self-training data. We generate strategically diverse data with two sampling methods: GROOT, a new method which constructs a hierarchical tree of approaches and samples distinct paths, and Verbalized Sampling (VS), adapted to produce an unstructured set of approaches. Across competitive programming and Next-Chapter Prediction domains, models trained on strategically sampled data outperform IID-trained counterparts on difficult tasks and provide strong initializations for RL and test-time scaling. Most strikingly, self-training on strategically diverse but incorrect traces from Qwen3-4B outperforms IID distillation from a 235B teacher. These results challenge prevailing assumptions about what makes useful self-training data and show that diversity of approaches can matter more than correctness or teacher scale.

Chinese Translation

许多大规模语言模型（LLM）的训练和推理方法，包括强化学习（RL）和测试时缩放，依赖于重复采样，但只有在响应有意义地不同的情况下才能获益。自我训练面临同样的挑战：训练数据通常通过采样独立同分布（IID）响应构建，并主要过滤正确性，从而过度代表模型已经偏好的策略。我们探讨战略多样性，即解决问题的方法之间的实质性变化，作为构建自我训练数据的另一原则。我们采用两种采样方法生成战略多样的数据：GROOT，一种新的方法，通过构建方法的层次树并采样不同路径来实现；以及口头化采样（Verbalized Sampling, VS），调整以生成无结构的方法集合。在竞争编程和下一章节预测领域，基于战略采样数据训练的模型在困难任务上优于基于IID训练的模型，并为强化学习和测试时缩放提供了强有力的初始化。最引人注目的是，在Qwen3-4B上进行的自我训练，即使是基于战略多样但不正确的轨迹，仍然优于来自235B教师的IID蒸馏。这些结果挑战了关于什么构成有用自我训练数据的普遍假设，并表明方法的多样性可能比正确性或教师规模更为重要。

View on arXiv Download PDF AI Translation

arXiv Papers

TinyCVIO: A Constellation-Aided Visual-Inertial Odometry System for Nanodrones

POIL: Point-based One-Shot Imitation Learning with Stable Dynamical Systems

Actively Resolving Contextual Uncertainty for Underspecified Tasks in Natural Language

WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving

VkVIO: Cross-platform GPU Acceleration for Visual-Inertial Odometry with Vulkan

Realizability Is Not Enough: Encoding, Liveness, and Auditing of Synthesized Robot Supervisors

DGT-Map: Directional Global Traversability Mapping Utilizing Multi-Task Learning for Heterogeneous Vehicles

Policy-Calibrated DAgger: Offline Calibrated Noise Injection for Imitation Learning

Learning-Based Pressure Predictive Control of a Vertebraic Soft Robotic Tail

Memory-Aware Multi-Sensor Perception for Efficient and Safe Navigation in Dynamic Environments

Tactile Sensing Array for Multi-Phalanx Sensing in Humanoid Hands

Aerial Manipulation in the Wild with Onboard Perception, Policy Learning, and Whole-Body Control

Containing Behavioral Cascades from Manipulated Claims in LLM-Powered Multi-Robot Systems

A Long-Legged, Direct-Drive Monopedal Robot Achieves Exceptional Jump Height

ST-pRRTC: Parallel Space-Time RRT-C with Adaptive Goal-Time Forests

GraspTwin: Zero-Shot Task-Oriented Grasp Optimization via a Digital Twin

Privacy-Preserving Prompted Policy Search for Robotic Control

Auditing Latent-Space Monitors for Autonomous Driving

SoGuDiff: Socially Guided Diffusion for Steerable, Norm-Grounded Robot Navigation

MOCHA: Multi-Objective Co-Design using Hypernetwork Architectures

HuGo: LLMs as Whole-Body Policy Code Designers for Humanoid Loco-Manipulation

Learning-Accelerated Narrow-Phase Collision Detection via Check Ordering for Sampling-Based Motion Planning

Audit Before You Commit: Locating Belief Failures in Active Identification for One-Shot Manipulation

Frequency-Modulated Piezoelectric Haptic Display

MR. POP: Multi-Robot Parallel Optimizing Planner for Almost-Surely Asymptotically Optimal Planning

Can a Robot Read Braille? - Learning to Adapt Contact via Imitation Learning for Tactile Braille Recognition

Multi-Objective Human-in-the-Loop Bayesian Optimization of a Lower-Limb Exoskeleton

Learning Vision-Based Agile Gap Traversal: Differentiable Simulation with a Warm-Started Critic

From Visual Search to Movement Control: A Priority Field for Artificial Agents

RoboMonitor: Label-Efficient Runtime Monitoring of Robot Task Execution via Predictive Representation Learning

Praxis: Distilling Physical Interaction Priors from Egocentric Videos for Generalizable Whole-Body Manipulation

Anatomy-Aware Dexterity-Driven Design Optimization of Surgical Continuum Robots

Design and Characterization of a Variable-Length Continuum Mechanism with Force Locking

NavGen: Visual Generative Models as a Scalable Data Engine for Embodied 3D Navigation

Moving Horizon Estimation for Quadrotors: An $\mathcal{L}_1$ Adaptive Optimizer Approach

SeA-RVINS: Semantic-Aware Tightly Coupled RTK-Visual-Inertial System with Correlation-Preserving Robust Estimation for Urban Navigation

Evaluation Is All You Need for Multi-Modal Autonomous Driving

HIRE: History-Conditioned Interaction Reasoning and High-Rate Execution for Visually Aliased Precision Manipulation

Fast Plans, Faithful Actions: Closing the Planning-Execution Gap in Hierarchical Vision-Language-Action Models

Impedance Cloning: Learning Equilibrium Point Parameters for Contact-Rich Manipulation

VLaRL: Augmenting Vision-Language-Action Models with Simulation-Trained Latent-Conditioned Residual RL

A Second Torque Port for Series Elastic Actuators: Parallel-Integrated Design and Time-Scale Torque Allocation

PHASE: Compliance-Enabled Tactile Phase Retrieval for Few-Shot Insertion Learning

Causeway: Restoring Task Accessibility for Instruction Switching in VLA Policies

Bundled Contact Gradients: Stabilizing Differentiable Simulation for Deployable Dynamic Tasks

VisTacAlign: Co-Training Dexterous Policies on Tactile Human and Robot Demonstrations

FRAM: Trajectory-Guided Visual Feature Selection for Compact Language-Conditioned Robot Manipulation

TACTIC: Understanding Tactile Encoders and Conditioning for Contact-rich Robot Manipulation Policies

Quadruped Obstacle Avoidance and Footstep Planning with Distributed Low-cost Time-of-Flight Sensors

Co-design of trajectory and morphology for a vertical jump-climbing robot

Precision at Speed: Sample-Efficient Online Model-Based Reinforcement Learning for Hydraulic Excavator Control

Compact Force Sensor for Dual-UAV Cable-Suspended Payload Transport with Tension-Aware Outer-Loop Control

Kintsugi-VLA: Turning Failed Robot Rollouts into Recovery Data through Interventional Recoverability

Accuracy Evaluation of INS/ZUPT Filtering Methods Based on Different Geometric Error Definitions

AuthGuard-R: Safety-Compliant Mission Hijacking and Dual-Gate Defense for LLM-Controlled Robots

DualManip: Agentic Dynamic Manipulation via Dual-Path Semantic Reasoning and Geometric Adaptation

Comparative Evaluation of an XR Pen-based Control Interface for Semi-Autonomous Mobile Robot Navigation in Service Environments

INTERACT: Interactive Planning for Autonomous Driving via Anchor-Conditioned Prediction and Trust-Region Refinement

Evaluating the Impact of Adaptive Extended Reality on Human-Robot Interaction Across the Reality-Virtuality Continuum

Onboard Wind-Preview Model Predictive Control Using Pitot-Static Sensing for Multirotor UAVs

Enabling a Unified Cross-Domain Representation for Two-Finger Gripper Manipulation via Interaction-Centric Modeling

CoralPlan: Observation Skill Selection and Execution for Underwater Robotic Inspection

Imp-ACT: Adaptive Impedance Control and Action Chunking with Transformers to Learn Contact-Rich Manipulation from Demonstrations

Cybflight: An Embedded Rust Autopilot for Aerial Robotics Research

Towards VLA-Dreamer: Refining VLA Behavior Using World Models

See to Reach, Feel to Grasp: Learning A Blind Grasp Reflex for Anthropomorphic Robotic Hands

Representation-Guided Generation and Integration of Executable Programs for Robot Manipulation

Transformer-based Monte Carlo Localization in Construction Meshes

RECAST: From Log Replay to Closed-Loop Driving Simulation with View-Complete Actors

Guiding End-to-End Driving Models with Endpoint-Constrained Trajectory Optimization

Modeling and Generative-AI-Based Design of Load-Adaptive Gravity Balancing Mechanisms

InternW0-$\Delta$: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data

Augmented Reality Interfaces for Human-Robot Collaboration: Development of a ROS 2-Based Sensor Streaming Framework and Validation via SLAM Algorithms

dRVG: Quadtree-Guided, Resolution-Complete Online Motion Planning for Polygonal Robots in Unknown Environments

CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping with an LLM Agent for Immersive Collaborative VR Teleoperation

ExoLaN: Physics-Consistent Context-Aware Dynamics Learning for Exoskeletons

Learning to Leverage Compliance: A Policy-Admittance Learning Framework for Robotic Insertion

Vision-Based 6-DoF Grasp Pose Estimation for Robot Cloth Unfolding

Generate, Track, Improve: Perceptive Multi-Skill Humanoid Locomotion with RL-Fine-Tuned Motion Generators