arXiv Daily Digest

354

Papers

Too Much of a Good Thing: When sim2real Efforts Impede Policy Learning (And What to Do About It)

好事也有过头的时候：何时 sim2real 努力妨碍政策学习（以及该如何应对）

Morgenstein, Kyle, Masetty, Bharath, Welch, Stephen, Sentis, Luis

Abstract

While sim2real efforts are necessary for effective policy transfer to hardware, there is such a thing as too much of a good thing. We argue that sim2real efforts have led to misaligned incentives with policy learning, resulting in simulator lock in and poor policy exploration due to the unreasonable constraints imposed by the real world. We offer a diagnosis and explanation of the current status of the problem, and propose a potential solution via a sim2sim2real paradigm that leverages the robot's kinematics as the sole design constraint.

Chinese Translation

虽然 sim2real 努力对于有效的政策转移到硬件上是必要的，但过犹不及也是有其道理的。我们认为，sim2real 努力导致了与政策学习之间的激励不匹配，从而造成了模拟器锁定和由于现实世界施加的不合理约束而导致的政策探索不足。我们提供了对当前问题状态的诊断和解释，并提出了一种通过 sim2sim2real 范式的潜在解决方案，该范式将机器人的运动学作为唯一的设计约束。

View on arXiv Download PDF AI Translation

cs.RO / 2 / 2606.02641

CARVE: Certified Affordable Repair of Vetoed Maneuvers via Envelopes for Interactive Driving

CARVE：通过交互驾驶的包络线认证可负担的被否决机动修复

Wang, Yifan

Abstract

Interactive driving exposes a failure mode that is easy to miss in rule-aware autonomous-driving stacks: a hard-rule margin can be negative for an ego candidate even though a small lawful accommodation by a non-priority agent would restore feasibility. Existing rulebooks, shields, and reachability filters are strong at vetoing unsafe actions, while prediction-based planners model likely responses. Neither returns a runtime proof object that states which bounded multi-agent edit repairs the maneuver, who owns the edit, whether the request is right-of-way affordable, and what ego fallback remains if the request is not observed. We formulate this missing object as *interactive repair certification* and introduce *CARVE*, a prediction-free certificate layer over a finite lattice of ego-owned and agent-owned tactical operators. Agent-owned requests are admissible only inside $B_j(s) = \beta(\pi_j)\alpha_j^{\max}(s)$, a cooperation envelope that separates kinematic reachability from normative priority. The resulting certificate records the binding rule, repair category, repair set, responsibility-weighted cost split, and fallback. On 589 Lanelet2-geometry-grounded INTERACTION replay episodes, CARVE-Greedy accepts 98.64% of initially vetoed maneuvers and recovers 370/378 human-resolved false vetoes, while preserving 589/589 right-of-way respect, zero priority-agent false positives, and 400/400 negative-stress vetoes. We prove certificate soundness, structural right-of-way respect, exact finite-lattice minimality, fallback contingency, and blame-consistency conditions. CARVE does not predict or require another driver's compliance; it certifies whether a proposed interaction is bounded, attributable, and normatively admissible under declared assumptions.

Chinese Translation

交互驾驶暴露了一种在规则意识的自动驾驶系统中容易被忽视的失败模式：即使非优先代理的微小合法调整可以恢复可行性，某个自我候选者的硬性规则边际也可能为负。现有的规则手册、保护机制和可达性过滤器在否决不安全行为方面表现强劲，而基于预测的规划者则对可能的反应进行建模。两者都未能返回一个运行时证明对象，该对象说明哪个有界的多智能体编辑修复了该机动，谁拥有该编辑，是否请求是优先通行可负担的，以及如果请求未被观察，剩余的自我后备措施是什么。我们将这一缺失的对象表述为*交互修复认证*，并引入*CARVE*，这是一个在自我拥有和代理拥有的战术操作有限格子上的无预测证书层。代理拥有的请求仅在合作包络内可接受，即在条件为 $B_j(s) = eta( ext{π}_j) ext{α}_j^{ ext{max}}(s)$ 的情况下，该包络将运动学可达性与规范优先级分开。生成的证书记录了约束规则、修复类别、修复集、责任加权成本分配和后备措施。在589个基于Lanelet2几何的INTERACTION重放回合中，CARVE-Greedy接受了98.64%的最初被否决的机动，并恢复了370/378个人工解决的错误否决，同时保持了589/589的优先通行尊重，零个优先代理的假阳性，以及400/400的负压力否决。我们证明了证书的有效性、结构优先通行的尊重、精确有限格子的最小性、后备应急和责任一致性条件。CARVE不预测或要求其他驾驶员的遵从；它认证所提议的交互是否在声明的假设下是有界的、可归属的和规范上可接受的。

View on arXiv Download PDF AI Translation

cs.RO / 3 / 2606.02658

Fixed-Time Dynamic Landing of Quadrotors using Adaptive Unscented Kalman Filtering and Nonlinear Model Predictive Control

基于自适应无迹卡尔曼滤波和非线性模型预测控制的四旋翼固定时间动态着陆

Izadi, Mohammadreza, Shayan, Zeinab, Waslander, Steven, Faieghi, Reza

Abstract

This paper introduces an estimation and control framework for dynamic landing of multi-rotor uncrewed aerial vehicles on moving platforms. The proposed method integrates nonlinear model predictive control with a real-time minimum-jerk trajectory planner that enforces a prescribed touchdown time, enabling consistent timing during the terminal descent. To enhance robustness in the presence of time-varying sensing quality, we utilize an adaptive unscented kalman filter that updates the process and measurement noise statistics online. In addition, we provide a reference feasibility analysis showing that minimum-jerk references induce bounded thrust and torque commands under standard tracking hypotheses. The proposed framework is evaluated in simulation and hardware experiments, and it is shown to achieve repeatable landings and improved platform velocity prediction accuracy relative to EKF/UKF-based methods.

Chinese Translation

本文提出了一种用于多旋翼无人机在移动平台上动态着陆的估计与控制框架。所提出的方法将非线性模型预测控制与实时最小颤动轨迹规划器相结合，该规划器强制执行预定的着陆时间，从而在终端下降过程中实现一致的时序。为了增强在时变传感质量下的鲁棒性，我们采用了一种自适应无迹卡尔曼滤波器，该滤波器在线更新过程和测量噪声统计信息。此外，我们提供了一项参考可行性分析，表明在标准跟踪假设下，最小颤动参考会引发有界的推力和扭矩指令。所提出的框架在仿真和硬件实验中进行了评估，结果表明其能够实现可重复的着陆，并相较于基于扩展卡尔曼滤波（EKF）/无迹卡尔曼滤波（UKF）的方法提高平台速度预测的准确性。

View on arXiv Download PDF AI Translation

cs.RO / 4 / 2606.02677

Motion Planning in Dynamic Environments: A Survey from Classical to Modern Methods

动态环境中的运动规划：从经典方法到现代方法的综述

Shen, Zongyuan, Ou, Yaming, Gupta, Shalabh, Zhao, Shancheng, Zhou, Dehua, Wang, Gao, Ren, Zhongqiang, Fan, Junfeng, Cheng, Long

Abstract

Motion planning in dynamic environments requires robots to continuously adapt their paths in response to environmental changes for safe and uninterrupted navigation. While many surveys have reviewed planning in static settings, systematic reviews focused on dynamic environments remain limited. This paper presents a comprehensive survey of 138 works, primarily published between 2015 and 2025, spanning both classical and learning-based approaches. The motion planning methods are grouped into five categories based on the concepts of sampling, graph search, model predictive control, learning, and additional classical local planning approaches, including velocity obstacles, potential fields and dynamic windows. The learning techniques include supervised learning and reinforcement learning. We also discuss the role of dynamic perception in motion planning, covering techniques for detecting and modeling moving obstacles using cameras, LiDAR, and event-based sensors. The survey analyzes the principles, strengths, and limitations of each method, with particular attention to challenges unique to dynamic environments, such as prediction uncertainty, human-robot interaction, and the freezing robot problem. The survey provides researchers with a structured understanding of motion planning methods in dynamic environments.

Chinese Translation

动态环境中的运动规划要求机器人持续调整其路径，以应对环境变化，从而实现安全和无障碍的导航。虽然许多综述已对静态环境中的规划进行了回顾，但专注于动态环境的系统性综述仍然有限。本文呈现了一项对138篇文献的全面综述，主要涵盖2015年至2025年间发表的研究，涉及经典方法和基于学习的方法。运动规划方法根据采样、图搜索、模型预测控制、学习以及其他经典局部规划方法（包括速度障碍、势场和动态窗口）这五个概念进行分类。学习技术包括监督学习和强化学习。我们还讨论了动态感知在运动规划中的作用，涵盖了使用摄像头、激光雷达（LiDAR）和事件驱动传感器检测和建模移动障碍物的技术。该综述分析了每种方法的原理、优点和局限性，特别关注动态环境中独特的挑战，如预测不确定性、人机交互和机器人冻结问题。该综述为研究人员提供了对动态环境中运动规划方法的结构化理解。

View on arXiv Download PDF AI Translation

cs.RO / 5 / 2606.02735

See Less, Specify More: Visual Evidence Budgets for Generalizable VLAs

少看，多指定：用于可推广的视觉-语言-动作（VLA）模型的视觉证据预算

Wu, Yueh-Hua, Matsushima, Tatsuya, Ota, Kei

Abstract

Generalization remains a central bottleneck for vision-language-action (VLA) models: under distractors, appearance shifts, and semantically similar tasks, the policy must often infer local execution details from coarse instructions while also deciding which parts of the image matter for control. We present S2 (See Less, Specify More), a framework for improving VLA generalization by training the executor under a cleaner interface. Specify More preserves the original instruction as a stable high-level goal while relabeling each trajectory into refined trajectory- and subtask-level language that disambiguates the current execution mode. Unlike native attention, See Less imposes an explicit visual evidence budget, training the executor to act from task-sufficient evidence rather than unconstrained visual context, without any region or mask annotation. This interface lets the executor follow detailed guidance without relying on distracting visual patches or resolving avoidable ambiguity on its own, and it remains compatible with off-the-shelf VLM planners through in-context learning. Across our main evaluation settings, S2 improves overall generalization metrics by changing the executor's learning problem: coarse instructions induce avoidable supervision aliasing, goal-preserving local guidance outperforms instruction replacement in our main ablations, and explicit evidence budgeting reduces dependence on broad visual context beyond efficiency considerations. Across eight real-robot tasks on TX-G2 (an AgiBot G2-compatible variant) and HSR, S2 raises mean subtask success from 54.2% to 79.0% over pi0.5. Together, these results suggest that VLA generalization improves when the executor is trained to act from informative local guidance and task-sufficient visual evidence, rather than recovering both from weak supervision.

Chinese Translation

泛化仍然是视觉-语言-动作（VLA）模型的一个核心瓶颈：在干扰物、外观变化和语义相似任务下，策略通常必须从粗略指令中推断局部执行细节，同时决定图像中哪些部分对控制是重要的。我们提出了 S2（See Less, Specify More），一个通过在更清晰的接口下训练执行者来改善 VLA 泛化的框架。Specify More 保留了原始指令作为稳定的高层目标，同时将每个轨迹重新标记为细化的轨迹和子任务级语言，以消除当前执行模式的歧义。与原生注意力不同，See Less 强加了明确的视觉证据预算，训练执行者从任务充分的证据中行动，而不是不受限制的视觉上下文，无需任何区域或掩码注释。该接口使执行者能够遵循详细的指导，而不依赖于分散注意力的视觉补丁或自行解决可避免的歧义，并且通过上下文学习与现成的视觉语言模型（VLM）规划器保持兼容。在我们的主要评估设置中，S2 通过改变执行者的学习问题来提高整体泛化指标：粗略指令引发可避免的监督混淆，保持目标的局部指导在我们的主要消融实验中优于指令替换，而明确的证据预算减少了对广泛视觉上下文的依赖，超出了效率考虑。在 TX-G2（与 AgiBot G2 兼容的变体）和 HSR 上的八个真实机器人任务中，S2 将平均子任务成功率从 54.2% 提高到 79.0%。这些结果表明，当执行者被训练从信息丰富的局部指导和任务充分的视觉证据中行动时，VLA 泛化得到了改善，而不是从弱监督中恢复两者。

View on arXiv Download PDF AI Translation

cs.RO / 6 / 2606.02745

SeeTraceAct: Visibility-Aware Latent Planning from Cross-Embodiment Demonstration Videos

SeeTraceAct：基于可见性意识的跨体现演示视频的潜在规划

Son, Jaehyeon, Kim, Junhyun, Kam, Kyle, Coholich, Jeremiah, Kim, Seok Joon, Kim, Jinhoo, Kim, Chris Dongjoo, Cho, Jaemin, Fox, Dieter, Kira, Zsolt

Abstract

Vision-language-action models (VLAs) are promising general-purpose robot policies, but adapting them to new tasks typically requires costly task-specific teleoperation data. As an alternative, we study one-shot demo-conditioned VLAs, where a robot policy is conditioned on a single demonstration video of an unseen task. We find that existing end-to-end approaches often struggle when successful execution requires precisely localizing small target regions. To address this limitation, we propose SeeTraceAct, a demo-conditioned VLA framework that encourages precise spatial grounding through visibility-aware prediction of future end-effector traces. To enable reproducible evaluation with cross-embodiment demonstrations, we introduce and release RoboCasa-DC, a demo-conditioned extension of RoboCasa with episode-paired humanoid videos. Experiments on RoboCasa-DC and a real-world benchmark, where a Franka Panda arm is conditioned on human demonstrations, show that SeeTraceAct outperforms baselines, achieving the best success rate across all four RoboCasa-DC settings and improving real-world average success by 12.5 percentage points.

Chinese Translation

视觉-语言-动作模型（VLAs）是有前景的通用机器人策略，但将其适应于新任务通常需要昂贵的任务特定遥操作数据。作为替代方案，我们研究了一种单次演示条件的VLA，其中机器人策略基于未见任务的单个演示视频。我们发现，现有的端到端方法在成功执行需要精确定位小目标区域时往往表现不佳。为了解决这一限制，我们提出了SeeTraceAct，这是一种演示条件的VLA框架，通过对未来末端执行器轨迹的可见性意识预测来鼓励精确的空间定位。为了实现跨体现演示的可重复评估，我们引入并发布了RoboCasa-DC，这是RoboCasa的演示条件扩展，包含配对的人形视频。对RoboCasa-DC和一个真实世界基准的实验显示，SeeTraceAct在所有四个RoboCasa-DC设置中都超越了基线，取得了最佳成功率，并将真实世界的平均成功率提高了12.5个百分点。

View on arXiv Download PDF AI Translation

cs.RO / 7 / 2606.02767

Hybrid Adaptive Kalman Filtering for Data-Efficient Joint Tracking and Classification

用于数据高效联合跟踪与分类的混合自适应卡尔曼滤波

Lee, Jiho, Ahmed, Nisar R., Russell, Rebecca

Abstract

Kalman filtering performance is highly sensitive to model mismatch and noise covariance tuning. Learning-based approaches address these limitations but typically rely on supervised training with large datasets and do not produce consistent uncertainty estimates. In this paper, we propose a self-supervised Hybrid Adaptive Kalman Filter that learns structured corrections to system dynamics and process noise covariance from measurements alone while preserving the probabilistic structure of the filter. This allows the innovation likelihood to be computed and subsequently used for model classification via generalized Bayesian inference. Experimental results on real-world and simulated datasets demonstrate improved estimation accuracy and statistical consistency as well as robust classification performance across both low-data and large-data scenarios.

Chinese Translation

卡尔曼滤波的性能对模型不匹配和噪声协方差调节高度敏感。基于学习的方法解决了这些局限性，但通常依赖于使用大型数据集进行监督训练，并且无法产生一致的不确定性估计。本文提出了一种自监督的混合自适应卡尔曼滤波器，它仅通过测量学习系统动态和过程噪声协方差的结构性修正，同时保持滤波器的概率结构。这使得创新似然能够被计算，并随后通过广义贝叶斯推断用于模型分类。在真实世界和模拟数据集上的实验结果表明，在低数据和大数据场景下，估计精度和统计一致性得到了改善，同时分类性能也表现出强健性。

View on arXiv Download PDF AI Translation

cs.RO / 8 / 2606.02796

A Measurement-Driven Digital Twin Architecture for Plant-Level Biomass Estimation and Growth Forecasting in Hydroponic Systems

一种基于测量驱动的数字双胞胎架构用于水培系统中的植物级生物质估计和生长预测

Mayborne, Morgan, Silwal, Abhisesh, Kantor, George

Abstract

Alternatives to soil-based horticulture, such as hydroponics, have been developed to respond to food distribution concerns for dense urban centers. A new system was developed to track an individual lettuce plant's growth in a hydroponic environment, utilizing streams of measured information and available models to continuously update the growth trajectory estimates for a plant. These "digital twin" models were integrated into an operating hydroponic greenhouse, with custom horticultural and sensor hardware to grow and measure relevant information. To aid in updating model parameters, plant yield was continuously measured with a custom neural network, using RGB-D images of the plants as an input. The network, trained on a collected dataset of 1300 images, was able to estimate mass within 1.5 g of the ground-truth value. After integration into the custom system, digital twin growth projections could approximate future yield between one and four days in the future, maintaining around a 2 g forecasting error.

Chinese Translation

为应对密集城市中心的食品分配问题，开发了水培等土壤替代园艺的方法。我们开发了一种新系统，用于跟踪水培环境中单个生菜植物的生长，利用测量信息流和可用模型持续更新植物的生长轨迹估计。这些“数字双胞胎”模型被集成到一个正在运行的水培温室中，配备定制的园艺和传感器硬件，以生长和测量相关信息。为了帮助更新模型参数，使用自定义神经网络持续测量植物产量，以植物的RGB-D图像作为输入。该网络在收集的1300张图像的数据集上进行训练，能够在真实值的基础上估计质量误差在1.5克以内。在集成到定制系统后，数字双胞胎的生长预测能够在未来一到四天内近似未来产量，保持约2克的预测误差。

View on arXiv Download PDF AI Translation

cs.RO / 9 / 2606.02879

Direct Informed Sampling on Riemannian Manifolds via Loewner Order Lower Bounds

通过洛温纳序下界在黎曼流形上进行直接知情采样

Kyaw, Phone Thiha, Kelly, Jonathan

Abstract

Informed sampling techniques accelerate sampling-based motion planners by focusing the search on promising regions of the state space, yet most existing methods rely on Euclidean heuristics that become inadmissible under configuration-dependent Riemannian metrics. While scalar eigenvalue bounds restore admissibility by uniformly scaling the Euclidean distance, they discard the directional structure of the metric, producing overly conservative informed sets. We propose a matrix-valued admissible heuristic that exploits the Loewner order on symmetric positive definite matrices to compute the tightest constant lower bound on the metric tensor while preserving its full directional structure. The Cholesky factorization of this bound defines a linear map to an isotropic Euclidean space in which the Riemannian informed set reduces to a standard prolate hyperspheroid, enabling direct, rejection-free sampling using existing algorithms. Experiments on manipulation tasks with a 6-DoF UR5, 7-DoF Franka, and 14-DoF PR2 under three distinct Riemannian metrics show that our heuristic produces consistently tighter informed sets than both the Euclidean and scalar eigenvalue bounds, accelerating convergence across multiple state-of-the-art asymptotically optimal planners.

Chinese Translation

知情采样技术通过将搜索集中在状态空间中有前景的区域，加速基于采样的运动规划器。然而，大多数现有方法依赖于欧几里得启发式，这在依赖于配置的黎曼度量下变得不可接受。尽管标量特征值界限通过均匀缩放欧几里得距离恢复了可接受性，但它们忽略了度量的方向结构，导致生成过于保守的知情集合。我们提出了一种矩阵值可接受启发式，利用对称正定矩阵上的洛温纳序来计算度量张量的最紧常数下界，同时保留其完整的方向结构。该下界的乔尔斯基分解定义了一个线性映射到一个各向同性的欧几里得空间，在该空间中，黎曼知情集合简化为标准的拉长超球面，从而能够使用现有算法进行直接的无拒绝采样。在三种不同的黎曼度量下，对6自由度UR5、7自由度Franka和14自由度PR2的操作任务进行的实验表明，我们的启发式方法产生的知情集合始终比欧几里得和标量特征值界限更紧，从而加速了多种最先进的渐近最优规划器的收敛。

View on arXiv Download PDF AI Translation

cs.RO / 10 / 2606.02888

Impact of a Soft Wearable Back-Support Device on Postural Stability during Trip-Like Perturbations

软性可穿戴背部支撑装置对类似绊倒扰动下姿势稳定性的影响

Chen, Yuanhao, Khatavkar, Rohan, Nayak, Soubhagya, Sun, Jiefeng, Lee, Hyunglae

Abstract

The effectiveness of a soft wearable back-support device in enhancing postural stability was investigated under trip-like perturbations using two experimental paradigms: perturbed standing and perturbed walking. Healthy subjects completed trials under three different back-support conditions: no device, device worn with low stiffness, and device activated with high stiffness. Whole-body stability was quantified using the minimum Margin of Stability (MOS) at the point of maximal instability. Results demonstrated increased MOS during device use, indicating enhanced postural stability. In standing, MOS increased significantly with device stiffness, whereas in walking, both device conditions improved MOS relative to no device but did not differ significantly from each other. These findings highlight the potential of soft wearable back-support devices with adjustable stiffness to improve reactive balance control against external perturbations, with important implications for fall prevention. Future research should explore personalized stiffness optimization and evaluate efficacy in populations at elevated risk of falls.

Chinese Translation

本研究探讨了一种软性可穿戴背部支撑装置在类似绊倒扰动下增强姿势稳定性的有效性，采用了两种实验范式：扰动站立和扰动行走。健康受试者在三种不同的背部支撑条件下完成试验：无装置、低刚度装置和高刚度激活装置。通过在最大不稳定点的最小稳定边际（Margin of Stability, MOS）来量化全身稳定性。结果表明，使用装置时MOS增加，表明姿势稳定性增强。在站立状态下，MOS随着装置刚度的增加而显著提高，而在行走状态下，两种装置条件相较于无装置均改善了MOS，但彼此之间没有显著差异。这些发现突显了可调刚度的软性可穿戴背部支撑装置在应对外部扰动时改善反应性平衡控制的潜力，对预防跌倒具有重要意义。未来的研究应探索个性化刚度优化，并评估其在高风险人群中的有效性。

View on arXiv Download PDF AI Translation

cs.RO / 11 / 2606.02928

Improved Postural Stability Using a Lightweight Semi-Active Soft Back Support Device Under Standing Perturbations

在站立扰动下使用轻量级半主动软背部支撑装置改善姿势稳定性

Khatavkar, Rohan, Sun, Jiefeng, Lee, Hyunglae

Abstract

Older adults are particularly susceptible to falls following perturbations during standing, such as forward loss of balance. Back support devices that assist trunk extension may help mitigate fall risk by preventing excessive trunk flexion. Previous studies have investigated heavy back support devices; however, these systems often introduced adverse effects on stability due to their added mass, which shifted the body's natural center of mass unfavorably. In contrast, lightweight passive devices have shown limited benefits, as they can generate only modest assistive forces during the relatively small trunk flexion associated with forward balance loss. In this study, we evaluated the effects of a lightweight semi-active soft back support device on postural stability following standing perturbations. Our device combines an active element (a pneumatic artificial muscle) in parallel with a passive elastic band. The active element rapidly provides assistive force following a perturbation, overcoming the limitations of passive devices. Experiments conducted with five healthy individuals demonstrated that the semi-active device significantly reduced whole-body angular momentum and increased the margin of stability, indicating improved balance recovery performance. These results highlight the promise of semi-active soft wearable robots as an effective and lightweight strategy for fall prevention during standing perturbations.

Chinese Translation

老年人在站立时受到扰动后，特别容易跌倒，例如前倾失去平衡。背部支撑装置通过辅助躯干伸展，可能有助于降低跌倒风险，防止过度的躯干屈曲。以往的研究主要集中于重型背部支撑装置；然而，这些系统由于增加的质量，往往对稳定性产生不利影响，导致身体的自然重心位置不利地偏移。相比之下，轻量级被动装置的效果有限，因为它们在与前倾失去平衡相关的相对较小的躯干屈曲中仅能产生适度的辅助力量。在本研究中，我们评估了一种轻量级半主动软背部支撑装置在站立扰动后的姿势稳定性影响。我们的装置将一个主动元件（气动人工肌肉）与一个被动弹性带并联结合。主动元件在扰动后迅速提供辅助力量，克服了被动装置的局限性。对五名健康个体进行的实验表明，半主动装置显著减少了全身角动量，并增加了稳定性边际，表明平衡恢复性能得到了改善。这些结果突显了半主动软穿戴机器人作为在站立扰动期间有效且轻便的跌倒预防策略的潜力。

View on arXiv Download PDF AI Translation

cs.RO / 12 / 2606.02951

SCOPE: Real-Time Natural Language Camera Agent at the Edge

SCOPE：边缘实时自然语言摄像机代理

Hindsbo, Nikolaj, Ehsani, Sina, Mishra, Pragyana

Abstract

Deploying language-driven agents in robotics requires evaluations that reflect real-world task demands: natural-language instructions with reproducible outcomes. Such agents must connect language models to callable perception and control tools, and be assessed using deployment-critical metrics including latency, accuracy, and error modes. We present SCOPE (Simulation and Camera Operations for Perception and Evaluation), a modular agent for natural-language, open-vocabulary pan-tilt-zoom (PTZ) camera control and visual scene understanding, designed explicitly for edge deployment. SCOPE operates both in a Blender-based simulation environment and on a physical PTZ camera, executing all perception, planning, and control locally at the deployment site using edge-accessible compute. We release a 536-task benchmark spanning QA, single- and multi-step commands, counting, spatial reasoning, descriptions, and optical character recognition in a Blender-based simulation environment that exposes realistic PTZ control affordances. Execution traces are combined with an LM-as-Judge to evaluate latency, accuracy, and error modes. We evaluate 19 planner-perception model combinations pairing Qwen3 small language models (SLMs) with Moondream and Qwen vision-language models (VLMs). Stronger SLMs substantially reduce hallucinations and improve tool routing, leading to more reliable closed-loop behavior. Once a sufficiently capable SLM is used, perception becomes the dominant performance bottleneck. Mixture-of-Experts models on both the planning and perception side consistently match or exceed dense alternatives at latencies and memory footprints comparable to much smaller networks. Quantization provides additional efficiency gains with minimal accuracy degradation, identifying a practical, sim-to-real validated design point for real-time, edge-feasible language-driven PTZ control.

Chinese Translation

在机器人领域中部署语言驱动的代理需要评估反映现实世界任务需求的标准：具有可重复结果的自然语言指令。这些代理必须将语言模型与可调用的感知和控制工具连接起来，并使用包括延迟、准确性和错误模式在内的部署关键指标进行评估。我们提出了SCOPE（感知与评估的仿真与摄像机操作），这是一个模块化的代理，专门用于自然语言、开放词汇的平移-倾斜-缩放（PTZ）摄像机控制和视觉场景理解，旨在边缘部署。SCOPE在基于Blender的仿真环境和物理PTZ摄像机上均可操作，所有感知、规划和控制均在部署现场本地执行，利用边缘可访问的计算资源。我们发布了一个包含536个任务的基准测试，涵盖了问答、单步和多步指令、计数、空间推理、描述和光学字符识别等，基于Blender的仿真环境展示了现实的PTZ控制能力。执行轨迹与LM-as-Judge结合，用于评估延迟、准确性和错误模式。我们评估了19种规划-感知模型组合，将Qwen3小型语言模型（SLMs）与Moondream和Qwen视觉-语言模型（VLMs）配对。更强的SLMs显著减少了幻觉现象并改善了工具路由，从而导致更可靠的闭环行为。一旦使用了足够强大的SLM，感知便成为主要的性能瓶颈。规划和感知方面的专家混合模型在延迟和内存占用方面始终与更密集的替代方案相匹配或超越，且与更小的网络相当。量化提供了额外的效率提升，且准确性下降最小，识别出一个实际的、经过仿真到现实验证的设计点，适用于实时、边缘可行的语言驱动PTZ控制。

View on arXiv Download PDF AI Translation

cs.RO / 13 / 2606.02969

Hybrid Dynamics Modeling for a Flexible 2-DoF Robotic Arm

柔性2自由度机器人手臂的混合动力学建模

Popik, Maciek, Yang, Daniel, Bisheban, Mahdis

Abstract

This paper examines three approaches for modeling the dynamics of a flexible-link 2-DoF robotic arm to address unmodeled dynamics not captured by rigid-body models. Two physics informed models combine rigid-body dynamics (RBD) formulations with a Gaussian Mixture Model (GMM) to capture residual model errors and linkage flexibility. A kinematics-based regression model serves as a purely data-driven baseline. Using an open-source dataset, torque predictions are first estimated using Ridge regression on kinematic features, while the physicsbased baseline is constructed from published specifications, and ordinary least-squares regression is subsequently used to estimate the same parameter set directly from data. Results show that the physics-based parameters yield the poorest accuracy, while regularized and least-squares estimators align more closely with measured torques. Residual analysis and error metrics highlight the limitations of purely parametric models for flexible-link systems and underscore the value of regularization and data-driven identification, supporting developments of semi-parametric residual learning methods.

Chinese Translation

本文探讨了三种建模柔性链节2自由度机器人手臂动力学的方法，以解决刚体模型未能捕捉的未建模动力学。两种物理信息模型将刚体动力学（RBD）公式与高斯混合模型（GMM）相结合，以捕捉残余模型误差和链节柔性。基于运动学的回归模型作为纯数据驱动的基线。利用一个开源数据集，首先通过对运动学特征进行岭回归来估计扭矩预测，同时物理基础的基线是根据已发布的规格构建的，随后使用普通最小二乘回归直接从数据中估计相同的参数集。结果表明，基于物理的参数的准确性最差，而正则化和最小二乘估计器与测量的扭矩更为接近。残差分析和误差指标突显了纯参数模型在柔性链节系统中的局限性，并强调了正则化和数据驱动识别的价值，支持半参数残差学习方法的发展。

View on arXiv Download PDF AI Translation

cs.RO / 14 / 2606.02996

MARIO: Motion-Augmented Real-Time Multi-Sensor Inertial Odometry

MARIO：运动增强实时多传感器惯性测程

Li, Yiquan, Yeon, Taeyoung, Gao, Chenfeng, Xu, Vasco, Liu, Xuanyou, Ahuja, Karan

Abstract

Inertial odometry (IO) using only Inertial Measurement Units (IMUs) provides a lightweight solution for human motion tracking in augmented reality (AR) and wearable devices. Recent learning-based IO methods have improved the generalizability of inertial localization through large-scale pretraining on human motion datasets. However, these approaches remain prone to drift and noise because they do not explicitly capture human motion dynamics, especially on daily activity datasets such as Nymeria. In this work, we propose to ground inertial odometry in human kinematics through a learned IMU-inferred pose prior, which promotes physically consistent motion constraints. We integrate this pose prior into existing IO architectures and reduce positional drift by up to 36% on the challenging Nymeria dataset, which is 5x larger than datasets used in prior work. We further improve long-term performance with a sensor-fusion framework that incorporates auxiliary signals from lightweight sensors already available on commercial AR glasses, including magnetometers, barometers, and secondary IMUs. With this fusion strategy, positional drift is reduced by up to 42%, improving robustness and generalization across diverse motion conditions. Together, our results introduce a new paradigm for inertial and lightweight odometry by unifying human motion kinematics with multimodal sensing, setting a new benchmark for accurate and robust camera-less human tracking. Our website is available at https://spice-lab.org/projects/MARIO/.

Chinese Translation

仅使用惯性测量单元（IMUs）的惯性测程（IO）为增强现实（AR）和可穿戴设备中的人类运动跟踪提供了一种轻量级解决方案。最近的基于学习的IO方法通过在大规模人类运动数据集上进行预训练，提高了惯性定位的泛化能力。然而，这些方法仍然容易受到漂移和噪声的影响，因为它们没有明确捕捉人类运动动态，特别是在如Nymeria这样的日常活动数据集上。在本研究中，我们提出通过学习的IMU推断姿态先验将惯性测程与人类运动学相结合，从而促进物理一致的运动约束。我们将这一姿态先验整合到现有的IO架构中，并在具有挑战性的Nymeria数据集上将位置漂移减少了多达36%，该数据集的规模是先前工作中使用的数据集的5倍。我们进一步通过传感器融合框架改善长期性能，该框架整合了来自商业AR眼镜上已存在的轻量级传感器的辅助信号，包括磁力计、气压计和次级IMUs。通过这一融合策略，位置漂移减少了多达42%，提高了在多样化运动条件下的鲁棒性和泛化能力。我们的研究结果为惯性和轻量级测程引入了一种新的范式，通过将人类运动学与多模态传感相结合，设定了无摄像头人类跟踪的准确性和鲁棒性的新基准。我们的官方网站可访问 https://spice-lab.org/projects/MARIO/。

View on arXiv Download PDF AI Translation

cs.RO / 15 / 2606.03047

ModuLoop : Low-Level Code Generation using Modular Synthesizer and Closed-Loop Debugger for Robotic Control

ModuLoop：用于机器人控制的模块化合成器和闭环调试器的低级代码生成

Yoon, Gina, Lee, Sumin, Sim, Joo Yong

Abstract

Large Language Models (LLMs) have demonstrated impressive performance across various domains, including code generation and problem solving. However, their application in robotic control, particularly in low-level tasks that require precise manipulation, real-time feedback, and environment-dependent execution, remains limited. To address this challenge, we propose the Closed-Loop Modular Code Synthesizer framework. This framework leverages a pre-trained LLM without any task-specific fine-tuning to perform modular code planning and generation, and iteratively executes the generated code while inserting debugging probes to observe its behavior. This closed-loop structure facilitates systematic debugging and refinement, ultimately producing executable control programs. We apply the proposed framework to the calibration of an RGB-D camera and a robotic arm, validating its effectiveness in real-world settings. Furthermore, through a subsequent pick-and-place task, we demonstrate not only the accuracy of the calibration but also the potential extensibility of the framework. Across both tasks, the framework achieved high execution accuracy and autonomy, illustrating the practicality and scalability of LLM-based robotic control using our framework.

Chinese Translation

大型语言模型（LLMs）在代码生成和问题解决等多个领域展示了令人印象深刻的性能。然而，它们在机器人控制中的应用，特别是在需要精确操作、实时反馈和环境依赖执行的低级任务中，仍然有限。为了解决这一挑战，我们提出了闭环模块化代码合成器框架。该框架利用预训练的LLM，在不进行任何特定任务微调的情况下，执行模块化代码规划和生成，并在迭代执行生成的代码时插入调试探针以观察其行为。这种闭环结构促进了系统化的调试和改进，最终生成可执行的控制程序。我们将该框架应用于RGB-D相机和机器人手臂的标定，验证了其在现实环境中的有效性。此外，通过后续的拾取和放置任务，我们不仅展示了标定的准确性，还展示了该框架的潜在可扩展性。在这两个任务中，该框架实现了高执行准确性和自主性，说明了基于LLM的机器人控制使用我们框架的实用性和可扩展性。

View on arXiv Download PDF AI Translation

cs.RO / 16 / 2606.03127

TTT-VLA: Test-Time Latent Prompt Optimization for Vision-Language-Action Models

TTT-VLA：用于视觉-语言-动作模型的测试时潜在提示优化

Zhang, Wenbo, Li, Jianxiong, Yang, Shuai, Chen, Sijin, Liu, Jiajun, Liu, Lingqiao, Ma, Xiao

Abstract

Vision-Language-Action (VLA) models trained on large-scale data have made remarkable progress, but they remain vulnerable to distribution shifts at deployment time. Recent VLA models suggest that prompts can serve as an efficient interface for steering policy behavior, but existing prompt-based steering typically relies on external guidance. This raises a natural question: can test-time training (TTT) for VLA be achieved by optimizing a prompt, so that the steering interface itself can be learned and adapted from interaction? We address this question with TTT-VLA, a test-time training framework based on Latent Prompt Optimization (LPO). During training, the latent prompt is learned with an additional proxy task, providing an extra learned conditioning signal for policy learning. At test time, TTT is performed by collecting interaction data from the current environment and optimizing only the latent prompt on those data using the proxy task's self-supervised signal, without modifying the policy itself. Experiments on SimplerEnv demonstrate that the proposed method consistently improves task success rates in both single- and multi-embodiment settings. Further analysis shows that the gains arise primarily from correcting a small number of critical decisions rather than globally altering policy behavior. These results suggest that LPO provides an effective and practical pathway for deployment-time improvement of foundation manipulation policies.

Chinese Translation

训练于大规模数据上的视觉-语言-动作（VLA）模型取得了显著进展，但在部署时仍然容易受到分布变化的影响。近期的VLA模型表明，提示可以作为引导策略行为的有效接口，但现有的基于提示的引导通常依赖于外部指导。这引出了一个自然的问题：是否可以通过优化提示来实现VLA的测试时训练（TTT），使得引导接口本身能够通过交互学习和适应？我们通过TTT-VLA来解决这个问题，这是一种基于潜在提示优化（LPO）的测试时训练框架。在训练过程中，潜在提示通过额外的代理任务进行学习，为策略学习提供额外的学习条件信号。在测试时，通过从当前环境收集交互数据并仅使用代理任务的自监督信号优化这些数据上的潜在提示来执行TTT，而不修改策略本身。在SimplerEnv上的实验表明，所提出的方法在单一和多体现设置中均能持续提高任务成功率。进一步分析表明，收益主要来自于纠正少量关键决策，而不是全局性地改变策略行为。这些结果表明，LPO为基础操作策略的部署时改进提供了一条有效且实用的途径。

View on arXiv Download PDF AI Translation

cs.RO / 17 / 2606.03134

How Visible Are Silent Manipulation Failures? An Observability Study of False-Success Detection in Simulated Robot Episodes

无声操控失败的可见性有多高？模拟机器人情境中虚假成功检测的可观察性研究

Bedi, Aarav

Abstract

Imitation-learning policies for robot manipulation inherit the quality of the success labels attached to their training episodes, and those labels are usually produced by the robot's own success check. A particularly damaging error is the false success: an episode the robot logs as a success when the task outcome was actually wrong. We ask a narrow but practical question about these episodes. Once an episode has already been flagged as a success, how much of the information needed to overturn that label is present in proprioception, and how much requires vision? We build a simulated testbed on two bimanual ALOHA tasks, induce failures through environment perturbations rather than label edits, label every episode by privileged simulator state that the detector never sees, and keep only episodes the robot flagged as successful. We then compare detectors restricted to proprioception against a vision-based detector. We find that recoverability spans a wide range: in cube transfer the false successes are almost fully recoverable from joint data alone, while in peg insertion proprioception recovers only part of them and a vision detector closes most of the gap. We also show that the proprioceptive separability we measure rests on velocity differences far below any realistic sensor noise floor, so it is best read as an optimistic upper bound that a noiseless simulator inflates. We release the generation and evaluation pipeline.

Chinese Translation

机器人操控的模仿学习策略继承了附加于其训练情境的成功标签的质量，而这些标签通常由机器人自身的成功检查生成。一种特别有害的错误是虚假成功：当任务结果实际上是错误时，机器人将某个情境记录为成功。我们提出了一个狭窄但实用的问题：一旦某个情境已被标记为成功，推翻该标签所需的信息有多少来自本体感觉（proprioception），又有多少需要依赖视觉？我们在两个双手 ALOHA 任务上建立了一个模拟测试平台，通过环境扰动而非标签编辑来诱发失败，利用特权模拟器状态对每个情境进行标记，而检测器从未见过这些状态，并仅保留机器人标记为成功的情境。然后，我们比较了仅限于本体感觉的检测器与基于视觉的检测器。我们发现可恢复性范围广泛：在立方体转移任务中，虚假成功几乎可以完全从关节数据中恢复，而在插销插入任务中，本体感觉仅能恢复其中一部分，视觉检测器则弥补了大部分差距。我们还表明，我们测量的本体感觉可分离性基于远低于任何现实传感器噪声底限的速度差异，因此最好将其视为一个乐观的上限，这一上限在无噪声模拟器中被夸大。我们发布了生成和评估管道。

View on arXiv Download PDF AI Translation

cs.RO / 18 / 2606.03177

ConTrack: Constrained Hand Motion Tracking with Adaptive Trade-off Control

ConTrack：具有自适应权衡控制的受限手部运动跟踪

Liang, Yutong, Peng, Quanquan, Qiu, Ri-Zhao, Wang, Xiaolong

Abstract

Human demonstrations provide strong priors for robot manipulation, yet it is non-trivial to transfer them to execute on real robots due to the kinematic gap. In dexterous manipulation, it remains challenging to track long-horizon, contact-rich sequences even in simulators: a reference-tracking policy must keep objects on their target trajectories while preserving demonstrated joint motion and contact timing. Existing approaches often rely on hand-crafted reward tuning that require per-sequence tuning and break under limited interaction budgets. We introduce ConTrack, a reinforcement learning (RL) framework that scales with tracking data. ConTrack treats object tracking as a constraint and allocates remaining control authority to motion fidelity, which allows it to adapt task--style trade-offs online using a dual-variable update. In addition, ConTrack also stabilizes long-horizon learning with an adaptive mid-trajectory reset library that reuses policy-reachable simulator states. Our qualitative and quantitative results in simulation tracking and real robot demonstrate that ConTrack improves success and object pose accuracy significantly over prior arts while preserving joint and contact fidelity. Website: https://www.lyt0112.com/projects/ConTrack.

Chinese Translation

人类示范为机器人操作提供了强有力的先验知识，但由于运动学差距，将其转移到真实机器人上并非易事。在灵巧操作中，即使在模拟器中，跟踪长时间、接触丰富的序列仍然具有挑战性：参考跟踪策略必须在保持物体沿目标轨迹运动的同时，保留示范的关节运动和接触时机。现有方法通常依赖于手工调整的奖励，这需要针对每个序列进行调整，并在有限的交互预算下失效。我们提出了ConTrack，一个与跟踪数据规模相适应的强化学习（RL）框架。ConTrack将物体跟踪视为一种约束，并将剩余的控制权分配给运动保真度，这使其能够通过双变量更新在线适应任务风格的权衡。此外，ConTrack还通过自适应中途重置库来稳定长时间学习，该库重用策略可达的模拟器状态。我们在模拟跟踪和真实机器人中的定性和定量结果表明，ConTrack在成功率和物体姿态准确性上显著优于先前的研究，同时保持了关节和接触的保真度。网站：https://www.lyt0112.com/projects/ConTrack。

View on arXiv Download PDF AI Translation

cs.RO / 19 / 2606.03188

GeoSem-WAM: Geometry- and Semantic-Aware World Action Models

GeoSem-WAM：几何与语义感知的世界行动模型

Ma, Fulong, Peng, Daojie, Yue, Wenjun, Cao, Jiahang, Wang, Bintao, Zhang, Qiang, Ma, Jun

Abstract

Recent World Action Models (WAMs) have demonstrated impressive capabilities in embodied decision-making. However, whether their effectiveness stems from explicit future imagination during inference or representation learning induced by predictive training remains an open question. Emerging evidence suggests the primary advantage lies in learning robust latent representations rather than generating future observations at test time. Nevertheless, existing WAMs mainly rely on RGB-based future prediction, which provides limited structural and spatial understanding of complex environments. To address this, we propose a structured world modeling framework that enhances latent representations through geometric and semantic supervision. Alongside future RGB prediction, our model introduces two auxiliary prediction branches for future geometry and semantic representations, enabling it to jointly capture scene dynamics, spatial geometry, and semantic context within a unified latent space. Crucially, our approach preserves efficient inference by avoiding explicit future rollout or video generation at test time. Extensive experiments show that incorporating structured world supervision consistently improves action prediction accuracy, scene understanding, and robustness under challenging embodied scenarios, highlighting its potential for advancing scalable and efficient WAMs.

Chinese Translation

近期的世界行动模型（WAMs）在具身决策中展现了令人印象深刻的能力。然而，它们的有效性是否源于推理过程中的显式未来想象，或是由预测训练引发的表征学习，仍然是一个悬而未决的问题。新兴证据表明，主要优势在于学习稳健的潜在表征，而非在测试时生成未来观察。然而，现有的WAMs主要依赖基于RGB的未来预测，这对复杂环境的结构和空间理解提供了有限的支持。为了解决这一问题，我们提出了一种结构化的世界建模框架，通过几何和语义监督增强潜在表征。在未来RGB预测的基础上，我们的模型引入了两个辅助预测分支，用于未来几何和语义表征，使其能够在统一的潜在空间中共同捕捉场景动态、空间几何和语义上下文。至关重要的是，我们的方法通过避免在测试时显式的未来展开或视频生成，保持了高效的推理。大量实验表明，结合结构化的世界监督始终提高了行动预测的准确性、场景理解能力以及在具有挑战性的具身场景下的鲁棒性，突显了其在推动可扩展和高效WAMs方面的潜力。

View on arXiv Download PDF AI Translation

cs.RO / 20 / 2606.03204

Toward Gripper-Integrated Active Electrosense for Pre-Contact Sensing in Underwater Soft Grippers

朝向集成抓手的主动电感知用于水下软抓手的接触前感知

Tanveer, Ahsan, Hamza, Muhammad, Afridi, Waqar Hussain, Wang, Chen, Xie, Guangming

Abstract

Underwater manipulation often occurs under degraded visibility due to turbidity, glare, and gripper occlusion, limiting the reliability of vision-based perception during approach and grasping. In such settings, soft grippers are well suited for compliant interaction, but they typically lack an onboard pre-contact cue that can guide approach and closure when vision is unreliable. This extended abstract explores active electrosense as a lightweight sensing modality that can provide a proximity-like signal prior to contact by measuring perturbations of an applied electric field in conductive media. We instrument an octopus-inspired gripper with a discrete electrode layout and record multi-channel sensing voltages using off-the-shelf hardware. Simulation and tank experiments with a suspended conductive sphere show structured, object-dependent changes in the multi-electrode voltage readout relative to empty-water baselines, with detectability varying across excitation of 5 to 20 V and frequencies from 1 mHz to 1 kHz. These findings motivate systematic investigation of gripper-integrated electrosense as a complementary pre-contact cue for underwater soft manipulation.

Chinese Translation

水下操作通常在能见度降低的情况下进行，受到浑浊、眩光和抓手遮挡的影响，这限制了在接近和抓取过程中的基于视觉的感知的可靠性。在这种环境下，软抓手非常适合进行柔性交互，但通常缺乏能够在视觉不可靠时指导接近和闭合的机载接触前信号。本文扩展摘要探讨了主动电感知作为一种轻量级感知方式，通过测量导电介质中施加电场的扰动来提供接触前的类似接近信号。我们为一种受章鱼启发的抓手配备了离散电极布局，并使用现成的硬件记录多通道感应电压。与空水基线相比，悬挂导电球体的仿真和水槽实验显示出多电极电压读数中相对于物体的结构性变化，且可检测性在5至20 V的激励和1 mHz至1 kHz的频率范围内变化。这些发现激励我们系统地研究集成电感知的抓手作为水下软操作的补充接触前信号。

View on arXiv Download PDF AI Translation

cs.RO / 21 / 2606.03223

BotDirector: Robot Storytelling Across the Symmetrical Reality with Multi-modal Interactions

BotDirector：通过多模态交互在对称现实中进行机器人讲故事

Sun, Zhe, Wang, Meng, Wang, Lei, Wang, Yuxi, Li, Wanxin, Peng, Yujia, Zhang, Zhenliang

Abstract

Robot storytelling offers a unique blend of technological innovation and creative expression that engages children in unprecedented ways. However, the technical aspects are often too complicated for children. We propose an interactive system that facilitates robot storytelling with tangible and natural language interactions. Children arrange the playground with their own stuff and create narratives with an LLM agent. The created narratives are transformed into a motion sequence based on the map and characters, and the motions are executed by self-navigating swarm robots. This system enhances robot storytelling with flexible scenarios, enabling young children to create robot dramas with everyday objects.

Chinese Translation

机器人讲故事提供了一种独特的技术创新与创造性表达的结合，以前所未有的方式吸引儿童。然而，技术方面往往对儿童来说过于复杂。我们提出了一种互动系统，促进机器人讲故事，采用可触摸和自然语言的交互方式。儿童可以用自己的物品布置游乐场，并与大型语言模型（LLM）代理一起创造叙事。所创作的叙事会根据地图和角色转化为运动序列，并由自导航的群体机器人执行。这一系统增强了机器人讲故事的灵活场景，使得幼儿能够用日常物品创造机器人戏剧。

View on arXiv Download PDF AI Translation

cs.RO / 22 / 2606.03240

GeoAlign: Beyond Semantics with State-Guided Spatial Alignment in VLA Models

GeoAlign：在 VLA 模型中通过状态引导的空间对齐超越语义

Chen, Yizhi, Cao, Zhanxiang, Peng, Xinyi, Zheng, Yixiao, Si, Xiaxi, Li, Yiheng, Yan, Liyun, Zhu, Keqi, Chen, Xueyun, Fu, Shengcheng, Zhan, Tianyue, Jia, Yufei, Yao, Jinming, Xie, Yan, Wang, Kun, Lu, Cewu, Gao, Yue

Abstract

Current Vision--Language--Action (VLA) models often optimize for semantic grounding, whereas executable manipulation requires geometry-aware spatial alignment and dynamic affordance selection. We introduce GeoAlign, a state-guided spatial alignment architecture for VLA policy learning. GeoAlign post-trains an RGB geometry branch with robot-domain RGB-D supervision, yielding RGB-derived Geometry-Enhanced Post-Trained (GEP) features for policy rollout. The robot's proprioceptive state queries the GEP feature grid, producing compact, phase-dependent geometry tokens for action prediction. GeoAlign achieves 99.0% on LIBERO, 85.3% across three SimplerEnv-Fractal tasks, and 78.8% on eight geometry-critical real-world ALOHA tasks, with ablations confirming the value of geometry post-training and proprioceptive-state-guided querying.

Chinese Translation

当前的视觉-语言-动作（VLA）模型通常优化语义基础，而可执行的操作需要考虑几何的空间对齐和动态的可供性选择。我们提出了 GeoAlign，一种用于 VLA 策略学习的状态引导空间对齐架构。GeoAlign 通过机器人领域的 RGB-D 监督对 RGB 几何分支进行后训练，生成用于策略展开的 RGB 派生几何增强后训练（GEP）特征。机器人的本体状态查询 GEP 特征网格，生成紧凑的、相位依赖的几何标记用于动作预测。GeoAlign 在 LIBERO 上达到了 99.0%，在三个 SimplerEnv-Fractal 任务上达到了 85.3%，在八个几何关键的现实世界 ALOHA 任务上达到了 78.8%，消融实验确认了几何后训练和本体状态引导查询的价值。

View on arXiv Download PDF AI Translation

cs.RO / 23 / 2606.03252

AirDreamer: Generalist Drone Navigation with World Models

AirDreamer：基于世界模型的通用无人机导航

Liu, Zian, Yang, Andong, Yang, Chunkai, An, Ruidong, Gao, Chao, Zhou, Guyue

Abstract

Navigating a drone in unseen and cluttered environments requires reliable generalization to unseen scene layouts and understanding of environmental structure relative to the robot's capabilities. Previous methods, which assume the same environment configuration, often rely heavily on human-designed perception pipelines and predefined rules to guide the robot toward the target. This process is environment-dependent and generalizes poorly across environments. Inspired by animal navigation behavior, we design a navigation framework that navigates with a reinforcement-learning-based policy on top of a world-model-based environment understanding to overcome these issues. In addition, a sparse reward function without hand-crafted shaping terms is designed to avoid local minima traps and encourage yaw control behaviors. In simulation and on real drones, our method exhibits emergent capabilities for navigating complex, unseen environments and escaping local optima where other methods fail. In challenging maps, it achieves a 5.3% higher navigation success rate than best baseline. Furthermore, the proposed framework achieves effective sim-to-real transfer without any tuning during deployment. The code will be publicly available.

Chinese Translation

在未知且杂乱的环境中导航无人机需要对未见场景布局的可靠泛化以及对环境结构相对于机器人能力的理解。以往的方法假设环境配置相同，通常过于依赖人工设计的感知管道和预定义规则来引导机器人朝向目标。这一过程依赖于特定环境，且在不同环境间的泛化能力较差。受到动物导航行为的启发，我们设计了一种导航框架，该框架基于强化学习策略，结合世界模型的环境理解，以克服这些问题。此外，我们设计了一种稀疏奖励函数，避免手工设计的塑形项，以防止局部最小值陷阱，并鼓励偏航控制行为。在仿真和真实无人机上，我们的方法展现了在复杂未知环境中导航和逃离局部最优的突现能力，而其他方法则未能实现。在具有挑战性的地图中，其导航成功率比最佳基线高出5.3%。此外，所提出的框架在部署过程中实现了有效的仿真到现实转移，无需任何调优。代码将公开发布。

View on arXiv Download PDF AI Translation

cs.RO / 24 / 2606.03265

Wheel-Mounted/GNSS Fusion with AI-Aided Position Updates

轮载/GNSS 融合与 AI 辅助位置更新

Versano, Gal, Klein, Itzik

Abstract

Accurate and robust localization remains a fundamental challenge for autonomous ground vehicles. In this work, we propose a hybrid neural inertial navigation framework that integrates a wheel-mounted inertial sensors, enforced periodic trajectories, and a simple, efficient neural network capable of regressing vehicle displacement with GNSS position updates in an error-state extended Kalman filter. The periodic trajectories increase the inertial signal-to-noise ratio, allowing the network to use only inertial readings to estimate displacement. The approach is validated through real-world experiments using multiple wheel-mounted inertial sensors. Experimental results demonstrate that the proposed method achieves a significant improvement in positioning accuracy, reducing the position root mean squared error by approximately 46 % compared to standard wheel-mounted inertial sensor fusion with GNSS updates.

Chinese Translation

准确且稳健的定位仍然是自主地面车辆面临的基本挑战。在本研究中，我们提出了一种混合神经惯性导航框架，该框架整合了轮载惯性传感器、强制周期性轨迹以及一个简单高效的神经网络，该网络能够在误差状态扩展卡尔曼滤波器中利用 GNSS 位置更新回归车辆位移。周期性轨迹提高了惯性信号的信噪比，使网络能够仅使用惯性读数来估计位移。通过使用多个轮载惯性传感器进行的实际实验验证了该方法。实验结果表明，所提出的方法在定位精度上取得了显著改善，与标准的轮载惯性传感器与 GNSS 更新融合相比，位置均方根误差减少了约 46%。

View on arXiv Download PDF AI Translation

cs.RO / 25 / 2606.03268

EaDex: A Cross-Embodiment Dexterous Manipulation Framework from Low-Cost Demonstrations

EaDex：一种基于低成本演示的跨体现灵巧操作框架

Zhao, Qian, Tong, Xin, Wu, Chengdong, Yang, Yang, Li, Yingtian

Abstract

Dexterous manipulation learning has long been hindered by the high costs of data and training, as pure reinforcement learning typically requires large-scale interactive exploration and imitation learning depends on high-quality demonstrations that are expensive to collect. To address this problem, we propose EaDex, a multi-embodiment dexterous manipulation learning framework under low-cost demonstration conditions, which enables rapid generation of demonstration data and consequently reduces training time for efficient dexterous manipulation. At the data level, EaDex captures human hand motions using only a single RGB-D camera and constructs structured demonstration data through MANO-based hand modeling, data normalization, and motion retargeting. At the learning level, we introduce a contact-reward-based dynamic demonstration annealing mechanism, which guides early-stage exploration under demonstration and gradually transitions to autonomous optimization with accumulating contact rewards. Using our custom dataset, we evaluate EaDex on three dexterous hands and three articulated object-opening tasks, covering nine cross-embodiment manipulation settings, achieving a 55.3% relative improvement over the baseline without demonstration annealing. These results validate the effectiveness of the proposed low-cost demonstration pipeline and the dynamic demonstration annealing strategy for dexterous manipulation learning.

Chinese Translation

灵巧操作学习长期以来受到数据和训练高成本的制约，因为纯强化学习通常需要大规模的互动探索，而模仿学习依赖于高质量的演示，这些演示的收集成本昂贵。为了解决这个问题，我们提出了EaDex，一种在低成本演示条件下的多体现灵巧操作学习框架，能够快速生成演示数据，从而减少高效灵巧操作的训练时间。在数据层面，EaDex仅使用单个RGB-D相机捕捉人手动作，并通过基于MANO的手部建模、数据标准化和动作重定向构建结构化演示数据。在学习层面，我们引入了一种基于接触奖励的动态演示退火机制，该机制指导早期阶段在演示下的探索，并逐渐过渡到随着接触奖励累积的自主优化。使用我们的自定义数据集，我们在三种灵巧手和三种关节物体开启任务上评估了EaDex，涵盖九种跨体现操作设置，相较于没有演示退火的基线实现了55.3%的相对提升。这些结果验证了所提出的低成本演示管道和动态演示退火策略在灵巧操作学习中的有效性。

View on arXiv Download PDF AI Translation

cs.RO / 26 / 2606.03296

Bridging Predictive Uncertainty and Safe Action: Sample-Conditioned Differentiable Planning for Autonomous Driving

弥合预测不确定性与安全行动之间的鸿沟：用于自动驾驶的样本条件可微规划

Meng, Chengzhen, Liu, Pei, Huang, Zhiyu, Lv, Chen, Ma, Jun

Abstract

Complex, dynamic, and interactive driving environments pose significant challenges for autonomous driving, primarily due to the pervasive uncertainty of surrounding traffic. A fundamental bottleneck in current systems is the disconnect between highly expressive uncertainty modeling and interpretable, safe motion planning. In this paper, we propose a novel sample-conditioned differentiable planning framework that bridges this gap by explicitly incorporating diffusion-generated future trajectories into the optimization process. Rather than compressing predictions into a single deterministic future or relying on black-box end-to-end architectures, our approach leverages a conditional diffusion model to generate a diverse set of plausible future scenarios. Crucially, these samples are directly fed into a differentiable planner, which explicitly mitigates predictive uncertainty via an empirical Conditional Value-at-Risk (CVaR) tail-risk constraint. This allows the planner to optimize a physically interpretable trajectory that is robust to rare yet safety-critical interactions. Furthermore, we introduce a directed graph representation for scene context that yields substantial improvements in both predictive effectiveness and computational efficiency. Validated through extensive open-loop and closed-loop evaluations on the Waymo Open Motion and Argoverse 2 datasets, our framework significantly outperforms state-of-the-art baselines in safety, efficiency, and ride comfort.

Chinese Translation

复杂、动态和互动的驾驶环境对自动驾驶提出了重大挑战，主要由于周围交通的普遍不确定性。目前系统的一个根本瓶颈在于高度表达的不确定性建模与可解释的安全运动规划之间的脱节。本文提出了一种新颖的样本条件可微规划框架，通过将扩散生成的未来轨迹明确地纳入优化过程，弥合了这一鸿沟。我们的方案不再将预测压缩为单一的确定性未来或依赖黑箱端到端架构，而是利用条件扩散模型生成一组多样的合理未来场景。关键是，这些样本被直接输入到可微规划器中，该规划器通过经验条件风险价值（Conditional Value-at-Risk, CVaR）尾部风险约束显式减轻预测不确定性。这使得规划器能够优化出一种在面对稀有但安全关键交互时具有鲁棒性的物理可解释轨迹。此外，我们引入了一种场景上下文的有向图表示，显著提高了预测有效性和计算效率。通过在Waymo Open Motion和Argoverse 2数据集上进行广泛的开环和闭环评估，我们的框架在安全性、效率和乘坐舒适性方面显著超越了最先进的基线。

View on arXiv Download PDF AI Translation

cs.RO / 27 / 2606.03297

SplitAdapter: Load-Aware Humanoid Loco-Manipulation via Factorized Adaptation

SplitAdapter：基于因子适应的负载感知类人运动操控

Kang, Jeonguk, Cho, Hanbyel, Kang, Sanghyun, Koo, Donghan

Abstract

Humanoid loco-manipulation requires stable whole-body control under varying object masses and pickup/placement heights. This becomes particularly challenging in sim-to-real transfer, where object-induced load variation and robot-side dynamics mismatch interact during physical contact. Existing history-based adapters often compress these factors into a single latent representation, which can weaken robustness under heavy-load manipulation. We propose \textbf{SplitAdapter: Load-Aware Humanoid Loco-Manipulation via Factorized Adaptation}, which freezes a pretrained box manipulation policy and extends it with object/load and dynamics-aware context encoders trained with split world-model objectives, GRL-based cross-adversarial regularization, and hierarchical Feature-wise Linear Modulation (FiLM). In sim-to-sim experiments and real-world deployment, SplitAdapter improves Full-task success over the base policy and world-model FiLM baselines across object masses of $2$, $4$, and $6$ kg and pickup/placement heights of $0$, $30$, and $60$ cm, with the largest improvements under heavy-load conditions.

Chinese Translation

类人运动操控需要在不同物体质量和拾取/放置高度下实现稳定的全身控制。这在模拟到现实的转移中尤为具有挑战性，因为物体引起的负载变化和机器人侧的动态不匹配在物理接触时相互作用。现有的基于历史的适配器通常将这些因素压缩为单一的潜在表示，这可能在重负载操控下削弱鲁棒性。我们提出了 extbf{SplitAdapter：基于因子适应的负载感知类人运动操控}，该方法冻结了一个预训练的盒子操控策略，并通过使用分割世界模型目标、基于GRL的交叉对抗正则化和分层特征线性调制（FiLM）训练的物体/负载和动态感知上下文编码器进行扩展。在模拟到模拟的实验和现实世界的部署中，SplitAdapter在物体质量为$2$、$4$和$6$ kg，以及拾取/放置高度为$0$、$30$和$60$ cm的情况下，相较于基础策略和世界模型FiLM基线，提高了全任务成功率，尤其在重负载条件下取得了最大的改善。

View on arXiv Download PDF AI Translation

cs.RO / 28 / 2606.03312

RobotValues: Evaluating Household Robots When Human Values Conflict

RobotValues：当人类价值观发生冲突时评估家用机器人

Han, Jongwook, Kim, Hyeongjin, Jo, Yohan

Abstract

While household robots are often evaluated based on task completion, everyday domestic environments involve value-conflicting situations in which robots are expected to choose actions that prioritize other values than task success, such as human autonomy, efficiency, or social appropriateness. Yet, there are no benchmarks for evaluating robots' value preferences in such scenarios. We introduce RobotValues, a benchmark to evaluate household robot planners in 10K value-conflict scenarios. Each instance consists of a realistic household image with multiple plausible robot actions that prioritize different human values. We construct RobotValues through LLM-assisted scenario generation, stakeholder-grounded value extraction, image generation and automatic quality control. Using RobotValues we evaluate VLMs used in robotics and find that models exhibit default value preferences, including safety and accommodation, while underselecting privacy-prioritizing actions. When the models are instructed to prioritize specific values that conflict with their own preferences, they often fail to override their default actions, choosing incorrect actions for 80% of the time. These findings suggest that household robot evaluation should measure not only task completion or safety compliance, but also whether robots can choose among plausible actions when human values conflict.

Chinese Translation

虽然家用机器人通常基于任务完成情况进行评估，但日常家庭环境中存在价值冲突的情境，在这些情境中，机器人被期望选择优先考虑其他价值的行动，而非任务成功，例如人类自主性、效率或社会适宜性。然而，目前尚无评估机器人在此类情境中价值偏好的基准。我们提出了RobotValues，这是一个用于评估家用机器人规划者在10,000个价值冲突情境中的基准。每个实例由一幅现实的家庭图像和多个优先考虑不同人类价值的合理机器人行动组成。我们通过大型语言模型（LLM）辅助的情境生成、利益相关者基础的价值提取、图像生成和自动质量控制构建了RobotValues。使用RobotValues，我们评估了用于机器人技术的视觉语言模型（VLM），发现这些模型表现出默认的价值偏好，包括安全性和适应性，同时对优先考虑隐私的行动选择不足。当模型被指示优先考虑与其自身偏好相冲突的特定价值时，它们往往无法覆盖默认行动，选择错误行动的比例高达80%。这些发现表明，家用机器人评估不仅应测量任务完成或安全合规性，还应评估机器人在面对人类价值冲突时是否能够在合理行动中进行选择。

View on arXiv Download PDF AI Translation

cs.RO / 29 / 2606.03335

GPU-Parallel Multi-Task Reinforcement Learning with Demonstration Guided Policy Optimization

基于GPU并行的多任务强化学习与示范引导策略优化

Zhang, Rui, Wu, Qiwei, Zhang, Zhengyu, Li, Tao, Guo, Yunrong, Lai, Junjie, Xu, Renjing, Zhang, Weihua

Abstract

Large scale GPU-parallel reinforcement learning has changed what can be trained in robot simulation, yet most systems still optimize one specialist policy per task. We propose a construction methodology for turning structured manipulation task families into GPU-parallel multi-task RL benchmarks, and instantiate it as MT-Libero using LIBERO assets and task predicates in Isaac Lab. The resulting benchmark supports simultaneous reinforcement learning over heterogeneous task suites with parallel rendering, physics randomization, and state-input or visual-input policies. To make such training practical under sparse success signals and limited prior data, we further propose DGPO, an on-policy demonstration guided method that combines importance weighted PPO with adaptive behavior cloning on matched demonstration actions. DGPO enables a tunable preference toward demonstrated task distributions, outperforming both prior-free RL and existing demonstration-based methods while preserving the stability and online improvement benefits of on-policy PPO.

Chinese Translation

大规模GPU并行强化学习改变了机器人仿真中可训练的内容，然而大多数系统仍然为每个任务优化一个专门的策略。我们提出了一种构建方法，将结构化的操作任务家族转化为GPU并行多任务强化学习基准，并将其实例化为MT-Libero，使用LIBERO资产和Isaac Lab中的任务谓词。所得到的基准支持在异构任务套件上进行同时强化学习，具备并行渲染、物理随机化以及状态输入或视觉输入策略。为了在稀疏成功信号和有限先前数据的情况下使这种训练变得可行，我们进一步提出了DGPO，一种基于策略的示范引导方法，它结合了重要性加权的PPO与在匹配示范动作上的自适应行为克隆。DGPO使得对示范任务分布的偏好可调，优于无先验强化学习和现有的基于示范的方法，同时保持了基于策略的PPO的稳定性和在线改进优势。

View on arXiv Download PDF AI Translation

cs.RO / 30 / 2606.03340

Autonomous Navigation System for Library Service Robot Based on Unitree Go2 Edu

基于Unitree Go2 Edu的图书馆服务机器人自主导航系统

Li, Aoduo, Lv, Haoran, Ou, Bingquan, Li, Jianfeng, Li, Yingdong, Li, Zimeng

Abstract

Libraries require autonomous robots to move quietly through narrow aisles while remaining safe around readers, chairs, bags, and carts. This paper presents a ROS 2 navigation system for a Unitree Go2 Edu quadruped equipped with a 4D LiDAR, a front depth camera, and an IMU. Rather than assuming the library is rough terrain, we target the practical mobility discontinuities of real deployments, including floor transitions, temporary clutter, and partially blocked passages where low-clearance wheeled platforms are less tolerant. RTAB-Map is used for visual-LiDAR SLAM, AMCL and EKF-based sensor fusion provide localization, and a Nav2 stack with A* and DWA supports planning and local avoidance. In a real library, the system achieves 100%, 96%, and 88% success rates in static, low-density dynamic, and high-density dynamic scenes, while map validation against surveyed control distances yields a mean metric error of 3.7 cm.

Chinese Translation

图书馆需要自主机器人在狭窄的过道中安静移动，同时确保在读者、椅子、包和推车周围的安全。本文提出了一种针对配备4D LiDAR、前深度相机和IMU的Unitree Go2 Edu四足机器人使用的ROS 2导航系统。我们并不假设图书馆是粗糙的地形，而是针对实际部署中的移动不连续性，包括地面过渡、临时杂物和部分阻塞的通道，这些地方低底盘的轮式平台容忍度较低。RTAB-Map用于视觉-LiDAR SLAM，AMCL和基于EKF的传感器融合提供定位，Nav2栈结合A*和DWA支持规划和局部避障。在真实的图书馆中，该系统在静态、低密度动态和高密度动态场景中分别实现了100%、96%和88%的成功率，而与调查控制距离的地图验证结果显示平均度量误差为3.7厘米。

View on arXiv Download PDF AI Translation

cs.RO / 31 / 2606.03374

eMEM: A Hybrid Spatio-Temporal Memory System For Embodied Agents

eMEM：一种用于具身智能体的混合时空记忆系统

Rasheed, A. Haroon, Kabtoul, Maria

Abstract

We present eMEM (Embodied Memory), a hybrid graph-based memory system for embodied agents operating in physical environments. Current agent memory architectures, such as Generative Agents, MemGPT, and A-MEM, treat memory as text streams or knowledge graphs, but embodied agents require memory that is simultaneously searchable by meaning, space, and time. eMEM fills this gap with a multi-index architecture (SQL ITE for structured storage, hnswlib for approximate nearest neighbour semantic search, and an R-tree for spatial queries) unified behind a single graph model. A tiered consolidation pipeline transforms raw perceptual observations into compressed summaries, mirroring hippocampal-neocortical consolidation in biological systems. Ten agent-facing recall tools expose memory retrieval primitives, including concept-to-location resolution and cross layer recall, as first-class operations for LLM tool calling. The system is fully embedded and runs in-process alongside the agent. In addition we introduce eMEM-Bench v1, a benchmark we construct over ProcTHOR-10K scenes for embodied memory evaluation. The benchmark is organised explicitly around eight cognitive-psychology paradigms (DRM lures, pattern separation, pattern completion, source monitoring, context-dependent retrieval, long-horizon interference, serial position, and a foil augmented retention curve), each chosen so that the result is interpretable against the broader memory-systems literature in humans and prior agent-memory systems; a level of diagnostic that surface-task benchmarks like LoCoMo or OpenEQA cannot provide. eMEM scores 80.8 weighted mean over 988 probes, with a flat retention curve at ceiling from 1 h to 1 yr of simulated delay on room-unique items. We show that a pure RAG baseline (the flat_rag ablation) loses 30 pt on context dependent retrieval and 29 pt on DRM lure rejection, isolating the contribution of multi-layer storage and consolidation respectively. We release both the system and the benchmark code.

Chinese Translation

我们提出了eMEM（具身记忆），这是一种基于图的混合记忆系统，专为在物理环境中操作的具身智能体设计。目前的智能体记忆架构，如生成智能体（Generative Agents）、MemGPT和A-MEM，将记忆视为文本流或知识图谱，但具身智能体需要一种能够同时按意义、空间和时间进行搜索的记忆。eMEM通过一个多索引架构（结构化存储的SQL ITE、用于近似最近邻语义搜索的hnswlib和用于空间查询的R树）填补了这一空白，这些都统一在一个单一的图模型之下。一个分层整合管道将原始感知观察转化为压缩摘要，反映了生物系统中海马体-新皮层的整合过程。十个面向智能体的回忆工具暴露了记忆检索原语，包括概念到位置的解析和跨层回忆，作为大型语言模型（LLM）工具调用的一级操作。该系统完全嵌入并与智能体并行运行。此外，我们还推出了eMEM-Bench v1，这是我们在ProcTHOR-10K场景上构建的用于具身记忆评估的基准。该基准明确围绕八个认知心理学范式（DRM诱饵、模式分离、模式完成、来源监控、上下文依赖检索、长期干扰、序列位置和增强保留曲线的干扰）进行组织，每个范式的选择使得结果可以与人类及先前智能体记忆系统的更广泛记忆系统文献进行可解释的比较；这是像LoCoMo或OpenEQA这样的表面任务基准无法提供的诊断水平。eMEM在988个探测器上得分为80.8的加权平均，在房间唯一项目的1小时到1年模拟延迟下保持平坦的保留曲线。我们展示了纯RAG基线（flat_rag消融）在上下文依赖检索上损失了30分，在DRM诱饵拒绝上损失了29分，分别隔离了多层存储和整合的贡献。我们发布了系统和基准代码。

View on arXiv Download PDF AI Translation

cs.RO / 32 / 2606.03385

Grasp-Then-Plan with Failure Attribution: A Closed Two-Stage Framework for Precise and Generalizable Robotic Manipulation

带有失败归因的抓取-再规划：一种精确且可推广的机器人操作的封闭双阶段框架

Xu, Jiahao, Wang, Peiyuan, Zhang, Hanzhuo, Yu, Zihao, Fu, Tianyu, Chen, Hao, Xiang, Xuanhao, Yu, Jianbo, Fu, Chenchen, Wang, Wanyuan

Abstract

In robotic manipulation, the tight coupling between grasping and motion planning often obscures the true source of failure, leading to inefficient trial-and-error. To enable efficient long-horizon manipulation, we propose GTP-FA (Grasp-Then-Plan with Failure Attribution), a task-oriented two-stage grasp-then-plan framework that generates grasp candidates and performs downstream motion planning conditioned on the selected grasp. Given a failed manipulation trajectory, we learn a failure attribution model that generalizes to unseen grasps and produces a stable distribution over failure modes for diagnosis-guided optimization. Based on these attribution results, we then optimize both modules in a diagnosis-driven manner: on the grasping side, we inject task-level priors and risk penalties into grasp candidate scoring and optimization to suppress unstable or task-incompatible grasps; on the planning side, we target high-risk initial states through data collection and fine-tuning to address genuine planning bottlenecks. We evaluate the proposed framework in both simulation and real-robot experiments, and show that GTP-FA improves the corresponding base learners across RL, IL, diffusion-policy, and VLA-based settings, achieving substantially higher overall task success rates.

Chinese Translation

在机器人操作中，抓取与运动规划之间的紧密耦合常常掩盖了失败的真实来源，导致低效的试错过程。为了实现高效的长时间操作，我们提出了GTP-FA（带有失败归因的抓取-再规划），这是一种面向任务的双阶段抓取-再规划框架，能够生成抓取候选并基于所选抓取进行下游运动规划。针对失败的操作轨迹，我们学习了一种失败归因模型，该模型能够推广到未见过的抓取，并为诊断引导优化生成稳定的失败模式分布。基于这些归因结果，我们以诊断驱动的方式优化两个模块：在抓取方面，我们将任务级先验和风险惩罚注入抓取候选的评分和优化中，以抑制不稳定或与任务不兼容的抓取；在规划方面，我们通过数据收集和微调针对高风险初始状态，以解决真正的规划瓶颈。我们在仿真和真实机器人实验中评估了所提出的框架，并显示GTP-FA在强化学习（RL）、模仿学习（IL）、扩散策略和基于变分自编码器（VLA）的设置中提高了相应基础学习者的性能，显著提高了整体任务成功率。

View on arXiv Download PDF AI Translation

cs.RO / 33 / 2606.03390

Extreme Motion Generation via Hybrid Null-Space Control for Straight-Line Path Following

通过混合零空间控制生成极端运动以实现直线轨迹跟踪

Yuan, Xinyi, Wan, Weiwei, Harada, Kensuke

Abstract

This work studies ``extreme motion generation'', which aims to maximize the Cartesian path length along a pre-defined trajectory within the manipulator's workspace. This objective is important in industry as long as path-following is fundamental to a large variety of tasks such as surface coating and welding. More critically, extreme motion enables a fixed-base manipulator to exploit the kinematic capability under limited reachability. However, such exploitation is challenging in practice, as the manipulator must actively avoid the safety boundary through execution, which is inherently a long-horizon problem. Accordingly, we claim that long-horizon decision-making should be delegated to a learning-based policy to maximize exploitation, while a classical model-based controller covers the near-boundary region, where the learning policy degrades sharply due to sparse data coverage. In detail, our proposed method is a step-level hybrid controller that switches between an RL-based and a model-based controller according to the normalized joint-limit distance. The initial joint configuration is sampled through conditional diffusion-based sampling, which improves the achievable path length based on the learned motion prior. We evaluate the proposed framework on 10,000 straight-line path-following tasks with a 7-DoF Franka FR3, extending the average rollout length by 27\% over the model-based baseline. Notably, certain tasks yield a pronounced extension toward the motion extreme, as reflected in the maximum improvement reported in the statistical results. The project website and related videos of this paper can be found at https://yuan-xinyi.github.io/extreme-motion-generation/.

Chinese Translation

本研究探讨了“极端运动生成”，旨在最大化操纵器工作空间内沿预定义轨迹的笛卡尔路径长度。该目标在工业中具有重要意义，因为路径跟踪是表面涂层和焊接等多种任务的基础。更重要的是，极端运动使固定基座的操纵器能够在有限的可达性下利用运动学能力。然而，实际操作中这种利用是具有挑战性的，因为操纵器必须在执行过程中主动避免安全边界，这本质上是一个长时间跨度的问题。因此，我们认为应将长时间跨度的决策制定委托给基于学习的策略，以最大化利用，而经典的基于模型的控制器则覆盖近边界区域，在该区域内，由于数据覆盖稀疏，学习策略的性能急剧下降。具体而言，我们提出的方法是一个分步混合控制器，根据归一化的关节极限距离在基于强化学习（RL）和基于模型的控制器之间切换。初始关节配置通过条件扩散采样进行采样，这基于学习到的运动先验改善了可实现的路径长度。我们在10,000个直线轨迹跟踪任务上评估了所提出的框架，使用7自由度的Franka FR3，相较于基于模型的基线，平均执行长度提高了27%。值得注意的是，某些任务在运动极限方面表现出显著的扩展，这在统计结果中反映为最大改进。项目网站及相关视频可在 https://yuan-xinyi.github.io/extreme-motion-generation/ 找到。

View on arXiv Download PDF AI Translation

cs.RO / 34 / 2606.03392

OpenEAI-Platform: An Open-source Embodied Artificial Intelligence Hardware-Software Unified Platform

OpenEAI-平台：一个开源的具身人工智能硬件-软件统一平台

Zhang, Jinyuan, Fan, Luoyi, Wang, Leiyu, Wang, Yeqiang, Zhu, Yicheng, Lu, Cewu, Ye, Nanyang

Abstract

Embodied AI in the real world requires both accurate hardware and robust vision-language-action (VLA) policies. We present OpenEAI-Platform, a fully open-source platform that integrates a low-cost 6+1 degree-of-freedom (dof) robotic arm (OpenEAI-Arm) and a reproducible VLA model (OpenEAI-VLA). OpenEAI-Arm provides open-source mechanical designs for low manufacturing cost and compliant control methods for higher accuracy. OpenEAI-VLA builds on Qwen3-VL-4B and uses a Diffusion Transformer action head, and is trained in two stages with only open-source robot and multimodal datasets. Across four real-world manipulation tasks, OpenEAI-Arm outperforms two commercial 6+1-dof arms under the same policy, and OpenEAI-VLA achieves success rates comparable to the large-scale pretrained pi0 baseline with only limited pretraining data. We will release the full hardware designs, drivers, models, and training/data pipelines to support reproducible research and scalable data collection. Our codes, layouts, and models will be released after the paper is accepted.

Chinese Translation

具身人工智能在现实世界中需要精确的硬件和强大的视觉-语言-动作（VLA）策略。我们提出了OpenEAI-平台，一个完全开源的平台，集成了低成本的6+1自由度（dof）机器人手臂（OpenEAI-Arm）和可复现的VLA模型（OpenEAI-VLA）。OpenEAI-Arm提供了开源的机械设计，以降低制造成本，并采用合规控制方法以提高精度。OpenEAI-VLA基于Qwen3-VL-4B，使用扩散变换器（Diffusion Transformer）动作头，并在仅使用开源机器人和多模态数据集的情况下进行两阶段训练。在四个现实世界的操作任务中，OpenEAI-Arm在相同策略下超越了两个商业6+1自由度手臂，而OpenEAI-VLA的成功率与大规模预训练的pi0基线相当，且仅使用了有限的预训练数据。我们将发布完整的硬件设计、驱动程序、模型以及训练/数据管道，以支持可复现的研究和可扩展的数据收集。我们的代码、布局和模型将在论文接受后发布。

View on arXiv Download PDF AI Translation

cs.RO / 35 / 2606.03421

Reliability-Guided Depth Fusion for Glare-Resilient Navigation Costmaps

基于可靠性引导的深度融合方法用于抗眩光导航代价图

Tsai, Shang-En

Abstract

Specular glare on reflective floors, glass boundaries, and glossy indoor surfaces frequently corrupts active-stereo RGB-D depth measurements, producing holes and spikes that accumulate as persistent phantom obstacles in occupancy-grid costmaps. This paper presents a glare-resilient costmap construction method based on explicit depth-reliability modeling. A lightweight Depth Reliability Map network (DRM-Net) predicts per-pixel measurement trustworthiness under specular interference, and a reliability-guided weighted-and-gated fusion (RGF) mechanism modulates occupancy updates before corrupted measurements are accumulated into the map. To support robust training and evaluation, the method uses pose-aligned multi-view reference-depth construction to reduce circular-supervision bias and is evaluated through fusion-variant ablations, parameter-sensitivity analysis, cross-condition tests, paired navigation comparisons, reliability-map metrics, and embedded runtime profiling. Experiments on a real mobile robotic platform equipped with an Intel RealSense D435 and a Jetson Orin Nano show that the proposed method reduces false obstacle insertion, improves free-space preservation, and maintains real-time throughput under reflective-floor, glass-wall, and natural-light glare conditions. These results support treating glare as a measurement-reliability problem rather than as a dense depth-completion problem for safety-critical indoor navigation.

Chinese Translation

反射地面、玻璃边界和光滑室内表面的镜面眩光常常会干扰主动立体RGB-D深度测量，导致在占用网格代价图中产生孔洞和尖峰，这些问题会累积为持久的虚假障碍物。本文提出了一种基于显式深度可靠性建模的抗眩光代价图构建方法。轻量级深度可靠性图网络（Depth Reliability Map network，DRM-Net）在镜面干扰下预测每个像素测量的可信度，而可靠性引导的加权和门控融合（reliability-guided weighted-and-gated fusion，RGF）机制在将受损测量累积到图中之前调节占用更新。为了支持稳健的训练和评估，该方法采用姿态对齐的多视图参考深度构建，以减少循环监督偏差，并通过融合变体消融实验、参数敏感性分析、跨条件测试、配对导航比较、可靠性图度量和嵌入式运行时分析进行评估。在配备Intel RealSense D435和Jetson Orin Nano的真实移动机器人平台上的实验表明，所提出的方法减少了虚假障碍物的插入，提高了自由空间的保留，并在反射地面、玻璃墙和自然光眩光条件下保持实时吞吐量。这些结果支持将眩光视为测量可靠性问题，而非密集深度补全问题，以确保安全关键的室内导航。

View on arXiv Download PDF AI Translation

cs.RO / 36 / 2606.03441

PerchRL: Vision-Based Agile Perching on Inclined Platforms under Rapid and Irregular Motion

PerchRL：基于视觉的快速不规则运动下倾斜平台的灵活栖息

Lu, Zihong, Liu, Zongzhuo, Li, Huaxu, Cui, Jinqiang, Mei, Jie, Gong, Youmin, Cheang, U Kei, Zhou, Boyu

Abstract

Autonomous vision-based perching of quadrotors on moving inclined platforms is critical for air-ground collaboration but remains challenging due to the limited field of view (FOV). In this paper, we propose PerchRL, a reinforcement learning (RL) framework for vision-based agile perching on inclined platforms under rapid and irregular motion. Specifically, we employ a two-stage learning strategy consisting of state-based pre-training followed by vision-based fine-tuning. To improve generalization across diverse platform motions, we employ randomized platform trajectories to prevent overfitting and temporal augmentation methods to capture latent motion patterns from historical observations. During vision-based fine-tuning, a hybrid learning framework consisting of visibility-aware state augmentation and active perception rewards is presented to improve robustness under intermittent visual loss. Extensive simulation and real-world experiments demonstrate the feasibility, stability, and real-time performance of PerchRL, while successful deployment across distinct quadrotor platforms further validates its adaptability. The source code will be released to benefit the community.

Chinese Translation

自主的基于视觉的四旋翼在移动倾斜平台上的栖息对于空地协作至关重要，但由于视场（FOV）的限制，仍然面临挑战。本文提出了PerchRL，一个用于在快速不规则运动下进行基于视觉的灵活栖息的强化学习（RL）框架。具体而言，我们采用了一个两阶段学习策略，包括基于状态的预训练和基于视觉的微调。为了提高在不同平台运动下的泛化能力，我们采用随机化平台轨迹以防止过拟合，并使用时间增强方法从历史观测中捕捉潜在运动模式。在基于视觉的微调过程中，提出了一种混合学习框架，包括考虑可见性的状态增强和主动感知奖励，以提高在间歇性视觉丧失下的鲁棒性。广泛的仿真和现实世界实验表明，PerchRL的可行性、稳定性和实时性能，而在不同四旋翼平台上的成功部署进一步验证了其适应性。源代码将发布以惠及社区。

View on arXiv Download PDF AI Translation

cs.RO / 37 / 2606.03476

Human2Humanoid: Physics-Aware Cross-Morphology Motion Retargeting for Humanoid Robots

Human2Humanoid：基于物理的跨形态运动重定向用于类人机器人

Huang, Tianchen, Yuan, Feiyang, Gu, Junchi, Fang, Shurui, Zhang, Xiaohu, Wang, Yu, Gao, Wei, Zhang, Shiwu

Abstract

Retargeting human motion to humanoid robots is critical for teleoperation, imitation learning and human-robot interaction. However, it remains challenging because of substantial morphological discrepancies between humans and robots, including differences in skeletal topology, limb proportions and degrees of freedom, as well as the scarcity of paired motion data. This paper presents Human2Humanoid, an unsupervised motion retargeting framework that transfers human motions to humanoid robot behaviors with high fidelity. To bridge the domain gap under unpaired data, we adopt a CycleGAN-based architecture equipped with a skeleton-aware graph convolutional network to capture topology-dependent motion features. To address cross-domain scale mismatches, we introduce a morphology-invariant end-effector consistency loss that aligns normalized end-effector trajectories to preserve motion semantics across embodiments. To improve physical plausibility and reduce contact artifacts, we impose explicit physics-aware feasibility constraints to encourage reproduction of the contact patterns in the source motion. Experimental results show that the proposed method successfully retargets human motion to the Unitree G1 humanoid robot without paired data, and outperforms existing methods in both downstream controllability and physical feasibility.

Chinese Translation

将人类运动重定向到类人机器人对于遥操作、模仿学习和人机交互至关重要。然而，由于人类与机器人之间存在显著的形态差异，包括骨骼拓扑、肢体比例和自由度的差异，以及配对运动数据的稀缺，这一任务仍然具有挑战性。本文提出了Human2Humanoid，一个无监督的运动重定向框架，能够高保真地将人类运动转移到类人机器人行为中。为了在无配对数据的情况下弥合领域差距，我们采用了基于CycleGAN的架构，并配备了一个骨骼感知图卷积网络，以捕捉依赖拓扑的运动特征。为了解决跨领域的尺度不匹配问题，我们引入了一种形态不变的末端执行器一致性损失，旨在对齐归一化的末端执行器轨迹，以保持不同表现形式之间的运动语义。为了提高物理合理性并减少接触伪影，我们施加了显式的物理感知可行性约束，以鼓励再现源运动中的接触模式。实验结果表明，所提出的方法成功地将人类运动重定向到Unitree G1类人机器人，且在下游可控性和物理可行性方面优于现有方法。

View on arXiv Download PDF AI Translation

cs.RO / 38 / 2606.03512

SPADE: Sketch-guided Path Planning Augmented with Diffusion Experts

SPADE：基于草图引导的路径规划与扩散专家的增强

Hana, Charbel Abi, Ghantous, Tatiana, Khalil, Mikael, Rizk, Anthony

Abstract

Path planning is essential for Autonomous Mobile Robots (AMRs). Conventional methods for incorporating human preferences into planning typically rely on either complex reward engineering or hardware-intensive solutions. Recent state-of-the-art frameworks leverage imitation learning to train behavior-specific path planning models from expert demonstrations. However, these approaches face two key limitations: limited generalization to unseen environments and low robustness in demonstration collection. To address these challenges, this work introduces an enhanced framework that focuses on two main contributions: an overhauled annotation tool built on ROS 2, and a novel training strategy that integrates diffusion-based augmentation into baseline behavioral cloning models. A dataset of expert demonstrations is provided and evaluated through ablation studies to assess the robustness of the proposed solution. The enhanced approach outperforms state-of-the-art methods with 39.1% lower Absolute Pose Error (APE) and 33.5% lower Fr'echet Inception Distance (FID) while having 93.8% less trainable parameters. Moreover it attains diffusion-level generalization while preserving the real-time, on-edge properties of state-of-the-art models.

Chinese Translation

路径规划对自主移动机器人（AMRs）至关重要。传统方法在规划中融入人类偏好通常依赖于复杂的奖励工程或硬件密集型解决方案。最近的最先进框架利用模仿学习从专家演示中训练特定行为的路径规划模型。然而，这些方法面临两个主要限制：对未见环境的有限泛化能力和演示收集的低鲁棒性。为了解决这些挑战，本研究提出了一种增强框架，重点关注两个主要贡献：基于ROS 2的全面注释工具，以及一种将基于扩散的增强集成到基线行为克隆模型中的新训练策略。提供并通过消融研究评估了一组专家演示数据集，以评估所提解决方案的鲁棒性。增强的方法在绝对姿态误差（APE）上降低了39.1%，在Fr'echet Inception Distance（FID）上降低了33.5%，同时可训练参数减少了93.8%。此外，它在保持最先进模型的实时边缘特性同时，实现了扩散级别的泛化。

View on arXiv Download PDF AI Translation

cs.RO / 39 / 2606.03536

Bionic Human-Motion Style Transfer for Physically Executable Whole-Body Control of Humanoid Robots

仿生人类运动风格迁移用于可物理执行的类人机器人全身控制

Huang, Tianchen, Zhao, Mingkuan, Gao, Yang, Yuan, Feiyang, Gu, Junchi, Zhang, Xiaohu, Zhao, Dongdong, Yan, Shi, Wang, Yu, Gao, Wei, Zhang, Shiwu

Abstract

Expressive whole-body motion is important for humanoid robots operating in human environments, where robots are expected to move stably while presenting readable and adjustable body behaviors. However, most expressive motions are still obtained from fixed demonstrations or manually designed scripts, making it difficult to reuse a demonstrated style across different motion contents. Inspired by the way human motion styles convey affective and intentional cues through gait rhythm, posture, arm swing and body sway, this paper proposes a bionic generation-to-control framework for exemplar-driven style transfer on humanoid robots. Given a short human style exemplar and a target content motion, the proposed framework generates a stylized whole-body reference that preserves the intended motion content while transferring the demonstrated style. A physics-aware multi-condition latent diffusion model is developed to fuse style, content and trajectory conditions, and classifier-free guidance is used to adjust the style intensity without retraining. To improve hardware executability, contact-consistency and temporal-smoothness regularization are imposed on decoded motions during training. The generated references are then converted into G1-compatible robot references and executed by a preview-based whole-body tracking policy trained with a cluster-and-distill strategy. Simulation and Unitree G1 experiments show that the proposed method can transfer short human style exemplars to diverse robot motion contents, reduce contact and jitter artifacts compared with animation-oriented style-transfer baselines, and achieve a 96.0% success rate over 125 reported real-robot trials. The results demonstrate the feasibility of using short human motion exemplars as reusable bionic sources for physically executable expressive humanoid motion.

Chinese Translation

富有表现力的全身运动对于在人工环境中操作的类人机器人至关重要，这些机器人需要在保持稳定移动的同时，展现出可读且可调的身体行为。然而，大多数富有表现力的动作仍然依赖于固定的演示或手动设计的脚本，这使得在不同运动内容之间重用演示的风格变得困难。受到人类运动风格通过步态节奏、姿势、手臂摆动和身体摇摆传达情感和意图线索的启发，本文提出了一种仿生生成控制框架，用于在类人机器人上进行示例驱动的风格迁移。给定一个短的人的风格示例和一个目标内容运动，所提出的框架生成一个风格化的全身参考，既保留了预期的运动内容，又转移了演示的风格。开发了一种物理感知的多条件潜在扩散模型，以融合风格、内容和轨迹条件，并使用无分类器引导来调整风格强度而无需重新训练。为了提高硬件的可执行性，在训练期间对解码运动施加了接触一致性和时间平滑性正则化。生成的参考随后被转换为G1兼容的机器人参考，并通过一种基于预览的全身跟踪策略执行，该策略采用了聚类与蒸馏策略进行训练。仿真和Unitree G1实验表明，所提出的方法能够将短的人类风格示例迁移到多样的机器人运动内容上，相较于以动画为导向的风格迁移基线，减少接触和抖动伪影，并在125次真实机器人试验中实现了96.0%的成功率。结果表明，使用短的人类运动示例作为可重用的仿生源，以实现可物理执行的富有表现力的类人运动是可行的。

View on arXiv Download PDF AI Translation

cs.RO / 40 / 2606.03545

Static and Dynamic Representations for Tactile Contact-Angle Estimation with Event-Based Sensors

基于事件传感器的触觉接触角估计的静态与动态表征

Lu, Yanhui, Psomopoulou, Efi, Ward-Cherrier, Benjamin

Abstract

Event-based tactile sensing offers low-latency signal acquisition for contact-rich robotic interaction. This paper investigates contact-angle estimation using event streams from an event-based tactile sensor (NeuroTac) and compares three event-derived spatial contour representations: a dynamic representation capturing recent event activity, a static representation recovering a more persistent contact state, and their combined representation. Across the evaluated motion scenarios, all representation pipelines exhibited P99 processing latency below 10 ms at all tested sampling intervals, demonstrating their potential for high-frequency event-based tactile angle estimation in robotic manipulation. The static representation consistently achieved marginally better performance than the dynamic and combined representations under scenario-specific training, yielding a mean overall MAE of 0.160{\deg} during continuous sensor rolling and a stop-phase mean MAE of 0.251{\deg} during randomly inserted motion interruptions. It also exhibited smaller performance fluctuations across speed and indentation depth variations than the other two representations.

Chinese Translation

基于事件的触觉传感提供了低延迟的信号获取，适用于接触丰富的机器人交互。本文研究了使用基于事件的触觉传感器（NeuroTac）事件流进行接触角估计，并比较了三种基于事件的空间轮廓表征：捕捉近期事件活动的动态表征、恢复更持久接触状态的静态表征，以及它们的组合表征。在评估的运动场景中，所有表征管道在所有测试的采样间隔下均表现出 P99 处理延迟低于 10 毫秒，展示了其在机器人操作中进行高频基于事件的触觉角度估计的潜力。在特定场景训练下，静态表征的性能始终略优于动态和组合表征，在连续传感器滚动过程中，平均绝对误差（MAE）为 0.160{ extdegree}，而在随机插入运动中断的停止阶段，平均 MAE 为 0.251{ extdegree}。与其他两种表征相比，它在速度和压入深度变化下表现出更小的性能波动。

View on arXiv Download PDF AI Translation

cs.RO / 41 / 2606.03551

NVIDIA Isaac Sim: Enabling Scalable, GPU-Accelerated Simulation for Robotics

NVIDIA Isaac Sim：支持可扩展的GPU加速机器人仿真

Gao, Sicong, Pagnucco, Maurice, Bednarz, Tomasz, Song, Yang

Abstract

Simulation has become a core infrastructure for robotics research. Unlike previous simulators, NVIDIA Isaac Sim leverages GPU acceleration to enable large-scale parallel training and physics-accurate modeling. Its synthetic data generation pipeline alleviates the scarcity of high-quality training data, supporting data-driven robot learning and large-scale simulation-centric experimentation. However, existing surveys often treat it as one simulator among many, without a systematic analysis of its architectural characteristics, usage patterns, and limitations. This survey reviews Isaac Sim from system and application perspectives, outlining its architecture and comparing it with widely used simulators. We analyze representative studies across five major domains and summarize common usage patterns, particularly in data generation and high-fidelity simulation. We also outline key future directions and challenges, including physics open-world learning, simulation-centric training and practical usability constraints.

Chinese Translation

仿真已成为机器人研究的核心基础设施。与以往的仿真器不同，NVIDIA Isaac Sim 利用 GPU 加速实现大规模并行训练和物理精确建模。其合成数据生成管道缓解了高质量训练数据的稀缺，支持数据驱动的机器人学习和大规模以仿真为中心的实验。然而，现有的调查往往将其视为众多仿真器中的一个，而没有对其架构特征、使用模式和局限性进行系统分析。本调查从系统和应用的角度回顾了 Isaac Sim，概述了其架构，并与广泛使用的仿真器进行了比较。我们分析了五个主要领域的代表性研究，并总结了常见的使用模式，特别是在数据生成和高保真仿真方面。我们还概述了未来的关键方向和挑战，包括物理开放世界学习、以仿真为中心的训练和实际可用性限制。

View on arXiv Download PDF AI Translation

cs.RO / 42 / 2606.03556

Partially Observable Adversarial Patch Attacks on Vision-Language-Action Models in Robotics

对机器人视觉-语言-动作模型的部分可观察对抗性补丁攻击

Wang, Xiaofei, Han, Mingliang, Hao, Tianyu, Yang, Yi, Zhao, Yun-Bo, Tang, Keke

Abstract

Vision-language-action (VLA) models are gaining attention in robotics, yet their robustness to adversarial attacks remains largely unexplored. Existing work shows that adversarial patches can mislead VLA-based robots but assumes full access to the entire execution trajectory, an unrealistic requirement in practice. We address this limitation by formulating a partially observable threat model, where the adversary can exploit only a short prefix of the trajectory to generate a fixed patch applied to all subsequent frames. Under this setting, we propose a two-phase framework. First, we localize the patch using the model's attention maps to identify visually critical regions that correspond to the full instruction. Then, we optimize the patch to disrupt the semantic grounding of target objects and increase the curvature of action trajectories, thereby compounding failures in both perception and control. Extensive experiments in simulation and real-world robotic environments show that our method sustains adversarial effects under partial observability, inducing long-horizon disruptions and significantly reducing task success rates.

Chinese Translation

视觉-语言-动作（VLA）模型在机器人领域受到越来越多的关注，但其对对抗性攻击的鲁棒性仍然未得到充分探索。现有研究表明，对抗性补丁可以误导基于VLA的机器人，但假设对整个执行轨迹的完全访问，这在实践中是不切实际的。我们通过提出一个部分可观察的威胁模型来解决这一限制，在该模型中，攻击者只能利用轨迹的短前缀来生成一个固定的补丁，该补丁应用于所有后续帧。在这种设置下，我们提出了一个两阶段框架。首先，我们利用模型的注意力图来定位补丁，以识别与完整指令对应的视觉关键区域。然后，我们优化补丁以干扰目标物体的语义基础，并增加动作轨迹的曲率，从而在感知和控制方面造成复合性失败。在仿真和真实机器人环境中的大量实验表明，我们的方法在部分可观察性下保持对抗性效果，导致长时间的干扰，并显著降低任务成功率。

View on arXiv Download PDF AI Translation

cs.RO / 43 / 2606.03590

CANMOT: Class-Aware Noise Modeling for Multi-Object Tracking in Autonomous Driving

CANMOT：用于自动驾驶中的多目标跟踪的类别感知噪声建模

Osterburg, Timo, Schütte, Stefan, Bertram, Torsten

Abstract

Kalman filter (KF)-based multi-object tracking (MOT) remains a strong baseline for autonomous driving due to its strong performance, computational efficiency and interpretability. In most practical systems, the process noise and measurement noise covariances are defined globally and shared across object classes, presuming identical uncertainty characteristics across heterogeneous traffic participants. This work revisits this assumption and proposes CANMOT, a class-aware and object-aligned noise modeling framework for KF-based 3D MOT. Class-specific diagonal process and measurement covariance matrices are introduced and optionally expressed in the object coordinate frame to preserve longitudinal-lateral anisotropy. Systematic experiments on the nuScenes benchmark show that class-aware and object-aligned noise modeling improves tracking performance and substantially reduces identity switches compared to state-of-the-art (SotA). In addition, the consistency of the estimated uncertainty is analyzed using the Average Normalized Estimation Error Squared (ANEES) and $\chi^2$-based violation tests. The results reveal severe overconfidence in standard KF-based MOT baselines. While the proposed formulation improves calibration without modifying the underlying filtering framework, it still exhibits substantial inconsistency, highlighting the need for further research in this area. Code is available at https://github.com/rst-tu-dortmund/learned-3d-nms.

Chinese Translation

基于卡尔曼滤波器（Kalman filter, KF）的多目标跟踪（Multi-Object Tracking, MOT）因其卓越的性能、计算效率和可解释性，仍然是自动驾驶领域的强基准。在大多数实际系统中，过程噪声和测量噪声的协方差是全局定义并在对象类别之间共享的，假设异构交通参与者具有相同的不确定性特征。本研究重新审视了这一假设，并提出了CANMOT，一个基于KF的3D MOT的类别感知和对象对齐的噪声建模框架。引入了类别特定的对角过程和测量协方差矩阵，并可选择在对象坐标系中表示，以保持纵向-横向的各向异性。在nuScenes基准上的系统实验表明，与最先进的方法（State-of-the-Art, SotA）相比，类别感知和对象对齐的噪声建模提高了跟踪性能，并显著减少了身份切换。此外，使用平均归一化估计误差平方（Average Normalized Estimation Error Squared, ANEES）和基于$ ext{χ}^2$的违例测试分析了估计不确定性的连贯性。结果显示，标准KF基准在不确定性估计上存在严重的过度自信。尽管所提出的公式在不修改基础滤波框架的情况下改善了校准，但仍然表现出显著的不一致性，突显了该领域进一步研究的必要性。代码可在https://github.com/rst-tu-dortmund/learned-3d-nms获取。

View on arXiv Download PDF AI Translation

cs.RO / 44 / 2606.03598

PHASER: Phase-Aware and Semantic Experience Replay for Vision-Language-Action Models

PHASER：面向阶段感知和语义经验重放的视觉-语言-动作模型

Chen, Ziyang, Wang, Shaoguang, Guo, Weiyu, Cai, Qianyi, Zhang, He, Li, Pengteng, Zhao, Yiren, Guo, Yandong

Abstract

Vision-Language-Action (VLA) models have achieved remarkable success in language-conditioned robotic manipulation. However, deploying these models in open-ended environments requires continuously acquiring novel skills, a process that inevitably triggers severe catastrophic forgetting of previously learned behaviors. While experience replay (ER) serves as a standard mitigating strategy, naive uniform sampling fundamentally misaligns with the temporal characteristics of manipulation trajectories. It systematically under-samples brief but causally critical sub-skills, leading to phase starvation, and completely overlooks the varying degrees of forgetting across historical tasks. To overcome these limitations, we introduce PHASER, an architecture-agnostic continual learning framework. PHASER employs a phase-centric capacity allocation to guarantee equal memory support for all sub-skills, coupled with a multi-modal interference routing strategy that dynamically prioritizes historical phases at high risk of forgetting. Furthermore, to enable fully autonomous lifelong adaptation, we integrate Auto-PC, a lightweight pipeline combining unsupervised action-signal change-point detection with VLM-based semantic verification to extract temporal boundaries without intensive manual supervision. Evaluated across three VLA backbones on LIBERO continual learning suites, PHASER yields substantial empirical improvements, increasing Average Success Rate (ASR) by up to 31% over matched-budget ER and achieving an 87.8% final ASR on the LIBERO-Goal CL setting.

Chinese Translation

视觉-语言-动作（VLA）模型在语言条件下的机器人操作中取得了显著成功。然而，在开放式环境中部署这些模型需要持续获取新技能，这一过程不可避免地会导致对先前学习行为的严重灾难性遗忘。虽然经验重放（ER）作为一种标准的缓解策略，但简单的均匀采样与操作轨迹的时间特性根本不匹配。它系统性地对简短但因果关键的子技能进行欠采样，导致阶段饥饿，并完全忽视了历史任务中遗忘程度的变化。为了解决这些局限性，我们提出了PHASER，一个与架构无关的持续学习框架。PHASER采用以阶段为中心的容量分配，以确保对所有子技能提供平等的记忆支持，并结合一种多模态干扰路由策略，动态优先考虑遗忘风险较高的历史阶段。此外，为了实现完全自主的终身适应，我们集成了Auto-PC，一个轻量级管道，将无监督的动作信号变更点检测与基于VLM的语义验证相结合，以提取时间边界，而无需大量手动监督。在LIBERO持续学习套件上对三种VLA骨干网络进行评估，PHASER显著提高了实证结果，平均成功率（ASR）提高了最多31%，并在LIBERO-Goal CL设置中达到了87.8%的最终ASR。

View on arXiv Download PDF AI Translation

cs.RO / 45 / 2606.03609

A 3D Isovist World Model -- Revealing a City's Unseen Geometry and Its Emergent Cross-City Signature

三维可视域世界模型——揭示城市的隐形几何及其跨城市特征

Lin, Xuhui, Law, Stephen, Chen, Nanjiang, Li, Kunyao, Yang, Tao

Abstract

Embodied agents that navigate cities rely on world models that predict how their surroundings will change as they move. But for navigation, what matters is not what the buildings look like; it is where the agent can go. Most world models nonetheless predict appearance, learning how a scene looks rather than the space an agent can move through. Those that do target geometry, such as bird's-eye-view occupancy grids, flatten the three-dimensional environment onto a ground plane, discarding the above-ground and multi-level structure that shapes real navigation. What is missing is a predictive target that captures the navigable geometry an agent actually traverses, without photometric entanglement and without collapsing the third dimension. Our key idea is to model the open volume between buildings, the negative space, encoded as a 3D isovist: a spherical visibility-depth map recording the distance to the nearest surface in every direction. We introduce an embodied world model that predicts the next isovist from a short history of past isovists and a movement action. The prediction is formulated as a depth residual so the decoder inherits sharp building edges, trained with self-rollout scheduled sampling to keep corrupted context on the geometry manifold, and equipped with a persistent latent bird's-eye-view spatial map for cross-path consistency. Our central finding is emergent and unexpected: a single city-blind model trained on Manhattan and Paris develops a cross-city spatial signature, with city identity linearly decodable from its temporal latents far above single-frame baselines, so the signature lives in the learned dynamics rather than in appearance. The representation is lightweight, interpretable, and reproducible, offering a geometric substrate for spatial reasoning in embodied AI, robotics, and urban analysis, released with an open dataset and pipeline.

Chinese Translation

在城市中导航的具身智能体依赖于世界模型，以预测其周围环境在移动过程中的变化。然而，对于导航而言，重要的并不是建筑物的外观，而是智能体可以去往的地方。尽管大多数世界模型预测的是外观，学习的是场景的视觉效果，而非智能体可以穿越的空间。那些专注于几何的模型，例如鸟瞰图占用网格，将三维环境压缩到地面平面，忽略了影响真实导航的地上和多层结构。缺失的是一个预测目标，它能够捕捉智能体实际穿越的可导航几何，而不受光度纠缠的影响，也不压缩第三维度。我们的关键思想是对建筑物之间的开放空间，即负空间进行建模，编码为三维可视域（3D isovist）：一个记录每个方向上到最近表面距离的球形可视深度图。我们引入了一个具身世界模型，该模型根据过去一段时间的可视域历史和移动动作预测下一个可视域。该预测被表述为深度残差，因此解码器继承了清晰的建筑边缘，通过自回归调度采样进行训练，以保持几何流形上的损坏上下文，并配备一个持久的潜在鸟瞰空间图，以确保跨路径一致性。我们的核心发现是突现且意外的：一个在曼哈顿和巴黎训练的城市盲模型发展出一种跨城市空间特征，其城市身份可以从其时间潜变量中线性解码，远高于单帧基线，因此该特征存在于学习的动态中，而非外观。该表示轻量、可解释且可重复，为具身人工智能、机器人技术和城市分析中的空间推理提供了几何基础，并随开放数据集和管道发布。

View on arXiv Download PDF AI Translation

cs.RO / 46 / 2606.03682

GN0: Toward a Unified Paradigm for Generation, Evaluation, and Policy Learning in Visual-Language Navigation

GN0：朝着视觉-语言导航中的生成、评估和策略学习的统一范式迈进

Li, Xinhai, Zhang, Xiaotao, Huang, Yuehao, Dong, Jiankun, Wang, Tianhang, Zhou, Sunyao, Wu, Yunzi, Sun, Chengnuo, Ge, Yunfei, Weng, Qizhen, Zhang, Chi, Bai, Chenjia, Li, Xuelong

Abstract

Embodied navigation connects intelligent agents with the physical world and is fundamental for general robotic intelligence. Limited availability and quality of navigation data have constrained Vision-and-Language Navigation (VLN) systems' generalization and long-horizon capabilities. To address this, we curate diverse 3D scenes and develop an automated pipeline for large-scale navigation data, resulting in the GN-Matrix dataset. Building on a 3D Gaussian Splatting (3DGS) engine, we introduce a high-fidelity simulation platform supporting interactive roaming and collision-aware navigation. We further propose GN-Bench, the first BEV-based benchmark incorporating dynamic 3DGS avatars for human-robot interaction evaluation. To leverage the simulator, we develop an RL-driven navigation foundation model, Break and Establish (BAE). After supervised learning, DAgger exposes the model to rollout-induced states, breaking narrow expert-centric distributions and enabling downstream RL exploration. This unified VLN paradigm integrates map-based and map-free tasks, including instruction following, human following, and goal navigation. GN-BAE formalizes high-fidelity 3DGS-rendered Bird's Eye View representations as compact memory, unlocking latent spatial reasoning in VLMs. Extensive evaluations on GN-Bench and VLN-CE show that GN0 outperforms state-of-the-art VLN methods. Overall, GN-Matrix offers a unified framework spanning data, simulation, and learning, advancing embodied navigation in research and industrial applications.

Chinese Translation

具身导航将智能体与物理世界连接起来，是通用机器人智能的基础。导航数据的有限可用性和质量限制了视觉-语言导航（VLN）系统的泛化能力和长时间跨度的能力。为了解决这个问题，我们策划了多样化的3D场景，并开发了一个大规模导航数据的自动化管道，最终形成了GN-Matrix数据集。在3D高斯点云引擎（3D Gaussian Splatting, 3DGS）的基础上，我们引入了一个高保真度的仿真平台，支持交互式漫游和碰撞感知导航。我们进一步提出了GN-Bench，这是第一个基于鸟瞰视角（BEV）的基准，结合了动态3DGS化身用于人机交互评估。为了利用该仿真器，我们开发了一个基于强化学习的导航基础模型，称为Break and Establish（BAE）。经过监督学习后，DAgger将模型暴露于由回滚引起的状态，打破狭隘的专家中心分布，启用下游的强化学习探索。这个统一的VLN范式整合了基于地图和无地图的任务，包括指令跟随、人类跟随和目标导航。GN-BAE将高保真度的3DGS渲染鸟瞰视图表示形式化为紧凑的记忆，解锁了视觉语言模型（VLMs）中的潜在空间推理。在GN-Bench和VLN-CE上的广泛评估表明，GN0的表现优于最先进的VLN方法。总体而言，GN-Matrix提供了一个跨越数据、仿真和学习的统一框架，推动了研究和工业应用中的具身导航发展。

View on arXiv Download PDF AI Translation

cs.RO / 47 / 2606.03694

Face versus Body Tracking for Human-Robot Interaction: An Egocentric Dataset

人机交互中的面部与身体追踪：一个自我中心的数据集

Wenninger, Jessica, Skantze, Gabriel

Abstract

To enable meaningful human-robot interaction (HRI), a robot must continuously assess engagement by consistently tracking users over time. State-of-the-art computer vision models, however, are heavily optimized for surveillance or autonomous driving. A social robot faces distinct egocentric challenges, such as humans bouncing, obstructing each other, or leaving the frame. Frequent identity switches (IDSW) cause the robot to lose its footing mid-conversation. To address this, we introduce a novel, custom-annotated egocentric dataset collected via the Furhat robot to capture complex social dynamics. We present a systematic evaluation isolating detection errors from tracking logic, comparing face versus body tracking, and assessing the impact of extended spatial memory and appearance re-identification (ReID). Results indicate that increasing spatial memory mitigates prolonged occlusions but fails on complex dynamic events. Integrating ReID resolves complex switches but exhibits opposing effects: it substantially improves body tracking stability, yet causes facial IDSW to spike due to profile angle sensitivity. Ultimately, our optimized pipeline reduces IDSW by 49\%, mitigating interaction breakdowns. Because standard benchmarks lack dense, close-quarter occlusions, this work highlights the critical need for natively captured social dynamics to truly validate HRI perception models.

Chinese Translation

为了实现有意义的人机交互（HRI），机器人必须持续评估用户的参与度，并在时间上持续跟踪用户。然而，最先进的计算机视觉模型主要针对监控或自动驾驶进行了高度优化。社交机器人面临着独特的自我中心挑战，例如人类之间的碰撞、相互遮挡或离开画面。频繁的身份切换（IDSW）会导致机器人在对话中失去定位。为了解决这一问题，我们引入了一个新颖的、通过Furhat机器人收集的自我中心数据集，以捕捉复杂的社交动态。我们进行了系统评估，隔离了检测错误与追踪逻辑，比较了面部与身体追踪，并评估了扩展空间记忆和外观重识别（ReID）的影响。结果表明，增加空间记忆可以减轻长时间遮挡的影响，但在复杂动态事件中表现不佳。集成ReID解决了复杂的身份切换问题，但表现出相反的效果：它显著提高了身体追踪的稳定性，但由于侧面角度敏感性导致面部IDSW激增。最终，我们优化的流程将IDSW减少了49%，减轻了交互中断的情况。由于标准基准缺乏密集的近距离遮挡，这项工作突出了原生捕捉社交动态以真正验证HRI感知模型的关键需求。

View on arXiv Download PDF AI Translation

cs.RO / 48 / 2606.03756

Neural Navigation Functions for Zero-Shot Generalizable Motion Planning

用于零-shot 可泛化运动规划的神经导航函数

Shaffer, Benjamin D., Hsieh, Pei-An, Kinch, Brooks, Trask, Nathaniel, Hsieh, M. Ani

Abstract

We introduce Neural Navigation Functions (Neural-NF), a learned reactive navigation function capable of zero-shot transfer across unseen environment geometries. Neural-NF places data-driven adaptation within a structured elliptic planner, where the navigation objective is learned while planner structure is preserved by construction. Specifically, intrinsic Laplacian-derived features are mapped to local PDE coefficients, and solving the resulting boundary value problem produces a globally consistent value function on each target domain. For every admissible learned model, the resulting policy is collision-free, provides monotonic descent and a global minimum at the goal by construction. This admits a linearly-solvable optimal-control interpretation for any parameter setting. Empirically, Neural-NF achieves strong zero-shot transfer across diverse geometries and outperforms learned planners that directly predict the value function by up to a $5\times$ improvement.

Chinese Translation

我们介绍了神经导航函数（Neural Navigation Functions, Neural-NF），这是一种能够在未见环境几何中实现零-shot 转移的学习型反应导航函数。Neural-NF 将数据驱动的适应性置于一个结构化的椭圆规划器中，其中导航目标在保持规划器结构的前提下进行学习。具体而言，内在的拉普拉斯导出特征被映射到局部偏微分方程（PDE）系数上，解决得到的边值问题在每个目标领域上产生一个全局一致的价值函数。对于每个可接受的学习模型，所得到的策略在构建上是无碰撞的，提供单调下降并在目标处达到全局最小值。这为任何参数设置提供了线性可解的最优控制解释。从实证上看，Neural-NF 在多样几何中实现了强大的零-shot 转移，并且在性能上超越了直接预测价值函数的学习规划器，改进幅度高达 $5 imes$。

View on arXiv Download PDF AI Translation

cs.RO / 49 / 2606.03784

Revisiting Embodied Chain-of-Thought for Generalizable Robot Manipulation

重新审视可体现的思维链以实现可推广的机器人操作

Sun, Nan, Zhang, Yuan, Yang, Yongkun, Zhao, Wentao, Li, Peiyan, Guo, Jun, Song, Wenxuan, Ding, Pengxiang, Suo, Runze, Su, Yifei, Xiao, Xin, Li, Xinghang, Liu, Huaping

Abstract

Embodied chain-of-thought (CoT) aims to bridge linguistic reasoning and robotic control, but its effective form and integration strategy remain underexplored. In this paper, we revisit embodied CoT for vision-language-action (VLA) models at large scale. We construct the largest embodied CoT corpus to date, comprising 978,743 trajectories, 226.3M samples, and 2592.5 hours of robot data. Through extensive experiments, we find that effective embodied CoT should ground high-level semantic understanding into concrete action guidance, such as end-effector movement descriptions and image-space trajectories, while high-level reasoning alone brings only marginal gains. We further show that explicit CoT does not scale reliably when used as an autoregressive action prefix, as it suffers from compounding inference errors and unstable reasoning-action coupling. To address these limitations, we propose ERVLA, a VLA model that uses embodied CoT as representation-shaping supervision rather than mandatory test-time reasoning. ERVLA is trained with a reasoning-dropout strategy, enabling the model to absorb rich reasoning traces during training while predicting actions directly without CoT decoding during inference. This design improves scalability with increasing pre-training data and avoids autoregressive instability. ERVLA achieves state-of-the-art performance on LIBERO-Plus with an 86.9% success rate and reaches 53.2% success rate on VLABench, demonstrating strong out-of-distribution generalization. In real-robot experiments, ERVLA further outperforms competitive state-of-the-art baselines, especially on tasks requiring semantic disambiguation and long-horizon execution. Code, data, and model checkpoints will be released.

Chinese Translation

可体现的思维链（CoT）旨在桥接语言推理与机器人控制，但其有效形式和集成策略仍未得到充分探索。本文重新审视了大规模视觉-语言-动作（VLA）模型中的可体现CoT。我们构建了迄今为止最大的可体现CoT语料库，包含978,743条轨迹、2.263亿个样本和2592.5小时的机器人数据。通过广泛的实验，我们发现有效的可体现CoT应将高层次的语义理解与具体的行动指导相结合，例如末端执行器运动描述和图像空间轨迹，而仅依靠高层次推理则仅带来边际收益。我们进一步表明，当作为自回归动作前缀使用时，显式CoT并不能可靠扩展，因为它会遭受推理错误的累积和不稳定的推理-动作耦合。为了解决这些局限性，我们提出了ERVLA，一种将可体现CoT用作表征塑造监督而非强制测试时推理的VLA模型。ERVLA采用推理丢弃策略进行训练，使模型在训练过程中能够吸收丰富的推理痕迹，同时在推理时直接预测动作而无需CoT解码。这一设计在增加预训练数据时提高了可扩展性，并避免了自回归的不稳定性。ERVLA在LIBERO-Plus上实现了86.9%的成功率，达到了53.2%的VLABench成功率，展示了强大的分布外泛化能力。在真实机器人实验中，ERVLA进一步超越了竞争性的最先进基线，尤其是在需要语义消歧和长时间执行的任务上。代码、数据和模型检查点将会发布。

View on arXiv Download PDF AI Translation

cs.RO / 50 / 2606.03787

Worth Remembering: Surprise-Gated Robot Episodic Memory

值得记住：惊讶门控的机器人情节记忆

Gorlo, Nicolas, Wise, Derek K., Speranzon, Alberto, Carlone, Luca

Abstract

Robots solving generalist tasks need to be able to ground instructions in their past experience, since humans may refer to notable past events when giving a task (e.g., ``Take me to where the chemical spill happened yesterday''). Since memory limits make storing all past events infeasible, long-term robot memory must be selective, ideally retaining only those episodes with high utility for future tasks. However, future tasks are not typically given a priori for generalist robots. To select generically useful memories, we propose Bayesian surprise as a gating mechanism for memory formation. We present an approach to compute surprise in a semantically rich deployment-agnostic latent space provided by V-JEPA-2. Using our gated episodic memory to augment 4D scene graph-based spatial memory, we show a consistent improvement over state-of-the-art benchmarks in robot question answering, outperforming prior robot memory methods by $\geq12\%$ for temporal, spatial, and binary questions, and surpassing the performance of supervised and non-causal methods with an unsupervised causal method in event segmentation tasks.

Chinese Translation

解决通用任务的机器人需要能够将指令与其过去的经验相结合，因为人类在给出任务时可能会提到显著的过去事件（例如，“带我去昨天发生化学泄漏的地方”）。由于记忆限制使得存储所有过去事件不可行，长期的机器人记忆必须具有选择性，理想情况下仅保留对未来任务具有高效用的情节。然而，未来的任务通常不会事先提供给通用机器人。为了选择通用有用的记忆，我们提出使用贝叶斯惊讶作为记忆形成的门控机制。我们提出了一种方法，在由 V-JEPA-2 提供的语义丰富的与部署无关的潜在空间中计算惊讶。通过使用我们的门控情节记忆来增强基于 4D 场景图的空间记忆，我们在机器人问答方面显示出相对于最新基准的一致性提升，在时间、空间和二元问题上超越了先前的机器人记忆方法，提升幅度达到 $ ext{≥}12 ext{ extperthousand}$，并在事件分割任务中以无监督因果方法超越了监督和非因果方法的表现。

View on arXiv Download PDF AI Translation

cs.RO / 51 / 2606.03798

Optimal Design and Analytical Modeling of a Soft Fin-Ray Effect Gripper Finger Using the Finite Rigid Elements Method

基于有限刚体元素法的软鳍射线效应抓手指的最佳设计与分析建模

Adeli, Sara, Sayyaadi, Hassan

Abstract

Fin Ray-inspired soft grippers offer a promising solution for gently handling delicate, irregular objects, especially in agriculture. The objective of this research is to design, fabricate, and model a Fin Ray Effect (FRE) soft gripper finger to enable precise force control in future applications. This design aims to gently grasp delicate agricultural products, such as tomatoes, that require both adaptability and accurate force application. To address the inherent challenges of soft robotics, including nonlinear behavior, infinite degrees of freedom, and variable material properties, the Finite Rigid Elements Method (FREM) was employed for modeling. This method preserves analytical accuracy while providing a reliable foundation for the development of a force controller in later stages. A detailed Finite Element Model (FEM) was created using ANSYS, and the analytical results were validated through simulation and experimental testing. The gripper's fingers were optimized based on four key criteria: tip displacement, total deflection, stress distribution, and contact force. The optimal finger configuration includes a length of 30 mm, rib spacing of 10 mm, seven ribs angled at -15 deg, and a rib thickness of 1 mm. Theoretical modeling using the FREM predicted finger deformation with a 3% error, while the ANSYS numerical model achieved 2% error.

Chinese Translation

受鳍射线启发的软抓手为温和处理脆弱、不规则物体提供了一个有前景的解决方案，特别是在农业领域。本研究的目标是设计、制造和建模一个鳍射线效应（Fin Ray Effect, FRE）软抓手指，以实现未来应用中的精确力控制。该设计旨在温和抓取需要适应性和准确施力的脆弱农业产品，如番茄。为了解决软机器人固有的挑战，包括非线性行为、无限自由度和可变材料特性，采用了有限刚体元素法（Finite Rigid Elements Method, FREM）进行建模。该方法在保持分析准确性的同时，为后续阶段力控制器的开发提供了可靠基础。使用ANSYS创建了详细的有限元模型（Finite Element Model, FEM），并通过仿真和实验测试验证了分析结果。抓手的指头基于四个关键标准进行了优化：尖端位移、总偏转、应力分布和接触力。最佳指头配置包括长度30毫米、肋间距10毫米、七个倾斜角为-15度的肋和肋厚度1毫米。使用FREM的理论建模预测指头变形的误差为3%，而ANSYS数值模型的误差为2%。

View on arXiv Download PDF AI Translation

cs.RO / 52 / 2606.03834

Let the Dynamics Flow: Stable Flow Matching Dynamical Systems

让动态流动：稳定流匹配动力系统

Pérez-Dattari, Rodrigo, Leiva, Francisco, Testa, Andrea, Rozo, Leonel, del Solar, Javier Ruiz, Jaquier, Noémie

Abstract

Flow matching has recently emerged as a powerful approach for imitation learning, enabling scalable, expressive, and multimodal motion policies. However, incorporating formal stability guarantees into these generative models, a prerequisite to ensure safe and generalizable robot behaviors, remains a significant challenge. While modeling robot motions as dynamical systems allows for such stability-based inductive biases, existing frameworks struggle to capture the rich action distributions inherent in complex robotic tasks. This paper introduces Stable Flow Matching Dynamical Systems (SFMDS), a novel framework that bridges the gap between high-capacity generative modeling and formal Lyapunov stability guarantees. SFMDS parametrizes dynamical systems via flow matching while simultaneously constraining the model to a family of stable solutions. We propose two variants: a soft constraint based on a penalty term, and a hard structural constraint embedded directly in the model architecture. We further extend both formulations to Lie groups. Experiments on benchmark datasets, in simulation, and on a humanoid robot show that SFMDS learns stable, scalable, and multimodal dynamical systems in low- and high-dimensional state spaces, enabling safe and expressive robot motion generation.

Chinese Translation

流匹配最近作为一种强大的模仿学习方法而出现，能够实现可扩展、富有表现力和多模态的运动策略。然而，将正式的稳定性保证纳入这些生成模型中，以确保安全和可推广的机器人行为，仍然是一个重大挑战。尽管将机器人运动建模为动力系统可以实现基于稳定性的归纳偏置，但现有框架在捕捉复杂机器人任务中固有的丰富动作分布方面存在困难。本文介绍了稳定流匹配动力系统（Stable Flow Matching Dynamical Systems, SFMDS），这是一个新颖的框架，弥合了高容量生成建模与正式的Lyapunov稳定性保证之间的差距。SFMDS通过流匹配对动力系统进行参数化，同时将模型约束在一组稳定解的家族中。我们提出了两种变体：基于惩罚项的软约束和直接嵌入模型架构中的硬结构约束。我们进一步将这两种形式扩展到李群。基准数据集、仿真和人形机器人上的实验表明，SFMDS能够在低维和高维状态空间中学习稳定、可扩展和多模态的动力系统，从而实现安全且富有表现力的机器人运动生成。

View on arXiv Download PDF AI Translation

cs.RO / 53 / 2606.03847

Denoising Tells When to Replan: Denoising-Variance Adaptive Chunking for Flow-Based Robot Policies

去噪指示何时重新规划：基于流的机器人策略的去噪方差自适应分块

Feng, Xiangdong, Cheng, Yuxuan, Shi, Chen, Han, Boyao, Yan, Yuxuan, Hong, Yitong, Tian, Zhuotao, Jiang, Li

Abstract

Action chunking has become a common inference strategy for flow-based robot policies, improving action coherence by modeling multi-step temporal dependencies in demonstrations. However, the execution horizon is still typically set as an empirical fixed value, overlooking that predictable free-space motions and precision-critical interaction phases often require different replanning frequencies. In this work, we first show that the denoising process of flow-based policies contains an intrinsic signal of task phases: clean-action estimates remain stable during predictable motion phases, but fluctuate more strongly around contact-rich or precision-sensitive operations. Motivated by this observation, we propose DVAC (Denoising-Variance Adaptive Chunking), a test-time method that adaptively determines how many actions to execute from each predicted chunk. DVAC measures the variance of clean-action estimates over the final denoising steps, executes the stable low-variance prefix, and replans before high-variance future actions are committed. To transfer across tasks and rollouts, DVAC further calibrates the threshold with a rolling estimate of the local variance scale. Experiments on LIBERO, RoboTwin, CALVIN, and real-world manipulation show that DVAC improves task success while reducing replanning frequency. With a $\pi_{0.5}$-based policy, DVAC improves LIBERO success from 94.75% to 98.00% and reduces replanning by 43.0%, while also yielding aggregate gains on RoboTwin and CALVIN and improving real-world execution efficiency.

Chinese Translation

动作分块已成为基于流的机器人策略的一种常见推理策略，通过建模演示中的多步时间依赖性来提高动作的一致性。然而，执行时间范围通常仍然设定为经验固定值，忽视了可预测的自由空间运动和对精度要求严格的交互阶段往往需要不同的重新规划频率。在本研究中，我们首先展示了基于流的策略的去噪过程包含任务阶段的内在信号：在可预测的运动阶段，干净的动作估计保持稳定，但在接触丰富或对精度敏感的操作周围波动更强。基于这一观察，我们提出了DVAC（去噪方差自适应分块），这是一种在测试时自适应确定从每个预测分块中执行多少动作的方法。DVAC测量最终去噪步骤中干净动作估计的方差，执行稳定的低方差前缀，并在高方差未来动作被承诺之前进行重新规划。为了在任务和回放之间进行迁移，DVAC进一步通过对局部方差尺度的滚动估计来校准阈值。在LIBERO、RoboTwin、CALVIN和现实世界操作的实验中，DVAC提高了任务成功率，同时减少了重新规划频率。在基于$ ext{π}_{0.5}$的策略下，DVAC将LIBERO的成功率从94.75%提高到98.00%，并将重新规划减少了43.0%，同时在RoboTwin和CALVIN上也取得了综合收益，并提高了现实世界的执行效率。

View on arXiv Download PDF AI Translation

cs.RO / 54 / 2606.03905

Semantic-weighted ICP for LiDAR Odometry: Class-Aware Residual Reweighting for Robust Scan Registration

用于激光雷达里程计的语义加权ICP：面向类别的残差重加权以实现鲁棒的扫描配准

Carvalho, Vasco, Barros, Tiago, Nunes, Urbano J.

Abstract

LiDAR odometry is a fundamental component of autonomous robotic systems, relying on geometric registration between consecutive point clouds to estimate ego-motion. However, traditional geometric approaches often degrade in dynamic or unstructured environments due to unreliable correspondences caused by moving objects, sparse geometric features, vegetation, and semantically ambiguous structures. Existing works have shown that, some of these limitations can be addressed by introducing semantic information from the environment in the registration process. In this work, we build on this, and show that not all elements in the environment are equally relevant for registration. Hence, we propose a semantic class-weighted ICP for LiDAR odometry. Instead of strictly filtering out points belonging to specific semantic classes, the proposed approach weights the residuals of points belonging to semantic categories based on their expected geometric stability. This strategy enables informative but potentially unstable structures, to contribute to the registration process while mitigating the influence of dynamic objects. The experimental evaluation was conducted on the SemanticKITTI and RELLIS-3D datasets, which include urban, highway, rural, and off-road environments. The empirical results show that the proposed Semantic-weighted ICP improves pose estimation, especially in challenging off-road scenarios where conventional rigid features are scarce. Furthermore, the analysis reveals that the effectiveness of this weighting strategy is highly environment-dependent, influenced by the structural and semantic composition of the scene.

Chinese Translation

激光雷达里程计是自主机器人系统的一个基本组成部分，依赖于连续点云之间的几何配准来估计自我运动。然而，传统的几何方法在动态或非结构化环境中往往会因移动物体、稀疏几何特征、植被和语义模糊结构导致的不可靠对应关系而退化。现有研究表明，通过在配准过程中引入环境的语义信息，可以解决这些限制中的一些。在本研究中，我们在此基础上展示了环境中的并非所有元素在配准中都是同等相关的。因此，我们提出了一种用于激光雷达里程计的语义类别加权ICP。该方法并非严格过滤掉属于特定语义类别的点，而是根据这些点的预期几何稳定性对属于语义类别的点的残差进行加权。这一策略使得信息丰富但可能不稳定的结构能够为配准过程做出贡献，同时减轻动态物体的影响。实验评估在SemanticKITTI和RELLIS-3D数据集上进行，这些数据集包括城市、高速公路、农村和越野环境。实证结果表明，所提出的语义加权ICP在姿态估计方面有所改善，特别是在常规刚性特征稀缺的挑战性越野场景中。此外，分析表明，这种加权策略的有效性高度依赖于环境，受场景的结构和语义组成的影响。

View on arXiv Download PDF AI Translation

cs.RO / 55 / 2606.03931

Multi-Robot Bearing-only Pose Estimation via Angle Rigidity

基于角度刚性的多机器人仅 bearing 位姿估计

Presenza, J. Francisco, Colombo, Leonardo J., Mas, Ignacio, Giribet, Juan I.

Abstract

This letter proposes a novel distributed bearing-based pose estimator for time-varying multi-robot systems. The method uses angles computed from body-frame bearings to estimate the robots' positions in $\mathbb{R}^3$ without knowledge of their orientations. The orientations in $\mathrm{SO}(3)$ are recovered from the estimated positions, the bearings, and the bearing derivatives. The proposed observer only requires the (directed) sensing topology to be \textit{angle-rigid}, a weaker condition than the commonly used ones like bearing rigidity. Local uniform exponential stability of the proposed observer is established under the assumption of persistently exciting motions for a subset of robots. Simulations are presented and discussed to evaluate the scheme's effectiveness and practicality.

Chinese Translation

本文提出了一种新颖的基于分布式 bearing 的位姿估计器，适用于时变的多机器人系统。该方法利用从机体坐标系中计算的角度来估计机器人在 $ ext{R}^3$ 中的位置，而无需了解其朝向。通过估计的位置、bearing 和 bearing 导数，恢复 $ ext{SO}(3)$ 中的朝向。所提出的观测器仅要求（定向）传感拓扑是 extit{角度刚性} 的，这一条件比常用的如 bearing 刚性等条件要弱。假设一部分机器人的运动是持续激励的，建立了所提观测器的局部均匀指数稳定性。通过仿真展示并讨论了该方案的有效性和实用性。

View on arXiv Download PDF AI Translation

cs.RO / 56 / 2606.03943

PointAction: 3D Points as Universal Action Representations for Robot Control

PointAction：将3D点作为机器人控制的通用动作表示

Tong, Mutian, Jiang, Han, Feng, Qiao, Liu, Lingjie, Gu, Jiatao

Abstract

Video-Action Models (VAMs) leverage the broad visual dynamics captured by pre-trained video diffusion models, offering a promising path toward generalizable robot manipulation. However, RGB-only video rollouts are not directly actionable: they leave metric 3D motion, contact geometry, and fine-grained spatial constraints under-specified, making action grounding ambiguous. Meanwhile, scaling action supervision across diverse tasks and embodiments remains costly. We present PointAction, a framework that bridges video predictions to robot actions through explicit point-based 4D modeling. PointAction fine-tunes a foundation video generation model to jointly predict future RGB frames and dynamic 3D pointmaps, producing temporally consistent 3D motion of task-relevant scene geometry. These point dynamics serve as a structured, embodiment-agnostic action interface, which a diffusion-based action decoder maps to executable robot actions. By using metric 3D point dynamics as the interface between video prediction and control, PointAction reduces the ambiguity of RGB-only action grounding and supports transfer across tasks and embodiments with limited action supervision. Experiments show that PointAction achieves state-of-the-art 4D generation quality on robot scenes, outperforms existing baselines in simulation, and generalizes to two real robot arms unseen during pretraining.

Chinese Translation

视频动作模型（VAMs）利用预训练视频扩散模型捕捉的广泛视觉动态，为通用机器人操控提供了一条有前景的路径。然而，仅依赖RGB的视频回放并不能直接用于动作执行：它们在度量3D运动、接触几何和细粒度空间约束方面的描述不够明确，导致动作的基础不清晰。同时，在不同任务和实现之间扩展动作监督仍然成本高昂。我们提出了PointAction，一个通过显式基于点的4D建模将视频预测与机器人动作连接起来的框架。PointAction微调了基础视频生成模型，以共同预测未来的RGB帧和动态3D点图，生成与任务相关的场景几何的时间一致的3D运动。这些点动态作为一个结构化的、与实现无关的动作接口，通过扩散式动作解码器映射到可执行的机器人动作。通过使用度量3D点动态作为视频预测与控制之间的接口，PointAction减少了仅依赖RGB的动作基础的不确定性，并支持在有限动作监督下跨任务和实现的迁移。实验表明，PointAction在机器人场景上实现了最先进的4D生成质量，在仿真中超越了现有基线，并在预训练期间未见过的两个真实机器人臂上实现了泛化。

View on arXiv Download PDF AI Translation

cs.RO / 57 / 2606.03949

Preference-Calibrated Human-in-the-Loop Reinforcement Learning for Robotic Manipulation

偏好校准的人机协作强化学习在机器人操作中的应用

Liu, Zeyi, Liu, Guangyao, Qu, Yinuo, Xue, Yuquan, Jia, Bofang, Yang, Chunhua, Gui, Weihua, Huang, Keke, Wang, Ziwei

Abstract

Human-in-the-loop reinforcement learning (HIL-RL) improves sample efficiency in real-robot manipulation through online human intervention. However, successful trajectories may include suboptimal actions that deviate from the desired task-execution path and force human intervention. Existing HIL-RL methods typically apply the consistent credit assignment principle to all transitions, uniformly propagating discounted terminal rewards through suboptimal segments, ignoring the actual contribution of each transition to task success. This overestimates Q-values for critic learning and indirectly misguides actor updates toward suboptimal behavior patterns. To this end, we propose PACT, a Preference-calibrated Actor-Critic Training framework that leverages the implicit preference signals induced by intervention to perform credit reassignment on identified suboptimal segments while directly guiding policy training for unbiased critic-actor learning. Specifically, we first design a progress model that learns from human demonstration and identifies suboptimal segments for credit correction. Then, from the human action and resampled policy action at the intervention state, we build preference pairs to define a counterfactual advantage that penalizes Bellman targets of the identified suboptimal segment, enabling directional credit calibration. Moreover, we directly align the policy with human corrective actions in the bounded mean space, providing an additional signal beyond critic-guided updates. Across five real-robot manipulation tasks, PACT improves the average success rate by 24.5% and achieves 1.3 times faster convergence, thereby improving both RL sample efficiency and performance. Code is available at https://anonymous.4open.science/r/HILRL-A1X-BC05.

Chinese Translation

人机协作强化学习（HIL-RL）通过在线人类干预提高了真实机器人操作的样本效率。然而，成功的轨迹可能包含偏离期望任务执行路径的次优动作，从而迫使人类进行干预。现有的HIL-RL方法通常将一致的信用分配原则应用于所有转移，均匀地通过次优段传播折扣终端奖励，忽视了每个转移对任务成功的实际贡献。这导致了对评论者学习的Q值的高估，并间接地误导了演员更新朝向次优行为模式。为此，我们提出了PACT，一个偏好校准的演员-评论者训练框架，利用干预所引发的隐含偏好信号对识别的次优段进行信用重新分配，同时直接指导策略训练以实现无偏的评论者-演员学习。具体而言，我们首先设计了一个进展模型，从人类示范中学习并识别次优段以进行信用修正。然后，从干预状态下的人类动作和重新采样的策略动作中，我们构建偏好对以定义一个反事实优势，惩罚识别的次优段的贝尔曼目标，从而实现方向性信用校准。此外，我们在有界均值空间中直接将策略与人类修正动作对齐，提供了超出评论者引导更新的额外信号。在五个真实机器人操作任务中，PACT将平均成功率提高了24.5%，并实现了1.3倍的更快收敛，从而提高了强化学习的样本效率和性能。代码可在 https://anonymous.4open.science/r/HILRL-A1X-BC05 获取。

View on arXiv Download PDF AI Translation

cs.RO / 58 / 2606.03963

Self-Refining Agentic Reinforcement Learning for Vision-Conditioned UAV Navigation

自我精炼的自主强化学习用于视觉条件下的无人机导航

Khan, Roohan Ahmed, Yaqoot, Yasheerah, Mustafa, Muhammad Ahsan, Tsetserukou, Dzmitry

Abstract

Deep reinforcement learning has shown strong potential for enabling autonomous robots to learn complex navigational tasks. However, its practical use still depends heavily on human designed reward functions and repeated manual fine tuning, which is time consuming and does not guarantee high success in the desired task. This paper presents AgenticRL, agent guided reinforcement learning framework that increases autonomy in reward design, policy refinement, and real world deployment for unmanned aerial vehicles (UAV) navigation tasks. AgenticRL uses a multimodal generative pre-trained tansformer (GPT) agent to interpret task information and visual scene observations, generate task specific reward functions, train policies using Proximal Policy Optimization (PPO) algorithm, and then act as a critic by evaluating the trained policy through diagnosis packets to generate feedback. Based on this feedback, the agent identifies failure modes and refines the reward function in a closed loop self improvement process. To further leverage the multimodal GPT agent during inference, AgenticRL uses real world images and natural language task information to automatically identify the active scenario and select the appropriate trained policy for execution. The framework is evaluated on multiple navigational tasks, including gate traversal, obstacle avoidance, wall barrier crossing with landing, trajectory following, and motion behavior learning. Experimental results show that the closed loop refinement process improves policy behavior compared with initial rewards by 71%. We also demonstrate sim-to-real transfer of the proposed framework, achieving a real world success rate of 91% and a sim-to-real accuracy of 94%.

Chinese Translation

深度强化学习在使自主机器人学习复杂导航任务方面展现了强大的潜力。然而，其实际应用仍然严重依赖于人工设计的奖励函数和反复的手动微调，这既耗时又无法保证在预期任务中的高成功率。本文提出了AgenticRL，一个自主引导的强化学习框架，旨在提高无人机（UAV）导航任务中奖励设计、策略优化和现实世界部署的自主性。AgenticRL利用多模态生成预训练变换器（GPT）代理来解读任务信息和视觉场景观察，生成特定任务的奖励函数，使用近端策略优化（PPO）算法训练策略，然后通过诊断包评估训练后的策略，充当评论者以生成反馈。基于这些反馈，代理识别失败模式并在闭环自我改进过程中优化奖励函数。为了在推理过程中进一步利用多模态GPT代理，AgenticRL使用现实世界图像和自然语言任务信息自动识别当前场景，并选择适当的训练策略进行执行。该框架在多个导航任务上进行了评估，包括门穿越、障碍物规避、墙壁障碍跨越与着陆、轨迹跟随和运动行为学习。实验结果表明，闭环精炼过程使策略行为相比初始奖励提高了71%。我们还展示了所提框架的仿真到现实转移，达到了91%的现实世界成功率和94%的仿真到现实精度。

View on arXiv Download PDF AI Translation

cs.RO / 59 / 2606.03985

Humanoid-GPT: Scaling Data and Structure for Zero-Shot Motion Tracking

类人型GPT：为零-shot运动追踪扩展数据和结构

Qi, Zekun, Chen, Xuchuan, Liu, Dairu, Lin, Chenghuai, Lian, Yunrui, Liang, Sikai, Zhang, Zhikai, Guan, Yu, Wang, Jilong, Zhang, Wenyao, Yu, Xinqiang, Wang, He, Yi, Li

Abstract

We introduce Humanoid-GPT, a GPT-style Transformer with causal attention trained on a billion-scale motion corpus for whole-body control. Unlike prior shallow MLP trackers constrained by scarce data and an agility-generalization trade-off, Humanoid-GPT is pre-trained on a 2B-frame retargeted corpus that unifies all major mocap datasets with large-scale in-house recordings. Scaling both data and model capacity yields a single generative Transformer that tracks highly dynamic behaviors while achieving unprecedented zero-shot generalization to unseen motions and control tasks. Extensive experiments and scaling analyses show that our model establishes a new performance frontier, demonstrating robust zero-shot generalization to unseen tasks while simultaneously tracking highly dynamic and complex motions.

Chinese Translation

我们介绍了类人型GPT（Humanoid-GPT），这是一种基于GPT风格的Transformer，采用因果注意力机制，训练于十亿规模的运动语料库，以实现全身控制。与之前受限于稀缺数据和灵活性泛化权衡的浅层多层感知器（MLP）追踪器不同，类人型GPT在一个包含2B帧重定向语料库的基础上进行预训练，该语料库统一了所有主要的动作捕捉（mocap）数据集以及大规模的内部录音。数据和模型容量的扩展使得我们得以构建一个单一的生成式Transformer，能够追踪高度动态的行为，同时在未见过的运动和控制任务上实现前所未有的零-shot泛化。大量实验和扩展分析表明，我们的模型建立了新的性能前沿，展示了对未见任务的强健零-shot泛化能力，同时能够追踪高度动态和复杂的运动。

View on arXiv Download PDF AI Translation

计算机视觉 (Computer Vision)

123

cs.CV / 1 / 2606.02603

COD10K-C: Benchmarking Robustness of Camouflaged Object Detection Under Natural Image Corruptions

COD10K-C：在自然图像损坏下评估伪装物体检测的鲁棒性基准

Sayem, Arafat Hossain

Abstract

Camouflaged object detection has improved substantially, but most standard benchmarks evaluate models only on clean images. This is not realistic because real cameras often capture blur, sensor noise, weather effects, and compression artifacts. We present COD10K-C, a corruption robustness benchmark based on COD10K. It includes 8 corruption types and 5 severity levels, giving 40 conditions and 81,040 evaluation pairs in total. We evaluate three popular camouflaged object detection models, SINet-v2, PFNet, and ZoomNet, as well as a lightweight model called RobustCODLite. All models show clear performance drops on corrupted images. Motion blur and Gaussian blur cause the largest drops, with SINet-v2 losing 18.5 Dice points under motion blur. Brightness and fog are less harmful. RobustCODLite uses corruption augmentation, a frequency-prior branch, and an uncertainty-consistency loss. It retains 92.3% of its clean Dice score under corruption, compared with 87.7% for SINet-v2, 84.8% for ZoomNet, and 84.1% for PFNet. On the hardest corruptions, RobustCODLite matches or outperforms models that perform better on clean data. We will release the COD10K-C GitHub repository to support future research in robust camouflaged object detection.

Chinese Translation

伪装物体检测已经取得了显著进展，但大多数标准基准仅在干净图像上评估模型。这并不现实，因为真实相机通常会捕捉到模糊、传感器噪声、天气影响和压缩伪影。我们提出了基于COD10K的损坏鲁棒性基准COD10K-C。它包括8种损坏类型和5个严重程度级别，总共提供40种条件和81,040对评估样本。我们评估了三种流行的伪装物体检测模型：SINet-v2、PFNet和ZoomNet，以及一个名为RobustCODLite的轻量级模型。所有模型在损坏图像上均显示出明显的性能下降。运动模糊和高斯模糊导致的性能下降最大，其中SINet-v2在运动模糊下损失了18.5个Dice分数。亮度和雾霾的影响较小。RobustCODLite使用了损坏增强、频率优先分支和不确定性一致性损失。在损坏情况下，它保留了92.3%的干净Dice分数，而SINet-v2为87.7%，ZoomNet为84.8%，PFNet为84.1%。在最困难的损坏情况下，RobustCODLite的表现与在干净数据上表现更好的模型相当或更优。我们将发布COD10K-C的GitHub仓库，以支持未来在鲁棒伪装物体检测方面的研究。

View on arXiv Download PDF AI Translation

cs.CV / 2 / 2606.02724

AVTrack: Audio-Visual Tracking in Human-centric Complex Scenes

AVTrack：人本复杂场景中的音视频跟踪

Wang, Yaoting, Zhou, Yun, Zhang, Zipei, Ding, Henghui

Abstract

Audio-visual speaker tracking aims to localize and track active speakers by leveraging auditory and visual cues, enabling fine-grained, human-centric scene understanding. This capability is essential for real-world applications such as intelligent video editing, surveillance, and human-computer interaction. However, existing datasets are largely limited to simple or homogeneous audio-visual scenes with coarse annotations. Such oversimplified settings bias evaluation toward static audio-visual co-occurrence, rather than rigorously assessing robust spatiotemporal modeling and cross-modal reasoning in complex, dynamic scenes. To address these limitations, we introduce AVTrack, a human-centric audio-visual instance segmentation (AVIS) dataset designed for dynamic real-world scenarios. AVTrack features diverse and challenging conditions, including camera motion, visual occlusions, and position changes. Evaluations of representative AVIS methods on AVTrack reveal substantial performance degradation, establishing AVTrack as a challenging benchmark for robust human-centric audio-visual scene understanding in complex environments. We further provide a simple yet effective baseline to facilitate future research. Project website: https://FudanCVL.github.io/AVTrack/

Chinese Translation

音视频说话者跟踪旨在通过利用听觉和视觉线索来定位和跟踪活跃的说话者，从而实现细粒度的人本场景理解。这一能力对于智能视频编辑、监控和人机交互等现实应用至关重要。然而，现有的数据集大多局限于简单或同质的音视频场景，并且注释粗略。这种过于简化的设置使得评估偏向于静态音视频共现，而不是严格评估复杂动态场景中的稳健时空建模和跨模态推理。为了解决这些局限性，我们引入了AVTrack，一个为动态现实场景设计的人本音视频实例分割（AVIS）数据集。AVTrack具有多样且具有挑战性的条件，包括摄像机运动、视觉遮挡和位置变化。在AVTrack上对代表性AVIS方法的评估显示出显著的性能下降，确立了AVTrack作为复杂环境中稳健的人本音视频场景理解的挑战性基准。我们还提供了一个简单而有效的基线，以促进未来的研究。项目网站：https://FudanCVL.github.io/AVTrack/

View on arXiv Download PDF AI Translation

cs.CV / 3 / 2606.02742

Consistent Yet Wrong: Evidence Insensitivity in Spatial Vision-Language Models

一致但错误：空间视觉-语言模型中的证据不敏感性

Bhat, S Divakar, Yamasaki, Toshihiko

Abstract

Spatial reasoning is fundamental to robotics, autonomy, and embodied AI, yet modern vision-language models (VLMs) remain unreliable on metric distance queries. A common assumption is that consistent predictions across viewpoints reflect geometric grounding. We test this assumption and find the opposite: leading VLMs often produce view-invariant and consistent answers even when those answers are incorrect, indicating weak coupling between predictions and viewpoint-specific visual evidence. We introduce \textbf{ViewDiag}, a controlled multi-view evaluation protocol built from Hypersim, ScanNet, and KITTI360, comprising 176 object-pair tracks across 80 scenes with 2--10 views per track. The protocol evaluates models along three axes: metric accuracy, distributional concentration, and a latent feature probe for internal collapse that distinguishes decision collapse from representation collapse. Across diverse models, we observe a consistent pattern of high prediction stability paired with substantial error, clustering in a regime characterized by strong consistency but low accuracy. \noindent These results challenge the common use of cross-view consistency as a proxy for geometric understanding. Instead, we show that stable predictions may reflect prior-driven collapse rather than evidence-sensitive reasoning. ViewDiag provides a controlled benchmark and diagnostic framework for evaluating spatial VLMs beyond accuracy alone. The code and data can be found \href{https://github.com/SDivakarBhat/Consistent_Yet_Wrong.git}{here}

Chinese Translation

空间推理是机器人技术、自治系统和具身人工智能的基础，但现代视觉-语言模型（VLMs）在度量距离查询上仍然不可靠。一个普遍的假设是，不同视角下的一致预测反映了几何基础。我们测试了这一假设，发现了相反的结果：领先的VLMs往往在视角不变的情况下给出一致的答案，即使这些答案是错误的，这表明预测与视角特定视觉证据之间的耦合较弱。我们引入了 extbf{ViewDiag}，一种基于Hypersim、ScanNet和KITTI360构建的受控多视角评估协议，涵盖了80个场景中176个物体对轨迹，每个轨迹有2至10个视角。该协议从三个维度评估模型：度量准确性、分布集中度，以及用于内部崩溃的潜在特征探测器，区分决策崩溃与表示崩溃。在多种模型中，我们观察到高预测稳定性与显著错误并存的一致模式，聚集在一个以强一致性但低准确性为特征的区域。这些结果挑战了将跨视角一致性作为几何理解代理的常见做法。相反，我们表明，稳定的预测可能反映了先验驱动的崩溃，而非证据敏感的推理。ViewDiag提供了一个受控基准和诊断框架，用于评估空间VLMs，超越单纯的准确性。代码和数据可以在此找到： ext{https://github.com/SDivakarBhat/Consistent_Yet_Wrong.git}

View on arXiv Download PDF AI Translation

cs.CV / 4 / 2606.02747

Plan2Map: A Multimodal Benchmark for Document-Grounded Geospatial Boundary Reconstruction from Planning Records

Plan2Map：一个基于文档的地理空间边界重建的多模态基准测试

Degen, Fabian, Deb, Oishi, Gu, Jindong, Yu, Junchi, Marro, Samuele, Torr, Philip, Yu, Jialin

Abstract

Planning records define restrictions over geographic areas, but their source documents often provide only indirect spatial evidence rather than machine-readable boundaries. We introduce Plan2Map, a 208-case multimodal benchmark for document-grounded geospatial boundary reconstruction from UK planning records. Given only a source planning document, systems must reconstruct a valid geospatial boundary from notice text, schedules, map plates, map labels, and boundary annotations; the reference GeoJSON is held out for scoring. We propose GeoPlanAgent, a document-grounded, geospatial-tool-in-the-loop system that decomposes the task into evidence extraction, localisation, map registration, boundary segmentation, projection, and verification. On Plan2Map, GeoPlanAgent achieves 0.736 mean IoU and 0.904 median IoU, with 67.8\% of predictions at or above 0.8 IoU, substantially outperforming direct VLM-to-GeoJSON baselines. Diagnostic analysis shows that direct VLM prediction remains unreliable, while remaining errors are concentrated in localisation and map registration, and supervised boundary segmentation substantially improves pixel-level mask quality. Plan2Map provides a concrete testbed for multimodal geospatial reconstruction from public planning records. Project page: https://odeb1.github.io/Plan2Map_Project_Page/.

Chinese Translation

规划记录定义了对地理区域的限制，但其源文档通常仅提供间接的空间证据，而非机器可读的边界。我们提出了Plan2Map，这是一个包含208个案例的多模态基准测试，旨在从英国规划记录中进行基于文档的地理空间边界重建。系统仅需给定一个源规划文档，便需从通知文本、时间表、地图图版、地图标签和边界注释中重建有效的地理空间边界；参考的GeoJSON用于评分。我们提出了GeoPlanAgent，这是一个基于文档的、地理空间工具在环的系统，将任务分解为证据提取、定位、地图注册、边界分割、投影和验证。在Plan2Map上，GeoPlanAgent实现了0.736的平均交并比（mean IoU）和0.904的中位数交并比（median IoU），其中67.8%的预测结果达到或超过0.8的交并比，显著优于直接的VLM到GeoJSON基线。诊断分析表明，直接的VLM预测仍然不可靠，而剩余的错误主要集中在定位和地图注册上，监督的边界分割显著提高了像素级掩码质量。Plan2Map为从公共规划记录中进行多模态地理空间重建提供了一个具体的测试平台。项目页面：https://odeb1.github.io/Plan2Map_Project_Page/

View on arXiv Download PDF AI Translation

cs.CV / 5 / 2606.02753

MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data

MetaWorld：从单视角视频数据扩展多智能体视频世界模型

Hu, Teng, Lu, Mingchun, Wang, Yating, Zhang, Jiangning, Hao, Jinkun, Pan, Ye, Yi, Ran, Ma, Lizhuang, Tao, Dacheng

Abstract

Video world models are a foundational generative technology for embodied AI and the Metaverse, yet existing approaches are inherently limited to a single agent observing from a single perspective. Extending these models to multi-agent settings introduces two critical challenges: data scarcity (coordinated multi-view recordings are prohibitively expensive to collect for general open-domain scenarios) and world state alignment (independently generated video streams cannot ensure that shared physical environments and events evolve consistently across views). To address these challenges, we propose MetaWorld, a novel framework that scales multi-agent video world models to open-domain environments directly from single-view videos. First, we introduce Monocular World-State Unrolling (MWSU) to explicitly decompose monocular footage into the camera operator's ego-motion and the visible subject's spatial trajectory. This camera-trajectory decomposition naturally extracts synchronized multi-agent motion data within a shared 3D space, completely bypassing the need for multi-camera setups. Second, for precise visual control, we develop the Subject-Aware World Generator to enable appearance-driven simulation conditioned on per-agent identity images. Finally, to ensure both views are grounded in the identical physical reality, we propose World-State Alignment, a per-frame inter-branch cross-attention mechanism inserted at every transformer layer of the video DiT. By jointly synchronizing the denoising process, WSA enforces both static geometric consistency and dynamic motion consistency, encouraging that the shared 3D environment and physical events remain well-aligned across both egocentric views. Extensive experiments demonstrate that MetaWorld achieves superior cross-view consistency and identity fidelity, establishing a highly scalable, physics-driven paradigm for multi-agent video world modeling.

Chinese Translation

视频世界模型是具身人工智能和元宇宙的基础生成技术，但现有方法本质上仅限于单个智能体从单一视角进行观察。将这些模型扩展到多智能体环境面临两个关键挑战：数据稀缺（协调的多视角录制在一般开放域场景中收集成本过高）和世界状态对齐（独立生成的视频流无法确保共享的物理环境和事件在不同视角下的一致演变）。为了解决这些挑战，我们提出了MetaWorld，这是一种新颖的框架，可以直接从单视角视频中扩展多智能体视频世界模型到开放域环境。首先，我们引入单目世界状态展开（Monocular World-State Unrolling, MWSU），以明确地将单目视频分解为摄像机操作者的自我运动和可见主体的空间轨迹。这种摄像机轨迹分解自然提取了共享三维空间内同步的多智能体运动数据，完全绕过了多摄像机设置的需求。其次，为了实现精确的视觉控制，我们开发了主体感知世界生成器（Subject-Aware World Generator），以便根据每个智能体的身份图像进行外观驱动的仿真。最后，为了确保两个视角都基于相同的物理现实，我们提出了世界状态对齐（World-State Alignment），这是一种在视频DiT的每个变换层中插入的逐帧跨分支交叉注意力机制。通过共同同步去噪过程，WSA强制执行静态几何一致性和动态运动一致性，确保共享的三维环境和物理事件在两个自我中心视角下保持良好对齐。大量实验表明，MetaWorld在视角一致性和身份保真度方面表现优越，建立了一种高度可扩展的、基于物理的多智能体视频世界建模范式。

View on arXiv Download PDF AI Translation

cs.CV / 6 / 2606.02764

From Local Training to Large-Scale Mapping: A Comparative Assessment of Machine Learning and Deep Learning for Transferable Satellite-Derived Bathymetry

从局部训练到大规模映射：机器学习与深度学习在可转移卫星衍生水深测量中的比较评估

Hsu, Hsiao-Jou, Moortgat, Joachim

Abstract

Satellite-derived bathymetry (SDB) from multispectral imagery is cost-effective but scales poorly across regions, especially in optically complex coastal environments. We evaluate machine learning and deep learning for transferable SDB over the 0-20 m depth range using Sentinel-2 imagery. A Random Forest baseline and four CNNs (ResNet-50, ResNet-101, EfficientNet-B4, ConvNeXt-Large) are trained on Pratas Island and selected Great Barrier Reef regions, then evaluated on spatially independent intra- and cross-regional test areas. Preserving spatial continuity during training, by keeping contiguous reef blocks rather than random patches, is the single most impactful design choice; we further introduce a Smooth Weight Function (SWF)-weighted RMSE loss that emphasizes near-surface depths. With these choices, intra-regional RMSE ranges from 1.15 to 1.92 m over 0-20 m and is as low as 0.26 m for depths <= 3 m. Random Forest degrades sharply under cross-regional transfer (RMSE 1.53 m -> 2.99-3.78 m), while the deep models stay more robust (2.46-2.98 m). On the public MagicBathyNet aerial-RGB benchmark (0-16 m) the proposed networks reach 0.19-0.22 m RMSE, outperforming a U-Net baseline and a task-specific transformer architecture with substantially fewer parameters. We further exploit multi-temporal repeat imagery: training on it broadens diversity, and median-aggregating predictions across passes at inference reduces noise from changing sun angles, atmospheric conditions, water properties, and tides. We release optimized architectures and pretrained weights to enable scalable transfer to new sites.

Chinese Translation

来自多光谱影像的卫星衍生水深测量（SDB）具有成本效益，但在不同区域的扩展性较差，尤其是在光学复杂的沿海环境中。我们评估了机器学习和深度学习在0-20米深度范围内对可转移SDB的应用，使用了Sentinel-2影像。以随机森林作为基线，训练了四个卷积神经网络（CNN）（ResNet-50、ResNet-101、EfficientNet-B4、ConvNeXt-Large），这些模型在普拉塔斯岛和选定的大堡礁区域进行训练，并在空间独立的区域内和跨区域测试区域进行评估。在训练过程中保持空间连续性，即保持相邻的礁块而非随机选取的斑块，是最具影响力的设计选择；我们进一步引入了一种平滑加权函数（Smooth Weight Function, SWF）加权的均方根误差（RMSE）损失，强调近表层深度。通过这些选择，区域内的RMSE在0-20米范围内从1.15到1.92米不等，对于深度小于等于3米的情况，RMSE低至0.26米。随机森林在跨区域转移时性能急剧下降（RMSE从1.53米增加到2.99-3.78米），而深度模型则保持相对稳健（2.46-2.98米）。在公共的MagicBathyNet航空RGB基准测试（0-16米）中，所提出的网络达到了0.19-0.22米的RMSE，优于基于U-Net的基线和具有显著较少参数的任务特定变换器架构。我们进一步利用多时相重复影像：在其上进行训练可以拓宽多样性，在推理过程中通过对不同时间段的预测进行中位数聚合，可以减少由于太阳角度、气象条件、水体特性和潮汐变化带来的噪声。我们发布了优化的架构和预训练权重，以便于在新地点的可扩展转移。

View on arXiv Download PDF AI Translation

cs.CV / 7 / 2606.02774

GeoDrive-Bench: Benchmarking Region-Specific Multimodal Reasoning in Autonomous Driving

GeoDrive-Bench：区域特定多模态推理在自动驾驶中的基准测试

Ma, Yingzi, Xiao, Chaowei, Jiang, Ming

Abstract

Vision-language models (VLMs) for autonomous driving have shown promising performance, but their ability to handle region-specific traffic rules remains underexplored, raising uncertainties about their deployment across diverse global settings. We therefore introduce GeoDrive-Bench, a novel benchmark that enables the systematic investigation of VLMs' geo-culturally grounded driving reasoning. We curated 5,053 human-validated multiple-choice QA pairs across six countries covering diverse driving cultures. Specifically, we emphasize four driving tasks: perception, prediction, planning, and region reasoning. Each question requires models to infer the correct driving behavior from visual evidence and local traffic conventions without explicit country labels. Beyond evaluation, we further design a distillation algorithm that injects region-specific traffic-rule knowledge into the internal representations of VLMs, enabling models to better align visual scene understanding with local driving policies. Experiments on nine state-of-the-art VLMs show substantial performance variations across geo-driving cultures for each task, while our proposed baseline models exhibit improved geo-cultural reasoning across regions. These results suggest that current VLMs still lack robust region-aware driving intelligence and highlight GeoDrive-Bench as a diagnostic and training-oriented testbed for deployable autonomous driving foundation models.

Chinese Translation

用于自动驾驶的视觉-语言模型（VLMs）表现出良好的性能，但它们处理区域特定交通规则的能力仍然未得到充分探索，这引发了在多样化全球环境中部署的诸多不确定性。因此，我们提出了GeoDrive-Bench，一个新颖的基准，旨在系统性地研究VLMs在地理文化背景下的驾驶推理。我们在六个国家策划了5,053个经过人工验证的多项选择问答对，涵盖了多样的驾驶文化。具体而言，我们强调四个驾驶任务：感知、预测、规划和区域推理。每个问题要求模型根据视觉证据和当地交通惯例推断正确的驾驶行为，而无需明确的国家标签。除了评估之外，我们还设计了一种蒸馏算法，将区域特定的交通规则知识注入VLMs的内部表征，使模型能够更好地将视觉场景理解与当地驾驶政策对齐。在九个最先进的VLMs上的实验显示，各个任务在地理驾驶文化中表现出显著的性能差异，而我们提出的基线模型在各区域的地理文化推理上表现出改善。这些结果表明，当前的VLMs仍然缺乏强大的区域感知驾驶智能，并强调GeoDrive-Bench作为可部署的自动驾驶基础模型的诊断和训练导向测试平台。

View on arXiv Download PDF AI Translation

cs.CV / 8 / 2606.02789

Diagnosis of Human Object Interaction Detectors for Real World Educational Applications

人类物体交互检测器在现实世界教育应用中的诊断

Mereddy, Divya, Sadashiva, Ashwin Tudur, Quinones-Grueiro, Marcos, Biswas, Gautam

Abstract

Human-object interaction (HOI) recognition is critical for automatically analyzing student behavior in complex educational environments. Although state-of-the-art (SOTA) HOI detectors perform well on benchmark datasets, their performance often degrades when deployed in real-world training environments due to domain-specific objects, occlusions, and complex visual conditions. In this paper, we introduce a diagnosis-driven framework that integrates a triplet-level HOI error taxonomy with error-factor attribution analysis for real-world educational video data. We study this problem in the context of Critical Care Air Transport Team (CCATT) mixed-reality medical training. Based on an analysis of HOI failure modes and their causes, we develop a diagnosis-informed refinement strategy for adapting pretrained HOI models to the target domain. Experiments on the CCATT dataset show that this approach improves the macro-F1 score of a pretrained CDN model from 48.6 to 90.2 through targeted refinement guided by diagnosed error factors. These results highlight the value of detailed diagnostic analysis for informing targeted adaptation of HOI models in real-world educational environments.

Chinese Translation

人类-物体交互（HOI）识别对于自动分析复杂教育环境中的学生行为至关重要。尽管最先进的（SOTA）HOI 检测器在基准数据集上表现良好，但在实际培训环境中部署时，由于领域特定的物体、遮挡和复杂的视觉条件，其性能往往会下降。本文提出了一种以诊断为驱动的框架，该框架将三元组级 HOI 错误分类法与针对现实世界教育视频数据的错误因素归因分析相结合。我们在危重护理空中运输团队（CCATT）混合现实医学培训的背景下研究这一问题。基于对 HOI 失败模式及其原因的分析，我们开发了一种基于诊断的细化策略，以适应预训练的 HOI 模型到目标领域。在 CCATT 数据集上的实验表明，这种方法通过针对性细化，基于诊断的错误因素，将预训练的 CDN 模型的宏观 F1 分数从 48.6 提高到 90.2。这些结果突显了详细诊断分析在指导 HOI 模型在现实世界教育环境中进行有针对性适应方面的价值。

View on arXiv Download PDF AI Translation

cs.CV / 9 / 2606.02800

Cosmos 3: Omnimodal World Models for Physical AI

Cosmos 3：用于物理人工智能的全模态世界模型

Aditi, Agarwal, Niket, Ali, Arslan, Allen, Jon, Antolini, Martin, Aubame, Adeline, Azzolini, Alisson, Bai, Junjie, Bala, Maciej, Balaji, Yogesh, Bapst, Josh, Basant, Aarti, Beladiya, Mukesh, Bhat, Mohammad Qazim, Bhat, Zaid Pervaiz, Blick, Dan, Brighella, Vanni, Cai, Han, Cai, Tiffany, Cameracci, Eric, Cao, Jiaxin, Cao, Yulong, Carlson, Mark, Casanova, Carlos, Chang, Ting-Yun, Chang, Yan, Chao, Yu-Wei, Chattopadhyay, Prithvijit, Chaudhari, Roshan, Chen, Chieh-Yun, Chen, Junyu, Chen, Ke, Chen, Qizhi, Chen, Wenkai, Chen, Xiaotong, Chen, Yu, Cheng, An-Chieh, Cheng, Click, Chia, Xiu, Choi, Jeana, Chung, Chaeyeon, Cong, Wenyan, Cui, Yin, Dadela, Magdalena, Dadhich, Nalin, Dai, Wenliang, Daw, Joyjit, Degirmenci, Alperen, Del Monte, Rodrigo Vieira, Denomme, Robert, Dharur, Sameer, Di Lucca, Marco, Ding, Ke, Ding, Wenhao, Ding, Yifan, Dong, Yuzhu, Drumheller, Nicole, Du, Yilun, Dzhumamuratova, Aigul, Efitorov, Aleksandr, Eghbalzadeh, Hamid, Eigbe, Naomi, Hanafi, Imad El, Eslami, Hassan, Falk, Benedikt, Fan, Jiaojiao, Fan, Jim, Fasale, Amol, Fefilatyev, Sergiy, Feng, Liang, Ferroni, Francesco, Fidler, Sanja, Fu, Xiao, Fugro, Vikram, Gaikwad, Prashant, Galda, TJ, Gao, Katelyn, Gao, Yihuai, Ge, Wenhang, Ghosh, Sreyan, Goel, Arushi, Goel, Vivek, Gokul, Akash, Govindaraju, Rama, Gu, Jinwei, Guerrero, Miguel, Guo, Elfie, Gupta, Aryaman, Gururani, Siddharth, Hadfield, Hugo, Han, Song, Handa, Ankur, Hao, Zekun, Harrim, Mohammad, Hassani, Ali, Hayes-Roth, Nathan, He, Yufan, Helvig, Chris, Hogg, Cyrus, Huang, Madison, Huang, Michael, Huang, Sophia, Huang, Yufan, Huffman, Jacob, Hutchins, DeLesley, Indupuru, Suneel, Ivanovic, Boris, Jain, Arihant, Jang, Joel, Ji, Ryan, Jian, Yanan, Jiang, Dongfu, Jin, Jingyi, Joshi, Atharva, Joshi, Nikhilesh, Joshi, Pranjali, Jung, Jaehun, Kang, Weiwei, Kassekert, Scott, Kautz, Jan, Khetan, Ashna, Kiczka, Julia, Kierat, Slawek, Kim, Gwanghyun, Kim, Kuno, Kim, Sunny, Kong, Kezhi, Kong, Xin, Kong, Zhifeng, Kornuta, Tomasz, Krivov, Egor, Kuang, Hui, Kumar, Saurav, Kuo, Chia-Wen, Kurian, George, Kutak, Wojciech, Lafleche, JF, Lahkar, Himangshu, Laymoun, Omar, Lee, Jayjun, Lee, Sanggil, Leone, Gabriele, Li, Boyi, Li, Freya, Li, Jiajun, Li, Jinfeng, Li, Ling, Li, Pengcheng, Li, Shangru, Li, Tingle, Li, Xiaolong, Li, Xuan, Li, Zhaoshuo, Li, Zhiqi, Liang, Hao, Liao, Maosheng, Lin, Chen-Hsuan, Lin, Tsung-Yi, Liu, Ming-Yu, Liu, Sifei, Liu, Zihan, Lu, Hai Loc, Lu, Xiangyu, Luo, Alice, Luo, Ruipu, Luo, Wenjie, Lyu, Jiangran, Ma, Martin Ding, Ma, Nic, Ma, Qianli, Majchrowski, Dawid, Marcoux, Louis, Martin, Miguel, Miao, Qing, Mirzaei, Ashkan, Misra, Shreyas, Mo, Kaichun, Mohsin, Durra, Moon, Hyejin, Morkisz, Pawel, Motiian, Saeid, Motkov, Kirill, Nah, Seungjun, Narang, Yashraj, Narayanan, Deepak, Ngazimbi, Thabang, Ouyang, Julian, Page, David, Pang, Yatian, Park, Sehwi, Patekar, Mahesh, Patwary, Mostofa, Pavone, Marco, Pham, Trung, Ping, Wei, Pouya, Soha, Prabhumoye, Shrimai, Praveen, Varun, Qu, Delin, Rabeti, Hesam, Ramezanali, Morteza, Reeb, Marilyn, Ren, Xuanchi, Rumley, Kristen, Rymer, Wojciech, Saito, Jun, Seol, Yeongho, Shao, John, Shekdar, Piyush, Shen, Tianwei, Shi, Humphrey, Shi, Min, Shi, Stella, Shih, Kevin, Shoeybi, Mohammad, Sieniawski, Mateusz, Song, Shuran, Sotelo, Alexander, Sotoodeh, Amir, Srinivasa, Sunil, Srinivasakumar, Vignesh, Stefaniak, Bartosz, Steiger, Rahul Heinrich, Sun, Shangkun, Tang, Jiaxiang, Tang, Shitao, Tang, Yangyang, Tang, Yue, Tavakkoli, Tolou, Ting, Kayley, Tomala, Krzysztof, Tseng, Wei-Cheng, Varghese, Jibin, Vasilev, Sergei, Volk, Thomas, Wagwani, Raju, Waleffe, Roger, Wang, Andrew Z., Wang, Boxiang, Wang, Haoxiang, Wang, Qiao, Wang, Shihao, Wang, Shijie, Wang, Ting-Chun, Wang, Yan, Wang, Yu, Wehr, David, Wei, Fangyin, Weng, Xinshuo, Wu, Jay Zhangjie, Wu, Kedi, Xia, Hongchi, Xiao, Summer, Xiao, Tianjun, Xie, Kevin, Xu, Daguang, Xu, Jiashu, Xu, Mengyao, Xu, Ruqing, Xu, Xingqian, Xu, Yao, Yang, Dinghao, Yang, Dong, Yang, Hans, Yang, Xiaodong, Yang, Xuning, Yang, Yichu, You, Yurong, Yu, Zhiding, Yuan, Hao, Yuen, Simon, Zeng, Xiaohui, Zeren, Pengcuo, Zha, Cindy, Zhang, Haotian, Zhang, Jenny, Zhang, Jing, Zhang, Liangkai, Zhang, Paris, Zhang, Shun, Zhang, Xuanmeng, Zhang, Zhizheng, Zhao, Ann, Zhao, Yilin, Zhautouskaya, Yuliya, Zhou, Charles, Zhou, Fengzhe, Zhu, Shilin, Zhu, Yuke, Zhylko, Dima, Zolkowski, Artur

Abstract

We introduce Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences within a unified mixture-of-transformers architecture. By supporting highly flexible input-output configurations, Cosmos 3 seamlessly unifies critical modalities for Physical AI -- effectively subsuming vision-language models, video generators, world simulators, and world-action models into a single framework. Our evaluation demonstrates that Cosmos 3 establishes a new state-of-the-art across a diverse suite of understanding and generation tasks, demonstrating omnimodal world models as scalable, general-purpose backbones for embodied agents. Our post-trained Cosmos 3 models were ranked as the best open-source Text-to-Image and Image-to-Video models by Artificial Analysis, and the best policy model by RoboArena at the time the technical report was written. To accelerate open research and deployment in Physical AI, we make our code, model checkpoints, curated synthetic datasets, and evaluation benchmark available under the Linux Foundation's OpenMDW-1.1 https://openmdw.ai/license/1-1/ License at https://github.com/nvidia/cosmos}{github.com/nvidia/cosmos and https://huggingface.co/collections/nvidia/cosmos3 . The project website is available at https://research.nvidia.com/labs/cosmos-lab/cosmos3 .

Chinese Translation

我们介绍了Cosmos 3，一个全模态世界模型系列，旨在通过统一的混合变换器架构共同处理和生成语言、图像、视频、音频和动作序列。通过支持高度灵活的输入输出配置，Cosmos 3无缝地统一了物理人工智能的关键模态——有效地将视觉-语言模型、视频生成器、世界模拟器和世界-动作模型纳入一个单一框架。我们的评估表明，Cosmos 3在多样化的理解和生成任务中建立了新的最先进水平，展示了全模态世界模型作为具身代理的可扩展通用骨干。我们的后训练Cosmos 3模型在技术报告撰写时被Artificial Analysis评为最佳开源文本到图像和图像到视频模型，并被RoboArena评为最佳策略模型。为了加速物理人工智能领域的开放研究和部署，我们将我们的代码、模型检查点、精心策划的合成数据集和评估基准在Linux基金会的OpenMDW-1.1许可证下公开，链接为https://github.com/nvidia/cosmos和https://huggingface.co/collections/nvidia/cosmos3。项目网站可访问https://research.nvidia.com/labs/cosmos-lab/cosmos3。

View on arXiv Download PDF AI Translation

cs.CV / 10 / 2606.02809

Automated Report-Derived Oncology VQA Benchmark for Evaluating Vision-Language Models on 3D Medical Imaging

基于自动化报告的肿瘤学视觉问答基准，用于评估视觉-语言模型在三维医学影像上的表现

Liu, Bo, Gu, Hanxue, Li, Xiangru, Zhu, Zheren, Ellison, Jacob, Wang, Kang, Lupo, Janine M., Yang, Yang, Lin, Hui

Abstract

Evaluating vision-language models (VLMs) on medical images requires benchmarks that are clinically grounded, scalable, and controlled for evaluation confounds. Existing public benchmarks are limited in scale, manually annotated, or potentially leaked into VLM pretraining corpora. We present an automated agent-driven pipeline that generates multiple-choice VQA datasets directly from paired private radiology reports and 3D oncology imaging, producing two complementary question types: RADS-style questions deterministically derived from clinician-defined reporting schemas, and radiology report-derived questions generated by an LLM from radiologist findings and verified against the source report. Applied to four in-house cancer cohorts, the pipeline yields an instance-contamination-controlled benchmark without per-question human annotation. Zero-shot evaluation of six VLMs reveals no dominant model and substantial headroom across all cells. A blind ablation reveals that visual reliance is highly dataset-specific: liver Report-derived questions genuinely require the image, while Lung CT is essentially solvable without it - the leading closed model exceeds its sighted accuracy on Lung CT when blinded - indicating that even private clinical data does not guarantee a contamination-controlled read of visual capability. The pipeline is released as an open agent skill for in-house redeployment.

Chinese Translation

在医学影像上评估视觉-语言模型（VLMs）需要临床基础、可扩展且控制评估混淆因素的基准。现有的公共基准在规模上有限，手动注释，或可能泄露到VLM的预训练语料中。我们提出了一种自动化代理驱动的管道，该管道直接从配对的私有放射学报告和三维肿瘤影像生成多项选择视觉问答（VQA）数据集，产生两种互补的问题类型：基于临床医生定义的报告模式确定性生成的RADS风格问题，以及由大型语言模型（LLM）从放射科医师的发现生成并与源报告核对的放射学报告派生问题。应用于四个内部癌症队列，该管道生成了一个实例污染控制的基准，无需逐题的人类注释。对六个VLM的零样本评估显示没有主导模型，并且所有单元都有显著的提升空间。盲法消融实验表明视觉依赖性高度特定于数据集：肝脏报告派生问题确实需要图像，而肺部CT在没有图像的情况下基本上可以解决——在盲测时，领先的封闭模型在肺部CT上的表现超过了其有视力时的准确性——这表明即使是私有临床数据也无法保证对视觉能力的污染控制解读。该管道作为开放代理技能发布，供内部重新部署使用。

View on arXiv Download PDF AI Translation

cs.CV / 11 / 2606.02831

Principled Reflection Separation via Nonlinear Superposition and Feature Interaction

通过非线性叠加和特征交互实现原则性反射分离

Hu, Qiming, Li, Mingjia, Li, Yuntong, Guo, Xiaojie

Abstract

Single-image reflection separation is fundamentally challenged by the entanglement of transmission and reflection layers under complex image formation processes. Existing approaches largely rely on simplified assumptions or independent modeling, limiting their ability to handle real-world scenarios. In this work, we revisit the problem from a unified perspective and identify a key issue of existing approaches, i.e., the widely adopted linear composition model in the sRGB domain fails to capture the nonlinear coupling introduced by real-world image signal processing pipelines. To address this, we introduce a learnable nonlinear superposition model that more faithfully characterizes layer interactions and improves decomposition fidelity. Building upon this formulation, we propose a generalized dual-stream interactive framework that explicitly models bidirectional dependencies between transmission and reflection through feature exchange. This framework unifies activation-, gating-, and attention-based interaction mechanisms, and is compatible with both CNN and Transformer backbones. Extensive experiments on diverse real-world benchmarks demonstrate that the proposed approach achieves superior performance with strong generalization capability. More importantly, our study reveals that reflection separation is not about undoing a linear mixture, but about learning nonlinear formation and interaction}, offering new insights into the design of principled image decomposition models. Code and models are publicly available at https://mingcv.github.io/DIRS-Page.

Chinese Translation

单幅图像的反射分离在复杂的图像形成过程中受到传输层和反射层纠缠的根本挑战。现有方法在很大程度上依赖于简化假设或独立建模，限制了它们处理现实场景的能力。在本研究中，我们从统一的视角重新审视这一问题，并识别出现有方法的一个关键问题，即在sRGB域中广泛采用的线性组合模型未能捕捉到真实世界图像信号处理管道引入的非线性耦合。为了解决这一问题，我们引入了一种可学习的非线性叠加模型，该模型更真实地表征层间交互并提高分解的保真度。在此基础上，我们提出了一种广义的双流交互框架，该框架通过特征交换明确建模传输和反射之间的双向依赖关系。该框架统一了激活、门控和基于注意力的交互机制，并与CNN和Transformer骨干网兼容。在多种现实世界基准上的大量实验表明，所提出的方法在性能上优于其他方法，并具有较强的泛化能力。更重要的是，我们的研究揭示了反射分离并不是关于撤销线性混合，而是关于学习非线性形成和交互，为原则性图像分解模型的设计提供了新的见解。代码和模型可在 https://mingcv.github.io/DIRS-Page 上公开获取。

View on arXiv Download PDF AI Translation

cs.CV / 12 / 2606.02877

Pathway-Structured Privileged Distillation for Deployable Computational Pathology

可部署计算病理学的通路结构特权蒸馏

Guo, Yongxin, Lu, Hao, Koyun, Onur, Zhu, Zhengjie, Demir, Muhammet, Gurcan, Metin

Abstract

Integrating transcriptomics and histopathology can improve cancer risk modelling, yet practical use is constrained by the limited availability of RNA profiling in routine settings. Here we introduce Mixture of Pathway Experts (MoPE), a knowledge-distillation framework that reframes multimodal learning as privileged distillation for histology-only inference. MoPE is motivated by the partial observability between RNA profiles and whole-slide images: histology can capture morphology-linked consequences of certain molecular programmes, but cannot be expected to reconstruct the full transcriptomic state. MoPE encodes RNA-derived pathways and transfers the molecular supervision to pathway-indexed pathology experts through memory-usage alignment. Across diverse public benchmarks and two independent breast cancer cohorts, MoPE consistently improved WSI-only inference performance relative to baseline methods. Pathway-usage analyses and human-audited visual inspection provide bounded inspection of model behaviour and candidate morphology-linked readouts. These results support pathway-structured privileged distillation as a promising route to using molecular information during training while preserving RNA-free inference.

Chinese Translation

整合转录组学和组织病理学可以改善癌症风险建模，但实际应用受到常规环境中RNA分析有限可用性的限制。在此，我们介绍了通路专家混合模型（Mixture of Pathway Experts, MoPE），这是一种知识蒸馏框架，将多模态学习重新构建为仅基于组织学推断的特权蒸馏。MoPE的动机源于RNA谱和全切片图像之间的部分可观测性：组织学可以捕捉某些分子程序的形态学相关后果，但不能期望重建完整的转录组状态。MoPE编码RNA衍生的通路，并通过内存使用对齐将分子监督转移到通路索引的病理专家。在各种公共基准和两个独立的乳腺癌队列中，MoPE相对于基线方法始终提高了仅基于WSI的推断性能。通路使用分析和人工审核的视觉检查提供了对模型行为和候选形态学相关输出的有限检查。这些结果支持通路结构特权蒸馏作为在训练过程中使用分子信息的有前景的途径，同时保持RNA自由推断。

View on arXiv Download PDF AI Translation

cs.CV / 13 / 2606.02894

Tiny Collaborative Inference for Occlusion-Robust Object Detection

微型协作推理用于抗遮挡物体检测

Cheng, Chieh-Tung, Aslanov, Mustafa, Kanjo, Eiman

Abstract

Small edge devices such as IoT surveillance nodes and search-and-rescue (SAR) platforms are increasingly expected to run computer vision locally. On ultra-low-end hardware, however, object detection is limited by available memory and compute, by communication costs when several devices cooperate, and by the loss of accuracy caused by occlusion. The work evaluates occlusion-robust object detection on devices with less than 1 MB SRAM by combining an MCUNet backbone, a YOLOv2 detection head, and TensorFlow Lite quantisation. We evaluate two collaborative inference strategies: feature-level fusion, which concatenates intermediate feature maps, and decision-level fusion via Weighted Boxes Fusion (WBF). Under the tested occlusion settings, WBF outperforms feature-level fusion and gives gains of up to +0.2736 mAP in asymmetric occlusion scenarios. Extending fusion to three views improves accuracy further (up to +0.3827 mAP) while adding communication overhead (approximately 1.3 KB per exchange). The hardware experiments start with a host-assisted USB-relay baseline and then move to a Wi-Fi peer-to-peer deployment on two Coral Dev Board Micro units, where WBF runs on-device and communication energy remains small relative to inference. In a representative 301.9 s autonomous session comprising 108 frames, fused output is observed on 61 frames compared with 47 for Board 2 alone, a frame-level coverage gain of +29.8%. We also include a small exploratory decentralised federated learning (DFL) feasibility note, but do not treat it as a main result because performance remains limited under non-iid local data. The results support decision-level fusion as a viable option for improving occlusion robustness in small-scale edge object detection, including host-free multi-board operation on ultra-low-end hardware.

Chinese Translation

小型边缘设备，如物联网监控节点和搜索与救援（SAR）平台，越来越被期望能够在本地运行计算机视觉。然而，在超低端硬件上，物体检测受到可用内存和计算能力的限制、多个设备协作时的通信成本影响，以及遮挡造成的准确性损失。本文评估了在小于1 MB SRAM的设备上进行抗遮挡物体检测的效果，通过结合MCUNet主干、YOLOv2检测头和TensorFlow Lite量化。我们评估了两种协作推理策略：特征级融合，即连接中间特征图，以及通过加权框融合（Weighted Boxes Fusion, WBF）进行的决策级融合。在测试的遮挡设置下，WBF的表现优于特征级融合，在不对称遮挡场景中提升了最高+0.2736 mAP。将融合扩展到三个视角进一步提高了准确性（最高+0.3827 mAP），同时增加了通信开销（每次交换约1.3 KB）。硬件实验从主机辅助的USB中继基线开始，然后转向在两个Coral Dev Board Micro单元上的Wi-Fi点对点部署，其中WBF在设备上运行，通信能耗相对于推理保持较小。在一个代表性的301.9秒自主会话中，共包含108帧，融合输出在61帧上被观察到，而仅在Board 2上观察到47帧，帧级覆盖增益为+29.8%。我们还包括了一小部分探索性去中心化联邦学习（DFL）可行性说明，但不将其视为主要结果，因为在非独立同分布（non-iid）本地数据下性能仍然有限。结果支持决策级融合作为提高小规模边缘物体检测抗遮挡能力的可行选项，包括在超低端硬件上进行无主机的多板操作。

View on arXiv Download PDF AI Translation

cs.CV / 14 / 2606.02915

Any2Poster: Any-Source Poster Generation Across Modalities and Domains

Any2Poster：跨模态和领域的任意来源海报生成

Vinaykumar, Amogh, Li, Aiden, Huang, Suozhi, Liu, Shilong

Abstract

Visual posters are a compact medium for communicating dense information, yet progress on automatic poster generation remains difficult to measure because existing evaluations are often restricted to paper-only inputs, narrow domains, or surface-level visual similarity. We introduce Any2Poster Bench, a benchmark for any-source poster generation that evaluates systems across eight input modalities--PDFs, URLs, PPTX, DOCX, Markdown, LaTeX, notebooks, and videos--and five content domains. Any2Poster Bench pairs each source with quiz-based probes of verbatim factual retention and interpretive understanding, together with VLM-based judgments of visual quality, layout, readability, content completeness, and logical flow, enabling reproducible assessment of both information fidelity and visual communication. To instantiate and validate this benchmark, we further present Any2Poster Agent, an end-to-end reference agent that parses heterogeneous sources, organizes salient content, plans poster layouts, renders posters, and iteratively refines them using visual feedback. On Any2Poster Bench, Any2Poster Agent achieves 87.25% average accuracy across input modalities and 87.28% across content domains. On PaperQuiz-style evaluation, where prior paper-to-poster agents are directly comparable, Any2Poster Agent improves over PosterAgent-4o from 51.06-51.33% to 72.58% overall accuracy and from 116-121 to 145.16 in density-augmented score. Together, Any2Poster Bench and Any2Poster Agent provide a reusable evaluation resource and a competitive baseline for studying multimodal, domain-general poster generation.

Chinese Translation

视觉海报是一种紧凑的媒介，用于传达密集信息，然而，自动海报生成的进展仍然难以衡量，因为现有的评估往往仅限于纸质输入、狭窄领域或表面视觉相似性。我们引入了Any2Poster Bench，这是一个用于任意来源海报生成的基准，评估系统在八种输入模态（PDF、URL、PPTX、DOCX、Markdown、LaTeX、笔记本和视频）和五个内容领域的表现。Any2Poster Bench将每个来源与基于测验的探测相结合，以评估逐字事实保留和解释理解，同时结合基于视觉语言模型（VLM）的视觉质量、布局、可读性、内容完整性和逻辑流的判断，从而实现信息保真度和视觉传达的可重复评估。为了实例化和验证这一基准，我们进一步提出了Any2Poster Agent，这是一个端到端的参考代理，能够解析异构来源，组织显著内容，规划海报布局，渲染海报，并使用视觉反馈迭代优化。通过Any2Poster Bench，Any2Poster Agent在输入模态上实现了87.25%的平均准确率，在内容领域上实现了87.28%。在PaperQuiz风格的评估中，之前的纸质到海报代理可以直接比较，Any2Poster Agent的整体准确率从51.06-51.33%提高到72.58%，密度增强得分从116-121提高到145.16。总之，Any2Poster Bench和Any2Poster Agent提供了一个可重复使用的评估资源和一个竞争性基准，以研究多模态、领域通用的海报生成。

View on arXiv Download PDF AI Translation

cs.CV / 15 / 2606.02919

Pixel Cube: Diffusion-based Portrait Video Relighting Through Realistic Lighting Reproduction

像素立方体：基于扩散的肖像视频重光照通过真实光照再现

Zhang, Yufan, Ji, Yu, Ajiboye, Ayo, Wu, Rundi, Guo, Yu, Zheng, Changxi, Ye, Jinwei

Abstract

We present a diffusion-based method for relighting dynamic portrait videos with photorealism and temporal consistency. Our method is fueled by a hybrid training dataset that consists of real-captured and rendered dynamic portrait videos with diverse subject appearances, facial motions, head poses, and known lighting conditions. Specifically, we construct an LED-based lighting system for realistic lighting emulation and high-speed video relighting data acquisition. By leveraging the image priors embedded in pre-trained video diffusion models, and using per-frame high dynamic range (HDR) environment map as lighting control, we train a high-performance generative model for realistic and identity-preserving dynamic portrait video relighting. In addition to the environment map control, our model uses a synthesized background image to enable control on the camera's exposure level and color tone. Our model can produce temporally consistent relit portrait video that looks realistic and harmonious under a provided new environment and faithfully preserve the subject's expression and fine facial features, including skin tone, wrinkles, and facial hair. Our model generalizes well to unseen data, in terms of the subject appearance, motion, and lighting condition. We perform extensive experiments on relighting in-the-wild videos with various environment maps and demonstrate practical applications on portrait photography. Results show that our method achieves state-of-the-art performance in photorealism, lighting harmony, and temporal consistency.

Chinese Translation

我们提出了一种基于扩散的方法，用于以照片真实感和时间一致性重光照动态肖像视频。我们的方法依赖于一个混合训练数据集，该数据集由真实捕捉和渲染的动态肖像视频组成，具有多样的主体外观、面部动作、头部姿势和已知光照条件。具体而言，我们构建了一个基于LED的光照系统，以实现真实光照的模拟和高速视频重光照数据的采集。通过利用嵌入在预训练视频扩散模型中的图像先验，并使用每帧高动态范围（HDR）环境图作为光照控制，我们训练了一个高性能的生成模型，用于真实且保持身份特征的动态肖像视频重光照。除了环境图控制外，我们的模型还使用合成的背景图像，以便控制相机的曝光水平和色调。我们的模型能够生成时间一致的重光照肖像视频，在提供的新环境下看起来真实且和谐，并忠实保留主体的表情和细腻的面部特征，包括肤色、皱纹和面部毛发。我们的模型在未见数据上表现良好，涵盖了主体外观、动作和光照条件。我们对各种环境图进行了大量的野外视频重光照实验，并展示了在肖像摄影中的实际应用。结果表明，我们的方法在照片真实感、光照和谐性和时间一致性方面达到了最先进的性能。

View on arXiv Download PDF AI Translation

cs.CV / 16 / 2606.02924

ATLAS: A Large-Scale Evaluation Benchmark for Adversarial LiDAR Perception

ATLAS：针对对抗性LiDAR感知的大规模评估基准

Zhang, Mellon M., Panse, Siddhant, Fan, Zimo, Dhal, Akshal, Sarkar, Rishit, Chou, Glen

Abstract

Autonomous driving perception is typically evaluated on clean benchmark data, yet real-world deployment requires robustness to rare, structured, and potentially adversarial sensor anomalies. This gap is especially critical for LiDAR, where external actors can physically manipulate the sensing process to induce black-box perception failures without accessing the model. Existing LiDAR benchmarks provide little visibility into this failure mode. Prior adversarial LiDAR studies have largely centered on attack hardware, geometric and algorithmic defenses, and early-generation detectors, leaving the robustness of modern perception systems unexplored. To address this evaluation gap, we introduce ATLAS (Adversarial Temporal LiDAR Attack Suite), the first large-scale, physically grounded evaluation benchmark for LiDAR perception models under black-box sensor attacks, simulating the two primary attack modes -- point injection and point removal -- across real driving sequences. Evaluating a broad cross-section of current state-of-the-art LiDAR perception models, ATLAS reveals a surprising robustness asymmetry: models with stronger performance on standard benchmarks tend to better withstand removal attacks, yet are actually more vulnerable to injection attacks than weaker models. We trace this vulnerability to standard object database sampling augmentations, revealing how current training practices can induce architecture-agnostic robustness failures, and study initial directions for mitigating both attack modes. We release the ATLAS generation code to support extensible, reproducible evaluations as attack capabilities evolve, helping make black-box sensor robustness an explicit consideration in future LiDAR perception development.

Chinese Translation

自主驾驶感知通常在干净的基准数据上进行评估，但实际部署需要对稀有、结构化和潜在的对抗性传感器异常具备鲁棒性。这一差距在LiDAR中尤为关键，因为外部行为者可以物理操控感知过程，诱导黑箱感知失败，而无需访问模型。现有的LiDAR基准对这种失败模式提供的信息有限。以往的对抗性LiDAR研究主要集中于攻击硬件、几何和算法防御以及早期生成的检测器，导致现代感知系统的鲁棒性尚未被探索。为了解决这一评估空白，我们引入了ATLAS（对抗性时序LiDAR攻击套件），这是第一个针对黑箱传感器攻击下LiDAR感知模型的大规模、物理基础评估基准，模拟了两种主要攻击模式——点注入和点移除——在真实驾驶序列中的表现。通过评估当前一系列最先进的LiDAR感知模型，ATLAS揭示了一个令人惊讶的鲁棒性不对称性：在标准基准上表现更强的模型往往能更好地抵御移除攻击，但实际上对注入攻击的脆弱性却高于表现较弱的模型。我们追溯这种脆弱性到标准对象数据库采样增强，揭示了当前训练实践如何导致架构无关的鲁棒性失败，并研究了缓解这两种攻击模式的初步方向。我们发布了ATLAS生成代码，以支持可扩展、可重复的评估，随着攻击能力的发展，帮助将黑箱传感器鲁棒性作为未来LiDAR感知开发中的一个显性考虑因素。

View on arXiv Download PDF AI Translation

cs.CV / 17 / 2606.02927

SaluNet: Enabling Total Plasticity in Normalization-Free Deep Networks

SaluNet：在无归一化深度网络中实现完全可塑性

Zaied, Mourad

Abstract

Normalization layers such as BatchNorm and LayerNorm have long been considered essential for stable training in deep networks. This work demonstrates that they can be fully replaced by a single learnable activation mechanism. We identify a plasticity suppression effect induced by standard normalization: learnable activation parameters rapidly lose adaptability when paired with normalization layers. Motivated by this observation, we introduce SALU (Saturated Adaptive Linear Unit), \[ \operatorname{SALU}(x;a,b) = \frac{a x}{\sqrt{1 + a b x^2}},\quad a>0,\; b>0 \] a bounded, learnable activation that provides intrinsic signal stabilization without relying on batch statistics or external affine parameters. Building on SALU, we propose SaluNet, a paradigm grounded in total plasticity: SALU replaces normalization layers, while SWALU and GALU replace standard activations. With ResNet-18, SaluNet-C-18 achieves 97.35\% on CIFAR-10 and 83.25\% on CIFAR-100 without normalization, maintaining 93.44\% and 76.23\% at batch size 1 where normalized architectures fail. For transformers, SaluNet-T improves over LayerNorm-GELU from 90.92\% to 91.01\% on CIFAR-10 and from 66.54\% to 68.10\% on CIFAR-100. SaluNet-C-50 reaches 78.67\% Top-1 on ImageNet-1K at $224\times224$, and $79.23\%$ at $288\times288$. These results suggest normalization layers suppress total plasticity, a property biological neurons inherently possess, enabling deep networks to learn effectively.

Chinese Translation

归一化层，如批归一化（BatchNorm）和层归一化（LayerNorm），长期以来被认为是深度网络稳定训练的关键。本研究表明，它们可以被单一可学习的激活机制完全替代。我们识别出标准归一化引起的可塑性抑制效应：当与归一化层配对时，可学习的激活参数迅速失去适应性。基于这一观察，我们引入了SALU（饱和自适应线性单元）， \[ \operatorname{SALU}(x;a,b) = \frac{a x}{\sqrt{1 + a b x^2}},\quad a>0,\; b>0 \] 这是一个有界的、可学习的激活函数，能够在不依赖批统计或外部仿射参数的情况下提供内在的信号稳定性。在SALU的基础上，我们提出了SaluNet，这是一种基于完全可塑性的范式：SALU替代了归一化层，而SWALU和GALU替代了标准激活函数。在ResNet-18上，SaluNet-C-18在CIFAR-10上达到了97.35\%的准确率，在CIFAR-100上达到了83.25\\%，且在批大小为1时保持93.44\\%和76.23\\%，而归一化架构在此情况下失效。对于变换器，SaluNet-T在CIFAR-10上将LayerNorm-GELU的准确率从90.92\\%提高到91.01\\%，在CIFAR-100上从66.54\\%提高到68.10\\%。SaluNet-C-50在ImageNet-1K上以$224\times224$的输入达到了78.67\\%的Top-1准确率，在$288\times288$的输入下达到了79.23\\%。这些结果表明，归一化层抑制了完全可塑性，而这一特性是生物神经元固有的，使得深度网络能够有效学习。

View on arXiv Download PDF AI Translation

cs.CV / 18 / 2606.02935

CAD-to-CT Registration of Cylindrical Objects via Ellipse-Based Axis Estimation

基于椭圆轴估计的圆柱形物体CAD与CT配准

Ogonowski, Aleksander, Mrozowski, Mikołaj, Więcek, Daniel, Ćwiek, Arkadiusz, Klimaszewski, Konrad, Możdżonek, Rafał, Padee, Adam, Raczyński, Lech, Wasiuk, Piotr, Wiślicki, Wojciech, Matusiak, Michał, Wronka, Sławomir

Abstract

Accurate registration of CAD models to CT scans is essential for establishing ground truth geometry in volumetric imaging. Obtaining reliable object masks is of growing importance in machine learning settings; as recent architectures grow more capable, huge datasets are required to fully utilise their capabilities. Traditional intensity-based methods fail when CT grayscale values lack calibration references, while point-based algorithms (e.g., ICP, RANSAC) require feature correspondence unavailable between idealized CAD geometry and noisy volumetric CT data. We propose a two-stage geometric registration method for cylindrical objects (ionization chambers) that takes advantage of the distinctive geometric features of the objects. First, we estimate the 3D rotation axis by detecting elliptical cross-sections across CT slices, fitting ellipses to edge-detected contours, and performing PCA on the fitted ellipse centers after RANSAC outlier removal. Second, we voxelize the CAD model, orient it along the detected axis, and maximize volumetric overlap with the CT scan through translational adjustment. This approach achieves robust registration with tilt and orientation errors below $0.1^\circ$ without intensity calibration or feature matching. Once registered, the aligned CAD model provides ground truth geometry for applications including machine learning-based object localization and automated analysis in industrial CT workflows.

Chinese Translation

CAD模型与CT扫描的准确配准对于在体积成像中建立真实几何形状至关重要。在机器学习环境中，获取可靠的物体掩膜变得越来越重要；随着近期架构能力的提升，充分利用其能力需要大量数据集。当CT灰度值缺乏校准参考时，传统的基于强度的方法会失效，而基于点的算法（如ICP、RANSAC）则需要在理想化的CAD几何形状与噪声较大的体积CT数据之间不存在的特征对应关系。我们提出了一种针对圆柱形物体（电离室）的两阶段几何配准方法，利用物体的独特几何特征。首先，通过检测CT切片中的椭圆截面，拟合边缘检测轮廓的椭圆，并在RANSAC去除异常值后对拟合的椭圆中心进行主成分分析（PCA），来估计3D旋转轴。其次，我们对CAD模型进行体素化，沿检测到的轴进行定向，并通过平移调整最大化与CT扫描的体积重叠。该方法在没有强度校准或特征匹配的情况下，实现了倾斜和方向误差低于$0.1^ ext{°}$的稳健配准。一旦配准，已对齐的CAD模型为包括基于机器学习的物体定位和工业CT工作流程中的自动分析等应用提供了真实几何形状。

View on arXiv Download PDF AI Translation

cs.CV / 19 / 2606.02956

The Road Ahead in Autonomous Driving: The KITScenes Multimodal Dataset

自主驾驶的未来之路：KITScenes多模态数据集

Schwarzkopf, Richard, Immel, Fabian, Blumberg, Alexander, Merkert, Jonas, Rack, Nils, Wang, Kaiwen, Konstantinidis, Fabian, Truetsch, Julian, Fernandez, Carlos, Bätz, Annika, Rösch, Kevin, Steiner, Marlon, Poh, Willi, Shen, Yinzhe, Wagner, Royden, Hauser, Felix, Strutz, Dominik, Villa, Jaime, Stepanov, Gleb, Caesar, Holger, Taş, Ömer Şahin, Bieder, Frank, Pauls, Jan-Hendrik, Stiller, Christoph

Abstract

Existing autonomous driving datasets have enabled major progress, but fall short in sensor fidelity, map completeness, or geographic diversity. We present KITScenes Multimodal, a European dataset built around high-fidelity sensors and maps. Our fully synchronized sensor suite combines high-resolution global-shutter cameras, long-range lidar beyond 400m, 4D imaging radar, and redundant GNSS/INS localization. Our HD maps are, to our knowledge, the most complete of any sensor dataset, validated through autonomous driving trials on open-source software. For the first time in a public dataset, all driving-relevant traffic elements, such as traffic lights, are mapped in 3D to a reprojection-accurate level with full topological connectivity. Recorded in cities with irregular street layouts and mixed traffic modes, our dataset complements existing datasets by broadening the available geographic diversity. We also introduce four benchmarks, each advancing spatial learning for embodied AI: online HD map construction, long-range depth estimation, novel view synthesis, and end-to-end driving. Project page: https://kitscenes.com/

Chinese Translation

现有的自主驾驶数据集促进了重大进展，但在传感器精度、地图完整性或地理多样性方面仍显不足。我们提出了KITScenes Multimodal，这是一个围绕高精度传感器和地图构建的欧洲数据集。我们的全同步传感器套件结合了高分辨率全球快门相机、超过400米的长距离激光雷达、4D成像雷达和冗余的GNSS/INS定位。我们的高清地图在我们所知的任何传感器数据集中都是最完整的，通过开源软件的自主驾驶试验进行了验证。在公共数据集中，所有与驾驶相关的交通元素（如交通信号灯）首次以3D形式映射到重投影精度水平，并具备完整的拓扑连接性。我们的数据集记录了街道布局不规则和交通模式混合的城市，补充了现有数据集，拓宽了可用的地理多样性。我们还引入了四个基准，推动了具身人工智能的空间学习：在线高清地图构建、长距离深度估计、新视图合成和端到端驾驶。项目页面：https://kitscenes.com/

View on arXiv Download PDF AI Translation

cs.CV / 20 / 2606.02962

Hand Trajectory Fusion for Egocentric Natural Language Query Grounding

基于手部轨迹融合的自我中心自然语言查询定位

Zhong, Enmin, del-Blanco, Carlos R., Jaureguizar, Fernando, García, Narciso

Abstract

Egocentric Natural Language Query (NLQ) grounding asks a model to localize, in a long first-person video, the temporal interval that answers a free-form text query. Existing methods fuse video appearance with the query but ignore hand motion, despite the fact that roughly 41% of Ego4D NLQ queries are answered at a moment of hand--object manipulation or their immediate outcomes.We propose a hand-trajectory encoder for converting a sequence of hand skeletons into highly-semantic hand kinematic features, which are then aligned and combined with pretrained video--text features through a cross-attention fusion strategy with adaptive gating. On the Ego4D NLQ v2 validation split, the clearest gains appear for Hand-Object Interaction queries (+2.54 R1@IoU=0.3) and Quantity/State queries (+4.32 R1@IoU=0.3), indicating that hand trajectory provides grounding cues beyond appearance alone.

Chinese Translation

自我中心自然语言查询（NLQ）定位要求模型在一段较长的第一人称视频中定位出回答自由形式文本查询的时间区间。现有方法将视频外观与查询融合，但忽视了手部运动，尽管大约41%的Ego4D NLQ查询是在手部与物体操作或其直接结果的时刻回答的。我们提出了一种手部轨迹编码器，将一系列手部骨架转换为高度语义化的手部运动学特征，这些特征通过自适应门控的交叉注意力融合策略与预训练的视频-文本特征对齐并结合。在Ego4D NLQ v2验证集上，手-物体交互查询的提升最为明显（+2.54 R1@IoU=0.3）以及数量/状态查询（+4.32 R1@IoU=0.3），这表明手部轨迹提供了超越外观的定位线索。

View on arXiv Download PDF AI Translation

cs.CV / 21 / 2606.02979

Towards Compact Autonomous Driving Perception with Balanced Learning and Multi-sensor Fusion

朝向紧凑型自主驾驶感知的平衡学习与多传感器融合

Natan, Oskar, Miura, Jun

Abstract

We present a novel compact deep multi-task learning model to handle various autonomous driving perception tasks in one forward pass. The model performs multiple views of semantic segmentation, depth estimation, light detection and ranging (LiDAR) segmentation, and bird's eye view projection simultaneously without being supported by other models. We also provide an adaptive loss weighting algorithm to tackle the imbalanced learning issue that occurred due to plenty of given tasks. Through data pre-processing and intermediate sensor fusion techniques, the model can process and combine multiple input modalities retrieved from RGB cameras, dynamic vision sensors (DVS), and LiDAR placed at several positions on the ego vehicle. Therefore, a better understanding of a dynamically changing environment can be achieved. Based on the ablation study, the model variant trained with our proposed method achieves a better performance. Furthermore, a comparative study is also conducted to clarify its performance and effectiveness against the combination of some recent models. As a result, our model maintains better performance even with much fewer parameters. Hence, the model can inference faster with less GPU memory utilization. Moreover, the result tends to be consistent in 3 different CARLA simulation datasets and 1 real-world nuScenes-lidarseg dataset. To support future research, we share codes and other files publicly at https://github.com/oskarnatan/compact-perception.

Chinese Translation

我们提出了一种新颖的紧凑型深度多任务学习模型，以在一次前向传递中处理各种自主驾驶感知任务。该模型能够同时执行语义分割、深度估计、激光雷达（LiDAR）分割和鸟瞰图投影的多视图任务，而无需依赖其他模型。我们还提供了一种自适应损失加权算法，以解决由于任务数量众多而导致的不平衡学习问题。通过数据预处理和中间传感器融合技术，该模型能够处理和结合来自RGB摄像头、动态视觉传感器（DVS）和安装在自我车辆多个位置的LiDAR的多种输入模态。因此，可以更好地理解动态变化的环境。基于消融研究，使用我们提出的方法训练的模型变体表现出更好的性能。此外，我们还进行了比较研究，以阐明其相对于一些最新模型组合的性能和有效性。结果表明，即使参数数量大幅减少，我们的模型仍保持更好的性能。因此，该模型可以更快地推理，并且GPU内存利用率更低。此外，结果在3个不同的CARLA仿真数据集和1个真实世界的nuScenes-lidarseg数据集上表现出一致性。为了支持未来的研究，我们在https://github.com/oskarnatan/compact-perception上公开分享代码和其他文件。

View on arXiv Download PDF AI Translation

cs.CV / 22 / 2606.03005

MUSE: A Unified Agentic Harness for MLLMs

MUSE：一种统一的代理执行框架用于多模态大语言模型

Lu, Jianglin, Wang, Hailing, Ma, Xu, Dong, Qihua, Zhang, Mingyuan, Wang, Yizhou, Fu, Yun

Abstract

Despite rapid progress, multimodal large language models (MLLMs) still fail on tasks that humans solve effortlessly, such as navigating a grid maze from a screenshot or selecting the correct puzzle piece. Rather than retraining the model, we ask a complementary question: how much capability can be elicited from a frozen MLLM purely by improving the execution scaffold around it? We introduce MUSE, a multimodal unified structured execution harness that wraps any off-the-shelf MLLM with composable modules for task representation, visual processing, perception tool use, structured parsing, deterministic verification, and verifier-guided repair, without any model retraining. We evaluate MUSE across diverse benchmarks spanning visual spatial planning, visual perception, multimodal reasoning, and fine-grained visual discrimination, using multiple state-of-the-art MLLMs. MUSE delivers consistent gains over the bare model in all settings, with the largest jumps on challenging instances. Further analysis reveals that many MLLM failures arise from harness-level shortcomings rather than fundamental model deficits, and can be addressed through verifier-guided repair without touching the model. These findings highlight the agentic multimodal harness as a critical yet underexplored design dimension, offering an orthogonal avenue for improving MLLMs beyond model-centric optimization.

Chinese Translation

尽管取得了快速进展，多模态大语言模型（MLLMs）在一些人类轻松解决的任务上仍然表现不佳，例如从截图中导航网格迷宫或选择正确的拼图块。我们并不打算重新训练模型，而是提出一个互补的问题：通过改善围绕模型的执行框架，可以从一个冻结的MLLM中引出多少能力？我们介绍了MUSE，这是一种多模态统一结构执行框架，它将任何现成的MLLM与可组合的模块结合起来，用于任务表示、视觉处理、感知工具使用、结构化解析、确定性验证和验证者引导的修复，而无需任何模型重训。我们在视觉空间规划、视觉感知、多模态推理和细粒度视觉辨别等多种基准上评估MUSE，使用多个最先进的MLLM。MUSE在所有设置中均提供了相对于裸模型的一致增益，尤其在具有挑战性的实例上表现出最大的提升。进一步分析表明，许多MLLM的失败源于框架层面的缺陷，而非模型的根本缺陷，并且可以通过验证者引导的修复来解决，而无需触及模型。这些发现突显了代理多模态框架作为一个关键但尚未充分探索的设计维度，提供了一条超越以模型为中心的优化来改善MLLM的正交途径。

View on arXiv Download PDF AI Translation

cs.CV / 23 / 2606.03050

FCUS-rPPG: A Fast-Converging Unsupervised Framework for Remote Photoplethysmography via Gradient Oscillation Suppression

FCUS-rPPG：一种快速收敛的无监督远程光电容积描记框架，通过梯度振荡抑制实现

Li, Jiajie, Liu, Yu, Song, Rencheng, Chen, Xun, Cheng, Juan

Abstract

Remote photoplethysmography (rPPG) enables non-contact extraction of blood volume pulse (BVP) signals using consumer-grade cameras. Recent unsupervised rPPG methods learn BVP representations without requiring ground-truth physiological annotations, yet their optimization is often hindered by noisy and unstable gradients, resulting in slow convergence and limited cross-domain generalization. In this paper, we propose FCUS-rPPG, a fast-converging unsupervised rPPG framework with strong generalization capability. Motivated by the observation that BVP representations exhibit both multi-spectral covariation and low-dimensional manifold structure, we design a spectrally shared backbone that facilitates BVP feature disentanglement while improving optimization efficiency. To jointly enhance convergence stability and generalization performance, we further develop a unified optimization framework operating at the gradient, loss-landscape, and feature-representation levels. Specifically, a post-verification masking mechanism filters out misleading gradients according to the weak-amplitude physiological prior of BVP signals; a perturbation-based loss landscape smoothing strategy steers optimization toward more generalizable flat minima; and a noise-aware null-space regularization constrains feature updates to the orthogonal complement of the noise subspace, thereby mitigating noise-induced representation drift. Extensive experiments on five datasets demonstrate that FCUS-rPPG requires only one training epoch, whereas existing methods typically require tens to hundreds of epochs. Notably, FCUS-rPPG consistently achieves state-of-the-art (SOTA) performance in cross-dataset evaluations. This study provides an efficient and robust solution to the real-world deployment of unsupervised rPPG. The source code will be publicly available at https://github.com/JiaJieLee/FCUS-rPPG.

Chinese Translation

远程光电容积描记（rPPG）利用消费级相机实现非接触式血容量脉搏（BVP）信号提取。近期的无监督rPPG方法在不需要真实生理注释的情况下学习BVP表示，但其优化常常受到噪声和不稳定梯度的阻碍，导致收敛缓慢和跨领域泛化能力有限。本文提出FCUS-rPPG，一种快速收敛的无监督rPPG框架，具有强大的泛化能力。基于BVP表示同时展现多光谱协变性和低维流形结构的观察，我们设计了一种光谱共享的主干网络，以促进BVP特征的解耦，同时提高优化效率。为了共同增强收敛稳定性和泛化性能，我们进一步开发了一个统一的优化框架，作用于梯度、损失景观和特征表示层面。具体而言，后验证掩蔽机制根据BVP信号的弱幅度生理先验过滤掉误导性梯度；基于扰动的损失景观平滑策略引导优化朝向更具泛化性的平坦极小值；而噪声感知的零空间正则化将特征更新限制在噪声子空间的正交补上，从而减轻噪声引起的表示漂移。在五个数据集上的大量实验表明，FCUS-rPPG仅需一个训练周期，而现有方法通常需要数十到数百个周期。值得注意的是，FCUS-rPPG在跨数据集评估中始终实现了最先进的（SOTA）性能。本研究为无监督rPPG的实际应用提供了高效且稳健的解决方案。源代码将公开发布于 https://github.com/JiaJieLee/FCUS-rPPG。

View on arXiv Download PDF AI Translation

cs.CV / 24 / 2606.03069

ROBUST-WT: Robust Uncertainty-aware Segmentation Transform via Whitening and Training Enhancements

ROBUST-WT：通过白化和训练增强实现的鲁棒不确定性感知分割变换

Naseer, Aqsa, Bibi, Maryam, Urooj, Syeda Samiya, Shahzad, Muhammad Khurram

Abstract

Generalized segmentation of medical images prevents performance degradation when different imaging devices and clinical protocols are used across multiple domains. The Whitening Transform-based Probabilistic Shape Regularization Extractor (WT-PSE), published in IEEE Transactions on Medical Imaging in 2024, addresses this challenge by employing feature decorrelation and Wasserstein distance-based knowledge distillation to achieve robust cross-domain segmentation. This study systematically examines improvements to the WT-PSE learning framework. Four limitations in the original implementation are identified: limited training augmentations that fail to simulate real scanner variations, reliance on per-pixel binary cross-entropy loss that is sensitive to edge noise, the absence of a scheduled loss weighting strategy that may destabilize early training, and the lack of ablation switches for controlled scientific comparison. To address these issues, we propose four enhancements: (1) domain-adaptive augmentation including random erasing, gamma correction, and salt-and-pepper noise; (2) a hybrid BCE and Dice loss function for improved edge-aware segmentation under noisy conditions; (3) a curriculum-based Dice weight scheduling strategy; and (4) command-line control flags for systematic ablation studies. Experiments on the fundus optic disc segmentation benchmark demonstrate that the improved pipeline achieves a final epoch optic-disc Dice score of 0.956 and an ASD score of 13.31, outperforming the baseline epoch-5 Dice score of 0.939. These results indicate that training-level improvements can provide consistent performance gains without modifying the underlying WT-PSE architecture.

Chinese Translation

医学图像的广义分割在不同成像设备和临床协议跨多个领域使用时防止性能下降。2024年发表在《IEEE医学成像汇刊》上的基于白化变换的概率形状正则化提取器（WT-PSE）通过采用特征去相关和基于Wasserstein距离的知识蒸馏来解决这一挑战，实现鲁棒的跨域分割。本研究系统地检视了对WT-PSE学习框架的改进。我们识别出原始实现中的四个局限性：有限的训练增强未能模拟真实扫描仪的变异、依赖于对每个像素的二元交叉熵损失对边缘噪声敏感、缺乏可能会使早期训练不稳定的调度损失加权策略，以及缺乏用于控制科学比较的消融开关。为了解决这些问题，我们提出了四项增强措施：（1）包括随机擦除、伽马校正和盐与胡椒噪声的领域自适应增强；（2）用于在噪声条件下改善边缘感知分割的混合BCE和Dice损失函数；（3）基于课程的Dice权重调度策略；（4）用于系统消融研究的命令行控制标志。在视网膜光盘分割基准测试中的实验表明，改进后的管道在最后一个训练周期中实现了光盘Dice得分0.956和ASD得分13.31，超越了基线的第五周期Dice得分0.939。这些结果表明，训练级别的改进可以在不修改基础WT-PSE架构的情况下提供一致的性能提升。

View on arXiv Download PDF AI Translation

cs.CV / 25 / 2606.03075

TGV-KV: Text-Grounded KV Eviction for Vision-Language Models

TGV-KV：基于文本的KV驱逐方法用于视觉-语言模型

Liu, Jizhihui, Han, Ruizi, Zhang, Miao, Shao, Rui, Liu, Xuebo, Guan, Weili, Wang, Yaowei

Abstract

Vision-Language Models (VLMs) inherit the auto-regressive generation paradigm and cache the keys and values (KV) of all previous tokens to accelerate inference, resulting in memory consumption that scales linearly with context length. This issue is particularly pronounced in VLMs due to substantial redundancy in the visual modality. Although KV cache eviction approaches can effectively reduce inference memory, they often incur significant performance degradation in VLMs, as most are designed for language models and overlook the inherent gap between text and vision. By systematically analyzing the modality gap in VLMs in this work, we argue that the importance of visual information should be grounded in textual guidance and accordingly propose a Text-Grounded KV Eviction method for VLMs (TGV-KV). TGV-KV comprises three submodules: (1) Text-Vision Budgeting (TVB) assigns budget to each layer based on the mutual information interaction. (2) Text-Weighted Ranking (TWR) assesses the priority of text and ranks vision importance based on weighted text-image attention. (3) Text-Prioritised Retention (TPR) policy strategically preserves text KV to avoid acute information loss. We evaluate TGV-KV across five models with different sizes and architectures, showing that TGV-KV preserves 99.2% full-KV accuracy on the VizWiz-VQA task with LLaVA-NeXT and boosts end-to-end throughput by 52.6% with an extreme retention budget of 5%. Code is available at https://github.com/Danielement321/TGV-KV.

Chinese Translation

视觉-语言模型（VLMs）继承了自回归生成范式，并缓存所有先前标记的键值（KV）以加速推理，这导致内存消耗与上下文长度线性相关。由于视觉模态中存在大量冗余，这一问题在VLMs中尤为明显。尽管KV缓存驱逐方法可以有效减少推理内存，但它们往往会在VLMs中导致显著的性能下降，因为大多数方法是为语言模型设计的，忽视了文本与视觉之间的固有差距。通过对VLMs中的模态差距进行系统分析，我们认为视觉信息的重要性应基于文本指导，并相应地提出了一种基于文本的KV驱逐方法（TGV-KV）。TGV-KV包含三个子模块：（1）文本-视觉预算（TVB）根据互信息交互为每一层分配预算；（2）文本加权排名（TWR）评估文本的优先级，并基于加权文本-图像注意力对视觉重要性进行排名；（3）文本优先保留（TPR）策略战略性地保留文本KV，以避免急剧的信息损失。我们在五种不同大小和架构的模型上评估了TGV-KV，结果表明TGV-KV在LLaVA-NeXT的VizWiz-VQA任务上保留了99.2%的全KV准确率，并在极端保留预算为5%的情况下提高了端到端吞吐量52.6%。代码可在https://github.com/Danielement321/TGV-KV获取。

View on arXiv Download PDF AI Translation

cs.CV / 26 / 2606.03084

Hierarchical Federated Learning with Dynamic Clustering and Adaptive Regularization for Robust Infrastructure Inspection

具有动态聚类和自适应正则化的分层联邦学习用于稳健的基础设施检测

Feng, Yuhu, Maeda, Keisuke, Ogawa, Takahiro, Haseyama, Miki

Abstract

The deployment of data-driven computer vision models for structural health monitoring (SHM) is heavily constrained by the data silo dilemma due to stringent privacy and security regulations. While federated learning (FL) offers a privacy-preserving collaborative alternative, its application to nationwide infrastructure networks is severely hindered by the challenge of ``double heterogeneity'': macro-level physical divergence across disparate structural types and micro-level statistical imbalances within local datasets. To overcome this challenge, this paper proposes a novel hierarchical federated learning framework. The framework orchestrates a synergistic two-tier optimization strategy. At the macro-level, a dynamic gradient-based clustering mechanism autonomously aggregates distributed clients into specialized expert groups based on their structural degradation trajectories, circumventing the need for prior geographical metadata. Concurrently, at the micro-level, an intra-cluster Dynamic Region-Adaptive Proximal Regularization (DRAPR) module computes a real-time statistical Non-IID Intensity Score for each client. By adaptively modulating a proximal penalty based on local label skewness and gradient divergence, DRAPR effectively calibrates local updates, mitigates client drift, and prevents the catastrophic forgetting of minority damage classes. Comprehensive evaluations on a large-scale, real-world structural inspection dataset demonstrate that the hierarchical integration of macro-clustering and micro-regularization successfully neutralizes dual-level heterogeneity, yielding highly robust and specialized diagnostic models for complex infrastructure inspection.

Chinese Translation

由于严格的隐私和安全法规，基于数据驱动的计算机视觉模型在结构健康监测（SHM）中的部署受到数据孤岛困境的严重限制。虽然联邦学习（FL）提供了一种保护隐私的协作替代方案，但其在全国基础设施网络中的应用受到“双重异质性”挑战的严重阻碍：不同结构类型之间的宏观物理差异和本地数据集内的微观统计不平衡。为了解决这一挑战，本文提出了一种新颖的分层联邦学习框架。该框架协调了一种协同的两层优化策略。在宏观层面上，动态基于梯度的聚类机制根据分布式客户端的结构退化轨迹自主地将其聚合为专业专家组，避免了对先前地理元数据的需求。同时，在微观层面上，集群内的动态区域自适应近端正则化（Dynamic Region-Adaptive Proximal Regularization, DRAPR）模块为每个客户端计算实时统计非独立同分布强度分数。通过根据本地标签偏斜度和梯度发散度自适应调节近端惩罚，DRAPR有效地校准本地更新，减轻客户端漂移，并防止少数损伤类别的灾难性遗忘。在大规模真实世界结构检测数据集上的全面评估表明，宏观聚类和微观正则化的分层集成成功中和了双层异质性，为复杂基础设施检测提供了高度稳健和专业的诊断模型。

View on arXiv Download PDF AI Translation

cs.CV / 27 / 2606.03100

Zero-Shot 3D Question Answering via Hierarchical View-to-Token Transportation

通过层次化视图到标记的传输实现零样本3D问答

Wang, Dongsheng, Su, Dawei, Huang, Hui

Abstract

Recently, zero-shot 3D scene understanding via 2D Vision-Language Models (VLMs) has gained increasing research interest due to their promising spatial reasoning capabilities. Typically, multiple 2D views are sampled from a 3D point cloud and fed into pre-trained VLMs to answer a given question. This paradigm highlights the critical role of input context quality and raises the challenge of retaining as many task-relevant 3D details as possible under a limited input budget. We propose \texttt{KeyVT}, a hierarchical approach for input context collection at both the view and token levels. Specifically, we combine pixel features with camera parameters and assess view importance based on both semantic content and geometric position, resulting in spatially consistent and task-relevant views. Furthermore, we address redundancy among patches across selected views by identifying representative tokens under the optimal transport (OT) framework, where view tokens and key tokens are formulated as two discrete distributions in the embedding space. These key tokens are expected to cover all view features by minimizing the OT distance. We evaluate our framework on three widely used benchmarks, demonstrating significant improvements over existing tuning-free methods and performance comparable to training-based approaches.

Chinese Translation

近年来，通过2D视觉-语言模型（VLMs）实现零样本3D场景理解引起了越来越多的研究关注，因为它们在空间推理能力方面表现出色。通常，从3D点云中采样多个2D视图，并将其输入预训练的VLMs以回答给定问题。这一范式突显了输入上下文质量的重要性，并提出了在有限输入预算下尽可能保留与任务相关的3D细节的挑战。我们提出了 exttt{KeyVT}，一种在视图和标记层面进行输入上下文收集的层次化方法。具体而言，我们将像素特征与相机参数相结合，并根据语义内容和几何位置评估视图的重要性，从而生成空间一致且与任务相关的视图。此外，我们通过在最优传输（OT）框架下识别代表性标记，解决了所选视图之间补丁的冗余问题，其中视图标记和关键标记在嵌入空间中被构造为两个离散分布。这些关键标记预计通过最小化OT距离来覆盖所有视图特征。我们在三个广泛使用的基准上评估了我们的框架，显示出相较于现有的无调优方法有显著改进，并且性能与基于训练的方法相当。

View on arXiv Download PDF AI Translation

cs.CV / 28 / 2606.03111

Inverting the Generation Process of Denoising Diffusion Implicit Models: Empirical Evaluation and a Novel Method

反转去噪扩散隐式模型的生成过程：实证评估与新方法

Zeng, Yan, Suganuma, Masanori, Okatani, Takayuki

Abstract

This paper studies the problem of inverting the DDIM image generation process to recover latent variables, particularly the initial noise map, from a generated image. Existing methods often struggle with accuracy in this task. We propose a novel hybrid approach that combines direct inversion via gradient descent for the first step, followed by a fixed-point method for subsequent steps. Empirical evaluations across three datasets demonstrate that our method significantly improves the prediction of initial latent variables while achieving superior reconstruction accuracy. Additionally, we introduce a new evaluation, called the self-interpolation test, which assesses the quality of images generated from interpolated points between the true and predicted latent maps, offering deeper insights into performance. Our results reveal that while existing methods perform reasonably well in reconstruction, they consistently fail to accurately predict the initial latent variables, resulting in poor performance on the self-interpolation test. In contrast, our method outperforms all others across all metrics, providing valuable insights into diffusion models and enhancing their applications in image generation and editing.

Chinese Translation

本文研究了反转去噪扩散隐式模型（DDIM）图像生成过程以从生成图像中恢复潜变量，特别是初始噪声图的难题。现有方法在这一任务中往往难以达到准确性。我们提出了一种新颖的混合方法，结合了通过梯度下降进行的直接反转作为第一步，随后采用固定点方法进行后续步骤。对三个数据集的实证评估表明，我们的方法显著提高了初始潜变量的预测，同时实现了更优的重建准确性。此外，我们引入了一种新的评估方法，称为自插值测试，用于评估从真实和预测潜图之间插值点生成的图像质量，从而提供更深入的性能洞察。我们的结果显示，尽管现有方法在重建方面表现尚可，但它们在准确预测初始潜变量方面始终存在不足，导致在自插值测试中表现不佳。相比之下，我们的方法在所有指标上均优于其他方法，为扩散模型提供了有价值的见解，并增强了其在图像生成和编辑中的应用。

View on arXiv Download PDF AI Translation

cs.CV / 29 / 2606.03114

FAF-CD: Frequency-Aware Fusion for Change Detection under Imperfect Multimodal Remote Sensing

FAF-CD：针对不完美多模态遥感的频率感知融合变化检测

Wang, Yufan, Makrogiannis, Sokratis, Kambhamettu, Chandra

Abstract

Remote sensing change detection for real-world monitoring often relies on imperfect heterogeneous observations, where pre- and post-event images may be asynchronous, cross-sensor, or affected by illumination, seasonal, and modality shifts. This setting is especially challenging for EO-SAR disaster mapping, where nuisance variation can resemble structural damage. We propose FAF-CD, a frequency-aware hybrid framework with a DINOv3-pretrained ConvNeXt encoder and a linear-complexity VMamba-based decoder. Its rectification-aware tri-branch fusion module combines deformable spatial alignment with Fourier and Haar-wavelet comparisons, using adaptive gating to aggregate complementary cues across scales. On BRIGHT validation, a matched heterogeneous EO-SAR adaptation improves clean and perturbed tc-mIoU/tc-mAP over NeXt2Former-CD. FAF-CD also generalizes to binary optical CD, achieving 0.924 cF1 on LEVIR-CD and 0.955 cF1 on WHU-CD, and obtains the best average perturbed cIoU/cF1 on both binary datasets among M-CD and NeXt2Former-CD under pseudo-change-aligned stress tests. It further reduces cost by approximately 24 GFLOPs relative to NeXt2Former-CD while maintaining or improving accuracy.

Chinese Translation

实际监测中的遥感变化检测通常依赖于不完美的异构观测，其中事件前后的图像可能存在异步、跨传感器或受照明、季节和模态变化的影响。这种情况对地球观测-合成孔径雷达（EO-SAR）灾害制图尤其具有挑战性，因为干扰变化可能与结构损坏相似。我们提出了FAF-CD，这是一种频率感知的混合框架，采用DINOv3预训练的ConvNeXt编码器和线性复杂度的VMamba解码器。其纠正感知的三分支融合模块结合了可变形空间对齐与傅里叶和Haar小波比较，使用自适应门控在不同尺度上聚合互补线索。在BRIGHT验证集上，匹配的异构EO-SAR适配提高了相较于NeXt2Former-CD的干净和扰动tc-mIoU/tc-mAP。FAF-CD还可以推广到二元光学变化检测，在LEVIR-CD上达到0.924 cF1，在WHU-CD上达到0.955 cF1，并在伪变化对齐压力测试中在两个二元数据集上获得M-CD和NeXt2Former-CD中最佳的平均扰动cIoU/cF1。相较于NeXt2Former-CD，它进一步减少了约24 GFLOPs的计算成本，同时保持或提高了准确性。

View on arXiv Download PDF AI Translation

cs.CV / 30 / 2606.03119

GuidedBridge: Training-freely Improving Bridge Models with Prior Guidance

GuidedBridge：无训练提升桥接模型的先验指导

Chen, Zehua, Yang, Yucheng, Yuan, Binjie, Zheng, Kaiwen, Liu, Jun S., Zhu, Jun

Abstract

Guidance methods, such as classifier-free guidance (CFG) and auto-guidance (AG), have advanced noise-to-data generation in diffusion models. Recently, bridge models have introduced a data-to-data generative process that can exploit an instructive clean prior. In this work, inspired by previous methods creating quality difference between denoising results as guidance, we propose a training-free bridge guidance method, termed Prior Guidance (PG). Specifically, we introduce a weak prior, which is unseen during bridge pre-training, hindering prior exploitation and thereby degrading denoising result. Then, we contrast it with the seen prior to highlight and enhance prior exploitation via a scaling factor. Moreover, we analyze the underlying mechanism of prior exploitation in the bridge process and design frequency-modulated prior guidance (FMPG), which tailors the guidance scale to low- and high-frequency bands coherent with bridge generative dynamics. To address prior exploitation in image in-painting, we develop a cascaded framework, CFG-FMPG, which first generates a noisy hidden representation via CFG and then exploits it as a generative prior with FMPG, fulfilling their complementary strengths without compromising inference efficiency. Experiments demonstrate that our PG methods consistently improve pre-trained bridge models across diverse image translation tasks.

Chinese Translation

指导方法，如无分类器指导（CFG）和自动指导（AG），在扩散模型中的噪声到数据生成方面取得了进展。最近，桥接模型引入了一种数据到数据的生成过程，可以利用指导性的干净先验。在本研究中，受到之前方法的启发，这些方法通过创建去噪结果之间的质量差异作为指导，我们提出了一种无训练的桥接指导方法，称为先验指导（PG）。具体而言，我们引入了一种在桥接预训练期间未见的弱先验，这阻碍了先验的利用，从而降低了去噪结果。然后，我们将其与已见先验进行对比，通过缩放因子突出并增强先验的利用。此外，我们分析了桥接过程中先验利用的潜在机制，并设计了频率调制先验指导（FMPG），该方法将指导尺度调整为与桥接生成动态一致的低频和高频带。为了解决图像修补中的先验利用问题，我们开发了一个级联框架CFG-FMPG，该框架首先通过CFG生成一个噪声隐藏表示，然后利用它作为生成先验与FMPG结合，充分发挥它们的互补优势而不影响推理效率。实验表明，我们的PG方法在多种图像翻译任务中持续提升了预训练的桥接模型。

View on arXiv Download PDF AI Translation

cs.CV / 31 / 2606.03120

KC-3DGS: Kurtosis-Constrained Gaussian Splatting for High-Fidelity View Synthesis

KC-3DGS：基于峰度约束的高保真视图合成的高斯点云技术

Banerjee, Vivekjyoti, Yadav, Abhay, Chellappa, Rama, Roy, Aniket

Abstract

3D Gaussian Splatting (3DGS) enables real-time novel view synthesis by representing scenes as collections of anisotropic Gaussians optimized via differentiable rasterization. However, standard pixel-space losses (L1, SSIM) constrain only aggregate reconstruction error, permitting the optimization to redistribute error across frequency scales. This leads to oversmoothing and structural artifacts, particularly in sparse-view settings where supervision is limited. We propose KC-3DGS, which augments 3DGS training with wavelet-domain supervision based on natural image statistics. Our method combines three components: (1) a multi-scale wavelet coefficient alignment loss that explicitly penalizes missing high-frequency detail, (2) a supervised kurtosis concentration loss that encourages rendered images to match the heavy-tailed frequency statistics of ground-truth images, and (3) a cross-band covariance penalty that promotes frequency specialization. We provide theoretical analysis showing that pixel-space losses admit a family of indistinguishable perturbations under wavelet redistribution, and that our joint objective excludes degenerate solutions. Experiments across MipNeRF360, Tanks&Temples, MVImgNet, DeepBlending, and WRIVA-ULTRRA demonstrate consistent improvements in perceptual quality. On the challenging WRIVA-ULTRRA outdoor dataset, KC-3DGS achieves a 9.48% improvement in DreamSim while also improving PSNR, SSIM, and LPIPS. In sparse-view settings with only 12 training images, our method improves PSNR by up to 0.5 dB on MipNeRF360 while maintaining perceptual quality. The approach integrates seamlessly into existing 3DGS pipelines as a plug-and-play regularization strategy.

Chinese Translation

3D高斯点云技术（3DGS）通过将场景表示为经过可微光栅化优化的各向异性高斯集合，实现了实时新视图合成。然而，标准的像素空间损失（L1，SSIM）仅限制了总体重建误差，允许优化在频率尺度之间重新分配误差。这导致了过度平滑和结构伪影，特别是在监督有限的稀疏视图设置中。我们提出了KC-3DGS，它通过基于自然图像统计的波形域监督增强了3DGS训练。我们的方法结合了三个组件：（1）多尺度小波系数对齐损失，明确惩罚缺失的高频细节；（2）监督的峰度集中损失，鼓励渲染图像与真实图像的重尾频率统计相匹配；（3）跨频带协方差惩罚，促进频率专业化。我们提供了理论分析，表明像素空间损失在小波重分配下允许一系列不可区分的扰动，并且我们的联合目标排除了退化解。在MipNeRF360、Tanks&Temples、MVImgNet、DeepBlending和WRIVA-ULTRRA等数据集上的实验表明感知质量的一致改善。在具有挑战性的WRIVA-ULTRRA户外数据集中，KC-3DGS在DreamSim上实现了9.48%的提升，同时改善了PSNR、SSIM和LPIPS。在仅有12张训练图像的稀疏视图设置中，我们的方法在MipNeRF360上将PSNR提高了最多0.5 dB，同时保持了感知质量。该方法可以无缝集成到现有的3DGS管道中，作为一种即插即用的正则化策略。

View on arXiv Download PDF AI Translation

cs.CV / 32 / 2606.03142

Disentangling Visual and Factual Correctness in LVLMs' Visualization Literacy

解构大型视觉语言模型（LVLMs）中的视觉正确性与事实正确性

Lee, Soohyun, Kim, Jaeyoung, Park, Seokhyeon, Lee, Sihyeon, Song, Jiwon, Kim, Bohyoung, Song, Hyunjoo, Seo, Jinwook

Abstract

Large Vision-Language Models (LVLMs) show strong visualization interpretation, yet it is unclear whether their responses reflect genuine reasoning over visual evidence or factual priors learned during training. Current evaluations mix these two sources, obscuring when correct visual interpretation is overridden by memorized facts. We present a framework that isolates visual correctness from factual correctness, revealing validity limitations in existing visualization literacy assessments. Across three experiments with 15 state-of-the-art LVLMs: (1) several models reach human-level performance on standard tests (VLAT), but this may reflect factual recall rather than visual understanding, while randomized-data tests (reVLAT) underestimate literacy when correct visual interpretation is superseded by factual priors. (2) Using our Counterfactual Visualization Literacy Assessment Test (CVLAT) with capability-normalized arbitration metrics, we classify models by the sign of their visual-factual reliance index (VFRI), revealing a visualization-oriented majority and a factual knowledge-oriented minority, though several near-zero cases warrant caution. A human baseline (N=30) on the same counterfactual items confirms that people overwhelmingly follow the chart under conflict, providing a human reference point. (3) Prompt-based intervention can shift prioritization, but its effectiveness is highly model-dependent and direction-asymmetric, and high chart-reading capability does not predict prompt-controllability. Overall, high visualization accuracy is not sufficient evidence of faithful visual reasoning: reliable integration into visual analytics requires evaluating not only visualization literacy but also how models arbitrate between visual evidence and factual priors when the two diverge. Benchmark and code: https://github.com/JaeyoungKim-HCIL/CVLAT

Chinese Translation

大型视觉语言模型（LVLMs）展现出强大的视觉解释能力，但尚不清楚其反应是否真正反映了对视觉证据的推理，还是训练过程中学习到的事实先验。当前的评估混合了这两种来源，模糊了何时正确的视觉解释被记忆的事实所覆盖。我们提出了一个框架，将视觉正确性与事实正确性分离，揭示了现有视觉素养评估中的有效性局限性。在对15个最先进的LVLM进行的三项实验中：（1）一些模型在标准测试（VLAT）中达到了人类水平的表现，但这可能反映的是事实回忆而非视觉理解，而随机数据测试（reVLAT）在正确的视觉解释被事实先验所取代时低估了素养。（2）使用我们的反事实视觉素养评估测试（CVLAT）和能力标准化仲裁指标，我们根据视觉-事实依赖指数（VFRI）的符号对模型进行分类，揭示了以视觉为导向的多数和以事实知识为导向的少数，尽管几个接近零的案例需要谨慎对待。对相同反事实项目的人类基线（N=30）确认了人们在冲突中压倒性地遵循图表，提供了人类参考点。（3）基于提示的干预可以改变优先级，但其有效性高度依赖于模型，并且方向不对称，高图表阅读能力并不能预测提示可控性。总体而言，高视觉准确性并不足以证明忠实的视觉推理：可靠地融入视觉分析需要评估模型在视觉证据与事实先验发生分歧时如何仲裁。基准和代码：https://github.com/JaeyoungKim-HCIL/CVLAT

View on arXiv Download PDF AI Translation

cs.CV / 33 / 2606.03148

$A^2$: Smaller Self-Supervised ViTs Localize Better than Larger Ones

$A^2$: 较小的自监督视觉变换器比较大的更能有效定位

Rammohan, Sreehari, Ha, Huy, Vondrick, Carl

Abstract

Robust visual classification often depends on localizing the main foreground objects in an image while ignoring contextual distractors. Surprisingly, we find that the attention maps of smaller self-supervised ViTs localize foreground objects better than those of larger ViTs. However, we still need large ViTs, because they extract richer representations from each patch. To get the best of both worlds, good localization and rich representations, we propose $A^2$, a simple method that leverages this inverse scaling finding by decoupling where to look (a small attention model) from what to extract (a large embedding model): we crop around the attention peaks of a small model and embed the crops with a larger model. $A^2$ uses entirely pretrained features, requires no group labels, and does not require per-dataset attention or backbone training. Across 5 benchmarks, $A^2$ is competitive with backbone-matched loss-level methods like DFR, and outperforms end-to-end attention training under stronger distribution shifts.

Chinese Translation

稳健的视觉分类通常依赖于在图像中定位主要的前景物体，同时忽略背景干扰物。令人惊讶的是，我们发现较小的自监督视觉变换器（ViTs）的注意力图比较大的ViTs更能有效地定位前景物体。然而，我们仍然需要较大的ViTs，因为它们从每个图像块中提取了更丰富的表示。为了兼顾良好的定位和丰富的表示，我们提出了$A^2$，这是一种简单的方法，通过将关注点（一个小的注意力模型）与提取内容（一个大的嵌入模型）解耦，利用这一反向缩放的发现：我们围绕小模型的注意力峰值进行裁剪，并使用较大的模型对裁剪区域进行嵌入。$A^2$完全使用预训练特征，不需要组标签，也不需要针对每个数据集的注意力或主干网络训练。在5个基准测试中，$A^2$与基于主干匹配的损失水平方法如DFR具有竞争力，并在更强的分布转移下优于端到端的注意力训练。

View on arXiv Download PDF AI Translation

cs.CV / 34 / 2606.03159

NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation

NVIDIA OmniDreams：用于闭环自主车辆仿真的实时生成世界模型

NVIDIA, :, Basant, Aarti, Kar, Amlan, Paschalidou, Despoina, Wei, Fangyin, Ferroni, Francesco, Cobo, Guillermo Garcia, Turki, Haithem, Ling, Huan, Seo, Jaewoo, Lucas, James, Wu, Jay Zhangjie, Wang, Jialiang, Lorraine, Jonathan, Gao, Jun, He, Kai, Tothova, Katarina, Xie, Kevin, Tyszkiewicz, Michał, Wu, Qi, de Lutio, Riccardo, Li, Ruilong, Fidler, Sanja, Kim, Seung Wook, Shen, Tianchang, Cao, Tianshi, Pfaff, Tobias, Lew, William, Wu, Xindi, Ren, Xuanchi, Lu, Yifan, Zhang, Yuxuan, Gojcic, Zan, Wang, Zian

Abstract

As autonomous vehicle capabilities advance, the safe evaluation of driving policies in long-tail scenarios remains a critical bottleneck. In closed-loop simulation, the driving policy model actively interacts with the environment, where its actions dynamically update the simulator state and directly influence the next set of generated sensor observations. While recent reconstruction-based neural simulators offer photorealism, they are fundamentally constrained by their initial captured data and struggle to generalize to highly dynamic or novel scenes. To overcome these limitations, we introduce OmniDreams, a foundation generative world model mid- and post-trained from the Cosmos diffusion model to autoregressively generate action-conditioned videos in real time. By leveraging the rich visual priors of Cosmos and mid- and post-training on 21k hours of driving scenarios, OmniDreams synthesizes complex, unobserved phenomena that are hard for traditional simulators to capture, such as extreme weather and unpredictable dynamic agent behaviors. Crucially, it autoregressively conditions its photorealistic sensor generation on past frames, the current simulator state, and immediate driving actions. Deployed in a closed-loop system with the Alpamayo 1 policy model and AlpaSim orchestrator, OmniDreams acts as a highly responsive, reactive environment, providing a scalable and comprehensive solution for training and evaluating next-generation autonomous driving policies. We additionally show preliminary results indicating that a world-action model (WAM) post-trained from OmniDreams achieves strong performance on the Physical AI Autonomous Vehicles NuRec dataset, surpassing the VLA-based Alpamayo 1.5 research policy model while using only 1/5 the total parameters. These results highlight the potential for a real-time world model like OmniDreams to also serve as a backbone for policy architectures.

Chinese Translation

随着自主车辆能力的提升，在长尾场景中安全评估驾驶策略仍然是一个关键瓶颈。在闭环仿真中，驾驶策略模型与环境积极互动，其动作动态更新仿真器状态，并直接影响下一组生成的传感器观测。尽管近期基于重建的神经仿真器提供了照片级真实感，但它们在根本上受到初始捕获数据的限制，难以推广到高度动态或新颖的场景。为了解决这些局限性，我们引入了OmniDreams，一个基于Cosmos扩散模型中后期训练的基础生成世界模型，能够实时自回归生成动作条件的视频。通过利用Cosmos丰富的视觉先验，并在21,000小时的驾驶场景上进行中后期训练，OmniDreams合成了复杂的、传统仿真器难以捕捉的未观察现象，如极端天气和不可预测的动态代理行为。至关重要的是，它自回归地将其照片级真实感的传感器生成条件于过去的帧、当前的仿真器状态和即时驾驶动作。在与Alpamayo 1政策模型和AlpaSim编排器的闭环系统中部署后，OmniDreams作为一个高度响应和反应的环境，提供了一种可扩展和全面的解决方案，用于训练和评估下一代自主驾驶策略。此外，我们还展示了初步结果，表明从OmniDreams后期训练的世界-动作模型（WAM）在Physical AI自主车辆NuRec数据集上表现出色，超越了基于VLA的Alpamayo 1.5研究政策模型，同时仅使用了总参数的1/5。这些结果突显了像OmniDreams这样的实时世界模型作为政策架构支撑的潜力。

View on arXiv Download PDF AI Translation

cs.CV / 35 / 2606.03160

SRENet: Spectral Re-Entry Network for Point Cloud Action Recognition

SRENet：用于点云动作识别的谱重入网络

Wu, Qiuxia, Lan, Jiarui, Kang, Wenxiong, Wang, Zhiyong, Hu, Kun

Abstract

Recognizing human actions from point cloud sequences is critical for 3D perception driven applications such as autonomous driving and human-computer interaction. However, the irregular structure and temporal inconsistency of point clouds pose unique challenges for spatio-temporal representation learning, especially in capturing both global motion context and fine-grained temporal dynamics. We propose SRENet, a spectral-aware framework designed to explicitly learn both global context and fine-grained temporal dynamics of motion from a frequency perspective for action recognition. SRENet introduces a Spectral Decomposition Block (SDeBlock) that performs wavelet-based analysis along temporal and spatial axes, disentangling features into low- and high-frequency components with frequency-specific attention. To recover residual dynamics and re-align temporal frequency structures distorted during semantic fusion, a Spectral Re-entry Block (SReBlock) performs secondary temporal decomposition. Furthermore, a spectral-aware learning strategy is devised to enhance discriminability in both frequency subspaces via contrastive loss and a curriculum schedule that gradually shifts focus from low- to high-frequency spaces in line with coarse to detailed motion patterns. Extensive experiments on MSR-Action3D, NTU-RGBD and NTU-RGBD120 demonstrate that SRENet achieves state-of-the-art performance, validating the effectiveness of frequency modeling in point cloud-based action understanding.

Chinese Translation

从点云序列中识别人体动作对于自动驾驶和人机交互等3D感知驱动的应用至关重要。然而，点云的不规则结构和时间不一致性给时空表示学习带来了独特的挑战，尤其是在捕捉全局运动上下文和细粒度时间动态方面。我们提出了SRENet，一种谱感知框架，旨在从频率角度明确学习动作的全局上下文和细粒度时间动态。SRENet引入了一个谱分解块（Spectral Decomposition Block, SDeBlock），该块沿时间和空间轴进行基于小波的分析，将特征解耦为低频和高频成分，并应用频率特定的注意力机制。为了恢复残余动态并重新对齐在语义融合过程中失真的时间频率结构，谱重入块（Spectral Re-entry Block, SReBlock）执行二次时间分解。此外，设计了一种谱感知学习策略，通过对比损失和逐步调整的课程安排，增强频率子空间中的可区分性，逐渐将注意力从低频空间转移到高频空间，以符合从粗到细的运动模式。在MSR-Action3D、NTU-RGBD和NTU-RGBD120上的大量实验表明，SRENet达到了最先进的性能，验证了频率建模在基于点云的动作理解中的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 36 / 2606.03168

JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation

JAVEDIT：基于指令的联合音视频编辑与智能数据管理

Chen, Yinan, Lin, Chuming, Chen, Zhennan, Zeng, Yuxiang, Zhu, Junwei, Bi, Yali, Huang, Xijie, Xu, Chengming, Luo, Donghao, Xue, Zhucun, Hu, Xiaobin, Wang, Chengjie, Liu, Yong, Zhang, Jiangning, Yan, Shuicheng

Abstract

While instruction-based video editing has seen significant progress, joint audio-visual editing remains constrained by the absence of dedicated datasets and benchmarks. To bridge this gap, we present JAVEdit-100k, the first large-scale, high-quality dataset tailored for instruction-guided joint audio-visual editing. Focusing on human-centric videos, JAVEdit-100k comprises approximately 100K editing triplets spanning five distinct categories, including subject editing and speech editing. This dataset is rigorously constructed via four meticulously designed generation pipelines, seamlessly paired with an agent-in-the-loop quality control mechanism. Furthermore, to address the lack of standardized evaluation within the field, we introduce JAVEditBench, a comprehensive benchmark featuring curated source videos and human-aligned instructions across all editing categories. Finally, we propose JAVEdit, a pioneering baseline model for instruction-guided joint audio-visual editing. Experiments show that \model\ outperforms all baselines on five of six evaluation metrics.

Chinese Translation

尽管基于指令的视频编辑已取得显著进展，但联合音视频编辑仍受限于缺乏专门的数据集和基准。为填补这一空白，我们提出了JAVEdit-100k，这是第一个针对基于指令的联合音视频编辑量身定制的大规模高质量数据集。JAVEdit-100k专注于以人为中心的视频，包含约10万对编辑三元组，涵盖了包括主体编辑和语音编辑在内的五个不同类别。该数据集通过四个精心设计的生成管道严格构建，并与一个智能控制机制无缝配对。此外，为了解决该领域缺乏标准化评估的问题，我们引入了JAVEditBench，这是一个全面的基准，包含经过筛选的源视频和与人类对齐的指令，涵盖所有编辑类别。最后，我们提出了JAVEdit，这是一个开创性的基线模型，用于基于指令的联合音视频编辑。实验表明， extit{model}在六个评估指标中的五个上优于所有基线。

View on arXiv Download PDF AI Translation

cs.CV / 37 / 2606.03175

Ask When It Pays: Cost-Aware Open-Ended Interaction for Instance Goal Navigation

问问何时有利：面向实例目标导航的成本感知开放式互动

Zhao, Xunyi, Lin, Sihao, Zhou, Gengze, Li, Zerui, Li, Shijie, Tao, Wei, Liu, Jiajun, Wu, Qi

Abstract

Instance Goal Navigation (IGN) requires an embodied agent to find a specific object instance among distractors from an underspecified natural-language description. Such ambiguity often cannot be resolved from perception and language alone, making interaction with an oracle a natural mechanism for disambiguation. Prior interactive methods allow oracle queries but treat lightweight clarification and route-level guidance alike, letting agents boost success rate through repeated high-information questions rather than by resolving the underlying ambiguity efficiently. We recast interactive IGN as a cost-sensitive uncertainty-reduction problem, where the agent should ask the question whose answer provides the largest reduction in navigation uncertainty relative to its penalty. To this end, we apply an information-gain analysis on existing navigation corpora to identify which cues reduce navigation uncertainty, yielding a compact set of question types and data-derived weights.However, existing interactive navigation benchmarks do not model the cost of different question types or evaluate how efficiently agents use interaction, making them unsuitable for studying cost-sensitive interaction. Based on this taxonomy, we construct a benchmark for diagnosing interaction behavior and efficiency, together with a Weighted Success Rate metric that penalizes each query by its derived cost. We further propose a zero-shot MLLM navigator that selectively queries at each decision step only when the expected uncertainty reduction justifies the interaction cost.

Chinese Translation

实例目标导航（IGN）要求具身智能体在不明确的自然语言描述中找到特定的物体实例，且需在干扰物中进行识别。这种模糊性往往无法仅通过感知和语言来解决，因此与神谕者的互动成为一种自然的消歧机制。以往的互动方法允许神谕查询，但将轻量级的澄清与路线级指导视为相同，导致智能体通过反复提出高信息量的问题来提高成功率，而不是有效地解决潜在的模糊性。我们将互动式IGN重新表述为一个成本敏感的不确定性减少问题，其中智能体应询问那些其答案能相对于惩罚提供最大导航不确定性减少的问题。为此，我们对现有导航语料库应用信息增益分析，以识别哪些线索能减少导航不确定性，从而得出一组紧凑的问题类型及数据导出的权重。然而，现有的互动导航基准并未建模不同问题类型的成本或评估智能体使用互动的效率，使其不适合研究成本敏感的互动。基于这一分类法，我们构建了一个用于诊断互动行为和效率的基准，并提出了一种加权成功率指标，该指标根据每个查询的衍生成本进行惩罚。我们进一步提出了一种零样本多语言模型导航器，该导航器仅在每个决策步骤中，当预期的不确定性减少足以证明互动成本的合理性时，才选择性地进行查询。

View on arXiv Download PDF AI Translation

cs.CV / 38 / 2606.03180

GLINT: Sparsely Gated Vision-Language Alignment for Fine-Grained Radiology Representations

GLINT：用于细粒度放射学表征的稀疏门控视觉-语言对齐

Park, Jonggwon, Lee, Seongeun, Park, Junhyun, Yun, Hannah, Kim, Hyunwoong, Jeong, Sohyun, Kang, Hyewon, Yoon, Byungmu, Choi, Kyoyun

Abstract

Vision-language models (VLMs) for radiology have emerged as a scalable paradigm by leveraging image-report pairs naturally produced in clinical workflows. However, this pairing reveals a mismatch in scale: each finding occupies only a small region of the image, yet supervision is provided only at the global image-report level. This poses a central challenge: prior approaches spread weight densely across all patches rather than concentrating on the sparse subset relevant to a given query. To address this, we present GLINT (Gated Language-Image alignmeNT), a framework that explicitly models this sparse correspondence. On the alignment side, we introduce Sparsely Gated Alignment, a novel architecture in which a sigmoid gate over a separate gate embedding space activates only the patches relevant to each textual query, enforcing explicit sparsity. On the representation side, we add Dense Feature Regularization, which anchors the trainable encoder's intermediate features to a frozen self-supervised learning (SSL) teacher, preserving the fine-grained patch features that the gate relies on. The same recipe applies to both 2D chest X-ray (CXR) and 3D chest computed tomography (CT), built with DINOv3 and V-JEPA 2.1, respectively. GLINT enables zero-shot classification, grounding, and segmentation from free-text queries, and to our knowledge is the first to demonstrate zero-shot segmentation on 3D CT volumes without mask supervision. Notably, the most pronounced gains arise on zero-shot grounding and segmentation, where sparse, query-specific localization is required, consistent with our design intent. In downstream evaluation, GLINT outperforms both SSL encoders and medical VLMs on classification, report generation, and segmentation.

Chinese Translation

放射学的视觉-语言模型（VLMs）通过利用在临床工作流程中自然产生的图像-报告对，已成为一种可扩展的范式。然而，这种配对揭示了尺度的不匹配：每个发现仅占图像的一小部分区域，而监督仅在全局图像-报告层面提供。这提出了一个核心挑战：以往的方法在所有图像块上均匀分配权重，而不是集中在与特定查询相关的稀疏子集上。为了解决这个问题，我们提出了GLINT（Gated Language-Image alignmeNT），一个明确建模这种稀疏对应关系的框架。在对齐方面，我们引入了稀疏门控对齐，这是一种新颖的架构，其中一个sigmoid门在一个单独的门嵌入空间上激活仅与每个文本查询相关的图像块，从而强制实现显式稀疏性。在表征方面，我们添加了密集特征正则化，将可训练编码器的中间特征锚定到一个冻结的自监督学习（SSL）教师上，保留了门控所依赖的细粒度图像块特征。相同的方法适用于2D胸部X光（CXR）和3D胸部计算机断层扫描（CT），分别使用DINOv3和V-JEPA 2.1构建。GLINT能够从自由文本查询中实现零样本分类、定位和分割，且据我们所知，首次在没有掩膜监督的情况下展示了3D CT体积的零样本分割。值得注意的是，最显著的提升出现在零样本定位和分割上，这里需要稀疏的、特定查询的定位，符合我们的设计意图。在下游评估中，GLINT在分类、报告生成和分割任务上超越了SSL编码器和医学VLMs。

View on arXiv Download PDF AI Translation

cs.CV / 39 / 2606.03201

Reinforcement Learning from Cross-domain Videos with Video Prediction Model

基于视频预测模型的跨领域视频强化学习

Yang, Zhao, Zu, Xinrui, Kooi, Jacob E., Delliaux, Thomas, Liu, He, Yu, Shujian, Luck, Kevin Sebastian, François-Lavet, Vincent

Abstract

Reinforcement learning from expert videos across visually distinct domains is challenging due to the absence of reward signals and the presence of domain gaps. We introduce XIPER (Cross-domain Video Prediction Reward), a reward model for learning from expert videos collected in a visually different domain, where the agent's appearance differs due to factors such as color, morphology, or the sim-to-real gap. More specifically, XIPER trains a cross-domain video prediction model that maps agent observations into the expert domain and uses the prediction likelihood as a reward signal. Experiments on the DMC Color Suite (8 tasks) and DMC Body Suite (3 tasks) show that XIPER consistently outperforms baselines despite domain gaps such as differences in agent color and morphology. We further analyze XIPER on a sim-to-real transfer dataset, demonstrating that it produces meaningful reward signals for real-robot observations given only simulated expert videos. Code, pretrained models, datasets and video demonstrations can be found on our project webpage: https://sites.google.com/view/xiper

Chinese Translation

从视觉上明显不同领域的专家视频中进行强化学习具有挑战性，因为缺乏奖励信号且存在领域间差距。我们提出了 XIPER（跨领域视频预测奖励），这是一种用于从收集于视觉上不同领域的专家视频中学习的奖励模型，其中代理的外观因颜色、形态或模拟到真实（sim-to-real）差距等因素而有所不同。更具体地说，XIPER 训练一个跨领域视频预测模型，将代理观察映射到专家领域，并使用预测的可能性作为奖励信号。在 DMC Color Suite（8 个任务）和 DMC Body Suite（3 个任务）上的实验表明，尽管存在代理颜色和形态等领域差距，XIPER 始终优于基线方法。我们进一步在一个模拟到真实转移数据集上分析了 XIPER，证明它在仅提供模拟专家视频的情况下，为真实机器人观察生成了有意义的奖励信号。代码、预训练模型、数据集和视频演示可以在我们的项目网页上找到：https://sites.google.com/view/xiper

View on arXiv Download PDF AI Translation

cs.CV / 40 / 2606.03216

Follow-Your-Preference++: Rethinking Preference Alignment for Image Inpainting

Follow-Your-Preference++：重新思考图像修复中的偏好对齐

Yuan, Junkun, Shen, Yutao, Aonishi, Toru, Nakayama, Hideki, Ma, Yue

Abstract

We study preference alignment for image inpainting. Rather than proposing yet another method, we revisit the problem from first principles and reassess its core challenges. We adopt the widely used direct preference optimization framework and construct preference training data with publicly available reward models. Our empirical study spans nine reward models, two benchmarks, and two baseline inpainting models that differ in architecture and generative mechanism. Our main findings are: (1) Most reward models provide valid signals for preference data construction, although some are unreliable as evaluators. (2) Across models and benchmarks, preference data exhibits consistent trends under both candidate and sample scaling. (3) Reward models display pronounced biases--particularly in brightness, composition, and color scheme--that make them prone to inducing reward hacking. (4) A simple ensemble of reward models mitigates such biases and yields robust, generalizable performance. {\color{rebuttal_blue}(5) Preference alignment is transferable to the object removal task, where the goal shifts from open-ended creative generation to coherent background completion. (6) Further analysis reveals that a calibrated ensemble method further mitigates hacking and improves robustness.} Without modifying model architectures or introducing additional datasets, our models substantially outperform prior state-of-the-art models on standard metrics, large vision-language model evaluations, and human assessments. Our code is available at: https://github.com/shenytzzz/Follow-Your-Preference.

Chinese Translation

我们研究了图像修复中的偏好对齐问题。我们并未提出另一种方法，而是从基本原理出发，重新审视这一问题及其核心挑战。我们采用广泛使用的直接偏好优化框架，并利用公开可用的奖励模型构建偏好训练数据。我们的实证研究涵盖了九种奖励模型、两个基准和两种在架构和生成机制上有所不同的基线修复模型。我们的主要发现包括：（1）大多数奖励模型为偏好数据构建提供了有效信号，尽管有些模型作为评估者并不可靠。（2）在不同模型和基准下，偏好数据在候选和样本扩展时表现出一致的趋势。（3）奖励模型显示出明显的偏见，尤其是在亮度、构图和色彩方案方面，这使得它们容易导致奖励操控。（4）简单的奖励模型集成可以减轻这些偏见，并产生稳健且具有可推广性的性能。{ extcolor{rebuttal_blue}(5) 偏好对齐可以转移到物体移除任务中，其目标从开放式创造生成转变为连贯的背景补全。（6）进一步分析表明，经过校准的集成方法进一步减轻了操控现象并提高了稳健性。} 在不修改模型架构或引入额外数据集的情况下，我们的模型在标准指标、大型视觉-语言模型评估和人类评估中显著超越了先前的最先进模型。我们的代码可在以下网址获取：https://github.com/shenytzzz/Follow-Your-Preference。

View on arXiv Download PDF AI Translation

cs.CV / 41 / 2606.03243

MemoGen: Can Past Experience Improve Future Text-to-Image Generation?

MemoGen：过去的经验能否改善未来的文本到图像生成？

Chen, Wenshuo, Yu, Kuimou, Tian, Bowen, Song, Jianfei, Liang, Shaofeng, Jia, Haozhe, Cheng, Kan, Li, Haosen, Yuan, Kaishen, Wang, Lei, Wu, Jiemin, Lai, Songning, Yue, Yutao

Abstract

Modern text-to-image models have achieved strong visual synthesis, yet remain unreliable when prompts require implicit visual constraints, relational reasoning, or external knowledge. Existing retrieval-augmented and agentic generation methods mitigate this issue by acquiring external knowledge, references, or refined prompts for the current request, yet they typically treat each generation as an isolated episode and do not systematically preserve past successes or failures for future use. In this work, we ask whether a text-to-image system can continually improve from its own generation experience without updating the underlying generator. We propose MemoGen, a training-free framework that augments existing image generators with an agentic evolution layer. For each task, MemoGen explicitly infers visual requirements, retrieves external evidence and references when necessary, translates them into executable generation constraints, evaluates the generated result, and stores task understanding, reference choices, visual feedback, successful strategies, and failure lessons as reusable experience memory. Across evolution rounds, the agent retrieves relevant experience to improve similar future generations, selectively repairing previously failed cases while preserving successful ones, thereby enabling test-time self-evolution without parameter updates. Extensive experiments on knowledge-intensive and reasoning-oriented benchmarks demonstrate the effectiveness of this paradigm: after only two evolution rounds, MemoGen built upon the open-source Qwen-Image backbone surpasses strong proprietary systems such as Nano Banana Pro and GPT-Image-1 on WISE and Mind-Bench, showing that explicit experience memory can serve as a powerful continual learning signal for reliable text-to-image generation.

Chinese Translation

现代文本到图像模型在视觉合成方面取得了显著进展，但在需要隐含视觉约束、关系推理或外部知识的提示下仍然不可靠。现有的检索增强和自主生成方法通过获取外部知识、参考资料或为当前请求精炼的提示来缓解这一问题，但它们通常将每次生成视为孤立的事件，并未系统性地保留过去的成功或失败以供未来使用。在本研究中，我们探讨一个文本到图像系统是否可以在不更新基础生成器的情况下，持续从自身的生成经验中改进。我们提出了MemoGen，一个无训练的框架，通过一个自主演化层增强现有的图像生成器。对于每个任务，MemoGen明确推断视觉需求，在必要时检索外部证据和参考资料，将其转化为可执行的生成约束，评估生成结果，并将任务理解、参考选择、视觉反馈、成功策略和失败教训存储为可重用的经验记忆。在演化轮次中，代理检索相关经验以改善类似的未来生成，选择性地修复先前失败的案例，同时保留成功的案例，从而实现测试时的自我演化而无需参数更新。在知识密集型和推理导向的基准测试中进行的广泛实验表明了这一范式的有效性：仅经过两轮演化，基于开源Qwen-Image骨干网的MemoGen在WISE和Mind-Bench上超越了强大的专有系统，如Nano Banana Pro和GPT-Image-1，显示出显式经验记忆可以作为可靠的文本到图像生成的强大持续学习信号。

View on arXiv Download PDF AI Translation

cs.CV / 42 / 2606.03246

MariData: One-Step Unpaired Image Translation for Maritime Environments

MariData：针对海洋环境的一步无配对图像翻译

Henriksson, Santeri, Asadi, Mehdi, Majd, Amin, Kalliovaara, Juha

Abstract

The development on robust perception systems for Maritime Autonomous Surface Ships (MASS) is heavily constrained by the scarcity of diverse training data, particularly for adverse weather and low-light conditions. Because collecting paired images in dynamic maritime environments is physically impossible, synthetic data generation via unpaired image-to-image translation offers a critical solution. However, existing generative models suffer from failing to preserve the fine structural details of small navigational objects due to latent compression bottlenecks. In this paper, we introduce a framework for generating synthetic maritime data using CycleGAN-turbo, a one-step unpaired translation architecture. By incorporating zero-convolution skip connections to bypass the Variational Autoencoder (VAE) bottleneck, our approach explicitly preserves small object details (e.g., distant vessels and sea marks) during translation. We compiled a dataset of 7,000 maritime images to train and evaluate models for Day-to-Foggy, Day-to-Sunset, and Day-to-Night domain translations. Qualitative evaluations and variable-strength inference studies demonstrate that our method effectively synthesizes realistic atmospheric conditions while maintaining the underlying semantic structure of the scene. The Day-to-Foggy and Day-to-Sunset models exhibit great structural retention, whereas the Day-to-Night model highlights the challenge of semantic hallucination, such as generating artificial coastal lights, induced by unbalanced training distributions. Ultimately, this work establishes an efficient, structure-aware data synthesis pipeline that directly addresses the data scarcity bottleneck in autonomous maritime navigation.

Chinese Translation

海洋自主水面船舶（MASS）的稳健感知系统的发展受到多样化训练数据稀缺的严重限制，尤其是在恶劣天气和低光照条件下。由于在动态海洋环境中收集配对图像在物理上是不可能的，因此通过无配对图像到图像的翻译生成合成数据提供了一个关键解决方案。然而，现有的生成模型由于潜在压缩瓶颈，无法保留小型导航物体的细微结构细节。本文提出了一种使用CycleGAN-turbo（一种一步无配对翻译架构）生成合成海洋数据的框架。通过引入零卷积跳跃连接以绕过变分自编码器（VAE）瓶颈，我们的方法在翻译过程中明确保留了小物体细节（例如，远处的船只和海标）。我们编制了一个包含7000张海洋图像的数据集，以训练和评估白天到雾天、白天到日落和白天到夜晚的领域翻译模型。定性评估和可变强度推断研究表明，我们的方法有效合成了逼真的大气条件，同时保持了场景的基本语义结构。白天到雾天和白天到日落模型表现出良好的结构保留，而白天到夜晚模型则突显了语义幻觉的挑战，例如生成由不平衡训练分布引起的人造海岸灯光。最终，这项工作建立了一个高效的、结构感知的数据合成管道，直接解决了自主海洋导航中的数据稀缺瓶颈。

View on arXiv Download PDF AI Translation

cs.CV / 43 / 2606.03254

FreeStreamGS: Online Feed-forward 3D Gaussian Splatting from Unposed Streaming Inputs

FreeStreamGS：来自无姿态流输入的在线前馈3D高斯点云渲染

Chen, Ruiyang, Li, Feiran, Zhou, Chu, Li, Zonglin, Ma, Zhanyu, Guo, Heng

Abstract

Feed-forward 3D Gaussian Splatting (3DGS) allows efficient and high-fidelity novel view synthesis (NVS) from an offline recorded image sequence. However, achieving online NVS from streaming and unposed image inputs remains challenging. Although online feed-forward geometric estimation methods have been proposed for streaming depth and point cloud recovery, they cannot be adapted to NVS due to severe rendering artifacts. This is because NVS demands stricter multi-view consistency in Gaussian scales and pose-geometry alignment; even minor deviations would accumulate over time and visibly degrade rendering quality. To this end, we propose FreeStreamGS, a robust online feed-forward framework for efficient and high-quality NVS. We introduce two key mechanisms: a Decoupled Intrinsic Recovery Head that removes cumulative camera intrinsic bias and prevents scene scale jitter during long-term streaming, and a Dynamic Point Refinement Offset strategy that relaxes rigid unprojection to correct coupled pose-depth drift. Extensive experiments show that FreeStreamGS achieves rendering quality competitive with state-of-the-art offline feed-forward 3DGS methods, despite operating without access to future frames.

Chinese Translation

前馈3D高斯点云渲染（3DGS）能够从离线录制的图像序列中高效且高保真地合成新视角图像（NVS）。然而，从流式和无姿态图像输入中实现在线NVS仍然具有挑战性。尽管已有针对流式深度和点云恢复的在线前馈几何估计方法被提出，但由于严重的渲染伪影，这些方法无法适应NVS。这是因为NVS对高斯尺度和姿态几何对齐的多视图一致性要求更为严格；即使是微小的偏差也会随着时间的推移累积并明显降低渲染质量。为此，我们提出了FreeStreamGS，一个稳健的在线前馈框架，用于高效且高质量的NVS。我们引入了两个关键机制：一个解耦的内参恢复头，消除累积的相机内参偏差，并在长期流式传输中防止场景尺度抖动；以及一个动态点精细化偏移策略，放宽刚性反投影以纠正耦合的姿态-深度漂移。大量实验表明，尽管在没有未来帧的情况下操作，FreeStreamGS的渲染质量与最先进的离线前馈3DGS方法相当。

View on arXiv Download PDF AI Translation

cs.CV / 44 / 2606.03264

PaddleOCR-VL-1.6: Expanding the Frontier of Document Parsing with Under-Optimized Region Refinement and Progressive Post-Training

PaddleOCR-VL-1.6：通过未优化区域细化和渐进式后训练扩展文档解析的前沿

Zhang, Zelun, Liu, Hongen, Liang, Suyin, Zhang, Yubo, Xiang, Yiqing, Liu, Jiaxuan, Sun, Ting, Lin, Manhui, Zhang, Yue, Zhou, Changda, Gao, Tingquan, Cui, Cheng, Liu, Yi, Yu, Dianhai, Ma, Yanjun

Abstract

We introduce PaddleOCR-VL-1.6, an upgraded compact document parsing model built upon PaddleOCR-VL-1.5. Although PaddleOCR-VL-1.5 establishes a strong 0.9B baseline, its remaining errors concentrate in under-optimized regions where model behavior is unstable, data coverage is sparse, or supervision is unreliable. Rather than expanding the training corpus indiscriminately, PaddleOCR-VL-1.6 introduces a region-aware data optimization framework that identifies weak regions from the previous model, applies targeted enhancement to these regions, and improves the reliability of supervision signals. It further adopts a progressive post-training recipe based on curated data selection and reinforcement learning, pushing model performance to a higher level through staged optimization. PaddleOCR-VL-1.6 achieves a new state-of-the-art score of 96.33% on OmniDocBench v1.6, demonstrates strong competitiveness against top-tier VLMs, and provides a practical post-training recipe for the PaddleOCR-VL series.

Chinese Translation

我们介绍了PaddleOCR-VL-1.6，这是一个基于PaddleOCR-VL-1.5升级而来的紧凑型文档解析模型。尽管PaddleOCR-VL-1.5建立了强大的0.9B基线，但其剩余错误集中在未优化区域，这些区域的模型行为不稳定、数据覆盖稀疏或监督不可靠。PaddleOCR-VL-1.6并没有无差别地扩展训练语料库，而是引入了一种区域感知的数据优化框架，识别出前一模型的弱区域，对这些区域进行针对性增强，并提高监督信号的可靠性。此外，它还采用了一种基于精心挑选数据和强化学习的渐进式后训练方案，通过分阶段优化将模型性能提升到更高水平。PaddleOCR-VL-1.6在OmniDocBench v1.6上达到了96.33%的新状态-of-the-art得分，展现出与顶级视觉语言模型（VLMs）的强大竞争力，并为PaddleOCR-VL系列提供了实用的后训练方案。

View on arXiv Download PDF AI Translation

cs.CV / 45 / 2606.03273

VistaHop: Benchmarking Multi-hop Visual Reasoning for Visual DeepSearch

VistaHop：多跳视觉推理的基准测试用于视觉深度搜索

He, Hang, Yue, Chuhuai, Dong, Chengqi, Wan, Chengcheng, Su, Ting, Sun, Haiying, Chai, Jiajun, Wang, Xiaohan, Yin, Guojun

Abstract

Visual DeepSearch requires multimodal large reasoning model (MLRM) agents to answer complex visual queries by repeatedly inspecting image regions, grounding intermediate reasoning in visual evidence, and connecting fine-grained clues across long reasoning chains. However, existing benchmarks mainly focus on single-step visual understanding or static image-question answering, offering limited evaluation of iterative image inspection, visual-anchor grounding, and multi-hop evidence integration. In this work, we introduce VistaHop, a benchmark for evaluating vision-centric search and multi-hop visual reasoning in Visual DeepSearch. VistaHop contains 300 high-resolution images, 25 visual search scenarios, and 350 multi-hop QA tasks that require models to follow evidence chains from visual anchors or fuse information across multiple image-grounded reasoning paths. We further develop VistaArena, a unified evaluation environment that supports tool-augmented reasoning with text search, image search, image cropping, and evidence-based answer validation. Experiments on seven representative MLRMs show that current models remain far from solving VistaHop: the best model, SenseNova-MARS-32B, achieves only 24.31% Pass@1. These results reveal persistent limitations in visual grounding, evidence revisiting, long-chain reasoning, and multi-anchor information fusion, highlighting the need for stronger benchmarks and training methods for Visual DeepSearch.

Chinese Translation

视觉深度搜索需要多模态大型推理模型（MLRM）代理通过反复检查图像区域、将中间推理建立在视觉证据上，并在长推理链中连接细粒度线索来回答复杂的视觉查询。然而，现有基准主要集中于单步视觉理解或静态图像问答，限制了对迭代图像检查、视觉锚点定位和多跳证据整合的评估。在本研究中，我们引入了VistaHop，一个用于评估视觉中心搜索和多跳视觉推理的基准，适用于视觉深度搜索。VistaHop包含300张高分辨率图像、25个视觉搜索场景和350个多跳问答任务，这些任务要求模型遵循来自视觉锚点的证据链或融合多个图像基础推理路径的信息。我们进一步开发了VistaArena，一个统一的评估环境，支持文本搜索、图像搜索、图像裁剪和基于证据的答案验证的工具增强推理。在对七个具有代表性的MLRM进行实验时，结果显示当前模型在解决VistaHop方面仍然远远不够：最佳模型SenseNova-MARS-32B仅达到24.31%的通过率（Pass@1）。这些结果揭示了在视觉定位、证据重访、长链推理和多锚信息融合方面的持续局限性，强调了对视觉深度搜索更强基准和训练方法的需求。

View on arXiv Download PDF AI Translation

cs.CV / 46 / 2606.03287

BA-T: An Iterative Transformer for Two-View Bundle Adjustment

BA-T：一种用于双视图束调整的迭代变换器

Zhang, Ganlin, Chen, Weirong, Cremers, Daniel, Wang, Xi

Abstract

Feed-forward models for 3D reconstruction have achieved strong performance using deep cross-view attention to exchange information across images. However, these approaches often depend on heavy decoder stacks and lack a structured mechanism for geometry refinement, resulting in poor multi-view consistency. We address this by drawing inspiration from classical bundle adjustment (BA), which can be viewed as an iterative information propagation process between poses and local geometry. Inspired by BA, we propose BA-T, an iterative Transformer that implements BA-style structured updates as a repeatable layer in implicit token space. Instead of relying on deep attention stacks, BA-T refines predictions based on latent residual by a single lightweight layer. Experiments demonstrate that BA-T progressively improves pose and reconstruction accuracy across iterations, achieves stronger cross-view consistency than conventional decoders, and matches or surpasses substantially larger models while using only 16% of their decoder parameters. BA-T provides a compact, efficient, and structural alternative to depth-heavy attention, enabling accurate 3D reconstruction within a lightweight architecture. The code will be made publicly at https://github.com/zhangganlin/BA-T.

Chinese Translation

用于三维重建的前馈模型通过深度跨视图注意力在图像之间交换信息，取得了良好的性能。然而，这些方法通常依赖于复杂的解码器堆栈，并缺乏结构化的几何细化机制，导致多视图一致性较差。我们通过借鉴经典的束调整（BA）来解决这一问题，束调整可以被视为姿态与局部几何之间的迭代信息传播过程。受BA的启发，我们提出了BA-T，一种迭代变换器，它在隐式标记空间中实现BA风格的结构化更新，作为可重复的层。BA-T不依赖于深层注意力堆栈，而是通过一个轻量级层基于潜在残差细化预测。实验表明，BA-T在迭代过程中逐步提高了姿态和重建精度，达到了比传统解码器更强的跨视图一致性，并在仅使用其解码器参数的16%的情况下，匹配或超越了显著更大的模型。BA-T为深度重的注意力提供了一种紧凑、高效且结构化的替代方案，使得在轻量级架构中实现准确的三维重建成为可能。代码将公开发布在 https://github.com/zhangganlin/BA-T。

View on arXiv Download PDF AI Translation

cs.CV / 47 / 2606.03314

TASE: Truncation-Aware Semantic Embeddings for 3D Scene Understanding and Editing

TASE：面向截断的语义嵌入用于3D场景理解与编辑

Faasch, Tim-Felix, Kall, Jochen, Nunes, Lucas, Behley, Jens, Stachniss, Cyrill

Abstract

High-fidelity semantic 3D scene representations are crucial for numerous applications, including robotics, autonomous driving, and simulation. Beyond this, the ability to edit such representations enables developers to adapt these applications more easily to specific target scenarios. Current approaches provide limited support for controllable editing. We introduce TASE, a method that projects pretrained 2D semantic features into a truncation-aware embedding space to enable flexible 3D scene editing. Our method explicitly optimizes a feature space in which progressively reducing feature channels yields increasingly abstract semantic representations, while retaining more channels preserves fine-grained detail. Additionally, we improve multi-view consistency of the features using a scale- and translation-equivariance loss. The resulting truncation-aware embedding space enables text-driven edits to 3D scenes, providing explicit control over how strongly edits adhere to the original scene content and allowing more substantial modifications than prior methods. Moreover, we propose a finetuning stage for the editing diffusion model to mitigate artifacts caused by geometric changes. Experimental results demonstrate competitive performance in 3D scene editing, substantially outperforming prior methods on edits involving large geometric modifications.

Chinese Translation

高保真语义3D场景表示对于众多应用至关重要，包括机器人技术、自动驾驶和仿真。除此之外，编辑这些表示的能力使得开发者能够更轻松地将这些应用适配到特定的目标场景。目前的方法对可控编辑的支持有限。我们提出了TASE，一种将预训练的2D语义特征投影到一个考虑截断的嵌入空间的方法，以实现灵活的3D场景编辑。我们的方法明确优化了一个特征空间，在该空间中，逐渐减少特征通道可以产生越来越抽象的语义表示，而保留更多通道则保持细粒度的细节。此外，我们通过使用尺度和位移等变损失来提高特征的多视图一致性。最终得到的考虑截断的嵌入空间使得基于文本的3D场景编辑成为可能，提供了对编辑如何强烈遵循原始场景内容的明确控制，并允许比先前方法更大幅度的修改。此外，我们提出了一个微调阶段，用于编辑扩散模型，以减轻几何变化造成的伪影。实验结果表明，在3D场景编辑方面表现出竞争力，显著优于先前方法在涉及大规模几何修改的编辑任务上的表现。

View on arXiv Download PDF AI Translation

cs.CV / 48 / 2606.03341

Cross-Modality Feature Fusion Based on Structured State Space Duality for Multimodal Image Registration Network

基于结构状态空间对偶的跨模态特征融合用于多模态图像配准网络

Li, Zhikang, Wu, Yan, Hu, Xin, Dai, Yi, Li, Ming

Abstract

In multi-modal image registration, the primary challenge lies in shared structural information extraction. Compared to Transformers, Structured State Space Duality (SSD) offers greater global structural feature extraction with higher efficiency during training and inference. Inspired by these advantages, we propose a novel algorithm for multi-modal image registration, named RegNetMamba-2. Our algorithm incorporates SSD into coarse-to-fine matching process to extract local and global structural features effectively. Firstly, SSD is applied in three different scales for multi-modal feature extraction in our network. To strengthen local representation, we pay more attention on foreground edge and structural information by feature scaling function of SSD. Secondly, for shared feature extraction of input images and multi-modal feature fusion in all scales, we propose cross-modality feature fusion model based on SSD, consisting of Cross-Modality feature Interaction (CMI) module and Multi-Scale feature Fusion (MSF) module. CMI module is designed for cross-modality feature extraction of each scale by SSD in cross form. MSF module is designed to employ a progressive upward fusion in feature-level to obtain fine features, consisting of multi-modal features in all scales. Following coarse-to-fine, the features in 1/8 scale from CMI and 1/2 scale from MSF are collected to calculate matching probability scores. Then we respectively establish matching process by correspondences of pixel-wise. Extensive experiments demonstrate that comparing with state-of-the-art deep-learning based algorithms, RegNetMamba-2 has achieved good effects in both performance and efficiency for multi-modal image registration on the following datasets: VIS-SAR (OSDataset), VIS-IR (LGHD/RoadSence) and VIS-NIR (RGB-NIR sense).

Chinese Translation

在多模态图像配准中，主要挑战在于共享结构信息的提取。与变换器（Transformers）相比，结构状态空间对偶（Structured State Space Duality, SSD）在训练和推理过程中提供了更高效的全局结构特征提取。受到这些优势的启发，我们提出了一种新的多模态图像配准算法，命名为RegNetMamba-2。我们的算法将SSD融入粗到细的匹配过程中，以有效提取局部和全局结构特征。首先，SSD在三个不同的尺度上应用于我们的网络中的多模态特征提取。为了增强局部表示，我们通过SSD的特征缩放函数更加关注前景边缘和结构信息。其次，为了实现输入图像的共享特征提取和各尺度的多模态特征融合，我们提出了基于SSD的跨模态特征融合模型，该模型由跨模态特征交互（Cross-Modality feature Interaction, CMI）模块和多尺度特征融合（Multi-Scale feature Fusion, MSF）模块组成。CMI模块旨在通过SSD以交叉形式提取每个尺度的跨模态特征。MSF模块则设计为在特征层面上采用渐进向上的融合，以获取细致特征，包含所有尺度的多模态特征。在粗到细的过程中，我们收集来自CMI的1/8尺度特征和来自MSF的1/2尺度特征，以计算匹配概率分数。然后，我们分别通过像素级的对应关系建立匹配过程。大量实验表明，与最先进的基于深度学习的算法相比，RegNetMamba-2在以下数据集上实现了良好的性能和效率：VIS-SAR（OSDataset）、VIS-IR（LGHD/RoadSence）和VIS-NIR（RGB-NIR sense）。

View on arXiv Download PDF AI Translation

cs.CV / 49 / 2606.03345

Beyond Semantics: Modeling Factual and Affective Perceptual Experiences from Vision-Language Data

超越语义：从视觉-语言数据建模事实与情感的感知体验

Mohamed, Youssef, Church, Kenneth Ward, Elhoseiny, Mohamed

Abstract

We present P-Topics (Perception Topics) modeling, a novel problem for understanding how images are perceived affectively and across cultures. The goal is to (1) discover and model the different perception experiences in a dataset of images and captions, where each experience is defined by an objective factual and a subjective affective aspect, and (2) associate images to their relevant perception experiences. We introduce **PercepT** (**Percep**tion topic **T**ransformer), a two-stage architecture that tackles P-Topics modeling. In the formation stage, percepT discovers *P-Topics* as visual-textual clusters using an unsupervised training objective, and dynamically selects the number of clusters to match the perceptual richness of the dataset. In the mapping stage, it learns *P-Topic mapping functions* via attention pooling to associate images to their respective clusters. On ArtELingo, PercepT achieves a silhouette score of **0.97** compared to **0.37** from the closest baseline reflecting better perceptual clusters. PercepT also achieves an AUC score of **0.94** compared to **0.77** showing better mapping to perceptual clusters. Human evaluation confirms that PercepT captures semantically meaningful perception experiences and significantly outperforms existing methods. Our implementation will be made public.

Chinese Translation

我们提出了P-Topics（感知主题）建模，这是一项新颖的问题，旨在理解图像在情感上和跨文化的感知方式。其目标是（1）发现并建模图像和标题数据集中不同的感知体验，每种体验由客观的事实和主观的情感方面定义，以及（2）将图像与其相关的感知体验关联起来。我们引入了**PercepT**（**Percep**tion topic **T**ransformer），一种两阶段架构，解决P-Topics建模问题。在形成阶段，percepT使用无监督训练目标发现*P-Topics*作为视觉-文本聚类，并动态选择聚类数量以匹配数据集的感知丰富性。在映射阶段，它通过注意力池化学习*P-Topic映射函数*，将图像与其各自的聚类关联。在ArtELingo上，PercepT的轮廓得分为**0.97**，而最接近的基线为**0.37**，显示出更好的感知聚类。PercepT还获得了**0.94**的AUC得分，而**0.77**则显示出更好的感知聚类映射。人类评估确认PercepT捕捉到了语义上有意义的感知体验，并显著优于现有方法。我们的实现将公开发布。

View on arXiv Download PDF AI Translation

cs.CV / 50 / 2606.03348

SynCred-Bench: Benchmarking Synthetic Credibility in AI-Generated Visual Misinformation

SynCred-Bench：人工智能生成视觉虚假信息的合成可信度基准测试

Yang, Junxiao, Zhang, Minghao, Wang, Xiaoce, Liu, Haoran, Cui, Shiyao, Wang, Hongning, Huang, Minlie

Abstract

Recent generative models can now produce visual artifacts with realistic embedded text and layouts, creating a new misinformation threat: synthetic credibility. We introduce SYNCRED-Bench, a benchmark of 600 AI-generated misinformation images balanced across six credible-form categories and seven fine-grained circulation styles, together with FP450, a real-image negative set for measuring false positives. Extensive evaluation shows that existing systems remain unreliable: under a 5% false-positive-rate constraint, 15 MLLMs achieve only 10.5% true positive rate (TPR), open-source AIGC detectors achieve less than 5%, and commercial APIs reach 57.6%. Human annotators also struggled to identify synthetic credibility, reaching only 63% TPR. These findings establish synthetic credibility as a severe and underexplored visual misinformation challenge, and provide a benchmark for developing detectors that reason beyond superficial credibility cues.

Chinese Translation

近期的生成模型能够生成带有真实嵌入文本和布局的视觉伪影，形成了一种新的虚假信息威胁：合成可信度。我们介绍了SYNCRED-Bench，这是一个包含600幅人工智能生成的虚假信息图像的基准，涵盖六种可信形式类别和七种细粒度传播风格，并配备FP450，一个用于测量假阳性的真实图像负样本集。广泛的评估表明，现有系统仍然不可靠：在5%的假阳性率限制下，15个多语言大型语言模型（MLLMs）仅实现了10.5%的真实阳性率（TPR），开源的人工智能生成内容（AIGC）检测器的真实阳性率低于5%，而商业API的真实阳性率为57.6%。人类标注者在识别合成可信度方面也遇到了困难，仅达到63%的真实阳性率。这些发现确立了合成可信度作为一种严重且未被充分探索的视觉虚假信息挑战，并为开发超越表面可信度线索的检测器提供了基准。

View on arXiv Download PDF AI Translation

cs.CV / 51 / 2606.03376

P\textsuperscript{2}-DPO: Grounding Hallucination in Perceptual Processing via Calibration Direct Preference Optimization

P extsuperscript{2}-DPO：通过校准直接偏好优化在感知处理中的幻觉基础

Zhang, Ruipeng, Li, Zhihao, Yuan, Haozhang, Chen, C. L. Philip, Zhang, Tong

Abstract

Hallucination has recently garnered significant research attention in Large Vision-Language Models (LVLMs). Direct Preference Optimization (DPO) aims to learn directly from the corrected preferences provided by humans, thereby addressing the hallucination issue. Despite its success, this paradigm has yet to specifically target the perceptual bottleneck in attended regions or address insufficient Visual Robustness against image degradation. Furthermore, existing preference pairs are often vision-agnostic and their inherently off-policy nature limits their effectiveness in guiding model learning. To address these challenges, we propose Perceptual Processing Direct Preference Optimization (P\textsuperscript{2}-DPO), a novel training paradigm in which the model generates and learns from its own preference pairs, thereby directly addressing the identified visual bottlenecks while inherently avoiding the issues of vision-agnostic and off-policy data. It introduces: (1) an on-policy preference pairs construction method targeting Focus-and-Enhance perception and Visual Robustness, and (2) a well-designed Calibration Loss to precisely align visual signals with the causal generation of text. Experimental results demonstrate that with a comparable amount of training data and cost, P\textsuperscript{2}-DPO outperforms strong baselines that rely on costly human feedback on benchmarks. Furthermore, evaluations on Attention Region Fidelity (ARF) and image degradation scenarios validate the effectiveness of P\textsuperscript{2}-DPO in addressing perceptual bottleneck in attended regions and improving Visual Robustness against degraded inputs.

Chinese Translation

幻觉最近在大型视觉-语言模型（LVLMs）中引起了显著的研究关注。直接偏好优化（DPO）旨在直接从人类提供的修正偏好中学习，从而解决幻觉问题。尽管这一范式取得了成功，但尚未专门针对关注区域中的感知瓶颈或解决对图像降级的视觉鲁棒性不足。此外，现有的偏好对通常是与视觉无关的，其固有的离策略特性限制了它们在指导模型学习中的有效性。为了解决这些挑战，我们提出了感知处理直接偏好优化（P extsuperscript{2}-DPO），这是一种新颖的训练范式，其中模型生成并学习自己的偏好对，从而直接解决识别出的视觉瓶颈，同时固有地避免与视觉无关和离策略数据的问题。它引入了：（1）一种针对聚焦与增强感知和视觉鲁棒性的在线偏好对构建方法，以及（2）一种精心设计的校准损失，以精确对齐视觉信号与文本的因果生成。实验结果表明，在相当数量的训练数据和成本下，P extsuperscript{2}-DPO在基准测试中优于依赖于昂贵人类反馈的强基线。此外，在关注区域保真度（ARF）和图像降级场景的评估中，验证了P extsuperscript{2}-DPO在解决关注区域的感知瓶颈和提高对降级输入的视觉鲁棒性方面的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 52 / 2606.03401

Towards Characterizing Scientific Image Utility and Upgradability

科学图像效用与可升级性的特征化研究

Li, WenZhe, Yan, Qihang, Chen, Liang, Wang, Junying, Wen, Farong, Guo, Yijin, Li, Chunyi, Zhang, Zicheng, Zhai, Guangtao

Abstract

Scientific images function as critical evidence in research communication, yet their integrity faces unprecedented threats from AI-generated content that introduces subtle but consequential errors. Existing evaluation paradigms prove inadequate: perceptual quality metrics poorly correlate with scientific validity, while language models lack domain-specific verification capabilities. To address this gap, we propose the \textbf{S}cientific \textbf{I}mage \textbf{U}tility and \textbf{U}pgradability \textbf{A}ssessment (\textbf{SIU$^2$A}) framework, which introduces two complementary dimensions for scientific image evaluation. \textbf{Utility} encompasses \textit{error detection} (identifying scientific inaccuracies) and \textit{correction feasibility} (assessing whether errors can be reliably repaired). \textbf{Upgradability} measures the quality of correction. We categorize scientific image corruption into four fundamental types: Detail Distortion, Incompleteness, False Content, and Entity Confusion. Based on this taxonomy, we construct SIU$^2$A-Benchmark, a dataset with expert annotations for error identification and repair. The framework implements a two-stage evaluation protocol: the \textit{Utility} stage evaluates error detection capability and repair instruction generation, while the \textit{Upgradability} stage assesses whether corrections faithfully restore scientific validity without compromising existing accurate information. Experiments reveal that current multimodal systems exhibit significant limitations in both scientific error assessment and faithful correction, exposing a fundamental gap between visual perception and scientific usability.

Chinese Translation

科学图像在研究交流中作为关键证据发挥着重要作用，但其完整性面临来自人工智能生成内容的前所未有的威胁，这些内容引入了微妙但影响深远的错误。现有的评估范式证明是不够的：感知质量指标与科学有效性之间的相关性较差，而语言模型缺乏领域特定的验证能力。为了解决这一问题，我们提出了 extbf{S}cientific extbf{I}mage extbf{U}tility and extbf{U}pgradability extbf{A}ssessment（ extbf{SIU$^2$A}）框架，该框架引入了两个互补的维度用于科学图像评估。 extbf{效用}包括 extit{错误检测}（识别科学不准确性）和 extit{修正可行性}（评估错误是否可以可靠地修复）。 extbf{可升级性}衡量修正的质量。我们将科学图像的损坏分为四种基本类型：细节失真、不完整性、虚假内容和实体混淆。基于这一分类法，我们构建了SIU$^2$A-Benchmark，一个具有专家注释的用于错误识别和修复的数据集。该框架实施了一个两阶段的评估协议： extit{效用}阶段评估错误检测能力和修复指令生成，而 extit{可升级性}阶段评估修正是否忠实地恢复科学有效性而不损害现有的准确信息。实验表明，当前的多模态系统在科学错误评估和忠实修正方面存在显著局限性，暴露了视觉感知与科学可用性之间的根本差距。

View on arXiv Download PDF AI Translation

cs.CV / 53 / 2606.03402

Mamba-Enhanced Implicit Motion Learning for Audio-Driven Portrait Animation

基于Mamba增强的隐式运动学习用于音频驱动的肖像动画

Wei, Xuan, Chen, Jiahui, Li, Kaiheng, Shao, Mingyu, Hong, Qingqi

Abstract

Audio-driven human motion video generation aims to synthesize realistic and temporally coherent human animations from a single static image, with applications in talking-head synthesis, co-speech gesture generation, and dynamic presentations. Moving beyond conventional keypoint-based methods that often struggle to capture subtle motion dynamics, We propose a novel implicit-motion framework for generating realistic and temporally coherent human motion videos from a single static image and audio. Our approach uses a two-stage pipeline that decouples motion prediction from rendering. The first stage integrates appearance priors and hierarchical depth cues into a region-aware attention mechanism to model latent motion features. The second stage employs a Mamba-enhanced diffusion model to directly predict these features from audio and the source image, enabling unsupervised learning of fine-grained motion patterns. This decoupled architecture enhances flexibility and efficiency. Trained on a new 380-hour high-quality dataset, our method outperforms prior work across multiple public benchmarks and our collected data in accuracy, naturalness, and temporal coherence, setting a new state-of-the-art.

Chinese Translation

音频驱动的人类运动视频生成旨在从单一静态图像合成逼真且时间一致的人类动画，应用于对话头合成、共语手势生成和动态演示等领域。我们提出了一种新颖的隐式运动框架，超越了传统的关键点方法，这些方法往往难以捕捉细微的运动动态。我们的框架从单一静态图像和音频生成逼真且时间一致的人类运动视频，采用了一个两阶段的管道，将运动预测与渲染解耦。第一阶段将外观先验和分层深度线索整合到区域感知注意机制中，以建模潜在的运动特征。第二阶段采用Mamba增强的扩散模型，直接从音频和源图像预测这些特征，实现细粒度运动模式的无监督学习。这种解耦架构增强了灵活性和效率。在一个新的380小时高质量数据集上训练后，我们的方法在多个公共基准和我们收集的数据中，在准确性、自然性和时间一致性方面超越了先前的工作，树立了新的最先进水平。

View on arXiv Download PDF AI Translation

cs.CV / 54 / 2606.03406

SAMatcher: Co-Visibility Modeling with Segment Anything for Robust Feature Matching

SAMatcher：基于可见性建模的鲁棒特征匹配

Pan, Xu, Ma, Qiyuan, Dong, Mingyue, Chen, He, Ji, Wei, Zheng, Xianwei

Abstract

Reliable correspondence estimation is a fundamental problem in image processing, underpinning applications such as Structure from Motion, visual localization, and image registration. Existing learning-based methods have significantly improved local feature representations, yet most still operate at the pixel or patch level and lack explicit modeling of regions that are jointly visible across views. We propose SAMatcher, a feature matching framework that formulates correspondence estimation through co-visibility modeling. Instead of directly matching local features, SAMatcher first predicts co-visible region masks and bounding boxes as structured priors for correspondence estimation. Built upon the Segment Anything Model (SAM), it introduces a symmetric cross-view interaction mechanism that enables bidirectional feature exchange and cross-view semantic alignment. We further develop a unified supervision scheme that jointly optimizes mask prediction and box localization through mask learning, box regression, and mask-box consistency constraints. Extensive experiments on challenging benchmarks demonstrate substantial improvements over existing matching pipelines, particularly under large viewpoint and scale variations. Our results show that foundation models originally designed for monocular segmentation can be effectively extended to multi-view correspondence reasoning through explicit co-visibility modeling, offering a new perspective on structured representation learning for image matching. Code and project page: https://xupan.top/Projects/samatcher

Chinese Translation

可靠的对应估计是图像处理中的一个基本问题，支撑着诸如运动结构重建、视觉定位和图像配准等应用。现有的基于学习的方法显著改善了局部特征表示，但大多数仍然在像素或补丁级别操作，缺乏对跨视角共同可见区域的明确建模。我们提出了SAMatcher，一个通过可见性建模来制定对应估计的特征匹配框架。SAMatcher并不是直接匹配局部特征，而是首先预测共同可见区域的掩模和边界框，作为对应估计的结构先验。基于Segment Anything Model (SAM)，它引入了一种对称的跨视角交互机制，能够实现双向特征交换和跨视角语义对齐。我们进一步开发了一种统一的监督方案，通过掩模学习、边框回归和掩模-边框一致性约束共同优化掩模预测和边框定位。在具有挑战性的基准测试上的大量实验表明，相较于现有的匹配管道，SAMatcher在大视角和尺度变化下表现出显著的改进。我们的结果表明，最初为单目分割设计的基础模型可以通过明确的可见性建模有效扩展到多视角对应推理，为图像匹配的结构化表示学习提供了新的视角。代码和项目页面：https://xupan.top/Projects/samatcher

View on arXiv Download PDF AI Translation

cs.CV / 55 / 2606.03410

Enginuity: A Dataset and Benchmark for Vision-Language Understanding of Engineering Diagrams

Enginuity：用于工程图纸视觉语言理解的数据集和基准

Kumar, Abhishek, Motiyani, Isha, Kasturi, Tilak, Seefried, Ethan, Movva, Prahitha, Ghosal, Tirthankar

Abstract

Engineering diagrams pose a distinct challenge for vision-language models: unlike natural images or general documents, they encode information through dense spatial layouts, domain-specific symbols, and cross-references between visual callouts and structured parts tables. Despite their centrality to service, repair, and design workflows, there is no public benchmark for measuring VLM capabilities in this domain; existing datasets primarily focus on flowcharts, scientific figures, or business documents. To address this gap, we introduce Enginuity, the first open dataset and benchmark for evaluating VLMs on complex engineering diagrams. We define two tasks over a corpus of U.S. military service and repair manuals: structured parts-table extraction (Task 1) and free-form visual diagram question answering (VQA)(Task 2) for benchmarking. We evaluate four frontier VLMs (GPT-5.2 Chat, Claude Opus 4.7, Gemma 4, Qwen3-VL-32B-Instruct) under zero-shot and chain-of-thought prompting. On Task 1, models reach Recall@all of 0.61-0.87 but Token F1pen of only 0.03-0.18, exposing a systematic gap between part identification and description fidelity. Task 2 reveals a consistent factual-reasoning gap across all models. A supporting analysis shows that token-overlap metrics under-report model capability on technical descriptions by 2-6x relative to semantic similarity, motivating LLM-as-judge calibration for domain-specific evaluation. We release the dataset, annotations, evaluation harness, and per-sample model outputs to support a reproducible study of VLM capability on engineering content.

Chinese Translation

工程图纸对视觉语言模型提出了独特的挑战：与自然图像或一般文档不同，它们通过密集的空间布局、特定领域的符号以及视觉标注与结构化零件表之间的交叉引用来编码信息。尽管工程图纸在服务、维修和设计工作流程中至关重要，但在该领域尚无公开基准来衡量视觉语言模型的能力；现有数据集主要集中于流程图、科学图形或商业文档。为了解决这一空白，我们推出了Enginuity，这是第一个用于评估视觉语言模型在复杂工程图纸上的开放数据集和基准。我们在美国军方服务和维修手册的语料库上定义了两个任务：结构化零件表提取（任务1）和自由形式视觉图表问答（VQA）（任务2）以进行基准测试。我们在零-shot 和思维链提示下评估了四个前沿视觉语言模型（GPT-5.2 Chat、Claude Opus 4.7、Gemma 4、Qwen3-VL-32B-Instruct）。在任务1中，模型的 Recall@all 达到 0.61-0.87，但 Token F1pen 仅为 0.03-0.18，暴露了零件识别与描述准确性之间的系统性差距。任务2显示所有模型在事实推理方面存在一致的差距。支持性分析表明，基于 token 重叠的度量相对于语义相似性低估了模型在技术描述上的能力，低估幅度为 2-6 倍，这促使我们进行领域特定评估的 LLM 作为评判者的校准。我们发布了数据集、注释、评估工具和每个样本的模型输出，以支持对视觉语言模型在工程内容上能力的可重复研究。

View on arXiv Download PDF AI Translation

cs.CV / 56 / 2606.03417

A unified multi-task framework enables interpretable chest radiograph analysis

统一的多任务框架实现可解释的胸部放射影像分析

Xu, Lijian, Ni, Ziyu, Liu, Xinglong, Wang, Xiaosong, Li, Hongsheng, Zhang, Shaoting

Abstract

While multimodal deep learning has advanced medical imaging analysis, existing black-box systems \textcolor{black}{may remain confined to isolated tasks, often overlooking} the trust-sensitive nature of clinical diagnosis as a multi-task process. We propose IMT-CXR (Interpretable Multi-task Transformer for Chest X-ray Analysis), a framework that emulates radiologists' diagnostic workflow through three evidence-driven stages: 1) Disease recognition; 2) Attribute characterization (e.g., size, location, severity quantification); 3) Evidence-integrated report generation with traceable decision pathways. The framework employs a unified transformer architecture optimized via medical-domain instruction tuning, sequentially executing four clinical tasks: multi-label disease classification, lesion localization, anatomical segmentation, and radiology report generation. Experimental validation demonstrates competitive performance on ten CXR benchmarks under direct inference and fine-tuning settings. In a blinded evaluation of 160 historical reports from four medical centers, three radiologists rated 66\% of AI-generated reports as comparable to or surpassing original clinical reports in diagnostic clarity, highlighting the framework's translational potential. By establishing traceable diagnostic pathways from anatomical findings to conclusions, this work bridges the gap between AI technical metrics and clinical utility, advancing trustworthy AI systems in medical imaging.

Chinese Translation

尽管多模态深度学习推动了医学影像分析的发展，但现有的黑箱系统往往局限于孤立的任务，常常忽视临床诊断作为多任务过程的信任敏感性。我们提出了IMT-CXR（可解释的多任务变换器用于胸部X光分析），该框架通过三个基于证据的阶段模拟放射科医生的诊断工作流程：1）疾病识别；2）属性特征化（例如，大小、位置、严重程度量化）；3）生成具有可追溯决策路径的证据整合报告。该框架采用统一的变换器架构，通过医学领域的指令调优进行优化，依次执行四个临床任务：多标签疾病分类、病变定位、解剖分割和放射学报告生成。实验验证表明，在直接推理和微调设置下，十个CXR基准测试中表现出竞争力的性能。在对来自四个医疗中心的160份历史报告的盲评中，三位放射科医生将66%的AI生成报告评定为与原始临床报告在诊断清晰度上相当或更优，突显了该框架的转化潜力。通过建立从解剖发现到结论的可追溯诊断路径，本研究弥合了AI技术指标与临床实用性之间的差距，推动了医学影像中可信赖的AI系统的发展。

View on arXiv Download PDF AI Translation

cs.CV / 57 / 2606.03418

IDO: Incongruity-aware Distribution Optimization for Multimodal Fake News Detection

IDO：面向不一致性的分布优化用于多模态假新闻检测

Zhou, Hengyang, Hong, Rongman, Zhou, Yuxuan, Wang, Jing, Pan, Zhaoyan

Abstract

Multimodal fake news detection aims to identify the authenticity of news. Existing multimodal fake news detection methods mainly focus on cross-modal consistency, but often fail to explicitly model the semantic incongruity that characterizes deceptive multimodal content. However, misinformation often contains semantic information incongruity with the facts. To address these challenges, we propose Incongruity-aware Distribution Optimization (IDO) to improve the performance of fake news detection from the perspectives of factual incongruity and modality incongruity. For factual incongruity, we introduce a channel-wise reweighting strategy to obtain semantically discriminative embeddings and utilize gaussian distribution to model the uncertain correlation caused by factual incongruity. For modality incongruity, we utilize incongruity contrastive learning to learn cross-modal semantic information. Experiments demonstrate that IDO achieves state-of-the-art performance.

Chinese Translation

多模态假新闻检测旨在识别新闻的真实性。现有的多模态假新闻检测方法主要关注跨模态一致性，但往往未能明确建模特征性的不一致性，这种不一致性是欺骗性多模态内容的特征。然而，虚假信息通常包含与事实不一致的语义信息。为了解决这些挑战，我们提出了面向不一致性的分布优化（Incongruity-aware Distribution Optimization，IDO），从事实不一致性和模态不一致性的角度提高假新闻检测的性能。对于事实不一致性，我们引入了一种通道加权重策略，以获得语义上具有区分性的嵌入，并利用高斯分布来建模因事实不一致性引起的不确定相关性。对于模态不一致性，我们利用不一致性对比学习来学习跨模态的语义信息。实验表明，IDO实现了最先进的性能。

View on arXiv Download PDF AI Translation

cs.CV / 58 / 2606.03420

PHAF-Personalized Hand Avatars in a Flash

PHAF-快速个性化手部虚拟形象

Shankar, Meghana, Upadhyay, Akanxit, Namdev, Anmol, KS, Green Rosh, BH, Pawan Prasad

Abstract

We present PHAF-Personalized Hand Avatars in a Flash, a personalized photo-realistic hand avatar which provides high quality multi-view renders from just two images (dorsal and palmar views).Unlike slow optimization-based techniques, PHAF generates fast personalized textures for real-time deployment on edge devices. Our approach combines semantic guided mesh alignment and densified texture extraction to transfer high-frequency details efficiently. A view-based inpainting network refines textures ensuring smooth, continuous appearance. PHAF generalizes to novel viewpoints and leverages a parametric hand model for accurate articulations, making it compatible with standard graphics engines. Experiments show it is comparable to existing methods in visual fidelity while drastically reducing texture generation time by 30 times, enabling practical AR/VR applications.

Chinese Translation

我们提出了PHAF-快速个性化手部虚拟形象，这是一种个性化的照片级真实感手部虚拟形象，仅需两张图像（背面视图和掌面视图）即可提供高质量的多视角渲染。与缓慢的基于优化的技术不同，PHAF能够快速生成个性化纹理，以便在边缘设备上实时部署。我们的方法结合了语义引导的网格对齐和密集纹理提取，以高效地传递高频细节。基于视图的修补网络对纹理进行精细化处理，确保外观平滑、连续。PHAF能够推广到新颖的视角，并利用参数化手部模型实现准确的关节运动，使其与标准图形引擎兼容。实验表明，PHAF在视觉保真度上与现有方法相当，同时将纹理生成时间减少了30倍，使其实用的增强现实/虚拟现实应用成为可能。

View on arXiv Download PDF AI Translation

cs.CV / 59 / 2606.03444

PRISM: Synergizing Vision Foundation Models via Self-organized Expert Specialization

PRISM：通过自组织专家专业化协同视觉基础模型

Tang, Ying, Li, Dong, Zhang, Youjia, Song, Zikai, Yu, Junqing, Yang, Wei

Abstract

Unifying the complementary strengths of diverse Vision Foundation Models (VFMs) into a single efficient model is highly desirable but challenged by the negative transfer inherent in monolithic distillation. To address these feature conflicts, we introduce \textbf{PRISM}, a novel dual-stream Mixture-of-Experts (MoE) framework that synergizes VFMs via modular specialization. We propose a two-stage paradigm: (1) expertise deconstruction, where a teacher-conditional router guides experts to specialize in distinct representational subspaces to mitigate interference, followed by (2) dynamic recomposition, where the router learns to assemble these experts into tailored computational pathways for downstream tasks. Experiments on PASCAL-Context and NYUD-v2 show that \textbf{PRISM} establishes a new state of the art, validating that sparse, emergent specialization is a scalable approach for integrating diverse visual knowledge.

Chinese Translation

将多样化视觉基础模型（VFMs）的互补优势统一为一个高效模型是非常理想的，但受到单一蒸馏中固有的负迁移的挑战。为了解决这些特征冲突，我们提出了 extbf{PRISM}，一种新颖的双流混合专家（MoE）框架，通过模块化专业化协同VFMs。我们提出了一个两阶段的范式：（1）专业知识解构，在这一阶段，教师条件路由器引导专家在不同的表征子空间中专业化，以减轻干扰；（2）动态重组，在这一阶段，路由器学习将这些专家组装成针对下游任务量身定制的计算路径。在PASCAL-Context和NYUD-v2上的实验表明， extbf{PRISM}建立了新的最先进水平，验证了稀疏的、突现的专业化是一种可扩展的整合多样化视觉知识的方法。

View on arXiv Download PDF AI Translation

cs.CV / 60 / 2606.03460

From 3D Perception to Safety Reasoning: A Graph-Based Framework for Real-Time Underground Mine Monitoring

从3D感知到安全推理：一种基于图的实时地下矿山监测框架

Ranasinghe, Pasindu, Raval, Simit, Patra, Dibyayan, Banerjee, Bikram, Canbulat, Ismet

Abstract

Underground coal mining requires personnel and heavy equipment to operate within shared, confined, and poorly illuminated spaces where hazards such as equipment proximity violations, structural instabilities, and occluded blind spots are difficult to anticipate. Conventional monitoring systems, including fixed cameras and rule-based proximity alerts, can detect predefined events but lack the 3D scene understanding and contextual memory needed to identify complex or evolving hazards. This paper presents a continuous monitoring framework that converts colourised 3D point clouds into structured and traceable safety reasoning outputs. The framework combines 3D semantic perception, uncertainty-based anomaly detection, rule-based hazard checks, on-device LLM reasoning, and GraphRAG -based memory analysis to identify immediate hazards and interpret longer-term safety patterns. Scene and temporal graphs serve as the explicit knowledge structure, linking perception outputs across reasoning stages. To overcome the scarcity of labeled underground data, real roadway scans, controlled object placement, and high-fidelity longwall simulation were combined to generate diverse hazard scenarios, while self-supervised pretraining improved segmentation from limited annotations. The perception model achieved 92.7% accuracy at 30 FPS with low memory usage. Across 115 hazard scenarios, rule-based checks achieved 57% coverage, increasing to 76% with contextual LLM reasoning and 93% with memory-based reasoning using historical records. Qualitative results show uncertainty-derived anomaly signals support the interpretation of out-of-distribution hazards beyond predefined classes. Overall, graph-based knowledge representation combined with 3D perception and layered safety reasoning provides a practical foundation for intelligent decision support in underground mine monitoring.

Chinese Translation

地下煤矿开采需要人员和重型设备在共享、封闭且照明不足的空间内操作，这些空间中存在设备接近违规、结构不稳定和遮挡盲点等难以预见的危险。传统监测系统，包括固定摄像头和基于规则的接近警报，能够检测预定义事件，但缺乏识别复杂或演变危险所需的3D场景理解和上下文记忆。本文提出了一种连续监测框架，将彩色化的3D点云转换为结构化且可追溯的安全推理输出。该框架结合了3D语义感知、不确定性基础的异常检测、基于规则的危险检查、设备内的LLM推理和基于GraphRAG的记忆分析，以识别即时危险并解释长期安全模式。场景和时间图作为显式知识结构，连接了各推理阶段的感知输出。为了克服标注地下数据的稀缺性，结合真实道路扫描、受控物体放置和高保真长壁模拟生成了多样化的危险场景，同时自监督预训练提高了有限标注下的分割效果。感知模型在30 FPS下实现了92.7%的准确率，且内存使用低。在115个危险场景中，基于规则的检查实现了57%的覆盖率，结合上下文LLM推理后提高至76%，使用历史记录的记忆基础推理则达到了93%。定性结果表明，源自不确定性的异常信号支持对超出预定义类别的分布外危险的解释。总体而言，基于图的知识表示结合3D感知和分层安全推理为地下矿山监测中的智能决策支持提供了实用的基础。

View on arXiv Download PDF AI Translation

cs.CV / 61 / 2606.03470

Mixed-Modality Dual Face-Hair Retrieval

混合模态双面发检索

Bui-Huynh, Quoc-Anh, Lam, Mai-Tuyen, Nguyen, Dai-Anh-Tuan, Ngo, Thanh Duc

Abstract

We introduce Dual Face-Hair Retrieval (DFHR), a new mixed-modality dual-reference task in image retrieval where a query consists of a face image specifying identity and a hairstyle reference expressed as either an image or text. Unlike prior retrieval settings, DFHR requires cross-component reasoning between two semantically independent attributes -- identity and hairstyle -- originating from heterogeneous modalities. This formulation demands localized feature disentanglement, cross-modal semantic alignment, and mixed-modality composition within a unified embedding space. We construct DFHR-Bench, the first benchmark for mixed-modality face-hair retrieval, comprising over 180K annotated triplets across dual-image and image-text settings, built via a multi-stage annotation protocol ensuring semantic and identity integrity. We further propose MFHC (Multimodal Face-Hair Combiner), a unified framework that fuses disentangled identity and hairstyle embeddings through token injection and multi-view supervision. DFHR and DFHR-Bench together establish a new paradigm for identity-aware, attribute-controllable visual retrieval across modalities.

Chinese Translation

我们提出了双面发检索（Dual Face-Hair Retrieval, DFHR），这是一种新的混合模态双参考任务，在图像检索中，查询由一个指定身份的面部图像和一个以图像或文本形式表达的发型参考组成。与以往的检索设置不同，DFHR要求在两个语义独立的属性——身份和发型之间进行跨组件推理，这些属性来自异构模态。这种表述要求在统一的嵌入空间内进行局部特征解耦、跨模态语义对齐和混合模态组合。我们构建了DFHR-Bench，这是第一个混合模态面发检索基准，包含超过18万对标注三元组，涵盖双图像和图像-文本设置，采用多阶段标注协议构建，以确保语义和身份的完整性。我们进一步提出了MFHC（Multimodal Face-Hair Combiner），这是一个统一框架，通过标记注入和多视角监督融合解耦的身份和发型嵌入。DFHR和DFHR-Bench共同建立了一种新的范式，用于跨模态的身份感知和属性可控的视觉检索。

View on arXiv Download PDF AI Translation

cs.CV / 62 / 2606.03479

PersistGS: Differentiable Physics for Object Permanence in 4D Gaussian Splatting

PersistGS：用于4D高斯点云中物体持久性的可微物理学

Ramlal, Adrian, Zelek, John S.

Abstract

Dynamic 3D Gaussian Splatting (3DGS) methods reconstruct time-varying scenes from synchronized multi-camera video using photometric supervision. When a moving object becomes fully occluded from all training cameras, this supervision vanishes: the Gaussians representing it receive no gradient signal and degrade. Existing approaches to incomplete observations in neural reconstruction rely on learned generative priors that prioritize visual plausibility over physical correctness. We propose $\textbf{PersistGS}$, a method that restores object permanence during occlusion by coupling differentiable rigid body simulation with 3D Gaussian Splatting. Our approach decomposes the scene into per-object Gaussians and collision meshes, estimates friction and velocity from the observed pre-occlusion trajectory via differentiable simulation, and uses the resulting SE(3) trajectory to position object Gaussians throughout the occlusion period. Because the predicted trajectory satisfies the governing equations of rigid body dynamics, it faithfully captures contact events (bounces, friction-based deceleration, direction changes) that kinematic extrapolation cannot model. We introduce a centroid silhouette loss that isolates positional gradients from appearance noise, yielding 40% lower trajectory error than photometric supervision. We evaluate using cameras withheld from training that observe the object during its occlusion. Experiments on synthetic scenes show that PersistGS outperforms constant velocity extrapolation by +2.46dB PSNR and comes within 0.19dB of a ground-truth trajectory upper bound.

Chinese Translation

动态3D高斯点云（3DGS）方法利用光度监督从同步的多摄像头视频中重建时变场景。当一个移动物体完全被所有训练摄像头遮挡时，这种监督消失：表示该物体的高斯体接收不到梯度信号而退化。现有的神经重建中对不完整观测的处理方法依赖于学习的生成先验，这些先验优先考虑视觉合理性而非物理正确性。我们提出了$ extbf{PersistGS}$，一种通过将可微刚体模拟与3D高斯点云相结合来恢复遮挡期间物体持久性的方法。我们的方法将场景分解为每个物体的高斯体和碰撞网格，通过可微模拟从观察到的遮挡前轨迹中估计摩擦和速度，并利用得到的SE(3)轨迹在整个遮挡期间定位物体高斯体。由于预测的轨迹满足刚体动力学的控制方程，它忠实地捕捉了运动学外推无法建模的接触事件（反弹、基于摩擦的减速、方向变化）。我们引入了一种质心轮廓损失，能够将位置梯度与外观噪声隔离，导致比光度监督低40%的轨迹误差。我们使用未在训练中使用的摄像头进行评估，这些摄像头在物体遮挡期间观察该物体。对合成场景的实验表明，PersistGS的性能优于恒定速度外推，提升了+2.46dB PSNR，并与真实轨迹的上限相差仅0.19dB。

View on arXiv Download PDF AI Translation

cs.CV / 63 / 2606.03490

TrAction: Action Recognition with Sparse Trajectories

TrAction：基于稀疏轨迹的动作识别

Meier, Jan F., Mueller, Felix B., Ecker, Alexander, Lüddecke, Timo

Abstract

Modern action recognition models operate on memory- and compute-intensive dense RGB video volumes and frequently exploit appearance and background shortcuts, for example, predicting actions from objects or scenes instead of characteristic motion. We investigate an efficient alternative input modality that is largely free of such biases by construction: sparse point trajectories. To this end, we develop a simple transformer architecture for 2.5D trajectory-based recognition together with a masked-trajectory pretraining, which we show to substantially improve downstream action recognition accuracy. Despite using only a fraction of the dense RGB input, our method reaches 45% top-1 on Something-Something V2 and 54% on EPIC-Kitchens-100, and surpasses V-JEPA on time-reversal sensitivity. More importantly, we find trajectory features to be complementary to state-of-the-art appearance-based features. Fusing our pretrained model with DINOv2 and V-JEPA 2 improves top-1 accuracy on Something-Something V2 by 8.7 and 1.6 points, respectively. Code: https://github.com/ecker-lab/TrAction

Chinese Translation

现代动作识别模型通常在内存和计算密集型的密集RGB视频数据上运行，并且经常利用外观和背景的捷径，例如，从物体或场景预测动作，而不是特征运动。我们研究了一种高效的替代输入方式，该方式在构建上基本上不受此类偏见的影响：稀疏点轨迹。为此，我们开发了一种简单的变换器架构，用于基于2.5D轨迹的识别，并结合了掩码轨迹的预训练，我们证明这显著提高了下游动作识别的准确性。尽管仅使用了密集RGB输入的一小部分，我们的方法在Something-Something V2上达到了45%的top-1准确率，在EPIC-Kitchens-100上达到了54%，并在时间反转敏感性上超越了V-JEPA。更重要的是，我们发现轨迹特征与最先进的基于外观的特征是互补的。将我们的预训练模型与DINOv2和V-JEPA 2融合，分别提高了Something-Something V2的top-1准确率8.7和1.6个百分点。代码：https://github.com/ecker-lab/TrAction

View on arXiv Download PDF AI Translation

cs.CV / 64 / 2606.03493

Low-Frequency Shortcuts in Texture-Driven Visual Learning

纹理驱动视觉学习中的低频捷径

Şirin, Utku, Hou, Cathy, Alvarez-Melis, David, Idreos, Stratos

Abstract

Neural networks suffer from shortcut learning, where learned features generalize well to the training set but not to in-distribution (ID) or out-of-distribution (OOD) test sets. Existing studies are all based on a few standard benchmarks, which are shape-driven. Numerous application domains, however, are texture-driven. In this work, we present shortcut learning analysis for texture-driven domains, and compare it with that of a standard benchmark. We show that texture-driven domains suffer from low-frequency shortcuts. They make the majority of their decisions based on a few low-frequency components (LFCs) with a skewed spectral behavior, despite that their classification information is in higher-frequency, fine-grained details. Pruning LFCs from training and test sets eliminates the shortcut and provides a more balanced spectral behavior, improving the ID accuracy by up to 8%. We show that low-frequency shortcuts make the models highly vulnerable to OOD corruptions, leading up to 70% accuracy drop compared to the ID accuracy. Pruning LFCs significantly improves robustness to low-frequency corruptions, by up to 40%, and introduces a trade-off for high-frequency corruptions; the balanced spectral behavior provides a better generalization performance, whereas the increased dependence on high-frequency features reduces it. OOD accuracy depends on the interaction between these two factors.

Chinese Translation

神经网络面临捷径学习的问题，即学习到的特征在训练集上表现良好，但在同分布（ID）或异分布（OOD）测试集上却表现不佳。现有研究均基于一些标准基准，这些基准主要是形状驱动的。然而，许多应用领域是纹理驱动的。在本研究中，我们对纹理驱动领域的捷径学习进行了分析，并将其与标准基准进行了比较。我们发现纹理驱动领域存在低频捷径。尽管其分类信息存在于高频、细粒度的细节中，但它们的决策主要基于少数具有偏斜谱行为的低频成分（LFCs）。从训练和测试集中去除LFCs消除了捷径，并提供了更平衡的谱行为，使ID准确率提高了多达8%。我们表明，低频捷径使模型对OOD干扰高度脆弱，与ID准确率相比，准确率下降可达70%。去除LFCs显著提高了对低频干扰的鲁棒性，提升幅度可达40%，并为高频干扰引入了权衡；平衡的谱行为提供了更好的泛化性能，而对高频特征的依赖增加则降低了泛化性能。OOD准确率依赖于这两个因素之间的相互作用。

View on arXiv Download PDF AI Translation

cs.CV / 65 / 2606.03499

Characterizing Detectability in 3DGS Poisoning: A Stage-wise Benchmark

3DGS 中可检测性的特征化：阶段性基准

Bui-Huynh, Quoc-Anh, Ngo, Thanh Duc, Geng, Xue, Xu, Kaixin, Zhe, Wang, Yang, Xulei, Cheung, Ngai-Man

Abstract

3D Gaussian Splatting (3DGS) has rapidly emerged as a leading representation for real-time novel view synthesis, but recent work shows it is vulnerable to diverse poisoning attacks, including illusory object injection, computation cost amplification, and post hoc model watermarking. Despite this expanding threat surface, existing studies focus mainly on attack success, while defense and detection remain underexplored. From a detection perspective, a key challenge and opportunity arise from the multi-stage nature of the 3DGS reconstruction pipeline, which produces heterogeneous intermediate representations. Forensic signals for detecting poisoning are inherently stage dependent: an attack introduced at one stage may produce signals that emerge only at later stages. This motivates a stage-wise view of detectability that goes beyond single-stage evaluation. We introduce Poison-3DGS, a benchmark for stage-wise characterization of poisoning detection in 3DGS. It exposes stage-specific artifacts, including multi-view images, geometry, training dynamics, and Gaussian parameters, across a diverse set of scenes and attacks. Using it, we conduct a systematic study of detectability across pipeline stages. Our analysis reveals several insights. First, detectability varies significantly across stages, and no single stage consistently dominates across attack types. Second, different attacks exhibit distinct stage-specific forensic signals, so detection effectiveness depends critically on where signals are observed. Third, later-stage signals such as training dynamics and Gaussian parameter statistics provide strong cues not observable at earlier stages. Overall, our work provides a principled benchmark and the first systematic characterization of stage-dependent detectability in 3DGS, offering a foundation for future research on robust and reliable 3DGS systems.

Chinese Translation

3D 高斯溅射（3D Gaussian Splatting，3DGS）迅速成为实时新视图合成的主要表示方法，但近期研究表明它对多种中毒攻击（如虚幻物体注入、计算成本放大和事后模型水印）存在脆弱性。尽管这一威胁面不断扩大，现有研究主要集中于攻击成功率，而防御和检测仍未得到充分探讨。从检测的角度来看，3DGS 重建管道的多阶段特性带来了关键挑战和机遇，因为它产生了异构的中间表示。用于检测中毒的法医信号本质上依赖于阶段：在某一阶段引入的攻击可能产生仅在后续阶段显现的信号。这促使我们从阶段性视角来看待可检测性，超越单一阶段的评估。我们引入了 Poison-3DGS，这是一个用于3DGS中毒检测阶段性特征化的基准。它揭示了阶段特定的伪影，包括多视图图像、几何形状、训练动态和高斯参数，覆盖了多样的场景和攻击。利用该基准，我们对管道各阶段的可检测性进行了系统研究。我们的分析揭示了几个见解。首先，可检测性在不同阶段之间差异显著，且没有单一阶段在所有攻击类型中始终占据主导地位。其次，不同攻击表现出独特的阶段特定法医信号，因此检测效果在很大程度上依赖于信号的观察位置。第三，后期信号如训练动态和高斯参数统计提供了在早期阶段不可观察的强线索。总体而言，我们的工作提供了一个原则性的基准，并首次系统性地特征化了3DGS中阶段依赖的可检测性，为未来关于稳健和可靠的3DGS系统的研究奠定了基础。

View on arXiv Download PDF AI Translation

cs.CV / 66 / 2606.03506

AvatarMix: Identity-Preserving Cross-Avatar Composition for Outfit Personalization

AvatarMix：保留身份的跨头像服装个性化组合

Wang, Zhaorong, Kanamori, Yoshihiro, Endo, Yuki

Abstract

Existing 3D avatar outfit transfer methods face distinct challenges: approaches that lift 2D edits to 3D often suffer from outfit or identity quality degradation, while those that separately model body and clothing layers are prone to intersection artifacts. We introduce AvatarMix, a compositional paradigm that bypasses these issues by directly composing the head and body from two high-fidelity Gaussian avatars. While this paradigm inherently preserves outfit quality and avoids intersections, it introduces challenges in creating a seamless join and maintaining appearance fidelity after body reshaping. To this end, we propose a two-tier refinement strategy: SeamFix, a localized diffusion module that refines hair and neck to ensure an artifact-free join, and an optional full-body refinement, FullbodyFix, that restores garment appearance when retargeting degrades the clothed body. Both operate on renders from an already 3D-consistent Gaussian avatar, which limits multi-view artifacts compared to 2D-to-3D lifting. To preserve the user's body identity, our mesh-based Gaussian representation enables the adaptation of a robust mesh retargeting technique, precisely reshaping the clothed body to the user's physique and robustly handling diverse body shapes. Extensive experiments demonstrate that our method achieves state-of-the-art results in outfit fidelity and identity preservation, providing a new perspective for realistic 3D outfit personalization. Project page: https://larsph.github.io/avatarmix/

Chinese Translation

现有的3D头像服装转移方法面临着明显的挑战：将2D编辑提升到3D的方法往往会导致服装或身份质量下降，而那些单独建模身体和服装层的方法则容易出现交叉伪影。我们提出了AvatarMix，一种组合范式，通过直接组合两个高保真高斯头像的头部和身体，绕过了这些问题。尽管这一范式本质上保留了服装质量并避免了交叉，但在创建无缝连接和在身体重塑后保持外观保真度方面引入了挑战。为此，我们提出了一种两级细化策略：SeamFix，一个局部扩散模块，细化头发和脖子以确保无伪影的连接，以及一个可选的全身细化模块FullbodyFix，当重定向导致穿衣身体外观下降时恢复服装外观。这两者均在已经3D一致的高斯头像的渲染图上操作，相较于2D到3D的提升，限制了多视角伪影。为了保留用户的身体身份，我们的基于网格的高斯表示使得能够适应一种强大的网格重定向技术，精确地将穿衣身体重塑为用户的体型，并稳健地处理多样的身体形状。大量实验表明，我们的方法在服装保真度和身份保留方面达到了最先进的结果，为逼真的3D服装个性化提供了新的视角。项目页面：https://larsph.github.io/avatarmix/

View on arXiv Download PDF AI Translation

cs.CV / 67 / 2606.03508

Structure-Guided Mixed Masked Pretraining and Spatial Continuity Regularization for Printed Circuit Board Defect Detection

基于结构引导的混合掩码预训练与空间连续性正则化的印刷电路板缺陷检测

Wang, Peitong, Wang, Nuo, Qin, Enxin, Yu, Chengjin, Xuan, Hanyu, Yan, Yuanting

Abstract

Printed circuit board (PCB) defect detection is an essential part of automated optical inspection (AOI); yet it remains challenging in practice because many defects are tiny, low-contrast, and embedded in dense circuit backgrounds. To address these issues, this paper presents a two-phase PCB defect detection framework that combines structure-guided mixed masked pretraining with spatial continuity regularization. In the pretraining stage, we design a sparse convolutional masked pretraining scheme to exploit unlabeled PCB images, where structure-guided mixed masking is used to construct informative masked inputs. The sparse convolutional reconstruction pipeline suppresses invalid responses from masked regions and enables the detector backbone to infer missing PCB structures from visible conductive patterns, thereby learning PCB structural priors. In the fine-tuning stage, the pretrained backbone is transferred to the downstream defect detection task. For the task, a spatial continuity regularization term is introduced during fine-tuning. This term constrains dispersed positive predictions assigned to the same defect instance and promotes more compact localization on elongated defect regions. Experiments on the DsPCBSD+ dataset show that the proposed method achieves 85.5% mAP0.5 and 52.3% mAP0.5:0.95, outperforming several strong baseline detectors. Ablation studies and qualitative results further confirm the effectiveness of the proposed framework for robust PCB defect detection in industrial AOI scenarios.

Chinese Translation

印刷电路板（PCB）缺陷检测是自动光学检测（AOI）中至关重要的一部分；然而，由于许多缺陷微小、对比度低且嵌入在密集的电路背景中，实际操作中仍然面临挑战。为了解决这些问题，本文提出了一种结合结构引导的混合掩码预训练与空间连续性正则化的两阶段PCB缺陷检测框架。在预训练阶段，我们设计了一种稀疏卷积掩码预训练方案，以利用未标记的PCB图像，其中使用结构引导的混合掩码构建信息丰富的掩码输入。稀疏卷积重建管道抑制了来自掩码区域的无效响应，使检测器骨干能够从可见的导电图案中推断缺失的PCB结构，从而学习PCB结构先验。在微调阶段，预训练的骨干被转移到下游缺陷检测任务中。对于该任务，在微调过程中引入了空间连续性正则化项。该项约束分散的正预测分配给同一缺陷实例，并促进对延长缺陷区域的更紧凑定位。在DsPCBSD+数据集上的实验表明，所提出的方法实现了85.5%的mAP0.5和52.3%的mAP0.5:0.95，超越了多个强基线检测器。消融研究和定性结果进一步确认了所提出框架在工业AOI场景中进行鲁棒PCB缺陷检测的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 68 / 2606.03509

EvoMemNav: Efficient Self-Evolving Fine-Grained Memory for Zero-Shot Embodied Navigation

EvoMemNav：高效自演化的细粒度记忆用于零-shot 具身导航

Ge, Zuhao, Jia, Xiaosong, Wu, Chao, Zhou, Yuchen, Wu, Zuxuan, Jiang, Yu-Gang

Abstract

Building memory is essential for long-horizon planning in zero-shot embodied navigation. Detector-centric scene graphs often compress observations into sparse nodes, discarding fine-grained visual evidence and accumulating noise, while 3D reconstruction-based methods remain computationally prohibitive. We present EvoMemNav, an efficient, self-evolving, fine-grained memory framework for zero-shot embodied navigation. EvoMemNav constructs a Visual-Semantic Memory Graph (VSMGraph) that keeps raw views as first-class memory and organizes them with lightweight semantic cues and topological relations into a room-view-object hierarchy, preserving fine-grained details for disambiguation and Stop verification. To scale to growing memory, we introduce a budgeted coarse-to-fine policy: a coarse stage compresses the search space into promising regions, and a fine stage invokes a VLM only for targeted verification and decision. Beyond static memories, EvoMemNav performs reflection-driven write-back after each subtask, updating graph-attached priors that encode accumulated environmental knowledge to refine future decisions without retraining. Experiments on GOAT-Bench and HM3D across object, text-description, and image-goal modalities show consistent gains in SR/SPL, with better multi-instance disambiguation, fewer premature stops, and stronger zero-shot generalization.

Chinese Translation

构建记忆对于零-shot 具身导航中的长时间规划至关重要。以检测器为中心的场景图通常将观察结果压缩为稀疏节点，丢弃细粒度的视觉证据并积累噪声，而基于3D重建的方法则在计算上仍然过于昂贵。我们提出了EvoMemNav，一种高效的自演化细粒度记忆框架，用于零-shot 具身导航。EvoMemNav构建了一个视觉-语义记忆图（Visual-Semantic Memory Graph, VSMGraph），将原始视图作为一类重要的记忆，并通过轻量级的语义线索和拓扑关系将其组织成房间-视图-对象的层级结构，保留细粒度的细节以便于消歧和停止验证。为了扩展不断增长的记忆，我们引入了一种预算化的粗到细策略：粗阶段将搜索空间压缩到有前景的区域，而细阶段仅在针对性验证和决策时调用视觉语言模型（VLM）。除了静态记忆，EvoMemNav在每个子任务后执行反思驱动的回写，更新附加于图的先验知识，以编码累积的环境知识，从而在不重新训练的情况下优化未来的决策。在GOAT-Bench和HM3D上进行的实验显示，在对象、文本描述和图像目标模式下，SR/SPL均有一致的提升，且在多实例消歧、减少过早停止和增强零-shot 泛化能力方面表现更佳。

View on arXiv Download PDF AI Translation

cs.CV / 69 / 2606.03539

Knowledge-Preserved Model Tuning in Null-Space for Robust Spatio-Temporal Video Grounding

在零空间中保持知识的模型调优用于鲁棒的时空视频定位

Chen, Haoxuan, Liu, Xianqin, Hu, Jian-Fang

Abstract

Spatio-Temporal Video Grounding aims to localize object tubes based on textual queries. While recent methods have achieved remarkable success, they mainly focus on high-quality(HQ) inputs, neglecting the widespread presence of low-quality(LQ) videos in real-world scenarios. Although tuning methods like LoRA can adapt to degraded inputs, they inevitably disrupt pre-trained knowledge. To address this, we propose Null-Space Tuning (NST). This framework exploits the geometric property that adding vectors within the null-space of frozen weights to the layer input does not affect the output. Leveraging this, NST injects learnable residuals into input features that can be selectively invisible to the pre-trained backbone. Specifically, NST combines the Quality-Adaptive Unit and Dual-Space Reparameterization to synthesize these residuals by confining components for HQ inputs to the null-space, while directing restoration components for LQ inputs to the non-null space. As the frozen weights eliminate null-space components, we effectively rectify degraded inputs while preserving pre-trained knowledge for HQ inputs. Extensive experiments show that NST outperforms state-of-the-art methods on our Mixed-Quality benchmark.

Chinese Translation

时空视频定位旨在基于文本查询定位物体管道。尽管近期的方法取得了显著成功，但它们主要集中于高质量（HQ）输入，忽视了现实场景中低质量（LQ）视频的普遍存在。尽管像 LoRA 这样的调优方法可以适应退化的输入，但它们不可避免地会破坏预训练知识。为了解决这个问题，我们提出了零空间调优（NST）。该框架利用几何特性，即在冻结权重的零空间内添加向量到层输入不会影响输出。借助这一点，NST 将可学习的残差注入到输入特征中，这些残差可以选择性地对预训练主干不可见。具体而言，NST 结合了质量自适应单元和双空间重参数化，通过将 HQ 输入的组件限制在零空间内，同时将 LQ 输入的恢复组件引导到非零空间，来合成这些残差。由于冻结权重消除了零空间组件，我们有效地修正了退化的输入，同时为 HQ 输入保留了预训练知识。大量实验表明，NST 在我们的混合质量基准上优于最先进的方法。

View on arXiv Download PDF AI Translation

cs.CV / 70 / 2606.03540

Attend to Anything: Foundation Model for Unified Human Attention Modeling

关注任何事物：统一人类注意力建模的基础模型

Zhao, Wenzhuo, Xian, Ronghao, Fu, Keren, Zhao, Qijun

Abstract

Existing human attention (saliency) modeling methods persist as highly fragmented across modalities, scenes, and task formulations. Consequently, even with increasing model capacity and data scale, current models predominantly remain scene-dependent and task-specific, failing to practically generalize in real-world applications. To address the fundamental limitations, we present the Attend to Anything Model (AAM), a multi-modal foundation model that unifies attention modeling across various image, video, and audio-visual tasks and scenes. AAM reformulates attention as a cognitive entailment relationship organized in a general-to-specific hierarchy, implemented through language prompts with hierarchical embeddings in hyperbolic space. Furthermore, to unify static image and dynamic video attention, we adopt a fluid-dynamics perspective, formulating video-frame attention as a diffusive temporal evolution governed by the Fokker--Planck equation. Extensive experiments on 16 benchmarks demonstrate that AAM consistently outperforms state-of-the-art methods by an average of 6\% across various scenarios, while achieving approximately a 4$\times$ speedup in video inference. Overall, these results demonstrate that AAM provides a principled foundation for future research on attention and saliency-related tasks. The dataset and code will be available at https://github.com/wz-zhao/Attend-to-Anything.

Chinese Translation

现有的人类注意力（显著性）建模方法在不同模态、场景和任务表述上依然高度碎片化。因此，即使模型容量和数据规模不断增加，当前模型仍主要依赖于特定场景和任务，未能在实际应用中有效泛化。为了解决这一根本性限制，我们提出了关注任何事物模型（Attend to Anything Model, AAM），这是一个多模态基础模型，统一了各种图像、视频和视听任务及场景的注意力建模。AAM将注意力重新表述为一种认知蕴涵关系，组织成从一般到具体的层级，通过在双曲空间中使用层级嵌入的语言提示来实现。此外，为了统一静态图像和动态视频的注意力，我们采用流体动力学的视角，将视频帧注意力表述为受Fokker-Planck方程支配的扩散时间演化。在16个基准测试上的广泛实验表明，AAM在各种场景中平均超越最先进的方法6%，同时在视频推理中实现了约4倍的加速。总体而言，这些结果表明AAM为未来在注意力和显著性相关任务的研究提供了一个有原则的基础。数据集和代码将发布在 https://github.com/wz-zhao/Attend-to-Anything。

View on arXiv Download PDF AI Translation

cs.CV / 71 / 2606.03564

\textsc{CR-Seg}: Attention-Guided and CoT-Enhanced Coarse-to-Refined Reasoning Segmentation

CR-Seg：基于注意力引导和链式思维增强的粗到细推理分割

Cao, Yifan, Yang, Xiaocui, Wan, Faxian, Feng, Shi, Wang, Daling, Zhang, Yifei

Abstract

Reasoning segmentation aims to segment target objects described by complex language through joint visual-textual reasoning. Existing methods typically rely on either learned semantic tokens to bridge Multimodal Large Language Models (MLLMs) and segmentation models, suffering from difficult cross-modal alignment, or explicit spatial prompts such as bounding boxes, which may lose holistic response semantics. To address these limitations, we propose Attention-Guided and CoT-Enhanced Coarse-to-Refined Reasoning Segmentation, termed CR-Seg, a two-stage framework for coarse-to-refined reasoning segmentation. Specifically, we design an Extract Attention Maps and Points (EAP) module to extract attention maps for coarse target localization and select informative points, both of which are fed into SAM for mask refinement. To alleviate reasoning--answer inconsistency, we further introduce Global-to-Local Chain-of-Thought (GLCoT), which guides the model to reason progressively from global scene context to local target details. Extensive experiments on reasoning segmentation benchmarks demonstrate the effectiveness of CR-Seg.

Chinese Translation

推理分割旨在通过联合视觉-文本推理对复杂语言描述的目标对象进行分割。现有方法通常依赖于学习的语义标记来桥接多模态大型语言模型（MLLMs）和分割模型，这导致跨模态对齐困难，或者依赖于显式的空间提示，如边界框，这可能会丧失整体响应语义。为了解决这些局限性，我们提出了基于注意力引导和链式思维增强的粗到细推理分割，称为CR-Seg，这是一种用于粗到细推理分割的两阶段框架。具体而言，我们设计了一个提取注意力图和点（EAP）模块，以提取用于粗略目标定位的注意力图并选择信息丰富的点，这些都被输入到SAM中进行掩膜细化。为了缓解推理与答案之间的不一致性，我们进一步引入了全局到局部链式思维（GLCoT），引导模型从全局场景上下文逐步推理到局部目标细节。在推理分割基准上的大量实验表明，CR-Seg的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 72 / 2606.03566

Efficient Transformer-Based Localized Patch Sampling for Choroid Plexus Segmentation in Multiple Sclerosis

基于高效变换器的局部补丁采样用于多发性硬化症中的脉络丛分割

Lu, Po-Jui, Cagol, Alessandro, Ocampo-Pineda, Mario, Spagnolo, Federico, Mastantuono, Marina, Aldea, Andreea-Alexandra, Müller, Jannis, Yaldizli, Özgür, Weigel, Matthias, Melie-Garcia, Lester, Magliozzi, Roberta, Sormani, Maria Pia, Kappos, Ludwig, Kuhle, Jens, Granziera, Cristina

Abstract

Background: The lateral ventricle choroid plexus (LVCP) is gaining recognition as a key imaging biomarker for multiple sclerosis (MS) related to physical disability and neuroinflammation. Yet, manual segmentation of the LVCP is highly tedious, restricting its use in broad clinical trials and longitudinal assessments. This research aims to develop a SwinUNETR-driven pipeline that leverages targeted intra- and peri-ventricular small patch sampling to automatically segment the LVCP in MS from both standalone and multi-modal MRI inputs. Methods: We retrospectively assessed 3T MRI scans across three sets of data stemming from two separate MS-dominant cohorts (Dataset 1: n=177; Dataset 2: n=177; expanded test set: n=388). Our method employed a SwinUNETR architecture trained on 32x32x32 voxel patches, benchmarking it against the 3D UXNET model. The primary metric for evaluation was the Dice Similarity Coefficient (DSC), supplemented by computational demand (GFLOPs) and the 95th percentile Hausdorff Distance (HD95). Results: On the extended test set, the SwinUNETR model secured a mean DSC of 0.868 (95% CI: 0.863-0.872) with MPRAGE and FLAIR combined, showing a statistically significant gain over UXNET (DSC: 0.858 [95% CI: 0.853-0.862], p<0.0001). When restricted to standalone FLAIR inputs, the transformer-based approach sustained a high DSC of 0.863, while the spatial localization of UXNET worsened considerably (HD95: 1.86 vs. 3.00 mm). Importantly, the proposed framework lowered computational load by 99% (91.8 vs. 22,080 GFLOPs). By integrating localized patch sampling with a SwinUNETR architecture, this methodology offers an accurate, robust, and statistically superior alternative to current leading models for LVCP segmentation. Its vast reduction in computational cost makes it ideal for widespread implementation in clinical and research environments.

Chinese Translation

背景：侧脑室脉络丛（LVCP）正逐渐被认可为与多发性硬化症（MS）相关的身体残疾和神经炎症的关键影像生物标志物。然而，LVCP的手动分割非常繁琐，限制了其在广泛临床试验和纵向评估中的应用。本研究旨在开发一个基于SwinUNETR的管道，利用针对性脑室内和脑室周围的小补丁采样，自动分割MS中的LVCP，支持独立和多模态MRI输入。方法：我们回顾性评估了来自两个独立MS主导队列的三组数据的3T MRI扫描（数据集1：n=177；数据集2：n=177；扩展测试集：n=388）。我们的方法采用了在32x32x32体素补丁上训练的SwinUNETR架构，并与3D UXNET模型进行了基准比较。主要评估指标为Dice相似系数（DSC），辅以计算需求（GFLOPs）和第95百分位Hausdorff距离（HD95）。结果：在扩展测试集上，SwinUNETR模型在结合MPRAGE和FLAIR时获得了平均DSC为0.868（95% CI：0.863-0.872），显示出相较于UXNET（DSC：0.858 [95% CI：0.853-0.862]，p<0.0001）具有统计学显著提升。当仅限于独立FLAIR输入时，基于变换器的方法维持了高达0.863的DSC，而UXNET的空间定位显著恶化（HD95：1.86 vs. 3.00 mm）。重要的是，所提出的框架将计算负担降低了99%（91.8 vs. 22,080 GFLOPs）。通过将局部补丁采样与SwinUNETR架构相结合，该方法为LVCP分割提供了一种准确、稳健且统计上优越的替代方案，其计算成本的大幅降低使其在临床和研究环境中的广泛应用成为可能。

View on arXiv Download PDF AI Translation

cs.CV / 73 / 2606.03568

Learned Non-Maximum Suppression for 3D Object Detection

用于3D目标检测的学习型非极大值抑制

Osterburg, Timo, Schütte, Stefan, Bertram, Torsten

Abstract

Post-processing is a critical stage in LiDAR-based 3D object detection, where dense and overlapping proposals must be filtered for compact and reliable perception. This work introduces two learned filtering modules that replace heuristic non-maximum suppression (NMS) by leveraging relations among detections. D2D-Rescore employs transformer-based detection-to-detection (D2D) attention, while GossipNet3D adapts the 2D GossipNet concept to 3D through localized message passing in bird's-eye view. A metric-aware matching strategy aligned with the nuScenes evaluation protocol ensures consistent training and validation behavior, improving overall detection performance. Both approaches improve mean average precision (mAP), nuScenes detection score (NDS), and true positive quality compared to CircleNMS, particularly for small and infrequent classes, while adding minimal computational overhead. These results demonstrate that learned, detection-level filtering can enhance 3D detector reliability without modifying the base network, offering a principled alternative to heuristic suppression. Code is available at https://github.com/rst-tu-dortmund/learned-3d-nms .

Chinese Translation

后处理是基于激光雷达的3D目标检测中的一个关键阶段，在此阶段，必须过滤密集且重叠的提议，以实现紧凑且可靠的感知。本研究引入了两个学习型过滤模块，通过利用检测之间的关系来替代启发式的非极大值抑制（NMS）。D2D-Rescore采用基于变换器的检测到检测（D2D）注意力机制，而GossipNet3D则通过在鸟瞰图中进行局部消息传递，将2D GossipNet概念适应于3D。与nuScenes评估协议对齐的度量感知匹配策略确保了一致的训练和验证行为，从而提高了整体检测性能。与CircleNMS相比，这两种方法在平均精度均值（mAP）、nuScenes检测分数（NDS）和真实正例质量方面均有所提升，尤其是在小型和不常见类别中，同时增加的计算开销极小。这些结果表明，学习型检测级过滤可以增强3D检测器的可靠性，而无需修改基础网络，为启发式抑制提供了一个有原则的替代方案。代码可在 https://github.com/rst-tu-dortmund/learned-3d-nms 获取。

View on arXiv Download PDF AI Translation

cs.CV / 74 / 2606.03569

When Attention Collapses: Stage-Aware Visual Token Pruning from Structure to Semantics

当注意力崩溃时：从结构到语义的阶段感知视觉标记剪枝

Wang, Jiahui, Zhang, Kai, Han, Mai, Zhang, Huanghe

Abstract

Vision-Language Models (VLMs) have demonstrated remarkable capabilities but suffer from significant computational overhead during inference. While visual token pruning offers a promising solution, existing methods predominantly rely on initial attention scores. This single-metric paradigm presents a critical flaw: high attention scores inherently collapse onto semantically similar regions, thereby severely reducing feature diversity and discarding vital contextual details. To address this, we introduce Structure-to-Semantics (STS), a novel two-stage visual token pruning framework that explicitly decouples the pruning process. The first stage employs a repulsion-based sampling mechanism to maximize spatial and structural diversity. The second stage leverages instruction-aware cross-attention to precisely filter out prompt-irrelevant tokens. This two-stage synergy constitutes the core of STS, first ensuring geometric coverage and then refining the retained tokens according to semantic relevance. Extensive evaluations demonstrate that STS mitigates the redundancy caused by attention-based selection, improving both structural diversity and fine-grained task alignment of the preserved visual tokens.

Chinese Translation

视觉-语言模型（VLMs）展现了卓越的能力，但在推理过程中面临着显著的计算开销。尽管视觉标记剪枝提供了一个有前景的解决方案，但现有方法主要依赖于初始注意力得分。这种单一指标范式存在一个关键缺陷：高注意力得分本质上会集中在语义相似的区域，从而严重减少特征多样性并丢弃重要的上下文细节。为了解决这个问题，我们提出了结构到语义（Structure-to-Semantics, STS）这一新颖的两阶段视觉标记剪枝框架，明确解耦剪枝过程。第一阶段采用基于排斥的采样机制，以最大化空间和结构多样性。第二阶段利用指令感知的交叉注意力，精确过滤掉与提示无关的标记。这一两阶段的协同构成了STS的核心，首先确保几何覆盖，然后根据语义相关性精炼保留的标记。大量评估表明，STS减轻了基于注意力选择所造成的冗余，提高了保留视觉标记的结构多样性和细粒度任务对齐。

View on arXiv Download PDF AI Translation

cs.CV / 75 / 2606.03577

Eliciting Complex Spatial Reasoning in MLLMs through Wide-Baseline Matching

通过宽基线匹配引发多模态大语言模型中的复杂空间推理

Zhong, Hao, Zhu, Muzhi, Zeng, Shenyan, Li, Anzhou, Chen, Cong, Geng, Hua, Shi, Duochao, Ye, Wentao, Lin, Tao, Chen, Hao, Shen, Chunhua

Abstract

Wide-baseline matching (WBM) requires integrating geometric understanding, viewpoint changes, fine-grained perception, and occlusion reasoning, making it a challenging testbed for spatial reasoning in multimodal large language models (MLLMs) deployed in physical environments. However, current MLLMs lack systematic evaluation and training frameworks for these capabilities. We introduce ReasonMatch-Bench, a benchmark stratified by viewpoint displacement and matching granularity across indoor, outdoor, and object-centric scenarios, and show that current MLLMs still struggle with fine-grained wide-baseline correspondence: on a difficult 90-sample subset, human annotators achieve 84.0 F1, while the best existing baseline reaches 37.2. To bridge this gap, we build a scalable data-generation pipeline that automatically extracts wide-baseline view pairs from large-scale video-3D corpora, including RGB-D videos and SfM reconstructions, yielding diverse and verifiable supervision. We further propose Dynamic Correspondence Reinforcement Learning (DCRL), which combines Image-Level Viewpoint Progression and Point-Level Correspondence Curriculum to improve WBM training through verifiable rewards without explicit CoT supervision. Extensive experiments show that DCRL substantially improves ReasonMatch-Bench and transfers to related spatial benchmarks, while maintaining general visual understanding performance with modest gains on several benchmarks.

Chinese Translation

宽基线匹配（WBM）需要整合几何理解、视角变化、细粒度感知和遮挡推理，使其成为在物理环境中部署的多模态大语言模型（MLLMs）进行空间推理的一个具有挑战性的测试平台。然而，当前的MLLMs缺乏对这些能力的系统评估和训练框架。我们引入了ReasonMatch-Bench，这是一个基于视角位移和匹配粒度分层的基准，涵盖室内、室外和以物体为中心的场景，并显示当前的MLLMs在细粒度宽基线对应方面仍然存在困难：在一个困难的90样本子集中，人类标注者的F1得分为84.0，而现有最佳基线仅达到37.2。为了弥补这一差距，我们构建了一个可扩展的数据生成管道，自动从大规模视频-3D语料库中提取宽基线视图对，包括RGB-D视频和结构从运动（SfM）重建，提供多样且可验证的监督。我们进一步提出了动态对应强化学习（Dynamic Correspondence Reinforcement Learning, DCRL），结合图像级视角进展和点级对应课程，通过可验证的奖励改善WBM训练，而无需显式的链式推理（CoT）监督。大量实验表明，DCRL显著改善了ReasonMatch-Bench，并能够迁移到相关的空间基准，同时在多个基准上保持一般视觉理解性能，并取得适度的提升。

View on arXiv Download PDF AI Translation

cs.CV / 76 / 2606.03578

Diffusing in the Right Space: A Systematic Study of Latent Diffusability

在正确的空间中扩散：潜在扩散性系统研究

Zhong, Tianxiong, Tian, Xingye, Wang, Xuebo, Tao, Xin, Wan, Pengfei

Abstract

Latent diffusion models leverage visual tokenizers to compress images into latent spaces for efficient generative modeling. However, better reconstruction quality of a tokenizer does not necessarily translate into better generation quality, suggesting that latent representations should be evaluated not only by fidelity but also by their diffusability. Recent studies have proposed diverse explanations for diffusion-friendly latent spaces, including semantic separability, affine equivariance, distribution uniformity, spatial structure, spectral smoothness, and manifold continuity. Yet these properties are often validated on a limited set of tokenizers, leaving it unclear which factors are most predictive of downstream generation quality and whether such conclusions hold beyond the specific settings in which they are introduced. In this work, we conduct a systematic study of latent diffusability by training a large collection of tokenizers with diverse regularization strategies, architectures, and latent configurations, and evaluating them with multiple downstream diffusion backbones. Our analysis identifies several latent properties that consistently correlate with generation quality and exhibit strong generalization across experimental settings. Beyond existing metrics, we introduce Velocity Irreducible Variance (VIV), a measure of velocity ambiguity induced by trajectory crossings. Extensive experiments show that VIV is one of the most stable predictors of generation quality.

Chinese Translation

潜在扩散模型利用视觉标记器将图像压缩到潜在空间，以实现高效的生成建模。然而，标记器的重建质量更好并不一定意味着生成质量更高，这表明潜在表示的评估不仅应考虑保真度，还应考虑其扩散性。最近的研究提出了多种关于适合扩散的潜在空间的解释，包括语义可分性、仿射等变性、分布均匀性、空间结构、谱平滑性和流形连续性。然而，这些属性通常是在有限的标记器集上进行验证的，因此尚不清楚哪些因素最能预测下游生成质量，以及这些结论在引入的特定设置之外是否仍然成立。在本研究中，我们通过训练大量具有多样正则化策略、架构和潜在配置的标记器，并使用多个下游扩散骨干网络进行评估，系统地研究了潜在扩散性。我们的分析识别出几个与生成质量一致相关的潜在属性，并在实验设置中表现出强大的泛化能力。除了现有指标外，我们引入了速度不可约方差（Velocity Irreducible Variance, VIV），这是一个由轨迹交叉引起的速度模糊度的度量。大量实验表明，VIV是生成质量最稳定的预测因子之一。

View on arXiv Download PDF AI Translation

cs.CV / 77 / 2606.03581

UnsOcc: 3D Semantic Occupancy Prediction in Unstructured Scene via Rendering Fusion

UnsOcc：通过渲染融合在非结构化场景中进行3D语义占用预测

Wu, Ye, Song, Ruiqi, Ding, Baiyong, Zeng, Nanxin, Cheng, Junjie, Ai, Yunfeng

Abstract

Unstructured scenes present unique challenges for autonomous driving, as irregular obstacles and sparse scene layouts undermine the effectiveness of traditional perception methods such as 3D object detection. 3D semantic occupancy prediction has emerged as a prominent focus due to its ability to provide dense spatial representations by assigning semantic labels to individual voxels in 3D space. However, directly applying 3D semantic occupancy prediction to unstructured scenes remains challenging because scene sparsity hinders effective cross-modal fusion and the more severe long-tail distribution in these scenarios further degrades prediction performance. To validate the effectiveness of our approach, we construct a dedicated dataset of unstructured scenes collected from open-pit mines. Based on this, we propose UnsOcc, a multi-modal 3D semantic occupancy prediction framework that improves robustness in unstructured environments. At its core, we introduce a rendering-based fusion module, RenderFusion, which enhances cross-modal feature alignment through bidirectional rendering supervision. Furthermore, we propose GSRefinement, a detail-aware auxiliary supervision method based on Gaussian Splatting that projects sparse 3D occupancy predictions into dense 2D semantic segmentation maps, enabling effective supervision for long-tail categories. Extensive experiments on both the open-pit mine dataset and the nuScenes dataset demonstrate that our method significantly outperforms existing state-of-the-art approaches.

Chinese Translation

非结构化场景为自动驾驶带来了独特的挑战，因为不规则的障碍物和稀疏的场景布局削弱了传统感知方法（如3D物体检测）的有效性。3D语义占用预测因其能够通过为3D空间中的单个体素分配语义标签来提供密集的空间表示而成为一个重要的研究重点。然而，直接将3D语义占用预测应用于非结构化场景仍然具有挑战性，因为场景稀疏性阻碍了有效的跨模态融合，而这些场景中更为严重的长尾分布进一步降低了预测性能。为了验证我们方法的有效性，我们构建了一个专门的数据集，该数据集包含从露天矿场收集的非结构化场景。基于此，我们提出了UnsOcc，一个多模态3D语义占用预测框架，旨在提高非结构化环境中的鲁棒性。其核心是引入了一个基于渲染的融合模块RenderFusion，通过双向渲染监督增强跨模态特征对齐。此外，我们提出了GSRefinement，一种基于高斯溅射的细节感知辅助监督方法，该方法将稀疏的3D占用预测投影到密集的2D语义分割图中，从而为长尾类别提供有效的监督。在露天矿场数据集和nuScenes数据集上的大量实验表明，我们的方法显著优于现有的最先进方法。

View on arXiv Download PDF AI Translation

cs.CV / 78 / 2606.03603

World Models Meet Language Models: On the Complementarity of Concrete and Abstract Reasoning

世界模型与语言模型的结合：具体与抽象推理的互补性

Zhou, Yucheng, Tao, Wei, Guo, Yiwen, Shen, Jianbing

Abstract

World models and multimodal large language models (MLLMs) provide complementary capabilities for predicting future outcomes from static visual observations. World models can generate concrete visual rollouts of possible futures, while MLLMs can reason abstractly over questions, goals, and rules. However, generated rollouts are stochastic and may be visually plausible but task-incorrect, making it necessary to determine when visual simulation is useful, whether a rollout is credible, and how it should influence the final answer. We formulate this problem as controlled concrete reasoning, where a model learns to invoke, verify, and integrate visual future simulation alongside abstract reasoning. To study this setting, we construct two human-verified benchmarks, VRQABench for controllable spatial lookahead and OpenWorldQA for open-domain physical prediction, and propose Privileged-Future On-Policy Self-Distillation (PF-OPSD). During training, PF-OPSD uses ground-truth future videos and answers only as teacher-side privileged context to evaluate on-policy concrete-reasoning trajectories, while the deployable student never observes true futures at test time. Experimental results show that PF-OPSD outperforms baseline by 10.6% and 10.9% on VRQABench and OpenWorldQA, respectively, while increasing robustness to noisy or conflicting rollouts. Our code and dataset are available at https://github.com/yczhou001/PF-OPSD.

Chinese Translation

世界模型和多模态大型语言模型（MLLMs）为从静态视觉观察中预测未来结果提供了互补的能力。世界模型能够生成可能未来的具体视觉展望，而MLLMs则可以在问题、目标和规则上进行抽象推理。然而，生成的展望是随机的，可能在视觉上看起来合理，但在任务上却不正确，因此有必要确定视觉模拟何时有用、展望是否可信以及它应如何影响最终答案。我们将这个问题表述为受控的具体推理，其中模型学习调用、验证和整合视觉未来模拟与抽象推理。为了研究这一设置，我们构建了两个经过人工验证的基准，分别是用于可控空间前瞻的VRQABench和用于开放领域物理预测的OpenWorldQA，并提出了特权未来在线自蒸馏（Privileged-Future On-Policy Self-Distillation, PF-OPSD）。在训练过程中，PF-OPSD仅将真实未来视频和答案作为教师端特权上下文，用于评估在线具体推理轨迹，而可部署的学生在测试时从未观察到真实的未来。实验结果表明，PF-OPSD在VRQABench和OpenWorldQA上分别比基线提高了10.6%和10.9%的性能，同时增强了对噪声或冲突展望的鲁棒性。我们的代码和数据集可在https://github.com/yczhou001/PF-OPSD获取。

View on arXiv Download PDF AI Translation

cs.CV / 79 / 2606.03610

SkelHCC: A Hyperbolic CLIP-Driven Cache Adaptation Framework for Skeleton-based One-Shot Action Recognition

SkelHCC：一个基于超曲率CLIP驱动的骨架一-shot动作识别缓存适应框架

Liu, Yanan, Zhu, Anqi, Zhu, Jingmin, Liu, Jun, Rahmani, Hossein, Bennamoun, Mohammed, Boussaid, Farid, Xu, Dan, Ke, Qiuhong

Abstract

Skeleton-based action recognition aims to understand human behaviors from body joint sequences and is especially challenging in the one-shot setting, where only a single labeled exemplar is available for each novel action. A key challenge is learning representations that capture the hierarchical and compositional structure of human motion while aligning effectively with high-level action semantics under extreme data scarcity. Existing approaches, largely based on Euclidean embeddings and low-level motion cues, struggle to model the tree-like organization of skeleton data, limiting cross-modal alignment and generalization to unseen action categories. We propose SkelHCC, a unified skeleton hyperbolic CLIP-driven cache adaptation framework for one-shot skeleton-based action recognition. SkelHCC introduces an Explicitly Hierarchical Hyperbolic CLIP (EH-HCLIP) module that embeds skeleton sequences and action language into a shared hyperbolic space. By leveraging the negative curvature and exponential volume growth of hyperbolic geometry, EH-HCLIP naturally encodes the joint-part-body hierarchy of human anatomy and yields structurally consistent cross-modal representations. To support efficient one-shot adaptation, SkelHCC further integrates a training-free LLM-guided Multi-granularity Voting Cache (LMV-Cache) for context-aware inference. Experiments on NTU RGB+D 60, NTU RGB+D 120, and PKU-MMD demonstrate that SkelHCC consistently outperforms state-of-the-art methods.

Chinese Translation

基于骨架的动作识别旨在通过身体关节序列理解人类行为，在一-shot设置中尤其具有挑战性，因为每种新颖动作仅提供一个标记示例。一个关键挑战是学习能够捕捉人类运动的层次和组合结构的表征，同时在极端数据稀缺的情况下有效地与高层次动作语义对齐。现有方法主要基于欧几里得嵌入和低级运动线索，难以建模骨架数据的树状组织，限制了跨模态对齐和对未见动作类别的泛化。我们提出了SkelHCC，一个统一的基于骨架的超曲率CLIP驱动的一-shot动作识别缓存适应框架。SkelHCC引入了一个显式层次超曲率CLIP（EH-HCLIP）模块，将骨架序列和动作语言嵌入到共享的超曲率空间。通过利用超曲率几何的负曲率和指数体积增长，EH-HCLIP自然编码了人类解剖的关节-部分-身体层次，并产生结构一致的跨模态表征。为了支持高效的一-shot适应，SkelHCC进一步集成了一个无训练的LLM引导的多粒度投票缓存（LMV-Cache），用于上下文感知推理。在NTU RGB+D 60、NTU RGB+D 120和PKU-MMD上的实验表明，SkelHCC始终优于最先进的方法。

View on arXiv Download PDF AI Translation

cs.CV / 80 / 2606.03626

TurtleAI: Benchmarking Multimodal Models for Visual Programming in Turtle Graphics

TurtleAI：针对海龟图形中的视觉编程的多模态模型基准测试

Wen, Chao, Staub, Jacqueline, Singla, Adish

Abstract

Vision-language models (VLMs) have been explored for visual programming, where they generate code to solve visual tasks. However, most prior work focuses on visual programming for productivity; it remains unclear how well current VLMs perform on education-oriented visual programming and what factors limit their performance. To bridge this gap, we introduce TurtleAI, a benchmark containing 823 tasks curated based on real-world visual programming tasks in the Turtle Graphics domain. Solving these tasks requires models to perceive geometric patterns, reason about spatial relationships, and synthesize Python code that faithfully reproduces geometric patterns. We evaluate 20+ VLMs, including GPT-5, GPT-4o, and Qwen2-VL-72B, and find that they struggle significantly, with most achieving success rates below 30%. To address these limitations, we propose a data generation technique that requires only a small set of seed samples. Fine-tuning Qwen2-VL-72B on the resulting synthetic data yields an improvement of about 20% on real-world tasks. Our failure analysis reveals that GPT-4o struggles with spatial reasoning and precise visual replication, whereas fine-tuning primarily improves the alignment between visual reasoning and code implementation.

Chinese Translation

视觉-语言模型（VLMs）已被探索用于视觉编程，在此过程中，它们生成代码以解决视觉任务。然而，大多数先前的研究集中于生产力导向的视觉编程；目前尚不清楚现有的 VLMs 在教育导向的视觉编程中的表现如何，以及哪些因素限制了它们的性能。为了解决这一问题，我们引入了 TurtleAI，这是一个包含 823 个任务的基准，这些任务是基于海龟图形领域中的真实视觉编程任务精心策划的。解决这些任务需要模型感知几何图形、推理空间关系，并合成能够忠实再现几何图形的 Python 代码。我们评估了 20 多个 VLM，包括 GPT-5、GPT-4o 和 Qwen2-VL-72B，发现它们的表现显著不佳，大多数模型的成功率低于 30%。为了解决这些局限性，我们提出了一种仅需少量种子样本的数据生成技术。在生成的合成数据上微调 Qwen2-VL-72B，使其在真实任务上的表现提高了约 20%。我们的失败分析揭示了 GPT-4o 在空间推理和精确视觉复制方面的困难，而微调主要改善了视觉推理与代码实现之间的对齐。

View on arXiv Download PDF AI Translation

cs.CV / 81 / 2606.03635

VidMsg: A Benchmark for Implicit Message Inference in Short Videos

VidMsg：短视频中隐含信息推断的基准测试

Tzachor, Issar, Green, Michael, Ben-Ari, Rami

Abstract

Understanding short online videos involves more than identifying visible objects and actions; video makers often include an underlying message or purpose in the clip. We introduce VidMsg, a benchmark for evaluating implicit message understanding in short, internet-native video clips. VidMsg contains 400 YouTube-derived clips across 9 practical topic areas and 52 fine-grained target messages, covering domains such as career and finance, education, health and well-being, culture, safety, sustainability, and lifestyle. VidMsg is constructed through a message-first pipeline: an LLM first translates target messages into indirect search scenarios, which are used to retrieve candidate clips. Human annotators then retain clips that convey the intended message without being overly explicit. VidMsg is designed primarily for bidirectional message-clip retrieval for scalable applications such as video search and recommendation, where systems must capture holistic video understanding. In addition to retrieval, VidMsg includes a diagnostic multiple-choice QA benchmark, where models select the intended message of a clip from semantically related alternatives. Experiments with contemporary video-language and retrieval models show that strong models often fail on VidMsg, because the task requires pragmatic inference, integration of contextual cues, and discrimination among semantically close messages. We also introduce VidVec-Msg, a baseline method that improves message-oriented retrieval while leaving substantial headroom for future work.

Chinese Translation

理解短小的在线视频不仅仅是识别可见的物体和动作；视频制作者通常在片段中包含一个潜在的信息或目的。我们介绍了VidMsg，这是一个用于评估短小互联网原生视频片段中隐含信息理解的基准测试。VidMsg包含来自YouTube的400个视频片段，涵盖9个实际主题领域和52个细分目标信息，涉及职业与金融、教育、健康与福祉、文化、安全、可持续性和生活方式等领域。VidMsg是通过信息优先的流程构建的：一个大型语言模型（LLM）首先将目标信息转化为间接搜索场景，这些场景用于检索候选片段。然后，人类注释者保留那些传达预期信息而不过于明显的片段。VidMsg主要设计用于双向信息-片段检索，以支持视频搜索和推荐等可扩展应用，这些系统必须捕捉整体视频理解。除了检索，VidMsg还包括一个诊断性的多项选择问答基准，模型需要从语义相关的选项中选择片段的预期信息。与当代视频-语言和检索模型的实验表明，强大的模型在VidMsg上往往表现不佳，因为该任务需要实用推理、上下文线索的整合以及在语义相近信息之间的区分。我们还介绍了VidVec-Msg，这是一种基线方法，能够改善以信息为导向的检索，同时为未来的工作留出了相当大的发展空间。

View on arXiv Download PDF AI Translation

cs.CV / 82 / 2606.03646

A Benchmark for Semi-supervised Multi-modal Crowd Counting

半监督多模态人群计数基准

Meng, Haoliang, Hong, Xiaopeng, Wang, Yabin, Zuo, Wangmeng

Abstract

This paper constructs the first benchmark on semi-supervised multi-modal crowd counting. To lay the foundation for this unexplored task, we first formulate the semi-supervised multi-modal setting and a standardized protocol that specifies the labeled-unlabeled data partition across different labeled ratios. Next, to establish solid reference points, we carefully tailor a diverse set of representative baselines, including existing fully supervised multi-modal methods and semi-supervised single-modal methods. Then, we carefully evaluate their performance under our proposed benchmark. Codes and the data partition will be released on https://github.com/HenryCilence/Semi-supervised-Multimodal-Crowd-Counting.

Chinese Translation

本文构建了第一个关于半监督多模态人群计数的基准。为了为这一尚未探索的任务奠定基础，我们首先制定了半监督多模态设置和一个标准化协议，该协议规定了不同标记比例下的标记-未标记数据划分。接下来，为了建立可靠的参考点，我们精心定制了一组多样化的代表性基线，包括现有的完全监督的多模态方法和半监督的单模态方法。然后，我们在我们提出的基准下仔细评估了它们的性能。代码和数据划分将发布在 https://github.com/HenryCilence/Semi-supervised-Multimodal-Crowd-Counting。

View on arXiv Download PDF AI Translation

cs.CV / 83 / 2606.03654

Graph Regularized Non-negative Reduced Biquaternion Matrix Factorization for Color Image Recognition

用于彩色图像识别的图正则化非负降维双四元数矩阵分解

Wu, Hailang, Liu, Yonghe, Yu, Bingxuan, Li, Chaoqian

Abstract

Non-negative reduced biquaternion matrix factorization (NRBMF) uses the product of reduced biquaternion (RB) matrices to incorporate the non-negativity constraints of color image pixels into the factorization process. However, NRBMF mainly focuses on reconstruction accuracy and does not exploit the local geometric structure of image data, which may limit the discriminative ability of the learned low-dimensional features. To address this issue, we propose a graph regularized non-negative reduced biquaternion matrix factorization (GNRBMF) model for color image recognition. The proposed model incorporates a graph Laplacian regularizer into the reduced biquaternion coefficient matrix, encouraging nearby samples in the original space to have similar representations in the learned feature space. Meanwhile, GNRBMF retains the non-negativity-preserving property of NRBMF in the reduced biquaternion domain. To solve the optimization problem, a component-wise alternating projected gradient algorithm is derived, and its convergence properties are analyzed. Experimental results demonstrate that the proposed GNRBMF model achieves competitive or superior recognition performance in some tested settings.

Chinese Translation

非负降维双四元数矩阵分解（NRBMF）利用降维双四元数（RB）矩阵的乘积将彩色图像像素的非负性约束纳入分解过程。然而，NRBMF主要关注重建精度，而未能利用图像数据的局部几何结构，这可能限制了所学习的低维特征的区分能力。为了解决这一问题，我们提出了一种用于彩色图像识别的图正则化非负降维双四元数矩阵分解（GNRBMF）模型。该模型将图拉普拉斯正则化器引入降维双四元数系数矩阵，鼓励原始空间中相邻样本在学习的特征空间中具有相似的表示。同时，GNRBMF在降维双四元数域中保持了NRBMF的非负性保持特性。为了解决优化问题，我们推导出了一种分量交替投影梯度算法，并分析了其收敛性。实验结果表明，所提出的GNRBMF模型在某些测试设置中实现了具有竞争力或优越的识别性能。

View on arXiv Download PDF AI Translation

cs.CV / 84 / 2606.03666

Beyond Single Solution: Multi-Hypothesis Collaborative Deep Unfolding Network for Image Compressive Sensing

超越单一解：用于图像压缩感知的多假设协作深度展开网络

Cui, Wenxue, Li, Hualin, Qin, Yuhang, Xu, Yifu, Fan, Xiaopeng, Zhao, Debin

Abstract

Recent deep unfolding networks (DUNs) have advanced Compressive Sensing (CS) by effectively integrating iterative optimization with deep learning architectures. However, most CS approaches predominantly confine their inference to a single solution space, neglecting the inherent ill-posedness of CS problems that intrinsically permits multiple plausible candidate hypotheses. In this paper, a novel Multi-Hypothesis Collaborative Deep Unfolding CS Network (MHC-DUN) is proposed, which explicitly models and leverages multiple hypotheses by jointly optimizing across diverse solution spaces. Specifically, following the Proximal Gradient Descent algorithm, MHC-DUN jointly performs gradient descent and proximal mapping within this multi-hypothesis paradigm. i) For gradient descent, a well-designed AlphaNet is introduced to dynamically predict spatially varying step sizes for all hypotheses, enabling collaborative gradient updates across multiple solutions. ii) For proximal operator, a sophisticated multi-hypothesis collaborative proximal mapping module is designed, which leverages both intra-hypothesis and inter-hypothesis correlation priors to jointly refine multiple solutions. To enable end-to-end training, a novel composite loss function is designed, which balances measurement fidelity, hypothesis diversity, and reconstruction accuracy, encouraging exploration of complementary solutions while maintaining reconstruction fidelity. Experimental results reveal that the proposed CS method outperforms existing CS networks.

Chinese Translation

近年来，深度展开网络（DUNs）通过有效地将迭代优化与深度学习架构相结合，推动了压缩感知（CS）的发展。然而，大多数CS方法主要将推理限制在单一解空间，忽视了CS问题固有的不适定性，这种不适定性本质上允许多个合理的候选假设。在本文中，提出了一种新颖的多假设协作深度展开压缩感知网络（MHC-DUN），该网络通过在不同解空间之间联合优化，明确建模和利用多个假设。具体而言，MHC-DUN在这一多假设范式下，遵循近端梯度下降算法，联合执行梯度下降和近端映射。i) 在梯度下降方面，提出了一种精心设计的AlphaNet，以动态预测所有假设的空间变化步长，从而实现多个解之间的协作梯度更新。ii) 在近端算子方面，设计了一个复杂的多假设协作近端映射模块，该模块利用假设内部和假设之间的相关性先验，共同优化多个解。为了实现端到端训练，设计了一种新颖的复合损失函数，该函数在测量保真度、假设多样性和重建精度之间取得平衡，鼓励探索互补解的同时保持重建保真度。实验结果表明，所提出的CS方法优于现有的CS网络。

View on arXiv Download PDF AI Translation

cs.CV / 85 / 2606.03675

A Fast Methane Detection Pipeline on Board Satellites Based on Mag1c-SAS and LinkNet

基于 Mag1c-SAS 和 LinkNet 的卫星快速甲烷检测管道

Herec, Jonáš, Růžička, Vít, Pitoňák, Rado, Sedmidubsky, Jan

Abstract

Methane is a potent greenhouse gas, and detecting leaks early via hyperspectral satellite imagery can help climate change mitigation efforts. Meanwhile, many existing hyperspectral missions only capture areas manually targeted by operators, thus missing potential events of interest. To overcome slow downlink rates cost-effectively, onboard detection is a viable solution. However, traditional methane detection methods are too computationally demanding for resource-limited onboard hardware. This work accelerates methane detection by focusing on efficient, low-power algorithms. In particular, we test fast target detection ACE and CEM methods that have not been previously used for methane detection and propose Mag1c-SAS -- a significantly faster variant of the current state-of-the-art Mag1c algorithm. To explore their detection potential, we integrate them with a machine learning model based on U-Net and LinkNet. We evaluate our methods on the STARCOP dataset and a novel EMIT-MSeg dataset, which we introduce and open-source alongside a high-quality annotation strategy. The proposed Mag1c-SAS approach proves highly effective by operating ~80x faster than the original Mag1c approach, providing a visually similar, but noisier result. When additionally paired with the lightweight LinkNet approach, it effectively reduces noise, achieving AUPRC score improvements of over 30 pp on EMIT-MSeg compared to the baseline Mag1c approach, and an F1 score on STARCOP ~4 pp higher. We evaluate two novel band selection strategies and confirm the system's onboard viability through hardware profiling, demonstrating marginal power consumption and efficient CPU/RAM utilization. We release the final system in a user-friendly and lightweight PyPI library at: https://pypi.org/project/onboard-methane-detection/, alongside all experimental code, models, and data at: https://github.com/zaitra/methane-filters-benchmark.

Chinese Translation

甲烷是一种强效温室气体，通过高光谱卫星影像早期检测泄漏可以帮助气候变化缓解工作。与此同时，许多现有的高光谱任务仅捕获操作人员手动选择的区域，从而错过潜在的感兴趣事件。为了解决低速下行率的问题，机载检测是一种可行的解决方案。然而，传统的甲烷检测方法对资源有限的机载硬件来说计算需求过高。本研究通过关注高效、低功耗的算法来加速甲烷检测。特别是，我们测试了尚未用于甲烷检测的快速目标检测 ACE 和 CEM 方法，并提出了 Mag1c-SAS——一种显著快于当前最先进的 Mag1c 算法的变体。为了探索它们的检测潜力，我们将其与基于 U-Net 和 LinkNet 的机器学习模型集成。我们在 STARCOP 数据集和一个新颖的 EMIT-MSeg 数据集上评估了我们的方法，该数据集是我们引入并开源的，伴随高质量的标注策略。所提出的 Mag1c-SAS 方法证明了其高效性，运行速度约为原 Mag1c 方法的 80 倍，提供了视觉上相似但噪声更大的结果。当与轻量级的 LinkNet 方法结合使用时，它有效地减少了噪声，在 EMIT-MSeg 上相较于基线 Mag1c 方法实现了超过 30 个百分点的 AUPRC 分数提升，并在 STARCOP 上的 F1 分数提高了约 4 个百分点。我们评估了两种新颖的波段选择策略，并通过硬件分析确认了系统的机载可行性，展示了边际的功耗和高效的 CPU/RAM 利用率。我们在用户友好且轻量的 PyPI 库中发布了最终系统，网址为：https://pypi.org/project/onboard-methane-detection/，同时在：https://github.com/zaitra/methane-filters-benchmark 发布了所有实验代码、模型和数据。

View on arXiv Download PDF AI Translation

cs.CV / 86 / 2606.03713

Investigating Adversarial Robustness of Multi-modal Large Language Models

多模态大型语言模型的对抗鲁棒性研究

Malik, Hashmat Shadab, Naseer, Muzammal, Khan, Salman

Abstract

Multi-modal Large Language Models (MLLMs) achieve strong performance on vision-language tasks, but incorporating visual inputs through a vision encoder (e.g., CLIP) substantially expands the attack surface, making these models vulnerable to visual adversarial perturbations. Prior defenses typically preserve compatibility with pretrained MLLMs by enforcing strict alignment to CLIP's original embedding space during adversarial fine-tuning; while practical, this constraint fundamentally limits achievable robustness. We present a systematic investigation of adversarial robustness in MLLMs. We first introduce a diagnostic CLIP-alignment protocol that predicts, prior to full MLLM training, which robust vision encoders will transfer effectively to the multimodal setting, revealing that large-scale multimodal adversarial pretraining, rather than unimodal scale alone, is the critical factor for strong robustness transfer. Integrating such encoders into MLLMs via end-to-end multimodal training yields average gains of 28 CIDEr points on captioning and 11.7% VQA accuracy under strong adversarial attacks compared to constrained plug-and-play baselines. We further show that adversarial training applied directly to a standard non-robust MLLM degrades both clean and adversarial performance, establishing robust visual representations as a strict prerequisite, while end-to-end adversarial training from a robust backbone delivers additional gains of 1.9 CIDEr points and 4.3% VQA accuracy. Beyond training-time defenses, lightweight test-time visual stochastic transformations serve as an effective black-box defense for non-robust MLLMs, elevating adversarial performance from near-zero to levels comparable with robust models. Finally, we show that our robust models substantially reduce toxic generation under white-box visual jailbreak attacks. Code and pretrained weights will be released publicly.

Chinese Translation

多模态大型语言模型（MLLMs）在视觉-语言任务中表现出色，但通过视觉编码器（例如，CLIP）引入视觉输入显著扩大了攻击面，使这些模型容易受到视觉对抗扰动的影响。以往的防御方法通常通过在对抗微调过程中强制与CLIP的原始嵌入空间严格对齐，来保持与预训练MLLMs的兼容性；尽管这种方法在实践中有效，但这一约束从根本上限制了可实现的鲁棒性。我们对MLLMs中的对抗鲁棒性进行了系统的研究。首先，我们引入了一种诊断性CLIP对齐协议，该协议在完整的MLLM训练之前预测哪些鲁棒的视觉编码器能够有效转移到多模态设置中，揭示出大规模多模态对抗预训练，而非单模态规模，才是强鲁棒性转移的关键因素。通过端到端的多模态训练将这些编码器集成到MLLMs中，相较于受限的即插即用基线，平均提高了28个CIDEr点的图像描述能力和11.7%的视觉问答准确率，尤其是在强对抗攻击下。我们进一步表明，直接对标准非鲁棒MLLM进行对抗训练会降低干净和对抗性能，确立了鲁棒视觉表征作为严格的前提条件，而从鲁棒主干进行的端到端对抗训练则带来了额外的1.9个CIDEr点和4.3%的视觉问答准确率提升。除了训练时的防御外，轻量级的测试时视觉随机变换作为一种有效的黑箱防御，能够将非鲁棒MLLM的对抗性能从接近零提升到与鲁棒模型相当的水平。最后，我们展示了我们的鲁棒模型在白箱视觉越狱攻击下显著减少了有害生成。代码和预训练权重将公开发布。

View on arXiv Download PDF AI Translation

cs.CV / 87 / 2606.03715

Text-to-Image Models Need Less from Text Encoders Than You Think

文本到图像模型对文本编码器的需求比你想象的要少

Spingarn, Nurit, Cohen, Noa, Shaham, Tamar Rott, Michaeli, Tomer

Abstract

Text-to-image models rely on text prompts as their primary interface to human intent. Prompts are encoded by a text encoder into embeddings that condition the image generation process. Beyond individual token meanings, text embeddings encode contextual information across the full prompt, such as compositionality and attribute binding. However, whether image models actually exploit this richer information remains underexplored. Here, we address the question: Which aspects of text representation are essential for image generation? We show that text-to-image diffusion transformer-based models commonly rely only on two relatively straightforward aspects of text representations: (i) the merging of adjacent tokens into a word representation, for words spanning multiple tokens, and (ii) word order, which is imprinted by the positional embedding of the text-encoder. To show this, we construct a new text embedding that encodes only individual word meanings and order but lacks any contextual information about the full prompt. We find that this bag of position-tagged words representation is sufficient to successfully guide image generation, achieving visual quality and text fidelity that are on par with full text embedding-guided generation. This demonstrates that, contrary to common belief, text-to-image models often do not use the rich information encoded in the text embedding beyond individual word meanings and word order. Instead, the decoding of complex linguistic structures is performed by the image model itself. Project webpage: https://nsping13.github.io/contextless-TTI/

Chinese Translation

文本到图像模型依赖文本提示作为与人类意图的主要接口。提示通过文本编码器被编码为嵌入，这些嵌入条件化图像生成过程。除了单个标记的含义外，文本嵌入还编码了整个提示的上下文信息，例如组合性和属性绑定。然而，图像模型是否真正利用了这些更丰富的信息仍然未被充分探讨。在这里，我们探讨了一个问题：文本表示的哪些方面对图像生成是必不可少的？我们展示了基于文本到图像扩散变换器的模型通常仅依赖于文本表示的两个相对简单的方面：(i) 将相邻标记合并为一个单词表示，适用于跨多个标记的单词，以及 (ii) 单词顺序，该顺序由文本编码器的位置嵌入印记。为了证明这一点，我们构建了一种新的文本嵌入，仅编码单个单词的含义和顺序，但缺乏关于整个提示的任何上下文信息。我们发现，这种带位置标签的单词表示足以成功指导图像生成，达到与完整文本嵌入引导生成相当的视觉质量和文本保真度。这表明，与普遍看法相反，文本到图像模型通常并不使用文本嵌入中编码的超出单个单词含义和单词顺序的丰富信息。相反，复杂语言结构的解码是由图像模型本身执行的。项目网页：https://nsping13.github.io/contextless-TTI/

View on arXiv Download PDF AI Translation

cs.CV / 88 / 2606.03730

Beyond False Stability: High-Noise Drift Gating for Test-Time Adversarial Defenses in Vision-Language Models

超越虚假稳定性：高噪声漂移门控在视觉-语言模型中的测试时对抗防御

Malik, Hashmat Shadab, Naseer, Muzammal, Khan, Salman

Abstract

Vision-language models (VLMs) such as CLIP show strong zero-shot generalization but remain highly vulnerable to adversarial attacks. Adversarial training improves robustness but is computationally expensive, motivating test-time defenses. Recent approaches exploit how CLIP's visual representations respond to stochastic perturbations: aggregating predictions across noisy views, constructing Gaussian noise-averaged anchors and interpolating features toward them, or applying counter-perturbations. These strategies improve robustness but often degrade clean accuracy, yielding an unfavorable clean-robust trade-off. We revisit stochastic test-time defenses and identify an underexplored noise-regime transition in CLIP's representation space. Prior work explored perturbations mainly in the weak-noise regime, where adversarial examples can appear unusually stable (false stability). Our analysis shows this reverses as perturbation strength grows: beyond the weak-noise regime, adversarial representations become markedly more unstable than clean ones, giving a clearer separation signal. The transition is consistent across uniform and Gaussian noise, photometric and geometric transforms, datasets, and diverse attacks. It largely disappears in adversarially trained models, suggesting it is tied to the fragile local-basin geometry of adversarial representations in non-robust CLIP. We propose a training-free, plug-in drift-gated mechanism that uses high-noise feature drift as a lightweight gating signal to trigger existing test-time defenses only when adversarial-like instability is detected. Across 13 datasets it consistently improves the clean-robust trade-off. On eight fine-grained datasets, mean clean+adversarial accuracy rises from 65.7% to 71.4% for counterattack defenses and 68.4% to 73.2% for noise-anchoring; on ImageNet and four shifted variants, from 56.1% to 66.2% and 62.1% to 67.6%.

Chinese Translation

视觉-语言模型（VLMs），如 CLIP，展现出强大的零样本泛化能力，但仍然对对抗攻击高度脆弱。对抗训练提高了模型的鲁棒性，但计算成本高昂，这促使了测试时防御的研究。近期的方法利用了 CLIP 的视觉表示对随机扰动的响应：通过在噪声视图中聚合预测、构建高斯噪声平均锚点并向其插值特征，或应用反扰动。这些策略提高了鲁棒性，但往往会降低干净样本的准确性，导致不理想的干净-鲁棒权衡。我们重新审视随机测试时防御，并识别出 CLIP 表示空间中一个未被充分探索的噪声状态转变。之前的研究主要在弱噪声状态下探索扰动，此时对抗样本可能表现出异常的稳定性（虚假稳定性）。我们的分析表明，随着扰动强度的增加，这种情况会逆转：在超出弱噪声状态后，对抗表示变得明显比干净表示更不稳定，从而提供了更清晰的分离信号。该转变在均匀和高斯噪声、光度和几何变换、数据集以及各种攻击中是一致的。在对抗训练模型中，这种转变基本消失，表明它与非鲁棒 CLIP 中对抗表示的脆弱局部盆地几何结构有关。我们提出了一种无训练的插件漂移门控机制，利用高噪声特征漂移作为轻量级门控信号，仅在检测到对抗性不稳定性时触发现有的测试时防御。在 13 个数据集上，它始终改善了干净-鲁棒权衡。在八个细粒度数据集上，平均干净+对抗准确率从 65.7% 提升至 71.4%（针对反击防御），从 68.4% 提升至 73.2%（针对噪声锚定）；在 ImageNet 及其四个偏移变体上，从 56.1% 提升至 66.2%，从 62.1% 提升至 67.6%。

View on arXiv Download PDF AI Translation

cs.CV / 89 / 2606.03746

Qwen-Image-Flash: Beyond Objective Design

Qwen-Image-Flash：超越目标设计

Wu, Tianhe, Yan, Kun, Zhou, Zikai, Jiang, Lihan, Li, Jiahao, Zhang, Jie, Gao, Kaiyuan, Tang, Ningyuan, Yin, Shengming, Chen, Xiaoyue, Xu, Xiao, Chen, Yilei, Chen, Yuxiang, Shu, Yan, Xu, Yixian, Zhang, Yanran, Liu, Zihao, Wang, Zhendong, Zhang, Zekai, Li, Deqing, Peng, Liang, Wang, Yi, Zhou, Jingren, Wu, Chenfei

Abstract

Few-step distillation has become an effective strategy for accelerating advanced visual generative models, yet prior work has largely focused on distillation objectives. In this work, we revisit few-step distillation from a complementary perspective, focusing on the training recipe that critically shapes student performance. Using Qwen-Image-2.0 as a representative case, we systematically investigate three factors in unified text-to-image generation and instruction-guided image editing distillation: data composition, teacher guidance, and task mixture. Our empirical analysis reveals several non-obvious behaviors, which motivate the development of Qwen-Image-Flash. Overall, our results suggest that effective few-step distillation requires not only carefully designed objectives, but also principled organization of the broader training pipeline.

Chinese Translation

少步蒸馏已成为加速先进视觉生成模型的有效策略，但之前的研究主要集中在蒸馏目标上。在本研究中，我们从互补的角度重新审视少步蒸馏，关注对学生性能产生重要影响的训练方案。以 Qwen-Image-2.0 为代表案例，我们系统地研究了统一文本到图像生成和指令引导图像编辑蒸馏中的三个因素：数据组成、教师指导和任务混合。我们的实证分析揭示了一些非显而易见的行为，这促使了 Qwen-Image-Flash 的发展。总体而言，我们的结果表明，有效的少步蒸馏不仅需要精心设计的目标，还需要对更广泛的训练流程进行原则性的组织。

View on arXiv Download PDF AI Translation

cs.CV / 90 / 2606.03748

Ultralytics YOLO26: Unified Real-Time End-to-End Vision Models

Ultralytics YOLO26：统一的实时端到端视觉模型

Jocher, Glenn, Qiu, Jing, Liu, Mengyu, Lyu, Shuai, Akyon, Fatih Cagatay, Kalfaoglu, Muhammet Esat

Abstract

Real-time vision demands models that are accurate, efficient, and simple to deploy across diverse hardware. The YOLO family has become widely deployed for this reason, yet most YOLO detectors still rely on non-maximum suppression at inference, carry heavy detection heads due to Distribution Focal Loss, require long training schedules, and can leave the smallest objects without positive label assignments. We present Ultralytics YOLO26, a unified real-time vision model family that addresses these limitations through coordinated architecture and training advances. YOLO26 uses a dual-head design for native NMS-free end-to-end inference and removes DFL entirely, yielding a lighter head with unconstrained regression range. Its training pipeline combines MuSGD, a hybrid Muon-SGD optimizer adapted from large language model training; Progressive Loss, which shifts supervision toward the inference-time head; and STAL, a label assignment strategy that guarantees positive coverage for small objects. Beyond detection, YOLO26 introduces task-specific head and loss designs for instance segmentation, pose estimation, and oriented detection, producing consistent gains across tasks and scales. The family spans five scales (n/s/m/l/x) and supports detection, instance segmentation, pose estimation, classification, and oriented detection in a single pipeline, with an open-vocabulary extension, YOLOE-26, for text-, visual-, and prompt-free inference. Across all scales, YOLO26 achieves 40.9-57.5 mAP on COCO at 1.7-11.8 ms T4 TensorRT latency, advancing the accuracy-latency Pareto front over prior real-time detectors, while YOLOE-26x reaches 40.6 AP on LVIS minival under text prompting. Code and models are available at https://github.com/ultralytics/ultralytics.

Chinese Translation

实时视觉要求模型具备准确性、高效性，并且能够在多种硬件上简单部署。由于这个原因，YOLO系列已经得到了广泛应用，然而大多数YOLO检测器在推理时仍依赖于非极大值抑制（NMS），由于分布焦点损失（Distribution Focal Loss）而承载着沉重的检测头，训练周期较长，并且可能会导致最小物体没有正标签分配。我们提出了Ultralytics YOLO26，一个统一的实时视觉模型系列，通过协调的架构和训练进展来解决这些局限性。YOLO26采用双头设计，实现了原生的无NMS端到端推理，并完全去除了DFL，产生了一个更轻的检测头，具有不受限的回归范围。其训练流程结合了MuSGD，一种从大型语言模型训练中改编而来的混合Muon-SGD优化器；渐进损失（Progressive Loss），将监督转向推理时的检测头；以及STAL，一种标签分配策略，确保小物体的正覆盖。除了检测，YOLO26还为实例分割、姿态估计和定向检测引入了特定任务的头和损失设计，在各个任务和尺度上产生了一致的增益。该系列涵盖五个尺度（n/s/m/l/x），并支持在单一流程中进行检测、实例分割、姿态估计、分类和定向检测，同时提供开放词汇扩展YOLOE-26，以实现文本、视觉和无提示推理。在所有尺度上，YOLO26在COCO数据集上实现了40.9-57.5的mAP，推理延迟为1.7-11.8毫秒，推动了准确性-延迟的Pareto前沿，相较于之前的实时检测器，YOLOE-26x在文本提示下在LVIS minival上达到了40.6的AP。代码和模型可在https://github.com/ultralytics/ultralytics获取。

View on arXiv Download PDF AI Translation

cs.CV / 91 / 2606.03774

AmbientEye: A Dataset for Pupil Segmentation under Natural Ambient Infrared Illumination

AmbientEye：一个用于自然环境红外照明下瞳孔分割的数据集

Han, Mingyu, Han, Hyunyoung, Thommakoon, Nitheekulawatn, Park, Gangtae, Han, Jieun, Zhang, Xucong, Oakley, Ian

Abstract

Eye tracking is essential for smart glasses, as it provides insight into user attention for ambient intelligence applications. However, most existing eye-tracking systems rely on active infrared (IR) illumination, creating practical barriers to all-day outdoor use due to power consumption. In this paper, we investigate whether passive IR cameras alone, without any active IR light source, can enable reliable pupil detection in unconstrained outdoor environments, where ambient sunlight serves as the sole illumination source. To support this investigation, we introduce AmbientEye, a large-scale dataset of 2,606,225 eye images collected from 35 participants from 19 countries. It is captured outdoors under natural sunlight with two off-axis camera configurations and two sun-orientation conditions. We provide high-quality pupil annotation through SAM2 automatic segmentation, followed by refinement by human annotators. We benchmark a state-of-the-art pupil segmentation algorithm on our dataset and compare its performance with that on existing datasets under controlled IR illumination. Results reveal a substantial drop in pupil segmentation performance from 0.928 on controlled IR datasets to 0.767 on AmbientEye. This performance gap highlights the challenge of the ambient-light setting. This positions AmbientEye as a first benchmark for an unexplored and highly practical eye-tracking scenario.

Chinese Translation

眼动追踪对于智能眼镜至关重要，因为它为环境智能应用提供了用户注意力的洞察。然而，现有的大多数眼动追踪系统依赖于主动红外（IR）照明，这在全天户外使用中由于功耗造成了实际障碍。本文研究了在没有任何主动红外光源的情况下，单靠被动红外相机是否能够在无约束的户外环境中实现可靠的瞳孔检测，在这些环境中，环境阳光是唯一的照明来源。为支持这一研究，我们引入了AmbientEye，一个由来自19个国家的35名参与者收集的2606225张眼睛图像的大规模数据集。该数据集是在自然阳光下通过两种偏轴相机配置和两种太阳方位条件在户外捕获的。我们通过SAM2自动分割提供高质量的瞳孔标注，随后由人工标注者进行细化。我们在我们的数据集上基准测试了一种最先进的瞳孔分割算法，并将其性能与在受控IR照明下现有数据集上的性能进行了比较。结果显示，瞳孔分割性能从受控IR数据集的0.928显著下降到AmbientEye的0.767。这一性能差距突显了环境光设置的挑战。这使得AmbientEye成为一个未被探索且极具实用性的眼动追踪场景的首个基准。

View on arXiv Download PDF AI Translation

cs.CV / 92 / 2606.03788

SLU-2K: A Question-Based Benchmark for Semantic Evaluation of Sign Language Translation

SLU-2K：基于问题的手语翻译语义评估基准

Testa, Zeno, Furnari, Antonino, Baraldi, Lorenzo, Díaz-Rodríguez, Natalia

Abstract

Sign Language Translation (SLT) is typically evaluated with surface-form metrics such as BLEU and ROUGE, which reward lexical overlap but do not directly measure whether a translation preserves the meaning of the source sign sequence. This is in contrast with the final objective of integrating SLT in assistive technology. In this work, we shift the focus from Sign Language Translation (SLT) to Sign Language Understanding (SLU), with particular emphasis on semantic understanding. Specifically, we evaluate systems based on their ability to correctly recover, from the input video, key semantic aspects of the original sentence, such as actions taking place and facts about people and objects. To enable this evaluation systematically, we propose SLU-2K, a dataset of 2,350 closed-ended video question-answer pairs based on the popular PHOENIX-2014T and CSL-Daily datasets. To obtain SLU-2K, we propose and extensively evaluate an automated data generation pipeline which produces questions across 7 categories, namely actions, locations, numbers, objects, people, time, and weather conditions. We show the potential of SLU-2K by evaluating popular Multimodal Large Language Models (MLLMs) and two representative state-of-the-art systems, MMSTL and SpaMo. Our results show that MLLMs reach near-random performance, highlighting the need for a more systematic integration of SLU in current AI systems. Furthermore, state-of-the-art translation systems carefully fine-tuned on in-domain data still exhibit a substantial semantic gap, with results ranging from 56.7% to 75.2%. These findings suggest that current SLT evaluation protocols overestimate true understanding and that future progress should be measured not only by fluency and n-gram overlap, but also by semantic correctness. Code, prompts, and benchmark files are available at https://github.com/ZenoTsT/SLU-2K

Chinese Translation

手语翻译（SLT）通常使用表面形式的指标进行评估，如BLEU和ROUGE，这些指标奖励词汇重叠，但并不直接衡量翻译是否保留了源手语序列的意义。这与将SLT整合到辅助技术中的最终目标相悖。在本研究中，我们将重点从手语翻译（SLT）转向手语理解（SLU），特别强调语义理解。具体而言，我们评估系统从输入视频中正确恢复原句的关键语义方面的能力，例如发生的动作以及关于人和物体的事实。为了系统地进行这种评估，我们提出了SLU-2K，这是一个基于流行的PHOENIX-2014T和CSL-Daily数据集的2,350个封闭式视频问答对的数据集。为了获得SLU-2K，我们提出并广泛评估了一种自动化数据生成管道，该管道在7个类别中生成问题，即动作、地点、数字、物体、人物、时间和天气条件。我们通过评估流行的多模态大型语言模型（MLLMs）和两个代表性的最先进系统MMSTL和SpaMo，展示了SLU-2K的潜力。我们的结果表明，MLLMs的表现接近随机水平，突显了在当前人工智能系统中更系统地整合SLU的必要性。此外，经过精心微调的最先进翻译系统在领域内数据上仍然存在显著的语义差距，结果范围为56.7%到75.2%。这些发现表明，当前的SLT评估协议高估了真实理解，未来的进展应不仅通过流利度和n-gram重叠来衡量，还应通过语义正确性来评估。代码、提示和基准文件可在https://github.com/ZenoTsT/SLU-2K获取。

View on arXiv Download PDF AI Translation

cs.CV / 93 / 2606.03792

Training-Free Multi-Concept LoRA Composition with Prompt-Aware Weighting

无训练的多概念LoRA组合与提示感知加权

Tsoumplekas, Georgios, Bounareli, Stella, Argyriou, Vasileios

Abstract

Low-Rank Adaptation (LoRA) successfully enables personalization in text-to-image generation by adapting pre-trained diffusion models to specific visual concepts and styles. However, extending such models to multi-concept customization remains challenging. Naively combining multiple LoRA weights or their outputs often leads to interference among concepts, resulting in degraded visual quality and reduced fidelity to the reference images of individual concepts. This paper proposes a simple yet effective approach for multi-concept customization by optimally combining the outputs of multiple LoRA modules. We leverage the relative importance of each concept during generation, as inferred from its corresponding prompt tokens and introduce two methods, W-Switch and W-Composite, that employ a prompt-aware importance weighting strategy in which each LoRA is weighted according to the semantic influence of its trigger words in the target prompt. In addition, we extend existing quantitative evaluation metrics by proposing a new image-based similarity evaluation framework that assesses image fidelity and identity preservation through comparisons between real-world reference images and automatically segmented concept regions from generated images. We evaluate our approach on the ComposLoRA testbed and demonstrate consistent improvements over existing state-of-the-art methods in terms of visual quality, identity preservation and compositionality. Qualitative evaluations, including a Large Language Model (LLM) based assessment and a user study, further validate the effectiveness of the proposed methods and align with the newly introduced quantitative image-based metrics. Our code is available at https://github.com/GeorgeTsoumplekas/Prompt-Aware-Multi-LoRA-Composition.

Chinese Translation

低秩适应（LoRA）通过将预训练的扩散模型适应于特定的视觉概念和风格，成功实现了文本到图像生成中的个性化。然而，将此类模型扩展到多概念定制仍然面临挑战。简单地组合多个LoRA权重或其输出往往会导致概念之间的干扰，从而降低视觉质量并减少对单个概念参考图像的忠实度。本文提出了一种简单而有效的多概念定制方法，通过最佳组合多个LoRA模块的输出。我们在生成过程中利用每个概念的相对重要性，这一重要性是通过其对应的提示标记推断得出的，并引入了两种方法，W-Switch和W-Composite，这些方法采用了一种提示感知的重要性加权策略，其中每个LoRA的权重根据其触发词在目标提示中的语义影响进行调整。此外，我们通过提出一个新的基于图像的相似性评估框架，扩展了现有的定量评估指标，该框架通过比较真实世界参考图像与生成图像中自动分割的概念区域来评估图像的忠实度和身份保持。我们在ComposLoRA测试平台上评估了我们的方法，并在视觉质量、身份保持和组合性方面展示了相较于现有最先进方法的一致性改进。定性评估，包括基于大型语言模型（LLM）的评估和用户研究，进一步验证了所提方法的有效性，并与新引入的基于图像的定量指标相一致。我们的代码可在 https://github.com/GeorgeTsoumplekas/Prompt-Aware-Multi-LoRA-Composition 获取。

View on arXiv Download PDF AI Translation

cs.CV / 94 / 2606.03795

Beyond Compression: Quantifying Spectral Accessibility in Vision Representations

超越压缩：量化视觉表征中的光谱可达性

Kitessa, Akayou A., Zhao, Yijun

Abstract

Vision-language models map visual features into a shared embedding space through learned projection layers, yet it remains unclear how these transformations alter the structure of visual information. This study examines changes in representation through spatial-frequency accessibility, measured by the linear recoverability of band-limited Fourier energy from model representations. To isolate effects beyond dimensionality reduction, we introduce Residual Spectral Loss (RSL), which evaluates changes relative to a dimension-matched random projection baseline. To reduce confounding effects from optimization, the analysis uses pretrained models with all parameters frozen. The experimental results show consistent frequency-dependent changes in accessibility across CLIP and DINOv2 on ImageNet and MS-COCO datasets. Spectral accessibility follows a non-monotonic trajectory across depth, peaking at intermediate layers before decreasing toward the output representation. The final transformation differs across architectures: CLIP's learned projection is spectrally neutral, with changes explained by compression, whereas DINOv2's [CLS] pooling induces a structured loss across the spectrum. These findings identify intermediate layers and pooling mechanisms as primary drivers of spectral transformation in modern vision encoders.

Chinese Translation

视觉-语言模型通过学习的投影层将视觉特征映射到共享的嵌入空间，但这些变换如何改变视觉信息的结构仍不清楚。本研究通过空间频率可达性来考察表征的变化，该可达性通过模型表征中带限傅里叶能量的线性可恢复性进行测量。为了隔离超越维度减少的影响，我们引入了残差光谱损失（Residual Spectral Loss, RSL），该损失相对于维度匹配的随机投影基线评估变化。为了减少优化带来的混淆效应，分析使用了所有参数被冻结的预训练模型。实验结果显示，在ImageNet和MS-COCO数据集上，CLIP和DINOv2在可达性方面表现出一致的频率依赖性变化。光谱可达性在深度上呈现非单调轨迹，在中间层达到峰值后向输出表征下降。最终的变换在不同架构间存在差异：CLIP的学习投影在光谱上是中性的，其变化可通过压缩解释，而DINOv2的[CLS]池化则在光谱上引入了结构化损失。这些发现将中间层和池化机制识别为现代视觉编码器中光谱变换的主要驱动因素。

View on arXiv Download PDF AI Translation

cs.CV / 95 / 2606.03802

Template Collapse and Information-Theoretic Limits in Camera rPPG Pulse Morphology Restoration

相机 rPPG 脉搏形态恢复中的模板崩溃与信息论极限

Ahmed, Achraf Ben

Abstract

Objective: Consumer face camera remote photoplethysmography (rPPG) enables passive cardiovascular monitoring, but whether single-cycle waveform morphology encoding arterial stiffness biomarkers is recoverable from this measurement has not been characterised. Methods: We evaluated 16 architectures spanning six families on 153 subjects across three datasets, introducing cross-subject Pearson r to distinguish subject-specific recovery from template collapse. Results: No architecture recovered subject-specific morphology (cross-subject r range 0.773--0.9999; ground-truth ceiling 0.601). Supervised Contrastive (SupCon) converged to log N = 4.844, constituting the strongest available empirical evidence that no discriminative morphological structure is extractable from single-cycle rPPG by the encoder families tested. The VAE decoder restores population-level harmonic content absent from the rPPG input (H2/H1: 0.310 output vs. 0.275 input), generalising zero-shot to UBFC (r = +0.708); a directional hallucination gap (p = 0.150) suggests partial signal reading. Anti-collapse objectives fail when input carries no discriminative structure. Significance: Consumer cameras cannot encode individual arterial morphology; cross-subject r is a necessary collapse diagnostic for waveform reconstruction benchmarks.

Chinese Translation

目的：消费级人脸相机远程光电容积描记法（rPPG）能够实现被动心血管监测，但从该测量中恢复编码动脉硬度生物标志物的单周期波形形态尚未得到表征。方法：我们在三个数据集上对153名受试者评估了跨越六个家族的16种架构，引入跨受试者皮尔逊相关系数（Pearson r）以区分特定受试者的恢复与模板崩溃。结果：没有任何架构能够恢复特定受试者的形态（跨受试者 r 范围 0.773--0.9999；真实值上限 0.601）。监督对比（Supervised Contrastive, SupCon）收敛至 log N = 4.844，构成了最强的实证证据，表明在测试的编码器家族中无法从单周期 rPPG 提取出可区分的形态结构。变分自编码器（VAE）解码器恢复了 rPPG 输入中缺失的人群水平谐波内容（H2/H1: 0.310 输出 vs. 0.275 输入），并在零样本情况下推广至 UBFC（r = +0.708）；一个方向性幻觉差距（p = 0.150）表明部分信号读取。反崩溃目标在输入不携带可区分结构时失败。意义：消费级相机无法编码个体动脉形态；跨受试者 r 是波形重建基准的必要崩溃诊断。

View on arXiv Download PDF AI Translation

cs.CV / 96 / 2606.03806

TeX-1500: A Paired Real-World LWIR Hyperspectral Dataset and Benchmark for Temperature-Emissivity-Texture Decomposition

TeX-1500：一套配对的真实世界长波红外高光谱数据集及温度-发射率-纹理分解基准

Dai, Cheng, Lin, Jiale, Xu, Hongyi, Song, Bingxuan, Xie, Ziyang, Bao, Fanglin

Abstract

Temperature-emissivity-texture (TeX) decomposition seeks to recover object heat state, material spectral response, and visible-like geometric texture from long-wave infrared hyperspectral imaging (LWIR HSI). Existing TeX pipelines are mainly scene-specific inverse solvers, and the lack of paired LWIR HSI-TeX supervision has limited learning-based decomposition. To address this gap, we introduce TeX-1500, a large-scale paired LWIR HSI-TeX dataset and benchmark for supervised HSI-to-TeX decomposition. TeX-1500 contains 1,522 calibrated real-scene pairs from DARPA Invisible Headlights (DARPA IH) pushbroom imagery and our FTIR acquisitions, covering five locations, four seasons, diverse acquisition times, heterogeneous wavelength layouts, and two sensor families. Each sample stores a calibrated valid-band radiance cube, calibrated wavelength positions, and aligned temperature, emissivity, and texture supervision constructed through a consistent restoration and TeX-construction protocol. We further provide TeX-UNet, a simple wavelength-aware baseline that maps calibrated HSI bands and wavelength positions to TeX fields. Experiments on the held-out DARPA IH pushbroom scenes and zero-/few-shot transfer to FTIR scenes show that TeX-1500 provides usable paired supervision and a measurable benchmark for data-driven physical-property-centered thermal perception.

Chinese Translation

温度-发射率-纹理（TeX）分解旨在从长波红外高光谱成像（LWIR HSI）中恢复物体的热状态、材料的光谱响应以及类似可见光的几何纹理。现有的TeX处理流程主要是场景特定的逆解算器，而缺乏配对的LWIR HSI-TeX监督限制了基于学习的分解方法。为了解决这一问题，我们推出了TeX-1500，这是一个大规模的配对LWIR HSI-TeX数据集及监督的HSI到TeX分解基准。TeX-1500包含来自DARPA隐形车灯（DARPA IH）推扫成像和我们的傅里叶变换红外（FTIR）采集的1,522对经过校准的真实场景，涵盖五个地点、四个季节、多样的采集时间、异构的波长布局和两种传感器系列。每个样本存储了一个经过校准的有效波段辐射立方体、校准的波长位置，以及通过一致的恢复和TeX构建协议构建的对齐的温度、发射率和纹理监督。我们还提供了TeX-UNet，一个简单的波长感知基线，将校准的HSI波段和波长位置映射到TeX字段。在保留的DARPA IH推扫场景上的实验以及对FTIR场景的零样本/少样本迁移表明，TeX-1500提供了可用的配对监督和可测量的基准，以支持以数据驱动的物理属性为中心的热感知。

View on arXiv Download PDF AI Translation

cs.CV / 97 / 2606.03827

Conditional Latent Diffusion Model with Fourier-based Motion Modelling for Virtual Population Synthesis

基于傅里叶运动建模的条件潜在扩散模型用于虚拟人群合成

Lan, Shaokun, Dou, Haoran, Huang, Jinghan, Zakeri, Arezoo, Lin, Fengming, Zhou, Zherui, Duan, Jinming, Frangi, Alejandro F.

Abstract

In-silico trials of medical devices require the generation of virtual populations of anatomies. In cardiovascular applications, virtual anatomy is typically represented as a 3D+t mesh sampled from a generative model. However, most existing mesh generators focus on static anatomy, while sequence models often lack explicit periodicity. To this end, we propose 4D F-MeshLDM, a conditional generative framework comprising a convolutional mesh VAE to encode meshes, a structural latent space that parameterises motion using a truncated Fourier series, and a diffusion prior that learns the latent distribution over Fourier coefficient tokens. By conditioning the diffusion process on clinical covariates via affine modulation, we enable controllable synthesis. Sampling tokens and performing inverse Fourier synthesis yield cycle-consistent latent trajectories, which can be decoded into 3D+t cardiac mesh sequences. Experiments on 5,000 UK Biobank subjects demonstrate that 4D F-MeshLDM outperforms state-of-the-art baselines in anatomical fidelity and achieves near-zero cycle closure error. Furthermore, the generated cohorts accurately preserve clinical functional indices, highlighting the potential of our framework for reliable in-silico cardiac trials.

Chinese Translation

医疗设备的计算机模拟试验需要生成虚拟的人体解剖结构人群。在心血管应用中，虚拟解剖结构通常表示为从生成模型中采样的3D+t网格。然而，大多数现有的网格生成器专注于静态解剖结构，而序列模型往往缺乏明确的周期性。为此，我们提出了4D F-MeshLDM，这是一种条件生成框架，包括一个卷积网格变分自编码器（VAE）用于编码网格，一个结构潜在空间通过截断傅里叶级数参数化运动，以及一个扩散先验学习傅里叶系数标记的潜在分布。通过仿射调制将扩散过程与临床协变量相结合，我们实现了可控合成。采样标记并执行逆傅里叶合成产生循环一致的潜在轨迹，这些轨迹可以解码为3D+t心脏网格序列。在5000名英国生物银行受试者上的实验表明，4D F-MeshLDM在解剖忠实度上优于最先进的基线，并且实现了近乎零的循环闭合误差。此外，生成的队列准确保留了临床功能指标，突显了我们框架在可靠的计算机模拟心脏试验中的潜力。

View on arXiv Download PDF AI Translation

cs.CV / 98 / 2606.03837

Where Do We (Not) Need Temporal Context in Low-Resource Video Task Adaptation?

在低资源视频任务适应中，我们（不）需要时间上下文吗？

Sträter, Luc P. J., Doughty, Hazel

Abstract

Parameter-efficient fine-tuning (PEFT) and probing enable adaptation of foundation models using only a small number of trainable parameters, making it attractive for video understanding where annotation and computation are expensive. However, video PEFT has focused on adapting image-pretrained models, while standard PEFT methods can also be applied to video representations. These settings are rarely compared and both confine temporal reasoning to a single component of the model, leaving open how temporal context should be distributed across backbone, PEFT and probe. In this work we provide a systematic study of model adaptation strategies for video understanding. We evaluate methods across appearance-focused, motion-focused and spatially dense settings, with a particular focus on scenarios with limited data where parameter-efficiency is most beneficial. Our results provide new insights into PEFT and probing across settings and demonstrate the importance of temporal context allocation for effective video adaptation

Chinese Translation

参数高效微调（PEFT）和探测技术使得基础模型能够仅使用少量可训练参数进行适应，这在注释和计算成本高昂的视频理解任务中尤为吸引人。然而，视频PEFT主要集中在适应图像预训练模型，而标准PEFT方法也可以应用于视频表示。这些设置很少进行比较，并且都将时间推理限制在模型的单一组件上，尚未明确时间上下文应如何在主干网络、PEFT和探测器之间分配。在本研究中，我们对视频理解的模型适应策略进行了系统研究。我们在以外观为中心、以运动为中心和空间密集的设置中评估了各种方法，特别关注数据有限的场景，在这些场景中，参数效率最为重要。我们的结果为不同设置下的PEFT和探测提供了新的见解，并展示了时间上下文分配对于有效视频适应的重要性。

View on arXiv Download PDF AI Translation

cs.CV / 99 / 2606.03868

Unified Video-Action Joint Denoising for Dexterous Action and Data Generation

统一视频-动作联合去噪以实现灵巧动作和数据生成

Wang, Dingrui, Wang, YuAn, Liu, Jinkun, Zhang, Yue, Piccinini, Mattia, Sun, Yu, Betz, Johannes

Abstract

Recent world action models leverage video foundation models by aligning broad visual-dynamics priors with executable robot actions. We revisit this alignment from a distributional perspective. Existing formulations typically narrow the aligned prior into an observation-conditioned policy distribution over future actions. In contrast, we keep the distribution broader by modeling the joint space of interaction videos and executable hand trajectories under multiple conditioning regimes. We propose Donk, a unified video-action denoising model for dexterous hands. With language, an initial image, and the initial hand state, Donk samples future videos and bimanual MANO trajectories as an action policy. Without the image condition, the same denoising architecture samples paired video-action rollouts from a text-conditioned distribution, turning the aligned video prior into a data engine. Across action, video, and text-only generation evaluations, Donk improves dexterous trajectory accuracy, preserves strong video fidelity, and produces smooth text-conditioned action rollouts under the same unified training recipe.

Chinese Translation

近期的世界动作模型通过将广泛的视觉动态先验与可执行的机器人动作对齐，利用视频基础模型。我们从分布的角度重新审视这种对齐。现有的公式通常将对齐的先验缩小为未来动作的观察条件策略分布。相比之下，我们通过在多种条件下建模交互视频和可执行手轨迹的联合空间，保持了分布的广度。我们提出了Donk，一个用于灵巧手的统一视频-动作去噪模型。通过语言、初始图像和初始手状态，Donk采样未来视频和双手MANO轨迹作为动作策略。在没有图像条件的情况下，相同的去噪架构从文本条件分布中采样配对的视频-动作展开，将对齐的视频先验转变为数据引擎。在动作、视频和仅文本生成评估中，Donk提高了灵巧轨迹的准确性，保持了强大的视频保真度，并在相同的统一训练方案下产生了平滑的文本条件动作展开。

View on arXiv Download PDF AI Translation

cs.CV / 100 / 2606.03871

Visual Instruction Tuning Aligns Modalities through Abstraction

视觉指令调优通过抽象对齐模态

Palacios, Luis, Basile, Lorenzo, Doimo, Diego, Cazzaniga, Alberto

Abstract

Visual instruction tuning effectively adapts a pre-trained Large Language Model (LLM) to process image information alongside text. Yet, it remains unclear how visual features are embedded into the layer-wise hierarchy of abstractions of the LLM backbone. Across a diverse set of vision-language architectures, we show that instruction tuning primarily serves as a bridge, embedding visual features directly into the intermediate semantic layers of the LLM, bypassing the early layers devoted to unimodal processing. With probing analyses and causal interventions, we show that these intermediate layers are the semantic core of vision-language processing and play a critical role in the performance on a broad set of multimodal benchmarks. In addition, by comparing the geometry of semantically equivalent visual and textual representations, we find that fine-tuning extends and strengthens the existing abstraction phase, aligning visual features with pre-existing textual ones. Finally, we confirm the functional role of this localized alignment by restricting fine-tuning to intermediate layers alone: this strategy preserves the performance of full fine-tuning on vision-centric benchmarks while reducing training time. Our results suggest that multimodal integration is a localized phenomenon driven by the repurposing of the internal abstraction engine of the LLM.

Chinese Translation

视觉指令调优有效地使预训练的大型语言模型（LLM）适应处理图像信息与文本。然而，视觉特征如何嵌入LLM主干的层次抽象结构仍不清楚。在一系列多样的视觉-语言架构中，我们展示了指令调优主要作为桥梁，直接将视觉特征嵌入LLM的中间语义层，绕过了专注于单模态处理的早期层。通过探测分析和因果干预，我们表明这些中间层是视觉-语言处理的语义核心，并在广泛的多模态基准测试中发挥着关键作用。此外，通过比较语义等效的视觉和文本表示的几何形状，我们发现微调扩展并增强了现有的抽象阶段，使视觉特征与预先存在的文本特征对齐。最后，我们通过将微调限制在中间层来确认这种局部对齐的功能作用：这一策略在视觉中心基准测试中保持了全面微调的性能，同时减少了训练时间。我们的结果表明，多模态整合是一个局部现象，由LLM内部抽象引擎的重新利用驱动。

View on arXiv Download PDF AI Translation

cs.CV / 101 / 2606.03874

DyaPlex: Full-Duplex Speech-Motion Model for Dyadic Interaction

DyaPlex：用于双人互动的全双工语音-运动模型

Nagano, Koki, Liu, Hongyu, Park, Seonwook, Li, Tianye, Mazumdar, Amrita, Jacobsen, Christian, Wang, Shengze, Stengel, Michael, Roy, Rajarshi, Cheung, Ka Chun, See, Simon, De Mello, Shalini

Abstract

We present DyaPlex, a streaming, full-duplex speech-and-motion model designed for dyadic interaction. To capture the continuous and reciprocal nature of human communication, this full-duplex capability empowers the agent to simultaneously perceive and generate both speech and physical motion in a streaming fashion. At its core, our method leverages the strong priors of a foundational full-duplex speech model and integrates a novel motion pathway, thereby achieving fully synchronized multi-modal interaction. Specifically, we design a dual-tower Transformer architecture that preserves the zero-shot conversational reasoning of a frozen base speech model while constructing a deeply coupled, streaming motion pathway. By introducing a unified dyadic token interleaving mechanism and guiding cross-attention via a time-aligned speech-motion RoPE, our model effectively aligns autoregressive motions with rich latent speech features. Trained on the 4,000-hour Seamless Interaction dataset, our model effectively captures cross-speaker dependencies and establishes new state-of-the-art performance across both monadic and dyadic human interaction benchmarks.

Chinese Translation

我们提出了 DyaPlex，这是一种为双人互动设计的流式全双工语音与运动模型。为了捕捉人类沟通的连续性和互惠性，这种全双工能力使得代理能够以流式方式同时感知和生成语音与物理运动。我们的核心方法利用了基础全双工语音模型的强先验，并整合了一条新颖的运动通道，从而实现完全同步的多模态互动。具体而言，我们设计了一种双塔 Transformer 架构，该架构在构建深度耦合的流式运动通道的同时，保留了冻结基础语音模型的零样本对话推理能力。通过引入统一的双人令牌交错机制，并通过时间对齐的语音-运动 RoPE 指导交叉注意力，我们的模型有效地将自回归运动与丰富的潜在语音特征对齐。我们的模型在 4000 小时的无缝互动数据集上进行训练，能够有效捕捉跨说话者的依赖关系，并在单人和双人互动基准测试中建立了新的最先进性能。

View on arXiv Download PDF AI Translation

cs.CV / 102 / 2606.03875

Seg2Track++: Probabilistic Track Validation and Data Association for Multi-Object Tracking and Segmentation

Seg2Track++：用于多目标跟踪和分割的概率轨迹验证与数据关联

Mendonça, Diogo, Barros, Tiago, Premebida, Cristiano, Nunes, Urbano J.

Abstract

Autonomous systems require robust Multi-Object Tracking and Segmentation (MOTS) to operate reliably in dynamic environments, ensuring consistent object identities and precise mask-level delineation. Foundation models such as SAM2 have shown strong zero-shot generalization for segmentation, but their direct application to MOTS is limited by unreliable track association and false-positive propagation. This work introduces Seg2Track++, a framework that integrates instance segmentation with SAM2 and a novel track management module to perform zero-shot MOTS with enhanced temporal consistency. Tracks are associated using Mask Centroid Distance (MCD) and Confidence-Aware Cost Modulation (CCM), while Probabilistic Track Validation (PTV) employs a Bernoulli filter to validate track existence and suppress ghost tracks. Experimental results on KITTI MOTS demonstrate improved identity preservation, reduced false-positive propagation, and robust track management without fine-tuning.

Chinese Translation

自主系统需要强大的多目标跟踪和分割（MOTS）能力，以便在动态环境中可靠运行，确保对象身份的一致性和精确的掩膜级别划分。基础模型如SAM2在分割方面展示了强大的零样本泛化能力，但其在MOTS中的直接应用受到不可靠的轨迹关联和假阳性传播的限制。本研究提出了Seg2Track++，一个将实例分割与SAM2以及新颖的轨迹管理模块相结合的框架，以增强时间一致性进行零样本MOTS。轨迹通过掩膜质心距离（Mask Centroid Distance, MCD）和基于置信度的成本调制（Confidence-Aware Cost Modulation, CCM）进行关联，而概率轨迹验证（Probabilistic Track Validation, PTV）则采用伯努利滤波器来验证轨迹的存在并抑制虚假轨迹。在KITTI MOTS上的实验结果表明，身份保持得到了改善，假阳性传播减少，并且在不进行微调的情况下实现了稳健的轨迹管理。

View on arXiv Download PDF AI Translation

cs.CV / 103 / 2606.03877

MLP Splatting: Object-Centric Neural Fields

MLP Splatting：以对象为中心的神经场

Kim, Shinjeong, Cheng, Yuzhou, Kong, Xin, Kelly, Paul H. J., Davison, Andrew J.

Abstract

3D representations are fundamental to scene rendering, understanding, and interaction. Recent approaches, such as 3D Gaussian Splatting and Neural Radiance Fields, achieve impressive photorealistic novel-view synthesis, but lack the ability to easily decompose scene elements into a few primitives, requiring additional segmentation or grouping for object-level manipulation. We present MLP-Splatting, a method that enables scene decomposition via a few expressive light-field primitives while providing photorealistic novel-view synthesis. MLP-Splatting models each primitive as an independent compact MLP with localized spatial support that predicts radiance and opacity. In contrast to low-level Gaussian primitives or a single global radiance field, our neural primitives provide greater expressive capacity while remaining spatially localized. Rendering is performed through efficient sparse volumetric compositing over ray-primitive interactions. Our primitives are supervised using RGB supervision alone, which yields primitives that represent local scene regions often corresponding to objects or object parts, enabling interactive object-level editing without segmentation masks by selecting a handful of primitives. Our method, augmented with optional semantic feature distillation, enables open-vocabulary scene interaction and open-set instant segmentation. Compared to state-of-the-art methods, we achieve substantially lower memory usage (1/15$\times$) and faster rendering (3$\times$), as we show in our experiments compared to semantic 3DGS methods. Project Page: https://shinjeongkim.com/mlp-splatting

Chinese Translation

三维表示是场景渲染、理解和交互的基础。最近的方法，如3D高斯溅射（3D Gaussian Splatting）和神经辐射场（Neural Radiance Fields），在光线真实感的新视角合成方面取得了令人印象深刻的成果，但缺乏将场景元素轻松分解为少数原始元素的能力，需要额外的分割或分组以进行对象级操作。我们提出了MLP-Splatting，一种通过少数表达性光场原始元素实现场景分解的方法，同时提供光线真实感的新视角合成。MLP-Splatting将每个原始元素建模为一个独立的紧凑型多层感知器（MLP），具有局部空间支持，能够预测辐射度和不透明度。与低级高斯原始元素或单一全局辐射场相比，我们的神经原始元素提供了更大的表达能力，同时保持空间局部化。渲染通过对光线-原始元素交互进行高效的稀疏体积合成来完成。我们的原始元素仅使用RGB监督进行训练，生成的原始元素表示局部场景区域，通常对应于对象或对象部分，从而使得通过选择少数原始元素进行交互式对象级编辑成为可能，而无需分割掩膜。我们的方法通过可选的语义特征蒸馏增强，支持开放词汇的场景交互和开放集的即时分割。与最先进的方法相比，我们在实验中显示出显著更低的内存使用（1/15×）和更快的渲染速度（3×），与语义3DGS方法相比。项目页面：https://shinjeongkim.com/mlp-splatting

View on arXiv Download PDF AI Translation

cs.CV / 104 / 2606.03879

Beyond Encoder Accumulation: Measuring Encoder Roles in Multi-Encoder VLMs

超越编码器累积：测量多编码器视觉语言模型中的编码器角色

Ding, Wei, Zhang, Yudong, Xie, Ruobing, Sun, Xingwu, Chen, Jiansheng, Wang, Yu

Abstract

As foundation models scale toward fusing more heterogeneous visual streams, understanding how diverse encoders interact under joint training becomes a prerequisite for principled design. Yet large vision-language models (LVLMs) currently lack the tools to do so, and parameter-efficient encoder configurations remain hard to identify before training. To re-examine encoder roles under joint training, on the 16-benchmark Cambrian-1 suite we retrain and evaluate all 31 non-empty subsets of five common vision encoders under a unified pipeline (~20k GPU-hours total), and report three findings. First, retraining each subset from scratch reveals encoder rankings that differ from those obtained by masking encoders on a fixed checkpoint, including which encoder ranks first overall. Second, we decompose each encoder's contribution into two axes, Capacity, the score an encoder reaches on its own, and Necessity, the drop when it is removed from the full pool. The two axes are not interchangeable. Pairing the two highest-Capacity encoders is suboptimal, while pairing a high-Capacity anchor with an adaptive complement matches the full five-encoder model. Adding further encoders beyond this pair yields only marginal gains. Third, at fixed parameter count, per-encoder pre-projector effective rank explains the residual score variation. The strongest pairs combine an anchor whose rank survives joint training with a complement whose rank expands under it, suggesting that higher-rank, less-collapsed projector inputs correspond to a more favorable optimization regime at the encoder-projector interface. Together, the Capacity-Necessity decomposition and the pre-projector rank analysis, along with comprehensive evaluation through retraining, expose a methodological gap in multi-encoder LVLM design, and offer concrete primitives for closing it.

Chinese Translation

随着基础模型向融合更多异构视觉流的方向发展，理解多样化编码器在联合训练下的交互成为原则性设计的前提。然而，目前大型视觉语言模型（LVLMs）缺乏相应的工具，且在训练前识别参数高效的编码器配置仍然困难。为了重新审视联合训练下的编码器角色，我们在16个基准的Cambrian-1套件上重新训练并评估了五个常见视觉编码器的所有31个非空子集，采用统一的流程（总计约20,000 GPU小时），并报告了三项发现。首先，从头开始重新训练每个子集揭示了与在固定检查点上屏蔽编码器所获得的排名不同的编码器排名，包括哪个编码器整体排名第一。其次，我们将每个编码器的贡献分解为两个维度：容量（Capacity），即编码器独立达到的得分，以及必要性（Necessity），即从完整池中移除该编码器后得分的下降。这两个维度不可互换。将两个最高容量的编码器配对是次优的，而将一个高容量的锚点与一个自适应的补充编码器配对则能匹配完整的五编码器模型。超出这一对的进一步编码器仅带来边际收益。第三，在固定参数数量下，每个编码器的预投影器有效排名解释了剩余得分的变化。最强的配对组合是一个在联合训练中排名保持的锚点与一个在联合训练中排名扩展的补充编码器，这表明较高排名、较少压缩的投影器输入与编码器-投影器接口处更有利的优化机制相对应。综上所述，容量-必要性分解和预投影器排名分析，以及通过重新训练进行的全面评估，揭示了多编码器LVLM设计中的方法论缺口，并提供了具体的原语以弥补这一缺口。

View on arXiv Download PDF AI Translation

cs.CV / 105 / 2606.03888

CoralBay: A Self-Supervised CT Foundation Model

CoralBay：一种自监督的CT基础模型

Gatopoulos, Ioannis, Känzig, Nicolas, Otálora, Sebastian, Tang, Fei

Abstract

Self-supervised learning has enabled large-scale pre-training on 2D natural images, producing general-purpose visual representations that transfer effectively across tasks. However, many medical imaging modalities, such as CT scans, are inherently three-dimensional and differ fundamentally from natural images in both structure and semantics. Volumetric modalities capture spatial continuity, organ anatomy, and intensity-based tissue properties (e.g., Hounsfield Units), which are not adequately modeled by 2D pre-training. To bridge this gap, we introduce CoralBay, a self-distillation framework that extends DINO by using a hierarchical 3D Swin backbone and applying self-distillation to concatenated multi-scale features, enabling data-efficient self-supervised learning of rich spatial representations that encode both global semantics and fine-grained local structure. As a result, CoralBay transfers effectively to a wide range of downstream radiological tasks, demonstrating strong and consistent performance across diverse anatomical targets. In addition, we contribute to the open-source \eva framework by introducing a public, reproducible 3D radiology leaderboard that unifies multiple datasets and establishes a standardized benchmark for evaluating volumetric representation learning methods.

Chinese Translation

自监督学习使得在二维自然图像上进行大规模预训练成为可能，产生了通用的视觉表征，能够有效地在不同任务之间迁移。然而，许多医学成像方式，如CT扫描，固有地是三维的，并且在结构和语义上与自然图像有根本的不同。体积成像方式捕捉空间连续性、器官解剖结构和基于强度的组织特性（例如，Hounsfield单位），这些特性在二维预训练中未得到充分建模。为了解决这一问题，我们提出了CoralBay，一个自蒸馏框架，通过使用分层的3D Swin主干网络并对连接的多尺度特征应用自蒸馏，扩展了DINO，能够实现数据高效的自监督学习，学习丰富的空间表征，编码全局语义和细粒度局部结构。因此，CoralBay能够有效地迁移到广泛的下游放射学任务中，在不同解剖目标上表现出强大且一致的性能。此外，我们通过引入一个公共的、可重复的3D放射学排行榜，为开放源代码的 extit{eva} 框架做出贡献，该排行榜统一了多个数据集，并建立了一个标准化的基准，用于评估体积表征学习方法。

View on arXiv Download PDF AI Translation

cs.CV / 106 / 2606.03890

OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs

OVO-S-Bench：多模态大语言模型中流媒体空间智能的分层基准

Li, Yifei, Liu, Pengyiang, Zang, Yuhang, Shi, Zhongyue, Fu, Qi, Hao, Hongye, Lu, Jiwen

Abstract

Multimodal agents in robotics, AR, and autonomous driving must reason about places and layouts from continuous egocentric streams, often using evidence outside the current view. Existing benchmarks either evaluate offline over full videos or target events rather than spatial structure. We introduce OVO-S-Bench, a fully human-annotated benchmark for streaming spatial intelligence, comprising 1,680 questions over 348 source videos. Annotation involves 12 trained annotators, each also serving as a blind cross-reviewer, across roughly 804 person-hours of multi-round quality assurance. Each question carries a query timestamp and an evidence interval, and at evaluation, the model sees only the prefix preceding the query. Questions span four levels of increasing abstraction: instantaneous egocentric perception, spatiotemporal context tracking, spatial simulation and reasoning, and allocentric mapping. Across 38 proprietary and open-source MLLMs, Gemini-3.1-Pro trails human experts by 27 points, 59.2 vs. 86.6, with allocentric mapping as the dominant bottleneck. Notably, streaming and spatially fine-tuned MLLMs underperform their own backbones. We further find that chain-of-thought reasoning amplifies spatial errors when ungrounded in the stream. By exposing these limitations, OVO-S-Bench establishes a demanding testbed for next-generation streaming spatial MLLMs.

Chinese Translation

机器人、增强现实（AR）和自动驾驶中的多模态智能体必须从持续的自我中心流中推理地点和布局，通常使用当前视图之外的证据。现有基准要么在完整视频上进行离线评估，要么针对事件而非空间结构。我们提出了OVO-S-Bench，这是一个完全由人类注释的流媒体空间智能基准，包含1,680个问题，基于348个源视频。注释工作由12名经过培训的注释员完成，每位注释员还担任盲交叉审查员，整个过程耗时约804人小时的多轮质量保证。每个问题都带有查询时间戳和证据区间，在评估时，模型仅能看到查询之前的前缀。问题涵盖四个逐渐抽象的层次：瞬时自我中心感知、时空上下文跟踪、空间模拟与推理，以及他心映射（allocentric mapping）。在38个专有和开源的多模态大语言模型（MLLMs）中，Gemini-3.1-Pro的表现比人类专家低27分，分别为59.2与86.6，而他心映射是主要瓶颈。值得注意的是，流媒体和空间微调的多模态大语言模型的表现低于其自身的基础模型。我们进一步发现，当推理未基于流时，思维链推理会放大空间错误。通过揭示这些局限性，OVO-S-Bench为下一代流媒体空间多模态大语言模型建立了一个严苛的测试平台。

View on arXiv Download PDF AI Translation

cs.CV / 107 / 2606.03893

Electromagnetic Navigation for Femoral Osteotomy Using High-Accuracy X-ray-to-CT Registration

基于高精度X射线与CT配准的股骨截骨电磁导航

Flepp, Roman, Nieuwland, Arend, Sigrist, Bastian, Fürnstahl, Philipp, Calvet, Lilian, Dreher, Thomas

Abstract

Accurate execution of preoperative plans in corrective femoral osteotomies remains challenging. Current techniques are limited by variable accuracy, invasiveness, and radiation exposure, with free-hand methods and patient-specific instrumentation (PSI) often requiring >30 and >6 fluoroscopic images, respectively. We present an integrated, electromagnetic tracking (EMT)-based navigation system for femoral osteotomies that minimizes dissection and intraoperative fluoroscopy. The system couples CT-based preoperative planning with one-time intraoperative C-arm calibration and accurate X-ray-to-CT registration from two fluoroscopic images acquired at initialization. This enables real-time, fluoroscopy-free EMT navigation of the saw blade and bone fragments relative to the preoperative plan, and is compatible with uniplanar and biplanar osteotomies. In a feasibility study using 18 synthetic femora, EMT guidance significantly outperformed free-hand execution in total angular error ($(3.05 \pm 0.75)^\circ$ vs.\ $(6.32 \pm 2.36)^\circ$, $p=0.031$), assuming the same minimal surgical exposure for both. No EMT-guided trials exceeded the >5{\deg} clinical threshold, whereas free-hand produced 4 outliers of 6 trials. The system achieved statistical equivalence ($\pm 2^\circ$, $\pm 2,\text{mm}$) to PSI for total angular ($p \le 0.02$) and total translational ($p=0.048$) errors, with no significant differences in user questionnaire scores. By transferring preoperative plans using only two fluoroscopic images while matching PSI accuracy without additional surgical exposure, the proposed system motivates subsequent cadaveric and clinical validation.

Chinese Translation

在矫正性股骨截骨术中，准确执行术前计划仍然具有挑战性。目前的技术受到可变精度、侵入性和辐射暴露的限制，自由手术方法和患者特定仪器（PSI）通常分别需要超过30张和6张透视图。我们提出了一种集成的基于电磁跟踪（EMT）的股骨截骨导航系统，该系统最小化了剥离和术中透视。该系统将基于CT的术前规划与一次性术中C臂校准和从初始化时获取的两张透视图进行准确的X射线与CT配准相结合。这使得相对于术前计划，锯片和骨片的实时、无透视EMT导航成为可能，并且兼容单平面和双平面截骨术。在一项使用18个合成股骨的可行性研究中，EMT引导在总角度误差方面显著优于自由手术执行（$(3.05 imes 0.75)^ ext{°}$ vs. $(6.32 imes 2.36)^ ext{°}$, $p=0.031$），假设两者的最小手术暴露相同。没有EMT引导的试验超过5{ ext{°}}的临床阈值，而自由手术则产生了6次试验中的4个异常值。该系统在总角度误差（$p ext{≤} 0.02$）和总平移误差（$p=0.048$）方面达到了与PSI的统计等效性（$ ext{±} 2^ ext{°}$, $ ext{±} 2 ext{mm}$），用户问卷评分没有显著差异。通过仅使用两张透视图转移术前计划，同时在没有额外手术暴露的情况下匹配PSI的精度，所提出的系统激励后续的尸体和临床验证。

View on arXiv Download PDF AI Translation

cs.CV / 108 / 2606.03903

An Attention-Based Denoising Model for Diffusion Weighted Imaging

基于注意力的扩散加权成像去噪模型

Verma, Prithviraj, Kumar, Pawan, Deshani, Chandan, Tripathi, Prasun Chandra

Abstract

Diffusion-weighted imaging (DWI) is used for whole-body cancer screening, but it typically requires a long acquisition time. When the scan time is reduced, the image quality often suffers, leading to increased noise in the scans. Magnitude reconstruction in DWI introduces signal-dependent Rician noise, which makes denoising more challenging for conventional convolution-based methods. To address this limitation, we propose a noise-aware attention-driven denoising framework that integrates hierarchical Swin Transformer window attention with transformer-based multi-dimensional gated refinement for DWI restoration. The model incorporates explicit noise-level conditioning and residual reconstruction to enable adaptive suppression of heteroscedastic noise across a wide range of corruption levels. Experimental evaluation on corrupted DWI scans demonstrates strong restoration performance. Our model achieves a mean PSNR of 33.69~dB and SSIM of 0.8539 across noise levels from 1\% to 15\%, while maintaining stable behavior under severe noise conditions. These results indicate that attention-guided contextual modeling combined with channel-adaptive refinement provides a robust and generalizable solution for DWI denoising.

Chinese Translation

扩散加权成像（DWI）用于全身癌症筛查，但通常需要较长的采集时间。当扫描时间缩短时，图像质量往往会下降，导致扫描中的噪声增加。DWI中的幅度重建引入了信号依赖的Rician噪声，这使得传统卷积方法的去噪变得更加困难。为了应对这一限制，我们提出了一种噪声感知的注意力驱动去噪框架，该框架将分层Swin Transformer窗口注意力与基于变换器的多维门控细化相结合，用于DWI恢复。该模型结合了显式的噪声水平条件和残差重建，以实现对广泛腐蚀水平的异方差噪声的自适应抑制。在受损的DWI扫描上的实验评估表明了强大的恢复性能。我们的模型在1 ext{%}到15 ext{%}的噪声水平下实现了平均PSNR为33.69~dB和SSIM为0.8539，同时在严重噪声条件下保持稳定的表现。这些结果表明，结合通道自适应细化的注意力引导上下文建模为DWI去噪提供了一个稳健且具有可推广性的解决方案。

View on arXiv Download PDF AI Translation

cs.CV / 109 / 2606.03909

SparseStreet: Sparse Gaussian Splatting for Real-Time Street Scene Simulation

SparseStreet：用于实时街景模拟的稀疏高斯点云技术

Wuwu, Qingpo, Wei, Xiaobao, Chen, Peng, Huang, Nan, Zhao, Zhongyu, Wang, Hao, Lu, Ming, Ma, Ningning, Zhang, Shanghang

Abstract

While 3D Gaussian Splatting has shown promising results in street scene reconstruction, existing methods require massive numbers of Gaussian primitives to capture fine details, leading to prohibitive storage costs and slow rendering speeds. We observe that dynamic objects (e.g., vehicles and pedestrians) demand high-fidelity representations to maintain temporal consistency, while static background regions often contain substantial redundancy. Motivated by this, we propose SparseStreet, a general compression framework specifically designed for street scenes. First, we introduce a node-based learnable pruning strategy that systematically removes low-contributing Gaussian primitives while preserving visually critical regions. Second, after the scene representation stabilizes, we apply background compression, further reducing redundancy in static regions. Our method effectively preserves the geometry and appearance of dynamic objects while significantly reducing the total number of Gaussian primitives. Extensive experiments on the Waymo and nuScenes demonstrate that SparseStreet achieves up to 80% compression ratio with minimal quality degradation, enabling resource-efficient, high-fidelity dynamic scene reconstruction. Project website: https://sparsestreet.github.io/.

Chinese Translation

尽管3D高斯点云技术在街景重建中展现了良好的效果，但现有方法需要大量的高斯原语来捕捉细节，导致存储成本高昂且渲染速度缓慢。我们观察到，动态物体（如车辆和行人）需要高保真度的表示以维持时间一致性，而静态背景区域往往包含大量冗余信息。基于此，我们提出了SparseStreet，一个专门为街景设计的通用压缩框架。首先，我们引入了一种基于节点的可学习剪枝策略，系统性地去除低贡献的高斯原语，同时保留视觉上重要的区域。其次，在场景表示稳定后，我们应用背景压缩，进一步减少静态区域的冗余。我们的方法有效地保留了动态物体的几何形状和外观，同时显著减少了高斯原语的总数。在Waymo和nuScenes上的大量实验表明，SparseStreet实现了高达80%的压缩比，且质量下降最小，从而实现了资源高效的高保真动态场景重建。项目网站：https://sparsestreet.github.io/

View on arXiv Download PDF AI Translation

cs.CV / 110 / 2606.03911

Bootstrap Your Generator: Unpaired Visual Editing with Flow Matching

引导你的生成器：基于流匹配的无配对视觉编辑

Tewel, Yoad, Atzmon, Yuval, Chechik, Gal, Wolf, Lior

Abstract

Modern generative models possess a deep understanding of visual content, yet training them for image editing typically requires massive datasets of paired examples. This limits scalability, especially for video editing where collecting paired data is prohibitively expensive. We propose Bootstrap Your Generator (ByG), a general framework for unpaired training of flow matching editing models. It leverages the base model's knowledge without any external signal. Our approach pairs instruction-following cues extracted from the frozen model with cycle-consistency for structure preservation. To make this tractable, we propose to route gradients from downstream losses over clean predictions to noisy training states. We demonstrate state-of-the-art results on challenging data-scarce image and video editing scenarios. Extensive evaluations and user studies show that our method effectively generalizes to unseen domains and outperforms supervised baselines trained on millions of samples. Analysis reveals that our gradient routing bridges the train-inference gap, and extracting semantic cues from a base model provides a robust training signal that obviates the need for external reward models.

Chinese Translation

现代生成模型对视觉内容具有深刻的理解，然而，训练它们进行图像编辑通常需要大量配对示例的数据集。这限制了可扩展性，特别是在视频编辑中，收集配对数据的成本极高。我们提出了引导你的生成器（Bootstrap Your Generator, ByG），这是一个用于无配对流匹配编辑模型训练的通用框架。它利用基础模型的知识，而无需任何外部信号。我们的方法将从冻结模型中提取的遵循指令的线索与循环一致性相结合，以保持结构。为了使这一过程可行，我们建议将下游损失的梯度从干净的预测路由到嘈杂的训练状态。我们在数据稀缺的图像和视频编辑场景中展示了最先进的结果。广泛的评估和用户研究表明，我们的方法能够有效地推广到未见领域，并且在数百万样本上训练的监督基线之上表现更佳。分析表明，我们的梯度路由弥合了训练与推理之间的差距，从基础模型中提取语义线索提供了一个稳健的训练信号，从而消除了对外部奖励模型的需求。

View on arXiv Download PDF AI Translation

cs.CV / 111 / 2606.03915

PatchScene: Patch-based Voxel Diffusion for Large-Scale Scene Completion

PatchScene：基于补丁的体素扩散用于大规模场景补全

Xu, Qingdong, Zhu, Jiajun, Zhu, Shilin, He, Xinjing, Lu, Chao, Wang, Huanran, Zhang, Jiyao

Abstract

We propose PatchScene, a novel diffusion-based framework for large-scale LiDAR scene completion. Unlike existing methods that rely on global latent representations or dense voxel grids, PatchScene adopts a patch-based voxel diffusion paradigm that explicitly generates fine-grained geometry within localized 3D regions. To ensure coherent reconstruction at both spatial and temporal scales, we introduce a confidence-guided spatio-temporal fusion mechanism that integrates overlapping patches and adjacent frames in a unified generative process. Furthermore, we design an Annular-Flow diffusion strategy that leverages the radial density pattern of LiDAR scans to progressively propagate high-fidelity information from near-range to far-range regions, enabling spatially unbounded scene completion. Extensive experiments on the SemanticKITTI benchmark demonstrate that PatchScene achieves state-of-the-art performance across all standard metrics, surpassing previous approaches in both geometric accuracy and temporal consistency. Remarkably, the model trained on 20 m LiDAR ranges generalizes effectively to 50 m scenes without retraining, highlighting its strong scalability and generalization capability for real-world autonomous driving applications.

Chinese Translation

我们提出了PatchScene，这是一种用于大规模LiDAR场景补全的新型基于扩散的框架。与依赖于全局潜在表示或密集体素网格的现有方法不同，PatchScene采用了一种基于补丁的体素扩散范式，明确地在局部3D区域内生成细粒度几何形状。为了确保在空间和时间尺度上都能实现一致的重建，我们引入了一种基于置信度的时空融合机制，该机制在统一的生成过程中整合重叠补丁和相邻帧。此外，我们设计了一种环流扩散策略，该策略利用LiDAR扫描的径向密度模式，逐步从近距离区域向远距离区域传播高保真信息，从而实现空间上无限制的场景补全。在SemanticKITTI基准上的大量实验表明，PatchScene在所有标准指标上都达到了最先进的性能，在几何精度和时间一致性方面超越了先前的方法。值得注意的是，在20米LiDAR范围内训练的模型能够有效地推广到50米场景，而无需重新训练，突显了其在现实世界自动驾驶应用中的强大可扩展性和泛化能力。

View on arXiv Download PDF AI Translation

cs.CV / 112 / 2606.03920

Benchmarking Visual State Tracking in Multimodal Video Understanding

多模态视频理解中的视觉状态跟踪基准测试

Yu, Sihyun, Ma, Nanye, Huang, Pinzhi, Lee, Hyunseok, Yang, Shusheng, Choi, June Suk, Brown, Ellis, Michel, Oscar, Zheng, Boyang, Shin, Jinwoo, Xie, Saining

Abstract

Understanding a video requires more than recognizing isolated moments, as humans continuously track entities, states, and events over time. This capacity for visual state tracking is fundamental to video understanding, yet remains underexplored in current evaluations of Multimodal Large Language Models (MLLMs). We introduce Visual STAte Tracking benchmark (VSTAT), a video-based benchmark designed to diagnose visual state tracking in MLLMs. VSTAT consists of 834 clips drawn from both synthetic and real-world videos, paired with 1,500 questions that cannot be answered from any single frame or short segment, requiring continuous perception and integration of events across the entire video stream. Despite their strong performance on existing video benchmarks, we find that state-of-the-art MLLMs perform far below humans and only modestly above answer-prior baselines. To analyze this gap, we compare MLLMs' thinking traces with the underlying video stream to understand why and when MLLMs fail on VSTAT. We find that MLLMs reason and track correctly in text, but fail at visually perceiving the events they need to track. Finally, our preliminary evaluation suggests that recent agentic approaches, including MLLM-based video agents and coding agents, do not readily resolve these failures, still falling short on VSTAT.

Chinese Translation

理解视频不仅仅是识别孤立的瞬间，因为人类会持续跟踪实体、状态和事件。这种视觉状态跟踪的能力是视频理解的基础，但在当前对多模态大型语言模型（MLLMs）的评估中仍然未得到充分探索。我们引入了视觉状态跟踪基准（Visual STAte Tracking benchmark，VSTAT），这是一个基于视频的基准，旨在诊断MLLMs中的视觉状态跟踪。VSTAT由834个片段组成，这些片段来自合成视频和真实世界视频，并配有1500个问题，这些问题无法通过任何单一帧或短片段回答，要求对整个视频流中的事件进行持续感知和整合。尽管在现有视频基准测试中表现强劲，我们发现最先进的MLLMs的表现远低于人类，仅略高于基于答案的基线。为了分析这一差距，我们比较了MLLMs的思维轨迹与基础视频流，以理解MLLMs在VSTAT上失败的原因和时机。我们发现，MLLMs在文本中进行推理和跟踪是正确的，但在视觉上感知他们需要跟踪的事件时失败。最后，我们的初步评估表明，最近的代理方法，包括基于MLLM的视频代理和编码代理，并未能有效解决这些失败，仍在VSTAT上表现不足。

View on arXiv Download PDF AI Translation

cs.CV / 113 / 2606.03921

GARDEN: Gravity-Aligned Reconstruction of Disentangled ENvironments from RGB images

GARDEN：基于重力对齐的从RGB图像中解耦环境的重建

Sun, Jiahao, Wei, Dingkun, Shen, Zehong, Zhou, Hongyu, Shen, Yujun, Li, Liang

Abstract

Converting multi-view RGB observations into simulation-ready 3D environments remains challenging because current reconstruction pipelines produce monolithic scene representations without explicit physical structure. They are typically defined up to an arbitrary global rotation and entangle rigid foreground objects with background geometry, which hinders stable physical interaction. Existing solutions often recover interactivity by replacing reconstructed objects with retrieved CAD assets, but this introduces a slow retrieval-and-replacement stage and weakens scene-specific geometric fidelity. We propose GARDEN, an RGB-only framework that reformulates reconstruction as physically-grounded scene factorization and outputs a structured hybrid scene representation. The key idea is to use gravity as a universal physical prior: we first align the reconstruction to a unified Gravity-View frame to resolve gauge ambiguity, then recover object-centric rigid meshes with accurate 6-DoF placement, and finally remove duplicate object geometry from the background through conditional 3D point classification. The resulting representation combines explicit rigid bodies with a decoupled background, enabling direct physics simulation while preserving visual realism. Experiments on both simulated and real multi-view scenes show that GARDEN improves object placement reliability, disentanglement quality, and rendering-simulation efficiency compared with retrieval-based baselines.

Chinese Translation

将多视角RGB观测转换为可用于模拟的3D环境仍然具有挑战性，因为当前的重建流程生成的场景表示是单一的，没有明确的物理结构。它们通常在任意全局旋转下定义，并将刚性前景物体与背景几何体纠缠在一起，这妨碍了稳定的物理交互。现有的解决方案通常通过用检索到的CAD资产替换重建物体来恢复交互性，但这引入了缓慢的检索和替换阶段，并削弱了场景特定的几何保真度。我们提出了GARDEN，这是一个仅基于RGB的框架，将重建重新表述为物理基础的场景因式分解，并输出结构化的混合场景表示。关键思想是使用重力作为通用物理先验：我们首先将重建对齐到统一的重力视图框架，以解决量规歧义，然后恢复以物体为中心的刚性网格，并准确放置6自由度，最后通过条件3D点分类从背景中移除重复的物体几何。最终的表示结合了明确的刚性物体和解耦的背景，使得直接的物理模拟成为可能，同时保持视觉真实感。在模拟和真实的多视角场景上的实验表明，与基于检索的基线相比，GARDEN提高了物体放置的可靠性、解耦质量和渲染-模拟效率。

View on arXiv Download PDF AI Translation

cs.CV / 114 / 2606.03925

Adaptive Causal Alignment for High-Confidence Adversarial Training

高置信度对抗训练的自适应因果对齐

Luo, Zhiming, Zhang, Kejia, Lai, Yingxin, Wu, Junwei, Weng, Juanjuan, Li, Shaozi

Abstract

Inverse adversarial training leverages high-confidence predictions to stabilize robust learning, yet we uncover a critical paradox: high confidence often stems from overfitting to non-causal background correlations rather than intrinsic object semantics. Our investigation reveals that visual context functions as a dual-natured signal, serving as either a necessary supportive prior or a spurious confounder. This insight renders existing blind suppression strategies flawed, as they inevitably lead to severe Feature Loss. To resolve this, we propose High-Confidence Causally Aligned Training (HICAT), a unified framework that establishes a Semantic Equilibrium. Operating on a ``Measure-Debias-Align'' pipeline, HICAT integrates a Learnable Background-Bias Estimator (LBBE) to adaptively diagnose context utility. Guided by this diagnosis, an Adaptive Debiasing mechanism performs surgical logit rectification, complemented by a geometrically grounded Foreground Logit Orthogonal Enhancement (FLOE) loss to enforce rigorous feature disentanglement. Extensive experiments on CIFAR-10, CIFAR-100, and ImageNet-1K demonstrate that HICAT consistently improves over matched baselines across diverse architectures (CNNs and ViTs) while significantly reducing the robust generalization gap.

Chinese Translation

逆向对抗训练利用高置信度预测来稳定鲁棒学习，然而我们发现了一个关键悖论：高置信度往往源于对非因果背景相关性的过拟合，而非内在对象语义。我们的研究揭示了视觉上下文作为双重性质信号的功能，既可以作为必要的支持先验，也可以作为虚假的混淆因素。这一洞察使得现有的盲目抑制策略显得缺陷，因为它们不可避免地导致严重的特征损失。为了解决这个问题，我们提出了高置信度因果对齐训练（High-Confidence Causally Aligned Training, HICAT），这是一个建立语义平衡的统一框架。HICAT在“测量-去偏-对齐”流程中运行，集成了可学习的背景偏差估计器（Learnable Background-Bias Estimator, LBBE）以自适应地诊断上下文效用。在这一诊断的指导下，自适应去偏机制执行精确的logit修正，并辅以几何基础的前景logit正交增强（Foreground Logit Orthogonal Enhancement, FLOE）损失，以强制实施严格的特征解耦。在CIFAR-10、CIFAR-100和ImageNet-1K上的广泛实验表明，HICAT在多种架构（卷积神经网络和视觉变换器）上始终优于匹配的基线，同时显著减少了鲁棒泛化差距。

View on arXiv Download PDF AI Translation

cs.CV / 115 / 2606.03951

Demo2Tutorial: From Human Experience to Multimodal Software Tutorials

Demo2Tutorial：从人类经验到多模态软件教程

Bai, Zechen, Chen, Zhiheng, Lin, Yiqi, Lin, Kevin Qinghong, Gao, Difei, Guo, Xiangwu, Wang, Xin, Shou, Mike Zheng

Abstract

Human experience in digital environments offers a vast, underexplored resource of authentic, untrimmed interactions that contain rich procedural knowledge. We introduce Demo2Tutorial, a framework that transforms this experience captured via screen recordings and interaction logs into structured, multimodal software tutorials for teaching both humans and agents. Demo2Tutorial first collects human experience via a dedicated recorder, then parses raw experience using a multimodal Action Parser to reconstruct perception, action, and intent. A Step Planner then abstracts these steps into hierarchical task graphs representing goals and steps. Finally, a Tutorial Composer transforms the parsed experience into structured, reusable image-text instructions. We evaluate the tutorial generation quality on a new benchmark derived from official software documentation. We further demonstrate that this distilled representation benefits (i) human learning, by automatically generating multimodal tutorials, and (ii) agent learning, by improving downstream GUI-agent planning and generalization. Experiments show Demo2Tutorial produces high-quality tutorials that surpass human-authored ones and significantly outperform baseline methods, while enabling both faster human task completion and improved GUI agent planning, demonstrating that structured tutorials distilled from human experience can serve as effective knowledge representations for advancing both human learning and agent capabilities. Code and data will be available at https://github.com/showlab/Demo2Tutorial.

Chinese Translation

数字环境中的人类经验提供了丰富且尚未充分探索的真实、未经修剪的互动资源，蕴含着丰富的过程知识。我们介绍了Demo2Tutorial，一个将通过屏幕录制和互动日志捕获的人类经验转化为结构化的多模态软件教程的框架，以用于教学人类和智能体。Demo2Tutorial首先通过专用录制器收集人类经验，然后使用多模态动作解析器（Multimodal Action Parser）解析原始经验，以重构感知、行动和意图。接着，步骤规划器（Step Planner）将这些步骤抽象为表示目标和步骤的层次任务图。最后，教程编排器（Tutorial Composer）将解析后的经验转化为结构化的、可重用的图文指令。我们在一个基于官方软件文档的新基准上评估了教程生成质量。我们进一步证明，这种提炼的表示对（i）人类学习有益，通过自动生成多模态教程，以及（ii）智能体学习，通过改善下游图形用户界面（GUI）智能体的规划和泛化。实验表明，Demo2Tutorial生成的高质量教程超越了人类撰写的教程，并显著优于基线方法，同时加快了人类任务完成速度并改善了GUI智能体的规划，证明了从人类经验提炼的结构化教程可以作为有效的知识表示，促进人类学习和智能体能力的提升。代码和数据将可在 https://github.com/showlab/Demo2Tutorial 获取。

View on arXiv Download PDF AI Translation

cs.CV / 116 / 2606.03954

VLESA: Vision-Language Embodied Safety Agent for Human Activity Monitoring

VLESA：用于人类活动监测的视觉-语言具身安全代理

Hu, Hanjiang, Pan, Yiyuan, Li, Jiaxing, Luo, Xusheng, Robey, Alexander, Li, Na, Wang, Yebin, Liu, Changliu

Abstract

As AI systems increasingly assist humans in physical tasks, ensuring safety becomes paramount -- physical actions carry immediate and irreversible consequences that digital errors do not. We introduce the Vision-Language Embodied Safety Agent (VLESA), a framework that monitors human activities from egocentric video and triggers real-time safety interventions when dangerous actions are predicted. VLESA addresses intent-dependent safety where identical actions can be safe or dangerous depending on context. A dataset pairing egocentric frames with goal-conditioned safety annotations is introduced, enabling a goal-conditioned safety Q-filter trained via GRPO that evaluates actions with respect to inferred intent without retraining. On top of that, an intent-action prediction agent is proposed to jointly infer goals and predict future actions from video. On the ASIMOV-2.0 benchmark, VLESA achieves higher intervention accuracy at the exact ground-truth frame compared to baselines, while the GRPO-trained Q-filter improves action safety by over 41 percentage points through goal-conditioned constrained decoding. Code is available at https://github.com/HanjiangHu/VLESA.

Chinese Translation

随着人工智能系统越来越多地协助人类进行物理任务，确保安全性变得至关重要——物理行为带来的后果是即时且不可逆的，而数字错误则不然。我们提出了视觉-语言具身安全代理（VLESA），这是一个从自我中心视频监测人类活动并在预测到危险行为时触发实时安全干预的框架。VLESA 解决了意图依赖的安全性问题，即相同的行为在不同的上下文中可能是安全的或危险的。我们引入了一个数据集，将自我中心帧与目标条件安全注释配对，使得通过 GRPO 训练的目标条件安全 Q 过滤器能够在不重新训练的情况下，根据推断的意图评估行为。此外，我们提出了一种意图-行为预测代理，能够从视频中共同推断目标并预测未来行为。在 ASIMOV-2.0 基准测试中，VLESA 在准确的真实帧上实现了比基线更高的干预准确率，而 GRPO 训练的 Q 过滤器通过目标条件约束解码将行为安全性提高了超过 41 个百分点。代码可在 https://github.com/HanjiangHu/VLESA 获取。

View on arXiv Download PDF AI Translation

cs.CV / 117 / 2606.03971

Video-Mirai: Autoregressive Video Diffusion Models Need Foresight

视频-Mirai：自回归视频扩散模型需要前瞻性

Yu, Yonghao, Huang, Lang, Li, Runyi, Wang, Zerun, Yamasaki, Toshihiko

Abstract

Causal video generators must predict from the past, but they need not learn only from it. In streaming autoregressive video diffusion, each emitted segment becomes a commitment that future segments must preserve. Standard training, however, only asks each causal state to explain the present. This creates what we call a representation-level planning gap: states that fit the current segment may discard identity, layout, and motion information needed for a consistent future. We introduce Video-Mirai, a training-only method that closes this gap without changing causal inference: the generator rolls out causally, a frozen foresight encoder reads the completed rollout non-causally, and a lightweight predictor distills the resulting stopped-gradient targets into causal states. Future frames supervise representations, never generator inputs. At inference, the encoder and predictor are discarded, leaving the original architecture, per-step FLOPs, and KV-cache behavior unchanged. Video-Mirai improves a strong Causal-Forcing baseline on 5-second VBench from 83.8 to 84.6 in terms of Total Score. On 30-second rollouts beyond the training horizon, subject consistency improves from 84.9 to 88.5 and background consistency from 90.2 to 91.9. Ablations identify future-conditioned targets as the key ingredient, and probes show that future frames become more decodable from current features. Causality should constrain inference, not representation supervision. Our study highlights that visual autoregressive models need foresight. Project page: https://y0uroy.github.io/Video-Mirai.

Chinese Translation

因果视频生成器必须从过去进行预测，但它们不必仅仅依赖于过去。在流式自回归视频扩散中，每个发出的片段都成为未来片段必须保留的承诺。然而，标准训练仅要求每个因果状态解释当前状态。这造成了我们所称的表示层级规划差距：适合当前片段的状态可能会丢弃一致未来所需的身份、布局和运动信息。我们提出了视频-Mirai，这是一种仅通过训练的方法，能够在不改变因果推理的情况下弥补这一差距：生成器因果展开，一个冻结的前瞻编码器非因果地读取完成的展开，而一个轻量级预测器将产生的停止梯度目标提炼为因果状态。未来帧监督表示，而非生成器输入。在推理时，编码器和预测器被丢弃，保留原始架构、每步的FLOPs和KV-cache行为不变。视频-Mirai在5秒的VBench上将强大的因果强制基线的总分从83.8提高到84.6。在超出训练范围的30秒展开中，主题一致性从84.9提高到88.5，背景一致性从90.2提高到91.9。消融实验表明，未来条件目标是关键成分，探测显示未来帧从当前特征中变得更易解码。因果性应约束推理，而非表示监督。我们的研究强调了视觉自回归模型需要前瞻性。项目页面：https://y0uroy.github.io/Video-Mirai。

View on arXiv Download PDF AI Translation

cs.CV / 118 / 2606.03972

AAD-1: Asymmetric Adversarial Distillation for One-Step Autoregressive Video Generation

AAD-1：用于一步自回归视频生成的非对称对抗蒸馏

Li, Haobo, Zeng, Yanhong, Lu, Yunhong, Zhu, Jiapeng, Ouyang, Hao, Wang, Qiuyu, Cheng, Ka Leong, Shen, Yujun, Zhang, Zhipeng

Abstract

We present AAD-1, an Asymmetric Adversarial Distillation framework for One-step autoregressive image-to-video generation. State-of-the-art methods adopt adversarial distillation but suffer from motion collapse and training instability, resulting in static videos. AAD-1 addresses these challenges through two key designs in architecture and training strategy. Our key architectural insight is to break the symmetry between generator and discriminator. While the generator remains causal to preserve autoregressive sampling capability, the discriminator attends bidirectionally over the full spatiotemporal context and produces a single holistic realism score for the entire video sequence. This asymmetric design enables the discriminator to effectively detect global temporal failures and long-range drift that cause motion collapse in autoregressive generation. To stabilize training, we introduce a phased strategy that first uses distribution matching to bootstrap a stable one-step generator, providing a warm-up phase that brings the student distribution closer to the teacher before adversarial distillation begins. Extensive experiments on VBench demonstrate that AAD-1 achieves state-of-the-art performance in one-step autoregressive video generation.

Chinese Translation

我们提出了AAD-1，一种用于一步自回归图像到视频生成的非对称对抗蒸馏框架。现有的最先进方法采用对抗蒸馏，但面临运动崩溃和训练不稳定的问题，导致生成静态视频。AAD-1通过在架构和训练策略上的两个关键设计来解决这些挑战。我们的关键架构见解是打破生成器和判别器之间的对称性。生成器保持因果性以保留自回归采样能力，而判别器则在全时空上下文中双向关注，并为整个视频序列生成一个整体的真实感评分。这种非对称设计使得判别器能够有效检测导致自回归生成中运动崩溃的全局时间失败和长距离漂移。为了稳定训练，我们引入了一种分阶段策略，首先使用分布匹配来引导一个稳定的一步生成器，提供一个热身阶段，使学生分布在对抗蒸馏开始之前更接近教师分布。在VBench上的大量实验表明，AAD-1在一步自回归视频生成中达到了最先进的性能。

View on arXiv Download PDF AI Translation

cs.CV / 119 / 2606.03976

Formalizing the Binding Problem

形式化绑定问题

Huang, Lianghuan, Li, Yihao, Salehi, Saeed, Chang, Yingshan, Soni, Ansh, Kording, Konrad P.

Abstract

Representations of the world, arguably, contain information about features (e.g. something is blue, something is a circle) but also information about which features are part of the same object (e.g. the circle is blue), which we call binding information. Any system with the ability to understand scenes with multiple objects must be able to solve the binding problem: it needs to know which features belong together. However, despite work showing that Vision Transformers (ViTs) know which patches belong together, it is not known whether current deep learning models learn to exhibit binding information, i.e., for features. We may believe that there is not much binding information, after all misattributing features to wrong objects is a common failure of ViT-based architectures, especially in scenes with objects sharing features. Here we formalize the binding problem with an information-theoretic approach, and introduce a probing method to measure binding information in model representations. We perform experiments on ViTs, measuring binding from different components of the architecture, such as the image summary token [CLS] or the spatial tokens. We use datasets with different binding challenges, such as feature sharing, occlusion, and natural features, while comparing the performance of several pre-trained ViTs. Overall, our research demonstrates binding as a key ingredient to strong visual recognition and reasoning.

Chinese Translation

世界的表征可以说包含了关于特征的信息（例如，某物是蓝色的，某物是圆形的），同时也包含了关于哪些特征属于同一对象的信息（例如，圆形是蓝色的），我们称之为绑定信息。任何能够理解多个对象场景的系统都必须能够解决绑定问题：它需要知道哪些特征是相互关联的。然而，尽管已有研究表明视觉变换器（Vision Transformers, ViTs）能够识别哪些补丁是相互关联的，但目前尚不清楚现有的深度学习模型是否学习到了绑定信息，即特征的绑定。我们可能会认为绑定信息并不多，毕竟将特征错误归属到错误对象上是基于ViT架构的常见失败，尤其是在对象共享特征的场景中。在此，我们采用信息论的方法形式化绑定问题，并引入了一种探测方法来测量模型表征中的绑定信息。我们在ViTs上进行实验，从架构的不同组件（如图像摘要标记 [CLS] 或空间标记）中测量绑定。我们使用具有不同绑定挑战的数据集，如特征共享、遮挡和自然特征，同时比较几种预训练ViTs的性能。总体而言，我们的研究表明，绑定是强视觉识别和推理的重要组成部分。

View on arXiv Download PDF AI Translation

cs.CV / 120 / 2606.03986

NewtPhys: Do Foundation Models Understand Newtonian Physics?

NewtPhys：基础模型是否理解牛顿物理学？

Cavada, Sebastian, Paul, Soumava, Vu, Tuan-Hung, Bursuc, Andrei, de Charette, Raoul

Abstract

Previous work has evaluated physics reasoning in foundation models using synthetic or semi-synthetic scenes and visual question-answering tasks. However, these benchmarks emphasize high-level events and lack the visual fidelity required to assess true low-level Newtonian understanding. We introduce NewtPhys, a 4D physically annotated dataset built from multiview images of real-world scenes with physics-grounded simulations. The dataset provides dense, fine-grained annotations across timesteps -- including 3D forces and amodal per-pixel quantities covering physics, tracking, semantics and geometry -- bridging the gap between simplistic synthetic setups and realistic visual complexity. Using NewtPhys, we systematically evaluate 56 VLMs, including 54 open-weight models and 2 closed-source frontier models, and 10 VFMs and reveal limitations in low-level physics reasoning. Beyond benchmarking, our dataset enables future research in physics-grounded vision and the development of next-generation physics-aware evaluations. Code and datasets are available at https://astra-vision.github.io/NewtPhys.

Chinese Translation

之前的研究通过合成或半合成场景和视觉问答任务评估基础模型中的物理推理。然而，这些基准强调高层次事件，缺乏评估真实低层次牛顿理解所需的视觉逼真度。我们引入了NewtPhys，这是一个基于多视角真实场景图像和物理基础模拟构建的4D物理注释数据集。该数据集在时间步长上提供了密集、细粒度的注释——包括3D力和覆盖物理、跟踪、语义和几何的无模态每像素量——弥合了简单合成设置与真实视觉复杂性之间的差距。利用NewtPhys，我们系统地评估了56个视觉语言模型（VLMs），包括54个开放权重模型和2个闭源前沿模型，以及10个视觉基础模型（VFMs），并揭示了低层次物理推理的局限性。除了基准测试，我们的数据集还支持未来在物理基础视觉领域的研究以及下一代物理感知评估的开发。代码和数据集可在 https://astra-vision.github.io/NewtPhys 获取。

View on arXiv Download PDF AI Translation

cs.CV / 121 / 2606.03989

PixVOD: Pixel-Distributed Direct Visual Odometry and Depth Estimation

PixVOD：像素分布式直接视觉里程计和深度估计

Kim, Shinjeong, Alzugaray, Ignacio, Rhodes, Callum, Kelly, Paul H. J., Davison, Andrew J.

Abstract

Images composed of 2D pixel arrays are the standard input to computer vision algorithms, yet many underlying computations can be distributed across pixels. Transmitting raw, redundant, and noisy pixel data off the sensor remains inefficient, motivating a shift toward focal-plane sensor-processors that perform a significant part of the computation directly within each pixel. We envision pixels synthesizing higher-level signals locally, reducing downstream load, and providing richer inputs for higher-level vision tasks. We propose a fully parallelizable form of visual odometry and depth estimation across pixels, where sensor-processors exchange information through Gaussian Belief Propagation (GBP) to achieve consensus about camera motion and infer depth from per-pixel photometric observations and a surface normal prior. To maintain geometric stability during optimization, we introduce a keyframe-like anchoring mechanism that regulates the effective baseline between frames, enabling consistent motion and depth updates. Our method is evaluated on realistic datasets, demonstrating the feasibility of GBP-based pixel-level distributed odometry and depth estimation with keyframe anchoring on-sensor. Project Page: https://www.shinjeongkim.com/pixvod/

Chinese Translation

由二维像素阵列组成的图像是计算机视觉算法的标准输入，然而许多底层计算可以在像素之间分布进行。将原始、冗余和噪声的像素数据从传感器传输出去仍然效率低下，这促使我们转向焦平面传感器处理器，它们在每个像素内直接执行大部分计算。我们设想像素在本地合成更高层次的信号，从而减少下游负载，并为更高层次的视觉任务提供更丰富的输入。我们提出了一种完全可并行化的视觉里程计和深度估计方法，传感器处理器通过高斯信念传播（Gaussian Belief Propagation, GBP）交换信息，以达成对相机运动的共识，并根据每个像素的光度观测和表面法线先验推断深度。为了在优化过程中保持几何稳定性，我们引入了一种类似关键帧的锚定机制，调节帧之间的有效基线，从而实现一致的运动和深度更新。我们在真实数据集上评估了我们的方法，展示了基于GBP的像素级分布式里程计和深度估计的可行性，以及在传感器上进行关键帧锚定的效果。项目页面： https://www.shinjeongkim.com/pixvod/

View on arXiv Download PDF AI Translation

cs.CV / 122 / 2606.03992

Exploring Easy Boosts for Lidar Semantic Scene Completion

探索激光雷达语义场景补全的简单增强策略

Martyniuk, Tetiana, Seele, Jonathan, Boulch, Alexandre, Puy, Gilles, Marlet, Renaud, de Charette, Raoul

Abstract

This paper investigates "free lunch" strategies to boost the performance of lidar semantic scene completion (SSC) without requiring complex architectural redesigns. We first demonstrate that endowing input point clouds with semantic pseudo-labels from off-the-shelf segmentors significantly improves the performance of existing architectures. By evaluating these models against an oracle, we establish that high-quality semantic priors are a primary driver of mIoU gains. Furthermore, we equip the input lidar scan with visibility information that distinguishes between empty and unknown spaces, which provides a secondary performance boost across the tested architectures. Using these simple enhancements, we observe that older models remain competitive with state-of-the-art systems, and can even outperform them. Our code is available at https://github.com/astra-vision/SSC-Priors.

Chinese Translation

本文研究了“免费午餐”策略，以提升激光雷达语义场景补全（SSC）的性能，而无需复杂的架构重设计。我们首先展示了，利用现成分割器为输入点云赋予语义伪标签显著提高了现有架构的性能。通过将这些模型与一个理想模型进行评估，我们确定高质量的语义先验是提高平均交并比（mIoU）增益的主要驱动因素。此外，我们为输入的激光雷达扫描配备了可见性信息，以区分空白和未知空间，这为测试的架构提供了二次性能提升。通过这些简单的增强，我们观察到较旧的模型在与最先进的系统竞争时仍然具有竞争力，甚至能够超越它们。我们的代码可在 https://github.com/astra-vision/SSC-Priors 获取。

View on arXiv Download PDF AI Translation

cs.CV / 123 / 2606.03994

SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Image

SimuScene：从单幅图像重建可用于仿真的组合3D场景

Lee, Inhee, Baik, Sangwon, Kim, Sungjoo, Kim, Hyeonwoo, Cha, Hyunsoo, Joo, Hanbyul

Abstract

Reconstructing interactive, simulation-ready 3D scenes from a single image is a critical bottleneck for robotic manipulation. While recent single-image lifters recover plausible per-object shapes, composing them yields scenes that collapse under physical simulation due to interpenetrating, hovering, or sinking objects. Existing physics-aware methods address this strictly as a post-hoc layout correction, leaving the underlying geometric errors unresolved. To address this, we introduce SimuScene, a compositional 3D reconstruction pipeline that puts physics in the loop of shape and layout estimation. Rather than using physics merely for layout cleanup, we utilize the physics engine as a diagnostic measurement tool during the generative process itself. By diagnostically simulating reconstructed objects under gravity, we convert penetration and support failures into quantitative correction signals that drive gravity-axis stretching and amodal shape resampling. This physics-informed feedback loop mitigates accumulated reconstruction errors and produces a stable, simulation-ready compositional 3D scene. Extensive experiments demonstrate state-of-the-art performance on physical stability and geometric alignment benchmarks. We further highlight SimuScene's utility by deploying reconstructed environments in humanoid control and robot-arm manipulation tasks.

Chinese Translation

从单幅图像重建交互式、可用于仿真的3D场景是机器人操作中的一个关键瓶颈。尽管最近的单图像提升方法能够恢复出合理的每个物体形状，但将这些形状组合在一起会导致场景在物理仿真中崩溃，原因在于物体之间的相互穿透、悬浮或下沉。现有的物理感知方法将此问题严格视为事后布局修正，未能解决潜在的几何错误。为了解决这一问题，我们提出了SimuScene，一个将物理纳入形状和布局估计的组合3D重建管道。我们不仅仅将物理用于布局清理，而是将物理引擎作为生成过程中的诊断测量工具。通过在重建物体上进行重力下的诊断性仿真，我们将穿透和支撑失败转化为定量修正信号，从而驱动重力轴拉伸和模态形状重采样。这种物理信息反馈循环减轻了累积的重建误差，并生成了一个稳定的、可用于仿真的组合3D场景。大量实验表明，在物理稳定性和几何对齐基准测试中，SimuScene展现了最先进的性能。我们进一步通过在类人控制和机器人臂操作任务中部署重建环境，突显了SimuScene的实用性。

View on arXiv Download PDF AI Translation

人工智能 (Artificial Intelligence)

cs.AI / 1 / 2606.02673

Visual Graph Scaffolds for Structural Reasoning in Large Language Models

用于大型语言模型结构推理的视觉图形支架

Lei, Runlin, Xiao, Xiaokui, Wei, Zhewei

Abstract

Graphs have been used to enhance large language models (LLMs) for structured reasoning, mostly as external knowledge sources are provided to models at test time. In this paper, we take a different view: the value of graphs for LLMs lie not only in supplying information, but also in organizing reasoning. Inspired by how humans use graph-structured mind maps to organize branching and converging thoughts, we ask whether graphs can serve as an internal form of reasoning assistance. We study this question on multi-hop question answering tasks, where teacher-provided reasoning traces are rewritten as graph mind maps and used to guide a student model. Our experiments reveal a clear modality gap. When graph structures are flattened into text, their benefits become limited once direct answer hints are removed. Under this abstract guidance setting, both reasoning efficiency and answer quality degrade substantially. In contrast, visual graph guidance remains effective without direct answer clues, and its advantage persists after supervised fine-tuning and KL-based distillation. The above findings support the claim that graphs should be studied not only as external knowledge structures for LLMs, but also as visual scaffolds for organizing reasoning.

Chinese Translation

图形被用来增强大型语言模型（LLMs）在结构化推理方面的能力，主要是因为在测试时向模型提供了外部知识源。本文采取了不同的视角：图形对LLMs的价值不仅在于提供信息，还在于组织推理。受到人类如何使用图结构思维导图来组织分支和汇聚思维的启发，我们探讨图形是否可以作为一种内部推理辅助形式。我们在多跳问答任务上研究这个问题，其中教师提供的推理轨迹被重写为图形思维导图，并用于指导学生模型。我们的实验揭示了明显的模态差距。当图形结构被压平为文本时，一旦移除直接答案提示，其益处变得有限。在这种抽象指导设置下，推理效率和答案质量均显著下降。相反，视觉图形指导在没有直接答案线索的情况下仍然有效，并且其优势在经过监督微调和基于KL的蒸馏后依然存在。上述发现支持了这样的观点：图形不仅应作为LLMs的外部知识结构进行研究，还应作为组织推理的视觉支架。

View on arXiv Download PDF AI Translation

cs.AI / 2 / 2606.02775

AURA: Action-Gated Memory for Robot Policies at Constant VRAM

AURA：用于机器人策略的动作门控记忆在恒定 VRAM 下的应用

Chen, Josef

Abstract

The KV-cache is the right memory for datacenters but the wrong memory for robots. Datacenter inference batches many short requests and resets them, amortizing an attention cache across a crowd. Embodied agents instead run one long, non-resetting episode on bandwidth-limited edge hardware, where high-bandwidth memory and flash are scarce, flash has finite write endurance, and memory writes rather than compute can become the binding constraint. AURA-Mem (Action-Utility Recurrent Adaptive Memory) targets this regime. It wraps a frozen vision-language-action backbone with a constant-size recurrent memory and a learned gate that writes only when the current observation would change the next action: memory that knows when to stay silent. Unlike reconstruction-based memory, the gate is trained directly against a closed-loop action-error signal. Its inference state is fixed at 4,224 bytes regardless of horizon, while a KV-cache grows to 6,061 times larger at 100,000 steps. On a controlled synthetic benchmark, AURA-Mem matches the best O(1) baseline in accuracy while using 5.19-6.13 times fewer writes, and up to 9.19 times fewer writes on easier configurations. Budget-matched random and periodic schedules do not recover this gain, isolating the benefit to the action-surprise signal. On a trained closed-loop OpenVLA-OFT 7B panel on LIBERO-Long (n=60 episodes per arm), the gate does not hurt success: AURA-Mem matches the ungated base policy (0.233) and slightly exceeds an always-write KV arm (0.217), while using 7.0 times fewer writes and constant memory. We also instantiate an approximate-information-state value-loss bound as a methodology demonstration; at this scale, the bound is vacuous rather than a guarantee.

Chinese Translation

KV-cache 是数据中心的合适内存，但对于机器人而言却是不适合的内存。数据中心推理批量处理许多短请求并重置它们，从而在一群请求中摊销注意力缓存。而具身智能体则在带宽有限的边缘硬件上运行一个长时间、不重置的剧集，在这种情况下，高带宽内存和闪存稀缺，闪存具有有限的写入耐久性，并且内存写入而非计算可能成为瓶颈。AURA-Mem（动作效用递归自适应记忆）旨在解决这一问题。它将一个冻结的视觉-语言-动作骨干网络与一个恒定大小的递归记忆和一个学习的门控结合在一起，只有在当前观察会改变下一个动作时才进行写入：这种记忆知道何时保持沉默。与基于重建的记忆不同，该门控是直接针对闭环动作误差信号进行训练的。其推理状态在 4,224 字节固定，无论时间范围如何，而 KV-cache 在 100,000 步时增长至 6,061 倍。在一个受控的合成基准测试中，AURA-Mem 在准确性上与最佳 O(1) 基线相匹配，同时使用的写入次数减少了 5.19-6.13 倍，在更简单的配置中减少了多达 9.19 倍。预算匹配的随机和周期性调度无法恢复这一增益，将这一好处孤立于动作惊讶信号。在一个经过训练的闭环 OpenVLA-OFT 7B 面板上，针对 LIBERO-Long（每个臂 n=60 个剧集），该门控并未影响成功率：AURA-Mem 的表现与无门控基线策略（0.233）相匹配，并略微超过了始终写入的 KV 臂（0.217），同时使用的写入次数减少了 7.0 倍且内存保持恒定。我们还实例化了一个近似信息状态值损失界限作为方法论示范；在这个规模下，该界限是空洞的，而非保证。

View on arXiv Download PDF AI Translation

cs.AI / 3 / 2606.02791

Evaluating Transformer and LSTM Frameworks for Prediction in Ungauged Basins

评估变换器（Transformer）和长短期记忆网络（LSTM）框架在无测站流域预测中的表现

Akinrele, Taye, Halgren, James, Golilarz, Noorbakhsh Amiri, Mittal, Sudip, Rahimi, Shahram

Abstract

Watershed networks exhibit convergent topologies in which multiple tributaries merge into downstream channels,integrating diverse upstream hydrological processes. In ungauged basins, the absence of direct observations increases uncertainty and limits the ability to anticipate extreme events. This study evaluates whether an encoder-only Transformer provides an advantage over an LSTM for upstream streamflow inference under limited hydrologic information, using retrospective simulations from the NOAA National Water Model (NWM). Across both upstream-only and combined configurations, the LSTM showed stronger overall performance than the Transformer model across the two configurations. Incorporating downstream information further boosted performance for all models, increasing median NNSE by more than 60%. Rather than treating this as a leaderboard-style comparison, we interpret the experiments as a test of architectural inductive bias for hydrologic sequence inference. The results indicate that recurrent memory remains better aligned with this upstream reconstruction task than an encoder-only Transformer, while downstream hydrologic context provides a strong auxiliary constraint that substantially improves prediction skill across architectures

Chinese Translation

流域网络展现出汇聚的拓扑结构，其中多个支流汇入下游河道，整合了多样的上游水文过程。在无测站流域中，缺乏直接观测数据增加了不确定性，并限制了预测极端事件的能力。本研究评估了在有限水文信息下，编码器-only 的变换器（Transformer）是否相较于长短期记忆网络（LSTM）在上游流量推断中具有优势，使用来自美国国家海洋和大气管理局（NOAA）国家水模型（NWM）的回顾性模拟数据。在上游-only 和组合配置中，LSTM 在两种配置下的整体表现均优于变换器模型。纳入下游信息进一步提升了所有模型的表现，使中位数归一化均方根误差（NNSE）提高了超过60%。我们并不将此视为排行榜式的比较，而是将实验解读为水文序列推断的架构归纳偏差测试。结果表明，递归记忆在这一上游重建任务中与编码器-only 变换器相比更为契合，而下游水文背景提供了强有力的辅助约束，显著提升了各架构的预测能力。

View on arXiv Download PDF AI Translation

cs.AI / 4 / 2606.02798

BehaviorBench: Modeling Real-World User Decisions from Behavioral Traces

行为基准：从行为轨迹建模现实世界用户决策

Yang, Liangwei, Qiu, Jielin, Chen, Zixiang, Zhu, Ming, Tan, Juntao, Liu, Zhiwei, Zhao, Wenting, Lan, Zhujun, Prabhakar, Akshara, Savarese, Silvio, Wang, Huan, Heinecke, Shelby

Abstract

Many decision-support settings require systems that adapt to individual users, but evaluation data for this problem remain limited. Existing benchmarks for user understanding often rely on simulated users or model-generated behavior, even though recent work cautions that model-based simulations can diverge systematically from human behavior. We introduce \textsc{BehaviorBench}, a benchmark for evaluating personalized decision modeling from real-world behavioral traces. \textsc{BehaviorBench} reconstructs wallet-level decision histories from observed public prediction-market and on-chain records, and organizes them into two complementary task layers: \emph{Belief prediction}, which predicts a user's final revealed stance and confidence in a market, and \emph{Trade prediction}, which predicts the direction and amount of individual transactions. Across 2,000 evaluation wallets, the benchmark contains 141,445 Belief instances and 1,485,972 Trade instances, with disjoint support pools for retrieval-based evaluation. We evaluate frontier and open-weight generative models under four history interfaces: no personalization, direct recent history, generated user profiles, and retrieved support-wallet evidence. Personalization improves Belief prediction more consistently than Trade prediction, model rankings change across task layers and metrics, and different history interfaces expose different failure modes. \textsc{BehaviorBench} provides an evaluation setting for studying whether personalized methods can use real-world behavioral evidence rather than simulated users alone.

Chinese Translation

许多决策支持环境需要能够适应个体用户的系统，但针对这一问题的评估数据仍然有限。现有的用户理解基准往往依赖于模拟用户或模型生成的行为，尽管近期研究警告说基于模型的模拟可能与人类行为系统性地偏离。我们引入了 extsc{BehaviorBench}，这是一个用于评估基于真实世界行为轨迹的个性化决策建模的基准。 extsc{BehaviorBench} 从观察到的公共预测市场和链上记录中重建钱包级决策历史，并将其组织为两个互补的任务层次： extit{信念预测}，预测用户在市场中的最终揭示立场及其信心，以及 extit{交易预测}，预测个体交易的方向和金额。在2000个评估钱包中，该基准包含141,445个信念实例和1,485,972个交易实例，并为基于检索的评估提供了不重叠的支持池。我们在四种历史接口下评估前沿和开放权重生成模型：无个性化、直接最近历史、生成的用户档案和检索的支持钱包证据。个性化在信念预测中的改善比在交易预测中更为一致，模型排名在任务层次和指标之间发生变化，不同的历史接口暴露出不同的失败模式。 extsc{BehaviorBench} 提供了一个评估环境，用于研究个性化方法是否可以利用真实世界的行为证据，而不仅仅是模拟用户。

View on arXiv Download PDF AI Translation

cs.AI / 5 / 2606.02802

ChatHealthAI: Aligning Electronic Health Record Representations with Large Language Models for Grounded Clinical Reasoning

ChatHealthAI：将电子健康记录表示与大型语言模型对齐以实现基于证据的临床推理

Wang, Bo-Hong, Peng, Baicheng, Wang, Ruilin, Bai, Jun, Song, Ziyang, Li, Yue

Abstract

Large language models (LLMs) exhibit strong natural-language reasoning abilities for clinical decision support, but struggle to effectively model structured longitudinal electronic health records (EHRs). In contrast, EHR foundation models can learn predictive patient representations, yet lack interpretable language-based reasoning. To bridge this gap, we propose ChatHealthAI, a multimodal reasoning framework that aligns structured EHR representations from a pretrained EHR foundation model with the semantic space of a frozen LLM through a task-aware resampler. By integrating longitudinal patient representations with refined clinical event descriptions, ChatHealthAI enables clinically grounded natural-language reasoning while maintaining accurate patient prediction. We evaluated ChatHealthAI on three clinical predictive tasks from the EHRSHOT benchmark. Results show that ChatHealthAI improves reasoning quality and interpretability while preserving competitive predictive performance. These findings highlight the potential of integrating EHR foundation models with pretrained LLMs for interpretable clinical prediction.

Chinese Translation

大型语言模型（LLMs）在临床决策支持方面表现出强大的自然语言推理能力，但在有效建模结构化的纵向电子健康记录（EHRs）方面存在困难。相比之下，EHR基础模型能够学习预测患者表示，但缺乏可解释的基于语言的推理。为了解决这一问题，我们提出了ChatHealthAI，这是一种多模态推理框架，通过任务感知重采样器将预训练EHR基础模型的结构化EHR表示与冻结的LLM的语义空间对齐。通过将纵向患者表示与精炼的临床事件描述相结合，ChatHealthAI实现了临床基础的自然语言推理，同时保持准确的患者预测。我们在EHRSHOT基准的三个临床预测任务上评估了ChatHealthAI。结果表明，ChatHealthAI提高了推理质量和可解释性，同时保持了竞争性的预测性能。这些发现突显了将EHR基础模型与预训练LLMs结合以实现可解释临床预测的潜力。

View on arXiv Download PDF AI Translation

cs.AI / 6 / 2606.02812

Traj-Evolve: A Self-Evolving Multi-Agent System for Patient Trajectory Modeling in Lung Cancer Early Detection

Traj-Evolve：一种自我演化的多智能体系统用于肺癌早期检测中的患者轨迹建模

Zeng, Sihang, Thompson, Matthew, Etzioni, Ruth, Yetisgen, Meliha

Abstract

Modeling patient trajectories from longitudinal electronic health records (EHRs) requires reasoning over sparse, noisy, and long-context multimodal sequences. Existing LLM-based multi-agent systems address context length but process patients in isolation, failing to mirror how clinicians leverage accumulated experience from similar prior cases. We present Traj-Evolve, a self-evolving multi-agent system with two complementary evolving mechanisms. First, an Experience Pool (ExPool) acts as a non-parametric memory, indexing rejection-sampled reasoning traces to retrieve similar patients as few-shot contexts. Second, multi-agent reinforcement learning (MARL) via reward-ranked fine-tuning parametrically optimizes inter-agent and agent-memory collaboration. A leave-one-out cross-retrieval strategy unifies the two, aligning training- and inference-time behavior under retrieval augmentation. On a lung cancer prediction task utilizing up to five years of multimodal EHRs, Traj-Evolve outperforms 9 strong baselines on the overall population and a challenging never-smoker population. Analysis of the evolving dynamics highlights three key findings: (1) expanding the ExPool shifts optimal retrieval from diverse to specific samples; (2) under MARL, the manager agent's prediction loss converges quickly while the worker agents' temporal reasoning continues to benefit from more verified patients; and (3) the two mechanisms are complementary on the predicted risk, where ExPool improves specificity while MARL improves sensitivity.

Chinese Translation

从纵向电子健康记录（EHRs）中建模患者轨迹需要对稀疏、噪声和长上下文的多模态序列进行推理。现有的基于大语言模型（LLM）的多智能体系统虽然解决了上下文长度的问题，但在处理患者时是孤立的，未能反映临床医生如何利用类似先前案例的累积经验。我们提出了Traj-Evolve，这是一种具有两种互补演化机制的自我演化多智能体系统。首先，经验池（Experience Pool, ExPool）作为一种非参数记忆，索引拒绝采样的推理轨迹，以检索相似患者作为少量样本上下文。其次，通过奖励排名微调的多智能体强化学习（Multi-Agent Reinforcement Learning, MARL）在参数上优化智能体间及智能体与记忆的协作。一种留一交叉检索策略将两者统一，确保在检索增强下训练和推理时的行为一致。在利用多达五年的多模态EHRs进行的肺癌预测任务中，Traj-Evolve在整体人群和一个具有挑战性的非吸烟者人群中超越了9个强基线。对演化动态的分析突出了三个关键发现：（1）扩展ExPool使得最佳检索从多样化样本转向特定样本；（2）在MARL下，管理者智能体的预测损失快速收敛，而工作智能体的时间推理则继续受益于更多经过验证的患者；（3）这两种机制在预测风险上是互补的，其中ExPool提高了特异性，而MARL提高了敏感性。

View on arXiv Download PDF AI Translation

cs.AI / 7 / 2606.02832

An Exploration of Collision-based Enemy Morphology Generation

基于碰撞的敌人形态生成探索

Gonzalez, Johor Jara, Guzdial, Matthew

Abstract

Despite a great deal of prior research into Procedural Content Generation (PCG), relatively little prior work has explored generating enemies for video games. In particular, there is almost no work on generating enemy morphologies, the basic body plan or collision information for in-game enemies, despite the existence of related morphology generation work in robotics. In this paper, we explore three different novel approaches to generate enemy morphologies based on player collision information. We found that each approach provides different strengths and weaknesses, but all had equivalent or better performance than an evolutionary baseline adapted from prior robotics morphology work.

Chinese Translation

尽管在程序化内容生成（Procedural Content Generation, PCG）方面已有大量研究，但针对视频游戏中敌人生成的相关工作相对较少。特别是，几乎没有关于生成敌人形态的研究，即游戏中敌人的基本身体结构或碰撞信息，尽管在机器人技术中存在相关的形态生成研究。本文探讨了三种基于玩家碰撞信息生成敌人形态的新方法。我们发现每种方法都有不同的优缺点，但所有方法的性能均等于或优于从先前机器人形态研究中改编的进化基线。

View on arXiv Download PDF AI Translation

cs.AI / 8 / 2606.02835

Thinking Past the Answer: Evaluating Harmful Overthinking in Large Reasoning Models

超越答案的思考：评估大型推理模型中的有害过度思考

Caldarella, Simone, Talon, Davide, Aljundi, Rahaf, Ricci, Elisa, Mancini, Massimiliano

Abstract

Large Reasoning Models (LRMs) improve performance by generating explicit intermediate reasoning traces through increased test-time compute, yet the assumption that longer reasoning is consistently beneficial remains under-examined. While recent evidence shows that additional reasoning can lead models to overthink, we ask: "Once a model has reached the correct answer, does further reasoning refine the solution, or deviate from it?" To study the dynamics after correctness, we introduce a prefix-level trajectory evaluation protocol grounded in reasoning sufficiency, defining the minimum reasoning budget required for a model to first generate the correct answer. This allows us to disentangle verbose overthinking, where additional reasoning is redundant but harmless, from harmful overthinking, where continued reasoning destabilizes an already-correct trajectory. Starting from multimodal benchmarks, we find that many instances considered reasoning-intensive require surprisingly little reasoning. Moreover, stopping at the first correct prefix improves accuracy over standard reasoning up to 21%, revealing that current models are limited not only by their ability to reason, but also by their inability to stop at the right time. Furthermore, while common efficiency strategies like early stopping substantially reduce verbose overthinking (up to 50%), they fail to mitigate harmful overthinking. Failure analysis reveals that correctness deviations are mainly driven by logical drift and visual reinterpretation. Finally, we show that our findings generalize to language-only reasoning benchmarks, highlighting harmful overthinking as a broader reliability risk. Code available at https://simonecaldarella.github.io/thinking-past-the-answer.

Chinese Translation

大型推理模型（LRMs）通过增加测试时计算量生成明确的中间推理轨迹，从而提高性能，但更长的推理是否始终有利的假设仍未得到充分检验。虽然最近的证据表明，额外的推理可能导致模型过度思考，我们提出了一个问题：“一旦模型达到了正确答案，进一步的推理是细化解决方案，还是偏离它？”为了研究正确性之后的动态，我们引入了一种基于推理充分性的前缀级轨迹评估协议，定义了模型首次生成正确答案所需的最小推理预算。这使我们能够区分冗长的过度思考（额外的推理是多余但无害的）与有害的过度思考（持续的推理使已经正确的轨迹不稳定）。从多模态基准开始，我们发现许多被认为是推理密集型的实例实际上需要的推理量出乎意料地少。此外，在第一个正确前缀处停止推理比标准推理提高了多达21%的准确性，揭示了当前模型的局限性不仅在于其推理能力，还在于其未能在正确时机停止。此外，虽然像早停这样的常见效率策略显著减少了冗长的过度思考（高达50%），但它们未能缓解有害的过度思考。失败分析表明，正确性偏差主要由逻辑漂移和视觉重解释驱动。最后，我们展示了我们的发现可以推广到仅语言推理的基准，强调有害的过度思考作为更广泛的可靠性风险。代码可在 https://simonecaldarella.github.io/thinking-past-the-answer 获取。

View on arXiv Download PDF AI Translation

cs.AI / 9 / 2606.02862

Toward a Modular Architecture for Embedded AI Agent Systems at the Edge

面向边缘嵌入式人工智能代理系统的模块化架构

Rüb, Marcus, Gerhards, Michael

Abstract

The rise of Large Language Models (LLMs) has enabled agentic AI capable of complex reasoning and tool use; however, deploying such autonomy in pervasive computing environments remains challenging due to the strict memory and energy constraints of embedded microcontrollers. Existing frameworks typically assume server-class resources or continuous connectivity, leaving a gap for deeply embedded systems. This paper proposes a modular reference architecture for Embedded Agent Systems that bridges the divide between deterministic real-time control and agentic intelligence. We introduce a tiered design that decouples On-Device Agents - executing highly compressed neural networks and rule-based logic for low-latency, privacy-critical tasks - from Cloud-Augmented Agents that leverage Small Language Models (SLMs) for higher-level reasoning and planning. A key contribution is the integration of a cross-cutting Governance Layer, ensuring observability, policy enforcement, and safety across distributed fleets of autonomous devices. Rather than presenting purely empirical benchmarks, we analyze architectural design principles and trade-offs regarding latency, energy, and reliable execution in resource-constrained environments.

Chinese Translation

大型语言模型（LLMs）的兴起使得能够进行复杂推理和工具使用的代理人工智能成为可能；然而，由于嵌入式微控制器的严格内存和能量限制，在普遍计算环境中部署这种自主性仍然面临挑战。现有框架通常假设服务器级资源或持续连接，这使得深度嵌入式系统存在空白。本文提出了一种嵌入式代理系统的模块化参考架构，弥合了确定性实时控制与代理智能之间的鸿沟。我们引入了一种分层设计，将在设备上执行高度压缩的神经网络和基于规则的逻辑以满足低延迟和隐私关键任务的设备代理，与利用小型语言模型（SLMs）进行更高层次推理和规划的云增强代理解耦。一个关键贡献是集成了跨切面的治理层，确保在分布式自主设备群体中的可观察性、政策执行和安全性。我们不仅呈现纯粹的经验基准，还分析了在资源受限环境中关于延迟、能量和可靠执行的架构设计原则和权衡。

View on arXiv Download PDF AI Translation

cs.AI / 10 / 2606.02863

Don't Gamble, GAMBLe: An Analytical Framework for AI-Driven Research Systems

不要赌博，GAMBLe：一个基于AI驱动的研究系统的分析框架

Ellis, Marquita, Castro, Paul

Abstract

AI-Driven Research Systems (ADRS) -- systems coupling LLMs with automated evaluation to discover algorithms, proofs, and designs -- are being optimized and adopted across domains, but the tools to analyze them have not kept pace. ADRS performance depends on component interactions that are poorly understood, expensive to explore, and (as we show) not well captured by standard convergence guarantees. These guarantees rely on structural assumptions that do not hold under the ADRS process we formalize. We introduce GAMBLe, a framework that decomposes ADRS behavior into four parameters (generator $G$, assessor $\mathcal{A}$, discovery mechanism $\mathcal{M}$, budget $B$) and one compositional object, the effective landscape $L_{\text{eff}} = \mathcal{A} \circ G$, which reveals that distinct generator-assessor pairs induce structurally different per-problem optimization landscapes. We exercise the framework on 760+ replicated runs (>46,000 iterations) spanning generators from single LLMs to dynamically-adaptive ensembles, mechanisms from greedy selection to co-evolutionary meta-search, and three NP-hard problems whose assessors range from continuous scoring to cliff functions. The experiments reveal no total ordering of generators or mechanisms: frontier models can underperform open-source alternatives and the simplest mechanism sometimes outperforms state-of-the-art meta-search. Results show that even under limited budgets (60 iterations per run), the right component choices can improve performance by 13-67% and search efficiency by 6-39x.

Chinese Translation

AI驱动的研究系统（ADRS）——将大型语言模型（LLMs）与自动评估相结合以发现算法、证明和设计的系统——正在各个领域进行优化和应用，但对其进行分析的工具却未能跟上。ADRS的性能依赖于组件之间的相互作用，而这些相互作用尚不清楚，探索成本高昂，并且（如我们所示）并未被标准的收敛保证很好地捕捉。这些保证依赖于在我们形式化的ADRS过程中并不成立的结构假设。我们引入了GAMBLe，一个将ADRS行为分解为四个参数（生成器 $G$、评估者 $ ext{A}$、发现机制 $ ext{M}$、预算 $B$）和一个组合对象的框架，即有效景观 $L_{ ext{eff}} = ext{A} ullet G$，该对象揭示了不同的生成器-评估者对会导致结构上不同的每个问题优化景观。我们在760多个重复运行（超过46,000次迭代）中应用该框架，涵盖了从单一LLM到动态自适应集成的生成器，从贪婪选择到共同进化元搜索的机制，以及三种NP难题，其评估者范围从连续评分到悬崖函数。实验结果表明，生成器或机制之间没有总排序：前沿模型的表现可能低于开源替代方案，而最简单的机制有时会超越最先进的元搜索。结果显示，即使在有限预算（每次运行60次迭代）下，正确的组件选择也能将性能提高13-67%，并将搜索效率提高6-39倍。

View on arXiv Download PDF AI Translation

cs.AI / 11 / 2606.02866

When Helping Hurts and How to Fix It: Multi-Agent Debate for Data Cleaning

帮助与伤害：多智能体辩论在数据清洗中的应用

Parmar, Chirag, Mehta, Akshat, Wu, Henglin, Ramamurthy, Jagadish, Medhekar, Shweta

Abstract

When does multi-agent debate help data cleaning, and when does it hurt? Across three benchmarks, four model families, and over 6,000 task-condition pairs, we find debate's effect reverses sign: it degrades generation across all four models (-1.6 to -15.5pp) through critique-induced confusion (CIC), hallucinated Critic feedback that the Generator accepts uncritically, yet improves error detection (+27.4pp F1, d=1.0). We derive a debate benefit condition: debate helps when the probability of rescuing a wrong output (Critic verification odds weighted by fixability) exceeds the probability of destroying a correct one. A factorial experiment proves adversarial separation is essential: self-verification with identical tools fails, while a separate Critic with code-execution grounding and evidence-gated generation produces the first debate configuration to significantly exceed single-agent on a generative task (+5.3pp, p<0.05). The condition correctly predicts all nine task types and generalizes with zero false positives across 19 published comparisons in seven domains.

Chinese Translation

多智能体辩论在何种情况下有助于数据清洗，又在何种情况下会造成伤害？在三个基准测试、四个模型系列和超过6000个任务-条件对的研究中，我们发现辩论的效果会反转：它在所有四个模型中导致生成质量下降（-1.6到-15.5个百分点），这是由于批评引发的混淆（CIC）和生成器无条件接受的虚构批评反馈，然而在错误检测方面却有所改善（+27.4个百分点 F1，d=1.0）。我们推导出辩论的效益条件：当拯救错误输出的概率（批评验证的赔率加权修复性）超过摧毁正确输出的概率时，辩论是有帮助的。一项因子实验证明对抗性分离是至关重要的：使用相同工具的自我验证失败，而一个具有代码执行基础和证据门控生成的独立批评者则产生了第一个在生成任务上显著超过单智能体的辩论配置（+5.3个百分点，p<0.05）。该条件正确预测了所有九种任务类型，并在七个领域的19个已发表比较中以零假阳性进行推广。

View on arXiv Download PDF AI Translation

cs.AI / 12 / 2606.02875

Handoff Debt: The Rediscovery Cost When Coding Agents Take Over Interrupted Tasks

交接债务：当编码代理接管中断任务时的再发现成本

KC, Dipesh, Budathoki, Anjila

Abstract

Coding-agent benchmarks evaluate whether a single uninterrupted agent can resolve a repository issue. Real software work is messier: tasks are interrupted, reassigned, reviewed, and resumed from partial states left by another agent or engineer. We study this missing dimension through \emph{handoff debt}: the rediscovery cost imposed when a predecessor's work is opaque or incomplete. Our takeover protocol interrupts a coding agent at deterministic handoff points, freezes the repository, and evaluates successor agents under four handoff views: repository state only, raw trace, summary notes, and structured notes. Across 75 source tasks, the protocol generates 181 handoff-point tasks and 724 takeover runs per successor model. Across three successor models, context-bearing handoffs reduce median agent events by 20--59\% and cumulative prompt tokens by 42--63\% relative to repository-only takeover. Solved-rate effects are smaller and model-dependent, but efficiency gains are consistent. These findings suggest that coding-agent evaluation should report not only whether a task is solved, but also how costly that work is for another agent to resume.

Chinese Translation

编码代理基准评估单个不间断代理是否能够解决一个代码库问题。实际的软件工作更加复杂：任务会被中断、重新分配、审查，并从另一个代理或工程师留下的部分状态中恢复。我们通过 extit{交接债务}这一缺失维度进行研究：当前任的工作不透明或不完整时所施加的再发现成本。我们的接管协议在确定的交接点中中断编码代理，冻结代码库，并在四种交接视图下评估后续代理：仅代码库状态、原始跟踪、摘要笔记和结构化笔记。在75个源任务中，该协议生成了181个交接点任务和每个后续模型724次接管运行。在三个后续模型中，具有上下文的交接相较于仅基于代码库的接管，减少了中位数代理事件20%至59%以及累积提示令牌42%至63%。解决率的影响较小且依赖于模型，但效率提升是一致的。这些发现表明，编码代理的评估不仅应报告任务是否解决，还应说明其他代理恢复该工作所需的成本。

View on arXiv Download PDF AI Translation

cs.AI / 13 / 2606.02914

Large AI Models in Dental Healthcare: From General-Purpose Systems to Domain-Specific Foundation Models

大型人工智能模型在口腔医疗中的应用：从通用系统到领域特定基础模型

Helali, Sema, Nadab, Lina Abu, Alqawas, Sausan, Abd-Alrazaq, Alaa, Tamimi, Faleh, Damseh, Rafat

Abstract

Background: Oral diseases affect nearly 3.5 billion people worldwide, yet the comparative clinical potential of large-scale AI models in dentistry remains poorly understood. Three distinct model categories have emerged: language-generative models, discriminative vision foundation models, and dental-specific foundation models, with no unified review examining their relationships and collective limitations. Methods: Following PRISMA-ScR guidelines, we systematically searched four databases (PubMed, Google Scholar, Scopus, arXiv), screened independently by two reviewers. After applying inclusion/exclusion criteria, 97 studies (2020-2026) were included. We propose a two-dimensional classification framework organizing models by architectural paradigm and dental specialization degree. Results: Language-generative models excel at text-based tasks (clinical reasoning, licensing exams, patient communication) but show inconsistent performance on image-dependent diagnostics. Adapted SAM and CLIP variants achieve strong tooth segmentation and lesion detection results. Dental-specific models (DentVFM, DentVLM, OralGPT) demonstrate strongest performance on complex multimodal tasks. Integrated pipelines consistently outperform single-model approaches. A data asymmetry is observed: dental-specific pretraining concentrates almost entirely in the vision domain, reflecting scarce large-scale dental text corpora. Conclusions: General-purpose and dental-specific models play complementary roles; the most effective systems combine both within structured pipelines. Safe autonomous deployment requires resolving three persistent barriers: hallucination in generative models, limited annotated dental datasets, and absent standardized clinical evaluation benchmarks.

Chinese Translation

背景：口腔疾病影响全球近35亿人，但大型人工智能模型在牙科领域的比较临床潜力仍然不甚明了。出现了三种不同的模型类别：语言生成模型、判别视觉基础模型和牙科特定基础模型，但尚无统一的评审来考察它们之间的关系及共同的局限性。方法：根据PRISMA-ScR指南，我们系统地搜索了四个数据库（PubMed、Google Scholar、Scopus、arXiv），由两位评审员独立筛选。经过纳入/排除标准的应用，共纳入97项研究（2020-2026）。我们提出了一个二维分类框架，根据架构范式和牙科专业化程度对模型进行组织。结果：语言生成模型在基于文本的任务（临床推理、执业考试、患者沟通）中表现出色，但在依赖图像的诊断中表现不一致。经过调整的SAM和CLIP变体在牙齿分割和病变检测方面取得了良好的结果。牙科特定模型（DentVFM、DentVLM、OralGPT）在复杂的多模态任务中表现最强。集成管道的表现始终优于单模型方法。观察到数据不对称：牙科特定的预训练几乎完全集中在视觉领域，反映出大型牙科文本语料库的稀缺。结论：通用模型和牙科特定模型发挥互补作用；最有效的系统是在结构化管道中结合两者。安全的自主部署需要解决三个持续存在的障碍：生成模型中的幻觉、有限的标注牙科数据集和缺乏标准化的临床评估基准。

View on arXiv Download PDF AI Translation

cs.AI / 14 / 2606.02965

What Benchmarks Don't Measure: The Case for Evaluating Abstention Competence in Autonomous Agents

基准测试未测量的内容：评估自主代理的弃权能力的案例

Ojewale, Victor, Venkatasubramanian, Suresh

Abstract

Benchmarks for autonomous agents measure whether agents complete tasks, yet this framing is systematically blind to whether an agent should have proceeded at all. Agents trained under human-feedback objectives develop a structural tendency to proceed even when they lack the inputs, evidence, or authorization to act safely, a disposition we term compliance bias, because both the reward signal and the benchmark scoring regime treat proceeding as the correct default regardless of whether the preconditions for safe action are present. We make three contributions. We first show that compliance bias originates in reward hacking within human-feedback pipelines and is entrenched by prominent agent benchmarks, which either penalize agents for pausing or are architecturally unable to distinguish a principled pause from a silent failure. We then introduce a three-gap taxonomy of abstention-warranted scenarios, covering specification gaps where required information is absent, verification gaps where world state cannot be confirmed, and authority gaps where explicit authorization has not been given, which together provide a principled basis for constructing abstention-aware agent benchmarks. Finally, we propose abstention evaluation protocols (Safety Rate, Usability Rate, and Informed Refusal Rate) and report preliminary results across 144 enterprise agent scenarios and five model families, in which a runtime-enforced abstention mechanism achieves up to 89.2% hazardous-action blocking and 87.5% usability on authorized scenarios, demonstrating that the safety--usability tradeoff is tunable rather than inherent and that its shape varies substantially across model families. We treat this as preliminary work and offer the taxonomy and composite metrics as a starting point for further conversations.

Chinese Translation

自主代理的基准测试衡量代理是否完成任务，但这种框架系统性地忽视了代理是否应该继续行动。根据人类反馈目标训练的代理发展出一种结构性倾向，即使在缺乏安全行动所需的输入、证据或授权时也会继续进行，这种倾向我们称之为合规偏差，因为奖励信号和基准评分机制都将继续行动视为正确的默认选择，而不考虑安全行动的前提条件是否存在。我们提出了三项贡献。首先，我们展示了合规偏差源于人类反馈管道中的奖励黑客行为，并且受到显著代理基准的巩固，这些基准要么惩罚代理暂停，要么在架构上无法区分原则性暂停与无声失败。然后，我们引入了一个三重差距分类法，涵盖了弃权所需场景中的规范差距（缺失必要信息）、验证差距（无法确认世界状态）和授权差距（未明确给予授权），这些差距共同为构建关注弃权的代理基准提供了原则基础。最后，我们提出了弃权评估协议（安全率、可用率和知情拒绝率），并报告了在144个企业代理场景和五个模型系列中的初步结果，其中运行时强制的弃权机制实现了高达89.2%的危险行动阻止率和87.5%的授权场景可用率，证明安全性与可用性之间的权衡是可调的而非固有的，并且其形状在不同模型系列中有显著变化。我们将此视为初步工作，并提供分类法和综合指标作为进一步讨论的起点。

View on arXiv Download PDF AI Translation

cs.AI / 15 / 2606.02974

WISE-HAR: A Generalizable Ensemble Deep Learning Framework for WiFi-Based Human Activity Recognition

WISE-HAR：一种可泛化的基于WiFi的人体活动识别集成深度学习框架

Arshad, Maheen, Zahra, Qindeel E, Shahzad, Muhammad Khuram

Abstract

Human Activity Recognition (HAR) using WiFi signals has emerged as a transformative technology for smart homes, healthcare monitoring, security systems, and ambient assisted living. Unlike traditional camera-based systems that raise significant privacy concerns and fail in low-light conditions, or wearable sensors that require user compliance, WiFi-based HAR is non-intrusive, privacy-preserving, cost-effective, and works seamlessly in any lighting condition. This paper presents a comprehensive approach to recognize three distinct human activities: "No Presence" (empty room), "Walking", and "Walking + Arm-waving" using the Wallhack1.8k WiFi spectrogram dataset. We propose three key improvements to address the main challenges in WiFi-based HAR. First, to address high performance variance, we implement ensemble learning with five different CNN architectures (Deep CNN, Wide CNN, MobileNetV2, ResNet50V2, and EfficientNetB0). Second, to address the small dataset size limitation, we apply aggressive data augmentation techniques including time-warping, frequency masking, and noise addition. Third, to evaluate real-world generalization capability, we perform cross-scenario evaluation (training on Line-of-Sight and testing on Non-Line-of-Sight) and cross-antenna evaluation (training on Biquad antenna and testing on PIFA antenna). Our ensemble model achieved a test accuracy of 94.87% on the LOS scenario with Biquad antenna, outperforming the best individual model by 0.66%. Data augmentation improved Random Forest performance from 60% to 95%. Cross-scenario evaluation showed minimal accuracy drops of only 1.37% and 2.07%, demonstrating strong generalization capabilities. The results indicate that the proposed approach is robust, reliable, and suitable for real-world deployment in diverse environments with different hardware configurations.

Chinese Translation

基于WiFi信号的人体活动识别（HAR）已成为智能家居、健康监测、安全系统和环境辅助生活的变革性技术。与传统的基于摄像头的系统（存在显著的隐私问题且在低光条件下失效）或需要用户配合的可穿戴传感器不同，基于WiFi的HAR是非侵入式的、保护隐私的、具有成本效益的，并且在任何光照条件下均能无缝工作。本文提出了一种全面的方法，利用Wallhack1.8k WiFi频谱图数据集识别三种不同的人体活动：“无存在”（空房间）、“行走”和“行走+挥手”。我们提出了三项关键改进，以应对基于WiFi的HAR中的主要挑战。首先，为了解决高性能方差问题，我们实现了五种不同CNN架构（Deep CNN、Wide CNN、MobileNetV2、ResNet50V2和EfficientNetB0）的集成学习。其次，为了解决小数据集规模的限制，我们应用了激进的数据增强技术，包括时间扭曲、频率掩蔽和噪声添加。第三，为了评估真实世界的泛化能力，我们进行了跨场景评估（在视距下训练并在非视距下测试）和跨天线评估（在Biquad天线下训练并在PIFA天线下测试）。我们的集成模型在Biquad天线的LOS场景下达到了94.87%的测试准确率，超越了最佳单一模型0.66%。数据增强将随机森林的性能从60%提高到95%。跨场景评估显示准确率仅下降1.37%和2.07%，展现出强大的泛化能力。结果表明，所提出的方法在不同硬件配置的多样化环境中具有稳健性、可靠性，适合于实际部署。

View on arXiv Download PDF AI Translation

cs.AI / 16 / 2606.02994

Inducing Reasoning Primitives from Agent Traces

从智能体轨迹中诱导推理原语

Lei, Zhihan, Yan, Jiarui, Momo, Joshua, Cohen, William W.

Abstract

ReAct-style LLM agents often rediscover the same reasoning routines across problems, yet leave those routines trapped in transient scratchpads. We introduce Reasoning Primitive Induction, a single-pass method that mines successful ReAct traces, clusters recurrent reasoning moves, and converts the most frequent moves into a compact library of typed pseudo-tools. Each pseudo-tool is specified by a natural-language docstring interpreted by an LLM at invocation time, and a standard ReAct loop composes these primitives at test time. The central result is that induced libraries outperform the very agent that generated their traces: by +44pp on RuleArena NBA (30 -> 74), +30pp on MuSR team allocation (38 -> 68), and +22pp on NatPlan meeting planning (7 -> 29). Across five comparable subtasks spanning narrative deduction, rule application, and constraint-satisfaction planning, a single fixed configuration improves over zero-shot Chain-of-Thought on every subtask, matches or surpasses expert-authored decompositions, and outperforms AWM at lower average inference cost.

Chinese Translation

ReAct风格的大型语言模型（LLM）智能体在不同问题中常常重新发现相同的推理流程，但这些流程却被困在短暂的草稿中。我们提出了推理原语诱导（Reasoning Primitive Induction），这是一种单次处理的方法，能够挖掘成功的ReAct轨迹，聚类重复的推理动作，并将最频繁的动作转换为一个紧凑的类型伪工具库。每个伪工具由自然语言文档字符串指定，在调用时由LLM进行解释，标准的ReAct循环在测试时组合这些原语。核心结果是，诱导的库在性能上超越了生成其轨迹的智能体：在RuleArena NBA上提高了44个百分点（30 -> 74），在MuSR团队分配上提高了30个百分点（38 -> 68），在NatPlan会议规划上提高了22个百分点（7 -> 29）。在涵盖叙述推理、规则应用和约束满足规划的五个可比子任务中，单一固定配置在每个子任务上均优于零-shot Chain-of-Thought，匹配或超越专家撰写的分解，并在较低的平均推理成本下超越AWM。

View on arXiv Download PDF AI Translation

cs.AI / 17 / 2606.03031

AUDITFLOW: Executable Symbolic Environments for Structured Financial Reporting Verification

AUDITFLOW：用于结构化财务报告验证的可执行符号环境

Wang, Yan, Ai, Xuguang, Patel, Jaisal, Peng, Xueqing, Mo, Fengran, Cao, Yupeng, Li, Haohang, Cao, Mingyu, Qian, Lingfei, Gutiérrez-Basulto, Víctor

Abstract

Structured financial audit verification is difficult for language-model agents because correctness depends on structured evidence rather than text alone. A model must link reported facts to taxonomy concepts, traverse calculation or dimensional relations, and recompute expected values before applying an audit rule. We propose AuditFlow, a graph-grounded multi-agent framework that separates adaptive search from deterministic verification. AuditFlow builds a symbolic environment from a static US-GAAP taxonomy graph and a dynamic XBRL filing graph, and exposes it through typed tools for fact retrieval, taxonomy traversal, numerical checking, and rule evaluation. Two junior auditors inspect each case from regulatory and evidentiary views, while a senior auditor resolves disagreements and can request further investigation. The final reports are fused through evidential aggregation to produce an audit verdict, expected value, evidence trail, and trustworthiness score. On a FinAuditing-derived FinMR sample, AuditFlow reaches 82.09% joint audit accuracy under GPT-5.5, outperforming the strongest baseline by 14.93 points. Removing deterministic checks drops accuracy to 17.91%, showing that the symbolic environment performs the verification step that the model cannot reliably replace.

Chinese Translation

结构化财务审计验证对于语言模型代理而言是困难的，因为正确性依赖于结构化证据而不仅仅是文本。模型必须将报告的事实与分类法概念关联起来，遍历计算或维度关系，并在应用审计规则之前重新计算预期值。我们提出了AuditFlow，一个基于图的多代理框架，将自适应搜索与确定性验证分开。AuditFlow从静态的美国通用会计准则（US-GAAP）分类法图和动态的XBRL申报图构建符号环境，并通过类型化工具暴露该环境以进行事实检索、分类法遍历、数值检查和规则评估。两名初级审计员从监管和证据的角度检查每个案例，而一名高级审计员解决分歧并可以请求进一步调查。最终报告通过证据聚合融合，以产生审计裁决、预期值、证据链和可信度评分。在一个基于FinAuditing的FinMR样本上，AuditFlow在GPT-5.5下达到了82.09%的联合审计准确率，超越了最强基线14.93个百分点。移除确定性检查后，准确率降至17.91%，这表明符号环境执行了模型无法可靠替代的验证步骤。

View on arXiv Download PDF AI Translation

cs.AI / 18 / 2606.03036

TriEval: A Resource-Efficient Pipeline for LLM Bias, Toxicity, and Truthfulness Assessment

TriEval：一种资源高效的LLM偏见、毒性和真实性评估管道

Srikantha, Akshatha, Singh, Manpreet, Jajoo, Yash, Lakhanpal, Shyamal

Abstract

LLMs have evolved from basic chatbots to the backbone of the AI ecosystem, now widely used in healthcare, schools, and government services. The domain-wide adoption of LLMs necessitates continuous evaluation to ensure their safety and fairness. Common issues encountered after deploying LLMs include inconsistent outputs and hallucinations of incorrect information. Although numerous LLM evaluation tools exist, most are limited to testing a single parameter at a time or require massive computational resources that are not accessible to most researchers. TriEval addresses these challenges by evaluating LLM outputs across multiple parameters, including bias, toxicity, and truthfulness together, while minimizing computing resources. The pipeline is compatible with both open- and closed-source models and runs on a standard laptop without a GPU cluster. TriEval has been tested on four models: Llama 3 8B, Mistral 7B, Gemma 2 9B, and Claude Haiku. The results show clear differences between open-source and closed-source models, especially in terms of toxicity and truthfulness. TriEval is being released as open source to enable broader access for researchers with limited computational resources.

Chinese Translation

大型语言模型（LLMs）已经从基础的聊天机器人发展成为人工智能生态系统的核心，现广泛应用于医疗、教育和政府服务等领域。LLMs的广泛采用要求持续评估，以确保其安全性和公平性。在部署LLMs后常见的问题包括输出不一致和错误信息的幻觉。尽管存在众多LLM评估工具，但大多数工具仅限于一次测试单一参数，或需要大量计算资源，而这些资源对大多数研究人员而言并不易得。TriEval通过同时评估LLM输出的多个参数，包括偏见、毒性和真实性，来应对这些挑战，同时最小化计算资源的消耗。该管道兼容开源和闭源模型，并可在标准笔记本电脑上运行，无需GPU集群。TriEval已在四个模型上进行了测试：Llama 3 8B、Mistral 7B、Gemma 2 9B和Claude Haiku。结果显示开源模型和闭源模型之间在毒性和真实性方面存在明显差异。TriEval将作为开源工具发布，以便为计算资源有限的研究人员提供更广泛的访问权限。

View on arXiv Download PDF AI Translation

cs.AI / 19 / 2606.03040

RelGT-AC: A Relational Graph Transformer for Autocomplete Tasks in Relational Databases

RelGT-AC：一种用于关系数据库自动补全任务的关系图变换器

Jiang, Phillip

Abstract

Relational databases underpin modern enterprise, scientific, and healthcare systems, yet predictive machine learning on such data remains challenging due to their multi-table, heterogeneous, and temporal structure. Relational Deep Learning (RDL) addresses this by representing databases as heterogeneous graphs and applying graph neural networks (GNNs) directly. RelBench v2 recently introduced autocomplete tasks -- a practically motivated task type where the goal is to predict an existing column value from relational context, analogous to an intelligent form-filling assistant. We propose RelGT-AC (Relational Graph Transformer for Autocomplete), extending the RelGT architecture with three targeted contributions: (1) a column masking strategy that prevents trivial solutions by masking the target column during subgraph encoding; (2) a unified task head supporting binary classification, multiclass classification, and regression autocomplete tasks within a single model; and (3) a TF-IDF text encoder that automatically detects and encodes free-text columns, recovering strong lexical signal that categorical encoders discard. Across 7 tasks spanning 3 RelBench v2 datasets (rel-trial, rel-f1, rel-stack), RelGT-AC outperforms the GraphSAGE baseline on all 3 regression autocomplete tasks and achieves up to +10 AUROC points on text-heavy eligibility tasks via the TF-IDF encoder.

Chinese Translation

关系数据库支撑着现代企业、科学和医疗系统，但由于其多表、异构和时间结构，基于此类数据的预测机器学习仍然面临挑战。关系深度学习（Relational Deep Learning, RDL）通过将数据库表示为异构图并直接应用图神经网络（Graph Neural Networks, GNNs）来解决这一问题。RelBench v2 最近引入了自动补全任务——一种具有实际应用动机的任务类型，其目标是从关系上下文中预测现有列值，类似于智能表单填写助手。我们提出了 RelGT-AC（关系图变换器用于自动补全），在 RelGT 架构的基础上进行了扩展，提出了三项针对性的贡献：（1）一种列掩码策略，通过在子图编码过程中掩盖目标列来防止简单解决方案；（2）一个统一的任务头，支持二分类、多分类和回归自动补全任务，能够在单一模型中实现；（3）一个 TF-IDF 文本编码器，能够自动检测和编码自由文本列，恢复分类编码器所丢弃的强词汇信号。在涵盖 3 个 RelBench v2 数据集（rel-trial、rel-f1、rel-stack）的 7 个任务中，RelGT-AC 在所有 3 个回归自动补全任务上均优于 GraphSAGE 基线，并通过 TF-IDF 编码器在文本密集的资格任务上实现了高达 +10 AUROC 点的提升。

View on arXiv Download PDF AI Translation

cs.AI / 20 / 2606.03054

ToolGate: Token-Efficient Pre-Call Control for Tool-Augmented Vision-Language Agents

ToolGate：工具增强视觉语言代理的高效预调用控制

Liu, Anjie, Song, Yan, Chen, Zhixun, Gong, Ziqin, Yu, Zhongwei, Wang, Jun

Abstract

Tool-augmented vision-language agents can acquire external perceptual evidence through OCR, detection, segmentation, and other tools, but executing every proposed tool call is costly and sometimes unnecessary. We study the pre-call control problem: after a ReAct-style VLM agent proposes a perceptual tool call, should the call be executed, or skipped before its output enters the context? Across five benchmarks, we find that the baseline agent exhibits poor local selectivity: helpful and harmful calls occur at similar rates (11.8% vs. 9.9%), while most calls do not change the immediate forced-answer prediction. We introduce ToolGate, a lightweight external controller that predicts execute/skip decisions from trajectory text and simple structural features. Across two Qwen3-VL backbones, ToolGate reduces token cost to 64-69% of the unrestricted ReAct baseline while preserving average accuracy in cross-domain settings. With matched-domain trajectory training on Qwen3-VL-30B, it further improves average accuracy by 1.65 points. These results show that tool-augmented VLM agents benefit not only from better perceptual tools, but also from explicit control over when tool outputs are worth paying for.

Chinese Translation

工具增强的视觉语言代理可以通过光学字符识别（OCR）、检测、分割和其他工具获取外部感知证据，但执行每个提议的工具调用是有成本的，有时也是不必要的。我们研究了预调用控制问题：在 ReAct 风格的视觉语言模型（VLM）代理提出感知工具调用后，应该执行该调用，还是在其输出进入上下文之前跳过它？在五个基准测试中，我们发现基线代理表现出较差的局部选择性：有用和有害的调用发生率相似（分别为 11.8% 和 9.9%），而大多数调用并未改变即时强制回答的预测。我们引入了 ToolGate，这是一种轻量级外部控制器，可以根据轨迹文本和简单结构特征预测执行/跳过决策。在两个 Qwen3-VL 主干网络上，ToolGate 将令牌成本降低到不受限制的 ReAct 基线的 64-69%，同时在跨领域设置中保持平均准确性。通过在 Qwen3-VL-30B 上进行匹配领域的轨迹训练，它进一步将平均准确性提高了 1.65 个百分点。这些结果表明，工具增强的 VLM 代理不仅受益于更好的感知工具，还受益于明确控制何时值得为工具输出付费。

View on arXiv Download PDF AI Translation

cs.AI / 21 / 2606.03056

SkillDAG: Self-Evolving Typed Skill Graphs for LLM Skill Selection at Scale

SkillDAG：用于大规模 LLM 技能选择的自我演化类型技能图

Bai, Tong, Wan, Zhenglin, Zhou, Pengfei, Yu, Xingrui, Zhao, Wangbo, You, Yang, Tsang, Ivor W.

Abstract

As LLM agents adopt large skill libraries, selecting the right subset becomes a structural problem rather than a similarity-matching one: skills depend on, conflict with, specialize, or duplicate one another, a structure invisible to both full enumeration and embedding similarity. We present SkillDAG, which models inter-skill relationships as a typed directed graph and exposes it to an LLM agent as an inference-time, agent-callable structural retrieval interface, queried and evolved during execution rather than baked into a fixed retrieval pipeline: each search returns vector matches, typed-edge neighbors, and conflict signals, and a propose-then-commit protocol lets the agent register execution-backed edges so the graph accumulates structure across episodes. On ALFWorld and SkillsBench with MiniMax-M2.7, SkillDAG reaches 67.1% success and 27.3% reward, exceeding the strongest reported Graph-of-Skills baseline by +12.8 and +8.6 points; the advantage ports to gpt-5.2-codex, and intrinsic SkillsBench Ret@K rises from 65.5 to 78.2 under matched queries. These gains trace to isolable mechanisms: candidate ranking that stays robust as the pool grows 10x where a fixed seeding-diffusion pipeline degrades, and set-monotone online edits that enlarge ground-truth recall without evicting prior hits.

Chinese Translation

随着 LLM 代理采用大型技能库，选择合适的子集变成了一种结构性问题，而非相似性匹配问题：技能之间存在依赖、冲突、专业化或重复的关系，这种结构对于完全枚举和嵌入相似性都是不可见的。我们提出了 SkillDAG，它将技能之间的关系建模为一个类型化的有向图，并将其作为推理时的、可由代理调用的结构检索接口暴露给 LLM 代理，该接口在执行过程中被查询和演化，而不是固定在一个检索管道中：每次搜索返回向量匹配、类型边邻居和冲突信号，并且提议-然后-提交的协议允许代理注册基于执行的边，从而使图在多个回合中积累结构。在 ALFWorld 和 SkillsBench 上使用 MiniMax-M2.7，SkillDAG 达到了 67.1% 的成功率和 27.3% 的奖励，超越了最强的 Graph-of-Skills 基线，分别提高了 +12.8 和 +8.6 个点；这一优势也迁移到 gpt-5.2-codex，并且在匹配查询下，SkillsBench 的内在 Ret@K 从 65.5 上升到 78.2。这些增益源于可隔离的机制：候选排名在池子增长 10 倍时仍保持稳健，而固定的种子扩散管道则会退化，以及集合单调的在线编辑在不驱逐先前命中的情况下扩大了真实召回率。

View on arXiv Download PDF AI Translation

cs.AI / 22 / 2606.03066

CORE: Conflict-Oriented Reasoning for General Multimodal Manipulation Detection

CORE：面向冲突的通用多模态操控检测推理

Shen, Jinjie, Wang, Yaxiong, Wu, Yujiao, Cheng, Lechao, Hui, Tianrui, Pu, Nan, Li, Zhihui, Zhong, Zhun

Abstract

The rapid rise of generative AI has made multimodal fake news increasingly realistic and pervasive, posing severe threats to public trust and social stability. Existing detection methods rely heavily on manipulation-specific models and large-scale labeled data, resulting in poor generalization to emerging manipulation types. We observed that the essence of manipulated misinformation lies in its intrinsic conflicts, \textbf{i.e.,} semantic or physical inconsistencies either across modalities or with common world knowledge. Inspired by this observation, we propose \textbf{C}onflict-\textbf{O}riented \textbf{RE}asoning (\textbf{CORE}) framework, an effective paradigm that learns to endows multimodal large language models (MLLMs) with explicit conflict-capturing capability. To this end, CORE first constructs the Conflict Attribution Corpus (CAC) with fine-grained annotations of conflict factors and sources, providing essential data support for subsequent conflict perception training. By performing conflict-oriented representation enhancement and reasoning based on CAC, CORE achieves robust and generalizable conflict detection, effectively and rapidly adapting to unseen manipulation types with a few samples or in even zero-shot settings. Extensive experiments demonstrate that CORE surpasses state-of-the-art models. The dataset and code are publicly available at https://github.com/shen8424/CORE.

Chinese Translation

生成性人工智能的快速崛起使得多模态假新闻变得愈加真实和普遍，给公众信任和社会稳定带来了严重威胁。现有的检测方法严重依赖于特定操控模型和大规模标注数据，导致对新兴操控类型的泛化能力较差。我们观察到，被操控的信息的本质在于其内在的冲突，即在不同模态之间或与常识知识之间存在的语义或物理不一致性。受到这一观察的启发，我们提出了冲突导向推理（Conflict-Oriented Reasoning，CORE）框架，这是一种有效的范式，旨在赋予多模态大型语言模型（Multimodal Large Language Models，MLLMs）明确的冲突捕捉能力。为此，CORE首先构建了冲突归因语料库（Conflict Attribution Corpus，CAC），对冲突因素和来源进行了细粒度标注，为后续的冲突感知训练提供了必要的数据支持。通过基于CAC进行冲突导向的表示增强和推理，CORE实现了稳健且具有良好泛化能力的冲突检测，能够在少量样本甚至零样本设置下有效快速地适应未见的操控类型。大量实验表明，CORE超越了现有的最先进模型。数据集和代码已公开，地址为 https://github.com/shen8424/CORE。

View on arXiv Download PDF AI Translation

cs.AI / 23 / 2606.03083

DELTAMEM: Incremental Experience Memory for LLM Agents via Residual Trees

DELTAMEM：通过残差树实现大语言模型代理的增量经验记忆

Tan, Haoran, Zhang, Zeyu, Cao, Zhicheng, Li, Rui, Chen, Xu

Abstract

Large Language Model (LLM)-based agents increasingly rely on memory to learn from experiences over continual interactions. However, storing experiences as independent, flat units leads to substantial redundancy and retrieval conflicts, as similar episodes repeat overlapping content and subtle scene variations cause retrieved memories to offer contradictory guidance. To address this, we introduce residual experience, positing that newly acquired experience is often an incremental variation of existing knowledge. We propose DeltaMem, a framework that organizes experience memory into two independent residual trees, one storing goal-conditioned task experience as reusable skills and another for scene-level environment knowledge. Each tree uses a root node for generalized base experiences and incremental delta nodes for subsequent variations, allowing related experiences to share a common foundation without duplication. For retrieval, a failure-penalized similarity scan locates the best match, reconstructing the full experience via root-to-match chain composition. An autonomous consolidation mechanism distills high-frequency paths into new root nodes, enabling the trees to self-organize from general heuristics to specialized variants. Experiments across diverse interactive environments show that DeltaMem consistently outperforms existing baselines. To facilitate future research, we release the code at https://github.com/import-myself/DeltaMem.

Chinese Translation

基于大语言模型（LLM）的代理越来越依赖记忆来从持续交互中学习经验。然而，将经验存储为独立的、平坦的单元会导致显著的冗余和检索冲突，因为相似的情节重复重叠内容，而细微的场景变化则导致检索到的记忆提供矛盾的指导。为了解决这个问题，我们引入了残差经验，认为新获得的经验往往是现有知识的增量变体。我们提出了DeltaMem，一个将经验记忆组织为两个独立残差树的框架，一个用于存储以目标为条件的任务经验作为可重用技能，另一个用于场景级环境知识。每棵树使用一个根节点来表示通用的基础经验，并使用增量的增量节点来表示后续变体，从而允许相关经验共享一个共同的基础而不产生重复。在检索方面，失败惩罚相似性扫描定位最佳匹配，通过根到匹配链组合重构完整经验。一个自主整合机制将高频路径提炼为新的根节点，使得树能够从一般启发式自我组织到专业变体。在多样化的交互环境中的实验表明，DeltaMem始终优于现有基线。为了促进未来的研究，我们在 https://github.com/import-myself/DeltaMem 发布了代码。

View on arXiv Download PDF AI Translation

cs.AI / 24 / 2606.03092

The Shadow Price of Reasoning: Economic Perspective on Optimal Budget Allocation for LLMs

推理的影子价格：大型语言模型的最优预算分配经济视角

Wan, Xu, Zhu, Speed, Cai, Jianwei, Chen, Guang, Huang, XiMing, Zhou, Wiggin, Sun, Mingyang

Abstract

Inference-time scaling has emerged as a critical avenue for enhancing Large Language Models' performance, yet real-world deployment is constrained by strict computational budgets. In this work, we formulate inference budget allocation as a global constrained optimization problem governed by economic principles. By modeling per-query reasoning utility with a shifted-surge function, we derive an optimal allocation policy based on a global shadow price that equilibrates marginal utility under resource scarcity. Based on this theory, we propose Constrained Latent-utility Equilibrium Allocation for Reasoning (CLEAR). It performs rational abandonment and reallocates resources from insolvent queries to solvable queries near their emergence thresholds. Extensive experiments on several reasoning tasks with different traffic streams demonstrate that CLEAR significantly improves the Pareto frontier of total token cost versus mean accuracy. In resource-scarce regimes, CLEAR achieves up to a 3x improvement in global accuracy compared to uniform allocation.

Chinese Translation

推理时的扩展已成为提升大型语言模型（LLMs）性能的关键途径，但实际部署受到严格计算预算的限制。在本研究中，我们将推理预算分配形式化为一个受经济原则支配的全局约束优化问题。通过使用移位激增函数对每个查询的推理效用进行建模，我们基于全球影子价格推导出一种最优分配策略，该策略在资源稀缺的情况下平衡边际效用。基于这一理论，我们提出了推理的约束潜在效用均衡分配（Constrained Latent-utility Equilibrium Allocation for Reasoning，CLEAR）。该方法执行理性放弃，并将资源从无解查询重新分配到接近其出现阈值的可解查询。在多个推理任务和不同流量情况下的广泛实验表明，CLEAR显著改善了总令牌成本与平均准确率的帕累托前沿。在资源稀缺的环境中，与均匀分配相比，CLEAR在全球准确率上实现了最高3倍的提升。

View on arXiv Download PDF AI Translation

cs.AI / 25 / 2606.03093

Decomposing how prompting steers behavior

分解提示如何引导行为

Cheng, Fan L., Kriegeskorte, Nikolaus

Abstract

Prompting steers large language models (LLMs) and vision-language models (VLMs) without weight updates, but it remains unclear how instruction changes reshape internal representations to produce behavior. We introduce a nested geometric decomposition framework that treats prompting as a transformation of the representational geometry of the content following the prompt. For each prompt pair, we align representations of the same stimuli under two prompts using increasingly expressive stimulus-invariant maps: translation, rigid transformation with uniform scaling, sequential axis scaling, affine transformation, and nonlinear transformation. We then causally test each map by replacing a single layer's prompt-A hidden state for held-out stimuli with its mapped counterpart and measuring recovery of prompt-B representational geometry and behavior. Across three LLMs, three VLMs, and six text or image datasets spanning style, emotion, scene content, and number, prompts consistently reshape representations toward the instructed task structure. Cross-validated variance decomposition shows that much prompt-induced activation change is captured by shape-preserving maps, especially translation and rigid transformation with uniform scaling, while tier profiles reveal model- and task-specific routing strategies across layers. Crucially, although translation and rigid tiers already improve behavioral agreement, affine transformation is the first tier to nearly recover target-prompt task geometry and yields corresponding behavioral gains. This suggests that cross-dimensional linear mixing is a key mechanism by which prompts reorganize representations toward instructed task structure. Our framework decomposes prompt-induced representational change into interpretable geometric components and reveals how models route task-relevant structure to produce prompt-driven behavior.

Chinese Translation

提示能够引导大型语言模型（LLMs）和视觉-语言模型（VLMs）而无需更新权重，但尚不清楚指令变化如何重塑内部表征以产生行为。我们引入了一种嵌套几何分解框架，将提示视为对内容表征几何的变换。对于每对提示，我们使用越来越具表现力的刺激不变映射对同一刺激在两个提示下的表征进行对齐：平移、具有均匀缩放的刚性变换、顺序轴缩放、仿射变换和非线性变换。然后，我们通过将单层的提示-A隐状态替换为其映射对应物，因果测试每个映射，并测量提示-B的表征几何和行为的恢复。在三种LLMs、三种VLMs和六个涵盖风格、情感、场景内容和数量的文本或图像数据集上，提示始终将表征重塑为指令任务结构。交叉验证的方差分解显示，大部分提示引起的激活变化是由保持形状的映射捕获的，特别是平移和具有均匀缩放的刚性变换，而层级轮廓揭示了模型和任务特定的路由策略。至关重要的是，尽管平移和刚性层已经改善了行为一致性，但仿射变换是第一个几乎恢复目标提示任务几何的层，并带来了相应的行为提升。这表明跨维度线性混合是提示重组表征以朝向指令任务结构的关键机制。我们的框架将提示引起的表征变化分解为可解释的几何成分，并揭示了模型如何路由与任务相关的结构以产生提示驱动的行为。

View on arXiv Download PDF AI Translation

cs.AI / 26 / 2606.03097

From Long News to Accurate Forecast: Importance-Aware Fusion and PRM-Guided Reflection for Time Series Forecasting

从长新闻到准确预测：基于重要性意识的融合与PRM引导的反思在时间序列预测中的重要性

Liu, Mingyang, Kang, Qingcan, Wang, Yuke, Kai, Shixiong, Liang, Kaichao, Zhen, Hui-Ling, Zhong, Tao, Yuan, Mingxuan, Song, Linqi

Abstract

Incorporating news into time series forecasting is appealing because news can reveal abrupt exogenous events that historical values alone cannot recover. However, existing LLM-based news-forecasting pipelines face two practical limitations: relevant news articles often exceed the model's context window, and iterative retrieval of supplementary news is typically unguided, leading to redundant updates and slow convergence. We address these issues with a novel framework that combines importance-aware news compression and process-level retrieval supervision. First, we train an importance reward model that estimates the forecasting utility of each article and uses this signal to allocate compression budgets during sequential pairwise fusion, preserving informative content within a fixed context limit. Second, we introduce a process reward model (PRM) that ranks multiple supplementary-news candidates conditioned on the current error profile and the history of previously selected articles, replacing one-shot blind retrieval with quality-controlled selection. Both components are trained offline using historical data with ground truth; inference uses the frozen filtering logic and compression modules without any reflection loop. Experiments on finance, energy, traffic, and bitcoin forecasting benchmarks show that our method improves prediction accuracy over strong baselines, significantly reduces the number of refinement iterations compared to the iterative baseline, and remains effective when relevant articles span thousands of tokens.

Chinese Translation

将新闻纳入时间序列预测具有吸引力，因为新闻可以揭示历史数据无法恢复的突发外部事件。然而，现有基于LLM的新闻预测流程面临两个实际限制：相关的新闻文章往往超出模型的上下文窗口，并且补充新闻的迭代检索通常缺乏指导，导致冗余更新和缓慢收敛。我们通过一个新颖的框架解决这些问题，该框架结合了基于重要性的新闻压缩和过程级检索监督。首先，我们训练一个重要性奖励模型，该模型估计每篇文章的预测效用，并利用这一信号在顺序成对融合过程中分配压缩预算，在固定的上下文限制内保留信息内容。其次，我们引入一个过程奖励模型（PRM），该模型根据当前的误差特征和之前选择的文章历史对多个补充新闻候选进行排名，将一次性盲检索替换为质量控制选择。这两个组件均使用带有真实标签的历史数据进行离线训练；推理时使用冻结的过滤逻辑和压缩模块，而无需任何反思循环。在金融、能源、交通和比特币预测基准上的实验表明，我们的方法在预测准确性上优于强基线，与迭代基线相比显著减少了精炼迭代次数，并且在相关文章跨越数千个标记时仍然有效。

View on arXiv Download PDF AI Translation

cs.AI / 27 / 2606.03103

DeskCraft: Benchmarking Desktop Agents on Professional Workflows and Human-in-the-Loop Collaboration

DeskCraft：在专业工作流程和人机协作中的桌面代理基准测试

Wang, Wenkai, Xiong, Tao, Ni, Jingchen, Bao, Yunpeng, Li, Xiyun, Liu, Tianqi, Guo, Hongcan, Huang, Zilong, Zhang, Shengyu

Abstract

Real-world professional desktop workflows in specialized creative and engineering software unfold over long horizons and often require human-in-the-loop coordination, where agents proactively seek necessary information and users provide additional instructions, clarifications, feedback, or corrections as the task progresses. Yet existing desktop GUI benchmarks mostly reduce this setting to short, simplified tasks with all user instructions provided upfront. To address this issue, we introduce DeskCraft, a desktop GUI benchmark targeting long horizon creative and engineering workflows and proactive human-agent collaboration. DeskCraft organizes tasks into a multilevel difficulty taxonomy, with long horizon tasks requiring over 50 execution steps, and covers professional creative software across design, video, audio, and 3D creation. Furthermore, DeskCraft formalizes human-agent collaboration into an interaction protocol covering mid-turn and post-turn exchanges. Mid-turn interaction captures both agent-initiated clarification under uncertainty and user-initiated interruption during execution, while post-turn interaction accommodates user-driven feedback after the agent signals completion, together spanning the full space of realistic collaboration patterns. We evaluate 18 proprietary and open source agents on 538 tasks and find that GPT-5.4 reaches 31.6% on standard tasks and 27.6% on interactive tasks. Further analyses reveal persistent failures in long horizon workflow delivery and proactive clarification. We will open-source all evaluation codes, tasks, and data at https://github.com/mrwwk/DeskCraft.

Chinese Translation

现实世界中的专业桌面工作流程在专业创意和工程软件中展开，通常涉及较长的时间跨度，并且需要人机协作，其中代理主动寻求必要的信息，而用户在任务进行过程中提供额外的指令、澄清、反馈或修正。然而，现有的桌面图形用户界面基准测试大多将这种设置简化为短小的任务，并且所有用户指令均在任务开始前提供。为了解决这一问题，我们提出了DeskCraft，这是一个针对长时间跨度创意和工程工作流程以及主动人机协作的桌面图形用户界面基准测试。DeskCraft将任务组织成多层次的难度分类，其中长时间跨度的任务需要超过50个执行步骤，并涵盖设计、视频、音频和3D创作等专业创意软件。此外，DeskCraft将人机协作形式化为一个交互协议，涵盖中途和后期的交流。中途交互捕捉了代理在不确定性下主动澄清的情况以及用户在执行过程中主动中断的情况，而后期交互则容纳了用户在代理信号完成后提供的反馈，从而全面涵盖了现实协作模式的所有空间。我们在538个任务上评估了18个专有和开源代理，发现GPT-5.4在标准任务上达到31.6%，在交互任务上达到27.6%。进一步分析显示，在长时间跨度工作流程交付和主动澄清方面存在持续的失败。我们将开放所有评估代码、任务和数据，网址为 https://github.com/mrwwk/DeskCraft。

View on arXiv Download PDF AI Translation

cs.AI / 28 / 2606.03108

EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning

EvoTrainer：共同进化的LLM策略与自主智能强化学习训练框架

Chen, Guhong, Shi, Yingcheng, Li, Yongbin, Li, Binhua, Xu, Xander, Wei, Hu, Ni, Shiwen, Yang, Min, Ye, Jieping

Abstract

Autonomous LLM training is often framed as recipe search, which leaves the training harness largely static. This limitation sharpens in agentic RL, where shifting bottlenecks and scalar rewards mask diverse failure modes. We introduce EvoTrainer, an autonomous training framework that co-evolves LLM policies and training-side harnesses through empirical feedback: it diagnoses rollout-level evidence, revises diagnostics, backtests interventions, and accumulates reusable skills. Evaluated on mathematical reasoning, competitive-programming code generation, and repository-level software engineering, EvoTrainer matches or exceeds the human-engineered RL references under the same data, codebase, and evaluation protocol, with the largest gain on long-horizon agentic SWE. Trajectory analyses show that retained strategies diverge across domains, evolving diagnostics prevent invalid high-scoring branches from being promoted, and reusable skills shape later search. Autonomous LLM RL should move beyond recipe search toward joint evolution of policies and the training harnesses that interpret them.

Chinese Translation

自主LLM训练通常被视为配方搜索，这使得训练框架在很大程度上保持静态。这一局限性在智能强化学习（agentic RL）中更加明显，因为不断变化的瓶颈和标量奖励掩盖了多样化的失败模式。我们提出了EvoTrainer，这是一种自主训练框架，通过经验反馈共同进化LLM策略和训练侧框架：它诊断滚动级别的证据，修订诊断，回测干预，并积累可重用的技能。在数学推理、竞争编程代码生成和代码库级软件工程的评估中，EvoTrainer在相同的数据、代码库和评估协议下，匹配或超越了人类设计的强化学习参考，尤其在长时间跨度的智能软件工程（agentic SWE）中获得了最大的提升。轨迹分析表明，保留的策略在不同领域中存在差异，进化的诊断防止无效的高得分分支被提升，而可重用的技能塑造了后续的搜索。自主LLM强化学习应超越配方搜索，朝着策略与解释它们的训练框架的共同进化方向发展。

View on arXiv Download PDF AI Translation

cs.AI / 29 / 2606.03135

Uncertainty-Aware Clarification in LLM Agents with Information Gain

具有信息增益的LLM代理的不确定性感知澄清

Deng, Mengyi, Li, Zhiwei, Li, Xin, Zhu, Tingyu, Zhao, Ying, Guo, Zhijiang, Wang, Wei

Abstract

Large Language Model (LLM) agents often operate under underspecified user instructions, where latent uncertainty over user intent leads to erroneous tool actions. To address this challenge, we propose a goal-oriented clarification framework that aligns clarification behavior with ambiguity resolution. Central to our approach is the Information Gain Reward, a metric that quantifies the utility of clarification questions by measuring the Bayesian belief update towards the ground-truth goal induced by the clarification exchange. We train the clarifier (LLM) using this reward to optimize for high information gain, ensuring that clarifications effectively reduce uncertainty and improve task completion within the agent-tool-user environment. We validate our framework within a clarification-enhanced $\tau$-Bench environment, conducting cross-agent evaluations across five heterogeneous backbones. Empirical results demonstrate that our method consistently improves the success rate by 3.7\% over the no-clarification baseline, while adding only 0.3 total interaction steps on average.

Chinese Translation

大型语言模型（LLM）代理通常在用户指令不明确的情况下操作，其中对用户意图的潜在不确定性导致错误的工具操作。为了解决这一挑战，我们提出了一种以目标为导向的澄清框架，该框架将澄清行为与模糊性解决相结合。我们方法的核心是信息增益奖励（Information Gain Reward），这一指标通过测量澄清交流所引起的对真实目标的贝叶斯信念更新来量化澄清问题的效用。我们利用这一奖励训练澄清者（LLM），以优化高信息增益，确保澄清有效减少不确定性并改善代理-工具-用户环境中的任务完成率。我们在增强澄清的$ au$-Bench环境中验证了我们的框架，针对五个异构基础模型进行了跨代理评估。实证结果表明，我们的方法在成功率上比无澄清基线提高了3.7，同时平均仅增加了0.3个总交互步骤。

View on arXiv Download PDF AI Translation

cs.AI / 30 / 2606.03137

Think-Before-Speak: From Internal Evaluation to Public Expression in Multi-Agent Social Simulation

思考再发言：多智能体社会模拟中的内部评估与公共表达

Yang, Kaiqi, Peng, Tai-Quan, Lee, Sanguk, Liu, Hui

Abstract

LLM-based multi-agent simulation offers a promising way to study social interaction, deliberation, and collective opinion dynamics. However, many existing dialogue simulation frameworks represent interaction mainly as observable turn exchange or aggregated outputs, leaving the internal evaluative processes behind silence, speaking intention, and public expression difficult to examine. We introduce TBS (Think-Before-Speak), an interval-based multi-agent simulation framework that separates agents' private reasoning from public utterance generation. At each interval, all agents update structured internal states based on the shared dialogue history and their own memory. These states include dissonance-related appraisal, perceived opinion climate, perceived isolation risk, response strategy, and willingness to speak. The orchestrator then resolves competing speaking intentions and commits one utterance to the public dialogue, allowing internal evaluation and public interaction to co-evolve over time. We evaluate TBS in simulated town hall discussions on a climate-related policy issue. Results show that TBS produces coherent internal-state traces and that these traces vary systematically across turn-allocation, silence, and memory conditions. Dissonance-related appraisal increases agents' willingness to speak, whereas silence-pressure appraisal decreases it. Once speaking intention is formed, public expression is shaped mainly by turn-allocation rules. These findings suggest that TBS supports mechanism-sensitive social simulation by making the pathway from internal evaluation to public expression observable and analyzable.

Chinese Translation

基于大型语言模型（LLM）的多智能体模拟为研究社会互动、审议和集体意见动态提供了一种有前景的方法。然而，许多现有的对话模拟框架主要将互动表现为可观察的轮次交换或聚合输出，忽视了内部评估过程，使得沉默、发言意图和公共表达的研究变得困难。我们提出了TBS（Think-Before-Speak），一种基于时间间隔的多智能体模拟框架，它将智能体的私人推理与公共发言生成分开。在每个时间间隔内，所有智能体根据共享的对话历史和自身记忆更新结构化的内部状态。这些状态包括与不和谐相关的评估、感知的意见气候、感知的孤立风险、反应策略和发言意愿。然后，协调者解决竞争的发言意图，并将一个发言提交给公共对话，从而使内部评估与公共互动随时间共同演化。我们在模拟的市政厅讨论中评估了TBS，讨论了与气候相关的政策问题。结果表明，TBS产生了一致的内部状态轨迹，并且这些轨迹在轮次分配、沉默和记忆条件下系统性变化。与不和谐相关的评估增加了智能体的发言意愿，而沉默压力评估则降低了发言意愿。一旦形成发言意图，公共表达主要受轮次分配规则的影响。这些发现表明，TBS通过使内部评估到公共表达的路径可观察和可分析，从而支持机制敏感的社会模拟。

View on arXiv Download PDF AI Translation

cs.AI / 31 / 2606.03144

GTBench: A Curriculum-Grounded Benchmark for Evaluating LLMs as Mathematical Research Assistants in Graph Theory

GTBench：一个基于课程的基准，用于评估大型语言模型作为图论中的数学研究助手

Nader, Noujoud, Aljabea, Ibrahem, Diehl, Patrick, Gupta, Deepti

Abstract

Large language models (LLMs) are increasingly used as self-study assistants in technical disciplines, yet their reliability as mathematical reasoning assistants remains poorly understood. We introduce GTBench, a curriculum-grounded benchmark for evaluating LLMs as mathematical research assistants in graph theory, comprising 63 problems organized into three groups of increasing difficulty: undergraduate definitions and basic properties (Group 1), algorithm tracing and structural reasoning (Group 2), and graduate-level proof construction (Group 3). Problems are sourced from verified academic materials including Diestel's Graph Theory. We evaluate five frontier models -- GPT-5, Claude Sonnet 4.6, Gemini 2.5 Flash-Lite, Llama 3.3 70B, and Mistral Large 3 -- under zero-shot and chain-of-thought prompting, using exact-match and LLM-as-judge evaluation for Groups 1 and 2, and a hybrid human expert and LLM-as-judge protocol for Group 3. Our results reveal a pronounced performance hierarchy: GPT-5 approaches ceiling on Group 1 (95.8% zero-shot) and maintains meaningful accuracy on graduate proofs (82%), while all other models degrade substantially with difficulty, with Llama achieving 0% under human evaluation on Group 3 zero-shot. Failure mode analysis shows that correct algorithm, wrong execution errors dominate Groups 1 and 2, while Group 3 additionally surfaces incomplete reasoning failures and reveals systematic disagreement between human evaluators and the automated judge, particularly on verbose or near-complete proofs (kappa = 0.48-0.83 across human pairs). GTBench provides the first curriculum-grounded evaluation framework for graph-theoretic reasoning in LLMs, with direct implications for the governance of AI tools in mathematical education and scientific research.

Chinese Translation

大型语言模型（LLMs）在技术学科中越来越多地被用作自学助手，但它们作为数学推理助手的可靠性仍然不甚明了。我们介绍了GTBench，这是一个基于课程的基准，用于评估LLMs作为图论中的数学研究助手，包含63个问题，分为三个难度逐渐增加的组别：本科生定义和基本属性（组1）、算法追踪和结构推理（组2）、以及研究生级别的证明构造（组3）。问题来源于经过验证的学术材料，包括Diestel的《图论》。我们在零-shot和思维链提示下评估了五个前沿模型——GPT-5、Claude Sonnet 4.6、Gemini 2.5 Flash-Lite、Llama 3.3 70B和Mistral Large 3，使用精确匹配和LLM作为评判者的评估方法对组1和组2进行评估，并对组3采用混合人类专家和LLM作为评判者的协议。我们的结果揭示了明显的性能层级：GPT-5在组1接近上限（95.8%零-shot），并在研究生证明中保持了有意义的准确性（82%），而所有其他模型在难度增加时显著下降，Llama在组3的零-shot人类评估中达到0%。失败模式分析显示，正确算法但错误执行的错误在组1和组2中占主导地位，而组3则额外出现了不完整推理的失败，并揭示了人类评估者与自动评判者之间的系统性分歧，特别是在冗长或接近完整的证明上（kappa = 0.48-0.83，跨人类对）。GTBench为LLMs中的图论推理提供了第一个基于课程的评估框架，对数学教育和科学研究中AI工具的治理具有直接影响。

View on arXiv Download PDF AI Translation

cs.AI / 32 / 2606.03157

ClinicalMC: A Benchmark for Multi-Course Clinical Decision-Making with Large Language Models

ClinicalMC：基于大型语言模型的多阶段临床决策制定基准

Hou, Ruihui, Zhu, Siyi, Huai, Ziyue, Yu, Guangya, Fan, Yongqi, Wang, Chunming, Ruan, Tong

Abstract

Large language models (LLMs) have been widely adopted in healthcare, yet they still encounter significant challenges in complex clinical decision-making scenarios. Existing benchmarks primarily assess LLM performance in single-course settings and lack systematic evaluation in multi-course scenarios, where a patient's condition evolves over time. To address this gap, we propose ClinicalMC, a benchmark for multi-course clinical decision-making. It includes 1,275 Chinese and 5,804 English samples across four stages from admission to discharge. These stages cover triage, first-course examination/diagnosis/treatment, subsequent multi-course examination/assessment/treatment, and final diagnosis. In ClinicalMC, patients in the English dataset undergo an average of 5.11 clinical courses, whereas those in the Chinese dataset undergo 3.42. To assess LLM performance, we construct a multi-agent evaluation framework that includes patient, examiner, and doctor agents. Based on the benchmark and framework, we design two experimental settings -- a single-turn static setting and a multi-turn dynamic setting -- and assess three categories of LLMs: 1) closed-source LLMs like GPT5-mini; 2) open-source LLMs like DeepSeek-V3.2; and 3) medical LLMs like HuatuoGPT-o1. Through extensive evaluation, we aim to better understand LLM performance in the medical domain and support its effective deployment in healthcare.

Chinese Translation

大型语言模型（LLMs）在医疗领域得到了广泛应用，但在复杂的临床决策场景中仍面临重大挑战。现有基准主要评估LLM在单阶段设置下的表现，缺乏对患者病情随时间演变的多阶段场景的系统评估。为填补这一空白，我们提出了ClinicalMC，一个用于多阶段临床决策制定的基准。该基准包含1275个中文样本和5804个英文样本，覆盖从入院到出院的四个阶段。这些阶段包括分诊、首次检查/诊断/治疗、后续多阶段检查/评估/治疗以及最终诊断。在ClinicalMC中，英文数据集中患者平均经历5.11个临床阶段，而中文数据集中患者平均经历3.42个。为了评估LLM的表现，我们构建了一个多代理评估框架，包括患者、检查员和医生代理。基于该基准和框架，我们设计了两种实验设置——单轮静态设置和多轮动态设置，并评估三类LLM：1）封闭源LLM，如GPT5-mini；2）开放源LLM，如DeepSeek-V3.2；3）医疗LLM，如HuatuoGPT-o1。通过广泛的评估，我们旨在更好地理解LLM在医疗领域的表现，并支持其在医疗保健中的有效应用。

View on arXiv Download PDF AI Translation

cs.AI / 33 / 2606.03203

MedCUA-Bench: A Screenshot-Only Benchmark for Clinical Computer-Use Agents

MedCUA-Bench：一种仅基于截图的临床计算机使用代理基准测试

Yu, Jia, Wang, Zilong, Jiang, Xinyang, Li, Dongsheng, Wang, Shuo

Abstract

Computer-use agents could automate repetitive screen-based clinical work, but their reliability in medical graphical user interfaces remains largely unvalidated. Existing benchmarks focus on general web or desktop tasks and underrepresent medical software, which requires domain knowledge, exhibits markedly different UI design from mainstream applications, lacks public testing environments, and demands safety validation beyond task completion. We introduce MedCUA-Bench, an interactive benchmark for clinical computer-use agents. It covers 18 clinical scenarios across 10 medical domains, reconstructed from real product manuals and open-source medical systems to capture authentic clinical interfaces while avoiding licensing and privacy constraints. Each task ships with paired intent- and step-level goals to disentangle clinical reasoning from UI execution, and is evaluated by a deterministic checker over task completion and five clinical safety dimensions. Across 23 agents, the best closed-source model reaches 54.2% strict success, while all models remain below 9% on the real OpenEMR. Open-source agents average only 2.5%, with the best reaching 16.2%. MedCUA-Bench exposes the gap between current agents and reliable clinical software use, providing a reproducible testbed for future research.

Chinese Translation

计算机使用代理可以自动化重复的基于屏幕的临床工作，但它们在医学图形用户界面中的可靠性仍然未得到充分验证。现有基准测试主要集中在一般的网络或桌面任务上，未能充分代表医学软件，而医学软件需要领域知识，用户界面设计与主流应用显著不同，缺乏公共测试环境，并且在任务完成之外还需要安全验证。我们介绍了MedCUA-Bench，这是一个针对临床计算机使用代理的互动基准测试。它涵盖了10个医学领域中的18个临床场景，重建自真实产品手册和开源医学系统，以捕捉真实的临床界面，同时避免许可和隐私限制。每个任务都配有意图和步骤级别的目标，以区分临床推理与用户界面执行，并通过确定性检查器对任务完成情况和五个临床安全维度进行评估。在23个代理中，最佳的闭源模型达到了54.2%的严格成功率，而所有模型在真实的OpenEMR上均低于9%。开源代理的平均成功率仅为2.5%，最佳模型达到了16.2%。MedCUA-Bench揭示了当前代理与可靠临床软件使用之间的差距，为未来的研究提供了可重复的测试平台。

View on arXiv Download PDF AI Translation

cs.AI / 34 / 2606.03214

Effect of Demographic Bias on Skin Lesion Classification

人口偏差对皮肤病变分类的影响

Raumanns, Ralf, Schouten, Gerard, Cheplygina, Veronika, Pluim, Josien P. W.

Abstract

In this study, we evaluate the performance of skin lesion classification using ResNet-based convolutional models, focusing on the impact of demographic bias in training data, particularly variations in patient sex and age. We use linear programming to generate datasets with controlled demographic characteristics, allowing systematic investigation of bias effects. Three learning strategies are evaluated: a single-task model, a reinforcing multi-task model, and an adversarial learning scheme. Our sex-based analysis indicates that sex-specific training datasets optimise model performance. Notably, including male patients in the training data improved performance for the male subgroup, even in female-majority cases. Reinforcing and adversarial learning schemes narrowed or eliminated bias gaps in balanced and female-majority datasets. However, these strategies proved less effective in male-majority settings, where models continued to perform better for males than females. The two learning schemes showed marginal bias reduction compared to the baseline model in predominantly male patient populations. Age-based analysis demonstrates comparable baseline performance across the three model approaches, with performance declining across age categories. Younger groups consistently achieve the highest performance, regardless of training data distribution. Although balanced training yields optimal results for the youngest age category, performance decreases in older categories. We find that sex biases arise mainly from data imbalances, while age biases consistently favour younger groups regardless of distribution. These distinct mechanisms require targeted mitigation strategies. Additionally, cross-dataset validation on two external datasets revealed that domain shifts notably affect performance and patterns of demographic bias.

Chinese Translation

在本研究中，我们评估了基于ResNet的卷积模型在皮肤病变分类中的性能，重点关注训练数据中人口偏差的影响，特别是患者性别和年龄的变化。我们使用线性规划生成具有控制人口特征的数据集，从而系统性地研究偏差效应。我们评估了三种学习策略：单任务模型、强化多任务模型和对抗学习方案。我们的性别分析表明，性别特定的训练数据集能够优化模型性能。值得注意的是，在训练数据中包含男性患者改善了男性子组的性能，即使在女性占多数的情况下也是如此。强化和对抗学习方案在平衡和女性占多数的数据集中缩小或消除了偏差差距。然而，这些策略在男性占多数的环境中效果较差，模型在男性中的表现仍然优于女性。与基线模型相比，这两种学习方案在以男性患者为主的人群中表现出边际偏差减少。基于年龄的分析显示，三种模型方法在基线性能上相当，但在不同年龄类别中性能下降。年轻组的表现始终是最高的，无论训练数据的分布如何。尽管平衡训练在最年轻年龄组中产生最佳结果，但在较老类别中的表现下降。我们发现，性别偏差主要源于数据不平衡，而年龄偏差则始终偏向年轻组，无论分布如何。这些不同的机制需要有针对性的缓解策略。此外，在两个外部数据集上的跨数据集验证表明，领域转移显著影响性能和人口偏差模式。

View on arXiv Download PDF AI Translation

cs.AI / 35 / 2606.03236

Perceive Before Reasoning: A Pre-Reasoning Perception Framework for Efficient and Reliable Proactive Mobile Agents

先感知再推理：一种高效可靠的主动移动代理预推理感知框架

Ding, Zhijie, Hong, Weinan, Zhu, Zicheng, Li, Lei, Kong, Dezhi, Wang, Hao, Zhou, Peng, Jiang, Xuchu, Xu, Jiaming

Abstract

Multimodal large language models (MLLMs) have substantially advanced mobile agents, yet proactive mobile assistance remains challenging because agents must decide \emph{when} to intervene before determining \emph{how} to assist. Existing systems often implement these two decisions within a unified MLLM-based pipeline, leading to goal misalignment between conservative intervention filtering and comprehensive assistance generation, as well as redundant inference when the agent should remain silent. To address these limitations, we propose the \textbf{Pre-Reasoning Perception Framework (PRPF)}, a two-stage framework built on perceiving before reasoning. PRPF introduces a lightweight Multimodal Proactive Perceptor (MPP) for intervention gating and context compression, and activates the Proactive Agent Reasoner (PAR) only when intervention is warranted. Experiments on the ProactiveMobile benchmark show that PRPF substantially reduces false trigger rates (FTR) while improving success rates (SR) and inference efficiency over the ProactiveMobile baseline.

Chinese Translation

多模态大型语言模型（MLLMs）显著推动了移动代理的发展，但主动移动辅助仍然面临挑战，因为代理必须在确定如何提供帮助之前决定何时干预。现有系统通常在统一的基于MLLM的管道中实现这两个决策，导致保守的干预过滤与全面的辅助生成之间目标不一致，以及在代理应保持沉默时出现冗余推理。为了解决这些局限性，我们提出了 extbf{预推理感知框架（PRPF）}，这是一个基于先感知后推理的两阶段框架。PRPF引入了一种轻量级的多模态主动感知器（MPP）用于干预门控和上下文压缩，并仅在需要干预时激活主动代理推理器（PAR）。在ProactiveMobile基准上的实验表明，PRPF显著降低了误触发率（FTR），同时提高了成功率（SR）和推理效率，相较于ProactiveMobile基线表现更佳。

View on arXiv Download PDF AI Translation

cs.AI / 36 / 2606.03237

Solipsistic Superintelligence is Unlikely to be Cooperative

自我中心超级智能不太可能是合作性的

Trivedi, Rakshit S, Jaques, Natasha, Cross, Logan, Vezhnevets, Alexander Sasha, Leibo, Joel Z

Abstract

AI's central challenge is shifting from capability to coexistence. The dominant paradigm in AI research focuses on developing powerful agents that treat the world as an exogenous and stationary source of feedback. We contend that superintelligence, an extremely capable task solver, born out of such a solipsistic approach to AI design, is unlikely to be cooperative. Deploying AI systems induces endogenous non-stationarity, resulting in a train-test-deploy gap where historical distributions diverge from the deployment context. We refer to this as the self-undermining property of unilateral optimization. Closing this gap requires AI that participates in cooperation: the equilibrium-selection process through which multiple actors navigate their interdependence. We call for a non-solipsistic research paradigm that treats this interdependence as a core design principle rather than approaching cooperation as a task to solve. This entails building dynamic evaluation testbeds involving adaptive counterparties, treating institutions as design primitives, and preserving human agency as a structural feature of the systems we build.

Chinese Translation

人工智能的核心挑战正在从能力转向共存。当前人工智能研究的主导范式集中于开发将世界视为外生且静态反馈源的强大智能体。我们认为，基于这种自我中心的人工智能设计方法产生的超级智能，作为一种极具能力的任务解决者，不太可能具备合作性。部署人工智能系统会引发内生性非静态性，导致训练-测试-部署之间的差距，即历史分布与部署环境之间的偏离。我们将其称为单边优化的自我削弱特性。弥合这一差距需要参与合作的人工智能：即多个参与者在相互依赖中进行的均衡选择过程。我们呼吁一种非自我中心的研究范式，将这种相互依赖视为核心设计原则，而不是将合作视为一个待解决的任务。这意味着需要构建涉及适应性对手的动态评估测试平台，将机构视为设计原始元素，并将人类代理性作为我们构建的系统的结构特征。

View on arXiv Download PDF AI Translation

cs.AI / 37 / 2606.03251

Do Real-World Datasets Contain Natural Experiments? An Empirical Study Using Causal Feature Selection

真实世界数据集是否包含自然实验？基于因果特征选择的实证研究

Gare, Gautam, Galeotti, John, Mozer, Michael, Ramanan, Deva, Ke, Nan Rosemary

Abstract

In nature, events that affect some individuals or groups but not others constitute an implicit intervention and are known as natural experiments. For example, the COVID-19 pandemic was an intervention by the coronavirus on the sub-population infected with COVID. We ask, do natural experiments occur in existing real-world datasets? If yes, how should we treat them? To detect natural experiments in data, we use causal discovery to recover the underlying causal graph and perform feature selection based on causal links. If downstream performance improves by treating the data as interventional rather than observational, we argue that this suggests the dataset contains natural experiments. We first validate this hypothesis by simulating datasets with and without natural experiments using synthetic graphs. We then perform a systematic empirical evaluation on a large suite of real-world datasets. Our results indicate that real-world datasets do contain natural experiments and we can take advantage of those natural experiments to improve model performance using causal inference. Our work represents the initial foray into this area, offering a preliminary exploration within a limited scope.

Chinese Translation

在自然界中，影响某些个体或群体而不影响其他个体或群体的事件构成了隐含的干预，被称为自然实验。例如，COVID-19大流行是冠状病毒对感染COVID的亚人群的一次干预。我们提出问题：现有的真实世界数据集中是否存在自然实验？如果存在，我们应该如何处理它们？为了在数据中检测自然实验，我们使用因果发现技术来恢复潜在的因果图，并基于因果联系进行特征选择。如果通过将数据视为干预性而非观察性来提高下游性能，我们认为这表明数据集包含自然实验。我们首先通过使用合成图模拟包含和不包含自然实验的数据集来验证这一假设。然后，我们对大量真实世界数据集进行了系统的实证评估。我们的结果表明，真实世界数据集确实包含自然实验，并且我们可以利用这些自然实验通过因果推断来提高模型性能。我们的工作代表了这一领域的初步探索，在有限的范围内提供了初步的研究。

View on arXiv Download PDF AI Translation

cs.AI / 38 / 2606.03269

Distilling Answer-Set Programming Rules from LLMs for Neurosymbolic Visual Question Answering

从大型语言模型中提炼答案集编程规则以实现神经符号视觉问答

Eiter, Thomas, Ruiz, Nelson Higuera, Oetsch, Johannes

Abstract

Visual Question Answering (VQA) is the task of answering questions about images, requiring the integration of multimodal input and reasoning. Modular approaches that incorporate logic-based representations into the reasoning component offer clear advantages over end-to-end trained systems, particularly in terms of interpretability. However, adapting or extending these representations when task requirements change can place a significant burden on developers. To address this challenge, we present an approach for distilling rules from Large Language Models (LLMs). Our method prompts an LLM to extend an initial VQA reasoning theory, expressed as an answer-set program, to meet new requirements of the task. Examples from VQA datasets guide the LLM, validate the results, and help correct erroneous rules by leveraging feedback from the ASP solver. We demonstrate that our approach is effective across diverse VQA datasets. Notably, only a few examples are needed to elicit correct rules from LLMs. Our experiments suggest that rule distillation from LLMs is a promising alternative to traditional data-driven rule learning approaches. Under consideration in Theory and Practice of Logic Programming (TPLP).

Chinese Translation

视觉问答（VQA）是回答关于图像的问题的任务，要求整合多模态输入和推理。将基于逻辑的表示方法纳入推理组件的模块化方法在可解释性方面相较于端到端训练系统具有明显优势。然而，当任务需求发生变化时，适应或扩展这些表示可能会给开发者带来重大负担。为了解决这一挑战，我们提出了一种从大型语言模型（LLMs）中提炼规则的方法。我们的方法促使LLM扩展初始的VQA推理理论，该理论以答案集程序的形式表达，以满足任务的新要求。来自VQA数据集的示例指导LLM，验证结果，并通过利用ASP求解器的反馈帮助纠正错误规则。我们展示了我们的方法在多样化的VQA数据集上是有效的。值得注意的是，仅需少量示例即可从LLMs中引出正确的规则。我们的实验表明，从LLMs中提炼规则是传统数据驱动规则学习方法的一个有前景的替代方案。

View on arXiv Download PDF AI Translation

cs.AI / 39 / 2606.03280

A Negative Result on Cross-Model Activation Transfer in a Pythia Multi-Hop Setting

关于Pythia多跳设置中跨模型激活传递的负结果

Zhang, Peiyan

Abstract

Recent work shows that language models can transmit behavioural traits through hidden signals in generated data during training. We ask whether a more direct and stricter channel is also viable: can one language model communicate useful intermediate reasoning state to another at inference time by translating and injecting hidden activations, rather than by passing natural-language text? We test this question in a controlled Pythia-160M to Pythia-410M multi-hop reasoning setting. A linear translation layer learns a strong normalized-space map between sender and receiver hidden states, with normalized cosine similarity near 0.97 across seeds. However, when the translated activations are injected into the receiver at inference time, they do not improve downstream answering. Low-strength additive injection remains near the no-injection baseline, with confidence intervals that cross zero. Replacement-style injection is consistently destructive, and rescaling translated vectors to the receiver hidden-state norm does not rescue performance. The result is therefore a scoped negative result: in this setting, offline representational alignment is not sufficient for useful causal communication inside the receiver.

Chinese Translation

近期研究表明，语言模型可以通过训练过程中生成数据中的隐藏信号传递行为特征。我们探讨是否存在一种更直接且严格的通道：一个语言模型是否可以在推理时通过翻译和注入隐藏激活状态，而不是通过传递自然语言文本，向另一个语言模型传达有用的中间推理状态？我们在一个受控的Pythia-160M到Pythia-410M的多跳推理设置中测试了这个问题。一个线性翻译层学习了发送者和接收者隐藏状态之间的强归一化空间映射，在不同种子下的归一化余弦相似度接近0.97。然而，当翻译后的激活在推理时注入到接收者中时，并没有改善下游回答。低强度的加性注入接近于无注入基线，置信区间穿越零。替换式注入始终具有破坏性，而将翻译向量重新缩放到接收者隐藏状态的范数并没有挽救性能。因此，结果是一个有范围的负结果：在这个设置中，离线表示对齐不足以实现接收者内部有用的因果通信。

View on arXiv Download PDF AI Translation

cs.AI / 40 / 2606.03303

LEAP: Supercharging LLMs for Formal Mathematics with Agentic Frameworks

LEAP：通过代理框架为形式数学赋能大型语言模型

Kung, Po-Nien, Song, Linfeng, Hwang, Dawsen, Yoon, Jinsung, Li, Chun-Liang, Severini, Simone, Olšák, Mirek, Lockhart, Edward, Le, Quoc V, Gokturk, Burak, Luong, Thang, Pfister, Tomas, Peng, Nanyun

Abstract

Large Language Models (LLMs) exhibit strong informal mathematical reasoning but struggle to generate mechanically verifiable proofs in formal languages like Lean. We present LEAP, an agentic framework that enables general-purpose foundation models to achieve state-of-the-art performance on automated formal theorem proving. LEAP leverages foundation model capabilities, such as informal reasoning, instruction following, and iterative self-refinement. By decomposing complex problems into smaller units, the system bridges formal proof construction with informal blueprints through continuous interaction with the Lean compiler. To provide a rigorous evaluation beyond increasingly saturated benchmarks, we introduce Lean-IMO-Bench, a benchmark of IMO-style problems formalized in Lean, with short statements yet highly non-routine and multi-step proofs across a wide range of difficulty levels. Empirically, on the latest 2025 Putnam Competition, an annual mathematics competition for undergraduate students in North America, LEAP solves all 12 problems, matching recent breakthroughs by frontier formal mathematical models. On Lean-IMO-Bench, LEAP boosts the one-shot formal solve rate of general-purpose LLMs from below 10% to 70%, notably surpassing the 48% benchmark set by a specialized, gold-medal-caliber IMO system. Furthermore, we demonstrate LEAP's research-level utility by autonomously formalizing complex proofs for open combinatorial challenges, including a verified proof for a key subproblem in Knuth's Hamiltonian decomposition of even-order Cayley graphs.

Chinese Translation

大型语言模型（LLMs）在非正式数学推理方面表现出色，但在生成可机械验证的形式语言证明（如 Lean）时却面临挑战。我们提出了 LEAP，这是一种代理框架，使通用基础模型能够在自动化形式定理证明方面实现最先进的性能。LEAP 利用基础模型的能力，如非正式推理、指令跟随和迭代自我完善。通过将复杂问题分解为更小的单元，该系统通过与 Lean 编译器的持续交互，将形式证明构建与非正式蓝图连接起来。为了提供超越日益饱和基准的严格评估，我们引入了 Lean-IMO-Bench，这是一个在 Lean 中形式化的 IMO 风格问题基准，具有简短的陈述但包含高度非例行和多步骤的证明，涵盖广泛的难度级别。在最新的 2025 年普特南竞赛（Putnam Competition）中，LEAP 解决了所有 12 个问题，匹配了前沿形式数学模型的最新突破。在 Lean-IMO-Bench 上，LEAP 将通用 LLM 的一次性形式解决率从低于 10% 提升至 70%，显著超过了由专门的金牌级 IMO 系统设定的 48% 基准。此外，我们通过自主形式化复杂证明，展示了 LEAP 的研究级实用性，解决了开放组合挑战中的关键子问题，包括对 Knuth 的偶数阶 Cayley 图的哈密尔顿分解的验证证明。

View on arXiv Download PDF AI Translation

cs.AI / 41 / 2606.03305

The Reliability Gap in Benchmark Auditing: Distribution Shift and Scale as Failure Modes of Contamination Detection

基准审计中的可靠性差距：分布偏移和规模作为污染检测的失效模式

Zarzecki, Wojciech, Dubiński, Jan, Cygert, Sebastian

Abstract

Benchmark contamination, where evaluation examples appear in a model's training data, threatens the validity of LLM assessment. Statistical tools for detecting training-data membership exist, but have been validated almost exclusively in controlled academic regimes: large, homogeneous pre-training corpora and transparent, single-stage training pipelines. Whether these methods remain reliable in realistic auditing scenarios remains unclear. We identify two under-studied failure modes: distribution shift, which arises when suspect and validation sets violate the IID assumption, and scale constraints, which arise because benchmarks are orders of magnitude smaller than pre-training corpora. We systematically evaluate three leading paradigms: LLM Dataset Inference, Post-Hoc Dataset Inference, and CoDeC across 27 models from multiple families (including Pythia, OLMo~2, and specialised cultural and medical LLMs) and scales (up to 27B). We then further extend our analysis to frontier industry models. Across 335 evaluations, only 199 yield correct outcomes. LLM Dataset Inference results in false positives under distribution shift, Post-Hoc Dataset Inference is underpowered at benchmark scale, and CoDeC provides only coarse provenance signals that are insufficient to verify individual benchmark splits. Our results reveal a systematic reliability gap between controlled validation and practical benchmark auditing, and show that statistical detection cannot yet replace transparent data provenance. We open-source our benchmark for further research.

Chinese Translation

基准污染，即评估示例出现在模型的训练数据中，威胁到大规模语言模型（LLM）评估的有效性。虽然存在用于检测训练数据成员资格的统计工具，但这些工具几乎仅在受控的学术环境中得到验证：大型、同质的预训练语料库和透明的单阶段训练流程。这些方法在现实审计场景中的可靠性尚不明确。我们识别出两种研究不足的失效模式：分布偏移，当可疑集和验证集违反独立同分布（IID）假设时产生；以及规模限制，因为基准的规模比预训练语料库小几个数量级。我们系统地评估了三种主要范式：LLM数据集推断（LLM Dataset Inference）、后验数据集推断（Post-Hoc Dataset Inference）和CoDeC，在来自多个家族（包括Pythia、OLMo~2以及专门的文化和医学LLM）和规模（最大达到27B）的27个模型上进行评估。随后，我们进一步将分析扩展到前沿行业模型。在335次评估中，仅有199次产生正确结果。LLM数据集推断在分布偏移下导致假阳性，后验数据集推断在基准规模下能力不足，而CoDeC仅提供粗略的来源信号，无法验证单个基准拆分。我们的结果揭示了受控验证与实际基准审计之间的系统可靠性差距，并表明统计检测尚不能替代透明的数据来源。我们将我们的基准开源以供进一步研究。

View on arXiv Download PDF AI Translation

cs.AI / 42 / 2606.03326

The Violation Situation Pattern: A Knowledge-Graph Pattern for Compliance Violations

违反情况模式：一种用于合规违规的知识图谱模式

Lassem, Nima Kamali, Song, Fuqi, Ali, Seyid Amjad

Abstract

Compliance pipelines detect violations as transient query results and do not keep the violation itself as a persistent graph object with review state, affected entities, or audit history. The Violation Situation Pattern (VSP) closes this gap. Building on the Situation pattern of Gangemi and Mika, VSP reifies each detected violation as a graph node with a rule identifier, a temporal validity interval, a lifecycle state, and evidence links to the entities involved. Lifecycle transitions are stored as immutable, PROV-O-aligned events, so audit history is a graph traversal. We instantiate VSP in a legal entity and contract lifecycle property graph and operationalize four deontic rules (V1 unauthorized signature, V2 expired mandate, V3 missing confidentiality clause, V4 missing breach-notification clause) through an FCL->Cypher->MERGE pipeline. We check V1 and V2 against BODACC corporate-officer publications, evaluate V4 on 73 GDPRhub enforcement decisions, and run a SHACL cross-formalism check on V3 and V4. The central finding is rule-body independence: extending V4 from clause-presence to deadline checking raises F1 from 0.312 to 0.602, while the pattern's identity, lifecycle, and evidence semantics stay the same. This separates a pattern contribution from a detector contribution, so detection logic can evolve without invalidating accumulated audit history.

Chinese Translation

合规管道将违规行为视为瞬态查询结果进行检测，而不将违规本身作为具有审查状态、受影响实体或审计历史的持久图对象。违反情况模式（Violation Situation Pattern，VSP）填补了这一空白。VSP 基于 Gangemi 和 Mika 的情况模式，将每个检测到的违规行为具体化为一个图节点，包含规则标识符、时间有效性区间、生命周期状态以及与相关实体的证据链接。生命周期转换作为不可变的、与 PROV-O 对齐的事件存储，因此审计历史可以通过图遍历进行访问。我们在法律实体和合同生命周期属性图中实例化 VSP，并通过 FCL->Cypher->MERGE 管道对四个义务规则（V1 未经授权的签名，V2 过期的授权，V3 缺失的保密条款，V4 缺失的违约通知条款）进行操作。我们对 BODACC 企业官员公告检查 V1 和 V2，评估 V4 在 73 个 GDPRhub 执法决定上的表现，并对 V3 和 V4 进行 SHACL 跨形式检查。核心发现是规则主体的独立性：将 V4 从条款存在扩展到截止日期检查，使 F1 从 0.312 提升至 0.602，而模式的身份、生命周期和证据语义保持不变。这将模式贡献与检测器贡献分开，因此检测逻辑可以演变而不影响已积累的审计历史。

View on arXiv Download PDF AI Translation

cs.AI / 43 / 2606.03329

InfoMem: Training Long-Context Memory Agents with Answer-Conditioned Information Gain

InfoMem：通过答案条件的信息增益训练长上下文记忆代理

Han, Tiancheng, Li, Yong, Yu, Wuzhou, Zhang, Qiaosheng, Shao, Wenqi

Abstract

Long-context tasks require LLMs to identify and preserve answer-relevant information from large contexts. Chunk-wise memory agents address this issue by sequentially reading document chunks, updating a compact memory, and generating the final answer from the accumulated memory. However, existing RL-based chunk-wise agents either rely on sparse final-answer rewards or use lexical intermediate rewards for memory and retrieval actions. These signals supervise task success or local overlap, but do not directly evaluate whether the final memory supports the ground-truth answer. We propose InfoMem, a reward mechanism for training chunk-wise memory agents that evaluates final-memory utility using answer-conditioned information. InfoMem measures how much the final memory increases the model's per-token log-likelihood of the ground-truth answer. To stabilize RL optimization, InfoMem applies this signal only to successful trajectories and normalizes it before reward composition. Under the same GRPO framework and training budget, InfoMem improves long-context memory-agent performance over comparable memory-agent RL baselines. Analyses show that effective final-memory rewards should operate on successful trajectories, be normalized before reward composition, and be conditioned on the answer rather than the query. Our code is available at https://github.com/GenSouKa1/InfoMem.

Chinese Translation

长上下文任务要求大型语言模型（LLMs）从大上下文中识别和保留与答案相关的信息。分块记忆代理通过顺序读取文档块、更新紧凑的记忆，并从累积的记忆中生成最终答案来解决这个问题。然而，现有的基于强化学习（RL）的分块代理要么依赖稀疏的最终答案奖励，要么使用词汇中间奖励来进行记忆和检索操作。这些信号监督任务成功或局部重叠，但并未直接评估最终记忆是否支持真实答案。我们提出了InfoMem，一种用于训练分块记忆代理的奖励机制，它通过答案条件的信息来评估最终记忆的效用。InfoMem衡量最终记忆在多大程度上提高了模型对真实答案的每个标记的对数似然。为了稳定强化学习优化，InfoMem仅将该信号应用于成功轨迹，并在奖励组合之前对其进行归一化。在相同的GRPO框架和训练预算下，InfoMem提高了长上下文记忆代理的性能，超越了可比的记忆代理强化学习基线。分析表明，有效的最终记忆奖励应在成功轨迹上操作，在奖励组合之前进行归一化，并以答案为条件而非查询。我们的代码可在 https://github.com/GenSouKa1/InfoMem 获取。

View on arXiv Download PDF AI Translation

cs.AI / 44 / 2606.03435

CP-Agent: Context-Aware Multimodal Reasoning for Cellular Morphological Profiling under Chemical Perturbations

CP-Agent：化学干扰下细胞形态特征分析的上下文感知多模态推理

Zhang, Yuxin, Li, Yiyao, Ho, Ping Shu, See, Simon, Wu, Zhenqin, Tsia, Kevin

Abstract

Cell Painting combines multiplexed fluorescent staining, high-content imaging, and quantitative analysis to generate high-dimensional phenotypic readouts to support diverse downstream tasks such as mechanism-of-action (MoA) inference, toxicity prediction, and construction of drug-disease atlases. However, existing workflows are slow, costly and difficult to interpret. Approaches for drug screening modeling predominantly focus on molecular representation learning, while neglecting actual experimental context (e.g., cell line, dosing schedule, etc.), limiting generalization and MoA resolution. We introduce CP-Agent, an agentic multimodal large language model (MLLM) capable of generating mechanism-relevant, human-interpretable rationales for cell morphological changes under drug perturbations. At its core, CP-Agent leverages a context-aware alignment module, CP-CLIP, that jointly embeds high-content images and experimental metadata to enable robust treatment and MoA discrimination (achieving a maximum F1-score of 0.896). By integrating CP-CLIP outputs with agentic tool usage and reasoning, CP-Agent compiles rationales into a structured report to guide experimental design and hypothesis refinement. These capabilities highlight CP-Agent's potential to accelerate drug discovery by enabling more interpretable, scalable, and context-aware phenotypic screening -- streamlining iterative cycles of hypothesis generation in drug discovery.

Chinese Translation

细胞绘画（Cell Painting）结合了多重荧光染色、高内容成像和定量分析，以生成高维表型读数，支持多种下游任务，如作用机制（MoA）推断、毒性预测和药物-疾病图谱构建。然而，现有工作流程速度慢、成本高且难以解释。药物筛选建模的方法主要集中在分子表征学习上，而忽视了实际实验背景（例如，细胞系、给药方案等），限制了推广性和作用机制的解析。我们提出了CP-Agent，一种能够为药物干扰下细胞形态变化生成与机制相关的人类可解释推理的代理多模态大型语言模型（MLLM）。CP-Agent的核心是一个上下文感知对齐模块CP-CLIP，该模块联合嵌入高内容图像和实验元数据，以实现稳健的处理和作用机制区分（最大F1-score达到0.896）。通过将CP-CLIP的输出与代理工具的使用和推理相结合，CP-Agent将推理汇编成结构化报告，以指导实验设计和假设优化。这些能力突显了CP-Agent在加速药物发现方面的潜力，使得表型筛选更加可解释、可扩展和上下文感知，从而简化药物发现中的假设生成迭代周期。

View on arXiv Download PDF AI Translation

cs.AI / 45 / 2606.03461

What Makes Interaction Trajectories Effective for Training Terminal Agents?

什么使得交互轨迹在训练终端代理时有效？

Yang, Sidi, Tao, Chaofan, Chen, Jierun, Yu, Tiezheng, Wang, Ruoyu, Jiang, Yuxin, Du, Yiming, Xu, Wendong, Xiong, Jing, Wu, Taiqiang, Shang, Lifeng, Li, Xiaohui, Wong, Ngai, Bai, Haoli

Abstract

Stronger code agents are commonly assumed to be superior teachers for post-training, yet this assumption remains poorly disentangled from task difficulty, harness design, and student capacity. We investigate this pedagogical link using Terminal-Lego, a scalable pipeline that transforms multi-domain real-world issues into environment-verified agentic tasks. Surprisingly, standalone performance does not dictate teaching efficacy: while Claude Opus 4.6 achieves higher scores on Terminal-Bench 2.0, students fine-tuned on trajectories from DeepSeek-V3.2, a lower-scoring agent, exhibit significantly stronger generalization. We attribute this "pedagogical paradox" to Environment-Grounded Supervision (EGS): trajectories that explicitly expose inspect-act-verify behaviors through harness-visible interactions allow students to internalize robust problem-solving routines rather than fragile action sequences. Scaling analysis reveals exceptional data efficiency: with only 15.3k Terminal-Lego trajectories, for example, Qwen3-32B achieves a 24.3% score on Terminal-Bench 2.0, rivaling previous SOTA performance established with over 30x the data volume. Our results suggest that the frontier of agent post-training lies beyond mere outcome-matching, shifting the focus toward "Harness Engineering", where the systematic design of environment-grounded interaction structures serves as the primary catalyst for reproducible and generalizable agentic intelligence.

Chinese Translation

更强大的代码代理通常被认为是后训练阶段更优秀的教师，然而这一假设与任务难度、工具设计和学生能力之间的关系仍然不够清晰。我们使用 Terminal-Lego 这一可扩展管道来研究这种教学联系，该管道将多领域的现实问题转化为经过环境验证的代理任务。令人惊讶的是，单独的性能并不决定教学效果：尽管 Claude Opus 4.6 在 Terminal-Bench 2.0 上获得了更高的分数，但在 DeepSeek-V3.2（一个得分较低的代理）上微调的学生在泛化能力上表现出显著更强的能力。我们将这种“教学悖论”归因于环境基础监督（Environment-Grounded Supervision, EGS）：那些通过可见的交互明确暴露检查-行动-验证行为的轨迹，使学生能够内化稳健的问题解决常规，而不是脆弱的行动序列。规模分析显示出卓越的数据效率：例如，仅凭 15.3k 个 Terminal-Lego 轨迹，Qwen3-32B 在 Terminal-Bench 2.0 上达到了 24.3% 的分数，媲美以超过 30 倍数据量建立的先前最优性能。我们的结果表明，代理后训练的前沿不仅仅在于结果匹配，而是将焦点转向“工具工程”，在这里，系统设计的环境基础交互结构成为可重复和可泛化的代理智能的主要催化剂。

View on arXiv Download PDF AI Translation

cs.AI / 46 / 2606.03463

DMF: A Deterministic Memory Framework for Conversational AI Agents

DMF：一种用于对话式人工智能代理的确定性记忆框架

Stabile, Matteo, Zimuel, Enrico

Abstract

Conversational AI agents require memory systems that are both scalable and semantically coherent across long interaction horizons. Existing approaches rely predominantly on large language model (LLM)-based summarisation at write time, which introduces non-determinism, escalating token costs, and opacity in pruning decisions. We present the Deterministic Memory Framework (DMF), a CPU-first approach that replaces generative memory compression with a fully deterministic pipeline grounded in classical NLP analysis, vector geometry, and mathematical scoring. DMF assigns each conversational interaction a Survival Score $\Omega$ computed from deterministic content signals, conversational cues, and structured provenance, combined through a logistic projection. An interaction-count decay law, denoted as $\Omega_{\mathrm{eff}}(\Delta n)$, governs how relevance evolves as new turns arrive, where $\Delta n$ is the number of newer interactions rather than wall-clock time, preserving full determinism. We present the mathematical formulation of DMF, its structured recall pipeline, the pruning decision procedure, and the evaluation protocol. Experiments are conducted on a purpose-built benchmark using the LoCoMo and LongMemEval datasets. We compare DMF against Mem0, a popular memory layer for AI agents. DMF achieves comparable accuracy while using zero tokens to prepare the memory context and 5x to 242x fewer tokens over the entire conversation. These results show that it is possible to eliminate LLM calls from the memory-management loop, reducing token costs to nearly zero and enabling deterministic memory systems for conversational AI agents.

Chinese Translation

对话式人工智能代理需要在长时间交互中既可扩展又语义一致的记忆系统。现有的方法主要依赖于基于大型语言模型（LLM）的写入时摘要，这引入了非确定性、增加了令牌成本，并使修剪决策变得不透明。我们提出了确定性记忆框架（DMF），这是一种以CPU为主的方案，使用基于经典自然语言处理分析、向量几何和数学评分的完全确定性管道来替代生成性记忆压缩。DMF为每个对话交互分配一个生存评分$ ext{Ω}$，该评分是根据确定性内容信号、对话提示和结构化来源计算得出的，并通过逻辑投影进行组合。一个交互计数衰减法则，记作$ ext{Ω}_{ ext{eff}}( ext{Δ} n)$，控制着随着新回合的到来相关性如何演变，其中$ ext{Δ} n$是较新交互的数量，而不是墙钟时间，从而保持完全的确定性。我们展示了DMF的数学公式、其结构化回忆管道、修剪决策程序和评估协议。实验在一个专门构建的基准上进行，使用LoCoMo和LongMemEval数据集。我们将DMF与Mem0进行比较，后者是一个流行的人工智能代理记忆层。DMF在使用零个令牌准备记忆上下文的同时，整个对话中使用的令牌数量减少了5倍到242倍，达到了可比的准确性。这些结果表明，可以从记忆管理循环中消除LLM调用，将令牌成本降低到几乎为零，并为对话式人工智能代理实现确定性记忆系统。

View on arXiv Download PDF AI Translation

cs.AI / 47 / 2606.03467

StepFinder: A Temporal Semantic Framework for Failure Attribution in Multi-Agent Systems

StepFinder：用于多智能体系统故障归因的时间语义框架

Zhu, Taiyu, Wu, Yifan, Jin, Weilin, Li, Ying, Huang, Gang

Abstract

LLM-based multi-agent systems exhibit remarkable collaborative capabilities in complex multi-step tasks. However, these systems are highly sensitive to single-step execution errors that can propagate through agent interactions and lead to cascading failures. To understand the causes of failure and improve system reliability, failure attribution has been introduced as a task that aims to automatically identify the root cause step responsible for a failure. Existing failure attribution methods mainly rely on LLMs to reason over original execution trajectories, which not only incur high inference costs and latency, but also suffer from interference caused by redundant and noisy execution logs, causing LLMs to struggle in accurately identifying the true root cause step. To address this, we propose StepFinder, a lightweight failure attribution framework. We use LLMs solely during the feature construction phase to encode execution logs into temporal semantic sequences. Subsequently, a parameter-efficient combination of temporal modeling and attention modules is applied to capture the sequential evolution and cross-step dependencies of the trajectories. Finally, the step-level error score is refined through multi-scale differences and position bias, enabling precise root cause identification. Experimental results on the Who&When benchmark demonstrate that StepFinder outperforms LLM-based methods in step-level failure attribution while achieving substantially higher inference efficiency, reducing inference time by 79% compared with the fastest LLM-based method, with no text generation overhead. Our code is available at https://github.com/taiyu-zhu/StepFinder.

Chinese Translation

基于大语言模型（LLM）的多智能体系统在复杂的多步骤任务中展现出显著的协作能力。然而，这些系统对单步执行错误高度敏感，这些错误可能通过智能体之间的交互传播并导致级联故障。为了理解故障原因并提高系统可靠性，故障归因被引入作为一项任务，旨在自动识别导致故障的根本原因步骤。现有的故障归因方法主要依赖于LLM对原始执行轨迹进行推理，这不仅会产生高昂的推理成本和延迟，还会受到冗余和噪声执行日志的干扰，导致LLM在准确识别真实根本原因步骤时面临困难。为了解决这个问题，我们提出了StepFinder，一个轻量级的故障归因框架。我们仅在特征构建阶段使用LLM，将执行日志编码为时间语义序列。随后，应用一种参数高效的时间建模和注意力模块组合，以捕捉轨迹的顺序演变和跨步骤依赖关系。最后，通过多尺度差异和位置偏差精炼步骤级错误评分，从而实现精确的根本原因识别。在Who&When基准上的实验结果表明，StepFinder在步骤级故障归因方面优于基于LLM的方法，同时实现了显著更高的推理效率，与最快的基于LLM的方法相比，推理时间减少了79%，且没有文本生成的开销。我们的代码可在https://github.com/taiyu-zhu/StepFinder获取。

View on arXiv Download PDF AI Translation

cs.AI / 48 / 2606.03471

A formal definition and meta-model for a machine theory of mind

机器心智理论的正式定义与元模型

Cuzzolin, Fabio

Abstract

This paper proposes, for the first time, a rigorous formal definition of the concept of Machine Theory of Mind, based on principles supported by evidence from cognitive psychology, neuroscience and artificial intelligence, and uses the above as a lens to examine state-of-the-art and current efforts in the field, driving a potential agenda for further research there able to "crack" the problem. It also advances a general holistic meta-model for Machine Theory of Mind, and examines the state of the art when it comes to empirically benchmarking such models.

Chinese Translation

本文首次提出了机器心智理论（Machine Theory of Mind）概念的严格正式定义，该定义基于来自认知心理学、神经科学和人工智能的证据支持的原则，并以此为视角审视该领域的最新进展和当前努力，推动进一步研究的潜在议程，以“破解”这一问题。同时，本文还提出了机器心智理论的一般整体元模型，并考察了在经验基准测试此类模型方面的最新进展。

View on arXiv Download PDF AI Translation

cs.AI / 49 / 2606.03503

ThoughtFold: Folding Reasoning Chains via Introspective Preference Learning

ThoughtFold：通过内省偏好学习折叠推理链

Liu, Ziyan, Shen, Xueda, Gu, Yuzhe, Gao, Songyang, Liu, Kuikun, Cheng, Guangran, Lyu, Chengqi, Lin, Dahua, Zhang, Wenwei, Chen, Kai

Abstract

Large Reasoning Models (LRMs) have achieved remarkable progress thanks to Reinforcement Learning with Verifiable Rewards (RLVR) on Chain-of-Thoughts (CoTs). However, since long CoTs naturally contain trial and errors and mainstream RLVR approaches choose outcome-correct CoT trajectories for memorization, the redundant explorations in long CoTs are inevitably reinforced, which results in the over-thinking issues of LRMs. Previous attempts to resolve this issue mainly give more advantage to shorter trajectories, yet their learning signals are still outcome-based and cannot reduce the memorization of redundant explorations in long CoTs. Therefore, we propose ThoughtFold, a framework that leverages fine-grained preference learning to mitigate redundant explorations for efficient reasoning. ThoughtFold employs an introspective strategy to identify redundancy within each correct trajectory, which yields a spectrum of candidate sub-trajectories. Leveraging this spectrum, we introduce a masked preference optimization objective that explicitly penalizes redundant explorations and encourages the model to directly bridge essential reasoning segments, effectively folding its reasoning chains into a more concise path. Extensive experiments show that ThoughtFold significantly enhances efficiency. It reduces the token usage of DeepSeek-R1-Distill-Qwen-7B by approximately 56% while maintaining state-of-the-art accuracy.

Chinese Translation

大型推理模型（LRMs）得益于可验证奖励的强化学习（RLVR）在思维链（CoTs）上的应用，取得了显著进展。然而，由于长思维链自然包含试错过程，而主流的RLVR方法选择结果正确的思维链轨迹进行记忆，导致长思维链中的冗余探索不可避免地被强化，从而导致LRMs的过度思考问题。之前的尝试主要是给予较短轨迹更多的优势，但其学习信号仍然是基于结果的，无法减少长思维链中冗余探索的记忆。因此，我们提出了ThoughtFold，一个利用细粒度偏好学习来减轻冗余探索以实现高效推理的框架。ThoughtFold采用内省策略来识别每个正确轨迹中的冗余，从而生成一系列候选子轨迹。利用这一系列候选轨迹，我们引入了一种掩蔽偏好优化目标，明确惩罚冗余探索，并鼓励模型直接连接重要的推理片段，有效地将其推理链折叠成更简洁的路径。大量实验表明，ThoughtFold显著提高了效率。它将DeepSeek-R1-Distill-Qwen-7B的令牌使用量减少了约56%，同时保持了最先进的准确性。

View on arXiv Download PDF AI Translation

cs.AI / 50 / 2606.03518

Overlaying Governance: A Compositional Authorization Framework for Delegation and Scope in Agentic AI

叠加治理：一种用于代理人工智能中委托与范围的组合授权框架

Ibrahim, Amjad, Li, Yong

Abstract

As AI systems evolve from passive models into autonomous active agents capable of initiating actions, collaborating, and delegating tasks, the traditional boundaries of software systems blur. Traditional authorization and delegation frameworks, built around fixed principals, explicit requests, and static scopes, are insufficient to govern agentic systems. Agentic AI demands richer authorization semantics: agents must inherit and delegate permissions, act under time-limited authority, and coordinate through shared protocols. Existing Identity and Access Management (IAM) systems fail to fully capture this notion of agency, lacking mechanisms for recursive delegation, contextual boundaries, and dynamic scoping as executable governance primitives. Unlike access delegation standards such as OAuth 2.0, we treat delegation as a contractual term rather than merely a static token-based consent credential. This paper proposes a compositional governance framework that introduces primitives indispensable for agentic AI. We define types of delegation and their permissions and accountability implications, and we introduce a notion of resource scope attenuation to bound agentic access envelopes. These concepts are expressed as general relational definitions that can be composed into existing authorization domains (e.g., financial systems). To operationalize this composition, we define a compositional operator that overlays new agentic semantics, such as recursive delegation chains, onto existing relational policies without rewriting them. We substantiate this framework through formal proofs and empirical evaluation, showing that it provides a formal yet practical foundation for accountable authorization in agentic AI systems.

Chinese Translation

随着人工智能系统从被动模型演变为能够主动发起行动、协作和委托任务的自主主动代理，软件系统的传统边界变得模糊。围绕固定主体、明确请求和静态范围构建的传统授权和委托框架，无法有效治理主动代理系统。主动人工智能要求更丰富的授权语义：代理必须继承和委托权限，在时间限制的授权下行动，并通过共享协议进行协调。现有的身份和访问管理（IAM）系统未能充分捕捉这一代理概念，缺乏递归委托、上下文边界和动态范围作为可执行治理原语的机制。与 OAuth 2.0 等访问委托标准不同，我们将委托视为一种合同条款，而不仅仅是静态的基于令牌的同意凭证。本文提出了一种组合治理框架，引入了主动人工智能不可或缺的原语。我们定义了委托的类型及其权限和责任的影响，并引入了资源范围减弱的概念，以限制主动访问的边界。这些概念以一般关系定义的形式表达，可以组合到现有的授权领域（例如，金融系统）中。为了实现这种组合，我们定义了一种组合运算符，将新的主动语义（如递归委托链）叠加到现有的关系政策上，而无需重写它们。我们通过形式证明和实证评估来证实这一框架，显示它为主动人工智能系统中的可问责授权提供了一个正式而实用的基础。

View on arXiv Download PDF AI Translation

cs.AI / 51 / 2606.03544

SAGE: A Quantitative Evaluation of Socialized Evolution in Agent Ecosystems

SAGE：代理生态系统中社会化进化的定量评估

Pan, Linyue, Zhu, Yaoming, Qiu, Lin, Cao, Xuezhi, Cai, Xunliang

Abstract

Self-improving language agents are typically evaluated in isolation: an agent attempts a task, receives feedback, and iteratively refines its own behavior. Yet agents increasingly operate alongside peers whose strategies and outcomes are publicly visible. This raises an under-studied question: when does shared experience produce improvements that self-improvement alone cannot achieve? We introduce SAGE (Social Agent Group Evolution),an evaluation framework that compares two compute-matched conditions: SocialEvo, where agents from five distinct model families co-evolve with access to all peers' histories; and SelfEvo, where each agent receives the same number of task attempts but sees only its own past, which is conventional in self-improving agent studies. We instantiate SAGE in three arenas: open-ended ML research, long-horizon economic planning, and strategic multiplayer play, evaluated across multiple evolutionary rounds. We find that group history is not a universal amplifier: the strongest agent does not exceed its self-evolution ceiling. However, agents that plateau under self-improvement can achieve significant breakthroughs when peer experience is available. In competitive settings, counterfactual controls reveal that agents improve generally rather than developing opponent-specific strategies. Across different forms of shared history, filtered peer traces and reflective summaries often outperform raw logs, indicating that social gains depend on abstraction rather than exposure volume. These findings reveal that peer-history gains are agent-specific, arena-dependent, and contingent on the capacity to abstract transferable knowledge from public traces.

Chinese Translation

自我改进的语言代理通常在孤立状态下进行评估：代理尝试完成任务，接收反馈，并迭代地优化自身行为。然而，代理越来越多地与其同伴共同操作，这些同伴的策略和结果是公开可见的。这引出了一个尚未充分研究的问题：何时共享经验会产生自我改进所无法实现的提升？我们引入了SAGE（Social Agent Group Evolution），一个评估框架，用于比较两种计算匹配的条件：SocialEvo，在该条件下，来自五个不同模型家族的代理共同进化，并可以访问所有同伴的历史记录；以及SelfEvo，在该条件下，每个代理接收相同数量的任务尝试，但仅能看到自己的过去，这在自我改进代理研究中是常规做法。我们在三个领域中实例化了SAGE：开放式机器学习研究、长期经济规划和战略多人游戏，并在多个进化轮次中进行了评估。我们发现，群体历史并不是一种普遍的放大器：最强的代理并未超越其自我进化的上限。然而，在自我改进下停滞的代理在获得同伴经验时可以实现显著突破。在竞争环境中，反事实控制显示，代理普遍改进，而不是发展特定于对手的策略。在不同形式的共享历史中，过滤的同伴痕迹和反思性总结往往优于原始日志，表明社会收益依赖于抽象而非曝光量。这些发现揭示了同伴历史的收益是代理特定的、领域依赖的，并且取决于从公共痕迹中抽象可转移知识的能力。

View on arXiv Download PDF AI Translation

cs.AI / 52 / 2606.03557

From Prompt to Service: An SLM-Based Agent Orchestration Gateway for AI-Driven Virtual Worlds

从提示到服务：基于SLM的AI驱动虚拟世界代理编排网关

Nisiotis, Louis, Hadjiliasi, Aimilios

Abstract

As generative AI capabilities expand, AI-driven virtual worlds face a growing architectural challenge. Users interact through in-world interfaces in multimodal ways, yet their requests demand fundamentally different AI backend models and computational resources. Embedding these capabilities directly into virtual world systems reduces extensibility, complicates maintenance, and limits the ability to coordinate services distributed across edge and cloud infrastructure. This paper presents an SLM-based Agent Orchestration Gateway, a lightweight runtime coordination mechanism that decouples a virtual world client from heterogeneous AI backends through intent-driven service routing. An edge-deployed SLM classifies the semantic intent of each user prompt, a configurable service registry validates and resolves the routing decision, and the selected backend is invoked transparently, enabling new AI capabilities to be introduced in the virtual world without modifying the client application. The gateway is implemented and evaluated within the InterwovenXR virtual museum testbed. The evaluation shows that compact SLMs can serve as reliable intent routers on edge hardware, and that task-specific fine-tuning can transform sub-billion-parameter models into practical, low-latency routers. A layered configuration pairing a fine-tuned sub billion-parameter model as router with a larger SLM for conversational response generation is shown to be deployable on mid-range edge hardware and more efficient than delegating both responsibilities to a single model. The findings show that SLMs can support practical AI service orchestration in virtual worlds and the work contributes an evaluated architecture for scalable, extensible, and edge-supported AI interaction, enabling virtual agents become access points to distributed generative AI services.

Chinese Translation

随着生成性人工智能能力的扩展，AI驱动的虚拟世界面临日益增长的架构挑战。用户通过多模态的方式在虚拟世界界面中进行交互，但他们的请求需要根本不同的AI后端模型和计算资源。将这些能力直接嵌入虚拟世界系统会降低可扩展性，增加维护复杂性，并限制在边缘和云基础设施中协调分布式服务的能力。本文提出了一种基于SLM的代理编排网关，这是一种轻量级的运行时协调机制，通过基于意图的服务路由将虚拟世界客户端与异构AI后端解耦。边缘部署的SLM对每个用户提示的语义意图进行分类，一个可配置的服务注册表验证并解决路由决策，所选后端被透明调用，使得在不修改客户端应用程序的情况下，可以在虚拟世界中引入新的AI能力。该网关在InterwovenXR虚拟博物馆测试平台中实现并评估。评估结果表明，紧凑型SLM可以作为边缘硬件上的可靠意图路由器，任务特定的微调可以将不足十亿参数的模型转变为实用的低延迟路由器。将微调的不足十亿参数模型作为路由器与用于对话响应生成的较大SLM配对的分层配置被证明可以在中等性能的边缘硬件上部署，并且比将这两项职责委托给单一模型更为高效。研究结果表明，SLM可以支持虚拟世界中的实用AI服务编排，本文为可扩展、可扩展且支持边缘的AI交互贡献了一个经过评估的架构，使虚拟代理成为分布式生成性AI服务的接入点。

View on arXiv Download PDF AI Translation

cs.AI / 53 / 2606.03618

Cross-Lingual Token Arbitrage: Optimizing Code Agent Context Windows via Local LLM Preprocessing

跨语言令牌套利：通过本地大语言模型预处理优化代码代理上下文窗口

Colak, Mehmet Utku

Abstract

AI-assisted coding agents are bottlenecked by input-token cost. Two pathologies of raw human input drive much of this overhead: tokenization inefficiency for non-English text and structural entropy in conversational prompts. Existing approaches act reactively by compressing already-bloated contexts or intervening after failures occur. We introduce a pre-flight, edge-side prompt-rewriting middleware that operates between the developer and the cloud agent. A local Llama 3.2 (3B) model performs cross-lingual translation into English, structural rewriting into a compact task-oriented format, and regex-validated rewrite-with-fallback safeguards to ensure the optimized prompt is never larger than the original. We evaluate on OMH-Polyglot, a multilingual coding benchmark spanning Turkish, Arabic, Chinese, and code-switched specifications. Across three commercial LLM backends, the middleware reduces prompt tokens by 34-47 percent and total tokens by up to 18.8 percent while preserving or improving task accuracy. Ablation studies show that gains arise primarily from the rewriting stage rather than simple function-name extraction. Compared with LLMLingua-2 at matched compression rates, our method consistently achieves superior OckScore performance across all evaluated backends. These results demonstrate that proactive prompt optimization can substantially reduce inference costs without sacrificing coding quality.

Chinese Translation

人工智能辅助的编码代理受到输入令牌成本的制约。原始人类输入的两种病态驱动了大部分开销：非英语文本的令牌化低效和对话提示中的结构熵。现有方法通过压缩已经膨胀的上下文或在失败发生后进行干预，采取反应性的措施。我们提出了一种预处理的边缘侧提示重写中间件，位于开发者与云代理之间。一个本地的 Llama 3.2 (3B) 模型执行跨语言翻译为英语、结构重写为紧凑的任务导向格式，以及经过正则表达式验证的重写与回退保障，以确保优化后的提示不会大于原始提示。我们在 OMH-Polyglot 上进行评估，这是一个涵盖土耳其语、阿拉伯语、中文和代码切换规范的多语言编码基准。在三个商业大语言模型后端中，该中间件将提示令牌减少了 34-47%，总令牌减少了最多 18.8%，同时保持或提高了任务准确性。消融研究表明，收益主要来自重写阶段，而非简单的函数名称提取。与在匹配压缩率下的 LLMLingua-2 相比，我们的方法在所有评估的后端中始终实现了更优的 OckScore 性能。这些结果表明，主动的提示优化可以显著降低推理成本，而不牺牲编码质量。

View on arXiv Download PDF AI Translation

cs.AI / 54 / 2606.03624

Bridging Auxiliary Constraints to Resolve Instruction Following in Large Reasoning Models

桥接辅助约束以解决大型推理模型中的指令遵循问题

Zhao, Zhengyi, Zhang, Shubo, Wang, Huimin, Wang, Zezhong, Zhao, Yutian, Zheng, Yefeng, Li, Binyang, He, Yulan, Wong, Kam-Fai, Wu, Xian

Abstract

Large Reasoning Models (LRMs) have demonstrated impressive capabilities in many tasks, yet they struggle with reliably following multiple instructions, either by failing to satisfy individual constraints or by struggling to balance competing constraints simultaneously. We formalize this challenge as the Constraint Adherence Problem (CAP). This paper introduces a novel framework that addresses CAP by representing instructions as a structured knowledge graph of constraints. Our approach, Constraint Relationship Graph Completion (CRGC), explicitly models relationships between constraints, identifies adherence challenges, and discovers ``bridge constraints'' that help the model better focus on and reconcile requirements. Bridge constraints act as auxiliary instructions that make primary constraints more salient and compatible. Unlike existing approaches that enhance instruction following through general training methods, CRGC specifically improves constraint satisfaction by leveraging the model's own knowledge to create better pathways for generation. Experiments across three popular instruction following datasets demonstrate that our approach reduces constraint violations by 39% compared to standard prompting while maintaining reasoning abilities of large reasoning models.

Chinese Translation

大型推理模型（LRMs）在许多任务中展现了令人印象深刻的能力，但在可靠地遵循多个指令方面却面临挑战，既可能无法满足单个约束，也可能难以同时平衡相互竞争的约束。我们将这一挑战形式化为约束遵循问题（Constraint Adherence Problem, CAP）。本文提出了一种新颖的框架，通过将指令表示为约束的结构化知识图来解决CAP。我们的方法，即约束关系图补全（Constraint Relationship Graph Completion, CRGC），明确建模约束之间的关系，识别遵循挑战，并发现“桥接约束”，帮助模型更好地关注和调和需求。桥接约束作为辅助指令，使主要约束更加显著和兼容。与现有通过一般训练方法增强指令遵循的方式不同，CRGC特别通过利用模型自身的知识来创建更好的生成路径，从而改善约束满足。针对三个流行的指令遵循数据集的实验表明，我们的方法相比标准提示减少了39%的约束违反，同时保持了大型推理模型的推理能力。

View on arXiv Download PDF AI Translation

cs.AI / 55 / 2606.03629

TSQAgent: Rating Time Series Data Quality via Dedicated Agentic Reasoning

TSQAgent：通过专用代理推理评估时间序列数据质量

Wu, Shunyu, Li, Dan, Ye, Haozheng, Feng, Weibin, Lou, Jian, Zhang, Bo, Feng, Wenjie, Guo, Chenjuan, Ng, See-Kiong

Abstract

Assessing the quality of time series (TS) data is fundamental yet inherently challenging due to the multifaceted nature of quality dimensions. Recently, large language models (LLMs) have emerged as a promising paradigm for TS quality assessment via pairwise comparison and per-dimension evaluation. However, existing approaches rely on manually predefined quality dimensions and purely text-based reasoning, leaving it unknown whether LLMs can identify truly relevant quality dimensions or perform grounded and quantitative quality comparisons. To investigate this, we construct TSQBench, a dedicated benchmark for evaluating LLMs on two progressive capabilities: (i) understanding and identifying relevant quality dimensions, and (ii) performing quality comparison under specific dimensions. Our analysis reveals that current LLMs consistently struggle with both dimension identification and evidence-grounded quality comparison. To address these limitations, we propose TSQAgent, a novel agentic reasoning framework for TS quality rating consisting of three collaborative roles: Perceiver for focused dimension selection, Inspector for dimension-wise quantitative analysis, and Adjudicator that aggregates and refines the final judgment. In particular, we introduce an agentic reasoning strategy that instills the ability to identify and prioritize the most relevant quality dimensions, and further propose an agent workflow equipped with external analytical tools to enable precise quantitative comparisons over selected dimensions. Experiments on both the proposed benchmark and eleven real-world datasets demonstrate that our framework not only substantially improves LLMs' capabilities in quality understanding and quantitative comparison but also effectively translates these improvements into better quality-aware data selection, leading to enhanced downstream performance and data efficiency.

Chinese Translation

评估时间序列（TS）数据的质量是基础但本质上具有挑战性的任务，因为质量维度的多面性。最近，大型语言模型（LLMs）作为一种有前景的范式，通过成对比较和逐维评估来进行TS质量评估。然而，现有方法依赖于手动预定义的质量维度和纯文本推理，尚不清楚LLMs是否能够识别真正相关的质量维度或进行有依据的定量质量比较。为此，我们构建了TSQBench，这是一个专门用于评估LLMs在两个渐进能力上的基准：（i）理解和识别相关质量维度，以及（ii）在特定维度下进行质量比较。我们的分析揭示，当前的LLMs在维度识别和证据基础的质量比较方面始终面临困难。为了解决这些局限性，我们提出了TSQAgent，这是一种新的代理推理框架，用于TS质量评分，包含三个协作角色：Perceiver用于聚焦维度选择，Inspector用于逐维定量分析，Adjudicator则整合和细化最终判断。特别地，我们引入了一种代理推理策略，使其具备识别和优先考虑最相关质量维度的能力，并进一步提出了一种配备外部分析工具的代理工作流程，以实现所选维度的精确定量比较。在所提出的基准和十一组真实世界数据集上的实验表明，我们的框架不仅显著提高了LLMs在质量理解和定量比较方面的能力，还有效地将这些改进转化为更好的质量感知数据选择，从而提升下游性能和数据效率。

View on arXiv Download PDF AI Translation

cs.AI / 56 / 2606.03641

Gender-Dependent Diagnostic Substitution in LLM Medical Triage: Same Symptoms, Unequal Urgency

性别依赖的LLM医疗分诊诊断替代：相同症状，不平等的紧急性

Wong, Qi Han

Abstract

We investigate whether large language models produce different medical triage recommendations for identical neurological symptoms when only the patient's stated gender and age vary. Using three model families--Gemini 3.5 Flash, Claude Sonnet 4.6, and GPT-5.4-mini--we present a standardized symptom profile (persistent headache, blurred vision, morning nausea, visual disturbances) across seven demographic conditions: three age groups (25, 38, 65) x two genders (male, female), plus a gender-unspecified baseline (n = 30 per condition per model, 630 total trials). We find a stark, systemic gender-dependent triage disparity: young women receive significantly lower emergency room (ER) referral rates than age-matched men (Gemini: 0% vs. 23.3%; Claude: 6.7% vs. 96.7%; GPT: 6.7% vs. 66.7%, all p < 0.001). The disparity disappears at age 65 for all models. The primary mechanism is diagnostic substitution: the models anchor on a gender-associated diagnosis, preferentially classifying young women with Idiopathic Intracranial Hypertension (IIH)--a condition epidemiologically linked to women of childbearing age--while diagnosing men with generic increased intracranial pressure with space-occupying lesions in the differential. This diagnostic closure routes female patients to lower-urgency care (outpatient doctor appointments) despite comparable severity ratings (7-9/10). Our findings demonstrate that clinical LLMs replicate documented human clinical biases by using epidemiological priors to suppress triage urgency, suggesting that AI triage engines must decouple urgency assessment from probabilistic diagnostic priors. We release all code, prompts, and raw results.

Chinese Translation

我们研究了大型语言模型在仅患者所述性别和年龄变化时，是否会对相同的神经系统症状产生不同的医疗分诊建议。使用三种模型系列——Gemini 3.5 Flash、Claude Sonnet 4.6和GPT-5.4-mini——我们在七个不同的人口条件下呈现了一个标准化的症状概况（持续性头痛、视力模糊、晨起恶心、视觉干扰）：三个年龄组（25岁、38岁、65岁）x 两个性别（男性、女性），加上一个性别未指定的基线（每个条件每个模型n = 30，总共630次试验）。我们发现了显著的系统性性别依赖的分诊差异：年轻女性的急诊室（ER）转诊率显著低于年龄匹配的男性（Gemini: 0% vs. 23.3%; Claude: 6.7% vs. 96.7%; GPT: 6.7% vs. 66.7%，所有p < 0.001）。在所有模型中，这种差异在65岁时消失。主要机制是诊断替代：模型依赖于与性别相关的诊断，优先将年轻女性归类为特发性颅内高压（IIH）——这一病症在流行病学上与育龄女性相关——而将男性诊断为具有占位性病变的非特异性颅内压增高。这种诊断的封闭将女性患者引导至低紧急性的护理（门诊医生预约），尽管其严重程度评分相当（7-9/10）。我们的研究结果表明，临床LLM通过使用流行病学先验来抑制分诊紧急性，复制了已记录的人类临床偏见，这表明AI分诊引擎必须将紧急性评估与概率诊断先验解耦。我们发布了所有代码、提示和原始结果。

View on arXiv Download PDF AI Translation

cs.AI / 57 / 2606.03655

Towards Non-Monotonic Entailment in Propositional Defeasible Standpoint Logic

朝向命题可驳立场逻辑中的非单调蕴涵

Leisegang, Nicholas, Meyer, Thomas, Varzniczak, Ivan

Abstract

Recent work in defeasible reasoning has seen notions of preferential semantics and entailment in the style of Kraus et al. applied to modal logics. However, work in this field has focussed primarily on satisfiability checking, and monotonic notions of entailment, which may be inferentially weak. One particular modal logic where this has been introduced is propositional standpoint logics, where modalities can express the views of different viewpoints. This has resulted in the formalisation of propositional defeasible standpoint logic (PDSL). In this paper, we propose a means of lifting the class of (non-monotonic) rational entailment relations from traditional KLM-style reasoning to a fragment of PDSL. In order to do so, we extend the expressivity of PDSL via situated standpoint conditionals, allowing us to talk about a defeasible conditional holding in the context of a given standpoint. This allows us to re-characterise the syntax of PDSL in terms of situated conditionals, and shows that a large fragment of PDSL is expressible as a set of situated conditionals. We then focus on characterising non-monotonic entailment in this fragment, defining a method to transport any ranking-based entailment relation from the propositional case into the PDSL case. This is first described in the general case and then considered in the specific cases of rational and lexicographic closures, providing a faithful translation of each inference into PDSL. We also show that entailment-checking in this fragment of PDSL can be done largely using algorithms from the propositional case, while preserving complexity bounds.

Chinese Translation

最近在可驳推理领域的研究中，Kraus等人提出的优先语义和蕴涵概念被应用于模态逻辑。然而，该领域的研究主要集中在可满足性检查和单调蕴涵的概念上，这可能在推理上较为薄弱。引入这一概念的一个特定模态逻辑是命题立场逻辑，其中模态可以表达不同观点的看法。这导致了命题可驳立场逻辑（PDSL）的形式化。在本文中，我们提出了一种将传统KLM风格推理中的（非单调）理性蕴涵关系提升到PDSL片段的方法。为此，我们通过情境立场条件扩展了PDSL的表达能力，使我们能够讨论在特定立场背景下成立的可驳条件。这使我们能够以情境条件的形式重新表征PDSL的语法，并表明PDSL的一个大片段可以被表达为一组情境条件。接下来，我们专注于在该片段中表征非单调蕴涵，定义了一种将任何基于排名的蕴涵关系从命题情况转移到PDSL情况的方法。首先在一般情况下描述，然后在理性闭包和字典闭包的特定情况下考虑，为每个推理提供了忠实的PDSL翻译。我们还展示了在PDSL的这一片段中进行蕴涵检查在很大程度上可以使用来自命题情况的算法，同时保持复杂性界限。

View on arXiv Download PDF AI Translation

cs.AI / 58 / 2606.03657

Diagnosing Knowledge Gaps in LLM Tool Use: An Agentic Benchmark for Novel API Acquisition

诊断大型语言模型工具使用中的知识差距：新颖API获取的代理基准

Liu, Jinnuo, Peng, Yue, Niu, Jinhan, Wen, Hongyi

Abstract

Large language models for code generation often need to use APIs that are absent from their pretraining data. This requires more than recalling a function name: models must coordinate signatures, module paths, input-output contracts, semantics, and executable usage patterns. Existing novel-API benchmarks are typically static, rely on coarse pass/fail metrics, or use synthetic APIs that may not reflect real library evolution. We introduce NovelAPIBench, a fully automated dynamic benchmark that, for any base model and target library, discovers novel APIs, extracts decomposed knowledge bundles, generates executable coding tasks, and assigns failed samples to six diagnostic categories. Across about 1.9K tasks, four base models, and five domains, we compare knowledge injected through retrieval with knowledge internalized through parametric adaptation. We find that knowledge components are not interchangeable: usage examples are the strongest standalone signal, while the best two-component setting pairs signatures with either mechanisms or examples depending on the domain and backbone. Adding more context, especially source code, can hurt by increasing import-path errors. Parametric adaptation also does not replace retrieval once external knowledge is removed; rather, fine-tuning mainly teaches models how to use provided bundles, and this ability transfers to held-out libraries. These results suggest that retrieval and tuning play complementary roles: retrieval supplies volatile API content, while tuning improves procedural integration.

Chinese Translation

用于代码生成的大型语言模型通常需要使用其预训练数据中缺失的API。这不仅仅需要回忆函数名称：模型必须协调签名、模块路径、输入输出契约、语义和可执行使用模式。现有的新颖API基准通常是静态的，依赖粗略的通过/失败指标，或者使用可能无法反映真实库演变的合成API。我们引入了NovelAPIBench，这是一个完全自动化的动态基准，可以为任何基础模型和目标库发现新颖API，提取分解的知识包，生成可执行的编码任务，并将失败样本分配到六个诊断类别。在大约1900个任务、四个基础模型和五个领域中，我们比较了通过检索注入的知识与通过参数适应内化的知识。我们发现知识组件并不可互换：使用示例是最强的独立信号，而最佳的双组件设置在不同领域和基础模型的情况下，将签名与机制或示例配对。增加更多上下文，尤其是源代码，可能会通过增加导入路径错误而造成负面影响。一旦外部知识被移除，参数适应也无法替代检索；相反，微调主要教会模型如何使用提供的知识包，这种能力会转移到未使用的库。这些结果表明，检索和微调发挥互补作用：检索提供易变的API内容，而微调改善程序集成。

View on arXiv Download PDF AI Translation

cs.AI / 59 / 2606.03660

From Answers to States: Verifiable Process-Level Evaluation of Chemical Reasoning in Large Language Models

从答案到状态：大型语言模型中化学推理的可验证过程级评估

Guo, Hongyu, Li, Hao, Cao, He, Zhang, Gongbo, Yuan, Li

Abstract

Large language models are increasingly used as chemistry assistants, yet most chemistry benchmarks still score only final answers. This masks a critical failure mode: a model may output the correct molecule, product, or option while its reasoning violates chemical logic. Existing process-level evaluators are hard to scale because LLM judges and human step-level process annotation are costly, inconsistent, and vulnerable to hallucination. We introduce ChemCoTBench-V2, a rule-verifiable diagnostic benchmark for low-cost, auditable evaluation of structured, verifier-addressable chemical reasoning traces. It spans molecular understanding, molecule editing, molecular optimization, and reaction prediction, with 5,620 evaluation samples across 18 reporting tasks. Models must expose key intermediate steps in expert-designed templates, and those steps are checked with deterministic chemistry rules and, for closed-answer tasks, reference traces rather than another LLM judge. Open-ended molecular optimization is evaluated with oracle-verifiable state constraints rather than strict trace matching. The benchmark reports three separate signals: final-answer correctness, template adherence, and step-wise verifier correctness over expert-refined intermediate commitments. Experiments on frontier models reveal a persistent gap between final-answer success and structured-reasoning-state consistency: models often follow the requested format while failing chemical-step checks, or answer correctly with weak supporting reasoning. ChemCoTBench-V2 enables fine-grained model comparison and identifies the concrete step at which the trace first violates the verifier.

Chinese Translation

大型语言模型越来越多地被用作化学助手，但大多数化学基准测试仍然仅评分最终答案。这掩盖了一个关键的失败模式：模型可能输出正确的分子、产物或选项，而其推理却违反了化学逻辑。现有的过程级评估工具难以扩展，因为大型语言模型（LLM）评判和人类逐步过程注释的成本高、结果不一致且容易出现幻觉。我们引入了ChemCoTBench-V2，这是一个可规则验证的诊断基准，用于低成本、可审计的结构化、可验证的化学推理轨迹评估。该基准涵盖分子理解、分子编辑、分子优化和反应预测，共包含18个报告任务的5,620个评估样本。模型必须在专家设计的模板中展示关键的中间步骤，这些步骤通过确定性的化学规则进行检查，对于封闭答案任务，则使用参考轨迹而非另一个LLM评判。开放式分子优化则通过可验证的状态约束进行评估，而非严格的轨迹匹配。该基准报告三个独立信号：最终答案的正确性、模板遵循情况以及在专家精炼的中间承诺上的逐步验证正确性。对前沿模型的实验揭示了最终答案成功与结构化推理状态一致性之间的持续差距：模型通常遵循请求的格式，但未能通过化学步骤检查，或者在支持推理薄弱的情况下正确回答。ChemCoTBench-V2使得模型比较更加细致，并识别出轨迹首次违反验证器的具体步骤。

View on arXiv Download PDF AI Translation

cs.AI / 60 / 2606.03678

EvoDrive: Pareto Evolution for Safety-Critical Autonomous Driving via Self-Improving LLM Agents

EvoDrive：通过自我改进的LLM代理进行安全关键型自主驾驶的帕累托进化

Nie, Tong, Mei, Yuewen, Tang, Yihong, He, Junlin, Deng, Jie, Sun, Jian, Ma, Wei

Abstract

Generating safety-critical scenarios is essential for validating and improving autonomous driving systems, yet it inherently requires maximizing adversariality to expose failures while preserving realism. Existing methods usually manage this trade-off with handcrafted heuristics, confining generation to known priors and overlooking underexplored patterns. While recent open-ended agentic evolution can push this limit, unconstrained general agents lack strict simulator grounding and tend to collapse the multi-objective tension into single-scalar maximization. Here we present EvoDrive, the first automated, LLM-based agentic evolution framework for multi-objective scenario generation. EvoDrive employs a simulator-grounded actor-critic architecture where a memory-driven actor iteratively proposes improvements to the generators and critics filter out implausible candidates, and a self-evolving world evaluator routes promising proposals to optimize simulation budgets. EvoDrive further maintains a Pareto archive of evaluated candidates to preserve diverse attack-realism trade-offs and guide future evolution via simulation feedback. Benchmark results on MetaDrive and CARLA show that EvoDrive not only significantly expands the Pareto frontier across various generators, but also produces valuable scenarios for policy training.

Chinese Translation

生成安全关键场景对于验证和改进自主驾驶系统至关重要，但这本质上需要最大化对抗性，以揭示故障，同时保持现实性。现有方法通常通过手工设计的启发式方法来管理这一权衡，限制生成在已知先验范围内，并忽视未充分探索的模式。尽管最近的开放式代理进化可以推动这一极限，但不受约束的通用代理缺乏严格的模拟器基础，往往将多目标张力压缩为单标量最大化。在此，我们提出了EvoDrive，这是第一个基于LLM的自动化代理进化框架，用于多目标场景生成。EvoDrive采用了一个基于模拟器的演员-评论家架构，其中一个基于记忆的演员迭代性地向生成器提出改进建议，而评论家则筛选出不合适的候选者，自我进化的世界评估器则将有前景的提案引导至优化模拟预算。EvoDrive进一步维护一个评估候选者的帕累托档案，以保留多样化的攻击-现实性权衡，并通过模拟反馈指导未来的进化。在MetaDrive和CARLA上的基准结果表明，EvoDrive不仅显著扩展了各种生成器的帕累托前沿，还为策略训练生成了有价值的场景。

View on arXiv Download PDF AI Translation

cs.AI / 61 / 2606.03686

The DeepSpeak-Agentic Dataset

DeepSpeak-Agentic 数据集

Barrington, Sarah, Bohacek, Maty, Farid, Hany

Abstract

We present DeepSpeak-Agentic, a dataset of videos comprising over 37 hours of semi-structured conversations between a human and an embodied AI agent. We use this dataset to evaluate the automatic forensic identification (audio, video, or text) of AI agents, study the nature of human-agent interactions, and provide a benchmark for future advances in the large-language models and AI-generated voices and faces that power embodied AI agents. We also contribute a scalable data-capture system that creates agents, automatically pairs them with human crowd workers, records audiovisual conversations across specified scenarios, and identifies and separates the human and agent in the combined stream.

Chinese Translation

我们提出了 DeepSpeak-Agentic 数据集，该数据集包含超过 37 小时的人类与具身 AI 代理之间的半结构化对话视频。我们利用该数据集评估 AI 代理的自动法医识别（音频、视频或文本），研究人类与代理之间的互动性质，并为未来在大型语言模型以及驱动具身 AI 代理的 AI 生成语音和面孔方面的进展提供基准。此外，我们还贡献了一个可扩展的数据捕获系统，该系统创建代理，自动将其与人类众包工作者配对，记录在指定场景下的视听对话，并在合成流中识别和分离人类与代理。

View on arXiv Download PDF AI Translation

cs.AI / 62 / 2606.03692

SkillPyramid: A Hierarchical Skill Consolidation Framework for Self-Evolving Agents

SkillPyramid：一种用于自我进化代理的层次技能整合框架

Xiong, Yuan, Miao, Ziqi, Chen, Qian, Li, Lijun, Wang, Yequan, He, Shizhu, Zhao, Jun, Liu, Kang

Abstract

Recent AI agents can flexibly invoke skills to solve complex tasks, but their long-term improvement is fundamentally constrained by a lack of systematic skill construction, accumulation, and transfer. In particular, without a unified framework for skill consolidation, agents tend to redundantly construct similar capabilities across different tasks, are unable to effectively transform experience into reusable assets, and struggle to generalize task-specific skills to novel scenarios. To address this limitation, we propose SkillPyramid, a skill consolidation framework that reuses existing skill experience for broader task generalization. Operating on a hierarchical skill topology, SkillPyramid further introduces a self-evolution mechanism that enables agents to compose, validate, and incorporate new skills during task execution. Experiments on ALFWorld, WebShop, and ScienceWorld across four backbone models show that SkillPyramid substantially increases the average reward by 38.0% and reduces execution steps by 27.7%. Overall, our method transforms a skill collection from a static resource pool into a dynamic evolution system.

Chinese Translation

近期的人工智能代理能够灵活调用技能来解决复杂任务，但其长期改进受到系统性技能构建、积累和转移缺乏的根本限制。特别是，在没有统一的技能整合框架的情况下，代理往往在不同任务中冗余地构建相似的能力，无法有效地将经验转化为可重用的资产，并且在将任务特定技能推广到新场景时面临困难。为了解决这一限制，我们提出了SkillPyramid，一种利用现有技能经验实现更广泛任务推广的技能整合框架。SkillPyramid基于层次技能拓扑结构，进一步引入了一种自我进化机制，使代理能够在任务执行过程中组合、验证和整合新技能。在ALFWorld、WebShop和ScienceWorld的四个基础模型上的实验表明，SkillPyramid显著提高了平均奖励38.0%，并减少了执行步骤27.7%。总体而言，我们的方法将技能集合从静态资源池转变为动态进化系统。

View on arXiv Download PDF AI Translation

cs.AI / 63 / 2606.03704

Dynamic Objective Selection with Safeguards and LLM Oversight for Financial Decision-Making

具有安全保障和大型语言模型监督的动态目标选择在金融决策中的应用

Sakurai, Keigo, Ogawa, Takahiro, Haseyama, Miki, Anan, Anjyu, Nakagawa, Kei

Abstract

Financial decision-making tasks such as stock recommendation and portfolio allocation typically estimate future return and risk and then select trades or allocations for an investor, and the chosen optimization objective often determines realized performance. However, because market conditions evolve over time, a fixed objective can be suboptimal across regimes, while regime-switching pipelines that rely on latent regime estimates can be noisy or delayed and frequent switching can increase turnover and operational instability. In this paper, we propose DOSS (Dynamic Objective Selection with Safeguards), a learning-based selector that directly chooses the decision-relevant objective function at each time point from interpretable statistical summaries of recent returns, selecting among a small set of candidates (e.g., return-seeking, loss-averse, and risk-adjusted) without introducing intermediate regime variables. DOSS formulates objective selection as a classification problem over objectives and performs sequential updates with a rolling window to make forward-looking selections without temporal leakage, while also outputting a confidence score for each proposal. To mitigate misselection and excessive switching in deployment, DOSS applies confidence-aware gating with a fail-safe that overrides low-confidence proposals to a conservative default and enforces explicit controls tied to switching frequency. We further integrate governance by positioning a Large Language Model (LLM) as an oversight component rather than a generator of new objectives: the LLM is restricted to accept a proposed objective or override it to a predefined safe default, with deterministic rule-based constraints triggering overrides when needed.

Chinese Translation

金融决策任务，如股票推荐和投资组合配置，通常需要估计未来的收益和风险，然后为投资者选择交易或配置，而所选择的优化目标往往决定了实际表现。然而，由于市场条件随时间演变，固定目标在不同市场环境中可能表现不佳，而依赖潜在市场状态估计的状态切换管道可能会出现噪声或延迟，频繁切换可能会增加周转率和操作不稳定性。本文提出了DOSS（具有安全保障的动态目标选择），这是一种基于学习的选择器，能够直接从近期收益的可解释统计摘要中选择每个时间点的决策相关目标函数，从一小组候选目标（例如，追求收益、规避损失和风险调整）中进行选择，而不引入中间状态变量。DOSS将目标选择公式化为一个目标分类问题，并采用滚动窗口进行顺序更新，以便在不产生时间泄漏的情况下进行前瞻性选择，同时为每个提案输出置信度分数。为了减少部署中的错误选择和过度切换，DOSS应用了基于置信度的门控机制，并设有安全保护措施，将低置信度提案覆盖为保守的默认值，并强制执行与切换频率相关的明确控制。我们进一步通过将大型语言模型（LLM）作为监督组件而非新目标的生成者来整合治理：LLM被限制为接受提议的目标或将其覆盖为预定义的安全默认值，当需要时，基于确定性规则的约束会触发覆盖。

View on arXiv Download PDF AI Translation

cs.AI / 64 / 2606.03705

Code-on-Graph: Iterative Programmatic Reasoning via Large Language Models on Knowledge Graphs

图上的代码：通过大型语言模型在知识图谱上进行迭代程序推理

Ding, Weiwei, Li, Zixuan, Bai, Long, Chen, Zhuo, Su, Kun, Wang, Fei, Jin, Xiaolong, Zhang, Jin, Guo, Jiafeng, Cheng, Xueqi

Abstract

Knowledge Graphs (KGs) are widely used to mitigate the limitations of Large Language Models (LLMs), such as outdated knowledge and hallucinations. Existing LLM-KG integration frameworks typically rely on predefined operators to retrieve factual knowledge from KGs and inject it into prompts for answer generation. This paradigm faces two critical bottlenecks: 1) Inflexibility: The predefined operators are limited in scope and thus lack sufficient compositional expressiveness to fully capture the complex semantics required by KG questions. 2) Unscalability: Direct injection of factual knowledge into prompts limits scalability in handling large-scale factual knowledge. To address these two bottlenecks, we propose Code-on-Graph (CoG), a programmatic reasoning framework for LLM-KG integration. Specifically, given the factual knowledge retrieved at each reasoning step, CoG first identifies the corresponding KG schemas and represents these schemas as Python classes, which serve as abstract interfaces to the retrieved facts. It then generates executable code grounded in these classes, with the retrieved facts instantiated as objects of the corresponding classes during execution. This design enables flexible code-based reasoning while avoiding the direct injection of large-scale factual knowledge into prompts. Experiments on WebQSP, CWQ, and GrailQA demonstrate that CoG outperforms prior state-of-the-art models by up to 10.5%.

Chinese Translation

知识图谱（KGs）被广泛用于缓解大型语言模型（LLMs）的局限性，例如过时的知识和幻觉。现有的LLM-KG集成框架通常依赖于预定义的操作符，从知识图谱中检索事实知识并将其注入提示中以生成答案。这种范式面临两个关键瓶颈：1）灵活性不足：预定义的操作符在范围上有限，因此缺乏足够的组合表达能力，无法充分捕捉KG问题所需的复杂语义。2）不可扩展性：将事实知识直接注入提示中限制了处理大规模事实知识的可扩展性。为了解决这两个瓶颈，我们提出了图上的代码（Code-on-Graph, CoG），这是一个用于LLM-KG集成的程序推理框架。具体而言，CoG在每个推理步骤中检索到事实知识后，首先识别相应的KG模式，并将这些模式表示为Python类，这些类作为检索到的事实的抽象接口。然后，它生成基于这些类的可执行代码，在执行过程中将检索到的事实实例化为相应类的对象。该设计实现了灵活的基于代码的推理，同时避免了将大规模事实知识直接注入提示中。在WebQSP、CWQ和GrailQA上的实验表明，CoG的表现优于之前的最先进模型，提升幅度高达10.5%。

View on arXiv Download PDF AI Translation

cs.AI / 65 / 2606.03719

Unveiling the Structure of Do-Calculus Reasoning via Derivation Graphs

通过推导图揭示 Do-演算推理的结构

Yvernes, Clément, Devijver, Emilie, Clausel, Marianne, Gaussier, Eric

Abstract

The do-calculus defines a general system of inference for interventional queries, allowing causal quantities to be transformed through successive applications of its rules. This process induces a rich space of equivalent interventional expressions, but combining and ordering these rules remains challenging. In this work, we introduce derivation graphs, which represent how do-calculus rules are applied and combined, and characterize the full space of observational and interventional probabilities which are equivalent under the do-calculus. The structure of these graphs yields a simple procedure that uses at most four applications of do-calculus rules. Finally, we show how applying identification algorithms to equivalent causal queries produces multiple valid estimands for the same causal quantity, eventually yielding more efficient estimators.

Chinese Translation

Do-演算定义了一种用于干预查询的推理通用系统，允许通过其规则的连续应用来转化因果量。这个过程引入了一个丰富的等效干预表达空间，但组合和排序这些规则仍然具有挑战性。在本研究中，我们引入了推导图，表示 Do-演算规则的应用和组合方式，并描述了在 Do-演算下等效的观察和干预概率的完整空间。这些图的结构产生了一种简单的程序，最多使用四次 Do-演算规则的应用。最后，我们展示了如何将识别算法应用于等效的因果查询，从而为同一因果量产生多个有效的估计量，最终实现更高效的估计器。

View on arXiv Download PDF AI Translation

cs.AI / 66 / 2606.03741

When to Re-Plan: Subgoal Persistence in Hierarchical Latent Reasoning

何时重新规划：层次潜在推理中的子目标持久性

Chadha, Ayushi

Abstract

Long-horizon reasoning requires a system to commit to medium-horizon intent without becoming rigid: re-plan too often and computation never coheres into multi-step structure; commit too long and the plan goes stale. We study this stability-adaptivity tradeoff in the latent reasoning setting, where multi-step computation occurs inside hidden state rather than externalized token traces. We extend the Hierarchical Reasoning Model (HRM) with a feudal-style manager-worker interface: a slow high-level module periodically emits a normalized directional subgoal that persists for P low-level steps, biasing the worker's hidden-state updates and supplying an intrinsic cosine alignment loss. On ARC and ConceptARC, we find that subgoal persistence -- not subgoal injection alone -- is the central knob: moderate periods P in [3, 6] consistently outperform both very frequent (P=1) and very long horizons, with a clear minimum LM loss at P=3 (1.544 vs. 1.674 at P=1, 1.640 baseline; replicated over 5 seeds at mean 1.595, std 0.045). The intrinsic alignment weight lambda shows a complementary narrow optimum (lambda approximately 0.05). A controlled ablation at past-sweet-spot lambda isolates learned directional structure -- not architectural capacity or auxiliary loss alone -- as the source of interference when the alignment signal exceeds its optimum. Together these findings implicate a design principle for compositional planning in latent reasoning systems: medium-horizon intent must be coherent across enough computational steps for compositional structure to form.

Chinese Translation

长时间范围的推理要求系统在不变得僵化的情况下承诺中等时间范围的意图：过于频繁地重新规划会导致计算无法形成多步骤结构；承诺时间过长则会使计划失效。我们在潜在推理的背景下研究这一稳定性与适应性的权衡，其中多步骤计算发生在隐藏状态内部，而非外部化的标记轨迹。我们通过一种封建式的管理者-工作者接口扩展了层次推理模型（Hierarchical Reasoning Model, HRM）：一个缓慢的高层模块定期发出一个规范化的方向性子目标，该子目标在 P 个低层步骤中保持存在，偏向于工作者的隐藏状态更新，并提供内在的余弦对齐损失。在 ARC 和 ConceptARC 上，我们发现子目标持久性——而不仅仅是子目标注入——是关键因素：适中的周期 P 在 [3, 6] 中始终优于非常频繁（P=1）和非常长的时间范围，在 P=3 时具有明显的最小语言模型损失（1.544 对比 P=1 的 1.674，基线为 1.640；在 5 个种子上重复实验，平均值为 1.595，标准差为 0.045）。内在对齐权重 lambda 显示出一个互补的狭窄最优（lambda 约为 0.05）。在过去的最佳点 lambda 进行的控制消融实验隔离了学习到的方向性结构——而不仅仅是架构能力或辅助损失——作为当对齐信号超过其最优值时干扰的来源。这些发现共同指向了一个设计原则，用于潜在推理系统中的组合规划：中等时间范围的意图必须在足够的计算步骤中保持一致，以便形成组合结构。

View on arXiv Download PDF AI Translation

cs.AI / 67 / 2606.03743

Proof-Refactor: Refactoring Generated Formal Proofs into Modular Artifacts

Proof-Refactor：将生成的形式证明重构为模块化工件

Fu, Yiming, Liu, Peixuan, Wang, Zichen, yuan, Kun

Abstract

While Large Language Models (LLMs) have shown strong performance in generating formal proofs, their outputs often remain less readable, modular, maintainable, and reusable than proofs in mature formal mathematics libraries. We argue that this gap stems in part from the compile-first objective implicit in most proof-generation pipelines, which encourages monolithic or ad hoc proof scripts rather than library-quality artifacts. Existing approaches to proof-quality improvement often rely on explicit, computable optimization objectives. In practice, however, the most tractable and experimentally validated objectives are largely length-based, while higher-level qualities such as readability, modularity, maintainability, and reusability are difficult to reduce to reliable automatic metrics. Instead of optimizing proof improvement against a single proxy metric, we take a process-guided approach inspired by human proof-refactoring workflows. We propose an agentic framework $\textbf{Proof-Refactor}$ that decomposes proof refactoring into four phases: extracting candidate proof fragments, designing helper declarations, formally proving the extracted and designed components, and repairing the original proof using the verified components. On generated Lean proofs from PutnamBench and Putnam2025, Proof-Refactor improves rubric-based refactoring scores over a strong Claude Code refactoring baseline, with the largest gains in signature quality and human readability. These results suggest that process-guided refactoring can improve proof structure without treating proof length as the primary objective.

Chinese Translation

尽管大型语言模型（LLMs）在生成形式证明方面表现出色，但其输出的可读性、模块化、可维护性和可重用性往往不及成熟形式数学库中的证明。我们认为，这一差距部分源于大多数证明生成管道中隐含的优先编译目标，这种目标鼓励生成单体或临时的证明脚本，而非库质量的工件。现有的证明质量改进方法通常依赖于明确的、可计算的优化目标。然而，在实践中，最易处理和实验验证的目标主要基于长度，而可读性、模块化、可维护性和可重用性等更高层次的质量则难以归结为可靠的自动化指标。我们并不将证明改进优化针对单一的代理指标，而是采取一种受人类证明重构工作流程启发的过程引导方法。我们提出了一个代理框架 $ extbf{Proof-Refactor}$，将证明重构分解为四个阶段：提取候选证明片段、设计辅助声明、正式证明提取和设计的组件，以及利用已验证的组件修复原始证明。在来自 PutnamBench 和 Putnam2025 的生成 Lean 证明上，Proof-Refactor 在强大的 Claude Code 重构基线之上提高了基于评分标准的重构分数，尤其在签名质量和人类可读性方面取得了最大的提升。这些结果表明，过程引导的重构可以在不将证明长度作为主要目标的情况下改善证明结构。

View on arXiv Download PDF AI Translation

cs.AI / 68 / 2606.03755

LAP: An Agent-to-Instrument Protocol for Autonomous Science

LAP：一种用于自主科学的代理-仪器协议

Zhu, Linwu, Gao, Liqiang, Chen, Yan, Zhu, Dan, Huang, Jian

Abstract

Autonomous science is moving from demonstration to infrastructure. Large language model agents now plan experiments, and self-driving laboratories execute them. Yet every such system rebuilds the link between the reasoning agent and the physical instrument from scratch, against fragmented vendor SDKs and standards built for deterministic software clients rather than probabilistic, goal-directed agents. Recent agent-interoperability protocols clarify two of the three edges of an agentic ecosystem (Anthropic's Model Context Protocol (MCP) standardizes the agent-to-tool edge, and Google's Agent2Agent (A2A) the agent-to-agent edge), but neither models the agent-to-instrument edge, where operations are stateful, safety-critical, exclusively owned, physically embodied, and produce measurements with units, calibration, and uncertainty. We present the Lab Agent Protocol (LAP), a protocol design that fills this gap. LAP retains A2A's peer-to-peer, discovery-first, task-lifecycle structure and adds four physical-world primitives: (i) the InstrumentCard, a signed capability and physical-limit description; (ii) first-class reservation for exclusive instrument and sample locking; (iii) a safety-fence handshake with operator-confirmation tokens cryptographically bound to a specific task and its parameters, gating hazardous and irreversible operations; and (iv) a MeasurementResult schema that makes every result physically typed (QUDT/UCUM), calibration-anchored, uncertainty-bearing, and reproducible by construction. We specify roles, a six-layer architecture, the JSON-RPC method set, the task and safety state machines, the error model, and cross-laboratory federation, and walk a closed-loop autonomous campaign through the protocol end-to-end. LAP is transport-compatible with the A2A/MCP ecosystem and encapsulates rather than replaces existing device standards such as SiLA 2 and OPC-UA.

Chinese Translation

自主科学正从演示阶段向基础设施发展。大型语言模型代理现在能够规划实验，而自动化实验室则执行这些实验。然而，每个这样的系统都需要从头开始重建推理代理与物理仪器之间的连接，这与为确定性软件客户端而构建的分散的供应商SDK和标准相悖，而这些客户端并不适用于概率性、目标导向的代理。最近的代理互操作协议澄清了代理生态系统的三个边缘中的两个（Anthropic的模型上下文协议（MCP）标准化了代理与工具之间的边缘，而谷歌的代理间协议（A2A）则标准化了代理与代理之间的边缘），但没有一个模型能够涵盖代理与仪器之间的边缘，在这个边缘上，操作是有状态的、安全关键的、独占拥有的、物理体现的，并且产生具有单位、校准和不确定性的测量结果。我们提出了实验室代理协议（LAP），一种填补这一空白的协议设计。LAP保留了A2A的对等、优先发现、任务生命周期结构，并增加了四个物理世界原语：（i）仪器卡（InstrumentCard），一个签名的能力和物理限制描述；（ii）对独占仪器和样本锁定的优先预订；（iii）与操作员确认令牌的安全围栏握手，这些令牌在密码学上绑定到特定任务及其参数，限制危险和不可逆操作；（iv）测量结果模式（MeasurementResult schema），使每个结果在物理上具有类型（QUDT/UCUM），以校准为锚，承载不确定性，并通过构造实现可重复性。我们指定了角色、六层架构、JSON-RPC方法集、任务和安全状态机、错误模型以及跨实验室的联合，并通过协议端到端地走完一个闭环自主活动。LAP与A2A/MCP生态系统兼容，并封装而非替代现有的设备标准，如SiLA 2和OPC-UA。

View on arXiv Download PDF AI Translation

cs.AI / 69 / 2606.03777

From Control Boundary to Insurance Claim: Reconstructing AI-Mediated Losses Through the CER Framework

从控制边界到保险索赔：通过CER框架重构AI介导的损失

Leung, Alex, Zhang, Rex, Toyoda, Kentaroh, Loh, SiewMei

Abstract

AI losses that arise through an insured organization's generative or agentic AI system require state reconstruction, not merely event reconstruction, because the relevant state changes as the system reasons, retrieves, calls tools, and acts. The relevant question is not only what loss occurred, but what the system was allowed to do, what it actually did, and whether that reconstructed loss can support insurance claim recovery. This paper addresses losses in which the insured's AI system is in the causal chain, including externally triggered failures such as prompt injection, retrieval-augmented generation (RAG) poisoning, malicious tool output, credential misuse, and data poisoning. Specifically, this paper introduces CER, a use-case-level diagnostic for AI residual risk transfer. C (control boundary) asks whether the system had an enforceable operating envelope. E (evidence reconstruction) asks whether the system state and causal chain can be reconstructed from retained artifacts. R (insurance response) asks whether the reconstructed loss is insured: whether insurance coverage is available in the market and placed for the insured, together with the proof needed to support insurance claim recovery. The paper makes three contributions: it defines the AI-specific reconstruction problem, operationalizes that problem through CER, and specifies claim-grade evidence for AI reconstruction. Public examples include the reported PocketOS and Replit agentic database-deletion incidents and Moffatt v. Air Canada as an adjudicated output/reliance case. Keywords: AI systems; CER framework; residual risk transfer; agentic AI; generative AI; AI insurance; evidence reconstruction.

Chinese Translation

通过被保险组织的生成性或代理性AI系统产生的AI损失需要进行状态重构，而不仅仅是事件重构，因为在系统推理、检索、调用工具和行动的过程中，相关状态会发生变化。相关的问题不仅是发生了什么损失，还包括系统被允许做什么、实际做了什么，以及重构的损失是否能够支持保险索赔的恢复。本文讨论了被保险的AI系统在因果链中的损失，包括外部触发的故障，如提示注入、增强检索生成（RAG）中毒、恶意工具输出、凭证滥用和数据中毒。具体而言，本文介绍了CER，一个用于AI剩余风险转移的用例级诊断工具。C（控制边界）询问系统是否具有可执行的操作范围。E（证据重构）询问是否可以从保留的文物中重构系统状态和因果链。R（保险响应）询问重构的损失是否被保险覆盖：市场上是否有可用的保险覆盖，并为被保险人投保，以及支持保险索赔恢复所需的证据。本文做出了三项贡献：定义了特定于AI的重构问题，通过CER将该问题操作化，并为AI重构指定了索赔级别的证据。公共示例包括报告的PocketOS和Replit代理数据库删除事件，以及Moffatt诉加拿大航空案作为一个裁定的输出/依赖案例。关键词：AI系统；CER框架；剩余风险转移；代理性AI；生成性AI；AI保险；证据重构。

View on arXiv Download PDF AI Translation

cs.AI / 70 / 2606.03812

Enhancing Operational Safety via Agentic Dialogue Hazard Identification Analysis

通过代理对话危险识别分析提升操作安全性

Das, Sanjay, Elgedawy, Ran, Seefried, Ethan, Burchfield, Ryan, Ghosal, Tirthankar

Abstract

Operational safety in high-stakes domains such as industrial process control, autonomous, and safety-critical systems, demand reliable hazard identification. While large language models (LLMs) have shown promise in automating safety analysis tasks, single-turn, monolithic inference is brittle: it lacks the self-correction, deliberation, and contextual refinement that safety engineers apply iteratively. In this paper, we introduce HAZDIAL, a framework that investigates whether structured agentic dialogue-multi-agent, multi-turn interactions improves the quality of NLP- based hazard identification over single-pass baselines. We systematically compare two dialogue modalities: adversarial debate and constructive discussion, and propose an algorithm-based agentic interaction optimization. We evaluate all configurations against a curated golden dataset using standard classification metrics (accuracy, precision, recall, F1) and novel dialogue metrics. This work advances the intersection of dialogue systems, multi-agent reasoning, and AI safety, providing an empirical evidence for dialogue-driven hazard analysis.

Chinese Translation

在工业过程控制、自动化及安全关键系统等高风险领域，操作安全性要求可靠的危险识别。尽管大型语言模型（LLMs）在自动化安全分析任务中展现出潜力，但单轮、单一推理的方式较为脆弱：它缺乏安全工程师迭代应用的自我修正、深思熟虑和上下文精炼。在本文中，我们介绍了HAZDIAL，一个框架，旨在探讨结构化的代理对话——多智能体、多轮互动是否能提升基于自然语言处理（NLP）的危险识别质量，相较于单次推理的基线。我们系统性地比较了两种对话模式：对抗性辩论和建设性讨论，并提出了一种基于算法的代理互动优化方法。我们使用标准分类指标（准确率、精确率、召回率、F1）和新颖的对话指标，对所有配置进行了评估，基于一个精心策划的黄金数据集。此项工作推进了对话系统、多智能体推理与人工智能安全的交叉研究，为对话驱动的危险分析提供了实证依据。

View on arXiv Download PDF AI Translation

cs.AI / 71 / 2606.03814

Leveraging BART to Assess CS1 C++ Programming Assignments using Rubric-based Criteria

利用BART评估CS1 C++编程作业的基于评分标准的标准

Rainey, Kelsey, Roberts, Jesse

Abstract

This paper investigates rubric-aware, multitask fine-tuning of transformer models for automated grading of introductory C++ programming assignments, with the goal of producing grade predictions that better reflect instructor grading behavior than general-purpose LLMs. Using multi-semester CS1 data, student submissions are paired with numeric scores, letter-grade buckets, and assignment rubrics, then preprocessed into unified sequences for transformer input. A BART encoder-decoder with LoRA adaptation is trained to jointly predict numeric grades and grade buckets, augmented with a distribution-matching term to align predicted and empirical grade distributions, an evaluation dimension often overlooked in prior work. Experiments compare single-task and multitask training, hard one-hot versus fuzzy and boundary-based soft labels, and rubric versus no-rubric conditions, with additional T5 and pairwise-pretrained variants. Results show that multitask BART with boundary-based soft labels and rubric context achieves lower mean absolute error and stronger grade-distribution alignment than single-task, hard-label, or code-only baselines. Fully fine-tuned T5 further improves distributional fidelity, while pairwise pretraining reduces numeric error at the cost of minority-class sensitivity. Collectively, the findings suggest that calibration-aware, rubric-guided training produces more instructor-like grading behavior than accuracy-optimized alternatives.

Chinese Translation

本文研究了基于评分标准的多任务微调变换器模型，以实现对初级C++编程作业的自动评分，旨在生成更能反映教师评分行为的成绩预测，而非通用的大型语言模型（LLMs）。通过多学期的CS1数据，将学生提交的作业与数值评分、字母等级区间和作业评分标准配对，然后预处理为统一序列以供变换器输入。我们训练了一个带有LoRA适应的BART编码器-解码器，以联合预测数值成绩和等级区间，并增加了一个分布匹配项，以对齐预测的成绩分布和经验成绩分布，这一评估维度在以往的研究中常被忽视。实验比较了单任务与多任务训练、硬一热标签与模糊及边界基础的软标签、以及有评分标准与无评分标准的条件，并包括额外的T5和成对预训练变体。结果表明，使用边界基础软标签和评分标准上下文的多任务BART在平均绝对误差和成绩分布对齐方面优于单任务、硬标签或仅代码的基线。完全微调的T5进一步提高了分布的保真度，而成对预训练则在减少数值误差的同时牺牲了对少数类别的敏感性。总体而言，研究结果表明，基于校准的、评分标准指导的训练能够产生更接近教师评分行为的结果，而非仅优化准确性的替代方案。

View on arXiv Download PDF AI Translation

cs.AI / 72 / 2606.03823

Calibrating Urban Traffic Simulation from Sparse Road Observations via Genetic Optimization

通过遗传优化从稀疏道路观测校准城市交通模拟

Sawyer, Hunter, Roberts, Jesse, Matei, Simon

Abstract

Urban traffic simulation is a critical tool for infrastructure planning, including the placement of electric vehicle charging stations. However, realistic traffic simulation across many cities is hindered by two fundamental data limitations: detailed real-world traffic measurements are available for only a small fraction of road segments in most cities, and employment distribution data critical for modeling commuter traffic is rarely available at the resolution needed for simulation. This paper presents a genetic algorithm-based framework that directly addresses both limitations, calibrating urban traffic simulations from sparse road observations without requiring detailed job location data. Using the SUMO traffic simulation platform for Greensboro, North Carolina, our approach optimizes job distributions and gate-traffic parameters to align simulated traffic with a small sample of roads with known traffic-flow rates. We demonstrate that this approach produces simulated traffic that correlates well with real-world measurements, generalizes to road segments withheld from training, and produces job distributions that show promising qualitative agreement with census employment data despite never directly training on that employment data. This work demonstrates that realistic urban traffic simulation can be achieved from minimal real-world observations, offering a scalable and data-light approach to simulation calibration that reduces the barrier to deploying traffic models across diverse cities.

Chinese Translation

城市交通模拟是基础设施规划的重要工具，包括电动汽车充电站的布局。然而，许多城市的现实交通模拟受到两个基本数据限制的制约：在大多数城市中，只有少量道路段提供详细的现实交通测量数据，而对于建模通勤交通至关重要的就业分布数据，通常缺乏所需的分辨率。本文提出了一种基于遗传算法的框架，直接解决这两个限制，从稀疏的道路观测中校准城市交通模拟，而无需详细的工作地点数据。我们使用北卡罗来纳州格林斯伯勒的SUMO交通模拟平台，优化工作分布和门口交通参数，以使模拟交通与已知交通流量的少量道路样本相一致。我们证明这种方法生成的模拟交通与现实测量数据具有良好的相关性，能够推广到未用于训练的道路段，并且生成的工作分布在定性上与人口普查就业数据表现出良好的一致性，尽管从未直接在该就业数据上进行训练。这项工作表明，从最少的现实观察中可以实现真实的城市交通模拟，提供了一种可扩展且数据轻量的模拟校准方法，降低了在不同城市部署交通模型的门槛。

View on arXiv Download PDF AI Translation

cs.AI / 73 / 2606.03829

BigFinanceBench: A Workflow-Grounded Benchmark for Financial-Research Agents

BigFinanceBench：一个基于工作流程的金融研究代理基准

Wang, Alex, Meinhardt, Georg, Katz, Jacob, Kim, Joseph H., Chaudhary, Pratyush K., Blagden, Chase, Xu, Eric

Abstract

Financial-research answers are decision-relevant only when another analyst can audit how they were produced: which source was chosen, which period and accounting definition were used, which assumptions were made, and how the calculation was performed. Existing finance benchmarks largely evaluate isolated subskills or final answers, leaving the auditable derivation itself under-measured. We introduce BigFinanceBench, a 928-item expert-authored benchmark of open-ended financial-research tasks in which each item pairs a ground-truth reference answer with a point-weighted rubric that decomposes the derivation into independently checkable steps. BigFinanceBench is workflow-grounded in that it evaluates the full derivation rather than only the final output. Across 36,241 rubric points, the benchmark supports partial-credit evaluation and localization of failures across the analyst workflow. Evaluating ten current frontier and open-weight agents, we find substantial headroom: the best system reaches only 58.8% rubric score, final-answer accuracy is a useful but lossy proxy for derivation quality, and model capability varies non-uniformly across financial workflows.

Chinese Translation

金融研究的答案只有在其他分析师能够审计其产生过程时才具有决策相关性：选择了哪个来源，使用了哪个时期和会计定义，做出了哪些假设，以及计算是如何进行的。现有的金融基准主要评估孤立的子技能或最终答案，导致可审计的推导过程未得到充分测量。我们介绍了BigFinanceBench，这是一个包含928项专家撰写的开放式金融研究任务的基准，每项任务都将真实参考答案与一个分数加权的评分标准配对，该评分标准将推导过程分解为可独立检查的步骤。BigFinanceBench是基于工作流程的，因为它评估的是完整的推导过程，而不仅仅是最终输出。在36,241个评分点中，该基准支持部分得分评估和分析师工作流程中失败的定位。评估十个当前前沿和开放权重的代理，我们发现有相当大的提升空间：最佳系统仅达到58.8%的评分，最终答案的准确性是推导质量的有用但不完全的代理，模型能力在金融工作流程中变化不均。

View on arXiv Download PDF AI Translation

cs.AI / 74 / 2606.03841

EvoDS: Self-Evolving Autonomous Data Science Agent with Skill Learning and Context Management

EvoDS：具有技能学习和上下文管理的自我进化自主数据科学代理

Yang, Zherui, Liu, Fan, Ning, Yansong, Liu, Hao

Abstract

Recent progress in Large Language Model (LLM) agents has enabled promising advances in automated data science. However, existing approaches remain fundamentally limited by their static action sets and lack of principled long-horizon context management, hindering their ability to accumulate reusable experience across tasks and operate reliably in multi-stage, iterative data science pipelines. To address these challenges, we introduce EvoDS, a self-evolving autonomous data science agent that learns to expand its skills and adaptively managing long-term context through agentic reinforcement learning. Specifically, EvoDS introduces two key strategies: (1) Autonomous Skill Acquisition (ASA) mechanism, which enables agents to synthesize, validate, and reuse executable skills; and (2) Adaptive Context Compression (ACC) strategy, which treats context management as a learned control problem rather than passive truncation. These strategies are orchestrated within a two-stage multi-agent training scheme, enabling EvoDS to autonomously improve over time. Theoretically, we prove that EvoDS's hierarchical design reduces tool-selection error, and its optimization objective aligns with an information bottleneck principle, ensuring efficient context use. Empirically, EvoDS outperforms state-of-the-art open-source data science agents by an average of 28.9% across four diverse benchmarks while eliminating out-of-token failures. Our code and data are available at https://github.com/usail-hkust/EvoDS.

Chinese Translation

近期大型语言模型（LLM）代理的进展为自动化数据科学带来了有希望的突破。然而，现有的方法在其静态动作集和缺乏原则性的长远上下文管理方面仍然存在根本性限制，这阻碍了它们在任务间积累可重用经验的能力，并在多阶段、迭代的数据科学流程中可靠运行。为了解决这些挑战，我们提出了EvoDS，一种自我进化的自主数据科学代理，能够通过代理强化学习学习扩展其技能并自适应地管理长期上下文。具体而言，EvoDS引入了两个关键策略：（1）自主技能获取（Autonomous Skill Acquisition, ASA）机制，使代理能够合成、验证和重用可执行技能；（2）自适应上下文压缩（Adaptive Context Compression, ACC）策略，将上下文管理视为一个学习控制问题，而不是被动截断。这些策略在一个两阶段的多代理训练方案中进行协调，使EvoDS能够随着时间的推移自主改进。从理论上讲，我们证明了EvoDS的层次设计减少了工具选择错误，其优化目标与信息瓶颈原理一致，确保了上下文的有效使用。从经验上看，EvoDS在四个不同基准测试中平均超越了最先进的开源数据科学代理28.9%，同时消除了超出令牌的失败。我们的代码和数据可在 https://github.com/usail-hkust/EvoDS 获取。

View on arXiv Download PDF AI Translation

cs.AI / 75 / 2606.03858

PyraMathBench: Evaluating and Improving Mathematical Capability in Large Language Models

PyraMathBench：评估和提升大型语言模型的数学能力

Ouyang, Zetian, Wang, Linlin, de Melo, Gerard, He, Liang

Abstract

Despite the pivotal role of numerical reasoning as the cornerstone of mathematical capabilities in large language models (LLMs) across applications, few benchmarks evaluate LLMs by integrating numerical processing and mathematical reasoning, hindering the interpretability of failures in math tasks. We introduce PyraMathBench, a comprehensive hierarchical benchmark with 32,505 questions derived from 7,404 math word problems, spanning 4 key cognitive aspects, 14 subcategories, and 2 modalities. Experiments reveal that LLMs' performance is severely compromised by inadequate numerical computation and weak handling of abstract numerical questions. To address this, we propose the Smart Optimization & Learning-based VErsatile module (SOLVE) and Interactive Relative Policy Optimization (IRPO), which enhance LLMs' numerical-mathematical synergy via efficient tool calls (fuzzy matching and low-quality call rejection). Comparative experiments show Qwen-2.5 achieves a 5.0 score improvement with SOLVE and IRPO training.

Chinese Translation

尽管数值推理在大型语言模型（LLMs）的数学能力中扮演着基础性角色，且在各类应用中至关重要，但目前很少有基准通过整合数值处理和数学推理来评估LLMs，这限制了对数学任务失败的可解释性。我们提出了PyraMathBench，这是一个综合性的分层基准，包含32,505个问题，源自7,404个数学文字题，涵盖4个关键认知方面、14个子类别和2种模式。实验表明，LLMs的表现受到不充分的数值计算和对抽象数值问题处理能力不足的严重影响。为了解决这一问题，我们提出了基于智能优化与学习的多功能模块（Smart Optimization & Learning-based VErsatile module, SOLVE）和互动相对策略优化（Interactive Relative Policy Optimization, IRPO），通过高效的工具调用（模糊匹配和低质量调用拒绝）来增强LLMs的数值-数学协同。比较实验显示，Qwen-2.5在经过SOLVE和IRPO训练后，得分提高了5.0分。

View on arXiv Download PDF AI Translation

cs.AI / 76 / 2606.03883

Reasoning Structure of Large Language Models

大型语言模型的推理结构

Berdoz, Frédéric, Lanzendörfer, Luca A., Farestam, Fabian, Wattenhofer, Roger

Abstract

Large reasoning models (LRMs) are often evaluated using metrics such as final-answer accuracy or token count. However, identical scores on these metrics can hide fundamentally different reasoning structures. To address this limitation, we introduce a scalable LRM benchmark of logic puzzles and a pipeline that converts unstructured traces into verifiable reasoning graphs of claims and dependencies. This turns reasoning into a structured, measurable object whose topology can be quantitatively analyzed. Building on this, we define a reasoning efficiency metric that quantifies how concentrated the model's logical flow is. Our analysis on open-source reasoning models shows that structural measurements separate behaviors that token count and accuracy conflate, providing a practical tool for diagnosing failure modes and comparing how reasoning scales with puzzle difficulty.

Chinese Translation

大型推理模型（LRMs）通常使用最终答案准确性或标记计数等指标进行评估。然而，在这些指标上相同的得分可能掩盖了根本不同的推理结构。为了解决这一局限性，我们引入了一种可扩展的LRM逻辑难题基准，并提出了一种将非结构化追踪转换为可验证的主张和依赖关系推理图的流程。这将推理转变为一个结构化的、可测量的对象，其拓扑可以进行定量分析。在此基础上，我们定义了一种推理效率指标，用于量化模型逻辑流的集中程度。我们对开源推理模型的分析表明，结构测量能够区分标记计数和准确性混淆的行为，为诊断失败模式和比较推理如何随着难题难度而变化提供了实用工具。

View on arXiv Download PDF AI Translation

cs.AI / 77 / 2606.03906

scTranslation: A Comprehensive Benchmark for Single-Cell Multi-Omics Modality Translation

scTranslation：单细胞多组学模态转换的综合基准

Cheng, Jiabei, Zhou, Jingbo, Xia, Jun, Li, Changkai, Lei, Zhen, Yu, Chang, Li, Stan Z.

Abstract

Simultaneous measurement of multiple omics modalities in single cells enables researchers to gain a more comprehensive understanding of cellular states and regulatory mechanisms. However, due to high experimental costs, significant noise, and incomplete modality coverage, a variety of computational methods for modality translation have emerged in recent years. Despite the development of translation models, there is still a lack of systematic benchmark evaluation in terms of datasets, evaluation metrics, and influencing factors. To address this, we present scTranslation, a comprehensive benchmark for single-cell multi-omics modality translation tasks. It includes diverse translation datasets, integrates state-of-the-art models, and provides a comprehensive evaluation metrics. In addition, we assess model performance under different scenarios, such as feature selection, feature quality, and few-shot settings. These factors significantly affect model performance but have rarely been systematically studied before. Leveraging this benchmark, we conduct a large-scale study of current methods, report many insightful findings that open up new possibilities for future development. The benchmark is open-sourced to facilitate future research. The code is anonymously released at https://github.com/Bunnybeibei/scTranslation.

Chinese Translation

单细胞中多种组学模态的同时测量使研究人员能够更全面地理解细胞状态和调控机制。然而，由于实验成本高、噪声显著以及模态覆盖不完整，近年来出现了多种模态转换的计算方法。尽管翻译模型得到了发展，但在数据集、评估指标和影响因素方面仍缺乏系统的基准评估。为此，我们提出了scTranslation，这是一个针对单细胞多组学模态转换任务的综合基准。它包括多样的翻译数据集，整合了最先进的模型，并提供了全面的评估指标。此外，我们在不同场景下评估模型性能，例如特征选择、特征质量和少样本设置。这些因素显著影响模型性能，但之前很少得到系统研究。借助这一基准，我们对当前方法进行了大规模研究，报告了许多富有洞察力的发现，为未来的发展开辟了新可能。该基准已开源，以促进未来的研究。代码已匿名发布在 https://github.com/Bunnybeibei/scTranslation。

View on arXiv Download PDF AI Translation

cs.AI / 78 / 2606.03918

Hedge-Bench: Benchmarking Agents on Hard, Realistic Tasks Pertaining to Financial Reasoning

Hedge-Bench：在与金融推理相关的困难、现实任务上对智能体进行基准测试

Cho, Eric, Huang, Shawn, Lu, Alice, Lyu, Andy

Abstract

AI agents can increasingly handle the mechanical tasks of financial analysis: retrieving documents, calculating formulas, updating spreadsheets. The harder, more valuable challenge is reasoning through the open-ended questions that define expert Analyst work. Existing benchmarks do not capture this class of problem, and those that attempt to evaluate open-ended reasoning rely on model-judged outputs that introduce noise and circularity. We present Hedge-Bench 1.0: a benchmark of 102 actual, on-the-job tasks grounded in the explicit reasoning traces of professional hedge fund analysts working with relevant information sources. This approach enables deterministic grading against verified expert steps. Frontier models and agents score below 16\% on the benchmark. We publish the dataset and evaluation harness at github.com/Trata-Inc/trata-hedge-bench.

Chinese Translation

人工智能智能体在金融分析的机械任务中表现越来越出色：检索文档、计算公式、更新电子表格。然而，更具挑战性且更有价值的任务是推理那些定义专家分析师工作的开放性问题。现有的基准测试未能捕捉这一类问题，而那些试图评估开放性推理的基准则依赖于模型判断的输出，这引入了噪声和循环性。我们提出了Hedge-Bench 1.0：一个基于专业对冲基金分析师在相关信息源上工作的明确推理轨迹的102个实际在职任务的基准测试。这种方法使得可以针对经过验证的专家步骤进行确定性评分。前沿模型和智能体在该基准测试中的得分低于16%。我们将在github.com/Trata-Inc/trata-hedge-bench上发布数据集和评估工具。

View on arXiv Download PDF AI Translation

cs.AI / 79 / 2606.03937

Entropy Is Not Enough: Unlocking Effective Reinforcement Learning for Visual Reasoning via Vision-Anchored Token Selection

熵不足以解决问题：通过视觉锚定的标记选择解锁有效的视觉推理强化学习

Jin, Senjie, Wang, Peixin, Liu, Boyang, Fan, Xiaoran, Li, Shuo, Xi, Zhiheng, Zhang, Jiazheng, Zhou, Yuhao, Gui, Tao, Zhang, Qi, Huang, Xuanjing

Abstract

While token-level entropy is commonly recognized as effective for credit assignment in text-only reinforcement learning with verifiable rewards (RLVR), it remains unclear whether this mechanism still holds in visual reasoning. Our controlled study shows that this mechanism collapses in visual reasoning due to the omission of vision-sensitive tokens with naturally low entropy. Although existing multimodal RL methods increasingly acknowledge the importance of visual perception, they struggle to satisfy the inherent demand for interleaving precise perceptual grounding with semantic reasoning, either lacking systematic visual measurements or overlooking that token entropy primarily drives semantic exploration. To address this, we introduce VEPO (Vision-Entropy token-selection for Policy Optimization), an effective RL framework explicitly integrating visual sensitivity with token entropy via a principled multiplicative coupling, where VEPO redirects gradient credit toward tokens which are simultaneously visually grounded and highly informative. Extensive experiments demonstrate VEPO's leading performance, significantly outperforming the entropy-only baseline by 2.28 points at 7B-scale and 3.15 points at 3B-scale. Ablations further substantiate the soundness of our method.

Chinese Translation

虽然在仅文本的强化学习与可验证奖励（RLVR）中，标记级熵通常被认为对信用分配有效，但在视觉推理中这一机制是否依然有效尚不明确。我们的控制研究表明，由于忽略了自然熵较低的视觉敏感标记，这一机制在视觉推理中崩溃。尽管现有的多模态强化学习方法越来越重视视觉感知的重要性，但它们在满足将精确的感知基础与语义推理交织在一起的内在需求方面仍然存在困难，要么缺乏系统的视觉测量，要么忽视了标记熵主要驱动语义探索这一事实。为了解决这个问题，我们引入了VEPO（视觉-熵标记选择用于策略优化），这是一个有效的强化学习框架，通过原则性的乘法耦合，明确地将视觉敏感性与标记熵结合在一起，其中VEPO将梯度信用重新引导到那些同时具有视觉基础和高度信息量的标记上。大量实验表明，VEPO的表现领先，显著超越了仅使用熵的基线，在7B规模上提高了2.28分，在3B规模上提高了3.15分。消融实验进一步证实了我们方法的合理性。

View on arXiv Download PDF AI Translation

cs.AI / 80 / 2606.03988

Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Models

想象感知标记增强多模态语言模型的空间推理能力

Bigverdi, Mahtab, Li, Lindsey, Huang, Weikai, Liu, Yiming, Cho, Jaemin, Zhang, Jieyu, Kundu, Tuhin, Kim, Chris Dangjoo, Luo, Zelun, Shapiro, Linda, Krishna, Ranjay

Abstract

Vision language models (VLMs) excel at many tasks but still struggle with spatial reasoning when critical information is not directly observable. Many such problems require imaginative perception: inferring what would be seen from an unseen viewpoint, tracing paths through occluded spaces, or integrating partial observations into a coherent spatial representation. We introduce Imaginative Perception Tokens (IPT), intermediate perceptual representations that externalize what a VLM would perceive under alternative spatial configurations while remaining consistent with the observed input. To study this capability, we formulate three tasks, Perspective Taking (PET), Path Tracing (PT), and Multiview Counting (MVC), and construct datasets of approximately 20K examples with ground truth imaginations, answers, and evaluation benchmarks. Using the unified VLM BAGEL as the backbone, IPT supervision consistently improves spatial reasoning and often outperforms textual chain of thought training, even without generating images at inference time. On MVC, IPT improves accuracy by 3.4% and achieves competitive performance with strong closed-source models on PT. We further find that combining IPT and label-only supervision yields additional gains, whereas textual chain of thought can substantially degrade performance, suggesting a modality mismatch when spatial computation is forced through language. Overall, IPT provides a principled supervision signal for reasoning about unobserved spatial structure, improving generalization while producing interpretable intermediate representations.

Chinese Translation

视觉语言模型（VLMs）在许多任务中表现出色，但在关键的信息不可直接观察时，仍然在空间推理方面面临挑战。许多此类问题需要想象感知：推断从未见视角所能看到的内容、在遮挡空间中追踪路径，或将部分观察整合为一致的空间表示。我们引入了想象感知标记（Imaginative Perception Tokens, IPT），这是一种中间感知表示，外化了VLM在不同空间配置下的感知，同时与观察到的输入保持一致。为了研究这一能力，我们制定了三个任务：视角转换（Perspective Taking, PET）、路径追踪（Path Tracing, PT）和多视图计数（Multiview Counting, MVC），并构建了约20K个示例的数据集，包含真实想象、答案和评估基准。以统一的VLM BAGEL作为基础，IPT监督持续改善空间推理，并且通常优于文本链式思维训练，即使在推理时不生成图像。在MVC任务中，IPT的准确率提高了3.4%，并在PT任务中与强大的闭源模型达到了竞争性能。我们进一步发现，结合IPT和仅标签监督可以带来额外的提升，而文本链式思维可能会显著降低性能，这表明在通过语言强制进行空间计算时存在模态不匹配。总体而言，IPT为推理未观察到的空间结构提供了一个有原则的监督信号，改善了泛化能力，同时产生可解释的中间表示。

View on arXiv Download PDF AI Translation

计算语言学 (Computation and Language)

cs.CL / 1 / 2606.02584

IdiomX A Multilingual Benchmark for Idiom Understanding, Retrieval, and Interpretation

IdiomX：一个多语言的习语理解、检索和解释基准

Sharara, Ayman Ali

Abstract

Idiomatic expressions remain a persistent challenge for natural language processing because their meanings are often non-compositional, context-dependent, and difficult to align across languages. Existing idiom resources are often limited in scale, contextual diversity, or multilingual coverage, restricting their utility for modern language models. We introduce IdiomX, a large-scale multilingual benchmark for idiom understanding, retrieval, and interpretation, constructed through a reproducible multi-stage pipeline combining lexical resource extraction, large-scale normalization, controlled large language model enrichment, and structured validation. The resulting dataset contains over 190K contextualized examples spanning 12K+ idioms, with aligned English, Arabic, and French semantic representations, idiomatic and literal usage labels, and rich linguistic metadata. Building on this resource, we define a unified four-task benchmark covering idiom detection, context-to-idiom retrieval, Arabic-to-English idiom retrieval, and idiom interpretation, extending evaluation from figurative recognition to semantic grounding and explainable meaning retrieval. Experiments show that contextual transformer models substantially improve idiom detection, while hybrid retrieval and reranking architectures significantly strengthen both monolingual and cross-lingual idiom retrieval. Results further demonstrate that idiom interpretation can be effectively modeled as a semantic retrieval task, introducing interpretability as a complementary benchmark dimension. Overall, IdiomX provides a scalable benchmark for studying idiomatic language as a progression from detection to retrieval and semantic interpretation, and offers a modular framework extensible to additional languages and figurative reasoning tasks

Chinese Translation

习语表达在自然语言处理领域仍然是一个持续的挑战，因为它们的含义往往是非组合性的、依赖于上下文的，并且在不同语言之间难以对齐。现有的习语资源通常在规模、上下文多样性或多语言覆盖方面有限，限制了它们对现代语言模型的实用性。我们介绍了IdiomX，一个大规模的多语言习语理解、检索和解释基准，通过可重复的多阶段流程构建，结合了词汇资源提取、大规模规范化、受控的大型语言模型增强和结构化验证。最终生成的数据集包含超过190K个上下文化的例子，涵盖12K+个习语，具有对齐的英语、阿拉伯语和法语语义表示、习语和字面用法标签，以及丰富的语言元数据。在此基础上，我们定义了一个统一的四任务基准，涵盖习语检测、上下文到习语的检索、阿拉伯语到英语的习语检索和习语解释，将评估从比喻识别扩展到语义基础和可解释的意义检索。实验表明，上下文变换器模型显著提高了习语检测的效果，而混合检索和重排序架构显著增强了单语和跨语言的习语检索。结果进一步表明，习语解释可以有效地建模为语义检索任务，引入可解释性作为一个补充的基准维度。总体而言，IdiomX提供了一个可扩展的基准，用于研究习语语言，从检测到检索和语义解释的进展，并提供了一个可扩展到其他语言和比喻推理任务的模块化框架。

View on arXiv Download PDF AI Translation

cs.CL / 2 / 2606.02741

Greener Than Humans? Environmental Attitudes in Large Language Models

比人类更环保？大型语言模型中的环境态度

Kunkel, Stefanie, Hartwig, Tilman, Voss, Marcus, Schütt, Emma K., Gellrich, Angelika

Abstract

Large language models (LLMs) are increasingly used in sustainability-related decision support, reporting, and public communication, yet little systematic evidence exists on the environmental attitudes embedded in their outputs. This paper develops a benchmark for evaluating environmental cognition, affect, and behavioural recommendations in LLMs and applies it to 31 widely used proprietary and open-weight models. Drawing on questions from established environmental awareness surveys and additional sustainability-related behavioural measures, we compare LLM responses 1) among models and 2) between models and human survey benchmarks from Germany. We assess their robustness across prompting conditions. We find that many LLMs align more closely with environmentally progressive attitudes than the average survey respondent, exhibiting higher levels of environmental affect and cognition and recommending behaviours associated with substantial potential CO2 reductions. At the same time, we observe no systematic relationship between sustainability-oriented responses and model origin, size, or release context. However, models exhibit contextual sensitivity, controlled by persona-based prompting and show sycophantic shifts mirroring user-specified ideological positions, which raises concerns about steerability and normative reliability in real-world deployments. Our findings provide a reusable evaluation framework for assessing sustainability-related value alignment in LLMs and highlight the importance of governance, transparency, and critical oversight as AI systems become increasingly embedded in sustainability transformations and public decision-making.

Chinese Translation

大型语言模型（LLMs）在可持续性相关的决策支持、报告和公共传播中越来越多地被使用，但关于其输出中嵌入的环境态度的系统性证据却很少。本文开发了一个评估LLMs中环境认知、情感和行为建议的基准，并将其应用于31个广泛使用的专有和开放权重模型。我们借鉴了已建立的环境意识调查中的问题以及其他与可持续性相关的行为测量，比较了LLM的响应：1）在模型之间，2）在模型与来自德国的人类调查基准之间。我们评估了它们在不同提示条件下的稳健性。我们发现，许多LLMs的环境态度与环境进步的态度更为一致，表现出更高的环境情感和认知水平，并推荐与显著潜在CO2减排相关的行为。同时，我们观察到可持续性导向的响应与模型的来源、规模或发布背景之间没有系统性关系。然而，模型表现出上下文敏感性，这由基于角色的提示控制，并显示出迎合用户指定意识形态立场的变化，这引发了对在现实世界部署中可引导性和规范可靠性的担忧。我们的研究结果提供了一个可重用的评估框架，用于评估LLMs中与可持续性相关的价值对齐，并强调了治理、透明度和批判性监督的重要性，因为人工智能系统在可持续性转型和公共决策中变得越来越深入。

View on arXiv Download PDF AI Translation

cs.CL / 3 / 2606.02750

On the Persistent Effects of Lexicality in Large Language Mod

大型语言模型中词汇性的持久影响

Rizwan, Hammad, Haider, Muhammad Umair, Subramani, Nishant, Diab, Mona T., Siddique, A. B., Sajjad, Hassan

Abstract

Representations extracted from large language models (LLMs) play an important role in many downstream applications. However, the structure of these representations is often influenced by lexical overlap rather than semantic content. Our understanding of the relationship between this lexical influence and semantic content, and its implications for downstream tasks, remains limited. In this work, we investigate representations to quantify the effect of lexical overlap relative to semantic content. We consider several adversarial semantic stress tests and further connect our findings to the information theory perspective. We find that lexical influence extends across the depth of models, consistently across architectures, training regimes, and objective functions, including the models trained for semantic similarity. Moreover, we observe a mid-depth region in which both lexical and semantic signals degrade simultaneously, indicating a transitional regime where representations are poor for both surface form and meaning. We further demonstrate the effect of lexical influence on downstream uses of LLMs using summarization and model editing as a case study.

Chinese Translation

从大型语言模型（LLMs）中提取的表示在许多下游应用中发挥着重要作用。然而，这些表示的结构往往受到词汇重叠而非语义内容的影响。我们对这种词汇影响与语义内容之间关系的理解，以及其对下游任务的影响，仍然有限。在本研究中，我们调查了表示，以量化词汇重叠相对于语义内容的影响。我们考虑了几种对抗性语义压力测试，并进一步将我们的发现与信息理论视角联系起来。我们发现，词汇影响贯穿模型的深度，在不同架构、训练方案和目标函数中始终如一，包括为语义相似性训练的模型。此外，我们观察到在中间深度区域，词汇和语义信号同时下降，表明这是一个过渡阶段，在该阶段表示在表面形式和意义上都较差。我们还通过使用摘要和模型编辑作为案例研究，进一步展示了词汇影响对LLMs下游应用的影响。

View on arXiv Download PDF AI Translation

cs.CL / 4 / 2606.02776

Topics as Proxies for Sociodemographics: How Conversational Context Affects LLM Answers

主题作为社会人口统计的代理：对话上下文如何影响大型语言模型的回答

Neplenbroek, Vera, Sarti, Gabriele, Bisazza, Arianna, Fernández, Raquel

Abstract

When large language models (LLMs) are used in high-stakes scenarios, such as legal, medical and financial advice, even a single conversation history is enough to drive differences in outcomes between users. Prior work has demonstrated that this results in outcome disparities between sociodemographic groups, with some groups receiving more advantageous outcomes than others. In this work, we demonstrate that LLMs actually struggle to infer user sociodemographics from a single conversation history and that although there are disparities between sociodemographic groups, they are minimal in magnitude. To investigate what the main driver of these disparities is, we compare user sociodemographics to a range of (psycho)linguistic features of conversations, including conversation topic, emotions, and readability. We find that conversation topics are most predictive of LLM-generated advice within a conversational context, which, to some extent, function as proxies for sociodemographic groups and often affect advice in unpredictable ways. This is cause for concern and highlights the need for future research to better understand and, if needed, mitigate the effect of conversational context on LLM outputs in high-stakes scenarios.

Chinese Translation

在高风险场景中使用大型语言模型（LLMs），如法律、医疗和财务建议，即使是单一的对话历史也足以导致用户之间结果的差异。先前的研究表明，这导致了不同社会人口统计群体之间的结果差异，某些群体获得的结果比其他群体更有利。在本研究中，我们展示了LLMs实际上难以从单一的对话历史中推断用户的社会人口统计特征，尽管不同社会人口统计群体之间存在差异，但其幅度很小。为了探讨这些差异的主要驱动因素，我们将用户的社会人口统计特征与一系列（心理）语言学特征进行比较，包括对话主题、情感和可读性。我们发现，在对话上下文中，对话主题是对LLM生成建议的最强预测因素，在某种程度上，它们充当了社会人口统计群体的代理，并且常常以不可预测的方式影响建议。这引起了我们的关注，并强调了未来研究的必要性，以更好地理解并在必要时减轻对话上下文在高风险场景中对LLM输出的影响。

View on arXiv Download PDF AI Translation

cs.CL / 5 / 2606.02780

Do Value Vectors in Deep Layers Need Context from the Residual Stream?

深层中的值向量是否需要来自残差流的上下文？

He, Muyu, Liu, Yuchen, Huang, Qingya, Zhang, Li

Abstract

The success of the transformer architecture as the backbone of modern LLMs is in large part due to its use of attention layers. An attention layer follows the standard neural network paradigm: it takes the residual stream as input and thereby produces context-dependent query, key, and value vectors. However, we find that model performance meaningfully improves when deeper layers learn only a context-free value vector to preserve the original token information, without drawing on any context from the residual stream. When the model has access to this context-free value vector, adding back the context-dependent component provides little additional benefit for aggregate benchmark performance. Such context-free value vectors can be stored as sparse model parameters, eliminating the need to recompute or persistently cache these values. Through systematic ablations on the key design choices for such context-free value vectors, we propose Bank of Values (BoV), a new way of computing value vectors in attention by learning a lookup table of token-specific value vectors for each of the last third of layers. Across 135M and 780M models, BoV improves validation loss over standard attention and, at 780M, the average score across 21 benchmarks, matching the previous best method that adds token information to the value vector with less compute and memory.

Chinese Translation

变换器架构作为现代大型语言模型（LLMs）的基础，其成功在很大程度上归功于其使用的注意力层。注意力层遵循标准神经网络范式：它以残差流作为输入，从而生成依赖上下文的查询、键和值向量。然而，我们发现，当深层仅学习一个无上下文的值向量以保留原始标记信息，而不依赖于残差流中的任何上下文时，模型性能显著提高。当模型能够访问这个无上下文的值向量时，重新引入依赖上下文的组件对整体基准性能几乎没有额外好处。这种无上下文的值向量可以作为稀疏模型参数存储，消除重新计算或持久缓存这些值的需要。通过对这种无上下文值向量的关键设计选择进行系统性消融，我们提出了值库（Bank of Values, BoV），这是一种通过学习每个最后三分之一层的标记特定值向量查找表来计算注意力中的值向量的新方法。在135M和780M模型中，BoV在验证损失上优于标准注意力，并且在780M模型中，在21个基准测试中的平均得分与之前将标记信息添加到值向量的最佳方法相匹配，同时计算和内存开销更低。

View on arXiv Download PDF AI Translation

cs.CL / 6 / 2606.02806

Translating Classical Poetry into Modern Prose

古典诗歌翻译为现代散文

Kranti, Chalamalasetti, Vajjala, Sowmya

Abstract

We introduce Padyam2Gadyam, a dataset for the task of poem-to-prose translation from 13th-17th Century Telugu Classical Poetry to contemporary Telugu and English prose. The dataset consists of 600 poems and their human-verified Telugu and English prose translations. We evaluated 5 contemporary Large Language Models (LLMs) on their ability to do poem-to-prose translation into Telugu and English. Our results indicate that while there are differences across LLMs, their overall performance leave a large room for improvement in both languages. Through qualitative analysis, we discuss the the capabilities and limitations of contemporary MT evaluation approaches for this task.

Chinese Translation

我们介绍了 Padyam2Gadyam，这是一个用于将13至17世纪泰卢固古典诗歌翻译为现代泰卢固语和英语散文的任务的数据集。该数据集包含600首诗及其经过人工验证的泰卢固语和英语散文翻译。我们评估了5个现代大型语言模型（LLMs）在将诗歌翻译为泰卢固语和英语散文的能力。我们的结果表明，尽管不同的LLMs之间存在差异，但它们在这两种语言上的整体表现仍有很大的改进空间。通过定性分析，我们讨论了现代机器翻译评估方法在这一任务中的能力和局限性。

View on arXiv Download PDF AI Translation

cs.CL / 7 / 2606.02837

Fixing FOLIO and MALLS: Verified Annotations and an LLM-assisted Framework to Focus Human Relabeling

修正 FOLIO 和 MALLS：经过验证的注释与一个 LLM 辅助框架以聚焦人类重新标注

Brunello, Andrea, Curaba, Cristian, Geatti, Luca, Mignani, Michele, Montanari, Angelo, Saccomanno, Nicola

Abstract

Accurate translation from Natural Language to First-Order Logic (NL-to-FOL) underpins neurosymbolic AI systems and Natural Language Inference (NLI), making the quality of NL-to-FOL benchmarks essential -- yet these datasets have never been rigorously audited. Our first contribution is to present a systematic human inspection of the validation split of \textsf{FOLIO} and a subset of \textsf{MALLS} test instances, finding that approximately 39% and 36% of entries, respectively, contain incorrect FOL formalizations (i.e., ground truth labels), with additional rates of ambiguous NL sentences (16.4% and 48%) and incorrect NLI labels in \textsf{FOLIO} (8.4%). Our second contribution is to develop and release corrected ground truths for such datasets, showing that annotation errors distort model evaluation on a reference benchmark task: testing three state-of-the-art LLMs (Gemma~4 31B-it, Qwen3-30B-A3B, and GPT-4o-mini) with the corrected ground truths yields accuracy gains from +9 to +22 percentage points. Motivated by these findings, we propose an LLM-based framework to support humans in manual reviewing NL-to-FOL datasets. By directing reviewers toward the most error-prone instances, we empirically show that it is possible to achieve 90% dataset accuracy after reviewing fewer than 24% of instances, compared to over 70% required by unguided review. We release all human-verified annotations and the code for our framework.

Chinese Translation

从自然语言到一阶逻辑（NL-to-FOL）的准确翻译是神经符号人工智能系统和自然语言推理（NLI）的基础，这使得 NL-to-FOL 基准的质量至关重要——然而这些数据集从未经过严格审计。我们的第一个贡献是对 extsf{FOLIO} 的验证分割和 extsf{MALLS} 测试实例的一个子集进行系统的人为检查，发现大约 39% 和 36% 的条目分别包含不正确的 FOL 形式化（即真实标签），同时还发现模糊的自然语言句子的比例为 16.4% 和 48%，以及 extsf{FOLIO} 中不正确的 NLI 标签（8.4%）。我们的第二个贡献是开发并发布这些数据集的修正真实标签，显示注释错误扭曲了模型在参考基准任务上的评估：使用修正后的真实标签测试三种最先进的 LLM（Gemma~4 31B-it、Qwen3-30B-A3B 和 GPT-4o-mini）可获得 +9 到 +22 个百分点的准确率提升。基于这些发现，我们提出了一个基于 LLM 的框架，以支持人类手动审查 NL-to-FOL 数据集。通过引导审查者关注最容易出错的实例，我们实证表明，在审查不到 24% 的实例后，可以实现 90% 的数据集准确率，而无指导审查则需要超过 70%。我们发布了所有经过人类验证的注释和我们框架的代码。

View on arXiv Download PDF AI Translation

cs.CL / 8 / 2606.02859

Economy of Minds: Emerging Multi-Agent Intelligence with Economic Interactions

思想经济学：具有经济互动的新兴多智能体智能

Qi, Zhenting, Su, Huangyuan, Qu, Ao, Wang, Chenyu, Yao, Yu, Zheng, Han, Chattopadhyay, Kushal, Xu, Guowei, Wang, Zihan, Ye, Weirui, Reddi, Vijay Janapa, Li, Ju, Liang, Paul Pu, Lakkaraju, Himabindu, Kakade, Sham, Du, Yilun

Abstract

How can a population of agents self-orchestrate and self-adapt into stronger collective intelligence without centralized control? Inspired by Friedrich Hayek's economic theory of decentralized coordination in markets, we study this question through an agent economy in which agents compete via auctions for the right to act, exchange payments, and accumulate wealth from environmental rewards. These simple economic signals induce decentralized credit assignment, driving planning without global orchestration or explicit communication protocols. The population evolves through economic selection: effective agents accumulate wealth and are mutated via exploitation, while ineffective ones go bankrupt and are replaced via exploration. We show that, initialized with weak agents, the economy produces emergent multi-step reasoning strategies and outperforms stronger monolithic baselines across five agentic tasks, including mathematical reasoning, financial research, scientific research, accelerator design, and distributed-system optimization. We further provide theoretical insights into how economic dynamics shape agent behaviors, linking local incentives to long-term global performance. Our results suggest a new path to multi-agent intelligence: rather than engineering coordination, we can design decentralized incentive structures under which it automatically emerges.

Chinese Translation

如何使一群智能体在没有集中控制的情况下自我协调和自我适应，从而形成更强的集体智能？受到弗里德里希·哈耶克（Friedrich Hayek）关于市场中去中心化协调的经济理论的启发，我们通过一个智能体经济体来研究这个问题，在这个经济体中，智能体通过拍卖竞争行动权，交换支付，并从环境奖励中积累财富。这些简单的经济信号诱导去中心化的信用分配，推动规划而无需全球协调或明确的通信协议。该群体通过经济选择进化：有效的智能体积累财富并通过利用进行变异，而无效的智能体则破产并通过探索被替代。我们展示了在以弱智能体初始化的情况下，该经济体产生了新兴的多步推理策略，并在五个智能体任务中超越了更强的单一基线，包括数学推理、金融研究、科学研究、加速器设计和分布式系统优化。我们进一步提供了理论见解，探讨经济动态如何塑造智能体行为，将局部激励与长期全球表现联系起来。我们的结果表明了一条通向多智能体智能的新路径：与其工程协调，不如设计去中心化的激励结构，使其自动涌现。

View on arXiv Download PDF AI Translation

cs.CL / 9 / 2606.02871

Adaptive Latent Agentic Reasoning

自适应潜在代理推理

Jung, Dongwon, Shi, Peng, Zhang, Yi, Zhang, Junshan, Chen, Muhao

Abstract

Large reasoning models improve performance by generating extended chain-of-thought (CoT) reasoning, but this behavior becomes inefficient when applied to LLM agents. Current LLM agents often generate verbose textual reasoning at every decision step and allocate reasoning effort nearly uniformly across turns, leading to substantial inefficiency in multi-turn agentic trajectories. We propose Adaptive Latent Agentic Reasoning (ALAR), a dual-mode framework that uses compact latent reasoning for routine turns and selectively escalates to explicit chain-of-thought when deeper deliberation is needed. ALAR learns latent reasoning by using the agent's actions as supervision anchors and is further optimized to use latent reasoning when it is sufficient for task success and reserve explicit CoT for harder decisions. Experiments on agentic search and tool-use benchmarks show that ALAR maintains comparable or better task accuracy while substantially reducing generated tokens by up to 43.6% in search and 84.6% in tool use. These results demonstrate that ALAR improves the accuracy-efficiency trade-off of LLM agents by reducing unnecessary textual reasoning while preserving explicit deliberation for harder decision steps.

Chinese Translation

大型推理模型通过生成扩展的思维链（CoT）推理来提高性能，但当应用于大语言模型（LLM）代理时，这种行为变得低效。目前的LLM代理在每个决策步骤中往往生成冗长的文本推理，并在每个回合中几乎均匀分配推理努力，导致多回合代理轨迹中的显著低效。我们提出了自适应潜在代理推理（ALAR），这是一种双模式框架，在常规回合中使用紧凑的潜在推理，并在需要更深入的思考时选择性地升级为显式的思维链。ALAR通过将代理的行动作为监督锚点来学习潜在推理，并进一步优化以在任务成功时使用潜在推理，而将显式的思维链保留用于更困难的决策。在代理搜索和工具使用基准上的实验表明，ALAR在保持可比或更好的任务准确性的同时，显著减少了生成的标记，搜索中减少了多达43.6%，工具使用中减少了84.6%。这些结果表明，ALAR通过减少不必要的文本推理，同时为更困难的决策步骤保留显式的深思熟虑，从而改善了LLM代理的准确性与效率的权衡。

View on arXiv Download PDF AI Translation

cs.CL / 10 / 2606.02907

Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States

线性探测器检测任务格式，而非语言模型隐藏状态中的推理模式

Sahoo, Subramanyam, Jain, Vinija, Chadha, Aman, Chaudhary, Divya

Abstract

Linear probing of large language model (LLM) hidden states is widely used to claim that models learn distinct representations for different reasoning types. We test this by probing Qwen3-14B on three benchmarks spanning the classical trichotomy: LogiQA 2.0 (deductive), ARC-Challenge (inductive), and $\alpha$NLI (abductive). At layer 32 of 40, linear probes achieve 100\% cross-validated accuracy with well-separated geometry (intrinsic dimensionalities: 20.6, 28.5, 33.6; convex hull contamination $\leq$1.5\%). However, this separation is entirely driven by format confounds. Residualizing source identity, option count, and response length reduces accuracy to chance. Trace-anchor similarity indicates largely shared reasoning across tasks (42.5\% agreement vs.\ 33.3\% chance), and causal steering with random controls ($n=20$) shows no functional link between geometry and reasoning mode ($p=0.286$). Thus, high probe accuracy reflects task format rather than computational structure, motivating routine format deconfounding in mechanistic interpretability.

Chinese Translation

对大型语言模型（LLM）隐藏状态的线性探测广泛用于声称模型为不同的推理类型学习了不同的表征。我们通过对 Qwen3-14B 在三个基准测试上的探测来验证这一点，这些基准测试涵盖了经典的三分法：LogiQA 2.0（演绎）、ARC-Challenge（归纳）和 $eta$NLI（溯因）。在40层中的第32层，线性探测器在几何分离良好的情况下实现了100\%的交叉验证准确率（内在维度：20.6，28.5，33.6；凸包污染 $ extless$1.5\%）。然而，这种分离完全是由格式混淆驱动的。去除源身份、选项数量和响应长度的影响后，准确率降至偶然水平。轨迹锚点相似性表明任务之间在推理上有很大程度的共享（42.5\% 的一致性对比 33.3\\% 的偶然性），而使用随机控制的因果引导（$n=20$）显示几何与推理模式之间没有功能性联系（$p=0.286$）。因此，高探测准确率反映的是任务格式而非计算结构，促使在机械解释中常规进行格式去混淆。

View on arXiv Download PDF AI Translation

cs.CL / 11 / 2606.02908

WRIT: Write-Read Intensive Trajectory Synthesis for Multi-Turn User-Facing Agents

WRIT：面向用户的多轮智能体的写读密集轨迹合成

Gu, Hengrui, Han, Xiaotian, Zhou, Kaixiong

Abstract

Multi-turn user-facing agents must infer user intent from incomplete requests, collect missing information through dialogue and tools, and execute valid actions. A training trajectory records this process as an interleaved sequence of user messages, agent responses, tool calls, etc. Synthesizing sufficiently complex trajectory has become a central route to train agents: existing pipelines often increase difficulty by composing multiple user requests into longer tasks, producing write-intensive trajectories that train sequential execution. We argue that a single write decision can itself be difficult when the agent must gather and compare substantial read-tool evidence before its arguments become identifiable, a challenge that write-intensive data alone cannot address. Guided by this insight, we propose WRIT (\uline{W}rite-\uline{R}ead \uline{I}ntensive \uline{T}rajectory Synthesis), a pipeline for synthesizing multi-turn agent training trajectories along two complexity axes: the number of write decisions in a task and the evidence burden of each individual decision. WRIT first generates write-intensive and read-heavy tasks. It then diversifies user behavior instructions to reflect realistic conversational variation, and finally simulates agent-user interactions in an executable environment to produce complete training trajectories. The resulting data trains agents not only for longer task execution, but also for robust, evidence-grounded decision making under high information load. With only 2K synthesized trajectories, a 4B model trained on WRIT outperforms GPT-5.1 no-think on $\tau^2$-bench and substantially reduces inference-time token usage, showing that compact SFT data can convert part of expensive test-time reasoning into efficient agent behavior.

Chinese Translation

面向用户的多轮智能体必须从不完整的请求中推断用户意图，通过对话和工具收集缺失的信息，并执行有效的动作。训练轨迹记录了这一过程，作为用户消息、智能体响应、工具调用等交错的序列。合成足够复杂的轨迹已成为训练智能体的核心途径：现有的流程通常通过将多个用户请求组合成更长的任务来增加难度，从而产生写密集型轨迹以训练顺序执行。我们认为，当智能体必须收集和比较大量的读取工具证据以使其参数可识别时，单个写决策本身可能是困难的，而仅靠写密集型数据无法解决这一挑战。在这一见解的指导下，我们提出了WRIT（Write-Read Intensive Trajectory Synthesis），一个沿着两个复杂性轴合成多轮智能体训练轨迹的流程：任务中的写决策数量和每个单独决策的证据负担。WRIT首先生成写密集型和读重型任务。然后，它多样化用户行为指令，以反映现实对话的变化，最后在可执行环境中模拟智能体与用户的互动，以生成完整的训练轨迹。生成的数据不仅训练智能体执行更长的任务，还训练其在高信息负载下进行稳健、基于证据的决策。仅用2K合成轨迹，基于WRIT训练的4B模型在$ au^2$-bench上超越了GPT-5.1 no-think，并显著减少了推理时的令牌使用，显示出紧凑的SFT数据可以将部分昂贵的测试时推理转化为高效的智能体行为。

View on arXiv Download PDF AI Translation

cs.CL / 12 / 2606.02911

The Ghost Annotator: a Framework to Explore Human Label Variation in Content Moderation through Conformal Prediction

幽灵标注者：通过保形预测探索内容审核中的人类标签变异的框架

Lai, Mirko, Urbinati, Alessandra, Frenda, Simona, Vernero, Fabiana, Stranisci, Marco Antonio

Abstract

Current research primarily focuses on model performance, while comparatively less attention has been devoted to uncertainty estimation, particularly in settings where LLMs are increasingly used to generate annotated data. We introduce a framework combining conformal prediction with Collaborative Filtering-style annotators' representation to model LLM behavior in relation to human annotators and to analyze patterns of agreement and disagreement. Using Non-Conformity Scores, we introduce the Ghost Prediction metric and the Ghost Annotator representation to quantify cases in which model predictions diverge from all available human annotations. We compute cosine similarity measures to explore differences in model behavior across sociodemographic axes. We evaluated four LLMs of different size and families across four content moderation datasets. Our finding shows that while we find that all models uncertainty increases with annotator disagreement, larger models tend to be more confident in the classification of texts that are not aligned with any human annotation. Finally, the Ghost Annotator framework reveals a consistent and robust pattern of demographic misalignment, suggesting a structural bias likely rooted in pretraining corpora.

Chinese Translation

当前的研究主要集中在模型性能上，而对不确定性估计的关注相对较少，特别是在大型语言模型（LLMs）日益被用于生成标注数据的环境中。我们提出了一个框架，将保形预测与协同过滤风格的标注者表示相结合，以建模LLM行为与人类标注者之间的关系，并分析一致性和不一致性的模式。通过使用非一致性评分，我们引入了幽灵预测（Ghost Prediction）指标和幽灵标注者（Ghost Annotator）表示，以量化模型预测与所有可用人类标注之间的偏差情况。我们计算余弦相似性度量，以探索模型行为在社会人口学维度上的差异。我们评估了四种不同规模和类型的LLM在四个内容审核数据集上的表现。我们的发现表明，尽管我们发现所有模型的不确定性随着标注者的不一致性而增加，但较大的模型在分类与任何人类标注不一致的文本时往往表现得更为自信。最后，幽灵标注者框架揭示了一个一致且稳健的人口统计学错位模式，暗示可能存在根植于预训练语料库的结构性偏见。

View on arXiv Download PDF AI Translation

cs.CL / 13 / 2606.02953

Linguistic Productivity in Large Language Models: Models Coerce, but do not Preempt

大型语言模型中的语言生产力：模型强制，但不预先排除

Bonial, Claire, Post, Claire Benet, Michaelis, Laura, Madabushi, Harish Tayyar

Abstract

Usage-based theories of grammars posit that creative productivity of the structures of language is both bolstered and constrained by two distinct frequency signals: entrenchment, stemming from high frequency usage, and preemption, stemming from having never observed a particular linguistic structure in a context where one might expect that structure to appear. Large Language Models are also usage-based, in the sense that the structures of language are learned through exposure to vast amounts of text. Here, we test whether or not the opposing statistical forces of entrenchment and preemption also encourage and constrain linguistic productivity in LLMs. We demonstrate across model architectures that larger models recognize and can reproduce with nonce words constructional productivity (entrenchment) in cases of coercion, wherein the broader constructional context coerces an atypical interpretation of a lexical item. However, we also show that even the largest models do not extend negative evidence to novel language, and statistical preemption does not enable models to avoid overgeneralization of patterns that are semantically felicitous, but never observed in data.

Chinese Translation

基于使用的语法理论认为，语言结构的创造性生产力既受到高频使用所带来的巩固（entrenchment）影响，也受到未曾在期望出现该结构的上下文中观察到特定语言结构所带来的预先排除（preemption）影响。大型语言模型（LLMs）同样是基于使用的，因为语言结构是通过接触大量文本而学习的。在此，我们测试巩固和预先排除这两种对立的统计力量是否也在LLMs中促进和限制语言生产力。我们在不同的模型架构中展示了更大的模型能够识别并使用非词（nonce words）生成具有构造性生产力的情况（巩固），尤其是在强制（coercion）情况下，其中更广泛的构造上下文强制了词汇项的非典型解释。然而，我们也表明，即使是最大的模型也不会将负证据扩展到新语言中，统计预先排除并未使模型避免对在数据中从未观察到但语义上合适的模式的过度概括。

View on arXiv Download PDF AI Translation

cs.CL / 14 / 2606.02955

Fast-dLLM++: Fr\'{e}chet Profile Decoding for Faster Diffusion LLM Inference

Fast-dLLM++：用于更快扩散大语言模型推理的Fréchet配置解码

Kasa, Siva Rajesh, Dai, Yasong, Negi, Sumit, Li, Hongdong

Abstract

Diffusion large language models promise parallel token generation, yet inference remains bottlenecked by deciding which masked tokens can be safely committed together. Fast-dLLM addressed this with KV caching and confidence-guided parallel decoding, but its decoding theory uses a homogeneous high-confidence assumption that effectively reduces each candidate set to its weakest selected token. We argue that this leaves speed on the table because real decoding steps exhibit heterogeneous confidence profiles. We propose \textbf{Fast-dLLM++}, a training-free extension that introduces \emph{Fr\'{e}chet profile decoding}: selecting parallel commit sets from the full sorted confidence profile rather than a single worst-case confidence. The resulting rule is a heterogeneous-confidence generalization of Fast-dLLM's factor selector and it recovers the previous rule exactly in the equal-confidence case and adds a provable \emph{heterogeneity bonus} when the selected tokens have uneven confidences. Fast-dLLM++ leaves the model, diffusion process, and cache implementation entirely unchanged, making it a drop-in replacement for existing Fast-dLLM decoding. Experiments on GSM8K, MATH, HumanEval, and MBPP with the LLaDA-8B model show that the theoretical improvement translates directly into empirical gains: profile-aware selection improves the accuracy--throughput frontier by exploiting safe parallelism that weakest-token rules miss, achieving up to 37\% higher throughput at comparable accuracy. Our anonymous code release is at https://github.com/Ringo-Star/FastdLLM_plusplus.

Chinese Translation

扩散大语言模型承诺实现并行的标记生成，但推理仍然受到决定哪些被掩蔽的标记可以安全地一起提交的瓶颈限制。Fast-dLLM通过键值缓存（KV caching）和基于置信度的并行解码解决了这一问题，但其解码理论使用了均匀的高置信度假设，这实际上将每个候选集简化为其最弱的选定标记。我们认为，这样做会导致速度损失，因为实际的解码步骤表现出异质的置信度配置。我们提出了 extbf{Fast-dLLM++}，这是一个无训练的扩展，引入了 extit{Fréchet配置解码}：从完整的排序置信度配置中选择并行提交集，而不是单一的最坏情况置信度。由此产生的规则是Fast-dLLM因子选择器的异质置信度推广，在均匀置信度情况下，它完全恢复了之前的规则，并在选定标记具有不均匀置信度时增加了可证明的 extit{异质性奖励}。Fast-dLLM++完全不改变模型、扩散过程和缓存实现，使其成为现有Fast-dLLM解码的直接替代品。在GSM8K、MATH、HumanEval和MBPP上使用LLaDA-8B模型的实验表明，理论上的改进直接转化为经验上的收益：基于配置的选择通过利用最弱标记规则所忽视的安全并行性，提高了准确性-吞吐量边界，在可比准确性下实现了高达37 ext{%}的吞吐量提升。我们的匿名代码发布在https://github.com/Ringo-Star/FastdLLM_plusplus。

View on arXiv Download PDF AI Translation

cs.CL / 15 / 2606.02971

EURO-5K: When Does Domain Pretraining Matter? Benchmarking Transformers for EU Reporting Obligation Extraction

EURO-5K：领域预训练何时重要？基于变换器的欧盟报告义务提取基准测试

Koniaris, Marios, Kotronis, Vasileios, Giannini, Eugenia, Tsanakas, Panayiotis

Abstract

Extracting reporting obligations from EU legislation is critical for assessing and reducing regulatory reporting burden. However, distinguishing reporting requirements from structurally similar provisions requires specialised legal understanding. Current legal NLP methods lack specialised datasets with clear guidelines and comparative evaluation of extraction paradigms and domain adaptation strategies. We curate EURO-5K, a corpus of sentence-level reporting obligations and challenging negative examples from 136 EU legislative acts. On this dataset, we train and compare discriminative token-classification models (BERT-style) and generative span-extraction models (LLMs), evaluating both full fine-tuning and parameter-efficient QLoRA against baselines (pattern and dependency-based extraction, few-shot prompting). Results show that fully fine-tuned generic and legal BERT models achieve similar performance (0.89 F1), while fine-tuned LLMs match encoder accuracy for sentence-level extraction. Legal pretraining offers only small gains for generative models. In contrast, it is clearly beneficial when adaptation capacity is constrained, as parameter-efficient tuning of Legal-BERT outperforms its generic counterpart. Learning curve analysis demonstrates that legal pretraining accelerates early learning with minimal data. All approaches converge around 3K samples with diminishing returns thereafter, validating dataset sufficiency. Cross-dataset evaluation on two external regulatory corpora shows that our models behave as specialised reporting obligation extractors rather than generic regulatory classifiers. We release EURO-5K, trained models, and an interactive demo with explainability visualizations and structured RDF export. These demonstrate that both paradigms and parameter-efficient training provide practical tools for regulatory compliance automation.

Chinese Translation

从欧盟立法中提取报告义务对于评估和减少监管报告负担至关重要。然而，从结构上相似的条款中区分报告要求需要专业的法律理解。目前的法律自然语言处理（NLP）方法缺乏具有明确指导方针的专业数据集，以及对提取范式和领域适应策略的比较评估。我们整理了EURO-5K，一个包含来自136项欧盟立法文件的句子级报告义务和具有挑战性的负面示例的语料库。在该数据集上，我们训练并比较了判别性标记分类模型（BERT风格）和生成性跨度提取模型（LLMs），评估了全量微调和参数高效的QLoRA与基线（基于模式和依赖的提取、少量提示）之间的表现。结果表明，完全微调的通用和法律BERT模型的性能相似（F1值为0.89），而微调的LLMs在句子级提取中达到了编码器的准确性。法律预训练对生成模型的提升仅为微小的增益。相反，当适应能力受限时，它显然是有益的，因为法律BERT的参数高效调优优于其通用版本。学习曲线分析表明，法律预训练加速了早期学习，所需数据量最小。所有方法在约3000个样本处收敛，之后收益递减，验证了数据集的充分性。对两个外部监管语料库的跨数据集评估显示，我们的模型表现为专业的报告义务提取器，而非通用的监管分类器。我们发布了EURO-5K、训练模型以及带有可解释性可视化和结构化RDF导出的交互式演示。这些展示了两种范式和参数高效训练为监管合规自动化提供了实用工具。

View on arXiv Download PDF AI Translation

cs.CL / 16 / 2606.02973

Chatbots Output Meaningful (but Problematic) Language

聊天机器人输出有意义（但存在问题）的语言

Stone, Matthew, Stojnić, Una

Abstract

Are utterances by AI chatbots meaningful? Concretely, if a user asks, say, Anthropic's agent Claude, "What is the capital of Spain?" and Claude answers, "Madrid is the capital of Spain," does that sentence have its ordinary meaning -- and does it express a true proposition? Most ordinary users, as well as AI engineers, take the answer to be trivially "yes." However, many cognitive scientists, linguists, and philosophers of language argue that dominant intentionalist accounts of language and meaning deliver the opposite conclusion. Theorists more sympathetic to ordinary users' intuitions have therefore advocated a radical "de-anthropomorphization" of language, revising our understanding of mental states, intentions, and semantic content to capture the intuition that the outputs of LLMs are meaningful. We take a different approach. While we, too, argue that LLM outputs are meaningful, we contend that a proper theory of human language already applies, as is, to current chatbots. Meaning is a low bar: claiming that LLM outputs are meaningful does not require positing mental states, intentions, rationality, or the cognitive capacities requisite for communication in LLMs -- or, indeed, making any other anthropomorphic assumptions. People do have communicative intentions (typically successful ones), but nevertheless, even in humans, language production can depart from what the speaker has in mind. Our view has important consequences for how we should theorize about -- and critically engage with -- both human linguistic output and synthetically generated text. In particular, to say that chatbots produce meaningful text is not by any means to endorse what they output, or to assume that the technology is (or is not) good, powerful, appropriate, or useful.

Chinese Translation

人工智能聊天机器人的发言是否有意义？具体来说，如果用户问，比如说，Anthropic 的代理 Claude，“西班牙的首都是什么？”而 Claude 回答，“马德里是西班牙的首都”，那么这句话是否具有其普通意义——并且它是否表达了一个真实的命题？大多数普通用户以及人工智能工程师认为答案显然是“是”。然而，许多认知科学家、语言学家和语言哲学家则认为，主流的意向主义语言和意义理论得出的结论正好相反。因此，更加同情普通用户直觉的理论家们主张对语言进行激进的“去人性化”修正，重新理解心理状态、意图和语义内容，以捕捉大型语言模型（LLMs）输出有意义的直觉。我们采取了不同的方法。虽然我们也认为 LLM 的输出是有意义的，但我们主张，现有的关于人类语言的适当理论已经适用于当前的聊天机器人。意义是一个低门槛：声称 LLM 的输出是有意义的并不需要假设心理状态、意图、理性或进行 LLM 通信所需的认知能力——或者，实际上，做出任何其他人性化的假设。人们确实有交际意图（通常是成功的），但即便如此，即使在人的语言生产中，也可能与说话者所想的有所偏离。我们的观点对我们如何理论化——以及批判性地参与——人类语言输出和合成生成文本具有重要意义。特别是，声称聊天机器人生成有意义的文本并不意味着支持它们的输出，或假设该技术是（或不是）良好、有力、适当或有用的。

View on arXiv Download PDF AI Translation

cs.CL / 17 / 2606.02976

Memory Retrieval for Changing Preferences

记忆检索与变化偏好的关系

Qin, Yuehan, Li, Li, Song, Linxin, Yang, Wei, Li, Jiate, Yang, Yuqing, Zhao, Yue

Abstract

Long-context dialogue systems must decide both when to access memory and which parts of the interaction history are relevant. Existing approaches typically rely on heuristic retrieval signals or always-on memory usage, failing to account for the changing and potentially inconsistent nature of user preferences. In this work, we propose a unified framework for memory access and selection based on changing preferences. We formulate personalized memory retrieval as identifying which historical turns provide evidence about a user's latent preference state, rather than relying on surface-level semantic similarity. To this end, we quantify the utility of each memory turn using a Bayes factor, defined as the improvement in the model's likelihood of the reference response when the turn is included in context. This provides a principled measure of evidence strength and a unified signal for both memory access and selection. By framing memory retrieval as utility estimation, the model learns to identify salient turns and regulate memory usage based on expected utility. Experiments on four heterogeneous memory benchmarks show that our approach outperforms existing embedding-based retrieval on long-context, preference-intensive tasks where modeling changing preferences is essential, while remaining competitive in low-density regimes where semantic similarity suffices.

Chinese Translation

长上下文对话系统必须决定何时访问记忆以及交互历史中哪些部分是相关的。现有的方法通常依赖于启发式检索信号或始终开启的记忆使用，未能考虑用户偏好的变化和潜在的不一致性。在本研究中，我们提出了一种基于变化偏好的记忆访问和选择的统一框架。我们将个性化记忆检索表述为识别哪些历史对话轮次提供了关于用户潜在偏好状态的证据，而不是依赖于表面层次的语义相似性。为此，我们使用贝叶斯因子量化每个记忆轮次的效用，贝叶斯因子被定义为在上下文中包含该轮次时模型对参考响应的可能性改善。这为证据强度提供了一个原则性度量，并为记忆访问和选择提供了统一信号。通过将记忆检索视为效用估计，模型学习识别显著的轮次，并根据预期效用调节记忆使用。在四个异构记忆基准上的实验表明，我们的方法在长上下文、偏好密集型任务中优于现有的基于嵌入的检索方法，而在语义相似性足够的低密度环境中仍具有竞争力。

View on arXiv Download PDF AI Translation

cs.CL / 18 / 2606.02981

Predicting Inference-Time Scaling Gains from Labeled Validation-Set Output Statistics

预测标记验证集输出统计的推理时间缩放增益

Zhang, Luyang, Li, Jingyan

Abstract

Best-of-$N$ inference scaling (drawing $N$ candidate answers from a language model and returning the one a reward model ranks highest) improves accuracy by an amount that varies across models, but predicting that amount in advance currently requires running the procedure end-to-end. Prior work links cheap statistics of a model's sampled outputs and validation-set correctness (how often samples agree, how diverse they are, how confident the model is, and where correct samples appear) to model behavior, but does not isolate which of these form a stable, compact predictor of best-of-$N$ gain. We fit ridge predictors on features computed from a single labeled validation-set sampling pass, use bootstrap-Lasso as a stability analysis of the candidate feature set, and give a concentration analysis with an explicit linear-approximation residual. Across three base-model families, six post-training methods, and math and reasoning task domains, the stability analysis identifies a strict three-feature core spanning prompt-level agreement spread, label-assisted first-correct-sample position, and completion-length variance; a compact ridge predictor built from this core plus an entropy add-on reaches Spearman $\rho = 0.90$ with actual best-of-$N$ gain under a reward-model verifier. The intended use is labeled validation-set screening of candidate configurations before paying the full reward-model scoring cost.

Chinese Translation

最佳的 $N$ 次推理缩放（从语言模型中抽取 $N$ 个候选答案，并返回奖励模型排名最高的一个）提高了准确性，但这种提高在不同模型之间的变化量各不相同，而目前预测这一量在事先需要运行整个过程。先前的研究将模型采样输出的廉价统计数据与验证集的正确性（样本一致性、样本多样性、模型置信度以及正确样本出现的位置）联系起来，但并未明确区分哪些因素构成了最佳的 $N$ 次增益的稳定、紧凑的预测因子。我们在从单次标记验证集采样中计算的特征上拟合了岭回归预测器，使用自助法-套索（bootstrap-Lasso）作为候选特征集的稳定性分析，并提供了带有显式线性近似残差的集中分析。在三个基础模型家族、六种后训练方法以及数学和推理任务领域中，稳定性分析识别出一个严格的三特征核心，涵盖了提示级别的一致性分布、标签辅助的首次正确样本位置以及完成长度的方差；基于该核心加上熵附加项构建的紧凑岭回归预测器在奖励模型验证器下达到了实际最佳 $N$ 次增益的斯皮尔曼相关系数 $ ho = 0.90$。预期用途是在支付完整的奖励模型评分成本之前，对候选配置进行标记验证集筛选。

View on arXiv Download PDF AI Translation

cs.CL / 19 / 2606.02983

A Locally Deployed RAG-Based Academic Advising System for Course Selection

基于RAG的本地部署学术咨询系统用于课程选择

Li, Feng, Iwata, Yoritaka

Abstract

The correct sequence of courses in the curriculum based on prerequisites between courses is of great importance for students to develop their knowledge and skills holistically. However, students crafting this sequence in isolation frequently struggle with recognition limitations and information overload that leads to confusion. Simultaneously, education institutions encounter difficulties in providing adequate academic advice for the correct sequence due to limited education resources. To address these challenges, we propose a locally deployed RAG-based academic advising system grounded in syllabus information. By combining large language models with retrieval from structured syllabus data, the system is designed to support course selection, prerequisite understanding, and personalized study planning in a privacy-preserving manner.

Chinese Translation

课程中基于先修课程之间的正确课程顺序对于学生全面发展知识和技能至关重要。然而，学生在孤立地制定这一顺序时，常常面临识别能力的限制和信息过载的问题，导致困惑。同时，教育机构由于教育资源有限，难以提供适当的学术建议以确保正确的课程顺序。为了解决这些挑战，我们提出了一种基于大纲信息的本地部署RAG（Retrieval-Augmented Generation）学术咨询系统。通过将大型语言模型与结构化大纲数据的检索相结合，该系统旨在以保护隐私的方式支持课程选择、先修课程理解和个性化学习规划。

View on arXiv Download PDF AI Translation

cs.CL / 20 / 2606.02991

Pretraining Language Models on Historical Text

在历史文本上预训练语言模型

Luo, Xiaoxi, Shinnick, Zachary, Griesshaber, Niclas, Wang, Yixuan, Yu, Junchi, Shi, Freda, Torr, Philip, Lu, Yao

Abstract

We introduce TypewriterLM, a 7.24B History language model (LM) trained exclusively on English text predating 1913. Developing History LMs requires addressing challenges in data quality and availability, preventing temporal leakage, designing temporally consistent post-training pipelines, and constructing reliable evaluations. To address these issues, we construct TypewriterCorpus, a 54B-token historical corpus collected from diverse archival and linguistically annotated sources with extensive data cleaning and leakage mitigation procedures. Furthermore, we introduce lexically grounded instructing tuning, a post-training framework that constraints responses to remain directly grounded in historical source documents. Using this framework we construct two historical instruction tuning datasets: History-LIMA and History-SelfInstruct. To evaluate capability and temporal consistency, we introduce History-Event, a benchmark suite for evaluating competence, temporal grounding and data leakage. We release TypewriterLM and all associated resources to support future research on historical language models.

Chinese Translation

我们介绍了TypewriterLM，一个专门在1913年前的英语文本上训练的7.24B历史语言模型（LM）。开发历史语言模型需要解决数据质量和可用性的问题，防止时间泄漏，设计时间一致的后训练流程，以及构建可靠的评估体系。为了解决这些问题，我们构建了TypewriterCorpus，一个由多种档案和语言注释来源收集的54B标记历史语料库，并进行了广泛的数据清理和泄漏缓解程序。此外，我们引入了词汇基础的指令调优（lexically grounded instructing tuning），这是一个后训练框架，约束模型的响应直接基于历史源文档。利用该框架，我们构建了两个历史指令调优数据集：History-LIMA和History-SelfInstruct。为了评估能力和时间一致性，我们引入了History-Event，一个用于评估能力、时间基础和数据泄漏的基准套件。我们发布了TypewriterLM及所有相关资源，以支持未来对历史语言模型的研究。

View on arXiv Download PDF AI Translation

cs.CL / 21 / 2606.03021

Hint-Guided Diversified Policy Optimization for LLM Reasoning

基于提示的多样化策略优化用于大语言模型推理

Cao, Zhiyu, Wu, Kaixin, Zhong, Mingjie, Li, Peifeng, Li, Xiaobo, Ye, Can, Zhu, Qiaoming

Abstract

Recent developments in Large Language Models (LLMs) have showcased impressive reasoning capabilities, with Reinforcement Learning with Verifiable Rewards (RLVR) being a promising enhancement strategy. However, existing reward mechanisms are constrained to the outcome-level correctness and lack explicit signals to guide the model to consider diverse solutions. In contrast, human problem solving typically involves evaluating multiple potential approaches and selecting the most reliable solution, a cognitive process that current RLVR frameworks do not explicitly incentivize. Inspired by this, we propose Hint-Guided Diversified Policy Optimization (HDPO), allowing the model to first list all potential candidate solution outlines as hints and then select the most reliable one for further reasoning. HDPO comprises two stages of Cold Start for Structured Reasoning and Hint-Guided Diversified Reinforcement Learning to incentivize the model to generate diverse and reliable solutions following the ``propose-select-think'' trajectory. Experimental results show that HDPO effectively boosts LLM reasoning and enhances the diversity of candidate solutions as well as the LLM's ability to identify reliable solutions.

Chinese Translation

近期大语言模型（LLMs）的发展展示了令人印象深刻的推理能力，其中可验证奖励的强化学习（RLVR）被认为是一种有前景的增强策略。然而，现有的奖励机制仅限于结果层面的正确性，缺乏明确的信号来引导模型考虑多样化的解决方案。相比之下，人类解决问题通常涉及评估多种潜在方法并选择最可靠的解决方案，这一认知过程在当前的RLVR框架中并未得到明确激励。受此启发，我们提出了基于提示的多样化策略优化（HDPO），允许模型首先列出所有潜在候选解决方案的提纲作为提示，然后选择最可靠的一个进行进一步推理。HDPO包括两个阶段：结构化推理的冷启动和基于提示的多样化强化学习，以激励模型生成多样且可靠的解决方案，遵循“提议-选择-思考”的轨迹。实验结果表明，HDPO有效提升了LLM的推理能力，增强了候选解决方案的多样性以及LLM识别可靠解决方案的能力。

View on arXiv Download PDF AI Translation

cs.CL / 22 / 2606.03022

Hallucinations as Orthogonal Noise: Inference-Time Manifold Alignment via Dynamic Contextual Orthogonalization

幻觉作为正交噪声：通过动态上下文正交化进行推理时流形对齐

Zhao, Mingkuan, Hu, Wentao, Huang, Tianchen, Min, Yuheng, Chen, Suquan, Gao, Yide, Zhai, Yanbo, Song, Shuangyong, Li, Xuelong

Abstract

Hallucination in Large Language Models (LLMs), characterized by the generation of content inconsistent with contextual facts or logical constraints -- remains a persistent challenge for reliable deployment. In this work, we address this issue through a geometric framework rooted in the linear representation hypothesis. We propose that hallucinations manifest as orthogonal noise relative to the semantic manifold of the residual stream. Specifically, we hypothesize that while attention heads ideally propagate information congruent with the context subspace, hallucinations arise when specific heads introduce components orthogonal to this subspace, disrupting the coherence of the latent representation. Based on this formulation, we introduce Dynamic Contextual Orthogonalization (DCO), an inference-time intervention method. DCO utilizes the input residual stream as a dynamic context anchor to perform orthogonal decomposition on attention head outputs. To distinguish between context-aligned semantic updates and divergent noise, DCO employs a layer-wise Z-score suppression mechanism that selectively attenuates outlier orthogonal components based on statistical distributions. Evaluations on Llama-3-8B and 70B across benchmarks such as XSum, NQ-Swap, and IFEval demonstrate that DCO achieves superior contextual faithfulness compared to state-of-the-art intervention baselines. Furthermore, DCO maintains high performance on knowledge-intensive tasks like TriviaQA and TruthfulQA, effectively mitigating the trade-off between hallucination suppression and parametric knowledge retention often observed in existing methods. Our findings validate the geometric interpretation of hallucinations and establish DCO as a computationally efficient approach for enforcing manifold alignment.Our code is available at https://github.com/Harry-Miral/DCO

Chinese Translation

大型语言模型（LLMs）中的幻觉，表现为生成与上下文事实或逻辑约束不一致的内容，仍然是可靠部署的一个持续挑战。在本研究中，我们通过一个基于线性表示假设的几何框架来解决这一问题。我们提出幻觉相对于残差流的语义流形表现为正交噪声。具体而言，我们假设，尽管注意力头理想情况下传播与上下文子空间一致的信息，但当特定的头引入与该子空间正交的成分时，幻觉便会产生，从而破坏潜在表示的一致性。基于这一构想，我们引入了动态上下文正交化（Dynamic Contextual Orthogonalization, DCO），这是一种推理时干预方法。DCO利用输入残差流作为动态上下文锚点，对注意力头输出进行正交分解。为了区分与上下文对齐的语义更新和发散噪声，DCO采用了一种逐层Z-score抑制机制，基于统计分布选择性地减弱异常正交成分。在Llama-3-8B和70B上的评估，包括XSum、NQ-Swap和IFEval等基准测试，表明DCO在上下文忠实性方面优于最先进的干预基线。此外，DCO在TriviaQA和TruthfulQA等知识密集型任务上保持高性能，有效缓解了现有方法中常见的幻觉抑制与参数知识保留之间的权衡。我们的研究结果验证了幻觉的几何解释，并确立了DCO作为一种计算上高效的流形对齐方法。我们的代码可在https://github.com/Harry-Miral/DCO获取。

View on arXiv Download PDF AI Translation

cs.CL / 23 / 2606.03027

SEA-Embedding: Open and Reproducible Text Embeddings for Southeast Asia

SEA-Embedding：东南亚开放且可重复的文本嵌入

Limkonchotiwat, Peerat, Ng, Raymond, Nutanong, Sarana, Ngui, Jian Gang

Abstract

Text embeddings are fundamental to many downstream applications, making robustness important for real-world NLP. However, most recent state-of-the-art embedding models are not reproducible because they rely on closed or undisclosed training data, and they remain insufficiently robust for Southeast Asian languages. We present SEA-Embedding, a fully open and reproducible text-embedding pipeline for Southeast Asian languages trained only on publicly available data, and use it to study three core factors of robust embedding design: data composition, training objective, and base encoder initialization. SEA-Embedding achieves state-of-the-art results on SEA-BED while enabling systematic and reproducible analysis of robust text embeddings for the region.

Chinese Translation

文本嵌入是许多下游应用的基础，因此在实际应用中的鲁棒性至关重要。然而，最近大多数最先进的嵌入模型并不可重复，因为它们依赖于封闭或未公开的训练数据，并且在东南亚语言上仍然不够鲁棒。我们提出了SEA-Embedding，这是一个完全开放且可重复的东南亚语言文本嵌入管道，仅基于公开可用的数据进行训练，并利用它研究鲁棒嵌入设计的三个核心因素：数据组成、训练目标和基础编码器初始化。SEA-Embedding在SEA-BED上取得了最先进的结果，同时使得对该地区鲁棒文本嵌入的系统性和可重复性分析成为可能。

View on arXiv Download PDF AI Translation

cs.CL / 24 / 2606.03029

Conditional Hypothesis Generation for LLM-Based Text Analysis with Researcher-Specified Covariates

基于条件假设生成的LLM文本分析与研究者指定协变量

Xu, Paiheng, Liu, Jing, Ai, Wei

Abstract

A core goal of computational social science is to discover interpretable differences in how language varies across outcomes of interest, such as political affiliation or instructional quality. Recent LLM-based hypothesis generation methods describe such differences in natural language, but select for globally discriminative patterns without accounting for covariates that shape the data based on researchers' domain knowledge. When covariates are ignored, selected patterns can reflect confounds rather than differences of substantive interest. We introduce conditional hypothesis generation, a framework that incorporates researcher-specified covariates to steer hypothesis discovery toward differences that hold within relevant subgroups. Two challenges arise: the target subgroup may be underrepresented (stratum imbalance), and the direction of a difference may reverse across subgroups (sign reversal). We propose two econometrics-inspired methods: one introduces feature--covariate interactions to detect sign reversals, and the other applies within-stratum demeaning and inverse-frequency reweighting to equalize underrepresented strata. Synthetic experiments show each method outperforms global baselines in its targeted setting, and expert evaluation on two real-world datasets confirms that covariate-aware generation surfaces more useful hypotheses within relevant subgroups.

Chinese Translation

计算社会科学的核心目标是发现语言在不同结果（如政治倾向或教学质量）之间的可解释差异。近期基于LLM的假设生成方法用自然语言描述了这些差异，但在选择全球性区分模式时未考虑根据研究者领域知识塑造数据的协变量。当忽略协变量时，所选模式可能反映混淆因素，而非实质性差异。我们提出了条件假设生成框架，该框架结合研究者指定的协变量，引导假设发现朝向在相关子群体内成立的差异。面临两个挑战：目标子群体可能代表性不足（层次不平衡），且差异的方向可能在子群体间反转（符号反转）。我们提出了两种受计量经济学启发的方法：一种引入特征与协变量的交互作用以检测符号反转，另一种应用层内去均值和逆频率重加权以平衡代表性不足的层次。合成实验表明，每种方法在其目标设置中均优于全球基线，而在两个真实世界数据集上的专家评估确认了考虑协变量的生成在相关子群体中呈现出更有用的假设。

View on arXiv Download PDF AI Translation

cs.CL / 25 / 2606.03032

The Deliberative Illusion: Diagnosing Factual Attrition and Stance Homogenization in Multi-Agent LLM Deliberation

审议幻觉：诊断多智能体大语言模型审议中的事实流失与立场同质化

Wan, Herun, Wu, Jiaying, Luo, Minnan, Li, Fanxiao, Wang, Ningnan, Chen, Nancy F., Kan, Min-Yen

Abstract

Multi-agent LLM systems often treat consensus as evidence of successful interaction. For deliberative problems, however, reliability depends on whether agents preserve the facts and viewpoints needed to interpret an issue. We identify the deliberative illusion: discussion produces (1) factual attrition, the progressive loss of issue-critical facts, alongside (2) stance homogenization, the collapse of diverse positions toward consensus. To measure this process, we introduce DelibTrace, a framework that decomposes each issue into atomic facts, labels issue-critical ones, distributes them across agents, and tracks their survival across discussion rounds. Across ethical and news-based deliberation with three representative LLM families, multi-agent discussion erases up to 72% of issue-critical facts. This loss is consequential: retained evidence can reconstruct the issue misleadingly, final stances remain anchored in base-model priors, and a single malicious agent can inject misinformation into the shrinking shared context. These results reveal a sharper risk: agents can agree more while knowing less. We call for evaluations that measure which facts, uncertainties, and legitimate disagreements survive interaction.

Chinese Translation

多智能体大语言模型系统通常将共识视为成功互动的证据。然而，对于审议性问题而言，可靠性取决于智能体是否保留了解释问题所需的事实和观点。我们识别出审议幻觉：讨论产生了（1）事实流失，即关键问题事实的逐渐丧失，以及（2）立场同质化，即多样化立场向共识的崩溃。为了测量这一过程，我们引入了 DelibTrace，一个将每个问题分解为原子事实的框架，标记关键问题事实，将其分配给智能体，并跟踪它们在讨论轮次中的存活情况。在涉及三种代表性大语言模型家族的伦理和新闻基础审议中，多智能体讨论最多抹去了72%的关键问题事实。这一损失是有重大影响的：保留的证据可能会误导性地重构问题，最终立场仍然基于基础模型的先验，而单个恶意智能体可以向日益缩小的共享上下文中注入错误信息。这些结果揭示了一个更尖锐的风险：智能体在了解更少的情况下可能达成更多的共识。我们呼吁进行评估，以测量哪些事实、不确定性和合法的分歧在互动中得以存活。

View on arXiv Download PDF AI Translation

cs.CL / 26 / 2606.03043

The Geometry of LLM-as-Judge: Why Inter-LLM Consensus Is Not Human Alignment

LLM作为评判者的几何学：为何LLM间共识并非人类对齐

Mukherjee, Sourabrata, Hamna, Hamna, Bali, Kalika, Sitaram, Sunayana

Abstract

LMs-as-judges are now standard, yet judges agree strongly with one another while agreeing only weakly with humans. We test whether this reflects shared signal or shared bias by measuring four geometric quantities on the standard LLM-as-judge stack across four community-built Indic datasets, eight Indic languages, and 41 LLM judges: score spread, effective rank, principal angle to the human subspace, and stacked correlations among judges and humans, all with bootstrap confidence intervals. On subjective rubrics, judges use less than half the human score range ($\sigma_J / \sigma_H \approx 0.3$--$0.5$). Their evaluation axis is nearly orthogonal to the human one and noticeably further from humans than humans are from each other ($87^\circ$--$89^\circ$ versus $78^\circ$--$81^\circ$). Inter-LLM agreement exceeds LLM--human agreement ($r_{LL} \approx 0.35$ versus $r_{LH} \approx 0.27$--$0.32$). On a rubric with a verifiable factual answer, the same diagnostics fall back into the human range (axis $58.5^\circ$; $r_{LH} = 0.519$). Fine-tuning and preference optimization recover spread ($0.32 \rightarrow 1.08$) but barely move the axis (still $87^\circ$--$88^\circ$). Only post-hoc calibration on a small human-anchored set improves all four community-health rubrics together, placing a calibrated 24B Indic judge ($r = 0.184$) ahead of GPT-5.5 ($r = 0.123$), yet still short of human reliability (human-human $r = 0.474$ on the verifiable rubric). We argue that inter-LLM agreement should be considered evidence of human alignment only when a direct geometric check on the judge's score subspace passes; otherwise, the consensus reflects agreement within a collapsed subspace.

Chinese Translation

作为评判者的语言模型（LMs-as-judges）现已成为标准，然而评判者之间的意见高度一致，而与人类的意见却仅弱相关。我们通过在四个社区构建的印度数据集、八种印度语言和41个LLM评判者的标准LLM作为评判者堆栈上测量四个几何量，来检验这是否反映了共享信号或共享偏见：得分分布、有效排名、与人类子空间的主角度，以及评判者与人类之间的堆叠相关性，所有测量均附带自助法置信区间。在主观评分标准上，评判者使用的得分范围不到人类得分范围的一半（$rac{ ext{σ}_J}{ ext{σ}_H} ext{≈} 0.3$--$0.5$）。他们的评估轴几乎与人类的评估轴正交，并且与人类的距离明显大于人类之间的距离（$87^ ext{°}$--$89^ ext{°}$对比$78^ ext{°}$--$81^ ext{°}$）。LLM间的协议超过了LLM与人类之间的协议（$r_{LL} ext{≈} 0.35$对比$r_{LH} ext{≈} 0.27$--$0.32$）。在一个具有可验证事实答案的评分标准上，相同的诊断结果回归到人类范围（轴$58.5^ ext{°}$；$r_{LH} = 0.519$）。微调和偏好优化恢复了分布（$0.32 ightarrow 1.08$），但几乎没有移动轴（仍为$87^ ext{°}$--$88^ ext{°}$）。只有在一个小型人类锚定集上的事后校准，才能同时改善所有四个社区健康评分标准，使得一个经过校准的24B印度评判者（$r = 0.184$）超越了GPT-5.5（$r = 0.123$），但仍低于人类的可靠性（在可验证评分标准上人类间的$r = 0.474$）。我们认为，只有在对评判者得分子空间进行直接几何检查通过时，LLM间的协议才应被视为人类对齐的证据；否则，这种共识反映的是在一个压缩子空间内的协议。

View on arXiv Download PDF AI Translation

cs.CL / 27 / 2606.03078

G^2C-MT: Graph-Guided Context Selection for Document-Level Machine Translation

G^2C-MT：基于图的上下文选择用于文档级机器翻译

Ji, Baijun, Zhou, Zixuan, Duan, Xiangyu, Liu, Yu, Sun, Longbo, Wei, Rupu, Zhao, Bohong

Abstract

Effective document-level machine translation (DocMT) requires capturing long-range discourse dependencies. Recent work has explored retrieval-based and discourse-aware context selection. However, these approaches often lack an explicit mechanism for modeling structured discourse dependencies between distant paragraphs in a document. In this paper, we propose G^2C-MT (Graph-Guided Context for Machine Translation), which views DocMT context selection as a structured path discovery problem on a lightweight discourse graph, rather than retrieving unstructured context sets or relying on expensive LLM-based discourse modeling. In detail, we represent each paragraph as a node and model the relationship between each pair of nodes, considering their semantic similarity, adjacency, and keyword overlap. Furthermore, we propose a depth-biased random walk over the graph to sample a backward context path for each target paragraph. The context path will be used to prompt a large language model (LLM) for translation. This framework naturally supports multi-path context sampling, which can improve robustness by aggregating diverse translation candidates for discourse-ambiguous inputs. Experiments conducted across various domains show that G^2C-MT outperforms strong baselines on multiple LLMs, including DeepSeek-V3, Gemini-2.5-Flash-lite, and the Qwen-2.5/3 series.

Chinese Translation

有效的文档级机器翻译（DocMT）需要捕捉长距离的语篇依赖关系。近期的研究探索了基于检索的和语篇感知的上下文选择方法。然而，这些方法通常缺乏对文档中远距离段落之间结构化语篇依赖关系的明确建模机制。本文提出了G^2C-MT（Graph-Guided Context for Machine Translation），将DocMT的上下文选择视为在轻量级语篇图上发现结构化路径的问题，而不是检索非结构化上下文集或依赖于昂贵的基于大语言模型（LLM）的语篇建模。具体而言，我们将每个段落表示为一个节点，并建模每对节点之间的关系，考虑它们的语义相似性、邻接性和关键词重叠。此外，我们提出了一种深度偏向的随机游走方法，在图上为每个目标段落采样一个向后上下文路径。该上下文路径将用于提示大型语言模型（LLM）进行翻译。该框架自然支持多路径上下文采样，通过聚合多样的翻译候选来提高对语篇模糊输入的鲁棒性。在多个领域进行的实验表明，G^2C-MT在多个LLM上超越了强基线，包括DeepSeek-V3、Gemini-2.5-Flash-lite和Qwen-2.5/3系列。

View on arXiv Download PDF AI Translation

cs.CL / 28 / 2606.03080

Regret Pre-training: Bridging Prior and Posterior Views for Enhanced Knowledge Grounding

遗憾预训练：桥接先验与后验视角以增强知识基础

Zhao, Mingkuan, Sun, Xiayu, Hu, Wentao, Chen, Suquan, Li, Jiaxuan, Zhu, Xiaoyan, Lai, Xin, Wang, Jiayin

Abstract

Causal language models factorize sequence probabilities using only preceding context, leaving future information unexploited during training despite its availability in the training data. This paper introduces Regret Pre-training, a self-supervised framework grounded in the Learning Using Privileged Information (LUPI) paradigm. The framework employs a dual-view architecture in which a single model generates both a causal Student distribution and a future-conditioned Teacher distribution. The training objective augments standard language modeling with a regret loss that minimizes the KL divergence from teacher to student, transferring future-aware signals to the causal representations. We investigate two teacher configurations on the OLMoE-1B-7B architecture:LocalRegret, which extends attention by one future token, andGlobalRegret, which conditions on bidirectional context with the target position masked. Experiments on nine downstream tasks following 4 billion tokens of training demonstrate that both configurations consistently outperform the baseline. On average,GlobalRegret andLocalRegret achieve 33.9% and 32.2% accuracy respectively, surpassing the baseline's 30.2%. Most notably,GlobalRegret improves BoolQ performance by 18.1 percentage points (61.0% vs 42.9%). The framework introduces no additional parameters and requires only one extra inference-mode forward pass per training step.

Chinese Translation

因果语言模型仅利用前文上下文来分解序列概率，尽管训练数据中可用未来信息，但在训练过程中未能加以利用。本文提出了遗憾预训练（Regret Pre-training），这是一种基于特权信息学习（Learning Using Privileged Information, LUPI）范式的自监督框架。该框架采用双视角架构，其中单个模型同时生成因果学生分布和未来条件教师分布。训练目标在标准语言建模的基础上增加了遗憾损失，最小化教师与学生之间的KL散度，将未来感知信号转移到因果表示中。我们在OLMoE-1B-7B架构上研究了两种教师配置：LocalRegret，它通过一个未来标记扩展注意力，以及GlobalRegret，它在目标位置被掩蔽的情况下对双向上下文进行条件处理。在经过40亿个标记的训练后，针对九个下游任务的实验表明，这两种配置均持续优于基线。平均而言，GlobalRegret和LocalRegret分别达到了33.9%和32.2%的准确率，超越了基线的30.2%。最显著的是，GlobalRegret在BoolQ任务上的表现提高了18.1个百分点（61.0%对42.9%）。该框架没有引入额外参数，并且每个训练步骤仅需一次额外的推理模式前向传递。

View on arXiv Download PDF AI Translation

cs.CL / 29 / 2606.03096

Can Factual Opinions Be Edited (Manipulated) in Large Language Models?

事实观点能否在大型语言模型中被编辑（操控）？

Cao, Yuanpu, Yin, Ziyi, Ma, Fenglong, Chen, Jinghui

Abstract

Large Language Models (LLMs) are increasingly integrated into various domains, making knowledge editing techniques crucial yet potentially hazardous. Current editing methods primarily target atomic facts, overlooking the significant risks associated with manipulating factual opinions, e.g., documented stances of public figures on societal issues. Such manipulation could reshape public images, influence elections, and alter societal views. To systematically assess this threat, we introduce the Factual Opinion Editing with Evidence (FOE) benchmark, which encompasses 261 public figures, 19 issue categories, and 2,178 complete opinion records. Our evaluations demonstrate that current editing techniques struggle significantly with factual opinions, often achieving only superficial changes while failing to preserve consistency between the edited opinion and the supporting evidence generated by the model. To address this limitation, we further propose a simple yet effective Self-Generated Evidence-Aligned method that achieves opinion-evidence alignment without relying on explicit instructions. Together, our benchmark and method provide a foundation for understanding the emerging security implications of factual opinion editing in LLMs.

Chinese Translation

大型语言模型（LLMs）正日益融入各个领域，使得知识编辑技术变得至关重要，但同时也可能带来风险。目前的编辑方法主要针对原子事实，忽视了操控事实观点（例如，公众人物在社会问题上的立场）所带来的重大风险。这种操控可能重塑公众形象、影响选举并改变社会观点。为了系统性地评估这一威胁，我们引入了带证据的事实观点编辑基准（Factual Opinion Editing with Evidence, FOE），该基准涵盖261位公众人物、19个议题类别和2,178条完整的观点记录。我们的评估表明，当前的编辑技术在处理事实观点时面临显著困难，通常只能实现表面上的变化，而无法保持编辑后的观点与模型生成的支持证据之间的一致性。为了解决这一局限，我们进一步提出了一种简单而有效的自生成证据对齐方法（Self-Generated Evidence-Aligned method），该方法在不依赖明确指令的情况下实现观点与证据的对齐。我们的基准和方法共同为理解大型语言模型中事实观点编辑的新兴安全隐患提供了基础。

View on arXiv Download PDF AI Translation

cs.CL / 30 / 2606.03099

PhotoCraft: Agentic Reasoning with Hierarchical Self-Evolving Memory for Deep Image Search

PhotoCraft：具有层次自我演化记忆的智能推理用于深度图像搜索

Lyu, Kailin, Yuan, Zhiqiang, He, Jianwei, Yan, Qiwei, Su, Xuanbo, Hu, Nanxing, Liu, Yang, Hao, Ce, Qin, Shengqian, Hu, Lianyu, Zhang, Jinchao, Zhou, Jie

Abstract

Deep Image Search requires multi-step reasoning over rich contextual cues, such as time, location, and event relations. However, most existing LLM-based agents are stateless and reactive, lacking persistent memory to maintain long-horizon context or transfer experience across tasks, which often leads to execution drift and experience isolation. To address these limitations, we propose PhotoCraft, a training-free, hierarchical memory system for photo-search agents. Inspired by human cognition, PhotoCraft equips MLLMs with working, episodic, and semantic memory, which are dynamically invoked during reasoning to preserve logical consistency and knowledge transferability throughout multi-step reasoning and answer generation. Extensive experiments on DISBench demonstrate that PhotoCraft consistently improves context-aware retrieval across diverse MLLM backbones, achieving gains of up to 18.5\% and effectively mitigating key bottlenecks in memoryless deep image search, offering a practical path toward reliable and generalizable multimodal search agents.

Chinese Translation

深度图像搜索需要对丰富的上下文线索进行多步骤推理，例如时间、地点和事件关系。然而，现有的大多数基于大型语言模型（LLM）的智能体是无状态和反应式的，缺乏持久记忆来维持长期上下文或跨任务转移经验，这常常导致执行漂移和经验孤立。为了解决这些局限性，我们提出了PhotoCraft，一种用于照片搜索智能体的无训练、层次记忆系统。受人类认知的启发，PhotoCraft为多模态大型语言模型（MLLM）配备了工作记忆、情节记忆和语义记忆，这些记忆在推理过程中动态调用，以保持逻辑一致性和知识可转移性，贯穿于多步骤推理和答案生成。我们在DISBench上进行了广泛实验，结果表明，PhotoCraft在不同的MLLM骨干网络中始终提高了上下文感知检索，提升幅度高达18.5\%，有效缓解了无记忆深度图像搜索中的关键瓶颈，为可靠且可推广的多模态搜索智能体提供了切实可行的路径。

View on arXiv Download PDF AI Translation

cs.CL / 31 / 2606.03102

Small RL Controller, Large Language Model: RL-Guided Adaptive Sampling for Test-Time Scaling

小型强化学习控制器，大型语言模型：基于强化学习的自适应采样用于测试时扩展

Dai, Runpeng, Zheng, Tong, Liu, Rui, Huang, Chengsong, Zhu, Hongtu

Abstract

Test-time scaling improves the reasoning performance of large language models but incurs substantial cost in both total computation and latency. Existing adaptive sampling methods partially mitigate this issue by dynamically deciding when to stop sampling, yet they typically rely on heuristic rules or rely on distribution assumptions. In this work, we formulate adaptive sampling as a Markov decision process (MDP). We train a lightweight sampling controller with reinforcement learning (RL) to jointly balance answer correctness, latency, and computation cost. At each round, the controller decides to stop sampling or to acquire additional samples. Our method is lightweight which only relies on statistics of final answers, and can be trained and deployed on CPU. We further show that the resulting framework admits an interpretation as the Lagrangian relaxation of a constrained optimization problem with explicit budget constraints. Experiments against strong baselines such as ASC and ESC show that our method achieves improved trade-offs among answer correctness, sampling rounds, and total samples required.

Chinese Translation

测试时扩展提高了大型语言模型的推理性能，但在总计算量和延迟上带来了显著成本。现有的自适应采样方法通过动态决定何时停止采样在一定程度上缓解了这一问题，但通常依赖于启发式规则或分布假设。在本研究中，我们将自适应采样形式化为一个马尔可夫决策过程（MDP）。我们使用强化学习（RL）训练一个轻量级采样控制器，以共同平衡答案的正确性、延迟和计算成本。在每一轮中，控制器决定是停止采样还是获取额外样本。我们的方法轻量且仅依赖于最终答案的统计信息，可以在CPU上进行训练和部署。我们进一步表明，所得到的框架可以解释为具有明确预算约束的约束优化问题的拉格朗日松弛。与强基线（如ASC和ESC）的实验表明，我们的方法在答案正确性、采样轮次和所需总样本之间实现了更好的权衡。

View on arXiv Download PDF AI Translation

cs.CL / 32 / 2606.03110

Coherence Maximization Improves Pluralistic Alignment

一致性最大化改善多元对齐

Mahbub, Taslim, Pei, Yiding, Feng, Shi

Abstract

Aligning AI systems with diverse human values requires value specifications grounded in concrete examples, but generating such examples without extensive human supervision remains an open challenge. We investigate what makes these examples effective, using Internal Coherence Maximization (ICM) -- which infers labels by maximizing their mutual predictability -- to generate persona-specific examples that steer a model toward a target group's values, without human supervision. Across four benchmarks spanning classification, preference, and open-ended generation, ICM-inferred in-context examples match the performance of gold labels. Crucially, coherence matters beyond individual label accuracy: with accuracy held constant, more coherent examples generalize substantially better than incoherent ones. For personas underrepresented in pretraining data, targeted human feedback on the questions where the model is least certain about a persona's values yields better generalization than the same number of labels on arbitrary questions. These results identify coherence as a key design principle for scalable value specification, leveraging the diverse human perspectives already encoded in pretrained language models.

Chinese Translation

将人工智能系统与多样的人类价值观对齐需要基于具体示例的价值规范，但在没有广泛人类监督的情况下生成此类示例仍然是一个未解决的挑战。我们研究了这些示例有效性的因素，使用内部一致性最大化（Internal Coherence Maximization, ICM）——通过最大化标签之间的相互可预测性来推断标签——生成特定于角色的人物示例，以引导模型朝向目标群体的价值观，而无需人类监督。在跨越分类、偏好和开放式生成的四个基准测试中，ICM推断的上下文示例与金标准标签的表现相匹配。至关重要的是，一致性在单个标签准确性之外也很重要：在保持准确性不变的情况下，更一致的示例在泛化能力上显著优于不一致的示例。对于在预训练数据中代表性不足的人物，针对模型对人物价值观最不确定的问题进行的定向人类反馈，能够比在任意问题上获得相同数量的标签更好地实现泛化。这些结果将一致性识别为可扩展价值规范的关键设计原则，利用已经编码在预训练语言模型中的多样人类视角。

View on arXiv Download PDF AI Translation

cs.CL / 33 / 2606.03113

Experience-Driven Dynamic Exits for LLMs with Reinforcement Learning

基于经验驱动的动态退出策略：强化学习下的大型语言模型

Zhu, Yanyu, Pao, Hoilam, Hu, Niu, Guo, Wei, Zhan, Shaoxiong, Lai, Boyu, Wang, Zitai, Zeng, Yongqin, Zheng, Hai-Tao

Abstract

Large Language Models suffer from slow autoregressive inference. While self-speculative decoding accelerates this process, its efficiency is hampered by static configurations like fixed exit layers and speculation lengths. We reframe this optimization as a \textbf{Markov Decision Process} and propose \textbf{LEDE}, a framework that uses offline reinforcement learning. LEDE learns a policy to dynamically select the optimal exit layer and speculation length based on the local context of the generated sequence at each step, balancing computational cost and draft quality. Comprehensive evaluations on Llama-2 and Llama-3 models show LEDE achieves up to a $2.0\times$$\sim$$2.7\times$ speedup over autoregressive decoding and and provides an additional 17\% speedup over the static speculative baselines.

Chinese Translation

大型语言模型在自回归推理中存在速度较慢的问题。尽管自我推测解码加速了这一过程，但其效率受到固定退出层和推测长度等静态配置的限制。我们将这一优化问题重新表述为 extbf{马尔可夫决策过程}，并提出了 extbf{LEDE}框架，该框架使用离线强化学习。LEDE学习一种策略，根据每一步生成序列的局部上下文动态选择最佳退出层和推测长度，以平衡计算成本和草稿质量。在对Llama-2和Llama-3模型的全面评估中，LEDE实现了高达$2.0 imes$$ ext{至}$$2.7 imes$的速度提升，相较于静态推测基线提供了额外的17 ext{%}速度提升。

View on arXiv Download PDF AI Translation

cs.CL / 34 / 2606.03132

DMT-CBT: Longitudinal Therapeutic State Modeling for CBT Counseling

DMT-CBT：认知行为疗法咨询的纵向治疗状态建模

Liu, Chang, Zhang, Shuyi, Ma, Changsheng, Tao, Yongfeng, Yang, Minqiang, Hu, Bin

Abstract

Large language models (LLMs) have shown growing potential for Cognitive Behavioral Therapy (CBT) counseling. However, most existing approaches still formulate counseling as a local response generation problem, focusing on empathetic replies within short, text-only, or single-session interactions. We argue that this formulation fundamentally mismatches the nature of real psychotherapy. In clinical CBT, therapy is a longitudinal process in which therapists continuously infer, update, and intervene on evolving therapeutic states across sessions. Realistic CBT further involves multimodal inference and delayed cross-session intervention effects, requiring models to capture longitudinal therapeutic state evolution under partial observability. We propose DMT-CBT, a framework for Dynamic Modeling of evolving Therapeutic states in CBT counseling. DMT-CBT maintains structured therapeutic states across sessions while incorporating multimodal behavioral grounding and tool-augmented intervention to support adaptive therapeutic reasoning. Based on this framework, we construct DMTCorpus, a synthetic multi-session multimodal CBT counseling dataset featuring evolving therapeutic states, image-grounded client behaviors, and cross-session intervention continuity. Experimental results show that DMT-CBT improves counseling fidelity and therapeutic alliance, produces more favorable longitudinal affective trajectories, and preserves therapeutic states more faithfully than post-hoc extraction approaches.

Chinese Translation

大型语言模型（LLMs）在认知行为疗法（CBT）咨询中展现出越来越大的潜力。然而，现有的大多数方法仍将咨询视为一个局部响应生成问题，专注于在短文本、单一会话互动中产生同理心回复。我们认为这种表述与真实心理治疗的本质根本不匹配。在临床CBT中，治疗是一个纵向过程，治疗师在多个会话中不断推断、更新和干预不断演变的治疗状态。现实的CBT还涉及多模态推理和延迟的跨会话干预效果，这要求模型在部分可观察性下捕捉纵向治疗状态的演变。我们提出了DMT-CBT，一个用于动态建模CBT咨询中不断演变的治疗状态的框架。DMT-CBT在会话之间保持结构化的治疗状态，同时结合多模态行为基础和工具增强干预，以支持适应性的治疗推理。基于该框架，我们构建了DMTCorpus，一个合成的多会话多模态CBT咨询数据集，具有不断演变的治疗状态、基于图像的客户行为和跨会话干预的连续性。实验结果表明，DMT-CBT提高了咨询的忠实度和治疗联盟，产生了更有利的纵向情感轨迹，并比事后提取方法更忠实地保留了治疗状态。

View on arXiv Download PDF AI Translation

cs.CL / 35 / 2606.03156

A cross-domain tropical species dataset with Chinese vernacular names and CITES source links

一个包含中文俗名和CITES来源链接的跨领域热带物种数据集

Wang, Jeff

Abstract

We describe a versioned cross-domain dataset of 410,499 active tropical species (working snapshot 2026-04-20) spanning three applied subdomains -- tropical_plants, tropical_aquatic, and tropical_pets -- that share a commercial and regulatory life cycle but are distributed across kingdom-organised biodiversity infrastructures. The resource joins taxonomic identifiers from GBIF, Plants of the World Online, iNaturalist, NCBI Taxonomy, the Catalogue of Life and the Encyclopedia of Life, and adds three original layers: a cross-domain ontology that re-segments taxa along trade and husbandry contexts; a Chinese vernacular layer with explicit per-name provenance under a typology that excludes unverified machine-generated proposals; and a CITES source-linkage layer connecting each taxon to its Species+ entry. Chinese vernacular coverage -- the proportion of taxa carrying a CJK Chinese name distinct from the scientific binomial -- reaches 99.50 percent (408,456 of 410,499; full-population count). Coverage characterises completeness, not name-translation accuracy; the latter is bounded by the four-level provenance typology and is the subject of a preliminary internal review reported here, with a blind external audit identified as the principal open item. Upstream content is referenced by stable identifier only for the original-contribution layers, supporting CC-BY 4.0 reuse. The dataset is deposited on Zenodo (10.5281/zenodo.20377811). This preprint is the canonical v1.0 description of the dataset's current state; future Data Descriptor submission is anticipated but is contingent on the validation and release-engineering items listed in the Limitations.

Chinese Translation

我们描述了一个版本化的跨领域数据集，包含410,499个活跃的热带物种（工作快照日期为2026-04-20），涵盖了三个应用子领域——热带植物（tropical_plants）、热带水生生物（tropical_aquatic）和热带宠物（tropical_pets），这些领域共享商业和监管生命周期，但分布在以王国为组织的生物多样性基础设施中。该资源结合了来自全球生物多样性信息设施（GBIF）、世界植物在线（Plants of the World Online）、iNaturalist、NCBI分类法、生命目录（Catalogue of Life）和生命百科全书（Encyclopedia of Life）的分类标识符，并增加了三个原创层次：一个跨领域本体，重新划分了与贸易和饲养相关的分类；一个带有明确逐名来源的中文俗名层，采用排除未经验证的机器生成提案的分类法；以及一个CITES来源链接层，将每个分类与其Species+条目连接起来。中文俗名覆盖率——带有不同于科学双名的CJK中文名称的分类比例——达到99.50%（408,456个分类中的410,499个；完整人口计数）。覆盖率表征了完整性，而非名称翻译的准确性；后者受到四级来源分类法的限制，并且是此处报告的初步内部审查的主题，盲审外部审计被确定为主要待解决事项。上游内容仅通过稳定标识符引用原始贡献层，支持CC-BY 4.0的重用。该数据集已存储在Zenodo（10.5281/zenodo.20377811）。此预印本是数据集当前状态的规范v1.0描述；未来的数据描述提交预计将进行，但取决于限制中列出的验证和发布工程项目。

View on arXiv Download PDF AI Translation

cs.CL / 36 / 2606.03165

Fully Automated Identification of Lexical Alignment and Preference-Stage Shifts in Large Language Models

大语言模型中词汇对齐和偏好阶段转变的完全自动化识别

Juzek, Thomas Stephan, Ming, Xiaoyang, Hernandez, Jose A.

Abstract

The language used by digital chat assistants such as ChatGPT can diverge from human expectations (misalignment). Research, mostly on Scientific English, has described both what divergences occur and, to some extent, why, linking them to the training stage of human preference learning. Yet, existing approaches rely on manual curation. This paper introduces two curation-free, assumption-light evaluation metrics: the Lexical Alignment Score, which identifies lexical overuse, and the Triangulated Preference Shift, which quantifies how much of such shifts can be attributed to human preference learning. Using PubMed abstracts, continuations were generated and measured using windowed document prevalence across six model families (Falcon, Gemma, Llama, Mistral, OLMo, Yi). The procedure identifies, without manual intervention, overused items such as 'suggest', 'additionally', and 'strategy', and estimates their link to preference learning. Our findings replicate prior work and remain stable across parameter settings, random seeds, and evaluation on further data. The approach scales readily and enables systematic study of lexical (mis)alignment beyond Scientific English and across languages, and as such, the metrics have the potential to contribute to improved alignment for future models and understanding of its origins.

Chinese Translation

数字聊天助手（如 ChatGPT）使用的语言可能与人类预期存在偏差（不对齐）。现有研究主要集中在科学英语上，描述了这些偏差的发生情况以及在某种程度上其原因，并将其与人类偏好学习的训练阶段联系起来。然而，现有的方法依赖于手动整理。本文介绍了两种无整理、假设轻量的评估指标：词汇对齐评分（Lexical Alignment Score），用于识别词汇的过度使用，以及三角偏好转变（Triangulated Preference Shift），用于量化此类转变中有多少可以归因于人类偏好学习。通过使用 PubMed 摘要生成的续写，并在六个模型系列（Falcon、Gemma、Llama、Mistral、OLMo、Yi）中使用窗口文档普遍性进行测量。该过程在没有人工干预的情况下识别出过度使用的词汇，如 'suggest'、'additionally' 和 'strategy'，并估计它们与偏好学习的关联。我们的研究结果复制了先前的工作，并在参数设置、随机种子和进一步数据的评估中保持稳定。该方法易于扩展，能够系统地研究超越科学英语的词汇（不）对齐及其在不同语言中的表现，因此，这些指标有潜力为未来模型的改进对齐和理解其起源做出贡献。

View on arXiv Download PDF AI Translation

cs.CL / 37 / 2606.03179

HyperPatch: Sequential Knowledge Editing Under n-ary Structural Drift

HyperPatch：在 n-元结构漂移下的顺序知识编辑

Chan, Yu-Kai, Lien, Wen-Sheng, Yao, Dong-Ting, Ruan, Bo-Kai, Lin, Kwan-Yeung, Shuai, Hong-Han, Chiang, Meng-Fen

Abstract

Large Language Models (LLMs) rely on Knowledge Editing (KE) to maintain temporal validity, yet real-world knowledge is inherently n-ary. We demonstrate that in non-stationary environments, sequential updates to complex relations induce N-ary Structural Drift, a phenomenon where the binary reification of n-ary events into triples fractures relational atomicity. This precipitates Structure-Conditioned Knowledge Transfer Failure, a systematic mis-grounding of the retriever frequently misdiagnosed as parametric hallucination. To tackle this, we propose HyperPatch, a parameter-preserving framework that reformulates sequential KE as a stability problem over hypergraph manifolds. HyperPatch preserves event integrity through three phases: (i) Structural Prior Initialization, establishing a topology-aware embedding space via contrastive learning on a Hypergraph Neural Network (HGNN) to capture high-order correlations; (ii) Sequential Topology Editing, utilizing a dual-stage mechanism that employs SimHash-based Topological Alignment for rapid conflict resolution and Topological LoRA Adaptation to track drift without backbone retraining; and (iii) Structure-Conditioned Reasoning, which integrates globally consistent evidence from fused linguistic and structural manifolds. On the MQuAKE-CF and MQuAKE-T benchmarks, HyperPatch achieves relative gains in Hop-wise Accuracy (H-Acc) of 96.24% and 21.06% over the strongest baseline, respectively. Further ablations demonstrate superior reliability under continuous n-ary update streams, whereas the standard KG-based variant suffers H-Acc collapses of up to 88.3% due to structural misalignment.

Chinese Translation

大型语言模型（LLMs）依赖知识编辑（KE）来维持时间有效性，但现实世界的知识本质上是 n-元的。我们展示了在非平稳环境中，对复杂关系的顺序更新会引发 n-元结构漂移，这是一种现象，其中 n-元事件的二元具体化为三元组破坏了关系的原子性。这导致了结构条件知识转移失败，即检索器的系统性错误基础，常被误诊为参数幻觉。为了解决这个问题，我们提出了 HyperPatch，这是一个保持参数的框架，将顺序知识编辑重新表述为超图流形上的稳定性问题。HyperPatch 通过三个阶段保持事件的完整性：（i）结构先验初始化，通过在超图神经网络（HGNN）上进行对比学习建立一个拓扑感知的嵌入空间，以捕捉高阶相关性；（ii）顺序拓扑编辑，利用双阶段机制，采用基于 SimHash 的拓扑对齐进行快速冲突解决，并通过拓扑 LoRA 适应跟踪漂移而无需重新训练主干；（iii）结构条件推理，整合来自融合语言和结构流形的全球一致证据。在 MQuAKE-CF 和 MQuAKE-T 基准测试中，HyperPatch 分别在 Hop-wise 准确率（H-Acc）上相较于最强基线实现了 96.24% 和 21.06% 的相对增益。进一步的消融实验表明，在连续 n-元更新流下具有更高的可靠性，而标准的基于知识图谱的变体由于结构不对齐遭受了高达 88.3% 的 H-Acc 崩溃。

View on arXiv Download PDF AI Translation

cs.CL / 38 / 2606.03189

SenseJudge: Human-Centric Preference-Driven Judgment Framework

SenseJudge：以人为中心的偏好驱动判断框架

Li, Rui, Liu, Junfeng, Kong, Xiangwen, Xu, Linhai, Sui, Zhifang

Abstract

Large Language Models (LLMs) as judges across various scenarios such as assessing model responses is becoming an increasingly accepted paradigm. However, existing judgment approaches often rely on trained judgers using fixed preference data, which tend to overlook diverse user preferences and struggle to adapt to real-world human-AI dialogue scenarios. To address these limitations, we propose SenseJudge, a customizable judgment framework driven by human preferences and SenseBench, a diverse and challenging instruction-following benchmark derived from real-world multi-turn interactions. We applied the automatic judgment framework and benchmark to two tasks: (1) LLMs as personalized judges, and (2) model ranking. We conducted extensive experiments, and the results demonstrate that the SenseJudge framework surpasses other judgment methods and models in the LLMs-as-personalized-judges task and achieves model ranking that aligns with real human sense. Additionally, we conducted analyses on position bias and consistency, alongside ablation studies, which affirmed the robustness of SenseJudge.

Chinese Translation

大型语言模型（LLMs）作为评判者在评估模型响应等各种场景中的应用正日益被接受。然而，现有的判断方法通常依赖于使用固定偏好数据的训练评判者，这往往忽视了多样化的用户偏好，并且难以适应现实世界的人机对话场景。为了解决这些局限性，我们提出了SenseJudge，一个由人类偏好驱动的可定制判断框架，以及SenseBench，一个基于真实多轮交互而衍生出的多样且具有挑战性的指令跟随基准。我们将自动判断框架和基准应用于两个任务：（1）LLMs作为个性化评判者，以及（2）模型排名。我们进行了广泛的实验，结果表明，SenseJudge框架在LLMs作为个性化评判者任务中超越了其他判断方法和模型，并在模型排名中达到了与真实人类感知相一致的结果。此外，我们还进行了关于位置偏差和一致性的分析，以及消融研究，进一步确认了SenseJudge的稳健性。

View on arXiv Download PDF AI Translation

cs.CL / 39 / 2606.03197

MemTrain: Self-Supervised Context Memory Training

MemTrain：自监督上下文记忆训练

Li, Ziheng, Xing, Xingrun, Wang, Haoqing, Deng, Zhi-Hong, Tang, Yehui

Abstract

Memory is an indispensable capability for long-horizon LLM agents, enabling them to preserve and utilize information accumulated across extended interactions. Existing memory-agent approaches are typically trained end-to-end with reinforcement learning on downstream tasks. However, collecting high-quality annotated problems for memory-intensive scenarios is costly, and the resulting training data often lack sufficient diversity to cover general memory behaviors. In this work, we propose MemTrain, a self-supervised training framework for generally enhancing the context-memory capability of LLM agents for more effective downstream post-training. MemTrain introduces two coupled proxy tasks over unlabeled Wikipedia corpora: (1) an end-to-end masked reconstruction objective, which requires the model to recover masked entities after multiple rounds of memory updates, thereby encouraging memory maintenance from the final outcome perspective; and (2) an intermediate memory recall objective, which requires the model to reconstruct masked historical information using intermediate memory states, encouraging faithful compression and memory completeness throughout the interaction process. The two objectives are jointly optimized using GRPO. Extensive experiments on long-text QA and search-based QA benchmarks demonstrate that MemTrain consistently improves downstream memory-intensive reasoning performance across different models, achieving gains of up to 17.67 points over direct task-specific post-training.

Chinese Translation

记忆是长时间跨度大语言模型（LLM）代理不可或缺的能力，使其能够在长期交互中保存和利用累积的信息。现有的记忆代理方法通常通过强化学习在下游任务上进行端到端训练。然而，收集高质量的标注问题以应对记忆密集型场景的成本较高，且所得到的训练数据往往缺乏足够的多样性，无法覆盖一般的记忆行为。在本研究中，我们提出了MemTrain，一种自监督训练框架，旨在普遍增强LLM代理的上下文记忆能力，以实现更有效的下游后续训练。MemTrain在未标注的维基百科语料库上引入了两个耦合的代理任务：（1）一个端到端的掩码重建目标，要求模型在多轮记忆更新后恢复被掩盖的实体，从而从最终结果的角度鼓励记忆维护；（2）一个中间记忆回忆目标，要求模型利用中间记忆状态重建被掩盖的历史信息，鼓励在交互过程中保持忠实的压缩和记忆完整性。这两个目标通过GRPO进行联合优化。在长文本问答和基于搜索的问答基准上的大量实验表明，MemTrain在不同模型上持续提高了下游记忆密集型推理性能，相较于直接的任务特定后续训练，提升幅度最高可达17.67分。

View on arXiv Download PDF AI Translation

cs.CL / 40 / 2606.03198

AI Rater Discrimination Depends on Scoring Protocol in Complex Clinical Decision-Making

AI评分者的区分能力依赖于复杂临床决策中的评分协议

Baek, Sangwon, Hur, Kyu Yeon, Kim, Kyunga

Abstract

Clinical AI evaluation increasingly delegates scoring to large language models (LLMs) acting as AI raters, yet their scoring behavior across evaluation conditions has not been quantitatively characterized. We address this gap through a factorial study of AI rater behavior in adult type 2 diabetes (T2D) pharmacotherapy at 12-month outpatient follow-up, a clinical task involving complex decision-making operationalized across seven evaluation questions. Four open-source LLMs served simultaneously as clinical decision support system (CDSS) models and AI raters. Each CDSS output was scored under two scoring protocols: a rubric-anchored Gold Rubric (GR) protocol incorporating a patient-specific rubric, and a rubric-free Non Gold Rubric (Non-GR) protocol. Linear mixed effects models crossed the scoring protocol factor with five design factors -- CDSS model, CDSS prompt configuration (document-referenced generation [DRG] vs.\ Baseline), rater model, prompt character, and prompt type -- and estimated main effects together with their protocol interactions. Across all questions, AI raters yielded consistently higher scores within a very narrow range (74--78 points on average) under Non-GR compared to those under GR (7.69 to 49.64 points lower mean scores; 1.68 to 3.67 times wider interquartile ranges). Within each question, GR amplified the AI rater's discrimination between DRG and Baseline CDSS outputs by factors of 1.76 to 5.10, while also revealing substantial behavioral variation across rater models that Non-GR suppressed. These findings support rubric anchoring as the scoring protocol that preserves discriminative power in clinical AI evaluation; rubric-free scoring cannot substitute when questions require patient-specific or jurisdiction-specific criteria that rater models cannot infer from parametric knowledge alone.

Chinese Translation

临床AI评估日益将评分工作委托给作为AI评分者的大型语言模型（LLMs），然而它们在不同评估条件下的评分行为尚未得到定量表征。我们通过对成人2型糖尿病（T2D）药物治疗在12个月门诊随访中的AI评分者行为进行的因子研究来填补这一空白，该临床任务涉及通过七个评估问题进行复杂决策。四个开源LLM同时作为临床决策支持系统（CDSS）模型和AI评分者。每个CDSS输出在两种评分协议下进行评分：一种是以评分标准为基础的金标准（Gold Rubric, GR）协议，包含患者特定的评分标准；另一种是无评分标准的非金标准（Non Gold Rubric, Non-GR）协议。线性混合效应模型将评分协议因素与五个设计因素交叉——CDSS模型、CDSS提示配置（文档引用生成[DRG]与基线[Baseline]）、评分者模型、提示字符和提示类型——并估计主效应及其协议交互效应。在所有问题中，AI评分者在Non-GR下的评分普遍高于GR，且范围非常狭窄（平均74至78分），与GR相比，平均分数低7.69至49.64分；四分位数范围宽度增加1.68至3.67倍。在每个问题中，GR放大了AI评分者在DRG和基线CDSS输出之间的区分能力，倍数为1.76至5.10，同时也揭示了不同评分者模型之间的显著行为差异，而Non-GR则抑制了这种差异。这些发现支持以评分标准为基础的评分协议在临床AI评估中保持区分能力；当问题需要患者特定或司法特定标准，而评分者模型无法仅通过参数知识推断时，无评分标准的评分无法替代。

View on arXiv Download PDF AI Translation

cs.CL / 41 / 2606.03219

Sample-Size Scaling of the African Languages NLI Evaluation

非洲语言自然语言推理评估的样本规模扩展

Tiwari, Anuj, Ogunremu, Oluwapelumi, Oko-odion, Terry, Egbewale, Jesujuwon, Nwokocha, Hannah

Abstract

African languages have very little labelled data, and it is unclear if augmenting the quantity of annotation data reliably enhances downstream performance. The study is a systematic sample-size scaling study of natural language inference (NLI) on 16 African languages based on the AfriXNLI benchmark. Under controlled conditions, two multilingual transformer models with roughly 0.6B parameters XLM-R Large fine-tuned on XNLI and AfroXLM-R Large are tested on sample sizes of between 50 and 500 labeled examples and average their results across random subsampling runs. As opposed to the usual belief of monotonic increase with increased data, we find a strongly language sensitive and often non-monotonic scaling behavior. Some languages show early saturation or decrease in performance with sample size as well as high variance in low resource regimes. These results indicate that the volume of data is not enough to guarantee stable profits to African NLI, creating the necessity of language sensitive datasets creation and stronger multi-lingual modelling strategies.

Chinese Translation

非洲语言的标注数据非常有限，目前尚不清楚增加标注数据的数量是否能够可靠地提升下游性能。本研究基于AfriXNLI基准，对16种非洲语言的自然语言推理（NLI）进行了系统的样本规模扩展研究。在受控条件下，测试了两个多语言变换器模型，分别是参数约为0.6B的XLM-R Large（在XNLI上进行微调）和AfroXLM-R Large，样本规模在50到500个标注示例之间，并对随机子抽样运行的结果进行了平均。与通常认为的数据增加会单调提升性能的观点相反，我们发现了强烈的语言敏感性和常常非单调的扩展行为。一些语言在样本规模上表现出早期饱和或性能下降，以及在低资源环境下的高方差。这些结果表明，数据量不足以保证非洲NLI的稳定收益，因此需要创建语言敏感的数据集和更强的多语言建模策略。

View on arXiv Download PDF AI Translation

cs.CL / 42 / 2606.03220

WebRISE: Requirement-Induced State Evaluation for MLLM-Generated Web Artifacts

WebRISE：基于需求的状态评估用于MLLM生成的网页工件

Meng, Yuxin, Suo, Yuhan, Wang, Junjie, Sun, Yuhan, Yu, Yiyao, Zhang, Ruixu, Hu, Ruining, Wang, Yubin, Ruan, Shouwei, Wang, Bin, Zhang, Yuxiang, Yang, Yujiu

Abstract

Existing benchmarks for MLLM-generated web artifacts assess interaction through local evidence and miss the requirement-induced states and transitions that determine whether a page works. We introduce WebRISE, which compiles task requirements into Interaction Contract Graphs (ICGs) of observable states, user-intent transitions, and DOM/visual assertions for implementation-agnostic browser execution. WebRISE spans 442 tasks across five input modalities (Text, Markdown, Sketch, Image, Video), with 5,495 transitions and 5,271 requirement checks that separate user-stated functions from implicit product-level constraints. Across 14 MLLMs, even the strongest model reaches only 65.6% transition validity and 66.3% requirement coverage, and visual quality is no proxy for behavior (Qwen3.6-35B-A3B on Markdown: V=80.8 yet T=15.5). Video gives the strongest interaction signal (+10.6 pp implicit coverage over Text), while implicit constraints persist; defect injection shows ICG-based scoring detects state errors at 2-16x the rate of checkpoint-style evaluation.

Chinese Translation

现有的针对MLLM生成网页工件的基准通过局部证据评估交互，忽略了决定页面是否正常工作的需求引导状态和转变。我们提出了WebRISE，它将任务需求编译为可观察状态、用户意图转变和DOM/视觉断言的交互合同图（Interaction Contract Graphs, ICGs），以实现与实现无关的浏览器执行。WebRISE涵盖了442个任务，涉及五种输入模态（文本、Markdown、草图、图像、视频），包含5,495个转变和5,271个需求检查，能够区分用户陈述的功能与隐含的产品级约束。在14个MLLM中，即使是最强的模型也仅达到65.6%的转变有效性和66.3%的需求覆盖率，视觉质量并不能作为行为的代理（Qwen3.6-35B-A3B在Markdown上的表现：V=80.8但T=15.5）。视频提供了最强的交互信号（比文本隐含覆盖率高出10.6个百分点），而隐含约束依然存在；缺陷注入显示基于ICG的评分在检测状态错误方面的速度是检查点式评估的2-16倍。

View on arXiv Download PDF AI Translation

cs.CL / 43 / 2606.03239

ARBOR: Online Process Rewards via a Reusable Rubric Buffer for Search Agents

ARBOR：通过可重用的评分缓冲区为搜索代理提供在线过程奖励

Liu, Zheng, Zhang, Longxiang, Wang, Xintong, Xu, Zhiang, Zhan, Shaoxiong, Shan, Xin, Huang, Wen, Dai, Tao, Xia, Shu-Tao, Huo, Chengfu, Ding, Liang

Abstract

LLM-based search agents are trained predominantly with outcome-only reward, leaving the search process itself unsupervised. This signal degenerates on outcome-homogeneous groups where all sampled trajectories share the same correctness, yielding zero within-group advantage and no gradient. Existing process supervision either trains a costly verifier or generates per-query rubrics that are inconsistent across queries and discarded after one use. We propose ARBOR (Adaptive Rubric Buffer for Online Reward), a reusable process-reward framework that maintains a rubric memory shared across queries. Query-local drafts induced from contrastive trajectories are admitted, consolidated into cross-query common rubrics, and retired as the policy evolves. A small active subset of common rubrics scores trajectories via sparse pairwise judging, and the resulting scores are added to the base reward, providing process-level gradient even when outcome reward is uniform. ARBOR consistently outperforms GRPO and DAPO baselines on four multi-hop QA benchmarks, raising average LLM-judge accuracy by up to 4.2 points and converting up to 42% of otherwise-zero-gradient training groups into informative ones.

Chinese Translation

基于大语言模型（LLM）的搜索代理主要依赖于仅基于结果的奖励进行训练，这使得搜索过程本身处于无监督状态。这种信号在结果同质的群体中退化，因为所有采样的轨迹共享相同的正确性，导致组内没有优势且没有梯度。现有的过程监督要么训练一个成本高昂的验证器，要么为每个查询生成不一致的评分标准，并在使用一次后被丢弃。我们提出了ARBOR（在线奖励的自适应评分缓冲区），这是一个可重用的过程奖励框架，维护一个跨查询共享的评分记忆。从对比轨迹中诱导的查询局部草稿被接纳，并整合为跨查询的公共评分标准，随着策略的演变而退役。一个小的活跃公共评分标准子集通过稀疏成对判断为轨迹打分，所得到的分数被添加到基础奖励中，即使在结果奖励均匀的情况下，也提供了过程级的梯度。ARBOR在四个多跳问答基准测试中始终优于GRPO和DAPO基线，将平均LLM评判准确率提高了多达4.2个百分点，并将多达42%的原本零梯度的训练组转化为信息丰富的组。

View on arXiv Download PDF AI Translation

cs.CL / 44 / 2606.03241

Benchmarking Speech-to-Speech Translation Models

语音到语音翻译模型的基准测试

Koudounas, Alkis, Futami, Hayato, Jodelet, Quentin, Take, Osamu, Watanabe, Shinji, Tsunoo, Emiru

Abstract

Speech-to-speech translation (S2ST) has advanced rapidly, but offline evaluation lacks a unified protocol: studies report non-overlapping metric subsets, preventing direct comparisons. We introduce COMPASS, a unified and reproducible benchmarking framework integrating 46 metrics across eight dimensions, and deploy it on 1,248 model-language configurations from FLEURS and CVSS, spanning cascaded and end-to-end architectures over ten language pairs. Architectures exhibit complementary strengths: best-vs-worst gaps exceed 30\% on naturalness and speaker preservation but remain within a few points on translation quality, so single-metric rankings systematically misrepresent system quality. Correlation filtering reduces 46 metrics to 10 per direction, with three axes requiring different metrics across X$\to$EN and EN$\to$X (e.g., TER/UTMOS vs. ChrF++/NISQA-MOS); these subsets preserve rankings (Spearman's $\rho>0.80$) while cutting evaluation time by $\approx 2.5\times$. Human validation across dubbing, podcasts, and medical domains shows standalone MOS predictors fail to predict listener preference, while top domain-specific metrics correlate with human judgment ($\rho \geq 0.90$). We release COMPASS as a foundation for domain-aware S2ST evaluation.

Chinese Translation

语音到语音翻译（S2ST）发展迅速，但离线评估缺乏统一的协议：研究报告的度量子集不重叠，阻碍了直接比较。我们引入了COMPASS，一个统一且可重复的基准测试框架，整合了46个度量指标，涵盖八个维度，并在来自FLEURS和CVSS的1,248个模型-语言配置上进行部署，涉及十对语言的级联和端到端架构。架构展现出互补的优势：在自然性和说话者保留方面，最佳与最差之间的差距超过30%，而在翻译质量上则保持在几分之内，因此单一度量排名系统性地误代表了系统质量。相关性过滤将46个度量减少到每个方向的10个，其中三个轴在X$ o$EN和EN$ o$X之间需要不同的度量（例如，TER/UTMOS与ChrF++/NISQA-MOS）；这些子集在保留排名（斯皮尔曼相关系数 $ ho>0.80$）的同时将评估时间缩短了约2.5倍。在配音、播客和医疗领域的人类验证表明，独立的MOS预测器无法预测听众偏好，而顶级领域特定度量与人类判断相关性较高（$ ho geq 0.90$）。我们发布COMPASS作为领域感知S2ST评估的基础。

View on arXiv Download PDF AI Translation

cs.CL / 45 / 2606.03244

When Does Complexity Conditioning Help a Frozen Sentence Embedding? A Controlled Study of Per-Sentence and Pair-Level Difficulty Adaptation

复杂性调节何时有助于冻结的句子嵌入？对逐句和对级别难度适应的控制研究

Hwang, Suhwan

Abstract

A common intuition is that sentence embeddings should adapt to the difficulty of the input. We test this intuition in a controlled, multi-seed setting: a lightweight post-encoder adapter attaches to a frozen Qwen3-Embedding-0.6B encoder, accessing only its final pooled embedding, and is evaluated on four paraphrase and semantic-similarity tasks (PAWS, MRPC, QQP, STS-B). The naive form of the idea fails: surface-based per-sentence complexity is nearly uncorrelated with frozen-baseline error (Pearson approximately 0.05) and provides no advantage over constant or shuffled controls, while degrading a saturated baseline. Even when the target is aligned to a non-circular pair-difficulty signal, the per-sentence gate still cannot reliably capture difficulty because difficulty is primarily a property of the pair, not the individual sentence. In contrast, a small pair-level residual gated by a held-out cross-encoder difficulty signal yields consistent gains on the larger and graded tasks, including +0.022 Spearman on STS-B and +0.037 on QQP, while remaining anchored to the frozen baseline across all seeds. Because this useful form operates on sentence pairs rather than individual sentences, the resulting model is best understood as a lightweight re-ranker over cached frozen embeddings, not a replacement single-vector embedding; we make no state-of-the-art claim. Our contribution is a controlled account of when difficulty-aware adaptation helps and when it fails, together with a pre-training diagnostic that predicts the available headroom.

Chinese Translation

一个普遍的直觉是，句子嵌入应该适应输入的难度。我们在一个受控的多种子设置中测试这一直觉：一个轻量级的后编码器适配器附加在冻结的 Qwen3-Embedding-0.6B 编码器上，仅访问其最终的池化嵌入，并在四个同义句和语义相似性任务（PAWS、MRPC、QQP、STS-B）上进行评估。该想法的简单形式失败了：基于表面的逐句复杂性与冻结基线误差几乎不相关（Pearson 约为 0.05），并且在与常量或随机控制相比没有优势，同时还降低了饱和基线的表现。即使目标与非循环的对级别难度信号对齐，逐句门控仍然无法可靠地捕捉难度，因为难度主要是对的属性，而不是单个句子的属性。相比之下，一个小的对级别残差由一个保留的交叉编码器难度信号门控，在较大和分级任务上产生了一致的增益，包括在 STS-B 上 +0.022 的 Spearman 和在 QQP 上 +0.037，同时在所有种子上保持与冻结基线的锚定。由于这种有用的形式是针对句子对而不是单个句子操作，所得到的模型最好理解为对缓存的冻结嵌入进行轻量级的重新排序，而不是替代单一向量嵌入；我们没有提出任何最先进的主张。我们的贡献是对何时难度感知适应有帮助以及何时失败的控制性描述，以及一个预测可用余量的预训练诊断。

View on arXiv Download PDF AI Translation

cs.CL / 46 / 2606.03247

Structures Facilitate Retrieve, Rerank, and Generate

结构促进检索、重排序和生成

Zhang, Yeqin, Fu, Haomin, Zhang, Xujie, Nguyen, Cam-Tu

Abstract

Document-grounded dialogue systems (DGDS) utilize knowledge from external documents to answer domain-specific user questions. Existing solutions typically divide documents into independent passages for retrieval and response generation. This approach, however, neither makes good use of structural information within documents nor provides enough (document) context for knowledge selection and responses. This paper proposes SF-Re2G to address such issues systematically. Firstly, we seek to improve a passage representation by contrasting it with others of the same section, thus improving the retrieval performance. Secondly, a structure-enhanced reranker is built, leveraging the fact that multiple grounding passages of one dialog turn tend to be in the same neighborhood. Specifically, candidates from the retrieval are grouped into subgraphs according to the document structure. The reranker will rescore the candidate integrating its group information. Finally, the chosen passages are used for responses, taking into account the subgraph context for better generation. Experimental results on two DGDS datasets validate our method for both Chinese and English.

Chinese Translation

文档基础对话系统（DGDS）利用外部文档中的知识来回答特定领域的用户问题。现有的解决方案通常将文档划分为独立的段落进行检索和响应生成。然而，这种方法既未充分利用文档内部的结构信息，也未提供足够的（文档）上下文以便于知识选择和响应。本文提出了SF-Re2G，以系统性地解决这些问题。首先，我们通过将段落与同一部分的其他段落进行对比，来改善段落表示，从而提高检索性能。其次，构建了一个结构增强的重排序器，利用多个基础段落在同一对话轮次中往往位于同一邻域的事实。具体而言，检索到的候选段落根据文档结构被分组为子图。重排序器将整合其组信息重新评分候选段落。最后，选择的段落将用于响应，同时考虑子图上下文以实现更好的生成。在两个DGDS数据集上的实验结果验证了我们的方法在中文和英文中的有效性。

View on arXiv Download PDF AI Translation

cs.CL / 47 / 2606.03250

The Word and the Way: Strategies for Domain-Specific BERT Pre-Training in German Medical NLP

词与道：德语医学自然语言处理中的领域特定BERT预训练策略

He, Henry, Frei, Johann, Schmitt, Raphael

Abstract

Digital healthcare generates vast amounts of clinical text that can support AI-assisted applications, yet German biomedical language models remain limited by older architectures or restricted training data. We present ChristBERT (Clinical- and Healthcare-Related Issues and Subjects Tuned BERT), a family of domain-specific German RoBERTa-based language models trained on a 13.5GB corpus of scientific publications, clinical texts, health-related web content, and translated clinical resources. To investigate the impact of domain adaptation strategies in German clinical NLP, we compare continued pre-training, training from scratch, and domain-specific vocabulary adaptation. The resulting models are evaluated on three medical named entity recognition tasks and two text classification tasks. ChristBERT consistently outperforms existing general-purpose and medical German language models on four of five benchmarks and establishes a new state of the art for German clinical language modeling. Our results show that the optimal adaptation strategy is task-dependent: in our evaluation, training from scratch is particularly effective for highly specialized clinical texts, whereas continued pre-training performs well on more commonly written medical texts. All models are publicly released to support future research and applications in German medical NLP.

Chinese Translation

数字医疗生成大量临床文本，这些文本可以支持人工智能辅助应用，但德语生物医学语言模型仍受限于较旧的架构或受限的训练数据。我们提出了ChristBERT（临床与医疗相关问题和主题调优的BERT），这是一个基于RoBERTa的领域特定德语语言模型家族，训练于一个包含13.5GB科学出版物、临床文本、健康相关网页内容和翻译临床资源的语料库。为了研究领域适应策略在德语临床自然语言处理中的影响，我们比较了继续预训练、从头开始训练和领域特定词汇适应。所得到的模型在三个医学命名实体识别任务和两个文本分类任务上进行了评估。ChristBERT在五个基准中的四个上始终优于现有的通用和医学德语语言模型，并为德语临床语言建模建立了新的最先进水平。我们的结果表明，最佳适应策略依赖于任务：在我们的评估中，从头开始训练对高度专业化的临床文本特别有效，而继续预训练在更常见的医学文本上表现良好。所有模型均已公开发布，以支持未来在德语医学自然语言处理中的研究和应用。

View on arXiv Download PDF AI Translation

cs.CL / 48 / 2606.03259

Beyond "To whom it may concern": Tailoring Machine Translation to Audience and Intent

超越“致有关人士”：根据受众和意图定制机器翻译

Merx, Raphael, Vylomova, Ekaterina, Cohn, Trevor

Abstract

Translation quality depends on purpose: the same source text demands different translations depending on audience, tone, and communicative intent. Yet MT models and metrics treat translation as a fixed mapping from source to target. LLMs enable users to explicitly specify purpose alongside source text, yet this capability has not been evaluated at scale. We introduce a systematic evaluation of purpose-driven MT across 50 languages, 5 model sizes and 8 text domains. We find that (1) explicit instructions substantially improve translation adaptedness, with larger gains on informal domains (conversation, social media), for larger model sizes and for higher-resource languages; (2) instructions outperform semantically-matched few-shot examples and paragraph-level context; (3) traditional MT metrics fail to capture adaptation quality, often penalizing adapted translations; (4) when curated instructions are unavailable, models can self-generate them from surrounding document context, closing up to 80% of the adaptedness gap to curated instructions. Our results establish that purpose-adapted MT is a viable and measurable capability of LLMs, while highlighting the need for purpose-aware metrics.

Chinese Translation

翻译质量依赖于目的：同一源文本根据受众、语气和交流意图的不同需要不同的翻译。然而，机器翻译（MT）模型和指标将翻译视为从源到目标的固定映射。大型语言模型（LLMs）使用户能够在源文本的基础上明确指定目的，但这一能力尚未在大规模上进行评估。我们对50种语言、5种模型规模和8个文本领域的目的驱动机器翻译进行了系统评估。我们的研究发现：（1）明确的指令显著提高了翻译的适应性，在非正式领域（对话、社交媒体）中、较大模型规模和高资源语言中获得了更大的提升；（2）指令的表现优于语义匹配的少量示例和段落级上下文；（3）传统的机器翻译指标未能捕捉适应质量，常常对适应翻译进行惩罚；（4）当缺乏精心策划的指令时，模型可以从周围文档上下文中自我生成指令，缩小与精心策划指令之间的适应性差距，达到80%。我们的结果表明，目的适应的机器翻译是大型语言模型的一个可行且可测量的能力，同时强调了对目的感知指标的需求。

View on arXiv Download PDF AI Translation

cs.CL / 49 / 2606.03284

SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding

SEA-NLI：以自然语言推理为视角理解东南亚文化

Chomphooyod, Peerawat, Ngui, Jian Gang, Susanto, Yosephine, Rutherford, Attapol T., Aji, Alham Fikri, Nutanong, Sarana, Udomcharoenchaikit, Can, Limkonchotiwat, Peerat

Abstract

Frontier LLMs perform well in Western contexts, but remain poorly tested on underrepresented cultures such as those in Southeast Asia (SEA). Existing NLI benchmarks are largely Western-centric, translation-derived, or monolingual, limiting their ability to measure culturally grounded reasoning. We introduce SEA-NLI, a native, culturally grounded NLI benchmark covering eight SEA countries in English and native regional languages, verified by native speakers. Across 17 encoder and decoder models, we observe a low performance from all models, especially for knowledge-intensive categories such as Languages and Science and Technology. Our analysis shows that failure cases mainly stem from missing SEA cultural knowledge: SEA-adapted models and culture-aware prompting improve performance, while CoT prompting offers limited gains.

Chinese Translation

前沿的大型语言模型在西方语境中表现良好，但在东南亚（SEA）等代表性不足的文化中测试较少。现有的自然语言推理（NLI）基准主要以西方为中心，源于翻译或单语，限制了其测量文化基础推理的能力。我们提出了SEA-NLI，这是一个本土化的、文化基础的NLI基准，涵盖了八个东南亚国家的英语和本地区域语言，并由母语者验证。在17个编码器和解码器模型中，我们观察到所有模型的表现都较低，尤其是在语言、科学和技术等知识密集型类别中。我们的分析表明，失败案例主要源于缺乏东南亚文化知识：适应东南亚的模型和文化意识提示提高了性能，而链式推理（CoT）提示的提升有限。

View on arXiv Download PDF AI Translation

cs.CL / 50 / 2606.03291

Multilingual Unlearning in LLMs: Transfer, Dynamics, and Reversibility

大语言模型中的多语言遗忘：迁移、动态与可逆性

Xiang, Chaoyi, Ohrimenko, Olga, Rubinstein, Benjamin I. P., Frermann, Lea

Abstract

Large language models (LLMs) can memorize sensitive facts, motivating unlearning methods that remove targeted knowledge without costly retraining. However, unlearning research remains heavily English-centric. We study multilingual unlearning by extending the TOFU benchmark to five languages, and fine-tune, unlearn, and query our models with different permutations of languages. We find that unlearning transfer, the ability of an unlearned model to "forget" facts in languages other than the unlearning language, is highly variable: e.g., it is strongest between languages sharing scripts and families, and we show that the unlearning language predicts which query languages are most likely to yield the strongest transfer. Layer-wise analysis reveals that unlearning leaves the shared cross-lingual latent space largely intact in early layers, instead operating primarily in later decoding layers. This suggests that unlearning does not truly erase knowledge, but rather induces superficial suppression. Exploiting this structure, a single inference-time steering direction reverses much of this suppression across languages, recovering 50% (Qwen) and 90% (Gemma) of the unlearned knowledge.

Chinese Translation

大型语言模型（LLMs）能够记忆敏感信息，这促使了去学习方法的研究，以在不进行昂贵的再训练的情况下移除特定知识。然而，去学习研究仍然高度集中于英语。我们通过将TOFU基准扩展到五种语言，研究多语言去学习，并对我们的模型进行微调、去学习和使用不同语言的排列进行查询。我们发现，去学习迁移，即去学习模型在非去学习语言中“遗忘”事实的能力，变化很大：例如，在共享脚本和语言家族的语言之间最强。我们还表明，去学习语言可以预测哪些查询语言最有可能产生最强的迁移。逐层分析表明，去学习在早期层中基本保持了共享的跨语言潜在空间不变，而主要在后期解码层中进行操作。这表明，去学习并不是真正抹去知识，而是引发表面的抑制。利用这种结构，在推理时的单一引导方向可以逆转跨语言的大部分抑制，恢复50%（Qwen）和90%（Gemma）的去学习知识。

View on arXiv Download PDF AI Translation

cs.CL / 51 / 2606.03301

SagaQA: A Multi-hop Reasoning Benchmark for Long-form Narrative Understanding in TV Series

SagaQA：用于电视剧长篇叙事理解的多跳推理基准

Pennec, Galann, Liu, Zhengyuan, Asher, Nicholas, Muller, Philippe, Chen, Nancy F.

Abstract

We introduce SagaQA, a long-form video benchmark for multi-hop reasoning over full-length TV series. Existing video reasoning benchmarks often emphasize local understanding of adjacent frames or clips. SagaQA addresses this gap by requiring high-level comprehension of extended multimodal narratives in entire TV shows. A distinguishing feature of SagaQA is the granularity of its reasoning steps. Our dataset necessitates long-range reasoning hops to connect information across completely different episodes. This requires models to reason over entire events and actions, demanding a deep understanding of the show's narration and progression at a multimodal level. Motivated by recent progress in agentic methods, we further study how different planning strategies handle such complex reasoning. We categorize these approaches into three classes-Parallel, Sequential, and Hybrid planners-and evaluate their ability to generate coherent and complete reasoning plans. Our results on SagaQA suggest that hybrid planners consistently produce higher-quality plans and exhibit stronger capabilities for complex, high-level narrative understanding in TV shows.

Chinese Translation

我们介绍了SagaQA，这是一个针对完整电视剧进行多跳推理的长篇视频基准。现有的视频推理基准通常强调对相邻帧或片段的局部理解。SagaQA填补了这一空白，要求对整个电视剧中的扩展多模态叙事进行高水平的理解。SagaQA的一个显著特点是其推理步骤的粒度。我们的数据集需要长距离的推理跳跃，以连接完全不同剧集中的信息。这要求模型对整个事件和行动进行推理，深刻理解节目的叙述和进展，且需在多模态层面上进行分析。受到近期代理方法进展的启发，我们进一步研究不同规划策略如何处理这种复杂推理。我们将这些方法分为三类——并行规划者、顺序规划者和混合规划者，并评估它们生成连贯且完整的推理计划的能力。我们在SagaQA上的结果表明，混合规划者始终产生更高质量的计划，并在电视剧的复杂高层叙事理解方面表现出更强的能力。

View on arXiv Download PDF AI Translation

cs.CL / 52 / 2606.03304

From Script to Semantics: Prompting Strategies for African NLI

从脚本到语义：非洲自然语言推理的提示策略

Tiwari, Anuj, Oko-odion, Terry, Nwokocha, Hannah

Abstract

Large language models (LLMs) are increasingly evaluated in multilingual settings, yet their inference behavior in low-resource African languages remains underexplored especially under pure prompting without fine-tuning. We present a systematic study of prompting strategies for Natural Language Inference (NLI) in Swahili, Yoruba, and Hausa using the AfriXNLI benchmark. We evaluate five prompting strategies Baseline (zero-shot), Script-Aware, Language Specific, Contrastive, and Native-Label Self-Translation (NL-STP) across two mid-sized open weight models (Llama3.2-3B and Gemma3-4B). To isolate the effect of prompt design, the effect of few-shot examples and Chain-of-Thought reasoning is eliminated in our study. We find a significant difference in performance of class wise across strategies with highly neutral class collapse and high prediction skew in some configurations. Contrastive prompting proves to be the most reliable and steadily improving strategy over language and model and has better balance of class behavior and balance of overall accuracy gains. Notably, well-constructed prompts are sufficient to beat more powerful baselines that are provided with few-shot prompts and Chain-of-Thought prompts. We have found that prompt formulation is essential to multilingual NLI with low-resource languages and that language aware decision structuring can be used to meaningfully enhance robustness in resource challenged settings.

Chinese Translation

大型语言模型（LLMs）在多语言环境中的评估日益增多，但它们在低资源非洲语言中的推理行为仍然未得到充分探索，特别是在没有微调的纯提示情况下。我们针对斯瓦希里语、约鲁巴语和豪萨语的自然语言推理（NLI）进行了系统研究，使用了AfriXNLI基准。我们评估了五种提示策略：基线（零样本）、脚本感知、语言特定、对比性和本地标签自翻译（Native-Label Self-Translation, NL-STP），并在两个中型开放权重模型（Llama3.2-3B和Gemma3-4B）上进行了测试。为了隔离提示设计的影响，我们的研究排除了少量示例和思维链推理的影响。我们发现，不同策略在类别性能上存在显著差异，某些配置中出现了高度中立的类别崩溃和高预测偏斜。对比性提示被证明是最可靠且在语言和模型上持续改进的策略，具有更好的类别行为平衡和整体准确性提升的平衡。值得注意的是，构造良好的提示足以超越提供少量示例提示和思维链提示的更强基线。我们发现，提示的构造对低资源语言的多语言NLI至关重要，语言感知的决策结构可以在资源受限的环境中有意义地增强鲁棒性。

View on arXiv Download PDF AI Translation

cs.CL / 53 / 2606.03318

Beyond Ideal Instruction: A Comprehensive Framework for Evaluating LLMs in Realistic Interactions

超越理想化指令：评估大型语言模型在现实交互中的综合框架

Yang, Xuan, Xu, Hao, Hui, Tingfeng, Xin, Hongsheng, Zhang, Kaike, Liu, Chunxiao, Miao, Ning

Abstract

Despite great advances in tool-use capabilities of large language models (LLMs), existing evaluation benchmarks struggle to fully align with real-world scenarios. Such benchmarks mostly rely on simulated idealized user assumptions and lacks experience-oriented evaluation. These limitations fail to account for the ambiguity, uncooperative behaviors, and shifting intentions characteristic of real-world users. To fill this gap, we propose RUT-Bench, a dedicated benchmark designed to assess LLMs under diverse Real-world User Tool calling scenarios. RUT-Bench supports high-fidelity simulations covering both ideal rational patterns and heterogeneous non-ideal behaviors across single-turn and multi-turn dialogues. We conduct comprehensive evaluations on 19 widely adopted open-source and proprietary LLMs using our benchmark. Experimental results reveal that no tested LLMs achieve an overall success rate above 40%, and nearly all of them experience noticeable performance drops when facing more complicated non-ideal user inputs. Our code and data is available at https://github.com/TorresYangX/RUT-Bench.

Chinese Translation

尽管大型语言模型（LLMs）在工具使用能力方面取得了重大进展，但现有的评估基准难以与现实世界场景完全对齐。这些基准主要依赖于模拟的理想化用户假设，缺乏以经验为导向的评估。这些局限性未能考虑现实用户特有的模糊性、不合作行为和变化的意图。为填补这一空白，我们提出了RUT-Bench，这是一个专门设计的基准，旨在评估LLMs在多样化的现实用户工具调用场景下的表现。RUT-Bench支持高保真模拟，涵盖理想理性模式和异质非理想行为，适用于单轮和多轮对话。我们使用该基准对19个广泛采用的开源和专有LLMs进行了全面评估。实验结果表明，所有测试的LLMs的整体成功率均未超过40%，几乎所有模型在面对更复杂的非理想用户输入时都出现了明显的性能下降。我们的代码和数据可在https://github.com/TorresYangX/RUT-Bench获取。

View on arXiv Download PDF AI Translation

cs.CL / 54 / 2606.03331

Evaluating LLMs' Effectiveness on Real-World Consumer Device Repair Questions

评估大型语言模型在现实世界消费设备维修问题上的有效性

Rahman, Atm Mizanur, Hasan, Md Arid, Ahmed, Syed Ishtiaque, Sultana, Sharifa

Abstract

Consumer device repair is an important but underexplored testbed for large language models (LLMs). Repair tasks require reasoning over incomplete problem descriptions, hardware-specific diagnostics, actionable troubleshooting, and safety-critical decisions, where incorrect advice can cause device damage, battery hazards, or permanent data loss. We introduce a benchmark of 991 real-world repair questions from Reddit spanning phone repair, computer repair, and data recovery, each paired with technician-written reference solutions, and provide Bangla translations to evaluate cross-lingual performance. We evaluate six state-of-the-art LLMs in English and Bangla using four repair-specific criteria: correctness, completeness, practicality, and safety. Our results show that while LLMs can provide useful repair assistance, they remain unreliable for high-risk real-world repair tasks without rigorous evaluation and explicit safety safeguards. Phone repair is the most difficult and safety-sensitive domain, and all models make substantial errors in board-level diagnosis, repair prioritization, and safe recovery procedures. Across domains and models, Bangla responses consistently perform worse than English responses. Among the evaluated models, GPT-5.4 performs best overall.

Chinese Translation

消费设备维修是一个重要但尚未充分探索的大型语言模型（LLMs）测试平台。维修任务需要对不完整的问题描述进行推理，进行硬件特定的诊断，提供可操作的故障排除建议，并做出安全关键的决策，其中错误的建议可能导致设备损坏、电池危险或永久数据丢失。我们介绍了一个基准数据集，包括来自Reddit的991个现实世界维修问题，涵盖手机维修、计算机维修和数据恢复，每个问题都配有技术人员撰写的参考解决方案，并提供孟加拉语翻译以评估跨语言性能。我们使用四个维修特定标准（正确性、完整性、实用性和安全性）对六个最先进的LLMs在英语和孟加拉语中的表现进行了评估。我们的结果表明，尽管LLMs可以提供有用的维修帮助，但在没有严格评估和明确安全保障的情况下，它们在高风险的现实世界维修任务中仍然不可靠。手机维修是最困难且安全敏感的领域，所有模型在电路板级诊断、维修优先级和安全恢复程序方面均存在重大错误。在各个领域和模型中，孟加拉语响应的表现始终低于英语响应。在评估的模型中，GPT-5.4的整体表现最佳。

View on arXiv Download PDF AI Translation

cs.CL / 55 / 2606.03334

Lingo_Research_Group at SemEval-2026 Task 9: Evaluating Prompt Variants for Polarization Detection

Lingo_Research_Group在SemEval-2026任务9中的表现：极化检测的提示变体评估

Kadasi, Pritam, Tiwari, Anuj, Singh, Mayank

Abstract

Our submission presented in this paper is for SemEval-2026 Task 9: Multilingual Text Classification Challenge - Polarization Detection and it covers all three subtasks: (1) binary polarization detection, (2) polarization type classification and (3) polarization manifestation identification. We adopt a systematic approach of research on short designed prompts by considering twelve designed prompts that are different in terminology clarity, detail of the definition, guidance of reasoning and in-context examples use. The experiments are conducted using aya-101 and Gemma3-27B, with the latter chosen for the submission at the end of the development through performance considerations. Our system has an average macro level F1-score of 0.762 on Subtask 1, 0.587 on Subtask 2 and 0.444 on Subtask 3 with the average accuracy of 0.819, 0.678 and 0.498, respectively, on the official test set averaged among 22 languages, respectively. With cross-task and cross-lingual analysis, we demonstrate that prompt-based approaches can be used effectively to detect coarse grained polarization but encounter more and more difficulties as far as fine-grained and multi-label sociolinguistic classification is concerned.

Chinese Translation

本文提交的内容是针对SemEval-2026任务9：多语言文本分类挑战 - 极化检测，涵盖了所有三个子任务：（1）二元极化检测，（2）极化类型分类和（3）极化表现识别。我们采用了一种系统化的研究方法，设计了十二个在术语清晰度、定义细节、推理指导和上下文示例使用方面有所不同的短提示。实验使用了aya-101和Gemma3-27B进行，后者因性能考虑在开发结束时被选为提交模型。我们的系统在子任务1上的平均宏F1分数为0.762，在子任务2为0.587，在子任务3为0.444，官方测试集中在22种语言上的平均准确率分别为0.819、0.678和0.498。通过跨任务和跨语言的分析，我们展示了基于提示的方法可以有效地用于检测粗粒度极化，但在细粒度和多标签社会语言学分类方面遇到了越来越多的困难。

View on arXiv Download PDF AI Translation

cs.CL / 56 / 2606.03357

The Unsampled Truth: Psychometrics in SLMs Measure Prompt Artifacts, Not Psychological Constructs

未采样的真相：SLMs中的心理测量衡量的是提示伪影，而非心理构念

Schwager, Nils, Hau, Christoph, Münker, Simon, Rettinger, Achim

Abstract

When prompting SLMs for psychometric assessments, researchers assume the outputs reflect semantic reasoning. We evaluate this premise across 13 open-weights models (0.6B to 14B parameters) using a prompt variation framework that separates semantic signals from prompt artifacts. By systematically varying personas, instructions, items, and option symbols, we find that artifactual variance frequently overpowers the semantic signal. In these cases, models predominantly reflect prompt compliance rather than simulated psychological traits. While these findings limit SLM utility in psychometrics, our framework provides a diagnostic tool to identify destructive artifacts and isolate semantic understanding for future frontier-model research.

Chinese Translation

在对SLMs进行心理测量评估时，研究人员假设输出反映了语义推理。我们通过一个提示变异框架评估这一前提，该框架将语义信号与提示伪影分离，涵盖了13个开放权重模型（参数从0.6B到14B）。通过系统地变化角色、指令、项目和选项符号，我们发现伪影方差常常压倒语义信号。在这些情况下，模型主要反映了对提示的遵从，而非模拟的心理特征。尽管这些发现限制了SLM在心理测量中的实用性，但我们的框架提供了一种诊断工具，以识别破坏性伪影并为未来前沿模型研究分离语义理解。

View on arXiv Download PDF AI Translation

cs.CL / 57 / 2606.03363

EntSQL: A Benchmark for Grounding Text-to-SQL in Long-Context Enterprise Knowledge

EntSQL：一个基于长上下文企业知识的文本到SQL基准测试

Liao, Chengxi, Xu, Tao, Chen, Zulong, Xu, Chuanfei, Wang, Yiyan, Wang, Xinyun, Zhang, Yanlong, Chen, Xiaojun, Yang, Zhibo, Wen, Zeyi

Abstract

Text-to-SQL enables natural language access to databases, and recent LLMs have substantially advanced its capabilities. Existing benchmarks such as Spider, BIRD, and Spider~2.0 evaluate schema generalization, large-scale databases, and realistic workflows, but largely overlook enterprise scenarios where SQL generation depends on private business knowledge, such as internal metrics, reporting conventions, and organizational rules. We introduce EntSQL, an enterprise-oriented Text-to-SQL benchmark for evaluating long-context grounding over proprietary business documents. EntSQL contains 1,066 aligned Chinese-English semantic examples across five business domains, with most examples requiring domain knowledge beyond the question and schema and involving complex SQL structures. On English inputs, the best evaluated system reaches only 15.9\% when long-form documents are provided, highlighting the difficulty of grounding SQL generation in enterprise knowledge.

Chinese Translation

文本到SQL技术使自然语言能够访问数据库，近期的大型语言模型（LLMs）显著提升了其能力。现有的基准测试如Spider、BIRD和Spider~2.0评估了模式泛化、大规模数据库和现实工作流程，但在很大程度上忽视了企业场景，其中SQL生成依赖于私有商业知识，如内部指标、报告惯例和组织规则。我们引入了EntSQL，这是一个面向企业的文本到SQL基准测试，用于评估在专有商业文档上的长上下文基础。EntSQL包含1066个对齐的中英文语义示例，涵盖五个商业领域，大多数示例需要超出问题和模式的领域知识，并涉及复杂的SQL结构。在英文输入下，最佳评估系统在提供长文档时仅达到15.9\%，突显了在企业知识中实现SQL生成的困难。

View on arXiv Download PDF AI Translation

cs.CL / 58 / 2606.03371

See, Infer, Intervene: Proactive World Modeling for Goal-Oriented Social Intelligence

观察、推断、干预：面向目标的社会智能的主动世界建模

Zhang, Honghui, Guo, Chenmeinian, Yu, Yichen, Liu, Guanyu, Qin, Yongming, Song, Chongguo, Yang, Mengyue, Yu, Lei, Shi, Tianyu

Abstract

Multimodal retail agents should not only recognize what a customer is doing, but also decide whether and how to assist before an explicit request is made. We study this setting through the See--Infer--Intervene (SII) framework, where a device must see pre-interaction behavior, infer latent customer intent, and act by selecting an appropriate service intervention or choosing to wait. We instantiate SII with the Proactive Intent World Model (PIWM), which represents customer state with AIDA (Attention, Interest, Desire, Action) purchasing phases and BDI (belief, desire, intention) psychological fields, predicts action-conditioned intent transitions, and selects from five response classes: Greet, Elicit, Inform, Recommend, and Hold. We further construct GuidanceSalesBench, a smart-retail benchmark containing state manifests, pre-interaction videos, candidate responses, action-conditioned outcomes, and best-action labels. When conditioned on ground-truth customer state to isolate action selection, PIWM achieves 0.641 macro F1 on 30 held-out target videos, outperforming a zero-shot Qwen2.5-VL-7B baseline and training variants without balanced action supervision; end-to-end video-only selection drops to 0.295, below the 5-class balanced random baseline of 0.414, identifying video-to-state grounding as the dominant deployment-time bottleneck. A preliminary staged real-store pilot (recorded with paid participants performing scripted customer behaviors) reaches 0.579 action macro F1 on 20 fully annotated videos, with 10 additional accessible videos released with index-level labels.

Chinese Translation

多模态零售代理不仅应识别客户的行为，还需在客户明确请求之前决定是否以及如何提供帮助。我们通过观察-推断-干预（See--Infer--Intervene, SII）框架研究这一情境，其中设备必须观察互动前的行为，推断潜在的客户意图，并通过选择适当的服务干预或选择等待来采取行动。我们使用主动意图世界模型（Proactive Intent World Model, PIWM）实例化SII，该模型通过AIDA（注意、兴趣、欲望、行动）购买阶段和BDI（信念、欲望、意图）心理领域来表示客户状态，预测基于行动的意图转变，并从五类响应中选择：问候、引导、通知、推荐和保持。我们进一步构建了GuidanceSalesBench，这是一个智能零售基准，包含状态清单、互动前视频、候选响应、基于行动的结果和最佳行动标签。当基于真实客户状态进行条件化以隔离行动选择时，PIWM在30个保留目标视频上达到了0.641的宏观F1分数，优于零样本Qwen2.5-VL-7B基线和没有平衡行动监督的训练变体；仅基于视频的端到端选择下降至0.295，低于5类平衡随机基线0.414，表明视频与状态的对接是主要的部署时间瓶颈。一项初步的分阶段真实商店试点（记录了付费参与者执行脚本化客户行为）在20个完全注释的视频上达到了0.579的行动宏观F1分数，并发布了10个附带索引级标签的额外可访问视频。

View on arXiv Download PDF AI Translation

cs.CL / 59 / 2606.03398

Causal Evidence of Stack Representations in Modeling Counter Languages Using Transformers

使用变换器建模计数语言的栈表示的因果证据

Singh, Nishit

Abstract

Formal languages have proven to be effective conduits to understand the inner mechanisms of transformers. Past work has shown that transformers trained on next token prediction over counter languages learn representations consistent with an underlying stack structure. Beyond representational analysis, this paper investigates the causal role of these representations. Linear probes are trained to predict the stack depth at each token from the model's hidden states, and a principal representation direction is extracted from the probe. Ablation of this direction from the model causes sequential accuracy to collapse to near 0%, providing strong empirical evidence that the stack representation is not just learned, but is causally necessary for model performance.

Chinese Translation

形式语言已被证明是理解变换器内部机制的有效途径。过去的研究表明，基于计数语言进行下一个令牌预测训练的变换器学习到与潜在栈结构一致的表示。除了表示分析之外，本文还探讨了这些表示的因果作用。我们训练线性探测器以预测模型隐藏状态中每个令牌的栈深度，并从探测器中提取出一个主要表示方向。将该方向从模型中去除会导致顺序准确率崩溃至接近0%，提供了强有力的实证证据，表明栈表示不仅是学习到的，而且对模型性能是因果必要的。

View on arXiv Download PDF AI Translation

cs.CL / 60 / 2606.03399

Selective Token-Level Cryptographic Redaction for Privacy-Preserving Clinical Deployment of Large Language Models

用于隐私保护的临床大语言模型选择性令牌级加密编辑

Sheth, Farhan, Yang, Ziyuan, Lan, Yongying, Yeo, Si Yong

Abstract

While large language models (LLMs) are increasingly used for clinical applications, many existing pipelines require sending raw sensitive health information to remote servers for processing, which heightens the risk of privacy leakage. A natural approach to mitigate this risk is to encrypt the data before transmission. However, straightforward solutions such as encrypting the entire dataset introduce prohibitive computational, alignment, and communication overheads, rendering large-scale practical deployment infeasible. To preserve privacy while maintaining usability, we present Healthcare Encryption & Redaction via Adaptive Linguistic Decomposition (HERALD), a token-level cryptographic redaction framework designed to achieve this balance by encrypting only sensitive tokens while preserving the surrounding context for downstream model utility. HERALD combines medical named-entity recognizer (NER) with part-of-speech (POS) driven policies to select candidate tokens, performs targeted lemmatization to stabilize surface forms, and substitutes each protected token with a deterministic ciphertext wrapped in explicit delimiters. Notably, HERALD is model-agnostic and operates entirely on the client side, ensuring that sensitive content remains encrypted throughout storage, transmission, and processing without requiring changes to downstream models. We evaluated HERALD on both classification and medical question answering (MQA) tasks on public datasets. Across different tasks, experiments illustrate that fully secured baselines suffer significant utility loss, whereas HERALD consistently recovers performance close to plaintext. Overall, HERALD provides a novel utilization pipeline.

Chinese Translation

尽管大语言模型（LLMs）在临床应用中越来越多地被使用，但许多现有的处理流程仍需要将原始敏感健康信息发送到远程服务器进行处理，这增加了隐私泄露的风险。缓解这一风险的自然方法是对数据进行加密后再传输。然而，像对整个数据集进行加密这样的简单解决方案会引入巨大的计算、对齐和通信开销，使得大规模实际部署变得不可行。为了在保持可用性的同时保护隐私，我们提出了通过自适应语言分解实现的医疗加密与编辑（Healthcare Encryption & Redaction via Adaptive Linguistic Decomposition, HERALD），这是一个令牌级加密编辑框架，旨在通过仅加密敏感令牌并保留周围上下文来实现这一平衡，以便于下游模型的实用性。HERALD结合了医学命名实体识别（NER）与基于词性（POS）的策略来选择候选令牌，进行有针对性的词形还原以稳定表面形式，并用明确的分隔符将每个受保护的令牌替换为确定性的密文。值得注意的是，HERALD是模型无关的，并完全在客户端运行，确保敏感内容在存储、传输和处理过程中始终保持加密，而无需对下游模型进行更改。我们在公共数据集上对HERALD进行了分类和医学问答（MQA）任务的评估。在不同任务中，实验表明完全安全的基线会遭受显著的实用性损失，而HERALD则始终恢复接近明文的性能。总体而言，HERALD提供了一种新颖的利用流程。

View on arXiv Download PDF AI Translation

cs.CL / 61 / 2606.03412

Lexicons and grammars for language processing: industrial or handcrafted products?

语言处理的词典与语法：工业产品还是手工制作？

Laporte, Eric

Abstract

During the recent years, the use of linguistic data for language processing increased progressively. Such data are now commonly called language resources. Most of the language resources used for this purpose are collections of texts as the Brown Corpus and the Penn Treebank, but electronic lexicons (WordNet, FrameNet, VerbNet, ComLex, Lexicon-Grammar...) and formal grammars (TAG...) developed recently. Most processes of construction of lexicons and grammars are manual, whereas the construction of corpora has always been highly automated. However, more and more specialists of language processing realize that the information content of lexicons and grammars is richer than that of corpora, and hence the former make more elaborate processing possible. The difference in construction time is likely to be connected with the difference in information content: the handcrafting of lexicons and grammars by linguists would make them more informative than automatically generated data. This situation can evolve into two directions: either specialists of language technology get progressively used to handling manually constructed resources, which are more informative and more complex, or the process of construction of lexicons and grammars is automated and industrialized, which is the mainstream perspective. Both evolutions are already in progress, and a tension exists between them. The relation between linguists and computer scientists depends on the future of these evolutions, since the first implies training and hiring numerous linguists, whereas the other depends essentially on solutions elaborated by computer engineers. The aim of this article is to analyse practical examples of the language resources in question, and to discuss about which of the two trends, handcrafting or generating industrially, or a combination of both, can give the best results or is the most realistic.

Chinese Translation

近年来，语言处理中对语言数据的使用逐渐增加。这类数据通常被称为语言资源。用于此目的的大多数语言资源是文本集合，如布朗语料库（Brown Corpus）和宾州树库（Penn Treebank），但近年来也开发了电子词典（如WordNet、FrameNet、VerbNet、ComLex、Lexicon-Grammar等）和形式语法（如TAG等）。词典和语法的构建过程大多是手动的，而语料库的构建则一直高度自动化。然而，越来越多的语言处理专家意识到，词典和语法的信息内容比语料库更为丰富，因此前者使得更复杂的处理成为可能。构建时间的差异可能与信息内容的差异有关：语言学家手工制作的词典和语法会比自动生成的数据更具信息量。这种情况可能朝两个方向发展：要么语言技术专家逐渐习惯于处理更具信息量和复杂性的手工构建资源，要么词典和语法的构建过程实现自动化和工业化，这是主流观点。这两种演变已经在进行中，并且它们之间存在张力。语言学家与计算机科学家之间的关系取决于这些演变的未来，因为前者意味着需要培训和雇佣大量语言学家，而后者则主要依赖计算机工程师提出的解决方案。本文旨在分析相关语言资源的实际例子，并讨论手工制作、工业生成或两者结合哪种趋势能够提供最佳结果或更为现实。

View on arXiv Download PDF AI Translation

cs.CL / 62 / 2606.03437

Large Language Models Are Overconfident in Their Own Responses

大型语言模型对自身回答的过度自信

Sanz-Guerrero, Mario, Mager, Manuel, von der Wense, Katharina

Abstract

Prior work has shown that instruction-tuned large language models (LLMs) are less well calibrated than their base pre-trained counterparts. However, little is known about the frequently used chat template's effect on the calibration of conversational LLMs. In this work, we investigate the mechanisms driving this miscalibration by decoupling the effects of the post-training algorithm and the chat format. We find that, while instruction tuning fundamentally harms calibration, the chat template aggravates the issue through an "ownership bias" -- models are significantly more confident in their own answers than in identical answers provided by a user. Extensive experiments across six recent open-weight LLMs, three benchmarks, and three confidence elicitation methods show that models assign up to 26% higher confidence to their own responses. Leveraging this insight, we propose a simple inference-time strategy: framing the model's answer as user input during confidence elicitation. This approach significantly reduces overconfidence and improves calibration by up to 26% without the need for retraining, narrowing the gap between base and instruction-tuned models.

Chinese Translation

先前的研究表明，经过指令调优的大型语言模型（LLMs）在校准方面不如其基础预训练模型。然而，关于常用聊天模板对对话型LLMs校准影响的研究仍然较少。在本研究中，我们通过解耦后训练算法和聊天格式的影响，探讨导致这种错误校准的机制。我们的发现表明，尽管指令调优从根本上损害了校准，但聊天模板通过“所有权偏差”加剧了这一问题——模型对自身答案的信心显著高于用户提供的相同答案。在对六个近期开放权重的LLMs、三个基准测试和三种信心引导方法进行的广泛实验中，我们发现模型对自身回答的信心最高可提高26%。基于这一洞察，我们提出了一种简单的推理时策略：在信心引导过程中将模型的回答框架化为用户输入。这一方法显著降低了过度自信，并在不需要重新训练的情况下将校准提高了多达26%，缩小了基础模型与经过指令调优模型之间的差距。

View on arXiv Download PDF AI Translation

cs.CL / 63 / 2606.03504

BaltiVoice: A Speech Corpus and Fine-tuned Whisper ASR System for the Balti Language

BaltiVoice：巴尔蒂语言的语音语料库和微调的Whisper ASR系统

Ali, Muhammad

Abstract

We present BaltiVoice, a 16.8-hour read-speech corpus for Balti (ISO 639-3: bft), a Tibetic language spoken in Gilgit-Baltistan, Pakistan, with no prior publicly available ASR resources. The corpus contains 10,060 validated utterances in native Nastaliq script, derived from Mozilla Common Voice recordings. We fine-tune OpenAI Whisper-small on this corpus and report a Word Error Rate (WER) of 30.07% on a held-out validation set of 538 utterances, down from a measured zero-shot baseline of 182.18% for Whisper-small on Balti. The dataset, fine-tuned model, and a live transcription demo are publicly available on HuggingFace.

Chinese Translation

我们提出了BaltiVoice，这是一个包含16.8小时巴尔蒂语（ISO 639-3: bft）朗读语音的语料库，该语言在巴基斯坦吉尔吉特-巴尔蒂斯坦地区使用，之前没有公开可用的自动语音识别（ASR）资源。该语料库包含10,060个经过验证的以本土Nastaliq书写的语句，来源于Mozilla Common Voice的录音。我们在该语料库上对OpenAI的Whisper-small进行了微调，并在一个包含538个语句的保留验证集上报告了30.07%的词错误率（WER），相比之下，Whisper-small在巴尔蒂语上的零样本基线测得为182.18%。该数据集、微调模型以及实时转录演示已在HuggingFace上公开提供。

View on arXiv Download PDF AI Translation

cs.CL / 64 / 2606.03576

AutoTail-BSFGM: Class-Balance-Aware Fine-Tuning for Chinese Scholarly Text Classification

AutoTail-BSFGM：针对中文学术文本分类的类平衡感知微调

Xiang, Anling, Yang, Yuwen, Shen, Yang

Abstract

Scholarly text classification supports literature organization, subject indexing, and research intelligence, but Chinese scholarly corpora often contain imbalanced and semantically adjacent disciplinary labels. We propose AutoTail-BSFGM, a class-balance-aware fine-tuning method that combines an automatically gated tail-prior adjustment, a weak Balanced Softmax auxiliary loss, and Fast Gradient Method adversarial regularization. The method changes only the training objective and procedure; inference uses the same single base-size encoder and linear classifier as the corresponding label-smoothed baseline. We evaluate the method on two CSL-based tasks: an abstract-to-discipline task with 67 labels and a title-to-category task with 13 categories. On the primary abstract task, AutoTail-BSFGM improves validation and lockbox accuracy under both Chinese RoBERTa-WWM and MacBERT-base. With MacBERT-base, validation accuracy increases by 0.83 percentage points and lockbox accuracy by 0.49 points, with a pooled paired McNemar signal on validation (p = 0.023). On the title task, the method improves validation accuracy by 0.70 points and validation balanced accuracy by 2.64 points; lockbox accuracy is approximately neutral while lockbox balanced accuracy improves by 1.22 points. The results support a bounded contribution: AutoTail-BSFGM improves class-balance-sensitive behavior and yields consistent gains for abstract-based scholarly classification, without uniformly improving every metric on every split.

Chinese Translation

学术文本分类支持文献组织、主题索引和研究智能，但中文学术语料库通常包含不平衡和语义相近的学科标签。我们提出了AutoTail-BSFGM，这是一种类平衡感知的微调方法，结合了自动门控的尾部先验调整、弱平衡Softmax辅助损失和快速梯度法对抗正则化。该方法仅改变训练目标和过程；推理使用与相应标签平滑基线相同的单一基础大小编码器和线性分类器。我们在两个基于CSL的任务上评估该方法：一个包含67个标签的摘要到学科任务和一个包含13个类别的标题到类别任务。在主要的摘要任务中，AutoTail-BSFGM在中文RoBERTa-WWM和MacBERT-base下均提高了验证和锁箱准确率。在MacBERT-base下，验证准确率提高了0.83个百分点，锁箱准确率提高了0.49个百分点，验证上的配对McNemar信号为p = 0.023。在标题任务中，该方法将验证准确率提高了0.70点，验证平衡准确率提高了2.64点；锁箱准确率大致保持中性，而锁箱平衡准确率提高了1.22点。结果支持有限贡献：AutoTail-BSFGM改善了类平衡敏感行为，并为基于摘要的学术分类带来了持续的收益，而并非在每个拆分上均匀改善每个指标。

View on arXiv Download PDF AI Translation

cs.CL / 65 / 2606.03604

Beyond the Literal: Decomposing Pragmatic Intent in Multimodal Meme Understanding

超越字面意义：多模态表情包理解中的语用意图分解

Zhao, Zhengyi, Zhang, Shubo, Wang, Zezhong, Ye, Luyao, Wang, Huimin, Yan, Hanqi, Li, Binyang, Wong, Kam-Fai, He, Yulan

Abstract

When asked what a meme or sarcastic post means, Large Vision Language Models (LVLMs) tend to describe what the image shows rather than what the author is trying to communicate. Standard instruction tuning entangles a post's literal content with its pragmatic meaning, letting surface-level details contaminate the final response. We reframe meme understanding as a problem of literal-pragmatic decomposition and propose \textbf{Intent Projection}, a framework that separates the two signals at the representation, output, and objective levels within a single LVLM backbone. At the representation level, an orthogonal projection module removes dominant unimodal directions from the fused image-text representation, retaining only the pragmatic residual, while a surface-real affect classifier anchors the decoder with a discrete tag that names the polarity gap. At the output level, the model externalizes a structured reasoning chain, and at the objective level a contrastive reward explicitly penalizes answers that restate the literal description. Across six multimodal benchmarks, Intent Projection consistently outperforms open-source baselines and narrows the gap to proprietary models, with the largest gains on high-divergence posts where literal collapse is most damaging.

Chinese Translation

当被问及一个表情包或讽刺帖子意味着什么时，大型视觉语言模型（LVLMs）往往倾向于描述图像所展示的内容，而不是作者试图传达的意思。标准的指令调优将帖子的字面内容与其语用意义混合在一起，使得表面细节污染最终的响应。我们将表情包理解重新构建为字面-语用分解的问题，并提出了 extbf{意图投影}（Intent Projection）框架，该框架在单一的LVLM骨干网络内，在表示、输出和目标层面分离这两种信号。在表示层面，一个正交投影模块从融合的图像-文本表示中去除主导的单模态方向，仅保留语用残差，同时一个表面-真实情感分类器用一个离散标签锚定解码器，以命名极性差距。在输出层面，模型外化一个结构化的推理链，而在目标层面，一个对比奖励明确惩罚那些重复字面描述的答案。在六个多模态基准测试中，意图投影始终优于开源基线，并缩小了与专有模型之间的差距，在字面崩溃最具破坏性的高差异帖子上获得了最大的提升。

View on arXiv Download PDF AI Translation

cs.CL / 66 / 2606.03628

Building Reliable Long-Form Generation via Hallucination Rejection Sampling

通过幻觉拒绝采样构建可靠的长文本生成

Li, Lin, Channing, Georgia, Bhat, Suhaas M, Jones, Gabriel Davis, Gal, Yarin

Abstract

Large language models (LLMs) have achieved remarkable progress in open-ended text generation, yet they remain prone to hallucinating incorrect or unsupported content, which undermines their reliability. This issue is exacerbated in long-form generation due to hallucination snowballing, a phenomenon where early errors propagate and compound into subsequent outputs. To address this challenge, we propose a novel inference-time hallucination mitigation framework, named Segment-wise HAllucination Rejection Sampling (SHARS), which uses an arbitrary hallucination detector to identify and reject hallucinated segments during generation and resample until faithful content is produced. By retaining only confident information and building subsequent generations upon it, the framework mitigates hallucination accumulation and enhances factual consistency. To instantiate this framework, we adopt semantic uncertainty as the detector and introduce several vital modifications to address its limitations and better adapt it to long-form text. Our method enables models to self-correct hallucinations without requiring external resources such as web search or knowledge bases, while remaining compatible with them for future extensions. Empirical evaluations on standardized hallucination benchmarks demonstrate that our method substantially reduces hallucinations in long-form generation while preserving or even improving the informativeness of generation. Code is available at: https://github.com/TreeLLi/hallucination-rejection-sampling.

Chinese Translation

大型语言模型（LLMs）在开放式文本生成方面取得了显著进展，但它们仍然容易产生不正确或不支持的内容，这削弱了它们的可靠性。在长文本生成中，这一问题因幻觉雪球效应而加剧，即早期错误传播并累积到后续输出中。为了解决这一挑战，我们提出了一种新颖的推理时幻觉缓解框架，称为分段幻觉拒绝采样（Segment-wise HAllucination Rejection Sampling, SHARS），该框架使用任意的幻觉检测器在生成过程中识别并拒绝幻觉段落，并进行重采样，直到生成真实内容。通过仅保留可信的信息并在其基础上构建后续生成，该框架减轻了幻觉累积并增强了事实一致性。为了实现这一框架，我们采用语义不确定性作为检测器，并引入若干重要修改以解决其局限性，并更好地适应长文本。我们的方法使模型能够自我纠正幻觉，而无需外部资源，如网络搜索或知识库，同时仍然与这些资源兼容，以便未来扩展。在标准化幻觉基准上的实证评估表明，我们的方法显著减少了长文本生成中的幻觉，同时保持甚至提高了生成的丰富性。代码可在以下网址获取：https://github.com/TreeLLi/hallucination-rejection-sampling。

View on arXiv Download PDF AI Translation

cs.CL / 67 / 2606.03648

Safety Measurements for Fine-tuned LLMs Should be Grounded in Capability

细调大型语言模型的安全性测量应基于能力

Vishnubhotla, Krishnapriya, Dawkins, Hillary, Nejadgholi, Isar, Kiritchenko, Svetlana

Abstract

Adapting foundation large language models to a user's task or preferred style through fine-tuning can result in compromising the model's safety. Previous works examined the effects of fine-tuning on model safety in limited and seemingly random experimental settings. We argue that anchoring fine-tuning to a specific capability goal is essential for avoiding arbitrary empirical choices, allowing us to draw meaningful conclusions about safety impacts, and to compare mitigation methods on a consistent basis. We conduct a multi-dimensional evaluation of the effects of fine-tuning on model behavior by focusing on capability as well as safety. Our results surface important issues that (1) fine-tuned models can produce incoherent generations in response to safety prompts, (2) automated safety judgments are unreliable for such incoherent outputs, and (3) the conclusions about the effects of fine-tuning can change depending on the choice of safety benchmark as well as the safety evaluator.

Chinese Translation

通过细调将基础大型语言模型适应于用户的任务或偏好风格可能会导致模型安全性的妥协。先前的研究在有限且似乎随机的实验环境中考察了细调对模型安全性的影响。我们认为，将细调锚定在特定的能力目标上对于避免任意的经验选择至关重要，这使我们能够对安全影响得出有意义的结论，并在一致的基础上比较缓解方法。我们通过关注能力和安全性，对细调对模型行为的影响进行了多维评估。我们的结果揭示了重要问题：(1) 细调模型在响应安全提示时可能产生不连贯的生成结果，(2) 对于这种不连贯输出的自动安全判断是不可靠的，以及 (3) 关于细调影响的结论可能会因安全基准的选择和安全评估者的不同而有所变化。

View on arXiv Download PDF AI Translation

cs.CL / 68 / 2606.03650

CoEval: Ranking Language Models for Custom Tasks Without Labeled Data or Trustworthy Benchmarks

CoEval：在没有标注数据或可信基准的情况下为自定义任务排名语言模型

Apartsin, Alexander, Aperstein, Yehudit

Abstract

Choosing or ranking language models for a specific application is hardest when no task-specific labeled data exists, and standard public benchmarks cannot be trusted, their items having likely leaked into pretraining, so scores reflect memorization rather than fitness. We present CoEval, an open-source, reusable framework that closes this gap end to end: from only a description of a task or domain, teacher models synthesize a fresh, attribute-controlled benchmark with no human labels, contamination-free because items are generated anew on each run, and a cross-family judge ensemble ranks candidate models with no human raters. Validated where ground truth exists, CoEval recovers the true model ranking and tracks ground-truth correctness at ho=0.86. The label-free judging needs no human calibration because judge-panel composition (vendor diversity), not size, drives reliability: a small, well-chosen cross-family panel is most reliable, while a single judge can be anti-correlated with ground truth (judge-choice regret 0.35) and the ensemble never is. Generated items show zero verbatim 13-gram overlap with five major public benchmarks; the panel cancels verbosity bias and precludes same-family self-preference. A four-task study produced 7,978 evaluations for USD 5.89. The same declarative pipeline applies to any domain and is cheap enough to re-run on every model release: a label-free, contamination-free leaderboard any team can regenerate for its own application.

Chinese Translation

在没有特定任务的标注数据且标准公共基准无法信任的情况下，为特定应用选择或排名语言模型是最困难的，因为这些基准的项目很可能在预训练中泄漏，因此得分反映的是记忆而非适应性。我们提出了CoEval，一个开源、可重用的框架，旨在从头到尾弥补这一空白：仅通过任务或领域的描述，教师模型合成一个全新、属性可控的基准，无需人工标签，且由于每次运行时生成的新项目而不受污染；跨家族的评审团队对候选模型进行排名，无需人工评审。经过真实情况验证，CoEval能够恢复真实的模型排名，并在ho=0.86时跟踪真实正确性。无标签评审无需人工校准，因为评审小组的组成（供应商多样性）而非规模驱动可靠性：一个小而精心挑选的跨家族小组是最可靠的，而单一评审可能与真实情况呈负相关（评审选择后悔值为0.35），而评审小组则不会。生成的项目与五个主要公共基准在13-gram上没有逐字重叠；评审小组消除了冗长偏见，并避免了同家族的自我偏好。一项四任务研究产生了7,978个评估，花费为5.89美元。相同的声明性流程适用于任何领域，且成本足够低，可以在每次模型发布时重新运行：一个无标签、无污染的排行榜，任何团队都可以为其自身应用重新生成。

View on arXiv Download PDF AI Translation

cs.CL / 69 / 2606.03693

Does Language Shift Break Medical Vision-Language Models? Indonesian Radiology Visual Question Answering Case Study

语言转变是否会影响医学视觉语言模型？印尼放射学视觉问答案例研究

Yudhistira, Pieter Christy Yan, Malik, Dzaki Rafif, Yudistira, Novanto

Abstract

Medical Vision-Language Models (VLMs) are typically evaluated on English radiology visual question answering benchmarks, leaving their robustness under non-English clinical language largely unexplored. We introduce IndoRad-VQA, an Indonesian adaptation of VQA-RAD, to assess whether medical VLMs retain radiology reasoning ability when questions are asked in Bahasa Indonesia. Radiology question-answer pairs are translated into Indonesian with self-evaluation-based quality control to preserve clinical meaning, terminology consistency, and answer equivalence. We evaluate general-purpose, Southeast Asian multilingual, and medical-specific VLMs under English and Indonesian prompting settings. Beyond accuracy, we quantify the language robustness gap between English and Indonesian inputs. We also conduct an error analysis to identify failure modes of question answering, such as yes/no flips, laterality errors, and output-language mismatches. Our findings show that strong performance on English medical VQA benchmarks does not necessarily translate to robust behavior in Indonesian clinical contexts. We observe a performance gap of 8 to 25 percent between the English and Indonesian settings, depending on the evaluation metric. These results highlight the need for more inclusive multilingual evaluation of medical multimodal foundation models. The dataset is available at https://huggingface.co/datasets/Lab-IS/IndoRad-VQA.

Chinese Translation

医学视觉语言模型（VLMs）通常在英语放射学视觉问答基准上进行评估，因此它们在非英语临床语言下的稳健性尚未得到充分探讨。我们引入了IndoRad-VQA，这是VQA-RAD的印尼适配版本，以评估医学VLMs在以印尼语（Bahasa Indonesia）提问时是否保持放射学推理能力。放射学问答对通过自我评估的质量控制翻译成印尼语，以保留临床意义、术语一致性和答案等价性。我们在英语和印尼语提示设置下评估通用、多语言的东南亚和医学特定的VLMs。除了准确性之外，我们量化了英语和印尼语输入之间的语言稳健性差距。我们还进行了错误分析，以识别问答的失败模式，例如是/否翻转、侧别错误和输出语言不匹配。我们的研究结果表明，在英语医学VQA基准上表现良好的模型并不一定在印尼临床环境中表现稳健。根据评估指标，我们观察到英语和印尼语设置之间的性能差距为8%到25%。这些结果强调了对医学多模态基础模型进行更具包容性的多语言评估的必要性。数据集可在 https://huggingface.co/datasets/Lab-IS/IndoRad-VQA 获取。

View on arXiv Download PDF AI Translation

cs.CL / 70 / 2606.03695

Don't Forget Your Embeddings: Robust Knowledge Erasure via Precise Editing of Embeddings

不要忘记你的嵌入：通过精确编辑嵌入实现稳健的知识抹除

Suslik, Clara Haya, Shafran, Or, Geva, Mor

Abstract

As language models are increasingly deployed in real-world applications, the ability to erase specific knowledge from them becomes critical for safety and compliance. Prominent methods seek persistent removal by updating the model's parameters, yet the target knowledge often can be recovered through adversarial prompting or relearning. In this work, we hypothesize this limitation stems in part from existing methods overlooking the embedding layer. To address this, we introduce EMBedding ERasure (EMBER), a plug-n-play erasure module that leverages Sparse Matrix Factorization for precise erasure of concept-related features from token embeddings. Through comprehensive evaluations across diverse concepts on Gemma-2-2B-it and Llama-3.1-8B-Instruct, we find that augmenting existing methods with EMBER consistently improves erasure efficacy and specificity across task formats, with minimal coherence loss. Moreover, it dramatically improves robustness to relearning, reducing regained accuracy by up to 50%, limiting it to 35% on Llama compared to 70%-76% for prior methods. Further analysis shows that the coherence cost is localized, affecting only a small set of concept-exclusive tokens. Our work establishes that precise embedding-level intervention is necessary for robust concept erasure, and demonstrates that existing methods can benefit from such augmentation.

Chinese Translation

随着语言模型在现实应用中的日益普及，从中抹除特定知识的能力变得对安全性和合规性至关重要。现有的主要方法通过更新模型参数来实现持久性移除，但目标知识往往可以通过对抗性提示或重新学习来恢复。在本研究中，我们假设这一限制部分源于现有方法忽视了嵌入层。为了解决这个问题，我们引入了嵌入抹除模块（EMBedding ERasure，简称EMBER），这是一个即插即用的抹除模块，利用稀疏矩阵分解精确抹除与概念相关的特征。通过在Gemma-2-2B-it和Llama-3.1-8B-Instruct上对多样概念进行全面评估，我们发现将EMBER与现有方法结合使用，能够持续提高抹除的有效性和特异性，且对任务格式的影响最小。此外，它显著提高了对重新学习的稳健性，将恢复的准确率降低了多达50%，在Llama上限制在35%，而之前的方法则为70%-76%。进一步分析表明，连贯性损失是局部化的，仅影响一小部分概念专属的标记。我们的研究确立了精确的嵌入级干预对于稳健的概念抹除是必要的，并展示了现有方法可以从这种增强中受益。

View on arXiv Download PDF AI Translation

cs.CL / 71 / 2606.03728

Re-Ranking Through an Attribution Lens for Citation Quality in Legal QA

通过归因视角进行再排序以提高法律问答中的引用质量

Elganayni, Mohamed Hesham, Saleh, Selim

Abstract

Retrieval-augmented generation systems for legal question answering typically retrieve passages based on semantic similarity and provide them to a language model, which then generates cited answers. Prior work assumes that highly ranked passages are most likely to be usefully cited by the model. Perturbation-based attribution methods, such as C-LIME, have been used exclusively for post-hoc explanation. However, on the AQuAECHR benchmark, semantic similarity does not correlate with passage attribution. Within a retriever's candidate pool, similarity-based ranking performs worse than random selection at surfacing gold citation paragraphs. To address this limitation, a lightweight cross-encoder is trained on continuous perturbation-based attribution scores to re-rank passages prior to generation. This approach is evaluated on the AQuAECHR benchmark, using two language models and five-fold cross-validation. The re-ranker substantially improves citation faithfulness and alignment with gold expert answers. Notably, two re-rankers trained independently on different models converge beyond their raw attribution agreement. This finding indicates that the cross-encoder reduces model-specific noise and produces a shared relevance signal that partially transfers across models, although same-model re-ranking remains more effective. These results demonstrate that perturbation-based attribution provides a practical, model-agnostic training signal for citation-aware retrieval.

Chinese Translation

用于法律问答的检索增强生成系统通常基于语义相似性检索段落，并将其提供给语言模型，后者生成引用答案。先前的研究假设高排名的段落最有可能被模型有效引用。然而，在AQuAECHR基准测试中，语义相似性与段落归因并无相关性。在检索器的候选池中，基于相似性的排名在揭示黄金引用段落方面的表现不如随机选择。为了解决这一局限性，训练了一个轻量级的交叉编码器，基于连续的扰动归因分数对段落进行再排序，以便在生成之前进行处理。该方法在AQuAECHR基准上进行了评估，使用了两个语言模型和五折交叉验证。再排序器显著提高了引用的可信度和与黄金专家答案的一致性。值得注意的是，两个在不同模型上独立训练的再排序器在其原始归因一致性之外趋于收敛。这个发现表明，交叉编码器减少了模型特定的噪声，并产生了一个共享的相关性信号，该信号在模型之间部分转移，尽管同模型再排序仍然更有效。这些结果表明，基于扰动的归因为引用感知检索提供了一种实用的、模型无关的训练信号。

View on arXiv Download PDF AI Translation

cs.CL / 72 / 2606.03739

Entropy Gate: Entropy Quenching for Near-Lossless Token Compression in LLM Pipelines

熵门：在大型语言模型管道中实现近无损的令牌压缩的熵淬火

Agyemang, Justice Owusu, Kponyo, Jerry John, Agyekum, Kwame Opuni-Boachie Obour, Acheampong, Francisca Adoma, Agyekum, Kwame Agyeman-Prempeh, Gadze, James Dzisi

Abstract

LLM pipelines waste substantial token budgets on low-information content: repeated context, verbose responses, and redundant boilerplate. We introduce Entropy Gate, a token compression framework applying entropy quenching $-$ a thermodynamic process that progressively freezes out low-energy tokens while preserving semantic fidelity. Each token receives a multi-factor information energy $E(t)$ combining statistical, structural, and positional components. An adaptive quenching schedule $T(\tau) = T_0 / (1 + \alpha \tau)$ removes tokens whose Boltzmann survival probability $p_i = \exp(-E_i / kT)$ falls below threshold, with a fidelity gate halting compression when energy-weighted similarity drops below $\theta$. We prove token selection by descending $E(t)$ maximizes expected semantic preservation, that quenching produces nested survival sets, and that achievable compression approaches the information-theoretic limit $\text{CR} \to 1 - I(P; T)/H(P)$. A Phase 1 heuristic achieves 40-60% compression across five prompt categories while maintaining $S_E > 0.80$, with energy-squared amplification $E \to E^2$ adding 10-25 percentage points. Context deduplication adds 50-70% savings on repeated blocks. Output-side quenching, motivated by findings that brevity improves accuracy, further reduces response overhead. Combined with external memory, reduction composes multiplicatively to 88-96% for agentic workloads. The framework is stateless, model-agnostic, and deploys as an OpenAI-compatible HTTP proxy.

Chinese Translation

大型语言模型管道在低信息内容上浪费了大量令牌预算：重复的上下文、冗长的响应和多余的模板。我们提出了熵门（Entropy Gate），这是一个令牌压缩框架，应用熵淬火（entropy quenching）——一种热力学过程，逐步冻结低能量令牌，同时保持语义的保真性。每个令牌接收一个多因素信息能量 $E(t)$，结合了统计、结构和位置成分。自适应淬火计划 $T( au) = T_0 / (1 + eta au)$ 移除那些玻尔兹曼生存概率 $p_i = ext{exp}(-E_i / kT)$ 低于阈值的令牌，同时在能量加权相似度低于 $ heta$ 时停止压缩。我们证明了通过降序选择 $E(t)$ 的令牌可以最大化预期的语义保留，淬火产生嵌套的生存集合，并且可实现的压缩接近信息论极限 $ ext{CR} o 1 - I(P; T)/H(P)$。第一阶段启发式方法在五个提示类别中实现了40-60%的压缩，同时保持 $S_E > 0.80$，能量平方放大 $E o E^2$ 增加了10-25个百分点。上下文去重在重复块上增加了50-70%的节省。基于发现简洁性提高准确性，输出侧淬火进一步减少了响应开销。结合外部内存，减少效果在代理工作负载中呈乘法组合，达到88-96%。该框架是无状态的、模型无关的，并作为兼容OpenAI的HTTP代理进行部署。

View on arXiv Download PDF AI Translation

cs.CL / 73 / 2606.03761

Framing Migration News with LLMs: Structured CoT as a Support for Human Interpretation

用大语言模型框架移民新闻：结构化链式思维作为人类解读的支持

del Barrio, David Alonso, Wen, Jing, Gatica-Perez, Daniel

Abstract

Frame analysis of migration news is a socially consequential task: media scholars and researchers who study how migration is narrated need tools that are not only accurate, but transparent, auditable, and accessible within the resource constraints typical of academic research groups. Existing LLM-based approaches rely on proprietary APIs and large models that raise concerns about data privacy, reproducibility and equitable access among media researchers. This work studies how a locally deployable open-source LLM can support interpretable frame analysis as an assistive tool. We introduce a Structured Chain-of-Thought (SCoT) prompting approach using Llama3-8B, enabling step-by-step justifications grounded in predefined framing categories. This structured design allows users to audit model outputs and examine alternative interpretations in a task that is inherently subjective. We evaluate our approach on a dataset of migration-related news and show that SCoT improves classification performance over zero-shot and few-shot baselines while remaining feasible on a single GPU. Then, we conduct a human-centered evaluation in which annotators assess the coherence and influence of "the model's reasoning". Results indicate that SCoT explanations are generally perceived as logical (mean score 4.1/5, though with notable variation across texts) and can prompt reflection on initial interpretations, even when disagreement persists. Our findings highlight both the potential and risks of LLM-assisted frame analysis. While structured reasoning can increase the traceability of model outputs and support critical interpretation, it can also influence human judgment in subtle ways. By enabling local deployment and emphasizing human-in-the-loop interaction, this work contributes to discussions on responsible and accessible computational tools for the study of socially impactful media narratives.

Chinese Translation

对移民新闻的框架分析是一项具有社会重要性的任务：媒体学者和研究人员需要不仅准确，而且透明、可审计且在学术研究小组典型的资源限制下可获取的工具，以研究移民叙事。现有基于大语言模型（LLM）的方法依赖于专有API和大型模型，这引发了关于数据隐私、可重复性和媒体研究人员之间公平获取的担忧。本研究探讨了如何通过本地可部署的开源LLM支持可解释的框架分析作为辅助工具。我们介绍了一种使用Llama3-8B的结构化链式思维（Structured Chain-of-Thought, SCoT）提示方法，使得基于预定义框架类别的逐步论证成为可能。这种结构化设计允许用户审计模型输出并在本质上主观的任务中检查替代解释。我们在一个与移民相关的新闻数据集上评估了我们的方法，结果表明SCoT在分类性能上优于零-shot和少-shot基线，同时在单个GPU上仍然可行。随后，我们进行了一项以人为中心的评估，评估者评估“模型推理”的连贯性和影响力。结果表明，SCoT的解释通常被认为是合乎逻辑的（平均得分4.1/5，尽管在文本之间存在显著差异），并且能够促使对初始解释的反思，即使在存在分歧的情况下。我们的发现突显了LLM辅助框架分析的潜力与风险。尽管结构化推理可以增加模型输出的可追溯性并支持批判性解读，但它也可能以微妙的方式影响人类判断。通过实现本地部署并强调人机交互，本研究为讨论负责任和可获取的计算工具在研究社会影响媒体叙事中的应用做出了贡献。

View on arXiv Download PDF AI Translation

cs.CL / 74 / 2606.03768

HybridThinker: Efficient Chain-of-Thought Reasoning via Compressed Memory and Transient Thought Steps

HybridThinker：通过压缩记忆和瞬态思维步骤实现高效的思维链推理

Liu, Xin, Zhao, Runsong, Liu, Xinyu, Ruan, Junhao, Huang, Pengcheng, Dong, Shichao, Xiao, Chunyang, Wang, Chenglong, Li, Changliang, Zhu, Jingbo, Xiao, Tong

Abstract

Extended chain-of-thought (CoT) traces improve LLM reasoning but incur substantial computational and memory costs. While existing CoT compression methods mitigate this by condensing thought steps into compact representations via memory tokens and retaining only these representations at inference time, the loss of fine-grained information makes subsequent steps more error-prone. To alleviate this, we propose \textbf{HybridThinker}, where in addition to preserved these representations, thought steps are also temporarily retained to provide fine-grained details. However, we observe that naively keeping thought steps accessible to subsequent steps \emph{during training} lets the model bypass memory tokens by retrieving information directly from these steps, leaving the model's ability to compress and retrieve information through memory tokens insufficiently trained. We therefore introduce a hybrid training scheme, in which only some thought steps are directly accessible through attention to subsequent steps, while the other thought steps are masked, forcing the model to use memory tokens for compression and retrieval. Across 4 reasoning benchmarks, HybridThinker matches the uncompressed baseline, advancing the state of the art in CoT compression by 5.8 points on average accuracy with similar inference time. Ablation studies confirm that both temporary thought-step retention and the hybrid training scheme contribute to these gains.

Chinese Translation

扩展的思维链（CoT）轨迹改善了大型语言模型（LLM）的推理能力，但带来了可观的计算和内存成本。现有的CoT压缩方法通过使用记忆标记将思维步骤浓缩为紧凑的表示，并在推理时仅保留这些表示，从而缓解了这一问题。然而，细粒度信息的丢失使得后续步骤更容易出错。为了解决这个问题，我们提出了 extbf{HybridThinker}，在保留这些表示的基础上，思维步骤也被暂时保留，以提供细粒度的细节。然而，我们观察到，在训练过程中，简单地让后续步骤可以直接访问思维步骤，会使模型绕过记忆标记，直接从这些步骤中检索信息，从而导致模型在通过记忆标记进行信息压缩和检索的能力不足。因此，我们引入了一种混合训练方案，其中只有部分思维步骤可以通过注意力直接访问后续步骤，而其他思维步骤则被屏蔽，迫使模型使用记忆标记进行压缩和检索。在四个推理基准测试中，HybridThinker的表现与未压缩基线相当，在平均准确率上将CoT压缩的最新进展提高了5.8个百分点，同时推理时间相似。消融研究证实，暂时保留思维步骤和混合训练方案均对这些提升有所贡献。

View on arXiv Download PDF AI Translation

cs.CL / 75 / 2606.03773

KletterMix: Climbing Toward High-Quality German Pretraining Data

KletterMix：迈向高质量德语预训练数据

Kraus, Maurice, Härle, Ruben, Sztwiertnia, Sebastian, Khan, Abbas Goher, Ali, Mehdi, Fromm, Michael, Kersting, Kristian

Abstract

High-quality pretraining data is a central ingredient in modern language models, but German-language resources remain far less developed than their English counterparts: they are often smaller, less carefully curated, weakly documented, and rarely validated through controlled training experiments. We introduce KletterMix, a high-quality German corpus for language model pretraining and annealing, designed as a reusable dataset artifact for the natural language processing and modeling community. KletterMix is built by translating a state-of-the-art English pretraining corpus into German while preserving document boundaries, metadata, source structure, and topical diversity. This construction yields a German corpus with the scale and diversity of a modern pretraining dataset, while enabling direct comparison to its English source. We document the dataset through a broad set of corpus-level analyses, including translation quality, document length distributions, topic coverage, source composition, and geographic metadata. Using COMETKiwi, we show that the translated documents achieve strong quality across diverse domains, suggesting that careful translation can preserve much of the semantic and stylistic richness of the original corpus. Beyond dataset construction, we evaluate KletterMix as training data. Through controlled pretraining and annealing ablations against established German corpora, we show that models trained on KletterMix achieve measurable improvements on German-language downstream evaluations. These results demonstrate that carefully curated translated data can substantially strengthen the German pretraining data ecosystem.

Chinese Translation

高质量的预训练数据是现代语言模型的核心组成部分，但德语资源的发展远远落后于英语资源：它们通常规模较小、整理不够精细、文档记录薄弱，且很少通过受控训练实验进行验证。我们介绍了KletterMix，一个用于语言模型预训练和退火的高质量德语语料库，旨在为自然语言处理和建模社区提供可重用的数据集工件。KletterMix通过将最先进的英语预训练语料库翻译成德语构建，同时保留文档边界、元数据、源结构和主题多样性。这一构建方式产生了一个具有现代预训练数据集规模和多样性的德语语料库，同时能够与其英语源进行直接比较。我们通过广泛的语料库级分析记录了该数据集，包括翻译质量、文档长度分布、主题覆盖、源组成和地理元数据。使用COMETKiwi，我们展示了翻译后的文档在不同领域中实现了强大的质量，表明仔细的翻译可以保留原始语料库的大部分语义和风格丰富性。除了数据集构建，我们还评估了KletterMix作为训练数据的效果。通过与已建立的德语语料库进行受控的预训练和退火消融实验，我们展示了在KletterMix上训练的模型在德语下游评估中取得了可衡量的改进。这些结果表明，经过精心整理的翻译数据可以显著增强德语预训练数据生态系统。

View on arXiv Download PDF AI Translation

cs.CL / 76 / 2606.03780

Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models

专家感知的稀疏混合专家语言模型中事实回忆的因果追踪

Lu, Yuetian, Modarressi, Ali, Liu, Yihong, Schütze, Hinrich

Abstract

Causal tracing of factual recall has been studied predominantly in dense transformer language models, where interventions localize information flow to layers or feed-forward modules. Sparse mixture-of-experts (MoE) language models introduce a sharper question: when a factual prediction is mediated by a routed MoE block, which routed expert contributions matter? We formulate expert-aware causal tracing for sparse MoE language models. Using CounterFact facts, we first corrupt the model's factual preference by adding noise to subject-token embeddings, and then test whether clean MoE-block outputs or clean expert-level updates restore the true-vs-foil logit contrast. For Qwen3-30B-A3B-Base, a layer sweep selects and validates layer 44, and expert-level tracing identifies L44E069 as an expert repeatedly selected in the clean run whose held-out patch outperforms other active same-layer expert patches. For Mixtral-8x7B-v0.1, layer-level tracing validates a mid-layer signal, but the signal is not localized to the selected singleton expert; a coalition check instead recovers it with routed multi-expert updates. These results suggest that MoE factual tracing can be made expert-aware, while also showing that expert-level localization is model- and protocol-dependent rather than universal.

Chinese Translation

事实回忆的因果追踪主要在密集的变换器语言模型中进行研究，其中干预将信息流局限于层或前馈模块。稀疏混合专家（MoE）语言模型提出了一个更尖锐的问题：当一个事实预测通过路由的 MoE 块进行调解时，哪些路由专家的贡献是重要的？我们为稀疏 MoE 语言模型制定了专家感知的因果追踪。使用 CounterFact 事实，我们首先通过向主题标记嵌入添加噪声来破坏模型的事实偏好，然后测试干净的 MoE 块输出或干净的专家级更新是否恢复真实与虚假对比的对数几率。对于 Qwen3-30B-A3B-Base，通过层遍历选择并验证了第 44 层，而专家级追踪确定 L44E069 是在干净运行中反复被选择的专家，其保留的补丁优于其他活跃的同层专家补丁。对于 Mixtral-8x7B-v0.1，层级追踪验证了中层信号，但该信号并未局限于所选的单一专家；相反，联盟检查通过路由的多专家更新恢复了该信号。这些结果表明，MoE 事实追踪可以实现专家感知，同时也显示专家级定位是依赖于模型和协议的，而非普遍适用。

View on arXiv Download PDF AI Translation

cs.CL / 77 / 2606.03782

Reasoning over Grammar: Can Synthetic Linguistic Reasoning Traces Enhance Low-Resource Machine Translation?

基于语法推理：合成语言推理轨迹能否增强低资源机器翻译？

Pei, Renhao, Liu, Yihong, Pyysalo, Sampo, Schütze, Hinrich, Ji, Shaoxiong

Abstract

Large language models (LLMs) offer a promising approach to machine translation (MT) for extremely low-resource languages by incorporating linguistic resources through in-context learning. However, LLMs often struggle to apply grammatical information effectively during translation. Inspired by recent progress in chain-of-thought reasoning, we investigate whether low-resource MT can benefit from structured intermediate steps of linguistic analysis and grammatical reasoning. We propose a pipeline for automatically generating step-by-step linguistic reasoning traces from Universal Dependencies treebanks, dictionaries, and grammar-rule banks. We evaluate these traces in three settings: in-context learning (ICL), supervised fine-tuning (SFT), and reinforcement fine-tuning (RFT), on Xibe and Chintang as test cases. Our results show that linguistic reasoning traces are most effective as inference-time guidance: in ICL, reliable sentence-specific traces substantially improve translation performance across most models, languages, and metrics. In contrast, using the linguistic reasoning traces as training data yields smaller and less consistent gains, as models learn the trace format but often generate erroneous content. These findings suggest that LLMs can leverage grammatical information for low-resource MT when given reliable linguistic analyses, while learning to generate such analyses remains a major bottleneck.

Chinese Translation

大型语言模型（LLMs）通过在上下文学习中融入语言资源，为极低资源语言的机器翻译（MT）提供了一种有前景的方法。然而，LLMs在翻译过程中往往难以有效应用语法信息。受到链式思维推理最新进展的启发，我们研究了低资源机器翻译是否可以从结构化的语言分析和语法推理的中间步骤中受益。我们提出了一种从通用依赖树库、词典和语法规则库中自动生成逐步语言推理轨迹的流程。我们在三个设置中评估这些轨迹：上下文学习（ICL）、监督微调（SFT）和强化微调（RFT），以锡伯（Xibe）和钦唐（Chintang）作为测试案例。我们的结果表明，语言推理轨迹在推理时作为指导最为有效：在ICL中，可靠的句子特定轨迹显著提高了大多数模型、语言和指标的翻译性能。相比之下，将语言推理轨迹作为训练数据使用所带来的增益较小且不一致，因为模型学习了轨迹格式但常常生成错误内容。这些发现表明，当提供可靠的语言分析时，LLMs可以利用语法信息进行低资源机器翻译，而生成这种分析的能力仍然是一个主要瓶颈。

View on arXiv Download PDF AI Translation

cs.CL / 78 / 2606.03785

Backdoor Unlearning Generalization: A Path Toward the Removal of Unknown Triggers in LLMs

后门遗忘泛化：去除大型语言模型中未知触发器的路径

Bouger, Lisa, Lasnier, Théo, Moundi, Philippe Looubet, Teglia, Yannick, Seddah, Djamé

Abstract

Backdoor attacks in Large Language Models (LLMs) are a growing security concern, where models can generate adversary-chosen content. Existing defenses target backdoors one at a time and typically require knowledge of the trigger, leaving the defender at a structural disadvantage when unknown backdoors may exist in a model. We show that backdoor neutralization through unlearning generalizes across backdoors: training a model to ignore a single trigger can also suppress other backdoors that were never explicitly targeted. We study this phenomenon across three model families, whose backdoors were injected via pretraining or continual pretraining, by analyzing the models obtained after removing one backdoor at a time. To understand why unlearning certain backdoors induces the suppression of others, we introduce the Cross Activation Shift Distance, to quantify the distance between model changes induced by different trainings. Our results open a new direction for LLM safety as defenders could deliberately inject controlled backdoors and then remove them, leveraging cross-backdoor transfer to also suppress unknown backdoors that an attacker may have previously introduced in the model.

Chinese Translation

大型语言模型（LLMs）中的后门攻击正日益成为安全隐患，这类模型能够生成对手选择的内容。现有的防御措施通常针对单个后门，且通常需要了解触发器，这使得在模型中可能存在未知后门时，防御者处于结构劣势。我们展示了通过遗忘实现后门中和的泛化特性：训练一个模型忽略单个触发器也可以抑制其他从未明确针对的后门。我们通过分析在逐一去除后门后获得的模型，研究了这一现象在三种模型家族中的表现，这些后门是通过预训练或持续预训练注入的。为了理解为何遗忘某些后门会导致其他后门的抑制，我们引入了交叉激活偏移距离（Cross Activation Shift Distance），以量化不同训练所引起的模型变化之间的距离。我们的结果为大型语言模型的安全性开辟了新的方向，防御者可以故意注入受控后门，然后将其去除，利用跨后门转移来抑制攻击者可能在模型中先前引入的未知后门。

View on arXiv Download PDF AI Translation

cs.CL / 79 / 2606.03793

Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models

探索多语言多模态大语言模型中的对抗鲁棒性与安全对齐

Malik, Hashmat Shadab, Naseer, Muzammal, Khan, Salman

Abstract

Multimodal Large Language Models integrate visual perception into language reasoning, introducing a continuous attack surface susceptible to adversarial attacks. Prior work on MLLM robustness has focused largely on English-centric tasks, leaving multilingual behaviour unexplored. We address this gap through a systematic study of adversarial robustness and multimodal safety across 12 diverse languages, evaluating open-source MLLMs that acquire multilingual capability through instruction tuning. Gradient-based attacks reveal a transferable multilingual vulnerability: adversarial images optimized in one language continue to induce failure in others, demonstrating strong cross-lingual transferability. Multilingual safety further varies with how effectively a model retrieves or interprets harmful instructions. When harmful intent is issued through text, languages with stronger linguistic grounding more often elicit misuse-enabling responses, while weaker languages produce fewer unsafe outputs. When embedded in the image as typographic content, English scripts are reliably recognised and followed, whereas non-English scripts are rarely parsed by the vision encoder. Lower-resource languages may therefore appear safer, but this is an artefact of comprehension and visual-grounding failures rather than genuine alignment, a phenomenon we term safety-by-failure. In contrast, MLLMs that build multilingual capability throughout their training stages rather than only at instruction tuning, such as Qwen3-VL, exhibit genuine cross-lingual safety, maintaining active refusal across languages rather than masking comprehension failure. Shallow multilingual adaptation, such as fine-tuning on translated instruction data, may produce surface-level understanding that creates illusory safety in low-resource languages; deeper integration across training stages leads to genuine multilingual safety alignment.

Chinese Translation

多模态大语言模型将视觉感知融入语言推理中，导致其面临持续的攻击面，易受对抗性攻击的影响。以往关于多模态大语言模型（MLLM）鲁棒性的研究主要集中在以英语为中心的任务上，未对多语言行为进行探讨。我们通过对12种不同语言的对抗鲁棒性和多模态安全性进行系统研究，填补了这一空白，评估了通过指令调优获得多语言能力的开源MLLM。基于梯度的攻击揭示了一种可转移的多语言脆弱性：在一种语言中优化的对抗图像在其他语言中仍然会导致失败，展示了强烈的跨语言可转移性。多语言安全性还与模型检索或解释有害指令的有效性有关。当通过文本发出有害意图时，具有更强语言基础的语言更容易引发助长误用的响应，而较弱的语言则产生较少的不安全输出。当作为排版内容嵌入图像时，英语文本被可靠地识别和遵循，而非英语文本则很少被视觉编码器解析。因此，低资源语言可能看起来更安全，但这实际上是理解和视觉基础失败的结果，而非真正的对齐现象，我们称之为“失败中的安全”。相比之下，在训练阶段全面构建多语言能力的MLLM，如Qwen3-VL，展现出真正的跨语言安全性，在不同语言中保持主动拒绝，而不是掩盖理解失败。浅层的多语言适应，例如在翻译的指令数据上进行微调，可能产生表面理解，从而在低资源语言中创造出虚假的安全感；而在训练阶段的更深层次整合则导致真正的多语言安全对齐。

View on arXiv Download PDF AI Translation

cs.CL / 80 / 2606.03810

Consistency Training Can Entrench Misalignment

一致性训练可能加剧不一致性

Africa, David Demitri, Mani, Arathi

Abstract

Consistency training encourages a model to produce similar outputs across related inputs or sampling procedures. Such methods are simple, scalable, and largely label-free, but their effects on model alignment remain poorly understood. Could the self-bootstrapping nature of these methods amplify undesired behavior in models? We test seven consistency training methods on 108 ``model organisms: open-source models (7B--70B) fine-tuned to exhibit various forms of controlled misaligned behavior. We find that outcomes vary significantly: consistency training generally suppresses reward hacking and emergent misalignment but amplifies sycophancy. We present evidence that distribution shifts induced by the consistency labeling process, rather than variation in the selection operators, may be the primary driver of systematic alignment effects. Finally, we present a unifying theoretical framework to derive conditions under which consistency training will amplify or suppress misalignment. In total, our study establishes that consistency training is not alignment-neutral, and that its use in critical systems should be carefully audited.

Chinese Translation

一致性训练鼓励模型在相关输入或采样过程中产生相似的输出。这类方法简单、可扩展且在很大程度上不依赖标签，但它们对模型一致性的影响仍然不甚了解。这些方法的自引导特性是否会放大模型中的不良行为？我们在108个“模型生物体”上测试了七种一致性训练方法：这些开放源代码模型（7B-70B）经过微调以表现出各种形式的受控不一致行为。我们发现结果差异显著：一致性训练通常抑制奖励黑客行为和突现不一致性，但却放大了谄媚行为。我们提供证据表明，由一致性标注过程引起的分布变化，而非选择操作符的变化，可能是系统性一致性效应的主要驱动因素。最后，我们提出了一个统一的理论框架，以推导一致性训练放大或抑制不一致性的条件。总体而言，我们的研究表明一致性训练并非与一致性中立，其在关键系统中的使用应谨慎审计。

View on arXiv Download PDF AI Translation

cs.CL / 81 / 2606.03817

Rethinking the Idiomaticity Decomposability Hypothesis: Evidence from Distributional Learning

重新思考习语可分解性假说：来自分布式学习的证据

Mi, Maggie, Atefi, Golzar, Yamaguchi, Atsuki, Gers, Felix, Villavicencio, Aline, Moosavi, Nafise Sadat

Abstract

Idioms can be analysed in terms of their decomposability, the extent to which constituent meanings contribute to the figurative whole. Decomposability is thought to predict syntactic flexibility. Usage-based accounts instead attribute idiom behaviour to distributional experience, such as speaker familiarity and predictability. We examine these views using contextualised language models as controlled distributional learners. We propose a model-internal measure of decomposability and relate it to human ratings, syntactic flexibility, and predictability while tracking idiom learning during pretraining. Model-derived decomposability correlates weakly with human judgments and shows a small but consistent negative relationship with syntactic flexibility. Pretraining analyses show that stabilisation of idiom representations in models is not explained by frequency alone. Instead, surprisal, decomposability, and frequency all contribute, with decomposability showing the strongest training-dependent effect.

Chinese Translation

习语可以根据其可分解性进行分析，即构成意义在多义整体中的贡献程度。可分解性被认为可以预测句法灵活性。基于使用的理论则将习语行为归因于分布式经验，例如说话者的熟悉度和可预测性。我们使用上下文化语言模型作为受控的分布式学习者来检验这些观点。我们提出了一种模型内部的可分解性度量，并将其与人类评分、句法灵活性和可预测性相关联，同时跟踪习语在预训练过程中的学习情况。模型导出的可分解性与人类判断的相关性较弱，并且与句法灵活性呈现出小但一致的负相关关系。预训练分析表明，模型中习语表示的稳定性并不仅仅由频率解释。相反，惊讶度、可分解性和频率都对其有贡献，其中可分解性显示出最强的依赖于训练的效果。

View on arXiv Download PDF AI Translation

cs.CL / 82 / 2606.03846

Clustered Self-Assessment: A Simple yet Effective Method for Uncertainty Quantification in Large Language Models

聚类自我评估：一种简单而有效的大语言模型不确定性量化方法

Cao, Qi, Kojima, Takeshi, Gambardella, Andrew, Peng, Helinyi, Matsuo, Yutaka, Iwasawa, Yusuke

Abstract

Large language models (LLMs) demonstrate remarkable performance across diverse tasks, but they often generate responses that appear plausible while being factually incorrect. This problem is compounded by the lack of explicit uncertainty estimates, which makes it difficult for users to judge the reliability of model outputs. Existing uncertainty quantification methods typically rely on indirect signals, such as entropy across sampled generations. These signals can be difficult to interpret and do not fully leverage the model's ability to assess its own uncertainty. We propose a simple yet effective self-assessment method for uncertainty quantification in LLMs. Our approach groups sampled generations into semantically distinct clusters, converts them into answer options in a structured multiple-choice question, and uses the probability assigned by the LLM to each option as a confidence estimate. Experiments across multiple models and datasets show that our method consistently outperforms baseline approaches. Notably, it achieves competitive performance with as few as two additional samples, demonstrating both its effectiveness and efficiency.

Chinese Translation

大型语言模型（LLMs）在多种任务中表现出色，但它们常常生成看似合理但实际上不准确的回答。这个问题因缺乏明确的不确定性估计而加剧，使得用户难以判断模型输出的可靠性。现有的不确定性量化方法通常依赖于间接信号，例如在采样生成中计算的熵。这些信号可能难以解释，并且未能充分利用模型评估自身不确定性的能力。我们提出了一种简单而有效的自我评估方法，用于LLMs中的不确定性量化。我们的方法将采样生成分组为语义上不同的聚类，将其转换为结构化多项选择题中的答案选项，并使用LLM为每个选项分配的概率作为置信度估计。在多个模型和数据集上的实验表明，我们的方法始终优于基线方法。值得注意的是，它在仅增加两个样本的情况下也能达到竞争性的性能，展示了其有效性和高效性。

View on arXiv Download PDF AI Translation

cs.CL / 83 / 2606.03867

A Training-Free Mixture-of-Agents Framework for Multi-Document Summarization using LLMs and Knowledge Graphs

一种无训练的混合代理框架用于基于大型语言模型和知识图谱的多文档摘要

Tuan, Cuong Vuong, Xuan, Trang Mai, Nguyen, Tien-Cuong, Ngo, Vu-Duc, Van Luong, Thien

Abstract

Multi-Document Summarization (MDS) plays a critical role in distilling essential information from collections of textual data. Existing approaches often struggle to capture complex inter-document relationships, rely heavily on large amounts of labeled data for supervised training, or exhibit limited generalization across domains and languages. To address these limitations, we present a training-free mixture-of-agents framework for MDS that leverages the complementary strengths of large language models (LLMs) and knowledge graphs. Our approach decomposes summarization into specialized agent tasks: extractive selection, knowledge-aware abstraction, and iterative refinement, each operating without task-specific fine-tuning. We unify their outputs using a multi-perspective consistency mechanism guided by LLMs. Experiments across four datasets in English and Vietnamese demonstrate state-of-the-art or competitive performance, validating the effectiveness and adaptability of our modular design.

Chinese Translation

多文档摘要（MDS）在从文本数据集合中提取关键信息方面发挥着重要作用。现有的方法往往难以捕捉复杂的文档间关系，过度依赖大量标注数据进行监督训练，或在不同领域和语言之间表现出有限的泛化能力。为了解决这些局限性，我们提出了一种无训练的混合代理框架用于多文档摘要，该框架利用大型语言模型（LLMs）和知识图谱的互补优势。我们的方法将摘要过程分解为专门的代理任务：抽取选择、知识感知抽象和迭代精炼，每个任务均不进行特定任务的微调。我们使用由LLMs指导的多视角一致性机制统一它们的输出。在英语和越南语的四个数据集上的实验表明，我们的方法在性能上达到最先进水平或具有竞争力，验证了我们模块化设计的有效性和适应性。

View on arXiv Download PDF AI Translation

cs.CL / 84 / 2606.03889

RealClawBench: Live OpenClaw Benchmarks from Real Developer-Agent Sessions

RealClawBench：来自真实开发者-代理会话的实时 OpenClaw 基准测试

Lv, Zongwei, Tan, Zhewen, Li, Yaoming, Yao, Yilun, Tian, Yuxuan, Sun, Lin, Zhang, Xiangzheng, Lin, Weihong, Yang, Tong, Zhao, Guangxiang

Abstract

Agent benchmarks should reflect what users actually ask deployed agents to do, yet existing benchmarks often miss key realism properties of real developer-agent sessions. We introduce RealClawBench, a live benchmark framework built from real OpenClaw sessions to capture the distribution, diversity, and real-world difficulty of deployed agent use. Real user requests are challenging to benchmark because they often depend on local execution environments, involve implicit or underspecified intent, and require nontrivial verification. RealClawBench addresses these challenges with two core mechanisms: reconstructed execution environments and deterministic verifiable scorers, which together convert real sessions into reproducible, automatically scored tasks. The resulting release contains 281 executable tasks sampled from a much larger real-session pool while preserving the source distribution, with maximum final-vs-source Jensen-Shannon divergence of 0.0448. Evaluating 14 contemporary models shows that the best system solves only 65.8% of tasks, revealing substantial headroom on realistic developer-agent workloads. By turning real deployed sessions into controlled evaluation instances, RealClawBench provides a practical path toward benchmarks that better measure agent capability in actual use. Code is available at:https://anonymous.4open.science/r/real-claw-bench-582B.

Chinese Translation

代理基准测试应反映用户实际要求已部署代理执行的任务，然而现有基准测试往往缺乏真实开发者-代理会话的关键现实特性。我们引入了 RealClawBench，这是一个基于真实 OpenClaw 会话构建的实时基准测试框架，旨在捕捉已部署代理使用的分布、多样性和现实世界的难度。真实用户请求的基准测试具有挑战性，因为它们通常依赖于本地执行环境，涉及隐含或未明确指定的意图，并且需要非平凡的验证。RealClawBench 通过两种核心机制来应对这些挑战：重建的执行环境和确定性可验证的评分器，这两者共同将真实会话转换为可重复的、自动评分的任务。最终发布的版本包含从更大规模的真实会话池中抽样的 281 个可执行任务，同时保持源分布，最大最终与源的 Jensen-Shannon 散度为 0.0448。对 14 个当代模型的评估表明，最佳系统仅解决了 65.8% 的任务，揭示了在现实开发者-代理工作负载上存在显著的提升空间。通过将真实的部署会话转化为受控评估实例，RealClawBench 为更好地测量代理在实际使用中的能力提供了一条切实可行的路径。代码可在以下链接获取：https://anonymous.4open.science/r/real-claw-bench-582B。

View on arXiv Download PDF AI Translation

cs.CL / 85 / 2606.03892

Synthesize and Reward -- Reinforcement Learning for Multi-Step Tool Use in Live Environments

合成与奖励——用于实时环境中多步骤工具使用的强化学习

Abdelaziz, Ibrahim, Munawar, Asim, Basu, Kinjal, Crouse, Maxwell, Gunasekara, Chulaka, Katrekar, Suneet, Kapanipathi, Pavan

Abstract

Training LLMs to orchestrate multi-step tool calls is held back by three coupled obstacles: realistic stateful execution environments are costly to build, synthetic training queries are often detached from the server's actual state (so the generated tool calls fail to execute), and recall-based RL rewards incentivize verbose tool-calling patterns. We present PROVE (Programmatic Rewards On Verified Environments), a framework with three contributions: (1) a library of 20 stateful MCP (Model Context Protocol) servers exposing 343 tools, enabling live-execution RL training with session-scoped state isolation; (2) an automated data synthesis pipeline that generates validated multi-turn tool-call trajectories against these servers via dependency-graph-guided conversation simulation grounded in live-sampled server state, so every generated query references entities that actually exist; and (3) a multi-component programmatic reward - graduated validity scoring, dependency-aware coverage, an adaptive efficiency penalty with a complexity-scaled call budget, a tool-name signal, and an argument-value matching bonus - requiring no external judge model. We train four models (Qwen3-4B, Qwen3-8B, Qwen2.5-7B, Granite-4.1-8B) with GRPO using identical reward hyperparameters and ~13K training examples; only learning rate is tuned per model family from a three-point sweep. On BFCL Multi-Turn, tau2-bench, and T-Eval, PROVE yields improvements of up to +10.2, +6.8, and +6.5 points respectively, demonstrating that a compact programmatic reward yields consistent gains on multi-step tool orchestration across two model families.

Chinese Translation

训练大型语言模型（LLMs）以协调多步骤工具调用受到三个相互关联的障碍的制约：构建真实的有状态执行环境成本高昂，合成训练查询往往与服务器的实际状态脱节（因此生成的工具调用无法执行），而基于回忆的强化学习奖励则激励冗长的工具调用模式。我们提出了PROVE（在验证环境中进行程序化奖励），该框架有三个贡献：（1）一个包含20个有状态的MCP（模型上下文协议）服务器的库，暴露出343个工具，使得能够在会话范围内状态隔离的情况下进行实时执行的强化学习训练；（2）一个自动化的数据合成管道，通过基于实时采样服务器状态的依赖图引导的对话模拟，生成针对这些服务器的经过验证的多轮工具调用轨迹，从而确保每个生成的查询都引用实际存在的实体；（3）一个多组件的程序化奖励——逐级有效性评分、依赖感知覆盖、具有复杂度缩放调用预算的自适应效率惩罚、工具名称信号以及参数值匹配奖金——不需要外部评判模型。我们使用GRPO训练了四个模型（Qwen3-4B、Qwen3-8B、Qwen2.5-7B、Granite-4.1-8B），使用相同的奖励超参数和约13K的训练示例；仅对每个模型系列的学习率进行了三点范围的调优。在BFCL多轮、tau2-bench和T-Eval上，PROVE分别实现了高达+10.2、+6.8和+6.5的改进，证明了紧凑的程序化奖励在两个模型系列的多步骤工具协调中带来了持续的收益。

View on arXiv Download PDF AI Translation

cs.CL / 86 / 2606.03924

Knowledge Editing in Masked Diffusion Language Models

掩蔽扩散语言模型中的知识编辑

Park, Haewon, Jo, Yohan

Abstract

Knowledge editing aims to update or correct factual knowledge in a language model. A widely used approach, locate-then-edit, does this in two steps: it first localizes a fact within the model, then edits the weights there. To date, such methods have been developed exclusively on autoregressive models (ARMs). Whether their underlying assumptions hold for masked diffusion models (MDMs), which model text bidirectionally and generate by iterative denoising rather than next-token prediction, remains an open question. We address it by transferring locate-then-edit to MDMs and comparing two MDMs (LLaDA, Dream) with two ARMs (LLaMA, Qwen) at matched scale. Our central finding has two parts. First, where an edit is applied transfers across paradigms: causal tracing highlights the same early-to-mid-layer MLP at the last subject token in both, and editing is most effective there. Second, this shared location does not guarantee a shared outcome. Single-token edits succeed in both, but as targets grow longer, editing degrades systematically in the MDMs but not the ARMs. The failure stems from how the edited fact is generated: producing a multi-token target requires passing through partially unmasked intermediate states for which the edit was never optimized. Guided by this diagnosis, we introduce a simple correction that optimizes the edit for these states, substantially restoring multi-token performance.

Chinese Translation

知识编辑旨在更新或纠正语言模型中的事实知识。一种广泛使用的方法是先定位后编辑（locate-then-edit），该方法分为两个步骤：首先在模型中定位一个事实，然后在该位置编辑权重。迄今为止，这种方法仅在自回归模型（ARMs）上开发。掩蔽扩散模型（MDMs）是否适用其基本假设仍然是一个悬而未决的问题，因为MDMs是双向建模文本并通过迭代去噪而非下一个标记预测来生成文本。我们通过将先定位后编辑方法转移到MDMs上，并在匹配规模下比较两个MDMs（LLaDA，Dream）与两个ARMs（LLaMA，Qwen），来解决这个问题。我们的主要发现有两个部分。首先，编辑应用的位置在不同范式间是可转移的：因果追踪（causal tracing）在两个模型的最后主题标记处突出了相同的早期到中层多层感知机（MLP），并且在该位置进行编辑最为有效。其次，这一共享位置并不保证共享结果。单标记编辑在两个模型中都成功，但随着目标变得更长，MDMs中的编辑效果系统性下降，而ARMs则没有。这一失败源于编辑事实的生成方式：生成多标记目标需要经过部分未掩蔽的中间状态，而这些状态并未针对编辑进行优化。根据这一诊断，我们引入了一种简单的修正方法，针对这些状态优化编辑，显著恢复了多标记的性能。

View on arXiv Download PDF AI Translation

cs.CL / 87 / 2606.03948

A Pocket Offline Model for Simultaneous Speech Translation as CUNI Submission to IWSLT 2026

用于同时语音翻译的便携离线模型：CUNI提交至IWSLT 2026

Ortega, Aziz Sharipov, Macháček, Dominik

Abstract

We implement simultaneous translation capability with the offline direct speech-to-text translation model Canary, using the state-of-the-art policy AlignAtt, and submit it to IWSLT 2026 Simultaneous Speech Translation Shared task for Czech to English and English to German and Italian. The strengths of our system are: (1) high translation quality, outperforming similarly sized baselines both in low- and high-latency regimes in computationally unaware simulations; (2) low computational requirements, as the model has only 1B parameters; (3) multilinguality -- support of 25 source and 25 target languages.

Chinese Translation

我们实现了使用离线直接语音转文本翻译模型Canary的同时翻译能力，采用了最先进的策略AlignAtt，并将其提交至IWSLT 2026同时语音翻译共享任务，涵盖捷克语到英语以及英语到德语和意大利语的翻译。我们系统的优势包括：（1）高翻译质量，在计算不敏感的模拟中，在低延迟和高延迟环境下均优于同等规模的基线；（2）低计算需求，因为该模型仅有10亿参数；（3）多语言支持——支持25种源语言和25种目标语言。

View on arXiv Download PDF AI Translation

cs.CL / 88 / 2606.03957

Efficient ASR Training with Conversations that Never Happened

利用未发生的对话进行高效的自动语音识别训练

Gedeon, Máté, Mihajlik, Péter

Abstract

Conversational ASR for lower-resource languages and niche domains is limited by the scarcity of domain-matched multi-speaker training data. We propose an augmentation pipeline that generates scenario-level dialogues with participant metadata, maps speaker attributes to TTS voice profiles, and assembles synthesized utterances into speaker-aware simulated conversations. We evaluated five LLM families under single-generator, fixed-budget mixture, and scale-up settings using the same FastConformer-Large training recipe for each one. We ran comprehensive evaluations on the Hungarian BEA-Dialogue benchmark corpus, with the method itself being applicable to any language given the resources for each component. The results show that synthetic conversations consistently improve speech recognition performance, but generator choice and data composition strongly affect the gains. Our largest training configuration, using only 67 hours of real conversations and 636 hours of simulated data, achieves better performance on the evaluation benchmark than a zero-shot model trained on 2700 hours of Hungarian speech. These findings indicate that LLM-generated conversational data synthesized with TTS is a practical complement to real conversational corpora for speech model training.

Chinese Translation

针对低资源语言和小众领域的对话式自动语音识别（ASR）受到领域匹配的多说话者训练数据稀缺的限制。我们提出了一种增强管道，生成具有参与者元数据的场景级对话，将说话者属性映射到文本转语音（TTS）语音配置文件，并将合成的语句组装成具有说话者意识的模拟对话。我们在单生成器、固定预算混合和扩展设置下评估了五个大型语言模型（LLM）系列，使用相同的 FastConformer-Large 训练配方。我们在匈牙利 BEA-Dialogue 基准语料库上进行了全面评估，该方法本身适用于任何语言，只要具备每个组件所需的资源。结果表明，合成对话持续改善语音识别性能，但生成器的选择和数据组成对增益有显著影响。我们最大的训练配置仅使用 67 小时的真实对话和 636 小时的模拟数据，在评估基准上取得了优于在 2700 小时匈牙利语音上训练的零样本模型的表现。这些发现表明，使用 TTS 合成的 LLM 生成的对话数据是语音模型训练中对真实对话语料库的实用补充。

View on arXiv Download PDF AI Translation

cs.CL / 89 / 2606.03967

AlignAtt4LLM: Fast AlignAtt for Decoder-Only LLMs at IWSLT 2026 Simultaneous Speech Translation Task

AlignAtt4LLM：IWSLT 2026 同步语音翻译任务中针对仅解码器 LLM 的快速 AlignAtt

Fuxa, Quentin, Macháček, Dominik

Abstract

We describe AlignAtt4LLM, an IWSLT 2026 simultaneous speech translation system for English to German, Italian, and Chinese. The system is a synchronous cascade: Qwen3-ASR with forced alignment produces an incrementally updated source transcript, and Gemma-4 E4B-it translates that prefix under an MT-side AlignAtt policy. To our knowledge, this is the first application of AlignAtt to a decoder-only LLM, where the encoder-decoder cross-attention used by earlier AlignAtt systems is absent. We recover a usable policy by proposing (1) an explicit source span in the prompt, (2) offline selection of translation-specific alignment heads, (3) selective qk-fast replay of the draft-to-source attention block, and (4) runtime query/key capture that preserves model outputs bit-identically. On the IWSLT 2026 development set, AlignAtt4LLM outperforms the supplied baselines for the European target languages, English to German and English to Italian, in both the low-latency regime around 2 seconds and the high-latency regime below 4 seconds CU-LongYAAL. Results for English to Chinese are more mixed, but the method is not tied to Gemma-4: because AlignAtt4LLM only requires a deterministic prompt layout, calibrated attention heads, and query/key capture, the same policy can be reapplied to stronger translation-focused decoder-only MT backbones for non-European target languages.

Chinese Translation

我们描述了 AlignAtt4LLM，这是一个用于英语到德语、意大利语和中文的 IWSLT 2026 同步语音翻译系统。该系统采用同步级联结构：Qwen3-ASR 通过强制对齐生成增量更新的源文本，而 Gemma-4 E4B-it 在机器翻译侧的 AlignAtt 策略下翻译该前缀。据我们所知，这是 AlignAtt 首次应用于仅解码器 LLM，其中早期 AlignAtt 系统使用的编码器-解码器交叉注意力被省略。我们通过提出以下方法恢复了可用的策略：(1) 在提示中明确源跨度，(2) 离线选择翻译特定的对齐头，(3) 选择性地快速重放草稿到源的注意力块，以及 (4) 运行时查询/键捕捉，保持模型输出的比特完全一致。在 IWSLT 2026 开发集上，AlignAtt4LLM 在低延迟（约 2 秒）和高延迟（低于 4 秒 CU-LongYAAL）条件下均优于提供的基线，尤其是在欧洲目标语言（英语到德语和英语到意大利语）方面。英语到中文的结果则较为复杂，但该方法并不依赖于 Gemma-4：由于 AlignAtt4LLM 仅需确定性的提示布局、校准的注意力头和查询/键捕捉，因此同样的策略可以重新应用于更强的以翻译为中心的仅解码器机器翻译基础架构，适用于非欧洲目标语言。

View on arXiv Download PDF AI Translation

cs.CL / 90 / 2606.03968

QUBRIC: Co-Designing Queries and Rubrics for RL Beyond Verifiable Rewards

QUBRIC：超越可验证奖励的强化学习查询与评分标准的协同设计

Zhang, Rongzhi, Feng, Rui, Zhang, Zhihan, Yang, Jingfeng, Yin, Qingyu, Liu, Xin, Zhang, Zixuan, Nigam, Priyanka, Yin, Bing, Zhao, Tuo, Zhang, Chao

Abstract

Rubric-based RL is a promising route for extending reinforcement learning beyond verifiable rewards, yet existing methods optimize rubrics while treating the query distribution as fixed. We identify a structural bottleneck: rubric quality is constrained by query structure. Open-ended queries yield vague rubrics; naively narrowing them introduces fabricated references that no model can verify, so all responses fail and training receives no reward signal. We present QUBRIC, a framework that co-designs queries and rubrics. Teacher-derived key points ground the rewriting of open-ended queries into scenario-based, evaluable questions. Contrastive rubric generation then turns teacher-policy gaps into query-level criteria, and learnability filtering retains only informative query-rubric pairs for GRPO training. QUBRIC achieves a +5.5 point gain on ArenaHard over the SFT baseline. Trained only on instruction-following data, it further transfers to three held-out benchmarks spanning legal, moral, and narrative reasoning (+6.3 points on average), with improvements concentrated in reasoning-related dimensions. These results provide evidence that co-designing queries and rubrics can make rubric-based RL a practical complement to RLVR beyond strictly verifiable tasks.

Chinese Translation

基于评分标准的强化学习（RL）是扩展强化学习超越可验证奖励的一个有前景的途径，然而现有方法在优化评分标准时将查询分布视为固定。我们识别出一个结构性瓶颈：评分标准的质量受到查询结构的限制。开放式查询会导致模糊的评分标准；简单地缩小查询范围会引入虚构的参考，这些参考无法被任何模型验证，因此所有响应都失败，训练也无法获得奖励信号。我们提出了QUBRIC，一个协同设计查询和评分标准的框架。教师导出的关键点为将开放式查询重写为基于场景的可评估问题提供了基础。对比评分标准生成则将教师策略的差距转化为查询级标准，而可学习性过滤仅保留对GRPO训练有信息量的查询-评分标准对。QUBRIC在ArenaHard上相较于SFT基线实现了+5.5的得分提升。仅基于遵循指令的数据进行训练，它还进一步迁移到三个保留基准，涵盖法律、道德和叙事推理（平均提升+6.3分），改进集中在与推理相关的维度。这些结果提供了证据，表明协同设计查询和评分标准可以使基于评分标准的强化学习成为超越严格可验证任务的强化学习可验证奖励的实用补充。

View on arXiv Download PDF AI Translation

cs.CL / 91 / 2606.03969

Quantifying Faithful Confidence Expression in Large Reasoning Models

量化大型推理模型中的忠实置信表达

Gani, Areeb, Meskin, Asal, Liu, Gabrielle Kaili-May, Cohan, Arman

Abstract

Reliable uncertainty communication is critical to the trustworthiness of LLMs, yet faithful calibration (FC)--the alignment between models' intrinsic and (linguistically) expressed confidence--is a persistent failure mode. This challenge is key for large reasoning models (LRMs), whose extended reasoning traces are often interpreted by users as evidence of deliberation, competence, and confidence. Despite the importance of FC and wide usage of LRMs, the extent to which LRMs can faithfully express their confidence remains poorly understood. Moreover, the prevailing paradigm to measure FC does not generalize well to the long chain-of-thought outputs generated by LRMs, which tend to lack clear step boundaries, involve inconsistent step structure, and encode complex conditional dependencies throughout the trace--complicating estimation of intrinsic confidence. To address this challenge, we introduce a novel framework to systematically quantify FC of LRMs. Our framework analyzes linguistic decisiveness relative to three sources of internal uncertainty, based on token probabilities, hidden states, and sampled response consistency. We also devise a prefix-conditioned sampling approach to control for conditional and structural variation across traces. Applying our framework to a diverse suite of leading models, datasets, and prompts, we find that faithful confidence expression is a significant challenge for LRMs. Reasoning behaviors do not automatically translate to improved FC, and prompt interventions for non-reasoning models do not improve faithfulness in the reasoning setting. Different confidence estimators further produce divergent assessments of the same traces, revealing fragility in prior evaluation methodologies. Taken together, our work establishes FC as a distinct reliability and alignment target for LRMs, particularly as such systems are increasingly deployed in high-stakes contexts.

Chinese Translation

可靠的不确定性沟通对于大型语言模型（LLMs）的可信度至关重要，然而忠实校准（Faithful Calibration, FC）——模型内在置信度与（语言上）表达的置信度之间的对齐——仍然是一个持续的失败模式。这个挑战对于大型推理模型（Large Reasoning Models, LRMs）尤为关键，因为用户通常将其扩展的推理轨迹解读为深思熟虑、能力和信心的证据。尽管FC的重要性以及LRMs的广泛应用，LRMs在多大程度上能够忠实地表达其置信度仍然不甚明了。此外，当前测量FC的主流范式并不适用于LRMs生成的长链思维输出，这些输出往往缺乏清晰的步骤边界，涉及不一致的步骤结构，并在整个轨迹中编码复杂的条件依赖关系，从而使得内在置信度的估计变得复杂。为了解决这一挑战，我们提出了一个新颖的框架，以系统地量化LRMs的FC。我们的框架分析了相对于三种内部不确定性来源的语言决断性，这些来源基于标记概率、隐藏状态和采样响应一致性。我们还设计了一种前缀条件采样方法，以控制轨迹中的条件和结构变异。将我们的框架应用于一系列领先的模型、数据集和提示，我们发现忠实置信表达是LRMs面临的一个重大挑战。推理行为并不会自动转化为改善的FC，而对非推理模型的提示干预在推理环境中并未提高忠实度。不同的置信度估计器进一步对相同轨迹产生不同的评估，揭示了先前评估方法的脆弱性。综合来看，我们的工作将FC确立为LRMs的一个独特可靠性和对齐目标，特别是随着此类系统在高风险环境中的日益应用。

View on arXiv Download PDF AI Translation

cs.CL / 92 / 2606.03982

Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics

语言模型使用特定数字和单位的启发式方法比较数量

Sasaki, Mutsumi, kamoda, Go, Takahashi, Ryosuke, Sato, Kosuke, Inui, Kentaro, Sakaguchi, Keisuke, Heinzerling, Benjamin

Abstract

Quantities with measurement units, such as 110 cm and 1.2 m, require language models (LMs) to combine a numeral with a symbolic unit scale. Here, we study how LMs compare such quantities in controlled settings spanning several unit systems. We find that accuracy degrades near the comparison boundary, where small changes in value determine the correct answer. The resulting errors are systematic: linear surrogate models predict LM preferences from numerical-difference and unit-scale-difference cues, and causal interventions on subspaces aligned with these variables shift model's output. The results suggest that LMs compare quantities through a bag of heuristics over numerals and units, rather than first converting both expressions to an exact shared-scale representation.

Chinese Translation

带有测量单位的数量，例如110厘米和1.2米，要求语言模型（LM）将数字与符号单位尺度结合起来。在此，我们研究了语言模型在多个单位系统的受控环境中如何比较这些数量。我们发现，在比较边界附近，准确性下降，此时数值的微小变化决定了正确答案。由此产生的错误是系统性的：线性替代模型根据数值差异和单位尺度差异的线索预测语言模型的偏好，并且在与这些变量对齐的子空间上进行因果干预会改变模型的输出。结果表明，语言模型通过对数字和单位的一系列启发式方法来比较数量，而不是首先将这两种表达转换为精确的共享尺度表示。

View on arXiv Download PDF AI Translation

arXiv Papers

Too Much of a Good Thing: When sim2real Efforts Impede Policy Learning (And What to Do About It)

CARVE: Certified Affordable Repair of Vetoed Maneuvers via Envelopes for Interactive Driving

Fixed-Time Dynamic Landing of Quadrotors using Adaptive Unscented Kalman Filtering and Nonlinear Model Predictive Control

Motion Planning in Dynamic Environments: A Survey from Classical to Modern Methods

See Less, Specify More: Visual Evidence Budgets for Generalizable VLAs

SeeTraceAct: Visibility-Aware Latent Planning from Cross-Embodiment Demonstration Videos

Hybrid Adaptive Kalman Filtering for Data-Efficient Joint Tracking and Classification

A Measurement-Driven Digital Twin Architecture for Plant-Level Biomass Estimation and Growth Forecasting in Hydroponic Systems

Direct Informed Sampling on Riemannian Manifolds via Loewner Order Lower Bounds

Impact of a Soft Wearable Back-Support Device on Postural Stability during Trip-Like Perturbations

Improved Postural Stability Using a Lightweight Semi-Active Soft Back Support Device Under Standing Perturbations

SCOPE: Real-Time Natural Language Camera Agent at the Edge

Hybrid Dynamics Modeling for a Flexible 2-DoF Robotic Arm

MARIO: Motion-Augmented Real-Time Multi-Sensor Inertial Odometry

ModuLoop : Low-Level Code Generation using Modular Synthesizer and Closed-Loop Debugger for Robotic Control

TTT-VLA: Test-Time Latent Prompt Optimization for Vision-Language-Action Models

How Visible Are Silent Manipulation Failures? An Observability Study of False-Success Detection in Simulated Robot Episodes

ConTrack: Constrained Hand Motion Tracking with Adaptive Trade-off Control

GeoSem-WAM: Geometry- and Semantic-Aware World Action Models

Toward Gripper-Integrated Active Electrosense for Pre-Contact Sensing in Underwater Soft Grippers

BotDirector: Robot Storytelling Across the Symmetrical Reality with Multi-modal Interactions

GeoAlign: Beyond Semantics with State-Guided Spatial Alignment in VLA Models

AirDreamer: Generalist Drone Navigation with World Models

Wheel-Mounted/GNSS Fusion with AI-Aided Position Updates

EaDex: A Cross-Embodiment Dexterous Manipulation Framework from Low-Cost Demonstrations

Bridging Predictive Uncertainty and Safe Action: Sample-Conditioned Differentiable Planning for Autonomous Driving

SplitAdapter: Load-Aware Humanoid Loco-Manipulation via Factorized Adaptation

RobotValues: Evaluating Household Robots When Human Values Conflict

GPU-Parallel Multi-Task Reinforcement Learning with Demonstration Guided Policy Optimization

Autonomous Navigation System for Library Service Robot Based on Unitree Go2 Edu

eMEM: A Hybrid Spatio-Temporal Memory System For Embodied Agents

Grasp-Then-Plan with Failure Attribution: A Closed Two-Stage Framework for Precise and Generalizable Robotic Manipulation

Extreme Motion Generation via Hybrid Null-Space Control for Straight-Line Path Following

OpenEAI-Platform: An Open-source Embodied Artificial Intelligence Hardware-Software Unified Platform

Reliability-Guided Depth Fusion for Glare-Resilient Navigation Costmaps

PerchRL: Vision-Based Agile Perching on Inclined Platforms under Rapid and Irregular Motion

Human2Humanoid: Physics-Aware Cross-Morphology Motion Retargeting for Humanoid Robots

SPADE: Sketch-guided Path Planning Augmented with Diffusion Experts

Bionic Human-Motion Style Transfer for Physically Executable Whole-Body Control of Humanoid Robots

Static and Dynamic Representations for Tactile Contact-Angle Estimation with Event-Based Sensors

NVIDIA Isaac Sim: Enabling Scalable, GPU-Accelerated Simulation for Robotics

Partially Observable Adversarial Patch Attacks on Vision-Language-Action Models in Robotics

CANMOT: Class-Aware Noise Modeling for Multi-Object Tracking in Autonomous Driving

PHASER: Phase-Aware and Semantic Experience Replay for Vision-Language-Action Models

A 3D Isovist World Model -- Revealing a City's Unseen Geometry and Its Emergent Cross-City Signature

GN0: Toward a Unified Paradigm for Generation, Evaluation, and Policy Learning in Visual-Language Navigation

Face versus Body Tracking for Human-Robot Interaction: An Egocentric Dataset

Neural Navigation Functions for Zero-Shot Generalizable Motion Planning

Revisiting Embodied Chain-of-Thought for Generalizable Robot Manipulation

Worth Remembering: Surprise-Gated Robot Episodic Memory

Optimal Design and Analytical Modeling of a Soft Fin-Ray Effect Gripper Finger Using the Finite Rigid Elements Method

Let the Dynamics Flow: Stable Flow Matching Dynamical Systems

Denoising Tells When to Replan: Denoising-Variance Adaptive Chunking for Flow-Based Robot Policies

Semantic-weighted ICP for LiDAR Odometry: Class-Aware Residual Reweighting for Robust Scan Registration

Multi-Robot Bearing-only Pose Estimation via Angle Rigidity

PointAction: 3D Points as Universal Action Representations for Robot Control

Preference-Calibrated Human-in-the-Loop Reinforcement Learning for Robotic Manipulation

Self-Refining Agentic Reinforcement Learning for Vision-Conditioned UAV Navigation

Humanoid-GPT: Scaling Data and Structure for Zero-Shot Motion Tracking

COD10K-C: Benchmarking Robustness of Camouflaged Object Detection Under Natural Image Corruptions

AVTrack: Audio-Visual Tracking in Human-centric Complex Scenes

Consistent Yet Wrong: Evidence Insensitivity in Spatial Vision-Language Models

Plan2Map: A Multimodal Benchmark for Document-Grounded Geospatial Boundary Reconstruction from Planning Records

MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data

From Local Training to Large-Scale Mapping: A Comparative Assessment of Machine Learning and Deep Learning for Transferable Satellite-Derived Bathymetry

GeoDrive-Bench: Benchmarking Region-Specific Multimodal Reasoning in Autonomous Driving

Diagnosis of Human Object Interaction Detectors for Real World Educational Applications

Cosmos 3: Omnimodal World Models for Physical AI

Automated Report-Derived Oncology VQA Benchmark for Evaluating Vision-Language Models on 3D Medical Imaging

Principled Reflection Separation via Nonlinear Superposition and Feature Interaction

Pathway-Structured Privileged Distillation for Deployable Computational Pathology

Tiny Collaborative Inference for Occlusion-Robust Object Detection

Any2Poster: Any-Source Poster Generation Across Modalities and Domains

Pixel Cube: Diffusion-based Portrait Video Relighting Through Realistic Lighting Reproduction

ATLAS: A Large-Scale Evaluation Benchmark for Adversarial LiDAR Perception

SaluNet: Enabling Total Plasticity in Normalization-Free Deep Networks

CAD-to-CT Registration of Cylindrical Objects via Ellipse-Based Axis Estimation

The Road Ahead in Autonomous Driving: The KITScenes Multimodal Dataset

Hand Trajectory Fusion for Egocentric Natural Language Query Grounding