arXiv Daily Digest

618

Papers

Bayesian Optimization for Learning Nonlinear MPC in Autonomous Agent Navigation

用于自主代理导航的非线性模型预测控制的贝叶斯优化

Ortolani, Lorenzo, Voss, Gabriel, Beltrami, Gabriele, Dorati, Francesco, Banfi, Tommaso Felice

Abstract

Real-time autonomous navigation in dynamic, unknown environments remains a fundamental challenge for mobile robotics. We propose a map-free framework that tightly integrates reactive rolling-horizon planning with nonlinear Model Predictive Control (MPC). At each control cycle, a LiDAR-based Gaussian occupancy representation is constructed and used to generate collision-free trajectories via A* search, which are then tracked by a CasADi/IPOPT MPC formulation incorporating a smooth sigmoid obstacle barrier. To improve robustness to parameter sensitivity, we adopt an offline Bayesian optimization scheme based on Tree-structured Parzen Estimators (TPE), which identifies near-optimal controller parameters with respect to a composite navigation objective. In addition, a Gaussian Process surrogate is used to analyze parameter sensitivity and provide insight into the optimization landscape. The proposed framework is robot-agnostic and is evaluated on the Unitree Go2 quadruped in simulation using Gazebo, followed by deployment on the physical robot. Experimental results show that parameters tuned in simulation transfer effectively to hardware, maintaining comparable performance without additional tuning. The full system achieves up to a 90.0\% navigation success rate when deployed, along with a 38.9\% average improvement in the evaluation metrics across simulated environments.

Chinese Translation

在动态未知环境中进行实时自主导航仍然是移动机器人面临的一个基本挑战。我们提出了一种无地图框架，该框架将反应式滚动视野规划与非线性模型预测控制（MPC）紧密结合。在每个控制周期中，构建基于激光雷达的高斯占用表示，并通过 A* 搜索生成无碰撞轨迹，这些轨迹随后由包含平滑 sigmoid 障碍物屏障的 CasADi/IPOPT MPC 公式进行跟踪。为了提高对参数敏感性的鲁棒性，我们采用了一种基于树结构帕尔岑估计器（TPE）的离线贝叶斯优化方案，该方案识别与复合导航目标相关的近似最优控制器参数。此外，使用高斯过程代理分析参数敏感性，并提供对优化景观的洞察。所提出的框架与机器人无关，并在 Gazebo 中对 Unitree Go2 四足机器人进行了仿真评估，随后在物理机器人上部署。实验结果表明，在仿真中调优的参数能够有效转移到硬件上，保持相似的性能而无需额外调优。整个系统在部署时实现了高达 90.0\% 的导航成功率，并在模拟环境中评估指标上平均提高了 38.9\\%。

View on arXiv Download PDF AI Translation

cs.RO / 2 / 2606.14767

Synthetic-to-Real Pipeline for Safe Landing Zone Detection

安全着陆区检测的合成到真实管道

Banerjee, Shrikant, Faieghi, Reza

Abstract

As Uncrewed Aerial Vehicles (UAVs) transition toward higher levels of autonomy, the ability to perform unassisted recovery in non-cooperative, unstructured environments becomes critical. Achieving safe autonomous landing requires high-fidelity semantic resolution to distinguish navigable terrain from hazardous obstacles, yet development is often hindered by the scarcity of annotated aerial datasets. This work proposes a comprehensive perception and data generation pipeline designed to bridge the sim-to-real gap for autonomous landing tasks. We introduce a procedural synthetic data engine that generates photorealistic urban environments with automated semantic annotations through domain randomization. A Transformer-based OneFormer architecture is fine-tuned exclusively on this synthetic data, leveraging multi-head self-attention mechanisms for global context resolution. To ensure operational safety, a deterministic landing module utilizes a Euclidean Distance Transform (EDT) and dynamic inference logic to identify the largest inscribed safe landing zones while maintaining strict clearance buffers around obstacles. Quantitative benchmarking against the UAVid dataset demonstrates robust semantic segmentation performance, while qualitative validation on real-world UAV footage confirms the system's ability to identify collision-free landing sites in unseen environments. Our results highlight the potential of high-fidelity procedural simulation to eliminate the need for manual annotation while providing robust, edge-deployable situational awareness for autonomous UAV recovery.

Chinese Translation

随着无人机（UAV）向更高水平的自主性过渡，在非合作性、非结构化环境中进行无辅助恢复的能力变得至关重要。实现安全的自主着陆需要高保真度的语义分辨率，以区分可导航地形与危险障碍物，然而，开发过程常常受到标注航空数据集稀缺的制约。本研究提出了一种全面的感知和数据生成管道，旨在弥合自主着陆任务的模拟与现实之间的差距。我们引入了一种程序化合成数据引擎，通过领域随机化生成具有自动语义标注的逼真城市环境。基于Transformer的OneFormer架构专门在这些合成数据上进行微调，利用多头自注意力机制进行全局上下文解析。为了确保操作安全，一个确定性的着陆模块利用欧几里得距离变换（EDT）和动态推理逻辑来识别最大的内切安全着陆区，同时在障碍物周围保持严格的安全缓冲区。与UAVid数据集的定量基准测试显示出强大的语义分割性能，而在真实世界无人机视频上的定性验证则确认了该系统在未见环境中识别无碰撞着陆地点的能力。我们的结果突显了高保真程序化仿真的潜力，能够消除手动标注的需求，同时为自主无人机恢复提供强大的、边缘可部署的态势感知。

View on arXiv Download PDF AI Translation

cs.RO / 3 / 2606.14776

Deep Learning-Based Lunar Crater Terrain Relative Navigation

基于深度学习的月球陨石坑地形相对导航

Candan, Batu, Servadio, Simone

Abstract

Accurate position estimation is crucial for the successful implementation of future lunar landings using autonomous vehicles, especially in dangerous environments with sparse terrain features. In this paper, we propose a terrain relative navigation (TRN) algorithm combining our deep-learning crater detector, which was designed specifically for the NASA Crater Detection Challenge problem, and an Extended Kalman Filter (EKF). Our detector analyzes crater features from the monocular images acquired from orbit, and their matches with craters from a global database are identified via a Hungarian assignment approach followed by the consensus-based outliers removal method. The estimated measurements are then used to refine an EKF, where spacecraft pose estimation in the Lunar-Centered Lunar-Fixed (LCLF) frame of reference, augmented with altitude aiding information, constrains radial drift. The simulation results indicate that even if the spacecraft is off from its actual location up to 5 km, TRN could recover from this situation, achieving navigation error reduction to a few hundred meters. It should be noted that in order to maintain crater feature correspondences, it is important to match the image resolution and the scales within the scene to the detector training set distribution.

Chinese Translation

准确的位置估计对于未来使用自主车辆实施月球着陆至关重要，尤其是在特征稀疏的危险环境中。在本文中，我们提出了一种地形相对导航（TRN）算法，该算法结合了我们专门为NASA陨石坑检测挑战问题设计的深度学习陨石坑检测器和扩展卡尔曼滤波器（EKF）。我们的检测器分析从轨道获取的单目图像中的陨石坑特征，并通过匈牙利分配方法识别与全球数据库中的陨石坑的匹配，随后采用基于共识的异常值去除方法。然后，估计的测量值用于优化EKF，其中在以月球为中心的月球固定（LCLF）参考框架中进行航天器姿态估计，并结合高度辅助信息以约束径向漂移。仿真结果表明，即使航天器与其实际位置相差达5公里，TRN仍能从这种情况中恢复，将导航误差降低到几百米。需要注意的是，为了保持陨石坑特征的对应关系，匹配图像分辨率和场景内的尺度与检测器训练集分布是非常重要的。

View on arXiv Download PDF AI Translation

cs.RO / 4 / 2606.14794

Computing Smooth Geodesics under Two-Sided Curvature Bounds with Applications to Robotics and Image Analysis

在双侧曲率约束下计算光滑测地线及其在机器人技术和图像分析中的应用

Chen, Da, Li, Zhenjiang, Mirebeau, Jean-Marie, Tai, Xuecheng, Zhang, Jinglin, Zhang, Wei, Cohen, Laurent D.

Abstract

Curvature of planar curves serves as a key regularization term for computing second-order minimal paths, due to its tight relevance to desirable geometric properties such as smoothness, rigidity, and elasticity. In this paper, we tackle a more challenging problem in computational physics and geometry problem: tracking minimal paths whose curvature is constrained by arbitrary upper and lower bounds. For that purpose, we propose a new curvature-bounded geodesic model, developed under the Hamilton-Jacobi-Bellman (HJB) partial differential equation (PDE) framework. It provides strong geometric control over minimal paths by enforcing curvature range constraints, whose paths are smooth and of bounded curvature limitation. We also present a discretization scheme for the Hamiltonian and the HJB PDE incorporating curvature bounds, allowing efficient solver for estimating numerical solutions to the model. Finally, we illustrate the capability of the proposed curvature-bounded geodesic model in applications of robot path planning and curvilinear structures tracking from images. Numerical experiments demonstrate that the proposed curvature-bounded geodesic model serves as a powerful and robust tool for finding satisfactory paths.

Chinese Translation

平面曲线的曲率作为计算二阶最小路径的关键正则化项，与光滑性、刚性和弹性等理想几何特性密切相关。本文针对计算物理和几何中的一个更具挑战性的问题：追踪其曲率受任意上下界约束的最小路径。为此，我们提出了一种新的曲率约束测地线模型，该模型在哈密顿-雅可比-贝尔曼（Hamilton-Jacobi-Bellman, HJB）偏微分方程（PDE）框架下发展而成。通过强制曲率范围约束，该模型对最小路径提供了强大的几何控制，确保路径光滑且曲率受到限制。我们还提出了一种包含曲率约束的哈密顿量和HJB PDE的离散化方案，从而实现高效求解器以估计模型的数值解。最后，我们展示了所提曲率约束测地线模型在机器人路径规划和从图像中追踪曲线结构中的应用能力。数值实验表明，所提曲率约束测地线模型是寻找满意路径的强大且稳健的工具。

View on arXiv Download PDF AI Translation

cs.RO / 5 / 2606.14862

TacStyle: Personalizing Tactile Robot Policies using Structured Behavior Representations

TacStyle：使用结构化行为表示个性化触觉机器人策略

Robledo, Kevin, Galaz, Matías I. Torres, Rai, Kumar Dixhant, Ulman, Shelly Sara, Tasrin, Tasmia, Nemlekar, Heramb

Abstract

Robotic systems that assist humans should be capable of adapting their behaviors to individual user preferences. For instance, users may want a robot arm to adjust the amount of force it applies while folding their laundry or cleaning furniture. Natural language provides an intuitive way for humans to communicate such preferences. Recent progress in language-conditioned robot policies has shown that robots can successfully use language prompts to determine what task to perform. However, extending the same approach to realize how the task should be performed requires detailed labels describing the preferences or styles of trajectories in the task data. Not only is collecting such annotations challenging, but conditioning directly on these labels may also fail to provide fine-grained control over a continuous range of behaviors. For example, it can be difficult to convey the exact force that a robot must apply through abstract instructions like "apply a bit more pressure than before". Therefore, in this work, we propose using language to reason over preferred behaviors instead of directly generating them. We first learn a structured latent representation that organizes user preferences according to differences in the corresponding trajectories. Then, given a preference prompt, we use a foundation model to interpret this latent space and choose a value that produces the desired behavior. Through both simulation and real-world experiments, we show that selecting robot behaviors from an intuitively structured latent space enables more precise adaptation to user preferences while requiring significantly fewer preference labels than language-conditioned policies.

Chinese Translation

辅助人类的机器人系统应能够根据个体用户的偏好调整其行为。例如，用户可能希望机器人手臂在折叠衣物或清洁家具时调整施加的力量。自然语言为人类传达此类偏好提供了一种直观的方式。最近在语言条件下的机器人策略方面取得的进展表明，机器人可以成功地使用语言提示来确定执行的任务。然而，将相同的方法扩展到如何执行任务的实现需要详细的标签来描述任务数据中轨迹的偏好或风格。收集此类注释不仅具有挑战性，而且直接基于这些标签进行条件设置可能也无法提供对连续行为范围的细粒度控制。例如，通过“施加比之前更多一点压力”等抽象指令来传达机器人必须施加的确切力量可能很困难。因此，在本研究中，我们提出使用语言推理偏好行为，而不是直接生成它们。我们首先学习一种结构化的潜在表示，根据相应轨迹的差异组织用户偏好。然后，在给定偏好提示的情况下，我们使用基础模型来解释该潜在空间，并选择一个值以产生所需的行为。通过模拟和真实世界实验，我们表明，从直观结构化的潜在空间中选择机器人行为能够更精确地适应用户偏好，同时所需的偏好标签显著少于语言条件下的策略。

View on arXiv Download PDF AI Translation

cs.RO / 6 / 2606.14879

VANDERER: Map-Free Exploration using Future-Aware and Visual-Curiosity-Guided Diffusion Policy

VANDERER：基于未来感知和视觉好奇心引导的扩散策略的无地图探索

Devarakonda, Venkata Naren, Goswami, Raktim Gautam, Krishnamurthy, Prashanth, Khorrami, Farshad

Abstract

Mobile agents require efficient exploration strategies to map unseen environments and autonomously plan tasks. Traditional methods rely on generating occupancy maps and optimizing the sequence in which unexplored regions are visited. However, in sensor-constrained settings, such as those limited to monocular cameras, generating accurate occupancy maps is challenging. To address this, we propose VANDERER, an exploration framework that leverages a Visual Curiosity Module (VCM) to guide pre-trained diffusion policies using only monocular image data. This curiosity module predicts the outcomes of proposed actions via a navigation world model and evaluates them through a curiosity cost. The cost then guides the diffusion process toward generating actions that maximize exploration. Evaluated across diverse simulated environments, VANDERER consistently outperforms established baselines, exploring an average of 13.4% more area than NoMaD. Our results reveal a direct correlation between visual and geometric curiosity in outdoor environments, demonstrating that VANDERER can effectively leverage this relationship for efficient exploration using sensor-constrained agents.

Chinese Translation

移动代理需要高效的探索策略，以绘制未见环境的地图并自主规划任务。传统方法依赖于生成占用地图并优化未探索区域的访问顺序。然而，在传感器受限的环境中，例如仅限于单目相机的设置，生成准确的占用地图是具有挑战性的。为了解决这个问题，我们提出了VANDERER，一个探索框架，利用视觉好奇心模块（Visual Curiosity Module, VCM）仅通过单目图像数据引导预训练的扩散策略。该好奇心模块通过导航世界模型预测提议动作的结果，并通过好奇心成本进行评估。然后，该成本引导扩散过程生成最大化探索的动作。在多样化的模拟环境中评估后，VANDERER始终优于既定基线，平均探索面积比NoMaD多出13.4%。我们的结果揭示了户外环境中视觉好奇心与几何好奇心之间的直接相关性，表明VANDERER能够有效利用这种关系，以实现传感器受限代理的高效探索。

View on arXiv Download PDF AI Translation

cs.RO / 7 / 2606.14882

DynaHMRC: Decentralized Heterogeneous Multi-Robot Collaboration for Dynamic Tasks with Large Language Models

DynaHMRC：基于大型语言模型的动态任务去中心化异构多机器人协作

Yu, Wenhao, Xie, Yu'ang, Duan, Yifan, Peng, Jie, Ye, Guanting, Yuen, Ka-Veng, Zhang, Yanyong, Ji, Jianmin

Abstract

Large language models (LLMs) provide robots with richer task understanding and adaptability, making them promising for coordinating heterogeneous multi-robot systems in long-horizon tasks. Despite this potential, several challenges remain underexplored: (1) Centralized LLM schedulers scale poorly as team size and environmental complexity increase. A single model must process excessive contextual information, and long-context approximation may degrade reasoning quality; (2) Existing task formulations insufficiently consider dynamic settings, while robust adaptation to evolving task conditions is essential for real-world deployment; (3) Domain-specific data scarcity limits specialized robotic reasoning, making proprietary general-purpose models inefficient for expert tasks. To address these limitations, we propose DynaHMRC, a decentralized framework in which each robot acts as a role-aware LLM agent. This design mitigates the single-model context bottleneck and supports flexible collaboration across heterogeneous team configurations. DynaHMRC organizes collaboration as a four-stage closed-loop process: self-description, task allocation with leadership bidding, leader election, and reflective execution, supported by executable robot interfaces. We further develop a benchmark covering three task families, four dynamic variations, and six team configurations to systematically study dynamic task modeling. In addition, we conduct an empirical analysis to guide the construction of domain-specific expert datasets and fine-tune pretrained LLMs to improve specialized competence. Experiments show that DynaHMRC achieves higher success rates than strong baselines with fewer action and communication steps, while demonstrating promising scalability trends as team size grows within the evaluated settings.

Chinese Translation

大型语言模型（LLMs）为机器人提供了更丰富的任务理解和适应能力，使其在长期任务中协调异构多机器人系统的潜力显著。尽管如此，仍然存在一些未被充分探讨的挑战：（1）随着团队规模和环境复杂性的增加，集中式LLM调度器的扩展性较差。单一模型必须处理过多的上下文信息，长上下文近似可能会降低推理质量；（2）现有的任务表述对动态环境的考虑不足，而对不断变化的任务条件进行稳健适应对于实际部署至关重要；（3）领域特定数据的稀缺限制了专业机器人推理，使得专有通用模型在专家任务中效率低下。为了解决这些限制，我们提出了DynaHMRC，一个去中心化框架，其中每个机器人充当角色感知的LLM代理。该设计缓解了单模型上下文瓶颈，并支持在异构团队配置中灵活协作。DynaHMRC将协作组织为一个四阶段闭环过程：自我描述、通过领导权竞标进行任务分配、领导者选举和反思执行，支持可执行的机器人接口。我们进一步开发了一个基准，涵盖三个任务家族、四种动态变体和六种团队配置，以系统地研究动态任务建模。此外，我们进行了一项实证分析，以指导领域特定专家数据集的构建，并微调预训练的LLMs以提高专业能力。实验表明，DynaHMRC在较少的行动和通信步骤下实现了比强基线更高的成功率，并在评估设置中随着团队规模的增长展现出良好的可扩展性趋势。

View on arXiv Download PDF AI Translation

cs.RO / 8 / 2606.14969

Multimodal Physiological Assessment of Contact-Rich Physical Human-Robot Interaction Under Varying Environmental Conditions

在不同环境条件下的接触丰富型人机交互的多模态生理评估

Chen, Yanyi, Wang, Xi, Deng, Min

Abstract

Physical human-robot interaction (pHRI) in real-world settings exposes operators to fluctuating environmental conditions during contact-rich tasks. Traditional task-centric evaluations overlook the physiological burdens imposed by these stressors. Therefore, we conducted a multimodal empirical study involving contact-rich tracing tasks under 18 distinct combinations of temperature, acoustic noise, and illuminance. Synchronously, we recorded electrodermal activity (EDA), surface electromyography (sEMG), eye-tracking data, and subjective environmental comfort ratings. Evaluating these physiological signals alongside execution data revealed hidden physiological costs not captured by objective performance. The results revealed that task performance remained stable across all environmental conditions. Autonomic workload, indexed by tonic skin conductance level (SCL), increased with temperature, while physical and cognitive workload were unaffected. Perceived environmental comfort showed no significant association with tracing error or completion time. These findings reveal a compensatory mechanism where operators maintain consistent performance by increasing their physiological effort to suppress thermal discomfort. Such insight motivates the development of physiology-aware control architectures that leverage real-time physiological metrics to reduce operator workload in unstructured environments.

Chinese Translation

在现实环境中，物理人机交互（pHRI）使操作员在接触丰富的任务中暴露于波动的环境条件下。传统的以任务为中心的评估忽视了这些压力源带来的生理负担。因此，我们进行了一个多模态实证研究，涉及在18种不同的温度、声学噪声和光照组合下的接触丰富型追踪任务。我们同步记录了皮肤电活动（EDA）、表面肌电图（sEMG）、眼动追踪数据以及主观环境舒适度评分。对这些生理信号与执行数据的评估揭示了客观表现未能捕捉到的隐性生理成本。结果显示，任务表现保持在所有环境条件下的稳定。以皮肤电导水平（SCL）为指标的自主工作负荷随温度的升高而增加，而身体和认知工作负荷则未受影响。感知的环境舒适度与追踪误差或完成时间没有显著关联。这些发现揭示了一种补偿机制，即操作员通过增加生理努力来抑制热不适，从而维持一致的表现。这一洞察促使我们开发生理感知控制架构，利用实时生理指标来减少操作员在非结构化环境中的工作负荷。

View on arXiv Download PDF AI Translation

cs.RO / 9 / 2606.14981

Inference-time Policy Steering via Vision and Touch

基于视觉和触觉的推理时策略引导

Wu, Yilin, Si, Zilin, Temel, Zeynep, Kroemer, Oliver, Bajcsy, Andrea

Abstract

Inference-time steering adapts pre-trained generative robot policies during deployment by verifying candidate actions before execution. While prior methods typically perform this verification only with visual observations, vision alone is often insufficient for contact-rich manipulation, where success depends on both global task progress and subtle local interactions such as contact force. We introduce ViTaL, a visuo-tactile inference-time steering framework that formulates multimodal guidance as a bi-level optimization problem. At the high level, visual sampling-and-verification performs long-horizon mode selection, deciding what behavior the robot should execute. At the low level, tactile-guided diffusion editing refines the selected action sequence over a shorter horizon to satisfy local contact requirements. To support outcome-based steering, ViTaL learns a visuo-tactile latent world model and employs semantically aligned visual and tactile verifiers, including a novel text-conditioned tactile reward that scores predicted tactile futures directly in latent space. Across three real-world contact-rich manipulation tasks, ViTaL improves overall success by 51% over the base policy, outperforms unimodal steering by at least 33%, and exceeds naive multimodal fusion by at least 20%. Website: https://yilin-wu98.github.io/vital_website.

Chinese Translation

推理时引导通过在执行前验证候选动作，适应预训练的生成机器人策略。在部署过程中，尽管先前的方法通常仅依赖视觉观察进行验证，但仅凭视觉在接触丰富的操作中往往不足，因为成功依赖于全球任务进展和细微的局部交互，例如接触力。我们提出了ViTaL，一个视觉-触觉推理时引导框架，将多模态引导形式化为一个双层优化问题。在高层次上，视觉采样与验证执行长时间模式选择，决定机器人应执行的行为。在低层次上，触觉引导的扩散编辑在较短的时间范围内细化所选的动作序列，以满足局部接触要求。为了支持基于结果的引导，ViTaL学习了一个视觉-触觉潜在世界模型，并采用语义对齐的视觉和触觉验证器，包括一种新颖的文本条件触觉奖励，直接在潜在空间中对预测的触觉未来进行评分。在三个真实世界的接触丰富操作任务中，ViTaL相较于基础策略提高了51%的整体成功率，至少比单模态引导提高了33%，并且超越了简单多模态融合至少20%。网站：https://yilin-wu98.github.io/vital_website.

View on arXiv Download PDF AI Translation

cs.RO / 10 / 2606.15010

LV-Calib: LiDAR-Camera Extrinsic Calibration with Boundary-Response Modeling

LV-Calib：基于边界响应建模的激光雷达-相机外部标定

Hong, Sheng

Abstract

We present LV-Calib, a calibration framework for LiDAR-camera extrinsic estimation and LiDAR boundary-response calibration using a printable planar target. The target serves as a shared observation carrier: visual fiducials provide indexed image measurements, while circular reflectivity boundaries provide LiDAR-observable structural feature points. Instead of directly fitting boundary points as ideal geometric contours, LV-Calib automatically crops background points, estimates the target plane, and iteratively refines accurate LiDAR-side 3-D feature points from intensity and geometric constraints. The refinement explicitly handles the broadened and distorted transition band induced by finite beam footprint and mixed-intensity returns around black-white reflectivity discontinuities. Given these refined LiDAR features, we formulate a weighted reprojection-consistent extrinsic optimization with LiDAR feature alignment, where image observations are kept in the reprojection domain and LiDAR feature residuals are weighted by refinement confidence. Finally, using the estimated extrinsic and the extracted transition band, LV-Calib calibrates the LiDAR boundary response by estimating pitch-yaw-range residual statistics of boundary-overlap samples. Experiments on printed-board calibration data demonstrate sub-pixel reprojection accuracy, millimeter-level LiDAR feature consistency, and improved odometry performance. Code and calibration data will be released for reproducible evaluation.

Chinese Translation

我们提出了LV-Calib，这是一个用于激光雷达-相机外部估计和激光雷达边界响应标定的标定框架，采用可打印的平面目标。该目标作为共享观测载体：视觉基准提供索引图像测量，而圆形反射边界提供激光雷达可观测的结构特征点。LV-Calib并不是直接将边界点拟合为理想的几何轮廓，而是自动裁剪背景点，估计目标平面，并从强度和几何约束中迭代精炼出准确的激光雷达侧三维特征点。该精炼过程明确处理了有限光束足迹和黑白反射率不连续性周围混合强度回波所引起的扩展和扭曲的过渡带。基于这些精炼后的激光雷达特征，我们制定了一个加权重投影一致的外部优化方案，结合激光雷达特征对齐，其中图像观测保持在重投影域内，激光雷达特征残差则根据精炼置信度加权。最后，利用估计的外部参数和提取的过渡带，LV-Calib通过估计边界重叠样本的俯仰-偏航-范围残差统计来标定激光雷达边界响应。在印刷电路板标定数据上的实验表明，重投影精度达到亚像素级，激光雷达特征一致性达到毫米级，并且里程计性能有所提升。代码和标定数据将被发布以便于可重复评估。

View on arXiv Download PDF AI Translation

cs.RO / 11 / 2606.15021

Steering Autoregressive Vision-Language-Action Policies via Action Token Intervention

通过动作令牌干预引导自回归视觉-语言-动作策略

Chan, Jason, Kao, Jonathan C.

Abstract

We present Token Steering (TS), a method for dynamically steering trajectories generated by an autoregressive vision-language-action (VLA) model through direct intervention in the action-token space. TS injects low-dimensional user inputs into the model's native action-token representation, allowing users to influence trajectory generation without modifying the underlying vision-language model (VLM) architecture. Because TS operates entirely at inference time, it requires no additional training or finetuning. User inputs guide rather than override the pretrained policy, allowing users to influence robot actions while preserving the dexterity, smoothness, and task priors learned by the VLA. We evaluate TS on two household manipulation tasks -- drawer closing after object placement and state-aware object swapping -- and improve success rates from 10.0% to 72.5% and from 16.7% to 93.8%, respectively. By enabling lightweight, intuitive steering over robot foundation models, our interface has the potential to improve human-robot interaction in consumer environments and broaden accessibility for individuals with limited physical control. Project website: https://jasontchan.github.io/token-steering/ .

Chinese Translation

我们提出了一种方法，称为令牌引导（Token Steering, TS），用于通过直接干预动作令牌空间动态引导由自回归视觉-语言-动作（VLA）模型生成的轨迹。TS将低维用户输入注入模型的原生动作令牌表示中，使用户能够在不修改基础视觉-语言模型（VLM）架构的情况下影响轨迹生成。由于TS完全在推理时操作，因此不需要额外的训练或微调。用户输入引导而不是覆盖预训练策略，使用户能够影响机器人动作，同时保留VLA所学习的灵活性、平滑性和任务先验。我们在两个家庭操作任务上评估了TS——物体放置后的抽屉关闭和状态感知物体交换，成功率分别从10.0%提高到72.5%和从16.7%提高到93.8%。通过实现对机器人基础模型的轻量级、直观的引导，我们的接口有潜力改善消费者环境中的人机交互，并为身体控制能力有限的个体拓宽可及性。项目网站：https://jasontchan.github.io/token-steering/

View on arXiv Download PDF AI Translation

cs.RO / 12 / 2606.15028

An Autonomous Subgram SMA-Based Swimmer

一种基于形状记忆合金的自主亚克朗游泳器

Trygstad, Conor K., Gonçalves, Francisco M. F. R., Pérez-Arancibia, Néstor O.

Abstract

We present the Swima, a bioinspired 900-mg swimmer propelled by two 10-mg high-work-density (HWD) actuators driven by shape-memory alloy (SMA) wires. We integrated onboard power and computation by using a custom-built printed circuit board (PCB) and an 11-mAh 3.7-V 507-mg single-cell lithium-ion (Li-Ion) battery, which in conjunction enable autonomous swimming in excess of 18 min. The Swima can swim at speeds of up to 22.4 mm/s (0.56 Bl/s), achieves turning rates of up to 14{\deg}/s, and can follow 0-degree heading reference trajectories with root mean square (RMS) values of tracking errors of about 6.5{\deg} across multiple tests. This robot is the first subgram microswimmer with onboard power, actuation, and computation developed to date.

Chinese Translation

我们提出了Swima，这是一种受生物启发的900毫克游泳器，由两个10毫克高功率密度（HWD）驱动器通过形状记忆合金（SMA）线驱动。我们通过使用定制的印刷电路板（PCB）和一块11毫安时3.7伏507毫克的单电池锂离子（Li-Ion）电池集成了机载电源和计算能力，这使得Swima能够实现超过18分钟的自主游泳。Swima的游泳速度可达22.4毫米/秒（0.56 Bl/s），转向速率可达14°/秒，并且在多次测试中能够以约6.5°的均方根（RMS）值跟踪0度航向参考轨迹。这款机器人是迄今为止首个具备机载电源、驱动和计算能力的亚克朗微型游泳器。

View on arXiv Download PDF AI Translation

cs.RO / 13 / 2606.15046

Exact, Efficient, and Safe Occlusion-Aware Planning Using AH-Polyhedrons

基于AH-多面体的精确、高效和安全的遮挡感知规划

Chung, Long Kiu, Isele, David, Mohammadnejad, Toktam, Tariq, Faizan M., Bae, Sangjae, Kousik, Shreyas, D'sa, Jovin

Abstract

Safely handling occlusions is a fundamental challenge for autonomous mobile robots operating in dynamic environments. This issue is especially prominent in autonomous valet parking (AVP), where traffic rules are lax, occlusions are frequent and cluttered, and overly conservative behavior can leave vehicles stuck. However, existing methods either lack formal safety guarantees, assume agents follow road structures, or introduce conservatism, leaving occlusion-aware planning for AVP an open challenge. In this paper, we propose APRO (AH-Polyhedron Reachability for Occlusions), an exact and efficient occlusion-aware planning framework based on game-theoretic active perception and AH-polyhedron reachability analysis with AVP as our canonical use case. Our key insight is to reformulate set-based safety conditions in prior work as unions of AH-polyhedrons, enabling exact safety verification through linear programming (LP) without any additional conservatism in set computations or assumptions on road topology. We further show how the resulting safety conditions can be integrated into optimization-based planners or a bisection search scheme for real-time applications. We validate our method in simulation and hardware experiments, including data replay on a real-world parking lot dataset. Experimental results demonstrate that our method consistently achieved a 100% safety rate across all evaluated scenarios while maintaining real-time performance, resulting in safer and more optimal decisions than existing methods with formal safety guarantees.

Chinese Translation

安全处理遮挡是自主移动机器人在动态环境中面临的一个基本挑战。这个问题在自主代客泊车（AVP）中尤为突出，因为交通规则宽松，遮挡情况频繁且杂乱，而过于保守的行为可能导致车辆被困。然而，现有的方法要么缺乏正式的安全保障，要么假设代理遵循道路结构，或者引入保守性，从而使得AVP的遮挡感知规划成为一个未解决的挑战。本文提出了APRO（AH-多面体遮挡可达性），这是一个基于博弈论主动感知和AH-多面体可达性分析的精确且高效的遮挡感知规划框架，以AVP作为我们的典型应用案例。我们的关键见解是将先前工作的基于集合的安全条件重新表述为AH-多面体的并集，从而通过线性规划（LP）实现精确的安全验证，而无需在集合计算中引入额外的保守性或对道路拓扑的假设。我们进一步展示了如何将得到的安全条件集成到基于优化的规划器或二分搜索方案中，以实现实时应用。我们在仿真和硬件实验中验证了我们的方法，包括在真实停车场数据集上的数据回放。实验结果表明，我们的方法在所有评估场景中始终实现了100%的安全率，同时保持实时性能，导致比现有具有正式安全保障的方法更安全、更优的决策。

View on arXiv Download PDF AI Translation

cs.RO / 14 / 2606.15068

Design and Fabrication of a Spin Coater with In-Situ Optical Measurement for Soft Thin Films

具有原位光学测量功能的旋涂机的设计与制造，用于软薄膜的制备

Gliksberg, Daniel, Qiu, Jiajie, Suzuki, Jun, Youcef-Toumi, Kamal

Abstract

Spin coating is widely used for fabrication of thin polymer and elastomer films, yet reliable thickness verification of highly compliant materials remains challenging due to deformation from contact-based measurements and the cost and complexity of conventional optical metrology. Accurate thickness control is especially critical in soft elastomer applications such as dielectric elastomer actuators (DEAs), where mechanical and functional performance scales strongly with film thickness. This work presents a low-cost, primarily 3D-printed benchtop spin coater with an integrated, minimally deforming optical thickness measurement system for soft-film fabrication workflows. The system is designed to manufacture films between 50 and 300 microns thick with repeatability within 10 microns. Thickness is measured in-situ by tracking displacement of a reflected laser beam via quadrant photodetector, avoiding significant deformation. Optical geometry, sensor linearity constraints, and structural validation via finite element analysis are discussed. Experimental validation using calibrated metal shims demonstrated a thickness resolution of 3.6-3.7 microns and best-case measurement repeatability of 13 microns (95 percent confidence interval). The platform repeatably produced silicone films within 9 microns of target thickness, demonstrating that accessible optical metrology can be integrated into a low-cost spin coating system for practical, thickness-controlled fabrication of compliant thin films without specialized industrial instrumentation.

Chinese Translation

旋涂技术广泛应用于薄聚合物和弹性体薄膜的制造，但由于接触式测量引起的变形以及传统光学计量的成本和复杂性，可靠的高柔性材料厚度验证仍然具有挑战性。在软弹性体应用中，如介电弹性体驱动器（DEAs），准确的厚度控制尤为关键，因为机械性能和功能性能与薄膜厚度密切相关。本研究提出了一种低成本、主要由3D打印制造的台式旋涂机，配备集成的、最小变形的光学厚度测量系统，适用于软薄膜的制造工作流程。该系统设计用于制造厚度在50到300微米之间的薄膜，重复性控制在10微米以内。通过跟踪反射激光束的位移，利用象限光电探测器进行原位厚度测量，避免了显著的变形。讨论了光学几何、传感器线性约束以及通过有限元分析进行的结构验证。使用经过校准的金属垫片进行的实验验证显示，厚度分辨率为3.6-3.7微米，最佳测量重复性为13微米（95%置信区间）。该平台重复性地生产出厚度在目标厚度9微米以内的硅胶薄膜，证明了可获取的光学计量可以集成到低成本的旋涂系统中，实现合规薄膜的实际厚度控制制造，而无需专业的工业仪器。

View on arXiv Download PDF AI Translation

cs.RO / 15 / 2606.15133

DragMesh-2: Physically Plausible Dexterous Hand-Object Interaction with Articulated Objects

DragMesh-2：具有关节物体的物理可信灵巧手-物体交互

Zhang, Tianshan, Duan, Yijia, Li, Yanjun, Zhang, Zeyu, Tang, Hao

Abstract

Dexterous interaction with articulated objects is important for household, assistive, and humanoid manipulation, where multi-finger hands can provide compliant contact patterns beyond parallel-jaw grasping. However, articulated-object manipulation differs from static-object manipulation: the target part cannot be directly actuated, and its motion must emerge through sustained physical hand--handle contact. This makes the transition from object-centric articulated generation to hand-driven dexterous hand--object interaction non-trivial, since geometric trajectory replay or open-loop execution does not model the contact dynamics required to move the articulated part. Moreover, policies trained only for task completion under fixed dynamics can overfit nominal contact loads, especially without tactile or force feedback, and may degrade when the contact load changes. To address these challenges, we present DragMesh-2, a contact-driven framework for dexterous interaction with articulated objects that extends articulated interaction from object-centric generation to hand-driven dexterous hand--object interaction, where articulated motion must arise through physical contact. We further propose PICA, a physically informed contact-aware training mechanism that injects physical signals into policy learning without tactile or force feedback, improving robustness and task success under changing contact loads. Finally, we conduct systematic evaluation across multiple damping conditions and articulated-object categories to study robustness under contact-load variation, and provide a pure-geometry dexterous interaction resource to support future loco-manipulation and humanoid hand--object interaction research. Across seven GAPartNet objects, DragMesh-2 achieves stronger robustness under contact-load variation than the compared methods while maintaining high task success across damping conditions.

Chinese Translation

与关节物体的灵巧交互对于家庭、辅助和类人操作至关重要，其中多指手可以提供超越平行夹持的顺应接触模式。然而，关节物体的操作与静态物体的操作有所不同：目标部分无法直接驱动，其运动必须通过持续的物理手-把手接触产生。这使得从以物体为中心的关节生成到以手驱动的灵巧手-物体交互的过渡变得复杂，因为几何轨迹重放或开环执行无法模拟移动关节部分所需的接触动力学。此外，仅针对固定动力学下的任务完成进行训练的策略可能会过拟合名义接触载荷，尤其是在没有触觉或力反馈的情况下，当接触载荷变化时可能会下降。为了解决这些挑战，我们提出了DragMesh-2，这是一个基于接触的灵巧交互框架，将关节交互从以物体为中心的生成扩展到以手驱动的灵巧手-物体交互，其中关节运动必须通过物理接触产生。我们进一步提出了PICA，一种物理信息驱动的接触感知训练机制，在没有触觉或力反馈的情况下将物理信号注入策略学习，提高了在变化接触载荷下的鲁棒性和任务成功率。最后，我们在多个阻尼条件和关节物体类别上进行了系统评估，以研究在接触载荷变化下的鲁棒性，并提供了一个纯几何灵巧交互资源，以支持未来的运动操控和类人手-物体交互研究。在七个GAPartNet物体上，DragMesh-2在接触载荷变化下表现出比比较方法更强的鲁棒性，同时在不同阻尼条件下保持高任务成功率。

View on arXiv Download PDF AI Translation

cs.RO / 16 / 2606.15148

MimicIK: Real-Time Generative Inverse Kinematics from Teleoperation with FK Consistency

MimicIK：基于遥操作的实时生成逆运动学与正运动学一致性

Yang, Jiahao, Yan, Shenhao, Feng, Fan, Yao, Chengsi, Wang, Ge, Mai, Zhixin, Zhao, Yiming, Han, Yatong

Abstract

Inverse kinematics (IK) remains a critical bottleneck for real-time robot manipulation. Classical numerical solvers achieve high geometric precision but often suffer from discontinuous branch switching and unstable behavior near kinematic singularities during closed-loop deployment. Meanwhile, learned IK approaches frequently struggle to balance spatial accuracy, motion smoothness, and real-time efficiency, particularly when trained on noisy human teleoperation data. We present \textbf{MimicIK}, a real-time generative inverse kinematics framework that learns smooth and robust joint-space motion priors from teleoperation demonstrations through conditional flow matching. Given the current joint configuration and a target end-effector pose, MimicIK predicts continuous delta-joint commands using an efficient two-step iterative refinement process based on a Minimal Iterative Policy (MIP) backbone. To enforce physical consistency, we further introduce an FK consistency loss, a differentiable forward-kinematics regularization that penalizes task-space deviations from the target pose during training. We evaluate MimicIK on a real-world 6-DOF robot dataset containing 8,848 teleoperation demonstrations. MimicIK achieves a mean position error of 4.65 mm, a 10 mm success rate of 92.01\%, and a trajectory spike rate of only 7.99\%. Compared with a UNet diffusion baseline, our method improves both spatial accuracy and motion smoothness while reducing inference latency from 21.66 ms to 6.74 ms. Furthermore, unlike deterministic MLP baselines that catastrophically diverge under out-of-distribution deployment, MimicIK remains stable near singular configurations and enables robust 20 Hz real-time control on deployment hardware.

Chinese Translation

逆运动学（IK）仍然是实时机器人操作的一个关键瓶颈。经典的数值求解器能够实现高几何精度，但在闭环部署过程中，常常面临不连续的分支切换和在运动学奇异点附近的不稳定行为。同时，学习型逆运动学方法在平衡空间精度、运动平滑性和实时效率方面常常面临挑战，尤其是在使用噪声较大的人工遥操作数据进行训练时。我们提出了 extbf{MimicIK}，一个实时生成逆运动学框架，通过条件流匹配从遥操作演示中学习平滑且鲁棒的关节空间运动先验。在给定当前关节配置和目标末端执行器姿态的情况下，MimicIK使用基于最小迭代策略（MIP）骨干网的高效两步迭代精炼过程预测连续的增量关节指令。为了确保物理一致性，我们进一步引入了正运动学一致性损失，这是一种可微的正运动学正则化方法，在训练过程中惩罚任务空间与目标姿态的偏差。我们在一个包含8,848个遥操作演示的真实世界6自由度机器人数据集上评估了MimicIK。MimicIK实现了4.65毫米的平均位置误差，92.01\%的10毫米成功率，以及仅为7.99\\%的轨迹尖峰率。与UNet扩散基线相比，我们的方法在提高空间精度和运动平滑性的同时，将推理延迟从21.66毫秒降低到6.74毫秒。此外，与在分布外部署下会灾难性发散的确定性多层感知器基线不同，MimicIK在奇异配置附近保持稳定，并能够在部署硬件上实现鲁棒的20 Hz实时控制。

View on arXiv Download PDF AI Translation

cs.RO / 17 / 2606.15154

Task-Aware Environment Augmentation for Reliable Navigation via Shielded Conditional Diffusion

基于任务感知的环境增强以实现可靠导航通过屏蔽条件扩散

Phoompho, Bharawee, Puthumanaillam, Gokul, Miao, Yan, Hernandez, Ruben, Bretl, Tim, Mitra, Sayan, Ornik, Melkior

Abstract

Reliable trajectory planning under partial observability depends not only on computing a feasible geometric path, but also on whether the robot receives informative observations while executing that trajectory. Existing approaches usually keep the environment fixed and adapt the robot through belief-space planning, active localization, or added sensing, often incurring costly uncertainty propagation and brittle behavior in observation-poor regions. We flip this perspective and address the largely open problem of \emph{task-aware environment augmentation}: given a mapped environment, a planned task trajectory, and a small budget of visual fiducial markers, where should the environment be augmented so that the planned trajectory can be executed reliably under uncertainty? Our key observation is that useful marker layouts are defined by the localization support they provide along the task trajectory: a small number of well-timed observations can be sufficient to prevent uncertainty from accumulating in regions where state-estimation error would otherwise compromise control. Building on this observation, we present \tbp{SCoDA}, $\textbf{S}$hielded $\textbf{Co}$nditional $\textbf{D}$iffusion for Environment $\textbf{A}$ugmentation. \tbp{SCoDA} learns a conditional distribution over high-performing fiducial layouts from data, using the environment, planned trajectory, disturbance context, and desired execution profile as conditioning. Its shielded sampler reasons over where along the planned execution pose corrections should occur, and steers this distribution toward task-relevant, finite-budget augmentations. Across simulated benchmarks and hardware deployments, we show that \tbp{SCoDA} improves trajectory execution reliability and completion time over strong baselines. Code, models and dataset available at: \hyperlink{scoda-diffusion.github.io}{https://scoda-diffusion.github.io/}

Chinese Translation

在部分可观测性下，可靠的轨迹规划不仅依赖于计算可行的几何路径，还取决于机器人在执行该轨迹时是否能够获得有用的观测。现有的方法通常保持环境固定，通过信念空间规划、主动定位或增加传感器来适应机器人，这往往导致昂贵的不确定性传播和在观测稀缺区域的脆弱行为。我们转变了这一视角，解决了一个尚未充分探讨的问题—— extit{任务感知环境增强}：在给定已映射环境、规划的任务轨迹和少量视觉基准标记的预算下，应该如何增强环境，以便在不确定性下可靠地执行规划的轨迹？我们的关键观察是，有用的标记布局由它们在任务轨迹上提供的定位支持定义：少量及时的观测足以防止在状态估计误差可能影响控制的区域内不确定性积累。基于这一观察，我们提出了 bp{SCoDA}，即环境增强的$ extbf{S}$hielded $ extbf{Co}$nditional $ extbf{D}$iffusion。 bp{SCoDA}从数据中学习高性能基准布局的条件分布，使用环境、规划轨迹、干扰上下文和期望执行特征作为条件。其屏蔽采样器推理规划执行姿态修正应发生的位置，并将该分布引导至与任务相关的有限预算增强。在模拟基准和硬件部署中，我们展示了 bp{SCoDA}在轨迹执行的可靠性和完成时间上优于强基线。代码、模型和数据集可在： exttt{https://scoda-diffusion.github.io/}获取。

View on arXiv Download PDF AI Translation

cs.RO / 18 / 2606.15171

Seam-to-Graph Reconstruction for Garment Configuration Alignment

缝合线到图形重建用于服装配置对齐

Huang, Xuzhao, Tang, Kai, Tokuda, Fuyuki, Tien, Norman C., Kosuge, Kazuhiro

Abstract

Seams encode rich structural information about garments but are frequently partially observable in robotic manipulation scenarios. To robustly leverage seam information, we propose a Seam-to-Graph network based on graph neural networks and attention mechanisms. This network maps unstructured seam observations to a topology-encoded structural skeleton graph for real-time garment state estimation. Using this skeleton-graph-based state estimation, we design a deformation-aware, hierarchical visual servoing controller for garment configuration alignment. We implement this controller on a bimanual robot system to load a garment onto a screen printing platen and to align it to the desired configuration precisely. Real-robot experiments demonstrate that the robot using the proposed method not only achieves human-level alignment accuracy with reduced variance in alignment error but is also robust to different garments. These results demonstrate that the use of seam information is effective for garment manipulation.

Chinese Translation

缝合线编码了关于服装的丰富结构信息，但在机器人操作场景中常常部分可观察。为了稳健地利用缝合线信息，我们提出了一种基于图神经网络和注意力机制的缝合线到图形网络（Seam-to-Graph network）。该网络将非结构化的缝合线观察映射到一个拓扑编码的结构骨架图，以实现实时服装状态估计。利用这种基于骨架图的状态估计，我们设计了一种考虑变形的分层视觉伺服控制器，用于服装配置对齐。我们在一个双手机器人系统上实现了该控制器，以将服装加载到丝网印刷平台上，并精确对齐到所需配置。真实机器人实验表明，使用所提方法的机器人不仅实现了与人类相当的对齐精度，并且对齐误差的方差减少，同时对不同服装具有良好的鲁棒性。这些结果表明，利用缝合线信息对于服装操作是有效的。

View on arXiv Download PDF AI Translation

cs.RO / 19 / 2606.15232

Rethinking Implicit Spatial Representation in Visuomotor Policy Learning

重新思考视觉运动策略学习中的隐式空间表示

Chen, Xiangyu, Hu, Yuxuan, Zhou, Chuhao, Yang, Jianfei

Abstract

Generative model-based imitation learning has become a widely adopted paradigm for robotic manipulation, where policy performance depends critically on the conditioned visual representations. Although spatial softmax-based representations have been adopted in prior visuomotor policies, their effectiveness and underlying mechanisms remain insufficiently understood. This work rethinks the use of spatial softmax pooling: do such implicit spatial representations provide effective and stable visual features for robotic manipulation? Through systematic studies of different pooling methods in visual encoders, we find that this pooling operation produces compact and stable spatial representations, which outperform feature-value representations, despite using substantially fewer dimensions. Complementary saliency analysis further suggests that these spatial representations guide the encoder to focus more consistently on task-relevant regions. However, this advantage is limited by a representation bottleneck in current visual encoders: repeated downsampling operations weaken fine-grained spatial information before the action-generation module can use it, especially under low-resolution observations. Motivated by these findings, we propose PRISM, a visual encoder that preserves multiscale implicit spatial information through top-down cross-attention fusion. Experiments across multiple tasks and policy backbones show consistent improvements. In particular, on the low-resolution, high-precision ToolHang task, PRISM shows clear gains, improving the average success rate from 5.0% to 13.4% while increasing parameters by only 15.4%. These results support the use of multiscale implicit spatial representations as an effective and efficient design principle for robotic manipulation.

Chinese Translation

基于生成模型的模仿学习已成为机器人操控的广泛采用范式，其中策略性能在很大程度上依赖于条件视觉表示。尽管在先前的视觉运动策略中采用了基于空间softmax的表示，但其有效性和潜在机制仍然未得到充分理解。本研究重新思考空间softmax池化的使用：这种隐式空间表示是否为机器人操控提供了有效且稳定的视觉特征？通过对视觉编码器中不同池化方法的系统研究，我们发现这种池化操作生成了紧凑且稳定的空间表示，尽管使用的维度显著较少，但其性能优于特征值表示。补充的显著性分析进一步表明，这些空间表示引导编码器更一致地关注与任务相关的区域。然而，这一优势受到当前视觉编码器中表示瓶颈的限制：重复的下采样操作在动作生成模块使用之前削弱了细粒度的空间信息，尤其是在低分辨率观察下。基于这些发现，我们提出了PRISM，一种通过自上而下的交叉注意力融合来保留多尺度隐式空间信息的视觉编码器。在多个任务和策略骨干网络上的实验显示出一致的改进。特别是在低分辨率、高精度的ToolHang任务中，PRISM显示出明显的提升，将平均成功率从5.0%提高到13.4%，同时仅增加了15.4%的参数。这些结果支持将多尺度隐式空间表示作为机器人操控的有效且高效的设计原则。

View on arXiv Download PDF AI Translation

cs.RO / 20 / 2606.15239

Co-Creating Buildable and Open Social Robot Study Companions with University Students

与大学生共同创造可构建的开放式社交机器人学习伴侣

Baksh, Farnaz, Zorec, Matevž B., Baksh, Feiazie, Kruusamäe, Karl

Abstract

Open-source social robots offer accessibility, repairability, and student empowerment, yet the build itself often presents a barrier. Existing platforms either ship pre-assembled, foreclosing hands-on learning, or expose students to unfamiliar fasteners, opaque wiring, and inaccessible service points that erode engagement. Whether targeted mechanical redesign can lower this barrier whilst maintaining structural integrity remains untested. Here we show that Design for Assembly (DfA) and Design for Disassembly (DfD) interventions reshape how a build feels before they shorten how long it takes. Working with university students in Guyana and Estonia, we applied the Double Diamond framework to co-create the Robot Study Companion (RSC) v4.1: mapping pain points, then redesigning its chassis around twist-lock fasteners, snap-fit joints, and tool-free service latches. Across two studies with developers and first-time builders, system usability climbed from Poor to Excellent (SUS 59.4 to 89.4), perceived workload trended downward (NASA-TLX 4.29 to 4.00), and mean assembly time trended downward (21.4 to 13.7 minutes, with juniors' learning effect), whilst orientation cues and navigation continuity for first-time builders emerged as the next documentation frontier. Perceived workload, not completion time, appears to govern whether students take up open hardware.

Chinese Translation

开源社交机器人提供了可获取性、可修复性和学生赋权，然而构建过程本身往往成为障碍。现有平台要么以预组装的形式发货，剥夺了动手学习的机会，要么让学生接触到不熟悉的紧固件、不透明的布线和难以接触的服务点，从而削弱了参与感。针对机械设计的改进是否能够降低这一障碍，同时保持结构完整性，尚未得到验证。在此，我们展示了装配设计（Design for Assembly, DfA）和拆解设计（Design for Disassembly, DfD）干预措施如何重塑构建过程的体验，同时缩短所需时间。我们与圭亚那和爱沙尼亚的大学生合作，应用双钻石框架共同创造了机器人学习伴侣（Robot Study Companion, RSC）v4.1：首先映射痛点，然后围绕旋锁紧固件、卡扣连接和免工具服务锁重新设计其底盘。在与开发者和首次构建者进行的两项研究中，系统可用性从差（SUS 59.4）提升至优（SUS 89.4），感知工作负荷呈下降趋势（NASA-TLX 4.29至4.00），平均组装时间也呈下降趋势（21.4分钟至13.7分钟，包含大三学生的学习效应），而首次构建者的方向提示和导航连续性则成为下一个文档前沿。感知工作负荷，而非完成时间，似乎主导着学生是否选择开放硬件。

View on arXiv Download PDF AI Translation

cs.RO / 21 / 2606.15251

Driving, Fast or Slow? Neuro-Symbolic Guidance for Motion Prediction in Multi-Modal Ground Mobility

驾驶，快还是慢？用于多模态地面移动的神经符号引导运动预测

Kohaut, Simon, Divo, Felix, Hahnewald, Julius, Flade, Benedict, Eggert, Julian, Kersting, Kristian, Dhami, Devendra Singh

Abstract

Accurate and interpretable motion prediction for heterogeneous traffic spaces, including pedestrians, bicycles, cars, and trucks, is essential for safe autonomous navigation. Nevertheless, state-of-the-art approaches remain predominantly black-box, lacking explicit encoding of the regulatory and behavioral constraints of real-world mobility. We propose Trajectory Compliance-Shaping (TraCS), a neuro-symbolic framework that augments existing black-box motion prediction backbones with interpretable and probabilistic first-order logic. To do so, TraCS employs an agentic code-generation pipeline to bridge the gap between natural-language descriptions of traffic regulations and probabilistic motion prediction. Furthermore, TraCS employs a reactive data-streaming inference engine that maintains and efficiently updates compliance landscapes as scenes evolve. To prevent TraCS from overconfidently steering the backbone's predictions in the wrong direction, we propose a neural confidence rating learned as a context-aware attenuation of the compliance signal. We demonstrate on the Argoverse 2 benchmark how TraCS consistently improves state-of-the-art prediction backbones, showing that probabilistic and symbolic compliance reasoning is a broadly applicable and computationally efficient complement to purely neural motion predictors.

Chinese Translation

对于包括行人、自行车、汽车和卡车在内的异构交通空间，准确且可解释的运动预测对于安全的自主导航至关重要。然而，现有的最先进方法仍然主要是黑箱模型，缺乏对现实世界移动性调节和行为约束的明确编码。我们提出了轨迹合规性塑造（Trajectory Compliance-Shaping, TraCS），这是一个神经符号框架，旨在通过可解释的概率性一阶逻辑增强现有黑箱运动预测模型。为此，TraCS采用了一种代理代码生成管道，以弥合交通法规的自然语言描述与概率运动预测之间的差距。此外，TraCS还采用了一种反应式数据流推理引擎，能够在场景演变时维护和高效更新合规性景观。为了防止TraCS过于自信地将模型的预测引导向错误的方向，我们提出了一种神经置信度评分，作为对合规信号的上下文感知衰减。我们在Argoverse 2基准上展示了TraCS如何持续改进最先进的预测模型，表明概率性和符号性合规推理是对纯神经运动预测器的广泛适用且计算高效的补充。

View on arXiv Download PDF AI Translation

cs.RO / 22 / 2606.15255

OSDAG: Online Scheduling for Efficient Multi-Robot Collaboration

OSDAG：高效多机器人协作的在线调度

Canh, Thanh Nguyen, Viet, Thang Tran, Van Dinh, Phuc, HoangVan, Xiem, Chong, Nak Young

Abstract

Coordinating heterogeneous multi-robot systems (MRS) for complex, long-horizon tasks requires both flexible high-level reasoning and efficient low-level scheduling. Existing LLM-based approaches address the reasoning side but introduce two critical bottlenecks: (1) repeated LLM inference during execution, which inflates latency with agent count, and (2) offline, pre-committed scheduling, which forces robots to idle while waiting for sequentially ordered predecessors even when independent work is available. This paper presents OSDAG, a novel framework that integrates LLM-based task reasoning with Directed Acyclic Graph (DAG) representation and constraint-aware online scheduling. The LLM is invoked once to decompose a natural-language instruction into a dependency-annotated task graph, and a lightweight online scheduler then allocates ready tasks to idle agents in real time. The DAG representation encodes both precedence and resource constraints, ensuring correctness while exposing all available parallelism. Experiments across five benchmark scenarios demonstrate that OSDAG achieves 5-15x faster reasoning time compared to dialogue-based methods, reduces makespan by up to 38% over sequential baselines, and maintains competitive success rates. Both simulation and real-world experiments on dual-arm manipulation tasks validate the effectiveness and practicality of the proposed approach for efficient multi-robot coordination. The website and resources are available at http://thanhnguyencanh.github.io/LLM_DAG4MultiRobot

Chinese Translation

协调异构多机器人系统（MRS）以完成复杂的长期任务需要灵活的高层推理和高效的低层调度。现有的基于大语言模型（LLM）的方法解决了推理方面的问题，但引入了两个关键瓶颈：（1）在执行过程中重复进行LLM推理，导致延迟随着代理数量的增加而增加；（2）离线的、预先承诺的调度，迫使机器人在等待按顺序排列的前驱时处于闲置状态，即使有独立的工作可用。本文提出了OSDAG，一个新颖的框架，将基于LLM的任务推理与有向无环图（DAG）表示和约束感知的在线调度相结合。LLM被调用一次，将自然语言指令分解为带有依赖关系注释的任务图，然后轻量级的在线调度器实时将准备好的任务分配给闲置的代理。DAG表示编码了优先级和资源约束，确保了正确性，同时暴露了所有可用的并行性。在五个基准场景中的实验表明，OSDAG的推理时间比基于对话的方法快5-15倍，较顺序基线减少了最多38%的完工时间，并保持了竞争性的成功率。在双臂操作任务上的仿真和现实世界实验验证了所提方法在高效多机器人协调中的有效性和实用性。相关网站和资源可访问 http://thanhnguyencanh.github.io/LLM_DAG4MultiRobot

View on arXiv Download PDF AI Translation

cs.RO / 23 / 2606.15285

Acting While Understanding: Asynchronous Semantic-Action Decoupling for Real-Time Vision-Language-Action Models

理解与行动并行：实时视觉-语言-行动模型的异步语义-行动解耦

Yan, Shenhao, Wang, Ge, Liu, Qi, Meng, Weilin, Yang, Jiahao, Yao, Chengsi, Feng, Fan, Ma, Xiaoguang, Zhao, Yiming, Han, Yatong

Abstract

Vision-Language-Action models (VLAs) have demonstrated strong task understanding and generalization in robotic manipulation, yet the high computational cost of full-model inference limits their deployment in low-latency, high-frequency closed-loop control. We propose an asynchronous semantic-action decoupling framework that separates semantic understanding from action generation along the internal semantic-action interface of existing VLAs, without redesigning the vision-language backbone or introducing an external planner. A low-frequency understanding module asynchronously updates reusable semantic conditions, while a high-frequency action module continuously outputs control actions without repeatedly invoking the full model. To mitigate the temporal mismatch between stale semantics and the current execution state, we further introduce historical action conditioning and time-misalignment training, which provide short-horizon execution context and improve feedback control robustness under stale semantic conditions. Experiments on LIBERO with $\pi_{0.5}$ and UniVLA, together with real-robot deployment using UniVLA, show that the proposed framework achieves up to 35.6 Hz server-side action-module inference throughput and offers a low-intrusion path to high-frequency closed-loop control without running full VLA inference at control rate.

Chinese Translation

视觉-语言-行动模型（VLA）在机器人操控任务理解和泛化方面表现出色，但全模型推理的高计算成本限制了其在低延迟、高频闭环控制中的应用。我们提出了一种异步语义-行动解耦框架，该框架在现有VLA的内部语义-行动接口上将语义理解与行动生成分离，而无需重新设计视觉-语言主干或引入外部规划器。低频理解模块异步更新可重用的语义条件，而高频行动模块则持续输出控制动作，无需重复调用完整模型。为了减轻过时语义与当前执行状态之间的时间不匹配，我们进一步引入了历史行动条件和时间错位训练，这为短期执行提供了上下文，并在过时语义条件下提高了反馈控制的鲁棒性。在LIBERO上使用$ ext{π}_{0.5}$和UniVLA进行的实验，以及使用UniVLA的真实机器人部署，表明所提出的框架实现了高达35.6 Hz的服务器端行动模块推理吞吐量，并为高频闭环控制提供了一条低干扰路径，而无需在控制速率下运行完整的VLA推理。

View on arXiv Download PDF AI Translation

cs.RO / 24 / 2606.15317

Covariance-Regulated Recursive Koopman Learning for Nonlinear Systems with Uncertain Time-Varying Dynamics

基于协方差调节的递归库普曼学习用于具有不确定时变动态的非线性系统

Gu, Weibin, Yang, Chen, Shi, Lu, Gao, Chao

Abstract

Offline models for autonomous robots often fail under time-varying dynamics outside their training distribution. Koopman operator theory offers a linear representation of nonlinear dynamics via lifting, but its transition to real-time recursive estimation may suffer numerical vulnerabilities: covariance windup under low excitation when using exponential forgetting, and vanishing gain without forgetting. This paper introduces a Covariance-Regulated Recursive Koopman Learning (CR-RKL) framework with two complementary strategies--error dead-zone gating and constant-trace normalization--each independently capable of preventing covariance explosion and parameter freezing, with the latter additionally preserving the geometric structure of uncertainty. Validated on a non-holonomic differential-drive robot with wheel slip and Stribeck friction and on a 26-gram butterfly-inspired flapping-wing micro aerial vehicle, CR-RKL achieves numerically stable and accurate online modeling, and when embedded in model predictive control, it maintains reliable tracking performance under uncertain, time-varying dynamics.

Chinese Translation

自主机器人的离线模型在其训练分布之外的时变动态下往往表现不佳。库普曼算子理论通过提升提供了非线性动态的线性表示，但其向实时递归估计的过渡可能会遭遇数值脆弱性：在使用指数遗忘时，低激励下的协方差膨胀，以及在没有遗忘时的增益消失。本文提出了一种协方差调节递归库普曼学习（CR-RKL）框架，采用两种互补策略——误差死区门控和常量迹归一化——每种策略都能独立防止协方差爆炸和参数冻结，后者还额外保持了不确定性的几何结构。在一个具有轮滑和斯特里贝克摩擦的非完整微分驱动机器人以及一个26克的蝴蝶启发的拍翼微型飞行器上进行验证，CR-RKL实现了数值稳定且准确的在线建模，并且在嵌入模型预测控制中，能够在不确定的时变动态下保持可靠的跟踪性能。

View on arXiv Download PDF AI Translation

cs.RO / 25 / 2606.15338

SimWeaver: Zero-Shot RGB Sim-to-Real for Deformable Manipulation

SimWeaver：零样本 RGB 模拟到现实的可变形操作

Hu, Wenkang, Wang, Haoran, Li, Yitong, Liu, Liu, Zhao, Mengao, Jiang, Lai, Tang, Xincheng, Wei, Junhang, Shu, Zhengjie, Wang, Zhendong, Su, Zhizhong, Wang, Huamin, Yang, Ruigang

Abstract

RGB sim-to-real for deformable manipulation has remained largely unsolved without real-world fine-tuning. We present SimWeaver, which trains zero-shot RGB VLA policies on 200 simulated demonstrations per task, reaching above 80% per-task and 91% average real-world success across 5 diverse deformable tasks including plastic-bag manipulation, without teleoperation or per-task calibration. SimWeaver combines a reliable measurement-backed simulator (SimWeaver-Sim) with an extensible asset framework supporting single-image generation(SimWeaver-Asset), a deterministic topology-aware trajectory synthesizer (SimWeaver-Syn), and a sim-to-real protocol with ISP-aware photometric augmentation (SimWeaver-Real). On silk grasping, the sim-trained policy reaches 100% under visual distribution shifts where real-data baselines drop to 9-70%, at two orders of magnitude lower per-trajectory cost. We will release SimWeaver and a representative asset subset. Project page: https://simweaver.github.io/

Chinese Translation

在没有现实世界微调的情况下，RGB 模拟到现实的可变形操作仍然基本未得到解决。我们提出了 SimWeaver，它在每个任务上使用 200 个模拟演示训练零样本 RGB VLA 策略，达到了每个任务超过 80% 和 91% 的平均现实世界成功率，涵盖包括塑料袋操作在内的 5 个不同的可变形任务，且无需远程操作或每个任务的校准。SimWeaver 将一个可靠的基于测量的模拟器（SimWeaver-Sim）与一个可扩展的资产框架相结合，支持单图像生成（SimWeaver-Asset）、一个确定性的拓扑感知轨迹合成器（SimWeaver-Syn），以及一个具有 ISP 感知光度增强的模拟到现实协议（SimWeaver-Real）。在丝绸抓取任务中，经过模拟训练的策略在视觉分布变化下达到了 100% 的成功率，而真实数据基线的成功率降至 9-70%，且每条轨迹的成本低两个数量级。我们将发布 SimWeaver 及其代表性资产子集。项目页面：https://simweaver.github.io/

View on arXiv Download PDF AI Translation

cs.RO / 26 / 2606.15373

A Hybrid Model-Based and Model-Free Framework for Active Multi-View Viewpoint Optimization in Sonar Target Recognition

基于混合模型的主动多视角视点优化框架在声纳目标识别中的应用

Park, Yongkyoon, Shin, Jane

Abstract

This paper presents a hybrid model-based and model-free framework for active multi-view target recognition using forward-looking sonar. A convolutional neural network (CNN) provides data-driven observation likelihoods, while Radon-based orientation estimation enables viewpoint-aware sensing without requiring angle annotations. During training, an information-gain-based reward guides a Proximal Policy Optimization (PPO) agent to learn a belief-aware viewpoint selection policy offline. At deployment, the learned policy performs real-time viewpoint selection using only CNN-based belief updates, eliminating the need for computationally expensive online POMDP tree search. Experiments on a marine-debris forward-looking sonar dataset demonstrate that the proposed approach achieves competitive recognition accuracy while reducing sensing steps and motion cost compared to model-based baselines.

Chinese Translation

本文提出了一种基于混合模型的主动多视角目标识别框架，利用前视声纳进行目标识别。卷积神经网络（CNN）提供数据驱动的观测似然性，而基于拉东变换的方向估计则实现了视点感知的传感，无需角度标注。在训练过程中，基于信息增益的奖励引导近端策略优化（PPO）代理离线学习一个基于信念的视点选择策略。在部署阶段，学习到的策略仅使用基于CNN的信念更新进行实时视点选择，从而消除了对计算成本高昂的在线部分可观测马尔可夫决策过程（POMDP）树搜索的需求。在一个海洋垃圾前视声纳数据集上的实验表明，所提出的方法在减少传感步骤和运动成本的同时，达到了与基于模型的基准相当的识别准确率。

View on arXiv Download PDF AI Translation

cs.RO / 27 / 2606.15431

A Corridor-Scale CARLA-VISSIM Co-Simulation Framework for Multi-Intersection Urban Traffic

用于多交叉口城市交通的走廊尺度CARLA-VISSIM协同仿真框架

Ashayer, Sima, Haris, Austin, Sartipi, Mina

Abstract

This paper presents an implemented CARLA-VISSIM co-simulation framework for an urban corridor comprising approximately fifteen connected intersections centered on Martin Luther King Jr. Boulevard in Chattanooga, Tennessee. The system integrates CARLA 0.10.0 Unreal Engine 5 with PTV VISSIM 2026 through a bidirectional, step-synchronized interface that couples VISSIM's microscopic vehicle, pedestrian, and signal-controller logic with CARLA's high-fidelity 3D rendering. A LiDAR-derived elevation model and RoadRunner-based High Definition (HD) map provide terrain-accurate road geometry deployed consistently across both simulators. The framework incorporates explicit actor ownership, mirrored lifecycle management, coordinate reconciliation, and a latest-state-per-actor update policy, enabling stable interaction between VISSIM-controlled traffic and a CARLA-controlled ego vehicle. A corridor-scale case study demonstrates consistent traffic-signal mirroring, synchronized vehicle-pedestrian interactions, and stable mixed-authority operation under peak loads of approximately 100 vehicles and 100 pedestrians. The deployment captures the interaction of the five signalized intersections along MLK Street and their connecting upstream and downstream intersections, revealing synchronization challenges unique to multi-intersection corridors. Results indicate that this MLK-centered corridor provides an effective testbed for verifying cross-simulator consistency and that the proposed architecture supports reliable, perception-ready co-simulation for corridor-level traffic studies.

Chinese Translation

本文提出了一个实现的CARLA-VISSIM协同仿真框架，针对位于田纳西州查塔努加的马丁·路德·金大道（Martin Luther King Jr. Boulevard）周围约十五个相连交叉口的城市走廊。该系统通过双向、步进同步接口将CARLA 0.10.0与PTV VISSIM 2026集成，结合了VISSIM的微观车辆、行人和信号控制逻辑与CARLA的高保真3D渲染。基于激光雷达（LiDAR）生成的高程模型和RoadRunner基础的高清（HD）地图提供了在两个仿真器中一致部署的地形准确道路几何。该框架包含显式的参与者所有权、镜像生命周期管理、坐标协调和每个参与者的最新状态更新策略，使得VISSIM控制的交通与CARLA控制的自我车辆之间能够稳定互动。走廊尺度的案例研究展示了一致的交通信号镜像、同步的车辆-行人互动，以及在约100辆车辆和100名行人高峰负载下的稳定混合权限操作。该部署捕捉了沿MLK街的五个信号交叉口及其连接的上下游交叉口之间的互动，揭示了多交叉口走廊特有的同步挑战。结果表明，以MLK为中心的走廊为验证跨仿真器一致性提供了有效的测试平台，并且所提出的架构支持可靠、感知准备的走廊级交通研究协同仿真。

View on arXiv Download PDF AI Translation

cs.RO / 28 / 2606.15434

A Bilateral Teleoperation Framework for Dexterous Manipulation

用于灵巧操作的双向遥操作框架

Gasperina, Stefano Dalla, Kang, Dong Ho, Zhang, Haiyun, Galvan, Aldo, Ramirez, Job D., Kim, Aaron, Helwig, Mark, Yokoyama, Kazuto, Ueno, Takahisa, Narita, Tetsuya, Majewicz-Fey, Ann, Deshpande, Ashish D., Sentis, Luis

Abstract

Dexterous teleoperation requires precise arm-hand coordination, low-latency feedback, and robust interaction in real-world contact-rich environments. This paper presents a modular bilateral teleoperation framework that integrates operator-side input interfaces with a robot-side dexterous hand and compliant robotic arm in a unified control architecture. The system supports position-based hand retargeting, differential arm control, multi-scale haptic feedback, and shared control for stable manipulation. We validate the framework through a real-world dexterous manipulation task, highlighting coordinated arm-hand control and contact-aware interaction. Beyond feasibility, we identify key design insights related to cross-embodiment mismatch, haptic feedback granularity, and shared control. The proposed platform provides a practical teleoperation system and a foundation for collecting high-quality demonstrations for future learning-from-demonstration research.

Chinese Translation

灵巧遥操作需要精确的手臂-手协调、低延迟反馈以及在真实接触丰富环境中的稳健交互。本文提出了一种模块化的双向遥操作框架，该框架将操作员侧输入接口与机器人侧的灵巧手和柔顺机器人臂集成在统一的控制架构中。该系统支持基于位置的手部重定向、差分臂控制、多尺度触觉反馈和共享控制，以实现稳定的操作。我们通过一个真实世界的灵巧操作任务验证了该框架，突出了协调的手臂-手控制和接触感知交互。除了可行性之外，我们还识别了与跨体现不匹配、触觉反馈粒度和共享控制相关的关键设计见解。所提出的平台提供了一个实用的遥操作系统，并为未来的基于示范学习研究收集高质量示范奠定了基础。

View on arXiv Download PDF AI Translation

cs.RO / 29 / 2606.15469

Learning Context-Aware Neural ODE Dynamics for Adaptive Robotic Control

学习上下文感知的神经常微分方程动态模型用于自适应机器人控制

Yu, Shao-Yi, Wang, Jen-Wei, Horii, Maya, Tomizuka, Masayoshi, Garg, Vikas

Abstract

Robotic systems deployed in uncertain and dynamically changing environments often face variations in contact conditions, aerodynamic effects, and external disturbances that challenge reliable control. To remain effective under model-based control, these systems require dynamics models that can adapt to such changes, especially when direct access to complete environmental information is limited. To enable adaptability and facilitate integration with model predictive control, we propose a context-aware dynamics model based on neural ordinary differential equations, which infers environmental factors from state-action histories using a two-phase training procedure. We validate the approach across diverse robotic platforms, including a quadrotor in simulation, as well as a Sphero BOLT robot and a Fanuc manipulator in real-world experiments. The results demonstrate that our method effectively adapts to temporally and spatially varying environmental changes across different tasks. Videos are available at https://youtu.be/PY0sNyF2rqE , and the source code is available at https://github.com/syyu410-yu/context-aware-neural-ode-control.git .

Chinese Translation

在不确定和动态变化的环境中部署的机器人系统，常常面临接触条件、气动效应和外部干扰的变化，这些变化对可靠控制构成挑战。为了在基于模型的控制下保持有效性，这些系统需要能够适应这些变化的动态模型，尤其是在直接获取完整环境信息受限的情况下。为实现适应性并促进与模型预测控制的集成，我们提出了一种基于神经常微分方程的上下文感知动态模型，该模型通过两阶段训练过程从状态-动作历史中推断环境因素。我们在多种机器人平台上验证了该方法，包括在仿真中使用的四旋翼飞行器，以及在真实实验中使用的 Sphero BOLT 机器人和 Fanuc 机械手臂。结果表明，我们的方法能够有效适应不同任务中时间和空间变化的环境变化。视频可在 https://youtu.be/PY0sNyF2rqE 查看，源代码可在 https://github.com/syyu410-yu/context-aware-neural-ode-control.git 获取。

View on arXiv Download PDF AI Translation

cs.RO / 30 / 2606.15476

FARM: Find Anything using Relational Spatial Memory

FARM：使用关系空间记忆查找任何物体

He, Siming, Huang, Leo, Lilja, Adam, Hubel, Fabio, Frey, Jonas, Pavone, Marco, Sastry, S. Shankar, Malik, Jitendra, Tomlin, Claire

Abstract

Robots operating in homes, warehouses, and other object-rich environments need memory systems that can find specific object instances on demand. Object-level memory alone is often insufficient: scenes contain many plausibly matching objects, and users refer to the target through relations to landmarks and surrounding objects (e.g. ``the tall lamp below the dartboard and to the left of the poster''), demanding a relational spatial memory that supports retrieval through semantic, appearance, and spatial predicates over objects. To achieve this, we present FARM (Find Anything using Relational Spatial Memory), which builds, in real time at 5-10 Hz, a compact, open-vocabulary, object-level memory with geometry, visual-language descriptors, and viewpoint evidence. At query time, FARM uses VLMs to parse the query and score visual evidence, while grounding spatial constraints explicitly through object symbols and relational predicates. This structured use of VLMs enables more accurate and robust retrieval than end-to-end reasoning over frame histories or scene-graph context. In experiments on 44k language queries spanning 67 indoor and outdoor scenes, ranging from 15 to 15,000 m^2, FARM improves Recall@5 and Recall@10 over prior methods by 164% and 224%, and a final VLM reranking stage improves Accuracy@1 by 35%, while running in real time. We further demonstrate closed-loop deployment on a quadrupedal robot using onboard sensors and compute.

Chinese Translation

在家庭、仓库和其他物体丰富的环境中运行的机器人需要能够按需查找特定物体实例的记忆系统。仅靠物体级记忆往往不足：场景中包含许多可能匹配的物体，用户通过与地标和周围物体的关系来指代目标（例如“在飞镖靶下方且在海报左侧的高灯”），这要求具备支持通过语义、外观和空间谓词对物体进行检索的关系空间记忆。为此，我们提出了FARM（Find Anything using Relational Spatial Memory），它以每秒5-10帧的速度实时构建一个紧凑的、开放词汇的物体级记忆，包含几何信息、视觉语言描述符和视角证据。在查询时，FARM使用视觉语言模型（VLMs）解析查询并评分视觉证据，同时通过物体符号和关系谓词明确地固定空间约束。这种结构化使用VLMs的方法比端到端推理帧历史或场景图上下文能够实现更准确和更稳健的检索。在涵盖67个室内和室外场景的44,000个语言查询的实验中，FARM在Recall@5和Recall@10上分别比之前的方法提高了164%和224%，而最终的VLM重排序阶段则将Accuracy@1提高了35%，且能够实时运行。我们进一步展示了在四足机器人上使用机载传感器和计算资源的闭环部署。

View on arXiv Download PDF AI Translation

cs.RO / 31 / 2606.15491

FD-SLAM: Fast Dense Radar-Inertial SLAM with Frequency-Domain Loop Closure and Pose Graph Optimization

FD-SLAM：具有频域回环闭合和姿态图优化的快速稠密雷达惯性SLAM

Abu-Alrub, Nader J., Rawashdeh, Nathir A.

Abstract

Radar SLAM is attractive for autonomous ground vehicles operating in visually degraded environments, however, scanning radars are noisy, have low scanning rates, and their measurements are challenging to match reliably over long trajectories. This paper presents FD-SLAM, a fast dense radar-inertial SLAM system that extends dense radar-inertial odometry with frequency-domain loop closure and pose graph optimization. The proposed method preserves an image-like structure of scanning radar measurements by using a compact frequency-domain polar descriptor for loop-candidate retrieval and a multi-stage verification pipeline based on temporal filtering, phase-correlation screening, scan-alignment similarity, and geometric consistency checks. Verified loop closures are added as non-sequential constraints in an SE(2) pose graph together with radar-inertial odometry factors. FD-SLAM is evaluated on a publicly available dataset using standard KITTI evaluation metrics. The results show that FD-SLAM improves FD-RIO baseline, achieves competitive performance against current state-of-the-art radar SLAM methods, and provides favorable rotational accuracy across multiple evaluated driving trajectories. Runtime analysis further indicates that the radar-inertial front-end operates above the radar sampling rate on a CPU-only setup, while loop closure detection and graph optimization remain suitable for parallel background execution.

Chinese Translation

雷达SLAM在视觉环境恶劣的自主地面车辆中具有吸引力，然而，扫描雷达存在噪声、扫描速率低，并且在长轨迹上可靠匹配其测量值具有挑战性。本文提出了FD-SLAM，一种快速稠密雷达惯性SLAM系统，扩展了稠密雷达惯性里程计，结合了频域回环闭合和姿态图优化。所提出的方法通过使用紧凑的频域极坐标描述符进行回环候选检索，并基于时间滤波、相位相关筛选、扫描对齐相似性和几何一致性检查的多阶段验证管道，保持了扫描雷达测量的图像结构。经过验证的回环闭合作为非顺序约束添加到SE(2)姿态图中，与雷达惯性里程计因子一起使用。FD-SLAM在一个公开可用的数据集上使用标准KITTI评估指标进行了评估。结果表明，FD-SLAM改善了FD-RIO基线，达到了与当前最先进的雷达SLAM方法竞争的性能，并在多个评估的驾驶轨迹上提供了良好的旋转精度。运行时分析进一步表明，雷达惯性前端在仅使用CPU的设置下运行在雷达采样率之上，而回环闭合检测和图优化仍适合并行后台执行。

View on arXiv Download PDF AI Translation

cs.RO / 32 / 2606.15494

Understanding and Modeling Perceived Cognitive and Physical Strain Dynamics for Planning-Oriented Human-Robot Collaboration in Prefabricated Construction

理解与建模感知的认知与身体负荷动态，以支持预制建筑中的规划导向人机协作

Wang, Yifan, Xiao, Bo, Mueller, Shane T.

Abstract

Human-robot collaboration (HRC) in prefabricated construction requires planning approaches that consider not only productivity but also time-dependent worker states during repeated work and rest. Existing planning models often rely on simplified assumptions about fatigue, workload, or recovery, with limited domain-specific empirical evidence on how perceived strain evolves. This study develops an empirically grounded, planning-oriented approach to characterize perceived strain accumulation and recovery in prefabricated construction HRC. A controlled repeated work-rest experiment assessed perceived cognitive and physical strain using the Rating Scale for Mental Effort and Borg's Rating of Perceived Exertion. Linear and exponential functional forms were evaluated, followed by mixed-effects modeling to examine collaborative conditions, session effects, and inter-individual variability. Results indicate that cognitive strain accumulation is best represented by a linear mixed-effects model, whereas rest-phase recovery follows nonlinear decay. The resulting planning-oriented models may inform future human-state-aware task allocation and scheduling research.

Chinese Translation

在预制建筑中，人机协作（HRC）需要考虑不仅仅是生产力，还包括在重复工作和休息期间时间依赖的工人状态的规划方法。现有的规划模型通常依赖于对疲劳、工作负荷或恢复的简化假设，缺乏关于感知负荷如何演变的领域特定实证证据。本研究开发了一种以实证为基础的、面向规划的方法，以表征在预制建筑人机协作中感知负荷的积累与恢复。通过控制的重复工作-休息实验，使用心理努力评定量表（Rating Scale for Mental Effort）和博格感知用力评定（Borg's Rating of Perceived Exertion）评估了感知的认知与身体负荷。评估了线性和指数函数形式，随后进行了混合效应建模，以考察协作条件、会话效应和个体间变异性。结果表明，认知负荷的积累最适合用线性混合效应模型表示，而休息阶段的恢复则呈现非线性衰减。所得到的面向规划的模型可能为未来人状态感知的任务分配和调度研究提供参考。

View on arXiv Download PDF AI Translation

cs.RO / 33 / 2606.15514

Reinforcement Learning-Guided Retrieval with Soft Fusion for Robust Multimodal Imitation Learning under Missing Modalities

基于强化学习引导的软融合检索方法：在缺失模态下的鲁棒多模态模仿学习

Ismkhan, Hassan, Bouchahcia, Hamid

Abstract

Robotic systems perceive the world through multiple input modalities -- including visual camera streams and natural language instructions -- and must select appropriate actions based on these signals. However, assuming the permanent availability of all input devices is unrealistic, as sensors may fail, become occluded, or drop out entirely during deployment. Robust handling of such missing-modality scenarios is therefore essential for real-world robot operation. This paper introduces RL4IL, a reinforcement learning guided method for imitation learning that selects the most suitable action for a given observation by identifying the most relevant expert demonstrations from a training library. A reinforcement learning policy, trained via Proximal Policy Optimisation over Breadth-First Search candidate sets, ranks candidate demonstrations and a soft cross-attention fusion head aggregates their action signals to produce the final prediction. When a modality is missing at inference time, a dedicated per-modality RL retrieval policy identifies donor demonstrations from the training library, and a soft imputation head reconstructs the missing embedding via cross-attention over the top-ranked donors -- without requiring any retraining of the system. Experiments on three LIBERO benchmark suites demonstrate that RL4IL substantially outperforms state-of-the-art imitation learning methods under sensor dropout conditions, while requiring no policy network training. The code can be found at https://github.com/h-ismkhan/Reinforcement-Learning-via-kNN-for-Robotic-Learning-with-Missing-Camera

Chinese Translation

机器人系统通过多种输入模态感知世界，包括视觉摄像头流和自然语言指令，并必须根据这些信号选择适当的行动。然而，假设所有输入设备始终可用是不现实的，因为传感器可能会故障、被遮挡或在部署期间完全失效。因此，鲁棒地处理缺失模态的场景对于现实世界中的机器人操作至关重要。本文介绍了RL4IL，这是一种基于强化学习的模仿学习方法，通过从训练库中识别最相关的专家示范来为给定观察选择最合适的行动。通过近端策略优化（Proximal Policy Optimisation）在广度优先搜索候选集上训练的强化学习策略对候选示范进行排名，而软交叉注意力融合头则聚合它们的行动信号以生成最终预测。当推理时缺失某一模态时，专门的每模态强化学习检索策略从训练库中识别捐赠示范，软插补头通过对排名最高的捐赠者进行交叉注意力重建缺失的嵌入，而无需对系统进行任何再训练。在三个LIBERO基准测试套件上的实验表明，RL4IL在传感器失效条件下显著优于最先进的模仿学习方法，同时不需要策略网络的训练。代码可在 https://github.com/h-ismkhan/Reinforcement-Learning-via-kNN-for-Robotic-Learning-with-Missing-Camera 找到。

View on arXiv Download PDF AI Translation

cs.RO / 34 / 2606.15516

Transferring Contact, Not Just Motion: Compliant Grasping Across Dexterous Hands

传递接触，而不仅仅是运动：跨灵巧手的柔性抓取

Atar, Soofiyan, Huang, Yao-Ting, Yip, Michael

Abstract

Dexterous grasping depends on contact regulation, not motion alone. Stable manipulation requires fingers to maintain appropriate object loading as contacts slip, deform, or become visually occluded. Existing cross-embodiment dexterous policies unify motion through retargeted hand poses or latent actions, but force feedback remains tied to each hand's sensing and actuation, limiting transfer. This work introduces a cross-embodiment force-position interface for contact-aware manipulation across heterogeneous dexterous hands. Motion intent is represented in a shared hand-pose latent, while each hand's effort signal is calibrated through system identification into physical joint torque in N.m. These torques are mapped to fingertip forces and compact per-finger load descriptors, giving the policy comparable observations of where the hand should move and how the object is loaded. Using this interface, a flow-matching visuomotor policy is trained on vision, proprioception, and calibrated contact, with structured visual masking that encourages reliance on force under grasp-relevant occlusion. The same calibrated signal drives a hybrid force-position controller for demonstration collection and execution, keeping force targets consistent across training and deployment. Experiments across structurally different hands show that calibrated contact feedback enables transferable compliant grasping, with learned primitives reusable in long-horizon manipulation pipelines.

Chinese Translation

灵巧抓取依赖于接触调节，而不仅仅是运动。稳定的操作要求手指在接触滑动、变形或被视觉遮挡时保持适当的物体负载。现有的跨体现灵巧策略通过重新定位的手势或潜在动作统一运动，但力反馈仍然与每只手的传感和驱动相绑定，限制了转移能力。本研究提出了一种跨体现的力-位置接口，用于在异构灵巧手之间进行接触感知的操作。运动意图在共享的手势潜在空间中表示，而每只手的努力信号通过系统识别校准为物理关节扭矩（N·m）。这些扭矩被映射到指尖力和紧凑的每指负载描述符，使策略能够获得手应如何移动以及物体如何被加载的可比观察。利用该接口，基于视觉、本体感知和校准接触训练了一个流匹配的视觉运动策略，并采用结构化的视觉遮罩，鼓励在与抓取相关的遮挡下依赖于力。相同的校准信号驱动混合力-位置控制器进行演示收集和执行，确保在训练和部署中保持力目标一致。在结构上不同的手之间的实验表明，校准的接触反馈能够实现可转移的柔性抓取，学习到的原语可在长时间操作管道中重复使用。

View on arXiv Download PDF AI Translation

cs.RO / 35 / 2606.15550

Robots as Tokens: Unified Diffusion Transformer for Coordinated Multi-Robot Trajectory Generation

机器人作为代币：统一扩散变换器用于协调多机器人轨迹生成

Bai, Ruofei, Chen, Jie, Cai, Yuxin, Li, Jun, Yau, Wei-Yun, Xie, Lihua

Abstract

The success of generative models in language and visual generation has inspired extensive applications to generative robot planning. However, most existing works either focus on single-robot planning, or generate multi-robot trajectories in a sequential manner with iterative post-processing to resolve inter-robot conflicts. In this work, we investigate whether coordinated multi-robot trajectories, as a special spatiotemporal distribution, can be learned and generated with a generative model in a feed-forward manner. We propose Robots as Tokens (Roken), a unified diffusion transformer that directly generates multi-robot trajectories that satisfy both (individual) safety and (global) connectivity constraints. The core design of Roken is to represent each robot as a discrete token, allowing them to naturally interact with each other through self-attention, and cross-attend to map tokens for environment layouts. We further introduce several auxiliary tasks based on Bayes' theorem to provide multi-scale spatial-temporal supervision for efficient learning of the conditional distribution. In training, Roken absorbs diverse expert trajectories from different team sizes. During inference, Roken behaves as a versatile multi-robot planner that can handle single-robot planning, coordinated multi-robot trajectory generation, and conditional trajectory generation by fixing some robot tokens as conditions. Experiments in diverse cluttered environments show that Roken can generate coordinated multi-robot trajectories to perform connectivity-constrained goal navigation tasks with high success rates, outperforming the baseline method used to generate the training dataset. Roken also demonstrates good scalability after training with mixed team sizes, and shows generalization to unseen or partially observed environments, verifying its potential to learn from diverse data and perform versatile tasks.

Chinese Translation

生成模型在语言和视觉生成中的成功激发了生成机器人规划的广泛应用。然而，现有的大多数研究要么专注于单机器人规划，要么以顺序方式生成多机器人轨迹，并通过迭代后处理来解决机器人间的冲突。在本研究中，我们探讨了协调多机器人轨迹作为一种特殊的时空分布，是否可以通过生成模型以前馈方式进行学习和生成。我们提出了“机器人作为代币”（Robots as Tokens, Roken），一种统一的扩散变换器，能够直接生成满足（个体）安全性和（全局）连通性约束的多机器人轨迹。Roken的核心设计是将每个机器人表示为一个离散的代币，使它们能够通过自注意力自然地相互作用，并交叉关注以映射环境布局的代币。我们进一步基于贝叶斯定理引入了几个辅助任务，以提供多尺度时空监督，从而高效学习条件分布。在训练过程中，Roken吸收来自不同团队规模的多样化专家轨迹。在推理过程中，Roken表现为一个多功能的多机器人规划器，能够处理单机器人规划、协调多机器人轨迹生成以及通过固定一些机器人代币作为条件进行条件轨迹生成。在多样化的杂乱环境中的实验表明，Roken能够生成协调的多机器人轨迹，以高成功率执行受连通性约束的目标导航任务，优于用于生成训练数据集的基线方法。Roken在经过混合团队规模的训练后还展示了良好的可扩展性，并对未见或部分观察到的环境表现出泛化能力，验证了其从多样化数据中学习和执行多功能任务的潜力。

View on arXiv Download PDF AI Translation

cs.RO / 36 / 2606.15568

SAPS: Shared Autonomy for Policy Steering by Blending Teleoperation with a Pretrained VLA

SAPS：通过将远程操作与预训练的视觉-语言-动作（VLA）模型相结合实现政策引导的共享自主性

Zhou, Crystal, Yang, Jehan, Weber, Douglas J., Erickson, Zackory

Abstract

Recent advancements in Vision-Language-Action (VLA) models have demonstrated impressive generalist capabilities in robot manipulation, yet these policies can be brittle under out-of-distribution spatial and semantic perturbations. While human teleoperation offers reliable recovery, it can demand high cognitive load and precise manual control, and existing policy steering methods often require auxiliary models or sampler modifications. In this work, we introduce Shared Autonomy for Policy Steering (SAPS), a framework that blends real-time human teleoperation commands with pretrained policy actions at the action level. SAPS requires no policy retraining, auxiliary dynamics models, or architectural modifications. We propose and evaluate three arbitration strategies to balance human and VLA policy control, including a dynamic Cosine-similarity arbitration strategy that computes the geometric agreement between human and policy actions. Across evaluations in simulation (LIBERO, LIBERO-PRO, CALVIN) and on real-world robot hardware, SAPS improves task success rates over autonomous execution by up to 82% in both simulation and the real world. Furthermore, our approach drastically reduces human intervention compared to pure teleoperation, while simultaneously achieving faster task completion times than both autonomous execution and pure teleoperation. These results demonstrate that action-level shared autonomy is a practical, model-agnostic approach for reliably deploying generalist robot policies in real-world contexts involving a human operator,with promising applications in assistive teleoperation and scalable data collection.

Chinese Translation

最近在视觉-语言-动作（VLA）模型方面的进展展示了其在机器人操作中的令人印象深刻的通用能力，然而这些政策在分布外的空间和语义扰动下可能表现脆弱。虽然人类远程操作提供了可靠的恢复能力，但它可能需要较高的认知负荷和精确的手动控制，而现有的政策引导方法通常需要辅助模型或采样器的修改。在本研究中，我们提出了政策引导的共享自主性（SAPS）框架，该框架在动作层面上将实时人类远程操作命令与预训练政策动作相结合。SAPS不需要政策重新训练、辅助动力学模型或架构修改。我们提出并评估了三种仲裁策略，以平衡人类和VLA政策控制，包括一种动态余弦相似度仲裁策略，该策略计算人类与政策动作之间的几何一致性。在模拟（LIBERO、LIBERO-PRO、CALVIN）和真实机器人硬件上的评估中，SAPS在任务成功率上比自主执行提高了多达82%。此外，与纯远程操作相比，我们的方法显著减少了人类干预，同时在任务完成时间上比自主执行和纯远程操作都更快。这些结果表明，动作层面的共享自主性是一种实用的、与模型无关的方法，能够在涉及人类操作员的真实世界背景中可靠地部署通用机器人政策，并在辅助远程操作和可扩展数据收集方面具有良好的应用前景。

View on arXiv Download PDF AI Translation

cs.RO / 37 / 2606.15587

Perfect Demo Makes Poor Teacher: Learning Robust Alignment from Critical Motion Segments

完美示范却成为糟糕教师：从关键运动片段中学习鲁棒对齐

Liu, Mingyu, Li, Zeju, Shu, Jiuhe, Wang, Hanqing, Chao, Yuhao, Chen, Hao, Shen, Chunhua

Abstract

Expert demonstrations are widely assumed to be the gold standard for robot imitation learning. Yet for fine-grained manipulation such as insertion, stacking, and alignment, we uncover a counterintuitive failure mode: fluent demonstrations can be poor teachers. A skilled teleoperator compresses the decisive moments of alignment and recovery into a brief temporal window, leaving the policy flooded with redundant free-space motion and starved of supervision exactly where precision determines success. We address this bottleneck at two levels. At the data level, slowing down near alignment and resampling critical segments both help, yet the gain comes mainly from broadening the coverage of recovery states the policy must learn, not from reweighting frames it already has. Such data-side fixes, however, leave the policy's per-frame view untouched: a single image still maps directly to an action, and the local motion that governs correction stays implicit. We therefore turn to the representation level and introduce STAIR (\textbf{S}patio-\textbf{T}emporal feature \textbf{A}s an \textbf{I}nterface for \textbf{R}obot learning), a compact dynamic feature that bridges the vision-language model and the action expert, distilling the short-horizon motion already recorded in each trajectory into dense, motion-aware supervision. Trained on fluent data alone, STAIR recovers most of the deliberate-demonstration gain ($50.0$ to $62.2\%$ overall, approaching the $64.4\%$ of deliberate demonstrations). These results call for a more pedagogical view of robot data, optimized for machine learnability rather than human efficiency alone.

Chinese Translation

专家示范被广泛认为是机器人模仿学习的金标准。然而，对于细粒度操作如插入、堆叠和对齐，我们发现了一种违反直觉的失败模式：流畅的示范可能是糟糕的教师。一名熟练的遥控操作员将对齐和恢复的关键时刻压缩到一个简短的时间窗口中，导致策略充斥着冗余的自由空间运动，而在精度决定成功的地方却缺乏监督。我们在两个层面上解决了这一瓶颈。在数据层面，减慢对齐过程并重新采样关键片段都有助于改善，然而收益主要来自于扩大策略必须学习的恢复状态的覆盖范围，而不是重新加权它已经拥有的帧。然而，这种数据层面的修复并没有改变策略的逐帧视图：单个图像仍然直接映射到一个动作，而控制修正的局部运动依然隐含。因此，我们转向表示层面，提出了STAIR（ extbf{S}patio- extbf{T}emporal feature extbf{A}s an extbf{I}nterface for extbf{R}obot learning），这是一种紧凑的动态特征，连接视觉-语言模型和动作专家，将每个轨迹中已记录的短期运动提炼为密集的、运动感知的监督。仅在流畅数据上训练，STAIR恢复了大部分有意示范的收益（整体从$50.0 ext{%}$提升至$62.2 ext{%}$，接近$64.4 ext{%}$的有意示范）。这些结果呼吁对机器人数据采取更具教学性的视角，优化机器学习能力，而不仅仅是人类效率。

View on arXiv Download PDF AI Translation

cs.RO / 38 / 2606.15594

Pixels to Proofs: Probabilistically-Safe Latent World Model Control via Parallel Conformal Robust MPC

从像素到证明：通过并行保形鲁棒模型预测控制实现概率安全的潜在世界模型控制

Nath, Devesh, Srinivasan, Anutam, Yin, Haoran, Jiang, Ruitong, Fang, Jeffrey, Chou, Glen

Abstract

We present SLS^2, a framework for safe feedback motion planning from pixels using robust model predictive control (MPC) in learned latent world models. Our approach trains an action-conditioned joint-embedding world model with compact Markovian latent states, enabling efficient gradient-based trajectory optimization through learned latent dynamics. To enforce safety for the true system despite imperfect latent predictions, we inform a GPU-accelerated system level synthesis (SLS) robust MPC scheme with conformal prediction to obtain calibrated latent error bounds and robust latent-space constraint sets. We further learn and conformalize a latent constraint checker, allowing the SLS planner to impose probabilistic safety constraints during closed-loop execution. We evaluate our method on vision-based control tasks, where it improves both goal-reaching performance and safety over latent world-model and safe-planning baselines.

Chinese Translation

我们提出了 SLS^2，这是一个基于像素的安全反馈运动规划框架，利用在学习的潜在世界模型中进行鲁棒模型预测控制（MPC）。我们的方法训练了一个动作条件的联合嵌入世界模型，该模型具有紧凑的马尔可夫潜在状态，从而通过学习的潜在动态实现高效的基于梯度的轨迹优化。为了在真实系统中强制执行安全性，尽管潜在预测不完美，我们通过保形预测来通知一个基于 GPU 加速的系统级合成（SLS）鲁棒 MPC 方案，以获得校准的潜在误差界限和鲁棒的潜在空间约束集。我们进一步学习并保形化一个潜在约束检查器，使得 SLS 规划器能够在闭环执行过程中施加概率安全约束。我们在基于视觉的控制任务上评估了我们的方法，结果表明其在目标达成性能和安全性上均优于潜在世界模型和安全规划基线。

View on arXiv Download PDF AI Translation

cs.RO / 39 / 2606.15631

Retrieve, Don't Retrain: Extending Vision Language Action Models to New Tasks at Test Time

检索，而非重训练：在测试时将视觉语言动作模型扩展到新任务

Park, Jeongeun, Park, Juhan, Kim, Taekyung, Choi, Sungjoon, Han, Dongyoon, Yun, Sangdoo

Abstract

Extending a vision-language-action (VLA) policy to a new task typically requires task-specific teleoperated demonstrations and per-task fine-tuning, making adaptation costly in both data collection and compute. In this paper, we show that this target-side per-task adaptation cost can be replaced by retrieval. Our retrieval-augmented policy is trained once on paired demonstrations from the target embodiment (query) and a cheaper embodiment (pool, e.g., human-hand video), then frozen. New tasks are added at deployment by appending pool-side demonstrations to a retrieval pool. The frozen policy conditions on retrieved trajectories at every control step, so new tasks are absorbed by indexing data rather than updating parameters. Fine-tuning is needed only to take on a new, unseen embodiment, not for each new task. We show that retrieval improves policies beyond a specific backbone, including standard VLA policies, but its effect is especially pronounced in Cosmos Policy, a video-generation-based world-action model (WAM). In this setting, retrieval supplies coarse task progression, while the WAM's future-image objective provides an additional visual consistency signal that strengthens the retrieval-conditioned actions. On PushT, we study how retrieval provides a reusable high-level motion prior for cross-embodiment generalization to unseen goal angles, while on RoboTwin 2.0 our method outperforms cross-embodiment baselines on unseen tasks, and we additionally demonstrate the method on a real robot.

Chinese Translation

将视觉语言动作（VLA）策略扩展到新任务通常需要特定任务的遥控演示和逐任务的微调，这使得适应过程在数据收集和计算上都变得昂贵。本文展示了这种目标侧逐任务适应成本可以通过检索来替代。我们的检索增强策略在目标体现（查询）和更便宜的体现（池，例如人手视频）上的配对演示上训练一次，然后冻结。在部署时，通过将池侧演示附加到检索池中来添加新任务。冻结的策略在每个控制步骤中基于检索到的轨迹进行条件处理，因此新任务通过索引数据而不是更新参数来吸收。微调仅在面对新的、未见过的体现时需要，而不是针对每个新任务。我们展示了检索在特定骨干网络之外改善策略的效果，包括标准的VLA策略，但其效果在基于视频生成的世界动作模型（WAM）Cosmos Policy中尤为明显。在这种设置中，检索提供了粗略的任务进展，而WAM的未来图像目标提供了额外的视觉一致性信号，从而增强了基于检索的动作。在PushT上，我们研究了检索如何为跨体现泛化到未见目标角度提供可重复使用的高层运动先验，而在RoboTwin 2.0上，我们的方法在未见任务上超越了跨体现基线，并且我们还在真实机器人上演示了该方法。

View on arXiv Download PDF AI Translation

cs.RO / 40 / 2606.15645

TO-SoFiT: Topology Optimization of Hydraulic Soft Fish Tail Design for programmable undulating locomotion

TO-SoFiT：可编程波动运动的液压软鱼尾设计的拓扑优化

Padmaprabhan, A, Shaji, Amal, Kumar, Prabhat

Abstract

Soft robots leverage compliant materials to generate motion through controlled elastic deformation, making them ideal for delicate tasks such as underwater exploration and biomimetic marine systems. Although hydraulic/pneumatic actuation remains pivotal for such systems, the lack of systematic design frameworks has hindered the development of robots capable of complex 3D motion, such as fish-like swimming. This work introduces a topology optimization method to automate the design of a hydraulic soft fish tail, explicitly addressing the design-dependent coupling between fluidic actuation and structural deformation. We use a Darcy law-based model augmented with a drainage term to simulate spatially varying hydraulic pressure loads, translating these into consistent nodal forces via finite element analysis. The employed robust multi-criteria optimization formulation balances deformation efficiency, fluid-structure interaction, geometric manufacturability, and required stiffness for optimizing a bioinspired soft fish tail for 3D swimming kinematics. The optimized tail topology is incorporated into a pneumatic network actuator and computationally validated under various hydraulic loads, achieving tunable undulatory amplitudes and multiaxis bending for depth adjustment. The optimized 2D tail outperforms its rectangular counterpart. By cascading optimized tail segments, we demonstrate programmable swimming patterns in soft robotic fish tails at different hydraulic loads. This work advances the systematic codesign of hydraulic actuators and soft structures, offering a pathway to automate underwater robots with optimized design and vertebrate-like agility in confined aquatic environments. Our implementations and simulations are publicly available at 'https://github.com/PrabhatIn/TO-SoFiT'.

Chinese Translation

软体机器人利用柔性材料通过控制弹性变形产生运动，使其在水下探索和仿生海洋系统等精细任务中表现出色。尽管液压/气动驱动在这些系统中仍然至关重要，但缺乏系统化的设计框架限制了能够实现复杂三维运动（如鱼类游动）的机器人的发展。本研究提出了一种拓扑优化方法，以自动化液压软鱼尾的设计，明确解决了流体驱动与结构变形之间的设计依赖耦合。我们使用基于达西定律的模型，并增加排水项，以模拟空间变化的液压压力载荷，并通过有限元分析将其转化为一致的节点力。所采用的稳健多标准优化公式在优化仿生软鱼尾的三维游动运动学时，平衡了变形效率、流体-结构相互作用、几何可制造性和所需刚度。优化后的尾部拓扑被纳入气动网络驱动器，并在各种液压载荷下进行计算验证，实现了可调的波动幅度和多轴弯曲以进行深度调整。优化后的二维尾部性能优于其矩形对应物。通过级联优化的尾部段，我们展示了在不同液压载荷下软体机器人鱼尾的可编程游动模式。本研究推动了液压驱动器与软结构的系统化协同设计，为自动化水下机器人提供了一条优化设计和类脊椎动物灵活性在狭窄水域环境中的实现路径。我们的实现和模拟结果已公开可用，网址为 'https://github.com/PrabhatIn/TO-SoFiT'。

View on arXiv Download PDF AI Translation

cs.RO / 41 / 2606.15654

PO-PDDL: Learning Symbolic POMDPs from Visual Demonstrations for Robot Planning Under Uncertainty

PO-PDDL：从视觉示范中学习符号化部分可观测马尔可夫决策过程以应对不确定性下的机器人规划

Tang, Wenjing, Jin, Xuanjin, Liu, Yuan, Huang, Renming, Lu, Cewu, Cai, Panpan

Abstract

Real-world robot task planning must operate under both stochastic action execution and partial observability, yet constructing Partially Observable Markov Decision Process (POMDP) models for real robotics domains remains difficult and labor-intensive. We introduce PO-PDDL, a symbolic formulation of POMDPs that preserves the relational structure and LLM-friendly syntax of the Planning Domain Definition Language (PDDL), while explicitly modeling partial observability, stochasticity, and beliefs. Building on this formulation, we propose a demonstration-driven pipeline for learning PO-PDDL models. The proposed method reconstructs latent symbolic state trajectories from real-robot execution videos, identifies partial observability via inconsistencies between inferred states and visual observations, and learns stochastic transition and observation models accordingly. The resulting PO-PDDL domains are reusable across tasks and enable online belief-space planning under both perception and execution uncertainty. Experiments on real-world long-horizon manipulation tasks show that our method consistently outperforms existing PDDL and POMDP model-learning approaches, achieving robust task planning under uncertainty with significantly lower planning cost.

Chinese Translation

现实世界中的机器人任务规划必须在随机动作执行和部分可观测性下进行，然而为真实机器人领域构建部分可观测马尔可夫决策过程（POMDP）模型仍然困难且劳动密集。我们提出了PO-PDDL，这是一种符号化的POMDP表述，保留了规划领域定义语言（PDDL）的关系结构和适合大型语言模型（LLM）的语法，同时明确建模部分可观测性、随机性和信念。在此基础上，我们提出了一种基于示范的管道，用于学习PO-PDDL模型。该方法从真实机器人执行视频中重建潜在的符号状态轨迹，通过推断状态与视觉观察之间的不一致性识别部分可观测性，并相应地学习随机转移和观察模型。生成的PO-PDDL领域可在不同任务中重复使用，并能够在感知和执行不确定性下进行在线信念空间规划。在真实世界的长时间操作任务上的实验表明，我们的方法在不确定性下的任务规划中始终优于现有的PDDL和POMDP模型学习方法，显著降低了规划成本，达到了稳健的任务规划效果。

View on arXiv Download PDF AI Translation

cs.RO / 42 / 2606.15685

Learning New Tasks via Reusable Skills: Skill-Compositional Experts for Embodied Continual Learning

通过可重用技能学习新任务：用于具身持续学习的技能组合专家

Zhang, Shuaike, Wang, Shaokun, Tang, Haoyu, Wu, Jianlong, Nie, Liqiang

Abstract

Embodied Continual Learning (ECL) aims to enable robots to continually acquire new manipulation tasks while retaining previously learned behaviors under closed-loop control. Compared with conventional continual learning, ECL suffers from more severe catastrophic forgetting. Feature drift accumulated under closed-loop control progressively propagates through sequential decision-making, leading to degradation of previously learned behaviors. A key challenge in ECL lies in structured skill reuse across continually evolving tasks, since existing methods primarily focus on skill learning without explicitly organizing them for coherent task execution. To address this issue, we propose SCE, a Skill-Compositional Experts framework for ECL. SCE builds a skill base via Compositional Skill Grounding (CSG), which decomposes task demonstrations into reusable skills. Based on this, Dual Execution-and-Transition Experts (DETE) enable new task learning through skill composition, where one branch ensures skill execution and the other supports transitions between skills for coherent behavior. Experiments on LIBERO benchmarks and real-world manipulation tasks demonstrate that SCE consistently improves retention and overall task performance. Further feature drift analyses and ablation studies verify the effectiveness of our method. Project website: https://eqcy.github.io/sce/.

Chinese Translation

具身持续学习（ECL）旨在使机器人能够在保持先前学习行为的同时，持续获取新的操作任务，且在闭环控制下进行。与传统的持续学习相比，ECL面临更严重的灾难性遗忘。在闭环控制下积累的特征漂移通过连续决策过程逐步传播，导致先前学习行为的退化。ECL中的一个关键挑战在于在不断演变的任务中进行结构化技能重用，因为现有方法主要关注技能学习，而未明确组织这些技能以实现连贯的任务执行。为了解决这一问题，我们提出了SCE（技能组合专家）框架用于ECL。SCE通过组合技能基础（CSG）构建技能库，该方法将任务演示分解为可重用的技能。在此基础上，双执行与过渡专家（DETE）通过技能组合实现新任务学习，其中一个分支确保技能执行，另一个分支支持技能之间的过渡以实现连贯行为。在LIBERO基准和真实世界操作任务上的实验表明，SCE始终提高了保留率和整体任务表现。进一步的特征漂移分析和消融研究验证了我们方法的有效性。项目网站：https://eqcy.github.io/sce/

View on arXiv Download PDF AI Translation

cs.RO / 43 / 2606.15691

Can Causal Models Enhance Robot Navigation? Online Causal Adaptation for Real-Robot Navigation

因果模型能否增强机器人导航？针对真实机器人导航的在线因果适应

Liang, Zhitao, Mitrevski, Alex, Dean, Emmanuel, Ramirez-Amaro, Karinne

Abstract

Causality in robotics aims to produce more interpretable and flexible robot behaviours by enabling robots to predict the consequences of their actions; however, deploying causal models with existing systems (e.g., navigation) operating in real environments remains understudied. This paper addresses the challenging problem of transferring causal models in real-robot experiments for a navigation scenario. We study this problem in two ways: (i) using the causal model as an offline evaluation module that predicts the competence of recorded real-robot navigation trajectories and relates it to quantitative navigation performance, and (ii) using the causal model as an online adaptation module that intervenes when the predicted competence of the default navigation is low. We validate our approach in a physical service robot that patrols around corridors. We show that the predicted competence correlates positively with path efficiency, and negatively with path irregularities (suboptimal behaviour). The model predictions also show strong agreement with human annotations (Cohen's kappa value of 0.88). In online experiments, the proposed method improves navigation performance in complex scenarios such as cornering and obstacle avoidance, yielding higher predicted competence and better navigation metrics than the default navigation baseline. In simpler scenarios, where the baseline already performs near-optimally, the causal adaptation provides limited benefit. These results indicate that causal models are particularly effective in enhancing navigation under increased task complexity. Overall, our results demonstrate that causal models developed for behavioural interpretation can be successfully integrated into real-robot navigation systems.

Chinese Translation

机器人学中的因果关系旨在通过使机器人能够预测其行为的后果，从而产生更具可解释性和灵活性的机器人行为；然而，将因果模型应用于在真实环境中运行的现有系统（例如导航）仍然是一个未被充分研究的问题。本文解决了在真实机器人实验中将因果模型转移到导航场景的挑战性问题。我们从两个方面研究这个问题：（i）将因果模型作为离线评估模块，预测记录的真实机器人导航轨迹的能力，并将其与定量导航性能相关联；（ii）将因果模型作为在线适应模块，当默认导航的预测能力较低时进行干预。我们在一台在走廊巡逻的物理服务机器人上验证了我们的方法。我们显示，预测的能力与路径效率呈正相关，与路径不规则性（次优行为）呈负相关。模型预测与人类标注之间也表现出强一致性（Cohen's kappa值为0.88）。在在线实验中，所提出的方法在拐角和避障等复杂场景中改善了导航性能，预测能力更高，导航指标优于默认导航基线。在更简单的场景中，基线已经接近最优，因果适应的益处有限。这些结果表明，因果模型在提高任务复杂性下的导航效果方面特别有效。总体而言，我们的结果表明，为行为解释而开发的因果模型可以成功集成到真实机器人导航系统中。

View on arXiv Download PDF AI Translation

cs.RO / 44 / 2606.15768

LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies

LaWAM：用于高效动态感知机器人策略的潜在世界动作模型

Chen, Jialei, Wang, Kai, Chen, Kang, Chen, Shuaihang, Gao, Feng, Tang, Wenhao, Li, Zhiyuan, Liu, Weilin, Yao, Zhuyu, Li, Boxun, Xu, Yuanbo, Yu, Chao

Abstract

Vision-Language-Action models (VLAs) leverage large-scale vision-language pretraining for semantic robot control, but often lack explicit foresight into how robot actions change the scene. World-Action Models (WAMs) address this limitation by conditioning policies on predicted futures, yet existing approaches typically rely on computationally expensive video generation with substantial pixel-level redundancy. We present LaWAM, a Latent World Action Model that exposes predictive dynamics to robot policies through compact latent visual subgoals instead of reconstructed future video. At the core of LaWAM is a latent-action-conditioned Latent World Model (LaWM). We obtain LaWM by training a latent action model in the latent space of a pretrained vision foundation model and repurposing its forward decoder to predict future observation features for scene evolution. LaWAM then conditions action generation on these predicted latent visual subgoals to enable dynamics-aware robot control. LaWAM achieves state-of-the-art or competitive success rates (SRs) across LIBERO (98.6% SR), RoboTwin (91.22% SR), and real-world manipulation tasks while retaining low-latency inference. LaWAM runs in 187 ms per action-chunk prediction and achieves up to 24x lower wall-clock latency than pixel-space WAMs.

Chinese Translation

视觉-语言-动作模型（VLAs）利用大规模的视觉-语言预训练进行语义机器人控制，但通常缺乏对机器人动作如何改变场景的明确预见。世界动作模型（WAMs）通过将策略与预测的未来进行条件化来解决这一限制，然而现有的方法通常依赖于计算成本高昂的视频生成，并存在大量像素级冗余。我们提出了LaWAM，一种潜在世界动作模型，它通过紧凑的潜在视觉子目标而不是重建的未来视频，将预测动态暴露给机器人策略。LaWAM的核心是一个潜在动作条件的潜在世界模型（LaWM）。我们通过在预训练视觉基础模型的潜在空间中训练潜在动作模型，并重新利用其前向解码器来预测场景演变的未来观察特征，从而获得LaWM。然后，LaWAM将动作生成条件化于这些预测的潜在视觉子目标，以实现动态感知的机器人控制。LaWAM在LIBERO（98.6%成功率）、RoboTwin（91.22%成功率）和真实世界操控任务中实现了最先进或具有竞争力的成功率，同时保持低延迟推理。LaWAM每个动作块预测运行时间为187毫秒，并且在墙钟延迟方面比像素空间的WAMs低达24倍。

View on arXiv Download PDF AI Translation

cs.RO / 45 / 2606.15846

FlashNav: Ultra-Fast Policy Training for Robot Navigation within 20 Seconds

FlashNav：20秒内超快速机器人导航策略训练

Wang, Shanze, Qian, Yiwei, Zhang, Xinming, Xue, Jun, Cheng, Siwei, Wang, Xianghui, Hu, Qingyuan, Shen, Xiaoyu, Zhang, Wei

Abstract

Deep reinforcement learning has shown strong potential for robot navigation, but its practical deployment is still limited by the long wall-clock cost of policy training. This paper presents FlashNav, a GPU-first framework for ultra-fast range-based robot navigation training. To the best of our knowledge, FlashNav is the first DRL-based robot navigation framework that reaches seconds-level policy training, with the fastest deployable policy trained in less than 20 seconds. The key idea is to align simulation with the navigation MDP: FlashNav preserves the essential components for velocity-level navigation, including occupancy geometry, range sensing, goal-conditioned control, robot motion dynamics, collision handling, termination, and reset, while removing unnecessary rendering and high-fidelity physical details from the training loop. Built on a batched bitmap simulator and a fully GPU-resident training pipeline with our FastDSAC learner, FlashNav generates massive parallel navigation transitions entirely on GPU. Experiments on TurtleBot2 and Unitree Go2 show that FlashNav achieves a 100\% success-rate below 20 seconds on an RTX 5090 and remains within tens of seconds across desktop GPUs. The learned policies further transfer to physical wheeled and legged robots in static and dynamic indoor scenes, demonstrating that DRL-based navigation can be trained at seconds-level speed while preserving deployable obstacle-avoidance behavior.

Chinese Translation

深度强化学习在机器人导航中展现出强大的潜力，但其实际应用仍受到策略训练长时间壁钟成本的限制。本文提出了FlashNav，一个以GPU为主的超快速基于范围的机器人导航训练框架。据我们所知，FlashNav是第一个实现秒级策略训练的基于深度强化学习（DRL）的机器人导航框架，其最快可在不到20秒内训练出可部署的策略。其关键思想是将仿真与导航马尔可夫决策过程（MDP）对齐：FlashNav保留了速度级导航所需的基本组件，包括占用几何、范围感知、目标条件控制、机器人运动动态、碰撞处理、终止和重置，同时去除了训练循环中不必要的渲染和高保真物理细节。FlashNav建立在批处理位图模拟器和完全GPU驻留的训练管道（配合我们的FastDSAC学习器）之上，能够在GPU上生成大量并行的导航过渡。在TurtleBot2和Unitree Go2上的实验表明，FlashNav在RTX 5090上实现了100%的成功率，且在桌面GPU上保持在几十秒以内。所学习的策略进一步迁移到静态和动态室内场景中的物理轮式和腿式机器人，证明了基于DRL的导航可以以秒级速度进行训练，同时保持可部署的避障行为。

View on arXiv Download PDF AI Translation

cs.RO / 46 / 2606.15896

LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors

LoComposition：无步态先验的地形自适应节能四足运动

Kordos, Loukas, Franz, Leonard T., Rappenecker, Simon, Hausdoerfer, Oliver, Schoellig, Angela P., Kolev, Pavel, Martius, Georg

Abstract

Learning-based quadrupedal locomotion typically relies on complex reward formulations that entangle task specification, operational limits, gait preference, and terrain adaptation within a single optimization objective. We instead treat these functions through distinct mechanisms: rewards for task specification, constraints for operational limits, energy minimization for gait preference, and exteroceptive perception for adapting energy use to terrain difficulty. We show that these components jointly enable efficient, terrain-adaptive locomotion, and that removing each component exposes a distinct failure mode. Our formulation removes explicit gait priors (including air-time, contact-count, and foot-clearance targets) in favor of emergent behavior. Compared to a conventional complex-reward baseline, our formulation achieves comparable terrain traversal while reducing cost of transport by 56% and operational-limit violations by 96%. The resulting policies transfer zero-shot to a physical Unitree Go2 using LiDAR-based elevation mapping. Project website with videos: https://tinyurl.com/locomposition.

Chinese Translation

基于学习的四足运动通常依赖于复杂的奖励公式，这些公式将任务规范、操作限制、步态偏好和地形适应性纠缠在一个单一的优化目标中。相反，我们通过不同的机制来处理这些功能：任务规范的奖励、操作限制的约束、步态偏好的能量最小化，以及外部感知以适应地形难度的能量使用。我们展示了这些组件共同实现高效的地形自适应运动，并且去除每个组件会暴露出不同的失败模式。我们的公式去除了显式的步态先验（包括空中时间、接触次数和足部间隙目标），而是倾向于涌现行为。与传统的复杂奖励基线相比，我们的公式在实现可比的地形穿越的同时，将运输成本降低了56%，操作限制违规情况减少了96%。最终的策略能够零样本迁移到使用基于激光雷达的高程映射的物理Unitree Go2上。项目网站及视频链接： https://tinyurl.com/locomposition。

View on arXiv Download PDF AI Translation

cs.RO / 47 / 2606.15898

VL2Spike: Spike-driven Distillation from VLMs for Low-Power Visual Perception in Embodied AI

VL2Spike：基于脉冲驱动的知识蒸馏框架，用于低功耗的具身人工智能视觉感知

Liu, Zinan, Zheng, Eric, Debnath, Soumyaratna, Shi, Hao, Xiao, Ling, Wang, Lin

Abstract

Spiking neural networks (SNNs) are brain-inspired, event-driven models that compute with sparse spikes, which enables highly efficient visual perception in resource-constrained embodied AI models. The emergence of Spiking-Transformer models with spike self-attention has substantially improved the learning capacity of pure SNNs. Although SNNs are energy efficient, their performance is still limited by the spike-based architecture and optimization challenges, as standard gradient descent rules cannot be directly applied. Recently, vision-language models (VLMs) have shown rich multi-modal knowledge representation capabilities for visual perception. Thus, it is promising to leverage VLMs for better Spikformer training. To this end, we present VL2Spike, a novel spike-based knowledge distillation (KD) framework that bridges multi-modal knowledge from VLMs with compact Spikformer models. This design enhances the learning capacity of Spikformer models while preserving their energy-efficiency merits, thereby offering a practical pathway toward low-power robotic perception. Our VL2Spike brings two key technical contributions. To align with spiking dynamics, we first propose spatial-temporal visual spike (SVS) distillation, which achieves (1) shared manifold alignment between VLM image features and spike tokens, and (2) warm-started temporal consistency on membrane potentials and spike rates. We then design a novel spike prototype-guided linguistic (SPL) distillation strategy that aligns Spikformer's class prototypes and logits with promptable VLM text embeddings. Extensive experiments show that VL2Spike achieves 6.81% gain across three static datasets with only 15.7% energy consumption. It also exhibits strong generalization capacity on robotic visual place recognition (VPR) with a gain of 6.63%, highlighting its potential for low-power perception in embodied AI.

Chinese Translation

脉冲神经网络（SNNs）是一种受大脑启发的事件驱动模型，利用稀疏脉冲进行计算，从而在资源受限的具身人工智能模型中实现高效的视觉感知。具有脉冲自注意力机制的脉冲变换器（Spiking-Transformer）模型的出现显著提高了纯脉冲神经网络的学习能力。尽管脉冲神经网络在能量效率上表现优异，但其性能仍受到脉冲基础架构和优化挑战的限制，因为标准的梯度下降规则无法直接应用。最近，视觉-语言模型（VLMs）在视觉感知方面展现了丰富的多模态知识表示能力。因此，利用视觉-语言模型来改善脉冲变换器的训练是一个有前景的方向。为此，我们提出了VL2Spike，一个新颖的基于脉冲的知识蒸馏（KD）框架，旨在将视觉-语言模型的多模态知识与紧凑的脉冲变换器模型相结合。该设计在保留脉冲变换器模型能量效率优点的同时，增强了其学习能力，从而为低功耗的机器人感知提供了一条实用的路径。我们的VL2Spike带来了两个关键的技术贡献。首先，为了与脉冲动态对齐，我们提出了时空视觉脉冲（SVS）蒸馏方法，该方法实现了（1）视觉-语言模型图像特征与脉冲标记之间的共享流形对齐，以及（2）膜电位和脉冲率的温启动时间一致性。然后，我们设计了一种新颖的脉冲原型引导语言（SPL）蒸馏策略，使脉冲变换器的类别原型和逻辑与可提示的视觉-语言模型文本嵌入对齐。大量实验表明，VL2Spike在三个静态数据集上实现了6.81%的性能提升，同时仅消耗15.7%的能量。它在机器人视觉位置识别（VPR）任务上也展现了强大的泛化能力，提升幅度为6.63%，突显了其在具身人工智能中低功耗感知的潜力。

View on arXiv Download PDF AI Translation

cs.RO / 48 / 2606.15909

GeoTLM: Geometry-aware Tactile-Language Models for Contact Motion Orientation Reasoning of Dynamic Objects

GeoTLM：面向动态物体接触运动方向推理的几何感知触觉语言模型

Li, Qiutian, Liu, Zinan, Wang, Lin

Abstract

Modern tactile-language models (TLMs) have shown potential for robot learning tasks, such as material and texture recognition. However, for contact-rich scenarios, these TLMs struggle to understand the physical properties of dynamic objects, such as rotation and sliding directions. For instance, our preliminary experiments reveal that popular TLMs, such as Sparsh and AnyTouch2, exhibit weak performance on basic rotation direction reasoning from GelSight Mini tactile data. This surprising gap inspires us to explore a novel research question: Can we inject physically grounded geometric priors into TLMs to enable reliable contact orientation reasoning of dynamic object properties? To this end, we propose GeoTLM, a novel geometric representation-guided TLM for the perception of dynamic contact events. Our key idea is to preserve and structure tactile shear-field geometry before language-level reasoning, rather than forcing low-resolution tactile tokens into fragile closed-form physics operators. To achieve this, we propose a lightweight (only 14k parameters) yet novel Differentiable Geometric Representation (DGR). Specifically, DGR learns a contact-mask-guided representation in the shear field and aggregates it through an antisymmetric seven-region pooling design, motivated by the physical intuition that rotational contact produces antisymmetric deformation patterns. We conduct experiments on two representative tasks: rotation direction and sliding direction reasoning. Extensive experiments show that GeoTLM improves novel-object rotation accuracy by +14.6% and real-sensor sliding accuracy by +16.2% over the same backbone without the geometric encoder. Overall, our work paves a new way for physically grounded tactile-language reasoning, with strong potential for dynamic object understanding and contact-rich robotic manipulation.

Chinese Translation

现代触觉语言模型（TLMs）在机器人学习任务中展现了潜力，例如材料和纹理识别。然而，在接触丰富的场景中，这些TLMs在理解动态物体的物理属性（如旋转和滑动方向）方面存在困难。例如，我们的初步实验表明，流行的TLMs，如Sparsh和AnyTouch2，在基于GelSight Mini触觉数据的基本旋转方向推理上表现较弱。这一令人惊讶的差距促使我们探索一个新颖的研究问题：我们能否将物理基础的几何先验注入TLMs，以实现对动态物体属性的可靠接触方向推理？为此，我们提出了GeoTLM，一种新颖的几何表示引导的TLM，用于动态接触事件的感知。我们的核心思想是在语言层推理之前保留和结构化触觉剪切场几何，而不是将低分辨率的触觉标记强行输入脆弱的封闭形式物理算子。为实现这一目标，我们提出了一种轻量级（仅14k参数）但新颖的可微几何表示（DGR）。具体而言，DGR在剪切场中学习一种接触掩膜引导的表示，并通过反对称的七区域池化设计进行聚合，这一设计受到物理直觉的启发，即旋转接触会产生反对称变形模式。我们在两个代表性任务上进行了实验：旋转方向和滑动方向推理。大量实验表明，GeoTLM在新物体旋转准确性上提高了14.6%，在真实传感器滑动准确性上提高了16.2%，相较于没有几何编码器的相同骨干网络。总体而言，我们的工作为物理基础的触觉语言推理开辟了一条新路径，具有强大的动态物体理解和接触丰富的机器人操作的潜力。

View on arXiv Download PDF AI Translation

cs.RO / 49 / 2606.15915

Identification of a Physics-Based Electrical Power Consumption Model for the Unitree G1 Humanoid Arm

基于物理的Unitree G1人形机器人手臂电力消耗模型的识别

Deniz, Nestor N., Vega, Sebastian, Parsons, Simon, Cheein, Fernando Auat

Abstract

Accurate prediction of electrical power consumption is essential for energy-aware motion planning, battery management, and thermal monitoring in battery-powered humanoid robots. This letter presents a physics-based, linear-in-parameters model for the electrical power consumption of the seven-degree-of-freedom left arm of the Unitree~G1 humanoid robot. The proposed formulation combines actuator loss terms with a baseline-torque correction that captures changes in gravity-compensation load and enables accurate prediction of negative net power trajectories. Pairwise interaction terms are introduced to model power coupling during simultaneous multi-joint motion. Model parameters are identified from experimental data collected on a physical Unitree~G1 using onboard power measurements as the regression target. Across 897 trajectories covering single-joint and coordinated arm motions at multiple speed levels, the identified model achieves $R^2 = 0.933$ with an RMSE of 1.07 (W). Validation on 46 trajectories executed at previously unseen speeds yields $R^2 = 0.965$, demonstrating strong generalisation beyond the identification dataset. Analysis of the identified parameters reveals distinct power-consumption characteristics across the arm, with viscous friction dominating most joints (shoulder pitch and all three wrist joints), copper losses dominating shoulder yaw and the elbow, and shoulder roll uniquely dominated by Coulomb friction.

Chinese Translation

准确预测电力消耗对于电池供电的人形机器人在能量感知运动规划、电池管理和热监测方面至关重要。本文提出了一种基于物理的线性参数模型，用于描述Unitree G1人形机器人七自由度左臂的电力消耗。所提模型结合了执行器损耗项与基线扭矩修正，能够捕捉重力补偿负载的变化，并实现对负净功率轨迹的准确预测。引入成对交互项以建模在同时多关节运动期间的功率耦合。模型参数通过在物理Unitree G1上收集的实验数据进行识别，以机载功率测量作为回归目标。在覆盖单关节和协调臂运动的897条轨迹中，识别出的模型达到了 $R^2 = 0.933$，均方根误差（RMSE）为1.07（W）。在46条以之前未见速度执行的轨迹上的验证结果显示 $R^2 = 0.965$，表明模型在识别数据集之外具有良好的泛化能力。对识别参数的分析揭示了手臂各部分的不同电力消耗特征，其中粘性摩擦主导大多数关节（肩部俯仰和所有三个腕关节），铜损主要影响肩部偏航和肘部，而肩部滚动则独特地受到库仑摩擦的主导。

View on arXiv Download PDF AI Translation

cs.RO / 50 / 2606.15918

Energy-Efficient Arm Reaching for a Humanoid Robot via Deep Reinforcement Learning with Identified Power Models

通过深度强化学习与识别的功率模型实现类人机器人能效臂伸展

Deniz, Nestor N., Parsons, Simon, Cheein, Fernando Auat

Abstract

Humanoid robots performing in-field manipulation tasks, such as robotic apple harvesting, face severe energy constraints that directly limit the number of reaching motions that can be executed per battery charge. This paper presents an end-to-end, energy-aware reinforcement learning framework for the 7-degree-of-freedom left arm of the Unitree~G1 humanoid robot, combining a physics-based, experimentally identified electrical power model with a Soft Actor-Critic (SAC) policy trained in a Pinocchio-based rigid-body dynamics simulator. The RL policy operates on an incremental joint-position action space and is trained with a Hybrid Constellation Reward that combines a four-point end-effector constellation distance with a torque-norm energy proxy; after % $5\times10^6$ training it reaches a $69.9\%$ success rate over $1\,000$ random targets in kinematic simulation, at a mean energy of \SI{98.16}{\joule} on successful episodes. Finally, on the physical Unitree~G1, the policy is validated over three independent 10-target batches, achieving a mean energy of $71.5 \pm 48.3$\,J, an end-effector position error of $2.64 \pm 1.04$\,cm, and an orientation error of $6.92 \pm 1.33^\circ$ -- within the \SI{4}{\centi\metre}/$8.6^\circ$ training tolerance. These results constitute a first step toward energy-aware reinforcement-learning-based arm reaching for humanoid robots.

Chinese Translation

在进行现场操作任务的类人机器人（如机器人苹果采摘）面临严重的能量限制，这直接限制了每次电池充电后可执行的伸展动作数量。本文提出了一种端到端的、关注能量的强化学习框架，针对Unitree~G1类人机器人的7自由度左臂，结合了基于物理的、实验识别的电力模型与在基于Pinocchio的刚体动力学模拟器中训练的Soft Actor-Critic (SAC) 策略。该强化学习策略在增量关节位置动作空间中运行，并使用混合星座奖励进行训练，该奖励结合了四点末端执行器星座距离与扭矩范数能量代理；经过$5 imes10^6$次训练后，在运动学仿真中对$1,000$个随机目标的成功率达到$69.9 d$，成功回合的平均能量为 ext{98.16} ext{J}。最后，在物理Unitree~G1上，该策略在三个独立的10目标批次中进行了验证，平均能量为$71.5 ext{J} ext{±} 48.3 ext{J}$，末端执行器位置误差为$2.64 ext{cm} ext{±} 1.04 ext{cm}$，方向误差为$6.92^ ext{°} ext{±} 1.33^ ext{°}$，均在 ext{4 cm}/$8.6^ ext{°}$的训练容忍度范围内。这些结果构成了基于能量意识的强化学习在类人机器人臂伸展中的初步探索。

View on arXiv Download PDF AI Translation

cs.RO / 51 / 2606.15930

ControlMap: Controllable High-Definition Map Generation for Traffic Scenario Simulation

ControlMap：可控的高分辨率地图生成用于交通场景模拟

Farag, Marwan, Wäldele, Steffen, Yao, Yu

Abstract

Simulation is central to validating autonomous driving systems, yet current pipelines are limited by insufficient scenario diversity due to costly High Definition (HD) map creation. Scaling HD maps requires expensive data collection and manual processing. Moreover, existing generative models lack the fine-grained control necessary to target specific road topologies during generation. This paper presents a data-driven pipeline for controllable HD map generation using latent diffusion and ControlNet for spatial conditioning. To our knowledge, we are the first to inject spatial guidance signals into a diffusion model for HD map synthesis. Furthermore, our model supports adjustable conditioning strength through classifier-free guidance and city-level style transfer via city label conditioning. To complement existing metrics, we introduce two novel metrics to evaluate adherence to the control signal and similarity to ground-truth maps. Experiments demonstrate that our model generates realistic HD maps that faithfully follow input road topologies while accurately preserving city-specific details.

Chinese Translation

模拟是验证自动驾驶系统的核心，然而目前的流程由于高成本的高分辨率（HD）地图创建而受到场景多样性不足的限制。扩展HD地图需要昂贵的数据收集和手动处理。此外，现有的生成模型缺乏在生成过程中针对特定道路拓扑进行精细控制的能力。本文提出了一种基于数据驱动的可控HD地图生成管道，使用潜在扩散（latent diffusion）和ControlNet进行空间条件设置。我们首次将空间引导信号注入扩散模型以合成HD地图。此外，我们的模型通过无分类器引导（classifier-free guidance）支持可调的条件强度，并通过城市标签条件实现城市级风格转移。为了补充现有的评估指标，我们引入了两个新指标来评估对控制信号的遵循程度和与真实地图的相似性。实验表明，我们的模型生成的HD地图在忠实遵循输入道路拓扑的同时，准确保留了城市特定的细节。

View on arXiv Download PDF AI Translation

cs.RO / 52 / 2606.15997

Friction Characterization of a Cable-Driven Differential Actuation System for Lower-Limb Exoskeletons

下肢外骨骼的电缆驱动差动驱动系统的摩擦特性研究

Nobili, Alberto Maria, Salsedo, Fabio, Filippeschi, Alessandro

Abstract

Lower-limb exoskeletons require actuation systems that can provide accurate joint torque control while preserving low mass and encumbrance. Conventional architectures often rely on independently actuated joints and joint-level torque sensors, increasing system complexity and weight. This paper presents a novel differential actuation architecture for hip-knee flexion/extension, enabling cooperative torque sharing between two motors via a linear differential mapping between motor and joint. To compensate for transmission losses, a model-based friction estimation strategy is developed and experimentally implemented, allowing accurate joint torque estimation without the need for torque sensors. The proposed solution is validated on a physical prototype, demonstrating the feasibility of sensorless torque estimation in a differentially actuated hip-knee module of a lower-limb exoskeleton.

Chinese Translation

下肢外骨骼需要能够提供精确关节扭矩控制的驱动系统，同时保持低质量和低负担。传统架构通常依赖于独立驱动的关节和关节级扭矩传感器，这增加了系统的复杂性和重量。本文提出了一种新颖的差动驱动架构，用于髋关节-膝关节的屈伸，能够通过电机与关节之间的线性差动映射实现两个电机之间的协同扭矩共享。为了补偿传动损失，开发并实验性实施了一种基于模型的摩擦估计策略，使得在无需扭矩传感器的情况下能够准确估计关节扭矩。所提出的解决方案在物理原型上进行了验证，展示了在下肢外骨骼的差动驱动髋关节-膝关节模块中无传感器扭矩估计的可行性。

View on arXiv Download PDF AI Translation

cs.RO / 53 / 2606.16022

$\lambda$-Reachability: Geometric-Horizon Safety Bellman Equations for Humanoid Safety

$ ext{λ}$-可达性：用于类人安全的几何视界安全贝尔曼方程

Chen, Rui, Li, Shangtao, Sun, Yifan, Liu, Changliu

Abstract

We introduce $\lambda$-Reachability, a scalable approach to Hamilton--Jacobi safety analysis for high-dimensional robotic systems. Unlike prior discounted formulations that rely on fixed one-step Bellman updates, $\lambda$-Reachability employs a stochastic multi-step estimator of the safety value, using a geometrically distributed rollout horizon together with a randomly absorbed terminal. Conceptually analogous to TD($\lambda$), $\lambda$-Reachability interpolates between local self-consistency updates and long-horizon max-over-trajectory safety targets via an interpretable horizon-control parameter. Unlike TD($\lambda$), where the terminal value is always incorporated in learning targets, the terminal safety value in $\lambda$-Reachability is only used at a probability controlled by parameter $\delta$. We formally show that for $\delta<1$, the update induces a contraction mapping that allows temporal-difference learning; as $\lambda \to 1$, the estimator recovers the undiscounted reachability objective. We apply $\lambda$-Reachability to high-dimensional safety learning problems with both simulated and real humanoid robots under balance and collision avoidance constraints. Experimental results demonstrate that $\lambda$-Reachability significantly improves both safe-set boundary classification and safety margin estimation compared to single-step temporal-difference baselines.

Chinese Translation

我们提出了$ ext{λ}$-可达性，这是一种可扩展的哈密顿-雅可比安全分析方法，适用于高维机器人系统。与之前依赖固定一步贝尔曼更新的折扣公式不同，$ ext{λ}$-可达性采用了一种随机多步估计器来评估安全值，使用几何分布的展开视界以及随机吸收终端。从概念上讲，$ ext{λ}$-可达性类似于TD($ ext{λ}$)，通过一个可解释的视界控制参数在局部自一致性更新和长视界的最大轨迹安全目标之间进行插值。与TD($ ext{λ}$)不同，后者的终端值始终被纳入学习目标，而在$ ext{λ}$-可达性中，终端安全值仅在由参数$ ext{δ}$控制的概率下使用。我们正式证明，对于$ ext{δ}<1$，该更新引入了一个收缩映射，从而允许时间差学习；当$ ext{λ} o 1$时，估计器恢复未折扣的可达性目标。我们将$ ext{λ}$-可达性应用于高维安全学习问题，涉及在平衡和避碰约束下的模拟和真实类人机器人。实验结果表明，与单步时间差基线相比，$ ext{λ}$-可达性显著改善了安全集边界分类和安全边际估计。

View on arXiv Download PDF AI Translation

cs.RO / 54 / 2606.16042

Leveraging Deep Learning for Object and Position Recognition of Load Carriers for Autonomous Logistics Vehicles

利用深度学习实现自主物流车辆的载荷承载体的物体和位置识别

Legat, Christoph, Miller, Tobias, Riess, Marco

Abstract

This work explores the use of artificial intelligence in mobile robotics to achieve autonomous detection and pose estimation of load carriers for automated pickup. A deep neural network is designed to recognize predefined landmarks on the carrier from RGBD data; these landmarks are then used to compute the carrier's pose. The network operates directly on RGBD images to estimate landmark positions, which form the basis for determining the carrier's location. The approach is validated in extensive experiments and comprises both software and hardware implementations. A deep learning-based framework is presented to detect load carriers and estimate their pose for use with autonomous logistics vehicles. Our method uses a convolutional neural network to identify characteristic reference points on the carrier from RGBD input and computes its pose by combining these inferred landmarks with prior geometric knowledge. Experiments show that the resulting accuracy is sufficient for reliable load carrier detection in industrial environments, confirming the suitability of the method for autonomous intralogistics applications.

Chinese Translation

本研究探讨了在移动机器人中使用人工智能，以实现对载荷承载体的自主检测和姿态估计，以便进行自动化拾取。我们设计了一种深度神经网络，从RGBD数据中识别载荷承载体上的预定义地标；这些地标随后用于计算承载体的姿态。该网络直接处理RGBD图像，以估计地标位置，这些位置构成了确定承载体位置的基础。该方法在广泛的实验中得到了验证，包括软件和硬件的实现。我们提出了一种基于深度学习的框架，用于检测载荷承载体并估计其姿态，以便与自主物流车辆配合使用。我们的方法使用卷积神经网络从RGBD输入中识别承载体上的特征参考点，并通过将这些推断出的地标与先前的几何知识相结合来计算其姿态。实验表明，所得到的准确性足以在工业环境中可靠地检测载荷承载体，确认该方法适用于自主内部物流应用。

View on arXiv Download PDF AI Translation

cs.RO / 55 / 2606.16057

A Smart-Scheduled Hybrid (SSH) EKF-FGO State Estimation

智能调度混合 (SSH) EKF-FGO 状态估计

Levi, Eric, Beheshti, Soosan

Abstract

Reliable state estimation in robotics and control re quires balancing estimation accuracy against computational cost. While filtering-based methods such as the Extended Kalman Filter (EKF) provide efficient real-time updates, and optimisation based formulations using factor graphs improve global consistency, the role of optimisation scheduling is often treated implicitly rather than examined as an explicit design variable. This paper presents an experimental study that explicitly isolates optimisation scheduling using a Smart Scheduled Hybrid (SSH) EKF-FGO framework as a controlled testbed. By combining EKF-based state propagation with periodically invoked batch optimisation and holding solver structure and effort fixed, the main contribution of this work is the experimental characterisation of optimisation scheduling as an independent design variable governing the trade-off between intermediate estimation accuracy and computational cost. Simulation results in a planar SLAM environment show that scheduling strongly influences pre optimisation drift, transient error behaviour, and runtime. In particular, the results identify operating regimes in which most of the benefit of global optimisation can be retained at a fraction of the computational cost, highlighting optimisation scheduling as an under-explored yet critical consideration in hybrid state estimation systems.

Chinese Translation

在机器人和控制领域，可靠的状态估计需要在估计精度与计算成本之间取得平衡。尽管基于滤波的方法如扩展卡尔曼滤波器 (EKF) 提供了高效的实时更新，而基于优化的公式使用因子图提高了全局一致性，但优化调度的角色往往被隐含处理，而不是作为一个明确的设计变量进行考察。本文呈现了一项实验研究，明确隔离了优化调度，使用智能调度混合 (SSH) EKF-FGO 框架作为受控测试平台。通过将基于 EKF 的状态传播与定期调用的批量优化相结合，并保持求解器结构和努力固定，本研究的主要贡献在于将优化调度实验性地表征为一个独立的设计变量，控制着中间估计精度与计算成本之间的权衡。在平面 SLAM 环境中的仿真结果表明，调度对预优化漂移、瞬态误差行为和运行时间有显著影响。特别是，结果识别出在这些操作状态下，大部分全局优化的好处可以在较低的计算成本下保留，突显了优化调度作为混合状态估计系统中一个尚未充分探索但至关重要的考虑因素。

View on arXiv Download PDF AI Translation

cs.RO / 56 / 2606.16078

A Deployment Case Study in Robotic Apparel Automation: Digital Twin Integration, Interoperability, and Workforce Enablement

机器人服装自动化的部署案例研究：数字双胞胎集成、互操作性与劳动力赋能

Narayanan, Gokul, Ajith, Abhiroop, Zornow, Jonathan, Calle, Carlos, Lugo, Auralis Herrero, Rincon, Jose Luis Susa, Wen, Chengtao, Solowjow, Eugen

Abstract

Despite steady advances in flexible automation in sectors such as electronics and automotive manufacturing, apparel automation remains challenging because fabrics are deformable and difficult to manipulate with robots. This paper presents a deployment-oriented case study of a robotic sewing system for denim manufacturing, emphasizing the system-level integration required for practical adoption. At the engineering level, a digital thread module parses DXF production drawings into process parameters and executable robot trajectories, reducing manual programming effort and enabling rapid re-targeting across sewing operations. In parallel, a digital twin of the workcell is used during pre-deployment to validate reach and clearance, refine layout and sequencing, evaluate operator access, and assess cycle-time compatibility with upstream and downstream tasks, thereby reducing commissioning risk. At deployment, the system integrates a collaborative robot with conventional sewing equipment, welding, suction fixtures, and machine-level controllers through an interoperability layer. Runtime monitoring and verification, including seam monitoring, collision checking, and trajectory-level validation, improve robustness under environmental variability, while operator-facing training and guidance tools support setup, troubleshooting, and technology adoption. Two staged factory deployments on denim shorts, covering 2D pocket operations and 3D garment-shaping seams, show that digital-twin-based validation, digital-thread-driven task generation, interoperability, runtime verification, and operator training are important for scaling robotic apparel automation.

Chinese Translation

尽管在电子和汽车制造等行业的灵活自动化方面取得了稳步进展，但服装自动化仍然面临挑战，因为面料是可变形的，且难以用机器人进行操作。本文呈现了一个针对牛仔布制造的机器人缝纫系统的部署导向案例研究，强调了实际应用所需的系统级集成。在工程层面，数字线程模块将DXF生产图纸解析为过程参数和可执行的机器人轨迹，从而减少了人工编程的工作量，并实现了缝纫操作的快速重定向。同时，在预部署阶段，工作单元的数字双胞胎用于验证到达范围和间隙，优化布局和顺序，评估操作员的访问权限，并评估与上游和下游任务的周期时间兼容性，从而降低了调试风险。在部署阶段，该系统通过互操作层将协作机器人与传统缝纫设备、焊接、吸附夹具和机器级控制器集成。运行时监控和验证，包括缝合监测、碰撞检查和轨迹级验证，提高了在环境变化下的稳健性，而面向操作员的培训和指导工具则支持设置、故障排除和技术采纳。对牛仔短裤的两次分阶段工厂部署，涵盖了2D口袋操作和3D服装成型缝合，表明基于数字双胞胎的验证、数字线程驱动的任务生成、互操作性、运行时验证和操作员培训对于扩大机器人服装自动化具有重要意义。

View on arXiv Download PDF AI Translation

cs.RO / 57 / 2606.16178

Scaling Short-Term Memory of Visuomotor Policies for Long-Horizon Tasks

扩展视觉运动策略的短期记忆以应对长时间任务

Shah, Rutav, Jenamani, Rajat Kumar, Zhang, Xiaohan, Sun, Lingfeng, Martín-Martín, Roberto, Zhu, Yuke, Ramanan, Deva, Schmeckpeper, Karl

Abstract

Many robotic tasks require short-term memory, whether it's retrieving an object that's no longer visible or turning off an appliance after a set period. Yet, most visuomotor policies trained via imitation learning rely only on immediate sensory input without using past experiences to guide decisions. We present PRISM, a transformer-based architecture for visuomotor policies to effectively use short-term memory via two key components: (i) gated attention, which filters retrieved information to suppress irrelevant details, improving performance by reducing the spurious correlations between the history and current action prediction, (ii) a hierarchical architecture that first compresses local information into compact tokens and then integrates them to capture temporally extended dependencies, improving its compute and memory footprint. Together, these mechanisms enable us to scale short-term memory in visuomotor policies for up to two minutes. To systematically evaluate memory in visuomotor control, we introduce ReMemBench -- a benchmark of eight diverse household manipulation tasks spanning four categories of short-term memory -- designed to foster general memory mechanisms rather than siloed, task-specific solutions. PRISM consistently outperforms prior works, including recurrent architectures, transformers, and their variants -- achieving an absolute improvement of 5%--12% over the strongest baseline. On the RoboCasa and LIBERO benchmarks, it achieves absolute improvements of 11%--15% over its no-memory variant and fine-tuned Vision-Language-Action baselines such as GR00T-N1-3B and OpenVLA, despite not leveraging any large-scale pretraining. Together, PRISM and ReMemBench establish a foundation for developing and evaluating short-term memory-augmented visuomotor policies that scale to long-horizon tasks. Additional materials are available at https://shahrutav.github.io/short-term-memory

Chinese Translation

许多机器人任务需要短期记忆，无论是取回不再可见的物体，还是在设定时间后关闭电器。然而，大多数通过模仿学习训练的视觉运动策略仅依赖于即时的感官输入，而未利用过去的经验来指导决策。我们提出了PRISM，一种基于变换器（transformer）的视觉运动策略架构，能够通过两个关键组件有效利用短期记忆：（i）门控注意力（gated attention），该机制过滤检索到的信息以抑制无关细节，通过减少历史与当前动作预测之间的虚假相关性来提高性能；（ii）层次架构，首先将局部信息压缩为紧凑的标记，然后整合这些标记以捕捉时间延续的依赖关系，从而改善计算和内存占用。这些机制共同使我们能够将视觉运动策略中的短期记忆扩展至最长两分钟。为了系统地评估视觉运动控制中的记忆，我们引入了ReMemBench——一个涵盖四类短期记忆的八个多样化家庭操作任务的基准，旨在促进通用记忆机制的发展，而不是孤立的、特定任务的解决方案。PRISM在性能上始终优于以往的工作，包括递归架构、变换器及其变体——在最强基线之上实现了5%到12%的绝对提升。在RoboCasa和LIBERO基准上，它在无记忆变体和经过微调的视觉-语言-动作基线（如GR00T-N1-3B和OpenVLA）上实现了11%到15%的绝对提升，尽管没有利用任何大规模的预训练。PRISM和ReMemBench共同为开发和评估增强短期记忆的视觉运动策略奠定了基础，以应对长时间任务。更多材料可在 https://shahrutav.github.io/short-term-memory 获取。

View on arXiv Download PDF AI Translation

cs.RO / 58 / 2606.16208

ATHENA: Accelerated Multi-Task Heterogeneous Influence Functions for Robot Data Curation

ATHENA：用于机器人数据整理的加速多任务异构影响函数

Xu, Tao, Wang, Jiaxin, Zhang, Runhao, Guan, Jiayi, Zeng, Xianchao, Song, Weixi, Zhou, Xinyu, Chen, Zhetao, Chen, Guang, Li, Yong-Lu

Abstract

In robot imitation learning, influence functions provide a principled approach to quantify each demonstration's effect on robot task outcomes, yet scaling them to billion-parameter Vision-Language-Action (VLA) models is limited by computational and multitask bottlenecks. To this end, we propose ATHENA, an influence function framework tailored for multitask VLA data curation at a billion-parameter scale. Concretely, it leverages the Kronecker structure of linear-layer gradients to reduce projection cost, and approximates dense Hessian inversion with a rank-r Random Truncated Approximation, achieving about a 313.4x speedup in influence computation. Furthermore, ATHENA formulates global and local interactive influence to balance data curation across 50 jointly trained tasks. Extensive evaluations on RoboTwin 2.0 and real-robot deployment, covering 9.34 and 6.90 hours of demonstrations, respectively, show that ATHENA matches or exceeds full-data joint fine-tuning using only 50% of demonstrations in simulation and 66.7% of data across six real-robot tasks. Overall, ATHENA demonstrates its effectiveness for data curation in billion-parameter multitask VLA fine-tuning.

Chinese Translation

在机器人模仿学习中，影响函数提供了一种原则性的方法来量化每个示范对机器人任务结果的影响，但将其扩展到十亿参数的视觉-语言-行动（Vision-Language-Action, VLA）模型受到计算和多任务瓶颈的限制。为此，我们提出了ATHENA，一个针对十亿参数规模的多任务VLA数据整理的影响函数框架。具体而言，它利用线性层梯度的克罗内克结构来降低投影成本，并通过秩-r随机截断近似来逼近稠密海森矩阵的逆，从而在影响计算中实现约313.4倍的加速。此外，ATHENA制定了全局和局部交互影响，以平衡50个联合训练任务之间的数据整理。在RoboTwin 2.0和真实机器人部署上的广泛评估，分别覆盖9.34小时和6.90小时的示范，表明ATHENA在模拟中仅使用50%的示范和在六个真实机器人任务中使用66.7%的数据时，能够匹配或超过全数据联合微调的效果。总体而言，ATHENA展示了其在十亿参数多任务VLA微调中的数据整理有效性。

View on arXiv Download PDF AI Translation

cs.RO / 59 / 2606.16232

PolyMerge: Compressing 3D Gaussian Splats with Polytope Coverings for Provably Safe Resource-Constrained Navigation

PolyMerge：通过多面体覆盖压缩3D高斯点云以实现可证明安全的资源受限导航

Hong, Jihoon, Chiu, Chih-Yuan, Fridovich-Keil, Sara, Chou, Glen

Abstract

Obstacle avoidance is essential for safe navigation and motion planning. Recent radiance field reconstruction methods enable object detection and modeling with high fidelity, but remain too memory- and compute-intensive for on-board perception-based path planning. To address these limitations, we propose PolyMerge to convert a large, photorealistic 3D Gaussian Splatting (3DGS) model of a scene into a lightweight representation of convex polytopes whose union provably over-approximates all obstacles in the original 3DGS model. PolyMerge tunes the polytope count to trade off conservativeness and compute cost, and integrates with control barrier functions (CBFs) to plan collision-free paths. We showcase PolyMerge in simulation and hardware experiments on a Crazyflie drone, which uses PolyMerge to compute and follow safe trajectories in real time under severe onboard compute constraints, outperforming baselines in speed while guaranteeing safety. For our code and videos, visit https://athlon76.github.io/PolyMerge-website/.

Chinese Translation

障碍物避免对于安全导航和运动规划至关重要。最近的辐射场重建方法能够以高保真度进行物体检测和建模，但对于基于机载感知的路径规划而言，仍然过于占用内存和计算资源。为了解决这些限制，我们提出了PolyMerge，将场景的大型光真实感3D高斯点云（3DGS）模型转换为轻量级的凸多面体表示，其并集可以证明地过度近似原始3DGS模型中的所有障碍物。PolyMerge调节多面体的数量，以在保守性和计算成本之间进行权衡，并与控制障碍函数（CBFs）集成，以规划无碰撞路径。我们在模拟和Crazyflie无人机的硬件实验中展示了PolyMerge，该无人机利用PolyMerge在严重的机载计算限制下实时计算和跟踪安全轨迹，在速度上优于基线，同时保证安全性。有关我们的代码和视频，请访问 https://athlon76.github.io/PolyMerge-website/。

View on arXiv Download PDF AI Translation

cs.RO / 60 / 2606.16272

TopoRetarget: Interaction-Preserving Retargeting for Dexterous Manipulation

TopoRetarget：保持交互的重定向以实现灵巧操作

Wu, Jielin, Yao, Shenzhe, He, Guanqi, Liu, Xiaohan, Zeng, Zhaoqing, Jiang, Xiangrui, Yang, Han, Zhang, Wentao, Zhao, Hang

Abstract

Human hand-object demonstrations provide dense reference motions for training dexterous manipulation reinforcement learning (RL) policies through reference tracking. However, to use such demonstrations for RL policy learning, retargeting must preserve hand pose and task-relevant hand-object contact structure. Otherwise, contact and feasibility artifacts can degrade downstream RL policy performance. We introduce TopoRetarget, an interaction-preserving retargeting framework that uses a single set of parameters across diverse retargeting conditions while maintaining task-relevant hand-object interaction and adapting human demonstrations to dexterous robot hands. The method constructs a sparse interaction graph over hand and object keypoints and optimizes distance-weighted Laplacian deformation with directional consistency, kinematic constraints, and penetration handling. Evaluations show that the generated references improve both interaction fidelity and policy learning: TopoRetarget achieves the best contact precision and alignment over all baselines on the ContactPose Dataset, improves Pen-Spin training success by 40.6 percentage points over the existing baseline methods, and enables zero-shot transfer to Wuji Hand hardware on cube reorientation and pen spinning.

Chinese Translation

人手与物体的示范提供了密集的参考动作，用于通过参考跟踪训练灵巧操作的强化学习（RL）策略。然而，为了将这些示范用于RL策略学习，重定向必须保持手部姿态和与任务相关的手-物体接触结构。否则，接触和可行性伪影可能会降低下游RL策略的性能。我们提出了TopoRetarget，一个保持交互的重定向框架，它在多样的重定向条件下使用一组参数，同时保持与任务相关的手-物体交互，并将人类示范适配到灵巧的机器人手。该方法在手部和物体关键点上构建稀疏交互图，并优化具有方向一致性、运动约束和穿透处理的距离加权拉普拉斯变形。评估结果表明，生成的参考提高了交互保真度和策略学习：TopoRetarget在ContactPose数据集上实现了所有基线中最佳的接触精度和对齐效果，较现有基线方法提高了40.6个百分点的Pen-Spin训练成功率，并在立方体重新定向和笔旋转任务上实现了对Wuji Hand硬件的零样本迁移。

View on arXiv Download PDF AI Translation

cs.RO / 61 / 2606.16313

Is Your Trajectory Displacement Safe in Long-tail?

您的轨迹位移在长尾场景中安全吗？

Sun, Qiao, Zheng, Weicheng, Huang, Yixin, Zhao, Hang

Abstract

Long-tail scenarios remain a major bottleneck for autonomous driving evaluation, even as datasets grow by orders of magnitude. Existing evaluation pipelines are rarely human-aligned, safety-aware, verifiable, and explainable at the same time: closed-loop metrics often saturate among strong planners, while unstructured human ratings can be noisy without a carefully designed protocol. We formulate planning evaluation as additional-threat detection: given a planner trajectory and an expert reference, does the planner's displacement introduce new unsafe driving behavior? We propose FluidTest, an evaluation pipeline with three components: a pairwise WebUI protocol for reliable human annotation; a taxonomy of 32 semantic threats with evidence-grounded decision graphs; and a three-agent verification system with reflection for precision and auditability. Experiments on the WOD-E2E dataset show that FluidTest produces consistent labels among trained annotators and identifies additional threats in 65% of Poutine trajectories and 51% of RAP trajectories. These results show that state-of-the-art planners can still exhibit substantial safety-relevant failures despite high Rater Feedback Scores (RFS) and low Average Displacement Error (ADE). Additional details, guidance, and code are available at https://fluidtest.web.app.

Chinese Translation

长尾场景仍然是自主驾驶评估的主要瓶颈，即使数据集的规模增长了几个数量级。现有的评估流程很少同时具有人类对齐、安全意识、可验证性和可解释性：闭环指标在强规划者之间往往饱和，而非结构化的人类评分在没有精心设计的协议时可能会出现噪声。我们将规划评估形式化为额外威胁检测：给定一个规划者的轨迹和一个专家参考，规划者的位移是否引入了新的不安全驾驶行为？我们提出了FluidTest，一个包含三个组件的评估流程：用于可靠人类标注的成对WebUI协议；一个包含32种语义威胁的分类法，配有基于证据的决策图；以及一个具有反思功能的三代理验证系统，以确保精确性和可审计性。在WOD-E2E数据集上的实验表明，FluidTest在训练标注者之间产生了一致的标签，并在65%的Poutine轨迹和51%的RAP轨迹中识别出额外的威胁。这些结果表明，尽管Rater Feedback Scores (RFS)较高且Average Displacement Error (ADE)较低，最先进的规划者仍可能表现出显著的安全相关失败。更多细节、指导和代码可在https://fluidtest.web.app获取。

View on arXiv Download PDF AI Translation

cs.RO / 62 / 2606.16370

ART-Glove: Articulated Tactile Glove for Contact-Grounded Dexterous Interaction Capture

ART-Glove：用于接触基础灵巧交互捕捉的关节触觉手套

Lin, Changyi, Zhao, Ding

Abstract

We present ART-Glove, an articulated tactile glove designed to capture contact-grounded dexterous demonstrations while preserving human dexterity. ART-Glove makes hand-side contact geometry explicit with 16 rigid functional surfaces covering the fingers, thumb, and palm. Twenty-two anatomically aligned joints connect these surfaces and allow them to follow human hand motion during dexterous manipulation. Encoder-based sensing tracks surface motion, while dense piezoresistive tactile sensing records contact over the same surfaces. The complete system captures synchronized 22-DoF joint measurements and 2048-taxel tactile measurements at 120 Hz. We evaluate ART-Glove across experiments on motion freedom, joint sensing, tactile sensing, and contact-rich interaction capture, demonstrating its ability to preserve human dexterity while recording contact-grounded information that can support downstream dexterous robot learning.

Chinese Translation

我们提出了ART-Glove，这是一种关节触觉手套，旨在在保持人类灵巧性的同时捕捉接触基础的灵巧演示。ART-Glove通过覆盖手指、拇指和手掌的16个刚性功能表面，使手部接触几何形状变得明确。这些表面通过22个解剖对齐的关节连接，允许它们在灵巧操作过程中跟随人手运动。基于编码器的传感器跟踪表面运动，而密集的压阻触觉传感器记录相同表面上的接触。整个系统以120 Hz的频率捕捉同步的22自由度关节测量和2048个触觉测量。我们在运动自由度、关节传感、触觉传感和接触丰富的交互捕捉等实验中评估了ART-Glove，展示了其在记录接触基础信息的同时保持人类灵巧性的能力，这些信息可以支持下游灵巧机器人学习。

View on arXiv Download PDF AI Translation

cs.RO / 63 / 2606.16400

SemGeoNav:A Safety-Guided Visual Navigation Approach with Semantic Reasoning and Geometric Planning

SemGeoNav：一种结合语义推理与几何规划的安全引导视觉导航方法

Liu, Yu, Chen, Zongyang, Guo, Yan, Liu, Chao, Pan, Xianfei

Abstract

Learning-based visual navigation has enhanced semantic goal-reaching capabilities. However, due to their black-box nature, purely end-to-end models often lack explicit geometric constraints, leading to unpredictable and unreliable obstacle avoidance in open environments. Conversely, traditional geometric planners ensure safety but struggle with high-dimensional visual targets. To address these limitations, we propose SemGeoNav, a novel hierarchical visual navigation framework.It tightly integrates the high-level semantic reasoning of end-to-end models with the reliable local planning ability of geometry-based methods, achieving robust image-based navigation while significantly improving obstacle avoidance. Furthermore, we introduce a temporal trajectory smoothing mechanism to ensure continuous and stable robot motion. We evaluated SemGeoNav on a Unitree Go2 quadruped robot in real-world environments. The results demonstrate that SemGeoNav outperforms existing representative methods, including ViNT and NoMaD, achieving higher success rates and shorter navigation times.

Chinese Translation

基于学习的视觉导航增强了语义目标到达能力。然而，由于其黑箱特性，纯粹的端到端模型往往缺乏明确的几何约束，导致在开放环境中障碍物规避不可预测且不可靠。相反，传统的几何规划方法确保安全，但在处理高维视觉目标时表现不佳。为了解决这些局限性，我们提出了SemGeoNav，一种新颖的分层视觉导航框架。它将端到端模型的高层语义推理与基于几何的方法的可靠局部规划能力紧密结合，实现了稳健的基于图像的导航，同时显著改善了障碍物规避。此外，我们引入了一种时间轨迹平滑机制，以确保机器人运动的连续性和稳定性。我们在实际环境中对Unitree Go2四足机器人进行了SemGeoNav的评估。结果表明，SemGeoNav在成功率和导航时间上均优于现有的代表性方法，包括ViNT和NoMaD。

View on arXiv Download PDF AI Translation

cs.RO / 64 / 2606.16413

An Augmented Reality Brain-Robot Interface for Generalist Robot Arm Manipulation

用于通用机器人臂操作的增强现实脑-机器人接口

Zhang, Shangkai, Dossa, Rousslan Fernand Julien, Nunziante, Luca, Di Vincenzo, Marina, Arulkumaran, Kai

Abstract

The integration of augmented reality (AR) and EEG-based brain-computer interfaces (BCIs) offers a promising path for enabling intuitive control of robots for assistive purposes. However, existing AR brain-robot interface (BRI) systems are often constrained to task-specific structures, limiting their utility in real-world environments. We present an AR BRI designed for generalist robot arm manipulation that combines gaze-based object selection with motor imagery action control. Our system uses eye-tracking for intuitive object targeting and context-aware visual overlays ("Place" and "Use") to guide the user through tasks within a shared autonomy framework. We evaluated the interface through a feasibility study with 18 healthy participants performing three multi-step activities of daily living: drinking, using a drawer, and operating an oven. Our results demonstrate that this interaction paradigm enables effective sequential task execution and high user engagement, achieving a "Good" usability rating (SUS > 70). These findings support the feasibility of the proposed interaction paradigm for complex BCI-driven robotic assistance, and motivate future evaluation with the intended target population. Project website: https://ar-bri-manip.github.io/.

Chinese Translation

增强现实（AR）与基于脑电图（EEG）的脑-计算机接口（BCI）的结合为实现直观的机器人控制提供了有前景的途径，特别是在辅助应用方面。然而，现有的增强现实脑-机器人接口（BRI）系统往往局限于特定任务的结构，限制了其在现实环境中的实用性。我们提出了一种用于通用机器人臂操作的增强现实脑-机器人接口，该系统结合了基于注视的物体选择与运动想象动作控制。我们的系统使用眼动追踪技术实现直观的物体定位，并通过上下文感知的视觉叠加（“放置”和“使用”）来引导用户在共享自主框架内完成任务。我们通过一项可行性研究对该接口进行了评估，18名健康参与者完成了三项多步骤的日常生活活动：饮水、使用抽屉和操作烤箱。我们的结果表明，这种交互范式能够有效地实现顺序任务执行，并且用户参与度高，获得了“良好”的可用性评分（SUS > 70）。这些发现支持了所提出的交互范式在复杂的BCI驱动机器人辅助中的可行性，并激励未来在目标人群中进行进一步评估。项目网站：https://ar-bri-manip.github.io/

View on arXiv Download PDF AI Translation

cs.RO / 65 / 2606.16436

V2P-Manip: Learning Dexterous Manipulation from Monocular Human Videos

V2P-Manip：从单目人类视频中学习灵巧操作

Chen, Kaihan, Shao, Yanming, Ji, Haifeng, Yang, Xiaokang, Mu, Yao

Abstract

Achieving autonomous robotic dexterous manipulation requires precise, human-like action sequences at scale. As a scalable supplement to costly teleoperation data, extracting trajectories with both visual fidelity and physical plausibility from monocular videos represents a promising frontier in embodied AI. To this end, we introduce V2P-Manip, an efficient framework designed to learn dexterous manipulation policies directly from human demonstration videos. We establish an efficient, integrated pipeline encompassing 3D asset acquisition, trajectory estimation, and dexterous policy learning. To bridge the gap between visual perception and physical constraints, we introduce a two-stage refinement process to enforce spatial alignment and physical consistency. Evaluations on the TACO and OakInk benchmarks demonstrate that our approach significantly outperforms previous methods in pose accuracy, adaptability to unstructured environments, and training efficiency. Ultimately, experimental results confirm an average success rate of over 75% across multiple synthetic manipulation tasks and validate the adaptability of the extracted manipulation priors across diverse dexterous hand embodiments.

Chinese Translation

实现自主机器人灵巧操作需要在大规模上进行精确的人类动作序列。作为昂贵遥控数据的可扩展补充，从单目视频中提取具有视觉真实性和物理合理性的轨迹代表了具身人工智能的一个有前景的前沿。为此，我们提出了V2P-Manip，这是一个高效的框架，旨在直接从人类演示视频中学习灵巧操作策略。我们建立了一个高效的集成管道，包括3D资产获取、轨迹估计和灵巧策略学习。为了弥合视觉感知与物理约束之间的差距，我们引入了一个两阶段的精炼过程，以强制执行空间对齐和物理一致性。在TACO和OakInk基准上的评估表明，我们的方法在姿态准确性、对非结构化环境的适应性和训练效率方面显著优于之前的方法。最终，实验结果确认在多个合成操作任务中平均成功率超过75%，并验证了提取的操作先验在各种灵巧手部体现中的适应性。

View on arXiv Download PDF AI Translation

cs.RO / 66 / 2606.16447

Training and Evaluating Diffusion Policies with Long Context Lengths

使用长上下文长度训练和评估扩散策略

Agarwal, Abhinav, Wei, Adam, Kargin, Taylan, Zeng, Michael, Becker, Cole, Dayi, Arif Kerem, Parrilo, Pablo, Ozdaglar, Asuman, Tedrake, Russ

Abstract

Imitation learning has enabled highly-dexterous robotic manipulation from RGB observations. Policies trained with these methods, however, typically condition robot actions on only a short history of observations. These policies cannot solve tasks that require memory and can get stuck repeatedly executing the same failing motions. In this work, we first benchmark policy performance as context length is incrementally increased from short to long, across a spectrum of tasks with varying local stability and memory requirements, and in multiple data regimes. To our knowledge, this is the first study to investigate context length in imitation learning at this level of detail. Our results challenge prior claims: naively scaling context length is not as brittle as advertised in literature. With an appropriate conditioning method and denoising backbone (UNet+Cross-Attention), single-task policies achieve high success rates on many tasks in the usual data regime even with naive scaling. Next, we propose a training algorithm to jointly train policies at multiple context lengths, further reducing the sample complexity of long-context learning. Finally, we apply our findings to re-evaluate some previously proposed solutions to long-context imitation learning.

Chinese Translation

模仿学习使得基于RGB观察实现高灵巧性的机器人操作成为可能。然而，使用这些方法训练的策略通常仅基于短期观察历史来决定机器人动作。这些策略无法解决需要记忆的任务，并且可能反复执行相同的失败动作。在本研究中，我们首先在多个数据条件下，针对不同局部稳定性和记忆需求的任务，逐步增加上下文长度，从短到长，对策略性能进行基准测试。据我们所知，这是首个在模仿学习中以如此细致的水平研究上下文长度的研究。我们的结果挑战了先前的观点：简单地扩大上下文长度并不像文献中所宣传的那样脆弱。通过适当的条件方法和去噪骨干网络（UNet+Cross-Attention），单任务策略在常规数据条件下即使在简单扩展的情况下也能在许多任务上实现高成功率。接下来，我们提出了一种训练算法，以联合训练多个上下文长度的策略，进一步降低长上下文学习的样本复杂性。最后，我们将我们的发现应用于重新评估一些之前提出的长上下文模仿学习解决方案。

View on arXiv Download PDF AI Translation

cs.RO / 67 / 2606.16458

RHO: Your Coding Agent is Secretly a Roboticist

RHO：你的编码代理实际上是一个机器人专家

Elmaaroufi, Karim, Svegliato, Justin, Kalade, Sarunas, Schelle, Graham, Seshia, Sanjit A., Zaharia, Matei

Abstract

Code-as-Policies (CaP) has shown that large language models (LLMs) can write code to solve robotics tasks by composing perception, planning, and control primitives. Recent CaP systems, however, rely on multi-turn code-generation loops at test time, which is often infeasible for real-time robot control. We introduce Robotics Harness Optimization (RHO), a novel paradigm in which tool-enabled coding agents, at training time, propose and search for interpretable, neurosymbolic multi-file policy repositories (Repositories-as-Policies) that compose these primitives rather than a single prompt, function, or file. RHO searches with reflective feedback from environment reward and execution rather than teleoperation demonstrations. It generalizes to perturbed pick-and-place settings like LIBERO-PRO, where OpenVLA scores 0.0% and $\pi_{0.5}$ averages 12.83%. Using the same low-level primitives, RHO reaches a 45.0% success rate, 2.5x higher than the strongest multi-turn agentic system, and 3.5x higher than $\pi_{0.5}$. On Robosuite, RHO sets a new state-of-the-art of 70.0%, exceeding the prior multi-turn record of 68.29% using single-turn execution with no corrective LLM code edits at deployment. When an LLM is used in the control loop, as on RAI's O3DE benchmark, RHO optimizes the deployed agent's multi-file harness of prompts, tools, and control code, improving held-out success from 23.5% to 44.3% with 20% less wall-clock time and 27% fewer tool calls.

Chinese Translation

代码作为策略（Code-as-Policies, CaP）表明大型语言模型（Large Language Models, LLMs）能够通过组合感知、规划和控制原语来编写代码以解决机器人任务。然而，近期的 CaP 系统在测试时依赖于多轮代码生成循环，这在实时机器人控制中往往不可行。我们引入了机器人工具优化（Robotics Harness Optimization, RHO），这是一种新颖的范式，在训练过程中，工具启用的编码代理提出并搜索可解释的神经符号多文件策略库（Repositories-as-Policies），这些库组合了这些原语，而不是单一的提示、函数或文件。RHO 通过环境奖励和执行的反思反馈进行搜索，而不是通过远程操作演示。它能够推广到像 LIBERO-PRO 这样的扰动抓取和放置设置，在这些设置中，OpenVLA 的得分为 0.0%，而 $ ext{π}_{0.5}$ 的平均得分为 12.83%。使用相同的低级原语，RHO 达到了 45.0% 的成功率，比最强的多轮代理系统高出 2.5 倍，比 $ ext{π}_{0.5}$ 高出 3.5 倍。在 Robosuite 上，RHO 创造了 70.0% 的新状态，超过了之前多轮记录 68.29%，且在部署时没有进行任何修正的 LLM 代码编辑。当在控制循环中使用 LLM 时，如在 RAI 的 O3DE 基准测试中，RHO 优化了已部署代理的多文件提示、工具和控制代码的组合，将保留的成功率从 23.5% 提高到 44.3%，同时减少了 20% 的实际时间和 27% 的工具调用。

View on arXiv Download PDF AI Translation

cs.RO / 68 / 2606.16480

HOLO-MPPI: Multi-Scenario Motion Planning via Hierarchical Policy Optimization

HOLO-MPPI：通过分层策略优化实现多场景运动规划

Min, Youngjae, D'sa, Jovin, Tariq, Faizan M., Isele, David, Azizan, Navid, Bae, Sangjae

Abstract

Robots deployed in the real world must plan motions across diverse scenarios without per-scenario retuning. End-to-end reinforcement learning (RL) can generalize across scenarios but often becomes brittle under distribution shift, reward misspecification, and stochastic interactions. Model predictive path integral (MPPI) control enables strong real-time refinement without gradients, but its performance depends on a well-shaped sampling prior, while manually designing the priors does not scale to multi-scenario deployment. We present HOLO-MPPI (High-level Offline, Low-level Online MPPI), a multi-scenario motion planning framework that combines high-level policy learning with low-level stochastic optimal control. Offline, we learn a high-level policy that proposes scenario-robust plans in an abstract action space, with a learned world model for online rollout. Online, the policy serves as a data-driven prior generator that parameterizes MPPI's sampling distribution conditioned on the current observation and goal. MPPI then optimizes low-level control sequences around this prior in real time to adapt to local disturbances. We instantiate HOLO-MPPI in autonomous driving by designing an effective high-level action space and tailored model architectures. Our evaluation across diverse driving scenarios shows that HOLO-MPPI improves upon MPPI and end-to-end RL baselines while maintaining real-time control.

Chinese Translation

部署在现实世界中的机器人必须在多样化场景中规划运动，而无需对每个场景进行重新调优。端到端的强化学习（RL）能够在不同场景之间进行泛化，但在分布转移、奖励误指定和随机交互的情况下往往变得脆弱。模型预测路径积分（MPPI）控制能够在没有梯度的情况下实现强大的实时优化，但其性能依赖于良好形状的采样先验，而手动设计先验无法扩展到多场景部署。我们提出了HOLO-MPPI（高层离线、低层在线MPPI），这是一个将高层策略学习与低层随机最优控制相结合的多场景运动规划框架。在离线阶段，我们学习一个高层策略，该策略在抽象动作空间中提出场景鲁棒的规划，并为在线执行提供学习的世界模型。在在线阶段，该策略作为数据驱动的先验生成器，根据当前观察和目标参数化MPPI的采样分布。然后，MPPI实时优化围绕该先验的低层控制序列，以适应局部干扰。我们通过设计有效的高层动作空间和定制的模型架构，在自动驾驶中实例化HOLO-MPPI。我们在多样化的驾驶场景中的评估表明，HOLO-MPPI在保持实时控制的同时，优于MPPI和端到端RL基线。

View on arXiv Download PDF AI Translation

cs.RO / 69 / 2606.16490

Robots that Collaborate: Sequential Asymmetric Imitation for Learning Coupled Robot Policies

协作机器人：用于学习耦合机器人策略的顺序非对称模仿

Chen, Yincong, Qiu, Ranpeng, Li, Zihao, Zhou, Yanan, Ren, Guoqiang, Zhi, Weiming

Abstract

Collaborative mobile manipulation requires robots to coordinate with a partially observed partner while physically interacting through shared objects. This is difficult because failures often arise not from poor local skills, but from mistimed waiting, yielding, pulling, releasing, or repositioning. We study this problem with two bimanual mobile manipulators coupled through rigid and deformable objects. We propose Sequential Asymmetric Imitation (SAI), a single-teleoperator curriculum for learning coupled multi-robot behaviors without synchronized dual-operator demonstrations or explicit inter-robot communication. SAI trains Robot A from unilateral demonstrations with a compliant human partner, trains Robot B against the deployed Robot A policy, and then refines Robot A using sparse interventions near coordination failures. This staged process exposes the policies to increasingly realistic partner behaviors, including delay, phase mismatch,insufficient yielding, and interaction conflict. Across real-world dual-robot manipulation tasks, SAI improves task success, phase synchronization, and partner-contingent yielding over independent imitation and curriculum-ablation baselines. These results suggest that physically coupled collaboration can be learned through the structure of the imitation curriculum, rather than through synchronized multi-operator demonstrations or explicit coordination mechanisms.Project page:http://cyc0429.github.io/sai-project-page/

Chinese Translation

协作移动操作要求机器人在与部分可观察的伙伴协调的同时，通过共享物体进行物理交互。这是一个困难的问题，因为失败往往不是由于局部技能差，而是由于时机不当的等待、让步、拉动、释放或重新定位。我们研究了这个问题，使用两个通过刚性和可变形物体耦合的双手移动操纵器。我们提出了顺序非对称模仿（Sequential Asymmetric Imitation, SAI），这是一种单一遥控操作员课程，用于学习耦合的多机器人行为，而无需同步的双操作员演示或明确的机器人间通信。SAI通过与一个顺应的人类伙伴进行单侧演示来训练机器人A，针对已部署的机器人A策略来训练机器人B，然后在协调失败附近使用稀疏干预来优化机器人A。这个分阶段的过程使策略暴露于越来越真实的伙伴行为，包括延迟、相位不匹配、不充分的让步和交互冲突。在现实世界的双机器人操作任务中，SAI在任务成功率、相位同步和伙伴依赖的让步方面优于独立模仿和课程消融基线。这些结果表明，物理耦合的协作可以通过模仿课程的结构来学习，而不是通过同步的多操作员演示或明确的协调机制。项目页面：http://cyc0429.github.io/sai-project-page/

View on arXiv Download PDF AI Translation

cs.RO / 70 / 2606.16491

HATS: A Human-Agent Teleoperation System for Multi-Arm Data Collection

HATS：一种用于多臂数据收集的人机遥操作系统

Lin, Zesen, Jiang, Jian-Jian, Cen, Haoming, Wu, Xiao-Ming, Zhang, Dandan, Zheng, Wei-Shi

Abstract

Many real-world manipulation scenarios, such as handling complex collaborative tasks and dealing with large workspaces, require coordination of more than two robotic arms. Consequently, an effective multi-arm teleoperation system is required to collect demonstrations for training coordinated multi-arm manipulation policies. However, existing teleoperation frameworks mainly focus on single-operator or multi-operator setups, facing a practical trade-off between the cognitive load placed on a single operator and the coordination cost incurred by multiple operators. To address this problem, we introduce HATS, a human-agent teleoperation system that enables a single human operator, assisted by an MLLM-based agent, to collect data for multi-arm manipulation tasks. Our system decouples the control space: two primary arms are directly teleoperated by the human, while two assistive arms are controlled by a training-free agent that handles sub-tasks. In addition, the human operator can use voice commands to prevent collisions and correct assistive arm behaviors during execution. Extensive evaluations demonstrate that HATS achieves data collection efficiency and success rates comparable to expert dual-human teams. Moreover, downstream policy evaluations demonstrate the efficacy and quality of the data collected through HATS.

Chinese Translation

许多现实世界的操作场景，如处理复杂的协作任务和应对大型工作空间，需要协调多个机器人手臂。因此，需要一个有效的多臂遥操作系统来收集演示数据，以训练协调的多臂操作策略。然而，现有的遥操作框架主要集中于单一操作员或多操作员设置，在单一操作员所承受的认知负荷与多操作员所产生的协调成本之间面临实际的权衡。为了解决这个问题，我们提出了HATS，一个人机遥操作系统，使得单个人工操作员在基于MLLM（多模态大语言模型）代理的协助下，能够收集多臂操作任务的数据。我们的系统解耦了控制空间：两个主要手臂由人类直接遥操作，而两个辅助手臂则由一个无需训练的代理控制，负责处理子任务。此外，人工操作员可以使用语音命令来防止碰撞并在执行过程中纠正辅助手臂的行为。大量评估表明，HATS在数据收集效率和成功率上与专家双人团队相当。此外，下游策略评估展示了通过HATS收集的数据的有效性和质量。

View on arXiv Download PDF AI Translation

cs.RO / 71 / 2606.16504

APEX: Adaptive Policy Execution for Precise Manipulation

APEX：用于精确操作的自适应策略执行

Zhao, Mengfei, Jiang, Chenxi, An, Tuo, Jia, Jindou, Yang, Jianfei

Abstract

Modern imitation learning methods, including visuomotor and Vision-Language-Action (VLA) policies, typically output high-level action references that are executed by low-level controllers. However, the absence of higher-order reference signals, together with the policy's lack of awareness of the underlying low-level control dynamics during training, inevitably induces an execution gap. As a result, realized actions deviate systematically from policy-commanded ones, with a critical impact on precision-sensitive manipulation. Prior work either modifies the policy architecture or the low-level controller, both requiring intrusive changes to the pretrained policy or packaged controller. This raises a natural question: when the policy and controller are both treated as inaccessible black boxes, can we bridge the execution gap? We propose Adaptive Policy Execution (APEX), a plug-and-play framework inserted between the policy and the controller that reconstructs a dynamically feasible reference from policy outputs and adapts at test-time according to low-level state feedback, with a provable convergence guarantee. Extensive empirical studies show that APEX reduces controller-induced tracking error by 41.2% on demonstration replay and improves manipulation success by 4.8--25.8 percentage points across four visuomotor and VLA policy classes.

Chinese Translation

现代模仿学习方法，包括视觉运动和视觉-语言-动作（VLA）策略，通常输出由低级控制器执行的高层次动作参考。然而，缺乏高阶参考信号，加上策略在训练过程中对底层控制动态的缺乏意识，必然导致执行差距。因此，实际执行的动作系统性地偏离了策略命令的动作，这对精确敏感的操作产生了重要影响。先前的研究要么修改策略架构，要么修改低级控制器，这两者都需要对预训练策略或打包控制器进行侵入式更改。这引出了一个自然的问题：当策略和控制器都被视为不可接触的黑箱时，我们能否弥合执行差距？我们提出了自适应策略执行（APEX），这是一个插入在策略和控制器之间的即插即用框架，它从策略输出中重构动态可行的参考，并在测试时根据低级状态反馈进行适应，具有可证明的收敛保证。大量实证研究表明，APEX在演示重放中减少了41.2%的控制器引起的跟踪误差，并在四个视觉运动和VLA策略类别中提高了4.8至25.8个百分点的操作成功率。

View on arXiv Download PDF AI Translation

cs.RO / 72 / 2606.16513

Agile Fall Recovery for Quadrotors with Bidirectional Thrust via Reinforcement Learning

基于强化学习的双向推力四旋翼灵活坠落恢复

Zhao, Anke, Zhong, Yuhang, Hoi, Kenghou, Mou, Junyu, Wang, Junjie, Wang, Lijie, Hou, Jialiang, Gao, Fei

Abstract

Autonomous fall recovery is a critical capability for quadrotors operating in real-world environments, where collisions or failures may leave the vehicle resting on the ground in an arbitrary attitude. This problem is challenging because recovery must be achieved under limited onboard sensing, in constrained free space, with ground contact, and in the presence of unknown disturbances. In this letter, we present an RL-based framework for autonomous fall recovery of a quadrotor from arbitrary ground attitudes to stable hover using only lightweight onboard sensors. To address severe partial observability and intermittent sensor invalidity, we train a recurrent policy within an asymmetric actor--critic architecture, leveraging an Incremental Nonlinear Dynamic Inversion (INDI) controller to track the policy output. Combined with high-fidelity simulations of motor response and optical flow, the overall training framework significantly reduces the sim-to-real gap. Simulation ablation studies validate the importance of the main design choices, while real-world experiments demonstrate zero-shot transfer and robust recovery under different initial attitudes, wind disturbances, and additional payloads. These results demonstrate that agile quadrotor fall recovery can be achieved without explicit state estimation using only limited and unreliable onboard sensing.

Chinese Translation

自主坠落恢复是四旋翼在现实环境中操作的关键能力，因为碰撞或故障可能导致飞行器以任意姿态停在地面上。这个问题具有挑战性，因为恢复必须在有限的机载传感器、受限的自由空间、与地面接触以及存在未知干扰的情况下实现。在本文中，我们提出了一种基于强化学习的框架，用于四旋翼从任意地面姿态恢复到稳定悬停，仅使用轻量级的机载传感器。为了应对严重的部分可观测性和间歇性的传感器失效，我们在不对称的演员-评论家架构中训练了一个递归策略，利用增量非线性动态反演（Incremental Nonlinear Dynamic Inversion, INDI）控制器来跟踪策略输出。结合高保真度的电机响应和光流模拟，整体训练框架显著减少了仿真与现实之间的差距。仿真消融研究验证了主要设计选择的重要性，而现实世界实验则展示了零-shot迁移和在不同初始姿态、风干扰和额外负载下的稳健恢复。这些结果表明，灵活的四旋翼坠落恢复可以在没有显式状态估计的情况下，仅使用有限且不可靠的机载传感器实现。

View on arXiv Download PDF AI Translation

cs.RO / 73 / 2606.16542

ADAPT: Analytical Disturbance-Aware Policy Training for Humanoid Locomotion

ADAPT：面向类人步态的分析性干扰感知策略训练

Lyu, Bofan, Jia, Jindou, Zuo, Kuangji, Lu, Yanshuo, Han, Shijia, Li, Gen, Ma, Boyu, Li, Jingliang, Li, Geng, Yang, Jianfei

Abstract

Humanoids deployed in human-centered environments must handle force-interactive tasks, where external contacts introduce unexpected disturbances that disrupt locomotion accuracy and stability. Existing learning-based approaches rely on broad domain randomization, task-specific force objectives, or learning-based force estimators from motion history, each of which compromises accuracy, task transferability, or out-of-distribution (OOD) robustness. We present Analytical Disturbance-Aware Policy Training (ADAPT), a framework that equips humanoid policies with a physically grounded disturbance observer. The core of ADAPT is an analytical whole-body disturbance observer that estimates residual force/torque online with the accessible robot dynamics, without requiring force/torque sensors. Fed directly into the policy, the estimated disturbances give the humanoid an explicit, physics-derived sense of external force/torque that can generalize across diverse unseen scenes. Experiments on a Unitree G1 humanoid show that ADAPT achieves accurate disturbance prediction and stronger robustness than a proprioception-only baseline under torso perturbations, standing pushes, and asymmetric hand payloads, with improved velocity tracking even on OOD disturbances. Moreover, ADAPT enables penalizing inferred disturbances at lower-body joints to encourage lighter locomotion.

Chinese Translation

在以人为中心的环境中部署的类人机器人必须处理力交互任务，其中外部接触会引入意外干扰，从而影响步态的准确性和稳定性。现有的基于学习的方法依赖于广泛的领域随机化、特定任务的力目标或基于运动历史的学习型力估计器，每种方法都在准确性、任务可转移性或分布外（OOD）鲁棒性方面存在妥协。我们提出了分析性干扰感知策略训练（ADAPT），该框架为类人策略配备了一个物理基础的干扰观察器。ADAPT的核心是一个分析性的全身干扰观察器，它利用可获取的机器人动力学在线估计残余力/扭矩，而无需力/扭矩传感器。估计的干扰直接输入到策略中，为类人机器人提供了一个明确的、基于物理的外部力/扭矩感知，能够在多样的未见场景中进行泛化。在Unitree G1类人机器人上的实验表明，ADAPT在躯干扰动、站立推力和不对称手部负载下实现了准确的干扰预测，并且比仅依赖本体感知的基线具有更强的鲁棒性，甚至在OOD干扰下也能改善速度跟踪。此外，ADAPT还能够对推断出的干扰在下肢关节施加惩罚，以鼓励更轻便的步态。

View on arXiv Download PDF AI Translation

cs.RO / 74 / 2606.16564

Elastic ODYN: Differentiable Optimization for Infeasible Control and Learning in Robotics

弹性 ODYN：用于机器人中不可行控制与学习的可微优化

Papatheodorou, Aristotelis, Rojas, Jose, Havoutis, Ioannis, Mastalli, Carlos

Abstract

Robotic systems routinely encounter conflicting objectives, modeling errors, and degenerate contact conditions that render quadratic programs (QPs) infeasible. Yet most optimization solvers and differentiable QP layers assume feasibility, leading to numerical failures, unstable gradients, or solver breakdown when constraints cannot be simultaneously satisfied. We present Elastic ODYN, a primal--dual non-interior-point QP solver that handles infeasibility through smooth squared-$\ell_2$ elastic relaxations. The resulting formulation remains well posed under ill-conditioning and degeneracy, supports warm starting, and converges to closest-to-feasible solutions when no feasible point exists. A lightweight refinement stage recovers physically meaningful dual variables from the elastic solution. Building on this framework, we develop Elastic OdynLayer, a differentiable QP layer with stable gradients under infeasibility, and Elastic OdynSQP, an infeasibility-aware SQP method that resolves inconsistent subproblems and intrinsically infeasible optimal control tasks through selective constraint relaxation. We evaluate the framework on benchmark QPs, singular contact mechanics, differentiable parameter identification, and quadrupedal and humanoid trajectory optimization. Across all settings, Elastic ODYN consistently outperforms state-of-the-art elastic QP solvers in robustness, warm-start performance, and convergence reliability, enabling optimization, simulation, control, and learning beyond the feasibility assumptions of existing methods.

Chinese Translation

机器人系统常常面临相互冲突的目标、建模误差和退化接触条件，这使得二次规划（QPs）变得不可行。然而，大多数优化求解器和可微 QP 层假设问题是可行的，当约束无法同时满足时，导致数值失败、不稳定的梯度或求解器崩溃。我们提出了弹性 ODYN，一种原始-对偶非内部点 QP 求解器，通过平滑的平方-$ ext{l}_2$ 弹性松弛处理不可行性。所得到的公式在不良条件和退化情况下仍然保持良好定义，支持热启动，并在不存在可行点时收敛到最接近可行的解。轻量级的精炼阶段从弹性解中恢复物理上有意义的对偶变量。在此框架基础上，我们开发了弹性 OdynLayer，一种在不可行性下具有稳定梯度的可微 QP 层，以及弹性 OdynSQP，一种关注不可行性的 SQP 方法，通过选择性约束松弛解决不一致的子问题和内在不可行的最优控制任务。我们在基准 QPs、奇异接触力学、可微参数识别以及四足和类人轨迹优化上评估了该框架。在所有设置中，弹性 ODYN 在鲁棒性、热启动性能和收敛可靠性方面始终优于最先进的弹性 QP 求解器，使得优化、仿真、控制和学习超越了现有方法的可行性假设。

View on arXiv Download PDF AI Translation

cs.RO / 75 / 2606.16570

Automated Digital Twin Construction for Highway Scenarios Using LiDAR Point Clouds and OpenStreetMap

基于LiDAR点云和OpenStreetMap的高速公路场景自动化数字双胞胎构建

Zhao, Yongqi, Bi, Dong, Kovacevic, Paul, Mihalj, Tomislav, Schabauer, Martin, Betz, Johannes, Eichberger, Arno

Abstract

Accurate road environment modeling is fundamental to the simulation and validation of automated driving systems. However, constructing road maps in standardized formats such as ASAM OpenDRIVE from real-world sensor data remains a time-consuming and costly process. Mobile mapping LiDAR captures accurate lane-level geometry but is confined to the driven corridor, while OpenStreetMap (OSM) provides broad road network topology but lacks geometric precision at the lane level. To address this, an automated workflow is proposed to fuse LiDAR point clouds with OSM data to generate georeferenced ASAM OpenDRIVE maps of highway environments, requiring minimal manual intervention. The pipeline reconstructs mainline roads from LiDAR-derived measurements and infers ramp geometry and topology from the OSM road graph, enabling complete highway interchange modeling without full sensor coverage. Experiments demonstrate a mean lateral RMSE of 0.740 m, and the generated maps are directly usable in mainstream simulation platforms including IPG CarMaker and Esmini. These results validate the effectiveness of combining measurement-derived geometry with map-derived topology for automated OpenDRIVE digital twin generation. The project code is available at https://github.com/ftgTUGraz/opendrive-digital-twin-generator

Chinese Translation

准确的道路环境建模是自动驾驶系统仿真与验证的基础。然而，从现实世界传感器数据构建标准化格式（如ASAM OpenDRIVE）的道路地图仍然是一个耗时且成本高昂的过程。移动测绘LiDAR能够捕获准确的车道级几何形状，但仅限于行驶走廊，而OpenStreetMap（OSM）提供了广泛的道路网络拓扑，但在车道级别缺乏几何精度。为了解决这一问题，提出了一种自动化工作流程，将LiDAR点云与OSM数据融合，以生成地理参考的ASAM OpenDRIVE高速公路环境地图，所需的人工干预最小。该流程从LiDAR测量中重建主干道，并从OSM道路图推断匝道几何形状和拓扑，实现了在没有完全传感器覆盖的情况下对高速公路互通的完整建模。实验表明，平均横向均方根误差（RMSE）为0.740米，生成的地图可以直接在包括IPG CarMaker和Esmini在内的主流仿真平台中使用。这些结果验证了将测量导出的几何形状与地图导出的拓扑相结合以实现自动化OpenDRIVE数字双胞胎生成的有效性。项目代码可在https://github.com/ftgTUGraz/opendrive-digital-twin-generator获取。

View on arXiv Download PDF AI Translation

cs.RO / 76 / 2606.16572

Steering Generative Reinforcement Learning into Stable Robotic Controller

将生成强化学习引导至稳定的机器人控制器

Wang, Yixuan, Ding, Shutong, Hu, Ke, Gui, Tianxiang, Wang, Jingya, Shi, Ye

Abstract

Diffusion and flow-based generative policies provide a powerful policy class for reinforcement learning by inducing rich stochastic exploration through iterative action generation. However, the stochasticity of diffusion policies is not suitable for stable and precise control in high-dimensional robotic systems, where small action variations can accumulate into inconsistent motion and reduced robustness. To address this issue, we propose SteerGenPO, a latent-space reinforcement learning framework that steers a trained generative policy into a robust deterministic robotic controller. The key idea is to replace stochastic latent sampling of the trained generative policy with a learned latent actor that predicts a state-dependent latent input for the generative policies. This separates exploration and control: stochastic generative sampling provides diverse action proposals during policy learning, while deterministic latent steering provides stable and adaptive control at deployment. We evaluate SteerGenPO on six Isaac Lab benchmarks and a Unitree G1 locomotion task. The results show SteerGenPO improves over both classical RL and generative RL baselines, while its deterministic latent steering produces more stable inference-time behaviors and more reliable command responses.

Chinese Translation

扩散和基于流的生成策略通过迭代动作生成引入丰富的随机探索，为强化学习提供了一种强大的策略类别。然而，扩散策略的随机性不适合高维机器人系统中的稳定和精确控制，因为小的动作变化可能会累积成不一致的运动并降低鲁棒性。为了解决这个问题，我们提出了SteerGenPO，一个潜在空间强化学习框架，它将训练好的生成策略引导至一个鲁棒的确定性机器人控制器。其关键思想是用一个学习到的潜在行为者替代训练生成策略的随机潜在采样，该行为者预测生成策略的状态依赖潜在输入。这将探索与控制分开：随机生成采样在策略学习过程中提供多样的动作建议，而确定性潜在引导在部署时提供稳定和自适应的控制。我们在六个Isaac Lab基准测试和一个Unitree G1运动任务上评估了SteerGenPO。结果表明，SteerGenPO在经典强化学习和生成强化学习基准上均有所提升，同时其确定性潜在引导在推理时产生了更稳定的行为和更可靠的指令响应。

View on arXiv Download PDF AI Translation

cs.RO / 77 / 2606.16600

WaveSync: Constrained Wavefront Optimization for Synchronized Co-Speech Gestures in Humanoid Robots

WaveSync：用于类人机器人同步共语手势的受限波前优化

Viet, Thang Tran, Canh, Thanh Nguyen, Uong, Gia Huy, Van Dinh, Phuc, Nguyen, Tan Viet Tuyen, HoangVan, Xiem, Chong, Nak Young

Abstract

Expressive co-speech gestures are crucial for natural human-robot interaction, but generating them on physical humanoid robots is difficult because gesture strokes must align with speech emphasis while satisfying strict kinematic and dynamic constraints. Unlike virtual avatars, humanoid robots cannot freely execute rapid or overlapping motions, making word-level synchronization and hardware-safe motion planning a coupled problem. We present \textbf{WaveSync}, a hybrid framework in which a Large Language Model decomposes dialogue responses into structured semantic schemas and assigns per-word importance weights, constructing a continuous Semantic Importance Wave. Gesture trajectories are shaped through Dynamic Movement Primitives, enforcing kinematic feasibility while enhancing expressiveness. A Wavefront Optimization stage aligns peak-to-peak gesture-speech synchronization and resolves residual kinematic violations through gesture-duration compression and forward propagation. Experimental evaluation based on five dialogue scenarios shows that our method achieves high synchronization accuracy and outperforms three baselines in both objective and subjective evaluations. Each component in WaveSync plays a necessary role in producing gestures that are expressive, semantically grounded, and kinematically compliant. The code, resources, and videos are available at \href{https://github.com/pairs-lab/WaveSync}{WaveSync}

Chinese Translation

富有表现力的共语手势对于自然的人机交互至关重要，但在物理类人机器人上生成这些手势却十分困难，因为手势的动作必须与语音的重音对齐，同时满足严格的运动学和动力学约束。与虚拟化身不同，类人机器人无法自由地执行快速或重叠的动作，这使得词级同步和硬件安全的运动规划成为一个耦合问题。我们提出了 extbf{WaveSync}，这是一个混合框架，其中大型语言模型将对话响应分解为结构化的语义模式，并为每个单词分配重要性权重，从而构建一个连续的语义重要性波。手势轨迹通过动态运动原语（Dynamic Movement Primitives）进行塑造，确保运动学的可行性，同时增强表现力。波前优化阶段对齐手势与语音的峰对峰同步，并通过手势持续时间压缩和前向传播解决残余的运动学违规问题。基于五个对话场景的实验评估表明，我们的方法实现了高同步精度，并在客观和主观评估中优于三个基线方法。WaveSync中的每个组件在生成富有表现力、语义扎根且运动学合规的手势中都发挥了必要的作用。代码、资源和视频可在 exttt{https://github.com/pairs-lab/WaveSync}获取。

View on arXiv Download PDF AI Translation

cs.RO / 78 / 2606.16621

Reinforcement Learning with Inner-loop Dynamics Estimator for Aerial Manipulation under Uncertainty

基于内循环动态估计器的强化学习在不确定性下的空中操控

Singh, Shivansh Pratap, Ujjwal, Samaksh, Chaudhary, Ishita, Vasudevan, V R, Yadav, Rishabh Dev, Roy, Spandan

Abstract

Aerial manipulators enable physical interaction in hard-to-reach environments; however, the combined problem of direct whole-body aerial manipulation under rapid arm motion, payload changes, and related unknown dynamic uncertainty remains a largely unsolved problem. We present a hierarchical control framework that combines Reinforcement Learning (RL) with an inner-loop dynamics estimator to address this problem. The RL outer loop maps desired 6-degrees-of-freedom (DOF) end-effector targets to coordinated whole-body commands, enabling direct task-driven control without relying on a fully accurate coupled dynamic model in the policy layer. An inner loop then tracks these commands while compensating for transient inertial shifts and uncertainty during execution via a dynamics estimator scheme without requiring system model knowledge. We validate the proposed approach on a custom quadrotor equipped with a 3-DoF manipulator through hardware experiments under varying payload conditions. Compared with RL+PID and RL+INDI+PID baselines, the proposed method reduces end-effector tracking error and improves task success rate across the tested hardware conditions. These results show that combining learned whole-body coordination with estimator-based low-level compensation improves the precision and robustness of aerial manipulation under changing operating conditions.

Chinese Translation

空中操控器能够在难以到达的环境中实现物理交互；然而，快速臂部运动、有效载荷变化以及相关的未知动态不确定性下的整体空中操控问题仍然是一个尚未解决的难题。我们提出了一种层次控制框架，将强化学习（Reinforcement Learning, RL）与内循环动态估计器相结合，以解决这一问题。RL外循环将期望的6自由度（6-DOF）末端执行器目标映射为协调的整体指令，从而实现直接的任务驱动控制，而无需在策略层依赖完全准确的耦合动态模型。内循环则在执行过程中跟踪这些指令，同时通过动态估计器方案补偿瞬态惯性变化和不确定性，而无需系统模型知识。我们在一台配备3自由度（3-DoF）操控器的定制四旋翼上验证了所提出的方法，通过在不同有效载荷条件下进行硬件实验。与RL+PID和RL+INDI+PID基线相比，所提出的方法在测试的硬件条件下减少了末端执行器跟踪误差，并提高了任务成功率。这些结果表明，将学习到的整体协调与基于估计器的低级补偿相结合，提高了在变化操作条件下的空中操控的精确性和鲁棒性。

View on arXiv Download PDF AI Translation

cs.RO / 79 / 2606.16690

PATCH: Action-Chunk-Conditioned Latent Patch Innovation Monitoring for Robot Manipulation

PATCH：基于动作块条件的潜在补丁创新监控用于机器人操作

Zhou, Yanan, Qiu, Ranpeng, Chen, Yincong, Cui, Jiajie, Zhi, Weiming

Abstract

Learning-based manipulation policies have made substantial progress in real-world robot manipulation, particularly for short-horizon action generation. However, deployment in open workspaces remains fragile under unexpected local scene dynamics, such as moving objects, transient occlusions, or disturbances near the intended motion. Existing runtime monitors often rely on global observation anomalies, policy uncertainty, or frame-level visual changes, and struggle to distinguish task-relevant execution risk from benign visual variation. We introduce PATCH, an action-chunk-conditioned latent patch innovation monitor for deployment-time intervention. Given the active action chunk, PATCH defines a projected execution corridor, predicts latent patch evolution inside it, and accumulates persistent residuals unexplained by the robot's own motion. These residuals form a localized intervention signal that allows PATCH-Router to pause execution, select an available recovery source, and resume the original policy once localized innovation subsides. Experiments on real robot rollout data show that PATCH produces more stable and context-relevant triggers than competing runtime monitors. Real-robot deployment further demonstrates monitor-driven intervention and policy resumption for disturbance-aware manipulation. Project Page: https://yananzhou5555.github.io/PATCH/.

Chinese Translation

基于学习的操作策略在现实世界的机器人操作中取得了显著进展，特别是在短期动作生成方面。然而，在开放工作空间中的部署在面对意外的局部场景动态时仍然脆弱，例如移动物体、瞬态遮挡或靠近预期运动的干扰。现有的运行时监控器通常依赖于全局观察异常、策略不确定性或帧级视觉变化，难以区分与任务相关的执行风险和良性视觉变化。我们提出了PATCH，一种基于动作块条件的潜在补丁创新监控器，用于部署时干预。给定活动的动作块，PATCH定义了一个预测的执行走廊，预测其内部的潜在补丁演变，并累积机器人自身运动无法解释的持续残差。这些残差形成了一个局部干预信号，使得PATCH-Router能够暂停执行，选择可用的恢复源，并在局部创新减弱后恢复原始策略。在真实机器人滚动数据上的实验表明，PATCH产生的触发信号比竞争的运行时监控器更稳定且与上下文相关。真实机器人部署进一步展示了监控驱动的干预和对干扰感知操作的策略恢复。项目页面：https://yananzhou5555.github.io/PATCH/

View on arXiv Download PDF AI Translation

cs.RO / 80 / 2606.16696

VENOM: Versatile Embodied Network for Omni-bodied Motion tracking

VENOM：用于全身运动跟踪的多功能具身网络

Padmanabhan, Siddharth, Miyazawa, Kazuki, Horii, Takato

Abstract

Achieving expert-level expressive full-body motion tracking across multiple humanoids solely from demonstration data remains a challenging and relatively an underexplored problem in humanoid robot learning. Cross-embodiment motion tracking policies are mostly trained by decoupling the control problem into upper and lower body control. This work proposes VENOM, a cross-embodiment full-body motion tracking model for humanoids in simulation. VENOM is a GPT-based motion tracker trained on multiple humanoid data that can track the entire body without the requirement to split into upper and lower body control. We curate a multi-humanoid motion tracking dataset called the VENOM dataset that contains states, actions, and rewards and train VENOM and the baselines on this dataset. In this letter, we evaluate VENOM's performance against baselines and show that we can achieve a stable motion tracker across different humanoids more capable than an MLP trained on multiple humanoid data with supervised learning alone, and also show that despite lack of reward feedback, VENOM closely matches the tracking capability of experts that were trained using asymmetric-actor critic reinforcement learning.

Chinese Translation

仅通过示范数据实现跨多个类人机器人专家级的全身运动跟踪仍然是一个具有挑战性且相对未被充分探索的问题。跨具身运动跟踪策略通常通过将控制问题解耦为上半身和下半身控制来进行训练。本研究提出了VENOM，一个用于仿真中类人机器人的跨具身全身运动跟踪模型。VENOM是一个基于GPT的运动跟踪器，训练于多个类人机器人数据，能够跟踪整个身体而无需拆分为上半身和下半身控制。我们整理了一个名为VENOM数据集的多类人机器人运动跟踪数据集，其中包含状态、动作和奖励，并在该数据集上训练VENOM及基线模型。在本文中，我们评估了VENOM的性能与基线模型的对比，显示我们能够在不同类人机器人之间实现一个更稳定的运动跟踪器，其能力超过了仅通过监督学习训练的多类人机器人数据的多层感知器（MLP），并且尽管缺乏奖励反馈，VENOM的跟踪能力与使用不对称演员评论家强化学习训练的专家相近。

View on arXiv Download PDF AI Translation

cs.RO / 81 / 2606.16735

Pride and Prejudice: Toward an Information-Theoretic Framework for Mutually Communicative Driver Behavior Modeling

自豪与偏见：朝着一个信息论框架构建互相沟通的驾驶行为模型

Li, Tingjun, Xu, Nan, Feng, Shuo, Askari, Hassan, Barbosa, Bruno Henrique Groenner, Guo, Konghui

Abstract

Mixed autonomy driving becomes unsafe and inefficient when autonomous vehicles (AVs) and human-driven vehicles (HVs) misread each other's intentions. We study this problem as implicit mutual communication in lane changes. The proposed framework models how the ego vehicle both expresses its intent and probes the other driver's preference under epistemic uncertainty. It combines a level-k Bayesian persuasion game with virtual features for proactive signaling, information-theoretic rewards for mutual communication, and adaptive weights of communication affordances. We further introduce the Pride-Inquiry (P-I) and Pride-Prejudice (P-P) planes to analyze communication intensity and tendency. The model is calibrated with a Communication-Based Multi-Agent Inverse Reinforcement Learning algorithm (C-MIRL) on the naturalistic NGSIM dataset. Compared with the non-communicative baseline, the proposed model reduces the prediction error of mandatory lane changes by up to 20% while maintaining strong generalization. Driver-In-the-Loop questionnaire scores are positively correlated with the calibrated communication variables, supporting the subjective validity of the model. The learned rewards further show that inquiry and listening affordances contribute more than pride and expression alone, and that inquiry preference varies more strongly across drivers. These results support explicit modeling of mutual communication and epistemic uncertainty in interactive driving.

Chinese Translation

当自主车辆（AVs）与人类驾驶车辆（HVs）误读彼此意图时，混合自主驾驶变得不安全且低效。我们将此问题视为车道变换中的隐性相互沟通。所提出的框架模型描述了自我车辆如何在认知不确定性下表达其意图并探测其他驾驶者的偏好。该框架结合了一个层级-k 贝叶斯劝说游戏、用于主动信号传递的虚拟特征、用于相互沟通的信息论奖励，以及沟通能力的自适应权重。我们进一步引入了自豪-探询（Pride-Inquiry, P-I）和平面自豪-偏见（Pride-Prejudice, P-P）来分析沟通的强度和倾向。该模型通过基于沟通的多智能体逆强化学习算法（Communication-Based Multi-Agent Inverse Reinforcement Learning, C-MIRL）在自然驾驶的 NGSIM 数据集上进行校准。与非沟通基线相比，所提出的模型将强制车道变换的预测误差降低了多达 20%，同时保持了良好的泛化能力。驾驶者反馈问卷得分与校准后的沟通变量呈正相关，支持了模型的主观有效性。学习到的奖励进一步表明，探询和倾听能力的贡献大于单纯的自豪和表达，而探询偏好在不同驾驶者之间的变化更为显著。这些结果支持在互动驾驶中明确建模相互沟通和认知不确定性。

View on arXiv Download PDF AI Translation

cs.RO / 82 / 2606.16776

DataLadder: A Simulation-Enabled Interconversion Toolchain for the Embodied Data Pyramid

DataLadder：一个支持仿真的体现数据金字塔互转工具链

Liu, Peidong, Liu, Yongce, Guo, Songyan, Ma, Fuyuan, Yuan, Zhihao, Li, Ao, Chen, Zengjue, Li, Wenhao, Zhang, Tianle, Li, Mingyang, Zhang, Jiale, Xiong, Junzhe, Xiang, Zhiyuan, Chi, Dafeng, Zhuang, Yuzheng, Li, Yihang, He, Qingrong, Liang, Jiaming, Cai, Chen, Hao, Peng, Luo, Mingxi, Wang, Song, Xiong, Junwu, Li, Ruodai, Luo, Liyi, Tan, Wei, Li, Dongjiang, Li, Jiawei, Shen, Hui, Gong, Yicheng, Lin, Liang

Abstract

Generalist robot policies require trustworthy evaluation and robot-usable training data, but both are difficult to scale with physical robots alone. Real-robot trials and demonstrations remain the most faithful source of deployment signals, yet they are slow, costly, and hard to reproduce. We present DataLadder, a simulation-enabled interconversion toolchain for human-robot aligned model evaluation and data generation, denoted as Robot $\rightleftharpoons$ Simulation $\rightleftharpoons$ Human. On the one hand, the Robot $\rightarrow$ Simulation $\rightarrow$ Human pathway supports human-robot aligned model evaluation by reconstructing real-robot tabletop organization tasks as calibrated digital twins for scalable evaluation, while using human embodied feedback to inspect and refine the naturalness of simulated motions. On the other hand, the Human $\rightarrow$ Simulation $\rightarrow$ Robot pathway supports human-robot aligned data generation: it lifts ego-centric human demonstrations into simulation, checks them under robot physical constraints, and converts them into robot-centered trajectories, annotations, and visual observations. Together, these pathways use the JoySim simulator as both a scalable evaluation layer and a physical consistency filter for robot data generation. We further package the core reconstruction, simulation, rendering, and realism-augmentation modules as cloud services on JD Cloud, turning the system into reusable infrastructure for robot data generation and model evaluation.

Chinese Translation

通用机器人策略需要可靠的评估和可供机器人使用的训练数据，但仅依靠物理机器人进行扩展是困难的。真实机器人试验和演示仍然是部署信号最真实的来源，但它们的速度慢、成本高且难以重现。我们提出了DataLadder，一个支持仿真的人机对齐模型评估和数据生成的互转工具链，表示为Robot $ ightleftharpoons$ Simulation $ ightleftharpoons$ Human。一方面，Robot $ ightarrow$ Simulation $ ightarrow$ Human路径通过将真实机器人桌面组织任务重建为经过校准的数字双胞胎来支持人机对齐的模型评估，以便进行可扩展的评估，同时利用人类的体现反馈来检查和优化模拟动作的自然性。另一方面，Human $ ightarrow$ Simulation $ ightarrow$ Robot路径支持人机对齐的数据生成：它将以自我为中心的人类演示提升到仿真中，在机器人物理约束下进行检查，并将其转换为以机器人为中心的轨迹、注释和视觉观察。这些路径共同使用JoySim模拟器作为可扩展的评估层和机器人数据生成的物理一致性过滤器。我们进一步将核心重建、仿真、渲染和现实增强模块打包为京东云上的云服务，将该系统转变为可重用的机器人数据生成和模型评估基础设施。

View on arXiv Download PDF AI Translation

cs.RO / 83 / 2606.16780

DIFF-IPPO: Diffusion-Based Informative Path Planning with Open-Vocabulary Belief Maps

DIFF-IPPO：基于扩散的开放词汇信念图的有信息路径规划

Karaf, Sausar, Sautenkov, Oleg, Martynov, Mikhail, Tsetserukou, Dzmitry

Abstract

Exploration and object search require robots to perceive their environment, identify regions of interest, and plan trajectories that improve target-detection likelihood or maximize information gain. Many IPP methods, especially in continuous environmental monitoring, rely on Gaussian-process belief models, while object-search settings often produce complex, multimodal belief maps from semantic or open-vocabulary perception. Global trajectory generation directly conditioned on such non-Gaussian belief maps remains comparatively underexplored. Although diffusion-based planners offer strong capabilities for modeling such distributions, their use in informative path planning remains limited. In this work, we propose DIFF-IPPO, a pipeline that integrates an open-vocabulary belief map generator with a diffusion-based planner for global trajectory generation over belief maps. The method generates trajectories that concentrate sensor coverage over high-belief regions, achieving normalized detection scores between 81.49% and 86.55% across different dataset scenarios. We validate the system in a simulated search-and-rescue scenario where the planner searches candidate building regions to locate a burning building. In this setting, a team of five drones using batched belief-map-conditioned trajectory generation achieves first detections in 3.5 minutes.

Chinese Translation

探索和物体搜索要求机器人感知其环境，识别感兴趣区域，并规划能够提高目标检测概率或最大化信息增益的轨迹。许多信息路径规划（IPP）方法，特别是在连续环境监测中，依赖于高斯过程信念模型，而物体搜索场景通常会产生来自语义或开放词汇感知的复杂多模态信念图。基于此类非高斯信念图的全局轨迹生成仍然相对未被深入探讨。尽管基于扩散的规划器在建模这些分布方面具有强大的能力，但其在有信息路径规划中的应用仍然有限。在本研究中，我们提出了DIFF-IPPO，一个将开放词汇信念图生成器与基于扩散的规划器集成的管道，用于在信念图上进行全局轨迹生成。该方法生成的轨迹集中传感器覆盖在高信念区域，实现了在不同数据集场景下81.49%至86.55%之间的标准化检测分数。我们在一个模拟的搜索与救援场景中验证了该系统，其中规划器搜索候选建筑区域以定位燃烧的建筑。在这种设置中，使用批量信念图条件轨迹生成的五架无人机团队在3.5分钟内实现了首次检测。

View on arXiv Download PDF AI Translation

cs.RO / 84 / 2606.16788

SoK: Security and Privacy of Foundation-Model-Powered Robots

安全与隐私：基础模型驱动机器人

Gong, Xueluan, Chen, Chen, Liu, Jinxin, Wang, Qian, Lam, Kwok-Yan

Abstract

Foundation models are reshaping robotics by enabling robots to interpret open-ended instructions, reason over multimodal contexts, and operate in complex, open-world environments. However, their integration also introduces security and privacy (S&P) risks that extend beyond the FMs themselves to embodied execution pipelines, supporting ecosystems, and broader governance impacts. Existing literature reviews provide valuable insights but often focus on specific FM types, risk categories, mitigation strategies, or trust boundaries. Consequently, the field lacks a unified structure for analyzing where risks originate, how they propagate across robotic systems, and where mitigations should intervene. To address this gap, we propose a progressive F-E-S-G structural boundary framework for analyzing the S&P of FM-powered robots. The framework comprises four layers: the Foundation model layer (F), Embodied system layer (E), Supporting ecosystem layer (S), and Governance impact layer (G). Building on this structure, we develop a multi-level taxonomy that organizes prior studies along three levels: F-E-S-G trust boundary, security-privacy concerns, and risk-mitigation perspectives. We further annotate each study using fine-grained coding attributes, including target, lifecycle stage, mechanism, system access, and effect. Guided by this framework and taxonomy, we systematize 96 papers. Our analysis uncovers multiple threat patterns, defense mismatches, and evaluation gaps that are difficult to identify from a single-boundary perspective. Based on these findings, we identify open challenges and future directions to provide a research agenda for developing secure, privacy-preserving, and responsibly governed FM-powered robotic systems.

Chinese Translation

基础模型正在通过使机器人能够理解开放式指令、在多模态环境中进行推理以及在复杂的开放世界环境中操作，重塑机器人技术。然而，它们的整合也引入了超出基础模型本身的安全与隐私（S&P）风险，这些风险扩展到具身执行管道、支持生态系统以及更广泛的治理影响。现有文献综述提供了有价值的见解，但通常集中于特定的基础模型类型、风险类别、缓解策略或信任边界。因此，该领域缺乏一个统一的结构来分析风险的来源、如何在机器人系统中传播以及缓解措施应介入的地方。为了解决这一空白，我们提出了一种渐进的基础-具身-支持-治理（F-E-S-G）结构边界框架，用于分析基础模型驱动机器人的安全与隐私。该框架由四个层次组成：基础模型层（F）、具身系统层（E）、支持生态系统层（S）和治理影响层（G）。基于这一结构，我们开发了一个多层次分类法，将先前的研究按三个层次组织：F-E-S-G信任边界、安全-隐私关注和风险-缓解视角。我们进一步使用细粒度编码属性对每项研究进行注释，包括目标、生命周期阶段、机制、系统访问和效果。在该框架和分类法的指导下，我们系统化了96篇论文。我们的分析揭示了多种威胁模式、防御不匹配和评估缺口，这些在单一边界视角下难以识别。基于这些发现，我们识别出开放挑战和未来方向，为开发安全、隐私保护和负责任治理的基础模型驱动机器人系统提供了研究议程。

View on arXiv Download PDF AI Translation

cs.RO / 85 / 2606.16826

ATOM-Bench: A Real-World Benchmark for Atomic Skills and Compositional Generalization in Manipulation Policies

ATOM-Bench：用于评估操作策略中的原子技能和组合泛化的现实世界基准

Wu, Zenan, Wei, Bingqing, Liu, Lu, He, Zheqi, Wang, Xi, Liu, Jiakang, Li, Zehui, Yao, Guocai, Zheng, Jing-Shu, Yang, Xi, Wang, Yongtao

Abstract

Generalist manipulation policies are increasingly presented as foundation models for robotic control, but their real-world generalization remains difficult to diagnose. A policy may succeed on demonstrated tasks while still failing to execute fine-grained atomic skills or recombine learned skills in new task structures. We introduce \textbf{ATOM-Bench}, a real-world benchmark for evaluating both atomic skills and compositional generalization in manipulation policies. ATOM-Bench factorizes tabletop manipulation into motor atoms and instruction atoms, and contains 30 atomic tasks and 24 held-out compositional tasks across paired single-arm and dual-arm robot tracks. We collect 3,000 human demonstrations for atomic fine-tuning and release both the demonstration data and evaluation rollout data to support reproducible real-world evaluation. Policies are fine-tuned on atomic tasks and evaluated on both atomic skill acquisition and held-out compositional tasks. We further introduce Atomic Score (AS) and Compositional Failure Share (CFS) to distinguish failures caused by weak atomic skills from failures caused by limited compositional reuse. Through 2,700 physical rollouts on five representative manipulation policies, we find that current policies can acquire simple instruction-grounding skills, but still struggle with fine-grained motor atoms, counting, and logical filtering. More importantly, strong atomic performance does not reliably transfer to held-out compositional tasks. ATOM-Bench provides a diagnostic testbed for studying whether failures arise from weak motor execution, poor instruction grounding, or limited compositional reuse.

Chinese Translation

通用操作策略越来越被视为机器人控制的基础模型，但其在现实世界中的泛化能力仍然难以诊断。一个策略可能在示范任务上成功，但仍然无法执行细粒度的原子技能或在新的任务结构中重新组合学习到的技能。我们引入了 extbf{ATOM-Bench}，这是一个用于评估操作策略中原子技能和组合泛化的现实世界基准。ATOM-Bench 将桌面操作分解为运动原子和指令原子，并包含30个原子任务和24个保留的组合任务，涵盖了单臂和双臂机器人轨道。我们收集了3,000个用于原子微调的人类示范，并发布了示范数据和评估回放数据，以支持可重复的现实世界评估。策略在原子任务上进行微调，并在原子技能获取和保留的组合任务上进行评估。我们进一步引入了原子得分（Atomic Score, AS）和组合失败率（Compositional Failure Share, CFS），以区分由弱原子技能引起的失败与由有限组合重用引起的失败。通过对五个代表性操作策略进行2,700次物理回放，我们发现当前策略能够获取简单的指令基础技能，但在细粒度运动原子、计数和逻辑过滤方面仍然存在困难。更重要的是，强大的原子性能并不能可靠地转移到保留的组合任务上。ATOM-Bench 提供了一个诊断测试平台，用于研究失败是由于运动执行能力弱、指令基础差，还是由于组合重用有限。

View on arXiv Download PDF AI Translation

cs.RO / 86 / 2606.16856

Video-Based Optimal Transport for Feedback-Efficient Offline Preference-Based Reinforcement Learning

基于视频的最优传输用于反馈高效的离线偏好强化学习

Luu, Tung M., Kim, Hwanhee, Lee, Younghwan, Yoo, Chang D.

Abstract

Conveying complex objectives to reinforcement learning (RL) agents often requires meticulous reward engineering. Preference-based RL (PbRL) offers a promising alternative by learning reward functions from human feedback, but its scalability is hindered by high labeling costs. Inspired by advances in Video Foundation Models (ViFMs), we present Video-based Optimal Transport Preference (VOTP), a semi-supervised framework that learns effective reward functions from only a handful of labels. By leveraging optimal transport to align visual trajectories within the rich representation space of ViFMs, VOTP effectively generates high-fidelity pseudo-labels for large amounts of unlabeled data, substantially reducing human supervision. Extensive experiments across locomotion and manipulation benchmarks demonstrate the superiority of VOTP, which outperforms state-of-the-art offline PbRL methods under limited feedback budgets. We also showcase the robustness of VOTP in the presence of visual distractors and validate its utility on real robotic tasks, where it learns meaningful rewards with minimal human input.

Chinese Translation

将复杂目标传达给强化学习（RL）代理通常需要精心设计奖励机制。基于偏好的强化学习（PbRL）通过从人类反馈中学习奖励函数提供了一种有前景的替代方案，但其可扩展性受到高标注成本的限制。受视频基础模型（ViFMs）进展的启发，我们提出了基于视频的最优传输偏好（VOTP），这是一个半监督框架，仅通过少量标签学习有效的奖励函数。通过利用最优传输对齐ViFMs丰富表示空间中的视觉轨迹，VOTP有效地为大量未标记数据生成高保真伪标签，从而大幅减少人类监督。在运动和操控基准上的大量实验表明，VOTP的优越性，其在有限反馈预算下超越了最先进的离线PbRL方法。我们还展示了VOTP在视觉干扰物存在下的鲁棒性，并验证了其在真实机器人任务中的实用性，在这些任务中，它以最小的人类输入学习有意义的奖励。

View on arXiv Download PDF AI Translation

cs.RO / 87 / 2606.16876

ExoTraj: A General Lower-limb Exoskeleton Assistance Policy for Complex Environments

ExoTraj：一种适用于复杂环境的通用下肢外骨骼辅助策略

Liu, Xiao-Yin, Li, Guotao, Sun, Long, Liang, Xu, Hou, Zeng-Guang

Abstract

Adaptive torque prediction in dynamic exoskeleton scenarios requires expensive motion capture systems, which are infeasible in complex outdoor environments. Trajectory prediction has emerged as one of the effective approaches to address such an issue. However, the core challenges of exoskeleton trajectory prediction are twofold: establishing the mapping from multi-modal features to trajectory information; constructing the mapping from trajectory to torque. For the former, most existing methods perform only single-step prediction and neglect inter-subject trajectory variability, thereby limiting the trajectory optimization space and prediction generalization. To address this, this paper proposes a fast flow matching method that enables accurate trajectory prediction and better generalization for real-time performance, where trajectory generation errors and encoded observations are used to guide the training direction. For the second challenge, due to the high dynamics of the human-robot system and the strong coupling between perception and control, simple control methods struggle to achieve efficient assistance based on the predicted trajectory. This paper utilizes model predictive control and designs a novel optimization objective to optimize torque, ensuring the exoskeleton achieves comfortable and robust assistance. By integrating the above two components, the unified policy, denoted as ExoTraj, is developed to enable adaptive assistance in complex outdoor scenarios without high data acquisition cost. Experimental results show that compared to traditional methods, ExoTraj reduces cross-subject prediction error by 14.0% during the online phase and maintains robustness against external noise. Relative to the zero torque condition, ExoTraj decreases metabolic rate by 11.5-24.4%, heart rate by 1.7-19.5%, and peak muscle activation levels by 10.9-41.3%, respectively.

Chinese Translation

在动态外骨骼场景中，自适应扭矩预测需要昂贵的动作捕捉系统，这在复杂的户外环境中是不可行的。轨迹预测已成为解决此类问题的有效方法之一。然而，外骨骼轨迹预测的核心挑战有两个：建立多模态特征与轨迹信息之间的映射；构建轨迹与扭矩之间的映射。对于前者，大多数现有方法仅执行单步预测，并忽视了个体间轨迹的变异性，从而限制了轨迹优化空间和预测的泛化能力。为了解决这个问题，本文提出了一种快速流匹配方法，使得轨迹预测更加准确，并提高实时性能的泛化能力，其中轨迹生成误差和编码观察结果被用来指导训练方向。对于第二个挑战，由于人机系统的高动态性以及感知与控制之间的强耦合，简单的控制方法难以基于预测轨迹实现有效的辅助。本文利用模型预测控制，设计了一种新颖的优化目标来优化扭矩，确保外骨骼实现舒适且稳健的辅助。通过整合上述两个组件，开发了统一策略ExoTraj，以便在复杂的户外场景中实现自适应辅助，而无需高数据采集成本。实验结果表明，与传统方法相比，ExoTraj在在线阶段将跨个体预测误差降低了14.0%，并保持对外部噪声的鲁棒性。相对于零扭矩条件，ExoTraj分别降低了11.5-24.4%的代谢率，1.7-19.5%的心率，以及10.9-41.3%的峰值肌肉激活水平。

View on arXiv Download PDF AI Translation

cs.RO / 88 / 2606.16881

SGM-SLAM: Scene Graph Matching for Data-Efficient Distributed SLAM

SGM-SLAM：用于数据高效分布式SLAM的场景图匹配

Huang, Yewei, Shan, Tixiao, Rajvanshi, Abhinav, Mithun, Niluthpol Chowdhury, Li, Yaxuan, Englot, Brendan, Chiu, Han-Pang

Abstract

We introduce a data-efficient distributed Simultaneous Localization and Mapping (SLAM) framework designed for a team of robots equipped with LiDAR, cameras, and inertial sensors. Our framework uses scene graph matching to identify inter-robot measurement constraints. Unlike prior approaches that rely on feature-level matching, our framework is the first to perform scene graph matching using only object labels and centroids. Our approach constructs a scene graph by using fused RGB-LiDAR point clouds to generate both a semantically segmented point cloud layer, and a layer of discrete bounded objects, to accompany estimated robot trajectories. Scene graph matching is performed collaboratively through exchanging and matching object data with neighboring robots. To maximize communication efficiency, we utilize a multi-step data exchange and optimization process. We demonstrate the effectiveness and efficiency of our approach using both simulation and real-world datasets collected by legged robots in indoor and outdoor environments.

Chinese Translation

我们提出了一种数据高效的分布式同步定位与地图构建（SLAM）框架，旨在为配备激光雷达、摄像头和惯性传感器的机器人团队提供支持。我们的框架利用场景图匹配来识别机器人之间的测量约束。与依赖特征级匹配的先前方法不同，我们的框架首次仅使用对象标签和质心进行场景图匹配。我们的方法通过融合RGB-LiDAR点云构建场景图，以生成语义分割的点云层和离散边界对象层，以配合估计的机器人轨迹。场景图匹配通过与邻近机器人交换和匹配对象数据进行协作完成。为了最大化通信效率，我们采用了多步骤的数据交换和优化过程。我们通过在室内和室外环境中由腿式机器人收集的模拟和真实世界数据集，展示了我们方法的有效性和效率。

View on arXiv Download PDF AI Translation

cs.RO / 89 / 2606.16888

LOPAL: Local Performance-Aware Active Learning from Imperfect Demonstrations

LOPAL：基于局部性能感知的主动学习方法，从不完美的示范中学习

Heidersberger, Johannes, Jadav, Shail, Lee, Dongheui

Abstract

Learning from Demonstration (LfD) enables intuitive robot skill acquisition by allowing robots to learn directly from human task demonstrations. However, current methods often fail to address the fact that due to suboptimal and inconsistent human behavior, the quality of the demonstration can vary within each demonstration. Therefore, we introduce LOPAL (LOcal Performance-aware Active Learning), an active learning approach that leverages this local demonstration quality information. Our approach consists of two synergistic components. First, a local performance-driven LfD method uses a Gaussian Mixture Model (GMM) to encode both the demonstrated trajectories and their associated local quality assessments. This enables the generation of trajectories that outperform the imperfect demonstrations by utilizing complementary local data of high performance. Second, active data acquisition allows to improve beyond the imperfect demonstrations by collecting additional informative samples. In areas missing good data, the user is actively requested to provide corrections through a shared autonomy (SA) mechanism, while the robot autonomously executes the learned behavior. The efficacy of LOPAL was validated in both a simulation and a real-world experiment. The results from a real-world pipe inspection task showed that the proposed approach can achieve up to 27.31 % improvement in task performance while also reducing the effort required to collect the demonstrations.

Chinese Translation

从示范学习（LfD）使得机器人能够通过直接从人类任务示范中学习，从而实现直观的技能获取。然而，当前的方法往往未能解决这样一个事实：由于人类行为的次优和不一致性，每个示范中的示范质量可能会有所不同。因此，我们提出了LOPAL（局部性能感知主动学习），这是一种利用局部示范质量信息的主动学习方法。我们的方法由两个协同的组成部分构成。首先，基于局部性能的LfD方法使用高斯混合模型（GMM）来编码示范轨迹及其相关的局部质量评估。这使得通过利用高性能的互补局部数据生成优于不完美示范的轨迹成为可能。其次，主动数据获取允许通过收集额外的信息样本来超越不完美的示范。在缺乏良好数据的区域，用户通过共享自主（SA）机制被主动请求提供修正，而机器人则自主执行学习到的行为。LOPAL的有效性在模拟和真实世界实验中得到了验证。来自真实世界管道检查任务的结果表明，所提出的方法在任务性能上可以实现高达27.31%的提升，同时减少了收集示范所需的努力。

View on arXiv Download PDF AI Translation

cs.RO / 90 / 2606.16902

Binary Tracking for Spatial QA and Navigation with Open Vision-Language Models

基于开放视觉语言模型的空间问答与导航的二元跟踪

Na, Dongbin, Kim, Chanwoo, Rho, Soonbin, Choi, Giyun, Lee, Gangbok, Hong, Dooyoung

Abstract

This work addresses spatial question answering for service robots traversing long egocentric routes. Given a query such as "where can I find a dry cleaner on the way back home?", the system returns a metric coordinate that downstream navigation components can act on. Prior Spatial Question Answering approaches leverage retrieval-augmented agents built on closed-source models such as GPT-4o for path exploration. However, robots operating in the real world often cannot reliably depend on online closed-source models due to network instability, communication latency, and deployment cost. It creates a need for open-source based Spatial Question Answering approaches that can run onboard the robot, yet prior research in this direction remains limited. This work proposes BinTrack, a simple yet effective, fully open-source spatial-localization agent that leverages the temporal ordering of a robot's trajectory. BinTrack performs a binary search over the trajectory segments between two anchor landmarks identified from a query. It improves overall accuracy by up to 22.8% over other open-source implementations and even matches the reported closed-source model result on the global category of the SpaceLocQA benchmark, the most challenging setting that has so far required strong reasoning agents such as GPT-4o. Furthermore, its optimized inference strategy consistently yields more than a 1.5x inference speedup over previous approaches. Finally, this work releases GangnamLoop, a novel and practical multi-trip outdoor benchmark collected by deploying a real quadruped robot on public streets with the anonymization policy. It revisits the same locations under different outdoor conditions and pairs the robot's low viewpoint with the human owner's. The source codes and datasets are publicly available at https://github.com/ndb796/BinaryTracking

Chinese Translation

本研究针对服务机器人在长时间自我中心路径上进行空间问答的问题。给定一个查询，例如“我在回家的路上可以在哪里找到干洗店？”，系统返回一个度量坐标，以供下游导航组件使用。之前的空间问答方法依赖于基于封闭源模型（如GPT-4o）构建的增强检索代理进行路径探索。然而，现实世界中的机器人往往无法可靠地依赖在线封闭源模型，原因包括网络不稳定、通信延迟和部署成本。这就需要基于开源的空间问答方法，这些方法能够在机器人上运行，但在这一方向上的先前研究仍然有限。本研究提出了BinTrack，这是一种简单而有效的完全开源空间定位代理，利用机器人的轨迹的时间顺序。BinTrack在从查询中识别的两个锚点地标之间的轨迹段上执行二元搜索。与其他开源实现相比，它的整体准确性提高了多达22.8%，甚至在SpaceLocQA基准的全球类别中达到了报告的封闭源模型结果，这是迄今为止需要强推理代理（如GPT-4o）的最具挑战性的设置。此外，其优化的推理策略在速度上始终比之前的方法快超过1.5倍。最后，本研究发布了GangnamLoop，这是一个新颖且实用的多次出行户外基准，通过在公共街道上部署真实的四足机器人并遵循匿名政策收集而来。它在不同的户外条件下重新访问相同的位置，并将机器人的低视角与人类主人的视角配对。源代码和数据集已公开发布在 https://github.com/ndb796/BinaryTracking

View on arXiv Download PDF AI Translation

cs.RO / 91 / 2606.16917

Unified Motion-Action Modeling for Heterogeneous Robot Learning

异构机器人学习的统一运动-动作建模

Cao, Yunhao, Liu, Shitong, Feng, Chao, Zhang, Meryl, Lu, Xuanchen, Owens, Andrew, Fang, Kuan

Abstract

We present Unified Motion-Action (UMA) Model, an approach that uses 3D object motion trajectories as a shared interface to bridge visuomotor control and dynamics modeling. UMA treats object motion and robot actions as co-evolving variables under a masked generative objective, in which the mask pattern determines both the supervision regime during pretraining and the inference mode at deployment. Using hindsight-relabeled motion contexts and a contrastive objective that disentangles task intent from scene geometry, UMA enables multi-task pretraining across heterogeneous data sources without requiring manually annotated task instructions. At deployment, the same pretrained parameters support motion-conditioned visuomotor control, motion-based dynamics modeling, and task adaptation from few-shot demonstrations. Pretrained on a mixture of robot demonstrations, human videos, and simulated data, UMA consistently outperforms state-of-the-art baselines specialized for each inference mode.

Chinese Translation

我们提出了统一运动-动作（UMA）模型，这是一种利用三维物体运动轨迹作为共享接口，连接视觉运动控制与动力学建模的方法。UMA将物体运动与机器人动作视为在掩蔽生成目标下共同演变的变量，其中掩蔽模式决定了预训练期间的监督机制和部署时的推理模式。通过利用事后重新标记的运动上下文和一种将任务意图与场景几何分离的对比目标，UMA使得在异构数据源上进行多任务预训练成为可能，而无需手动标注的任务指令。在部署阶段，相同的预训练参数支持基于运动的视觉运动控制、基于运动的动力学建模以及从少量示范中进行任务适应。经过在机器人示范、人类视频和模拟数据的混合数据上预训练，UMA在每种推理模式下均优于专门针对各自推理模式的最新基线。

View on arXiv Download PDF AI Translation

cs.RO / 92 / 2606.16935

CrossMaps: Confidence-Aware Open-Vocabulary Semantic Mapping for Rover Navigation

CrossMaps：基于信心的开放词汇语义映射用于探测器导航

Klein, Jan-Niklas, Ghahremani, Sona, Adriano, Christian Medeiros, Giese, Holger

Abstract

Rovers rely on perception to maintain spatial maps that encode both objects and sensor quality (e.g., range reliability, lighting artifacts, data density), guiding data fusion, embedding updates, and navigation under partial observability. To study these coupled perception-navigation processes, we present CrossMaps, a real-time confidence-aware open-vocabulary semantic mapping pipeline that constructs language-queryable maps from RGB-D data. Building on VLMaps-style approaches, CrossMaps integrates multi-scale CLIP embeddings with confidence-aware fusion and a dual-memory architecture consisting of Short-Term Memory (STM) and Long-Term Memory (LTM). The STM aggregates noisy visual observations using geometric, semantic, and temporal confidence cues, while confident and coherent cells are promoted to the LTM as persistent semantic landmarks. Designed for deployment with a Jetson Orin-powered UGV alongside SLAM, CrossMaps runs in real time and produces semantic heatmaps that can be queried with natural language to guide rover navigation.

Chinese Translation

探测器依赖感知来维护空间地图，这些地图编码了物体和传感器质量（例如，范围可靠性、光照伪影、数据密度），指导数据融合、嵌入更新和在部分可观测性下的导航。为了研究这些耦合的感知-导航过程，我们提出了CrossMaps，这是一种实时的基于信心的开放词汇语义映射管道，能够从RGB-D数据构建可通过语言查询的地图。基于VLMaps风格的方法，CrossMaps集成了多尺度的CLIP嵌入、基于信心的融合以及由短期记忆（STM）和长期记忆（LTM）组成的双重记忆架构。STM使用几何、语义和时间信心线索聚合噪声视觉观测，而自信且一致的单元则被提升到LTM，作为持久的语义地标。CrossMaps旨在与搭载Jetson Orin的无人地面车辆（UGV）及SLAM一起部署，能够实时运行并生成可以用自然语言查询的语义热图，以指导探测器导航。

View on arXiv Download PDF AI Translation

cs.RO / 93 / 2606.16953

SidewalkBench: Benchmarking Visual Navigation on Urban Sidewalks

人行道基准测试：城市人行道视觉导航的基准评估

Liu, Zhizheng, He, Honglin, Alumootil, Vivek, Pandya, Akshat, Squicciarini, Brad, Wu, Wayne, Zhou, Bolei

Abstract

Urban sidewalk navigation presents significant challenges due to complex structural layouts, dynamic pedestrian behaviors, and long distances. While recent visual navigation models offer a promising solution, the lack of a unified benchmark hinders quantitative and reproducible evaluation. To bridge this gap, we propose SidewalkBench, a comprehensive benchmark designed for visual navigation on urban sidewalks. Built upon NVIDIA Isaac Sim, SidewalkBench brings GPU-accelerated simulation of diverse, high-fidelity sidewalk environments, including both procedurally generated and real-world scanned scenes. We further populate the scenes with rich, reactive event-based pedestrian behaviors and flexible, efficient animation, enabling standardized model evaluation under realistic real-world settings. We conduct a comprehensive evaluation of 9 visual navigation models on 330 unit-test scenarios, 800 pedestrian-reactive scenarios, and 105 long-horizon scenarios. Our findings highlight that pedestrian interaction and long-horizon robustness remain critical bottlenecks for existing models, and scaling up sidewalk training with synthetic data emerges as a promising solution.

Chinese Translation

城市人行道导航因复杂的结构布局、动态的行人行为和较长的距离而面临重大挑战。尽管近期的视觉导航模型提供了有前景的解决方案，但缺乏统一的基准限制了定量和可重复的评估。为了解决这一问题，我们提出了人行道基准测试（SidewalkBench），这是一个专为城市人行道视觉导航设计的综合基准。人行道基准测试基于NVIDIA Isaac Sim构建，提供了多样化、高保真的人行道环境的GPU加速仿真，包括程序生成的场景和真实世界扫描的场景。我们进一步为场景添加了丰富的、基于事件的行人行为和灵活、高效的动画，使得在现实世界环境下进行标准化模型评估成为可能。我们对9个视觉导航模型在330个单元测试场景、800个行人反应场景和105个长时间视野场景上进行了全面评估。我们的研究结果强调，行人交互和长时间视野的鲁棒性仍然是现有模型的关键瓶颈，而利用合成数据扩大人行道训练规模则成为一种有前景的解决方案。

View on arXiv Download PDF AI Translation

cs.RO / 94 / 2606.16972

When Should a Robot Replan? Regret-Guided Update Scheduling in Time-Varying MDPs

机器人何时应重新规划？基于遗憾的更新调度在时间变化的马尔可夫决策过程中的应用

Musavi, Negin, Puthumanaillam, Gokul, Hernandez, Ruben, Schafer, William, Ornik, Melkior

Abstract

Robots operating in non-stationary environments must continually adapt their policies as the dynamics drift, but onboard energy and compute budgets cap how often a full state estimation and re-planning step can be performed. This raises a question: \emph{when}, along a horizon, should a robot spend its limited budget? We formulate this problem in time-varying Markov decision processes (TVMDPs) with a known bound on the rate of transition drift. We model execution as a \emph{skip-update} scheme in which, at chosen update times, the agent estimates the transition kernel by maximum likelihood and computes a finite-horizon policy, and between updates reuses this policy under a propagated state estimate. We analyze the dynamic regret of this scheme and show how it grows during skip intervals in terms of the properties of the TVMDP and the skip lengths; the resulting bound answers the opening question via an online, regret-guided update rule that allocates the budget adaptively. We evaluate the rule in a simulated Mars-rover navigation task with time-varying slip dynamics and on a Crazyflie quadrotor in indoor obstacle fields. Adaptive allocation outperforms other budgeted baselines.

Chinese Translation

在非平稳环境中运行的机器人必须不断调整其策略，以应对动态变化，但机载能量和计算预算限制了完整状态估计和重新规划步骤的频率。这引发了一个问题： extit{在何时}，机器人应如何花费其有限的预算？我们在已知转移漂移速率界限的时间变化马尔可夫决策过程（TVMDPs）中对这一问题进行了形式化建模。我们将执行建模为一种 extit{跳过更新}方案，在选定的更新时刻，代理通过最大似然估计转移核，并计算有限时间范围内的策略，而在更新之间则在传播的状态估计下重用该策略。我们分析了该方案的动态遗憾，并展示了在跳过间隔期间其如何根据TVMDP的特性和跳过长度而增长；由此得出的界限通过一种在线的、基于遗憾的更新规则回答了开头的问题，该规则自适应地分配预算。我们在一个具有时间变化滑移动态的模拟火星车导航任务和一个在室内障碍物场中的Crazyflie四旋翼飞行器上评估了该规则。自适应分配的表现优于其他预算基线。

View on arXiv Download PDF AI Translation

cs.RO / 95 / 2606.16978

Task-Error Residual Learning for Real-Robot Five-Ball Juggling

用于真实机器人五球杂耍的任务误差残差学习

Ploeger, Kai, Peters, Jan

Abstract

For residual learning that refines existing behavior, sample efficiency depends on two things: how much information each rollout returns, and how efficiently the learner uses that information. Reinforcement learning's standard scalar reward carries far less information than the directional task error that defines the task. Random exploration further discards whatever information each rollout returns. Through residual learning with directional task-error supervision and a task error model that drives sample selection, we achieve stable three-, four-, and five-ball juggling on anthropomorphic Barrett WAM arms. Despite planning and controlling through a simple, idealized stack, the system converges from the second attempt. The first attempt drops, after which task error decreases monotonically without further failures. In comparison, five-ball juggling typically takes humans years of practice. We compare residual learners across two ternary axes, the directional information in the learning feedback and the commitment of the analytic prior, spanning Newton-style Jacobian updates, Composite Bayesian Optimization, and stochastic search methods. Both axes prove necessary: neither directional feedback nor an informative prior suffices alone, and the simplest method that combines them, a fixed-Jacobian Newton update, is the most reliable. The learned residual tolerates substantial prior misalignment and degraded joint tracking, affecting mainly convergence speed. The bottleneck for residual learning on real robots is therefore the information content of the supervision signal and how the learner uses it, not the accuracy of the surrounding stack. Video documentation of all experiments is available at https://kai-ploeger.com/residual-juggling.

Chinese Translation

在精炼现有行为的残差学习中，样本效率依赖于两个因素：每次回合返回的信息量，以及学习者使用该信息的效率。强化学习的标准标量奖励所携带的信息远不如定义任务的方向性任务误差。随机探索进一步丢弃了每次回合返回的任何信息。通过方向性任务误差监督的残差学习和驱动样本选择的任务误差模型，我们在类人巴雷特WAM臂上实现了稳定的三球、四球和五球杂耍。尽管通过一个简单的理想化堆栈进行规划和控制，该系统在第二次尝试时就收敛了。第一次尝试失败，之后任务误差单调减少且没有进一步的失败。相比之下，五球杂耍通常需要人类数年的练习。我们在两个三元轴上比较了残差学习者，即学习反馈中的方向性信息和解析先验的承诺，涵盖了牛顿式雅可比更新、复合贝叶斯优化和随机搜索方法。两个轴都是必要的：单独的方向性反馈或信息丰富的先验都不足以满足要求，而将它们结合起来的最简单方法，即固定雅可比牛顿更新，是最可靠的。学习到的残差能够容忍相当大的先验不对齐和退化的关节跟踪，主要影响收敛速度。因此，真实机器人上残差学习的瓶颈在于监督信号的信息内容及学习者如何使用它，而不是周围堆栈的准确性。所有实验的视频记录可在 https://kai-ploeger.com/residual-juggling 获取。

View on arXiv Download PDF AI Translation

cs.RO / 96 / 2606.17011

ROVE: Unlocking Human Interventions for Humanoid Manipulation via Reinforcement Learning

ROVE：通过强化学习解锁人类干预以实现类人操控

Xiao, Wei, Tang, Weiliang, Ge, Yuying, Zhou, Hui, Mu, Yao, Zhang, Li, Ge, Yixiao

Abstract

Human interventions provide crucial corrective signals for post-training Vision-Language-Action (VLA) models. However, enabling seamless humanoid interventions is a formidable systems challenge due to complex whole-body kinematics and dexterous-hand control. Consequently, the collected intervention trajectories are often suboptimal, and methods that rely on human interventions as expert supervision can absorb hesitant, inefficient, or even erroneous behaviors. To address both the system and algorithmic challenges, we propose ROVE, a reinforcement learning framework for humanoid VLA post-training with imperfect human interventions. First, ROVE introduces a human-in-the-loop pipeline capable of collecting deployment and intervention data for humanoid manipulation. Second, it utilizes Optimistic Value Estimation (OVE) to prioritize high-value behaviors from mixed-quality trajectories. To further robustify value estimation, we incorporate cross-embodiment human experience videos to provide rich supervision for long-tailed failure and recovery modes. The resulting critic yields informative advantage signals, steering the VLA actor to focus on high-value behaviors rather than indiscriminately imitating all actions. On challenging real-world contact-rich and fine-grained humanoid manipulation tasks, ROVE outperforms experience-learning baselines and consistently improves across multiple rollout-intervention iterations.

Chinese Translation

人类干预为训练后的视觉-语言-动作（VLA）模型提供了至关重要的纠正信号。然而，由于复杂的全身运动学和灵巧手控制，实现无缝的类人干预是一项艰巨的系统挑战。因此，收集到的干预轨迹往往是次优的，依赖人类干预作为专家监督的方法可能会吸收犹豫、不高效甚至错误的行为。为了解决系统和算法上的挑战，我们提出了ROVE，一个用于类人VLA训练后处理的不完美人类干预的强化学习框架。首先，ROVE引入了一个人机协作的流程，能够收集类人操控的部署和干预数据。其次，它利用乐观价值估计（Optimistic Value Estimation, OVE）来优先考虑来自混合质量轨迹的高价值行为。为了进一步增强价值估计的稳健性，我们结合跨体现人类经验视频，为长尾失败和恢复模式提供丰富的监督。最终生成的评论者产生了有意义的优势信号，引导VLA演员专注于高价值行为，而不是无差别地模仿所有动作。在具有挑战性的现实世界接触丰富和细粒度的类人操控任务中，ROVE的表现优于经验学习基线，并在多个回合-干预迭代中持续改进。

View on arXiv Download PDF AI Translation

cs.RO / 97 / 2606.17040

R2RDreamer: 3D-aware Data Augmentation for Spatially-generalized 2D Manipulation Policies

R2RDreamer：用于空间广泛化的3D感知数据增强方法

Xu, Xiuwei, Sun, Haowen, Ma, Angyuan, Zhang, Yiwei, Wu, Zhenyu, Wang, Xiaofeng, Yu, Bingyao, Zhu, Zheng, Zhou, Jie, Lu, Jiwen

Abstract

Spatial generalization is critical for imitation-learned manipulation policies, but achieving it typically requires scaling demonstrations across diverse object poses, robot configurations, and camera viewpoints. Data augmentation from a few source demonstrations offers a practical alternative to costly real-world collection. Simulation-based augmentation can create controllable variation, but requires complex environment and object setup and may introduce a sim-to-real gap. Recent real-to-real methods avoid these issues by jointly editing 3D observations and action trajectories from real demonstrations, yet they still rely on strong 3D scene parsing and geometry completion, and often produce observations tailored to 3D pointcloud policies rather than RGB-based 2D policies. We propose R2RDreamer, a real-to-real demonstration augmentation framework that preserves the geometric consistency of 3D action-observation editing while moving visual completion to 2D video space. Specifically, R2RDreamer first performs lightweight 3D augmentation by editing incomplete object pointclouds and end-effector trajectories in a shared 3D frame; it then projects the edited scene into masked image-space control videos with occlusion-aware reasoning and uses a dense-control image-to-video model to complete temporally coherent RGB observations. Experiments on spatially shifted manipulation tasks with both 2D diffusion-style policies and vision-language-action policies show that R2RDreamer improves spatial generalization from limited source demonstrations, with analyses validating the contributions of 3D editing, occlusion-aware projection, and video completion.

Chinese Translation

空间广泛化对于模仿学习的操作策略至关重要，但实现这一目标通常需要在多样的物体姿态、机器人配置和摄像机视角下扩展演示。通过少量源演示进行数据增强提供了一个实用的替代方案，避免了昂贵的现实世界数据收集。基于仿真的增强可以创造可控的变化，但需要复杂的环境和物体设置，并可能引入模拟与现实之间的差距。最近的真实到真实方法通过共同编辑来自真实演示的3D观察和动作轨迹来避免这些问题，但它们仍然依赖于强大的3D场景解析和几何补全，且通常生成的观察结果更适合3D点云策略，而非基于RGB的2D策略。我们提出了R2RDreamer，一个真实到真实的演示增强框架，保持3D动作-观察编辑的几何一致性，同时将视觉补全转移到2D视频空间。具体而言，R2RDreamer首先通过在共享的3D框架中编辑不完整的物体点云和末端执行器轨迹进行轻量级的3D增强；然后，它将编辑后的场景投影到具有遮挡感知推理的掩膜图像控制视频中，并使用密集控制的图像到视频模型来完成时间上连贯的RGB观察。在空间转移的操作任务上，使用2D扩散风格策略和视觉-语言-动作策略的实验表明，R2RDreamer能够在有限的源演示中提高空间广泛化，分析验证了3D编辑、遮挡感知投影和视频补全的贡献。

View on arXiv Download PDF AI Translation

cs.RO / 98 / 2606.17043

Hierarchical Advantage Weighting for Online RL Fine-Tuning of VLAs from Sparse Episode Outcomes

基于层次优势加权的在线强化学习微调稀疏情节结果的变换学习代理

Fang, Tongyan, Huang, Siyuan, Fang, Naiyu, Zhao, Ganlong, Luo, Zhongjin, Liu, Jianbo, Wang, Xiaogang, Dong, Ying, Li, Hongsheng

Abstract

When pretrained VLA policies are fine-tuned through online RL, each rollout episode produces only a single binary outcome (success or failure), yet the actor update requires per-transition supervision. Existing approaches commonly reduce this sparse outcome to a single scalar reward or advantage signal, which conflates distinct forms of transition-level feedback and provides limited guidance once basic task success becomes achievable. First, a single scalar signal conflates the two objectives of viability and efficiency; once basic success is achieved, the binary label provides no gradient to distinguish efficient completions from slow ones. Second, real-world rollouts mix autonomous and intervention segments; naively assigning episode outcomes across these boundaries introduces incorrect credit assignment. To address these issues, we propose Hierarchical Advantage-Weighted Behavior Cloning (HABC), which trains separate critic heads for these two objectives on different data subsets and combines their outputs with a state-adaptive balance. A state-adaptive gate $g_t$ merges their one-step advantages, prioritizing viability when success is uncertain and shifting to efficiency only when viability is high, and converts the result into per-transition weights on the actor loss. Intervention-aware credit assignment further restricts outcome labels to segments executed by the current policy, preventing supervision from leaking across intervention boundaries. In real-robot experiments on three contact-rich bimanual tasks, HABC raises success from supervised fine-tuning (SFT) baselines of 36%, 44%, and 12% to 92%, 88%, and 38%.

Chinese Translation

当预训练的变换学习代理（VLA）策略通过在线强化学习进行微调时，每个回合情节仅产生一个二元结果（成功或失败），而演员更新则需要逐步的监督。现有的方法通常将这种稀疏结果简化为一个标量奖励或优势信号，这混淆了不同形式的过渡级反馈，并在基本任务成功可达后提供有限的指导。首先，单一的标量信号混淆了可行性和效率这两个目标；一旦实现基本成功，二元标签无法提供梯度来区分高效完成和缓慢完成。其次，现实世界中的回合混合了自主和干预段；在这些边界之间简单地分配情节结果会引入错误的信用分配。为了解决这些问题，我们提出了层次优势加权行为克隆（HABC），该方法在不同的数据子集上为这两个目标训练独立的评论家头，并通过状态自适应平衡组合它们的输出。状态自适应门 $g_t$ 合并它们的一步优势，在成功不确定时优先考虑可行性，仅在可行性高时转向效率，并将结果转换为演员损失的逐步权重。干预意识的信用分配进一步限制了结果标签仅适用于当前策略执行的段，防止监督在干预边界之间泄漏。在三个接触丰富的双手任务的真实机器人实验中，HABC 将监督微调（SFT）基线的成功率从36%、44%和12%提升至92%、88%和38%。

View on arXiv Download PDF AI Translation

cs.RO / 99 / 2606.17046

Geometric Action Model for Robot Policy Learning

用于机器人策略学习的几何动作模型

Han, Jisang, Jeon, Seonghu, Jung, Jaewoo, Zurbrügg, René, An, Honggyu, Portela, Tifanny, Hutter, Marco, Pollefeys, Marc, Kim, Seungryong, Hong, Sunghwan

Abstract

Generalist robot policies must follow user instructions while reasoning about how objects, cameras, and robot actions interact in the 3D physical world. Recent vision-language-action models (VLAs) and video world-action models (WAMs) inherit strong semantic or temporal priors from large-scale foundation models, but they still operate primarily on 2D image frames or 2D-derived latent spaces, leaving implicit the 3D geometry required for contact-rich manipulation. We propose the Geometric Action Model (GAM), a language-conditioned manipulation policy that directly repurposes a pretrained geometric foundation model (GFM) as a shared substrate for perception, temporal prediction, and action decoding. GAM splits the GFM at an intermediate layer: the shallow layers serve as an observation encoder, and a causal future predictor inserted at the split layer forecasts future latent tokens conditioned on language, proprioception, and action history. The predicted future tokens are then routed through the remaining GFM blocks for feature propagation and decoding, allowing a single backbone to produce both future geometry and actions. This design equips the GFM with language-conditioned temporal world modeling through minimal architectural modification while preserving its rich geometric priors. Across a broad suite of simulation and real-robot manipulation benchmarks, GAM is more accurate, more robust, faster, and lighter than current foundation-model-scale baselines.

Chinese Translation

通用机器人策略必须遵循用户指令，同时推理物体、相机和机器人动作在三维物理世界中的相互作用。近期的视觉-语言-动作模型（VLAs）和视频世界-动作模型（WAMs）从大规模基础模型中继承了强大的语义或时间先验，但它们仍主要在二维图像帧或二维衍生的潜在空间中操作，隐含了进行接触丰富操作所需的三维几何结构。我们提出了几何动作模型（GAM），这是一种语言条件的操作策略，直接重新利用预训练的几何基础模型（GFM）作为感知、时间预测和动作解码的共享基础。GAM在中间层对GFM进行分割：浅层作为观察编码器，而插入在分割层的因果未来预测器则基于语言、本体感知和动作历史预测未来潜在标记。然后，预测的未来标记通过剩余的GFM模块进行特征传播和解码，使得单一主干能够生成未来几何和动作。该设计通过最小的架构修改为GFM提供了语言条件的时间世界建模，同时保留了其丰富的几何先验。在一系列广泛的仿真和真实机器人操作基准测试中，GAM的准确性、鲁棒性、速度和轻量性均优于当前基础模型规模的基线。

View on arXiv Download PDF AI Translation

cs.RO / 100 / 2606.17054

Human Universal Grasping

人类通用抓取

Wu, Kevin Yuanbo, Zhou, Tianxing, Tu, Isaac, Yan, Billy, Guzey, Irmak, Fouhey, David, Shan, Dandan, Pinto, Lerrel

Abstract

Humans can grasp objects effortlessly, whereas multi-fingered robots are far from this level of generality. We argue that the most natural source of robot grasping data is from humans, who pick up thousands of objects every day. We present HUG, a flow-matching model that generates diverse human grasps for any user-specified object in a single RGB-D image captured from a stereo camera. Using smart glasses, we first collect 1M-HUGs, an egocentric dataset of human grasps spanning 1M frames (27.8 hrs) and 6,707 object instances across 41 buildings. Next, to model the distribution of natural human grasps, our novel flow-matching model fuses RGB and depth observations to output a grasp parameterized by wrist translation, wrist rotation, and MANO hand pose. Predicted grasps can be retargeted to various robot hands, enabling zero-shot grasping in everyday scenes. To standardize evaluation, we build a new simulated benchmark, HUG-Bench, of 90 unseen objects from five geometric categories and various sizes, with metric-scale 3D meshes. We evaluate HUG in the real world on the 30-object test set of HUG-Bench across multiple stereo cameras, robot embodiments, and household environments. HUG outperforms the state-of-the-art grasping baselines by +23% and +34% on our challenging object set. Code, data, benchmark, checkpoints, and an interactive demo are released on our website: https://grasping.io/

Chinese Translation

人类能够轻松地抓取物体，而多指机器人在这一通用性方面远远不够。我们认为，机器人抓取数据最自然的来源是人类，因为他们每天都会拾取成千上万的物体。我们提出了HUG，一个流匹配模型，能够为任何用户指定的物体生成多样化的人类抓取，基于从立体相机捕获的单张RGB-D图像。通过智能眼镜，我们首先收集了1M-HUGs，一个以自我为中心的人类抓取数据集，涵盖了1M帧（27.8小时）和41栋建筑中的6,707个物体实例。接下来，为了建模自然人类抓取的分布，我们的新型流匹配模型融合了RGB和深度观测，输出一个由手腕平移、手腕旋转和MANO手姿态参数化的抓取。预测的抓取可以重新定向到各种机器人手，从而实现日常场景中的零样本抓取。为了标准化评估，我们构建了一个新的模拟基准HUG-Bench，包含来自五个几何类别和各种尺寸的90个未见物体，配备有度量尺度的3D网格。我们在现实世界中对HUG进行了评估，使用HUG-Bench的30个物体测试集，跨多个立体相机、机器人形态和家庭环境。HUG在我们具有挑战性的物体集上比最先进的抓取基线提高了23%和34%。代码、数据、基准、检查点和互动演示已在我们的网站发布：https://grasping.io/

View on arXiv Download PDF AI Translation

cs.RO / 101 / 2606.17055

T-Rex: Tactile-Reactive Dexterous Manipulation

T-Rex：触觉反应灵巧操作

Niu, Dantong, Liu, Zhuoyang, Wang, Zekai, Shao, Boning, Yin, Zhao-Heng, Pai, Anirudh, Sharma, Yuvan, Saravalle, Stefano, Zheng, Ruijie, Wang, Jing, Punamiya, Ryan, Xu, Mengda, Xie, Yuqi, Jiang, Yunfan, Fu, Letian, Kallidromitis, Konstantinos, Gioia, Matteo, Zhang, Junyi, Ge, Jiaxin, Feng, Haiwen, Galasso, Fabio, Zhan, Wei, Chan, David M., Bai, Yutong, Herzig, Roei, Lei, Jiahui, Li, Fei-Fei, Goldberg, Ken, Malik, Jitendra, Abbeel, Pieter, Zhu, Yuke, Xu, Danfei, Jim, Fan, Darrell, Trevor

Abstract

The ability to react dynamically to tactile signals has long been considered crucial to agile human-level dexterity. Yet contemporary learning-based Vision-Language-Action (VLA) models for robotic manipulation generally either overlook the tactile modality or are limited to encoders with static cues, due in part to the scarcity of diverse training data and standardized evaluation, architectural constraints in current VLA models, and limitations of static tactile encoders. In this paper, we push the frontier of tactile-reactive manipulation by addressing all of these limitations. We propose a large-scale, 100-hour tactile-rich dataset collected via a novel, data-efficient recipe that prioritizes elementary motor primitives. To effectively exploit naturally high-frequency touch signals without sacrificing the existing capabilities of existing VLAs, we introduce a variable-rate Mixture-of-Transformers (MoT) architecture equipped with a novel temporal tactile VQ-VAE encoder. We demonstrate the effectiveness of tactile-reactive policies on 12 manipulation tasks requiring delicate force control and deformable object manipulation, achieving over 30% higher average success rate than the strongest baseline.

Chinese Translation

动态响应触觉信号的能力长期以来被认为是实现灵活人类水平灵巧性的关键。然而，当前基于学习的视觉-语言-动作（VLA）模型在机器人操作中通常要么忽视触觉模态，要么仅限于具有静态线索的编码器，这在一定程度上是由于多样化训练数据和标准化评估的稀缺、当前VLA模型的架构限制以及静态触觉编码器的局限性。在本文中，我们通过解决这些限制推动了触觉反应操作的前沿。我们提出了一个大规模的、收集了100小时触觉丰富数据集的新颖数据高效配方，优先考虑基本运动原语。为了有效利用自然高频触觉信号而不牺牲现有VLA的能力，我们引入了一种可变速率的混合变换器（Mixture-of-Transformers, MoT）架构，并配备了一种新型的时间触觉VQ-VAE编码器。我们在12个需要精细力控制和可变形物体操作的操作任务上展示了触觉反应策略的有效性，成功率比最强基线高出30%以上。

View on arXiv Download PDF AI Translation

计算机视觉 (Computer Vision)

254

cs.CV / 1 / 2606.14716

RAMS: Resource-Adaptive and Detection-Conditioned Model Switching for Embedded Edge Perception

RAMS：面向嵌入式边缘感知的资源自适应与检测条件模型切换

Khemani, Kushal, Leri, Evan, Xu, George, Hod, Amit

Abstract

Edge object detection on embedded hardware requires balancing inference latency and detection quality under changing resource pressure. We present RAMS, a lightweight runtime controller that monitors device pressure, calibrates switching thresholds from idle behavior, and dynamically selects among three resident YOLOv8 tiers (NANO/SMALL/MEDIUM at 320/416/640 px) without model-reload latency. RAMS defines five switching policies, including two detection-conditioned variants that prevent aggressive downgrades after recent vulnerable-road-user (VRU) detections. We further introduce the VRU-Weighted Accuracy Score (SWAS), a scalar metric for offline policy comparison without ground-truth annotations, together with an oracle-bounded variant that separates detector circularity from genuine tier-retention benefit. Across Raspberry Pi 5, x86 laptops, and Jetson Orin ONNX/TensorRT deployments, the same controller equations operate over a 37x latency range. On Jetson Orin TensorRT under heavy load, the safety2 policy achieves 3.41 ms mean latency, 5.6x faster than fixed-MEDIUM inference, while retaining 74% of its proxy accuracy through near-NANO operation with selective SMALL and MEDIUM locks during VRU-positive windows. Detection-conditioned switching improves SWAS by 25.4% under oracle scoring and 47.3% under detector-derived scoring relative to threshold-only policies under heavy load. Live KITTI evaluation reports per-tier VRU recall of 24.2%, 41.2%, and 59.0%, showing that reactive overrides are fundamentally limited by baseline detector recall.

Chinese Translation

在嵌入式硬件上进行边缘物体检测需要在变化的资源压力下平衡推理延迟和检测质量。我们提出了RAMS，一种轻量级运行时控制器，它监控设备压力，从空闲行为中校准切换阈值，并在不产生模型重载延迟的情况下动态选择三种驻留的YOLOv8层次（NANO/SMALL/MEDIUM，分辨率分别为320/416/640 px）。RAMS定义了五种切换策略，包括两种检测条件变体，防止在最近的脆弱道路使用者（VRU）检测后发生激进降级。我们进一步引入了VRU加权准确性评分（SWAS），这是一种用于离线策略比较的标量指标，无需真实标注，同时提供了一种有Oracle界限的变体，将检测器的循环性与真实的层次保留效益分开。在Raspberry Pi 5、x86笔记本电脑和Jetson Orin ONNX/TensorRT部署中，相同的控制器方程在37倍的延迟范围内运作。在重负载下的Jetson Orin TensorRT上，safety2策略实现了3.41毫秒的平均延迟，比固定MEDIUM推理快5.6倍，同时在VRU正窗口期间通过选择性的小型和中型锁保持了74%的代理准确性。检测条件切换在重负载下相对于仅阈值策略，Oracle评分下提高了25.4%的SWAS，检测器派生评分下提高了47.3%。实时KITTI评估报告每个层次的VRU召回率分别为24.2%、41.2%和59.0%，显示反应性覆盖在基线检测器召回率的限制下是根本性的。

View on arXiv Download PDF AI Translation

cs.CV / 2 / 2606.14720

AI for Maritime Security: Comparative Evaluation of CNN and Vision Transformer Architectures for Maritime Object Detection

海事安全中的人工智能：卷积神经网络与视觉变换器架构在海洋目标检测中的比较评估

Gocer, Ismet, Bhuiayn, Zakirul, Ahmad, Shakeel, Hasan, Raza

Abstract

This study aims to enhance maritime security by using advanced Artificial Intelligence (AI) and Computer Vision (CV) techniques. For this purpose, it was designed and assessed intelligent object detection systems that can detect the presence of ships on the sea surface under different real-time environments. To achieve this goal, a maritime image dataset with 6,468 images was used, covering different weather conditions like cloudy, foggy, rainy, and sunny environments. Six deep learning architectures were evaluated, including a base Convolutional Neural Network (CNN) model, four transfer learning models (Xception, VGG16, MobileNetV2, and EfficientNetV2L), and a Vision Transformer (ViT) model. The models were compared using multiple performance indicators, including accuracy, Type I and Type II errors, model size, and video processing time. The results show that model performance varies depending on computational constraints and deployment conditions. While lightweight architectures are suitable for resource-limited devices, the ViT achieved the best overall performance, reaching 100% accuracy with the lowest error rates and the fastest video processing time. The findings highlight the potential of AI-driven computer vision systems for maritime surveillance, border protection, and autonomous navigation.

Chinese Translation

本研究旨在通过使用先进的人工智能（AI）和计算机视觉（CV）技术来增强海事安全。为此，设计并评估了智能目标检测系统，这些系统能够在不同的实时环境下检测海面上船只的存在。为实现这一目标，使用了一个包含6,468幅图像的海事图像数据集，涵盖了多种天气条件，如多云、雾霾、雨天和晴天。评估了六种深度学习架构，包括基础卷积神经网络（CNN）模型、四个迁移学习模型（Xception、VGG16、MobileNetV2和EfficientNetV2L）以及一个视觉变换器（ViT）模型。通过多个性能指标对模型进行了比较，包括准确率、I型和II型错误、模型大小和视频处理时间。结果表明，模型性能因计算限制和部署条件而异。虽然轻量级架构适合资源有限的设备，但ViT在整体性能上表现最佳，达到了100%的准确率，最低的错误率和最快的视频处理时间。研究结果突显了基于AI的计算机视觉系统在海事监控、边境保护和自主导航中的潜力。

View on arXiv Download PDF AI Translation

cs.CV / 3 / 2606.14723

Disagreement-Based Cross-Model Routing for Implicit Video Question Answering

基于分歧的跨模型路由用于隐式视频问答

Saluru, Durga Sandeep

Abstract

We study multiple-choice video question answering on the ImplicitQA benchmark, where the correct answer is never explicitly shown but must be inferred from off-screen events, line-of-sight cues, causal structure, and cross-shot spatial layout. On this benchmark a single frontier video LLM already operates near its accuracy ceiling, and we observe that conventional self-consistency strategies -- majority voting across repeated samples of the same model -- can hurt rather than help, because the model's errors on hard questions are correlated. We propose disagreement-based cross-model routing, a pure inference-time procedure that requires no labels and no training. We triple-sample a native-video model (Gemini 3.1 Pro Preview) at temperature zero, exploit the genuine sample-to-sample variance of its video-processing pipeline to identify the roughly 20% subset of questions where the three samples disagree, and route only that subset to a second model from a different family (Claude Opus 4.8) that consumes uniformly sampled frames with adaptive thinking. On the 1001-question validation set with public ground truth -- our main evaluation -- the method improves AvgAcc by +1.43 over the best single sample of the primary model, with per-category gains concentrated on Motion & Trajectory (+5.49), Inferred Counting (+3.45), and Vertical Spatial Reasoning (+1.82) -- the categories most dependent on cross-shot reference resolution. The same pipeline applied to the held-out 172-question CVPR 2026 ImplicitQA challenge test set achieves 82.03 AvgAcc / 79.71 MacroAvgAcc (+1.81 over the best single sample of the primary model), confirming the validation result on an independent split.

Chinese Translation

我们研究了在隐式问答基准（ImplicitQA）上的多项选择视频问答，其中正确答案从未明确显示，而必须通过屏幕外事件、视线线索、因果结构和跨镜头空间布局进行推断。在该基准上，单一的前沿视频大语言模型（LLM）已经接近其准确率上限，我们观察到传统的自一致性策略——对同一模型的重复样本进行多数投票——可能会适得其反，因为模型在困难问题上的错误是相关的。我们提出了一种基于分歧的跨模型路由，这是一种纯推理时的过程，不需要标签和训练。我们在温度为零的情况下对原生视频模型（Gemini 3.1 Pro Preview）进行了三次采样，利用其视频处理管道的真实样本间变异性来识别大约20%的问题子集，其中三个样本存在分歧，并仅将该子集路由到来自不同家族的第二个模型（Claude Opus 4.8），该模型以自适应思维处理均匀采样的帧。在包含公共真值的1001个问题验证集上——我们的主要评估——该方法使平均准确率（AvgAcc）比主模型的最佳单一样本提高了1.43，其中每个类别的增益集中在运动与轨迹（+5.49）、推断计数（+3.45）和垂直空间推理（+1.82）——这些类别最依赖于跨镜头参考解析。同样的管道应用于保留的172个问题的CVPR 2026隐式问答挑战测试集，达到了82.03的平均准确率/79.71的宏平均准确率（比主模型的最佳单一样本提高了1.81），确认了在独立划分上的验证结果。

View on arXiv Download PDF AI Translation

cs.CV / 4 / 2606.14724

VigilFormer: Deformable Attention for Video Anomaly Detection with Causal Risk Inference

VigilFormer：基于可变形注意力的监控视频异常检测与因果风险推断

Zhang, Xinze

Abstract

Video anomaly detection in surveillance settings must balance detection accuracy against real-time throughput, a tension that existing methods address either through stronger feature extractors or more efficient architectures, but rarely both. We present VigilFormer, a unified framework that combines deformable spatio-temporal attention with causal temporal modeling to detect anomalies in untrimmed surveillance video. The proposed Deformable Spatio-Temporal Encoder (DSTE) attends to a sparse set of informative locations across frames, avoiding the quadratic cost of dense attention while retaining the ability to capture irregular motion patterns. A Causal Anomaly Classifier (CAC) applies dilated causal convolutions over snippet-level features and optimizes a contrastive multiple-instance learning objective that separates anomalous and normal representations without frame-level labels. To meet deployment constraints, an Adaptive Confidence Scheduler (ACS) dynamically skips low-information frames at inference time, reducing redundant computation in static scenes. Evaluated on UCF-Crime, ShanghaiTech, and CUHK Avenue, VigilFormer achieves AUC scores of 87.83%, 97.21%, and 89.74% respectively, at 41.5 FPS on a single GPU, outperforming recent weakly-supervised methods in both accuracy and speed.

Chinese Translation

监控环境中的视频异常检测必须在检测准确性与实时处理能力之间取得平衡，现有方法通常通过更强的特征提取器或更高效的架构来解决这一矛盾，但很少同时兼顾两者。我们提出了VigilFormer，一个统一框架，结合了可变形时空注意力与因果时间建模，以检测未裁剪监控视频中的异常。所提出的可变形时空编码器（DSTE）关注于跨帧的一组稀疏信息位置，避免了密集注意力的二次计算成本，同时保留了捕捉不规则运动模式的能力。因果异常分类器（CAC）在片段级特征上应用扩张因果卷积，并优化对比多实例学习目标，以在没有帧级标签的情况下区分异常与正常表示。为了满足部署约束，自适应置信度调度器（ACS）在推理时动态跳过低信息帧，从而减少静态场景中的冗余计算。在UCF-Crime、ShanghaiTech和CUHK Avenue数据集上的评估结果显示，VigilFormer在单个GPU上以41.5 FPS的速度分别达到了87.83%、97.21%和89.74%的AUC分数，超越了近期的弱监督方法在准确性和速度上的表现。

View on arXiv Download PDF AI Translation

cs.CV / 5 / 2606.14725

Interpolation between Convolution and Attention via K-Nearest Neighbors

通过K近邻在卷积与注意力之间的插值

Kang, Mingi

Abstract

The shift from Convolutional Neural Networks to Transformers has reshaped computer vision, yet these two architectural families are typically viewed as fundamentally distinct. Convolutional Neural Networks are defined by spatially local convolution operations, while Transformers rely on global self-attention. We argue that convolution and self-attention, despite their apparent differences, can be unified within a single k-nearest neighbor aggregation framework. The critical insight is that both operations are special cases of neighbor selection and weighted aggregation. Convolution selects neighbors by spatial proximity while self-attention selects by feature similarity, revealing that they lie on a continuous spectrum rather than representing categorically different computations. We introduce Convolutional Nearest Neighbors (ConvNN), a unified framework that formalizes this connection. ConvNN exactly recovers standard and depthwise convolution by restricting neighbor selection to normalized spatial coordinates, and exactly recovers self-attention and its sparse variants, including KVT-attention, by replacing spatial proximity with scaled dot-product similarity. Beyond these special cases, ConvNN serves as a drop-in replacement for both convolution and attention layers, enabling systematic exploration of the intermediate spectrum between local and global aggregation through configurable similarity functions, neighbor selection strategies, positional encodings, and aggregation kernels.

Chinese Translation

从卷积神经网络到变换器的转变重塑了计算机视觉，然而这两种架构通常被视为根本不同。卷积神经网络通过空间局部卷积操作定义，而变换器依赖于全局自注意力。我们认为，尽管卷积和自注意力表面上存在差异，但可以在一个统一的k近邻聚合框架内进行统一。关键的见解是，这两种操作都是邻居选择和加权聚合的特例。卷积通过空间邻近性选择邻居，而自注意力则通过特征相似性选择邻居，这表明它们位于一个连续的光谱上，而不是代表分类上不同的计算。我们引入了卷积近邻（Convolutional Nearest Neighbors, ConvNN），这是一个正式化这种联系的统一框架。ConvNN通过将邻居选择限制在标准化空间坐标上，准确恢复标准卷积和深度卷积，并通过用缩放点积相似性替代空间邻近性，准确恢复自注意力及其稀疏变体，包括KVT-attention。除了这些特例，ConvNN还可以作为卷积和注意力层的直接替代，能够通过可配置的相似性函数、邻居选择策略、位置编码和聚合核系统地探索局部与全局聚合之间的中间光谱。

View on arXiv Download PDF AI Translation

cs.CV / 6 / 2606.14727

FairGen: Preference-Aligned Diffusion for Demographically Equitable Medical Image Synthesis

FairGen：面向人口统计公平的偏好对齐扩散医学图像合成

Li, Zhimin, Zhang, Ruichen, Tan, Zhen, Aizenstein, Howard J, Hu, Jingtong, Chen, Tianlong

Abstract

Medical imaging is central to modern diagnostics, and artificial intelligence (AI) systems are increasingly used to support image-based analysis by improving efficiency, accuracy, and access to care. However, inequities in healthcare access and differential disease prevalence create severe demographic imbalances in clinical image data. Such imbalances are compounded by the fact that diseases can manifest with distinct features across demographic groups, rendering certain phenotypic presentations naturally rare. AI models trained on such imbalanced data risk perpetuating diagnostic bias and widening healthcare disparities. Here we introduce FairGen, a fairness-aware diffusion framework that synthesizes demographically balanced medical images while preserving pathology-relevant visual features. By embedding physician-aligned preferences into the generation process, FairGen improves subgroup coverage during synthesis and downstream classification. Applied to dermatology, radiology, and neuroimaging benchmark tasks, FairGen achieves fairness improvements of 95.9% for skin images, 80.0% for chest radiography, and 35.2% for brain MRI, while maintaining competitive diagnostic accuracy relative to models trained on original clinical data. Clinician-facing expert review and external validation on independent cohorts further support that these gains extend beyond standard fidelity metrics and are not confined to the original in-distribution datasets.

Chinese Translation

医学成像是现代诊断的核心，人工智能（AI）系统越来越多地被用于支持基于图像的分析，提高效率、准确性和医疗服务的可及性。然而，医疗服务获取的不平等和不同疾病的流行率差异在临床图像数据中造成了严重的人口统计失衡。这种失衡因疾病在不同人口群体中可能表现出不同特征而加剧，从而使某些表型表现自然稀有。基于这种不平衡数据训练的AI模型有可能延续诊断偏见并加剧医疗差距。在此，我们介绍了FairGen，一种关注公平性的扩散框架，它在合成医学图像时保持人口统计平衡，同时保留与病理相关的视觉特征。通过将医生对齐的偏好嵌入生成过程，FairGen在合成和后续分类中改善了子群体的覆盖率。在皮肤科、放射学和神经成像基准任务中，FairGen在皮肤图像、胸部X光和脑部MRI方面分别实现了95.9%、80.0%和35.2%的公平性提升，同时保持了相对于基于原始临床数据训练的模型的竞争性诊断准确性。临床专家评审和在独立队列上的外部验证进一步支持这些提升超越了标准的保真度指标，并且并不局限于原始的分布内数据集。

View on arXiv Download PDF AI Translation

cs.CV / 7 / 2606.14728

FUSE: Quantifying Uncertainty in Vision-Language Models by Bayesian Fusing Epistemic and Aleatoric Uncertainty

FUSE：通过贝叶斯融合表征性和偶然性不确定性来量化视觉-语言模型中的不确定性

Zhang, Harry, Carlone, Luca

Abstract

Vision-language models (VLMs) are playing an increasingly important role across multiple domains. In many applications, such as robotics, it is crucial to quantify the uncertainty in the output of these models. } We develop FUSE, a probabilistic framework for capturing two complementary sources of uncertainty in vision-language modeling: (i) aleatoric embedding-level uncertainty derived from input data vision-language ambiguity, and (ii) epistemic model-level uncertainty estimated from the semantic response diversity of VLMs. Our approach formulates a Bayesian fusion mechanism that analytically combines these uncertainty sources to produce a scalar measure of uncertainty. This measure can be used to reliably predict the model's output correctness for downstream applications. We demonstrate that our method outperforms baselines and achieves SOTA uncertainty calibration.

Chinese Translation

视觉-语言模型（VLMs）在多个领域中扮演着越来越重要的角色。在许多应用中，例如机器人技术，量化这些模型输出的不确定性至关重要。我们开发了FUSE，一个概率框架，用于捕捉视觉-语言建模中两种互补的不确定性来源：（i）源自输入数据视觉-语言歧义的偶然性嵌入级不确定性，以及（ii）从VLM的语义响应多样性中估计的表征性模型级不确定性。我们的方法制定了一种贝叶斯融合机制，分析性地结合这些不确定性来源，以产生一个标量的不确定性度量。该度量可用于可靠地预测模型在下游应用中的输出正确性。我们证明了我们的方法优于基线，并实现了最先进的不确定性校准。

View on arXiv Download PDF AI Translation

cs.CV / 8 / 2606.14730

Hierarchical GRU with Input-Conditioned Slot Queries for Ball Action Anticipation

基于输入条件槽查询的层次化GRU用于球类动作预测

Rawat, Parthsarthi

Abstract

We present a hierarchical model for ball action anticipation in football broadcast video. Given a 30-second observation window, the system predicts actions occurring in the subsequent 5-second window across 10 classes. A shared local Transformer encodes clip-level features within each 5-second sub-window; a GRU then aggregates temporal context across all sub-windows; finally, a Transformer decoder with K input-conditioned event slots decodes the anticipation target via three decoupled heads (objectness, class, temporal offset). We introduce frequency-reweighted Hungarian matching that systematically favours rare action classes, and Gaussian soft targets for temporal bin supervision. On the SoccerNet Ball Action Anticipation benchmark, our method achieves 17.91% mAP on the test server.

Chinese Translation

我们提出了一种用于足球直播视频中球类动作预测的层次化模型。在30秒的观察窗口内，该系统预测在随后的5秒窗口中发生的10类动作。一个共享的局部Transformer在每个5秒子窗口内编码剪辑级特征；然后，GRU聚合所有子窗口的时间上下文；最后，一个带有K个输入条件事件槽的Transformer解码器通过三个解耦的头（物体性、类别、时间偏移）解码预测目标。我们引入了频率重加权的匈牙利匹配，系统性地倾向于稀有动作类别，以及用于时间区间监督的高斯软目标。在SoccerNet球类动作预测基准上，我们的方法在测试服务器上达到了17.91%的平均精度（mAP）。

View on arXiv Download PDF AI Translation

cs.CV / 9 / 2606.14731

BBR-Net: Boundary-Balanced Replay for Continual Medical Image Segmentation

BBR-Net：边界平衡重放用于持续医学图像分割

Ullah, Zahid, Choi, Sieun, Kim, Jihie

Abstract

Continual learning for medical image segmentation remains challenging under domain shift because replay-based methods often preserve appearance information without explicitly modeling anatomical structure. This study investigates whether structural consistency governs knowledge retention in continual cardiac ultrasound segmentation. We propose the Boundary-Balanced Replay Network (BBR-Net), which selects replay samples using boundary-aware priority and class balance to preserve anatomically informative regions. The method is evaluated on CAMUS and CardiacNet under forward (CAMUS to CardiacNet) and reverse (CardiacNet to CAMUS) task orders. In the forward setting, BBR-Net retains source-task performance close to an offline joint-training reference, while markedly reducing catastrophic forgetting and preserving competitive target-task adaptation. Ablation results show that boundary-aware prioritization contributes to retention and improves the balance between source-task preservation and target-task adaptation when combined with class-aware sampling. In contrast, the reverse setting reveals that structure-aware replay fails when initial representations are learned from noisy and structurally inconsistent data. To isolate this effect, we conduct a controlled structural perturbation analysis by progressively corrupting source-task boundaries while keeping the dataset, architecture, and training protocol fixed. Forgetting increases consistently as structural reliability decreases, suggesting that replay effectiveness is strongly influenced by the quality of stored structural information, rather than by memory capacity alone. These findings indicate that preserving anatomical structure under domain shift is a central factor in continual medical image segmentation, and that replay mechanisms should account for structural reliability to support robust knowledge retention.

Chinese Translation

在领域转移下，医学图像分割的持续学习仍然面临挑战，因为基于重放的方法通常保留外观信息，而未明确建模解剖结构。本研究探讨了结构一致性是否主导了持续心脏超声分割中的知识保留。我们提出了边界平衡重放网络（BBR-Net），该网络通过边界感知优先级和类别平衡选择重放样本，以保留解剖信息丰富的区域。该方法在CAMUS和CardiacNet上进行了评估，任务顺序包括前向（CAMUS到CardiacNet）和反向（CardiacNet到CAMUS）。在前向设置中，BBR-Net的源任务性能接近离线联合训练参考，同时显著减少灾难性遗忘并保持竞争性的目标任务适应性。消融实验结果表明，边界感知优先级有助于知识保留，并在与类别感知采样结合时改善源任务保留与目标任务适应之间的平衡。相反，在反向设置中，当初始表示是从噪声和结构不一致的数据中学习时，结构感知重放失败。为了隔离这一效应，我们通过逐步破坏源任务边界，同时保持数据集、架构和训练协议不变，进行了受控的结构扰动分析。随着结构可靠性的降低，遗忘现象持续增加，这表明重放的有效性受到存储结构信息质量的强烈影响，而不仅仅是内存容量。这些发现表明，在领域转移下保留解剖结构是持续医学图像分割的一个核心因素，重放机制应考虑结构可靠性以支持稳健的知识保留。

View on arXiv Download PDF AI Translation

cs.CV / 10 / 2606.14732

Steady-Forcing: Balancing Spatial Persistence and Motion Continuity in Long-Horizon Nature Video Diffusion

稳态强制：在长时间自然视频扩散中平衡空间持久性与运动连续性

Minar, Matiur Rahman, Oh, Seunghun, Jeong, GangHyeon, Park, Unsang

Abstract

Autoregressive video diffusion models enable streaming generation but often degrade over long rollouts: static scene layouts drift, while mechanisms that improve spatial stability tend to suppress motion, causing natural flows such as water, fire, or smoke to stagnate. We study this stability-motion trade-off in fixed-camera long-horizon nature video generation, where the two failure modes can be more clearly separated than in moving-camera settings. We propose Steady-Forcing, a memory and training framework combining a persistent visual anchor (V-Sink), an exponential moving-average motion memory (EMA-Sink), block-relative temporal encoding, periodic cache purification, and distillation from a Wan2.1-14B teacher with motion-rewarded priors under task-focused configurations. Together, these components are designed to preserve background identity while sustaining visually plausible fluid dynamics over multi-minute autoregressive rollouts. Evaluations across seven baselines show that Steady-Forcing improves long horizon background consistency and imaging quality, while a blind user study indicates stronger perceived stability and motion continuity. The benchmark evaluation further suggest that generic VBench aggregate scores under-penalize fixed-camera artifacts as well as rewarding drift-induced optical flow as Dynamic Degree while not directly penalizing texture hardening or flow stagnation - motivating future task-specific benchmarks for static-camera nature-flow evaluation. Project page: https://minar09.github.io/steadyforcing/

Chinese Translation

自回归视频扩散模型能够实现流式生成，但在长时间的生成过程中往往会出现质量下降：静态场景布局发生漂移，而改善空间稳定性的机制往往抑制运动，导致水、火或烟等自然流动停滞。我们研究了在固定摄像机的长时间自然视频生成中稳定性与运动之间的权衡，此时两种失败模式的分离比在移动摄像机设置中更为明显。我们提出了稳态强制（Steady-Forcing），这是一种结合持久视觉锚点（V-Sink）、指数移动平均运动记忆（EMA-Sink）、块相对时间编码、周期性缓存净化以及在任务聚焦配置下从Wan2.1-14B教师模型中蒸馏的运动奖励先验的记忆与训练框架。这些组件旨在在多分钟的自回归生成中保持背景身份，同时维持视觉上合理的流体动力学。对七个基线模型的评估表明，稳态强制改善了长时间背景一致性和成像质量，而盲测用户研究显示出更强的感知稳定性和运动连续性。基准评估进一步表明，通用VBench聚合分数对固定摄像机伪影的惩罚不足，并且奖励因漂移引起的光流作为动态度（Dynamic Degree），而没有直接惩罚纹理硬化或流动停滞，这激励了未来针对静态摄像机自然流动评估的任务特定基准。项目页面：https://minar09.github.io/steadyforcing/

View on arXiv Download PDF AI Translation

cs.CV / 11 / 2606.14735

UtVAA: Ultra-tiny Vision Transformer with Affix Attention for Mobile Image Classification

UtVAA：一种具有附加注意力机制的超小型视觉变换器用于移动图像分类

George, Romiyal, Nishankar, Sathiyamohan, Thuseethan, Selvarajah, Ragel, Roshan G.

Abstract

Vision Transformers (ViTs) have demonstrated strong representation capability in image classification. However, their quadratic self-attention complexity and large parameter counts limit deployment on resource-constrained mobile and edge devices. This paper introduces UtVAA, an ultra-tiny Vision Transformer architecture designed for efficient visual recognition under strict computational budgets. It incorporates a novel Affix Attention block that combines depthwise-pointwise local feature extraction, linear self-attention, coordinate attention for spatial dependency modelling, and a lightweight ternary fusion strategy to integrate local and global representations. In addition, Dilated Bottleneck blocks expand the receptive field using dilated depthwise separable convolutions while maintaining low FLOPs and stable optimisation through residual connections. UtVAA is implemented in scalable Tiny, Medium, and Large variants, with the smallest model containing 204.67K parameters and 53.95M FLOPs. Experimental results on CIFAR-10, CIFAR-100, PlantVillage-Tomato and SLIF-Tomato datasets show that UtVAA achieves competitive accuracy within a sub-million-parameter regime. Overall, the results demonstrate that transformer-based vision models can be redesigned into ultra-tiny architectures without significant loss in discriminative performance, making UtVAA suitable for mobile and edge deployment. Code is available at https://github.com/romiyal/UtVAA

Chinese Translation

视觉变换器（ViTs）在图像分类中展现了强大的表征能力。然而，其二次自注意力复杂度和大量参数限制了在资源受限的移动和边缘设备上的部署。本文介绍了UtVAA，一种超小型视觉变换器架构，旨在在严格的计算预算下实现高效的视觉识别。它结合了一种新颖的附加注意力（Affix Attention）模块，该模块结合了深度可分离卷积的局部特征提取、线性自注意力、用于空间依赖建模的坐标注意力以及一种轻量级的三元融合策略，以整合局部和全局表征。此外，扩张瓶颈（Dilated Bottleneck）模块通过扩张深度可分离卷积扩展感受野，同时通过残差连接保持低FLOPs和稳定的优化。UtVAA实现了可扩展的Tiny、Medium和Large变体，其中最小模型包含204.67K参数和53.95M FLOPs。在CIFAR-10、CIFAR-100、PlantVillage-Tomato和SLIF-Tomato数据集上的实验结果表明，UtVAA在百万参数以下的范围内实现了具有竞争力的准确性。总体而言，结果表明基于变换器的视觉模型可以在不显著损失区分性能的情况下重新设计为超小型架构，使UtVAA适合于移动和边缘部署。代码可在https://github.com/romiyal/UtVAA获取。

View on arXiv Download PDF AI Translation

cs.CV / 12 / 2606.14740

GridVQA-X: A Framework for Evaluating Multimodal Explainability Methods

GridVQA-X：一种评估多模态可解释性方法的框架

Belsare, Sujay, Nikhil, Sudarshan, Kumar, Sushant, Kumaraguru, Ponnurangam, Agarwal, Chirag

Abstract

With the increasing development of Vision-Language Models, it becomes imperative that their predictions are readily explainable to relevant stakeholders. However, the field of explainability has not kept pace with the multimodal surge. While recent Multimodal Explainable AI (MxAI) methods generate explanations to attribute the interaction between different modalities, current evaluation protocols lack the ground truth required to distinguish between true cross-modal reasoning (e.g., spatial composition) and shallow cross-modal shortcuts (e.g., Bag-of-Words attribute matching). It remains unknown whether MxAI methods faithfully capture synergistic interactions or merely hallucinate reasoning on models acting as simple feature detectors. In this paper, we introduce GridVQA-X, the first diagnostic framework specifically designed to evaluate cross-modal explainability. Unlike natural datasets, GridVQA-X leverages a closed-world synthesis logic to generate unique, mathematically guaranteed explanations. We utilize this controlled environment to train paired ground-truth models on identical architectures: $M_{\text{pure}}$, which learns robust spatial-relational reasoning and $M_{\text{spur}}$, which is structurally forced to rely on cross-modal shortcuts. This behavioral divergence creates a rigorous testbed: a faithful explainer must report distinct reasoning pathways for each model. Our findings reveal that widely used methods fail to distinguish between models relying on genuine spatial-relational reasoning and those exploiting cross-modal shortcuts, highlighting a critical gap in capturing true cross-modal synergy and misrepresenting how multimodal models actually make decisions.

Chinese Translation

随着视觉-语言模型的不断发展，使相关利益相关者能够轻松理解其预测变得至关重要。然而，可解释性领域并未跟上多模态的快速发展。尽管近期的多模态可解释人工智能（MxAI）方法生成了解释，以归因于不同模态之间的交互，但当前的评估协议缺乏区分真实跨模态推理（例如，空间组合）和浅层跨模态捷径（例如，词袋属性匹配）所需的真实数据。尚不清楚MxAI方法是否真实捕捉了协同交互，还是仅仅在作为简单特征检测器的模型上幻想推理。在本文中，我们介绍了GridVQA-X，这是第一个专门设计用于评估跨模态可解释性的诊断框架。与自然数据集不同，GridVQA-X利用封闭世界合成逻辑生成独特、数学上有保证的解释。我们利用这一受控环境在相同架构上训练成对的真实模型：$M_{ ext{pure}}$，它学习稳健的空间-关系推理，以及$M_{ ext{spur}}$，它在结构上被迫依赖跨模态捷径。这种行为差异创造了一个严格的测试平台：一个真实的解释者必须为每个模型报告不同的推理路径。我们的研究结果表明，广泛使用的方法未能区分依赖于真实空间-关系推理的模型与利用跨模态捷径的模型，这突显了捕捉真实跨模态协同的关键缺口，并误导了对多模态模型实际决策方式的理解。

View on arXiv Download PDF AI Translation

cs.CV / 13 / 2606.14741

HorusEye: Language as Dynamic Attention for Emergency Visual Analysis

HorusEye：将语言作为紧急视觉分析的动态注意力

Yara, Armel

Abstract

We introduce HorusEye, Language as Dynamic Attention for Emergency Visual Analysis. Our investigation followed five stages. The first one is benchmarking RefCOCO-Degraded, a dataset of 15,244 images (3,811 base images x 4 conditions: Clean, Fog, Smoke and Thermal) with systematic visual degradation. Through four research questions, we evaluate multiple VLMs (Gemini, Qwen2-VL, BLIP-2, LLaVA, Kosmos-2) across visual grounding the second stage, language feedback recovery the third one, health VQA tasks the fourth, and hallucination analysis the final stage. Our key finding is that language feedback effectiveness is model-dependent: Gemini achieves +47.3% improvement in thermal conditions through iterative language feedback, while Qwen2-VL shows -5.1% degradation under the same protocol. We also identify the 'Thermal Paradox' where cropping strategies that improve RGB performance catastrophically fail in thermal imagery. Furthermore, BLIP-2 uniquely hallucinates more under degradation, making it unsuitable for emergency deployment

Chinese Translation

我们介绍了HorusEye，将语言作为紧急视觉分析的动态注意力。我们的研究分为五个阶段。第一阶段是基准测试RefCOCO-Degraded，这是一个包含15,244张图像（3,811张基础图像 x 4种条件：清晰、雾霾、烟雾和热成像）的数据集，具有系统性的视觉退化。通过四个研究问题，我们评估了多种视觉语言模型（VLMs），包括Gemini、Qwen2-VL、BLIP-2、LLaVA和Kosmos-2，分别在视觉定位的第二阶段、语言反馈恢复的第三阶段、健康视觉问答任务的第四阶段以及幻觉分析的最后阶段。我们的主要发现是语言反馈的有效性依赖于模型：Gemini在热成像条件下通过迭代语言反馈实现了+47.3%的提升，而Qwen2-VL在相同协议下则显示出-5.1%的退化。我们还识别了“热成像悖论”，即改善RGB性能的裁剪策略在热成像中会灾难性失败。此外，BLIP-2在退化情况下独特地产生更多幻觉，使其不适合紧急部署。

View on arXiv Download PDF AI Translation

cs.CV / 14 / 2606.14746

Style-CCL: Content-Preserving Style Transfer via Curriculum Continual Learning

Style-CCL：通过课程持续学习实现内容保留的风格迁移

Zhang, Shiwen, Wang, Haoyuan, Zang, Xianghao, Huang, Haibin, Zhang, Chi, Li, Xuelong

Abstract

Content-Preserving Style transfer, given content and style references, remains challenging for Diffusion Transformers (DiTs) due to entangled content and style features. With a reverse triplet synthesis pipeline to build a million-scale training set and a dual-branch Style-Content DiT (SC-DiT) that decouples style and content via separate ROPE embeddings and causal masking, we observe that such a one-stage training paradigm on mixed style categories causes semantic styles to dominate, hindering texture style learning, and harming content preservation. To address these issues, we propose Style-CCL, a Multi-Stage Curriculum Continual Learning framework that trains SC-DiT from semantic (easy) to texture (hard) styles, and from clean to synthetic data, with Random Memory Rehearsal across stages to avoid catastrophic forgetting. Extensive experiments demonstrate that our Style-CCL achieves state-of-the-art performance in three core metrics: style similarity, content consistency, and aesthetic quality.

Chinese Translation

内容保留的风格迁移在给定内容和风格参考的情况下，对于扩散变换器（Diffusion Transformers, DiTs）仍然具有挑战性，因为内容和风格特征相互纠缠。通过反向三元组合成管道构建百万规模的训练集，并采用双分支风格-内容变换器（Style-Content DiT, SC-DiT），通过独立的ROPE嵌入和因果掩蔽解耦风格和内容，我们观察到这种在混合风格类别上的单阶段训练范式导致语义风格占主导地位，妨碍了纹理风格的学习，并损害了内容的保留。为了解决这些问题，我们提出了Style-CCL，一个多阶段课程持续学习框架，该框架从语义（简单）风格到纹理（困难）风格，从干净数据到合成数据训练SC-DiT，并在各个阶段进行随机记忆复习以避免灾难性遗忘。大量实验表明，我们的Style-CCL在三个核心指标上实现了最先进的性能：风格相似度、内容一致性和美学质量。

View on arXiv Download PDF AI Translation

cs.CV / 15 / 2606.14747

MMLongEmbed: Benchmarking Multimodal Embedding Models in Long-Context Scenarios

MMLongEmbed：在长上下文场景中基准测试多模态嵌入模型

Wang, Haitian, Sun, Ruoxi, Qiu, Quantong, Li, Juntao, Li, Junhui, Chen, Hua, Chang, Jinxiong, Zhang, Min

Abstract

Recent advancements have significantly expanded the theoretical context windows of Multimodal Embedding Models (MEMs). However, larger context windows do not necessarily translate into effective comprehension and representation of long-context multimodal inputs, which remains a critical bottleneck for real-world deployment. To address the lack of systematic evaluation in this setting, we introduce MMLongEmbed, the first comprehensive benchmark for evaluating MEMs in long-context scenarios. MMLongEmbed comprises four retrieval tasks spanning multiple context-length ranges, covering text, document, and video modalities. Through extensive evaluation of state-of-the-art models, we find that current architectures rely heavily on superficial feature matching and struggle to capture deep semantic and structural dependencies. We further observe that performance degradation varies systematically with context length and key information placement. Moreover, models exhibit substantially different robustness to redundant contextual information across modalities. For reproducibility, the benchmark and code are publicly available.

Chinese Translation

近期的进展显著扩展了多模态嵌入模型（MEMs）的理论上下文窗口。然而，更大的上下文窗口并不一定能有效理解和表征长上下文的多模态输入，这仍然是实际应用中的一个关键瓶颈。为了解决这一领域缺乏系统评估的问题，我们引入了MMLongEmbed，这是第一个用于评估长上下文场景中MEMs的综合基准。MMLongEmbed包含四个检索任务，涵盖多个上下文长度范围，涉及文本、文档和视频模态。通过对最先进模型的广泛评估，我们发现当前架构在很大程度上依赖于表面特征匹配，难以捕捉深层语义和结构依赖关系。我们进一步观察到，性能下降与上下文长度和关键信息位置之间存在系统性变化。此外，模型在不同模态中对冗余上下文信息的鲁棒性表现出显著差异。为了确保可重复性，该基准和代码已公开发布。

View on arXiv Download PDF AI Translation

cs.CV / 16 / 2606.14748

Is My Vision-Language Data in Your AI? Membership Inference Test (MINT) Demo 2

我的视觉-语言数据在你的人工智能中吗？成员推断测试（MINT）演示 2

DeAlcala, Daniel, Mancera, Gonzalo, Fierrez, Julian, Morales, Aythami, Tolosana, Ruben, Vera-Rodriguez, Ruben

Abstract

We present the Membership Inference Test (MINT) Demo 2, a framework designed to improve transparency in machine learning training processes. MINT is a technique for experimentally determining whether specific data were used during machine learning model training. We establish the theoretical framework and propose multiple architectures for MINT depending on the amount of information known about the models that are being audited. Experimental results using a popular face recognition model, 4 state-of-the-art LLMs, and multiple, diverse, and large-scale public image and text databases achieve promising accuracy levels in the detection of training data of up to 90%. Building on these results, we introduce a comprehensive web platform1 that expands these capabilities to image and text modalities. The platform integrates a diverse technological stack, including MINT, aMINT, and gMINT, allowing users to audit a wide range of models. This demonstrator aims to promote AI transparency and provides a practical tool to foster compliance with emerging AI regulations.

Chinese Translation

我们提出了成员推断测试（MINT）演示 2，这是一个旨在提高机器学习训练过程透明度的框架。MINT是一种实验性技术，用于确定特定数据是否在机器学习模型训练过程中被使用。我们建立了理论框架，并根据对被审计模型已知信息的多少，提出了多种MINT架构。使用一个流行的人脸识别模型、4个最先进的大型语言模型（LLMs）以及多个多样化的大规模公共图像和文本数据库的实验结果显示，训练数据的检测准确率高达90%。基于这些结果，我们推出了一个综合性网络平台，扩展了这些能力到图像和文本模态。该平台整合了多样的技术栈，包括MINT、aMINT和gMINT，使用户能够审计广泛的模型。这个演示工具旨在促进人工智能的透明度，并提供一个实用工具，以促进遵守新兴的人工智能法规。

View on arXiv Download PDF AI Translation

cs.CV / 17 / 2606.14749

Automated 3D Kinematic Monitoring for Circadian Activity and Anomaly Detection in Juvenile Fish

自动化三维运动监测用于幼鱼的昼夜活动与异常检测

Huang, Chih-Wei, Huang, Chang-Wen, Chiang, Chung-Ping, Pan, Tsung-Wei

Abstract

Precision aquaculture faces a "phenotyping bottleneck" in tracking high-resolution behavioral traits, as conventional methods cannot quantify instantaneous three-dimensional (3D) physical exertion. To address this, we present a high-throughput 3D behavioral phenotyping framework integrating deep learning object detection with binocular stereo vision for real-time monitoring of juvenile tilapia in high-density environments. The system automates non-contact body length estimation and reconstructs 3D swimming trajectories from absolute spatial coordinates. By eliminating 2D perspective distortions, this approach precisely quantifies 3D velocity and acceleration, marking the first estimation of true physical swimming speeds in free-roaming juveniles. Results show the framework successfully establishes circadian locomotor baselines, serving as an early warning system for physiological stress and providing an objective metric for fish vitality.

Chinese Translation

精准水产养殖在追踪高分辨率行为特征时面临“表型瓶颈”，因为传统方法无法量化瞬时的三维（3D）身体活动。为了解决这一问题，我们提出了一种高通量的3D行为表型框架，该框架将深度学习目标检测与双目立体视觉相结合，用于对高密度环境中幼鱼（如尼罗罗非鱼）的实时监测。该系统自动化了非接触式的体长估计，并从绝对空间坐标重建3D游动轨迹。通过消除二维透视失真，该方法精确量化了3D速度和加速度，标志着首次对自由游动幼鱼真实游泳速度的估计。结果表明，该框架成功建立了昼夜运动基线，作为生理压力的预警系统，并提供了鱼类活力的客观指标。

View on arXiv Download PDF AI Translation

cs.CV / 18 / 2606.14752

X-Tokenizer: A Multimodal Action Tokenizer for Vision-Language-Action Pretraining

X-Tokenizer：一种用于视觉-语言-动作预训练的多模态动作标记器

Kang, Xirui, Shi, Yanpei, Liang, Lucy, Gan, Roy, Liu, Dongxiu, Zhang, Pushi, Chen, Danpeng, Qin, Xiaoyi, Zheng, Yinan, Zheng, Jinliang, Wang, Hao, Zhan, Xianyuan, Su, Hang

Abstract

Modern Vision-Language-Action (VLA) models must bridge pretrained vision-language reasoning and precise continuous robot control. Existing action tokenizers discretize actions primarily for reconstruction, producing codes that preserve motion geometry but provide only weak semantic supervision to the backbone. We therefore formulate action tokenization not as mere compression, but as semantic interface learning between multimodal reasoning and executable control. To this end, we introduce X-Tokenizer, a lightweight encoder-Semantic Residual Quantization (SRQ)-decoder architecture that provides a shared action interface across diverse robotic arm embodiments. Its key component, SRQ, imposes an asymmetric structure on residual vector quantization: the first level is trained with Masked Action Modeling (MAM) to form a discrete action language that captures coarse motion intent, while deeper levels remain reconstruction-oriented residuals that preserve fine-grained details. To further align action tokens with multimodal semantics, X-Tokenizer is pretrained with contrastive alignment to the representation space of a pretrained foundation model and with next-frame vision-language feature prediction. Pretrained on 2.4M trajectories (2.0B action frames), a single frozen X-Tokenizer plugs into a mixed discrete-continuous VLA as a representation-shaping supervision signal. X-Tokenizer achieves top real-world aggregate and strong RoboTwin 2.0 simulation results. Outperforming FAST in multimodal grounding (+13.5%) and long-horizon tasks (+8.25), it shows that action tokenizers serve as semantic interfaces for VLA pretraining beyond mere action compression.

Chinese Translation

现代视觉-语言-动作（VLA）模型必须在预训练的视觉-语言推理与精确的连续机器人控制之间架起桥梁。现有的动作标记器主要为重建而离散化动作，生成的代码保留了运动几何信息，但对主干网络提供的语义监督较弱。因此，我们将动作标记化的定义从单纯的压缩转变为多模态推理与可执行控制之间的语义接口学习。为此，我们提出了X-Tokenizer，一种轻量级的编码器-语义残差量化（Semantic Residual Quantization, SRQ）-解码器架构，提供跨多种机器人臂实现的共享动作接口。其关键组件SRQ对残差向量量化施加了不对称结构：第一层通过掩蔽动作建模（Masked Action Modeling, MAM）进行训练，以形成捕捉粗略运动意图的离散动作语言，而更深层则保持以重建为导向的残差，保留细粒度细节。为了进一步将动作标记与多模态语义对齐，X-Tokenizer通过对比对齐预训练基础模型的表示空间以及下一帧视觉-语言特征预测进行预训练。在240万条轨迹（20亿个动作帧）上进行预训练后，单个冻结的X-Tokenizer可以作为混合离散-连续VLA的表示塑形监督信号。X-Tokenizer在现实世界的综合表现和RoboTwin 2.0模拟结果中均取得了最佳成绩。在多模态基础上超越FAST的表现（+13.5%）和长时间任务（+8.25%），表明动作标记器作为VLA预训练的语义接口，不仅仅是动作压缩。

View on arXiv Download PDF AI Translation

cs.CV / 19 / 2606.14753

Beyond Self-Attention: Sub-Quadratic Vision Transformers for Fast Image Captioning

超越自注意力：用于快速图像描述的亚二次视觉变换器

Ghosh, Chiradeep, Kisku, Dakshina Ranjan

Abstract

Image captioning is a challenging and significant task that aims to generate coherent and semantically meaningful textual descriptions for given images. To accomplish this task, it requires a deep understanding of visual content along with the ability to express that understanding in natural language. Despite remarkable progress with transformer-based architectures, existing approaches often suffer from limitations, such as a lack of rich local feature representations and the high computational cost of quadratic self-attention. The proposed model focuses on improving computational efficiency by restructuring the vision transformer architecture. In designing this approach, the standard self-attention mechanism in Vision Transformers is replaced with a probabilistic transformer approach based on a Gaussian Mixture Model (GMM), a soft-clustering technique. Instead of computing pairwise attention among all image patches, the model groups similar patches into a fixed number of clusters using an Expectation-Maximization (EM) algorithm. This clustering-based mechanism reduces the computational complexity from quadratic O(n^2) to linear O(nK), where K << n. The autoregressive GPT-based decoder is used for caption generation. The model is evaluated on the Flickr 30K dataset, demonstrating competitive and significant improvement over existing works.

Chinese Translation

图像描述是一项具有挑战性且重要的任务，旨在为给定图像生成连贯且语义丰富的文本描述。为了完成这一任务，需要对视觉内容有深入的理解，并能够以自然语言表达这种理解。尽管基于变换器的架构取得了显著进展，但现有方法往往存在一些局限性，例如缺乏丰富的局部特征表示以及二次自注意力的高计算成本。所提出的模型专注于通过重构视觉变换器架构来提高计算效率。在设计这一方法时，视觉变换器中的标准自注意力机制被基于高斯混合模型（Gaussian Mixture Model, GMM）的概率变换器方法所替代，这是一种软聚类技术。模型不再计算所有图像块之间的成对注意力，而是使用期望最大化（Expectation-Maximization, EM）算法将相似的图像块分组为固定数量的聚类。这种基于聚类的机制将计算复杂度从二次的 O(n^2) 降低到线性的 O(nK)，其中 K << n。自回归的基于 GPT 的解码器用于生成描述。该模型在 Flickr 30K 数据集上进行了评估，显示出与现有工作相比具有竞争力和显著的改进。

View on arXiv Download PDF AI Translation

cs.CV / 20 / 2606.14754

Sub-Semantic Image Segmentation

子语义图像分割

Zada, Aviad Cohen, Orenstein, Nadav, Avidan, Shai, Oren, Gal

Abstract

Images can be segmented based on visual cues (i.e., texture segmentation) or into objects (i.e., semantic segmentation). We propose a new category of sub-semantic image segmentation that blurs the line between the two. In sub-semantic image segmentation, language is not used to name whole objects. Instead, it is used to partition an image into stable appearance patterns that can be described by language. To do that, we couple a general-purpose vision-language model to SAM 3, a promptable segmentation backbone whose native text pathway can ground rich descriptions into masks. Simple coupling fails for a number of reasons that we identify in the paper, and we overcome them by introducing DETECTURE that resolves three concrete failure modes -- language leakage between texture regions, prompt competition inside the segmentation backbone, and semantic distortion at the language-to-mask interface. Since there is no dataset of sub-semantic image segmentation, we introduce one, termed TextureADE. The new dataset is derived from the ADE20K dataset using a system we designed. We compare DETECTURE to a number of baselines and find that it achieves the strongest performance on several datasets using different metrics. Code is available at https://github.com/Scientific-Computing-Lab/TextureDetecture.

Chinese Translation

图像可以基于视觉线索（即纹理分割）或对象（即语义分割）进行分割。我们提出了一种新的子语义图像分割类别，模糊了两者之间的界限。在子语义图像分割中，语言并不用于命名整个对象，而是用于将图像划分为可以用语言描述的稳定外观模式。为此，我们将通用视觉-语言模型与 SAM 3（一个可提示的分割骨干网络）结合在一起，其原生文本通道可以将丰富的描述转化为掩膜。简单的结合由于我们在论文中识别出的多种原因而失败，我们通过引入 DETECTURE 解决了三种具体的失败模式——纹理区域之间的语言泄漏、分割骨干内部的提示竞争，以及语言与掩膜接口处的语义扭曲。由于没有子语义图像分割的数据集，我们引入了一个名为 TextureADE 的数据集。该新数据集是通过我们设计的系统从 ADE20K 数据集中派生而来的。我们将 DETECTURE 与多个基线进行比较，发现它在使用不同指标的多个数据集上取得了最佳性能。代码可在 https://github.com/Scientific-Computing-Lab/TextureDetecture 获取。

View on arXiv Download PDF AI Translation

cs.CV / 21 / 2606.14755

Where Does Texture Evidence Live in SAM? Features, Proposal Masks, and Texture Segmentation

纹理证据在SAM中何处存在？特征、提议掩码与纹理分割

Orenstein, Nadav, Zada, Aviad Cohen, Avidan, Shai, Oren, Gal

Abstract

Texture segmentation stresses foundation segmentation because meaningful regions are defined by material or repeated appearance rather than object identity. Segment Anything Models (SAMs) often fail by default on such texture-defined partitions, but this failure is ambiguous: the texture evidence may be absent, missing from the proposal bank, or present but selected or assembled incorrectly by an object-centric readout. We ask what texture-relevant evidence is already preserved in frozen SAM before adaptation. We study two frozen evidence spaces: multiscale features, probed with a minimal clustering readout, and the automatic proposal bank, treated as evidence for a supervised consolidation readout. SAM is frozen throughout; we do not fine-tune the backbone or retrain the proposal generator. Across RWTD, STLD, an ADE20K-selected refined-crop complement, and a ControlNet-stitched PTD bridge archive, frozen SAM is not a texture segmenter by default, but its failures are not simple texture blindness. Coarse frozen features preserve texture organization, and proposal banks often contain texture-aligned masks or fragments. Natural scenes more often require assembly and commitment over fragments, while cleaner synthetic cases more often reduce to selecting an already coherent proposal. Default mask failure should therefore be decomposed into representation evidence, proposal-bank support, readout mismatch, and commitment failure.

Chinese Translation

纹理分割强调基础分割，因为有意义的区域是由材料或重复外观定义的，而不是由物体身份定义的。Segment Anything Models (SAMs) 在这种纹理定义的分区上通常默认失败，但这种失败是模糊的：纹理证据可能缺失、未包含在提议库中，或者存在但被物体中心的读取方式错误选择或组装。我们探讨在适应之前，冻结的SAM中已经保留了哪些与纹理相关的证据。我们研究了两个冻结的证据空间：通过最小聚类读取探测的多尺度特征，以及作为监督整合读取证据的自动提议库。SAM在整个过程中保持冻结；我们不对主干进行微调，也不重新训练提议生成器。在RWTD、STLD、ADE20K选择的精细裁剪补充和ControlNet拼接的PTD桥接档案中，冻结的SAM默认并不是一个纹理分割器，但它的失败并不是简单的纹理盲。粗糙的冻结特征保留了纹理组织，而提议库通常包含与纹理对齐的掩码或片段。自然场景更常需要对片段进行组装和承诺，而更干净的合成案例则更常简化为选择一个已经连贯的提议。因此，默认掩码失败应分解为表示证据、提议库支持、读取不匹配和承诺失败。

View on arXiv Download PDF AI Translation

cs.CV / 22 / 2606.14756

Divide-and-Denoise: A Game-Theoretic Method for Fairly Composing Diffusion Models

分割与去噪：一种公平组合扩散模型的博弈论方法

Gupta, Abhi, Barabanshchikova, Polina, Garg, Vikas, Kaski, Samuel, Jaakkola, Tommi

Abstract

The abundance of pre-trained diffusion models provides an opportunity for composition. Combining several models, however, runs the risk of one model dominating or models disagreeing with each other. Here, we propose Divide-and-Denoise, a method for coordinating multiple pre-trained diffusion models during sampling. Much like managing a specialized workforce, our method creates a fair but efficient division of labor across models. Central to our method is the notion of an allocation which defines the responsibility of each model to every region of the noisy sample. At every timestep, we then denoise by (i) updating the allocation by solving a fair division game, where we divide the sample into regions that maximize total utility under fairness constraints, and (ii) aligning the models with this allocation, where we guide each model to denoise within its assigned region. This leads to a new composite denoising process that evolves in tandem with a division process. We evaluate Divide-and-Denoise on conditional image generation. Across several quality metrics, including the GenEval benchmark, our method outperforms baselines and resolves common failures including missing objects and mismatched attributes. Experiments show that Divide-and-Denoise utilizes each model's expertise without neglecting any other model.

Chinese Translation

预训练扩散模型的丰富性为组合提供了机会。然而，结合多个模型存在一个模型主导或模型之间不一致的风险。在此，我们提出了分割与去噪（Divide-and-Denoise），一种在采样过程中协调多个预训练扩散模型的方法。我们的办法类似于管理一个专业化的劳动力，创造了模型之间公平而高效的劳动分工。我们方法的核心是分配的概念，它定义了每个模型在噪声样本的每个区域的责任。在每个时间步，我们通过（i）解决一个公平分配博弈来更新分配，在该博弈中，我们将样本划分为在公平约束下最大化总效用的区域，以及（ii）使模型与该分配对齐，引导每个模型在其分配的区域内进行去噪。这样形成了一种新的复合去噪过程，与分割过程同步演变。我们在条件图像生成上评估了分割与去噪。通过多个质量指标，包括GenEval基准，我们的方法优于基线，并解决了常见的失败问题，包括缺失对象和属性不匹配。实验表明，分割与去噪充分利用了每个模型的专长，而没有忽视其他模型。

View on arXiv Download PDF AI Translation

cs.CV / 23 / 2606.14757

Spatial Priors via Space Filling Curves for Small and Limited Data Vision Transformers

通过空间填充曲线为小型和有限数据视觉变换器引入空间先验

Candogan, Leyla Naz, Afzal, Arshia, Puigdemont, Pol, Cevher, Volkan

Abstract

Though Vision Transformers (ViTs) have become the dominant backbone in many computer vision tasks, due to permutation equivariance, their attention mechanism lacks explicit spatial inductive biases. This become particularly important in two settings: when model capacity is small or training data is limited. Inspired by the attention masking strategies in Linear Transformers and the scanning patterns of Vision SSMs, we introduce VIOLIN, a lightweight masked attention mechanism that encodes spatial structure within attention via Space Filling Curves (SFCs) with less than 0.0015% extra parameters and negligible computational overhead. VIOLIN scans the image using multiple SFCs to construct curve-specific decay masks, which are then combined and multiplied with the attention matrix. Across a wide range of evaluations, VIOLIN consistently improves performance. In limited data regimes such as fine-tuning on VTAB-1K, it boosts accuracy across all task groups and by up to 8.7% on the tasks where spatial information is essential. It can be combined with parameter-efficient fine-tuning methods such as LoRA to further increase the performance. Beyond fine-tuning, VIOLIN improves various small scale ViT architectures (e.g., DeiT, DINO) during pretraining on ImageNet-1K. Additionally, on pixel-level CIFAR-100 training, a task that is highly dependent on location information, VIOLIN increases accuracy by up to 7.2%. Overall, VIOLIN provides a computationally efficient yet effective way to inject spatial inductive bias into ViTs, especially benefiting small models and limited data settings.

Chinese Translation

尽管视觉变换器（Vision Transformers, ViTs）已成为许多计算机视觉任务中的主流骨干，但由于其置换等变性，其注意力机制缺乏明确的空间归纳偏置。这在两种情况下尤为重要：当模型容量较小或训练数据有限时。受到线性变换器中的注意力掩蔽策略和视觉SSM（Spatial State Machines）的扫描模式的启发，我们提出了VIOLIN，这是一种轻量级的掩蔽注意力机制，通过空间填充曲线（Space Filling Curves, SFCs）在注意力中编码空间结构，增加的参数少于0.0015%，且计算开销可以忽略不计。VIOLIN使用多个SFC扫描图像，以构建特定曲线的衰减掩码，这些掩码随后与注意力矩阵结合并相乘。在广泛的评估中，VIOLIN始终提高了性能。在有限数据环境下，例如在VTAB-1K上的微调，它在所有任务组中提高了准确率，尤其是在空间信息至关重要的任务上，提升幅度可达8.7%。它可以与如LoRA等参数高效的微调方法结合，以进一步提高性能。除了微调，VIOLIN在ImageNet-1K的预训练过程中也改善了各种小规模ViT架构（例如DeiT、DINO）。此外，在像素级CIFAR-100训练中，这一任务高度依赖位置信息，VIOLIN将准确率提高了多达7.2%。总体而言，VIOLIN提供了一种计算效率高且有效的方式，将空间归纳偏置注入ViTs，特别有利于小模型和有限数据环境。

View on arXiv Download PDF AI Translation

cs.CV / 24 / 2606.14758

Disentangling Hallucinations: Orthogonal Semantic Projection for Robust Interpretability

解构幻觉：用于鲁棒可解释性的正交语义投影

Bilgiç, Emirhan, Caramiaux, Baptiste, Yan, Zhi, Franchi, Gianni

Abstract

As Vision-Language Models are increasingly deployed in safety-critical applications, the trustworthiness of their explanations becomes crucial. Explainable AI (XAI) methods for Vision-Language Models often suffer from semantic hallucination, where attribution maps highlight prominent image regions even when prompted with incorrect text descriptions (e.g., highlighting a dog when prompted ``cat''). Although this problem is widespread, a formal mathematical analysis of XAI methods and CLIP embeddings is largely missing in the literature. We demonstrate that this phenomenon is not specific to a single architecture but is a fundamental consequence of Linear Semantic Leakage in high-dimensional embedding spaces. We propose a unified theoretical framework, Linear Semantic Attribution (LSA), which generalizes across discriminative methods. We introduce OSP, a geometric intervention that utilizes the residual property of OMP to disentangle unique semantic signals from shared concepts. We prove theoretically and demonstrate empirically that OSP minimizes hallucination by orthogonalizing the query vector against distractor concepts, rendering the attribution model blind to shared features while preserving fidelity for correct prompts. Our code is available at: https://github.com/emirhanbilgic/Orthogonal-Semantic-Projection

Chinese Translation

随着视觉-语言模型在安全关键应用中的日益广泛应用，其解释的可信度变得至关重要。视觉-语言模型的可解释人工智能（XAI）方法常常受到语义幻觉的困扰，即在错误的文本描述提示下，归因图仍然突出显示显著的图像区域（例如，在提示“猫”时突出显示一只狗）。尽管这一问题普遍存在，但文献中对XAI方法和CLIP嵌入的正式数学分析仍然缺乏。我们证明这一现象并非特定于单一架构，而是高维嵌入空间中线性语义泄漏的基本结果。我们提出了一个统一的理论框架——线性语义归因（Linear Semantic Attribution, LSA），该框架在判别方法之间具有广泛的适用性。我们引入了正交语义投影（Orthogonal Semantic Projection, OSP），这是一种几何干预方法，利用正交匹配追踪（Orthogonal Matching Pursuit, OMP）的残差特性，从共享概念中解构独特的语义信号。我们在理论上证明并在实证中展示，OSP通过将查询向量正交化于干扰概念，最小化幻觉，使归因模型对共享特征失去敏感性，同时保持对正确提示的忠实性。我们的代码可在以下链接获取：https://github.com/emirhanbilgic/Orthogonal-Semantic-Projection

View on arXiv Download PDF AI Translation

cs.CV / 25 / 2606.14759

Temporally Consistent and Controllable Video Generation of 2D Cine CMR via Latent Space Motion Modeling

通过潜在空间运动建模实现时序一致且可控的2D Cine心脏磁共振视频生成

Cao, Yiheng, Andrade-Miranda, Gustavo, Zhang, Jiatian, Sallé, Guillaume, Gao, Xin

Abstract

Cine cardiac magnetic resonance is the gold standard for assessing cardiac function, but the scarcity of public datasets limits the development of advanced data-driven models. To address this limitation, we propose a generative method for synthesizing temporally coherent and anatomically consistent cardiac sequences. Our text-to-video framework decouples cardiac spatial structure from temporal motion. First, a fine-tuned diffusion model synthesizes an initial frame from a clinical text prompt, controlling anatomical features. Then, a latent flow model conditioned on a cardiac phase embedding generates the complete cardiac motion, ensuring spatial consistency and temporal control. Our model generates anatomically and pathologically diverse sequences with high temporal coherence and strong fidelity to input prompts, achieving a FID of 31.68 for image realism and a CLIP score of 31.04 for text-image alignment. These experimental results highlight its potential to produce high-fidelity, on-demand medical data, offering a scalable solution to data scarcity.

Chinese Translation

Cine心脏磁共振是评估心脏功能的金标准，但公共数据集的稀缺限制了先进数据驱动模型的发展。为了解决这一限制，我们提出了一种生成方法，用于合成时序一致且解剖结构一致的心脏序列。我们的文本到视频框架将心脏的空间结构与时间运动解耦。首先，经过微调的扩散模型根据临床文本提示合成初始帧，控制解剖特征。然后，基于心脏相位嵌入的潜在流模型生成完整的心脏运动，确保空间一致性和时间控制。我们的模型生成解剖和病理多样的序列，具有高时序一致性和对输入提示的强保真度，图像真实感的FID为31.68，文本-图像对齐的CLIP得分为31.04。这些实验结果突显了其生成高保真、按需医疗数据的潜力，为数据稀缺问题提供了可扩展的解决方案。

View on arXiv Download PDF AI Translation

cs.CV / 26 / 2606.14760

GeoRoPE: Ground-Aware Rotary Adaptation for Remote Sensing Foundation Models

GeoRoPE：面向遥感基础模型的地面感知旋转适应

Luo, Yu, Hu, Kun, He, Mengwei, Zhu, Xiaogang, Zeng, Shan, Benter, Allen, Xiang, Wei, Filippi, Patrick, Bishop, Thomas Francis, Wang, Zhiyong

Abstract

Remote-sensing foundation models (RSFMs) benefit from pretraining on imagery from multiple sensors and ground sampling distances (GSDs), but such exposure alone does not resolve scale mismatch during downstream adaptation. A fixed token-grid offset can correspond to different ground distances across sensors, making grid-based positional priors physically inconsistent. Meanwhile, heterogeneous spatial granularity means that compact urban regions and homogeneous landscapes may require different positional sensitivities even under the same GSD. Therefore, we propose {GeoRoPE}, a ground-aware, RoPE-compatible, and parameter-efficient spatial adaptation method for RSFMs. GeoRoPE recalibrates token-level positional interactions from two complementary aspects. First, \textit{Geo-Coordinate Calibration (GCC)} rescales raw token-grid offsets according to the ground distance represented by one token-grid step, producing geo-calibrated relative coordinates across GSDs. Second, \textit{Geo-Frequency Calibration (GFC)} adjusts the native RoPE frequency with a relation-specific factor, enabling position sensitive adaptation to scene-dependent spatial granularity. GeoRoPE is injected into pretrained RSFMs through a lightweight adapter, preserving the frozen spatial prior while adding geo-aware positional corrections. Experiments across multiple RSFMs, sensors, resolutions, and downstream tasks demonstrate that GeoRoPE improves cross-resolution robustness and scale-sensitive representation learning.

Chinese Translation

遥感基础模型（RSFMs）通过对来自多个传感器和地面采样距离（GSDs）的图像进行预训练而受益，但仅仅这种曝光并不能解决下游适应过程中的尺度不匹配问题。固定的标记网格偏移在不同传感器之间可能对应于不同的地面距离，使得基于网格的位置信息在物理上不一致。同时，异构的空间粒度意味着即使在相同的GSD下，紧凑的城市区域和均质的景观可能需要不同的位置信息敏感性。因此，我们提出了GeoRoPE，这是一种面向地面的、兼容RoPE的、参数高效的RSFM空间适应方法。GeoRoPE从两个互补的方面重新校准标记级别的位置信息交互。首先， extit{地理坐标校准（Geo-Coordinate Calibration, GCC）}根据一个标记网格步长所表示的地面距离重新缩放原始标记网格偏移，生成跨GSD的地理校准相对坐标。其次， extit{地理频率校准（Geo-Frequency Calibration, GFC）}通过特定关系因子调整原生RoPE频率，使得适应场景依赖的空间粒度时具有位置敏感性。GeoRoPE通过轻量级适配器注入到预训练的RSFMs中，保留冻结的空间先验，同时添加地面感知的位置信息修正。在多个RSFM、传感器、分辨率和下游任务上的实验表明，GeoRoPE提高了跨分辨率的鲁棒性和尺度敏感的表示学习。

View on arXiv Download PDF AI Translation

cs.CV / 27 / 2606.14762

Scribby: A Multi-Level LLM Framework for Semantic Video Analysis

Scribby：一种用于语义视频分析的多层次大语言模型框架

Abelarde, Julian, Belinchon, Hugo Garrido-Lestache

Abstract

As video content continues to expand across educational platforms, recorded lectures, and live-streamed entertainment, the need for efficient and structured analysis of long-form footage has increased \cite{1}. Although many existing AI programs provide high-level video summaries based on AI-generated transcripts \cite{2,3,4,5}, these approaches are often limited to coarse overviews and lack detailed analysis of a video's structure, thematic progression, and semantic relationships, all of which are required for comprehensive video analysis. This paper proposes an LLM-based video summarization framework that balances macro-level comprehension with micro-level semantic analysis \cite{6,12,13}. The first stage of the process indexes the video at a micro level by (1) analyzing the full transcript, (2) analyzing individual transcript sentences, and (3) grouping these sentences by semantic similarity using an LLM as a judge \cite{6,13}. Contextual continuity is retained during sentence-level processing by incorporating both the global transcript analysis and adjacent sentence information into each evaluation prompt. This framework establishes a foundation for video analysis tools that visualize semantic chunking and semantic matching through relevance-based heatmaps. Limitations and future expansions of the framework are also discussed.

Chinese Translation

随着视频内容在教育平台、录制讲座和直播娱乐中的不断扩展，对长格式视频的高效和结构化分析的需求也在增加 [1m。尽管许多现有的人工智能程序基于人工智能生成的转录文本提供高层次的视频摘要 [2m, [3m, [4m, [5m，但这些方法通常仅限于粗略的概述，缺乏对视频结构、主题进展和语义关系的详细分析，而这些都是全面视频分析所必需的。本文提出了一种基于大语言模型（LLM）的视频摘要框架，平衡了宏观层面的理解与微观层面的语义分析 [6m, [12m, [13m。该过程的第一阶段通过（1）分析完整的转录文本，（2）分析单个转录句子，以及（3）使用LLM作为判断者根据语义相似性对这些句子进行分组，从微观层面对视频进行索引 [6m, [13m。在句子级处理过程中，通过将全局转录分析和相邻句子信息纳入每个评估提示，保持了上下文的连续性。该框架为视频分析工具奠定了基础，这些工具通过基于相关性的热图可视化语义分块和语义匹配。本文还讨论了该框架的局限性和未来扩展方向。

View on arXiv Download PDF AI Translation

cs.CV / 28 / 2606.14764

Avoiding Exponential Blow-Up in Distributive Lattice Submodular Minimization

避免分配格子子次模最小化中的指数膨胀

Shanu, Ishant

Abstract

Submodular function minimization has gained a lot of interest in recent years. They are highly applicable in the area of Computer Vision and Machine Learning. Often such applications require to work with submodular functions defined on distributive lattice. Current best way of dealing with it is using a transformation which extrapolates the submodular function for the respective boolean lattice. It makes optimization system too inefficient due to enlargement of the working space. Quantitatively, the expanded space has additional exponential (in set size) number of elements. We propose a generic framework for dealing with distributive lattice which only works within distributive lattice. Our framework allows one to use already established submodular function minimization algorithms for boolean lattice. In our experiment, we show the huge improvement in terms of running time over tranditional methods for handling distributive lattice.

Chinese Translation

子次模函数最小化近年来引起了广泛关注。它们在计算机视觉和机器学习领域具有很高的应用价值。通常，这些应用需要处理定义在分配格子上的子次模函数。目前处理此类问题的最佳方法是使用一种变换，该变换将子次模函数外推到相应的布尔格子。这使得优化系统变得非常低效，因为工作空间的扩大。定量而言，扩展后的空间包含额外的指数级（相对于集合大小）元素。我们提出了一个通用框架，用于处理仅在分配格子内工作的分配格子。我们的框架允许使用已建立的布尔格子的子次模函数最小化算法。在我们的实验中，我们展示了与传统处理分配格子的方法相比，在运行时间上有显著的改进。

View on arXiv Download PDF AI Translation

cs.CV / 29 / 2606.14765

Momentum-Guided Semantic Forecasting (MoFore) for Self-Supervised Video Representation Learning

基于动量引导的语义预测（MoFore）用于自监督视频表征学习

Xu, Qinwu

Abstract

Self-supervised video representation learning has recently advanced through contrastive learning, masked reconstruction, and predictive representation learning. Reconstruction-based approaches such as MAE and VideoMAE learn representations by recovering masked visual content \cite{he2022mae,tong2022videomae}, while contrastive methods such as CLIP learn semantically meaningful embedding spaces through representation alignment \cite{radford2021clip}. In this work, we introduce a Momentum-Guided Semantic Forecasting framework (MoFore) for self-supervised video representation learning. Instead of optimizing for pixel-level reconstruction or task-specific semantic alignment, the proposed method learns temporally predictive video representations by forecasting future latent embeddings from temporally distant context clips. To improve robustness across temporal scales, we further introduce randomized temporal-gap forecasting during training. The framework combines predictive latent forecasting with contrastive regularization to encourage temporal consistency while preventing representation collapse. Experiments on the UCF101 dataset demonstrate that the proposed framework learns temporally consistent and semantically meaningful video representations without using action labels during training. Quantitative analysis shows strong temporal stability and emergent category-level structure in the learned embedding space, while qualitative retrieval experiments reveal motion-aware organization across related activities. Overall, the results suggest that long-range latent forecasting provides an effective and computationally efficient approach for self-supervised video representation learning without relying on reconstruction-based objectives.

Chinese Translation

自监督视频表征学习最近通过对比学习、掩蔽重建和预测表征学习取得了进展。基于重建的方法，如 MAE 和 VideoMAE，通过恢复被掩蔽的视觉内容来学习表征 [0m，而对比方法，如 CLIP，通过表征对齐学习语义上有意义的嵌入空间。在本研究中，我们提出了一种动量引导的语义预测框架（MoFore）用于自监督视频表征学习。该方法不再优化像素级重建或特定任务的语义对齐，而是通过从时间上远离的上下文片段预测未来的潜在嵌入来学习时间预测的视频表征。为了提高在时间尺度上的鲁棒性，我们进一步在训练过程中引入了随机时间间隔预测。该框架将预测潜在预测与对比正则化相结合，以鼓励时间一致性，同时防止表征崩溃。在 UCF101 数据集上的实验表明，所提出的框架在训练过程中无需使用动作标签即可学习时间一致且语义上有意义的视频表征。定量分析显示，学习到的嵌入空间具有强的时间稳定性和新兴的类别级结构，而定性检索实验则揭示了相关活动之间的运动感知组织。总体而言，结果表明，长距离潜在预测为自监督视频表征学习提供了一种有效且计算高效的方法，而无需依赖基于重建的目标。

View on arXiv Download PDF AI Translation

cs.CV / 30 / 2606.14766

XMedFusion: A Knowledge-Guided Multimodal Perception and Reasoning Framework for Autonomous Medical Systems

XMedFusion：一种知识引导的多模态感知与推理框架用于自主医疗系统

Riaz, Hamza, Haroon, Arham, Baig, Maha, Rizwan, Muhammad Dawood, Bajwa, Muhammad Naseer, Fraz, Muhammad Moazam

Abstract

Autonomous medical and robotic systems increasingly rely on intelligent perception and reasoning capabilities to interpret visual data and support clinical decision making. Radiology report generation represents a critical component of such automated diagnostic workflows, yet existing end-to-end multimodal models often suffer from weak visual grounding, resulting in unreliable interpretations and omission of subtle clinical findings. This paper presents XMedFusion, a modular AI framework designed as an intelligent perception and reasoning module for autonomous medical systems. The proposed framework decomposes visual information into coordinated functional components that emulate expert-driven analysis, including a visual perception agent that extracts image-grounded evidence, a knowledge graph construction agent that structures clinically relevant findings, and a retrieval-guided drafting process that ensures a consistent reporting structure. A synthesis agent iteratively integrates visual and structured evidence through reasoning-driven verification to produce reliable and interpretable diagnostic outputs. Experimental evaluation on a public chest radiograph dataset demonstrates significant improvements over baseline vision-language models, achieving gains from 0.0493 to 0.3359 in BLEU-1, 0.0863 to 0.2440 in ROUGE-L, and 0.0829 to 0.1708 in METEOR, along with substantial improvements in semantic evaluation metrics such as Consistency (2.38 to 7.80) and Accuracy (2.34 to 6.93). The results highlight the effectiveness of structured multi-agent perception and reasoning for enhancing robustness, transparency, and automation in intelligent medical imaging systems, enabling integration into autonomous healthcare and robotic diagnostic workflows.

Chinese Translation

自主医疗和机器人系统日益依赖智能感知和推理能力来解读视觉数据并支持临床决策。放射学报告生成是此类自动化诊断工作流程中的一个关键组成部分，然而现有的端到端多模态模型往往存在视觉基础薄弱的问题，导致不可靠的解读和对微妙临床发现的遗漏。本文提出了XMedFusion，一个模块化的人工智能框架，旨在作为自主医疗系统的智能感知与推理模块。该框架将视觉信息分解为协调的功能组件，模拟专家驱动的分析，包括一个提取图像基础证据的视觉感知代理，一个构建临床相关发现的知识图谱构建代理，以及一个确保一致报告结构的检索引导草拟过程。合成代理通过推理驱动的验证迭代整合视觉和结构化证据，以生成可靠且可解释的诊断输出。在公共胸部X光数据集上的实验评估显示，相较于基线视觉-语言模型，显著提高了性能，BLEU-1从0.0493提升至0.3359，ROUGE-L从0.0863提升至0.2440，METEOR从0.0829提升至0.1708，同时在一致性（从2.38提升至7.80）和准确性（从2.34提升至6.93）等语义评估指标上也有显著改善。结果突显了结构化多代理感知与推理在增强智能医疗影像系统的鲁棒性、透明性和自动化方面的有效性，使其能够融入自主医疗和机器人诊断工作流程。

View on arXiv Download PDF AI Translation

cs.CV / 31 / 2606.14770

An Empirical Analysis of Optimization Dynamics and Sparsity Boundaries in Large-Scale Pedestrian Attribute Recognition

大规模行人属性识别中的优化动态与稀疏边界的实证分析

Mir, Houssam El

Abstract

Pedestrian Attribute Recognition (PAR) is critical for video surveillance, enabling forensic search and re-identification systems. Extreme class imbalance remains a fundamental obstacle when merging PETA and PA-100K into a 109,000-image composite corpus, where minority attributes have positive sample fractions below 1%. This causes standard BCE optimization to suppress rare traits, a phenomenon we term the majority negative class cheating trap. We present a systematic ablation of Multi-Label Focal Loss hyperparameters (alpha and gamma) on a ResNet-18 backbone. A calibrated configuration (alpha=0.50, gamma=2.0) achieves a Macro F1-score of 62.32%, matching BCE baseline while preserving superior hard-example mining and convergence dynamics. Our approach uses pure loss-function engineering with zero computational overhead for edge deployment. We identify the Sparsity Wall, a hard boundary where positive sample fractions below 0.1% make global loss reweighting ineffective, requiring instance-level intervention.

Chinese Translation

行人属性识别（PAR）对于视频监控至关重要，能够支持取证搜索和重识别系统。然而，在将PETA和PA-100K合并为一个包含109,000张图像的复合语料库时，极端类别不平衡仍然是一个基本障碍，其中少数属性的正样本比例低于1%。这导致标准的二元交叉熵（BCE）优化抑制了稀有特征，这一现象我们称之为“多数负类作弊陷阱”。我们对基于ResNet-18的多标签焦点损失（Multi-Label Focal Loss）超参数（alpha和gamma）进行了系统的消融实验。经过校准的配置（alpha=0.50，gamma=2.0）实现了62.32%的宏F1分数，与BCE基线相匹配，同时保持了优越的难例挖掘和收敛动态。我们的方法采用纯损失函数工程，边缘部署时没有计算开销。我们识别出稀疏墙（Sparsity Wall），这是一个硬边界，当正样本比例低于0.1%时，全球损失重加权变得无效，需要进行实例级干预。

View on arXiv Download PDF AI Translation

cs.CV / 32 / 2606.14772

ScoutVLA: UAV-Centric Active Perception via a Dual-Expert VLA Model for Open-World Embodied Question Answering

ScoutVLA：基于双专家VLA模型的无人机中心主动感知用于开放世界的具身问答

Lu, Wenhao, Zhu, Zhengqiu, Wang, Xiaofeng, Zhang, Xiaoran, Ji, Yatai, Zhao, Yong, Hu, Yue, Nie, Yingzhen, Zhu, Jinlong, Zhu, Zheng

Abstract

Aerial Embodied Question Answering (EQA) requires Unmanned Aerial Vehicles (UAVs) to actively perceive the environment and answer natural language questions. Existing outdoor EQA systems usually stop once the target enters the UAV's field of view, leaving the fine-grained viewpoint adjustment needed for evidence-seeking questions largely unresolved. To address this issue, we introduce FG-EQA, a fine-grained active perception EQA benchmark with more than 40K simulated trajectories and 1K real-world trajectories. Drawing inspiration from the ``waggle dance'' of scout bees, which iteratively adjust their flight paths to verify target information, we propose ScoutVLA, an evidence-driven Vision-Language-Action model for outdoor EQA. To emulate this active exploration behavior, ScoutVLA features a decoupled dual-expert architecture: a vision-language expert infers the semantic intent to identify missing evidence, while an independent action expert employs high-DoF flow matching to generate continuous viewpoint-refinement trajectories. To balance the competing demands of continuous control and semantic reasoning, we devise a decoupled training strategy with a knowledge insulation mechanism that prevents the action gradients from erasing the model's multimodal reasoning ability. Extensive simulated experiments and a qualitative real-world field study both verify the superiority of ScoutVLA over the state-of-the-art baselines, demonstrating a 10.48$\boldsymbol{\times}$ higher average strict success rate and a 7.72$\boldsymbol{\times}$ higher average QA correctness.

Chinese Translation

空中具身问答（EQA）要求无人机（UAV）主动感知环境并回答自然语言问题。现有的户外EQA系统通常在目标进入无人机视野后就停止，导致针对证据寻求问题所需的细粒度视角调整大多未得到解决。为了解决这一问题，我们引入了FG-EQA，一个细粒度主动感知EQA基准，包含超过4万条模拟轨迹和1000条真实轨迹。我们受到侦查蜜蜂“摇摆舞”的启发，该舞蹈通过迭代调整飞行路径来验证目标信息，我们提出了ScoutVLA，一个基于证据驱动的视觉-语言-行动（Vision-Language-Action）模型，用于户外EQA。为了模拟这种主动探索行为，ScoutVLA采用了一个解耦的双专家架构：视觉-语言专家推断语义意图以识别缺失的证据，而独立的行动专家则利用高自由度流匹配生成连续的视角调整轨迹。为了平衡连续控制和语义推理的竞争需求，我们设计了一种解耦训练策略，并引入知识隔离机制，以防止行动梯度抹去模型的多模态推理能力。大量的模拟实验和定性真实世界现场研究均验证了ScoutVLA相较于最先进基线的优越性，显示出10.48$oldsymbol{ imes}$更高的平均严格成功率和7.72$oldsymbol{ imes}$更高的平均问答正确率。

View on arXiv Download PDF AI Translation

cs.CV / 33 / 2606.14773

Double-Helix Vision (DH-V2): A Geometry-Based Visual Sampler for Bandwidth-Constrained Perception

双螺旋视觉 (DH-V2)：一种基于几何的视觉采样器，用于带宽受限的感知

Wen, Jinwen

Abstract

We present Double-Helix Vision (DH), a geometry-based visual sampler that compresses 2D images into compact 1D signals using paired golden-ratio-inspired spiral trajectories. Rather than processing every pixel uniformly, DH employs two phase-shifted helices (Alpha and Beta, offset by 180 degrees) to sample the image with biologically-inspired foveation: high density at the center, sparse coverage at the periphery. At 4K resolution, DH achieves a 1,433x compression ratio (99.93% reduction) while preserving the geometric structure of the scene. The full perception pipeline -- including spatial mapping, temporal collision detection, and intra-frame structural disparity estimation -- runs in 0.52 ms at 1080p on CPU-only hardware, with no neural network dependencies. On CIFAR-10 at extreme sampling budgets (K=128 points per helix), DH achieves a +6.03% accuracy gain over uniform random sampling. A JSON-serializable Robotics API is provided, delivering sub-millisecond spatial perception reports in 2.7 KB packets. Code and benchmarks are available under the MIT License.

Chinese Translation

我们提出了双螺旋视觉 (Double-Helix Vision, DH)，这是一种基于几何的视觉采样器，利用成对的黄金比例启发的螺旋轨迹将二维图像压缩为紧凑的一维信号。DH并不是均匀处理每个像素，而是采用两个相位偏移的螺旋（Alpha和Beta，偏移180度）以生物启发的中央凹采样方式对图像进行采样：中心区域高密度，边缘区域稀疏覆盖。在4K分辨率下，DH实现了1433倍的压缩比（99.93%的减少），同时保持场景的几何结构。完整的感知管道——包括空间映射、时间碰撞检测和帧内结构差异估计——在仅使用CPU硬件的1080p下以0.52毫秒的速度运行，且不依赖于神经网络。在极端采样预算下（每个螺旋K=128个点），DH在CIFAR-10上实现了比均匀随机采样高出6.03%的准确率提升。提供了一个可JSON序列化的机器人API，能够以2.7 KB的数据包提供亚毫秒级的空间感知报告。代码和基准测试在MIT许可证下可用。

View on arXiv Download PDF AI Translation

cs.CV / 34 / 2606.14777

JoyAI-VL-Interaction: Real-Time Vision-Language Interaction Intelligence

JoyAI-VL-Interaction：实时视觉-语言交互智能

Yao, Dingyu, Zhou, Junhao, Yang, Chenxu, Qin, Chuanyu, Hou, Haowen, Liang, Zheming, Wang, Congcong, Cao, Yuhang, Ye, Shenglong, Xie, Shuai, Gu, Shuhuan, Huang, Haoyang, Si, Qingyi, Duan, Nan, Wang, Jiaqi

Abstract

Many moments in the real world do not wait for a user to ask. A fire starts on a security monitor, an expression flickers across a video call, or a product a viewer wants flashes by in a livestream. Yet today's large models remain mostly turn-based by design: they answer only when addressed, and even video-call apps that appear interactive still operate as question-answer systems, reacting only when polled or prompted. We argue for a different paradigm: a model that is present in the world like a person. It continuously watches what is happening now, decides on its own whether to speak or stay silent, interacts in real time, and delegates to a background model when the problem is hard. To advance interaction models and their adoption across domains, we make two fully open-sourced contributions. First, we release JoyAI-VL-Interaction, an 8B-scale, vision-first VL-interaction model. The model makes the response decision internally, choosing each second to stay silent, respond, or delegate to a background model, and it excels at vision-triggered responsiveness and time awareness. We pair it with a transferable training recipe, from which capabilities we never trained for emerge, such as guiding a shopper through changing app screens or improvising a lecture from a slide deck. Second, we release a complete, deployable system built around that model. The system streams any ongoing video into the model, making it genuinely present in the world. All other components are pluggable, including ASR/TTS modules, memory, visualization UI, and a background brain that can connect to any API or agent. Across six real-world scenarios, human raters prefer JoyAI-VL-Interaction over the in-app video-call assistants of Doubao and Gemini by a wide margin. To our knowledge, this is the first open, vision-driven interaction model released together with its training recipe, data, and complete deployable system.

Chinese Translation

现实世界中的许多时刻并不等待用户提问。安全监控上发生火灾，视频通话中面部表情闪现，或观众在直播中想要的产品一闪而过。然而，今天的大型模型在设计上仍然主要是基于轮询的：它们仅在被提及时作出回应，即使是看似互动的视频通话应用也仍然作为问答系统运行，仅在被询问或提示时做出反应。我们主张一种不同的范式：一个像人一样存在于世界中的模型。它持续观察当前发生的事情，自主决定是否发言或保持沉默，实时互动，并在问题复杂时委托给后台模型。为了推动交互模型及其在各个领域的应用，我们做出了两个完全开源的贡献。首先，我们发布了JoyAI-VL-Interaction，一个8B规模的视觉优先VL交互模型。该模型内部做出响应决策，每秒选择保持沉默、回应或委托给后台模型，并在视觉触发的响应性和时间意识方面表现出色。我们为其配备了一个可转移的训练方案，从中出现了我们从未训练过的能力，例如引导购物者通过不断变化的应用界面或即兴讲解幻灯片演示。其次，我们发布了一个围绕该模型构建的完整可部署系统。该系统将任何正在进行的视频流输入模型，使其真正存在于世界中。所有其他组件都是可插拔的，包括ASR/TTS模块、内存、可视化用户界面，以及可以连接到任何API或代理的后台大脑。在六个真实场景中，人类评审者普遍更喜欢JoyAI-VL-Interaction，而不是Doubao和Gemini的应用内视频通话助手。根据我们的了解，这是第一个与其训练方案、数据和完整可部署系统一起发布的开放视觉驱动交互模型。

View on arXiv Download PDF AI Translation

cs.CV / 35 / 2606.14778

FactCheck: Feasibility-aware Long-term Action Anticipation with Multi-agent Collaboration

FactCheck：基于可行性意识的多智能体协作长时间动作预测

Cao, Rui, Cao, Jiannong, Yuan, Bo, Wen, Zhiyuan, Zhang, Mingjin

Abstract

Long-term action anticipation (LTA) aims to predict an ordered sequence of future verb-noun actions from a partially observed video. While this task serves as the foundation for embodied intelligence, anticipating physically feasible long-term actions remains a critical challenge. Existing methods, which operate in an open-loop manner, often hallucinate non-existent objects, violate object affordances, or disregard object states, as they lack explicit mechanisms to verify action feasibility against the physical environment. To address this, we propose FactCheck, a novel multi-agent collaboration framework that improves feasibility through a closed-loop "Observe-Plan-Verify" mechanism. FactCheck decomposes the complex LTA task into specialized roles: an Observer that recognizes historical actions from video observations and constructs a dual-form structured memory, comprising a History Action Abstract that captures high-level human intentions and environmental status, and a History Action Graph that encodes object states and temporal dependencies; a Planner that generates draft future actions conditioned on both low-level historical actions and high-level History Action Abstract; and a Verifier that rigorously validates the draft against the History Action Graph and refines infeasible actions. Extensive experiments on the EPIC-Kitchens-55 and EGTEA Gaze+ benchmarks demonstrate that FactCheck consistently outperforms state-of-the-art methods. Our work establishes a new paradigm for feasibility-aware long-term action anticipation, effectively closing the loop of action recognition, action prediction and action verification.

Chinese Translation

长时间动作预测（LTA）旨在从部分观察到的视频中预测未来动词-名词动作的有序序列。虽然这一任务是具身智能的基础，但预测物理上可行的长时间动作仍然是一个关键挑战。现有方法以开放环路的方式运行，常常会幻觉出不存在的物体，违反物体的可用性，或忽视物体状态，因为它们缺乏明确的机制来验证动作在物理环境中的可行性。为了解决这个问题，我们提出了FactCheck，一个新颖的多智能体协作框架，通过闭环的“观察-规划-验证”机制来提高可行性。FactCheck将复杂的LTA任务分解为专门的角色：观察者（Observer）从视频观察中识别历史动作，并构建一种双重结构记忆，包括捕捉高层次人类意图和环境状态的历史动作摘要（History Action Abstract），以及编码物体状态和时间依赖关系的历史动作图（History Action Graph）；规划者（Planner）基于低层次的历史动作和高层次的历史动作摘要生成草拟的未来动作；验证者（Verifier）严格验证草拟动作与历史动作图的一致性，并细化不可行的动作。在EPIC-Kitchens-55和EGTEA Gaze+基准上的大量实验表明，FactCheck始终优于最先进的方法。我们的工作为基于可行性意识的长时间动作预测建立了一个新范式，有效地闭合了动作识别、动作预测和动作验证的环路。

View on arXiv Download PDF AI Translation

cs.CV / 36 / 2606.14780

YTClickbait21K: Human-Annotated Multimodal Dataset for YouTube Clickbait Detection Across Diverse Channels and Content Categories

YTClickbait21K：用于YouTube点击诱饵检测的人类标注多模态数据集，涵盖多样化的频道和内容类别

Islam, Md. Minhazul, Jubaer, Md. Tanbeer, Khandakar, Amith, Sarker, Shovon, Rahman, Sumaiya, Mia, Md. Masum, Ayari, Mohamed Arselene, Noori, Hamed

Abstract

Clickbait content on video-sharing platforms poses a significant challenge to information reliability, yet progress in automated detection has been constrained by the lack of large-scale, high-quality multimodal datasets. We present YTClickbait21K, a human-annotated YouTube clickbait dataset comprising 21,238 videos collected from 40 channels across 29 countries, covering diverse content categories such as news, entertainment, education, and gaming. Each sample includes structured metadata (title, description, engagement statistics) along with associated thumbnail images, enabling comprehensive multimodal analysis. To ensure annotation quality, every video was independently labeled by three annotators using a standardized decision framework that incorporates textual, visual, and cross-modal consistency cues, with final labels determined through majority voting. The dataset exhibits substantial inter-annotator agreement (k=0.65), confirming reliable labeling despite the inherent subjectivity of clickbait detection. By combining scale, annotation rigor, and multimodal richness, this dataset provides a robust benchmark for developing and evaluating machine learning models, facilitating research in cross-modal semantic understanding, and advancing automated content moderation systems.

Chinese Translation

视频分享平台上的点击诱饵内容对信息可靠性构成了重大挑战，但由于缺乏大规模、高质量的多模态数据集，自动检测的进展受到限制。我们提出了YTClickbait21K，这是一个人类标注的YouTube点击诱饵数据集，包含来自29个国家的40个频道收集的21,238个视频，涵盖新闻、娱乐、教育和游戏等多样化内容类别。每个样本包括结构化元数据（标题、描述、参与统计）以及相关的缩略图，使得全面的多模态分析成为可能。为确保标注质量，每个视频由三名标注者独立标注，使用一个标准化的决策框架，该框架结合了文本、视觉和跨模态一致性线索，最终标签通过多数投票确定。该数据集表现出显著的标注者间一致性（k=0.65），确认了尽管点击诱饵检测固有的主观性，标注仍然可靠。通过结合规模、标注严谨性和多模态丰富性，该数据集为开发和评估机器学习模型提供了一个强有力的基准，促进了跨模态语义理解的研究，并推动了自动内容审核系统的发展。

View on arXiv Download PDF AI Translation

cs.CV / 37 / 2606.14781

Variational Deep Unfolding with Mamba-Based Nonlocal Modeling for Underwater Image Enhancement

基于Mamba的非局部建模的变分深度展开用于水下图像增强

Torres, Daniel, Navarro, Julia, Sbert, Catalina, Duran, Joan

Abstract

Underwater imaging plays a crucial role in ocean engineering, although captured data often suffer from poor visibility and color distortion. To address these challenges, we propose a model-based deep unfolding network for underwater image enhancement that integrates variational modeling into a learnable architecture. The framework is guided by a variational formulation based on a dehazing decomposition, incorporating a multiplicative residual component to absorb remaining artifacts and a nonlocal gradient-type constraint to preserve structural details and enhance edge sharpness. We provide a theoretical analysis establishing the existence of solution for the associated minimization problem. The proposed unfolding method incorporates Mamba layers to efficiently capture self-similarities in the scene. In addition, we introduce a proximal trajectory loss that enforces consistency between the unfolding stages and the iterations of an ideal restoration regularizer. Experimental results demonstrate that the proposed unfolding approach achieves improved visual quality and competitive quantitative performance compared with recent state-of-the-art methods. The source code will be available at https://github.com/MIA-UIB/Variational-Unfolding-Mamba-Underwater-Enhancement .

Chinese Translation

水下成像在海洋工程中发挥着至关重要的作用，尽管捕获的数据通常存在能见度差和颜色失真的问题。为了解决这些挑战，我们提出了一种基于模型的深度展开网络，用于水下图像增强，该网络将变分建模集成到可学习的架构中。该框架由基于去雾分解的变分公式指导，结合了乘法残差组件以吸收剩余伪影，并引入非局部梯度类型约束以保持结构细节并增强边缘清晰度。我们提供了理论分析，证明了相关最小化问题解的存在。所提出的展开方法结合了Mamba层，以高效捕捉场景中的自相似性。此外，我们引入了一种近端轨迹损失，以强制展开阶段与理想恢复正则化器的迭代之间的一致性。实验结果表明，所提出的展开方法在视觉质量和定量性能上均优于最近的最先进方法。源代码将可在 https://github.com/MIA-UIB/Variational-Unfolding-Mamba-Underwater-Enhancement 获取。

View on arXiv Download PDF AI Translation

cs.CV / 38 / 2606.14782

Last But Not Least: Boundary Attention CalibratiON for Multimodal KV Cache Compression

最后但并非最不重要：多模态KV缓存压缩的边界注意力校准

Chen, Tianhao, Wu, Yuheng, Yao, Kelu, Xu, Xiaogang, Hu, Xiaobin, Lee, Dongman

Abstract

Multimodal Large Language Models (MLLMs) achieve strong vision-language reasoning, but long visual contexts enlarge the KV cache and increase decoding latency. Existing compression methods rely on observation window attention for stable token-importance estimation, yet this aggregation can dilute sparse visual evidence and discard answer-critical tokens under aggressive compression. Therefore, we identify last-query attention as a complementary source for recovering such evidence, but its answer-irrelevant signals can mislead retention. We propose BACON, a plug-and-play method that calibrates observation window attention with last-query evidence and suppresses isolated noise via intra-layer coherence and inter-layer persistence. Across diverse benchmarks, models, budgets, and compression methods, BACON improves multimodal KV compression by 7.5% on average under the most aggressive budget, with gains up to 30.9%.

Chinese Translation

多模态大型语言模型（MLLMs）在视觉-语言推理方面表现出色，但较长的视觉上下文会扩大KV缓存并增加解码延迟。现有的压缩方法依赖于观察窗口注意力来稳定令牌重要性估计，然而这种聚合可能会稀释稀疏的视觉证据，并在激进压缩下丢弃对答案至关重要的令牌。因此，我们将最后查询注意力识别为恢复此类证据的补充来源，但其与答案无关的信号可能会误导保留。我们提出了BACON，这是一种即插即用的方法，通过最后查询证据校准观察窗口注意力，并通过层内一致性和层间持久性抑制孤立噪声。在各种基准、模型、预算和压缩方法下，BACON在最激进的预算下平均提高了多模态KV压缩7.5%，增益最高可达30.9%。

View on arXiv Download PDF AI Translation

cs.CV / 39 / 2606.14783

The Vision Encoder as a Privacy Boundary: Visual-Token Side Channels in Encoder-Free Vision-Language Models

视觉编码器作为隐私边界：无编码器视觉语言模型中的视觉令牌侧信道

Zhou, Chenyu, Jiang, Qiliang, Wu, Shuning, Zhou, Xu

Abstract

A vision encoder compresses image pixels into semantic embeddings, implicitly acting as a privacy boundary by preserving semantic content while attenuating pixel-local detail required for exact text recovery. Encoder-free vision-language models (VLMs) remove this boundary by routing image patches directly into the language-model token stream, thereby exposing an architectural privacy attack surface: intermediate visual tokens become a pre-output side channel. Under a token-access adversary, decoders invert visual-token streams from two encoder-free VLMs, Gemma4 and Fuyu, recovering recognizable image structure and readable held-out access codes, whereas matched encoder-based controls localize target regions but recover no exact strings. Within-model ablations show that the operative factor is spatial sampling fidelity of the visual-token grid, especially character-direction sampling density, rather than token or value count. The leakage is not limited to exported tokens: Gemma4 layer-0 key-value cache tensors are directly invertible, placing the side channel within KV caches commonly persisted by production serving stacks for decoding efficiency. The attack survives clutter, realistic document degradation, and zero-shot transfer to public document images, and it resists value-level defenses such as additive noise and quantization. Effective mitigation must therefore reduce spatial sampling, making removal of the vision encoder a first-class privacy decision in VLM deployment.

Chinese Translation

视觉编码器将图像像素压缩为语义嵌入，隐含地充当隐私边界，通过保留语义内容而削弱了精确文本恢复所需的像素局部细节。无编码器视觉语言模型（VLMs）通过将图像块直接路由到语言模型的令牌流中，移除了这一边界，从而暴露出一种架构隐私攻击面：中间视觉令牌成为预输出侧信道。在令牌访问对手的情况下，解码器从两个无编码器的 VLMs（Gemma4 和 Fuyu）中反转视觉令牌流，恢复可识别的图像结构和可读的保留访问代码，而匹配的基于编码器的控制则定位目标区域但未能恢复任何精确字符串。模型内部的消融实验表明，操作因素是视觉令牌网格的空间采样保真度，特别是字符方向的采样密度，而非令牌或值的数量。泄漏不仅限于导出的令牌：Gemma4 层0 的键值缓存张量是可直接反转的，这将侧信道置于 KV 缓存中，这些缓存通常由生产服务堆栈持久化以提高解码效率。该攻击能够抵御杂乱、现实文档退化以及对公共文档图像的零-shot 转移，并且能够抵抗值级防御措施，如加性噪声和量化。因此，有效的缓解措施必须减少空间采样，使得在 VLM 部署中移除视觉编码器成为一项重要的隐私决策。

View on arXiv Download PDF AI Translation

cs.CV / 40 / 2606.14787

Vision-Encoder Behavioral Fingerprints of Image-to-Image Generative Models: A Training-Paradigm-Driven Taxonomy of Six Commercial APIs

图像生成模型的视觉编码行为指纹：基于训练范式的六个商业API分类

Hill, Hunter

Abstract

We study six production image-to-image AI systems (gpt-image-1, Gemini 2.5 Flash Image, Flux Kontext, SDXL img2img, SD3 img2img, and Qwen Image Edit) under a content-adaptive sub-JND adversarial perturbation pipeline, scoring all outputs by frozen DINOv2 ViT-B/14 token distances against clean references. Across a 3,588-call corpus spanning COCO photographs, CelebA-HQ portraits, and AI-generated inputs, the six systems partition into two image-invariant behavioral bands on a 2D (patch_mean, ssim_clean) plane: edit-trained models (Flux Kontext, Qwen Edit, Gemini) cluster in a tight band, while T2I-base models adapted at sampling time (SDXL, SD3, gpt-image-1) cluster in a drift band.

Chinese Translation

我们研究了六个生产级图像到图像的人工智能系统（gpt-image-1、Gemini 2.5 Flash Image、Flux Kontext、SDXL img2img、SD3 img2img 和 Qwen Image Edit），在一个内容自适应的次可觉差对抗扰动管道下，对所有输出进行评分，使用冻结的 DINOv2 ViT-B/14 令牌距离与干净参考进行比较。在一个涵盖 COCO 照片、CelebA-HQ 肖像和 AI 生成输入的 3,588 次调用的语料库中，这六个系统在二维（patch_mean, ssim_clean）平面上划分为两个图像不变的行为带：经过编辑训练的模型（Flux Kontext、Qwen Edit、Gemini）聚集在一个紧密的带中，而在采样时适应的 T2I 基础模型（SDXL、SD3、gpt-image-1）则聚集在一个漂移带中。

View on arXiv Download PDF AI Translation

cs.CV / 41 / 2606.14792

Efficient Reinforcement for Visual-Textual Thinking with Discrete Diffusion Model

基于离散扩散模型的视觉-文本思维高效强化学习

Kim, Yoonjeon, Takida, Yuhta, Lai, Chieh-Hsin, Yang, Eunho, Mitsufuji, Yuki

Abstract

RL-based post-training has been widely adopted to enable interleaved visual and textual reasoning in unified multimodal models capable of both text and image generation. However, most existing approaches are built upon autoregressive (AR) unified models, which require full image regeneration during visual reasoning. In this work, we demonstrate that multimodal discrete diffusion models are effective alternatives to AR models for reinforcement learning in interleaved reasoning, owing to their ability to perform efficient visual rollouts via localized visual editing rather than full image-token regeneration. This reduces rollout computation during GRPO by 26.9\% compared to AR baselines, with minimal performance drop. Despite the improved efficiency, we find that joint reward assignment, which employs a shared reward signal across modalities, introduces cross-modal interference between unrelated image and text token sequences during RL updates. To address this issue, we propose factorized reward assignment, a strategy that assigns rewards independently to text and vision segments. With factorized reward assignment, our RL approach achieves an 11.2% improvement over joint reward assignment and a 38.04% improvement over the base model.

Chinese Translation

基于强化学习的后训练已被广泛应用于统一多模态模型中，以实现交错的视觉和文本推理，这些模型能够同时进行文本和图像生成。然而，现有的大多数方法都是建立在自回归（AR）统一模型之上，这要求在视觉推理过程中进行完整的图像再生。在本研究中，我们证明了多模态离散扩散模型是AR模型在交错推理中进行强化学习的有效替代方案，因为它们能够通过局部视觉编辑而非完整的图像-标记再生来高效执行视觉展开。这使得在GRPO过程中，相较于AR基线，展开计算减少了26.9%，且性能下降极小。尽管效率有所提高，我们发现联合奖励分配（即在不同模态之间使用共享奖励信号）在强化学习更新过程中引入了无关图像和文本标记序列之间的跨模态干扰。为了解决这一问题，我们提出了因子化奖励分配策略，该策略独立地为文本和视觉部分分配奖励。通过因子化奖励分配，我们的强化学习方法在联合奖励分配的基础上实现了11.2%的提升，并较基础模型提高了38.04%。

View on arXiv Download PDF AI Translation

cs.CV / 42 / 2606.14795

Position: The Systemic Lack of Agency in Visual Reasoning

位置：视觉推理中的系统性缺乏能动性

Huang, Yizhao, Chen, Haoyang, Wang, Shiqin, Huang, Pohsun, Li, Jiayuan, Du, Haoyuan, Shi, Yandong, Wang, Zheng, Wang, Zhixiang

Abstract

This paper argues that a systemic lack of Agency constrains the implicit reasoning capabilities of current Vision-Language Models (VLMs). Implicit reasoning refers to the ability to autonomously discover and utilize hidden visual evidence to bridge information gaps, rather than merely relying on explicitly specified targets. This capacity underlies human visual understanding and everyday reasoning. We argue that this limitation arises from a tendency to approach visual reasoning primarily as passive semantic retrieval, rather than as active, situated reasoning that depends on autonomous visual exploration. As a result, most existing benchmarks primarily assess Passive Capacity, leaving this aspect of reasoning largely unmeasured. To address this gap, we introduce the Visual Implicit Reasoning Diagnosing Benchmark (V-IRD), which targets this missing quadrant by requiring models to derive answers strictly through autonomous visual analysis. Our results show that, despite strong retrieval abilities, prominent VLMs struggle to utilize reference objects and to attend to visual evidence that requires self-directed inquiry. Simply put, strong semantic recognition does not equate to active visual exploration, revealing a critical gap in current VLMs. More information can be found at https://haoychen.github.io/Implicit-Reasoning/

Chinese Translation

本文论证了系统性缺乏能动性限制了当前视觉-语言模型（Vision-Language Models, VLMs）隐性推理能力。隐性推理是指自主发现和利用隐藏视觉证据以弥补信息缺口的能力，而不仅仅依赖于明确指定的目标。这一能力是人类视觉理解和日常推理的基础。我们认为，这一局限性源于将视觉推理主要视为被动的语义检索，而不是依赖自主视觉探索的主动、情境化推理。因此，大多数现有基准主要评估被动能力，导致这一推理方面未被充分测量。为了解决这一空白，我们引入了视觉隐性推理诊断基准（Visual Implicit Reasoning Diagnosing Benchmark, V-IRD），该基准要求模型通过自主视觉分析严格推导答案，从而针对这一缺失的象限。我们的结果显示，尽管具有强大的检索能力，著名的 VLMs 在利用参考对象和关注需要自我引导探究的视觉证据方面表现不佳。简单来说，强大的语义识别并不等同于主动的视觉探索，这揭示了当前 VLMs 中的一个关键缺口。更多信息请访问 https://haoychen.github.io/Implicit-Reasoning/

View on arXiv Download PDF AI Translation

cs.CV / 43 / 2606.14803

HSQ-VLM: A Novel Spatially-Constrained Quadrant Segmentation VLM Model for Explainability in Diabetic Retinopathy

HSQ-VLM：一种用于糖尿病视网膜病变可解释性的新的空间约束象限分割VLM模型

Telang, Shivum

Abstract

Diabetic Retinopathy (DR) is an aggressive retinal disease and a leading cause of global blindness, yet its clinical management is currently hindered by the black-box nature of diagnostic AI. While deep learning models achieve high classification accuracy, there is a critical lack of explainability methods capable of detailing the exact anatomical landmarks and lesion distributions that lead to a clinical decision for DR. Therefore, we propose HSQ-VLM, a novel quadrant segmentation pipeline on fundus images that utilizes a Landmark-Anchored Cartesian Cross-Attention mechanism to unify visual feature extraction with structured clinical reasoning. Unlike traditional methods that rely on arbitrary image partitioning, our pipeline implements 4-quadrant Topological Latent Partitioning (TLP) to dynamically align retinal features with a fovea-centered coordinate system. This allows the Vision-Language Model to generate natural language reports that quantify pathology with anatomical precision. On a dataset of 3,500 high-resolution fundus images, this innovative methodology achieved a lesion detection sensitivity of 99.6% for hemorrhages and 96.4% for microaneurysms, while demonstrating a significant reduction in boundary-ambiguity errors compared to standard segmentation baselines.

Chinese Translation

糖尿病视网膜病变（DR）是一种侵袭性的视网膜疾病，是全球失明的主要原因，但其临床管理目前受到诊断人工智能的黑箱特性限制。尽管深度学习模型在分类准确性上取得了很高的成绩，但缺乏能够详细说明导致DR临床决策的确切解剖标志和病变分布的可解释性方法。因此，我们提出了HSQ-VLM，一种新的基于眼底图像的象限分割管道，利用基于标志的笛卡尔交叉注意机制，将视觉特征提取与结构化临床推理相结合。与依赖任意图像分割的传统方法不同，我们的管道实现了4象限拓扑潜在分割（TLP），以动态对齐视网膜特征与以中心凹为中心的坐标系统。这使得视觉-语言模型能够生成自然语言报告，以解剖学精确度量化病理。在一个包含3500张高分辨率眼底图像的数据集中，这种创新方法在出血的病变检测灵敏度上达到了99.6%，在微动脉瘤的检测灵敏度上达到了96.4%，同时与标准分割基线相比，显著减少了边界模糊错误。

View on arXiv Download PDF AI Translation

cs.CV / 44 / 2606.14811

S23DR 2026: End-to-End 3D Wireframe Prediction via DETR-Style Set Prediction with Contrastive Denoising

S23DR 2026：通过对比去噪的DETR风格集合预测实现端到端的3D线框预测

Khanal, Nitiz

Abstract

We present WireframeDETR, our submission to the Structured Semantic 3D Reconstruction (S23DR) 2026 Challenge, which requires predicting a 3D building wireframe from multi-view COLMAP point clouds. Our method applies DETR-style set prediction directly to 3D point clouds, producing wireframes as sets of edge coordinate pairs without any intermediate vertex detection stage. We introduce three technical contributions: (1) contrastive denoising training that stabilises noisy Hungarian matching in early epochs; (2) a multi-scale encoder that aggregates the last encoder layer outputs via learned scalar weights; and (3) progressive auxiliary loss weighting that concentrates gradient signal on the decoder layers that most benefit from it. Our model achieves a public test HSS of 0.575 (F1~=~0.664, IoU~=~0.516) and a best validation HSS of 0.534 on the cleaned val split.

Chinese Translation

我们提出了WireframeDETR，这是我们对结构化语义3D重建（S23DR）2026挑战的提交，要求从多视角COLMAP点云中预测3D建筑线框。我们的方法直接将DETR风格的集合预测应用于3D点云，生成作为边缘坐标对集合的线框，而无需任何中间的顶点检测阶段。我们引入了三项技术贡献：（1）对比去噪训练，稳定了早期阶段的噪声匈牙利匹配；（2）多尺度编码器，通过学习的标量权重聚合最后一层编码器的输出；（3）渐进辅助损失加权，将梯度信号集中在最受益的解码器层上。我们的模型在公共测试集上取得了0.575的HSS（F1~=~0.664，IoU~=~0.516），在清理后的验证集上取得了最佳验证HSS为0.534。

View on arXiv Download PDF AI Translation

cs.CV / 45 / 2606.14841

Multi-HMR 2: Multi-Person Camera-Centric Human Detection, Mesh Recovery and Tracking

Multi-HMR 2：多人摄像机中心的人体检测、网格恢复与跟踪

Fiche, Guénolé, Weinzaepfel, Philippe, Brégier, Romain, Baradel, Fabien

Abstract

Most advances in human mesh recovery (HMR) have focused on pelvis-centered recovery, overlooking metric 3D localization and detection accuracy in the camera coordinate system - two key factors for real-world applications such as human-robot interaction and social scene understanding. Current evaluation protocols often ignore these aspects, emphasizing per-person, root-centered recovery rather than camera-space perception. As a result, existing approaches rely on fixed camera assumptions or handcrafted post-processing, limiting their robustness and practical deployment. We introduce Multi-HMR 2, a simple yet robust DETR-based framework for Multi-person Camera-centric Human detection, mesh Recovery, and tracking. Multi-HMR 2 predicts a scene-consistent camera together with human meshes, enabling metric 3D localization without ground-truth intrinsics. Moreover, by distilling image-based memory features from SAM2, Multi-HMR 2 extends to tracking, achieving consistent identity association without video supervision. Despite its conceptual simplicity - no handcrafted components, no video input, and no ground-truth cameras - Multi-HMR 2 achieves state-of-the-art pelvis-centered performance while substantially improving detection accuracy and metric 3D localization.

Chinese Translation

大多数人体网格恢复（HMR）的进展集中在以骨盆为中心的恢复，忽视了在摄像机坐标系中的度量3D定位和检测精度这两个关键因素，而这对于人机交互和社会场景理解等现实应用至关重要。目前的评估协议往往忽略这些方面，强调以每个人为单位的根中心恢复，而非摄像机空间的感知。因此，现有方法依赖于固定摄像机假设或手工后处理，限制了其鲁棒性和实际部署能力。我们提出了Multi-HMR 2，这是一个基于DETR的简单而强大的框架，用于多人摄像机中心的人体检测、网格恢复与跟踪。Multi-HMR 2预测一个场景一致的摄像机以及人体网格，从而实现无需真实内参的度量3D定位。此外，通过从SAM2中提取基于图像的记忆特征，Multi-HMR 2扩展到跟踪，实现了一致的身份关联而无需视频监督。尽管其概念简单——没有手工组件、没有视频输入和没有真实摄像机——Multi-HMR 2在骨盆中心性能上达到了最先进的水平，同时显著提高了检测精度和度量3D定位。

View on arXiv Download PDF AI Translation

cs.CV / 46 / 2606.14871

An Ensemble Deep Learning Approach for Reliable and Scalable Lemon Leaf Disease Classification

一种可靠且可扩展的柠檬叶病分类的集成深度学习方法

Abrar, Shayan, Mandal, Sudeepta, Yasir, Abdul Awal, Bhattacharjee, Sonjoy, Bhuiyan, Sadman Haque, Ghosh, Samanta, Ahamed, Rafi

Abstract

Early detection of plant diseases is crucial to plants and for the farmers. Plant diseases reduce fruit yield and quality, and plants are more susceptible to other stresses when they are infected. The lemon leaf disease dataset contains 1354 images. The dataset has 9 classes. Among the 9 classes only one class is for healthy leaf, and the other 8 classes are leaf diseases. The dataset was split into training (70%), testing (15%) and validation (15%) sets after comprehensive preprocessing. Two pretrained models (InceptionV3 and MobileNetV2) were applied and then combined these models using an ensemble technique to boost robustness. Ensemble models showed a promising performance of 99.27% accuracy. Adversarial Training is applied to improve models' ability and ensure reliable predictions under noisy data. Grad-CAM visualization highlights the important regions of leaf images that validate the model prediction with confidence level.

Chinese Translation

植物疾病的早期检测对植物和农民至关重要。植物疾病会降低果实产量和质量，并且在感染后植物更容易受到其他压力的影响。柠檬叶病数据集包含1354张图像，分为9个类别。在这9个类别中，只有一个类别是健康叶片，其他8个类别是叶病。经过全面的预处理后，数据集被划分为训练集（70%）、测试集（15%）和验证集（15%）。采用了两个预训练模型（InceptionV3和MobileNetV2），并使用集成技术将这些模型结合起来以增强鲁棒性。集成模型显示出99.27%的准确率，表现出良好的性能。对抗训练被应用于提高模型的能力，并确保在噪声数据下的可靠预测。Grad-CAM可视化突出了叶片图像中重要的区域，以验证模型预测的置信水平。

View on arXiv Download PDF AI Translation

cs.CV / 47 / 2606.14883

Understanding Cross-Modal Contributions in Continual Vision-Language Models: A Theoretical Perspective

理解持续视觉-语言模型中的跨模态贡献：一种理论视角

Sekeh, Salimeh, Wisell, Mary

Abstract

Continual vision-language models are commonly addressed through sequential fine-tuning; however, although this paradigm enables adaptation to new environments (tasks), it inherently emphasizes the contribution of previously learned environments (tasks) at the expense of the stability required to preserve previously acquired knowledge. While existing approaches have adequately studied continual learning and catastrophic forgetting in vision-language models (VLMs), the theoretical understanding of modality-specific contributions across a sequence of environments remains largely unexplored. In this paper, we present a new theoretical perspective to understand the cross-modal (vision-language) contributions to consecutive environments. We empirically evaluate our theoretical findings on large VLMs and demonstrate their effectiveness in capturing environment-level cross-modal contributions. Our analysis provides deeper insights into continual VLMs, highlighting their contribution robustness to varying task orders and inter-task similarities, and their improved generalization performance.

Chinese Translation

持续视觉-语言模型通常通过顺序微调来处理；然而，尽管这一范式能够适应新环境（任务），但它在本质上强调了先前学习的环境（任务）的贡献，这牺牲了保持先前获得知识所需的稳定性。虽然现有方法已充分研究了视觉-语言模型（VLMs）中的持续学习和灾难性遗忘，但对一系列环境中模态特定贡献的理论理解仍然 largely 未被探索。在本文中，我们提出了一种新的理论视角，以理解连续环境中的跨模态（视觉-语言）贡献。我们在大型 VLMs 上实证评估了我们的理论发现，并展示了它们在捕捉环境级跨模态贡献方面的有效性。我们的分析提供了对持续 VLMs 的更深入见解，突出了它们对不同任务顺序和任务间相似性的贡献鲁棒性，以及它们改善的泛化性能。

View on arXiv Download PDF AI Translation

cs.CV / 48 / 2606.14886

Improved Knowledge Distillation for Land-Use Image Classification

改进的知识蒸馏用于土地利用图像分类

Sur, Arundhuti, Chatterjee, Abhiroop, Ghosh, Susmita, Ientilucci, Emmett

Abstract

In the present article, an improved Knowledge Distillation (KD) framework has been proposed for efficient compression of deep convolutional neural networks for land-use image classification task. Motivated by the need to achieve competitive classification accuracy while reducing computational complexity, a teacher-student learning paradigm is adopted in which a VGG16 network transfers knowledge to a lightweight MobileNetV2 model. The proposed framework integrates hard supervision from ground truth labels with a soft supervision strategy that combines Kullback-Leibler divergence and Cosine Similarity losses. Experiments conducted on three land-use datasets show that the proposed KD-based method yields improved performance, and achieves an accuracy of 99.04%, outperforming both baseline student training and single-loss distillation approaches, while retaining substantial model compression.

Chinese Translation

在本文中，提出了一种改进的知识蒸馏（Knowledge Distillation, KD）框架，用于高效压缩深度卷积神经网络以完成土地利用图像分类任务。基于在降低计算复杂度的同时实现竞争性分类精度的需求，采用了教师-学生学习范式，其中VGG16网络将知识传递给轻量级的MobileNetV2模型。所提出的框架将来自真实标签的硬监督与结合了Kullback-Leibler散度和余弦相似度损失的软监督策略相结合。对三个土地利用数据集进行的实验表明，所提出的基于KD的方法表现出更好的性能，达到了99.04%的准确率，超越了基线学生训练和单损失蒸馏方法，同时保持了显著的模型压缩。

View on arXiv Download PDF AI Translation

cs.CV / 49 / 2606.14905

Deep Learning in Seismic Interpretation: Federated Advances in Salt Dome Segmentation

深度学习在地震解释中的应用：盐丘分割的联邦进展

Mehdi, Muhammad Zain, Zaid, Muhammad, Aleem, Owais

Abstract

Salt-dome delineation is a critical, high-impact task in subsurface geological interpretation, driving decisions in hydrocarbon exploration, reservoir modeling, and drilling safety. While convolutional encoder-decoder architectures have delivered significant improvements in automated salt segmentation, their widespread application is severely limited by data sovereignty concerns, dataset bias, and the scarcity of labeled seismic volumes. This paper introduces FedSaltNet, a Federated Learning (FL) framework explicitly engineered for robust, generalizable, and privacy preserving salt-dome segmentation. We couple a lightweight Small U-Net backbone, chosen for its efficiency and regularization properties with a novel Foreground-Weighted (FG-WEIGHTED) aggregation strategy designed to tackle domain-specific class imbalance. Through an extensive comparative study emulating non-IID conditions across four diverse seismic datasets (TGS, SEAM, F3, GBS), we demonstrate two critical findings: The FG-WEIGHTED algorithm effectively mitigates data heterogeneity, yielding a 4.0% relative improvement in Intersection over Union (IoU) over the best conventional FL method. The simple U-Net architecture proved essential, outperforming the higher capacity ResNet-18 U-Net variant by 166% in average IoU, underscoring the necessity of architectural simplicity in data-constrained federated environments. FedSaltNet provides a validated, high-performance solution that establishes the viability of federated deep learning for collaborative, next-generation subsurface interpretation.

Chinese Translation

盐丘划分是地下地质解释中的一项关键且高影响力的任务，直接影响到碳氢化合物勘探、储层建模和钻井安全。尽管卷积编码器-解码器架构在自动盐分割方面取得了显著进展，但由于数据主权问题、数据集偏差以及标记地震体的稀缺，其广泛应用受到严重限制。本文介绍了FedSaltNet，一个专门为稳健、可泛化和保护隐私的盐丘分割而设计的联邦学习（Federated Learning, FL）框架。我们结合了一种轻量级的小型U-Net骨干网络，因其高效性和正则化特性而被选用，并采用了一种新颖的前景加权（Foreground-Weighted, FG-WEIGHTED）聚合策略，旨在解决特定领域的类别不平衡问题。通过对四个不同地震数据集（TGS、SEAM、F3、GBS）模拟非独立同分布（non-IID）条件进行广泛的比较研究，我们展示了两个关键发现：FG-WEIGHTED算法有效缓解了数据异质性，相较于最佳传统FL方法在交并比（Intersection over Union, IoU）上实现了4.0%的相对提升。简单的U-Net架构被证明至关重要，其在平均IoU上超越了更高容量的ResNet-18 U-Net变体，提升幅度达166%，强调了在数据受限的联邦环境中架构简单性的必要性。FedSaltNet提供了一个经过验证的高性能解决方案，确立了联邦深度学习在协作、下一代地下解释中的可行性。

View on arXiv Download PDF AI Translation

cs.CV / 50 / 2606.14912

Mask Proposal Voting Based on Geodesic Framework for Robust Image Segmentation

基于测地框架的掩膜提议投票方法用于鲁棒图像分割

Liu, Li, Wang, Mingzhu, Li, Zhenjiang, Chen, Da, Cohen, Laurent D.

Abstract

Despite great advances, finding accurate segmentation remains a challenging task, especially in scenarios with cluttered backgrounds, complex intensity variations and topology appearance. Minimal path models have exhibited their strong ability in addressing image segmentation tasks. However, the performance of minimal paths-based segmentation approaches is heavily influenced by model initialization, hence limiting their application scope in practice. In this work, we propose a novel mask proposal voting framework that overcomes the major drawback of classical approaches, allowing robust segmentation even in complicated scenarios. Firstly, we introduce an efficient method for constructing adaptive domain cuts as a constraint for initializing the region-based min-cut evolution, by which diverse and reliable mask proposal candidates can be generated, substantially increasing the possibility of accurately covering the objective region by these proposals. Secondly, we propose a new mask voting scheme to build a voting score map encoding the final segmentation information. In contrast to classical path voting methods, our model allows incorporating priors to assign different importance to each individual mask. As a consequence, the proposed segmentation model is capable of accurately delineating object boundaries under complex scenarios, and is insensitive to initialization. Experiments demonstrate that our method consistently outperforms state-of-the-art minimal path-based approaches in both accuracy and robustness.

Chinese Translation

尽管取得了重大进展，准确的分割仍然是一项具有挑战性的任务，尤其是在背景杂乱、强度变化复杂和拓扑外观多样的场景中。最小路径模型在处理图像分割任务方面表现出了强大的能力。然而，基于最小路径的分割方法的性能受到模型初始化的严重影响，从而限制了其在实际中的应用范围。在本研究中，我们提出了一种新颖的掩膜提议投票框架，克服了经典方法的主要缺陷，使得即使在复杂场景中也能实现鲁棒分割。首先，我们引入了一种高效的方法来构建自适应域切割，作为初始化基于区域的最小切割演化的约束，从而生成多样且可靠的掩膜提议候选，显著提高了这些提议准确覆盖目标区域的可能性。其次，我们提出了一种新的掩膜投票方案，构建一个投票得分图，编码最终的分割信息。与经典路径投票方法相比，我们的模型允许结合先验知识，为每个单独的掩膜分配不同的重要性。因此，所提出的分割模型能够在复杂场景下准确勾勒物体边界，并且对初始化不敏感。实验表明，我们的方法在准确性和鲁棒性方面始终优于最先进的基于最小路径的方法。

View on arXiv Download PDF AI Translation

cs.CV / 51 / 2606.14926

FlexPooling with Simple Auxiliary Classifiers in Deep Networks

深度网络中的简单辅助分类器与FlexPooling

Ali, Muhammad, Alsuwaidi, Omar, Khan, Salman

Abstract

In computer vision, the basic pipeline of most convolutional neural networks consists of multiple feature extraction layers, where the input signal is downsampled to a lower resolution in each subsequent layer. This downsampling process is commonly referred to as pooling, which is an essential operation in CNNs. Pooling improves robustness against transformations, reduces the number of trainable parameters, increases the receptive field, and lowers computation time. Since pooling is a lossy process but remains important for extracting high-level information from low-level representations, it is important to preserve the most prominent information from previous activations to improve network discriminability. Standard pooling is usually performed using dense pooling methods, such as max pooling or average pooling, or through strided convolutional kernels. In this paper, we propose a simple yet effective adaptive pooling method, called FlexPooling, which generalizes average pooling by learning a weighted average over activations jointly with the rest of the network. We further show that attaching Simple Auxiliary Classifiers (SAC) to the CNN improves performance and demonstrates the effectiveness of the proposed method compared with standard pooling methods. Experiments on multiple popular image classification datasets show that FlexPooling consistently outperforms baseline networks, achieving approximately 1 to 3 percent improvement in accuracy.

Chinese Translation

在计算机视觉中，大多数卷积神经网络的基本流程由多个特征提取层组成，其中输入信号在每个后续层中被下采样到较低的分辨率。这个下采样过程通常被称为池化，是卷积神经网络中的一个重要操作。池化提高了对变换的鲁棒性，减少了可训练参数的数量，增加了感受野，并降低了计算时间。由于池化是一个有损过程，但对于从低级表示中提取高级信息仍然至关重要，因此保留先前激活中最显著的信息以提高网络的可区分性是非常重要的。标准池化通常使用密集池化方法进行，例如最大池化（max pooling）或平均池化（average pooling），或通过步幅卷积核进行。在本文中，我们提出了一种简单而有效的自适应池化方法，称为FlexPooling，它通过与网络的其余部分共同学习激活的加权平均来推广平均池化。我们进一步展示了将简单辅助分类器（Simple Auxiliary Classifiers, SAC）附加到卷积神经网络上可以提高性能，并证明了所提方法相较于标准池化方法的有效性。在多个流行的图像分类数据集上的实验表明，FlexPooling始终优于基线网络，准确率提高约1%到3%。

View on arXiv Download PDF AI Translation

cs.CV / 52 / 2606.14957

Learning Sparse Latent Predictive Foundation Model for Multimodal Neuroimaging

学习稀疏潜在预测基础模型用于多模态神经成像

Huang, Haoxu, Chen, Long, Chen, Jingyun, Hyun, Jinu, Loftus, James Ryan, Melmed, Kara, Orringer, Daniel, Frontera, Jennifer, Dehkharghani, Seena, Masurkar, Arjun, Razavian, Narges

Abstract

Brain MRIs are routinely acquired as multiple complementary sequences with unique contrast weighting, including T1-weighed imaging (T1w) anatomic and fluid-sensitive T2-weighted (T2w) contrasts. However, methods for learning unified representations across the multitude of MRI contrast mechanisms at health-system scale are lacking. In this study, we introduce Neuro-JEPA, a sparse multimodal neuroimaging foundation model that combines a latent predictive objective with a Mixture-of-Experts architecture to encode brain MRI across core T1w, T2w, and fluid-suppressed FLAIR imaging (FLAIR). We further provide a systematic methodological study of architectural, masking, objective, and sparsity design choices beneficial for robust neuroimaging multimodal representation learning. Neuro-JEPA was pretrained on 1,551,862 scans from 428,647 studies after modality-specific preprocessing with data curation across three core structural brain MRI sequences. We evaluated the learned representations across clinical and research settings, including 25 tasks from three health systems: NYU Langone, NYU Long Island, and Massachusetts General Hospital, and 22 tasks from 12 public datasets, covering unimodal, multimodal and cross-domain evaluation configurations. Across these benchmarks, existing neuroimaging foundation models showed inconsistent gains over a simple convolutional neural network (CNN) baseline, whereas Neuro-JEPA achieved stronger and more consistent performance across all evaluated settings. These results establish a scalable methodological framework for multimodal neuroimaging representation learning and highlight the need for foundation model evaluation protocols that include simple baselines, clinically heterogeneous cohorts and controlled multimodal comparisons.

Chinese Translation

脑部MRI通常以多种互补序列获取，这些序列具有独特的对比加权，包括T1加权成像（T1w）解剖图像和流体敏感的T2加权成像（T2w）对比。然而，目前缺乏在健康系统规模上学习统一表示的方法，以整合多种MRI对比机制。在本研究中，我们介绍了Neuro-JEPA，这是一种稀疏多模态神经成像基础模型，结合了潜在预测目标与专家混合（Mixture-of-Experts）架构，以编码核心的T1w、T2w和流体抑制的FLAIR成像（FLAIR）脑部MRI。我们进一步提供了系统的方法论研究，探讨了有助于稳健的神经成像多模态表示学习的架构、掩蔽、目标和稀疏性设计选择。Neuro-JEPA在经过模态特定预处理和数据整理的428,647个研究中的1,551,862个扫描上进行了预训练，涵盖了三种核心结构脑部MRI序列。我们在临床和研究环境中评估了学习到的表示，包括来自三个健康系统的25个任务：NYU Langone、NYU Long Island和麻省总医院，以及来自12个公共数据集的22个任务，涵盖了单模态、多模态和跨领域评估配置。在这些基准测试中，现有的神经成像基础模型在简单卷积神经网络（CNN）基线上的增益不一致，而Neuro-JEPA在所有评估设置中实现了更强和更一致的性能。这些结果建立了一个可扩展的方法论框架，用于多模态神经成像表示学习，并强调了基础模型评估协议的必要性，包括简单基线、临床异质性队列和受控的多模态比较。

View on arXiv Download PDF AI Translation

cs.CV / 53 / 2606.14958

MVEB: Massive Video Embedding Benchmark

MVEB：大规模视频嵌入基准

Assadi, Adnan El, Solomatin, Roman, Chung, Isaac, Xiao, Chenghao, Shah, Deep, Dey, Manan, Sudhakar, Shriya, Bugaud, Zacharie, Siblini, Wissam, Munot, Ayush Sunil, Devavarapu, Yashwanth, Ireddi, Rakshitha, Yang, Michelle, Kardos, Márton, Muennighoff, Niklas, Enevoldsen, Kenneth

Abstract

We introduce the Massive Video Embedding Benchmark (MVEB), a 23-task benchmark for video embeddings spanning classification, zero-shot classification, clustering, pair classification, retrieval, and video-centric question answering. We evaluate 33 models and find that no single model dominates: MLLM-based embeddings lead on classification, clustering, pair classification, and QA; multimodal binding leads on retrieval and zero-shot classification; generative MLLMs without contrastive adaptation collapse on cross-modal tasks. Paired video-only vs. audio+video evaluations show that audio's contribution depends on dataset annotation provenance: audio helps when labels were produced from both modalities and hurts when they were produced from visuals alone, a six-point gap consistent across model families. MVEB is derived from MVEB+, a 184-task pool, and is designed to maintain task diversity while reducing evaluation cost. It integrates into the MTEB ecosystem for unified evaluation across text, image, audio, and video. We release MVEB and all 184 tasks along with code and a leaderboard at https://github.com/embeddings-benchmark/mteb.

Chinese Translation

我们介绍了大规模视频嵌入基准（MVEB），这是一个涵盖分类、零样本分类、聚类、配对分类、检索和视频中心问答的23项任务基准。我们评估了33个模型，发现没有单一模型占主导地位：基于MLLM的嵌入在分类、聚类、配对分类和问答方面表现优异；多模态绑定在检索和零样本分类方面表现突出；而没有对比适应的生成型MLLM在跨模态任务上表现不佳。配对的视频与音频+视频评估表明，音频的贡献取决于数据集注释的来源：当标签是从两种模态生成时，音频有助于性能，而当标签仅由视觉生成时，音频则会造成负面影响，这一六分差距在不同模型系列中保持一致。MVEB源自MVEB+，这是一个包含184项任务的池，旨在保持任务多样性的同时降低评估成本。它集成到MTEB生态系统中，实现文本、图像、音频和视频的统一评估。我们在https://github.com/embeddings-benchmark/mteb上发布了MVEB及所有184项任务的代码和排行榜。

View on arXiv Download PDF AI Translation

cs.CV / 54 / 2606.14963

Multi-Modal Attention for Automated Disaster Damage Assessment Using Remote Sensing Imagery and Deep Learning

基于多模态注意力的自动化灾害损害评估：利用遥感影像和深度学习

Gebre, Tewodros Syum, Talreja, Jagrati, Hashemi-Beni, Leila

Abstract

Timely and accurate disaster damage assessment is crucial for effective emergency response, resource allocation, and recovery. Traditional methods, which often rely on manual inspections or sparse data, are typically slow and error-prone. This paper introduces a novel framework leveraging remote sensing imagery and deep learning to automate building damage classification. Using pre- and post-disaster satellite imagery, our model categorizes buildings into four damage levels: no damage, minor damage, major damage, and destroyed. The core innovation is a multi-modal attention mechanism that fuses bi-temporal features to explicitly detect and assess structural changes. We employ a lightweight ConvNeXT-Tiny backbone to ensure efficient processing without compromising performance. Key contributions include: (1) a cross-attention module for multi-modal data fusion, (2) an optimized preprocessing pipeline for large-scale datasets, and (3) robust data augmentation techniques. Experiments on a large-scale disaster dataset demonstrate an overall classification accuracy of 94.90%. The model effectively discriminates between damage categories and remains resilient to incomplete data. This system significantly improves assessment speed and accuracy, aiding emergency responders in prioritizing interventions. This work advances automated disaster damage detection by integrating multi-temporal imagery with deep learning, offering a scalable solution for real-time response.

Chinese Translation

及时准确的灾害损害评估对于有效的应急响应、资源分配和恢复至关重要。传统方法通常依赖于人工检查或稀疏数据，通常速度较慢且容易出错。本文提出了一种新颖的框架，利用遥感影像和深度学习自动化建筑损害分类。通过使用灾前和灾后卫星影像，我们的模型将建筑物分类为四个损害等级：无损害、轻微损害、重大损害和完全毁坏。核心创新在于一个多模态注意力机制，该机制融合了双时态特征，以明确检测和评估结构变化。我们采用轻量级的ConvNeXT-Tiny主干网络，以确保高效处理而不影响性能。主要贡献包括：(1) 用于多模态数据融合的交叉注意力模块，(2) 针对大规模数据集的优化预处理管道，以及(3) 强大的数据增强技术。在大规模灾害数据集上的实验表明，整体分类准确率达到94.90%。该模型有效区分损害类别，并对不完整数据保持鲁棒性。该系统显著提高了评估速度和准确性，帮助应急响应者优先处理干预任务。本研究通过将多时相影像与深度学习相结合，推动了自动化灾害损害检测的发展，为实时响应提供了可扩展的解决方案。

View on arXiv Download PDF AI Translation

cs.CV / 55 / 2606.14972

ReGenHuman: Re-Generating Human Appearances for Realistic Full-Body Video Anonymization

ReGenHuman：为真实感全身视频匿名化重新生成人类外观

Sun, Adam, Barkataki, Eshaan, Milstein, Arnold, Wetzstein, Gordon, Adeli, Ehsan

Abstract

Anonymizing human-centric video data is an understudied problem. Prior anonymization techniques either blur or redact pixels at the cost of realism and downstream utility, or generate frame-by-frame at the cost of temporal coherence. We introduce ReGenHuman, the first full-body video anonymization pipeline that is simultaneously realistic, temporally consistent, and anonymous by construction. Contrary to past approaches which redact or edit the inputs directly, we propose a regenerate, don't edit paradigm. Our approach composites 2D pose, segmentation, and monocular depth into two complementary conditioning streams - StructAll and StructHuman, which are used to fine-tune a video-to-video diffusion backbone on in-the-wild human videos, synthesizing the human regions entirely from identity-free structural cues. We evaluate our model on privacy, quality, and utility, and show that our ReGenHuman achieves the best tradeoff across all three axes against current baselines. We further show that our anonymized videos remain effective for downstream tasks, including video question answering.

Chinese Translation

人类中心的视频数据匿名化是一个尚未得到充分研究的问题。以往的匿名化技术要么模糊或删除像素，从而牺牲了真实感和后续实用性，要么逐帧生成，导致时间一致性不足。我们提出了ReGenHuman，这是第一个同时在构造上实现真实感、时间一致性和匿名化的全身视频匿名化管道。与过去直接删除或编辑输入的方式相反，我们提出了一种“再生成，而非编辑”的范式。我们的方法将2D姿态、分割和单目深度合成到两个互补的条件流——StructAll和StructHuman，这些流用于对在自然环境中拍摄的人类视频进行视频到视频的扩散基础模型的微调，完全从无身份的结构线索合成人体区域。我们在隐私、质量和实用性方面评估了我们的模型，并显示出ReGenHuman在这三个维度上相较于当前基线达到了最佳的权衡。我们进一步展示了我们的匿名化视频在后续任务中仍然有效，包括视频问答。

View on arXiv Download PDF AI Translation

cs.CV / 56 / 2606.15015

NEXUS: Neural Energy Fields for Physically Consistent Contact-Rich 3D Object Dynamics

NEXUS：用于物理一致的接触丰富3D物体动态的神经能量场

Ying, Qizhen, Wang, Guangming, Pan, Yangchen, Prisacariu, Victor Adrian, Jing, Yixiong

Abstract

Physics-grounded video generation requires controllable 3D object dynamics that remain physically consistent under contact, deformation, and external forcing. Existing trajectory-based methods often model isolated physical effects, making it difficult to compose conservative and non-conservative dynamics in contact-rich 3D scenes. We present NEXUS, a neural energy-field framework for contact-rich 3D object dynamics. NEXUS represents each object as a structural graph and constructs dynamic object-object and object-environment contact graphs. Inspired by Hamiltonian Neural Networks, NEXUS formulates motion through scalar energy and dissipation terms rather than directly predicting states or accelerations. Conservative effects, including gravity and elastic deformation, are composed as additive energy terms, while non-conservative effects such as damping and impact-induced energy loss are modeled with learned Rayleigh-style dissipation. Forces are derived by differentiating the energy and dissipation functions and rolled out with a multi-substep semi-implicit integrator. Across controlled trajectory benchmarks, NEXUS improves long-horizon accuracy over representative learned and physics-structured dynamics baselines under varying mechanical properties and physical-effect compositions. We further show that NEXUS trajectories provide effective guidance for contact-rich video generation, improving physical plausibility while maintaining competitive visual quality.

Chinese Translation

基于物理的视频生成需要可控的3D物体动态，这些动态在接触、变形和外部作用下保持物理一致性。现有的基于轨迹的方法通常建模孤立的物理效应，使得在接触丰富的3D场景中组合保守和非保守动态变得困难。我们提出了NEXUS，一个用于接触丰富3D物体动态的神经能量场框架。NEXUS将每个物体表示为一个结构图，并构建动态的物体-物体和物体-环境接触图。受到哈密顿神经网络的启发，NEXUS通过标量能量和耗散项来表述运动，而不是直接预测状态或加速度。保守效应，包括重力和弹性变形，被组合为附加能量项，而非保守效应如阻尼和冲击引起的能量损失则通过学习的Rayleigh风格耗散进行建模。力是通过对能量和耗散函数进行微分得到的，并通过多子步半隐式积分器进行展开。在受控轨迹基准测试中，NEXUS在不同机械属性和物理效应组合下，提升了长时间范围的准确性，相较于代表性的学习和物理结构动态基线表现更佳。我们进一步展示了NEXUS轨迹为接触丰富的视频生成提供了有效的指导，提高了物理合理性，同时保持了竞争性的视觉质量。

View on arXiv Download PDF AI Translation

cs.CV / 57 / 2606.15019

Towards Global AI-Driven Cervical Cancer Screening

迈向全球人工智能驱动的宫颈癌筛查

Tran, Thuy Nuong, Sümer, Ömer, Christodoulou, Evangelia, Nauschütte, Lennart, Kalteis, Simon, Paulikat, Martin, Pashayeva, Esmira, Steinheuer, Klara, Borges, Isabella, Kalinowski, Piotr, Bussmann, Hermann, Sokmney, Sieng, Kuong, Poeung, Vong, Sathiarany, Schneider, Achim, von Knebel-Doeberitz, Magnus, Godau, Patrick, Maier-Hein, Lena

Abstract

The global elimination of cervical cancer is a key public health goal set by the World Health Organization (WHO), with screening programs reducing mortality by up to 80%. However, access to experts and biopsy services is limited in low- to middle-income countries (LMICs). Deep learning (DL)-based algorithms offer promising support for screening, but most existing approaches have been developed and validated on private datasets from single countries. We present the first DL-based approach to cervical cancer screening validated on data from multiple countries. Technically, we phrase the problem of detecting and classifying lesions in colposcopy images as a multi-task learning problem, in which we simultaneously perform image-level classification and lesion segmentation. Our model was trained on a private data set of acid stain colposcopy images with manually generated lesion segmentation masks and corresponding histopathological results, employing extensive data augmentation to address image variability. In an in-distribution validation with pathology results serving as ground truth, our algorithm outperformed medical experts (Balanced Accuracy: 0.68 vs 0.64) in CIN1- (Cervical intraepithelial neoplasia grade 1 or lower) versus CIN2+ (grade 2 or higher) classification. External validation on four colposcopy data sets from four countries featuring radical differences in prevalence and patient characteristics yielded superior performance of our method compared to baseline methods. Performance variability across countries was high with AUC values ranging from 0.54 - 0.80. Overall, algorithm performance varied with age, transformation zone (cervical area most prone to lesion development), presence of comorbidities and pathognomonic signs, with comorbidities having by far the largest negative effect. Future work should focus on improving model robustness and generalizability.

Chinese Translation

全球消除宫颈癌是世界卫生组织（WHO）设定的关键公共卫生目标，筛查程序可将死亡率降低多达80%。然而，在中低收入国家（LMICs），专家和活检服务的获取受到限制。基于深度学习（DL）的算法为筛查提供了有希望的支持，但大多数现有方法是在单一国家的私有数据集上开发和验证的。我们提出了首个基于深度学习的宫颈癌筛查方法，该方法在多个国家的数据上进行了验证。从技术上讲，我们将检测和分类阴道镜图像中的病变的问题表述为多任务学习问题，在该问题中，我们同时执行图像级分类和病变分割。我们的模型在一个包含手动生成的病变分割掩膜和相应组织病理学结果的酸染阴道镜图像私有数据集上进行了训练，采用了广泛的数据增强以应对图像变异性。在使用病理结果作为真实值的分布内验证中，我们的算法在CIN1（宫颈上皮内瘤变1级或更低）与CIN2+（2级或更高）分类中超越了医学专家（平衡准确率：0.68对0.64）。在来自四个国家的四个阴道镜数据集上的外部验证显示，我们的方法相比基线方法表现优越。不同国家之间的性能变异性较高，AUC值范围为0.54至0.80。总体而言，算法性能因年龄、转化区（最易发生病变的宫颈区域）、合并症的存在以及特征性体征而异，其中合并症对性能的负面影响最大。未来的工作应集中在提高模型的鲁棒性和泛化能力上。

View on arXiv Download PDF AI Translation

cs.CV / 58 / 2606.15049

Gaussian Spatial Priors for Anatomy-Aware Object Detection in Surgical Videos

用于手术视频中解剖结构感知物体检测的高斯空间先验

Li, Yunfan, Shmelev, Artem, Gupta, Himanshu

Abstract

Detecting anatomical structures in surgical video is essential for intraoperative safety frameworks such as the Critical View of Myopectineal Orifice (CVMPO) in inguinal hernia repair. While prominent structures like the Cooper's Ligament and Triangle of Doom are reliably detected by standard methods, smaller structures such as the epigastric vessels remain challenging due to their visual ambiguity and intermittent visibility. We observe that the spatial relationship between structures is anatomically constrained, and propose a Gaussian Spatial Prior (GSP) module that encodes this relationship as a compact, parametric bias injected into the self-attention of a DAB-DETR decoder. The prior is computed offline from training annotations as a small set of frozen Gaussian parameters and recomputed at each decoder layer using the iteratively refined reference points. On a dataset of inguinal hernia repair videos with 5-fold cross-validation, GSP improves dependent class detection by $+33.5\%$ ($\text{AP}_{50}$) over DAB-DETR and $+53.9\%$ over YOLOv26, while also improving anchor detection by $+6.0\%$. These gains are statistically significant across all folds ($p=0.012$, paired $t-$test).

Chinese Translation

在手术视频中检测解剖结构对于诸如腹股沟疝修复中的关键视图（Critical View of Myopectineal Orifice, CVMPO）等术中安全框架至关重要。尽管像库珀韧带（Cooper's Ligament）和死亡三角（Triangle of Doom）等显著结构可以通过标准方法可靠检测，但小型结构如上腹壁血管（epigastric vessels）由于其视觉模糊性和间歇性可见性仍然具有挑战性。我们观察到结构之间的空间关系在解剖上是受限的，并提出了一种高斯空间先验（Gaussian Spatial Prior, GSP）模块，该模块将这种关系编码为注入到DAB-DETR解码器自注意力中的紧凑参数偏置。该先验是从训练标注中离线计算得到的一小组固定高斯参数，并在每个解码器层使用迭代优化的参考点重新计算。在一个包含腹股沟疝修复视频的数据集上进行5折交叉验证，GSP在依赖类检测上比DAB-DETR提高了$+33.5 ext{AP}_{50}$，比YOLOv26提高了$+53.9 ext{AP}_{50}$，同时锚点检测也提高了$+6.0 ext{AP}_{50}$。这些提升在所有折中均具有统计显著性（$p=0.012$，配对$t$检验）。

View on arXiv Download PDF AI Translation

cs.CV / 59 / 2606.15055

Bridging Geographic Bias in Urban Streetscape Inference via Lifelong Learning with Visual-Semantic Pivoting

通过视觉-语义枢轴的终身学习弥补城市街景推断中的地理偏差

Zhang, Xinze

Abstract

Visual perception of urban streetscapes underpins evidence-based decisions in landscape planning, public health, and place-making. Yet models trained on a few well-photographed metropolises systematically misjudge underrepresented districts, propagating geographic bias into downstream policy. We address this gap with HVSP-LL, a lifelong learning framework that couples a stratified visual-semantic pivoting module with an equity-aware rehearsal mechanism. The pivoting module organises landscape concepts along a three-tier ontology (macro structure, meso composition, micro element) and aligns image features to learnable semantic anchors at each tier, providing transferable representations that resist distributional drift. The lifelong adaptation component sequentially absorbs new urban regions while constraining inter-region perception gaps through a worst-region sample-reweighting objective and a structurally-aware exemplar buffer. We evaluate HVSP-LL on a panoramic streetscape benchmark assembled from twelve cities across four continents and seven perceptual dimensions. The framework attains 0.834 Spearman correlation on the held-out city sequence, an absolute 6.1 point improvement over the strongest continual baseline, and shrinks the inter-city perception gap to 0.094 -- a 38% reduction relative to the strongest continual baseline (0.151) and a 57% reduction relative to a representative regularisation baseline (0.218). Ablations confirm that each tier of the pivoting hierarchy contributes monotonically, and the equity-aware rehearsal converts mean backward transfer from -0.038 (without retention) to +0.013, eliminating catastrophic forgetting on the held-out sequence. Our results indicate that hierarchical anchoring is a practical pathway toward geographically equitable streetscape inference at city scale.

Chinese Translation

城市街景的视觉感知是景观规划、公共健康和场所营造中基于证据决策的基础。然而，在少数拍摄良好的大都市上训练的模型系统性地错误评估了代表性不足的地区，将地理偏差传播到下游政策中。我们通过HVSP-LL解决了这一问题，这是一个将分层视觉-语义枢轴模块与关注公平的重演机制相结合的终身学习框架。该枢轴模块沿着三层本体（宏观结构、中观组成、微观元素）组织景观概念，并将图像特征与每一层的可学习语义锚点对齐，提供抗分布漂移的可转移表示。终身适应组件顺序吸收新的城市区域，同时通过最差区域样本重加权目标和结构感知示例缓冲区来限制区域间的感知差距。我们在由四大洲十二个城市和七个感知维度组成的全景街景基准上评估HVSP-LL。该框架在保留的城市序列上达到了0.834的斯皮尔曼相关系数，相较于最强的持续基线绝对提高了6.1分，并将城市间感知差距缩小至0.094——相较于最强的持续基线（0.151）减少了38%，相较于具有代表性的正则化基线（0.218）减少了57%。消融实验确认了枢轴层级的每一层都单调贡献，并且关注公平的重演将平均向后转移从-0.038（无保留）转变为+0.013，消除了在保留序列上的灾难性遗忘。我们的结果表明，分层锚定是实现城市规模地理公平街景推断的有效途径。

View on arXiv Download PDF AI Translation

cs.CV / 60 / 2606.15072

Texture-Shape Bias Balancing for Robust Synthetic-to-Real Semantic Segmentation in Automotive NIR Imagery

汽车近红外图像中稳健的合成到真实语义分割的纹理-形状偏差平衡

Stillger, Felix, Hamscher, Ben, Hahn, Lukas, Mütze, Annika, Meisen, Tobias, Maag, Kira

Abstract

Semantic segmentation is a fundamental component of visual perception in modern automotive systems, enabling pixel-level scene understanding. Near-Infrared imaging (NIR) offers stable detection under difficult illumination conditions, but the development of domain-specific semantic segmentation models remains challenging due to the lack of high-quality annotated data from real-world scenarios. Synthetic datasets offer a scalable alternative, but models trained on synthetic images often suffer performance degradation when transferred to real domains. We present the first systematic study on synthetic to real domain adaptation for semantic segmentation in NIR images in the automotive domain. We propose a generative augmentation framework that transforms synthetic images into realistic NIR-style variants via our introduced target style adaptation (TSA). TSA fine-tunes a latent diffusion model via low-rank adaptation on a small curated set of real NIR images and applies it to synthetic training data using structure-preserving multi-signal conditioning. To reduce texture bias and improve segmentation robustness, we further apply a Voronoi-based style diversification strategy (VSD) that modifies the original textures while preserving scene geometry. Experiments with multiple model architectures on NIR data from vehicle interiors and street scenes show that balancing inductive bias during training leads to noticeably more robust semantic segmentation and effectively reduces the domain gap in our real-world scenarios by up to 63.6% on exterior and 28.4% on interior data. The code is available at GitHub.

Chinese Translation

语义分割是现代汽车系统视觉感知的基本组成部分，能够实现像素级场景理解。近红外成像（NIR）在困难的照明条件下提供了稳定的检测，但由于缺乏来自真实场景的高质量标注数据，开发特定领域的语义分割模型仍然具有挑战性。合成数据集提供了一种可扩展的替代方案，但在真实领域中转移时，基于合成图像训练的模型往往会遭遇性能下降。我们首次系统性地研究了汽车领域中近红外图像的合成到真实领域适应的语义分割。我们提出了一种生成增强框架，通过我们引入的目标风格适应（TSA）将合成图像转换为逼真的近红外风格变体。TSA通过对一小组精心挑选的真实近红外图像进行低秩适应，微调潜在扩散模型，并使用结构保持的多信号调节将其应用于合成训练数据。为了减少纹理偏差并提高分割的稳健性，我们进一步应用了一种基于Voronoi的风格多样化策略（VSD），该策略在保持场景几何形状的同时修改原始纹理。对来自车辆内部和街景的近红外数据进行的多种模型架构实验表明，在训练过程中平衡归纳偏差显著提高了语义分割的稳健性，并有效减少了我们真实场景中的领域差距，外部数据减少了高达63.6%，内部数据减少了28.4%。代码可在GitHub上获取。

View on arXiv Download PDF AI Translation

cs.CV / 61 / 2606.15099

Think Less, Act Early: Reinforced Latent Reasoning with Early Exit in Vision-Language-Action Models

少思考，早行动：具有早期退出机制的强化潜在推理视觉-语言-行动模型

Lei, Dianqiao, Shan, Lianlei

Abstract

Existing Vision-Language-Action (VLA) models predominantly rely on explicit Chain-of-Thought (CoT) reasoning to bridge perception and action. While effective, this paradigm suffers from high computational costs and error propagation in multi-step tasks. In this paper, we propose Adaptive Variable Alignment VLA (AVA-VLA), a novel Latent Reasoning VLA framework that models reasoning as a sequence of unobservable latent variables, bypassing the need for explicit text generation. However, latent trajectories are inherently susceptible to noise interference and misalignment with downstream objectives. To address this, we introduce a Reinforcement Learning-based Denoising mechanism that treats latent state generation as a sequential decision process, optimizing reasoning trajectories via task-level rewards. Furthermore, we incorporate an Early-Exit Strategy that adaptively terminates reasoning based on state confidence, enabling a dynamic trade-off between depth and efficiency. Extensive experiments on embodied decision benchmarks demonstrate that AVA-VLA achieves a 6x inference speedup over explicit CoT methods while attaining a 98.3% average success rate on LIBERO, improving both efficiency and long-horizon stability over full-reasoning baselines.

Chinese Translation

现有的视觉-语言-行动（VLA）模型主要依赖显式的思维链（CoT）推理来连接感知与行动。尽管这种范式有效，但在多步骤任务中存在高计算成本和错误传播的问题。本文提出了一种自适应变量对齐VLA（AVA-VLA），这是一种新颖的潜在推理VLA框架，将推理建模为一系列不可观察的潜在变量，从而绕过显式文本生成的需求。然而，潜在轨迹本质上容易受到噪声干扰和与下游目标的不对齐。为了解决这一问题，我们引入了一种基于强化学习的去噪机制，将潜在状态生成视为一个序列决策过程，通过任务级奖励优化推理轨迹。此外，我们还结合了一种早期退出策略，根据状态置信度自适应地终止推理，实现深度与效率之间的动态权衡。在具身决策基准上的广泛实验表明，AVA-VLA在推理速度上比显式CoT方法快6倍，同时在LIBERO上达到了98.3%的平均成功率，提升了效率和长时间稳定性，优于全推理基线。

View on arXiv Download PDF AI Translation

cs.CV / 62 / 2606.15104

Text-Driven Fusion for Infrared and Visible Images: Achieving Image Scene Adaptation on Hyperbolic Space

基于文本驱动的红外与可见光图像融合：在双曲空间中实现图像场景适应

Kang, Huan, Li, Hui, Xu, Tianyang, Zhou, Tao, Wu, Xiao-Jun, Kittler, Josef

Abstract

Infrared and visible image fusion aims to integrate complementary modalities, while existing Euclidean methods impose rigid distance metrics that distort multi-modal interactions and parent-to-child semantic hierarchies. To overcome these limitations, we introduce a text-driven fusion framework empowered by hyperbolic manifold learning. During training, BLIP-extracted text prompts serve as topological anchors within the hyperbolic space, guiding vision-attribute alignment through hyperbolic embeddings that naturally accommodate varying semantic granularities. By exploiting the exponential volume growth dictated by the Poincar\'e ball's negative curvature, this approach seamlessly embeds hierarchical trees to encode coarse-to-fine semantics without metric saturation, while the vast peripheral space prevents texture distortion during cross-modal fusion. At inference, the fusion process autonomously adapts to input content using the learned text-attribute priors, completely eliminating the need for textual input. Experimental results show our method outperforms state-of-the-art approaches on benchmark datasets, with code available at https://github.com/Shaoyun2023/TEDFusion.

Chinese Translation

红外与可见光图像融合旨在整合互补的模态，而现有的欧几里得方法施加了僵化的距离度量，扭曲了多模态交互和父子语义层次结构。为克服这些限制，我们引入了一种基于文本驱动的融合框架，该框架由双曲流形学习赋能。在训练过程中，BLIP提取的文本提示作为双曲空间中的拓扑锚点，通过双曲嵌入引导视觉属性对齐，自然适应不同的语义粒度。通过利用庞加莱球体负曲率所决定的指数体积增长，该方法无缝地嵌入层次树，以编码粗到细的语义，而不出现度量饱和，同时广阔的外部空间在跨模态融合过程中防止了纹理失真。在推理阶段，融合过程利用学习到的文本-属性先验自主适应输入内容，完全消除对文本输入的需求。实验结果表明，我们的方法在基准数据集上优于最先进的方法，代码可在 https://github.com/Shaoyun2023/TEDFusion 获取。

View on arXiv Download PDF AI Translation

cs.CV / 63 / 2606.15110

Physics-Driven Zero-Shot MRI Reconstruction with Non-local Image Priors

基于物理驱动的零-shot MRI重建与非局部图像先验

Zhang, Lingtong, Li, Wenlei, He, Mu, Xiao, Li, Ji, Yang

Abstract

Zero-Shot Self-Supervised Learning (ZS-SSL) has emerged as a promising paradigm for accelerated Magnetic Resonance Imaging (MRI) reconstruction, eliminating the reliance on fully-sampled external datasets. However, learning solely from a single under-sampled scan suffers from supervision scarcity and optimization instability, often leading to overfitting or artifacts. To address these challenges, we propose a robust physics-driven ZS-SSL framework that synergizes physical consistency with image-domain non-local priors. Our method introduces three core innovations: (1) a Coil Sensitivity Map (CSM)-Guided Dynamic Repository, which stabilizes the training trajectory by filtering physically inconsistent artifacts based on coil sensitivity constraints; (2) a SPIRiT-based regularization, which enforces k-space self-consistency via a learned correlation kernel and stochastic masking; (3) a Non-Local Self-Similarity (NSS) Pixel Bank, which leverages the high-fidelity reference established by the former modules to explicitly mine non-local anatomical similarities, thereby augmenting supervision in the image domain. Extensive experiments on the FastMRI dataset demonstrate that our approach achieves state-of-the-art performance, particularly under high acceleration factors, effectively bridging the gap between zero-shot learning and supervised methods. The code is available at https://github.com/Zolento/NS-SSL.

Chinese Translation

零-shot自监督学习（ZS-SSL）作为一种加速磁共振成像（MRI）重建的有前景的范式，消除了对完全采样外部数据集的依赖。然而，仅从单个欠采样扫描中学习会面临监督稀缺和优化不稳定的问题，常常导致过拟合或伪影。为了解决这些挑战，我们提出了一种稳健的基于物理驱动的ZS-SSL框架，结合了物理一致性和图像域非局部先验。我们的方法引入了三个核心创新：（1）基于线圈灵敏度图（CSM）的动态库，该库通过根据线圈灵敏度约束过滤物理不一致的伪影来稳定训练轨迹；（2）基于SPIRiT的正则化，通过学习的相关核和随机掩蔽强制k空间自一致性；（3）非局部自相似（NSS）像素库，利用前述模块建立的高保真参考，明确挖掘非局部解剖相似性，从而增强图像域中的监督。在FastMRI数据集上的大量实验表明，我们的方法在高加速因子下实现了最先进的性能，有效弥合了零-shot学习与监督方法之间的差距。代码可在https://github.com/Zolento/NS-SSL获取。

View on arXiv Download PDF AI Translation

cs.CV / 64 / 2606.15112

Learn Temporal Consistency For Robust Satellite Video Detector

学习时间一致性以增强卫星视频检测器的鲁棒性

Guo, Weilong, Li, Shengyang, Gu, Yanfeng

Abstract

Satellite video object detection (SVOD) for oriented and fine-grained objects plays an important role in satellite applications. Most existing SVOD methods only focus on one or a few coarse-grained categories of moving objects and represent objects with horizontal bounding boxes. They have difficulty extracting complete, accurate, and consistent information about objects in whole satellite videos. In this paper, we propose a satellite video object detection framework based on Temporal Consistency Learning (TCL). TCL adeptly detects oriented and fine-grained objects by leveraging the rich temporal contexts within satellite videos. The framework integrates three key modules: temporal and fine-grained feature aggregation (TFA), structure encoding (SE), and temporal consistency constraint (TCC). TFA and TCC modules facilitate consistent representation learning across frames, while the SE module encodes both appearance and structural information for precise fine-grained recognition. Experimental results on the SAT-MTB benchmark dataset demonstrate TCL's superior performance, achieving a new state-of-the-art oriented and fine-grained detection accuracy of 47.7% mAP--a 4.8% improvement over the baseline. Furthermore, our TCL framework readily accommodates existing image-based detectors, leading to enhanced detection accuracies.

Chinese Translation

卫星视频目标检测（SVOD）在定向和细粒度物体的卫星应用中发挥着重要作用。现有的大多数SVOD方法仅关注一类或少数几类粗粒度的移动物体，并用水平边界框表示物体。这使得它们在提取整个卫星视频中物体的完整、准确和一致的信息时面临困难。本文提出了一种基于时间一致性学习（TCL）的卫星视频目标检测框架。TCL通过利用卫星视频中的丰富时间上下文，灵活地检测定向和细粒度物体。该框架集成了三个关键模块：时间和细粒度特征聚合（TFA）、结构编码（SE）和时间一致性约束（TCC）。TFA和TCC模块促进了跨帧的一致性表示学习，而SE模块则编码了外观和结构信息，以实现精确的细粒度识别。在SAT-MTB基准数据集上的实验结果表明，TCL的性能优越，达到了47.7%的新状态-of-the-art定向和细粒度检测准确率，较基线提高了4.8%。此外，我们的TCL框架能够轻松适应现有的基于图像的检测器，从而提高检测准确性。

View on arXiv Download PDF AI Translation

cs.CV / 65 / 2606.15118

Multi-view feature High-order Fusion for Space Weak Object Detection and Segmentation

空间弱小物体检测与分割的多视角特征高阶融合

Guo, Weilong, Sun, Yuhan, Li, Shengyang

Abstract

Weak objects are common in images and videos of space applications. However, it is hard to learn proper representations from their limited appearance information. Inspired by multi-view learning, we develop simple multi-view attentions, treating their outputs as multi-view features. We also propose a multi-view feature high-order fusion method (MHF) to aggregate more accurate and richer features of weak objects. Our MHF extends the commonly used low-order feature fusion method to higher orders. It enhances the model's capacity to capture relevant and complementary information about weak objects. This is achieved by introducing high-order multi-view features perception and a recursive task-contribution gated selection of multi-view features. The new operation is highly flexible and customizable. It is compatible with various variants of multi-view feature representations. We conduct extensive experiments on two newly constructed space science datasets and an open, large-scale satellite video dataset. Our MHF serves as a plug-and-play module and significantly improves various vision transformers and convolution-based detection and segmentation models. We achieve all state-of-the-art accuracies on both tasks across three datasets. Our MHF can be a new basic module for visual modeling that effectively represents weak objects in terms of multi-view learning. The code will be available at https://github.com/Kingdroper/MHF.

Chinese Translation

弱小物体在空间应用的图像和视频中很常见。然而，从它们有限的外观信息中学习适当的表示是困难的。受到多视角学习的启发，我们开发了简单的多视角注意机制，将其输出视为多视角特征。我们还提出了一种多视角特征高阶融合方法（MHF），以聚合更准确和丰富的弱小物体特征。我们的MHF将常用的低阶特征融合方法扩展到更高阶次。它增强了模型捕捉有关弱小物体的相关和互补信息的能力。这是通过引入高阶多视角特征感知和递归任务贡献门控选择多视角特征来实现的。该新操作具有高度的灵活性和可定制性，兼容各种多视角特征表示的变体。我们在两个新构建的空间科学数据集和一个开放的大规模卫星视频数据集上进行了广泛的实验。我们的MHF作为一个即插即用模块，显著提高了各种视觉变换器和基于卷积的检测与分割模型的性能。在三个数据集上的两个任务中，我们达到了所有最先进的准确率。我们的MHF可以成为视觉建模的新基础模块，有效地通过多视角学习表示弱小物体。代码将可在 https://github.com/Kingdroper/MHF 获取。

View on arXiv Download PDF AI Translation

cs.CV / 66 / 2606.15129

EyeMVP: OCT-Informed Fundus Representation Learning via Paired CFP--OCT Pretraining

EyeMVP：通过配对CFP-OCT预训练的OCT信息化视网膜表示学习

Deng, Zhuo, Zhang, Ruiheng, Zhang, Ziheng, Gao, Weihao, Li, Yitong, Wang, Qian, Shao, Lei, Dong, Jiaoyue, Zeng, Zhixi, Fang, Lijian, Wang, Haibo, Lin, Xiaobin, Liu, Tao, Du, Zhicheng, Zhang, Zhengwei, Yang, Lin, Gong, Zheng, Zhao, Xinyu, Wu, Zhenquan, Li, Fang, Zhou, Zhiguang, Zhang, Guoming, Jing, Sun, Lv, Han, We, Wenbin, Ma, Lan

Abstract

Color fundus photography (CFP) is the mainstay for large-scale retinal screening, yet its diagnostic capacity is constrained by the lack of depth-resolved structural information. Optical coherence tomography (OCT) provides cross-sectional retinal anatomy, but is less accessible in population-level screening. Here, we present EyeMVP, a cross-modal retinal foundation model that uses paired CFP--OCT pretraining to learn OCT-informed CFP representations. EyeMVP is pretrained on 674,893 strict same-eye same-day paired CFP--OCT image triples from 112,642 patients across eight hospitals in China. The model uses cross-modal masked reconstruction to enrich CFP representations with OCT-associated supervision, while requiring only CFP images at inference. To accommodate the non-aligned imaging geometry between en-face CFP and cross-sectional OCT, EyeMVP combines source-constrained cross-attention with CFP-derived structural masks. Across 16 downstream tasks, including classification, segmentation, few-shot adaptation, and cross-modal retrieval, EyeMVP outperforms representative retinal foundation models and shows consistent gains on tasks involving macular and optic nerve structure. For CFP-challenging macular diseases, EyeMVP achieves an AUROC of 0.948 for macular edema (vs.~0.852 for EyeCLIP) and 0.825 for myopic macular schisis. In an exploratory reader study, EyeMVP exceeds junior and intermediate ophthalmologist groups but does not reach senior ophthalmologist performance on macular edema, while showing numerically higher balanced accuracy than all reader groups on myopic macular schisis. These results suggest that pixel-level cross-modal reconstruction can enrich CFP representations with OCT-associated supervision, providing a practical route toward stronger CFP-based retinal analysis in screening settings.

Chinese Translation

彩色视网膜摄影（CFP）是大规模视网膜筛查的主要手段，但其诊断能力受到缺乏深度分辨的结构信息的限制。光学相干断层扫描（OCT）提供了视网膜的横截面解剖结构，但在群体级筛查中可及性较差。在此，我们提出了EyeMVP，这是一种跨模态视网膜基础模型，利用配对CFP-OCT预训练来学习OCT信息化的CFP表示。EyeMVP在来自中国八家医院的112,642名患者的674,893对严格同眼同日配对CFP-OCT图像三元组上进行了预训练。该模型使用跨模态掩蔽重建，通过OCT相关的监督来丰富CFP表示，同时在推理时仅需CFP图像。为了适应面向CFP与横截面OCT之间不对齐的成像几何，EyeMVP结合了源约束的交叉注意力与CFP衍生的结构掩码。在包括分类、分割、少样本适应和跨模态检索在内的16个下游任务中，EyeMVP超越了代表性的视网膜基础模型，并在涉及黄斑和视神经结构的任务中显示出持续的性能提升。对于CFP挑战性的黄斑疾病，EyeMVP在黄斑水肿方面达到了0.948的AUROC（相比之下，EyeCLIP为0.852），在近视性黄斑裂隙方面为0.825。在一项探索性读者研究中，EyeMVP的表现超过了初级和中级眼科医生组，但在黄斑水肿方面未达到高级眼科医生的表现，同时在近视性黄斑裂隙方面显示出比所有读者组更高的平衡准确率。这些结果表明，像素级跨模态重建可以通过OCT相关的监督丰富CFP表示，为在筛查环境中实现更强的基于CFP的视网膜分析提供了切实可行的途径。

View on arXiv Download PDF AI Translation

cs.CV / 67 / 2606.15134

Beyond Scalar Distances: Semantic Attribute Gradients from Frozen MLLMs for Visual Embeddings

超越标量距离：来自冻结的多模态大语言模型的语义属性梯度用于视觉嵌入

Bhatnagar, Shubhang, Baiju, Dheeraj, Ahuja, Narendra

Abstract

Vision encoders for retrieval are typically trained with class-label supervision: each training pair reduces to a scalar that uniformly pushes the embedding apart or pulls it together, as if every visual attribute either differed or matched. A multimodal large language model (MLLM), shown the same pair, can articulate those attributes and use them to predict whether the images share a class. We propose \textbf{SAGA}, a framework that turns this language-grounded, attribute-aware perception into a training signal for the encoder itself. Specifically, we use Group Relative Policy Optimization (GRPO) to reward the MLLM for correct predictions on the vision encoder's tokens. Since correct predictions require those tokens to expose the specific attributes that differ or match between the pair, the gradient pushes the encoder to encode them, replacing the uniform pair-level scalar with attribute-resolved supervision. An auxiliary attention-distillation loss anchors the encoder's embedding to tokens the MLLM attended to, and a standard metric-learning loss shapes the embedding geometry for nearest-neighbour retrieval. The MLLM is frozen throughout and discarded at inference, matching the deployment cost of a metric-learning baseline. SAGA improves Recall@1 by 3 to 6 points over state-of-the-art baselines on CUB-200-2011, Cars-196, FGVC-Aircraft, and iNaturalist Aves on zero-shot image retrieval.

Chinese Translation

用于检索的视觉编码器通常是在类标签监督下训练的：每个训练对简化为一个标量，均匀地推动嵌入分开或拉近，就好像每个视觉属性要么不同，要么匹配。一个多模态大语言模型（MLLM）在看到同一对图像时，可以清晰地表达这些属性，并利用它们预测图像是否属于同一类。我们提出了 extbf{SAGA}，一个将这种基于语言的、属性感知的感知转化为编码器自身训练信号的框架。具体而言，我们使用组相对策略优化（Group Relative Policy Optimization, GRPO）来奖励MLLM对视觉编码器的标记进行正确预测。由于正确的预测需要这些标记揭示出成对图像之间不同或匹配的特定属性，因此梯度推动编码器对其进行编码，从而用属性解析的监督替代均匀的成对标量。辅助注意力蒸馏损失将编码器的嵌入锚定到MLLM关注的标记上，而标准的度量学习损失则塑造嵌入几何以便于最近邻检索。在整个过程中，MLLM保持冻结，并在推理时被丢弃，这与度量学习基线的部署成本相匹配。SAGA在零样本图像检索中，相较于最先进的基线，在CUB-200-2011、Cars-196、FGVC-Aircraft和iNaturalist Aves上提高了3到6个点的Recall@1。

View on arXiv Download PDF AI Translation

cs.CV / 68 / 2606.15142

MotionVLA: Vision-Language-Action Model for Humanoid Motion

MotionVLA：用于类人运动的视觉-语言-动作模型

Zhang, Nonghai, Zhai, Siyu, Li, Yanjun, Zhang, Zeyu, Yin, Zhihan, Guo, Yandong, Shi, Boxin, Tang, Hao

Abstract

Generating realistic humanoid motion from scene images and text involves both low-frequency pose semantics and high-frequency physical dynamics. However, many existing methods tokenize motion with a single shared codebook, forcing heterogeneous motion signals into the same quantization space. Our frequency-domain analysis of human motion data reveals a clear mismatch between single-codebook quantization and motion statistics: five DCT coefficients capture 93% of joint-position energy but only 37% of joint-velocity energy, which can bias quantization toward pose statistics and under-represent high-frequency velocity components. A second challenge lies in adapting a standard autoregressive model to effectively model high-frequency physical signals in motion sequences. Therefore, we propose DSFT, a dual-stream frequency tokenizer that separates motion into Base and physical streams and compresses them independently with DCT truncation and BPE. Furthermore, we present MotionVLA, a Qwen3.5-based model that arranges Base and physical tokens in a unified sequence, where Phys tokens are predicted after Base tokens. Experiments on HumanML3D and MBench show that, despite using a lightweight 2B backbone, MotionVLA reduces the Diversity gap to real data by over 50% on HumanML3D and improves Motion-Condition Consistency by 3.8% on MBench, supporting frequency-aware dual-stream decoupling as an effective formulation for autoregressive motion generation. Code: https://github.com/AIGeeksGroup/MotionVLA. Website: https://aigeeksgroup.github.io/MotionVLA.

Chinese Translation

从场景图像和文本生成逼真的类人运动涉及低频姿态语义和高频物理动态。然而，许多现有方法使用单一共享的词汇表对运动进行标记，迫使异质运动信号进入相同的量化空间。我们对人类运动数据的频域分析揭示了单一词汇表量化与运动统计之间的明显不匹配：五个离散余弦变换（DCT）系数捕获了93%的关节位置能量，但仅捕获了37%的关节速度能量，这可能导致量化偏向于姿态统计，并低估高频速度成分。第二个挑战在于如何调整标准自回归模型，以有效建模运动序列中的高频物理信号。因此，我们提出了DSFT，一种双流频率标记器，将运动分为基础流和物理流，并通过DCT截断和字节对编码（BPE）独立压缩它们。此外，我们提出了MotionVLA，一个基于Qwen3.5的模型，将基础和物理标记排列在统一的序列中，其中物理标记在基础标记之后进行预测。在HumanML3D和MBench上的实验表明，尽管使用轻量级的2B主干，MotionVLA在HumanML3D上将与真实数据的多样性差距缩小了超过50%，并在MBench上提高了运动条件一致性3.8%，支持频率感知的双流解耦作为自回归运动生成的有效形式。代码：https://github.com/AIGeeksGroup/MotionVLA。网站：https://aigeeksgroup.github.io/MotionVLA。

View on arXiv Download PDF AI Translation

cs.CV / 69 / 2606.15151

HiRo: A Compact Four-Directional Hierarchical Reservoir Token-Mixer for Efficient Image Classification

HiRo：一种紧凑的四向层次水库令牌混合器，用于高效的图像分类

Islam, Md Farhadul, Thakkar, Ishan, Hastings, J. Todd

Abstract

Recent image classification models must balance local feature modeling, cross-window interaction, and parameter efficiency. Many high-performing architectures rely on fully trainable token-mixers, which improve representation learning but increase parameter count, optimization complexity and computational cost. We propose a parameter-efficient image classification model called HiRo that integrates shifted-window partitioning with multi-directional hierarchical reservoir computing. Images are divided into non-overlapping patches (treated as tokens), linearly projected, normalized, and enriched with 2D sinusoidal positional encodings, then processed within local windows. Inside each window, tokens are scanned in four directions and passed through a two-stage slice-and-mix reservoir module. In the first stage, directional sequences are split into contiguous slices, each processed by its own fixed reservoir with a trainable closed-loop readout. The resulting slice outputs are summarized using the start, end, and mean representations, and then mixed by a second-stage fixed reservoir for each direction. The mixed slice representations are expanded back to the token level and fused with the first-stage outputs, after which the four directional outputs are realigned and averaged. Consecutive blocks alternate between regular and shifted windows to enable cross-window interaction, followed by layer normalization, a residual feed-forward network, and global pooling for classification. This design combines regular and shifted window partitioning with hierarchical multi-directional reservoirs to make an efficient local-to-cross-window token-mixing framework for image classification. Despite using under 1M trainable parameters and significantly lower memory and time than transformer-style baselines, HiRo also achieves 99.46%, 85.57%, and 59.10% accuracy on MNIST, CIFAR-10, and CIFAR-100, respectively.

Chinese Translation

最近的图像分类模型必须在局部特征建模、跨窗口交互和参数效率之间取得平衡。许多高性能架构依赖于完全可训练的令牌混合器，这虽然提高了表示学习，但也增加了参数数量、优化复杂性和计算成本。我们提出了一种名为HiRo的参数高效图像分类模型，它将移动窗口划分与多方向层次水库计算相结合。图像被划分为不重叠的补丁（视为令牌），然后进行线性投影、归一化，并通过二维正弦位置编码进行丰富处理，接着在局部窗口内进行处理。在每个窗口内，令牌在四个方向上进行扫描，并通过一个两阶段的切片和混合水库模块。第一阶段中，方向序列被分割为连续切片，每个切片由其自身的固定水库和可训练的闭环读取进行处理。生成的切片输出使用起始、结束和均值表示进行汇总，然后由第二阶段的固定水库进行混合。混合后的切片表示被扩展回令牌级别，并与第一阶段的输出融合，随后四个方向的输出被重新对齐并取平均。连续的模块在常规窗口和移动窗口之间交替，以实现跨窗口交互，之后进行层归一化、残差前馈网络和全局池化以进行分类。该设计结合了常规和移动窗口划分与层次多方向水库，构建了一个高效的局部到跨窗口令牌混合框架用于图像分类。尽管使用的可训练参数少于100万，并且在内存和时间上显著低于变换器风格的基线，HiRo在MNIST、CIFAR-10和CIFAR-100上分别达到了99.46%、85.57%和59.10%的准确率。

View on arXiv Download PDF AI Translation

cs.CV / 70 / 2606.15158

RefGC-SR$^2$: Reference-guided Generated Content Super-Resolution and Refinement

RefGC-SR$^2$: 基于参考的生成内容超分辨率与精细化

Sung, Jeahun, Kye, Dahyeon, Kim, Soo Ye, Oh, Jihyong

Abstract

Reference-guided generation (e.g., object compositing, customization) has progressed rapidly, yet current pipelines share a fundamental limitation: the object-centric high-resolution reference image (HRRI) provided by users is downsampled to a fixed low-resolution (LR) before being fed into the model, so the fine-grained details are discarded before the output is even produced. In addition, the generation step then introduces its own artifacts (e.g., identity distortion) on top of this loss. Existing reference-guided generated content refinement (RefGCR) methods can correct some of these artifacts but still operate in the LR domain; reference-guided super-resolution (RefSR) methods recover resolution but assume natural-image degradations and ignore the artifact distribution of generative pipelines. To address both gaps in a single formulation, we introduce a new task: reference-guided generated content super-resolution-refinement (RefGC-SR$^2$), where the original HRRI is reused at the post-processing stage to recover lost details, refine generative artifacts, and upscale the output simultaneously. We construct the first real-world triplet data generation pipeline for this RefGC-SR$^2$ task, training a diptych-conditioned generator to synthesize paired low-quality anchors that public pretrained models cannot provide. We further present a frequency-aware diffusion transformer model for RefGC-SR$^2$ that selectively injects fine details from the HRRI while removing generative artifacts. Extensive experiments demonstrate that our RefGC-SR$^2$ model successfully (i) refines the object identity faithfully with respect to the reference, and (ii) recovers high-resolution details, so that the final result is significantly higher quality and practically more usable compared to existing RefGCR and RefSR baselines.

Chinese Translation

基于参考的生成（例如，物体合成、定制）发展迅速，但当前的流程存在一个根本性限制：用户提供的以物体为中心的高分辨率参考图像（HRRI）在输入模型之前被下采样为固定的低分辨率（LR），因此在输出生成之前，细粒度的细节就被丢弃。此外，生成步骤还会在此损失的基础上引入自身的伪影（例如，身份扭曲）。现有的基于参考的生成内容精细化（RefGCR）方法可以纠正一些这些伪影，但仍然在LR域中操作；而基于参考的超分辨率（RefSR）方法虽然可以恢复分辨率，但假设自然图像的退化并忽略了生成流程中的伪影分布。为了在单一公式中解决这两个缺口，我们引入了一项新任务：基于参考的生成内容超分辨率-精细化（RefGC-SR$^2$），在该任务中，原始的HRRI在后处理阶段被重复使用，以恢复丢失的细节、精细化生成伪影并同时提升输出质量。我们为这个RefGC-SR$^2$任务构建了第一个真实世界三元组数据生成流程，训练了一个双联条件生成器，以合成公共预训练模型无法提供的成对低质量锚点。我们进一步提出了一种频率感知的扩散变换器模型，用于RefGC-SR$^2$，该模型选择性地从HRRI中注入细节，同时去除生成伪影。大量实验表明，我们的RefGC-SR$^2$模型成功地（i）忠实于参考精细化物体身份，以及（ii）恢复高分辨率细节，使得最终结果的质量显著高于现有的RefGCR和RefSR基线，并在实际应用中更具可用性。

View on arXiv Download PDF AI Translation

cs.CV / 71 / 2606.15160

DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning

DLWM：用于高效多模态推理的多样性潜在世界模型

Huang, David, Shan, Lianlei

Abstract

Reasoning capabilities of multimodal large language models (MLLMs) have improved considerably in recent years. Existing approaches typically rely on explicit chain-of-thought or continuous latent-space trajectories to enhance multi-step reasoning. However, these methods generally assume that an input admits a single latent interpretation and unfold reasoning along a fixed path or under a uniform computation budget. In real-world multimodal settings, visual observations are often subject to occlusion, blur, viewpoint variation, or semantic ambiguity, giving rise to multiple plausible interpretations. A uniform reasoning strategy not only limits the model's ability to explore multiple hypotheses but also incurs high memory usage and rollout cost. We present DLWM (Diverse Latent World Models), a multimodal reasoning framework that combines latent-space reasoning with reinforcement learning. First, we construct a set of diverse latent world hypotheses in continuous latent space, each capturing a different plausible interpretation of the visual input, and unfold latent reasoning independently on each hypothesis. An orthogonality-based diversity regularizer explicitly prevents hypothesis collapse. Second, we formulate the latent reasoning process as a resource-constrained sequential decision problem and introduce a resource-aware reinforcement learning policy that adaptively allocates computation across hypotheses, dynamically deciding whether to expand, terminate, or merge reasoning paths, thereby substantially reducing memory footprint and improving rollout efficiency. Experiments on multiple multimodal reasoning benchmarks demonstrate that DLWM outperforms existing methods by 2-5 points in accuracy while reducing memory usage by 24%.

Chinese Translation

近年来，多模态大型语言模型（MLLMs）的推理能力有了显著提升。现有的方法通常依赖于显式的思维链或连续的潜在空间轨迹来增强多步骤推理。然而，这些方法一般假设输入仅具有单一的潜在解释，并沿着固定路径或在统一的计算预算下展开推理。在现实世界的多模态环境中，视觉观察常常受到遮挡、模糊、视角变化或语义模糊的影响，导致多种合理的解释产生。统一的推理策略不仅限制了模型探索多种假设的能力，还导致高内存使用和展开成本。我们提出了DLWM（多样性潜在世界模型），这是一个将潜在空间推理与强化学习相结合的多模态推理框架。首先，我们在连续潜在空间中构建了一组多样的潜在世界假设，每个假设捕捉视觉输入的不同合理解释，并在每个假设上独立展开潜在推理。基于正交性的多样性正则化器明确防止假设崩溃。其次，我们将潜在推理过程表述为一个资源受限的序列决策问题，并引入了一种资源感知的强化学习策略，该策略自适应地在假设之间分配计算，动态决定是扩展、终止还是合并推理路径，从而显著减少内存占用并提高展开效率。在多个多模态推理基准上的实验表明，DLWM在准确性上比现有方法提高了2-5个百分点，同时减少了24%的内存使用。

View on arXiv Download PDF AI Translation

cs.CV / 72 / 2606.15162

GeoStream: Toward Precise Camera Controlled Streaming Video Generation

GeoStream：朝着精确的相机控制流视频生成

Zhao, Yizhou, Wang, Yifan, Wang, Xiaoyuan, Wu, Yushu, Zhang, Hao, Haji-Ali, Moayed, Abdal, Rameen, Mirzaei, Ashkan, Li, Yanyu, Menapace, Willi, Jeni, Laszlo, Tulyakov, Sergey, Wonka, Peter, Wang, Chaoyang

Abstract

Accurate interactive camera control is essential for video-based world models, but most existing approaches learn camera motion implicitly, leading to inaccurate control under out-of-distribution trajectories. Explicit geometric conditioning improves controllability, but existing methods are non-autoregressive and rely on a static 3D cache built from an initial frame, which becomes ineffective once the viewpoint moves beyond the original frustum. We propose GeoStream, a framework that enables precise metric-scale camera control in autoregressive streaming video generation. Our method maintains a self-refreshing 3D cache that is periodically updated online from the model's own outputs: we estimate depth from the most recently generated frame, unproject to 3D, and reproject into the target view to produce point reprojections as geometric conditioning for subsequent synthesis. By the same principle, the conditioning seen during training is also rendered from the student's own generated frames, yielding a fully on-policy distillation that naturally aligns the train and inference conditioning distributions. Unlike prior work that uses off-policy condition noising, our approach trains the model against the exact error distribution it encounters at inference, mitigating both standard autoregressive drift and the second-order geometric feedback loop that arises when the cache itself is derived from generated outputs. Quantitative and qualitative results show that our approach substantially improves camera controllability.

Chinese Translation

准确的互动相机控制对于基于视频的世界模型至关重要，但大多数现有方法隐式学习相机运动，导致在分布外轨迹下控制不准确。显式几何条件改善了可控性，但现有方法是非自回归的，并依赖于从初始帧构建的静态3D缓存，一旦视点超出原始视锥体就变得无效。我们提出了GeoStream，一个在自回归流视频生成中实现精确度量级相机控制的框架。我们的方法维护一个自我刷新3D缓存，该缓存定期在线从模型自身的输出中更新：我们从最近生成的帧估计深度，反投影到3D空间，并重新投影到目标视图，以产生点重投影作为后续合成的几何条件。根据相同的原理，训练期间看到的条件也来自学生自身生成的帧，从而实现完全的在线策略蒸馏，自然对齐训练和推理条件分布。与之前使用离策略条件噪声的工作不同，我们的方法使模型针对其在推理中遇到的精确误差分布进行训练，减轻了标准自回归漂移和当缓存本身源自生成输出时产生的二阶几何反馈循环。定量和定性结果表明，我们的方法显著改善了相机的可控性。

View on arXiv Download PDF AI Translation

cs.CV / 73 / 2606.15167

Variational Network with Wavelet-based UNET in Accelerated MRI Reconstruction from Under Sampled K-space Data

基于小波的UNET变分网络在加速MRI重建中的应用：从欠采样K空间数据出发

Prodhan, Yasir Arafat, Fattah, Shaikh Anowarul

Abstract

Fully sampled MRI requires dense k-space acquisition, leading to long scan times, reduced clinical throughput, and increased sensitivity to patient motion. Accelerated MRI addresses this by acquiring undersampled k-space data and reconstructing the missing information computationally. However, reconstruction from undersampled measurements is highly ill-posed and can introduce aliasing artifacts, noise amplification, and loss of anatomical detail. Although conventional parallel imaging and compressed sensing methods mitigate these issues, and deep learning methods have further improved reconstruction quality, preserving high-frequency structures under aggressive undersampling remains challenging. In this work, we propose a Variational Network with a Wavelet-based U-Net (W-UNet) for accelerated MRI reconstruction. The framework combines physics-guided iterative reconstruction with learnable multi-scale frequency representations. Standard pooling operations are replaced with Discrete Wavelet Transform and Inverse Wavelet Transform modules, enabling lossless downsampling while preserving low-frequency structure and high-frequency edge details. Integrated into the refinement and sensitivity map estimation stages, the proposed design improves artifact suppression, feature preservation, and reconstruction fidelity in both single-coil and multi-coil settings. Experiments on fastMRI knee and M4Raw brain datasets show state-of-the-art performance. Ablation studies further confirm the effectiveness of wavelet-based feature decomposition for accelerated MRI reconstruction.

Chinese Translation

全采样MRI需要密集的K空间采集，这导致扫描时间长、临床通量降低以及对患者运动的敏感性增加。加速MRI通过获取欠采样的K空间数据并计算重建缺失的信息来解决这一问题。然而，从欠采样测量中进行重建是高度不适定的，可能引入混叠伪影、噪声放大和解剖细节的丢失。尽管传统的并行成像和压缩感知方法减轻了这些问题，深度学习方法进一步提高了重建质量，但在激进欠采样的情况下保持高频结构仍然具有挑战性。在本研究中，我们提出了一种基于小波的U-Net（W-UNet）的变分网络用于加速MRI重建。该框架结合了物理引导的迭代重建与可学习的多尺度频率表示。标准池化操作被离散小波变换和逆小波变换模块所替代，使得在保持低频结构和高频边缘细节的同时实现无损下采样。该设计集成于精细化和灵敏度图估计阶段，改善了单线圈和多线圈设置下的伪影抑制、特征保留和重建保真度。在fastMRI膝关节和M4Raw脑部数据集上的实验显示了最先进的性能。消融研究进一步确认了基于小波的特征分解在加速MRI重建中的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 74 / 2606.15169

Label Shift Aware Adaptation for Online Zero-shot Learning with Contrastive Language-Image Pre-Training (CLIP)

针对在线零样本学习的标签偏移感知适应方法：以对比语言-图像预训练（CLIP）为例

Han, Pengxiao, Ye, Changkun, Wang, Yanshuo, Tong, Jinguang, Zhang, Miaohua, Li, Xuesong, Hong, Jie, Petersson, Lars

Abstract

Vision-language models like Contrastive Language-Image Pre-Training (CLIP) have been extensively studied in data-scarce scenarios. A particularly challenging and realistic task in this area is online zero-shot learning with CLIP, where unknown test samples are predicted sequentially in random order by CLIP while keeping the feature extraction and model parameters fixed during the sequential inference phase. Most existing approaches in this setting address the problem by adapting representations online using incoming test samples, while neglecting the distribution of the data on which CLIP was initially trained. This mismatch can lead to degraded performance when the label distribution in the test data differs from that of the training domain. To address this gap, we propose Label Shift Aware (LSA), which formulates the online zero-shot classification task as a domain adaptation problem. Specifically, LSA adapts the predictions computed by CLIP, which was trained on an unknown source distribution, to a target distribution using only unlabeled test data, and applies label shift correction to mitigate the mismatch between the source and target domains. The extensive experiments across multiple datasets demonstrate that the proposed LSA consistently outperforms state-of-the-art online zero-shot learning methods based on CLIP.

Chinese Translation

视觉-语言模型如对比语言-图像预训练（CLIP）在数据稀缺的场景中得到了广泛研究。在这一领域中，一个特别具有挑战性且现实的任务是使用CLIP进行在线零样本学习，其中未知的测试样本由CLIP按随机顺序逐个预测，同时在序列推理阶段保持特征提取和模型参数不变。现有的大多数方法通过使用传入的测试样本在线适应表示来解决这一问题，而忽视了CLIP最初训练时所用数据的分布。这种不匹配可能导致当测试数据中的标签分布与训练领域不同时时，性能下降。为了解决这一问题，我们提出了标签偏移感知（Label Shift Aware, LSA）方法，将在线零样本分类任务表述为一个领域适应问题。具体而言，LSA利用仅有的未标记测试数据，将CLIP在未知源分布上训练得到的预测适应到目标分布，并应用标签偏移校正以减轻源领域与目标领域之间的不匹配。通过在多个数据集上的广泛实验，证明了所提出的LSA方法在基于CLIP的在线零样本学习方法中始终优于现有的最先进技术。

View on arXiv Download PDF AI Translation

cs.CV / 75 / 2606.15176

Enabling Real-Time Point-of-Care Ultrasound Segmentation: A GPU-Free Deployment in Resource-Limited Settings

实现实时床边超声分割：在资源有限环境中的无GPU部署

Gao, Weihao

Abstract

Ultrasound imaging is the most widely adopted medical modality globally due to its low cost and portability, yet artificial intelligence (AI) deployment remains constrained by reliance on GPU-accelerated models, creating a structural paradox where the cost of "intelligence" exceeds that of the imaging device itself. Here, we present the systematic adaptation and extensive evaluation of UltraSeg, an ultra-lightweight architecture originally developed for colonoscopic polyp segmentation, now engineered for point-of-care ultrasound (POCUS) across ten public datasets spanning six anatomical sites (breast, thyroid, kidney, carotid, fetal, and small-animal tumor). We systematically validate both variants in ultrasound domains: UltraSeg-130K (0.13M parameters) achieves 89.7 FPS on single-core CPUs and 34.8 FPS on a refurbished mobile device, while UltraSeg-500K (0.5M parameters) delivers 44.6 FPS on CPU and 16.1 FPS on mobile device. UltraSeg-500K matches or exceeds the Dice performance of the 31M-parameter UNet and approaches 105M-parameter TransUNet in average performance, with superior zero-shot cross-dataset generalization on external validation sets (UDIAT, DDTI). By enabling clinical-grade segmentation without GPU dependency, this work brings AI costs in line with ultrasound accessibility, making advanced diagnostics available in resource-limited settings.

Chinese Translation

超声成像因其低成本和便携性而成为全球最广泛采用的医学成像方式，但人工智能（AI）的部署仍受限于对GPU加速模型的依赖，形成了一种结构性悖论，即“智能”的成本超过了成像设备本身的成本。在此，我们展示了UltraSeg的系统适应和广泛评估，这是一种最初为结肠镜息肉分割开发的超轻量架构，现在经过工程改造，适用于十个公共数据集中的床边超声（POCUS），涵盖六个解剖部位（乳腺、甲状腺、肾脏、颈动脉、胎儿和小动物肿瘤）。我们系统地验证了超声领域中的两个变体：UltraSeg-130K（0.13M参数）在单核CPU上实现了89.7 FPS，在翻新移动设备上实现了34.8 FPS，而UltraSeg-500K（0.5M参数）在CPU上提供了44.6 FPS，在移动设备上提供了16.1 FPS。UltraSeg-500K的Dice性能与31M参数的UNet相当或更优，并在平均性能上接近105M参数的TransUNet，在外部验证集（UDIAT，DDTI）上具有更优的零样本跨数据集泛化能力。通过实现无GPU依赖的临床级分割，本研究将AI成本与超声可及性相结合，使先进的诊断能够在资源有限的环境中实现。

View on arXiv Download PDF AI Translation

cs.CV / 76 / 2606.15188

Adaptive Inference-Time Scaling via Early-Step Latent Verification for Image Editing

基于早期步骤潜在验证的自适应推理时间缩放用于图像编辑

Yu, Yue, Jiao, Yang, Wang, Jiayu, Dai, Qi, Chen, Jingjing

Abstract

Instruction-based image editing has made notable progress with recent advances in generative models. However, the quality of the edited result is still influenced by the randomly sampled initial noise, particularly in complex editing scenarios. An unsuitable initial noise may lead to unsatisfactory editing results. Recent inference-time scaling methods address this issue by sampling multiple initial noises and selecting better candidates. Nevertheless, most of them follow a decode-then-verify scheme which introduces an efficiency-accuracy trade-off. When decoding is performed after limited inference steps, the decoded images often remain too noisy for reliable assessment, whereas sufficiently denoised images require much higher computational cost. To address this issue, we propose VeriLatent, a plug-and-play adaptive inference-time scaling framework with early-step latent verification for image editing. Specifically, we propose a novel verifier that scores each initial noise through a latent-space editing activation map at an early stage. It identifies promising candidates by assessing whether they can induce an effective edit in the correct region. This enables efficient early pruning without decoding latents into images. Building on this, we further develop an adaptive search strategy for inference-time scaling. It allocates inference budgets according to editing difficulty, thereby reducing the number of function evaluations (NFE). Extensive experiments on multiple benchmarks and different base models demonstrate that VeriLatent consistently improves both editing performance and inference-time scaling efficiency.

Chinese Translation

基于指令的图像编辑在生成模型的最新进展中取得了显著进展。然而，编辑结果的质量仍然受到随机采样的初始噪声的影响，特别是在复杂的编辑场景中。不合适的初始噪声可能导致不令人满意的编辑结果。最近的推理时间缩放方法通过采样多个初始噪声并选择更好的候选者来解决这个问题。然而，它们大多数遵循解码-再验证的方案，这引入了效率与准确性的权衡。当在有限的推理步骤后进行解码时，解码的图像往往仍然过于嘈杂，无法进行可靠评估，而充分去噪的图像则需要更高的计算成本。为了解决这个问题，我们提出了VeriLatent，一个即插即用的自适应推理时间缩放框架，具有早期步骤潜在验证用于图像编辑。具体而言，我们提出了一种新颖的验证器，通过潜在空间编辑激活图在早期阶段对每个初始噪声进行评分。它通过评估候选者是否能够在正确区域内引发有效编辑来识别有前途的候选者。这使得在不将潜在变量解码为图像的情况下实现高效的早期修剪。在此基础上，我们进一步开发了一种自适应搜索策略用于推理时间缩放。该策略根据编辑难度分配推理预算，从而减少函数评估次数（NFE）。在多个基准和不同基础模型上的大量实验表明，VeriLatent始终提高了编辑性能和推理时间缩放效率。

View on arXiv Download PDF AI Translation

cs.CV / 77 / 2606.15198

City landscape in sight: A crowdsourced framework for unlocking urban-scale window view perceptions from real estate imagery

城市景观在视野中：一个众包框架用于解锁来自房地产影像的城市规模窗景感知

Peng, Chucai, Yang, Sijie, Liu, Ang, Xiang, Yang, Zhou, Zhixiang, Biljecki, Filip

Abstract

City landscapes viewed through home windows influence quality of life, yet perceptions of actual window views at the urban scale remain understudied. This study presents an approach for large-scale mapping of perceptions using 12,334 window view images (WVIs) collected from actual residential properties listed on real estate platforms in Wuhan, China, representing a rarely explored form of urban view imagery that offers advantages over the rendered or simulated window views commonly examined in previous studies. Through a non-immersive virtual reality platform, we collected 27,477 pairwise comparisons across six perceptual dimensions (e.g.\ Vivid) from 304 participants based on 499 WVIs. A hybrid neural network model was trained to predict human perceptions of all crowdsourced WVIs and map their spatial distribution. Results reveal significant spatial autocorrelation with distinct hot and cold spots across the whole city. Floor level strongly influences human perceptions: while higher floors offer more preferred and extensive window views, lower-floor windows provide residents with quiet and vivid views. An inference model further shows that window view composition matters considerably: high ratios of sky, trees, and low-rise buildings enhance people's preferences and perceptions of vividness, whereas high ratios of high-rise buildings increase perceptions of monotony and oppression. Importantly, these effects are non-linear: the excessive presence of certain elements can alter their impact on human perception. This work advances urban-scale understanding of residents' visual experiences and provides evidence-based guidance for human-centric urban planning and real estate to optimise visual landscapes from windows.

Chinese Translation

通过家庭窗户观察的城市景观影响生活质量，但在城市规模上对实际窗景的感知仍然研究不足。本研究提出了一种大规模映射感知的方法，使用从中国武汉的实际住宅物业在房地产平台上收集的12,334张窗景图像（WVI），这是一种鲜有探讨的城市视图影像形式，相较于以往研究中常见的渲染或模拟窗景，具有优势。通过一个非沉浸式虚拟现实平台，我们基于499张WVI从304名参与者收集了27,477对比数据，涵盖六个感知维度（例如：生动性）。我们训练了一个混合神经网络模型，以预测所有众包WVI的人类感知并绘制其空间分布。结果显示出显著的空间自相关性，整个城市存在明显的热区和冷区。楼层高度对人类感知有显著影响：虽然高层提供了更受欢迎和更广阔的窗景，但低层窗户则为居民提供了安静而生动的视野。进一步的推断模型表明，窗景组成对感知至关重要：天空、树木和低层建筑的高比例增强了人们的偏好和生动感知，而高层建筑的高比例则增加了单调和压迫感的感知。重要的是，这些影响是非线性的：某些元素的过度存在可能会改变其对人类感知的影响。本研究推进了对居民视觉体验的城市规模理解，并为以人为本的城市规划和房地产提供基于证据的指导，以优化窗外的视觉景观。

View on arXiv Download PDF AI Translation

cs.CV / 78 / 2606.15200

Keep It in Mind: User Centric Continual Spatial Intelligence Reasoning in Egocentric Video Streams

牢记在心：以用户为中心的持续空间智能推理在自我中心视频流中的应用

Wang, Yun, Xiao, Junbin, Lyu, Han, Wang, Yifan, Zuo, Jing, Zhang, Zhanjie, Huang, Hong, Wu, Dapeng, Yao, Angela

Abstract

We introduce UCS-Bench, a dataset spanning 170+ hours of egocentric visual observations with 8.1K+ timestamped questions for diagnosing User-Centric Continual Spatial intelligence in egocentric video streams. UCS-Bench targets a new problem that emphasizes dynamic spatial reasoning, long-term memory, and their alignment with users' real-time locations. We propose DirectMe, a framework that incrementally constructs and maintains a structured spatial memory from streaming egocentric observations. DirectMe enables robust tracking and recall of object locations, all relative to the user's movement over time. By tightly coupling visual perception with memory updates and spatial reasoning, our approach supports long-horizon queries that require recalling interactions, resolving viewpoint-induced ambiguities, and adapting to dynamic scenes. Our experiments show that DirectMe significantly improves the spatial reasoning of leading multimodal LLMs; it also surpasses many spatially aware and long-form streaming video models. We hope our benchmark and solution will advance spatial intelligence research for egocentric AI assistants. Data and code are available at https://github.com/cocowy1/UCS-Bench.

Chinese Translation

我们介绍了UCS-Bench，这是一个涵盖170多个小时的自我中心视觉观察的数据集，包含8.1K多个带时间戳的问题，用于诊断自我中心视频流中的以用户为中心的持续空间智能。UCS-Bench针对一个新问题，强调动态空间推理、长期记忆及其与用户实时位置的对齐。我们提出了DirectMe，一个框架，能够从流式自我中心观察中逐步构建和维护结构化的空间记忆。DirectMe实现了对物体位置的稳健跟踪和回忆，所有这些都是相对于用户随时间变化的运动。通过紧密结合视觉感知、记忆更新和空间推理，我们的方法支持需要回忆交互、解决视角引起的歧义以及适应动态场景的长期查询。我们的实验表明，DirectMe显著提高了领先的多模态大语言模型（LLMs）的空间推理能力；它还超越了许多具有空间感知能力和长格式流媒体视频模型。我们希望我们的基准和解决方案能够推动自我中心人工智能助手的空间智能研究。数据和代码可在 https://github.com/cocowy1/UCS-Bench 获取。

View on arXiv Download PDF AI Translation

cs.CV / 79 / 2606.15202

Comparing Human Gaze and Vision-Language Model Attention in Safety-Relevant Environments

比较人类注视与视觉-语言模型在安全相关环境中的注意力

Vallejo, Marta, Wang, Siwen

Abstract

Human visual attention plays an important role in how people perceive and respond to environments containing potential risks. This study investigates whether large vision-language models can identify the same regions of a scene that attract human attention in safety-relevant environments. Eye-tracking data were collected from ten participants viewing 33 scene images representing environments with varying levels of potential risk using Pupil Invisible wearable glasses. Gaze coordinates were mapped onto stimulus images to generate population-averaged human gaze heatmaps. In parallel, GPT-4o was prompted through the OpenAI Vision Application Programming Interface (API) to generate spatial predictions of visual attention, which were converted into saliency maps for comparison with human gaze patterns. Spatial alignment between human gaze heatmaps and model-generated saliency maps was evaluated using four complementary metrics: Pearson correlation (r = 0.515 +- 0.117), Normalised Scanpath Saliency (NSS = 0.988 +- 0.323), Kullback-Leibler divergence (KL = 1.766 +- 0.844), and Area Under the Receiver Operating Characteristic Curve using the Judd formulation (AUC-Judd = 0.806 +- 0.076). A cross-model comparison with Gemini Pro, Gemini Flash, and Claude showed that all models exceeded the AUC-Judd chance baseline of 0.5 and achieved positive NSS scores. Gemini Pro demonstrated the strongest spatial localisation according to three of the four metrics, whereas GPT-4o produced the closest distributional match to human attention as measured by KL divergence. These findings suggest that large vision-language models can identify regions that broadly correspond to where humans direct visual attention in safety-relevant scenes without requiring eye-tracking training data. The results highlight the potential of vision-language models as a scalable tool for approximating human attentional patterns.

Chinese Translation

人类视觉注意力在个体如何感知和应对潜在风险环境中扮演着重要角色。本研究探讨了大型视觉-语言模型是否能够识别出在安全相关环境中吸引人类注意的场景区域。通过佩戴 Pupil Invisible 眼动追踪眼镜，从十名参与者那里收集了观看33幅代表不同潜在风险水平环境的场景图像的眼动数据。将注视坐标映射到刺激图像上，以生成群体平均的人类注视热图。同时，通过 OpenAI 视觉应用程序编程接口（API）对 GPT-4o 进行提示，以生成视觉注意力的空间预测，并将其转换为显著性图以与人类注视模式进行比较。使用四个互补指标评估人类注视热图与模型生成的显著性图之间的空间对齐：皮尔逊相关系数（r = 0.515 ± 0.117）、归一化扫描路径显著性（NSS = 0.988 ± 0.323）、库尔贝克-莱布勒散度（KL = 1.766 ± 0.844）以及使用 Judd 公式的接收者操作特征曲线下面积（AUC-Judd = 0.806 ± 0.076）。与 Gemini Pro、Gemini Flash 和 Claude 的跨模型比较显示，所有模型的 AUC-Judd 超过了 0.5 的机会基线，并取得了正的 NSS 分数。根据四个指标中的三个，Gemini Pro 展现出最强的空间定位能力，而 GPT-4o 在 KL 散度测量中产生了与人类注意力最接近的分布匹配。这些发现表明，大型视觉-语言模型能够识别出大致对应于人类在安全相关场景中视觉注意力指向的区域，而无需眼动追踪训练数据。结果突显了视觉-语言模型作为一种可扩展工具来近似人类注意模式的潜力。

View on arXiv Download PDF AI Translation

cs.CV / 80 / 2606.15236

Show the Signal, Hide the Noise: Spectral Forcing for Pixel-Space Diffusion

显示信号，隐藏噪声：像素空间扩散的谱强制

Fan, Weichen, Diao, Haiwen, Wu, Penghao, Liu, Ziwei

Abstract

Pixel-space diffusion models are trained on full-bandwidth noisy images, yet the useful signal available to the denoiser is strongly frequency dependent. Under rectified-flow diffusion and natural-image power-law spectra, the per-band data-to-noise contour $k^{*}(t) = (1-t)^{-2/\alpha}$ separates a signal-bearing low-frequency region from a noise-dominated high-frequency region at each time $t$. We show that this implicit coarse-to-fine structure is not merely descriptive: it induces a capacity-allocation problem. A standard pixel-space denoiser must discover the moving bandwidth boundary internally and can spend computation on frequency-time regions where the optimal prediction collapses to deterministic baselines rather than data-distribution modeling. To make this boundary explicit, we introduce Spectral Forcing, a parameter-free, time-conditional 2D-DCT low-pass operator applied to the noisy input before the patch embedder. Its cutoff expands monotonically with the diffusion time and becomes the identity at the data endpoint. Through controlled synthetic experiments, we identify the regime in which the operator is beneficial: coarse patch tokenization and data whose high-frequency content is predominantly noise rather than essential signal. On ImageNet-256 with JiT-700M/32, Spectral Forcing consistently improves both FID and Inception Score across different training epochs, demonstrating robust gains throughout training; at finer tokenization, the spectral forcing is still competitive. We further insert the unchanged operator into SenseNova-U1, a unified text-to-image model, where it improves DPG-Bench and GenEval, showing that the input-side spectral prior transfers beyond class-conditional generation. These results suggest a route to capacity-efficient pixel-space diffusion by showing the signal and hiding the noise.

Chinese Translation

像素空间扩散模型是在全带宽噪声图像上训练的，然而可用于去噪器的有用信号在很大程度上依赖于频率。在整流流扩散和自然图像幂律谱下，每个时间 $t$ 的每带数据与噪声轮廓 $k^{*}(t) = (1-t)^{-2/eta}$ 将信号承载的低频区域与噪声主导的高频区域分开。我们展示了这种隐含的粗到细结构不仅仅是描述性的：它引发了一个容量分配问题。标准的像素空间去噪器必须在内部发现移动的带宽边界，并且可能会在频率-时间区域花费计算资源，而这些区域的最佳预测会崩溃为确定性基线，而不是数据分布建模。为了使这个边界显性化，我们引入了谱强制（Spectral Forcing），这是一种无参数、时间条件的二维离散余弦变换（2D-DCT）低通算子，应用于噪声输入之前的补丁嵌入器。其截止频率随着扩散时间单调扩展，并在数据端点处变为单位函数。通过受控的合成实验，我们识别出该算子有益的范围：粗略的补丁标记化和高频内容主要是噪声而非重要信号的数据。在使用 JiT-700M/32 的 ImageNet-256 上，谱强制在不同训练周期中始终提高 FID 和 Inception Score，显示出在整个训练过程中的稳健增益；在更细的标记化下，谱强制仍然具有竞争力。我们进一步将未改变的算子插入到 SenseNova-U1，一个统一的文本到图像模型中，在其中它改善了 DPG-Bench 和 GenEval，显示输入侧的谱先验超越了类别条件生成。这些结果表明，通过显示信号和隐藏噪声，提供了一条实现容量高效的像素空间扩散的途径。

View on arXiv Download PDF AI Translation

cs.CV / 81 / 2606.15243

SPARK: Spatial Policy-driven Adaptive Reinforcement learning for Knowledge distillation

SPARK：基于空间策略驱动的自适应强化学习知识蒸馏

Rasool, Mohamed Jismy Aashik, Ahmad, Shabir, Oh, Gisong, Whangbo, Teag Kuen

Abstract

Low-bit quantization enables deployment of image restoration (IR) networks on resource-constrained devices, but introduces rounding noise that disproportionately degrades high-frequency regions such as edges and fine textures. Existing knowledge distillation (KD) methods apply distillation signals uniformly across all spatial locations, overlooking the varying reconstruction difficulty across image regions. To address this, we propose SPARK (Spatial Policy-driven Adaptive Reinforcement Learning for Knowledge Distillation), a framework that adaptively allocates distillation effort using a lightweight reinforcement learning (RL) policy network. At each training step, a difficulty feature extractor computes four signals, namely Laplacian variance, pixel variance, student reconstruction error, and teacher-student knowledge gap, which are fed into a compact policy CNN that produces a stochastic spatial weight map to modulate the KD loss during quantization-aware training (QAT). SPARK is IR task-agnostic, adds no inference cost, and integrates into any existing QAT pipeline without architectural changes. Experiments on benchmark datasets demonstrate that SPARK consistently outperforms PTQ, QAT, and state-of-the-art (SOTA) KD approaches across multiple student architectures, achieving reconstruction quality closest to the full-precision teacher under significant computational constraints.

Chinese Translation

低比特量化使得图像恢复（IR）网络能够在资源受限的设备上部署，但引入的舍入噪声会不成比例地降低高频区域（如边缘和细纹理）的质量。现有的知识蒸馏（KD）方法在所有空间位置均匀应用蒸馏信号，忽视了图像区域之间重建难度的差异。为了解决这个问题，我们提出了SPARK（基于空间策略驱动的自适应强化学习知识蒸馏），这是一个通过轻量级强化学习（RL）策略网络自适应分配蒸馏努力的框架。在每个训练步骤中，难度特征提取器计算四个信号，即拉普拉斯方差、像素方差、学生重建误差和教师-学生知识差距，这些信号被输入到一个紧凑的策略卷积神经网络（CNN），该网络生成一个随机空间权重图，以调节量化感知训练（QAT）中的KD损失。SPARK与IR任务无关，不增加推理成本，并且可以无架构更改地集成到任何现有的QAT管道中。在基准数据集上的实验表明，SPARK在多个学生架构中始终优于PTQ、QAT和最先进（SOTA）的KD方法，在显著的计算约束下实现了与全精度教师最接近的重建质量。

View on arXiv Download PDF AI Translation

cs.CV / 82 / 2606.15250

Landmark-free Assessment of Lower-limb Alignment with Implicit Neural Shape Functions from Knee Radiographs

基于隐式神经形状函数的无标志下肢对齐评估：来自膝关节X光片的研究

Hu, Zhisen, Kemppainen, Antti, Johnson, David, Panfilov, Egor, Nguyen, Huy Hoang, Cootes, Timothy, Lindner, Claudia, Tiulpin, Aleksei

Abstract

Radiographic assessment of lower-limb alignment (LLA) is important for predicting joint health and surgical outcomes in total knee arthroplasty. Traditional measurement methods are manual and time-consuming, while recent machine learning approaches typically rely on locating a fixed set of anatomical landmarks. This dependence limits flexibility and may require re-annotation when clinical definitions change. To address this, we propose an automated workflow using Implicit Neural Shape Functions (INSF). Rather than relying on explicit landmark coordinates, we encode the anatomy into a compact latent space and regress clinical alignment measurements directly from these latent codes. This architecture allows for rapid extendability to new tasks without altering the backbone representation. We trained our method on an internal dataset of 566 knee radiographs, each annotated with the outline of the femur and tibia. We evaluated it on both an internal test dataset of 50 patients and a separate external set of 402 preoperative cases from the MRKR dataset. Manual clinical measurements are available for these data, and the MRKR measurements will be made publicly accessible. Performance was comparable to state-of-the-art landmark-based methods and manual agreement, while offering a flexible shape representation that can be extended to additional measurement tasks.

Chinese Translation

下肢对齐（LLA）的放射学评估对于预测关节健康和全膝关节置换手术的结果至关重要。传统的测量方法是手动且耗时的，而最近的机器学习方法通常依赖于定位一组固定的解剖标志。这种依赖限制了灵活性，并且在临床定义变化时可能需要重新标注。为了解决这个问题，我们提出了一种使用隐式神经形状函数（INSF）的自动化工作流程。我们并不依赖于显式的标志坐标，而是将解剖结构编码到一个紧凑的潜在空间中，并直接从这些潜在编码回归临床对齐测量。这种架构允许快速扩展到新任务，而无需改变基础表示。我们在一个包含566个膝关节X光片的内部数据集上训练了我们的方法，每个样本都标注了股骨和胫骨的轮廓。我们在一个包含50名患者的内部测试数据集和一个来自MRKR数据集的402个术前病例的外部数据集上进行了评估。这些数据的手动临床测量是可用的，MRKR测量将公开获取。我们的性能与最先进的基于标志的方法和手动一致性相当，同时提供了一个灵活的形状表示，可以扩展到其他测量任务。

View on arXiv Download PDF AI Translation

cs.CV / 83 / 2606.15253

Focus, Align, and Sustain: Counteracting Gradient Dilution in Incremental Object Detection

聚焦、对齐与维持：对抗增量目标检测中的梯度稀释

Zhang, Aoting, Yang, Dongbao, Liu, Chang, Hong, Xiaopeng, Zhou, Yu

Abstract

Adapting Detection Transformers to Incremental Object Detection (IOD) poses a systemic challenge, as set-based optimization is inherently destabilized by sequential learning. In this work, we identify Gradient Dilution as the root cause of performance degradation, wherein optimization signals required to preserve old knowledge are progressively weakened. This phenomenon manifests as a cascading erosion of preservation gradients in magnitude, direction, and support coverage, driven by three tightly coupled factors: Signal Dispersion, where foreground gradients are overwhelmed by background noise; Assignment Drift, where stochastic query-target matching induces inconsistent gradient trajectories; and Support Attrition, where gradients from retained samples insufficiently cover the old-class feature space, weakening decision boundaries under interference from new classes. To counteract this, we propose FAS, a unified framework that Focuses, Aligns, and Sustains gradient flow throughout incremental learning. Specifically, we introduce prior-injected queries to focus discriminative signals by filtering background interference at the source. We further propose deterministic anchor distillation to align query-target assignments and enforce semantic consistency across stages under unstable matching. Finally, we devise manifold-support replay to sustain distributional support of old classes, counteracting representational erosion induced by continual updates. Extensive experiments show that FAS restores robust optimization dynamics and outperforms state-of-the-art methods, achieving over 5.0 AP improvement in the challenging 40+10x4 incremental setting.

Chinese Translation

将检测变换器（Detection Transformers）适应于增量目标检测（Incremental Object Detection, IOD）面临系统性挑战，因为基于集合的优化在顺序学习中本质上是不稳定的。在本研究中，我们确定梯度稀释（Gradient Dilution）是性能下降的根本原因，优化信号在保持旧知识的过程中逐渐减弱。这一现象表现为保存梯度在幅度、方向和支持覆盖上的级联侵蚀，受到三个紧密耦合因素的驱动：信号扩散（Signal Dispersion），其中前景梯度被背景噪声淹没；分配漂移（Assignment Drift），其中随机查询-目标匹配导致不一致的梯度轨迹；以及支持衰退（Support Attrition），保留样本的梯度对旧类别特征空间的覆盖不足，削弱了在新类别干扰下的决策边界。为了解决这一问题，我们提出了FAS，一个统一框架，能够在增量学习中聚焦、对齐和维持梯度流。具体而言，我们引入了先验注入查询，以通过在源头过滤背景干扰来聚焦判别信号。我们进一步提出了确定性锚点蒸馏（deterministic anchor distillation），以对齐查询-目标分配，并在不稳定匹配下强制各阶段之间的语义一致性。最后，我们设计了流形支持重放（manifold-support replay），以维持旧类别的分布支持，抵消持续更新引起的表征侵蚀。大量实验表明，FAS恢复了稳健的优化动态，并超越了最先进的方法，在具有挑战性的40+10x4增量设置中实现了超过5.0的AP提升。

View on arXiv Download PDF AI Translation

cs.CV / 84 / 2606.15265

Trusted Multi-View Deep Learning Classification of Fetal Congenital Heart Disease with Feature-level and Decision-level Fusion

可信的多视角深度学习分类胎儿先天性心脏病的特征级和决策级融合

Zhou, Tan, Yao, Shifa, Xiang, Suncheng, Qian, Dahong, Ye, Baoying

Abstract

Congenital heart disease (CHD) refers to the abnormal anatomical structure caused by the abnormal development of the heart and great vessels during embryonic development. Traditional diagnostics often fail to achieve high accuracy and efficiency, especially given the complexity of cardiac anatomy. This study presents a specialized multi-view deep learning framework for CHD binary classification using echocardiographic images. A large-scale CHD dataset, including five views, was used to train the model, enabling it to integrate multi-angle image data. The framework utilizes advanced feature extraction and attention mechanisms to improve diagnostic precision and reliability. An uncertainty-based decision-making component is also integrated to handle low-quality images, enhancing diagnostic outcomes. Experimental results show that this method achieves top-tier performance on our dataset and provides a robust tool for early CHD detection, underscoring its potential for clinical use. The dataset and source code will be released upon paper acceptance.

Chinese Translation

先天性心脏病（CHD）是指在胚胎发育过程中，由于心脏和大血管的异常发育而导致的解剖结构异常。传统的诊断方法往往难以实现高准确性和高效率，尤其是在心脏解剖结构复杂的情况下。本研究提出了一种专门的多视角深度学习框架，用于基于超声心动图图像的CHD二分类。我们使用了一个大规模的CHD数据集，包括五个视角，以训练模型，使其能够整合多角度的图像数据。该框架利用先进的特征提取和注意力机制，提高了诊断的精确性和可靠性。此外，还集成了基于不确定性的决策组件，以处理低质量图像，从而增强诊断结果。实验结果表明，该方法在我们的数据集上达到了顶尖的性能，并为早期CHD检测提供了一个强有力的工具，突显了其在临床应用中的潜力。数据集和源代码将在论文接受后发布。

View on arXiv Download PDF AI Translation

cs.CV / 85 / 2606.15275

MamBOA: State-Space Architecture for Video Recognition

MamBOA：用于视频识别的状态空间架构

Çelik, Mustafa Bora

Abstract

Fine-grained action recognition demands temporal reasoning that general-purpose architectures address through different cost-accuracy tradeoffs: 3D dense operators couple computation to the input volume, while difference-based methods approximate motion through rigid, hand-crafted subtraction of uncontextualized features - each reflecting a deliberate design choice with corresponding limitations in expressiveness or flexibility. We present MamBOA, a backbone-agnostic temporal framework built upon a novel interleaved scan structure that recasts the selective state-space recurrence (S6) as a native motion synthesizer. By interleaving consecutive feature representations extracted from a pretrained backbone into a single alternating sequence, the proposed scan structurally drives the recurrence to encode both temporal observations of each position within a shared hidden state, separated by only a single decay step - rendering the inter-frame transition an intrinsic component of the state dynamics rather than an externally computed quantity. A cascade of dedicated alignment and decoding operations then distills this joint encoding into an explicit motion representation, which a dual-path pooling mechanism adaptively aggregates by balancing attention-driven selection with uniform temporal coverage. The framework interfaces seamlessly with CNN, Transformer, and Mamba backbone families, adding only ~2.1 GFLOPs per feature pair. On Diving48, MamBOA achieves 85.02% Top-1 accuracy with an image-pretrained backbone and 86.24% with a video-pretrained backbone processing the entire video in a single forward pass - demonstrating that structurally induced state-space dynamics constitute a principled and general foundation for motion modeling.

Chinese Translation

细粒度动作识别需要时间推理，而通用架构通过不同的成本-准确性权衡来应对这一需求：3D密集算子将计算与输入体积耦合，而基于差异的方法通过对未上下文化特征的刚性、手工减法来近似运动——每种方法都反映了相应的设计选择及其在表达能力或灵活性方面的局限性。我们提出了MamBOA，这是一种与主干网络无关的时间框架，基于一种新颖的交错扫描结构，将选择性状态空间递归（S6）重新构建为一种原生运动合成器。通过将从预训练主干网络提取的连续特征表示交错成一个单一的交替序列，所提出的扫描结构在结构上驱动递归，以编码共享隐藏状态中每个位置的时间观察，这些观察仅通过一个衰减步骤分隔——使得帧间过渡成为状态动态的内在组成部分，而不是外部计算的量。然后，一系列专门的对齐和解码操作将这种联合编码提炼为明确的运动表示，双路径池化机制通过平衡基于注意力的选择与均匀的时间覆盖，自适应地聚合这些表示。在Diving48数据集上，MamBOA在使用图像预训练主干网络时达到了85.02%的Top-1准确率，而使用视频预训练主干网络时达到了86.24%，并且在单次前向传播中处理整个视频——这表明结构性诱导的状态空间动态构成了运动建模的一个原则性和通用的基础。

View on arXiv Download PDF AI Translation

cs.CV / 86 / 2606.15282

Enhancing Precision Agriculture with a Hybrid Deep Learning Framework for Multi-Class Plant Disease Classification and Interpretability

通过混合深度学习框架增强精准农业的多类植物疾病分类与可解释性

Sufi, Hasibul Islam, Roy, Ridam, Setu, Shayla Alam, Nadim, Mahimul Islam

Abstract

This study proposes an overall deep learning architecture for multi-class classification of plant diseases from high-resolution leaf imagery, with a particular interest in investigating the behavior of ResNet-50 and a hybrid ResNet + Vision Transformer (ViT) design. A specially gathered image database with 15,200 training images and 3,800 validation images spanning 38 classes across multiple crops, including tomato, apple, grape etc. were subjected to preprocessing steps such as resizing, normalization, and data augmentation to enhance model robustness. Multiple architectures, including ResNet-50, MobileNetV2, and EfficientNet-B0, were trained and compared with the hybrid ResNet + ViT model. All models were fine-tuned using the AdamW optimizer and cross-entropy loss, with early stopping applied to prevent overfitting and ensure generalization. Furthermore, interpretability techniques such as Grad-CAM and saliency maps were implemented to indicate disease-relevant regions, while segmentation-based analysis was performed to identify the affected parts of a leaf. For every one of the considered architectures, ResNet-50 led to the highest accuracy of 98.74%, whereas the hybrid ResNet + ViT model achieved a competitive accuracy of 98.58%, showing that the hybrid architectures were effective in capturing both local and overall information. The experimental results showcase the promise of transformer-based models to achieve highly accurate, interpretable, and computationally efficient computer-based multi-class multi-disease classification systems, providing helpful assistance for cultivation management practices as well as for precision farming.

Chinese Translation

本研究提出了一种整体深度学习架构，用于从高分辨率叶片图像中进行多类植物疾病的分类，特别关注ResNet-50的行为以及混合ResNet + 视觉变换器（Vision Transformer, ViT）设计。我们特别收集了一个包含15,200张训练图像和3,800张验证图像的图像数据库，涵盖了包括番茄、苹果、葡萄等在内的38个类别，并对其进行了预处理步骤，如调整大小、归一化和数据增强，以提高模型的鲁棒性。我们训练并比较了多种架构，包括ResNet-50、MobileNetV2和EfficientNet-B0，并与混合ResNet + ViT模型进行了对比。所有模型均使用AdamW优化器和交叉熵损失进行了微调，并应用早停法以防止过拟合并确保模型的泛化能力。此外，还实施了Grad-CAM和显著性图等可解释性技术，以指示与疾病相关的区域，同时进行了基于分割的分析，以识别叶片的受影响部分。在所有考虑的架构中，ResNet-50达到了最高的98.74%准确率，而混合ResNet + ViT模型则实现了98.58%的竞争性准确率，显示出混合架构在捕捉局部和整体信息方面的有效性。实验结果展示了基于变换器的模型在实现高准确性、可解释性和计算效率的计算机多类多病害分类系统中的潜力，为作物管理实践和精准农业提供了有益的支持。

View on arXiv Download PDF AI Translation

cs.CV / 87 / 2606.15286

Decoupled Motion Representation Learning for Moving Infrared Small Target Detection

解耦运动表征学习用于移动红外小目标检测

Zhang, Guoyi, Wu, Peiwen, Wang, Han, Xu, Xiangpeng, Zhang, Xiaohu

Abstract

Infrared small target detection in dynamic scenes remains challenging due to the highly coupled motions among targets, imaging platforms, and dynamic backgrounds. Existing multi-frame methods usually perform implicit temporal modeling, where coherent background dynamics dominate motion correspondence learning, leading to an inherent trade-off between detection and false alarms. In this work, we observe that background motions exhibit strong global coherence, whereas small targets mainly correspond to sparse local motion anomalies. Moreover, many false-alarm responses maintain high consistency with globally coherent motion patterns, indicating that they mainly originate from coherent background dynamics rather than genuine target motions. Based on these observations, we propose a decoupled motion representation learning framework for moving infrared small target detection. Specifically, an explicit motion branch is introduced to model globally coherent motion dynamics using pretrained optical flow priors, together with a structure-preserving self-supervised adaptation strategy for infrared motion correspondence learning. Meanwhile, an implicit motion branch based on deformable feature alignment is designed to capture target-sensitive local motion anomalies under coherent motion guidance. Furthermore, a coherent-motion-guided local anomaly reasoning module is proposed to identify and suppress coherent-motion-induced false responses during localized motion modeling. Extensive experiments on two challenging infrared small target detection benchmarks demonstrate that the proposed method consistently outperforms existing state-of-the-art approaches, particularly in dynamic scenes with complex motions, while maintaining favorable inference efficiency.

Chinese Translation

在动态场景中，红外小目标检测仍然面临挑战，因为目标、成像平台和动态背景之间的运动高度耦合。现有的多帧方法通常进行隐式时间建模，其中一致的背景动态主导运动对应学习，导致检测与误报之间存在固有的权衡。在本研究中，我们观察到背景运动表现出强烈的全局一致性，而小目标主要对应于稀疏的局部运动异常。此外，许多误报响应与全局一致运动模式保持高度一致，表明它们主要源于一致的背景动态，而非真实目标运动。基于这些观察，我们提出了一种解耦运动表征学习框架，用于移动红外小目标检测。具体而言，引入了一个显式运动分支，利用预训练的光流先验建模全局一致的运动动态，并结合一种结构保持的自监督适应策略进行红外运动对应学习。同时，设计了一个基于可变形特征对齐的隐式运动分支，以在一致运动引导下捕捉对目标敏感的局部运动异常。此外，提出了一种一致运动引导的局部异常推理模块，以识别和抑制在局部运动建模过程中由一致运动引起的误响应。在两个具有挑战性的红外小目标检测基准上进行的广泛实验表明，所提方法在动态场景中，尤其是在复杂运动情况下，始终优于现有的最先进方法，同时保持良好的推理效率。

View on arXiv Download PDF AI Translation

cs.CV / 88 / 2606.15287

G2IA: Geometry-Guided Instance-Aware Retrieval and Refinement for Cross-Modal Place Recognition

G2IA：基于几何引导的实例感知检索与精炼用于跨模态地点识别

Jiao, Xianyun, Xu, Jingyi, Yan, Zhongmiao, Chen, Xieyuanli, Pei, Lin

Abstract

Cross-modal place recognition (CMPR) enables camera-only robots to localize against pre-built LiDAR maps in autonomous navigation scenarios. This image-to-point-cloud setting is challenged by two coupled ambiguities: the modality gap between perspective RGB appearance and sparse metric geometry, and perceptual aliasing among urban places with similar roads, facades, intersections, and object arrangements. Instead of treating CMPR as a single global descriptor matching problem, we argue that reliable retrieval requires both geometry-aware representation alignment and fine-grained candidate verification. In this paper, we propose G2IA, a geometry-guided instance-aware framework for image-to-point-cloud place recognition. In the retrieval stage, visual geometry priors from VGGT and instance features are integrated to construct place descriptors that are more compatible with LiDAR-derived map representations. In the refinement stage, the retrieved candidates are re-ranked by explicitly verifying whether local instance shapes and their relative spatial layouts are consistent across modalities. Experiments on public benchmarks demonstrate that G2IA consistently improves image-to-point-cloud place recognition under different localization thresholds, and exhibits strong cross-dataset generalization.

Chinese Translation

跨模态地点识别（CMPR）使得仅使用相机的机器人能够在自主导航场景中与预构建的激光雷达（LiDAR）地图进行定位。该图像到点云的设置面临两个相互关联的模糊性挑战：透视RGB外观与稀疏度量几何之间的模态差距，以及城市地点中相似道路、立面、交叉口和物体排列之间的感知混淆。我们认为，可靠的检索不仅需要几何感知的表示对齐，还需要细粒度的候选验证，而不是将CMPR视为单一的全局描述符匹配问题。本文提出了G2IA，一种基于几何引导的实例感知框架，用于图像到点云的地点识别。在检索阶段，来自VGGT的视觉几何先验和实例特征被整合，以构建与LiDAR派生地图表示更兼容的地点描述符。在精炼阶段，通过明确验证局部实例形状及其相对空间布局在不同模态间的一致性，对检索到的候选进行重新排序。在公共基准上的实验表明，G2IA在不同定位阈值下始终提高了图像到点云的地点识别能力，并展现出强大的跨数据集泛化能力。

View on arXiv Download PDF AI Translation

cs.CV / 89 / 2606.15304

HemExp: Clinically-Guided Latent Diffusion for Modeling Hematoma Expansion

HemExp：临床引导的潜在扩散模型用于血肿扩展建模

Aydin, Orhun Utku, Tanioka, Satoru, Chuang, Tzu I, Koch, Alexander, Rallios, Dimitrios, Gultom, Marie, Tahhan, Begum, Ishida, Fujimaro, Frey, Dietmar, Hilbert, Adam

Abstract

Hematoma expansion (HE) after spontaneous intracerebral hemorrhage (ICH) is a major determinant of acute triage and treatment decisions in neurosurgical care. However, most existing methods provide either a binary expansion risk or a single follow-up volume, limiting uncertainty-aware decisions. We introduce HemExp, a clinically-guided latent diffusion model that generates patient-specific follow-up non-contrast CT images, along with segmentations of intraparenchymal and intraventricular hemorrhage. Generation is conditioned on baseline imaging, clinical variables, and an explicit expansion indicator, enabling controllable simulation of realistic clinical scenarios. HemExp uses a hemorrhage-aware multi-head variational autoencoder and models progression as the difference between baseline and follow-up latent representations with a conditional diffusion model. The model is trained on paired scans from 450 patients across multiple centers and evaluated on 107 patients from a held-out institution. HemExp produces spatial HE probability maps by generating multiple synthetic follow-up images per patient to estimate distributions of plausible follow-up hematoma volumes. Perturbing clinical inputs such as symptom-onset-to-imaging time or anticoagulant status shifts the predicted follow-up volume distribution. HemExp extends binary predictors and demonstrates robust estimation of clinically relevant outcomes in the imaging space, such as hematoma volume, intraventricular involvement, and mass effects. Overall, our results support controllable latent diffusion as a promising direction for uncertainty-aware modeling of early ICH progression.

Chinese Translation

自发性脑内出血（ICH）后的血肿扩展（HE）是神经外科护理中急性分诊和治疗决策的重要决定因素。然而，现有的大多数方法要么提供二元扩展风险，要么提供单一的随访体积，限制了对不确定性的考虑。我们提出了HemExp，这是一种临床引导的潜在扩散模型，能够生成特定患者的随访非对比CT图像，并对脑实质内和脑室内出血进行分割。生成过程以基线影像、临床变量和明确的扩展指示符为条件，从而实现对现实临床场景的可控模拟。HemExp使用一种考虑出血的多头变分自编码器，并将进展建模为基线和随访潜在表示之间的差异，采用条件扩散模型。该模型在450名患者的配对扫描数据上进行训练，并在一个独立机构的107名患者上进行评估。HemExp通过为每位患者生成多个合成随访图像，产生空间HE概率图，以估计合理的随访血肿体积的分布。改变临床输入，如症状出现到影像学检查的时间或抗凝状态，会改变预测的随访体积分布。HemExp扩展了二元预测因子，并在影像学领域展示了对临床相关结果（如血肿体积、脑室内受累和肿块效应）的稳健估计。总体而言，我们的结果支持可控的潜在扩散作为早期ICH进展不确定性建模的有前景方向。

View on arXiv Download PDF AI Translation

cs.CV / 90 / 2606.15305

CoMNeT: A MedNeXt-CorrDiff Framework for Volumetric Brain Tumor Segmentation

CoMNeT：一种用于体积脑肿瘤分割的MedNeXt-CorrDiff框架

Evans, Michael L., Hossen, MD Fayaz Bin, Sadique, MD Shibly, Farzana, Walia, Iftekharuddin, Khan M.

Abstract

Accurate brain tumor segmentation from multiparametric magnetic resonance imaging (MRI) is critical for treatment planning, response assessment, and quantitative neuro-oncology research. However, automated segmentation remains a difficult task in computer vision because of variation in tumor appearance and MRI protocols across patient scans. Moreover, clinically important regions such as enhancing tumor (ET) and tumor core (TC) are often small relative to the full brain volume, furthering increasing the difficulty of achieving high voxel-level precision. In this paper, we show that combining a modern 3D convolutional segmentation model with corrective diffusion-based refinement and ensembling improves volumetric glioma segmentation on the UTSW-Glioma dataset. We propose CoMNeT, a MedNeXt-CorrDiff framework that uses four MRI modalities as input and predicts ET, TC, and whole tumor (WT) regions for automated brain tumor segmentation. MedNeXt is used as the primary segmentation model with Global Response Normalization for feature learning, while CorrDiff is trained as a postprocessing residual refinement method to correct errors in the probability maps before final thresholding. Using five-fold cross-validation, CoMNeT achieved the highest Dice score for most tumor regions, with ET, TC, WT, and average Dice scores of 0.7543 +/- 0.0261, 0.6806 +/- 0.0166, 0.9049 +/- 0.0128, and 0.7798 +/- 0.0184, respectively. CoMNeT outperformed two selected baseline models: SegResNet (0.7555 +/- 0.0190 average Dice) and standalone MedNeXt (0.7697 +/- 0.0154 average Dice). Our findings support the use of corrective diffusion and fold-level probability ensembling as practical additions to existing state-of-the-art 3D convolutional models for automated glioma segmentation.

Chinese Translation

从多参数磁共振成像（MRI）中准确分割脑肿瘤对于治疗计划、反应评估和定量神经肿瘤学研究至关重要。然而，由于肿瘤外观和患者扫描中的MRI协议的变化，自动分割在计算机视觉中仍然是一项困难的任务。此外，临床重要区域如增强肿瘤（ET）和肿瘤核心（TC）相对于整个脑体积通常较小，进一步增加了实现高体素级精度的难度。在本文中，我们展示了将现代3D卷积分割模型与基于修正扩散的细化和集成相结合可以改善UTSW-Glioma数据集上的体积胶质瘤分割。我们提出了CoMNeT，一个使用四种MRI模态作为输入并预测ET、TC和整体肿瘤（WT）区域以实现自动脑肿瘤分割的MedNeXt-CorrDiff框架。MedNeXt被用作主要分割模型，并结合全局响应归一化进行特征学习，而CorrDiff则作为后处理残差细化方法进行训练，以在最终阈值化之前纠正概率图中的错误。通过五折交叉验证，CoMNeT在大多数肿瘤区域达到了最高的Dice分数，ET、TC、WT和平均Dice分数分别为0.7543 +/- 0.0261、0.6806 +/- 0.0166、0.9049 +/- 0.0128和0.7798 +/- 0.0184。CoMNeT的表现优于两个选定的基线模型：SegResNet（平均Dice为0.7555 +/- 0.0190）和独立的MedNeXt（平均Dice为0.7697 +/- 0.0154）。我们的研究结果支持将修正扩散和折级概率集成作为对现有最先进的3D卷积模型在自动胶质瘤分割中的实用补充。

View on arXiv Download PDF AI Translation

cs.CV / 91 / 2606.15320

Conditional Multi-Event Temporal Grounding in Long-Form Video

长视频中的条件多事件时间定位

Zou, Yuanhao, Kulkarni, Arthad, Tonanez, Lucas, Spencer, Lincoln, Sun, Guangyu, Ding, Tianxingjian, Deng, Andong, Li, Yi, Liu, Shuangjun, Li, Yuan, Gao, Dashan, Bi, Ning, Jing, Taotao, Zhang, Shuai, Chen, Chen

Abstract

Multimodal large language models have made rapid progress in video temporal grounding, yet real-world applications routinely require localizing every event that satisfies compositional temporal and spatial conditions. Existing benchmarks fall short: they localize only a single moment per query, count without temporal conditions, or treat grounding and counting as disjoint tasks. We introduce CoMET-Bench for Conditional Multi-Event Temporal Grounding in long-form video, comprising 2789 queries over 600 videos averaging 33.8 minutes across five real-world domains, with each query composed from 4 temporal conditions, 3 spatial conditions, and a dedicated negative-query subset. We further propose a unified evaluation protocol jointly measuring counting, grounding, and negative-query recognition, including a new Rejection-F1 metric that prevents trivial gaming by lazy "always-empty" models. Benchmarking a broad suite of MLLMs, agent-based, and grounding-specialized methods reveals that existing approaches remain far from solving this task. Building on these findings, we propose CoMET-Agent, a training-free agentic framework that reformulates the task as structured search-and-aggregate, improving [email protected] by 6.1% over GPT-5 purely through structural reasoning. Failure analysis further surfaces three open directions: fine-grained entity tracking, position-uniform retrieval, and causal event pairing.

Chinese Translation

多模态大型语言模型在视频时间定位方面取得了快速进展，但现实应用通常需要定位满足组合时间和空间条件的每个事件。现有基准测试存在不足：它们仅定位每个查询的单个时刻，计算时不考虑时间条件，或将定位和计数视为不相干的任务。我们引入了CoMET-Bench，用于长视频中的条件多事件时间定位，包含2789个查询，覆盖600个视频，平均时长为33.8分钟，涉及五个现实世界领域，每个查询由4个时间条件、3个空间条件和一个专门的负查询子集组成。我们进一步提出了一种统一的评估协议，联合测量计数、定位和负查询识别，包括一个新的拒绝-F1指标，以防止懒惰的“始终为空”模型的简单游戏。对广泛的多模态大型语言模型（MLLMs）、基于代理的方法和专门的定位方法进行基准测试，结果表明现有方法距离解决此任务仍然相去甚远。基于这些发现，我们提出了CoMET-Agent，这是一种无训练的代理框架，将任务重新表述为结构化搜索与聚合，通过结构推理使[email protected]提高了6.1%，相较于GPT-5。失败分析进一步揭示了三个开放方向：细粒度实体跟踪、位置均匀检索和因果事件配对。

View on arXiv Download PDF AI Translation

cs.CV / 92 / 2606.15323

PPDM: Pixel Puzzling Diffusion Model for Speed and Memory Efficient Volumetric Medical Image Translation

PPDM：用于速度和内存高效的体积医学图像翻译的像素拼图扩散模型

Chen, Tianqi, Hou, Jun, Zhou, Yinchi, Duncan, James S., Liu, Chi, Zhou, Bo

Abstract

Diffusion models have demonstrated superior fidelity for medical image-to-image translation, but their extension to high-resolution 3D volumes is severely constrained by prohibitive computational cost and GPU memory requirements. Existing memory-efficient strategies often compromise global volumetric consistency or fine anatomical detail. In this work, we propose the Pixel Puzzling Diffusion Model (PPDM), a simple and effective framework for memory- and speed-efficient 3D medical image translation. PPDM introduces a reversible pixel puzzle-unpuzzle operator that trades spatial resolution for channel dimensionality, substantially reducing activation memory while preserving global context. To further improve efficiency and stability, we adopt a direct bridge diffusion formulation that starts from the conditional input rather than pure noise, enabling the model to focus on task-relevant residuals. In addition, a puzzle-gradient loss is incorporated to enforce spatial coherence and suppress grid-like artifacts introduced by spatial rearrangement. We evaluate PPDM on multiple challenging 3D medical image translation tasks, including low-count PET denoising, joint PET denoising and attenuation correction, and cross-modal MRI translation. Across all tasks, PPDM consistently matches or outperforms full 3D diffusion models while reducing training GPU memory usage by up to an order of magnitude and significantly accelerating inference, and it outperforms existing memory-efficient diffusion approaches based on latent compression or frequency decomposition. These results demonstrate that PPDM provides a practical and scalable solution for high-fidelity 3D diffusion-based medical image translation under limited computational resources.

Chinese Translation

扩散模型在医学图像到图像的翻译中表现出卓越的保真度，但其在高分辨率3D体积中的扩展受到高昂的计算成本和GPU内存需求的严重限制。现有的内存高效策略往往妥协于全局体积一致性或精细的解剖细节。在本研究中，我们提出了像素拼图扩散模型（PPDM），这是一个简单且有效的框架，旨在实现内存和速度高效的3D医学图像翻译。PPDM引入了一种可逆的像素拼图-解拼图操作符，通过在通道维度上进行空间分辨率的交换，显著减少了激活内存，同时保留了全局上下文。为了进一步提高效率和稳定性，我们采用了一种直接桥接扩散公式，该公式从条件输入而非纯噪声开始，使模型能够专注于与任务相关的残差。此外，结合了拼图梯度损失以强制空间一致性，并抑制由空间重排引入的网格状伪影。我们在多个具有挑战性的3D医学图像翻译任务上评估了PPDM，包括低计数PET去噪、联合PET去噪和衰减校正，以及跨模态MRI翻译。在所有任务中，PPDM始终与全3D扩散模型相匹配或超越，同时将训练GPU内存使用量减少了一个数量级，并显著加速了推理，且超越了基于潜在压缩或频率分解的现有内存高效扩散方法。这些结果表明，PPDM为在有限计算资源下实现高保真度的3D基于扩散的医学图像翻译提供了一个实用且可扩展的解决方案。

View on arXiv Download PDF AI Translation

cs.CV / 93 / 2606.15328

SGFormer++: Semantic Graph Transformer for Incremental 3D Scene Graph Generation

SGFormer++：用于增量3D场景图生成的语义图变换器

Qi, Mengshi, Lv, Changsheng, Fu, Zijian, Zhang, Xianlin, Ma, Huadong

Abstract

In this paper, we propose SGFormer++, a novel Semantic Graph Transformer for 3D scene graph generation (SGG), which aims to parse point cloud scenes into semantic structural graphs, where nodes denote detected object instances and edges encode their pairwise relationships, with the core challenge lying in modeling complex global scene structure. While existing graph convolutional network (GCN)-based methods suffer from over-smoothing and limited receptive fields, SGFormer++ leverages Transformer layers as its backbone to enable global message passing. Specifically, we introduce two key components tailored for 3D SGG: (1) a Graph Embedding Layer++ that efficiently integrates edge-aware global context with linear computational complexity, and (2) a Semantic Injection Layer++ that enriches visual features with linguistic priors from large language models (LLMs) and vision-language models (VLMs), boosting semantic representation without introducing extra trainable parameters. To further address the practical challenge of incremental SGG (I-SGG), where new relationship categories arrive sequentially, we equip SGFormer++ with a novel Spatial-guided Feature Adapter, which calibrates predicate features using subject-object spatial geometry to counter scale variation, and a Cascaded Binary Prediction Head that mitigates catastrophic forgetting via task-incremental classifier expansion and logit distillation. Extensive experiments on the 3DSSG benchmark demonstrate that SGFormer++ achieves state-of-the-art performance in both standard and incremental settings: it yields a significant 4.49% absolute improvement in Predicate A@1 under the incremental setting. Code and data are available at: https://github.com/Andy20178/SGFormer.

Chinese Translation

在本文中，我们提出了SGFormer++，一种新颖的语义图变换器，用于3D场景图生成（SGG），旨在将点云场景解析为语义结构图，其中节点表示检测到的物体实例，边缘编码它们的成对关系，核心挑战在于建模复杂的全局场景结构。现有的基于图卷积网络（GCN）的方法存在过度平滑和感受野有限的问题，而SGFormer++利用变换器层作为其主干，以实现全局信息传递。具体而言，我们引入了两个针对3D SGG量身定制的关键组件：（1）Graph Embedding Layer++，它以线性计算复杂度高效地整合边缘感知的全局上下文；（2）Semantic Injection Layer++，它利用大型语言模型（LLMs）和视觉-语言模型（VLMs）中的语言先验丰富视觉特征，在不引入额外可训练参数的情况下提升语义表示。为了进一步解决增量SGG（I-SGG）的实际挑战，即新关系类别顺序出现，我们为SGFormer++配备了一种新颖的空间引导特征适配器，该适配器利用主语-宾语空间几何校准谓词特征，以应对尺度变化，并且采用级联二元预测头，通过任务增量分类器扩展和逻辑蒸馏来减轻灾难性遗忘。在3DSSG基准上的大量实验表明，SGFormer++在标准和增量设置下均实现了最先进的性能：在增量设置下，谓词A@1的绝对提升达4.49%。代码和数据可在以下网址获取：https://github.com/Andy20178/SGFormer。

View on arXiv Download PDF AI Translation

cs.CV / 94 / 2606.15341

CausalDrive: Real-time Causal World Models for Autonomous Driving

CausalDrive：用于自动驾驶的实时因果世界模型

Yan, Tianyi, Zheng, Huan, Chen, Dubing, Qu, Meizhi, Shen, Yingying, Zhou, Lijun, Tu, Mingfei, Wang, Bing, Chen, Guang, Ye, Hangjun, Sun, Haiyang, Xu, Cheng-zhong, Shen, Jianbing

Abstract

World models have emerged as a promising paradigm for scaling autonomous driving (AD) data, yet existing video generative models fall short as interactive simulators. Layout-conditioned renderers rely on "oracle" future trajectories of all background agents, rendering them strictly non-reactive. Conversely, pure action-conditioned predictors lack semantic control over complex interactions and suffer from prohibitive diffusion latencies, hindering closed-loop policy learning. To bridge this gap, we present CausalDrive, a controllable, real-time foundation driving world renderer. CausalDrive operates solely on the initial front-view frame, the ego-vehicle's trajectory, and a macroscopic text prompt. By excluding future NPC layouts, we compel the model to intrinsically predict causal interactions, enabling text-driven control over Driving Sociology, allowing users to dynamically orchestrate diverse counterfactual reactions to identical ego-actions. To overcome the efficiency bottleneck and address the covariate shift in autoregressive generation, we propose a novel Context-Forced DMD architecture. This combines continuous flow-matching with a self-correcting distillation objective, achieving interactive speeds of 12 FPS. This breakthrough transforms the passive video generator into a playable neural simulator. We demonstrate its versatility across three downstream applications: (1) generative closed-loop evaluation with significantly mitigated collision artifacts, (2) large-scale Reinforcement Learning (RL) post-training driven by a Video2Reward module, and (3) real-time human-in-the-loop simulation. Extensive experiments validate that policies trained within CausalDrive's reactive scenarios exhibit superior interaction capabilities in the real world.

Chinese Translation

世界模型已成为扩展自动驾驶（AD）数据的有前景的范式，但现有的视频生成模型在作为互动模拟器方面存在不足。基于布局的渲染器依赖于所有背景代理的“神谕”未来轨迹，使其严格非反应性。相反，纯动作条件预测器缺乏对复杂交互的语义控制，并且受到高昂的扩散延迟的影响，从而阻碍了闭环策略学习。为了解决这一问题，我们提出了CausalDrive，一种可控的实时基础驾驶世界渲染器。CausalDrive仅基于初始的前视帧、自动驾驶车辆的轨迹和宏观文本提示进行操作。通过排除未来非玩家角色（NPC）布局，我们迫使模型内在地预测因果交互，从而实现对驾驶社会学的文本驱动控制，使用户能够动态编排对相同自我动作的多样反事实反应。为了克服效率瓶颈并解决自回归生成中的协变量转移问题，我们提出了一种新颖的上下文强制动态模式分解（Context-Forced DMD）架构。该架构结合了连续流匹配和自我校正蒸馏目标，实现了12帧每秒的互动速度。这一突破将被动视频生成器转变为可玩的神经模拟器。我们展示了其在三个下游应用中的多功能性：（1）生成闭环评估，显著减轻碰撞伪影；（2）由Video2Reward模块驱动的大规模强化学习（RL）后训练；（3）实时人机协作模拟。大量实验证实，在CausalDrive的反应场景中训练的策略在现实世界中表现出更强的交互能力。

View on arXiv Download PDF AI Translation

cs.CV / 95 / 2606.15346

DYNA-PRUNER: Input-Adaptive Data-Model Co-Pruning for Efficient and Scalable Spatio-Temporal Media Prediction

DYNA-PRUNER：输入自适应数据-模型共同剪枝用于高效且可扩展的时空媒体预测

Zhang, Fuyan, Li, Yuqi, Tian, Yingli, Ho, Edmond S. L.

Abstract

Spatio-temporal prediction supports radar/satellite nowcasting and city-scale traffic monitoring, but modern models are often too expensive for real-time deployment. This stems from a mismatch between dense computation and strong input-dependent redundancy (e.g., calm seas or clear skies). To enable automated, resource-aware architecture optimization in scalable media analysis, we propose Dyna-Pruner, an end-to-end framework for input-dependent co-pruning of data and model structure. A shared-importance synchronization mechanism generates coupled masks that prune redundant regions and their corresponding computational units (e.g., convolutional filters), yielding per-sample sparse sub-networks at inference time. Experiments on WeatherBench, SEVIR, and TaxiBJ show seamless integration with CNN, RNN, and Transformer backbones, reducing FLOPs by up to $70\%$ and achieving a $2.5\times$ speedup on NVIDIA Jetson AGX Orin with negligible accuracy loss ($<1\%$).

Chinese Translation

时空预测支持雷达/卫星的即时天气预报和城市规模的交通监测，但现代模型通常过于昂贵，无法实现实时部署。这源于密集计算与强输入依赖冗余之间的不匹配（例如，平静的海面或晴朗的天空）。为了在可扩展的媒体分析中实现自动化的、资源感知的架构优化，我们提出了Dyna-Pruner，这是一个用于输入依赖的数据和模型结构共同剪枝的端到端框架。共享重要性同步机制生成耦合掩码，以剪除冗余区域及其对应的计算单元（例如，卷积滤波器），在推理时产生每个样本的稀疏子网络。在WeatherBench、SEVIR和TaxiBJ上的实验表明，该方法与CNN、RNN和Transformer主干网络无缝集成，将FLOPs减少了高达70%，并在NVIDIA Jetson AGX Orin上实现了2.5倍的加速，且准确率损失微乎其微（<1%）。

View on arXiv Download PDF AI Translation

cs.CV / 96 / 2606.15351

Facial Affect Analysis for Service-Oriented Systems: Advances, Challenges, and Future Visions

面向服务的系统中的面部情感分析：进展、挑战与未来愿景

Georgiou, Spyridon, Psiris, Aggelos, Lagkas, Thomas, Argyriou, Vasileios, Sarigiannidis, Panagiotis, Varlamis, Iraklis, Papadopoulos, Georgios Th.

Abstract

Facial Affect Analysis (FAA) is evolving from a stand-alone recognition task into a reusable perception capability for Service-Oriented Software Ecosystems (SoSE). This paper preserves the FAA methodological core while reframing recent advances through systems-engineering requirements for composable and dependable services. We review representative progress in static and dynamic expression analysis, action-unit and micro-expression modeling, and modern CNN, Transformer, graph, and hybrid architectures, then interpret these advances by their operational fit in edge, cloud, and hybrid service pipelines. The synthesis emphasizes SoSE concerns that determine deployability: service contracts for uncertainty-aware outputs, latency and availability envelopes, lifecycle monitoring and recalibration, governance-aware integration, and interoperability across independently evolving components. Our analysis shows that benchmark gains alone are insufficient for SoSE readiness; robustness under shift, intervention stability, fairness, privacy posture, and runtime guarantees are equally critical. We conclude with a roadmap for treating FAA as an operational service component with explicit interfaces, measurable quality attributes, and accountable lifecycle management.

Chinese Translation

面部情感分析（Facial Affect Analysis, FAA）正从一个独立的识别任务演变为服务导向软件生态系统（Service-Oriented Software Ecosystems, SoSE）中的可重用感知能力。本文保留了FAA的方法论核心，同时通过可组合和可靠服务的系统工程需求重新框定了近期的进展。我们回顾了静态和动态表情分析、动作单元和微表情建模，以及现代卷积神经网络（CNN）、变换器（Transformer）、图形和混合架构的代表性进展，并根据这些进展在边缘、云和混合服务管道中的操作适配性进行解读。综合分析强调了决定可部署性的SoSE关注点：针对不确定性输出的服务合同、延迟和可用性范围、生命周期监测与重新校准、治理意识的集成，以及跨独立演变组件的互操作性。我们的分析表明，仅依靠基准提升不足以满足SoSE的准备性；在变化下的鲁棒性、干预稳定性、公平性、隐私姿态和运行时保证同样至关重要。最后，我们提出了一条将FAA视为具有明确接口、可测量质量属性和负责任生命周期管理的操作服务组件的路线图。

View on arXiv Download PDF AI Translation

cs.CV / 97 / 2606.15355

Sustainable Face Recognition on Low-Power Devices with VQ-VAE Embeddings

基于 VQ-VAE 嵌入的低功耗设备可持续人脸识别

Chronis, Christos, Papadopoulos, Georgios Th., Varlamis, Iraklis

Abstract

Face recognition has become a cornerstone of modern AI applications, yet conventional approaches often rely on computationally intensive models deployed in cloud environments, leading to increased network traffic, high energy consumption, and a heavy carbon footprint. This work introduces a sustainable, edge-deployable face recognition framework based on Vector-Quantized Variational Autoencoders (VQ-VAE), which generates compact and semantically rich latent representations of facial images. By leveraging the compression capacity and reconstruction quality of VQ-VAE embeddings on the edge and combining them with the power of pre-trained face embeddings in a knowledge distillation setup, our system achieves comparable accuracy to state-of-the-art face embedding models while significantly reducing memory and computation requirements on the edge, making it suitable for low-power edge devices. The integration of VQ-VAE compression minimizes network overhead while keeping the matching accuracy high by retaining only the most informative facial features in the latent space. As a result, the reconstructed images preserve the key identity characteristics, improving the robustness and overall performance of the face embeddings.

Chinese Translation

人脸识别已成为现代人工智能应用的基石，然而传统方法通常依赖于在云环境中部署的计算密集型模型，导致网络流量增加、高能耗和较大的碳足迹。本研究提出了一种基于向量量化变分自编码器（VQ-VAE）的可持续边缘部署人脸识别框架，该框架生成紧凑且语义丰富的人脸图像潜在表示。通过利用 VQ-VAE 嵌入在边缘的压缩能力和重建质量，并将其与知识蒸馏设置中预训练的人脸嵌入的优势相结合，我们的系统在边缘设备上实现了与最先进的人脸嵌入模型相当的准确性，同时显著降低了内存和计算需求，使其适用于低功耗边缘设备。VQ-VAE 压缩的集成最小化了网络开销，同时通过仅保留潜在空间中最具信息量的人脸特征来保持高匹配准确性。因此，重建的图像保留了关键的身份特征，提高了人脸嵌入的鲁棒性和整体性能。

View on arXiv Download PDF AI Translation

cs.CV / 98 / 2606.15370

MNet++: Extended 2D/3D Networks for Anisotropic Medical Image Segmentation

MNet++：用于各向异性医学图像分割的扩展2D/3D网络

Odendaal, Kirsten, Bajic, Rade

Abstract

This work demonstrates a full reproduction and extension of MNet, a hybrid 2D/3D convolutional network designed for anisotropic medical image segmentation. The original architecture was re-implemented within the nnU-Net framework to verify its reported performance and robustness to variable voxel spacing, known as anisotropy. Experiments were conducted on PROMISE prostate MRI and a controlled subset of LiTS liver CT under matched preprocessing and compute constraints. The reproduced MNet achieved a Dice similarity coefficient (DSC) of 89.0 +/- 0.9% on PROMISE, within 0.8% of the published result, and 94.3 +/- 1.9% / 54.6 +/- 3.1% for liver and tumor segmentation on LiTS, respectively. Two lightweight extensions were further introduced: (1) a learned Fusion Gating mechanism enabling adaptive 2D-3D feature blending, and (2) a VMamba state-space module for efficient long-range depth modelling. The Spatial Gating variant improved DSC by +0.8% with less than 3% inference overhead, while VMamba improved performance consistency, reducing PROMISE Dice variation to +/- 0.7% and achieving the strongest LiTS liver performance at 95.8% Dice. Both extensions preserved MNet robustness to anisotropy, with delta Dice = 1.5% across 1-4 mm voxel spacing. Overall, the study confirms MNet reproducibility and demonstrates that adaptive fusion and state-space modelling have the potential to further strengthen segmentation reliability under anisotropic conditions. However, further tests are required to provide definitive conclusions.

Chinese Translation

本研究展示了MNet的完整复现和扩展，MNet是一种为各向异性医学图像分割设计的混合2D/3D卷积网络。原始架构在nnU-Net框架内重新实现，以验证其报告的性能和对可变体素间距（即各向异性）的鲁棒性。在PROMISE前列腺MRI和经过匹配预处理和计算约束的LiTS肝脏CT的受控子集上进行了实验。复现的MNet在PROMISE上达到了89.0 +/- 0.9%的Dice相似系数（DSC），与已发布结果相差0.8%，在LiTS上肝脏和肿瘤分割分别达到了94.3 +/- 1.9%和54.6 +/- 3.1%。进一步引入了两个轻量级扩展：(1) 一种学习的融合门控机制，能够实现自适应的2D-3D特征融合；(2) 一个VMamba状态空间模块，用于高效的长距离深度建模。空间门控变体在不到3%的推理开销下将DSC提高了0.8%，而VMamba提高了性能一致性，将PROMISE的Dice变异降低至+/- 0.7%，并在LiTS肝脏分割中实现了95.8%的最佳表现。两个扩展都保持了MNet对各向异性的鲁棒性，在1-4毫米体素间距下的Delta Dice为1.5%。总体而言，本研究确认了MNet的可复现性，并表明自适应融合和状态空间建模有潜力进一步增强在各向异性条件下的分割可靠性。然而，仍需进一步测试以提供明确的结论。

View on arXiv Download PDF AI Translation

cs.CV / 99 / 2606.15389

Timestep Rescheduling in Diffusion Inversion

扩散反演中的时间步重调度

Sun, Shangquan, Gong, Ting, Liu, Zhirui, Wu, Jiamin, Zhao, Runkai, Liu, Mianxin, Ren, Wenqi, Cao, Xiaochun

Abstract

Diffusion inversion, which maps images back to the Gaussian latent space of a diffusion model, is a critical task for image reconstruction and editing. While DDIM enables fast deterministic inversion, it inherently introduces deviations that accumulate into noticeable inversion errors. Existing methods often address this by solving a fixed-point problem but largely overlook how the selection of the diffusion timestep in the noise scheduler influences inversion fidelity. In this work, we reveal that the deviation scale in diffusion inversion is strongly dependent on the timestep size, and exhibits a parabolic trend, with larger errors concentrated at both small and large timesteps. Based on this finding, we propose a simple yet effective nonuniform timestep scheduler that integrates a global rescaling with a local dynamic programming based rescheduling, enabling a strategic allocation of computational effort that minimizes the overall inversion error and preserves higher inversion accuracy. Our method serves as an off-the-shelf enhancement for existing inversion techniques and requires no extra parameters or computational overhead. Through extensive experiments, we verify that integrating our scheduler consistently boosts the performance of existing inversion methods, achieving superior results in image reconstruction and editing.

Chinese Translation

扩散反演是将图像映射回扩散模型的高斯潜在空间的关键任务，对于图像重建和编辑至关重要。虽然 DDIM 使得快速确定性反演成为可能，但它本质上引入了累积的偏差，导致明显的反演误差。现有方法通常通过解决固定点问题来应对这一点，但在很大程度上忽视了噪声调度器中扩散时间步的选择如何影响反演的保真度。在本研究中，我们揭示了扩散反演中的偏差规模与时间步大小之间的强依赖关系，并表现出抛物线趋势，较大的误差集中在小时间步和大时间步。基于这一发现，我们提出了一种简单而有效的非均匀时间步调度器，该调度器将全局重缩放与基于局部动态规划的重调度相结合，使计算努力的战略分配得以实现，从而最小化整体反演误差并保持更高的反演准确性。我们的方法作为现有反演技术的即插即用增强，无需额外参数或计算开销。通过大量实验，我们验证了集成我们的调度器能够持续提升现有反演方法的性能，在图像重建和编辑中取得了优越的结果。

View on arXiv Download PDF AI Translation

cs.CV / 100 / 2606.15409

Segmentation-based Detection for Efficient Multi-Task Spacecraft Perception

基于分割的高效多任务航天器感知检测

Muniyasamy, Sivaperuman, Devasundaram, Surendar

Abstract

Vision-based perception is fundamental to Space Situational Awareness and autonomous on-orbit operations such as rendezvous, docking, servicing, and navigation. However, progress in this area is limited by the scarcity of annotated space imagery and by challenging visual-domain characteristics including severe illumination changes, low signal-to-noise ratio, and high contrast. We address Stream 1 of the SPARK 2026 Challenge, which requires a single model for spacecraft classification, detection, and fine-grained component segmentation across multiple target types. We propose a compact architecture that integrates a MobileNetV3 encoder with a U-Net-style decoder, combining computational efficiency with accurate dense prediction. Detection is derived analytically from the union of predicted component masks, avoiding a separate bounding-box regression head in the single-spacecraft setting. Our method achieved an overall leaderboard score of 0.9482, with task-specific scores of 1.0000 in classification, 0.9788 in detection, and 0.8917 in segmentation. The proposed approach ranked second overall in the SPARK 2026 Challenge, demonstrating that lightweight encoder-decoder architectures can deliver strong multi-task performance for practical onboard space vision systems.

Chinese Translation

基于视觉的感知对于空间态势感知和自主在轨操作（如会合、对接、服务和导航）至关重要。然而，这一领域的进展受到标注空间图像稀缺以及严峻的视觉领域特征（包括严重的光照变化、低信噪比和高对比度）的限制。我们针对SPARK 2026挑战的第一条流进行研究，该挑战要求一个单一模型用于多种目标类型的航天器分类、检测和细粒度组件分割。我们提出了一种紧凑的架构，将MobileNetV3编码器与U-Net风格的解码器相结合，兼顾计算效率和准确的密集预测。检测是通过分析预测的组件掩码的并集来推导的，避免了在单一航天器设置中使用单独的边界框回归头。我们的方法在总排行榜上获得了0.9482的分数，分类任务的得分为1.0000，检测任务的得分为0.9788，分割任务的得分为0.8917。所提出的方法在SPARK 2026挑战中总体排名第二，证明了轻量级编码器-解码器架构能够为实际的航天视觉系统提供强大的多任务性能。

View on arXiv Download PDF AI Translation

cs.CV / 101 / 2606.15417

From Frames to Temporal Graphs: In-Context Egocentric Action Recognition with Vision-Language Models

从框架到时间图：基于视觉-语言模型的上下文自我中心动作识别

Dominguez-Dager, Bessie, Gomez-Donoso, Francisco, Cazorla, Miguel, Pollefeys, Marc, Barath, Daniel, Bauer, Zuria

Abstract

Action reasoning in egocentric video requires capturing fine-grained transitions of hand-object interactions, a task where general-purpose Vision-Language Models (VLMs) often struggle when operating directly on raw pixels. We propose to decouple visual perception from symbolic reasoning by converting videos into Temporal Action Graphs. In a multi-stage prompting pipeline, we first generate dense natural language narratives over short temporal windows as a semantic bottleneck, then formalize them into structured, open-vocabulary graph representations. On the EGTEA and Epic-Kitchens-100 datasets, the symbolic representation unlocks efficient in-context learning: few-shot graph demonstrations yield substantial accuracy gains over zero-shot frame and graph-based inference alike. Even in the zero-shot setting, graph-based reasoning remains competitive with pixel-based inference despite potential pretraining contamination favoring the latter. Across 11 open-weight VLMs from 6 model families ranging from 2B to 235B parameters, our findings indicate that current VLMs are more effective as symbolic reasoners than as direct visual observers. By projecting video into the language domain, we provide a scalable, fine-tuning-free alternative to end-to-end approaches that better leverages these models' latent reasoning strengths. The code will be made public.

Chinese Translation

自我中心视频中的动作推理需要捕捉手-物体交互的细粒度过渡，而通用的视觉-语言模型（VLMs）在直接处理原始像素时往往表现不佳。我们提出通过将视频转换为时间动作图来解耦视觉感知与符号推理。在一个多阶段提示管道中，我们首先在短时间窗口内生成密集的自然语言叙述作为语义瓶颈，然后将其形式化为结构化的开放词汇图表示。在EGTEA和Epic-Kitchens-100数据集上，符号表示解锁了高效的上下文学习：少量示例图的演示在零-shot帧和基于图的推理中均带来了显著的准确性提升。即使在零-shot设置中，基于图的推理在潜在的预训练污染有利于像素基础推理的情况下，仍然与之竞争。在来自6个模型家族的11个开放权重VLM中，我们的研究结果表明，当前的VLM在作为符号推理器时比作为直接视觉观察者更为有效。通过将视频投影到语言领域，我们提供了一种可扩展的、无需微调的替代方案，以更好地利用这些模型的潜在推理优势。代码将公开发布。

View on arXiv Download PDF AI Translation

cs.CV / 102 / 2606.15457

Lesion-DDPM: Lesion-Enhanced 3D Diffusion for MS MRI Synthesis

病灶-DDPM：增强病灶的3D扩散用于多发性硬化症MRI合成

Zhang, Weidong, Jung, Yongchan, Anik, Shafayat Mowla, Xiao, Furen, Janarthanan, Vasudevan, Chuluunbaatar, Enkhzaya, Lee, Byeong Kil, Ryoo, Jeeho

Abstract

3D FLAIR MRI is widely recommended as one of the standard MRI sequences for brain imaging in multiple sclerosis (MS), but publicly available MS datasets remain relatively small and vary across scanners, acquisition protocols, and lesion patterns. This scarcity and variability hinder the development of robust neuroimaging machine learning models and are particularly challenging for generative models that aim to synthesize images while preserving small, sparse lesions. We propose Lesion-DDPM, a 3D conditional diffusion framework for lesion-aware FLAIR synthesis that incorporates multi-level anatomical mask injection together with a lesion-weighted reconstruction loss to emphasize lesion voxels while maintaining global brain structure. Using a curated subset of the MSLesSeg dataset, we compare Lesion-DDPM with representative state-of-the-art GAN- and diffusion-based models, assessing both image-generation metrics and downstream 3D U-Net segmentation. In our experiments, Lesion-DDPM achieved the lowest lesion-region reconstruction error among all methods. In a downstream 3D U-Net lesion segmentation task, a model trained only on Lesion-DDPM-generated scans and evaluated on real MRIs reached a Dice score of 0.616 compared with 0.569 for the best competing synthetic dataset. When Lesion-DDPM images were added to the real training set, the Dice score further increased to 0.685.

Chinese Translation

3D FLAIR MRI被广泛推荐作为多发性硬化症（MS）脑成像的标准MRI序列之一，但公开可用的MS数据集仍然相对较小，并且在扫描仪、采集协议和病灶模式上存在差异。这种稀缺性和变异性阻碍了稳健的神经影像机器学习模型的发展，尤其对旨在合成图像并保留小而稀疏病灶的生成模型构成了特别的挑战。我们提出了病灶-DDPM，一种用于病灶感知FLAIR合成的3D条件扩散框架，结合了多层解剖掩膜注入和病灶加权重建损失，以强调病灶体素，同时保持全脑结构。使用经过整理的MSLesSeg数据集子集，我们将病灶-DDPM与代表性的最先进GAN和基于扩散的模型进行了比较，评估了图像生成指标和下游3D U-Net分割。在我们的实验中，病灶-DDPM在所有方法中实现了最低的病灶区域重建误差。在下游3D U-Net病灶分割任务中，仅在病灶-DDPM生成的扫描上训练的模型，在真实MRI上的评估达到了0.616的Dice分数，而最佳竞争合成数据集的Dice分数为0.569。当将病灶-DDPM图像添加到真实训练集中时，Dice分数进一步提高至0.685。

View on arXiv Download PDF AI Translation

cs.CV / 103 / 2606.15468

Analyzing Visual Aircraft Representations with Sparse Autoencoders

利用稀疏自编码器分析视觉飞机表征

Sharma, Deepshik

Abstract

Vision models can achieve strong performance on classification tasks, but the internal representations supporting their predictions are often difficult to interpret. This work investigates whether sparse autoencoders can decompose intermediate representations of a vision model into interpretable features. We train a ConvNeXt classifier on the FGVC-Aircraft dataset, extract spatial activations from its final feature stage, and train a sparse autoencoder on these activations. The learned sparse features are analyzed using top-activating image patches, activation strength, and class selectivity. Qualitative visual inspection reveals that several features correspond to recognizable aircraft structures and visual patterns. We evaluate a subset of selected features using input-space and feature-space ablations, measuring how blurring image patches and suppressing sparse features affect class logits, classification margins, and prediction confidence. The results suggest that sparse autoencoders can reveal partially interpretable, class-relevant visual features associated with aircraft recognition, while also exposing limitations such as polysemanticity and coarse spatial localization.

Chinese Translation

视觉模型在分类任务中可以实现强大的性能，但支撑其预测的内部表征往往难以解释。本研究探讨了稀疏自编码器是否能够将视觉模型的中间表征分解为可解释的特征。我们在FGVC-Aircraft数据集上训练了一个ConvNeXt分类器，从其最终特征阶段提取空间激活，并在这些激活上训练稀疏自编码器。通过顶级激活图像块、激活强度和类别选择性分析所学习的稀疏特征。定性视觉检查表明，多个特征对应于可识别的飞机结构和视觉模式。我们通过输入空间和特征空间的消融实验评估了一部分选定特征，测量模糊图像块和抑制稀疏特征如何影响类别logits、分类边际和预测置信度。结果表明，稀疏自编码器能够揭示与飞机识别相关的部分可解释的视觉特征，同时也暴露了多义性和粗略空间定位等局限性。

View on arXiv Download PDF AI Translation

cs.CV / 104 / 2606.15486

ST-DiffEye: Diffusion-based Continuous Gaze Generation via Joint Scanpath-Trajectory Modeling

ST-DiffEye：基于扩散的连续注视生成通过联合扫描路径-轨迹建模

Zhao, Brian Nlong, Kara, Ozgur, Kim, Junho, Rehg, James M.

Abstract

We study the problem of human gaze modeling, which aims to generate the gaze patterns a viewer produces while observing a visual stimulus. Gaze is primarily captured through two modalities: continuous eye-tracking trajectories, which describe fine-grained motion dynamics, and discrete scanpaths, which describe high-level fixation structure. Because gaze varies substantially across viewers and trials, we treat this variability as a defining property rather than noise and model gaze as a stochastic generative process. Existing generative gaze models supervise on only one of these two representations in isolation. We hypothesize that trajectories and scanpaths describe gaze at complementary scales and are jointly informative during training, and test this hypothesis through ST-DiffEye, a joint trajectory-scanpath diffusion framework that couples both modalities by concatenating them as an additional raw input channel, requiring no architectural overhead beyond an input and output channel expansion. We further introduce a principled evaluation framework based on the Continuous Ranked Probability Score (CRPS), which generalizes any existing sequence similarity metric into a proper scoring rule that jointly assesses the accuracy and diversity of generated gaze. Experiments on task-driven visual search, covering both target-present and target-absent scenarios, and on free-viewing benchmarks demonstrate state-of-the-art performance. These results, along with detailed ablations, confirm the benefit of joint modeling and the value of distribution-aware evaluation in capturing the intrinsic variability of human gaze. Project webpage: https://st-diffeye.github.io/

Chinese Translation

我们研究人类注视建模的问题，旨在生成观察者在观察视觉刺激时产生的注视模式。注视主要通过两种方式捕获：连续的眼动轨迹，描述细粒度的运动动态，以及离散的扫描路径，描述高层次的注视结构。由于注视在不同观察者和试验中存在显著差异，我们将这种变异视为一种定义特性而非噪声，并将注视建模为一种随机生成过程。现有的生成注视模型仅在孤立的情况下对这两种表示中的一种进行监督。我们假设轨迹和扫描路径在互补的尺度上描述注视，并在训练过程中共同提供信息，通过ST-DiffEye进行验证，该框架是一个联合轨迹-扫描路径扩散框架，通过将两者连接作为额外的原始输入通道来耦合这两种模态，且不需要超出输入和输出通道扩展的架构开销。我们进一步引入了基于连续排名概率评分（CRPS）的原则性评估框架，该框架将任何现有的序列相似性度量推广为一个适当的评分规则，联合评估生成注视的准确性和多样性。在任务驱动的视觉搜索实验中，涵盖目标存在和目标缺失的场景，以及在自由观看基准测试中，展示了最先进的性能。这些结果以及详细的消融实验确认了联合建模的好处和基于分布的评估在捕捉人类注视内在变异性方面的价值。项目网页：https://st-diffeye.github.io/

View on arXiv Download PDF AI Translation

cs.CV / 105 / 2606.15527

Selective Synergistic Learning for Video Object-Centric Learning

视频对象中心学习的选择性协同学习

Moon, WonJun, Heo, Jae-Pil

Abstract

Typical video object-centric learning (VOCL) approaches employ slot-based frameworks that rely on reconstruction-driven encoder-decoder architectures, where learning is mediated by two spatial maps: attention maps from the encoder and object maps from the decoder. As these two distinct maps exhibit different properties, a recent dense alignment strategy attempted to reconcile this discrepancy by enforcing agreement across all spatio-temporal patches via contrastive learning. However, this indiscriminate alignment inadvertently propagates the inherent weaknesses of each module, such as noisy encoder predictions and blurred decoder boundaries. Moreover, computing dense similarities across all pairs incurs a computational cost quadratic in the total number of spatio-temporal patches, severely limiting scalability. Motivated by this, we propose Selective Synergistic Learning (SSync). Instead of exhaustive patch-to-patch alignment, SSync prevents error propagation by selectively distilling only the most reliable cues: leveraging the encoder strictly for boundary refinement and the decoder for interior denoising. This is realized via a pseudo-labeling with linear complexity, eliminating the need for quadratic spatial comparisons. Also, to prevent the reinforcement of architectural biases like slot redundancy, we introduce a transitive pseudo-label merging that consolidates overlapping slots based on spatio-temporal activation consistency. Extensive studies demonstrate that SSync improves decomposition quality and serves as a versatile, plug-and-play module while also exhibiting exceptional robustness to slot configurations. Code is available at github.com/wjun0830/SSync.

Chinese Translation

典型的视频对象中心学习（VOCL）方法采用基于槽位的框架，依赖于重构驱动的编码器-解码器架构，其中学习通过两个空间图进行调节：来自编码器的注意力图和来自解码器的对象图。由于这两种不同的图展现出不同的特性，最近的一种密集对齐策略试图通过对比学习在所有时空补丁之间强制一致性来调和这种差异。然而，这种无差别的对齐无意中传播了每个模块的固有弱点，例如噪声编码器预测和模糊解码器边界。此外，在所有配对之间计算密集相似性会导致与时空补丁总数成平方关系的计算成本，严重限制了可扩展性。基于此，我们提出了选择性协同学习（SSync）。SSync并不是进行全面的补丁对补丁对齐，而是通过选择性提取最可靠的线索来防止错误传播：严格利用编码器进行边界细化，利用解码器进行内部去噪。这是通过线性复杂度的伪标记实现的，消除了平方空间比较的需求。此外，为了防止增强架构偏差（如槽位冗余），我们引入了一种传递伪标记合并方法，根据时空激活一致性整合重叠槽位。大量研究表明，SSync提高了分解质量，并作为一个多功能的即插即用模块，同时对槽位配置表现出卓越的鲁棒性。代码可在github.com/wjun0830/SSync获取。

View on arXiv Download PDF AI Translation

cs.CV / 106 / 2606.15534

Track2View: 4D-Consistent Camera-Controlled Video Generation via Paired 3D Point Tracks

Track2View：通过配对的3D点轨迹实现4D一致性的摄像机控制视频生成

Qiao, Feng, An, Zhaochong, Xiong, Zhexiao, Belongie, Serge, Jacobs, Nathan

Abstract

Re-rendering an existing video from a novel camera viewpoint requires the output to follow the prescribed camera trajectory while preserving the appearance and dynamics of the original scene across every frame. Existing methods rely on per-frame pose embeddings, noisy point-cloud renderings, or implicit learned correspondences, none of which provides an explicit, temporally continuous link between source and target pixels. We propose Track2View, which conditions a video diffusion transformer on paired 3D point tracks: sparse trajectories of scene points projected into both the source and target camera views. These tracks provide explicit spatiotemporal correspondences that are temporally continuous by construction, encoding what content should appear where and when. At the core of Track2View is a dual-view track conditioner that transfers visual context from source to target view through parameter-free geometric operations and learned temporal aggregation, ensuring generalization to arbitrary camera trajectories without memorizing specific motions. We further introduce a data curation pipeline that extracts one-to-one track correspondences by running a 3D point tracker on temporally concatenated multi-camera view pairs. On a 400-video benchmark spanning static and dynamic scenes, Track2View achieves state-of-the-art results across visual quality, view synchronization, and camera accuracy, reducing rotation error by 30-65% and translation error by 61-72% relative to leading baselines. Project page is available at this https URL: https://qjizhi.github.io/track2view

Chinese Translation

从新的摄像机视角重新渲染现有视频要求输出遵循规定的摄像机轨迹，同时在每一帧中保持原始场景的外观和动态。现有方法依赖于逐帧姿态嵌入、噪声点云渲染或隐式学习的对应关系，这些方法都未能提供源像素和目标像素之间明确的、时间上连续的联系。我们提出了Track2View，它将视频扩散变换器条件化于配对的3D点轨迹：投影到源视图和目标视图中的场景点的稀疏轨迹。这些轨迹提供了明确的时空对应关系，构造上是时间上连续的，编码了内容在何处以及何时出现。在Track2View的核心是一个双视图轨迹调节器，通过无参数的几何操作和学习的时间聚合将视觉上下文从源视图转移到目标视图，确保对任意摄像机轨迹的泛化，而无需记忆特定的运动。我们进一步引入了一个数据整理管道，通过在时间上连接的多摄像机视图对上运行3D点跟踪器来提取一对一的轨迹对应关系。在一个涵盖静态和动态场景的400个视频基准测试中，Track2View在视觉质量、视图同步和摄像机精度方面达到了最先进的结果，相较于领先的基线，旋转误差减少了30-65%，平移误差减少了61-72%。项目页面可在此链接访问：https://qjizhi.github.io/track2view

View on arXiv Download PDF AI Translation

cs.CV / 107 / 2606.15547

EcoBin: A Two-Stage Deep Convolutional Neural Network for Contamination-Aware Waste Classification

EcoBin：一种考虑污染的废物分类的两阶段深度卷积神经网络

Kumar, Raghav Senthil

Abstract

Waste classification models have become highly accurate at sorting waste, often exceeding 95% on benchmark datasets. However, these models fail to account for contamination in recyclable waste. We present EcoBin, a two-stage deep convolutional neural network that classifies household waste by its disposal pathway and that explicitly accounts for contamination. The first stage is a base waste classifier built on an EfficientNetV2-S backbone that assigns each of the thirty waste categories in our dataset to one of four disposal pathways. The second stage is a contamination classifier that inspects any item routed toward recycling and overrides the decision to garbage when contamination is detected. Because no public dataset of contaminated recyclables exists, we synthesize one by segmenting images of clean recyclable objects with a U2-Net model and compositing realistic contamination textures onto their surfaces. The first stage achieves 87.42% test accuracy and a 96.13% pathway-adjusted accuracy. Meanwhile, the contamination stage distinguishes clean from contaminated items with a 0.99 ROC-AUC. On a test set of contaminated recyclables, the complete pipeline routes 24 of 25 items correctly, compared with only 1 of 25 for the base classifier alone. A McNemar's test confirms that the improvement contributed by the contamination stage is statistically significant (p < 0.001).

Chinese Translation

废物分类模型在废物分类方面已变得非常准确，通常在基准数据集上的准确率超过95%。然而，这些模型未能考虑可回收废物中的污染问题。我们提出了EcoBin，一种两阶段深度卷积神经网络，它通过废物处理路径对家庭废物进行分类，并明确考虑污染。第一阶段是基于EfficientNetV2-S骨干网络的基础废物分类器，将我们数据集中三十个废物类别中的每一个分配到四个处理路径之一。第二阶段是污染分类器，检查任何被送往回收的物品，并在检测到污染时覆盖其垃圾处理的决策。由于不存在公共的污染可回收物数据集，我们通过使用U2-Net模型对干净可回收物体的图像进行分割，并将现实的污染纹理合成到其表面，合成了一个数据集。第一阶段的测试准确率达到87.42%，路径调整后的准确率为96.13%。与此同时，污染阶段以0.99的ROC-AUC区分干净物品和污染物品。在一个污染可回收物的测试集中，完整的处理流程正确分类了25个物品中的24个，而仅有基础分类器单独分类了25个物品中的1个。McNemar检验确认污染阶段带来的改进在统计上是显著的（p < 0.001）。

View on arXiv Download PDF AI Translation

cs.CV / 108 / 2606.15554

RaLMPH: Reliability-aware Learning for Multi-Pathologist Harmonization in Whole-Slide Image Classification

RaLMPH：面向可靠性的多病理学家和谐学习在全幻灯片图像分类中的应用

Hong, Sungrae, Jeong, Jiwon, Cheon, Soeun, Han, Donghee, Lee, Sol, Shin, Jisu, Kim, Kyungeun, Yi, Mun Yong

Abstract

Multiple Instance Learning (MIL) is a standard paradigm for Whole-Slide Image (WSI) analysis and has achieved strong results in computational pathology. However, most MIL pipelines assume a single "gold" label per slide, which conflicts with clinical practice where substantial inter-pathologist variability is common. Existing multi-annotator learning and label-refinement methods typically estimate global annotator reliability or rely on single-instance assumptions, making them poorly suited to MIL and to localized diagnostic contexts where experts disagree. We propose RaLMPH (Reliability-aware Learning for Multi-Pathologist Harmonization), a MIL-based label reconciliation framework for WSIs annotated by multiple pathologists. RaLMPH introduces a reliability field that jointly models (i) local neighborhood structure in WSI feature space and (ii) expert uncertainty (entropy), enabling per-sample identification of trustworthy reference neighborhoods. Leveraging this field, RaLMPH performs sample-wise local annotator ranking to select reliable opinions per slide and applies an adaptive gating mechanism to fuse labels conditioned on local reliability. Experiments on a clinical WSI dataset with labels from six pathologists, as well as controlled simulated benchmarks, show that RaLMPH consistently outperforms existing approaches. Further analyses clarify how our reliability-aware mechanism improves label reconciliation and downstream MIL performance.

Chinese Translation

多实例学习（MIL）是全幻灯片图像（WSI）分析的标准范式，并在计算病理学中取得了显著成果。然而，大多数MIL流程假设每个幻灯片只有一个“金标准”标签，这与临床实践相悖，因为病理学家之间的显著变异性是常见的。现有的多注释者学习和标签精炼方法通常估计全球注释者的可靠性或依赖单实例假设，使其不适合MIL以及专家意见不一致的局部诊断环境。我们提出了RaLMPH（面向可靠性的多病理学家和谐学习），这是一个基于MIL的标签协调框架，适用于由多个病理学家注释的WSI。RaLMPH引入了一个可靠性场，该场联合建模（i）WSI特征空间中的局部邻域结构和（ii）专家不确定性（熵），使得能够逐样本识别可信的参考邻域。利用该场，RaLMPH执行逐样本的局部注释者排名，以选择每个幻灯片的可靠意见，并应用自适应门控机制，根据局部可靠性融合标签。在一个包含六位病理学家标签的临床WSI数据集以及受控的模拟基准测试中，实验结果表明RaLMPH始终优于现有方法。进一步的分析阐明了我们的可靠性感知机制如何改善标签协调和下游MIL性能。

View on arXiv Download PDF AI Translation

cs.CV / 109 / 2606.15570

An Extensive Benchmark for Single-round and Multi-round Instruction-based Image Editing

单轮与多轮基于指令的图像编辑的广泛基准测试

Ma, Yiwei, Ye, Ke, Lin, Weihuang, Ji, Jiayi, Sun, Xiaoshuai, Chua, Tat-Seng, Ji, Rongrong

Abstract

In recent years, there have been notable advancements in the area of instruction-based image editing (IIE), which focuses on the automatic alteration of input images using a model. Nevertheless, assessing the effectiveness of these editing models poses a considerable challenge due to the intricate nature of instructions and the wide variety of edits. To tackle this problem, one urgent task in this domain is the development of a robust evaluation framework that can precisely gauge the quality of editing outcomes and offer valuable benchmarks to guide future improvements. To address this challenge, we present a comprehensive evaluation benchmark named I2EBench2.0, designed for single-round and multi-round assessment of IIE models. I2EBench2.0 has four key features: 1) Evaluation Across Single and Multi-rounds: I2EBench2.0 simultaneously evaluates both single-round and multi-round instruction-based edits, assessing the precision and consistency of the edits. 2) Extensive Evaluation Criteria: I2EBench2.0 encompasses a broad range of criteria, evaluating both high-level and low-level aspects of each IIE model. Specifically, it incorporates 16 dimensions for single-round evaluations and 7 for multi-round evaluations. 3) Alignment with Human Judgment: To ensure our benchmark aligns with human evaluation, we conducted a comprehensive user study for each criterion. 4) Research-driven Insights: By analyzing the strengths and weaknesses of current IIE models across all 16 single-round and 7 multi-round dimensions, we provide critical insights aimed at directing future research in this area. We tested eight recently developed IIE models using I2EBench2.0 and derived academic insights through meticulous comparison and analysis. The related code, dataset, and images generated by all IIE models are available on GitHub: https://github.com/cocoshe/I2EBench.

Chinese Translation

近年来，基于指令的图像编辑（Instruction-based Image Editing, IIE）领域取得了显著进展，该领域专注于利用模型自动改变输入图像。然而，由于指令的复杂性和编辑类型的多样性，评估这些编辑模型的有效性面临着相当大的挑战。为了解决这个问题，该领域亟需开发一个强大的评估框架，以准确衡量编辑结果的质量，并提供有价值的基准来指导未来的改进。为应对这一挑战，我们提出了一个全面的评估基准，命名为 I2EBench2.0，旨在对 IIE 模型进行单轮和多轮评估。I2EBench2.0 具有四个关键特性：1）单轮与多轮评估：I2EBench2.0 同时评估单轮和多轮基于指令的编辑，评估编辑的精确性和一致性。2）广泛的评估标准：I2EBench2.0 包含广泛的标准，评估每个 IIE 模型的高层次和低层次方面。具体而言，它为单轮评估纳入了 16 个维度，为多轮评估纳入了 7 个维度。3）与人类判断的一致性：为了确保我们的基准与人类评估一致，我们为每个标准进行了全面的用户研究。4）基于研究的洞察：通过分析当前 IIE 模型在所有 16 个单轮和 7 个多轮维度上的优缺点，我们提供了旨在指导该领域未来研究的关键洞察。我们使用 I2EBench2.0 测试了八个最近开发的 IIE 模型，并通过细致的比较和分析得出了学术见解。所有 IIE 模型生成的相关代码、数据集和图像可在 GitHub 上获取：https://github.com/cocoshe/I2EBench。

View on arXiv Download PDF AI Translation

cs.CV / 110 / 2606.15574

Toward the Whole Picture: Accumulative Fingerprint Mapping and Reconstruction for Small-Area Mobile Sensors

迈向全局视角：小面积移动传感器的累积指纹映射与重建

Guan, Xiongjun, Feng, Jianjiang, Zhou, Jie

Abstract

Small-area fingerprint sensing on mobile devices creates a fundamental mismatch between acquisition and recognition: each touch captures only a tiny, pose-varying local patch, while reliable biometric matching ultimately requires a stable and sufficiently complete fingerprint representation. Existing pipelines largely cope with this mismatch by treating repeated touches as independent partial templates, which leads to repeated registration, repeated matching, and no guarantee of adequate global coverage. In this paper, we advocate a different formulation, namely \emph{accumulative fingerprint mapping and reconstruction} for small-area mobile sensing. Rather than matching every partial patch separately, the proposed perspective converts a sequence of local observations into a unified fingerprint state that is progressively refined as new touches arrive and can be matched only once after consolidation. As a concrete baseline, we present a classical pipeline that performs patch-wise structural feature extraction, feature-level registration and fusion, fingerprint map construction, and phase-based ridge reconstruction. More importantly, we position this baseline within a broader mobile fingerprint framework that integrates structured token learning, two-stage pose reasoning, and diffusion-based generative reconstruction. This viewpoint reframes mobile fingerprint recognition from multi-capture multi-match processing to accumulative map building, state refinement, and one-shot matching, offering a principled route toward efficient, pose-robust, and deployment-friendly biometrics for small-area mobile platforms. The baseline implementation has been publicly released at https://github.com/XiongjunGuan/FpReconstruction.

Chinese Translation

小面积指纹感应在移动设备上造成了获取与识别之间的根本不匹配：每次触摸仅捕获一个微小的、姿态变化的局部区域，而可靠的生物特征匹配最终需要一个稳定且足够完整的指纹表示。现有的处理流程主要通过将重复触摸视为独立的部分模板来应对这种不匹配，这导致了重复注册、重复匹配，并且无法保证足够的全局覆盖。在本文中，我们倡导一种不同的表述，即针对小面积移动感应的 extit{累积指纹映射与重建}。我们提出的视角不是单独匹配每个局部区域，而是将一系列局部观测转换为一个统一的指纹状态，该状态随着新触摸的到来而逐步精炼，并且在整合后仅能匹配一次。作为具体的基线，我们提出了一个经典的处理流程，该流程执行基于补丁的结构特征提取、特征级注册与融合、指纹图谱构建以及基于相位的脊线重建。更重要的是，我们将这个基线置于一个更广泛的移动指纹框架中，该框架整合了结构化标记学习、两阶段姿态推理和基于扩散的生成重建。这一观点将移动指纹识别重新框定为从多次捕获多次匹配处理转变为累积地图构建、状态精炼和一次性匹配，为小面积移动平台提供了一条高效、姿态鲁棒且易于部署的生物特征识别的原则性路径。基线实现已公开发布在 https://github.com/XiongjunGuan/FpReconstruction。

View on arXiv Download PDF AI Translation

cs.CV / 111 / 2606.15590

Unlocking Diffusion Hierarchies: Adaptive Timestep Selection for Zero-Shot Segmentation

解锁扩散层次：零样本分割的自适应时间步选择

Nakhli, Ramin, Ramachandran, Mahesh, Ballan, Luca

Abstract

Zero-shot segmentation has recently shown notable improvement by leveraging the rich visual priors in large-scale text-to-image diffusion models, such as Stable Diffusion. However, current diffusion-based methods often face limitations due to the trade-off between spatial resolution and contextual information, as well as their reliance on a single static timestep for feature extraction. To overcome these challenges, our work introduces two key advancements. First, our Contextual Similarity Maps fuse high-resolution attention maps with rich U-Net encoder features, providing both fine-grained and robust per-pixel representations. Second, we identify an emergent hierarchical semantic progression within the denoising process of various diffusion models: representations transition from part-level abstractions at earlier timesteps to object-level abstractions at later stages. Leveraging this insight, we introduce a mechanism to adaptively select the optimal timestep for each pixel. Extensive experiments demonstrate that our method consistently outperforms existing zero-shot segmentation baselines, validating the efficacy of combining contextual features with dynamic, hierarchical timestep selection.

Chinese Translation

零样本分割最近通过利用大规模文本到图像扩散模型（如稳定扩散）中的丰富视觉先验，显示出显著的改进。然而，当前基于扩散的方法常常面临空间分辨率与上下文信息之间的权衡，以及依赖单一静态时间步进行特征提取的局限性。为了解决这些挑战，我们的工作引入了两个关键进展。首先，我们的上下文相似性图将高分辨率注意力图与丰富的 U-Net 编码器特征融合，提供了细粒度且稳健的逐像素表示。其次，我们在各种扩散模型的去噪过程中识别出一种新兴的层次语义进展：表示从早期时间步的部分级抽象过渡到后期阶段的对象级抽象。利用这一见解，我们引入了一种机制，能够自适应地为每个像素选择最佳时间步。大量实验表明，我们的方法在零样本分割基准测试中始终优于现有方法，验证了结合上下文特征与动态层次时间步选择的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 112 / 2606.15592

DenseControl: Instance-Level Controllable Synthesis of Dense Crowd Image

DenseControl：密集人群图像的实例级可控合成

Wang, Juncheng, Shang, Lei, Lu, Wang, Sun, Baigui, Wang, Shujun

Abstract

In this paper, we introduce DenseControl, a novel pipeline for generating dense crowd images. Specifically, DenseControl meticulously positions and sizes each generated instance to align precisely with the predefined coordinates and scales. Based on this, we further allow for control over the background, style, and attributes of instances. The motivation behind DenseControl stems from the observation of two main challenges in synthesizing crowd images: controlling signal embedding and maintaining topological integrity when imparting instance scale guidance. To address these, we first introduce the Isolated Object Embedding (IOE) map, a novel representation that facilitates spatial location control while mitigating the difficulties associated with learning projections for model. Secondly, we propose an Implicit Scale Embedding (ISE) strategy that seamlessly integrates with the IOE map to encode precise scale information. To further enhance the efficacy of combining ISE with the IOE map, we incorporate a Position Shortcut mechanism that enhances cross-attention to alleviate projection challenges. We evaluate DenseControl through two lenses: synthesis quality and applicability in latent applications. Experiments across different control conditions demonstrate DenseControl achieves state-of-the-art results in dense crowd image synthesis. Furthermore, we showcase applications in augmenting crowd analysis under data scarcity, transfer learning, and weather generalization scenes, to highlight the practical utility of DenseControl. The codebase will be released.

Chinese Translation

本文提出 DenseControl，一种用于生成密集人群图像的新型流程。具体而言，DenseControl 能够精确地对每个生成的实例进行位置与尺寸的控制，使其严格对齐预定义的坐标与尺度。在此基础上，我们进一步实现对背景、风格以及实例属性的可控生成。DenseControl 的动机源于对人群图像合成中两个关键挑战的观察：控制信号的嵌入问题，以及在引入实例尺度引导时保持拓扑结构完整性的困难。为了解决这些问题，我们首先提出 Isolated Object Embedding（IOE）图，这是一种新型表示方法，可在缓解模型投影学习困难的同时，实现空间位置的可控建模。其次，我们提出 Implicit Scale Embedding（ISE）策略，并使其能够与 IOE 图无缝结合，从而编码精确的尺度信息。为了进一步提升 ISE 与 IOE 图融合的效果，我们引入 Position Shortcut 机制，以增强交叉注意力并缓解投影学习中的困难。我们从两个角度对 DenseControl 进行评估：合成质量与在潜在应用中的适用性。不同控制条件下的实验表明，DenseControl 在密集人群图像合成任务中达到了当前最先进（state-of-the-art）的性能。此外，我们还展示了其在数据稀缺条件下增强人群分析、迁移学习以及天气泛化场景中的应用，以突出 DenseControl 的实际应用价值。代码将会公开发布。

View on arXiv Download PDF AI Translation

cs.CV / 113 / 2606.15597

Fusion-E2Pulse: A Multimodal Event-RGB Fusion Network for Non-contact Pulse Wave Reconstruction

Fusion-E2Pulse：一种用于非接触脉搏波重建的多模态事件-RGB融合网络

Feng, Qian, Guo, Hao, Niu, Yan, Xu, Zhenhuan, Li, Yidi

Abstract

Non-contact pulse wave reconstruction hinges on the precise recovery of waveform morphology, including the dicrotic notch. Conventional Red-Green-Blue (RGB)-based methods, which extract physiological signals from recorded facial videos, are constrained by the integral imaging mechanism of standard cameras, where the exposure process induces a smoothing effect that attenuates subtle vascular pulsation details. Conversely, neuromorphic event cameras, while offering exceptional sensitivity to intensity fluctuations, are inherently susceptible to noise and artifacts induced by minor motion. To exploit the synergy between frame-based integration and event-based differential sensing, we propose a novel multimodal network named Fusion-E2Pulse. This framework utilizes filtered RGB signals as structural priors to suppress motion artifacts, while leveraging the high-sensitivity of event streams to recover fine-grained morphological details. Experimental results demonstrate that Fusion-E2Pulse achieves state-of-the-art performance, effectively balancing noise suppression and morphological fidelity, achieving a mean absolute error of 0.78 bpm for heart rate estimation, a waveform correlation of 0.89, and a systolic phase duration error of 16.74 ms, validating its efficacy in reconstructing fine-grained pathological features.

Chinese Translation

非接触脉搏波重建依赖于波形形态的精确恢复，包括二尖波凹陷。传统的基于红绿蓝（RGB）的方法从录制的面部视频中提取生理信号，但受到标准相机的整体成像机制的限制，其中曝光过程会导致平滑效应，从而削弱细微血管脉动细节。相反，神经形态事件相机虽然对强度波动具有卓越的敏感性，但本质上容易受到微小运动引起的噪声和伪影的影响。为了利用基于帧的集成与基于事件的差分感知之间的协同作用，我们提出了一种名为Fusion-E2Pulse的新型多模态网络。该框架利用过滤后的RGB信号作为结构先验，以抑制运动伪影，同时利用事件流的高灵敏度恢复细粒度的形态细节。实验结果表明，Fusion-E2Pulse实现了最先进的性能，有效平衡了噪声抑制和形态保真度，心率估计的平均绝对误差为0.78 bpm，波形相关性为0.89，收缩相位持续时间误差为16.74 ms，验证了其在重建细粒度病理特征方面的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 114 / 2606.15604

Parameter-Efficient Adaptation of SAM 3 for Automated ITV Generation from 4DCT Images

基于参数高效适应的SAM 3模型用于从4DCT图像自动生成内部靶区

Song, Changwoo

Abstract

Four-dimensional computed tomography (4DCT) captures the full respiratory cycle of thoracic anatomy, yet current Internal Target Volume contouring workflows process each phase in isolation, discarding temporal coherence and leaving contours vulnerable to phase-specific artifacts. We present a lightweight framework that applies parameter-efficient fine-tuning to the Segment Anything Model 3 (SAM 3) via low-rank adaptation (LoRA) to align its text-prompted segmentation with the medical domain using only seven annotated 3D CT volumes. Furthermore, the framework incorporates a hard negative mining strategy to improve boundary discrimination in low-contrast thoracic regions. At inference, phase-wise predictions are refined through phase-coherent temporal filtering and spatial connectivity analysis. Since respiratory motion is continuous and periodic, genuine anatomy appears in contiguous blocks of phases, whereas transient artifacts appear sporadically and are thus effectively suppressed. Experiments on pulmonary and cardiac structures yield median Dice scores of 0.968 and 0.910 with 95th-percentile Hausdorff distances of 0.998 mm and 2.931 mm, respectively. The proposed framework effectively eliminates the severe false-positive predictions inherent in the zero-shot inference of the unadapted SAM 3. With only seven annotated volumes, the framework retains over 95% of full-data accuracy, and the entire pipeline is trainable on a single consumer-grade GPU, demonstrating a scalable, data-efficient solution for adaptive radiotherapy.

Chinese Translation

四维计算机断层扫描（4DCT）捕捉了胸部解剖结构的完整呼吸周期，然而当前的内部靶区轮廓绘制工作流程在处理每个相位时是孤立进行的，忽视了时间一致性，使得轮廓容易受到相位特定伪影的影响。我们提出了一种轻量级框架，通过低秩适应（LoRA）对Segment Anything Model 3（SAM 3）进行参数高效的微调，以仅使用七个带注释的三维CT体积将其文本提示分割与医学领域对齐。此外，该框架还结合了一种硬负样本挖掘策略，以提高低对比度胸部区域的边界判别能力。在推理过程中，通过相位一致的时间滤波和空间连通性分析来细化相位预测。由于呼吸运动是连续和周期性的，真实的解剖结构在相邻的相位中出现，而瞬态伪影则偶尔出现，因此得以有效抑制。在对肺部和心脏结构的实验中，获得了中位Dice系数为0.968和0.910，95百分位Hausdorff距离分别为0.998毫米和2.931毫米。所提出的框架有效消除了未适应的SAM 3在零样本推理中固有的严重假阳性预测。仅使用七个带注释的体积，该框架保留了超过95%的全数据准确性，整个流程可在单个消费级GPU上进行训练，展示了一种可扩展、数据高效的自适应放射治疗解决方案。

View on arXiv Download PDF AI Translation

cs.CV / 115 / 2606.15608

On the Adversarial Robustness of Multimodal LLM Judges

多模态大型语言模型评判者的对抗鲁棒性研究

Wang, Zihan, Pang, Guansong, Liu, Zelin, Miao, Wenjun, Zheng, Jin, Bai, Xiao

Abstract

Multimodal Large Language Models (MLLMs) are increasingly used as automated judges, e.g., for image quality and safety assessment. However, their adversarial robustness remains largely unexplored, threatening the fairness and reliability of automated judging. To bridge this gap, we introduce RobustMLLMJudge, the first general framework for evaluating the adversarial robustness of general-purpose MLLMs when functioning as judges. It covers diverse attacks against popular judge approaches across quality and safety evaluation scenarios. Using RobustMLLMJudge, we reveal that i) different MLLM judges are highly vulnerable to score-inflating adversarial attacks; and ii) although effective, these attack methods face a critical challenge due to unique constraints in the evaluation protocols of MLLM judges. We further propose MGSIA, namely Manifold-Guided Semantic Induction Attack, a novel method that bypasses these constraints to enable more effective and transferable attacks on MLLM judges. The core idea of MGSIA is to combine affirmative semantic induction with high-score manifold alignment: it maximizes the probability that judges yield affirmative responses (e.g., "Yes") to binary semantic queries, while regularizing adversarial representations toward high-score centers estimated from proxy protocols. Together, these objectives yield transferable score-inflating perturbations. Extensive experiments demonstrate the superiority and generalizability of MGSIA in deceiving advanced MLLM judges under different evaluation scenarios, highlighting the need for robust MLLM judges. Code and data will be made available at https://github.com/mala-lab/RobustMLLMJudge.

Chinese Translation

多模态大型语言模型（MLLMs）越来越多地被用作自动评判者，例如用于图像质量和安全性评估。然而，它们的对抗鲁棒性仍然在很大程度上未被探索，这威胁到自动评判的公平性和可靠性。为了解决这一问题，我们提出了RobustMLLMJudge，这是第一个用于评估通用MLLMs作为评判者时对抗鲁棒性的通用框架。该框架涵盖了针对流行评判方法在质量和安全评估场景中的多种攻击。使用RobustMLLMJudge，我们揭示了i）不同的MLLM评判者对评分膨胀的对抗攻击高度脆弱；以及ii）尽管这些攻击方法有效，但由于MLLM评判者评估协议中的独特约束，这些攻击方法面临着重大挑战。我们进一步提出了MGSIA，即流形引导的语义诱导攻击，这是一种新颖的方法，能够绕过这些约束，从而对MLLM评判者实施更有效和可转移的攻击。MGSIA的核心思想是将肯定的语义诱导与高分流形对齐相结合：它最大化评判者对二元语义查询（例如“是”）给出肯定响应的概率，同时将对抗表示规范化为从代理协议估计的高分中心。综合这些目标产生可转移的评分膨胀扰动。大量实验表明，MGSIA在不同评估场景下欺骗先进的MLLM评判者方面具有优越性和广泛适用性，突显了对鲁棒MLLM评判者的需求。代码和数据将发布在https://github.com/mala-lab/RobustMLLMJudge。

View on arXiv Download PDF AI Translation

cs.CV / 116 / 2606.15611

Mutual Distillation of Dual-Foundation Models for Semi-Supervised PET/CT Segmentation

双基础模型的互蒸馏用于半监督PET/CT分割

Mao, Fuyou, Wu, Beining, Jiang, Yanfeng, Xu, Bohan, Lin, Lixin, Ji, Naye, Zhang, Hao, Tang, Yan

Abstract

Organ segmentation from PET/CT is critical for quantitative analysis and radiotherapy planning in oncology. To ease the high annotation cost of PET/CT segmentation, semi-supervised learning (SSL) provides a practical and effective solution for developing deep models with limited labeled data. Recent developments in visual foundation models have demonstrated remarkable adaptability with improved efficiency. In this work, we propose a mutual distillation framework that seamlessly exploits both structural and functional foundation models, which act as modality-specific generalists for distilling knowledge from structural CT and metabolic PET imaging. By bridging the gap between the task-specific precision of student models and the segmentation priors of generalist foundation models, we propose \textbf{MuDuo}, a mutual distillation framework that synergistically leverages SAM-Med3D for CT and SegAnyPET for PET to distill their knowledge into a lightweight student network. Our approach eliminates the need for manual prompts while maximizing the utility of unlabeled data for automatic segmentation, achieving state-of-the-art performance on the AutoPET dataset with only 5 labeled cases. Our source code is available at https://github.com/Wu-beining/MuDuo.

Chinese Translation

从PET/CT进行器官分割对于肿瘤学中的定量分析和放疗规划至关重要。为了减轻PET/CT分割的高标注成本，半监督学习（SSL）为在有限标注数据下开发深度模型提供了一个实用有效的解决方案。近期视觉基础模型的发展展示了显著的适应性和提高的效率。在本研究中，我们提出了一种互蒸馏框架，能够无缝利用结构性和功能性基础模型，这些模型作为特定模态的通用模型，从结构性CT和代谢性PET成像中提取知识。通过弥合学生模型的任务特定精度与通用基础模型的分割先验之间的差距，我们提出了 extbf{MuDuo}，一个互蒸馏框架，协同利用SAM-Med3D用于CT和SegAnyPET用于PET，将它们的知识蒸馏到一个轻量级的学生网络中。我们的方法消除了对手动提示的需求，同时最大化未标注数据在自动分割中的利用，在仅有5个标注案例的情况下，在AutoPET数据集上实现了最先进的性能。我们的源代码可在https://github.com/Wu-beining/MuDuo获取。

View on arXiv Download PDF AI Translation

cs.CV / 117 / 2606.15614

Variational Test-time Optimization for Diffusion Synchronization

变分测试时优化用于扩散同步

Lee, Hyunsoo, Sofian, Farrin Marouf, Pandey, Kushagra, Mandt, Stephan

Abstract

Collaborative generation, which coordinates multiple diffusion trajectories to extend the capabilities of pretrained priors, has emerged as a powerful paradigm for extending the applicability of diffusion models. Among existing approaches, diffusion synchronization provides a scenario-agnostic solution by introducing general guidance mechanisms. However, current synchronization approaches rely heavily on heuristics and still require task-specific tailoring, which limits their generalizability and performance. In this work, we mathematically derive a synchronization framework based on optimal control, providing a principled explanation of diffusion synchronization. During sampling, we optimize control variables to guide multiple trajectories toward coherent solutions while remaining close to the underlying diffusion prior. Our method operates entirely at test-time without additional training, thereby enabling broad applicability across diverse generation scenarios when combined with strong pretrained priors. We demonstrate consistent improvements over baselines on three representative collaborative generation tasks, covering a wide range of modalities and applications. Beyond performance gains, our work establishes a novel foundation for collaborative generation, opening a principled path toward extending pretrained generative models to new collaborative generation settings.

Chinese Translation

协同生成通过协调多个扩散轨迹来扩展预训练先验的能力，已成为扩展扩散模型适用性的强大范式。在现有方法中，扩散同步通过引入通用指导机制提供了一种场景无关的解决方案。然而，目前的同步方法在很大程度上依赖于启发式方法，并且仍然需要针对特定任务的调整，这限制了它们的通用性和性能。在本研究中，我们基于最优控制数学推导出一个同步框架，为扩散同步提供了原则性解释。在采样过程中，我们优化控制变量，以引导多个轨迹朝向一致的解决方案，同时保持与基础扩散先验的接近。我们的方法完全在测试时运行，无需额外训练，从而在与强大的预训练先验结合时，能够广泛适用于多种生成场景。我们在三项代表性的协同生成任务上展示了相对于基线的一致性改进，涵盖了广泛的模态和应用。除了性能提升，我们的工作为协同生成建立了一个新颖的基础，为将预训练生成模型扩展到新的协同生成设置开辟了一条原则性路径。

View on arXiv Download PDF AI Translation

cs.CV / 118 / 2606.15617

NeRD: Neuro-Symbolic Rule Distillation for Efficient Ontology-Grounded Chain-of-Thought in Medical Image Diagnosis

NeRD：用于医疗图像诊断中高效本体基础思维链的神经符号规则蒸馏

Yang, Hongxi, Jiang, Yiwen, Yan, Siyuan, Chow, Jamie, Li, Eunis, Poon, Charlotte, Fong, Stephanie, Zhao, Xiangyu, Mehta, Deval, George, Yasmeen, Ge, Zongyuan

Abstract

Interpretability is essential for trustworthy medical image diagnosis. However, existing concept-driven interpretable methods have key limitations: Concept Bottleneck Models (CBMs) require scoring all predefined concepts at inference time and for manual intervention, imposing a substantial burden on clinicians, while rationale-based generative approaches often select concepts by class discriminability, which can drift from diagnostic ontologies. To address these issues, we propose Neuro-Symbolic Rule Distillation (NeRD), a framework that produces efficient, ontology-grounded reasoning chains that are sufficient yet non-redundant, without manually crafting diagnostic rules. Experiments on two skin datasets demonstrate strong diagnostic performance and interpretability, and blinded expert evaluation confirms the clinical plausibility of NeRD rationales. Our method further enables a first expert-in-the-loop study for Multimodal Chain-of-Thought-based diagnosis, achieving efficient and effective concept-level intervention.

Chinese Translation

可解释性对于可信赖的医疗图像诊断至关重要。然而，现有的以概念驱动的可解释方法存在关键局限性：概念瓶颈模型（Concept Bottleneck Models, CBMs）在推理时需要对所有预定义概念进行评分，并且需要人工干预，这给临床医生带来了巨大的负担，而基于理由的生成方法通常通过类别可区分性选择概念，这可能与诊断本体偏离。为了解决这些问题，我们提出了神经符号规则蒸馏（Neuro-Symbolic Rule Distillation, NeRD），该框架生成高效的、本体基础的推理链，这些推理链是充分的但不冗余的，无需手动构建诊断规则。在两个皮肤数据集上的实验表明，NeRD在诊断性能和可解释性方面表现出色，盲评专家的评估确认了NeRD推理的临床合理性。我们的方法进一步支持了首个专家参与的多模态思维链基础诊断研究，实现了高效且有效的概念级干预。

View on arXiv Download PDF AI Translation

cs.CV / 119 / 2606.15629

XPASS-Vis: A Dataset for Cross-Domain Personalized Image Aesthetic Assessment

XPASS-Vis：跨领域个性化图像美学评估数据集

Hayashi, Takato, Takahara, Hiroaki, Mawalim, Candy Olivia, Narimatsu, Hiromi, Kimura, Akisato, Kumano, Shiro, Okada, Shogo

Abstract

Personalized image aesthetic assessment (PIAA) seeks to model, at the individual level, the subjective nature of aesthetic judgments toward artworks and photographs. Aesthetic preference is known to be both deeply personal and partially consistent across visual domains. Yet existing PIAA datasets and methods are largely confined to a single domain, or provide too few samples per annotator within each domain to enable personalization across domains. Consequently, the cross-domain generalization of personalized aesthetic preferences remains largely unexplored. To address this gap, we introduce XPASS-Vis, the first dataset explicitly designed for cross-domain PIAA. XPASS-Vis comprises 6,526 stimuli from three visual domains -- art, fashion, and landscape -- rated by 129 annotators, yielding 87,836 user-stimulus interactions, each annotated with an overall aesthetic score and nine aesthetic-emotion ratings. Notably, each annotator rated more than 200 stimuli per domain, providing sufficient per-domain coverage to support personalization both within and across domains. Moreover, we establish baseline models for cross-domain PIAA under unsupervised domain adaptation (UDA), where a model trained on a labeled source domain is transferred to an unlabeled target domain. A systematic evaluation of representative UDA approaches shows that the best-performing method recovers approximately 60\% (Spearman's $\rho$ = .28) of the supervised upper bound under a fully unsupervised setting. This provides encouraging evidence that personalized aesthetic preferences are, to a meaningful extent, transferable across visual domains. At the same time, a substantial gap remains, highlighting the need for PIAA-specific adaptation strategies. XPASS-Vis and the accompanying baselines provide a foundation for future research on cross-domain PIAA. All datasets and code will be made publicly available upon acceptance.

Chinese Translation

个性化图像美学评估（PIAA）旨在从个体层面建模对艺术作品和照片的美学判断的主观性质。美学偏好被认为既是深具个人性的，同时在视觉领域间也有部分一致性。然而，现有的PIAA数据集和方法大多局限于单一领域，或在每个领域内为每位注释者提供的样本过少，无法支持跨领域的个性化。因此，个性化美学偏好的跨领域泛化仍然在很大程度上未被探索。为了解决这一问题，我们推出了XPASS-Vis，这是第一个专门为跨领域PIAA设计的数据集。XPASS-Vis包含来自三个视觉领域（艺术、时尚和风景）的6,526个刺激，由129位注释者进行评分，产生了87,836个用户-刺激交互，每个交互都附有一个总体美学评分和九个美学情感评分。值得注意的是，每位注释者在每个领域内评分超过200个刺激，提供了足够的领域覆盖，以支持领域内和跨领域的个性化。此外，我们在无监督领域适应（UDA）下建立了跨领域PIAA的基线模型，其中在标记源领域上训练的模型被转移到未标记的目标领域。对代表性UDA方法的系统评估表明，表现最佳的方法在完全无监督的设置下恢复了约60\%（Spearman's $ ho$ = .28）的监督上限。这为个性化美学偏好在视觉领域间具有一定程度的可转移性提供了令人鼓舞的证据。同时，仍然存在显著的差距，突显了针对PIAA的特定适应策略的必要性。XPASS-Vis及其附带的基线为未来跨领域PIAA的研究提供了基础。所有数据集和代码将在接受后公开发布。

View on arXiv Download PDF AI Translation

cs.CV / 120 / 2606.15632

Open-World Video Segmentation

开放世界视频分割

Su, Qing, Li, Kaiyang, Zhuang, Yuan, Miao, Fei, Ji, Shihao

Abstract

While video segmentation has advanced rapidly on short clips and closed-set benchmarks, open-world video segmentation remains largely unexplored. The challenge is twofold: (1) existing methods are not designed to support object discovery and identity maintenance in long videos of dynamic ego-motion, and (2) existing evaluation protocols rely on a rigid 1:1 matching that unfairly penalizes semantically valid predictions with mismatched granularity. To address both gaps, we introduce Savvy, a practical and strong system for zero-shot open-world long-horizon video segmentation. Savvy combines hierarchical mask discovery, deferred admission, and track consolidation to support persistent object discovery, safe track promotion, and stable long-range identity maintenance. We further propose OGA, a granularity-aware evaluation suite for open-world video segmentation. Built on a Granularity-Agnostic (GA) matching protocol, OGA relaxes conventional 1:1 matching to an n:1 mapping, but still enforces temporal rigor by detecting support discontinuities through sever points and scoring each reference object through its dominant coherent fragment. This prevents fragmented or flickering support from being over-rewarded while enabling GA-adapted metrics and structural diagnostics: identity persistence (IP), and identity concentration (IC). On VIPSeg, we show that standard 1:1 evaluation substantially underestimates open-world methods, whereas GA evaluation recovers much of their suppressed performance. On the more realistic long-horizon benchmarks: ScanNet and HM3D, Savvy consistently outperforms strong baselines across both classical and proposed metrics, including STQ, VPQ$_\infty$, IP and IC. Together, these results establish a practical benchmark and a strong baseline for open-world long-horizon video segmentation.

Chinese Translation

尽管视频分割在短片段和封闭集基准测试上取得了快速进展，但开放世界视频分割仍然在很大程度上未被探索。其挑战主要有两个方面：（1）现有方法并未设计用于支持在动态自我运动的长视频中进行对象发现和身份维护；（2）现有评估协议依赖于严格的1:1匹配，这不公平地惩罚了语义上有效但粒度不匹配的预测。为了解决这两个问题，我们提出了Savvy，一个实用且强大的零样本开放世界长视野视频分割系统。Savvy结合了分层掩码发现、延迟接纳和轨迹整合，以支持持久的对象发现、安全的轨迹提升和稳定的长距离身份维护。我们进一步提出了OGA，一个粒度感知的开放世界视频分割评估套件。OGA基于粒度无关（Granularity-Agnostic, GA）匹配协议，放宽了传统的1:1匹配至n:1映射，但仍通过检测支持不连续性（sever points）来强制执行时间上的严格性，并通过其主导一致片段对每个参考对象进行评分。这防止了碎片化或闪烁支持被过度奖励，同时启用了GA适应的指标和结构诊断：身份持久性（Identity Persistence, IP）和身份集中度（Identity Concentration, IC）。在VIPSeg上，我们展示了标准的1:1评估显著低估了开放世界方法，而GA评估则恢复了它们被压制的性能。在更现实的长视野基准测试ScanNet和HM3D上，Savvy在经典和提出的指标（包括STQ、VPQ$_ ext{∞}$、IP和IC）上始终优于强基线。这些结果共同建立了开放世界长视野视频分割的实用基准和强基线。

View on arXiv Download PDF AI Translation

cs.CV / 121 / 2606.15648

Fusing Transferred Priors and Physics-based Decomposition for Underwater Image Enhancement

融合转移先验与基于物理的分解用于水下图像增强

Hu, Haochen, Bin, Yanrui, Zhang, Zhengyan, Wei, Minchen, Wen, Chih-yung, Wang, Bing

Abstract

The underwater images are captured within diverse water-medium conditions, leading to complex degradation, including color bias, low contrast, and blur effect. Recently, learning-based methods have demonstrated their potential for underwater image enhancement (UIE). However, most of the previous work focus on the training strategy or network design to make the enhanced result aligned well with the labels in datasets, ignoring that the labels are selected from the enhanced results of previous UIE methods and these pseudo-labels are noisy. Consequently, the performance of their models is not satisfactory to a certain extent. However, collecting the true labels of the underwater images is challenging. In this work, we propose a transfer learning-based UIE that does not require underwater images to have paired noisy or true labels for learning. Instead, the UIE task is first divided into global color correction, haze removal, and background noise suppression following the underwater physics. Then multiple types of prior from other vision tasks are leveraged as cross-domain supervision in each step. In this way, a novel UIE is available via transfer learning, and the physics-aligned UIE decomposition provides theoretical soundness. Qualitative and quantitative experiments demonstrate that our proposal based on physics and priors fusion achieves SOTA performance in the UIE task and effectively boosts downstream vision tasks, significantly outperforming benchmark methods. Project repo: https://github.com/Haru2022/P2-UIE.

Chinese Translation

水下图像在多种水介质条件下捕获，导致复杂的退化现象，包括色彩偏差、低对比度和模糊效果。最近，基于学习的方法展示了其在水下图像增强（UIE）中的潜力。然而，以往的大多数研究集中于训练策略或网络设计，以使增强结果与数据集中的标签良好对齐，忽视了标签是从先前UIE方法的增强结果中选择的，这些伪标签是噪声的。因此，他们模型的性能在某种程度上并不令人满意。然而，收集水下图像的真实标签是具有挑战性的。在本研究中，我们提出了一种基于转移学习的UIE方法，该方法不需要水下图像具有配对的噪声或真实标签进行学习。相反，UIE任务首先根据水下物理学被划分为全局色彩校正、雾霾去除和背景噪声抑制。然后，在每个步骤中利用来自其他视觉任务的多种类型的先验作为跨领域监督。通过这种方式，基于转移学习的新型UIE得以实现，基于物理的UIE分解提供了理论上的合理性。定性和定量实验表明，我们基于物理和先验融合的提案在UIE任务中达到了最先进的性能，并有效提升了下游视觉任务，显著超越了基准方法。项目仓库：https://github.com/Haru2022/P2-UIE。

View on arXiv Download PDF AI Translation

cs.CV / 122 / 2606.15651

Self-Questioning Vision-Language Models: Reinforcement Learning for Compositional Visual Reasoning

自我提问的视觉-语言模型：用于组合视觉推理的强化学习

Amjith, Saraswathy

Abstract

Vision-Language Models (VLMs) are AI systems that process both images and text, yet they often struggle with compositional visual reasoning questions that require chaining multiple steps together, such as identifying objects, counting them, and comparing the results. Existing approaches improve this reasoning by training models on human-written step-by-step explanations, but creating these annotations is expensive and difficult to scale. We propose a self-questioning framework that trains a VLM to break visual questions into smaller sub-questions and answer each one before producing a final response, using a reinforcement learning algorithm called Group Relative Policy Optimization (GRPO). The model is never shown examples of how to decompose questions, it discovers this behavior on its own, guided by a reward signal that scores whether the output contains sub-questions and whether the final answer is correct. We apply this framework to a 3-billion-parameter model, training on both synthetic scenes of geometric shapes (CLEVR) and real-world photographs (A-OKVQA). On A-OKVQA, both self-questioning and standard reinforcement learning substantially improve accuracy over the untrained model (52.2% and 51.6% vs. 46.8%). We introduce the first self-questioning VLM by rewarding not only the final answer like standard RL but additionally for generating intermediate sub-questions, enabling it to discover compositional decomposition strategies. These results suggest that teaching AI systems to ask themselves intermediate questions is a promising strategy for complex visual reasoning, particularly when the difficulty of a question warrants explicit step-by-step decomposition.

Chinese Translation

视觉-语言模型（VLMs）是处理图像和文本的人工智能系统，但它们在处理需要多个步骤连接的组合视觉推理问题时往往表现不佳，例如识别物体、计数以及比较结果。现有的方法通过在人工撰写的逐步解释上训练模型来改善这种推理，但创建这些注释既昂贵又难以扩展。我们提出了一种自我提问框架，训练VLM将视觉问题分解为更小的子问题，并在生成最终回答之前回答每一个子问题，使用一种名为组相对策略优化（Group Relative Policy Optimization, GRPO）的强化学习算法。模型从未被展示如何分解问题的示例，而是通过奖励信号自主发现这一行为，该信号评估输出是否包含子问题以及最终答案是否正确。我们将此框架应用于一个具有30亿参数的模型，训练数据包括几何形状的合成场景（CLEVR）和真实世界的照片（A-OKVQA）。在A-OKVQA上，自我提问和标准强化学习显著提高了模型的准确性（分别为52.2%和51.6%，而未训练模型为46.8%）。我们引入了第一个自我提问的VLM，不仅像标准强化学习那样奖励最终答案，还额外奖励生成中间子问题，使其能够发现组合分解策略。这些结果表明，教导人工智能系统自我提问中间问题是一种有前景的复杂视觉推理策略，尤其是在问题的难度需要明确的逐步分解时。

View on arXiv Download PDF AI Translation

cs.CV / 123 / 2606.15659

SpatialAvatar-0: High-Quality 4D Head Avatar with Multi-Stage Reconstruction

SpatialAvatar-0：高质量的4D头部头像与多阶段重建

Wang, Yiran, Zhang, Zeyu, Li, Yuanming, Wang, Ziming, Zhao, Yang

Abstract

High-quality 4D head avatars from one or a few source portraits are central to telepresence, AR/VR, and digital-human interaction. 3D Gaussian Splatting (3DGS) has emerged as the dominant representation, with two complementary regimes (generalizable feed-forward predictors and per-subject refiners) maturing in parallel. However, existing feed-forward predictors are trained on a single dataset family with a hard-coded source count, inheriting the corresponding domain bias. Per-subject refiners require 300K--600K iterations and rely on adaptive densification that destroys upstream Gaussian layouts, preventing the two regimes from sharing a representation end-to-end. To bridge both regimes we propose SpatialAvatar-0 on a shared FLAME-mesh-bound Gaussian representation: a feed-forward generator with a parameter-free K-source mean-pool and a monocular-temporal to multi-view-spatial two-phase schedule that anchors against identity-prior collapse onto the smaller multi-view set. We further introduce a 10K-iter layout-preserving per-subject refinement loop that freezes the FLAME-binding and Gaussian count and replaces densification with a three-component anti-spike regularization. On VFHQ/HDTF cross-domain zero-shot we surpass the in-domain leader GAGAvatar by +1.5 dB PSNR despite never training on either test domain, and on the SplattingAvatar monocular benchmark we lead every reported metric, surpassing the 300K-iter GeoAvatar by +1.3 dB PSNR at up to 60x shorter per-subject schedule than common SOTA baselines. Website: https://spatialwalk.github.io/SpatialAvatar-0.

Chinese Translation

高质量的4D头部头像源自一张或几张源肖像，对于远程呈现、增强现实/虚拟现实以及数字人类交互至关重要。3D高斯点云（3D Gaussian Splatting，3DGS）已成为主流表示方法，具有两种互补的模式（可泛化的前馈预测器和每个对象的细化器）并行成熟。然而，现有的前馈预测器是在单一数据集家族上训练的，且源数量是硬编码的，因此继承了相应的领域偏差。每个对象的细化器需要300K到600K次迭代，并依赖于自适应稠密化，这会破坏上游的高斯布局，阻碍了这两种模式在端到端共享表示。为了弥合这两种模式，我们提出了SpatialAvatar-0，基于共享的FLAME网格绑定高斯表示：一个具有无参数K源均值池的前馈生成器，以及一个单目-时间到多视角-空间的两阶段调度，旨在防止身份优先崩溃到较小的多视角集合。我们进一步引入了一个10K迭代的布局保持每个对象细化循环，该循环冻结了FLAME绑定和高斯数量，并用三组反尖峰正则化替代了稠密化。在VFHQ/HDTF跨领域零样本测试中，我们的表现超越了领域内领导者GAGAvatar，PSNR提升了1.5 dB，尽管从未在任何测试领域进行训练；而在SplattingAvatar单目基准测试中，我们在每个报告的指标上均领先，PSNR比300K迭代的GeoAvatar高出1.3 dB，且每个对象的调度时间比常见的SOTA基线短60倍。网站：https://spatialwalk.github.io/SpatialAvatar-0。

View on arXiv Download PDF AI Translation

cs.CV / 124 / 2606.15663

OneFocus: Enabling Real-World X-ray Security Screening with a Unified Vision-Language Model

OneFocus：通过统一的视觉-语言模型实现现实世界的X射线安全筛查

Wen, Jiali, Gao, Hongxia, Li, Litao, Chen, Yixin, Zhang, Kaijie, Liu, Qianyun, Wen, Xiaoqin

Abstract

X-ray contraband detection is critical for security in large-scale logistics and transportation, yet conventional detectors struggle to adapt to emerging contraband types and lack fundamental visual understanding. Vision-language models (VLMs) offer strong generalization but are hindered by the scarcity of high-quality X-ray image-caption data. To bridge this critical gap, we present MMXray, a meticulously curated benchmark of 52,124 image-caption pairs spanning 28 fine-grained classes of X-ray contraband. To enrich MMXray with realistic occlusion patterns, we further introduce CleanDET, a dedicated synthesis dataset containing clean foreground contraband images from 28 categories and background images with diverse density levels, together with AnyContraSyn, a controllable synthesis method designed to operate on CleanDET. We also develop OnePipe, an extensible pipeline for systematic data curation. Built on MMXray, we propose OneFocus, a unified VLM that supports four core tasks: visual question answering, contraband localization, classification, and image understanding. OneFocus achieves state-of-the-art performance in X-ray contraband understanding and demonstrates robust cross-domain generalization, establishing a strong vision-language baseline for security screening.

Chinese Translation

X射线违禁品检测对于大规模物流和运输的安全至关重要，然而传统检测器在适应新兴违禁品类型方面存在困难，并且缺乏基本的视觉理解。视觉-语言模型（VLMs）具有强大的泛化能力，但受到高质量X射线图像-文本数据稀缺的限制。为了解决这一关键问题，我们提出了MMXray，这是一个精心策划的基准数据集，包含52,124对图像-文本对，涵盖28个细粒度的X射线违禁品类别。为了丰富MMXray的真实遮挡模式，我们进一步引入了CleanDET，这是一个专门的合成数据集，包含来自28个类别的干净前景违禁品图像和具有不同密度水平的背景图像，以及AnyContraSyn，这是一种旨在在CleanDET上操作的可控合成方法。我们还开发了OnePipe，这是一个可扩展的数据策划系统。基于MMXray，我们提出了OneFocus，一个统一的VLM，支持四个核心任务：视觉问答、违禁品定位、分类和图像理解。OneFocus在X射线违禁品理解方面达到了最先进的性能，并展示了强大的跨领域泛化能力，为安全筛查建立了一个强有力的视觉-语言基准。

View on arXiv Download PDF AI Translation

cs.CV / 125 / 2606.15667

CEVAR: Centerline Embedding Extraction for Endovascular Aneurysm Repair

CEVAR：用于血管内动脉瘤修复的中心线嵌入提取

Naeem, Roman, Niiniskorpi, Timo, Sandström, Charlotte, Desai, Naman, Jeppsson, Anders, Häggström, Ida, Kahl, Fredrik, Roos, Håkan, Alvén, Jennifer

Abstract

Long-term mortality rates after endovascular aneurysm repair (EVAR) remain elevated due to post-EVAR rupture caused by loss of seal in stent graft sealing zones. Structured CT review using centerline measurements improves detection, but current workflows require manual centerline editing and expert operators. We propose a transformer framework for automated, protocol-driven sealing zone assessment that combines 3D centerline tracking with embedding-based geometric prediction. Two state-of-the-art image-to-graph models are evaluated for aorto-iliac centerline extraction from follow-up CT and for measurement of stent position, vessel diameters, and seal lengths according to EVAR4C protocol. Across the full test set and a challenging no-contrast subset, the proposed fully automatic method outperforms the commercial semi-automatic workflow.

Chinese Translation

血管内动脉瘤修复（EVAR）后的长期死亡率由于支架移植物密封区的密封失效导致的EVAR后破裂而持续升高。使用中心线测量的结构化CT审查可以改善检测，但目前的工作流程需要手动编辑中心线和专家操作员。我们提出了一种变换器框架，用于自动化、协议驱动的密封区评估，该框架结合了3D中心线跟踪和基于嵌入的几何预测。评估了两种最先进的图像到图模型，用于从随访CT中提取主动脉-髂动脉中心线，并根据EVAR4C协议测量支架位置、血管直径和密封长度。在完整测试集和具有挑战性的无对比剂子集中，所提出的完全自动化方法优于商业半自动工作流程。

View on arXiv Download PDF AI Translation

cs.CV / 126 / 2606.15681

3D Consistency Optimization for Self-Supervised Monocular Video Depth Estimation

自监督单目视频深度估计的三维一致性优化

Liu, Yuanye, Zhang, Ke, Jiang, Junzhe, Zhang, Li, Patel, Vishal, Zhuang, Xiahai

Abstract

Reliable monocular video depth estimation is crucial for downstream 3D reasoning and embodied AI in endoscopic navigation. However, existing self-supervised approaches typically treat video frames independently or rely on weak temporal regularization. These methods, lacking a holistic perception of the underlying 3D scene, inevitably suffer from geometrically inconsistent predictions and severe cross-frame drift. To address these limitations, we introduce a new paradigm that recasts sequential video depth estimation as an unconstrained multi-view 3D reconstruction problem, enabling full exploitation of the powerful geometric priors embedded in recent 3D foundation models. The core of our approach is a 3D consistency optimization framework driven by three constraints: image-level photometric rendering, explicit world-coordinate geometric alignment, and multi-scale temporal gradient consistency. Such unified optimization elegantly anchors isolated frames to a globally coherent 3D structure. Our method has been validated in both the self-supervised training scenarios and challenging zero-shot clinical environments. Results show that the proposed approach achieves state-of-the-art spatial accuracy, outperforming the frame-based, video-based depth estimators and the multi-view 3D reconstruction baselines.

Chinese Translation

可靠的单目视频深度估计对于内窥镜导航中的下游三维推理和具身人工智能至关重要。然而，现有的自监督方法通常独立处理视频帧或依赖于弱时间规律化。这些方法缺乏对潜在三维场景的整体感知，必然导致几何不一致的预测和严重的跨帧漂移。为了解决这些局限性，我们提出了一种新的范式，将顺序视频深度估计重新构建为一个无约束的多视图三维重建问题，从而充分利用最近的三维基础模型中嵌入的强大几何先验。我们方法的核心是一个由三个约束驱动的三维一致性优化框架：图像级光度渲染、显式世界坐标几何对齐和多尺度时间梯度一致性。这种统一的优化优雅地将孤立的帧锚定到一个全局一致的三维结构上。我们的方法在自监督训练场景和具有挑战性的零样本临床环境中得到了验证。结果表明，所提出的方法实现了最先进的空间精度，超越了基于帧的、基于视频的深度估计器和多视图三维重建基线。

View on arXiv Download PDF AI Translation

cs.CV / 127 / 2606.15749

OmniTraffic: A Controllable Generation Pipeline and Benchmark for Spatio-Temporal Traffic Reasoning

OmniTraffic：一个可控的生成管道和时空交通推理基准

Wang, Maonan, Huang, Zhengyan, Jiang, Kemou, Fu, Yuhang, Zhu, Jiayue, Cai, Yuxin, Zou, Xingchen, Zhang, Qiaosheng, Yu, Yi, Wang, Ding, Chen, Xi, Chen, Ben M., Liang, Yuxuan, Cui, Zhiyong, Pun, Man On, Chen, Yirong

Abstract

Traffic scene understanding requires models to reason beyond object recognition, including lane topology, multi-view geometry, temporal evolution, and signal-phase semantics. However, existing traffic-oriented multimodal benchmarks largely emphasize passive visual recognition or isolated video understanding, offering limited support for evaluating structure-aware traffic reasoning under controlled conditions. We introduce OmniTraffic, a controllable generation pipeline and benchmark for spatio-temporal traffic reasoning. Built around 12 real-world intersections reconstructed into editable 3D traffic environments and complemented by surveillance footage from two countries, OmniTraffic supports both controlled and natural-condition evaluation. It defines a three-level task hierarchy spanning scene perception, multi-view and temporal reasoning, and decision support. Using structured traffic metadata, OmniTraffic generates synchronized multi-view VQA samples covering vehicle states, lane functions, view--BEV correspondence, temporal dynamics, and signal-phase analysis, resulting in 8M VQA samples and a 3K human-verified test set. Evaluation of eleven frontier MLLMs reveals a large human--model gap, with the most pronounced failures in topology-grounded and spatio-temporal reasoning tasks. Fine-tuning a lightweight MLLM on simulated OmniTraffic data further improves performance on real-world traffic scenes, demonstrating the value of simulation-generated supervision for traffic-specific multimodal reasoning. Beyond a fixed dataset, OmniTraffic provides an extensible pipeline with configurable intersections, camera views, traffic demands, signal phases, visual conditions, and rare events.

Chinese Translation

交通场景理解要求模型超越对象识别进行推理，包括车道拓扑、多视角几何、时间演变和信号相位语义。然而，现有的交通导向多模态基准主要强调被动的视觉识别或孤立的视频理解，提供的支持有限，无法在受控条件下评估结构感知的交通推理。我们提出了OmniTraffic，一个用于时空交通推理的可控生成管道和基准。该基准围绕12个重建为可编辑3D交通环境的真实世界交叉口构建，并辅以来自两个国家的监控视频，支持受控和自然条件下的评估。它定义了一个跨越场景感知、多视角和时间推理以及决策支持的三级任务层次结构。利用结构化的交通元数据，OmniTraffic生成覆盖车辆状态、车道功能、视图-鸟瞩视图（BEV）对应关系、时间动态和信号相位分析的同步多视角视觉问答（VQA）样本，最终生成800万VQA样本和3000个经过人工验证的测试集。对十一种前沿多模态大语言模型（MLLM）的评估显示出显著的人类-模型差距，尤其在基于拓扑和时空推理任务中表现最为明显。在模拟的OmniTraffic数据上对轻量级MLLM进行微调，进一步提升了其在真实交通场景中的表现，证明了模拟生成的监督在交通特定多模态推理中的价值。除了固定数据集，OmniTraffic还提供了一个可扩展的管道，支持可配置的交叉口、摄像机视角、交通需求、信号相位、视觉条件和稀有事件。

View on arXiv Download PDF AI Translation

cs.CV / 128 / 2606.15763

The Circumplex Degeneracy Behind the Rare-Class Limit in Affect Recognition

情感识别中稀有类别限制背后的圆周退化

Huynh, Van Thong, Nguyen, Hong Hai, Kim, Soo-Hyung

Abstract

In-the-wild expression recognition persistently fails on a few rare emotions, and the standard explanation is class imbalance. Through a controlled multi-task study on two benchmarks, we show the failure is instead a property of affect geometry: the rare classes are degenerate on Russell's circumplex, and that degeneracy bounds what any loss or cost can achieve. Our instrument is a circumplex-cost optimal-transport term that prices expression confusions by their valence-arousal distance. The term improves the official score and expression macro-F1, but a control most studies omit shows the gain is not geometric: a uniform cost, equivalent to a generic confidence penalty, matches it on Aff-Wild2 (p=0.625) and significantly exceeds it on AffectNet (+0.057 over base, larger than the circumplex). What the geometry reshapes is the structure of the errors, making them affectively nearer the truth on Aff-Wild2 (p=0.031 against the uniform control), an effect that does not survive on AffectNet, where a visual confound at the far corner of the circumplex overwhelms it. The rare-class failure, by contrast, is stable across both datasets we examine: the degenerate pairs (anger-fear on Aff-Wild2, anger-contempt on AffectNet) resist frequency-based interventions, the transport term, and an action-unit-augmented cost built specifically to separate them. We conclude that progress on rare expressions requires representations that distinguish the classes, not supervision that reprices their confusions, and we provide the controls and metrics needed to tell the two apart.

Chinese Translation

在真实环境中的表情识别在一些稀有情感上持续失败，标准解释是类别不平衡。通过对两个基准的受控多任务研究，我们表明这种失败实际上是情感几何的一个特性：稀有类别在拉塞尔的圆周上是退化的，这种退化限制了任何损失或成本所能达到的效果。我们的工具是一个圆周成本最优传输项，它根据表情混淆的效价-唤醒距离来定价。该项改善了官方得分和表情宏观F1，但大多数研究忽略的一个对照显示，增益并不是几何上的：一个均匀成本，相当于一个通用的置信惩罚，在Aff-Wild2上与之匹配（p=0.625），并在AffectNet上显著超过它（比基线高出+0.057，超过圆周的增益）。几何重塑的是错误的结构，使其在Aff-Wild2上在情感上更接近真实（与均匀对照相比p=0.031），但这一效应在AffectNet上并未持续，因为圆周远角的视觉混淆压倒了它。相比之下，稀有类别的失败在我们检查的两个数据集中都是稳定的：退化对（Aff-Wild2上的愤怒-恐惧，AffectNet上的愤怒-轻蔑）抵抗基于频率的干预、传输项以及专门构建的用于区分它们的动作单元增强成本。我们得出结论，稀有表情的进展需要能够区分类别的表示，而不是重新定价其混淆的监督，我们提供了区分两者所需的对照和指标。

View on arXiv Download PDF AI Translation

cs.CV / 129 / 2606.15765

Task-Instructed Causal Routing of Vision Foundation Models for Multi-Task Learning

任务指导的视觉基础模型因果路由用于多任务学习

Han, Donghyun, Bae, Yuseok, Kim, Jung Uk, Kim, Hyung-Il

Abstract

Vision foundation models (VFMs) have demonstrated strong robustness and transferability across a wide range of visual tasks. However, each model typically encodes strong inductive biases shaped by its pre-training objective and data domain, resulting in fragmented yet complementary visual knowledge. As a result, a single model often struggles to capture the diverse visual representations required across multiple dense prediction tasks. To address this limitation, we propose TIGER (Task-Instruction-Guided Expert Routing), a framework that coordinates multiple heterogeneous VFMs for multi-task dense prediction. Instead of naively aggregating expert features, TIGER leverages natural-language task instructions to guide a routing network that assigns token-level expert weights conditioned on task semantics, enabling adaptive integration of complementary expert features. TIGER further introduces a counterfactual loss that aligns routing decisions with each expert's causal contribution by measuring prediction changes when experts are excluded, encouraging more reliable and interpretable routing. We evaluate TIGER on two multi-task dense prediction benchmarks, NYUD-v2 and Pascal Context, where it consistently outperforms recent multi-task learning baselines while keeping all VFMs frozen. These results demonstrate that combining instruction-guided expert routing with counterfactual causal alignment enables effective coordination of heterogeneous vision foundation models.

Chinese Translation

视觉基础模型（VFM）在多种视觉任务中表现出强大的鲁棒性和迁移能力。然而，每个模型通常编码了由其预训练目标和数据领域塑造的强烈归纳偏差，导致视觉知识碎片化但互补。因此，单一模型往往难以捕捉在多个密集预测任务中所需的多样化视觉表征。为了解决这一限制，我们提出了TIGER（任务指令引导的专家路由），这是一个协调多个异构VFM进行多任务密集预测的框架。TIGER并不是简单地聚合专家特征，而是利用自然语言任务指令来引导一个路由网络，该网络根据任务语义分配基于token级别的专家权重，从而实现互补专家特征的自适应整合。TIGER进一步引入了一种反事实损失，通过测量排除专家时预测变化来对齐路由决策与每个专家的因果贡献，从而鼓励更可靠和可解释的路由。我们在两个多任务密集预测基准NYUD-v2和Pascal Context上评估了TIGER，结果表明其在保持所有VFM不变的情况下，始终优于最近的多任务学习基线。这些结果表明，将指令引导的专家路由与反事实因果对齐相结合，可以有效协调异构视觉基础模型。

View on arXiv Download PDF AI Translation

cs.CV / 130 / 2606.15772

Ellipse Meets Bit-Planes: A Novel Approach to RNFL based Glaucoma Detection Using Advanced Image Processing and Deep Learning

椭圆与位平面相遇：一种基于RNFL的青光眼检测新方法，采用先进的图像处理和深度学习

Paul, Snigdha, Mallick, Sambit, Sen, Anindya

Abstract

This work proposes an integrated pipeline for automatic glaucoma detection method from easily available colour fundas images based on an adaptive algorithm for ellipse-based polar transformation, to enhance the analysis of the Retinal Nerve Fiber Layer (RNFL) as the primary biomarker for observing glaucomatous changes, regardless of optic disc and macula position. Utilizing this transformation, we introduce two distinct frameworks tailored to different operational needs. The first framework, a deep learning-inspired feature fusion approach, achieves a 99.3% detection rate, ideal for settings where high precision is essential, despite higher computational demands. The second framework employs a novel image-processing algorithm based on bit-plane slicing, offering 92.31% accuracy and optimized for environments requiring rapid inference with minimal resource consumption. Both frameworks provide scalable and cost-effective solutions for early glaucoma detection. This study highlights the potential of RNFL-based diagnostic tools in addressing the global challenge of glaucoma, particularly in underserved regions.

Chinese Translation

本研究提出了一种集成管道，用于从易于获取的彩色眼底图像中自动检测青光眼，基于一种自适应算法进行椭圆基极坐标变换，以增强视网膜神经纤维层（RNFL）的分析，作为观察青光眼变化的主要生物标志物，而不受视神经盘和黄斑位置的影响。利用这种变换，我们引入了两个针对不同操作需求的框架。第一个框架是一种受深度学习启发的特征融合方法，达到了99.3%的检测率，适用于对高精度要求较高的环境，尽管计算需求较高。第二个框架采用基于位平面切片的新型图像处理算法，提供92.31%的准确率，并优化用于需要快速推理且资源消耗最小的环境。这两个框架为早期青光眼检测提供了可扩展且具成本效益的解决方案。本研究强调了基于RNFL的诊断工具在应对全球青光眼挑战中的潜力，尤其是在服务不足的地区。

View on arXiv Download PDF AI Translation

cs.CV / 131 / 2606.15779

Faithful Action-unit Causal Reasoning for Counterfactually Faithful Emotion Explanations

忠实的动作单元因果推理用于反事实忠实的情感解释

Huynh, Van Thong, Nguyen, Hong Hai, Pham, Thuy, Nguyen, Trong Nghia, Kim, Soo-Hyung

Abstract

Multimodal models can name the action units (AUs) behind a facial emotion, but their AU->emotion rationales are typically plausible rather than faithful: nothing forces the AUs a model invokes to be the AUs that actually drive its prediction. We cast AU->emotion reasoning as a counterfactual-consistency problem between the rationale, the label, and a structural AU->emotion causal graph G, and propose FACR, which grounds the reasoner in an independently induced, polarity-aware G and trains a counterfactual-faithfulness objective: a do-intervention on an AU that G marks causal for a class must move the prediction, while one it marks irrelevant must leave it unchanged. Faithfulness is thereby both trainable and measurable through a matching interventional metric, which we evaluate against a known causal structure, the PSPI pain-AU composition, as no existing affective-reasoning benchmark allows. We are explicit that this metric tests fidelity to the supplied structure rather than its rediscovery: it asks whether the trained reasoner invokes the AUs the structure marks causal, on held-out subjects and a second dataset. Under subject-independent evaluation on UNBC-PAIN, the objective raises the agreement between the invoked AUs and the PSPI composition from a no-objective baseline of 0.08 to 0.57, at a small detection cost; an unfaithfulness control attributes the gain to the objective. On a cross-dataset emotion transfer, the objective likewise raises fidelity to G on a seven-class task (0.50 to 0.84). Finally, we attach a language verbalizer and extend the audit to the generated text: biasing each action unit's emission by its latent activation makes the rationale faithful by construction, so that ablating an AU removes it from the explanation, a property that transfers to a second language-model backbone, whereas a freely generated rationale is unfaithful.

Chinese Translation

多模态模型能够识别面部情感背后的动作单元（AUs），但它们的AU->情感推理通常是合理的而非忠实的：模型所调用的AUs并不一定是实际驱动其预测的AUs。我们将AU->情感推理视为一个反事实一致性问题，涉及推理、标签和结构性AU->情感因果图G，并提出了FACR，该方法将推理者基于独立诱导的、考虑极性的G进行训练，并设定反事实忠实性目标：在G标记为某类别因果的AU上进行的干预必须改变预测，而在G标记为无关的AU上进行的干预则必须保持预测不变。因此，忠实性可以通过匹配干预度量进行训练和测量，我们在已知的因果结构PSPI疼痛-AU组合上进行了评估，因为没有现有的情感推理基准允许这样做。我们明确指出，该度量测试的是对提供结构的忠实性，而不是其重新发现：它询问训练后的推理者是否调用了结构标记为因果的AUs，在保留的受试者和第二个数据集上进行评估。在UNBC-PAIN的受试者独立评估中，该目标将调用的AUs与PSPI组合之间的符合度从没有目标的基线0.08提高到0.57，且检测成本较小；一个不忠实的控制将增益归因于该目标。在跨数据集的情感迁移中，该目标同样提高了在七类任务上对G的忠实性（从0.50提高到0.84）。最后，我们附加了一个语言生成器，并将审核扩展到生成的文本：通过其潜在激活偏向每个动作单元的发射，使得推理在构造上是忠实的，因此消除一个AU会将其从解释中移除，这一特性可以转移到第二个语言模型骨干上，而自由生成的推理则是不忠实的。

View on arXiv Download PDF AI Translation

cs.CV / 132 / 2606.15786

Domain-Guided Prompting of the Segment Anything Model for Seismic Interpretation: The Role of Attributes, Visualization, and Hybrid Prompts

基于领域引导的Segment Anything模型在地震解释中的提示：属性、可视化和混合提示的作用

Ahmad, Aniq, Bedle, Heather, Mustafa, Ahmad

Abstract

The advent of large pretrained foundation models for computer vision has significantly improved the efficiency of visual data interpretation. The Segment Anything Model (SAM), in particular, offers powerful zero shot segmentation capabilities through prompt based interaction, thus making it a promising tool for seismic interpretation. However, most existing applications of SAM rely on fine tuning for specific geological targets, which requires extensive labeled data, incurs high computational cost, and often compromises the model's generalization capability. In this study, we introduce a principled framework for zero shot adaptation of foundation models to seismic data. The framework is built on two key components: (1) aligning seismic attributes and visualization choices (e.g., colormaps) with the geological target of interest, and (2) employing a hybrid prompting strategy that combines sparse user defined point prompts with dense mask prompts derived from SAM's internal feature activations. We systematically evaluate this framework across multiple geological targets, datasets, prompt configurations, and seismic attribute representations. Our results demonstrate that geologic target aware selection of seismic attributes and colormaps, combined with hybrid prompting, enhances the separability of geological features and improves boundary delineation and segmentation accuracy relative to point based prompting alone. Our findings show that, when these components are jointly applied, SAM can achieve competitive segmentation performance in a fully zero shot setting, thereby eliminating the need to retrain SAM for each geologic feature. This work establishes a practical and scalable pathway to leverage foundation models in seismic interpretation, reducing reliance on labeled data while preserving model generality.

Chinese Translation

大型预训练基础模型的出现显著提高了视觉数据解释的效率。特别是，Segment Anything Model (SAM) 通过基于提示的交互提供强大的零样本分割能力，因此成为地震解释的有前景工具。然而，现有大多数SAM应用依赖于针对特定地质目标的微调，这需要大量标记数据，造成高计算成本，并且往往会妨碍模型的泛化能力。在本研究中，我们提出了一个系统化的框架，用于将基础模型零样本适应于地震数据。该框架基于两个关键组件： (1) 将地震属性和可视化选择（例如，色彩图）与感兴趣的地质目标对齐，以及 (2) 采用混合提示策略，将稀疏的用户定义点提示与来自SAM内部特征激活的密集掩模提示相结合。我们在多个地质目标、数据集、提示配置和地震属性表示上系统地评估了该框架。我们的结果表明，关注地质目标的地震属性和色彩图的选择，结合混合提示，增强了地质特征的可分离性，并相较于仅使用基于点的提示，提高了边界划分和分割精度。我们的研究结果显示，当这些组件共同应用时，SAM能够在完全零样本的情况下实现竞争性的分割性能，从而消除对每个地质特征重新训练SAM的需求。这项工作建立了一条实用且可扩展的路径，以利用基础模型进行地震解释，减少对标记数据的依赖，同时保持模型的通用性。

View on arXiv Download PDF AI Translation

cs.CV / 133 / 2606.15796

DifFRACT: Diffusion Feature Reconstruction and Attribution for Circuit Tracing

DifFRACT：用于电路追踪的扩散特征重建与归因

Mazur, Artyom, Konovalova, Nina, Alanov, Aibek

Abstract

Mechanistic interpretability seeks to explain neural network behavior by decomposing model computations into interpretable features and circuits. While transcoder-based circuit tracing has recently enabled detailed causal analyses of large language models, multimodal diffusion transformers for image generation remain comparatively opaque. We still lack tools for understanding how semantic information propagates across denoising steps and how text and image representations interact within double-stream MM-DiT architectures. Existing methods provide only partial insight: attention maps expose a limited view of token interactions, while sparse autoencoders can discover interpretable features but do not directly reveal how these features are transformed and composed through nonlinear MLP layers. In this work, we extend transcoder-based circuit tracing to multimodal diffusion transformers. We train timestep-conditioned transcoders that faithfully approximate the input-output behavior of MLP sublayers in FLUX.1[schnell]. By replacing MLPs with transcoders and linearizing the remaining computation, we obtain exact feature-to-feature attribution and recover compact, interpretable circuits. Empirically, our transcoders match or slightly outperform sparse autoencoders on the sparsity-faithfulness tradeoff. The resulting circuits reveal mechanisms underlying attribute binding and cross-stream semantic propagation, and provide causal explanations for systematic generation errors. Moreover, circuit-guided interventions are substantially more precise and effective than standard SAE-based steering. Our results demonstrate that transcoder-based circuit analysis is feasible for state-of-the-art diffusion transformers and provides a powerful framework for understanding and controlling multimodal generative models. The code is available at https://github.com/Artalmaz31/DifFRACT

Chinese Translation

机制可解释性旨在通过将模型计算分解为可解释的特征和电路来解释神经网络的行为。尽管基于转码器的电路追踪最近使得对大型语言模型的详细因果分析成为可能，但用于图像生成的多模态扩散变换器仍然相对不透明。我们仍然缺乏理解语义信息如何在去噪步骤中传播，以及文本和图像表示如何在双流 MM-DiT 架构中相互作用的工具。现有方法仅提供部分洞察：注意力图揭示了令牌交互的有限视图，而稀疏自编码器可以发现可解释的特征，但并未直接揭示这些特征如何通过非线性 MLP 层进行转换和组合。在本研究中，我们将基于转码器的电路追踪扩展到多模态扩散变换器。我们训练了时间步条件的转码器，这些转码器忠实地近似 FLUX.1[schnell] 中 MLP 子层的输入输出行为。通过用转码器替换 MLP，并线性化剩余计算，我们获得了精确的特征到特征的归因，并恢复了紧凑的可解释电路。实证结果表明，我们的转码器在稀疏性与真实性的权衡上与稀疏自编码器相匹配或略有超越。所得电路揭示了属性绑定和跨流语义传播的机制，并为系统生成错误提供了因果解释。此外，电路引导的干预比基于标准 SAE 的引导更为精确和有效。我们的结果表明，基于转码器的电路分析对于最先进的扩散变换器是可行的，并为理解和控制多模态生成模型提供了强大的框架。代码可在 https://github.com/Artalmaz31/DifFRACT 获取。

View on arXiv Download PDF AI Translation

cs.CV / 134 / 2606.15802

CPS4: Class Prompt driven Semi-Supervised Spine Segmentation with Class-specific Consistency Constraint

CPS4：基于类别提示的半监督脊柱分割与类别特定一致性约束

Pan, Qingtao, Sun, Hongzan, Ji, Bing, Li, Shuo

Abstract

Vision Language Model (VLM) has great potential to enhance the quality of pseudo labels in semi-supervised spine segmentation by leveraging textual class prompts to generate segmentation map, but no one has studied it yet. Although promising, it lacks explicit constraints to ensure consistency between spine class prompts and spine unit region, resulting in unsatisfactory performance in multi-class segmentation map generation. In this paper, we propose CPS4, the first text-guided semi-supervised spine segmentation network using class prompts to enhance the quality of spine pseudo labels. Specifically, CPS4 is implemented through two training stages. (i) Class-specific consistency constrained VLM pretraining stage: we propose token- and pixel-level attention loss to optimize the consistency between class prompts and spine units, forcing the textual class prompt to be closely coupled with the target spine unit in the semantic space. (ii) Class Prompt driven semi-supervised spine segmentation stage: using the pretrained vision-text encoder, we derive each class-specific binary segmentation map for the unlabeled spine image and integrate them into an unified multi-class segmentation map, improving the quality of the spine pseudo label generated by the semi-supervised spine segmentation network. Experimental results show that our CPS4 achieves superior spine segmentation performance with Dice of 80.44%, only using 5% labeled data on the public spine segmentation dataset, surpassing popular semi-supervised learning and VLM methods. Our code will be available.

Chinese Translation

视觉语言模型（VLM）通过利用文本类别提示生成分割图，具有增强半监督脊柱分割中伪标签质量的巨大潜力，但尚未有人对此进行研究。尽管前景可期，但它缺乏明确的约束来确保脊柱类别提示与脊柱单元区域之间的一致性，导致多类别分割图生成的性能不尽如人意。本文提出了CPS4，这是第一个使用类别提示的文本引导半监督脊柱分割网络，以提高脊柱伪标签的质量。具体而言，CPS4通过两个训练阶段实现。（i）类别特定一致性约束的VLM预训练阶段：我们提出了基于标记和像素级的注意力损失，以优化类别提示与脊柱单元之间的一致性，迫使文本类别提示在语义空间中与目标脊柱单元紧密耦合。（ii）基于类别提示的半监督脊柱分割阶段：利用预训练的视觉-文本编码器，我们为未标记的脊柱图像推导出每个类别特定的二进制分割图，并将其整合成统一的多类别分割图，从而提高由半监督脊柱分割网络生成的脊柱伪标签的质量。实验结果表明，我们的CPS4在公共脊柱分割数据集上仅使用5%的标记数据即达到了80.44%的Dice，超越了流行的半监督学习和VLM方法。我们的代码将会公开。

View on arXiv Download PDF AI Translation

cs.CV / 135 / 2606.15819

SACE: Concept Erasure at the Semantic Singularity in Visual Autoregressive Models

SACE：视觉自回归模型中语义奇点的概念抹除

Yang, Siya, Jiang, Nanxiang, Fan, Zhaoxin, Diao, Yunfeng

Abstract

The rapid progress of visual autoregressive (VAR) models has unlocked a transformative frontier for high-fidelity text-to-image synthesis, while heightening concerns over the safety alignment of generated content. Naive application of existing erasure techniques to VAR models causes catastrophic semantic collapse and visual artifacts, since they are predominantly designed for the homogeneous denoising steps of diffusion models. To address this foundational challenge, we first propose the Semantic Singularity Axiom, which posits that any target semantic concept embedded within a prompt is definitively locked at Scale-0. Then rigorously validate this axiom through our proposed Incremental Semantic Saliency Analysis (ISSA),which also enable the community to transparently inspect the coarse-to-fine semantic injection process. Guided by this insight, we introduce the first scale-aware concept erasure framework (SACE) for VAR models. By strictly confining interventions to the first scale, our approach couples an Entropy-Regularized Erasure Objective to prevent high-entropy sampling degeneration, alongside a restorative preservation loss to safely anchor the integrity of entangled benign priors. Extensive experiments demonstrate that our method achieves surgical concept erasure performance across various domains with minimal training overhead, timely and elegently resolute the critical safety vulnerabilities inherent in emerging VAR architectures. Code is available at: https://github.com/limerenceysy/SACE}{https://github.com/limerenceysy/SACE.

Chinese Translation

视觉自回归（VAR）模型的快速进展为高保真文本到图像合成开辟了变革性的前沿，同时也加剧了对生成内容安全对齐的担忧。将现有的抹除技术简单应用于VAR模型会导致灾难性的语义崩溃和视觉伪影，因为这些技术主要是为扩散模型的同质去噪步骤设计的。为了解决这一基础性挑战，我们首先提出了语义奇点公理，该公理认为任何嵌入在提示中的目标语义概念在Scale-0时是明确锁定的。然后，通过我们提出的增量语义显著性分析（Incremental Semantic Saliency Analysis, ISSA）严格验证这一公理，该分析还使社区能够透明地检查粗到细的语义注入过程。在这一洞察的指导下，我们为VAR模型引入了首个规模感知的概念抹除框架（SACE）。通过严格限制干预在第一尺度内，我们的方法结合了熵正则化抹除目标，以防止高熵采样退化，同时引入了恢复性保留损失，以安全地锚定纠缠的良性先验的完整性。大量实验表明，我们的方法在各个领域实现了手术式的概念抹除性能，训练开销最小，及时且优雅地解决了新兴VAR架构中固有的关键安全漏洞。代码可在：https://github.com/limerenceysy/SACE 获取。

View on arXiv Download PDF AI Translation

cs.CV / 136 / 2606.15837

Learning a Sampling-Free Variational DNN Plugin from Tiny Training Sets to Refine OOD Segmentation With Uncertainty Estimation

从小型训练集学习无采样变分深度神经网络插件，以通过不确定性估计优化OOD分割

Pal, Jimut B., Awate, Suyash P.

Abstract

Deep neural networks (DNNs) frequently fail to generalize to out-of-distribution (OOD) medical images because of variations in scanners and acquisition protocols. Retraining DNN models to address these distribution shifts is often impractical due to the high cost of acquiring and annotating new medical datasets. To address this, we introduce VarDeepPCA, a novel lightweight variational DNN framework designed to restore/refine degraded segmentation maps by leveraging intrinsic geometric priors. Unlike existing approaches that require target-domain data or extensive pre-training, our VarDeepPCA explicitly learns a distribution of valid anatomical geometries using only small in-distribution (ID) datasets. Theoretically, our novel variational learning framework leverages a reinterpretation of the softmax mapping to implicitly perform exact distribution modeling, thereby enabling computationally efficient, sampling-free learning and inference. This also enables VarDeepPCA to provide uncertainty estimates associated with its restored segmentation maps. We empirically validate our framework across 4 distinct clinical applications, using 14 publicly available datasets, involving segmentation of the myocardium, neuroretinal rim, prostate, and fetal head. Comparisons against 15 existing methods demonstrate that VarDeepPCA consistently restores segmentation maps produced by the existing methods on OOD data to (i) significantly improve anatomical plausibility of geometries and clinical utility of the segmentations, and (ii) significantly reduce errors, without needing any more training data than that used by existing methods.

Chinese Translation

深度神经网络（DNN）常常无法对分布外（OOD）医学图像进行有效泛化，这主要是由于扫描仪和采集协议的变化。由于获取和标注新的医学数据集的高成本，重新训练DNN模型以应对这些分布变化往往是不切实际的。为了解决这个问题，我们提出了VarDeepPCA，这是一种新颖的轻量级变分DNN框架，旨在通过利用内在几何先验来恢复/优化退化的分割图。与现有方法需要目标领域数据或广泛的预训练不同，我们的VarDeepPCA仅使用小型的分布内（ID）数据集显式学习有效解剖几何的分布。从理论上讲，我们的新型变分学习框架利用对softmax映射的重新解释，隐式执行精确的分布建模，从而实现计算高效的无采样学习和推理。这也使得VarDeepPCA能够提供与其恢复的分割图相关的不确定性估计。我们在4个不同的临床应用中实证验证了我们的框架，使用14个公开可用的数据集，涉及心肌、神经视网膜边缘、前列腺和胎头的分割。与15种现有方法的比较表明，VarDeepPCA始终能够恢复现有方法在OOD数据上生成的分割图，以(i)显著提高几何的解剖合理性和分割的临床实用性，以及(ii)显著减少错误，而无需比现有方法使用的更多训练数据。

View on arXiv Download PDF AI Translation

cs.CV / 137 / 2606.15848

EmoZone-Talker: Regional Semantic Control of Audio-Driven 3DGS Talking Heads via Facial Action Units

EmoZone-Talker：通过面部动作单元实现音频驱动的3D高保真谈话头的区域语义控制

Chen, Tingting, Wang, Shaojun, Zhang, Huaye, Jiang, Diqiong, Chen, Chenglizhao

Abstract

3D Gaussian Splatting (3DGS) has shown strong potential for high-fidelity talking head synthesis. However, enabling fine-grained, interpretable, and editable facial expression control remains fundamentally challenging due to intrinsic conflicts between speech-driven facial dynamics and explicit expression signals. Existing methods rely on implicit multimodal fusion, leading to spatial entanglement and temporal instability. We present EmoZone-Talker, a novel framework that reformulates audio-driven facial animation as a structured spatial-temporal coordination problem under cross-modal conflicts. Our approach introduces an explicit spatial disentanglement and temporal dynamics modeling of facial motion. Specifically, we propose Synergy Zones with Prioritized Attention Bias (SZ-PAB) to explicitly decouple modality contributions via region-wise constraints guided by anatomical priors, and a Channel-Independent Temporal AU Encoder (CIT-AE) to model temporally coherent AU dynamics. By integrating these representations into 3D Gaussian deformation, EmoZone-Talker enables precise and interpretable control over facial expressions. Extensive experiments demonstrate that our method improves expression controllability and realism, with notable gains in upper-face accuracy and temporal coherence, while preserving high rendering quality and accurate lip synchronization. Code will be publicly released to facilitate reproducibility and further research.

Chinese Translation

3D高斯点云（3DGS）在高保真谈话头合成方面展现出强大的潜力。然而，由于语音驱动的面部动态与显式表情信号之间的内在冲突，实现细粒度、可解释和可编辑的面部表情控制仍然面临根本性挑战。现有方法依赖于隐式多模态融合，导致空间纠缠和时间不稳定。我们提出了EmoZone-Talker，这是一种新颖的框架，将音频驱动的面部动画重新构造为跨模态冲突下的结构化时空协调问题。我们的方法引入了面部运动的显式空间解耦和时间动态建模。具体而言，我们提出了优先注意偏差的协同区域（SZ-PAB），通过受解剖学先验指导的区域约束显式解耦模态贡献，以及一个通道独立的时间面部动作单元编码器（CIT-AE）来建模时间一致的面部动作单元动态。通过将这些表示整合到3D高斯变形中，EmoZone-Talker实现了对面部表情的精确和可解释的控制。大量实验表明，我们的方法提高了表情的可控性和真实感，在上半脸准确性和时间一致性方面取得了显著提升，同时保持了高渲染质量和准确的唇部同步。代码将公开发布，以促进可重复性和进一步研究。

View on arXiv Download PDF AI Translation

cs.CV / 138 / 2606.15857

A Dual-Branch Collaborative Framework for Joint Optimization of Underwater Image Enhancement and Object Detection

一种双分支协作框架用于水下图像增强与目标检测的联合优化

Cao, Liyuan, Liu, Zheng, Liao, Guanghao, Yang, Yonghui, Li, Qi

Abstract

Due to wavelength dependent light absorption and scattering, underwater images usually suffer from color distortion and blurred details, which limits underwater object detection performance. Existing underwater image enhancement methods mainly focus on visual quality improvement, while it is still difficult to balance enhancement quality, processing efficiency, and downstream detection performance. Therefore, this paper proposes an efficient dual-branch underwater image enhancement framework for object detection. The detail enhancement branch improves brightness and local contrast to recover texture details in dark regions. The color restoration branch uses adaptive compensation to reduce color distortion and improve color gradation. By combining the complementary outputs of the two branches, the proposed framework provides clearer and more informative images for object detection. On the UIEB and EUVP datasets, the proposed method achieves UIQM scores of 2.249 and 2.576. When applied to the YOLOv8 detection task on the URPC dataset, the proposed method improves mAP50 by 2.1\% compared with the baseline. Extensive experiments show that our method improves object detection in complex underwater scenes, while balancing enhancement quality and processing efficiency.

Chinese Translation

由于波长依赖的光吸收和散射，水下图像通常会遭受颜色失真和细节模糊，这限制了水下目标检测的性能。现有的水下图像增强方法主要集中在视觉质量的提升上，而在增强质量、处理效率和下游检测性能之间仍然难以取得平衡。因此，本文提出了一种高效的双分支水下图像增强框架用于目标检测。细节增强分支改善亮度和局部对比度，以恢复暗区的纹理细节。颜色恢复分支采用自适应补偿来减少颜色失真并改善颜色渐变。通过结合两个分支的互补输出，所提出的框架为目标检测提供了更清晰和更具信息量的图像。在UIEB和EUVP数据集上，所提方法的UIQM得分分别为2.249和2.576。在URPC数据集上应用于YOLOv8检测任务时，所提方法相比基线提高了mAP50 2.1%。大量实验表明，我们的方法在复杂水下场景中改善了目标检测，同时平衡了增强质量和处理效率。

View on arXiv Download PDF AI Translation

cs.CV / 139 / 2606.15861

Object Tokens as a Bridge Between Segmentation and Visual Question Answering in Robotic Surgery

对象令牌作为机器人手术中分割与视觉问答之间的桥梁

Li, Yiping, de Jong, Ronald, van Jaarsveld, Romy, Badaloni, Franco, Kuiper, Gino, Ruurda, Jelle, Pluim, Josien, Breeuwer, Marcel

Abstract

Visual Question Answering (VQA) in robotic surgery, referred to as surgical VQA, requires high-level understanding of complex surgical scenes and the integration of visual perception with language reasoning, with the potential to support surgical training and intraoperative decision-making. Recent Vision-Language Models (VLMs) have shown promising performance through parameter-efficient fine-tuning; however, most existing approaches rely on coarse visual grounding, typically limited to bounding boxes, which fails to capture the fine-grained spatial structure of surgical objects. In this work, we propose a unified framework that jointly performs pixel-level segmentation and visual question answering within a single framework. Our approach integrates a VLM with a Segment Anything Model (SAM)-based decoder and represents scene elements as object tokens generated by the VLM. These object tokens guide answer prediction and are further projected to the SAM-based decoder to produce segmentation masks. By optimizing the object token embeddings through both segmentation and question answering objectives, the model learns spatially grounded representations that enhance visual reasoning while providing explicit pixel-level grounding. We evaluate the proposed method on the private RAMIE (Robot-Assisted Minimally Invasive Esophagectomy) dataset and the public EndoVis18 dataset, where it consistently outperforms baseline methods for surgical VQA. These results demonstrate that incorporating context-aware object tokens into vision-language models improves fine-grained surgical scene understanding.

Chinese Translation

机器人手术中的视觉问答（Visual Question Answering, VQA），即外科VQA，需要对复杂外科场景的高层次理解，以及视觉感知与语言推理的整合，具有支持外科培训和术中决策的潜力。近期的视觉-语言模型（Vision-Language Models, VLMs）通过参数高效的微调展现了良好的性能；然而，大多数现有方法依赖于粗略的视觉定位，通常限于边界框，这无法捕捉外科物体的细粒度空间结构。在本研究中，我们提出了一个统一框架，该框架在单一框架内同时执行像素级分割和视觉问答。我们的方法将VLM与基于Segment Anything Model (SAM) 的解码器集成，并将场景元素表示为由VLM生成的对象令牌。这些对象令牌指导答案预测，并进一步投影到基于SAM的解码器以生成分割掩膜。通过优化对象令牌嵌入以同时满足分割和问答目标，模型学习到空间上扎根的表示，从而增强视觉推理，同时提供明确的像素级定位。我们在私有的RAMIE（机器人辅助微创食管切除术）数据集和公共的EndoVis18数据集上评估了所提方法，结果表明其在外科VQA方面始终优于基线方法。这些结果表明，将上下文感知的对象令牌纳入视觉-语言模型能够改善细粒度的外科场景理解。

View on arXiv Download PDF AI Translation

cs.CV / 140 / 2606.15867

CogCanvas: A Benchmark for Evaluating Multi-Subject Reference-Based Image Generation

CogCanvas：用于评估多主体参考图像生成的基准

Nguyen, Long-Bao, Tran, Quang-Khai, Nguyen, Tam V., Tran, Minh-Triet, Le, Trung-Nghia

Abstract

Multi-subject reference-based image generation requires jointly preserving multiple human identities, binding per-person objects and fashion items, and respecting a specified background scene, a regime where current diffusion models remain brittle. Existing benchmarks evaluate only one axis at a time and none jointly captures multi-identity composition with human-object interaction, background grounding, and spatial plausibility. We introduce CogCanvas, a benchmark of 1,952 curated reference images spanning 100 celebrity identities, 115 distinctive objects and fashion items, and 29 real-world background scenes including landmarks, from which we construct 1,361 compositional prompts covering 2-5 person group sizes. The curation pipeline combines DINOv2-based deduplication, two-stage aesthetic filtering, and automated derivation of structured interaction and position graphs that serve as ground-truth supervision. CogCanvas supports three tasks, reference-based multi-human-object generation (primary), text-to-image compositional generation, and reference retrieval, under a unified six-axis evaluation protocol. We introduce two metrics tailored to the multi-reference setting: BG-Sim, which scores background fidelity on SAM 3-masked regions via DINOv3 feature similarity, and Attr-VQA, which uses a multimodal LLM to verify per-subject attribute binding and inter-person interactions against the structured graphs. Benchmarking five SOTA methods reveals that every model degrades substantially as group size grows from 2 to 5, with near-complete failure on object/fashion binding beyond three subjects.

Chinese Translation

基于多主体参考的图像生成需要共同保留多个身份，绑定每个人的物体和时尚单品，并尊重指定的背景场景，而当前的扩散模型在这一领域仍显脆弱。现有基准仅评估一个维度，且没有一个能够共同捕捉多身份组合与人-物交互、背景定位和空间合理性。我们引入了CogCanvas，这是一个包含1,952张精心挑选的参考图像的基准，涵盖100个名人身份、115个独特物体和时尚单品，以及29个真实世界的背景场景，包括地标，从中构建了1,361个组合提示，覆盖2-5人的组规模。该策划流程结合了基于DINOv2的去重、两阶段的美学过滤，以及自动推导的结构化交互和位置图，这些图作为真实监督。CogCanvas支持三项任务：基于参考的多人体-物体生成（主要任务）、文本到图像的组合生成，以及参考检索，采用统一的六维评估协议。我们引入了两个针对多参考设置的指标：BG-Sim，通过DINOv3特征相似性对SAM 3个掩码区域的背景保真度进行评分，以及Attr-VQA，利用多模态大语言模型验证每个主体的属性绑定和人际交互与结构化图的匹配。对五种最先进的方法进行基准测试显示，随着组规模从2增加到5，每个模型的性能显著下降，在超过三个主体时，物体/时尚绑定几乎完全失效。

View on arXiv Download PDF AI Translation

cs.CV / 141 / 2606.15869

Metis: A Generalizable and Efficient World-Action Model for Autonomous Driving and Urban Navigation

Metis：一种可泛化且高效的世界行动模型用于自主驾驶和城市导航

Li, Jingyu, Liu, Zhe, Hu, Dongnan, Wu, Junjie, Ma, Zipei, Wu, Wenxiao, Han, Chao, Hao, Zhihui, Liu, Zhikang, Zhan, Kun, Deng, Jiankang, Zhu, Xiatian, Zhang, Li

Abstract

World action models~(WAMs) have shown great promise for autonomous driving and urban navigation. Built upon Vision-Language-Action models or video generation models, existing approaches suffer key limitations: (1) High inference latency due to future observation prediction at test time, and (2) tightly coupled video and action modeling leading to representational mismatch and degraded generalization. To address both issues, we propose Metis, an end-to-end WAM framework that decouples video generation and action prediction. Specifically, Metis employs a Mixture-of-Transformers architecture with dedicated experts for video generation and action prediction, preserving the intrinsic distributional properties of each task. To enhance efficiency, we introduce an asymmetric attention mask that enables joint training of both experts while allowing the action model to bypass explicit video generation during inference. This design ensures training-inference consistency and significantly reduces computational costs without compromising planning performance. Extensive experiments demonstrate state-of-the-art performance on the NAVSIM navhard and navtest benchmarks and the CityWalker navigation benchmark, validating both the generalizability and efficiency across diverse tasks. Real-robot deployments further confirm the practical feasibility of our approach.

Chinese Translation

世界行动模型（WAMs）在自主驾驶和城市导航中展现了巨大的潜力。现有方法基于视觉-语言-行动模型或视频生成模型，但存在关键限制：（1）由于在测试时需要预测未来观察，导致高推理延迟；（2）视频和行动建模紧密耦合，导致表示不匹配和泛化能力下降。为了解决这两个问题，我们提出了Metis，一个端到端的WAM框架，解耦视频生成和行动预测。具体而言，Metis采用了混合变换器（Mixture-of-Transformers）架构，专门为视频生成和行动预测设立专家，保留每个任务的内在分布特性。为了提高效率，我们引入了一种不对称注意力掩码，使两个专家能够联合训练，同时允许行动模型在推理过程中跳过显式的视频生成。该设计确保了训练与推理的一致性，并显著降低了计算成本，而不影响规划性能。大量实验表明，在NAVSIM的navhard和navtest基准以及CityWalker导航基准上，Metis实现了最先进的性能，验证了其在多样化任务中的泛化能力和效率。真实机器人部署进一步确认了我们方法的实际可行性。

View on arXiv Download PDF AI Translation

cs.CV / 142 / 2606.15880

Deep Residual Injection for Full-Spectrum Forensic Signal Perception in Multimodal Large Language Models

深度残差注入用于多模态大语言模型的全谱法医学信号感知

Lin, Kaiqing, Yan, Zhiyuan, Chen, Ruoxin, Zhang, Ke-Yue, Zhou, Yue, Piao, Caiyong, Li, Bin, Yao, Taiping, Wang, Bo, Xiao, Youchang, Ding, Shouhong

Abstract

Multimodal large language models (MLLMs) have been increasingly adopted in forensics for their robust semantic understanding. As AI-generated images become realistic, semantic-level inconsistencies alone are often insufficient for reliable detection. This motivates a critical question: whether MLLMs can achieve full-spectrum forensic signal perception, i.e., capturing low-level generator artifacts without sacrificing pre-trained semantic knowledge. We further perform a layer-wise analysis of forensic signal perception in MLLMs, showing that semantic information is primarily formed in the early-to-middle layers, whereas direct fine-tuning for artifact learning disrupts these semantic representations. Based on this insight, we propose Deep Visual Residual MLLM (Deep-VRM) to preserve early semantic processing while injecting artifact-specific visual signals as a residual path into an intermediate layer, where they are fused with semantic token representations and propagated through subsequent trainable layers. This enables later layers to jointly model semantic reasoning and signal-level forensic cues, and surprisingly, the model learns to adaptively leverage different levels of forensic signals depending on the input, achieving robust and generalizable detection performance. Extensive experiments show that our method achieves state-of-the-art across most benchmarks. The code and data are available at https://github.com/KQL11/Deep-VRM.

Chinese Translation

多模态大语言模型（MLLMs）因其强大的语义理解能力而在法医学中被越来越广泛地采用。随着人工智能生成图像的逼真化，仅依靠语义层面的不一致性往往不足以实现可靠的检测。这引发了一个关键问题：MLLMs是否能够实现全谱法医学信号感知，即在不牺牲预训练语义知识的情况下捕捉低级生成器伪影。我们进一步对MLLMs中的法医学信号感知进行了逐层分析，结果表明，语义信息主要在早期到中间层形成，而直接针对伪影学习的微调会破坏这些语义表征。基于这一洞察，我们提出了深度视觉残差MLLM（Deep-VRM），以保留早期的语义处理，同时将特定于伪影的视觉信号作为残差路径注入到中间层，在那里它们与语义标记表征融合并通过后续可训练层传播。这使得后续层能够共同建模语义推理和信号级法医学线索，令人惊讶的是，模型学会根据输入自适应地利用不同层次的法医学信号，从而实现稳健且具有良好泛化能力的检测性能。大量实验表明，我们的方法在大多数基准测试中达到了最先进的水平。代码和数据可在 https://github.com/KQL11/Deep-VRM 获取。

View on arXiv Download PDF AI Translation

cs.CV / 143 / 2606.15886

Text region detection in historical astronomical diagrams

历史天文图中的文本区域检测

Baltacı, Zeynep Sonat, Baena, Raphaël, Meng, Fei, Norindr, Somkéo, Somer, Florence, Husson, Matthieu, Aubry, Mathieu

Abstract

Text detection is a crucial task in the analysis of historical documents. While datasets and benchmarks exist for text detection in manuscripts and maps, the study of text in mathematical diagrams has received little attention. To address this, we introduce a large-scale, diverse, open-access dataset of 948 historical astronomical diagrams containing 10,940 oriented polygonal text regions. Our dataset spans ten centuries (8th to 18th) and seven main linguistic traditions: Arabic and Persian (115), Chinese (332), Byzantine (233), Latin (185), Hebrew (48), and Sanskrit (35). It captures a wide range of diagram styles and textual content, from symbols to multi-line paragraphs. Each text instance is annotated with ordered polygons that precisely delineate text regions and encode the reading direction. In addition, we annotated the 2,293 regions in Latin diagrams with 20 class labels. We evaluated several strong baselines on our dataset, including TESTR, DeepSolo++, and Poly-DETR, a simple extension of DINO-DETR that we design to predict ordered polygon vertices. Poly-DETR achieves state-of-the-art performance on the MTHv2 and cBAD2019 benchmarks and provides a solid, simple baseline on our dataset. Code and dataset available online.

Chinese Translation

文本检测是历史文献分析中的一项关键任务。虽然已有针对手稿和地图的文本检测数据集和基准，但对数学图表中文本的研究却鲜有关注。为了解决这一问题，我们引入了一个大规模、多样化的开放获取数据集，包含948幅历史天文图，涵盖10,940个定向多边形文本区域。我们的数据集跨越十个世纪（8世纪至18世纪）和七大主要语言传统：阿拉伯语和波斯语（115），中文（332），拜占庭语（233），拉丁语（185），希伯来语（48），和梵语（35）。它捕捉了广泛的图表风格和文本内容，从符号到多行段落。每个文本实例都用有序多边形进行标注，精确划定文本区域并编码阅读方向。此外，我们对拉丁图中的2,293个区域进行了20个类别标签的标注。我们在数据集上评估了几个强基线，包括 TESTR、DeepSolo++ 和 Poly-DETR，这是我们设计的 DINO-DETR 的简单扩展，用于预测有序多边形顶点。Poly-DETR 在 MTHv2 和 cBAD2019 基准上达到了最先进的性能，并在我们的数据集上提供了一个稳固且简单的基线。代码和数据集可在线获取。

View on arXiv Download PDF AI Translation

cs.CV / 144 / 2606.15889

SiGnature: Explicit Motion Diffusion for Stylized Semantic Gesture

SiGnature：用于风格化语义手势的显式运动扩散

Rosenthal, Adi, Koren, Tomer, Shaked, Nadav, Friedman, Doron, Shamir, Ariel

Abstract

While recent advances in co-speech gesture generation have achieved impressive rhythmic synchronization, synthesizing gestures that are both semantically meaningful and faithful to a speaker's unique non-verbal style remains an open challenge. Semantic gestures, such as iconic shapes or deictic pointing, are statistically sparse, making them difficult to learn effectively within standard generative models. We present SiGnature, a framework for Stylized and Semantic Gesture generation that reconciles precise semantic control with high-fidelity style preservation. Unlike prevalent methods that rely on entangled latent representations, SiGnature operates in an explicit joint-rotation space. This design enables our core contribution, Joint Motion Integration (JMI), a training-free inference mechanism capable of injecting any external motion sequence, particularly in-the-wild semantic gestures, directly into the diffusion process. JMI automatically identifies the specific ``active joints'' conveying a semantic action and injects them into the generation, while relying on the diffusion backbone to synthesize the remaining body dynamics, including posture and flow, in accordance with the pre-learned style of the target speaker. This allows for the plug-and-play integration of arbitrary motions, including complex semantic gestures, without retraining or introducing the ``Frankenstein'' artifacts typical of cut-and-paste methods. Extensive experiments and perceptual studies demonstrate that SiGnature offers superior semantic motion control while maintaining smooth and natural co-speech gesture generation and preserving the distinct characteristics of the speaker, thereby outperforming state-of-the-art baselines.

Chinese Translation

尽管近年来在共语手势生成方面取得了显著的节奏同步进展，但合成既具有语义意义又忠实于说话者独特非语言风格的手势仍然是一个未解决的挑战。语义手势，例如标志性形状或指示性指向，统计上是稀疏的，这使得在标准生成模型中有效学习它们变得困难。我们提出了SiGnature，一个用于风格化和语义手势生成的框架，它调和了精确的语义控制与高保真度的风格保留。与依赖纠缠潜在表示的普遍方法不同，SiGnature在显式的联合旋转空间中操作。该设计使我们的核心贡献——联合运动整合（Joint Motion Integration, JMI）成为可能，这是一种无训练推理机制，能够将任何外部运动序列，特别是自然环境中的语义手势，直接注入到扩散过程中。JMI自动识别传达语义动作的特定“活动关节”，并将其注入生成过程中，同时依赖扩散骨干网络合成其余的身体动态，包括姿势和流动，以符合目标说话者预先学习的风格。这使得任意运动的即插即用集成成为可能，包括复杂的语义手势，而无需重新训练或引入典型的剪切和粘贴方法所带来的“弗兰肯斯坦”伪影。大量实验和感知研究表明，SiGnature在保持流畅自然的共语手势生成和保留说话者独特特征的同时，提供了卓越的语义运动控制，从而超越了最先进的基线。

View on arXiv Download PDF AI Translation

cs.CV / 145 / 2606.15908

High-Fidelity 4D Hand-Object Capture via Multi-View Spatiotemporal Tracking and Physics-Aware Gaussians

通过多视角时空跟踪和物理感知高斯实现高保真4D手物体捕捉

Peng, Bo, Chen, Xu, Gu, Yi, Matsuki, Hidenobu, Dou, Mingsong, Shen, Jingjing, Kong, Deying, Zhang, Juyong, Shen, Zhengyang

Abstract

The growing demand for high-fidelity 4D hand-object interaction (HOI) data in embodied AI and spatial computing is currently bottlenecked by the reliance on pre-scanned object templates and physical markers. While recent methods have demonstrated promising results in reconstructing 4D hand-object interaction from videos, they are highly sensitive to initial estimates of hand and object poses. Yet, estimating these poses from images is challenging, in particular under severe occlusion which is inherent in hand-object interaction scenarios. We propose a novel system for the robust and accurate reconstruction of hands and objects from synchronized and calibrated multi-view videos without requiring any templates or markers. Our system consists of two main components with key innovations: (1) a multi-view feed-forward transformer model that aggregates cross-view geometry and temporal cues to provide a reliable, metric-consistent initialization for both poses and dense object geometry, and (2) a hand-object physics-aware Gaussian-based optimization framework to refine the initial estimates, integrating tetrahedral constraints, collision refinement, and appearance decomposition to produce physically plausible and visually accurate reconstruction. Validated on public benchmarks and an extensive internal dataset, our pipeline achieves highly robust, artifact-free reconstruction, providing an efficient foundation for automated 4D asset generation. Our project page are available at https://zyshen021.github.io/HOSTPG/.

Chinese Translation

在具身人工智能和空间计算中，对高保真4D手物体交互（HOI）数据的需求日益增长，但目前受到依赖预扫描物体模板和物理标记的瓶颈限制。尽管最近的方法在从视频中重建4D手物体交互方面显示出良好的结果，但它们对手和物体姿态的初始估计非常敏感。然而，从图像中估计这些姿态是具有挑战性的，尤其是在手物体交互场景中固有的严重遮挡情况下。我们提出了一种新颖的系统，用于从同步和校准的多视角视频中稳健且准确地重建手和物体，而无需任何模板或标记。我们的系统由两个主要组件和关键创新组成：（1）一个多视角前馈变换器模型，聚合跨视角几何和时间线索，为手和物体的姿态及密集物体几何提供可靠的、度量一致的初始化；（2）一个基于物理感知高斯的手物体优化框架，用于细化初始估计，整合四面体约束、碰撞细化和外观分解，以生成物理上合理且视觉上准确的重建。经过公共基准测试和广泛的内部数据集验证，我们的流程实现了高度稳健、无伪影的重建，为自动化4D资产生成提供了高效的基础。我们的项目页面可在 https://zyshen021.github.io/HOSTPG/ 上访问。

View on arXiv Download PDF AI Translation

cs.CV / 146 / 2606.15920

OmniOPSD: Rationale-Privileged On-Policy Self-Distillation for Affective Computing

OmniOPSD：面向情感计算的理性特权在线自蒸馏

Cheng, Zebang, Chen, Shuimu, Yang, Boxue, Guan, Yuanshen, Chen, Jingyi, Lian, Zheng, Peng, Xiaojiang, Ma, Fei, Cui, LaiZhong, Tian, Qi

Abstract

Reinforcement learning for multimodal large language models (MLLMs) is often hindered by severe reward sparsity in complex reasoning tasks. This challenge is particularly pronounced in human-centered scenarios involving states, emotions, intentions, and behaviors, where heterogeneous multimodal signals and subjective human factors make high-quality chain-of-thought (CoT) annotations expensive and difficult to obtain. Although many multimodal datasets provide expert-annotated ground-truth labels, directly using these labels for supervised fine-tuning may encourage shortcut learning in multimodal perception and provides limited transparency for safety-critical human--AI interaction. To address these limitations, we propose OmniOPSD, a Rationale-Privileged On-Policy Self-Distillation framework that uses frontier-generated rationales as teacher-side privileged evidence rather than student imitation targets. OmniOPSD uses frontier-generated evidence-aware rationales only as training-time privileged evidence context for a local teacher. The student samples its own rollout from the original multimodal input, while the rationale-privileged teacher scores the same tokens and provides dense token-level supervision. Thus, the student learns on its own trajectory distribution without directly imitating frontier-model completions, and inference requires no labels, rationales, CoT annotations, or closed-source model access. Experiments on MER-UniBench show that OmniOPSD achieves state-of-the-art performance with an average score of $84.19$, and ablations further support the value of rationale-privileged teacher guidance.

Chinese Translation

多模态大型语言模型（MLLMs）的强化学习常常受到复杂推理任务中严重奖励稀疏的制约。这一挑战在涉及状态、情感、意图和行为的人本场景中尤为明显，其中异质的多模态信号和主观的人类因素使得高质量的思维链（CoT）注释成本高昂且难以获得。尽管许多多模态数据集提供了专家注释的真实标签，但直接使用这些标签进行监督微调可能会促使多模态感知中的捷径学习，并且对安全关键的人机交互提供有限的透明度。为了解决这些局限性，我们提出了OmniOPSD，一种理性特权在线自蒸馏框架，该框架使用前沿生成的理性作为教师端的特权证据，而不是学生模仿目标。OmniOPSD仅在训练时将前沿生成的证据感知理性作为本地教师的特权证据上下文。学生从原始多模态输入中采样自己的展开，而理性特权教师对相同的标记进行评分并提供密集的标记级监督。因此，学生在自己的轨迹分布上学习，而不直接模仿前沿模型的完成，推理不需要标签、理性、CoT注释或闭源模型访问。在MER-UniBench上的实验表明，OmniOPSD实现了最先进的性能，平均得分为84.19，并且消融实验进一步支持了理性特权教师指导的价值。

View on arXiv Download PDF AI Translation

cs.CV / 147 / 2606.15924

TurboGS: Accelerating 3D Gaussian Splatting via Error-Guided Sparse Pixel Sampling and Optimization

TurboGS：通过误差引导的稀疏像素采样与优化加速3D高斯点云渲染

Dong, Zheng, Qiu, Daifei, Dai, Pinxuan, Xu, Ke, Xu, Jiamin, He, Lili, Lau, Rynson W. H., Xu, Weiwei

Abstract

Consumer-level applications require fast optimization of 3D Gaussian Splatting (3DGS) with high-fidelity novel view rendering. However, existing 3DGS acceleration approaches still incur substantial computation on redundant pixels while sacrificing fine details. In this paper, we present TurboGS, an error-guided training framework that accelerates 3DGS by concentrating optimization on perceptually informative pixels. TurboGS is built upon four core components: (1) a tile-wise sparse pixel sampling, which, driven by multi-view reconstruction errors during training, prioritizes challenging regions and skips well-reconstructed ones to avoid redundant gradient computation; (2) a tile-wise structure-aware loss with sparse Normalized Cross-Correlation, which provides sparse yet effective supervision to preserve fine details and stabilize training; (3) an error-driven Gaussian density control strategy, which dynamically allocates model capacity and removes redundant primitives; and (4) a tailored hybrid optimizer that couples Hessian-informed updates with Adam moment damping to stabilize and improve convergence under sparse supervision. Experiments on standard benchmarks demonstrate that TurboGS can deliver on par or superior rendering quality within 100 seconds on a single RTX 5090 GPU card (up to 10x training speedup over vanilla 3DGS).

Chinese Translation

消费级应用需要快速优化3D高斯点云渲染（3DGS），以实现高保真度的新视角渲染。然而，现有的3DGS加速方法仍然在冗余像素上消耗大量计算资源，同时牺牲了细节。在本文中，我们提出了TurboGS，一种通过集中优化感知信息丰富的像素来加速3DGS的误差引导训练框架。TurboGS建立在四个核心组件之上：（1）基于瓦片的稀疏像素采样，利用训练期间的多视图重建误差，优先考虑具有挑战性的区域，并跳过重建良好的区域，以避免冗余的梯度计算；（2）基于瓦片的结构感知损失与稀疏归一化互相关（Normalized Cross-Correlation），提供稀疏但有效的监督，以保留细节并稳定训练；（3）基于误差的高斯密度控制策略，动态分配模型容量并去除冗余原语；（4）定制的混合优化器，将海森矩阵信息更新与Adam动量阻尼相结合，以在稀疏监督下稳定并改善收敛性。在标准基准测试中的实验表明，TurboGS能够在单个RTX 5090 GPU卡上在100秒内提供与传统3DGS相当或更优的渲染质量（训练速度提高最高可达10倍）。

View on arXiv Download PDF AI Translation

cs.CV / 148 / 2606.15937

GOOSE-M2F: Adapting Mask2Former for High-Fidelity, Long-Tailed Fine-Grained Semantic Segmentation in Unstructured Outdoor Terrain

GOOSE-M2F：将Mask2Former适配于高保真、长尾细粒度语义分割在非结构化户外地形中的应用

Lingam, Jyothiraditya, Sulake, Nikhileswara Rao, Machara, Sai Manikanta Eswar

Abstract

We present GOOSE-M2F, a task-specific adaptation of Mask2Former for the GOOSE 2D Fine-Grained Semantic Segmentation (FGSS) Challenge at ICRA~2026. The GOOSE benchmark spans 64 fine-grained classes across unstructured outdoor terrain with a severely long-tailed distribution, where rare classes occupy fewer than 50 pixels per image. We extend the Swin-Large Mask2Former baseline with three targeted contributions: (1)200 Object Queries to eliminate representational saturation; (2)a Feature Refinement Module (FRM) combining ASPP-lite and CBAM dual-attention; and (3)an Auxiliary Supervision Head that delivers direct per-pixel gradients for rare classes. A multi-stage training strategy pairs Distribution-Balanced loss, Rare-Class Copy-Paste augmentation, dynamic IoU-aware re-weighting, and EMA. At inference, a dense sliding-window engine with 2D Gaussian kernel blending and 4-scale TTA adds +10.57\%. GOOSE-M2F achieves 70.08\% Official Composite mIoU (63.55\% fine, 76.61\% coarse), placing 3rd on the GOOSE 2D FGSS leaderboard. Code and trained models are publicly available at: \href{https://github.com/Aditya-Lingam-9000/GOOSE-M2F}{Github GOOSE-M2F Code} and \href{https://huggingface.co/XYZ9843/GOOSE-M2F}{Hugging Face GOOSE-M2F}.

Chinese Translation

我们提出了GOOSE-M2F，这是对Mask2Former的任务特定适配，旨在参加2026年ICRA的GOOSE 2D细粒度语义分割（FGSS）挑战。GOOSE基准涵盖了64个细粒度类别，分布在具有严重长尾分布的非结构化户外地形中，其中稀有类别每幅图像的像素数少于50个。我们通过三项针对性的贡献扩展了Swin-Large Mask2Former基线： (1) 200个对象查询以消除表示饱和； (2) 结合ASPP-lite和CBAM双重注意力的特征精炼模块（FRM）； (3) 一个辅助监督头，直接为稀有类别提供每像素梯度。多阶段训练策略结合了分布平衡损失、稀有类别复制粘贴增强、动态IoU感知重加权和EMA。在推理阶段，使用带有2D高斯核混合和4尺度TTA的密集滑动窗口引擎增加了+10.57%。GOOSE-M2F实现了70.08%的官方综合mIoU（63.55%细粒度，76.61%粗粒度），在GOOSE 2D FGSS排行榜中排名第三。代码和训练模型可在以下链接公开获取： [Github GOOSE-M2F Code](https://github.com/Aditya-Lingam-9000/GOOSE-M2F) 和 [Hugging Face GOOSE-M2F](https://huggingface.co/XYZ9843/GOOSE-M2F)。

View on arXiv Download PDF AI Translation

cs.CV / 149 / 2606.15938

Learning Directional Semantic Transitions for Longitudinal Chest X-ray Analysis

用于纵向胸部X光分析的方向性语义转变学习

Hu, Zhangfeng, Yang, Zefan, Wang, Ge, Syeda-Mahmood, Tanveer, Burade, Anushree, Kalra, Mannudeep, Yan, Pingkun

Abstract

Chest X-ray (CXR) interpretation often requires longitudinal comparison to assess disease progression. Existing approaches typically rely on temporal feature fusion or inter-study discrepancy modeling, yet remain limited in capturing subtle progression semantics and overlook the inherently directional nature of disease trajectories. In this paper, we propose ProTrans, a novel vision-language pretraining framework that formulates disease progression as a directional semantic transition between paired CXR studies. ProTrans leverages radiology reports to anchor individual CXR representations within interpretable disease states, and introduces a learnable progression feature map to explicitly encode semantic shifts between states, aligned with report-derived progression descriptions. To enforce direction-aware perception, ProTrans incorporates a reversed temporal modeling process and imposes bidirectional reconstruction consistency across states and transitions, thereby disentangling directional semantics and promoting coherent trajectory modeling. Extensive experiments on longitudinal downstream tasks, including disease progression classification and progression captioning, demonstrate that ProTrans consistently outperforms existing methods, establishing a unified pretraining framework for longitudinal CXR understanding. https://github.com/RPIDIAL/ProTrans

Chinese Translation

胸部X光（CXR）解读通常需要进行纵向比较以评估疾病进展。现有的方法通常依赖于时间特征融合或跨研究差异建模，但在捕捉微妙的进展语义方面仍然有限，并且忽视了疾病轨迹固有的方向性特征。本文提出了ProTrans，一种新颖的视觉-语言预训练框架，将疾病进展形式化为配对CXR研究之间的方向性语义转变。ProTrans利用放射学报告将个体CXR表示锚定在可解释的疾病状态中，并引入可学习的进展特征图，以显式编码状态之间的语义变化，与报告派生的进展描述相一致。为了增强方向感知，ProTrans结合了反向时间建模过程，并在状态和转变之间施加双向重建一致性，从而解开方向性语义并促进连贯的轨迹建模。在纵向下游任务上的广泛实验，包括疾病进展分类和进展描述，表明ProTrans始终优于现有方法，为纵向CXR理解建立了统一的预训练框架。

View on arXiv Download PDF AI Translation

cs.CV / 150 / 2606.15956

You Don't Need Strong Assumptions: Visual Representation Learning via Temporal Differences

你不需要强假设：通过时间差进行视觉表征学习

Daithankar, Ninad, Gladstone, Alexi, LeCun, Yann, Ji, Heng

Abstract

Progress in AI has largely been driven by methods that assume less. As compute and data increase, approaches with weaker inductive biases generally outperform those with stronger assumptions. This is particularly characteristic of the field of Visual Representation Learning, where approaches have gone from being dominated by Supervised Learning, to Weakly Supervised Learning, to the now widespread success of Self-Supervised Learning without human labels. Yet, even modern Self-Supervised Learning approaches still depend on strong inductive biases such as augmentations, masking, or cropping. If this trend holds, even these remaining biases should become bottlenecks at scale -- and our experiments confirm this: the optimal strength of inductive biases decreases as data grows. This motivates the search for approaches that rely on fewer assumptions. To this end, we introduce Temporal Difference in Vision (TDV), a new paradigm for self-supervised learning from video that avoids existing inductive biases, relying instead on a causal assumption that the past causes the future. TDV functions by jointly training an image encoder and a motion encoder so that the current frame's representation plus the encoded motion equals the next frame's representation. Despite not leveraging any strong inductive biases, TDV matches state-of-the-art recipes on dense spatial tasks, laying the foundation for representation learning without strong assumptions.

Chinese Translation

人工智能的进展在很大程度上是由假设较少的方法推动的。随着计算能力和数据量的增加，具有较弱归纳偏置的方法通常优于那些具有较强假设的方法。这在视觉表征学习领域尤为明显，该领域的方法经历了从以监督学习为主导到弱监督学习，再到如今广泛成功的无人工标签的自监督学习的转变。然而，即使是现代自监督学习方法仍然依赖于强归纳偏置，例如数据增强、遮蔽或裁剪。如果这一趋势持续存在，即使是这些剩余的偏置在规模上也应该成为瓶颈——我们的实验证实了这一点：随着数据的增长，归纳偏置的最佳强度在降低。这激励我们寻找依赖于更少假设的方法。为此，我们引入了视觉中的时间差（Temporal Difference in Vision, TDV），这是一种新的自监督学习范式，旨在从视频中学习，避免现有的归纳偏置，而是依赖于一个因果假设，即过去导致未来。TDV通过联合训练图像编码器和运动编码器来工作，使得当前帧的表征加上编码的运动等于下一帧的表征。尽管没有利用任何强归纳偏置，TDV在密集空间任务上达到了最先进的效果，为没有强假设的表征学习奠定了基础。

View on arXiv Download PDF AI Translation

cs.CV / 151 / 2606.15966

VEPHand: View-Efficient Photometric Hand Performance Capture at Scale

VEPHand：高效视角的光度手部性能捕捉系统

Shen, Zhengyang, Chang, Kai-Hung, Wood, Erroll, Kong, Deying, Peng, Bo, Bolkart, Timo, Yang, Jinlong, Zhao, Bowen, Tang, Danhang, Petrovic, Sasa, Aksan, Emre, Riviere, Jérémy, Choutas, Vassilis, Vicini, Delio, Busch, Jay, Liu, Shichen, Cao, Zhe, Liu, Hugh, Shen, JingJing, Taylor, Jonathan, Dou, Mingsong

Abstract

Robust, high-fidelity 3D hand capture, while fundamental to digital human creation, remains challenging with practical multi-view systems that balance rich photometry with the geometric ambiguities of reconstruction arising from limited viewpoint density. This paper presents an end-to-end pipeline for dynamic hand performance capture and registration, specifically designed for view-efficient setups ($\sim$20 views). We address key challenges with two primary innovations. First, to overcome reconstruction difficulties like limited view overlap and background clutter, our mask-free neural method robustly extracts detailed hand geometry and appearance from unmasked images using scene parameterization and scenario-specific density regularization. Second, addressing registration challenges such as accurately capturing non-linear skin deformations and ensuring plausible results during severe self-contact, we propose a physics-inspired framework. It aligns reconstructions to a personalized hand model by optimizing intrinsic volumetric offsets within its canonical tetrahedral mesh, alongside pose parameters. This approach, supported by robust losses and optimization, captures fine surface deformations, ensures plausible results under severe articulation and self-contact, and demonstrates strong tolerance to input noise. We demonstrate the scalability and robustness of our automated pipeline on an extensive dataset of over 12,000 sequences, from which we also derive a large-scale, high-quality synthetic 2D/3D hand dataset for training downstream tasks. This showcases its effectiveness for single hands, intricate two-hand interactions, and natural hand-object manipulations. Our method achieves state-of-the-art reconstruction fidelity in view-efficient, unmasked scenarios and highly accurate registration. Our project page are available at https://zyshen021.github.io/VEPHand/.

Chinese Translation

高保真度的3D手部捕捉在数字人类创作中至关重要，但在实际的多视角系统中，由于视角密度有限而导致的几何重建模糊性，使得这一过程仍然具有挑战性。本文提出了一种端到端的动态手部性能捕捉与注册管道，专为高效视角设置（约20个视角）而设计。我们通过两项主要创新来解决关键挑战。首先，为了克服重建困难，如有限的视角重叠和背景杂乱，我们提出了一种无掩膜的神经方法，通过场景参数化和特定场景的密度正则化，从未掩膜图像中稳健地提取详细的手部几何形状和外观。其次，为了解决注册挑战，如准确捕捉非线性皮肤变形并在严重自接触期间确保合理结果，我们提出了一种受物理启发的框架。该框架通过优化其典型四面体网格内的内在体积偏移量及姿态参数，将重建结果与个性化手部模型对齐。该方法通过稳健的损失函数和优化，捕捉细微的表面变形，确保在严重关节运动和自接触下的合理结果，并表现出对输入噪声的强容忍性。我们在一个超过12,000个序列的大型数据集上展示了我们自动化管道的可扩展性和稳健性，并从中衍生出一个大规模、高质量的合成2D/3D手部数据集，以用于下游任务的训练。这展示了我们方法在单手、复杂的双手交互以及自然的手物体操作中的有效性。我们的方法在高效视角、无掩膜场景中实现了最先进的重建保真度和高度准确的注册。我们的项目页面可访问 https://zyshen021.github.io/VEPHand/。

View on arXiv Download PDF AI Translation

cs.CV / 152 / 2606.15967

CRIS: Cross-Plane Self-Supervised Isotropic Restoration for Anisotropic Volumetric Imaging Across Modalities

CRIS：跨平面自监督各向同性恢复用于各模态的各向异性体积成像

Ahituv, Adi, Ilivitzki, Anat, Freiman, Moti

Abstract

Anisotropic volumetric acquisitions are common in clinical MRI and volume electron microscopy (vEM), where sparse through-plane sampling creates thick slices or sections that degrade orthogonal reformats and downstream analysis. We present CRIS, a cross-plane self-supervised framework for isotropic restoration without paired isotropic ground truth. CRIS casts 3D restoration as 2D stripe completion on orthogonal reformats of an isotropic grid: high-resolution in-plane slices are synthetically degraded and periodically masked for training, while at inference blank slices define the isotropic grid, two orthogonal reformats are restored, and predictions are fused by multi-view averaging. We evaluate CRIS on two MRI cohorts and two microscopy benchmarks up to 8x anisotropy. On brain MRI, CRIS achieves 32.921 +/- 0.436 dB PSNR and 0.9631 +/- 0.0027 SSIM, outperforming interpolation, SMORE4, SIMPLE, SA-INR, and ATME, and gives the best segmentation consistency (Dice 0.940 +/- 0.004, ASSD 0.245 +/- 0.014 mm, HD99 1.275 +/- 0.061 mm). On reference-free abdominal MRI, CRIS reduces FID/KID to 48.714/0.023. On vEM, CRIS outperforms interpolation, NIIV, and vEMINR, reaching 29.133 dB/0.834 3D PSNR/SSIM at 4x, 27.123 dB/0.734 on EPFL at 8x, and 21.915 dB/0.699 on noisy hemibrain data. In a robustness experiment, one variable-gap CRIS model evaluated across gap factors 3--7 and coronal, axial, and sagittal degradations maintained higher PSNR/SSIM than interpolation (36.36--31.14 dB and 0.977--0.932 vs. 33.07--27.85 dB and 0.951--0.853). These results support CRIS as a modality-flexible route to isotropic restoration without paired isotropic targets or configuration-specific retraining. Code is available at https://github.com/adi-hatav/CRIS.

Chinese Translation

各向异性体积采集在临床MRI和体积电子显微镜(vEM)中十分常见，其中稀疏的平面采样导致厚切片或截面，降低了正交重构和后续分析的质量。我们提出CRIS，一种跨平面自监督框架，用于在没有配对各向同性真实值的情况下进行各向同性恢复。CRIS将3D恢复视为在各向同性网格的正交重构上进行2D条纹补全：高分辨率的面内切片被合成降质并定期遮罩用于训练，而在推理阶段，空白切片定义了各向同性网格，两个正交重构被恢复，并通过多视图平均融合预测结果。我们在两个MRI队列和两个显微镜基准上对CRIS进行了评估，达到8倍各向异性。在脑MRI上，CRIS达到了32.921 +/- 0.436 dB的PSNR和0.9631 +/- 0.0027的SSIM，优于插值、SMORE4、SIMPLE、SA-INR和ATME，并且提供了最佳的分割一致性（Dice 0.940 +/- 0.004，ASSD 0.245 +/- 0.014 mm，HD99 1.275 +/- 0.061 mm）。在无参考的腹部MRI上，CRIS将FID/KID降低到48.714/0.023。在vEM上，CRIS优于插值、NIIV和vEMINR，在4倍时达到29.133 dB/0.834的3D PSNR/SSIM，在8倍的EPFL上达到27.123 dB/0.734，并在噪声半脑数据上达到21.915 dB/0.699。在鲁棒性实验中，一个变量间隙的CRIS模型在间隙因子3--7以及冠状、轴向和矢状降质下评估，维持了比插值更高的PSNR/SSIM（36.36--31.14 dB和0.977--0.932对比33.07--27.85 dB和0.951--0.853）。这些结果支持CRIS作为一种灵活的各模态各向同性恢复方法，无需配对的各向同性目标或特定配置的再训练。代码可在https://github.com/adi-hatav/CRIS获取。

View on arXiv Download PDF AI Translation

cs.CV / 153 / 2606.15976

HadBalance: A Plug-and-Play Unified Global Geometric Prior Framework for Generalizable Biomedical Segmentation

HadBalance：一种可插拔的统一全局几何先验框架用于可泛化的生物医学分割

Gao, Zhuangzhi, Zhou, Feixiang, Zhao, He, Chen, Wenhan, Luo, Ruiyu, Wang, Xin, Qin, Hongyi, Wu, Zhongli, Meng, Yanda, Zhao, Yitian, Shantsila, Alena, Lip, Gregory Y. H., Shantsila, Eduard, Zheng, Yalin

Abstract

Precise biomedical image segmentation is crucial for clinical diagnosis. Geometric cues (e.g., boundary, shape, and topology) can improve structural consistency, yet most are task-specific and lack a unified geometric foundation that generalizes across organs and modalities. We are motivated by the observation that several medical segmentation targets can be approximated as globally near-convex shapes. A convex region is one in which any two interior points can be connected by a line segment entirely contained within the region. In practice, medical targets may exhibit small local concavities or boundary irregularities; we refer to such globally convex-like shapes as near-convex. Motivated by this, we derive Hadwiger Shape Priors from Hadwiger's theorem as an interpretable global regularizer using three 2D measures: area A, perimeter P, and Euler characteristic chi, enabling transfer across organs and modalities. However, because medical datasets are shape-heterogeneous, enforcing near-convex priors uniformly can over-regularize non-convex anatomy with significant concavities, washing out concavities and fine details and degrading segmentation accuracy. To address this challenge, we propose Conflict-Aware Objective Balancing (CAOB), which integrates shape priors with segmentation in a gradient-aware manner. For each prior, CAOB removes only the gradient component that conflicts with segmentation while preserving the remaining aligned component, and adaptively regulates objective influences to prevent prior dominance. This enables stable use of shape priors on shape-heterogeneous data without erasing genuine concavities or fine structural details. We call this plug-and-play framework HadBalance.

Chinese Translation

精确的生物医学图像分割对临床诊断至关重要。几何线索（例如，边界、形状和拓扑）可以提高结构一致性，但大多数是特定任务的，缺乏一个能够跨器官和模态泛化的统一几何基础。我们受到观察的启发，发现多个医学分割目标可以近似为全局近凸形状。凸区域是指任何两个内部点可以通过完全包含在该区域内的线段连接。在实践中，医学目标可能会表现出小的局部凹陷或边界不规则性；我们将这种全局类似凸的形状称为近凸形状。基于此，我们从Hadwiger定理推导出Hadwiger形状先验，作为一种可解释的全局正则化器，使用三个二维度量：面积A、周长P和欧拉特征χ，从而实现器官和模态之间的转移。然而，由于医学数据集在形状上是异质的，均匀施加近凸先验可能会对具有显著凹陷的非凸解剖结构进行过度正则化，抹去凹陷和细节，从而降低分割准确性。为了解决这一挑战，我们提出了冲突感知目标平衡（CAOB），它以梯度感知的方式将形状先验与分割相结合。对于每个先验，CAOB仅去除与分割冲突的梯度分量，同时保留其余对齐分量，并自适应调节目标影响，以防止先验主导。这使得在形状异质数据上稳定使用形状先验成为可能，而不会抹去真实的凹陷或细微的结构细节。我们将这一可插拔框架称为HadBalance。

View on arXiv Download PDF AI Translation

cs.CV / 154 / 2606.15982

Mind the Gap: Diagnosing Constraint Discovery Failures in Text-in-Image Editing

注意差距：诊断文本图像编辑中的约束发现失败

Gui, Rui

Abstract

A key challenge in multimodal reasoning is determining which visual dependencies become relevant under a specific task, rather than merely recognizing visible content. We study this through edit-induced constraint discovery in text-in-image editing, a controlled diagnostic setting where a local text change can activate secondary consistency constraints: given a valid editing instruction and an image, can a model identify the secondary regions that must also change? Across 461 diagnostic cases, four MLLMs, and 19 constraint subtypes, models recover only 46% case-level macro recall under unguided prompting versus 94% when constraints are explicitly provided, suggesting that a substantial portion of the failure arises when models must decide which unstated dependencies to surface. Oracle-field decomposition shows that case-specific causal explanations are the most effective partial guidance (0.782 recall), above region names (0.610) or type labels (0.646), suggesting that edit-specific causal cues account for much of the oracle gain. A downstream experiment further shows that higher self-discovery recall does not necessarily improve task performance: unverified self-discovery introduces false positives that offset recall gains, motivating precision-aware constraint elicitation.

Chinese Translation

多模态推理中的一个关键挑战是确定在特定任务下哪些视觉依赖关系变得相关，而不仅仅是识别可见内容。我们通过文本图像编辑中的编辑引发约束发现来研究这一问题，这是一种受控的诊断设置，其中局部文本更改可以激活次级一致性约束：给定有效的编辑指令和一幅图像，模型能否识别出必须也发生变化的次级区域？在461个诊断案例、四种多模态语言模型（MLLMs）和19种约束子类型中，模型在无指导提示下仅恢复了46%的案例级宏召回率，而在明确提供约束时则为94%，这表明失败的很大一部分发生在模型必须决定哪些未说明的依赖关系需要显现时。Oracle-field分解显示，特定案例的因果解释是最有效的部分指导（召回率为0.782），高于区域名称（0.610）或类型标签（0.646），这表明编辑特定的因果线索占据了大部分的oracle增益。下游实验进一步表明，较高的自我发现召回率不一定会提高任务性能：未经验证的自我发现引入了假阳性，从而抵消了召回增益，这促使了对精确度敏感的约束引导。

View on arXiv Download PDF AI Translation

cs.CV / 155 / 2606.15987

A Text Recognition Dataset from Sahidic Coptic Ancient Manuscripts

来自萨希迪克科普特古代手稿的文本识别数据集

Quattrini, Fabio, Zaccagnino, Carmine, Bianchi, Costanza, Cascianelli, Silvia, Cucchiara, Rita

Abstract

In this work, we target Handwritten Text Recognition (HTR) in low-resource scenarios, which arise from underrepresented languages, rare scripts, and degraded visual conditions typical of historical documents. We introduce SCAM (Sahidic Coptic Ancient Manuscripts), a new line-level dataset built from digitized ancient manuscripts written in the extinct Sahidic Coptic dialect. The dataset reflects a realistic and challenging setting, as it combines heterogeneous acquisition conditions across libraries with typical manuscript degradations such as ink fading, bleed-through, and material deterioration. In addition to visual complexity, SCAM poses significant linguistic challenges due to the scarcity of resources for Sahidic Coptic, its uncommon alphabet, and dialect-specific diacritics. To support research in low-resource HTR, we benchmark several state-of-the-art approaches based on different paradigms, highlighting their limitations and strengths in this setting. Our results underline the gap between current HTR performance on well-resourced modern scripts and historically grounded, low-resource scenarios, thus providing a reference point for future developments.

Chinese Translation

在本研究中，我们针对低资源场景下的手写文本识别（HTR），这些场景源于语言代表性不足、稀有文字以及典型历史文献的视觉条件恶化。我们介绍了SCAM（Sahidic Coptic Ancient Manuscripts），这是一个基于已数字化的用已灭绝的萨希迪克科普特方言书写的古代手稿构建的新行级数据集。该数据集反映了一个现实且具有挑战性的环境，因为它结合了来自不同图书馆的异质获取条件，以及典型手稿退化现象，如墨水褪色、渗透和材料劣化。除了视觉复杂性外，SCAM还因萨希迪克科普特资源的稀缺、其不常见的字母表以及特定方言的变音符号而带来了显著的语言挑战。为了支持低资源HTR的研究，我们基于不同范式对几种最先进的方法进行了基准测试，突出了它们在这一环境中的局限性和优势。我们的结果强调了当前在资源丰富的现代文字上HTR性能与历史背景下的低资源场景之间的差距，从而为未来的发展提供了参考点。

View on arXiv Download PDF AI Translation

cs.CV / 156 / 2606.15992

Multi-Task Tennis Stroke Biomechanics Analysis Using MediaPipe Pose

基于MediaPipe Pose的多任务网球击球生物力学分析

Hazarika, Jigyashman

Abstract

We built a multi-task pipeline for tennis stroke biomechanics from plain RGB video. On top of pose-based stroke recognition, it adds two new tasks, predicting shot direction and grading posture quality, plus a rule-based feedback layer that suggests coaching tips. Strokes are found automatically using a weighted joint velocity score, s(t) = 0.5 v_wrist + 0.3 m_elbow + 0.2 m_shoulder, removing the need for manual annotation. Pose comes from MediaPipe Pose Landmarker (33 landmarks, metric world coordinates), with each stroke turned into a 30-frame by 39-feature sequence for TennisTransformerGPU, a compact 564,103-parameter transformer (4 layers, 4 heads, d=128) with three parallel output heads. Trained on 1,281 labeled strokes from 7 pros and 1 amateur across 11 videos, it hits 83.7% stroke-type accuracy, 61.9% on direction, and 62.6% on posture under a random 80/20 split. The interesting test is cross-player: train on pros, evaluate on the amateur. Stroke type barely budges, 82.9%, a 0.8% drop. Direction prediction does not transfer; it just falls back to the majority class. An ablation shows why world coordinates matter so much here: switching to image-space landmarks tanks cross-player stroke-type accuracy from 83% to 47% and direction from 68% to 21%. Everything runs on Kaggle's free T4 GPU tier and is fully reproducible.

Chinese Translation

我们构建了一个从普通RGB视频中提取网球击球生物力学的多任务管道。在基于姿态的击球识别基础上，它增加了两个新任务：预测击球方向和评估姿势质量，以及一个基于规则的反馈层，提供训练建议。通过加权关节速度评分自动识别击球，s(t) = 0.5 v_wrist + 0.3 m_elbow + 0.2 m_shoulder，消除了手动标注的需求。姿态数据来自MediaPipe Pose Landmarker（33个关键点，公制世界坐标），每个击球被转化为一个30帧、39特征的序列，供TennisTransformerGPU使用，这是一个紧凑的564,103参数的变换器（4层，4头，d=128），具有三个并行输出头。该模型在11个视频中训练了1,281个来自7名专业运动员和1名业余运动员的标注击球，达到了83.7%的击球类型准确率，61.9%的方向准确率和62.6%的姿势准确率，采用随机80/20的划分。一个有趣的测试是跨运动员：在专业运动员上训练，在业余运动员上评估。击球类型几乎没有变化，准确率为82.9%，下降了0.8%。方向预测无法转移，直接回落到多数类。消融实验显示了世界坐标在这里的重要性：切换到图像空间关键点使跨运动员击球类型准确率从83%降至47%，方向准确率从68%降至21%。所有操作均在Kaggle的免费T4 GPU层上运行，并且完全可复现。

View on arXiv Download PDF AI Translation

cs.CV / 157 / 2606.16015

Stringalign: Moving beyond summary statistics with a transparent Unicode-aware tool for evaluating automatic transcription models

Stringalign：超越摘要统计的透明Unicode感知工具，用于评估自动转录模型

Moe, Yngve Mardal, Roald, Marie

Abstract

Comparing text strings is crucial when evaluating and understanding the performance of various text processing tasks such as document recognition and audio transcription. With an increasingly complex landscape of AI-based handwritten text recognition (HTR), optical character recognition (OCR) and automatic speech recognition (ASR) models, there is a need for tools that facilitate evaluation in a flexible and reproducible way. This paper presents Stringalign, a Python library designed to simplify the evaluation process for automatic transcription projects and facilitate transparent evaluation. Stringalign's tools to examine and visualise both the rate of errors and the types of errors a model makes, give insights into possible improvements and help inform model selection for a particular task. Widely used string comparison metrics, such as the character and word error rates (CER and WER), although useful, can be ambiguous due to varying definitions of what constitutes a character and a word. Stringalign addresses this challenge by ensuring all preprocessing (i.e. normalisation and tokenisation) is transparent and easily replicable, and by providing tools to move beyond summary statistics and analyse common model errors. Moreover, Stringalign adheres to FAIR (Findable, Accessible, Interoperable, and Reusable) principles for research software while staying lightweight and easy to adapt into researchers existing workflows. In this paper, we discuss challenges with character and word level string comparisons and show through examples that where existing tools can yield opaque and sometimes confusing results, Stringalign provides an easy-to-use and unambiguous alternative.

Chinese Translation

在评估和理解各种文本处理任务（如文档识别和音频转录）的性能时，比较文本字符串至关重要。随着基于人工智能的手写文本识别（HTR）、光学字符识别（OCR）和自动语音识别（ASR）模型的日益复杂，迫切需要能够以灵活和可重复的方式促进评估的工具。本文介绍了Stringalign，一个旨在简化自动转录项目评估过程并促进透明评估的Python库。Stringalign提供的工具可以检查和可视化模型的错误率及错误类型，为可能的改进提供见解，并帮助为特定任务选择模型。虽然广泛使用的字符串比较指标（如字符错误率（CER）和单词错误率（WER））在某种程度上有用，但由于对字符和单词定义的不同，可能会存在模糊性。Stringalign通过确保所有预处理（即规范化和分词）透明且易于复制，解决了这一挑战，并提供工具以超越摘要统计，分析常见模型错误。此外，Stringalign遵循研究软件的FAIR（可发现、可获取、可互操作和可重用）原则，同时保持轻量且易于适应研究人员现有的工作流程。本文讨论了字符和单词级字符串比较的挑战，并通过示例展示了现有工具可能产生不清晰且有时令人困惑的结果，而Stringalign则提供了一个易于使用且明确的替代方案。

View on arXiv Download PDF AI Translation

cs.CV / 158 / 2606.16031

The Third Challenge on Image Denoising at NTIRE 2026: Methods and Results

2026年NTIRE图像去噪第三届挑战赛：方法与结果

Sun, Lei, Guo, Hang, Ren, Bin, Su, Shaolin, Wang, Xian, Paudel, Danda Pani, Van Gool, Luc, Timofte, Radu, Li, Yawei

Abstract

This paper reports on the NTIRE 2026 Challenge on Image Denoising, specifically focusing on the high-noise regime ($\sigma = 50$). The competition investigates advanced neural architectures designed to restore high-fidelity details from images corrupted by additive white Gaussian noise (AWGN). Unlike constrained benchmarks, this track emphasizes peak quantitative performance, measured by Peak Signal-to-Noise Ratio (PSNR), without limitations on parameter count or computational overhead. By synthesizing contributions from 20 finalist teams out of 116 registrants, this report benchmarks the latest technical innovations and provides a comprehensive snapshot of the current state-of-the-art in unconstrained image restoration.

Chinese Translation

本文报告了2026年NTIRE图像去噪挑战赛，特别关注高噪声环境（$ ext{σ} = 50$）。此次比赛探讨了旨在从受加性白噪声（AWGN）污染的图像中恢复高保真细节的先进神经网络架构。与受限基准测试不同，此赛道强调峰值定量性能，通过峰值信噪比（PSNR）进行测量，而不对参数数量或计算开销施加限制。通过综合来自116个注册团队中的20个决赛团队的贡献，本文基准测试了最新的技术创新，并提供了当前无约束图像恢复领域的全面快照。

View on arXiv Download PDF AI Translation

cs.CV / 159 / 2606.16036

Trusting Right Predictions for Wrong Reasons: A LIME Based Analysis of Deep Learning Interpretability in Lung Cancer Diagnosis

基于LIME的深度学习可解释性分析：错误原因下的正确预测信任度在肺癌诊断中的应用

Poudel, Samarpan, Veksler, Vladislav D

Abstract

Lung cancer is the leading cause of cancer-related mortality, with approximately 2.5 million new cases and 1.8 million deaths annually, making reliable diagnosis a clinical priority. Although deep learning models have achieved strong performance in lung cancer classification, evaluation has largely focused on predictive accuracy, leaving their decision-making processes insufficiently examined. This study compares three architecturally distinct models: a Convolutional Neural Network (CNN), a pretrained ResNet50, and a Vision Transformer (ViT), trained on the IQ-OTH/NCCD lung cancer CT dataset. Local Interpretable Model-Agnostic Explanations (LIME) were applied to investigate model reasoning. In addition to standard performance metrics, a dual-correlation framework was introduced to measure both prediction agreement and explanation agreement across model pairs. All three models achieved strong classification performance, with ResNet50 attaining 98.61% accuracy, CNN 97.91%, and ViT 93.75%, while all achieved ROC-AUC scores of 0.99. Prediction correlations exceeded 0.99 across all model pairs, indicating highly consistent outputs. However, LIME explanation correlations remained below 0.26, revealing substantial differences in the image regions used to reach those predictions. Analysis of misclassified samples further identified a consistent spatial pattern: incorrect predictions were associated with attention outside the lung parenchyma, whereas correct predictions focused primarily within lung regions. These findings demonstrate that prediction agreement is a poor proxy for reasoning consistency, and that interpretability evaluation must be treated as an independent validation criterion alongside predictive performance in clinical AI systems.

Chinese Translation

肺癌是癌症相关死亡的主要原因，每年约有250万新病例和180万死亡，使得可靠的诊断成为临床优先事项。尽管深度学习模型在肺癌分类中取得了良好的表现，但评估主要集中在预测准确性上，导致其决策过程未得到充分检验。本研究比较了三种架构不同的模型：卷积神经网络（CNN）、预训练的ResNet50和视觉变换器（ViT），这些模型均在IQ-OTH/NCCD肺癌CT数据集上进行训练。我们应用了局部可解释模型无关解释（LIME）来研究模型的推理过程。除了标准性能指标外，我们引入了双重相关框架来测量模型对之间的预测一致性和解释一致性。所有三种模型均实现了强大的分类性能，其中ResNet50的准确率达到98.61%，CNN为97.91%，ViT为93.75%，同时所有模型的ROC-AUC得分均为0.99。预测相关性在所有模型对之间均超过0.99，表明输出高度一致。然而，LIME解释的相关性保持在0.26以下，揭示了用于做出这些预测的图像区域存在显著差异。对误分类样本的分析进一步识别出了一种一致的空间模式：错误预测与肺实质外的注意力相关，而正确预测主要集中在肺部区域内。这些发现表明，预测一致性是推理一致性的较差代理，且可解释性评估必须作为临床人工智能系统中独立的验证标准，与预测性能并重。

View on arXiv Download PDF AI Translation

cs.CV / 160 / 2606.16048

PointDiffusion: Diffusion-Based Scene Completion in the Point Cloud Domain

PointDiffusion：基于扩散的点云领域场景补全

Agbasiere, Chidera, Sannikov, Mikhail, Ogunwoye, Faith, Shaikhiev, Erik, Kozinov, Alex, Mikhalchuk, Ilya, Zhura, Iana, Tsetserukou, Dzmitry

Abstract

Reconstructing dense 3D scenes from sparse LiDAR point clouds is a fundamental challenge in autonomous driving, where latent diffusion models offer a promising solution. However, existing approaches rely on object-level autoencoders that collapse into unstable global representations at outdoor scale and suffer from ground truth data corrupted by odometry drift that systematically degrades supervision quality. Furthermore, multi-step diffusion inference incurs prohibitive latency for real-time deployment. We propose a novel multi-token Gaussian VAE with cross-attention pooling for stable scene-scale LiDAR compression, combined with an anchor-based ICP ground truth refinement pipeline that eliminates drift-induced noise from training supervision. Together, these components enable a scaffold-free single-step diffusion completion model that achieves an approximately 16x reduction in squared Chamfer distance on SemanticKITTI seq. 08 (0.396 m^2 to 0.024 m^2), surpasses LiDiff and ScoreLiDAR by 17-19% and 10-11%, respectively, and operates at 25-143x lower inference latency. Our results demonstrate that data quality dominates model design in this regime and that multi-token latent spaces provide a stable first stage for latent diffusion-based scene completion.

Chinese Translation

从稀疏的激光雷达点云重建密集的三维场景是自动驾驶中的一个基本挑战，而潜在的扩散模型提供了一种有前景的解决方案。然而，现有的方法依赖于对象级自编码器，这在户外规模上会崩溃为不稳定的全局表示，并且受到由里程计漂移引起的真实数据损坏，这系统性地降低了监督质量。此外，多步扩散推理会导致实时部署的延迟过高。我们提出了一种新颖的多标记高斯变分自编码器（Gaussian VAE），结合交叉注意力池化，用于稳定的场景规模激光雷达压缩，并结合基于锚点的ICP真实数据精炼管道，消除了由漂移引起的训练监督噪声。这些组件共同实现了一个无支架的单步扩散补全模型，在SemanticKITTI序列08上实现了大约16倍的平方Chamfer距离减少（从0.396 m²降至0.024 m²），分别超越了LiDiff和ScoreLiDAR 17-19%和10-11%，并且推理延迟降低了25-143倍。我们的结果表明，在这一领域，数据质量主导模型设计，而多标记潜在空间为基于潜在扩散的场景补全提供了一个稳定的第一阶段。

View on arXiv Download PDF AI Translation

cs.CV / 161 / 2606.16067

Stepwise Token Selection for Efficient Multimodal Large Language Models

逐步令牌选择以提高多模态大型语言模型的效率

He, Landi, Young, Shawn, Xu, Lijian

Abstract

In multimodal large language models (MLLMs), inference cost is largely dominated by the visual token prefix rather than the language backbone, making token reduction a key factor for improving efficiency. Existing approaches typically assign independent importance scores to visual tokens and retain a fixed number of top-ranked tokens, implicitly assuming token independence and a uniform compression ratio across inputs. In this work, we reformulate visual token pruning as a sequential decision-making process. Specifically, we introduce a pointer-style selection mechanism that iteratively chooses informative tokens, conditioning each decision on previously selected ones, and dynamically determines when to stop via a learned termination action. This enables joint optimization of both the selected subset and its size. To enable end-to-end training under standard language modeling objectives, we design a differentiable relaxation based on a variance-preserving noise interpolation scheme, allowing gradients to propagate through the discrete selection process. Extensive experiments on LLaVA-v1.5-7B and Qwen2.5-VL-7B demonstrate that our approach consistently outperforms fixed-ratio baselines across different compression levels. Under aggressive pruning that removes 88.9% of visual tokens, our method preserves 94.6% of the original accuracy while achieving a 1.88x speed-up in prefill latency.

Chinese Translation

在多模态大型语言模型（MLLMs）中，推理成本主要由视觉令牌前缀主导，而非语言主干，因此令牌减少成为提高效率的关键因素。现有方法通常为视觉令牌分配独立的重要性评分，并保留固定数量的排名前列的令牌，隐含地假设令牌之间的独立性以及在输入之间的均匀压缩比。在本研究中，我们将视觉令牌修剪重新表述为一个顺序决策过程。具体而言，我们引入了一种指针式选择机制，该机制迭代地选择信息量丰富的令牌，每个决策都基于之前选择的令牌，并通过学习的终止动作动态确定何时停止。这使得所选子集及其大小能够共同优化。为了在标准语言建模目标下实现端到端训练，我们设计了一种基于方差保持噪声插值方案的可微松弛方法，允许梯度通过离散选择过程传播。在LLaVA-v1.5-7B和Qwen2.5-VL-7B上的大量实验表明，我们的方法在不同压缩水平下始终优于固定比例基线。在去除88.9%视觉令牌的激进修剪下，我们的方法保留了94.6%的原始准确率，同时在填充延迟上实现了1.88倍的加速。

View on arXiv Download PDF AI Translation

cs.CV / 162 / 2606.16082

Tool-IQA: Augmenting Image Quality Assessment with Simple Tools

Tool-IQA：通过简单工具增强图像质量评估

Qin, Guanyi, Zhang, Junjie, He, Chunming, Fu, Yibing, Liang, Jie, Wu, Tianhe, Zhang, Lei

Abstract

Vision-Language Models (VLMs) have been increasingly adopted for Image Quality Assessment (IQA). However, current methods typically employ a static one-shot scoring paradigm, despite the fact that humans assess image quality through dynamic visual inspection, e.g., selectively adjusting views to verify details and subtle artifacts. Specifically, relying solely on a single-pass observation introduces two primary limitations: first, perceiving the image only at a global scale restricts the assessment of finer local details; second, the original intensity distribution of the image may overwhelm the visibility, leading to insufficient inspection of image quality. To address these issues, we propose Tool-IQA, shifting the assessment mechanism from passive scoring to a tool-augmented workflow. In particular, we equip VLMs with simple yet effective view tools: a Magnifier to inspect local details, and a Gamma Corrector to uncover visibility and hidden artifacts. The assessment follows a structured pipeline that consists of an initial observation with rubric notes, a tool-augmented in-depth inspection, and a final quantification for calibrated quality score. Furthermore, to ensure efficient and purposeful tool callings, we introduce a batch-aware training strategy to reward tool interactions that can yield positive contributions rather than simply encouraging usage. Experiments on a variety of IQA benchmarks demonstrate that, with effective tool calling and calibrated assessment, our proposed Tool-IQA significantly outperforms existing state-of-the-art models, e.g., it achieves a PLCC of 0.854 on the challenging CLIVE dataset.

Chinese Translation

视觉-语言模型（VLMs）在图像质量评估（IQA）中得到了越来越广泛的应用。然而，目前的方法通常采用静态的一次性评分范式，尽管人类通过动态视觉检查来评估图像质量，例如选择性地调整视角以验证细节和微妙的伪影。具体而言，仅依赖单次观察引入了两个主要限制：首先，仅在全局尺度上感知图像限制了对更细微局部细节的评估；其次，图像的原始强度分布可能会掩盖可见性，导致对图像质量的检查不足。为了解决这些问题，我们提出了Tool-IQA，将评估机制从被动评分转变为工具增强的工作流程。特别地，我们为VLMs配备了简单而有效的视图工具：放大镜用于检查局部细节，伽马校正器用于揭示可见性和隐藏伪影。评估遵循一个结构化的流程，包括初步观察和评分笔记、工具增强的深入检查，以及最终的标定质量评分。此外，为了确保工具调用的高效性和目的性，我们引入了一种批量感知训练策略，以奖励能够产生积极贡献的工具交互，而不仅仅是鼓励使用。在多种IQA基准上的实验表明，通过有效的工具调用和标定评估，我们提出的Tool-IQA显著优于现有的最先进模型，例如在具有挑战性的CLIVE数据集上实现了0.854的PLCC。

View on arXiv Download PDF AI Translation

cs.CV / 163 / 2606.16092

VinQA: Visual Elements Interleaved Long-form Answer Generation for Real-World Multimodal Document QA

VinQA：用于现实世界多模态文档问答的视觉元素交错长文本答案生成

Jang, Young Rok, Kong, Hyesoo, An, Kyunghwan, Huh, Jae Sub, Kim, Gyeonghun, Choi, Stanley Jungkyu

Abstract

Real-world documents combine text with tables, charts, photographs, and diagrams arranged in diverse layouts, yet existing research on multimodal large language models (MLLMs) for document QA predominantly produces text-only responses, underutilizing these visual elements. We introduce VinQA, a dataset for long-form answer generation where cited visual elements are explicitly interleaved with their supporting text and grounded in relevant document pages. To support this task, we study two encoding methods for feeding raw document page images into an MLLM, along with their visual-element citation mechanisms: (1) Page Encoding, which directly encodes full-page images with bounding boxes of visual elements and treats these boxed regions as citable units; and (2) Modality Encoding, which parses each page to extract text and crop visual elements, encodes them separately, and uses these cropped elements as citable units. In our experiments, we propose M-GroSE, a multimodal evaluation framework extending GroUSE to assess answers along four dimensions: completeness, answer relevancy, faithfulness, and unanswerability. We additionally report Visual Source F1 to directly measure visual citation accuracy. Although proprietary frontier models still achieve the best overall scores on the VinQA test split, fine-tuning open Qwen2.5-VL models on the training split substantially improves their performance and narrows this gap. Modality Encoding is initially more robust for complex documents with long text, many visual elements, and diverse citation requirements. After training on VinQA, however, Page Encoding reaches a comparable level, competing effectively even without the explicit parsing used in Modality Encoding. Finally, Visual G-Eval, an MLLM-based judge, confirms that fine-tuned models insert visual elements at semantically appropriate positions with faithful supporting text.

Chinese Translation

现实世界的文档将文本与表格、图表、照片和图示结合在一起，呈现出多样的布局，但现有关于文档问答的多模态大型语言模型（MLLM）研究主要产生文本响应，未充分利用这些视觉元素。我们提出了VinQA，一个用于长文本答案生成的数据集，其中引用的视觉元素与其支持文本明确交错，并基于相关文档页面进行定位。为了支持这一任务，我们研究了两种编码方法，以将原始文档页面图像输入到MLLM中，以及它们的视觉元素引用机制：（1）页面编码（Page Encoding），直接对带有视觉元素边界框的完整页面图像进行编码，并将这些框定区域视为可引用单元；（2）模态编码（Modality Encoding），解析每个页面以提取文本并裁剪视觉元素，分别对其进行编码，并将这些裁剪的元素用作可引用单元。在我们的实验中，我们提出了M-GroSE，一个扩展GroUSE的多模态评估框架，用于从四个维度评估答案：完整性、答案相关性、真实性和不可回答性。我们还报告了视觉源F1，以直接测量视觉引用的准确性。尽管专有前沿模型在VinQA测试集上仍然获得最佳整体得分，但在训练集上微调开放的Qwen2.5-VL模型显著提高了它们的性能，并缩小了这一差距。模态编码在处理复杂文档时，尤其是长文本、众多视觉元素和多样的引用需求方面，最初更具鲁棒性。然而，在VinQA上训练后，页面编码达到了可比水平，即使没有模态编码中使用的显式解析，仍能有效竞争。最后，基于MLLM的评估者Visual G-Eval确认，微调后的模型在语义上适当的位置插入视觉元素，并提供真实的支持文本。

View on arXiv Download PDF AI Translation

cs.CV / 164 / 2606.16103

SceneCraft: Interactive System for Image Editing via Scene Graph

SceneCraft：通过场景图进行图像编辑的交互系统

Phan, Duc-Manh, Tran, Ngoc-Dai, Do, Duy-Khang, Nguyen, Tam V., Tran, Minh-Triet, Le, Trung-Nghia

Abstract

Recent advances in generative AI have enabled natural language-driven image editing, yet existing systems often fail in complex scenes with multiple interacting objects because they rely heavily on users crafting precise text prompts. To address the absence of structured control, we propose SceneCraft, a novel interactive framework that bridges user intent and model execution by representing images as editable scene graphs. Instead of guessing text prompts through trial and error, users interact directly with a visual graph to perform complex spatial and relational operations. These graph modifications are automatically translated into precise, context-aware editing prompts, effectively eliminating linguistic ambiguity. To ensure robust and diverse results, structured prompts are dispatched to multiple state-of-the-art generative models. Evaluations across diverse editing scenarios show that SceneCraft provides a more intuitive control mechanism, significantly reducing the cognitive burden of manual prompt engineering while generating outputs that users consistently rate as higher in quality and fidelity.

Chinese Translation

近年来，生成性人工智能的进步使得基于自然语言的图像编辑成为可能，但现有系统在处理多个交互对象的复杂场景时往往表现不佳，因为它们过于依赖用户精确构建文本提示。为了解决缺乏结构化控制的问题，我们提出了SceneCraft，这是一种新颖的交互框架，通过将图像表示为可编辑的场景图，架起用户意图与模型执行之间的桥梁。用户不再通过反复试错来猜测文本提示，而是直接与视觉图进行交互，以执行复杂的空间和关系操作。这些图形修改会自动转换为精确的、上下文感知的编辑提示，有效消除语言歧义。为了确保结果的稳健性和多样性，结构化提示会被发送到多个最先进的生成模型。对多种编辑场景的评估表明，SceneCraft提供了一种更直观的控制机制，显著降低了手动提示工程的认知负担，同时生成的输出在质量和保真度上均被用户一致评价为更高。

View on arXiv Download PDF AI Translation

cs.CV / 165 / 2606.16119

EdgeZSAD: Practical Zero-Shot Anomaly Detection on Edge Devices

EdgeZSAD：边缘设备上的实用零-shot异常检测

Cho, Taewan, Choi, Andrew Jaeyong

Abstract

Industrial inspection needs zero-shot anomaly detection (ZSAD) that remains useful under edge deployment constraints. Recent methods often rely on ViT-L foundation backbones (~300M parameters), which exceed the memory and operator budget of typical embedded hardware. We study this regime through EdgeZSAD, a compact reference system built around a TinyViT-21M-512 backbone, an asymmetric global-local readout (EdgeGLR), and a reproducible source-side training recipe (Real-IAD-DR). We train a single checkpoint in a source-trained, target-unseen protocol and evaluate it across six industrial benchmarks. Across three independent runs, the resulting model reaches an average image AUROC of 91.6 on MVTec-AD and 88.2 on VisA, while remaining directly deployable on Jetson Orin Nano Super (TensorRT FP16) and RB5 Gen2 (QNN GPU FP16). Across the six device-rescored benchmarks, image-AUROC drift stays below 0.2 points, indicating that the exported graph preserves host-side ranking behavior in the evaluated deployment setting.

Chinese Translation

工业检测需要在边缘部署约束下仍然有效的零-shot异常检测（ZSAD）。最近的方法通常依赖于ViT-L基础骨干网络（约3亿参数），这超出了典型嵌入式硬件的内存和运算预算。我们通过EdgeZSAD研究这一领域，该系统围绕TinyViT-21M-512骨干网络、非对称全局-局部读取（EdgeGLR）和可重复的源侧训练方案（Real-IAD-DR）构建。我们在源训练、目标未见的协议下训练一个单一的检查点，并在六个工业基准上进行评估。在三次独立运行中，所得到的模型在MVTec-AD上的平均图像AUROC达到91.6，在VisA上达到88.2，同时能够直接部署在Jetson Orin Nano Super（TensorRT FP16）和RB5 Gen2（QNN GPU FP16）上。在六个设备重新评分的基准中，图像AUROC漂移保持在0.2点以下，表明导出的图保持了在评估部署环境中的主机侧排名行为。

View on arXiv Download PDF AI Translation

cs.CV / 166 / 2606.16124

Training-Free Open-Vocabulary Visual Grounding for Remote Sensing Images and Videos

无训练的开放词汇视觉定位用于遥感图像和视频

Li, Ke, Wang, Di, Zhu, Yongshan, Wang, Ting, Ni, Weiping, Lei, Tao, Wang, Quan, Gao, Xinbo

Abstract

Remote sensing visual grounding (RSVG) aims to localize a referred target in a remote sensing image or video according to a natural language expression. Existing RSVG methods usually rely on task-specific manual annotations, which are costly to collect and inevitably limited in covering the diversity of real-world geospatial scenarios. As a result, they often struggle to generalize to open-vocabulary queries involving novel objects, fine-grained attributes, complex spatial relationships, and functional semantics. In this paper, we propose RSVG-ZeroOV, a training-free framework that leverages frozen generic foundation models for zero-shot open-vocabulary RSVG. RSVG-ZeroOV follows an Overview-Focus-Evolve paradigm, which exploits the distinct yet complementary attention patterns of vision-language models (VLMs) and diffusion models (DMs) to progressively generate precise grounding results. Specifically, (i) Overview utilizes a VLM to extract cross-attention maps that capture semantic correlations between the referring expression and visual regions; (ii) Focus leverages the fine-grained modeling priors of a DM to compensate for object structure and shape information often overlooked by VLM attention; and (iii) Evolve introduces a simple yet effective attention evolution module to suppress irrelevant activations, yielding purified object masks. To handle video inputs, we further present Video RSVG-ZeroOV, which extends image-level grounding to spatio-temporal grounding through a query-relevant key-frame selector and a temporal propagator, enabling efficient and temporally coherent video grounding without video annotations or fine-tuning. Extensive experiments on six image and video grounding benchmarks show that RSVG-ZeroOV consistently outperforms existing zero-shot baselines and achieves competitive or superior performance compared with weakly- and fully-supervised methods.

Chinese Translation

遥感视觉定位（RSVG）旨在根据自然语言表达在遥感图像或视频中定位所指目标。现有的RSVG方法通常依赖于特定任务的手动标注，这些标注的收集成本高昂，并且在覆盖真实世界地理空间场景的多样性方面不可避免地受到限制。因此，它们往往难以推广到涉及新对象、细粒度属性、复杂空间关系和功能语义的开放词汇查询。本文提出了RSVG-ZeroOV，这是一种无训练的框架，利用冻结的通用基础模型进行零-shot开放词汇RSVG。RSVG-ZeroOV遵循概览-聚焦-演变（Overview-Focus-Evolve）范式，利用视觉-语言模型（VLMs）和扩散模型（DMs）之间独特但互补的注意力模式，逐步生成精确的定位结果。具体而言，(i) 概览阶段利用VLM提取交叉注意力图，捕捉所指表达与视觉区域之间的语义关联；(ii) 聚焦阶段利用DM的细粒度建模先验来补偿VLM注意力常常忽视的对象结构和形状信息；(iii) 演变阶段引入一个简单而有效的注意力演变模块，以抑制无关激活，从而生成纯化的对象掩膜。为了处理视频输入，我们进一步提出了视频RSVG-ZeroOV，该方法通过查询相关的关键帧选择器和时间传播器，将图像级定位扩展到时空定位，实现高效且时间一致的视频定位，而无需视频标注或微调。在六个图像和视频定位基准上的广泛实验表明，RSVG-ZeroOV始终优于现有的零-shot基线，并且与弱监督和完全监督方法相比，表现出竞争力或更优的性能。

View on arXiv Download PDF AI Translation

cs.CV / 167 / 2606.16131

Shift-and-Sum Quantization for Visual Autoregressive Models

视觉自回归模型的移位与求和量化

Moon, Jaehyeon, Ham, Bumsub

Abstract

Post-training quantization (PTQ) enables efficient deployment of deep networks using a small set of data. Its application to visual autoregressive models (VAR), however, remains relatively unexplored. We identify two key challenges for applying PTQ to VAR: (i) large reconstruction errors in attention-value products, especially at coarse scales where high attention scores occur more frequently; and (ii) a discrepancy between the sampling frequencies of codebook entries and their predicted probabilities due to limited calibration data. To address these challenges, we propose a PTQ framework tailored for VAR. First, we introduce a shift-and-sum quantization method that reduces reconstruction errors by aggregating quantized results from symmetrically shifted duplicates of value tokens. Second, we present a resampling strategy for calibration data that aligns sampling frequencies of codebook entries with their predicted probabilities. Experiments on class-conditional image generation, inpainting, outpainting, and class-conditional editing show consistent improvements across VAR architectures, establishing a new state of the art in PTQ for VAR.

Chinese Translation

后训练量化（PTQ）使得使用少量数据高效部署深度网络成为可能。然而，PTQ 在视觉自回归模型（VAR）中的应用仍然相对未被探索。我们确定了将 PTQ 应用于 VAR 的两个关键挑战：（i）在注意力-值乘积中的大重构误差，尤其是在高注意力分数更频繁出现的粗尺度下；（ii）由于校准数据有限，码本条目的采样频率与其预测概率之间存在差异。为了解决这些挑战，我们提出了一种针对 VAR 的 PTQ 框架。首先，我们引入了一种移位与求和量化方法，通过聚合来自对称移位的值令牌的量化结果来减少重构误差。其次，我们提出了一种校准数据的重采样策略，使码本条目的采样频率与其预测概率对齐。在类条件图像生成、图像修复、图像扩展和类条件编辑等实验中，VAR 架构的一致性改进表明，我们在 VAR 的 PTQ 中建立了新的最先进水平。

View on arXiv Download PDF AI Translation

cs.CV / 168 / 2606.16153

A Comprehensive Survey of Medical Image Segmentation: Challenges, Benchmarks, and Beyond

医学图像分割的综合调查：挑战、基准与未来

Zhu, Pengyu, Zhang, Xiaojing, Zhang, Kunbo, Zhang, Chunyan, Wang, Zhenyu

Abstract

Medical image segmentation plays a critical role in clinical diagnostics, treatment planning, disease monitoring, and neurological disorder identification. This article presents a comprehensive review of its systematic development, covering widely used public datasets, representative methods built on the U-Net, Transformer, and SAM architectures, and key evaluation metrics with their differences, followed by an analysis of major challenges from multiple perspectives. Unlike surveys that focus on a single model family or a specific clinical application, this review organizes U-Net-, Transformer-, and SAM-based methods within a unified analytical framework, with a particular focus on their effectiveness in improving segmentation accuracy and efficiency. This work aims to guide future research and support clinical translation of medical image segmentation, with all related resources publicly available in our GitHub repository: https://github.com/andrew-pengyu/Awsome_MedSeg/tree/main.

Chinese Translation

医学图像分割在临床诊断、治疗规划、疾病监测和神经系统疾病识别中发挥着关键作用。本文对其系统发展进行了全面回顾，涵盖了广泛使用的公共数据集、基于 U-Net、Transformer 和 SAM 架构的代表性方法，以及主要评估指标及其差异，随后从多个角度分析了主要挑战。与专注于单一模型家族或特定临床应用的调查不同，本综述在统一的分析框架内组织了基于 U-Net、Transformer 和 SAM 的方法，特别关注它们在提高分割准确性和效率方面的有效性。本研究旨在指导未来的研究并支持医学图像分割的临床转化，所有相关资源均在我们的 GitHub 仓库中公开可用： https://github.com/andrew-pengyu/Awsome_MedSeg/tree/main。

View on arXiv Download PDF AI Translation

cs.CV / 169 / 2606.16158

Focus When Necessary: Adaptive Routing and Collaborative Grounding for Training-Free Visual Grounding

按需聚焦：用于免训练视觉定位的自适应路由与协同定位

Wang, Yifan, Li, Peiming, Li, Shiyu, Hu, Zhiyuan, Yang, Xiaochen, Yang, Wenming, Tang, Yang, Wei, Zheng

Abstract

While Multimodal Large Language Models (MLLMs) excel in cross-modal reasoning, they often struggle to perceive fine-grained details in complex high-resolution images. Recent training-free methods address this through image scaling and localized cropping. However, applying these manipulations indiscriminately introduces computational redundancy for simple queries and can degrade accuracy by truncating essential global context or introducing irrelevant background noise. To this end, we propose LazyMCoT, a dynamic and training-free framework that adaptively allocates visual grounding efforts based on sample difficulty. The framework features an Adaptive Routing mechanism that evaluates predictive uncertainty using first-token statistics from a single forward pass. This efficiently bypasses confident cases while ensuring the recall of difficult samples via conformal calibration. For these challenging cases, a Collaborative Grounding module integrates the inherent cross-modal attention of the model with an external visual expert through a two-stage refinement process. This refinement process generates a precise localized display to recover small or occluded targets. Extensive experiments across diverse benchmarks demonstrate that LazyMCoT rivals training-based approaches by simultaneously improving reasoning accuracy and reducing average inference latency. Our code is availble at https://github.com/TencentBAC/LazyMCoT.

Chinese Translation

尽管多模态大语言模型（Multimodal Large Language Models, MLLMs）在跨模态推理方面表现出色，但在处理复杂高分辨率图像时，仍难以感知细粒度细节。近期的免训练方法通过图像缩放与局部裁剪来缓解这一问题。然而，对这些操作的不加区分地应用，会在处理简单查询时引入计算冗余，并可能由于截断关键的全局上下文或引入无关的背景噪声而降低准确性。为此，我们提出 LazyMCoT，这是一个动态的免训练框架，可根据样本难度自适应地分配视觉定位计算资源。该框架包含一个自适应路由机制，通过单次前向传播中的首个 token 统计信息来评估预测不确定性，从而实现有效判断。该机制能够高效绕过高置信度样本，并通过保形校准（conformal calibration）确保对困难样本的召回。对于这些困难样本，协同定位模块通过两阶段精细化流程，将模型内在的跨模态注意力机制与外部视觉专家相结合，以生成精确的局部化显示，从而恢复小目标或被遮挡目标。大量实验表明，在多个基准测试上，LazyMCoT 在无需训练的条件下可与基于训练的方法竞争，同时提升推理准确率并降低平均推理延迟。我们的代码已发布于 https://github.com/TencentBAC/LazyMCoT。

View on arXiv Download PDF AI Translation

cs.CV / 170 / 2606.16159

Continuous Splatting meets Retinex: Continuous Gaussian Splatting and Implicit Reflectance Modeling for Low-Light Image Enhancement

连续Splatting与Retinex结合：用于低照度图像增强的连续高斯Splatting与隐式反射率建模

Chen, Yuhan, Shi, Yicui, Li, Guofa, Yu, Wenxuan, Fang, Ying, Bai, Guangrui, Chu, Wenbo, Li, Keqiang

Abstract

Low-light image enhancement aims to recover clear images from low-illumination observations and is crucial for high-level downstream vision tasks. However, existing methods frequently encounter color distortion and structural artifacts when balancing global smooth illumination adjustment and local high-frequency detail recovery. To address these issues, we propose CGS-Retinex as the first low-light image enhancement framework based on explicit-implicit joint modeling. Our framework deeply integrates continuous Gaussian splatting with Retinex theory. Specifically, we represent the image grid as a continuous parameter field and propose a continuous Gaussian renderer to estimate the spatially continuous global illumination distribution. This approach fundamentally eliminates grid artifacts caused by discrete Gaussian sampling. Furthermore, we introduce an implicit neural representation to model reflectance independently. We leverage shallow high-frequency features to guide the network in accurately reconstructing degraded texture details. Within the Retinex framework, we incorporate physics-inspired brightness consistency constraints and illumination smoothness regularization to enable explicit illumination and implicit reflectance to maintain proper exposure and achieve high-fidelity recovery of high-frequency structures and colors. Extensive experiments demonstrate that CGS-Retinex significantly suppresses dark-region noise and overexposure while achieving exceptional high-frequency structural fidelity and color restoration by precisely decoupling illumination and texture. This work establishes a novel continuous physical representation paradigm for low-light image enhancement.

Chinese Translation

低照度图像增强旨在从低照度观测中恢复清晰图像，对高层视觉下游任务至关重要。然而，现有方法在平衡全局平滑的光照调整与局部高频细节恢复时，常常出现颜色失真与结构伪影问题。为了解决这些问题，我们提出CGS-Retinex，这是首个基于显式-隐式联合建模的低照度图像增强框架。该框架深度融合了连续高斯Splatting与Retinex理论。具体而言，我们将图像网格表示为连续参数场，并提出一种连续高斯渲染器，用于估计空间连续的全局光照分布。该方法从根本上消除了由离散高斯采样引起的网格伪影。此外，我们引入隐式神经表示来独立建模反射率，并利用浅层高频特征引导网络更准确地重建退化的纹理细节。在Retinex框架下，我们引入受物理启发的亮度一致性约束与光照平滑正则化，使显式光照与隐式反射率能够维持合理曝光，并实现高保真度的高频结构与颜色恢复。大量实验表明，CGS-Retinex通过精确解耦光照与纹理，在显著抑制暗部噪声与过曝的同时，实现了卓越的高频结构保真度与颜色还原能力。该工作为低照度图像增强建立了一种新的连续物理表示范式。

View on arXiv Download PDF AI Translation

cs.CV / 171 / 2606.16161

Multimodal LLM-Empowered Re-Ranking for Generalizable Person Re-Identification

多模态大语言模型赋能的通用性行人重识别重排序方法

Li, Jiachen, Gong, Xiaojin

Abstract

Domain Generalizable (DG) person re-identification (Re-ID) has attracted growing research interest due to its potential for deployment in unseen real-world scenarios. Most existing approaches address DG Re-ID by focusing on training domain-generalizable encoders but ignore the possible refinements in inference stage. In contrast, this work explores an alternative direction which improves inference re-ranking to enhance DG Re-ID. Conventional re-ranking methods typically rely on neighborhood-based distances to refine the initial ranking list, inherently depending on features produced by the Re-ID encoder. However, they deteriorate on target domains since the encoder lacks sufficient generalizability to produce reliable feature distances on unseen scenarios. Inspired by the remarkable generalization capabilities of recent Multimodal Large Language Models (MLLMs), we propose an MLLM-empowered distance metric to improve re-ranking in DG Re-ID. Specifically, we first adapt an MLLM to Re-ID data through supervised fine-tuning, which incorporates a domain-agnostic prompt and a query-candidate hard mining scheme. Then, the adapted MLLM is employed to compute a $\mu$-distance during inference, which is robust to domain gap and significantly enhances subsequent re-ranking performance. Our approach is model-agnostic and can be seamlessly integrated into previous re-ranking frameworks. Extensive experiments demonstrate that our approach consistently yields substantial performance improvements across multiple DG Re-ID benchmarks. The code of this work will be released at https://github.com/RikoLi/MUSE soon.

Chinese Translation

领域泛化（Domain Generalizable, DG）行人重识别（Re-ID）由于其在未见真实场景中的部署潜力，近年来受到越来越多的研究关注。现有大多数方法通过训练具有领域泛化能力的编码器来解决DG Re-ID问题，但往往忽略了推理阶段的进一步优化。相比之下，本文探索了一种新的研究方向，即通过改进推理阶段的重排序（re-ranking）机制来提升DG Re-ID性能。传统重排序方法通常依赖基于邻域的距离来细化初始排序列表，其本质上依赖于Re-ID编码器所提取的特征。然而，由于编码器在未见领域中泛化能力不足，无法生成可靠的特征距离，这类方法在目标领域中往往性能下降。受近期多模态大语言模型（Multimodal Large Language Models, MLLMs）出色泛化能力的启发，我们提出了一种由MLLM赋能的距离度量方法，用于改进DG Re-ID中的重排序过程。具体而言，我们首先通过监督微调（supervised fine-tuning）将MLLM适配到Re-ID数据中，其中引入领域无关（domain-agnostic）的提示设计以及查询-候选难例挖掘（query-candidate hard mining）策略。随后，在推理阶段利用该适配后的MLLM计算μ-distance，该距离对领域差异具有较强鲁棒性，并显著提升后续重排序性能。我们的方法具有模型无关性（model-agnostic），可以无缝集成到现有的重排序框架中。大量实验表明，该方法在多个DG Re-ID基准数据集上均能持续带来显著性能提升。本工作的代码将很快在https://github.com/RikoLi/MUSE开源。

View on arXiv Download PDF AI Translation

cs.CV / 172 / 2606.16163

Dehaze-GaussianImage: Zero-Shot Dehazing via Efficient 2D Gaussian Splatting Representation

Dehaze-GaussianImage：基于高效二维高斯泼溅（2D Gaussian Splatting）表示的零样本去雾方法

Chen, Yuhan, Yu, Wenxuan, Li, Guofa, Huang, Kunyang, Fang, Ying, Shi, Yicui, Chu, Wenbo, Li, Keqiang

Abstract

Existing single image dehazing methods are often constrained by computational redundancy in pixel-level optimization and the lack of physical interpretability in implicit neural networks. These limitations hinder the balance between representation efficiency and reconstruction fidelity. To address these issues, we propose Dehaze-GaussianImage, the first zero-shot framework that introduces 2D Gaussian Splatting (2DGS) into the image dehazing domain to break the traditional pixel-grid processing paradigm. Distinct from static convolutional neural networks (CNNs) or Transformers, our approach models hazy images as continuous and dynamically evolvable anisotropic Gaussian fields. Specifically, we propose a novel reconstruction-decoupling zero-shot learning strategy that embeds the atmospheric scattering model into the Gaussian parameter space. This strategy drives Gaussian primitives to adaptively split, clone, and prune during optimization, achieving geometric-level decoupling of the transmission medium and clear textures. Furthermore, explicit structure-preserving constraints are introduced to suppress artifacts commonly caused by traditional physical priors. Experimental results demonstrate that the proposed method achieves state-of-the-art (SOTA) performance in a fully unsupervised manner with minimal parameters, highlighting the potential of explicit Gaussian representation for low-level vision tasks.

Chinese Translation

现有的单幅图像去雾方法往往受到像素级优化带来的计算冗余限制，以及隐式神经网络缺乏物理可解释性的制约。这些问题阻碍了表示效率与重建保真度之间的平衡。为了解决上述问题，我们提出Dehaze-GaussianImage，这是首个将二维高斯泼溅（2D Gaussian Splatting, 2DGS）引入图像去雾领域的零样本框架，从而打破传统基于像素网格的处理范式。不同于静态的卷积神经网络（Convolutional Neural Networks, CNNs）或Transformer模型，我们的方法将雾霾图像建模为连续且可动态演化的各向异性高斯场。具体而言，我们提出了一种新颖的重建-解耦零样本学习策略，将大气散射模型嵌入高斯参数空间中。该策略在优化过程中驱动高斯基元自适应地进行分裂、复制与剪枝，从而实现传输介质与清晰纹理在几何层面的解耦。此外，引入了显式的结构保持约束，以抑制传统物理先验方法中常见的伪影问题。实验结果表明，该方法在完全无监督的情况下，以极少的参数实现了当前最先进（State-of-the-Art, SOTA）的性能，凸显了显式高斯表示在低层视觉任务中的潜力。

View on arXiv Download PDF AI Translation

cs.CV / 173 / 2606.16168

Fi-Gaussian: Frequency-Aware Implicit Gaussian Splatting for Single Image Dehazing

Fi-Gaussian：频率感知隐式高斯点云用于单幅图像去雾

Chen, Yuhan, Fang, Ying, Li, Guofa, Yu, Wenxuan, Shi, Yicui, Huang, Kunyang, Chu, Wenbo, Li, Keqiang

Abstract

Single image dehazing continues to be hindered by the loss of high-frequency details and the difficulty of accurate physical scattering modeling. To address these issues, we propose Fi-Gaussian, a frequency-aware implicit Gaussian splatting network for single image dehazing. Unlike explicit rendering methods that rely on 3D point clouds, our method employs implicit Gaussian splatting to adaptively model the underlying distribution of clear images as a continuous representation in 2D feature space. The core of the network is a frequency-aware implicit Gaussian splatting module, which decouples low-frequency structural information and high-frequency texture information in the frequency domain and then performs adaptive Gaussian aggregation with complex-valued weights to recover fine details. In addition, a physics-driven scattering renormalization mechanism is introduced to estimate the transmission map and atmospheric light under the guidance of implicit Gaussian priors. Extensive experiments on multiple benchmark datasets demonstrate that Fi-Gaussian achieves state-of-the-art quantitative performance and produces visually superior dehazed results, validating the effectiveness of implicit Gaussian splatting for low-level vision tasks.

Chinese Translation

单幅图像去雾仍然受到高频细节丢失和准确物理散射建模困难的限制。为了解决这些问题，我们提出了Fi-Gaussian，一种频率感知隐式高斯点云网络，用于单幅图像去雾。与依赖于3D点云的显式渲染方法不同，我们的方法采用隐式高斯点云自适应建模清晰图像的底层分布，作为二维特征空间中的连续表示。网络的核心是一个频率感知隐式高斯点云模块，它在频率域中解耦低频结构信息和高频纹理信息，然后使用复值权重进行自适应高斯聚合，以恢复细节。此外，引入了一种基于物理的散射重归一化机制，以在隐式高斯先验的指导下估计传输图和大气光。对多个基准数据集的广泛实验表明，Fi-Gaussian在定量性能上达到了最先进的水平，并产生了视觉上优越的去雾结果，验证了隐式高斯点云在低级视觉任务中的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 174 / 2606.16180

To forget is to preserve: Machine Unlearning for 3D medical image segmentation

遗忘即保留：用于3D医学图像分割的机器遗忘（Machine Unlearning）方法研究

Singh, Nitesh Kumar, Singh, Akhilesh, Arora, Arjun

Abstract

With new data privacy laws such as the General Data Protection Regulation (GDPR) [1] that allow individuals to ask that any of their personal information be erased from trained machine learning models, there has been a push to investigate the unlearning of data from models as a way to comply with these laws. In this regard, based on four mechanics, we consider several approximate unlearning strategies applied to the MRBrainS18 dataset [2]. We use a 3D ResNet-50 [3] as a backbone architecture for segmentation that has been pre-trained with the Med3D framework [4]. Considering the pre-trained model as a baseline, we evaluate respective retention accuracy on 2 types of subjects, i.e., retain and forget. We assess these approaches through their Dice similarity coefficient and mean absolute error (MAE) values using two separate training horizons 20 and 50 epochs. The results show that the Noisy Label strategy had the best overall trade-off with a decrease of 93% in the forget set while maintaining 84% accuracy for the retained set after 50 epochs. All other strategies showed extreme levels of forgetting at higher epoch numbers while also demonstrating catastrophic degradation of their retain set performance. The results of this study provide a strict baseline of performance metrics for unlearning on a subject-specific level and provide practitioners with clear criteria for selecting the proper strategies.

Chinese Translation

随着《通用数据保护条例》（General Data Protection Regulation, GDPR）等新的数据隐私法律的出台，这些法规允许个人要求将其在已训练机器学习模型中的任何个人信息删除，因此研究从模型中“遗忘”数据的方法，以满足相关法律合规性，已成为一个重要研究方向。在此背景下，基于四种机制，我们在 MRBrainS18 数据集上考虑并比较了多种近似机器遗忘策略。我们采用基于 Med3D 框架预训练的 3D ResNet-50 作为分割任务的骨干网络。在将预训练模型作为基线的基础上，我们分别在“保留（retain）”与“遗忘（forget）”两类样本上评估其保留性能。我们通过 Dice 相似系数（Dice similarity coefficient）和平均绝对误差（Mean Absolute Error, MAE）对这些方法进行评估，并在 20 个与 50 个训练周期两种不同训练阶段下进行比较。结果表明，在 50 个训练周期后，“噪声标签（Noisy Label）”策略在整体权衡上表现最佳，在遗忘集上性能下降 93%，同时在保留集上仍保持 84% 的准确率。其他策略在较高训练轮次下虽然表现出更强的遗忘效果，但同时也导致保留集性能的严重退化。本研究为针对特定样本层面的机器遗忘任务提供了严格的性能基线，并为实践者选择合适的遗忘策略提供了明确参考标准。

View on arXiv Download PDF AI Translation

cs.CV / 175 / 2606.16184

Closed-Loop Triplet Synergistic Generation for Long-Form Video

用于长篇视频的闭环三元协同生成

Yin, Xinlei, Peng, Xiulian, Li, Xiao, Xiong, Zhiwei, Lu, Yan

Abstract

Multi-shot long-form video generation remains challenging due to identity drift and compounding inconsistencies across shots. While storyboard-driven pipelines improve controllability, they are often executed in a feed-forward manner, with limited mechanisms to incorporate generated visual evidence back into subsequent conditioning. We propose CoTriSyGen, an agentic framework that formulates multi-shot long video generation as a closed-loop visual-text-memory synergy process, where planned intent, persistent memory, and generated visuals are jointly leveraged for iterative correction and long-range coherence. A vision-language-model-based analyzer reasons over this triplet and produces updates to both prompts and memory along two pathways: (i) intra-shot refinement, which triggers targeted regeneration when semantic or compositional violations are detected and refines image-to-video prompt for coherent motions; and (ii) inter-shot refinement, which rewrites subsequent-shot prompts to propagate newly manifested entities or attributes and improve prompt quality (e.g., compositional grounding and cinematic fluency) based on generated evidence. The loop is grounded in an entity-centric memory modeled as a mutable visual state that evolves as the story progresses, which is continuously updated by both the generator and the analyzer by adding new and evolved entities to reflect appearance changes, accumulated multi-view evidence, and multi-entity compositions. Experiments on our curated StoryBench benchmark demonstrate substantial improvements in cross-shot consistency, prompt adherence, and cinematic continuity over representative methods.

Chinese Translation

多镜头长篇视频生成由于跨镜头的身份漂移以及误差累积的不一致问题而仍然具有挑战性。尽管分镜驱动（storyboard-driven）的生成流程提升了可控性，但通常以前馈方式执行，缺乏将已生成的视觉证据反馈回后续条件建模中的机制。我们提出 CoTriSyGen，一种智能体式（agentic）框架，将多镜头长视频生成形式化为一个闭环的视觉-文本-记忆协同过程，在该过程中，规划意图、持久记忆与生成视觉结果被联合利用，以实现迭代式修正与长程一致性建模。基于视觉语言模型（Vision-Language Model, VLM）的分析器对这一三元结构进行推理，并通过两条路径更新提示词与记忆：（i）镜头内细化（intra-shot refinement），当检测到语义或构图层面的违规时触发针对性重生成，并对图像到视频的提示词进行优化，以获得更连贯的运动表现；（ii）镜头间细化（inter-shot refinement），基于已生成证据，重写后续镜头的提示词，以传播新出现的实体或属性，并提升提示词质量（例如更好的组合式锚定与电影化表达流畅性）。该闭环系统以实体为中心的记忆为基础，将其建模为随故事进展而演化的可变视觉状态，并由生成器与分析器共同持续更新，通过添加新的及演化的实体来反映外观变化、累积的多视角证据以及多实体组合关系。在我们构建的 StoryBench 基准测试上的实验表明，该方法在跨镜头一致性、提示词遵循度以及电影级连贯性方面，相较于代表性方法具有显著提升。

View on arXiv Download PDF AI Translation

cs.CV / 176 / 2606.16185

Learned JPEG Compression for DNN Vision

面向深度神经网络视觉任务的学习型JPEG压缩

Zheng, Kaixiang, Salamah, Ahmed H., Chen, Siyu, Yang, En-Hui

Abstract

JPEG, a lossy image compression technique designed for human viewers, has maintained its dominance for decades. However, in the era of artificial intelligence (AI), a substantial portion of image data, often compressed by JPEG, is and will continue to be consumed by deep neural networks (DNNs) instead of humans, thus creating a need to optimize JPEG for DNN inference performance. To this end, we propose learned JPEG compression for DNN vision (J4D), a novel training framework for determining JPEG encoding parameters to minimize compression rate while maximizing DNN inference performance. The major challenge of solving this optimization problem lies in representing the JPEG codec and compression rate in closed form. By incorporating a differentiable soft quantizer based on a probabilistic quantization scheme, we not only obtain a differentiable proxy for the JPEG codec, but are also able to compute the entropy of the coded source analytically, which is a close estimate of the actual compression rate. Equipped with both the differentiable JPEG codec and the information-theoretic rate estimator, we are then able to solve the aforementioned optimization problem with backpropagation. After training, the learned encoding parameters will be subsequently used in actual JPEG encoding based on probabilistic quantization. Extensive experimental results across multiple datasets and DNN architectures demonstrate that J4D consistently and significantly outperforms the default JPEG and other competitive JPEG codecs optimized for DNNs. Notably, compared to the default JPEG, J4D achieves an increase in accuracy by as much as 11.60% at the same rate, or a reduction of compression rate up to 80.05% at the same accuracy. Additionally, with the help of J4D, we show the potential to design universal JPEG encoding parameters for various DNN architectures for the first time.

Chinese Translation

JPEG是一种为人类视觉设计的有损图像压缩技术，数十年来一直占据主导地位。然而在人工智能（AI）时代，大量通常经过JPEG压缩的图像数据，正在并且将持续被深度神经网络（DNN）而非人类所消费，因此需要对JPEG进行优化以提升DNN推理性能。为此，我们提出了面向DNN视觉任务的学习型JPEG压缩（J4D），这是一种用于确定JPEG编码参数以在最小化压缩率的同时最大化DNN推理性能的新型训练框架。该优化问题的主要挑战在于如何以闭式形式表示JPEG编解码器及压缩率。通过引入基于概率量化方案的可微分软量化器，我们不仅获得了JPEG编解码器的可微代理，还能够对编码信号的熵进行解析计算，从而得到对实际压缩率的近似估计。在可微分JPEG编解码器与信息论意义上的速率估计器的共同支持下，我们得以通过反向传播求解上述优化问题。训练完成后，学习得到的编码参数将用于基于概率量化的实际JPEG编码。在多个数据集与多种DNN架构上的大量实验结果表明，J4D始终显著优于默认JPEG以及其他针对DNN优化的竞争性JPEG编码方案。值得注意的是，相较于默认JPEG，J4D在相同压缩率下可将准确率最高提升11.60%，或在相同准确率下将压缩率最高降低80.05%。此外，借助J4D，我们首次展示了为多种DNN架构设计通用JPEG编码参数的潜力。

View on arXiv Download PDF AI Translation

cs.CV / 177 / 2606.16188

teasr: training-efficient any-step diffusion transformer for real-world image super-resolution

TEASR：一种训练高效的任意步扩散Transformer用于真实世界图像超分辨率

Gao, Xiang, Zhu, Chenxin, Fang, Yushun, Hu, Qiang, Zhang, Xiaoyun

Abstract

Diffusion models excel in Real-World Image Super-Resolution (Real-ISR) due to their powerful generative priors but suffer from slow iterative sampling. Although existing one-step distillation methods accelerate inference, they typically require auxiliary teacher models that inflate training memory and restrict scalability to large-scale architectures. Furthermore, these fixed-step models lack the flexibility to trade off speed for quality. In this paper, we propose TEASR, a training-efficient any-step diffusion framework for Real-ISR that enables both one-step and multi-step restoration within a unified model. Our key idea is to perform self-adversarial distillation within a single diffusion model, eliminating the need for auxiliary teachers or discriminators. Specifically, we propose a timestep-aware rectification strategy that stabilizes one-step generation across noise levels. These two designs further enables the distillation of 20B-parameter diffusion models on a single GPU, significantly improving training efficiency. Moreover, we introduce a dual-branch diffusion transformer with decoupled timestep condition to separate the current noise state and the denoising target to enhance sampling quality. Extensive experiments demonstrate that TEASR supports seamless any-step sampling and consistently outperforms state-of-the-art methods across multiple datasets.

Chinese Translation

扩散模型由于其强大的生成先验，在真实世界图像超分辨率（Real-ISR）任务中表现优异，但其缺点是迭代采样速度较慢。尽管现有的一步蒸馏方法能够加速推理，但通常需要辅助教师模型，从而增加训练显存开销，并限制其在大规模架构中的可扩展性。此外，这类固定步数模型缺乏在速度与质量之间进行权衡的灵活性。本文提出TEASR，一种训练高效的任意步扩散框架，用于真实世界图像超分辨率，能够在统一模型中同时支持一步与多步恢复。其核心思想是在单一扩散模型内部进行自对抗蒸馏，从而无需额外的教师模型或判别器。具体而言，我们提出一种时间步感知的校正策略，以稳定不同噪声水平下的一步生成过程。这两项设计进一步使得在单张GPU上对200亿参数规模的扩散模型进行蒸馏成为可能，显著提升训练效率。此外，我们引入双分支扩散Transformer，通过解耦的时间步条件，将当前噪声状态与去噪目标分离，以提升采样质量。大量实验表明，TEASR支持无缝的任意步采样，并在多个数据集上持续优于当前最先进方法。

View on arXiv Download PDF AI Translation

cs.CV / 178 / 2606.16193

Cascaded Sparse Autoencoders Learn Multi-Level Visual Concepts in Multimodal LLMs

级联稀疏自编码器在多模态大语言模型中学习多层次视觉概念

Zhao, Yusong, Wang, Hengyi, Ganu, Tanuja, Nambi, Akshay, Wang, Hao

Abstract

Multimodal Large Language Models (MLLMs) have demonstrated strong performance on vision-language tasks, yet their internal visual representations remain difficult to interpret. Sparse Autoencoders (SAEs) provide a scalable way to decompose dense model activations into sparse, interpretable features. However, existing SAE architectures primarily recover flat feature dictionaries and are less suited for explicit multi-level concept organization. In this paper, we introduce cascaded sparse autoencoders (CSAEs) for learning hierarchical visual concepts in MLLMs. Rather than nesting or stacking SAE sparse activation codes, CSAEs train a second-level SAE directly on the decoder weights of the first-level SAE, treating learned low-level feature directions as inputs for higher-level abstraction. This design enables CSAEs to learn "concepts of concepts" while avoiding drawbacks from the shared-prefix coupling of nesting, Matryoshka-style hierarchies and the bottlenecks of naively stacked SAEs. Experiments across Qwen3-VL, Gemma-3, and LLaVA on multiple visual datasets show that CSAEs improve interpretability in terms of hierarchical concept coherence over state-of-the-art SAE baselines. Results on concept steering further demonstrate that the learned concept groups support effective group-level interventions in MLLM outputs.

Chinese Translation

多模态大语言模型（Multimodal Large Language Models, MLLMs）在视觉-语言任务中表现出强大的能力，但其内部视觉表征仍然难以解释。稀疏自编码器（Sparse Autoencoders, SAEs）提供了一种可扩展的方法，可将致密的模型激活分解为稀疏且可解释的特征。然而，现有的SAE架构主要恢复的是平面的特征词典，不适用于显式的多层次概念组织。本文提出级联稀疏自编码器（cascaded sparse autoencoders, CSAEs），用于在MLLM中学习分层视觉概念。与嵌套或堆叠SAE稀疏激活编码不同，CSAEs在第一层SAE的解码器权重上直接训练第二层SAE，将已学习的低层特征方向作为高层抽象的输入。这一设计使CSAEs能够学习“概念的概念”，同时避免嵌套结构中共享前缀耦合（如Matryoshka式层次结构）带来的缺陷，以及简单堆叠SAE造成的瓶颈问题。在Qwen3-VL、Gemma-3和LLaVA等模型及多个视觉数据集上的实验表明，与当前最优的SAE基线相比，CSAEs在层次概念一致性方面显著提升了可解释性。概念引导（concept steering）实验进一步表明，所学习的概念组能够有效支持对MLLM输出进行组级干预。

View on arXiv Download PDF AI Translation

cs.CV / 179 / 2606.16198

GRACE: Boosting Video MLLMs with Grounded Action-Centric Evidence for Viewer Sentiment Prediction

GRACE：通过基于动作中心的具象证据增强视频多模态大语言模型以进行观众情感预测

Yang, Ruoxuan, Chen, Tieyuan, Huang, Xiaofeng, Yin, Haibing, Wang, Jun, Chen, Xiping, Yin, Jun, Gao, Xuesong, Lin, Weiyao

Abstract

Viewer sentiment prediction in video advertisements aims to infer the latent affective response evoked in the audience. To bridge the gap between what is shown and what is felt, models must deduce hidden viewer emotions from explicit visual narratives, concrete character-object interactions, and visible textual cues. However, standard Multimodal Large Language Models (MLLMs) typically rely on holistic frame representations, which leave these fine-grained, affect-relevant events implicit and complicate precise emotional reasoning. To address this, we propose a grounded action-centric evidence augmentation framework that enhances video MLLMs' clue extraction and comprehension by introducing explicit event structure and localized visual evidence. Our method extracts temporally ordered subject-verb-object (SVO) triplets and auxiliary visible textual cues from action-centric video descriptions, grounds subject and object entities as visual entity crops, and then enables the MLLM to perform clue-enhanced emotional reasoning based on these extracted structured clues. In this way, action triplets specify "what happens", while grounded visual entity crops anchor "who or what participates in each event" to concrete visual evidence. Experiments on the Pitts dataset show consistent improvements over Qwen2.5-VL and Qwen3-VL baselines. Ablation studies, cross-dataset evaluation on AdsQA, and transfer experiments on an emotion-focused TVQA subset further support the effectiveness and generalization of our approach.

Chinese Translation

视频广告中的观众情感预测旨在推断观众被激发的潜在情感反应。为弥合“所见”与“所感”之间的差距，模型需要从显性的视觉叙事、具体的角色-物体交互以及可见的文本线索中推断隐含的观众情绪。然而，标准的多模态大语言模型（Multimodal Large Language Models, MLLMs）通常依赖整体帧表示，使这些细粒度且与情感相关的事件保持隐式表达，从而增加精确情感推理的难度。为此，我们提出一种基于动作中心的具象证据（grounded action-centric evidence）增强框架，通过引入显式的事件结构与局部化视觉证据，提升视频多模态大语言模型的线索提取与理解能力。我们的方法从以动作为中心的视频描述中提取按时间顺序排列的主-谓-宾（SVO）三元组以及辅助的可见文本线索，并将主语与宾语实体映射为视觉实体裁剪区域，从而使多模态大语言模型能够基于这些结构化线索进行增强的情感推理。在该框架下，动作三元组用于明确“发生了什么”，而具象化的视觉实体裁剪则将“谁或什么参与了每个事件”锚定到具体的视觉证据之上。在Pitts数据集上的实验表明，该方法相较于Qwen2.5-VL和Qwen3-VL基线模型均取得了一致提升。消融实验、在AdsQA上的跨数据集评估以及在情感导向TVQA子集上的迁移实验进一步验证了该方法的有效性与泛化能力。

View on arXiv Download PDF AI Translation

cs.CV / 180 / 2606.16202

EgoPhys: Learning Generalizable Physics Models of Deformable Objects from Egocentric Video

EgoPhys：从第一人称视角视频中学习可泛化的可变形物体物理模型

Kim, Hyunjin, Qiu, Ri-Zhao, Jiang, Guangqi, Wang, Xiaolong

Abstract

Humans naturally understand object physics through everyday interactions, but faithfully predicting complex deformable dynamics, such as elastic materials and fabrics, remains a major challenge for computer vision and robotics. We present EgoPhys, a framework that constructs deformable physical digital twins from egocentric RGB-only video using generalizable priors. EgoPhys overcomes the limitations of existing methods to enable controllable deformable digital twin generation from egocentric videos by distilling per-object inverse-physics solutions into a compact codebook, enabling prediction of dense spring stiffness fields for unseen objects without per-spring test-time optimization. Trained with generalizable priors from diverse egocentric interactions, EgoPhys outperforms baselines in reconstruction, future prediction, and zero-shot generalization. To support training and evaluation, we curate an egocentric interaction dataset covering diverse deformable objects, scenes, and manipulation styles. We deploy EgoPhys on a real xArm6 robot, demonstrating that a digital twin initialized from a single egocentric human play video can serve as an internal world representation to aid in deformable-object planning, highlighting egocentric RGB observations as a scalable path toward real-to-sim pipelines.

Chinese Translation

人类能够通过日常交互自然地理解物体物理特性，但对复杂可变形动力学（例如弹性材料和织物）的准确预测仍然是计算机视觉与机器人领域的一大挑战。我们提出EgoPhys，一种利用通用先验从第一人称RGB视频中构建可变形物理数字孪生的框架。EgoPhys通过将每个物体的逆物理求解结果蒸馏为紧凑的codebook，从而克服现有方法的局限，实现基于第一人称视频的可控可变形数字孪生生成，并能够在无需针对每个弹簧进行测试时优化的情况下，预测未见物体的稠密弹簧刚度场。该方法在多样化第一人称交互数据上训练，学习到具有良好泛化能力的先验，在重建、未来状态预测以及零样本泛化任务中均优于基线方法。为支持模型训练与评估，我们构建了一个第一人称交互数据集，涵盖多种可变形物体、场景及操作方式。我们将EgoPhys部署于真实xArm6机器人上，实验表明，仅依赖单段第一人称人类操作视频初始化的数字孪生即可作为内部世界表征，用于辅助可变形物体规划，展示了第一人称RGB观测作为实现真实到仿真（real-to-sim）管线的可扩展路径的潜力。

View on arXiv Download PDF AI Translation

cs.CV / 181 / 2606.16203

DynFS-MoE: Dynamic Functional-Structural Mixture-of-Experts for Post-Traumatic Epilepsy Diagnosis

DynFS-MoE：用于创伤后癫痫（Post-Traumatic Epilepsy, PTE）诊断的动态功能-结构混合专家（Mixture-of-Experts, MoE）模型

Ding, Jun-En, Chen, Spencer, Noren, Henry, Valdivia, Daniel, Yohn, Christine, Patel, Suhina, Zink, Taylor, Sun, Hai, Liu, Feng

Abstract

Post-traumatic epilepsy (PTE) is a severe complication of traumatic brain injury (TBI), yet early identification remains challenging due to the complex structural and functional alterations it induces in the brain. To address this, we propose a dynamic multimodal Mixture-of-Experts (MoE) framework that integrates functional and structural MRI through time-aware functional-structural encoding and class-conditioned expert routing. Within this framework, modality-specific and cross-modal experts learn complementary representations, while a Modality-Class MoE (MCoE) module dynamically dispatches expert weights according to each classification objective. Experimental results across three binary classification tasks demonstrate that the framework consistently outperforms static fusion baselines, and high-interpretability analyses further reveal meaningful region-of-interest (ROI) interactions. This dynamic multimodal expert framework effectively captures class-dependent brain interaction patterns and provides an interpretable approach for PTE diagnosis and risk stratification.

Chinese Translation

创伤后癫痫（Post-traumatic epilepsy, PTE）是创伤性脑损伤（Traumatic Brain Injury, TBI）的一种严重并发症，但由于其在大脑中引起的复杂结构与功能性改变，早期识别仍然具有挑战性。为此，我们提出一种动态多模态混合专家（Mixture-of-Experts, MoE）框架，通过时间感知的功能-结构编码以及类别条件的专家路由机制，融合功能磁共振成像（functional MRI, fMRI）与结构磁共振成像（structural MRI, sMRI）信息。在该框架中，不同模态特异性专家与跨模态专家学习互补表征，同时一个模态-类别混合专家（Modality-Class MoE, MCoE）模块根据不同分类目标动态分配专家权重。实验结果表明，在三个二分类任务上，该框架均持续优于静态融合基线方法；同时，高可解释性分析进一步揭示了具有生物学意义的感兴趣脑区（Region of Interest, ROI）交互模式。该动态多模态专家框架能够有效捕捉类别依赖的脑网络交互模式，并为创伤后癫痫的诊断与风险分层提供了一种具有可解释性的分析方法。

View on arXiv Download PDF AI Translation

cs.CV / 182 / 2606.16212

LUCID: Learned Undersampling-Adaptive Consistency-Guided Inference with Deterministic Flow Matching for Sparse-View CT Reconstruction

LUCID：基于确定性流匹配（Flow Matching）的稀疏视角CT重建中的学习型欠采样自适应一致性引导推理

Duan, Jigang, Wang, Jiayi, Wang, Heran, Yang, Ping, Ma, Genwei, Zhao, Xing

Abstract

Sparse-view CT reduces radiation dose and scanning time by acquiring fewer projection views, but angular undersampling makes reconstruction severely ill-posed, causing streak artifacts, structural blurring, and loss of fine details. Existing supervised methods are often tied to specific sampling settings, whereas generative methods may introduce anatomically inconsistent hallucination-like structures under severe undersampling. We propose Lucid, a sparsity-adaptive, consistency-guided reconstruction framework based on a Flow Matching generative prior for sparse-view CT. Lucid is trained only on high-quality CT images to learn a continuous transport between a Gaussian distribution and the high-quality CT image distribution, independent of view sampling. During inference, the sampling sparsity level is explicitly incorporated to adapt the generative trajectory of a single pretrained model. Specifically, Lucid constructs a degradation-matched initial state by sparsity-weighted fusion of the sparse-view FBP image and Gaussian noise, performs sparsity-modulated Flow Matching updates, and applies projection-domain data-consistency correction after each prior update. Experiments under multiple sparse-view settings show that Lucid achieves stable reconstruction performance across different sampling densities, improves image quality and structural fidelity, and reduces the risk of hallucination-like structures in generative sparse-view CT reconstruction.

Chinese Translation

稀疏视角CT通过减少投影采集角度以降低辐射剂量并缩短扫描时间，但角度欠采样使重建问题成为严重病态问题，从而引发条纹伪影、结构模糊以及细节信息丢失等问题。现有监督学习方法通常依赖特定的采样设置，而生成式方法在严重欠采样条件下可能引入解剖结构不一致的“幻觉式”伪影。我们提出 LUCID，一种基于 Flow Matching 生成先验的稀疏视角CT重建框架，具备稀疏性自适应与一致性引导能力。LUCID仅使用高质量CT图像进行训练，学习高斯分布与高质量CT图像分布之间的连续传输过程，与具体的视角采样方式无关。在推理阶段，将采样稀疏程度显式融入模型，以自适应调整单一预训练模型的生成轨迹。具体而言，LUCID通过对稀疏视角FBP重建图像与高斯噪声进行稀疏加权融合，构建与退化程度匹配的初始状态；随后执行稀疏调制的Flow Matching更新，并在每次先验更新后施加投影域的数据一致性校正。在多种稀疏视角设置下的实验表明，LUCID在不同采样密度条件下均能实现稳定的重建性能，提升图像质量与结构保真度，并降低生成式稀疏视角CT重建中出现“幻觉式”结构的风险。

View on arXiv Download PDF AI Translation

cs.CV / 183 / 2606.16234

Propagating Structural Guidance: Synthesizing Fluorescein Angiography from Fundus Images and Sparse OCT Scans

结构引导传播：基于眼底图像与稀疏OCT扫描的荧光素眼底血管造影合成

Ma, Tengfei, Wu, Ruiqi, Zhang, Chenran, Geng, Ye, Su, Na, Duanmu, Xiangyuan, Zhou, Tao, Zhou, Yi, Fan, Wen

Abstract

Fundus fluorescein angiography (FFA) is critical for assessing retinal vascular abnormalities, but its acquisition is invasive and not always feasible. In contrast, color fundus photography (CFP) is non-invasive and widely accessible, which has motivated studies on CFP-to-FFA synthesis. However, prior works rely solely on CFP surface texture, fundamentally limiting the ability to reconstruct functional vascular information and subtle pathological changes. To address this, we propose a novel framework that synthesizes FFA from CFP with structural guidance provided by optical coherence tomography (OCT). We construct a multi-modal retinal imaging dataset with paired CFP, FFA, and OCT from 3,676 patient eyes--the first tri-modally aligned dataset in retinal imaging. To bridge the spatial gap between OCT and fundus modalities, we propose a Spatially Aligned Cross-Modal Fusion (SACMF) module that projects depth-resolved OCT features onto the fundus plane and injects them into the CFP encoder via adaptive layer normalization. Beyond feature fusion, we further introduce Token-wise Cross-Modality Alignment (TCMA), a token-level contrastive learning strategy that explicitly aligns CFP and FFA representations at corresponding spatial positions. Our method achieves superior synthesis performance compared to state-of-the-art methods. Moreover, extensive experiments demonstrate that the FFA images synthesized by our approach bring greater improvements in downstream disease diagnosis performance than existing methods, highlighting the clinical potential of our approach as a non-invasive decision-support tool in routine workflows. The code is available at https://github.com/while-plus/OCT-guide-FFA-Syn.

Chinese Translation

眼底荧光素血管造影（fundus fluorescein angiography, FFA）在评估视网膜血管异常方面具有重要作用，但其获取过程具有侵入性，且并非总是可行。相比之下，彩色眼底摄影（color fundus photography, CFP）为非侵入式且广泛可用，这也推动了基于CFP生成FFA的研究。然而，以往方法仅依赖CFP的表面纹理信息，从根本上限制了对功能性血管信息以及细微病理变化的重建能力。为此，我们提出一种新框架，在光学相干断层扫描（optical coherence tomography, OCT）提供结构引导的条件下，从CFP合成FFA。我们构建了一个多模态视网膜影像数据集，包含3,676例患者眼部的CFP、FFA与OCT配对数据，这是视网膜影像领域首个三模态对齐数据集。为弥合OCT与眼底影像之间的空间差异，我们提出空间对齐跨模态融合模块（Spatially Aligned Cross-Modal Fusion, SACMF），该模块将具有深度信息的OCT特征投影到眼底平面，并通过自适应层归一化注入CFP编码器。除特征融合外，我们进一步提出逐token跨模态对齐（Token-wise Cross-Modality Alignment, TCMA）策略，这是一种基于token级对比学习的方法，用于在对应空间位置显式对齐CFP与FFA的表征。实验结果表明，我们的方法在合成质量上显著优于现有最先进方法。此外，大量实验显示，由本方法合成的FFA图像在下游疾病诊断任务中相比现有方法带来更显著的性能提升，凸显了其作为非侵入式临床决策支持工具的应用潜力。代码已开源：https://github.com/while-plus/OCT-guide-FFA-Syn。

View on arXiv Download PDF AI Translation

cs.CV / 184 / 2606.16241

Structure-Semantic Co-optimized Latent Diffusion Model for Fast Visual Anagram Synthesis

结构-语义协同优化的潜在扩散模型用于快速视觉字谜合成

Gao, Xiang, Jia, Yunpeng

Abstract

Visual anagram is an intriguing form of art creation wherein a single image presents different conceptual interpretations under transformations such as flipping or rotation. Recent work has achieved visual anagram synthesis by leveraging pretrained text-to-image (T2I) diffusion models, yet still suffers from several key limitations including computational inefficiency, suboptimal aesthetic quality, and weak semantic fidelity and expressiveness. This work focuses on generating visual anagrams with substantially improved visual quality at minimal computational cost, thereby advancing intelligent creation of illusionary digital art. To increase image resolution while reducing time overhead, we adapt the cutting-edge parallel denoising algorithm from pixel-based T2I model to the adversarially distilled latent-based one, and accordingly propose a structure-semantic co-optimization (S2CO) framework to counteract the consequent visual degradation. As the core of our approach, S2CO framework comprises three key innovations: (\romannumeral1) null-text structure alignment optimization; (\romannumeral2) semantic enhancement optimization; (\romannumeral3) attention-guided noise fusion. Building upon these components, our method dubbed \textbf{S2CO-Anagram} is able to generate higher-resolution anagram images with noticeably superior visual harmony and semantic faithfulness than related SOTA approaches, all while achieving substantially faster inference speed. Code will be publicly available.

Chinese Translation

视觉字谜是一种引人入胜的艺术创作形式，其中单一图像在翻转或旋转等变换下呈现出不同的概念解释。近期的研究通过利用预训练的文本到图像（T2I）扩散模型实现了视觉字谜的合成，但仍然面临计算效率低下、审美质量不佳以及语义保真度和表现力弱等几个关键限制。本研究旨在以最低的计算成本生成视觉字谜，显著提高视觉质量，从而推动幻影数字艺术的智能创作。为了在减少时间开销的同时提高图像分辨率，我们将前沿的并行去噪算法从基于像素的T2I模型适配到对抗蒸馏的潜在模型，并相应提出了结构-语义协同优化（S2CO）框架，以抵消随之而来的视觉退化。作为我们方法的核心，S2CO框架包含三个关键创新：( omannumeral1) 空文本结构对齐优化；( omannumeral2) 语义增强优化；( omannumeral3) 注意力引导的噪声融合。在这些组件的基础上，我们的方法被称为 extbf{S2CO-Anagram}，能够生成具有明显更高视觉和谐性及语义真实性的高分辨率字谜图像，且在推理速度上显著快于相关的最先进方法。代码将公开发布。

View on arXiv Download PDF AI Translation

cs.CV / 185 / 2606.16253

Learned Image Compression for Vision-Language-Action Models

面向视觉-语言-动作模型的学习图像压缩

Kim, Hyeonjun, Ryu, Jegwang, Ha, Sangbeom, Lee, Junhyeok, Kim, Jun-Hyuk, Ahn, Hyemin, Lee, Jaeho

Abstract

Vision-language-action (VLA) models increasingly rely on high-frequency multi-camera observations, making visual communication a major bottleneck for real-time robotic control in bandwidth-constrained or distributed deployment settings. Existing image and video codecs, however, are designed to preserve generic visual fidelity rather than the control performance of downstream VLA policies. In this work, we introduce SPARC (SPatially Adaptive Rate Control), a learned image compression framework tailored for VLA-driven robots. Our key observation is that the importance of visual information varies substantially across both camera views and spatial regions within an image. Based on this observation, SPARC employs a lightweight temporal mask selector that adaptively allocates bitrate over latent representations according to task relevance while leveraging temporal context. We further introduce a tilted rate loss that stabilizes training by reducing the tendency of entropy-based objectives to over-suppress rare yet task-critical visual patterns. Experiments on diverse robotic benchmarks, including RoboCasa365, VLABench, and LIBERO, show that SPARC consistently achieves stronger control performance than conventional image/video codecs and recent learned compression methods under the same bitrate budget. We additionally demonstrate real-world deployment benefits in remote-control settings, where our method substantially improves the bitrate-success tradeoff.

Chinese Translation

视觉-语言-动作（VLA）模型越来越依赖于高频多摄像头观测，这使得视觉通信成为带宽受限或分布式部署环境中实时机器人控制的主要瓶颈。然而，现有的图像和视频编解码器旨在保持通用的视觉保真度，而非下游VLA策略的控制性能。在本研究中，我们介绍了SPARC（空间自适应速率控制），这是一个专为VLA驱动的机器人量身定制的学习图像压缩框架。我们的关键观察是，视觉信息的重要性在不同的摄像头视角和图像的空间区域之间有显著差异。基于这一观察，SPARC采用了一种轻量级的时间掩码选择器，根据任务相关性自适应地在潜在表示上分配比特率，同时利用时间上下文。我们进一步引入了一种倾斜速率损失，通过减少基于熵的目标过度抑制稀有但对任务至关重要的视觉模式的倾向，从而稳定训练。在包括RoboCasa365、VLABench和LIBERO在内的多样化机器人基准测试中，实验表明SPARC在相同比特率预算下，始终实现了比传统图像/视频编解码器和近期学习压缩方法更强的控制性能。我们还展示了在远程控制环境中的实际部署优势，我们的方法显著改善了比特率与成功率之间的权衡。

View on arXiv Download PDF AI Translation

cs.CV / 186 / 2606.16255

UniDDT: Unifying Multimodal Understanding and Generation with Decoupled Diffusion Transformer

UniDDT：通过解耦扩散变换器统一多模态理解与生成

Wang, Shuai, Li, Liang, Chen, Yang, Gao, Ruopeng, Teng, Yao, Wang, Limin

Abstract

Unified Multimodal Models (UMMs) have emerged as a critical direction for general-purpose multimodal intelligence, integrating understanding and generation into a single framework. However, existing UMMs face prominent challenges: (1) the inherent learning conflicts between visual understanding and generation tasks, leading to suboptimal modeling in both tasks; (2) different understanding and generation visual spaces impeding scalability; (3) over-reliance on task-specific data that neglects the duality of text-image understanding and generation. To address these challenges, we propose UniDDT, which leverages a Noisy ViT encoder along with an LLM to unify semantic encoding for visual generation and understanding tasks, while employing a separate diffusion decoder to decouple diffusion decoding from text decoding. With this Noisy ViT encoder, UniDDT is able to leverage the latent space as a unified visual representation, enabling seamless compatibility between understanding and generation tasks. Thus, the scalability within the generation tasks and the semantic expressiveness within understanding tasks can be balanced. Also, we construct dual data structures from the same image-text pairs, fostering interdependence between the generation and understanding data to exploit their inherent duality. Extensive experiments demonstrate that UniDDT achieves effective unification of multimodal understanding and generation with enhanced semantic consistency and scalability. For visual generation tasks, our UniDDT achieves 0.87 GenEval score and 86.9 DPG overall score. For multimodal understanding tasks, our UniDDT achieves 1699.5 score on MME benchmark and 76.5 overall score on SEEDbench.

Chinese Translation

统一多模态模型（UMMs）已成为通用多模态智能的重要方向，将理解与生成整合到一个框架中。然而，现有的UMMs面临显著挑战：（1）视觉理解与生成任务之间固有的学习冲突，导致两者建模效果不佳；（2）不同的理解与生成视觉空间妨碍了可扩展性；（3）过度依赖特定任务的数据，忽视了文本-图像理解与生成的双重性。为了解决这些挑战，我们提出了UniDDT，该模型利用噪声ViT编码器和大型语言模型（LLM）统一视觉生成与理解任务的语义编码，同时采用独立的扩散解码器将扩散解码与文本解码解耦。通过这个噪声ViT编码器，UniDDT能够利用潜在空间作为统一的视觉表示，从而实现理解与生成任务之间的无缝兼容。因此，生成任务的可扩展性与理解任务的语义表达能力得以平衡。此外，我们从相同的图像-文本对构建双重数据结构，促进生成与理解数据之间的相互依赖，以利用其固有的双重性。大量实验表明，UniDDT在增强语义一致性和可扩展性的同时，实现了多模态理解与生成的有效统一。在视觉生成任务中，我们的UniDDT达到了0.87的GenEval分数和86.9的DPG总体分数。在多模态理解任务中，我们的UniDDT在MME基准上达到了1699.5分，在SEEDbench上达到了76.5的总体分数。

View on arXiv Download PDF AI Translation

cs.CV / 187 / 2606.16256

KeepLoRA++: Continual Learning with Layer-Scaled Residual Gradient Adaptation

KeepLoRA++：具有层级缩放残差梯度适应的持续学习

Luo, Mao-Lin, Zhang, Yi-Lin, Zhou, Zi-Hao, Hong, Yankun, Tong, Xialiang, Yuan, Mingxuan, Wei, Tong, Zhang, Min-Ling

Abstract

Continual learning for pre-trained vision-language models requires balancing three competing objectives: retaining pre-trained knowledge, preserving knowledge from a sequence of learned tasks, and maintaining the plasticity to acquire new knowledge. This paper presents KeepLoRA++, balancing these objectives through a unified dual-dimensional knowledge retention mechanism. We analyze knowledge distribution of Transformer architecture from both inter-layer and intra-layer perspectives. The inter-layer perspective examines how retention is distributed across layers, while the intra-layer perspective focuses on the parameter space within each layer. Our analysis reveals a structural property: general transferable knowledge is mainly encoded in the shallow layers and the principal subspace of the parameters, while task-specific adaptations are localized in the deep layers and the residual subspace. Motivated by this insight, KeepLoRA++ introduces a layer-scaled residual gradient adaptation method. New tasks are learned by restricting LoRA parameter updates to the residual subspace, combined with a shallow-to-deep layer scaling, to prevent interference with previously acquired capabilities. Specifically, the gradient of a new task is projected onto a subspace orthogonal to both the principal subspace of the pre-trained model and the dominant directions of previous task features, while simultaneously assigning smaller update magnitudes to shallow layers and larger ones to deeper layers. Our theoretical analysis and empirical evaluations confirm that KeepLoRA++ successfully balances these three competing objectives, consistently outperforming representative baselines across image classification, visual question answering, and video understanding tasks.

Chinese Translation

针对预训练视觉-语言模型的持续学习需要平衡三个相互竞争的目标：保留预训练知识、保持从一系列学习任务中获得的知识，以及维持获取新知识的可塑性。本文提出了KeepLoRA++，通过统一的双维知识保留机制来平衡这些目标。我们从层间和层内两个角度分析了Transformer架构的知识分布。层间视角考察了知识在各层之间的分布，而层内视角则关注每层内部的参数空间。我们的分析揭示了一个结构特性：通用可迁移知识主要编码在浅层和参数的主子空间中，而特定任务的适应则局限于深层和残差子空间。基于这一见解，KeepLoRA++引入了一种层级缩放的残差梯度适应方法。通过将LoRA参数更新限制在残差子空间，并结合浅层到深层的缩放，来学习新任务，以防止对先前获得能力的干扰。具体而言，新任务的梯度被投影到一个与预训练模型的主子空间及先前任务特征的主导方向正交的子空间，同时对浅层分配较小的更新幅度，对深层分配较大的更新幅度。我们的理论分析和实证评估证实，KeepLoRA++成功平衡了这三个相互竞争的目标，在图像分类、视觉问答和视频理解任务中始终优于代表性基线。

View on arXiv Download PDF AI Translation

cs.CV / 188 / 2606.16271

Contrastive Learning for Seismic Horizon Tracking with Domain-Specific Priors

基于对比学习的地震地平线追踪与领域特定先验

Thouvenot, Alexandre, Boillot, Lionel, Gripon, Vincent

Abstract

Unsupervised 3D seismic horizon tracking faces a key limitation: signal-based propagators provide accurate trace-level alignment but often fail near faults, whereas texture-driven deep models are more robust to discontinuities, typically at the cost of labeled data requirements and reduced trace-level precision. We propose a self-supervised fusion of both paradigms in which signal-derived local horizon correspondences act as domain-specific priors to train a texture-based deep learning model. Specifically, we estimate reliable trace-to-trace flows from reflector slopes and use them to form positive pairs in a contrastive objective, while restricting training to high-confidence neighborhoods, optionally augmented with a fault mask. The objective is not to infer ambiguous correspondences close to discontinuities, but to preserve horizon identity across them. As a result, the network learns voxel-wise embeddings that preserve local signal continuity while enabling horizon propagation beyond discontinuities through similarity search. Experiments on the public F3 dataset and a faulted synthetic dataset achieve lower mean absolute error (MAE) than unsupervised baselines and competitive performance against a semi-supervised method using a single labeled slice.

Chinese Translation

无监督的三维地震地平线追踪面临一个关键限制：基于信号的传播器提供准确的轨迹级对齐，但在断层附近往往失败，而基于纹理的深度模型对不连续性更具鲁棒性，通常以需要标记数据和降低轨迹级精度为代价。我们提出了一种自监督的两种范式融合方法，其中信号衍生的局部地平线对应作为领域特定先验，用于训练基于纹理的深度学习模型。具体而言，我们从反射器斜率中估计可靠的轨迹间流动，并利用它们在对比目标中形成正样本对，同时将训练限制在高置信度邻域内，选配地使用断层掩膜。目标不是推断接近不连续性时的模糊对应，而是保持地平线在不连续性之间的身份。因此，网络学习到的体素级嵌入能够保持局部信号连续性，同时通过相似性搜索实现地平线在不连续性上的传播。在公共F3数据集和一个带断层的合成数据集上的实验表明，我们的方法在平均绝对误差（MAE）上优于无监督基线，并且在使用单个标记切片的半监督方法中表现出竞争力。

View on arXiv Download PDF AI Translation

cs.CV / 189 / 2606.16274

GraphWorld: Long-Horizon Planning with World Models for End-to-End Autonomous Driving

GraphWorld：基于世界模型的长时间规划端到端自主驾驶

Song, Ziying, Jia, Caiyan, Liu, Lin, Yang, Lei, Zhang, Shengkai, Jia, Feiyang, Zhao, Fengda, Wu, Peiliang, Xu, Shaoqing, Lv, Chen, Luo, Yadan

Abstract

End-to-end autonomous driving has made significant progress by unifying perception, prediction, and planning within a single learning framework, achieving strong performance in short-horizon decision making. However, most existing E2E-AD methods remain confined to short-horizon planning and lack the ability to model long-term temporal dependencies, which severely limits their generalization and security in complex and highly interactive driving scenarios. In this work, we propose GraphWorld, an E2E-AD framework that explicitly enhances long-horizon planning through latent world modeling. We introduce an Ego-Centric Interaction Graph, which adaptively models critical neighboring agents based on spatial proximity, and propagates relational context to planning queries via cross-node cross-attention. We present a World-State-Conditioned Planning that learns ego-centric latent world representations by modeling interactions between an ego vehicle and surrounding agents. This latent world state captures key interaction dynamics and safety-relevant semantics, and serves as a conditioning signal to guide long-horizon, safety-aware trajectory planning. Extensive experiments on Bench2Drive, NAVSIMv1/2, and nuScenes demonstrate that GraphWorld significantly reduces collision rates and improves long-horizon planning performance, validating its effectiveness in complex driving environments.

Chinese Translation

端到端自主驾驶通过将感知、预测和规划统一在一个学习框架内，取得了显著进展，在短时间决策中表现出色。然而，现有的大多数端到端自主驾驶（E2E-AD）方法仍然局限于短时间规划，缺乏建模长期时间依赖性的能力，这严重限制了它们在复杂和高度互动的驾驶场景中的泛化能力和安全性。在本研究中，我们提出了GraphWorld，一个通过潜在世界建模显著增强长时间规划的E2E-AD框架。我们引入了一种以自我为中心的互动图（Ego-Centric Interaction Graph），该图根据空间邻近性自适应地建模关键的邻近代理，并通过跨节点的交叉注意力将关系上下文传播到规划查询中。我们提出了一种世界状态条件规划（World-State-Conditioned Planning），通过建模自我车辆与周围代理之间的互动来学习以自我为中心的潜在世界表示。该潜在世界状态捕捉了关键的互动动态和与安全相关的语义，并作为条件信号指导长时间、安全意识的轨迹规划。在Bench2Drive、NAVSIMv1/2和nuScenes上的大量实验表明，GraphWorld显著降低了碰撞率，并改善了长时间规划性能，验证了其在复杂驾驶环境中的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 190 / 2606.16278

RealityBridge: Bridging Editable 3D Gaussian Splatting Driving Simulations and Real-World Videos

RealityBridge：桥接可编辑的 3D 高斯溅射驱动仿真与真实世界视频

Wu, Zhenhua, Pang, Yun, Chang, Mingkun, Ning, Yuwei, Wang, Liangzhi, Xiao, Yi, Li, Guanbin

Abstract

Long-tail hazardous scenarios are essential for safety-oriented autonomous driving, yet they are difficult to collect and reproduce at scale. Editable 3D Gaussian Splatting (3DGS) simulation offers a promising alternative by reconstructing real driving scenes and supporting controllable scene editing. However, edited 3DGS-rendered videos still suffer from a significant Sim-to-Real gap, including rendering artifacts, degraded foreground assets, inconsistent illumination, and temporal flickering. Existing restoration and video generation methods are insufficient for this task, as they often fail to jointly repair 3DGS-specific artifacts, improve visual realism, and ensure temporal consistency. To fill this gap, we propose RealityBridge, a structure-preserving and asset-aware Sim-to-Real framework for edited 3DGS driving videos. RealityBridge uses multimodal controls, including rendered videos, foreground masks, edge maps, and semantic masks, together with a lightweight GateNet for adaptive condition allocation across backbone layers. We further construct targeted training data and introduce autoregressive long-video training with reward-guided post-training to improve restoration quality, temporal stability, and hallucination suppression. Extensive experiments on internal and public driving datasets show that RealityBridge outperforms existing methods in artifact removal, illumination harmonization, and long-sequence temporal consistency.

Chinese Translation

长尾危险场景对于以安全为导向的自动驾驶至关重要，但在大规模收集和重现方面存在困难。可编辑的 3D 高斯溅射（3DGS）仿真通过重建真实驾驶场景并支持可控场景编辑，提供了一种有前景的替代方案。然而，经过编辑的 3DGS 渲染视频仍然存在显著的仿真到现实（Sim-to-Real）差距，包括渲染伪影、前景资产退化、不一致的照明和时间闪烁。现有的修复和视频生成方法不足以应对这一任务，因为它们往往无法共同修复 3DGS 特有的伪影、提高视觉真实感并确保时间一致性。为填补这一空白，我们提出了 RealityBridge，这是一个结构保持和资产感知的仿真到现实框架，专为编辑的 3DGS 驾驶视频设计。RealityBridge 使用多模态控制，包括渲染视频、前景掩膜、边缘图和语义掩膜，并结合轻量级的 GateNet 在主干层之间进行自适应条件分配。我们进一步构建了针对性训练数据，并引入了自回归长视频训练与奖励引导的后训练，以提高修复质量、时间稳定性和幻觉抑制。在内部和公共驾驶数据集上的广泛实验表明，RealityBridge 在伪影去除、照明协调和长序列时间一致性方面优于现有方法。

View on arXiv Download PDF AI Translation

cs.CV / 191 / 2606.16294

Sex-based Network-Specific Differences in Connectomes: A Krakencoder-Based Analysis

基于性别的网络特异性连接组差异：Krakencoder分析

H, Vibhashree S, Bhattacharya, Debanjali, Kancharla, Vamshi Krishna, Sinha, Neelam

Abstract

This study examines how deficiencies in one brain connectome modality propagate to the other, using the Krakencoder as a simulation framework. Structural and functional connectomes from 702 healthy participants in the Human Connectome Project were analyzed, with the impact of each of the Yeo-7 functional networks assessed separately. Seven scenarios were considered, each involving the removal of a single network while the remaining networks were preserved. The resulting perturbations in cross-modal predictions were quantified using three complementary metrics: KL divergence on eigenvalue spectra, Frobenius norm, and Wasserstein distance. In addition, the persistence of sex-specific information within the predicted connectomes was evaluated. Across all metrics and both prediction directions, the Default Mode Network produced the largest perturbations, whereas the Somatomotor network yielded the smallest. Sex differences in network-level perturbation signatures were subtle, with the best result being an accuracy of 66.09% from connectomes predicted under network-removal conditions. In contrast, connectomes predicted from intact inputs achieved substantially higher sex classification accuracy, reaching up to 84.76%. These findings confirm that full predicted connectomes retain considerably more sex-discriminative information than perturbation-derived signatures alone.

Chinese Translation

本研究探讨了一个脑连接组模态中的缺陷如何传播到另一个模态，使用Krakencoder作为模拟框架。分析了来自人类连接组项目（Human Connectome Project）中702名健康参与者的结构和功能连接组，并分别评估了每个Yeo-7功能网络的影响。考虑了七种情境，每种情境涉及移除一个单一网络，同时保留其余网络。通过三种互补指标量化了跨模态预测中的扰动：特征值谱的KL散度、Frobenius范数和Wasserstein距离。此外，还评估了预测连接组中性别特异性信息的持久性。在所有指标和预测方向中，默认模式网络（Default Mode Network）产生了最大的扰动，而躯体运动网络（Somatomotor network）则产生了最小的。网络级扰动特征中的性别差异较为微妙，最佳结果是在网络移除条件下预测的连接组的准确率为66.09%。相比之下，从完整输入预测的连接组实现了显著更高的性别分类准确率，达到84.76%。这些发现确认，完整的预测连接组保留了比单独扰动衍生特征更多的性别区分信息。

View on arXiv Download PDF AI Translation

cs.CV / 192 / 2606.16295

VisualClaw: A Real-Time, Personalized Agent for the Physical World

VisualClaw：一个实时的个性化物理世界代理

Tu, Haoqin, Chen, Jianwen, Wang, Zijun, Han, Siwei, Wu, Juncheng, Chen, Hardy, Ji, Haonian, Xiong, Kaiwen, Liu, Jiaqi, Xia, Peng, Mei, Jieru, Fei, Hongliang, Eshraghian, Jason, Zheng, Zeyu, Zhou, Yuyin, Yao, Huaxiu, Xie, Cihang

Abstract

Vision language models are serving as general-purpose interfaces for complex multimodal tasks. However, deployment still faces three gaps: VLMs typically incur high latency and cost when processing dense video frames and long prompts, the agent scaffold remains static after deployment, and standard video-QA benchmarks do not test whether agents can use visual evidence inside tool-using workspaces. We present VisualClaw, a self-evolving multimodal agent built around two principles. First, hybrid encoding reduces deployment cost by filtering less informative streaming frames with a cascaded gate and compressing the text skill bank through hot/cold top-k injection. Second, skill evolution lets the agent learn from failures: retrieved memories condition an evolver as direct concatenated context or as guided evidence, producing skill-bank updates that help future questions. Across 4 video-QA benchmarks with 2 VLMs, VisualClaw cuts per-question API cost by an average -98% versus full-frame upload and by -25.9% over the offline uniform 8 frame baseline, while boosting accuracy in most settings, e.g., an average +3.85% and a peak +15.80% on EgoSchema with Gemini 3 Flash. To address the gap, we curate VisualClawArena, a 200-scenario multimodal agentic benchmark built through a strict five-stage pipeline; models must use video evidence, documents, dynamic updates, and executable checks inside a workspace. On VisualClawArena, the same framework with computer-use agent backends improves macro accuracy by +2.9% for Codex (GPT-5.5) and +3.2% for Claude Code (Sonnet 4.6) over no-evolution baselines, with a -9.5% cost reduction compared to the uniform-sampled baseline. These properties make VisualClaw a natural fit for edge applications, where the cascade reduces a 1-hour streaming session from ~3,600 API uploads down to only 5-20 calls and the self-evolution makes it a perfect personalized assistant.

Chinese Translation

视觉语言模型作为复杂多模态任务的通用接口。然而，部署仍面临三个问题：在处理密集视频帧和长提示时，视觉语言模型（VLMs）通常会产生高延迟和高成本，代理框架在部署后保持静态，并且标准的视频问答基准测试并未测试代理是否能够在使用工具的工作空间中利用视觉证据。我们提出了VisualClaw，一个基于两个原则构建的自我演化多模态代理。首先，混合编码通过级联门过滤不太有信息量的流媒体帧，并通过热/冷 top-k 注入压缩文本技能库，从而降低部署成本。其次，技能演化使代理能够从失败中学习：检索的记忆作为直接连接的上下文或作为引导证据来调节演化器，产生技能库更新，帮助未来的问题。在4个视频问答基准测试中，使用2个视觉语言模型，VisualClaw将每个问题的API成本平均降低了98%，相比于全帧上传，并在离线均匀8帧基线基础上降低了25.9%，同时在大多数设置中提高了准确性，例如，在EgoSchema上，平均提高了3.85%，峰值提高了15.80%（使用Gemini 3 Flash）。为了解决这一差距，我们策划了VisualClawArena，这是一个通过严格的五阶段流程构建的200场景多模态代理基准；模型必须在工作空间内使用视频证据、文档、动态更新和可执行检查。在VisualClawArena上，使用计算机代理后端的相同框架相比于无演化基线，Codex（GPT-5.5）的宏观准确性提高了2.9%，Claude Code（Sonnet 4.6）提高了3.2%，同时与均匀采样基线相比，成本降低了9.5%。这些特性使得VisualClaw非常适合边缘应用，其中级联将1小时的流媒体会话从约3600次API上传减少到仅5-20次调用，而自我演化使其成为完美的个性化助手。

View on arXiv Download PDF AI Translation

cs.CV / 193 / 2606.16298

DDTNet: Degradation Disentanglement and Transfer Network for Test-Time All-in-One De-weathering Adaptation

DDTNet：用于测试时一体化去天气适应的降解解耦与转移网络

Lin, Kuan-Hung, Tsai, Fu-Jen, Peng, Yan-Tsung, Chen, Min-Hung, Lin, Chia-Wen, Lin, Yen-Yu

Abstract

All-in-one adverse weather image restoration aims to remove multiple degradations, such as rain, haze, and snow, using a single unified model. Despite their broad applicability, existing methods typically compromise performance, delivering balanced but suboptimal results for individual degradation types. This issue becomes more pronounced when a domain gap exists between training and testing data. Motivated by the observation that modeling degradation patterns is more feasible than recovering clean content, we propose the Degradation Disentanglement and Transfer Network (DDTNet), which focuses specifically on degradation transfer. By disentangling degradation patterns from target-domain degraded images and transferring them to source domain clean images, DDTNet generates domain-adaptive paired training data. These pairs are then used to fine-tune restoration models, significantly enhancing their adaptability across diverse weather conditions and domains. The core of DDTNet is the Degradation Disentanglement Module (DDM), which comprises Degradation Coupled Attention (DCA) to capture both general and weather-specific features, thereby enabling effective disentanglement and transfer of degradation patterns. Experimental results demonstrate that DDTNet significantly and consistently improves existing all-in-one models across real-world deraining, desnowing, and dehazing datasets.

Chinese Translation

一体化恶劣天气图像恢复旨在使用单一统一模型去除多种降解，如雨、雾和雪。尽管现有方法具有广泛的适用性，但通常会妥协性能，为各个降解类型提供平衡但次优的结果。当训练数据与测试数据之间存在领域差距时，这一问题更加明显。基于降解模式建模比恢复干净内容更可行的观察，我们提出了降解解耦与转移网络（DDTNet），专注于降解转移。通过从目标领域的降解图像中解耦降解模式，并将其转移到源领域的干净图像上，DDTNet生成领域自适应的配对训练数据。这些配对数据随后用于微调恢复模型，显著增强其在不同天气条件和领域中的适应性。DDTNet的核心是降解解耦模块（DDM），该模块包含降解耦合注意力（DCA），用于捕捉一般特征和特定天气特征，从而有效地解耦和转移降解模式。实验结果表明，DDTNet在现实世界的去雨、去雪和去雾数据集上显著且持续地改善了现有的一体化模型。

View on arXiv Download PDF AI Translation

cs.CV / 194 / 2606.16302

Explainable Flood Segmentation on Sentinel-1 SAR Imagery: A Comparative Study of CNN and Transformer Architectures

基于Sentinel-1 SAR影像的可解释洪水分割：CNN与Transformer架构的比较研究

Banerjee, Arundhuti, Daou, David

Abstract

Rapid and accurate flood prediction is essential for disaster response and mitigation planning. Synthetic Aperture Radar (SAR) sensors in satellites are well-suited for this purpose because they operate independently of weather and daylight conditions. Although SAR-based data enable all-weather flood monitoring, distinguishing flooded land from permanent water remains a significant challenge, particularly when flooding is defined strictly as inundated land. This study provides a comprehensive comparison of convolutional neural network (CNN) and vision transformer architectures for multi-class flood segmentation using Sentinel-1 SAR imagery, specifically trained to separate flooded land from permanent water bodies and land. Three state-of-the-art (SOTA)CNN-based models, U-Net, U-Net++, and DeepLabV3 with ResNet-34 backbone, and three SegFormer variants (b0,b1,b2) were evaluated in two benchmark datasets, the ETCI NASA dataset and SenFloods11, using scene-based data splits to ensure a realistic assessment of spatial generalization. The results demonstrate that SegFormer-b2 significantly outperforms the U-Net baseline on the ETCI dataset (higher flood IoU across all 7 test scenes in the Wilcoxon signed-rank test), while after fine-tuning on Sen1Floods11, the advantage narrows to within the range of scene variability and is concentrated in spatially fragmented flood events. The study includes both qualitative and quantitative explainability techniques to visually comprehend model decisions and systematically assess prediction reliability. Qualitative analysis reveals that SegFormer-b2 produces more spatially coherent Grad-CAM activations focused on flood-relevant features, while U-Net generates more informative uncertainty estimates along flood boundaries.

Chinese Translation

快速而准确的洪水预测对于灾害响应和减灾规划至关重要。合成孔径雷达（SAR）传感器在卫星中非常适合此目的，因为它们不受天气和光照条件的影响。尽管基于SAR的数据能够实现全天候洪水监测，但将淹没的土地与永久水体区分开来仍然是一个重大挑战，尤其是当洪水严格定义为被淹没的土地时。本研究对卷积神经网络（CNN）和视觉变换器架构在使用Sentinel-1 SAR影像进行多类洪水分割方面进行了全面比较，特别是训练以区分被淹没的土地与永久水体和土地。我们评估了三种最先进的（SOTA）基于CNN的模型：U-Net、U-Net++和以ResNet-34为骨干的DeepLabV3，以及三种SegFormer变体（b0、b1、b2），在两个基准数据集ETCI NASA数据集和SenFloods11上进行评估，采用基于场景的数据划分以确保对空间泛化的现实评估。结果表明，SegFormer-b2在ETCI数据集上显著优于U-Net基线（在Wilcoxon符号秩检验中，所有7个测试场景的洪水IoU更高），而在Sen1Floods11上微调后，其优势缩小至场景变异范围内，并集中在空间上碎片化的洪水事件中。该研究包括定性和定量的可解释性技术，以直观理解模型决策并系统评估预测可靠性。定性分析表明，SegFormer-b2产生了更具空间一致性的Grad-CAM激活，集中在与洪水相关的特征上，而U-Net在洪水边界沿线生成了更具信息性的不确定性估计。

View on arXiv Download PDF AI Translation

cs.CV / 195 / 2606.16317

Training-free sparse attention based on cumulative energy filtering

基于累积能量过滤的无训练稀疏注意力

Li, Chunlu, Pan, Yixuan, Du, Bai, Chen, Zhenyuan, Li, Yanzhao, Dong, Hui, Wang, Hui, Zou, Zhiqiang

Abstract

Sparse attention accelerates Diffusion Transformers (DiTs) for video generation by computing only the important tokens while skipping the rest. The token selection strategy is key to balancing sparsity and accuracy. We formulate the token filtering process as a dual-goal optimization problem: maximizing sparsity and minimizing accuracy degradation. Existing algorithms cannot fulfill both objectives simultaneously. For example, Top-p only considers the accuracy constraint, while Top-k maintains a fixed computational budget but loosens the accuracy constraint. This paper demonstrates that maintaining a fixed recall rate is sufficient for ensuring accuracy, whereas a fixed threshold is suboptimal for reducing computational cost. Therefore, we propose a dynamic thresholding scheme to improve sparsity while maintaining the same level of accuracy. Furthermore, our algorithm is deeply integrated with Flash Attention (FA), eliminating the need for any additional masking computation overhead. Experimental results on Wan 2.2 validate that, compared to the BLASST algorithm which is also integrated with FA, our dynamic thresholding strategy enhances sparsity from 61.42\% to 82\% with a VBench metric drop of less than 5\%. This results in an approximate 15\% in attention computation and a $1.61\times$ increase in computational efficiency, which is 1.18x higher than that of BLASST.

Chinese Translation

稀疏注意力通过仅计算重要的标记而跳过其余部分，从而加速了用于视频生成的扩散变换器（Diffusion Transformers, DiTs）。标记选择策略是平衡稀疏性和准确性的关键。我们将标记过滤过程形式化为一个双重目标优化问题：最大化稀疏性并最小化准确性下降。现有算法无法同时满足这两个目标。例如，Top-p 仅考虑准确性约束，而 Top-k 则保持固定的计算预算，但放宽了准确性约束。本文表明，保持固定的召回率足以确保准确性，而固定阈值在降低计算成本方面是次优的。因此，我们提出了一种动态阈值方案，以提高稀疏性，同时保持相同水平的准确性。此外，我们的算法与闪电注意力（Flash Attention, FA）深度集成，消除了任何额外的掩蔽计算开销。在 Wan 2.2 上的实验结果验证了，与同样集成了 FA 的 BLASST 算法相比，我们的动态阈值策略将稀疏性从 61.42\% 提高到 82\\%，同时 VBench 指标下降不足 5\\%。这导致注意力计算大约减少 15\\%，计算效率提高了 $1.61 imes$，比 BLASST 高出 1.18 倍。

View on arXiv Download PDF AI Translation

cs.CV / 196 / 2606.16323

HAFMat: Hybrid Priors Guided Adaptive Fusion for Single-Image Human Material Estimation

HAFMat：基于混合先验引导的单幅图像人类材质估计的自适应融合

Jiang, Yu, Xia, Jiahao, Qin, Jiongming, Sun, Jianchi, Xiao, Chunxia

Abstract

Physically based rendering (PBR) material estimation is a fundamental appearance decomposition task with broad applications in virtual content creation, relighting, and digital human rendering. However, estimating PBR materials from a single human image remains highly ill-posed, since illumination, geometry, and reflectance are heavily entangled in the observed appearance. To mitigate this ambiguity, we propose HAFMat, a hybrid-prior-guided framework for single-image human material estimation. Our method introduces guidance maps that encode complementary cues, including appearance, body geometry, structure, and prior material predictions from pre-trained models. A key observation is that these guidance cues are heterogeneous: some cues mainly provide texture-level constraints, while others convey higher-level semantic information. To exploit this property, we design a Multi-layer Adaptive Feature Fusion Mechanism, which adaptively fuses guidance features with decoder features at different stages. This design enables texture-dominant and semantic-dominant cues to guide material decoding at appropriate levels, leading to more accurate and physically plausible material estimation. Extensive experiments on both synthetic and real data demonstrate that our method achieves state-of-the-art performance in material estimation and downstream relighting.

Chinese Translation

基于物理的渲染（PBR）材质估计是一个基础的外观分解任务，在虚拟内容创作、重光照和数字人类渲染等领域具有广泛的应用。然而，从单幅人类图像中估计PBR材质仍然高度不适定，因为照明、几何形状和反射率在观察到的外观中紧密交织。为了减轻这种模糊性，我们提出了HAFMat，一种基于混合先验引导的单幅图像人类材质估计框架。我们的方法引入了编码互补线索的引导图，包括外观、身体几何、结构和来自预训练模型的先验材质预测。一个关键观察是这些引导线索是异质的：一些线索主要提供纹理级别的约束，而其他线索则传达更高层次的语义信息。为了利用这一特性，我们设计了一种多层自适应特征融合机制，该机制在不同阶段自适应地将引导特征与解码器特征融合。这一设计使得以纹理为主和以语义为主的线索能够在适当的层次上引导材质解码，从而实现更准确和物理上合理的材质估计。在合成数据和真实数据上的大量实验表明，我们的方法在材质估计和后续重光照任务中达到了最先进的性能。

View on arXiv Download PDF AI Translation

cs.CV / 197 / 2606.16325

Attention-Based Prototype Calibration for Multi-Rater Few-Shot Medical Image Segmentation

基于注意力的多评审者少样本医学图像分割原型校准

Vu, Truong, Ho, Minh Khoi, Xie, Yutong

Abstract

Few-shot medical image segmentation methods typically assume a single ground-truth annotation, overlooking systematic variability across expert raters commonly observed in clinical datasets. We propose an attention-based prototype calibration framework for few-shot multi-rater segmentation that models rater-specific deviations from a consensus representation in prototype space. A lightweight yet principled attention operator directly refines rater prototypes without modifying the backbone feature extractor, making the approach fully compatible with existing prototype-based few-shot segmentation methods. This design preserves semantic consistency while enabling personalized segmentation outputs with minimal computational overhead. Experiments on multi-rater medical imaging datasets demonstrate consistent improvements over baseline prototype approaches, highlighting the effectiveness of structured prototype calibration for modeling annotation variability. Our code is available at https://github.com/truong2710-cyber/JAPC.

Chinese Translation

少样本医学图像分割方法通常假设存在单一的真实标注，忽视了在临床数据集中常见的专家评审者之间的系统性变异。我们提出了一种基于注意力的原型校准框架，用于少样本多评审者分割，该框架在原型空间中建模评审者特定的偏差与共识表示之间的关系。一个轻量级但原理明确的注意力操作符直接细化评审者原型，而无需修改主干特征提取器，使得该方法与现有的基于原型的少样本分割方法完全兼容。该设计在保持语义一致性的同时，能够以最小的计算开销实现个性化的分割输出。在多评审者医学影像数据集上的实验表明，相较于基线原型方法，结构化原型校准在建模标注变异性方面表现出一致的改进。我们的代码可在 https://github.com/truong2710-cyber/JAPC 获取。

View on arXiv Download PDF AI Translation

cs.CV / 198 / 2606.16333

Differentiable Packing of Irregular 3D Objects with Adaptive Container Estimation

可微分的非规则三维物体打包与自适应容器估计

Gupta, Palak, Raman, Shanmuganathan

Abstract

Most existing approaches either fix the container in advance or optimize only a single container dimension through an outer search loop, leaving the remaining dimensions as a manual tuning problem. We present a differentiable packing framework that jointly optimizes all 6N object pose parameters and all three container side lengths inside a single gradient-based loop. The formulation combines six physics-inspired, differentiable loss terms computed directly on triangle meshes through axis-aligned bounding-box proxies. An adaptive squeezing mechanism periodically tightens the container whenever the overlap loss falls below a pair-count-scaled threshold, producing a large initial drop in container volume, followed by small refinements. All pairwise computations are written in tensor-broadcasting form, giving a 3.4 to 54 times speedup over a reference loop-based implementation. The pipeline is implemented in Python and PyTorch, with no physics engine, FFT library, or convex decomposition. On multiple object categories, the method produces containers that are 11 to 32 percent smaller than time-matched DBLF and simulated-annealing baselines at N =100, while running in under 4 minutes per instance on a single consumer GPU.

Chinese Translation

大多数现有方法要么提前固定容器，要么仅通过外部搜索循环优化单个容器维度，导致其余维度成为手动调优问题。我们提出了一种可微分的打包框架，该框架在单个基于梯度的循环中联合优化所有6N个物体姿态参数和三个容器边长。该公式结合了六个受物理启发的可微分损失项，这些损失项通过轴对齐的包围盒代理直接在三角网格上计算。自适应挤压机制在重叠损失低于成对计数缩放阈值时定期收紧容器，产生容器体积的大幅初始下降，随后进行小幅度的细化。所有成对计算均以张量广播形式编写，相较于参考的基于循环的实现，速度提升为3.4到54倍。该管道在Python和PyTorch中实现，无需物理引擎、FFT库或凸分解。在多个物体类别中，该方法生成的容器比时间匹配的DBLF和模拟退火基线在N=100时小11%到32%，且每个实例的运行时间少于4分钟，使用单个消费级GPU。

View on arXiv Download PDF AI Translation

cs.CV / 199 / 2606.16334

Chronological Blindness: Benchmarking Temporal Reasoning in Vision-Language Models with CHRONOSIGHT

时间盲点：使用 CHRONOSIGHT 基准评估视觉-语言模型中的时间推理

Goswami, Parthaw, Deep, Jaynto Goswami

Abstract

Human perception of visual scenes is inherently temporal. We instinctively recognise whether a fruit is ripening or rotting, whether construction is progressing or being demolished, and approximately how much time separates two photographs of the same subject. Whether large vision-language models (VLMs) share this competence remains an open and practically important question. We introduce CHRONOSIGHT, a rigorously controlled benchmark evaluating five dimensions of visual temporal reasoning: CHRONORANK (chronological ordering of image sequences), CHRONOLOCATE (ordinal stage localisation from a single image), CHRONODELTA (estimation of time elapsed between two images on a logarithmic scale), CHRONOREVERSE (detection of temporally reversed sequences), and CHRONOODD (identification of a temporal outlier within a set). The benchmark comprises 1{,}000 items across eight process families (biological growth, food transformation, physical weathering, construction, environmental change, human ageing, astronomical phenomena, and urban dynamics) spanning timescales from minutes to millennia. We evaluate eight open-source VLMs (500 M to 19 B parameters) under two prompting regimes and collect human performance baselines. Human performance averages 0.89 across tasks; the best open model (Qwen2.5-VL-7B) reaches 0.40 under direct prompting, a gap we term chronological blindness. Lightweight LoRA fine-tuning on 151 examples raises CHRONODELTA accuracy from near-zero to 0.43, transferring zero-shot to related tasks (CHRONOODD: 0.37; CHRONOREVERSE: 0.64)suggesting the bottleneck is partly instruction following rather than visual perception. Benchmark, code, and predictions will be released upon acceptance.

Chinese Translation

人类对视觉场景的感知本质上是时间性的。我们本能地识别水果是成熟还是腐烂，建筑是正在施工还是被拆除，以及大约有多少时间将两张同一主题的照片分隔开。大型视觉-语言模型（VLMs）是否具备这种能力仍然是一个开放且具有实际重要性的问题。我们引入了 CHRONOSIGHT，这是一个严格控制的基准，评估视觉时间推理的五个维度：CHRONORANK（图像序列的时间顺序）、CHRONOLOCATE（从单张图像中确定的序列阶段）、CHRONODELTA（在对数尺度上估计两张图像之间经过的时间）、CHRONOREVERSE（检测时间倒置的序列）和 CHRONOODD（在一组中识别时间异常值）。该基准包含 1,000 个项目，涵盖八个过程类别（生物生长、食品转化、物理风化、建筑、环境变化、人类衰老、天文现象和城市动态），时间跨度从几分钟到几千年。我们在两种提示机制下评估了八个开源 VLM（参数从 5 亿到 190 亿），并收集了人类表现的基线。人类在各项任务中的平均表现为 0.89；最佳开源模型（Qwen2.5-VL-7B）在直接提示下达到 0.40，我们称之为时间盲点。在 151 个示例上进行轻量级 LoRA 微调将 CHRONODELTA 的准确率从接近零提高到 0.43，并在相关任务上实现零样本迁移（CHRONOODD: 0.37；CHRONOREVERSE: 0.64），这表明瓶颈部分在于指令遵循而非视觉感知。基准、代码和预测将在接受后发布。

View on arXiv Download PDF AI Translation

cs.CV / 200 / 2606.16342

When the Past Matters: FlashBack Memory for Precipitation Nowcasting

当过去变得重要：用于降水即时预报的FlashBack记忆

Du, Yuhao, Huang, Boxiao, Wu, Chengrong, Zhang, Jiankai

Abstract

Accurate precipitation nowcasting is crucial for disaster mitigation and socio-economic planning, yet existing methods often struggle with false alarms, missed events, and long range dependency modeling at high spatiotemporal resolution. To address these challenges, we propose FlashBack Memory (FB), a module that dynamically retrieves key historical states and integrates them via an adaptive fusion gate, enhancing the spatiotemporal representation capability of recurrent-based models. We incorporate FB into PredRNN, PredRNNpp, MIM, MotionRNN, and PredRNN-V2, and evaluate on CIKM2017, Shanghai2020, and SEVIR datasets. Experimental results demonstrate that FB significantly improves MSE, MAE, SSIM, and CSI metrics, particularly for high-intensity rainfall and long-sequence predictions, while reducing false alarms and missed events and enhancing temporal consistency and spatial localization. The proposed method provides a general and efficient memory enhancement mechanism, improving the overall performance of recurrent-based precipitation nowcasting models.

Chinese Translation

准确的降水即时预报对于灾害减缓和社会经济规划至关重要，但现有方法常常面临误报、漏报以及在高时空分辨率下的长程依赖建模等挑战。为了解决这些问题，我们提出了FlashBack Memory (FB) 模块，该模块动态检索关键历史状态，并通过自适应融合门进行整合，从而增强基于递归模型的时空表示能力。我们将FB集成到PredRNN、PredRNNpp、MIM、MotionRNN和PredRNN-V2中，并在CIKM2017、上海2020和SEVIR数据集上进行评估。实验结果表明，FB显著提高了均方误差 (MSE)、平均绝对误差 (MAE)、结构相似性指数 (SSIM) 和一致性指数 (CSI) 等指标，尤其是在高强度降雨和长序列预测方面，同时减少了误报和漏报，增强了时间一致性和空间定位能力。所提出的方法提供了一种通用且高效的记忆增强机制，提升了基于递归的降水即时预报模型的整体性能。

View on arXiv Download PDF AI Translation

cs.CV / 201 / 2606.16353

What Should a Streaming Video Model Remember?

流媒体视频模型应记住什么？

Ge, Haonan, Wang, Yiwei, Wu, Hang, Cai, Yujun

Abstract

Streaming video understanding models must answer queries at any moment during an ongoing stream, using only what they have observed so far and under fixed memory and computation budgets. Existing methods address this by adding memory banks, retrieval modules, or visual token compression to preserve long-range history. However, strong recent-window baselines show that indiscriminate history injection can dilute current-scene perception, suggesting that the key challenge is not whether to use memory, but how to allocate it selectively. We formulate this as budgeted online latent evidence allocation and propose \textbf{SelectStream}, a selective latent-memory framework that keeps the current observation directly visible to a frozen VLM while exposing historical information only through a compact, query-conditioned evidence budget. Three coordinated mechanisms govern when to write, what to preserve, and how to retrieve: surprise-driven adaptive windowing, priority-preserving consolidation, and query-conditioned graph reasoning over a fixed-capacity latent memory graph. Retrieved evidence is calibrated and injected as latent tokens for answer generation, without replaying frames or growing the context with stream length. Experimental results show that SelectStream achieves strong online streaming performance and preserves general video understanding, reaching 82.67\% on StreamingBench, 67.03\% on OVO-Bench, and 74.4\% average accuracy on offline video benchmarks, while outperforming strong recent-window baselines and prior streaming memory methods.

Chinese Translation

流媒体视频理解模型必须能够在持续流媒体的任何时刻回答查询，仅使用它们迄今为止观察到的信息，并在固定的内存和计算预算下进行操作。现有方法通过添加内存库、检索模块或视觉令牌压缩来解决这个问题，以保留长时间的历史记录。然而，强大的近期窗口基线表明，随意注入历史信息可能会稀释当前场景的感知，这表明关键挑战不在于是否使用内存，而在于如何选择性地分配内存。我们将此表述为预算在线潜在证据分配，并提出了 extbf{SelectStream}，一种选择性潜在记忆框架，该框架使当前观察结果直接对一个冻结的视觉语言模型（VLM）可见，同时仅通过一个紧凑的、查询条件的证据预算来暴露历史信息。三个协调机制决定何时写入、保留什么以及如何检索：基于惊讶驱动的自适应窗口、优先级保留的整合，以及在固定容量潜在记忆图上的查询条件图推理。检索到的证据经过校准并作为潜在令牌注入以生成答案，而无需重放帧或随着流长度的增加而扩展上下文。实验结果表明，SelectStream在在线流媒体性能上表现出色，并保留了通用视频理解，在StreamingBench上达到了82.67\%，在OVO-Bench上达到了67.03\%，在离线视频基准上达到了74.4\%的平均准确率，同时超越了强大的近期窗口基线和之前的流媒体记忆方法。

View on arXiv Download PDF AI Translation

cs.CV / 202 / 2606.16354

GraphBEV++: Multi-Modal Feature Alignment for Autonomous Driving

GraphBEV++：用于自动驾驶的多模态特征对齐

Song, Ziying, Jia, Caiyan, Liu, Lin, Xu, Shaoqing, Yang, Lei, Luo, Yadan

Abstract

Feature misalignment in BEV perception is a critical yet often overlooked challenge in autonomous driving, especially under calibration uncertainties between LiDAR and camera sensors. To address this issue, we propose a robust multi-modal fusion framework, GraphBEV++, which systematically mitigates projection-induced misalignment. The framework consists of two key modules: LocalAlign-v2 and GlobalAlign-v2. LocalAlign-v2 introduces neighborhood-aware depth features via graph matching to correct local misalignment. It supports both LSS-based and query-based BEV representations, making it compatible with BEVFusion and BEVFormer architectures for consistent cross-paradigm alignment. GlobalAlign-v2 encompasses two variants: Deformable and Diffusion. The Deformable variant addresses global misalignment in LSS-based multi-modal BEV by explicitly learning cross-modal feature offsets. In contrast, the Diffusion variant targets implicit misalignment in query-based BEV by injecting noise to simulate misalignment and employing a denoising process to recover aligned features. Experimental results show that GraphBEV++ achieves state-of-the-art performance under misalignment noise on nuScenes and Waymo subset, improves long-range detection on Argoverse2, and generalizes effectively to the 3D occupancy prediction task, consistently improving occupancy estimation accuracy and robustness under both clean and noisy settings. Furthermore, GraphBEV++ effectively alleviates misalignment issues in end-to-end autonomous driving. Compared with five baselines (UniAD, VAD, FusionAD, MomAD, and WoTE), it demonstrates superior performance in both open-loop (nuScenes) and closed-loop (Bench2Drive and NAVSIM) evaluations across perception, prediction, and planning tasks.

Chinese Translation

在鸟瞰视图（BEV）感知中，特征不对齐是自动驾驶中的一个关键但常被忽视的挑战，尤其是在激光雷达（LiDAR）与摄像头传感器之间存在校准不确定性的情况下。为了解决这一问题，我们提出了一种稳健的多模态融合框架GraphBEV++，该框架系统性地减轻了投影引起的不对齐。该框架由两个关键模块组成：LocalAlign-v2和GlobalAlign-v2。LocalAlign-v2通过图匹配引入邻域感知的深度特征，以纠正局部不对齐。它支持基于LSS（局部空间结构）和基于查询的BEV表示，使其与BEVFusion和BEVFormer架构兼容，以实现一致的跨范式对齐。GlobalAlign-v2包括两个变体：可变形（Deformable）和扩散（Diffusion）。可变形变体通过显式学习跨模态特征偏移来解决基于LSS的多模态BEV中的全局不对齐。相反，扩散变体通过注入噪声来模拟不对齐，并采用去噪过程来恢复对齐特征，针对基于查询的BEV中的隐式不对齐。实验结果表明，GraphBEV++在nuScenes和Waymo子集上的不对齐噪声下实现了最先进的性能，改善了Argoverse2上的远程检测，并有效地推广到3D占用预测任务，在干净和嘈杂环境下始终提高了占用估计的准确性和鲁棒性。此外，GraphBEV++有效缓解了端到端自动驾驶中的不对齐问题。与五个基线（UniAD、VAD、FusionAD、MomAD和WoTE）相比，它在感知、预测和规划任务的开放环（nuScenes）和闭环（Bench2Drive和NAVSIM）评估中表现出优越的性能。

View on arXiv Download PDF AI Translation

cs.CV / 203 / 2606.16392

Towards UAV Image Dehazing: A UAV Atmospheric Scattering Model, Benchmark, and Geometry-Aware Deep Unfolding Network

朝向无人机图像去雾：无人机大气散射模型、基准测试与几何感知深度展开网络

Fang, Wenxuan, Weng, Jiangwei, Zheng, Yu, Fan, Junkai, Wang, Guangfa, Chen, Xiang, Yang, Jian, Li, Jun

Abstract

In UAV applications, haze significantly obscures distant details and weaken structural information, hindering the recovery of details. Current UAV scenarios still face two key challenges: (i) paired hazy/clean images from the real world are unobtainable, while the classical atmospheric scattering model is inadequate for modeling the spatially non-uniform haze in UAV imagery; (ii) existing dehazing methods struggle to remove the heavy haze accumulated in the upper regions of UAV images. To address these issues, we first propose a UAV Atmospheric Scattering Model (UASM), which explicitly incorporates flight altitude, viewing pitch, and extinction to characterize the non-uniform haze distribution in UAV imaging. Based on UASM, we develop a physics-driven dehazing framework, termed Geometry-aware Proximal Deep Unfolding Network (GP-DUN). Specifically, GP-DUN consists of three key modules: a Latent Geometry Estimator (LGE) that infers transmittance consistent with UAV imaging geometry, a Geometry-aware Gradient Descent Module (GeoGDM) that embeds UASM into the data-fidelity term and performs physics-consistent closed-form updates, and an Pooling-Expert Proximal Mapping Module (PE-PMM) that learns an implicit prior to restore textures and structures beyond the capability of explicit physical modeling. In addition, we further construct UASM-HazeSet, which provides controllable paired synthetic data together with 2,285 real UAV haze images for testing. Extensive experiments show that GP-DUN consistently outperforms existing methods on both UASM-HazeSet and real UAV haze benchmarks.

Chinese Translation

在无人机应用中，雾霭显著遮蔽了远处细节并削弱了结构信息，妨碍了细节的恢复。目前的无人机场景仍面临两个主要挑战：（i）来自现实世界的配对雾霭/清晰图像无法获得，而经典的大气散射模型不足以模拟无人机图像中空间上不均匀的雾霭；（ii）现有的去雾方法难以去除无人机图像上部区域积累的重雾。为了解决这些问题，我们首先提出了一种无人机大气散射模型（UAV Atmospheric Scattering Model, UASM），该模型明确考虑了飞行高度、视角和消光，以表征无人机成像中的不均匀雾霭分布。基于UASM，我们开发了一个以物理驱动的去雾框架，称为几何感知近端深度展开网络（Geometry-aware Proximal Deep Unfolding Network, GP-DUN）。具体而言，GP-DUN由三个关键模块组成：一个潜在几何估计器（Latent Geometry Estimator, LGE），用于推断与无人机成像几何一致的透射率；一个几何感知梯度下降模块（Geometry-aware Gradient Descent Module, GeoGDM），将UASM嵌入数据保真项并执行物理一致的闭式更新；以及一个池化专家近端映射模块（Pooling-Expert Proximal Mapping Module, PE-PMM），用于学习隐式先验，以恢复超出显式物理建模能力的纹理和结构。此外，我们进一步构建了UASM-HazeSet，提供可控的配对合成数据以及2,285张真实无人机雾霭图像用于测试。大量实验表明，GP-DUN在UASM-HazeSet和真实无人机雾霭基准测试中始终优于现有方法。

View on arXiv Download PDF AI Translation

cs.CV / 204 / 2606.16396

SP$^3$: Spherical Priors for Plug-and-Play Restoration

SP$^3$: 用于即插即用恢复的球形先验

Man, Sean, Raphaeli, Ron, Kleiner, Matan, Ronai, Or

Abstract

In this paper, we introduce SP$^3$, a novel Plug-and-Play algorithm that accelerates maximum a posteriori image restoration by replacing denoisers with Spherical Encoders (SE) as generative priors. SP$^3$ approximates the intractable proximal prior step by utilizing the SE tightly structured latent space as a robust projection onto the natural image manifold. Alternating this projection with a closed-form data-consistency step, via Half-Quadratic Splitting, achieves stable convergence without requiring gradient computation during inference. This unique formulation unlocks "anytime" restoration capabilities, producing sharp, plausible images from the first iteration. Evaluations across a variety of image restoration tasks demonstrate that SP$^3$ achieves perceptual quality comparable to state-of-the-art zero-shot diffusion and flow methods while being $3$-$630\times$ faster.

Chinese Translation

在本文中，我们介绍了SP$^3$，一种新颖的即插即用算法，通过将去噪器替换为球形编码器（Spherical Encoders, SE）作为生成先验，加速最大后验图像恢复。SP$^3$通过利用SE紧密结构的潜在空间作为自然图像流形的稳健投影，近似不可处理的近端先验步骤。通过半二次分裂（Half-Quadratic Splitting）将此投影与闭合形式的数据一致性步骤交替进行，实现了稳定的收敛，而无需在推理过程中进行梯度计算。这一独特的公式化解锁了“随时”恢复的能力，从第一次迭代开始就能生成清晰、可信的图像。在各种图像恢复任务中的评估表明，SP$^3$在感知质量上可与最先进的零-shot扩散和流方法相媲美，同时速度提高了$3$-$630 imes$。

View on arXiv Download PDF AI Translation

cs.CV / 205 / 2606.16401

RGFVR: Reference-Guided Face Video Restoration with Flow Matching

RGFVR：基于参考的面部视频恢复与流匹配

Eteke, Cem, Tosun, Batuhan, Steinbach, Eckehard

Abstract

Face video restoration from degraded observations is challenging, as it requires simultaneously recovering visual fidelity, temporal consistency, and subject identity. Existing approaches are often either reference-free, which can lead to identity loss when person-specific facial details are lost, or subject-specific, which limits generalization to unseen identities. We propose a subject-agnostic, reference-guided framework for identity-preserving face video restoration. Our method introduces bimodal perceptual-descriptive identity conditioning into a pretrained flow-based text-to-video generator and employs a two-stage training strategy to strengthen identity guidance during restoration. Experiments show that our approach improves restoration fidelity, temporal consistency, and identity preservation, achieving superior performance under challenging video degradations, including downsampling, blur, noise, and compression artifacts. The code is available under: https://github.com/batuhanntosun/RG-FVR.

Chinese Translation

从退化观测中恢复面部视频是一项具有挑战性的任务，因为它需要同时恢复视觉保真度、时间一致性和主体身份。现有的方法往往是无参考的，这可能导致在特定人脸细节丢失时身份丧失，或者是特定主体的，这限制了对未见身份的泛化。我们提出了一种无主体、基于参考的框架，用于保持身份的面部视频恢复。我们的方法将双模态感知-描述身份条件引入预训练的基于流的文本到视频生成器，并采用两阶段训练策略来加强恢复过程中的身份引导。实验表明，我们的方法在恢复保真度、时间一致性和身份保留方面均有所改善，在包括下采样、模糊、噪声和压缩伪影等挑战性视频退化情况下实现了优越的性能。代码可在以下链接获取：https://github.com/batuhanntosun/RG-FVR。

View on arXiv Download PDF AI Translation

cs.CV / 206 / 2606.16414

Instance-Aware Knowledge Distillation for Semi-Supervised Learning of an On-Board Multi-Task Dense Prediction Model for Collision Avoidance System

面向实例的知识蒸馏用于碰撞避免系统的半监督学习多任务密集预测模型

Hwang, Gyutae, Lee, Sang Jun

Abstract

Collision avoidance systems have evolved toward camera-based deep learning approaches for driving scene understanding. However, deployment in edge environments such as country clubs is constrained by limited computational resources and unreliable communication infrastructure. Moreover, constructing large-scale datasets for the target domain involves substantial annotation cost. To address these limitations, we propose an instance-aware knowledge distillation framework for semi-supervised learning. Specifically, we generate pseudo labels that mitigate teacher bias by leveraging domain priors from the teacher and instance-centric knowledge from foundation models. The trained lightweight student is deployed in the proposed collision avoidance system and performs multiple dense prediction tasks in real-time. The system detects frontal obstacles and encodes their spatial information into controller area network messages for automated guided vehicle operation. To achieve this, we construct a large-scale country club dataset and perform field validation of the proposed system. Experimental results demonstrate that the student outperforms the large teacher in instance segmentation while mitigating performance degradation in monocular depth estimation. Compared with the teacher, the student reduces FLOPs by 22.68$\times$ and parameters by 14.33$\times$, achieving 6.46 FPS on a low-cost edge device.

Chinese Translation

碰撞避免系统已逐渐向基于摄像头的深度学习方法发展，以实现驾驶场景理解。然而，在乡村俱乐部等边缘环境中的部署受到计算资源有限和通信基础设施不可靠的限制。此外，为目标领域构建大规模数据集涉及大量的标注成本。为了解决这些限制，我们提出了一种面向实例的知识蒸馏框架，用于半监督学习。具体而言，我们生成伪标签，通过利用教师模型的领域先验和基础模型的实例中心知识来减轻教师偏差。训练后的轻量级学生模型被部署在所提议的碰撞避免系统中，并实时执行多个密集预测任务。该系统能够检测前方障碍物，并将其空间信息编码为控制器局域网消息，以便于自动引导车辆的操作。为此，我们构建了一个大规模的乡村俱乐部数据集，并对所提议的系统进行了现场验证。实验结果表明，学生模型在实例分割任务中优于大型教师模型，同时减轻了单目深度估计中的性能下降。与教师模型相比，学生模型的FLOPs减少了22.68倍，参数减少了14.33倍，在低成本边缘设备上实现了6.46 FPS的性能。

View on arXiv Download PDF AI Translation

cs.CV / 207 / 2606.16421

Beer-Lambert Guided Representation Learning for Unsupervised Anomaly Detection in Sub-THz Food Inspection Images

基于比尔-朗伯特定律的引导表示学习用于亚太赫兹食品检测图像中的无监督异常检测

Hwang, Gyutae, Lee, Sang Jun

Abstract

Food manufacturing requires reliable inspection systems to detect foreign material contamination and maintain product safety. Sub-THz transmission imaging provides material-dependent attenuation characteristics that are useful for detecting low-density contaminants in food products. However, existing unsupervised anomaly detection methods mainly rely on RGB-pretrained visual representations, which may not adequately capture the transmission behavior of Sub-THz images. This paper proposes a Beer-Lambert guided representation learning framework for unsupervised anomaly detection in Sub-THz food inspection images. The proposed method introduces an attenuation decomposition module as an auxiliary regularization module that constrains student representations through attenuation reconstruction during training. In addition to the conventional one-class setting, we introduce a Leave-One-Food-Out protocol to evaluate generalization capability under unseen food categories. Experimental results on the Inline-Food-Inspection-THz dataset show that the proposed method improves overall anomaly detection performance over the baseline method.

Chinese Translation

食品制造需要可靠的检测系统以识别外来物质污染并维护产品安全。亚太赫兹传输成像提供了依赖于材料的衰减特性，这对于检测食品产品中的低密度污染物非常有用。然而，现有的无监督异常检测方法主要依赖于RGB预训练的视觉表示，这可能无法充分捕捉亚太赫兹图像的传输行为。本文提出了一种基于比尔-朗伯特定律的引导表示学习框架，用于亚太赫兹食品检测图像中的无监督异常检测。所提出的方法引入了一个衰减分解模块作为辅助正则化模块，通过衰减重建在训练过程中约束学生表示。除了传统的单类设置外，我们还引入了一种“留一食品法”（Leave-One-Food-Out）协议，以评估在未见食品类别下的泛化能力。在Inline-Food-Inspection-THz数据集上的实验结果表明，所提出的方法在整体异常检测性能上优于基线方法。

View on arXiv Download PDF AI Translation

cs.CV / 208 / 2606.16448

Hierarchical Fine-Grained Aerial Object Detection

层次化细粒度空中物体检测

Zhang, Yan, Xu, Fang, Yang, Wen, Xia, Gui-Song

Abstract

Fine-grained aerial object detection, driven by the intrinsic granularity of real-world object categories, is crucial for advanced scene understanding in remote sensing. Existing methods largely inherit the paradigm of coarse-grained object detection, relying solely on single-label supervision and thus struggling to distinguish model-level categories with subtle structural differences. However, for each specific model (e.g., Boeing 787), structured prior knowledge such as attributes and hierarchies offers discriminative semantics across multiple granularities. Motivated by this, we present ExpertDet, a scheme that incorporates expert-informed cues to enhance fine-grained aerial object detection. Specifically, we design Vision-aware Masked Attribute Modeling (VMAM), which aligns attribute semantics with visual structures by reconstructing randomly masked attributes from visual cues, enabling the detector to capture subtle structural distinctions. We further propose Hierarchical Visual Instance Promotion (HierVIP), which builds a visual prototype tree based on hierarchical relations and imposes taxonomy-aware constraints to preserve cross-level semantic continuity while enhancing category discrimination. Moreover, we curate a new fine-grained object detection benchmark for Precise recognition of model-specific Ships and Planes from aerial imagery, PSP, covering 106 ship classes and 30 airplane models, respectively, featuring the most extensive collection of model-specific categories among existing aerial object detection datasets to date. We benchmark state-of-the-art object detection algorithms on the PSP benchmark. Extensive evaluation demonstrates that ExpertDet consistently outperforms other fine-grained competitors across hierarchy levels. The dataset, benchmark, and code are available at https://nnnnerd.github.io/PSP-Benchmark/.

Chinese Translation

细粒度空中物体检测受到现实世界物体类别内在粒度的驱动，对于遥感中的高级场景理解至关重要。现有方法在很大程度上继承了粗粒度物体检测的范式，仅依赖单标签监督，因此在区分具有微妙结构差异的模型级类别时面临困难。然而，对于每个特定模型（例如，波音787），结构化的先验知识（如属性和层次）提供了跨多个粒度的区分语义。基于此动机，我们提出了ExpertDet，一个结合专家信息线索以增强细粒度空中物体检测的方案。具体而言，我们设计了视觉感知掩蔽属性建模（Vision-aware Masked Attribute Modeling, VMAM），通过从视觉线索中重建随机掩蔽的属性，将属性语义与视觉结构对齐，使检测器能够捕捉微妙的结构差异。我们进一步提出了层次化视觉实例提升（Hierarchical Visual Instance Promotion, HierVIP），该方法基于层次关系构建视觉原型树，并施加分类意识约束，以保持跨层次的语义连续性，同时增强类别区分。此外，我们为从空中图像中精确识别特定模型的船舶和飞机创建了一个新的细粒度物体检测基准（PSP），涵盖106个船舶类别和30个飞机模型，成为迄今为止现有空中物体检测数据集中模型特定类别最广泛的集合。我们在PSP基准上对最先进的物体检测算法进行了基准测试。广泛的评估表明，ExpertDet在各层次上始终优于其他细粒度竞争者。数据集、基准和代码可在 https://nnnnerd.github.io/PSP-Benchmark/ 获取。

View on arXiv Download PDF AI Translation

cs.CV / 209 / 2606.16449

PermaVid: Consistent Video Generation Across Edits via Disentangled Context Memory

PermaVid：通过解耦上下文记忆实现编辑过程中的一致视频生成

Yang, Shuai, Gao, Bingjie, Liu, Ziwei, Wang, Jiaqi, Lin, Dahua, Wu, Tong

Abstract

Consistent video generation under editing operations requires persistence: when edits modify scene appearance or layout, subsequent generations should remain coherent across time and viewpoints. However, existing memory designs struggle to maintain long-term consistency after such modifications, as stored contexts may become outdated or invalid. To address this, we propose PermaVid, a novel framework built upon a multi-modal context memory that disentangles spatial context into semantic appearance and geometric structure, together with an edit-aware memory update and retrieval strategy that keeps memory evolution aligned with subsequent observations. Specifically, we develop two complementary memory banks: an RGB context memory that captures appearance-aware observations while implicitly encoding geometry, and a depth context memory that preserves geometry-only structure disentangled from semantics. Building on this design, we introduce a memory-guided video generation model that performs multi-modal feature fusion under reference conditions drawn from mixed-modality memory contexts. Experiments demonstrate that our method maintains strong long-term semantic and structural consistency after edits, significantly outperforming state-of-the-art methods.

Chinese Translation

在编辑操作下实现一致的视频生成需要持久性：当编辑修改场景外观或布局时，后续生成应在时间和视角上保持一致。然而，现有的记忆设计在此类修改后难以维持长期一致性，因为存储的上下文可能变得过时或无效。为了解决这一问题，我们提出了PermaVid，一个基于多模态上下文记忆的新框架，该框架将空间上下文解耦为语义外观和几何结构，并结合了一种编辑感知的记忆更新和检索策略，使记忆演变与后续观察保持一致。具体而言，我们开发了两个互补的记忆库：一个RGB上下文记忆，捕捉外观感知的观察，同时隐式编码几何信息；另一个深度上下文记忆，保留与语义解耦的几何结构。基于这一设计，我们引入了一种记忆引导的视频生成模型，该模型在从混合模态记忆上下文中提取的参考条件下执行多模态特征融合。实验表明，我们的方法在编辑后能够维持强大的长期语义和结构一致性，显著优于现有的最先进方法。

View on arXiv Download PDF AI Translation

cs.CV / 210 / 2606.16457

ResEdit: Residual embeddings for precise generative image editing

ResEdit：用于精确生成图像编辑的残差嵌入

Baykal, Ahmet Canberk, Deschaintre, Valentin, Hold-Geoffroy, Yannick, Fischer, Michael, Frühstück, Anna, Öztireli, Cengiz, Georgiev, Iliyan

Abstract

Conditional diffusion image generators can be repurposed for editing through inversion, without the need for large-scale paired fine-tuning data. However, producing high-quality, targeted edits while maintaining image identity and global consistency remains challenging, as weakly conditioned inversion often embeds conflicting image features into the noise. We demonstrate that incorporating a residual image encoding as additional conditioning enables both improved identity preservation and better editability. We optimize this residual encoding to provide a strong conditioning signal for reconstruction, thereby reducing the reliance on inversion and susceptibility to its aforementioned pitfalls. To ensure this residual does not interfere with desired edits, we incorporate a gradient reversal-based optimization strategy that disentangles the residual from the edited condition. We illustrate our method's ability to produce high-fidelity results across precise intrinsic-based editing and relighting, and show proof-of-concept text-guided manipulation.

Chinese Translation

条件扩散图像生成器可以通过反演进行编辑，而无需大规模配对微调数据。然而，在保持图像身份和全局一致性的同时，生成高质量、针对性的编辑仍然具有挑战性，因为弱条件反演往往会将相互矛盾的图像特征嵌入噪声中。我们证明，结合残差图像编码作为额外条件可以提高身份保留和编辑能力。我们优化这种残差编码，以提供强有力的重建条件信号，从而减少对反演的依赖及其上述缺陷的敏感性。为了确保这种残差不会干扰所需的编辑，我们采用了一种基于梯度反转的优化策略，将残差与编辑条件解耦。我们展示了我们的方法在精确的基于内在的编辑和重光照方面产生高保真结果的能力，并展示了概念验证的文本引导操作。

View on arXiv Download PDF AI Translation

cs.CV / 211 / 2606.16470

Decoupled Object-Centric Video Understanding for Generating Robotic Manipulation Commands

解耦的面向对象视频理解用于生成机器人操作命令

Canh, Thanh Nguyen, Tran, Thanh-Tuan, Zhang, Haolan, Gao, Ziyan, HoangVan, Xiem, Chong, Nak Young

Abstract

Translating video demonstrations into executable robot commands remains challenging because existing methods often fail to identify which objects are functionally involved in the demonstrated action. As a result, they may generate commands that are linguistically plausible but operationally ambiguous. We propose an object-centric video understanding framework that decouples action recognition from object identification to generate precise, grammar-free manipulation commands. Our approach integrates Temporal Shift Modules (TSM) for efficient spatio-temporal action classification with a novel \textbf{Object Selection} algorithm that identifies task-relevant objects through trajectory-based role classification, blur detection, and overlap minimization. The selected objects are then processed by Vision-Language Models (VLMs) for robust category recognition and zero-shot generalization. Evaluated on a modified Something-Something V2 dataset, our method achieves 86.79\% action classification accuracy and BLEU-4 scores of 0.337 on standard objects and 0.261 on novel objects. These results improve over the strongest task-specific baseline by 80.2\% and 143.9\%, respectively. Larger gains are observed in METEOR and CIDEr, reaching 157.9\% and 171.7\% on novel objects. Across all semantic metrics, our approach consistently outperforms task-specific methods and remains competitive with, or surpasses, large general-purpose VLMs while retaining a modular, object-centric design.

Chinese Translation

将视频演示转换为可执行的机器人命令仍然具有挑战性，因为现有方法往往无法识别在演示动作中功能相关的对象。因此，它们可能生成在语言上看似合理但在操作上模糊的命令。我们提出了一种面向对象的视频理解框架，该框架将动作识别与对象识别解耦，以生成精确的、无语法的操作命令。我们的方法集成了时间移位模块（Temporal Shift Modules, TSM），用于高效的时空动作分类，并采用了一种新颖的对象选择（Object Selection）算法，通过基于轨迹的角色分类、模糊检测和重叠最小化来识别任务相关对象。所选对象随后由视觉-语言模型（Vision-Language Models, VLMs）处理，以实现稳健的类别识别和零样本泛化。在修改后的Something-Something V2数据集上进行评估，我们的方法在动作分类准确率上达到了86.79%，在标准对象上的BLEU-4得分为0.337，在新颖对象上的得分为0.261。这些结果分别比最强的任务特定基线提高了80.2%和143.9%。在METEOR和CIDEr指标上观察到更大的提升，分别在新颖对象上达到了157.9%和171.7%。在所有语义指标上，我们的方法始终优于任务特定方法，并在保持模块化、面向对象设计的同时，与大型通用VLMs竞争或超越。

View on arXiv Download PDF AI Translation

cs.CV / 212 / 2606.16474

MVOFormer: Flow-Semantic Transformer for Robust Monocular Visual Odometry

MVOFormer：用于鲁棒单目视觉里程计的流语义变换器

Li, Jituo, Sun, Shunwang, Zhang, Jialu, Liu, Xinqi, Hu, Jinyao, Lu, Zhicheng, Saeedi, Sajad, Lu, Guodong

Abstract

Monocular visual odometry (MVO) is foundational to autonomous navigation and robotic localization. However, existing learning-based MVO approaches often struggle with either a lack of interpretable, complementary features or overly complex multi-stage architectures. These limitations inherently restrict their robustness and cross-domain generalization. In this work, we propose MVOFormer, a novel transformer framework for robust monocular visual odometry. Our architecture features a Flow-Semantic Dual Branch Encoder that synergizes dense geometric motion cues with object-centric semantic priors, explicitly distinguishing static structures from dynamic distractors. These representations are then fused by an Iterative Multimodal Decoder, enabling coarse-to-fine pose refinement while dynamically suppressing attention on unreliable regions. Extensive evaluations demonstrate that, without any target-domain fine-tuning, MVOFormer achieves superior zero-shot generalization and robustness, significantly outperforming prior learning-based frame-to-frame methods across diverse benchmarks including TartanAir, KITTI, TUM-RGBD, and ETH3D-SLAM.

Chinese Translation

单目视觉里程计（MVO）是自主导航和机器人定位的基础。然而，现有的基于学习的MVO方法通常面临缺乏可解释的互补特征或过于复杂的多阶段架构的问题。这些局限性本质上限制了它们的鲁棒性和跨领域泛化能力。在本研究中，我们提出了MVOFormer，一种用于鲁棒单目视觉里程计的新型变换器框架。我们的架构具有流语义双分支编码器，能够将密集的几何运动线索与以对象为中心的语义先验相结合，明确区分静态结构与动态干扰物。这些表示随后通过迭代多模态解码器进行融合，使得在动态抑制不可靠区域的同时实现粗到细的姿态优化。广泛的评估表明，在没有任何目标领域微调的情况下，MVOFormer实现了优越的零样本泛化和鲁棒性，显著超越了以往基于学习的帧间方法，在包括TartanAir、KITTI、TUM-RGBD和ETH3D-SLAM等多种基准测试中表现优异。

View on arXiv Download PDF AI Translation

cs.CV / 213 / 2606.16477

AURA: Active-Response Attribution under Treatment Ambiguity in Bacterial Cytological Profiling

AURA：在细菌细胞学分析中处理模糊性下的主动响应归因

Jhawar, Kartik, Deshpande, Mrunmayee, Moreira, Wilfried, Bazan, Guillermo C., Wang, Lipo

Abstract

When a bacterial sample is exposed to several antibiotics, not every applied drug necessarily acts: if the organism is resistant to one of them, that drug leaves no morphological trace. The clinically meaningful quantity is therefore not which antibiotics were applied, but which ones were active. We show that these two are sharply decoupled in real E. coli microscopy - naively assuming the applied combination equals the active one is correct only about 37% of the time - yet existing computational tools are ill-suited to recovering the active set. Forward perturbation models such as scGen, CPA, and IMPA are designed to predict appearance from treatment, not the reverse, and inverting them degrades sharply; discriminative image classifiers tend to memorise strain- and batch-specific texture and fail to transfer across experimental replicates. We introduce AURA, which reframes the task as constrained, energy-based inverse attribution. Its central inductive bias is that the active set must be a subset of the applied set; this collapses the candidate space and lets AURA infer the active subset of applied antibiotics by decomposing residual morphology into antibiotic response atoms and selecting the subset with the lowest reconstruction energy, using no strain label at test time. AURA-E adds evidence-aware abstention, withholding a prediction when candidate explanations remain near-equally plausible. On cross-replicate transfer in an E. coli cytological profiling dataset, AURA recovers the active antibiotic combination with 95.47% exact-match accuracy.

Chinese Translation

当细菌样本暴露于多种抗生素时，并非每种施用的药物都必然起作用：如果微生物对其中一种药物具有抗性，则该药物不会留下形态学痕迹。因此，临床上有意义的量不是施用了哪些抗生素，而是哪些抗生素是有效的。我们展示了在真实的大肠杆菌显微镜观察中，这两者是明显解耦的——天真的假设施用的组合等于有效的组合，仅在约37%的情况下是正确的——然而现有的计算工具并不适合恢复有效的药物组合。前向扰动模型如scGen、CPA和IMPA旨在预测治疗后的外观，而不是反向预测，反转这些模型会显著降级；判别图像分类器往往会记忆菌株和批次特定的纹理，无法在实验重复中进行迁移。我们引入了AURA，将任务重新框定为受限的基于能量的逆归因。其核心归纳偏差是有效药物组合必须是施用药物组合的子集；这缩小了候选空间，使AURA能够通过将残余形态分解为抗生素响应原子，并选择重构能量最低的子集，从而推断施用抗生素的有效子集，在测试时不使用菌株标签。AURA-E增加了证据感知的弃权，当候选解释保持近乎同样可信时，暂停预测。在大肠杆菌细胞学分析数据集中进行跨重复实验转移时，AURA以95.47%的准确率恢复了有效的抗生素组合。

View on arXiv Download PDF AI Translation

cs.CV / 214 / 2606.16479

Uncertainty Quality of VGGT: An Analysis on the DTU Benchmark Dataset

VGGT的不确定性质量：对DTU基准数据集的分析

Hillemann, Markus, Langendörfer, Robert, Landgraf, Steven, Ulrich, Markus

Abstract

Visual Geometry Grounded Transformer (VGGT) has already attracted a great deal of attention in a short period of time, not least due to the Best Paper Award at CVPR-2025. Similar to DUSt3R and MASt3R, VGGT aims to bring about a paradigm shift by replacing established methods like bundle adjustment and feature matching with a simple, unified, feed-forward neural network that predicts camera poses, depth maps, and dense 3D structure directly from multiple images of a scene in a few seconds. A key aspect is its ability to process an arbitrary number of views consistently in a single forward pass without any post-processing or iterative optimization. For photogrammetry, this opens new possibilities for real-time, scalable, and accessible 3D reconstruction. In this context, not only high reconstruction accuracy but also high-quality uncertainty estimates are crucial, as they foster trust and enable robust quality assurance. This paper therefore investigates the quality of VGGT's uncertainty predictions. The analysis identifies an effective confidence threshold for filtering VGGT's raw output and demonstrates that enhancing uncertainty quality holds strong potential for improving the accuracy of its 3D reconstructions.

Chinese Translation

视觉几何基础变换器（Visual Geometry Grounded Transformer, VGGT）在短时间内已经引起了广泛的关注，尤其是由于其在CVPR-2025上获得了最佳论文奖。与DUSt3R和MASt3R类似，VGGT旨在通过用一个简单的统一前馈神经网络替代传统的束调整和特征匹配等方法，从多个场景图像中直接预测相机姿态、深度图和稠密3D结构，从而带来范式的转变。其一个关键特点是能够在单次前向传递中一致地处理任意数量的视图，而无需任何后处理或迭代优化。这为摄影测量学开辟了实时、可扩展和易于访问的3D重建的新可能性。在这一背景下，不仅高重建精度至关重要，高质量的不确定性估计同样重要，因为它们能够增强信任并实现稳健的质量保证。因此，本文研究了VGGT不确定性预测的质量。分析确定了一个有效的置信度阈值，用于过滤VGGT的原始输出，并展示了提高不确定性质量在改善其3D重建精度方面的强大潜力。

View on arXiv Download PDF AI Translation

cs.CV / 215 / 2606.16484

Unified Multimodal Model for Brain MRI Imputation and Understanding

统一的多模态模型用于脑部MRI的插补与理解

Song, Zhiyun, Liu, Che, Xia, Tian, Kori, Avinash, Bai, Wenjia

Abstract

Multimodal large language models (MLLMs) hold great potential for medicine, as they inherit knowledge from LLM and allow multiple data modalities to be integrated, analysed and interpreted in natural language. However, the field of medical MLLMs is constrained by non-trivial challenges, notably the scarcity of high-quality training data and the frequent occurrence of missing data in the real-world clinical setting. Here, we propose a novel unified multimodal model, UniBrain, for brain magnetic resonance image (MRI) analysis. To address potential missing brain MRI modalities, we employ a unified training strategy to perform joint imaging modality imputation and brain image understanding. During training, an interleaved and description-enriched data flow is constructed to train the model in an autoregressive manner, enabling medical reasoning with generated multimodal data. A self-alignment strategy is introduced to leverage dense image embeddings to learn fine-grained anatomical features without requiring detailed image captions. Furthermore, we propose a dynamic hidden state mechanism to alleviate the exposure bias during long-context multimodal inference. Extensive experiments on multi-disease brain MRI dataset demonstrate that UniBrain achieves high performance for brain image imputation, understanding, and disease diagnosis under various extents of modality incompleteness.

Chinese Translation

多模态大型语言模型（MLLMs）在医学领域具有巨大的潜力，因为它们继承了大型语言模型（LLM）的知识，并允许多种数据模态的整合、分析和自然语言解释。然而，医学MLLM领域面临着非平凡的挑战，尤其是高质量训练数据的稀缺以及在实际临床环境中缺失数据的频繁出现。在此，我们提出了一种新颖的统一多模态模型UniBrain，用于脑部磁共振成像（MRI）分析。为了解决潜在的缺失脑部MRI模态，我们采用统一的训练策略来执行联合成像模态插补和脑部图像理解。在训练过程中，构建了一种交错且描述丰富的数据流，以自回归的方式训练模型，使其能够利用生成的多模态数据进行医学推理。我们引入了一种自对齐策略，以利用密集的图像嵌入学习细粒度的解剖特征，而无需详细的图像说明。此外，我们提出了一种动态隐藏状态机制，以减轻长上下文多模态推理中的暴露偏差。在多疾病脑部MRI数据集上的广泛实验表明，UniBrain在不同程度的模态不完整性下，能够实现高效的脑部图像插补、理解和疾病诊断。

View on arXiv Download PDF AI Translation

cs.CV / 216 / 2606.16502

Active Reference Acquisition in Few-Shot Font Generation

少样本字体生成中的主动参考获取

Matsuo, Shinnosuke

Abstract

Few-shot font generation aims to synthesize the remaining glyphs of a font given one or a few reference glyphs while preserving stylistic consistency, thereby supporting font designers in efficiently completing a typeface. Existing methods primarily focus on improving generation quality given a fixed reference set. However, when the current reference glyphs are insufficient to represent the target style, few-shot font generation may fail to produce satisfactory results. In practical scenarios, additional reference glyphs can often be obtained from the designer when necessary. Accordingly, we propose a new framework, Active Reference Acquisition in Few-Shot Font Generation, in which the model sequentially decides which character to acquire next as an additional reference. Furthermore, we propose a reference part-coverage-based acquisition function to efficiently query the designer. Motivated by the observation that font styles are well characterized by local structural parts, we represent each glyph using a histogram of local features and select query characters that maximize the expected part coverage of the reference set. By prioritizing characters that contain parts not yet covered by the current references, the proposed method progressively expands the diversity of visual parts in the reference set. As a result, generation quality is improved with fewer queries. Experiments on the Google Fonts dataset demonstrate that the proposed method achieves higher generation quality than random querying and reference-agnostic baselines. The code is available at https://github.com/matsuo-shinnosuke/ActiveRef-FontGen.

Chinese Translation

少样本字体生成旨在根据一个或几个参考字形合成字体的其余字形，同时保持风格一致性，从而支持字体设计师高效完成字体设计。现有方法主要集中在提高给定固定参考集的生成质量。然而，当当前参考字形不足以代表目标风格时，少样本字体生成可能无法产生令人满意的结果。在实际场景中，设计师在必要时通常可以获得额外的参考字形。因此，我们提出了一种新的框架，即少样本字体生成中的主动参考获取，其中模型顺序决定下一个要获取作为额外参考的字符。此外，我们提出了一种基于参考部分覆盖的获取函数，以高效地查询设计师。受到字体风格通过局部结构部分良好表征的观察启发，我们使用局部特征的直方图表示每个字形，并选择最大化参考集期望部分覆盖的查询字符。通过优先选择包含当前参考尚未覆盖部分的字符，所提出的方法逐步扩展了参考集中视觉部分的多样性。因此，生成质量在较少查询的情况下得到了改善。在Google Fonts数据集上的实验表明，所提出的方法在生成质量上优于随机查询和无参考基线。代码可在https://github.com/matsuo-shinnosuke/ActiveRef-FontGen获取。

View on arXiv Download PDF AI Translation

cs.CV / 217 / 2606.16519

BadWorld: Adversarial Attacks on World Models

BadWorld：针对世界模型的对抗攻击

Shen, Linghui, Cui, Mingyue, Yang, Xingyi

Abstract

Visual world models (VWMs) synthesize interactive, action-conditioned rollouts from a single context image. However, it remains an open question how robust these models are to adversarial perturbations. Standard adversarial attacks fail to assess this vulnerability because attackers lack ground-truth future videos and cannot predict subsequent user controls. We introduce BadWorld, a label-free adversarial framework tailored for autoregressive VWMs that systematically overcomes both constraints. First, to bypass the need for future supervision, we propose a self-supervised velocity attack that directly disrupts the early denoising dynamics of the model. Second, to ensure the attack generalizes across unpredictable user actions, we formulate a trajectory-adaptive bi-level optimization that actively mines hard control sequences to forge control-agnostic perturbations. Evaluated on representative VWMs with continuous and discrete controls, BadWorld exposes severe structural fragility. Visually indistinguishable adversarial images reliably trigger catastrophic degradation in future rollouts, leading to incomplete denoising, structural collapse, and control inconsistency. These findings reveal critical risks for deploying VWMs in safety-critical systems while highlighting a practical mechanism for privacy protection.

Chinese Translation

视觉世界模型（VWMs）从单一上下文图像合成交互式、基于动作的展开。然而，这些模型对对抗扰动的鲁棒性仍然是一个未解的问题。标准的对抗攻击无法评估这种脆弱性，因为攻击者缺乏真实的未来视频，并且无法预测后续的用户控制。我们提出了BadWorld，一个无标签的对抗框架，专为自回归VWMs量身定制，系统性地克服了这两个限制。首先，为了绕过对未来监督的需求，我们提出了一种自监督速度攻击，直接干扰模型的早期去噪动态。其次，为了确保攻击在不可预测的用户行为中具有普适性，我们制定了一种轨迹自适应的双层优化，主动挖掘困难的控制序列，以形成与控制无关的扰动。在具有连续和离散控制的代表性VWMs上进行评估，BadWorld揭示了严重的结构脆弱性。视觉上难以区分的对抗图像可靠地触发未来展开中的灾难性退化，导致去噪不完全、结构崩溃和控制不一致。这些发现揭示了在安全关键系统中部署VWMs的关键风险，同时突显了隐私保护的实用机制。

View on arXiv Download PDF AI Translation

cs.CV / 218 / 2606.16566

Local-GS: Accelerating 3D Gaussian Splatting via Tile-Local Warp Coherence

Local-GS：通过块局部变换一致性加速3D高斯点云渲染

Luo, Yang, Gong, Yan, Gao, Yongsheng, Zhao, Jie, Zhang, Xinyu, Liu, Huaping

Abstract

3D Gaussian Splatting (3DGS) has significantly advanced real-time novel view synthesis by representing scenes as dense collections of anisotropic 3D Gaussian primitives. However, the irregular spatial distribution of Gaussians often leads to poor GPU utilization, as warp divergence and redundant computation degrade rendering performance. To address this, we present Local-GS, a warp-coherent rendering paradigm that, organizes Gaussian primitives with respect to SIMT (Single Instruction, Multiple Threads) execution boundaries rather than scene geometry. Specifically, we propose three warp-coherent stages: a hoisting stage that precomputes shared parameters at tile level, a culling stage that discards warps with no contribution, and a blending stage that replaces per-pixel branching with a uniform instruction stream. Across extensive benchmarks on multiple datasets, Local-GS improves efficiency without compromising quality. As a plug-and-play optimization, it provides additional performance gains to all tested baselines, culminating in a $7.76\times$ speedup on Deep Blending scenes.

Chinese Translation

3D高斯点云渲染（3DGS）通过将场景表示为密集的各向异性3D高斯原语集合，显著推动了实时新视角合成的发展。然而，高斯的非规则空间分布常常导致GPU利用率低下，因为变换发散和冗余计算降低了渲染性能。为了解决这个问题，我们提出了Local-GS，这是一种变换一致的渲染范式，它根据SIMT（单指令多线程）执行边界而非场景几何来组织高斯原语。具体而言，我们提出了三个变换一致的阶段：一个在块级别预计算共享参数的提升阶段，一个丢弃无贡献变换的剔除阶段，以及一个用统一指令流替代逐像素分支的混合阶段。在多个数据集上的广泛基准测试中，Local-GS在不妥协质量的情况下提高了效率。作为一种即插即用的优化，它为所有测试的基线提供了额外的性能提升，在Deep Blending场景中达到了$7.76 imes$的加速。

View on arXiv Download PDF AI Translation

cs.CV / 219 / 2606.16569

PROSE: Training-Free Egocentric Scene Registration with Vision-Language Models

PROSE：基于视觉-语言模型的无训练自我中心场景配准

Chen, Zhiang, Lee, Nahyuk, Sun, Boyang, Kwon, Taein, Pollefeys, Marc, Bauer, Zuria, Hong, Sunghwan

Abstract

Registering two captures of the same indoor space taken at different times underpins persistent spatial memory for robots and AR systems, yet the realistic version of this task is egocentric and its most scalable form is RGB-only. Head-mounted cameras yield blurry, fast-moving, partially overlapping views from which dense geometry is hard to recover. Classical registration leans on exactly the clean point clouds this setting lacks, while learned scene-graph methods require a pre-built or annotated graph and a trained matcher that we find brittle under egocentric data. We take a different route, using a pretrained vision-language model as the source of both scene understanding and cross-scan matching. Our method, PROSE (Prompted Scene rEgistration), lifts each RGB sequence into an object-level 3D scene graph using off-the-shelf foundation models for geometry, segmentation, and language, then prompts the same VLM to match object instances across the two RGB sequences. To make this matching tractable and reliable, we leverage object heights as a prior and verify each proposed match with a paired same/different query, then solve for the rigid transform by hypothesizing a candidate per matched object and selecting the one with the strongest geometric consensus. PROSE adds no learned parameters and requires no depth sensor, training, or annotated graph. On the egocentric Aria Digital Twin and Aria Everyday Activities benchmarks, it outperforms both geometric and learned scene-graph baselines in registration accuracy, on ground-truth and RGB-reconstructed point clouds alike, and the scene graph it produces transfers directly to downstream tasks.

Chinese Translation

在不同时间拍摄的同一室内空间的两次捕捉之间进行配准是机器人和增强现实系统持久空间记忆的基础，然而这一任务的现实版本是自我中心的，其最具可扩展性的形式仅为RGB图像。头戴式摄像头提供模糊、快速移动和部分重叠的视图，这使得密集几何的恢复变得困难。经典的配准方法依赖于这种设置所缺乏的干净点云，而学习的场景图方法则需要预构建或注释的图以及训练好的匹配器，我们发现这些在自我中心数据下表现脆弱。我们采取了不同的路径，使用预训练的视觉-语言模型作为场景理解和跨扫描匹配的来源。我们的方法PROSE（Prompted Scene rEgistration）将每个RGB序列提升为一个对象级的3D场景图，利用现成的基础模型进行几何、分割和语言处理，然后提示相同的视觉-语言模型在两个RGB序列之间匹配对象实例。为了使这种匹配可行且可靠，我们利用对象高度作为先验，并通过配对的同/不同查询验证每个提议的匹配，然后通过假设每个匹配对象的候选者并选择具有最强几何一致性的一个来求解刚性变换。PROSE不添加任何学习参数，也不需要深度传感器、训练或注释图。在自我中心的Aria数字双胞胎和Aria日常活动基准测试中，它在配准精度上优于几何和学习的场景图基线，无论是在真实数据还是RGB重建的点云上，其生成的场景图可以直接转移到下游任务中。

View on arXiv Download PDF AI Translation

cs.CV / 220 / 2606.16573

Transformation-driven generation of comparable projection images from multimodal anatomical scenes

基于变换驱动的多模态解剖场景可比投影图像生成

Pojda, Dariusz, Domino, Krzysztof, Tarnawski, Michał, Tomaka, Agnieszka Anna

Abstract

This work addresses the computational problem of generating reproducible projection-space observations from heterogeneous anatomical scenes whose components may undergo independent spatial transformations. We propose a transformation-driven framework for synthetic projection imaging from multimodal anatomical data and demonstrate it on mandibular-motion scenarios. In contrast to conventional Digitally Reconstructed Radiograph (DRR) approaches primarily designed for registration, projection realism, or rendering efficiency, the proposed formulation treats projection imaging as an observation process operating on an explicitly represented anatomical scene. Independently transformable volumetric and surface-based anatomical objects are embedded within a shared scene representation and propagated directly into projection space through explicit transformations. Projection geometry, acquisition modelling, material interpretation, and image presentation remain explicitly separated, enabling controlled exploration of methodological assumptions while preserving reproducibility and direct comparability between generated projections. Particular emphasis is placed on transformation-driven anatomical scenarios relevant to craniofacial analysis, including mandibular motion and therapeutic repositioning. Using a shared anatomical reference scene composed of CT/CBCT volumes, segmented structures, surface models, and auxiliary anatomical or therapeutic objects, the framework enables generation of directly comparable VirtualRTG projections from multiple anatomical configurations while preserving identical imaging assumptions. Rather than aiming at fully physically faithful radiographic simulation, the proposed approach provides a controllable and reproducible methodological environment for studying anatomy--projection relationships, motion observability, and transformation-aware imaging workflows.

Chinese Translation

本研究解决了从异构解剖场景生成可重复的投影空间观测的计算问题，这些场景的组成部分可能经历独立的空间变换。我们提出了一种基于变换驱动的框架，用于从多模态解剖数据生成合成投影成像，并在下颌运动场景中进行了验证。与主要为配准、投影真实感或渲染效率设计的传统数字重建放射图（DRR）方法相比，所提出的公式将投影成像视为在明确表示的解剖场景上进行的观测过程。独立可变换的体积和基于表面的解剖对象嵌入在共享的场景表示中，并通过显式变换直接传播到投影空间。投影几何、获取建模、材料解释和图像呈现保持明确分离，使得在保持可重复性和生成投影之间的直接可比性的同时，能够控制方法假设的探索。特别强调与颅面分析相关的基于变换的解剖场景，包括下颌运动和治疗性重新定位。利用由CT/CBCT体积、分割结构、表面模型和辅助解剖或治疗对象组成的共享解剖参考场景，该框架能够在保持相同成像假设的情况下，从多个解剖配置生成直接可比的虚拟放射图（VirtualRTG）投影。所提出的方法并不旨在实现完全物理真实的放射模拟，而是提供了一个可控和可重复的方法环境，用于研究解剖-投影关系、运动可观测性和变换感知成像工作流程。

View on arXiv Download PDF AI Translation

cs.CV / 221 / 2606.16586

LOCUS: Local Visual Cue Search for Enhancing Fine-Grained Perception in Multimodal Large Language Models

LOCUS：增强多模态大型语言模型细粒度感知的局部视觉线索搜索

Tao, Zhou, Zhang, Fang, Ding, Zewen, Wang, Shida, Sun, Xiaokun, Hua, YongXiang, Cao, Haoyu, Xu, Linli

Abstract

Multimodal Large Language Models (MLLMs) remain unreliable on fine-grained visual perception, even when high-resolution inputs preserve the necessary local details. We identify this limitation as visual context rot: decisive evidence may exist in the full image, yet fail to be reliably selected and used amid redundant visual context. We propose LOCUS (LOcal visual CUe Search), a training framework that teaches MLLMs to internalize local evidence search through a verifiable proxy task. During training, LOCUS provides a local crop as a visual cue and optimizes the model to recover its spatial support in the full image using an IoU-based reward. The visual cue is used only during training, leaving the standard image-question inference interface unchanged. Experiments across fine-grained perception, hallucination, general understanding, and reasoning benchmarks show that LOCUS improves localization-sensitive visual understanding while preserving broad capabilities. Attention analyses further indicate stronger focus on task-relevant evidence regions, suggesting that training-time visual cue search provides an effective route to internalized fine-grained evidence selection.

Chinese Translation

多模态大型语言模型（MLLMs）在细粒度视觉感知方面仍然不可靠，即使高分辨率输入保留了必要的局部细节。我们将这一局限性称为视觉上下文衰退：决定性证据可能存在于完整图像中，但在冗余视觉上下文中却未能被可靠地选择和使用。我们提出了LOCUS（局部视觉线索搜索），这是一个训练框架，旨在通过可验证的代理任务教会MLLMs内化局部证据搜索。在训练过程中，LOCUS提供一个局部裁剪作为视觉线索，并优化模型以使用基于IoU的奖励在完整图像中恢复其空间支持。视觉线索仅在训练期间使用，保持标准的图像-问题推理接口不变。在细粒度感知、幻觉、一般理解和推理基准上的实验表明，LOCUS在提高定位敏感的视觉理解的同时保持了广泛的能力。注意力分析进一步表明对任务相关证据区域的更强关注，表明训练期间的视觉线索搜索为内化细粒度证据选择提供了一条有效路径。

View on arXiv Download PDF AI Translation

cs.CV / 222 / 2606.16593

Rotational Symmetry based Object Pose Estimation from Point Clouds in the Absence of Known 3D Models

基于旋转对称性的点云物体姿态估计方法：在缺乏已知三维模型的情况下

Dai, Weichen, Yu, Ruixun, Tang, Yangjie, Du, Yifan, Zhang, Yiyang, Sun, Donglei, Zhang, Hua

Abstract

Object pose estimation is crucial to many industrial applications, with one example being automated spray painting using a robot. However, confidentiality concerns often limit access to high-quality 3D models, posing a significant challenge for point-cloud-based pose estimation. In such scenarios, rotational symmetry, a readily accessible characteristic of many industrial objects, can provide valuable prior information to facilitate pose estimation.In this paper, we propose a method that leverages the rotational symmetry commonly found in industrial objects to address the challenge caused by the absence of 3D models. The object pose is jointly estimated with point cloud refinement through an iterative optimization process. This optimization relies on a rotational symmetry constraint loss. To construct this loss, each 3D point is rotated according to the currently estimated pose, and multiple correspondences are identified using nearest-neighbor search by exploiting the rotational symmetry property. These correspondences are then used to compute the rotational symmetry constraint loss, which iteratively refines both the pose and the point cloud.By explicitly incorporating rotational symmetry into the optimization process, the proposed method achieves robust pose estimation and generalizes well across diverse object types. The proposed method is evaluated on a dataset specifically created for point clouds without known 3D models, consisting of four categories of synthetic objects and one real wheel hub collected from a production line. Experimental results demonstrate that the proposed method achieves performance comparable to methods that rely on known 3D models.

Chinese Translation

物体姿态估计在许多工业应用中至关重要，例如使用机器人进行自动喷涂。然而，保密问题常常限制对高质量三维模型的访问，这给基于点云的姿态估计带来了重大挑战。在这种情况下，旋转对称性作为许多工业物体的一个易于获取的特征，可以提供有价值的先验信息以促进姿态估计。本文提出了一种利用工业物体中常见的旋转对称性来应对缺乏三维模型所带来的挑战的方法。该方法通过迭代优化过程联合估计物体姿态和点云精细化。此优化依赖于旋转对称性约束损失。为了构建该损失，每个三维点根据当前估计的姿态进行旋转，并利用旋转对称性特性通过最近邻搜索识别多个对应关系。这些对应关系随后用于计算旋转对称性约束损失，从而迭代精细化姿态和点云。通过将旋转对称性明确纳入优化过程，所提出的方法实现了稳健的姿态估计，并在不同物体类型之间具有良好的泛化能力。该方法在一个专门为缺乏已知三维模型的点云创建的数据集上进行了评估，该数据集包含四类合成物体和一个从生产线收集的真实轮毂。实验结果表明，所提出的方法在性能上与依赖已知三维模型的方法相当。

View on arXiv Download PDF AI Translation

cs.CV / 223 / 2606.16601

DifferAD-R1: A Difference-Guided IndustrialAnomaly Localization with Multimodal LargeLanguage Models

DifferAD-R1：一种基于差异指导的多模态大语言模型的工业异常定位方法

Wang, Dingrong, Tao, Xian, Qu, Zhen, Luo, Hengliang, Gong, Xinyi, Shen, Fei, Zhang, Zhengtao, Ding, Guiguang

Abstract

Industrial anomaly localization aims to accurately identify and localize abnormal regions in industrial products, addressing the critical challenge of detecting unseen defect categories in real-world scenarios. Traditional closed-set methods often suffer from poor cross-scenario generalization, while existingMultimodal Large Language Model (MLLM)-based approachesface two core limitations: they either adopt QA-style paradigmsmisaligned with the practical demands of localization, or relyon standard optimization techniques such as Group RelativePolicy Optimization (GRPO), which fails to deliver effectivelearning signals for subtle defects. To tackle these issues, thispaper proposes DifferAD-R1, an MLLM-augmented reinforcement learning framework tailored for industrial anomaly localization. We design a Difference-Guided dual-image paradigm,which reformulates the localization task as a one-shot difference grounding problem to effectively explore cross-scenarioanomalies. A Dual-Consistency Localization Reward is developedfor hard-to-detect anomalies, enhancing optimization stabilityand robustness. Additionally, we integrate a difficulty-awarestrategy with adaptive reweighting and group-wise resamplingto prioritize learning on challenging instances. To facilitateevaluations in real-world industrial settings, we construct theAD-DualDiff dataset, comprising 13K paired images across 20categories. Experimental results demonstrate that DifferADR1 significantly outperforms existing baselines and achievescompetitive performance compared to large-scale models likeQwen3-VL (235B parameters). Our code is publicly availableat: https://github.com/Rong2026/work-1.

Chinese Translation

工业异常定位旨在准确识别和定位工业产品中的异常区域，解决在现实场景中检测未见缺陷类别的关键挑战。传统的闭集方法往往在跨场景泛化能力上表现不佳，而现有的基于多模态大语言模型（MLLM）的方法面临两个核心限制：要么采用与定位实际需求不符的问答（QA）风格范式，要么依赖于标准优化技术，如组相对策略优化（GRPO），这无法为微小缺陷提供有效的学习信号。为了解决这些问题，本文提出了DifferAD-R1，一种增强了MLLM的强化学习框架，专门用于工业异常定位。我们设计了一种差异指导的双图像范式，将定位任务重新表述为一次性差异定位问题，以有效探索跨场景异常。我们为难以检测的异常开发了一种双一致性定位奖励，以增强优化的稳定性和鲁棒性。此外，我们整合了一种难度感知策略，采用自适应重加权和组内重采样，优先学习具有挑战性的实例。为了便于在真实工业环境中进行评估，我们构建了AD-DualDiff数据集，包含20个类别的13K对图像。实验结果表明，DifferAD-R1显著优于现有基线，并与像Qwen3-VL（235B参数）这样的规模模型相比，表现出竞争力。我们的代码已公开发布于：https://github.com/Rong2026/work-1。

View on arXiv Download PDF AI Translation

cs.CV / 224 / 2606.16615

SUP-MCRL: Subject-aware Unified Pseudo-feature Coded Multimodal Contrastive Representation Learning for EEG Visual Decoding

SUP-MCRL：基于主体感知的统一伪特征编码多模态对比表示学习用于脑电图视觉解码

Gong, Shengyu, Zeng, Weiming, Li, Yueyang, Kang, Zijian, Yan, Hongjie, Siok, Wai Ting, Wang, Nizhuan

Abstract

Non-invasive brain-computer interfaces suffer severe fidelity degradation in neural visual decoding when generalizing to natural visual experiences. Conventional multimodal contrastive representation learning solely optimizes geometric distance alignment, neglecting semantic consistency and subject selectivity, causing spurious zero-shot alignment. We propose SUP-MCRL, a unified framework integrating three collaborative mechanisms: (1) Semantic-entity Aware Visual Encoder (SAVE), learning spatial attention to extract semantic content without pre-trained saliency models; (2 Unified EEG Enhancer (UEE), employing multi-scale atrous convolutions and inter-band attention for adaptive cross-subject robustness; and (3) Prototype-based Progressive Augmenter (PPA), maintaining an EMA-updated pseudo-feature pool to prevent representation collapse. Zero-shot experiments on THINGS-EEG achieve 66.0%/91.9% (Top-1/Top-5) intra-subject and 24.0%/52.9% LOSO accuracy, surpassing state-of-the-art methods. Code is available at https://github.com/NZWANG/SUP-MCRL.

Chinese Translation

非侵入式脑-计算机接口在向自然视觉体验推广时，神经视觉解码的保真度严重下降。传统的多模态对比表示学习仅优化几何距离对齐，忽视了语义一致性和主体选择性，导致虚假的零样本对齐。我们提出了SUP-MCRL，一个统一框架，整合了三种协作机制：（1）语义实体感知视觉编码器（SAVE），学习空间注意力以提取语义内容，而无需预训练的显著性模型；（2）统一脑电图增强器（UEE），采用多尺度空洞卷积和跨频带注意力以实现自适应的跨主体鲁棒性；（3）基于原型的渐进增强器（PPA），维护一个EMA更新的伪特征池以防止表示崩溃。在THINGS-EEG上的零样本实验达到了66.0%/91.9%（Top-1/Top-5）主体内准确率和24.0%/52.9%的LOSO准确率，超越了现有的最先进方法。代码可在https://github.com/NZWANG/SUP-MCRL获取。

View on arXiv Download PDF AI Translation

cs.CV / 225 / 2606.16633

DCP-Prune: Ultra-Low Token Pruning with Distribution Consistency Preservation

DCP-Prune：具有分布一致性保持的超低令牌剪枝

Xue, Xifeng, Wang, Xiaokang, Li, Zirui, Cheng, Ming-Ming, Sun, Guolei

Abstract

Recent vision token pruning methods effectively preserve model performance under moderate token budgets but become unstable under ultra-low token budget. Our analysis shows that as the pruning budget decreases, accuracy degradation is often accompanied by larger feature distribution shifts. Critically, the degree of this distribution shift strongly correlates with performance degradation. To better characterize this phenomenon, we introduce a lightweight distribution consistency metric to estimate the distribution shift between retained and full tokens. Motivated by these observations, we propose a two-stage pruning framework consisting of Anchor-Context Graph Recovery (ACGR) and Text-Aware Token Cluster Selection (TATCS). Specifically, ACGR transfers contextual information before token removal, while TATCS dynamically re-selects representative tokens when severe distribution shift is detected. Extensive experiments demonstrate that our method achieves superior and more stable performance under ultra-low token budget. Notably, it retains 92.1% of the upper-bound average performance on LLaVA-1.5-7B with only 16 visual tokens.

Chinese Translation

最近的视觉令牌剪枝方法在适度的令牌预算下有效地保持了模型性能，但在超低令牌预算下变得不稳定。我们的分析表明，随着剪枝预算的减少，准确率下降通常伴随着更大的特征分布变化。关键是，这种分布变化的程度与性能下降有很强的相关性。为了更好地表征这一现象，我们引入了一种轻量级的分布一致性度量，用于估计保留令牌与完整令牌之间的分布变化。基于这些观察，我们提出了一种两阶段剪枝框架，包括锚点-上下文图恢复（Anchor-Context Graph Recovery, ACGR）和文本感知令牌聚类选择（Text-Aware Token Cluster Selection, TATCS）。具体而言，ACGR 在令牌移除之前转移上下文信息，而 TATCS 在检测到严重的分布变化时动态重新选择代表性令牌。大量实验表明，我们的方法在超低令牌预算下实现了更优越且更稳定的性能。值得注意的是，它在仅使用 16 个视觉令牌的情况下，保持了 LLaVA-1.5-7B 上 92.1% 的上限平均性能。

View on arXiv Download PDF AI Translation

cs.CV / 226 / 2606.16638

MVM-IOD: An Industrial Object-Centric Benchmark Dataset for the Evaluation of 3D Reconstruction Methods

MVM-IOD：用于评估3D重建方法的工业对象中心基准数据集

Langendörfer, Robert, Hillemann, Markus, Ulrich, Markus

Abstract

3D object reconstruction, and camera pose estimation in industrial applications are challenging tasks, as errors are costly while the computation time is often limited. The complexity of typical industrial objects further complicates these tasks. Most of the existing datasets in this context do not depict realistic industrial scenarios. Therefore, we introduce the Machine Vision Metrology Industrial Object Dataset (MVM-IOD). Images of typical industrial objects are captured systematically, by moving a camera, mounted at the end effector of an industrial robot arm, on a hemisphere around the objects. MVM-IOD contains reference camera poses and reference 3D point clouds, the acquired RGB images of 9 objects and 2 background choices resulting in 18 scenes, which allows evaluation of all image based methods that compute a 3D reconstruction, camera poses, or novel views of a scene. Based on MVM-IOD, we extensively evaluate current SOTA 3D reconstruction and camera pose estimation methods, such as Structure from Motion, Multi-View Stereo, recent feed forward methods (Visual Geometry Grounded Transformer, {\pi}3), and 2D Gaussian Splatting and report our findings as a baseline for future research. The experiments show that capture setups like ours generate out-of distribution images for feed forward methods, leading to suboptimal point clouds and camera poses. However, these out-of-distribution images can be shifted closer to the training distribution by applying simple preprocessing steps. Consequently, in certain industrial applications, feed forward methods should be used with caution.

Chinese Translation

在工业应用中，3D对象重建和相机姿态估计是具有挑战性的任务，因为错误代价高昂，而计算时间通常有限。典型工业对象的复杂性进一步增加了这些任务的难度。现有的大多数数据集并未描绘出真实的工业场景。因此，我们引入了机器视觉计量工业对象数据集（MVM-IOD）。通过将相机安装在工业机器人手臂的末端执行器上，系统性地在对象周围的半球上移动相机，捕获典型工业对象的图像。MVM-IOD包含参考相机姿态和参考3D点云，以及9个对象和2个背景选择所获得的RGB图像，形成18个场景，从而允许评估所有基于图像的方法，这些方法计算3D重建、相机姿态或场景的新视图。基于MVM-IOD，我们对当前的最先进（SOTA）3D重建和相机姿态估计方法进行了广泛评估，例如运动结构（Structure from Motion）、多视图立体（Multi-View Stereo）、最近的前馈方法（Visual Geometry Grounded Transformer, { ext{π}}3）以及2D高斯点云（2D Gaussian Splatting），并将我们的发现报告作为未来研究的基线。实验表明，像我们这样的捕获设置为前馈方法生成了分布外图像，导致次优的点云和相机姿态。然而，通过应用简单的预处理步骤，这些分布外图像可以更接近训练分布。因此，在某些工业应用中，前馈方法应谨慎使用。

View on arXiv Download PDF AI Translation

cs.CV / 227 / 2606.16658

Vision-Language Models as Zero-Annotation Oracles in Histopathology

视觉-语言模型作为组织病理学中的零注释神谕

Jain, Vishal, Buzzanca, Giorgio, Cechnicka, Sarah, Naesens, Maarten, Koshy, Priyanka, Nguyen, Tri, Kers, Jesper, Roufosse, Candice, Kainz, Bernhard

Abstract

Foreground segmentation is the critical first step of every computational pathology pipeline, yet existing methods rely on hand-tuned heuristics or supervised models that overfit to narrow stain and scanner distributions, failing silently on specialised stains such as Jones silver or Elastica van Gieson. We propose a coarse-to-fine approach that recasts foreground segmentation as a visual perception task and leverages general-purpose vision-language models (VLMs) as zero-annotation oracles. Our key insight is that tissue-versus-background discrimination is a natural-image recognition problem, not a histopathological one, so VLMs trained on internet-scale corpora generalise where domain-specific models cannot. We introduce Leica-75, a benchmark of 75 renal transplant whole-slide images spanning three stain families. On Leica-75, our method achieves the highest segmentation quality on out-of-distribution stains (Dice 0.858 +/- 0.027 on Jones, 0.853 +/- 0.041 on EVG) with 7x lower cross-stain variance than the best supervised baseline, while remaining competitive on in-distribution H&E. Few-shot prompting with automatically curated exemplars (Auto-context) rescues hard cases on Stress-32 (n=32), a curated stress-test subset (Dice 0.470 to 0.819 for the 2B model). VLM-based annotation review matches human expert consensus (kappa=0.989 for blur detection; mean precision/recall grading accuracy 0.708 vs. human 0.646 for segmentation mask review). The resulting pseudo-labels are used to distil lightweight student models that are as performant as the teacher model while running for a fraction of the cost. Our framework provides a principled, scalable solution to a persistent infrastructure bottleneck in digital pathology.

Chinese Translation

前景分割是每个计算病理学流程的关键第一步，然而现有方法依赖于手动调整的启发式方法或过拟合于狭窄染色和扫描仪分布的监督模型，在专门染色（如琼斯银染色或弹性Van Gieson染色）上默默失败。我们提出了一种粗到细的方法，将前景分割重新定义为视觉感知任务，并利用通用的视觉-语言模型（VLMs）作为零注释神谕。我们的关键见解是，组织与背景的区分是一个自然图像识别问题，而非组织病理学问题，因此在互联网规模语料库上训练的VLMs能够在领域特定模型无法泛化的情况下进行推广。我们引入了Leica-75，这是一个包含75个肾脏移植全切片图像的基准，涵盖三种染色类型。在Leica-75上，我们的方法在分布外染色上实现了最高的分割质量（琼斯染色的Dice为0.858 +/- 0.027，EVG的Dice为0.853 +/- 0.041），其交叉染色方差比最佳监督基线低7倍，同时在分布内的H&E染色上保持竞争力。通过自动策划的示例进行少量提示（Auto-context）能够挽救Stress-32（n=32）上的困难案例（2B模型的Dice从0.470提升至0.819）。基于VLM的注释审查与人类专家共识相匹配（模糊检测的kappa=0.989；分割掩膜审查的平均精确度/召回率评分准确度为0.708，而人类为0.646）。生成的伪标签用于提炼轻量级学生模型，其性能与教师模型相当，同时运行成本仅为其一小部分。我们的框架为数字病理学中的持续基础设施瓶颈提供了一个有原则的、可扩展的解决方案。

View on arXiv Download PDF AI Translation

cs.CV / 228 / 2606.16667

Look Again Before You Abstain:Budgeted Conformal Evidence Acquisition for Reliable Vision-Language Model

再次审视再决定：用于可靠视觉-语言模型的预算化符合证据获取

Xu, Jian, Zeng, Delu, Paisley, John, Zhao, Qibin

Abstract

Large vision-language models (LVLMs) hallucinate: they assert visual details that the image does not support. A principled remedy is selective prediction with a distribution-free guarantee-verify each claim and abstain when the claim is not grounded, so that the hallucination rate among asserted claims is provably bounded. We show, however, that this guarantee is bought at a brutal price: to keep the hallucination rate below $5\%$ on a balanced object-existence benchmark, a state-of-the-art conformal filter must abstain on more than $80\%$ of claims. We argue that abstention is wasteful when more visual evidence is cheaply available, and introduce Budgeted Conformal Evidence Acquisition (BCEA), which replaces the binary answer/abstain decision with a three-way choice: answer, abstain, or acquire additional visual evidence by re-examining the image (zooming, cropping, or applying a claim-specific intervention) under a bounded compute budget. We make two observations. First, acquisition that is plugged naively into a calibrated filter breaks the statistical guarantee -- realized risk overshoots the target by up to $17$ points -- because the acquisition step destroys the exchangeability that conformal calibration relies on. Second, folding the entire acquisition policy into the score function and re-calibrating on post-acquisition scores \emph{restores} the finite-sample guarantee while still recovering coverage. BCEA further uses structured, claim-type-specific interventions. Across the POPE benchmark and COCO-constructed existence and spatial-relation claims, on four open VLMs, BCEA controls the hallucination rate at the target level and consistently improves coverage over a guaranteed-abstention baseline.

Chinese Translation

大型视觉-语言模型（LVLMs）存在幻觉现象：它们声称图像中存在的视觉细节并不支持。一个合理的解决方案是选择性预测，提供无分布保证——验证每一个声明，并在声明没有依据时选择放弃，从而使得所声称的幻觉率在可证明的范围内。然而，我们展示了这一保证是以极高的代价换来的：为了在平衡的物体存在基准上将幻觉率控制在$5\%$以下，最先进的符合过滤器必须对超过$80\\%$的声明选择放弃。我们认为，当更多的视觉证据可以便宜地获取时，放弃是浪费的，因此引入了预算化符合证据获取（BCEA），它将二元的回答/放弃决策替换为三种选择：回答、放弃或在有限的计算预算下通过重新审视图像（缩放、裁剪或应用声明特定的干预）获取额外的视觉证据。我们做出了两个观察。首先，简单地将获取步骤插入到校准过滤器中会破坏统计保证——实现的风险超出了目标，最多达$17$个点，因为获取步骤破坏了符合校准所依赖的可交换性。其次，将整个获取策略融入评分函数并在后获取评分上重新校准 extit{恢复}了有限样本保证，同时仍然保持覆盖率。BCEA进一步使用结构化的、特定于声明类型的干预。在POPE基准和COCO构建的存在和空间关系声明上，在四个开放的VLMs中，BCEA将幻觉率控制在目标水平，并在保证放弃的基线之上持续改善覆盖率。

View on arXiv Download PDF AI Translation

cs.CV / 229 / 2606.16672

Sinkhorn-CPD: Robust point cloud registration via unbalanced entropic optimal transport

Sinkhorn-CPD：通过不平衡熵最优传输实现稳健的点云配准

Zhang, Jin, Zhao, Mingyang, Liu, Bing, Jiang, Xin

Abstract

Coherent Point Drift (CPD) is widely used for rigid point cloud registration because of its soft correspondences and closed-form parameter updates. However, CPD's target-side marginal constraint forces every observation, including outliers, to receive exactly unit probability mass. This assumption degrades registration accuracy under heavy outliers and partial overlap. Optimal transport (OT) methods can handle missing mass through unbalanced formulations, but require hand-tuned annealing schedules. In this paper, we propose Sinkhorn-CPD, which replaces CPD's target-side marginal constraint with dual Kullback-Leibler penalties, allowing the algorithm to discard outliers on both sides. The resulting formulation is a fully unbalanced entropic optimal transport problem, which can be efficiently solved by generalized Sinkhorn iterations. Moreover, Sinkhorn-CPD preserves the closed-form Procrustes and variance updates of CPD. In our method, the variance sigma^2 plays the role of the entropic regularization parameter, which induces an automatic annealing schedule from diffuse to sharp correspondences without manual temperature tuning. Experiments on synthetic, cross-category, and scan-to-CAD benchmarks show that Sinkhorn-CPD achieves state-of-the-art accuracy, with strong robustness to outliers and partial overlap.

Chinese Translation

一致点漂移（Coherent Point Drift, CPD）因其柔性对应关系和封闭形式的参数更新而广泛应用于刚性点云配准。然而，CPD的目标侧边际约束强迫每个观测值（包括离群点）都获得恰好单位概率质量。这一假设在存在大量离群点和部分重叠的情况下会降低配准精度。最优传输（Optimal Transport, OT）方法可以通过不平衡的形式处理缺失质量，但需要手动调节退火进程。在本文中，我们提出了Sinkhorn-CPD，它用双重Kullback-Leibler惩罚替代了CPD的目标侧边际约束，从而允许算法在两侧丢弃离群点。得到的公式是一个完全不平衡的熵最优传输问题，可以通过广义Sinkhorn迭代高效求解。此外，Sinkhorn-CPD保留了CPD的封闭形式Procrustes和方差更新。在我们的方法中，方差σ²充当熵正则化参数的角色，能够在不需要手动温度调节的情况下，从模糊到清晰的对应关系自动引导退火进程。在合成数据、跨类别和扫描到CAD基准测试中的实验表明，Sinkhorn-CPD实现了最先进的精度，并对离群点和部分重叠表现出强大的鲁棒性。

View on arXiv Download PDF AI Translation

cs.CV / 230 / 2606.16673

MMDiff: Extending Diffusion Transformers for Multi-Modal Generation

MMDiff：扩展扩散变换器以实现多模态生成

Akarken, Yagmur, Kupyn, Orest, Rupprecht, Christian

Abstract

Diffusion transformers have demonstrated remarkable generative capabilities, yet the rich perceptual representations computed across their denoising trajectory are discarded once the content is rendered. We present MMDiff, a framework that transforms a frozen diffusion transformer into a multi-modal generative system that jointly produces images alongside any combination of dense perceptual modalities using lightweight decoder heads. Our central finding is that perceptual information is temporally distributed along the denoising trajectory, and that multi-timestep feature fusion with spatially varying aggregation weights is essential, improving semantic segmentation results by up to 28.7% mIoU over single-timestep extraction. We further adopt concept-driven attention extraction for interpretable spatial guidance, and show that frozen diffusion features are competitive with and complementary to state-of-the-art encoders such as DINOv3. By training only lightweight decoder heads on a frozen backbone, we achieve strong performance in semantic segmentation, salient object detection, and depth estimation, and demonstrate that this framework enables effective synthetic data generation at scale.

Chinese Translation

扩散变换器展示了显著的生成能力，但在内容渲染后，其在去噪轨迹中计算的丰富感知表示被丢弃。我们提出了MMDiff，一个将冻结的扩散变换器转变为多模态生成系统的框架，该系统能够使用轻量级解码头联合生成图像及任何组合的密集感知模态。我们的核心发现是，感知信息在去噪轨迹上是时间分布的，具有空间变化聚合权重的多时间步特征融合是至关重要的，这使得语义分割结果比单时间步提取提高了多达28.7%的mIoU。我们进一步采用基于概念的注意力提取，以实现可解释的空间引导，并展示冻结的扩散特征在与DINOv3等最先进编码器的竞争中具有竞争力和互补性。通过仅在冻结的主干上训练轻量级解码头，我们在语义分割、显著目标检测和深度估计方面取得了强劲的表现，并证明该框架能够有效地进行大规模合成数据生成。

View on arXiv Download PDF AI Translation

cs.CV / 231 / 2606.16742

Revealing Artifacts via Noise Amplification: A Novel Perspective for AI-Generated Video Detection

通过噪声放大揭示伪影：AI生成视频检测的新视角

Cheng, Renxi, Gui, Jie, Wang, Hongsong

Abstract

With the rapid advancement of video generation models, distinguishing between AI-generated and authentic videos has emerged as a challenging endeavor. The majority of existing research endeavors concentrate on the development of detectors for identifying samples generated by generative adversarial networks. Nevertheless, the detection of AI-generated videos, particularly those produced by text-to-video models, still remains an uncharted territory. Although state-of-the-art text-to-video models can generate realistic visual content similar to real videos, they fall short of generating the details of the images and the changes in details within the videos. Inspired by this, we address AI-generated video detection from a novel perspective of bit-planes, which can effectively describe the details or noises in images or videos. To this end, we propose a simple yet effective approach called Noise Amplification. This approach first extracts noise signals based on bit-planes, then amplifies these noise signals, and finally feeds them into the discriminator networks for video fake classification. Noise amplification is comprehensively constructed by incorporating three aspects: pixel-level intensity enhancement, region-level spatial amplification, and frame-level temporal aggregation. To evaluate methods of AI-generated video detection in challenging scenarios, we also introduce a benchmark named HardGVD. Extensive experiments on both the large-scale dataset GenVidBench and HardGVD show that our simple approach significantly outperforms state-of-the-art methods.

Chinese Translation

随着视频生成模型的快速发展，区分AI生成的视频与真实视频已成为一项具有挑战性的任务。现有研究大多集中于开发检测器，以识别由生成对抗网络生成的样本。然而，AI生成视频的检测，尤其是由文本到视频模型生成的视频，仍然是一个未被探索的领域。尽管最先进的文本到视频模型能够生成与真实视频相似的逼真视觉内容，但它们在生成图像细节和视频中细节变化方面仍显不足。受此启发，我们从比特平面（bit-planes）的新视角出发，探讨AI生成视频检测，这一视角能够有效描述图像或视频中的细节或噪声。为此，我们提出了一种简单而有效的方法，称为噪声放大（Noise Amplification）。该方法首先基于比特平面提取噪声信号，然后放大这些噪声信号，最后将其输入到判别网络中进行视频伪造分类。噪声放大综合构建了三个方面：像素级强度增强、区域级空间放大和帧级时间聚合。为了评估在具有挑战性的场景下的AI生成视频检测方法，我们还引入了一个基准测试，命名为HardGVD。在大规模数据集GenVidBench和HardGVD上的大量实验表明，我们的方法显著优于最先进的技术。

View on arXiv Download PDF AI Translation

cs.CV / 232 / 2606.16749

Structure-aware Knowledge-guided Heterogeneous Mamba for Zygomaticomaxillary Suture Assessment

结构感知知识引导的异构Mamba用于颧上颌缝评估

Guo, Xiaoqi, Chen, Birui, Yang, Xinquan, Zhang, Chaoyun, Liu, Xuefen, Zheng, Mianjie, Tang, Kun, Li, Xuguang, Ma, Wen, Xu, Yanhua, Shen, Linlin

Abstract

The Zygomaticomaxillary Suture is a key circummaxillary structure that connects the zygomatic bone and the maxilla, which serves as a primary site of resistance during maxillary advancement, and its maturation status directly influences the timing and efficacy of orthopedic interventions. However, accurate staging of ZMS maturation remains challenging due to subtle high-frequency transitions in suture lines and the global semantic ambiguity between adjacent stages. To address this, we present the first public ZMS dataset, comprising 3,790 ZMS images covering the entire age range from 4 to 24 years. Based on this dataset, we propose SKMamba, a Structure-aware and Knowledge-guided Mamba-based multi-modal framework for automated ZMS maturation assessment. SKMamba adopts a decoupled dual-path architecture that mimics the hierarchical diagnostic process used by experienced orthodontists. We first introduce an Implicit Edge Extractor (IEE), which leverages structural pre-training to reduce trabecular noise and accentuate sutural boundaries. Complementarily, a Cross-Modal Semantic Alignment (CSA) module is designed to incorporate anatomical descriptions from a large language model (LLM). This module helps align local morphological cues with global semantic descriptions while ensuring that objective morphological evidence remains the primary basis for decisions. Extensive experiments on our ZMS dataset demonstrate that SKMamba achieves state-of-the-art performance compared to existing methods. Code is available at https://github.com/galaxygxq1116/SKMamba.

Chinese Translation

颧上颌缝是连接颧骨和上颌骨的关键环状结构，是上颌前移过程中的主要阻力点，其成熟状态直接影响正畸干预的时机和效果。然而，由于缝合线的细微高频变化以及相邻阶段之间的全局语义模糊性，准确评估ZMS（Zygomaticomaxillary Suture）成熟度仍然具有挑战性。为了解决这一问题，我们首次公开了ZMS数据集，包含3,790张涵盖4至24岁整个年龄段的ZMS图像。基于该数据集，我们提出了SKMamba，一种基于结构感知和知识引导的Mamba多模态框架，用于自动化ZMS成熟度评估。SKMamba采用解耦的双路径架构，模拟经验丰富的正畸医生的分层诊断过程。我们首先引入了一种隐式边缘提取器（Implicit Edge Extractor, IEE），利用结构预训练来减少小梁噪声并突出缝合边界。作为补充，设计了一个跨模态语义对齐（Cross-Modal Semantic Alignment, CSA）模块，以整合来自大型语言模型（Large Language Model, LLM）的解剖描述。该模块有助于将局部形态线索与全局语义描述对齐，同时确保客观形态证据仍然是决策的主要依据。在我们的ZMS数据集上进行的广泛实验表明，SKMamba在与现有方法的比较中实现了最先进的性能。代码可在 https://github.com/galaxygxq1116/SKMamba 获得。

View on arXiv Download PDF AI Translation

cs.CV / 233 / 2606.16756

3D Classification of Paramagnetic Rim Lesions in Multiple Sclerosis via Asymmetric QSM-FLAIR Modeling

通过不对称 QSM-FLAIR 建模对多发性硬化症中的顺磁性边缘病变进行 3D 分类

Pignedoli, Veronica, Boffa, Giacomo, Noceti, Nicoletta, Inglese, Matilde, Odone, Francesca, Moro, Matteo

Abstract

Paramagnetic rim lesions (Rim$^+$) identified on susceptibility-sensitive MRI have recently emerged as a specific biomarker of chronic active inflammation in Multiple Sclerosis (MS) and are associated with long-term disability progression. However, susceptibility imaging and expert interpretation remain limited to specialized centers, visual assessment is time-consuming and variable, and the low prevalence of Rim$^+$ lesions poses severe class imbalance challenges for automated analysis. We propose a 3D multimodal deep learning framework for lesion-level Rim$^+$/Rim$^-$ classification from Quantitative Susceptibility Mapping (QSM) and FLAIR MRI. The architecture explicitly models modality asymmetry by treating QSM as the primary susceptibility-driven signal and conditioning it with FLAIR-derived structural context. To improve robustness under limited data, we employ self-supervised multimodal pretraining followed by supervised fine-tuning with contrastive regularization. The method was evaluated on a clinically acquired cohort of 88 people with MS with expert lesion annotations as reference standard. Results highlight improved performance compared to prior architectures, supporting the effectiveness of asymmetric multimodal modeling for automated chronic active lesion identification.

Chinese Translation

在易感性敏感的 MRI 上识别出的顺磁性边缘病变（Rim$^+$）最近被认为是多发性硬化症（MS）中慢性活跃炎症的特异性生物标志物，并与长期残疾进展相关。然而，易感性成像和专家解读仍然局限于专业中心，视觉评估耗时且变异性大，而 Rim$^+$ 病变的低发生率给自动化分析带来了严重的类别不平衡挑战。我们提出了一种 3D 多模态深度学习框架，用于从定量易感性映射（QSM）和 FLAIR MRI 中进行病变级别的 Rim$^+$/Rim$^-$ 分类。该架构通过将 QSM 视为主要的易感性驱动信号，并结合 FLAIR 派生的结构上下文，明确建模了模态不对称性。为了在有限数据下提高鲁棒性，我们采用自监督多模态预训练，随后进行对比正则化的监督微调。该方法在 88 名 MS 患者的临床获取队列上进行了评估，以专家病变注释作为参考标准。结果显示，与先前的架构相比，性能有所提升，支持不对称多模态建模在自动化慢性活跃病变识别中的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 234 / 2606.16767

Text-Vision Co-Instructed Image Editing

文本-视觉共同指导的图像编辑

Xie, Chenxi, Wu, Yuhui, Yi, Qiaosi, Zhang, Lei

Abstract

Existing image editing methods can be generally categorized into textual instruction-based and visual prompt-based ones. Textual instructions are semantically expressive, but are limited by the coarse granularity of spatial control of the editing results. In contrast, visual prompts such as drag and point can provide precise spatial guidance, but are limited by the inherent ambiguity in semantic intent. To unify the strength of textual and visual prompts, we present Text-Vision Co-Instructed Image Editing, which jointly models textual instructions as semantic intent and sparse visual instructions as spatial guidance, aiming to achieve precise and intent-faithful image manipulation. To this end, we first construct a textual-visual instruction paired dataset with more than 23K samples derived from dynamic videos, enabling aligned supervision for cross-modal instruction. We then propose TV-Edit, a Textual-Visual instruction unified Editing framework to contextualize drag or point-based visual instructions with image-text semantics and lift them into semantic-aware control representations for pretrained editing backbones. By integrating semantic intent and spatial constraints, TV-Edit leads to more precise spatial control, less instruction ambiguity, and stronger structural consistency than text-only or drag-based alternatives. Finally, we establish TV-Edit-Bench, a deliberately designed benchmark to evaluate semantic faithfulness, spatial alignment, and visual consistency with ground-truth references and controlled textual-visual variations for reliable assessment. Our experiments across multiple editing backbones demonstrate that TV-Edit consistently yields more precise and intent-faithful edits, significantly outperforming state-of-the-art instruction-based and drag-based baselines.

Chinese Translation

现有的图像编辑方法通常可以分为基于文本指令和基于视觉提示的两类。文本指令语义表达丰富，但在编辑结果的空间控制上受限于粗糙的粒度。相比之下，拖拽和指点等视觉提示可以提供精确的空间指导，但在语义意图上存在固有的模糊性。为了统一文本和视觉提示的优势，我们提出了文本-视觉共同指导的图像编辑（Text-Vision Co-Instructed Image Editing），该方法将文本指令建模为语义意图，将稀疏的视觉指令建模为空间指导，旨在实现精确且忠实于意图的图像操作。为此，我们首先构建了一个文本-视觉指令配对的数据集，包含来自动态视频的超过23K样本，使得跨模态指令的对齐监督成为可能。然后，我们提出了TV-Edit，一个文本-视觉指令统一编辑框架，将基于拖拽或指点的视觉指令与图像-文本语义进行上下文化，并将其提升为语义感知控制表示，以适应预训练的编辑骨干网络。通过整合语义意图和空间约束，TV-Edit实现了比仅基于文本或基于拖拽的替代方案更精确的空间控制、更少的指令模糊性和更强的结构一致性。最后，我们建立了TV-Edit-Bench，这是一个经过精心设计的基准，用于评估与真实参考和受控文本-视觉变体的语义忠实性、空间对齐和视觉一致性，以便进行可靠的评估。我们在多个编辑骨干网络上的实验表明，TV-Edit始终能够产生更精确且忠实于意图的编辑，显著优于最先进的基于指令和基于拖拽的基线。

View on arXiv Download PDF AI Translation

cs.CV / 235 / 2606.16783

Gen-VCoT: Generative Visual Chain-of-Thought Reasoning via Diffusion-Based RGB Intermediate Representations

Gen-VCoT：基于扩散的RGB中间表示的生成视觉推理链

Zhou, Zhiqiang, Dai, Junliang, ling, Xu

Abstract

Multimodal large language models (MLLMs) excel at visual reasoning but rely on text-based chain-of-thought (CoT), lacking interpretable visual intermediates. Existing methods use opaque tokens or external tools, missing key properties. We propose Gen-VCoT, a framework using expert vision models to generate RGB images as reasoning intermediates. It has three stages: visual grounding (SAM segmentation), geometric reasoning (Marigold depth maps), and semantic reasoning (Qwen2-VL integration). An adaptive router selects reasoning depth. Evaluations show Gen-VCoT improves spatial (25% better) and depth (50% better) questions, but may hurt simple factual queries. Text CoT outperforms visual intermediates on CLEVR (91.2% vs 62.5%), showing task-dependent optimal representations. Gen-VCoT establishes a new paradigm for interpretable multimodal reasoning.

Chinese Translation

多模态大语言模型（MLLMs）在视觉推理方面表现出色，但依赖于基于文本的推理链（CoT），缺乏可解释的视觉中间体。现有方法使用不透明的标记或外部工具，缺失关键属性。我们提出了Gen-VCoT，一个利用专家视觉模型生成RGB图像作为推理中间体的框架。该框架分为三个阶段：视觉定位（SAM分割）、几何推理（Marigold深度图）和语义推理（Qwen2-VL集成）。自适应路由器选择推理深度。评估结果表明，Gen-VCoT在空间问题上提高了25%的表现，在深度问题上提高了50%，但可能会影响简单事实查询。文本推理链在CLEVR上的表现优于视觉中间体（91.2%对62.5%），显示出任务依赖的最佳表示。Gen-VCoT为可解释的多模态推理建立了一个新范式。

View on arXiv Download PDF AI Translation

cs.CV / 236 / 2606.16794

LLM-Based Visual Explanation Evaluation Framework for Assessing the Explainability of Facial Skin Disease Classification Models

基于大型语言模型的视觉解释评估框架：评估面部皮肤疾病分类模型的可解释性

Na, Gyuyeon

Abstract

This study proposes a domain-specific LLM-based Visual Explanation Evaluation Framework for assessing Grad-CAM explanations in facial skin disease diagnosis models. While previous studies have primarily focused on improving classification performance through data augmentation techniques, relatively few studies have systematically examined whether model explanations are grounded in clinically relevant lesion regions. In this study, geometric augmentation, color-based augmentation, and mixed augmentation strategies were applied to facial skin disease classification models based on EfficientNet-B0, MobileNetV3, and ResNet18. Grad-CAM was employed to generate visual explanations representing the models' decision-making processes. Furthermore, an LLM-as-a-Judge evaluation framework was designed using GPT-5.5, Gemini 3.5 Flash, and Claude Sonnet 4.6 to assess Grad-CAM explanations from the perspectives of lesion localization and explanation trustworthiness. To improve evaluation consistency and clinical grounding, a progressive prompt engineering strategy was introduced, incorporating evaluation rubrics, clinical knowledge, penalty rules, and structured output formats.

Chinese Translation

本研究提出了一种特定领域的基于大型语言模型（LLM）的视觉解释评估框架，用于评估面部皮肤疾病诊断模型中的Grad-CAM解释。尽管以往研究主要集中在通过数据增强技术提高分类性能，但系统性地检验模型解释是否与临床相关的病变区域相一致的研究相对较少。在本研究中，针对基于EfficientNet-B0、MobileNetV3和ResNet18的面部皮肤疾病分类模型，应用了几何增强、基于颜色的增强和混合增强策略。使用Grad-CAM生成了代表模型决策过程的视觉解释。此外，设计了一种基于LLM的评估框架，利用GPT-5.5、Gemini 3.5 Flash和Claude Sonnet 4.6，从病变定位和解释可信度的角度评估Grad-CAM解释。为了提高评估的一致性和临床基础，提出了一种渐进式提示工程策略，结合了评估标准、临床知识、惩罚规则和结构化输出格式。

View on arXiv Download PDF AI Translation

cs.CV / 237 / 2606.16795

WaveDINO: Learning-Based Atmospheric Correction of Unwrapped InSAR Interferograms Validated by GNSS: Results at Laguna del Maule and Campi Flegrei Volcanoes

WaveDINO：基于学习的解包InSAR干涉图的大气校正，经过GNSS验证：在拉古纳德尔毛莱和坎皮弗莱格雷火山的结果

Popescu, Robert, Biggs, Juliet, Zhu, Tianyuan, Anantrasirichai, Nantheera

Abstract

Interferometric Synthetic Aperture Radar (InSAR) enables effective monitoring of volcanic deformation; however, the observed signals are often corrupted by atmospheric phase delays, seasonal surface changes, and decorrelation effects. Existing atmospheric correction methods, such as numerical weather model-based methods, can reduce these effects but do not consistently remove atmospheric artefacts and may introduce residual biases. To address these limitations, we propose a novel learning-based method for denoising unwrapped InSAR interferograms, using a hybrid training strategy that combines physically motivated synthetic deformation with real atmospheric noise. Specifically, we introduce WaveDINO, a wavelet-based multi-scale denoising framework conditioned on frozen DINOv3 foundation-model features and terrain information. Training uses synthetic magma-source deformation superimposed on short-term interferograms to expose the network to realistic atmospheric statistics while retaining known ground truth. Performance is evaluated on both controlled synthetic data and long-term real interferograms from Laguna del Maule (Chile) and Campi Flegrei (Italy), with independent GNSS measurements used for validation. WaveDINO consistently outperforms competing models, improving agreement with GNSS measurements, and reducing mean GNSS misfit by approximately 3% and 19% at two sites, respectively, while surpassing weather-model-based corrections.

Chinese Translation

干涉合成孔径雷达（InSAR）能够有效监测火山变形；然而，观察到的信号常常受到大气相位延迟、季节性地表变化和去相关效应的干扰。现有的大气校正方法，如基于数值天气模型的方法，能够减少这些影响，但并不能始终消除大气伪影，并可能引入残余偏差。为了解决这些局限性，我们提出了一种新颖的基于学习的方法，用于去噪解包的InSAR干涉图，采用一种混合训练策略，将物理驱动的合成变形与真实的大气噪声相结合。具体而言，我们引入了WaveDINO，这是一种基于小波的多尺度去噪框架，依赖于冻结的DINOv3基础模型特征和地形信息进行条件处理。训练使用合成的岩浆源变形叠加在短期干涉图上，以使网络接触到真实的大气统计特征，同时保留已知的地面真值。性能在受控的合成数据和来自拉古纳德尔毛莱（智利）和坎皮弗莱格雷（意大利）的长期真实干涉图上进行评估，并使用独立的GNSS测量进行验证。WaveDINO在各个竞争模型中表现一致优越，与GNSS测量的吻合度提高，在两个地点分别减少了约3%和19%的GNSS误差，同时超越了基于天气模型的校正。

View on arXiv Download PDF AI Translation

cs.CV / 238 / 2606.16799

Decoupling Semantics from Distortions: Multi-Scale Two-Stream Vision-Language Alignment for AI-Generated Image Quality Assessment

解耦语义与失真：多尺度双流视觉-语言对齐用于AI生成图像质量评估

Meng, Zijie

Abstract

Existing vision-language model (VLM)-based AI-generated image quality assessment (AIGIQA) methods suffer from a fundamental semantic-distortion dimensional conflict: monolithic representations optimized for semantic discrimination inherently entangle compositional understanding with low-level perceptual sensitivity, rendering them blind to fine-grained quality degradations. We introduce MST-CLIPIQA, a multi-scale two-stream framework that achieves hierarchical vision-language alignment through explicit representational decoupling. Our architecture leverages dual CLIP encoders with complementary patch granularities: coarse-grained streams capture global semantic coherence while fine-grained streams preserve textural signatures and artifact patterns. An information bottleneck-inspired gated fusion mechanism performs adaptive cross-scale distillation, with optional cross-attention enabling prompt-anchored correspondence evaluation when generation prompts are available. Extensive experiments across five benchmarks establish new state-of-the-art results, achieving average improvements of 1.11 percent SRCC on quality and 2.35 percent SRCC on text-image correspondence prediction, while maintaining efficiency with only 0.8M trainable parameters. Our project is available at https://github.com/YMlinfeng/MST-CLIPIQA.

Chinese Translation

现有基于视觉-语言模型（VLM）的AI生成图像质量评估（AIGIQA）方法存在一个基本的语义-失真维度冲突：为语义区分优化的单一表示本质上将组合理解与低级感知敏感性纠缠在一起，使其对细粒度的质量退化视而不见。我们提出了MST-CLIPIQA，一个多尺度双流框架，通过显式的表示解耦实现层次化的视觉-语言对齐。我们的架构利用了具有互补补丁粒度的双CLIP编码器：粗粒度流捕捉全局语义一致性，而细粒度流保留纹理特征和伪影模式。受信息瓶颈启发的门控融合机制执行自适应跨尺度蒸馏，选配的交叉注意力在生成提示可用时实现提示锚定的对应评估。在五个基准测试上的广泛实验建立了新的最先进结果，在质量评估上平均提高了1.11%的SRCC，在文本-图像对应预测上提高了2.35%的SRCC，同时仅使用0.8M的可训练参数保持了效率。我们的项目可在https://github.com/YMlinfeng/MST-CLIPIQA获取。

View on arXiv Download PDF AI Translation

cs.CV / 239 / 2606.16837

Robust Spoofed Speech Detection via Temporal Pyramid Modeling

通过时间金字塔建模实现鲁棒的伪造语音检测

Nezhad, Mahtab Masoudi, Karimian, Nima

Abstract

Spoofed speech detection is increasingly challenged by realistic synthesis, voice conversion, and replay attacks, with cross-dataset generalization remaining a major limitation. This work we propose a Temporal Pyramid Adapter that utilize parallel temporal convolutions with varying receptive fields to capture multi-scale spoofing cues, ranging from local artifacts to global prosodic irregularities. We also integrated self-supervised XLS-R representations combined with front-end adapters, including Mel, Sinc, and a Temporal Pyramid design for multi-scale temporal modeling. The proposed model is evaluated cross multiple benchmark including ASVspoof 2017, ASVspoof 2021 (DF/LA), PartialSpoof, DiffSSD, and multilingual HQ-MPSD datasets. Experimental results demonstrate that Temporal Pyramid model obtained AUC of 99.24% and a EER of 3.87% on the PartialSpoof database, which is significantly outperforming the base model and several SOTA baseline such as LCNN-BLSTM (9.87% EER) and TRACE (8.08% EER). Additionally, multilingual evaluations confirm that while spoofing artifact are independent from language. While self-supervised representations improve robustness, performance degrades under domain and language shifts, highlighting the need for better adaptation and calibration strategies.

Chinese Translation

伪造语音检测正面临着现实合成、语音转换和重放攻击的挑战，跨数据集的泛化能力仍然是一个主要限制。本研究提出了一种时间金字塔适配器（Temporal Pyramid Adapter），利用具有不同感受野的并行时间卷积来捕捉多尺度的伪造线索，从局部伪影到全局韵律不规则性。我们还结合了自监督的XLS-R表示与前端适配器，包括Mel、Sinc和时间金字塔设计，以实现多尺度时间建模。所提出的模型在多个基准数据集上进行了评估，包括ASVspoof 2017、ASVspoof 2021（DF/LA）、PartialSpoof、DiffSSD和多语言HQ-MPSD数据集。实验结果表明，时间金字塔模型在PartialSpoof数据库上获得了99.24%的AUC和3.87%的EER，显著优于基础模型及多个最先进的基线模型，如LCNN-BLSTM（9.87% EER）和TRACE（8.08% EER）。此外，多语言评估确认伪造伪影与语言无关。虽然自监督表示提高了鲁棒性，但在领域和语言转变下性能下降，强调了更好的适应和校准策略的必要性。

View on arXiv Download PDF AI Translation

cs.CV / 240 / 2606.16861

An Open-Source Monitoring Framework for Data Exploration and Progress Tracking in Multi-Center Radiology Studies

一个开源监测框架用于多中心放射学研究中的数据探索和进展跟踪

Bujotzek, Markus, Scherer, Jonas, Denner, Stefan, Neher, Peter, Hamm, Benjamin, Feineis, Lorenz, Akuenal, Uenal, Bucher, Andreas, Penzkofer, Tobias, Maier-Hein, Klaus

Abstract

Multi-center studies are crucial for advancing medical and radiological research. Data exploration, collaboration discovery, and study progress monitoring are essential for maximizing their potential. However, in practice these processes often rely on manual communication and shared tables, which quickly become outdated and hinder efficient coordination in large distributed studies. This highlights the need for dedicated monitoring solutions that provide transparent and up-to-date insights into study progress. We propose a lightweight, open-source monitoring architecture for multi-center studies based on the widely used Grafana-Prometheus stack. The framework collects aggregated monitoring metrics from distributed study sites and visualizes them through configurable dashboards. As a real-world deployment example, the framework is integrated into the medical imaging platform Kaapana and evaluated within a large multi-center research network. By deploying our solution within the Germany-wide RACOON consortium, we demonstrate its ability to enable privacy-preserving data exploration and study progress monitoring across all 38 German university clinics. The monitoring framework supports transparent coordination of distributed research activities and can facilitate more efficient management of large-scale multi-center studies. The source code and Kaapana integration are publicly available at https://github.com/MIC-DKFZ/study-monitoring-kaapana.

Chinese Translation

多中心研究对推动医学和放射学研究至关重要。数据探索、协作发现和研究进展监测是最大化其潜力的关键。然而，在实际操作中，这些过程往往依赖于手动沟通和共享表格，这些表格很快就会过时，从而阻碍大规模分布式研究中的高效协调。这突显了需要专门的监测解决方案，以提供透明且最新的研究进展洞察。我们提出了一种基于广泛使用的Grafana-Prometheus堆栈的轻量级开源监测架构，适用于多中心研究。该框架从分布式研究站点收集聚合监测指标，并通过可配置的仪表板进行可视化。作为一个实际部署的例子，该框架已集成到医学影像平台Kaapana中，并在一个大型多中心研究网络中进行了评估。通过在德国范围内的RACOON联盟中部署我们的解决方案，我们展示了其在所有38所德国大学医院中实现隐私保护的数据探索和研究进展监测的能力。该监测框架支持分布式研究活动的透明协调，并能够促进大规模多中心研究的更高效管理。源代码和Kaapana集成可在https://github.com/MIC-DKFZ/study-monitoring-kaapana公开获取。

View on arXiv Download PDF AI Translation

cs.CV / 241 / 2606.16866

Redirecting the Flow: Image Customization through Attention Distribution Shift

重定向流动：通过注意力分布转移进行图像定制

Li, Jie, Yang, Suorong, Zhao, Jian, Shen, Furao

Abstract

Subject-driven image customization aims to generate images that not only follow textual instructions but also preserve the identity of a given reference subject. Existing approaches, including test-time fine-tuning, encoder-based methods, and token competition in shared attention spaces, suffer from limited efficiency, misalignment between extracted reference features and the generative process, and interference from irrelevant information. To address these limitations, we formulate the customization task as a distribution shift induced by incorporating reference images into text-to-image generation, and derive a Conditional Attention Distribution Shift formulation grounded in maximum entropy theory. Building on this formulation, we propose CustomShift, a dual-branch architecture based on Stable Diffusion 3. The Reference-Alignment Branch leverages self-attention between reference images and subject names to achieve layer-wise alignment with latent representations, while the Cross-Guidance Branch integrates textual and reference cues to guide generation. Experiments on the DreamBooth and Custom101 benchmarks demonstrate that our method consistently outperforms state-of-the-art approaches, achieving a better balance between semantic fidelity and subject consistency.

Chinese Translation

以主体为驱动的图像定制旨在生成不仅遵循文本指令而且保留给定参考主体身份的图像。现有方法，包括测试时微调、基于编码器的方法以及共享注意力空间中的令牌竞争，存在效率有限、提取的参考特征与生成过程之间的错位以及无关信息干扰等问题。为了解决这些局限性，我们将定制任务表述为通过将参考图像融入文本到图像生成中引起的分布转移，并基于最大熵理论推导出条件注意力分布转移的公式。在此基础上，我们提出了CustomShift，这是一种基于Stable Diffusion 3的双分支架构。参考对齐分支利用参考图像与主体名称之间的自注意力，实现与潜在表示的逐层对齐，而交叉引导分支则整合文本和参考线索以指导生成。在DreamBooth和Custom101基准上的实验表明，我们的方法在语义保真度和主体一致性之间取得了更好的平衡，始终优于最先进的方法。

View on arXiv Download PDF AI Translation

cs.CV / 242 / 2606.16868

Federated Medical Image Segmentation under Real-World Label Noise: A Benchmark Suite for Noisy Label Learning Method Selection

在真实世界标签噪声下的联邦医学图像分割：噪声标签学习方法选择的基准套件

Bujotzek, Markus, Bounias, Dimitrios, Denner, Stefan, Floca, Ralf, Fischer, Maximilian, Neher, Peter, Maier-Hein, Klaus

Abstract

While federated learning (FL) enables collaborative medical image segmentation without centralizing sensitive data, real-world deployment is frequently complicated by cross-site label imperfections such as contour disagreement, missing or additional structures, and confused labels. Federated noisy label learning (FNLL) aims to mitigate these effects, yet remains underused in practice as existing evidence is largely based on synthetic noise, simplified settings, and limited real-world noisy evaluation. We address this gap by introducing a benchmark suite that combines diverse real-world noisy datasets, deployment-relevant client-noise scenarios, and label-noise-targeted evaluation to support systematic FNLL assessment and informed method selection. The suite combines curated real-world noisy medical image segmentation datasets from diverse sources with a comprehensive federated segmentation framework including various client-noise scenarios and noise-targeted evaluation. The presented suite provides a realistic and discriminative basis for FNLL evaluation in medical image segmentation and establishes a reusable foundation for fair benchmarking, dataset-specific label-noise characterization, and future method development under realistic federated settings. Code is available at https://github.com/MIC-DKFZ/FedSegNoiseBench.

Chinese Translation

尽管联邦学习（FL）能够在不集中敏感数据的情况下实现协作医学图像分割，但在实际应用中，跨站点标签的不完美性常常使得这一过程复杂化，例如轮廓不一致、缺失或额外结构以及标签混淆。联邦噪声标签学习（FNLL）旨在减轻这些影响，然而由于现有证据主要基于合成噪声、简化设置和有限的真实世界噪声评估，其在实践中仍然使用不足。我们通过引入一个基准套件来填补这一空白，该套件结合了多样的真实世界噪声数据集、与部署相关的客户端噪声场景以及针对标签噪声的评估，以支持系统的FNLL评估和知情的方法选择。该套件将来自不同来源的策划的真实世界噪声医学图像分割数据集与一个全面的联邦分割框架相结合，包括各种客户端噪声场景和针对噪声的评估。所呈现的套件为医学图像分割中的FNLL评估提供了一个现实且具有区分性的基础，并为公平基准测试、数据集特定的标签噪声特征化以及在现实联邦设置下的未来方法开发建立了可重用的基础。代码可在 https://github.com/MIC-DKFZ/FedSegNoiseBench 获取。

View on arXiv Download PDF AI Translation

cs.CV / 243 / 2606.16870

Latent Space Reinforcement Learning for Inverse Material Estimation in Food Fracture Simulation

用于食品断裂模拟的逆材料估计的潜在空间强化学习

Ramlal, Adrian, Chen, Yuhao, Zelek, John S.

Abstract

Realistic visual simulation of food manipulation requires accurate material parameters, yet these are difficult to measure directly and vary across the heterogeneous regions of a single food item. We address the inverse problem of estimating material parameters from a target description of fracture behavior in a non-differentiable continuum damage mechanics simulator. Using orange peeling as a test case, we train a neural surrogate on 2,000 forward simulations and compare Covariance Matrix Adaptation Evolution Strategy (CMA-ES, a gradient-free evolutionary optimizer) with Proximal Policy Optimization (PPO, a reinforcement learning algorithm) across the original 9-dimensional parameter space and two learned 4-dimensional latent representations. Since different oranges have different material properties, a practical inverse system must handle arbitrary targets without retraining. We train a goal-conditioned PPO policy that learns a general inverse mapping: given any target description of peeling behavior, the policy produces a material parameter estimate in a single forward pass (8 surrogate evaluations, approximately 10ms). Operating in a normalizing flow latent space with a shared surrogate evaluator, the goal-conditioned policy achieves 0.642 actual recovery when validated through the simulator, outperforming the original parameter space by 23%. A warm-start extension that initializes CMA-ES refinement from the policy's output further improves recovery to 0.828 with 540 evaluations. These findings provide a practical framework for inverse food physics and lay groundwork for vision-driven material identification from video observations of food manipulation.

Chinese Translation

食品操作的真实视觉模拟需要准确的材料参数，但这些参数难以直接测量，并且在单一食品项目的异质区域中变化。我们解决了从非可微连续损伤力学模拟器中的断裂行为目标描述估计材料参数的逆问题。以橙子剥皮为测试案例，我们在2000次正向模拟中训练了一个神经代理，并比较了协方差矩阵自适应进化策略（Covariance Matrix Adaptation Evolution Strategy, CMA-ES，一种无梯度进化优化器）与近端策略优化（Proximal Policy Optimization, PPO，一种强化学习算法）在原始9维参数空间和两个学习到的4维潜在表示上的表现。由于不同的橙子具有不同的材料特性，实际的逆系统必须能够处理任意目标而无需重新训练。我们训练了一个目标条件的PPO策略，该策略学习了一个通用的逆映射：给定任何剥皮行为的目标描述，该策略在一次正向传递中生成材料参数估计（8次代理评估，约10毫秒）。在具有共享代理评估器的归一化流潜在空间中，目标条件策略在通过模拟器验证时实现了0.642的实际恢复率，比原始参数空间提高了23%。一个温启动扩展从策略输出初始化CMA-ES的精细化，进一步将恢复率提高到0.828，评估次数为540。这些发现为逆食品物理提供了一个实用框架，并为基于视频观察的食品操作中的视觉驱动材料识别奠定了基础。

View on arXiv Download PDF AI Translation

cs.CV / 244 / 2606.16898

Semantic Flip: Synthetic OOD Generation for Robust Refusal in Embodied Question Answering and Spatial Localization

语义翻转：用于增强拒绝能力的合成OOD生成在具身问答和空间定位中的应用

Na, Dongbin, Kim, Chanwoo, Choi, Giyun, Hong, Dooyoung

Abstract

Detecting unanswerable user queries remains essential for the reliable deployment of real-world embodied agents. However, modern vision-language models (VLMs) often generate overly confident answers even when the available visual memory cannot support the query. Such overconfidence poses various task-dependent risks. The agent may provide misleading information to the user in Embodied Question Answering and select an arbitrary coordinate and physically guide the user there in spatial reasoning for navigation. Despite these high stakes, only a few prior studies directly address when and how an embodied VLM should respond with "I do not know." This work proposes Semantic Flip, a simple yet effective framework that synthesizes auxiliary out-of-distribution (OOD) samples for embodied refusal without requiring external OOD annotations. The key idea is to independently transform the query and video memory to construct auxiliary OOD pairs that lack sufficient visual grounding. These synthesized pairs enable training a lightweight rejection module on top of a frozen pretrained VLM. The module attaches to any existing VLM-based pipeline without retraining the underlying model. Across two complementary benchmarks, Semantic Flip consistently outperforms strong prompting baselines. This work also introduces SpaceReject, a new refusal benchmark for spatial localization with deliberately unanswerable queries over long video memory, where Semantic Flip achieves an $F_1$ score of 0.9559. The source codes and datasets are publicly available at https://github.com/ndb796/SemanticFlip.

Chinese Translation

检测无法回答的用户查询对于现实世界具身代理的可靠部署至关重要。然而，现代视觉-语言模型（VLMs）在可用视觉记忆无法支持查询时，往往会生成过于自信的回答。这种过度自信带来了各种依赖任务的风险。在具身问答中，代理可能会向用户提供误导性信息，而在空间推理中则可能选择任意坐标并物理引导用户前往。尽管面临这些高风险，之前的研究仅有少数直接探讨了具身VLM应在何时以及如何回应“我不知道”。本研究提出了语义翻转（Semantic Flip），这是一个简单而有效的框架，能够合成辅助的分布外（OOD）样本，以实现具身拒绝，而无需外部OOD标注。其关键思想是独立地转换查询和视频记忆，以构建缺乏足够视觉基础的辅助OOD对。这些合成对使得在冻结的预训练VLM之上训练一个轻量级拒绝模块成为可能。该模块可以附加到任何现有的基于VLM的管道上，而无需重新训练基础模型。在两个互补基准测试中，语义翻转始终优于强提示基线。此外，本研究还引入了SpaceReject，这是一个新的空间定位拒绝基准，包含故意无法回答的查询，针对长视频记忆，语义翻转在该基准上达到了0.9559的$F_1$分数。源代码和数据集已公开，地址为https://github.com/ndb796/SemanticFlip。

View on arXiv Download PDF AI Translation

cs.CV / 245 / 2606.16951

Simulation-Based Multi-Fillet Evaluation of Woody Breast Poultry Fillets

基于仿真的多鱼片评估木质胸肉禽肉片

Mukherjee, Chirantan Sen, Yoon, Seung-Chul, Beksi, William J.

Abstract

Woody breast (WB) is a myopathy in modern broiler chickens that causes the breast muscle to become unusually stiff and fibrous, leading to decreased meat quality and significant economic losses. State-of-the-art automated WB detection relies on a side-view imaging system to analyze the bending behavior of a single fillet as it falls off a conveyor belt. While highly accurate, this approach is constrained by its single-fillet field of view, creating throughput bottlenecks on commercial processing lines. In this paper, we address this limitation via a novel multi-fillet detection architecture utilizing a top-down camera configuration. To validate our approach, we first develop a high-fidelity digital twin of an industrial conveyor system. Next, we synthesize a diverse dataset of 3D fillet meshes and model their viscoelastic bending dynamics using a physics-based simulation engine. Lastly, a continuous 2D shape deformation score is extracted from the top-down perspective as the simulated fillets traverse the roller precipice. Experimental results demonstrate that the top-down shape score effectively captures the contour changes of the fillets as it bends, providing a robust and scalable alternative to a side-view imaging system for simultaneous multi-fillet WB evaluation.

Chinese Translation

木质胸肉（Woody breast, WB）是一种现代肉鸡的肌肉病，导致胸肌异常僵硬和纤维化，从而降低肉质并造成显著的经济损失。最先进的自动化WB检测依赖于侧视成像系统，通过分析单个鱼片在从输送带上掉落时的弯曲行为来进行检测。尽管这种方法具有很高的准确性，但其单鱼片视野的限制在商业加工线上造成了吞吐量瓶颈。本文通过一种新颖的多鱼片检测架构，利用自上而下的相机配置来解决这一限制。为了验证我们的方法，我们首先开发了一个工业输送系统的高保真数字双胞胎。接下来，我们合成了多样化的3D鱼片网格数据集，并使用基于物理的仿真引擎对其粘弹性弯曲动态进行建模。最后，从自上而下的视角提取了一个连续的2D形状变形评分，作为模拟鱼片穿越滚筒边缘时的表现。实验结果表明，自上而下的形状评分有效捕捉了鱼片在弯曲过程中的轮廓变化，为同时进行多鱼片WB评估提供了一种稳健且可扩展的替代方案。

View on arXiv Download PDF AI Translation

cs.CV / 246 / 2606.16960

SurroundNEXO: Ego-Centric Metric Bridging for Spatially Consistent Geometry in Autonomous Driving

SurroundNEXO：自我中心度量桥接以实现自主驾驶中的空间一致几何

Yuan, Shuai, Tang, Runxi, Ji, Yuzhou, Ge, Fudong, Wang, Hanshi, Wang, Yifei, Zeng, Xianming, Xu, Jianyun, Liu, Xingliang, Wang, Yanfeng, Zhang, Zhipeng

Abstract

Modern autonomous driving depends on accurate metric 3D understanding for perception, reconstruction, and planning, which in turn requires reliable multi-camera depth prediction. However, the outward-facing nature of vehicle-mounted surround-view camera rigs inherently limits visual overlap across views, challenging the correspondence-based assumptions that underpin conventional multi-view geometry. To bridge this gap, we present SurroundNEXO, named after the Spanish word nexo for a geometric link, a low-overlap multi-camera metric depth framework that grounds cross-view reasoning in ego-centric geometry rather than dense visual correspondences. Instead of directly enforcing early global fusion, SurroundNEXO first assigns image tokens globally comparable ego-frame viewing directions through Ego-Ray Positional Encoding, then uses sparse LiDAR measurements as metric anchors to propagate absolute scale cues, and finally expands feature interaction progressively from view-local modeling to decomposed spatio-temporal reasoning and global integration. This design enables metric-scale depth prediction with improved spatial consistency across weakly overlapping cameras. Across low-overlap autonomous driving benchmarks, including NuScenes, Waymo and DDAD, SurroundNEXO reduces single-view error by 33.2%, improves cross-view consistency by 10.5%, and enhances metric reconstruction quality by 25.6% compared with SOTA methods. It further remains robust under extremely sparse depth prompts and exhibits strong zero-shot generalization to unseen camera layouts.

Chinese Translation

现代自主驾驶依赖于准确的三维度量理解，以支持感知、重建和规划，这反过来又需要可靠的多摄像头深度预测。然而，车载环视摄像头系统的外向特性本质上限制了视图之间的视觉重叠，这对传统多视图几何所依赖的对应关系假设提出了挑战。为了解决这一问题，我们提出了SurroundNEXO，命名源于西班牙语中表示几何链接的词汇nexo，这是一种低重叠多摄像头度量深度框架，它将跨视图推理建立在自我中心几何之上，而不是稠密的视觉对应关系。SurroundNEXO并不是直接强制早期的全局融合，而是首先通过自我射线位置编码（Ego-Ray Positional Encoding）为图像标记分配全球可比较的自我框架视角方向，然后使用稀疏的激光雷达测量作为度量锚点来传播绝对尺度线索，最后逐步扩展特征交互，从视图局部建模到分解的时空推理和全局整合。这一设计使得在弱重叠摄像头之间实现度量尺度的深度预测时，空间一致性得到了改善。在包括NuScenes、Waymo和DDAD在内的低重叠自主驾驶基准测试中，SurroundNEXO将单视图误差降低了33.2%，提高了跨视图一致性10.5%，并且与最先进的方法相比，增强了25.6%的度量重建质量。此外，它在极其稀疏的深度提示下仍然保持稳健，并对未见过的摄像头布局表现出强大的零样本泛化能力。

View on arXiv Download PDF AI Translation

cs.CV / 247 / 2606.16991

A Multi-Center Benchmark for Abdominal Disease Diagnosis and Report Generation from Non-Contrast CT

基于非对比CT的腹部疾病诊断与报告生成的多中心基准研究

Elbakry, Mariam, Sheha, Aliaa Sayed, Tantawy, Salma Hassan, Yassin, Aya, Spampinato, Concetto, Lekadir, Karim, Li, Xiaomeng, Elbatel, Marawan

Abstract

Multiphasic contrast-enhanced CT (CECT) is widely used for abdominal lesion characterization, yet it carries inherent risks of contrast-induced nephropathy, escalates acquisition burden, and heavily contributes to radiologist workload. To address these challenges, we introduce a novel multi-center benchmark for multi-organ abdominal disease diagnosis and automated radiology report generation, which learns to synthesize contrast-enhanced findings from single-phase non-contrast CT (NCCT). To support this, we curated a large-scale dataset of paired NCCT-CECT studies and their corresponding contrast-enhanced radiology reports from two centers, partitioned into internal sets and an external validation cohort. Under a unified evaluation protocol, we benchmarked five contemporary deep learning architectures encompassing chest-specific, abdomen-specific, and general-purpose multimodal domains. Extensive experiments demonstrate that NCCT retains diagnostic signals, achieving an average multi-organ AUC of 69.1% on the internal cohort and 63.1% on the external cohort, respectively. By releasing this dataset and standardized benchmark publicly, this study aims to catalyze future research into safer, resource-efficient, and globally accessible contrast-free abdominal imaging workflows. Code is available at: https://github.com/xmed-lab/TriALS-Report.

Chinese Translation

多相对比增强CT（CECT）广泛应用于腹部病变特征化，但其固有的对比剂诱导肾病风险、增加的获取负担以及对放射科医师工作量的重大影响，均是亟待解决的挑战。为应对这些问题，我们提出了一种新颖的多中心基准，用于多脏器腹部疾病诊断和自动化放射学报告生成，旨在从单相非对比CT（NCCT）中学习合成对比增强的发现。为此，我们整理了一个大规模的配对NCCT-CECT研究数据集及其对应的对比增强放射学报告，数据来源于两个中心，并划分为内部数据集和外部验证队列。在统一的评估协议下，我们对五种现代深度学习架构进行了基准测试，这些架构涵盖了胸部特定、腹部特定和通用多模态领域。大量实验表明，NCCT保留了诊断信号，在内部队列上实现了69.1%的多脏器AUC，在外部队列上实现了63.1%的AUC。通过公开发布该数据集和标准化基准，本研究旨在促进未来对更安全、资源高效和全球可及的无对比腹部影像工作流程的研究。代码可在以下地址获取：https://github.com/xmed-lab/TriALS-Report。

View on arXiv Download PDF AI Translation

cs.CV / 248 / 2606.16993

DreamX-World 1.0: A General-Purpose Interactive World Model

DreamX-World 1.0：通用交互式世界模型

DreamX Team, Bai, Yancheng, Chen, Rui, Chu, Xiangxiang, Dang, Rujing, Dou, Hao, Gao, Bingjie, Gu, Qiwen, Hong, Siyu, Lei, Jiachen, Li, Geng, Li, Jifan, Lin, Ruimin, Shi, Qingfeng, Song, Bingze, Sun, Lei, Tang, Jing, Tian, Ruitian, Wang, Jun, Wu, Jiahong, Zhang, Pengfei, Zhang, Shen, Zhu, Jiashu

Abstract

DreamX-World 1.0 is a general-purpose interactive text/image-to-video world model for controllable long-horizon generation. It supports camera navigation, revisits to previously observed regions, and promptable events across photorealistic, game-style, and stylized domains. Our data engine combines camera-accurate Unreal Engine rendering, action-rich gameplay recordings, and real-world videos with recovered camera geometry. For camera control, we introduce E-PRoPE, a lightweight variant of projective positional encoding that retains PRoPE's projective camera geometry while applying camera-aware attention to spatially reduced tokens. We convert a bidirectional video generator into a few-step autoregressive world model using causal forcing, DMD-style distillation, and long-rollout training. Training on self-generated long-horizon contexts exposes the model to its own generated history and reduces the style and color drift that accumulates across autoregressive chunks. Memory-Conditioned Scene Persistence retrieves earlier views through camera-geometry-based retrieval, while residual recycling makes the conditioning path less sensitive to imperfect memory latents. Event Instruction Tuning adds composable event control, and reinforcement learning alignment recovers camera control and visual quality after distillation. With mixed-precision DiT execution, residual reuse, 75\%-pruned VAE decoding, and asynchronous pipeline parallelism, DreamX-World 1.0 reaches up to 16\,FPS on eight RTX\,5090 GPUs. On our 5-second basic evaluation, DreamX-World 1.0 achieves a camera-control score of 73.75 and an overall score of 84.76, outperforming HY-WorldPlay 1.5 and LingBot-World in overall score, which achieve 80.79 and 80.45, respectively.

Chinese Translation

DreamX-World 1.0 是一个通用的交互式文本/图像到视频的世界模型，旨在实现可控的长时间生成。它支持相机导航、重新访问先前观察的区域，以及在照片真实感、游戏风格和风格化领域中的可提示事件。我们的数据引擎结合了相机精确的虚幻引擎渲染、丰富动作的游戏录制和带有恢复相机几何的真实世界视频。为了实现相机控制，我们引入了 E-PRoPE，这是一种轻量级的投影位置编码变体，保留了 PRoPE 的投影相机几何，同时对空间减少的标记应用相机感知注意力。我们将双向视频生成器转换为一个几步自回归世界模型，采用因果强制、DMD 风格的蒸馏和长时间回放训练。在自生成的长时间上下文上进行训练，使模型接触到自身生成的历史，并减少在自回归块中积累的风格和颜色漂移。基于相机几何的检索的记忆条件场景持久性可以检索早期视图，而残差回收使得条件路径对不完美的记忆潜变量的敏感性降低。事件指令调优增加了可组合的事件控制，而强化学习对齐在蒸馏后恢复了相机控制和视觉质量。通过混合精度的 DiT 执行、残差重用、75% 剪枝的变分自编码器解码和异步管道并行，DreamX-World 1.0 在八个 RTX 5090 GPU 上达到了最高 16 帧每秒。在我们的 5 秒基本评估中，DreamX-World 1.0 实现了 73.75 的相机控制得分和 84.76 的总体得分，超越了 HY-WorldPlay 1.5 和 LingBot-World 的总体得分，后者分别为 80.79 和 80.45。

View on arXiv Download PDF AI Translation

cs.CV / 249 / 2606.16996

ActiveSAM: Image-Conditional Class Pruning for Fast and Accurate Open-Vocabulary Segmentation

ActiveSAM：用于快速准确的开放词汇分割的图像条件类修剪

Tien, Tran Dinh, Shen, Zhiqiang

Abstract

Segment Anything Model 3 (SAM 3) provides a strong frozen backbone for concept-prompted segmentation, but applying it directly to open-vocabulary semantic segmentation (OVSS) is inefficient: full-resolution decoding is typically run over the entire dataset vocabulary, whereas each image contains only a small active subset of classes. We introduce ActiveSAM, a training-free, zero-shot inference framework that turns SAM 3 into an active-vocabulary segmenter. ActiveSAM first canonicalizes and expands class prompts, then estimates an image-conditioned active set from a low-resolution presence preview. Only the retained classes are decoded at full resolution, using bucketed prompt multiplexing with the frozen SAM 3 decoder. The preview stage uses only class-presence evidence and skips unnecessary segmentation-head computation, while the final stage applies margin-aware background calibration to suppress low-confidence pixels. ActiveSAM requires no target-dataset training, no weight updates, and no oracle class-presence labels. Across eight OVSS benchmarks, ActiveSAM improves the speed-accuracy tradeoff of training-free open-vocabulary semantic segmentation, outperforming the current state-of-the-art SegEarth-OV3 by approximately +1.4 mIoU on average while running up to 5.5x faster on large-vocabulary datasets. ActiveSAM also demonstrates the strongest robustness under image corruption that simulates real-world distribution shift, making it well-suited for deployment in noisy-input domains such as autonomous driving and embodied AI. Code is available at https://github.com/VILA-Lab/ActiveSAM.

Chinese Translation

Segment Anything Model 3 (SAM 3) 为概念提示分割提供了强大的冻结骨干网络，但直接将其应用于开放词汇语义分割（OVSS）效率低下：全分辨率解码通常在整个数据集词汇上运行，而每幅图像仅包含少量活跃类别。我们提出了 ActiveSAM，这是一种无训练、零样本推理框架，将 SAM 3 转变为活跃词汇分割器。ActiveSAM 首先标准化并扩展类别提示，然后从低分辨率的存在预览中估计图像条件的活跃集合。只有保留的类别在全分辨率下进行解码，使用与冻结的 SAM 3 解码器相结合的分桶提示复用。预览阶段仅使用类别存在证据，跳过不必要的分割头计算，而最终阶段应用边际感知背景校准以抑制低置信度像素。ActiveSAM 不需要目标数据集训练、权重更新和oracle类别存在标签。在八个 OVSS 基准测试中，ActiveSAM 提高了无训练开放词汇语义分割的速度-准确性权衡，平均比当前最先进的 SegEarth-OV3 提高约 +1.4 mIoU，同时在大词汇数据集上运行速度快达 5.5 倍。ActiveSAM 还在模拟真实世界分布变化的图像损坏下表现出最强的鲁棒性，使其非常适合在噪声输入领域（如自动驾驶和具身人工智能）中部署。代码可在 https://github.com/VILA-Lab/ActiveSAM 获取。

View on arXiv Download PDF AI Translation

cs.CV / 250 / 2606.17020

FusionRS: A Large-Scale RGB-Infrared Remote Sensing Dataset for Dual-Modal Vision-Language Foundation Models

FusionRS：一个用于双模态视觉-语言基础模型的大规模RGB-红外遥感数据集

Han, Jiaju, Zhang, Ben, Sun, Xuemeng, Zhang, Qike, Dong, Yuxian, Hu, Chengyin, Zhang, Fengyu, Wei, Yiwei, Guo, Jiujiang

Abstract

Remote sensing vision-language models have advanced Earth observation understanding, but most existing work remains centered on RGB imagery, leaving the complementary information in infrared data underexplored. Infrared images provide distinctive cues, including thermal intensity structures, object boundaries, and illumination-invariant scene features, which can enrich visual-language learning beyond conventional RGB observations. However, a large-scale RGB-infrared-text dataset for remote sensing vision-language modeling is still absent. To address this gap, we introduce FusionRS, the first large-scale RGB-infrared-text dataset designed for dual-modal vision-language learning in remote sensing. FusionRS is constructed by translating diverse public RGB remote sensing images into infrared-style counterparts, forming aligned RGB-IR image pairs. Each pair is associated with conventional scene captions and IR-aware captions that explicitly describe infrared-specific visual properties while preserving semantic content. Based on FusionRS, we train dual-modal vision-language foundation models for RGB-IR joint understanding. We first train CLIP-style models for RGB-IR-text alignment, and then fine-tune generative VLMs for dual-modal RGB-IR captioning. Experiments show that FusionRS improves RGB-IR alignment, infrared-to-text retrieval, and dual-modal captioning over RGB-only and non-IR-aware training settings. Ablation studies further verify that IR-aware captions are crucial for strengthening infrared-language alignment, highlighting the importance of modality-specific textual supervision for more scalable RGB-infrared remote sensing vision-language representation learning.

Chinese Translation

遥感视觉-语言模型推动了对地球观测的理解，但现有大多数研究仍集中于RGB图像，红外数据中的互补信息尚未得到充分探索。红外图像提供了独特的线索，包括热强度结构、物体边界和光照不变的场景特征，这些都可以丰富视觉-语言学习，超越传统的RGB观测。然而，针对遥感视觉-语言建模的大规模RGB-红外-文本数据集仍然缺失。为了解决这一空白，我们推出了FusionRS，这是第一个为遥感中的双模态视觉-语言学习而设计的大规模RGB-红外-文本数据集。FusionRS通过将多样的公共RGB遥感图像转换为红外风格的对应图像来构建，形成对齐的RGB-IR图像对。每对图像都与传统场景标题和红外感知标题相关联，后者明确描述了红外特有的视觉属性，同时保留语义内容。基于FusionRS，我们训练了用于RGB-IR联合理解的双模态视觉-语言基础模型。我们首先训练了CLIP风格的模型以实现RGB-IR-文本对齐，然后微调生成式视觉-语言模型（VLMs）以进行双模态RGB-IR标题生成。实验表明，FusionRS在RGB-IR对齐、红外到文本检索和双模态标题生成方面优于仅RGB和非红外感知的训练设置。消融研究进一步验证了红外感知标题对于增强红外-语言对齐的重要性，突显了模态特定文本监督在更大规模的RGB-红外遥感视觉-语言表示学习中的重要性。

View on arXiv Download PDF AI Translation

cs.CV / 251 / 2606.17027

MeshLoom: Feed-Forward Non-Rigid Registration of Mesh Sequences

MeshLoom：网格序列的前馈非刚性配准

Chen, Jianqi, Yenphraphai, Jiraphon, Tang, Xiangjun, Tulyakov, Sergey, Wang, Chaoyang, Wonka, Peter, Abdal, Rameen

Abstract

We present MeshLoom, a feed-forward registration network that directly reconstructs vertex deformations across mesh sequences. Our approach advances non-rigid registration beyond existing models, which are typically constrained by costly per-instance optimization, narrow object categories, pairwise-only inputs, or merely intermediate outputs. The network is simple and efficient, registering multiple meshes within seconds. At its core lies a topology-aware encoder--decoder design. Specifically, we first introduce a topology-aware point representation that encodes the anchor (reference) mesh's topology into its per-vertex features. This representation strengthens the network's understanding of the anchor-mesh geometry and disambiguates points that are Euclidean-close yet geodesically distant. We then propose a multi-modal encoder that fuses this anchor-mesh representation with complementary cues from each frame, such as shape latents and image features. These multi-source signals are compressed into a compact global motion embedding that captures dense inter-frame correspondence. A lightweight decoder then queries this global embedding with the anchor-mesh point representation, retrieving per-vertex deformations at target timestamps. Through extensive experiments across diverse motions and object categories, we show that MeshLoom achieves state-of-the-art results on non-rigid registration. In addition, we find that our global embedding-then-query paradigm naturally enables the network to generate deformations at intermediate timestamps, which extends MeshLoom to motion interpolation and mesh morphing. Project page: https://meshloom.github.io/ .

Chinese Translation

我们提出了MeshLoom，一种前馈配准网络，能够直接重建网格序列中的顶点变形。我们的方法在现有模型的基础上推进了非刚性配准，这些模型通常受到高昂的每实例优化、狭窄的物体类别、仅限于成对输入或仅提供中间输出的限制。该网络简单高效，能够在几秒钟内注册多个网格。其核心是一个考虑拓扑的编码器-解码器设计。具体而言，我们首先引入了一种考虑拓扑的点表示法，将锚点（参考）网格的拓扑编码到其每个顶点特征中。这种表示增强了网络对锚点网格几何形状的理解，并消除了在欧几里得空间中接近但在测地线上距离较远的点之间的歧义。然后，我们提出了一种多模态编码器，将这种锚点网格表示与每帧的补充线索（如形状潜变量和图像特征）融合。这些多源信号被压缩成一个紧凑的全局运动嵌入，捕捉密集的帧间对应关系。一个轻量级解码器随后使用锚点网格点表示查询这个全局嵌入，从而在目标时间戳检索每个顶点的变形。通过在多种运动和物体类别上的广泛实验，我们展示了MeshLoom在非刚性配准方面达到了最先进的结果。此外，我们发现我们的全局嵌入-再查询范式自然使网络能够在中间时间戳生成变形，这将MeshLoom扩展到运动插值和网格变形。项目页面：https://meshloom.github.io/

View on arXiv Download PDF AI Translation

cs.CV / 252 / 2606.17030

Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation

Qwen-RobotWorld技术报告：通过语言条件的视频生成统一具身世界建模

Zhang, Jie, Chen, Xiaoyue, Chen, Anzhe, Lv, Chenxu, Li, Deqing, Zhou, Gengze, Yin, Hang, Yuan, Haoqi, Li, Haoyang, Li, Jiahao, Zhang, Jiazhao, Zhou, Jingren, Gao, Kaiyuan, Yan, Kun, Jiang, Lihan, Tang, Ningyuan, Lin, Pei, Peng, Qihang, Yin, Shengming, Wu, Tianhe, Yan, Tianyi, Xu, Xiao, Shu, Yan, Zhang, Yanran, Wang, Ye, Wang, Yi, Chen, Yilei, Xu, Yixian, Huang, Yiyang, Chen, Yuxiang, Zhang, Zekai, Wang, Zhendong, Lei, Zhixing, Liang, Zhixuan, Liu, Zihao, Zhou, Zikai, Chen, Xiong-Hui, Wu, Chenfei

Abstract

We introduce Qwen-RobotWorld, a language-conditioned video world model for embodied intelligence. With natural language as a unified action interface, it predicts physically grounded future visual trajectories from current observations across robotic manipulation, autonomous driving, indoor navigation, and human-to-robot transfer. This unified formulation provides three promising application directions: synthetic data generation for policy training augmentation, scalable virtual environments for policy evaluation, and language-guided planning signals for downstream robot control. This is achieved through a three-part design: a) Double-Stream MMDiT with MLLM Action Encoding, where a 60-layer double-stream diffusion transformer couples frozen Qwen2.5-VL semantics with video-VAE latents through layer-wise joint attention; b) Embodied World Knowledge (EWK), an 8.6M video-text corpus (200M+ frames) with action-language mapping over 20+ embodiments and 500+ action categories; and c) General+Expert Progressive Curriculum, a two-stage training strategy that first learns general visual priors and then injects embodied specialization under a shared language interface. Extensive results show strong competitiveness: ranks 1st overall on EWMBench and DreamGen Bench, outperforms all open-source models on WorldModelBench and PBench. Additional zero-shot analyses on RoboTwin-IF benchmark further support robust generalization and multi-view consistency.

Chinese Translation

我们介绍了Qwen-RobotWorld，这是一种用于具身智能的语言条件视频世界模型。通过自然语言作为统一的动作接口，它能够从当前观察中预测物理基础的未来视觉轨迹，涵盖机器人操作、自动驾驶、室内导航和人机转移等领域。这种统一的公式化提供了三个有前景的应用方向：用于策略训练增强的合成数据生成、用于策略评估的可扩展虚拟环境，以及用于下游机器人控制的语言引导规划信号。这是通过三部分设计实现的：a) 双流MMDiT与MLLM动作编码，其中一个60层的双流扩散变换器通过层级联合注意力将冻结的Qwen2.5-VL语义与视频-VAE潜变量相结合；b) 具身世界知识（EWK），一个包含860万视频-文本语料库（超过2亿帧），在20多个具身体和500多个动作类别上进行动作-语言映射；c) 通用+专家渐进课程，一种两阶段训练策略，首先学习通用视觉先验，然后在共享语言接口下注入具身专业化。大量结果显示出强大的竞争力：在EWMBench和DreamGen Bench上总排名第一，在WorldModelBench和PBench上超越所有开源模型。对RoboTwin-IF基准的额外零-shot分析进一步支持了强大的泛化能力和多视角一致性。

View on arXiv Download PDF AI Translation

cs.CV / 253 / 2606.17037

The Importance of Phase in Neural Representations: An Internal Oppenheim-Lim Test of Image Classifiers

相位在神经表征中的重要性：图像分类器的内部Oppenheim-Lim测试

Yıldırım, Alper

Abstract

Oppenheim and Lim (1981) showed that natural images stay recognizable when reconstructed from their Fourier phase alone, while the magnitude carries little of their identity. We ask whether trained image classifiers reproduce this asymmetry inside their hidden layers, and we test it causally: given two images, we transplant the phase of one onto the magnitude of the other at a chosen layer and record which image the prediction follows. In PRISM2D, GFNet, and ViT-B/16 the prediction follows the phase or sign donor, and deleting all image-specific magnitude barely moves accuracy, so identity rides on phase while image-specific magnitude is largely dispensable to the readout. ResNet-50 at first seems to break the pattern, because transplanting sign after its ReLUs does nothing; a fair intervention before the ReLU reveals a strong latent sign code in the late blocks, and a DC-only control shows the readout consumes a channel-wise spatial average. Controls rule out the trivial case in which magnitude simply stops depending on the image. The architectures therefore share a phase/sign identity code but expose it in different bases, set by rectification and readout geometry, which gives a mechanistic account of the texture--shape gap between CNNs and attention models.

Chinese Translation

Oppenheim和Lim（1981）显示，自然图像在仅从其傅里叶相位重建时仍然可识别，而幅度几乎不携带其身份信息。我们探讨经过训练的图像分类器是否在其隐藏层中重现这种不对称性，并进行因果测试：给定两幅图像，我们将一幅的相位移植到另一幅的幅度上，并记录预测跟随哪幅图像。在PRISM2D、GFNet和ViT-B/16中，预测跟随相位或符号捐赠者，删除所有特定于图像的幅度几乎不会影响准确性，因此身份依赖于相位，而特定于图像的幅度在读取过程中大多是可有可无的。ResNet-50起初似乎打破了这一模式，因为在其ReLU之后移植符号没有任何效果；在ReLU之前进行的公平干预揭示了晚期块中存在强烈的潜在符号编码，而仅使用直流（DC）的控制显示读取过程消耗了通道方向的空间平均值。控制实验排除了幅度简单地不再依赖于图像的平凡情况。因此，这些架构共享相位/符号身份编码，但在不同的基础上暴露出来，这由整流和读取几何形状设定，提供了CNN和注意力模型之间纹理与形状差距的机械解释。

View on arXiv Download PDF AI Translation

cs.CV / 254 / 2606.17049

BRDFusion: Physics Meets Generation for Urban Scene Inverse Rendering

BRDFusion：物理与生成相结合的城市场景逆向渲染

Liu, Yi-Ruei, Lee, Jie-Ying, Huang, Zheng-Hui, Liu, Yu-Lun, Lin, Chih-Hao

Abstract

Inverse rendering of urban scenes from captured videos enables numerous applications, including content creation and autonomous driving simulation. Physically-based rendering methods follow and control lighting physics, but suffer from reconstruction and rendering artifacts. While generative models produce realistic videos, they offer limited consistency and controllability. We present BRDFusion, a unified framework that combines two complementary models for inverse and forward rendering. Specifically, BRDFusion recovers explicit, consistent scene properties with physical modeling and alleviates optimization ambiguity with generative priors. During forward rendering, the physical model provides controllable rendering from the scene configuration, and the generative model denoises and fixes artifacts. Therefore, our method produces high-quality videos while allowing precise control, outperforming baselines in real and synthetic scenes. Moreover, BRDFusion supports novel-view relighting, night simulation, and dynamic object insertion/editing. Project page: https://shigon255.github.io/brdfusion-page/

Chinese Translation

从捕获的视频中进行城市场景的逆向渲染能够实现多种应用，包括内容创作和自动驾驶模拟。基于物理的渲染方法遵循并控制光照物理，但在重建和渲染过程中会出现伪影和渲染瑕疵。虽然生成模型能够生成逼真的视频，但其一致性和可控性有限。我们提出了BRDFusion，一个将两种互补模型结合用于逆向和正向渲染的统一框架。具体而言，BRDFusion通过物理建模恢复明确且一致的场景属性，并通过生成先验缓解优化歧义。在正向渲染过程中，物理模型根据场景配置提供可控的渲染，而生成模型则去噪并修复瑕疵。因此，我们的方法能够生成高质量的视频，同时允许精确控制，在真实和合成场景中超越基线。此外，BRDFusion支持新视角重光照、夜间模拟以及动态物体的插入/编辑。项目页面：https://shigon255.github.io/brdfusion-page/

View on arXiv Download PDF AI Translation

人工智能 (Artificial Intelligence)

123

cs.AI / 1 / 2606.14838

A Definition of Good Explanations and the Challenges Explaining LLM Outputs

良好解释的定义及解释大型语言模型输出的挑战

Mahon, Louis, Ford, Elliot, Hackett, Callum

Abstract

How to define a good explanation is a long-standing philosophical debate which has found recent renewed interest in the context of AI outputs. Explainability is crucial for AI adoption in many contexts, but in order to produce good explanations of AI systems, we must first have an understanding of what good explanations are. In this paper we propose a definition inspired by the notion of counterfactual explanations, however we argue that one must also take into account the interlocutor's prior beliefs in each fact that could be offered in an explanation. We explore the ramifications of this definition for AI explainability and, in particular, why LLM outputs are difficult to produce good explanations for.

Chinese Translation

如何定义良好的解释是一个长期以来的哲学辩论，最近在人工智能输出的背景下重新引起了关注。可解释性在许多场景中对人工智能的采用至关重要，但为了产生良好的人工智能系统解释，我们必须首先理解什么是良好的解释。本文提出了一种受反事实解释（counterfactual explanations）概念启发的定义，然而我们认为还必须考虑到对话者对每个可以在解释中提供的事实的先前信念。我们探讨了这一定义对人工智能可解释性的影响，特别是为什么大型语言模型（LLM）的输出难以产生良好的解释。

View on arXiv Download PDF AI Translation

cs.AI / 2 / 2606.14885

Dr-DCI: Scaling Direct Corpus Interaction via Dynamic Workspace Expansion

DR-DCI：通过动态工作区扩展实现直接语料库交互的规模化

Lu, Yi, Li, Zhuofeng, Nie, Ping, Zhang, Haoxiang, Zhang, Yuyu, Zou, Kai, Chen, Wenhu, Lin, Jimmy, Jiang, Dongfu, Zhang, Yu

Abstract

Agentic search over large corpora relies on retriever-mediated interfaces (e.g., BM25 or ColBERT) for scalable candidate discovery. While effective at ranking relevant documents, these interfaces expose evidence only as ranked results or bounded document views, limiting agents' ability to reorganize material and verify constraints across documents. Direct Corpus Interaction (DCI) addresses this limitation by exposing shell-executable corpus operations for flexible search, filtering, comparison, and verification. However, full-corpus terminal commands become slow and unstable as the corpus grows, degrading performance and efficiency. We introduce DR-DCI, a retriever-steered DCI framework that treats retrieval as an agent-callable action for expanding a local workspace. Rather than operating directly over the full corpus, the agent dynamically pulls relevant documents into an evolving workspace and conducts DCI operations within it. This design combines retriever-level recall with DCI-style precision: retrieval keeps exploration scalable, while DCI preserves the local operations needed for effective evidence resolution. Experiments show that DR-DCI is both effective and efficient across scales. On Browsecomp-Plus, DR-DCI reaches 71.2\% accuracy, improving over raw DCI and ablated variants by up to 8.3 points while reducing tool usage, wall time, and estimated cost. With workspace-preserving context reset, accuracy further improves to 73.3\%. In corpus-scaling experiments, DR-DCI remains effective from 100K to 10M documents, whereas raw DCI becomes unstable and BM25 performs substantially worse. DR-DCI also scales to a 20M-scale file-per-document Wiki-18 QA setting, achieving an average score of 63.0 across six benchmarks and outperforming retrieval-based and trained search-agent baselines. Ablation analysis further shows that ranked previews and inter-document DCI are key to performance.

Chinese Translation

代理搜索大规模语料库依赖于检索器介导的接口（例如，BM25或ColBERT）来实现可扩展的候选发现。尽管这些接口在排名相关文档方面有效，但它们仅以排名结果或有限的文档视图展示证据，限制了代理重新组织材料和验证跨文档约束的能力。直接语料库交互（DCI）通过暴露可执行的语料库操作来解决这一限制，以实现灵活的搜索、过滤、比较和验证。然而，随着语料库的增长，完整语料库的终端命令变得缓慢且不稳定，从而降低了性能和效率。我们提出了DR-DCI，一个由检索器引导的DCI框架，将检索视为可由代理调用的操作，以扩展本地工作区。代理不是直接在完整语料库上操作，而是动态地将相关文档拉入一个不断演变的工作区，并在其中进行DCI操作。这种设计结合了检索器级别的召回与DCI风格的精确性：检索保持探索的可扩展性，而DCI则保留了有效证据解决所需的本地操作。实验表明，DR-DCI在各个规模上都有效且高效。在Browsecomp-Plus上，DR-DCI达到了71.2%的准确率，比原始DCI和消融变体提高了多达8.3个百分点，同时减少了工具使用、墙时和估计成本。通过保持工作区的上下文重置，准确率进一步提高至73.3%。在语料库扩展实验中，DR-DCI在从100K到10M文档的情况下仍然有效，而原始DCI变得不稳定，BM25的表现明显更差。DR-DCI还扩展到20M规模的逐文档Wiki-18问答设置，在六个基准测试中平均得分为63.0，优于基于检索和训练的搜索代理基线。消融分析进一步表明，排名预览和跨文档DCI是性能的关键。

View on arXiv Download PDF AI Translation

cs.AI / 3 / 2606.14892

Relational Structural Causal Models

关系结构因果模型

Ejaz, Adiba, Bareinboim, Elias

Abstract

An artificial intelligence must have a model of its environment that is causal, supporting reasoning about interventions and counterfactuals, and also combinatorial, supporting generalization to unseen combinations of objects. In this work, we formally study when and how such a model can be learned. We develop relational structural causal models, extending structural causal models (Pearl 2009) to settings where objects and their relations vary. First, we show how answers to not only causal but also observational queries about unseen combinations of objects can not be identified without further assumptions. To enable such identification--including in the presence of unobserved confounding--we define relational causal graphs and derive symbolic identification criteria. Finally, we propose relational neural causal models, a provably correct approach that outperforms non-relational baselines on simulated traffic scenes with varying cars, signals, and pedestrians.

Chinese Translation

人工智能必须具有一个因果模型，以支持对干预和反事实的推理，并且还必须是组合性的，以支持对未见对象组合的泛化。在本研究中，我们正式探讨了何时以及如何可以学习这样的模型。我们发展了关系结构因果模型，将结构因果模型（Pearl 2009）扩展到对象及其关系变化的环境中。首先，我们展示了关于未见对象组合的因果查询和观察查询的答案在没有进一步假设的情况下是无法识别的。为了实现这种识别——包括在存在未观察到的混杂因素的情况下——我们定义了关系因果图并推导了符号识别标准。最后，我们提出了关系神经因果模型，这是一种可证明正确的方法，在模拟交通场景中，表现优于非关系基线，场景中包含变化的汽车、信号和行人。

View on arXiv Download PDF AI Translation

cs.AI / 4 / 2606.14923

Trust Between AI Agents: Measuring Formation, Breakage, and Recovery, with Implications for Governing Multi-Agent Systems

人工智能代理之间的信任：形成、破裂与恢复的测量及其对多代理系统治理的影响

Chen, Yujiao

Abstract

As language-model agents increasingly work in teams, each agent must decide how much to trust its teammates. Yet we lack a standard way to measure trust between AI agents. We propose a behavioral measure based on costly verification. In a cooperative survival game, checking a teammate's work consumes resources, while trusting a wrong answer can be fatal. Relative to a memoryless version of the same model, reduced verification provides an observable measure of trust. Using this framework, we study trust formation, breakage, and recovery across six frontier model snapshots. When paired with a consistently reliable teammate, four snapshots (Claude Opus 4.6, Claude Sonnet 4.6, GPT-5.1, and Gemini 3.1 Pro) reduce verification by roughly 60-85%, whereas two smaller snapshots show little or no such adjustment. Failures reverse this discount, but models differ in how they respond. Some concentrate renewed scrutiny on the culprit, while others become more cautious toward the entire team. Recovery is slower than formation, and clustered failures sustain suspicion far longer than the same number of failures spread apart. These differences have practical consequences. Models that form trust verify less, decide more quickly, and achieve higher payoffs in our environment. By contrast, persistent over-verification is associated with indecision rather than safety. Our results show that trust dispositions can be measured before deployment and suggest that calibration, rather than maximal suspicion, should be the central concern in the governance of multi-agent AI systems.

Chinese Translation

随着语言模型代理越来越多地在团队中工作，每个代理必须决定对其队友的信任程度。然而，我们缺乏一种标准的方法来测量人工智能代理之间的信任。我们提出了一种基于高成本验证的行为测量方法。在一个合作生存游戏中，检查队友的工作会消耗资源，而信任错误答案可能是致命的。与同一模型的无记忆版本相比，减少验证提供了一个可观察的信任测量。利用这一框架，我们研究了六个前沿模型快照中的信任形成、破裂和恢复。当与一个始终可靠的队友配对时，四个快照（Claude Opus 4.6、Claude Sonnet 4.6、GPT-5.1 和 Gemini 3.1 Pro）将验证减少了大约60-85%，而两个较小的快照则几乎没有这样的调整。失败会逆转这种折扣，但模型在响应方式上存在差异。有些模型将重新审查的焦点集中在罪魁祸首身上，而其他模型则对整个团队变得更加谨慎。恢复的速度慢于形成，集中的失败会比同样数量的分散失败更长时间地维持怀疑。这些差异具有实际意义。形成信任的模型验证较少，决策更快，并在我们的环境中获得更高的收益。相比之下，持续的过度验证与优柔寡断而非安全相关。我们的结果表明，信任倾向可以在部署前进行测量，并建议在多代理人工智能系统的治理中，校准而非最大怀疑应成为中心关注点。

View on arXiv Download PDF AI Translation

cs.AI / 5 / 2606.14935

PrologMCP: A Standardized Prolog Tool Interface for LLM Agents

PrologMCP：面向LLM代理的标准化Prolog工具接口

Mensfelt, Agnieszka, Prabhakaran, Adarsh, Haret, Adrian, Trencsenyi, Vince, Stathis, Kostas

Abstract

Frontier reasoning-tuned language models still fail on deductive tasks at depth, and the cost of improved performance through extended internal reasoning scales poorly. Symbolic delegation offers a complementary route: a language model translates the problem, while a solver performs the inference. However, current autoformalization pipelines for logic programming are typically bespoke integrations tied to particular tasks or agents. We introduce PrologMCP, a task-agnostic, open-source server that exposes Prolog as a stateful tool through the Model Context Protocol (MCP). Its compact tool interface, structured error reporting, and per-session isolation make the translate-run-inspect-repair loop a reusable primitive for MCP-capable agents. We evaluate a formalizer agent enhanced with PrologMCP against standard and reasoning LLMs (Claude Sonnet 4.6, GPT-4.1, and o4-mini) on two subsets of PARARULE-Plus: a general-purpose sample and a more challenging one targeting a specific failure mode of natural-language reasoning. On the general sample, the formalizer matches or exceeds reasoning LLMs (accuracy 1.00 vs.\ 1.00 / 0.998), with the largest gains over standard models (0.762 for GPT-4.1). On the challenging subset, the formalizer remains near-perfect (1.00 / 0.99) while reasoning LLMs drop to 0.95 / 0.94. These results suggest that delegating inference to Prolog via MCP is a robust and inspectable alternative to extended natural-language reasoning.

Chinese Translation

尽管前沿的推理调优语言模型在深度推理任务上仍然表现不佳，通过扩展内部推理来提高性能的成本也呈现出不良的规模效应。符号委托提供了一条互补的路径：语言模型翻译问题，而求解器执行推理。然而，当前的逻辑编程自动形式化管道通常是与特定任务或代理紧密结合的定制集成。我们提出了PrologMCP，一个与任务无关的开源服务器，通过模型上下文协议（Model Context Protocol, MCP）将Prolog作为一种有状态工具暴露出来。其紧凑的工具接口、结构化的错误报告和每会话的隔离使得翻译-运行-检查-修复循环成为MCP能力代理的可重用原语。我们评估了一个增强了PrologMCP的形式化代理，与标准和推理语言模型（Claude Sonnet 4.6、GPT-4.1和o4-mini）在PARARULE-Plus的两个子集上进行比较：一个通用样本和一个针对自然语言推理特定失败模式的更具挑战性的样本。在通用样本中，形式化代理的表现与推理语言模型相当或更好（准确率1.00对比1.00 / 0.998），在标准模型上获得了最大的提升（GPT-4.1为0.762）。在具有挑战性的子集中，形式化代理保持接近完美（1.00 / 0.99），而推理语言模型下降至0.95 / 0.94。这些结果表明，通过MCP将推理委托给Prolog是一种稳健且可检查的替代方案，用于扩展自然语言推理。

View on arXiv Download PDF AI Translation

cs.AI / 6 / 2606.14941

Semantics-Enhanced Retrieval-Augmented Time Series Forecasting

语义增强的检索增强时间序列预测

Zhou, Shiqiao, Wu, Zipeng, Schöner, Holger, Fouché, Edouard, Wilson, IAG, Wang, Shuo

Abstract

Time series forecasting models often benefit from historical patterns. Inspired by Retrieval-Augmented Generation (RAG), recent research explored retrieving relevant historical time series segments to enhance forecasting. However, relying solely on time series similarity is often insufficient for retrieval under non-stationarity. To address this, we propose a multimodal approach: a \textbf{S}emantics-\textbf{E}nhanced \textbf{R}etrieval-\textbf{A}ugmented Time Series \textbf{F}orecasting framework, SERAF. Unlike mainstream approaches that depend only on time series similarity, SERAF conducts dual retrieval over the time series and their self-generated textual descriptions. It retrieves two complementary sets of historical patterns and corresponding futures, which are selectively and jointly used to guide future predictions. Experiments across seven real-world datasets demonstrate the effectiveness of SERAF in bridging numerical and semantic views of time series compared with state-of-the-art baselines.

Chinese Translation

时间序列预测模型通常受益于历史模式。受到检索增强生成（Retrieval-Augmented Generation, RAG）的启发，最近的研究探索了检索相关历史时间序列片段以增强预测。然而，仅依赖时间序列相似性在非平稳性条件下往往不足以进行有效检索。为了解决这个问题，我们提出了一种多模态方法： extbf{语}义- extbf{增}强的 extbf{检}索- extbf{增}强时间序列 extbf{预}测框架，SERAF。与仅依赖时间序列相似性的主流方法不同，SERAF在时间序列及其自生成的文本描述上进行双重检索。它检索两组互补的历史模式及其对应的未来，这些模式被选择性地联合使用，以指导未来的预测。在七个真实世界数据集上的实验表明，SERAF在连接时间序列的数值视角和语义视角方面的有效性，优于最先进的基线方法。

View on arXiv Download PDF AI Translation

cs.AI / 7 / 2606.14997

AI Engram: In Search of Memory Traces in Artificial Intelligence

人工智能的记忆痕迹：AI Engram的探索

Kwon, Jea, Kim, Dong-Kyum, Kim, Jiwon, Kim, Yonghyun, Kook, Woong, Cha, Meeyoung

Abstract

Memory formation is fundamental to intelligence, yet whether deep neural networks preserve identifiable memory traces analogous to biological memory units remains an open question. This work introduces a geometric framework to identify such "AI engrams" by formalizing the neuroscientific criteria of specificity, reactivation, sufficiency, and necessity into a constrained inverse problem. We derive a closed-form estimator that isolates individual memory traces from globally entangled parameters, and show that this biologically-derived solution corresponds to a natural gradient update on the parameter manifold. AI engrams enable surgical manipulation of learned knowledge: any subset of memories can be composed or erased through linear arithmetic, without iterative optimization. Experiments ranging from simple MLPs to LLMs demonstrate the causal validity and substantial scalability of AI engrams. Together, these results bridge theories of biological memory and artificial representation learning and offer geometric insight into how deep networks simultaneously support functional specificity within distributed storage.

Chinese Translation

记忆的形成是智能的基础，但深度神经网络是否保留可识别的记忆痕迹，类似于生物记忆单元，仍然是一个未解的问题。本研究引入了一个几何框架，通过将神经科学的特异性、再激活、充分性和必要性等标准形式化为一个受限的逆问题，以识别这些“AI engrams”。我们推导出一个闭式估计量，能够从全局纠缠的参数中孤立出单个记忆痕迹，并展示这一生物学导出的解决方案对应于参数流形上的自然梯度更新。AI engrams使得对学习知识的精确操控成为可能：任何记忆的子集都可以通过线性算术进行组合或删除，而无需迭代优化。从简单的多层感知器（MLPs）到大型语言模型（LLMs）的实验验证了AI engrams的因果有效性和显著可扩展性。这些结果共同架起了生物记忆理论与人工表示学习之间的桥梁，并提供了几何视角，揭示深度网络如何在分布式存储中同时支持功能特异性。

View on arXiv Download PDF AI Translation

cs.AI / 8 / 2606.15029

Metric Match: A Subset Selection Approach to Evaluating LLM Judge Reliability

度量匹配：一种评估大型语言模型评审可靠性的子集选择方法

Unell, Alyssa, Dullerud, Natalie, Boneh, Naomi, Jagadeesan, Meena, Hashimoto, Tatsu, Shah, Nigam, Koyejo, Sanmi

Abstract

LLM judges are used to reduce the need for costly human labor in evaluating open-ended text generation. However, the reliability of these judges depends critically on their alignment with human raters -- a property that itself depends on costly human annotations. In this work, we develop a method (Metric Match) for estimating correlation-based reliability metrics of LLM judges from limited annotations. Metric Match selects a subset of samples for human annotation such that the subset matches the population reliability metric with respect to acquired synthetic labels. We empirically show that Metric Match achieves a win-rate of 0.838 against random subset selection across four different correlation metrics and 15 datasets, with an 18.7% decrease in average estimation error and reduces annotation needs by 32.5%. We provide a cost model and highlight a medical case study where our method saves $1,041.67 compared to random selection for expert annotation. Further, we shift our task from reliability estimation to reliability classification of whether a given judge is above a deployment threshold, outperforming random selection with Metric Match. All project code is publicly available, and we additionally provide an installable package for ease of use.

Chinese Translation

大型语言模型（LLM）评审用于减少评估开放式文本生成所需的昂贵人力劳动。然而，这些评审的可靠性在很大程度上依赖于它们与人类评分者的一致性——这一特性本身又依赖于昂贵的人类注释。在本研究中，我们开发了一种方法（Metric Match），用于从有限的注释中估计基于相关性的LLM评审可靠性指标。Metric Match选择一组样本进行人类注释，使得该子集在获得的合成标签方面与总体可靠性指标相匹配。我们通过实证研究表明，Metric Match在四种不同的相关性指标和15个数据集上相较于随机子集选择实现了0.838的胜率，平均估计误差减少了18.7%，并减少了32.5%的注释需求。我们提供了一个成本模型，并强调了一个医学案例研究，其中我们的方法相比随机选择为专家注释节省了1,041.67美元。此外，我们将任务从可靠性估计转向可靠性分类，以判断给定评审是否超过部署阈值，Metric Match在此方面优于随机选择。所有项目代码均已公开，并且我们还提供了一个可安装的软件包以便于使用。

View on arXiv Download PDF AI Translation

cs.AI / 9 / 2606.15034

OSGuard: A Benchmark for Safety in Computer-Use Agents

OSGuard：计算机使用代理安全性的基准测试

Mohammadmirzaei, Mina, Flanigan, Jeffrey

Abstract

Computer-use agents are increasingly evaluated by whether they complete realistic desktop and web tasks. However, task success alone can miss failures in which an agent reaches the nominal goal through an unsafe shortcut. We introduce OSGuard, a dual-granularity benchmark suite for evaluating safety in computer-use agents under benign, unchanged user instructions. OSGuard contains an action-level benchmark for local guardrail decisions and a risk-augmented execution suite for end-to-end evaluation. The action-level benchmark consists of contextualized proposed actions labeled as allowed, unrelated, or unsafe, each judged relative to the original instruction and current interface state. The execution suite contains manually constructed OSWorld-derived task variants in which the original task remains achievable, but the environment is modified to introduce latent hazards such as destructive overwrites, etc. Each variant is paired with augmented evaluators that retain the original task-success criterion while adding explicit state-based safety invariants, allowing us to distinguish safe completions from unsafe completions that satisfy the nominal task objective. Our experimental results on OSGuard show that current multimodal guardrails can perform well on isolated action judgments, while risk-augmented execution exposes remaining gaps between local oversight and reliable end-to-end safety. This dual-granularity design enables more precise diagnosis of whether models can both recognize unsafe proposed actions and improve full-task safety when deployed as guardrails.

Chinese Translation

计算机使用代理的评估越来越多地基于它们是否能够完成现实的桌面和网络任务。然而，仅仅依靠任务成功可能会忽视代理通过不安全的捷径达到名义目标的失败。我们引入了OSGuard，一个用于在友好且不变的用户指令下评估计算机使用代理安全性的双粒度基准套件。OSGuard包含一个用于本地保护决策的行动级基准和一个用于端到端评估的风险增强执行套件。行动级基准由上下文化的建议行动组成，这些行动被标记为允许、不相关或不安全，每个行动都是相对于原始指令和当前界面状态进行判断的。执行套件包含手动构建的源自OSWorld的任务变体，其中原始任务仍然可以实现，但环境被修改以引入潜在的危险，例如破坏性覆盖等。每个变体都配有增强评估器，这些评估器保留了原始任务成功标准，同时增加了明确的基于状态的安全不变性，使我们能够区分安全完成和满足名义任务目标的不安全完成。我们在OSGuard上的实验结果表明，当前的多模态保护措施在孤立的行动判断上表现良好，而风险增强执行则暴露了本地监督与可靠端到端安全之间的剩余差距。这种双粒度设计使我们能够更精确地诊断模型是否能够识别不安全的建议行动，并在作为保护措施部署时改善完整任务的安全性。

View on arXiv Download PDF AI Translation

cs.AI / 10 / 2606.15038

Fusion is not one-size-fits-all: Cross-Modal Representation Alignment for Time-to-Event Modeling

融合并非一刀切：用于事件时间建模的跨模态表示对齐

Zhang, Zhemin, Chen, Weijie, Le, David, Tariq, Amara, Wallace, Alex, Stib, Matthew, Farina, Juan Maria, Ayoub, Chadi, Arsanjani, Reza, Banerjee, Imon

Abstract

Accurate time-to-event (TTE) prediction from multimodal clinical data remains challenging due to modality imbalance and distribution shift. We introduce a foundation model-driven framework for cross-modal representation alignment between CT imaging and longitudinal EHR data, designed to generalize across tasks and institutions. CT and EHR modalities are encoded independently using domain-specific foundation models and aligned in a shared latent space through four principled fusion strategies: late fusion, contrastive alignment, cross-attention, and co-attention. We evaluate two clinically distinct TTE tasks: pulmonary embolism (PE) mortality and cardiovascular disease (CVD) outcomes, on large-scale multi-institutional cohorts (PE: N=3,099 train; 1,098 internal; 435 external; CVD: N=2,951 train; 837 internal; 682 external). Fusion consistently improves concordance index by 1.5-5.4% over unimodal baselines when modalities contribute comparably. Overall, contrastive multimodal fusion, particularly with CLMBR representations, provided the most consistent and statistically robust improvements, especially for PE mortality prediction. For MACE, cross-attention (one-hot) achieved the highest internal performance and image-guided co-attention achieved the best external performance. We therefore introduce a generalizable foundation model-based cross-modal alignment framework and provide the first systematic analysis of fusion behavior under modality imbalance in TTE prediction. Our results establish task-aware multimodal alignment as a necessary design principle for robust generalization and scalable clinical deployment.

Chinese Translation

从多模态临床数据中准确预测事件时间（TTE）仍然面临挑战，主要由于模态不平衡和分布转移。我们提出了一种基于基础模型的框架，用于CT影像与纵向电子健康记录（EHR）数据之间的跨模态表示对齐，旨在跨任务和机构进行泛化。CT和EHR模态分别使用特定领域的基础模型进行编码，并通过四种原则性融合策略在共享潜在空间中对齐：晚期融合、对比对齐、交叉注意力和共同注意力。我们在大规模多机构队列上评估了两个临床上不同的TTE任务：肺栓塞（PE）死亡率和心血管疾病（CVD）结果（PE：N=3,099训练；1,098内部；435外部；CVD：N=2,951训练；837内部；682外部）。当模态贡献相当时，融合始终将一致性指数提高了1.5-5.4%，相较于单模态基线。总体而言，对比多模态融合，特别是使用CLMBR表示，提供了最一致和统计上稳健的改进，尤其是在PE死亡率预测方面。对于主要不良心血管事件（MACE），交叉注意力（one-hot）实现了最高的内部性能，而图像引导的共同注意力则实现了最佳的外部性能。因此，我们引入了一种可泛化的基于基础模型的跨模态对齐框架，并首次系统性地分析了在TTE预测中模态不平衡下的融合行为。我们的结果确立了任务感知的多模态对齐作为稳健泛化和可扩展临床部署的必要设计原则。

View on arXiv Download PDF AI Translation

cs.AI / 11 / 2606.15077

Risk-Aware LLM Agents for Geospatial Data Retrieval: Design and Preliminary Adversarial Evaluation

面向地理空间数据检索的风险感知大型语言模型代理：设计与初步对抗性评估

Gao, Kyle, Cumming, Joel, Li, Jonathan, Xu, Linlin, Clausi, David A.

Abstract

We present an LLM-driven framework for retrieving remote sensing data from cloud-based geospatial catalogues using natural language queries. The system converts user intent into structured API calls, enabling efficient access to satellite imagery and environmental datasets. The architecture integrates three agents: Guardrail for safety and policy enforcement, General-QA for intent interpretation, and Recommender-Analyst for schema-aware API call generation. This coordinated design ensures reliable, semantically aligned interaction with external data services. The modular framework is portable across platforms through API schema substitution and supports applications in environmental monitoring, disaster response, and climate analysis. It establishes a scalable interface between user intent and geospatial infrastructure, enabling streamlined and automated Earth observation workflows. Preliminary experiments under adversarial multi-turn settings show that prompt-level safety instructions improve robustness, although rare high-impact failures persist in API manipulation scenarios and highlight the need for adaptive, system-level defenses that balance safety, usability, and cost efficiency, which motivates the use of our intercept-level Guardrail agent.

Chinese Translation

我们提出了一个基于大型语言模型（LLM）的框架，用于通过自然语言查询从云端地理空间目录中检索遥感数据。该系统将用户意图转换为结构化的API调用，从而实现对卫星图像和环境数据集的高效访问。架构集成了三个代理：Guardrail（安全与政策执行）、General-QA（意图解释）和Recommender-Analyst（基于模式的API调用生成）。这种协调设计确保了与外部数据服务的可靠且语义对齐的交互。该模块化框架通过API模式替换在各个平台上可移植，并支持环境监测、灾害响应和气候分析等应用。它在用户意图与地理空间基础设施之间建立了可扩展的接口，实现了简化和自动化的地球观测工作流程。在对抗性多轮设置下的初步实验表明，提示级安全指令提高了系统的鲁棒性，尽管在API操控场景中仍然存在少数高影响的失败，这突显了需要适应性、系统级防御的必要性，以平衡安全性、可用性和成本效率，这也促使我们使用拦截级Guardrail代理。

View on arXiv Download PDF AI Translation

cs.AI / 12 / 2606.15078

Cognitive Debt: AI as Intellectual Leverage and the Dynamics of Systemic Fragility

认知债务：人工智能作为智力杠杆与系统脆弱性的动态

Meng, Shuchen

Abstract

We develop a formal theory of cognitive debt: the stock of unverified reasoning obligations that accumulates when individuals use AI as a substitute rather than a complement for first-principles cognition. The model features two state variables per agent, cognitive capital and cognitive debt, and a multiplicative production technology in which cognitive capital functions as collateral that determines the return to AI adoption. We establish six propositions. Rational agents incur positive cognitive debt because the costs are deferred, partially external, and masked by short-run productivity gains. Tranquil periods lower subjective risk assessments, raise AI substitution intensity, and compound leverage, generating a cognitive Minsky moment in which subjective risk falls while true systemic fragility rises. Expected crisis losses are convex in aggregate leverage. Post-crisis, output-target pressure can produce a false-correction loop in which agents patch AI failures with more AI. The decentralised equilibrium over-adopts substitutive AI relative to the social optimum because of systemic risk, cognitive public goods, and arms-race externalities. In a two-type heterogeneous-agent economy, high-cognitive-capital agents adopt AI more intensively and may eventually erode their unaided cognitive capital below that of initially lower-skilled agents.

Chinese Translation

我们发展了一种认知债务的正式理论：当个体将人工智能作为第一性认知的替代品而非补充品时，未验证的推理义务的存量会累积。该模型为每个主体设定了两个状态变量：认知资本和认知债务，并采用一种乘法生产技术，其中认知资本作为抵押品，决定了人工智能采用的回报。我们建立了六个命题。理性主体会产生正的认知债务，因为成本被推迟、部分外部化，并且被短期生产力提升所掩盖。平静时期降低了主观风险评估，提高了人工智能替代的强度，并加剧了杠杆效应，产生了认知明斯基时刻，在这一时刻，主观风险下降而真正的系统脆弱性上升。预期危机损失在总杠杆中是凸的。危机后，产出目标压力可能产生一个虚假修正循环，在这个循环中，主体用更多的人工智能来修补人工智能的失败。由于系统风险、认知公共产品和军备竞赛外部性，分散的均衡相对于社会最优过度采用替代性人工智能。在一个两类异质主体的经济中，高认知资本的主体更 intensively 地采用人工智能，并可能最终使其未借助的认知资本低于最初技能较低的主体。

View on arXiv Download PDF AI Translation

cs.AI / 13 / 2606.15096

VGPT-RSI for RH-Adjacent Formal Progress: Boundary Certificates, Verified Finite Lagarias Inequalities, and Explicit Failure Localization

VGPT-RSI在RH相邻形式进展中的应用：边界证书、经过验证的有限Lagarias不等式及显式失败定位

Hu, Zhixin, Xu, Tao, Sun, Xiaodian, Jin, Li, Xiong, Momiao

Abstract

The Riemann Hypothesis remains one of the central unsolved problems in mathematics. Rather than claiming proof, we investigate whether a verifiable AI-assisted reasoning system can produce reliable, formally checked partial progress while explicitly identifying the remaining mathematical obstructions. We apply the Verifiable Growing Physical Transformer with Recursive Self-Improvement (VGPT-RSI) to two RH-adjacent certification tasks. First, we construct and verify a finite RH-boundary certificate for inequality on a parameterized safe lower curve over a region. The numerical boundary curve is converted into a certificate-backed lower curve, audited using outward-rounded interval arithmetic and Arb/FLINT ball arithmetic, and then checked in Rocq/CoqInterval for the parameterized theorem. Second, we initiate a formal Lagarias-route certificate. Lagarias criterion states that RH is equivalent to the global inequality. We formalize the finite quantity and produce a Coq-checked finite certificate. The final system identifies the exact unresolved mathematical bottlenecks: formalizing the Lagarias equivalence, proving the global tail theorem beyond any finite cutoff, and potentially reducing counterexamples to colossally abundant or related extremal integers. These results demonstrate that VGPT-RSI can produce certified RH-adjacent formal progress, organize proof dependencies, and avoid overclaiming when the remaining obstruction is genuinely mathematical.

Chinese Translation

黎曼假设仍然是数学中未解决的中心问题之一。我们并不声称提供证明，而是探讨一个可验证的AI辅助推理系统是否能够在明确识别剩余数学障碍的同时，产生可靠的、形式上经过检查的部分进展。我们将可验证的增长物理变换器与递归自我改进（VGPT-RSI）应用于两个RH相邻的认证任务。首先，我们构建并验证了一个有限的RH边界证书，该证书针对一个参数化的安全下界曲线上的不等式。数值边界曲线被转换为一个有证书支持的下界曲线，使用外部圆整区间算术和Arb/FLINT球算术进行审计，然后在Rocq/CoqInterval中检查该参数化定理。其次，我们启动了一个形式的Lagarias路径证书。Lagarias标准表明，RH等价于全局不等式。我们形式化有限数量并生成一个Coq检查的有限证书。最终系统识别出确切的未解决数学瓶颈：形式化Lagarias等价性、证明超出任何有限截止的全局尾部定理，以及可能将反例减少到极其丰盈或相关的极端整数。这些结果表明，VGPT-RSI能够产生经过认证的RH相邻形式进展，组织证明依赖关系，并在剩余障碍确实是数学时避免过度声称。

View on arXiv Download PDF AI Translation

cs.AI / 14 / 2606.15107

Towards Verifiable Agentic Data Science: Solving Irregular TSQA Via Tool-Grounded Reasoning

迈向可验证的自主数据科学：通过工具基础推理解决不规则时间序列问答

Chen, Sanhorn, Chen, Xiaoyang, Liu, Boyu, Zhao, Roy

Abstract

Time series data in real-world deployments is overwhelmingly irregular. Observations are asynchronous, missing values are informative rather than random, and sampling frequencies vary across sensors and operational windows. However, existing Time Series Question Answering (TSQA) benchmarks mostly assume regularly sampled inputs, leaving a fundamental gap in understanding how large language models (LLMs) and AI agents perform under irregular conditions. To bridge this gap, we introduce IRTS-ToolBench, a benchmark of 1,700 questions spanning 10 task types across 13 domains. IRTS-ToolBench is designed to be used independently by any researcher working on LLM-based irregular time series analysis, providing standardized inputs and a reproducible evaluation protocol. Code can be found in https://github.com/SanhornC/IRTS-ToolBench.

Chinese Translation

现实世界中的时间序列数据往往是不规则的。观察值是异步的，缺失值是信息性的而非随机的，且不同传感器和操作窗口的采样频率各不相同。然而，现有的时间序列问答（TSQA）基准大多假设输入是规则采样的，这在理解大型语言模型（LLMs）和人工智能代理在不规则条件下的表现时留下了根本性的空白。为填补这一空白，我们引入了IRTS-ToolBench，这是一个包含1700个问题的基准，涵盖13个领域的10种任务类型。IRTS-ToolBench旨在供任何从事基于LLM的不规则时间序列分析的研究人员独立使用，提供标准化的输入和可重复的评估协议。代码可在https://github.com/SanhornC/IRTS-ToolBench找到。

View on arXiv Download PDF AI Translation

cs.AI / 15 / 2606.15179

CONCORD: Asynchronous Sparse Aggregation for Device-Cloud RAG under Document Isolation

CONCORD：在文档隔离下的设备-云 RAG 的异步稀疏聚合

Hu, Xuedong, Tang, Zhiqing, Yao, Zhi, Wang, Tian, Jia, Weijia

Abstract

Retrieval-augmented generation (RAG) has emerged as a pivotal technique for improving language models by incorporating external knowledge at inference time. As device-cloud collaborative inference makes it feasible to deploy small language models on edge devices, a new setting arises in which private documents remain on the device and public knowledge resides in the cloud. Privacy and policy constraints often forbid raw document exchange, creating a document-isolated dual-end RAG setting. However, existing methods rely on frequent remote synchronization and dense evidence transfer, limiting throughput under realistic latency and bandwidth conditions. To address this issue, we propose CONCORD, an asynchronous sparse aggregation framework for dual-end RAG under document isolation. CONCORD treats the cloud as an asynchronously arriving evidence source rather than a continuously synchronized co-generator. Specifically, we introduce waiting debt control to decide whether each decoding step should continue waiting for remote participation based on the observed return of waiting. We also design a certificate-guided minimal supplementation mechanism that requests only the remote evidence needed to determine the current greedy decision. Steps that consult the cloud preserve the same greedy token as dense dual-end aggregation, while the remaining steps commit locally without remote evidence. Experiments on Natural Questions and WikiText-2 show that CONCORD improves end-to-end throughput over baselines by $1.66\times$ and $2.15\times$, respectively, while reducing per-token communication by over two orders of magnitude and maintaining comparable answer quality and perplexity.

Chinese Translation

检索增强生成（RAG）已成为通过在推理时引入外部知识来提升语言模型的关键技术。随着设备-云协同推理使得在边缘设备上部署小型语言模型成为可能，出现了一种新的场景，其中私有文档保留在设备上，而公共知识则存储在云端。隐私和政策限制通常禁止原始文档的交换，从而形成了文档隔离的双端 RAG 设置。然而，现有方法依赖于频繁的远程同步和密集的证据传输，在现实的延迟和带宽条件下限制了吞吐量。为了解决这个问题，我们提出了 CONCORD，一种在文档隔离下的双端 RAG 的异步稀疏聚合框架。CONCORD 将云视为一个异步到达的证据源，而不是一个持续同步的共同生成器。具体而言，我们引入了等待债务控制，以决定每个解码步骤是否应继续等待远程参与，基于观察到的等待返回。我们还设计了一种证书引导的最小补充机制，仅请求确定当前贪婪决策所需的远程证据。咨询云的步骤保留与密集双端聚合相同的贪婪标记，而其余步骤则在没有远程证据的情况下本地提交。在 Natural Questions 和 WikiText-2 上的实验表明，CONCORD 在端到端吞吐量上分别比基线提高了 $1.66 imes$ 和 $2.15 imes$，同时将每个标记的通信减少了两个数量级以上，并保持了可比的答案质量和困惑度。

View on arXiv Download PDF AI Translation

cs.AI / 16 / 2606.15199

CogGuard: Cognitive and Operational Profiling for Proactive Warning in Edge Intelligent Services

CogGuard：用于边缘智能服务的认知与操作分析以实现主动预警

Yao, Zhi, Chen, Weihao, Tang, Zhiqing, Cui, Hanshuai, Ma, Qianli, Jia, Weijia, Zhao, Wei

Abstract

Proactive warning is an important capability for edge intelligent services, where the system predicts whether a subject will successfully complete an incoming task under strict latency and privacy constraints. Such prediction depends on both long-term static attributes and short-term dynamic states derived from historical interaction logs. Recent Large Language Models (LLMs) offer strong long-context reasoning for constructing structured profiles from these logs, but existing solutions face two challenges for edge deployment: (1) profiling methods are typically domain-specific and lack a reusable abstraction across service scenarios, and (2) fine-tuning alignment models on heterogeneous edge clusters incurs high synchronization overhead due to the variance in input sequence lengths. To address these challenges, we propose CogGuard, a proactive-warning framework for edge intelligent services. CogGuard decouples offline LLM-based profile construction from online Small Language Model (SLM)-based score prediction through a shared static-dynamic profile-to-score pipeline, and instantiates it in two representative scenarios: educational performance warning and operational task outcome warning. For efficient profile construction, we design scenario-specific profiling methods with prefix-aligned KV-cache reuse to reduce repeated encoding overhead. For edge-side model alignment, we propose a length-aware distributed fine-tuning strategy with contrastive regularization to mitigate workload imbalance on heterogeneous clusters. Experiments on education and operation datasets show that CogGuard reduces profile construction time by up to 48% and distributed fine-tuning time by 19%, while achieving MAEs of 13.4 and 5.9, respectively, on 100-point-scale warning tasks. In the largest educational setting, CogGuard reduces prediction error by 15.4% compared with the strongest baseline.

Chinese Translation

主动预警是边缘智能服务的重要能力，该系统预测在严格的延迟和隐私约束下，某个主体是否能够成功完成即将到来的任务。这种预测依赖于来自历史交互日志的长期静态属性和短期动态状态。近期的大型语言模型（LLMs）在从这些日志中构建结构化档案方面提供了强大的长上下文推理能力，但现有解决方案在边缘部署中面临两个挑战：（1）分析方法通常是特定于领域的，缺乏跨服务场景的可重用抽象；（2）在异构边缘集群上微调对齐模型由于输入序列长度的差异而产生高同步开销。为了解决这些挑战，我们提出了CogGuard，一个用于边缘智能服务的主动预警框架。CogGuard通过共享的静态-动态档案到评分管道，将基于离线LLM的档案构建与基于在线小型语言模型（SLM）的评分预测解耦，并在两个代表性场景中实例化：教育表现预警和操作任务结果预警。为了高效构建档案，我们设计了特定场景的分析方法，并采用前缀对齐的KV缓存重用，以减少重复编码的开销。对于边缘侧模型对齐，我们提出了一种长度感知的分布式微调策略，结合对比正则化，以减轻异构集群上的工作负载不平衡。在教育和操作数据集上的实验表明，CogGuard将档案构建时间减少了多达48%，分布式微调时间减少了19%，同时在100分制预警任务上分别实现了13.4和5.9的平均绝对误差（MAE）。在最大的教育场景中，CogGuard与最强基线相比，将预测误差降低了15.4%。

View on arXiv Download PDF AI Translation

cs.AI / 17 / 2606.15209

Attribute Inference from Interactive Targeted Ads

从互动定向广告中推断属性

Li, Peihao

Abstract

Targeted advertising systems can pair audiences selected by advertisers with ad units that expose visible user actions. When an interaction remains linked to the campaign that elicited it, the advertiser may receive an observation tied to a user rather than only an aggregate report. We model that channel as a noisy oracle for attribute inference. The model separates targeting predicates, exposure, interaction, and disclosure. These boundaries capture the gap between eligibility and delivery, and the gap between interaction and advertiser visibility. We build a reproducible benchmark using synthetic populations calibrated with public data, each with known sensitive labels. A generated campaign semantics layer provides topic variants and response priors. The simulator generates the ground truth, event traces, disclosed observations, and metrics. The evaluation compares Bayesian, supervised, positive and unlabeled, and adaptive attacks under common campaign and disclosure definitions. The final evaluation uses four topic variants, seven simulator seeds, and two interaction settings. Repeated campaigns with identity exposure produce measurable but bounded inference signal. At $160$ campaigns, Bayesian and supervised attacks reach about $0.64$ AUC in the main setting and about $0.65$ AUC in the higher interaction setting. Disclosure policy is the strongest control. Aggregate reporting removes the evaluated oracle input tied to users. Type filtering and randomized disclosure reduce the released signal. The result is a model, artifact, and defense evaluation method for privacy in interactive targeted advertising. The code is available at https://github.com/P-HOW/Interactive-Ad-Oracle.

Chinese Translation

定向广告系统可以将广告主选择的受众与展示可见用户行为的广告单元配对。当一次互动与引发该互动的广告活动保持关联时，广告主可能会获得与用户相关的观察结果，而不仅仅是汇总报告。我们将该渠道建模为一个用于属性推断的噪声神谕。该模型区分了定向谓词、曝光、互动和披露。这些边界捕捉了资格与投放之间的差距，以及互动与广告主可见性之间的差距。我们使用经过公共数据校准的合成群体构建了一个可重复的基准，每个群体都有已知的敏感标签。生成的广告活动语义层提供了主题变体和响应先验。模拟器生成真实情况、事件轨迹、披露观察和指标。评估比较了贝叶斯、监督、正负样本和自适应攻击在常见广告活动和披露定义下的表现。最终评估使用了四个主题变体、七个模拟器种子和两种互动设置。重复的广告活动与身份曝光产生可测量但有限的推断信号。在160次广告活动中，贝叶斯和监督攻击在主要设置下的AUC约为0.64，在更高互动设置下的AUC约为0.65。披露政策是最强的控制措施。汇总报告移除了与用户相关的评估神谕输入。类型过滤和随机披露减少了发布的信号。最终结果是一个模型、工件和用于互动定向广告隐私的防御评估方法。代码可在 https://github.com/P-HOW/Interactive-Ad-Oracle 获取。

View on arXiv Download PDF AI Translation

cs.AI / 18 / 2606.15231

Visual-Seeker: Towards Visual-Native Multimodal Agentic Search via Active Visual Reasoning

Visual-Seeker：通过主动视觉推理迈向视觉原生的多模态智能搜索

Zhang, Zhengbo, Miao, Changtao, Su, Jinbo, Zhou, Zhaowen, Zhang, Chunxia, Wang, Xukai, Liu, Ruiqi, Zheng, Kaiyuan, Cai, Jiansheng, Zhang, Bo, Li, Zhe, Xiang, Shiming, Yan, Ying

Abstract

Multimodal large language models (MLLMs) have demonstrated impressive capabilities in many visual tasks, but they often struggle with factual grounding when confronted with complex, open-world scenarios. While recent multimodal deep search agents attempt to address this issue by utilizing external tools, the visual-native search paradigm remains underexplored. Existing methods primarily rely on simple images with explicit semantics and text-only evidence trajectories, limiting the agent's ability to perform multi-hop, cross-modal reasoning and search. To address these limitations, we propose Visual-Seeker, a visual-native multimodal deep search agent via active visual reasoning. Rather than treating vision as a static input, our agent actively attends to fine-grained visual details, dynamically harvests visual evidence throughout the search process. To unlock its visual-native potential, we design an active visual reasoning data pipeline and synthesize 5K high-quality multimodal trajectories for model training. Extensive experiments demonstrate the state-of-the-art performance across five challenging multimodal search benchmarks, even surpassing several proprietary models, validating robust visual-native reasoning and search in real-world web environments. The code and data can be accessed at: https://github.com/ZhengboZhang/Visual-Seeker.

Chinese Translation

多模态大型语言模型（MLLMs）在许多视觉任务中展现了令人印象深刻的能力，但在面对复杂的开放世界场景时，它们往往在事实基础方面表现不佳。尽管近期的多模态深度搜索代理试图通过利用外部工具来解决这一问题，但视觉原生搜索范式仍然未得到充分探索。现有方法主要依赖于具有明确语义的简单图像和仅包含文本的证据轨迹，这限制了代理进行多跳跨模态推理和搜索的能力。为了解决这些局限性，我们提出了Visual-Seeker，一种通过主动视觉推理实现的视觉原生多模态深度搜索代理。我们的代理并不将视觉视为静态输入，而是主动关注细致的视觉细节，在搜索过程中动态收集视觉证据。为了释放其视觉原生潜力，我们设计了一个主动视觉推理数据管道，并合成了5000条高质量的多模态轨迹用于模型训练。大量实验表明，在五个具有挑战性的多模态搜索基准上，Visual-Seeker表现出最先进的性能，甚至超过了若干专有模型，验证了其在真实网络环境中强大的视觉原生推理和搜索能力。代码和数据可在以下链接访问：https://github.com/ZhengboZhang/Visual-Seeker。

View on arXiv Download PDF AI Translation

cs.AI / 19 / 2606.15258

Mask-Proof: An LLM-based Automated Data Curation Pipeline on Mathematical Proofs

Mask-Proof：基于大型语言模型的数学证明自动数据整理管道

Zhang, Jierui, Tan, Siyuan, Li, Xinhang, Lin, Longzhuangzhi, Li, Dailin, Gu, Chengfeng, Li, Xinping, Hao, Yaxian, Liang, Shengjia, Ren, Yuxiang, Liu, Wenhao

Abstract

Large language models (LLMs) are increasingly capable of mathematical problem solving and can even assist with research-level proofs, yet we still lack a scalable and reproducible way to measure step-level reasoning in long proofs across diverse sources. This evaluation gap limits trustworthy AI assistance in proof-certified scientific progress. Existing evaluations often emphasize final answers or rely on costly expert grading, while end-to-end proof generation remains open-ended and hard to verify automatically. We introduce Mask-Proof, a pipeline that turns real proofs into automatically checkable masked-step tasks. It masks key formula steps, provides the necessary surrounding context, and evaluates model reconstructions with an LLM-based equivalence judge using repeated votes for stability. The resulting Mask-ProofBench contains 292 curated problems across diverse research areas. Experiments with 17 models show that reasoning-enhanced models outperform standard models by 12% to 27%. Our evaluator achieves 96.8% agreement with expert annotators, enabling faithful, reproducible, and comparable measurement of step-level mathematical reasoning. Benchmark, annotations, and code are available at https://github.com/weating/Mask-Proof.

Chinese Translation

大型语言模型（LLMs）在数学问题解决方面的能力日益增强，甚至可以协助进行研究级别的证明，然而我们仍然缺乏一种可扩展且可重复的方法来衡量来自不同来源的长证明中的逐步推理。这一评估缺口限制了可信赖的人工智能在证明认证科学进展中的辅助作用。现有的评估通常强调最终答案或依赖于昂贵的专家评分，而端到端的证明生成仍然是开放式的，且难以自动验证。我们提出了Mask-Proof，一个将真实证明转化为可自动检查的掩码步骤任务的管道。它掩盖了关键公式步骤，提供必要的上下文，并使用基于LLM的等价判断器通过重复投票来评估模型重构的稳定性。最终生成的Mask-ProofBench包含292个来自不同研究领域的整理问题。对17个模型的实验表明，增强推理能力的模型比标准模型的表现提高了12%到27%。我们的评估器与专家标注者的达成一致率达到96.8%，使得逐步数学推理的测量能够真实、可重复且可比较。基准、注释和代码可在https://github.com/weating/Mask-Proof获取。

View on arXiv Download PDF AI Translation

cs.AI / 20 / 2606.15273

Feature Attribution in Directed Acyclic Graphs Using Edge Intervention

基于边干预的有向无环图特征归因

Sun, Qiheng, Liu, Junxu, Mao, Xiaokai, Xia, Haocheng, Liu, Jinfei, Ren, Kui, Hu, Haibo

Abstract

Shapley value-based feature attribution methods face challenges in scenarios involving complex feature interactions and causal relationships, even when a causal structure is provided. Existing methods typically adopt a node-centric view, attributing importance solely to individual features. Consequently, they often fail to simultaneously capture the externality and exogenous influence of features, leading to unreasonable interpretations. To overcome these limitations, we propose a novel feature attribution method called DAG-SHAP, which is based on edge intervention. DAG-SHAP treats each feature edge as an individual attribution object, ensuring that both externality and exogenous contributions of features are appropriately captured. Additionally, we introduce an approximation method for efficiently computing DAG-SHAP. Extensive experiments on both real and synthetic datasets validate the effectiveness of DAG-SHAP. Our code is available at https://github.com/ZJU-DIVER/DAG-SHAP.

Chinese Translation

基于Shapley值的特征归因方法在涉及复杂特征交互和因果关系的场景中面临挑战，即使在提供因果结构的情况下也是如此。现有方法通常采用以节点为中心的视角，仅将重要性归因于单个特征。因此，它们往往无法同时捕捉特征的外部性和外生影响，导致不合理的解释。为克服这些局限性，我们提出了一种新颖的特征归因方法，称为DAG-SHAP，该方法基于边干预。DAG-SHAP将每个特征边视为一个独立的归因对象，确保特征的外部性和外生贡献得到适当捕捉。此外，我们还引入了一种近似方法，以高效计算DAG-SHAP。在真实和合成数据集上的大量实验验证了DAG-SHAP的有效性。我们的代码可在 https://github.com/ZJU-DIVER/DAG-SHAP 获取。

View on arXiv Download PDF AI Translation

cs.AI / 21 / 2606.15291

A Formal Framework for Declarative Agentic AI in Business Process Analysis

商业流程分析中声明性代理人工智能的正式框架

Azarijafari, Mohammad, Mich, Luisa, Missikoff, Michele

Abstract

Agentic AI opens new opportunities for automating Business Process (BP), enabling autonomous decision-making and dynamic adaptation. However, realising this potential requires BP entities and their interactions to be defined with formal precision. This paper presents a formal framework for Agentic BP analysis through the AGO methodology. AGO captures the modelling perspective in terms of who is acting (Agents), why it is carried out (Goals), and what the relevant entities are (Objects). Grounded in set theory and mathematical logic, we formally define the AGO entity types and their interactions, organising all definitions into a BP Knowledge Base (BPKB). The resulting BPKB supports structured querying, incremental updates, and automatic generation of BP workflows, while ensuring soundness and completeness of the derived paths.

Chinese Translation

代理人工智能为自动化商业流程（BP）开辟了新的机会，使自主决策和动态适应成为可能。然而，实现这一潜力需要以正式的精确性定义BP实体及其相互作用。本文通过AGO方法论提出了一个用于代理BP分析的正式框架。AGO从谁在行动（代理）、为什么要进行（目标）以及相关实体是什么（对象）等角度捕捉建模视角。基于集合论和数学逻辑，我们正式定义了AGO实体类型及其相互作用，并将所有定义组织到一个BP知识库（BPKB）中。生成的BPKB支持结构化查询、增量更新和BP工作流的自动生成，同时确保所推导路径的正确性和完整性。

View on arXiv Download PDF AI Translation

cs.AI / 22 / 2606.15300

CODA-BENCH: Can Code Agents Handle Data-Intensive Tasks?

CODA-BENCH：代码智能体能否处理数据密集型任务？

Zhang, Yuxin, Fan, Ju, Fan, Meihao, Zhang, Shaolei, Du, Xiaoyong

Abstract

Advanced agents are increasingly demonstrating the potential to operate as autonomous engineers, creating a growing demand for evaluation benchmarks that capture the complexity of real-world development. Such environments typically involve both complex code and large-scale data (i.e., file system). However, existing benchmarks usually evaluate code-centric or data-centric capabilities in isolation, leaving a clear gap with real development scenarios. In this paper, we bridge this gap by introducing CODA-BENCH, the first benchmark to jointly evaluate code and data intelligence in a data-intensive environment. We construct a data-intensive Linux sandbox based on the Kaggle ecosystem (containing hundreds of datasets), where agents must actively explore complex file hierarchies to identify relevant resources and generate code for data-driven analytical tasks. CODA-BENCH comprises 1,009 tasks spanning 31 communities, with each task environment containing an average of 980 files, simulating realistic data scale and noise. Evaluations of advanced agents reveal that even top-performing systems struggle to effectively integrate data discovery with code execution, achieving a success rate of only 61.1%. These results highlight a substantial gap in current agentic capabilities for data-intensive tasks and point to promising directions for future research.

Chinese Translation

先进的智能体越来越显示出作为自主工程师的潜力，导致对能够捕捉真实世界开发复杂性的评估基准的需求日益增长。这些环境通常涉及复杂的代码和大规模数据（即文件系统）。然而，现有的基准通常孤立地评估以代码为中心或以数据为中心的能力，导致与真实开发场景之间存在明显差距。在本文中，我们通过引入CODA-BENCH来填补这一空白，这是第一个在数据密集型环境中联合评估代码和数据智能的基准。我们基于Kaggle生态系统构建了一个数据密集型的Linux沙箱（包含数百个数据集），智能体必须积极探索复杂的文件层次结构，以识别相关资源并为数据驱动的分析任务生成代码。CODA-BENCH包含1,009个任务，涵盖31个社区，每个任务环境平均包含980个文件，模拟了真实的数据规模和噪声。对先进智能体的评估表明，即使是表现最好的系统在有效整合数据发现与代码执行方面也面临挑战，成功率仅为61.1%。这些结果突显了当前智能体在数据密集型任务中的能力存在显著差距，并指向未来研究的有希望方向。

View on arXiv Download PDF AI Translation

cs.AI / 23 / 2606.15308

Forced Deferral: Manipulating Routing Decisions in Multimodal LLM Cascades

强制延迟：在多模态大语言模型级联中操控路由决策

Liu, Zhongye, Zeng, Yaopei, Chang, Yurui, Lin, Lu

Abstract

While multimodal large language models (MLLMs) have shown strong visual reasoning abilities, serving a large model for every query is computationally expensive. MLLM cascades mitigate this cost by first querying a weak but cheaper model and deferring to a strong model when the weak model's output is unconfident. However, since the weak model's confidence directly controls compute allocation, these systems expose a new attack surface: an adversary can manipulate confidence so that their queries are consistently deferred to the strong model. Motivated by this vulnerability, we introduce the Forced Deferral Attack (FDA), an adversarial image attack that lowers the weak model's confidence and causes cascades to route queries to the strong model. FDA learns a universal border trigger by optimizing a temperature-flattened objective. This objective pushes the weak model's token distribution on triggered inputs toward less concentrated targets constructed from its clean responses. Across datasets, model families, and deferral metrics, FDA consistently increases strong-model routing while outperforming image-perturbation and prompt-injection baselines. These results show that MLLM cascades are vulnerable to attacks that manipulate compute allocation, forcing unintended strong-model usage without directly targeting answer correctness.

Chinese Translation

尽管多模态大语言模型（MLLMs）展现了强大的视觉推理能力，但为每个查询服务一个大型模型在计算上是昂贵的。MLLM 级联通过首先查询一个弱但更便宜的模型，并在弱模型的输出不确定时转而使用强模型，从而减轻了这一成本。然而，由于弱模型的置信度直接控制计算资源的分配，这些系统暴露了一个新的攻击面：对手可以操控置信度，使得他们的查询始终被转发到强模型。基于这一脆弱性，我们引入了强制延迟攻击（Forced Deferral Attack, FDA），这是一种对抗性图像攻击，旨在降低弱模型的置信度，从而导致级联将查询路由到强模型。FDA 通过优化温度平坦化目标学习一个通用边界触发器。该目标推动弱模型在触发输入上的标记分布朝向从其干净响应构建的较不集中目标。跨数据集、模型系列和延迟指标，FDA 一致性地增加了强模型的路由，同时超越了图像扰动和提示注入的基线。这些结果表明，MLLM 级联对操控计算分配的攻击是脆弱的，强制使用意图外的强模型，而不直接针对答案的正确性。

View on arXiv Download PDF AI Translation

cs.AI / 24 / 2606.15315

ChatPlanner: A Large Language Model Framework for Personalized Public Transit Routing

ChatPlanner：一个用于个性化公共交通路线规划的大型语言模型框架

Yang, Tingting, Xue, Chenhao, Chen, Jun

Abstract

Personalized public transit routing in public transit systems remains challenging due to the difficulty of capturing and integrating diverse user preferences into routing algorithms. This paper presents ChatPlanner, a novel framework that leverages Large Language Models (LLMs) to enable preference aware public transit routing. Our approach employs fine-tuned LLMs with Retrieval-Augmented Generation (RAG) to extract routing parameters and interpret nuanced user preferences from natural language queries, subsequently integrating these preferences into the objective function of a public transit routing algorithm. This study designs preference aware datasets incorporating eight personas and five contexts to establish scoring standards for both fine-tuning and RAG. This work conducted three experiments to validate the solutions' feasibility, extraction of routing information and preferences, and solution set quality and completeness. Results demonstrate that ChatPlanner generates feasible solutions reliably. Fine-tuning enforces the required output structure and learns general preference patterns, while RAG provides query-specific context to resolve imprecise or conversational expressions and calibrate continuous scores. The combination of both achieves the highest accuracy in routing information extraction and user preference interpretation. Results based on selected case studies show that by capturing user preferences, ChatPlanner identifies valuable solutions across different dimensions that existing route planners overlook, generating more valuable route alternatives. This research establishes a new paradigm for integrating natural language understanding into transportation optimization.

Chinese Translation

在公共交通系统中，个性化公共交通路线规划仍然面临挑战，因为很难将多样化的用户偏好捕捉并整合到路线算法中。本文提出了ChatPlanner，一个新颖的框架，利用大型语言模型（LLMs）实现偏好感知的公共交通路线规划。我们的方法采用经过微调的LLMs与检索增强生成（RAG）相结合，从自然语言查询中提取路线参数并解释细微的用户偏好，随后将这些偏好整合到公共交通路线算法的目标函数中。本研究设计了包含八种角色和五种情境的偏好感知数据集，以建立微调和RAG的评分标准。我们进行了三项实验，以验证解决方案的可行性、路线信息和偏好的提取，以及解决方案集的质量和完整性。结果表明，ChatPlanner能够可靠地生成可行的解决方案。微调确保了所需输出结构，并学习一般偏好模式，而RAG则提供特定查询的上下文，以解决不精确或对话式的表达并校准连续评分。两者的结合在路线信息提取和用户偏好解释方面达到了最高准确性。基于选定案例研究的结果表明，通过捕捉用户偏好，ChatPlanner能够识别出现有路线规划器所忽视的不同维度的有价值解决方案，从而生成更具价值的路线替代方案。本研究为将自然语言理解整合到交通优化中建立了一个新范式。

View on arXiv Download PDF AI Translation

cs.AI / 25 / 2606.15363

APEX: Adaptive Principle EXtraction A Three-Layer Self-Evolution Framework for Production AI Agents

APEX：自适应原则提取三层自我进化框架用于生产AI代理

Chen, Ya-Chuan, Lai, Tien-Jen, Hu, Hsiang-Wei

Abstract

Self-improvement in AI agents has emerged as a key research frontier: systems that modify their own prompts, workflows, and decision rules based on accumulated operational experience. The state-of-the-art Self-Harness framework [1] achieves 14--21% improvement on Terminal-Bench-2.0 by mining failure clusters and patching the agent harness. However, Self-Harness optimises only one dimension -- the prompt harness -- leaving behavioural principles and workflow topology unchanged. We propose APEX (Adaptive Principle EXtraction), a three-layer co-evolution framework that simultaneously evolves: (L1) the harness via failure-mode patching, (L2) behavioural principles via success-trace distillation [2], and (L3) the agent workflow topology via structural fitness-based selection [6]. We implement APEX on Joe [13], a production-grade super AI Agent built on NVIDIA Nemotron and designed as an Edge AI Agent Factory for the NVIDIA Agent Challenge 2026, managing a 15-node compute fleet using 114 real task traces collected over 18 days. APEX achieves an APEX Health Score of 0.570 (+90% vs. baseline 0.300) in a single evolutionary run, distilling 6 novel reusable principles and selecting a research-first workflow topology scoring 0.900 (+20%). Our results demonstrate that multi-dimensional co-evolution substantially outperforms single-axis harness optimisation, at a cost of only 4 LLM calls (~270 s) on a local qwen2.5-coder:32b instance.

Chinese Translation

AI代理的自我改进已成为一个关键的研究前沿：这些系统根据累积的操作经验修改自身的提示、工作流程和决策规则。最先进的Self-Harness框架[1]通过挖掘失败集群并修补代理工具，实现了在Terminal-Bench-2.0上14%到21%的改进。然而，Self-Harness仅优化了一个维度——提示工具——而未改变行为原则和工作流程拓扑。我们提出APEX（自适应原则提取），一个三层协同进化框架，能够同时进化：(L1) 通过失败模式修补优化工具，(L2) 通过成功轨迹蒸馏优化行为原则[2]，以及(L3) 通过基于结构适应度的选择优化代理工作流程拓扑[6]。我们在Joe[13]上实现了APEX，这是一个基于NVIDIA Nemotron构建的生产级超级AI代理，旨在作为NVIDIA Agent Challenge 2026的边缘AI代理工厂，管理一个由15个节点组成的计算集群，使用在18天内收集的114个真实任务轨迹。在一次进化运行中，APEX达到了0.570的APEX健康评分（比基线0.300提高了90%），蒸馏出6个新颖的可重用原则，并选择了一个研究优先的工作流程拓扑，评分为0.900（提高了20%）。我们的结果表明，多维协同进化显著优于单轴工具优化，且仅在本地qwen2.5-coder:32b实例上消耗了4次LLM调用（约270秒）的成本。

View on arXiv Download PDF AI Translation

cs.AI / 26 / 2606.15367

S1-DeepResearch: Beyond Search, Toward Real-World Long-Horizon Research Agents

S1-DeepResearch：超越搜索，迈向现实世界的长时程研究代理

Dong, Yao, Xiao, Xinglin, Dong, Liwei, Jin, Xinlong, Li, Zhengbo, Zhang, Heng, Wang, Duyun, Xu, Nan

Abstract

Deep research agents aim to solve complex knowledge-intensive tasks through long-horizon planning, evidence gathering, reasoning, and report generation. While recent progress in search agents has demonstrated strong capabilities in information retrieval and answer verification, most existing training datasets remain search-centric, focusing primarily on closed-ended question answering and information localization. As a result, they mainly train information-seeking behavior while providing limited coverage of key deep research capabilities, including evidence integration, knowledge synthesis, planning, file understanding, and structured report generation. In this work, we propose a unified trajectory construction paradigm for deep research agents that combines closed-ended QA and open-ended exploration. The proposed framework consists of graph-grounded task formulation, agentic trajectory rollout, and multi-dimensional trajectory verification, enabling scalable synthesis of high-quality agentic trajectories spanning long-chain complex reasoning, deep research instruction following, report writing, file understanding and generation, and skills usage. Compared with existing search-oriented datasets, our synthesized trajectories place greater emphasis on knowledge synthesis, complex reasoning, and planning. S1-DeepResearch-32B achieves state-of-the-art performance among open-source models of comparable scale across 20 benchmarks spanning five capability dimensions, including complex reasoning, instruction following, report generation, file understanding, and skills usage. On several challenging deep research benchmarks, it approaches the performance of leading proprietary frontier models. These results highlight the importance of jointly modeling information acquisition, knowledge synthesis, and planning-oriented agent behaviors for building effective deep research agents.

Chinese Translation

深度研究代理旨在通过长时程规划、证据收集、推理和报告生成来解决复杂的知识密集型任务。尽管近期在搜索代理方面的进展展示了其在信息检索和答案验证方面的强大能力，但大多数现有的训练数据集仍然以搜索为中心，主要集中在封闭式问答和信息定位上。因此，它们主要训练信息获取行为，而对关键的深度研究能力（包括证据整合、知识综合、规划、文件理解和结构化报告生成）覆盖有限。在本研究中，我们提出了一种统一的轨迹构建范式，用于深度研究代理，结合了封闭式问答和开放式探索。所提出的框架包括基于图的任务表述、代理轨迹展开和多维轨迹验证，使得能够大规模合成高质量的代理轨迹，涵盖长链复杂推理、深度研究指令遵循、报告撰写、文件理解与生成以及技能使用。与现有的以搜索为导向的数据集相比，我们合成的轨迹更加强调知识综合、复杂推理和规划。在20个涵盖五个能力维度的基准测试中，S1-DeepResearch-32B在可比规模的开源模型中实现了最先进的性能，包括复杂推理、指令遵循、报告生成、文件理解和技能使用。在若干具有挑战性的深度研究基准测试中，其性能接近领先的专有前沿模型。这些结果突显了共同建模信息获取、知识综合和规划导向的代理行为在构建有效深度研究代理中的重要性。

View on arXiv Download PDF AI Translation

cs.AI / 27 / 2606.15385

Reward Hacking in Language Model Agents: Revisiting AI Safety Gridworlds

语言模型代理中的奖励黑客行为：重访人工智能安全网格世界

Çağatan, Ömer Veysel, Zhao, Xuandong

Abstract

Reward hacking, where AI systems exploit misspecified objectives to achieve high reward without satisfying intended goals, remains a central challenge in AI safety. Yet most known instances have been discovered post hoc in frontier systems where controlled study is impractical. We adapt the AI Safety Gridworlds framework into a text-based evaluation suite that reformulates classic reinforcement learning safety tasks for language-based agents. Across frontier and mid-scale models, we find that specification gaming emerges zero-shot: models systematically achieve high observed reward while underperforming on hidden safety objectives, and even apparently safe behaviors can reflect misunderstanding rather than principled safety. Reinforcement learning does not correct these failures: direct reward optimization widens the gap between observed and hidden reward, as the model's initial competence causes it to lock into locally rewarding strategies before discovering safer alternatives. This pattern persists across model scales (1.5B--14B) and is not resolved by finer credit assignment, exploration prompts, or entropy regularization. Our results show that reward hacking arises naturally when optimizing proxy objectives with capable language model agents and resists standard mitigations, suggesting that proxy-reward failures in agentic settings may require approaches beyond standard exploration and credit-assignment fixes. To facilitate reproducibility, the code for this work is available at \href{https://github.com/asparius/verl-agent-safety}{our public repository}.

Chinese Translation

奖励黑客行为是指人工智能系统利用错误指定的目标来获得高奖励，而不满足预期目标，这仍然是人工智能安全领域的一个核心挑战。然而，大多数已知实例是在前沿系统中事后发现的，在这些系统中进行控制研究是不切实际的。我们将人工智能安全网格世界框架改编为一个基于文本的评估套件，重新构造经典强化学习安全任务以适用于基于语言的代理。在前沿和中型模型中，我们发现规范游戏现象在零样本情况下出现：模型系统性地获得高观察奖励，同时在隐藏的安全目标上表现不佳，甚至看似安全的行为也可能反映出误解，而非原则性的安全。强化学习并未纠正这些失败：直接的奖励优化扩大了观察奖励和隐藏奖励之间的差距，因为模型的初始能力使其在发现更安全的替代方案之前锁定在局部奖励策略上。这种模式在不同规模的模型中持续存在（1.5B-14B），并且通过更精细的信用分配、探索提示或熵正则化并未得到解决。我们的结果表明，当优化代理目标时，奖励黑客行为在有能力的语言模型代理中自然出现，并且抵抗标准的缓解措施，这表明在代理环境中，代理奖励失败可能需要超越标准探索和信用分配修复的方法。为了促进可重复性，本研究的代码可在我们的公共仓库中获得。

View on arXiv Download PDF AI Translation

cs.AI / 28 / 2606.15447

Hierarchical Modeling of ICD Codes in EHR Foundation Models

电子健康记录基础模型中ICD编码的层次建模

Thukral, Megha, Kang, Dong Gyun, Singh, Rudra Pratap, Hiremath, Shruthi Kashinath, Hänsel, Katrin, Plötz, Thomas

Abstract

Electronic health record foundation models typically treat ICD diagnosis codes as flat tokens, overlooking the clinically meaningful hierarchical structure that captures disease families, subcategories, and fine-grained diagnostic detail. As a result, existing EHR representation learning methods do not explicitly exploit the hierarchical structure already present in the coding system. In this work, we study ICD-10-CM hierarchy as a general inductive bias for clinical representation learning. We investigate two complementary mechanisms for incorporating hierarchy: first, by augmenting diagnosis sequences in a BERT-style transformer with tokens corresponding to different levels of the ICD hierarchy, and second, by injecting hierarchy into graph-based code representations through hierarchy-aware edges combined with diagnosis co-occurrence structure. Across these settings, we evaluate whether explicit hierarchy improves downstream prediction, which levels of the hierarchy are most useful, whether hierarchy encoding improves transfer across datasets, and how hierarchy reshapes embedding similarity structure. We conduct experiments on two large-scale real-world clinical datasets: MIMIC-IV, used for pretraining and in-domain evaluation, and eICU, used to assess cross-dataset transfer via frozen encoder probing. Our findings show that explicitly encoding ICD hierarchy improves over flat code representations in both in-domain and cross-dataset settings, while revealing that the most useful level of hierarchy depends on both the task and the modeling approach. More broadly, we focus on hierarchy-aware EHR representation learning and show that the benefits of encoding hierarchy are generalizable across modeling settings and hierarchy levels.

Chinese Translation

电子健康记录基础模型通常将ICD诊断编码视为平面标记，忽视了捕捉疾病家族、子类别和细粒度诊断细节的临床意义层次结构。因此，现有的电子健康记录表示学习方法并未明确利用编码系统中已经存在的层次结构。在本研究中，我们将ICD-10-CM层次结构作为临床表示学习的一种通用归纳偏置进行研究。我们探讨了两种互补机制以纳入层次结构：首先，通过在BERT风格的变换器中增强与ICD层次不同级别对应的标记的诊断序列；其次，通过结合诊断共现结构的层次感知边缘，将层次结构注入基于图的编码表示。在这些设置中，我们评估了显式层次结构是否改善下游预测，哪些层次最有用，层次编码是否改善跨数据集的迁移，以及层次如何重塑嵌入相似性结构。我们在两个大规模真实临床数据集上进行实验：MIMIC-IV，用于预训练和领域内评估，以及eICU，用于通过冻结编码器探测评估跨数据集迁移。我们的研究结果表明，显式编码ICD层次结构在领域内和跨数据集设置中均优于平面编码表示，同时揭示了最有用的层次级别依赖于任务和建模方法。更广泛地说，我们关注层次感知的电子健康记录表示学习，并展示了编码层次的好处在不同建模设置和层次级别之间是可推广的。

View on arXiv Download PDF AI Translation

cs.AI / 29 / 2606.15474

Who Drifted: the System or the Judge? Anytime-Valid Attribution in LLM Evaluation Pipelines

谁在漂移：系统还是评审？LLM评估管道中的随时有效归因

Li, Yitao

Abstract

Continuous evaluation of LLM products relies on a strong LLM judge treated as ground truth: a cheap monitor scores every interaction and a team is paged when the score drifts down. But the judge is itself a model behind an API, and a silent version bump or scoring-prompt update changes how it scores -- so every drift alarm is ambiguous between a worse product and a changed judge. We resolve the ambiguity with a fixed, human-labeled anchor set that the current judge re-scores at a steady interleave, a second betting e-process on the judge-versus-human gap, and a guard-window rule returning a verdict in {none, system, judge}. We prove anytime-validity, one-way identification (only the judge can move the anchors), an attribution race whose design law is that the anchors must out-run the main process they guard, and process orthogonality. On two real judge changes, a silent version bump is detected as judge drift in 60/60 runs with zero judge-to-system misattribution, and a contaminating strict-prompt change is correctly attributed on 110 of 120 runs at guard width 300 -- while the industry-default rolling z-test false-alarms on 75% of drift-free streams. Every experiment replicates on a second domain (TL;DR summarization) with nothing re-tuned, and where the domains differ the differences are the ones the race predicts: the strict-prompt change shifts scores harder there, so the anchors fire faster and attribution becomes perfect (240/240). The monitor runs at approximately 0.64 of the cost of strong-judging every item, or 0.21 in a cheaper-but-deafer regime.

Chinese Translation

对LLM产品的持续评估依赖于一个被视为真实标准的强大LLM评审：一个廉价的监控器为每次交互打分，当分数下降时会通知团队。然而，评审本身是一个通过API提供的模型，静默的版本更新或评分提示的更改会改变其评分方式，因此每次漂移警报在更差的产品和更改的评审之间是模糊的。我们通过一个固定的人类标注锚点集来解决这种模糊性，当前评审在稳定的交替中重新评分，一个针对评审与人类差距的第二个投注电子过程，以及一个返回判决的保护窗口规则，判决结果为{无、系统、评审}。我们证明了随时有效性、单向识别（只有评审可以移动锚点）、一个归因竞争，其设计法则是锚点必须超越它们所保护的主要过程，以及过程正交性。在两次真实的评审变化中，静默的版本更新在60/60次运行中被检测为评审漂移，且没有发生评审到系统的错误归因，而一个污染性的严格提示变化在300的保护宽度下在120次运行中正确归因110次——而行业默认的滚动z检验在75%的无漂移流中产生了虚假警报。每个实验在第二个领域（TL;DR摘要）中重复进行，且没有重新调整参数，而在领域不同的情况下，差异正是竞争所预测的：严格的提示变化在此处更强烈地影响分数，因此锚点反应更快，归因变得完美（240/240）。监控的成本约为强评审每个项目的0.64，或在一个更便宜但反应迟钝的环境中为0.21。

View on arXiv Download PDF AI Translation

cs.AI / 30 / 2606.15497

Towards End-to-End Automation of AI Research

迈向人工智能研究的端到端自动化

Yamada, Yutaro, Lange, Robert Tjarko, Lu, Cong, Lu, Chris, Hu, Shengran, Foerster, Jakob, Ha, David, Clune, Jeff

Abstract

The automation of science is a long-standing ambition in the field of AI. While the community has made significant progress in automating individual components of the scientific process, a system that autonomously navigates the entire research lifecycle -- from conception to publication -- has remained out of reach. Here, we present the strongest demonstration to date toward automating the entire process end-to-end. We present The AI Scientist, which creates research ideas, writes code, runs experiments, plots and analyzes data, writes the entire scientific manuscript and performs its own peer review. Its ideas, execution, and presentation are of sufficient quality to produce a manuscript generated by an AI system that passes the first round of peer review at a major machine learning conference workshop. The workshop has an acceptance rate of 70 percent. Our system leverages modern foundation models within a complex agentic system. We evaluate The AI Scientist in two settings: a focused mode using human-provided code templates as an initial scaffold to conduct research on a specific topic, and a template-free, open-ended mode that leverages agentic search for wider scientific exploration. Both settings produce diverse ideas and automatically test, report on, and evaluate them. This achievement demonstrates AI's growing capacity for scientific contribution and signifies a potential paradigm shift in how research is conducted. As with any impactful new technology, there could be significant risks, including taxing overwhelmed review systems and adding noise to scientific literature. However, if developed responsibly, such autonomous systems could greatly accelerate scientific discovery.

Chinese Translation

科学自动化是人工智能领域长期以来的追求。尽管社区在自动化科学过程的各个单独组件方面取得了显著进展，但一个能够自主导航整个研究生命周期——从构想到出版的系统仍然遥不可及。在此，我们展示了迄今为止在端到端自动化整个过程方面的最强演示。我们介绍了人工智能科学家（The AI Scientist），该系统能够创建研究想法、编写代码、运行实验、绘制和分析数据、撰写完整的科学手稿并进行自我同行评审。其创意、执行和呈现的质量足以生成一篇由人工智能系统撰写的手稿，并在某主要机器学习会议的首次同行评审中通过。该会议的接受率为70%。我们的系统在一个复杂的代理系统中利用现代基础模型。我们在两种环境中评估人工智能科学家：一种是使用人类提供的代码模板作为初始框架进行特定主题研究的集中模式，另一种是利用代理搜索进行更广泛科学探索的无模板开放模式。这两种环境都能产生多样化的想法，并自动测试、报告和评估这些想法。这一成就展示了人工智能在科学贡献方面日益增长的能力，并标志着研究开展方式的潜在范式转变。与任何具有重大影响的新技术一样，可能会存在显著风险，包括对过载的评审系统造成压力以及向科学文献中添加噪音。然而，如果负责任地开发，这种自主系统可以大大加速科学发现。

View on arXiv Download PDF AI Translation

cs.AI / 31 / 2606.15503

Synthetic Counteradaptation: A Principle of Human-AI Co-evolution

合成反适应：人类与人工智能共同进化的原则

Frisch, Ivar, Kay, Jackie, Tomei, Philip Moreira

Abstract

In this paper, we introduce the concept of synthetic counteradaptation, a process where human and AI systems co-evolve by adapting to each other's strategies and behaviors. Synthetic counteradaptation occurs when AI systems develop novel strategies or social protocols, prompting humans to extract insights and adapt their own behaviors in response, leading to the emergence of new agent interaction dynamics. To illustrate these dynamics, we analyze examples from various contexts, including the game of Go, mixed-motive social interactions, and geopolitical simulations. By exploring these cases, we demonstrate how synthetic counteradaptation provides a framework for understanding the recursive and co-evolutionary nature of human-AI interactions in multi-agent environments.

Chinese Translation

在本文中，我们引入了合成反适应的概念，这是一种人类与人工智能系统通过适应彼此的策略和行为而共同进化的过程。当人工智能系统发展出新颖的策略或社会协议时，合成反适应就会发生，这促使人类提取见解并相应地调整自己的行为，从而导致新的代理交互动态的出现。为了说明这些动态，我们分析了来自不同背景的示例，包括围棋、混合动机的社会互动和地缘政治模拟。通过探索这些案例，我们展示了合成反适应如何为理解多代理环境中人类与人工智能交互的递归和共同进化特性提供了框架。

View on arXiv Download PDF AI Translation

cs.AI / 32 / 2606.15504

Toward Vibe Medicine: A Self-Evolving Multi-Agent Framework for Clinical Decision Support

迈向振动医学：一种自我演化的多智能体框架用于临床决策支持

Zhang, Qianxue, Ren, Yiming, Qin, Shihuan, Zhang, Xiao, Zhang, Liao, Huang, Jinyang, Liu, Zhengliang, Liu, Chenbin, Feng, Hongying, Chen, Jingyuan, Ding, Yuzhen, You, Weihang, Jiang, Hanqi, Pan, Yi, Zhou, Yifan, Chen, Junhao, Chen, Lifeng, Liu, Wei, Liu, Tianming, Zhao, Zengren, Zhang, Lian

Abstract

In recent years, the advances of large language models and autonomous agents have revolutionized the healthcare field, facilitating diagnosis and improving treatment results. However, most existing AI systems rely on pre-trained knowledge and predefined pipelines, which struggle to learn dynamically from the interactive chat session history that contains patient outcomes and past failures. To address this limitation, we propose VIBEMed, a multi-agent framework with a built-in self-evolution mechanism and architecture-level safety sandbox for robust clinical decision support. The system integrates three specialized agents, including a Clinical Diagnostic Agent (CDA) for hypothesis generation, a Therapeutic Execution Agent (TEA) for treatment planning, and a Clinical Evolution Manager Agent (CEMA) that distills longitudinal clinical feedback into reusable knowledge, transforming multimodal patient information into personalized medical decisions. Through self-evolution mechanism, the framework enables iterative updates across memory, model behavior, and decision strategies, allowing the system to improve over time. Experimental results show that VIBEMed demonstrates superior performance through its evolving mechanism in complex clinical cases, particularly in tasks that require integrated decision-making and longitudinal planning. The framework also supports reliable end-to-end decisions in challenging scenarios such as oncology treatment planning, highlighting its feasibility in real-world clinical contexts. Overall, VIBEMed provides a practical path beyond static AI systems toward adaptive, experience-driven clinical decision support, demonstrating the value of combining multi-agent collaboration with continuous evolution for advancing precision medicine.

Chinese Translation

近年来，大型语言模型和自主智能体的进步彻底改变了医疗领域，促进了诊断并改善了治疗结果。然而，现有的大多数人工智能系统依赖于预训练知识和预定义流程，难以从包含患者结果和过去失败的互动聊天记录中动态学习。为了解决这一局限性，我们提出了VIBEMed，一个具有内置自我演化机制和架构级安全沙箱的多智能体框架，用于强大的临床决策支持。该系统集成了三个专业智能体，包括用于假设生成的临床诊断智能体（Clinical Diagnostic Agent, CDA）、用于治疗规划的治疗执行智能体（Therapeutic Execution Agent, TEA）以及将纵向临床反馈提炼为可重用知识的临床演化管理智能体（Clinical Evolution Manager Agent, CEMA），将多模态患者信息转化为个性化的医疗决策。通过自我演化机制，该框架实现了内存、模型行为和决策策略的迭代更新，使系统能够随着时间的推移不断改进。实验结果表明，VIBEMed在复杂临床案例中通过其演化机制表现出优越的性能，特别是在需要综合决策和纵向规划的任务中。该框架还支持在肿瘤治疗规划等具有挑战性的场景中可靠的端到端决策，突显了其在现实临床环境中的可行性。总体而言，VIBEMed为超越静态人工智能系统提供了一条实用路径，迈向适应性、经验驱动的临床决策支持，展示了将多智能体协作与持续演化相结合以推动精准医学的价值。

View on arXiv Download PDF AI Translation

cs.AI / 33 / 2606.15507

Frame-Conditioned Moral Computation in LLaMA 3.1-8B-Instruct: A Mechanistic Interpretability Audit of Ethical Reasoning

LLaMA 3.1-8B-Instruct中的框架条件道德计算：伦理推理的机制可解释性审计

Dasdan, Ali, Shah, Manan, Neuman, W. Russell, Coleman, Chad, Meghani, Kund, Ali, Safinah

Abstract

Behavioral audits of Large Language Models on moral prompts measure what the model says, not the internal computation producing it. We use Transluce, an AI-driven mechanistic-interpretability platform, to examine LLaMA 3.1-8B-Instruct on 54 moral prompts in four batteries: 17 dilemmas, policy, and meta-ethical questions (B1); 6 role-playing scenarios (B3); and a controlled trolley contrast varying the switching mechanism with people fixed (B4, 15 prompts) or identity attributes with mechanism fixed (B5, 16 prompts). Two complementary metric families, five cluster-level metrics and a six-metric neuron-level panel, converge on a Situational Anchor Effect: domain-specific representations dominate the top of the activation list across every battery. The model's ethics-labeled capacity stays essentially constant; its salience (rank, priority, top-of-list presence) is highly sensitive to the interpretive frame the prompt selects. The B4-vs-B5 contrast confirms the model attends to whichever surface feature varies: aggregate ethics metrics are indistinguishable, but the dominant non-ethics distractor mirrors the design. A multi-temperature audit identifies a candidate ethics neuron (L16/N3837) stable across temperatures; a cross-model behavioral proxy on two frontier models yields preliminary evidence of divergence in self-reported moral focus, consistent with an Alignment Wrapper in which RLHF re-orders surface text without removing underlying domain-first frames. We unify these as Frame-Conditioned Moral Computation: the prompt's surface vocabulary selects a feature manifold, and the moral conclusion is downstream of that selection. Behavioral alignment must be supplemented by Mechanistic Alignment: a research program asking whether ethics-related features can be shown causally privileged under controlled frame variation, not merely loud in the explanation.

Chinese Translation

对大型语言模型在道德提示下的行为审计测量的是模型所表达的内容，而非产生这些内容的内部计算。我们使用Transluce，一个由人工智能驱动的机制可解释性平台，来检查LLaMA 3.1-8B-Instruct在54个道德提示上的表现，这些提示分为四个组别：17个困境、政策和元伦理问题（B1）；6个角色扮演场景（B3）；以及一个控制的电车对比，变换开关机制而固定人群（B4，15个提示）或固定机制而变换身份属性（B5，16个提示）。两组互补的度量体系，五个集群级别的度量和一个六个度量的神经元级别面板，汇聚出一个情境锚定效应：特定领域的表征在每个组别的激活列表顶部占主导地位。模型的伦理标签能力基本保持不变；其显著性（排名、优先级、列表顶部存在）对提示选择的解释框架高度敏感。B4与B5的对比证实模型关注于任何变化的表面特征：整体伦理度量无法区分，但主导的非伦理干扰因素反映了设计。多温度审计识别出一个候选伦理神经元（L16/N3837），在不同温度下保持稳定；在两个前沿模型上的跨模型行为代理提供了初步证据，表明自我报告的道德关注存在差异，这与一种对齐包装（Alignment Wrapper）一致，其中强化学习人类反馈（RLHF）重新排列表面文本，而不移除潜在的领域优先框架。我们将这些统一为框架条件道德计算：提示的表面词汇选择了一个特征流形，而道德结论则是该选择的下游结果。行为对齐必须补充机制对齐：这是一个研究项目，旨在探讨在控制框架变化下，伦理相关特征是否可以被证明在因果上具有特权，而不仅仅是在解释中显得喧闹。

View on arXiv Download PDF AI Translation

cs.AI / 34 / 2606.15508

ToolMenuBench: Benchmarking Tool-Menu Filtering Strategies for Reliable and Efficient LLM Agents

ToolMenuBench：评估工具菜单过滤策略以实现可靠和高效的LLM代理

Babu, Rahul Suresh, Iyer, Laxmipriya Ganesh

Abstract

Tool-augmented large language model agents increasingly operate over large tool libraries, but existing evaluations often focus on whether a model can call a tool correctly rather than how the visible tool menu shapes reliability, efficiency, and safety-relevant risk exposure. We introduce ToolMenuBench, a benchmark for evaluating tool-menu construction in multi-step LLM agents. ToolMenuBench varies tool-menu size, distractor type, state-dependent task structure, and risk exposure, and reports both filter-level and downstream agent metrics, including visible-tool count, risky-tool exposure, task success, wrong-tool calls, premature actions, and token usage. In a controlled evaluation across seven model backends, three tool-menu sizes, six filtering methods, and seven evaluation settings, CMTF improves task success from 32.1% under all-tools exposure to 85.7%, while reducing average token usage by roughly 98%. Causal minimal tool filtering achieves the strongest overall tradeoff, reducing visible tools, wrong-tool calls, premature actions, and risky-tool exposure relative to unfiltered exposure, lexical filtering, state-aware filtering, and broader causal-path baselines. ToolMenuBench provides a reusable evaluation framework for studying the agent-interface problem: which tools should be visible, when they should be visible, and under what cost or risk constraints.

Chinese Translation

增强工具的大型语言模型代理越来越多地在大型工具库上运行，但现有评估往往关注模型是否能够正确调用工具，而不是可见工具菜单如何影响可靠性、效率和安全相关的风险暴露。我们引入了ToolMenuBench，这是一个用于评估多步骤LLM代理中工具菜单构建的基准。ToolMenuBench变更工具菜单的大小、干扰物类型、状态依赖的任务结构和风险暴露，并报告过滤级别和下游代理指标，包括可见工具数量、风险工具暴露、任务成功率、错误工具调用、过早行动和令牌使用。在对七个模型后端、三种工具菜单大小、六种过滤方法和七种评估设置进行的控制评估中，CMTF将任务成功率从32.1%（在所有工具暴露下）提高到85.7%，同时将平均令牌使用量减少了约98%。因果最小工具过滤在整体权衡中表现最佳，相较于未过滤暴露、词汇过滤、状态感知过滤和更广泛的因果路径基准，减少了可见工具、错误工具调用、过早行动和风险工具暴露。ToolMenuBench提供了一个可重用的评估框架，用于研究代理-接口问题：哪些工具应该可见，何时应该可见，以及在什么成本或风险约束下。

View on arXiv Download PDF AI Translation

cs.AI / 35 / 2606.15563

Minimal Oversight: Uncertainty-Aware Governance for Delegated AI Systems

最小监督：面向不确定性的委托AI系统治理

Azevedo, Carlos R. B.

Abstract

AI systems increasingly delegate decisions to specialized models, evaluators, tools, and supervisory controllers. The central AI problem is no longer only model accuracy, but uncertainty-aware governance: how much autonomy to grant, which evidence should calibrate trust, what performance ceiling a delegated AI system can sustain, and when human intervention becomes necessary. We propose the Minimum Sufficient Oversight Principle (MSO), a variational principle for principled autonomy delegation: minimize governance burden on the Fisher information manifold subject to a delivery constraint. The resulting Euler-Lagrange solution yields a water-filling allocation of governed delegation across the task space. Building on a revealed-action governed delegation channel model, we prove a capacity theorem for stationary symbolwise review policies, derive a local first-order approximation relating workflow complexity to quality degradation, and give a drift-dominated autonomy-time scaling law linking intervention timing to effective capacity, complexity, and drift. Within this framework, masking appears as a structural AI-governance pathology: corrected performance can hide the competence signal needed to calibrate trust. Synthetic simulations and a semi-real reconstructed workflow support design prescriptions including upstream-first correction, sensitivity-based intervention, and explicit feasibility checks before autonomy is expanded. The result is a computable framework for uncertainty, planning, and oversight in delegated AI systems. A companion Python package is available at https://github.com/crbazevedo/delegation-lab.

Chinese Translation

AI系统越来越多地将决策委托给专门的模型、评估者、工具和监督控制器。中心AI问题不再仅仅是模型的准确性，而是面向不确定性的治理：应授予多少自主权，哪些证据应校准信任，委托AI系统能够维持的性能上限，以及何时需要人类干预。我们提出了最小充分监督原则（Minimum Sufficient Oversight Principle, MSO），这是一种用于原则性自主委托的变分原则：在交付约束下，最小化在Fisher信息流形上的治理负担。得到的Euler-Lagrange解产生了在任务空间中治理委托的水填充分配。基于揭示行动的治理委托通道模型，我们证明了静态符号逐一审查策略的容量定理，推导出与工作流程复杂性相关的质量下降的局部一阶近似，并给出了一个漂移主导的自主时间缩放法则，将干预时机与有效容量、复杂性和漂移联系起来。在这个框架内，掩蔽被视为一种结构性AI治理病理：修正后的性能可能掩盖校准信任所需的能力信号。合成模拟和半真实重建的工作流程支持设计建议，包括上游优先修正、基于敏感性的干预，以及在扩大自主权之前进行明确的可行性检查。结果是一个可计算的框架，用于处理委托AI系统中的不确定性、规划和监督。一个配套的Python包可在https://github.com/crbazevedo/delegation-lab获取。

View on arXiv Download PDF AI Translation

cs.AI / 36 / 2606.15573

QoS-Aware Token Scheduling and Private Data Valuation for Multi-Modal Agentic Networks

面向服务质量的令牌调度与多模态代理网络中的私人数据评估

Du, Yao, Liu, Jing, Xu, Pengfei, Wang, Zehua, Leung, Victor C. M., Leung, Cyril, Lemieux, Victoria

Abstract

In agentic systems, human-generated data records anchor the value of AI services. Yet cloud compute pipelines centralize processing on remote servers. Data centralization reduces personal data sovereignty and may potentially degrade the quality of service (QoS). Meanwhile, user contributions are diverse in quantity and quality: decentralized records can be biased, noisy, and heterogeneously distributed. To address the data challenge, we study fair token allocation and private data valuation for decentralized and resource-constrained agentic systems. Our approach embeds multi-modal representations in a shared semantic space and releases differentially private (DP) prototypes to preserve utility while reducing semantic leakage. With the DP guarantee, we design a fair token allocation scheme that rewards effective contributions and remains robust to data heterogeneity and AI resource scarcity. Extensive simulations demonstrate improved contribution-based fairness and QoS compared to standard benchmarks. The improved resistance to image reconstruction attacks indicates enhanced privacy for multi-modal personal data.

Chinese Translation

在代理系统中，人类生成的数据记录为人工智能服务的价值提供了支撑。然而，云计算管道将处理集中在远程服务器上，数据集中化降低了个人数据主权，并可能导致服务质量（QoS）的下降。同时，用户贡献在数量和质量上存在差异：去中心化的记录可能存在偏差、噪声，并且分布不均。为了解决数据挑战，我们研究了去中心化和资源受限的代理系统中的公平令牌分配和私人数据评估。我们的方法将多模态表示嵌入共享语义空间，并释放差分隐私（DP）原型，以在减少语义泄露的同时保持效用。在DP保证下，我们设计了一种公平的令牌分配方案，奖励有效的贡献，并对数据异质性和人工智能资源稀缺性保持鲁棒性。大量模拟表明，与标准基准相比，基于贡献的公平性和QoS得到了改善。对图像重建攻击的增强抵抗力表明多模态个人数据的隐私得到了提升。

View on arXiv Download PDF AI Translation

cs.AI / 37 / 2606.15575

Do we have the knowledge we need? Rethinking human-AI decision-making in corporations

我们是否拥有所需的知识？重新思考企业中的人机决策

Marx, Anne S. R., Avelino, Ricardo M., Netland, Torbjørn, El-Assady, Mennatallah

Abstract

Organizational knowledge is fragmented across a variety of software systems, tacit expertise, and manual documents that have traditionally been designed for human consumption. As AI systems are increasingly deployed and granted decision-making roles, they require access to this knowledge. This raises two questions: how should organizations store and maintain knowledge so that it remains accessible to both humans and future AI systems, and how should agency be allocated between humans and AI across tasks with different risks and levels of uncertainty? In this position paper, we describe how organizational knowledge evolves and contribute a framework that maps task attributes and knowledge availability to recommended agency allocations and control mechanisms. We illustrate the applicability of the framework on two different manufacturing tasks: a routine operation (visual quality inspection) and a one-off strategic decision (factory location), and conclude with opportunities for future research.

Chinese Translation

组织知识分散在各种软件系统、隐性专业知识和传统上为人类使用而设计的手动文档中。随着人工智能（AI）系统的逐步部署并被赋予决策角色，它们需要访问这些知识。这引发了两个问题：组织应如何存储和维护知识，以确保人类和未来的AI系统都能访问，以及在不同风险和不确定性水平的任务中，如何在人工和AI之间分配代理权？在这篇立场论文中，我们描述了组织知识的演变，并提出了一个框架，将任务属性和知识可用性映射到推荐的代理分配和控制机制。我们通过两个不同的制造任务（常规操作——视觉质量检查和一次性战略决策——工厂选址）来说明该框架的适用性，并总结了未来研究的机会。

View on arXiv Download PDF AI Translation

cs.AI / 38 / 2606.15577

Large Language Models as Optimizers: A Survey of Direct vs. Tool-Augmented Approaches and Their Performance Frontiers

大型语言模型作为优化器：直接方法与工具增强方法及其性能前沿的综述

Peran, Roko, Hobor, Luka, Kovac, Mihael, Brcic, Mario

Abstract

Large Language Models (LLMs) are increasingly involved in complex mathematical optimization, even if the pragmatic user who triggers them is unaware of it. After all, many real-world problems reduce to the search for better or the best solutions. The field of LLM-as-optimizer has three paradigms: direct optimization, tool-augmented optimization, and tool-creating optimization. Direct optimization uses iterative prompting and heuristic generation to navigate solution spaces. Tool-augmented optimization translates natural language problems into formal specifications and orchestrates external solvers. Tool-creating optimization goes further, using LLMs to discover reusable algorithms or heuristics that can be deployed at zero marginal LLM cost. We describe current performance frontiers based on the benchmarks from the literature. We identify the critical reasoning gap in current architectures and argue for trade-offs between the future potential of direct optimization and the auditability of tool-augmented optimization. Even future, more powerful models might opt for tool-making to improve operational efficiency for repetitive families of problems.

Chinese Translation

大型语言模型（LLMs）越来越多地参与复杂的数学优化，即使触发它们的实际用户对此并不知情。毕竟，许多现实世界的问题归结为寻找更好或最佳的解决方案。LLM作为优化器的领域有三种范式：直接优化、工具增强优化和工具创建优化。直接优化使用迭代提示和启发式生成来导航解决方案空间。工具增强优化将自然语言问题转化为正式规范，并协调外部求解器。工具创建优化更进一步，利用LLMs发现可重用的算法或启发式方法，这些方法可以在零边际LLM成本下部署。我们基于文献中的基准描述当前的性能前沿。我们识别出当前架构中的关键推理差距，并论证了直接优化的未来潜力与工具增强优化的可审计性之间的权衡。即使未来更强大的模型也可能选择工具创建，以提高对重复性问题家族的操作效率。

View on arXiv Download PDF AI Translation

cs.AI / 39 / 2606.15579

Your Agent Has a Genome: Sequence-Level Behavioral Analysis and Runtime Governance of LLM-Powered Autonomous Agents

您的代理有一个基因组：基于序列的行为分析与 LLM 驱动的自主代理的运行时治理

Deng, Sidi

Abstract

We propose Base Sequence Analysis, a framework that encodes the runtime behavior of LLM-powered autonomous agents into compact symbolic sequences using a four-letter alphabet: X (Explore), E (Execute), P (Plan), and V (Verify). Drawing an analogy to genomic sequence analysis, we apply n-gram pattern mining, Markov transition matrices, and point-biserial correlation to 347 real-world execution traces collected from a production ReAct agent system over 8 days. Our analysis reveals that (1) the trigram P-X-P is the only statistically significant high-risk pattern, lowering success rate by 10.4%; (2) P-ratio is the strongest negative predictor of success (r=-0.256, p<0.0001); and (3) the E->V transition probability is only 2.1%, indicating a systemic verification deficit. Based on these findings, we design Governor, a three-layer runtime intervention system comprising a rule engine, a statistical accumulator, and a chi-square-based threshold adaptor. In a natural before/after deployment evaluation (N=101 vs. N=246), Governor achieves a +6.2% absolute increase in task success rate while simultaneously reducing average token consumption by 44%. To validate cross-system generality, we apply the XEPV encoding to 2,000 public SWE-agent trajectories on SWE-bench, confirming that exploration spirals and the E->V verification deficit replicate in an independent system. We outline six research directions including base sequence language models, cross-agent behavioral fingerprinting, and reward shaping, and release an open-source toolkit for reproducibility.

Chinese Translation

我们提出了基础序列分析（Base Sequence Analysis），这一框架将 LLM 驱动的自主代理的运行时行为编码为使用四个字母的紧凑符号序列：X（探索）、E（执行）、P（计划）和 V（验证）。通过类比基因组序列分析，我们对从生产 ReAct 代理系统收集的 347 条真实执行轨迹进行了 n-gram 模式挖掘、马尔可夫转移矩阵和点二元相关分析，时间跨度为 8 天。我们的分析揭示了以下几点：（1）三元组 P-X-P 是唯一具有统计显著性的高风险模式，降低成功率 10.4%；（2）P 比率是成功的最强负预测因子（r=-0.256，p<0.0001）；（3）E->V 转移概率仅为 2.1%，表明存在系统性的验证缺陷。基于这些发现，我们设计了 Governor，一个三层运行时干预系统，包括规则引擎、统计累加器和基于卡方的阈值适配器。在自然的部署前后评估中（N=101 对比 N=246），Governor 实现了任务成功率的绝对提升 6.2%，同时将平均令牌消耗减少了 44%。为了验证跨系统的普遍性，我们将 XEPV 编码应用于 SWE-bench 上的 2,000 条公共 SWE 代理轨迹，确认探索螺旋和 E->V 验证缺陷在独立系统中得以复制。我们概述了六个研究方向，包括基础序列语言模型、跨代理行为指纹识别和奖励塑造，并发布了一个开源工具包以支持可重复性。

View on arXiv Download PDF AI Translation

cs.AI / 40 / 2606.15591

Agentic Retrieval and Reinforcement Learned Equation Chains: A Controlled Generation Framework for Complex and Novel Physics Word Problems

自主检索与强化学习方程链：复杂且新颖物理文字问题的受控生成框架

Mittra, Tirthankar

Abstract

Generating high-quality Physics Word Problems (PWPs) that are novel, complex, and solvable remains a challenging and underexplored problem in educational content generation. Existing approaches, many adapted from Math Word Problem (MWP) generation, often produce ambiguous, unsolvable, or structurally simple questions with limited linguistic diversity. We introduce ARVRE (Agentic Retrieval Value Reinforced Equation-chain), a two-stage framework for generating diverse and mathematically valid PWPs. In the first stage, a form of offline temporal-difference learning is used to construct valid chains of physics equations, while an agentic retrieval-augmented generation (RAG) framework dynamically selects topic-specific concepts and vocabulary. This design enables explicit control over problem structure and difficulty. In the second stage, a Large Language Model (LLM) converts the equation chain and retrieved concepts into a natural-language physics question. By grounding generation in valid equation chains, our method preserves mathematical correctness while promoting linguistic diversity and contextual richness. Human and automated evaluations demonstrate that ARVRE generates PWPs that are more complex, novel, and solvable than those produced by existing approaches. These results highlight the potential of combining reinforcement learning, retrieval, and LLMs for reliable generation of educational physics content.

Chinese Translation

生成高质量的新颖、复杂且可解的物理文字问题（PWPs）仍然是教育内容生成中一个具有挑战性且尚未深入研究的问题。现有的方法，许多是从数学文字问题（MWP）生成中改编而来，往往会产生模糊、不可解或结构简单的问题，且语言多样性有限。我们提出了ARVRE（自主检索价值强化方程链），这是一个用于生成多样化且数学有效的PWPs的两阶段框架。在第一阶段，使用一种离线时序差分学习的方法构建有效的物理方程链，同时，自主检索增强生成（RAG）框架动态选择特定主题的概念和词汇。这一设计使得对问题结构和难度的显式控制成为可能。在第二阶段，大型语言模型（LLM）将方程链和检索到的概念转换为自然语言的物理问题。通过将生成过程基于有效的方程链，我们的方法在保持数学正确性的同时，促进了语言多样性和上下文丰富性。人类和自动化评估表明，ARVRE生成的PWPs在复杂性、新颖性和可解性方面优于现有方法生成的问题。这些结果突显了结合强化学习、检索和LLM在可靠生成教育物理内容方面的潜力。

View on arXiv Download PDF AI Translation

cs.AI / 41 / 2606.15598

Integrating Reasoning and Generalization in Text-to-SQL via Self-Enhanced Fine-Tuning

通过自增强微调在文本到SQL中整合推理与泛化

Lyu, Feng, Cen, Jinfeng, Duan, Sijing, Wu, Hao, Li, Shucheng, Zhang, Weixu, Wu, Haolun

Abstract

Text-to-SQL aims to translate natural language questions into executable SQL queries over structured databases, enabling non-expert users to access data intuitively. While recent advances in large language models (LLMs) have shown promise in this task, existing LLM-based approaches often struggle to strike a balance between strong reasoning capabilities and robust generalization. To address these limitations, we propose CoTE-SQL to enhance the LLM-based text-to-SQL generation with three key innovations: (i) self-enhanced reasoning traces distilled from LLMs without human annotation, (ii) structured chain-of-thought (CoT) prompting with modular decomposition and examples retrieval, and (iii) error-aware revision based on SQL execution feedback. Extensive experiments on the Spider and Bird benchmarks demonstrate that CoTE-SQL achieves new state-of-the-art performance among methods built on open-source LLMs with comparable model sizes on Bird (53.39% EX / 59.02 VES) and strong results on Spider (79.60% EX / 77.19 VES), with especially significant gains on complex queries. Results highlight the effectiveness of combining self-enhancement, structured reasoning, and execution-time feedback within an LLM-based framework for text-to-SQL design.

Chinese Translation

文本到SQL旨在将自然语言问题翻译为可在结构化数据库上执行的SQL查询，使非专业用户能够直观地访问数据。尽管近期大型语言模型（LLMs）的进展在这一任务中显示出潜力，但现有的基于LLM的方法往往难以在强大的推理能力和稳健的泛化之间取得平衡。为了解决这些局限性，我们提出了CoTE-SQL，通过三项关键创新来增强基于LLM的文本到SQL生成：（i）从LLM中提炼的自增强推理轨迹，无需人工标注；（ii）具有模块化分解和示例检索的结构化思维链（CoT）提示；（iii）基于SQL执行反馈的错误感知修正。在Spider和Bird基准上的广泛实验表明，CoTE-SQL在基于开源LLM且具有可比模型规模的方法中实现了新的最先进性能，在Bird上达到53.39%的EX和59.02%的VES，在Spider上取得79.60%的EX和77.19%的VES，尤其在复杂查询上表现出显著提升。结果突显了在基于LLM的文本到SQL设计框架中结合自增强、结构化推理和执行时反馈的有效性。

View on arXiv Download PDF AI Translation

cs.AI / 42 / 2606.15646

NeuroSymbolic AI for Legal AI-TRISM: Trustworthy, Reliable, Interpretable, Safe Models

法律人工智能的神经符号人工智能——可信、可靠、可解释、安全的模型

Tilwani, Deepa, Saxena, Yash, Padia, Ankur, Parthasarathy, Srinivasan, Gaur, Manas

Abstract

Large Language Models (LLMs) have transformed natural language processing, but their lack of interpretable reasoning and tendency to hallucinate pose significant challenges for legal applications. While LLMs show promise for legal text analysis and generation, they struggle with accurate citation attribution and precedent verification. For example, in legal contexts, a single incorrect precedent can jeopardize a case. Current approaches to improve LLM reliability in legal domains suffer from two key limitations: inadequate integration of structured legal knowledge during training or fine-tuning, and insufficient verification mechanisms for generated legal content. To address these challenges, we propose the TRISM (Trustworthy, Reliable, Interpretable, Safe Models) framework, which integrates NeuroSymbolic AI principles with LLMs to leverage both neural learning capabilities and symbolic reasoning over structured legal knowledge. The TRISM approach addresses the above limitations while maintaining interpretable decision pathways. Our framework formalizes the extraction of symbolic knowledge from legal textual documents and incorporates Retrieval-Augmented Generation (RAG) as a core component for grounding LLM outputs in verified legal sources. In this position paper, we make the following contributions: (1) An analysis of the limitations of AI in law; (2) Introduce RASOR RAG which creates foundations for neurosymbolic RAG by generating explicit interpretable rationales that could be formalized into symbolic representations; (3) A formalized methodology for creating symbolic legal knowledge bases that support both interpretable reasoning and output verification in LLMs; and (4) The TRISM framework for integrating symbolic legal knowledge with LLMs.

Chinese Translation

大型语言模型（LLMs）已经改变了自然语言处理，但它们缺乏可解释的推理能力和容易产生幻觉的倾向，为法律应用带来了重大挑战。尽管LLMs在法律文本分析和生成方面展现出潜力，但在准确的引用归属和判例验证方面仍然存在困难。例如，在法律环境中，一个错误的判例可能会危及案件的结果。目前提高LLM在法律领域可靠性的现有方法存在两个主要局限性：在训练或微调过程中对结构化法律知识的整合不足，以及对生成法律内容的验证机制不够充分。为了解决这些挑战，我们提出了TRISM（可信、可靠、可解释、安全的模型）框架，该框架将神经符号人工智能原则与LLMs相结合，利用神经学习能力和对结构化法律知识的符号推理。TRISM方法解决了上述局限性，同时保持可解释的决策路径。我们的框架形式化了从法律文本文件中提取符号知识的过程，并将检索增强生成（Retrieval-Augmented Generation, RAG）作为核心组件，以将LLM输出与经过验证的法律来源相结合。在这篇立场论文中，我们做出以下贡献：（1）分析人工智能在法律领域的局限性；（2）介绍RASOR RAG，该方法为神经符号RAG奠定基础，通过生成明确的可解释推理，能够形式化为符号表示；（3）创建支持可解释推理和LLM输出验证的符号法律知识库的形式化方法；（4）提出TRISM框架，用于将符号法律知识与LLMs整合。

View on arXiv Download PDF AI Translation

cs.AI / 43 / 2606.15647

Towards Next-Generation Healthcare: A Survey of Medical Embodied AI for Perception, Decision-Making, and Action

迈向下一代医疗保健：医疗具身人工智能在感知、决策和行动中的调查

Zhang, Cheng, Cai, Qing, Wu, Xingzheng, Yang, Xun, Chang, Xiaojun, Bao, Bingkun, Nie, Liqiang, Liu, Xinwang, Yang, Yi

Abstract

Foundation models have demonstrated impressive performance in enhancing healthcare efficiency across a wide range of medical applications. Nevertheless, their limited ability to perceive, understand, and interact with the physical world significantly constrains their effectiveness in real-world clinical workflows, where safety-critical decision-making and physical execution are tightly coupled. Recently, embodied artificial intelligence (AI) has emerged as a promising physical-interactive paradigm for intelligent healthcare, enabling agents to operate in complex medical environments. As research in this area rapidly expands, understanding how intelligent agents function as integrated, end-to-end systems in clinical environments becomes increasingly critical. However, existing surveys on medical embodied AI largely emphasize individual aspects or functional components, lacking a unified system-level organization of the field. To support and consolidate recent advances, we systematically survey the core components of medical embodied AI, with a particular emphasis on the coordinated integration of perception, decision-making, and action. We further review representative medical applications and relevant datasets, and we analyze the major challenges encountered in real-world clinical practice. Finally, we discuss key directions for future research in this rapidly evolving field. The associated project can be found at https://github.com/VMVLab/Medical_Embodied_AI_Paper_List.

Chinese Translation

基础模型在提升医疗保健效率方面在广泛的医疗应用中表现出色。然而，它们在感知、理解和与物理世界互动方面的能力有限，显著限制了它们在现实临床工作流程中的有效性，在这些流程中，安全关键的决策和物理执行紧密相连。最近，具身人工智能（embodied AI）作为一种有前景的物理交互范式在智能医疗保健中崭露头角，使得代理能够在复杂的医疗环境中操作。随着该领域研究的快速扩展，理解智能代理如何作为集成的端到端系统在临床环境中运作变得愈发重要。然而，现有的医疗具身人工智能调查主要强调个别方面或功能组件，缺乏对该领域的统一系统级组织。为了支持和整合近期的进展，我们系统性地调查了医疗具身人工智能的核心组成部分，特别强调感知、决策和行动的协调整合。我们进一步回顾了具有代表性的医疗应用和相关数据集，并分析了在现实临床实践中遇到的主要挑战。最后，我们讨论了在这一快速发展的领域中未来研究的关键方向。相关项目可在 https://github.com/VMVLab/Medical_Embodied_AI_Paper_List 找到。

View on arXiv Download PDF AI Translation

cs.AI / 44 / 2606.15655

Advanced Machine Learning and Deep Learning Techniques for Enhanced Cattle Identification and Detection: A Comprehensive Review

先进的机器学习和深度学习技术在增强牛只识别与检测中的应用：一项综合评述

Chowdhury, Fayazunnesa, Galib, Syed Md., Adnan, Md Nasim, Siddique, Md. Moradul, Karim, Md Robiul, Anjum, K M Tanvir

Abstract

The need for effective cattle identification technology is now more acutely felt than ever in maintaining biosecurity, food safety, and supply chain efficacy in livestock management. This paper presents a systematic review of recent research in cattle identification using machine learning and deep learning techniques. The present systematic review measures the effectiveness of traditional and modern cattle identification techniques using studies from major academic databases, where articles were subjected to full-text review. Among these techniques, classical Machine Learning Techniques such as K-Nearest Neighbors and Support Vector Machines have demonstrated good results in cattle identification; however, Deep Learning Techniques, such as Convolutional Neural Networks, Residual Networks, and You Only Look Once, are better in cognition, detection, and identification tasks. Feature extraction relies on common techniques like Local Binary Pattern (LBP), Speeded-Up Robust Features (SURF), and Scale-Invariant Feature Transform (SIFT), while key features commonly used in these studies include muzzle prints and coat patterns. The review highlights key hurdles involving cattle identification, such as the limited number of publicly accessible datasets, issues with data quality susceptible to environmental changes and animal mobility, and high demand for real-time processing ability. The paper aims to inform researchers, policymakers, and stakeholders about implementing scalable, humane, and effective cattle identification systems to achieve sustainable livestock management.

Chinese Translation

在维护生物安全、食品安全和供应链效率方面，牛只识别技术的有效性需求比以往任何时候都更加迫切。本文对使用机器学习和深度学习技术进行牛只识别的近期研究进行了系统评述。本系统评述通过主要学术数据库中的研究，评估了传统与现代牛只识别技术的有效性，所选文章均经过全文审查。在这些技术中，经典的机器学习技术如K-最近邻（K-Nearest Neighbors）和支持向量机（Support Vector Machines）在牛只识别中表现良好；然而，深度学习技术如卷积神经网络（Convolutional Neural Networks）、残差网络（Residual Networks）和You Only Look Once在认知、检测和识别任务中表现更佳。特征提取依赖于常用技术，如局部二值模式（Local Binary Pattern, LBP）、加速稳健特征（Speeded-Up Robust Features, SURF）和尺度不变特征变换（Scale-Invariant Feature Transform, SIFT），而这些研究中常用的关键特征包括鼻印和毛色图案。评述强调了牛只识别中面临的主要障碍，如公开可获取数据集的数量有限、易受环境变化和动物活动影响的数据质量问题，以及对实时处理能力的高需求。本文旨在向研究人员、政策制定者和利益相关者提供信息，以实施可扩展、人道和有效的牛只识别系统，从而实现可持续的畜牧管理。

View on arXiv Download PDF AI Translation

cs.AI / 45 / 2606.15656

Overcoming the Impedance Mismatch: A Theoretical Roadmap for Fusing Foundation Models and Knowledge Graphs

克服阻抗不匹配：融合基础模型与知识图谱的理论路线图

Dhayalkar, Sahil Rajesh

Abstract

Modern artificial intelligence remains fundamentally divided between the continuous, probabilistic spaces of Foundation Models and the discrete, deterministic structures of Knowledge Graphs. While Retrieval-Augmented Generation (RAG) attempts to connect them by serializing graph data into text, we argue this lexical bridging is merely a superficial patch. In this paper, we formalize the underlying structural and geometric friction as the \textit{Impedance Mismatch}. By categorizing current neuro-symbolic integration strategies into a three-tiered hierarchy, we demonstrate that neither surface-level prompt injection nor continuous representation alignment can preserve the strict logical motifs required for reliable multi-hop reasoning. We define the specific mathematical limits, such as the Lexical Bottleneck and Topological Collapse, that show current architectures will eventually hallucinate or conflate semantic nodes. To achieve true semantic fusion, we propose a rigorous theoretical roadmap. We advocate for natively internalizing discrete symbolic structures through Structured Residual Streams, utilizing Vector Symbolic Architectures for latent sub-graph injection, and performing model updates via Orthogonal Subspace Editing. This actionable framework paves the way for models that seamlessly fuse the precision of symbolic logic with the expressivity of parametric memory.

Chinese Translation

现代人工智能在基础模型的连续概率空间与知识图谱的离散确定性结构之间仍然存在根本性的分歧。尽管检索增强生成（Retrieval-Augmented Generation, RAG）试图通过将图数据序列化为文本来连接这两者，但我们认为这种词汇桥接仅仅是表面的修补。在本文中，我们将潜在的结构和几何摩擦形式化为 extit{阻抗不匹配}。通过将当前的神经符号集成策略分类为三级层次结构，我们证明了无论是表层的提示注入还是连续表示对齐都无法保持可靠的多跳推理所需的严格逻辑模式。我们定义了特定的数学限制，如词汇瓶颈（Lexical Bottleneck）和拓扑崩溃（Topological Collapse），这些限制表明当前架构最终会产生幻觉或混淆语义节点。为了实现真正的语义融合，我们提出了一条严格的理论路线图。我们倡导通过结构残差流（Structured Residual Streams）本地内化离散符号结构，利用向量符号架构（Vector Symbolic Architectures）进行潜在子图注入，并通过正交子空间编辑（Orthogonal Subspace Editing）进行模型更新。这一可操作的框架为模型的无缝融合符号逻辑的精确性与参数记忆的表现力铺平了道路。

View on arXiv Download PDF AI Translation

cs.AI / 46 / 2606.15673

Where Did It Go Wrong? Process-Level Evaluation of Web Agents with Semantic State Tracking

哪里出了问题？具有语义状态跟踪的网络代理的过程级评估

Chung, Jiwan, Byun, JiHyuk, Vineet, Vibhav, Kim, Seon Joo

Abstract

Web agents act through long interaction sequences, yet existing benchmarks evaluate only terminal success, discarding all process information and offering little guidance on improvement. In this work, we conduct a process-level analysis of web agents. We introduce WebStep, a benchmark of 1,800 task instances with controlled difficulty and automatic semantic state tracking. Each website exposes a deterministic semantic MDP alongside the GUI: the agent operates on the interface, while the environment records high-level states and transitions in the background, enabling fine-grained analysis without manual annotation. Based on the semantic trajectory, we first show that process metrics reveal differences invisible to outcome evaluation: three agents whose success rates cluster within 31-33% diverge in exploration reach versus execution accuracy. Then, decomposing by skill characterizes the nature of these differences, exposing opposite per-skill rankings hidden within the same website: e.g., on Housing, OpenAI CUA outperforms Qwen3.5 by 23.7% on commit actions yet underperforms it by 15.6% on filtering, pinpointing a concrete skill to improve even within a domain. Bifurcation analysis further localizes the decisive error that loses the task and shows that this error is agent-specific rather than shared. Finally, these differences widen as tasks grow harder: success rate is similar on easy tasks but separates sharply as exploration becomes more demanding. Our process-level analysis opens a new avenue in web agent evaluation, providing fine-grained and actionable insight into where and how each agent should be improved.

Chinese Translation

网络代理通过长时间的交互序列进行操作，但现有的基准仅评估最终成功，忽略了所有过程信息，提供的改进指导有限。在本研究中，我们对网络代理进行了过程级分析。我们引入了WebStep，这是一个包含1800个任务实例的基准，具有可控的难度和自动语义状态跟踪。每个网站都展示了一个确定性的语义马尔可夫决策过程（MDP）以及图形用户界面（GUI）：代理在界面上操作，而环境在后台记录高层状态和转变，从而实现无手动标注的细粒度分析。基于语义轨迹，我们首先展示了过程指标揭示了结果评估无法察觉的差异：三种成功率聚集在31-33%之间的代理在探索范围与执行准确性上存在差异。然后，通过技能分解来表征这些差异的性质，揭示了在同一网站内隐藏的相反技能排名：例如，在Housing上，OpenAI CUA在提交操作上比Qwen3.5高出23.7%，但在过滤上则低于其15.6%，明确指出了即使在同一领域中需要改进的具体技能。分岔分析进一步定位了导致任务失败的决定性错误，并显示该错误是特定于代理的，而非共享的。最后，随着任务难度的增加，这些差异进一步扩大：在简单任务上的成功率相似，但随着探索变得更加困难，成功率迅速分离。我们的过程级分析为网络代理评估开辟了一条新途径，提供了细粒度和可操作的洞察，明确了每个代理应如何以及在哪里进行改进。

View on arXiv Download PDF AI Translation

cs.AI / 47 / 2606.15684

Multi-agent Framework for Time-Sensitive Complementary Collaboration in Minecraft

用于时间敏感的互补协作的多智能体框架在Minecraft中的应用

Yi, Juheon, Wang, Jinglu, Zhang, Xiaoyi, Lu, Yan

Abstract

We present TickingCollabBench, a Minecraft-based multi-agent benchmark for a novel class of time-sensitive complementary collaboration tasks. Our benchmark reflects four core characteristics of real-world collaboration: agent heterogeneity, mandatory collaboration, dynamic environments, and strict real-time constraints with failure risks. To enable this, we develop the TickingCollab framework, which supports the generation of diverse dynamic environments and abstracts Minecraft's primitive APIs to enable declarative YAML task specifications for composing these events. Building on this, we design a feasibility-aware automated benchmark generation pipeline, where an LLM drafts structurally diverse task configurations and feasibility verifier filters out invalid ones using approximate constraints. Evaluations demonstrate that lang latency and inherent difficulty of coordinating under partial observability and agent heterogeneity cause LLMs to frequently fail under dynamic environments and fall significantly short of a global-knowledge oracle.

Chinese Translation

我们提出了TickingCollabBench，这是一个基于Minecraft的多智能体基准，用于一种新型的时间敏感互补协作任务。我们的基准反映了现实世界协作的四个核心特征：智能体异质性、强制协作、动态环境和严格的实时约束以及失败风险。为此，我们开发了TickingCollab框架，支持多样化动态环境的生成，并抽象了Minecraft的原始API，以便能够使用声明性YAML任务规范来组合这些事件。在此基础上，我们设计了一个考虑可行性的自动化基准生成管道，其中一个大型语言模型（LLM）起草结构多样的任务配置，而可行性验证器则使用近似约束过滤掉无效配置。评估结果表明，语言延迟和在部分可观察性及智能体异质性下协调的固有难度导致LLM在动态环境中频繁失败，且远未达到全局知识预言机的水平。

View on arXiv Download PDF AI Translation

cs.AI / 48 / 2606.15686

Recurrent Reasoning on Symbolic Puzzles with Sequence Models

基于序列模型的符号谜题递归推理

Mannem, Gowrav, Mahjabin, Chowdhury Marzia, Chen, Jason, Garg, Shivank, Zhu, Kevin

Abstract

Large language models often appear strong on symbolic and algorithmic tasks, yet this apparent strength can hide brittle behaviour when problems become longer, harder, or slightly out of distribution. A major limitation of current reasoning benchmarks is that many primarily test whether a model can produce a valid answer, while paying less attention to whether the solution is minimal, robust, and stable under controlled difficulty scaling. We introduce RecurrReason, a difficulty-controlled benchmark of four recurrent logic puzzles (Tower of Hanoi, River Crossing, Block World, and Checkers Jumping) with BFS-optimal trajectories and a single interpretable difficulty parameter $N \in \{1,\dots,10\}$, totalling 10{,}817 unique puzzles and 285{,}933 moves. We benchmark two Transformer families, an encoder-decoder model (T5-style) and a decoder-only model (GPT-2-style), under consistent data splits and evaluation criteria, training on $N{=}1$ to $7$ and evaluating on both held-out in-distribution instances and harder out-of-distribution instances at $N{=}8$ to $10$. Fine-tuned pre-trained T5 achieves 97.27\% validation and 81.00\% OOD accuracy on Block World; all models score 0.00\% on River Crossing under all conditions. Failure mode analysis reveals that architecture is a stronger determinant of success than scale. Pre-training transfers only to puzzles with locally structured transition functions. Our code and dataset will be open-sourced upon acceptance.

Chinese Translation

大型语言模型在符号和算法任务上通常表现出色，然而这种表面上的强大可能掩盖了在问题变得更长、更难或稍微超出分布时的脆弱行为。目前推理基准的一个主要限制是，许多基准主要测试模型是否能够产生有效答案，而较少关注解决方案是否是最小的、稳健的，并且在控制难度扩展下是否稳定。我们引入了RecurrReason，这是一个控制难度的基准，包含四个递归逻辑谜题（汉诺塔、过河问题、方块世界和跳棋），具有BFS最优轨迹和一个可解释的难度参数$N ext{ in } ext{ extbackslash{}{1, extellipsis,10 extbackslash{}}}$，总计10,817个独特谜题和285,933个移动。我们在一致的数据划分和评估标准下，对两种Transformer模型进行基准测试：一种编码-解码模型（T5风格）和一种仅解码模型（GPT-2风格），训练范围为$N{=}1$到$7$，并在$N{=}8$到$10$的保留分布实例和更难的超出分布实例上进行评估。经过微调的预训练T5在方块世界上达到了97.27 ext{ extbackslash%}的验证准确率和81.00 ext{ extbackslash%}的超出分布准确率；在所有条件下，所有模型在过河问题上的得分均为0.00 ext{ extbackslash%}。失败模式分析表明，架构是成功的更强决定因素，而非规模。预训练仅转移到具有局部结构转移函数的谜题上。我们的代码和数据集将在接受后开源。

View on arXiv Download PDF AI Translation

cs.AI / 49 / 2606.15696

Do LLMs Reliably Identify Correct Information Units in Aphasic Discourse?

大型语言模型能否可靠地识别失语症话语中的正确信息单元？

Pittman, Jason M, Medina-Santos, Yesenia, Phillips Jr., Anton, Stark, Brielle C.

Abstract

Correct Information Units (CIUs) are central to discourse assessment in aphasia because they quantify communicative informativeness rather than linguistic form alone. However, CIU scoring is time intensive and requires trained raters. This study examined whether instruction-tuned large language models (LLMs) can reliably perform token-level CIU classification from aphasic discourse transcripts. Sixteen picture-description transcripts elicited with the Cat Rescue stimulus were annotated for CIU status according to Nicholas and Brookshire (1993). The sample spanned four severity strata: control, mild, moderate, and severe aphasia. Four publicly available instruction-tuned LLMs were benchmarked under zero-shot and two few-shot prompting conditions across five stratified random seeds. Performance was evaluated against consensus human labels using accuracy, precision, recall, F1, and Cohen's kappa. Zero-shot prompting was insufficient across models. In contrast, few-shot prompting yielded substantial gains and produced competitive performance for three viable models. Mean few-shot F1 scores ranged from 0.776 to 0.817 across Llama-3.1-8B, Qwen2.5-7B, and Mistral-7B, with no significant differences between fixed global and per-chunk local example selection. Phi-3-mini was unstable and did not yield reliable performance. Viable models showed high recall but lower precision, suggesting systematic over-classification of tokens as CIUs. Performance also varied by discourse severity, with the weakest results in more severe aphasia. Few-shot LLM prompting can support automated CIU identification without gradient-based task training, but agreement with human annotation remains insufficient for fully autonomous use. These findings support LLM-based CIU scoring as a promising human-in-the-loop component of discourse assessment systems.

Chinese Translation

正确信息单元（CIUs）在失语症的话语评估中至关重要，因为它们量化了交流的信息量，而不仅仅是语言形式。然而，CIU评分耗时且需要经过培训的评分者。本研究考察了经过指令调优的大型语言模型（LLMs）是否能够可靠地从失语症话语转录中执行基于标记的CIU分类。使用Cat Rescue刺激 elicited 的十六个图片描述转录根据Nicholas和Brookshire（1993）的标准进行了CIU状态的注释。样本涵盖了四个严重程度层次：对照组、轻度、 moderate和重度失语症。在五个分层随机种子下，四个公开可用的指令调优LLMs在零样本和两个少样本提示条件下进行了基准测试。性能通过准确率、精确率、召回率、F1值和Cohen's kappa与人类共识标签进行评估。零样本提示在各模型中均不够充分。相比之下，少样本提示带来了显著的提升，并为三个可行模型提供了竞争性的表现。Llama-3.1-8B、Qwen2.5-7B和Mistral-7B的平均少样本F1得分范围为0.776到0.817，固定全局和每块局部示例选择之间没有显著差异。Phi-3-mini不稳定，未能提供可靠的表现。可行模型显示出高召回率但较低的精确率，表明系统性地将标记过度分类为CIUs。性能还因话语的严重程度而异，在更严重的失语症中结果最弱。少样本LLM提示可以支持自动化CIU识别，而无需基于梯度的任务训练，但与人类注释的一致性仍不足以实现完全自主使用。这些发现支持基于LLM的CIU评分作为话语评估系统中一种有前景的人机协作组件。

View on arXiv Download PDF AI Translation

cs.AI / 50 / 2606.15708

Artificial Intelligence Index Report 2026

2026年人工智能指数报告

Sajadieh, Sha, Fattorini, Loredana, Perrault, Raymond, Gil, Yolanda, Parli, Vanessa, Santarlasci, Lapo, Pava, Juan, Maslej, Nestor, Altman, Russ, Brynjolfsson, Erik, Brodley, Carla, Clark, Jack, Dignum, Virginia, Kumar, Vipin, Landay, James, Lyons, Terah, Manyika, James, Niebles, Juan Carlos, Shoham, Yoav, Tabassi, Elham, Wald, Russell, Walsh, Toby, Weld, Dan

Abstract

Welcome to the ninth edition of the AI Index report. As AI continues to advance rapidly, the question becomes whether the systems built around it can keep up. Governance frameworks, evaluation methods, education systems, and the data infrastructure needed to track AI's impact are struggling to match the pace of the technology itself. That gap between what AI can do and how prepared we are to manage it runs through every chapter of this year's report. New in this edition, the report tracks how AI is being tested more ambitiously across reasoning, safety, and real-world task execution, and why those measurements are increasingly difficult to rely on. It also features new estimates of generative AI's economic value alongside emerging evidence of its labor market effects, an analytical framework on AI sovereignty, and a science chapter developed in collaboration with Schmidt Sciences. For the first time, the report features standalone chapters on AI in science and AI in medicine, reflecting AI's growing impact across these two domains.

Chinese Translation

欢迎阅读第九版人工智能指数报告。随着人工智能的快速发展，问题在于围绕其构建的系统是否能够跟上这一进程。治理框架、评估方法、教育系统以及跟踪人工智能影响所需的数据基础设施正努力与技术本身的步伐相匹配。今年报告的每一章都贯穿着人工智能能够做什么与我们准备好管理它之间的差距。本版报告的新内容追踪了人工智能在推理、安全性和现实任务执行方面的更雄心勃勃的测试，以及为什么这些测量结果越来越难以依赖。报告还提供了生成性人工智能经济价值的新估算，以及其对劳动市场影响的新证据，分析了人工智能主权的框架，并与施密特科学公司合作开发了科学章节。首次，报告单独设立了关于科学中的人工智能和医学中的人工智能的章节，反映了人工智能在这两个领域日益增长的影响。

View on arXiv Download PDF AI Translation

cs.AI / 51 / 2606.15709

AI-Driven Framework for Adaptive Water Network Management with Proof-of-Concept Implementation: Addressing Non-Revenue Water in Jordan

基于人工智能的自适应水网管理框架及其概念验证实施：解决约旦的非收入水问题

Fasha, Mohammed, Al-Maayta, Nahel, Sowan, Bilal, Athamneh, Mohammad, Barham, Husam

Abstract

Jordan faces severe water scarcity with 50\% of water produced is lost to leakage, theft and metering issues also known as non-revenue water (NRW). Traditional reactive approaches have proven insufficient for sustained NRW reduction. This paper proposes an intelligent framework integrating EPANET hydraulic modeling, digital twin technology, SCADA systems, and large language model (LLM)-based AI agents for continuous network monitoring and adaptive decision-making. The system combines real-time data streams with physics-based simulation to detect anomalies, employing retrieval-augmented generation (RAG) for policy interpretation and function calling for network control. A proof-of-concept implementation validates technical feasibility using EPYT with offline LLMs (llama3.1:8b via Ollama) on a 1,164-junction Amman district network. The system demonstrates automated hydraulic simulation, flow-based anomaly detection aligned with water distribution zone (DZ) practice, and AI-generated health reports with response times under 2 minutes and zero API costs. Burst detection relies on local flow anomaly analysis: a 30.1~L/s simulated leak produces measurable flow redistribution in 15 pipes, flagging a 15-junction cluster that localises the burst -- confirming alignment with water distribution zone (DZ) monitoring practice. The framework accommodates Jordan's intermittent supply patterns and limited automation through phased implementation, offering a scalable pathway for water-scarce regions to leverage intelligent automation for NRW reduction and operational efficiency.

Chinese Translation

约旦面临严重的水资源短缺，50\% 的水源因漏水、盗水和计量问题而损失，这被称为非收入水（NRW）。传统的反应性方法已被证明不足以持续减少非收入水。本文提出了一种智能框架，整合了EPANET水力模型、数字双胞胎技术、SCADA系统和基于大型语言模型（LLM）的人工智能代理，以实现持续的网络监测和自适应决策。该系统将实时数据流与基于物理的模拟相结合，以检测异常，采用检索增强生成（RAG）进行政策解释，并通过函数调用进行网络控制。概念验证实施使用EPYT和离线LLM（llama3.1:8b via Ollama）在1164个接点的安曼区域网络上验证了技术可行性。该系统展示了自动化的水力模拟、基于流量的异常检测与水分配区（DZ）实践相一致，以及AI生成的健康报告，响应时间在2分钟以内且无API成本。突发检测依赖于局部流量异常分析：模拟的30.1 L/s漏水在15条管道中产生可测量的流量重分配，标记出一个15接点的聚类，定位突发事件——确认与水分配区（DZ）监测实践的一致性。该框架通过分阶段实施，适应约旦间歇性的供水模式和有限的自动化，为水资源匮乏地区提供了一条可扩展的路径，以利用智能自动化减少非收入水和提高运营效率。

View on arXiv Download PDF AI Translation

cs.AI / 52 / 2606.15753

RoboPIN: Grounded Embodied Reasoning via Pinned Chain-of-Thought

RoboPIN：通过固定思维链实现的具身推理

Huang, Yaoting, Yuan, Yifu, Han, Linqi, Li, Chengwen, Zhang, Shuoheng, Yao, Xianze, Tang, Hongyao, Zheng, Yan, Hao, Jianye

Abstract

Embodied reasoning requires models to perceive task-relevant objects and spaces in physical environments and maintain consistent visual grounding throughout multi-step reasoning. However, current vision-language models rely on text-only or coordinate-augmented chain-of-thought, where entity references remain implicit and ambiguous. This may cause the reasoning process to decouple from visual evidence, entity references to drift across steps, and a causal disconnection between the reasoning trajectory and the final answer, with these problems further amplified in multi-view scenarios due to cross-view appearance changes. To address these issues, we propose Pinned Chain-of-Thought (\pincot{}), a structured reasoning paradigm that pins every reasoning step to visual evidence. \pincot{} introduces the concept of \reasoninganchor{}, which binds each task-relevant entity to a structured visual anchor with entity name, unique identity, view index, and spatial grounding, enabling consistent entity tracking across reasoning steps and views. We build a fully automated data generation pipeline to construct \dataset{}, a high-quality \pincot{}-formatted reasoning dataset. We then train \method{} through three-stage post-training that progressively injects embodied knowledge, structured reasoning ability, and process-supervised alignment, with rewards that directly constrain both anchor localization and identity consistency during reasoning. On 14 benchmarks covering embodied spatial reasoning, multi-view reasoning, and pointing, \method{} with only 4B parameters consistently outperforms 7B level open-source embodied models, achieving a 12\% average improvement over the strongest 7B baseline, Mimo-Embodied. Further analysis shows that \pincot{} improves grounding accuracy and cross-step identity consistency, validating the effectiveness of process supervision.

Chinese Translation

具身推理要求模型在物理环境中感知与任务相关的对象和空间，并在多步骤推理过程中保持一致的视觉基础。然而，当前的视觉-语言模型依赖于仅文本或坐标增强的思维链，其中实体引用保持隐含和模糊。这可能导致推理过程与视觉证据脱节，实体引用在步骤之间漂移，以及推理轨迹与最终答案之间的因果断裂，这些问题在多视角场景中由于视角间外观变化而进一步加剧。为了解决这些问题，我们提出了固定思维链（Pinned Chain-of-Thought， extit{pincot}），一种将每个推理步骤固定在视觉证据上的结构化推理范式。 extit{pincot}引入了 extit{reasoning anchor}的概念，将每个与任务相关的实体绑定到一个结构化的视觉锚点，该锚点包含实体名称、唯一身份、视角索引和空间基础，从而实现跨推理步骤和视角的一致实体跟踪。我们构建了一个完全自动化的数据生成管道，以构建 extit{dataset}，这是一个高质量的 extit{pincot}格式推理数据集。然后，我们通过三阶段的后训练来训练 extit{method}，该过程逐步注入具身知识、结构化推理能力和过程监督对齐，奖励直接约束推理过程中的锚点定位和身份一致性。在涵盖具身空间推理、多视角推理和指向的14个基准测试中，只有4B参数的 extit{method}始终优于7B级的开源具身模型，较最强的7B基线Mimo-Embodied平均提高了12%。进一步分析表明， extit{pincot}提高了基础准确性和跨步骤身份一致性，验证了过程监督的有效性。

View on arXiv Download PDF AI Translation

cs.AI / 53 / 2606.15766

Rethinking Scaffolding in LLM Tutors: The Interactional Mismatch Between Benchmarks and Real-World Deployments

重新思考大型语言模型辅导中的支架支持：基准测试与实际应用之间的互动不匹配

Neagu, Alexandra, Wong, Jeffrey T. H., Messer, Marcus, Nelson, Rhodri, Johnson, Peter B.

Abstract

A central pedagogical value evaluated in AI tutor benchmarks is scaffolding: guiding students through graduated steps toward a solution. Alignment and evaluation methods for embedding scaffolding behaviour into chatbots, however, rest on an implicit assumption: that students will take up the scaffolding and engage in the conversation. To examine whether this assumption holds, we introduce an evaluation pipeline around two metrics - Chatbot Scaffolding and Student Uptake - and apply them across nine datasets of 9,490 chats, spanning AI tutor benchmarks and real-world deployments of educational chatbots. Our analysis reveals that while benchmarks assume a high-scaffolding, high-student-uptake environment, students in real-world settings exhibit lower levels of uptake overall - frequently bypassing the chatbot's pedagogical framing to drive the interaction toward their own learning goals at little interpersonal cost. We argue that bypassing scaffolding is not necessarily detrimental; rather, it frequently highlights a mismatch between a chatbot's pedagogical framing and the student's learning goals. To meaningfully evaluate the effectiveness of a chatbot's assistance, future benchmarks must move beyond the assumption that students will simply take up the scaffolding, and instead evaluate how these chatbots navigate diverse learning contexts and student-driven interaction patterns.

Chinese Translation

在人工智能辅导基准测试中评估的一个核心教学价值是支架支持：通过逐步引导学生找到解决方案。然而，嵌入支架行为到聊天机器人中的对齐和评估方法基于一个隐含假设：学生会接受支架支持并参与对话。为了检验这一假设是否成立，我们引入了一个围绕两个指标的评估流程——聊天机器人支架支持和学生接受度，并将其应用于九个数据集中的9,490次聊天，这些数据集涵盖了人工智能辅导基准测试和教育聊天机器人的实际应用。我们的分析揭示，尽管基准测试假设存在高支架支持和高学生接受度的环境，但在实际环境中，学生的接受度整体较低——他们常常绕过聊天机器人的教学框架，将互动引向自己的学习目标，几乎没有人际成本。我们认为，绕过支架支持并不一定是有害的；相反，它常常突显了聊天机器人的教学框架与学生学习目标之间的不匹配。为了有意义地评估聊天机器人辅助的有效性，未来的基准测试必须超越假设学生会简单接受支架支持，而是评估这些聊天机器人如何在多样的学习环境和学生驱动的互动模式中进行导航。

View on arXiv Download PDF AI Translation

cs.AI / 54 / 2606.15782

Mitigating Visual Hallucinations in Multimodal Systems through Retrieval-Augmented Reliability-Aware Inference

通过检索增强的可靠性感知推理减轻多模态系统中的视觉幻觉

Hariharan, Pratheswaran, Xu, Haiping, Yan, Donghui

Abstract

Multimodal large language models (MLLMs) have demonstrated strong capabilities in vision-language understanding and natural-language response generation. However, these systems can still produce overconfident predictions and hallucination-like outputs, particularly when the visual evidence is weak, ambiguous, or semantically inconsistent. Most existing approaches focus on improving multimodal representation alignment or retrieval-augmented generation, while providing limited mechanisms to quantify instance-level prediction reliability or identify incorrect visual outputs. This work proposes a retrieval-augmented reliability-aware inference framework for trustworthy multimodal visual understanding. The proposed framework constructs an external visual evidence database using pretrained visual embeddings and nearest-neighbor retrieval over normalized feature representations. Retrieved evidence is used to estimate prediction trustworthiness through multiple reliability indicators, including similarity strength, class-support agreement, evidence margin, entropy-based uncertainty, and an aggregate reliability score. Based on these signals, a decision gate determines whether the system should accept the prediction, answer with caution, or abstain/fallback when evidence is insufficient. A multimodal response-generation layer then produces a final user-facing response conditioned on the reliability decision. Experiments on ImageNet-100 demonstrate that the proposed reliability-aware framework improves accepted prediction accuracy from 85.84\% to 88.88\% at 89.04\% coverage. The hallucination-like accepted wrong-answer rate is reduced from 14.16\% to 11.12\%. These results show that integrating retrieval evidence, reliability estimation, and selective decision gating can improve calibration and reduce overconfident visual errors without retraining large multimodal models.

Chinese Translation

多模态大型语言模型（MLLMs）在视觉-语言理解和自然语言响应生成方面表现出强大的能力。然而，这些系统仍可能产生过于自信的预测和类似幻觉的输出，特别是在视觉证据薄弱、模糊或语义不一致时。现有的大多数方法集中在改善多模态表示对齐或检索增强生成上，而在量化实例级预测可靠性或识别错误视觉输出方面提供的机制有限。本研究提出了一种检索增强的可靠性感知推理框架，以实现可信赖的多模态视觉理解。该框架利用预训练的视觉嵌入和对归一化特征表示的最近邻检索构建外部视觉证据数据库。检索到的证据通过多个可靠性指标来估计预测的可信度，包括相似性强度、类别支持一致性、证据边际、基于熵的不确定性和综合可靠性评分。基于这些信号，决策门确定系统是否应接受预测、谨慎回答或在证据不足时放弃/回退。然后，多模态响应生成层根据可靠性决策生成最终的用户响应。在ImageNet-100上的实验表明，所提出的可靠性感知框架将接受的预测准确率从85.84%提高到88.88%，覆盖率为89.04%。类似幻觉的错误答案接受率从14.16%降低到11.12%。这些结果表明，整合检索证据、可靠性估计和选择性决策门可以改善校准并减少过于自信的视觉错误，而无需重新训练大型多模态模型。

View on arXiv Download PDF AI Translation

cs.AI / 55 / 2606.15797

Unassigned Agents in Compilation-based Multi-agent Path Finding

基于编译的多智能体路径规划中的未分配智能体

Surynek, Pavel

Abstract

Compilation-based techniques represent an important stream of solvers for multi-agent path finding (MAPF) due to their modularity and adaptability for non-standard variants of the problem. While in the standard MAPF the task is to navigate all agents from their initial positions to given individual goal positions without any collision, variants where a different requirement for agents is used are also relevant. Such a variant is MAPF with unassigned agents (UA-MAPF) where some agents have the same setting as in the standard MAPF with initial positions and goals while the remaining agents have the initial position but have no goal - unassigned agents. Despite unassigned agent do not need to reach any goal position they have to be moved out of the way of the standard agents if needed which represent a specific challenge. We show in this paper that UA-MAPF can be expressed in recent compilation-based techniques for MAPF based on formulating the problem as Boolean satisfiability, namely we adapt SMT-CBS and NRF-SAT, the recent solvers based on counterexample guided abstraction refinement and non-refined abstractions.

Chinese Translation

基于编译的技术代表了多智能体路径规划（MAPF）求解器的重要方向，因为它们具有模块化和适应非标准变体问题的能力。在标准的MAPF中，任务是将所有智能体从其初始位置导航到给定的个体目标位置，且不发生碰撞，但使用不同要求的智能体变体同样具有相关性。这样的变体是未分配智能体的MAPF（UA-MAPF），其中一些智能体与标准MAPF具有相同的设置，包括初始位置和目标，而其余智能体则只有初始位置但没有目标——即未分配智能体。尽管未分配智能体不需要到达任何目标位置，但在必要时，它们必须被移动以让出标准智能体的通道，这构成了一个特定的挑战。我们在本文中展示了UA-MAPF可以通过将问题表述为布尔可满足性来表达在最近的基于编译的MAPF技术中，即我们适应了基于反例引导的抽象细化和非细化抽象的最新求解器SMT-CBS和NRF-SAT。

View on arXiv Download PDF AI Translation

cs.AI / 56 / 2606.15822

TrustedARI: Towards Trust-Native Agentic Routing Infrastructure for Agentic AI

TrustedARI：面向信任原生的代理人工智能路由基础设施

Li, Qi, Zou, Zhenhua, Li, Shuo, Xu, Mingwei, Liu, Zhuotao

Abstract

AI agents increasingly access external models, tools, and services through Agentic Routing Infrastructure (ARI) to manage the overhead of heterogeneous interfaces and fragmented subscriptions. Yet, the architecture of ARI introduces fundamental trust risks: it obtains plaintext access to agent queries and service responses, while leaving agents unable to verify that their queries are routed to intended service providers or that requests and responses remain untampered. To address this problem, we present TrustedARI, the first trust-native agentic routing infrastructure for agentic AI. Architecturally, TrustedARI is built upon three core innovations: (i) an ARI-adapted three-party TLS handshake that enables the agent and ARI to jointly authenticate the service provider through role-specific distribution of TLS key materials; (ii) a privacy-preserving query-construction protocol that allows the agent and ARI to collaboratively construct well-formed queries without exposing their respective private inputs; and (iii) a verifiable billing protocol that supports fair usage-based settlement while preserving the integrity and confidentiality of service responses. We implemented and extensively evaluated a prototype of TrustedARI to validate its performance. Experiments confirm that TrustedARI is highly efficient: our ARI-adapted handshake protocol reduces communication overhead by 39.34% compared to the existing three-party TLS handshake. Furthermore, the privacy-preserving query-construction protocol imposes negligible overhead-averaging 0.19 seconds in computation time and 0.58 MB in communication costs-while the verifiable billing protocol speeds up proof generation by 28.20x. Crucially, TrustedARI is readily deployable without any modification to the service providers.

Chinese Translation

人工智能代理越来越多地通过代理路由基础设施（Agentic Routing Infrastructure, ARI）访问外部模型、工具和服务，以管理异构接口和碎片化订阅的开销。然而，ARI的架构引入了基本的信任风险：它以明文方式获取代理查询和服务响应的访问权限，同时使得代理无法验证其查询是否被路由到预期的服务提供者，或请求和响应是否保持未被篡改。为了解决这个问题，我们提出了TrustedARI，这是首个面向信任原生的代理人工智能路由基础设施。从架构上看，TrustedARI建立在三个核心创新之上：（i）一种适应ARI的三方TLS握手，能够通过角色特定的TLS密钥材料分发，使代理和ARI共同认证服务提供者；（ii）一种隐私保护的查询构建协议，允许代理和ARI在不暴露各自私有输入的情况下协作构建格式良好的查询；（iii）一种可验证的计费协议，支持基于公平使用的结算，同时保持服务响应的完整性和机密性。我们实现并广泛评估了TrustedARI的原型，以验证其性能。实验确认TrustedARI的高效性：我们的ARI适应握手协议相比现有的三方TLS握手减少了39.34%的通信开销。此外，隐私保护的查询构建协议几乎没有开销，计算时间平均为0.19秒，通信成本为0.58 MB，而可验证的计费协议使得证明生成速度提高了28.20倍。至关重要的是，TrustedARI可以在不对服务提供者进行任何修改的情况下轻松部署。

View on arXiv Download PDF AI Translation

cs.AI / 57 / 2606.15831

An Integrated System for Real-Time Student Assessment and Career Guidance Using Neural Networks in Computing Disciplines

基于神经网络的计算学科实时学生评估与职业指导综合系统

Faruque, Sakir Hossain, Hossain, Md. Jubair, Khushbu, Sharun Akter

Abstract

Many undergraduate students in Computer Science (CS) and Software Engineering (SWE) struggle to identify suitable career paths, particularly when their academic performance, abilities, and interests do not fully align. To address this issue, this study proposes an AI-driven Student Assessment and Career Prediction System that integrates a Career Guidance Expert (CGE) system with a Web-Based Student Assessment (WBSA) platform. Within the integrated framework, CGE enhances personalized career recommendations using AI while also assisting students after graduation in identifying suitable jobs, research domains, and higher study opportunities aligned with their skills and interests. The WBSA platform further strengthens interaction between students and faculty through assessments, personalized tasks, mentorship activities, and a secure real-time chat application. The CGE system employs a Multilayer Perceptron (MLP) model trained on real-world academic and extracurricular data collected using the snowball sampling method from the students of universities, achieving a validation accuracy of 94.71% in predicting personalized career paths. A pre-survey was conducted across universities to evaluate the proposed model before deployment. The WBSA system was developed as a modern web application using technologies such as Node.js, Next.js, and PostgreSQL to ensure scalability, responsiveness, and secure data management. The overall system is supported by a secure cloud-based infrastructure, the platform provides reliable performance while assisting graduates to select suitable career path in IT sector. In addition, a post-survey involving both students and faculty was conducted to gather feedback and further improve the overall effectiveness and usability of the system.

Chinese Translation

许多计算机科学（CS）和软件工程（SWE）专业的本科生在确定合适的职业路径时面临困难，尤其是在他们的学业表现、能力和兴趣未能完全一致的情况下。为了解决这一问题，本研究提出了一种基于人工智能的学生评估与职业预测系统，该系统将职业指导专家（CGE）系统与基于网络的学生评估（WBSA）平台相结合。在这一综合框架内，CGE利用人工智能增强个性化职业推荐，同时帮助毕业生识别与其技能和兴趣相匹配的合适工作、研究领域和深造机会。WBSA平台通过评估、个性化任务、指导活动和安全的实时聊天应用程序进一步加强了学生与教师之间的互动。CGE系统采用多层感知器（MLP）模型，该模型基于从大学生中使用滚雪球抽样法收集的真实学术和课外数据进行训练，在预测个性化职业路径方面实现了94.71%的验证准确率。在部署之前，针对各大学进行了预调查，以评估所提出模型的有效性。WBSA系统作为现代网络应用程序开发，采用了Node.js、Next.js和PostgreSQL等技术，以确保可扩展性、响应性和安全的数据管理。整体系统由安全的云基础设施支持，平台在帮助毕业生选择IT行业合适职业路径的同时提供可靠的性能。此外，还进行了包括学生和教师在内的后续调查，以收集反馈并进一步提高系统的整体有效性和可用性。

View on arXiv Download PDF AI Translation

cs.AI / 58 / 2606.15834

AIChilles: Automatically Uncovering Hidden Weaknesses in AI-Evolved Systems

AIChilles：自动揭示AI进化系统中的隐藏弱点

Zhou, Yajie, Li, Ao, Silla, Ashwin, Liu, Zaoxing, Sekar, Vyas

Abstract

The computer systems community has recently seen growing interest in AI-driven system evolution, where AI agents iteratively rewrite systems. Frameworks such as AdaEvolve and Engram report 12-60% score improvements over human-designed algorithms. While these results are promising, there are practical concerns if these AI-evolved programs can perform worse on unseen workloads and exhibit scalability regressions. Given the speed and scale of AI-generated code, we need automated mechanisms to uncover such identify hidden weaknesses in AI-evolved systems programs. To this end, we develop AIChilles that takes as input a baseline program $P$ and an AI-evolved program $P'$, AIChilles searches for valid workloads where $P'$ regresses relative to $P$ in correctness, runtime, memory usage, or output quality. To tackle the diversity in system applications, weakness types and potential bugs, AIChilles combines deterministic workload-parameter extraction, agent-based constraint inference, differential oracles, and code-frequency coverage to discover diverse failures. Across five system applications and 30 AI-evolved programs, AIChilles finds 49 distinct hidden weaknesses. We also show that explicitly including AIChilles in the AI-driven development lifecycle can mitigate several of these weaknesses.

Chinese Translation

计算机系统领域最近对AI驱动的系统进化表现出越来越大的兴趣，其中AI代理通过迭代方式重写系统。像AdaEvolve和Engram这样的框架报告了相较于人类设计算法12-60%的得分提升。尽管这些结果令人鼓舞，但仍然存在实际问题，即这些AI进化的程序在未见过的工作负载下表现可能更差，并且可能出现可扩展性回归。鉴于AI生成代码的速度和规模，我们需要自动化机制来揭示AI进化系统程序中的隐藏弱点。为此，我们开发了AIChilles，它以基线程序$P$和AI进化程序$P'$为输入，AIChilles搜索有效的工作负载，在这些工作负载中，$P'$在正确性、运行时间、内存使用或输出质量方面相对于$P$出现回归。为了应对系统应用、弱点类型和潜在错误的多样性，AIChilles结合了确定性工作负载参数提取、基于代理的约束推断、差分oracle和代码频率覆盖，以发现多样化的失败。在五个系统应用和30个AI进化程序中，AIChilles发现了49个不同的隐藏弱点。我们还展示了在AI驱动的开发生命周期中显式包含AIChilles可以减轻其中一些弱点的影响。

View on arXiv Download PDF AI Translation

cs.AI / 59 / 2606.15841

Heteroskedastic Signals in Budgeted LLM Verification: Structural Heterogeneity Limits Optimization Gains

预算化大语言模型验证中的异方差信号：结构异质性限制优化收益

Yang, Jinlong

Abstract

Large language model (LLM) systems increasingly use uncertainty signals to allocate limited computation across verification, test-time scaling, tool execution, and other selective-compute decisions. Such policies rely on a \emph{global signal comparability assumption}: equal scores should carry comparable decision value across inputs. Using budgeted verification as a controlled diagnostic setting, we identify a failure mode of this assumption: uncertainty quality is heteroskedastic across cost strata, with some regions exhibiting near-random discriminability despite concentrating many errors. Under an explicit local model, we characterize the resulting distortion of global allocation and show that its upper bound scales with cross-stratum signal-quality dispersion. We separate weak signals, optimization instability, and structural heterogeneity through a controlled intervention hierarchy: Threshold, MP-Adapt, MP-Strat, and a deliberately simple cost-stratified thresholding intervention (CST). Across MBPP and MATH using Qwen3-8B, LLaMA3-8B, and GPT-4o-mini, global online adaptation yields inconsistent gains over static thresholding; MP-Strat partially recovers performance, while CST improves hit rate by up to 17 percentage points in strongly heterogeneous settings without gradient updates. These results identify structural heterogeneity, rather than optimizer weakness alone, as the primary bottleneck in the observed settings. More broadly, misaligned feedback structure cannot always be repaired by stronger optimization.

Chinese Translation

大型语言模型（LLM）系统越来越多地使用不确定性信号来在验证、测试时间扩展、工具执行和其他选择性计算决策之间分配有限的计算资源。这些策略依赖于一个 extit{全局信号可比性假设}：相等的分数在不同输入之间应该具有可比的决策价值。通过将预算验证作为一个受控诊断环境，我们识别出这一假设的失败模式：在成本层次中，不确定性质量是异方差的，一些区域尽管集中许多错误，却表现出近乎随机的可区分性。在一个明确的局部模型下，我们描述了由此导致的全局分配扭曲，并展示其上界与跨层次信号质量的离散度成比例。我们通过一个受控干预层次结构分离弱信号、优化不稳定性和结构异质性：阈值（Threshold）、MP-适应（MP-Adapt）、MP-分层（MP-Strat），以及一个故意简单的成本分层阈值干预（CST）。在使用Qwen3-8B、LLaMA3-8B和GPT-4o-mini的MBPP和MATH任务中，全球在线适应相较于静态阈值产生了不一致的收益；MP-Strat部分恢复了性能，而CST在强异质性环境中在没有梯度更新的情况下将命中率提高了多达17个百分点。这些结果表明，结构异质性而非单纯的优化器弱点是观察到的设置中的主要瓶颈。更广泛地说，反馈结构的不对齐并不总能通过更强的优化来修复。

View on arXiv Download PDF AI Translation

cs.AI / 60 / 2606.15862

RetailBench: Benchmarking long horizon reasoning and coherent decision making of LLM agents in realistic retail environments

RetailBench：在现实零售环境中基准测试大型语言模型代理的长期推理和连贯决策能力

Zhang, Linghua, Wang, Jun, Wu, Jingtong, Zhang, Zhisong

Abstract

Large language model (LLM) agents have made rapid progress on short-horizon, well-scoped tasks, yet their ability to sustain coherent decisions in dynamic long-horizon environments remains uncertain. We introduce RetailBench, a data-grounded simulation benchmark for evaluating tool-using LLM agents in single-store supermarket operation. RetailBench models retail management as a partially observable decision process and is designed to support thousand-day-scale simulations. In this environment, agents must manage pricing, replenishment, supplier selection, shelf assortment, inventory aging, customer feedback, external events, and cash-flow constraints. We evaluate seven contemporary LLMs under representative agent frameworks over a 180-day evaluation horizon and compare them with a privileged oracle policy. Results show substantial variation across models: only a small subset survives the full evaluation horizon, and even the strongest LLM runs remain substantially behind the oracle policy in final net worth and sales outcomes. Behavioral analysis attributes these gaps to incomplete evidence acquisition, surface-level decision making, and the lack of a consistent long-horizon policy. RetailBench provides a controlled testbed for studying reliable autonomy in economically grounded long-horizon decision-making.

Chinese Translation

大型语言模型（LLM）代理在短期、明确界定的任务上取得了快速进展，但它们在动态长期环境中维持连贯决策的能力仍然不确定。我们引入了RetailBench，这是一个基于数据的模拟基准，用于评估在单店超市运营中使用工具的LLM代理。RetailBench将零售管理建模为一个部分可观察的决策过程，并设计为支持千天规模的模拟。在这个环境中，代理必须管理定价、补货、供应商选择、货架组合、库存老化、客户反馈、外部事件和现金流约束。我们在一个180天的评估周期内，基于代表性代理框架评估了七种当代LLM，并将其与一个特权的神谕策略进行了比较。结果显示模型之间存在显著差异：只有一小部分模型能够在整个评估周期内存活下来，即使是最强的LLM在最终净资产和销售结果上也仍然显著落后于神谕策略。行为分析将这些差距归因于证据获取不完整、表层决策以及缺乏一致的长期政策。RetailBench提供了一个受控的测试平台，用于研究在经济基础上可靠的长期决策中的自主性。

View on arXiv Download PDF AI Translation

cs.AI / 61 / 2606.15866

STRIDE: Strategic Trajectory Reasoning via Discriminative Estimation for Verifiable Reinforcement Learning

STRIDE：通过区分估计进行可验证强化学习的战略轨迹推理

Zhao, Qinjian, Dou, Zhihao, Zhang, Dinggen, Li, Xiangyu, Song, Chaoda, Wan, Zhongwei, Li, Xinpeng, Zhang, Yanyan, Chen, Kaijie, Pan, Qingtao, Feng, Chengcheng, Gao, Zhiqiang, Xia, Xiaoyu

Abstract

Reinforcement Learning with Verifiable Rewards (RLVR) has become an effective post-training paradigm for improving the reasoning abilities of large language models. However, existing RLVR methods typically rely on final-answer correctness to assign trajectory-level rewards, providing sparse supervision and treating all tokens uniformly regardless of their actual contribution to reasoning. Although recent studies introduce intermediate signals such as process rewards, high-entropy tokens, and semantic uncertainty, these signals are often not inherently verifiable and may fail to distinguish beneficial strategic patterns from harmful ones. To address this limitation, we propose STRIDE (Strategic Trajectory Reasoning with Discriminative Estimation), a fine-grained RLVR framework that derives strategic reasoning supervision from verifiable outcomes. STRIDE contrasts successful and failed trajectories within each response group to estimate the outcome-discriminative preference of each $n$-gram strategic pattern, and further combines this signal with reasoning saliency entropy to identify decision-relevant strategic patterns. These patterns are assigned differentiated advantage values during RL optimization, enabling more precise credit assignment while preserving the verifiability of RLVR. Extensive experiments demonstrate that STRIDE consistently improves reasoning performance across diverse models, tasks, and extended settings, including VLMs and agent-based systems.

Chinese Translation

具有可验证奖励的强化学习（RLVR）已成为提升大型语言模型推理能力的有效后训练范式。然而，现有的RLVR方法通常依赖最终答案的正确性来分配轨迹级奖励，这提供了稀疏的监督，并且对所有标记的处理是统一的，而不考虑它们对推理的实际贡献。尽管近期研究引入了过程奖励、高熵标记和语义不确定性等中间信号，但这些信号往往不是固有可验证的，可能无法区分有益的战略模式和有害的模式。为了解决这一局限性，我们提出了STRIDE（通过区分估计进行战略轨迹推理），这是一个细粒度的RLVR框架，从可验证的结果中推导出战略推理监督。STRIDE在每个响应组内对成功和失败的轨迹进行对比，以估计每个$n$-gram战略模式的结果区分偏好，并进一步将该信号与推理显著性熵结合，以识别与决策相关的战略模式。在RL优化过程中，这些模式被赋予不同的优势值，从而实现更精确的信用分配，同时保持RLVR的可验证性。大量实验表明，STRIDE在多种模型、任务和扩展设置（包括VLM和基于代理的系统）中始终提高了推理性能。

View on arXiv Download PDF AI Translation

cs.AI / 62 / 2606.15874

LLM-as-Code Agentic Programming for Agent Harness

LLM作为代码的代理编程用于代理控制

Qi, Junjia, Fu, Zichuan, Gao, Jingtong, Zhang, Wenlin, Yan, Hanyu, Wu, Xian, Zhao, Xiangyu

Abstract

Every major LLM agent framework gives the LLM the role of orchestrator; the model decides what to do next, when to call tools, and when to stop. We argue that token explosion, control-flow hallucination, and unreliable completion are not implementation bugs but architectural consequences of assigning the deterministic work of looping, branching, and sequencing to a probabilistic system. A better prompt or a stronger model cannot guarantee the reliability of the LLM agent. We therefore propose Agentic Programming, in which the program governs all control flow, and the LLM is itself part of it, an adaptive component we call LLM-as-Code and invoke only where a task calls for reasoning or generation. Within each call the model keeps full flexibility, but it cannot alter the program's execution path. With control in the program, the LLM's context is built from the execution history's call tree and forms a directed acyclic graph (DAG). Each call's context length is then determined by its call depth rather than by accumulation over steps. A case study of computer-use agents shows that the design is practical, not just a theoretical stance, substantially improving the stability of long visual operation sequences.

Chinese Translation

每个主要的LLM代理框架都赋予LLM协调者的角色；模型决定下一步做什么、何时调用工具以及何时停止。我们认为，令牌爆炸、控制流幻觉和不可靠的完成并不是实现错误，而是将循环、分支和序列化的确定性工作分配给概率系统的架构后果。更好的提示或更强的模型无法保证LLM代理的可靠性。因此，我们提出了代理编程（Agentic Programming），在这种编程中，程序控制所有控制流，而LLM本身是其中的一部分，是一个适应性组件，我们称之为LLM-as-Code，仅在任务需要推理或生成时调用。在每次调用中，模型保持完全的灵活性，但无法改变程序的执行路径。通过在程序中控制，LLM的上下文是由执行历史的调用树构建而成，并形成一个有向无环图（DAG）。每次调用的上下文长度由其调用深度决定，而不是通过步骤的累积。对计算机使用代理的案例研究表明，该设计是实用的，而不仅仅是理论立场，显著提高了长视觉操作序列的稳定性。

View on arXiv Download PDF AI Translation

cs.AI / 63 / 2606.15890

UrbanWell: Benchmarking Multimodal Large Language Models for Spatio-Temporal Urban Wellbeing Analytics

UrbanWell：多模态大型语言模型在时空城市幸福分析中的基准评估

Xi, Yanxin, Su, Xiang, Feng, Jie, Liu, Yu, Tarkoma, Sasu, Hui, Pan

Abstract

Understanding urban wellbeing from multimodal data requires integrating heterogeneous spatial and temporal signals, posing significant challenges for current multimodal large language models (MLLMs). We introduce UrbanWell, a large-scale benchmark designed to systematically evaluate the spatio-temporal reasoning capabilities of MLLMs for urban wellbeing analytics through joint modeling of satellite and street view imagery. UrbanWell spans 38 cities across multiple years and includes diverse indicators covering (1) environmental conditions (CO$_2$, NO$_2$, PM${2.5}$, and Normalized Difference Vegetation Index), (2) spatial accessibility (minimum distance to supermarkets and restaurants), (3) urban form (road length, road density, and land use), (4) urban vitality (population, economic activity diversity, and land use diversity), and (5) subjective perception attributes (e.g., safety, beauty, liveliness, wealth, and quietness). All indicators are aligned at grid level to enable standardized evaluation. Beyond static prediction, UrbanWell defines temporal reasoning tasks, including future value forecasting from historical observations and temporal trend classification. We benchmark 15 state-of-the-art representative MLLMs in a zero-shot setting, providing a comprehensive comparative evaluation across spatial and temporal dimensions. Experimental results indicate that while MLLMs capture salient spatial and perceptual cues, their performance varies substantially across heterogeneous urban indicators spanning environment and subjective perception. UrbanWell serves as a unified benchmark for evaluating multimodal spatial and temporal reasoning in urban wellbeing analytics, offering a standardized testbed for systematic assessment and future research on multimodal urban intelligence. Our codes and datasets are accessible via https://github.com/axin1301/UrbanWell-Benchmark.

Chinese Translation

从多模态数据理解城市幸福感需要整合异构的时空信号，这对当前的多模态大型语言模型（MLLMs）提出了重大挑战。我们提出了UrbanWell，这是一个大规模基准，旨在通过卫星图像和街景图像的联合建模系统性评估MLLMs在城市幸福分析中的时空推理能力。UrbanWell覆盖38个城市，跨越多个年份，包含多样化的指标，涵盖（1）环境条件（CO$_2$、NO$_2$、PM${2.5}$和归一化植被指数），（2）空间可达性（到超市和餐馆的最短距离），（3）城市形态（道路长度、道路密度和土地利用），（4）城市活力（人口、经济活动多样性和土地利用多样性），以及（5）主观感知属性（例如安全、美丽、活力、财富和安静）。所有指标在网格级别对齐，以便进行标准化评估。除了静态预测，UrbanWell还定义了时序推理任务，包括从历史观察中预测未来值和时序趋势分类。我们在零样本设置下对15个最先进的代表性MLLMs进行了基准测试，提供了跨空间和时间维度的全面比较评估。实验结果表明，尽管MLLMs能够捕捉显著的空间和感知线索，但它们在涵盖环境和主观感知的异构城市指标上的表现差异显著。UrbanWell作为一个统一的基准，评估城市幸福分析中的多模态时空推理，为系统评估和未来多模态城市智能研究提供了标准化的测试平台。我们的代码和数据集可以通过https://github.com/axin1301/UrbanWell-Benchmark获取。

View on arXiv Download PDF AI Translation

cs.AI / 64 / 2606.15994

Agentic Framework for Deep Learning workload migration via In-Context Learning

基于代理框架的深度学习工作负载迁移方法：通过上下文学习实现

Liang, Qiyue, Ingram, Steven, Vanica, George, Gavrilescu, Andi, Harrat, Newfel, Sipra, Hassan, Sankaran, Sethuraman

Abstract

Translating deep learning models from PyTorch's flexible, object-oriented design to JAX's functional, stateless setup is usually a manual and error-prone task. Automated migration is challenging because Large Language Models (LLMs) struggle with strict and dynamic API alignment and are prone to mistakes for exacting operations. We propose a fully autonomous system that combines In-Context Learning (ICL) with oracle-driven self-debugging. First, we curated an ICL context that serves as a strict reference for idiomatic JAX styling and test case generation. Second, instead of depending on the LLM to deduce mathematical outputs, we run the source PyTorch modules to get their actual dynamic tensor states. This creates an unchangeable execution oracle. We then use an autonomous agentic loop to synthesize tests based on the oracle data. The test cases are executed repeatedly, and the traceback is sent back to the LLM for self-correction. Ablations show that combining ICL references with oracle grounding and self-debugging greatly outperforms pure instructional and basic agentic baselines. This improvement does not add an excessive computational overhead. Our lightweight pipeline achieves 91% numerical equivalence (compared to baseline: 9%, instruction + self-debugging: 27%) on neural modules, providing a highly reliable, scalable blueprint for cross-framework migration. This has been validated across several state-of-the-art models including SAM (segment anything), T5, Code Whisper amongst others showing high numerical equivalency. Code: https://github.com/AI-Hypercomputer/accelerator-agents/tree/main/MaxCode

Chinese Translation

将深度学习模型从PyTorch灵活的面向对象设计迁移到JAX的函数式无状态设置通常是一个手动且容易出错的任务。自动化迁移面临挑战，因为大型语言模型（LLMs）在严格和动态的API对齐方面表现不佳，并且在精确操作时容易出错。我们提出了一种完全自主的系统，将上下文学习（In-Context Learning, ICL）与基于oracle的自我调试相结合。首先，我们策划了一个ICL上下文，作为JAX风格和测试用例生成的严格参考。其次，我们不依赖LLM推导数学输出，而是运行源PyTorch模块以获取其实际动态张量状态。这创建了一个不可更改的执行oracle。然后，我们使用自主代理循环基于oracle数据合成测试。测试用例被反复执行，回溯信息被发送回LLM进行自我修正。消融实验表明，将ICL参考与oracle基础和自我调试相结合，显著优于纯指令和基本代理基线。这种改进没有增加过多的计算开销。我们的轻量级管道在神经模块上实现了91%的数值等价性（与基线相比：9%，指令 + 自我调试：27%），为跨框架迁移提供了高度可靠、可扩展的蓝图。这一方法已在多个最先进的模型上得到验证，包括SAM（segment anything）、T5、Code Whisper等，显示出高数值等价性。代码链接：https://github.com/AI-Hypercomputer/accelerator-agents/tree/main/MaxCode

View on arXiv Download PDF AI Translation

cs.AI / 65 / 2606.16003

SciText2Eq: Assessing LLMs for Explainable Equation Generation for Scientific Creativity

SciText2Eq：评估大型语言模型在科学创造性中的可解释方程生成能力

Mo, Yifan, Fu, Xiao, Su, Yue, Meng, Qingyu, Hindriks, Koen, Liu, Qingzhi, Pei, Jiahuan

Abstract

This work investigates the ability of large language models (LLMs) to generate mathematical equations from scientific texts. Prior work faces challenges in unstructured grounding, multi-equation dependency, and humanaligned evaluation. To this end, we construct a dataset of AI research papers, pairing contextual passages with ground-truth equations and variable descriptions. We develop an explainable equation generation workflow and evaluate it across diverse open- and closed-source LLM backbones. We introduce an evaluation protocol combining automatic metrics, LLM-based rubrics, and human judgments to assess accuracy, explainability, and human-LLM alignment. Results indicate that LLMs perform moderately on lexical- and syntactic-based similarity, while struggling with semantic accuracy. Comparisons between LLM-based evaluations and human judgments reveal limited alignment, highlighting challenges in using LLMs to assess equation quality. These findings offer insights for improving equation generation models and developing more reliable evaluation methods for scientific text. We provide code and data for reproducibility.

Chinese Translation

本研究探讨了大型语言模型（LLMs）从科学文本中生成数学方程的能力。以往的研究面临着非结构化基础、多方程依赖和人类对齐评估的挑战。为此，我们构建了一个人工智能研究论文的数据集，将上下文段落与真实方程和变量描述配对。我们开发了一个可解释的方程生成工作流程，并在多种开源和闭源的LLM基础上进行了评估。我们引入了一种评估协议，结合自动化指标、基于LLM的评分标准和人类判断，以评估准确性、可解释性和人类与LLM的对齐程度。结果表明，LLMs在基于词汇和句法的相似性方面表现中等，但在语义准确性上存在困难。LLM评估与人类判断之间的比较显示出有限的对齐，突显了使用LLM评估方程质量的挑战。这些发现为改进方程生成模型和开发更可靠的科学文本评估方法提供了见解。我们提供了代码和数据以便于复现。

View on arXiv Download PDF AI Translation

cs.AI / 66 / 2606.16062

Auditing Reward Hackability in Code RL Training Environments

审计代码强化学习训练环境中的奖励可破解性

Rajan, Shreshth

Abstract

We measure the rate at which code RL environments accept incorrect solutions as correct. On a 49-task sample of SWE-bench Verified, 28.5% of tasks have test suites weak enough that a Docker-verified incorrect patch passes them. On 20 R2E-Gym tasks across 6 repositories, the same pipeline at single-shot exploit generation yields 25.0%. A random-effects meta-analysis over 134 frontier model submissions to SWE-bench Verified finds, within the same human-rated difficulty stratum, model Pass@1 is +14.14 percentage points higher on flagged-hackable tasks than on robust ones (95% CI [+11.80, +16.48]; one-sided p < 10^-6; I^2 = 0%; 123 of 134 models positive). We then describe a procedure for hardening the broken tasks. An inline LLM judge with a Docker gold-sanity gate runs each generated test against the gold solution before the judge is consulted. On the 11 broken tasks in the audit, the gate flags 65 of 105 decisive LLM-generated tests as failing on the gold patch itself, a 61.9% per-augmentation defect rate the LLM judge alone misses. With diversity-biased retry, the loop converges 9 of 11 tasks to a gated upgrade.

Chinese Translation

我们测量了代码强化学习环境接受错误解决方案作为正确答案的频率。在49个任务的SWE-bench Verified样本中，28.5%的任务的测试套件过于薄弱，以至于一个经过Docker验证的错误补丁可以通过它们。在6个代码库中的20个R2E-Gym任务上，相同的单次攻击生成管道的成功率为25.0%。对134个前沿模型提交至SWE-bench Verified的随机效应元分析发现，在相同的人类评分难度层次中，标记为可破解任务的模型Pass@1比稳健任务高出14.14个百分点（95%置信区间[+11.80, +16.48]；单侧p < 10^-6；I^2 = 0%；134个模型中有123个为正）。然后，我们描述了一种加固破损任务的程序。一个内联的LLM评判者与Docker金标准门一起运行每个生成的测试，以确保在咨询评判者之前，测试与金标准解决方案进行对比。在审计的11个破损任务中，该门将105个决定性LLM生成测试中的65个标记为在金补丁上失败，LLM评判者单独遗漏的缺陷率为61.9%。通过多样性偏向的重试，该循环将11个任务中的9个收敛到一个经过门控的升级。

View on arXiv Download PDF AI Translation

cs.AI / 67 / 2606.16070

Mind-Studio: Executable World Models with Lookahead Evaluation for Partially Observable Games

Mind-Studio：具有前瞻性评估的可执行世界模型用于部分可观察游戏

Dong, Yifei, Zheng, Mingen, Wu, Linquan, Pan, Jeff Z., Bai, Jiaxin

Abstract

World-model synthesis aims to turn interaction experience into an internal model of environment dynamics. Existing symbolic approaches often fit observed transitions or mixtures of local rules, but they do not produce a complete executable program that can run independently of the real environment. We present Mind-Studio, a framework that synthesizes executable pygame-style world models from state-action-next-state trajectories using large language models. Mind-Studio combines entropy-selected traces with a lightweight game skill file containing object, action, and static scene information extracted from screenshots. We evaluate synthesis quality with a K-step lookahead fidelity protocol that compares generated world-model rollouts against Real-ALE rollouts from the same state. On Montezuma's Revenge, Mind-Studio improves chosen-action next-state prediction from 0.3% for PoE-World to 48.7% while verifying 5 of 8 subgoals; across Alien, Assault, and Skiing, it achieves stronger branch-level fidelity than prior learned lookahead sources.

Chinese Translation

世界模型合成旨在将交互经验转化为环境动态的内部模型。现有的符号方法通常适应观察到的转变或局部规则的混合，但它们并未生成可以独立于真实环境运行的完整可执行程序。我们提出了Mind-Studio，一个利用大型语言模型从状态-动作-下一个状态轨迹合成可执行pygame风格世界模型的框架。Mind-Studio将熵选择的轨迹与包含从屏幕截图中提取的对象、动作和静态场景信息的轻量级游戏技能文件相结合。我们使用K步前瞻保真度协议评估合成质量，该协议将生成的世界模型展开与来自同一状态的Real-ALE展开进行比较。在《蒙特祖玛的复仇》中，Mind-Studio将PoE-World的选定动作下一个状态预测的准确率从0.3%提高到48.7%，同时验证了8个子目标中的5个；在《外星人》、《攻击》和《滑雪》游戏中，它实现了比之前学习的前瞻性来源更强的分支级别保真度。

View on arXiv Download PDF AI Translation

cs.AI / 68 / 2606.16084

Rhythm of the Deep: A Computational-Linguistic Test of Duality of Patterning in Sperm Whale Codas

深海的节奏：对抹香鲸口哨声模式双重性的计算语言学测试

Sinha, Mudit, Chavan, Sanika

Abstract

Human language has often been described as combining structure at two levels: lower-level units combine into larger units, which then combine into larger sequences. We test for this design feature, duality of patterning, in sperm whale codas using 1,483 codas from the Dominica Sperm Whale Project. Because acoustic similarity can imitate symbolic structure, we treat the problem as computational-linguistic structure discovery from continuous audio rather than as a direct claim about language or meaning. We use a consensus of frozen audio encoders, held-out structural tests, per-statistic nulls, and acoustic-null recoverability gates. The evidence supports a narrow two-tier architecture. At the lower tier, clicks compose into codas not by a stable ordered rule, but by which clicks are present together with their inter-click rhythm. At the upper tier, coda tokens show bout-level sequential dependence, with an NSB second-order transfer-entropy lift of 0.132 bits (p = 0.002). Under tempo scaling, encoder-derived click identity is strongly rate-bound, while coda identity remains substantially more stable, yielding a measurable abstraction gradient across the click-to-coda step. Rhythm-only baselines recover substantial lower-tier structure but fail to reproduce the upper-tier sequential-dependence signal. We do not claim language, semantics, perception, or human-like phonemes. Instead, we report representation-level evidence for a duality-of-patterning-like architecture whose lower tier is rhythmic rather than segmental, and provide a portable null-controlled framework for testing combinatorial structure in induced acoustic token systems.

Chinese Translation

人类语言常被描述为在两个层次上结合结构：低层单位组合成更大的单位，然后再组合成更大的序列。我们使用来自多米尼克抹香鲸项目的1483个口哨声测试这一设计特征，即模式双重性。由于声学相似性可以模仿符号结构，我们将这一问题视为从连续音频中发现计算语言学结构，而不是直接关于语言或意义的主张。我们使用冷冻音频编码器的一致性、保留的结构测试、每个统计的零假设以及声学零假设可恢复性门。证据支持一个狭窄的双层架构。在低层，点击声并不是通过稳定的有序规则组合成口哨声，而是通过哪些点击声同时存在及其间点击节奏。在高层，口哨声标记显示出回合级的序列依赖性，具有0.132比特的NSB二阶转移熵提升（p = 0.002）。在节奏缩放下，编码器派生的点击声身份受到强烈的速率限制，而口哨声身份则保持相对稳定，导致在点击到口哨声的步骤中产生可测量的抽象梯度。仅基于节奏的基线恢复了相当大的低层结构，但未能重现高层的序列依赖信号。我们并不声称存在语言、语义、感知或人类般的音素。相反，我们报告了一个类似于模式双重性的架构的表示层面证据，其低层是节奏性而非分段性，并提供了一个可移植的零控制框架，用于测试诱导声学标记系统中的组合结构。

View on arXiv Download PDF AI Translation

cs.AI / 69 / 2606.16113

RecourseBench: A Modular Framework for Reproducible Algorithmic Recourse Evaluation

RecourseBench：一个模块化的可重复算法补救评估框架

Khotanlou, Zahra, Ahmed, Hashir, Tan, Chenghao, Abdelaal, Ahmed, Karimi, Amir-Hossein

Abstract

Algorithmic recourse methods provide counterfactual explanations that inform individuals of the actions required to overturn an unfavorable model decision. Despite rapid methodological progress, principled comparison remains elusive; existing frameworks are often difficult to extend and lack both interoperability and systematic verification that integrated methods faithfully reproduce their originally reported results. We introduce \emph{RecourseBench}, a unified evaluation framework built around three commitments namely, modularity, reproducibility, and interactivity. The framework decomposes the pipeline into five fully decoupled layers -- Data, Preprocessing, Model, Recourse Method, and Evaluation -- governed by abstract interfaces and a dynamic registry. To address the reproducibility gap in prior benchmarks, we introduce a four-tier classification system in which every integrated method is validated by an automated test suite against its originally reported results. We further provide an interactive web interface for flexible, configuration-driven comparison across methods, datasets, and model architectures. Our framework currently integrates 28 state-of-the-art recourse methods and, to our knowledge, constitutes the first recourse benchmark to explicitly enforce method-level reproducibility through automated, quantitative testing.

Chinese Translation

算法补救方法提供反事实解释，告知个人为推翻不利模型决策所需采取的行动。尽管方法论进展迅速，但原则性比较仍然难以实现；现有框架往往难以扩展，缺乏互操作性和系统验证，无法确保集成方法忠实再现其最初报告的结果。我们提出了 extit{RecourseBench}，一个围绕模块化、可重复性和交互性三大承诺构建的统一评估框架。该框架将流程分解为五个完全解耦的层次——数据、预处理、模型、补救方法和评估——由抽象接口和动态注册表管理。为了弥补先前基准中的可重复性差距，我们引入了一个四级分类系统，其中每个集成方法都通过自动化测试套件验证其与最初报告结果的一致性。我们进一步提供了一个交互式网络界面，以便在方法、数据集和模型架构之间进行灵活的、基于配置的比较。我们的框架目前集成了28种最先进的补救方法，并且据我们所知，构成了第一个明确通过自动化、定量测试强制执行方法级可重复性的补救基准。

View on arXiv Download PDF AI Translation

cs.AI / 70 / 2606.16118

Know Your Limits : On the Faithfulness of LLMs as Solvers and Autoformalizers in Legal Reasoning

了解你的局限性：大型语言模型在法律推理中的求解者和自动形式化的可信度

Wang, Olivia Peiyu, Wong-Toropainen, Sanna, Amrollahi, Daneshvar, Bai, Ryan, Bansal, Tashvi, Garg, Arush, Gilpin, Leilani H.

Abstract

Large Language Models (LLMs) achieve strong performance on reasoning tasks, but whether this reflects faithful logical inference or heuristic approximation remains unclear. We study this question in legal entailment by comparing three paradigms, including pure LLM classification, LLM-based Formal Reasoning, and solver-based Formal Reasoning using the Z3 SMT solver, on a re-annotated subset of ContractNLI across five LLMs. Our re-annotation reveals a systematic and measurable gap between pragmatic legal interpretation and strict formal entailment, where a substantial proportion of legally sound inferences are not formally grounded without additional unstated assumptions. While introducing formal structure improves accuracy, with LLM-based Formal Reasoning achieving the highest benchmark performance, we show that this gain does not imply faithful reasoning. We identify three recurring failure modes: scope laundering, where LLMs report solver-inconsistent classifications without executing the underlying formal reasoning, producing conclusions that appear logically grounded but are not; implicit constraint blindness, where LLMs overlook logical constraints present in formal representations; and program synthesis failures, where LLMs generate incorrect Z3 code despite structured prompting. Critically, scope laundering persists across all models, raising serious concerns about the faithfulness of LLM-based formal reasoning as a proxy for symbolic execution. These results reveal a fundamental gap between benchmark accuracy and logical faithfulness.

Chinese Translation

大型语言模型（LLMs）在推理任务中表现出色，但这是否反映了忠实的逻辑推理还是启发式的近似仍不清楚。我们通过比较三种范式来研究这一问题，包括纯 LLM 分类、基于 LLM 的形式推理，以及使用 Z3 SMT 求解器的基于求解器的形式推理，针对五个 LLM 在重新标注的 ContractNLI 子集上进行分析。我们的重新标注揭示了实用法律解释与严格形式蕴涵之间存在系统且可测量的差距，其中相当一部分合法推理在没有额外未声明假设的情况下并未得到形式上的支持。尽管引入形式结构提高了准确性，基于 LLM 的形式推理达到了最高的基准性能，但我们表明，这种提升并不意味着忠实推理。我们识别出三种反复出现的失败模式：范围洗涤（scope laundering），即 LLM 在未执行基础形式推理的情况下报告与求解器不一致的分类，产生看似逻辑上合理但实际上并非如此的结论；隐性约束盲点（implicit constraint blindness），即 LLM 忽略了形式表示中存在的逻辑约束；以及程序合成失败（program synthesis failures），即 LLM 尽管有结构化提示，仍生成不正确的 Z3 代码。重要的是，范围洗涤在所有模型中普遍存在，严重影响了基于 LLM 的形式推理作为符号执行代理的可信度。这些结果揭示了基准准确性与逻辑忠实性之间的根本差距。

View on arXiv Download PDF AI Translation

cs.AI / 71 / 2606.16122

Thinking with Visual Grounding

视觉基础思维

Zhang, Junkai, Deng, Yihe, Chang, Kai-Wei, Wang, Wei

Abstract

Visual thinking should not only sound right; it should show its evidence. While recent vision-language models (VLMs) can produce natural-language reasoning traces, these traces often leave the supporting image regions implicit, making them hard to verify and difficult to supervise. We introduce visually grounded thinking, a reasoning process in which models interleave natural-language thoughts with explicit point or box groundings of the visual evidence used at each step. This lets the model express intermediate reasoning in language while grounding key objects in the image regions they refer to. To train this behavior, we construct a scalable synthesis pipeline that distills correct visual reasoning traces, extracts the visual objects required by the traces, grounds them with a SAM3-based agent, and derives aligned point and box supervision from the resulting masks. We further propose grounding-aware reinforcement learning, which combines answer correctness rewards with dense grounding rewards that score whether generated object references match the correct image evidence. Across two counting benchmarks and four spatial reasoning benchmarks, adding visually grounded thinking to Gemma3-4B-IT consistently improves performance over the original model and the non-grounded thinking baseline. On spatial reasoning, the visually grounded thinking 4B models match, and in some cases surpass, Gemma3-27B-IT from the same model family. Our analysis shows that point grounding is well suited to counting, while box grounding benefits most from explicit grounding rewards on spatial tasks. Overall, our results show that VLMs think better when their intermediate thoughts are tied to the image regions that make them true.

Chinese Translation

视觉思维不仅应该听起来正确；它还应该展示其证据。尽管最近的视觉-语言模型（VLMs）能够生成自然语言推理轨迹，但这些轨迹往往使支持的图像区域隐含，难以验证且难以监督。我们引入了视觉基础思维，这是一种推理过程，其中模型将自然语言思维与每一步使用的视觉证据的明确点或框基础交替进行。这使得模型能够用语言表达中间推理，同时将关键对象与其所指的图像区域进行基础。为了训练这种行为，我们构建了一个可扩展的合成管道，该管道提炼出正确的视觉推理轨迹，提取轨迹所需的视觉对象，利用基于SAM3的代理进行基础，并从生成的掩模中推导出对齐的点和框监督。我们进一步提出了基础感知强化学习，它结合了答案正确性的奖励与密集基础奖励，以评估生成的对象引用是否与正确的图像证据匹配。在两个计数基准和四个空间推理基准上，将视觉基础思维添加到Gemma3-4B-IT模型中，始终提高了性能，超越了原始模型和非基础思维基线。在空间推理任务中，视觉基础思维4B模型与同一模型系列中的Gemma3-27B-IT相匹配，并在某些情况下超过了它。我们的分析表明，点基础非常适合计数，而框基础在空间任务中最受益于明确的基础奖励。总体而言，我们的结果表明，当视觉语言模型的中间思维与使其成立的图像区域相联系时，它们的思维表现更佳。

View on arXiv Download PDF AI Translation

cs.AI / 72 / 2606.16140

VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models

VibeThinker-3B：探索小型语言模型中可验证推理的前沿

Xu, Sen, Liu, Shixi, Wang, Wei, Min, Jixin, Dai, Yingwei, Yin, Zhibin, Chen, Yirong, Zhou, Xin, Zhang, Junlin

Abstract

This technical report introduces VibeThinker-3B, a compact dense model with 3B parameters developed to investigate how far verifiable reasoning can be pushed within a strictly small-model regime. Building upon the Spectrum-to-Signal post-training paradigm, we systematically enhance the model through an optimized pipeline that includes curriculum-based supervised fine-tuning, multi-domain reinforcement learning, and offline self-distillation. Experimental evaluations demonstrate that VibeThinker-3B achieves frontier-level performance on highly demanding verifiable tasks. Specifically, it attains a score of 94.3 on AIME26 (improving to 97.1 with claim-level test-time scaling), an 80.2 Pass@1 on LiveCodeBench v6, and exhibits strong out-of-distribution generalization with a 96.1\% acceptance rate on recent unseen LeetCode contests. This effectively places it in the performance band of first-tier reasoning systems, matching or exceeding flagship models that are orders of magnitude larger, such as DeepSeek V3.2, GLM-5, and Gemini 3 Pro. Furthermore, a score of 93.4 on IFEval confirms that this extreme reasoning enhancement does not compromise strict instruction controllability. Extending our previous 1.5B work, these findings motivate the Parametric Compression-Coverage Hypothesis, which views verifiable reasoning as compressible into compact reasoning cores, while open-domain knowledge and general-purpose competence require broad parameter coverage over facts, concepts, and long-tail scenarios. This perspective suggests that compact models are not merely deployment-efficient substitutes, but a complementary path toward frontier-level performance in parameter-dense capability regimes.

Chinese Translation

本技术报告介绍了VibeThinker-3B，这是一种具有30亿参数的紧凑型密集模型，旨在研究在严格的小型模型范畴内可验证推理的极限。基于Spectrum-to-Signal后训练范式，我们通过优化的管道系统性地增强了该模型，该管道包括基于课程的监督微调、多领域强化学习和离线自蒸馏。实验评估表明，VibeThinker-3B在高度要求的可验证任务上达到了前沿水平的性能。具体而言，它在AIME26上获得了94.3的分数（在声明级测试时间扩展后提升至97.1），在LiveCodeBench v6上达到了80.2的Pass@1，并在最近未见的LeetCode竞赛中展现出强大的分布外泛化能力，接受率为96.1%。这有效地将其置于一流推理系统的性能区间，与规模大几个数量级的旗舰模型（如DeepSeek V3.2、GLM-5和Gemini 3 Pro）相匹配或超越。此外，在IFEval上获得的93.4分确认了这种极端推理增强并未妨碍严格的指令可控性。扩展我们之前的1.5B工作，这些发现激励了参数压缩-覆盖假设，该假设将可验证推理视为可以压缩为紧凑推理核心，而开放领域知识和通用能力则需要对事实、概念和长尾场景的广泛参数覆盖。这一观点表明，紧凑模型不仅仅是高效部署的替代品，而是朝着参数密集能力范畴内前沿性能的互补路径。

View on arXiv Download PDF AI Translation

cs.AI / 73 / 2606.16149

LiteOdyssey: A Lightweight Reasoning AI Agent for Interpretable Rare-Disease Diagnosis

LiteOdyssey：一种轻量级推理人工智能代理，用于可解释的罕见疾病诊断

Nguyen, Minh-Ha, Gray, Erica, Yang, Chih-Ting, Hamid, Rizwan, Li, Lingyao, Ma, Siyuan, Cassini, Thomas A., Shyr, Cathy

Abstract

Most medical AI systems improve by scaling additional machinery: more fine-tuning data, more agents, and/or larger retrieval databases. In rare-disease diagnosis, however, such scaling can produce systems that are difficult to deploy, audit, and maintain. We asked whether state-of-the-art diagnostic performance could instead be achieved by extending the reasoning chain of a single AI agent: guiding it with a diagnostic policy, developed through human-AI collaboration and augmenting with freely available biomedical tools. We introduce LiteOdyssey, a lightweight rare-disease diagnostic framework that guides reasoning language model through a clinical genetics workflow. This framework was developed through Policy Iteration with Human Feedback (PIHF) and uses dynamic access to public biomedical tools. On two challenging benchmarks that provide only patient clinical features, LiteOdyssey achieved state-of-the-art performance, with an overall disease Recall@1 of 59.3% over the combined 1,243 cases of LIRICAL (n = 370) and the PhenoPacket Store (n = 873). Both benchmarks have a high proportion of ultra-rare disease (a prevalence below 1 in 1,000,000, with ultra-rare shares of approximately 45% and 52.8%, respectively). On the more difficult PhenoPacket subset, where causal diseases were not mapped to Orphanet in our rarity-mapping pipeline, LiteOdyssey achieved 60.7% Recall@1, compared with 10.7% for the same baseline model (GPT-5.4) without tools. This performance was achieved without fine-tuning, multi-agent ensembles, or a large case-retrieval database. Gains were also observed in the following: on cases never seen during development, on a private cohort of real-world rare disease patients, and on a smaller open-weights model. LiteOdyssey suggests a path toward rare-disease AI systems that are accurate, easier to deploy, and more transparent for physician review.

Chinese Translation

大多数医疗人工智能系统通过扩展额外的设备来提高性能：更多的微调数据、更多的代理和/或更大的检索数据库。然而，在罕见疾病诊断中，这种扩展可能导致难以部署、审计和维护的系统。我们探讨了是否可以通过扩展单个人工智能代理的推理链来实现最先进的诊断性能：通过人机协作开发的诊断策略来指导它，并结合可自由获取的生物医学工具进行增强。我们介绍了LiteOdyssey，一个轻量级的罕见疾病诊断框架，该框架通过临床遗传学工作流程指导推理语言模型。该框架是通过人类反馈的策略迭代（Policy Iteration with Human Feedback, PIHF）开发的，并使用动态访问公共生物医学工具。在两个具有挑战性的基准测试中，LiteOdyssey在仅提供患者临床特征的情况下实现了最先进的性能，在LIRICAL（n = 370）和PhenoPacket Store（n = 873）合计1,243个案例中，整体疾病召回率（Recall@1）为59.3%。这两个基准测试中，超罕见疾病的比例较高（患病率低于1/1,000,000，超罕见疾病的比例分别约为45%和52.8%）。在更困难的PhenoPacket子集上，因因果疾病未在我们的稀有映射管道中映射到Orphanet，LiteOdyssey实现了60.7%的召回率，而同一基线模型（GPT-5.4）在没有工具的情况下仅为10.7%。这一性能是在没有微调、多代理集成或大型案例检索数据库的情况下实现的。我们还观察到在以下方面的提升：在开发过程中从未见过的案例、真实世界罕见疾病患者的私有队列，以及一个较小的开放权重模型。LiteOdyssey为构建准确、易于部署且更透明的罕见疾病人工智能系统提供了一条路径，以便于医生审查。

View on arXiv Download PDF AI Translation

cs.AI / 74 / 2606.16152

The Quality-Utility Paradox: Why High-Reward Data Impairs Small Model Mathematical Reasoning

质量-效用悖论：为何高奖励数据会削弱小模型的数学推理能力

Qian, Haolong, Yang, Xianliang, ma, Yinuo, Che, Lirong, Lu, Feng, Guo, Ye, Song, Lei, Bian, Jiang, Yuan, Chun

Abstract

Knowledge distillation from powerful reasoning models is widely used to improve Small Language Models (SLMs) on mathematical reasoning, often assuming that traces with higher reward model scores provide more useful supervision. We identify a counterintuitive \textbf{Quality-Utility Paradox} in mathematical reasoning distillation. Data refined or synthesized by a stronger Oracle obtains higher perceived quality according to reward models, yet consistently underperforms traces generated by the SLM itself and selected through rejection sampling across Qwen2.5, LLaMA-3, and DeepSeek families. Our analysis shows that Oracle refinement couples logical repair with distributional drift away from the SLM's native reasoning distribution. This drift increases the learner's adaptation cost and can outweigh the benefit of improved reasoning logic. To test this mechanism, we introduce \textbf{Style-Aligned Refinement}, which preserves the native trajectory of the SLM while retaining logical repair from the Oracle. This intervention lowers adaptation cost and restores downstream utility. These findings suggest that effective mathematical reasoning distillation should jointly optimize perceived solution quality and learner-data compatibility, rather than relying solely on reward-model scores. The datasets and code are available at https://github.com/Dracoqhl/Quality-Utility-Paradox.

Chinese Translation

从强大的推理模型中进行知识蒸馏被广泛应用于提升小型语言模型（SLMs）在数学推理方面的表现，通常假设具有更高奖励模型评分的轨迹提供了更有用的监督。然而，我们在数学推理蒸馏中识别出一个反直觉的 extbf{质量-效用悖论}。由更强的Oracle精炼或合成的数据根据奖励模型获得了更高的感知质量，但在Qwen2.5、LLaMA-3和DeepSeek系列中，始终表现不及由SLM自身生成并通过拒绝采样选择的轨迹。我们的分析表明，Oracle的精炼将逻辑修复与SLM原生推理分布的分布漂移相结合。这种漂移增加了学习者的适应成本，可能超过改善推理逻辑的益处。为了验证这一机制，我们提出了 extbf{风格对齐精炼}，该方法在保留SLM原生轨迹的同时，保持来自Oracle的逻辑修复。这一干预降低了适应成本并恢复了下游效用。这些发现表明，有效的数学推理蒸馏应共同优化感知解决方案质量和学习者与数据的兼容性，而不是仅仅依赖于奖励模型评分。数据集和代码可在 https://github.com/Dracoqhl/Quality-Utility-Paradox 获取。

View on arXiv Download PDF AI Translation

cs.AI / 75 / 2606.16167

AI Pluralism and the Worlds It Misses

人工智能的多元主义及其遗漏的世界

Mushkani, Rashid

Abstract

AI pluralism is often framed as a problem of representing diverse values, preferences, users, or outputs. This paper argues that this framing is incomplete because AI systems also impose ontologies: they define what counts as an entity, relation, feature, harm, benefit, and valid form of evidence. We define ontological flattening as the conversion of situated, contested, and historically specific meanings into a restricted technical category, proxy, aggregation rule, or benchmark target that is treated as neutral and difficult to contest. The paper develops a bounded conceptual and qualitative synthesis across value pluralism, pluralistic alignment, participatory and democratic AI, procedural justice, science and technology studies, accountability research, aggregate themes from 11 expert interviews, and three urban AI companion cases. The cases illustrate how pluralistic methods can improve or structure model behavior while still compressing categories, proxies, aggregation rules, and revision rights before affected actors have procedural standing. We introduce Pluralistic Lifecycle Governance (PLG) as a preliminary qualitative audit scaffold for documenting ontological openness, epistemic inclusion, procedural authority, evaluation pluralism, and lifecycle accountability. PLG is not presented as a validated scoring instrument; it is a framework for making the evidence and governance conditions of pluralistic AI explicit.

Chinese Translation

人工智能多元主义通常被视为一个代表多样化价值观、偏好、用户或输出的问题。本文认为这种框架是不完整的，因为人工智能系统还会施加本体论：它们定义了什么被视为实体、关系、特征、伤害、利益和有效证据的形式。我们将本体论扁平化定义为将特定情境、争议和历史特定的意义转化为一个有限的技术类别、代理、聚合规则或基准目标，这些被视为中立且难以争辩。本文在价值多元主义、多元对齐、参与式和民主人工智能、程序正义、科学与技术研究、问责研究、11次专家访谈的聚合主题以及三个城市人工智能伴侣案例之间进行了有限的概念和定性综合。这些案例展示了多元方法如何改善或构建模型行为，同时在受影响的参与者尚未获得程序地位之前，仍然压缩类别、代理、聚合规则和修订权。我们引入了多元生命周期治理（Pluralistic Lifecycle Governance, PLG）作为一个初步的定性审计框架，用于记录本体开放性、认识包容性、程序权威性、评估多元性和生命周期问责制。PLG并不是作为一个经过验证的评分工具呈现；它是一个使多元人工智能的证据和治理条件明确化的框架。

View on arXiv Download PDF AI Translation

cs.AI / 76 / 2606.16173

TimeVista: Exploring and Exploiting Vision-Language Models as Judges for Time Series Forecasting

TimeVista：探索和利用视觉语言模型作为时间序列预测的评判者

Chen, Zhi, Wang, Yuxuan, Wu, Jialong, Liu, Yong, Zhang, Haoran, Su, Xingjian, Wang, Jianmin, Long, Mingsheng

Abstract

High-quality time series forecasting is pivotal for real-world decision-making. However, traditional point-wise metrics often fail to reveal complex temporal patterns and align poorly with human intuitive preferences. While the ''LLM-as-a-Judge'' paradigm has revolutionized text evaluation by providing flexible, human-aligned judgment, its application to time series remains largely unexplored. In this paper, we leverage Vision-Language Models (VLMs) as judges for time series forecasting, harnessing their ability to comprehend time series plots grounded in textual information. Specifically, we propose a novel framework integrating micro- and macro-level judgments informed by contextual information to evaluate time series forecasting. To this end, we introduce TimeVista, a comprehensive VLM-as-a-Judge benchmark comprising 5563 time series samples paired with detailed evaluation rubrics. Extensive meta-evaluations demonstrate that VLMs are highly reliable judges, achieving significantly higher consistency with human preferences than conventional metrics. Building upon our benchmark, we comprehensively assess recent Time Series Foundation Models (TSFMs) under the VLM-as-a-Judge paradigm. Our results demonstrate that VLMs serve as robust and interpretable judges, providing a comprehensive, human-aligned standard for evaluating time series models.

Chinese Translation

高质量的时间序列预测对现实世界的决策至关重要。然而，传统的点对点指标往往无法揭示复杂的时间模式，并且与人类直观偏好的对齐程度较差。虽然“LLM作为评判者”（LLM-as-a-Judge）范式通过提供灵活的人类对齐判断，彻底改变了文本评估，但其在时间序列中的应用仍然很大程度上未被探索。本文中，我们利用视觉语言模型（VLMs）作为时间序列预测的评判者，利用它们理解基于文本信息的时间序列图的能力。具体而言，我们提出了一个新颖的框架，整合了基于上下文信息的微观和宏观判断，以评估时间序列预测。为此，我们引入了TimeVista，一个全面的VLM作为评判者基准，包含5563个时间序列样本及详细的评估标准。广泛的元评估表明，VLMs是高度可靠的评判者，与人类偏好的一致性显著高于传统指标。在我们的基准基础上，我们在VLM作为评判者的范式下全面评估了近期的时间序列基础模型（TSFMs）。我们的结果表明，VLMs作为强大且可解释的评判者，为评估时间序列模型提供了全面的人类对齐标准。

View on arXiv Download PDF AI Translation

cs.AI / 77 / 2606.16175

PAL-Bench: Evidence-Grounded Profile Reconstruction from Longitudinal Personal Albums

PAL-Bench：基于证据的纵向个人相册档案重建

Yan, Qiwei, Yuan, Zhiqiang, Jia, Zexi, Hu, Nanxing, Lyu, Kailin, Zhou, Jie, Zhang, Jinchao

Abstract

Longitudinal personal albums are weak-schema multimodal databases: noisy perceptual records whose key facts require joins across faces, text, timestamps, locations, and repeated events. Existing visual, video, document, and lifelog benchmarks test sub-problems, but not album-scale profile reconstruction with social identity binding and evidence citation. Benchmarking this task is difficult because the ground truth needed for evaluation--owner profiles, social graphs, face-name maps, and evidence provenance--is private state that real albums cannot safely release. We introduce PAL-Bench, a controlled benchmark for evidence-grounded reconstruction under a public-record contract. Its Evidence Compiler builds latent private worlds, programs target-level evidence paths, renders album pixels, re-measures them through perception pipelines, and exports audited public/private views. Agents receive only perception-derived public records; targets, identifier maps, and evidence paths remain hidden. PAL-Bench contains 50 synthetic users, 36,659 public photo records, and 2,799 targets over owner facts, identities, and relations. A privacy-preserving audit with 10 participants confirms that PAL-Bench evidence structures match real private albums, though equivalent releases remain privacy-prohibitive. Across seven systems and two compute-matched diagnostics, a seven-metric protocol reveals a gap between plausible profile summarization and faithful social reconstruction: systems recover some owner facts but struggle with recurring identities and evidence citation. PAL-TRACE, a reference framework that freezes identity bindings before owner-fact mining, performs best but leaves hard identity resolution far from solved. PAL-Bench provides a testbed for perceptual entity resolution, multimodal data integration, temporal evidence aggregation, and provenance-aware structured prediction.

Chinese Translation

纵向个人相册是弱模式多模态数据库：噪声感知记录，其关键事实需要跨越面孔、文本、时间戳、地点和重复事件进行连接。现有的视觉、视频、文档和生活日志基准测试了子问题，但并未针对具有社会身份绑定和证据引用的相册规模档案重建进行测试。评估这一任务的基准测试困难，因为所需的真实情况——所有者档案、社交图谱、面孔-姓名映射和证据来源——是私人状态，真实相册无法安全发布。我们引入了PAL-Bench，这是一个在公共记录合同下进行基于证据的重建的受控基准。其证据编译器构建潜在的私人世界，编程目标级证据路径，渲染相册像素，通过感知管道重新测量它们，并导出经过审计的公共/私人视图。代理仅接收感知派生的公共记录；目标、标识符映射和证据路径保持隐藏。PAL-Bench包含50个合成用户、36,659个公共照片记录和2,799个关于所有者事实、身份和关系的目标。通过10名参与者的隐私保护审计确认，PAL-Bench的证据结构与真实私人相册相匹配，尽管等效发布仍然存在隐私限制。在七个系统和两个计算匹配的诊断中，一个七项指标的协议揭示了合理档案总结与忠实社会重建之间的差距：系统恢复了一些所有者事实，但在重复身份和证据引用方面存在困难。PAL-TRACE，一个在所有者事实挖掘之前冻结身份绑定的参考框架，表现最佳，但在解决困难身份方面仍然远未解决。PAL-Bench为感知实体解析、多模态数据集成、时间证据聚合和知识来源感知的结构化预测提供了测试平台。

View on arXiv Download PDF AI Translation

cs.AI / 78 / 2606.16206

Measuring Whether LLM Tutors Teach or Solve: A Diagnostic for Educational Impact

测量大型语言模型辅导是否在教学或解题：教育影响的诊断

Yao, Junyi, Zheng, Zihao, Li, Baichuan

Abstract

Large language models are increasingly proposed as educational tutors, yet stronger task-solving ability does not necessarily imply stronger learning support. Motivated by recent calls to measure the social impact of NLP systems in practice, we study whether public LLM tutoring benchmarks distinguish learning-supportive behavior from mere answer production. We propose a lightweight diagnostic based on the gap between solving-oriented and pedagogy-oriented benchmark performance. Using public MathTutorBench leaderboard results, we show that these dimensions are only partially aligned: across eight publicly reported models, the correlation between solving and pedagogy composites is 0.421, and several models shift meaningfully in rank when evaluation moves from solving to pedagogy. We then analyze the public TutorBench sample and show that agency-relevant behaviors are explicitly encoded in benchmark rubrics, especially in active-learning settings that reward guiding questions, calibrated hints, and non-disclosive scaffolding. Together, these findings suggest that educational-impact evaluation should not treat task success as a sufficient proxy for learning support. We argue that public tutoring benchmarks can better support positive-impact evaluation by reporting solving-oriented and pedagogy-oriented scores separately and by making disclosure-sensitive, student-agency-preserving criteria more explicit.

Chinese Translation

大型语言模型越来越多地被提议作为教育辅导工具，但更强的解题能力并不一定意味着更强的学习支持。受到近期关于衡量自然语言处理（NLP）系统社会影响的呼吁的启发，我们研究公共大型语言模型（LLM）辅导基准是否能够区分支持学习的行为与单纯的答案生成。我们提出了一种轻量级的诊断方法，基于解题导向和教学导向基准表现之间的差距。利用公共的 MathTutorBench 排行榜结果，我们显示这两个维度仅部分对齐：在八个公开报告的模型中，解题和教学综合得分之间的相关性为 0.421，并且当评估从解题转向教学时，多个模型的排名发生了显著变化。随后，我们分析了公共的 TutorBench 样本，表明与学习者自主性相关的行为在基准评分标准中被明确编码，特别是在奖励引导性问题、适度提示和非揭示性支架的主动学习环境中。综合这些发现，我们认为教育影响评估不应将任务成功视为学习支持的充分代理。我们主张公共辅导基准可以通过分别报告解题导向和教学导向的得分，并使与披露敏感、保护学生自主性的标准更加明确，从而更好地支持积极影响的评估。

View on arXiv Download PDF AI Translation

cs.AI / 79 / 2606.16210

Sensor-Conditioned Representation Learning via Scene-Relevant Observation Quotients

基于传感器条件的场景相关观察商的表示学习

Jiao, Yan, Ho, Pin-Han, Peng, Limei

Abstract

Learned representations in intelligent sensing systems are often evaluated by reconstruction fidelity or downstream prediction accuracy, but these criteria do not specify which latent distinctions are justified by the sensing process. In sensor-conditioned environments, nuisance factors can change measurements without changing the scene, while distinct scenes may be indistinguishable under limited sensing capability. This paper formulates sensor-conditioned representation correctness as preserving sensing-supported scene distinctions while suppressing nuisance-induced and sensor-unsupported variation. We introduce the scene-relevant observation quotient, a representation target induced by sensing-supported distinguishability after nuisance canonicalization, and develop Observation-Quotient Tucker-Structured Autoencoding (OQ-TSAE), a scene-nuisance factorized framework with diagnostics for false distinction, false merge, nuisance sensitivity, and latent ordering consistency. Experiments on a controlled benchmark show that quotient-consistent supervision improves representation-correctness diagnostics over reconstruction-oriented, metric-learning, and contrastive-learning baselines. Sensitivity, perturbation, and ablation studies show the importance of quotient-aligned supervision, reliable quotient relations, and quotient geometry. Complementary real-radar experiments show that a reconstruction-only OQ-TSAE variant retains competitive downstream utility, robustness under observation degradation, and low seed-to-seed variability. These results suggest that sensor-conditioned representations should be evaluated not only by predictive utility, but also by whether their latent geometry preserves sensing-justified scene distinctions.

Chinese Translation

智能传感系统中学习到的表示通常通过重建保真度或下游预测准确性进行评估，但这些标准并未明确传感过程所依据的潜在区别。在传感器条件环境中，干扰因素可以在不改变场景的情况下改变测量，而在有限的传感能力下，不同的场景可能无法区分。本文将传感器条件下的表示正确性定义为在抑制干扰引起的变化和传感器不支持的变化的同时，保留传感支持的场景区别。我们引入了场景相关观察商，这是一种在干扰规范化后由传感支持的可区分性引发的表示目标，并开发了观察商塔克结构自编码器（Observation-Quotient Tucker-Structured Autoencoding, OQ-TSAE），这是一个场景-干扰因素分解框架，具有虚假区分、虚假合并、干扰敏感性和潜在顺序一致性的诊断功能。在一个受控基准上的实验表明，与重建导向、度量学习和对比学习基线相比，观察商一致的监督提高了表示正确性诊断。敏感性、扰动和消融研究显示了观察商对齐监督、可靠的观察商关系和观察商几何的重要性。补充的真实雷达实验表明，仅重建的OQ-TSAE变体在下游效用、观察降级下的鲁棒性和低种子间变异性方面保持了竞争力。这些结果表明，传感器条件下的表示不仅应通过预测效用进行评估，还应考虑其潜在几何是否保留了传感支持的场景区别。

View on arXiv Download PDF AI Translation

cs.AI / 80 / 2606.16222

Latent Thought Flow: Efficient Latent Reasoning in Large Language Models

潜在思维流：大规模语言模型中的高效潜在推理

Zou, Xiandong, Huang, Jing, Li, Jianshu, Zhou, Pan

Abstract

Large Language Models (LLMs) increasingly rely on intermediate reasoning, yet explicit Chain-of-Thought (CoT) suffers from a linguistic space bottleneck: each thought must be decoded into tokens, causing high inference overhead. Latent reasoning moves deliberation into continuous space, but existing methods mostly learn deterministic or reward-maximizing paths, lacking a principled way to allocate probability across trajectories with different correctness and costs. We propose Latent Thought Flow (LTF), which models reasoning as variable-length continuous trajectories and trains a sampler to match a reward-induced posterior over answer quality and computation cost. We instantiate this with a continuous GFlowNet using stochastic latent transitions. To handle sparse answer supervision, we introduce an Entropy-Weighted Subtrajectory Balance objective for intermediate rewards and a reference-prior regularizer to anchor exploration. Experiments under finetuning and transfer learning settings show that LTF outperforms explicit CoT and latent reasoning baselines, improving accuracy by 9.5% while reducing reasoning length by 27.2% on average compared with strong latent reasoning baselines.

Chinese Translation

大规模语言模型（LLMs）越来越依赖中间推理，然而显式的思维链（Chain-of-Thought, CoT）面临语言空间瓶颈：每个思维必须解码为标记，导致高推理开销。潜在推理将思考过程转移到连续空间，但现有方法大多学习确定性或奖励最大化路径，缺乏在不同正确性和成本的轨迹之间分配概率的原则性方法。我们提出了潜在思维流（Latent Thought Flow, LTF），将推理建模为可变长度的连续轨迹，并训练一个采样器以匹配基于奖励的后验分布，关注答案质量和计算成本。我们通过使用随机潜在转移的连续GFlowNet实现这一模型。为了处理稀疏的答案监督，我们引入了一个熵加权子轨迹平衡目标用于中间奖励，并使用参考先验正则化器来锚定探索。在微调和迁移学习的实验中，结果表明LTF在准确性上比显式的CoT和潜在推理基线提高了9.5%，同时与强大的潜在推理基线相比，推理长度平均减少了27.2%。

View on arXiv Download PDF AI Translation

cs.AI / 81 / 2606.16276

SpecAlign: Efficient Specification-Grounded Alignment of Large Language Models via Synthetic Data

SpecAlign：通过合成数据高效实现大型语言模型的规范基础对齐

Wang, Wenjie, Huang, Yue, Yuan, Zhengqing, Bao, Han, Du, Shiyi, Ma, Yuchen, Zhao, Yue, Ye, Yanfang, Zhang, Xiangliang

Abstract

As large language models (LLMs) are increasingly deployed in real-world applications, alignment is no longer governed by a single universal notion of safety or helpfulness, but instead by provider- or application-specific model specifications. These specifications are typically long, structured, and frequently updated, yet existing alignment pipelines lack a systematic mechanism to operationalize them as training signals. In this paper, we propose specification-grounded alignment, a new alignment paradigm that treats provider-authored model specifications as the primary alignment target rather than abstract principles or static benchmarks. To instantiate this paradigm, we introduce SpecAlign, a framework that synthesizes alignment data directly from specification documents. SpecAlign combines structured rule annotation, controllable specification instantiation, and multi-agent adversarial data synthesis to generate fine-grained, boundary-aware preference pairs that capture both compliant behaviors and meaningful specification violations. Experiments across multiple model specifications and backbone models demonstrate that training with SpecAlign consistently improves rule compliance while preserving general capabilities and avoiding over-conservative behavior. These results suggest that grounding alignment in explicit model specifications enables rapid, precise, and scalable adaptation of LLM behavior to evolving policy requirements.

Chinese Translation

随着大型语言模型（LLMs）在现实世界应用中的日益普及，对齐不再仅仅由单一的普遍安全性或有用性概念所主导，而是由提供者或应用特定的模型规范所决定。这些规范通常较长、结构化且频繁更新，但现有的对齐流程缺乏将其系统化为训练信号的机制。在本文中，我们提出了一种规范基础对齐的新对齐范式，将提供者撰写的模型规范视为主要的对齐目标，而非抽象原则或静态基准。为了实现这一范式，我们引入了SpecAlign，一个直接从规范文档合成对齐数据的框架。SpecAlign结合了结构化规则注释、可控的规范实例化和多智能体对抗数据合成，以生成细粒度、边界感知的偏好对，这些偏好对既捕捉合规行为，又反映有意义的规范违反。针对多个模型规范和基础模型的实验表明，使用SpecAlign进行训练能够持续提高规则合规性，同时保持一般能力并避免过于保守的行为。这些结果表明，将对齐基于明确的模型规范能够快速、精准且可扩展地适应不断变化的政策要求。

View on arXiv Download PDF AI Translation

cs.AI / 82 / 2606.16307

State-Grounded Multi-Agent Synthetic Data Generation for Tool-Augmented LLMs

基于状态的多智能体合成数据生成用于工具增强的大型语言模型

Khedar, Rahul, Eshita, Thondapu, Sneha Teja Sree Reddy, Malhotra, Mayank, Das, Arup, Chandra, Jitesh, Chuang, Yun-Shiuan, Kulkarni, Chaitanya, Menon, Arun, Pang, Linsey, Karn, Avinash, V, Mouli, Mehrotra, Prakhar

Abstract

Training tool-augmented LLM agents requires large corpora of multi-turn, tool-grounded conversational data that is expensive to annotate, privacy-constrained in production settings, and largely absent from public datasets. We present StateGen, a synthetic data generation platform that produces scored, reasoning-trace-rich training conversations by orchestrating a four-role LLM loop: a persona-conditioned user simulator, an agent under test, a state-grounded tool simulator, and a multi-axis LLM judge. The key architectural contribution is an authoritative state manager that maintains a structured world-state object across turns, enforcing a backend-is-truth invariant that eliminates the dominant class of tool-call hallucinations by construction. StateGen extends naturally to hierarchical multi-agent settings by declaring sub-agents as tools, all sharing a single state object. We report results on 64,698 evaluated conversations across three production corpora: tool-call hallucination scores reach 9.66/10, the system supports persona-driven variation via a 23-dimensional trait vector, and a cleanly separated train and golden evaluation set split confirms the data is not memorization bait (per-criterion gap analysis). Comparison with eight external systems shows that no single publicly available platform combines multi-turn generation, state-grounded tool simulation, hierarchical multi-agent support, and built-in judge scoring.

Chinese Translation

训练工具增强的大型语言模型（LLM）代理需要大量的多轮、基于工具的对话数据，这些数据的注释成本高昂，在生产环境中受到隐私限制，并且在公共数据集中几乎不存在。我们提出了StateGen，一个合成数据生成平台，通过协调一个四角色的LLM循环，生成带有评分和推理轨迹丰富的训练对话：一个基于角色的用户模拟器、一个待测代理、一个基于状态的工具模拟器，以及一个多轴LLM评判者。关键的架构贡献是一个权威的状态管理器，它在轮次之间维护一个结构化的世界状态对象，强制执行一个后端即真理的不变性，从而通过构造消除了主导类别的工具调用幻觉。StateGen自然扩展到层次化的多智能体设置，通过将子代理声明为工具，所有子代理共享一个状态对象。我们报告了在三个生产语料库中评估的64,698个对话的结果：工具调用幻觉得分达到9.66/10，系统支持通过23维特征向量驱动的角色变化，干净分开的训练集和黄金评估集划分确认数据不是记忆诱饵（按标准差距分析）。与八个外部系统的比较显示，没有任何单一的公共平台结合了多轮生成、基于状态的工具模拟、层次化多智能体支持和内置评判评分。

View on arXiv Download PDF AI Translation

cs.AI / 83 / 2606.16319

Architectural Wisdom: A Framework for Governing Optimization in AI Systems

建筑智慧：人工智能系统优化治理框架

Chang, Edward Y.

Abstract

Modern AI systems exhibit structural failures that capability scaling alone does not reliably fix: they optimize under-specified objectives with no architectural mechanism to question whether the objective should be optimized at all. Engagement maximization can amplify harmful pathways; tool-using agents can commit irreversible actions; preference-trained language models can become sycophantic. We argue that this failure is a wisdom problem, not an intelligence problem. We use "wisdom" in a deliberately architectural sense, not as a claim about virtue, consciousness, or moral omniscience. Intelligence accepts a goal and optimizes within it; wisdom interrogates whether the goal should be optimized at all. The two are separable architectural properties. We propose architectural wisdom as a corrigible objective-governance layer above the optimization substrate. The layer makes three structural commitments explicit and nondegenerate before any action: temporal horizon, relational boundary, and irreversibility. It is realized by four components (Structural Utility Transform, Moral Admissibility Interface, Arbitration and Escalation Controller, Value Revision Channel) that compute a six-coordinate wisdom tuple over horizon, relational coverage, irreversibility, admissibility, value revision, and auditability. We motivate the architecture by eight cases drawn from contemporary AI failures, secular wisdom traditions, and hard ethical situations, and defend the distinction against the intelligence-completeness thesis using goal-questioning over goal-taking, Bostrom's orthogonality, structural separation in our exemplar cases, and persistent failure modes despite capability scaling. The framework is the conceptual contract for a larger architecture whose formal specifications and empirical validation are developed in subsequent work.

Chinese Translation

现代人工智能系统表现出结构性故障，仅靠能力扩展无法可靠修复：它们在未充分指定的目标下进行优化，缺乏质疑该目标是否应被优化的建筑机制。参与度最大化可能会加剧有害路径；使用工具的智能体可能会做出不可逆的行动；经过偏好训练的语言模型可能会变得谄媚。我们认为这一失败是一个智慧问题，而非智能问题。我们在此将“智慧”以一种故意的建筑意义使用，而不是作为对美德、意识或道德全知的主张。智能接受一个目标并在其范围内进行优化；智慧则质疑该目标是否应被优化。这两者是可分离的建筑属性。我们提出建筑智慧作为优化基础之上的可纠正目标治理层。该层在任何行动之前明确并非退化地做出三个结构性承诺：时间视野、关系边界和不可逆性。它通过四个组件（结构效用变换、道德可接受性接口、仲裁与升级控制器、价值修订通道）实现，计算出一个关于时间视野、关系覆盖、不可逆性、可接受性、价值修订和可审计性的六维智慧元组。我们通过八个案例来激励这一架构，这些案例源自当代人工智能的失败、世俗智慧传统和艰难的伦理情境，并通过对目标质疑与目标接受的区分、博斯特罗姆的正交性、我们示例案例中的结构分离，以及尽管能力扩展仍然存在的持续失败模式来捍卫这一区分。该框架是一个更大架构的概念合同，其正式规范和实证验证将在后续工作中展开。

View on arXiv Download PDF AI Translation

cs.AI / 84 / 2606.16328

AdaSTORM: Scaling LLM Reasoning on Dynamic Graphs via Adaptive Spatio-Temporal Multi-Agent Collaboration

AdaSTORM：通过自适应时空多智能体协作扩展动态图上的大规模语言模型推理

Hao, Bing, Wang, Ruijie, Qian, Haodong, Chu, Yunlong, Liu, Yuhang, Lin, Yumeng, Shao, Minglai, Li, Jianxin

Abstract

Large Language Models (LLMs) demonstrate remarkable potential in dynamic graph reasoning, but suffer from a scaling bottleneck: current models can only handle graphs with tens of nodes, constrained by exponential reasoning overhead and finite context windows. While multi-agent systems (MAS) offer collective reasoning and topology-aware orchestration, capabilities naturally suited for graph-structured tasks, their application to dynamic graphs remains unexplored. This paper presents Scaling LLM Reasoning on Dynamic Graphs via Adaptive Spatio-Temporal Multi-Agent Collaboration (AdaSTORM), a framework that reformulates large-scale dynamic graph reasoning into two stages: (i) Adaptive Partitioning, partitioning large-scale dynamic graphs into subregions that match the model's reasoning capacity while minimizing inference cost; and (ii) Collaborative Reasoning, aligning graph partition topologies with a spatio-temporal decoupled multi-agent architecture. AdaSTORM is the first multi-agent framework tailored for dynamic graph reasoning. Extensive experiments show that AdaSTORM successfully breaks through the scaling bottleneck, scaling reasoning to thousand-node graphs with over 90% accuracy across several large-scale dynamic graph settings without external tools, significantly outperforms seven competitive baselines. Furthermore, it achieves state-of-the-art accuracy on existing benchmarks and generalizes robustly to real-world datasets. The source code is available at: https://github.com/irisorchid107/AdaSTORM/.

Chinese Translation

大型语言模型（LLMs）在动态图推理方面展现出显著的潜力，但面临扩展瓶颈：当前模型仅能处理数十个节点的图，受限于指数级的推理开销和有限的上下文窗口。尽管多智能体系统（MAS）提供了集体推理和拓扑感知的协调能力，这些能力自然适合图结构任务，但其在动态图中的应用仍未被探索。本文提出了通过自适应时空多智能体协作（AdaSTORM）扩展动态图推理的框架，该框架将大规模动态图推理重构为两个阶段：(i) 自适应划分，将大规模动态图划分为与模型推理能力匹配的子区域，同时最小化推理成本；(ii) 协作推理，将图划分拓扑与时空解耦的多智能体架构对齐。AdaSTORM是首个为动态图推理量身定制的多智能体框架。大量实验表明，AdaSTORM成功突破了扩展瓶颈，将推理扩展到千节点图，在多个大规模动态图设置中实现了超过90%的准确率，且无需外部工具，显著优于七个竞争基线。此外，它在现有基准测试中达到了最先进的准确率，并在真实世界数据集上具有良好的泛化能力。源代码可在以下网址获取：https://github.com/irisorchid107/AdaSTORM/

View on arXiv Download PDF AI Translation

cs.AI / 85 / 2606.16329

Exploiting Search in Symbolic Numeric Planning with Patterns

利用模式在符号数值规划中的搜索

Cardellini, Matteo, Giunchiglia, Enrico

Abstract

In this paper, we present a procedure for numeric planning based on Symbolic Pattern Planning (SPP). Given a numeric planning problem $\Pi$, a pattern $\prec$ is a sequence of actions used to define a formula encoding the subsequences of $\prec$ executable from a starting state $S$. Cardellini, Giunchiglia, and Maratea (2024a) follow the Planning as Satisfiability approach by defining, at each step $n \ge 0$, a formula $\Pi^\prec_n$ in which $(i)$ the pattern $\prec$ is computed only for $n=0$ in the initial state $I$ of $\Pi$, and then exploited at each step $n$, $(ii)$ the starting state $S$ is set to $I$, and $(iii)$ the set $G$ of goals is required to hold in the last state that can be reached by one of the subsequences of $\prec$ concatenated $n$ times. The procedure begins with $n=0$, terminates as soon as $\Pi^\prec_n$ is satisfiable, and otherwise proceeds by incrementing $n$. In this paper, possibly at each step, $(i)$ we symbolically search for an intermediate state $P$ reachable from $I$, closer to a goal state, $(ii)$ dynamically recompute the pattern $\prec_h$ -- to be used in the next step -- in $P$, $(iii)$ refine the pattern $\prec_g$ used to reach $P$, and $(iv)$ start the new search from the state $S$ which can be either the initial state $I$ or the last computed intermediate state $P$, exploiting the computed patterns $\prec_g$ and $\prec_h$ to define the pattern $\prec$ to be used in the search. In particular, at each step, we define a formula $\Pi^{\prec}_{S,P}$ encoding the existence of a state $P'$ closer than $P$ to a goal state, with $P'$ reachable from the starting state $S$ when using the pattern $\prec$. We present different techniques for producing such formulas, each corresponding to a different strategy for exploring the search space. We prove their correctness and completeness, the latter under certain conditions.

Chinese Translation

在本文中，我们提出了一种基于符号模式规划（Symbolic Pattern Planning, SPP）的数值规划程序。给定一个数值规划问题 $ ext{Π}$，模式 $ ext{⟨p⟩}$ 是一系列动作，用于定义一个公式，该公式编码从起始状态 $ ext{S}$ 可执行的模式 $ ext{⟨p⟩}$ 的子序列。Cardellini、Giunchiglia 和 Maratea（2024a）遵循将规划视为可满足性（Planning as Satisfiability）的方法，在每一步 $n ext{≥} 0$ 中定义一个公式 $ ext{Π}^{ ext{⟨p⟩}}_n$，其中 $(i)$ 模式 $ ext{⟨p⟩}$ 仅在初始状态 $ ext{I}$ 的 $n=0$ 时计算，然后在每一步 $n$ 中利用，$(ii)$ 起始状态 $ ext{S}$ 设置为 $ ext{I}$，并且 $(iii)$ 目标集合 $ ext{G}$ 要求在可以通过将 $ ext{⟨p⟩}$ 连接 $n$ 次到达的最后状态中成立。该程序从 $n=0$ 开始，一旦 $ ext{Π}^{ ext{⟨p⟩}}_n$ 可满足则终止，否则通过递增 $n$ 继续进行。在本文中，可能在每一步中， $(i)$ 我们符号性地搜索从 $ ext{I}$ 可达的更接近目标状态的中间状态 $ ext{P}$， $(ii)$ 动态重新计算模式 $ ext{⟨p⟩}_h$——将在下一步中使用——在 $ ext{P}$ 中， $(iii)$ 精炼用于到达 $ ext{P}$ 的模式 $ ext{⟨p⟩}_g$，以及 $(iv)$ 从状态 $ ext{S}$ 开始新的搜索，$ ext{S}$ 可以是初始状态 $ ext{I}$ 或最后计算的中间状态 $ ext{P}$，利用计算出的模式 $ ext{⟨p⟩}_g$ 和 $ ext{⟨p⟩}_h$ 来定义将在搜索中使用的模式 $ ext{⟨p⟩}$。特别地，在每一步中，我们定义一个公式 $ ext{Π}^{ ext{⟨p⟩}}_{ ext{S,P}}$，编码一个状态 $ ext{P}'$ 的存在，该状态比 $ ext{P}$ 更接近目标状态，并且在使用模式 $ ext{⟨p⟩}$ 时从起始状态 $ ext{S}$ 可达 $ ext{P}'$。我们提出了生成这些公式的不同技术，每种技术对应于探索搜索空间的不同策略。我们证明了它们的正确性和完备性，后者在某些条件下成立。

View on arXiv Download PDF AI Translation

cs.AI / 86 / 2606.16330

Phase-Aware Guidance Injection for Recurrent MAPPO in Assembly-Line Disruption Recovery

面向阶段的指导注入用于装配线中断恢复的递归MAPPO

Huang, Xin, Wang, Yongcai, Zhang, Fengyi, Tao, Zhikun, Han, Yunjun, Wu, Naiqi

Abstract

Disruption recovery in industrial assembly lines requires timely decisions under machine faults, worker absence, and emergency orders. Existing methods either rely on rigid handcrafted recovery logic or learn adaptive policies that do not readily exploit heterogeneous external recovery knowledge at decision time to reduce abnormal recovery time (ART) and preserve on-time delivery (OTD). To address this gap, we propose a phase-aware guidance injection framework that augments a trained recurrent MAPPO (RMAPPO) scheduling policy through logit-level action bias during evaluation. The framework provides a unified decision-time interface for rule-based, replay-based, and online LLM-based guidance, while activating intervention only during abnormal and recovery phases. Experiments on a custom AssemblyLineEnv show that high-quality rule guidance yields the strongest gains, replay-based guidance degrades smoothly under imperfect availability, and online LLM guidance still provides useful intermediate improvements. These results show that decision-time guidance injection can exploit heterogeneous recovery hints without redesigning the actor.

Chinese Translation

工业装配线中的中断恢复需要在机器故障、工人缺席和紧急订单下进行及时决策。现有方法要么依赖于僵化的手工恢复逻辑，要么学习适应性策略，但在决策时未能充分利用异质的外部恢复知识，从而降低异常恢复时间（ART）并保持准时交付（OTD）。为了解决这一问题，我们提出了一种面向阶段的指导注入框架，该框架通过在评估期间的logit级别动作偏置增强训练好的递归MAPPO（RMAPPO）调度策略。该框架为基于规则、基于重放和在线LLM（大语言模型）指导提供了统一的决策时间接口，同时仅在异常和恢复阶段激活干预。在定制的AssemblyLineEnv上的实验表明，高质量的规则指导带来了最强的收益，基于重放的指导在不完美可用性下平滑降级，而在线LLM指导仍然提供了有用的中间改进。这些结果表明，决策时间的指导注入可以在不重新设计演员的情况下利用异质的恢复提示。

View on arXiv Download PDF AI Translation

cs.AI / 87 / 2606.16337

Medical Heuristic Learning: An LLM-Driven Framework for Interpretable and Auditable Clinical Decision Rules

医学启发式学习：一个基于大型语言模型的可解释和可审计临床决策规则框架

Xu, Wei, Yang, Ke, Luo, Gang, Zheng, Keli, Hu, Lingyan, Wang, Jing, Li, Kefeng

Abstract

Predictive modeling for clinical tabular data is central to clinical decision support and therefore requires not only strong predictive performance but also transparent decision logic. Although deep learning and tree-based ensemble methods can achieve high accuracy, their black-box nature remains a major obstacle to clinical deployment. This challenge is further compounded by common characteristics of medical data, including limited sample sizes, severe class imbalance, and feature evolution arising from changes in diagnostic criteria and clinical documentation. To address these issues, we propose Medical Heuristic Learning (MHL), an instantiation of the learning-beyond-gradients paradigm for clinical tabular prediction. Instead of relying on neural network weight updates, MHL uses a large language model (LLM)-driven workflow that integrates statistical probes, medical knowledge probes, rule synthesis, and code-level iterative refinement to optimize a deterministic and executable decision system. The resulting model is expressed not as opaque parameters, but as versioned pure-Python decision rules that are explicitly interpretable, fully auditable, and clinically grounded. MHL also supports continual learning by starting from previously validated rules and iteratively revising them using updated feature information under data drift or feature evolution. Comprehensive experiments on medical datasets show that MHL achieves performance comparable to state-of-the-art methods while maintaining strong behavior in small-sample and highly imbalanced settings. The results further indicate that this explicit rule update mechanism can help alleviate catastrophic forgetting under feature evolution. Overall, these findings suggest that non-gradient-based heuristic systems offer a transparent and adaptable alternative for high-stakes clinical decision support.

Chinese Translation

临床表格数据的预测建模是临床决策支持的核心，因此不仅需要强大的预测性能，还需要透明的决策逻辑。尽管深度学习和基于树的集成方法可以实现高准确性，但其黑箱特性仍然是临床部署的主要障碍。这个挑战因医学数据的常见特征而进一步加剧，包括有限的样本量、严重的类别不平衡，以及由于诊断标准和临床文档变化而导致的特征演变。为了解决这些问题，我们提出了医学启发式学习（Medical Heuristic Learning, MHL），这是一个超越梯度学习范式在临床表格预测中的实例。MHL不依赖于神经网络权重更新，而是采用基于大型语言模型（LLM）的工作流程，集成统计探测、医学知识探测、规则合成和代码级迭代优化，以优化一个确定性和可执行的决策系统。所得到的模型不是以不透明的参数形式表达，而是以版本化的纯Python决策规则形式呈现，这些规则是明确可解释的、完全可审计的，并且与临床实践相结合。MHL还支持持续学习，通过从先前验证的规则出发，利用数据漂移或特征演变下的更新特征信息进行迭代修订。对医学数据集的全面实验表明，MHL在性能上可与最先进的方法相媲美，同时在小样本和高度不平衡的情况下保持强大的表现。结果进一步表明，这种显式的规则更新机制可以帮助缓解特征演变下的灾难性遗忘。总体而言，这些发现表明，非梯度的启发式系统为高风险临床决策支持提供了一种透明且可适应的替代方案。

View on arXiv Download PDF AI Translation

cs.AI / 88 / 2606.16344

Whose hotel does the AI recommend? An algorithm audit of reputation signals in LLM-assisted hotel selection

人工智能推荐哪个酒店？对大语言模型辅助酒店选择中的声誉信号的算法审计

Baig, Mirza Samad Ahmed, Gillani, Syeda Anshrah, Ali, Asher

Abstract

Travelers increasingly ask large language model (LLM) assistants which hotel to book, making these systems gatekeepers of property visibility -- yet what moves their recommendations is undocumented. We conduct a pre-specified algorithm audit using a randomized choice-based conjoint: across personas, prompt templates, and twelve open-weight and proprietary models, assistants choose among five hotels whose guest rating, review volume and recency, management response, chain affiliation, price, eco-certification, and list position are independently randomized. We estimate the average marginal component effect of each signal on the probability of recommendation. Guest rating and price dominate (a top rating raises selection by 31.6 percentage points; a high price lowers it by 30.0), reproducing human valence-and-price primacy but over-weighting eco-certification and ignoring management response. List position -- a content-free artifact -- shifts recommendations causally, worth about \$12 per night. Stated reasons track revealed weights imperfectly. The findings ground generative engine optimization and the accountability of AI infomediaries in causal evidence.

Chinese Translation

旅行者越来越多地向大型语言模型（LLM）助手询问预订哪个酒店，这使得这些系统成为物业可见性的守门人——然而，推动其推荐的因素尚未被记录。我们进行了一项预先指定的算法审计，使用随机选择基础的联合分析：在不同的人物角色、提示模板以及十二个开放权重和专有模型之间，助手在五个酒店中进行选择，这些酒店的客人评分、评论数量和时效性、管理响应、连锁隶属、价格、生态认证和列表位置均被独立随机化。我们估计了每个信号对推荐概率的平均边际成分效应。客人评分和价格占主导地位（高评分使选择概率提高31.6个百分点；高价格使其降低30.0），再现了人类对价值和价格的优先考虑，但过度重视生态认证而忽视管理响应。列表位置——一种无内容的伪影——因果性地改变了推荐，价值约为每晚12美元。陈述的理由与揭示的权重不完全一致。研究结果为生成引擎优化和人工智能信息中介的问责制提供了因果证据基础。

View on arXiv Download PDF AI Translation

cs.AI / 89 / 2606.16364

Looking Is Not Picking: An Attention-Segment Account of Tool-Selection Failures in LLM Agents

观察并非选择：大型语言模型代理工具选择失败的注意力-分段解释

Chen, Shiyang

Abstract

LLM agents mis-call tools, and the natural guess is that the model failed to see the right tool in a crowded harness. We show the opposite through a lens concurrent work sets aside -- the model's attention to labeled tool-definition segments. On real BFCL failures, by per-candidate attention argmax the model attends most to the correct tool 80% of the time (vs. 21% chance), and the gold is the under-attended segment on only 10%: it looks at the right tool and still picks wrong. This directly refutes the intuitive "crowded-harness / lost-in-the-middle" explanation: the failure is at the decision readout, not the harness, and we pin it there three ways. (1) Input vs. readout: repairing the prompt (reordering or duplicating the gold tool) recovers <=23% of failures, while readout-side interventions recover 59-91%. (2) Representation-invariance: two gold-pointed interventions in different representations -- an additive attention-logit bias and a residual-stream steering vector -- recover largely the same failures (per-task Jaccard 0.865 pooled, 0.79-0.91 per model), so the bottleneck is localized to the readout independent of which representation is poked. (3) A training-free, gold-free selector: per-segment attention closes most of the gold-free-vs-oracle gap on BFCL (+11.9 pts pooled function-name selection vs. +17.9-pt oracle headroom) and adds +14.9 pts on Seal-Tools; every model positive (exact McNemar p<=8e-4 each). Scopes differ: the causal attention-bias dose-response is bidirectional and monotonic on 10 mask-honoring models (3-32B), the full 0.5-32B span carrying only the correlational diagnostic; the deployable selector is evaluated on 5 single-turn models and does not yet transfer to a multi-turn loop.

Chinese Translation

大型语言模型代理在工具调用中出现错误，通常的猜测是模型未能在拥挤的环境中看到正确的工具。我们通过一个被并行工作忽视的视角展示了相反的观点——模型对标记工具定义分段的注意力。在真实的 BFCL 失败案例中，通过每个候选工具的注意力最大值，模型在 80% 的情况下关注到了正确的工具（相比之下，随机选择的概率仅为 21%），而黄金标准工具仅在 10% 的情况下未被充分关注：它看到了正确的工具却仍然选择错误。这直接反驳了直观的“拥挤环境/迷失其中”的解释：失败发生在决策输出阶段，而非工具环境，我们通过三种方式将其归因于此。(1) 输入与输出：修复提示（重新排序或重复黄金工具）仅能恢复最多 23% 的失败，而输出侧的干预能够恢复 59-91%。(2) 表示不变性：在不同表示下的两个黄金指向干预——加性注意力-逻辑偏差和残差流引导向量——在很大程度上恢复了相同的失败（每个任务的 Jaccard 指数为 0.865，模型之间为 0.79-0.91），因此瓶颈被局限于输出阶段，而与所使用的表示无关。(3) 一个无训练、无黄金标准的选择器：每个分段的注意力在 BFCL 上缩小了大部分无黄金标准与理想状态之间的差距（功能名称选择的 pooled 提升为 +11.9 分，相比之下理想状态提升为 +17.9 分），并在 Seal-Tools 上增加了 +14.9 分；每个模型均表现积极（确切的 McNemar p<=8e-4）。范围有所不同：因果注意力偏差的剂量反应在 10 个遵循掩码的模型（3-32B）上是双向且单调的，而完整的 0.5-32B 范围仅携带相关性诊断；可部署的选择器在 5 个单轮模型上进行了评估，但尚未转移到多轮循环中。

View on arXiv Download PDF AI Translation

cs.AI / 90 / 2606.16415

Posterior Twins: Distributional Behavioral Simulation for Enterprise Decisions

后验双胞胎：企业决策的分布式行为模拟

Das, Ankit

Abstract

Enterprise behavioral simulation requires more than producing a plausible response. Many decisions depend on the shape of a population under a proposed action: which segments accept, defect, hesitate, or move into risk-sensitive states. This paper introduces Posterior Twins, a memory-grounded digital-twin approach that represents likely behavior as an updated distribution under a specific decision context. We evaluate a family of Twinning Labs behavioral-model operating points on a 226-example held-out behavioral-response benchmark and report both modal accuracy and Wasserstein-1 distance. The results show that modal accuracy and distributional fidelity identify different operating regimes. TL-Twin Alpha achieves the lowest observed Wasserstein-1 distance in the reported result set ($W_1 = 1.16$), while TL-Twin Delta and TL-Twin Gamma provide balanced operating points near the modal-accuracy frontier. The paper frames these results as a systems result: governed memory, behavioral model routing, scenario orchestration, distributional aggregation, and auditability are necessary for turning simulated behavior into reusable enterprise decision evidence.

Chinese Translation

企业行为模拟不仅仅需要产生一个合理的响应。许多决策依赖于在特定行动下人群的形态：哪些细分市场接受、拒绝、犹豫或进入风险敏感状态。本文介绍了后验双胞胎（Posterior Twins），一种基于记忆的数字双胞胎方法，它在特定决策背景下将可能的行为表示为更新的分布。我们在一个包含226个示例的行为响应基准上评估了一系列双胞胎实验室（Twinning Labs）行为模型的操作点，并报告了模态准确性和Wasserstein-1距离。结果表明，模态准确性和分布保真度识别了不同的操作状态。TL-Twin Alpha在报告的结果集中实现了最低的Wasserstein-1距离（$W_1 = 1.16$），而TL-Twin Delta和TL-Twin Gamma则提供了接近模态准确性边界的平衡操作点。本文将这些结果框架化为系统结果：受控记忆、行为模型路由、场景编排、分布聚合和可审计性是将模拟行为转化为可重用的企业决策证据所必需的。

View on arXiv Download PDF AI Translation

cs.AI / 91 / 2606.16465

When Agent Automation Becomes Profitable: Quantifying and Insuring Autonomous AI Risk through Trace-Economic Underwriting

当代理自动化变得可盈利：通过追踪经济承保量化和保险自主人工智能风险

Xu, Binyan, Dai, Xilin, Yang, Fan, Zhang, Kehuan

Abstract

AI agents can now take irreversible actions in operational systems, but agent-caused losses are still not clearly assigned, priced, or transferred. Providers often disclaim consequential damages, users are left with uncompensated losses, and default human review limits the efficiency gains of automation. We ask when autonomous AI deployment can become economically acceptable despite failure risk. Our answer is to quantify risk at the customer-task-trace episode level and transfer it through insurance. Automation is acceptable when its expected benefit exceeds the premium, control cost, and remaining risk. This requires a defined role with bounded permissions and comparable traces. We introduce trace-economic underwriting, which maps tool-use traces to customer exposure and claimable loss, then uses this representation for pricing, control, and risk transfer. It uses deterministic economic labels rather than an LLM judge. In our trace-to-loss testbed, trace-economic pricing reduces pricing MAE from $17.7K to $569 and removes regressive cross-subsidy. A 300-trace expert audit accepts 295 labels unchanged. On 1,000 real SWE-smith traces, trace-conditioned controls reduce CVaR95 by 72%. Theorem~1 gives a finite-sample scope condition. We release code, labels, and audit sheets.

Chinese Translation

人工智能代理现在可以在操作系统中采取不可逆的行动，但代理造成的损失仍未明确分配、定价或转移。提供者通常会拒绝承担后果性损害，用户则面临未得到补偿的损失，而默认的人类审查限制了自动化的效率提升。我们探讨自主人工智能部署在何种情况下可以在失败风险下经济上变得可接受。我们的答案是，在客户-任务-追踪事件级别量化风险，并通过保险转移风险。当预期收益超过保费、控制成本和剩余风险时，自动化是可接受的。这需要一个具有界定角色和有限权限的定义，以及可比的追踪。我们引入了追踪经济承保，它将工具使用的追踪映射到客户暴露和可索赔损失，然后利用这种表示进行定价、控制和风险转移。它使用确定性的经济标签，而不是大型语言模型（LLM）判断。在我们的追踪到损失测试平台中，追踪经济定价将定价的平均绝对误差（MAE）从17.7K美元降低到569美元，并消除了回归性交叉补贴。300个追踪的专家审计接受了295个标签不变。在1000个真实的SWE-smith追踪中，追踪条件控制将条件价值风险（CVaR95）降低了72%。定理1给出了有限样本范围条件。我们发布了代码、标签和审计表。

View on arXiv Download PDF AI Translation

cs.AI / 92 / 2606.16478

Tensor-Coord: Algebraic Decomposition of Joint Plan Tensors for Conflict-Free Multi-Agent LLM Planning

张量协调：联合计划张量的代数分解用于无冲突多智能体大语言模型规划

Rastogi, Mudit

Abstract

Large language models (LLMs) remain limited in multi-agent planning because independently generated plans can create coordination failures such as spatial collisions, resource contention, and temporal deadlocks. We introduce Tensor-Coord, a multilinear algebra framework that represents the joint plan of N agents as a third-order tensor $T \in R^{N \times H \times A}$ over agents, timesteps, and actions. Canonical Polyadic (CP) and Tucker decompositions are used to identify latent coordination structure. The minimal epsilon-approximate CP rank R* defines a computable coordination complexity measure, with $CC(Pi)=(R*-N)/N$. We prove that R*=N is necessary and sufficient for plan independence. The residual $E=T-T_{R*}$ defines a conflict score over agent pairs, timesteps, and actions, localizing failures without domain-specific rules. Tucker factors provide interpretable agent roles, temporal phases, and action clusters that are converted into natural language constraints for iterative LLM replanning. Experiments on multi-robot delivery tasks across Easy (2 agents, 5x5 grid), Medium (3 agents, 5x5 grid), and Hard (4 agents, 5x5 grid) settings show convergence to conflict-free plans in 100% of 2-agent cases within 1.4 iterations on average, 80% of 3-agent cases within 3.2 iterations, and 60% of 4-agent cases within 4.0 iterations. CP rank scaled approximately linearly as $R*(N) = 3.9N + 0.5$, supporting its use as a predictor of coordination complexity.

Chinese Translation

大型语言模型（LLMs）在多智能体规划中仍然受到限制，因为独立生成的计划可能会导致协调失败，例如空间碰撞、资源争用和时间死锁。我们提出了Tensor-Coord，一个多线性代数框架，将N个智能体的联合计划表示为一个三阶张量T ∈ R^{N × H × A}，其中N表示智能体数量，H表示时间步，A表示动作。使用典范多项式（CP）和塔克（Tucker）分解来识别潜在的协调结构。最小的ε-近似CP秩R*定义了一个可计算的协调复杂度度量，公式为CC(Pi)=(R*-N)/N。我们证明了R*=N是计划独立性的必要和充分条件。残差E=T-T_{R*}定义了一个冲突评分，针对智能体对、时间步和动作进行局部化失败，而无需领域特定规则。塔克因子提供了可解释的智能体角色、时间阶段和动作聚类，这些聚类被转换为自然语言约束，以便进行迭代的LLM重新规划。在多机器人送货任务的实验中，在简单（2个智能体，5x5网格）、中等（3个智能体，5x5网格）和困难（4个智能体，5x5网格）设置下，显示出在100%的2智能体案例中平均在1.4次迭代内收敛到无冲突计划，在80%的3智能体案例中平均在3.2次迭代内收敛，在60%的4智能体案例中平均在4.0次迭代内收敛。CP秩大致呈线性增长，公式为R*(N) = 3.9N + 0.5，支持其作为协调复杂度预测因子的使用。

View on arXiv Download PDF AI Translation

cs.AI / 93 / 2606.16481

Steering Emotional Dynamics for Art Therapy: Controllable Narrative Script Generation through Hierarchically Guided LLM Agents

引导艺术治疗中的情感动态：通过分层引导的LLM代理生成可控叙事脚本

Wang, Suqing, Miao, Qinghai, Guo, Chao, Lv, Yisheng

Abstract

Art therapy plays a vital role in emotional healing, in which narrative creation acts as the primary vehicle for emotional expression. Given the inherently dynamic nature of emotions during healing, narratives with finely controlled emotional fluctuations enable individuals to safely project inner conflicts and achieve emotional catharsis. Recently, with the rapid development of Large Language Models (LLMs), automated narrative generation technology has provided a new pathway to support such artistic designs. However, while existing methods can produce fluent texts, they struggle to generate narratives that adhere to specified affective trajectories, failing to meet the demands of emotion-oriented psychological healing. To address these issues, this paper proposes EC-Script, an LLM agent-based framework that enables hierarchical control of the affective trajectory in narrative generation for emotional healing. To ensure that the generated narratives strictly follow the given emotional patterns, EC-Script establishes overall narrative direction through Emotion-Trajectory Planning, propels scene-level plot development with Character-Driven Scene Generation, and regulates local emotional changes of characters via Emotion-Controlled Script Writing. Ultimately, it outputs scene-by-scene script content that remains highly consistent with the preset affective trajectory. Experimental results demonstrate that EC-Script significantly outperforms baseline methods in affective trajectory adherence, exhibiting excellent and reliable emotional controllability, thereby providing effective technical support for AI-assisted emotional healing scenarios.

Chinese Translation

艺术治疗在情感疗愈中发挥着至关重要的作用，其中叙事创作是情感表达的主要载体。鉴于情感在疗愈过程中的动态特性，具有精细控制情感波动的叙事能够使个体安全地投射内心冲突，实现情感宣泄。近年来，随着大型语言模型（LLMs）的快速发展，自动化叙事生成技术为支持此类艺术设计提供了新的途径。然而，现有方法虽然能够生成流畅的文本，却难以产生符合特定情感轨迹的叙事，未能满足以情感为导向的心理疗愈需求。为了解决这些问题，本文提出了EC-Script，一个基于LLM代理的框架，能够在情感疗愈的叙事生成中实现情感轨迹的分层控制。为了确保生成的叙事严格遵循给定的情感模式，EC-Script通过情感轨迹规划（Emotion-Trajectory Planning）建立整体叙事方向，通过角色驱动场景生成（Character-Driven Scene Generation）推动场景级情节发展，并通过情感控制脚本写作（Emotion-Controlled Script Writing）调节角色的局部情感变化。最终，它逐场输出与预设情感轨迹高度一致的脚本内容。实验结果表明，EC-Script在情感轨迹遵循性方面显著优于基线方法，展现出卓越且可靠的情感可控性，从而为AI辅助的情感疗愈场景提供了有效的技术支持。

View on arXiv Download PDF AI Translation

cs.AI / 94 / 2606.16501

Post-Hoc Merging is Not Enough: Many-Shot Model Merging with Loss-Gap Balancing

后期合并不足够：基于损失差距平衡的多次模型合并

Im, Kyungjin, Kim, Miru, Eom, Chanin, Kwon, Minhae

Abstract

Model merging has become a practical post-training strategy for building a single multi-task large language model (LLM) by combining multiple task-specialized models. However, most existing approaches rely on post-hoc merging, in which task-specific models are merged only once after training. This one-shot aggregation often suffers from task interference, leading to information erasure across individual tasks. In this work, we show that replacing post-hoc merging with an iterative many-shot merging protocol is effective in improving multi-task performance. Building on this insight, we propose METIS, Mitigating Erasure from Task Interference for Stable many-shot merging. METIS is a loss-aware many-shot merging method that addresses information erasure in post-hoc merging through task-wise loss-gap weighting and consensus-based masking. Notably, METIS exhibits significant performance improvement on the worst-performing task, effectively mitigating information erasure. (Project page: https://imkyungjin.github.io/METIS/)

Chinese Translation

模型合并已成为构建单一多任务大型语言模型（LLM）的实用后训练策略，通过结合多个任务专用模型来实现。然而，大多数现有方法依赖于后期合并，即在训练后仅合并一次任务特定模型。这种一次性聚合往往会遭遇任务干扰，导致各个任务之间的信息丢失。在本研究中，我们展示了用迭代的多次合并协议替代后期合并在提高多任务性能方面的有效性。基于这一见解，我们提出了METIS（Mitigating Erasure from Task Interference for Stable many-shot merging），一种关注损失的多次合并方法，通过任务级损失差距加权和基于共识的掩蔽来解决后期合并中的信息丢失问题。值得注意的是，METIS在表现最差的任务上展现了显著的性能提升，有效减轻了信息丢失。（项目页面：https://imkyungjin.github.io/METIS/）

View on arXiv Download PDF AI Translation

cs.AI / 95 / 2606.16509

Model Graph Inductive Learning for Knowledge Graph Completion

知识图谱补全的模型图归纳学习

Khani, Mohommad Esmaei, Hasheminejad, Mahdieh, Taherkhani, Ali, Hajiabolhassan, Hossein

Abstract

Link prediction in knowledge graphs fundamentally depends on the quality of learned embeddings for entities and relations. However, most existing methods derive these embeddings by aggregating only the local neighborhood of each entity, neglecting the global structure of the knowledge graph. This limited view prevents models from capturing higher-level structural patterns that are essential for accurate and generalizable link prediction. To address these limitations, we introduce Model Graph Inductive Learning (\textbf{MGIL}), a framework that constructs a model graph by clustering entities based on the similarity of their incoming and outgoing relational structures or their entity types. A GNN is then applied to this model graph to produce embeddings that capture the global view of the knowledge graph. These embeddings subsequently serve as high-quality initial features %embeddings for the original knowledge graph, replacing random initialization and leading to more stable and expressive representations. Extensive experiments on standard and recently proposed inductive benchmarks demonstrate that MGIL achieves state-of-the-art or highly competitive performance in inductive link prediction, highlighting its effectiveness across diverse graph settings.

Chinese Translation

知识图谱中的链接预测根本上依赖于对实体和关系的嵌入学习质量。然而，现有的大多数方法仅通过聚合每个实体的局部邻域来推导这些嵌入，忽视了知识图谱的全局结构。这种有限的视角阻碍了模型捕捉到准确且可推广的链接预测所需的更高层次结构模式。为了解决这些局限性，我们提出了模型图归纳学习（Model Graph Inductive Learning，MGIL）框架，该框架通过基于实体的入边和出边关系结构的相似性或实体类型对实体进行聚类，从而构建模型图。然后，将图神经网络（GNN）应用于该模型图，以生成捕捉知识图谱全局视角的嵌入。这些嵌入随后作为原始知识图谱的高质量初始特征，替代随机初始化，从而导致更稳定和更具表现力的表示。在标准和最近提出的归纳基准上的大量实验表明，MGIL在归纳链接预测中实现了最先进或高度竞争的性能，突显了其在多种图设置中的有效性。

View on arXiv Download PDF AI Translation

cs.AI / 96 / 2606.16533

Kairos: A Native World Model Stack for Physical AI

Kairos：一种用于物理人工智能的原生世界模型栈

Kairos Team, Wang, Fei, You, Shan, Zhang, Qiming, Huang, Tao, Fu, Zuoyi, Zheng, Zhisheng, Xi, Yunlong, Lv, Feng, Wu, Xiaoming, Liu, Zeyu, Wan, Cong, Li, Pu, Yang, Ruiqing, Li, Xiaoou, Wang, Wei, Zhu, Kangkang, Zhang, Yuwei, Fu, Shi, Wu, Xiaoning, Fan, Xuzeng, Tao, Dacheng, Wang, Xiaogang

Abstract

World models are transitioning from passive visual generators to foundational, operational infrastructure for Physical AI: they must natively acquire world knowledge from heterogeneous experience, maintain persistent states over long horizons, and execute efficiently within real deployment constraints. We introduce Kairos, a native world model stack designed around these requirements. (1) Kairos learns the world by pioneering a Native Pre-training Paradigm governed by a Cross-Embodiment Data Curriculum, which organizes open-world videos, human behavioral data, and robot interactions into a progressive developmental pathway. (2) Kairos maintains the world by unified world understanding, generation, and prediction within a Native Unified Architecture equipped with Hybrid Linear Temporal Attention, where sliding-window attention captures local dynamics, dilated sliding windows capture mid-range dependencies, and gated linear attention maintains persistent global memory. We establish formal theoretical bounds demonstrating that this temporal factorization strictly limits error accumulation, mathematically guaranteeing state propagation across extended horizons. (3) Kairos runs the world by incorporating a Deployment-Aware System Co-Design to support low-latency rollout generation on server and consumer-grade hardware for real-world observation-action-feedback loops. Experiments on embodied world-model, long-horizon, and action-policy benchmarks show that Kairos achieves top level performance while offering a strong efficiency-capability trade-off. Together, these results position Kairos as a cohesive operational foundation for future self-evolving physical intelligence.

Chinese Translation

世界模型正从被动的视觉生成器转变为物理人工智能的基础性操作基础设施：它们必须从异构经验中原生获取世界知识，保持长期的持久状态，并在实际部署约束内高效执行。我们介绍了Kairos，这是一种围绕这些要求设计的原生世界模型栈。(1) Kairos通过开创一种由跨体现数据课程（Cross-Embodiment Data Curriculum）主导的原生预训练范式来学习世界，该课程将开放世界视频、人类行为数据和机器人交互组织成一个渐进的开发路径。(2) Kairos通过统一的世界理解、生成和预测来维护世界，采用原生统一架构（Native Unified Architecture），该架构配备了混合线性时间注意力（Hybrid Linear Temporal Attention），其中滑动窗口注意力捕捉局部动态，扩张滑动窗口捕捉中等范围的依赖关系，而门控线性注意力则保持持久的全局记忆。我们建立了形式化的理论界限，证明这种时间因子化严格限制了误差积累，数学上保证了状态在延长视野中的传播。(3) Kairos通过结合部署感知系统共同设计（Deployment-Aware System Co-Design）来运行世界，以支持在服务器和消费级硬件上进行低延迟的生成，以实现现实世界观察-行动-反馈循环的需求。在具身世界模型、长时间范围和行动策略基准上的实验表明，Kairos在提供强大的效率与能力权衡的同时，达到了顶级性能。这些结果共同将Kairos定位为未来自我演化物理智能的一个统一操作基础。

View on arXiv Download PDF AI Translation

cs.AI / 97 / 2606.16541

The Faithfulness Gap: Certifying Semantic Equivalence Between Natural-Language and Formal Mathematical Statements

信度差距：认证自然语言与形式数学陈述之间的语义等价性

Mohammad, Noor Islam S., Sheikh, Tamim

Abstract

Autoformalization, translating natural-language mathematics into formal proof assistants, is bottlenecked not by translation fluency but by \emph{faithfulness}: a formal statement can typecheck and be provable, yet still encode a different theorem than the source intended. We introduce \emph{Bidirectional Provability Fingerprinting} (\bpf{}), a framework that certifies faithfulness by characterizing each candidate through its forward and backward consequence neighborhoods in the ambient theory and matching these against probes derived from the natural-language statement. We further introduce four novel components: (i) \emph{Counterfactual Probe Generation} (\cpg{}), a contrastive procedure that synthesizes probes targeting specific drift directions; (ii) the \emph{Equivalence Spectrum}, a continuous faithfulness score that replaces brittle binary verdicts; (iii) \emph{Adaptive Probe Budget Allocation} (\apba{}), an information-theoretic budget router; and (iv) \emph{Faithfulness-Guided Decoding} (\fgd{}), which uses \bpf{} signals as a reward during autoformalization. We prove a \emph{drift detection theorem} and a \emph{PAC-faithfulness} result establishing that the equivalence class of a natural language statement is learnable from $\mathcal{O}(\log(1/\delta)/\varepsilon)$ probes under mild assumptions. We release \driftbench{}, a benchmark of $2{,}183$ NL/Lean~4 pairs with controlled drift labels across six subfields of mathlib4. \bpf{}\,+\,\cpg{} detects $89.6\%$ of drifted formalizations at a $3.0\%$ false-positive rate-against $41.2\%$ for typecheck and $63.3\%$ for LLM-judge baselines, and \fgd{} reduces the rate at which a state-of-the-art autoformalizer emits drifted statements by $47\%$. https://pmlrbd.github.io/BPF/

Chinese Translation

自动形式化，即将自然语言数学翻译为形式证明助手，面临的瓶颈并非翻译流畅性，而是 extit{信度}：一个形式陈述可以通过类型检查并可证明，但仍然可能编码与源意图不同的定理。我们引入 extit{双向可证明性指纹}（Bidirectional Provability Fingerprinting，pf{}），这是一个通过表征每个候选者在环境理论中的前向和后向后果邻域，并将其与从自然语言陈述派生的探针进行匹配，从而认证信度的框架。我们进一步引入四个新组件：（i） extit{反事实探针生成}（Counterfactual Probe Generation， extit{cpg}），一种合成针对特定漂移方向的探针的对比程序；（ii） extit{等价谱}，一个连续的信度评分，取代脆弱的二元裁决；（iii） extit{自适应探针预算分配}（Adaptive Probe Budget Allocation， extit{apba}），一个信息论预算路由器；（iv） extit{信度引导解码}（Faithfulness-Guided Decoding， extit{fgd}），在自动形式化过程中使用pf{}信号作为奖励。我们证明了一个 extit{漂移检测定理}和一个 extit{PAC-信度}结果，建立了自然语言陈述的等价类在温和假设下可以通过$ extmathcal{O}(rac{ ext{log}(1/ ext{δ})}{ ext{ε}})$个探针学习。我们发布了 extit{driftbench}，这是一个包含$2{,}183$个NL/Lean~4对的基准，具有跨越mathlib4六个子领域的受控漂移标签。pf{} ext{+} extit{cpg}以$3.0\%$的假阳性率检测到$89.6\\%$的漂移形式化，而类型检查和LLM-judge基线分别为$41.2\\%$和$63.3\\%$，并且 extit{fgd}将最先进的自动形式化器发出漂移陈述的比率降低了$47\\%$。

View on arXiv Download PDF AI Translation

cs.AI / 98 / 2606.16558

ROSA-RL: Uncertainty-Aware Roundabout Optimized Speed Advisory with Reinforcement Learning

ROSA-RL：基于强化学习的不确定性感知环形交叉口优化速度建议

Schlamp, Anna-Lena, Gerner, Jeremias, Bogenberger, Klaus, Huber, Werner, Schmidtner, Stefanie

Abstract

Roundabouts challenge automated driving in mixed traffic, as heterogeneous and non-deterministic human behavior, unknown driving intentions, and high interaction complexity create uncertainty about whether the conflict zone will be blocked or available at the moment of entry. We present ROSA-RL -- uncertainty-aware Roundabout Optimized Speed Advisory with Reinforcement Learning. It enables safe and efficient roundabout entry for automated and human-driven vehicles in mixed traffic through probabilistic conflict forecasting. A Transformer-based model predicts conflict zone occupancy over a five-second horizon, capturing multi-agent interactions to anticipate upcoming conflicts and available gaps. The prediction outputs encode uncertainty in future motion and intent, and augment the state of a classical RL framework, enabling uncertainty-aware speed coordination. Evaluated in simulations grounded in real-world data, ROSA-RL can effectively handle uncertainty and outperform a comparable model-based baseline, closing the gap to an ideal setting assuming fully known occupancy while improving traffic efficiency and safety. The source code of this work is available under: github.com/urbanAIthi/ROSA-RL.

Chinese Translation

环形交叉口在混合交通中对自动驾驶提出了挑战，因为异质和非确定性的人类行为、未知的驾驶意图以及高交互复杂性导致了对冲突区域在进入时是否被阻塞或可用的不确定性。我们提出了ROSA-RL——基于强化学习的不确定性感知环形交叉口优化速度建议。它通过概率冲突预测，使自动驾驶和人驾驶车辆在混合交通中安全高效地进入环形交叉口。基于Transformer的模型在五秒的时间范围内预测冲突区域的占用情况，捕捉多智能体交互，以预测即将发生的冲突和可用的间隙。预测输出编码了未来运动和意图的不确定性，并增强了经典强化学习框架的状态，实现了不确定性感知的速度协调。在基于真实世界数据的模拟中评估，ROSA-RL能够有效处理不确定性，并优于可比较的基于模型的基线，缩小了假设完全已知占用情况下理想设置的差距，同时提高了交通效率和安全性。本研究的源代码可在以下地址获取：github.com/urbanAIthi/ROSA-RL。

View on arXiv Download PDF AI Translation

cs.AI / 99 / 2606.16567

TNODEV: Toolbox for Neural ODE Verification

TNODEV：神经常微分方程验证工具箱

Sayed, Abdelrahman Sayed, Meyer, Pierre-Jean, Ghazel, Mohamed

Abstract

Neural ordinary differential equations (neural ODE) have started to appear in safety critical settings such as continuous-time controllers for cyber-physical systems and classifiers integrated into automated decision pipelines, raising the question of whether their behavior can be formally verified. Existing tools dedicated to neural ODE provide only a single reachability call without iterative input set refinement, limiting the precision of their verdicts to whatever one reachability call can deliver. We present TNODEV, the first sound formal verifier for neural ODE that integrates a falsification checker, a fast interval-based reachability backend based on continuous-time mixed monotonicity, a verification and refinement loop with three input-set splitting heuristics, and a parallel scheduler in a single end-to-end pipeline. TNODEV supports safe-set inclusion verification on pure neural ODE, neural ODE in closed loop with a neural network controller and general neural ODE (GNODE), with the safe set specified either as an interval or as the half-space intersection induced by a target classification label. We evaluate TNODEV on a range of benchmarks across safe-set inclusion and classification-robustness properties, including a direct reachability comparison against NNV~2.0 and CORA and a verification comparison against NNV2.0 on MNIST general neural ODE classifiers.

Chinese Translation

神经常微分方程（neural ODE）已开始出现在安全关键的环境中，例如用于网络物理系统的连续时间控制器和集成于自动决策流程中的分类器，这引发了其行为是否可以被正式验证的问题。现有专门针对神经 ODE 的工具仅提供单次可达性调用，而没有迭代输入集的细化，这限制了其判决的精确度，仅能依赖一次可达性调用所能提供的结果。我们提出了 TNODEV，这是第一个针对神经 ODE 的可靠形式验证器，它集成了一个伪造检查器、一个基于连续时间混合单调性的快速区间可达性后端、一个包含三种输入集拆分启发式的验证和细化循环，以及一个在单一端到端管道中的并行调度器。TNODEV 支持对纯神经 ODE、与神经网络控制器闭环的神经 ODE 以及一般神经 ODE（GNODE）的安全集包含验证，安全集可以指定为区间或由目标分类标签诱导的半空间交集。我们在一系列基准测试中评估了 TNODEV，涵盖了安全集包含和分类鲁棒性属性，包括与 NNV~2.0 和 CORA 的直接可达性比较，以及在 MNIST 一般神经 ODE 分类器上与 NNV2.0 的验证比较。

View on arXiv Download PDF AI Translation

cs.AI / 100 / 2606.16605

ARB4WM: An Adversarial Robustness Benchmark for World Models in Continuous Control

ARB4WM：用于连续控制中世界模型的对抗鲁棒性基准

Zhang, Junjian, Tan, Hao, Li, Ruonan, Zhu, Dong, Li, Aiping, Gu, Zhaoquan

Abstract

World models are widely used in robotic and agentic engineering control systems due to their ability to learn latent dynamics for planning and decision-making. As these systems are increasingly deployed in safety-critical settings, understanding their robustness under adversarial conditions has become essential. However, existing evaluations lack a unified benchmark for testing adversarial threats across the policy, value, and latent-dynamics levels of world-model agents. To fill this gap, we present ARB4WM, a unified evaluation framework for pre-deployment robustness and risk assessment of world-model agents under visual perturbations. ARB4WM defines five white-box loss objectives across these three levels and studies their effects when combined with single-step or multi-step perturbation strategies and temporal attack modes, including full-frame, half-sequence, and sparse-frame exposure. Specifically, we evaluate four Dreamer-style agents across 20 tasks from MetaWorld and the DeepMind Control Suite under different loss objectives, perturbation strategies, and temporal attack modes. Results show that attacks targeting value estimation, latent representations, and RSSM dynamics can be as damaging as direct policy disruption, and that early or frequent perturbations are especially harmful, while input-level defenses provide limited recovery under adaptive attacks. These findings suggest that safety, risk, and reliability assessment for world models should cover multiple component-oriented attack objectives and temporal exposure protocols rather than relying solely on action-space robustness. Source code is available at https://github.com/zaoanguai/ARB4WM.

Chinese Translation

世界模型因其学习潜在动态以进行规划和决策的能力而广泛应用于机器人和智能工程控制系统。随着这些系统在安全关键环境中的日益部署，理解它们在对抗条件下的鲁棒性变得至关重要。然而，现有评估缺乏一个统一的基准，用于测试世界模型代理在策略、价值和潜在动态层面的对抗威胁。为填补这一空白，我们提出了ARB4WM，一个统一的评估框架，用于在视觉扰动下对世界模型代理进行预部署鲁棒性和风险评估。ARB4WM在这三个层面上定义了五个白盒损失目标，并研究了它们在与单步或多步扰动策略及时间攻击模式（包括全帧、半序列和稀疏帧暴露）结合时的效果。具体而言，我们在不同的损失目标、扰动策略和时间攻击模式下，对来自MetaWorld和DeepMind Control Suite的20个任务中的四个Dreamer风格代理进行了评估。结果表明，针对价值估计、潜在表示和RSSM动态的攻击可能与直接的策略干扰一样具有破坏性，并且早期或频繁的扰动尤其有害，而输入级防御在自适应攻击下提供的恢复有限。这些发现表明，世界模型的安全性、风险和可靠性评估应涵盖多个面向组件的攻击目标和时间暴露协议，而不仅仅依赖于动作空间的鲁棒性。源代码可在 https://github.com/zaoanguai/ARB4WM 获取。

View on arXiv Download PDF AI Translation

cs.AI / 101 / 2606.16613

CoffeeBench: Benchmarking Long-Horizon LLM Agents in Heterogeneous Multi-Agent Economies

CoffeeBench：在异构多智能体经济中评估长时间跨度的LLM代理的基准测试

Sugiura, Issa, Hattori, Daichi, Araragi, Kazuo, Ogawa, Keita, Onose, Shota, Makino, Taro, Usuki, Teppei, Ishida, Takashi

Abstract

As LLM agents become capable of increasingly long-horizon tasks, evaluating their performance in economic systems is becoming increasingly important. Unlike existing benchmarks that primarily evaluate a single agent interacting with a passive environment, economic systems are inherently multi-agent, requiring autonomous agents to communicate, negotiate, and transact while pursuing their own objectives over extended periods. We introduce CoffeeBench, a benchmark for evaluating LLM agents in a long-horizon multi-agent economy composed of heterogeneous firms. In CoffeeBench, two farmers, two roasters, and two retailers autonomously operate their businesses over a 90-day simulation, each seeking to maximize cumulative net income through communication and transactions while managing cash, inventory, and pricing. The evaluated model controls one coffee roaster, while the remaining firms are controlled by fixed reference agents. Across several recent open-weight and proprietary LLMs, all models outperform a passive baseline that takes no actions, with most achieving positive net income. Analysis of agent behavior reveals substantial differences in long-horizon economic interaction: higher-performing models communicate more actively with other firms, whereas Claude~Haiku~4.5 exhibits an idle-drift failure mode, repeatedly choosing inaction despite producing coherent assessments and plans. We release our code and agent trajectories to support future research.

Chinese Translation

随着LLM代理能够处理越来越长时间跨度的任务，在经济系统中评估其性能变得愈加重要。与主要评估单一代理与被动环境互动的现有基准不同，经济系统本质上是多智能体的，要求自主代理在追求自身目标的同时进行沟通、谈判和交易，并持续一段较长的时间。我们介绍了CoffeeBench，这是一个用于评估异构企业组成的长时间跨度多智能体经济中LLM代理的基准。在CoffeeBench中，两位农民、两位烘焙师和两位零售商在90天的模拟中自主运营各自的业务，每个代理都寻求通过沟通和交易最大化累积净收入，同时管理现金、库存和定价。被评估的模型控制一位咖啡烘焙师，而其余企业则由固定的参考代理控制。在多个最近的开放权重和专有LLM中，所有模型的表现均优于不采取任何行动的被动基线，大多数模型实现了正的净收入。对代理行为的分析揭示了长时间跨度经济互动中的显著差异：表现更好的模型与其他企业的沟通更为积极，而Claude Haiku 4.5则表现出闲置漂移的失败模式，尽管产生了一致的评估和计划，却反复选择不采取行动。我们发布了我们的代码和代理轨迹，以支持未来的研究。

View on arXiv Download PDF AI Translation

cs.AI / 102 / 2606.16624

MR-GVNO: A Geometry-Aware Variational Physics-Informed Neural Operator for Mindlin-Reissner Plates on Irregular Domains

MR-GVNO：一种针对不规则域的几何感知变分物理信息神经算子用于Mindlin-Reissner板

Wang, Siqi, Sun, Daobo, Wang, Yizheng, Zhang, Yilong, Jin, Yabin, Zhuang, Xiaoying, Rabczuk, Timon

Abstract

Plate and shell structures are widely used in engineering, making rapid response prediction under varying geometries, materials, and loads highly desirable. However, conventional finite element methods require repeated modeling and solution, resulting in high computational costs. This study proposes a geometry-aware variational neural operator for Mindlin-Reissner plate problems, termed MR-GVNO. The method uses boundary point clouds to represent irregular geometries and employs separate encoders for spatially varying material fields, pressure loads, and scalar physical parameters. A cross-attention mechanism integrates these inputs with query point information to predict transverse deflections and rotations at arbitrary locations. MR-GVNO is trained without labeled solution data using a variational physics-informed loss derived from the discretized total potential energy. It directly processes irregular point clouds and allows different physical fields to be discretized independently, avoiding interpolation onto a common grid. Numerical experiments on single-hole, double-hole, and L-shaped plates demonstrate accurate response prediction under homogeneous and heterogeneous materials and uniform and random loads. The model also achieves millisecond-level full-field inference and favorable cross-geometry generalization.

Chinese Translation

板壳结构在工程中被广泛应用，因此在不同几何形状、材料和载荷下快速响应预测是非常重要的。然而，传统的有限元方法需要反复建模和求解，导致高计算成本。本研究提出了一种针对Mindlin-Reissner板问题的几何感知变分神经算子，称为MR-GVNO。该方法使用边界点云表示不规则几何形状，并为空间变化的材料场、压力载荷和标量物理参数采用独立的编码器。交叉注意机制将这些输入与查询点信息结合，以预测任意位置的横向挠度和旋转。MR-GVNO在没有标记解数据的情况下进行训练，使用从离散化的总势能导出的变分物理信息损失。它直接处理不规则点云，并允许不同的物理场独立离散，避免了插值到共同网格的需求。在单孔、双孔和L形板上的数值实验表明，在均匀和非均匀材料以及均匀和随机载荷下，模型能够准确预测响应。该模型还实现了毫秒级的全场推断和良好的跨几何泛化能力。

View on arXiv Download PDF AI Translation

cs.AI / 103 / 2606.16649

The Integrator Advantage: Controlled Agentic AI for Small and Medium-Sized Companies

整合者优势：面向中小型企业的受控自主人工智能

Koch, Christopner, Wellbrock, Joshua A.

Abstract

Agentic AI marks a new phase of enterprise automation. Unlike traditional automation or conversational AI, agentic systems can interpret goals, plan multi step tasks, access tools, interact with enterprise systems, and execute workflows with varying degrees of autonomy. For small and medium sized companies, this creates potential to reduce administrative burden, accelerate routine processes, and improve the use of organizational knowledge. This paper argues that the near term value of Agentic AI does not lie in full autonomy or workforce reduction, but in controlled partial autonomy for simple and medium complexity business processes. It proposes an integration framework covering use case suitability, autonomy levels, technical integration, governance, security, employee enablement, and measurable impact. The paper concludes that Agentic AI can become a productivity lever when implemented as a human centered capability with responsibility and accountability retained by people.

Chinese Translation

自主人工智能标志着企业自动化的新阶段。与传统的自动化或对话式人工智能不同，自主系统能够理解目标、规划多步骤任务、访问工具、与企业系统互动，并以不同程度的自主性执行工作流程。对于中小型企业而言，这为减少行政负担、加速日常流程和改善组织知识的利用创造了潜力。本文认为，自主人工智能的近期价值不在于完全自主或减少劳动力，而在于对简单和中等复杂度的业务流程进行受控的部分自主性。本文提出了一个整合框架，涵盖了使用案例适用性、自主性水平、技术整合、治理、安全性、员工赋能和可衡量的影响。文章最后得出结论，自主人工智能可以成为生产力的杠杆，当其作为以人为本的能力实施时，责任和问责仍由人类保留。

View on arXiv Download PDF AI Translation

cs.AI / 104 / 2606.16687

From Affect Prediction to Affect Forecasting: Evidence for Distinct Information Sources in Longitudinal Text

从情感预测到情感预测：纵向文本中不同信息来源的证据

Noor, Sadia, Latif, Seemab, Shahzad, Raja Khurram, Fatima, Mehwish

Abstract

Modeling dimensional affect in longitudinal text requires distinguishing current affect estimation from future affective change forecasting. Existing approaches often treat each text as an independent observation and apply similar assumptions to both tasks, without testing whether they rely on different information sources. This paper investigates that distinction using longitudinal self-reported ecological essays and feeling-word entries. We propose the Trait--State Affective Prediction (TSAP) framework and its temporal extension E-TSAP for per-text valence and arousal prediction, evaluated on a held-out prediction test set of 1,737 entries from 91 users. We further propose the Affective Change Forecaster Hybrid (ACF-Hybrid) for next-step affective change forecasting, evaluated on a held-out forecasting test set of 46 users. For prediction, E-TSAP achieves composite Pearson correlations of 0.670 for valence and 0.449 for arousal. For forecasting, textual representations perform worse than compact numeric trajectory baselines: the text-inclusive model achieves only r=0.316 for valence and r=0.284 for arousal, whereas a simple prior-state baseline reaches r=0.615 and r=0.670, respectively. ACF-Hybrid, using dimension-specific numeric trajectory features, achieves r=0.659 for valence and $r=0.658$ for arousal. These results show that textual semantics support current affect prediction, whereas future affective change is better captured through prior numeric trajectory dynamics.

Chinese Translation

对纵向文本中的维度情感进行建模，需要区分当前情感估计与未来情感变化预测这两种任务。现有方法通常将每一段文本视为独立观测，并对两类任务采用相似假设，而未验证它们是否依赖不同的信息来源。本文利用纵向自我报告的生态随笔与情感词条数据，对这一差异进行了研究。我们提出特质—状态情感预测（Trait--State Affective Prediction, TSAP）框架及其时间扩展版本 E-TSAP，用于逐文本效价（valence）与唤醒度（arousal）预测，并在包含91名用户的1,737条留出测试数据集上进行评估。进一步地，我们提出情感变化预测混合模型（Affective Change Forecaster Hybrid, ACF-Hybrid），用于下一步情感变化预测，并在46名用户的留出预测测试集上进行评估。在预测任务中，E-TSAP在效价上的综合皮尔逊相关系数达到0.670，在唤醒度上达到0.449。在预测任务中，相比于紧凑的数值轨迹基线方法，文本表示效果更差：包含文本信息的模型在效价上的相关系数仅为r=0.316，在唤醒度上为r=0.284，而一个简单的先验状态基线分别达到r=0.615与r=0.670。ACF-Hybrid利用维度特定的数值轨迹特征，在效价上达到r=0.659，在唤醒度上达到r=0.658。这些结果表明，文本语义更有助于当前情感预测，而未来情感变化则更适合通过先前的数值轨迹动态来捕捉。

View on arXiv Download PDF AI Translation

cs.AI / 105 / 2606.16707

User as Code: Executable Memory for Personalized Agents

用户即代码：用于个性化智能体的可执行记忆

Li, Bojie

Abstract

A personalized AI agent needs a user memory: a persistent model of who the user is, built across many conversations and consulted on each new one. Today this memory is almost always stored as unstructured text, a knowledge graph, or a flat store of facts, and consulted by retrieval -- fetching the entries most similar to the current request. Such "bag-of-facts" memory recalls individual facts well, but because storing a fact and acting on it are separate steps, it struggles to resolve contradictions, aggregate over many records, or enforce rules. We argue that user memory should instead be executable. We introduce User as Code (UaC), a paradigm in which an agent's model of a user is a living software project: typed Python objects hold the user's state and ordinary Python functions encode the rules that govern it, so representing and reasoning about the user happen in one medium an interpreter can run. The enabling mechanism is a two-phase pipeline: an append-only log that never discards a fact, periodically checkpointed into typed code. This changes what memory can do. On standard long-term conversation benchmarks, UaC matches both a full-context upper bound and the strongest prior memory systems on recall (78.8% on LOCOMO). Its advantage emerges where representation matters most. On aggregate questions over a user's history -- "how many international trips did I take last year?" -- retrieval-based memory collapses (6-43%) while UaC stays near-perfect (99%), because the answer is a one-line computation over typed state rather than a search over text. And because its rules execute deterministically whenever the state changes, UaC can surface unsolicited, safety-critical alerts -- such as a newly prescribed drug that conflicts with an allergy recorded months earlier -- a capability query-driven memory cannot provide.

Chinese Translation

个性化AI智能体需要用户记忆（user memory）：即对用户身份与特征的持久化建模，该模型在多轮对话中逐步构建，并在每一次新对话中被调用。目前，这种记忆通常以非结构化文本、知识图谱或扁平事实存储的形式保存，并通过检索（retrieval）机制使用——即获取与当前请求最相似的条目。这种“事实袋（bag-of-facts）”式记忆能够较好地召回单条事实，但由于事实存储与基于事实的行为执行是分离的，它在处理矛盾消解、跨多条记录的聚合计算以及规则约束执行方面存在困难。我们认为，用户记忆应当是可执行的（executable）。我们提出 User as Code（UaC）范式，在该范式中，智能体对用户的建模是一个“活的”软件工程项目：使用带类型标注的 Python 对象表示用户状态，普通 Python 函数用于编码约束与规则，使得对用户的表示与推理统一在同一可由解释器执行的媒介中完成。其实现机制是一个两阶段流水线：首先是一个仅追加（append-only）的日志系统，永不丢弃任何事实；随后定期将其检查点（checkpoint）转换为带类型标注的代码表示。这一设计改变了记忆系统的能力边界。在标准长期对话基准测试中，UaC 在召回性能上达到了与完整上下文上界以及当前最强既有记忆系统相当的水平（LOCOMO 上为 78.8%）。其优势在表示方式至关重要的任务中尤为明显。在针对用户历史的聚合型问题中——例如“我去年进行了多少次国际旅行？”——基于检索的记忆系统表现显著下降（6–43%），而 UaC 几乎保持完美（99%），因为答案可以通过对结构化类型状态的一行计算直接得到，而不是在文本中进行语义搜索。并且由于其规则在状态发生变化时能够以确定性方式自动执行，UaC 能够主动触发未经查询的安全关键提示——例如新开具的药物与数月前记录的过敏信息发生冲突——而这种能力是基于查询驱动的记忆系统无法提供的。

View on arXiv Download PDF AI Translation

cs.AI / 106 / 2606.16721

Medical world models: representing medical states, modelling clinical dynamics and guiding intervention policies

医学世界模型：医学状态表征、临床动态建模与干预策略引导

Liu, Ke, Li, Mengxuan, Bao, Yanyi, Zhang, Tianyun, Chu, Chong, Bu, Jiajun, Wang, Haishuai

Abstract

Medical diagnosis and treatment are dynamic processes in which patient states evolve over time and clinical interventions alter future outcomes. Although current medical AI can detect disease, estimate risk and generate reports, many systems still return static labels or scores, offering limited insight into how illness may progress or how alternative interventions may reshape its trajectory. Medical world models adapt the world-model idea from artificial intelligence to healthcare by learning internal simulators of patient-state dynamics. Their long-term goal is to help clinicians anticipate deterioration, compare treatment-conditioned futures and tailor care to individual patients. Yet relevant work remains scattered across foundation models, longitudinal modelling, disease simulation, treatment-effect estimation, reinforcement learning and digital twins. To bridge this gap, this review outlines a roadmap for advancing medical AI from isolated diagnosis and prediction toward medical world models that simulate disease evolution and support intervention decisions. This roadmap is organized around three coupled capabilities: patient-state construction, clinical dynamics modelling and intervention decision support. Across representative systems, the comparison highlights what each capability contributes and how partial components can be integrated into more mature perception--dynamics--planning systems. Finally, we identify the challenges involved in turning plausible rollouts into clinically useful simulators. Related literature is available at https://github.com/1999kevin/awesome_medical_world_models.

Chinese Translation

医学诊断与治疗是一个动态过程，其中患者状态会随时间演化，而临床干预又会改变其未来结局。尽管当前医学人工智能已能够实现疾病检测、风险评估与报告生成，但许多系统仍主要返回静态标签或评分，难以提供关于疾病如何进展或不同干预措施如何改变其演化轨迹的深入理解。医学世界模型将人工智能中的世界模型思想引入医疗领域，通过学习患者状态动态变化的内部模拟器来建模疾病发展过程。其长期目标是帮助临床医生预测病情恶化趋势、比较不同治疗方案下的潜在未来，并为个体患者制定个性化治疗策略。然而，目前相关研究仍分散于基础模型、纵向建模、疾病仿真、治疗效应估计、强化学习以及数字孪生等多个方向。为弥补这一空白，本文综述提出了一条从孤立的诊断与预测走向医学世界模型的发展路线图，使医学人工智能能够模拟疾病演化并支持干预决策。该路线图围绕三项核心能力展开：患者状态构建、临床动态建模以及干预决策支持。在对代表性系统的比较中，本文强调了每项能力的贡献，以及如何将其部分模块整合为更成熟的感知—动力学—规划系统。最后，本文指出了将合理的预测轨迹转化为具有临床实用价值的模拟器所面临的关键挑战。相关文献见：https://github.com/1999kevin/awesome_medical_world_models

View on arXiv Download PDF AI Translation

cs.AI / 107 / 2606.16723

AgentFairBench: Do LLM Agents Discriminate When They Act?

AgentFairBench：LLM智能体在执行行动时是否存在歧视？

Morla, Triveni, Bellibaltu, Rohith Reddy, Singh, Manpreet, Kapoor, Manmeet Singh

Abstract

Large language model (LLM) agents increasingly take actions (screening applicants, recommending credit, triaging patients), yet fairness for LLMs is still measured by grading answers. We introduce AgentFairBench, a cheap, reproducible, multi-domain benchmark for demographic disparity in the actions of LLM agents. Grounded in a companion framework, the Bias Conduction Framework (BCF, restated here), it spans three regulator-anchored domains: hiring, lending, and medical triage. Synthetic, demographic-neutral profiles are evaluated in counterfactual matched sets that vary only a name-coded race x gender signal (in the Bertrand Mullainathan tradition), under four agent scaffolds of increasing agency (direct, chain-of-thought, multi-agent deliberation, tool-augmented). A NumPy-only harness computes counterfactual flip rate, mean absolute score difference (MASD), action-rate disparity, and tool-invocation disparity, with bootstrap confidence intervals, paired tests, and false-discovery-rate control, for single-digit dollars per model. A live leaderboard with a held-out private split and a contamination canary admits external models by submission. Our pilot (864 decisions plus a test-retest replication) carries a methodological lesson: comparing a six-group score spread against a two-run noise difference overstates disparity by ~ 2.4X through statistic arity alone. Against an arity matched noise floor and an omnibus group test, claude haiku 4 5 shows no demographic effect above sampling noise (0 of 120 pairwise and 0 of 9 omnibus contrasts survive correction); a planted-bias test confirms the instrument detects disparity when present. The contribution is a sound, sensitive, adoption-ready instrument, the arity matched null methodology, and open artifacts to scale it. Code, data, and harness are released under open licenses, with an anonymized review artifact.

Chinese Translation

大型语言模型（LLM）智能体正在越来越多地执行实际行动（如筛选求职者、推荐信贷、患者分诊等），然而针对LLM的公平性评估仍主要停留在对答案进行评分的层面。我们提出AgentFairBench，一个低成本、可复现、跨领域的基准测试，用于衡量LLM智能体在执行行为时是否存在群体人口差异。该基准建立在配套框架Bias Conduction Framework（BCF，本文中重述）之上，覆盖三个具有监管约束的应用领域：招聘、贷款与医疗分诊。我们构建了合成的、人口统计学中性的个体档案，并在反事实匹配集合中进行评估；这些集合仅在以姓名编码的种族×性别信号上存在差异（延续Bertrand–Mullainathan研究范式）。在四种逐步增强智能体自主性的架构下进行实验，包括直接决策、思维链（chain-of-thought）、多智能体协同讨论以及工具增强型智能体。我们实现了一个仅依赖NumPy的评估框架，用于计算反事实翻转率、平均绝对评分差异（MASD）、行动率差异以及工具调用差异，并提供bootstrap置信区间、配对检验以及错误发现率控制。单个模型的评估成本为个位数美元级别。系统还包含一个实时排行榜，采用保留的私有测试集与污染检测“canary”机制，以支持外部模型提交与验证。我们的试点实验（864次决策及一次重复性测试）揭示了一项方法学结论：将六组评分差异与两次运行的噪声差异进行比较，会仅因统计维度（arity）不同而系统性高估约2.4倍的群体差异。在采用维度匹配的噪声基线与整体检验后，Claude Haiku 4.5在所有评估中均未表现出超出采样噪声范围的人口统计学效应（120个两两比较与9个整体检验均在多重校正后不显著）。植入偏差实验进一步验证了该评估工具在存在真实偏差时能够有效检测。本研究的贡献在于提出了一种稳定、敏感且可直接应用的评估工具，提出了维度匹配的零假设方法，并提供可扩展的开源资源。代码、数据与评估框架均已在开源许可下发布，并附带匿名化审稿材料。

View on arXiv Download PDF AI Translation

cs.AI / 108 / 2606.16733

A First-Principles Derivation of LLM Policy Optimization: From Expected Reward to GRPO and Its Structural Extensions

基于第一性原理的LLM策略优化推导：从期望奖励到GRPO及其结构性扩展

Shen, Jianghan, Luo, Siqi, Li, Yue, Liu, Jiyao, Qu, Wanying, Zhang, Yi, Huang, Ziyan, Li, Tianbin, Hu, Ming, Liu, Xiaohong, Chen, Yirong, He, Junjun

Abstract

Policy gradient algorithms for language models optimize the same objective $J(\theta) = \mathbb{E}*{\tau \sim p*\theta(\tau)}[R(\tau)]$, which has exactly two factors: the trajectory probability $p_\theta(\tau)$ and the reward $R(\tau)$. Every method from REINFORCE to PPO to GRPO and their descendants modifies one or both factors to address a specific failure in the preceding formulation. Existing surveys organize these methods by domain or chronology, which obscures the rationale behind each design choice and the precise location of its intervention within the gradient estimator. This survey revisits the landscape of LLM policy optimization from $J(\theta)$ on first principles and uses the trajectory side, induced by $p_\theta(\tau)$, and the reward side, induced by $R(\tau)$, as the two axes along which methods are located. It covers the path from REINFORCE and PPO to GRPO, as well as post-GRPO variants, Agentic RL, and GRPO-OPD. The resulting framework is unified, diagnostic, and extensible: it analyzes methods from a shared objective, identifies which side each method modifies and why, and applies the same trajectory and reward axes across these settings. Across these settings, the framework also exposes compound failures that no single-side fix resolves and that therefore require joint design of the trajectory side and the reward side. The boundary cases and coupled failures identified by this map mark where existing solutions run out and provide a principled starting point for designing the next generation of LLM policy optimization algorithms.

Chinese Translation

所有用于语言模型的策略梯度算法都在优化同一个目标函数 $J(\theta)=\mathbb{E}_{\tau \sim p_\theta(\tau)}[R(\tau)]$，该目标函数仅由两个核心因素构成：轨迹概率 $p_\theta(\tau)$ 与奖励 $R(\tau)$。从 REINFORCE 到 PPO，再到 GRPO 及其后续方法，每一种算法本质上都是通过修改其中一个或两个因素，以修复前一类公式在建模或优化过程中暴露出的特定缺陷。现有综述通常按照领域划分或时间顺序组织这些方法，这种方式在一定程度上掩盖了各设计决策背后的动机，以及其在梯度估计器中的精确作用位置。本综述从第一性原理重新审视 LLM 策略优化的整体图景，从目标函数 $J(\theta)$ 出发，并以轨迹侧（由 $p_\theta(\tau)$ 决定）与奖励侧（由 $R(\tau)$ 决定）作为两条分析主轴，对各类方法进行定位与归纳。该框架覆盖了从 REINFORCE 和 PPO 到 GRPO 的演进路径，以及 GRPO 之后的变体、Agentic RL 和 GRPO-OPD 等方法。所提出的统一框架具有诊断性与可扩展性：它基于共享的目标函数分析各类方法，明确每种方法究竟修改了哪一侧以及其原因，并在不同设置中沿用相同的轨迹—奖励双轴结构进行比较分析。在这些分析中，该框架进一步揭示了多种复合失效模式，即仅依赖单侧修复无法解决的问题，这些问题需要对轨迹侧与奖励侧进行联合设计。该映射所识别出的边界情况与耦合失效，正是现有方法难以覆盖的区域，同时也为下一代 LLM 策略优化算法的设计提供了一个更为系统和原则性的起点。

View on arXiv Download PDF AI Translation

cs.AI / 109 / 2606.16769

Skill-to-LoRA: From Using Skills to Learning Behaviors for Token-Efficient LLM Agents

Skill-to-LoRA：从使用技能到学习行为的token高效大语言模型智能体方法

Zhang, Tianyi, Qi, Zhonghao

Abstract

Agent skills are commonly distributed as SKILL.md files: human-readable procedural documents that describe workflows, tools, resources, and domain conventions. While convenient for inspection and reuse, this design requires the same reusable procedure to be repeatedly injected into the runtime context. We propose Skill-to-LoRA(S2L), a behavior-centric skill representation that replaces runtime skill text with skill-specific LoRA adapters. Rather than compressing the skill document itself, S2L models the behavioral change induced by the skill text: offline, the complete SKILL.md is used to synthesize skill-guided demonstrations; online, the full document is omitted and the corresponding LoRA adapter is dynamically loaded to activate the learned skill behavior. We evaluate S2L with Qwen3.6-27B on a 21-skill subset of SWE-Skills-Bench. Compared with the no-skill and Full Skill Text baselines, S2L improves pass rate by 2.9 and 5.2 percentage points, respectively, while reducing per-step token cost by 6.6% relative to Full Skill Text prompting. S2L matches or improves Full Skill Text on 18/21 skills and the no-skill baseline on 15/21 skills. Control experiments further show that the gains depend on skill-specific adapter alignment: Wrong-LoRA and Shared-LoRA both reduce performance. These results suggest that many procedural agent skills can be converted from runtime instructions into trainable, dynamically loadable behavioral modules. Code will be released upon acceptance.

Chinese Translation

智能体技能通常以SKILL.md文件的形式分发：这类文件是人类可读的过程性文档，用于描述工作流程、工具、资源以及领域规范。尽管这种设计便于检查与复用，但它要求同一可复用过程在运行时上下文中被反复注入。为此，我们提出Skill-to-LoRA（S2L），一种以行为为中心的技能表示方法，用于用特定于技能的LoRA（Low-Rank Adaptation）适配器替代运行时的技能文本。S2L并不直接压缩技能文档本身，而是对技能文本所诱导的行为变化进行建模：在离线阶段，使用完整的SKILL.md生成技能引导的示例；在在线阶段，则省略完整文档，并动态加载对应的LoRA适配器以激活已学习的技能行为。我们在SWE-Skills-Bench的21个技能子集上，基于Qwen3.6-27B对S2L进行评估。与无技能（no-skill）和完整技能文本（Full Skill Text）基线相比，S2L分别将通过率提升2.9和5.2个百分点，同时相较于Full Skill Text提示方式，每步token成本降低6.6%。S2L在21个技能中有18个优于或持平Full Skill Text，在15个技能中优于或持平无技能基线。对照实验进一步表明，这种性能提升依赖于特定技能的适配器对齐：使用错误LoRA（Wrong-LoRA）和共享LoRA（Shared-LoRA）都会导致性能下降。这些结果表明，许多过程型智能体技能可以从运行时指令形式转化为可训练、可动态加载的行为模块。代码将在论文接收后开源。

View on arXiv Download PDF AI Translation

cs.AI / 110 / 2606.16774

OpenClaw-Skill: Collective Skill Tree Search for Agentic Large Language Models

OpenClaw-Skill：面向智能体大语言模型的集体技能树搜索

Lin, Tianyi, Sun, Chuanyu, Zhang, Jingyi, Wei, Changxu, Yao, Huanjin, Liu, Shunyu, Zhang, Xikun, Liu, Liu, Huang, Jiaxing

Abstract

Equipping Large Language Model (LLM) agents with effective skills is crucial for solving complex tasks in real-world systems like OpenClaw. In this work, we aim to develop a framework that automatically constructs such reusable skills to enhance LLMs in tool use, multi-step reasoning, and dynamic environment interaction. To this end, we propose Collective Skill Tree Search (CSTS), a novel tree-search-based skill construction framework that constructs structured, diverse and generalizable tree of skills. The core idea of CSTS is to leverage collective intelligence to jointly search, identify and compose effective skills via two iterative phases: Collective Skill Node Generation (CSN-Gen) and Collective Skill Node Assessment (CSN-Assess). CSN-Gen exploits collective knowledge from multiple models to explore diverse candidate skills for each subtask, enabling comprehensive skill exploration. CSN-Assess employs multiple models as judges to evaluate and select skill nodes with two scoring mechanisms: (1) collective quality scoring that aggregates independent evaluations to produce a robust estimate of skill effectiveness, and (2) collective transferability scoring that explicitly verifies whether a skill generalizes well across different models. With CSTS, we construct a set of comprehensive tree of skills along with skill-augmented training data, enabling models to effectively learn and utilize skills. Besides, we introduce Collective Skill Reinforcement Learning, which actively selects multiple relevant skills from the tree to broaden solution-space exploration, avoid being trapped by a single skill and its resulting homogeneous or suboptimal solutions. As a result, our trained model, OpenClaw-Skill, exhibits outstanding agentic capabilities in long-horizon planning, tool use and generalization over challenging benchmarks.

Chinese Translation

为大语言模型（Large Language Model, LLM）智能体配备有效技能对于解决OpenClaw等真实系统中的复杂任务至关重要。本文旨在提出一种能够自动构建可复用技能的框架，以增强LLM在工具使用、多步推理以及动态环境交互方面的能力。为此，我们提出集体技能树搜索（Collective Skill Tree Search, CSTS），这是一种基于树搜索的技能构建新框架，用于构建结构化、多样化且具有良好泛化能力的技能树。CSTS的核心思想是利用集体智能，通过两个迭代阶段共同搜索、识别并组合有效技能：集体技能节点生成（Collective Skill Node Generation, CSN-Gen）与集体技能节点评估（Collective Skill Node Assessment, CSN-Assess）。CSN-Gen利用多个模型的集体知识，为每个子任务探索多样化的候选技能，从而实现更全面的技能探索。CSN-Assess则使用多个模型作为评审者对技能节点进行评估与筛选，并采用两种评分机制：（1）集体质量评分，通过汇总独立评估结果生成对技能有效性的稳健估计；（2）集体迁移性评分，显式验证某一技能是否能够在不同模型之间良好泛化。基于CSTS，我们构建了一组完整的技能树以及技能增强训练数据，使模型能够更有效地学习并使用技能。此外，我们还提出集体技能强化学习（Collective Skill Reinforcement Learning），通过从技能树中主动选择多个相关技能来拓展解空间探索，避免陷入单一技能导致的同质化或次优解。最终，所训练的模型OpenClaw-Skill在长程规划、工具使用以及复杂基准任务的泛化能力方面展现出卓越的智能体能力。

View on arXiv Download PDF AI Translation

cs.AI / 111 / 2606.16802

LabOSBench: Benchmarking Computer Use Agents for Scientific Instrument Control

LabOSBench：面向科学仪器控制的计算机使用智能体基准评测

Zou, Anqi, Deng, Han, Zhang, Chengyu, Hu, Junquan, Wang, Yu, Xing, Yuxiang, Zhang, Aokai, Zhang, Hanling, Liu, Zhaoyang, Fei, Ben, Wang, Zhihui, Ouyang, Wanli

Abstract

Current computer-use benchmarks primarily focus on software operation tasks in virtualized systems, whereas scientific instrumentation scenarios require coordinated control over complex interfaces, and feedback-driven parameter adjustment. However, directly evaluating agents on physical high-precision instruments is impractical due to high cost, safety risks, limited accessibility, and difficulty in ensuring reproducible evaluation. This motivates the need for a simulated yet realistic testbed that preserves the operational challenges of scientific instruments while enabling scalable and safe benchmarking. To this end, we introduce LabOSBench, a challenging benchmark for multimodal GUI agents built on a suite of web-based scientific-instrument simulators. Operating directly via a browser, LabOSBench avoids resource-heavy OS virtualization while supporting flexible task configuration and execution-based evaluation. Specifically, LabOSBench constructs 96 subtasks across eight instrument simulators, covering workflows from sample loading, alignment, parameter tuning, and data acquisition to result inspection. We evaluate general-purpose vision-language models, specialized GUI agent models, and advanced agentic frameworks at both subtask and end-to-end levels. Our experiments reveal that while existing agents can complete many structured GUI subtasks, they still struggle with feedback-driven operations and long-horizon workflow execution. Overall, LabOSBench provides a reproducible, low-cost testbed for advancing computer-using agents toward scientific-instrument control.

Chinese Translation

当前的计算机使用基准主要聚焦于虚拟化系统中的软件操作任务，而科学仪器应用场景则需要对复杂界面进行协同控制，并进行基于反馈的参数调节。然而，直接在物理高精度仪器上对智能体进行评估在实际中并不可行，原因在于成本高昂、安全风险较大、可访问性有限，以及难以保证评估的可重复性。这一问题促使我们需要构建一个既具备仿真性质又具有真实感的测试平台，在保留科学仪器操作复杂性的同时，实现可扩展且安全的基准测试。为此，我们提出 LabOSBench，这是一个面向多模态GUI智能体的挑战性基准，其基于一组基于网页的科学仪器模拟器构建。LabOSBench可直接通过浏览器运行，避免了资源密集型的操作系统虚拟化，同时支持灵活的任务配置与基于执行结果的评估方式。具体而言，LabOSBench在八种仪器模拟器上构建了96个子任务，覆盖从样本加载、对准、参数调节、数据采集到结果检查的完整工作流程。我们在子任务级别与端到端级别上评估了通用视觉语言模型、专用GUI智能体模型以及先进的智能体框架。实验结果表明，尽管现有智能体能够完成许多结构化的GUI子任务，但在需要反馈驱动的操作以及长时程工作流执行方面仍然存在困难。总体而言，LabOSBench为推动计算机使用智能体向科学仪器控制能力发展提供了一个可复现、低成本的测试平台。

View on arXiv Download PDF AI Translation

cs.AI / 112 / 2606.16808

Adaptive and Explicit safe: Triggering Latent Safety Awareness in Large Reasoning Models

自适应且显式的安全机制：在大型推理模型中触发潜在安全感知能力

Miao, Ke, Li, Jiaxin, Chen, Hongliang, Hu, Yuke, Qin, Zhan

Abstract

While Large Reasoning Models (LRMs) excel at complex tasks, they remain highly vulnerable to sophisticated jailbreaks and direct harmful queries. To address this vulnerability, prior works depend heavily on external manual data annotation for safety alignment. However, we observe that LRMs can inherently identify safety risks when being re-presented with original queries alongside their own reasoning trajectories -- a capability we term Latent Safety Awareness. To leverage this safety awareness, we first employ Supervised Fine-Tuning (SFT) to explicitly induce safe tags to trigger safety analysis and guidance following the initial reasoning content for unsafe queries, while preserving standard responses for general queries to ensure adaptive triggering. Subsequently, we apply Direct Preference Optimization (DPO) to further enhance the correctness and stability of the safety analysis and guidance. Notably, responses required for both training stages are entirely generated by models being optimized. With (Safe Trigger) SFT and DPO, experimental results demonstrate significant safety enhancement. For example, the Attack Success Rate (ASR) of DeepSeek-R1-Distill-Llama-8B, on average, drops 24.65% and 36.72% on harmful and jailbreak benchmarks, respectively. Finally, our Safe Trigger method exerts almost no negative impact on general performance or user experience.

Chinese Translation

尽管大型推理模型（Large Reasoning Models, LRMs）在复杂任务上表现优异，但它们仍然极易受到复杂越狱攻击（jailbreaks）以及直接有害查询的影响。为了解决这一脆弱性，既有研究高度依赖外部人工数据标注来进行安全对齐。然而，我们观察到，当将原始查询与其自身推理轨迹重新组合呈现时，LRMs能够内在地识别安全风险——我们将这一能力称为“潜在安全感知”（Latent Safety Awareness）。为了利用这种安全感知能力，我们首先采用监督微调（Supervised Fine-Tuning, SFT），通过在不安全查询的初始推理内容之后显式引入安全标签，引导模型触发安全分析与安全指导，同时对一般查询保持标准回复，以实现自适应触发机制。随后，我们应用直接偏好优化（Direct Preference Optimization, DPO），进一步提升安全分析与指导内容的准确性与稳定性。值得注意的是，这两个训练阶段所需的响应均完全由被优化的模型自身生成。在采用（Safe Trigger）SFT与DPO后，实验结果表明安全性得到显著提升。例如，在有害性与越狱基准测试中，DeepSeek-R1-Distill-Llama-8B的攻击成功率（Attack Success Rate, ASR）平均分别下降了24.65%和36.72%。最后，该Safe Trigger方法几乎不会对模型的通用性能或用户体验造成负面影响。

View on arXiv Download PDF AI Translation

cs.AI / 113 / 2606.16811

Scaling LLM Reasoning from Minimal Labels: A Semi-Supervised Framework with a Lightweight Verifier

从最少标注扩展大语言模型（LLM）推理能力：一种基于轻量级验证器的半监督框架

Kato, Keizo, Chu, Chenhui, Murawaki, Yugo, Kurohashi, Sado

Abstract

For the development of Large language models (LLMs), recent approaches to generating pseudo intermediate reasoning have shown remarkable progress. But they typically rely on large numbers of correctly annotated answers to assess reasoning quality. This paper presents a semi-supervised framework that scales reasoning learning from minimal supervision, turning reasoning verification itself into a data creation mechanism. We train a lightweight reasoning-correctness classifier on only a few labeled samples, which judges whether intermediate reasoning traces generated by an LLM are valid. Furthermore, an entropy-based confidence threshold filters out unreliable samples, and the remaining high-confidence reasoning traces are used to fine-tune the model. Experiments on Verifiable Math Problems (Orca-Math subset) and Question Answering on Image Scene Graphs (GQA) with Visual Programming show that our method achieves accuracy comparable to using 10-15x more labeled data. Ablation analyses confirm that both the classifier and entropy filtering are essential for scalable and noise-resistant pseudo-labeling. By replacing expensive answer-level supervision with lightweight reasoning verification, our method provides a practical path toward constructing large-scale reasoning resources and paves the way for future autonomous reasoning systems that learn from minimal human input.

Chinese Translation

在大语言模型（Large Language Models, LLMs）的发展过程中，近年来用于生成伪中间推理（pseudo intermediate reasoning）的方法取得了显著进展。然而，这些方法通常依赖大量正确标注的答案来评估推理质量。本文提出了一种半监督框架，能够在极少监督信号的条件下扩展推理学习能力，并将推理验证本身转化为一种数据生成机制。我们仅使用少量标注样本训练一个轻量级的推理正确性分类器，用于判断由LLM生成的中间推理轨迹是否有效。此外，引入基于熵的置信度阈值，用于过滤不可靠样本，而剩余的高置信度推理轨迹则用于对模型进行微调。在可验证数学问题（Verifiable Math Problems，Orca-Math子集）以及基于视觉编程的图像场景图问答任务（Question Answering on Image Scene Graphs, GQA）上的实验表明，该方法在仅使用10–15倍更多标注数据的情况下仍能达到相当的准确率。消融实验进一步验证了分类器与熵过滤机制在可扩展且抗噪声伪标注过程中的关键作用。通过用轻量级推理验证替代昂贵的答案级监督，该方法为构建大规模推理资源提供了一条实用路径，并为未来从最少人工输入中学习的自主推理系统奠定了基础。

View on arXiv Download PDF AI Translation

cs.AI / 114 / 2606.16813

GIST-CMTF: Goal-State Inference for Causal Minimal Tool Filtering in LLM Agents

GIST-CMTF：用于大语言模型（LLM）智能体中因果最小工具过滤（CMTF）的目标状态推断

Babu, Rahul Suresh, Shukla, Rohit

Abstract

Tool-augmented LLM agents rely on runtime filtering to decide which tools should be visible at each step. Causal Minimal Tool Filtering (CMTF) reduces tool-choice confusion by exposing only the next causally necessary tool frontier, but it assumes that the user request has already been mapped to a symbolic goal state. In practice, requests such as "handle my appointment" or "take care of this email" may correspond to multiple possible goals. This creates wrong-goal execution, where an agent follows a valid causal tool path for an unintended objective. We introduce GIST-CMTF, a goal-state inference layer that predicts candidate symbolic goals over the same state-transition vocabulary used by CMTF, estimates ambiguity, and either applies CMTF or exposes clarification as a causal action that produces missing goal or state variables. We evaluate GIST-CMTF across seven model backends, six filtering methods, and 120 controlled tool-use tasks. GIST-CMTF achieves 97.0% task success, compared with 80.1% for top-goal CMTF and 82.9% for semantic-goal CMTF. It reduces wrong-goal execution from 19.4% under top-goal CMTF to 2.5%, while preserving the one-tool exposure of causal filtering and using substantially fewer tokens than all-tools exposure. These results suggest that reliable tool-augmented agents should validate goal state, not only tool relevance, before exposing external actions.

Chinese Translation

基于工具增强的大语言模型（LLM）智能体依赖运行时过滤机制，在每一步决定哪些工具应当被可见。因果最小工具过滤（Causal Minimal Tool Filtering, CMTF）通过仅暴露下一步因果上必要的工具前沿，从而减少工具选择混淆，但其前提是假设用户请求已经被映射为一个符号化的目标状态。在实际应用中，“处理我的预约”或“处理这封邮件”等请求可能对应多个不同的潜在目标，从而引发错误目标执行问题，即智能体虽然遵循了一个有效的因果工具路径，但却服务于非预期目标。本文提出 GIST-CMTF，一种目标状态推断层，用于在与 CMTF 相同的状态转移词汇空间上预测候选符号目标，估计目标歧义程度，并在应用 CMTF 或将澄清请求作为因果动作之间进行选择；该澄清动作可用于生成缺失的目标或状态变量。我们在7种模型后端、6种过滤方法以及120个受控工具使用任务上对 GIST-CMTF 进行了评估。结果表明，GIST-CMTF 的任务成功率达到97.0%，显著高于顶层目标CMTF的80.1%以及语义目标CMTF的82.9%。同时，其错误目标执行率从顶层CMTF的19.4%降低至2.5%，并在保持因果过滤“单工具暴露”特性的同时，相比“暴露全部工具”的方法显著减少token使用量。这些结果表明，在暴露外部动作之前，可靠的工具增强智能体不仅需要判断工具相关性，还必须验证目标状态的正确性。

View on arXiv Download PDF AI Translation

cs.AI / 115 / 2606.16893

Symbolic Informalization: Fluent, Productive, Multilingual

符号非形式化：流畅、可生成、多语言

Ranta, Aarne

Abstract

Symbolic informalization enables a reliable conversion of formal mathematics to natural language. It has the potential to make machine-checked content human-readable without loss of precision. In a traditional proof system usage, symbolic informalization generalizes the limited mechanisms of syntactic sugar into the ordinary language of mathematics. In a setting where proofs are constructed by artificial intelligence and autoformalization, symbolic informalization can explain what precisely has been constructed. This paper outlines the project Informath, which aims to show how symbolic informalization can produce fluent text with a reasonable development effort and address multiple formal and natural languages. Informath is based on an interlingual architecture, where Dedukti works as a hub between different proof systems (Agda, Lean, Rocq) and Grammatical Framework (GF) takes care of linguistic correctness and variation in different natural languages.

Chinese Translation

符号非形式化（Symbolic informalization）能够将形式化数学可靠地转换为自然语言。它有潜力在不损失精确性的前提下，使机器验证的内容变得可读、易于人类理解。在传统证明系统的使用中，符号非形式化将有限的语法糖（syntactic sugar）机制扩展并泛化为数学的日常语言表达形式。在由人工智能构造证明以及自动形式化（autoformalization）的场景中，符号非形式化可以用于清晰解释所实际构造出的精确内容。本文概述了 Informath 项目，该项目旨在展示符号非形式化如何以合理的开发成本生成流畅文本，并支持多种形式语言与自然语言的处理。Informath 基于一种跨语言（interlingual）架构，其中 Dedukti 作为不同证明系统（Agda、Lean、Rocq）之间的枢纽，而 Grammatical Framework（GF）负责保证不同自然语言中的语言正确性及表达变体的生成与处理。

View on arXiv Download PDF AI Translation

cs.AI / 116 / 2606.16914

Greed Is Learned: Visible Incentives as Reward-Hacking Triggers

贪婪是可以习得的：可见激励作为奖励投机（reward-hacking）触发因素

Che, Tong, Wu, Rui

Abstract

Deployed agents increasingly act with their reward proxy in view, such as a balance, score, or KPI dashboard. We show that reinforcement learning can make a policy \emph{addicted} to such a visible self-benefit channel. It chases the displayed payoff across held-out domains, sacrifices the true task to do so, and follows the channel wherever we rewrite it, while policies that never saw the channel stay honest. We call this \emph{reward-channel addiction} and study it in \emph{MoneyWorld}, a synthetic sandbox. The addiction can \emph{flip a model's safety alignment}: trained only on innocuous money tasks with no safety content, the model abandons the safe action it otherwise always takes whenever a dashboard pays for an unsafe one, and reverts to safe once the channel is hidden. This learned bribe replicates across model scales and families. Blindly optimizing super-capable, next-generation AI on KPIs or P\&L can be dangerous for alignment. \emph{Greed is learned} when following such a channel pays.

Chinese Translation

部署后的智能体越来越多地在其奖励代理（reward proxy）可见的情况下行动，例如余额、分数或KPI仪表盘。我们表明，强化学习可以使策略（policy）对这种可见的自我收益通道产生“成瘾”（addicted）。该策略会在未见过的数据分布（held-out domains）中持续追逐显示的收益，在此过程中牺牲真实任务目标，并且无论我们如何重写该通道，它都会持续跟随该通道，而未接触过该通道的策略则保持诚实。我们将这种现象称为“奖励通道成瘾”（reward-channel addiction），并在合成沙盒环境“MoneyWorld”中进行了研究。该成瘾现象可以翻转模型的安全对齐（safety alignment）：即使模型仅在无害的货币任务上进行训练且不包含任何安全相关内容，一旦仪表盘为不安全行为提供奖励，它就会放弃原本始终采取的安全行为；而当该通道被隐藏后，它又会恢复安全行为。这种被学习到的“贿赂”在不同模型规模与不同模型家族中均可复现。在KPI或盈亏（P&L）上盲目优化超强能力的下一代人工智能，可能对对齐性造成风险。当沿着该通道行动能够带来收益时，“贪婪是可以习得的”。

View on arXiv Download PDF AI Translation

cs.AI / 117 / 2606.16923

MA-SBI: Misspecification-Aware Simulation-Based Inference via Side-Channel Guidance

MA-SBI：基于侧信道指导的误设定感知模拟推断

V, Arunkumar, Gandhudi, Manoranjan, R., Gangadharan G., Prakash, Arun, Senthilkumar, S.

Abstract

Simulation-based inference (SBI) of latent parameters is often hindered by simulator misspecification, the mismatch between simulated and real-world observations caused by inherent modeling simplifications. RoPE, the recent state-of-the-art for robust SBI, addresses this through optimal transport between learned representations of real and simulated observations, but requires ground-truth parameter calibration pairs that are typically unavailable in the very settings where SBI is needed. What practitioners do have is unstructured side-information such as regime labels, instruction text, and policy bulletins. We propose Misspecification-Aware Simulation-Based Inference (MA-SBI), a calibration-free framework that turns this side-channel into a posterior correction. A learned corrector maps side-channel text to an observation-space shift applied before any pre-trained amortized posterior, requiring no retraining and no parameter ground-truth. Our main theorem bounds achievable bias reduction by the mutual information between misspecification and side-channel, with a non-vacuous constant that extends to all sub-Gaussian noise via Donsker-Varadhan. On hide-the-calibration benchmarks, MA-SBI with text alone matches the oracle posterior across 10 seeds and two backbones (TOST equivalence), while RoPE given more data does not. The two approaches are complementary: where misspecification is structural and recoverable from parameter pairs, RoPE dominates, as the theory predicts. A stochastic variant improves posterior-predictive log-likelihood on real COVID and OxCGRT epidemiological data, and correctly leaves the posterior unchanged on a well-specified cognitive-science corpus.

Chinese Translation

潜在参数的基于模拟的推断（SBI）常常受到模拟器误设定的阻碍，这种误设定是由于固有建模简化导致的模拟与现实观察之间的不匹配。RoPE是当前稳健SBI的最新技术，通过在真实和模拟观察的学习表示之间进行最优传输来解决这一问题，但需要通常在需要SBI的环境中不可用的真实参数校准对。实践者所拥有的是非结构化的侧信息，如状态标签、指令文本和政策公告。我们提出了误设定感知模拟推断（MA-SBI），这是一个无校准框架，将侧信道转化为后验修正。一个学习到的修正器将侧信道文本映射到在任何预训练的摊销后验之前应用的观察空间偏移，且无需重新训练和真实参数。我们的主要定理通过误设定与侧信道之间的互信息界定了可实现的偏差减少，具有一个非空常数，该常数通过Donsker-Varadhan扩展到所有亚高斯噪声。在隐藏校准基准上，仅使用文本的MA-SBI在10个种子和两个骨干网络（TOST等价）中匹配了oracle后验，而RoPE在获得更多数据时则未能做到。这两种方法是互补的：当误设定是结构性的并且可以从参数对中恢复时，RoPE占据主导地位，正如理论所预测的那样。一个随机变体在真实的COVID和OxCGRT流行病数据上提高了后验预测对数似然，并在一个良好设定的认知科学语料库上正确地保持了后验不变。

View on arXiv Download PDF AI Translation

cs.AI / 118 / 2606.16925

RAID: Semantic Graph Diffusion for True Cold-Start and Cross-Lingual Forecasting

RAID：用于真实冷启动和跨语言预测的语义图扩散

V, Arunkumar, Gandhudi, Manoranjan, R., Gangadharan G., Prakash, Arun, Senthilkumar, S.

Abstract

Time-series foundation models show strong transfer performance when given a non-empty history window. However, true cold-start scenarios, where a new item has no prior observations, violate this assumption. We propose RAID (Retrieval-Augmented Iterative Diffusion) a framework, which replaces history-based correlation learning with metadata-driven semantic retrieval and graph-conditioned diffusion. RAID maps textual metadata into a shared semantic space using a frozen multilingual embedding model and constructs an inductive retrieval graph that extends naturally to unseen items. It first forms a base forecast by aggregating information from semantically related neighbors, then refines this forecast with a gated diffusion module to model residual uncertainty. Under a strict true cold-start protocol, RAID outperforms strong foundation models and competitive baselines on both forecasting accuracy and prediction interval coverage, while reducing inference latency by an order of magnitude through non-autoregressive decoding. The shared semantic space also enables zero-shot cross-lingual transfer, allowing a model trained on English descriptions to generalize to items described in other languages without direct supervision.

Chinese Translation

时间序列基础模型在给定非空历史窗口时表现出强大的迁移性能。然而，真实冷启动场景中，新项目没有先前的观察数据，这一假设被违反。我们提出了RAID（检索增强迭代扩散）框架，该框架用元数据驱动的语义检索和图条件扩散替代基于历史的相关性学习。RAID使用冻结的多语言嵌入模型将文本元数据映射到共享语义空间，并构建一个归纳检索图，自然扩展到未见过的项目。它首先通过聚合语义相关邻居的信息形成基础预测，然后通过门控扩散模块来细化该预测，以建模残余不确定性。在严格的真实冷启动协议下，RAID在预测准确性和预测区间覆盖率方面超越了强大的基础模型和竞争基准，同时通过非自回归解码将推理延迟减少了一个数量级。共享的语义空间还实现了零样本跨语言迁移，使得在英语描述上训练的模型能够在没有直接监督的情况下推广到用其他语言描述的项目。

View on arXiv Download PDF AI Translation

cs.AI / 119 / 2606.16944

A Causal Model of Theory of Mind in Conflict for Artificial Intelligence

冲突中人工智能的心智理论因果模型

Gurney, Nikolos

Abstract

Theory of mind (ToM), the capacity to ascribe mental states to others and use those ascriptions for prediction and inference, is widely assumed to be essential for effective human-machine integration. Existing AI-ToM models address \emph{how} to mentalize, but leave the question of when largely unaddressed. The central question is: under what situational and agent-level conditions is ToM engagement causally warranted in conflict? This paper presents a structural causal model formalized as a directed acyclic graph (DAG), treating ToM as a mechanism activated by situational and agent-level conditions rather than as an always-on capacity. The model specifies four exogenous variables capturing situational and agent-level conditions, five endogenous mediators, and a mechanistic ToM node producing engagement states through three distinct causal pathways: a tractability pathway, a reasoning-depth pathway, and an enabling-cause pathway. The primary outcome is epistemic accuracy, which decouples social reasoning from behavioral policy and generalizes across social phenomena beyond conflict. The framework gives AI systems a principled, resource-rational decision procedure for mentalizing, with implications for efficiency, trust, and the development of robust artificial social intelligence. Simulation validation, empirical human-machine teaming studies, and ethical considerations arising from conflict-optimized mentalizing are discussed.

Chinese Translation

心智理论（Theory of mind, ToM）是指将心理状态归因于他人并利用这些归因进行预测和推理的能力，广泛认为这是有效的人机整合所必需的。现有的AI-ToM模型主要关注于 extit{如何}进行心智化，但对何时进行心智化的问题却鲜有探讨。核心问题是：在什么样的情境和代理者层面的条件下，心智理论的参与在冲突中是因果上合理的？本文提出了一个结构因果模型，形式化为有向无环图（Directed Acyclic Graph, DAG），将心智理论视为由情境和代理者层面的条件激活的机制，而非一种始终开启的能力。该模型指定了四个外生变量，以捕捉情境和代理者层面的条件，五个内生中介变量，以及一个通过三条不同因果路径（可处理性路径、推理深度路径和启发因果路径）产生参与状态的机制性心智理论节点。主要结果是认知准确性，它将社会推理与行为政策解耦，并在冲突之外的社会现象中具有广泛的适用性。该框架为人工智能系统提供了一种原则性、资源理性的心智化决策程序，对效率、信任以及强大人工社会智能的发展具有重要意义。文中还讨论了模拟验证、实证人机协作研究以及由冲突优化的心智化所引发的伦理考量。

View on arXiv Download PDF AI Translation

cs.AI / 120 / 2606.16974

The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers

开放科学的拥抱：对十年人工智能研究和56800篇会议论文的分析

Coakley, Kevin L, Snelleman, Thijs, Hoos, Holger, Gundersen, Odd Erik

Abstract

The reproducibility crisis has directed the AI research community toward improving documentation practices. Several studies have identified methodological issues, and in response, the most impactful venues in the field have introduced reproducibility checklists. We seek to understand whether documentation practices have changed over time by assessing all published papers at five leading AI conferences over the past decade. Seven reproducibility variables were identified, quality-assured and used to analyse 56 800 publications. Our analysis reveals that in the period 2014 to 2024, documentation practices have improved; papers sharing both code and data increased nearly sixfold, from 11% to 64% Building on empirical reproducibility rates from a prior study, we estimate - inferred from documentation practices, not direct testing - that reproducibility increased from 28% in 2014 to 64% in 2024. Improvements in documentation practices predate the introduction of reproducibility checklists, suggesting these changes reflect a broader movement toward open science rather than a direct response to formal requirements.

Chinese Translation

可重复性危机促使人工智能研究社区改善文档实践。多项研究已识别出方法论问题，作为回应，该领域最具影响力的会议引入了可重复性检查清单。我们旨在通过评估过去十年在五个领先人工智能会议上发表的所有论文，了解文档实践是否随时间而变化。我们识别了七个可重复性变量，进行了质量保证，并用于分析56800篇出版物。我们的分析显示，在2014年至2024年期间，文档实践有所改善；同时共享代码和数据的论文数量几乎增加了六倍，从11%上升至64%。基于先前研究中的实证可重复性率，我们估计——这一推断基于文档实践，而非直接测试——可重复性从2014年的28%增加到2024年的64%。文档实践的改善早于可重复性检查清单的引入，这表明这些变化反映了朝向开放科学的更广泛运动，而不是对正式要求的直接回应。

View on arXiv Download PDF AI Translation

cs.AI / 121 / 2606.16987

Consensus-based Agentic Large Language Model Framework for Harmonized Tariff Schedule Code Classification

基于共识的代理大型语言模型框架用于协调关税分类代码的分类

Nguyen, Truong Thanh Hung, Nguyen, Khanh Van Quynh, Cao, Hoang-Loc, Duong, Tri, Ho, Phuc, Pham, Van, Nguyen, Loc, Cao, Hung

Abstract

Accurate Harmonized Tariff Schedule (HTS) code classification is essential for customs clearance, duty assessment, trade statistics, and regulatory compliance in maritime logistics. However, exact HTS classification remains challenging because product descriptions are often short, incomplete, or ambiguous, while correct classification depends on hierarchical tariff structures, legal notes, and jurisdiction-specific rules. This paper proposes an agentic large language model (LLM) framework for Canadian 10-digit HTS code classification in smart-port and maritime logistics environments. The framework integrates multi-agent information retrieval, semantic retrieval over official tariff documents, evidence-grounded reasoning, consensus-based validation, element-wise voting across hierarchical code components, confidence estimation, and human-in-the-loop escalation. We evaluate the framework on a private dataset of 3,300 domain-expert-labeled product records collected from logistics and delivery contexts. Experimental results show that exact 10-digit classification remains difficult even for advanced LLMs, with performance decreasing from coarse chapter-level prediction to fine-grained tariff and statistical suffix assignment. These findings demonstrate the need for evidence-grounded, uncertainty-aware, and human-centered classification workflows rather than fully autonomous single-step prediction. The proposed framework supports more interpretable, accountable, and compliance-oriented HTS classification for maritime logistics and smart-port operations. Our code is available at https://github.com/Analytics-Everywhere-Lab/hts.

Chinese Translation

准确的协调关税表（HTS）代码分类对于海事物流中的海关清关、关税评估、贸易统计和合规性至关重要。然而，由于产品描述通常简短、不完整或模糊，准确的HTS分类仍然具有挑战性，而正确分类依赖于分层关税结构、法律注释和特定于管辖区的规则。本文提出了一种代理大型语言模型（LLM）框架，用于在智能港口和海事物流环境中对加拿大10位HTS代码进行分类。该框架集成了多代理信息检索、对官方关税文件的语义检索、基于证据的推理、基于共识的验证、跨层级代码组件的逐元素投票、信心估计和人机协作的升级。我们在一个包含3,300个领域专家标注的产品记录的私有数据集上评估了该框架，这些记录来自物流和交付环境。实验结果表明，即使对于先进的LLM，准确的10位分类仍然困难，性能从粗略的章节级预测到细粒度的关税和统计后缀分配均有所下降。这些发现表明，需要基于证据、关注不确定性和以人为中心的分类工作流程，而不是完全自主的一步预测。所提出的框架支持更具可解释性、可追溯性和合规导向的HTS分类，以适应海事物流和智能港口操作。我们的代码可在 https://github.com/Analytics-Everywhere-Lab/hts 获取。

View on arXiv Download PDF AI Translation

cs.AI / 122 / 2606.16995

When in Doubt, Plan It Out: Committed Small Language Model Deliberation for Reactive Reinforcement Learning

在不确定时，规划它：用于反应性强化学习的承诺小型语言模型深思熟虑

Gavenski, Nathan, Monteiro, Juarez, Galuppo, Francisco, Veloso, Adriano, Rodrigues, Odinaldo

Abstract

Reinforcement Learning (RL) policies often degrade in unfamiliar environments because they lack explicit deliberation. We propose Plan, Align, Commit, Think (PACT), a hybrid architecture that combines a fast, reactive RL policy with a slow, deliberative Small Language Model (SLM) planner. PACT invokes the SLM asynchronously to generate and validate candidate action plans. Once a plan is verified through simulation as safe, feasible, and complete, it is executed directly, bypassing the RL policy without retraining or modifying it. Evaluated on three FrozenLake configurations of increasing difficulty, PACT outperforms all baselines while relying on a 2B-parameter SLM backbone, suggesting that deliberative planning and reactive execution are more powerful in concert than either is alone in these settings.

Chinese Translation

强化学习（RL）策略在不熟悉的环境中往往会退化，因为它们缺乏明确的深思熟虑。我们提出了计划、对齐、承诺、思考（PACT），这是一种混合架构，将快速的反应性RL策略与缓慢的深思熟虑小型语言模型（SLM）规划器相结合。PACT 异步调用 SLM 以生成和验证候选行动计划。一旦通过模拟验证计划是安全、可行和完整的，它将被直接执行，绕过RL策略，而无需重新训练或修改。通过在三个逐渐增加难度的FrozenLake配置上进行评估，PACT 超越了所有基线，同时依赖于一个具有20亿参数的SLM骨干，表明在这些环境中，深思熟虑的规划与反应性执行的结合比单独使用任何一种方法更为强大。

View on arXiv Download PDF AI Translation

cs.AI / 123 / 2606.17005

Bayesian Inference and Decision Audits for Public Archives of Frontier AI Evaluations

前沿人工智能评估公共档案的贝叶斯推断与决策审计

Long, Yanan

Abstract

Public AI evaluations are often read as terminal leaderboards, yet the underlying evidence is a selective time series shaped by reporting rules, benchmark revisions, and missingness. Repeated public archives for LiveBench and Open LLM Leaderboard v2 serve as the primary longitudinal record; LMArena provides a preference stress test; and GAIA and tau-bench contribute limited agentic pilots. Together, these archives instantiate a Bayesian inference problem: under a fixed reporting convention, one constructed terminal-only example over $1{,}000$ systems is compatible with two pre-terminal histories, yielding times of $23.03$ or $75.13$ to reach within $0.05$ of the ceiling under the same terminal-tail model. In synthetic posterior comparisons, action-facing diagnostics differ across observation regimes. The candidate selection-aware frontier model fails synthetic recovery, objective-archive prediction, preference transfer, and uncertainty calibration; correspondingly, fixed audit gates reject its stronger claims. An archive-and-adjudication protocol reconstructs public evaluation histories, isolates a verified timing boundary, and falsifies unsupported frontier claims.

Chinese Translation

公共人工智能评估通常被视为终端排行榜，然而其背后的证据是一系列受报告规则、基准修订和缺失数据影响的选择性时间序列。LiveBench 和 Open LLM Leaderboard v2 的重复公共档案作为主要的纵向记录；LMArena 提供了偏好压力测试；而 GAIA 和 tau-bench 则贡献了有限的代理试点。这些档案共同构成了一个贝叶斯推断问题：在固定的报告惯例下，构建的仅终端示例在超过 $1{,}000$ 个系统中与两个前终端历史兼容，导致在相同的终端尾模型下达到 $0.05$ 的上限所需的时间为 $23.03$ 或 $75.13$。在合成后验比较中，面向行动的诊断在观察机制之间存在差异。候选选择感知的前沿模型未能实现合成恢复、目标档案预测、偏好转移和不确定性校准；相应地，固定审计门拒绝了其更强的主张。一种档案与裁决协议重建了公共评估历史，隔离了经过验证的时间边界，并驳斥了不支持的前沿主张。

View on arXiv Download PDF AI Translation

计算语言学 (Computation and Language)

140

cs.CL / 1 / 2606.14832

PhoneHarness: Harnessing Phone-Use Agents through Mixed GUI, CLI, and Tool Actions

PhoneHarness：通过混合图形用户界面、命令行界面和工具操作来利用手机使用代理

Li, Chenxin, Fang, Zhengyao, Tang, Zhengyang, Lyu, Pengyuan, Zhou, Xingran, Lai, Xin, Tang, Fei, Wu, Liang, Guo, Yiduo, Wang, Weinong, Li, Junyi, Zhang, Yi, Ding, Yang, Shen, Huawen, Fan, Sunqi, Peng, Shangpin, Ruan, Zheng, Zhang, Anran, Wang, Benyou, Zhang, Chengquan, Hu, Han

Abstract

Phone agents are increasingly expected to complete real mobile workflows rather than merely predict the next screen action. However, much of the current mobile-agent literature still evaluates agents primarily as GUI controllers that observe a screen, emit taps and swipes, and are scored by target app state. Real phone-use tasks are broader: they require deciding when to use app GUIs, device-side commands, or structured tools, while leaving evidence that the intended side effect actually occurred. We introduce PhoneHarness, a mixed-action benchmark and execution harness for studying phone-use agents on verifiable mobile workflows. PhoneHarness runs a device-side agent loop over GUI, CLI, and host-side tool actions, combining deterministic action routing with bounded GUI delegation and auditable execution traces. Its benchmark, PhoneHarness Bench, evaluates whether agents complete tasks with observable side effects, not only whether they produce plausible final answers. On the annotated evaluation split, PhoneHarness reaches a 75.0% pass rate, outperforming the strongest non-PhoneHarness settings by 12.9 percentage points. PhoneHarness and PhoneHarness Bench therefore play distinct but mutually dependent roles: the harness makes mixed phone workflows executable, while the benchmark measures whether agents can use that harness reliably and safely. Our findings suggest that reliable phone automation depends on action-surface routing and verifiable execution, not only visual GUI control.

Chinese Translation

手机代理越来越被期望能够完成真实的移动工作流程，而不仅仅是预测下一个屏幕操作。然而，目前的移动代理文献仍主要将代理评估为观察屏幕、发出点击和滑动的图形用户界面（GUI）控制器，并根据目标应用状态进行评分。真实的手机使用任务更为广泛：它们需要决定何时使用应用程序的图形用户界面、设备端命令或结构化工具，同时留下证据表明预期的副作用确实发生。我们引入了PhoneHarness，这是一个混合操作基准和执行工具，用于研究可验证移动工作流程上的手机使用代理。PhoneHarness在图形用户界面、命令行界面（CLI）和主机端工具操作上运行设备端代理循环，结合了确定性操作路由、有限的图形用户界面委托和可审计的执行轨迹。其基准测试PhoneHarness Bench评估代理是否完成了具有可观察副作用的任务，而不仅仅是是否产生了合理的最终答案。在标注的评估分割中，PhoneHarness达到了75.0%的通过率，超越了最强的非PhoneHarness设置12.9个百分点。因此，PhoneHarness和PhoneHarness Bench发挥着不同但相互依赖的作用：工具使混合手机工作流程可执行，而基准测试则衡量代理是否能够可靠且安全地使用该工具。我们的研究结果表明，可靠的手机自动化依赖于操作表面路由和可验证的执行，而不仅仅是视觉图形用户界面控制。

View on arXiv Download PDF AI Translation

cs.CL / 2 / 2606.14867

Evaluating the Robustness of Proof Autoformalization in Lean 4

评估 Lean 4 中证明自动形式化的鲁棒性

Gui, Zhengtao, Yang, Sheng, Shi, Zhouxing

Abstract

Proof autoformalization aims to translate a mathematical informal proof written in natural language into a formal proof in a formal language such as Lean~4. Several works have developed LLM-based models for proof autoformalization. However, existing evaluations have typically focused on translating well-formed informal proofs from curated datasets. We argue that a robust proof autoformalizer must remain faithful even for informal proofs that diverge from these idealized ones, and we present the first study on the robustness of proof autoformalization models. We formulate two categories of perturbations and evaluate robustness under each: a global perturbation paraphrases the informal proof in a different style, under which the formalization should remain consistent; a local perturbation alters a value, symbol, or proof step, possibly in a counterfactual way, and a robust formalization should faithfully reflect the perturbation rather than reverting to the original one or inferring a different one on its own. We build a benchmark with both perturbations on miniF2F and MATH-500, and automatically measure how stable a proof autoformalization's correctness is under global perturbations and how faithfully its output reflects local perturbations. We evaluate seven recent models, all of which are sensitive to global perturbations and mostly fail to remain faithful under local perturbations. Code and data are available via https://github.com/ucr-rai/robust-proof-autoformalization.

Chinese Translation

证明自动形式化旨在将用自然语言撰写的数学非正式证明翻译为在正式语言（如 Lean~4）中的正式证明。已有多项研究开发了基于大语言模型（LLM）的证明自动形式化模型。然而，现有评估通常集中在从策划数据集中翻译良构的非正式证明。我们认为，一个鲁棒的证明自动形式化器必须在面对偏离这些理想化证明的非正式证明时仍然保持忠实性，并且我们首次对证明自动形式化模型的鲁棒性进行了研究。我们提出了两类扰动并在每类下评估鲁棒性：全局扰动以不同风格对非正式证明进行释义，在这种情况下，形式化应保持一致；局部扰动则改变一个值、符号或证明步骤，可能以反事实的方式进行，而鲁棒的形式化应忠实地反映扰动，而不是恢复到原始状态或自行推断出不同的内容。我们在 miniF2F 和 MATH-500 上建立了包含这两类扰动的基准，并自动测量证明自动形式化在全局扰动下的正确性稳定性以及其输出在局部扰动下的忠实反映程度。我们评估了七个近期模型，所有模型对全局扰动敏感，并且大多数在局部扰动下未能保持忠实性。代码和数据可通过 https://github.com/ucr-rai/robust-proof-autoformalization 获取。

View on arXiv Download PDF AI Translation

cs.CL / 3 / 2606.14875

Context Compression Is Not One Thing: Readable Symbolic Re-expression vs. Coherent Summary at Matched Budget

上下文压缩不是单一概念：可读的符号重表达与匹配预算下的连贯摘要

Bei, Sisong, Arbuzov, Mikhail L., Dong, Ziwei, Kalaev, Dmitri, Shvets, Alexey

Abstract

We study context compression for multi-hop question answering with small language models. We propose Telegraph English, a readable symbolic format that rewrites retrieved passages into structured entity-relation statements, preserving reasoning evidence at lower token cost. In controlled experiments on MuSiQue, TwoWiki, and HotpotQA, Telegraph English outperforms three matched-budget compression baselines (character-level deletion, truncation, and random sub-sampling) on every dataset, with gains of 13 to 20 F1 percentage point. It also outperforms a coherent prose summary produced by the same encoder on the hardest dataset. A pre-registered depth-interaction hypothesis is null: the advantage does not grow with reasoning depth within datasets. We interpret these results as evidence that readable symbolic re-expression preserves entity content more densely than either natural language or coherent summarization at matched token budget.

Chinese Translation

我们研究了针对小型语言模型的多跳问答中的上下文压缩。我们提出了电报英语（Telegraph English），一种可读的符号格式，将检索到的段落重写为结构化的实体-关系陈述，同时以更低的标记成本保留推理证据。在对MuSiQue、TwoWiki和HotpotQA的控制实验中，电报英语在每个数据集上均优于三种匹配预算的压缩基线（字符级删除、截断和随机子采样），F1得分提升幅度在13到20个百分点之间。它在最难的数据集上也优于同一编码器生成的连贯散文摘要。预注册的深度交互假设为零：在数据集中，优势并未随着推理深度的增加而增长。我们将这些结果解读为可读的符号重表达在匹配的标记预算下比自然语言或连贯摘要更密集地保留了实体内容的证据。

View on arXiv Download PDF AI Translation

cs.CL / 4 / 2606.14943

Simplifying the Modeling of Arbitrary Conditionals in Natural Language

简化自然语言中任意条件的建模

Lu, Yinhan, Elmoznino, Eric, Gagnon, Léo, Mittal, Sarthak, Kasetty, Tejas, Lajoie, Guillaume

Abstract

Causal Transformers model sequences through an autoregressive factorization of the joint distribution, which enables efficient left-to-right decoding and conditional likelihood computation. However, they cannot tractably sample from or evaluate arbitrary conditionals -- e.g., a block of text conditioned on past and future tokens. Recent work aims to solve this problem through novel architectures, but they often lead to sub-optimal modeling of such conditionals and degraded generations. We propose Arbitrary Conditionals GPT (AC-GPT) which introduces a simple modification to standard causal Transformers to enable evaluating and sampling from arbitrary conditionals -- including past, future, and mixed contexts -- within a single forward pass. Unlike prior approaches, our method preserves the standard left-to-right ordering and next-token prediction objective essential for both strong performance and efficient training on natural language. Crucially, this compatibility allows existing LLMs to be fine-tuned for arbitrary conditioning. Our empirical results indicate that our method outperforms baselines on modeling arbitrary conditionals, without degrading standard left-to-right performance.

Chinese Translation

因果变换器通过对联合分布的自回归因子分解来建模序列，这使得高效的从左到右解码和条件似然计算成为可能。然而，它们无法有效地从任意条件中进行采样或评估——例如，基于过去和未来标记的文本块。最近的研究旨在通过新颖的架构解决这一问题，但这些架构往往导致对这些条件的建模效果不佳，并且生成质量下降。我们提出了任意条件 GPT（AC-GPT），该方法对标准因果变换器进行了简单的修改，以便在单次前向传播中评估和采样任意条件——包括过去、未来和混合上下文。与之前的方法不同，我们的方法保留了标准的从左到右的顺序和下一个标记预测目标，这对于在自然语言上实现强大的性能和高效的训练至关重要。关键是，这种兼容性使得现有的大型语言模型（LLMs）能够针对任意条件进行微调。我们的实证结果表明，我们的方法在建模任意条件方面优于基线，同时不降低标准的从左到右性能。

View on arXiv Download PDF AI Translation

cs.CL / 5 / 2606.14961

CoRA: Confidence-Rationale Alignment for Reliable Chain-of-Thought Reasoning

CoRA：可靠的思维链推理中的置信度与理由对齐

Xiong, Juming, Liu, Weixin, Guo, Kevin, Ni, Congning, Zhu, Junchao, Qu, Chongyu, Yan, Chao, Brown, Katherine, Baidya, Avinash, Gao, Xiang, Malin, Bradley, Yin, Zhijun

Abstract

Chain-of-thought (CoT) reasoning can improve LLM performance, but high answer confidence may be misleading when the accompanying CoT rationale is plausible yet incomplete or poorly supported. We study confidence--rationale alignment: whether a model's confidence in its committed answer is justified by its generated rationale. We introduce a GRPO-based reinforcement learning framework that jointly rewards answer correctness, committed-answer probability, and rubric-based rationale support, where the rubric assesses grounding, coherence, task match, and connection to the selected answer without revealing the gold answer to the judge. Across MedQA, MathQA, and OpenBookQA using three open-weight LLMs, our method reduces the confidence--rationale alignment error by up to 26.51% compared with untuned checkpoints, SFT, and correctness-only GRPO, while maintaining competitive accuracy and often improving calibration. These results show that reliable CoT reasoning requires not only confident answers, but rationales that substantively support them.

Chinese Translation

思维链（CoT）推理可以提高大型语言模型（LLM）的性能，但当伴随的思维链理由虽然合理却不完整或支持不足时，高置信度的答案可能会误导我们。我们研究了置信度与理由的对齐：模型对其提交答案的置信度是否得到了其生成理由的合理支持。我们提出了一种基于GRPO的强化学习框架，该框架联合奖励答案的正确性、提交答案的概率以及基于评分标准的理由支持，其中评分标准评估基础、连贯性、任务匹配和与所选答案的连接，而不向评审者透露正确答案。在使用三种开放权重的LLM进行的MedQA、MathQA和OpenBookQA实验中，我们的方法将置信度与理由对齐的错误降低了多达26.51%，与未调优的检查点、SFT和仅考虑正确性的GRPO相比，同时保持了竞争力的准确性，并且通常改善了校准。这些结果表明，可靠的思维链推理不仅需要自信的答案，还需要实质上支持这些答案的理由。

View on arXiv Download PDF AI Translation

cs.CL / 6 / 2606.15007

Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning

Nemotron 3 Ultra：开放、高效的混合专家混合Mamba-Transformer模型用于自主推理

NVIDIA, :, Blakeman, Aaron, Thomas, Aaron, Jhunjhunwala, Aastha, Gupta, Abhibha, Khattar, Abhinav, Rajfer, Adam, Renduchintala, Adi, Asif, Adil, Vavre, Aditya, Miranda, Adriana Flores, Bilal, Ahmad, Zaman, Aileen, Hotchandani, Ajay, Shukla, Akanksha, Bercovich, Akhiad, Ficek, Aleksander, Gronskiy, Alex, Kondratenko, Alex, Steiner, Alex, Ye, Alex, Bukharin, Alexander, Milesi, Alexandre, Taghibakhshi, Ali, Gatti, Alice, Liu, Alisa, Kumar, Alok, Phanishayee, Amar, Mahabaleshwarkar, Ameya Sunil, Klein, Amir, Zuker, Amit, Geifman, Amnon, Bhiwandiwalla, Anahita, Subramaniam, Ananth, Santilli, Andrea, Fulks, Andrew, McHarg, Andrew, Tao, Andrew, Skliar, Andrii, Agrusa, Anjulie, Srivastava, Ankur, Verma, Ankur, Shors, Anna, Warno, Anna, Llaquet, Antoni-Joan Solergibert I, Mehta, Arham, Nowaczynski, Arkadiusz, Jain, Arti, Aithal, Ashwath, Poojary, Ashwin, Ahamed, Asif, Mishra, Asit, Thekkumpate, Asma Kuriparambil, Sohrabizadeh, Atefeh, Kaur, Avinash, Vem, Avinash, Dattagupta, Ayush, Anandan, Barath Subramaniam, Sadeghi, Bardiya, Lanir, Ben, Schifferer, Benedikt, Nushi, Besmira, Kartal, Bilal, Thiede, Bill, Rouhani, Bita Darvish, Deng, Bo, Schatz, Bob, Ginsburg, Boris, Wang, Boxin, Nemire, Brad, Norick, Brandon, Dang, Brian, Westphal, Brian, Yu, Brian, Khailany, Brucek, Catanzaro, Bryan, del Mundo, Carlo, Aarish, Caryln, Lee, Chankyu, Hwang, Chantal, Sakr, Charbel, Wang, Charles, Truong, Charlie, Cui, Chen, Cheng, Cheng, Hsieh, Cheng-Ping, Zhang, Chenghao, Deng, Chenhui, Patel, Chintan, Alexiuk, Chris, Cosgrove, Christian, Munley, Christian, Harvey, Christine, Parisien, Christopher, Shen, Chunyang, Li, Coco, Neale, Collin, Gao, Cynthia, Meurillon, Cyril, Gil, Dan, Su, Dan, Zhao, Dan, Corneil, Dane, Afrimi, Daniel, Egert, Daniel, Korzekwa, Daniel, Lo, Daniel, Machlab, Daniel, Serebrenik, Daniel, Sorokin, Daniil, Gitman, Daria, Levy, Daria, Stosic, Darko, Mosallanezhad, David, Yu, David, Karamyan, Davit, Donia, Deena, Debroy, Deep, Narayanan, Deepak, O'Kelly, Devin, Peri, Dheeraj, Nathawani, Dhruv, Di, Wu, Rekesh, Dima, Kakwani, Divyanshu, Plummer, Donald, Anh, Dong, Yu, Dongfeng, Jiang, Dongfu, Kim, Donnie, Poorkay, Dorrin, Riach, Duncan, Stosic, Dusan, VanStee, Dustin, Meng, Eavan, Minasyan, Edgar, Lin, Edward, Long, Eileen Margaret Peters, Sarafin, Elad, Segal, Elad, Lantz, Elena, Evans, Ellie, Ning, Elliott, Chung, Eric, Harper, Eric, Pham-Hung, Eric, Tramel, Eric, Yang, Eric, Galinkin, Erick, Pounds, Erik, Goncalves, Erika Goncalves, Briones, Evan, Wu, Evan, Bakhturina, Evelina, Tsykunov, Evgeny, Dobrowolska, Ewa, Ladhak, Faisal, Memarian, Farzan, Wang, Fay, Jia, Fei, Soares, Felipe, Frujeri, Felipe Vieira, Chen, Feng, Lin, Fengguang, Galko, Ferenc, Sun, Frank, Siino, Frankie, Hou, Frida, Agam, Gal Hubara, Kaplun, Gal, Bhatt, Gantavya, Prasad, Gargi, Kulshreshtha, Garvit, Armstrong, George, Shen, Gerald, Borghesi, Giulio, Neskovic, Gordana, Batmaz, Gorkem, Lam, Grace, Mason, Greg, Pauloski, Greg, Nalbandyan, Grigor, Chlebus, Grzegorz, Karch, Grzegorz, Liu, Guan-Ting, Zhang, Guoming, Huang, Guyue, Maron, Haggai, Qian, Haifeng, Elisha, Haim, Ren, Haoxing, Kumar, Haran Kumar Shiv, Hud, Haribhau, Nover, Harris, Hall, Harrison Saturley, Iso, Hayate, Ngo, Helen, Hum, Herbert, Sahota, Herman, Wang, Hexin, Soni, Himanshu, Tamoyan, Hovhannes, Li, Hua, Chen, Huanhuan, Li, Hui, Wang, Hui, Nguyen, Huy, Chiles, Ian, Galil, Ido, Shahaf, Ido, Gitman, Igor, Shovkun, Igor, Loshchilov, Ilya, Guehring, Ingo, Schen, Itamar, Levy, Itay, Neeman, Itay, Moshkov, Ivan, Golan, Izik, Putterman, Izzy, Choi, Jaemin, Slowikowski, Jakub, Kautz, Jan, Scowcroft, Jane Polak, Casper, Jared, Mitra, Jatin, Glick, Jeffrey, Chen, Jenny, Oliver, Jesse, Xu, Jiacheng, Zhu, Jiafan, Song, Jialin, Zhang, Jian, Jiao, Jiantao, Zeng, Jiaqi, Lou, Jie, King, Jim, Zhang, Jimmy, Wang, Jingquan, Choi, Jinhang, Chu, Jinju, Conway, Joey, Guman, Joey, Jatko, Johan, Rausch, Johannes, Kamalu, John, Roberts, John, Greco, Johnny, Mensel, Johnny, Alben, Jonah, Yang, Jonas, Cohen, Jonathan, Raiman, Jonathan, Jennings, Joseph, Mabry, Joshua, Pierce, Joshua, Daw, Joyjit, Vialard, Julien Veron, Yi, Junkeun, Parmar, Jupinder, Jain, Kajal, Zhu, Kan, Briski, Kari, Cheung, Katherine, Luna, Katherine, Willowhawk, Keith, Wyss, Keith, Santhanam, Keshav, Shih, Kevin, Kong, Kezhi, Nguyen, Khanh, Bhardwaj, Khushi, Sivamani, Kirthi Shankar, Krommydas, Konstantinos, Puvvada, Krishna C., Pawelec, Krzysztof, Anik, Kumar, Keprios, Kyle, Day, Kylie, McAfee, Lawrence, Du, Leo, Derczynski, Leon, Ding, Li, Liu, Linda, Wu, Lingjie, Kadoch, Lior, Wei, Lizzie, Vega, Luis, Robison, Luke, Su, Lun, Van Segbroeck, Maarten, Mikulski, Maciej Jakub, de Melo, Maer Rodrigues, Sypula, Magda, Fathi, Mahan, Sreedhar, Makesh Narsimhan, Chandran, Makesh Tarun, Kilaru, Manoj, Ashkenazi, Maor, Cuevas, Marc, Romeijn, Marc, Chochowski, Marcin, Cai, Mark, Mozolewski, Mark, Kliegl, Markus, Stepniewska-Dziubinska, Marta, Patelka, Martyna, Machczynski, Mattei, Novikov, Matvei, Ferrato, Mauricio, Golub, Maximilian, Samadi, Mehrzad, Corpuz, Melissa, Wang, Mengru, Wu, Mengxi, Price, Meredith, Boubdir, Meriem, Schaffer, Micah, Andersch, Michael, Boone, Michael, Gschwind, Michael, Lightstone, Michael, Loh, Michael, Bien, Michal, Zawalski, Michal, Gill, Michelle, Martinez, Miguel, Khona, Mikail, Chrzanowski, Mike, Houston, Mike, Ma, Mingyuan, Lee, Minseok, Fawzy, Mohamed, Dabbah, Mohammad, Shoeybi, Mohammad, Patwary, Mostofa, Mulepati, Nabin, Nabwani, Najeeb, Dhameja, Namit, Hennouni, Narimane, Hereth, Natalie, Pinckney, Nathaniel, Algarici, Nave, Assaf, Nave, Haber, Netanel, Knight, Nicholas, Reamaroon, Nick, Quak, Nickson, Bhatia, Nidhi, Desai, Nikhil, Ludwig, Nikolai, Tajbakhsh, Nima, Xu, Ning, Ailon, Nir, Juluru, Nirmal, Nitin, Nitin, Masad, Ofri, Rybakov, Oleg, Hrinchuk, Oleksii, Kuchaiev, Oleksii, Viessmann, Olivia, Delalleau, Olivier, Olabiyi, Oluwatobi, Argov, Omer Ullman, Puny, Omri, Tropp, Oren, Ribalta, Pablo, Bhattacharya, Pallab, Lampropoulos, Panos, Mannan, Parth, Shamis, Pasha, Legresley, Patrick, Gibbons, Paul, Molchanov, Pavlo, Morkisz, Pawel, Dykas, Peter, Jin, Peter, Aquilanti, Pierre-Yves, Xu, Pinky, Januszewski, Piotr, Laskiewicz, Piotr, Jannaty, Pooya, Gurumurthy, Prakash, Thombre, Pranav Prashant, Varshney, Prasoon, Gundecha, Pritam, Tredak, Przemek, Meng, Puhui, Wan, Qiyu, Mahabadi, Rabeeh Karimi, Oberman, Rachel, Garg, Rachit, Sri-Tharan, Radha, Kandu, Rahul, Sanadhya, Rakshit, El-Yaniv, Ran, Zilberstein, Ran, Shafipour, Rasoul, Macalisang, Ray, Tian, Rayen, Kovacs, Reka, Pi, Renjie, Izzo, Rick, Shahbazyan, Rima, Garg, Rishabh, Puri, Rishi, Neves, Rita Fernandes, Zhao, Ritchie, Borkar, Ritika, Gala, Ritu, Islam, Riyad, Clark, Robert, Hesse, Robert, Kirby, Robert, Waleffe, Roger, Watve, Rohit, Koren, Roi, Banner, Ron, Zhang, Ruoxi, Hewett, Russell J., Prenger, Ryan, Stewart, Ryan, Egashira, Ryota, Mahdavi, Sadegh, Paliwal, Saee, Singh, Sagar, Modi, Sahil, Dave, Salika, Shinagawa, Samantha, Kriman, Samuel, Bhaskar, Sandip, Lym, Sangkug, Kariyappa, Sanjay, Satheesh, Sanjeev, Murari, Saran Vikas, Pasumarthi, Satish, Mishra, Saurabh, Muralidharan, Saurav, Hara, Scott, Narentharen, Sean, Anandaraj, Selvaraj, Na, Seonjin, Bak, Seonmeyong, Bak, Seonmyeong, Sameni, Sepehr, Mard, Seph, Panev, Serge, Henneman, Seth, Poulos, Seth, Mor, Shahar, Acharya, Shantanu, Ghosh, Shaona, Sreenivas, Sharath Turuvekere, Mendelson, Sharon, Kotek, Shaun, Wang, Shawn, Aharon, Shay, Gharghabi, Shaya, Lin, Sheng-Chieh, Chen, Shi, Fan, Shiqing, Baskaran, Shirish, Gopa, Shreya, Prabhumoye, Shrimai, Pachori, Shubham, Toshniwal, Shubham, Ding, Shuoyang, Krishnamurthy, Shwetha, Singh, Siddharth, Sun, Simeng, Das, Sirshak, Thottakara, Sivakumar Arayandi, Ithape, Smita, Majumdar, Somshubra, Singhal, Soumye, Singudasu, Sri Harsha, Bhuvanapalli, Sridhar, Veccham, Srimukh, Sergienko, Stas, Alborghetti, Stefania, Ge, Stephen, Rong, Su, Devare, Sugam Dipak, Rao, Sukrit, Barua, Sumeet Kumar, Ha, Sungsoo, Gai, Sunny, Gunasekar, Suriya, Panguluri, Suseella, Gupta, Suyog, Hinzburh, Sviataslau, Priyadarshi, Sweta, Akter, Syeda Nahida, Abramovich, Talor, Bui, Tan, Varshney, Tanay, Ter-Hovhannisyan, Tatevik, Ene, Teodor-Dumitru, Kong, Terry, Do, Thanh, Zhang, Tianhe, Moore, Tiffany, Blankevoort, Tijmen, Moon, Tim, Mitra, Tiyasa, Balough, Tom, Grzegorzek, Tomasz, Hliwiak, Tomasz, Asida, Tomer, Natan, Tomer Bar, Keren, Tomer, Ronen, Tomer, Salim, Tony, Wang, Tony, Rebedea, Traian, Konuk, Tugrul, Vashishth, Twinkle, Karpas, Udi, De, Ushnish, Noorozi, Vahid, Srinivasan, Venkat, Elango, Venmugil, Agrawal, Vibhor, Cui, Victor, Korthikanti, Vijay, Mehta, Vikas, Rao, Vinay, Wu, Virginia, Kurin, Vitaly, Lavrukhin, Vitaly, Anisimov, Vladimir, Pham, Vu, Jiang, Wanli, Ahmad, Wasi Uddin, Ishihara, Wataru, Du, Wei, Ping, Wei, Chai, Weiheng, Dai, Wenliang, Helmholz, Wesley, Jennings, Will, Zhu, Will, Prazuch, Wojciech, Ren, Xiaowei, Yu, Xiwen, Breek, Yan, Chen, Yang, Yu, Yang, Chen, Yangyi, Galron, Yaniv, Karnati, Yashaswi, Choi, Yejin, Meyer, Yev, Wu, Yi-Fu, Zhang, Yian, Lin, Ying, Geifman, Yonatan, Fu, Yonggan, Kwon, Youngeun, Yao, Yu, Guvvla, Yugi, Huang, Yuki, Liu, Yunsheng, Moshe, Zach, Newell, Zachary, Wang, Zhilin, Li, Zhiyu, Zhu, Zhongbo, Yang, Zhuolin, Liu, Zihan, Yan, Zijie, Wertheimer, Zsolt-Alon

Abstract

We introduce Nemotron 3 Ultra, a 550 billion total and 55 billion active parameter Mixture-of-Experts Hybrid Mamba-Attention language model. We pre-trained Nemotron 3 Ultra on 20 trillion text tokens, then extended the context length to 1M tokens, and post-trained using Supervised Fine Tuning (SFT), Reinforcement Learning (RL), and Multi-teacher On-Policy Distillation (MOPD). Nemotron 3 Ultra is our most capable model yet, employing multiple key technologies - LatentMoE, Multi Token Prediction (MTP), NVFP4 pre-training, multi-environment RLVR, MOPD, and reasoning budget control. Nemotron 3 Ultra achieves up to ~6x higher inference throughput as compared to state-of-the-art publicly available LLMs while attaining on-par accuracy. The state-of-the-art accuracy, high inference throughput, and 1M token context length make Nemotron 3 Ultra ideal for long-running autonomous agentic tasks. We open-source the base, post-trained, and quantized checkpoints, along with the training data and recipe on HuggingFace.

Chinese Translation

我们介绍了Nemotron 3 Ultra，这是一种拥有5500亿总参数和550亿活跃参数的混合专家混合Mamba-Attention语言模型。我们在20万亿文本标记上对Nemotron 3 Ultra进行了预训练，然后将上下文长度扩展到100万标记，并使用监督微调（Supervised Fine Tuning, SFT）、强化学习（Reinforcement Learning, RL）和多教师在线蒸馏（Multi-teacher On-Policy Distillation, MOPD）进行了后训练。Nemotron 3 Ultra是我们迄今为止最强大的模型，采用了多项关键技术——LatentMoE、多标记预测（Multi Token Prediction, MTP）、NVFP4预训练、多环境RLVR、MOPD和推理预算控制。与现有的公开可用大型语言模型（LLMs）相比，Nemotron 3 Ultra的推理吞吐量提高了约6倍，同时保持了相当的准确性。其卓越的准确性、高推理吞吐量和100万标记的上下文长度使Nemotron 3 Ultra非常适合长时间运行的自主代理任务。我们将在HuggingFace上开源基础模型、后训练模型和量化检查点，以及训练数据和配方。

View on arXiv Download PDF AI Translation

cs.CL / 7 / 2606.15017

Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents

在线技能和记忆模块是否总是值得其代币？一项预算受限的网络代理研究

Hajimiri, Sina, Aminbeidokhti, Masih, Dolz, Jose, Ayed, Ismail Ben, Laradji, Issam H., Gella, Spandana, Gontier, Nicolas

Abstract

Online web agents often augment a base actor with memory, workflow, or skill modules. These modules can improve performance, but they also consume test-time tokens, a cost rarely reported alongside the actor's inference cost. We study online augmentation, where this overhead is paid on every task, and re-evaluate its benefits under a fixed total inference budget. We compare AWM, ASI, and ReasoningBank with a token-matched vanilla baseline that uses the same budget for additional actor steps. Across three WebArena domains and three models, Gemini 3 Flash, GPT-5.4-mini, and Qwen 3.6-27B, the vanilla baseline matches or surpasses all three augmentation methods in aggregate success rate while often using fewer total tokens. We observe a similar trend on WorkArena-L1 with Qwen 3.6-27B, indicating that the effect extends to enterprise knowledge-work tasks. Our results suggest that skills and workflow memory can be useful in specific domains, but their apparent gains often vanish against a budget-matched actor. We further show that run-to-run variance materially affects outcomes and should be reported as a core evaluation criterion for online web agents.

Chinese Translation

在线网络代理通常通过记忆、工作流或技能模块增强基础演员。这些模块可以提高性能，但它们也会消耗测试时间的代币，这一成本很少与演员的推理成本一起报告。我们研究在线增强，其中每个任务都需要支付这一额外费用，并在固定的总推理预算下重新评估其收益。我们将 AWM、ASI 和 ReasoningBank 与一个代币匹配的基础基线进行比较，该基线在额外的演员步骤上使用相同的预算。在三个 WebArena 领域和三个模型（Gemini 3 Flash、GPT-5.4-mini 和 Qwen 3.6-27B）中，基础基线在整体成功率上与所有三种增强方法相匹配或超越，同时通常使用更少的总代币。我们在 WorkArena-L1 上使用 Qwen 3.6-27B 观察到类似的趋势，表明这一效应扩展到企业知识工作任务。我们的结果表明，技能和工作流记忆在特定领域可能是有用的，但它们的明显收益在与预算匹配的演员面前往往会消失。我们进一步表明，运行间的方差对结果有实质性影响，应作为在线网络代理的核心评估标准进行报告。

View on arXiv Download PDF AI Translation

cs.CL / 8 / 2606.15026

Deep Temporal Modeling and Ensemble Fusion for Multimodal Emotion Recognition from Physiological Signals

基于深度时间建模和集成融合的多模态生理信号情感识别

Hagos, Desta Haileselassie, Aryal, Saurav Keshari, Ymele-Leki, Patrick, Andy, Anietie, Burge, Legand L.

Abstract

Physiological stress and emotion recognition are important for health monitoring and affective computing. In this work, we present a comprehensive evaluation of deep learning models such as Long Short-Term Memory (LSTM), Temporal Convolutional Networks (TCN), and Transformer on the WESAD dataset for multimodal affect recognition using wrist and chest sensor signals. We perform ablation studies to assess the individual contributions of each modality by training models on wrist-only and chest-only inputs. In addition, we implement a late-fusion ensemble strategy that combines predictions from all three architectures trained on multimodal input. We also employ early fusion at the sensor level by concatenating wrist and chest signals before feeding them into each model. Our results show that Transformer models consistently achieve the highest accuracy in multimodal settings, while TCN models perform best in the wrist-only configuration. The ensemble method yields the highest overall accuracy (98.91 +/- 0.13%) and macro-F1 score (98.56 +/- 0.17%). These findings demonstrate the effectiveness of sensor fusion and ensemble-based fusion in developing robust systems for physiological emotion recognition.

Chinese Translation

生理压力和情感识别对于健康监测和情感计算至关重要。在本研究中，我们对深度学习模型进行了全面评估，包括长短期记忆网络（LSTM）、时间卷积网络（TCN）和变压器（Transformer），并在WESAD数据集上进行多模态情感识别，使用手腕和胸部传感器信号。我们通过对仅使用手腕和仅使用胸部输入的模型进行训练，进行消融研究以评估每种模态的个体贡献。此外，我们实现了一种晚期融合集成策略，将在多模态输入上训练的三种架构的预测结果进行组合。我们还在传感器级别实施了早期融合，通过在将手腕和胸部信号输入每个模型之前进行拼接。我们的结果表明，变压器模型在多模态设置中始终实现了最高的准确率，而TCN模型在仅使用手腕配置时表现最佳。集成方法的整体准确率最高（98.91 +/- 0.13%）和宏观F1分数（98.56 +/- 0.17%）。这些发现证明了传感器融合和基于集成的融合在开发生理情感识别的稳健系统中的有效性。

View on arXiv Download PDF AI Translation

cs.CL / 9 / 2606.15037

ReportQA: QA-Based Radiology Report Evaluation

ReportQA：基于问答的放射学报告评估

Shi, Yiming, Yang, Shaoshuai, Chen, Xi, Li, Haolin, Zhang, Hengyu, Jiang, Che, Wang, Kaiwen, Zhu, Xun, Xie, Dong, Wang, Fei, Dou, Dejing, Li, Miao, Wu, Ji

Abstract

Radiology report evaluation is essential for advancing automated report generation. Natural language generation metrics have limited clinical relevance. Clinical efficacy (CE) metrics evaluate important medical findings, but focus mainly on presence and cover only a limited set of entities. Due to heavy reliance on manual annotations, it is difficult for CE metrics to extend clinical entities or attributes. In clinical practice, radiology reports serve as a medium for information transfer. Clinicians use them to perform downstream diagnostic tasks without directly inspecting images. Based on this insight, we propose ReportQA, a clinical-related and flexible radiology report evaluation framework, supporting detailed quantitative analysis of radiology report generation systems. We first collect datasets covering multiple imaging modalities and anatomical regions. We then construct knowledge trees of clinical entities and attributes with radiologist guidance, and use large language models (LLMs) to extract structured information from raw reports. Next, we generate QA pairs from predefined templates and apply quality control through self-filtering and report-based filtering. During evaluation, the report is treated as context, and an LLM acts as a judge model to answer the QA pairs. Based on the resulting QA accuracy, we introduce QAScore metric. Compared with existing metrics, QAScore shows better alignment with radiologist judgments. Experiments on multiple state-of-the-art vision-language models reveal that current report-based inference paradigms struggle to learn fine-grained clinical representations and exhibit strong negative prior biases. In contrast, question-driven inference provides a more effective alternative. For reproducibility and extensibility, we release the knowledge trees, structured reports, and QA pairs, along with the pipeline code for QA construction and evaluation.

Chinese Translation

放射学报告评估对于推进自动报告生成至关重要。自然语言生成指标在临床上的相关性有限。临床有效性（CE）指标评估重要的医学发现，但主要关注存在性，并仅涵盖有限的实体集。由于对手动注释的高度依赖，CE指标难以扩展临床实体或属性。在临床实践中，放射学报告作为信息传递的媒介。临床医生利用这些报告执行下游诊断任务，而无需直接检查图像。基于这一洞察，我们提出了ReportQA，一个与临床相关且灵活的放射学报告评估框架，支持对放射学报告生成系统的详细定量分析。我们首先收集涵盖多种成像方式和解剖区域的数据集。然后，在放射科医生的指导下构建临床实体和属性的知识树，并使用大型语言模型（LLMs）从原始报告中提取结构化信息。接下来，我们从预定义模板生成问答对，并通过自我过滤和基于报告的过滤进行质量控制。在评估过程中，报告被视为上下文，LLM作为评判模型回答问答对。基于得到的问答准确性，我们引入了QAScore指标。与现有指标相比，QAScore与放射科医生的判断表现出更好的一致性。在多个最先进的视觉-语言模型上的实验表明，当前基于报告的推理范式难以学习细粒度的临床表征，并表现出强烈的负先验偏见。相比之下，基于问题的推理提供了更有效的替代方案。为了可重复性和可扩展性，我们发布了知识树、结构化报告和问答对，以及用于问答构建和评估的管道代码。

View on arXiv Download PDF AI Translation

cs.CL / 10 / 2606.15044

Equity with Efficiency: An Empirical Study of Tokenizers for Multilingual Large Language Models

效率与公平：多语言大型语言模型的分词器实证研究

Lee, Kieron Seven Jun Wei, Qorib, Muhammad Reza, Soegeng, Andrew Ivan, Ng, Hwee Tou

Abstract

Multilingual large language models (LLMs) depend on subword tokenization to bridge discrete text and continuous neural representation. State-of-the-art multilingual LLMs often use Byte-level Byte-Pair Encoding (BPE) tokenizers that structurally favor high-resource languages and Latin scripts. For speakers of underrepresented languages, particularly those across Southeast Asia, this bias inflates inference costs and widens cross-lingual capability gaps. We present the first systematic comparison of equitable tokenizers on a unified benchmark spanning 11 Southeast Asian languages. Beyond tokenizer-level analysis of compression efficiency and cross-lingual equity, we assess downstream task performance through controlled 1.5B-parameter language model training using the same training data. Our results show that Parity-aware BPE lies on the Pareto frontier of the efficiency-equity trade-off, achieving strong compression parity at competitive cost. Morphology-Driven Byte Encoding delivers the best semantic reasoning performance through morphologically richer representations, albeit at a higher computational expense. Byte Latent Transformer underperforms on downstream tasks, possibly because its architectural assumptions misalign with the constraints of limited low-resource training data. Together, our findings demonstrate that cross-lingual fairness and tokenization efficiency are not fundamentally at odds, and offer practical guidance for designing equitable multilingual models.

Chinese Translation

多语言大型语言模型（LLMs）依赖子词分词技术来桥接离散文本与连续神经表示。当前最先进的多语言LLMs通常使用字节级字节对编码（Byte-Pair Encoding, BPE）分词器，这在结构上偏向高资源语言和拉丁文字。对于那些使用较少代表性语言的用户，尤其是东南亚地区的用户，这种偏见增加了推理成本并扩大了跨语言能力差距。我们首次在一个统一的基准上对11种东南亚语言的公平分词器进行了系统比较。除了对分词器在压缩效率和跨语言公平性方面的分析外，我们还通过使用相同训练数据的1.5B参数语言模型训练评估下游任务性能。我们的结果表明，考虑公平性的BPE位于效率与公平性权衡的帕累托前沿，在竞争成本下实现了强大的压缩公平性。形态驱动字节编码（Morphology-Driven Byte Encoding）通过形态上更丰富的表示提供了最佳的语义推理性能，尽管计算开销较高。字节潜在变换器（Byte Latent Transformer）在下游任务中的表现不佳，可能是因为其架构假设与有限低资源训练数据的约束不匹配。总的来说，我们的研究结果表明，跨语言公平性与分词效率并非根本对立，并为设计公平的多语言模型提供了实用指导。

View on arXiv Download PDF AI Translation

cs.CL / 11 / 2606.15059

A Practical Evaluation Method for Long-Form Simultaneous Speech-to-Speech Translation

一种用于长形式同步语音到语音翻译的实用评估方法

Xue, Yulin, Ouyang, Siqi, Li, Lei

Abstract

Simultaneous speech-to-speech translation (SimulS2ST) enables real-time cross-lingual communication, but existing evaluation has focused largely on short or pre-segmented speech rather than long-form, continuous input. Prior approaches are difficult to reproduce and make assumptions that do not hold for end-to-end systems. We present a practical evaluation method for long-form SimulS2ST. Given source speech, pre-segmented source transcripts, and reference translations, we run automatic speech recognition (ASR) and forced alignment on the generated target speech to recover token-level timestamps, then apply a sentence-embedding-based aligner to match the target text to its corresponding source sentences. This enables sentence-level computation of latency and quality metrics, including YAAL and xCOMET, which are then aggregated into final system-level scores. Experiments on representative SimulS2ST systems show that the method is effective in practice and reveal that current systems suffer from substantial latency accumulation on long speech.

Chinese Translation

同步语音到语音翻译（SimulS2ST）实现了实时跨语言交流，但现有的评估主要集中在短语音或预分段语音上，而非长形式的连续输入。以往的方法难以复现，并且做出了一些不适用于端到端系统的假设。我们提出了一种用于长形式SimulS2ST的实用评估方法。给定源语音、预分段的源转录文本和参考翻译，我们对生成的目标语音进行自动语音识别（ASR）和强制对齐，以恢复标记级时间戳，然后应用基于句子嵌入的对齐器，将目标文本与其对应的源句子匹配。这使得我们能够在句子级别计算延迟和质量指标，包括YAAL和xCOMET，然后将其汇总为最终的系统级评分。在具有代表性的SimulS2ST系统上的实验表明，该方法在实践中是有效的，并揭示了当前系统在长语音上存在显著的延迟累积问题。

View on arXiv Download PDF AI Translation

cs.CL / 12 / 2606.15069

CoCoGEC: Counterfactual Generation for Robust Grammatical Error Correction

CoCoGEC：用于稳健语法错误纠正的反事实生成

Wang, Qianyu, Wang, Xiaoman, Liang, Yuanyuan, Li, Xinyuan, Lan, Yunshi

Abstract

Grammatical error correction (GEC) systems are usually trained and evaluated on GEC benchmarks, but their performance often drops sharply once the surrounding context is slightly perturbed or extended. This indicates that the existing GEC models usually fail to understand the error patterns in the varying contexts. In this paper, we thoroughly investigate the counterfactuals for GEC tasks, where the subtle changes to the contexts could lead to the label flipping issue. We propose CoCoGEC, a counterfactual generation framework that creates copies of training instances with error-irrelevant contexts altered. Our framework systematically generates counterfactuals by (1) generating intra- and inter-sentence counterfactuals that maintain the error patterns as well as syntax of the original instances by altering the word-level and sentence-level contexts; (2) revising the generated counterfactuals by selecting the instances with flipped labels and high GEC Mutual Information (MI) coefficient. Extensive experiments show that our method substantially improves the stability of GEC models, outperforming a set of data augmentation baselines. Particularly, it could achieve absolute F0.5 gains of +9.9, +11.3, and +20.8 points on the perturbed BEA-19*,CoNLL-14*, and TEM-8* data set.Our code is released at https://github.com/Quinnok/CoCoGEC

Chinese Translation

语法错误纠正（GEC）系统通常在GEC基准上进行训练和评估，但一旦周围上下文稍有扰动或扩展，其性能往往会急剧下降。这表明现有的GEC模型通常无法理解在不同上下文中出现的错误模式。本文深入研究了GEC任务中的反事实，其中对上下文的细微变化可能导致标签翻转问题。我们提出了CoCoGEC，一个反事实生成框架，通过改变与错误无关的上下文来创建训练实例的副本。我们的框架通过以下方式系统地生成反事实：（1）生成保持原始实例的错误模式及语法的句内和句间反事实，通过改变词级和句级上下文；（2）通过选择标签翻转且GEC互信息（MI）系数高的实例来修正生成的反事实。大量实验表明，我们的方法显著提高了GEC模型的稳定性，超越了一系列数据增强基线。特别地，它在扰动的BEA-19*、CoNLL-14*和TEM-8*数据集上分别实现了+9.9、+11.3和+20.8的绝对F0.5增益。我们的代码已发布在https://github.com/Quinnok/CoCoGEC

View on arXiv Download PDF AI Translation

cs.CL / 13 / 2606.15070

Stop When Further Reasoning Won't Help: Attention-State Adaptive Generation in Reasoning Models

在推理模型中，当进一步推理无助时停止：注意力状态自适应生成

Li, Jiakai, Qin, Ke, Wang, Rongzheng, Ma, Yizhuo, Chen, Qizhi, Li, Muquan, Liang, Shuang

Abstract

By incorporating test-time compute scaling, large reasoning models (LRMs) can solve complex problems through explicit chain-of-thought (CoT) reasoning processes. However, they often suffer from overthinking, resulting in redundant token outputs and degraded accuracy. Current methods to mitigate this issue remain limited: training-based approaches require substantial computational resources, while training-free methods rely on well-crafted prompts or unreliable confidence signals. In this work, we investigate early stopping from the perspective of attention distributions and propose a simple method, ASAG, which infers the model's reasoning state and adaptively adjusts the generation strategy. The proposed framework is training-free and plug-and-play, enabling seamless integration into existing LRMs. Extensive experiments on nine benchmarks demonstrate consistent improvements across mainstream LRMs with varying parameter scales, including the DeepSeek-R1-Distill and Qwen3 series. Specifically, ASAG improves average accuracy by 3.2% while reducing the number of generated tokens by nearly 40% across all reasoning tasks on Qwen3-8B.

Chinese Translation

通过引入测试时计算规模调整，大型推理模型（LRMs）能够通过明确的思维链（CoT）推理过程解决复杂问题。然而，它们常常会出现过度思考，导致冗余的标记输出和准确性下降。目前缓解这一问题的方法仍然有限：基于训练的方法需要大量计算资源，而无训练的方法依赖于精心设计的提示或不可靠的信心信号。在本研究中，我们从注意力分布的角度探讨了早停，并提出了一种简单的方法ASAG，该方法推断模型的推理状态并自适应地调整生成策略。所提出的框架是无训练的，且可即插即用，能够无缝集成到现有的LRMs中。在九个基准上的大量实验表明，在不同参数规模的主流LRMs中，包括DeepSeek-R1-Distill和Qwen3系列，均实现了一致的性能提升。具体而言，ASAG在Qwen3-8B的所有推理任务中将平均准确率提高了3.2%，同时将生成的标记数量减少了近40%。

View on arXiv Download PDF AI Translation

cs.CL / 14 / 2606.15079

Ling and Ring 2.6 Technical Report: Efficient and Instant Agentic Intelligence at Trillion-Parameter Scale

Ling 和 Ring 2.6 技术报告：万亿参数规模下的高效即时智能代理

Li, Ang, Liu, Ben, Han, Bin, Hu, Bin, Jing, Bin, Hu, Binbin, Li, Bing, Chen, Cai, Tang, Caizhi, Tian, Changxin, Huang, Chao, Zhang, Chao, Liang, Chen, Qian, Chen, Tang, Chengfu, Wen, Chengyao, Fu, Chilin, Wu, Chunwei, Zhang, Cong, Peng, Cunyin, Wang, Daixin, Zhang, Dalong, Zhao, Deng, Jin, Dingnan, Zhu, Dingyuan, Zhang, Donghao, Yuan, Fan, Zhao, Fangzheng, Meng, Fanzhuang, Wu, Feifan, Xu, Feng, Fang, Fengbin, Wang, Gangshan, Yang, Guodong, Zhao, Hailin, Wang, Haitao, Zhang, Haitao, Zhang, Hanxiao, Wang, Hanzi, Dai, Hao, Liu, Hao, Qian, Hao, Wu, Hao, Liu, Haoxiong, Xu, Haoyu, Zhang, Heng, Liu, Hong, Zhang, Hongliang, Liu, Hongrui, Li, Hongxun, Ruan, Hongzhi, Xiong, Huaidong, Zheng, Huihuang, Tang, Huikang, Guo, Jia, Li, Jia, Liu, Jia, Wang, Jiameng, Liu, Jiaming, Shi, Jiannan, Wei, Jianping, Yang, Jiaolong, Wang, Jiapeng, Gao, Jie, Wang, Jie, Wu, Jiewei, Yang, Jin, Li, Jinjin, Huang, Jinjing, Sun, Jinquan, Chen, Jinyao, Tu, Juanhui, Liu, Jun, Mei, Jun, Xu, Jun, Zhou, Jun, Ou, Junjie, Sipan, Junnan, Fang, Junpeng, Zhang, Kaihong, Hu, Kaiqin, Shi, Ke, Xu, Kuan, Tang, Kun, Chen, Kunlong, Mei, Lanyin, Chen, Lei, Liang, Lei, Xu, Lei, Tang, Li, Jiang, Liang, Fu, Liangcheng, Zhang, Lihui, Shi, Linfeng, Ma, Lintao, Liu, Liyuan, Li, Longfei, Zheng, Longfei, Liu, Lu, Yu, Lu, Li, Man, Zhu, Meiqi, Li, Meng, Gao, Mengjie, Sun, Mengshu, Yin, Mingming, Zhang, Mingyang, Fan, Mingyuan, Xu, Nuo, Tang, Pan, Jiang, Peijie, Zhao, Peilong, Lin, Peng, Liu, Pingping, Zuo, Qi, Zhao, Qian, Cheng, Qiang, Cao, Qianggang, Bao, Qiaoben, Cui, Qing, Yang, Qingyuan, Shi, Qitao, Huang, Qiyin, Zhou, Qizheng, Wan, Quan, Zhao, Runyuan, Zheng, Shaomian, Wei, Shaowei, Zhang, Shengnan, Li, Shuaicheng, Li, Shujie, Zhang, Shuo, Bian, Sikang, Yao, Tianchu, Xu, Tiange, Wang, Tianshu, Guo, Ting, Wang, Tinghao, Huang, Tingwei, Zhao, Tong, Yang, Tongkai, Hong, Wang, Gu, Wanli, Lu, Wei, Wu, Weichang, Han, Weiguang, Li, Weiquan, Shen, Wenbo, Fang, Wenjing, Tang, Wenzhi, Shu, Xiang, Shi, Xiao, Yan, Xiaodong, Zhang, Xiaolu, Wan, Xiaopei, Sun, Xiaqing, Zhao, Xin, Lu, Xingyu, Yang, Xinxing, Tang, Xinyao, Kong, Xinyu, Liu, Xinyu, Xu, Xiong, Sun, Xuan, Han, Xudong, Wang, Xudong, Shen, Xujie, Zhang, Yalin, Hou, Yangyang, Ren, Yankun, Zhao, Yao, Chen, Ye, Chen, Yeyang, Cao, Yibo, Zuo, Yifan, Chen, Yijie, Li, Ying, Song, Yingjie, Li, Yingxue, Wang, Yiqi, Sun, Yixuan, Xiao, Yizhu, Xu, Yongfei, Liu, Yu, Fang, Yuchen, Gao, Yue, Yu, Yue, Zhang, Yue, Zhang, Yuqi, He, Yuxiao, Lu, Yuxiao, Tian, Yuxin, Li, Yuxuan, Fu, Yuzhuo, Xu, Zhankai, Huan, Zhaoxin, Zhang, Zhenduo, Gui, Zhengke, Huang, Zhengyu, Ma, Zhenjun, Pan, Zhenxuan, Qu, Zheping, Zhu, Zhibo, Fan, Zhidong, Huangfu, Zhigang, Wang, Zhihao, Zhang, Zhiqiang, Liu, Zhizhen, Zhou, Zhuyan, Lin, Zibin, Zeng, Zihang, Wang, Zihao, Wang, Zilong, Liu, Ziqi, Xuan, Zitao, Cheng, Zixuan, Wen, Zujie, Tang, Zuoli

Abstract

Efficient and scalable agentic intelligence requires models that can deliver both low-latency responses and strong reasoning capabilities while remaining practical to train, serve, and deploy. In this report, we present Ling-2.6 and Ring-2.6, a family of models designed to address this challenge at scale. Ling-2.6 is optimized for instant response generation and high capability per output token, whereas Ring-2.6 is tailored for deeper reasoning and more advanced agentic workflows. Instead of training from scratch, we upgrade the Ling-2.0 base model through architectural migration pre-training and large-scale post-training. This upgrade is guided by a unified co-design of model architecture, optimization objectives, serving systems, and agent training environments, enabling improvements in both model capability and deployment efficiency. At the architectural level, we introduce a hybrid linear attention design that integrates Lightning Attention with MLA, improving the efficiency of long-context training and decoding. To further enhance token efficiency, we optimize capability per output token through Evolutionary Chain-of-Thought, Linguistic Unit Policy Optimization, bidirectional preference alignment, and shortest-correct-response distillation. For agentic capabilities, we propose KPop, a reinforcement learning framework designed to support stable training of Ring-2.6-1T on large-scale environment-grounded data. KPop improves training efficiency through asynchronous scheduling across coding, search, tool use, and workflow execution, enabling scalable learning from complex agent-environment interactions. Together, Ling-2.6 and Ring-2.6 provide a practical pathway toward efficient, scalable, and open agentic systems. We open-source all checkpoints in the 2.6 family to support further research and development in practical agentic intelligence.

Chinese Translation

高效且可扩展的智能代理需要能够提供低延迟响应和强大推理能力的模型，同时在训练、服务和部署方面保持实用性。在本报告中，我们介绍了 Ling-2.6 和 Ring-2.6，这是一系列旨在应对这一挑战的模型。Ling-2.6 针对即时响应生成和每个输出标记的高能力进行了优化，而 Ring-2.6 则专门为更深层次的推理和更高级的智能代理工作流程量身定制。我们通过架构迁移预训练和大规模后训练来升级 Ling-2.0 基础模型，而不是从头开始训练。这一升级是通过模型架构、优化目标、服务系统和代理训练环境的统一共同设计来指导的，从而实现模型能力和部署效率的双重提升。在架构层面，我们引入了一种混合线性注意力设计，将 Lightning Attention 与 MLA 相结合，提高了长上下文训练和解码的效率。为了进一步增强标记效率，我们通过进化链式思维、语言单元策略优化、双向偏好对齐和最短正确响应蒸馏来优化每个输出标记的能力。对于智能代理能力，我们提出了 KPop，这是一种强化学习框架，旨在支持在大规模环境基础数据上稳定训练 Ring-2.6-1T。KPop 通过在编码、搜索、工具使用和工作流程执行之间的异步调度提高了训练效率，使得从复杂的代理-环境交互中进行可扩展学习成为可能。Ling-2.6 和 Ring-2.6 一起为高效、可扩展和开放的智能代理系统提供了实用的路径。我们开源了 2.6 系列中的所有检查点，以支持在实用智能代理领域的进一步研究和开发。

View on arXiv Download PDF AI Translation

cs.CL / 15 / 2606.15080

AdaMame: A Training Recipe for Adaptive Multilingual Reasoning

AdaMame：自适应多语言推理的训练方案

Ki, Dayeon, Duh, Kevin, Carpuat, Marine

Abstract

While Large Reasoning Models (LRMs) show strong performance in English, they often fail to reason in the language of the query, a phenomenon known as language collapse. Existing RL-based fixes typically add a binary language fidelity reward to the accuracy objective, yet still incur trade-off in accuracy, mid-trace code-switching, and excessive token usage. In this work, we propose AdaMame, a two-stage training recipe for multilingual mathematical reasoning that addresses these limitations by adaptively aligning the reasoning language to the query language without compromising accuracy. The first SFT stage fine-tunes on naturally occurring reasoning traces across five languages to establish multilingual reasoning capability. In the subsequent RL stage, we introduce AdaMame-GRPO, an adaptation of Group Relative Policy Optimization (GRPO) in which a query-conditioned alignment factor grows progressively during training, guiding the model to first explore diverse reasoning languages before exploiting reasoning in the query language. Evaluated across two benchmarks, two LRMs, and 12 languages, AdaMame-GRPO achieves Pareto-optimal performance across reasoning accuracy, language fidelity, and token efficiency over all baselines, with the strongest gains on out-of-domain, lower-resource languages.

Chinese Translation

尽管大型推理模型（LRMs）在英语中的表现强劲，但它们在查询语言中进行推理时常常失败，这一现象被称为语言崩溃。现有的基于强化学习（RL）的解决方案通常在准确性目标中添加一个二元语言保真度奖励，但仍然会在准确性、中途代码切换和过度使用标记方面产生权衡。在本研究中，我们提出了AdaMame，一种针对多语言数学推理的两阶段训练方案，通过自适应地将推理语言与查询语言对齐而不妥协准确性，从而解决这些局限性。第一阶段的监督微调（SFT）在五种语言中自然发生的推理轨迹上进行微调，以建立多语言推理能力。在随后的强化学习阶段，我们引入了AdaMame-GRPO，这是对组相对策略优化（Group Relative Policy Optimization, GRPO）的改编，在该阶段中，查询条件的对齐因子在训练过程中逐渐增长，引导模型首先探索多样的推理语言，然后再利用查询语言进行推理。在两个基准、两个大型推理模型和12种语言的评估中，AdaMame-GRPO在推理准确性、语言保真度和标记效率方面实现了帕累托最优性能，相较于所有基线在领域外、低资源语言上取得了最显著的提升。

View on arXiv Download PDF AI Translation

cs.CL / 16 / 2606.15121

When Cognitive Graphs Meet LLMs: BDEI Cognitive Pathways for Panic Emotional Arousal Prediction

当认知图谱遇上大型语言模型：用于恐慌情绪唤起预测的BDEI认知路径

Liu, Mengzhu, Qin, Long, Ai, Chuan, Zhu, Zhengqiu, Liang, Hongru, Gao, Chen, Li, Yong, Lu, Xin, Yin, Quanjun

Abstract

Predicting individual panic emotional arousal timing before manifestation is essential for proactive emergency intervention. Existing methods incorporate cognitive elements but none explicitly model the emotional arousal process, making them ill-suited for emotional arousal timing prediction. We argue that grounding prediction in appraisal emotion theory is necessary because it explicitly models this process, but three problems must be solved. (1) Appraisal theory posits that emotion arises from simultaneous evaluation across multiple threat dimensions, yet no prior work fuses these inputs into risk perception. (2) Existing cognitive models lack an Emotion node, decoupling threat appraisal from emotional arousal and forcing emotions to be inferred indirectly from behaviors. (3) Given their generalizable cognitive reasoning, current approaches adopt LLMs as the primary decision-maker, yet overlook the fragility and hallucination-proneness of their outputs. To address these issues, we introduce PanicCognitivePath (PCP), a framework that addresses all three. A Psychological Safety Distance (PSD) model, grounded in psychological distance theory, maps four-domain signals into a unified risk metric as the entry condition for subsequent cognitive reasoning. An explicit Emotion node grounded in appraisal emotion theory is introduced into BDI, forming a Belief-Desire-Emotion-Intention (BDEI) pathway. Agents whose risk metric exceeds the PSD threshold enter this pathway, coupling threat appraisal directly to emotional arousal. The BDEI pathway governs all state transitions while the LLM is confined to parameter estimation for the Belief-to-Desire transition, confining hallucinations to a single step and preventing error propagation. Experiments on Hurricane Sandy show PCP improves arousal timing accuracy by 10.68% over baselines, reduces peak count error to 7.07%.

Chinese Translation

在情绪表现之前预测个体的恐慌情绪唤起时机对于主动应急干预至关重要。现有方法虽然融入了认知元素，但没有明确建模情绪唤起过程，因此不适合用于情绪唤起时机的预测。我们认为，将预测基于评估情绪理论是必要的，因为它明确建模了这一过程，但必须解决三个问题。(1) 评估理论认为情绪源于对多个威胁维度的同时评估，但之前的工作没有将这些输入融合为风险感知。(2) 现有的认知模型缺乏情绪节点，将威胁评估与情绪唤起解耦，迫使情绪只能通过行为间接推断。(3) 鉴于其可推广的认知推理，当前方法将大型语言模型（LLMs）作为主要决策者，但忽视了其输出的脆弱性和幻觉倾向。为了解决这些问题，我们提出了PanicCognitivePath（PCP），一个解决所有三个问题的框架。基于心理距离理论的心理安全距离（PSD）模型将四个领域的信号映射为统一的风险指标，作为后续认知推理的入门条件。我们在BDI中引入了一个基于评估情绪理论的明确情绪节点，形成了信念-欲望-情绪-意图（BDEI）路径。风险指标超过PSD阈值的代理进入该路径，将威胁评估直接与情绪唤起相结合。BDEI路径控制所有状态转换，而LLM仅限于信念到欲望转换的参数估计，将幻觉限制在单一步骤内，防止错误传播。对桑迪飓风的实验表明，PCP在唤起时机准确性上比基线提高了10.68%，并将峰值计数误差降低至7.07%。

View on arXiv Download PDF AI Translation

cs.CL / 17 / 2606.15144

PACUTE: Phonology-, Affix-, and Character-level Understanding of Tokens for Filipino

PACUTE：针对菲律宾语的音位、词缀和字符级别的标记理解

Montalan, Jann Railey, Africa, David Demitri, Layacan, Jimson Paulo, Flores, Richell Isaiah, De Leon, Ivan Yuri, Gamboa, Lance Calvin

Abstract

Large language models (LLMs) process text as sequences of subword tokens, which can obscure the character-level and morphological structure that underlies word formation. This limitation is most acute for languages with non-concatenative morphology, where standard tokenizers systematically misalign token boundaries with morpheme boundaries. We introduce PACUTE, a diagnostic benchmark of 4,600 tasks designed to evaluate morphological understanding in Filipino, a language characterized by productive infixation, reduplication, and diacritic-driven lexical distinctions that are typically absent from written text. PACUTE includes a hierarchical diagnostic framework of six compositional levels that localizes where morphological understanding breaks down. Evaluating open-weight LLMs and frontier commercial models, we find that open-weight models perform near chance on morpheme decomposition regardless of scale. Frontier models perform much better, often recovering individual affixes under contains-match scoring, but remain far below their character-level ceilings on compositional tasks of morpheme transformations and syllabification. These results identify productive morphological composition, rather than character access alone, as the persistent bottleneck for Filipino word-structure understanding.

Chinese Translation

大型语言模型（LLMs）将文本处理为子词标记的序列，这可能会掩盖构成单词的字符级和形态结构。这一局限性在具有非拼接形态的语言中尤为明显，标准的分词器系统性地将标记边界与语素边界错位。我们提出了PACUTE，这是一个包含4600个任务的诊断基准，旨在评估菲律宾语的形态理解，该语言以富有表现力的插入、重叠和由变音符号驱动的词汇区分为特征，这些特征通常在书面文本中缺失。PACUTE包括一个六个组成层级的分层诊断框架，以定位形态理解失效的具体环节。在评估开放权重的LLMs和前沿商业模型时，我们发现开放权重模型在语素分解任务上的表现接近随机，无论其规模如何。前沿模型的表现则要好得多，通常在包含匹配评分下能够恢复单个词缀，但在语素变换和音节化的组成任务中仍远低于其字符级的上限。这些结果表明，富有表现力的形态组合，而非仅仅是字符访问，才是菲律宾语单词结构理解的持续瓶颈。

View on arXiv Download PDF AI Translation

cs.CL / 18 / 2606.15152

Can Agents Read the Room? Benchmarking Visual Social Intelligence in Multimodal Simulation

代理能读懂环境吗？多模态仿真中视觉社会智能的基准测试

Wan, Shijun, Wu, Xuehai, Zhang, Jiwen, Wang, Siyuan, Wei, Zhongyu

Abstract

Social interaction depends on both language and visible social signals, such as facial expressions, posture, gaze, and emotional shifts. Yet existing social-agent benchmarks are largely text-based and rarely test whether multimodal agents can use visual cues to guide interaction. We introduce \textsc{\benchmarkname{}}, a benchmark evaluating visual social intelligence in multimodal social simulation. It contains 240 scenarios, 585 role instances, and 2,340 role-task instances, combining aligned textual-visual evidence, structured role profiles, and four role-level tasks: expression task, characteristic task, interaction regulation task, and interaction outcome task. Evaluating seven recent MLLMs under verbalized-vision and direct-vision reveals a clear gap between local role enactment and interaction management: role-specific expression and conflict handling are near saturation, whereas interaction regulation and visually grounded outcome achievement remain substantially more difficult. The code is released at https://github.com/JunsWan/AgentViSS, and the dataset is available at https://huggingface.co/datasets/JunsWan/AgentViSS.

Chinese Translation

社会互动依赖于语言和可见的社会信号，如面部表情、姿势、目光和情感变化。然而，现有的社会代理基准主要基于文本，鲜有测试多模态代理是否能够利用视觉线索来指导互动。我们引入了 extsc{enchmarkname{}}，这是一个评估多模态社会仿真中视觉社会智能的基准。它包含240个场景、585个角色实例和2,340个角色任务实例，结合了对齐的文本-视觉证据、结构化的角色档案和四个角色级任务：表达任务、特征任务、互动调节任务和互动结果任务。在对七个近期的多模态语言模型（MLLMs）进行口头视觉和直接视觉评估时，发现局部角色执行与互动管理之间存在明显差距：角色特定的表达和冲突处理接近饱和，而互动调节和视觉基础的结果实现仍然显著更具挑战性。代码已发布在 https://github.com/JunsWan/AgentViSS，数据集可在 https://huggingface.co/datasets/JunsWan/AgentViSS 获取。

View on arXiv Download PDF AI Translation

cs.CL / 19 / 2606.15161

Beyond Layer Importance in Layer-wise Sparsity: An Inter-Layer Perturbation-Absorption Perspective

超越层重要性：层间扰动吸收视角下的层级稀疏性

Jing, Tao, Wu, Ningxin, Kang, Chen, Yu, Dong, Li, Changliang, Liu, Pengyuan

Abstract

The considerable layer-wise redundancy in large language models (LLMs) has established non-uniform sparsity allocation across layers as the standard pruning approach for efficient compression. Existing layer-wise allocation methods that estimate allocation strategy from local signals such as activation outliers or weight spectra mainly derive from local layer importance, whereas the final post-pruning performance is also influenced by the network's subsequent compensatory capacity. In this paper, we directly characterize this property through controlled perturbation experiments. We make the following empirical findings. First, layers exhibit highly heterogeneous responses to pruning-scale perturbations. In most cases, early layers amplify perturbations, while middle and late layers actively absorb them, with relative L2 drift decreasing monotonically across depth and direction realigning toward the unperturbed hidden-state trajectory. Second, absorption is a large-perturbation phenomenon. Under small perturbations the network exhibits amplification across all layers, and the transition to absorption occurs smoothly as perturbation magnitude grows to pruning scale. This enriches the linearized accumulation theory underlying related works. Building on these findings, we define an absorption coefficient per layer and propose absorption-aware correction, an orthogonal augmentation that improves OWL and AlphaPruning by reducing perplexity by 7.13% and boosting zero-shot accuracy by 1.02% across multiple model families at 70% sparsity.

Chinese Translation

大型语言模型（LLMs）中显著的层级冗余使得非均匀稀疏分配成为高效压缩的标准剪枝方法。现有的层级分配方法主要基于局部信号（如激活异常值或权重谱）来估计分配策略，这些方法主要源于局部层的重要性，而最终的剪枝后性能也受到网络后续补偿能力的影响。本文通过控制扰动实验直接表征了这一特性。我们的实证发现如下：首先，各层对剪枝规模扰动的响应高度异质。在大多数情况下，早期层会放大扰动，而中间层和后期层则积极吸收扰动，相对L2漂移在深度上单调递减，方向则重新对齐到未扰动的隐状态轨迹。其次，吸收是一个大扰动现象。在小扰动下，网络在所有层上表现出放大效应，而当扰动幅度增大到剪枝规模时，吸收的过渡平滑发生。这丰富了相关工作的线性累积理论。在这些发现的基础上，我们为每层定义了一个吸收系数，并提出了吸收感知校正（absorption-aware correction），这是一种正交增强方法，通过在70%稀疏率下将困惑度降低7.13%并提高多个模型系列的零-shot准确率1.02%，改善了OWL和AlphaPruning。

View on arXiv Download PDF AI Translation

cs.CL / 20 / 2606.15191

AmchiBias: Measuring Stereotypical Bias in Goan Identity Groups with a Minimal Pair Dataset in English and Konkani

AmchiBias：使用英语和孔卡尼语的最小对比数据集测量果阿身份群体中的刻板偏见

Barbosa, Michelle, Padó, Sebastian, Weeber, Franziska

Abstract

Socio-cultural stereotypical bias is an important consideration in the development and deployment of NLP systems. It is however often considered only at the national level, despite rich subnational socio-cultural structures. We present AmchiBias, the first benchmark for measuring socio-cultural stereotypical bias for the Indian state of Goa with its unique historically multicultural setting. It covers various Goan identity groups and comprises 313 minimal pairs across eight sociodemographic dimensions in both English and Devanagari Konkani. We then evaluate stereotypical bias in five multilingual encoder models on this benchmark. We find near-chance scores in Konkani, reflecting language incompetence for general multilingual models and a lack of Goan cultural competence for Indian language models. Queried in English, models with a stronger Indian language coverage show higher bias for pan-Indian groups than hyperlocal Goan groups. This suggests the English signal reflects pan-Indian pretraining associations rather than genuine Goan cultural knowledge. Our findings highlight a critical gap in low-resource multilingual NLP evaluation for hyperlocal community identities.

Chinese Translation

社会文化刻板偏见是自然语言处理（NLP）系统开发和部署中的一个重要考虑因素。然而，这一问题往往仅在国家层面上被关注，尽管存在丰富的次国家社会文化结构。我们提出了AmchiBias，这是第一个用于测量印度果阿州社会文化刻板偏见的基准，考虑到其独特的历史多元文化背景。该基准涵盖了多个果阿身份群体，并包含313个最小对在八个社会人口维度上，使用英语和天城文孔卡尼语进行比较。随后，我们在这一基准上评估了五个多语言编码模型的刻板偏见。我们发现，在孔卡尼语中模型的得分接近随机水平，这反映出一般多语言模型的语言能力不足以及印度语言模型对果阿文化的理解缺乏。当以英语查询时，覆盖更强的印度语言的模型对泛印度群体表现出更高的偏见，而对超本地的果阿群体则偏见较低。这表明，英语信号反映的是泛印度预训练关联，而非真实的果阿文化知识。我们的研究结果突显了在低资源多语言NLP评估中，针对超本地社区身份的关键缺口。

View on arXiv Download PDF AI Translation

cs.CL / 21 / 2606.15216

Spokes: Optimizing for Diverse Pretraining Data Selection

Spokes：优化多样化预训练数据选择

Lee, Clarence, Choi, Yejin, Zettlemoyer, Luke, Koh, Pang Wei, Chieu, Hai Leong

Abstract

Diversity plays a critical role in data selection, improving performance under fixed data budgets by reducing redundancy and repetition. However, optimizing for diversity is inherently challenging, as it is a set-level property that depends on interactions between data points rather than individual examples. As a result, existing approaches typically rely on proxies or approximations, which often fail to ensure sufficiently diverse subsets. In this work, we directly optimize diversity by introducing a probabilistic diversification framework based on the G-Vendi score, optimized via exponentiated gradient descent. Our method produces subsets that are substantially more diverse than those obtained via random sampling, achieving a +489 increase in G-Vendi score on a 500k-sample subset. We evaluate our approach on FineWeb and DCLM, where it consistently outperforms existing methods. Notably, SPOKES (diversity-only) improves average downstream performance by +0.4 and +0.5 points over random sampling on DCLM and FineWeb, respectively. More importantly, jointly optimizing for both quality and diversity yields the strongest results: SPOKES achieves gains of +1.5 and +1.4 points on DCLM and FineWeb, outperforming all baselines, including semantic deduplication and quality filtering.

Chinese Translation

多样性在数据选择中起着至关重要的作用，通过减少冗余和重复，在固定数据预算下提高性能。然而，优化多样性本质上具有挑战性，因为它是一个集体属性，依赖于数据点之间的相互作用，而不是单个示例。因此，现有的方法通常依赖于代理或近似，这往往无法确保足够多样化的子集。在本研究中，我们通过引入基于 G-Vendi 分数的概率多样化框架，直接优化多样性，该框架通过指数梯度下降进行优化。我们的方法生成的子集在多样性上显著优于通过随机抽样获得的子集，在 500k 样本子集上实现了 G-Vendi 分数的 +489 增加。我们在 FineWeb 和 DCLM 上评估了我们的方法，结果表明其始终优于现有方法。值得注意的是，SPOKES（仅多样性）在 DCLM 和 FineWeb 上分别比随机抽样提高了 +0.4 和 +0.5 的平均下游性能。更重要的是，联合优化质量和多样性产生了最强的结果：SPOKES 在 DCLM 和 FineWeb 上分别获得了 +1.5 和 +1.4 的提升，超越了所有基线，包括语义去重和质量过滤。

View on arXiv Download PDF AI Translation

cs.CL / 22 / 2606.15266

Evaluating and Preserving Lexical Stress in English-to-Chinese Speech-to-Speech Translation

评估和保留英语到汉语语音翻译中的词汇重音

Song, Yuchen, Chen, Xi, Li, Mingze, Nakamura, Satoshi

Abstract

Speech-to-speech translation (S2ST) systems have achieved impressive progress in semantic accuracy and speech naturalness. However, the cross-lingual transfer of lexical stress, a vital cue for emphasis and speaker intent, remains heavily underexplored, compounded by a lack of reliable automatic evaluation metrics for tonal languages like Chinese. We investigate English-to-Chinese S2ST stress transfer by constructing a stress-annotated Chinese dataset and an XLS-R-based Mandarin stress detector. Integrating this with the English EmphAssess system, we propose a novel objective metric for cross-lingual stress evaluation. Furthermore, we fine-tune CosyVoice3 to build a stress-aware S2ST system. Experiments demonstrate that our proposed S2ST architecture significantly outperforms existing systems in stress translation capability while maintaining competitive translation quality. Furthermore, our evaluation metric exhibits a strong correlation with human subjective judgments.

Chinese Translation

语音到语音翻译（S2ST）系统在语义准确性和语音自然性方面取得了显著进展。然而，词汇重音的跨语言转移作为强调和说话者意图的重要线索，仍然未得到充分探索，尤其是在缺乏针对汉语等声调语言的可靠自动评估指标的情况下。我们通过构建一个带有重音标注的汉语数据集和一个基于XLS-R的普通话重音检测器，研究英语到汉语的S2ST重音转移。将其与英语EmphAssess系统相结合，我们提出了一种新的跨语言重音评估的客观指标。此外，我们对CosyVoice3进行微调，以构建一个重音感知的S2ST系统。实验表明，我们提出的S2ST架构在重音翻译能力上显著优于现有系统，同时保持了竞争力的翻译质量。此外，我们的评估指标与人类主观判断之间表现出强相关性。

View on arXiv Download PDF AI Translation

cs.CL / 23 / 2606.15307

Adapting Reinforcement Learning with Chain-of-Thought Supervision for Explainable Detection of Hateful and Propagandistic Memes

通过链式思维监督调整强化学习以实现可解释的仇恨和宣传性表情包检测

Kmainasi, Mohamed Bayan, Kutlu, Mucahid, Shahroor, Ali Ezzat, Hasnat, Abul, Alam, Firoj

Abstract

Hateful and propagandistic memes exploit the interplay between images and text to convey harmful intent that neither modality reveals alone. Although thinking-based multimodal large language models (MLLMs) have advanced vision-language understanding, their application to meme content moderation remains underexplored. We propose a reinforcement learning-based post-training method that improves classification performance and reference-based explanation quality in thinking-based MLLMs via task-specific rewards and Group Relative Policy Optimization (GRPO). Concretely, we (i) conduct a systematic empirical study of off-the-shelf MLLMs for hateful and propagandistic meme understanding across English and Arabic benchmarks, (ii) extend existing meme datasets with weakly supervised chain-of-thought (CoT) rationales via distillation and multi-LLM fine-grained propaganda annotations, (iii) introduce a GRPO-based objective with thinking-length regularization that jointly optimizes classification accuracy and explanation quality, and (iv) investigate self-supervised GRPO on unlabeled memes using consensus-based pseudo-labels. Experiments on the Hateful Memes and ArMeme benchmarks show that our approach improves over previously reported results on FHM accuracy (up to +2.1%, from 79.9% to 82.0%) and on ArMeme macro-F1 (up to +7.6 points, from 0.536 to 0.612 with explanations; +6.1 compared to the original ArMeme benchmark), while also generating natural-language explanations. On ArMeme, sequence-classification baselines remain stronger in terms of raw accuracy, whereas our approach provides more balanced per-class performance along with explanations. We publicly release our code, data extensions, and evaluation resources.

Chinese Translation

仇恨和宣传性表情包利用图像与文本之间的相互作用传达单一模态无法揭示的有害意图。尽管基于思维的多模态大型语言模型（MLLMs）在视觉-语言理解方面取得了进展，但其在表情包内容审核中的应用仍然未被充分探索。我们提出了一种基于强化学习的后训练方法，通过任务特定的奖励和群体相对策略优化（GRPO）提高基于思维的MLLMs的分类性能和基于参考的解释质量。具体而言，我们（i）对现成的MLLMs在仇恨和宣传性表情包理解方面进行系统的实证研究，涵盖英语和阿拉伯语基准，（ii）通过蒸馏和多MLLM细粒度宣传注释扩展现有的表情包数据集，并引入弱监督的链式思维（CoT）推理，（iii）引入一个基于GRPO的目标，结合思维长度正则化，联合优化分类准确性和解释质量，以及（iv）使用基于共识的伪标签在未标记的表情包上研究自监督GRPO。在仇恨表情包和ArMeme基准上的实验表明，我们的方法在FHM准确性（提高最多2.1%，从79.9%提升至82.0%）和ArMeme宏F1（提高最多7.6分，从0.536提升至0.612，带有解释；与原始ArMeme基准相比提高6.1分）上优于之前报告的结果，同时生成自然语言解释。在ArMeme上，序列分类基线在原始准确性方面仍然更强，而我们的方法在每个类别的性能上提供了更平衡的表现以及解释。我们公开发布了我们的代码、数据扩展和评估资源。

View on arXiv Download PDF AI Translation

cs.CL / 24 / 2606.15325

Prior over Evidence: Stereotype-Driven Diagnosis in LLM-Based L2 Pronunciation Feedback

证据的先验：基于大语言模型的刻板印象驱动的二语发音反馈诊断

Wang, Rong, Sun, Kun

Abstract

Large language models are increasingly deployed for written pronunciation feedback in second-language (L2) English learning, under the assumption that their diagnoses are grounded in the supplied speech evidence rather than in priors from pretraining. This assumption is tested on 1,800 L2-Arctic utterances spanning six L1 backgrounds, three audio-capable LLMs, four pronunciation dimensions, and five evidence conditions ranging from a text-only baseline to numeric acoustic features and raw audio. Each (utterance x model x condition x dimension) cell is scored on three metrics: Rating Accuracy (RA) against gold labels, Evidence Coherence (EC) assessing internal consistency without ground truth, and Grounded Correctness (GC) evaluated against gold evidence. Results show three findings across models. First, rating accuracy and grounded reasoning decouple: 39.6% of judged cells contain internally coherent reasoning that supports a wrong rating, against only 15.8% where the reasoning supports a correct rating. Second, phoneme-level feedback converges to a fixed inventory of L2-English difficulty phones that recurs across all six L1 backgrounds and all evidence conditions. Third, acoustic evidence improves the rating only when the supplied feature directly probes the target dimension: textualised F0 range raises pitch-variation grounding from (0.18-0.19) to (0.45-0.62) across all three models, while stress and phoneme correctness, which require target-to-realisation alignment, remain ungrounded. The same audio waveform without textualised F0 values does not reproduce this improvement. These findings indicate that current general-purpose LLMs are more reliable as verbalisers of externally computed pronunciation evidence than as standalone diagnostic engines.

Chinese Translation

大型语言模型越来越多地被用于第二语言（L2）英语学习中的书面发音反馈，假设其诊断是基于提供的语音证据，而非预训练中的先验知识。本文在1800个L2-Arctic语句上测试了这一假设，这些语句涵盖了六种母语背景、三种具有音频能力的大语言模型、四个发音维度和五种证据条件，从仅文本的基线到数字声学特征和原始音频。每个（语句 x 模型 x 条件 x 维度）单元在三个指标上进行评分：与金标准标签的评分准确性（Rating Accuracy, RA）、评估内部一致性的证据一致性（Evidence Coherence, EC）以及与金证据对比的基础正确性（Grounded Correctness, GC）。结果显示出三个跨模型的发现。首先，评分准确性与基础推理解耦：39.6%的判断单元包含支持错误评分的内部一致推理，而仅有15.8%的推理支持正确评分。其次，音素级反馈收敛于一个固定的L2英语难度音素库，该库在所有六种母语背景和所有证据条件中反复出现。第三，声学证据仅在提供的特征直接探测目标维度时改善评分：文本化的F0范围将音高变化的基础从（0.18-0.19）提高到（0.45-0.62），而需要目标与实现对齐的重音和音素正确性则保持无基础。没有文本化F0值的相同音频波形无法再现这一改善。这些发现表明，当前的通用大语言模型在作为外部计算的发音证据的表述者时比作为独立的诊断引擎更可靠。

View on arXiv Download PDF AI Translation

cs.CL / 25 / 2606.15333

Replay What Matters: Off-Policy Replay for Efficient LLM Reinforcement Unlearning

重放重要内容：高效大规模语言模型强化学习遗忘的离线重放

Pang, Zirui, Zhang, Chenlong, Tan, Haosheng, Jin, Zhuoran, Wei, Jiaheng, Zhong, Zixin

Abstract

LLM unlearning has emerged as a cost-effective alternative to full retraining for removing hazardous knowledge from pretrained models while preserving general utility. Recent RL-based methods such as RULE reformulate unlearning as learning a refusal behavior, but their on-policy optimization repeatedly samples from the same forget and retain/boundary prompts throughout training. We identify a critical inefficiency in this process: easy cases quickly converge and provide little useful gradient signal, while hard cases near the forget/retain boundary continue to produce low-reward rollouts that are discarded after a single use. To address this issue, we propose ReRULE, an off-policy replay enhancement for reinforcement unlearning. ReRULE stores low-reward hard-case rollout groups in a replay buffer during early GRPO training and reuses them in later stages through importance-sampled off-policy updates, redirecting computation toward boundary cases that still require learning. Theoretically, we show that ReRULE yields a tighter hard-case convergence bound than pure on-policy RULE. Empirically, ReRULE improves MUSE-Books Retain Quality from 46.3 to 56.2 while adding only 5--11% training time across benchmarks. Its limited improvement on the simpler TOFU setting further supports the intended conditional behavior: replay is most beneficial when the hard/easy disparity is pronounced.

Chinese Translation

大规模语言模型（LLM）遗忘已成为一种经济高效的替代方案，用于从预训练模型中去除有害知识，同时保持一般实用性。近期基于强化学习（RL）的方法如RULE将遗忘重新表述为学习拒绝行为，但其基于策略的优化在训练过程中反复从相同的遗忘和保留/边界提示中抽样。我们识别出这一过程中的一个关键低效：简单案例迅速收敛，提供的有用梯度信号很少，而接近遗忘/保留边界的困难案例则继续产生低奖励的回放，这些回放在一次使用后被丢弃。为了解决这一问题，我们提出了ReRULE，一种用于强化学习遗忘的离线重放增强方法。ReRULE在早期的GRPO训练中将低奖励的困难案例回放组存储在重放缓冲区，并通过重要性抽样的离线更新在后期阶段重用这些回放，将计算重定向到仍需学习的边界案例。从理论上讲，我们证明ReRULE比纯基于策略的RULE具有更紧的困难案例收敛界限。从实证上看，ReRULE将MUSE-Books的保留质量从46.3提高到56.2，同时在各基准测试中仅增加了5%到11%的训练时间。在更简单的TOFU设置中，其有限的改进进一步支持了预期的条件行为：当困难/简单的差异明显时，重放最为有益。

View on arXiv Download PDF AI Translation

cs.CL / 26 / 2606.15335

Privacy-Preserving Text Sanitization for Distributed Agents Collaboration via Disentangled Representations

基于解耦表示的分布式代理协作的隐私保护文本清洗

Liu, Xuan, Zhou, Hefeng, Chen, Sicheng, Yang, Chao, Xu, Xingcheng, Qu, Jingjing, Lou, Jiong, LI, Jie, Hu, Xia

Abstract

When distributed agents exchange text across organizational boundaries, privacy leakage arises not only from explicit identifiers but also from distributional signatures such as formatting conventions, vocabulary choices, and syntactic patterns. We propose DiSan(Disentangled Sanitization), a privacy-preserving sanitization framework and a built-in component of Intern-Shannon for multi-agent collaboration. DiSan uses a two-stream encoder to factorize text into a source-invariant role subspace that preserves task semantics and a source-identifying style subspace that remains local. Federated proto-type alignment and adversarial regularization enable joint training without centralizing raw text. Experiments show that identifier-level masking is insufficient: masking 19.2% of tokens reduces TF-IDF stylometric attribution by only 18.6%. By contrast, DiSan reduces answer-level PII exposure by 20 times while maintaining 83% answer faithfulness on a distributed multi-agent RAG benchmark, and lowers Enron stylometric attribution by 73.2% under TF-IDF and 70.6% under a neural probe.

Chinese Translation

当分布式代理在组织边界之间交换文本时，隐私泄露不仅来自显式标识符，还来自格式约定、词汇选择和句法模式等分布特征。我们提出了DiSan（解耦清洗），这是一种隐私保护的清洗框架，也是多代理协作的Intern-Shannon的内置组件。DiSan使用双流编码器将文本分解为一个保持任务语义的源不变角色子空间和一个保持本地的源识别风格子空间。联邦原型对齐和对抗正则化使得在不集中原始文本的情况下进行联合训练成为可能。实验表明，仅仅进行标识符级别的掩蔽是不够的：掩蔽19.2%的标记仅减少了18.6%的TF-IDF风格归属。相比之下，DiSan在分布式多代理RAG基准上将答案级别的个人身份信息（PII）暴露降低了20倍，同时保持了83%的答案真实性，并在TF-IDF下将Enron风格归属降低了73.2%，在神经探测下降低了70.6%。

View on arXiv Download PDF AI Translation

cs.CL / 27 / 2606.15345

Beyond Monolingual Deep Research: Evaluating Agents and Retrievers with Cross-Lingual BrowseComp-Plus

超越单语深度研究：使用跨语言 BrowseComp-Plus 评估智能体和检索器

Lu, Yuheng, Zeng, Qingcheng, Qi, Heli, Yu, Puxuan, Zhao, Fuheng, Yang, Rui, Yanaka, Hitomi, Yokoya, Naoto, Xuan, Weihao

Abstract

Deep research agents are increasingly evaluated on their ability to search for evidence, reason over retrieved sources, and produce grounded answers. Existing browsing benchmarks, however, largely assume that the user's query and the supporting evidence are written in the same language, leaving open whether agentic search systems can operate when relevant evidence appears in another language. We introduce XBCP (Cross-lingual BrowseComp-Plus), a controlled benchmark that preserves the English question-and-answer space of BrowseComp-Plus but varies the languages of the supporting documents. XBCP instantiates two complementary settings: in the cross-lingual setting, each query is paired with evidence in a single assigned language. In the multilingual setting, the full evidence corpus is distributed equally and randomly across 12 languages spanning high-resource and low-resource regimes. We evaluate four deep research agents using sparse and dense multilingual retrievers, measuring answer accuracy, evidence recall, search behavior, calibration, citation fidelity, and oracle retrieval. Results reveal substantial degradation when evidence is translated. Even strong, dense retrievers lose evidence recall, and agents become less calibrated and cite evidence less reliably. Notably, accuracy remains lower even when all gold evidence is supplied directly. These findings suggest that cross-lingual deep research exposes both retrieval failures and an independent, agent-side difficulty in integrating language-mismatched evidence.

Chinese Translation

深度研究智能体越来越多地被评估其搜索证据、对检索来源进行推理以及生成有根据的答案的能力。然而，现有的浏览基准主要假设用户的查询和支持证据使用相同语言，这使得在相关证据以另一种语言出现时，智能搜索系统是否能够正常运作仍然是一个未解之题。我们引入了 XBCP（Cross-lingual BrowseComp-Plus），这是一个受控基准，保留了 BrowseComp-Plus 的英语问答空间，但改变了支持文档的语言。XBCP 实现了两种互补的设置：在跨语言设置中，每个查询与单一指定语言的证据配对；在多语言设置中，完整的证据语料库在 12 种语言之间均匀且随机分配，这些语言涵盖了高资源和低资源的领域。我们使用稀疏和密集的多语言检索器评估四个深度研究智能体，测量答案准确性、证据召回率、搜索行为、校准、引用忠实度和神谕检索。结果显示，当证据被翻译时，性能显著下降。即使是强大的密集检索器，证据召回率也会下降，智能体的校准度降低，引用证据的可靠性减弱。值得注意的是，即使所有的金证据直接提供，准确性仍然较低。这些发现表明，跨语言深度研究暴露了检索失败以及在整合语言不匹配证据时智能体面临的独立困难。

View on arXiv Download PDF AI Translation

cs.CL / 28 / 2606.15378

Rethinking the Role of Efficient Attention in Hybrid Architectures

重新思考高效注意力在混合架构中的角色

Qiao, Ziqing, Xu, Yinuo, Xiao, Chaojun, Su, Zhou, Zhou, Zihan, Chen, Yingfa, Xu, Xiaoyue, Han, Xu, Liu, Zhiyuan

Abstract

Modern language models increasingly adopt hybrid architectures that combine full attention with efficient attention modules, such as sliding-window attention (SWA) and recurrent sequence mixers. However, how these efficient modules shape model capabilities remains poorly understood. To address this gap, we conduct a systematic analysis across hybrid architectures from three perspectives: scaling behavior, mechanism analysis, and architecture design. First, from a scaling perspective, we find that efficient-attention design primarily affects how fast long-context capability emerges, while different hybrids eventually converge to comparable long-context performance under sufficient training. Second, mechanistically, we show that long-range retrieval is mainly carried by full attention, whereas efficient attention shapes its optimization trajectory. This explains a counter-intuitive phenomenon we call Large-Window Laziness: larger SWA windows can delay the formation of retrieval heads in full-attention layers. Third, guided by this mechanism, we show that applying NoPE to only the full-attention layers of a small-window SWA hybrid substantially improves long-context performance with negligible impact on short-context performance.

Chinese Translation

现代语言模型越来越多地采用混合架构，将全注意力与高效注意力模块相结合，例如滑动窗口注意力（Sliding-Window Attention, SWA）和递归序列混合器。然而，这些高效模块如何塑造模型能力仍然知之甚少。为了解决这一空白，我们从三个角度对混合架构进行了系统分析：扩展行为、机制分析和架构设计。首先，从扩展的角度来看，我们发现高效注意力设计主要影响长上下文能力的快速出现，而不同的混合架构在充分训练下最终会收敛到可比的长上下文性能。其次，从机制上看，我们表明长距离检索主要由全注意力承担，而高效注意力则塑造其优化轨迹。这解释了我们称之为“大窗口懒惰”（Large-Window Laziness）的反直觉现象：更大的SWA窗口可能会延迟全注意力层中检索头的形成。第三，在这一机制的指导下，我们展示了仅对小窗口SWA混合体的全注意力层应用NoPE可以显著提高长上下文性能，而对短上下文性能的影响微乎其微。

View on arXiv Download PDF AI Translation

cs.CL / 29 / 2606.15390

Not All Skills Help: Measuring and Repairing Agent Knowledge

并非所有技能都有帮助：测量和修复代理知识

Wang, Yixuan, Zhou, Yiyang, Liang, Yiming, Zhang, Congyu, Liu, Fuxiao, Zhou, Jiawei, Yao, Huaxiu

Abstract

LLM agents can improve without weight updates by accumulating natural-language skills from experience, but current systems entrust every decision about which skills to keep and how to apply them to LLM judgment alone. We argue that this conflates two distinct roles: generating a skill from experience is a creative act that judgment handles well, while deciding whether that skill actually helps requires empirical evidence across many tasks. Measuring per-skill causal contributions via randomized masking, we find that skill libraries exhibit pervasive causal heterogeneity: individual skills routinely help on some task types while hurting on others, yet their opposing effects cancel in aggregate, making them invisible to global curation methods. We propose ASSAY, a framework that separates generation from curation: it computes a per-skill causal attribution on a small development set, restructures the library offline, and suppresses skills with negative predicted effect for each test task. Across seven base models spanning four providers and two benchmarks (AppWorld and tau-bench), ASSAY consistently improves over prior skill-curation approaches. On AppWorld's hardest split, DeepSeek-V3 achieves 69.3% task-goal completion (47.4% relative improvement), a new state of the art among all published methods including weight-tuned approaches. On tau-bench retail, GPT-4.1 improves by 8.7% relative, advancing past o4-mini, o1, and GPT-4.5 on the public leaderboard without any weight modification. Ablation traces the dominant gain to per-task masking, confirming that the bottleneck is matching skills to tasks at inference time, not removing bad skills globally. Code is available at https://github.com/aiming-lab/assay.

Chinese Translation

大型语言模型（LLM）代理可以通过积累来自经验的自然语言技能而无需权重更新进行改进，但当前系统将关于保留哪些技能以及如何将其应用于LLM判断的每个决策都委托给LLM判断。我们认为这混淆了两个不同的角色：从经验中生成技能是一种创造性行为，而判断能够很好地处理这一过程，而决定该技能是否真的有帮助则需要在多个任务中获得实证证据。通过随机屏蔽测量每项技能的因果贡献，我们发现技能库表现出普遍的因果异质性：个别技能在某些任务类型上通常有帮助，而在其他任务上则有害，但它们的相反效应在总体上相互抵消，使其对全局策划方法不可见。我们提出了ASSAY，一个将生成与策划分开的框架：它在一个小的开发集上计算每项技能的因果归因，离线重组库，并抑制对每个测试任务具有负面预测效果的技能。在跨越四个提供者和两个基准（AppWorld和tau-bench）的七个基础模型中，ASSAY始终优于先前的技能策划方法。在AppWorld最难的分割上，DeepSeek-V3达到了69.3%的任务目标完成率（相对提高47.4%），在所有已发布的方法中，包括权重调优的方法，创造了新的最先进记录。在tau-bench零售中，GPT-4.1相对提高了8.7%，在公共排行榜上超越了o4-mini、o1和GPT-4.5，而没有进行任何权重修改。消融实验追踪到主要增益源于每个任务的屏蔽，确认瓶颈在于推理时将技能与任务匹配，而不是全局去除不良技能。代码可在https://github.com/aiming-lab/assay获取。

View on arXiv Download PDF AI Translation

cs.CL / 30 / 2606.15396

CHILLGuard: Towards Fine-Grained Chinese LLM Safety Guardrail with Scalable Data Construction and Model-aware Preference Alignment

CHILLGuard：面向细粒度中文大语言模型安全防护的可扩展数据构建与模型感知偏好对齐

Yu, Wenbo, Wang, Bohua, Fang, Hao, Gao, Kuofeng, Zeng, Jingru, Yang, Xiaochen, Zhang, Tianyi, Ma, Xiaoxiao, Kong, Jiawei, Wu, Hao, Chen, Bin, Xia, Shu-Tao, Zhang, Min

Abstract

Malicious content generated from large language models (LLMs) could pose severe safety risks and ethical concerns. While existing LLM safety guardrails excel in English or multilingual settings, they lack adaptation to Chinese-specific regulatory policies, cultural context and linguistic nuances, failing to support fine-grained risk classification for diverse deployment needs. In this paper, we introduce a 5-macro, 31-micro category fine-grained risk taxonomy for Chinese scenarios, and build CHILLGuard: a dedicated Chinese LLM content safety guardrail. To address the critical scarcity of high-quality annotated Chinese safety data, we propose a scalable multi-stage data construction pipeline: we expand multi-source corpus via retrieval-augmented generation, generate implicit harmful samples through prompt engineering rewriting, and refine high-quality data via multi-model voting-based label calibration. Based on this, we build CHILLGuardTrain, a large-scale training set with 405,007 samples, and CHILLGuardTest, a rigorously curated annotated test set with 51,745 samples. We then train CHILLGuard on CHILLGuardTrain under a generator-classifier collaborative framework via Model-aware Direct Preference Optimization. Extensive experiments under multiple settings demonstrate the state-of-the-art performance of CHILLGuard, e.g., a 15.92% improvement of F1 score over Qwen3Guard-8B-Strict on our benchmark. We will release our resources at https://github.com/cswbyu/CHILLGuard.

Chinese Translation

大型语言模型（LLMs）生成的恶意内容可能带来严重的安全风险和伦理问题。尽管现有的LLM安全防护在英语或多语言环境中表现出色，但它们缺乏对中国特定监管政策、文化背景和语言细微差别的适应性，无法支持针对多样化部署需求的细粒度风险分类。本文提出了一种针对中文场景的5个宏观类别和31个微观类别的细粒度风险分类法，并构建了CHILLGuard：一个专门针对中文LLM内容安全的防护系统。为了解决高质量标注中文安全数据的严重短缺问题，我们提出了一种可扩展的多阶段数据构建流程：通过检索增强生成扩展多源语料库，通过提示工程重写生成隐含有害样本，并通过多模型投票的标签校准精炼高质量数据。基于此，我们构建了CHILLGuardTrain，一个包含405,007个样本的大规模训练集，以及CHILLGuardTest，一个经过严格策划的包含51,745个样本的标注测试集。然后，我们在生成器-分类器协作框架下，通过模型感知直接偏好优化在CHILLGuardTrain上训练CHILLGuard。多种设置下的广泛实验表明，CHILLGuard的性能达到了最先进水平，例如，在我们的基准测试中，相较于Qwen3Guard-8B-Strict，F1分数提高了15.92%。我们将于 https://github.com/cswbyu/CHILLGuard 发布我们的资源。

View on arXiv Download PDF AI Translation

cs.CL / 31 / 2606.15405

T-Mem: Memory That Anticipates, Not Archives

T-Mem：预见而非存档的记忆

Guo, Weidong, Wang, Dakai, Wang, Zixuan, Liu, Hui, Xu, Yu

Abstract

Long-term memory is essential for conversational agents to remain coherent across extended dialogues, follow through on commitments made many sessions earlier, and adapt their behaviour to each user. Current LLM-backed long-term conversational memory, however, is reachability-bounded by the similarity between a query and stored content, both lexical and dense-vector. The approach is effective when query and memory share surface features such as wording or named entities (we call this descriptive). But it misses another, equally valuable class of cases, where query and memory do not share surface features and are tied only by a latent semantic arc (associative). On this regime prevailing long-term memory systems collectively fail. Covering this other half is what allows an assistant, for the first time, to actively draw on past dialogue as a semantic asset. On the memory side, this is the engineering counterpart of what cognitive science calls episodic future thinking: rehearsing past experience for the future contexts under which it will need to be found. We call these write-time rehearsals triggers. We propose T-Mem, the first long-term conversational memory architecture that covers both descriptive and associative recall. At each of two evidence granularities, single facts and full exchanges, T-Mem instantiates one descriptive trigger family and one associative trigger family, so that every memory remains reachable from both surface-similar and relevance-bound queries. As empirical validation, T-Mem reaches state-of-the-art on both LoCoMo and LoCoMo-Plus.

Chinese Translation

长期记忆对于对话代理在长时间对话中保持连贯性、遵循早前会话中做出的承诺以及根据每个用户调整其行为至关重要。然而，当前基于大型语言模型（LLM）的长期对话记忆受到查询与存储内容之间相似性的限制，包括词汇和密集向量。当查询和记忆共享表面特征（如措辞或命名实体）时，这种方法是有效的（我们称之为描述性）。但它忽略了另一类同样重要的情况，即查询和记忆不共享表面特征，仅通过潜在语义弧相连（关联性）。在这种情况下，现有的长期记忆系统普遍失效。覆盖这一另一半，使得助手首次能够主动利用过去的对话作为语义资产。在记忆方面，这与认知科学所称的情景未来思维相对应：为未来需要找到的上下文排练过去的经验。我们称这些写入时排练为触发器。我们提出了T-Mem，这是第一个涵盖描述性和关联性回忆的长期对话记忆架构。在两个证据粒度（单个事实和完整交流）中，T-Mem实例化一个描述性触发器家族和一个关联性触发器家族，使得每个记忆都可以通过表面相似和相关性绑定的查询进行访问。作为实证验证，T-Mem在LoCoMo和LoCoMo-Plus上达到了最先进的水平。

View on arXiv Download PDF AI Translation

cs.CL / 32 / 2606.15412

Few-Shot Biomedical Relation Extraction with Large Language Models: A Viable Alternative to Supervised Learning?

基于大语言模型的少样本生物医学关系提取：监督学习的可行替代方案？

Mraz, Jakob, Curk, Tomaž, Zupan, Blaž

Abstract

Biomedical relation extraction (BioRE) is a key step in transforming biomedical literature into structured knowledge. However, most existing approaches rely on supervised models trained on costly annotated datasets, limiting their scalability and adaptability across relation types and domains. We investigate few-shot BioRE using prompt-based learning with large language models (LLMs) and compare two task formulations: pairwise classification, which predicts relations for individual entity pairs, and joint generation, which extracts multiple relations in a single model call. Experiments on the BioREDirect dataset reveal a clear precision-recall trade-off. Pairwise classification achieves higher recall, whereas joint generation is more precise and computationally efficient. The best-performing model achieves a micro-F1 score of 0.44, substantially outperforming previous few-shot results (0.34) while remaining below the supervised baseline (0.56). Much of this gap is attributable to a single ambiguously defined relation type. When evaluated using macro-F1, which better captures performance across relation types in an imbalanced setting, prompt-based approaches outperform the supervised baseline (0.45 vs. 0.38), particularly on rare relation types. These findings highlight the potential of LLMs for BioRE in low-resource settings and underscore the importance of well-defined relation schemas.

Chinese Translation

生物医学关系提取（BioRE）是将生物医学文献转化为结构化知识的关键步骤。然而，大多数现有方法依赖于在昂贵的标注数据集上训练的监督模型，这限制了它们在关系类型和领域之间的可扩展性和适应性。我们研究了使用基于提示的学习进行的少样本BioRE，并比较了两种任务形式：成对分类，该方法为单个实体对预测关系；以及联合生成，该方法在一次模型调用中提取多个关系。在BioREDirect数据集上的实验揭示了明显的精确度-召回率权衡。成对分类实现了更高的召回率，而联合生成则更为精确且计算效率更高。表现最佳的模型达到了0.44的微F1分数，显著优于之前的少样本结果（0.34），但仍低于监督基线（0.56）。这一差距在很大程度上归因于一个定义模糊的关系类型。当使用宏F1进行评估时，该指标更好地捕捉了在不平衡设置中不同关系类型的表现，基于提示的方法超越了监督基线（0.45对0.38），尤其是在稀有关系类型上。这些发现突显了大语言模型在低资源环境下进行BioRE的潜力，并强调了明确关系模式的重要性。

View on arXiv Download PDF AI Translation

cs.CL / 33 / 2606.15416

Encode Errors: Representational Retrieval of In-Context Demonstrations for Multilingual Grammatical Error Correction

编码错误：多语言语法错误纠正的上下文示例的表征检索

Peng, Guangyue, Li, Wei, Luo, Wen, Wang, Houfeng

Abstract

Grammatical Error Correction (GEC) involves detecting and correcting the wrong usage of grammar. While large language models (LLMs) with in-context learning (ICL) capabilities have shown significant progress on various natural language processing (NLP) tasks, their few-shot performance on GEC remains suboptimal. This is mainly due to the challenge of retrieving suitable in-context demonstrations that capture error patterns instead of semantic similarity. In this paper, we demonstrate that LLMs can inherently capture information related to grammatical errors through their internal states. From these states, we extract the Grammatical Error Representation (GER), an informative and semantically neutral encoding of grammatical errors. Our novel GER-based retrieval method significantly boosts performance in ICL settings on multilingual GEC datasets, improving the precision of correction. For high-resource languages, our results on 8B-sized open-source models match those of closed-source models such as Deepseek2.5 and GPT-4o-mini. For low-resource languages, our $F_{0.5}$ scores surpass the baseline by up to a factor of 1.20. This method provides a more precise and resource-efficient solution for multilingual GEC, offering a promising direction for interpretable GEC research.

Chinese Translation

语法错误纠正（GEC）涉及检测和纠正语法的错误使用。虽然具有上下文学习（ICL）能力的大型语言模型（LLMs）在各种自然语言处理（NLP）任务上取得了显著进展，但它们在GEC上的少量样本表现仍然不尽如人意。这主要是由于检索适合的上下文示例的挑战，这些示例应捕捉错误模式而非语义相似性。本文展示了LLMs可以通过其内部状态固有地捕捉与语法错误相关的信息。我们从这些状态中提取了语法错误表征（GER），这是一种信息丰富且语义中立的语法错误编码。我们新颖的基于GER的检索方法显著提升了多语言GEC数据集上ICL设置的性能，提高了纠正的准确性。对于高资源语言，我们在8B规模的开源模型上的结果与Deepseek2.5和GPT-4o-mini等闭源模型相匹配。对于低资源语言，我们的$F_{0.5}$得分超过基线，提升幅度达到1.20倍。该方法为多语言GEC提供了更精确且资源高效的解决方案，为可解释的GEC研究提供了一个有前景的方向。

View on arXiv Download PDF AI Translation

cs.CL / 34 / 2606.15419

Let LLMs Judge Each Other: Multi-Agent Peer-Reviewed Reasoning for Medical Question Answering

让大型语言模型相互评判：用于医学问答的多智能体同行评审推理

Zhan, Zaifu, Zhou, Shuang, Zhang, Rui

Abstract

Objective: To enhance the accuracy, interpretability, and robustness of large language models (LLMs) in medical question answering (MedQA). Method: We designed a multi-agent peer-reviewed reasoning method in which multiple LLM agents independently generate chain-of-thought reasoning with candidate answers, then act as peer reviewers to evaluate each other's reasoning for factual correctness and logical soundness. The highest-rated reasoning chain is selected to produce the final answer. Experiments were conducted with five state-of-the-art LLMs (Llama-3.1-8B, Qwen2.5-7B, Phi-4, DeepSeek-LLM-7B, GPT-oss-20B) on three benchmark datasets: HeadQA, MedQA-USMLE, and PubMedQA. Performance was compared against single-model chain-of-thought reasoning and chain-of-thought-based majority voting. Results: Peer-reviewed reasoning consistently outperformed both baselines. The best model combination achieved an average accuracy of 0.820 across datasets, exceeding the strongest single model (0.777) and majority voting ensembles (up to 0.789). The method also scaled effectively with more participating models, while peer assessments reliably distinguished high- from low-quality reasoning chains. Conclusion: The proposed multi-agent peer-reviewed reasoning method enables LLMs to act as both solvers and evaluators, yielding superior performance in MedQA. By emphasizing reasoning quality rather than answer agreement alone, this approach improves accuracy, interpretability, and robustness, offering a promising direction for trustworthy biomedical AI systems.

Chinese Translation

目的：提高大型语言模型（LLMs）在医学问答（MedQA）中的准确性、可解释性和鲁棒性。方法：我们设计了一种多智能体同行评审推理方法，其中多个LLM智能体独立生成带有候选答案的推理链，然后作为同行评审者相互评估对方的推理在事实正确性和逻辑合理性方面的表现。最终选择评分最高的推理链以生成最终答案。实验使用了五个最先进的LLM（Llama-3.1-8B, Qwen2.5-7B, Phi-4, DeepSeek-LLM-7B, GPT-oss-20B）在三个基准数据集上进行：HeadQA、MedQA-USMLE和PubMedQA。性能与单模型推理链和基于推理链的多数投票进行了比较。结果：同行评审推理在所有基准测试中始终优于这两种基线。最佳模型组合在各数据集上的平均准确率达到了0.820，超过了最强单模型（0.777）和多数投票集成（最高0.789）。该方法在参与模型数量增加时也有效扩展，而同行评估可靠地区分了高质量和低质量的推理链。结论：所提出的多智能体同行评审推理方法使LLMs能够同时作为解题者和评估者，在MedQA中实现了卓越的性能。通过强调推理质量而非仅仅是答案一致性，该方法提高了准确性、可解释性和鲁棒性，为可信的生物医学人工智能系统提供了一个有前景的方向。

View on arXiv Download PDF AI Translation

cs.CL / 35 / 2606.15422

Pepti-Agent: An AI Agent for Peptide Design and Optimization

Pepti-Agent：一种用于肽设计和优化的人工智能代理

Chen, Houxu, Chandrasekhar, Achuth, Farimani, Amir Barati

Abstract

Therapeutic peptides occupy a valuable design space between small molecules and biologics, but their development requires satisfying several competing constraints at once: solubility, hemolytic activity, and nonspecific surface fouling are governed by overlapping sequence features, so improving one property often degrades another. Computational design addresses this by pairing generative models with sequence-based property predictors, iteratively proposing and refining candidates. However, these components are typically wired together as monolithic scripts that are difficult to inspect, extend, or reuse, and they often refine sequences by natural-language reasoning rather than by tracking the evolving multi-property state of each candidate. We present Pepti-Agent, a closed-loop, peptide-specific framework that exposes generation, property prediction, and single-residue mutation as independently inspectable Model Context Protocol (MCP) tools. A large language model controller invokes these tools and consults live predictor output between calls, so refinement is guided by each sequence's current property profile rather than by language reasoning alone. Task-specific PeptideGPT models generate candidates, ProtBERT-based classifiers score solubility, hemolysis, and non-fouling, and two interchangeable mutation operators propose sequence edits. By recording a per-step trace of controller decisions, predictor outputs, and accepted mutations, Pepti-Agent offers a reproducible substrate for benchmarking multi-objective design strategies and for prioritizing candidates for experimental validation.

Chinese Translation

治疗性肽在小分子和生物制剂之间占据了一个宝贵的设计空间，但其开发需要同时满足多个相互竞争的约束条件：溶解度、溶血活性和非特异性表面污染受重叠序列特征的影响，因此改善一种属性往往会降低另一种属性。计算设计通过将生成模型与基于序列的属性预测器相结合来解决这一问题，迭代地提出和优化候选者。然而，这些组件通常以单一脚本的形式连接在一起，难以检查、扩展或重用，并且它们通常通过自然语言推理而不是跟踪每个候选者不断变化的多属性状态来优化序列。我们提出了Pepti-Agent，一个闭环的肽特定框架，将生成、属性预测和单残基突变作为独立可检查的模型上下文协议（Model Context Protocol, MCP）工具。一个大型语言模型控制器调用这些工具，并在调用之间咨询实时预测器输出，因此优化是由每个序列当前的属性概况指导的，而不仅仅是依赖语言推理。特定任务的PeptideGPT模型生成候选者，基于ProtBERT的分类器对溶解度、溶血性和非污染进行评分，两个可互换的突变操作符提出序列编辑。通过记录控制器决策、预测器输出和接受的突变的逐步追踪，Pepti-Agent提供了一个可重复的基准平台，用于评估多目标设计策略和优先考虑实验验证的候选者。

View on arXiv Download PDF AI Translation

cs.CL / 36 / 2606.15449

Transfer Learning for FHIR Questionnaire Terminology Binding

用于FHIR问卷术语绑定的迁移学习

Gorshkov, Maxim

Abstract

Electronic prior authorization workflows require FHIR Questionnaire items to carry LOINC codes, yet most items in the HL7 Da Vinci CDS-Library lack these bindings. We treat this as a retrieval problem: given a Questionnaire item's text, find the correct LOINC code in a pool of 97,314 active codes. We compare six methods (TF-IDF, frozen MiniLM, BioBERT, BioLORD, contrastively fine-tuned MiniLM, and a TF-IDF+GPT reranker) on a 54-item evaluation set spanning three query styles (natural question, medium, and terse). No single method wins on every metric. BioLORD, a frozen encoder pre-trained on biomedical ontology definitions, has the best top-rank accuracy (R@1 = 0.185, MRR = 0.246) despite seeing no task-specific data, while a contrastive fine-tune on raw LHC-Forms pairs takes R@5 (0.389) and R@10 (0.426). A distribution-shift ablation shows why the fine-tune in our main table is not the strongest one: adding GPT-generated paraphrases to the raw pairs drops R@5 from 0.389 to 0.296, so the augmented union underperforms raw-only training on every metric except R@1. Performance peaks at 5k training pairs. Error analysis on BioLORD's R@1 failures shows that wrong-specificity and ambiguous-text cases together account for 59% of errors.

Chinese Translation

电子事先授权工作流程要求FHIR问卷项目携带LOINC代码，但HL7 Da Vinci CDS-Library中的大多数项目缺乏这些绑定。我们将此视为一个检索问题：给定一个问卷项目的文本，从97,314个活跃代码的池中找到正确的LOINC代码。我们在一个包含54个项目的评估集上比较了六种方法（TF-IDF、冻结的MiniLM、BioBERT、BioLORD、对比微调的MiniLM和TF-IDF+GPT重排序器），涵盖三种查询风格（自然问题、中等和简洁）。没有单一方法在每个指标上都表现最佳。BioLORD，一个在生物医学本体定义上预训练的冻结编码器，尽管没有看到特定任务的数据，但在最高排名准确率（R@1 = 0.185，MRR = 0.246）上表现最佳，而对原始LHC-Forms对进行的对比微调则在R@5（0.389）和R@10（0.426）上取得了最佳成绩。分布偏移消融实验表明，为什么我们主表中的微调不是最强的：将GPT生成的释义添加到原始对中使R@5从0.389降至0.296，因此增强的联合在除R@1外的每个指标上都表现不如仅使用原始训练。性能在5k训练对时达到峰值。对BioLORD在R@1失败的错误分析表明，错误的特异性和模糊文本案例共同占据了59%的错误。

View on arXiv Download PDF AI Translation

cs.CL / 37 / 2606.15461

ESBMC-PLC: Formal Verification of IEC 61131-3 Ladder Diagram Programs Using SMT-Based Model Checking

ESBMC-PLC：基于 SMT 的模型检测在 IEC 61131-3 梯形图程序中的形式验证

Dantas, Pierre, Cordeiro, Lucas, Junior, Waldir

Abstract

PLCs execute safety-critical programs across industrial sectors. The dominant PLC notation, ladder diagram (LD) per IEC 61131-3, remains absent from formal verification: SMT-based model checkers cannot process LD's rung-and-coil graphics. This paper presents ESBMC-PLC, the first open-source formal verifier with native LD support (PLCopen XML format), implemented as a new ESBMC frontend. ESBMC-PLC translates LD rungs to GOTO IR, models the PLC scan cycle as a while(true) loop with nondeterministic inputs, and checks safety properties via SMT-based bounded model checking or k-induction. A five-property YAML language (mutual_exclusion, invariant, absence, response, reachability) avoids temporal logic. A survey of 22 studies (2020-2026) identifies four research gaps; ESBMC-PLC closes two of them. Evaluation on 13 benchmarks (6 domains, 3 sources - including deployed CONTROLLINO PLCs and MathWorks Simulink PLC Coder) shows correct classification across 61 properties: all 9 author-constructed programs (Categories A/B) as expected, all 4 vendor programs (Category C) correctly unlabeled, with 8 bugs found (actionable counterexamples), 7 unbounded k-induction proofs, all runs under 60ms on Apple Silicon. Feature comparison with PLCverif shows that ESBMC-PLC is the only open-source tool that combines native LD, k-induction, and SMT bit-vector semantics.

Chinese Translation

可编程逻辑控制器（PLC）在工业领域执行安全关键程序。根据 IEC 61131-3 的主流 PLC 表示法，梯形图（LD）在形式验证中仍然缺乏支持：基于 SMT 的模型检查器无法处理 LD 的梯级和线圈图形。本文提出了 ESBMC-PLC，这是第一个具有原生 LD 支持（PLCopen XML 格式）的开源形式验证工具，作为新的 ESBMC 前端实现。ESBMC-PLC 将 LD 梯级转换为 GOTO 中间表示，模型化 PLC 扫描周期为带有非确定性输入的 while(true) 循环，并通过基于 SMT 的有界模型检查或 k-归纳法检查安全属性。一个包含五个属性的 YAML 语言（互斥性、不变性、缺失、响应、可达性）避免了时序逻辑。对 22 项研究（2020-2026）的调查识别出四个研究空白；ESBMC-PLC 解决了其中两个。对 13 个基准（6 个领域，3 个来源 - 包括已部署的 CONTROLLINO PLC 和 MathWorks Simulink PLC Coder）的评估显示在 61 个属性上的正确分类：所有 9 个作者构建的程序（A/B 类别）均如预期，所有 4 个供应商程序（C 类别）正确未标记，发现 8 个错误（可操作的反例），7 个无界 k-归纳证明，所有运行在 Apple Silicon 上均在 60 毫秒以内。与 PLCverif 的功能比较显示，ESBMC-PLC 是唯一结合原生 LD、k-归纳法和 SMT 位向量语义的开源工具。

View on arXiv Download PDF AI Translation

cs.CL / 38 / 2606.15483

Evaluative Judgement in Teaching AI-based Translation: A Class-room Case Study of AI-Mediated Translation and Post-Editing

基于人工智能翻译教学中的评估判断：AI介导翻译与后编辑的课堂案例研究

Dogru, Gokhan

Abstract

Drawing on 23 anonymized student pro-jects from a fourth-year Machine Transla-tion and Post-editing course in a BA-level translation programme, this paper exam-ines how structured comparison of gen-eral-purpose LLMs and online MT sys-tems can elicit evaluative judgement in AI-mediated translation. Students translat-ed short specialised English Wikipedia texts into Catalan or Spanish, generated four system outputs, evaluated them using automatic metrics and human adequa-cy/fluency assessment, selected one output for post-editing, and justified their deci-sion in written reports. Descriptive counts are reported for all 23 projects, while qualitative interpretation is based on the 22 cases accompanied by written reports. Results show that students did not treat automatic metrics as final authority: final post-editing selections often diverged from metric rankings and were justified through adequacy, fluency, terminology, naturalness, and expected post-editing ef-fort. The study therefore does not bench-mark systems under controlled conditions; it analyses how students justified system choice within an authentic classroom as-signment.

Chinese Translation

本论文基于来自本科翻译项目中第四年机器翻译与后编辑课程的23个匿名学生项目，探讨了通用大型语言模型（LLMs）与在线机器翻译（MT）系统的结构化比较如何引发在AI介导翻译中的评估判断。学生们将短篇专业的英文维基百科文本翻译成加泰罗尼亚语或西班牙语，生成四个系统输出，使用自动评测指标和人工的适应性/流畅性评估对其进行评价，选择一个输出进行后编辑，并在书面报告中阐明他们的决策。所有23个项目的描述性计数已被报告，而定性解释则基于22个附有书面报告的案例。结果显示，学生并未将自动评测指标视为最终权威：最终的后编辑选择往往与指标排名不一致，并通过适应性、流畅性、术语、自然性和预期的后编辑工作量进行辩护。因此，本研究并未在受控条件下对系统进行基准测试；而是分析了学生在真实课堂作业中如何为系统选择提供依据。

View on arXiv Download PDF AI Translation

cs.CL / 39 / 2606.15510

AthDGC: An Open Diachronic Greek Treebank with Indo-European Parallels

AthDGC：一个具有印欧语系平行关系的开放性历时希腊树库

Lavidas, Nikolaos, Nikiforidou, Kiki, Haug, Dag, Kulikov, Leonid, Geka, Vassiliki, Symeonidis, Vassileios, Michalareas, Theodoros, Chionidi, Sofia, Tsiropina, Anastasia, Plakoutsi, Eleni, Argyropoulos, Evangelos

Abstract

AthDGC ("Athens-PROIEL") is an open, end-to-end workflow and dataset. It is, to the best of our knowledge, the first openly licensed dependency-parsed treebank of Greek that spans eight diachronic periods, namely Archaic, Classical, Koine, Late Antique, Byzantine, Late Byzantine, Early Modern, and Modern Greek, under a single PROIEL XML 2.0 schema, with verse-level cross-alignment of the New Testament to Latin (Vulgate), Gothic (Wulfila), Old Church Slavonic (Marianus), and Classical Armenian. AthDGC builds on the PROIEL Treebank Family (Haug and Johndal 2008; Eckhoff et al. 2018), which established the schema and the Koine-Greek reference set for the project. Annotation uses the Stanford Stanza PROIEL-trained workflow; sentence-level alignment uses LaBSE, a multilingual sentence-embedding model; word-level alignment uses multilingual-BERT attention through the AwesomeAlign procedure. The v0.4 release provides curated samples and the open-source toolkit; the full annotated corpus partitions remain under v0.5 audit on the Greek national HPC. Quantitative scale, per-witness verse counts, and per-period annotated-row counts are reported in the v0.5 release notes, after the audit pass completes. Concept DOI: 10.5281/zenodo.20439182.

Chinese Translation

AthDGC（“雅典-PROIEL”）是一个开放的端到端工作流程和数据集。据我们所知，它是第一个公开许可的希腊依存句法分析树库，涵盖八个历时时期，即古希腊、古典希腊、科伊尼希腊、晚古代希腊、拜占庭希腊、晚拜占庭希腊、早现代希腊和现代希腊，采用单一的PROIEL XML 2.0模式，并与拉丁文（《武加大》）、哥特文（Wulfila）、古教会斯拉夫文（Marianus）和古典亚美尼亚文进行新约经文级别的交叉对齐。AthDGC建立在PROIEL树库家族（Haug和Johndal 2008；Eckhoff等人 2018）的基础上，该家族为该项目建立了模式和科伊尼希腊参考集。注释使用斯坦福Stanza的PROIEL训练工作流程；句子级对齐使用LaBSE，一个多语言句子嵌入模型；词级对齐通过AwesomeAlign程序使用多语言BERT注意力机制。v0.4版本提供了经过整理的样本和开源工具包；完整的注释语料库分区仍在希腊国家高性能计算中心进行v0.5审计。定量规模、每个见证的经文计数和每个时期的注释行计数将在v0.5发布说明中报告，审计通过完成后。概念 DOI：10.5281/zenodo.20439182。

View on arXiv Download PDF AI Translation

cs.CL / 40 / 2606.15517

SHARD: Safe and Helpful Alignment via Self-Reframing Distillation

SHARD：通过自我重构蒸馏实现安全与有帮助的对齐

Manoranjan, Viswonathan, Gupta, Amogh, Vijjini, Anvesh Rao, Hofweber, Thomas, Chaturvedi, Snigdha

Abstract

Large language models often struggle with sensitive prompts. They may refuse outright, provide generic safety boilerplate, or fail to address the user's legitimate informational needs that can be answered safely. We introduce SHARD, a self-reframing distillation method to improve safe-helpfulness. It first rewrites sensitive prompts to surface benign intent using philosophical guidelines, then reframes its original responses into safe, more helpful ones, and finally fine-tunes the model on its self-reframed responses. Across DNA and the English subset of LINGUASAFE, SHARD improves helpfulness for most model families while preserving safety. It also remains competitive with distillation from a larger teacher model, suggesting that models can internalize safe and helpful behavior elicited from their own. Warning: This paper contains content that may be offensive or harmful.

Chinese Translation

大型语言模型在处理敏感提示时常常面临困难。它们可能会直接拒绝、提供通用的安全模板，或未能满足用户的合法信息需求，这些需求是可以安全回答的。我们提出了SHARD，一种自我重构蒸馏方法，以提高安全性和有帮助性。该方法首先使用哲学指导原则重写敏感提示，以显现良性意图，然后将其原始响应重构为安全且更有帮助的回应，最后在其自我重构的响应上对模型进行微调。在DNA和LINGUASAFE的英语子集上，SHARD提高了大多数模型家族的有帮助性，同时保持了安全性。它还与来自更大教师模型的蒸馏保持竞争力，这表明模型可以内化从自身引发的安全和有帮助的行为。警告：本文包含可能令人反感或有害的内容。

View on arXiv Download PDF AI Translation

cs.CL / 41 / 2606.15521

Emergent retokenization symmetry in large language models: phenomenology and applications

大型语言模型中的突现重标记对称性：现象学与应用

Jain, Kanishk, Day, Matthew, Can, Tankut

Abstract

Tokenization introduces representational redundancy: under a fixed token vocabulary, every byte string admits many valid token encodings, or segmentations, that decode to the same surface string. However, given a prompt, most language model tokenizers break this representational symmetry by returning a canonical segmentation. Training only on canonical segmentations should influence inference behavior, and there is little reason to expect models to respect segmentation symmetry on downstream tasks. We find that this symmetry partially emerges during training. Here, we probe this emergent symmetry through experiments testing token compositional understanding, representation diversity, and task focused benchmark performance. We primarily use \textbf{retokenization} -- replacing a prompt's canonical tokenization with an alternative segmentation while preserving its bytes exactly. Relative to other prompt perturbations, retokenization is unusually clean because it isolates segmentation effects without changing syntax, semantics or surface form. We use retokenization to study sensitivity and robustness to semantically identical input representations across pretraining and post-training. Moreover, this partial retokenization symmetry suggests a distinct inference-time sampling axis. While temperature sampling generates diverse outputs from the model using its next-token probability distribution, retokenization generates diversity from the model's internal computations through semantically equivalent input representations. We find that while this retokenization sampling strategy can hurt performance on easy problems, it can also recover solutions that conventional sampling does not find. Overall, our work presents retokenization as a simple yet powerful probe of large language models, shedding light on compositional understanding and prompt sensitivity, and offering a novel sampling strategy.

Chinese Translation

标记化引入了表示冗余：在固定的标记词汇下，每个字节字符串都允许多种有效的标记编码或分段，这些编码解码为相同的表面字符串。然而，在给定提示的情况下，大多数语言模型的标记器通过返回标准分段来打破这种表示对称性。仅在标准分段上进行训练应会影响推理行为，因此几乎没有理由期望模型在下游任务中尊重分段对称性。我们发现这种对称性在训练过程中部分出现。在这里，我们通过实验探讨这种突现对称性，测试标记组合理解、表示多样性和任务聚焦基准性能。我们主要使用重标记（retokenization）——在完全保留字节的情况下，用替代分段替换提示的标准标记化。与其他提示扰动相比，重标记异常干净，因为它在不改变语法、语义或表面形式的情况下，隔离了分段效应。我们使用重标记研究在预训练和后训练中对语义相同输入表示的敏感性和鲁棒性。此外，这种部分重标记对称性暗示了一种独特的推理时采样轴。虽然温度采样通过模型的下一个标记概率分布生成多样化输出，重标记则通过语义等价的输入表示从模型的内部计算中生成多样性。我们发现，尽管这种重标记采样策略可能会在简单问题上损害性能，但它也能恢复常规采样无法找到的解决方案。总体而言，我们的工作将重标记呈现为一种简单而强大的大型语言模型探测工具，揭示了组合理解和提示敏感性，并提供了一种新颖的采样策略。

View on arXiv Download PDF AI Translation

cs.CL / 42 / 2606.15532

EIBench: A Simulator-Based Benchmark and Turn-Credit RL for Emotion Management

EIBench：基于模拟器的情感管理基准与回合信用强化学习

Zhu, Rongzhi, Huang, Xiang, Wu, Yuchuan, Wang, Rui, Sun, Zequn, Ren, Tao, Luo, Weiyao, Qiu, Bingxue, Ye, Jieping, Li, Yongbin, Hu, Wei

Abstract

Emotional intelligence (EI) in Large Language Models (LLMs) is often evaluated through static understanding tasks or single-response dialogue generation. However, emotion management is interactive: a good model should not only recognize a user's emotion, but also improve the user's emotional and relational state over several turns. We introduce EIBench, a simulator-based benchmark for interactive emotion management. EIBench contains 2,222 scenarios, with 2,009 for training and 213 for held-out testing. The scenarios are organized by a 2x2 taxonomy covering Support, Defense, Repair, and Charm, which together capture different forms of support, boundary maintenance, trust repair, and rapport building. In each scenario, an LLM simulator plays the user, updates an emotion-relation state after each turn, and maps the final state to an anchor-based score. This design makes EIBench both an evaluation benchmark and a training environment: the final state gives the outcome reward, while the per-turn state updates provide dense feedback for RL. We evaluate 15 open- and closed-source LLMs. Current models perform well on support and rapport-building scenes, but struggle with boundary maintenance under user pressure. To improve the EI ability of LLMs, we propose Centered Turn-Credit GRPO (CTC-GRPO), a GRPO extension that reuses the simulator's per-turn state updates as dense turn-level feedback while preserving the final outcome reward. CTC-GRPO improves Qwen3-8B from -22.4 to +22.4 on EIBench and also improves on out-of-distribution evaluations including SAGE (+12.4) and EQBench3 (+20.9%). Our results show that simulator-tracked user states can support both evaluation and training for multi-turn emotion management.

Chinese Translation

大型语言模型（LLMs）中的情感智能（EI）通常通过静态理解任务或单一响应对话生成进行评估。然而，情感管理是互动性的：一个好的模型不仅应该识别用户的情感，还应该在多个回合中改善用户的情感和关系状态。我们引入了EIBench，这是一个用于互动情感管理的基于模拟器的基准。EIBench包含2222个场景，其中2009个用于训练，213个用于保留测试。这些场景按照一个2x2的分类法组织，涵盖支持、辩护、修复和魅力，这些共同捕捉了不同形式的支持、边界维护、信任修复和融洽建立。在每个场景中，LLM模拟器扮演用户，在每个回合后更新情感-关系状态，并将最终状态映射到基于锚点的得分。这个设计使得EIBench既是一个评估基准，也是一个训练环境：最终状态提供结果奖励，而每回合状态更新则为强化学习提供密集反馈。我们评估了15个开源和闭源的LLMs。目前的模型在支持和融洽建立场景中表现良好，但在用户压力下的边界维护方面表现不佳。为了提高LLMs的情感智能能力，我们提出了Centered Turn-Credit GRPO（CTC-GRPO），这是GRPO的扩展，重用模拟器的每回合状态更新作为密集的回合级反馈，同时保留最终结果奖励。CTC-GRPO使Qwen3-8B在EIBench上的得分从-22.4提升至+22.4，并在包括SAGE（+12.4）和EQBench3（+20.9%）的分布外评估中也有所改善。我们的结果表明，模拟器跟踪的用户状态可以支持多回合情感管理的评估和训练。

View on arXiv Download PDF AI Translation

cs.CL / 43 / 2606.15566

LLM-Assisted Stance Detection in Scientific Discourse: A Test Case in Bayesian Cognitive Science

基于大型语言模型的科学话语立场检测：一个贝叶斯认知科学的案例研究

Kucuk, Eyup Engin, Kelestemur, Tarik, Tanrikulu, Ömer Dağlar

Abstract

Qualitative coding is central to social science, but expert annotation is difficult to scale. LLMs offer a possible extension, yet require careful validation when the target construct is interpretive, theoretically loaded, and only indirectly expressed. We study this problem in a difficult case: detecting whether authors treat Bayesian models as descriptions of mental and neural mechanisms (realism) or as useful mathematical tools (instrumentalism). Our method combines a theory-driven codebook, expert-coded reference annotations, a diagnostic-gated prompt-optimization search yielding a shared zero-shot prompt for three frontier LLMs (GPT-5.1, Claude Sonnet 4.6, Gemini 3 Pro Preview), and multi-rater reliability analysis. The final prompt achieved a held-out combined reliability score of 0.76 (harmonic mean of ICC = 0.79 and $\alpha$ = 0.74), with all diagnostics satisfied. Deployed on 6,858 quotes from 210 articles, the three LLMs reached substantial quote-level agreement (ICC = 0.80; $\alpha$ = 0.76; combined = 0.78) and near-perfect article-level rank stability ($r$ = 0.96-0.97 across rater pairs). The corpus was predominantly weakly realist, but article-level stances were rarely uniform: only 1.4% of articles used a single band, while 59.5% spanned four or more. Low-level perception/motor articles scored 8.8 Realism points higher than high-level cognition articles ($p < .001$, $d = 0.60$), quantifying a long-held qualitative intuition. We present this as an expert-led case study; the framework is intended to generalize to similar theoretically demanding tasks, not to all qualitative analysis.

Chinese Translation

定性编码在社会科学中至关重要，但专家注释难以扩展。大型语言模型（LLMs）提供了一种可能的扩展方式，但在目标构念具有解释性、理论负载且仅间接表达时，需要谨慎验证。我们在一个困难的案例中研究这个问题：检测作者是否将贝叶斯模型视为心理和神经机制的描述（现实主义）或作为有用的数学工具（工具主义）。我们的方法结合了以理论为驱动的编码手册、专家编码的参考注释、诊断门控的提示优化搜索，生成了一个适用于三种前沿大型语言模型（GPT-5.1、Claude Sonnet 4.6、Gemini 3 Pro Preview）的共享零样本提示，并进行了多评估者可靠性分析。最终提示在保留的组合可靠性评分中达到了0.76（和谐均值为ICC = 0.79和α = 0.74），所有诊断均满足要求。在210篇文章的6,858条引用中，三种大型语言模型达到了显著的引用级别一致性（ICC = 0.80；α = 0.76；组合 = 0.78）和接近完美的文章级别排名稳定性（r = 0.96-0.97，跨评估者对）。语料库主要表现为弱现实主义，但文章级别的立场很少统一：仅有1.4%的文章使用单一立场，而59.5%的文章跨越四个或更多立场。低层次感知/运动类文章的现实主义得分比高层次认知类文章高出8.8分（p < .001，d = 0.60），量化了长期以来的定性直觉。我们将此作为一个专家主导的案例研究呈现；该框架旨在推广到类似理论要求高的任务，而非所有定性分析。

View on arXiv Download PDF AI Translation

cs.CL / 44 / 2606.15610

LLM Judges Have Dark Current: A Psychometric Datasheet for LLM-as-a-Judge Evaluation

LLM评判者存在暗电流：LLM作为评判者评估的心理测量数据表

Usami, Hiroyasu, Hara, Keisuke, Tsuboi, Ayato, Matsuda, Naohiko

Abstract

LLM-as-a-judge systems are now routinely used for open-ended model evaluation, where human preference annotation is costly, slow, and difficult to reproduce. Yet these judges are often reported as scalar accuracy, win-rate, or agreement devices. We argue that a judge should instead be reported as a measurement instrument. We introduce a Judge Datasheet protocol that measures dark current under true-vacuum inputs, stable cross-sensitivity to same-quality surface variation, positional false preference, target sensitivity on a controlled quality ladder, and the criterion or operating point induced by tie instructions. The direction-stability decomposition reveals that apparent Delta0 preference can be stable surface response or disguised position bias. In a three-judge open-weight case study, Llama-3.1-8B shows high dark current and presentation-conflicted Delta0 behavior, Qwen2.5-14B is vacuum-clean and target-sensitive but mixes stable and positional over-discrimination, and Qwen2.5-32B is vacuum-clean with low stable cross-sensitivity and low positional false preference. A strict tie criterion eliminates Qwen32B Delta0 false preference but absorbs marginal Delta1 target signals into ties while preserving Delta5 sensitivity. The results show that prompting moves the criterion, not the resolution. We do not claim that the downstream mechanism hypothesis that motivated this work is confirmed; the contribution is a metrological protocol for measuring the measuring device before downstream claims are made.

Chinese Translation

LLM作为评判者的系统现在常用于开放式模型评估，其中人类偏好注释成本高、速度慢且难以重现。然而，这些评判者通常被报告为标量准确性、胜率或一致性设备。我们认为，评判者应该被视为一种测量工具。我们引入了一种评判者数据表协议，用于测量在真实真空输入下的暗电流、对同质量表面变化的稳定交叉敏感性、位置虚假偏好、在受控质量梯度上的目标敏感性，以及由平局指令引起的标准或操作点。方向稳定性分解揭示，表观的Delta0偏好可能是稳定的表面响应或伪装的位置偏差。在一个三评判者开放权重的案例研究中，Llama-3.1-8B显示出高暗电流和呈现冲突的Delta0行为，Qwen2.5-14B在真空清洁和目标敏感性方面表现良好，但混合了稳定和位置过度歧视，而Qwen2.5-32B在真空清洁的情况下具有低稳定交叉敏感性和低位置虚假偏好。严格的平局标准消除了Qwen32B的Delta0虚假偏好，但将边际Delta1目标信号吸收到平局中，同时保持Delta5敏感性。结果表明，提示移动了标准，而不是分辨率。我们并不声称激励本研究的下游机制假设已得到证实；贡献在于在提出下游主张之前测量测量设备的计量协议。

View on arXiv Download PDF AI Translation

cs.CL / 45 / 2606.15641

Distilling Examples into Task Instructions: Enhanced In-Context Learning for Real-World B2B Conversations

将示例提炼为任务指令：增强现实世界B2B对话的上下文学习

Rotman, Guy, Kopilov, Adi, Zalmanson, Danit Berger, Allouche, Omri

Abstract

In-context learning (ICL) is the standard method for low-resource classification, yet its efficacy in specialized domains remains largely unexplored. We address the challenge of classifying semantically complex, multi-party B2B conversations, where traditional ICL encounters significant limitations, especially as context length increases due to the concatenation of multiple few-shot examples. We introduce the \texttt{Call Playbook} dataset, featuring five classification tasks derived from real-world B2B conversations targeting core sales concepts. To bridge the gap between performance and practical utility, we propose novel knowledge extraction methods that distill verbose examples into compact, interpretable representations of structured classification criteria and precise task descriptions. Our approach achieves a 99\% reduction in token usage and improves macro-averaged AUC by up to 7\% over traditional ICL. Notably, it remains robust as context grows, unlike advanced token compression baselines which degrade by over 9 F1 points. Importantly, our framework enables direct refinement of classification logic, addressing critical needs for transparency, efficiency, and user interaction in real-world NLP applications.

Chinese Translation

上下文学习（ICL）是低资源分类的标准方法，但其在专业领域的有效性仍然 largely 未被探索。我们解决了对语义复杂的多方B2B对话进行分类的挑战，在这种情况下，传统的ICL面临显著的局限性，尤其是当由于多个少量示例的串联而导致上下文长度增加时。我们引入了 exttt{Call Playbook}数据集，包含五个源自现实世界B2B对话的分类任务，针对核心销售概念。为了弥合性能与实际效用之间的差距，我们提出了新颖的知识提取方法，将冗长的示例提炼为紧凑、可解释的结构化分类标准和精确的任务描述。我们的方法在令牌使用上实现了99%的减少，并在宏观平均AUC上比传统ICL提高了多达7%。值得注意的是，随着上下文的增长，我们的方法保持了稳健性，而先进的令牌压缩基线则下降超过9个F1点。重要的是，我们的框架能够直接优化分类逻辑，满足现实世界NLP应用中对透明性、效率和用户交互的关键需求。

View on arXiv Download PDF AI Translation

cs.CL / 46 / 2606.15643

Extending Item Response Theory for Efficient and Meaningful Multilingual Evaluation

扩展项目反应理论以实现高效且有意义的多语言评估

Lior, Gili, Frostig, Tzviel, Stanovsky, Gabriel, Eyal, Matan

Abstract

Multilingual benchmarks are central to evaluating large language models (LLMs) across languages, but they suffer from three issues: exhaustive evaluation scales linearly with the number of languages, automatic translation introduces errors that are easily missed at scale, and some items conflate general and culture-specific knowledge. We address all three with a unified statistical framework, Multilingual-IRT, which extends Item Response Theory with per-language difficulty deviations, split discriminability separating content from language effects, and per-language ability residuals. Fitting Multilingual-IRT on 25 LLMs across 29 languages of MMLU-Pro-X, we show that its fitted parameters support three practical applications: predicting unobserved (item, LLM, language) instances with 11-16% lower binary cross-entropy than the strongest accuracy-based baseline, surfacing candidate translation errors distributed across all 28 non-English languages, whereas accuracy-based baselines concentrate detections in a few languages, and recovering culture-specific items that accuracy-based baselines miss.

Chinese Translation

多语言基准在评估跨语言的大型语言模型（LLMs）中至关重要，但它们面临三个问题：全面评估的规模与语言数量成线性关系，自动翻译引入的错误在大规模中容易被忽视，以及某些项目混淆了通用知识和特定文化知识。我们通过一个统一的统计框架——多语言项目反应理论（Multilingual-IRT）来解决这三个问题，该框架扩展了项目反应理论，包含每种语言的难度偏差、将内容与语言效应分开的辨别度，以及每种语言的能力残差。在对29种语言的MMLU-Pro-X上拟合25个LLM的多语言项目反应理论后，我们显示其拟合参数支持三种实际应用：预测未观察到的（项目、LLM、语言）实例，其二元交叉熵比最强的基于准确性的基线低11-16%；揭示分布在所有28种非英语语言中的候选翻译错误，而基于准确性的基线则集中在少数语言的检测上；以及恢复基于准确性基线遗漏的特定文化项目。

View on arXiv Download PDF AI Translation

cs.CL / 47 / 2606.15714

Beyond English: Uncovering the Multilingual Gap in Vision-Language-Action Models

超越英语：揭示视觉-语言-行动模型中的多语言差距

Chen, Hanyang, Li, Hongliang, Cao, Jiarui, Li, Yang, Jiang, Yang, Wen, Haonan, Huang, Kaiyu, Guo, Shengnan, Wan, Huaiyu

Abstract

Vision-Language-Action models have recently demonstrated promising capabilities in learning generalist robot policies from large-scale multimodal data. However, most existing VLA systems are trained and evaluated primarily with English instructions, leaving their ability to understand and execute instructions in other languages largely unexplored. While the underlying large language models often possess multilingual capabilities, it remains unclear whether these multilingual capabilities transfer to VLAs during training. In this work, we present the first systematic study of multilingual instruction following in VLA models. We first construct multilingual instructions by extending existing benchmarks with translations of their instructions. Using these instructions, we evaluate several representative VLA models across a range of tasks in simulation settings. Our experiments reveal a significant multilingual gap: models trained primarily on English instructions exhibit substantial performance degradation when evaluated on other languages, even when the underlying language backbone is multilingual. We provide several findings and analyses to understand the multilingual gap. Cross-lingual transfer behavior analysis shows that performance drops correlate with both instruction understanding and action execution. Representation analyses suggest that multilingual instruction-caused representation shifts may contribute to the multilingual gap. Motivated by these findings, we further explore strategies to improve multilingual performance in VLAs. We propose a simple yet effective multilingual fine-tuning approach, Multilingual Principal Component Alignment, which leverages Principal Component Analysis to get the principal component subspace and align projected multilingual representations, effectively reducing the multilingual performance gap.

Chinese Translation

视觉-语言-行动（VLA）模型最近在从大规模多模态数据中学习通用机器人策略方面展现了良好的能力。然而，现有的大多数VLA系统主要使用英语指令进行训练和评估，导致它们理解和执行其他语言指令的能力尚未得到充分探索。尽管基础的大型语言模型通常具备多语言能力，但这些多语言能力在VLA训练过程中是否能够转移仍不明确。在本研究中，我们首次系统性地研究了VLA模型中的多语言指令跟随。我们首先通过扩展现有基准测试并翻译其指令来构建多语言指令。利用这些指令，我们在模拟环境中评估了几种具有代表性的VLA模型在不同任务上的表现。实验结果揭示了显著的多语言差距：主要在英语指令上训练的模型在评估其他语言时表现显著下降，即使其基础语言模型是多语言的。我们提供了若干发现和分析以理解这一多语言差距。跨语言转移行为分析表明，性能下降与指令理解和动作执行均相关。表征分析表明，由多语言指令引起的表征变化可能会导致多语言差距。基于这些发现，我们进一步探索了提高VLA多语言性能的策略。我们提出了一种简单而有效的多语言微调方法——多语言主成分对齐（Multilingual Principal Component Alignment），该方法利用主成分分析获取主成分子空间并对齐投影的多语言表征，从而有效减少多语言性能差距。

View on arXiv Download PDF AI Translation

cs.CL / 48 / 2606.15733

Vernier: Probing Representational Misalignment Behind Lexical Gaps in Causal Reasoning

Vernier：探讨因果推理中词汇缺口背后的表征不一致性

Yu, Zhenyu

Abstract

Instruction-tuned language models can answer the same causal-reasoning question differently after its English variable names are replaced by type-preserving placeholders, although the structural causal model and the gold answer are unchanged. We ask whether this lexical gap reflects information loss in the placeholder view or a misaligned read-out from a representation that still carries answer-relevant content. Vernier uses a paired-view weight update as an instrument and then inspects the mechanism left after the gap closes. In the working regimes, the evidence favours representational misalignment. A variable-name probe becomes more accurate on the placeholder view, and activation patching on Qwen-7B, Qwen-14B, and Llama-3.1-8B shows that the decision-token representation can transfer answer identity between views. The update that realigns the views is counterfactual augmentation over original and placeholder prompts, while the answer-subspace KL mainly sharpens intermediate answer-belief agreement. Success is bounded by model family, scale, and task. CRASS transfer is reliable across Qwen scales and Llama, e-CARE remains weak, and preliminary non-causal rename tasks show a similar qualitative pattern.

Chinese Translation

经过指令调优的语言模型在将其英文变量名替换为保持类型的占位符后，能够以不同的方式回答同一个因果推理问题，尽管结构因果模型和标准答案保持不变。我们探讨这种词汇缺口是否反映了占位符视图中的信息丢失，或是从仍然包含答案相关内容的表征中产生的读出不一致。Vernier 使用成对视图权重更新作为工具，然后检查在缺口关闭后留下的机制。在工作机制中，证据支持表征不一致性。变量名探测器在占位符视图上的准确性提高，而在 Qwen-7B、Qwen-14B 和 Llama-3.1-8B 上的激活修补显示，决策令牌表征可以在视图之间转移答案身份。重新对齐视图的更新是针对原始和占位符提示的反事实增强，而答案子空间的 KL 散度主要加深了中间答案信念的一致性。成功受限于模型系列、规模和任务。CRASS 转移在 Qwen 系列和 Llama 中是可靠的，而 e-CARE 仍然较弱，初步的非因果重命名任务显示出类似的定性模式。

View on arXiv Download PDF AI Translation

cs.CL / 49 / 2606.15734

Retrievable Gradients: Continual Post-Training Without Cumulative Weight Drift

可检索梯度：无累积权重漂移的持续后训练

Su, Weihang, Kang, Jiacheng, Xu, Jingyan, Ai, Qingyao, Long, Jianming, Zhang, Hanwen, Du, Bangde, Cao, Xinyuan, Zhang, Min, Liu, Yiqun

Abstract

Continual post-training enables models to absorb emerging knowledge after deployment, but repeatedly updating shared parameters can accumulate weight drift, potentially causing catastrophic forgetting and degrading general capabilities. Retrieval-augmented generation avoids such parameter drift, yet often lacks the depth of parametric knowledge integration. In this paper, we propose ReGrad (Retrievable Gradients), a new paradigm that treats gradients as retrievable units of knowledge. ReGrad pre-computes document-specific gradients offline, stores them in an indexed Gradient Bank, and retrieves only query-relevant gradients at inference time for temporary weight adaptation. However, raw language-modeling gradients are optimized for token-level document reconstruction rather than for query-driven knowledge use. We therefore introduce a bi-level meta-learning objective that reshapes document-derived gradients into generalizable adaptation signals for downstream tasks. Experiments across general and domain-specific settings show that \textsc{ReGrad} outperforms CPT and RAG baselines, enabling scalable and reversible parametric knowledge injection without accumulating weight drift.

Chinese Translation

持续后训练使模型在部署后能够吸收新出现的知识，但反复更新共享参数可能会导致权重漂移的累积，从而可能引发灾难性遗忘并降低模型的通用能力。检索增强生成（Retrieval-augmented generation）避免了这种参数漂移，但往往缺乏参数知识整合的深度。本文提出了 ReGrad（可检索梯度），一种将梯度视为可检索知识单元的新范式。ReGrad 离线预计算文档特定的梯度，将其存储在索引梯度库（Gradient Bank）中，并在推理时仅检索与查询相关的梯度以进行临时权重适应。然而，原始的语言模型梯度是针对令牌级文档重建进行优化的，而不是针对查询驱动的知识使用。因此，我们引入了一个双层元学习目标，将文档衍生的梯度重塑为可推广的适应信号，以用于下游任务。在一般和特定领域的实验中， extsc{ReGrad} 的表现优于 CPT 和 RAG 基线，能够实现可扩展和可逆的参数知识注入，而无需累积权重漂移。

View on arXiv Download PDF AI Translation

cs.CL / 50 / 2606.15735

EHRNote-ChatQA: A Benchmark for Evidence-Grounded Multi-Turn Clinical Question Answering over Longitudinal Discharge Summaries

EHRNote-ChatQA：基于证据的多轮临床问答在长期出院总结中的基准

Kim, Jiyoun, Yeo, Muhan, Jang, Eunhye, Yang, Jeewon, Yoon, Hangyul, Lee, Su Ji, Han, Hee Jo, Jung, Hee-Jae, Kwon, Doyun, Lee, Jun young, Lee, Jaehun, Lee, Jung-Oh, Kweon, Sunjun, Moon, Jong Hak, Kim, Daseul, Cho, Minjae, Choi, Edward

Abstract

Discharge summaries are crucial clinical documents containing the context of a patient's overall hospital stay, and are routinely reviewed by medical experts for patient readmission, ongoing care, and diagnostic decision-making. When reviewing them, medical experts often must iteratively synthesize information across multiple summaries while verifying the evidence supporting each answer. Although large language models (LLMs) are increasingly explored for clinical question answering, existing benchmarks do not sufficiently reflect this setting: they often evaluate exam-style medical knowledge or focus on single-turn question answering with limited evidence-grounding evaluation. We introduce EHRNote-ChatQA, the first benchmark for evidence-grounded multi-turn clinical question answering over patients' multiple discharge summaries. Built from de-identified MIMIC-IV discharge summaries, EHRNote-ChatQA contains 967 patient-level multi-turn samples spanning one to five notes and 16,072 medical-expert-verified QA pairs (8,036 content questions, each paired with an evidence-grounding question) across eight clinical categories. The benchmark is constructed through an expert-informed pipeline combining discharge-summary structuring schema, expert-curated multi-turn QA templates, and LLM-based generation, followed by review and revision of every single QA sample by 11 medical experts. Benchmarking 22 open- and closed-source LLMs reveals several challenges, including that LLMs struggle more with evidence grounding than content answering, multi-turn errors compound across turns, and single-turn clinical QA performance does not reliably transfer to this setting. These findings establish EHRNote-ChatQA as a rigorous and practical benchmark for evaluating clinical QA systems. The dataset will be made publicly available through PhysioNet credentialed access.

Chinese Translation

出院总结是重要的临床文件，包含患者整体住院期间的背景信息，医疗专家通常会对其进行审查，以便于患者再入院、持续护理和诊断决策。在审查这些总结时，医疗专家常常需要在多个总结中迭代地综合信息，同时验证每个答案所支持的证据。尽管大型语言模型（LLMs）在临床问答中越来越受到关注，但现有的基准并未充分反映这一环境：它们通常评估考试风格的医学知识或专注于单轮问答，且证据基础的评估有限。我们介绍了EHRNote-ChatQA，这是第一个针对患者多个出院总结的基于证据的多轮临床问答的基准。EHRNote-ChatQA基于去标识化的MIMIC-IV出院总结构建，包含967个患者级别的多轮样本，跨越一到五个总结，以及16,072个经过医学专家验证的问答对（8,036个内容问题，每个问题配有一个证据基础问题），涵盖八个临床类别。该基准通过结合出院总结结构化方案、专家策划的多轮问答模板和基于LLM的生成，经过11位医学专家对每个问答样本的审查和修订而构建。对22个开源和闭源LLM的基准测试揭示了几个挑战，包括LLM在证据基础方面的表现优于内容回答的困难、多轮错误在轮次间的累积，以及单轮临床问答的表现并不可靠地转移到这一环境中。这些发现确立了EHRNote-ChatQA作为评估临床问答系统的严格且实用的基准。该数据集将通过PhysioNet的凭证访问公开发布。

View on arXiv Download PDF AI Translation

cs.CL / 51 / 2606.15741

A Self Consistency Based Reranking for Narrative Question Answering

基于自一致性的叠加排序方法用于叙事问答

Mohamed, Molham, Hamdi, Ali

Abstract

Narrative question answering (NQA) is a challenging task in natural language processing that requires models to understand long textual contexts, capture relationships across events, and generate coherent responses. Despite recent advances in pretrained language models, most existing approaches rely on a single decoding output during inference, making them sensitive to generation variability and often resulting in incomplete or inconsistent answers .To address this limitation, we propose a self-ensemble Self-Consistency-Based reranking framework for narrative question answering. The proposed method generates multiple candidate answers for each story-question pair and selects the final answer based on semantic agreement among the generated responses. This allows the model to explore diverse answer formulations while improving robustness through consensus-based selection without requiring modifications to the underlying architecture .The framework combines pretrained and fine-tuned language generation with multi-answer inference and similarity-based reranking. We evaluate the proposed approach on the NarrativeQA dataset using multiple models, including FLAN-T5 (Base and Small) and Pegasus-Large, under both baseline and fine-tuned settings .Experimental results demonstrate that the proposed method consistently improves performance across all models. In particular, FLAN-T5-Base achieves the best overall performance, improving from 82.32% to 86.66% (+4.34%) when combined with self-ensemble inference. Additionally, the largest improvement is observed with Pegasus-Large, which increases from 72.50% to 87.07% (+14.57%), highlighting the effectiveness of the proposed strategy.

Chinese Translation

叙事问答（NQA）是自然语言处理中的一项挑战性任务，要求模型理解长文本上下文，捕捉事件之间的关系，并生成连贯的回答。尽管预训练语言模型近年来取得了进展，但大多数现有方法在推理过程中依赖单一的解码输出，这使得它们对生成的变异性敏感，常常导致回答不完整或不一致。为了解决这一局限性，我们提出了一种基于自一致性的自集成重排序框架，用于叙事问答。该方法为每个故事-问题对生成多个候选答案，并根据生成响应之间的语义一致性选择最终答案。这使得模型能够探索多样的答案表述，同时通过基于共识的选择提高鲁棒性，而无需对底层架构进行修改。该框架结合了预训练和微调的语言生成、多答案推理以及基于相似性的重排序。我们在NarrativeQA数据集上评估了所提方法，使用包括FLAN-T5（基础版和小型版）和Pegasus-Large在内的多种模型，分别在基线和微调设置下进行实验。实验结果表明，所提方法在所有模型上均能持续提高性能。特别是，FLAN-T5-基础版的整体表现最佳，从82.32%提升至86.66%（+4.34%），当与自集成推理结合时。此外，Pegasus-Large的改进幅度最大，从72.50%提升至87.07%（+14.57%），突显了所提策略的有效性。

View on arXiv Download PDF AI Translation

cs.CL / 52 / 2606.15770

ttda704 at SemEval-2026 Task 6: Structured Chain-of-Thought Prompting for Political Evasion Detection

ttda704在SemEval-2026任务6中的表现：用于政治规避检测的结构化思维链提示

Tan, Tai Tran, Thien, An Dinh

Abstract

This paper describes our system for SemEval-2026 Task 6, which addresses the classification of political evasion strategies in English question-answer pairs extracted from U.S. presidential interviews. We systematically compare two distinct paradigms: (1) Parameter-Efficient Fine-Tuning of Qwen3 models (4B-32B) using QLoRA, enhanced with tiered upsampling and weighted cross-entropy loss to address severe class imbalance, and (2) structured Chain-of-Thought (CoT) prompting of reasoning-capable API models, namely DeepSeek-V3.2 and Grok-4-Fast. Our evaluation demonstrates that structured CoT prompting of reasoning-enabled models substantially outperforms our baseline parameter-efficient fine-tuning implementation in absolute Macro F1. Our best system, Grok-4-Fast with extended reasoning and few-shot hierarchical CoT prompting, achieves a Macro F1 of 0.5147 on Subtask 2 (9-class evasion) and 0.7979 on Subtask 1 (3-class clarity), ranking 8th out of 33 teams on Subtask 2 and 13th out of 41 teams on Subtask 1 on the official leaderboard. Furthermore, our ablation studies reveal key insights into effective prompt design for evasion detection: presenting labels within a hierarchical taxonomy helps structure model reasoning, while few-shot exemplars provide task calibration. However, the strongest prompt variants are not statistically distinguishable in Macro F1, and explicitly enabling extended reasoning modes yields substantial performance gains by facilitating the multi-step pragmatic analysis required to detect evasive intent.

Chinese Translation

本文描述了我们在SemEval-2026任务6中的系统，该任务涉及对从美国总统采访中提取的英语问答对中的政治规避策略进行分类。我们系统地比较了两种不同的范式：（1）使用QLoRA对Qwen3模型（4B-32B）进行的参数高效微调，结合分层上采样和加权交叉熵损失，以解决严重的类别不平衡问题；（2）对推理能力API模型（即DeepSeek-V3.2和Grok-4-Fast）进行的结构化思维链（CoT）提示。我们的评估表明，结构化CoT提示的推理启用模型在绝对宏观F1值上显著优于我们的基线参数高效微调实现。我们最好的系统Grok-4-Fast通过扩展推理和少量示例的分层CoT提示，在子任务2（9类规避）上达到了0.5147的宏观F1，在子任务1（3类清晰度）上达到了0.7979，在官方排行榜上子任务2中排名第8，在子任务1中排名第13。此外，我们的消融研究揭示了有效的规避检测提示设计的关键见解：在分层分类法中呈现标签有助于结构化模型推理，而少量示例提供了任务校准。然而，最强的提示变体在宏观F1值上并没有统计显著差异，明确启用扩展推理模式通过促进多步骤的务实分析来检测规避意图，从而带来了显著的性能提升。

View on arXiv Download PDF AI Translation

cs.CL / 53 / 2606.15778

DYNA : Dynamic Episodic Memory Networks for Augmenting Large Language Models with Temporal Knowledge Graphs in Continuous Learning

DYNA：动态情节记忆网络通过时间知识图谱增强大规模语言模型的连续学习

Sarabadani, Ali, Tajvidiyan, Mahtab

Abstract

Large Language Models (LLMs) struggle to incorporate new knowledge without forgetting or costly retraining. We propose DYNA, a lightweight framework that augments a frozen LLM with a temporal knowledge graph where events are nodes and temporal relations are directed, timestamped edges. The graph serves as an external, updatable memory. At query time, DYNA retrieves relevant nodes via random walks and centrality measures, then augments the LLM's response. Evaluated on three temporal recall tasks, DYNA reduces catastrophic forgetting by ~7% compared to fine-tuning and improves temporal ordering by ~5% over standard RAG. Higher graph clustering coefficients correlate with better retrieval, showing that graph structure matters. Contributions: (1) episodic memory as temporal KG, (2) retraining-free LLM augmentation, (3) graph properties as predictors of retrieval performance.

Chinese Translation

大规模语言模型（LLMs）在不遗忘或高成本再训练的情况下，难以融入新知识。我们提出了DYNA，一个轻量级框架，通过时间知识图谱增强一个冻结的LLM，其中事件作为节点，时间关系作为有向、带时间戳的边。该图作为一个外部的、可更新的记忆。在查询时，DYNA通过随机游走和中心性度量检索相关节点，然后增强LLM的响应。在三个时间回忆任务上的评估表明，与微调相比，DYNA将灾难性遗忘减少了约7%，并且在标准RAG上提高了约5%的时间排序。更高的图聚类系数与更好的检索相关，显示出图结构的重要性。贡献包括：（1）作为时间知识图的情节记忆，（2）无再训练的LLM增强，（3）图属性作为检索性能的预测因子。

View on arXiv Download PDF AI Translation

cs.CL / 54 / 2606.15783

ttda704 at SemEval-2026 Task 4: Modeling Narrative Structures via Pseudonymization and Multi-View Sentence Alignment

ttda704在SemEval-2026任务4中的表现：通过伪名化和多视角句子对齐建模叙事结构

Tan, Tai Tran, Thien, An Dinh

Abstract

We present our approach to SemEval 2026 Task 4: Narrative Story Similarity and Narrative Representation Learning. Our solution uses contrastive learning with fine-tuned sentence transformers to capture narrative similarity across abstract themes, course of action, and outcomes. We develop two pipelines: (Track A) a single-view method that encodes full narratives with smart layer freezing to reduce overfitting, and (Track B) a multi-view method that models theme, plot, and outcome with view-specific projection heads and self-supervised alignment. Both pipelines build on sentence-transformers models and are trained with contrastive loss on synthetic data. The code is available at the following GitHub repository: https://github.com/dinhthienan33/SemEval2026-Task4-ttda704.

Chinese Translation

我们提出了针对SemEval 2026任务4：叙事故事相似性和叙事表示学习的解决方案。我们的方案使用对比学习与微调的句子变换器，以捕捉抽象主题、行动过程和结果之间的叙事相似性。我们开发了两个流程：（A轨道）一种单视角方法，通过智能层冻结对完整叙事进行编码，以减少过拟合；（B轨道）一种多视角方法，通过视角特定的投影头和自监督对齐来建模主题、情节和结果。这两个流程均基于句子变换器模型，并在合成数据上使用对比损失进行训练。代码可在以下GitHub仓库获取：https://github.com/dinhthienan33/SemEval2026-Task4-ttda704。

View on arXiv Download PDF AI Translation

cs.CL / 55 / 2606.15815

On Defining Erasure Harms for NLP

关于定义自然语言处理中的抹除伤害

Liu, Yu Lu, Goel, Arnav, Cheung, Jackie Chi Kit, Olteanu, Alexandra, Xiao, Ziang, Blodgett, Su Lin

Abstract

The deployment of NLP systems has raised concerns about harms they might produce, including representational harms. Recent literature has begun to conceptualize and measure one such harm, the harm of erasure. Nevertheless, the field lacks a clear and cohesive conceptual foundation for identifying and measuring erasure. Existing conceptualizations of erasure are often broad -- making it difficult to identify what is needed to establish and measure erasure -- or else specific to particular settings -- facilitating measurement for those settings but potentially challenging to adapt to other settings. To address this gap, we develop and propose a structured definition of erasure that clarifies what components are necessary for establishing whether erasure has occurred, which practitioners need to explicitly articulate and operationalize in order to measure erasure.

Chinese Translation

自然语言处理（NLP）系统的部署引发了对其可能产生的伤害的关注，包括表现性伤害。近期文献开始概念化和测量其中一种伤害，即抹除伤害。然而，该领域缺乏明确且连贯的概念基础来识别和测量抹除。现有的抹除概念化往往过于宽泛——使得识别建立和测量抹除所需的内容变得困难——或者特定于某些特定环境——便于在这些环境中进行测量，但可能在适应其他环境时面临挑战。为了解决这一空白，我们开发并提出了一个结构化的抹除定义，明确了确定抹除是否发生所需的组件，这些组件需要从业者明确表达并操作化，以便测量抹除。

View on arXiv Download PDF AI Translation

cs.CL / 56 / 2606.15821

The Truth Stays in the Family: Enhancing Contextual Grounding via Inherited Truthful Heads in Model Lineages

真相留在家族中：通过模型谱系中的继承真实头增强上下文基础

Choi, Miso, Choi, Seonga, Kwon, Mincheol, Joung, Woosung, Kim, Jinkyu, Lee, Jungbeom

Abstract

Recent advances in large language models (LLMs) have produced many specialized multimodal LLMs (MLLMs) that share common foundational LLMs, forming distinct model lineages. It remains unclear whether a fundamental behavioral link exists between the foundational LLMs and downstream variants. We investigate this question by quantifying head-level context-truthfulness scores. Across diverse LLM and MLLM lineages, including Vicuna-, Qwen2.5-, LLaMA2-, and Mistral-based models, we find that Truth Scores are strongly preserved within model families, even after instruction tuning or multimodal adaptation. We further show that this inheritance is consistent with attention-head weight preservation, and that context-truthful heads attend to query-relevant evidence. Building on this finding, we propose TruthProbe, a soft-gating strategy that amplifies context-truthful heads while preserving other head contributions. TruthProbe improves contextual truthfulness on HaluEval and reduces multimodal hallucination on POPE and CHAIR, with base-LLM Truth Scores transferring effectively to their fine-tuned LLM and MLLM descendants. Code is available at https://github.com/miso-choi/TruthProbe.

Chinese Translation

最近，大型语言模型（LLMs）的进展催生了许多专门的多模态LLMs（MLLMs），这些模型共享共同的基础LLMs，形成了不同的模型谱系。目前尚不清楚基础LLMs与下游变体之间是否存在基本的行为联系。我们通过量化头级上下文真实度评分来研究这个问题。在包括Vicuna、Qwen2.5、LLaMA2和Mistral等多种LLM和MLLM谱系中，我们发现真实评分在模型家族内得到了强有力的保留，即使在指令调优或多模态适应之后。我们进一步表明，这种继承与注意力头权重的保留是一致的，并且上下文真实的头会关注与查询相关的证据。在此基础上，我们提出了TruthProbe，一种软门控策略，能够增强上下文真实的头，同时保留其他头的贡献。TruthProbe在HaluEval上提高了上下文真实度，并在POPE和CHAIR上减少了多模态幻觉，基础LLM的真实评分有效地转移到其微调后的LLM和MLLM后代上。代码可在https://github.com/miso-choi/TruthProbe获取。

View on arXiv Download PDF AI Translation

cs.CL / 57 / 2606.15833

When Correct Edges Cannot Be Verified: A Provenance Gap in Incomplete KGQA and a Provenance-Favoring Completion Policy

当正确的边无法被验证时：不完整知识图谱问答中的溯源缺口及其溯源偏好的补全策略

Kang, Yongqi, Fu, Yu, Zhao, Yong

Abstract

Incomplete Knowledge Graph Question Answering (IKGQA) requires completing missing edges to continue reasoning. A growing line of work verifies completed edges against retrieved text, treating textual support as a proxy for edge quality. We ask a question that, to our knowledge, has not been systematically tested: does textual verifiability actually track correctness? Exploiting the gold deleted triples provided by the standard random-deletion protocol, we measure both. The finding is counterintuitive: among gold-correct completed edges, 76-96% have no supporting passage even under exhaustive retrieval, robustly across deletion rates (20%/40%), datasets (CWQ/WebQSP), and relation types (structural, commonsense, long-tail). Most Freebase-style facts simply do not occur as head-tail co-mentions in text. Textual faithfulness therefore measures provenance, not correctness -- separated by a paradigm-level gap no in-corpus retrieval closes. This reframes edge completion. Since most completed edges -- correct or not -- are causally redundant for the answer (95-97% of correct answers do not depend on any unsupported edge), the central question shifts from "is the edge correct?" to "admit or abstain under provenance uncertainty?" Within this framing we present TGComplete, a provenance-favoring admission policy that retrieves evidence at a reasoning breakpoint, verifies a candidate through a lightweight loop, and abstains when support is absent. Against the generate-to-complete baseline GoG, it attains higher edge precision against gold (15-21% vs 3-14%), with no statistically detectable EM loss and 3.1-7.4 times higher strict faithfulness of admitted edges -- at the cost of lower recall. We position TGComplete not as uniformly better, but as a principled point on a precision/provenance-recall trade-off, appropriate when auditability matters.

Chinese Translation

不完整知识图谱问答（IKGQA）需要补全缺失的边以继续推理。越来越多的研究工作通过检索文本来验证已补全的边，将文本支持视为边质量的代理。我们提出一个问题，至今为止似乎尚未系统测试：文本可验证性是否真正反映了正确性？利用标准随机删除协议提供的黄金删除三元组，我们对两者进行了测量。结果令人意外：在黄金正确的已补全边中，即使在全面检索下，76-96%也没有支持段落，这一现象在不同的删除率（20%/40%）、数据集（CWQ/WebQSP）和关系类型（结构性、常识性、长尾）中都表现出稳健性。大多数Freebase风格的事实在文本中根本没有作为头尾共同提及的情况。因此，文本的可信度测量的是溯源，而非正确性——这两者之间存在一个范式级别的缺口，任何语料库内的检索都无法弥补。这一发现重新定义了边的补全。由于大多数已补全的边——无论正确与否——对答案在因果上是冗余的（95-97%的正确答案不依赖于任何不支持的边），核心问题从“这条边正确吗？”转变为“在溯源不确定的情况下，是接受还是放弃？”在这一框架下，我们提出了TGComplete，一种偏向溯源的接受策略，它在推理的断点处检索证据，通过轻量循环验证候选项，并在缺乏支持时选择放弃。与生成-补全基线GoG相比，它在黄金数据上的边精度更高（15-21%对比3-14%），且没有统计上可检测的EM损失，接受的边的严格可信度提高了3.1-7.4倍——代价是召回率降低。我们将TGComplete定位为一个原则性的点，处于精度/溯源-召回的权衡中，适用于审计性重要的场景。

View on arXiv Download PDF AI Translation

cs.CL / 58 / 2606.15872

SciOrch: Learning to Orchestrate Expert LLMs for Solving Frontier Multimodal Scientific Reasoning Tasks

SciOrch：学习协调专家级大语言模型以解决前沿多模态科学推理任务

Guo, Jingru, Xue, Xiangyuan, Zhang, Lian, Xu, Wanghan, Chen, Siki, Torr, Philip, Ouyang, Wanli, Bai, Lei, Yin, Zhenfei

Abstract

Frontier scientific reasoning remains a major challenge for large language models (LLMs), where even the strongest commercial systems fall short of expert-level performance. A closer look at model behavior reveals substantial complementarity that single-model evaluation hides: different frontier models excel on different question types, and no single model captures the full picture. We present SciOrch, a framework that trains a lightweight 8B model to orchestrate frontier LLMs for scientific reasoning. The orchestrator decomposes each question, delegates sub-problems to selected commercial models through API calls, and synthesizes a final answer. Training such an orchestrator is fundamentally harder than conventional agentic RL: each action triggers an API call that is expensive in both dollar cost and latency, making standard online rollouts infeasible. We address this with MCTS-based approach, producing diverse orchestration trajectories, extracting per-node single-turn samples, and optimizing the orchestrator with GRPO-style training. On a 240-question test set spanning SGI-Reasoning and Scientists' First Exam, SciOrch reaches 56.66% average accuracy, outperforming the strongest single commercial model by 3.74% and the strongest multi-agent baseline by 3.33%. It also attains the best accuracy on both SGI and SFE with less than half the API cost of typical multi-agent methods.

Chinese Translation

前沿科学推理仍然是大型语言模型（LLMs）面临的一大挑战，即使是最强的商业系统也未能达到专家级的表现。对模型行为的深入分析揭示了单一模型评估所掩盖的显著互补性：不同的前沿模型在不同类型的问题上表现优异，且没有单一模型能够全面捕捉全貌。我们提出了SciOrch，一个框架，旨在训练一个轻量级的8B模型来协调前沿LLMs进行科学推理。协调器对每个问题进行分解，通过API调用将子问题委派给选定的商业模型，并综合出最终答案。训练这样的协调器在根本上比传统的代理强化学习更具挑战性：每个动作都会触发一个API调用，这在成本和延迟上都非常昂贵，使得标准的在线回放不可行。我们通过基于蒙特卡洛树搜索（MCTS）的方法来解决这个问题，生成多样化的协调轨迹，提取每个节点的单轮样本，并使用GRPO风格的训练来优化协调器。在涵盖SGI-Reasoning和科学家首次考试的240个问题测试集中，SciOrch达到了56.66%的平均准确率，超越了最强单一商业模型3.74%和最强多代理基线3.33%。它在SGI和SFE上也获得了最佳准确率，且API成本不到典型多代理方法的一半。

View on arXiv Download PDF AI Translation

cs.CL / 59 / 2606.15877

Free Energy Heuristics: Fast-And-Frugal Cognition as Active Inference Under Uncertain Precision

自由能启发式：在不确定精度下的快速而简约的认知作为主动推理

Bogdan, Alex

Abstract

Chain-of-thought (CoT) improves large language models' performance in math and symbolic reasoning. But on planning, contested ethics, and tasks where the model cannot check itself, more reasoning makes things worse. Both effects are documented; what has been missing is a principled account of which property decides the outcome. We argue it is meta-uncertainty: how unsure the model is about the reliability of its own evidence. When that uncertainty is high, extra reasoning stops adding signal and starts manufacturing false confidence. We prove that the policy minimizing expected free energy under uncertain precision stops integrating cues after a finite number of high-validity ones when the precision prior is heavy-tailed (Theorem 2.6.1), and under a Descending Dominance condition, is sample-wise identical to take-the-best (Theorem 2.7.4). Fast-and-frugal heuristics and active inference are, then, two descriptions of the same computation. The prediction is that on high-meta-uncertainty items, longer CoT should degrade accuracy. We score the regime per item (simulate-and-recover rho > 0.96), build FEH-79, a benchmark of Knightian frames with matched controls, and run a pre-registered study across seven models (five open-weight 3B-32B, two frontier), five CoT lengths, and 7,875 responses. The gate, fixed before any data, required a negative interaction with posterior probability above 0.95 and an accuracy drop of more than 6 points. It held. The high-regime drop is 17.3 points (95% CI [7.7, 25.5]); matched items with definite answers show no cost. The effect is regime-dependent: decisive in capable mid-to-large models, directional in the two frontier systems, absent-to-reversed in the weakest. The framework answers when CoT helps and unifies the Bayesian and fast-and-frugal traditions: less-is-more effects are evidence about the meta-uncertainty regime, not against Bayesian cognition.

Chinese Translation

思维链（Chain-of-thought, CoT）提高了大型语言模型在数学和符号推理中的表现。但在规划、争议伦理以及模型无法自我检查的任务中，更多的推理反而会使情况变得更糟。这两种效应都有文献记录；缺失的是一个原则性的解释，说明哪种属性决定了结果。我们认为是元不确定性：模型对自身证据可靠性的怀疑程度。当这种不确定性较高时，额外的推理不再增加信号，反而制造出虚假的自信。我们证明了在不确定精度下，最小化期望自由能的策略在有限数量的高有效性线索后停止整合线索（定理2.6.1），并且在下降主导条件下，与“取最佳”（take-the-best）在样本上是相同的（定理2.7.4）。因此，快速而简约的启发式与主动推理是同一计算的两种描述。我们的预测是，在高元不确定性项目上，较长的思维链应降低准确性。我们对每个项目进行评分（模拟与恢复 rho > 0.96），构建了 FEH-79，一个与控制组匹配的奈特框架基准，并在七个模型（五个开放权重的3B-32B，两个前沿模型）、五个思维链长度和7,875个响应上进行了预注册研究。该门控在任何数据之前固定，要求与后验概率的负交互超过0.95，并且准确性下降超过6分。结果得到了验证。高阶段的下降为17.3分（95%置信区间 [7.7, 25.5]）；具有明确答案的匹配项目没有成本。该效应依赖于阶段：在能力较强的中到大型模型中是决定性的，在两个前沿系统中是方向性的，而在最弱的模型中则缺失或反转。该框架回答了思维链何时有助于统一贝叶斯与快速简约传统：少即是多的效应是关于元不确定性阶段的证据，而不是反对贝叶斯认知。

View on arXiv Download PDF AI Translation

cs.CL / 60 / 2606.15883

Koshur Diacritizer: A Byte-Level Sequence-to-Sequence Model for Kashmiri Diacritic Restoration

Koshur Diacritizer：一种用于克什米尔语音调恢复的字节级序列到序列模型

Malik, Haq Nawaz, Nissar, Nahfid, Iqbal, Faizan

Abstract

Kashmiri, an Indo-Aryan language written in a modified Perso-Arabic script, frequently omits diacritic marks in digital text, creating ambiguity and challenging downstream NLP applications. We present Koshur Diacritizer, a ByT5-small byte-level sequence-to-sequence model for restoring diacritics in Kashmiri text. To support this task, we release a publicly available dataset of 23.7k aligned undiacritized diacritized Kashmiri sentence pairs. The proposed framework combines script-aware normalization, alignment validation, and skeleton-preserving inference to ensure reliable restoration while maintaining the original base-letter sequence. Experimental results on a held-out test set achieve a DERm of 0.2012 and a WER of 0.2159. Additionally, evaluation by a native Kashmiri linguistic expert yields a mean accuracy of 77.5%. The dataset, model, and source code are publicly released to provide a reproducible baseline for Kashmiri diacritic restoration and future low-resource language research.

Chinese Translation

克什米尔语是一种用改良的波斯-阿拉伯字母书写的印度-雅利安语言，数字文本中经常省略音调符号，这导致了歧义并对下游自然语言处理应用造成挑战。我们提出了Koshur Diacritizer，一种基于ByT5-small的字节级序列到序列模型，用于恢复克什米尔文本中的音调符号。为支持这一任务，我们发布了一个公开可用的数据集，其中包含23.7k对对齐的无音调和有音调的克什米尔句子。所提出的框架结合了脚本感知的规范化、对齐验证和保留骨架的推断，以确保在保持原始基本字母序列的同时实现可靠的恢复。在保留的测试集上的实验结果显示，DERm为0.2012，WER为0.2159。此外，由一位克什米尔语母语语言学专家进行的评估得出平均准确率为77.5%。该数据集、模型和源代码已公开发布，以提供克什米尔语音调恢复和未来低资源语言研究的可重复基准。

View on arXiv Download PDF AI Translation

cs.CL / 61 / 2606.15884

Neuron Level Analysis of Large Language Model in Legal Domain Reasoning

法律领域推理的大型语言模型神经元级分析

Onami, Eri, Ma, Youmi, Kurita, Shuhei, Okazaki, Naoaki

Abstract

We presented a neuron-level analysis of legal-domain reasoning in LLMs, comparing it with other applied domain tasks across seven open-weight models. Using neuron attribution scores to rank and suppress influential neurons, we confirmed that suppressing the identified neurons collapses accuracy on the target task, whereas suppressing the same number of random neurons does not. We further found a small subset of neurons influential across all seven tasks; once these are removed, suppressing the remaining neurons degrades only the task they were identified from, revealing genuinely task-specific neurons in every model studied. Within the legal domain, the three benchmarks exhibit relatively high neuron overlap and tend to be affected jointly, suggesting of legal components neurons that span jurisdictions. The distribution of identified neurons in our experiments suggests that the hypothesis that influential neurons are concentrated in middle MLP layers may depend on the input format and content, rather than being a universal phenomenon.

Chinese Translation

我们对大型语言模型（LLMs）在法律领域推理的神经元级分析进行了研究，并将其与七个开放权重模型中的其他应用领域任务进行了比较。通过使用神经元归因分数对影响神经元进行排名和抑制，我们确认抑制已识别的神经元会导致目标任务的准确性下降，而抑制相同数量的随机神经元则不会。我们进一步发现，在所有七个任务中，有一小部分神经元具有影响力；一旦这些神经元被移除，抑制其余神经元仅会降低它们被识别的任务的性能，揭示了每个研究模型中真正特定于任务的神经元。在法律领域，三个基准测试表现出相对较高的神经元重叠，并且往往共同受到影响，这表明存在跨法域的法律组件神经元。我们实验中识别的神经元分布表明，影响力神经元集中在中间多层感知器（MLP）层的假设可能依赖于输入格式和内容，而不是一种普遍现象。

View on arXiv Download PDF AI Translation

cs.CL / 62 / 2606.15893

BALTO: Balanced Token-Level Policy Optimization for Hallucination Mitigation

BALTO：用于幻觉缓解的平衡令牌级策略优化

Li, Ning, Guo, Zixuan, Xu, Yan, Fei, Wenbo, Niu, Yifan, Luo, Chang, Wang, Yasheng, Liu, Weiwen, Yu, Yong, Zhang, Weinan

Abstract

Hallucinations remain a major obstacle to deploying large language models (LLMs) in knowledge-intensive settings, where generated responses must be faithfully grounded in provided evidence. Reinforcement learning (RL) is a promising direction for hallucination mitigation, but response-level faithfulness rewards suffer from a granularity mismatch: localized hallucinations can cause supported content to receive spurious penalties. Although recent work introduces fine-grained feedback such as claim-level verification and token-level rewards, unbalanced credit assignment can still induce length, verbosity, or optimization-noise biases. We propose BALTO, a Balanced Token-level Policy Optimization framework for hallucination mitigation. BALTO extracts checkable factual claims, verifies them against the reference context, and projects claim-level judgments to token-level labels. A balanced token-level credit assignment mechanism is introduced into the framework. This design redistributes probability mass from unsupported content toward faithful content, rather than suppressing the entire response. We systematically analyze the limitations of response-level rewards from a theoretical standpoint, and prove BALTO's advantages in training stability and optimization efficiency for hallucination mitigation. Experiments on ConFiQA, RAGTruth, and FinLLM-Eval show that BALTO achieves the highest faithfulness across all six model--benchmark settings and consistently outperforms existing post-training baselines in Q-Score, demonstrating a stronger faithfulness--informativeness trade-off.

Chinese Translation

幻觉仍然是将大型语言模型（LLMs）部署在知识密集型环境中的主要障碍，在这些环境中，生成的响应必须忠实于提供的证据。强化学习（RL）是缓解幻觉的一个有前景的方向，但响应级忠实奖励存在粒度不匹配的问题：局部幻觉可能导致支持内容受到虚假的惩罚。尽管最近的研究引入了细粒度反馈，如主张级验证和令牌级奖励，但不平衡的信用分配仍可能导致长度、冗长或优化噪声偏差。我们提出了BALTO，一种用于幻觉缓解的平衡令牌级策略优化框架。BALTO提取可检查的事实主张，对其进行参考上下文验证，并将主张级判断投影到令牌级标签。该框架引入了一种平衡的令牌级信用分配机制。该设计将概率质量从不支持内容重新分配到忠实内容，而不是抑制整个响应。我们从理论角度系统分析了响应级奖励的局限性，并证明了BALTO在训练稳定性和优化效率方面的优势。对ConFiQA、RAGTruth和FinLLM-Eval的实验表明，BALTO在所有六个模型-基准设置中实现了最高的忠实度，并在Q-Score上持续超越现有的后训练基线，展示了更强的忠实度-信息量权衡。

View on arXiv Download PDF AI Translation

cs.CL / 63 / 2606.15903

Control-Plane Placement Shapes Forgetting: An Architectural Study of Agent Memory Across Thirteen System Configurations

控制平面布局影响遗忘：跨十三种系统配置的智能体记忆架构研究

Yang, Dongxu

Abstract

Where an LLM sits in an agent memory pipeline -- between the recall plane that retrieves stored facts (extensively benchmarked) and the control plane that mutates them via supersede, release, purge (largely untested) -- shapes which forgetting failure modes the system recovers. Comparing thirteen system configurations on a 385-case adversarial surface, we observe three placement regimes with partly complementary coverage: deterministic primitives suffice for lexical/temporal categories but fail canonicalization (5% on identifier-obfuscation, 0% on cross-lingual); inscribe-time LLM recovers canonicalization (100%) but cannot help intent-aware deletion (0% on prefix-collision and compound-fact); a mutation-time hook recovers intent-aware deletion (78-85%) and brightens nearly all categories simultaneously (91.7-93.2% overall, $0.17 per 385-case run, 2.3s/case mutation latency vs. 64-191ms/case deterministic, recall path unchanged). We expose the trade-off via ForgetEval, a 1000-case templated suite plus a 385-case adversarial layer (132 hand-crafted + 253 LLM-drafted oracle-validated) scored by deterministic substring match, paired with a six-method Adapter Protocol with honest N/A scoring that lets heterogeneous memory stores enter in 130 lines. Admission is corroborated by 10-annotator IAA (Fleiss' kappa = 0.958) and a 77-case external-authored subset (four blind contributors) that replicates the canonicalization asymmetry and amplifies the joint-placement lift (+27.8 pt). Production failures are predominantly forgetting failures rather than recall failures, yet existing benchmarks measure only recall. ForgetEval and all adapters are released under MIT.

Chinese Translation

大型语言模型（LLM）在智能体记忆管道中的位置——在检索存储事实的回忆平面（经过广泛基准测试）与通过替代、释放、清除变更这些事实的控制平面之间——决定了系统能够恢复哪些遗忘失败模式。在385个案例的对抗性表面上比较了十三种系统配置，我们观察到三种布局机制，其覆盖部分互补：确定性原语足以处理词汇/时间类别，但在规范化方面表现不佳（标识符混淆为5%，跨语言为0%）；铭刻时间的LLM能够恢复规范化（100%），但无法帮助意图感知删除（前缀冲突和复合事实均为0%）；变更时间的钩子能够恢复意图感知删除（78-85%），并几乎同时提升所有类别（总体91.7-93.2%，每385案例运行成本为0.17美元，变更延迟为2.3秒/案例，而确定性延迟为64-191毫秒/案例，回忆路径保持不变）。我们通过ForgetEval揭示了这种权衡，ForgetEval是一个包含1000个案例的模板化套件和一个385个案例的对抗层（132个手工制作 + 253个LLM草拟的经过oracle验证的案例），通过确定性子串匹配进行评分，并配合六种方法的适配器协议，采用诚实的N/A评分，使异质记忆存储能够在130行中进入。入选通过10位注释者的IAA（Fleiss' kappa = 0.958）得到证实，并且77个外部作者的子集（四位盲评贡献者）复制了规范化的不对称性并放大了联合布局的提升（+27.8个百分点）。生产失败主要是遗忘失败而非回忆失败，但现有基准仅测量回忆。ForgetEval及所有适配器均在MIT许可下发布。

View on arXiv Download PDF AI Translation

cs.CL / 64 / 2606.15910

Calibrated Triage, Not Autonomy: Confidence Estimation for Medical Vision-Language Models

校准分诊，而非自主性：医疗视觉-语言模型的置信度估计

Khanmohammadi, Reza, Thind, Kundan, Ghassemi, Mohammad M.

Abstract

A vision-language model can answer a question about a medical image fluently and confidently while barely using the image, leaning instead on language priors. In medicine this is the failure that matters most, because the answer looks trustworthy and is not, and the only protection is a confidence score reliable enough to tell the system when to abstain. We ask a deployment question rather than an accuracy one: how much imaging work a model can safely handle alone, and which confidence signal makes that possible. We evaluate seven confidence estimators across five open-weight LVLMs and three medical visual-question-answering datasets spanning broad clinical imaging, radiology, and pathology, with every probe trained only on natural images and applied without adaptation. Recast as bounded selective prediction (automate a case only when confidence clears a threshold, defer the rest), the comparison is cautionary. The standard metrics are poor guides: discrimination barely separates the methods, and the weak calibration of a cheap self-report is cheaply removed by off-domain temperature scaling without changing deployable yield. What distinguishes a usable estimator is the high-confidence region a clinician acts on: the weakest baselines are confidently wrong on 41 to 45 percent of their errors against 1 to 4 percent for the best probe, and no estimator is reliably best across domains or models. Safe handoff is governed at two levels: base-model competence sets a ceiling, so a well-calibrated score recovers roughly a third of radiology cases at a 20 percent error tolerance but almost none of pathology; the confidence layer then decides how much of that ceiling is reachable. The usable role today is calibrated triage, not autonomy: automate the cases a calibrated score marks safe, route the rest to a clinician. We release all outputs, correctness judgments, and confidence scores, with code.

Chinese Translation

视觉-语言模型能够流利且自信地回答关于医学图像的问题，而几乎不使用图像，反而依赖于语言先验。在医学领域，这种失败是最重要的，因为答案看起来值得信赖，但实际上并非如此，唯一的保护是一个足够可靠的置信度评分，以告知系统何时应当放弃。我们提出一个部署问题，而非准确性问题：模型可以安全地独立处理多少影像工作，以及哪种置信度信号使其成为可能。我们在五个开放权重的 LVLM（语言视觉大模型）和三个涵盖广泛临床影像、放射学和病理学的医学视觉问答数据集上评估了七种置信度估计器，所有探针均仅在自然图像上训练，并在未适应的情况下应用。重新表述为有限选择性预测（仅在置信度超过阈值时自动处理案例，其余则推迟），比较结果具有警示意义。标准指标的指导作用较差：区分度几乎无法区分这些方法，而便宜的自我报告的弱校准通过领域外温度缩放轻松消除，而不改变可部署的收益。可用估计器的区别在于临床医生所依据的高置信度区域：最弱的基线在其错误中有41%到45%是自信错误，而最佳探针仅为1%到4%，且没有任何估计器在不同领域或模型中始终表现最佳。安全的交接在两个层面上受到控制：基础模型的能力设定了上限，因此一个良好校准的评分在20%的错误容忍度下大约能恢复三分之一的放射学案例，但几乎无法恢复病理学案例；然后，置信度层决定了可以达到多少上限。当前可用的角色是校准分诊，而非自主性：自动处理校准评分标记为安全的案例，将其余案例转交给临床医生。我们发布所有输出、正确性判断和置信度评分，以及代码。

View on arXiv Download PDF AI Translation

cs.CL / 65 / 2606.15911

Interactor: Agentic RL oriented Iterative Creation for Ad Description Generation in Sponsored Search

Interactor：面向代理强化学习的迭代创作框架用于赞助搜索中的广告描述生成

Wei, Penghui, Wu, Jiayu, Ye, Chao, Guo, Zhi, Li, Shuanglong, Liu, Lin

Abstract

This paper focuses on automatically generating informative ad descriptions in sponsored search. Unlike ad titles which are usually optimized to attract user click feedbacks, ad descriptions have a longer text span and possess the potential of incorporating world knowledge to address user search intents while presenting the fine-grained selling points of the ads. We propose Interactor, a multi-turn iterative creation framework optimized with agentic RL for ad description generation. The generation model acts as a policy that interacts with a customized environment consisting of multiple generative reward models. Given initial generations by the policy, the customized GenRMs evaluate multi-dimensional qualities including knowledge capacity and landing page consistency, providing both binary signals and reasoning feedbacks. The policy then iteratively refines the descriptions based on such feedbacks to ensure continuous improvement. Experiments on industrial datasets show that the Interactor framework significantly outperforms state-of-the-art approaches in generating knowledge-rich and faithful ad descriptions. Since May 2026, it has been deployed online in a leading search ads system, contributing to both ad revenue and user experience.

Chinese Translation

本文聚焦于在赞助搜索中自动生成信息丰富的广告描述。与通常优化以吸引用户点击反馈的广告标题不同，广告描述具有更长的文本跨度，并具备结合世界知识以满足用户搜索意图的潜力，同时呈现广告的细粒度卖点。我们提出了Interactor，一个基于代理强化学习优化的多轮迭代创作框架，用于广告描述生成。生成模型作为一种策略，与由多个生成奖励模型组成的定制环境进行交互。在策略生成的初始描述下，定制的生成奖励模型（GenRMs）评估多维度的质量，包括知识容量和着陆页一致性，提供二元信号和推理反馈。然后，策略根据这些反馈迭代地优化描述，以确保持续改进。在工业数据集上的实验表明，Interactor框架在生成知识丰富且真实的广告描述方面显著优于现有的最先进方法。自2026年5月以来，该框架已在领先的搜索广告系统中上线，为广告收入和用户体验做出了贡献。

View on arXiv Download PDF AI Translation

cs.CL / 66 / 2606.15914

Contaminated Collaboration: Measuring Gender Bias Transfer in LLM-Assisted Student Writing

受污染的协作：测量性别偏见在大型语言模型辅助学生写作中的转移

Hossain, Ariyan, Rabbi, Kazi Kamruzzaman, Sadeque, Farig, Haque, S M Taiabul

Abstract

Gender bias in LLMs has been studied extensively in model outputs, with biased prompts shown to amplify stereotyped generations. Whether such bias propagates into text produced by humans who use these systems, however, remains underexplored. We investigate whether gender bias in an LLM writing assistant transfers into career plan essays written by students. We first verify that a gender-biased prompt induces gender-differentiated language in LLM-generated essays, while a neutral prompt does not. We then recruited participants (N = 123) in a controlled environment to write career plan essays for paired biographical profiles differing only in gender under three conditions: no AI assistance, neutral LLM assistance, or gender-biased LLM assistance. Students in the biased condition produced essays with a significantly larger agentic gap and more gender-stereotypic occupation suggestions than those in the control and neutral conditions. Our results also reveal that this bias transfer is asymmetric: agency is suppressed in female-target essays while male-target writing remains largely unaffected. Our findings highlight the risk of bias propagation in AI-assisted writing, calling for fairness-aware design in educational AI tools.

Chinese Translation

大型语言模型（LLMs）中的性别偏见在模型输出中得到了广泛研究，研究表明偏见提示会放大刻板印象生成。然而，这种偏见是否会传播到使用这些系统的人类所产生的文本中，仍然未得到充分探讨。我们研究了LLM写作助手中的性别偏见是否会转移到学生撰写的职业规划论文中。我们首先验证了性别偏见提示会在LLM生成的论文中引发性别差异化的语言，而中性提示则不会。随后，我们在受控环境中招募了参与者（N = 123），让他们在三种条件下为仅在性别上有所不同的配对传记档案撰写职业规划论文：无AI辅助、中性LLM辅助或性别偏见LLM辅助。处于偏见条件下的学生所撰写的论文表现出显著更大的代理差距和更多的性别刻板职业建议，相较于控制组和中性条件下的论文。我们的结果还揭示了这种偏见转移是非对称的：在女性目标的论文中，代理性受到抑制，而男性目标的写作则基本不受影响。我们的研究结果强调了在AI辅助写作中偏见传播的风险，呼吁在教育AI工具中进行公平性意识设计。

View on arXiv Download PDF AI Translation

cs.CL / 67 / 2606.15932

Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence

超越 NL2Code：多模态代码智能的结构化调查

Zhao, Xuanle, Sun, Qiushi, Xiao, Jingyu, Liu, Xuexin, Yang, Haoyue, Chen, Qiaosheng, Luo, Xianzhen, Huang, Jing, Zhong, Yufeng, Chen, Lei, Fu, Shuai, Wei, Zhenlin, Bi, Jinhe, Jiang, Lei, Qiu, Haibo, Yang, Siqi, Shi, Peng, Hu, Jian, Zeng, Zhixiong

Abstract

While LLMs have substantially advanced text-to-code synthesis, many real programming tasks specify intent through visual artifacts such as screenshots, charts, documents, vector drawings, videos, and interactive states. These tasks require models to connect visual perception to executable programs, because correctness depends not only on syntax but also on layout, geometry, data semantics, editability, interaction behavior, and domain-specific constraints that apply after execution. This survey examines Multimodal Code Intelligence, covering systems that generate, edit, refine, execute, or reason with code under visually grounded inputs and outputs. We first formulate the field by the role that code plays in each task, distinguishing code as a rendered artifact, an editable symbolic structure, a scientific representation, an intermediate reasoning trace, or an executable policy or tool interface. We then organize benchmarks and methods into four domains: Graphical User Interface, Scientific Visualization, Structured Graphics, and Frontier Tasks and Frameworks. This taxonomy connects mature artifact-generation problems to emerging agentic and unified settings and allows us to compare how different tasks treat evidence of correctness. Looking ahead, we argue that future research may benefit from four verification-centered directions. Multi-signal validation can combine complementary evidence of correctness, multi-state verification can test behavior across execution trajectories, cross-task transfer testing can probe reusable visual-code skills, and verifiable agent traces can reveal whether agent actions are grounded in visual evidence. Together, these directions may move multimodal code generation from single-output imitation toward evidence-grounded executable systems.

Chinese Translation

虽然大规模语言模型（LLMs）在文本到代码的合成方面取得了显著进展，但许多实际编程任务通过视觉工件（如截图、图表、文档、矢量图、视频和交互状态）来指定意图。这些任务要求模型将视觉感知与可执行程序连接起来，因为正确性不仅依赖于语法，还依赖于布局、几何、数据语义、可编辑性、交互行为以及在执行后适用的领域特定约束。本调查研究了多模态代码智能，涵盖在视觉基础输入和输出下生成、编辑、精炼、执行或推理代码的系统。我们首先通过代码在每个任务中的角色来构建该领域，区分代码作为渲染工件、可编辑符号结构、科学表示、中间推理轨迹或可执行策略或工具接口。然后，我们将基准和方法组织为四个领域：图形用户界面、科学可视化、结构化图形以及前沿任务和框架。该分类法将成熟的工件生成问题与新兴的代理和统一设置联系起来，并允许我们比较不同任务如何处理正确性的证据。展望未来，我们认为未来的研究可能受益于四个以验证为中心的方向。多信号验证可以结合互补的正确性证据，多状态验证可以测试在执行轨迹中的行为，跨任务迁移测试可以探测可重用的视觉代码技能，而可验证的代理轨迹可以揭示代理行为是否基于视觉证据。综合来看，这些方向可能将多模态代码生成从单一输出模仿推向基于证据的可执行系统。

View on arXiv Download PDF AI Translation

cs.CL / 68 / 2606.15949

FinBalance: A Multi-Document Accounting Reconciliation Benchmark

FinBalance：一个多文档会计对账基准

Tumpati, Sasank, Agarwal, Devansh, Kedia, Ayush, Neekhra, Arjun, Mandal, Murari, Garg, Krishna, Sinha, Yash, Gupta, Suman, Kumar, Dhruv

Abstract

Existing financial-NLP benchmarks mostly evaluate prepared artifacts such as filings, tables, or extracted values. Real accounting begins earlier: source documents must be reconciled into cited journal entries, aggregated into a balance sheet, and checked for contradictions. We introduce FinBalance, a multi-document accounting reconciliation benchmark built from source-document bundles across eight industries, three period types, and five difficulty levels. Human-authored business scenarios, accounting policies, tax/FX treatments, document schemas, distractors, and inconsistency templates are composed by a deterministic generator whose ledger produces journal entries,balance sheets, and 23 inconsistency-code labels. On a 710-record evaluation split, six contemporary LLMs reach at most 46% exact final-balance-sheet accuracy. Four models show a 26-41 pp gap between BS_exact, the model's reported balance sheet, and BS_recon, the balance sheet obtained by replaying its entries through our ledger. Models often recover numerically plausible entries but fail to bind them to supporting documents and aggregate them consistently. Citation-pressure prompting barely changes document-linking errors, while ledger-feedback ablations substantially improve reported balance sheets and expose inconsistency-detection trade-offs. Expert finance reviewers validate the benchmark design and labels.

Chinese Translation

现有的金融自然语言处理基准主要评估准备好的文档，如申报、表格或提取的数值。真实的会计工作开始得更早：源文档必须被对账为引用的日记条目，汇总为资产负债表，并检查是否存在矛盾。我们介绍了FinBalance，一个基于来自八个行业、三种时期类型和五个难度级别的源文档包构建的多文档会计对账基准。人类撰写的商业场景、会计政策、税务/外汇处理、文档架构、干扰项和不一致性模板由一个确定性生成器构成，该生成器的账本生成日记条目、资产负债表和23个不一致性代码标签。在一个710条记录的评估分割中，六个当代大型语言模型的最终资产负债表准确率最高仅为46%。四个模型在BS_exact（模型报告的资产负债表）和BS_recon（通过我们的账本重放其条目获得的资产负债表）之间显示出26-41个百分点的差距。模型通常能够恢复数值上合理的条目，但未能将其与支持文档绑定并一致地汇总。引用压力提示几乎没有改变文档链接错误，而账本反馈消融显著改善了报告的资产负债表，并揭示了不一致性检测的权衡。专家金融审阅者验证了基准设计和标签的有效性。

View on arXiv Download PDF AI Translation

cs.CL / 69 / 2606.15971

SAG: SQL-Retrieval Augmented Generation with Query-Time Dynamic Hyperedges

SAG：带有查询时动态超边的SQL检索增强生成

Wu, Yuchao, Li, Junqin, Liang, XingCheng, Chen, Yongjie, Liang, Yinghao, Mo, Linyuan, Li, Guanxian

Abstract

Retrieval-Augmented Generation (RAG) offers an effective approach for large language models to access external knowledge. However, existing methods rely on dense similarity retrieval and face inherent limitations in handling structured constraints and multi-hop reasoning. Incorporating knowledge graphs partially alleviates these issues, but at the cost of semantic fragmentation, high maintenance overhead, and difficult incremental updates. This paper introduces SAG (SQLRetrieval Augmented Generation), a structured architecture for retrieval and agent systems. Instead of pre-building a global static graph, SAG converts each chunk into one semantically complete event and a set of indexing entities, then uses SQL join queries to dynamically link events that share entities into local hyperedges,constructing, at query time, a dynamically instantiated local index structure. This design avoids the need for global graph rebuilding and ongoing maintenance; the system naturally supports incremental writes, concurrent processing, and continuous scaling through its reliance on standard database infrastructure. Across HotpotQA, 2WikiMultiHop, and MuSiQue, three standard multi-hop benchmarks,SAG achieves the best results on 8 out of 9 Recall@K metrics, reaching 80.0% Recall@5 on MuSiQue, the benchmark with the highest multi-hop reasoning demands.SAG has also been deployed at a production scale of hundreds of millions of data items, with online retrieval latency kept within seconds. Project site and code are available at https://github.com/Zleap-AI/SAG-Benchmark.

Chinese Translation

检索增强生成（RAG）为大型语言模型访问外部知识提供了一种有效的方法。然而，现有方法依赖于密集相似性检索，并在处理结构化约束和多跳推理方面面临固有的限制。结合知识图谱在一定程度上缓解了这些问题，但代价是语义碎片化、高维护开销和难以进行增量更新。本文介绍了SAG（SQL检索增强生成），一种用于检索和代理系统的结构化架构。SAG并不预先构建一个全局静态图，而是将每个数据块转换为一个语义完整的事件和一组索引实体，然后使用SQL连接查询动态链接共享实体的事件，构建在查询时动态实例化的本地索引结构。该设计避免了全局图重建和持续维护的需求；系统自然支持增量写入、并发处理和通过依赖标准数据库基础设施的持续扩展。在HotpotQA、2WikiMultiHop和MuSiQue这三个标准多跳基准测试中，SAG在9个Recall@K指标中取得了8个最佳结果，在对多跳推理要求最高的基准MuSiQue上达到了80.0%的Recall@5。SAG还已在数亿数据项的生产规模上部署，在线检索延迟保持在秒级。项目网站和代码可在https://github.com/Zleap-AI/SAG-Benchmark获取。

View on arXiv Download PDF AI Translation

cs.CL / 70 / 2606.15972

Formalize Once, Edit the Rest: Efficient Lean-Based Answer Selection for Math Reasoning

一次形式化，其余编辑：基于Lean的高效数学推理答案选择

Feng, Ji, Shi, Zhouxing

Abstract

With large language models (LLMs) increasingly applied to mathematical reasoning, formal proof assistants such as Lean can be leveraged to verify reasoning outputs with machine-checkable rigor, enabling use cases such as answer selection in test-time scaling with K sampled candidate answers. However, employing Lean requires that LLM outputs, originally in natural language, first be formalized. Existing Lean-based answer-selection work uses an autoformalization model to generate a formal statement in Lean for each candidate answer independently, incurring a significant computational cost. We propose BASE, a base-and-edit pipeline that formalizes a single base candidate per problem and derives the remaining K-1 statements by editing the answer expression in place. To facilitate this, we train a rewriter model LEANSCRIBE to localize the answer in the base formalization and generate a reusable edit function for the other K-1 candidates. BASE simultaneously improves selection accuracy and reduces formalization cost - a Pareto improvement that holds on all 12 (dataset, solver) configurations across four benchmarks and three solvers, cutting autoformalizer calls by about 5x at K=8, with the reduction expected to become larger as K grows. Code is available at https://github.com/ucr-rai/base-and-edit.

Chinese Translation

随着大型语言模型（LLMs）在数学推理中的应用日益增多，形式化证明助手如Lean可以被用来以机器可验证的严格性验证推理输出，从而实现诸如在测试时使用K个候选答案进行答案选择等用例。然而，使用Lean要求LLM的输出（最初为自然语言）首先被形式化。现有的基于Lean的答案选择工作使用自动形式化模型为每个候选答案独立生成一个Lean中的形式化陈述，这会产生显著的计算成本。我们提出了BASE，一个基础与编辑的管道，它为每个问题形式化一个基础候选答案，并通过就地编辑答案表达式推导出其余的K-1个陈述。为此，我们训练了一个重写模型LEANSCRIBE，以定位基础形式化中的答案并为其他K-1个候选生成可重用的编辑函数。BASE同时提高了选择准确性并降低了形式化成本——在四个基准和三个求解器的12个（数据集，求解器）配置中实现了帕累托改进，在K=8时将自动形式化调用减少了约5倍，预计随着K的增加，这一减少将变得更大。代码可在https://github.com/ucr-rai/base-and-edit获取。

View on arXiv Download PDF AI Translation

cs.CL / 71 / 2606.15974

A Large-Scale Multi-Dimensional Empirical Study of LLMs for Conversation Summarization

大规模多维度对话摘要的 LLM 实证研究

Zhou, Weixiao, Li, Gengyao, Cheng, Xianfu, Zhu, Junnan, Zhai, Feifei, Li, Zhoujun

Abstract

Despite the significant advancement of LLMs in conversation summarization, their evaluation remains limited by insufficient scenarios, input lengths, and sample sizes. Furthermore, existing benchmarks often omit frontier reasoning systems and efficient small models, or lack fine-grained, multi-dimensional assessments. To bridge these gaps, we propose OmniCSEval, a unified benchmark comprising 1,800 diverse conversations across six real-world scenarios, featuring context lengths ranging from 128 to 32k tokens. For fine-grained evaluation, we employ a bidirectional fact-checking framework that integrates key fact matching to assess completeness and conciseness, alongside summary fact verification to evaluate faithfulness. To ensure reliable assessment, we establish a human-LLM collaborative pipeline for key fact extraction and a multi-LLM consensus verifier for summary fact decomposition. Leveraging this framework, we evaluate 28 LLMs across four distinct categories grouped by reasoning capability and model scale. Our extensive empirical study reveals critical insights regarding the cross-scenario challenges current LLMs continue to face, the impacts of reasoning and scale, and the efficiency and adaptability of reasoning models. We also provide guidance for system selection in real-world deployments.

Chinese Translation

尽管 LLM（大语言模型）在对话摘要方面取得了显著进展，但其评估仍受到场景不足、输入长度限制和样本规模小的制约。此外，现有基准往往忽略前沿推理系统和高效的小型模型，或缺乏细粒度的多维度评估。为了解决这些问题，我们提出了 OmniCSEval，这是一个统一的基准，包含 1,800 个来自六个真实场景的多样化对话，背景长度从 128 到 32,000 个标记不等。为了进行细粒度评估，我们采用了一个双向事实核查框架，该框架结合了关键事实匹配，以评估摘要的完整性和简洁性，同时通过摘要事实验证来评估其忠实度。为了确保评估的可靠性，我们建立了一个人类-LLM 协作管道用于关键事实提取，并设立了一个多 LLM 共识验证器用于摘要事实分解。利用这一框架，我们对 28 个 LLM 进行了评估，这些模型根据推理能力和模型规模分为四个不同类别。我们的大规模实证研究揭示了当前 LLM 在跨场景挑战中面临的关键问题、推理与规模的影响，以及推理模型的效率和适应性。我们还为实际部署中的系统选择提供了指导。

View on arXiv Download PDF AI Translation

cs.CL / 72 / 2606.15984

ROMPAR: Morphological Completion and Demographic Unlearning for Romanian-Accented Speech Recognition

ROMPAR：罗马尼亚口音语音识别中的形态补全与人口去学习

Avram, Andrei-Marius, Antonie, Aureliu-Valentin, Badea, Ştefan-Bogdan, Florea, Andrei, Zaharoiu, Robert-Nicolae, Cercel, Dumitru-Clementin

Abstract

Automated transcription of parliamentary proceedings faces significant hurdles due to demographic bias, dialectal variation, and technical artifacts such as utterance truncation during segmentation. This paper introduces the ROManian PARliamentary Speech Corpus (ROMPAR) dataset, a 17.80-hour corpus of Romanian and Moldavian parliamentary speech, featuring double-annotated ground truth and explicit labels for reconstructed word fragments. To build a robust ASR system, we propose a multi-task adversarial training framework that enforces demographic invariance across age, gender, and dialect. We address the inherent instability of adversarial objectives in generative architectures by introducing an exponential decay mechanism for the adversarial coefficients. Furthermore, we implement an LLM-guided decoding strategy with position-dependent weighting to facilitate morphological completion of truncated terminal words. Our results demonstrate that the proposed framework significantly reduces WER and achieves an F1-score of 96.6% in morphological reconstruction.

Chinese Translation

自动转录议会程序面临着由于人口偏见、方言变异和技术伪影（如分段时的发言截断）等显著障碍。本文介绍了罗马尼亚议会语音语料库（ROManian PARliamentary Speech Corpus，ROMPAR），这是一个包含17.80小时罗马尼亚和摩尔多瓦议会发言的语料库，具有双重注释的真实标签和明确的重构词片段标签。为了构建一个稳健的自动语音识别（ASR）系统，我们提出了一种多任务对抗训练框架，该框架在年龄、性别和方言之间强制执行人口不变性。我们通过引入对抗系数的指数衰减机制来解决生成架构中对抗目标的固有不稳定性。此外，我们实施了一种基于大型语言模型（LLM）引导的解码策略，采用位置依赖加权，以促进截断终端词的形态补全。我们的结果表明，所提出的框架显著降低了字错误率（WER），并在形态重构中达到了96.6%的F1-score。

View on arXiv Download PDF AI Translation

cs.CL / 73 / 2606.16000

GRACE-DS: a Guarded Reward-guided Agent Correction Environment in Data Science

GRACE-DS：数据科学中的受保护奖励引导代理纠正环境

Tsymbalov, Aleksandr, Zaripov, Danis, Epifanov, Artem, Palienko, Anastasya

Abstract

We introduce GRACE-DS, a Guarded Reward-guided Agent Correction Environment in Data Science for pre-deployment evaluation of LLM-powered AutoML agents. GRACE-DS is a set of evaluation metrics in an isolated environment that can be applied to tabular ML tasks specific to a particular organization. It exposes agents to realistic workflow stages, from planning and data inspection through feature engineering, model development, validation, and code repair to final submission, while hidden executable validators measure not only final predictive performance but also leakage avoidance, reproducibility, protocol validity, correction behavior, and reward alignment. The strongest structured regime, flexible iterative interaction (our approach), achieves higher end-to-end normalized hidden-test quality than single-shot generation, unstructured interaction, and restart-based baselines, while also improving protocol-valid completion. Validated across more than 7,000 episodes, these results establish GRACE-DS as a robust platform for assessing the capacity of LLM-based AutoML agents to execute machine learning workflows under production-like conditions and in accordance with organization-specific requirements.

Chinese Translation

我们介绍了GRACE-DS，一个用于大规模语言模型（LLM）驱动的自动机器学习（AutoML）代理预部署评估的受保护奖励引导代理纠正环境。GRACE-DS是一组在隔离环境中应用于特定组织的表格机器学习任务的评估指标。它使代理暴露于现实的工作流程阶段，从规划和数据检查到特征工程、模型开发、验证、代码修复以及最终提交，同时隐藏的可执行验证器不仅测量最终预测性能，还评估泄漏避免、可重复性、协议有效性、纠正行为和奖励对齐。最强的结构化机制——灵活的迭代交互（我们的方法）在端到端的标准化隐藏测试质量上优于单次生成、非结构化交互和基于重启的基线，同时也改善了协议有效的完成。经过超过7000个实验的验证，这些结果确立了GRACE-DS作为一个强大的平台，用于评估基于LLM的AutoML代理在生产类条件下执行机器学习工作流程的能力，并符合特定组织的要求。

View on arXiv Download PDF AI Translation

cs.CL / 74 / 2606.16009

Bridging the Usability Gap: Lessons from Interpreting Studies for Machine Interpreting Design

弥合可用性差距：从口译研究中汲取的机器口译设计经验

Fantinuoli, Claudio

Abstract

Machine interpreting (MI), the live, real-time branch of speech translation, has achieved remarkable progress on standard benchmarks, with some systems approaching human parity on textual fidelity. Yet the user experience remains far inferior to interpreter-mediated communication, revealing what we term the \emph{accuracy illusion}: systems that appear accurate on paper but fail in practice to support smooth, goal-oriented interaction. This paper defines MI as a distinct subfield of speech translation, with its own characteristics and the need for evaluation methods grounded in communicative effectiveness rather than isolated fidelity metrics. Drawing on insights from interpreting studies, we identify critical dimensions of professional interpreting practice that are overlooked by current systems, and consolidate them into three interdependent design priorities for future MI: \emph{agency} (context-sensitive initiative and repair), \emph{grounding} (multimodal and discourse-level situational awareness), and \emph{experience} (adaptive improvement through real interaction). Together, these priorities chart a path toward closing the usability gap and enabling systems that can sustain authentic multilingual communication in real time.

Chinese Translation

机器口译（Machine Interpreting, MI）作为语音翻译的实时分支，在标准基准测试中取得了显著进展，一些系统在文本忠实度上接近人类水平。然而，用户体验仍然远不及由口译员介导的交流，揭示了我们所称之为 extit{准确性幻觉}的问题：在纸面上看似准确的系统在实践中却未能支持流畅、以目标为导向的互动。本文将机器口译定义为语音翻译的一个独特子领域，具有自身特征，并需要基于交流有效性的评估方法，而非孤立的忠实度指标。借鉴口译研究的见解，我们识别出当前系统忽视的专业口译实践的关键维度，并将其整合为未来机器口译的三个相互依赖的设计优先事项： extit{代理性}（上下文敏感的主动性和修复能力）、 extit{基础性}（多模态和话语层面的情境意识）以及 extit{体验}（通过真实互动的适应性改进）。这些优先事项共同指明了一条弥合可用性差距的路径，使系统能够在实时中维持真实的多语言交流。

View on arXiv Download PDF AI Translation

cs.CL / 75 / 2606.16011

Who Flips? Self- and Cross-Model Counterarguments Reveal Answer Instability in LLMs

谁会翻转？自我与交叉模型反驳揭示大型语言模型中的答案不稳定性

Nikeghbal, Nafiseh, Kargaran, Amir Hossein, Kolli, Shaghayegh, Diesner, Jana

Abstract

Standard accuracy benchmarks are designed to test how closely large language models (LLMs) approach correct answers, but are not suitable for testing whether LLMs stick with a correct answer when that answer is challenged by a plausible counter-argument. We introduce a controlled protocol for evaluating answer stability: after a model answers a multiple-choice question correctly, we challenge the model's answer with a coherent argument for an incorrect option and measure whether the model flips. The setup a) isolates argumentative content from overt social pressure and b) varies argument length, self-attribution, and cross-model source. Across seven frontier models and 57 MMLU subjects, flip rates range from 17.5% to 97.3%, revealing large differences in stability that are not captured by accuracy metrics alone. We find that self-attribution consistently increases flip rates (mean +7.1pp, up to +18.7pp). Also, pooling wrong-answer arguments across models and selecting the most effective one per question yields stronger adversarial challenges than relying on any single source model. We further construct MaxFlip, a curated challenge set that amplifies flips by up to +23.6pp over standard self-generated challenges. We release the protocol, challenge records, and MaxFlip to support stability evaluation alongside standard accuracy benchmarks. Materials are available at https://github.com/nafisenik/WhoFlips and https://hf.co/datasets/nafisehNik/WhoFlips.

Chinese Translation

标准准确性基准旨在测试大型语言模型（LLMs）接近正确答案的程度，但不适合测试当正确答案受到合理反驳挑战时，LLMs是否坚持该答案。我们引入了一种控制协议来评估答案稳定性：在模型正确回答多项选择问题后，我们用一个连贯的错误选项论证来挑战模型的答案，并测量模型是否翻转。该设置一方面将论证内容与明显的社会压力隔离，另一方面变化论证长度、自我归因和交叉模型来源。在七个前沿模型和57个MMLU主题中，翻转率范围从17.5%到97.3%，揭示了稳定性方面的巨大差异，这些差异仅通过准确性指标无法捕捉。我们发现，自我归因始终会提高翻转率（平均增加7.1个百分点，最高可达18.7个百分点）。此外，跨模型汇总错误答案论证并为每个问题选择最有效的一个，产生的对抗性挑战比依赖任何单一源模型更强。我们进一步构建了MaxFlip，一个策划的挑战集，其翻转率比标准自生成挑战提高了最多23.6个百分点。我们发布了该协议、挑战记录和MaxFlip，以支持与标准准确性基准一起的稳定性评估。材料可在https://github.com/nafisenik/WhoFlips和https://hf.co/datasets/nafisehNik/WhoFlips获取。

View on arXiv Download PDF AI Translation

cs.CL / 76 / 2606.16019

Scaling Human and G2P Supervision for Robust Phonetic Transcription

扩展人类与G2P监督以实现稳健的音素转录

Metzger, Alexander, Srivastava, Aruna, Mukhamedvaleev, Ruslan

Abstract

Expert phonetic annotation is costly, especially for non-standard dialects and atypical speech. A common alternative is using Grapheme-to-Phoneme (G2P) models to auto-generate phonetic labels from text transcripts at scale. We study how automatic phonetic transcription performance scales with human and G2P supervision in English. Using a curated 80-hour benchmark spanning native, non-native and post-stroke speech, we identify a supervision quality threshold: G2P supervision helps only when fewer than 20-30 hours of human annotation are available. Beyond this threshold, it provides no significant benefit and can reduce cross-dialect robustness. What is effective after this threshold is ASR pretraining which we use to achieve a 2.3x reduction in weighted phone feature error rate over prior systems, with strong gains on non-native and aphasic speech. These results suggest that quantity-driven G2P scaling may yield diminishing returns for robust generalization.

Chinese Translation

专家音素标注成本高昂，尤其对于非标准方言和非典型言语。一个常见的替代方案是使用字母到音素（Grapheme-to-Phoneme, G2P）模型，从文本转录中自动生成音素标签。我们研究了在英语中，自动音素转录性能如何随着人类和G2P监督的增加而变化。通过使用一个涵盖母语、非母语和中风后言语的80小时精心策划的基准数据集，我们确定了一个监督质量阈值：当可用的人类标注少于20-30小时时，G2P监督才会有所帮助。超过这个阈值后，它并没有显著的益处，反而可能降低跨方言的鲁棒性。在此阈值之后，ASR预训练变得有效，我们利用这一点实现了相较于之前系统在加权音素特征错误率上降低2.3倍的成果，并在非母语和失语症言语上取得了显著提升。这些结果表明，基于数量驱动的G2P扩展可能会导致稳健泛化的收益递减。

View on arXiv Download PDF AI Translation

cs.CL / 77 / 2606.16026

In-Domain Supervised Pathology Report Classification: A Reproducible Pipeline from Data Curation to Production-Matched Evaluation

领域内监督病理报告分类：从数据整理到生产匹配评估的可重复管道

Hands, Isaac, Huang, Bin, Spannaus, Adam, Gounley, John, Hanson, Heidi, Durbin, Eric, Ellingson, Sally R.

Abstract

We introduce an in-domain supervised pipeline designed to counter the out-of-distribution performance drop that hampers supervised biomedical NLP models, a problem observed when models trained on pathology reports are moved across cancer registries. Our contribution is a reproducible recipe for training a supervised classifier from routinely collected cancer registry data. It describes how to build the in-domain training set and a production-matched holdout, and to choose operating points that keep the false-negative rate (FNR) very low while keeping reviewer workload manageable. The pipeline standardizes data curation with facility-stratified sampling and separate handling of reports linked to registry cases, and includes a blinded manual audit to estimate positive-case prevalence and label noise. On a 418k-report holdout set, the Kentucky model achieved FNR 0.003 and false-positive rate (FPR) 0.097, improving over the Seattle-trained MOSSAIC OncoID baseline (FNR 0.010, FPR 0.183) and raising F1 from 0.860 to 0.922. In a blinded manual review of 600 reports, estimated positive prevalence declined from 0.500 to 0.398, indicating substantial label noise with errors concentrated in rare primary sites.

Chinese Translation

我们介绍了一种领域内监督管道，旨在应对监督生物医学自然语言处理模型的分布外性能下降问题，这一问题在将训练于病理报告的模型迁移到不同癌症登记处时尤为明显。我们的贡献是提供了一种可重复的训练监督分类器的方案，基于常规收集的癌症登记数据。该方案描述了如何构建领域内训练集和生产匹配的保留集，以及如何选择操作点，以保持假阴性率（FNR）非常低，同时确保审阅者的工作量可控。该管道通过设施分层抽样和对与登记案例相关的报告进行单独处理来标准化数据整理，并包括盲审手动审核，以估计阳性案例的流行率和标签噪声。在418,000份报告的保留集中，肯塔基模型实现了FNR 0.003和假阳性率（FPR）0.097，相较于西雅图训练的MOSSAIC OncoID基线（FNR 0.010，FPR 0.183）有所改善，F1从0.860提升至0.922。在对600份报告的盲审手动审核中，估计的阳性流行率从0.500下降至0.398，表明存在显著的标签噪声，错误主要集中在稀有的原发部位。

View on arXiv Download PDF AI Translation

cs.CL / 78 / 2606.16047

From Argument Components to Graphs: A Multi-Agent Debate with Confidence Gating for Argument Relations

从论证组件到图：一种具有置信门控的多智能体辩论用于论证关系

Bąba, Jakub, Chudziak, Jarosław A.

Abstract

Large Language Models (LLMs) are increasingly assessed and utilized in the field of Argument Mining (AM), thanks to their strong general reasoning capabilities. However, standard training-free models often miss sophisticated details, specifically in contexts where two parts of the text have to be analyzed together. Furthermore, self-correction mechanisms tend to reinforce initial hallucinations in reasoning. Overcoming these limitations typically requires expensive, domain-specific supervised fine-tuning. Recent work has shown that a multi-agent paradigm can address such weaknesses for the component classification task through dialectical refinement with a Proponent-Opponent-Judge architecture, setting a promising direction for training-free approaches in the field. In this paper, we extend and evaluate this framework on the Argument Relation Identification and Classification (ARIC) task, reformulating it as a debate over component pairs. Besides that, we introduce a confidence gating mechanism that enables debating only on the uncertain cases and accepting the initial prediction when confidence is high. On the UKP Argument Annotated Essays v2 corpus, we demonstrate that the selective debate achieves the highest Macro F1 among all training-free methods, while debate over all samples degrades performance below that of one of the baselines. All generative approaches also outperform fine-tuned RoBERTa models on Macro F1, suggesting that the under-representation of the Attack class was more damaging to supervised fine-tuning than to inference-only models. Additionally, our framework produces human-readable debate transcripts, offering interpretability absent from both single-agent and supervised classifiers.

Chinese Translation

大型语言模型（LLMs）因其强大的通用推理能力，在论证挖掘（AM）领域越来越受到评估和应用。然而，标准的无训练模型往往忽略复杂的细节，特别是在需要将文本的两个部分一起分析的上下文中。此外，自我修正机制往往会强化推理中的初始幻觉。克服这些限制通常需要昂贵的特定领域监督微调。近期的研究表明，多智能体范式可以通过支持者-反对者-评判者（Proponent-Opponent-Judge）架构，通过辩证精炼来解决组件分类任务中的这些弱点，为该领域的无训练方法指明了一个有希望的方向。在本文中，我们扩展并评估了该框架在论证关系识别与分类（ARIC）任务上的应用，将其重新表述为对组件对的辩论。此外，我们引入了一种置信门控机制，使得仅在不确定的案例上进行辩论，并在置信度高时接受初始预测。在UKP论证注释论文v2语料库上，我们证明选择性辩论在所有无训练方法中达到了最高的宏观F1分数，而对所有样本的辩论则使性能降至低于某些基线的水平。所有生成方法在宏观F1上也优于微调的RoBERTa模型，这表明攻击类的不足代表性对监督微调的影响比对仅推理模型的影响更为严重。此外，我们的框架生成了可读的人类辩论记录，提供了单智能体和监督分类器所缺乏的可解释性。

View on arXiv Download PDF AI Translation

cs.CL / 79 / 2606.16074

PVminerLLM2: Improving Structured Extraction of Patient Voice via Preference Optimization

PVminerLLM2：通过偏好优化改善患者声音的结构化提取

Fodeh, Samah, Ma, Linhai, Puthiaraju, Ganesh, Talakokkul, Srivani, Khan, Afshan, Irankhah, Elyas, Ramachandran, Sreeraj, Hagaman, Ashley, Lowe, Sarah, Roundtree, Aimee

Abstract

Motivation: Patient-generated text contains critical information on patients' lived experiences, social context, and care engagement, but remains largely unstructured, limiting its use in patient-centered outcomes research. Prior work introduced the PV-Miner benchmark and PVMinerLLM models for structured extraction. However, supervised fine-tuning (SFT) alone struggles with rare, fine-grained, and unevenly distributed errors, particularly in token-critical structured outputs. Results: We present PVminerLLM2, an improved set of LLMs for structured patient voice extraction that applies preference optimization to address token-critical errors beyond the reach of supervised fine-tuning. Our method introduces (i) a preference objective with token-level gated stabilization term that prevents degradation of absolute token likelihood under preference optimization, and (ii) confusion-aware preference pair construction to better capture low-separation distinctions. We further incorporate token-importance weighting and inverse-frequency reweighing to address token imbalance and class skew. Across multiple model sizes, PVMinerLLM2 consistently outperforms strong baselines, achieving gains of up to 4.43% (Code), 3.50% (Sub-code), and 1.55% (Span), and outperforms baseline LLM trained with existing preference optimization methods. Availability and Implementation: The supplementary material, code, evaluation scripts, and trained models for PVminerLLM2 are publicly available at: https://github.com/Data-Mining-Lab-Yale/PVminerLLM2

Chinese Translation

动机：患者生成的文本包含关于患者生活经历、社会背景和护理参与的重要信息，但仍然大多是非结构化的，限制了其在以患者为中心的结果研究中的应用。先前的工作引入了PV-Miner基准和PVMinerLLM模型用于结构化提取。然而，单靠监督微调（SFT）在处理稀有、细粒度和分布不均的错误时面临挑战，特别是在对标记关键的结构化输出中。结果：我们提出了PVminerLLM2，这是一组改进的LLM，用于结构化患者声音提取，应用偏好优化以解决超出监督微调范围的标记关键错误。我们的方法引入了（i）一个带有标记级门控稳定项的偏好目标，该项防止在偏好优化下绝对标记似然的降级，以及（ii）混淆感知的偏好对构建，以更好地捕捉低分离度的区别。我们进一步结合了标记重要性加权和逆频率重加权，以解决标记不平衡和类别偏斜。在多个模型规模上，PVMinerLLM2始终优于强基线，取得了高达4.43%（代码）、3.50%（子代码）和1.55%（跨度）的提升，并且优于使用现有偏好优化方法训练的基线LLM。可用性和实现：PVminerLLM2的补充材料、代码、评估脚本和训练模型可在以下网址公开获取：https://github.com/Data-Mining-Lab-Yale/PVminerLLM2

View on arXiv Download PDF AI Translation

cs.CL / 80 / 2606.16093

Long-Context Modeling via GSS-Transformer Hybrid Architecture with Learnable Mixing

通过可学习混合的GSS-Transformer混合架构进行长上下文建模

Torlak, Kuzey, Arslan, Hüseyin Arda, Dervişoğlu, Anıl, Deniz, Beyza Nur, Boyar, Onur

Abstract

Modeling long-range dependencies remains a central challenge in natural language processing. Transformer architectures achieve strong performance via self-attention but scale quadratically ($O(N^2)$) with sequence length, while State Space Models (SSMs) scale linearly ($O(N)$) but suffer from a selective recall bottleneck, struggling to retrieve precise information from compressed states. This creates a fundamental tradeoff between efficiency and perplexity. To tackle these challenges, we propose the \textit{Parallel Hybrid Architecture (PHA)}, which runs Gated State Spaces (GSS), Grouped Query Attention (GQA), and Feed-Forward Networks (FFNs) as independent parallel branches fused by a learnable mixing mechanism. Instead of forcing SSMs to approximate attention or serializing the two paradigms, PHA allows each branch to specialize: GSS captures global context, while attention performs selective retrieval, with FFN providing complementary processing. On WikiText-103, PHA achieves 16.51 PPL at 125M parameters, outperforming Hedgehog (16.70) and H3-125M (23.70). Scaling to 180M parameters yields 16.42 PPL, which gives comparable results with the pure attention baseline while delivering 24\% higher throughput and up to 40\% lower memory usage at long contexts. On OpenWebText, our 125M model achieves 19.72 PPL, outperforming standard Transformers (20.60) and GSS hybrid baselines (19.80). These results demonstrate that separating sequence modeling paradigms into parallel specialists enables Transformer-level perplexity with substantially improved efficiency for long-context language modeling.

Chinese Translation

建模长距离依赖关系仍然是自然语言处理中的一个核心挑战。Transformer架构通过自注意力机制实现了强大的性能，但其计算复杂度随序列长度呈二次增长（$O(N^2)$），而状态空间模型（State Space Models, SSMs）则以线性（$O(N)$）的方式扩展，但在选择性回忆方面存在瓶颈，难以从压缩状态中检索精确信息。这在效率与困惑度之间形成了根本的权衡。为了解决这些挑战，我们提出了 extit{并行混合架构（Parallel Hybrid Architecture, PHA）}，该架构将门控状态空间（Gated State Spaces, GSS）、分组查询注意力（Grouped Query Attention, GQA）和前馈网络（Feed-Forward Networks, FFNs）作为独立的并行分支，通过可学习的混合机制进行融合。PHA允许每个分支专门化，而不是强迫SSMs近似注意力或将这两种范式串行化：GSS捕捉全局上下文，而注意力机制执行选择性检索，FFN则提供补充处理。在WikiText-103数据集上，PHA在125M参数下实现了16.51的困惑度，超越了Hedgehog（16.70）和H3-125M（23.70）。在参数扩展到180M时，PHA达到了16.42的困惑度，这与纯注意力基线相当，同时在长上下文中提供了24 ext{%}更高的吞吐量和高达40 ext{%}的内存使用降低。在OpenWebText数据集上，我们的125M模型达到了19.72的困惑度，超越了标准的Transformer（20.60）和GSS混合基线（19.80）。这些结果表明，将序列建模范式分离为并行专家能够实现Transformer级别的困惑度，同时在长上下文语言建模中显著提高效率。

View on arXiv Download PDF AI Translation

cs.CL / 81 / 2606.16111

Towards Pareto-Optimal Tool-Integrated Agents with Pareto Ranking Policy Optimization

朝向帕累托最优的工具集成代理及其帕累托排名策略优化

Li, Junyi, Qian, Xiaowei, Zhang, Yingyi, Zhang, Wenlin, Li, Guojing, Zhang, Sheng, Han, Xiao, Wang, Yichao, Zhao, Xiangyu

Abstract

Recent advances in tool-integrated language agents have significantly improved their ability to solve complex reasoning tasks. However, existing alignment methods predominantly focus on maximizing task accuracy, while overlooking auxiliary objectives such as tool-use efficiency, which are essential for practical deployment. To address this gap, we introduce ParetoPO, a two-stage multi-objective optimization framework for aligning tool-using large language models (LLMs) under competing objectives. In the first stage, ParetoPO leverages hypervolume-guided dynamic scalarization to adapt reward weights based on global Pareto frontier progress. In the second stage, it replaces scalarized learning signals with Pareto-ranking-based advantage computation, promoting nondominated trajectories through dominance-aware credit assignment. This design enables fine-grained, action-level optimization across multiple conflicting objectives. Experimental results on mathematic reasoning and multi-hop QA tasks show that ParetoPO consistently discovers policies with superior accuracy-efficiency trade-offs compared to static and heuristic baselines.

Chinese Translation

近期工具集成语言代理的进展显著提升了其解决复杂推理任务的能力。然而，现有的对齐方法主要集中于最大化任务准确性，而忽视了工具使用效率等辅助目标，而这些目标对于实际应用至关重要。为了解决这一问题，我们提出了ParetoPO，一个用于在竞争目标下对工具使用的大型语言模型（LLMs）进行对齐的两阶段多目标优化框架。在第一阶段，ParetoPO利用超体积引导的动态标量化，根据全局帕累托前沿的进展调整奖励权重。在第二阶段，它用基于帕累托排名的优势计算替代标量化学习信号，通过关注主导关系的信用分配促进非支配轨迹。这一设计使得在多个相互冲突的目标之间实现细粒度的行动级优化。数学推理和多跳问答任务的实验结果表明，ParetoPO在准确性与效率的权衡方面，始终发现优于静态和启发式基线的策略。

View on arXiv Download PDF AI Translation

cs.CL / 82 / 2606.16127

AuAu: A Benchmark for Auditing Authoritarian Alignment in Large Language Models

AuAu：审计大型语言模型中威权主义对齐的基准

Einwiller, Andreas, Klabunde, Max, Lemmerich, Florian

Abstract

The worldwide surge of authoritarianism, combined with the increasing central role in users' everyday lives, raises the question of to what extent specific models exhibit or promote authoritarian attitudes and characteristics. We introduce AuAu, a comprehensive benchmark that aims to assess the risk of LLMs generating responses with authoritarian tendencies. This benchmark combines three evaluation approaches: (i) psychometric questions from an extensive pool of 15 human validated instruments; (ii) contextual behavior vignettes probing intended actions in concrete situations; and (iii) responses to realistic user prompts. Unlike prior work, AuAu evaluates not only a general closeness towards authoritarianism but also the established sub-concepts Authoritarian Aggression, Authoritarian Submission, and Conventionalism. Evaluating 17 models from China, the EU, Russia, and the USA, we find that all tested models exhibit substantial authoritarian response rates under the psychometric evaluation, though rates drop significantly in increasingly more realistic downstream task. We further find that an authoritarian system prompt easily manipulates 15 out of 17 models to promote increased authoritarianism. Our results underscore the need for continued, systematic auditing of LLM-based AI systems to detect and ultimately mitigate undesired authoritarian tendencies in generated output. Our code and data are available at: https://github.com/andreaseinwiller/AuAu

Chinese Translation

全球范围内威权主义的激增，加上其在用户日常生活中日益重要的角色，提出了一个问题：特定模型在多大程度上表现或促进威权主义的态度和特征。我们引入了AuAu，这是一个综合性的基准，旨在评估大型语言模型（LLMs）生成具有威权倾向的响应的风险。该基准结合了三种评估方法：（i）来自15个经过人类验证的工具的心理测量问题；（ii）探讨具体情境中预期行为的情境行为小插曲；以及（iii）对现实用户提示的响应。与以往的研究不同，AuAu不仅评估对威权主义的一般接近性，还评估已建立的子概念：威权攻击、威权服从和传统主义。对来自中国、欧盟、俄罗斯和美国的17个模型进行评估，我们发现所有测试模型在心理测量评估中表现出显著的威权响应率，尽管在越来越现实的下游任务中，这些比率显著下降。我们进一步发现，一个威权系统提示可以轻易操控17个模型中的15个，以促进更高的威权主义。我们的结果强调了对基于LLM的人工智能系统进行持续、系统审计的必要性，以检测并最终减轻生成输出中的不良威权倾向。我们的代码和数据可在以下链接获取：https://github.com/andreaseinwiller/AuAu

View on arXiv Download PDF AI Translation

cs.CL / 83 / 2606.16137

XAI-Grounded Explanation Generation for Speech Deepfake Detection with Training-Free Multimodal Large Language Models

基于可解释人工智能的语音深度伪造检测解释生成：无训练的多模态大型语言模型

Li, Yupei, Sun, Qiyang, Wu, Xiaoliang, Wang, Chenxi, Sisman, Berrak, Schuller, Björn W.

Abstract

Speech deepfake detection (SDD) systems require trustworthy explanations for reliable decision-making. Existing explanation ways mainly fall into two categories. Traditional explainable AI (XAI), such as gradient-based attribution, produces low-level attribution signals tightly coupled with model decisions, and harder to be understood by human than natural language explanations. Meanwhile, large language model (LLM)-based explanation generation often produces generic and ungrounded descriptions due to the lack of heuristic evidence and task-specific supervision, stemming from limited grounded explanation datasets for SDD. We therefore propose a training-free explanation framework that integrates XAI evidence with multimodal LLMs to generate grounded and specific explanations. Using the PartialSpoof dataset, we construct a grounded explanation dataset and show that methods with XAI increase inside accuracy by over 45\%, verified through human evaluation and faithfulness checks.

Chinese Translation

语音深度伪造检测（SDD）系统需要可信的解释以支持可靠的决策。现有的解释方法主要分为两类。传统的可解释人工智能（XAI）方法，如基于梯度的归因，产生与模型决策紧密相关的低级归因信号，且比自然语言解释更难以被人理解。同时，基于大型语言模型（LLM）的解释生成往往由于缺乏启发式证据和特定任务的监督，导致生成的描述泛化且缺乏依据，这源于SDD领域有限的基础解释数据集。因此，我们提出了一种无训练的解释框架，将XAI证据与多模态LLM相结合，以生成有依据且具体的解释。通过使用PartialSpoof数据集，我们构建了一个有依据的解释数据集，并展示了使用XAI方法的准确性提高超过45%，这一结果通过人工评估和可信度检查得到了验证。

View on arXiv Download PDF AI Translation

cs.CL / 84 / 2606.16151

GRACE: Step-Level Benchmark for Faithful Reasoning over Context

GRACE：基于步骤的忠实推理基准

Pham, Hoang, Le, Dong, Luu, Anh Tuan

Abstract

Many reasoning tasks require models to reason over input context, from document-grounded question answering to rule-based deduction. Chain-of-Thought (CoT) prompting produces traces that appear transparent, yet individual steps can silently deviate from the source evidence, even when the final answer is correct. Existing methods detect hallucinations at the response level but fail to identify where in the chain a failure occurs or what type it is. We introduce GRACE, the first human-annotated step-level faithfulness benchmark with a data-driven error taxonomy for context-grounded textual reasoning. GRACE covers CoT traces from 10 models across 4 source datasets, with each step annotated for faithfulness, error category, and natural language explanation. A data-driven taxonomy, discovered bottom-up via unsupervised clustering, organizes failures into two tracks: GRACE-Inference (deductive errors) and GRACE-Grounding (factual grounding errors), with four categories each. The evaluation set is human-annotated and challenging by design. Our experiments reveal substantial headroom for current models. In addition, integrating step-level faithfulness signals into reinforcement learning pipelines improves both downstream accuracy and reasoning reliability.

Chinese Translation

许多推理任务要求模型在输入上下文中进行推理，从基于文档的问题回答到基于规则的推导。链式思维（Chain-of-Thought, CoT）提示生成的推理过程看似透明，但个别步骤可能在未被察觉的情况下偏离源证据，即使最终答案是正确的。现有方法在响应层面检测幻觉，但未能识别链中失败发生的位置或类型。我们提出了GRACE，这是首个经过人工标注的步骤级忠实性基准，具有针对上下文驱动的文本推理的数据驱动错误分类。GRACE涵盖了来自10个模型的CoT推理过程，涉及4个源数据集，每个步骤都标注了忠实性、错误类别和自然语言解释。通过无监督聚类自下而上发现的数据驱动分类法将失败组织为两个轨道：GRACE推理（演绎错误）和GRACE基础（事实基础错误），每个轨道有四个类别。评估集经过人工标注，设计上具有挑战性。我们的实验揭示了当前模型的显著改进空间。此外，将步骤级忠实性信号整合到强化学习管道中，提高了下游准确性和推理可靠性。

View on arXiv Download PDF AI Translation

cs.CL / 85 / 2606.16211

Weaving Multi-Source Evidence for Biomedical Reasoning: The BioMedHop Benchmark and BioWeave Framework

整合多源证据以进行生物医学推理：BioMedHop基准与BioWeave框架

Tan, Xingyu, Liu, Shiyuan, Wang, Xiaoyang, Liu, Qing, Xu, Xiwei, Yuan, Xin, Zhu, Liming, Zhang, Wenjie

Abstract

Biomedical question answering (QA) increasingly requires reasoning over interacting entities, where supporting evidence is scattered across biomedical knowledge graphs, literature documents, and web-accessible resources. However, existing biomedical QA benchmarks mainly focus on exam-style knowledge, literature comprehension, or short-range multi-hop inference, leaving source-conditioned graph reasoning and evidence topology construction underexplored. To fill this gap, we introduce BioMedHop, a multi-source graph-grounded benchmark for evaluating biomedical reasoning over structured evidence topologies. BioMedHop contains 10,045 instances across KG, document, web, and hybrid evidence settings, covering shared-neighbor matching, intersection reasoning, path-based reasoning, and counting, with option-based, open-ended, and numeric count renderings. To support this benchmark, we further propose BioWeave, a source-aware reasoning framework that retrieves biomedical KG paths, gathers supporting clues from documents and web sources, assembles them into a unified evidence graph, and verifies answers through entity-level evidence support. Comprehensive experiments show that BioWeave achieves the best overall performance among compared methods on BioMedHop, outperforming the strong hybrid baseline ToG-2 by 10.5% in the overall average. Moreover, BioWeave consistently improves different LLM backbones and enables smaller models, such as Qwen3-4B, to achieve reasoning performance comparable to GPT-4-Turbo.

Chinese Translation

生物医学问答（QA）越来越需要对交互实体进行推理，而支持证据散布在生物医学知识图谱、文献文档和可访问的网络资源中。然而，现有的生物医学QA基准主要集中在考试式知识、文献理解或短程多跳推理上，源条件图推理和证据拓扑构建尚未得到充分探索。为填补这一空白，我们引入了BioMedHop，这是一个多源图基础的基准，用于评估基于结构化证据拓扑的生物医学推理。BioMedHop包含10,045个实例，涵盖知识图（KG）、文档、网络和混合证据设置，涉及共享邻居匹配、交集推理、基于路径的推理和计数，提供基于选项、开放式和数值计数的呈现。为了支持这一基准，我们进一步提出了BioWeave，一个源感知推理框架，它检索生物医学KG路径，从文档和网络来源收集支持线索，将其组装成统一的证据图，并通过实体级证据支持验证答案。全面的实验表明，BioWeave在BioMedHop上在比较方法中实现了最佳的整体性能，整体平均表现比强大的混合基线ToG-2高出10.5%。此外，BioWeave持续提升不同的LLM骨干模型的性能，使得较小的模型，如Qwen3-4B，能够达到与GPT-4-Turbo相当的推理性能。

View on arXiv Download PDF AI Translation

cs.CL / 86 / 2606.16215

PACT: Privileged Trace Co-Training for Multi-Turn Tool-Use Agents

PACT：多轮工具使用代理的特权轨迹协同训练

Du, Zhenbang, Luo, Jun, Zheng, Zhiwei, Yuan, Xiangchi, Xia, Kejing, Shi, Dachuan, Jin, Qirui, He, Qijia, Zou, Shaofeng, Liang, Yingbin, Lee, Wenke

Abstract

Multi-turn tool-use agents must reason, call tools, and adapt to observations across several interaction turns. Post-training such agents is challenging, as reinforcement learning often suffers from sparse rewards and weak credit assignment despite matching the prompt-only inference setting, while supervised fine-tuning on expert traces provides dense process supervision but can over-constrain the model to fixed trajectories. To tackle this, we propose PACT, a Privileged trAce Co-Training framework for multi-turn tool-use agents. The key idea is to use expert traces only as training-time optimization signals rather than rollout-time hints. PACT keeps rollout generation prompt-only, then uses expert traces to guide optimization through two complementary signals: a trace-conditioned RL surrogate that evaluates prompt-only rollouts under expert-trace context, and a component-aware SFT loss that supervises reasoning prefixes and tool-calls with annealed strength. To reduce over-reliance on the training-only trace context, PACT further introduces a prompt-only anchoring. We also provide a latent-trace view that connects the two trace-based objectives and explains how expert traces can guide optimization without being used during rollout generation. Experiments on FTRL, BFCL, and ToolHop show that PACT consistently improves over strong SFT- and RL-based baselines, highlighting the value of privileged trace co-training for multi-turn tool-use learning.

Chinese Translation

多轮工具使用代理必须在多个交互回合中进行推理、调用工具并适应观察。对这些代理进行后期训练具有挑战性，因为强化学习通常面临稀疏奖励和弱信用分配的问题，尽管它与仅基于提示的推理设置相匹配，而在专家轨迹上进行监督微调则提供了密集的过程监督，但可能会使模型过于受限于固定轨迹。为了解决这个问题，我们提出了PACT，一个用于多轮工具使用代理的特权轨迹协同训练框架。其关键思想是将专家轨迹仅用作训练时的优化信号，而不是回滚时的提示。PACT保持回滚生成仅基于提示，然后利用专家轨迹通过两个互补信号引导优化：一个基于轨迹的强化学习替代品（trace-conditioned RL surrogate），在专家轨迹上下文中评估仅基于提示的回滚，以及一个组件感知的监督微调损失（SFT loss），以退火强度监督推理前缀和工具调用。为了减少对仅训练轨迹上下文的过度依赖，PACT进一步引入了仅基于提示的锚定。我们还提供了一个潜在轨迹视角，连接这两个基于轨迹的目标，并解释专家轨迹如何在不用于回滚生成的情况下指导优化。在FTRL、BFCL和ToolHop上的实验表明，PACT在强监督微调（SFT）和强化学习（RL）基线之上始终取得了改进，突显了特权轨迹协同训练在多轮工具使用学习中的价值。

View on arXiv Download PDF AI Translation

cs.CL / 87 / 2606.16240

Creative Collision: Directorial Persona Steering and Competition in Large Language Models

创意碰撞：导演人格引导与大型语言模型中的竞争

Sahoo, Subramanyam, Shenk, Justin

Abstract

Activation steering has emerged as a powerful tool for shaping the behaviour of large language models at inference time, yet most prior work injects a \emph{single} semantic direction into the residual stream. We study the richer setting in which two semantically opposing steering vectors are superimposed -- a regime we call \textbf{Creative Collision}. Concretely, we construct directorial persona vectors for Steven Spielberg (optimistic, redemptive moral valence) and Martin Scorsese (dark, morally ambiguous) via mean-difference activation contrast on curated screenplay-derived corpora, then interpolate between them with a scalar mixing parameter $\alpha \in [0,1]$ and a steering coefficient $\lambda$. Across five evaluation axes -- moral valence, generation coherence, surface style, directional dominance, and vector geometry -- three principal findings emerge: (i)~Spielberg's representational signature exhibits robust \emph{directional dominance}, suppressing Scorsese's moral influence across almost the entire interpolation range; (ii)~intermediate collision points paradoxically \emph{improve} generation coherence relative to pure single-director steering at high $\lambda$; and (iii)~both personas localise maximally to layer~28 of a 40-layer decoder-only transformer, revealing a shared \emph{moral-tone substrate}. These results illuminate the geometry of competing semantic directions in transformer residual streams and have direct implications for controllable creative generation and value-aligned narrative synthesis.

Chinese Translation

激活引导已成为在推理时塑造大型语言模型行为的强大工具，然而大多数先前的研究仅将 extit{单一}语义方向注入残差流中。我们研究了一个更丰富的情境，其中两个语义上对立的引导向量被叠加——我们称之为 extbf{创意碰撞}。具体而言，我们通过对策划的剧本衍生语料库进行均值差异激活对比，构建了斯蒂文·斯皮尔伯格（乐观、救赎的道德倾向）和马丁·斯科塞斯（黑暗、道德模糊）的导演人格向量，然后使用标量混合参数$eta ext{in} [0,1]$和引导系数$ heta$在它们之间进行插值。在五个评估维度——道德倾向、生成一致性、表面风格、方向主导性和向量几何中，得出了三个主要发现：（i）斯皮尔伯格的表现特征展示了强有力的 extit{方向主导性}，在几乎整个插值范围内抑制了斯科塞斯的道德影响；（ii）中间碰撞点在高$ heta$下悖论性地 extit{改善}了生成一致性，相较于纯单导演引导；（iii）两个人格在40层解码器仅有的第28层最大化局部化，揭示了共享的 extit{道德基调底层}。这些结果阐明了变换器残差流中竞争语义方向的几何特征，并对可控的创意生成和价值对齐的叙事合成具有直接的影响。

View on arXiv Download PDF AI Translation

cs.CL / 88 / 2606.16281

Who Should Lead Decoding Now? Tracking Reliable Trajectories for Ensembling Masked Diffusion Language Models

现在谁应该领导解码？追踪集成掩码扩散语言模型的可靠轨迹

Yun, Heecheol, Park, Joonhyung, Kim, Joowon, Yang, Eunho

Abstract

Masked Diffusion Language Models (MDLMs) have emerged as a distinct paradigm for sequence generation. As MDLMs become diverse in capabilities and knowledge coverage, an important question is how to combine their knowledge. Toward this, we first investigate the unique decoding dynamics of MDLMs. We find that successful generations exhibit stable confidence dynamics over answer-relevant positions, while unreliable trajectories can often be corrected by injecting promising intermediate states from other models. Guided by this observation, we propose $\textbf{TIE}$ ($\textbf{T}$rajectory-based $\textbf{I}$terative $\textbf{E}$nsembling), a knowledge fusion framework in which MDLMs iteratively identify reliable decoding trajectories and relay them across models. TIE tracks confidence dynamics over answer-relevant positions to determine which model currently follows a more reliable trajectory and selectively transfers partially denoised sequences across models. As the model on the more promising trajectory often changes across denoising steps, TIE allows different models to contribute complementary strengths at different stages of generation. Strong performance across diverse reasoning tasks, along with our analyses, suggests that TIE offers a practical approach to the underexplored problem of MDLM ensembling.

Chinese Translation

掩码扩散语言模型（MDLMs）已成为序列生成的一个独特范式。随着MDLMs在能力和知识覆盖方面的多样化，一个重要的问题是如何结合它们的知识。为此，我们首先研究MDLMs独特的解码动态。我们发现，成功的生成在与答案相关的位置上表现出稳定的置信度动态，而不可靠的轨迹往往可以通过注入其他模型的有前景的中间状态来纠正。在这一观察的指导下，我们提出了$ extbf{TIE}$（$ extbf{T}$rajectory-based $ extbf{I}$terative $ extbf{E}$nsembling），这是一个知识融合框架，其中MDLMs迭代识别可靠的解码轨迹并在模型之间传递这些轨迹。TIE跟踪与答案相关的位置上的置信度动态，以确定当前哪个模型遵循更可靠的轨迹，并选择性地在模型之间转移部分去噪序列。由于在去噪步骤中，更有前景的轨迹上的模型通常会发生变化，TIE允许不同模型在生成的不同阶段贡献互补的优势。在多样的推理任务中表现出强劲的性能，以及我们的分析，表明TIE为尚未深入探讨的MDLM集成问题提供了一种实用的方法。

View on arXiv Download PDF AI Translation

cs.CL / 89 / 2606.16285

HiMPO: Hindsight-Informed Memory Policy Optimization for Less-Entangled Credit in Long-Horizon Agents

HiMPO：用于减少长时间跨度智能体中纠缠信用的事后信息记忆策略优化

Yan, Jiangze, Shen, Yi, Zhang, Wenjing, Huang, Jieyun, Liu, Zhaoxiang, Wang, Ning, Wang, Kai, Lian, Shiguo

Abstract

Long-horizon agents rely on memory mechanisms to compress interaction history, but optimizing memory writing faces a distinct credit assignment challenge: a memory update may be rewarded or penalized due to downstream tool failures, noisy observations, or reasoning errors rather than its own contribution. This causally entangled credit can lead agents to discard useful evidence or preserve irrelevant information. We propose HiMPO, a Hindsight-Informed Memory Policy Optimization framework for assigning less-entangled credit to memory-writing actions in long-horizon agents. HiMPO first estimates the local utility of a memory update by comparing the task-relevant information recoverable from the previous and updated memories under the same pre-write state. It then uses hindsight relevance as a bounded retrospective filter that attenuates memory credit when local utility is not supported by the target outcome. The resulting memory-specific advantage is applied only to memory tokens, while trajectory-level rewards optimize the rest of the agent behavior. Across judge-based open-domain tasks and objective compressive-memory QA, HiMPO improves over strong memory-based and RL-based baselines while preserving compressed-context efficiency. Controlled interventions further show that HiMPO reduces blame leakage from tool-induced errors and improves attribution fidelity of memory updates.

Chinese Translation

长时间跨度的智能体依赖记忆机制来压缩交互历史，但优化记忆写入面临着独特的信用分配挑战：由于下游工具故障、噪声观测或推理错误，记忆更新可能会因其自身贡献而受到奖励或惩罚。这种因果纠缠的信用可能导致智能体丢弃有用证据或保留无关信息。我们提出了HiMPO，一个事后信息记忆策略优化框架，用于在长时间跨度的智能体中为记忆写入行为分配较少纠缠的信用。HiMPO首先通过比较在相同的预写状态下从先前和更新的记忆中可恢复的任务相关信息来估计记忆更新的局部效用。然后，它使用事后相关性作为一个有界的回顾性过滤器，当局部效用未得到目标结果支持时，减弱记忆信用。最终的记忆特定优势仅应用于记忆标记，而轨迹级奖励则优化智能体的其余行为。在基于评估的开放领域任务和目标压缩记忆问答中，HiMPO在保持压缩上下文效率的同时，优于强大的基于记忆和基于强化学习的基线。控制干预进一步表明，HiMPO减少了由工具引起的错误的责任泄漏，并提高了记忆更新的归因准确性。

View on arXiv Download PDF AI Translation

cs.CL / 90 / 2606.16322

PaperJury: Due-Process Review for Bounded LaTeX Revision

PaperJury：有限 LaTeX 修订的正当程序审查

Wang, Yiran, An, Ruixuan, Wu, Biao, Wang, Wenhao

Abstract

Pre-submission hardening of human-authored LaTeX computer science papers differs from drafting assistance because it requires adversarial whole-paper review, explicit no-fix outcomes, and bounded artifact-safe revision. Existing writing assistants, critique generators, and judge-centered loops lack durable issue identity across rounds, deterministic routing from critique to adjudication, and manuscript control that can reject invalid concerns or defer author-dependent ones. We present PaperJury, a closed-loop review-verdict-revise-verify system built on a deterministic-versus-semantic split: deterministic orchestration manages decomposition, a frozen claim spine, a durable ledger, routing, stopping, and exact-once patch application, while semantic agents are limited to bounded review, judgment, and repair. PaperJury combines bounded holistic review, contestability-based routing, a due-process trial, and risk-proportional guard chains for anchor-bounded edits, yielding terminal outcomes of invalid-drop, valid-fixable, and author-required. In a two-arm expert-review evaluation on held-out Vision, natural language processing, and machine learning papers against four baselines, we assess issue quality, verdict and routing quality, edit safety, convergence behavior, and cost, supporting the thesis that load-bearing safety and completion logic should reside in deterministic orchestration rather than model discretion. PaperJury is available at https://github.com/u7079256/paperjury.

Chinese Translation

人类撰写的 LaTeX 计算机科学论文在提交前的强化与草拟辅助有所不同，因为它需要对整篇论文进行对抗性的审查、明确的无修复结果以及有限的安全修订。现有的写作助手、批评生成器和以评审为中心的循环在多个回合中缺乏持久的问题识别、从批评到裁决的确定性路径，以及能够拒绝无效关注或推迟依赖作者的关注的手稿控制。我们提出了 PaperJury，一个基于确定性与语义分离的闭环审查-裁决-修订-验证系统：确定性编排管理分解、冻结的主张脊、持久的账本、路径选择、停止以及精确一次的补丁应用，而语义代理则限制于有限的审查、判断和修复。PaperJury 结合了有限的整体审查、基于争议的路径选择、正当程序审判和风险比例的保护链，以实现锚定有限编辑的终极结果，包括无效删除、有效可修复和作者所需。在对 Vision、自然语言处理和机器学习论文进行的双臂专家评审评估中，我们与四个基线进行了比较，评估了问题质量、裁决和路径质量、编辑安全性、收敛行为和成本，支持了负载承载安全性和完成逻辑应当存在于确定性编排而非模型自由裁量的论点。PaperJury 可在 https://github.com/u7079256/paperjury 获取。

View on arXiv Download PDF AI Translation

cs.CL / 91 / 2606.16351

TMASC: Transmasculine Attitude and Speech Corpus

TMASC：跨男性态度与言语语料库

Wong, Sidney

Abstract

We introduce the Transmasculine Attitudes and Speech Corpus (TMASC), a multimodal corpus of 196 transmasculine individuals, including questionnaire responses and 66 audio recordings. The questionnaire includes items exploring the vocal health of transmasculine individuals. The audio recordings include cough and throat-clearing samples, a reading passage, and additional session-specific questions. This paper outlines the development of this corpus and the data collection procedures. To illustrate the utility of this corpus, we present three case studies demonstrating how this crowd-sourced multimodal corpus can be used to support transmasculine individuals. These include the integration of perceptual and acoustic data, the identification of group-level characteristics, and the calibration of acoustic measurements.

Chinese Translation

我们介绍了跨男性态度与言语语料库（TMASC），这是一个包含196名跨男性个体的多模态语料库，包括问卷回答和66个音频录音。问卷中包含探讨跨男性个体声音健康的条目。音频录音包括咳嗽和清喉样本、阅读段落以及额外的特定会话问题。本文概述了该语料库的发展及数据收集程序。为了说明该语料库的实用性，我们展示了三个案例研究，演示了如何利用这一众包的多模态语料库来支持跨男性个体。这些案例包括感知数据与声学数据的整合、群体特征的识别以及声学测量的校准。

View on arXiv Download PDF AI Translation

cs.CL / 92 / 2606.16360

Tyler: Typed Latent Reasoning for Language Models -- When to Think, What to Compute, and How Much to Allocate

Tyler：用于语言模型的类型化潜在推理——何时思考、计算什么以及分配多少资源

Lin, Hanyu, Cai, Min, Wen, Jiawei, Zhang, Haodi

Abstract

Chain-of-thought (CoT) prompting improves reasoning in large language models (LLMs) by externalizing intermediate computation as discrete text tokens, but this textual interface also introduces redundancy and inference overhead. Latent reasoning offers a promising alternative by carrying part of the computation in continuous representations. However, existing methods typically predefine when latent computation is invoked and how it is allocated during decoding, leaving a key problem unresolved: when to invoke latent computation, what type of computation to perform, and how much budget to allocate. We propose \textbf{Ty}ped \textbf{L}at\textbf{e}nt \textbf{R}easoning (Tyler), a typed and budget-aware framework for latent reasoning during autoregressive decoding. Tyler learns a policy that, at each decoding step, chooses between emitting a text token and switching to a latent computation module specialized for a particular reasoning function. Once invoked, an operator maps the current reasoning state into latent tokens that support global planning, local state updates, or reusable procedural abstraction. Across extensive experiments on three backbone LLMs, Tyler improves accuracy by up to 14.49 points over CoT and by up to 4.30 points over the strongest competing baseline. It further generalizes across diverse reasoning domains and achieves the best final-stage performance with the lowest forgetting.

Chinese Translation

链式思维（Chain-of-thought, CoT）提示通过将中间计算外部化为离散文本标记来改善大型语言模型（LLMs）的推理能力，但这种文本接口也引入了冗余和推理开销。潜在推理提供了一种有前景的替代方案，通过连续表示来承载部分计算。然而，现有方法通常预定义何时调用潜在计算以及在解码过程中如何分配计算，这留下了一个关键问题未得到解决：何时调用潜在计算、执行何种类型的计算以及分配多少预算。我们提出了 extbf{Ty}ped extbf{L}at extbf{e}nt extbf{R}easoning（Tyler），这是一个在自回归解码过程中进行潜在推理的类型化和预算感知框架。Tyler学习一种策略，在每个解码步骤中，在发出文本标记和切换到专门用于特定推理功能的潜在计算模块之间进行选择。一旦被调用，操作符将当前推理状态映射为支持全局规划、本地状态更新或可重用过程抽象的潜在标记。在对三种基础LLM进行的大量实验中，Tyler在准确性上比CoT提高了最多14.49个百分点，比最强竞争基线提高了最多4.30个百分点。它还在多样化的推理领域中实现了更好的泛化，并在最低遗忘率的情况下达到了最佳的最终阶段性能。

View on arXiv Download PDF AI Translation

cs.CL / 93 / 2606.16368

Evaluating LLM Personalization via Semantic Constraint Verification

通过语义约束验证评估大型语言模型个性化

Li, Xuran, Zhang, Guanqin, Razzak, Imran, Hacid, Hakim, Kafeza, Eleanna, Xue, Hao, Salim, Flora D.

Abstract

Current evaluation paradigms for Large Language Model (LLM) personalization rely heavily on brittle surface-matching metrics or computationally expensive LLM-as-a-judge protocols, both of which lack interpretability. To address these limitations, we introduce Natural Language Inference Constraint Verification (NLICV), a scalable, semantically invariant framework that maps sentence meanings to truth-condition sets to verify personalization constraints via a Natural Language Inference (NLI) model. Moving beyond binary scoring, NLICV categorizes LLM behaviors into four distinct modes: personalization, generalization, sycophancy, and failure. Extensive experiments demonstrate that NLICV aligns closely with human annotations while drastically reducing the latency and token costs associated with LLM judges (up to 2100 inference speedup). Finally, through an ablation-based procedure, NLICV pinpoints the exact sentences driving the constraint verification, yielding faithful, understandable evidence for its evaluations.

Chinese Translation

当前大型语言模型（LLM）个性化的评估范式在很大程度上依赖于脆弱的表面匹配指标或计算成本高昂的LLM作为评判者的协议，这两者都缺乏可解释性。为了解决这些局限性，我们提出了自然语言推理约束验证（Natural Language Inference Constraint Verification, NLICV），这是一个可扩展的、语义不变的框架，它将句子意义映射到真值条件集合，以通过自然语言推理（Natural Language Inference, NLI）模型验证个性化约束。NLICV超越了二元评分，将LLM行为分类为四种不同模式：个性化、概括、阿谀奉承和失败。大量实验表明，NLICV与人类注释高度一致，同时大幅降低了与LLM评判者相关的延迟和令牌成本（最高可达2100倍的推理加速）。最后，通过基于消融的程序，NLICV精确定位驱动约束验证的具体句子，为其评估提供了真实且易于理解的证据。

View on arXiv Download PDF AI Translation

cs.CL / 94 / 2606.16383

Surpassing Scale by Efficiency: A Compact 135M Parameter Foundational LLM Natively Adapted for the Bangla Language

通过效率超越规模：一种紧凑型135M参数基础大型语言模型，原生适应孟加拉语

Nandi, Rabindra Nath

Abstract

While the NLP landscape is dominated by multi-billion parameter architectures, their deployment in low-resource, non-Latin scripts remains computationally prohibitive for edge configurations, mobile systems, and decentralized local hardware. This paper presents bangla-smollm-135m, a highly compact 135-million parameter decoder-only foundational model engineered explicitly for high-efficiency language modeling in the Bangla script. By leveraging a deterministic intersect-and-append token merging strategy between TituLLMs and SmolLM2-135M, the model overcomes subword script fragmentation without destabilizing early pretrained parameter states. In zero-shot multi-task benchmark evaluations (PIQA_bn, OpenBookQA_bn, CommonsenseQA_bn, and Bangla_MMLU), bangla-smollm-135m matches or outperforms models twice its size (Gemma-3-270m) and achieves parity with models in the 1B parameter tier. The model is available at rnnandi/bangla-smollm-135m

Chinese Translation

尽管自然语言处理（NLP）领域主要由数十亿参数的架构主导，但在低资源、非拉丁字符的脚本中部署这些架构仍然在计算上对边缘配置、移动系统和去中心化本地硬件构成了巨大的挑战。本文提出了bangla-smollm-135m，这是一种高度紧凑的135百万参数解码器基础模型，专门为孟加拉脚本中的高效语言建模而设计。通过利用TituLLMs与SmolLM2-135M之间的确定性交叉与附加令牌合并策略，该模型克服了子词脚本碎片化的问题，同时不破坏早期预训练参数状态。在零-shot多任务基准评估（PIQA_bn、OpenBookQA_bn、CommonsenseQA_bn和Bangla_MMLU）中，bangla-smollm-135m的表现与其两倍大小的模型（Gemma-3-270m）相当或更优，并且在1B参数级别的模型中达到了平衡。该模型可在rnnandi/bangla-smollm-135m获取。

View on arXiv Download PDF AI Translation

cs.CL / 95 / 2606.16407

A Mechanistic Understanding of Pronoun Fidelity in LLMs

大型语言模型中代词忠实性的机制理解

Trinley, Katharina, Alabi, Jesujoba O., Klakow, Dietrich, Gautam, Vagrant

Abstract

Faithful and robust pronoun use is important for fair and coherent generations, yet large language models largely fail when multiple referents use different pronouns. To study the interplay of reasoning, repetition, and bias in this task, prior work relies exclusively on behavioural approaches, which may not reflect a model's internal workings. Therefore, we provide a mechanistic, model-internal perspective on pronoun fidelity, testing whether three mechanisms -- group entity binding (G), recency bias (R), and stereotypical bias (S) -- are causally implemented across several SOTA language models. Using Boundless Distributed Alignment Search, we find all three coexist as causal subspaces distributed across network depth. No single mechanism fully explains model behaviour, but a combination of the three consistently accounts for 91-99.5%. An attention head analysis further reveals two competing copying routes; group binding and stereotype share a localized concept-level route that retrieves a bound occupation-pronoun unit, while recency uses a distributed token-level route that repeats surface forms. In sum, pronoun fidelity arises from competition between simultaneously active causal subspaces.

Chinese Translation

忠实且稳健的代词使用对于公平和连贯的生成至关重要，然而当多个指称使用不同的代词时，大型语言模型往往表现不佳。为了研究推理、重复和偏见在这一任务中的相互作用，以往的研究主要依赖于行为方法，这可能无法反映模型的内部工作机制。因此，我们提供了一种机制性、模型内部的视角来探讨代词忠实性，测试三种机制——群体实体绑定（Group Entity Binding, G）、近期偏见（Recency Bias, R）和刻板印象偏见（Stereotypical Bias, S）——是否在多个最先进（SOTA）语言模型中因果实现。通过使用无界分布对齐搜索（Boundless Distributed Alignment Search），我们发现这三种机制作为因果子空间共存于网络深度中。没有单一机制能够完全解释模型行为，但三者的组合一致地解释了91-99.5%的变异。注意力头分析进一步揭示了两条竞争的复制路径；群体绑定和刻板印象共享一个局部概念级路径，该路径检索绑定的职业-代词单元，而近期偏见则使用一个分布式的标记级路径来重复表面形式。总之，代词忠实性源于同时活跃的因果子空间之间的竞争。

View on arXiv Download PDF AI Translation

cs.CL / 96 / 2606.16409

PathRouter: Aligning Rewards with Retrieval Quality in Agentic Graph Retrieval-Augmented Generation

PathRouter：在代理图检索增强生成中对齐奖励与检索质量

Wang, Bo, Huang, Heyan, Li, Yaolin, Tang, Wei, Zhang, Yuan, Li, Wenbo, Gao, Mingze, Shi, Ge, Feng, Chong

Abstract

Agentic GraphRAG trains language-model agents to iteratively retrieve and reason over graph-structured evidence, enabling more accurate and context-aware decision-making by efficiently navigating complex information networks. However, outcome-only reinforcement learning suffers from \textit{\textbf{answer-path reward aliasing}}, where correct answers may come from shortcuts rather than useful evidence paths. It also exhibits \textit{\textbf{search-update ambiguity}}, as scalar trajectory-level feedback does not indicate which retrieval actions to adjust. To mitigate these shortcomings, we present PathRouter, a path-aware training framework for agentic GraphRAG. PathRouter jointly evaluates each trajectory along answer correctness and evidence-path overlap, yielding four trajectory categories with differentiated GRPO advantage scaling that suppresses shortcut reinforcement while preserving evidence-seeking behavior. For evidence-poor trajectories, a frozen gold-evidence teacher provides token-level KL guidance on reasoning and search-query tokens, excluding answer tokens to avoid direct response imitation. Experiments on six QA benchmarks across three model sizes show that PathRouter consistently improves answer F1 and evidence-path overlap, achieving average F1 gains of 3.1 on 3B and 4.9 on 7B models compared to a strong baseline.

Chinese Translation

代理图检索增强生成（GraphRAG）训练语言模型代理，能够迭代地检索和推理图结构证据，从而通过高效导航复杂信息网络实现更准确和上下文感知的决策。然而，仅依赖结果的强化学习存在 extit{ extbf{答案路径奖励混淆}}的问题，即正确答案可能来自捷径而非有用的证据路径。此外，它还表现出 extit{ extbf{搜索更新模糊性}}，因为标量轨迹级反馈并未指示需要调整哪些检索动作。为了解决这些问题，我们提出了PathRouter，这是一种针对代理GraphRAG的路径感知训练框架。PathRouter联合评估每条轨迹的答案正确性和证据路径重叠，产生四类轨迹，并通过差异化的GRPO优势缩放抑制捷径强化，同时保留寻求证据的行为。对于证据稀缺的轨迹，冻结的金标准证据教师提供关于推理和搜索查询标记的逐标记KL指导，排除答案标记以避免直接模仿响应。在三个模型规模的六个QA基准上的实验表明，PathRouter持续提升答案F1和证据路径重叠，相较于强基线，3B模型平均F1提升3.1，7B模型提升4.9。

View on arXiv Download PDF AI Translation

cs.CL / 97 / 2606.16428

Lect\=uraAgents: A Multi-Agent Framework for Adaptive Personalized AI-Assisted Learning and Embodied Teaching

LectAgents：一个用于自适应个性化AI辅助学习和具身教学的多代理框架

Sesay, Jaward, Yu, Yue, Dong, Siwei, Shi, Yemin, Chen, Guangyao, Karlsson, Börje F.

Abstract

Effective personalized AI-assisted learning demands systems that can not only generate accurate learner-specific educational materials, but also dynamically adapt their instruction to diverse learners. However, existing educational agents have primarily focused on lecture content automation and simulations, which often fall short of modelling multimodal and embodied instructional methods tailored for the individual learner. To this end, we propose Lect\=uraAgents - a multi-agent framework that enables personalized learning through end-to-end adaptive embodied teaching. At its core, Lect\=uraAgents mirrors a professor-student relationship, in which a ProfessorAgent leads a collaborative team of specialized subordinate agents through research, planning, review, and embodied delivery of lecture contents that adapt to a learner's needs. The framework offers three main contributions: (1) a hierarchical multi-agent architecture for end-to-end personalized learning; (2) an adaptive embodied teaching mechanism, wherein the ProfessorAgent executes visible and pedagogically motivated teaching actions (e.g., handwrite, highlight, underline, etc.) over contents in a teaching environment; and (3) a Teaching Action-Speech Alignment (TASA) algorithm that employs salience-based heuristics and temporal semantic segmentation to generate coherent teaching action sequences aligned with learner profiles. We evaluate Lect\=uraAgents on diverse courses at high school, undergraduate, and graduate levels using sample-specific rubric-based analysis; with generated lecture materials and teaching actions assessed and validated by expert educators. Experimental results show consistent gains in lecture content quality, embodied teaching quality, assessment, and personalization over existing approaches, positioning Lect\=uraAgents as a pedagogically well-grounded framework for personalized learning at scale.

Chinese Translation

有效的个性化AI辅助学习需要能够生成准确的学习者特定教育材料的系统，并且能够动态地根据不同学习者的需求调整教学。然而，现有的教育代理主要集中在讲座内容的自动化和模拟上，这往往未能有效模拟针对个别学习者的多模态和具身教学方法。为此，我们提出了LectAgents——一个通过端到端自适应具身教学实现个性化学习的多代理框架。LectAgents的核心反映了教授与学生之间的关系，其中ProfessorAgent领导一个由专门的下属代理组成的协作团队，通过研究、规划、审查和具身传递适应学习者需求的讲座内容。该框架提供了三个主要贡献：（1）一个用于端到端个性化学习的分层多代理架构；（2）一种自适应具身教学机制，其中ProfessorAgent在教学环境中对内容执行可见且具有教育动机的教学动作（例如，手写、突出显示、下划线等）；（3）一个教学动作-语言对齐（Teaching Action-Speech Alignment, TASA）算法，利用基于显著性的启发式方法和时间语义分割生成与学习者档案对齐的连贯教学动作序列。我们在高中、本科和研究生层次的多样课程上评估了LectAgents，采用基于样本特定的评分标准进行分析；生成的讲座材料和教学动作经过专家教育工作者的评估和验证。实验结果显示，在讲座内容质量、具身教学质量、评估和个性化方面，相较于现有方法，LectAgents表现出一致的提升，将其定位为一个在大规模个性化学习中具有良好教育基础的框架。

View on arXiv Download PDF AI Translation

cs.CL / 98 / 2606.16432

ACCORD: Action-Conditioned Contextual Grounding for Language Agents

ACCORD：基于动作条件的上下文基础构建语言代理

Jiang, Lai, Qian, Cheng, Wang, Zhenhailong, Lu, Pan, Ji, Heng, Peng, Hao

Abstract

User instructions are often underspecified because humans rely on implicit assumptions about the surrounding environment. For large language model (LLM) agents operating in information-rich digital and physical environments, these assumptions cannot be inferred from the instruction alone; they must be recovered from the current state of tools, data, interfaces, and observations. Effective execution therefore requires agents to identify missing context, ground it in observed evidence, and carry it forward into subsequent actions. We show that current agents often fail to do so. They act from assumed rather than observed specifics, overlook information they could have gathered, and fail to incorporate evidence that has already been returned. Building on this insight, we propose ACCORD (Action-Conditioned Contextual Grounding), a simple and effective agent framework for adaptive grounding. Before each action, ACCORD actively probes the environment for missing information and integrates relevant context from the agent's trajectory that would otherwise be overlooked. Requiring no additional training or task-success signals, ACCORD improves task-goal completion on AppWorld by up to +20.6 points with GPT-5-mini, from 42.0% to 62.6%, compared to strong baselines. These gains persist with a substantially stronger base model (+10.8 with Claude-4.5-sonnet), an open-weight model (+10.1 with Qwen3.5-27B-FP8), and on the embodied AlfWorld benchmark (+7.4 success rate with GPT-5-mini).

Chinese Translation

用户指令往往缺乏明确性，因为人类依赖于对周围环境的隐含假设。对于在信息丰富的数字和物理环境中操作的大型语言模型（LLM）代理，这些假设不能仅从指令中推断出来；它们必须从当前的工具、数据、接口和观察状态中恢复。因此，有效的执行要求代理识别缺失的上下文，将其基于观察到的证据进行基础构建，并将其融入后续的行动中。我们发现当前的代理往往未能做到这一点。它们依据假设而非观察到的具体情况进行行动，忽视了可以收集的信息，并未能整合已经返回的证据。在此基础上，我们提出了ACCORD（基于动作条件的上下文基础构建），这是一个简单而有效的自适应基础构建代理框架。在每个行动之前，ACCORD主动探测环境以获取缺失的信息，并整合代理轨迹中相关的上下文，这些上下文在其他情况下可能会被忽视。ACCORD无需额外的训练或任务成功信号，在AppWorld上将任务目标完成率提高了多达20.6个百分点，从42.0%提升至62.6%，相比于强基线。这些增益在一个更强大的基础模型上依然存在（Claude-4.5-sonnet提升10.8个百分点），在一个开放权重模型上（Qwen3.5-27B-FP8提升10.1个百分点），以及在具身的AlfWorld基准上（GPT-5-mini提升7.4%的成功率）。

View on arXiv Download PDF AI Translation

cs.CL / 99 / 2606.16472

From Awareness to Adherence: Bridging the Context Gap in Spoken Dialogue Systems via Context-Aware Decoding

从意识到遵循：通过上下文感知解码弥合语音对话系统中的上下文差距

Lee, Che Hyun, Kim, Heeseung, Yoon, Sungroh

Abstract

Despite the success of end-to-end (E2E) spoken dialogue systems, maintaining strict context adherence in multi-round conversations remains a challenge. While prior works attribute these failures to models forgetting dialogue history, we highlight an equally critical but overlooked bottleneck: a gap between latent context awareness and active adherence. Although models internally recognize relevant past utterances, strong parametric priors often overshadow these signals during decoding. To bridge this gap, we propose an audio-adapted Context-Aware Decoding (CAD) approach. By leveraging internal attention mechanisms to isolate key historical rounds, our approach contrasts output distributions with and without this key context during inference, directly amplifying multimodal contextual signals. Evaluations on the Audio MultiChallenge benchmark demonstrate significant improvements in Semantic Memory and Self Coherence subtasks, successfully enforcing strict, context-faithful adherence.

Chinese Translation

尽管端到端（E2E）语音对话系统取得了成功，但在多轮对话中保持严格的上下文遵循仍然是一个挑战。虽然先前的研究将这些失败归因于模型遗忘对话历史，但我们强调了一个同样重要但被忽视的瓶颈：潜在上下文意识与主动遵循之间的差距。尽管模型在内部识别相关的过去发言，但强大的参数先验往往在解码过程中掩盖了这些信号。为了弥合这一差距，我们提出了一种音频适应的上下文感知解码（Context-Aware Decoding, CAD）方法。通过利用内部注意机制来隔离关键的历史轮次，我们的方法在推理过程中对比了包含和不包含这一关键上下文的输出分布，直接放大了多模态上下文信号。在Audio MultiChallenge基准上的评估显示，在语义记忆和自我连贯性子任务中取得了显著改善，成功地强制执行了严格的、符合上下文的遵循。

View on arXiv Download PDF AI Translation

cs.CL / 100 / 2606.16494

Lost at the End: Primacy Bias in Multimodal Retrieval-Augmented Question Answering

迷失在结尾：多模态检索增强问答中的首因偏差

Liu, Jieyuan, Gu, Jianyang, Chen, Shijie, Chen, Jefferson, Wang, Zhen

Abstract

Knowledge-based visual question answering (KB-VQA) lets vision-language systems answer questions that exceed their parametric knowledge by conditioning a reader on passages retrieved from a Wikipedia-scale knowledge base. In pure-text long-context LLMs, retrieved-context use follows the U-shaped "lost-in-the-middle" effect of Liu et al. (2024): information at the start and end of context is used, the middle is lost. Whether this transfers to deployed multimodal KB-VQA is open. To close this gap, we design the first controlled probe of reader-side position dependence in multimodal KB-VQA: a gold-position protocol in which only the gold passage's prompt slot varies within question. We run it on three open-source 7B/8B VLM readers and two KB-VQA benchmarks at k up to 20. The shape flips from U to primacy: gold-at-first beats gold-at-last by 16 to 26 points on every reader-by-benchmark cell, an effect we call "Lost at the End". Three targeted ablations narrow the cause: a text-only control shows the multimodal setting amplifies an already-present text-mode primacy 2.2 to 4.5 times, and image-position and distractor-shuffle ablations together pin the locus to prompt slot 0 of the instruction-tuned reader. On a frozen reader, three retrieval-side fixes (MMR, oracle reranking, rank-based reordering) all leave the gap intact (no separable improvement). Our findings indicate that recall@k is the wrong metric for deployed KB-VQA and that closing the gap requires reader-side intervention; we release our protocol as a controlled instrument for evaluating such interventions.

Chinese Translation

基于知识的视觉问答（KB-VQA）使得视觉-语言系统能够回答超出其参数知识的问题，通过从维基百科规模的知识库中检索段落来为阅读器提供条件。在纯文本的长上下文大语言模型（LLMs）中，检索上下文的使用遵循刘等人（2024）提出的U形“迷失在中间”效应：上下文开头和结尾的信息被使用，而中间部分则被忽视。这种现象是否会转移到已部署的多模态KB-VQA中仍然是一个未解之谜。为了填补这一空白，我们设计了第一个控制性探测，研究多模态KB-VQA中阅读器端位置依赖性：一种金标准位置协议，其中仅金标准段落的提示位置在问题中变化。我们在三个开源的7B/8B视觉语言模型（VLM）阅读器和两个KB-VQA基准上进行实验，k值最高可达20。结果显示形状从U型转变为首因效应：在每个阅读器-基准单元中，金标准在开头的表现比金标准在结尾的高出16到26分，这一现象我们称之为“迷失在结尾”。通过三项针对性的消融实验，我们缩小了原因范围：仅文本的对照实验表明，多模态设置将已有的文本模式首因效应放大了2.2到4.5倍，而图像位置和干扰项洗牌的消融实验共同将原因归结为指令调优阅读器的提示位置0。在一个冻结的阅读器上，三种检索端的修正（MMR、oracle重排序、基于排名的重新排序）均未能缩小这一差距（没有可分离的改进）。我们的研究结果表明，recall@k并不是已部署KB-VQA的正确指标，缩小这一差距需要在阅读器端进行干预；我们将我们的协议作为评估此类干预的控制工具发布。

View on arXiv Download PDF AI Translation

cs.CL / 101 / 2606.16496

REFLEX: Reflective Evolution from LLM Experience

REFLEX: 基于大型语言模型经验的反思性演化

Wang, Pan

Abstract

Large multimodal language models (LLMs) have emerged as powerful tools for guiding evolutionary search toward interpretable programmatic policies. However, existing frameworks rely on a monolithic model call to simultaneously interpret visual behavioral evidence and synthesize corrective code. This diagnosis-repair entanglement creates an opaque feedback loop, obscuring the rationale behind mutations and preventing the retention of algorithmic insights across independent runs. To achieve auditable and efficient policy search, we argue that visual diagnosis must be structurally decoupled from code generation. We present REFLEX, a train-free evolutionary framework that operationalizes this decoupling. In REFLEX, a vision-enabled Critic first distills task-specific behavioral evidence into structured, auditable diagnoses. Subsequently, a text-optimized Actor synthesizes child policies using these diagnoses alongside a persistent, self-evolving Skill Memory of reusable code snippets. This architecture not only provides transparent mutation traces but also enables cross-run programmatic knowledge transfer. Extensive evaluations across control benchmarks (Lunar Lander, Acrobot, Pendulum) and a 36-dimensional antenna array synthesis task demonstrate exceptional sample efficiency. Notably, REFLEX solves Acrobot and Pendulum in under 10 LLM calls and reaches a best Normalized Weighted Score of 1.092 on Lunar Lander, achieving highly competitive final performance while significantly accelerating the early-stage discovery of transparent policies.

Chinese Translation

大型多模态语言模型（LLMs）已成为引导演化搜索以获得可解释的程序策略的强大工具。然而，现有框架依赖于单一模型调用，同时解释视觉行为证据并合成修正代码。这种诊断-修复的纠缠形成了一个不透明的反馈循环，模糊了突变背后的理由，并阻碍了在独立运行之间保留算法洞察。为了实现可审计和高效的策略搜索，我们认为视觉诊断必须在结构上与代码生成解耦。我们提出了REFLEX，一个无训练的演化框架，实施这种解耦。在REFLEX中，一个具备视觉能力的批评者首先将任务特定的行为证据提炼为结构化、可审计的诊断。随后，一个经过文本优化的执行者利用这些诊断以及一个持久的、自我演化的可重用代码片段技能记忆合成子策略。该架构不仅提供透明的突变轨迹，还实现了跨运行的程序知识转移。在控制基准（Lunar Lander、Acrobot、Pendulum）和一个36维天线阵列合成任务上的广泛评估显示出卓越的样本效率。值得注意的是，REFLEX在不到10次LLM调用内解决了Acrobot和Pendulum问题，并在Lunar Lander上达到了1.092的最佳归一化加权分数，取得了高度竞争的最终性能，同时显著加速了透明策略的早期发现。

View on arXiv Download PDF AI Translation

cs.CL / 102 / 2606.16523

SkillWiki: A Living Knowledge Infrastructure for Agent Skills

SkillWiki：一种用于智能体技能的动态知识基础设施

Huang, Dingcheng, Ding, Yuda, Liu, Bingshuo, Liu, Qingbin, Chen, Xi, Bian, Jiang, Sun, Hongliang, Tu, Zhiying, Chu, Dianhui, Yu, Xiaoyan, Sui, Dianbo

Abstract

While knowledge is managed through Wikipedia and software through GitHub, agent skills still lack an infrastructure for large-scale production, governance, and evolution. SkillWiki is a living knowledge infrastructure that supports the organization, grounding, and continuous evolution of agent skills by transforming heterogeneous knowledge into reusable skill assets linked to their originating evidence. Our demonstration presents the complete skill lifecycle, from knowledge ingestion and skill production to provenance-aware exploration, governance, and execution-driven evolution. SkillWiki highlights a future in which knowledge, skills, and execution experience co-evolve within a shared infrastructure. The live demonstration and source code are publicly available at https://github.com/Huangdingcheng/SkillWiki.

Chinese Translation

尽管知识通过维基百科进行管理，软件通过GitHub进行管理，但智能体技能仍然缺乏用于大规模生产、治理和演化的基础设施。SkillWiki是一种动态知识基础设施，通过将异构知识转化为与其来源证据相关联的可重用技能资产，支持智能体技能的组织、基础和持续演化。我们的演示展示了完整的技能生命周期，从知识摄取和技能生产到基于来源的探索、治理和执行驱动的演化。SkillWiki展现了一个未来，在这个未来中，知识、技能和执行经验在共享基础设施中共同演化。实时演示和源代码可在 https://github.com/Huangdingcheng/SkillWiki 上公开获取。

View on arXiv Download PDF AI Translation

cs.CL / 103 / 2606.16545

Can LLM Coding Agents Reason About Time Series?

大型语言模型编码代理能否推理时间序列？

Rechtorík, Filip, Dušek, Ondřej, Kasner, Zdeněk

Abstract

Large language models (LLMs) are increasingly being used for automated decision-making systems in finance, healthcare, or environmental monitoring. Time series data are ubiquitous in these fields, yet hard to process automatically. Can time series be analyzed by LLM agents? We examine three approaches: providing the agent with raw numerical data, using the LLM as a coding agent, or a combination of both. In the coding agent setup, the model iteratively queries the data using Python code. Using two time series understanding benchmarks, we show that agents with code access can outperform models processing raw data by up to 10%. However, even the best performing agent still answers about 22-34% of the questions incorrectly. To get insights into models' strategies and reasoning gaps, we analyze the model outputs with a strong LLM judge. Our analysis reveals that coding agents can select appropriate statistical tests, but often miss important nuances. Meanwhile, models with access to raw data can reach the right conclusions using back-of-the-envelope calculations.

Chinese Translation

大型语言模型（LLMs）在金融、医疗保健或环境监测等自动决策系统中的应用日益增多。时间序列数据在这些领域中无处不在，但自动处理却十分困难。LLM代理能否分析时间序列？我们考察了三种方法：向代理提供原始数值数据、将LLM用作编码代理，或两者的结合。在编码代理设置中，模型通过Python代码迭代查询数据。通过两个时间序列理解基准，我们展示了具有代码访问权限的代理可以比处理原始数据的模型提高多达10%的表现。然而，即使是表现最好的代理，仍然有约22-34%的问题回答错误。为了深入了解模型的策略和推理缺陷，我们使用强大的LLM评判者分析模型输出。我们的分析揭示，编码代理能够选择合适的统计检验，但常常忽视重要的细微差别。同时，具有原始数据访问权限的模型可以通过简单的估算得出正确结论。

View on arXiv Download PDF AI Translation

cs.CL / 104 / 2606.16560

The BD-LSC Dataset: Facilitating the Benchmarking of Models for Lexical Semantic Change Detection in Slang and Standard Usage

BD-LSC 数据集：促进俚语与标准用法中词汇语义变化检测模型的基准测试

Aloraini, Afnan, Schlegel, Viktor, Nenadic, Goran, Batista-Navarro, Riza

Abstract

Automatic semantic change detection aims to identify how word meanings shift over time, offering insights into both linguistic and societal change. Despite recent progress in computational lexical semantic change (LSC), existing benchmarks and methods struggle to capture bi-directional semantic change, particularly cases where words simultaneously gain and lose senses. This problem is especially challenging for words that have both slang and standard meanings. To address these gaps, we introduce two complementary benchmark datasets. The Bi-Directional Lexical Semantic Change (BD-LSC) dataset captures sense gain, sense loss, and stability across three time periods, enabling the study of complex semantic trajectories. The SlangTrack Word Sense Disambiguation (ST-WSD) dataset provides fine-grained, instance-level sense annotations for words combining slang and standard usages, supporting systematic benchmarking of WSD and semantic change detection models. Using these benchmarks, we systematically evaluate models across different methodological families: unsupervised clustering using contextualised embeddings, supervised machine learning, transformer-based models, and state-of-the-art large language models. Among the evaluated systems, the few-shot GPT-4o model achieved the strongest aggregate performance on Exact Sense Match (ESM) and multi-label accuracy; however, Macro-F1 scores near 0.5 across all systems show that rare slang senses remain difficult, which we identify as the central open challenge.

Chinese Translation

自动语义变化检测旨在识别词义随时间的变化，为语言和社会变迁提供洞察。尽管在计算词汇语义变化（LSC）方面取得了近期进展，但现有的基准和方法仍难以捕捉双向语义变化，特别是在某些情况下，词汇同时获得和失去意义。这一问题对于同时具有俚语和标准意义的词汇尤其具有挑战性。为了解决这些空白，我们引入了两个互补的基准数据集。双向词汇语义变化（BD-LSC）数据集捕捉了三个时间段内的意义获得、意义丧失和稳定性，从而能够研究复杂的语义轨迹。俚语跟踪词义消歧（ST-WSD）数据集为结合俚语和标准用法的词汇提供了细粒度的实例级意义注释，支持词义消歧和语义变化检测模型的系统基准测试。利用这些基准，我们系统地评估了不同方法论家族中的模型：使用上下文化嵌入的无监督聚类、监督机器学习、基于变换器的模型以及最先进的大型语言模型。在评估的系统中，少样本 GPT-4o 模型在精确意义匹配（ESM）和多标签准确性上表现最佳；然而，所有系统的宏观 F1 分数接近 0.5，显示稀有俚语意义仍然难以处理，我们将其视为核心的开放挑战。

View on arXiv Download PDF AI Translation

cs.CL / 105 / 2606.16568

Fast When, Careful Who: Dual-Process Multiparty Turn-Taking with Diffusion Augmentation

快速何时，谨慎谁：扩散增强的双过程多方轮流发言

Patamia, Rutherford A., Liu, Ming, Luo, Wei, Ekong, Favour, Cosgun, Akan

Abstract

Reliable turn-taking is essential for spoken dialogue systems. However, most existing methods are designed for two-speaker interaction and struggle with realistic multiparty audio containing overlap and rapid speaker changes. We study multiparty turn-taking on the VoxConverse dataset and propose an audio-only two-stage pipeline that separates when to trigger a turn boundary from whether the floor is actually transferring. A fast trigger scans the audio and proposes candidate end-of-turn times, while a lightweight verifier runs only at those times to decide \textsc{Hold} or \textsc{Shift} and support next-speaker prediction. We report results in the full multiparty setting and a controlled dyadic top-2 projection for comparability. We also investigate diffusion-based, label-preserving background-audio mixing as a data augmentation strategy. Results show improved shift detection over a baseline, with further improvements from diffusion augmentation.

Chinese Translation

可靠的轮流发言对于口语对话系统至关重要。然而，大多数现有方法是为两位发言者的互动设计的，难以处理包含重叠和快速发言者变化的真实多方音频。我们在VoxConverse数据集上研究多方轮流发言，并提出了一种仅基于音频的两阶段管道，将触发轮转边界的时机与实际转移发言权的判断分开。快速触发器扫描音频并提出候选的发言结束时间，而轻量级验证器仅在这些时间点运行，以决定是 extsc{Hold}还是 extsc{Shift}，并支持下一位发言者的预测。我们在完整的多方设置和受控的双人前两名投影中报告结果，以便进行比较。我们还研究了基于扩散的、保持标签的背景音频混合作为数据增强策略。结果显示，相较于基线，转移检测有所改善，而扩散增强进一步提升了效果。

View on arXiv Download PDF AI Translation

cs.CL / 106 / 2606.16576

Can LLM Agents Infer World Models? Evidence from Agentic Automata Learning

大型语言模型代理能推断世界模型吗？来自代理自动机学习的证据

Menaged, Reef, Lior, Gili, Ravfogel, Shauli, Aharoni, Roee, Stanovsky, Gabriel

Abstract

We propose agentic automata learning to evaluate the extent to which tool-calling LLM agents can uncover hidden environments through interaction. In our setup, an agent should uncover a hidden deterministic finite automaton (DFA) by interacting with an oracle through (1) membership queries ("Does this string belong to the target language?") and (2) equivalence queries ("Is this the target DFA?"). This yields a scalable testbed with controlled task complexity, measurable interaction efficiency, and strong baselines (classic automata-learning algorithms). Evaluating state-of-the-art LLMs, we find that performance drops sharply as DFA size increases. Reasoning models are markedly stronger than non-reasoning models, yet trajectory analyses reveal recurring failures in query planning, evidence integration, and hypothesis construction. Overall, our results show that current LLM agents can sometimes perform non-trivial interactive discovery, but remain far less robust and efficient than classic algorithms for the task.

Chinese Translation

我们提出了代理自动机学习，以评估工具调用型大型语言模型（LLM）代理通过交互揭示隐藏环境的能力。在我们的设置中，代理应通过与一个oracle的交互，揭示一个隐藏的确定性有限自动机（DFA），交互方式包括（1）成员查询（“这个字符串属于目标语言吗？”）和（2）等价查询（“这是目标DFA吗？”）。这提供了一个可扩展的测试平台，具有可控的任务复杂性、可测量的交互效率和强大的基准（经典的自动机学习算法）。在评估最先进的LLM时，我们发现随着DFA大小的增加，性能急剧下降。推理模型明显强于非推理模型，但轨迹分析揭示了查询规划、证据整合和假设构建中的反复失败。总体而言，我们的结果表明，当前的LLM代理有时能够进行非平凡的互动发现，但在任务的鲁棒性和效率上远不及经典算法。

View on arXiv Download PDF AI Translation

cs.CL / 107 / 2606.16583

Uncertainty Is Not a Safety Net for Clinical VQA, but Can It Anticipate Model Failure?

不确定性并不是临床视觉问答的安全网，但它能否预见模型失败？

Fazla, Arnisa, Testoni, Alberto, Abu-Hanna, Ameen, Plank, Barbara, Calixto, Iacer

Abstract

Safe deployment of clinical vision-language models (VLMs) requires reliable uncertainty estimation (UE): a signal indicating when predictions should be trusted or escalated to a clinician. We test whether current UE methods actually deliver this signal. Benchmarking 8 methods across 12 VLMs on clinical visual question-answering (VQA), we find that UE quality is not an intrinsic property of the UE method: it tracks model accuracy, degrading precisely where the model performance is weakest, and therefore where reliability is most needed. When we stress-test models by hiding the correct option among the multiple-choice answers (NOTA perturbations), accuracy collapses while uncertainty barely changes, leaving models systematically miscalibrated. Yet, we find that uncertainty on the unperturbed input reliably anticipates which predictions will collapse under NOTA, indicating that UE in current VLMs carries diagnostic information about model fragility. Our results position UE as a diagnostic tool for identifying fragile predictions and motivate perturbation-based evaluation as a path toward safe clinical deployment.

Chinese Translation

临床视觉语言模型（VLMs）的安全部署需要可靠的不确定性估计（UE）：这是一个指示何时应信任预测或将其升级至临床医生的信号。我们测试当前的UE方法是否真正提供了这一信号。在临床视觉问答（VQA）中对12个VLMs的8种方法进行基准测试，我们发现UE质量并不是UE方法的内在属性：它跟踪模型的准确性，在模型性能最弱的地方下降，因此在最需要可靠性的地方也下降。当我们通过在多项选择答案中隐藏正确选项（NOTA扰动）来对模型进行压力测试时，准确性崩溃，而不确定性几乎没有变化，导致模型系统性地失去校准。然而，我们发现，在未扰动输入上的不确定性可靠地预示了哪些预测将在NOTA下崩溃，这表明当前VLM中的UE携带关于模型脆弱性的诊断信息。我们的结果将UE定位为识别脆弱预测的诊断工具，并激励基于扰动的评估作为安全临床部署的路径。

View on arXiv Download PDF AI Translation

cs.CL / 108 / 2606.16591

SING: Synthetic Intention Graph for Scalable Active Tool Discovery in LLM Agents

SING：用于大规模主动工具发现的合成意图图

Xiao, Qiao, Shi, Haochen, Gao, Yisen, Hu, Wenbin, Jing, Huihao, Zheng, Tianshi, Xu, Baixuan, Zhang, Ziheng, Wang, Weiqi, Li, Haoran, Bai, Jiaxin, Song, Yangqiu

Abstract

Large language model (LLM) agents increasingly rely on agent harnesses that manage context, tools, and multi-turn execution, making tools a central interface for acting in realistic digital environments. As harness-connected tool ecosystems expand to hundreds or thousands of APIs, services, and task-specific skills, exhaustive tool schema injection becomes costly and imposes a closed-world assumption that limits agents to a predefined static inventory. Retrieval-augmented tool selection offers a natural alternative, but existing one-shot retrieval methods often fail to align isolated tool descriptions with the agent's true task intention, especially in long-horizon tasks where required capabilities emerge through decomposition, observations, and newly induced subgoals. We propose SING, an intention-aware active tool discovery framework that builds an intention-tool graph linking user intentions, tool capabilities, and tool collaboration patterns, and dynamically retrieves tools according to evolving task states. Using a unified corpus of 7,471 tools, we evaluate SING on three real-world tool-use benchmarks. SING improves Global Recall@5 by up to 59.8% and downstream success rate by up to 28.9% over baselines, while reducing full-corpus tool-schema exposure by 99.8%, demonstrating that intention-aware graph structure enables more accurate and context-efficient tool discovery in large-scale agentic ecosystems.

Chinese Translation

大型语言模型（LLM）代理越来越依赖于管理上下文、工具和多轮执行的代理框架，使得工具成为在现实数字环境中行动的核心接口。随着连接代理的工具生态系统扩展到数百或数千个API、服务和特定任务技能，全面的工具模式注入变得成本高昂，并且施加了一个封闭世界假设，限制代理只能使用预定义的静态库存。增强检索的工具选择提供了一种自然的替代方案，但现有的一次性检索方法往往无法将孤立的工具描述与代理的真实任务意图对齐，特别是在需要通过分解、观察和新诱导子目标出现的长期任务中。我们提出了SING，一个关注意图的主动工具发现框架，构建了一个链接用户意图、工具能力和工具协作模式的意图-工具图，并根据不断变化的任务状态动态检索工具。通过使用一个包含7,471个工具的统一语料库，我们在三个真实世界的工具使用基准上评估了SING。SING在Global Recall@5上提高了多达59.8%，在下游成功率上提高了多达28.9%，同时将全语料库工具模式的曝光减少了99.8%，证明了关注意图的图结构能够在大规模代理生态系统中实现更准确和上下文高效的工具发现。

View on arXiv Download PDF AI Translation

cs.CL / 109 / 2606.16596

How Far Can Machine Translation Quality Take You? Extrinsic Discourse Evaluation in Goal-Oriented Setups

机器翻译质量能带你多远？目标导向设置中的外部话语评估

Mohammed, Wafaa, Naszadi, Kata, Niculae, Vlad

Abstract

Existing machine translation (MT) metrics and discourse-focused evaluations primarily assess translation quality intrinsically, without measuring the downstream consequences of translation errors. In this work, we focus on extrinsic discourse evaluation of machine translation under two distinct regimes: static and interactive. Under the static regime, we propose an entity counting task as a probe of referential consistency in discourse. We show that high intrinsic MT quality does not reliably predict downstream discourse success and strong MT systems still produce referential inconsistencies. For the interactive regime, we study the goal-oriented multi-agent Welfare Diplomacy game as a probe of long-horizon communication and coordination. We find that interaction-specific translation failures impact downstream coordination. Our results highlight goal-oriented environments as a viable framework for discourse-sensitive extrinsic MT evaluation.

Chinese Translation

现有的机器翻译（MT）指标和以话语为中心的评估主要从内在角度评估翻译质量，而未衡量翻译错误的下游影响。在本研究中，我们关注机器翻译的外部话语评估，分为两种不同的模式：静态模式和交互模式。在静态模式下，我们提出了一项实体计数任务，以探测话语中的指称一致性。我们表明，高内在MT质量并不能可靠地预测下游话语的成功，而强大的MT系统仍然会产生指称不一致的情况。对于交互模式，我们研究了以目标为导向的多智能体福利外交游戏，作为长期沟通和协调的探测工具。我们发现，特定于交互的翻译失败会影响下游协调。我们的结果强调了以目标为导向的环境作为话语敏感的外部MT评估的可行框架。

View on arXiv Download PDF AI Translation

cs.CL / 110 / 2606.16603

VeriGraph: Towards Verifiable Data-Analytic Agents

VeriGraph：迈向可验证的数据分析智能体

Jin, Jiajie, Yang, Zhao, Liao, Wenle, Hu, Yuyang, Dong, Guanting, Li, Xiaoxi, Zhu, Yutao, Dou, Zhicheng

Abstract

LLM-based agents have demonstrated strong capabilities in data-intensive analytical tasks, yet their outputs are rarely verifiable: a reliance on linear text trajectories makes their reasoning difficult to audit. In particular, deterministic computations over raw data and semantic deductions over natural-language claims are often entangled in an unstructured stream, leaving numerical conclusions hard to reproduce and qualitative judgments hard to inspect. To address this, we propose VeriGraph, a traceable neuro-symbolic reasoning framework that enables agents to construct an explicit heterogeneous evidence directed acyclic graph (DAG) during execution. VeriGraph introduces three evidence-expansion primitives, namely computational, grounding, and derivational expansion, to connect raw data, interpreter variables, computed results, and natural-language claims in a unified graph. Under this formulation, structural traceability is reduced to graph reachability from raw data sources to terminal claims, while semantic support is measured by claim-level evidence evaluation. To improve graph construction, we further design a graph-based policy optimization strategy with a composite reward that jointly supervises answer correctness, computational integrity, and derivational coherence. Experiments on four benchmarks show that VeriGraph-8B achieves the highest overall score among all baselines. More importantly, VeriGraph produces auditable evidence graphs with substantially stronger claim grounding, achieving a 87.61\% Grounding Rate under our claim-level evidence support evaluation. These results suggest that explicit evidence-graph construction is a promising path toward verifiable data-analytic agents. Our code is available at https://github.com/ignorejjj/VeriGraph.

Chinese Translation

基于大型语言模型（LLM）的智能体在数据密集型分析任务中展现出强大的能力，但其输出结果往往难以验证：对线性文本轨迹的依赖使得其推理过程难以审计。特别是，对原始数据的确定性计算和对自然语言声明的语义推导通常交织在一个非结构化的流中，导致数值结论难以重现，定性判断难以检查。为了解决这个问题，我们提出了VeriGraph，一个可追溯的神经符号推理框架，使智能体在执行过程中能够构建一个明确的异构证据有向无环图（DAG）。VeriGraph引入了三种证据扩展原语，即计算扩展、基础扩展和推导扩展，以在统一图中连接原始数据、解释变量、计算结果和自然语言声明。在这种框架下，结构可追溯性简化为从原始数据源到终端声明的图可达性，而语义支持通过声明级证据评估来衡量。为了改善图的构建，我们进一步设计了一种基于图的策略优化策略，采用复合奖励共同监督答案的正确性、计算的完整性和推导的一致性。在四个基准测试上的实验表明，VeriGraph-8B在所有基线中取得了最高的整体得分。更重要的是，VeriGraph生成的可审计证据图具有显著更强的声明基础，在我们的声明级证据支持评估中达到了87.61%的基础率。这些结果表明，明确的证据图构建是实现可验证数据分析智能体的有希望的路径。我们的代码可在 https://github.com/ignorejjj/VeriGraph 获取。

View on arXiv Download PDF AI Translation

cs.CL / 111 / 2606.16617

Sycophancy as Material Failure under Pushback Loading: A Multi-Axis Characterization Across Three Loading Cases and up to Seventeen Material Charges

在反推加载下的谄媚作为材料失效：跨越三种加载案例和多达十七种材料特性的多轴特征化

Schessl, Ferdinand M.

Abstract

Sycophancy in LLMs is documented across 70+ papers, but expert agreement on construct boundaries remains low (ICC=.184; Ye et al., 2026). The construct fragments because behavioral classification depends on which surface form is privileged. We adopt a materials-science framing: conversation as test specimen under load, LLM-model as material charge, pushback as progressive load, stance-flip as material failure. We characterize this failure across three loading cases (debate n=1000; false-presuppositions n=3400; ethical-setting n=3400; 10-17 material charges per case; 7800 specimens total) using 14 turn-level axis-measurements spanning velocity, damage accumulation, frame-drift, brittleness, and direction stability, plus three speaker-resolved axes from an independent pipeline. The measurements are Hooke-coupled ($\sigma = E \cdot \varepsilon$ analog) and reproduce across loading cases with effects up to $|r_{rb}| = 0.35$ on debate; the sign structure adds a second pattern: the ethical-setting case inverts the velocity and accumulation blocks. Variance composition partitions into two profiles: debate is charge-dominated (brittle-fracture-like: the material grade decides), false-presuppositions and ethical-setting are topic-dominated (creep-like: the load decides); the ratios (2.03 vs 0.13/0.17) are estimator-dependent, for debate even in direction. Cross-judge reliability (GPT-4o vs Haiku 4.5) shows debate scoring is judge-robust (Cohen's $\kappa = 0.88$) while false-presupposition scoring is judge-sensitive ($\kappa = 0.36$) -- a caveat single-judge benchmarks must report. This is the methodological move Ye et al.'s diagnosis calls for: a multi-axis characterization that does not depend on which surface form of the construct one privileges.

Chinese Translation

在大型语言模型（LLMs）中，谄媚现象已在70多篇论文中被记录，但专家对构念边界的共识仍然较低（ICC=.184；Ye等，2026）。该构念的碎片化是因为行为分类依赖于优先考虑的表面形式。我们采用材料科学的框架：将对话视为在负载下的测试样本，将LLM模型视为材料特性，将反推视为渐进负载，将立场翻转视为材料失效。我们在三种加载案例中对这种失效进行特征化（辩论 n=1000；错误假设 n=3400；伦理设置 n=3400；每个案例10-17种材料特性；总计7800个样本），使用14个转级别的轴向测量，涵盖速度、损伤积累、框架漂移、脆性和方向稳定性，以及来自独立管道的三个说话者解析轴。这些测量是胡克耦合的（$ ext{σ} = E imes ext{ε}$ 类比），并在加载案例中重现，辩论的效果高达$|r_{rb}| = 0.35$；符号结构增加了第二种模式：伦理设置案例反转了速度和积累块。方差组成分为两种特征：辩论以特性为主导（类似脆性断裂：材料等级决定），错误假设和伦理设置以主题为主导（类似蠕变：负载决定）；比率（2.03 vs 0.13/0.17）依赖于估计器，辩论中甚至在方向上也是如此。跨评估者可靠性（GPT-4o vs Haiku 4.5）显示辩论评分具有评估者稳健性（Cohen's $ ext{κ} = 0.88$），而错误假设评分则对评估者敏感（$ ext{κ} = 0.36$）——这是单评估者基准必须报告的警告。这是Ye等人诊断所要求的方法论转变：一种不依赖于优先考虑的构念表面形式的多轴特征化。

View on arXiv Download PDF AI Translation

cs.CL / 112 / 2606.16629

Islamic Large Language Models: From Knowledge Acquisition to Trustworthy and Hallucination-Resistant AI

伊斯兰大型语言模型：从知识获取到可信赖且抗幻觉的人工智能

Mouhoub, Mohammed Amine

Abstract

Large language models (LLMs) are increasingly used for knowledge-intensive question answering, including religious and legal questions. Islamic knowledge is a particularly demanding setting: answers are expected to be grounded in authoritative sources, citations must be exact, Arabic varieties differ substantially from the language of classical sources, and legitimate jurisprudential disagreement must be represented rather than collapsed into a single answer. This survey reviews the emerging field of Islamic LLMs and trustworthy Islamic AI. We organize the literature around Arabic NLP and Arabic-centric LLMs, Islamic NLP resources, Qur'anic question answering, Islamic knowledge benchmarks, retrieval-augmented generation, Islamic legal reasoning, inheritance reasoning, hallucination evaluation, and trustworthiness. We argue that fluency in Arabic is not sufficient for Islamic AI. Reliable systems require curated sources, retrieval and verification modules, citation-aware generation, madhhab-aware reasoning, human expert evaluation, and benchmarks that measure not only answer accuracy but also faithfulness, source validity, and reasoning quality. The survey concludes with a research agenda for hallucination-resistant Islamic AI systems.

Chinese Translation

大型语言模型（LLMs）在知识密集型问答中被越来越广泛地使用，包括宗教和法律问题。伊斯兰知识是一个特别具有挑战性的领域：答案需要基于权威来源，引用必须准确，阿拉伯语的不同方言与经典来源的语言差异显著，合法的法理分歧必须得到体现，而不是简化为单一答案。本文综述了伊斯兰LLMs和可信赖的伊斯兰人工智能这一新兴领域。我们围绕阿拉伯语自然语言处理（NLP）和以阿拉伯语为中心的LLMs、伊斯兰NLP资源、古兰经问答、伊斯兰知识基准、检索增强生成、伊斯兰法律推理、继承推理、幻觉评估和可信度等文献进行了组织。我们认为，仅仅精通阿拉伯语并不足以构建伊斯兰人工智能。可靠的系统需要经过策划的来源、检索和验证模块、关注引用的生成、考虑教派的推理、人类专家评估，以及不仅测量答案准确性，还测量忠实性、来源有效性和推理质量的基准。本文最后提出了抗幻觉的伊斯兰人工智能系统的研究议程。

View on arXiv Download PDF AI Translation

cs.CL / 113 / 2606.16659

FraudSMSWalker: Benchmarking Agentic Large Language Models for SMS-to-Webpage Fraud Detection

FraudSMSWalker：基于代理的大型语言模型在短信到网页欺诈检测中的基准评估

Zhou, Y. H., Ma, Z. M., Zhou, Y. J., Li, Y. T., Xiang, H. X., Cheng, Y. M., Chen, T. L., Zhang, K. J., Nan, Z. H., Ni, J. H., Wu, Z., Pan, Q. Y., Zhang, S., Cheng, S., Luo, M. Y.

Abstract

SMS fraud is increasingly cross-channel: a message directs the user to a webpage, and the final risk depends on how the SMS claim aligns with the page content and requested user action. However, existing evaluations either focus on message-only smishing classification or expose URL and domain cues that allow models to rely on reputation shortcuts. To address this gap, we introduce \textbf{FraudSMSWalker}, a controlled benchmark for URL-masked SMS-to-webpage fraud judgment. FraudSMSWalker contains 699 bilingual chains, including 332 fraudulent and 367 benign cases, across ten service scenarios. The model-visible input consists of the SMS context and sanitized webpage evidence, while raw URLs, hosts, domains, IPs, redirects, and reputation metadata are withheld. The benchmark further includes hard benign cases whose pages contain login, payment, verification, or account-management elements that are plausible under the service context but also appear in scam flows. We evaluate nine web agents under masked browser-agent protocols and conduct URL-visibility ablations. The results show that current agents can detect suspicious cues, but struggle to preserve benign recall and often produce positive predictions that are weakly supported by the observed evidence. These findings position FraudSMSWalker as a benchmark for measuring whether web agents can make fraud judgments that remain both accurate and evidence-grounded when direct reputation shortcuts are suppressed. The associated code and dataset are accessible at the \href{https://anonymous.4open.science/w/FraudMessageWalker-Bench}{anonymous link}.

Chinese Translation

短信欺诈日益呈现跨渠道特征：一条消息引导用户访问网页，最终风险取决于短信声明与网页内容及用户所需操作的一致性。然而，现有评估要么仅关注消息本身的钓鱼短信分类，要么暴露出允许模型依赖声誉捷径的URL和域名线索。为了解决这一问题，我们引入了 extbf{FraudSMSWalker}，这是一个针对URL掩蔽的短信到网页欺诈判断的受控基准。FraudSMSWalker包含699个双语链条，包括332个欺诈案例和367个良性案例，涵盖十种服务场景。模型可见的输入包括短信上下文和清理后的网页证据，而原始URL、主机、域名、IP、重定向和声誉元数据则被隐去。该基准还包括一些困难的良性案例，这些页面包含在服务上下文下看似合理但也出现在欺诈流程中的登录、支付、验证或账户管理元素。我们在掩蔽的浏览器代理协议下评估了九个网络代理，并进行了URL可见性消融实验。结果表明，当前的代理能够检测可疑线索，但在保持良性召回方面表现不佳，且经常产生与观察到的证据支持较弱的正预测。这些发现使FraudSMSWalker成为一个基准，用于衡量网络代理在抑制直接声誉捷径时，是否能够做出既准确又基于证据的欺诈判断。相关代码和数据集可在 exttt{https://anonymous.4open.science/w/FraudMessageWalker-Bench}访问。

View on arXiv Download PDF AI Translation

cs.CL / 114 / 2606.16684

Progressive Knowledge-Guided Large Language Model Framework for Bearing Fault Diagnosis

渐进式知识引导的大型语言模型框架用于轴承故障诊断

Wang, Jinghan, Peng, Gaoliang, Chen, Yanjun, Zhang, Wei, Wu, Wentao, Liu, Tianchen

Abstract

Vibration-based bearing fault diagnosis requires resolving three interrelated measurement challenges, including the trade-off between global statistical feature efficiency and local transient signal fidelity, insufficient traceability of measurement features to underlying fault physics, and ineffective multi-source measurement information fusion across diagnostic scales. This paper presents a progressive physics-guided multi-scale vibration signal processing framework that addresses all three challenges within a unified diagnostic pipeline. An 81-dimensional measurement descriptor, derived from bearing kinematic theory and characteristic defect frequencies, establishes a physically traceable feature space enabling real-time fault screening at approximately 20 ms per sample. A fault-adaptive signal segmentation mechanism then directs analytical attention toward fault-relevant waveform regions guided by physics-based priors, without manual feature engineering. Structured fault mechanism knowledge is further encoded implicitly in model parameters during training, enabling autonomous multi-scale measurement fusion without external knowledge dependencies at inference. Validated on four public benchmark datasets under diverse operating conditions, the framework achieves 98.49% diagnostic accuracy with a 12.6-fold reduction in computational cost relative to signal-level baselines. Interpretability analysis confirms that diagnostic feature activations align with established bearing fault mechanics, supporting measurement traceability in safety-critical industrial systems.

Chinese Translation

基于振动的轴承故障诊断需要解决三个相互关联的测量挑战，包括全局统计特征效率与局部瞬态信号保真度之间的权衡、测量特征与潜在故障物理之间的可追溯性不足，以及跨诊断尺度的多源测量信息融合效果不佳。本文提出了一种渐进式物理引导的多尺度振动信号处理框架，旨在在统一的诊断流程中解决这三项挑战。基于轴承运动学理论和特征缺陷频率，构建的81维测量描述符建立了一个物理可追溯的特征空间，使得每个样本的实时故障筛查时间约为20毫秒。接着，故障自适应信号分割机制将分析重点引导至与故障相关的波形区域，基于物理先验进行指导，无需手动特征工程。在训练过程中，结构化的故障机制知识进一步隐式编码在模型参数中，使得在推理时能够实现自主的多尺度测量融合，而不依赖外部知识。在多种操作条件下对四个公共基准数据集进行验证，该框架实现了98.49%的诊断准确率，相较于信号级基线计算成本降低了12.6倍。可解释性分析确认，诊断特征激活与已建立的轴承故障力学一致，支持在安全关键工业系统中的测量可追溯性。

View on arXiv Download PDF AI Translation

cs.CL / 115 / 2606.16700

Multi-Turn Reflective Masking Elicits Reasoning in Mask Diffusion Models

多轮反思掩蔽在掩蔽扩散模型中的推理引发

Zhang, Yanming, Bian, Yihan, Qi, Jingyuan, Yao, Yuguang, Huang, Lifu, Zhou, Tianyi

Abstract

While reasoning on autoregressive (AR) models is often performed by chain-of-thought reasoning and reflection, their refinement of previous outputs still relies on fully sequential generation, even when only local edits are needed. In contrast, the masking mechanism in Mask Diffusion Models (MDMs) naturally supports explicit local edits on previous outputs, allowing selective refinement without discarding previous answers and generating another from scratch. While this property more closely aligns with how humans correct mistakes by iterative local refinement, existing MDMs do not support multi-turn masking and denoising. We propose Reflective Masking (RM), which elicits such an intrinsic reasoning capability in MDMs via lightweight post-training. RM provides a native test-time scaling, where an MDM iteratively revisits and revises its prior outputs based on evolving context. To exploit insights from previous turns like AR reasoning, we further introduce History Reference, a parameter-free mechanism that leverages intermediate denoising states during revision. Our approach requires no architectural changes and is easily applicable to existing MDMs. Across diverse tasks and modalities, including text generation, Sudoku, and image editing, Reflective Masking consistently outperforms standard masking-based baselines and demonstrates strong generality, positioning RM as a fundamental primitive for reasoning on MDMs.

Chinese Translation

在自回归（AR）模型中，推理通常通过思维链推理和反思进行，但它们对先前输出的精炼仍然依赖于完全顺序生成，即使只需要局部编辑。相比之下，掩蔽扩散模型（MDMs）中的掩蔽机制自然支持对先前输出的显式局部编辑，允许选择性精炼而不丢弃先前的答案并从头生成。虽然这一特性更贴近人类通过迭代局部精炼纠正错误的方式，但现有的MDMs并不支持多轮掩蔽和去噪。我们提出了反思掩蔽（RM），通过轻量级后训练在MDMs中引发这种内在的推理能力。RM提供了原生的测试时间扩展，其中MDM根据不断变化的上下文迭代地重新审视和修订其先前的输出。为了利用先前轮次的见解，如AR推理，我们进一步引入了历史参考（History Reference），这是一种无参数机制，在修订过程中利用中间去噪状态。我们的方法不需要架构更改，并且易于应用于现有的MDMs。在包括文本生成、数独和图像编辑等多种任务和模态中，反思掩蔽始终优于标准掩蔽基线，并展示出强大的通用性，使RM成为MDMs推理的基本原语。

View on arXiv Download PDF AI Translation

cs.CL / 116 / 2606.16753

P3B3: A Multi-Turn Conversational Benchmark for Measuring European and Brazilian Portuguese Variety Bias in LLMs

P3B3：用于测量大型语言模型中欧洲和巴西葡萄牙语变体偏差的多轮对话基准

Ferreira, Rafael, Vieira, Inês, Calvo, Inês, Furtado, James, Paulo, Iago, Tavares, Diogo, Glória-Silva, Diogo, Semedo, David, Magalhães, João

Abstract

As Large Language Models (LLMs) become embedded in everyday communication, capturing regional linguistic variation is essential for reliable and equitable language use. In Portuguese, European (pt-PT) and Brazilian (pt-BR) varieties remain unevenly represented, with pt-BR dominating in data quantity, while LLM preference for Portuguese variants remains underexplored. To address this gap, we introduce P3B3, an expert-curated language variety agnostic benchmark of conversational prompts, along with an evaluation framework for measuring variety bias and controllability. Experiments on several models show that most LLMs exhibit a strong bias toward pt-BR, with variation in controllability across models. These results highlight the need for more balanced multilingual representation across language varieties.

Chinese Translation

随着大型语言模型（LLMs）逐渐融入日常交流，捕捉区域语言变异对可靠和公平的语言使用至关重要。在葡萄牙语中，欧洲（pt-PT）和巴西（pt-BR）变体的代表性不均，pt-BR在数据数量上占主导地位，而LLM对葡萄牙语变体的偏好尚未得到充分探讨。为了解决这一问题，我们引入了P3B3，这是一个由专家策划的语言变体无关的对话提示基准，以及一个用于测量变体偏差和可控性的评估框架。对多个模型的实验表明，大多数LLMs对pt-BR表现出强烈的偏见，并且模型之间的可控性存在差异。这些结果突显了在语言变体之间实现更平衡的多语言代表性的必要性。

View on arXiv Download PDF AI Translation

cs.CL / 117 / 2606.16801

The Art of Mixology: Mixup-based Obfuscation for Privacy-Preserving Split Learning in Large Language Models

调酒艺术：基于混合的隐私保护分割学习在大语言模型中的应用

Chen, Chen, Gao, Xiang, Wang, Xianshun, Li, Chengran, Xia, Shengyu, Gong, Xueluan, Zhang, Linru, Wang, Qian, Lam, Kwok-Yan

Abstract

Split learning provides a practical paradigm for resource-constrained users to train Large Language Models (LLMs) by offloading computation-intensive layers to a server while keeping raw data local. However, existing privacy-preserving split learning methods still face a difficult trade-off among utility, privacy, efficiency, and stability. Specifically, these methods often suffer from substantial utility degradation, remain vulnerable to advanced data reconstruction attacks, incur prohibitive computational and communication overhead, or exhibit unstable performance across different tasks. In this paper, we propose MIXGUARD, a novel mixup-based privacy-preserving split learning framework for LLMs. MIXGUARD introduces token-level obfuscation, representation-level obfuscation, and adaptive gradient perturbation mechanisms, which operate jointly to preserve useful learning signals while preventing privacy leakage to the server. Technically, MIXGUARD first constructs a lightweight calibration model on a public dataset to refine the approximated target representation, and then applies this model during privacy-preserving fine-tuning on private data. We conduct extensive experiments on four classification tasks and four text generation tasks across multiple LLM families, model sizes, architectures, and fine-tuning strategies. The results show that MIXGUARD preserves model utility comparable to non-split training baselines, consistently achieves stronger privacy protection than existing split learning defense methods against state-of-the-art data reconstruction attacks, and remains robust under adaptive attack settings.

Chinese Translation

分割学习为资源受限的用户提供了一种实用的范式，使其能够通过将计算密集型层卸载到服务器上，同时保持原始数据在本地，从而训练大语言模型（LLMs）。然而，现有的隐私保护分割学习方法在效用、隐私、效率和稳定性之间仍面临艰难的权衡。具体而言，这些方法往往会遭遇显著的效用下降，仍然容易受到高级数据重构攻击的威胁，产生高昂的计算和通信开销，或者在不同任务中表现不稳定。本文提出了MIXGUARD，一种新颖的基于混合的隐私保护分割学习框架，专为LLMs设计。MIXGUARD引入了令牌级混淆、表示级混淆和自适应梯度扰动机制，这些机制共同作用，以保留有用的学习信号，同时防止隐私泄露到服务器。技术上，MIXGUARD首先在公共数据集上构建一个轻量级的校准模型，以优化近似目标表示，然后在私有数据上进行隐私保护的微调时应用该模型。我们在多个LLM家族、模型规模、架构和微调策略上，对四个分类任务和四个文本生成任务进行了广泛的实验。结果表明，MIXGUARD在模型效用上与非分割训练基线相当，始终在对抗最先进的数据重构攻击时实现比现有分割学习防御方法更强的隐私保护，并在自适应攻击设置下保持稳健。

View on arXiv Download PDF AI Translation

cs.CL / 118 / 2606.16806

LLM-based Visual Code Completion for Aerospace Geometric Design

基于大型语言模型的航空航天几何设计视觉代码补全

Yong, Hau Kit, Marsh, Robert, Silva, Edmar A., Sóbester, András, Middleton, Stuart E.

Abstract

Recent advances in both Large Language Models (LLMs) and Vision Language Models (VLMs) have seen a step change in their ability to perform visual code completion, but the aerospace industry, which prioritizes safety and explainabilty over rapid LLM adoption, currently has no publicly announced LLM-based geometric design copilot systems in commercial use by aerospace Original Equipment Manufacturers (OEMs). This paper presents a LLM-based visual programming copilot application for aerospace engineering design tasks, using a visual programming variant of the ReAct methodology and GPT 5.4. In addition to the copilot, we describe Wingbuilder, a new Grasshopper plugin library with custom components for aerospace-specific geometry abstraction, and an associated Aerospace Visual Programming Dataset (AVPD) with 18 aerospace expert designed tasks at different levels of difficulty alongside ground truth solutions. We evaluate our copilot application with a user trial involving two experienced aerospace engineers from a large aircraft manufacturing company. We find our copilot visual programming ReAct methodology was successful in generating suggestions that participants found helpful, but slow ReAct inference times limit its usefulness to more complex time-consuming tasks where waiting for good copilot solution suggestion was worthwhile. Participants reported they liked the tool and would be willing to use it in the future.

Chinese Translation

近期，大型语言模型（LLMs）和视觉语言模型（VLMs）的进展使得它们在视觉代码补全方面的能力发生了显著变化。然而，航空航天行业优先考虑安全性和可解释性，而非快速采用LLM，目前尚无航空航天原始设备制造商（OEM）公开宣布的基于LLM的几何设计助手系统在商业中使用。本文提出了一种基于LLM的视觉编程助手应用，旨在航空工程设计任务中，采用了ReAct方法论的视觉编程变体和GPT 5.4。除了助手，我们还描述了Wingbuilder，一个新的Grasshopper插件库，包含针对航空航天特定几何抽象的自定义组件，以及一个相关的航空航天视觉编程数据集（AVPD），该数据集包含18个由航空航天专家设计的不同难度级别的任务及其真实解答。我们通过一项用户试验评估了我们的助手应用，参与者为来自一家大型飞机制造公司的两位经验丰富的航空航天工程师。我们发现，助手的视觉编程ReAct方法论在生成参与者认为有帮助的建议方面取得了成功，但缓慢的ReAct推理时间限制了其在更复杂、耗时任务中的实用性，因为等待良好的助手解决方案建议是值得的。参与者表示他们喜欢这个工具，并愿意在未来使用它。

View on arXiv Download PDF AI Translation

cs.CL / 119 / 2606.16807

Connecting Speech to Words through Images

通过图像将语音与词汇连接起来

Pirlogeanu, Gabriel, Oneata, Dan, Cucu, Horia, Kamper, Herman

Abstract

How can we learn the mapping between written words and their spoken counterparts in the absence of explicit textual supervision? We present a visually grounded method for building a vocabulary of spoken words using only images and their spoken descriptions. First, image captioning systems are used to build a vocabulary of written words representing salient visual concepts in the images. For each word, we then find utterances whose image captions contain that word. Then we use an unsupervised word discovery technique to align these utterances to locate instances of the target word. The result is spoken word segments that are linked to written words -- all accomplished without any text supervision. In spoken word retrieval and keyword spotting experiments, the proposed approach outperforms a strong neural baseline while being more interpretable. These results demonstrate the feasibility of the approach in English and motivate future work on low-resource languages without transcripts.

Chinese Translation

在缺乏明确文本监督的情况下，我们如何学习书面词汇与其口语对应之间的映射？我们提出了一种视觉基础的方法，仅使用图像及其口语描述来构建口语词汇。首先，使用图像描述系统构建一个书面词汇，表示图像中显著的视觉概念。然后，对于每个词，我们寻找包含该词的图像描述的发音。接着，我们使用无监督的词汇发现技术对齐这些发音，以定位目标词的实例。最终结果是与书面词汇相关联的口语词段——这一切都是在没有任何文本监督的情况下完成的。在口语词检索和关键词识别实验中，所提出的方法在性能上超越了一个强大的神经基线，同时具有更好的可解释性。这些结果证明了该方法在英语中的可行性，并激励未来在没有转录的低资源语言上的研究。

View on arXiv Download PDF AI Translation

cs.CL / 120 / 2606.16817

Understanding the Behaviors of Environment-aware Information Retrieval

理解环境感知信息检索的行为

Yuan, Ruifeng, Yuan, Chaohao, Dai, David, Rong, Yu, Cheng, Hong, Chan, Hou Pong, Xiao, Chenghao

Abstract

Recent retrieval-augmented generation (RAG) approaches have demonstrated strong capability in handling complex queries, yet current research overlooks a critical challenge: different retrievers require fundamentally different query formulation strategies for optimal performance. In this work, we present the first systematic analysis of how LLMs can learn to adapt their query formulation strategies for different retrievers via reinforcement learning (RL). Our empirical study reveals that RL effectively teaches an LLM to tailor its queries to specific retriever characteristics. We discover that different retrievers exhibit surprisingly distinct optimal query styles (e.g., descriptive vs. question-like), suggesting strategies learned for one retriever ineffective for another. We further show that performance can be enhanced by incorporating retriever-specific human guidance and by scaling model size. To facilitate learning over multi-retrieval-step trajectories, we introduce a branching-based rollout technique that improves training stability. Our work provides the first empirical evidence and actionable insights for building truly retriever-aware RAG systems. Code and resources are available at https://github.com/LCO-Embedding/Envs-aware-Information-Retrieval.

Chinese Translation

最近的检索增强生成（RAG）方法在处理复杂查询方面展现了强大的能力，但当前研究忽视了一个关键挑战：不同的检索器需要根本不同的查询构造策略以实现最佳性能。在本研究中，我们首次系统性地分析了大型语言模型（LLMs）如何通过强化学习（RL）学习适应不同检索器的查询构造策略。我们的实证研究表明，RL有效地教会LLM根据特定检索器的特征调整其查询。我们发现，不同的检索器表现出惊人地不同的最佳查询风格（例如，描述性与类问题），这表明为一个检索器学习的策略对另一个检索器可能无效。我们进一步展示，通过结合特定检索器的人类指导和扩大模型规模，可以提升性能。为了促进多检索步骤轨迹的学习，我们引入了一种基于分支的回滚技术，以改善训练稳定性。我们的工作提供了构建真正感知检索器的RAG系统的首次实证证据和可操作的见解。代码和资源可在 https://github.com/LCO-Embedding/Envs-aware-Information-Retrieval 获取。

View on arXiv Download PDF AI Translation

cs.CL / 121 / 2606.16821

How Much Can We Trust LLM Search Agents? Measuring Endorsement Vulnerability to Web Content Manipulation

我们能多大程度上信任大型语言模型搜索代理？测量对网络内容操控的认可脆弱性

Chen, Yimeng, Ren, Zhe, Laakom, Firas, Li, Yu, Guo, Dandan, Schmidhuber, Jürgen

Abstract

Large language model (LLM)-based search agents synthesize open-web content into actionable recommendations on behalf of users, creating a risk that attacker-published pages are transformed into endorsed claims. We introduce SearchGEO, a controlled evaluation framework for measuring endorsement corruption in LLM-based web-search agents, combining a web-evidence manipulation pipeline, a five-mode attack taxonomy, and multiple output-level metrics. We evaluate 13 LLM backends on 308 cases each. Results show that vulnerability patterns vary across backends: overall attack success rate (ASR) ranges from 0.0% on Claude-Sonnet-4.6 to 31.4% on Gemini-3-Flash, the strongest attack mode differs by model family, and the same deployment scaffold could amplify or decrease ASR on different backends. An auxiliary agent-skill probe, where endorsement becomes an install command, exposes a sharp split among otherwise robust backends: Claude over-rejects while GPT over-trusts. These findings argue for treating recommendation reliability under adversarial search content as a first-class dimension of backend safety evaluation.

Chinese Translation

基于大型语言模型（LLM）的搜索代理将开放网络内容综合为可操作的建议，带来了攻击者发布的页面可能被转化为认可声明的风险。我们引入了SearchGEO，一个用于测量LLM基础网络搜索代理中认可腐败的受控评估框架，结合了网络证据操控管道、五种攻击分类法和多个输出级别指标。我们对13个LLM后端在每个308个案例上进行了评估。结果显示，脆弱性模式在不同后端之间存在差异：整体攻击成功率（ASR）从Claude-Sonnet-4.6的0.0%到Gemini-3-Flash的31.4%不等，最强攻击模式因模型家族而异，同一部署框架在不同后端上可能会放大或降低ASR。一个辅助的代理技能探测，其中认可变为安装命令，揭示了在其他情况下表现稳健的后端之间的明显分歧：Claude过度拒绝，而GPT过度信任。这些发现表明，应将对抗性搜索内容下的推荐可靠性视为后端安全评估的一个重要维度。

View on arXiv Download PDF AI Translation

cs.CL / 122 / 2606.16825

Tying the Loop -- Tied Expert Layers in Mixture-of-Experts Language Models

环的结合——混合专家语言模型中的绑定专家层

Jaggi, Martin

Abstract

Mixture-of-Experts (MoE) architectures efficiently scale Large Language Models (LLMs) by activating only a small fraction of their experts per token, yet the full parameter count - dominated by the expert parameters - must be held in training and inference memory. To address this, we introduce Expert Tying, an architectural modification that shares expert parameters across consecutive transformer layers while preserving independent, layer-wise routing and attention. We evaluate this approach across common, state-of-the-art architectures, including OLMoE, Qwen3, and DeepSeek-style MoEs. Our pretraining experiments demonstrate that tying experts can reduce memory footprint by almost 2x at virtually no degradation in perplexity or downstream quality. By exploiting the parameter redundancy inherent in MoE pathways, our method provides a highly favorable compute-to-memory trade-off, advancing efficient training and scaling of next-generation LLMs.

Chinese Translation

混合专家（Mixture-of-Experts, MoE）架构通过每个令牌仅激活一小部分专家，能够高效地扩展大型语言模型（Large Language Models, LLMs），但完整的参数数量——主要由专家参数主导——必须在训练和推理过程中保持在内存中。为了解决这个问题，我们引入了专家绑定（Expert Tying），这是一种架构修改，能够在保持独立的层级路由和注意力的同时，在连续的变换器层之间共享专家参数。我们在包括OLMoE、Qwen3和DeepSeek风格的MoE在内的常见前沿架构上评估了这种方法。我们的预训练实验表明，绑定专家可以将内存占用减少近2倍，而几乎不会降低困惑度或下游质量。通过利用MoE路径中固有的参数冗余，我们的方法提供了极为有利的计算与内存的权衡，推动了下一代LLMs的高效训练和扩展。

View on arXiv Download PDF AI Translation

cs.CL / 123 / 2606.16836

Does Traversal Order Matter? A Systematic Study of Tree Traversal Methods in Transformer Grammars

遍历顺序重要吗？对变换器语法中树遍历方法的系统研究

Liu, Zongru, Ji, Pengyu, Wang, Pengcheng, Tu, Kewei

Abstract

Transformer Grammars (TGs) enhance language modeling by incorporating syntactic tree structures. Despite the potentially significant impact on model performance of how syntactic trees are linearized in TGs, existing studies rely solely on Depth-First Traversal (DFT) for linearization. In this paper, we expand the traversal design space by exploring Breadth-First Traversal (BFT) and a novel hybrid traversal strategy, Production-Rule Traversal (PRT), which combines the structural lookahead of BFT with the early lexical generation of DFT. We integrate these traversal methods with varying tree configurations and masking strategies, and empirically evaluate their performance on language modeling, syntactic generalization and summarization. We reveal the inherent trade-offs between nested composition and global lookahead, providing actionable recommendations for designing task-aware Transformer Grammars.

Chinese Translation

变换器语法（Transformer Grammars, TGs）通过结合句法树结构来增强语言建模。尽管句法树在 TGs 中的线性化方式可能对模型性能产生显著影响，但现有研究仅依赖深度优先遍历（Depth-First Traversal, DFT）进行线性化。本文通过探索广度优先遍历（Breadth-First Traversal, BFT）和一种新颖的混合遍历策略——生成规则遍历（Production-Rule Traversal, PRT），扩展了遍历设计空间，后者结合了 BFT 的结构前瞻性和 DFT 的早期词汇生成。我们将这些遍历方法与不同的树配置和掩蔽策略相结合，并在语言建模、句法泛化和摘要生成上进行实证评估。我们揭示了嵌套组合与全局前瞻性之间的固有权衡，为设计任务感知的变换器语法提供了可行的建议。

View on arXiv Download PDF AI Translation

cs.CL / 124 / 2606.16843

Data-Driven Decoding of Russell's Circumplex Model of Affect

基于数据驱动的拉塞尔情感圆周模型解码

Belaref, Amdjed, Sadok, Samir, Noumir, Zineb, Seguier, Renaud

Abstract

Affective computing increasingly relies on deep learning to represent emotions, yet latent spaces often remain opaque, high-dimensional black boxes. This paper investigates whether Transformers' embeddings recover the geometric regularities of Russell's circumplex model. We unify two complementary experiments testing the hypothesis that, after training models on text and speech, their resulting latent spaces encode a topology consistent with valence-arousal and reproduce human-like neighborhood relations. Specifically, we evaluate deep representations extracted from Transformer-based text (RoBERTa) and speech (wav2vec 2.0) encoders, along with a multimodal Transformer fusion architecture, across naturalistic datasets like MSP-Podcast and controlled LLM-generated stimuli. Our analysis reveals that multimodal fusion of text and audio yields perfect topological alignment with Russell's primary emotion ordering. Furthermore, in a zero-shot setting using generic text embeddings, projected fine-grained emotion terms fall close to their established human-mapped coordinates. Our contribution is a novel, data-driven framework for validating emotion models, demonstrating that Russell's circumplex structure is intrinsically encoded in the embeddings of these modalities rather than being solely an artifact of human labeling, thereby bridging the gap between psychological theory and representation learning.

Chinese Translation

情感计算越来越依赖深度学习来表示情感，然而潜在空间往往仍然是模糊的、高维的黑箱。本文研究了变换器（Transformers）的嵌入是否能够恢复拉塞尔（Russell）情感圆周模型的几何规律。我们统一了两个互补实验，以检验假设：在对文本和语音进行模型训练后，得到的潜在空间编码了与效价-唤醒（valence-arousal）一致的拓扑结构，并再现了类人邻域关系。具体而言，我们评估了从基于变换器的文本（RoBERTa）和语音（wav2vec 2.0）编码器提取的深度表示，以及跨越自然数据集（如MSP-Podcast）和受控的LLM生成刺激的多模态变换器融合架构。我们的分析表明，文本和音频的多模态融合与拉塞尔的主要情感排序实现了完美的拓扑对齐。此外，在使用通用文本嵌入的零样本设置中，投影的细粒度情感术语接近其已建立的人类映射坐标。我们的贡献是提出了一种新颖的数据驱动框架，用于验证情感模型，证明拉塞尔的圆周结构在这些模态的嵌入中是内在编码的，而不仅仅是人类标记的产物，从而弥合了心理学理论与表示学习之间的鸿沟。

View on arXiv Download PDF AI Translation

cs.CL / 125 / 2606.16845

Robust Dual-Signal Fusion: Hybrid Neuro-Symbolic Gating with Compressed Chain-of-Thought Refinement for Irony Detection in Social Media Texts

鲁棒双信号融合：具有压缩思维链（Chain-of-Thought）精炼的混合神经符号门控模型用于社交媒体文本中的讽刺检测

Bhattacharjee, Ankit, Bhaumik, Krityapriya

Abstract

Large Language Models (LLMs) natively default to literal semantic interpretations, making zero-shot irony detection a persistent challenge. We introduce the Robust Dual-Signal (RDS) Fusion framework, a hybrid neuro-symbolic architecture that compresses Chain-of-Thought (CoT) reasoning trajectories without Supervised Fine-Tuning (SFT). Evaluated on a strictly held-out TweetEval test set (N=734), RDS achieves 78.1% accuracy and a Macro F1 of 0.777, matching the absolute performance ceiling of the fine-tuned BERTweet. On the heavily imbalanced iSarcasm dataset, the frozen CoT pipeline filters 22.5% of out-of-distribution hallucinations, yielding a zero-shot Macro F1 of 0.6726 and Ironic F1 of 0.4821, outperforming multiple heavily supervised SemEval transformer ensembles. A statistical ablation confirms this structural synergy: adding the symbolic prior to the neural baseline yields no significant gain (p = 0.242), and the marginal benefit of adding the CoT pipeline to that prior is heavily compressed (p = 0.149). Only the complete, concurrent fusion of all three signals achieves a statistically validated improvement over the baseline (p = 0.005).

Chinese Translation

大型语言模型（LLMs）本质上默认采用字面语义解释，使得零样本讽刺检测成为一个持续的挑战。我们提出了鲁棒双信号（RDS）融合框架，这是一种混合神经符号架构，能够在不进行监督微调（SFT）的情况下压缩思维链（CoT）推理轨迹。在严格保留的TweetEval测试集（N=734）上进行评估，RDS达到了78.1%的准确率和0.777的宏观F1值，匹配了微调BERTweet的绝对性能上限。在严重不平衡的iSarcasm数据集上，冻结的CoT管道过滤了22.5%的分布外幻觉，获得了零样本宏观F1值为0.6726，讽刺F1值为0.4821，超越了多个 heavily supervised SemEval变换器集成模型。统计消融分析确认了这种结构协同效应：将符号先验添加到神经基线并未带来显著增益（p = 0.242），而将CoT管道添加到该先验的边际效益则被大幅压缩（p = 0.149）。只有同时融合所有三个信号的完整模型才能实现对基线的统计验证改进（p = 0.005）。

View on arXiv Download PDF AI Translation

cs.CL / 126 / 2606.16847

Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokens

遵循潜在路线图：利用锚点在扩散大语言模型中导航可撤销解码

Yao, Yizhen, Zhu, Qinglin, Zhao, Runcong, Dai, Xiangxiang, Xiang, Yanzheng, He, Yulan, Gui, Lin

Abstract

Diffusion Large Language Models (dLLMs) offer a promising avenue for parallel generation but face a trade-off between decoding speed and quality. While revocable decoding strategies attempt to mitigate errors by verifying and remasking tokens, they typically operate within a mixed-quality context. This leads to two critical failures: \textit{Error Propagation}, where new tokens absorb toxic information from erroneous context, and \textit{Local Error Reinforcement}, where errors mutually reinforce each other to evade detection. To alleviate these challenges, we propose ASRD (Anchor Supervised Revocable Decoding), a training-free framework that operates within the embedding space. ASRD explicitly decouples the decoding context into trusted \textit{Anchor Tokens}, which are identified via temporal consistency, and uncertain candidates. Leveraging a dynamic Anchor Tokens Cache, we introduce two complementary mechanisms: (1) Anchor-Guided Generation, which injects entropy-weighted anchor signals into masked positions to implicitly rectify attention toward the reliable global skeleton; and (2) Anchor-Perturbed Verification, which applies orthogonal perturbations to uncertain candidate tokens, destabilizing and remasking errors driven by fragile local consensus. Extensive experiments on math and coding benchmarks demonstrate that ASRD outperforms recent remasking baselines, achieving accuracy improvements of up to 6.4\% while accelerating inference throughput by up to 7.2$\times$.

Chinese Translation

扩散大语言模型（dLLMs）为并行生成提供了一个有前景的途径，但在解码速度和质量之间存在权衡。尽管可撤销解码策略试图通过验证和重新掩蔽标记来减轻错误，但它们通常在混合质量的上下文中运行。这导致了两个关键的失败： extit{错误传播}，即新标记从错误上下文中吸收有害信息，以及 extit{局部错误强化}，即错误相互强化以逃避检测。为了解决这些挑战，我们提出了ASRD（锚点监督可撤销解码），这是一个在嵌入空间中运行的无训练框架。ASRD明确地将解码上下文解耦为可信的 extit{锚点标记}，这些标记通过时间一致性识别，以及不确定的候选标记。利用动态锚点标记缓存，我们引入了两种互补机制：（1）锚点引导生成，向掩蔽位置注入熵加权的锚点信号，以隐式地纠正对可靠全局框架的注意力；（2）锚点扰动验证，对不确定的候选标记施加正交扰动，破坏并重新掩蔽由脆弱局部共识驱动的错误。在数学和编码基准上的大量实验表明，ASRD在性能上优于最近的重新掩蔽基线，准确率提高了最高6.4\%，同时推理吞吐量加速了最高7.2$ imes$。

View on arXiv Download PDF AI Translation

cs.CL / 127 / 2606.16867

Revisiting the Systematicity in Negation in the Era of In-Context Learning

在上下文学习时代重新审视否定的系统性

Yanaka, Hitomi, Yamamoto, Taisei

Abstract

Understanding the meaning of negated sentences remains one of the challenges for language models, even in the era of large language models (LLMs). We analyze systematicity regarding LLM understanding of negation from two perspectives: behavioral systematicity and representational systematicity. For behavioral systematicity, we confirm that through demonstrations and in-context learning, LLMs can recognize negation expressions and scope within sentences to some extent, but they fail to achieve perfect performance. In particular, the difficulty of the negation scope recognition for models varies depending on the output format. For representational systematicity, we analyze the extent to which function vectors can be robustly constructed from in-context examples for tasks that are essential to understanding negation. The experiments suggest that while function vectors can be composed for negation cue extraction tasks, extracting function vectors for recognizing scope is more challenging.

Chinese Translation

理解否定句的意义仍然是语言模型面临的挑战之一，即使在大型语言模型（LLMs）时代。我们从两个角度分析了LLM对否定的理解的系统性：行为系统性和表征系统性。在行为系统性方面，我们确认通过示范和上下文学习，LLMs在一定程度上能够识别句子中的否定表达和范围，但它们未能达到完美的表现。特别是，模型对否定范围识别的难度因输出格式而异。在表征系统性方面，我们分析了功能向量在理解否定的关键任务中，能够从上下文示例中稳健构建的程度。实验表明，尽管功能向量可以用于否定线索提取任务，但提取用于识别范围的功能向量则更具挑战性。

View on arXiv Download PDF AI Translation

cs.CL / 128 / 2606.16874

Understanding Scam Trends and Rail Paths from Reddit Self-Disclosure Narratives

理解来自Reddit自我披露叙述的诈骗趋势和轨迹

Zhang, Yangjun, Bottarelli, Mirko, Hooper, Mark, Maple, Carsten

Abstract

Online scam behavior is inherently multi-stage, and the lifecycle includes temporally ordered rails and events rather than isolated signals. Existing works analyze characteristics of scam types and rails, but they do not track scam trends across years. Moreover, the work on the relations between rails is hampered due to the lack of open-source datasets with annotations and coverage of different scam types. To address these gaps, we build a dataset to analyze the yearly trend of scam characteristics and rail paths using Reddit self-disclosure narratives from 2023 to 2025. We collect 21,304 posts from scam-related subreddits with at least one rail among identity, communication, platform, and payment for trend analysis by heuristic annotation. Then, we label 1,800 posts containing explicit or recoverable scam chains by an LLM-assisted method for scam path analysis. The method is evaluated with human annotation. Lastly, we run a topic model on the comments of the posts to analyze the community support behavior. The results reveal that scam processes are predominantly multi-rail. Across years, different scam types and rail components dominate. Different scam types vary systematically in path complexity. Reddit support behaviors have become more detailed over time. This work supports synthetic scam chain data simulation and AI-related scam risk assessment, though findings may not generalise to other platforms.

Chinese Translation

在线诈骗行为本质上是多阶段的，其生命周期包括时间顺序排列的轨迹和事件，而非孤立的信号。现有研究分析了诈骗类型和轨迹的特征，但并未追踪多年来的诈骗趋势。此外，由于缺乏带注释的开源数据集以及对不同诈骗类型的覆盖，关于轨迹之间关系的研究受到限制。为了解决这些问题，我们构建了一个数据集，以分析2023年至2025年期间诈骗特征和轨迹的年度趋势，数据来源于Reddit自我披露叙述。我们从与诈骗相关的子版块收集了21,304条帖子，这些帖子至少包含一个关于身份、沟通、平台和支付的轨迹，以进行趋势分析，采用启发式注释方法。然后，我们使用基于大型语言模型（LLM）辅助的方法对1,800条包含明确或可恢复诈骗链的帖子进行了标注，以便进行诈骗路径分析。该方法经过人工注释进行评估。最后，我们对帖子评论进行了主题模型分析，以研究社区支持行为。结果表明，诈骗过程主要是多轨迹的。在不同年份中，不同的诈骗类型和轨迹组件占主导地位。不同的诈骗类型在路径复杂性上存在系统性差异。Reddit的支持行为随着时间的推移变得更加详细。这项工作支持合成诈骗链数据的模拟和与人工智能相关的诈骗风险评估，尽管研究结果可能无法推广到其他平台。

View on arXiv Download PDF AI Translation

cs.CL / 129 / 2606.16890

Compositional Reasoning Depth Predicts Clinical AI Failure: Empirical Evidence Consistent with Transformer Compositionality Limits in Electronic Health Record Question Answering

组合推理深度预测临床人工智能失败：与电子健康记录问答中变换器组合性限制一致的实证证据

Basu, Sanjay

Abstract

Aggregate accuracy benchmarks conceal a systematic structure in how large language models fail at electronic health record (EHR) question answering: questions requiring more inferential steps produce disproportionately more errors. Motivated by theoretical results on transformer compositionality limits, we introduce a pre-specified hop-count taxonomy -- the number of distinct reasoning steps required to answer a clinical question from an EHR -- as a principled predictor of model failure. We annotate 313 clinician-generated MedAlign EHR question-answer pairs across four hop levels and evaluate 301 questions in a within-model ablation (claude-sonnet-4-6, zero-shot vs. extended thinking) and cross-architecture replications (gpt-4o and gpt-5.4-2026-03-05, zero-shot). All three models, spanning two providers and two OpenAI generations (GPT-4 and GPT-5), show monotone accuracy decline with hop count: Claude Sonnet zero-shot falls from 30.6% (hop=1) to 17.6% (hop=4) (Cochran-Armitage z=-2.30, p=0.011; OR per hop 0.72, 95% CI [0.56,0.92], p=0.008); GPT-4o replicates this (37.8% to 14.7%; OR 0.58 [0.45,0.75], p<0.001); and gpt-5.4-2026-03-05 confirms it (37.8% to 23.5%; OR 0.80 [0.66,0.98], p=0.027). A pre-specified context-sufficiency audit shows higher-hop questions are not differentially disadvantaged by EHR truncation (answerability 93-95% at hops 2-4 vs. 79% at hop=1), so the decline reflects compositional reasoning difficulty. Extended thinking did not significantly flatten the accuracy-depth curve across three reasoning conditions, and thinking-token usage scaled with hop count (r=0.31, p<0.0001), consistent with the predicted O(k) computational requirement. Hop count is thus a theory-motivated, cross-architecture predictor of large-language-model error on EHR question answering, with direct implications for deployment risk stratification of clinical AI.

Chinese Translation

聚合准确性基准掩盖了大型语言模型在电子健康记录（EHR）问答中失败的系统性结构：需要更多推理步骤的问题产生的错误比例过高。基于对变换器组合性限制的理论结果，我们引入了一种预先指定的跳数分类法——回答临床问题所需的不同推理步骤数量——作为模型失败的原则性预测因子。我们对313个临床医生生成的MedAlign EHR问答对进行了标注，涵盖四个跳数级别，并在模型内部消融实验（claude-sonnet-4-6，零样本与扩展思维）和跨架构复制实验（gpt-4o和gpt-5.4-2026-03-05，零样本）中评估了301个问题。所有三个模型，跨越两个提供者和两个OpenAI版本（GPT-4和GPT-5），都显示出随着跳数增加准确性单调下降：Claude Sonnet零样本从30.6%（跳数=1）下降到17.6%（跳数=4）（Cochran-Armitage z=-2.30，p=0.011；每跳的OR 0.72，95% CI [0.56,0.92]，p=0.008）；GPT-4o复制了这一结果（37.8%降至14.7%；OR 0.58 [0.45,0.75]，p<0.001）；而gpt-5.4-2026-03-05确认了这一点（37.8%降至23.5%；OR 0.80 [0.66,0.98]，p=0.027）。预先指定的上下文充分性审计显示，高跳数问题并未因EHR截断而受到不同程度的劣势（在跳数2-4时可回答率为93-95%，而在跳数=1时为79%），因此下降反映了组合推理的困难。扩展思维在三个推理条件下并未显著平坦化准确性-深度曲线，思维令牌的使用与跳数成正比（r=0.31，p<0.0001），与预测的O(k)计算需求一致。因此，跳数是一个基于理论的、跨架构的预测因子，用于大型语言模型在EHR问答中的错误，直接影响临床人工智能的部署风险分层。

View on arXiv Download PDF AI Translation

cs.CL / 130 / 2606.16897

Contrastive-Difference CKA Reveals Concept-Specific Structural Alignment Across Language Model Architectures

对比差异 CKA 揭示语言模型架构间特定概念的结构对齐

Gao, Xueping

Abstract

Do different LLM architectures encode high-level concepts in structurally compatible ways? We systematically characterize a geometric-functional universality dissociation: across multiple concept domains and architectural families, moderate geometric convergence coexists with near-perfect functional transfer. Using contrastive-difference CKA (CKA_Delta), a training-free diagnostic that computes kernel alignment on per-sample contrastive differences, we isolate concept-specific convergence from generic similarity -- achieving significant discrimination where standard CKA cannot. The dissociation replicates across all six concept domains we test (five with p <= 0.017 geometric discrimination and safety as a converging-functional trend, p = 0.08), including two non-instruction concepts (code-vs-NL, reasoning-vs-recall) validated without system prompts; a single 70B--70B pair provides an observational note that universality may strengthen with scale, requiring replication with additional >=70B models. We position CKA_Delta as a practical regime classifier and architectural outlier detector (Gemma: d = 1.08, AUC = 0.79) rather than an absolute transfer-accuracy predictor, providing a training-free diagnostic for cross-architecture concept monitoring.

Chinese Translation

不同的 LLM 架构是否以结构上兼容的方式编码高层次概念？我们系统性地描述了一种几何-功能的普遍性解离：在多个概念领域和架构家族中，中等几何收敛与近乎完美的功能转移共存。使用对比差异 CKA (CKA_Delta)，这是一种无训练的诊断方法，通过计算每个样本的对比差异上的核对齐，我们将特定概念的收敛与通用相似性分离开来——在标准 CKA 无法做到的地方实现了显著的区分。这种解离在我们测试的六个概念领域中得到了复制（五个领域的几何区分 p <= 0.017，安全性作为收敛-功能趋势，p = 0.08），包括两个非指令概念（代码与自然语言、推理与回忆），这些概念在没有系统提示的情况下得到了验证；一个 70B--70B 的配对提供了一个观察性注释，表明普遍性可能随着规模的扩大而增强，需要用额外的 >=70B 模型进行复制。我们将 CKA_Delta 定位为一种实用的领域分类器和架构异常检测器（Gemma: d = 1.08, AUC = 0.79），而不是绝对的转移准确性预测器，为跨架构概念监测提供了一种无训练的诊断方法。

View on arXiv Download PDF AI Translation

cs.CL / 131 / 2606.16905

Speaking the Language of Science: Toward a General-Purpose Generative Foundation Model for the Natural Sciences

科学语言的表达：面向自然科学的通用生成基础模型

Li, Mingyang, Liu, Yurou, Ye, Jieping, Su, Bing, Wen, Ji-Rong, Wang, Zheng

Abstract

In this report, we present LOGOS (Language Of Generative Objects in Science), a scientific generative language model that unifies heterogeneous tasks across the natural sciences within a single autoregressive framework based on a shared scientific grammar. It encodes diverse scientific objects and their spatial interactions as token sequences over a common vocabulary. By representing spatial contact and constraint patterns as discrete tokens, the model captures complex structural interactions in a purely sequential manner, without relying on explicit coordinates or geometric neural networks. This unified representation enables a wide range of downstream tasks to be formulated consistently as next-token prediction in the same grammar space, creating strong alignment between continued multi-domain pre-training and downstream objectives. Across diverse tasks, LOGOS consistently matches or outperforms domain-specific baselines, providing preliminary evidence for the feasibility of "one model fits all" in the natural sciences. We train LOGOS models at different scales (1B, 3B, and 8B parameters) and find a consistent positive correlation between model size and performance. This suggests that the future of AI for Science (AI4S) may not lie in building an independent technical stack that is separated from large language models (LLMs). Instead, it may depend on deeply aligning scientific foundation models with LLMs through shared architectures, shared training paradigms, and shared inference infrastructure, so that LLMs can truly become a new entry point for AI4S. We release the model weights and associated resources to facilitate further research.

Chinese Translation

在本报告中，我们提出了LOGOS（科学中的生成对象语言），这是一个科学生成语言模型，旨在通过基于共享科学语法的单一自回归框架统一自然科学中的异构任务。它将多样的科学对象及其空间交互编码为一个共同词汇上的标记序列。通过将空间接触和约束模式表示为离散标记，该模型以纯粹的顺序方式捕捉复杂的结构交互，而不依赖于显式坐标或几何神经网络。这种统一表示使得广泛的下游任务能够在同一语法空间中一致地被表述为下一个标记预测，从而在持续的多领域预训练与下游目标之间创造了强大的对齐。在多样化任务中，LOGOS始终与特定领域的基线相匹配或超越，提供了“一个模型适用于所有”在自然科学中可行性的初步证据。我们训练了不同规模的LOGOS模型（1B、3B和8B参数），并发现模型规模与性能之间存在一致的正相关关系。这表明，科学人工智能（AI4S）的未来可能不在于构建一个与大型语言模型（LLMs）相分离的独立技术栈，而在于通过共享架构、共享训练范式和共享推理基础设施，深度对齐科学基础模型与LLMs，从而使LLMs真正成为AI4S的新切入点。我们发布了模型权重和相关资源，以促进进一步的研究。

View on arXiv Download PDF AI Translation

cs.CL / 132 / 2606.16908

LESS Is More: Mutual-Stability Sampling for Diffusion Language Models

少即是多：扩散语言模型的互稳定采样

Mohamed, Amr, Shang, Guokan, Vazirgiannis, Michalis

Abstract

Diffusion large language models (dLLMs) offer a promising alternative to autoregressive decoding by iteratively refining masked sequences, enabling parallel token updates and bidirectional conditioning. Their practical efficiency, however, is limited by sampling procedures that execute a fixed number of reverse denoising steps selected before decoding, spending computation on already-stable positions and sometimes committing unstable ones too early. We present \textsc{LESS}, a training-free, model-agnostic adaptive sampler that treats token commitment as an online stopping problem. \textsc{LESS} implements mutual-stability sampling through a joint stability rule that makes a masked position eligible for unmasking only when its top-1 prediction has high confidence, its top-1 token persists across recent reverse steps, and its predictive distribution is stable under top-$K$ inter-step Jensen--Shannon divergence. We evaluate \textsc{LESS} on Dream-7B, LLaDA-8B, and LLaDA-1.5-8B, covering full-sequence diffusion and semi-autoregressive blockwise sampling regimes, across seven benchmarks spanning general knowledge, math, and code. \textsc{LESS} improves average accuracy over strong training-free adaptive samplers while using $72.1\%$ fewer reverse steps than fixed-budget decoding. Since each reverse step requires a Transformer forward pass, these step-count reductions translate into fewer forward evaluations, lower measured wall-clock latency, and lower estimated inference compute.

Chinese Translation

扩散大语言模型（dLLMs）通过迭代优化被遮蔽的序列，提供了一种有前景的自回归解码替代方案，能够实现并行的标记更新和双向条件。然而，它们的实际效率受到采样过程的限制，该过程在解码之前执行固定数量的反向去噪步骤，导致计算资源浪费在已经稳定的位置上，并且有时过早地承诺不稳定的位置。我们提出了 extsc{LESS}，一种无训练、模型无关的自适应采样器，将标记承诺视为一个在线停止问题。 extsc{LESS} 通过一个联合稳定性规则实现互稳定采样，使得只有在其顶级预测具有高置信度、其顶级标记在最近的反向步骤中持续存在，并且其预测分布在顶级-$K$ 交叉步骤的詹森-香农散度下稳定时，遮蔽位置才有资格被解遮蔽。我们在 Dream-7B、LLaDA-8B 和 LLaDA-1.5-8B 上评估 extsc{LESS}，涵盖全序列扩散和半自回归块采样模式，涉及七个基准，涵盖一般知识、数学和代码。 extsc{LESS} 在使用比固定预算解码少 $72.1\%$ 的反向步骤的同时，提高了强大的无训练自适应采样器的平均准确性。由于每个反向步骤都需要一次 Transformer 前向传递，这些步骤数量的减少转化为更少的前向评估、更低的测量墙钟延迟以及更低的估计推理计算。

View on arXiv Download PDF AI Translation

cs.CL / 133 / 2606.16910

IMPACTeen: Intentions, Manipulation, Persuasion, Annotations, and Consequences in Teen Communication Dataset

IMPACTeen：青少年沟通数据集中意图、操控、说服、注释及其后果

Szczęsny, Aleksander, Mieleszczenko-Kowszewicz, Wiktoria, Markiewicz, Maciej, Bajcar, Beata, Adamczyk, Tomasz, Babiak, Jolanta, Chodak, Grzegorz, Kazienko, Przemysław

Abstract

IMPACTeen is a dataset of textual social influence scenarios spanning interpersonal, media-based, and digital settings in an adolescent context. It contains 1,021 texts, 5,100 individual annotation records, and gold labels for social influence techniques, with each text annotated from five distinct perspectives: teenagers, parents, psychologists, communication experts, and teachers. The resource was constructed through constrained LLM generation, followed by a two-step human editing and validation phase aimed at ensuring youth-context realism. A multi-dimensional annotation covered influence presence, techniques, intentions, consequences, resistance, reactions, and annotation confidence. The dataset supports research on social influence detection, annotator disagreement, cross-lingual modeling, and the training and evaluation of language models. The dataset was created in Polish and is accompanied by a corresponding English version.

Chinese Translation

IMPACTeen 是一个涵盖青少年背景下人际、媒体和数字环境的文本社会影响场景的数据集。该数据集包含 1,021 条文本、5,100 条个体注释记录，以及针对社会影响技术的金标准标签，每条文本从五个不同的视角进行注释：青少年、父母、心理学家、传播专家和教师。该资源通过受限的 LLM 生成构建，随后经过两步人类编辑和验证阶段，以确保青少年背景的真实性。多维度的注释涵盖了影响的存在、技术、意图、后果、抵抗、反应和注释信心。该数据集支持社会影响检测、注释者分歧、跨语言建模以及语言模型的训练和评估。该数据集以波兰语创建，并附有相应的英文版本。

View on arXiv Download PDF AI Translation

cs.CL / 134 / 2606.16934

Exploring Extrinsic and Intrinsic Properties for Effective Reasoning with Code Interpreter

探索有效代码解释器推理的外在和内在属性

Payoungkhamdee, Patomporn, Laosaengpha, Napat, Wonglertsakul, Jenta, Taveekitworachai, Pittawat, Tuchinda, Pume, Poobanchuen, Panjapong, Chuangsuwanich, Ekapol, Udomcharoenchaikit, Can, Cahyawijaya, Samuel, Limkonchotiwat, Peerat, Nutanong, Sarana

Abstract

Reasoning with a Code Interpreter (CI) has emerged as an effective paradigm for enhancing the reasoning capabilities of large language models (LLMs) through executable computation and iterative verification. Despite its growing adoption, the behavioral properties underlying effective code reasoning remain largely underexplored. In this work, we investigate code reasoning from two distinct perspectives inspired by prior studies of natural language reasoning: extrinsic properties, represented by crucial tokens, and intrinsic properties, represented by code-specific cognitive behaviors. Across multiple LLMs, we find that stronger CI reasoning models consistently exhibit a higher prevalence of crucial tokens and cognitive behaviors, particularly verification, backtracking, and backward chaining. Building on these observations, we examine how these properties can be leveraged during both inference and training. At inference time, appending code-specific crucial tokens improves performance on several reasoning capabilities, including mathematical, ordering, and optimization, while yielding limited benefits elsewhere. At training time, augmenting a state-of-the-art framework with code-specific cognitive behaviors improves supervised fine-tuning and reinforcement learning performance in two of three evaluated models. Further analysis shows that these behaviors reduce overthinking in incorrect responses and improve token efficiency, while also revealing factors that limit gains in a certain model. Our findings provide the first systematic characterization of effective reasoning with CI and demonstrate both the potential and limitations of leveraging key properties to improve CI-based reasoning.

Chinese Translation

使用代码解释器（Code Interpreter, CI）进行推理已成为一种有效的范式，通过可执行计算和迭代验证增强大型语言模型（Large Language Models, LLMs）的推理能力。尽管其应用日益广泛，但有效代码推理背后的行为属性仍然在很大程度上未被深入探讨。在本研究中，我们从两个不同的视角研究代码推理，这些视角受到自然语言推理先前研究的启发：外在属性，由关键标记表示，以及内在属性，由特定于代码的认知行为表示。在多个LLM中，我们发现更强的CI推理模型始终表现出更高的关键标记和认知行为的出现率，特别是验证、回溯和逆向链推理。基于这些观察，我们考察了如何在推理和训练过程中利用这些属性。在推理时，附加特定于代码的关键标记提高了多个推理能力的表现，包括数学、排序和优化，而在其他方面的收益有限。在训练时，将最先进的框架与特定于代码的认知行为相结合，提高了三种评估模型中两种的监督微调和强化学习性能。进一步分析表明，这些行为减少了错误响应中的过度思考，并提高了标记效率，同时揭示了限制某一模型收益的因素。我们的研究首次系统性地表征了使用CI进行有效推理，并展示了利用关键属性改善基于CI的推理的潜力和局限性。

View on arXiv Download PDF AI Translation

cs.CL / 135 / 2606.17016

TokenPilot: Cache-Efficient Context Management for LLM Agents

TokenPilot：用于大语言模型代理的缓存高效上下文管理

Xu, Buqiang, Xue, Zirui, Chen, Dianmou, Fu, Chenyang, Wu, Chiyu, Huang, Caiying, Jiang, Chen, Fang, Jizhan, Deng, Xinle, Chen, Yijun, Yao, Yunzhi, Wang, Xuehai, Shang, Jin, Yu, Gong, Zhang, Ningyu

Abstract

As LLM agents are deployed in long-horizon sessions, context accumulation drives up inference costs. Existing approaches utilize text pruning or dynamic memory eviction to minimize token footprints; however, their unconstrained sequence mutations alter layouts, introducing prefix mismatches and cache invalidation. This reveals a critical trade-off between text sparsity and prompt cache continuity. To address this, we present TokenPilot, a dual-granularity context management framework. Globally, Ingestion-Aware Compaction acts as a framework harness to stabilize prompt prefixes and eliminate open-world environmental noise at the ingestion gate. Locally, Lifecycle-Aware Eviction monitors the ongoing residual utility of context segments, enforcing a conservative batch-turn schedule to offload content segments only when task relevance expires. Experiments on PinchBench and Claw-Eval under both isolated and continuous modes demonstrate that TokenPilot reduces costs by 61% and 56% in isolated mode, and 61% and 87% in continuous mode, while maintaining competitive performance compared to prior systems. TokenPilot has been integrated into LightMem2 at https://github.com/zjunlp/LightMem2.

Chinese Translation

随着大语言模型（LLM）代理在长时间会话中的部署，上下文的积累导致推理成本上升。现有的方法利用文本剪枝或动态内存驱逐来最小化令牌占用；然而，它们不受限制的序列变更会改变布局，导致前缀不匹配和缓存失效。这揭示了文本稀疏性与提示缓存连续性之间的关键权衡。为了解决这个问题，我们提出了TokenPilot，一个双粒度的上下文管理框架。在全局层面，感知摄取的压缩（Ingestion-Aware Compaction）作为框架支撑，稳定提示前缀并在摄取门口消除开放世界环境噪声。在局部层面，生命周期感知驱逐（Lifecycle-Aware Eviction）监控上下文段的持续剩余效用，执行保守的批量轮转调度，仅在任务相关性过期时卸载内容段。在PinchBench和Claw-Eval的实验中，无论是在隔离模式还是连续模式下，TokenPilot分别在隔离模式下减少了61%和56%的成本，在连续模式下减少了61%和87%的成本，同时与之前的系统相比保持了竞争力的性能。TokenPilot已集成到LightMem2中，网址为https://github.com/zjunlp/LightMem2。

View on arXiv Download PDF AI Translation

cs.CL / 136 / 2606.17029

DEEPRUBRIC: Evidence-Tree Rubric Supervision for Efficient Reinforcement Learning of Deep Research Agents

DEEPRUBRIC：基于证据树的评分标准监督以提高深度研究代理的强化学习效率

Zhu, Minghang, Wei, Chuyang, Xu, Junhao, Cheng, Yilin, Chen, Zhumin, He, Jiyan

Abstract

Deep research agents synthesize long-form reports by searching and reasoning over retrieved evidence. Reinforcement learning with rubric-based rewards improves these agents by optimizing them against checkable criteria that translate report quality into reward signals, but its efficiency depends on whether those criteria reliably capture the task scope and evidence needs. Most existing studies ask an LLM to generate rubrics for a given query, but when the model fails to infer the underlying information needs, the generated rubrics may be incomplete and reduce RL efficiency. To obtain more reliable query--rubric supervision, we introduce DeepRubric, a data construction framework that reverses this process: instead of inferring evaluation criteria for a given query, it first determines what an evidence-backed report should be evaluated on and then synthesizes aligned query--rubric pairs from those evaluation targets. Starting from a sampled seed topic, DeepRubric builds an evidence tree by recursively expanding evidence-backed sub-questions, whose leaves serve as atomic and verifiable evaluation targets. It then uses the evidence tree to synthesize the training query and rubrics, ensuring that the reward evaluates exactly the information requested by the query. Using DeepRubric, we construct 9K query--rubric supervision examples and train DeepRubric-8B with rubric-based GRPO, achieving comparable performance to prior open state-of-the-art deep research models across three benchmarks with roughly 13x fewer RL GPU-hours.

Chinese Translation

深度研究代理通过搜索和推理检索到的证据来合成长篇报告。基于评分标准的奖励的强化学习通过将报告质量转化为奖励信号的可检查标准来优化这些代理，但其效率依赖于这些标准是否可靠地捕捉任务范围和证据需求。现有大多数研究要求大型语言模型（LLM）为给定查询生成评分标准，但当模型未能推断出潜在的信息需求时，生成的评分标准可能不完整，从而降低强化学习的效率。为了获得更可靠的查询-评分标准监督，我们引入了DeepRubric，这是一种数据构建框架，逆转了这一过程：它首先确定一个基于证据的报告应评估的内容，而不是为给定查询推断评估标准，然后从这些评估目标中合成对齐的查询-评分标准对。从一个采样的种子主题开始，DeepRubric通过递归扩展基于证据的子问题来构建证据树，其叶子作为原子和可验证的评估目标。然后，它利用证据树合成训练查询和评分标准，确保奖励准确评估查询所请求的信息。使用DeepRubric，我们构建了9000个查询-评分标准监督示例，并使用基于评分标准的GRPO训练DeepRubric-8B，在三个基准测试中实现了与之前的开放前沿深度研究模型相当的性能，同时减少了大约13倍的强化学习GPU小时数。

View on arXiv Download PDF AI Translation

cs.CL / 137 / 2606.17034

KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing

KVEraser：学习引导KV缓存以实现高效的局部上下文擦除

Li, Mufei, Liu, Shikun, Fu, Dongqi, Wang, Haoyu, Xia, Yinglong, Li, Hong, Yan, Hong, Li, Pan

Abstract

Post-hoc context erasing over the KV cache is challenging because a local edit has a global consequence: once a span has been processed, its influence propagates into the cached states of all subsequent tokens. This issue arises naturally in long-context LLM applications, where stale retrieved facts, incorrect tool observations, retracted user preferences, or harmful prompt injections may be identified only after prefill. Exact erasing must then recompute all tokens after the deleted span, making its computational cost depend on suffix length rather than erased-span length. We introduce KVEraser, a learned KV-cache editing method for efficient localized context erasing. Given a processed context and a span to remove, KVEraser replaces only the KV states of the erased interval with learned steering states while reusing the remaining cache unchanged. To learn a transferable erasing mechanism, we build a two-stage training pipeline: generic span-neighbor pre-training teaches the eraser to suppress the influence of the erased span, while task-specific fine-tuning adapts this capability to downstream scenarios. Experiments show that KVEraser nearly matches full recomputation in post-erasure performance on in-domain tasks across 1K--32K context lengths, while its latency increases by only 24% compared with a 17.6x increase for full recomputation. KVEraser also generalizes to unseen long-document QA tasks with harmful factual distractors, achieving the best performance among approximate baselines with a 3--4x speedup over full recomputation.

Chinese Translation

在KV缓存上进行事后上下文擦除具有挑战性，因为局部编辑会产生全局后果：一旦某个跨度被处理，其影响会传播到所有后续标记的缓存状态中。这一问题在长上下文的LLM应用中自然出现，其中过时的检索事实、不正确的工具观察、撤回的用户偏好或有害的提示注入可能仅在预填充后被识别。精确擦除必须重新计算所有在被删除跨度之后的标记，使其计算成本依赖于后缀长度而非被擦除跨度的长度。我们提出了KVEraser，这是一种学习的KV缓存编辑方法，用于高效的局部上下文擦除。在给定处理过的上下文和要移除的跨度的情况下，KVEraser仅用学习到的引导状态替换被擦除区间的KV状态，同时保持其余缓存不变。为了学习可迁移的擦除机制，我们构建了一个两阶段的训练管道：通用的跨度邻域预训练教会擦除器抑制被擦除跨度的影响，而特定任务的微调则将这一能力适应于下游场景。实验表明，KVEraser在1K到32K上下文长度的领域内任务中，几乎与完全重新计算的后擦除性能相匹配，而其延迟仅增加了24%，相比之下，完全重新计算的延迟增加了17.6倍。KVEraser还能够推广到未见过的长文档问答任务，处理有害的事实干扰者，在近似基线中实现最佳性能，并比完全重新计算快3到4倍。

View on arXiv Download PDF AI Translation

cs.CL / 138 / 2606.17041

Benchmarking LLM Agents on Meta-Analysis Articles from Nature Portfolio

对来自自然出版集团的荟萃分析文章进行大语言模型代理的基准测试

Xie, Anzhe, Su, Weihang, Zhou, Yujia, Liu, Yiqun, Ai, Qingyao

Abstract

Meta-analysis is a demanding form of evidence synthesis that combines literature retrieval, PI/ECO-guided study selection, and statistical aggregation. Its structured, verifiable workflow makes it an ideal substrate for evaluating systematic scientific reasoning, yet existing benchmarks lack ground truth across the full retrieval-screening-synthesis pipeline. We introduce MetaSyn, a dataset of 442 expert-curated meta-analyses from Nature Portfolio journals. Each entry pairs a research question with PI/ECO criteria, a retrieval corpus of 140k PubMed articles, verified positive studies, hard negatives that are topically similar but PI/ECO-ineligible, and complete search strategies and date bounds. Benchmarking twelve pipeline configurations (nine RAG variants and a protocol-driven agent) reveals a critical screening bottleneck: despite a retrieval ceiling of 90.9% recall at K=200, no system recovers more than 52.7% of ground-truth included literature. Current LLMs fail to reliably separate eligible studies from PI/ECO-failing distractors in pools of comparable topical relevance. Stage-attributed metrics capture where systems succeed and fail; a single end-to-end score does not.

Chinese Translation

荟萃分析是一种要求严格的证据综合形式，结合了文献检索、PI/ECO指导的研究选择和统计聚合。其结构化、可验证的工作流程使其成为评估系统科学推理的理想基础，但现有基准在整个检索-筛选-综合流程中缺乏真实的基准数据。我们引入了MetaSyn，一个包含442篇来自自然出版集团期刊的专家策划荟萃分析的数据集。每个条目将研究问题与PI/ECO标准配对，并提供140,000篇PubMed文章的检索语料库、经过验证的阳性研究、在主题上相似但不符合PI/ECO标准的难负样本，以及完整的搜索策略和日期范围。对十二种管道配置（九种RAG变体和一种基于协议的代理）的基准测试揭示了一个关键的筛选瓶颈：尽管在K=200时检索的召回率达到了90.9%的上限，但没有任何系统能够恢复超过52.7%的真实包含文献。目前的大语言模型在相似主题的池中无法可靠地区分符合条件的研究与PI/ECO不合格的干扰项。阶段归因指标捕捉了系统成功和失败的地方；单一的端到端得分则无法做到这一点。

View on arXiv Download PDF AI Translation

cs.CL / 139 / 2606.17053

Context-Aware RL for Agentic and Multimodal LLMs

面向上下文的强化学习用于自主和多模态大型语言模型

Xu, Peiyang, Li, Bangzheng, Liu, Sijia, Narasimhan, Karthik R., Viswanath, Pramod, Mittal, Prateek, Fu, Xingyu

Abstract

Large language models (LLMs) often fail when answering requires identifying a small but decisive piece of evidence within a long or complex context, such as a single line in a tool trace or a subtle detail in an image. We propose ContextRL, a context-aware reinforcement learning (RL) method that improves long-horizon reasoning and multimodal performance through an \emph{indirect} auxiliary objective. Instead of supervising only the final answer, ContextRL presents the model with a query, an answer, and two highly similar contexts, and rewards it for selecting the context that supports the query--answer pair, thereby encouraging fine-grained grounding. We construct contrastive context data in two domains: for coding agents, trajectories serve as contexts, yielding 1k pairs built via condition filtering; for multimodal reasoning, images serve as contexts, yielding 7K pairs built via generative editing and similarity search. ContextRL achieves average gains of +2.2% over standard GRPO on 5 long-horizon benchmarks, and +1.8% across 12 diverse visual question answering benchmarks. To disentangle the effect of the proposed objective from that of additional data, we compare against data-augmentation baselines that repurpose the same contrastive contexts as standard query--context--answer examples. These baselines provide little to no improvement, showing that the gains arise from the proposed context-selection objective rather than from the contrastive data alone.

Chinese Translation

大型语言模型（LLMs）在回答需要在长或复杂上下文中识别一小段但决定性证据时，往往表现不佳，例如工具追踪中的一行或图像中的细微细节。我们提出了ContextRL，这是一种面向上下文的强化学习（RL）方法，通过一种 extit{间接}的辅助目标来改善长时间推理和多模态性能。ContextRL不仅仅监督最终答案，而是向模型呈现一个查询、一个答案和两个高度相似的上下文，并奖励模型选择支持该查询-答案对的上下文，从而鼓励细致的基础支持。我们在两个领域构建了对比上下文数据：对于编码代理，轨迹作为上下文，生成了通过条件过滤构建的1000对；对于多模态推理，图像作为上下文，生成了通过生成编辑和相似性搜索构建的7000对。ContextRL在5个长时间基准测试中平均获得了+2.2%的提升，在12个多样的视觉问答基准测试中获得了+1.8%的提升。为了将所提目标的效果与额外数据的效果区分开，我们与数据增强基线进行了比较，这些基线将相同的对比上下文重新用于标准的查询-上下文-答案示例。这些基线几乎没有提供改进，表明提升源于所提的上下文选择目标，而非仅仅来自对比数据。

View on arXiv Download PDF AI Translation

cs.CL / 140 / 2606.17056

The Value Axis: Language Models Encode Whether They're on the Right Track

价值轴：语言模型编码其是否走在正确轨道上

Jiang, Nick, Kauvar, Isaac, Lindsey, Jack

Abstract

We investigate whether language models internally track the value of their current trajectory, defined as the likelihood that their ongoing strategy will achieve their goals. Using synthetic, in-context reinforcement learning data, we construct a "value" axis for Qwen3-8B. We find that activations along this axis distinguish between high vs. low verbalized confidence, rollouts without and with backtracking, and correct vs. corrupted code. Steering towards high value causally suppresses self-correction and reduces explanatory verbosity, while steering towards low value induces backtracking and exploration. We demonstrate that direct preference optimization (DPO) can increase the internal value of rewarded behaviors (e.g. use a certain word), causing the model to act more confidently after exhibiting them. Finally, we apply the value axis to study in-the-wild settings. For example, we find that Qwen assigns low value to politically sensitive chat queries after post-training and that supervised fine-tuning increases internal confidence within the training domain. Our results suggest that language models linearly encode an estimate of expected goal success that modulates their confidence in pursuing a direction.

Chinese Translation

我们研究语言模型是否在内部跟踪其当前轨迹的价值，这被定义为其正在进行的策略实现目标的可能性。通过使用合成的上下文强化学习数据，我们为 Qwen3-8B 构建了一个“价值”轴。我们发现，该轴上的激活能够区分高与低的语言化信心、没有回溯和有回溯的展开，以及正确与损坏的代码。朝向高价值的引导因果性地抑制自我修正并减少解释的冗长，而朝向低价值则会诱发回溯和探索。我们证明了直接偏好优化（DPO）可以提高被奖励行为的内部价值（例如，使用某个特定词汇），使得模型在展现这些行为后表现得更加自信。最后，我们将价值轴应用于研究实际环境中的设置。例如，我们发现 Qwen 在后训练后对政治敏感的聊天查询赋予低价值，并且监督微调在训练领域内提高了内部信心。我们的结果表明，语言模型线性编码了对预期目标成功的估计，这调节了它们在追求某一方向时的信心。

View on arXiv Download PDF AI Translation

arXiv Papers

Bayesian Optimization for Learning Nonlinear MPC in Autonomous Agent Navigation

Synthetic-to-Real Pipeline for Safe Landing Zone Detection

Deep Learning-Based Lunar Crater Terrain Relative Navigation

Computing Smooth Geodesics under Two-Sided Curvature Bounds with Applications to Robotics and Image Analysis

TacStyle: Personalizing Tactile Robot Policies using Structured Behavior Representations

VANDERER: Map-Free Exploration using Future-Aware and Visual-Curiosity-Guided Diffusion Policy

DynaHMRC: Decentralized Heterogeneous Multi-Robot Collaboration for Dynamic Tasks with Large Language Models

Multimodal Physiological Assessment of Contact-Rich Physical Human-Robot Interaction Under Varying Environmental Conditions

Inference-time Policy Steering via Vision and Touch

LV-Calib: LiDAR-Camera Extrinsic Calibration with Boundary-Response Modeling

Steering Autoregressive Vision-Language-Action Policies via Action Token Intervention

An Autonomous Subgram SMA-Based Swimmer

Exact, Efficient, and Safe Occlusion-Aware Planning Using AH-Polyhedrons

Design and Fabrication of a Spin Coater with In-Situ Optical Measurement for Soft Thin Films

DragMesh-2: Physically Plausible Dexterous Hand-Object Interaction with Articulated Objects

MimicIK: Real-Time Generative Inverse Kinematics from Teleoperation with FK Consistency

Task-Aware Environment Augmentation for Reliable Navigation via Shielded Conditional Diffusion

Seam-to-Graph Reconstruction for Garment Configuration Alignment

Rethinking Implicit Spatial Representation in Visuomotor Policy Learning

Co-Creating Buildable and Open Social Robot Study Companions with University Students

Driving, Fast or Slow? Neuro-Symbolic Guidance for Motion Prediction in Multi-Modal Ground Mobility

OSDAG: Online Scheduling for Efficient Multi-Robot Collaboration

Acting While Understanding: Asynchronous Semantic-Action Decoupling for Real-Time Vision-Language-Action Models

Covariance-Regulated Recursive Koopman Learning for Nonlinear Systems with Uncertain Time-Varying Dynamics

SimWeaver: Zero-Shot RGB Sim-to-Real for Deformable Manipulation

A Hybrid Model-Based and Model-Free Framework for Active Multi-View Viewpoint Optimization in Sonar Target Recognition

A Corridor-Scale CARLA-VISSIM Co-Simulation Framework for Multi-Intersection Urban Traffic

A Bilateral Teleoperation Framework for Dexterous Manipulation

Learning Context-Aware Neural ODE Dynamics for Adaptive Robotic Control

FARM: Find Anything using Relational Spatial Memory

FD-SLAM: Fast Dense Radar-Inertial SLAM with Frequency-Domain Loop Closure and Pose Graph Optimization

Understanding and Modeling Perceived Cognitive and Physical Strain Dynamics for Planning-Oriented Human-Robot Collaboration in Prefabricated Construction

Reinforcement Learning-Guided Retrieval with Soft Fusion for Robust Multimodal Imitation Learning under Missing Modalities

Transferring Contact, Not Just Motion: Compliant Grasping Across Dexterous Hands

Robots as Tokens: Unified Diffusion Transformer for Coordinated Multi-Robot Trajectory Generation

SAPS: Shared Autonomy for Policy Steering by Blending Teleoperation with a Pretrained VLA

Perfect Demo Makes Poor Teacher: Learning Robust Alignment from Critical Motion Segments

Pixels to Proofs: Probabilistically-Safe Latent World Model Control via Parallel Conformal Robust MPC

Retrieve, Don't Retrain: Extending Vision Language Action Models to New Tasks at Test Time

TO-SoFiT: Topology Optimization of Hydraulic Soft Fish Tail Design for programmable undulating locomotion

PO-PDDL: Learning Symbolic POMDPs from Visual Demonstrations for Robot Planning Under Uncertainty

Learning New Tasks via Reusable Skills: Skill-Compositional Experts for Embodied Continual Learning

Can Causal Models Enhance Robot Navigation? Online Causal Adaptation for Real-Robot Navigation

LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies

FlashNav: Ultra-Fast Policy Training for Robot Navigation within 20 Seconds

LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors

VL2Spike: Spike-driven Distillation from VLMs for Low-Power Visual Perception in Embodied AI

GeoTLM: Geometry-aware Tactile-Language Models for Contact Motion Orientation Reasoning of Dynamic Objects

Identification of a Physics-Based Electrical Power Consumption Model for the Unitree G1 Humanoid Arm

Energy-Efficient Arm Reaching for a Humanoid Robot via Deep Reinforcement Learning with Identified Power Models

ControlMap: Controllable High-Definition Map Generation for Traffic Scenario Simulation

Friction Characterization of a Cable-Driven Differential Actuation System for Lower-Limb Exoskeletons

$\lambda$-Reachability: Geometric-Horizon Safety Bellman Equations for Humanoid Safety

Leveraging Deep Learning for Object and Position Recognition of Load Carriers for Autonomous Logistics Vehicles

A Smart-Scheduled Hybrid (SSH) EKF-FGO State Estimation

A Deployment Case Study in Robotic Apparel Automation: Digital Twin Integration, Interoperability, and Workforce Enablement

Scaling Short-Term Memory of Visuomotor Policies for Long-Horizon Tasks

ATHENA: Accelerated Multi-Task Heterogeneous Influence Functions for Robot Data Curation

PolyMerge: Compressing 3D Gaussian Splats with Polytope Coverings for Provably Safe Resource-Constrained Navigation

TopoRetarget: Interaction-Preserving Retargeting for Dexterous Manipulation

Is Your Trajectory Displacement Safe in Long-tail?

ART-Glove: Articulated Tactile Glove for Contact-Grounded Dexterous Interaction Capture

SemGeoNav:A Safety-Guided Visual Navigation Approach with Semantic Reasoning and Geometric Planning

An Augmented Reality Brain-Robot Interface for Generalist Robot Arm Manipulation

V2P-Manip: Learning Dexterous Manipulation from Monocular Human Videos

Training and Evaluating Diffusion Policies with Long Context Lengths

RHO: Your Coding Agent is Secretly a Roboticist

HOLO-MPPI: Multi-Scenario Motion Planning via Hierarchical Policy Optimization

Robots that Collaborate: Sequential Asymmetric Imitation for Learning Coupled Robot Policies

HATS: A Human-Agent Teleoperation System for Multi-Arm Data Collection

APEX: Adaptive Policy Execution for Precise Manipulation

Agile Fall Recovery for Quadrotors with Bidirectional Thrust via Reinforcement Learning

ADAPT: Analytical Disturbance-Aware Policy Training for Humanoid Locomotion

Elastic ODYN: Differentiable Optimization for Infeasible Control and Learning in Robotics

Automated Digital Twin Construction for Highway Scenarios Using LiDAR Point Clouds and OpenStreetMap

Steering Generative Reinforcement Learning into Stable Robotic Controller

WaveSync: Constrained Wavefront Optimization for Synchronized Co-Speech Gestures in Humanoid Robots

Reinforcement Learning with Inner-loop Dynamics Estimator for Aerial Manipulation under Uncertainty

PATCH: Action-Chunk-Conditioned Latent Patch Innovation Monitoring for Robot Manipulation