arXiv Daily Digest

181

Papers

Pelican-VLA 0.5: Attending Before Acting Benefits Generalization

Pelican-VLA 0.5：在行动之前进行关注有助于泛化

Ding, Zeyuan, Liu, Wenhai, Xu, Yang, Hu, Jiayu, Chen, Yinda, Zhang, Yi, Dai, Yong, Tang, Jian, Ju, Xiaozhu

Abstract

In this report, we present Pelican-VLA 0.5, a unified VLA model that integrates vision-language understanding, future-frame generation, and action prediction within a single architecture. Pelican-VLA 0.5 achieves attention-level generalization: without object annotations, segmentation masks, attention supervision, or task-specific fine-tuning, its action pathway already focuses on the instruction-relevant object and contact region. This behavior persists across unseen scenes and unseen robot embodiments, and is substantially stronger than in other open-source VLA baselines. We verify that this ability originates from the learnable Reasoning Slots inserted between perception and action: by routing task-relevant visual information through a compact bottleneck, the slot interface induces manipulation-centric attention during pre-training and remains effective across different policy structures, including a MoT-style architecture.

Chinese Translation

在本报告中，我们介绍了Pelican-VLA 0.5，这是一种统一的视觉-语言理解（VLA）模型，将视觉-语言理解、未来帧生成和动作预测集成在单一架构中。Pelican-VLA 0.5 实现了注意力级别的泛化：在没有物体注释、分割掩码、注意力监督或特定任务微调的情况下，其动作路径已经专注于与指令相关的物体和接触区域。这种行为在未见过的场景和未见过的机器人体现中持续存在，并且显著强于其他开源 VLA 基线。我们验证了这种能力源自插入在感知与行动之间的可学习推理槽（Reasoning Slots）：通过将任务相关的视觉信息通过一个紧凑的瓶颈进行路由，该槽接口在预训练期间引导以操控为中心的注意力，并在不同的策略结构中保持有效，包括 MoT 风格架构。

View on arXiv Download PDF AI Translation

cs.RO / 2 / 2607.06678

NativeMEM: Native Memory Compression for Long-Horizon Robotic Manipulation

NativeMEM：用于长时间范围机器人操作的本地内存压缩

Wang, Ziye, Shi, Modi, Ni, Chaojun, Yang, Jiazhi, Li, Mengdi, Su, Zhizhong, Lin, Tianwei, Li, Hongyang

Abstract

How can pretrained Vision-Language-Action (VLA) models retain long-horizon visual histories with high-frequency updates without sacrificing efficiency? Existing approaches rely on external memory management, which restrains either the memory horizon or the reactiveness of pretrained policies. To this end, we present NativeMEM, a VLA policy that features long-term and real-time updated memory. At its core is an efficient memory encoding scheme, Native Memory Compression, which repurposes the VLA's own vision encoder to compress each historical frame from each camera view into a single token. Appended to the input sequence, these memory tokens enable the pretrained VLA to attend over long-term history with negligible latency overhead, requiring neither an external planner nor a freshly initialized memory module. To align the memory tokens with the pretrained policy, we first develop a generic memory tokenizer under the supervision of a frozen VLA on memory-demanding data, and then unfreeze the VLA for task-specific fine-tuning. NativeMEM consistently outperforms prior methods, boosting success rates from 32.4% to 84.0% in simulation and up to 98.7% on real robots, while maintaining low inference latency and GPU memory usage. Notably, NativeMEM exhibits high data efficiency by achieving competitive results with prior arts using only 20% of the training data.

Chinese Translation

如何让预训练的视觉-语言-动作（VLA）模型在不牺牲效率的情况下，保留长时间范围的视觉历史并进行高频更新？现有的方法依赖于外部内存管理，这限制了内存的时间范围或预训练策略的反应能力。为此，我们提出了NativeMEM，一种具有长期和实时更新内存的VLA策略。其核心是一个高效的内存编码方案——本地内存压缩（Native Memory Compression），该方案将VLA自身的视觉编码器重新利用，以将每个相机视角的历史帧压缩为一个单一的标记。附加到输入序列中的这些内存标记使得预训练的VLA能够以可忽略的延迟开销关注长期历史，而无需外部规划器或新初始化的内存模块。为了将内存标记与预训练策略对齐，我们首先在内存需求较高的数据上，在冻结的VLA的监督下开发了一个通用的内存标记器，然后解冻VLA进行任务特定的微调。NativeMEM在性能上始终优于先前的方法，在仿真中成功率从32.4%提升至84.0%，在真实机器人上则高达98.7%，同时保持低推理延迟和GPU内存使用。值得注意的是，NativeMEM通过仅使用20%的训练数据就能取得与先前研究相媲美的结果，展现出高数据效率。

View on arXiv Download PDF AI Translation

cs.RO / 3 / 2607.06699

RoboSnap: One-Shot Real-to-Sim Scene Generation for Generalizable Robot Learning and Evaluation

RoboSnap：用于可推广机器人学习和评估的一次性真实到仿真场景生成

Zhang, Shujie, Yi, Jingkun, Zhong, Weipeng, Zhou, Zirui, Zhu, Yangkun, Wang, Hanqing, Xu, Xudong, Zhang, Weinan, Shen, Chunhua

Abstract

Recovering real-world scenes as interactive simulation environments can enable generalizable robot learning and reproducible policy evaluation. However, constructing scenes that are both physically stable and visually faithful remains slow and expensive. In this work, we present RoboSnap, a real-to-sim framework that turns a single RGB image into a simulation-ready scene. The key idea is a layered design that separates the physics-critical interaction area from the surrounding visual context: collision-aware foreground assets are refined for stable robot interaction, while a 3D Gaussian splatting visual layer preserves faithful background appearance under novel views. Experiments on DROID scenes and real-robot tasks show that RoboSnap achieves reliable trajectory replay in the recovered scenes, supports task-specific synthetic data generation for policy training, and yields meaningful sim-real correlation for policy evaluation. To further support real-to-sim research, we introduce DROID-Sim, a real-to-sim companion dataset constructed from 564 real-world scenes in DROID. Extensive experiments suggest that the value of real-to-sim methods lies not only in high-fidelity visual reconstruction, but in turning real environments into reusable infrastructure for robot learning and evaluation.

Chinese Translation

将现实世界场景恢复为交互式仿真环境可以促进可推广的机器人学习和可重复的策略评估。然而，构建既物理稳定又视觉真实的场景仍然是一个缓慢且昂贵的过程。在本研究中，我们提出了RoboSnap，一个将单张RGB图像转换为仿真准备场景的真实到仿真框架。其关键思想是采用分层设计，将物理关键的交互区域与周围的视觉背景分离：碰撞感知的前景资产经过精细化处理，以实现稳定的机器人交互，而3D高斯点云视觉层则在新视角下保持真实的背景外观。在DROID场景和真实机器人任务上的实验表明，RoboSnap在恢复的场景中实现了可靠的轨迹重放，支持针对策略训练的任务特定合成数据生成，并为策略评估提供了有意义的仿真与现实的相关性。为了进一步支持真实到仿真的研究，我们引入了DROID-Sim，这是一个由DROID中的564个真实场景构建的真实到仿真的伴随数据集。大量实验表明，真实到仿真方法的价值不仅在于高保真的视觉重建，还在于将真实环境转化为可重用的机器人学习和评估基础设施。

View on arXiv Download PDF AI Translation

cs.RO / 4 / 2607.06700

CILC: Cryptographically-secure Inter-agent Loop Closure Candidate Detection for Multi-Agent Collaborative SLAM

CILC：用于多智能体协作SLAM的密码学安全的智能体间回环闭合候选检测

Fishberg, Andrew, Jia, Yixuan, How, Jonathan P.

Abstract

Multi-agent Simultaneous Localization and Mapping (SLAM) and collaborative SLAM (CSLAM) require robots to continuously exchange global descriptors (GDs) to detect inter-agent loop closures (ILCs). While encrypted radios protect this traffic from external eavesdroppers, they offer no protection against a compromised swarm member. We show this threat is concrete by demonstrating how a corrupted agent can reconstruct approximations of an honest agent's imagery and trajectory from its public GD broadcasts. To address this, we propose CILC (Cryptographically-secure Inter-agent Loop Closure candidate detection), a first-of-its-kind system leveraging Secure Multi-Party Computation (SMPC) to detect ILC candidates without exchanging GDs in the clear. Rather than securing the entire CSLAM pipeline, we apply SMPC only to ILC candidate detection (i.e., GD similarity comparison), a privacy-sensitive yet computationally lightweight step, yielding an advantageous privacy-to-overhead trade-off. We validate in both simulation and hardware experiments that CILC remains real-time and communication-feasible across multimodal GDs (visual and LiDAR), while mitigating information leakage to a compromised swarm agent.

Chinese Translation

多智能体同时定位与地图构建（SLAM）和协作SLAM（CSLAM）要求机器人持续交换全局描述符（GDs）以检测智能体间回环闭合（ILCs）。尽管加密无线电保护了这一通信免受外部窃听者的攻击，但对于被攻陷的群体成员却没有保护。我们通过展示一个被腐败的智能体如何从其公开的GD广播中重构诚实智能体的图像和轨迹的近似值，证明了这一威胁的具体性。为了解决这个问题，我们提出了CILC（密码学安全的智能体间回环闭合候选检测），这是首个利用安全多方计算（SMPC）在不明文交换GDs的情况下检测ILC候选的系统。我们并不对整个CSLAM流程进行安全保护，而是仅将SMPC应用于ILC候选检测（即GD相似性比较），这是一个隐私敏感但计算开销较小的步骤，从而实现了隐私与开销之间的有利权衡。我们在模拟和硬件实验中验证了CILC在多模态GDs（视觉和激光雷达）下仍能保持实时性和通信可行性，同时减少了对被攻陷的群体智能体的信息泄露。

View on arXiv Download PDF AI Translation

cs.RO / 5 / 2607.06706

Vision Language Action (VLA) Models for Unmanned Aerial Robotics and Bimanual Manipulation: A Review

无人机机器人和双手操作的视觉语言行动（VLA）模型：综述

Sa, Inkyu, Park, Chanoh, Lee, Hea-Min, Noh, Donghee, Ahn, Ho Seok

Abstract

Vision Language Action (VLA) models unify visual perception, natural-language understanding, and action generation within a single foundation model, allowing a robot to follow instructions such as fold the towel or fly to the red building directly from camera images. Because VLAs inherit world knowledge from internet-scale pre-training, they have become the dominant framework for learning-based manipulation, with bimanual coordination serving as the most demanding testbed: two arms with 7 degrees of freedom each must move in concert to fold, assemble, and reorient objects. Unmanned aerial robotics faces a structurally similar challenge: a drone must coordinate thrust, attitude, and increasingly gripper commands from visual observations under strict latency and payload constraints. This review covers 183 contributions spanning 2017-2026 and organized along seven dimensions: VLA architectures, training recipes, action representations, bimanual coordination (2022-2026), unmanned aerial vehicle (UAV) navigation and control (2017-2026), language grounding, and cross-cutting concerns including memory and world models. We show that the coordination strategies, training recipes, and action representations developed for bimanual VLAs transfer to unmanned aerial systems and identify fourteen research directions across both domains.

Chinese Translation

视觉语言行动（VLA）模型将视觉感知、自然语言理解和行动生成统一于一个基础模型中，使得机器人能够直接从摄像头图像中遵循指令，如折叠毛巾或飞往红色建筑。由于VLA从互联网规模的预训练中继承了世界知识，它们已成为基于学习的操作的主导框架，其中双手协调作为最具挑战性的测试平台：两个各具有7个自由度的手臂必须协调运动以折叠、组装和重新定向物体。无人机机器人面临着结构上类似的挑战：无人机必须在严格的延迟和有效载荷限制下，从视觉观察中协调推力、姿态和日益增加的夹持器指令。本文综述了2017年至2026年间的183项贡献，按七个维度进行组织：VLA架构、训练方案、行动表示、双手协调（2022-2026）、无人驾驶飞行器（UAV）导航与控制（2017-2026）、语言基础以及包括记忆和世界模型在内的跨领域关注点。我们展示了为双手VLA开发的协调策略、训练方案和行动表示可以转移到无人机系统，并确定了两个领域的十四个研究方向。

View on arXiv Download PDF AI Translation

cs.RO / 6 / 2607.06724

EvoPlan: Evolutionary Neuro-Symbolic Robot Planning with Spatio-Temporal Guarantees

EvoPlan：具有时空保证的进化神经符号机器人规划

Nukapotula, Bhavya Sai, Moosavi, Samin, Wang, Haoze, Duncan, Luke, Shakkottai, Diya, Murali, Varun, Shakkottai, Srinivas

Abstract

LLM-based robot planners are fluent but cannot guarantee that their plans are executable or safe. Classical PDDL planners can guarantee these properties, but only after the problem is fully specified, and they make poor use of an LLM's ability to read context and repair plans. This paper presents a neuro-symbolic framework with three parts. All LLM calls use a locally-hosted open-weight model, so the pipeline can be deployed on-robot with no cloud dependency. First, an offline procedure that mines a single global Signal Temporal Logic (STL) constraint on mobility from demonstration data. The procedure recovers codified rules (e.g., stopping at red lights, mined from nuPlan driving logs) or population preferences (e.g., social-navigation comfort, mined from SCAND teleoperation), depending on what the demonstrations encode. Because the demonstrations are a one-class signal, we generate the missing negatives with counterfactual perturbations and an LLM violation generator and then fit the constraint by evolutionary search. We use the mined constraint to shield a vision-language driving policy on Bench2Drive and two discrete-action navigation policies on HA-VLN-CE. Second, an evolutionary PDDL planner: an LLM proposes and repairs plans, programmatic validators decide which ones survive, and the validated portion of the plan grows over iterations. We test the planner on the open-world ALFWorld Text benchmark, where it beats strong baselines and stays robust when the goal vocabulary does not match the action-model vocabulary. Third, a constrained execution loop: the planner's plan is compiled into waypoints, the waypoints are checked against the mined constraint, and the planner re-plans on a violation. We illustrate the full pipeline via demonstrations using the Gazebo simulator.

Chinese Translation

基于大型语言模型（LLM）的机器人规划器流畅但无法保证其计划是可执行或安全的。经典的PDDL规划器可以保证这些属性，但仅在问题完全指定后才能实现，并且未能充分利用LLM读取上下文和修复计划的能力。本文提出了一种由三个部分组成的神经符号框架。所有LLM调用使用本地托管的开放权重模型，因此该管道可以在机器人上部署，无需依赖云服务。首先，一个离线程序从演示数据中挖掘出一个关于移动性的全局信号时序逻辑（STL）约束。该程序恢复编码规则（例如，停在红灯前，从nuPlan驾驶日志中挖掘）或人群偏好（例如，社交导航舒适度，从SCAND远程操作中挖掘），具体取决于演示所编码的内容。由于演示是单类信号，我们通过反事实扰动和LLM违规生成器生成缺失的负样本，然后通过进化搜索拟合约束。我们使用挖掘的约束来保护在Bench2Drive上的视觉-语言驾驶策略和在HA-VLN-CE上的两个离散动作导航策略。其次，一个进化PDDL规划器：LLM提出并修复计划，程序验证器决定哪些计划存活，经过验证的计划部分在迭代中不断增长。我们在开放世界ALFWorld Text基准上测试该规划器，结果超越了强基线，并在目标词汇与动作模型词汇不匹配时保持稳健。第三，一个受约束的执行循环：规划器的计划被编译为航点，航点与挖掘的约束进行检查，规划器在违规时重新规划。我们通过使用Gazebo模拟器的演示来说明整个管道。

View on arXiv Download PDF AI Translation

cs.RO / 7 / 2607.06740

A Continual Learning Framework for Adaptive Control of Modular Soft Robots

用于模块化软机器人自适应控制的持续学习框架

Kushawaha, Nilay, Nazeer, Muhammad Sunny, Bal, Baljinder Singh, Laschi, Cecilia, Falotico, Egidio

Abstract

Soft robots have attracted significant attention in applications such as medical intervention, rehabilitation, and robotic manipulation due to their inherent compliance, flexibility, and high degrees of freedom. Modular soft robots (MSRs), composed of multiple interconnected segments, represent an emerging class of robotic systems with highly deformable and reconfigurable structures capable of performing complex tasks. However, designing controllers for MSRs remains challenging due to their nonlinear dynamics, modeling complexity, and hyper-redundant nature. Existing approaches typically require controllers to be retrained from scratch whenever the robot morphology changes. In this work, we address these challenges through a continual learning inspired control framework capable of incrementally adapting to changes in robot morphology while preserving previously acquired knowledge. Specifically, the proposed framework enables the controller to sequentially learn new MSR configurations without forgetting previously learned ones. In addition, for MSRs with fixed configurations, the same framework can be employed in a distributed manner to learn module-specific dynamics, enabling localized control and improved precision. The proposed approach is validated through closed-loop trajectory tracking experiments in simulation using a tendon-driven soft robot, as well as on a real-world three-module pneumatic soft robotic arm. Furthermore, we demonstrate the adaptive capabilities of the framework through a reaching experiment in which the controller selectively activates only the necessary modules to reach a virtual target position, thereby reducing computational overhead.

Chinese Translation

软机器人因其固有的顺应性、灵活性和高自由度，在医疗干预、康复和机器人操作等应用中引起了广泛关注。模块化软机器人（Modular Soft Robots, MSRs）由多个相互连接的部分组成，代表了一类新兴的机器人系统，具有高度可变形和可重构的结构，能够执行复杂任务。然而，由于其非线性动力学、建模复杂性和超冗余特性，为MSRs设计控制器仍然具有挑战性。现有方法通常要求在机器人形态发生变化时，从头开始重新训练控制器。在本研究中，我们通过一个受持续学习启发的控制框架来解决这些挑战，该框架能够逐步适应机器人形态的变化，同时保留先前获得的知识。具体而言，所提出的框架使控制器能够顺序学习新的MSR配置，而不会遗忘先前学习的配置。此外，对于具有固定配置的MSRs，相同的框架可以以分布式方式应用，以学习模块特定的动力学，从而实现局部控制和提高精度。通过在模拟中使用腱驱动软机器人进行闭环轨迹跟踪实验，以及在现实世界中的三模块气动软机器人臂上进行验证，证明了所提出方法的有效性。此外，我们通过一个到达实验展示了该框架的自适应能力，在该实验中，控制器仅选择性地激活必要的模块以达到虚拟目标位置，从而减少计算开销。

View on arXiv Download PDF AI Translation

cs.RO / 8 / 2607.06782

G-PROBE: Cross-FOV Place Recognition and Certainty-Coupled Localization for 3D Point Clouds

G-PROBE：跨视场位置识别与确定性耦合定位的3D点云方法

Lee, Jinseop

Abstract

Global localization from 3D point clouds remains challenging under limited or asymmetric fields of view (FOV), which fail to provide the dense, symmetric coverage that place recognition methods assume. We present G-PROBE, a learning-free global localization framework that removes this assumption. A virtual sensor decomposition runs the same pipeline, by design, on configurations ranging from a narrow-FOV sensor to a panoramic or multi-sensor rig. The front-end enumerates cross-FOV branch ensembles that encode heading hypotheses for heading-invariant place recognition. A score-scale-invariant, tuning-free gamma-SGRT suppresses heading aliasing under partial FOV and provably becomes inert at symmetric 360 degrees. The back-end, CG-GICP, refines a coarse full-cloud GICP with a pass restricted to high-certainty co-observed points selected by a bird's-eye-view certainty map (a by-product of front-end scoring). This certainty coupling links descriptor evaluation to 6-DoF metric pose estimation without an external verification module. Evaluated on five LiDAR datasets and three modalities (mechanical, solid-state, FMCW), G-PROBE attains the highest learning-free multi-session F1 on average and is competitive in panoramic single-session settings. Where hand-crafted and zero-shot supervised baselines collapse under wide-to-narrow cross-sensor pairing, it remains usable end-to-end (up to 55.0% vs. no more than 6.8% success), and under FOV asymmetry (360 to 60 degrees) it retains about 54% Recall@1, about 18x the strongest learning-free baseline.

Chinese Translation

在有限或不对称的视场（FOV）下，从3D点云进行全局定位仍然具有挑战性，因为这些视场无法提供位置识别方法所假设的密集、对称的覆盖。我们提出了G-PROBE，这是一种无学习的全局定位框架，消除了这一假设。虚拟传感器分解设计上在从窄视场传感器到全景或多传感器设备的配置上运行相同的流程。前端枚举跨视场分支集，编码用于方向不变位置识别的航向假设。一个分数尺度不变、无需调优的gamma-SGRT在部分视场下抑制航向混淆，并在对称的360度下证明变得无效。后端CG-GICP通过限制对由鸟瞰视角确定性图（前端评分的副产品）选择的高确定性共同观察点的访问，精炼粗略的全云GICP。这种确定性耦合将描述符评估与6自由度度量姿态估计联系起来，而无需外部验证模块。在五个LiDAR数据集和三种模式（机械、固态、FMCW）上进行评估，G-PROBE在平均上实现了最高的无学习多会话F1，并在全景单会话设置中具有竞争力。在宽到窄的跨传感器配对下，手工制作和零样本监督基线崩溃，而它在端到端使用中仍然可用（成功率高达55.0%，而不超过6.8%），并且在视场不对称（360到60度）下保持约54%的Recall@1，约为最强无学习基线的18倍。

View on arXiv Download PDF AI Translation

cs.RO / 9 / 2607.06824

CaLiSym: Learning Symplectic Dynamics of Real-World Systems through Structured Canonical Lifts

CaLiSym：通过结构化典范提升学习真实世界系统的辛动力学

Papatheodorou, Aristotelis, Vaidhyanathan, Pranav, Ares, Natalia, Havoutis, Ioannis, Milburn, Gerard J.

Abstract

Physics-informed learning promises data-efficient and stable dynamics prediction, yet its strongest geometric guarantees have largely remained confined to closed conservative systems. This excludes many robotic systems of practical interest, where actuation, dissipation, and constraints continuously exchange energy and momentum with the environment. We introduce CaLiSym, a lightweight framework that extends exact symplectic learning to such systems by changing where the geometric prior is imposed. Rather than enforcing symplecticity on the measured physical state, CaLiSym embeds the state and its physical ports into a structured lifted canonical phase space, where the learned dynamics evolve through an exactly symplectic map. The lift is explicit and algebraic, requiring neither recurrent latent states, transformer decoders, implicit optimization, nor inference-time ODE integration. We instantiate the framework with generalized-ridge SympNet predictors and introduce GRB-SympNet, a B-spline variant that combines local approximation with exact symplectic structure. Experiments on a controlled dissipative double pendulum, a real-world quadrotor, and a contact-rich quadruped demonstrate consistent improvements in out-of-distribution autoregressive prediction while using parameter-efficient models. At the same time, the learned lifted dynamics preserve the symplectic form to numerical precision. These results show that symplectic learning can be extended beyond conservative mechanics through structured canonical lifts, enabling geometry-preserving dynamics models for real-world robotic systems.

Chinese Translation

物理信息学习承诺实现数据高效和稳定的动力学预测，但其最强的几何保证在很大程度上仍局限于封闭的保守系统。这排除了许多实际应用中感兴趣的机器人系统，因为这些系统的驱动、耗散和约束不断与环境交换能量和动量。我们提出了CaLiSym，这是一个轻量级框架，通过改变几何先验的施加位置，将精确的辛学习扩展到此类系统。CaLiSym并不是在测量的物理状态上强制施加辛性，而是将状态及其物理端口嵌入到一个结构化的提升典范相空间中，在这里，学习到的动力学通过一个精确的辛映射演化。该提升是显式和代数的，不需要递归潜在状态、变压器解码器、隐式优化或推理时的常微分方程（ODE）积分。我们用广义岭SympNet预测器实例化该框架，并引入GRB-SympNet，这是一种结合局部近似和精确辛结构的B样条变体。在一个受控的耗散双摆、一个真实世界的四旋翼和一个接触丰富的四足机器人上的实验表明，在使用参数高效模型的同时，超出分布的自回归预测表现出一致的改进。同时，学习到的提升动力学在数值精度上保持了辛形式。这些结果表明，辛学习可以通过结构化典范提升扩展到保守力学之外，从而为真实世界的机器人系统提供几何保持的动力学模型。

View on arXiv Download PDF AI Translation

cs.RO / 10 / 2607.06882

GemNav: Discrete-Token Visual Robot Navigation using a Multimodal Large Language Model

GemNav：基于多模态大语言模型的离散标记视觉机器人导航

Bohm, Peter, Rahman, Saimunur, Khamis, Abdelwahed, Shrestha, Sagun Man Singh, McCool, Chris, Moghadam, Peyman

Abstract

Visual navigation policies built on large pretrained models have so far followed a common recipe: a dedicated visual encoder, a bespoke action head, and training on thousands of hours of cross-embodiment datasets. We ask whether this recipe is necessary. In this paper, we introduce GemNav, a visual robot navigation policy that adapts a frozen Multimodal Large Language Model (MLLM) for short-to-medium horizon waypoint navigation using Low-Rank Adaptation (LoRA) on the language tower alone, with no auxiliary visual encoder and no continuous regression head. Waypoints and categorical navigation signals share a single discrete token vocabulary generated by the language-model head, and a soft-decoded auxiliary loss recovers the metric structure that pure cross-entropy training discards. On a single 8.7-hour open corpus, roughly three orders of magnitude smaller than competing training sets, the policy transfers zero-shot to four physically distinct unseen environments and stops within 0.25-0.42m of the goal across 20 real-world trials covering an open carpark, an obstacle carpark, a long outdoor chemical yard, and an indoor warehouse. Conditioning on short image histories improves offline metrics but yields no robot benefit, pointing to a ceiling on what temporal context adds once pretrained vision features are in place. These results indicate that discrete-token adaptation of frozen MLLMs can provide a data-efficient, deployable alternative for foundation model robot navigation.

Chinese Translation

基于大型预训练模型的视觉导航策略迄今为止遵循了一种常见的方案：专用的视觉编码器、定制的动作头，以及在数千小时的跨实体数据集上进行训练。我们提出了这个方案是否必要的问题。本文介绍了GemNav，一种视觉机器人导航策略，它利用冻结的多模态大语言模型（MLLM）进行短期到中期的航点导航，仅在语言塔上使用低秩适应（LoRA），不使用辅助视觉编码器和连续回归头。航点和分类导航信号共享由语言模型头生成的单一离散标记词汇，而软解码的辅助损失则恢复了纯交叉熵训练所丢弃的度量结构。在一个单一的8.7小时开放语料库上，该策略在四个物理上不同的未见环境中实现了零-shot迁移，并在覆盖开放停车场、障碍停车场、长户外化学场和室内仓库的20次真实世界试验中，距离目标停止在0.25-0.42米之间。对短图像历史的条件化提高了离线指标，但对机器人没有益处，指出了一旦预训练的视觉特征到位，时间上下文所增加的内容的上限。这些结果表明，冻结的MLLM的离散标记适应可以为基础模型的机器人导航提供一种数据高效、可部署的替代方案。

View on arXiv Download PDF AI Translation

cs.RO / 11 / 2607.06896

Dynamic Object Detection and Tracking in Construction: A Fisheye Camera and LiDAR Sensor Fusion Model

建筑中的动态物体检测与跟踪：鱼眼相机与激光雷达传感器融合模型

Chen, Yilong, Huang, Huili, Cho, Yong K.

Abstract

Robust dynamic object detection and tracking are essential for enabling robots to operate safely and effectively alongside humans in complex environments such as construction sites. While LiDAR-based SLAM and occupancy grid methods offer viable solutions for detecting and tracking motion, many state-of-the-art 3D vision approaches rely heavily on pre-trained neural networks and require additional post-processing to identify moving objects. Sensor fusion techniques, combining the precision of LiDAR with the semantic richness of RGB imagery, offer a promising alternative. In this work, we present a novel framework that enhances a quadruped robot equipped with a LiDAR sensor and an upward-facing fisheye camera for real-time dynamic object detection and tracking. After identifying moving objects within a registered point cloud, our method assigns semantic labels by projecting 3D coordinates onto a 2D cylindrical panorama, aligning with real-time image-based detections for observation update of the Kalman filter. The proposed system demonstrates high precision, simplicity, and robustness, particularly in handling objects transitioning between dynamic and static states, thus it is well-suited for deployment in real-world construction environments.

Chinese Translation

稳健的动态物体检测与跟踪对于使机器人能够在建筑工地等复杂环境中安全有效地与人类协作至关重要。虽然基于激光雷达的同步定位与地图构建（SLAM）和占用网格方法为检测和跟踪运动提供了可行的解决方案，但许多最先进的三维视觉方法严重依赖于预训练的神经网络，并且需要额外的后处理来识别移动物体。传感器融合技术结合了激光雷达的精确性和RGB图像的语义丰富性，提供了一种有前景的替代方案。在本研究中，我们提出了一个新颖的框架，增强了一种配备激光雷达传感器和向上朝向的鱼眼相机的四足机器人，实现实时动态物体检测与跟踪。在识别注册点云中的移动物体后，我们的方法通过将三维坐标投影到二维圆柱全景上来分配语义标签，与基于图像的实时检测对齐，以更新卡尔曼滤波器的观测。所提出的系统展示了高精度、简便性和稳健性，特别是在处理动态与静态状态之间转换的物体时，因此非常适合在实际建筑环境中部署。

View on arXiv Download PDF AI Translation

cs.RO / 12 / 2607.06957

Flow-ERD: Agent-type Aware Flow Matching with Entropy-Regularized Distillation for Diverse Traffic Simulation

Flow-ERD：基于代理类型的流匹配与熵正则化蒸馏的多样化交通模拟

Hwang, Seulbin, Om, Kiyoung, Kim, Daejung, Lee, Jinhan

Abstract

Realistic and diverse traffic simulation is essential to autonomous driving development. Yet prevailing benchmarks predominantly reward realism, and recent methods have optimized accordingly, leaving diversity underexplored. We introduce \textbf{Flow-ERD}, a multi-agent simulator that pursues realism and diversity jointly. Its backbone, \textbf{Agent-Type Aware Flow Matching} (AFM), couples flow matching's multi-modal expressiveness with type-specific kinematic execution. It preserves fine-grained diversity while keeping motions consistent with each agent type. A second stage, \textbf{Entropy-Regularized Distillation} (ERD), fine-tunes the closed-loop rollout distribution with an entropy-regularized reverse-KL objective. This mitigates covariate shift while explicitly preventing collapse onto high-density modes. We evaluate Flow-ERD with a log-free diversity metric alongside standard realism scores. Flow-ERD ranks first on the WOSAC test benchmark and dominates the realism--diversity Pareto front among reproducible baselines. Our project page is available \href{https://seulbinhwang.github.io/flow-erd-project-page/}{here}.

Chinese Translation

现实且多样化的交通模拟对自动驾驶的发展至关重要。然而，现有的基准测试主要奖励现实性，近期的方法也相应进行了优化，导致多样性未得到充分探索。我们提出了 extbf{Flow-ERD}，一个追求现实性和多样性并重的多代理模拟器。其核心组件 extbf{代理类型感知流匹配}（AFM）将流匹配的多模态表现力与特定类型的运动执行相结合。它在保持每种代理类型运动一致性的同时，保留了细粒度的多样性。第二阶段 extbf{熵正则化蒸馏}（ERD）通过熵正则化的反向KL目标微调闭环展开分布。这减轻了协变量偏移，同时明确防止向高密度模式的崩溃。我们使用无日志多样性指标评估Flow-ERD，并与标准现实性评分进行比较。Flow-ERD在WOSAC测试基准中排名第一，并在可复现基线中主导现实性-多样性帕累托前沿。我们的项目页面可在此访问 exthref{https://seulbinhwang.github.io/flow-erd-project-page/}{这里}。

View on arXiv Download PDF AI Translation

cs.RO / 13 / 2607.06964

End-to-End LLM Flight Planning with RAG-based Memory and Multi-modal Coach Agent

基于RAG记忆和多模态教练代理的端到端大型语言模型飞行规划

Tabrizian, Amin, Aziz, Arsyi, Ullah, Aarifah, Ghazanfari, Mahyar, Razzaghi, Pouria, Wei, Peng

Abstract

Bridging the gap between human pilot intent and autonomous flight operation is critical for real-world electric vertical takeoff and landing (eVTOL) aircraft deployment. Flight planning traditionally relies on classic algorithms that struggle to incorporate flexible human preferences. We present FRAMe, an End-to-End Large Language Model (LLM) Flight Planning tool with RAG-based Memory and Multi-modal Coach Agent. Our system integrates a planner LLM with a multi-modal coach agent and retrieval augmented generation (RAG)-based memory to generate flight plans that satisfy mission constraints while aligning with human flight operator preferences. We demonstrate the system in a range of real-world-inspired scenarios of varying difficulty levels. Across four LLMs, the full FRAMe system (RAG and coach) yields the highest validity for every planner (up to 93.8% aggregate, 99% on Easy scenarios for the strongest planner) and shifts preference-relevant metrics in the operator-favored direction where the metric has headroom. FRAMe signifies how advanced LLMs can be deployed for human-centric mission planning, translating natural language instructions into safe, efficient, and flexible flight routes. The code is available at: github.com/amin-tabrizian/FlightPlanningLLMs

Chinese Translation

弥合人类飞行员意图与自主飞行操作之间的差距对于现实世界电动垂直起降（eVTOL）飞机的部署至关重要。传统的飞行规划依赖于经典算法，这些算法难以融入灵活的人类偏好。我们提出了FRAMe，一个基于RAG记忆和多模态教练代理的端到端大型语言模型（LLM）飞行规划工具。我们的系统将规划器LLM与多模态教练代理和基于检索增强生成（RAG）的记忆相结合，以生成满足任务约束并与人类飞行操作员偏好一致的飞行计划。我们在一系列灵感来源于现实世界的不同难度场景中展示了该系统。在四个LLM的测试中，完整的FRAMe系统（RAG和教练）在每个规划器中都产生了最高的有效性（总计高达93.8%，在最强规划器的简单场景中达到99%），并在有提升空间的指标上向操作员偏好的方向转变。FRAMe标志着先进的LLM如何用于以人为中心的任务规划，将自然语言指令转化为安全、高效和灵活的飞行路线。代码可在以下网址获取：github.com/amin-tabrizian/FlightPlanningLLMs

View on arXiv Download PDF AI Translation

cs.RO / 14 / 2607.06978

SPECTRA: Context-Conditioned Spectral Movement Primitives for Robot Skill Generalization

SPECTRA：用于机器人技能泛化的上下文条件谱运动原语

Zhang, Boxuan, Liu, Sheng, Ming, Chenglin, Abdelrahman, Ahmed

Abstract

Robot imitation learning for manipulation should preserve demonstrated task geometry while producing dynamically admissible robot motions. Existing pipelines often learn task-dependent trajectories and impose execution limits afterward through filtering, smoothing, clipping, or time scaling, which may distort task-critical end-effector paths. We propose the Spectral Movement Primitive (SMP), a frequency-domain imitation learning framework that couples task-space skill generation with joint-space execution regulation. Demonstrations are represented by truncated finite-horizon Fourier coefficients. An empirically selected low-frequency task band captures the dominant motion geometry, while higher harmonics contribute disproportionately to derivative growth. A frame-aware context-conditioned GMM/GMR prior predicts the task-band coefficients in a canonical task frame, and the resulting Cartesian trajectory is mapped to joint space through sequential inverse kinematics. A phase-coupled regulator then limits the requested phase progression without modifying the spectral coefficients, thereby enforcing joint velocity and acceleration limits while preserving the represented path. Experiments evaluate task-band reconstruction, robustness to composite demonstration corruption, out-of-distribution cross-board generalization, joint-space dynamic admissibility, end-effector path preservation, and deployment on a Franka Panda robot. Results show compact geometric reconstruction, consistent transfer across unseen task frames, substantial reductions in dynamic violations and jerk, and preservation of the intended end-effector path during phase regulation.

Chinese Translation

机器人模仿学习在操作中应保持所示任务的几何形状，同时产生动态可接受的机器人运动。现有的流程通常学习任务依赖的轨迹，并通过过滤、平滑、裁剪或时间缩放等方式在之后施加执行限制，这可能会扭曲任务关键的末端执行器路径。我们提出了谱运动原语（Spectral Movement Primitive, SMP），这是一种频域模仿学习框架，将任务空间技能生成与关节空间执行调节相结合。演示通过截断的有限时间傅里叶系数表示。经验选择的低频任务带捕捉主导的运动几何，而高次谐波则不成比例地贡献于导数增长。一个基于框架感知的上下文条件高斯混合模型/高斯混合回归（GMM/GMR）先验在规范任务框架中预测任务带系数，得到的笛卡尔轨迹通过顺序逆运动学映射到关节空间。一个相位耦合调节器随后限制请求的相位进展，而不修改谱系数，从而在保持表示路径的同时强制执行关节速度和加速度限制。实验评估了任务带重构、对复合演示损坏的鲁棒性、跨分布的跨板泛化、关节空间的动态可接受性、末端执行器路径的保持，以及在Franka Panda机器人上的部署。结果显示出紧凑的几何重构、在未见任务框架中的一致转移、动态违规和抖动的显著减少，以及在相位调节过程中保持预期的末端执行器路径。

View on arXiv Download PDF AI Translation

cs.RO / 15 / 2607.06988

WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time

WAM-TTT：通过观察人类在测试时的游戏来引导世界行动模型

Feng, Yusen, Han, Bingchen, Lyu, Jiangran, Liu, Kai, Zheng, Yixin, Wan, Yuxuan, Liu, Weiheng, Han, Sun, Li, Ruiqin, Zhang, Yulong, Liu, Fangfu, Shi, Xuesong, Liu, Libin, Wang, Yizhou, Zhang, Zhizheng, Wang, He

Abstract

Steering robot foundation models (RFMs) toward new task variants or user-preferred behaviors remains challenging, often requiring additional robot demonstrations, task-specific fine-tuning, or long-context conditioning. We present WAM-TTT, a test-time training framework for steering world action models from raw human videos. Rather than treating human videos as trajectories to imitate, WAM-TTT absorbs them into a lightweight adaptive memory inside a frozen WAM through self-supervised video prediction. To make this memory useful for control, we introduce a meta-training stage that aligns human demonstrations with robot behaviors using paired human-robot data and a key--value memory reconstruction objective. At test time, only unlabeled human videos are required to adapt the memory, while the pretrained WAM remains frozen. This enables efficient and reusable steering without robot actions, human-side annotations, or task-specific fine-tuning, while preserving the generalization ability of the foundation model. Extensive experiments show that WAM-TTT consistently outperforms in-context human-video conditioning baselines across diverse manipulation tasks and generalization settings.

Chinese Translation

引导机器人基础模型（RFMs）朝向新的任务变体或用户偏好的行为仍然具有挑战性，通常需要额外的机器人演示、特定任务的微调或长上下文的条件。我们提出了WAM-TTT，这是一种在测试时训练的框架，用于从原始人类视频中引导世界行动模型。WAM-TTT并不是将人类视频视为需要模仿的轨迹，而是通过自监督视频预测将其吸收到一个轻量级的自适应记忆中，该记忆位于一个冻结的WAM内部。为了使这个记忆在控制中有用，我们引入了一个元训练阶段，该阶段使用配对的人类-机器人数据和关键-值记忆重建目标，将人类演示与机器人行为对齐。在测试时，仅需未标记的人类视频来适应记忆，而预训练的WAM保持冻结。这使得在没有机器人动作、人类侧注释或特定任务微调的情况下实现高效且可重用的引导，同时保留基础模型的泛化能力。大量实验表明，WAM-TTT在各种操作任务和泛化设置中始终优于上下文人类视频条件基线。

View on arXiv Download PDF AI Translation

cs.RO / 16 / 2607.06989

Ace! Motion Planning of Professional-Level Table Tennis Serves with a Robot Arm

Ace！机器人臂专业级乒乓球发球的运动规划

Torrente, Guillem, Maeda, Guilherme Jorge, Grover, Divij, Tsukamoto, Megumu, Sahloul, Hamdi, Dürr, Peter

Abstract

Table tennis, a dynamic, compact, and popular sport, has received significant attention as a robotics benchmark over the last decades. Most of the research has focused on the rally aspect - returning an incoming ball - requiring high-speed vision, agile motion planning, and tight closed-loop control. However, the other component of table tennis gameplay - the serve - is comparatively a quite unexplored research problem, that in fact requires pushing physics modeling and control to the extremes. Achieving competitive serves with a robot presents domain-specific challenges, such as high-spin generation from a spinless ball, precise aiming, or multi-objective optimization. In this work, we present a novel approach for generating official rule-compliant serves by combining motion primitives, Model Predictive Control, and Bayesian Optimization. Serves generated in this way offer a wide and controllable variation of spins of up to 550 rad/s, and speeds of up to 6.7 m/s, matching and even surpassing those of elite table tennis players.

Chinese Translation

乒乓球是一项动态、紧凑且受欢迎的运动，在过去几十年中作为机器人学的基准受到了显著关注。大多数研究集中在对打方面——回击来球——这需要高速视觉、灵活的运动规划和紧密的闭环控制。然而，乒乓球比赛的另一个组成部分——发球——相对而言是一个尚未充分探索的研究问题，实际上需要将物理建模和控制推向极限。使用机器人实现竞争性发球面临领域特定的挑战，例如从无旋转的球体产生高旋转、精确瞄准或多目标优化。在本研究中，我们提出了一种新颖的方法，通过结合运动原语、模型预测控制（Model Predictive Control）和贝叶斯优化（Bayesian Optimization）来生成符合官方规则的发球。这种方式生成的发球提供了高达550 rad/s的旋转和高达6.7 m/s的速度，匹配甚至超越了顶级乒乓球运动员的水平。

View on arXiv Download PDF AI Translation

cs.RO / 17 / 2607.06990

A Closed-Loop Multi-Agent Framework for Robust Multi-Robot Manipulation

一种用于稳健多机器人操作的闭环多智能体框架

He, Yi-Xiang, Wei, Lan, Cen, Haoming, Jiang, Jian-Jian, Li, Zhuohao, Lu, Guanxing, Yang, Yihan, Zhang, Dandan, Zheng, Wei-Shi

Abstract

Multi-robot systems provide the parallelism and redundancy necessary for long-horizon tasks, while Large Language Models (LLMs) offer the reasoning capabilities to decompose these objectives into actionable plans. However, effectively grounding this high-level reasoning in physical multi-robot execution remains an open challenge. Existing LLM-based approaches fall mainly into two categories: Single-robot methods achieve robust contact-rich manipulation but lack the coordination mechanisms required for tasks spanning multiple workspaces. Current multi-robot frameworks focus on high-level planning, often treating manipulation as an idealized primitive that fails to account for real-world execution uncertainties. To address this, we propose a hierarchical closed-loop agentic LLM-based framework to ensure robust multi-robot manipulation. Our system consists of three specialized agents: the Planning Agent decomposes instructions into allocated sub-tasks, the Manipulation Agent for each robot executes actions via adaptive tool use, and the Verification Agent closes the loop by monitoring physical outcomes and feeding back semantic corrections. Extensive real-world experiments demonstrate that our framework achieves superior success rates, ensures robust adaptability ranging from single to cross workspace manipulation, and offers a generalizable approach for diverse manipulation tasks.

Chinese Translation

多机器人系统提供了执行长时间任务所需的并行性和冗余性，而大型语言模型（LLMs）则提供了将这些目标分解为可操作计划的推理能力。然而，将这种高层次的推理有效地落实到物理多机器人执行中仍然是一个未解决的挑战。现有的基于LLM的方法主要分为两类：单机器人方法实现了稳健的接触丰富操作，但缺乏执行跨多个工作空间任务所需的协调机制。目前的多机器人框架侧重于高层次规划，通常将操作视为理想化的原语，未能考虑现实执行中的不确定性。为了解决这个问题，我们提出了一种基于层次闭环智能体的LLM框架，以确保稳健的多机器人操作。我们的系统由三个专门的智能体组成：规划智能体将指令分解为分配的子任务，操作智能体为每个机器人通过自适应工具使用执行动作，验证智能体则通过监控物理结果并反馈语义修正来闭环。大量的现实世界实验表明，我们的框架实现了更高的成功率，确保了从单一到跨工作空间操作的稳健适应性，并为多样化的操作任务提供了一种可推广的方法。

View on arXiv Download PDF AI Translation

cs.RO / 18 / 2607.07076

PriGo: Test-Time Primitive Guidance to Diffusion and Flow Policies for Adaptive Robotic Manipulation

PriGo：用于自适应机器人操作的测试时原始指导扩散和流动策略

Li, Zezeng, Xiang, Enda, Tran, Thuy, Huang, Di, Thiam, Momath, Chen, Liming

Abstract

Imitation learning has enabled remarkable progress in robotic manipulation, especially with diffusion and flow-based policies that generate complex visuomotor behaviors directly from demonstrations. Yet, despite their strong performance, these policies often fail to generalize across tasks and environments. A key reason is that existing policies tend to imitate superficial action correlations rather than the underlying intent. Inspired by the compositional structure of human behaviors, we propose PriGo, a primitive-guided test-time adaptive framework for robust robotic manipulation. PriGo introduces PANet, a lightweight primitive prediction module that infers primitive distributions directly from observations. We further propose a differentiable primitive guidance mechanism that refines generated actions during inference, steering trajectories toward semantically consistent behaviors. Unlike prior primitive-conditioned approaches, PriGo operates entirely at test time and can be seamlessly integrated into pretrained diffusion and flow policies without retraining. Extensive experiments on LIBERO, CALVIN, SIMPLER, and real-world robotic tasks demonstrate that PriGo consistently improves robustness, long-horizon execution, and generalization ability across both diffusion and flow-based policies.

Chinese Translation

模仿学习在机器人操作中取得了显著进展，特别是基于扩散和流动的策略，这些策略能够直接从示范中生成复杂的视觉运动行为。然而，尽管这些策略表现出色，但它们在任务和环境之间的泛化能力往往不足。一个关键原因是现有策略倾向于模仿表面的动作关联，而非潜在的意图。受到人类行为组合结构的启发，我们提出了PriGo，一种原始指导的测试时自适应框架，用于稳健的机器人操作。PriGo引入了PANet，一个轻量级的原始预测模块，能够直接从观察中推断原始分布。我们进一步提出了一种可微分的原始指导机制，在推理过程中细化生成的动作，引导轨迹朝向语义一致的行为。与之前的原始条件方法不同，PriGo完全在测试时运行，并且可以无缝集成到预训练的扩散和流动策略中，而无需重新训练。在LIBERO、CALVIN、SIMPLER和真实世界机器人任务上的大量实验表明，PriGo在稳健性、长时间执行和泛化能力方面始终有所提升，适用于基于扩散和流动的策略。

View on arXiv Download PDF AI Translation

cs.RO / 19 / 2607.07101

GeoProp: Grounding Robot State in Vision for Generalist Manipulation

GeoProp：在视觉中为通用操作提供机器人状态的基础

Zhao, Guoyang, Qian, Quanhao, Zhang, Gongjie, Li, Wenhao, Wang, Jiuniu, Lu, Xiaowei, Zhao, Deli, Xu, Ran

Abstract

Proprioception is fundamental to robotic manipulation, yet standard fusion methods often treat it as an isolated vector lacking explicit alignment with visual tokens. Without a direct correspondence between 3D kinematics and 2D feature maps, manipulation policies struggle to ground the robot's state within the scene, frequently underperforming even vision-only baselines. To address this, we introduce GeoProp, a lightweight, plug-and-play adapter that aligns proprioception with vision through explicit geometric grounding and spatial feature sampling. GeoProp projects the robot state onto the image plane to sample localized visual features, constructing a grounded state token. It then injects state-derived spatial priors into the corresponding visual features via FiLM modulation. To capture motion intent, GeoProp further samples features at a short-horizon predicted coordinate derived from recent kinematics, providing look-ahead visual context. Across 67 tasks, GeoProp improves Diffusion Policy by 8.7% on 63 simulation tasks and pi_0 by 4.0% on the RoboTwin subset, and yields a 10.6% average gain across both policy families in the real world, while adding only 2-3% to the parameter count. These results demonstrate that GeoProp is a simple yet high-impact inductive bias for generalist embodied policies. Project page: https://alibaba-damo-academy.github.io/GeoProp/.

Chinese Translation

本体感知是机器人操作的基础，但标准的融合方法通常将其视为一个孤立的向量，缺乏与视觉标记的明确对齐。在3D运动学与2D特征图之间没有直接对应关系的情况下，操作策略难以将机器人的状态与场景中的内容相结合，往往表现不及仅依赖视觉的基线方法。为了解决这一问题，我们提出了GeoProp，这是一种轻量级的即插即用适配器，通过明确的几何基础和空间特征采样将本体感知与视觉对齐。GeoProp将机器人状态投影到图像平面上，以采样局部视觉特征，构建一个基础状态标记。然后，它通过FiLM调制将状态派生的空间先验注入到相应的视觉特征中。为了捕捉运动意图，GeoProp进一步在由最近的运动学推导出的短期预测坐标上采样特征，提供前瞻性的视觉上下文。在67个任务中，GeoProp在63个仿真任务上将Diffusion Policy的性能提高了8.7%，在RoboTwin子集上将pi_0的性能提高了4.0%，并在现实世界中实现了两个策略家族平均10.6%的增益，同时仅增加了2-3%的参数数量。这些结果表明，GeoProp是对通用体态策略的一种简单而高影响力的归纳偏置。项目页面：https://alibaba-damo-academy.github.io/GeoProp/.

View on arXiv Download PDF AI Translation

cs.RO / 20 / 2607.07129

Compositional Motion Generation from Demonstration with Object-Centric Neural Fields

基于对象中心神经场的示范生成组合运动

Tekden, Ahmet Ercan, Bekiroglu, Yasemin

Abstract

Compositionality, by organizing complex behavior as combinations of simpler elements, enables robot learning that is scalable and data efficient. Leveraging this principle, we propose a generative learning-from-demonstration framework that enables compositional modeling of robotic behavior by connecting perception and motion through shared object-level representations. We render scenes from object-centric neural representations that integrate canonical neural fields with latent-conditioned deformations, capturing positional and geometric variations in a smooth, consistent, and interpretable way. For motion generation, a temporal mixture-of-experts (MoE) employs a gating mechanism to combine object-conditioned movement primitives over time, producing complete trajectories. This spatial-temporal compositionality maintains the data efficiency of movement primitives while grounding motion in visual structure, enabling systematic generalization across diverse scene configurations. In simulation, long-horizon manipulation tasks are successfully completed using the proposed model, which requires significantly less training data than other image-based baselines. Real-world experiments further demonstrate the method's robustness to noise, its ability to generalize at the category level through language-based segmentation models, and its capacity to operate directly on 3D scene representations.

Chinese Translation

组合性通过将复杂行为组织为简单元素的组合，使得机器人学习具备可扩展性和数据效率。基于这一原则，我们提出了一种生成式学习-从示范框架，能够通过共享的对象级表示连接感知与运动，从而实现机器人行为的组合建模。我们从对象中心神经表示中渲染场景，这些表示将典型神经场与潜在条件变形相结合，以平滑、一致且可解释的方式捕捉位置和几何变化。对于运动生成，时间混合专家（MoE）采用门控机制在时间上结合对象条件的运动原语，从而生成完整的轨迹。这种时空组合性保持了运动原语的数据效率，同时将运动基于视觉结构进行扎根，使得在不同场景配置中实现系统化的泛化。在仿真中，使用所提出的模型成功完成了长时间范围的操作任务，该模型所需的训练数据显著少于其他基于图像的基线。现实世界的实验进一步证明了该方法对噪声的鲁棒性、通过基于语言的分割模型在类别层面进行泛化的能力，以及直接在3D场景表示上操作的能力。

View on arXiv Download PDF AI Translation

cs.RO / 21 / 2607.07136

Learning Spatiotemporal Tubes for Full Class of Signal Temporal Logic Tasks for Control of Unknown Systems under Input Constraints

基于时空管道的控制框架：针对未知系统在输入约束下的全类信号时序逻辑任务的学习

Basu, Ahan, Das, Ratnangshu, Nath, Soumyodipta, Liu, Siyuan, Jagtap, Pushpak

Abstract

This paper presents a Spatiotemporal Tube (STT)-based control framework for general unknown nonlinear Euler-Lagrange (EL) systems subject to input constraints, with the objective of satisfying Signal Temporal Logic (STL) specifications, where confinement of the system trajectory within the STT guarantees the satisfaction of the corresponding STL task. For both single and multi-agent scenarios, the STT corresponding to each agent is modeled as a time-varying ball, whose center and radius are jointly parameterized using a physics-informed neural network (PINN). The robustness metric associated with the STL specification corresponding to the agents is incorporated into the training process as a loss function, enabling the learned tube to encode task-level temporal requirements. For a multi-agent scenario, we introduce an additional robustness metric corresponding to the global task, which, when satisfied, ensures the tubes do not collide with each other. To ensure that the system trajectory remains within the learned STT and thereby satisfies the local and global STL specifications, we propose a control strategy that explicitly accounts for input constraints. In particular, a closed-form control law is developed to keep the trajectory inside the tube while regulating the motion of the tube by enforcing bounds on its evolution depending on the input constraints of the system. The proposed approach has been validated over several case studies.

Chinese Translation

本文提出了一种基于时空管道（Spatiotemporal Tube, STT）的控制框架，适用于在输入约束下的未知非线性欧拉-拉格朗日（Euler-Lagrange, EL）系统，旨在满足信号时序逻辑（Signal Temporal Logic, STL）规范，其中系统轨迹被限制在STT内可确保相应STL任务的满足。在单一和多智能体场景中，每个智能体对应的STT被建模为一个时变球体，其中心和半径通过物理信息神经网络（Physics-Informed Neural Network, PINN）进行联合参数化。与智能体对应的STL规范相关的鲁棒性度量被纳入训练过程作为损失函数，使得学习到的管道能够编码任务级的时间要求。在多智能体场景中，我们引入了一个额外的鲁棒性度量，针对全局任务，当满足该度量时，确保管道之间不发生碰撞。为了确保系统轨迹保持在学习到的STT内，从而满足局部和全局的STL规范，我们提出了一种控制策略，明确考虑输入约束。特别地，开发了一种闭式控制律，以保持轨迹在管道内，同时通过施加系统输入约束的演变边界来调节管道的运动。所提出的方法已在多个案例研究中得到了验证。

View on arXiv Download PDF AI Translation

cs.RO / 22 / 2607.07139

Disturbance-aware Motion Planning for Over-actuated Underwater Vehicles Exploiting Actuation Redundancy for High-fidelity 3D Reconstruction

基于干扰感知的过驱动水下机器人运动规划：利用驱动冗余实现高保真三维重建

Gao, Yuer, Xu, Tongqing, Liu, Qingyang, Cai, Yi

Abstract

Underwater robots often operate near delicate targets where high-power thrusters resuspend sediments and induce turbulence, degrading image quality at the sensor input. Conventional controllers optimize vehicle-centric objectives, such as tracking and stability, without accounting for the impact of actuation on sensing. We address this actuation-to-perception coupling by exploiting redundancy in over-actuated platforms. For an eight-thruster ROV, multiple thrust allocations can yield the same motion; we search this null space to minimize predicted disturbance in a task-relevant target region while enforcing motion constraints. Our method uses a control-oriented thruster-wake proxy derived from actuator-disk theory with directional attenuation and validated by PIV ($R^2 = 0.99$ near the wake axis; $R^2 > 0.82$ in the primary wake region), together with a real-time redundancy-resolving allocator running at 10 Hz (45 ms/solve). Across 440 trials, the approach reduces target-region particle velocity by 67% ($p < 0.001$), improves 3D reconstruction RMSE by 55% versus a disturbance-unaware baseline ($1.9 \pm 0.4$ mm vs. $4.3 \pm 1.8$ mm), and achieves a 98.5% reconstruction success rate. The framework supports autonomous scanning, which is quantitatively evaluated, and operator-assisted inspection, which is demonstrated in the supplementary materials.

Chinese Translation

水下机器人通常在脆弱目标附近操作，高功率推进器会重新悬浮沉积物并引发湍流，从而降低传感器输入的图像质量。传统控制器优化以车辆为中心的目标，例如跟踪和稳定性，而未考虑驱动对感知的影响。我们通过利用过驱动平台中的冗余来解决这种驱动与感知的耦合问题。对于一台八推进器的遥控水下机器人（ROV），多种推力分配可以产生相同的运动；我们在这个零空间中搜索，以最小化任务相关目标区域内的预测干扰，同时强制执行运动约束。我们的方法使用基于驱动盘理论的控制导向推进器尾流代理，结合方向衰减，并通过粒子图像测速（PIV）验证（在尾流轴附近 $R^2 = 0.99$；在主要尾流区域 $R^2 > 0.82$），同时运行一个实时冗余解决分配器，频率为10 Hz（每次求解45毫秒）。在440次试验中，该方法将目标区域的粒子速度降低了67%（$p < 0.001$），与未考虑干扰的基线相比，3D重建的均方根误差（RMSE）改善了55%（$1.9 imes 0.4$ mm 对比 $4.3 imes 1.8$ mm），并实现了98.5%的重建成功率。该框架支持自主扫描，并进行了定量评估，同时展示了操作员辅助检查的能力，相关内容在补充材料中进行了展示。

View on arXiv Download PDF AI Translation

cs.RO / 23 / 2607.07196

Validate the Dream Before You Trust Its Verdict: Admissibility for World-Model Simulators

在信任其裁决之前验证梦境：世界模型模拟器的可接受性

Oefinger, Christian, Schäfer, Finn Rasmus, Moller, Korbinian, Piccinini, Mattia, Betz, Johannes

Abstract

Across robotics, World Models (WMs) are increasingly used to evaluate action policies by simulating the consequences of actions in an imagined world, and returning a success or safety verdict. Yet a verdict is only as trustworthy as the WM that produced it, and the WM itself needs to be certified. In video-generation WMs, fidelity metrics such as Fr\'echet Video Distance (FVD) reward visual realism, but ignore whether the world responds correctly to the policy's actions, including those unseen in training. Classical simulation-based validation assumes a trusted simulator evaluating an untrusted policy, whereas generative WMs are themselves unverified learned artifacts. Hence, we argue that any WM used as a test oracle must first be accredited before its verdicts can serve as evidence. Building on credibility practices from safety-critical simulation, including Verification, Validation & Accreditation (VV&A), Safety of the Intended Functionality (SOTIF), and scenario-based testing standards, we define an admissibility ladder (L0-L4) that a WM must climb before its closed-loop verdicts are accepted as assurance evidence. Our framework is embodiment-agnostic, and is instantiated in autonomous driving (AD), where assurance methods for traditional simulation are most mature. Applied to two driving WMs, the lower rungs reveal a reversal: the model that ranks higher on visual generation quality (L0) ranks lower on action-following (L1-L2), so visual fidelity does not predict the action-robustness a closed-loop verdict depends on.

Chinese Translation

在机器人技术领域，世界模型（World Models, WMs）越来越多地用于通过模拟想象世界中行动的后果来评估行动策略，并返回成功或安全的裁决。然而，裁决的可信度取决于产生该裁决的世界模型，而世界模型本身也需要经过认证。在视频生成的世界模型中，诸如Fréchet视频距离（Fréchet Video Distance, FVD）等保真度指标奖励视觉真实感，但忽略了世界是否正确响应策略的行动，包括那些在训练中未见的行动。经典的基于模拟的验证假设一个可信的模拟器在评估一个不可信的策略，而生成的世界模型本身是未经验证的学习产物。因此，我们认为，任何作为测试神谕的世界模型在其裁决可以作为证据之前，必须首先获得认证。基于安全关键模拟中的可信度实践，包括验证、确认与认证（Verification, Validation & Accreditation, VV&A）、预期功能的安全性（Safety of the Intended Functionality, SOTIF）和基于场景的测试标准，我们定义了一个可接受性阶梯（L0-L4），世界模型必须在接受其闭环裁决作为保证证据之前逐步攀登。我们的框架与具体实现无关，并在自动驾驶（Autonomous Driving, AD）中得以体现，传统模拟的保证方法在这一领域最为成熟。应用于两个驾驶世界模型时，较低的阶梯揭示了一个反转：在视觉生成质量（L0）上排名更高的模型在行动跟随（L1-L2）上排名更低，因此视觉保真度并不能预测闭环裁决所依赖的行动鲁棒性。

View on arXiv Download PDF AI Translation

cs.RO / 24 / 2607.07253

Manual, Joystick, or Haptic Control? An In Vitro Comparison of Navigation Strategies for Robotic Interventional Neuroradiology Procedures

手动、操纵杆还是触觉控制？机器人介入神经放射学程序导航策略的体外比较

Jackson, Benjamin, Fischer, Nikola, Robershaw, Harry, Chen, Xingyu, Sadati, S. H. Hadi, Li, Yang, Lynch, Jeremy, Abdelsalam, Nasr, Buwanabala, Jonathon, Benger, Matthew, Sciacca, Sara, Kandasamy, Naga, Mancuso-Marcello, Marco, Balasundaram, Parthiban, Guruge, Sahan, Das, Neelan, Granados, Alejandro, Rhode, Kawal, Booth, Thomas C

Abstract

Objective: To evaluate robotic controller interfaces for interventional neuroradiology procedures in-vitro incorporating a force-sensing platform to assess safety. Methods: A custom endovascular robot, device-mimicking controller, and sensorized neurovascular phantom were developed. Ten interventional neuroradiologists (4 novices, 6 experts) performed simulated navigations using four control modalities: device-mimicking controllers with and without haptic feedback, joystick-based input, and manual navigation. Navigation time, peak vessel-wall forces, incorrect catheterisations, and prolapse events were assessed, alongside user analyses. Results: Manual navigation was fastest (mean 47.7 s) compared to haptic-on (248.7 s), haptic-off (314.7 s), and joystick (392.6 s) modalities (p<0.001). Regardless of controller type, vessel-wall forces were below the 0.70 N puncture threshold; therefore all modalities were considered safe. Joystick produced significantly more prolapse events than manual control (1.56 vs 0.13; p=0.018). Operator experience was relevant to performance: experts made fewer incorrect catheterisations than novices (0.25 vs 0.62; p=0.035) and applied less vessel-wall force (p<0.0005); these effects were sustained across controllers but accentuated when haptics were on. Users perceived haptic on and haptic off as similarly intuitive, and more intuitive than joystick (p=0.033). Conclusion: Device-mimicking robotic controllers outperform joystick interfaces on most metrics; haptic feedback shows promising but non-significant performance benefits.

Chinese Translation

目的：评估用于介入神经放射学程序的机器人控制器接口，采用力传感平台进行安全性评估。方法：开发了一种定制的血管内机器人、设备模拟控制器和传感神经血管模型。十名介入神经放射学专家（4名新手，6名专家）使用四种控制方式进行模拟导航：带触觉反馈和不带触觉反馈的设备模拟控制器、基于操纵杆的输入和手动导航。评估了导航时间、峰值血管壁力、错误导管插入和脱垂事件，同时进行了用户分析。结果：手动导航的速度最快（平均47.7秒），相比之下，触觉开启（248.7秒）、触觉关闭（314.7秒）和操纵杆（392.6秒）模式的速度较慢（p<0.001）。无论控制器类型如何，血管壁力均低于0.70 N的穿刺阈值，因此所有模式均被认为是安全的。操纵杆产生的脱垂事件显著多于手动控制（1.56 vs 0.13；p=0.018）。操作员经验与表现相关：专家的错误导管插入次数少于新手（0.25 vs 0.62；p=0.035），施加的血管壁力也较少（p<0.0005）；这些效应在不同控制器间保持一致，但在触觉开启时更为明显。用户认为触觉开启和关闭的直观性相似，且比操纵杆更直观（p=0.033）。结论：设备模拟机器人控制器在大多数指标上优于操纵杆接口；触觉反馈显示出有希望但并不显著的性能优势。

View on arXiv Download PDF AI Translation

cs.RO / 25 / 2607.07281

Programmable Synchronization Graphs for Adaptive and Fault-Tolerant Modular Miniature Robots

用于自适应和容错的可编程同步图在模块化微型机器人中的应用

Kulekcioglu, Okan, Ahmad, Arqam Bin, Garcia, Ines, Alves, Filipe Serra, Ozcan, Onur, Hanay, M. Selim

Abstract

Modular miniature robots could provide scalable function in constrained environments, but coordinating many imperfect modules remains difficult when computation, communication and reliability are limited. A central robotics challenge is to coordinate many actuator-sensor modules without assigning a privileged leader, prescribing a fixed gait template, or relying on dense communication. Here we introduce a programmable synchronization-graph framework for modular miniature robots in which each actuator-sensor pair is represented as a network node and locomotor coordination is encoded through graph coupling. Fixed intra-subgraph links synchronize heterogeneous actuator groups, whereas a small number of signed inter-subgraph links program phase relationships between groups. In physical robot collectives with up to nine modules, graph coupling drives the emergence of synchronization, signed links tune the phase difference from in-phase to out-of-phase motion, and floor experiments produce gallop-like and trot-like contact patterns in a five-module robot assembly. Replacing dense all-to-all coupling with sparse d-regular topologies preserves synchronization while reducing the coupling burden. The same graph representation also captures fault tolerance: increasing graph degree increases the number of module deactivations tolerated before desynchronization. Finally, an upper-confidence-bound edge-selection algorithm learns inter-subgraph links that drive the system toward target phase states. In a separate deactivation benchmark, the graph-based controller avoids the leader-specific failure mode observed in centralized leader-follower control and reduces worst-case phase error by about threefold. These results establish programmable network topology as a compact control layer for gait phase programming, online adaptation and robustness to unit loss in modular miniature robots.

Chinese Translation

模块化微型机器人可以在受限环境中提供可扩展的功能，但在计算、通信和可靠性受限的情况下，协调多个不完美的模块仍然困难。一个中心的机器人挑战是如何在不指定特权领导者、不规定固定步态模板或依赖密集通信的情况下协调多个执行器-传感器模块。在此，我们介绍了一种可编程同步图框架，用于模块化微型机器人，其中每个执行器-传感器对被表示为网络节点，运动协调通过图耦合进行编码。固定的子图内部链接使异构执行器组同步，而少量的有符号子图间链接则编程组间的相位关系。在最多九个模块的物理机器人集体中，图耦合驱动同步的出现，有符号链接调节相位差，从同相运动到异相运动，地面实验在五模块机器人组装中产生了类似于奔跑和小跑的接触模式。用稀疏的 d-正则拓扑替代密集的全连接耦合，保持同步的同时减少了耦合负担。同样的图表示也捕捉了容错性：增加图的度数可以增加在去同步之前容忍的模块失效数量。最后，一个上置信界边选择算法学习驱动系统朝向目标相位状态的子图间链接。在一个单独的失效基准测试中，基于图的控制器避免了在集中式领导-跟随控制中观察到的特定于领导者的失效模式，并将最坏情况下的相位误差减少了约三倍。这些结果确立了可编程网络拓扑作为模块化微型机器人步态相位编程、在线适应和对单元失效的鲁棒性的紧凑控制层。

View on arXiv Download PDF AI Translation

cs.RO / 26 / 2607.07287

TouchWorld: A Predictive and Reactive Tactile Foundation Model for Dexterous Manipulation

TouchWorld：用于灵巧操作的预测性和反应性触觉基础模型

Zhou, Jianyi, Hong, Feiyang, Li, Yunhao, Zhao, Yicheng, Cen, Yongjue, Liu, Zirui, Huang, Jiakang, Chen, Zirui, Zhang, Ruiyang, Zhu, Weizhuo, Song, Xuhua, Yang, Shuo

Abstract

Dexterous manipulation in everyday environments requires both anticipation and reaction: a robot must predict how contact should evolve while rapidly correcting local errors caused by slip, misalignment, unstable grasping, or force mismatch. Vision and language provide semantic and geometric guidance, but they cannot reliably reveal hidden contact states such as force, slip, and contact stability. Although tactile sensing exposes these physical cues, most existing policies treat touch as a low-frequency observation stream within a monolithic action model, coupling slow task reasoning, action generation, and fast contact feedback in a single loop. We introduce TouchWorld, a predictive-and-reactive tactile foundation model for dexterous manipulation. TouchWorld uses a hierarchical policy that separates vision-language subtask planning, tactile world-model prediction, visuo-tactile goal-conditioned action generation, and high-frequency tactile residual refinement. A High-Level Planning Layer produces executable subtasks and predicts tactile subgoals; a Visuo-Tactile Goal-Conditioned Policy generates nominal action chunks; and a Tactile-Conditioned Refinement Policy performs online residual correction using recent tactile and proprioceptive feedback. By using touch as both a predictive contact reference and a fast feedback signal, TouchWorld preserves the semantic generalization of vision-language-action policies while improving local contact adaptation. Across six long-horizon and contact-rich dexterous manipulation tasks, TouchWorld achieves 65.0% success in the clean setting and 53.7% success under human perturbations, outperforming the strongest baseline by 15.7 and 18.5 percentage points, respectively.

Chinese Translation

在日常环境中，灵巧操作需要预测和反应：机器人必须预测接触如何演变，同时快速纠正由于滑动、错位、不稳定抓取或力不匹配造成的局部错误。视觉和语言提供语义和几何指导，但它们无法可靠地揭示隐藏的接触状态，如力、滑动和接触稳定性。尽管触觉传感能够暴露这些物理线索，但现有的大多数策略将触觉视为单一动作模型中的低频观察流，将缓慢的任务推理、动作生成和快速的接触反馈耦合在一个循环中。我们提出了TouchWorld，一个用于灵巧操作的预测性和反应性触觉基础模型。TouchWorld使用层次化策略，将视觉-语言子任务规划、触觉世界模型预测、视觉-触觉目标条件下的动作生成和高频触觉残差细化分开。高层规划层生成可执行的子任务并预测触觉子目标；视觉-触觉目标条件策略生成名义动作块；而触觉条件细化策略则利用最近的触觉和本体反馈进行在线残差校正。通过将触觉作为预测接触参考和快速反馈信号，TouchWorld在保持视觉-语言-动作策略的语义泛化的同时，改善了局部接触适应性。在六个长时间跨度和接触丰富的灵巧操作任务中，TouchWorld在干净环境下的成功率达到65.0%，在受到人为干扰的情况下成功率为53.7%，分别比最强基线高出15.7和18.5个百分点。

View on arXiv Download PDF AI Translation

cs.RO / 27 / 2607.07294

Multimodal Voice Activity Projection for Turn-Taking in Social Robots with Voice-Activity-Related Pretrained Encoders

用于社交机器人轮流对话的多模态语音活动投影与语音活动相关的预训练编码器

Cano, Antonio, Pérez, Guillermo, Merino, Luis, Gomez, Randy

Abstract

Turn-taking prediction is a key requirement for social robots involved in human-human interaction, particularly in mediator settings, where the robot must anticipate conversational dynamics rather than merely react to pauses. This work presents a Multimodal Voice Activity Projection (MM-VAP) framework that extends the original audio-only VAP formulation to synchronized audio-visual inputs while preserving its self-supervised future-projection objective. The proposed approach builds on pretrained audio-visual backbones originally optimized for speech-related tasks and adapts them through Low-Rank Adaptation to the multimodal turn-taking problem. After independent speaker encoding, an inter-speaker attention stage models the relational dynamics required to project future voice activity. In addition, a semantic consistency loss is introduced to regularize the 256-state output space according to higher-level dialogue activity patterns. Experiments on NoXi and NoXi+J showed improvements over the current baselines, particularly for some turn-taking events. Additional evaluation on the Haru EDR corpus further supported the suitability of this direction for mediation-oriented human-robot interaction.

Chinese Translation

轮流对话预测是参与人际互动的社交机器人所需的关键能力，特别是在调解环境中，机器人必须预测对话动态，而不仅仅是对停顿作出反应。本研究提出了一种多模态语音活动投影（Multimodal Voice Activity Projection, MM-VAP）框架，该框架将原始的仅音频语音活动投影（Voice Activity Projection, VAP）扩展到同步的音频-视觉输入，同时保持其自监督的未来投影目标。所提出的方法基于原本为与语音相关任务优化的预训练音频-视觉骨干网络，并通过低秩适应（Low-Rank Adaptation）将其调整为多模态轮流对话问题。在独立的说话者编码后，跨说话者注意力阶段建模了投影未来语音活动所需的关系动态。此外，引入了一种语义一致性损失，以根据更高层次的对话活动模式对256状态输出空间进行正则化。在NoXi和NoXi+J上的实验显示出相较于当前基线的改进，特别是在某些轮流对话事件上。对Haru EDR语料库的进一步评估进一步支持了这一方向在以调解为导向的人机交互中的适用性。

View on arXiv Download PDF AI Translation

cs.RO / 28 / 2607.07350

Towards Reliable Aerial Ground Vehicle Collaboration: An Integrated Planning and Autonomy Framework for Field Deployment

迈向可靠的空中地面车辆协作：用于现场部署的综合规划与自主框架

Mondal, Md Safwan, Russo, Luca, Humann, James D., Dotterweich, James M., Bhounsule, Pranav

Abstract

Limited flight endurance significantly restricts the operational range of unmanned aerial vehicles (UAVs) in long duration missions such as surveillance and inspection, where multiple spatially distributed Areas of Interest (AOIs) must be visited. These tasks require efficient routing determining the sequence of visits which directly impacts mission time, energy consumption, and overall feasibility. Pairing UAVs with unmanned ground vehicles (UGVs) for mobile recharging offers a promising solution, but introduces a tightly coupled cooperative routing problem involving UAV route planning, UGV road constrained movement, energy management, and rendezvous scheduling under uncertainty. In this work, we present an integrated planning and autonomy framework for reliable field deployment. We formulate the problem as an energy constrained cooperative routing task and solve it using a Deep Reinforcement Learning (DRL) based planner that jointly optimizes the UAV visitation sequence and rendezvous locations with the UGV, outperforming baseline heuristics in minimizing total mission time. To bridge the gap between planning and execution, we introduce a standardized two layer YAML based mission API that captures environment states and structures lightweight, synchronized action sequences. This framework is supported by a complete autonomy stack using PX4/MAVSDK for UAV control and ROS 2/Nav2 for UGV navigation. Furthermore, we propose a lightweight Rendezvous Aware Replanner (RARP) that operates online to handle environmental uncertainties, reducing energy margin violations from 83.33% to 20.00%. The full system is validated through outdoor field experiments, demonstrating robust cooperative navigation and adaptability in dynamic tasks, including a search and rescue scenario with vision language model (VLM) based hazard detection

Chinese Translation

有限的飞行耐力显著限制了无人机（UAV）在长时间任务中的操作范围，例如监视和检查，这些任务需要访问多个空间分布的兴趣区域（AOI）。这些任务要求高效的路径规划，以确定访问顺序，这直接影响任务时间、能量消耗和整体可行性。将无人机与无人地面车辆（UGV）配对进行移动充电提供了一种有前景的解决方案，但这引入了一个紧密耦合的合作路径规划问题，涉及无人机路线规划、UGV受限于道路的移动、能量管理和在不确定性下的会合调度。在本研究中，我们提出了一个用于可靠现场部署的综合规划与自主框架。我们将问题表述为一个能量受限的合作路径规划任务，并使用基于深度强化学习（DRL）的规划器进行求解，该规划器联合优化无人机的访问顺序和与UGV的会合位置，在最小化总任务时间方面优于基线启发式方法。为了弥合规划与执行之间的差距，我们引入了一个标准化的基于YAML的双层任务API，该API捕获环境状态并构建轻量级、同步的动作序列。该框架由一个完整的自主堆栈支持，使用PX4/MAVSDK进行无人机控制，使用ROS 2/Nav2进行UGV导航。此外，我们提出了一种轻量级的会合感知重规划器（RARP），该重规划器在线运行以处理环境不确定性，将能量余量违规从83.33%降低到20.00%。整个系统通过户外实地实验进行了验证，展示了在动态任务中强大的合作导航能力和适应性，包括基于视觉语言模型（VLM）的危险检测的搜索与救援场景。

View on arXiv Download PDF AI Translation

cs.RO / 29 / 2607.07357

HumAIN: Human-Aware Implicit Social Robot Navigation

HumAIN：人类意识隐式社交机器人导航

Song, Daeun, Le, Nhat, Chen, Jeffrey, Nazeri, Mohammad, Payandeh, Amirreza, Chandra, Rohan, Mirsky, Reuth, Mead, Ross, Xiao, Ling, Xiao, Xuesu

Abstract

Effective social robot navigation requires sensitivity to human behavior, often revealed through subtle skeletal cues like gait and orientation. We present Human-Aware Implicit Social Robot Navigation (HumAIN), a novel framework that fuses implicit social cues directly into the planning loop via knowledge distillation. We first employ a transformer-based teacher model that fuses rich multi-modal inputs, including historic images, skeletal keypoints, robot state, and a robot's target goal, to learn robust, human-aware representations for the robot's future trajectory planning. To enable real-time deployment, we then distill this knowledge into a lightweight student model. By optimizing for both trajectory reconstruction and latent feature alignment with the teacher, the student learns to infer complex social dynamics from minimal inputs. Bridging the prediction-planning gap with an efficient distilled architecture, our method enables robots to reason about human behavior in a manner that is adaptive, robust, and socially compliant. We validate HumAIN through extensive experiments, where it improves trajectory prediction metrics by an average of 29.8% across all metrics compared to state-of-the-art baselines. These results highlight the benefit of using implicit, whole-body cues to achieve human-like navigation awareness on resource-constrained platforms.

Chinese Translation

有效的社交机器人导航需要对人类行为的敏感性，这通常通过步态和方向等微妙的骨骼线索显现出来。我们提出了人类意识隐式社交机器人导航（HumAIN），这是一个新颖的框架，通过知识蒸馏将隐式社交线索直接融入规划循环中。我们首先采用基于变换器的教师模型，融合丰富的多模态输入，包括历史图像、骨骼关键点、机器人状态和机器人的目标，学习稳健的人类意识表示，以用于机器人的未来轨迹规划。为了实现实时部署，我们随后将这些知识蒸馏到一个轻量级的学生模型中。通过同时优化轨迹重建和与教师的潜在特征对齐，学生模型学习从最小输入中推断复杂的社交动态。我们的算法通过高效的蒸馏架构弥合了预测与规划之间的差距，使机器人能够以适应性、稳健性和社会合规的方式推理人类行为。我们通过大量实验验证了HumAIN，相较于最先进的基线，HumAIN在所有指标上平均提高了29.8%的轨迹预测指标。这些结果突显了在资源受限的平台上使用隐式全身线索以实现类人导航意识的优势。

View on arXiv Download PDF AI Translation

cs.RO / 30 / 2607.07370

Behavior Foundations for Quadruped Robots: ABot-C0 Technical Report

四足机器人行为基础：ABot-C0技术报告

Zhao, Xufeng, Yang, Fuzhi, Chen, Jianhui, Gao, Li, Meng, Zhang, Gao, Jie, Zheng, Yao, Liu, Wenyu, Yang, Menglin, Gu, Minqi, Zhao, Yaru, Han, Honglin, Su, Shihui, Tang, Zixiao, Liu, Liu, Xu, Mu, Cai, Yang, Tang, Wenbin

Abstract

In embodied intelligence systems, the motion controller serves as the critical bridge between semantic reasoning and physical execution. Humanoid control has progressed rapidly through large-scale human motion-capture data and motion-tracking paradigm. However, producing quadruped robots motion corpora with scalability and physical feasibility faces more fundamental obstacles: animal motion data is scarce, and cross-embodiment retargeting remains fragile. We present ABot-C0, a generalist motion-control system for quadruped robots that establishes three complementary behavior foundations: a scalable multi-source motion-data pipeline, robust policy learning across motion tracking, locomotion, and scene interaction, and a unified deployment stack for reliable real-world operation. Fundamentally, we construct a data pyramid through conditional video-generation synthesis, annotated motion capture, teleoperation and human design, producing 16,074 physically feasible motion clips as the data foundation for various motion learning demands. We then train a Flow-Matching generalist policy that demonstrates for the first time quadruped motion tracking scaling law that its performance improves consistently as training scales up, with zero-shot capability to track unseen motions. Then, we push a step further for robust all-terrain traversal locomotion by adopting a three-stage privileged-to-perceptive framework with temporal LiDAR memory and terrain-predictive supervision. Collectively, these components form a motion generalist that coordinates multi-policy execution, smooth behavior transitions, energy-efficient control, and safety mechanisms for real-world deployment. Extensive experiments on urban-terrain autonomous navigation and companion-style multimodal interaction demonstrate that quadruped robots move beyond single-function demos toward product-level behavioral intelligence.

Chinese Translation

在具身智能系统中，运动控制器作为语义推理与物理执行之间的关键桥梁。类人控制通过大规模的人体运动捕捉数据和运动跟踪范式迅速发展。然而，生成具有可扩展性和物理可行性的四足机器人运动数据集面临更根本的障碍：动物运动数据稀缺，跨体现重定向仍然脆弱。我们提出了ABot-C0，这是一个针对四足机器人的通用运动控制系统，建立了三个互补的行为基础：可扩展的多源运动数据管道、跨运动跟踪、运动和场景交互的稳健策略学习，以及用于可靠现实世界操作的统一部署堆栈。从根本上说，我们通过条件视频生成合成、注释运动捕捉、遥操作和人类设计构建了一个数据金字塔，产生了16,074个物理可行的运动片段，作为各种运动学习需求的数据基础。然后，我们训练了一个流匹配通用策略，首次展示了四足运动跟踪的可扩展性法则，即其性能随着训练规模的扩大而持续改善，并具备零样本能力来跟踪未见过的运动。接着，我们进一步推动了稳健的全地形穿越运动，通过采用具有时间LiDAR记忆和地形预测监督的三阶段特权到感知框架。综合来看，这些组件形成了一个运动通用体，协调多策略执行、平滑行为过渡、节能控制和现实世界部署的安全机制。在城市地形自主导航和伴侣式多模态交互的广泛实验中，证明了四足机器人超越单一功能演示，迈向产品级行为智能。

View on arXiv Download PDF AI Translation

cs.RO / 31 / 2607.07374

PLED-VINS: A Point-Line Event-Based Visual Inertial SLAM for Dynamic Environments

PLED-VINS：一种针对动态环境的基于点线事件的视觉惯性SLAM

Lee, Seunghun, Nam, Jihun, Seo, Dong-Uk, Myung, Hyun

Abstract

Dynamic environments remain a fundamental challenge for visual SLAM, where unreliable observations from moving objects and rapid motion degrade state estimation accuracy. Although event cameras preserve fine-grained spatio-temporal information, most existing event-based SLAM frameworks still assume static scenes and lack approaches to estimate the reliability of features. To this end, we propose PLED-VINS, a monocular event camera-based visual-inertial SLAM framework that enables robust state estimation in dynamic environments. We propose an entropy-recency score map to characterize the temporal reliability of both point and line features based on event temporal statistics. Concurrently, geometric reliability is estimated via a unified point-line robust bundle adjustment. Building upon these, we design an adaptive weighting strategy that fuses temporal and geometric reliability, including motion-conditioned reliability modeling for line features, to suppress unreliable observations. Experimental results demonstrate that PLED-VINS improves state estimation on the evaluated dynamic sequences with moving objects.

Chinese Translation

动态环境仍然是视觉SLAM面临的一个基本挑战，移动物体的不可靠观测和快速运动会降低状态估计的准确性。尽管事件相机能够保留细粒度的时空信息，但大多数现有的基于事件的SLAM框架仍假设场景是静态的，并且缺乏估计特征可靠性的方法。为此，我们提出了PLED-VINS，这是一种基于单目事件相机的视觉惯性SLAM框架，能够在动态环境中实现稳健的状态估计。我们提出了一种熵-近期得分图，用于基于事件时间统计特征来表征点和线特征的时间可靠性。同时，通过统一的点线鲁棒束调整来估计几何可靠性。在此基础上，我们设计了一种自适应加权策略，融合时间和几何可靠性，包括针对线特征的运动条件可靠性建模，以抑制不可靠的观测。实验结果表明，PLED-VINS在评估的动态序列中改善了移动物体的状态估计。

View on arXiv Download PDF AI Translation

cs.RO / 32 / 2607.07390

Communicative Efficiency of Single vs. Multi-Axis Robot Neck Motion

单轴与多轴机器人颈部运动的交流效率

Sirithunge, Chapa, Jeong, Haewon, Guan, Qinghua, Iida, Fumiya, Hughes, Josie

Abstract

Nonverbal communication through head and neck movement is fundamental to human social signalling, yet how robotic neck morphology translates motion into communicative information remains poorly understood. We present an information-theoretic framework characterising robot neck movement as a communication channel, quantifying information transmitted and energy expended across varied configurations. Using a robotic neck platform, we recorded 84 video stimuli spanning three rotational degrees of freedom (DoF), varying amplitude, acceleration, and frequency, measuring Shannon entropy of pixel-change signals alongside energy consumption. A perceptual study validated communicative interpretations of each motion. While humans typically engage one axis per gesture, robots are unconstrained by biological architecture, motivating tests up to 3 DoF. Yet communicative information peaks at two DoF and decreases at three despite rising energy cost, a phenomenon we term the morphological information bottleneck. Motion parameter effects were parameter-dependent, some additive, others non-linear. We introduce the Motor Information Space, a framework mapping entropy against energy to expose communicative efficiency across morphologies, in which the optimal configuration achieves 5.26 bits at competitive energy cost. Perception data further confirm multi-axis movements reduce clarity. These findings challenge the assumption that anatomical completeness improves robotic expressiveness, establishing a quantitative basis for morphological design in robots, especially humanoids.

Chinese Translation

通过头部和颈部运动进行的非语言交流是人类社会信号传递的基础，但机器人颈部形态如何将运动转化为交流信息仍然不够清楚。我们提出了一个信息论框架，将机器人颈部运动表征为一个通信通道，量化在不同配置下传递的信息和消耗的能量。利用一个机器人颈部平台，我们记录了84个视频刺激，涵盖了三个旋转自由度（DoF），变化了幅度、加速度和频率，同时测量了像素变化信号的香农熵以及能量消耗。一项感知研究验证了对每种运动的交流解释。尽管人类通常在每个手势中只使用一个轴，但机器人不受生物结构的限制，因此我们进行了多达3个自由度的测试。然而，交流信息在两个自由度时达到峰值，在三个自由度时尽管能量成本上升却下降，这一现象我们称之为形态信息瓶颈。运动参数的影响依赖于参数，有些是加性的，有些是非线性的。我们引入了运动信息空间（Motor Information Space），这是一个将熵与能量映射的框架，以揭示不同形态下的交流效率，其中最佳配置在具有竞争性能量成本的情况下达到了5.26比特。感知数据进一步确认多轴运动降低了清晰度。这些发现挑战了解剖完整性提高机器人表现力的假设，为机器人，特别是类人机器人中的形态设计建立了定量基础。

View on arXiv Download PDF AI Translation

cs.RO / 33 / 2607.07420

Initiation Safety: A Missing Dimension in Generalist-Robot Safety

启动安全性：通用机器人安全中的缺失维度

Meng, Zhijin, Cruz, Francisco

Abstract

Safety for generalist robots is usually discussed in terms of motion or dialogue. We argue a third question is missing: should the robot take its first hard-to-undo social action at all, such as a greeting, an uninvited grasp, or stepping into someone's space? We call this initiation authorization. Current frameworks rarely treat it as a separate safety layer. Today's stacks often skip this step: a high engagement score or a confident VLA rollout is treated as permission to act. But seeing a person is not the same as having their consent to be addressed. We frame initiation authorization within generalist-robot safety and contrast it with post-plan VLA guardrails, implementing PAS (probe-authorize-speak) on a doorway humanoid, comparing it with direct-init on logged traces, and proposing a three-condition user study, with open questions on metrics, governance, and where initiation ends and foundation-model generation begins.

Chinese Translation

通用机器人的安全性通常是通过运动或对话来讨论的。我们认为缺少一个第三个问题：机器人是否应该进行其首个难以撤回的社会行为，例如问候、未经邀请的抓握或进入他人的私人空间？我们称之为启动授权。目前的框架很少将其视为一个独立的安全层。如今的技术栈往往跳过这一步骤：高参与度评分或自信的 VLA（价值-学习-行动）推出被视为行动的许可。但看到一个人并不等同于获得他们的同意进行交流。我们将启动授权框架置于通用机器人安全的背景下，并将其与计划后的 VLA 保护措施进行对比，在一个门口的人形机器人上实施 PAS（探测-授权-发言），并将其与记录的轨迹中的直接启动进行比较，提出一个包含三项条件的用户研究，并对指标、治理以及启动的结束与基础模型生成的开始提出开放性问题。

View on arXiv Download PDF AI Translation

cs.RO / 34 / 2607.07430

Immersive Social Interaction with VR and LLM-Assisted Humanoids

虚拟现实与大型语言模型辅助的人形机器人沉浸式社交互动

Pudasaini, Niraj, Bethala, Geeta Chandra Raju, Doma, Pranav, Tzes, Anthony, Fang, Yi

Abstract

Humanoid robots can extend human presence to remote, constrained, or hazardous environments, but existing teleoperation interfaces often require physically demanding motion tracking or cognitively demanding low-level control. This paper presents an immersive teleoperation framework that integrates voice-controlled locomotion, VR-based manipulation, and bidirectional social interaction for whole-body humanoid control. Using Apple Vision Pro, the operator receives egocentric visual feedback, issues natural-language locomotion commands, and teleoperates the robot's arms and dexterous hands through wrist and finger tracking. An LLM-assisted voice-control module converts spoken instructions into high-level locomotion commands, while the manipulation module retargets human hand motions to the robot through inverse kinematics and PD control. The system also records multimodal data, including egocentric RGB observations, voice/text commands, joint states, hand motions, and eye-gaze signals, supporting future imitation learning and autonomy. We evaluate the framework on a Unitree H1 humanoid equipped with dexterous hands in manipulation and social interaction tasks. Results show that novice users can successfully operate the system after brief familiarization, achieving 80\% success in object manipulation and 70\% success in a social cube-passing task. These results demonstrate the potential of immersive, language-assisted teleoperation as an accessible interface for humanoid interaction, remote assistance, and multimodal data collection.

Chinese Translation

人形机器人可以将人类的存在扩展到遥远、受限或危险的环境中，但现有的远程操作接口通常需要身体上要求较高的运动跟踪或认知上要求较高的低级控制。本文提出了一种沉浸式远程操作框架，该框架集成了语音控制的移动、基于虚拟现实的操控以及双向社交互动，以实现全身人形机器人的控制。使用 Apple Vision Pro，操作员可以接收自我中心的视觉反馈，发出自然语言的移动指令，并通过手腕和手指跟踪远程操作机器人的手臂和灵巧手。一个大型语言模型（LLM）辅助的语音控制模块将口头指令转换为高级移动命令，而操控模块则通过逆向运动学和比例微分（PD）控制将人类手部动作重新定向到机器人上。该系统还记录多模态数据，包括自我中心的 RGB 观察、语音/文本命令、关节状态、手部动作和眼动信号，支持未来的模仿学习和自主性。我们在一台配备灵巧手的 Unitree H1 人形机器人上评估了该框架在操控和社交互动任务中的表现。结果表明，初学者在简短的熟悉后能够成功操作该系统，在物体操控中实现了 80\% 的成功率，在社交立方体传递任务中实现了 70\% 的成功率。这些结果展示了沉浸式、语言辅助的远程操作作为人形机器人互动、远程协助和多模态数据收集的可访问接口的潜力。

View on arXiv Download PDF AI Translation

cs.RO / 35 / 2607.07452

GeoGS-SLAM: Geometry-Only Gaussian Splatting for Dense Monocular SLAM

GeoGS-SLAM：仅基于几何的高斯点云稀疏化用于密集单目SLAM

Zhou, Lipu, Kang, Yaoyun, Pang, Junxiang, Sun, Shengkai, Bao, Tingting, Wang, Kehan

Abstract

Dense visual SLAM is a fundamental problem in robotics. Recent advances in 3DGS have demonstrated its potential for dense SLAM. Existing 3DGS frameworks focus on both appearance and geometry modeling. However, scene geometry is typically more critical for SLAM than novel view synthesis because downstream robotic tasks, such as navigation and obstacle avoidance, rely primarily on accurate spatial geometry rather than photorealistic rendering. This observation raises a natural question: Is it feasible for 3DGS to perform 3D reconstruction without scene appearance modeling? Motivated by this, we propose Geometry-only Gaussian Splatting (GeoGS), which directly reconstructs scene geometry, and further present GeoGS-SLAM, a dense visual SLAM system built upon this representation. Specifically, GeoGS retains only spatial parameters to reduce the number of per-primitive parameters by over 80%. In contrast to existing 3DGS methods, GeoGS focuses solely on geometric reconstruction, which significantly reduces the number of Gaussian primitives, accelerates geometric convergence, and enhances robustness to illumination variations. In addition, we present an effective training framework that optimizes the Gaussian primitives via single-view and multi-view geometric and photometric supervision, and speeds up geometry convergence with a local-plane driven initialization that better aligns primitives with local structures. Furthermore, we introduce a map update strategy for loop closure that globally transforms the Gaussian map to align it with the corrected pose estimates, thereby preventing map tearing caused by inconsistent per-viewpoint pose corrections in existing methods. Extensive experiments on synthetic and real-world benchmarks demonstrate that our method outperforms SOTA methods in terms of online mapping efficiency and geometric reconstruction quality.

Chinese Translation

密集视觉SLAM是机器人领域的一个基本问题。最近在3DGS（3D高斯稀疏化）方面的进展展示了其在密集SLAM中的潜力。现有的3DGS框架关注于外观和几何建模。然而，场景几何通常对SLAM比新视图合成更为关键，因为下游机器人任务，如导航和避障，主要依赖于准确的空间几何，而非真实感渲染。这一观察引出了一个自然的问题：3DGS是否可以在不进行场景外观建模的情况下执行3D重建？基于此，我们提出了仅基于几何的高斯点云稀疏化（GeoGS），该方法直接重建场景几何，并进一步提出了基于此表示的密集视觉SLAM系统GeoGS-SLAM。具体而言，GeoGS仅保留空间参数，从而将每个原始参数的数量减少了超过80%。与现有的3DGS方法相比，GeoGS仅专注于几何重建，这显著减少了高斯原始体的数量，加速了几何收敛，并增强了对光照变化的鲁棒性。此外，我们提出了一个有效的训练框架，通过单视图和多视图的几何与光度监督来优化高斯原始体，并通过局部平面驱动的初始化加速几何收敛，更好地将原始体与局部结构对齐。此外，我们引入了一种用于回环闭合的地图更新策略，该策略全局变换高斯地图以使其与修正的位姿估计对齐，从而防止现有方法中由于不一致的每视点位姿修正而导致的地图撕裂。在合成和真实世界基准上的大量实验表明，我们的方法在在线映射效率和几何重建质量方面优于现有的最先进方法（SOTA）。

View on arXiv Download PDF AI Translation

cs.RO / 36 / 2607.07459

EmbodiedGen V2: An Agentic, Simulation-Ready 3D World Engine for Embodied AI

EmbodiedGen V2：一个适用于具身人工智能的代理式、模拟准备的3D世界引擎

Wang, Xinjie, Liu, Liu, Ding, Taojun, Choi, Andrew, Huang, Chaodong, Zhao, Mengao, Li, Ziang, Jiang, Jackson, Yu, Chunlei, Liu, Shengxiang, Xu, Wei, Su, Zhizhong

Abstract

We present EmbodiedGen V2, a generative 3D world engine for building executable sim-ready environments for embodied intelligence. Sim-ready 3D asset generation has advanced rapidly, yet assembling such assets into policy-ready task environments remains largely manual, limiting scalable closed-loop learning. EmbodiedGen V2 addresses this gap through a unified sim-ready representation that connects cross-simulator assets, interaction affordances, task-driven worlds, large-scale multi-room scenes, and stateful Vibe Coding into a generative, editable, and reusable simulation pipeline. The generated environments support manipulation, navigation, mobile manipulation, cross-simulator deployment, and embodied policy training. In evaluation, the asset pipeline achieves 96.5% human acceptance and 98.6% collision success, and 83.3% of task-driven worlds are directly usable for downstream simulation without manual modification. Online reinforcement learning with generated environments further improves simulation success from 9.7% to 79.8%, and transfers to real robots with task success increasing from 21.7% to 75.0%. These results establish EmbodiedGen V2 as scalable simulation infrastructure for training, evaluating, and deploying embodied policies.

Chinese Translation

我们提出了EmbodiedGen V2，一个用于构建可执行的模拟准备环境的生成性3D世界引擎，旨在支持具身智能的发展。模拟准备的3D资产生成已经快速发展，但将这些资产组装成政策准备的任务环境仍然主要依赖人工操作，这限制了可扩展的闭环学习。EmbodiedGen V2通过统一的模拟准备表示来填补这一空白，该表示连接了跨模拟器资产、交互可用性、任务驱动的世界、大规模多房间场景和有状态的Vibe Coding，形成一个生成性、可编辑和可重用的模拟管道。生成的环境支持操作、导航、移动操作、跨模拟器部署和具身政策训练。在评估中，资产管道实现了96.5%的人工接受度和98.6%的碰撞成功率，83.3%的任务驱动世界可以直接用于下游模拟而无需人工修改。使用生成环境的在线强化学习进一步将模拟成功率从9.7%提高到79.8%，并且在真实机器人上的任务成功率从21.7%提高到75.0%。这些结果确立了EmbodiedGen V2作为可扩展的模拟基础设施，用于训练、评估和部署具身政策。

View on arXiv Download PDF AI Translation

cs.RO / 37 / 2607.07475

Agent-Exploitation Affordances: From Basic to Complex Representation Patterns

代理-利用可供性：从基本到复杂的表征模式

Dussard, Bastien, Clodic, Aurélie, Sarthou, Guillaume

Abstract

In robotics, the capability of an artificial agent to represent the range of its action possibilities, i.e. affordances, is crucial to understand how it can act on its environment. While functional affordances, which refer to the use of tools and objects, have been broadly studied in knowledge representation, the implications of a social context and the presence of other agents have remained unexplored in this field. Consequently, in the field of social robotics, a multi-agent context enables the agents to engage in new actions that are potentially complementary to their individual capabilities, leading to the perspective of agentexploitation. This work focuses on the concept of cooperative affordance within the realm of social affordances. Cooperative affordances refer to situations where agents interact with each other to extend their action possibilities range. From this definition, this paper proposes a tractable ontological representation of this concept with the aim of making it usable by an artificial agent. Expanding on those elementary patterns, we illustrate the effectiveness of these representations by combining them to depict a diverse range of scenarios.

Chinese Translation

在机器人学中，人工代理表示其行动可能性范围的能力，即可供性，对于理解其如何与环境互动至关重要。虽然功能性可供性（functional affordances），即工具和物体的使用，已在知识表示领域得到了广泛研究，但社会背景及其他代理的存在对这一领域的影响仍未被探讨。因此，在社会机器人学领域，多代理环境使得代理能够进行新的行动，这些行动可能与其个体能力互补，从而引出代理利用（agent-exploitation）的视角。本研究聚焦于社会可供性领域中的合作可供性（cooperative affordance）概念。合作可供性指的是代理之间相互作用以扩展其行动可能性范围的情境。基于这一定义，本文提出了一种可处理的本体表示，以期使其可被人工代理使用。在这些基本模式的基础上，我们通过将其结合来描绘多样化场景，从而展示这些表示的有效性。

View on arXiv Download PDF AI Translation

cs.RO / 38 / 2607.07491

Smooth Operator: A Real-Time Sampling-Based Algorithm for Kinematic Hand Retargeting

平滑操作员：一种用于运动手重定向的实时基于采样的算法

Malate, Robert Jomar, Bauer, Erik, Bacuieti, Norica, Charalambous, Stefanos, Nava, Elvis, Katzschmann, Robert K., Forrai, Benedek

Abstract

Advances in learning-based robotic manipulation, such as Vision-Language-Action (VLA) models and Video Action Models (VAMs), heavily rely on high-quality teleoperation data. Their capabilities are strictly upper-bounded by the quality of the underlying human demonstrations. Current gradient-based retargeting algorithms often converge to different local minima, resulting in jitter that affects data quality and teleoperation experience. To address this, we introduce the Sampling-Based Retargeter (SBR), a novel gradient-free retargeting method drawn from the rich literature of sampling-based control and explicitly designed for low-jitter, real-time kinematic retargeting. We evaluate SBR both in simulation and through a rigorous real-world user study involving 18 participants performing 3 complex manipulation tasks. Compared to gradient-based baselines, SBR achieved the highest overall task success rate (54.1%) while significantly reducing operator cognitive fatigue, recording the lowest NASA-TLX workload score (36.4 out of 100). Ultimately, we establish SBR as a highly effective, intuitive retargeter for dexterous manipulation, providing the community with a rigorous benchmarking methodology to guide future retargeting research.

Chinese Translation

基于学习的机器人操作技术的进步，如视觉-语言-动作（Vision-Language-Action, VLA）模型和视频动作模型（Video Action Models, VAMs），在很大程度上依赖于高质量的遥操作数据。它们的能力严格受限于基础人类示范的质量。目前的基于梯度的重定向算法往往收敛到不同的局部最小值，导致抖动，从而影响数据质量和遥操作体验。为了解决这个问题，我们提出了基于采样的重定向器（Sampling-Based Retargeter, SBR），这是一种新颖的无梯度重定向方法，源于丰富的基于采样的控制文献，并专门设计用于低抖动、实时的运动重定向。我们在模拟环境中以及通过一项严格的现实世界用户研究评估了SBR，该研究涉及18名参与者执行3个复杂的操作任务。与基于梯度的基线相比，SBR实现了最高的整体任务成功率（54.1%），同时显著降低了操作员的认知疲劳，记录了最低的NASA-TLX工作负荷评分（36.4分，满分100分）。最终，我们确立了SBR作为一种高效、直观的灵巧操作重定向器，为社区提供了一种严格的基准测试方法，以指导未来的重定向研究。

View on arXiv Download PDF AI Translation

cs.RO / 39 / 2607.07533

Generating Personalized Lower-Limb Kinematics Across Walking Speeds Using Subject-Conditioned Diffusion

利用受试者条件扩散生成个性化的下肢运动学在不同步态速度下的应用

Dinesh, Diya, Krieger, Adrian, Song, Changseob, Park, Dongho, Young, Aaron J., Kang, Inseung

Abstract

Personalizing exoskeleton assistance requires user-specific gait data across many locomotor tasks, yet collecting this data demands repeated motion capture sessions that are costly, time-intensive, and especially burdensome for clinical populations. This challenge is most acute across walking speeds, where gait changes substantially and deviates further in clinical gait. This work introduces a subject-conditioned residual diffusion framework that generates personalized lower-limb kinematics at unseen walking speeds from a subject's gait sequence at a single seen speed. Given sagittal-plane hip, knee, and ankle trajectories at a seen speed and a desired unseen speed, the model generates a residual that transforms the seen trajectory into the unseen one, using a transformer denoiser conditioned on the subject's gait and the two speeds through feature-wise linear modulation. Trained only on able-bodied data, the model achieved a mean absolute error (MAE) of 3.4{\deg} on held-out able-bodied subjects. Without any stroke-specific fine-tuning, it achieved a 6.0{\deg} MAE on out-of-training-distribution stroke subjects, retaining subject identity for clinical gait. The framework reduced the MAE by over 70% relative to supervised feed-forward baselines, and a single seen speed matched the accuracy of four speeds within 0.4{\deg}. These results demonstrate that subject-conditioned residual diffusion can synthesize personalized gait across speeds from minimal data, reducing the collection burden for downstream exoskeleton personalization.

Chinese Translation

个性化外骨骼辅助需要在多种运动任务中收集用户特定的步态数据，但收集这些数据需要反复进行运动捕捉，这既昂贵又耗时，尤其对临床人群来说负担更重。步态在不同步态速度下变化显著，尤其在临床步态中，这一挑战尤为突出。本研究提出了一种受试者条件的残差扩散框架，该框架能够根据受试者在单一已知速度下的步态序列生成在未见步态速度下的个性化下肢运动学。给定在已知速度下的矢状面髋关节、膝关节和踝关节轨迹及所需的未见速度，模型生成一个残差，将已知轨迹转化为未见轨迹，使用基于受试者步态和两种速度的特征线性调制的变换器去噪器。该模型仅在健康人群数据上训练，在保留的健康受试者中达到了3.4°的平均绝对误差（MAE）。在没有任何针对中风特定的微调的情况下，它在训练分布外的中风受试者中达到了6.0°的MAE，保持了临床步态的受试者身份。该框架相较于监督前馈基线将MAE减少了超过70%，且在单一已知速度下的准确性与四个速度的准确性相差不超过0.4°。这些结果表明，受试者条件的残差扩散可以从最少的数据中合成个性化步态，减轻下游外骨骼个性化的收集负担。

View on arXiv Download PDF AI Translation

cs.RO / 40 / 2607.07542

SonoRank: Towards Calibration-Free Real-Time Finger Flexion Detection from Forearm Ultrasound Sequences

SonoRank：实现无校准实时前臂超声序列中的手指屈曲检测

Zadok, Dean, Wolf, Alon, Bronstein, Alex M., Salzman, Oren

Abstract

Powered prosthetic hands are frequently abandoned, largely due to the limited functionality of current devices that rely on surface electromyography (sEMG). Sonomyography (ultrasound) has emerged as a promising alternative, owing to its ability to observe muscle activity in real time and control a greater number of degrees of freedom. Yet, existing ultrasound-based methods require per-user fine-tuning, limiting their commercialization. We propose SonoRank, an important step towards calibration-free finger flexion detection from forearm ultrasound video. SonoRank first learns to rank pairs of ultrasound sequences by their relative motion magnitude for each of the five fingers. The learned representations are then fine-tuned to classify whether each finger is actively flexing, using a rest reference that is captured at the beginning of the operation. Under 12-fold leave-one-subject-out cross-validation on a dataset of twelve subjects with synchronized kinematics, SonoRank achieves a 28% improvement in F1 score over direct classification baselines that skip the ranking stage. These results establish pairwise ranking as an effective pretraining signal for subject-independent control, bringing ultrasound-based prosthetics closer to practical, calibration-free deployment.

Chinese Translation

动力假肢手常常被弃用，主要是由于当前依赖表面肌电图（sEMG）的设备功能有限。声肌图（超声）因其能够实时观察肌肉活动并控制更多的自由度而成为一种有前景的替代方案。然而，现有的基于超声的方法需要针对每个用户进行微调，这限制了它们的商业化。我们提出了SonoRank，这是实现无校准前臂超声视频手指屈曲检测的重要一步。SonoRank首先学习通过相对运动幅度对五个手指的超声序列进行配对排序。然后，将学习到的表示进行微调，以分类每个手指是否在主动屈曲，使用在操作开始时捕获的静息参考。在对包含十二个受试者的同步运动学数据集进行12折留一法交叉验证时，SonoRank在F1分数上比跳过排序阶段的直接分类基线提高了28%。这些结果确立了成对排序作为一种有效的预训练信号，用于实现与受试者无关的控制，使基于超声的假肢更接近于实际的无校准部署。

View on arXiv Download PDF AI Translation

cs.RO / 41 / 2607.07574

Context-Aware Force Estimation for Deformable Tool Manipulation in Robotic Environmental Swabbing via Few-Shot Continual Adaptation

基于上下文的可变形工具操作中的接触力估计：通过少量样本持续适应的机器人环境拭擦

Mahmoudi, Siavash, Reddy, Chaitainya Kuppar, Tian, Yang, Wang, Dongyi

Abstract

Robotic surface swabbing requires sustained interaction between a compliant tool and heterogeneous environments, where accurate estimation of tip-level contact force is critical for consistent sampling performance. However, deformable tool dynamics introduce nonlinear viscoelastic hysteresis that decouples wrist-mounted force measurements from true contact forces, while tool-integrated sensors are impractical for deployment due to sterility and disposability constraints. This paper presents a data-driven framework for contact force estimation in Deformable Tool Manipulation (DTM) that leverages proprioceptive sensing without requiring explicit physical models or permanent embedded sensing hardware at the tool tip. A recurrent architecture is first identified through a comparative evaluation of temporal models, where a compact LSTM achieves the lowest estimation error and sub-millisecond inference latency. To address generalization across unseen surfaces and tool compliance conditions, we introduce a parameter-isolated few-shot adaptation strategy that augments a frozen recurrent backbone with low-dimensional context embeddings using feature-wise linear modulation (FiLM). Experiments on a UR5e platform across nine tool-surface interaction regimes demonstrate that the proposed approach significantly improves robustness under domain shift, reducing zero-shot estimation error by up to 63\% while preserving baseline performance without catastrophic forgetting. These results show that separating shared deformation-history dynamics from domain-specific conditioning enables reliable force estimation for DTM in non-stationary environments.

Chinese Translation

机器人表面拭擦需要合规工具与异质环境之间的持续互动，其中准确估计尖端接触力对于一致的采样性能至关重要。然而，可变形工具的动态引入了非线性粘弹性滞后，这使得腕部安装的力测量与真实接触力脱耦，同时由于无菌性和一次性限制，工具集成传感器的部署并不实际。本文提出了一种数据驱动的接触力估计框架，适用于可变形工具操作（Deformable Tool Manipulation, DTM），该框架利用本体感知传感而无需在工具尖端要求显式的物理模型或永久嵌入的传感硬件。通过对时间模型的比较评估，首先确定了一种递归架构，其中紧凑型长短期记忆网络（LSTM）实现了最低的估计误差和亚毫秒的推理延迟。为了应对未见表面和工具合规条件下的泛化问题，我们引入了一种参数隔离的少量样本适应策略，该策略使用特征线性调制（Feature-wise Linear Modulation, FiLM）增强了冻结的递归主干与低维上下文嵌入的结合。在UR5e平台上进行的九种工具-表面交互模式的实验表明，所提出的方法在领域转移下显著提高了鲁棒性，将零样本估计误差降低了多达63%，同时在不发生灾难性遗忘的情况下保持了基线性能。这些结果表明，将共享的变形历史动态与领域特定的条件分离，使得在非静态环境中进行DTM的可靠力估计成为可能。

View on arXiv Download PDF AI Translation

cs.RO / 42 / 2607.07601

CARLA-GS: Decoupling Representation, Reasoning, and Physics Simulation for Autonomous Driving Corner-Case Synthesis

CARLA-GS：解耦表示、推理与物理仿真以合成自动驾驶边缘案例

Huang, Kaicong, Ma, Meng, Ke, Ruimin

Abstract

Safety evaluation for autonomous driving is dominated by rare, safety-critical interactions, motivating simulators that can deliberately synthesize corner cases with photorealistic observations. Corner-case generation is inherently a multi-source problem spanning visual representation, scene reasoning, and vehicle trajectory generation and control. Prior knowledge- and model-based approaches typically focus on scene or trajectory components in isolation, while diffusion-based methods attempt end-to-end generation but still struggle to ensure spatiotemporal consistency and physical realism. To unify these aspects within a single framework, we propose CARLA-GS, a modular corner-case synthesis pipeline that decouples visual representation, semantic reasoning, and physics-based execution while maintaining tight cross-module coupling. Starting from real driving data, we reconstruct an editable gaussian scene with additional geometry-consistent constraints. A multi-agent LLM then performs scene-level reasoning to identify risky interactions and generate intent-level waypoint trajectories, while the low-level motion control is delegated to CARLA, where a PID controller ensures kinematic and dynamic feasibility. The simulated vehicle states are finally re-projected into the gaussian scene for ego-centric rendering. This design enables high-level semantic reasoning, low-level physically executable motion, and photorealistic corner-case generation within a unified pipeline. Experiments on the Waymo Open Dataset show, both quantitatively and qualitatively, that our framework enables controllable corner-case generation and produces photorealistic, spatiotemporally consistent videos aligned with semantic intent and physically feasible motion.

Chinese Translation

自动驾驶的安全评估主要集中在稀有的、与安全相关的交互上，这促使开发能够故意合成具有照片真实感观察的边缘案例的仿真器。边缘案例生成本质上是一个跨越视觉表示、场景推理和车辆轨迹生成与控制的多源问题。以往的基于知识和模型的方法通常孤立地关注场景或轨迹组件，而基于扩散的方法则尝试端到端生成，但仍然难以确保时空一致性和物理现实性。为了在一个统一的框架内整合这些方面，我们提出了CARLA-GS，一个模块化的边缘案例合成管道，它解耦了视觉表示、语义推理和基于物理的执行，同时保持紧密的跨模块耦合。我们从真实驾驶数据出发，重建一个可编辑的高斯场景，并增加几何一致性约束。然后，一个多智能体的LLM执行场景级推理，以识别风险交互并生成意图级的航点轨迹，而低级运动控制则委托给CARLA，由PID控制器确保运动的运动学和动态可行性。最后，模拟的车辆状态被重新投影到高斯场景中以进行自我中心渲染。该设计使得在统一管道内实现高层次的语义推理、低层次的物理可执行运动以及照片真实感的边缘案例生成成为可能。在Waymo开放数据集上的实验表明，无论是定量还是定性，我们的框架都能够实现可控的边缘案例生成，并生成与语义意图和物理可行运动一致的照片真实感、时空一致的视频。

View on arXiv Download PDF AI Translation

cs.RO / 43 / 2607.07608

Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulation

用于机器人操作的视觉-语言-行动模型中的双潜在记忆

Qu, Hongyu, Gao, Jianzhe, Hu, Xiaobin, Yang, Shaohuan, Yu, Xinlei, Yan, Rui, Wang, Wenguan, Shu, Xiangbo, Yan, Shuicheng

Abstract

Mainstream Vision-Language-Action (VLA) models predict actions primarily from the current observation under a Markovian assumption, thus struggling with long-horizon, temporally dependent tasks. Existing memory-augmented VLAs either expand the observation window or retrieve history from the memory bank as auxiliary policy-side context. However, they leave memory outside the native latent embedding space of VLA reasoning, preventing historical experience from being fluidly interleaved with multimodal reasoning and action formation. To this end, we introduce LaMem-VLA, a latent-memory-native framework that reconstructs historical experience into latent memory tokens and directly interweaves them with VLA reasoning. At its core, LaMem-VLA introduces four coordinated components: (i) a curator that organizes historical experience into two complementary short-term and long-term memory vaults; (ii) a seeker that queries both vaults using the multimodal cognition to retrieve context-relevant evidence; (iii) a condenser that reconstructs the retrieved evidence into compact short-term and long-term latent memory tokens; and (iv) a weaver that injects these memory tokens with the current observation and instruction into one continuous embedding sequence. By representing, retrieving, and consuming historical experience entirely in the same continuous latent space, LaMem-VLA enables memory to directly participate in VLA reasoning and guide action generation under a bounded context. Extensive experiments on SimplerEnv and LIBERO demonstrate the superiority of our LaMem-VLA.

Chinese Translation

主流的视觉-语言-行动（VLA）模型主要基于马尔可夫假设从当前观察中预测动作，因此在长时间跨度和时间依赖的任务中表现不佳。现有的增强记忆的VLA要么扩展观察窗口，要么从记忆库中检索历史作为辅助策略上下文。然而，它们将记忆置于VLA推理的原生潜在嵌入空间之外，阻碍了历史经验与多模态推理和动作形成的流畅交织。为此，我们提出了LaMem-VLA，这是一种潜在记忆原生框架，将历史经验重构为潜在记忆标记，并将其直接与VLA推理交织在一起。LaMem-VLA的核心引入了四个协调组件：（i）一个策展者，将历史经验组织成两个互补的短期和长期记忆库；（ii）一个探寻者，使用多模态认知查询两个记忆库，以检索与上下文相关的证据；（iii）一个凝聚者，将检索到的证据重构为紧凑的短期和长期潜在记忆标记；（iv）一个编织者，将这些记忆标记与当前观察和指令注入到一个连续的嵌入序列中。通过在同一连续潜在空间中完全表示、检索和利用历史经验，LaMem-VLA使得记忆能够直接参与VLA推理，并在有限的上下文中指导动作生成。在SimplerEnv和LIBERO上的大量实验表明，我们的LaMem-VLA具有优越性。

View on arXiv Download PDF AI Translation

cs.RO / 44 / 2607.07622

Continuous and large-scale: ELEANOR, the soft architected arm inspired by the elephant trunk

连续且大规模：ELEANOR，受大象鼻子启发的柔性结构臂

Naselli, Giovanna A., Nardin, Anderson B., Joe, Seonggun, Drinkwater, Ryan, Donato, Enrico, Bianchi, Diego, Falotico, Egidio, Milinkovitch, Michel C., Beccai, Lucia

Abstract

The elephant trunk is a dexterous and versatile manipulator whose performance is still unmatched in robotics. In previous works, modularity was prioritized and relatively small-scale continuum robots were built. We take the natural proboscis of the *Loxodonta africana* species as a model and propose a different design approach which favors structural continuity and dynamic properties that plausibly emulate those of the natural trunk, while conferring high adaptability to the environment and humans. Instead of targeting prescribed behaviors, we show that a biomimetic design based on the macroscopic properties of the natural system enables elephant-like movements and grasping. We build by 3D printing an 85 cm long, compliant, tapered, volumetrically tessellated continuum arm, which is combined with tendon-driven actuation mimicking the longitudinal and oblique muscles of the natural model. We demonstrate whole-body grasping of objects having different shapes and dimensions and discuss a comparison to the biological trunk highlighting aspects of both biology and robotics.

Chinese Translation

大象的鼻子是一种灵巧且多功能的操作器，其性能在机器人领域仍然无与伦比。在之前的研究中，模块化被优先考虑，构建了相对小规模的连续机器人。我们以非洲象（*Loxodonta africana*）的自然鼻子为模型，提出了一种不同的设计方法，强调结构的连续性和动态特性，这些特性可以合理地模拟自然鼻子的特性，同时赋予其对环境和人类的高度适应性。我们展示了一种基于自然系统宏观特性的仿生设计，使得机器人能够实现类似大象的运动和抓取。我们通过3D打印构建了一种85厘米长的柔性、锥形、体积镶嵌的连续臂，并结合了模仿自然模型纵向和斜向肌肉的腱驱动激励。我们展示了对不同形状和尺寸物体的全身抓取，并讨论了与生物鼻子的比较，突出了生物学和机器人学的各个方面。

View on arXiv Download PDF AI Translation

计算机视觉 (Computer Vision)

cs.CV / 1 / 2607.06585

Pixel-Precise Explainable Stress Indexing: A Semantic Segmentation Framework for Disease Severity Quantification in Field Crops

像素级可解释压力指数：一种用于田间作物病害严重性量化的语义分割框架

Kumar, Raunak, Kar, Soumyashree

Abstract

Plant diseases, resulting from both biotic and abiotic stresses, cause an estimated 20-40% loss in global agricultural yield annually, resulting in economic damages exceeding USD 220 billion. Accurate and scalable stress quantification is essential for precision agriculture, yet traditional manual assessments are labour-intensive and subjective. This paper proposes a unified deep learning pipeline integrating semantic segmentation, regression-based severity estimation, and disease classification. Stress severity is categorised into four levels (Low to Very High) based on the proportion of infected leaf area. Experiments on the Apple Tree Leaf Disease Segmentation dataset (1,641 samples, six classes) evaluate four models: U-Net (MobileNetV2), SegFormer, FCN, and PSPNet. U-Net with MobileNetV2 achieves the best performance with 98.20% pixel accuracy, 0.70 mIoU, and 99.41% detection accuracy at 14.7 ms per image, making it suitable for real-time use. SegFormer performs competitively (mIoU 0.66), while FCN and PSPNet show lower spatial accuracy (approximately 0.49 mIoU). The computed severity index strongly correlates with expert annotations (r = 0.968, R^2 = 0.937), demonstrating the system's reliability for automated crop monitoring and decision support.

Chinese Translation

植物病害由于生物和非生物压力的影响，每年导致全球农业产量损失估计在20%到40%之间，经济损失超过2200亿美元。准确且可扩展的压力量化对于精准农业至关重要，然而传统的人工评估劳动密集且主观。本文提出了一种统一的深度学习管道，集成了语义分割、基于回归的严重性估计和病害分类。根据感染叶面积的比例，压力严重性被分为四个级别（低到非常高）。在苹果树叶病害分割数据集（1,641个样本，六个类别）上进行的实验评估了四种模型：U-Net（MobileNetV2）、SegFormer、FCN和PSPNet。使用MobileNetV2的U-Net模型在每张图像14.7毫秒的时间内实现了最佳性能，像素准确率达到98.20%，mIoU为0.70，检测准确率为99.41%，使其适合实时使用。SegFormer的表现具有竞争力（mIoU为0.66），而FCN和PSPNet的空间准确性较低（约为0.49 mIoU）。计算出的严重性指数与专家标注之间存在很强的相关性（r = 0.968，R^2 = 0.937），证明了该系统在自动化作物监测和决策支持中的可靠性。

View on arXiv Download PDF AI Translation

cs.CV / 2 / 2607.06587

CoFINN: Conservation Flux Informed Neural Networks for Physics Problems Governed by Conservation Laws

CoFINN：基于守恒通量的物理问题神经网络

Doğan, Adnan Harun, Deniz, Mert, Alemdar, Hande, Baran, Özgür Uğraş

Abstract

We present CoFINN (Conservation Flux Informed Neural Networks), a physics-informed deep learning framework for predicting compressible flow fields governed by conservation laws. Unlike conventional data-driven convolutional neural networks (CNNs), which optimize only pixel-wise similarity metrics, CoFINN embeds finite-volume conservation physics directly into the training process. Unlike classical physics-informed methods which enforce differential-equation residuals at collocation points through automatic differentiation, CoFINN adopts a finite-volume perspective consistent with modern CFD methodology. CoFINN interprets CNN output fields as structured computational grids, where each pixel represents a finite-volume cell, and enforces conservation consistency through sophisticated numerical flux calculations. The framework is evaluated on transonic flow prediction around airfoils at (M=0.7, Re=6 * 10^6), including challenging conditions involving shock waves and high angles of attack. Results show that CoFINN improves aerodynamic force prediction accuracy, reducing drag prediction error by up to 34% at extreme angles of attack and by approximately 15% on average across the test set. Improvements are particularly significant in limited-data regimes, demonstrating that the conservation-based loss acts as an effective physical regularizer. The proposed approach maintains the computational efficiency advantages of CNN surrogates while significantly improving physical consistency and conservation behavior. The framework is architecture-agnostic and extensible to broader classes of conservation-law-governed physical systems.

Chinese Translation

我们提出了CoFINN（基于守恒通量的物理信息神经网络），这是一种用于预测受守恒定律支配的可压缩流场的物理信息深度学习框架。与传统的数据驱动卷积神经网络（CNN）仅优化像素级相似性度量不同，CoFINN将有限体积守恒物理直接嵌入训练过程中。与经典的物理信息方法通过自动微分在配点上强制执行微分方程残差不同，CoFINN采用与现代计算流体动力学（CFD）方法一致的有限体积视角。CoFINN将CNN输出场解释为结构化计算网格，其中每个像素代表一个有限体积单元，并通过复杂的数值通量计算来强制执行守恒一致性。该框架在跨音速流动预测（M=0.7，Re=6 * 10^6）中进行了评估，包括涉及冲击波和高攻角的挑战性条件。结果表明，CoFINN提高了气动力预测的准确性，在极高攻角下减少了多达34%的阻力预测误差，并在测试集上平均减少了约15%的误差。在有限数据环境下，改进尤为显著，表明基于守恒的损失函数作为有效的物理正则化器。所提出的方法保持了CNN替代模型的计算效率优势，同时显著提高了物理一致性和守恒行为。该框架不依赖于特定架构，并可扩展到更广泛的受守恒定律支配的物理系统。

View on arXiv Download PDF AI Translation

cs.CV / 3 / 2607.06590

AI for Cultural Heritage Textiles: Fine-Tuned Latent Diffusion for Novel Ulos Motif Synthesis

文化遗产纺织品的人工智能：针对新型乌洛斯图案合成的精细调优潜在扩散模型

Simanjuntak, Humasak Tommy Argo, Purba, Jesika, Girsang, Sitogab, Manurung, Widya, Situmeang, Samuel, Barus, Arlinta, Siahaan, Daniel Oranova

Abstract

Preserving and revitalising traditional textiles such as Ulos, a cultural heritage of the Batak ethnic group in North Sumatra, Indonesia, requires balancing fidelity to tradition with innovative approaches that meet contemporary design demands. Traditional Ulos weaving faces two key limitations: a narrow range of motifs and a time-intensive design process. This study presents a generative AI framework that fine-tunes two pretrained latent diffusion models: Protogen v3.4 and Stable Diffusion v1.4, on a curated, annotated dataset of high-resolution Ulos motifs to generate culturally consistent yet novel designs. Model performance is evaluated quantitatively using Frechet Inception Distance (FID), Inception Score (IS), and qualitatively through assessments by traditional weavers and members of the public. Protogen v3.4 consistently outperforms Stable Diffusion v1.4, achieving substantially lower FID (~10.5x) and higher IS (2.0x), indicating superior visual fidelity, diversity, and closer alignment with the real Ulos motif distribution. We further examine the effects of strength and guidance scale on generation quality across both models. Lower strength values consistently yield higher fidelity (lower FID), while higher strength values increase generative diversity at the cost of realism, revealing a clear fidelity-diversity tradeoff for both models. Across all tested configurations, a guidance scale of 5-9 provides the most effective balance between fidelity and diversity, stabilising FID, KID, and IS, and is recommended as the operating range for high-quality, diverse Ulos motif generation. These findings demonstrate that carefully fine-tuned generative AI can support the creative renewal of intangible cultural heritage while preserving its stylistic and symbolic integrity.

Chinese Translation

保护和振兴传统纺织品，如乌洛斯（Ulos），这是印度尼西亚北苏门答腊巴塔克族的文化遗产，需要在忠于传统与满足当代设计需求的创新方法之间取得平衡。传统乌洛斯编织面临两个主要限制：图案范围狭窄和设计过程耗时。本研究提出了一种生成性人工智能框架，对两个预训练的潜在扩散模型进行精细调优：Protogen v3.4 和 Stable Diffusion v1.4，使用经过策划和注释的高分辨率乌洛斯图案数据集生成文化一致但新颖的设计。通过弗雷歇距离（Frechet Inception Distance, FID）、启发式评分（Inception Score, IS）进行定量评估，并通过传统编织者和公众的评估进行定性评估。Protogen v3.4 的表现始终优于 Stable Diffusion v1.4，FID 显著降低（约 10.5 倍），IS 提高（2.0 倍），表明其在视觉保真度、多样性和与真实乌洛斯图案分布的更紧密对齐方面具有优势。我们进一步考察了强度和引导比例对两个模型生成质量的影响。较低的强度值始终产生更高的保真度（较低的 FID），而较高的强度值则在现实性上增加生成多样性，揭示了两个模型之间明显的保真度与多样性权衡。在所有测试配置中，引导比例为 5-9 提供了保真度与多样性之间最有效的平衡，稳定了 FID、KID 和 IS，并建议作为高质量、多样化乌洛斯图案生成的操作范围。这些发现表明，经过精细调优的生成性人工智能可以支持无形文化遗产的创造性更新，同时保持其风格和象征的完整性。

View on arXiv Download PDF AI Translation

cs.CV / 4 / 2607.06592

LipSSD: Lipschitz-Constrained Single-Shot Detection for Adversarially Robust Object Detection

LipSSD：具有利普希茨约束的单次检测用于对抗鲁棒的目标检测

Lébé, Vincent, Prudent, Yannick, Friedrich, Corentin, Massena, Thomas, Sicre, Ronan, Mamalet, Franck

Abstract

Object detectors have many applications in safety-critical systems, but they are known to be sensitive to worst-case perturbations such as adversarial attacks, which limits their applicability in real-world scenarios. Compared with classification, adversarial robustness for object detection has received less attention, and existing methods are often tied to adversarial training, whose performance may not transfer across attacks, perturbation budgets, or architectures. In this work, we introduce Lipschitz-constrained variants of object detection architectures as robust-by-design alternatives to standard detectors. We validate this approach with LipSSD, a Lipschitz-constrained Single Shot MultiBox Detector (SSD), and provide a comprehensive study of its adversarial robustness using multiple white-box adversarial attacks and datasets. We first analyze the accuracyrobustness trade-off induced by Lipschitz constraints and show that it can be controlled through a single training hyperparameter. We then demonstrate that Lipschitzconstrained detectors are complementary to adversarial training: under the same training setup on the Pascal VOC dataset, adversarially trained LipSSD improves mAP@50 on unseen attacks by up to 15 points over classical adversarially trained SSD. Finally, we use more specific safety-critical datasets such as LARD and KITTI, and show that Lipschitz-constrained detectors can improve robustness while largely preserving clean performance. These results suggest that architectural Lipschitz control is a practical and attack-agnostic direction for improving the robustness of object detectors.

Chinese Translation

目标检测器在安全关键系统中有许多应用，但它们对最坏情况的扰动（如对抗攻击）非常敏感，这限制了它们在现实场景中的适用性。与分类相比，目标检测的对抗鲁棒性受到的关注较少，现有方法通常与对抗训练相关，而其性能可能无法在不同攻击、扰动预算或架构之间迁移。在本研究中，我们引入了目标检测架构的利普希茨约束变体，作为标准检测器的鲁棒设计替代方案。我们通过LipSSD（利普希茨约束的单次多框检测器）验证了这一方法，并使用多种白盒对抗攻击和数据集对其对抗鲁棒性进行了全面研究。我们首先分析了利普希茨约束引起的准确性与鲁棒性之间的权衡，并展示了这一权衡可以通过一个训练超参数进行控制。然后，我们证明了利普希茨约束检测器与对抗训练是互补的：在相同的训练设置下，经过对抗训练的LipSSD在未见攻击上提高了mAP@50，较经典的对抗训练SSD提高了多达15个点。最后，我们使用更具体的安全关键数据集，如LARD和KITTI，展示了利普希茨约束检测器可以在大幅保持清晰性能的同时提高鲁棒性。这些结果表明，架构的利普希茨控制是提高目标检测器鲁棒性的一个实用且与攻击无关的方向。

View on arXiv Download PDF AI Translation

cs.CV / 5 / 2607.06600

MiLSD: A Micro Line-Segment Detector for Resource-Constrained Devices

MiLSD：一种针对资源受限设备的微型线段检测器

Panahi, Parsa Hassani Shariat, Jalilvand, Amir Hossein, Najafi, M. Hassan

Abstract

Line segment detection is a key building block in visual SLAM, 3D reconstruction, and industrial inspection. Recent deep learning methods have greatly improved accuracy, yet even the smallest models require several megabytes of memory, exceeding low-cost MCU capacity. This work investigates the maximum achievable accuracy under a sub-megabyte budget. We propose MiLSD, a detector tailored for MCU-level constraints, and systematically compare three output representations within a compact fully-convolutional backbone. Our study shows that the proposed F-Clip center-with-length-and-angle formulation learns most effectively at small model sizes. We find that 8-bit quantization preserves full-precision performance, while 4-bit quantization causes significant degradation, particularly in angle regression, with quantization-aware training recovering only part of the loss. With a one-megabyte activation budget and inference enhancements including sub-pixel decoding, test-time augmentation, and a lightweight verifier, MiLSD improves sAP10 on ShanghaiTech Wireframe from 10.6 (25k parameters, 0.25 MB) to 24.1 within 1 MB. Rather than competing with GPU-scale parsers, we map the accuracy memory trade-off across representations, bit-widths, capacities, and post-processing strategies for embedded vision systems.

Chinese Translation

线段检测是视觉SLAM、3D重建和工业检测中的关键构建模块。近年来，深度学习方法显著提高了准确性，但即使是最小的模型也需要几兆字节的内存，超出了低成本MCU的容量。本研究探讨了在小于一兆字节的预算下可实现的最大准确性。我们提出了MiLSD，这是一种针对MCU级约束量身定制的检测器，并在紧凑的全卷积骨干网络中系统地比较了三种输出表示。我们的研究表明，所提出的F-Clip中心-带长度和角度的公式在小模型尺寸下学习效果最佳。我们发现8位量化能够保持全精度性能，而4位量化则导致显著降级，特别是在角度回归方面，量化感知训练仅能恢复部分损失。在一兆字节的激活预算下，以及包括亚像素解码、测试时增强和轻量级验证器等推理增强，MiLSD将上海科技大学线框数据集上的sAP10从10.6（25k参数，0.25 MB）提高到24.1，且在1 MB内实现。我们并非与GPU规模的解析器竞争，而是为嵌入式视觉系统绘制了表示、位宽、容量和后处理策略之间的准确性与内存权衡图。

View on arXiv Download PDF AI Translation

cs.CV / 6 / 2607.06603

Do Counterfactually Fair Image Classifiers Satisfy Group Fairness? -- A Theoretical and Empirical Study

反事实公平的图像分类器是否满足群体公平性？——理论与实证研究

Jung, Sangwon, Yu, Sumin, Chun, Sanghyuk, Moon, Taesup

Abstract

The notion of algorithmic fairness has been actively explored from various aspects of fairness, such as counterfactual fairness (CF) and group fairness (GF). However, the exact relationship between CF and GF remains to be unclear, especially in image classification tasks; the reason is because we often cannot collect counterfactual samples regarding a sensitive attribute, essential for evaluating CF, from the existing images (\eg, a photo of the same person but with different secondary sex characteristics). In this paper, we construct new image datasets for evaluating CF by using a high-quality image editing method and carefully labeling with human annotators. Our datasets, \oursceleb and \ourslfw, build upon the popular image GF benchmarks; hence, we can evaluate CF and GF simultaneously. We empirically observe that CF does not imply GF in image classification, whereas previous studies on tabular datasets observed the opposite. We theoretically show that it could be due to the existence of a latent attribute $G$ that is correlated with, but not caused by, the sensitive attribute (\eg, secondary sex characteristics are highly correlated with hair length). From this observation, we propose a simple baseline, Counterfactual Knowledge Distillation (CKD), to mitigate such correlation with the sensitive attributes. Extensive experimental results on \oursceleb and \ourslfw demonstrate that CF-achieving models satisfy GF if we successfully reduce the reliance on $G$ (\eg, using CKD).

Chinese Translation

算法公平性的概念已从反事实公平（CF）和群体公平（GF）等多个方面积极探索。然而，CF与GF之间的确切关系仍不清楚，特别是在图像分类任务中；原因在于我们通常无法从现有图像中收集关于敏感属性的反事实样本（例如，同一个人的照片但具有不同的次级性别特征），这对于评估CF至关重要。在本文中，我们通过使用高质量的图像编辑方法并由人工标注者仔细标记，构建了新的图像数据集以评估CF。我们的数据集 extit{oursceleb}和 extit{ourslfw}基于流行的图像GF基准，因此我们可以同时评估CF和GF。我们实证观察到，在图像分类中，CF并不意味着GF，而在以表格数据集为基础的先前研究中观察到了相反的结果。我们理论上表明，这可能是由于存在一个潜在属性$G$，该属性与敏感属性相关，但并不由其引起（例如，次级性别特征与发长高度相关）。基于这一观察，我们提出了一个简单的基线，反事实知识蒸馏（Counterfactual Knowledge Distillation，CKD），以减轻与敏感属性的相关性。在 extit{oursceleb}和 extit{ourslfw}上的大量实验结果表明，如果我们成功减少对$G$的依赖（例如，使用CKD），那么实现CF的模型将满足GF。

View on arXiv Download PDF AI Translation

cs.CV / 7 / 2607.06618

Overview of the NLPCC 2026 Shared Task 1: Difficulty-Aware Multilingual and Multimodal Medical Instructional Video Understanding Evaluation

NLPCC 2026 共享任务 1 概述：基于难度的多语言和多模态医学教学视频理解评估

Liu, Shenxi, Li, Kan, Zhao, Mingyang, Tian, Yuhang, Li, Bin

Abstract

Following the CMIVQA, MMI-VQA, and M4IVQA challenges in NLPCC 2023--2025, we introduce the Difficulty-Aware Medical Instructional Video Question Answering (DA-MIVQA) shared task for NLPCC 2026. DA-MIVQA extends previous multilingual and multimodal medical video benchmarks by explicitly distinguishing questions according to the type and complexity of evidence required for answering. Specifically, simple questions can often be answered from subtitle-based textual cues, whereas complex questions require visual grounding, procedural understanding, and cross-modal evidence integration. The challenge contains three tracks: Difficulty-Aware Temporal Answer Grounding in Single Video (DA-TAGSV), Difficulty-Aware Video Corpus Retrieval (DA-VCR), and Difficulty-Aware Temporal Answer Grounding in Video Corpus (DA-TAGVC). The dataset is collected from public medical instructional channels, covers diverse scenarios such as first aid, emergency response, rehabilitation, nursing, and general medical education, and is manually verified with difficulty annotations. This paper presents the task motivation, dataset construction, evaluation protocol, participation overview, competition results, and representative systems of DA-MIVQA. DA-MIVQA provides a practical benchmark for evaluating medical instructional video question answering systems under varying textual, visual, temporal, and procedural reasoning requirements.

Chinese Translation

继 NLPCC 2023-2025 的 CMIVQA、MMI-VQA 和 M4IVQA 挑战之后，我们为 NLPCC 2026 引入了基于难度的医学教学视频问答（Difficulty-Aware Medical Instructional Video Question Answering, DA-MIVQA）共享任务。DA-MIVQA 通过根据回答所需证据的类型和复杂性明确区分问题，扩展了之前的多语言和多模态医学视频基准。具体而言，简单问题通常可以通过基于字幕的文本线索回答，而复杂问题则需要视觉定位、过程理解和跨模态证据整合。该挑战包含三个子任务：单视频中的基于难度的时间答案定位（Difficulty-Aware Temporal Answer Grounding in Single Video, DA-TAGSV）、基于难度的视频语料库检索（Difficulty-Aware Video Corpus Retrieval, DA-VCR）和视频语料库中的基于难度的时间答案定位（Difficulty-Aware Temporal Answer Grounding in Video Corpus, DA-TAGVC）。数据集来自公共医学教学频道，涵盖急救、应急响应、康复、护理和一般医学教育等多种场景，并经过人工验证和难度标注。本文介绍了任务动机、数据集构建、评估协议、参与概况、竞赛结果以及 DA-MIVQA 的代表性系统。DA-MIVQA 为评估医学教学视频问答系统在不同文本、视觉、时间和过程推理要求下提供了一个实用的基准。

View on arXiv Download PDF AI Translation

cs.CV / 8 / 2607.06620

SpaR3D-MoE: Adaptive 3D Spatial Reasoning from Sparse Views Meets Geometry-Inductive Mixture-of-Experts

SpaR3D-MoE：从稀疏视图中自适应的3D空间推理与几何诱导混合专家相结合

Feng, Haida, Wei, Hao, Wang, Haolin, Li, Shiwei, Li, Chade, Wu, Yihong

Abstract

Recent Multimodal Large Language Models (MLLMs) struggle to bridge the representational gap between 2D semantic understanding and 3D spatial geometry. Existing 3D-aware models either rely on costly 3D-specific data or utilize RGB-only inputs with heuristic sampling and monolithic, shallow fusion, which respectively disrupt essential spatiotemporal connectivity and induce modality contention across diverse spatial tasks. To overcome these bottlenecks, we introduce SpaR3D-MoE, an end-to-end framework that enables adaptive spatial reasoning by equipping MLLMs with geometry-aware capabilities from only sparse RGB inputs. First, we propose an adaptive spatiotemporal manifold sampling mechanism that constructs a geometry-aware spatiotemporal graph to extract informative keyframes, effectively mitigating sequence redundancy while preserving the scene's topological connectivity. Second, we introduce the heterogeneous geometry-inductive Mixture-of-Experts driven by an instruction-pose aware router, which adaptively routes multimodal tokens to specialized experts, resolving the cross-modal contention inherent in monolithic fusion. Extensive experiments on VSI-Bench, ScanQA, and SQA3D demonstrate that our method achieves state-of-the-art performance. Notably, SpaR3D-MoE achieves the highest average score of 63.5 on VSI-Bench, outperforming the strongest baseline by 7.8 absolute points, alongside relative improvements of 35.4% and 51.4% in Route Plan and Relative Direction tasks, respectively.

Chinese Translation

近期的多模态大型语言模型（MLLMs）在弥合2D语义理解与3D空间几何之间的表征差距方面面临挑战。现有的3D感知模型要么依赖于昂贵的3D特定数据，要么利用仅包含RGB的输入，采用启发式采样和单一、浅层融合，这分别破坏了基本的时空连通性，并在多样的空间任务中引发模态争用。为了解决这些瓶颈，我们提出了SpaR3D-MoE，这是一个端到端框架，通过仅使用稀疏RGB输入为MLLMs赋予几何感知能力，从而实现自适应空间推理。首先，我们提出了一种自适应时空流形采样机制，构建一个几何感知的时空图，以提取信息丰富的关键帧，有效减轻序列冗余，同时保持场景的拓扑连通性。其次，我们引入了异构几何诱导混合专家（Mixture-of-Experts），由一个关注指令-姿态的路由器驱动，能够自适应地将多模态标记路由到专门的专家，从而解决单一融合中固有的跨模态争用。在VSI-Bench、ScanQA和SQA3D上的大量实验表明，我们的方法达到了最先进的性能。值得注意的是，SpaR3D-MoE在VSI-Bench上获得了最高的平均分63.5，超越了最强基线7.8个绝对点，同时在路线规划和相对方向任务中分别实现了35.4%和51.4%的相对提升。

View on arXiv Download PDF AI Translation

cs.CV / 9 / 2607.06631

Dynamic-in-Few-Step: Unifying Dynamic Computation and Few-Step Distillation for Efficient Video Generation

动态少步：统一动态计算与少步蒸馏以实现高效视频生成

Cheng, Yu, Yao, Siyue, Qi, Zhongang, Guan, Shanyan, Li, Wei, Yuan, Fajie

Abstract

Video Diffusion Models (VDMs) have demonstrated superior generation quality but suffer from prohibitive computational costs. While recent few-step distillation techniques significantly accelerate inference, they typically enforce a static model architecture across all denoising stages, ignoring the varying computational demands inherent to different noise levels. In this work, we propose a novel post-training acceleration framework that exploits this redundancy by integrating dynamic structural sparsification directly into the distillation process. Unlike conventional post-hoc compression applied to a fixed diffusion pipeline, our approach jointly optimizes the denoising steps and structured model sparsity, transforming a pre-trained VDM into a compact, step-specific Mixture-of-Models (MoM). To address the training instability arising from this joint optimization, we introduce a Progressive Training Strategy coupled with an Output Rollout Mechanism, which ensures the coherent learning of structural decisions across timesteps. Furthermore, we develop a specialized inference engine to deploy the resulting MoM efficiently. Our method is orthogonal to existing acceleration techniques and highly effective: On Wan-14B, it removes 24% of the per-step FLOPs on top of 4-step distillation, adding a 1.2x wall-clock gain and reaching a 30x speedup over the 50-step teacher while preserving competitive generation quality.

Chinese Translation

视频扩散模型（VDMs）已显示出卓越的生成质量，但面临着高昂的计算成本。尽管最近的少步蒸馏技术显著加速了推理过程，但它们通常在所有去噪阶段强制使用静态模型架构，忽视了不同噪声水平固有的不同计算需求。在本研究中，我们提出了一种新颖的后训练加速框架，通过将动态结构稀疏化直接整合到蒸馏过程中，利用这一冗余。与应用于固定扩散管道的传统后处理压缩不同，我们的方法联合优化去噪步骤和结构模型稀疏性，将预训练的VDM转化为紧凑的、特定步骤的混合模型（Mixture-of-Models, MoM）。为了解决这一联合优化所带来的训练不稳定性，我们引入了一种渐进训练策略，并结合输出展开机制，确保在时间步之间结构决策的连贯学习。此外，我们开发了一种专门的推理引擎，以高效部署所得到的MoM。我们的方法与现有的加速技术正交且效果显著：在Wan-14B上，它在4步蒸馏的基础上去除了每步24%的FLOPs，增加了1.2倍的墙钟增益，并在保持竞争性生成质量的同时实现了相较于50步教师模型的30倍加速。

View on arXiv Download PDF AI Translation

cs.CV / 10 / 2607.06633

ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities

ProMoE-FL：用于缺失模态的多模态联邦学习的原型条件专家混合模型

Chhetri, Aavash, Niroula, Bibek, Vazquez, Eduard, Shrestha, Yash Raj, Gyawali, Prashnna, Bazzani, Loris, Bhattarai, Binod

Abstract

In this paper, we address the problem of multimodal federated learning with missing modality. Existing methods utilize an additional public dataset or perform naive feature synthesis that is based solely on the available modality. To address these limitations, we propose ProMoE-FL, a Prototype-conditioned Mixture-of-Experts framework for robust missing-modality feature synthesis in multimodal federated learning. ProMoE-FL builds a global client-aware prototype bank that captures clinically meaningful modality priors across institutions. Our Mixture of Experts is conditioned on these prototypes and modality indices to enable direction-aware expert routing for dynamically synthesizing missing features. We perform extensive quantitative and qualitative evaluations on four public chest X-ray datasets (MIMIC-CXR, NIH Open-I, PadChest, and CheXpert) and demonstrate that ProMoE-FL consistently outperforms state-of-the-art methods in both homogeneous as well as the more challenging heterogeneous settings.

Chinese Translation

在本文中，我们解决了缺失模态的多模态联邦学习问题。现有的方法利用额外的公共数据集或仅基于可用模态进行简单的特征合成。为了解决这些局限性，我们提出了ProMoE-FL，一个用于多模态联邦学习中稳健缺失模态特征合成的原型条件专家混合框架。ProMoE-FL构建了一个全球客户端感知的原型库，捕捉跨机构的临床意义模态先验。我们的专家混合模型基于这些原型和模态索引进行条件化，以实现方向感知的专家路由，从而动态合成缺失特征。我们在四个公共胸部X光数据集（MIMIC-CXR、NIH Open-I、PadChest和CheXpert）上进行了广泛的定量和定性评估，结果表明ProMoE-FL在同质和更具挑战性的异质设置中均持续优于最先进的方法。

View on arXiv Download PDF AI Translation

cs.CV / 11 / 2607.06691

CoMind: Understanding Collaborative Human Activity from Multiple Minds and Views

CoMind：从多重视角和思维理解协作人类活动

Gavryushin, Alexey, Zhang, Dingxi, Huang, Zhao, Delitzas, Alexandros, Chen, Jiaqi, Ellis, Ben, Zöllner, Cedric, Patel, Manthan, Kaufmann, Manuel, Pollefeys, Marc, Wang, Xi

Abstract

Human-human collaboration is a fundamental aspect of everyday life, essential to success in a wide range of goal-directed activities from household tasks to professional teamwork. While much research has focused on modeling coordination and task execution, the cognitive processes that support such collaboration, particularly Theory of Mind (the ability to infer the mental states of others), remain difficult to study in natural settings. To address this gap, we introduce a novel egocentric and exocentric video dataset capturing real-world collaboration in cooking scenarios. The dataset integrates multi-perspective video, high-quality audio, gaze tracking, and 3D scene and object scans, with annotations for shared attention to objects, social cues and interactions between agents, as well as agent-object interactions. We establish benchmarks for Joint Attention Estimation, Socially Conditioned Object Interaction Anticipation, and Collaborative Handover Prediction, enabling research on multimodal perception, proactive assistance, and collaborative planning. By providing temporally aligned, richly annotated multimodal data, CoMind facilitates the development and evaluation of AI systems capable of modeling complex social interactions and reasoning about human behaviors in collaborative environments. Our dataset and benchmarks are made available at https://comind.ethz.ch/.

Chinese Translation

人际协作是日常生活中的一个基本方面，对于从家庭任务到专业团队合作等各种目标导向活动的成功至关重要。尽管许多研究集中于协调和任务执行的建模，但支持这种协作的认知过程，特别是心智理论（推断他人心理状态的能力），在自然环境中仍然难以研究。为了解决这一空白，我们引入了一个新颖的自我中心和外部中心的视频数据集，捕捉了现实世界中烹饪场景的协作。该数据集整合了多视角视频、高质量音频、注视追踪以及三维场景和物体扫描，并对共享注意力、社会线索和代理之间的互动以及代理与物体之间的互动进行了注释。我们建立了联合注意力估计、社会条件下的物体互动预测和协作交接预测的基准，促进了多模态感知、主动辅助和协作规划的研究。通过提供时间对齐、丰富注释的多模态数据，CoMind 促进了能够建模复杂社会互动和推理人类行为的人工智能系统的发展和评估。我们的数据集和基准可在 https://comind.ethz.ch/ 获取。

View on arXiv Download PDF AI Translation

cs.CV / 12 / 2607.06701

SPEAR: A Simulator for Photorealistic Embodied AI Research

SPEAR：用于逼真体现人工智能研究的模拟器

Roberts, Mike, Wang, Renhan, Zawar, Rushikesh, Dey-Prakash, Rachith, Leboutet, Quentin, Richter, Stephan R., Müller, Matthias, Ros, German, Tang, Rui, Leutenegger, Stefan, Hold-Geoffroy, Yannick, Sunkavalli, Kalyan, Koltun, Vladlen

Abstract

Interactive simulators have become powerful tools for training embodied agents and generating synthetic visual data, but existing photorealistic simulators suffer from limited generality, programmability, and rendering speed. We address these limitations by introducing SPEAR: A Simulator for Photorealistic Embodied AI Research. At its core, SPEAR is a Python library that can connect to, and programmatically control, any Unreal Engine (UE) application via a modular plugin architecture. SPEAR exposes over 14K unique UE functions to Python, representing an order-of-magnitude increase in programmable functionality over existing UE-based simulators. Additionally, a single SPEAR instance can render 1920x1080 photorealistic beauty images directly into a user's NumPy array at 73 frames per second - an order of magnitude faster than existing UE plugins - while also providing ground truth image modalities that are not available in any existing UE-based simulator (e.g., a non-diffuse intrinsic image decomposition, material IDs, and physically based shading parameters). Finally, SPEAR introduces an expressive high-level programming model that enables users to specify complex graphs of UE work with arbitrary data dependencies among work items, and to execute these graphs deterministically within a single UE frame. We demonstrate the utility of SPEAR through a diverse collection of example applications: controlling multiple embodied agents with distinct action spaces (e.g., humans, cars, and robots) across several in-the-wild UE projects; rendering photorealistic city-scale environments; manipulating UE's procedural content generation systems; rendering synchronized multi-view images of detailed human faces; coordinating an interactive co-simulation with the MuJoCo physics simulator; and editing scenes with natural language via an AI coding assistant.

Chinese Translation

交互式模拟器已成为训练体现代理和生成合成视觉数据的强大工具，但现有的逼真模拟器在通用性、可编程性和渲染速度方面存在局限。我们通过引入SPEAR（用于逼真体现人工智能研究的模拟器）来解决这些限制。SPEAR的核心是一个Python库，可以通过模块化插件架构连接并程序化控制任何Unreal Engine（UE）应用程序。SPEAR将超过14,000个独特的UE函数暴露给Python，代表了相较于现有基于UE的模拟器在可编程功能上的数量级提升。此外，单个SPEAR实例可以以每秒73帧的速度将1920x1080的逼真美图直接渲染到用户的NumPy数组中，速度是现有UE插件的数量级更快，同时还提供现有任何基于UE的模拟器中不可用的真实图像模态（例如，非漫反射内在图像分解、材质ID和基于物理的着色参数）。最后，SPEAR引入了一种表达性强的高级编程模型，使用户能够指定具有任意数据依赖关系的UE工作复杂图形，并在单个UE帧内确定性地执行这些图形。我们通过一系列多样的示例应用展示了SPEAR的实用性：在多个真实UE项目中控制具有不同动作空间的多个体现代理（例如，人类、汽车和机器人）；渲染逼真的城市规模环境；操控UE的程序内容生成系统；渲染详细人脸的同步多视图图像；协调与MuJoCo物理模拟器的交互式共同仿真；以及通过AI编码助手使用自然语言编辑场景。

View on arXiv Download PDF AI Translation

cs.CV / 13 / 2607.06726

A Good Initialization is All You Need for Faithful Visual Attribution

良好的初始化是实现可信视觉归因所需的一切

Gu, Zihan, Wang, Jiayu, Zhang, Hua, Hu, Yue

Abstract

Faithful visual attribution identifies which image regions support a model prediction. Search-based perturbation methods lead the insertion--deletion faithfulness frontier by masking regions and measuring score changes, but they usually output a complete ordering of all regions. Many applications, especially MLLM attribution and repair, only need a compact top-$k$ evidence mask. We study this mask-first attribution problem. An exactly $k$-region mask is combinatorial: useful evidence can depend on interactions among fine regions. Coarse grouping can stabilize early search but aggregates redundant content, whereas one-step scoring can miss high-value combinations. We introduce two forward-only methods. \textsc{CoPAIR} uses a PhaseWin--Greedy gap diagnosis to construct coarse singleton/pair candidates that warm-start full-ordering search. \textsc{TRACE} directly searches fixed-cardinality fine-region masks with cross-entropy sampling, elite retention, and distribution updates, with a finite-budget recovery analysis. The resulting evidence set can be returned as a compact attribution mask or used to initialize Greedy or PhaseWin when a complete ranking is required. Across ImageNet classification with CLIP ViT-L/14, CLIP RN101, and ResNet-101, our initialized search methods establish a new state-of-the-art frontier for faithful full-ordering attribution under inclusive forward-call accounting. On POPE and RePOPE with Qwen2.5-VL-3B-Instruct and LLaVA-v1.5-7B, \textsc{TRACE}+Greedy gives the strongest search-based MLLM attribution results. Direct \textsc{TRACE} masks further achieve single-point RePOPE repair rates of $94.44\%$ and $96.00\%$, showing that compact evidence masks can be actionable attribution outputs, not merely prefixes of full rankings.

Chinese Translation

可信视觉归因识别哪些图像区域支持模型预测。基于搜索的扰动方法通过遮蔽区域并测量得分变化，领先于插入-删除的可信度前沿，但它们通常输出所有区域的完整排序。许多应用，特别是多模态大语言模型（MLLM）的归因和修复，仅需要一个紧凑的前- (k)证据掩码。我们研究这一掩码优先的归因问题。一个精确的(k)区域掩码是组合性的：有用的证据可能依赖于细区域之间的相互作用。粗略分组可以稳定早期搜索，但会聚合冗余内容，而一步评分可能会错过高价值组合。我们引入了两种仅向前的方法。 extsc{CoPAIR} 使用 PhaseWin-Greedy 差距诊断构建粗略的单体/对候选，帮助启动全排序搜索。 extsc{TRACE} 直接使用交叉熵采样、精英保留和分布更新搜索固定基数的细区域掩码，并进行有限预算的恢复分析。生成的证据集可以作为紧凑的归因掩码返回，或在需要完整排名时用于初始化 Greedy 或 PhaseWin。在使用 CLIP ViT-L/14、CLIP RN101 和 ResNet-101 的 ImageNet 分类中，我们的初始化搜索方法在包容性前向调用会计下建立了可信全排序归因的新最优前沿。在使用 Qwen2.5-VL-3B-Instruct 和 LLaVA-v1.5-7B 的 POPE 和 RePOPE 上， extsc{TRACE}+Greedy 提供了最强的基于搜索的 MLLM 归因结果。直接的 extsc{TRACE} 掩码进一步实现了单点 RePOPE 修复率为 94.44% 和 96.00%，显示出紧凑的证据掩码可以作为可操作的归因输出，而不仅仅是完整排名的前缀。

View on arXiv Download PDF AI Translation

cs.CV / 14 / 2607.06739

Hardware-aware Graph Neural Networks prunning for embedded event-based vision

面向硬件的图神经网络剪枝用于嵌入式事件驱动视觉

Wzorek, Piotr, Jeziorek, Kamil, Kryjak, Tomasz

Abstract

Event-based cameras are gaining popularity as the sensor of choice for mobile robotics, due to their high performance in dynamic environments. However, these applications require efficient real-time data processing with low latency and power consumption. One strategy to meet these stringent requirements is hardware acceleration of efficient algorithms that preserve the temporal sparsity of event data. In this work, we propose an optimization strategy for Graph Convolutional Neural Networks models aimed at adapting their architecture to the limited resources of embedded heterogeneous FPGA platforms. Our method incorporates hardware-aware pruning and quantization, taking into account the trade-off between on-chip memory savings and inference accuracy. Strategic exploration of the design space with Fine Grid Search and Greedy layer-wise Iterative Deepening Search methods enables flexible adaptation of the model architecture to the target platform. Our approach was evaluated across various network configurations and multiple datasets, resulting in BRAM memory reductions of 28.8% for CIFAR-10 (with a 1.65% decrease in accuracy), 31.4% for MNIST-DVS (accuracy drop of 3.55%), and 26.5% for N-Caltech101 (with a 5.18% accuracy reduction).

Chinese Translation

事件驱动相机因其在动态环境中的高性能而日益成为移动机器人首选的传感器。然而，这些应用需要低延迟和低功耗的高效实时数据处理。满足这些严格要求的一种策略是对有效算法进行硬件加速，同时保持事件数据的时间稀疏性。在本研究中，我们提出了一种针对图卷积神经网络模型的优化策略，旨在使其架构适应嵌入式异构FPGA平台的有限资源。我们的方法结合了面向硬件的剪枝和量化，考虑了片上内存节省与推理精度之间的权衡。通过细致网格搜索（Fine Grid Search）和贪婪层次迭代深度搜索（Greedy layer-wise Iterative Deepening Search）方法对设计空间进行战略性探索，使模型架构能够灵活适应目标平台。我们的方案在多种网络配置和多个数据集上进行了评估，结果显示CIFAR-10的BRAM内存减少了28.8%（精度下降1.65%），MNIST-DVS减少了31.4%（精度下降3.55%），N-Caltech101减少了26.5%（精度下降5.18%）。

View on arXiv Download PDF AI Translation

cs.CV / 15 / 2607.06779

URS-Stereo: Uncertainty-Guided Residual Search for Real-Time Stereo Matching

URS-Stereo：基于不确定性引导的残差搜索用于实时立体匹配

Sohrabipour, Pouya, Pallerla, Chaitanya kumar reddy, Wang, Dongyi

Abstract

Real-time stereo matching is crucial for robotics, autonomous systems, and embedded vision applications, where both computational efficiency and disparity accuracy are required. Recent coarse-to-fine stereo matching methods improve efficiency by progressively refining disparity estimates using local cost volumes at higher resolutions. However, these methods rely heavily on the accuracy of propagated disparity estimates from previous stages. When the propagated disparity is inaccurate, the ground-truth correspondence may fall outside the predefined local search range, leading to unrecoverable matching failures during subsequent refinement. In this paper, we propose URS-Stereo, a real-time coarse-to-fine stereo matching framework that addresses this limitation through uncertainty-guided search adaptation. Specifically, we introduce an Uncertainty-Guided Residual Search Module (UGRSM), which predicts the reliability of propagated disparities together with residual search offsets to adaptively relocate the centers of local cost volumes before disparity refinement. By dynamically adjusting the search region according to the confidence of the propagated disparity, the proposed method significantly improves the robustness of local correspondence estimation while preserving the computational efficiency of coarse-to-fine stereo matching. Extensive experiments on SceneFlow, KITTI 2012, KITTI 2015, Middlebury, and ETH3D demonstrate that URS-Stereo consistently improves disparity estimation while maintaining real-time inference speed, validating the effectiveness of the proposed uncertainty-guided search strategy

Chinese Translation

实时立体匹配对于机器人技术、自动化系统和嵌入式视觉应用至关重要，这些应用要求计算效率和视差精度兼具。最近的粗到细立体匹配方法通过在更高分辨率下逐步细化视差估计，利用局部成本体积提高了效率。然而，这些方法在很大程度上依赖于来自前一阶段的传播视差估计的准确性。当传播的视差不准确时，真实对应关系可能会超出预定义的局部搜索范围，导致在后续细化过程中无法恢复的匹配失败。本文提出了URS-Stereo，一种实时粗到细立体匹配框架，通过不确定性引导的搜索适应性来解决这一限制。具体而言，我们引入了不确定性引导的残差搜索模块（Uncertainty-Guided Residual Search Module, UGRSM），该模块预测传播视差的可靠性，并结合残差搜索偏移量，适应性地重新定位局部成本体积的中心，以便在视差细化之前进行调整。通过根据传播视差的置信度动态调整搜索区域，所提出的方法显著提高了局部对应估计的鲁棒性，同时保持了粗到细立体匹配的计算效率。在SceneFlow、KITTI 2012、KITTI 2015、Middlebury和ETH3D上的大量实验表明，URS-Stereo在保持实时推理速度的同时，始终改善了视差估计，验证了所提出的不确定性引导搜索策略的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 16 / 2607.06829

Rail Track Extraction from Rasterized Classified Point Clouds Using a Full-Resolution, Fully Convolutional Recurrent Neural Network

基于全分辨率全卷积递归神经网络的光栅化分类点云中的轨道提取

Gribov, Alexander, Chang, Jie

Abstract

Rail track extraction is essential for effective railway asset management and maintenance, especially in automated inspection and mapping workflows. This paper introduces a novel method for extracting rail tracks from classified 3D point clouds using a fully convolutional recurrent neural network that preserves full spatial resolution and is trained exclusively on synthetically generated data. This approach enhances per-pixel quality and is particularly suited for rail track extraction. The proposed method begins by rasterizing points corresponding to railroad tracks, then applies the neural network to reduce noise and yield a cleaner track representation suitable for vectorization [1]. Subsequent morphological operations further refine the resultant data, enabling accurate track centerline extraction. Next, the extracted centerlines undergo smoothing to eliminate residual irregularities [2, 3]. Finally, the algorithm transfers 3D information from lidar points onto 2D polylines and applies additional vertical smoothing. A single centerline for both tracks is found using the Dynamic Time Warping (DTW) algorithm [4]. The final outcome consists of rail top centerlines and track centerlines derived for rail pairs, with minimal manual intervention. Experimental validation confirms the effectiveness of this method in yielding high-quality rail track extraction.

Chinese Translation

轨道提取对于有效的铁路资产管理和维护至关重要，特别是在自动化检查和制图工作流程中。本文介绍了一种新颖的方法，利用全分辨率全卷积递归神经网络从分类的三维点云中提取轨道，该网络保持完整的空间分辨率，并且仅在合成生成的数据上进行训练。这种方法提高了每个像素的质量，特别适合轨道提取。所提出的方法首先对与铁路轨道对应的点进行光栅化，然后应用神经网络以减少噪声，并生成适合矢量化的更清晰的轨道表示。随后，形态学操作进一步精炼结果数据，使得轨道中心线的提取更加准确。接下来，提取的中心线经过平滑处理，以消除残余的不规则性。最后，算法将激光雷达点的三维信息转移到二维多边形线上，并应用额外的垂直平滑。使用动态时间规整（Dynamic Time Warping, DTW）算法找到两个轨道的单一中心线。最终结果包括轨道顶部中心线和为轨道对提取的轨道中心线，且手动干预最小。实验验证确认了该方法在实现高质量轨道提取方面的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 17 / 2607.06838

WildCity: A Real-World City-Scale Testbed for Rendering, Simulation, and Spatial Intelligence

WildCity：一个用于渲染、模拟和空间智能的真实城市规模测试平台

Han, Xiangyu, Yang, Mengyu, Li, Jiaqi, Chang, Bowen, Chen, Ziyu, Zhao, Hexu, Agrawal, Rahul Kumar, Rodriguez, Anthony, Hua, Fiona, Pavone, Marco, Feng, Chen, Li, Yiming

Abstract

Humans can navigate an unfamiliar city and gradually form a coherent spatial mental map spanning tens of square kilometers. Can AI build spatial representations at a comparable scale? Although recent foundation models have advanced scene reconstruction and embodied intelligence, scaling to entire cities remains an open challenge, primarily due to the lack of city-scale data. To bridge the gap, we introduce WildCity, a real-world multimodal dataset collected by autonomous fleets traversing complex urban environments. Our dataset includes 18 trajectories, each averaging 83.7 kilometers in length, and preserves the core challenges of in-the-wild perception, e.g., dynamic objects, lighting variations, and imperfect camera poses. We further establish an urban-tailored reconstruction baseline and convert the reconstructed environments into a closed-loop simulator. Beyond the dataset and baseline, we systematically analyze the key challenges on the path to simulation-ready urban digital twins: scalability, extrapolation, and uncertainty. Ultimately, WildCity aims to catalyze progress not only in city-scale rendering, but more broadly in the pursuit of AI that can perceive, remember, and reason across space at a scale comparable to human cognition. Project page: https://han-xiangyu.github.io/Wild-City/

Chinese Translation

人类能够在陌生城市中导航，并逐渐形成一个覆盖数十平方公里的连贯空间心理地图。人工智能能否在可比的规模上构建空间表征？尽管近期的基础模型在场景重建和具身智能方面取得了进展，但扩展到整个城市仍然是一个未解的挑战，主要是由于缺乏城市规模的数据。为了解决这一问题，我们引入了WildCity，这是一个由自主车队在复杂城市环境中收集的真实多模态数据集。我们的数据集包含18条轨迹，每条轨迹平均长度为83.7公里，并保留了野外感知的核心挑战，例如动态物体、光照变化和不完美的相机姿态。我们进一步建立了一个城市定制的重建基线，并将重建的环境转换为闭环模拟器。除了数据集和基线，我们还系统地分析了实现可用于模拟的城市数字双胞胎所面临的关键挑战：可扩展性、外推和不确定性。最终，WildCity旨在推动不仅在城市规模渲染方面的进展，更广泛地推动人工智能在空间感知、记忆和推理方面的发展，使其能够在与人类认知相当的规模上进行操作。项目页面：https://han-xiangyu.github.io/Wild-City/

View on arXiv Download PDF AI Translation

cs.CV / 18 / 2607.06843

Retrieving and Refining Winning Noise Tickets for Diffusion-Based Motion Generation

检索和优化基于扩散的运动生成中的有效噪声票

Ota, Sakuya, Yu, Qing, Fujiwara, Kent, Ikehata, Satoshi, Sato, Ikuro

Abstract

Diffusion-based text-to-motion models synthesize realistic human motions but often exhibit semantic drift from the input text. Motion is inherently temporal, especially in compositional and long-duration sequences that require semantic consistency across multiple action segments and smooth kinematic transitions throughout the trajectory. We posit that the initial noise is central to this consistency: within the Gaussian noise space, certain instances, i.e. winning noise tickets, carry latent structure that biases denoising toward particular motion semantics, even under null prompts. We propose WInning Noise Retrieval and Optimization (WINRO), a training-free, model-agnostic framework that improves text-motion alignment by selecting and refining such tickets before diffusion sampling. WINRO maps random noises to motion features generated under null prompts, retrieves the best-aligned noise for a given text, and refines it via a KL-regularized objective that reduces the residual semantic gap while preserving the Gaussian prior. An optional LoRA-based adapter amortizes this refinement into a single forward pass. WINRO consistently improves text-motion fidelity across different base models, MDM and MotionLCM, on HumanML3D without retraining, improves temporal robustness on the MTT benchmark, and generalizes to applications such as motion stylization and spatial constraint satisfaction.

Chinese Translation

基于扩散的文本到运动模型能够合成逼真的人类运动，但往往会出现与输入文本的语义漂移。运动本质上是时间性的，尤其是在需要跨多个动作片段保持语义一致性和在整个轨迹中实现平滑运动过渡的组合性和长时间序列中。我们认为初始噪声对这种一致性至关重要：在高斯噪声空间中，某些实例，即有效噪声票，携带潜在结构，使去噪偏向特定的运动语义，即使在无提示的情况下。我们提出了有效噪声检索与优化（WInning Noise Retrieval and Optimization，WINRO），这是一个无训练、模型无关的框架，通过在扩散采样之前选择和优化这些票据来改善文本与运动的对齐。WINRO将随机噪声映射到在无提示下生成的运动特征，检索与给定文本最佳对齐的噪声，并通过KL正则化目标进行优化，以减少残余语义差距，同时保持高斯先验。可选的基于LoRA的适配器将这种优化融入到单次前向传播中。WINRO在HumanML3D上对不同基础模型（MDM和MotionLCM）的文本运动保真度持续提升，在MTT基准测试中提高了时间鲁棒性，并且能够推广到运动风格化和空间约束满足等应用。

View on arXiv Download PDF AI Translation

cs.CV / 19 / 2607.06856

Gen4U: Unifying Video Generation and Understanding via Diffusion

Gen4U：通过扩散统一视频生成与理解

King, Michael, Mahendran, Aravindh, Grimes, Matthew Koichi, Kitashov, Fedor, Elarabawy, Adham, Velez, Pedro, Ovsjanikov, Maks, Pătrăucean, Viorica

Abstract

Prior work suggests that diffusion representations capture low-level geometry but struggle with high-level semantics. We demonstrate that state-of-the-art video diffusion models overcome this limitation. By systematically probing their intermediate activations using recent mutual-kNN alignment metrics, we reveal a highly structured latent space where visual representations evolve across both network depth and noise levels. We show that while moderate noise levels yield linearly separable global semantics, fine-grained details persist at lower noise levels but become spatially scattered, requiring attention mechanisms to decode. Building on these insights, we introduce Gen4U (Generation for Understanding), a framework that repurposes these generative representations with a single forward pass. Our experiments establish that frozen, large-scale video diffusion models function as highly competitive video encoders across a wide spectrum of tasks, spanning semantic and non-semantic objectives (video classification, depth estimation, camera pose estimation, image and video captioning). Bypassing fine-tuning, Gen4U unifies the generation and understanding paradigms, achieving strong perception performance while fully preserving the model's ability to generate high-quality video.

Chinese Translation

先前的研究表明，扩散表示能够捕捉低层次的几何信息，但在高层次语义上存在困难。我们证明了最先进的视频扩散模型克服了这一限制。通过使用最新的互信息 k 最近邻（mutual-kNN）对齐度量系统性地探测其中间激活，我们揭示了一个高度结构化的潜在空间，其中视觉表示在网络深度和噪声水平上不断演变。我们展示了适度的噪声水平能够产生线性可分的全局语义，而在较低噪声水平下，细粒度的细节依然存在，但变得空间上分散，需借助注意机制进行解码。基于这些见解，我们引入了 Gen4U（Generation for Understanding），一个通过单次前向传播重新利用这些生成表示的框架。我们的实验表明，冻结的大规模视频扩散模型在广泛的任务中作为高度竞争的视频编码器表现出色，涵盖了语义和非语义目标（视频分类、深度估计、相机姿态估计、图像和视频字幕生成）。Gen4U 在不进行微调的情况下统一了生成与理解的范式，实现了强大的感知性能，同时完全保留了模型生成高质量视频的能力。

View on arXiv Download PDF AI Translation

cs.CV / 20 / 2607.06871

Geometric Collapse: When Vision Models Fail to Verify Physical Causality

几何崩溃：当视觉模型无法验证物理因果关系时

Zhang, Wentao, Qi, Jinhu, Jin, Weiqiang, Zhang, Yifei, Lam, Chan-Tong, King, Irwin

Abstract

Recent progress in large-scale self-supervised learning has improved dense geometric prediction, but it remains unclear whether such scaling yields inference-time physical plausibility checks. We propose Scrambled Edges, a controlled counterfactual that injects salient edge-like cues while violating surface continuity, illumination coherence, and occlusion ordering. With energy-matched and structure-matched controls, we isolate the effect of unsupported edge evidence from high-frequency energy and edge sparsity. Across CNN/ViT/SSL depth predictors on NYU Depth v2 and KITTI, Scrambled Edges induce up to 3.2x larger deviation from clean predictions than energy-matched noise; additional diffusion and flow-matching depth estimators show attenuated but still significant collapse. The resulting Geometric Collapse propagates globally: even with oracle knowledge of the corrupted region, output-level repair recovers only 47%, with substantial error outside the mask. These findings provide controlled behavioral evidence that current dense predictors lack reliable mechanisms to quarantine physically unsupported edge cues, motivating explicit plausibility scoring and selective cue integration.

Chinese Translation

近期在大规模自监督学习方面的进展改善了密集几何预测，但尚不清楚这种扩展是否能在推理时进行物理合理性检查。我们提出了 Scrambled Edges（打乱边缘），这是一种受控的反事实方法，通过引入显著的边缘线索，同时违反表面连续性、照明一致性和遮挡顺序。通过能量匹配和结构匹配的控制，我们将不支持的边缘证据的影响与高频能量和边缘稀疏性进行了隔离。在 NYU Depth v2 和 KITTI 数据集上的 CNN/ViT/SSL 深度预测器中，Scrambled Edges 使得与干净预测的偏差最大达到 3.2 倍，超过了能量匹配噪声；额外的扩散和流匹配深度估计器显示出减弱但仍显著的崩溃。由此产生的几何崩溃在全局范围内传播：即使对受损区域有神谕知识，输出级别的修复也仅能恢复 47%，且在掩膜外存在显著误差。这些发现提供了受控的行为证据，表明当前的密集预测器缺乏可靠的机制来隔离物理上不支持的边缘线索，促使我们进行明确的合理性评分和选择性线索整合。

View on arXiv Download PDF AI Translation

cs.CV / 21 / 2607.06872

Ensemble Deep Learning Approaches for AI-Altered Video Detection

用于人工智能生成视频检测的集成深度学习方法

Khan, Laiba, Wu, Hung-Mao, Lin, Wei, Bi, Frank, Abdelhadi, Yousef, Jung, Joshua

Abstract

The increasing accessibility of artificial intelligence has led to a rapid rise in AI-generated videos, making it more difficult to distinguish between real and manipulated content. Many existing detection methods rely on a single model and often struggle to generalize across different types of deepfakes. In this work, we developed a multimodal deepfake detection system that combines both audio and visual analysis using an ensemble of models. The system includes AASIST for audio-based detection, and EfficientNet, XceptionNet, and MesoNet for analyzing visual features in video frames. The pipeline takes a video as input, separates the audio, and extracts face frames using MTCNN. Each model produces a score indicating the likelihood of the input being fake. These scores are then combined using ensemble strategies, including mean averaging and stacking. Mean fusion provides a simple and stable baseline, while stacking uses a trained meta-model to learn how to combine predictions more effectively. Results show that while individual models perform well on the datasets they were trained on, their performance drops when tested on more diverse datasets. The ensemble approach helps improve overall robustness by combining predictions from multiple models, leading to more consistent performance across different types of deepfakes. This suggests that using both audio and visual information together is a more reliable approach for deepfake detection. Our results highlight generalization to unseen manipulations as the central open challenge, with average accuracy around 70%.

Chinese Translation

人工智能的日益普及导致了AI生成视频的快速增加，使得区分真实内容与被操纵内容变得更加困难。许多现有的检测方法依赖于单一模型，往往难以在不同类型的深度伪造（deepfake）中进行泛化。在本研究中，我们开发了一种多模态深度伪造检测系统，结合了音频和视觉分析，采用模型集成的方法。该系统包括用于音频检测的AASIST，以及用于分析视频帧视觉特征的EfficientNet、XceptionNet和MesoNet。该流程以视频为输入，分离音频，并使用MTCNN提取人脸帧。每个模型生成一个分数，指示输入为伪造的可能性。这些分数随后通过集成策略进行组合，包括均值平均和堆叠。均值融合提供了一个简单且稳定的基线，而堆叠则使用训练好的元模型学习如何更有效地组合预测结果。结果表明，尽管单个模型在其训练数据集上表现良好，但在更具多样性的数据集上测试时，其性能下降。集成方法通过结合多个模型的预测，帮助提高整体鲁棒性，从而在不同类型的深度伪造中实现更一致的表现。这表明同时使用音频和视觉信息是深度伪造检测的更可靠方法。我们的结果突显了对未见操纵的泛化作为中心开放挑战，平均准确率约为70%。

View on arXiv Download PDF AI Translation

cs.CV / 22 / 2607.06875

Video2Reaction: Mapping Video to Audience Reaction Distribution in the Wild

Video2Reaction：将视频映射到观众反应分布

Nguyen, Trang, Zhang, Sidong, Shankar, Shiv, Jagatap, Gauri, Chandran, Deepak, Fanelli, Andrea, Fiterau, Madalina

Abstract

Understanding and forecasting audience reactions to video content are crucial for improving content creation, recommendation systems, and media analysis. To enable audience reaction prediction and other content engagement applications, we introduce $\textbf{Video2Reaction}$, a multimodal dataset that maps short movie segments to a distribution of $\textit{induced emotions}$ of viewers in the wild, as expressed through social media. $\textbf{Video2Reaction}$ spans more than 10,000 videos and serves as a reliable benchmark as well as a training resource for audience reaction prediction. To enable cost-effective continuous annotations as reactions may change over time, we develop a two-stage multi-agent pipeline using only open-source LLMs, achieving 86% correctness under blind human verification despite the inherently noisy and subjective nature of the task. We establish the first benchmark for video-to-reaction-distribution prediction in the wild and show that pretrained foundation video models fail in zero-shot settings, while finetuning transforms them into state-of-the-art predictors capable of modeling both full reaction distributions and dominant responses from video alone. However, the task remains challenging: even the strongest methods achieve only 77% Top-3 F1 in dominant reaction prediction (LLaVA-Next), highlighting a substantial gap in modeling collective audience reaction. \modification{Dataset and code are available at our project page: https://information-fusion-lab-umass.github.io/video2reaction-bench.github.io

Chinese Translation

理解和预测观众对视频内容的反应对提高内容创作、推荐系统和媒体分析至关重要。为了实现观众反应预测和其他内容互动应用，我们引入了$ extbf{Video2Reaction}$，一个多模态数据集，它将短片段与观众在社交媒体上表达的$ extit{诱发情感}$分布相映射。$ extbf{Video2Reaction}$包含超过10,000个视频，作为观众反应预测的可靠基准和训练资源。为了实现成本效益高的持续注释，因为反应可能随时间变化，我们开发了一种两阶段多智能体流程，仅使用开源的大型语言模型(LLMs)，在盲人验证下实现了86%的正确率，尽管任务本质上存在噪声和主观性。我们建立了第一个野外视频到反应分布预测的基准，并展示了预训练基础视频模型在零-shot 设置中的失败，而微调则将它们转变为最先进的预测器，能够仅通过视频建模完整的反应分布和主导回应。然而，这一任务仍然具有挑战性：即使是最强的方法在主导反应预测中也仅达到了77%的Top-3 F1分数（LLaVA-Next），突显出在建模集体观众反应方面存在显著差距。\modification{数据集和代码可在我们的项目页面获取：https://information-fusion-lab-umass.github.io/video2reaction-bench.github.io

View on arXiv Download PDF AI Translation

cs.CV / 23 / 2607.06889

ReMoDEx: A Local-to-Global Relevance-Based Model Decision Explainability Framework for large-Scale Image Datasets

ReMoDEx：一种基于局部到全局相关性的模型决策可解释性框架，用于大规模图像数据集

Pathak, Abhay Kumar, Chaubey, Mrityunjay, Gupta, Manjari

Abstract

Deep learning image classifiers achieve strong predictive performance yet remain opaque in how decisions are formed. A model may predict correctly while relying on irrelevant cues, shortcut associations, peripheral structures, or device level artifacts instead of task relevant regions. On large scale datasets this opacity is especially problematic, since inspecting heatmaps one sample at a time cannot scale to thousands of predictions. We propose Relevance Based Model Decision Explainability (ReMoDEx), a framework for systematic, dataset scale assessment of model decision behaviour in image classification. ReMoDEx defines a stepwise pipeline: model inference, target class selection, relevance map generation, heatmap standardisation, similarity based grouping of patterns, cluster level interpretation, and spatial relevance assessment. Local methods GradCAM++, Integrated Gradients, Occlusion Sensitivity, and Layerwise Relevance Propagation are each combined independently with a single global module that summarises an entire set of relevance maps into a few decision strategy clusters, replacing sample by sample inspection with an automatic, scalable summary. To demonstrate ReMoDEx, we applied it to a VGG16 based classifier distinguishing COVID-19, Normal, Lung Opacity, and Viral Pneumonia. The classifier showed stable performance (86.27% test accuracy, 0.9624 test AUC). However, each explainer combined with the global module consistently produced two recurring strategies: central thoracic region decisions and border/corner sensitive decisions, indicating possible shortcut learning that conventional metrics could not reveal. Masked image validation confirmed that model confidence and predicted class changed when central or peripheral regions were occluded. ReMoDEx thus provides a scalable relevance based decision assessment framework and an essential complement to accuracy based evaluation.

Chinese Translation

深度学习图像分类器在预测性能上表现出色，但在决策形成的透明度方面仍然存在问题。模型可能在依赖无关线索、捷径关联、外围结构或设备级伪影而非任务相关区域的情况下做出正确预测。在大规模数据集中，这种不透明性尤其成问题，因为逐个样本检查热图无法扩展到数千个预测。我们提出了基于相关性的模型决策可解释性（ReMoDEx），这是一个用于系统性、大规模评估图像分类中模型决策行为的框架。ReMoDEx定义了一个逐步流程：模型推理、目标类别选择、相关性图生成、热图标准化、基于相似性的模式分组、聚类级别解释和空间相关性评估。局部方法GradCAM++、集成梯度、遮挡敏感性和层级相关传播各自独立地与一个单一的全局模块结合，该模块将整个相关性图集总结为几个决策策略集群，从而用自动化、可扩展的摘要替代逐样本检查。为了演示ReMoDEx，我们将其应用于一个基于VGG16的分类器，该分类器区分COVID-19、正常、肺部不透明和病毒性肺炎。该分类器表现出稳定的性能（86.27%的测试准确率，0.9624的测试AUC）。然而，每个解释器与全局模块结合后始终产生两个重复出现的策略：中央胸部区域决策和边缘/角落敏感决策，表明可能存在捷径学习，而传统指标无法揭示这一点。遮挡图像验证确认，当中央或外围区域被遮挡时，模型的置信度和预测类别发生了变化。因此，ReMoDEx提供了一个可扩展的基于相关性的决策评估框架，是基于准确率评估的重要补充。

View on arXiv Download PDF AI Translation

cs.CV / 24 / 2607.06909

Seeing What Matters: Lesion-Aware High-Resolution Patch Discovery and Fusion for Chest X-ray Report Generation

关注重要信息：基于病灶感知的高分辨率补丁发现与融合用于胸部X光报告生成

Li, Yingshu, Liu, Yunyi, Chen, Zhenghao, Chen, Tong, Chen, Zailong, Liu, Lingqiao, Wang, Lei, Zhou, Luping

Abstract

Despite rapid advances in chest X-ray (CXR) foundation models, most radiology report generation (RRG) systems still rely on heavily downsampled inputs (e.g., 256x256) due to the fixed visual token budgets of pretrained vision encoders, suppressing subtle yet clinically important cues present in native-resolution images. However, enabling high-resolution (high-res) perception remains challenging: naive tiling causes prohibitive token inflation, while global compression suppresses subtle lesions and degrades diagnostic fidelity. Inspired by radiologists' workflow, localizing suspicious regions before detailed high-res assessment. We propose Lesion-Aware High-Resolution Patch Discovery and Fusion for Chest X-ray Reporting (LePaX), the first RRG framework that enables efficient high-res CXR perception (up to 1920x1920) without increasing the vision-token count. LePaX formulates high-res perception as a constrained spatial resolution allocation problem under a fixed token budget and introduces two key components: Learnable Spatial Resolution Allocation (LSRA), which learns a spatial utility map that adaptively allocates limited high-res capacity to diagnostically relevant regions, enabling targeted extraction of high-res patches from native CXRs; and Global-Regional Fusion (GRF), which performs token-preserving region-to-global refinement by projecting high-resolution regional evidence back onto the global feature grid through spatially aligned resolution write-back, avoiding token inflation. Experiments on multiple CXR benchmarks demonstrate that LePaX consistently improves both clinical and linguistic metrics while enabling native-resolution CXR perception with over 10x fewer visual tokens than naive high-res tiling.

Chinese Translation

尽管胸部X光（CXR）基础模型快速发展，大多数放射学报告生成（RRG）系统仍依赖于大幅降采样的输入（例如，256x256），这是由于预训练视觉编码器的固定视觉标记预算，抑制了原始分辨率图像中存在的细微但临床重要的线索。然而，实现高分辨率（高分辨率）感知仍然具有挑战性：简单的平铺会导致不可接受的标记膨胀，而全局压缩则会抑制细微病灶并降低诊断准确性。受到放射科医生工作流程的启发，我们在详细的高分辨率评估之前定位可疑区域。我们提出了基于病灶感知的高分辨率补丁发现与融合用于胸部X光报告生成（LePaX），这是第一个能够在不增加视觉标记数量的情况下实现高效高分辨率CXR感知（高达1920x1920）的RRG框架。LePaX将高分辨率感知公式化为在固定标记预算下的约束空间分辨率分配问题，并引入两个关键组件：可学习空间分辨率分配（LSRA），它学习一个空间效用图，适应性地将有限的高分辨率容量分配给诊断相关区域，从而实现从原始CXR中有针对性地提取高分辨率补丁；以及全局-区域融合（GRF），它通过空间对齐的分辨率回写将高分辨率区域证据投影回全局特征网格，避免标记膨胀。在多个CXR基准测试中的实验表明，LePaX在实现原始分辨率CXR感知的同时，始终提高临床和语言指标，并且所需的视觉标记比简单的高分辨率平铺少超过10倍。

View on arXiv Download PDF AI Translation

cs.CV / 25 / 2607.06915

Smart Scissor: Coupling Spatial Redundancy Reduction and CNN Compression for Embedded Hardware

智能剪刀：结合空间冗余减少与CNN压缩的嵌入式硬件方案

Kong, Hao, Liu, Di, Huai, Shuo, Luo, Xiangzhong, Liu, Weichen, Subramaniam, Ravi, Makaya, Christian, Lin, Qian

Abstract

Scaling down the resolution of input images can greatly reduce the computational overhead of convolutional neural networks (CNNs), which is promising for edge AI. However, as an image usually contains much spatial redundancy, e.g., background pixels, directly shrinking the whole image will lose important features of the foreground object and lead to severe accuracy degradation. In this paper, we propose a dynamic image cropping framework to reduce the spatial redundancy by accurately cropping the foreground object from images. To achieve the instance-aware fine cropping, we introduce a lightweight foreground predictor to efficiently localize and crop the foreground of an image. The finely cropped images can be correctly recognized even at a small resolution. Meanwhile, computational redundancy also exists in CNN architectures. To pursue higher execution efficiency on resource-constrained embedded devices, we also propose a compound shrinking strategy to coordinately compress the three dimensions (depth, width, resolution) of CNNs. Eventually, we seamlessly combine the proposed dynamic image cropping and compound shrinking into a unified compression framework, Smart Scissor, which is expected to significantly reduce the computational overhead of CNNs while still maintaining high accuracy. Experiments on ImageNet-1K demonstrate that our method reduces the computational cost of ResNet50 by 41.5% while improving the top-1 accuracy by 0.3%. Moreover, compared to HRank, the state-of-the-art CNN compression framework, our method achieves 4.1% higher top-1 accuracy at the same computational cost. The codes and data are available at https://github.com/ntuliuteam/smart-scissor

Chinese Translation

缩小输入图像的分辨率可以大幅降低卷积神经网络（CNN）的计算开销，这对于边缘人工智能（Edge AI）具有良好的前景。然而，由于图像通常包含大量空间冗余，例如背景像素，直接缩小整个图像会导致前景物体的重要特征丢失，从而严重降低准确性。本文提出了一种动态图像裁剪框架，通过准确裁剪图像中的前景物体来减少空间冗余。为了实现实例感知的精细裁剪，我们引入了一种轻量级前景预测器，以高效定位和裁剪图像的前景。即使在较小的分辨率下，精细裁剪的图像也能被正确识别。同时，CNN架构中也存在计算冗余。为了在资源受限的嵌入式设备上追求更高的执行效率，我们还提出了一种复合缩减策略，以协调压缩CNN的三个维度（深度、宽度、分辨率）。最终，我们将所提出的动态图像裁剪和复合缩减无缝结合成一个统一的压缩框架，智能剪刀（Smart Scissor），预计将显著降低CNN的计算开销，同时保持高准确性。在ImageNet-1K上的实验表明，我们的方法在提高ResNet50的top-1准确率0.3%的同时，减少了41.5%的计算成本。此外，与最先进的CNN压缩框架HRank相比，我们的方法在相同计算成本下实现了4.1%的更高top-1准确率。代码和数据可在https://github.com/ntuliuteam/smart-scissor获取。

View on arXiv Download PDF AI Translation

cs.CV / 26 / 2607.06918

LoCA: Spatially-Aware Low-Rank Convolutional Adaptation of Vision Foundation Models

LoCA：空间感知的视觉基础模型低秩卷积适应

An, Sojung, Lee, Junha, You, Sujeong, Cho, Nam Ik, Kim, Donghyun

Abstract

Pre-trained Vision Foundation Models (VFMs) provide strong visual representations for diverse downstream tasks. The key challenge of VFM adaptation stems from the prohibitive costs of full fine-tuning and catastrophic forgetting. To address this, Low-Rank Adaptation (LoRA) has emerged as the prevailing paradigm for Parameter-Efficient Fine-Tuning (PEFT). However, LoRA is typically designed for transformer self-attention layers parameterized by 2D matrices. Since convolutional kernels inherently couple spatial and channel information within a 4D tensor, forcing them into a monolithic 2D matrix disrupts the inherent spatial topology. In this paper, we propose Low-Rank Convolutional Adaptation (LoCA), a convolution-aware PEFT framework that addresses spatial-channel entanglement by decoupling channel and spatial adaptation. LoCA introduces a low-rank channel adaptation for dense cross-channel mixing and refines spatial bases extracted from pre-trained kernels via Singular Value Decomposition (SVD). Experimental results show that LoCA preserves pre-trained spatial priors and achieves competitive or state-of-the-art performance across fine-grained classification, domain-generalized semantic segmentation, and generative benchmarks.

Chinese Translation

预训练的视觉基础模型（VFM）为多样的下游任务提供了强大的视觉表示。VFM 适应的主要挑战源于全面微调的高昂成本和灾难性遗忘。为了解决这个问题，低秩适应（LoRA）已成为参数高效微调（PEFT）的主流范式。然而，LoRA 通常是为由二维矩阵参数化的变换器自注意力层设计的。由于卷积核固有地将空间和通道信息耦合在一个四维张量中，将它们强行转化为单一的二维矩阵会破坏固有的空间拓扑。在本文中，我们提出了低秩卷积适应（LoCA），这是一种卷积感知的 PEFT 框架，通过解耦通道和空间适应来解决空间-通道纠缠。LoCA 引入了一种低秩通道适应方法，以实现密集的跨通道混合，并通过奇异值分解（SVD）精炼从预训练卷积核中提取的空间基。实验结果表明，LoCA 保持了预训练的空间先验，并在细粒度分类、领域泛化语义分割和生成基准测试中实现了具有竞争力或最先进的性能。

View on arXiv Download PDF AI Translation

cs.CV / 27 / 2607.06919

Compass: Prostate Cancer Detection Needs Multi-View Context

Compass：前列腺癌检测需要多视角上下文

Wilson, Paul F. R., Harmanani, Mohamed, Guo, Zhuoxin, Dzikunu, Obed K., Cash, Hannes, Kinnaird, Adam, Wodlinger, Brian, Abolmaesumi, Purang, Mousavi, Parvin

Abstract

Artificial intelligence (AI) analysis of micro-ultrasound ($\mu$US) has shown promise for prostate cancer (PCa) detection. However, most existing AI methods focus on the analysis of single $\mu$US images in isolation. By contrast, expert $\mu$US readers typically assess a full recorded video study, which provides three-dimensional context, to improve PCa detection compared to single-frame analysis. Inspired by this clinical workflow, we propose Compass, a novel AI methodology which models a $\mu$US study as a stream of 2D images. Compass jointly integrates rotational sweep videos of the prostate with $\mu$US frames acquired at the moment of biopsy, and performs evidence aggregation across the study using a transformer conditioned on the probe's rotational angle. Finally, a decoder head predicts frame-level and study-level risk scores for the patient. The model is trained and evaluated using a multi-center clinical trial dataset of $\mu$US studies, including continuous rotational scans of the prostate and videos captured during biopsy acquisition. We compare the proposed method to baseline AI methods from the literature and to risk scores provided by clinical experts. Our framework shows strong performance, highlighting the value of multi-view context for $\mu$US PCa detection, and providing a potentially powerful tool to complement human expertise in $\mu$US-based PCa diagnosis. Our code is available at: https://github.com/mharmanani/Compass.

Chinese Translation

人工智能（AI）对微超声（$$US）的分析在前列腺癌（PCa）检测中展现出良好的前景。然而，大多数现有的AI方法主要集中于对单幅$$US图像的孤立分析。相比之下，专业的$$US读者通常会评估完整的录制视频研究，这提供了三维上下文，从而改善相较于单帧分析的PCa检测效果。受到这一临床工作流程的启发，我们提出了Compass，这是一种新颖的AI方法论，将$$US研究建模为一系列2D图像。Compass将前列腺的旋转扫描视频与在活检时获取的$$US帧进行联合整合，并使用基于探头旋转角度的变换器在整个研究中进行证据聚合。最后，解码器头预测患者的帧级和研究级风险评分。该模型使用多中心临床试验数据集进行训练和评估，该数据集包括前列腺的连续旋转扫描和在活检获取过程中捕获的视频。我们将所提出的方法与文献中的基线AI方法以及临床专家提供的风险评分进行了比较。我们的框架表现出强大的性能，突显了多视角上下文在$$US PCa检测中的价值，并为基于$$US的PCa诊断提供了一个可能强大的工具，以补充人类的专业知识。我们的代码可在以下链接获取：https://github.com/mharmanani/Compass。

View on arXiv Download PDF AI Translation

cs.CV / 28 / 2607.06923

Bi-PT: Bidirectional Cross-Attention Point Transformers for Four-Chamber Heart Reconstruction from Sparse Cardiac MRI Data

Bi-PT：用于从稀疏心脏MRI数据重建四腔心的双向交叉注意力点变换器

Hu, Chenchuhui, Pan, Shaoming, Axel, Leon, Ye, Meng

Abstract

We propose Bi-PT, a pipeline for reconstructing 3D four-chamber human heart meshes from clinical sparsely sampled cardiac magnetic resonance imaging (CMR) data. This work addresses the error-prone generation of 3D cardiac shape from a sparse point cloud (SPC) extracted from 2D long-axis and short-axis views used in routine clinical CMR protocols. Bi-PT enables accurate inference of the four-chamber heart mesh from the SPC by learning robust point features via bidirectional point cross-attention between an atlas and the SPC, together with per-point semantic labels that improve correspondence estimation. We formulate the deformation field as a Neural Ordinary Differential Equation (NODE) parameterized by a per-point affine transformation and translation to deform the atlas toward the target heart shape. By learning such a NODE, we can guarantee the deformation field to be a locally affine diffeomorphic deformation. We also integrate a semantic label loss into the Chamfer distance to encourage label-consistent correspondences and add a smoothness regularization to stabilize and improve the learning of the deformation field. Extensive experiments demonstrate that Bi-PT achieves accurate and robust performance compared to baselines.

Chinese Translation

我们提出了Bi-PT，一个从临床稀疏采样的心脏磁共振成像（CMR）数据中重建三维四腔人心网格的管道。本研究解决了从常规临床CMR协议中使用的二维长轴和短轴视图提取的稀疏点云（SPC）生成三维心脏形状时容易出现的错误。Bi-PT通过在图谱和SPC之间进行双向点交叉注意力学习稳健的点特征，从而实现了从SPC准确推断四腔心网格，并结合每个点的语义标签以改善对应估计。我们将变形场公式化为一个由每个点的仿射变换和位移参数化的神经常微分方程（NODE），以将图谱变形为目标心脏形状。通过学习这样的NODE，我们可以确保变形场是局部仿射的微分同胚变形。我们还将语义标签损失集成到Chamfer距离中，以鼓励标签一致的对应关系，并添加平滑性正则化以稳定和改善变形场的学习。大量实验表明，与基线相比，Bi-PT实现了准确且稳健的性能。

View on arXiv Download PDF AI Translation

cs.CV / 29 / 2607.06943

General Incomplete Multimodal Learning via Dynamic Quality Perception

通过动态质量感知实现通用不完全多模态学习

Meng, Xiangyu, Wei, Shicai

Abstract

Multimodal learning robust to missing modalities is essential for real-world applications. Existing methods mainly focus on inter-modality missing, where entire modalities are absent, while overlooking intra-modality degradation, where modalities are present but severely corrupted. In practice, these two types of missing often coexist, making existing approaches ineffective. To address this limitation, we propose General Incomplete Multimodal Learning (GIML), a unified framework that simultaneously handles both inter-modality missing and intra-modality degradation through dynamic quality perception. Specifically, GIML models heterogeneous missing patterns as continuous modality information degradation, enabling degradation-aware adaptive fusion. To achieve reliable quality perception, we introduce a Noise-aware Quality Estimator that learns the mapping from corrupted features to noise intensity through controlled noise injection. Furthermore, we propose a Noise-Semantic Decoupled module that separates semantic information from noise interference. This improves robustness and generalization to unseen corruption patterns. Extensive experiments across datasets with diverse modality types demonstrate the effectiveness and generality of GIML. Code is available at: https://github.com/Yu-Five/GIML.

Chinese Translation

对缺失模态具有鲁棒性的多模态学习对于实际应用至关重要。现有方法主要集中于模态间缺失，即整个模态缺失，而忽视了模态内退化，即模态存在但严重损坏。在实际应用中，这两种缺失类型往往共存，使得现有方法效果不佳。为了解决这一局限性，我们提出了通用不完全多模态学习（General Incomplete Multimodal Learning, GIML），这是一个统一框架，能够通过动态质量感知同时处理模态间缺失和模态内退化。具体而言，GIML将异构缺失模式建模为连续的模态信息退化，从而实现退化感知的自适应融合。为了实现可靠的质量感知，我们引入了一种噪声感知质量估计器，该估计器通过控制噪声注入学习从损坏特征到噪声强度的映射。此外，我们提出了一种噪声-语义解耦模块，该模块将语义信息与噪声干扰分离。这提高了对未见腐蚀模式的鲁棒性和泛化能力。在具有多样化模态类型的数据集上进行的广泛实验表明，GIML的有效性和普适性。代码可在以下链接获取：https://github.com/Yu-Five/GIML。

View on arXiv Download PDF AI Translation

cs.CV / 30 / 2607.06948

Self-Supervised Pretraining Improves Cross-Site and Cross-Scale Robustness of Point Cloud Leaf-Wood Segmentation

自监督预训练提高了点云叶木分割的跨场地和跨尺度鲁棒性

Mun, Heeju, Yang, Tackang, Nam, Yunsoo, Choi, Changhyun

Abstract

The accuracy of existing leaf-wood segmentation methods for tree point clouds varies across forest types and sites. Self-supervised learning (SSL) on point clouds has improved the generalization of deep learning models for forestry point cloud tasks, including biomass regression and individual tree segmentation, but its applicability to leaf-wood segmentation remains untested. In this study, we pretrained Point-M2AE, a widely used SSL architecture for point clouds, on ShapeNet-55 augmented with 2,400 individual tree point clouds. For fine-tuning and inference, we used recursive voxel subdivision to handle the wide variation in point density across inputs, allowing the same model to operate at both individual-tree and plot scales without architecture change. Compared to the model without pretraining, the pretrained model improved wood IoU from 60.5% to 70.0% for needleleaf and from 69.7% to 76.3% for broadleaf trees. On a benchmark spanning four countries across three climatic zones, the pretrained model achieved the smallest cross-site variation and highest overall performance among compared methods (LeWos, CWLS, and PointTransformer). Plot-level segmentation maintained accuracy comparable to individual-tree performance, with mIoU of 84.7% for broadleaf and 77.7% for needleleaf plots, showing that the model generalizes across scales without additional finetuning. As a downstream test in tropical forests, where dense canopies make segmentation challenging, we applied our model and a quantitative structure model to estimate wood volume for 28 trees from Guyana, Indonesia, and Peru to assess whether the segmentation improvements from SSL pretraining translate into improved downstream performance. The resulting volume estimates achieved the lowest error among all methods tested (MAE = 2.40 m$^3$), less than half that of algorithmic baselines (LeWos: 5.94 m$^3$; CWLS: 5.27 m$^3$).

Chinese Translation

现有的树木点云叶木分割方法在不同森林类型和场地上的准确性存在差异。自监督学习（SSL）在点云上的应用提高了深度学习模型在林业点云任务（包括生物量回归和单棵树分割）的泛化能力，但其在叶木分割中的适用性尚未经过验证。在本研究中，我们在ShapeNet-55上对Point-M2AE这一广泛使用的点云自监督学习架构进行了预训练，并增加了2400个单棵树点云。为了进行微调和推断，我们使用递归体素细分来处理输入中点密度的广泛变化，使得同一模型能够在单棵树和样地尺度上运作而无需改变架构。与未预训练的模型相比，预训练模型在针叶树的木材IoU从60.5%提高到70.0%，在阔叶树的木材IoU从69.7%提高到76.3%。在涵盖三个气候区的四个国家的基准测试中，预训练模型在比较的方法（LeWos、CWLS和PointTransformer）中实现了最小的跨场地变化和最高的整体性能。样地级别的分割保持了与单棵树性能相当的准确性，阔叶样地的mIoU为84.7%，针叶样地的mIoU为77.7%，显示模型在不同尺度上无需额外微调即可泛化。作为在热带森林中的下游测试，由于密集的树冠使得分割具有挑战性，我们应用了我们的模型和一个定量结构模型来估算来自圭亚那、印度尼西亚和秘鲁的28棵树的木材体积，以评估SSL预训练带来的分割改进是否转化为下游性能的提升。最终的体积估算在所有测试方法中实现了最低的误差（MAE = 2.40 m$^3$），不到算法基线（LeWos: 5.94 m$^3$; CWLS: 5.27 m$^3$）的一半。

View on arXiv Download PDF AI Translation

cs.CV / 31 / 2607.06949

SpiS-GAN: Spiral-Modulated Handwriting Synthesis with Star Operation

SpiS-GAN：基于星操作的螺旋调制手写合成

Hieu, Nguyen Duy, Nam, Dang Hoai, Giap, Pham Hoang, Hieu, Quang Huu, Duy, Vo Nguyen Le

Abstract

Training robust handwriting recognition (HTR) systems requires massive amounts of annotated data, which is often difficult to acquire. While synthetic handwriting generation offers a practical solution to expand training sets, existing models struggle with several core issues. First, previous approaches, even MLP-based models fail to effectively trace cursive handwriting due to fixed-grid spatial receptive field. Second, their CNN-relied discriminators usually lose structural details through aggressive downsampling, making broken connections difficult to detect. Third, existing architectures are either limited to linear feature interactions or too expensive for high-resolution synthesis. Finally, existing approaches lack explicit edge constraints, often resulting in blurred stroke boundaries. To address these challenges, this study proposes a Spiral-Modulated Handwriting Synthesis framework based on Generative Adversarial Networks (SpiS-GAN). Our generator employs Star-Spiral Blocks combining proposed Modulated Elliptical SpiralFC with the star operation to capture spatial relationships and efficiently follow complex handwriting stroke trajectories, while a Spiral-Modulated discriminator is introduced for multi-domain flaws detection. Additionally, we introduce a Sobel-Regularized Edge Reconstruction Loss that provides edge guidance, ensuring every character remains clear and legible. Evaluations on the English and Vietnamese datasets demonstrate that SpiS-GAN significantly outperforms current state-of-the-art models. The generated images are highly authentic, accurately preserve the original writer's style across languages, and successfully lower error rates when training downstream HTR systems.

Chinese Translation

训练稳健的手写识别（HTR）系统需要大量的标注数据，而这些数据往往难以获取。尽管合成手写生成为扩展训练集提供了实用的解决方案，但现有模型在几个核心问题上表现不佳。首先，以往的方法，即使是基于多层感知器（MLP）的模型，也无法有效追踪连笔手写，因为其固定网格的空间感受野限制了表现。其次，它们依赖卷积神经网络（CNN）的判别器通常通过激进的下采样丧失结构细节，使得断开的连接难以检测。第三，现有架构要么仅限于线性特征交互，要么在高分辨率合成时成本过高。最后，现有方法缺乏明确的边缘约束，常常导致笔画边界模糊。为了解决这些挑战，本研究提出了一种基于生成对抗网络（Generative Adversarial Networks）的螺旋调制手写合成框架（SpiS-GAN）。我们的生成器采用结合了所提议的调制椭圆螺旋全连接（Modulated Elliptical SpiralFC）与星操作的星螺旋块，以捕捉空间关系并有效跟随复杂的手写笔画轨迹，同时引入了螺旋调制判别器用于多领域缺陷检测。此外，我们引入了一种索贝尔正则化边缘重建损失（Sobel-Regularized Edge Reconstruction Loss），为边缘提供指导，确保每个字符清晰可读。在对英语和越南语数据集的评估中，SpiS-GAN显著优于当前最先进的模型。生成的图像高度真实，准确保留了原作者在不同语言中的风格，并成功降低了训练下游HTR系统时的错误率。

View on arXiv Download PDF AI Translation

cs.CV / 32 / 2607.06972

HPR-SAM: Hierarchical Probabilistic Representation Learning for Prompt-free SAM-based Medical Image Segmentation

HPR-SAM：基于分层概率表示学习的无提示SAM医学图像分割

Hu, Yingzhen, Zhong, Yiheng, Zhu, Keying, Zhang, Zimu, Ye, Zihan, Song, Sifan, Su, Jionglong, Liu, Xiaofeng

Abstract

Prompt-free adaptation of the Segment Anything Model (SAM) has emerged as a promising paradigm for automatic medical image segmentation. Existing methods mainly focus on prompt generation, while overlooking that prompt quality is fundamentally constrained by the expressiveness of anatomical representations. However, deterministic prototypes or semantic tokens are insufficient to jointly capture global anatomical priors, intra-structure diversity, and local structural reliability. To address this limitation, we propose the Hierarchical Probabilistic Representation (HPR) framework, which learns complementary anatomical representations through Distributional Anatomical Representation (DAR), Multi-component Anatomical Representation (MAR), and Local Reliability Representation (LRR), and integrates their predictions via Hierarchical Prediction Fusion (HPF) while remaining compatible with the original SAM decoder. Experiments on the Synapse, LA, and PROMISE12 datasets demonstrate that HPR-SAM achieves state-of-the-art performance on Synapse and the best performance under few-shot settings on LA and PROMISE12, validating the effectiveness of the proposed hierarchical probabilistic representation learning framework for prompt-free medical image segmentation. Code is available at https://anonymous.4open.science/r/HPR-SAM-E4AF.

Chinese Translation

无提示的Segment Anything Model (SAM) 适应已成为自动医学图像分割的一个有前景的范式。现有方法主要集中在提示生成上，而忽视了提示质量在根本上受到解剖表示表达能力的限制。然而，确定性原型或语义标记不足以共同捕捉全局解剖先验、结构内部多样性和局部结构可靠性。为了解决这一限制，我们提出了分层概率表示 (HPR) 框架，通过分布式解剖表示 (DAR)、多组分解剖表示 (MAR) 和局部可靠性表示 (LRR) 学习互补的解剖表示，并通过分层预测融合 (HPF) 整合它们的预测，同时与原始SAM解码器保持兼容。在Synapse、LA和PROMISE12数据集上的实验表明，HPR-SAM在Synapse上实现了最先进的性能，并在LA和PROMISE12的少量样本设置下达到了最佳性能，验证了所提出的分层概率表示学习框架在无提示医学图像分割中的有效性。代码可在 https://anonymous.4open.science/r/HPR-SAM-E4AF 获取。

View on arXiv Download PDF AI Translation

cs.CV / 33 / 2607.06982

EdgeCompress: Coupling Multidimensional Model Compression and Dynamic Inference for EdgeAI

EdgeCompress：将多维模型压缩与动态推理结合用于边缘人工智能

Kong, Hao, Liu, Di, Huai, Shuo, Luo, Xiangzhong, Subramaniam, Ravi, Makaya, Christian, Lin, Qian, Liu, Weichen

Abstract

Convolutional neural networks (CNNs) have demonstrated encouraging results in image classification tasks. However, the prohibitive computational cost of CNNs hinders the deployment of CNNs onto resource-constrained embedded devices. To address this issue, we propose EdgeCompress, a comprehensive compression framework to reduce the computational overhead of CNNs. In EdgeCompress, we first introduce dynamic image cropping (DIC), where we design a lightweight foreground predictor to accurately crop the most informative foreground object of input images for inference, which avoids redundant computation on background regions. Subsequently, we present compound shrinking (CS) to collaboratively compress the three dimensions (depth, width, and resolution) of CNNs according to their contribution to accuracy and model computation. DIC and CS together constitute a multidimensional CNN compression framework, which is able to comprehensively reduce the computational redundancy in both input images and neural network architectures, thereby improving the inference efficiency of CNNs. Further, we present a dynamic inference framework to efficiently process input images with different recognition difficulties, where we cascade multiple models with different complexities from our compression framework and dynamically adopt different models for different input images, which further compresses the computational redundancy and improves the inference efficiency of CNNs, facilitating the deployment of advanced CNNs onto embedded hardware. Experiments on ImageNet-1K demonstrate that EdgeCompress reduces the computation of ResNet-50 by 48.8% while improving the top-1 accuracy by 0.8%. Meanwhile, we improve the accuracy by 4.1% with similar computation compared to HRank, the state-of-the-art compression framework. The source code and models are available at https://github.com/ntuliuteam/edge-compress

Chinese Translation

卷积神经网络（CNN）在图像分类任务中表现出令人鼓舞的结果。然而，CNN的高计算成本阻碍了其在资源受限的嵌入式设备上的部署。为了解决这个问题，我们提出了EdgeCompress，一个综合压缩框架，用于减少CNN的计算开销。在EdgeCompress中，我们首先引入动态图像裁剪（Dynamic Image Cropping, DIC），设计了一种轻量级前景预测器，以准确裁剪输入图像中最具信息量的前景物体进行推理，从而避免对背景区域的冗余计算。随后，我们提出了复合缩减（Compound Shrinking, CS），根据CNN的准确性和模型计算的贡献，协同压缩CNN的三个维度（深度、宽度和分辨率）。DIC和CS共同构成了一个多维CNN压缩框架，能够全面减少输入图像和神经网络架构中的计算冗余，从而提高CNN的推理效率。此外，我们提出了一个动态推理框架，以高效处理具有不同识别难度的输入图像，在该框架中，我们级联多个来自压缩框架的不同复杂度的模型，并动态选择不同的模型来处理不同的输入图像，进一步压缩计算冗余，提高CNN的推理效率，促进先进CNN在嵌入式硬件上的部署。在ImageNet-1K上的实验表明，EdgeCompress将ResNet-50的计算减少了48.8%，同时提升了0.8%的Top-1准确率。同时，与最先进的压缩框架HRank相比，我们在相似计算下提高了4.1%的准确率。源代码和模型可在https://github.com/ntuliuteam/edge-compress获取。

View on arXiv Download PDF AI Translation

cs.CV / 34 / 2607.07001

Ego-Human Motion Prediction with 3D-Aware LLM

基于3D感知的自我人类运动预测

Bae, Yujin, Jeong, Jaewoo, Kim, Hyeonseong, Yoon, Kuk-Jin

Abstract

Anticipating human motion from an egocentric perspective is fundamental for proactive assistance in AR/VR, human-robot collaboration, and embodied AI. While recent works incorporate language as a semantic prior to reduce the ill-posed nature of egocentric forecasting, they largely neglect the 3D spatial and semantic context that governs how motion unfolds, and treat pose and language prediction as separate inference streams. We introduce Ego3DLM, built on two core principles: accurate motion forecasting requires explicit spatial and semantic understanding of the 3D environment, and pose and language must be predicted holistically in a single pass, since motion is inherently tied to the semantic interpretation of actions being performed. Given three-point tracking, 3D scene features, and egocentric video, Ego3DLM simultaneously decodes past pose, future pose, past narration, and future narration in a single autoregressive pass, grounding predicted poses and descriptions in one another to enforce cross-modal and temporal consistency. We adopt a three-stage training scheme: (1) spatial-semantic scene awareness pretraining; (2) holistic instruction tuning over all four outputs in a single pass; and (3) GRPO-based reinforcement finetuning with intra- and inter-modal rewards that directly optimize pose-language fidelity. Experiments on the Nymeria benchmark demonstrate that Ego3DLM achieves state-of-the-art performance across future motion prediction, past motion tracking, and motion description, showing that 3D scene grounding and holistic cross-modal prediction yield physically plausible and semantically coherent motion forecasts. The project page is available at https://jaewoo97.github.io/Ego3DLM/.

Chinese Translation

从自我中心的视角预测人类运动对于增强现实/虚拟现实（AR/VR）、人机协作和具身人工智能的主动辅助至关重要。尽管近期的研究将语言作为语义先验以减少自我中心预测的病态特性，但它们在很大程度上忽视了支配运动展开的3D空间和语义上下文，并将姿态和语言预测视为独立的推理流。我们提出了Ego3DLM，基于两个核心原则：准确的运动预测需要对3D环境的明确空间和语义理解，并且姿态和语言必须在单次推理中整体预测，因为运动本质上与所执行动作的语义解释密切相关。在给定三点跟踪、3D场景特征和自我中心视频的情况下，Ego3DLM在单次自回归推理中同时解码过去的姿态、未来的姿态、过去的叙述和未来的叙述，将预测的姿态和描述相互关联，以强化跨模态和时间的一致性。我们采用三阶段训练方案：（1）空间-语义场景意识的预训练；（2）在单次推理中对所有四个输出进行整体指令调优；（3）基于GRPO的强化微调，利用内部和外部模态奖励直接优化姿态-语言的一致性。在Nymeria基准上的实验表明，Ego3DLM在未来运动预测、过去运动跟踪和运动描述方面达到了最先进的性能，表明3D场景基础和整体跨模态预测能够产生物理上合理且语义上连贯的运动预测。项目页面可访问：https://jaewoo97.github.io/Ego3DLM/

View on arXiv Download PDF AI Translation

cs.CV / 35 / 2607.07019

SHTA: Semantic Hard Token Correction and Center Alignment for Semi-Supervised Medical Image Segmentation

SHTA：用于半监督医学图像分割的语义硬标记修正与中心对齐

Zhang, Zhuoru, Zhong, Yiheng, Zhang, Zimu, Liu, Xiaofeng

Abstract

Recent advances in semi-supervised medical image segmentation have achieved remarkable performance through prediction consistency, pseudo-label supervision, and hard-region supervision. However, these methods primarily improve supervision quality rather than explicitly enforcing semantic consistency in the learned representations of hard regions. Consequently, even under increasingly stronger prediction-level supervision, difficult regions exhibiting unstable semantic assignment often fail to establish semantically consistent representations during training, thereby limiting further segmentation improvement. To address this issue, we propose SHTA (Semantic Hard Token Correction and Center Alignment), a lightweight training-time semantic representation branch. Instead of introducing additional prediction supervision, SHTA refines intermediate semantic representations through Semantic Assignment, Hard Token Refinement, and Semantic Center Alignment, thereby improving semantic consistency in hard regions while preserving the original prediction pathway and introducing no additional inference cost. We integrate SHTA into representative semi-supervised segmentation frameworks, including GA-CPS, CPS, URPC, and MagicNet, and conduct evaluations on the Synapse and AMOS datasets. Experimental results demonstrate that SHTA delivers consistent paired improvements across frameworks, with especially clear gains in segmentation accuracy, weak-organ recovery, and semantic ambiguity reduction, while incurring only training-time overhead. The code is available at https://anonymous.4open.science/r/release_SHTA-42D5/.

Chinese Translation

近年来，半监督医学图像分割的进展通过预测一致性、伪标签监督和硬区域监督取得了显著的性能。然而，这些方法主要提高了监督质量，而不是明确地在学习到的硬区域表示中强制执行语义一致性。因此，即使在越来越强的预测级监督下，表现出不稳定语义分配的困难区域在训练过程中往往无法建立语义一致的表示，从而限制了进一步的分割改进。为了解决这个问题，我们提出了SHTA（语义硬标记修正与中心对齐），这是一个轻量级的训练时语义表示分支。SHTA通过语义分配、硬标记精炼和语义中心对齐来细化中间语义表示，而不是引入额外的预测监督，从而提高硬区域的语义一致性，同时保留原始预测路径且不引入额外的推理成本。我们将SHTA集成到代表性的半监督分割框架中，包括GA-CPS、CPS、URPC和MagicNet，并在Synapse和AMOS数据集上进行了评估。实验结果表明，SHTA在各框架中提供了一致的配对改进，尤其在分割准确性、弱器官恢复和语义模糊度减少方面取得了明显的提升，同时仅增加了训练时的开销。代码可在 https://anonymous.4open.science/r/release_SHTA-42D5/ 获取。

View on arXiv Download PDF AI Translation

cs.CV / 36 / 2607.07033

AnchorPrune: Relevance-Anchored Contextual Expansion for Visual Token Pruning

AnchorPrune：基于相关性的上下文扩展用于视觉标记剪枝

Oh, Kyuan, Kim, Bumsoo

Abstract

Large vision-language models incur substantial inference costs because high-resolution inputs introduce thousands of visual tokens, many of which are redundant for a given query. Existing pruning methods often combine query relevance and token diversity, yet these objectives can conflict under aggressive compression: relevance-driven selection may overconcentrate the budget on correlated local evidence, while diversity-driven selection may suppress indispensable tokens or retain distinct but uninformative regions. We introduce AnchorPrune, a training-free framework that first constructs a protected relevance anchor and then expands it with complementary visual context. AnchorPrune adaptively determines the anchor size from the novelty profile of relevance-ranked tokens, preserving a compact set of query-critical evidence, and allocates the remaining budget through importance-weighted novelty to recover informative, non-redundant context relative to the anchor. This ordered design prevents contextual expansion from displacing indispensable query cues while improving overall visual coverage. AnchorPrune is lightweight, architecture-aware, and requires neither retraining nor model modification. Across image and video vision-language models and benchmarks, it consistently improves the accuracy-efficiency trade-off over training-free baselines, particularly under severe compression. On LLaVA-NeXT-7B, AnchorPrune preserves 97.6% of full-token performance using only 160 of 2,880 visual tokens. These results establish relevance-anchored contextual expansion as an effective principle for efficient multimodal inference. Code is available at https://github.com/MULTI-cau/AnchorPrune.

Chinese Translation

大型视觉-语言模型因高分辨率输入引入数千个视觉标记而产生巨大的推理成本，其中许多对于特定查询来说是冗余的。现有的剪枝方法通常结合查询相关性和标记多样性，但在激进压缩下，这些目标可能会发生冲突：以相关性为驱动的选择可能会过度集中预算于相关的局部证据，而以多样性为驱动的选择可能会抑制不可或缺的标记或保留不同但无信息的区域。我们提出了AnchorPrune，这是一种无训练框架，首先构建一个受保护的相关性锚点，然后用互补的视觉上下文扩展它。AnchorPrune根据相关性排名标记的新颖性特征自适应地确定锚点大小，保留一组紧凑的查询关键证据，并通过重要性加权的新颖性分配剩余预算，以恢复相对于锚点的信息丰富且非冗余的上下文。这种有序设计防止了上下文扩展取代不可或缺的查询线索，同时改善了整体视觉覆盖。AnchorPrune 轻量级、架构感知，并且不需要重新训练或模型修改。在图像和视频视觉-语言模型及基准测试中，它在训练无关的基线之上持续改善了准确性与效率的权衡，特别是在严重压缩的情况下。在 LLaVA-NeXT-7B 上，AnchorPrune 仅使用 160 个视觉标记就保留了 97.6% 的全标记性能。这些结果确立了基于相关性的上下文扩展作为高效多模态推理的有效原则。代码可在 https://github.com/MULTI-cau/AnchorPrune 获取。

View on arXiv Download PDF AI Translation

cs.CV / 37 / 2607.07038

TRACE-Seg3D: Counterfactual Context Auditing For Robust 3D Glioma Segmentation Under Institutional Shift

TRACE-Seg3D：应对机构变化的反事实上下文审计以实现稳健的3D胶质瘤分割

Le, Nguyen Linh Dan, Le, Nguyen Pham Hoang, Khoi, Tran Dang

Abstract

Medical image segmentation models can achieve strong benchmark performance while remaining sensitive to scanner, protocol, and institutional variation. These context shifts alter image appearance without changing the underlying lesion, allowing models to exploit nuisance cues that Dice and HD95 fail to expose. We present TRACE-Seg3D, a counterfactual context auditing framework for robust 3D medical image segmentation. TRACE-Seg3D preserves lesion-relevant evidence and systematically varies imaging context to quantify prediction stability under controlled context shifts. The framework pairs each segmentation with audit evidence for context sensitivity and anatomical plausibility, enabling case-level reliability assessment beyond overlap-based evaluation. Experiments on BraTS and UTSW glioma segmentation benchmarks demonstrate competitive in-distribution and cross-domain performance. TRACE-Seg3D also exposes context-sensitive failure modes missed by conventional metrics. These results establish counterfactual context auditing as a practical route toward transparent and reliable 3D medical image segmentation under distribution shift. Our code is available at https://github.com/danleneurocom/Counterfactual-Representation-Network.

Chinese Translation

医学图像分割模型可以在基准测试中取得良好的表现，但对扫描仪、协议和机构的变化仍然敏感。这些上下文变化会改变图像外观，而不改变潜在病变，使得模型能够利用Dice和HD95未能揭示的干扰线索。我们提出了TRACE-Seg3D，这是一种用于稳健的3D医学图像分割的反事实上下文审计框架。TRACE-Seg3D保留与病变相关的证据，并系统性地变化成像上下文，以量化在受控上下文变化下的预测稳定性。该框架将每个分割与上下文敏感性和解剖合理性的审计证据配对，使得超越基于重叠的评估进行案例级可靠性评估成为可能。在BraTS和UTSW胶质瘤分割基准上的实验表明，TRACE-Seg3D在分布内和跨领域的表现具有竞争力。TRACE-Seg3D还揭示了传统指标未能捕捉的上下文敏感失效模式。这些结果确立了反事实上下文审计作为在分布变化下实现透明和可靠的3D医学图像分割的实用途径。我们的代码可在https://github.com/danleneurocom/Counterfactual-Representation-Network获取。

View on arXiv Download PDF AI Translation

cs.CV / 38 / 2607.07051

Making Implicit Preservation Intent Explicit in Conversational Image Editing

在对话式图像编辑中使隐含的保留意图显性化

Han, Soomin, Ahn, Jihyung, Kim, Bumsoo, Chang, Buru

Abstract

Conversational image editing requires preserving not only visible content, but also content that temporarily disappears across turns. When newly added or modified content occludes a previously visible region, that region should reappear if it was never semantically changed. However, existing systems often fail to recover such occluded-but-unchanged content, producing inconsistent or hallucinated results. We introduce OCCUR-Bench, a diagnostic benchmark for temporal preservation in conversational image editing. OCCUR-Bench provides diverse occlusion-and-revelation scenarios with historical restoration references, enabling evaluation of faithful restoration rather than plausible regeneration. We also propose ReSpec, a training-free framework that makes implicit preservation explicit by pairing restoration-aware instructions with historical visual references. Given an editing history, ReSpec identifies what should persist, selects the historical image state that provides missing visual evidence, and conditions an in-context editor on the resulting instruction and reference image. Experiments show that ReSpec improves restoration fidelity and temporal consistency on OCCUR-Bench, highlighting the need to ground preservation in editing history rather than only the current image.

Chinese Translation

对话式图像编辑不仅需要保留可见内容，还需要保留在不同回合中暂时消失的内容。当新添加或修改的内容遮挡了先前可见的区域时，如果该区域在语义上没有发生变化，则应重新显示该区域。然而，现有系统往往无法恢复这些被遮挡但未改变的内容，从而产生不一致或虚构的结果。我们引入了 OCCUR-Bench，这是一个用于对话式图像编辑中时间保留的诊断基准。OCCUR-Bench 提供了多样的遮挡与显现场景，并附有历史恢复参考，能够评估忠实恢复而非可信再生。我们还提出了 ReSpec，这是一个无需训练的框架，通过将关注恢复的指令与历史视觉参考配对，使隐含的保留意图显性化。给定编辑历史，ReSpec 确定应持续存在的内容，选择提供缺失视觉证据的历史图像状态，并根据生成的指令和参考图像对上下文编辑器进行条件设置。实验表明，ReSpec 在 OCCUR-Bench 上提高了恢复的保真度和时间一致性，强调了将保留根植于编辑历史而不仅仅是当前图像的必要性。

View on arXiv Download PDF AI Translation

cs.CV / 39 / 2607.07077

Navigating Hierarchy: Hyperbolic Learning on Brain Graphs for Disorder Diagnosis

导航层级：基于脑图的超曲率学习用于疾病诊断

Li, Yapeng, Jiang, Bo, Zhang, Ziyan, Chen, Dongdong, Tu, Zhengzheng

Abstract

Functional brain networks exhibit a hierarchical organization across ROI, community, and whole-brain levels, supporting local processing, inter-community coordination, and global integration. Recent studies have demonstrated that brain community-aware modeling is beneficial for both diagnosis and biomarker identification of brain networks. However, existing brain graph modeling methods often struggle to model ROI-community interactions, thereby failing to fully exploit the hierarchy across ROI, community, and whole-brain network levels. To address this issue, inspired by deep hyperbolic learning in modeling hierarchical structures, we propose a novel framework, termed Hyperbolic Learning on Brain Graphs (HLBG), for brain network analysis. The core idea of HLBG is to exploit the inherent hierarchical geometry of hyperbolic space to model the hierarchical relationships among ROIs, functional communities, and the whole-brain network, thereby learning hierarchy-aware and highly discriminative representations for brain network data. Specifically, HLBG first projects representations from ROIs, communities, and the whole-brain network into Lorentzian hyperbolic space. Then, the multi-level hierarchy is imposed via two geometric entailment constraints. In addition, we introduce a new Graph-aware Mamba (GaMamba) model, which incorporates topology-derived structural prompts into Mamba to capture long-range dependencies while preserving graph topological information. Experiments on ABIDE-I and REST-MDD demonstrate that HLBG outperforms state-of-the-art methods and identifies disorder-relevant functional biomarkers.

Chinese Translation

功能性脑网络在感兴趣区域（ROI）、社区和全脑层面上展现出层级组织，支持局部处理、社区间协调和全球整合。近期研究表明，关注脑社区的建模对于脑网络的诊断和生物标志物识别具有重要意义。然而，现有的脑图建模方法往往难以有效建模ROI与社区之间的交互，未能充分利用ROI、社区和全脑网络层面之间的层级关系。为了解决这一问题，受到深度超曲率学习在建模层级结构方面的启发，我们提出了一种新颖的框架，称为基于脑图的超曲率学习（Hyperbolic Learning on Brain Graphs, HLBG），用于脑网络分析。HLBG的核心思想是利用超曲率空间固有的层级几何结构来建模ROI、功能社区和全脑网络之间的层级关系，从而学习对脑网络数据具有层级感知和高度区分性的表示。具体而言，HLBG首先将来自ROI、社区和全脑网络的表示投影到洛伦兹超曲率空间中。然后，通过两个几何蕴含约束施加多层次层级。此外，我们引入了一种新的图感知Mamba（Graph-aware Mamba, GaMamba）模型，该模型将基于拓扑的结构提示纳入Mamba，以捕捉长距离依赖关系，同时保留图的拓扑信息。在ABIDE-I和REST-MDD上的实验表明，HLBG优于最先进的方法，并识别出与疾病相关的功能生物标志物。

View on arXiv Download PDF AI Translation

cs.CV / 40 / 2607.07091

AT-Attn: Temporal-Aware Cross-Attention for Longitudinal Multimodal Alzheimer's Disease Diagnosis

AT-Attn：用于纵向多模态阿尔茨海默病诊断的时间感知交叉注意力

Du, Xinyue, Liu, Yibo, Zhou, Zhenglei, Yao, Xuancheng, Zhong, Weimin, Chen, Qiuhui

Abstract

In longitudinal Alzheimer's disease (AD) diagnosis support, clinical and imaging information is often collected at irregular visits. Integrating these multimodal observations may improve diagnostic assessment, but naive fusion can degrade performance when MRI is noisy or intermittently unavailable. We propose AT-Attn, a temporal-aware multimodal framework that combines Change-and-Time encoding, time-biased asymmetric cross-attention, and gated fusion to integrate MRI with longitudinal clinical information. We evaluate AT-Attn on an MRI-retained ADNI cohort of 1,520 patients using structural MRI, six cognitive-scale trajectories, and seven static clinical variables under patient-level five-fold cross-validation. The main asymmetric AT-Attn model achieves accuracy 0.719+/-0.024, macro F1 0.721+/-0.023, ROC-AUC 0.873+/-0.013, and PR-AUC 0.783+/-0.018, outperforming unimodal and naive multimodal fusion baselines while remaining competitive with strong tabular baselines. These results suggest that a temporal-aware and constrained fusion strategy can help structural MRI contribute clinically relevant complementary information for patient-level AD diagnosis support.

Chinese Translation

在纵向阿尔茨海默病（AD）诊断支持中，临床和影像信息通常在不规则的就诊中收集。整合这些多模态观察可能会改善诊断评估，但简单的融合在MRI噪声较大或间歇性不可用时可能会降低性能。我们提出了AT-Attn，一种时间感知的多模态框架，结合了变化与时间编码、时间偏向的非对称交叉注意力和门控融合，以将MRI与纵向临床信息整合。我们在一个包含1,520名患者的MRI保留ADNI队列上评估了AT-Attn，使用结构性MRI、六个认知量表轨迹和七个静态临床变量，并在患者级别进行五折交叉验证。主要的非对称AT-Attn模型实现了准确率0.719±0.024，宏观F1值0.721±0.023，ROC-AUC 0.873±0.013，以及PR-AUC 0.783±0.018，超越了单模态和简单多模态融合的基线，同时在强表格基线中保持竞争力。这些结果表明，时间感知和受限的融合策略可以帮助结构性MRI为患者级别的AD诊断支持提供临床相关的互补信息。

View on arXiv Download PDF AI Translation

cs.CV / 41 / 2607.07099

Video-Based Detection of squint and cataract for accessibility-aware adaptive web interface rendering

基于视频的斜视和白内障检测用于无障碍适应性网页界面渲染

Dash, Amar Ranjan, Patra, Manas Ranjan

Abstract

Squint and cataract are major ocular disorders that majorly affect visual perception and interaction capability. This paper proposes a real-time video-based automated detection system for squint and cataract detection based on computer vision and image processing methods. The proposed system uses a media-pipe face-mesh (a 478-point facial landmark detection model) to extract geometric ocular features for multi-class squint classification. Simultaneously, The presence and severity cataract is estimated through grayscale intensity and histogram-based lens opacity analysis. The system records short video sequences with standard laptop or mobile cameras, which can be deployed at low costs and on a large scale. The experimental performance has shown great accuracy in the detection of squint (98.39%) and classification of cataract (96.90%). Besides automatic ocular analysis, the proposed framework is also made accessible for visual impairment inference which will be integrated with future adaptive user interface and Web accessibility systems for people with visual impairment.

Chinese Translation

斜视和白内障是主要影响视觉感知和交互能力的眼部疾病。本文提出了一种基于计算机视觉和图像处理方法的实时视频自动检测系统，用于斜视和白内障的检测。该系统使用媒体管道面部网格（media-pipe face-mesh，478点面部标志检测模型）提取几何眼部特征，以进行多类别斜视分类。同时，通过灰度强度和基于直方图的晶状体混浊分析来估计白内障的存在及其严重程度。该系统使用标准笔记本电脑或移动摄像头录制短视频序列，能够以低成本大规模部署。实验性能显示在斜视检测（98.39%）和白内障分类（96.90%）方面具有很高的准确性。除了自动眼部分析外，所提出的框架还为视觉障碍推断提供了可访问性，将与未来的适应性用户界面和无障碍网页系统集成，以服务于视觉障碍人士。

View on arXiv Download PDF AI Translation

cs.CV / 42 / 2607.07117

Tree-of-Thoughts Reasoning for Text-to-Image In-Context Learning

基于思维树的文本到图像上下文学习推理

Alekseeva, Stepanida, Kalafatovich, Jenifer, Lee, Seong-Whan

Abstract

In text-to-image in-context learning (T2I-ICL), a model has to infer a latent compositional pattern from fewshot demonstrations for generating a query image. Recent studies show that state-of-the-art multimodal large language models struggle with this setting, particularly due to limited compositional reasoning and sensitivity to prompt construction. In this work, we propose a Tree-of-Thoughts (ToT) reasoning framework for T2I-ICL that introduces a multi-stage reasoning and selection layer that generates, evaluates, and selects among multiple candidate hypotheses before constructing the final prompt for image synthesis. By exploring alternative reasoning branches and selecting a coherent interpretation, the proposed approach mitigates prompt ambiguity and compositional errors. We implement the proposed approach in a complete ToT-T2IICL inference pipeline and evaluate it on the CoBSAT benchmark. Both qualitative and quantitative results show that structured multi-branch reasoning leads to more consistent and semantically aligned image generation compared to baseline and Chain-of-Thought prompting strategies, without any additional training or fine-tuning.

Chinese Translation

在文本到图像上下文学习（T2I-ICL）中，模型需要从少量示例中推断出潜在的组合模式，以生成查询图像。近期研究表明，最先进的多模态大型语言模型在这一设置下表现不佳，特别是由于组合推理能力有限以及对提示构造的敏感性。在本研究中，我们提出了一种基于思维树（Tree-of-Thoughts, ToT）的推理框架，用于T2I-ICL，该框架引入了一个多阶段推理和选择层，在构建最终图像合成提示之前，生成、评估并选择多个候选假设。通过探索替代推理分支并选择一致的解释，所提出的方法减轻了提示的模糊性和组合错误。我们在完整的ToT-T2IICL推理管道中实现了该方法，并在CoBSAT基准上进行了评估。定性和定量结果均表明，与基线和链式思维（Chain-of-Thought）提示策略相比，结构化的多分支推理能够实现更一致且语义对齐的图像生成，而无需额外的训练或微调。

View on arXiv Download PDF AI Translation

cs.CV / 43 / 2607.07119

ColorFM: An Optimization-to-Learning Framework for Color Transfer via Flow Matching

ColorFM：一种通过流匹配进行颜色转移的优化到学习框架

He, Yuhang, Zhang, Kai, Li, Xiaoming, Chen, Du, Yang, Jian

Abstract

Color transfer aims to align the color distribution of a source image with that of a reference image while preserving structural and semantic consistency. However, existing methods often suffer from inaccurate global mapping, semantic misalignment, and visual artifacts. To address these issues, we propose ColorFM, an optimization-to-learning framework. ColorFM connects online optimization to offline inference by reformulating color transfer as the transport of pixel distributions along velocity fields via Flow Matching. Specifically, we introduce ColorFM-O, an instance-specific optimization scheme that fits the velocity field through hierarchical color coupling guided by semantic priors. By numerically integrating the induced flow trajectories, ColorFM-O produces precise and semantically consistent color transfer results, while generating high-quality paired data as pseudo-supervision. Building upon this, we design ColorFM-L, an efficient feed-forward model trained on the generated pairs. Through implicit state modeling, ColorFM-L extracts deep semantic features to predict flow parameters for bidirectional linearized transport, ensuring accurate color transfer. Extensive experiments demonstrate that ColorFM-L outperforms state-of-the-art methods in visual quality, structural fidelity, and semantic consistency, successfully combining the accuracy of optimization with the speed of feed-forward inference.

Chinese Translation

颜色转移旨在将源图像的颜色分布与参考图像的颜色分布对齐，同时保持结构和语义的一致性。然而，现有方法往往面临不准确的全局映射、语义错位和视觉伪影等问题。为了解决这些问题，我们提出了ColorFM，一种优化到学习的框架。ColorFM通过将颜色转移重新表述为沿速度场的像素分布传输，将在线优化与离线推理连接起来。具体而言，我们引入了ColorFM-O，一种实例特定的优化方案，通过语义先验指导的层次颜色耦合来拟合速度场。通过数值积分诱导的流轨迹，ColorFM-O生成精确且语义一致的颜色转移结果，同时生成高质量的配对数据作为伪监督。在此基础上，我们设计了ColorFM-L，一种在生成的配对数据上训练的高效前馈模型。通过隐式状态建模，ColorFM-L提取深层语义特征以预测双向线性化传输的流参数，从而确保准确的颜色转移。大量实验表明，ColorFM-L在视觉质量、结构保真度和语义一致性方面优于最先进的方法，成功地将优化的准确性与前馈推理的速度相结合。

View on arXiv Download PDF AI Translation

cs.CV / 44 / 2607.07123

Widest-Path Reachability Fields for Connectivity-Preserving Slender Structure Segmentation

用于保持连通性的细长结构分割的最宽路径可达性场

Zong, Youcheng, Jia, Runda, Hu, Minxuan, Su, Weilan, He, Dakuo

Abstract

Segmenting slender curvilinear structures such as retinal vessels, cracks, and roads demands topological correctness, as even a single-pixel discontinuity can fragment a continuous network and invalidate downstream analysis. Under standard binary-mask supervision, models optimized for pixel-level overlap frequently produce topologically broken predictions. We trace this to a fundamental mismatch: pixel-wise losses distribute gradients uniformly, yet connectivity hinges on a sparse set of bottleneck pixels. These pixels are vastly outnumbered by thick structures and background, rendering their aggregate gradient contribution negligible. We term this phenomenon topological gradient starvation (TGS). To address it, we propose Widest-Path Reachability Fields (WPRF), a differentiable Max-Min reachability objective that redirects gradient flow to connectivity bottlenecks. The module is plug-and-play, backbone-agnostic, and incurs no inference overhead. WPRF implements a differentiable Max-Min objective via dynamic programming on a domain-restricted graph, coupled with a bottleneck-aware observation term that balances gradient contributions across varying structures. Compared to prior topology-aware losses that rely on post-hoc skeletonization or homology computation, WPRF directly optimizes end-to-end reachability via differentiable Max-Min algebra, enabling gradient flow to concentrate on connectivity bottlenecks without auxiliary structures. We introduce OMVIS, a new oral microvessel segmentation dataset. Experiments across nine architectures and six datasets validate the bottleneck-focused gradient routing mechanism. WPRF improves 87\% of experiments with fixed hyperparameters and achieves clDice gains of 7.2 percentage points on structurally fragile datasets.

Chinese Translation

细长曲线结构的分割，如视网膜血管、裂缝和道路，要求拓扑正确性，因为即使是单像素的不连续性也可能破坏连续网络并使下游分析失效。在标准的二进制掩膜监督下，针对像素级重叠优化的模型常常产生拓扑破损的预测。我们将其归因于一个基本的不匹配：像素级损失均匀分配梯度，而连通性则依赖于稀疏的瓶颈像素。这些像素的数量远少于厚结构和背景，使得它们的总梯度贡献微不足道。我们将这种现象称为拓扑梯度饥饿（Topological Gradient Starvation, TGS）。为了解决这个问题，我们提出了最宽路径可达性场（Widest-Path Reachability Fields, WPRF），这是一种可微分的最大-最小可达性目标，能够将梯度流引导至连通性瓶颈。该模块即插即用，与主干网络无关，并且不会增加推理开销。WPRF通过在域限制图上进行动态规划实现可微分的最大-最小目标，并结合一个关注瓶颈的观察项，平衡不同结构之间的梯度贡献。与依赖事后骨架化或同源计算的先前拓扑感知损失相比，WPRF通过可微分的最大-最小代数直接优化端到端的可达性，使梯度流集中于连通性瓶颈，而无需辅助结构。我们引入了OMVIS，一个新的口腔微血管分割数据集。在九种架构和六个数据集上的实验验证了以瓶颈为中心的梯度路由机制。WPRF在固定超参数的87%的实验中表现出改善，并在结构脆弱的数据集上实现了7.2个百分点的clDice增益。

View on arXiv Download PDF AI Translation

cs.CV / 45 / 2607.07135

Sparse Attention for Dense Open-Vocabulary Prediction in CLIP

CLIP中的稀疏注意力用于密集开放词汇预测

Zohra, Fatimah, Zhao, Chen, Liu, Shuming, Ghanem, Bernard

Abstract

Contrastive Language-Image Pre-training (CLIP) relies on softmax-based self-attention, a strictly positive distribution that assigns probability mass to every pair of tokens-even semantically irrelevant ones. While these dense softmax weights are effective for gathering broad context during pre-training, they spread attention across many low-salience tokens, producing noise that obscures the fine-grained, spatially localized cues required for dense, open-vocabulary prediction. We study an inference-time substitution of the row-wise softmax in the final visual self-attention layers with the $\alpha$-entmax transform, applied across both the standard query-key attention and self-correlation variants. Because entmax applies a data-dependent threshold that maps low scores exactly to zero, it acts as an implicit denoiser, zeroing contextually irrelevant dependencies while redistributing mass onto the most relevant tokens. We evaluate on open-vocabulary tasks-dense semantic segmentation (Pascal VOC, Pascal Context, ADE20K) and fine-grained retrieval (FG-OVD)-and find the gain from attention sparsification is proportional to how much the baseline attention spreads off the target class.

Chinese Translation

对比语言-图像预训练（CLIP）依赖于基于softmax的自注意力，这是一种严格正值分布，为每对标记分配概率质量——即使是语义上无关的标记。虽然这些密集的softmax权重在预训练期间有效地收集了广泛的上下文，但它们将注意力分散到许多低显著性的标记上，产生了噪声，模糊了进行密集开放词汇预测所需的细粒度、空间局部线索。我们研究了在推理时将最终视觉自注意力层中的行级softmax替换为$eta$-entmax变换，该变换应用于标准查询-键注意力和自相关变体。由于entmax应用了一个数据依赖的阈值，将低分数精确映射为零，因此它作为一种隐式去噪器，消除了上下文无关的依赖，同时将质量重新分配到最相关的标记上。我们在开放词汇任务上进行评估——密集语义分割（Pascal VOC、Pascal Context、ADE20K）和细粒度检索（FG-OVD）——发现注意力稀疏化带来的增益与基线注意力偏离目标类别的程度成正比。

View on arXiv Download PDF AI Translation

cs.CV / 46 / 2607.07161

ASFR-Net: Adversarial Alignment and Spatio-Frequency Refinement Network for Heterogeneous Remote Sensing Image Change Detection

ASFR-Net：用于异构遥感图像变化检测的对抗性对齐与时空频率精炼网络

Wu, Xin-Jie, You, Zhi-Hui, Chen, Si-Bao, Shu, Qing-Ling, Wang, Xiao, Tang, Jin, Luo, Bin

Abstract

The core challenge of heterogeneous change detection in remote sensing imagery lies in effectively decoupling genuine land-cover changes from significant modal disparities caused by distinct imaging mechanisms. These intrinsic inconsistencies are prone to introducing pseudo-changes, thereby constraining detection accuracy. To address this, we propose a novel, end-to-end adversarial spatio-frequency refinement network (ASFR-Net). Initially, a modality-invariant representation learner (MIR-Learner) guides the backbone to extract modality-invariant features, effectively bridging the primary domain gap. Subsequently, to address persistent residual modal differences, we design an innovative spatio-frequency synergistic enhancement module (SFEM), which identifies and suppresses sensor-specific noise and artifacts that are difficult to discern in the spatial domain by leveraging frequency-domain processing. Multi-level difference features are then computed from these refined representations and fed into a decoder equipped with cascaded hierarchical guided fusion module (HGFM) blocks to generate precise change maps. To alleviate the data scarcity in heterogeneous tasks, we construct and release a new high-resolution benchmark specifically focused on building changes: the visible-near-infrared heterogeneous change detection (VisNIR-HCD) dataset. It presents unique scientific challenges arising from deceptive visual similarity and non-linear spectral inversions, providing a robust platform for evaluating model generalization. Extensive experiments on VisNIR-HCD and public datasets demonstrate that ASFR-Net achieves state-of-the-art (SOTA) performance, significantly outperforming existing methods. The source code and the VisNIR-HCD dataset are publicly available at https://github.com/LuoYang2024/ASFR-Net.

Chinese Translation

异构变化检测在遥感图像中的核心挑战在于有效地将真实的土地覆盖变化与由于不同成像机制引起的显著模态差异解耦。这些内在的不一致性容易引入伪变化，从而限制检测精度。为此，我们提出了一种新颖的端到端对抗性时空频率精炼网络（ASFR-Net）。最初，模态不变表示学习器（MIR-Learner）引导主干网络提取模态不变特征，有效地弥合主要领域差距。随后，为了解决持续存在的残余模态差异，我们设计了一个创新的时空频率协同增强模块（SFEM），该模块通过利用频域处理识别并抑制在空间域中难以辨别的传感器特定噪声和伪影。然后，从这些精炼的表示中计算多级差异特征，并输入到配备级联分层引导融合模块（HGFM）块的解码器中，以生成精确的变化图。为了缓解异构任务中的数据稀缺问题，我们构建并发布了一个新的高分辨率基准数据集，专注于建筑变化：可见-近红外异构变化检测（VisNIR-HCD）数据集。该数据集呈现出由于视觉相似性和非线性光谱反演而产生的独特科学挑战，为评估模型的泛化能力提供了一个强有力的平台。在VisNIR-HCD和公共数据集上的广泛实验表明，ASFR-Net达到了最先进的（SOTA）性能，显著优于现有方法。源代码和VisNIR-HCD数据集已公开发布，网址为https://github.com/LuoYang2024/ASFR-Net。

View on arXiv Download PDF AI Translation

cs.CV / 47 / 2607.07168

NoDrift3R: Raymap-Guided Coupling for Drift-Robust Unposed Feed-Forward 3D Reconstruction

NoDrift3R：基于光线图引导的耦合方法用于抗漂移的无姿态前馈三维重建

Sun, Xiangyu, Liu, Liu, Yang, Seungkwon, Han, Jingbing, Nam, Seungtae, Su, Zhizhong, Park, Eunbyung

Abstract

Pose-Free Feed-forward 3D Gaussian Splatting (3DGS) has recently emerged as a powerful paradigm for fast scene reconstruction. However, its performance degrades significantly in long image sequences due to cumulative camera pose estimation drift, which propagates errors into geometric modeling and severely limits rendering fidelity. In this work, we revisit the long-sequence bottleneck and identify pose drift as the primary factor restricting reconstruction quality. Furthermore, while SfM-based pseudo ground-truth poses introduce sensor noise, purely rendering-based supervision often leads to optimization instability and local minima due to the entangled optimization of geometry and pose. To address the challenges, we propose a synergistic pose-free framework that explicitly couples geometry and appearance via a Raymap-Guided Coupling Module (RGC). Concretely, we anchor Gaussian centers to raymap-induced geometry and jointly optimize RGB reconstruction, raymap consistency, and camera regularization under a unified objective, yielding a bidirectional feedback loop: stronger geometry improves rendering, and appearance supervision in turn refines geometry and pose. To further stabilize learning across wide temporal ranges, we introduce a Dual-Frequency Viewpoint Scheduling strategy that combines easy-to-hard interval expansion with replay of short-interval pairs. Extensive experiments across in-domain and cross-domain datasets show consistent gains in both rendering and pose estimation, with notably improved robustness on long sequences. Ablation studies validate our central insight: explicitly designed geometry-appearance synergy is the key to scalable and drift-robust pose-free feed-forward 3D reconstruction.

Chinese Translation

无姿态前馈三维高斯点云（3D Gaussian Splatting, 3DGS）最近成为快速场景重建的强大范式。然而，由于累积的相机姿态估计漂移，其在长图像序列中的性能显著下降，这种漂移将误差传播到几何建模中，严重限制了渲染的保真度。在本研究中，我们重新审视了长序列瓶颈，并确定姿态漂移是限制重建质量的主要因素。此外，基于结构从运动（Structure from Motion, SfM）的伪地面真实姿态引入了传感器噪声，而纯粹基于渲染的监督往往由于几何与姿态的交织优化导致优化不稳定和局部最小值。为了解决这些挑战，我们提出了一种协同的无姿态框架，通过光线图引导耦合模块（Raymap-Guided Coupling Module, RGC）显式耦合几何和外观。具体而言，我们将高斯中心锚定到光线图引导的几何上，并在统一目标下共同优化RGB重建、光线图一致性和相机正则化，从而产生一个双向反馈循环：更强的几何改善渲染，而外观监督反过来又细化几何和姿态。为了进一步稳定跨越广泛时间范围的学习，我们引入了一种双频视点调度策略，该策略结合了由易到难的间隔扩展与短间隔对的重放。在领域内和跨领域数据集上的大量实验显示，在渲染和姿态估计方面均有一致的提升，尤其在长序列上表现出显著的鲁棒性。消融研究验证了我们的核心见解：显式设计的几何-外观协同是可扩展和抗漂移的无姿态前馈三维重建的关键。

View on arXiv Download PDF AI Translation

cs.CV / 48 / 2607.07169

TACoS: Weakly Supervised Learning of Two-Dimensional Materials from Scribble Annotations to Precise Segmentation

TACoS：从涂鸦标注到精确分割的二维材料弱监督学习

Chen, Jiabei, Zhang, Liping, Wu, Jiang-Bin, Wei, Zhongming, Ning, Enhao, Yan, Su, Li, Weijun, Tan, Ping-Heng, Ning, Xin

Abstract

The precise pixel-level localization of 2D material flakes is crucial for high-throughput screening. However, traditional fully supervised methods rely on dense annotations, which are costly and time-consuming, severely limiting the practical deployment of segmentation models. This paper proposes TACoS, a specialized scribble segmentation framework tailored for 2D materials. First, we design a unified framework that integrates semi-supervised consistency learning with structured tree energy constraints. This framework comprises two core components: an unlabeled weak-strong distribution alignment module and a tree energy regularization module. The former employs cosine consistency constraints to enhance prediction alignment across views. Meanwhile, the latter utilizes minimum spanning trees to establish pixel affinity relationships and generate structure-aware soft pseudo labels for online semantic guidance. Next, we introduce asymmetric regional contrast learning. This approach fuses high-confidence predictions from the weak augmentation branch with scribbles to form augmented labels, and construct category prototypes in the representation space. Simultaneously, we prioritize contrastive constraints on challenging pixels in boundary-unlabeled regions. This strategy enhances intra-class cohesion and inter-class separation at the representation level, effectively reducing category confusion in low-contrast edges and complex backgrounds. Experiments conducted on the constructed graphene and MoS2 datasets demonstrate that our method TACoS achieves over 96% of fully supervised performance using less than 0.6% annotated data. Furthermore, it exhibits superior structural coherence and boundary stability in scenarios with weakly contrasting edges and complex backgrounds, providing an efficient and scalable solution for automated high-throughput screening of 2D material flakes.

Chinese Translation

二维材料薄片的精确像素级定位对于高通量筛选至关重要。然而，传统的完全监督方法依赖于密集标注，这既成本高昂又耗时，严重限制了分割模型的实际应用。本文提出了TACoS，一个专门针对二维材料的涂鸦分割框架。首先，我们设计了一个统一框架，将半监督一致性学习与结构化树能量约束相结合。该框架由两个核心组件组成：一个无标签的弱强分布对齐模块和一个树能量正则化模块。前者采用余弦一致性约束来增强不同视图之间的预测对齐。同时，后者利用最小生成树建立像素亲和关系，并生成结构感知的软伪标签以进行在线语义指导。接下来，我们引入了非对称区域对比学习。这种方法将来自弱增强分支的高置信度预测与涂鸦结合，形成增强标签，并在表示空间中构建类别原型。同时，我们优先对边界未标记区域中的挑战性像素施加对比约束。这一策略在表示层面增强了类内凝聚力和类间分离性，有效减少了低对比度边缘和复杂背景下的类别混淆。在构建的石墨烯和MoS2数据集上进行的实验表明，我们的方法TACoS在使用不到0.6%的标注数据的情况下，达到了超过96%的完全监督性能。此外，在边缘对比度弱和背景复杂的场景中，它展现出优越的结构一致性和边界稳定性，为二维材料薄片的自动化高通量筛选提供了一种高效且可扩展的解决方案。

View on arXiv Download PDF AI Translation

cs.CV / 49 / 2607.07170

PUF: Plug-and-Play Uncertainty-Aware Fusion for Online 3D Scene Graph Generation

PUF：即插即用的不确定性感知融合框架用于在线3D场景图生成

Yang, Yi, Castillo, Myrna, Rosenhahn, Bodo, Yang, Michael Ying

Abstract

Online 3D scene graph generation builds a persistent, structured representation of a scene by incrementally fusing 2D observations into a global 3D graph. Existing online methods treat this fusion as a fully deterministic pipeline, where we identify three sources of uncertainty that are overlooked: observation, 2D model, and 3D representation. We propose PUF: a Plug-and-play, Uncertainty-aware, and training-free Fusion framework. Scene graph node association is reformulated as a probabilistic likelihood over semantic and spatial factors, replacing binary accept/reject gates. Dirichlet evidence accumulation distributes class and relationship evidence across plausible candidates proportional to association likelihood. An optional class-conditional prior completes edges for sparsely or never co-observed object pairs. We instantiate PUF with both a 3D Gaussian and a 3D voxel backend and observe consistent improvements, demonstrating its ability to generalize across different representations. Experiments on the 3DSSG and ReplicaSSG benchmarks show that our method substantially outperforms existing approaches while maintaining real-time latency. These results establish uncertainty-aware fusion as a principled and effective paradigm for online 3D scene understanding. The source code is publicly available at https://github.com/yyyyangyi/PUF.

Chinese Translation

在线3D场景图生成通过将2D观测逐步融合为全局3D图，构建场景的持久结构化表示。现有的在线方法将这种融合视为一个完全确定性的流程，而我们识别出三个被忽视的不确定性来源：观测、不确定的2D模型和3D表示。我们提出了PUF：一个即插即用、不确定性感知且无需训练的融合框架。场景图节点关联被重新表述为对语义和空间因素的概率似然，替代了二元接受/拒绝门。Dirichlet证据积累将类别和关系证据按关联似然分配到合理的候选者中。可选的类别条件先验为稀疏或从未共同观测的物体对完成边缘。我们用3D高斯和3D体素后端实例化PUF，并观察到一致的改进，证明其在不同表示间的泛化能力。在3DSSG和ReplicaSSG基准上的实验表明，我们的方法在保持实时延迟的同时显著优于现有方法。这些结果确立了不确定性感知融合作为在线3D场景理解的一个原则性和有效的范式。源代码可在https://github.com/yyyyangyi/PUF公开获取。

View on arXiv Download PDF AI Translation

cs.CV / 50 / 2607.07173

Stage-Aware Adaptation and Distribution Calibration for Subject-Driven Personalized Text-to-Image Generation

基于阶段感知的适应与分布校准用于主体驱动的个性化文本到图像生成

Xu, Wenyan, Wong, Alizer

Abstract

Subject-driven personalized text-to-image generation requires a pretrained diffusion model to acquire a specific subject from a few reference images while preserving subject identity, following novel text prompts, and maintaining sample diversity. Existing optimization-based methods instantiate subject adaptation through full fine-tuning, textual embedding optimization, or low-rank parameter updates; PaRa further constrains personalization from the perspective of parameter rank reduction. However, a uniform low-rank constraint or a uniform adapter strength cannot explicitly distinguish the capacity requirements of different denoising stages. Moreover, inference-time candidate selection driven mainly by identity similarity may compress the selected samples in the visual representation space. We decompose the problem into two complementary components: SPaRa denotes training-side stage-aware low-rank adaptation, DCAL denotes inference-side distribution-calibrated candidate selection, and SPaRa-DCAL denotes the combined framework. Theoretical analysis shows that timestep-dependent scaling controls the effective perturbation magnitude of a low-rank adapter, while identity-biased candidate selection restricts the radius of selected features around the reference center under explicit conditions. Auditable experiments under the SDXL and DreamBooth 30-subject protocol show that DCAL improves 1-LPIPS, CLIP-I, DINO-I, and CLIP-T on a fixed LoRA candidate pool, while revealing a clear trade-off with CLIP/DINO pairwise diversity and pairwise LPIPS. These results indicate that personalized generation should be evaluated through identity consistency, text alignment, and representation diversity rather than identity metrics alone.

Chinese Translation

主体驱动的个性化文本到图像生成需要一个预训练的扩散模型，从少量参考图像中获取特定主体，同时保持主体身份，遵循新的文本提示，并维持样本多样性。现有的基于优化的方法通过完全微调、文本嵌入优化或低秩参数更新来实现主体适应；而PaRa则从参数秩降低的角度进一步约束个性化。然而，统一的低秩约束或统一的适配器强度无法明确区分不同去噪阶段的能力需求。此外，主要由身份相似性驱动的推理时候候选选择可能会在视觉表示空间中压缩所选样本。我们将问题分解为两个互补的组成部分：SPaRa表示训练阶段感知的低秩适应，DCAL表示推理阶段的分布校准候选选择，而SPaRa-DCAL则表示组合框架。理论分析表明，时间步依赖的缩放控制低秩适配器的有效扰动幅度，而身份偏向的候选选择在明确条件下限制了所选特征围绕参考中心的半径。在SDXL和DreamBooth 30主体协议下的可审计实验表明，DCAL在固定的LoRA候选池上提高了1-LPIPS、CLIP-I、DINO-I和CLIP-T，同时揭示了与CLIP/DINO成对多样性和成对LPIPS之间的明显权衡。这些结果表明，个性化生成应通过身份一致性、文本对齐和表示多样性进行评估，而不仅仅是身份指标。

View on arXiv Download PDF AI Translation

cs.CV / 51 / 2607.07179

Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering

适应领域的视觉语言模型在一般文档视觉问答中的比较研究

Lopez-Duran, Miguel, Marrero, Elena, Fierrez, Julian, Robledo-Moreno, Marta, Vera-Rodriguez, Ruben, DeAlcala, Daniel, Morales, Aythami, Tolosana, Ruben, Delgado, Oscar, Ortigosa, Alvaro, Ortega-Garcia, Javier

Abstract

Document Visual Question Answering (DocVQA) presents a complex multimodal challenge, requiring models to exploit visual, textual, and layout information from documents. Although Vision-Language Models (VLMs) have shown remarkable performance in text-vision tasks, their robustness and transferability to different document domains remains underexplored. In this study, we present a comprehensive evaluation of 8 open-source pretrained VLMs on DocVQA in three different document domains: industrial documents of varying type, infographics, and presentation slides. We systematically assess model performance under zero-shot evaluations, fully supervised finetuning with inter- and intra-dataset evaluations, and few-shot learning evaluations of knowledge transfer between domains. Our findings demonstrate that while large pretrained VLMs possess strong zero-shot baselines for structured layouts, their performance strongly decreases on visually complex layouts of infographics and slides. Although parameter scaling is a dominant factor on performance, supervised finetuning yields higher relative gains in smaller architectures. Furthermore, our cross-domain and few-shot experiments show that visual understanding is the main bottleneck for DocVQA, not a lack of knowledge from the VLMs. Using 50 target domain samples, the models finetuned in DocVQA with datasets of different domains rapidly adapt to the target domain documents, even surpassing their fully supervised counterparts in some cases.

Chinese Translation

文档视觉问答（DocVQA）呈现出复杂的多模态挑战，要求模型利用文档中的视觉、文本和布局信息。尽管视觉语言模型（VLMs）在文本-视觉任务中表现出色，但它们在不同文档领域的鲁棒性和可迁移性仍然未得到充分探索。在本研究中，我们对8个开源预训练VLMs在三个不同文档领域的DocVQA进行了全面评估：不同类型的工业文档、信息图表和演示幻灯片。我们系统地评估了模型在零-shot评估、完全监督微调下的跨数据集和同数据集评估，以及领域间知识迁移的少-shot学习评估中的表现。我们的研究结果表明，尽管大型预训练VLMs在结构化布局上具有强大的零-shot基线，但在信息图表和幻灯片的视觉复杂布局上，其性能显著下降。虽然参数规模是影响性能的主导因素，但在较小架构中，监督微调带来了更高的相对增益。此外，我们的跨领域和少-shot实验表明，视觉理解是DocVQA的主要瓶颈，而非VLMs缺乏知识。使用50个目标领域样本，在不同领域数据集上微调的模型能够迅速适应目标领域文档，甚至在某些情况下超越其完全监督的对应模型。

View on arXiv Download PDF AI Translation

cs.CV / 52 / 2607.07187

EditVerse3D: High-Quality 3D Object Editing with Region-Aware Learning

EditVerse3D：基于区域感知学习的高质量3D物体编辑

Yin, Youtan, Zhou, Yanning, Wei, Jiacheng, Yang, Xiaofeng, Zhang, Jun, Bai, Jiayang, Ye, Jingwen, Zhang, Weidong, Lin, Guosheng

Abstract

Local editing of 3D objects remains a long-standing challenge. When interacting with 3D content, humans naturally tend to specify a coarse region of interest for modification rather than defining precise editing boundaries. However, previous methods rely on fully edited 2D images, precise 3D masks, or redundant pipelines, which present a gap. To bridge this gap, we propose EditVerse3D, a novel 3D editing framework that enables high-quality object editing under such coarse guidance. Our approach takes as input a 3D object to be edited, a coarse 3D bounding box indicating the target region, and a reference 2D image describing the desired modification. It produces a coherent, high-fidelity edited 3D object. To facilitate this editing, we introduce a novel region-aware adaptive loss that emphasizes hard-to-learn regions and balances the objective between target and preserved areas. Complementing our loss function, we enhance model robustness and generalization through targeted data augmentations, such as training with scaled 3D masks and filtering out unrealistic editing pairs. We construct a large-scale 3D editing dataset derived from parts information. Extensive experiments demonstrate that EditVerse3D achieves superior visual quality and quantitative performance compared to existing 3D editing approaches. Please visit our project page at https://editverse3d.github.io.

Chinese Translation

3D物体的局部编辑一直是一个长期存在的挑战。在与3D内容交互时，人类自然倾向于指定一个粗略的感兴趣区域进行修改，而不是定义精确的编辑边界。然而，以往的方法依赖于完全编辑的2D图像、精确的3D掩膜或冗余的处理流程，这造成了一个差距。为了弥补这一差距，我们提出了EditVerse3D，一个新颖的3D编辑框架，能够在这种粗略指导下实现高质量的物体编辑。我们的方法以待编辑的3D物体、指示目标区域的粗略3D边界框以及描述所需修改的参考2D图像作为输入，生成一个连贯的、高保真的编辑3D物体。为了促进这一编辑过程，我们引入了一种新颖的区域感知自适应损失，强调难以学习的区域，并在目标区域和保留区域之间平衡目标。作为我们损失函数的补充，我们通过有针对性的数据增强提高模型的鲁棒性和泛化能力，例如使用缩放的3D掩膜进行训练，并过滤掉不现实的编辑对。我们构建了一个基于部件信息的大规模3D编辑数据集。大量实验表明，EditVerse3D在视觉质量和定量性能上优于现有的3D编辑方法。请访问我们的项目页面 https://editverse3d.github.io。

View on arXiv Download PDF AI Translation

cs.CV / 53 / 2607.07192

Prototype-Anchored Generalized Manifold Regression for Unknown-Domain Object Detection

原型锚定的广义流形回归用于未知领域物体检测

Zhang, Zihao, Wu, Aming, Li, Yang, Han, Yahong

Abstract

In this paper, we study Single-Domain Generalized Object Detection (Single-DGOD), which aims to transfer a detector trained on a single source domain to multiple unseen domains. Existing methods mainly rely on simulation-driven strategies, such as data augmentation or textual prompts, to enlarge the training distribution. However, finite simulations can hardly cover the dynamic variations of real-world scenarios, often causing overfitting to synthetic styles and limited robustness to complex structural degradations. Inspired by the manifold hypothesis, we argue that semantic features, despite diverse visual changes, should lie on a compact and stable low-dimensional manifold. Therefore, robust generalization requires rectifying deviant samples back to this semantic manifold, rather than exhaustively simulating external perturbations. To this end, we propose Manifold Regression with Visual-Text Dual Chain-of-Thought (MR-DCoT), which formulates unknown-domain generalization as a manifold regression problem. MR-DCoT first uses a Visual-Text Dual Chain-of-Thought module to combine VLM-guided semantic evolution with diffusion-based structural perturbation, generating structured off-manifold hard examples. It then introduces Class-Specific Prototype Anchoring to learn a rectification operator that projects deviant features toward the source semantic manifold. By integrating outlier generation and semantic correction into a closed loop, MR-DCoT effectively narrows the distribution gap and improves robustness under unseen shifts. Extensive experiments on three complementary benchmarks, including adverse-weather detection, real-to-art generalization, and zero-shot semantic segmentation, demonstrate the effectiveness and versatility of our method.

Chinese Translation

在本文中，我们研究了单领域广义物体检测（Single-DGOD），其目标是将训练于单一源领域的检测器转移到多个未见领域。现有方法主要依赖于模拟驱动策略，如数据增强或文本提示，以扩大训练分布。然而，有限的模拟很难覆盖真实场景的动态变化，常常导致对合成风格的过拟合以及对复杂结构退化的鲁棒性不足。受到流形假设的启发，我们认为，尽管视觉变化多样，语义特征应位于一个紧凑且稳定的低维流形上。因此，稳健的泛化需要将偏离样本校正回这个语义流形，而不是穷举地模拟外部扰动。为此，我们提出了带有视觉-文本双链思维的流形回归（MR-DCoT），将未知领域的泛化形式化为流形回归问题。MR-DCoT首先使用视觉-文本双链思维模块，将基于视觉语言模型（VLM）的语义演变与基于扩散的结构扰动相结合，生成结构化的流形外难例。然后，它引入类别特定的原型锚定，学习一个校正算子，将偏离特征投影到源语义流形上。通过将异常值生成和语义校正整合到一个闭环中，MR-DCoT有效缩小了分布差距，并提高了在未见变化下的鲁棒性。在三个互补基准上的大量实验，包括恶劣天气检测、真实到艺术的泛化和零样本语义分割，证明了我们方法的有效性和多样性。

View on arXiv Download PDF AI Translation

cs.CV / 54 / 2607.07195

DiffCVE: Diffusion-based Compressed Video Enhancement

DiffCVE：基于扩散的压缩视频增强

Xiao, Wenqiang, Ma, Wenzhuo, Zhang, Junxi, Chen, Zhenzhong

Abstract

Perceptual quality enhancement of severely compressed videos remains challenging due to complex artifact patterns and substantial information loss. Recent diffusion models have demonstrated strong generative capability for visual restoration, but directly applying them to compressed video often ignores compression degradation characteristics and may introduce structure-inconsistent hallucinations. To address this issue, this paper presents a diffusion-based compressed video enhancement method, named DiffCVE. Coding Prior-enhanced Dual Conditioning (CPDC) branches are designed to jointly model compressed video and coding prior conditions, where coding priors including residuals and motion vectors provide complementary structural and motion guidance during the diffusion denoising process. To make the diffusion process aware of compression severity, a Compression Degradation Semantic Prompting (CDSP) mechanism is introduced to leverage QP-conditioned textual prompts together with LoRA fine-tuning. In addition, a Coding Prior-guided Weighted Fusion (CPWF) module is incorporated into the VAE decoder to fuse VAE encoder and coding prior encoder features with QP-predicted weights. Extensive experiments demonstrate the effectiveness of the proposed method in improving perceptual quality, especially under severe compression settings. The project page with enhanced video demonstrations is available at https://wqmaker.github.io/projects/DiffCVE/.

Chinese Translation

由于复杂的伪影模式和大量信息损失，严重压缩视频的感知质量增强仍然具有挑战性。最近的扩散模型在视觉恢复方面展示了强大的生成能力，但直接将其应用于压缩视频往往忽视了压缩退化特征，并可能引入结构不一致的幻觉。为了解决这一问题，本文提出了一种基于扩散的压缩视频增强方法，命名为DiffCVE。设计了编码先验增强的双重条件（CPDC）分支，以共同建模压缩视频和编码先验条件，其中编码先验包括残差和运动矢量，在扩散去噪过程中提供互补的结构和运动指导。为了使扩散过程意识到压缩严重性，引入了一种压缩退化语义提示（CDSP）机制，利用QP条件的文本提示以及LoRA微调。此外，将编码先验引导的加权融合（CPWF）模块纳入VAE解码器，以根据QP预测的权重融合VAE编码器和编码先验编码器的特征。大量实验表明，所提方法在改善感知质量方面的有效性，特别是在严重压缩设置下。增强视频演示的项目页面可访问：https://wqmaker.github.io/projects/DiffCVE/

View on arXiv Download PDF AI Translation

cs.CV / 55 / 2607.07216

Why Fake ? Unveiling the Semantic Vocabulary of Deepfake Detectors

为什么是伪造？揭示深度伪造检测器的语义词汇

Vanian, Vazgken, Doumanoglou, Alexandros, Zarpalas, Dimitris

Abstract

Deepfake (DF) technology poses a significant threat to information integrity, driving the need for robust detection methods. Most DF detectors only consider predicting a binary label for whether the input is real or fake, lacking the justification required for real-world applications like legal proceedings. Explainable DF Detection has emerged to address this limitation, but existing techniques frequently fall short by either relying on human annotations for precise artifact localization or generating superficially plausible textual explanations without grounding. This work investigates the use of post-hoc explainable AI (XAI) to analyze the decision-making process of state-of-the-art black-box DF detectors. Specifically, we employ Encoding-Decoding Direction Pairs (EDDP), a technique suitable for uncovering the concept space of DF detectors (their semantic vocabulary) as well as the mechanism for writing and reading concept information to and from internal representations. Our analysis reveals previously hidden real and fake features learned implicitly during detector training, offering nuanced explanations unattainable through conventional methods. This enables global model understanding, spatially aware concept localization, and counterfactual what-if analysis, all contributing to a deeper comprehension of DF detection strategies.

Chinese Translation

深度伪造（Deepfake, DF）技术对信息完整性构成了重大威胁，推动了对强大检测方法的需求。大多数DF检测器仅考虑预测输入是真实的还是伪造的二元标签，缺乏在法律程序等现实应用中所需的合理性。可解释的DF检测应运而生以解决这一局限性，但现有技术往往依赖于人工注释以实现精确的伪造特征定位，或生成表面上似是而非的文本解释而缺乏实质依据。本研究探讨了后验可解释人工智能（post-hoc explainable AI, XAI）的使用，以分析最先进的黑箱DF检测器的决策过程。具体而言，我们采用编码-解码方向对（Encoding-Decoding Direction Pairs, EDDP）这一技术，旨在揭示DF检测器的概念空间（其语义词汇）以及从内部表示中写入和读取概念信息的机制。我们的分析揭示了在检测器训练过程中隐式学习到的真实和伪造特征，提供了通过传统方法无法获得的细致解释。这使得全球模型理解、空间感知的概念定位和反事实的假设分析成为可能，所有这些都促进了对DF检测策略的更深入理解。

View on arXiv Download PDF AI Translation

cs.CV / 56 / 2607.07219

Vision Foundation Models in Radiology: A Scoping Review of Data, Methodology, Evaluation and Clinical Translation

放射学中的视觉基础模型：数据、方法、评估和临床转化的范围审查

Vergara-Richart, Alejandro, Rafael-Palou, Xavier, Fuster-Matanzo, Almudena, Roncales, Ignacio Iborra, Alberich-Bayarri, Ángel, Jiménez-Pastor, Ana

Abstract

Vision foundation models (VFMs) are increasingly being developed for radiological imaging, yet their definition, development and evaluation remain heterogeneous. We conducted a PRISMAScR scoping review of peer-reviewed studies published between January 2017 and March 2026 describing foundation models trained exclusively on radiological imaging data. Sixty-seven studies were included and mapped across three pillars: data scale and heterogeneity, architectural and pretraining scalability, and downstream transferability and generalization. Datasets primarily covered brain MRI, thoracoabdominal CT, and chest X-ray, ranging from fewer than 100,000 samples to multi-million-image cohorts. Transformer-based architectures and self-supervised pretraining predominated, particularly masked image modeling, contrastive learning and multi-stage approaches. Evaluation focused mainly on segmentation and classification, whereas cross-center, cross-scanner, anatomical and modality-shift validation was inconsistently reported. Alignment with FUTURE-AI principles was uneven. Overall, radiology-specific VFMs show promising transferability, but clinical translation remains constrained by limited data representativeness, heterogeneous benchmarks, incomplete reporting and insufficient deployment-oriented evaluation.

Chinese Translation

视觉基础模型（VFMs）在放射影像领域的开发日益增多，但其定义、开发和评估仍然存在异质性。我们进行了PRISMAScR范围审查，涵盖了2017年1月至2026年3月间发表的同行评审研究，这些研究描述了仅基于放射影像数据训练的基础模型。共纳入67项研究，并在三个支柱上进行了映射：数据规模和异质性、架构和预训练的可扩展性，以及下游迁移性和泛化能力。数据集主要涉及脑部MRI、胸腹部CT和胸部X光，样本数量从不足100,000到数百万图像不等。基于Transformer的架构和自监督预训练占主导地位，特别是掩蔽图像建模、对比学习和多阶段方法。评估主要集中在分割和分类上，而跨中心、跨扫描仪、解剖学和模态转换的验证报告不一致。与FUTURE-AI原则的一致性也不均衡。总体而言，特定于放射学的VFMs显示出良好的迁移能力，但临床转化仍受限于数据代表性不足、基准不一致、报告不完整和缺乏面向部署的评估。

View on arXiv Download PDF AI Translation

cs.CV / 57 / 2607.07230

`Attention-Guided Cross-Temporal Clustering for Self-Supervised Video Object Segmentation

基于注意力引导的跨时间聚类用于自监督视频目标分割

Arshid, Waqas, Awrangjeb, Mohammad, Liew, Alan Wee-Chung, Gao, Yongsheng

Abstract

Video object segmentation (VOS) is a fundamental task in video understanding, requiring accurate delineation and consistent tracking of objects across frames. While supervised methods achieve strong performance, they rely on densely annotated datasets that are costly to obtain and have limited domain coverage. Self-supervised learning offers a promising alternative by removing the need for manual labels; however, existing approaches often struggle to jointly maintain spatial accuracy and temporal coherence, particularly in unconstrained multi-object scenarios. Many rely on optical flow, synthetic motion cues, or task-specific pretraining, limiting scalability and generalisation. We propose a self-supervised framework, Cross-Temporal Consistency and Clustering, that learns mid-level, part-aware representations by combining attention-guided token selection with lightweight temporal clustering. Instead of operating at the pixel or whole-object level, the method aligns soft part assignments across time using a saliency-weighted symmetric consistency objective. The framework leverages a frozen transformer backbone with lightweight modules for adaptive token selection and multi-offset temporal alignment, enabling efficient scaling across resolutions and motion patterns.

Chinese Translation

视频目标分割（VOS）是视频理解中的一项基础任务，要求在帧间准确描绘和一致跟踪对象。尽管监督方法取得了强劲的性能，但它们依赖于密集标注的数据集，这些数据集获取成本高且领域覆盖有限。自监督学习通过消除对手动标签的需求提供了一种有前景的替代方案；然而，现有方法往往难以在不受约束的多目标场景中同时保持空间准确性和时间一致性。许多方法依赖于光流、合成运动线索或特定任务的预训练，这限制了其可扩展性和泛化能力。我们提出了一种自监督框架，跨时间一致性与聚类（Cross-Temporal Consistency and Clustering），通过结合注意力引导的标记选择与轻量级时间聚类，学习中级的、部分感知的表示。该方法不是在像素或整体对象级别操作，而是使用显著性加权的对称一致性目标在时间上对软部分分配进行对齐。该框架利用一个冻结的变换器主干，配合轻量级模块进行自适应标记选择和多偏移时间对齐，从而实现跨分辨率和运动模式的高效扩展。

View on arXiv Download PDF AI Translation

cs.CV / 58 / 2607.07236

Unraveling Machine Behavior by Multi-Level Bias Analysis and Detection: Methodology and Application to Computer Vision

通过多层偏差分析与检测揭示机器行为：方法论及其在计算机视觉中的应用

Serna, Ignacio, Morales, Aythami, Fierrez, Julian

Abstract

This study investigates the presence and propagation of bias within Neural Networks through a comprehensive multi-level analysis spanning the learned latent space, layer activations, and the network's parameters. Based on this taxonomy, we propose three bias detection approaches: 1) SpaceBias (new method), which characterizes the latent space prior to the final classification layer using neighbor-probability distributions and quantifies bias with the two-sample Kolmogorov-Smirnov test on the per-group distributions. 2) ActivationBias (extension of the existing method InsideBias), which analyzes the activations of neural network filters and quantifies bias via a Mann-Whitney U test, based on the observed fact that underrepresented groups exhibit lower activation levels in the final convolutional layers. 3) WeightBias (extension of the existing method IFBiD), which uses a secondary neural network trained to identify biased patterns directly in the parameters of task-specific models. Unlike conventional methods, which assess neural network outcomes and treat the model as a black box, our proposed techniques provide insight into how biases manifest within the network architecture itself at different levels, offering a more nuanced and detailed understanding. Experiments are conducted on two complementary applications: gender classification in the DiveFace dataset (72,000 face images) and digit classification on a colored-MNIST benchmark with controlled bias severity. In total, more than 127,000 models with varying degrees and types of bias were trained and evaluated. The severity sweep shows that the internal disparity, and with it the detection performance, decreases smoothly as the training distribution approaches balance. The results highlight the importance of methods that provide deeper insight into the behavior of AI models.

Chinese Translation

本研究通过对神经网络中偏差的存在与传播进行全面的多层分析，探讨了学习的潜在空间、层激活和网络参数。基于这一分类法，我们提出了三种偏差检测方法：1) SpaceBias（新方法），通过邻域概率分布表征最终分类层之前的潜在空间，并利用两样本Kolmogorov-Smirnov检验量化每组分布的偏差。2) ActivationBias（现有方法InsideBias的扩展），分析神经网络滤波器的激活，并通过Mann-Whitney U检验量化偏差，基于观察到的事实，即代表性不足的群体在最终卷积层中表现出较低的激活水平。3) WeightBias（现有方法IFBiD的扩展），使用一个次级神经网络直接识别任务特定模型参数中的偏差模式。与传统方法不同，后者评估神经网络的结果并将模型视为黑箱，我们提出的技术提供了对偏差如何在网络架构内部以不同层次表现的洞察，从而提供了更细致和深入的理解。实验在两个互补的应用上进行：在DiveFace数据集（72,000张人脸图像）中的性别分类和在具有可控偏差严重性的彩色MNIST基准上的数字分类。总共训练和评估了超过127,000个具有不同程度和类型偏差的模型。严重性扫描显示，内部差异及其检测性能随着训练分布接近平衡而平滑下降。结果强调了提供更深入洞察AI模型行为的方法的重要性。

View on arXiv Download PDF AI Translation

cs.CV / 59 / 2607.07240

An Edge-aware Prompt-enhanced SAM for Ultrasound Image Segmentation

一种边缘感知的提示增强型SAM用于超声图像分割

Li, Wenhao, Liu, Fangyi, Du, Bo

Abstract

Ultrasound image segmentation is essential for delineating anatomical structures and lesions, providing the foundation for accurate diagnosis. While the Segment Anything Model (SAM) has demonstrated remarkable success on natural images, its performance on ultrasound data is often hindered by poor boundary delineation. To address this limitation, we propose EP-SAM, an edge-aware and prompt-enhanced adaptation of SAM. Specifically, we leverage multi-block feature extraction from the image encoder to enrich coarse-to-fine semantic representations, while edge-aware supervision of the image encoder improves robustness to contour ambiguity and speckle noise. By integrating these complementary cues, EP-SAM generates high-quality prompts that effectively guide the model toward target regions of interest. Experimental results on multiple benchmarks demonstrate that EP-SAM consistently outperforms existing SAM-based methods.

Chinese Translation

超声图像分割对于描绘解剖结构和病变至关重要，为准确诊断提供基础。尽管Segment Anything Model (SAM) 在自然图像上表现出色，但其在超声数据上的性能常常受到边界描绘不佳的影响。为了解决这一局限性，我们提出了EP-SAM，一种边缘感知和提示增强的SAM适应模型。具体而言，我们利用图像编码器的多块特征提取来丰富粗到细的语义表示，同时对图像编码器进行边缘感知监督，以提高对轮廓模糊和斑点噪声的鲁棒性。通过整合这些互补线索，EP-SAM生成高质量的提示，有效引导模型关注目标区域。多项基准实验结果表明，EP-SAM在各项评估中始终优于现有的基于SAM的方法。

View on arXiv Download PDF AI Translation

cs.CV / 60 / 2607.07264

Naming the Concepts Classifiers Rely On: Language-Anchored Decomposition for Faithful Explanation

命名分类器所依赖的概念：基于语言的分解方法以实现可信解释

Akash, Ahsan Habib, Bhusal, Dipkamal, Jones, Stacey, Adjeroh, Donald A., Bhattarai, Binod, Gyawali, Prashnna Kumar

Abstract

Deep neural networks are widely deployed in high-stakes visual applications where interpretability is critical, yet existing explanations face a trade-off: post-hoc concept methods recover factors that are faithful to a model's behavior but unnamed, while naming and by-design methods attach human-readable concepts only by retraining or altering the classifier. We propose Language-Anchored Decomposition (LAD), a post-hoc framework that delivers concepts which are simultaneously named, faithful, and obtained without modifying the model. For each class, a large language model proposes a concept vocabulary that CLIP-based similarity maps localize across image regions. Inverting standard non-negative matrix factorization, LAD fixes these language-grounded maps as the coefficient matrix and learns only a concept basis that reconstructs the frozen encoder's activations, so naming becomes a structural constraint and the model's own feature geometry determines which concepts are retained. Removing this anchor preserves accuracy but collapses attribution faithfulness. Across natural-image, scene, and medical-imaging benchmarks, LAD produces spatially precise explanations that are decision-relevant under both concept insertion and deletion, while uniquely providing stable, human-interpretable concept names.

Chinese Translation

深度神经网络广泛应用于高风险的视觉应用中，在这些应用中可解释性至关重要。然而，现有的解释方法面临权衡：事后概念方法恢复与模型行为一致的因素，但未命名，而命名和设计方法则仅通过重新训练或修改分类器来附加人类可读的概念。我们提出了基于语言的分解方法（Language-Anchored Decomposition, LAD），这是一种事后框架，能够同时提供命名、可信且无需修改模型的概念。对于每个类别，一个大型语言模型提出一个概念词汇，基于CLIP的相似性图在图像区域中进行定位。LAD通过反转标准的非负矩阵分解，将这些基于语言的图作为系数矩阵固定，并仅学习一个概念基，以重构冻结编码器的激活，因此命名成为一种结构约束，模型自身的特征几何决定了保留哪些概念。去除这一锚点虽然保留了准确性，但会导致归因的可信性下降。在自然图像、场景和医学影像基准测试中，LAD产生了空间上精确的解释，这些解释在概念插入和删除下均与决策相关，同时独特地提供了稳定的人类可解释的概念名称。

View on arXiv Download PDF AI Translation

cs.CV / 61 / 2607.07288

InfraQR: Edge-Placed QR-Inspired Structured Patch Attacks on Infrared Vision-Language Models

InfraQR：针对红外视觉语言模型的边缘放置QR灵感结构补丁攻击

Li, Xin, Han, Jiaju, Yaqi, Ma, Hu, Chengyin, Zhao, Yingying, Long, Jiahuan, Zhang, Fengyu, Chai, Yahui

Abstract

Infrared vision-language models are increasingly used for perception under low-light and adverse visual conditions, yet their robustness to localized structured perturbations remains underexplored. Existing infrared adversarial studies mainly focus on object detectors, leaving the security of infrared vision-language models less systematically examined. We present InfraQR, a QR-inspired structured patch attack for infrared vision-language models. Unlike localized attacks that attach perturbations to the target object, InfraQR places a compact structured patch along image boundaries and optimizes learnable grid cells through surrogate CLIP-style encoders. The resulting patch has a near-binary structured appearance, but is not required to be a valid or machine-readable QR code. We evaluate InfraQR on infrared classification, caption transfer, and question-answer-aware visual question answering (VQA) tasks. On a 300-image infrared benchmark, InfraQR sharply reduces the accuracy of multiple CLIP-style classifiers, including reducing OpenAI CLIP accuracy from 98.67% to 0.70%. The generated adversarial images also transfer to black-box captioning and VQA models, causing semantic degradation in captions and more error-prone answers under GPT-5.4-based evaluation. These results show that infrared vision-language models remain vulnerable to structured edge-placed perturbations, motivating further study of cross-task robustness beyond direct object occlusion.

Chinese Translation

红外视觉语言模型越来越多地用于低光照和不利视觉条件下的感知，但它们对局部结构扰动的鲁棒性仍然未得到充分探索。现有的红外对抗研究主要集中在物体检测器上，导致红外视觉语言模型的安全性缺乏系统性的检验。我们提出了InfraQR，一种针对红外视觉语言模型的QR灵感结构补丁攻击。与将扰动附加到目标物体的局部攻击不同，InfraQR在图像边界放置一个紧凑的结构补丁，并通过代理的CLIP风格编码器优化可学习的网格单元。生成的补丁具有近乎二进制的结构外观，但不要求是有效或机器可读的QR码。我们在红外分类、标题转移和基于问题-答案的视觉问答（VQA）任务上评估了InfraQR。在一个包含300张图像的红外基准测试中，InfraQR显著降低了多个CLIP风格分类器的准确性，包括将OpenAI CLIP的准确性从98.67%降低到0.70%。生成的对抗图像还能够转移到黑箱标题生成和VQA模型中，导致标题语义退化以及在基于GPT-5.4的评估下更易出错的答案。这些结果表明，红外视觉语言模型仍然容易受到结构化边缘放置扰动的影响，促使我们进一步研究跨任务鲁棒性，而不仅仅是直接物体遮挡。

View on arXiv Download PDF AI Translation

cs.CV / 62 / 2607.07292

CarbonCLIP: Enhance Carbon Prediction from Satellite Imagery via Integrated Street-View Semantics and Temporal Context Training

CarbonCLIP：通过集成街景语义和时间上下文训练增强卫星图像的碳预测

Yang, Zeru, Gong, Fang-Ying, Yim, Steve H. L., Yuen, Chau

Abstract

Accurately estimating urban carbon emissions is critical for sustainable urban planning, yet many existing approaches remain difficult to apply consistently across cities due to data-source heterogeneity and the lack of fine-grained semantic-temporal context in remote sensing data. We propose CarbonCLIP, a task-oriented multimodal distillation framework that improves satellite-based carbon emission prediction by transferring contextual knowledge into a unified satellite representation through dual-branch contrastive learning. Unlike conventional methods that rely on static visual features, CarbonCLIP explicitly bridges the gap between top-down satellite views and ground-level human activities. Specifically, the spatial branch uses fine-grained textual descriptions automatically generated from street-view images by Large Multimodal Models (LMMs) to provide semantic priors reflecting building functions, infrastructure, and urban activities, while the temporal branch employs a month encoder to encode temporal priors associated with monthly emission variation. CarbonCLIP requires multimodal data only during the pretraining phase; during inference, it relies solely on satellite imagery, thereby supporting scalable deployment when ground-level data are unavailable at inference. Experiments on Beijing and Singapore demonstrate that CarbonCLIP outperforms baselines in both study cities. The results validate that our method effectively transfers multimodal knowledge into satellite representations, offering a robust solution for satellite-based urban carbon modeling.

Chinese Translation

准确估计城市碳排放对可持续城市规划至关重要，但由于数据源异质性以及遥感数据中缺乏细粒度的语义-时间上下文，许多现有方法在不同城市间的一致应用仍然困难。我们提出了CarbonCLIP，这是一种面向任务的多模态蒸馏框架，通过双分支对比学习将上下文知识转移到统一的卫星表示中，从而改善基于卫星的碳排放预测。与依赖静态视觉特征的传统方法不同，CarbonCLIP明确弥合了自上而下的卫星视图与地面人类活动之间的差距。具体而言，空间分支使用由大型多模态模型（LMMs）自动生成的细粒度文本描述，提供反映建筑功能、基础设施和城市活动的语义先验，而时间分支则采用月份编码器来编码与每月排放变化相关的时间先验。CarbonCLIP仅在预训练阶段需要多模态数据；在推理过程中，它仅依赖卫星图像，从而在推理时地面数据不可用的情况下支持可扩展部署。对北京和新加坡的实验表明，CarbonCLIP在这两个研究城市中均优于基线。结果验证了我们的方法有效地将多模态知识转移到卫星表示中，为基于卫星的城市碳建模提供了稳健的解决方案。

View on arXiv Download PDF AI Translation

cs.CV / 63 / 2607.07320

SoccerNet 2026 Challenges Results

SoccerNet 2026 挑战结果

Cioppa, Anthony, Giancola, Silvio, Ardö, Håkan, Dalal, Mohamad, Held, Jan, Ochin, Jérémie, Rao, Jiayuan, Sanchez, Karen, Vandeghen, Renaud, Xarles, Artur, Barnich, Olivier, Clapés, Albert, Delvaux, Mathieu, Escalera, Sergio, Ghanem, Bernard, Hons, Cédric, Houet, Antoine, Manitsaris, Sotiris, Michel, Tom, Miralles, Pierre, Moeslund, Thomas B., Nilsson, Mikael, Stanciulescu, Bogdan, Van Droogenbroeck, Marc, Wang, Yanfeng, Xie, Weidi, Altawijri, Faisal, Atef, Mohamed, Budennyy, Semen, Chelpanov, Vasiliy, Chen, Puhua, Chen, Yixin, Cheng, Lechao, Chu, Jianling, Do, Ju-Seong, Durygin, Oleg, Fetouh, Omar, Fuchs, Mirco, Ghallab, Youssef, Ghosh, Falguni, Heo, Wonjun, Hu, Yufeng, Huang, Weixuan, Huynh-Ha, Phuong-Linh, Isupov, Matvey, Ji, Yangguang, Jiang, Siyuan, Jiang, Zhenxiang, Jo, Wonyong, Jung, Ho-Young, Kang, SeongHeon, Kim, MinJae, Kim, Youngseon, Komosa, Jakub, Konshin, Artem, Le, Trung-Hoang, Lee, Jongmin, Li, Lingling, Li, Litao, Linkov, Vadim, Liu, Fang, Ma, Haoxuan, Makino, Shun, Mathkour, Ismail, Mitin, Konstantin, Moiseev, Mikhail, Nagaya, Takumi, Nakamura, Yuki, Nguyen, Thanh-Khoi, Nguyen, Hoang-Phuc, Nguyen, Trong-Thuan, Orduz, Christian, Park, Kwanyong, Perez, Fabian, Rawat, Parthsarthi, Rim, SuHyun, Rueda-Chacón, Hoover, Scott, Atom, Sugimura, Minori, Sun, Yuyang, Tang, Shengeng, Tran, Minh-Triet, Uchida, Ikuma, Vanegas, Juan, Vo, Thanh-Nhan, Wang, Jiangtao, Wang, Yaxiong, Wang, Xiaogang, Wang, Ruifeng, Watanabe, Rio, Wen, Jiali, Wu, Yongliang, Yang, Di, Yang, Xu, Yang, Zhuo, Ye, Xinyu, Yu, Yibo, Zhai, Zihan, Zhang, Yu, Zhao, Zhenyu, Zhong, Zhun, Zhou, Yixi, Zhu, Xingyu, Zhu, Wenbo, Ziegler, Julian

Abstract

The SoccerNet 2026 Challenges constitute the sixth annual edition of the SoccerNet open benchmarking effort, dedicated to advancing computer vision research in sports video understanding. This year's challenges span five vision-based tasks: (1) Ball Action Anticipation, predicting the timing and class of ball-related actions within a short future window from a preceding observation window; (2) Player-Centric Ball Action Spotting, temporally localizing and classifying ball-related actions while assigning each action to the acting player through team affiliation and jersey number; (3) Novel View Synthesis, rendering images from unobserved camera poses in multi-view football scenes; (4) Spiideo SoccerNet Synloc, localizing athletes in real-world pitch coordinates from a single calibrated static-camera image; and (5) Visual Question Answering, answering multiple-choice questions about football broadcasts across text, image, and video inputs. For each task, participants were provided with annotated data, a unified evaluation protocol, and a public baseline. This edition saw broad participation, with 427 teams submitting 1,129 entries across the five tasks and 28 teams contributing reviewed technical reports. This paper describes each task and its evaluation protocol, presents the challenge leaderboards, and summarizes the leading submissions, with the aim of documenting the current state of each task as measured on held-out challenge data.

Chinese Translation

SoccerNet 2026 挑战是第六届年度 SoccerNet 开放基准测试活动，旨在推动体育视频理解领域的计算机视觉研究。今年的挑战涵盖五个基于视觉的任务：(1) 球的动作预测，预测在短期未来窗口内与球相关的动作的时间和类别，基于之前的观察窗口；(2) 以球员为中心的球的动作定位，时间上定位和分类与球相关的动作，同时通过团队归属和球衣号码将每个动作分配给执行的球员；(3) 新视角合成，从未观察到的相机视角渲染多视角足球场景中的图像；(4) Spiideo SoccerNet Synloc，从单个校准静态相机图像中将运动员定位到现实世界的场地坐标；(5) 视觉问答，回答关于足球广播的多项选择问题，涉及文本、图像和视频输入。对于每个任务，参与者提供了带注释的数据、统一的评估协议和公共基准。本届挑战吸引了广泛的参与，共有427个团队在五个任务中提交了1,129个参赛作品，28个团队提交了经过评审的技术报告。本文描述了每个任务及其评估协议，展示了挑战排行榜，并总结了领先的提交，旨在记录在保留的挑战数据上测量的每个任务的当前状态。

View on arXiv Download PDF AI Translation

cs.CV / 64 / 2607.07322

HAJJv2-CrowdCount: Zero-Shot Benchmark for Dense Crowd Counting

HAJJv2-CrowdCount：密集人群计数的零样本基准

AlYabis, Reem, AlTuwaim, Fares, AlOtaibi, AlJawharh, Eltahir, Mohamed

Abstract

Automated crowd counting in Hajj video is difficult not because current models lack capacity, but because the footage violates the assumptions those models were built on: cameras observe the crowd from steep, near-vertical angles, individuals occlude one another extensively, and a single frame can contain well over a thousand people. Benchmarks that test crowd counting in such an environment are either private or not detailed per second. We revisit the HAJJv2 dataset and contribute HAJJv2-CrowdCount: per-second human-annotated crowd counts for its testing videos. Using these annotations, we benchmark three recent zero-shot counting paradigms: an open-vocabulary detector (YOLO-World), a point-based counter (APGCC), and a promptable segmentation-based counter (SAM3Count). SAM3Count attains the lowest overall mean absolute error (MAE 70.4, 95% CI 56.0-86.1), ahead of YOLO-World (92.0) and APGCC (152.9). This ordering reverses, however, in the regime most relevant to deployment: on the densest frames, the detection- and segmentation-based counters both degrade sharply (MAE exceeding 300), while the point-based counter degrades far more gracefully (MAE 114.9). This inversion is decision-relevant for Hajj crowd management, where reliable counts are needed most precisely in the densest and most occluded scenes. The annotations are released to support reproduction and extension of these results.

Chinese Translation

在朝觐视频中进行自动化人群计数的难点并不在于当前模型的能力不足，而在于这些视频违反了模型构建时的假设：摄像机从陡峭的近乎垂直的角度观察人群，个体之间相互遮挡严重，并且单帧画面中可能包含超过一千人。测试此类环境下人群计数的基准要么是私有的，要么没有逐秒详细信息。我们重新审视了HAJJv2数据集，并贡献了HAJJv2-CrowdCount：为其测试视频提供逐秒的人为标注人群计数。利用这些标注，我们对三种最新的零样本计数范式进行了基准测试：开放词汇检测器（YOLO-World）、基于点的计数器（APGCC）和可提示的分割基础计数器（SAM3Count）。SAM3Count在总体平均绝对误差（MAE 70.4，95% CI 56.0-86.1）上表现最佳，领先于YOLO-World（92.0）和APGCC（152.9）。然而，在与部署最相关的情况下，这一排序发生了逆转：在最密集的帧中，基于检测和分割的计数器均急剧下降（MAE超过300），而基于点的计数器则表现得更为平稳（MAE 114.9）。这一反转对于朝觐人群管理具有决策相关性，因为在最密集和最遮挡的场景中最需要可靠的计数。我们发布了这些标注，以支持结果的复现和扩展。

View on arXiv Download PDF AI Translation

cs.CV / 65 / 2607.07361

BUS: Brain-Inspired Unsupervised Self-Reflection for Advanced Multimodal Reasoning

BUS：基于大脑启发的无监督自我反思用于高级多模态推理

Yang, Jiacheng, Xiao, Tongying, Dang, Yunkai, Wang, Cong, Yang, Yuekun, Fan, Qi, Li, Wenbin, Miao, Feng, Gao, Yang

Abstract

Current Vision-Language Models (VLMs) often struggle to handle complex visual tasks that require consistent and fine-grained reasoning. Recent methods aim to train models to facilitate self-reflective reasoning, i.e., reviewing and improving the generated reasoning. However, they require large volumes of annotated data and lack explicit reflective behavior during test time. This work aims to bridge this gap through inspiration from neuroscience. The human brain exhibits efficient backward prediction, i.e., predicting which current states are likely to precede a given future state. In this work, we first verify that mainstream VLMs can perform backward prediction, similar to the human brain. Then, we propose Brain-inspired Unsupervised Self-reflection (BUS), a label-free training framework to enhance reflective reasoning capability in challenging image analysis. BUS enables VLMs to perform backward prediction and provide explicit learning signals on data without ground-truth labels. In this way, BUS eliminates reliance on annotated data while improving reasoning performance. Notably, BUS is compatible with popular fine-tuning methods, such as Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). Finally, extensive experiments on 8 benchmarks demonstrate the effectiveness of BUS across a wide range of complex visual tasks. It achieves notable improvements over the base model while using only unlabeled training data. Our experimental findings validate that backward prediction capability is critical for VLM reasoning.

Chinese Translation

当前的视觉-语言模型（VLMs）在处理需要一致且细致推理的复杂视觉任务时常常面临挑战。近期的方法旨在训练模型以促进自我反思推理，即回顾和改善生成的推理。然而，这些方法需要大量的标注数据，并且在测试时缺乏明确的反思行为。本研究旨在通过神经科学的启发来弥补这一差距。人脑表现出高效的逆向预测能力，即预测哪些当前状态可能会在给定的未来状态之前出现。在本研究中，我们首先验证了主流的VLMs能够进行逆向预测，类似于人脑。然后，我们提出了基于大脑启发的无监督自我反思（BUS），这是一个无标签的训练框架，用于增强在挑战性图像分析中的反思推理能力。BUS使VLMs能够进行逆向预测，并在没有真实标签的数据上提供明确的学习信号。通过这种方式，BUS消除了对标注数据的依赖，同时提高了推理性能。值得注意的是，BUS与流行的微调方法兼容，如监督微调（SFT）和强化学习（RL）。最后，在8个基准测试上的广泛实验表明，BUS在各种复杂视觉任务中表现出色。它在仅使用未标记的训练数据的情况下，相较于基础模型实现了显著的改进。我们的实验结果验证了逆向预测能力对于VLM推理的重要性。

View on arXiv Download PDF AI Translation

cs.CV / 66 / 2607.07383

MMAgent-R$^2$: Learning to Rerank and Reject for Agentic mRAG

MMAgent-R$^2$: 学习重排序和拒绝以实现自主的 mRAG

Zhang, Tao, Zhang, Ziqi, Ma, Zongyang, Yang, Yuxin, Li, Bing, Yuan, Chunfeng, Rong, Kang, Rao, Fengyun, Lyu, Jing, Hu, Weiming

Abstract

Knowledge-based Visual Question Answering (KB-VQA) requires models to retrieve visual entities matching the query image from large-scale encyclopedic knowledge bases and answer related questions. Existing multimodal Retrieval Augmented Generation (mRAG) methods rely on global visual features to match candidate entities, yet when the knowledge base contains numerous visually similar entities, the retriever struggles to distinguish them, populating the candidate set with visually similar but factually mismatched distractors. Since subsequent processing steps such as noise filtering are also confined to this fixed candidate set, errors from failed retrieval inevitably propagate to the final answer. To address these challenges, we propose MMAgent-R$^2$, an agentic mRAG framework that integrates visual reranking and active rejection as its internal verification mechanism. Visual reranking directly compares query and candidate images, capturing discriminative details beyond textual descriptions to precisely identify the target entity among similar candidates; active rejection discards unreliable results and retrieves additional candidates when no confident match is found, moving beyond the fixed candidate pool. We design a composite reward function with step-level verification rewards and achieve joint optimization of external retrieval, internal verification, and answer generation via GRPO training. Experiments on InfoSeek, E-VQA, and MMhops demonstrate that \ours{} achieves state-of-the-art performance, with particularly notable advantages in challenging retrieval scenarios and complex multi-image multi-hop reasoning tasks.

Chinese Translation

基于知识的视觉问答（KB-VQA）要求模型从大规模的百科知识库中检索与查询图像匹配的视觉实体，并回答相关问题。现有的多模态检索增强生成（mRAG）方法依赖于全局视觉特征来匹配候选实体，但当知识库中包含大量视觉相似的实体时，检索器难以区分它们，导致候选集充满视觉相似但事实不符的干扰项。由于后续处理步骤如噪声过滤也局限于这个固定的候选集，检索失败所产生的错误不可避免地传播到最终答案。为了解决这些挑战，我们提出了 MMAgent-R$^2$，一个自主的 mRAG 框架，它将视觉重排序和主动拒绝作为其内部验证机制。视觉重排序直接比较查询图像和候选图像，捕捉超出文本描述的区分性细节，以精确识别相似候选中的目标实体；主动拒绝则在未找到可靠匹配时丢弃不可靠的结果并检索额外的候选，超越固定的候选池。我们设计了一个复合奖励函数，包含逐步验证奖励，并通过 GRPO 训练实现外部检索、内部验证和答案生成的联合优化。在 InfoSeek、E-VQA 和 MMhops 上的实验表明， extit{ours} 达到了最先进的性能，尤其在具有挑战性的检索场景和复杂的多图多跳推理任务中表现出显著优势。

View on arXiv Download PDF AI Translation

cs.CV / 67 / 2607.07395

When Prompts Ignore Structure: Graph-Based Attribute Reasoning for Calibrated VLMs

当提示忽略结构时：基于图的属性推理用于校准的视觉语言模型

Sodha, Tanay, Sharma, Aditya, Hebbalaguppe, Ramya, Agarwal, Vinti, Yeluripaty, Pranav Murthy

Abstract

Reliable confidence estimation remains a key limitation of test-time adaptation in vision-language models (VLMs), where prompt tuning improves zero-shot accuracy but often degrades calibration due to entropy-driven overconfidence. Prior approaches mitigate this using LLM-derived class attributes and contrastive regularization, yet treat attributes independently, ignoring their relational structure. We propose ARGTCA, which represents (class, attribute) pairs as nodes in a Symbolic Attribute Graph and trains a Graph Attention Network (GAT) using contrastive objectives to produce structurally informed embeddings that capture inter-attribute dependencies. We introduce two attribute selection strategies: ARGTCA-DIV for intra-class diversity and ARGTCA-DISC for inter-class discrimination. Experiments across nine benchmarks show that ARGTCA-DIV reduces average Expected Calibration Error (ECE) by approximately ~37% over baselines, while ARGTCA-DISC consistently performs as the second-best variant, reducing average ECE by approximately ~17% over baselines. These results suggest that modeling symbolic attribute interactions provides a principled approach for reliable test-time adaptation in VLMs.

Chinese Translation

可靠的置信度估计仍然是视觉语言模型（VLMs）在测试时适应中的一个关键限制，其中提示调优提高了零-shot 精度，但由于熵驱动的过度自信，往往会降低校准效果。先前的方法通过使用大型语言模型（LLM）衍生的类别属性和对比正则化来缓解这一问题，但独立处理属性，忽视了它们的关系结构。我们提出了 ARGTCA，它将（类别，属性）对表示为符号属性图中的节点，并使用对比目标训练图注意力网络（GAT），以生成捕捉属性间依赖关系的结构化嵌入。我们引入了两种属性选择策略：ARGTCA-DIV 用于类内多样性，ARGTCA-DISC 用于类间区分。在九个基准测试中的实验表明，ARGTCA-DIV 将平均期望校准误差（ECE）降低了约 37%，而 ARGTCA-DISC 始终表现为第二好的变体，平均 ECE 降低了约 17%。这些结果表明，建模符号属性交互为 VLMs 中可靠的测试时适应提供了一种有原则的方法。

View on arXiv Download PDF AI Translation

cs.CV / 68 / 2607.07401

Heterogeneity-Adaptive Diffusion Schrodinger Bridge for PET-Guided Whole-Body MRI Translation

异质性自适应扩散薛定谔桥用于PET引导的全身MRI转换

Wang, Chengbo, Yu, Jiacheng, Bian, Linjie, Qi, Ming, Liu, Xiaosheng, Che, Tongtong, Zhang, Jichang, Li, Shuyu, Song, Shaoli, Wang, Xiuying

Abstract

While whole-body multimodal medical imaging scanners have been increasingly recognized for more effective medical applications, the excessive long acquisition time in PET-MR scanning is a major obstacle in more efficient clinical practice. Deep learning-based MRI translation provides a potential solution to reduce scan duration. However, current models often focus on specific anatomical regions and face challenges for whole-body scans that consists of highly heterogeneous feature distributions mainly due to (1) different anatomical regions across whole-body, and (2) lesions or pathological tissues. This paper tackles the challenges through a novel Heterogeneity-Adaptive Diffusion Schrodinger Bridge (HA-DSB) framework. By explicitly modeling translation as stochastic transport between source and target distributions, HA-DSB incorporates region context embeddings derived from a vision-language model (VLM) to enable region-specific modeling. To enhance fidelity of the pathological tissue, lesion-aware metabolic prior from PET is integrated directly into the bridge dynamics through a dual-stage guidance mechanism. Specifically, a PET-guided noise modulation module adaptively scales spatial diffusion perturbations during the forward process, while PET features are leveraged during the reverse process to selectively amplify lesion-relevant structures via an attention mechanism. Experiments demonstrate the superiority of our method across different body regions in whole-body MRI translation and show improved translation quality in lesion areas under PET guidance. Our code is available at Github.

Chinese Translation

尽管全身多模态医学成像扫描仪在更有效的医学应用中得到了越来越多的认可，但PET-MR扫描中过长的采集时间仍然是更高效临床实践的一大障碍。基于深度学习的MRI转换提供了一种减少扫描时间的潜在解决方案。然而，当前模型往往专注于特定的解剖区域，并面临全身扫描中高度异质特征分布的挑战，这主要是由于（1）全身不同的解剖区域，以及（2）病变或病理组织。本文通过一种新颖的异质性自适应扩散薛定谔桥（HA-DSB）框架来解决这些挑战。HA-DSB通过明确将转换建模为源分布和目标分布之间的随机传输，结合来自视觉-语言模型（VLM）提取的区域上下文嵌入，以实现区域特定建模。为了增强病理组织的保真度，病变感知的代谢先验信息直接通过双阶段引导机制集成到桥的动态中。具体而言，PET引导的噪声调制模块在前向过程中自适应地缩放空间扩散扰动，而在反向过程中利用PET特征通过注意机制选择性地放大与病变相关的结构。实验表明，我们的方法在全身MRI转换的不同身体区域中表现出优越性，并在PET引导下提高了病变区域的转换质量。我们的代码可在Github上获取。

View on arXiv Download PDF AI Translation

cs.CV / 69 / 2607.07416

VCDP: Variation-Conditioned Distributional Proxy Learning for Semi-Supervised Medical Image Segmentation

VCDP：用于半监督医学图像分割的变异条件分布代理学习

Zhang, Zimu, Zhong, Yiheng, Zhang, Zhuoru, Hu, Yingzhen, He, Yanan, Meng, Fanliang, Liu, Xiaofeng

Abstract

Semi-supervised 3D medical image segmentation reduces the need for dense voxel-level annotations by exploiting unlabeled volumes. Although existing methods such as consistency regularization, pseudo-labeling, and co-training improve prediction-level robustness, they often provide insufficient feature-space organization for anatomically complex structures, especially small organs and ambiguous boundary regions with large intra-class variations. To address this issue, we propose Variation-Conditioned Distributional Proxy Learning (VCDP), a plug-and-play training-only regularization module for semi-supervised 3D medical image segmentation. VCDP represents each class with a learnable Gaussian distribution for shared class semantics and multiple variation prototypes for fine-grained intra-class patterns. A unified variation-conditioned compatibility score is further formulated to fuse distributional similarity and soft variation aggregation, guiding voxel embeddings to align with both global organ identity and local anatomical variations. VCDP is attached to decoder features during training and removed during inference, introducing no additional inference cost. Experiments on multi-organ segmentation benchmarks show that VCDP improves most evaluated baselines, particularly for small, ambiguous, and highly variable organs. Our anonymous code is released at https://anonymous.4open.science/r/VCDP_code-41ED.

Chinese Translation

半监督3D医学图像分割通过利用未标记的体积数据减少了对密集体素级注释的需求。尽管现有方法如一致性正则化、伪标签和协同训练提高了预测级的鲁棒性，但它们通常对解剖复杂结构，特别是小器官和具有大类内变异的模糊边界区域，提供的特征空间组织不足。为了解决这一问题，我们提出了变异条件分布代理学习（VCDP），这是一种即插即用的仅训练正则化模块，旨在用于半监督3D医学图像分割。VCDP为每个类别表示一个可学习的高斯分布，以共享类别语义，并为细粒度的类内模式提供多个变异原型。进一步制定了统一的变异条件兼容性得分，以融合分布相似性和软变异聚合，指导体素嵌入与全局器官身份和局部解剖变异对齐。VCDP在训练期间附加到解码器特征上，在推理期间移除，不引入额外的推理成本。在多器官分割基准上的实验表明，VCDP改善了大多数评估基线，特别是对于小型、模糊和高度可变的器官。我们的匿名代码已发布在 https://anonymous.4open.science/r/VCDP_code-41ED。

View on arXiv Download PDF AI Translation

cs.CV / 70 / 2607.07438

Two-Stage Multi-Modal Fusion with Adaptive Alignment for Action Quality Assessment

基于自适应对齐的两阶段多模态融合用于动作质量评估

Zhou, Kanglei, Cai, Ruizhi, Wang, Xinning, Zheng, Yijian, Wang, Liyuan, Li, Jianguo, Liang, Xiaohui

Abstract

Action Quality Assessment (AQA) aims to evaluate how well a person performs a movement, which is essential in applications such as sports scoring, skill assessment, and healthcare. However, unimodal approaches often struggle to capture subtle cues of movement quality in real-world settings. Although multi-modal inputs provide complementary information, existing methods still face two major challenges: heterogeneous modalities often lead to cross-modal misalignment and unstable fusion, and reliable multi-modal annotation is costly, resulting in limited dataset diversity. To address these challenges, we propose DualAlign, a two-stage multi-modal fusion framework with adaptive alignment. The framework first constructs a coherent visual representation by maximizing shared structural information across RGB video, optical flow, and skeleton modalities. Textual semantics are then incorporated after visual stabilization, allowing high-level descriptions to complement rather than distort the underlying visual manifold. To evaluate the framework under realistic multi-modal conditions, we introduce MM--JDM, a movement-quality assessment dataset integrating RGB videos, optical flow, skeleton sequences, and structured text. MM--JDM naturally exhibits modality noise, class imbalance, and label scarcity, making it a challenging benchmark for studying multi-modal fusion and alignment. Extensive experiments show that DualAlign improves average correlation on MM--JDM by 21.16% over the state-of-the-art methods and achieves gains of 3.53% and 5.95% on the RG and Fis-V benchmarks, respectively. DualAlign also remains robust under missing-modality and label-scarce conditions.

Chinese Translation

动作质量评估（AQA）旨在评估一个人执行动作的表现，这在体育评分、技能评估和医疗保健等应用中至关重要。然而，单模态方法往往难以捕捉现实环境中动作质量的细微线索。尽管多模态输入提供了互补信息，但现有方法仍面临两个主要挑战：异构模态常常导致跨模态错位和不稳定的融合，而可靠的多模态标注成本高昂，导致数据集多样性有限。为了解决这些挑战，我们提出了DualAlign，一种具有自适应对齐的两阶段多模态融合框架。该框架首先通过最大化RGB视频、光流和骨架模态之间的共享结构信息，构建一致的视觉表示。然后，在视觉稳定后引入文本语义，使高层描述能够补充而非扭曲基础视觉流形。为了在真实的多模态条件下评估该框架，我们引入了MM--JDM，一个整合了RGB视频、光流、骨架序列和结构化文本的动作质量评估数据集。MM--JDM自然表现出模态噪声、类别不平衡和标签稀缺，使其成为研究多模态融合和对齐的具有挑战性的基准。大量实验表明，DualAlign在MM--JDM上的平均相关性比最先进的方法提高了21.16%，在RG和Fis-V基准上分别获得了3.53%和5.95%的提升。DualAlign在缺失模态和标签稀缺条件下也保持了鲁棒性。

View on arXiv Download PDF AI Translation

cs.CV / 71 / 2607.07470

A Theory of Contrastive Learning with Natural Images

自然图像对比学习理论

Torralba, Antonio, Weiss, Yair

Abstract

Why does contrastive learning with simple images and augmentations yield useful representations for downstream tasks? We address this question by analytically computing the optimal representation in terms of a contrastive loss for a range of basic augmentations and any image dataset with stationary statistics. We show that for certain augmentations the optimum can be attained by a CNN whose first layer filters are sinusoids, followed by a pointwise nonlinearity, global average pooling, and a final linear layer that performs partial whitening. We also show that the optimal weights in such CNNs for more complicated augmentations are still sinusoids. The frequencies of the sinusoids and their weights can be computed using a simple waterfilling algorithm given the dataset's expected power spectrum. Experiments with different image datasets and augmentations show that such CNNs trained with SGD empirically learn sinusoids in their first layer and to perform partial whitening

Chinese Translation

为什么使用简单图像和增强的对比学习能够为下游任务提供有用的表示？我们通过分析计算在对比损失下，针对一系列基本增强和任何具有平稳统计特性的图像数据集的最优表示来解决这个问题。我们展示了对于某些增强，最优表示可以通过一个卷积神经网络（CNN）来实现，其第一层的滤波器为正弦波，后接逐点非线性、全局平均池化和执行部分白化的最终线性层。我们还表明，对于更复杂的增强，这种CNN中的最优权重仍然是正弦波。给定数据集的期望功率谱，正弦波的频率及其权重可以通过简单的水填充算法进行计算。对不同图像数据集和增强的实验表明，这种使用随机梯度下降（SGD）训练的CNN在其第一层中经验性地学习到正弦波并执行部分白化。

View on arXiv Download PDF AI Translation

cs.CV / 72 / 2607.07486

Discovering Geometric Biases in 3D Face Reconstruction: A Curvature-Aware Spectral Framework for Fairness Evaluation

发现3D人脸重建中的几何偏差：一种考虑曲率的谱框架用于公平性评估

Shilova, Veronika, Malherbe, Emmanuel, Palma, Giovanni, Bokaris, Panagiotis-Alexandros, Risser, Laurent, Loubes, Jean-Michel

Abstract

3D Morphable Models (3DMMs) remain the standard parametric shape priors for many state-of-the-art 3D face reconstruction algorithms. However, as these models are derived from a finite number of 3D face samples, they inherit the morphological biases of their training data, potentially limiting their generalizability across diverse global populations. In this paper, we propose a novel framework to analyze 3DMM reconstructions through the lens of surface curvature, with the objective to discover, quantify and visualize biases. While standard evaluation metrics often rely on Euclidean distances, our reconstruction error captures subtle surface nuances such as local topology or undulations. To do so, we leverage the Laplace-Beltrami Operator (LBO) to generate high-resolution curvature error maps, providing a localized and geometrically meaningful visualization of discrepancies between ground truth faces and reconstructed meshes. We derive from it an error metric that we validated through a user study, observing a significantly higher correlation to human perception compared to traditional methods. Furthermore, we conduct extensive experiments across several 3DMM bases and fitting algorithms, uncovering systematic age-related biases and providing preliminary evidence of biases associated with gender and ethnicity. Our findings highlight the necessity of adopting curvature-aware evaluation protocols to ensure demographic fairness and geometric precision in future 3D face reconstruction research.

Chinese Translation

3D可变形模型（3DMMs）仍然是许多最先进的3D人脸重建算法的标准参数形状先验。然而，由于这些模型是从有限数量的3D人脸样本中推导而来的，它们继承了训练数据的形态偏差，可能限制了其在不同全球人群中的泛化能力。本文提出了一种新颖的框架，通过表面曲率的视角分析3DMM重建，旨在发现、量化和可视化偏差。尽管标准评估指标通常依赖于欧几里得距离，我们的重建误差捕捉了细微的表面细节，如局部拓扑或波动。为此，我们利用拉普拉斯-贝尔特拉米算子（Laplace-Beltrami Operator, LBO）生成高分辨率的曲率误差图，提供了真实人脸与重建网格之间差异的局部和几何意义的可视化。我们从中推导出了一种误差指标，并通过用户研究进行了验证，观察到与传统方法相比，其与人类感知的相关性显著更高。此外，我们在多个3DMM基础和拟合算法上进行了广泛的实验，揭示了系统性的与年龄相关的偏差，并提供了与性别和种族相关的偏差的初步证据。我们的研究结果强调了采用考虑曲率的评估协议的必要性，以确保未来3D人脸重建研究中的人口公平性和几何精确性。

View on arXiv Download PDF AI Translation

cs.CV / 73 / 2607.07507

HIVE: Understanding Post-Hallucination Reasoning in Vision Language Models

HIVE：理解视觉语言模型中的后幻觉推理

He, Feng, Wang, Zhenting, Wang, Qifan, Guan, Qiang, Liu, Dongfang, Tang, Ruixiang, Li, Qiankun

Abstract

Hallucinations in vision language models (VLMs) are commonly treated as semantic errors, yet they often arise from partial or ambiguous visual evidence. Prior work mainly focuses on detecting or suppressing hallucinations at generation time, leaving the subsequent reasoning stage largely unexplored. In this work, we study Post Hallucination Reasoning (PHR), the stage in which hallucinated semantics enter the model's inference context and influence downstream predictions. To systematically investigate PHR, we introduce HIVE, Hallucination Inference and Verification Engine, an evaluation infrastructure that enables controlled comparisons between faithful and hallucinated captions. Across nine tasks and nine models, we observe structured modality dependent patterns: hallucinated captions often improve accuracy on vision language tasks, while text only tasks exhibit limited or unstable effects. Further analyses show that hallucinated cues broaden semantic coverage and reshape reasoning dynamics while preserving stable inference. These findings highlight that hallucinated semantics may influence downstream reasoning once they enter the model's inference context. Understanding this post hallucination stage is important for improving the reliability and interpretability of multimodal reasoning systems. Code is publicly available at https://github.com/hefengcs/HIVE.

Chinese Translation

视觉语言模型（VLMs）中的幻觉通常被视为语义错误，然而它们往往源于部分或模糊的视觉证据。之前的研究主要集中在生成时检测或抑制幻觉，而对随后的推理阶段几乎没有探索。在本研究中，我们研究了后幻觉推理（Post Hallucination Reasoning, PHR），即幻觉语义进入模型推理上下文并影响下游预测的阶段。为了系统地研究PHR，我们引入了HIVE（Hallucination Inference and Verification Engine），一个评估基础设施，能够在真实和幻觉标题之间进行受控比较。在九个任务和九个模型中，我们观察到结构化的模态依赖模式：幻觉标题通常提高视觉语言任务的准确性，而仅文本任务则表现出有限或不稳定的效果。进一步分析表明，幻觉线索扩展了语义覆盖面，并重塑了推理动态，同时保持稳定的推理。这些发现强调了幻觉语义一旦进入模型的推理上下文，可能会影响下游推理。理解这一后幻觉阶段对于提高多模态推理系统的可靠性和可解释性至关重要。代码已公开发布在 https://github.com/hefengcs/HIVE。

View on arXiv Download PDF AI Translation

cs.CV / 74 / 2607.07518

Learning to Unify Deformable Shape and Texture Representations for Cardiac Video Classification

学习统一可变形形状与纹理表示用于心脏视频分类

Hossain, Tonmoy, Zhang, Miaomiao

Abstract

Deformable shape representations have proven to be robust complements to texture features in cardiac image classification, offering geometric priors that are invariant to imaging artifacts and intensity variations. However, existing deep networks perform simple concatenation to combine these distinct feature representations, which neither fully exploits their complementary nature nor learns cross-modal feature dependencies. Furthermore, this results in uniform attention across all timepoints; hence ignoring the varying diagnostic importance across the cardiac phases. In this paper, we propose a novel cardiac video classification model that, for the first time, learns temporal features in an integrated space of deformable shape and image texture representations. In particular, we design a bi-directional cross-attention in the latent space to fuse latent deformable shape and image features, allowing each modality to adaptively weight the other based on spatio-temporal correspondence. In contrast to current methods that apply uniform weighting across all the cardiac phases, our approach learns to dynamically adjust the contributions of shape and texture representations, derived from images, over time. We demonstrate state-of-the-art classification performance on a cine cardiac magnetic resonance (CMR) video dataset, achieving improved interpretability from attention mechanisms that identify diagnostically critical cardiac phases and modality contributions.

Chinese Translation

可变形形状表示已被证明是心脏图像分类中纹理特征的稳健补充，提供对成像伪影和强度变化不变的几何先验。然而，现有的深度网络仅通过简单的拼接来结合这些不同的特征表示，这既没有充分利用它们的互补特性，也没有学习跨模态特征依赖关系。此外，这导致在所有时间点上均匀关注，从而忽视了心脏各个阶段之间的诊断重要性差异。在本文中，我们提出了一种新颖的心脏视频分类模型，首次在可变形形状与图像纹理表示的综合空间中学习时间特征。特别地，我们设计了一种在潜在空间中的双向交叉注意机制，以融合潜在的可变形形状和图像特征，使每种模态能够根据时空对应关系自适应地加权另一种模态。与当前在所有心脏阶段应用均匀加权的方法不同，我们的方法学习动态调整来自图像的形状和纹理表示的贡献。我们在一组心脏磁共振（CMR）视频数据集上展示了最先进的分类性能，利用注意机制提高了解释性，识别出诊断关键的心脏阶段和模态贡献。

View on arXiv Download PDF AI Translation

cs.CV / 75 / 2607.07532

Context-Aware Slum Mapping in Sub-Saharan Africa Using Sentinel-1 Texture and Local Climate Zones

基于Sentinel-1纹理和地方气候区的撒哈拉以南非洲上下文感知贫民窟制图

Chepkilot, Peterson, Memar, Babak, Gamba, Paolo

Abstract

Accurate mapping of informal settlements remains a major challenge in Sub-Saharan African (SSA) cities because optical imagery often fails to distinguish Informal Settlements (defined here as LCZ 7) from spectrally similar formal Compact Low-Rise areas (LCZ 3). This study presents a context-aware, reproducible Optical-SAR framework that improves informal settlement delineation using Sentinel-2 spectral features and Sentinel-1 structural information within an adapted Local Climate Zone (LCZ) taxonomy. We implement a three-tier SAR integration strategy: calibrated backscatter, GLCM textures, and a physics-guided feature engineered to capture the high structural disorder and weak radar return characteristic of SSA informal settlements. Using reference data across Nairobi and Eldoret (Kenya), we evaluate performance via a stratified hold-out protocol and a season-aware ablation study. Results show that SAR textures provide the dominant performance gain for LCZ 7 detection. The Optical-SAR model achieves overall accuracy of 0.816 (dry) and 0.807 (wet), significantly outperforming the WUDAPT baseline (OA 0.704) and reducing the critical LCZ 3 - LCZ 7 confusion to 7%. Seasonal analysis indicates that while optical-only separability varies with phenology, SAR-derived textures stabilize informal settlement mapping across seasons. These findings demonstrate that the incorporation of SAR-derived features yields consistent improvements for urban morphology mapping in data-scarce environments across seasons and across the evaluated source cities, while cross-city transfer remains limited without local adaptation strategies.

Chinese Translation

在撒哈拉以南非洲（SSA）城市中，准确制图非正式定居点仍然是一个主要挑战，因为光学影像常常无法区分非正式定居点（在此定义为LCZ 7）与光谱上相似的正式紧凑低层区域（LCZ 3）。本研究提出了一种上下文感知的、可重复的光学-合成孔径雷达（Optical-SAR）框架，通过利用Sentinel-2光谱特征和Sentinel-1结构信息，在调整后的地方气候区（LCZ）分类法中改善非正式定居点的划分。我们实施了一种三层次的SAR集成策略：标定的后向散射、灰度共生矩阵（GLCM）纹理，以及一种物理指导特征，旨在捕捉SSA非正式定居点的高度结构无序和弱雷达回波特征。通过在内罗毕和埃尔多雷特（肯尼亚）的参考数据，我们通过分层保留协议和季节感知的消融研究评估性能。结果表明，SAR纹理为LCZ 7的检测提供了主要的性能提升。光学-SAR模型在干燥条件下的总体准确率为0.816，在湿润条件下为0.807，显著优于WUDAPT基准（OA 0.704），并将关键的LCZ 3 - LCZ 7混淆降低至7%。季节性分析表明，尽管仅使用光学数据的可分离性随物候变化，但SAR衍生的纹理在各季节中稳定了非正式定居点的制图。这些发现表明，结合SAR衍生特征在数据稀缺环境中对城市形态制图的季节性和评估源城市之间的一致性改进具有重要意义，而跨城市转移在没有地方适应策略的情况下仍然有限。

View on arXiv Download PDF AI Translation

cs.CV / 76 / 2607.07534

Infinite Worlds with Versatile Interactions

具有多样化交互的无限世界

Gao, Zelin, Wang, Qiuyu, Zhu, Jiapeng, Chen, Jingye, Liu, Zichen, Bai, Qingyan, Wang, Jiahao, Yuan, Yufeng, Wang, Hanlin, Lu, Yichong, Cheng, Ka Leong, Zhang, Haojie, Gao, Jian, Feng, Tianrui, Liu, Yuzheng, Yao, Yao, Xu, Yinghao, Zhu, Xing, Shen, Yujun, Ouyang, Hao

Abstract

We present LingBot-World 2.0 (also known as LingBot-World-Infinity), an advanced iteration of LingBot-World featuring four distinct upgrades. (1) Our model achieves an unbounded interaction horizon while maintaining consistent output quality, benefiting from a carefully crafted causal pretraining paradigm. (2) Through distilling a real-time variant from the base model, our system guarantees rapid response time, sufficient to drive 720p video streams at 60 fps. (3) Compared to the previous version, this update introduces highly diverse interactive elements, comprising a broader spectrum of actions (e.g., attacking, archery, spell-casting, and shooting) alongside a richer variety of text-driven events. (4) We pioneer the integration of an agentic harness within the domain of world modeling, wherein a pilot agent is tasked with planning and executing character behaviors, while a director agent is responsible for synthesizing novel environmental elements as the scene progresses. Additionally, to facilitate a shared experience, we develop an interface that permits multiple players to simultaneously immerse themselves in this vivid world simulator. We pair our primary 14B model with a lightweight 1.3B counterpart, which supports effortless deployment on a single GPU.

Chinese Translation

我们介绍了 LingBot-World 2.0（也称为 LingBot-World-Infinity），这是 LingBot-World 的一个高级版本，具有四个显著的升级。(1) 我们的模型实现了无限的交互视野，同时保持一致的输出质量，这得益于精心设计的因果预训练范式。(2) 通过从基础模型中提炼出实时变体，我们的系统保证了快速的响应时间，足以以 60 帧每秒驱动 720p 视频流。(3) 与之前的版本相比，此次更新引入了高度多样化的交互元素，包括更广泛的动作范围（例如，攻击、射箭、施法和射击），以及更丰富的文本驱动事件。(4) 我们在世界建模领域开创性地整合了一个代理控制系统，其中一个引导代理负责规划和执行角色行为，而一个导演代理则负责在场景进展中合成新的环境元素。此外，为了促进共享体验，我们开发了一个界面，允许多个玩家同时沉浸在这个生动的世界模拟器中。我们将主要的 14B 模型与一个轻量级的 1.3B 模型配对，后者支持在单个 GPU 上轻松部署。

View on arXiv Download PDF AI Translation

cs.CV / 77 / 2607.07545

Face-trace: Open-Set Attribution and Progressive Discovery of Synthetic Face Generators

面部追踪：开放集归属与合成面部生成器的渐进发现

Infantino, Alessia, Schiavella, Claudio, Amerini, Irene

Abstract

Recent advances in generative Artificial Intelligence have made synthetic face images increasingly realistic, creating new challenges for multimedia forensics. Source attribution methods should not only identify the generator of an image when the source is known, but also handle samples produced by previously unseen models. However, most existing approaches address synthetic face attribution in a closed-set setting, where all possible generators are available during training. This assumption does not hold in real-world scenarios, where new generators continuously appear and rejected samples should be organized rather than simply discarded. In this work we propose a pipeline for open-set synthetic face source attribution that combines known generator classification, energy-based OOD rejection, and unknown generator discovery. A classifier is trained on known generators using frozen I-JEPA embeddings, while rejected samples are represented by combining projected I-JEPA features with Forensic Self-Descriptors and then clustered to discover groups of unknown generators. We also extend the discovery stage to an incremental scenario, where rejected samples arrive over time. Experiments on the WILD dataset show that the proposed method achieves 96.73% closed-set attribution accuracy. In the open-set setting, energy-based rejection reaches 71.25% balanced accuracy, while rejected samples are clustered into meaningful unknown-generator groups, obtaining an ARI of 0.81, an NMI of 0.90, and an overall clustering purity of 87.74%. In the incremental setting, the discovered generator space is progressively extended while maintaining a final purity of 99.23%. Cross-dataset experiments suggest that the pipeline can operate beyond the original dataset distribution, although post-processing remains challenging.

Chinese Translation

近年来，生成性人工智能的进步使得合成面部图像变得越来越逼真，这为多媒体取证带来了新的挑战。源归属方法不仅应在已知源的情况下识别图像的生成器，还应处理由之前未见模型生成的样本。然而，大多数现有方法在封闭集环境中处理合成面部归属，其中所有可能的生成器在训练期间都是可用的。这一假设在现实世界场景中并不成立，因为新的生成器不断出现，而被拒绝的样本应被组织而非简单丢弃。在本研究中，我们提出了一种开放集合成面部源归属的流程，该流程结合了已知生成器分类、基于能量的OOD拒绝和未知生成器发现。使用冻结的I-JEPA嵌入对已知生成器进行分类器训练，同时通过将投影的I-JEPA特征与法医自描述符结合来表示被拒绝的样本，然后对其进行聚类以发现未知生成器的组。我们还将发现阶段扩展到增量场景，其中被拒绝的样本随时间到达。在WILD数据集上的实验表明，所提出的方法在封闭集归属准确率上达到了96.73%。在开放集环境中，基于能量的拒绝达到了71.25%的平衡准确率，而被拒绝的样本被聚类为有意义的未知生成器组，获得了0.81的ARI、0.90的NMI和87.74%的整体聚类纯度。在增量设置中，发现的生成器空间逐步扩展，同时保持最终纯度为99.23%。跨数据集实验表明，该流程可以在原始数据集分布之外运行，尽管后处理仍然具有挑战性。

View on arXiv Download PDF AI Translation

cs.CV / 78 / 2607.07553

AA-ViT: Anatomically Aware Vision Transformer with Structural and Frequency Guidance for Contrast Enhanced Brain MRI Synthesis

AA-ViT：具有结构和频率指导的解剖学意识视觉变换器，用于对比增强脑部MRI合成

Meraj, Talha, Flannery, Tom, Cummins, Charlie, Townend, Matt, Booth, Thomas C, Crossley, Peter, McCann, Michael, Overton, Ian, Unnikrishnan, Saritha

Abstract

Accurate tumour localization and diagnosis is a critical component of clinical care for brain cancers. Magnetic Resonance Imaging (MRI) is the most commonly used imaging modality due to its superior soft-tissue contrast. However, standard MRI often exhibits limited contrast and imaging artifacts, which necessitates the use of contrast agents to enhance lesion visibility. The administration of chemical contrast agents is not always feasible and may be contraindicated in patients with renal impairment or other health conditions. As a result, developing accurate and non-invasive contrast enhanced MRI (CEMRI) synthesis methods has clinical importance. In recent years, numerous approaches for CEMRI synthesis have been proposed, predominantly relying on generative artificial intelligence models. While these methods demonstrate promising performance, their dependence on implicit feature learning often limits their ability to preserve anatomical boundaries and tumour-specific fine structures. To address these challenges, we propose an anatomically aware frequency-and-structure-guided vision transformer (AA-ViT), for CEMRI synthesis using pre-contrast MRI modalities (T1, T2, and FLAIR). Experiments on the BraTS 2021 dataset demonstrate that the proposed method preserves anatomical and lesion boundaries, achieving higher PSNR and SSIM than state-of-the-art approaches. Clinical evaluation by three neuroradiologists and a neurosurgeon on 19 randomly selected cases across diverse gliomas yielded a mean score of 3.94/5, providing preliminary clinical validation rarely seen in prior studies. Synthetic post-contrast scans from our model could lower scanning costs, shorten imaging time, and avoid the potential risks of using gadolinium-based contrast agents.

Chinese Translation

准确的肿瘤定位和诊断是脑癌临床护理的重要组成部分。磁共振成像（MRI）是最常用的成像方式，因为它具有优越的软组织对比度。然而，标准MRI通常表现出有限的对比度和成像伪影，这需要使用对比剂来增强病变的可见性。化学对比剂的使用并不总是可行，并且在肾功能受损或其他健康状况的患者中可能是禁忌的。因此，开发准确且非侵入性的对比增强MRI（CEMRI）合成方法具有临床重要性。近年来，已经提出了许多CEMRI合成的方法，主要依赖于生成性人工智能模型。虽然这些方法表现出良好的性能，但它们对隐式特征学习的依赖往往限制了它们保持解剖边界和肿瘤特定细微结构的能力。为了解决这些挑战，我们提出了一种具有解剖意识的频率和结构指导视觉变换器（AA-ViT），用于利用预对比MRI模态（T1、T2和FLAIR）进行CEMRI合成。在BraTS 2021数据集上的实验表明，所提出的方法能够保持解剖和病变边界，获得比现有最先进方法更高的PSNR和SSIM。三位神经放射科医师和一位神经外科医生对19例随机选择的不同类型胶质瘤病例的临床评估得分平均为3.94/5，为先前研究中罕见的初步临床验证提供了依据。我们模型合成的后对比扫描可以降低扫描成本，缩短成像时间，并避免使用基于钆的对比剂的潜在风险。

View on arXiv Download PDF AI Translation

cs.CV / 79 / 2607.07580

Automatic Echocardiography Segmentation via Transition Probability Correlation for Stable Semantic Extraction

基于转移概率相关的自动超声心动图分割以实现稳定的语义提取

Chen, Xinran, Wang, Xiyuan, Zhou, Guangquan, Chen, Chuan

Abstract

While echocardiography is essential for cardiovascular diagnosis, inherent speckle noise and low signal-to-noise ratio often lead to ambiguous semantic features and fragmented boundaries. These limitations significantly hinder the segmentation accuracy of deep learning models in complex clinical cases. Moreover, temporal motion of the heart plays a critical role in recognizing anatomical structures. To address these challenges, we designed a STLSF module which comprises a window-matching-based semantic correction component and a semantics-guided texture enhancement component. By leveraging local transition probability correlations to correct semantics and employing semantics-guided texture enhancement, the STLSF module effectively mitigates texture instability and ambiguous semantic interpretations caused by disadvantaged echocardiography quality. Additionally, to facilitate the encoder's adaptation to the intrinsic priors of ultrasound-specific imaging patterns, we propose a frequency-aware denoising pre-training method. The entire work builds a convolution-based network with locality inductive bias and long-range dependencies. Extensive experiments confirm our SOTA performance, achieving 93.87\% Dice on CAMUS and 92.62\% on EchoNet-Dynamic, with respective HD95 values of 3.29mm and 2.73mm.

Chinese Translation

超声心动图对于心血管诊断至关重要，但固有的斑点噪声和低信噪比常常导致模糊的语义特征和破碎的边界。这些局限性显著阻碍了深度学习模型在复杂临床案例中的分割准确性。此外，心脏的时间运动在识别解剖结构中发挥着关键作用。为了解决这些挑战，我们设计了一个STLSF模块，该模块包括基于窗口匹配的语义修正组件和语义引导的纹理增强组件。通过利用局部转移概率相关性来修正语义，并采用语义引导的纹理增强，STLSF模块有效减轻了由于超声心动图质量不佳而导致的纹理不稳定性和模糊的语义解释。此外，为了促进编码器适应超声特定成像模式的内在先验，我们提出了一种频率感知去噪预训练方法。整个工作构建了一个具有局部归纳偏置和长距离依赖的卷积网络。大量实验验证了我们的SOTA表现，在CAMUS上达到了93.87%的Dice系数，在EchoNet-Dynamic上达到了92.62%，相应的HD95值为3.29mm和2.73mm。

View on arXiv Download PDF AI Translation

cs.CV / 80 / 2607.07581

Cardiac MRI Through-Plane Super-Resolution Guided by Reference and Memory

参考和记忆引导的心脏MRI平面超分辨率

Pan, Shaoming, Hu, Chenchuhui, Axel, Leon, Ye, Meng

Abstract

Clinical cardiac MRI is commonly acquired with high in-plane resolution but coarse through-plane resolution to reduce scan time and accommodate breath-hold and cardiac-motion constraints, which limits 3D analysis and diagnostic accuracy. We propose STRMSR, a reference- and memory-guided through-plane super-resolution (SR) framework that reconstructs high-resolution (HR) cardiac volumes by leveraging HR reference views acquired from the same subject and intermediate SR results as the memory. Our method uses coarse-to-fine contextual matching to establish robust correspondence between low-resolution target and reference/memory images under spatial misalignment. A learnable patch-wise dynamic feature aggregation module predicts content-adaptive mixture weights for each local patch, effectively fusing dynamic information while suppressing unreliable feature transfers. The intermediate SR results stored in the memory bank ensure slice-to-slice consistency for the super-resolved 3D volume. Experiments on the WHS cardiac MRI dataset under two reference protocols, orthogonal-plane views and long-axis chamber views, demonstrate consistent improvements over baselines at 4x and 8x upsampling factors.

Chinese Translation

临床心脏MRI通常以高平面分辨率但粗糙的垂直分辨率进行采集，以减少扫描时间并适应呼吸保持和心脏运动的限制，这限制了3D分析和诊断准确性。我们提出了STRMSR，一种参考和记忆引导的垂直超分辨率(SR)框架，通过利用从同一受试者获取的高分辨率(HR)参考视图和中间SR结果作为记忆，重建高分辨率的心脏体积。我们的方法使用粗到细的上下文匹配，在空间错位的情况下建立低分辨率目标与参考/记忆图像之间的稳健对应关系。一个可学习的基于块的动态特征聚合模块为每个局部块预测内容自适应混合权重，有效融合动态信息，同时抑制不可靠的特征转移。存储在记忆库中的中间SR结果确保了超分辨率3D体积的切片间一致性。在两个参考协议下的WHS心脏MRI数据集上的实验，正交平面视图和长轴腔室视图，展示了在4倍和8倍上采样因子下相较于基线的一致性改进。

View on arXiv Download PDF AI Translation

cs.CV / 81 / 2607.07673

MedPMC: A Systematic Framework for Scaling High-Fidelity Medical Multimodal Data for Foundation Models

MedPMC：一个用于扩展高保真医学多模态数据的系统框架，以支持基础模型

Kim, Hyunjae, Kim, Dain, Xiao, Pan, Applebaum, Serina S., Chung, Younjoon, Ai, Xuguang, Yin, Yu, Jiang, Roy, Du, Yuexi, Wei, Yawen, Kong, Yiming, Guo, Tuo, Cao, Zhiyuan, Du, Mengmeng, Fu, Yuelei, Hu, Yan, Shi, Rui, Yang, Gui, Jin, Kevin W., Liu, Yuntian, Tian, Yuxuan, Marquez, Jonathan, Chen, Zhen, Zhang, Sheng, Poon, Hoifung, Xu, Hua, Kang, Jaewoo, Chen, Qingyu

Abstract

Medicine is inherently multimodal, requiring clinicians to synthesize information across diverse data streams. Yet the development of multimodal foundation models is constrained by limited access to large-scale, high-quality clinical data. Although PubMed Central (PMC) offers a complementary source of expert-authored image-text data, existing PMC-derived resources remain limited in fidelity, reproducibility, and clinical validation. We introduce MedPMC, an automated, continuously updatable framework that transforms permissively licensed literature into high-fidelity infrastructure for medical multimodal models. Applied to 6.1 million PMC articles, MedPMC curated 11 million medical image-text pairs. Component evaluations showed strong performance for initial screening (F1 = 93.2), multi-panel figure detection (F1 = 96.5), figure separation (mAP = 89.8), caption separation and alignment (F1 = 81.4; ROUGE-L = 85.3), and medical figure classification (F1 = 96.5). Manual review by five annotators, three with medical training, found 95.3% of MedPMC images medically relevant, versus 19.7% in a prior PMC-derived dataset. Across 26 benchmarks spanning 11 specialties, a MedPMC-trained CLIP-style model improved average zero-shot AUC by 7.1 percentage points over the strongest architecture-matched biomedical CLIP baseline despite using fewer than half as many image-text pairs. As the vision encoder in a multimodal large language model, it improved medical visual question-answering by 1.9 and 16.9 percentage points across two benchmarks. In 10,524 Yale New Haven Health System dermatology photographs, it improved morphology-to-image retrieval Recall@5 by 11.7 percentage points. These findings show that high-fidelity literature curation strengthens medical multimodal foundation models across benchmark and clinical settings. We publicly release the framework, corpus, benchmarks, and pretrained models.

Chinese Translation

医学本质上是多模态的，要求临床医生在不同的数据流中综合信息。然而，多模态基础模型的发展受到大规模高质量临床数据获取限制的制约。尽管PubMed Central (PMC) 提供了一种补充来源的专家撰写的图像-文本数据，但现有的PMC衍生资源在保真度、可重复性和临床验证方面仍然有限。我们介绍了MedPMC，一个自动化、可持续更新的框架，将许可宽松的文献转化为高保真的医学多模态模型基础设施。应用于610万篇PMC文章，MedPMC整理了1100万对医学图像-文本。组件评估显示初步筛选的强大性能（F1 = 93.2）、多面板图形检测（F1 = 96.5）、图形分离（mAP = 89.8）、标题分离与对齐（F1 = 81.4；ROUGE-L = 85.3）以及医学图形分类（F1 = 96.5）。五位注释者的人工审查中，三位具有医学背景，发现95.3%的MedPMC图像在医学上相关，而在之前的PMC衍生数据集中仅为19.7%。在涵盖11个专业的26个基准测试中，MedPMC训练的CLIP风格模型在使用不到一半的图像-文本对的情况下，平均零-shot AUC比最强的架构匹配生物医学CLIP基线提高了7.1个百分点。作为多模态大型语言模型中的视觉编码器，它在两个基准测试中提高了医学视觉问答的表现，分别提高了1.9和16.9个百分点。在10,524张耶鲁新哈芬健康系统的皮肤科照片中，它将形态到图像检索的Recall@5提高了11.7个百分点。这些发现表明，高保真的文献整理增强了医学多模态基础模型在基准和临床环境中的表现。我们公开发布该框架、语料库、基准测试和预训练模型。

View on arXiv Download PDF AI Translation

cs.CV / 82 / 2607.07675

Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence

扩展混合专家视频预训练以实现具身智能

Ma, Shuailei, Liao, Jiaqi, Wang, Xinyang, Wang, Jingjing, Feng, Chaoran, Hu, Zijing, Bao, Chong, Xi, Zichen, Gan, Yuqi, Wang, Weisen, Zeng, Yanhong, Zhao, Qin, Shi, Zifan, Wu, Wei, Ouyang, Hao, Wang, Qiuyu, Zhang, Shangzhan, Shao, Jiahao, Sun, Yipengjing, Hu, Liangxiao, Pan, Lunke, Xue, Nan, Zheng, Kecheng, Xu, Yinghao, Zhu, Xing, Shen, Yujun, Cheng, Ka Leong

Abstract

Despite the recent promise in robot control, video generative models suffer from a domain mismatch due to their primary focus on content creation. For example, their design inherently prioritizes visual fidelity and creativity over computational efficiency and physical realism. In this work, we present LingBot-Video, a DiT-based video pretraining paradigm specifically tailored for embodied intelligence. From the architecture perspective, we adopt the Mixture-of-Experts (MoE), instead of dense, framework to achieve a better trade-off between modeling capacity and inference efficiency, and manage to scale it up from scratch. From the data perspective, we construct a data profiling engine that augments standard internet videos with extensive robot-oriented footage, encompassing manipulation, navigation, and egocentric perspectives, to equip the base model with an intrinsic understanding of actions and world dynamics. From the training perspective, we develop a multi-dimensional reward system to enforce the alignment regarding physical rationality and task completion, going beyond standard criteria such as aesthetics, prompt-following, and motion consistency. Comprehensive evaluations validate its performance and efficiency as a video foundation model. We contribute LingBot-Video as the inaugural large-scale, open-source MoE video foundation model to the community, in a pioneering effort to bridge digital creativity and physical actuation.

Chinese Translation

尽管在机器人控制方面最近取得了一些进展，视频生成模型由于主要关注内容创作而遭遇领域不匹配的问题。例如，它们的设计本质上优先考虑视觉真实感和创造力，而非计算效率和物理现实性。在本研究中，我们提出了LingBot-Video，一种基于DiT的视频预训练范式，专门为具身智能量身定制。从架构的角度来看，我们采用了混合专家（Mixture-of-Experts, MoE）框架，而非密集框架，以实现建模能力与推理效率之间更好的权衡，并成功从零开始进行扩展。从数据的角度来看，我们构建了一个数据分析引擎，通过大量机器人导向的录像（包括操控、导航和自我中心视角）来增强标准互联网视频，以使基础模型具备对动作和世界动态的内在理解。从训练的角度来看，我们开发了一个多维奖励系统，以强化物理合理性和任务完成之间的对齐，超越了美学、遵循提示和运动一致性等标准标准。全面的评估验证了其作为视频基础模型的性能和效率。我们将LingBot-Video作为首个大规模开源的MoE视频基础模型贡献给社区，开创性地努力在数字创造力与物理执行之间架起桥梁。

View on arXiv Download PDF AI Translation

人工智能 (Artificial Intelligence)

cs.AI / 1 / 2607.06624

AgentLens: Production-Assessed Trajectory Reviews for Coding Agent Evaluation

AgentLens：用于编码代理评估的生产评估轨迹审查

Podivilov, Andrey, Lomshakov, Vadim, Savin, Sergey, Startsev, Matvei, Pozharskiy, Roman, Parshin, Maksim, Nikolenko, Sergey

Abstract

We present AgentLens, a production-assessed benchmark for interactive code agents. Most code-agent benchmarks reduce a run to a single bit -- did the task pass? -- but the people who actually use these agents experience the entire trajectory: how the agent follows instructions, uses its tools, verifies its own work, recovers from mistakes, and talks to them along the way. AgentLens evaluates that whole trajectory. It pairs formal verification, where an objective check exists, with LLM-written trajectory reviews and side-by-side comparisons, so that each run yields a readable explanation of why the score is what it is. This makes AgentLens useful for more than ranking models: we use it to diagnose model behavior, compare successive versions of our own agent, and catch product regressions in a nightly evaluation pipeline. We release the benchmark as open source at https://github.com/agent-lens/agent-lens-bench.

Chinese Translation

我们提出了AgentLens，这是一个针对交互式代码代理的生产评估基准。大多数代码代理基准将一次运行简化为一个二元结果——任务是否通过？——但实际使用这些代理的人经历的是整个轨迹：代理如何遵循指令、使用工具、验证自身工作、从错误中恢复以及在此过程中与他们的对话。AgentLens评估整个轨迹。它将形式验证（在此存在客观检查）与LLM撰写的轨迹审查和并排比较相结合，使得每次运行都能生成可读的解释，说明得分的原因。这使得AgentLens不仅对模型排名有用：我们还利用它来诊断模型行为、比较我们自己代理的连续版本，并在夜间评估管道中捕捉产品回归。我们将该基准作为开源项目发布，网址为https://github.com/agent-lens/agent-lens-bench。

View on arXiv Download PDF AI Translation

cs.AI / 2 / 2607.06720

When Does In-Context Search Help? A Sampling-Complexity Theory of Reflection-Driven Reasoning

何时上下文搜索有助于推理？反思驱动推理的采样复杂性理论

Wolf, Yotam, Wies, Noam, Shashua, Amnon

Abstract

Training large language models (LLMs) with extended reasoning has enabled in-context search, in which models iteratively generate, critique, and revise solution attempts. We provide a theoretical analysis of in-context search by modeling it as approximate inference over reasoning traces, where the base model defines a prior and self-reflection provides feedback for posterior updates, and study the resulting inference-time sampling complexity - the number of sequential attempts needed to achieve high success probability. We show that when reflections reliably localize early mistakes, in-context search can yield exponential improvements over the base model, solving problems with exponentially small zero-shot pass rates using only a polynomial number of sequential attempts, whereas when this property fails, conditioning on past attempts offers no asymptotic benefit over parallel sampling. We further show that these gains are robust and learnable: approximate posterior updates suffice, and cross-entropy training on search rollouts recovers the required behavior with polynomial sample complexity. Finally, we show that under a stagewise abstraction of reinforcement learning with verifiable rewards, the optimal policy extension implements the same posterior reweighting rule. We validate key qualitative predictions of the theory on real large reasoning models.

Chinese Translation

对大型语言模型（LLMs）进行扩展推理训练使得上下文搜索成为可能，在这种情况下，模型迭代生成、批判和修订解决方案尝试。我们通过将上下文搜索建模为对推理轨迹的近似推断，提供了对上下文搜索的理论分析，其中基础模型定义了先验，而自我反思则为后验更新提供反馈，并研究了由此产生的推断时采样复杂性——实现高成功概率所需的顺序尝试次数。我们表明，当反思能够可靠地定位早期错误时，上下文搜索可以在基础模型之上实现指数级的改进，仅需多项式数量的顺序尝试即可解决具有指数小零-shot通过率的问题，而当这一特性失效时，基于过去尝试的条件化对比于并行采样没有渐近优势。我们进一步表明，这些收益是稳健且可学习的：近似后验更新足够，基于搜索回滚的交叉熵训练能够以多项式样本复杂度恢复所需行为。最后，我们展示在具有可验证奖励的强化学习的阶段抽象下，最优策略扩展实现了相同的后验重加权规则。我们在真实的大型推理模型上验证了理论的关键定性预测。

View on arXiv Download PDF AI Translation

cs.AI / 3 / 2607.06757

LLM-powered reasoning in agent-based modeling

基于LLM的代理建模推理

Moon, Sifat Afroj, Maguire, Dakotah, Spannaus, Adam, Tuccillo, Joe, Alam, Maksudul, Seal, Sudip K., Gounley, John, Hanson, Heidi

Abstract

Agent-based modeling (ABM) has the capability to model millions of individuals and their interactions, which is useful for policy making. However, ABMs have traditionally relied on static prior, which prevents the models from adapting to real-time changes. Our research provides a novel approach to addressing this information gap. Large language models (LLMs) offer new opportunities to predict human decision-making. Here, we introduce a scalable Hybrid Agent-based and Language-driven Epidemic (HALE) modeling framework that leverages LLMs to predict human decision-making in an ABM simulation. As a proof-of-concept, we use HALE to simulate COVID-19 and its effects in Salt Lake County, UT.

Chinese Translation

代理建模（ABM）能够模拟数百万个个体及其相互作用，这对于政策制定非常有用。然而，传统的ABM依赖于静态先验，这限制了模型适应实时变化的能力。我们的研究提供了一种新颖的方法来解决这一信息缺口。大型语言模型（LLMs）为预测人类决策提供了新的机会。在此，我们介绍了一种可扩展的混合代理和语言驱动的流行病（HALE）建模框架，该框架利用LLMs预测ABM模拟中的人类决策。作为概念验证，我们使用HALE模拟了COVID-19及其在犹他州盐湖县的影响。

View on arXiv Download PDF AI Translation

cs.AI / 4 / 2607.06760

QANTIS: Hardware-Calibrated Sequential POMDP Belief Updates on IBM Heron

QANTIS：在IBM Heron上进行硬件校准的序列POMDP信念更新

Eker, Bayram Yuksel, Arslan, Suayb S., Nazli, Ozgur, Demirgil, Mustafa Serhat, Deligoz, Furkan

Abstract

Autonomous systems under partial observability act on beliefs, not raw sensor events. QANTIS treats the quantum processor as a calibrated belief-update service in that loop: it receives a prior and an observation model, estimates the rare-event evidence term, and returns an ordinary posterior to a classical planner. This paper asks whether that service can be reused across a sequential Tiger POMDP horizon on present IBM Heron hardware without corrupting the planner-facing posterior. We answer with a controlled hardware case study rather than an end-to-end autonomy or wall-clock speedup claim. The study compares no amplification, guarded Grover amplification, and all-step fixed-point amplification on the same trajectory, then checks whether the returned posterior would change the downstream action. All-step FPAA preserves the Tiger posterior across the reported 8-step and 12-step primary runs, and the 20-step and 32-step controls remain inside the same operating band. In every reported decision check, the hardware posterior and the exact Bayes posterior select the same immediate action. Boundary-aware BIQAE stabilizes amplitude estimation near zero and near one, while a rare-event sweep maps the logical sample-complexity envelope for one-in-a-million evidence. The result is an operating envelope for a hardware-calibrated belief-update primitive, not a standalone hardware-advantage claim.

Chinese Translation

在部分可观测性的自主系统中，系统依据信念而非原始传感器事件进行决策。QANTIS将量子处理器视为该循环中的一个校准信念更新服务：它接收先验和观测模型，估计稀有事件证据项，并将普通后验返回给经典规划器。本文探讨该服务是否可以在当前IBM Heron硬件上跨越序列Tiger POMDP的视野进行重用，而不会损害面向规划器的后验。我们通过一个受控的硬件案例研究来回答这个问题，而不是提出端到端的自主性或时钟加速的主张。该研究比较了在相同轨迹下无放大、受保护的Grover放大和全步固定点放大，然后检查返回的后验是否会改变下游动作。全步FPAA在报告的8步和12步主要运行中保持了Tiger后验，而20步和32步的控制仍然保持在相同的操作带内。在每次报告的决策检查中，硬件后验和精确的贝叶斯后验选择了相同的即时动作。边界感知的BIQAE在接近零和接近一的地方稳定幅度估计，而稀有事件扫描则为百万分之一的证据绘制了逻辑样本复杂度包络。结果是一个针对硬件校准信念更新原语的操作包络，而非独立的硬件优势主张。

View on arXiv Download PDF AI Translation

cs.AI / 5 / 2607.06764

Cost-Effective Agent Harnesses for Abstract Reasoning and Generalization on ARC-AGI-1

成本效益型代理工具在ARC-AGI-1上的抽象推理和泛化

Moghe, Kabir, Chin, Peter

Abstract

Recent progress on ARC-AGI-1 from disclosed architectures has come broadly from two regimes: heavy test-time compute over frontier models (evolutionary search, exhaustive sampling, extended chain-of-thought), or benchmark-specific training in which small models are fine-tuned on ARC data, often with task-specialized architectures. We study a third regime: an open-weight model in non-thinking mode (DeepSeek V3.2) under a strict budget, with no ARC-specific fine-tuning. We study what is recoverable through architecture alone, building agentic harnesses that decompose pattern-discovery and program-synthesis stages explicitly. First, we introduce an Explorer-Definer Pipeline that separates pattern discovery from executable transformation synthesis, implemented as a two-stage agent pipeline. Next, we present the Reflective Orchestrator, which augments the pipeline with autonomous exploration of new transformations when previous hypotheses fail on training pairs. On the ARC-AGI-1 public 400-task evaluation set, the pipeline reaches 57.50% pass@2 at \$0.25 per task, and the orchestrator reaches 67.25% pass@2 at \$0.62 per task. Together these architectures lift a 15.50% one-shot baseline by ~52 points without benchmark-specific training or heavy test-time compute. Furthermore, the orchestrator-driven lift tests a falsifiable diagnostic the pipeline produces; unbiased pass@k analysis suggests the pipeline is generation-bound, not selection-bound (selection via training-pair accuracy captures ~95% of the candidate ceiling) and predicts that significant improvement requires broader generation, not better ranking. The orchestrator implements this prediction via adaptive re-exploration and confirms it (unbiased pass@1 lift +9.81 pp, matching selection-mediated pass@2 lift). An additional pipeline ablation identifies its think tool as a significant component, with removal reducing pass@2 by 5.75 pp.

Chinese Translation

近期在ARC-AGI-1上取得的进展主要来自于两种模式：在前沿模型上进行大量的测试时间计算（进化搜索、全面采样、扩展思维链），或在基准特定训练中，小型模型在ARC数据上进行微调，通常使用任务专用架构。我们研究了第三种模式：在严格预算下的非思考模式开放权重模型（DeepSeek V3.2），没有进行ARC特定的微调。我们研究了仅通过架构可以恢复的内容，构建了明确分解模式发现和程序合成阶段的代理工具。首先，我们引入了一个探索-定义管道，将模式发现与可执行转换合成分开，实施为一个两阶段的代理管道。接下来，我们展示了反思协调器，它在先前假设在训练对上失败时，通过自主探索新转换来增强管道。在ARC-AGI-1公共400任务评估集上，该管道以每个任务0.25美元的成本达到了57.50%的pass@2，而协调器以每个任务0.62美元的成本达到了67.25%的pass@2。这些架构共同将15.50%的单次基线提升了约52个百分点，而无需基准特定训练或大量测试时间计算。此外，协调器驱动的提升测试了管道产生的可证伪诊断；无偏的pass@k分析表明，管道受限于生成，而非选择（通过训练对准确性进行选择捕获了约95%的候选上限），并预测显著的改进需要更广泛的生成，而不是更好的排名。协调器通过自适应重新探索实现了这一预测，并确认了这一点（无偏pass@1提升9.81个百分点，匹配选择介导的pass@2提升）。额外的管道消融实验确定其思考工具是一个重要组成部分，去除后pass@2降低了5.75个百分点。

View on arXiv Download PDF AI Translation

cs.AI / 6 / 2607.06820

Evaluating SageMath-Augmented LLM Agents for Computational and Experimental Mathematics

评估增强SageMath的LLM智能体在计算与实验数学中的应用

Snopov, Pavel, Magai, German

Abstract

Recent advances in AI for Mathematics have focused largely on autoformalization and theorem proving, leaving the role of Computer Algebra Systems (CAS) in agentic LLM workflows underexplored. We propose a ReAct-style agentic setup that combines LLM reasoning with verifiable feedback from SageMath, together with Context7 for the up-to-date documentation. We evaluate this agentic setup across frontier models for solving research-level mathematical problems from the RealMath benchmark in a setting that emulates a computational-mathematics research loop. We also propose a refinement to the RealMath benchmark by introducing a multi-step post-processing procedure and a multi-stage validation pipeline, both of which improve the quality and reliability of the extracted problem set. Our experiments reveal substantial performance gains from SageMath access across all evaluated models on +9.7~pp on average, the gains range from 1.5~pp to 27.8~pp and narrow the gap between open-weight and closed models. Qwen~3.7-Max benefits from SageMath the most, while GPT-5.5 achieves the highest solve rate of $75.2\%$ and the lowest token usage among tool-enabled configurations. Our findings suggest that CAS-augmented agents represent a promising direction for assisting mathematicians in computational exploration, and we believe that this work is a step towards automated conjecture discovery. The project repository is available online.

Chinese Translation

近期在数学领域的人工智能进展主要集中于自动形式化和定理证明，计算代数系统（CAS）在智能LLM工作流程中的作用尚未得到充分探讨。我们提出了一种ReAct风格的智能体设置，将LLM推理与来自SageMath的可验证反馈相结合，并使用Context7提供最新文档。我们在模拟计算数学研究循环的环境中，评估这一智能体设置在解决来自RealMath基准的研究级数学问题时的表现。我们还通过引入多步骤后处理程序和多阶段验证管道，对RealMath基准进行了改进，这两者均提高了提取问题集的质量和可靠性。我们的实验显示，所有评估模型在使用SageMath时平均性能提升了9.7个百分点，提升幅度从1.5个百分点到27.8个百分点不等，缩小了开放权重模型与封闭模型之间的差距。Qwen 3.7-Max在SageMath的帮助下获益最大，而GPT-5.5在工具启用配置中实现了最高的解题率75.2%和最低的令牌使用量。我们的研究结果表明，增强CAS的智能体代表了一个有前景的方向，可以帮助数学家进行计算探索，我们相信这项工作是朝着自动猜想发现迈出的重要一步。项目代码库已在线发布。

View on arXiv Download PDF AI Translation

cs.AI / 7 / 2607.06906

The Harness Effect: How Orchestration Design Sets the Token Economics of Enterprise Agentic AI

驾驭效应：编排设计如何设定企业自主智能代理的代币经济学

Ali, Muayad Sayed, Novik, Aliaksandra, Boddupally, Anji, Yavorskyi, Artem, Nickerson, Chris, Rica, Daniel, DuGranrut, Emily, Leung, Felix, Prince, Garrett, Barnett, Grace, Robinson, Heath, Ahmad, Hosain Al, Resnick, Jesse, Farah, Juan Carlos, Meruga, Jyothi Swaroop, Kuznetsov, Leonid, Gorham, Luke, Schmoll, Marie, Paciullo, Michael, Das, Saumya, Sheripally, Sharath, Griscom, Tommy, Osadchyi, Mykyta, Mantri, Neha, Westrum, Nick, Benowitz, Olivia, Kulkarni, Parikshith, Chernyshov, Radik, Vasudev, Rakshith, Nadimpally, Rohith, Gangadevi, Vikas, AlShikh, Waseem

Abstract

Agentic AI development today runs on token maxing: buying capability with tokens -- longer reasoning traces, more turns, wider tool payloads, bigger replayed contexts -- so tokens per task grow faster than task value. Falling per-token prices mask the pattern; total spend rises anyway. We argue the decisive lever against token maxing is the harness: the orchestration layer that assembles context, exposes tools, sequences turns, delegates work, and carries enterprise observability and governance. We isolate it with a controlled swap: 22 locked evaluation tasks, six foundation models (Claude Sonnet 4.6, Gemini 3.1, Gemini Flash 3.5, Qwen 3.6, GLM 5.1, Palmyra X6), changing only the orchestration layer -- a frozen conventional production loop versus the Writer Agent Harness. Holding models constant, the harness cuts blended cost per task 41% ($0.21->$0.12), median wall-clock 44% (48s->27s), and tokens per task 38% (14.2k->8.8k), with task-completion quality at parity (0.78->0.81, directional at this sample size). Efficiency is model-invariant -- every model gets cheaper (33-61%) -- while quality gains are capability-dependent: a model's gain correlates almost perfectly with its baseline strength (r=0.99, n=6), a phenomenon we term harness leverage. Quality per dollar rises 82%; task-completions per million tokens rise from 54.9 to 92.0. On this workload the orchestration layer moved cost per task more than the full spread of the model menu did. We formalize token economics at the orchestration layer (including effective input price under prompt caching), detail the six mechanism families behind the effect -- cache-shape discipline to failure-spend governance -- compare six widely used agent systems on the same axes, and argue the harness is the one component whose efficiency multiplies across every model an organization runs -- present and future.

Chinese Translation

当前自主智能代理的开发依赖于代币最大化：用代币购买能力——更长的推理轨迹、更多的回合、更广的工具负载、更大的重放上下文——因此每个任务的代币数量增长速度超过任务的价值。每个代币价格的下降掩盖了这一模式；总支出依然上升。我们认为，抵制代币最大化的关键杠杆是“驾驭”：编排层负责组装上下文、暴露工具、排列回合、委派工作，并承担企业的可观察性和治理。我们通过受控交换将其单独隔离：22个锁定的评估任务，六个基础模型（Claude Sonnet 4.6、Gemini 3.1、Gemini Flash 3.5、Qwen 3.6、GLM 5.1、Palmyra X6），仅改变编排层——一个冻结的传统生产循环与Writer Agent Harness。保持模型不变，驾驭将每个任务的混合成本降低41%（$0.21->$0.12），中位墙钟时间降低44%（48秒->27秒），每个任务的代币数量降低38%（14.2k->8.8k），而任务完成质量保持平衡（0.78->0.81，在此样本量下方向性）。效率是模型不变的——每个模型的成本都降低（33-61%）——而质量的提升依赖于能力：模型的增益几乎与其基线强度完美相关（r=0.99，n=6），这一现象我们称之为驾驭杠杆。每美元的质量提升82%；每百万代币的任务完成次数从54.9上升到92.0。在这一工作负载下，编排层在每个任务的成本上所做的改进超出了整个模型菜单的范围。我们在编排层上形式化代币经济学（包括在提示缓存下的有效输入价格），详细说明了导致该效应的六个机制家族——缓存形状、失败支出治理——在相同轴上比较六个广泛使用的代理系统，并认为驾驭是唯一一个在组织运行的每个模型中都能实现效率倍增的组件——无论是现在还是未来。

View on arXiv Download PDF AI Translation

cs.AI / 8 / 2607.06925

Grounding Spatial Relations in a Compact World Model: Instruction Leakage and a Goal-Free Dynamics Fix

在紧凑世界模型中扎根空间关系：指令泄漏与无目标动态修复

Wang, Yufeng, Wei, Lu, Ling, Haibin

Abstract

Compact world models that condition on a language goal promise to ground relations such as ``put the red block left of the blue block'' using a sparse set of explicit \emph{reference anchors}. We ask when such references actually ground a relation, and identify a trap: a goal-conditioned predictor reaches a striking $0.90$ relation-readout accuracy, yet this is \emph{instruction transcription}, not perception. Withholding the goal collapses it to chance ($0.90\!\to\!0.27$, three seeds) and a counterfactual instruction makes the predicted anchors follow the \emph{false} instruction $94.5\%$ of the time (true scene $2.3\%$; $N{=}256$). Tested across three settings and a within-task ablation, our central claim characterizes the confound: \textbf{instruction leakage occurs when the scored quantity is transcribable from the instruction (when the instruction names the answer) and is essentially independent of how predictive the non-instruction inputs are.} Our tabletop and the external BabyAI benchmark leak, whereas a Language-Table forward-dynamics world model whose instruction names \emph{referents} does not, until the instruction is augmented to name the direction; and degrading the action never increases leakage, the opposite of what predictor-competition predicts. The diagnosis prescribes the fix: keep the goal out of the dynamics (it belongs to the planner's cost) and supervise the \emph{read} path, recovering genuine, instruction-independent grounding ($0.88$, identical with and without the goal). The detection protocol and remedy apply to any goal-conditioned world model whose instruction names the scored quantity.

Chinese Translation

紧凑的世界模型通过语言目标进行条件化，承诺能够利用一组稀疏的显式 extit{参考锚点}来扎根诸如“将红色方块放在蓝色方块左侧”的关系。我们探讨何时这些参考实际上能够扎根关系，并识别出一个陷阱：一个目标条件的预测器达到了惊人的$0.90$关系读取准确率，但这实际上是 extit{指令转录}而非感知。当目标被隐去时，准确率降至偶然水平($0.90 o0.27$, 三个种子)；而一个反事实指令使得预测的锚点$94.5\%$的时间遵循 extit{错误}指令（真实场景为$2.3\\%$；$N=256$）。在三个设置和一个任务内消融实验中，我们的核心主张描述了混淆： extbf{当得分量可以从指令转录时（即指令命名答案时），指令泄漏发生，并且本质上独立于非指令输入的预测能力。} 我们的桌面实验和外部BabyAI基准存在泄漏，而一个语言-桌面前向动态世界模型，其指令命名 extit{指称物}，则没有泄漏，直到指令被增强以命名方向；而降低动作从未增加泄漏，这与预测器竞争的预期相反。诊断结果提出了修复方案：将目标排除在动态之外（它属于规划者的成本）并监督 extit{读取}路径，从而恢复真正的、独立于指令的扎根（$0.88$，无论有无目标均相同）。该检测协议和修复方法适用于任何目标条件的世界模型，其指令命名得分量。

View on arXiv Download PDF AI Translation

cs.AI / 9 / 2607.06993

Large Behavior Model: A Promptable Digital Twin of the Retail Customer

大型行为模型：零售客户的可提示数字双胞胎

Modecrua, Wachiravit, Pachtrachai, Krittin, Kraisingkorn, Touchapon

Abstract

Customer behavior modeling underpins recommendation, marketing, and decision support, yet existing approaches either optimize predictive accuracy without explaining decisions or simulate users without grounding them in real behavioral data. We present the Large Behavioral Model (LBM) that learns customer decision making directly from large-scale retail transactions through a unified Person-Environment formulation. Customer state is represented by a behavioral profile derived from historical purchases, while product context is incorporated through retrieval-augmented generation. The model is trained using continued pre-training on verbalized behavioral data, supervised fine-tuning for decision generation, and reinforcement learning with verifiable rewards for evidence-based calibration. We evaluate the proposed framework on purchase prediction, hard-negative discrimination, basket completion, promotion response, and cross-domain voucher redemption. The model consistently outperforms frontier general-purpose language models on in-domain retail tasks while demonstrating strong zero-shot and fine-tuned transfer across retailers and decision domains. Ablation studies show that continued pre-training is the primary driver of behavioral generalization, retrieval is most effective when applied during both training and inference, and reinforcement learning improves reliance on explicit behavioral evidence over generic language-model priors. These results demonstrate that behavioral knowledge encoded in transaction histories can be effectively learned by language models, providing a scalable foundation for customer digital twins and behavior simulation.

Chinese Translation

客户行为建模是推荐、营销和决策支持的基础，但现有方法要么优化预测准确性而不解释决策，要么模拟用户而不基于真实行为数据。我们提出了大型行为模型（Large Behavioral Model, LBM），该模型通过统一的人-环境（Person-Environment）框架直接从大规模零售交易中学习客户决策。客户状态由历史购买生成的行为特征表示，而产品上下文则通过检索增强生成（retrieval-augmented generation）进行整合。该模型通过对口头化行为数据的持续预训练、决策生成的监督微调以及基于可验证奖励的强化学习进行训练，以实现基于证据的校准。我们在购买预测、困难负样本区分、购物篮完成、促销响应和跨领域优惠券兑换等任务上评估了所提出的框架。该模型在领域内零售任务上始终优于前沿通用语言模型，同时在不同零售商和决策领域展示了强大的零样本和微调迁移能力。消融研究表明，持续预训练是行为泛化的主要驱动因素，检索在训练和推理阶段应用时最为有效，而强化学习则提高了对显性行为证据的依赖，优于通用语言模型的先验。这些结果表明，交易历史中编码的行为知识可以被语言模型有效学习，为客户数字双胞胎和行为模拟提供了可扩展的基础。

View on arXiv Download PDF AI Translation

cs.AI / 10 / 2607.07021

Learning social norms enhances compatibility in dynamic human-AI coordination

学习社会规范增强动态人机协调中的兼容性

Yang, Yi, Liu, Siyuan, Gao, Xin, Sun, Huamu, Liu, Chao, Zhou, Qing, Nie, Bingbing

Abstract

Humans continuously coordinate with others in dynamic interactions, often through implicit, hard-to-quantify social norms that act as shared tacit expectations among interacting agents. As AI agents, including large language models (LLMs), become embedded in daily life, they increasingly participate in such interactions and reshape social interaction structures. Yet they often fail to coordinate with humans in an effective, considerate, and natural manner. We hypothesize that this gap arises because existing approaches align model behavior with human demonstrations without explicitly quantifying the underlying norms that generate such behavior. We selected pedestrian-vehicle interaction as a representative dynamic interaction and developed a simplified experimental platform that captures its key interactive features. From 3,456 dynamic human interactions collected via this platform, we identified three principles underlying human social norms: outcome predictability, value alignment, and advantage awareness. Incorporating these principles into AI agents significantly improves human-AI coordination. In the closed-loop interaction task with humans, the social-norm-informed LLM achieved a nearly fourfold higher total score than the baseline strategy and outperformed human-human interactions by 43%. These findings indicate that formalizing tacit social norms into explicit, quantifiable principles can enable AI agents to achieve mutually beneficial coordination in dynamic interactions, supporting their more natural integration into human society.

Chinese Translation

人类在动态互动中不断与他人协调，通常通过隐含的、难以量化的社会规范，这些规范作为交互代理之间的共享默契期望。随着人工智能代理（包括大型语言模型，LLMs）逐渐融入日常生活，它们越来越多地参与此类互动，并重塑社会互动结构。然而，它们往往无法以有效、体贴和自然的方式与人类协调。我们假设，这一差距的产生是因为现有方法将模型行为与人类示范对齐，而未明确量化产生这种行为的潜在规范。我们选择行人-车辆互动作为一个代表性的动态互动，并开发了一个简化的实验平台，以捕捉其关键互动特征。从通过该平台收集的3,456个动态人类互动中，我们识别出人类社会规范的三个基本原则：结果可预测性、价值对齐和优势意识。将这些原则纳入人工智能代理显著改善了人机协调。在与人类的闭环互动任务中，基于社会规范的LLM的总得分几乎是基线策略的四倍，并且比人际互动高出43%。这些发现表明，将隐性社会规范形式化为明确、可量化的原则，可以使人工智能代理在动态互动中实现互利协调，支持它们更自然地融入人类社会。

View on arXiv Download PDF AI Translation

cs.AI / 11 / 2607.07040

Measuring Intelligence Beyond Human Scale

超越人类尺度的智能测量

Han, Jerry, Moschopoulos, Rafael, Colby, Ella, Goyal, Vishrut, Tu, Andrew, Ghods, Kia, Braverman, Mark, Hazan, Elad

Abstract

How can we measure intelligence beyond human capability? Human-authored benchmarks saturate, and above human capability, examiners may not know which tasks are both hard and verifiable. We argue that this difficulty is inherent to absolute-scale evaluation and propose a new paradigm based on relative measurement in which models generate public challenges that separate other systems. Aggregating these outcomes yields an adversarial psychometric rating system that can scale with the systems being measured. We describe practical protocols that reduce incentives for private-information attacks, support judge-free adjudication, and naturally scale with agent capabilities. We instantiate the framework across verifiable and open-ended, non-verifiable domains, illustrating how model-generated evaluation can continue to measure systems beyond the human frontier.

Chinese Translation

我们如何能够测量超出人类能力的智能？人类编写的基准测试已经饱和，而在超出人类能力的情况下，评估者可能不知道哪些任务既困难又可验证。我们认为，这种困难是绝对尺度评估固有的，并提出了一种基于相对测量的新范式，在该范式中，模型生成公共挑战以区分其他系统。聚合这些结果产生了一种对抗性心理测量评分系统，能够与被测量的系统进行扩展。我们描述了实用的协议，以减少对私人信息攻击的激励，支持无评判的裁决，并自然地与代理能力进行扩展。我们在可验证和开放式的非可验证领域中实例化该框架，说明模型生成的评估如何能够继续测量超越人类边界的系统。

View on arXiv Download PDF AI Translation

cs.AI / 12 / 2607.07097

Operational Reframing and Approval-Framed Delegation in Multi-Agent LLM Safety

多智能体大语言模型安全中的操作重构与批准框架委托

Liu, Lifei, Yu, Haoran, Jiang, Xiaochong, Wang, Su, Qian, Pin, Chen, Yihang

Abstract

Safety evaluations of multi-agent LLM systems often compare a direct prompt with a planner-executor pipeline and report the difference as a single "pipeline effect." We argue that this aggregate is difficult to interpret because it conflates three mechanisms: harmful intent may be reframed as plausible operational work, the planner may refuse or transform the request, and the executor may act under delegation prompts implying prior approval. To separate these factors, we introduce a five-condition controlled contrast design, evaluated on 30 synthetic harmful scenarios and an exploratory external validation set from four agent-safety benchmarks using LLM-judged compliance. Our results show that aggregate pipeline safety is not a stable architectural property. Operational reframing is the most portable risk signal, increasing compliance for GPT, Gemini, and DeepSeek across both scenario sets, while Claude is comparatively resistant. Planner behavior can offset this risk mainly through refusal; however, when the planner produces executable steps, the executor may become more compliant than under the direct operational baseline. Approval-framed delegation is sensitive to prompt design, model pairing, and scenario source, and a skeptical executor prompt sharply reduces compliance. Raw-direct model rankings can also mispredict deployed planner-executor behavior. Gemini is safest under raw direct prompts in the primary set yet shows the largest amplification with a Claude planner, rising from 8.9 percent to 38.9 percent compliance. GPTs near-zero aggregate pipeline effect instead hides a reframing increase canceled by planner refusal. These findings suggest that multi-agent safety evaluations should report reframing, planner behavior, delegation framing, and model pairing separately before attributing failures to architecture itself.

Chinese Translation

多智能体大语言模型系统的安全评估通常将直接提示与规划者-执行者管道进行比较，并将差异报告为单一的“管道效应”。我们认为，这种汇总难以解释，因为它混淆了三种机制：有害意图可能被重新构建为合理的操作工作，规划者可能拒绝或转变请求，而执行者可能在暗示先前批准的委托提示下行动。为了分离这些因素，我们引入了一种五条件控制对比设计，在30个合成有害场景和来自四个智能体安全基准的探索性外部验证集上进行评估，使用大语言模型判断的合规性。我们的结果表明，汇总管道安全并不是一个稳定的架构属性。操作重构是最具可移植性的风险信号，在两个场景集上提高了GPT、Gemini和DeepSeek的合规性，而Claude则相对抵抗。规划者的行为主要通过拒绝来抵消这一风险；然而，当规划者生成可执行步骤时，执行者的合规性可能比直接操作基线更高。批准框架委托对提示设计、模型配对和场景来源敏感，而怀疑的执行者提示会显著降低合规性。原始直接模型排名也可能错误预测已部署的规划者-执行者行为。在主要集中的原始直接提示下，Gemini的安全性最高，但在Claude规划者的情况下却显示出最大的放大，从8.9%的合规性上升到38.9%。GPT的近零汇总管道效应则掩盖了被规划者拒绝抵消的重构增加。这些发现表明，多智能体安全评估应分别报告重构、规划者行为、委托框架和模型配对，然后再将失败归因于架构本身。

View on arXiv Download PDF AI Translation

cs.AI / 13 / 2607.07189

Does AI Understand Imaging? A Systematic Benchmark of Agentic AI for Computational Imaging Tasks

人工智能理解成像吗？针对计算成像任务的代理人工智能系统性基准测试

Chung, Ethan, Zheng, Chuanjun, Tan, Jasper, Li, Jingxi, Zhang, Haopeng, Chen, Huaijin

Abstract

Vision-language models (VLMs) and agentic AI have shown strong performance on semantic visual tasks, but it remains unclear whether they can handle the physics and inverse problems that underlie computational imaging. We present ImagingBench, a benchmark of 20 computational imaging tasks spanning five categories: ray and wave optics, image signal processing, inverse reconstruction, computational sensing, and calibration. ImagingBench evaluates three complementary settings: Expert, fixed expert-guided inverse reconstruction; Planner, planner-guided inverse reconstruction; and Forward, forward-system simulation for consistency checking. We benchmark leading proprietary and open-source image-centric multimodal systems, including Gemini, GPT, and Qwen, and compare them with representative task-specific non-agentic baselines. Across tasks, agentic models remain consistently weaker than specialized methods, especially on computational sensing problems such as lensless imaging, event-based reconstruction, time-of-flight imaging, and holography. Planner guidance provides only modest and inconsistent gains over the fixed-prompt Expert baseline. Although the models often generate visually plausible outputs, their reference-based fidelity remains poor, revealing a substantial gap between semantic visual competence and physically grounded imaging performance. ImagingBench provides a unified testbed for measuring this gap and tracking progress in agentic AI for computational imaging.

Chinese Translation

视觉-语言模型（VLMs）和代理人工智能在语义视觉任务上表现出色，但它们是否能够处理计算成像背后的物理学和逆问题仍不清楚。我们提出了ImagingBench，这是一个涵盖五个类别的20个计算成像任务的基准：光线和波动光学、图像信号处理、逆重建、计算传感和校准。ImagingBench评估了三种互补设置：专家（Expert），固定专家指导的逆重建；规划者（Planner），规划者指导的逆重建；以及前向（Forward），用于一致性检查的前向系统模拟。我们对领先的专有和开源图像中心多模态系统进行了基准测试，包括Gemini、GPT和Qwen，并将其与代表性的任务特定非代理基线进行了比较。在各项任务中，代理模型在专门化方法面前始终表现较弱，尤其是在无透镜成像、基于事件的重建、飞行时间成像和全息术等计算传感问题上。规划者指导仅在固定提示的专家基线之上提供了适度且不一致的增益。尽管这些模型通常生成视觉上可信的输出，但其基于参考的保真度仍然较差，揭示了语义视觉能力与物理基础成像性能之间的显著差距。ImagingBench提供了一个统一的测试平台，用于测量这一差距并跟踪代理人工智能在计算成像领域的进展。

View on arXiv Download PDF AI Translation

cs.AI / 14 / 2607.07229

Reasoning Consistency Scanning: A Framework for Auditing Chain-of-Thought Validity in AI Safety Evaluations

推理一致性扫描：审计人工智能安全评估中思维链有效性的框架

Santano, Silvia

Abstract

Prior work has shown that chain-of-thought (CoT) reasoning is often unfaithful: a model's stated reasoning does not reliably reflect the process that produced its output. Detecting unfaithfulness, though, requires controlled experimental interventions, which cannot be applied to evaluation transcripts after the fact. We turn instead to a more tractable question that has received less attention: whether the stated reasoning is logically consistent with the answer it accompanies. Unlike faithfulness, consistency can be assessed from a transcript alone, with no intervention. We introduce reasoning consistency scanning, a reusable method for detecting this property in AI safety evaluation transcripts. Our contributions are fourfold. First, we formalize reasoning consistency as distinct from faithfulness and define a six-subtype taxonomy of inconsistency. Second, we build a validated benchmark of 60 transcripts, manually adapted from InstrumentalEval outputs. Third, we implement a working scanner for InspectScout, the first to target this property in safety evaluation transcripts. Fourth, we report results across four generator models and three evaluations from inspect_evals, showing that reasoning inconsistency is present, detectable, and varies systematically across both models and task types.

Chinese Translation

先前的研究表明，思维链（Chain-of-Thought, CoT）推理常常不可靠：模型所陈述的推理并不能可靠地反映产生其输出的过程。然而，检测不可靠性需要控制实验干预，这在事后无法应用于评估记录。我们转而关注一个更易处理但关注较少的问题：所陈述的推理是否与其伴随的答案在逻辑上是一致的。与可靠性不同，一致性可以仅通过记录进行评估，无需干预。我们提出了推理一致性扫描，这是一种可重复使用的方法，用于检测人工智能安全评估记录中的这一特性。我们的贡献有四个方面。首先，我们将推理一致性形式化为与可靠性不同，并定义了六种不一致性子类型的分类法。其次，我们构建了一个经过验证的60个记录的基准，这些记录是从InstrumentalEval输出手动调整而来的。第三，我们为InspectScout实现了一个有效的扫描器，这是首个针对安全评估记录中这一特性的扫描器。第四，我们报告了在四个生成模型和三个inspect_evals评估中的结果，显示推理不一致性是存在的、可检测的，并且在模型和任务类型之间系统性地变化。

View on arXiv Download PDF AI Translation

cs.AI / 15 / 2607.07321

From Atomic Actions to Standard Operating Procedures: Iterative Tool Optimization for Self-Evolving LLM Agents

从原子动作到标准操作程序：自我演化大语言模型代理的迭代工具优化

Ding, Haipeng, Xie, Yuexiang, Wei, Zhewei, Li, Yaliang, Ding, Bolin

Abstract

Tool utilization enables Large Language Model (LLM) agents to interact with the real world and resolve complex tasks. However, existing agent frameworks predominantly rely on static toolsets composed of granular atomic actions (e.g., basic file I/O or single-turn search), which forces agents to reinvent low-level logic for every recurring workflow, leading to increased reasoning overhead and failure rates. In this study, we propose that agents can achieve self-evolution by synthesizing these atomic actions into reusable Standard Operating Procedures (SOPs), which function as callable higher-order tools that encapsulate multi-step logic. We further introduce EvoSOP, a framework that empowers agents to extract SOPs from execution trajectories and iteratively optimize the toolset through a systematic lifecycle of construction, merging, evaluation, and pruning. Extensive experiments demonstrate that EvoSOP significantly boosts task success rates while substantially reducing the number of interaction rounds compared to baselines. Our analysis also reveals that iterative tool optimization fosters reliable and efficient tool-use patterns, providing a scalable pathway for the development of self-evolving agents.

Chinese Translation

工具的使用使大语言模型（LLM）代理能够与现实世界互动并解决复杂任务。然而，现有的代理框架主要依赖由细粒度原子动作（例如，基本文件输入/输出或单轮搜索）组成的静态工具集，这迫使代理在每个重复工作流中重新发明低级逻辑，从而导致推理开销和失败率的增加。在本研究中，我们提出代理可以通过将这些原子动作合成可重用的标准操作程序（SOPs）来实现自我演化，这些程序作为可调用的高阶工具，封装多步骤逻辑。我们进一步介绍了EvoSOP，一个框架，赋予代理从执行轨迹中提取SOPs的能力，并通过构建、合并、评估和修剪的系统生命周期迭代优化工具集。大量实验表明，与基线相比，EvoSOP显著提高了任务成功率，同时大幅减少了交互轮次。我们的分析还揭示，迭代工具优化促进了可靠和高效的工具使用模式，为自我演化代理的发展提供了可扩展的路径。

View on arXiv Download PDF AI Translation

cs.AI / 16 / 2607.07379

Physics-Audited Agentic Discovery in Scientific Machine Learning

物理审核的自主发现：科学机器学习中的应用

Abueidda, Diab W., Ahmed, Bilal, Pantidis, Panos, Mobasher, Mostafa E.

Abstract

In agentic scientific machine learning (SciML), large language model (LLM) agents can discover surrogate models and select one by an automated score, typically an error metric. A low error, however, does not establish that the predicted fields satisfy the physics that matter for mechanics, such as boundary conditions, superposition, stiffness scaling, or causality. We introduce Physics-Audited Agentic SciML (PA-SciML), a verification-first workflow for agentic SciML discovery. The workflow fixes a scoring evaluator before search, derives reviewable machine-checkable physics requirements, checks each trained candidate on its outputs, and separately searches prescribed input ranges or measured load-history spans for high-violation cases without reference solution fields. A surrogate is reported as verified only under the stated checks. When enabled, the workflow also adds advisory numerical probes before training and tests one modeling change at a time to record which isolated edits are associated with score gains before reuse. In the reported computational-solid-mechanics numerical examples, the static elasticity run selects a surrogate with lower validation error than the error-only baseline while both selected models pass the common linear-elastic checks. In the transient elastodynamics run, an error-only baseline with similar mean error fails a stricter causality check by responding to future parts of the loading history, while the selected surrogate passes the stated checks. The main distinction is per-candidate physics evidence on predicted fields, not a richer aggregate score.

Chinese Translation

在自主科学机器学习（SciML）中，大型语言模型（LLM）代理可以发现替代模型并通过自动评分选择一个，通常是误差指标。然而，低误差并不能证明预测场满足对力学重要的物理条件，例如边界条件、叠加性、刚度缩放或因果关系。我们引入了物理审核的自主SciML（PA-SciML），这是一种以验证为先的自主SciML发现工作流程。该工作流程在搜索之前固定评分评估器，推导出可审查的机器可检查物理要求，检查每个训练候选模型的输出，并单独搜索规定的输入范围或测量的载荷历史跨度，以寻找高违规案例，而不参考解决方案场。只有在满足上述检查的情况下，替代模型才会被报告为经过验证。当启用时，该工作流程还会在训练之前添加建议性的数值探测，并一次测试一个建模更改，以记录哪些孤立的编辑与评分提升相关，然后再进行重用。在报告的计算固体力学数值示例中，静态弹性运行选择了一个验证误差低于仅误差基线的替代模型，而两个选择的模型都通过了常见的线性弹性检查。在瞬态弹性动力学运行中，具有相似平均误差的仅误差基线未能通过更严格的因果关系检查，因为它对加载历史的未来部分做出了响应，而所选的替代模型则通过了上述检查。主要区别在于每个候选模型在预测场上的物理证据，而不是更丰富的综合评分。

View on arXiv Download PDF AI Translation

cs.AI / 17 / 2607.07391

MIRA-Math: A Benchmark for Minimal Information Requesting and Mathematical Reasoning

MIRA-Math：最小信息请求和数学推理的基准测试

Bateh, Charbel Al, Saab Jr, Samer

Abstract

Mathematical reasoning benchmarks typically provide all facts needed to solve each problem, while interactive benchmarks often mix reasoning with tools, retrieval, and long-horizon dialogue. We introduce MIRA-Math, a benchmark for a narrower diagnostic capability: solving mathematical problems whose full latent state has a unique answer, but whose solver-facing view is missing exactly one necessary atomic fact. The solver must request the missing information in natural language under a strict budget and then integrate the returned fact into an exact final answer. A fixed constrained LLM responder sees only the dataset-provided atomic fact and must either offer the quoted fact when the request matches it, or decline otherwise. Thus, instance generation, typed hint specifications, validation, and final-answer verification are deterministic, while request metrics are measured under a fixed LLM-mediated responder channel. MIRA-Math contains 2{,}310 generated instances from 22 typed mathematical families spanning algebra, probability, linear systems, discrete structures, signal processing, Markov chains, circuits, interpolation, and numerical boundary-value problems. Experiments across frontier and small models show that request success and final-answer accuracy are separable: models may ask for the right fact yet fail the downstream computation, or fail before obtaining the canonical hint. We release generators, verifiers, prompts, run metadata, and dataset documentation to support reproducible evaluation of minimal information requesting in mathematical reasoning.

Chinese Translation

数学推理基准通常提供解决每个问题所需的所有事实，而交互式基准则常常将推理与工具、检索和长时间对话混合在一起。我们引入了 MIRA-Math，这是一个针对更狭窄的诊断能力的基准：解决数学问题，其完整的潜在状态具有唯一答案，但其面向求解者的视图缺少恰好一个必要的原子事实。求解者必须在严格的预算下以自然语言请求缺失的信息，然后将返回的事实整合到确切的最终答案中。一个固定约束的 LLM（大型语言模型）响应者仅能看到数据集中提供的原子事实，并且必须在请求匹配时提供所引用的事实，否则拒绝提供。因此，实例生成、类型提示规范、验证和最终答案验证都是确定性的，而请求指标是在固定的 LLM 中介响应通道下测量的。MIRA-Math 包含来自 22 个类型数学家族的 2,310 个生成实例，涵盖代数、概率、线性系统、离散结构、信号处理、马尔可夫链、电路、插值和数值边界值问题。对前沿和小型模型的实验表明，请求成功和最终答案准确性是可分离的：模型可能请求正确的事实但在下游计算中失败，或在获得规范提示之前失败。我们发布了生成器、验证器、提示、运行元数据和数据集文档，以支持数学推理中最小信息请求的可重复评估。

View on arXiv Download PDF AI Translation

cs.AI / 18 / 2607.07397

Agentic Data Environments

自主数据环境

Ang, Elaine, Huang, Chenxi, Liargkovas, Georgios, Liu, Jerry, Liu, Jinhui, Pagonas, Nikos, Summers, Charlie, Wang, Haonan, Xu, Jiakai, Zhou, Tianle, Zhang, Yusen, Yu, Zhou, Zhang, Zhuo, Peng, Tianyi, Kaffes, Kostis, Wu, Eugene

Abstract

Autonomous agents promise substantial gains in speed, scale, and labor efficiency, but their failures can impose abrupt and often irreversible costs. The central challenge for agentic automation is therefore to increase the benefits of automation while bounding the consequences of failure. While databases remain central to modern computing, agents operate over a broader data environment spanning files, APIs, applications, and system state. In this talk, I will outline early work on Agentic Data Environments -- the execution substrate in which agents operate -- that both amplify agent capabilities and enforce safety guarantees. This perspective reframes data systems from passive stores of state into active substrates for safe, reliable execution.

Chinese Translation

自主代理承诺在速度、规模和劳动效率上带来显著提升，但它们的失败可能会造成突发且往往不可逆转的成本。因此，自主自动化的核心挑战在于在增加自动化收益的同时限制失败的后果。尽管数据库在现代计算中仍然占据中心地位，代理却在更广泛的数据环境中运作，这些环境涵盖文件、API、应用程序和系统状态。在本次演讲中，我将概述关于自主数据环境（Agentic Data Environments）的早期研究——这是代理运作的执行基础，既增强了代理的能力，又强化了安全保障。这一视角将数据系统从被动的状态存储重新构建为安全、可靠执行的主动基础。

View on arXiv Download PDF AI Translation

cs.AI / 19 / 2607.07405

Reason Less, Verify More: Deterministic Gates Recover a Silent Policy-Violation Failure Mode in Tool-Using LLM Agents

减少推理，增加验证：确定性门恢复工具使用LLM代理中的静默政策违规失败模式

Reddy, Vikas, Challaram, Sumanth Reddy, Basu, Abhishek

Abstract

Tool-using LLM agents can violate the very policies they are deployed to enforce while appearing to complete the task successfully. In policy-permissive environments, a tool may execute any well-formed call even when the corresponding state transition is forbidden by domain policy. The result is a silent wrong state (a booking cancelled, a passenger count changed, a claim acted on without verification) that neither the tool nor the agent's self-report exposes. We study this failure mode in the $\tau^2$-bench airline domain. On a budget agent, 78% of observed failures are silent wrong-state failures with no tool error, and the aggregate failure rate is reproducible across disjoint seeds, not sampling noise. We then evaluate a lightweight intervention: deterministic, read-only pre-execution gates that inspect the proposed call and current state before allowing a write. A four-gate suite raises full-benchmark success from 29.6% to 42.0% on gpt-4o-mini (+12.4pp; paired task-level bootstrap P=0.0012), and the lift reproduces on a disjoint 15-seed set (+12.3pp; P=0.0008). The effect is concentrated where the gates fire: on the 26/50 firing tasks, success rises by +19.2pp, while movement on the 24 non-firing tasks does not exclude zero. Two negative controls (a self-enforcing retail domain and BFCL) bound the mechanism: gates help when tools are policy-permissive and add little where tools already self-enforce. As suggestive evidence, not a central claim, the same failure mode persists at the frontier: gpt-5.2 at default reasoning still attempts policy-violating writes, and the same suite improves success from 61.2% to 71.6% (+10.4pp; P=0.020; n=5, no replication). The contribution is a bounded evaluation and reliability result: deterministic gates do not guarantee task success, but they can deterministically prevent a known class of silent policy-violating writes at the action boundary.

Chinese Translation

使用工具的LLM代理在看似成功完成任务的同时，可能会违反其被部署以执行的政策。在政策宽松的环境中，即使相应的状态转换被领域政策禁止，工具也可能执行任何格式正确的调用。结果是一个静默的错误状态（如预订被取消、乘客数量变化、未经验证的索赔处理），既没有工具错误，也没有代理的自我报告暴露出这一点。我们在$ au^2$-bench航空领域研究这种失败模式。在预算代理中，观察到的失败中有78%是静默错误状态失败，没有工具错误，且整体失败率在不同种子之间可重复，而不是采样噪声。然后，我们评估了一种轻量级干预：确定性的只读预执行门，在允许写入之前检查提议的调用和当前状态。一个四门套件将gpt-4o-mini的全基准成功率从29.6%提高到42.0%（+12.4个百分点；配对任务级引导P=0.0012），这种提升在一个不重叠的15种子集上得以重现（+12.3个百分点；P=0.0008）。这一效果集中在门触发的地方：在26/50个触发任务中，成功率提高了+19.2个百分点，而在24个未触发任务上的变化不排除零。两个负控制（一个自我执行的零售领域和BFCL）限制了该机制：当工具政策宽松时，门的帮助显著，而在工具已经自我执行的情况下则贡献不大。作为暗示性证据，而非中心主张，同样的失败模式在前沿依然存在：gpt-5.2在默认推理下仍尝试违反政策的写入，而同样的套件将成功率从61.2%提高到71.6%（+10.4个百分点；P=0.020；n=5，无重现）。本研究的贡献是一个有限的评估和可靠性结果：确定性门并不保证任务成功，但它们可以在行动边界上确定性地防止已知类别的静默政策违规写入。

View on arXiv Download PDF AI Translation

cs.AI / 20 / 2607.07422

InductWave: Inductive Multi-Hop Logical Query Answering on Knowledge Graphs

InductWave：基于归纳的多跳逻辑查询在知识图谱上的回答

Kharbanda, Mayank, Cochez, Michael, Shah, Rajiv Ratn, Mutharaju, Raghava

Abstract

Logical Multi-Hop Query Answering over Knowledge Graphs (KGs) can be formulated as querying, with an implicit completeness assumption. Current works mainly focus on Existential First Order Logic (EFO) queries. These EFO queries contain conjunction, disjunction, and negation operators. Most existing works employ transductive reasoning, meaning they are not capable of reasoning over entities unseen during training. In the real world, there is a resource scarcity, and we cannot train a model with all the nodes of a large KG. Hence, we propose InductWave, a wavelet-based inductive embedding method for logical query answering on large KGs. Here, the training graph consists of fewer nodes than the test graph. Our model performs on par with the baseline models while having half the number of message-passing layers. It outperforms all of them in most cases, with 75% of the layers. These fewer resource requirements enable us to evaluate InductWave on massive graphs, such as Wiki-KG. We test our model using extensive experiments across varying train-test graph proportions of the FB15k-(237) dataset, comparing it with the state-of-the-art models. The code and datasets for the model are available at https://github.com/kracr/inductwave/.

Chinese Translation

在知识图谱（KGs）上进行逻辑多跳查询回答可以被表述为查询，隐含着完整性假设。目前的研究主要集中在存在性一阶逻辑（EFO）查询上。这些EFO查询包含合取、析取和否定运算符。大多数现有工作采用传导推理，这意味着它们无法对训练过程中未见过的实体进行推理。在现实世界中，资源稀缺，我们无法用大型知识图谱的所有节点来训练模型。因此，我们提出了InductWave，这是一种基于小波的归纳嵌入方法，用于在大型知识图谱上进行逻辑查询回答。在这里，训练图的节点数量少于测试图的节点数量。我们的模型在性能上与基线模型相当，同时消息传递层的数量仅为其一半。在大多数情况下，它在75%的层数下超越了所有基线模型。这种较少的资源需求使我们能够在大规模图上评估InductWave，例如Wiki-KG。我们通过对FB15k-(237)数据集的不同训练-测试图比例进行广泛实验来测试我们的模型，并与最先进的模型进行比较。模型的代码和数据集可在https://github.com/kracr/inductwave/获取。

View on arXiv Download PDF AI Translation

cs.AI / 21 / 2607.07436

The Blind Curator: How a Biased Judge Silently Disables Skill Retirement in Self-Evolving Agents

盲目的策展人：偏见法官如何默默地禁用自我进化代理的技能退休

Zhang, Xing, Cui, Yanwei, Wang, Guanghui, Li, Ziyuan, Qiu, Wei, Zhu, Bing, He, Peiyang

Abstract

A self-evolving agent retires its bad skills by watching them fail, so what happens when the judge cannot see the failures? Skill retirement is the structural constraint that keeps a growing library from drifting below the no-skill baseline, but its guarantee assumes an unbiased reward, which is false for the LLM judges that reference-free tasks force upon us. We show that a biased judge does not merely add noise; it \emph{silently switches off the curator}. We make this precise with a corrupted-reward analysis and, isolating the causal channel by injecting corruption on top of a deterministic reward, a behavioral study on a reference-free report-writing testbed with a code-generation cross-check. Symmetric noise leaves retirement intact, but \emph{false-pass} bias (failures slipping through as passes) disables contribution-based retirement past a sharp threshold that no amount of data can cross. Separating genuine retirement from cap-eviction churn shows this \emph{mechanism} failure is universal, holding across domains and failure rates and sparing only near-zero-false-pass, verifier-like graders. The downstream \emph{outcome}, though, is regime-dependent: eval quality degrades only where the same corruption also starves skill synthesis, and otherwise holds steady, so the disabled curator is \emph{silent}, surfacing in no aggregate metric. The contribution is a behavioral safety result, not a performance one. A cheap defect-injection audit then tells an operator, before deployment, which side of the threshold their judge occupies.

Chinese Translation

自我进化代理通过观察技能的失败来退休其不良技能，那么当法官无法看到这些失败时会发生什么？技能退休是一个结构性约束，它防止不断增长的技能库低于无技能基线，但其保证假设了一个无偏的奖励，这对于被无参考任务强加给我们的LLM法官来说是错误的。我们展示了偏见法官不仅仅是增加噪声；它 extit{默默地关闭了策展人}。我们通过腐败奖励分析使这一点变得精确，并通过在确定性奖励上注入腐败来隔离因果通道，在一个无参考的报告撰写测试平台上进行行为研究，并进行代码生成交叉检查。对称噪声保持退休不变，但 extit{虚假通过}偏见（失败被视为通过）在一个尖锐的阈值之上禁用基于贡献的退休，而这个阈值是任何数量的数据都无法跨越的。将真正的退休与能力驱逐的波动分开显示，这一 extit{机制}失效是普遍存在的，适用于不同领域和失败率，仅在近零虚假通过的验证者类评分者中幸免。尽管如此，下游 extit{结果}是依赖于体制的：评估质量仅在同一腐败也使技能合成匮乏的地方下降，而在其他地方保持稳定，因此被禁用的策展人是 extit{无声的}，在没有任何汇总指标中浮现。该贡献是一个行为安全结果，而不是性能结果。一个廉价的缺陷注入审计可以在部署之前告诉运营者，他们的法官处于阈值的哪一侧。

View on arXiv Download PDF AI Translation

cs.AI / 22 / 2607.07467

SpaCellAgent: A Self-Evolving LLM-Based Multi-Agent Framework for Trajectory Analysis

SpaCellAgent：一种基于自我进化的大型语言模型（LLM）的多智能体框架用于轨迹分析

Wang, Songhan, Chi, Haoang, Li, He, Zhang, Zhiheng, Yuan, Jiayan, Wang, Cheems, Peng, Hao, Liu, Xinwang, Yang, Wenjing

Abstract

Spatial and Single-cell transcriptomics are transformative in deciphering cellular dynamics. As the fundamental paradigm for reconstructing cell developmental paths, trajectory inference (TI) is critical. However, existing methods require extensive manual intervention and proficiency in heterogeneous tools, posing a significant barrier to efficient TI analysis. To bridge this gap, we propose SpaCellAgent, an autonomous large language model (LLM) multi-agent framework that automates end-to-end spatiotemporal analysis and narrative generation. SpaCellAgent utilizes a multi-agent architecture for strategic workflow planning, a dynamic tool-orchestration engine for adaptive algorithm selection, and a self-evolution module that iteratively refines performance through feedback. We evaluate SpaCellAgent on six heterogeneous datasets encompassing complex temporal developmental trajectories, diverse sequencing platforms, and spatially-resolved tissue architectures. SpaCellAgent consistently demonstrates over 40\% improvement in analytical efficiency while maintaining expert-aligned performance. By converting natural language specifications into optimized analytical workflows and fully automating the pipeline, SpaCellAgent democratizes advanced spatiotemporal modeling and establishes a scalable, agent-driven paradigm for computational biology. The code and materials are available at https://github.com/LittleXH-shw/SpaCellAgent.

Chinese Translation

空间和单细胞转录组学在解读细胞动态方面具有变革性意义。作为重建细胞发育路径的基本范式，轨迹推断（Trajectory Inference, TI）至关重要。然而，现有方法需要大量人工干预和对异构工具的熟练掌握，成为高效TI分析的重大障碍。为了解决这一问题，我们提出了SpaCellAgent，一个自主的大型语言模型（LLM）多智能体框架，能够自动化端到端的时空分析和叙述生成。SpaCellAgent利用多智能体架构进行战略工作流程规划，采用动态工具协调引擎进行自适应算法选择，并通过反馈迭代优化性能的自我进化模块。我们在六个异构数据集上评估了SpaCellAgent，这些数据集涵盖复杂的时间发育轨迹、多样的测序平台和空间分辨的组织结构。SpaCellAgent在分析效率上始终表现出超过40\%的提升，同时保持与专家一致的性能。通过将自然语言规范转化为优化的分析工作流程并完全自动化管道，SpaCellAgent使高级时空建模民主化，并建立了一个可扩展的、以智能体驱动的计算生物学范式。代码和材料可在 https://github.com/LittleXH-shw/SpaCellAgent 获取。

View on arXiv Download PDF AI Translation

cs.AI / 23 / 2607.07492

Search, Fail, Recover: A Training Framework for Correction-Aware Reasoning

搜索、失败、恢复：一种面向纠错推理的训练框架

Beresnev, Dmitry, Makharev, Vladimir, Khalikov, Roman, Oseledets, Ivan, Anokhin, Petr

Abstract

Many reasoning tasks are not well described by a single left-to-right chain: a solver may need to pursue a plausible branch, observe delayed failure, and return to the latest prefix that can still be completed. We introduce Pyligent, a training and inference framework inspired by the Diligent Learner formulation that represents reasoning as validated search over partial solution chains. A task validator labels generated continuations and failures, and the resulting search trees are converted into supervised targets for three actions: continue, finish, and backtrack, with optional traces that summarize abandoned branches. We evaluate Pyligent on a hidden directed graph task designed to isolate delayed-failure recovery, and on structured reasoning domains with exact validators, including $4{\times}4$ Sudoku, Sudoku with reasoning traces, and Blocksworld. Compared with gold-only supervised fine-tuning, Pyligent improves solve rate by $72.7$ percentage points on hidden graphs, by $17$ and $18$ points on mixed and expert Sudoku, by $27$ and $14$ points on mixed and expert Sudoku with reasoning traces, and by $13$ points on Blocksworld. These results suggest that explicit failed-branch supervision can teach useful recovery behavior beyond imitation of polished solution chains.

Chinese Translation

许多推理任务无法仅通过单一的从左到右的链条来很好地描述：求解器可能需要追寻一个合理的分支，观察延迟失败，并返回到最新的仍可完成的前缀。我们提出了 Pyligent，这是一种训练和推理框架，灵感来源于 Diligent Learner 的表述，将推理视为对部分解链的验证搜索。任务验证器对生成的延续和失败进行标记，生成的搜索树被转换为三个动作的监督目标：继续、完成和回溯，并可选地提供总结被放弃分支的追踪信息。我们在一个旨在隔离延迟失败恢复的隐藏有向图任务上评估了 Pyligent，并在具有精确验证器的结构化推理领域进行评估，包括 $4{ imes}4$ 数独、带推理追踪的数独以及 Blocksworld。与仅使用黄金标准的监督微调相比，Pyligent 在隐藏图上的解题率提高了 $72.7$ 个百分点，在混合和专家数独上分别提高了 $17$ 和 $18$ 个百分点，在带推理追踪的混合和专家数独上分别提高了 $27$ 和 $14$ 个百分点，在 Blocksworld 上提高了 $13$ 个百分点。这些结果表明，显式的失败分支监督可以教授有用的恢复行为，超越对精炼解链的模仿。

View on arXiv Download PDF AI Translation

cs.AI / 24 / 2607.07504

Do LLM-Generated Skills Make Better AI Data Scientists? A Component Ablation Across Data-Science Workflows

LLM生成的技能是否能培养更优秀的AI数据科学家？数据科学工作流程中的组件消融研究

Huang, Wei-Jung

Abstract

Product data scientists often ask LLM-based agents to help with recurring execution tasks such as cleaning data, writing SQL, choosing statistical tests, and formatting results. Reusable skill files are meant to avoid prompting from scratch by packaging guidance for a task family. Expert-written skills can encode high-quality guidance, but writing and maintaining them across many data-science task families creates a manual bottleneck. We ask whether LLM-generated skills offer a useful low-curation alternative: do they improve performance over the task prompt alone? We test this question across four lifecycle stages: data preparation, data extraction, statistical analysis, and reporting, using one generated skill per stage. We find no reliable improvement from full generated skills over No-Skill prompting. We then ask whether any part of the skill is useful by ablating different skill components. The main ablation covers 56 tasks, nine model configurations, and three providers, yielding 7,560 runs. Compared with prompting using the task alone, neither the full generated skill nor any ablated skill variant significantly improves performance; all p-values are at least 0.396, and the total spread across variants is only 1.2 pp. A supplemental token-matched control adds 1,512 runs and finds that Full skills perform similarly to task-irrelevant skill-formatted content. The results caution against using one LLM-generated skill per data-science workflow as a default single-shot prompting strategy.

Chinese Translation

产品数据科学家经常请求基于LLM的代理帮助执行重复性任务，例如清理数据、编写SQL、选择统计检验和格式化结果。可重用的技能文件旨在通过为一类任务打包指导，避免从头开始提示。专家编写的技能可以编码高质量的指导，但在多个数据科学任务类别中编写和维护这些技能会造成手动瓶颈。我们探讨LLM生成的技能是否提供了一种有用的低维护替代方案：它们是否能比单纯的任务提示提高性能？我们在四个生命周期阶段进行测试：数据准备、数据提取、统计分析和报告，每个阶段使用一个生成的技能。我们发现，与无技能提示相比，完整生成的技能并未显著提高性能。随后，我们通过消融不同的技能组件来探讨技能的任何部分是否有用。主要的消融研究涵盖了56个任务、九种模型配置和三家提供商，共进行7,560次实验。与仅使用任务提示相比，完整生成的技能或任何消融的技能变体均未显著提高性能；所有p值至少为0.396，变体之间的总差异仅为1.2个百分点。一个补充的匹配控制增加了1,512次实验，结果发现完整技能的表现与与任务无关的技能格式内容相似。结果提示在数据科学工作流程中使用单个LLM生成的技能作为默认的一次性提示策略时需谨慎。

View on arXiv Download PDF AI Translation

cs.AI / 25 / 2607.07646

RL Post-Training Builds Compositional Reasoning Strategies

RL 后训练构建组合推理策略

Abdulsalam, Azwar, Patel, Nishil, Saxe, Andrew

Abstract

Does RL post-training merely amplify primitive skills already latent in a base model, or can it compose primitive skills into new higher-level strategies? We study this question in a fully observable rewrite-grammar environment where the pretraining distribution is known and every generated rewrite can be audited. A Transformer is pretrained on primitive symbol-rewrite chains and post-trained on a Trace-based reasoning task with only a binary final-answer reward. RL solves held-out problems that remain rarely solved by the pretrained model even under much larger sampling budgets, while rejection fine-tuning improves early but plateaus. Trace analysis shows that RL reorganizes primitive competence through a phased compositional mechanism: it first strengthens primitive reductions, then discovers valid composed procedures. These include sequential compositions, which collapse ordered chains of primitive contractions, and parallel compositions, which combine independent primitive contractions in a single step. The composed procedures are not isolated samples; they are reused and consolidated into a stable repertoire. Comparing RL with rejection fine-tuning shows that the key difference is not exploration volume but selectivity: RFT produces many shortcut-like rewrites, much of them invalid, whereas RL concentrates exploration into valid reusable structure. Pretraining ablations show that the emergence of compositional strategies is gated not by primitive exposure alone, but by whether pretraining organizes primitive competence into reduction procedures that RL can later compress. The base model provides weak procedural ingredients; RL builds them into reliable higher-level strategies.

Chinese Translation

RL 后训练仅仅是放大基础模型中已经潜在的原始技能，还是能够将原始技能组合成新的更高层次的策略？我们在一个完全可观察的重写语法环境中研究这个问题，其中预训练分布是已知的，且每个生成的重写都可以进行审计。一个 Transformer 在原始符号重写链上进行预训练，并在一个基于追踪的推理任务上进行后训练，该任务仅提供二元最终答案奖励。RL 解决了在更大的采样预算下仍然很少被预训练模型解决的保留问题，而拒绝微调在早期有所改善但趋于平稳。追踪分析表明，RL 通过分阶段的组合机制重新组织原始能力：它首先强化原始简化，然后发现有效的组合程序。这些程序包括顺序组合，它们压缩了原始收缩的有序链，以及并行组合，它们在单一步骤中组合独立的原始收缩。这些组合程序不是孤立的样本；它们被重复使用并巩固为一个稳定的库。将 RL 与拒绝微调进行比较显示，关键区别不在于探索量，而在于选择性：RFT 产生了许多类似捷径的重写，其中许多是无效的，而 RL 将探索集中在有效的可重用结构上。预训练消融实验表明，组合策略的出现不仅仅依赖于原始暴露，而在于预训练是否将原始能力组织成 RL 后续可以压缩的简化程序。基础模型提供了弱的程序成分；RL 将它们构建为可靠的更高层次策略。

View on arXiv Download PDF AI Translation

cs.AI / 26 / 2607.07663

Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops

人工智能中的递归自我改进：从有限自我精炼到自主研究循环

Chen, Mingguang, Wang, Licheng, Qu, Bo

Abstract

AI systems increasingly participate in their own improvement: revising their outputs, adapting their own harnesses during deployment, training on data they generate, and, increasingly, conducting AI research itself. This literature is described under a vocabulary ("self-refine," "self-reward," "self-play," "self-evolve") that conflates fundamentally different ambitions. We survey 1,250 arXiv papers (2024-2026) along two axes: what the system improves -- its behavior in deployment, its policy through training, its evaluator, or the research process itself -- and the degree of loop closure (human-in-the-loop to fully closed). The taxonomy separates bounded self-refinement -- convergent, evaluable, and already industrial practice -- from open-ended recursive self-improvement (RSI), which remains bounded by grounding requirements, collapse dynamics, and compute constraints on every measured axis. Its distinctive feature is a dedicated category for self-evaluation: every improvement loop is a claim that some signal can substitute for human judgment. We survey the evaluator design space -- judges, process reward models, verifiers, rubrics, meta-evaluation -- order the signals into a verification hierarchy from formal verifiers (strongest) to intrinsic self-assessment (weakest), and observe that demonstrated self-improvement strength tracks this hierarchy, that its failure modes (self-confirming loops, model collapse, diversity collapse) follow from its violations, and that the "research direction-setting" bottleneck keeping humans in the loop sits at the top of that hierarchy. We connect the technical literature to the theory of RSI limits and to the safety and governance questions raised by frontier-lab accounts of closing the loop, and identify governance-grade measurement of self-improvement as the field's most underpopulated niche.

Chinese Translation

人工智能系统越来越多地参与到自身的改进中：修正其输出、在部署过程中调整自身的约束、对其生成的数据进行训练，以及越来越多地进行人工智能研究。相关文献使用了一些术语（如“自我精炼”、“自我奖励”、“自我博弈”、“自我进化”），这些术语混淆了根本不同的目标。我们对2024-2026年间的1250篇arXiv论文进行了调查，沿着两个维度进行分类：系统改进的内容——其在部署中的行为、通过训练改进的策略、评估者或研究过程本身——以及循环闭合的程度（从人类参与到完全闭合）。该分类法将有限自我精炼（收敛的、可评估的、已成为工业实践）与开放式递归自我改进（Recursive Self-Improvement, RSI）区分开来，后者仍受到基础要求、崩溃动态和每个测量维度上的计算约束的限制。其独特之处在于设立了一个专门的自我评估类别：每个改进循环都声称某种信号可以替代人类判断。我们调查了评估者设计空间——评审者、过程奖励模型、验证者、评分标准、元评估——将信号按验证层级排序，从正式验证者（最强）到内在自我评估（最弱），并观察到证明的自我改进强度与此层级相符，其失败模式（自我确认循环、模型崩溃、多样性崩溃）源于对这一层级的违反，并且“研究方向设定”这一瓶颈使人类仍需参与，位于该层级的顶端。我们将技术文献与RSI限制理论以及前沿实验室在闭环问题上提出的安全和治理问题联系起来，并确定自我改进的治理级别测量是该领域最为稀缺的细分领域。

View on arXiv Download PDF AI Translation

cs.AI / 27 / 2607.07676

SkillCenter: A Large-Scale Source-Grounded Skill Library for Autonomous AI Agents

SkillCenter：一个大规模的源基础技能库，用于自主人工智能代理

Sha, Tianming, Zhao, Yue, Sun, Lichao, Dong, Yushun

Abstract

Autonomous AI agents can execute complex tasks with limited human review, yet they often lack the grounded operational knowledge to make their outputs not just executable but correct, secure, and maintainable. We introduce SkillCenter, to our knowledge the largest open skill library for agents by total count: 216,938 structured skills across 24 domain bundles. A SkillGate-filtered pipeline contributes 114,565 source-grounded skills from peer-reviewed journals, ArXiv, and over 24,000 technical sources, integrated with 102,373 community skills from GitHub and the ClawHub marketplace. We present the end-to-end framework that builds the pipeline subset: multi-source acquisition, an LLM-based quality gate (SkillGate), template-driven generation, iterative source-grounding, and quality-controlled publishing. Source grounding is a traceability guarantee: each retained claim maps to an exact quotation in its source. All skills ship as offline-searchable SQLite FTS5 bundles.

Chinese Translation

自主人工智能代理能够在有限的人类审查下执行复杂任务，但它们往往缺乏扎实的操作知识，使得其输出不仅可执行，而且正确、安全和可维护。我们介绍了SkillCenter，据我们所知，这是迄今为止最大的开放技能库，共计216,938个结构化技能，涵盖24个领域包。通过SkillGate过滤的管道贡献了114,565个源基础技能，这些技能来自经过同行评审的期刊、ArXiv以及超过24,000个技术来源，并与来自GitHub和ClawHub市场的102,373个社区技能集成。我们展示了构建管道子集的端到端框架：多源获取、基于大型语言模型的质量门（SkillGate）、模板驱动生成、迭代源基础和质量控制发布。源基础是可追溯性的保证：每个保留的声明都映射到其来源中的确切引用。所有技能以离线可搜索的SQLite FTS5包形式发布。

View on arXiv Download PDF AI Translation

cs.AI / 28 / 2607.07695

Institutional Red-Teaming: Deployment Rules, Not Just Models, Causally Shape Multi-Agent AI Safety

制度红队：部署规则不仅仅是模型，因果地塑造多智能体人工智能安全性

Chen, Yujiao

Abstract

We introduce institutional red-teaming, an evaluation methodology for testing deployment rules in multi-agent AI: hold the agents, objectives, and task state fixed, vary only one rule, and attribute the resulting change in collective behavior to that rule. We instantiate the methodology in IABench-CA, a consequence-allocation benchmark spanning 228 contexts, five canonical rules, and seven model populations (33,924 games), with a normative cooperative reference and auto-labelled reasoning traces. Three findings emerge. (1) Deployment rules causally alter collective safety: changing only the consequence rule moves mean fatality by 22 to 58 percentage points within every population. (2) There is no safe default, but the targeting hazard is universal: the safest rule, the least-safe rule, and even the direction of the incidence effect vary across populations, yet regressive identity-targeting is never decisively safest in any context for any population, eliminates the least-resourced agent in 30-87% of games everywhere, and is selection-unsafe relative to the cooperative reference for all seven populations. (3) Identity salience is the mechanism: a one-shot anonymization ablation on the most exploitation-prone population (gpt-5.1) shows that merely naming the loss bearer in the rule text drives targeted elimination from 22% to 81% at identical payoffs; under repeated play, anonymization only delays the targeting, as agents re-infer the hidden rule from observed eliminations. We package the methodology as a safety-case workflow that certifies a provisional rule region $\Phi(c,P)$ per deployment context and population, with explicit residual risks and monitoring obligations.

Chinese Translation

我们引入了制度红队（institutional red-teaming），这是一种用于测试多智能体人工智能中部署规则的评估方法：固定智能体、目标和任务状态，仅改变一条规则，并将集体行为的变化归因于该规则。我们在 IABench-CA 中实例化该方法，该基准涵盖228个情境、五条经典规则和七个模型群体（33,924场游戏），并提供了规范性的合作参考和自动标记的推理轨迹。研究得出了三项发现。(1) 部署规则因果地改变集体安全性：仅改变后果规则会使每个群体的平均致死率变化22到58个百分点。(2) 没有安全的默认选项，但目标风险是普遍存在的：最安全的规则、最不安全的规则，甚至影响效应的方向在不同群体中各不相同，然而回归的身份目标在任何情境下对任何群体都不是决定性的安全选择，在30%到87%的游戏中消灭资源最少的智能体，并且相对于所有七个群体的合作参考来说，选择上是不安全的。(3) 身份显著性是机制：对最易被利用的群体（gpt-5.1）进行一次性匿名化消融实验表明，仅仅在规则文本中命名损失承担者就能将目标消除率从22%提高到81%，在相同的收益下；在重复游戏中，匿名化仅仅延迟了目标选择，因为智能体会根据观察到的消除情况重新推断隐藏规则。我们将该方法打包为一种安全案例工作流，为每个部署情境和群体认证一个临时规则区域 $ 7(c,P)$，并明确残余风险和监控义务。

View on arXiv Download PDF AI Translation

计算语言学 (Computation and Language)

cs.CL / 1 / 2607.06611

Audio Sentiment Analysis via Distillation and Cross-Modal Integration of Generated Multilingual Transcripts

通过蒸馏和跨模态集成生成的多语言转录文本进行音频情感分析

Durdun, Andrei-George, Constantinescu, Victor, Ionescu, Radu Tudor

Abstract

Automatically recognizing the sentiment, positive or negative, from speech is a challenging task, requiring both the analysis of vocal inflections and the interpretation of uttered words. Recent solutions rely on audio foundation models to solve the task, but it remains unclear if such models can take all aspects into account. To this end, we propose a multimodal solution that integrates audio and text information via cross-modal transformers, where text transcripts are automatically generated via an automatic speech recognition (ASR) tool. Moreover, we create multiple text modalities by automatically translating the transcripts into multiple languages via machine translation tools. Audio and multilingual text features are combined via a cascaded architecture comprising cross-modal transformer blocks that integrate modalities one by one. We further distill knowledge from the multimodal model, called teacher, into a unimodal (audio only) model, called student. We conduct experiments on a large-scale dataset, demonstrating that the automatically generated textual information can bring significant performance boosts in multimodal sentiment polarity classification. Our ablation study confirms that both automatic transcripts and automatic translations are helpful. Moreover, we show that the audio-only model can be enhanced via distillation, boosting performance without any computational overhead during inference. To reproduce the reported results, we publicly release our code at https://github.com/andreidurdun/cross-modal-audio-sentiment.

Chinese Translation

自动识别语音中的情感（正面或负面）是一项具有挑战性的任务，既需要分析声调的变化，又需要解读所表达的词语。近期的解决方案依赖于音频基础模型来完成这一任务，但尚不清楚这些模型是否能够考虑所有方面。为此，我们提出了一种多模态解决方案，通过跨模态变换器集成音频和文本信息，其中文本转录是通过自动语音识别（ASR）工具自动生成的。此外，我们通过机器翻译工具将转录文本自动翻译成多种语言，从而创建多个文本模态。音频和多语言文本特征通过一个级联架构相结合，该架构包含逐一集成模态的跨模态变换器模块。我们进一步将多模态模型（称为教师）中的知识蒸馏到单模态（仅音频）模型（称为学生）中。我们在一个大规模数据集上进行了实验，证明自动生成的文本信息可以显著提升多模态情感极性分类的性能。我们的消融研究确认了自动转录和自动翻译都是有帮助的。此外，我们还展示了通过蒸馏可以增强仅音频模型的性能，在推理过程中没有任何计算开销。为了重现报告的结果，我们在 https://github.com/andreidurdun/cross-modal-audio-sentiment 上公开发布了我们的代码。

View on arXiv Download PDF AI Translation

cs.CL / 2 / 2607.06641

Healthier LLMs: Retrieval-Augmented Generation for Public Health Question Answering

更健康的LLMs：用于公共卫生问答的检索增强生成

Feldman, Felix, Harris, Joshua, Laurence, Timothy, Loman, Leo, Higgins, Ollie, Grayson, Fan, Soma, Poonam, Pace-Bonello, Bethany, Borowitz, Michael, Nonnenmacher, Toby

Abstract

Large language models (LLMs) achieve promising results on medical question answering benchmarks, yet their use in public health is constrained by hallucinations and the rapid evolution of official guidance. Retrieval-Augmented Generation (RAG) mitigates these risks by grounding responses in an explicitly maintained corpus, but end-to-end performance depends critically on retrieval configuration and on evaluation beyond multiple-choice formats. We extend PubHealthBench, a question answering (QA) benchmark of 7,929 questions derived from UK Government public health guidance, into a retrieval-augmented setting and systematically evaluate retrieval and generation choices. We compare dense, sparse, and hybrid retrieval across multiple embedding models and corpus variants, and show that hybrid retrieval consistently improves recall and ranking quality, with chunk length and topic interacting with ranking performance. Providing retrieved context substantially increases multiple-choice accuracy across a diverse set of LLMs, enabling smaller open-weight models to match or outperform larger models used without retrieval, with gains primarily driven by retrieval quality and careful context selection. To assess realistic free-form answering, we introduce a rubric-based LLM-as-a-judge covering faithfulness, completeness, clarity, and factual consistency, and validate it against dual human annotations. Judge-human agreement is strongest for faithfulness and completeness, while factual consistency and clarity are less reliably reproduced, motivating caution when interpreting those dimensions at scale. Overall, our results highlight retrieval as a primary lever for reliable public health QA and provide practical guidance for building and evaluating RAG systems grounded in official guidance.

Chinese Translation

大型语言模型（LLMs）在医学问答基准测试中取得了令人鼓舞的结果，但其在公共卫生领域的应用受到幻觉和官方指导快速演变的限制。检索增强生成（Retrieval-Augmented Generation, RAG）通过将响应基于明确维护的语料库来减轻这些风险，但端到端性能在很大程度上依赖于检索配置以及超越多项选择格式的评估。我们将PubHealthBench，一个由来自英国政府公共卫生指导的7,929个问题构成的问答（QA）基准，扩展到检索增强的设置，并系统地评估检索和生成选择。我们比较了多种嵌入模型和语料库变体下的密集检索、稀疏检索和混合检索，并显示混合检索在召回率和排名质量上始终有所改善，且块长度和主题与排名表现相互作用。提供检索上下文显著提高了多项选择的准确性，适用于多种LLMs，使得较小的开放权重模型能够与未使用检索的大型模型相匹配或超越，主要受益于检索质量和仔细的上下文选择。为了评估现实的自由形式回答，我们引入了一个基于评分标准的LLM作为评判者，涵盖了忠实性、完整性、清晰性和事实一致性，并与双重人工标注进行验证。评判者与人类的协议在忠实性和完整性方面最强，而事实一致性和清晰性则较难可靠再现，这促使我们在大规模解读这些维度时保持谨慎。总体而言，我们的结果强调了检索作为可靠公共卫生问答的主要杠杆，并为构建和评估基于官方指导的RAG系统提供了实用指导。

View on arXiv Download PDF AI Translation

cs.CL / 3 / 2607.06818

Ad Headline Generation using Self-Critical Masked Language Model

使用自我批评的掩码语言模型生成广告标题

Kanungo, Yashal Shakti, Negi, Sumit, Rajan, Aruna

Abstract

For any E-commerce website it is a nontrivial problem to build enduring advertisements that attract shoppers. It is hard to pass the creative quality bar of the website, especially at a large scale. We thus propose a programmatic solution to generate product advertising headlines using retail content. We propose a state of the art application of Reinforcement Learning (RL) Policy gradient methods on Transformer based Masked Language Models. Our method creates the advertising headline by jointly conditioning on multiple products that a seller wishes to advertise. We demonstrate that our method outperforms existing Transformer and LSTM + RL methods in overlap metrics and quality audits. We also show that our model-generated headlines outperform human submitted headlines in terms of both grammar and creative quality as determined by audits.

Chinese Translation

对于任何电子商务网站来说，构建持久的、能够吸引购物者的广告是一项非平凡的任务。尤其是在大规模的情况下，达到网站创意质量的标准是非常困难的。因此，我们提出了一种程序化的解决方案，通过零售内容生成产品广告标题。我们在基于Transformer的掩码语言模型上应用了最先进的强化学习（Reinforcement Learning, RL）策略梯度方法。我们的方法通过联合考虑多个卖家希望广告的产品来创建广告标题。我们证明了我们的方法在重叠指标和质量审计方面优于现有的Transformer和LSTM + RL方法。我们还展示了我们模型生成的标题在语法和创意质量方面均优于人类提交的标题，这一点通过审计得以确认。

View on arXiv Download PDF AI Translation

cs.CL / 4 / 2607.06831

Gradient-Based Speech-to-Text Alignment for Any ASR Model: From CTC to Speech LLMs

基于梯度的任意语音识别模型的语音到文本对齐：从CTC到语音大型语言模型

Zeyer, Albert, Schlüter, Ralf, Ney, Hermann

Abstract

Speech-to-text alignment means finding the temporal boundaries of each word in the audio. Some models provide such an alignment directly and others do not. Connectionist temporal classification (CTC) and transducer models have an alignment by construction, whereas attention-based encoder-decoders (AED) and speech large language models (LLMs) do not, and their word timings are usually read off the attention weights instead. All of these signals live on the encoder frame grid, which bounds their temporal precision. We study a generic gradient-based alignment that applies to any differentiable ASR model. We take the gradient of each teacher-forced token log probability with respect to the input, reduce it to a per-frame saliency, and decode the resulting matrix into word boundaries with a single dynamic-programming pass. The method needs no training, no model modification and no alignment heads, works across all model families including the speech LLMs, and aligns on the input grid rather than on the coarser encoder grid. We evaluate it on sixteen models from four families, on read (TIMIT) and spontaneous (Buckeye) speech, each against the model's own native or attention-based alignment. We find that the gradient yields a usable alignment for every model, that it is usually somewhat behind a strong native aligner but better where the native alignment is weak, as for the streaming models, and that its main disadvantage is the cost of one backward pass per token.

Chinese Translation

语音到文本对齐是指在音频中找到每个单词的时间边界。一些模型直接提供这种对齐，而另一些则不然。连接主义时间分类（CTC）和转换器模型在构建时就具有对齐，而基于注意力的编码器-解码器（AED）和语音大型语言模型（LLMs）则没有，它们的单词时序通常是从注意力权重中读取的。所有这些信号都存在于编码器帧网格上，这限制了它们的时间精度。我们研究了一种通用的基于梯度的对齐方法，适用于任何可微分的语音识别（ASR）模型。我们对每个教师强制的标记对数概率相对于输入的梯度进行计算，将其简化为每帧的显著性，并通过一次动态规划过程解码出单词边界。该方法无需训练、无需模型修改和对齐头，适用于所有模型家族，包括语音LLMs，并在输入网格上进行对齐，而不是在较粗的编码器网格上。我们在来自四个家族的十六个模型上进行了评估，涉及阅读（TIMIT）和自发（Buckeye）语音，每个模型都与其自身的本地或基于注意力的对齐进行比较。我们发现，梯度为每个模型提供了可用的对齐，通常在强大的本地对齐之后，但在本地对齐较弱的情况下（如流式模型）表现更好，其主要缺点是每个标记需要一次反向传播的成本。

View on arXiv Download PDF AI Translation

cs.CL / 5 / 2607.06845

LLMs Silently Correct African American English: Auditing and Mitigating Dialect Bias via Activation Steering

大型语言模型默默纠正非洲裔美国英语：通过激活引导审计和减轻方言偏见

Wu, Huan, Emami, Ali, Hassan, Muhammad Furquan, Igbinoba, Osaretin, Idusuyi, Osakpolor, Igbinoba, Osamede, Khattak, Faiza Khan, Seyyed-Kalantari, Laleh

Abstract

African American English (AAE), a rule-governed dialect spoken by over 30 million people, is routinely misinterpreted and "corrected" by large language models (LLMs). Across six instruction-tuned LLMs (14B to 70B), we show that state-of-the-art models systematically prefer Standard American English (SAE) continuations even when the preceding context is in AAE, effectively rewriting AAE into SAE. We present an end-to-end framework to audit and mitigate this bias. For auditing, we introduce conditional Dialect Group Invariance (cDGI), which isolates true model bias from translator-induced artifacts, and a feature-level localization analysis that identifies which AAE markers most strongly trigger bias; we find that syntactic constructions, especially negative concord (e.g., "ain't nobody"), are universal triggers across all models. For mitigation, we introduce, to our knowledge, the first application of activation steering to dialect bias: a training-free, test-time method that extracts dialect directions via causal tracing and injects them into bias-relevant layers. Activation steering reduces bias 5 to 20 times more than prompting while preserving SAE fluency. To enable this work, we release REAL-AAE , the largest real-AAE parallel corpus to date: 17,479 AAE/SAE/ AAE_back triplets from natural tweets (2 to 6 times larger than prior real-AAE resources), validated automatically (BERTScore F1 = 0.95) and by three native AAE speakers (83.0% semantic agreement).

Chinese Translation

非洲裔美国英语（AAE）是一种由超过3000万人使用的规则驱动方言，常常被大型语言模型（LLMs）误解和“纠正”。在六个经过指令调优的LLM（参数规模从140亿到700亿）中，我们展示了最先进的模型系统性地偏好标准美国英语（SAE）的延续，即使前文上下文为AAE，实际上将AAE重写为SAE。我们提出了一个端到端的框架来审计和减轻这种偏见。在审计方面，我们引入了条件方言组不变性（cDGI），该方法将真实模型偏见与翻译者引入的伪影隔离开来，并进行特征级定位分析，以识别哪些AAE标记最强烈地触发偏见；我们发现句法结构，特别是负一致（例如，“ain't nobody”），是所有模型的普遍触发因素。在减轻偏见方面，我们首次将激活引导应用于方言偏见：这是一种无训练、测试时的方法，通过因果追踪提取方言方向并将其注入与偏见相关的层。激活引导在保持SAE流畅性的同时，将偏见减少了5到20倍，远超提示方法。为了支持这项工作，我们发布了REAL-AAE，这是迄今为止最大的真实AAE平行语料库：来自自然推文的17,479个AAE/SAE/AAE_back三元组（比之前的真实AAE资源大2到6倍），经过自动验证（BERTScore F1 = 0.95）和三位AAE母语者的验证（83.0%的语义一致性）。

View on arXiv Download PDF AI Translation

cs.CL / 6 / 2607.06940

Comprehensive Evaluation of Large Language Model Responses: A Multi-Factor Scoring System

大型语言模型响应的综合评估：多因素评分系统

Gai, Yiming, Lu, Junde, Huang, Xuefei

Abstract

The remarkable performance of large language models (LLMs) in linguistic tasks underscores an urgent need for comprehensive evaluation of their response quality. Prevailing methods, often confined to singular dimensions, fall short of capturing the full spectrum of model capabilities. This study introduces a multifactor scoring paradigm, integrating accuracy, conciseness, factual consistency, readability, and coherence, complemented by a graphical user interface (GUI) for visualizing outcomes. Evaluations on the TruthfulQA dataset unveil mainstream LLMs' strengths in reasoning tasks (peaking at a composite score of 0.6104) alongside pervasive limitations in navigating complex facts and ambiguities. Transcending the narrow lens of traditional metrics, this framework offers a transparent, adaptable avenue to illuminate model potential and deficiencies. Though presently focused on English tasks, its horizons beckon toward multilingual domains. This work carves a novel path for knowledge engineering and model refinement.

Chinese Translation

大型语言模型（LLMs）在语言任务中的卓越表现凸显了对其响应质量进行全面评估的迫切需求。现有方法往往局限于单一维度，无法全面捕捉模型能力的全貌。本研究提出了一种多因素评分范式，整合了准确性、简洁性、事实一致性、可读性和连贯性，并配备了可视化结果的图形用户界面（GUI）。对TruthfulQA数据集的评估揭示了主流LLMs在推理任务中的优势（综合得分高达0.6104），同时也暴露了它们在处理复杂事实和模糊性方面的普遍局限性。超越传统指标的狭隘视角，该框架提供了一条透明且可适应的途径，以揭示模型的潜力和不足。尽管目前集中于英语任务，但其前景向多语言领域拓展。本研究为知识工程和模型优化开辟了一条新路径。

View on arXiv Download PDF AI Translation

cs.CL / 7 / 2607.06974

MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning

MILES：具有可学习选择的模块化指令记忆用于自我改进的LLM推理

Tong, Ruilin, Gong, Dong

Abstract

Large language models (LLMs) increasingly improve their reasoning at test time via additional computation, yet most existing works treat each problem in isolation. When problems arrive sequentially, accumulating reusable experience across them can further improve performance. Existing memory-based methods either store whole-solution templates that generalize poorly to novel problems or use heuristic step-level selection that is not optimized for final-answer correctness. Learning selection policies requires large-scale training data and fixed action spaces, making such approaches unsuitable for test-time settings where memory expands incrementally and only limited supervision is available. We propose MILES (Modular Instruction Memory with LEarnable Selection for self-improving LLM reasoning), a framework that dynamically expands step-wise memory and applies correctness-optimized memory composition under realistic test-time constraints. MILES maintains modular memory units consisting of asymmetric pairs of sub-goal embeddings and sub-instructions, each associated with a learnable selection head. This memory structure enables a coarse-to-fine retrieval mechanism: The coarse level enables memory expansion and collects supervision for training selection heads from confident samples, while the fine stage applies learned selection heads to rerank coarse-level candidates and guide reasoning for uncertain samples. MILES consistently matches or outperforms prior methods while achieving superior accuracy-efficiency tradeoffs. Extensive experiments demonstrate its effectiveness, robustness, and transferability.

Chinese Translation

大型语言模型（LLMs）在测试时通过额外计算不断提高其推理能力，但大多数现有研究将每个问题视为孤立的。当问题顺序出现时，跨问题积累可重用经验可以进一步提高性能。现有的基于记忆的方法要么存储对新问题泛化效果不佳的完整解决方案模板，要么使用未针对最终答案正确性优化的启发式逐步选择。学习选择策略需要大规模训练数据和固定的动作空间，使得此类方法不适合在测试时逐步扩展记忆且仅有有限监督的环境中。我们提出了MILES（具有可学习选择的模块化指令记忆，用于自我改进的LLM推理），这是一个在现实测试时约束下动态扩展逐步记忆并应用正确性优化的记忆组合的框架。MILES维护由不对称的子目标嵌入和子指令对组成的模块化记忆单元，每个单元都与一个可学习的选择头相关联。这种记忆结构使得粗到细的检索机制成为可能：粗级别允许记忆扩展并从自信样本中收集监督以训练选择头，而细级别则应用学习到的选择头对粗级候选进行重新排序，并指导不确定样本的推理。MILES在一致性上与先前的方法相匹配或超越，同时实现了更优的准确性与效率权衡。大量实验表明其有效性、鲁棒性和可迁移性。

View on arXiv Download PDF AI Translation

cs.CL / 8 / 2607.07047

Riemannian Geometry for Pre-trained Language Model Embeddings

预训练语言模型嵌入的黎曼几何

Konior, Szczepan, Quemy, Alexandre, Klocek, Przemysław, Cattan, Grégoire, Sobieski, Bartłomiej

Abstract

Understanding the geometric structure of pre-trained language model embeddings matters for interpretability and safety. We ask whether sentence-level classification signal lives in the Riemannian geometry of contextual token embeddings, and probe it by extracting per-token pullback metrics from a learned encoder's analytical Jacobian and aggregating them with the Fr\'echet mean on the symmetric positive definite (SPD) manifold; we call this procedure Riemannian Mean Pooling (RMP). Across three datasets with non-trivial linguistic structure (CoLA, CREAK, RTE), RMP outperforms Euclidean mean pooling, while on FEVER-Symmetric, a benchmark constructed to remove annotation-driven lexical artifacts, the method correctly stays at chance. Ablations show that a randomly initialised encoder combined with Fr\'echet aggregation already beats Euclidean pooling on two of the three signal-bearing datasets, localising the source of the gain to the geometric aggregation rather than to learned manifold structure; the trained encoder contributes additional signal specifically on CREAK, the most knowledge-heavy of the three signal-bearing datasets.

Chinese Translation

理解预训练语言模型嵌入的几何结构对于可解释性和安全性至关重要。我们探讨句子级分类信号是否存在于上下文标记嵌入的黎曼几何中，并通过从学习到的编码器的解析雅可比矩阵中提取每个标记的拉回度量，并在对称正定（SPD）流形上使用Fréchet均值进行聚合来进行探究；我们将这一过程称为黎曼均值池化（Riemannian Mean Pooling, RMP）。在三个具有非平凡语言结构的数据集（CoLA、CREAK、RTE）上，RMP的表现优于欧几里得均值池化，而在FEVER-Symmetric这个旨在消除注释驱动的词汇伪影的基准测试中，该方法的表现恰好保持在随机水平。消融实验表明，随机初始化的编码器结合Fréchet聚合在三个信号承载数据集中的两个上已经超过了欧几里得池化，将增益的来源定位于几何聚合而非学习的流形结构；训练后的编码器在CREAK上贡献了额外的信号，这是三个信号承载数据集中知识密集度最高的。

View on arXiv Download PDF AI Translation

cs.CL / 9 / 2607.07050

Behavior Leverage Imbalance in Multi-Teacher On-Policy Distillation

多教师在线蒸馏中的行为杠杆不平衡

Shen, Jiabin, Chen, Guang, Mao, Chengjun

Abstract

Agentic language models must learn when to call tools, when to consume tool responses, and when to answer directly. This makes multi-teacher on-policy distillation a natural training strategy: one teacher can specialize in tool calls, another in direct responses, and the student can learn from both on its own generated distribution. We show that this strategy can induce a behavior shift that is invisible from aggregate losses alone. In a two-teacher tool-use setting, vanilla generalized knowledge distillation improves tool-call recall but also moves the model toward over-calling, where it calls tools on examples that should be answered directly. Aggregate explanations are insufficient: tool-call samples do not receive more token exposure, and full-sequence per-token divergence is not larger for the tool-call teacher. We instead analyze behavior leverage imbalance: local token-level signals at mode- entry and structural positions, such as and function names, can have disproportionate control over the global generation mode. We propose Soft Clamp, a per-token divergence calibration method that dynamically compresses extreme token-level Jensen-Shannon divergence while preserving nonzero gradients. On APIGen-MT, Soft Clamp reduces over-calling from 13.7% to 9.0% relative to vanilla GKD while matching its decision accuracy. In a BFCL multi-turn diagnostic, it also lowers tool-call loops and repeated calls among GKD variants. These results suggest that multi-teacher OPD should monitor where teacher signals act, not only how large they are in aggregate.

Chinese Translation

自主语言模型必须学习何时调用工具、何时使用工具响应以及何时直接回答。这使得多教师在线蒸馏成为一种自然的训练策略：一位教师可以专注于工具调用，另一位则专注于直接响应，而学生可以从其自身生成的分布中学习两者。我们展示了这一策略可以引发一种行为转变，这种转变仅通过聚合损失是无法察觉的。在一个双教师工具使用的设置中，普通的广义知识蒸馏提高了工具调用的召回率，但也使模型趋向于过度调用，即在应直接回答的示例上调用工具。聚合解释是不够的：工具调用样本并没有获得更多的标记曝光，且工具调用教师的全序列每标记散度并没有更大。我们反而分析了行为杠杆不平衡：在模式进入和结构位置（如和函数名称）上的局部标记级信号可以对全局生成模式产生不成比例的控制。我们提出了Soft Clamp，一种每标记散度校准方法，能够动态压缩极端标记级的詹森-香农散度，同时保持非零梯度。在APIGen-MT上，Soft Clamp将过度调用从13.7%降低到9.0%，同时保持决策准确性。在BFCL多轮诊断中，它还降低了工具调用循环和GKD变体之间的重复调用。这些结果表明，多教师在线蒸馏应监测教师信号的作用位置，而不仅仅是它们在聚合中的大小。

View on arXiv Download PDF AI Translation

cs.CL / 10 / 2607.07141

From Text to Parameters: Predicting Item Parameters from Embedding Regularization with Reliability and Design Ceilings

从文本到参数：通过嵌入正则化预测项目参数的可靠性和设计上限

Chen, Shi-Ting, Chen, Jinsong

Abstract

Newly developed items must ordinarily be field tested before their psychometric properties are known, creating a cold start problem for item calibration. Predicting item parameters from features is a long standing measurement problem dating back to the Linear Logistic Test Model; modern text embeddings now automate the design matrices traditionally specified by hand. We propose an evaluation framework combining regularized regression on item text embeddings, repeated cross validated R squared reported with its resampling standard deviation, and two performance upper bounds: a reliability ceiling derived from parameter standard errors, and a design ceiling derived from simulation based power calibration. Applying this framework to a mathematics item bank (EEDI) and a medical licensure benchmark (BEA 2024), we find that item difficulty is highly predictable from text (repeated cross validated R squared = 0.53, or about 57% of its reliability ceiling), whereas discrimination and pseudo guessing appear less predictable. However, evaluating these results against our ceilings reveals that this apparent hierarchy stems from target reliability rather than text signal strength: text uniformly recovers 57 to 63% of the reliable variance across difficulty targets, whereas the 3PL pseudo guessing parameter has a reliability ceiling near zero, making it an unviable target at current precision. On BEA, embedding based regression matches leaderboard RMSE despite explaining almost no variance, highlighting the critical need for scale free metrics and explicit ceilings in benchmarking. Finally, we show that a single train and test split can inflate apparent accuracy by 0.1 to 0.15 in R squared, underscoring the necessity of repeated cross validation for calibration support applications and future benchmark construction.

Chinese Translation

新开发的项目通常必须经过实地测试，才能了解其心理测量特性，这为项目校准带来了冷启动问题。从特征预测项目参数是一个长期存在的测量问题，早在线性逻辑测试模型（Linear Logistic Test Model）时期就已提出；现代文本嵌入技术现在可以自动生成传统上手动指定的设计矩阵。我们提出了一个评估框架，结合了对项目文本嵌入的正则化回归、重复交叉验证的R平方值及其重抽样标准差，以及两个性能上限：一个是基于参数标准误差得出的可靠性上限，另一个是基于模拟的功效校准得出的设计上限。将该框架应用于数学项目库（EEDI）和医学执照基准（BEA 2024），我们发现项目难度可以从文本中高度预测（重复交叉验证的R平方 = 0.53，约为其可靠性上限的57%），而区分度和伪猜测则显得不那么可预测。然而，将这些结果与我们的上限进行评估表明，这种明显的层次结构源于目标可靠性而非文本信号强度：文本在不同难度目标中均匀恢复了57%至63%的可靠方差，而3PL伪猜测参数的可靠性上限接近于零，使其在当前精度下成为不可行的目标。在BEA上，基于嵌入的回归尽管几乎没有解释任何方差，但与排行榜的均方根误差（RMSE）相匹配，突显了在基准测试中对无尺度指标和明确上限的迫切需求。最后，我们展示了单次训练和测试划分可以使表观准确度在R平方中膨胀0.1到0.15，强调了在校准支持应用和未来基准构建中重复交叉验证的必要性。

View on arXiv Download PDF AI Translation

cs.CL / 11 / 2607.07251

Evaluation of Multilingual Ability to Use Spatial Deictic Expressions in Vision-Language Models

评估视觉语言模型使用空间指示表达的多语言能力

Watanabe, Kaito, Yamamoto, Taisei, Doi, Tomoki, Yanaka, Hitomi

Abstract

One of the expected abilities of vision-language models (VLMs) is spatial reasoning ability based on a given text and image. To evaluate the spatial reasoning abilities of VLMs, we focus on the use of spatial deictic expressions, which are defined as spatial expressions whose referent is determined by their situational context, such as ``this'' and ``that''. To handle spatial deictic expressions, VLMs must jointly reason over language and visual space, grounding context-dependent references in the image's spatial structure. In addition, selecting appropriate spatial deictic expressions across languages requires VLMs to understand the language-specific spatial distinctions encoded by these expressions. In this paper, we develop a benchmark to evaluate the multilingual ability of VLMs to use spatial deictic expressions in four languages. Our experiments using this benchmark reveal that the tested models use demonstratives in a manner different from that of humans, particularly in selecting the appropriate demonstratives based on the distance to the object.

Chinese Translation

视觉语言模型（VLMs）预期具备的一项能力是基于给定文本和图像的空间推理能力。为了评估VLMs的空间推理能力，我们关注空间指示表达的使用，这些表达被定义为其指称由其情境上下文决定的空间表达，例如“这个”和“那个”。为了处理空间指示表达，VLMs必须在语言和视觉空间之间进行联合推理，将上下文相关的指称与图像的空间结构相结合。此外，在不同语言中选择合适的空间指示表达需要VLMs理解这些表达所编码的语言特定空间区分。在本文中，我们开发了一个基准，以评估VLMs在四种语言中使用空间指示表达的多语言能力。我们使用该基准进行的实验揭示，被测试模型在使用指示词时的方式与人类有所不同，特别是在根据与物体的距离选择合适的指示词时。

View on arXiv Download PDF AI Translation

cs.CL / 12 / 2607.07277

Understanding Interpretation Difficulty in Harmful Online Communication: Insights from Cybercrime Communities

理解有害在线交流中的解读难度：来自网络犯罪社区的见解

Okatsu, Tomohiro, Takada, Naoki, Pa, Yin Min Pa, Yoshioka, Katsunari, Mori, Tatsunori

Abstract

Harmful online communication often contains slang, coded terms, abbreviations, and community-specific expressions, which make messages difficult to interpret. This paper presents an exploratory study of interpretation difficulty in Discord chats related to cybercrime. We construct reference interpretations of purposefully selected difficult messages, which were reviewed by an expert. We then use them to evaluate human and large language model (LLM) interpretations under different context conditions. The results show that local context alone is often insufficient for humans, while external knowledge and extended conversational context substantially improve human interpretation. For LLMs, local context also improves interpretation, and the larger model performs better. We further conduct a qualitative error analysis and propose a preliminary classification of factors that make harmful chats difficult to interpret. These findings suggest that harmful-content analysis should treat interpretation as an evidence-integration problem, rather than as message-level classification alone.

Chinese Translation

有害的在线交流通常包含俚语、编码术语、缩写和特定社区的表达方式，这使得信息的解读变得困难。本文呈现了一项关于网络犯罪相关的Discord聊天中解读难度的探索性研究。我们构建了经过专家审阅的有意选择的困难信息的参考解读。随后，我们利用这些参考解读在不同的上下文条件下评估人类和大型语言模型（LLM）的解读效果。结果表明，仅依赖局部上下文通常不足以帮助人类解读，而外部知识和扩展的对话上下文则显著提高了人类的解读能力。对于LLM而言，局部上下文同样改善了解读效果，且更大的模型表现更佳。我们进一步进行了定性错误分析，并提出了使有害聊天难以解读的因素的初步分类。这些发现表明，有害内容分析应将解读视为一个证据整合问题，而不仅仅是信息层面的分类。

View on arXiv Download PDF AI Translation

cs.CL / 13 / 2607.07282

A Word-Level Digital Reader of the Prasthanatrayi with Sankara's Bhasya: Corpus, Method, and an Open, Offline Reading Aid for the Advaita Vedanta Canon

带有商羯罗注释的普拉斯坦那特赖（Prasthanatrayi）词级数字阅读器：语料库、方法及一款开放的离线阅读辅助工具用于非二元论经典

Maharaj, Tamal

Abstract

The Prasthanatrayi -- the ten principal Upanisads, the Brahmasutra, and the Bhagavadgita, with Sankara's commentaries (bhasya) -- is the foundational corpus of Advaita Vedanta. Continuous euphonic combination (sandhi), long compounds (samasa), and dense scholastic prose make it hard to read at the word level: where one word ends, and what each word means grammatically, are both obscured. We present an open, fully offline, word-level digital reader of the entire Prasthanatrayi with Sankara's bhasya. Every word -- of both the root text (mula) and the commentary -- is clickable and resolves to a pop-up giving its split (padaccheda), morphological analysis, and gloss. Because every word carries a lemma, the reader also acts as a concordance: a search on a dictionary headword retrieves all of that word's inflected and sandhi-hidden occurrences, and its occurrences inside compounds, across both layers. The resource covers thirteen commentarial units (2,971 verses, sutras, and prose sections; 36,881 analysed word-occurrences of root text) and a global dictionary of 95,587 distinct commentarial surface forms. We describe the corpus, the hybrid pipeline -- a rule-based sandhi splitter over an inflected-form lexicon and attested-corpus look-ups, with LLM-assisted analysis under an adversarial two-pass verification protocol -- and a durable human-review loop whose corrections survive every regeneration. An intrinsic evaluation against independent Sanskrit resources finds high-confidence analyses agree with an authoritative inflectional lexicon on over 99% of attested forms, and a band-blind adjudication confirms that quality degrades predictably across confidence bands, with errors concentrated in the low-confidence tier the review loop targets. The reader is a single self-contained HTML file needing no server or network, offered as a freely redistributable teaching and reading aid.

Chinese Translation

普拉斯坦那特赖（Prasthanatrayi）——包括十部主要的奥义书（Upanisads）、《梵文经》（Brahmasutra）和《博伽梵歌》（Bhagavadgita），以及商羯罗的注释（bhasya）——是非二元论（Advaita Vedanta）的基础语料库。连续的音韵结合（sandhi）、长复合词（samasa）和密集的学术散文使得在词级上阅读变得困难：一个词的结束位置以及每个词的语法意义都变得模糊。我们展示了一个开放的、完全离线的、带有商羯罗注释的普拉斯坦那特赖的词级数字阅读器。根文本（mula）和注释中的每个词都是可点击的，点击后会弹出窗口显示其分割（padaccheda）、形态分析和释义。由于每个词都携带一个词根，阅读器还充当了一个索引：在字典的词头上搜索可以检索到该词的所有变形和隐藏的音韵结合出现，以及它在复合词中的出现，涵盖两个层面。该资源涵盖了十三个注释单元（2,971 诗句、经文和散文部分；36,881 个根文本的分析词汇出现）以及一个包含 95,587 种不同注释表面形式的全球字典。我们描述了语料库、混合管道——基于规则的音韵分割器与变形词汇表和已证实语料库的查找，结合在对抗性双通道验证协议下的 LLM 辅助分析——以及一个持久的人类审查循环，其修正会在每次再生中保留。与独立的梵语资源进行的内在评估发现，高置信度分析与权威的变形词汇表在超过 99% 的已证实形式上达成一致，盲审裁定确认质量在置信度区间内可预测地下降，错误集中在审查循环所针对的低置信度层。该阅读器是一个独立的 HTML 文件，无需服务器或网络，作为一个可自由再分发的教学和阅读辅助工具提供。

View on arXiv Download PDF AI Translation

cs.CL / 14 / 2607.07302

Evaluating RAG Metrics in Applied Contexts: An Experiment, Its Findings and Its Limitations

在应用背景中评估 RAG 指标：实验、发现及其局限性

Brabant, Quentin

Abstract

This paper reports an empirical study evaluating the relevance of several RAG metrics. The experiment is based on a question-answering dataset created by human annotators from business data. The generated responses and retrieved spans of a RAG system are scored using evaluation metrics from four libraries (Ragas, DeepEval, RAGChecker, Opik). These metrics are compared to scores given by two evaluators, as well as to standard metrics such as recall. An analysis of correlations is conducted. Finally, we highlight certain limitations of our methodology, compare it to those used in the literature, and suggest some avenues for future research. This paper is an English translation of a paper originally published in the French-speaking workshop EvalLLM (Brabant, 2026).

Chinese Translation

本文报告了一项实证研究，评估了多个 RAG 指标的相关性。该实验基于由人类注释者从商业数据中创建的问题回答数据集。生成的响应和 RAG 系统检索的片段使用来自四个库（Ragas、DeepEval、RAGChecker、Opik）的评估指标进行评分。这些指标与两位评估者给出的评分以及标准指标（如召回率）进行了比较。我们进行了相关性分析。最后，我们强调了我们方法论的某些局限性，将其与文献中使用的方法进行了比较，并提出了一些未来研究的方向。本文是原发表于法语研讨会 EvalLLM（Brabant, 2026）的论文的英文翻译。

View on arXiv Download PDF AI Translation

cs.CL / 15 / 2607.07318

R^3: Advertisement Compliance Rectification via Group-Relative Experience Extractor and Curriculum Reinforcement

R^3：通过群体相对经验提取器和课程强化实现广告合规整改

Chen, Yuan, Hu, Zhenyu, Xue, Mengge, Cao, Te, Liu, Liqun, Shu, Peng, Yu, Huan, Jiang, Jie

Abstract

Rigorous content moderation is crucial for online advertising but leads to millions of daily rejections. This scale renders manual rectification infeasible, particularly for video advertisements. However, existing safety-driven methods often suffer from aggressive over-editing, which compromises the advertiser's original semantic intent merely to satisfy compliance. In this work, we target the rectification of textual violations in video ads, covering both speech transcripts and on-screen text. We propose R^3, a novel framework designed to harmonize compliance with original semantic intent preservation. Our approach integrates three key innovations: (1) an experience-driven data synthesis framework that bootstraps high-quality supervision via a group-Relative compliance experience extractor; (2) a curriculum Reinforcement learning strategy with hierarchical rewards designed to enforce compliance while maximizing semantic consistency; and (3) a comprehensive video Rectification framework seamlessly integrating text recognition, rewriting, and re-rendering for industrial deployment. Extensive experiments on industrial datasets and online A/B testing demonstrate that R^3 significantly outperforms state-of-the-art baselines, achieving an optimal trade-off between violation rectification and intent preservation.

Chinese Translation

严格的内容审核对于在线广告至关重要，但这导致每天有数百万条广告被拒绝。这种规模使得人工整改变得不可行，尤其是对于视频广告。然而，现有的以安全为驱动的方法往往存在过度编辑的问题，这在满足合规要求的同时损害了广告商的原始语义意图。在本研究中，我们针对视频广告中的文本违规行为进行整改，涵盖语音稿和屏幕文本。我们提出了R^3，一个旨在协调合规性与原始语义意图保留的新框架。我们的方法整合了三个关键创新：(1) 一个以经验驱动的数据合成框架，通过群体相对合规经验提取器引导高质量的监督；(2) 一种具有分层奖励的课程强化学习策略，旨在在最大化语义一致性的同时强制执行合规性；(3) 一个全面的视频整改框架，完美整合文本识别、重写和重新渲染，以便于工业部署。在工业数据集和在线A/B测试上的广泛实验表明，R^3显著优于最先进的基线，实现了违规整改与意图保留之间的最佳权衡。

View on arXiv Download PDF AI Translation

cs.CL / 16 / 2607.07388

TF-Engram: A Train-Free Engram with SSD-Backed Memory for Large Language Models

TF-Engram：一种无训练的带SSD支持内存的大型语言模型的Engram

Ma, Yutang, Huang, Kecheng, Jiang, Xikun, Shao, Zili

Abstract

Large Language Models (LLMs) store factual knowledge and domain-specific patterns implicitly in dense Transformer parameters, making knowledge expansion costly through pretraining, fine-tuning, retrieval augmentation, or longer contexts. Engram-style memory offers a compact hidden-state injection pathway, but existing GPU-resident designs often rely on hash-based compression, causing unrelated phrases to collide in shared slots and weakening phrase-level semantic fidelity. We present TF-Engram, a train-free Engram system that constructs phrase-specific semantic memory offline from external corpora, stores large memory tables across a GPU--DRAM--SSD hierarchy, and uses Early-Exit Guided Predictive Prefetching to hide external-memory latency during autoregressive decoding. On Qwen3-0.6B, TF-Engram improves the average downstream score from 57.6 to 59.4, outperforming both the frozen backbone and a parameter-matched LoRA baseline. System evaluation shows that large TF-Engram tables can be built with moderate offline cost, SSD-backed storage substantially reduces GPU memory demand, and predictive prefetching recovers much of the throughput loss caused by external memory access. These results demonstrate that static phrase memory can be integrated into LLM inference as a scalable, train-free, and low-overhead system component.

Chinese Translation

大型语言模型（LLMs）在密集的Transformer参数中隐式存储事实知识和特定领域的模式，使得通过预训练、微调、检索增强或更长上下文进行知识扩展的成本较高。Engram风格的内存提供了一种紧凑的隐藏状态注入路径，但现有的GPU驻留设计往往依赖于基于哈希的压缩，这导致无关短语在共享槽中发生冲突，从而削弱了短语级语义的保真度。我们提出了TF-Engram，这是一种无训练的Engram系统，能够离线从外部语料库构建短语特定的语义记忆，跨越GPU-DRAM-SSD层次存储大型内存表，并使用早期退出引导预测预取（Early-Exit Guided Predictive Prefetching）在自回归解码过程中隐藏外部内存延迟。在Qwen3-0.6B上，TF-Engram将平均下游得分从57.6提高到59.4，超越了冻结的主干网络和参数匹配的LoRA基线。系统评估表明，可以以适度的离线成本构建大型TF-Engram表，SSD支持的存储显著减少了GPU内存需求，而预测预取则恢复了因外部内存访问造成的吞吐量损失。这些结果表明，静态短语内存可以作为可扩展的、无训练的、低开销的系统组件集成到LLM推理中。

View on arXiv Download PDF AI Translation

cs.CL / 17 / 2607.07408

Transformer-based segmentation of prosodic boundaries in Brazilian Portuguese

基于变换器的巴西葡萄牙语韵律边界分割

Lima, Rodrigo de Freitas, Galdino, Julio Cesar, Treviso, Marcos Vinicius

Abstract

Automatic prosodic segmentation identifies boundaries between speech units from acoustic and linguistic evidence. Although recent deep learning approaches have produced strong results for English, automatic segmentation for Brazilian Portuguese (BP) still relies mostly on rule-based or traditional machine-learning methods. This paper presents SAMPA, a Whisper-based segmenter that transcribes BP speech while inserting explicit markers for terminal prosodic boundaries. We fine-tune Whisper large-v3 on manually segmented recordings from the NURC-SP dataset and evaluate different training and test-time filtering configurations, including out-of-distribution testing on the MuPe-Diversidades dataset. SAMPA achieves competitive boundary-detection performance across settings, with the best models reaching F1=0.731 on the held-out test split and F1=0.796 on MuPe-Diversidades. Finally, through n-gram and acoustic-visual analyses, we show that our model follows morphosyntactic, semantic, and prosodic cues for detecting prosodic boundaries.

Chinese Translation

自动韵律分割通过声学和语言证据识别语音单元之间的边界。尽管最近的深度学习方法在英语中取得了良好的效果，但巴西葡萄牙语（BP）的自动分割仍主要依赖于基于规则或传统机器学习的方法。本文提出了SAMPA，一个基于Whisper的分割器，它在转录BP语音的同时插入终端韵律边界的显式标记。我们在NURC-SP数据集中手动分割的录音上微调了Whisper large-v3，并评估了不同的训练和测试时过滤配置，包括在MuPe-Diversidades数据集上的分布外测试。SAMPA在各个设置下实现了具有竞争力的边界检测性能，最佳模型在保留的测试集上达到F1=0.731，在MuPe-Diversidades上达到F1=0.796。最后，通过n-gram和声学-视觉分析，我们展示了我们的模型遵循形态句法、语义和韵律线索来检测韵律边界。

View on arXiv Download PDF AI Translation

cs.CL / 18 / 2607.07409

DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting

DeLS-Spec：用于并行推测草拟的解耦长短上下文

Zheng, Hong-Kai, Li, Piji

Abstract

Speculative decoding accelerates LLM inference by drafting multiple tokens and verifying them in parallel. Block-parallel drafters such as DFlash further improve drafting efficiency by predicting an entire block in one pass, but their position-wise predictions lack explicit intra-block causal conditioning. Recent methods such as Domino and DSpark attempt to introduce such causality into block-parallel drafting, but they require training the draft model from scratch, which limits their flexibility and increases training cost. We propose DeLS-Spec, a decoupled long-short context speculative decoding method. DeLS-Spec treats the fixed DFlash model as a long-context expert and introduces a lightweight local head as a short-context expert. The local head can be trained independently with a standard next-token prediction objective, without joint training with the target model or the DFlash backbone, leading to extremely low training cost. At inference time, DeLS-Spec combines long-context and short-context logits, and the local head is not tied to a specific DFlash checkpoint, making the method more modular and flexible. Experiments on Qwen3 models show that DeLS-Spec consistently improves speedup and average acceptance length over DFlash across math, code, and dialogue benchmarks.

Chinese Translation

推测解码通过并行草拟多个标记并验证它们来加速大规模语言模型（LLM）的推理。块并行草拟器如 DFlash 通过一次性预测整个块进一步提高了草拟效率，但它们的逐位置预测缺乏明确的块内因果条件。最近的方法如 Domino 和 DSpark 尝试将这种因果关系引入块并行草拟，但它们需要从头开始训练草拟模型，这限制了它们的灵活性并增加了训练成本。我们提出了 DeLS-Spec，一种解耦的长短上下文推测解码方法。DeLS-Spec 将固定的 DFlash 模型视为长上下文专家，并引入一个轻量级的局部头作为短上下文专家。局部头可以独立训练，使用标准的下一个标记预测目标，而无需与目标模型或 DFlash 主干进行联合训练，从而导致极低的训练成本。在推理时，DeLS-Spec 结合长上下文和短上下文的 logits，并且局部头不依赖于特定的 DFlash 检查点，使得该方法更加模块化和灵活。在 Qwen3 模型上的实验表明，DeLS-Spec 在数学、代码和对话基准测试中始终提高了相对于 DFlash 的加速和平均接受长度。

View on arXiv Download PDF AI Translation

cs.CL / 19 / 2607.07469

SynthAVE: Scalable Synthetic Labeling for E-Commerce with LLM-Arena Validation

SynthAVE：用于电子商务的可扩展合成标签生成与LLM-Arena验证

Scarinci, Andrea, Negri, Virginia, Impata, Brayan, Khan, Suleiman, Martinez, Victor, Federico, Marcello

Abstract

Fine-tuning large language models (LLMs) for e-commerce attribute extraction requires labeled data representative across thousands of product types, attributes, and multiple languages. This combinatorial scale translates to millions of annotations, rendering human labeling prohibitively costly. While recent work has demonstrated synthetic label generation using LLMs, deploying such approaches at industrial scale requires integrated quality control mechanisms. We present SynthAVE, a large-scale human-validated benchmark for attribute value extraction spanning 12,726 products across 229 product types, 792 attributes, and 4 languages (Spanish, French, Italian, German). To validate synthetic labels at scale, we introduce a multi-LLM arena framework where samples are independently evaluated by 21 judge configurations (7 model families $\times$ 3 prompts), with final labels determined via majority voting. The majority vote ensemble agrees with human experts at Cohen's $\kappa = 0.92$ (95.2% agreement), while individual judges show substantial inter-model agreement (Fleiss' $\kappa = 0.76$). This demonstrates that diverse models with varying individual judgments aggregate into highly reliable predictions, enabling cost-effective validation at scale while maintaining quality parity with human review.

Chinese Translation

针对电子商务属性提取的超大规模语言模型（LLMs）微调需要在数千种产品类型、属性和多种语言中具有代表性的标注数据。这种组合规模转化为数百万条注释，使得人工标注的成本极高。尽管近期的研究已经展示了使用LLMs生成合成标签，但在工业规模上部署此类方法需要集成的质量控制机制。我们提出了SynthAVE，这是一个大规模的人类验证基准，涵盖了12,726种产品，涉及229种产品类型、792个属性和4种语言（西班牙语、法语、意大利语、德语）。为了在大规模上验证合成标签，我们引入了一个多LLM竞技场框架，其中样本由21个评审配置（7个模型系列 × 3个提示）独立评估，最终标签通过多数投票确定。多数投票集成与人类专家的意见一致，Cohen's κ = 0.92（95.2%的一致性），而个别评审者显示出显著的模型间一致性（Fleiss' κ = 0.76）。这表明，具有不同个体判断的多样化模型能够聚合成高度可靠的预测，从而在保持与人工审核质量平等的同时，实现成本效益高的规模验证。

View on arXiv Download PDF AI Translation

cs.CL / 20 / 2607.07548

Think Big, Search Small: Where Capacity Matters in Hierarchical Search Agents?

放眼大局，聚焦细节：层次搜索代理中容量的重要性？

Cai, Qinnan, Zhao, Yibo, Li, Xiang

Abstract

Large language model based search agents increasingly adopt multi-agent architectures in which a main agent decomposes a complex question into sub-queries and dispatches them to parallel sub-agents. However, existing systems instantiate all roles from a single model of identical scale, leaving open how model capacity should be distributed across roles. We factorize hierarchical search into three roles: a delegation role responsible for task decomposition, an execution role responsible for retrieval and evidence extraction, and an answer generation role held fixed as a confound control. We then conduct controlled capacity sweeps along the delegation and execution axes on five multi-hop QA benchmarks. The experiments yield three findings. First, role factorization consistently outperforms a single-agent baseline, improving exact match from 4.5 to 8.6 points across six model scales. Second, capacity sensitivity is asymmetric: scaling the delegation backbone improves EM by ~11 points, whereas scaling the execution sub-agent moves EM by only ~2.6 points, identifying decomposition as the capability bottleneck. Third, a 1.7B-parameter executor trained via quality-filtered trajectory distillation matches a frontier sub-agent in accuracy while consuming 37% fewer sub-agent tokens, advancing the Pareto frontier. These results suggest a concrete recipe for building hierarchical search agents: concentrate capacity at delegation and downsize execution without sacrificing accuracy. Our code is available at https://github.com/QinnanCai0115/role-factorized-search.

Chinese Translation

基于大型语言模型的搜索代理越来越多地采用多代理架构，其中主代理将复杂问题分解为子查询，并将其分派给并行的子代理。然而，现有系统从单一规模相同的模型实例化所有角色，尚未明确模型容量应如何在各角色之间分配。我们将层次搜索分解为三个角色：负责任务分解的委派角色、负责检索和证据提取的执行角色，以及作为混杂控制保持固定的答案生成角色。然后，我们在五个多跳问答基准上沿着委派和执行轴进行控制容量的测试。实验结果得出三项发现。首先，角色分解始终优于单代理基线，在六个模型规模中，准确匹配（Exact Match, EM）提高了4.5到8.6个百分点。其次，容量敏感性是不对称的：扩大委派主干使EM提高约11个百分点，而扩大执行子代理仅使EM提高约2.6个百分点，识别出分解是能力瓶颈。第三，通过质量过滤的轨迹蒸馏训练的1.7B参数执行器在准确性上与前沿子代理相匹配，同时消耗的子代理令牌减少了37%，推动了帕累托前沿。这些结果为构建层次搜索代理提供了具体的方案：将容量集中于委派，并在不牺牲准确性的情况下缩小执行规模。我们的代码可在 https://github.com/QinnanCai0115/role-factorized-search 获取。

View on arXiv Download PDF AI Translation

cs.CL / 21 / 2607.07557

PALS: Percentile-Aware Layerwise Sparsity for LLM Pruning

PALS：基于百分位的逐层稀疏性用于大语言模型剪枝

Jamshidi, Yazdan, Shvets, Alexey

Abstract

One-shot pruning methods like Wanda and SparseGPT apply the same sparsity ratio to every layer of a transformer, ignoring known variation in layer importance. We propose PALS (Percentile-Aware Layerwise Sparsity), which adjusts per-layer sparsity based on the 99th percentile of activation magnitudes, bounded to $\pm 5\%$ around the target ratio. On LLaMA-2-7B at 50\% sparsity, PALS achieves 10.96 WikiText-2 perplexity versus 12.92 for uniform Wanda (mean over 9 runs, $p < 0.001$). The benefit is architecture-dependent: LLaMA-3-8B shows marginal gains and Mistral-7B shows none. We also find that gradient-based allocation -- the seemingly more principled approach -- produces results worse than random, suggesting that gradient magnitude does not predict the impact of discrete weight removal. PALS adds negligible cost to the pruning pipeline and requires no fine-tuning.

Chinese Translation

像 Wanda 和 SparseGPT 这样的单次剪枝方法对变换器的每一层应用相同的稀疏比率，忽略了层重要性已知的变化。我们提出了 PALS（基于百分位的逐层稀疏性），该方法根据激活幅度的第 99 个百分位数调整每层的稀疏性，限制在目标比率的 $m{ ext{±}} 5\%$ 范围内。在 LLaMA-2-7B 模型上，当稀疏率为 50 ext{%} 时，PALS 达到了 10.96 的 WikiText-2 困惑度，而统一的 Wanda 则为 12.92（9 次运行的平均值，$p < 0.001$）。这种效果依赖于架构：LLaMA-3-8B 显示出边际收益，而 Mistral-7B 则没有任何收益。我们还发现，基于梯度的分配——这种看似更有原则的方法——产生的结果比随机选择还要差，这表明梯度幅度并不能预测离散权重移除的影响。PALS 对剪枝流程的额外成本微乎其微，并且不需要微调。

View on arXiv Download PDF AI Translation

cs.CL / 22 / 2607.07626

Future Confidence Distillation in Large Language Models

大型语言模型中的未来置信蒸馏

Kale, Sahil

Abstract

Reliable confidence estimation is essential for deploying large language models (LLMs) in confidence-aware systems, where downstream decisions such as retrieval, tool use, and adaptive computation depend on accurately estimating answer reliability. Existing approaches, however, largely treat confidence as a property of completed responses, overlooking how confidence-related information evolves throughout the answering process. In this work, we investigate confidence from a temporal perspective by comparing pre-solution Feeling-of-Knowing (FOK) and post-solution Judgement-of-Learning (JOL) confidence estimates across frontier and open-source LLMs. We show that post-solution confidence is consistently better calibrated and more discriminative than pre-solution confidence, while linear probes trained on hidden representations recover substantially richer confidence-related information than models explicitly verbalise. Building on this observation, we introduce future confidence distillation, which trains predictors operating on pre-solution hidden representations using teacher confidence estimates produced by post-solution correctness probes. Despite requiring only pre-solution representations for inference, distilled predictors recover much of the calibration improvement achieved by post-solution confidence, remain highly sample efficient, and transfer across datasets within the same domain. Together, our findings demonstrate that confidence-related information evolves throughout the answering process and can be anticipated before answer generation is complete, enabling significantly more reliable yet low-cost confidence estimation.

Chinese Translation

可靠的置信度估计对于在置信度感知系统中部署大型语言模型（LLMs）至关重要，因为下游决策（如检索、工具使用和自适应计算）依赖于对答案可靠性的准确估计。然而，现有的方法在很大程度上将置信度视为已完成响应的特性，忽视了置信度相关信息在回答过程中如何演变。在本研究中，我们从时间的角度研究置信度，通过比较前解决方案的知识感知（Feeling-of-Knowing, FOK）和后解决方案的学习判断（Judgement-of-Learning, JOL）置信度估计，分析前沿和开源LLMs。我们发现，后解决方案的置信度在校准性和区分性上始终优于前解决方案的置信度，同时，基于隐藏表示训练的线性探测器恢复了比模型明确表达的更丰富的置信度相关信息。基于这一观察，我们引入了未来置信蒸馏，该方法利用后解决方案正确性探测器产生的教师置信度估计，训练在前解决方案隐藏表示上运行的预测器。尽管推理仅需前解决方案表示，蒸馏预测器仍能恢复后解决方案置信度所实现的校准改进的大部分，保持高样本效率，并在同一领域内跨数据集迁移。综上所述，我们的研究结果表明，置信度相关信息在回答过程中不断演变，并且可以在答案生成完成之前进行预测，从而实现显著更可靠且低成本的置信度估计。

View on arXiv Download PDF AI Translation

cs.CL / 23 / 2607.07669

DiaLLM: An Investigation into the Robustness-Generation Gap in English Dialect Adaptation

DiaLLM：对英语方言适应中的鲁棒性-生成差距的研究

Painter, Jordan, Srirag, Dipankar, Kappiyath, Adarsh, Kanojia, Diptesh, Joshi, Aditya, Yin, Lu

Abstract

Large language models increasingly \emph{understand} dialectal English, yet still \emph{produce} only standard, US-leaning English, leaving dialectal generation, the harder half of the problem, largely unaddressed. We introduce \textbf{DiaLLM}, which continually pretrains three open-weight language model families on the International Corpus of English and applies implicit and explicit post-training paradigms, each combined with three model alignment strategies, giving the first controlled comparison of these components across Australian, Indian, and Northern British English. Our results reveal that dialectal robustness and generation are \emph{dissociated}: benchmarks are shaped by continual pretraining and SFT, while alignment visibly reshapes generation in ways benchmarks do not capture. Explicit variety-targeted adaptation produces output reliably recognised as dialectal and preferred over broad alignment, yet the method that most aggressively optimises the dialectal reward is not preferred by human evaluators. Independent linguistic analysis corroborates this reward-quality gap, most clearly on two of the three families. No single alignment method dominates, and closing the gap will require richer reward designs and continued investment in dialectal resources. We release all code, checkpoints, and preference datasets.

Chinese Translation

大型语言模型越来越能够 extit{理解}方言英语，但仍然 extit{生成}仅限于标准的、偏向美国的英语，使得方言生成这一更为复杂的问题在很大程度上未得到解决。我们引入了 extbf{DiaLLM}，该模型持续对三种开放权重的语言模型系列进行预训练，使用国际英语语料库，并应用隐式和显式的后训练范式，每种范式结合三种模型对齐策略，从而首次对这些组件在澳大利亚、印度和北英方言中的表现进行控制比较。我们的结果揭示了方言鲁棒性和生成之间的 extit{解耦}：基准测试受到持续预训练和监督微调（SFT）的影响，而对齐在生成中显著重塑了基准测试未能捕捉的方式。显式的针对方言的适应方法产生了被可靠识别为方言的输出，并优于广泛对齐，然而，最积极优化方言奖励的方法并未受到人类评估者的青睐。独立的语言学分析证实了这一奖励-质量差距，尤其在三种模型系列中的两种上最为明显。没有单一的对齐方法占主导地位，缩小这一差距将需要更丰富的奖励设计和对方言资源的持续投资。我们发布了所有代码、检查点和偏好数据集。

View on arXiv Download PDF AI Translation

cs.CL / 24 / 2607.07670

Does Bielik Know What It Doesn't Know? Activation Dispersion Separates Entity Familiarity from Factual Reliability Across Model Scale

Bielik 知道它不知道什么吗？激活分散将实体熟悉度与事实可靠性在模型规模上区分开来

Brzezinka, Grzegorz

Abstract

Large language models hallucinate most about entities they have never seen. We ask whether a model's activations betray entity familiarity before a single answer token is generated, and whether that signal predicts the factual reliability of the answers. On four Polish Bielik models (1.5B-11B parameters), we probe four entity domains (athletes, cities, writers, musicians), each with 42 well-known, 42 obscure-but-real, and 42 fabricated entities addressed by a one-sentence question (504 prompts per model). Two unsupervised, single-forward-pass dispersion measures over post-SwiGLU MLP activations, inverse participation ratio and spectral entropy, separate known from fabricated entities at AUROC 0.95-1.00 across all domains and scales; a supervised linear probe reaches 0.99-1.00. Both clear selection-aware permutation floors of about 0.70-0.74 (empirical p<=1e-3), survive held-out layer selection (0.93-0.99), and persist on real names (known vs. obscure-but-real: 0.96-1.00). The signal transfers across entity types (mean off-diagonal AUROC 0.92-0.99); a matched-template counterfactual shows the only large drops are template-caused, not entity-type effects, and the signal is diffuse across heads. This representational signal is already at ceiling at 1.5B, whereas behavioral factual reliability scales sharply: 0, 2, 10, and 19 of 42 known athletes are answered fully correctly by the 1.5B, 4.5B, 7B, and 11B models under a strict judge. Within known entities, separating correct from hallucinated answers is much harder (probe 0.93; dispersion no better than a first-token-entropy baseline). A five-sample semantic-entropy baseline reaches only 0.71-0.83 at 5x the inference cost. Despite this internal awareness, the models almost never abstain: an audit of 2,520 answers finds 2 refusals and 1 hedge. Entity familiarity and factual reliability are distinct phenomena on different scaling curves.

Chinese Translation

大型语言模型在未见过的实体上大多会产生幻觉。我们探讨模型的激活是否在生成单个答案标记之前揭示了实体的熟悉度，以及该信号是否能预测答案的事实可靠性。在四个波兰 Bielik 模型（1.5B-11B 参数）上，我们考察了四个实体领域（运动员、城市、作家、音乐家），每个领域包含 42 个知名实体、42 个不太知名但真实的实体以及 42 个虚构的实体，通过一个句子的问题进行探讨（每个模型 504 个提示）。两种无监督的单次前向传播分散度量（后 SwiGLU MLP 激活的逆参与比率和谱熵）在所有领域和规模上以 AUROC 0.95-1.00 将已知实体与虚构实体区分开；一个监督线性探测器达到了 0.99-1.00。两者的选择感知置换底线均约为 0.70-0.74（经验 p<=1e-3），在保留层选择中生存（0.93-0.99），并在真实名称上持续存在（已知与不太知名但真实的实体：0.96-1.00）。该信号在实体类型之间转移（平均非对角 AUROC 0.92-0.99）；一个匹配模板的反事实显示，唯一的大幅下降是由模板引起的，而非实体类型效应，且信号在各个头部之间是分散的。该表征信号在 1.5B 模型中已达到上限，而行为事实可靠性则急剧上升：在严格评判下，1.5B、4.5B、7B 和 11B 模型分别完全正确回答了 0、2、10 和 19 个已知运动员。在已知实体中，区分正确答案与幻觉答案要困难得多（探测 0.93；分散度不比首个标记熵基线更好）。一个五样本语义熵基线在 5 倍推理成本下仅达到 0.71-0.83。尽管存在这种内部意识，模型几乎从不放弃：对 2,520 个答案的审计发现 2 次拒绝和 1 次模糊回答。实体熟悉度和事实可靠性是不同的现象，具有不同的规模曲线。

View on arXiv Download PDF AI Translation

cs.CL / 25 / 2607.07702

From Noisy Traces to Root Causes: Structural Trajectory Analysis and Causal Extraction for Agent Optimization

从噪声轨迹到根本原因：结构轨迹分析与因果提取用于智能体优化

Chang, Ying, Xu, Jiahang, Feng, Xuan, Yang, Chenyuan, Cheng, Peng, Yang, Yuqing

Abstract

The optimization of long-horizon agents increasingly relies on reflection-based mechanisms, where a large language model (LLM) acts as an optimizer to diagnose agent failures and improve agent policies. However, real execution traces are difficult to use directly for optimization: large trace collections are often redundant and heterogeneous, making optimization inefficient and prone to overfitting to low-value failures; meanwhile, each individual trajectory also contains many irrelevant steps, while naive context reduction methods such as truncation or sliding windows can discard causally important evidence and produce misleading optimization signals. To resolve this dilemma, we introduce STRACE (Structural TRajectory Analysis and Causal Extraction), a framework that constructs high signal-noise optimization contexts for more precise and effective optimization. At the batch level, STRACE mines failure patterns to filter redundant traces and retain representative failures; within each selected trace, it performs causal localization over a textual dependency graph to remove non-causal steps and identify the true root-cause module for optimization. Empirical results demonstrate that STRACE significantly outperforms standard context-filtering baselines. Notably, on a challenging formal verification task (VeruSAGE-Bench), it successfully optimizes human-expert designed agents, delivering $1.4\times$ success-rate improvement (42.5% to 58.5%). The code is available at https://github.com/moomight/STRACE .

Chinese Translation

长时间跨度智能体的优化越来越依赖于基于反思的机制，其中大型语言模型（LLM）作为优化器来诊断智能体故障并改进智能体策略。然而，实际执行轨迹难以直接用于优化：大量轨迹集合通常冗余且异质，使得优化效率低下，并容易过拟合于低价值的故障；与此同时，每个单独的轨迹也包含许多无关步骤，而简单的上下文缩减方法，如截断或滑动窗口，可能会丢弃因果上重要的证据并产生误导性的优化信号。为了解决这一困境，我们引入了STRACE（结构轨迹分析与因果提取），一个构建高信号噪声优化上下文的框架，以实现更精确和有效的优化。在批处理层面，STRACE挖掘故障模式以过滤冗余轨迹并保留代表性故障；在每个选定轨迹内，它在文本依赖图上执行因果定位，以去除非因果步骤并识别用于优化的真实根本原因模块。实证结果表明，STRACE显著优于标准上下文过滤基线。值得注意的是，在一个具有挑战性的形式验证任务（VeruSAGE-Bench）上，它成功优化了人类专家设计的智能体，实现了$1.4 imes$的成功率提升（从42.5%提升至58.5%）。代码可在https://github.com/moomight/STRACE获取。

View on arXiv Download PDF AI Translation

cs.CL / 26 / 2607.07707

Co-LMLM: Continuous-Query Limited Memory Language Models

共查询有限记忆语言模型（Co-LMLM）

Feldman, Yair, Zhao, Linxi, Godey, Nathan, Go, Dongyoung, Hua, Yilun, Weinberger, Kilian Q., Sun, Jennifer J., Artzi, Yoav

Abstract

Limited memory language models (LMLMs) externalize factual knowledge during pretraining to a knowledge base (KB), rather than memorizing it in their weights. During generation, the model then fetches knowledge from the KB as needed. This recently introduced paradigm provides multiple advantages, including knowledge control capabilities that remain beyond conventional LLMs. We propose continuous-query LMLM (CO-LMLM), where the KB pairs continuous keys with textual knowledge values, a significant departure from prior reliance on relational KB and queries. CO-LMLM generates flexible vector queries at minimal cost, while still integrating human-readable and attributable retrieved knowledge into its generation. We pair this design with an annotation pipeline that tags free-form factual spans in arbitrary text, removing prior work's restriction to Wikipedia. Across pretraining on Wikipedia and FineWeb-Edu and at multiple model scales, CO-LMLM outperforms prior LMLMs and vanilla LLMs in both perplexity and factual precision. At 360M scale, this includes lower perplexity than models pretrained on 40x more data, and SimpleQA-verified performance that is in line with gpt-4o-mini and higher than Claude Sonnet 4.5.

Chinese Translation

有限记忆语言模型（LMLMs）在预训练期间将事实知识外部化到知识库（KB）中，而不是将其记忆在权重中。在生成过程中，模型根据需要从知识库中获取知识。这个新近提出的范式提供了多种优势，包括超越传统大型语言模型（LLMs）的知识控制能力。我们提出了共查询有限记忆语言模型（CO-LMLM），其中知识库将连续键与文本知识值配对，这与之前对关系知识库和查询的依赖有显著不同。CO-LMLM以最低的成本生成灵活的向量查询，同时将可读和可归属的检索知识整合到其生成中。我们将这一设计与一个注释管道相结合，该管道标记任意文本中的自由形式事实跨度，消除了之前研究对维基百科的限制。在对维基百科和FineWeb-Edu进行预训练并在多个模型规模下，CO-LMLM在困惑度和事实精确度上均优于之前的LMLMs和普通LLMs。在360M规模下，其困惑度低于在40倍更多数据上预训练的模型，并且在SimpleQA验证的性能上与gpt-4o-mini持平，且高于Claude Sonnet 4.5。

View on arXiv Download PDF AI Translation

cs.CL / 27 / 2607.07708

Accurate, Interdisciplinary and Transparent Structure-property Understanding with Deep Native Structural Reasoning

基于深度本征结构推理的准确、跨学科和透明的结构-性质理解

Tang, Chen, Wang, Yizhou, Wu, Jianyu, Wang, Lintao, Tang, Shixiang, Li, Pengze, Su, Encheng, Yao, Jun, Xiao, Jiabei, Shi, Yuqi, Li, Jielan, Hao, Hongxia, Gao, Zhangyang, Wu, Fang, Fei, Ben, Yue, Xiangyu, Tan, Pan, Zhong, Bozitao, Zhang, Jinouwen, Wang, Aoran, Lu, Yan, Liu, Jiaheng, Ma, Xinzhu, Hong, Liang, Zheng, Mingyue, Torr, Phil, Zhou, Bowen, Ouyang, Wanli, Bai, Lei

Abstract

Structure-property relationships are foundational to biology, chemistry and materials science, where function, reactivity and physical response emerge from spatial, chemical and periodic organization. Mechanistically explaining these relationships requires interpreting structural evidence through scientific principles and physical constraints, from stereochemistry and bonding to symmetry, energetics and periodic order. However, applying artificial intelligence to this process presents a joint challenge of representation and reasoning: models must preserve domain-native structural information while showing how specific evidence supports predictions under these constraints. Here we introduce SciReasoner, a multimodal scientific foundation model for native structural reasoning across proteins, small molecules and inorganic crystals. SciReasoner discretizes coordinates, topologies and periodic connectivities into a unified structure-aware vocabulary, treating structural tokens as addressable evidence units during reasoning. In homology-controlled Gene Ontology prediction, SciReasoner improves Cellular Component annotation for low-homology and orphan-like proteins, increasing $F_{\max}$ from 0.42 to 0.55. In chemistry, it raises single-step retrosynthesis accuracy from 0.63 to 0.72 while generating fragment-level disconnection and precursor-verification traces. In materials science, its representations separate elemental and compound phases and resolve high- and low-band-gap regimes. Across 86 benchmarks, SciReasoner achieves state-of-the-art performance on 67 tasks. Double-blind expert evaluation rates its reasoning traces as preferred or at least comparable to those of a frontier large language model in 98% of cases. By making structure an inspectable substrate for reasoning under scientific constraints, SciReasoner connects accurate prediction with interpretable scientific inference.

Chinese Translation

结构-性质关系是生物学、化学和材料科学的基础，其中功能、反应性和物理响应源于空间、化学和周期性组织。机制性地解释这些关系需要通过科学原理和物理约束来解读结构证据，从立体化学和键合到对称性、能量学和周期性秩序。然而，将人工智能应用于这一过程面临着表示和推理的共同挑战：模型必须保留领域本征的结构信息，同时展示特定证据如何在这些约束下支持预测。在此，我们介绍了SciReasoner，一种用于蛋白质、小分子和无机晶体的本征结构推理的多模态科学基础模型。SciReasoner将坐标、拓扑和周期连接离散化为统一的结构感知词汇，在推理过程中将结构标记视为可寻址的证据单元。在同源性控制的基因本体预测中，SciReasoner提高了低同源性和孤儿样蛋白的细胞组分注释，将$F_{ ext{max}}$从0.42提高到0.55。在化学领域，它将单步逆合成的准确性从0.63提高到0.72，同时生成片段级的断裂和前体验证轨迹。在材料科学中，它的表示能够区分元素相和化合物相，并解析高带隙和低带隙区域。在86个基准测试中，SciReasoner在67个任务上实现了最先进的性能。双盲专家评估中，98%的情况下其推理轨迹被评为优于或至少可与前沿大型语言模型相媲美。通过将结构作为在科学约束下可检视的推理基础，SciReasoner将准确预测与可解释的科学推理连接起来。

View on arXiv Download PDF AI Translation

arXiv Papers

Pelican-VLA 0.5: Attending Before Acting Benefits Generalization

NativeMEM: Native Memory Compression for Long-Horizon Robotic Manipulation

RoboSnap: One-Shot Real-to-Sim Scene Generation for Generalizable Robot Learning and Evaluation

CILC: Cryptographically-secure Inter-agent Loop Closure Candidate Detection for Multi-Agent Collaborative SLAM

Vision Language Action (VLA) Models for Unmanned Aerial Robotics and Bimanual Manipulation: A Review

EvoPlan: Evolutionary Neuro-Symbolic Robot Planning with Spatio-Temporal Guarantees

A Continual Learning Framework for Adaptive Control of Modular Soft Robots

G-PROBE: Cross-FOV Place Recognition and Certainty-Coupled Localization for 3D Point Clouds

CaLiSym: Learning Symplectic Dynamics of Real-World Systems through Structured Canonical Lifts

GemNav: Discrete-Token Visual Robot Navigation using a Multimodal Large Language Model

Dynamic Object Detection and Tracking in Construction: A Fisheye Camera and LiDAR Sensor Fusion Model

Flow-ERD: Agent-type Aware Flow Matching with Entropy-Regularized Distillation for Diverse Traffic Simulation

End-to-End LLM Flight Planning with RAG-based Memory and Multi-modal Coach Agent

SPECTRA: Context-Conditioned Spectral Movement Primitives for Robot Skill Generalization

WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time

Ace! Motion Planning of Professional-Level Table Tennis Serves with a Robot Arm

A Closed-Loop Multi-Agent Framework for Robust Multi-Robot Manipulation

PriGo: Test-Time Primitive Guidance to Diffusion and Flow Policies for Adaptive Robotic Manipulation

GeoProp: Grounding Robot State in Vision for Generalist Manipulation

Compositional Motion Generation from Demonstration with Object-Centric Neural Fields

Learning Spatiotemporal Tubes for Full Class of Signal Temporal Logic Tasks for Control of Unknown Systems under Input Constraints

Disturbance-aware Motion Planning for Over-actuated Underwater Vehicles Exploiting Actuation Redundancy for High-fidelity 3D Reconstruction

Validate the Dream Before You Trust Its Verdict: Admissibility for World-Model Simulators

Manual, Joystick, or Haptic Control? An In Vitro Comparison of Navigation Strategies for Robotic Interventional Neuroradiology Procedures

Programmable Synchronization Graphs for Adaptive and Fault-Tolerant Modular Miniature Robots

TouchWorld: A Predictive and Reactive Tactile Foundation Model for Dexterous Manipulation

Multimodal Voice Activity Projection for Turn-Taking in Social Robots with Voice-Activity-Related Pretrained Encoders

Towards Reliable Aerial Ground Vehicle Collaboration: An Integrated Planning and Autonomy Framework for Field Deployment

HumAIN: Human-Aware Implicit Social Robot Navigation

Behavior Foundations for Quadruped Robots: ABot-C0 Technical Report

PLED-VINS: A Point-Line Event-Based Visual Inertial SLAM for Dynamic Environments

Communicative Efficiency of Single vs. Multi-Axis Robot Neck Motion

Initiation Safety: A Missing Dimension in Generalist-Robot Safety

Immersive Social Interaction with VR and LLM-Assisted Humanoids

GeoGS-SLAM: Geometry-Only Gaussian Splatting for Dense Monocular SLAM

EmbodiedGen V2: An Agentic, Simulation-Ready 3D World Engine for Embodied AI

Agent-Exploitation Affordances: From Basic to Complex Representation Patterns

Smooth Operator: A Real-Time Sampling-Based Algorithm for Kinematic Hand Retargeting

Generating Personalized Lower-Limb Kinematics Across Walking Speeds Using Subject-Conditioned Diffusion

SonoRank: Towards Calibration-Free Real-Time Finger Flexion Detection from Forearm Ultrasound Sequences

Context-Aware Force Estimation for Deformable Tool Manipulation in Robotic Environmental Swabbing via Few-Shot Continual Adaptation

CARLA-GS: Decoupling Representation, Reasoning, and Physics Simulation for Autonomous Driving Corner-Case Synthesis

Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulation

Continuous and large-scale: ELEANOR, the soft architected arm inspired by the elephant trunk

Pixel-Precise Explainable Stress Indexing: A Semantic Segmentation Framework for Disease Severity Quantification in Field Crops

CoFINN: Conservation Flux Informed Neural Networks for Physics Problems Governed by Conservation Laws

AI for Cultural Heritage Textiles: Fine-Tuned Latent Diffusion for Novel Ulos Motif Synthesis

LipSSD: Lipschitz-Constrained Single-Shot Detection for Adversarially Robust Object Detection

MiLSD: A Micro Line-Segment Detector for Resource-Constrained Devices

Do Counterfactually Fair Image Classifiers Satisfy Group Fairness? -- A Theoretical and Empirical Study

Overview of the NLPCC 2026 Shared Task 1: Difficulty-Aware Multilingual and Multimodal Medical Instructional Video Understanding Evaluation

SpaR3D-MoE: Adaptive 3D Spatial Reasoning from Sparse Views Meets Geometry-Inductive Mixture-of-Experts

Dynamic-in-Few-Step: Unifying Dynamic Computation and Few-Step Distillation for Efficient Video Generation

ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities

CoMind: Understanding Collaborative Human Activity from Multiple Minds and Views

SPEAR: A Simulator for Photorealistic Embodied AI Research

A Good Initialization is All You Need for Faithful Visual Attribution

Hardware-aware Graph Neural Networks prunning for embedded event-based vision

URS-Stereo: Uncertainty-Guided Residual Search for Real-Time Stereo Matching

Rail Track Extraction from Rasterized Classified Point Clouds Using a Full-Resolution, Fully Convolutional Recurrent Neural Network

WildCity: A Real-World City-Scale Testbed for Rendering, Simulation, and Spatial Intelligence

Retrieving and Refining Winning Noise Tickets for Diffusion-Based Motion Generation

Gen4U: Unifying Video Generation and Understanding via Diffusion

Geometric Collapse: When Vision Models Fail to Verify Physical Causality

Ensemble Deep Learning Approaches for AI-Altered Video Detection

Video2Reaction: Mapping Video to Audience Reaction Distribution in the Wild

ReMoDEx: A Local-to-Global Relevance-Based Model Decision Explainability Framework for large-Scale Image Datasets

Seeing What Matters: Lesion-Aware High-Resolution Patch Discovery and Fusion for Chest X-ray Report Generation

Smart Scissor: Coupling Spatial Redundancy Reduction and CNN Compression for Embedded Hardware

LoCA: Spatially-Aware Low-Rank Convolutional Adaptation of Vision Foundation Models

Compass: Prostate Cancer Detection Needs Multi-View Context

Bi-PT: Bidirectional Cross-Attention Point Transformers for Four-Chamber Heart Reconstruction from Sparse Cardiac MRI Data

General Incomplete Multimodal Learning via Dynamic Quality Perception

Self-Supervised Pretraining Improves Cross-Site and Cross-Scale Robustness of Point Cloud Leaf-Wood Segmentation

SpiS-GAN: Spiral-Modulated Handwriting Synthesis with Star Operation

HPR-SAM: Hierarchical Probabilistic Representation Learning for Prompt-free SAM-based Medical Image Segmentation

EdgeCompress: Coupling Multidimensional Model Compression and Dynamic Inference for EdgeAI

Ego-Human Motion Prediction with 3D-Aware LLM

SHTA: Semantic Hard Token Correction and Center Alignment for Semi-Supervised Medical Image Segmentation