arXiv Daily Digest

596

Papers

GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation

GigaWorld-1：构建机器人政策评估世界模型的路线图

GigaWorld Team, Ma, Angyuan, Wang, Boyuan, Li, Bohan, Ni, Chaojun, Li, Guo, Huang, Guan, Zhao, Guosheng, Li, Hao, Li, Hengtao, Liu, Jingyu, Lu, Jiwen, Deng, Qiuping, Yu, Tingdong, Xu, Xuancheng, Zhou, Xinyu, Xu, Xiuwei, Chen, Xinze, Wang, Xiaofeng, Tian, Xiaoyu, Wang, Yang, Chang, Yifan, Zhou, Yukun, Ye, Yun, Wu, Zhenyu, Wu, Zhanqian, Zhu, Zheng

Abstract

Evaluating embodied robot foundation models remains a critical bottleneck; unlike large language models efficiently assessed via digital benchmarks, robotic policies require slow, costly real-world rollouts limited by hardware and human supervision, which has driven interest in world models as surrogate policy evaluators, yet the key properties that make a world model reliable for policy assessment remain poorly understood. This work presents a systematic study of world models for robotic policy evaluation and introduces WMBench, a benchmark constructed from real-robot teleoperation data and matched policy rollouts covering diverse manipulation tasks to enable controlled comparisons across model families, action encodings, rollout horizons, and evaluation metrics. Using WMBench, we analyze 7 video world models, 4 action representation schemes, and over 324,000 simulated policy rollouts paired with real robot executions, further enriching our analysis with large-scale community submissions from the CVPR 2026 GigaBrain Challenge, curated synthetic trajectories, and a training videos spanning more than 12,000 hours. Our experiments deliver three core insights: evaluator quality is dominated by long-horizon, action-faithful rollout consistency rather than short-term visual realism; pretraining gains stem not only from data scale but from balancing general world knowledge with robot-specific controllability; and architectural choices including action encoding, memory design, and evaluator-focused post-training strongly determine alignment with real-world robot behavior. Drawing on these results, we derive a practical design roadmap and realize it in \textit{GigaWorld-1}, a world model specially optimized for policy evaluation, and we fully release our code, models, datasets, and toolkits to advance scalable evaluation research for embodied foundation models.

Chinese Translation

评估具身机器人基础模型仍然是一个关键瓶颈；与通过数字基准高效评估的大型语言模型不同，机器人政策需要缓慢且昂贵的现实世界实施，这受到硬件和人类监督的限制，这推动了对世界模型作为替代政策评估者的兴趣，但使世界模型在政策评估中可靠的关键特性仍然不甚明了。本研究对机器人政策评估中的世界模型进行了系统研究，并引入了WMBench，这是一个基于真实机器人遥操作数据和匹配政策实施的基准，涵盖多样的操控任务，以便在模型家族、动作编码、实施视野和评估指标之间进行受控比较。利用WMBench，我们分析了7种视频世界模型、4种动作表示方案以及超过324,000个与真实机器人执行配对的模拟政策实施，并通过来自CVPR 2026 GigaBrain Challenge的大规模社区提交、策划的合成轨迹和超过12,000小时的训练视频进一步丰富我们的分析。我们的实验提供了三个核心见解：评估者质量主要受长视野、动作一致性实施的影响，而非短期视觉现实主义；预训练的收益不仅源于数据规模，还源于平衡通用世界知识与机器人特定可控性；包括动作编码、记忆设计和以评估者为中心的后训练的架构选择在很大程度上决定了与现实世界机器人行为的对齐。基于这些结果，我们推导出一个实用的设计路线图，并在 extit{GigaWorld-1}中实现，该世界模型专门优化用于政策评估，我们全面发布我们的代码、模型、数据集和工具包，以推动具身基础模型的可扩展评估研究。

View on arXiv Download PDF AI Translation

cs.RO / 2 / 2607.02646

EVA-Client: A Unified Data Collection, Inference, and Deployment Framework for Embodied Policies on Real Robots

EVA-Client：一个统一的数据收集、推理和部署框架，用于真实机器人上的具身策略

Yang, Heqing, Yi, Yang, Wang, Liyao, Zhong, Linqing, Yang, Donglin, Wu, Ruipu, Bai, Zitong, Chen, Fengjiao, Zhang, Manyuan, Huang, Linjiang, Liu, Si

Abstract

We present EVA-Client, an open-source framework for deployment, data collection, and evaluation of trained manipulation policies on real robots. Sitting between a policy server and the physical hardware, EVA-Client unifies the real-robot stages of the policy iteration loop within a single codebase. It makes three contributions. First, a component-decoupled architecture in which robot backends, inference strategies, and transport middlewares form an orthogonal grid: adding a robot or a strategy touches only its own layer. Second, inspectable execution through Debug, Collect, and Eval workflows, with modes ranging from open-loop simulation to continuous real-time control. Third, every evaluation run doubles as a data collection, recording full rollouts in training-ready format alongside exhaustive logs and a side-by-side comparison viewer, so each evaluation feeds the next round of training rather than ending as an unrecorded impression. EVA-Client further consolidates major real-time inference strategies, synchronous and asynchronous execution, ACT-style temporal ensembling, Real-Time Chunking, and a naive-async ablation baseline, behind a single configuration surface.

Chinese Translation

我们提出了EVA-Client，这是一个开源框架，用于在真实机器人上部署、收集数据和评估训练好的操作策略。EVA-Client位于策略服务器和物理硬件之间，将策略迭代循环的真实机器人阶段统一在一个代码库中。它有三个主要贡献。首先，采用组件解耦的架构，其中机器人后端、推理策略和传输中间件形成一个正交网格：添加一个机器人或策略只需触及其自身层。其次，通过Debug、Collect和Eval工作流实现可检查的执行，模式范围从开环仿真到连续实时控制。第三，每次评估运行同时作为数据收集，记录完整的训练准备格式的回放，以及详尽的日志和并排比较查看器，因此每次评估为下一轮训练提供数据，而不是作为未记录的印象结束。EVA-Client进一步整合了主要的实时推理策略、同步和异步执行、ACT风格的时间集成、实时分块以及一个简单的异步消融基线，所有这些都通过一个统一的配置界面进行管理。

View on arXiv Download PDF AI Translation

cs.RO / 3 / 2607.02764

MorphQuad: Morphable Quadrotor for Superhuman Maneuverability, Manipulation, and Resiliency

MorphQuad：可变形四旋翼无人机实现超人类的机动性、操控能力和韧性

Pacheco, J. Diaz Peon Gonzalez, Xu, J., Zhao, A., Zhou, H., Navsalkar, A., Scheffer, A., Reddi, A. Malli, Shankar, S., Bao, Y., Tzoumas, V.

Abstract

Infrastructure maintenance, contact-based inspection, and emergency response can benefit from aerial vehicles that act as a flying human hand with extreme maneuverability, manipulation, and resiliency (MMR): maneuverability to fly in arbitrary orientations to reach remote and tight locations; manipulation to point sensors, turn valves, and press tools at arbitrary orientations; resiliency to maintain accurate motion and force control despite disturbances from arbitrary directions, such as wind, ground effects, and friction. Realizing MMR on aerial vehicles requires not only omnidirectional flight; it also requires (I) vectoring of maximum thrust in any direction, to maximize capacity for contact-force application and disturbance rejection, (II) global stability, to enable control over any orientation/position, and (III) compact, standard designs that build upon platforms such as quadrotors to inherit technological know-how. No current aerial vehicle simultaneously enables I--III, due to structural and control limitations that constrain actuation. We present MorphQuad: a morphable quadrotor that enjoys MMR. Key to our approach is a hardware and control co-design: on hardware, we independently articulate each of the four rotor systems via two-axis gimbals; on control, we introduce globally-stable control, and energy-optimal thrust allocation that permits inter-rotor thrust cancellations only to avoid downwash interference and gimbal lock. With fully-onboard autonomy, MorphQuad demonstrates multi-revolution rotation while translating or hovering, for pipe inspection and target tracking (maneuverability); valve turning, perching, and object pressing and pushing with human-level strengths (manipulation); and wind rejection from any direction, even directed to a single rotor, and push-pull recovery (resiliency).

Chinese Translation

基础设施维护、基于接触的检查和紧急响应可以从具备极高机动性、操控能力和韧性（MMR）的空中飞行器中受益：机动性使其能够以任意方向飞行，抵达偏远和狭小的位置；操控能力使其能够在任意方向指向传感器、旋转阀门和按压工具；韧性使其能够在受到来自任意方向的干扰（如风、地面效应和摩擦）时，保持精确的运动和力控制。在空中飞行器上实现MMR不仅需要全向飞行，还需要（I）在任何方向上最大推力的矢量化，以最大化接触力应用和干扰拒绝的能力，（II）全球稳定性，以便控制任意方向/位置，以及（III）基于四旋翼等平台的紧凑标准设计，以继承技术知识。目前没有任何空中飞行器能够同时实现I-III，因为结构和控制的限制约束了驱动能力。我们提出了MorphQuad：一种可变形四旋翼无人机，具备MMR。我们方法的关键在于硬件与控制的协同设计：在硬件方面，我们通过双轴云台独立控制四个旋翼系统；在控制方面，我们引入全球稳定控制和能量最优的推力分配，仅在避免下洗干扰和云台锁定时允许旋翼间的推力抵消。MorphQuad具备完全自主能力，展示了在平移或悬停时的多圈旋转，用于管道检查和目标跟踪（机动性）；以人类水平的力量进行阀门旋转、栖息和物体按压与推动（操控能力）；以及抵御来自任意方向的风，即使是针对单个旋翼的风，以及推拉恢复（韧性）。

View on arXiv Download PDF AI Translation

cs.RO / 4 / 2607.02827

Metallic Ultrasound Waveguides as a Distributed Tactile Sensing Platform for Contact Localization, Force Estimation, and Material Class Discrimination

金属超声波导作为接触定位、力估计和材料分类的分布式触觉传感平台

Rosakis, Alexandros, Tamborini, Alessio, Fakhoury, Basile, Bailey, Cole, Gharib, Morteza

Abstract

Tactile sensing is central to how robotic systems interact with the real world, yet current solutions face a tradeoff between sensing area and system complexity. This work investigates metallic ultrasound waveguides as distributed tactile sensors fully interrogated from a single proximal transducer. Using cylindrical indenters, we characterized the acoustic response to single and multi-point contacts with varying forces and contact materials. For single point indentation, the applied force was well captured by a linear relationship with the ratio of the reflection to transmission coefficients (F = a * R/T) across all nine tested materials (R2 >= 0.95). The calibration slope, a, correlated strongly with the material's effective contact modulus (log--log Pearson r=-0.98). The reflected energy partition was found to be a load-independent parameter related to the contacting material's properties, enabling material classification independent of force. For the two-indenter experiment, both contact forces were recovered from the waveguide signal and were in close agreement with reference load cell measurements (contact 1, R2 = 0.97; contact 2, R2=0.95). The approach was extended to two-dimensional metallic sheets, confirming both contact localization and material-dependent effects. Overall, these results validate metallic waveguides as a robust platform for distributed tactile sensing, providing contact localization, force estimation, and material-class discrimination for the contacting body.

Chinese Translation

触觉传感是机器人系统与现实世界互动的核心，然而当前的解决方案在传感区域和系统复杂性之间存在权衡。本研究探讨了金属超声波导作为分布式触觉传感器的应用，能够通过单个近端传感器进行全面探测。使用圆柱形压头，我们对不同力和接触材料下的单点和多点接触的声学响应进行了表征。对于单点压入，施加的力与反射系数与透射系数的比值之间呈线性关系（F = a * R/T），在所有九种测试材料中均表现良好（R2 >= 0.95）。校准斜率a与材料的有效接触模量呈强相关（对数-对数皮尔逊相关系数 r=-0.98）。反射能量分配被发现是一个与载荷无关的参数，与接触材料的特性相关，从而实现了独立于力的材料分类。在双压头实验中，两个接触力均从波导信号中恢复，并与参考负载传感器的测量结果高度一致（接触1，R2 = 0.97；接触2，R2=0.95）。该方法扩展到二维金属板，确认了接触定位和材料依赖效应。总体而言，这些结果验证了金属波导作为分布式触觉传感的稳健平台，能够提供接触定位、力估计和接触物体的材料分类。

View on arXiv Download PDF AI Translation

cs.RO / 5 / 2607.02840

TACO: TActile World Model as a Self-COrrector forScalable VLA Post-Training

TACO：作为可扩展 VLA 后训练的自我修正触觉世界模型

Liu, Shengbang, Jia, Yueru, Yan, Yuyang, Liu, Jiaming, Zhang, Xinran, Feng, Qiuxuan, Guo, Yandong, Zhou, Shiji, Shi, Boxin, Zhang, Shanghang

Abstract

Vision-Language-Action (VLA) models have shown promising generalization in robotic manipulation, but they still struggle with contact-rich tasks, where minor contact perturbations can cause unrecoverable failures that are hard to detect from vision alone. Since these failures are localized rather than task-level semantic errors, tactile-aware corrective post-training offers an efficient way to improve recovery. However, scaling such supervision through human intervention is costly. Recent works have explored world models to synthesize imagined rollouts for policy improvement, but vision-only world models may produce visually plausible yet contact-inconsistent trajectories. We therefore introduce TACO, a tactile-aware world-model-driven framework for scalable VLA post-training in contact-rich manipulation. Given real robot rollouts, TACO follows a Recognize-Imagine-Label loop with a tactile-aware world model: a unified progress-action model recognizes failure-adjacent states using progress estimates, a visuo-tactile generation model imagines local correction segments, and the progress-action model labels them with executable corrective actions. To incorporate tactile corrective supervision into VLA post-training, TACO combines knowledge-insulated tactile adaptation with advantage-conditioned training, enabling the policy to learn from imagined corrections without degrading pretrained visual-language priors. These components enable TACO to convert real-world failures into imagined visuo-tactile corrections for iterative VLA post-training. Experiments on real-world contact-rich manipulation tasks show that TACO achieves 44% absolute success rate improvement over the base policy and 32% over the policy without knowledge-insulated tactile adaptation.

Chinese Translation

视觉-语言-动作（VLA）模型在机器人操作中展现了良好的泛化能力，但在接触丰富的任务中仍然面临挑战，其中微小的接触扰动可能导致无法恢复的失败，而这些失败仅通过视觉很难检测到。由于这些失败是局部的，而不是任务级的语义错误，因此基于触觉的纠正后训练提供了一种有效的改进恢复的方法。然而，通过人工干预来扩展这种监督是成本高昂的。最近的研究探索了世界模型，以合成想象中的回滚以改善策略，但仅基于视觉的世界模型可能会产生视觉上合理但接触不一致的轨迹。因此，我们引入了 TACO，一个基于触觉的世界模型驱动的框架，用于在接触丰富的操作中进行可扩展的 VLA 后训练。给定真实的机器人回滚，TACO 采用识别-想象-标记循环，结合触觉感知的世界模型：一个统一的进展-动作模型使用进展估计来识别失败相邻状态，一个视觉-触觉生成模型想象局部修正段，而进展-动作模型则为这些段标记可执行的纠正动作。为了将触觉纠正监督纳入 VLA 后训练，TACO 结合了知识隔离的触觉适应与优势条件训练，使策略能够从想象中的修正中学习，而不会降低预训练的视觉-语言先验。这些组件使 TACO 能够将现实世界的失败转化为想象中的视觉-触觉修正，以进行迭代的 VLA 后训练。在真实的接触丰富操作任务上的实验表明，TACO 相较于基础策略实现了 44% 的绝对成功率提升，相较于没有知识隔离触觉适应的策略提升了 32%。

View on arXiv Download PDF AI Translation

cs.RO / 6 / 2607.02841

CLEAR: Closed-Loop Reinforcement Learning at Scale for End-to-End Autonomous Driving

CLEAR：大规模闭环强化学习用于端到端自动驾驶

Shi, Yunxiao, Cai, Hong, Ghavamzadeh, Mohammad, Porikli, Fatih

Abstract

End-to-end autonomous driving (E2E-AD) aims to directly map raw sensor information to driving actions. Recently, with the rapid advancement of multi-modal large language models (MLLMs), researchers have proposed the paradigm of Vision-Language-Action (VLA) models for E2E-AD, where it seeks to integrate visual perception, language understanding and action prediction within a single policy. However, existing VLA-based policies largely adopts imitation learning, where it only learns to drive by optimizing distance-based metrics w.r.t. logged expert trajectories. Such distribution shift between open-loop training and closed-loop inference leads to suboptimal performance in closed-loop planning. To close this gap, we present CLEAR, a system that enables closed-loop training using Reinforcement Learning (RL) at scale for E2E-AD. We propose to learn a novel residual waypoint policy around the waypoint prior from pretrained VLA policies, effectively harnessing the knowledge within. On another front, one of the key challenges to scale up RL for vision-based policies is the number of parallel simulation environments since RL is data hungry. To that end, we design a heterogeneous pipeline that places the simulator and the VLA learner on distinct compute groups, which allows us to dramatically increase the number of simulation environments running in parallel while avoiding resource contention and maintaining training stability. We show that with a simple reward, CLEAR significantly outperforms previous methods and sets new state-of-the-art performance on the challenging benchmarks of CARLA longest6 v2 and Bench2Drive.

Chinese Translation

端到端自动驾驶（E2E-AD）旨在将原始传感器信息直接映射到驾驶动作。最近，随着多模态大型语言模型（MLLMs）的快速发展，研究人员提出了视觉-语言-动作（VLA）模型的范式用于E2E-AD，试图在单一策略中整合视觉感知、语言理解和动作预测。然而，现有的基于VLA的策略主要采用模仿学习，仅通过优化与记录的专家轨迹相关的基于距离的指标来学习驾驶。这种开放式训练与闭环推理之间的分布转移导致了闭环规划中的次优性能。为了解决这一问题，我们提出了CLEAR，一个能够大规模使用强化学习（RL）进行闭环训练的系统，用于E2E-AD。我们建议围绕预训练的VLA策略中的航点先验学习一种新颖的残差航点策略，有效利用其中的知识。在另一个方面，扩展基于视觉的策略的RL的关键挑战之一是并行仿真环境的数量，因为RL对数据的需求量很大。为此，我们设计了一个异构管道，将仿真器和VLA学习者放置在不同的计算组中，这使我们能够显著增加并行运行的仿真环境数量，同时避免资源争用并保持训练稳定性。我们展示了通过简单的奖励，CLEAR显著优于以前的方法，并在CARLA longest6 v2和Bench2Drive的挑战基准上设定了新的最先进性能。

View on arXiv Download PDF AI Translation

cs.RO / 7 / 2607.02845

Differential Amplifier-Inspired AmpAttention for Multi-View Robotic Manipulation

基于差分放大器的AmpAttention在多视角机器人操控中的应用

Yang, Jin, Wei, Ping, Zheng, Nanning

Abstract

Multi-view robotic manipulation methods with the attention mechanism have recently achieved significant progress in both training efficiency and task performance. However, the inherent redundancy, occlusion, and viewpoint dependency in robotic view images often lead to severe attention drift. To address this challenge, we propose AmpAttention, a novel attention mechanism inspired by differential amplifiers in analog circuits. It aims to suppress attention noise and capture high signal-to-noise ratio signals for more reliable perception. Based on this, we introduce the RVAF model, which integrates task-guided intra-view and inter-view AmpAttention. Compared to previous state-of-the-art methods, RVAF achieves the optimal average success rate across 18 RLBench tasks (249 variations) while reducing training time by 33.3\%. RVAF also demonstrates strong potential in real-world high-precision tasks, exemplified by its ability to pick up a dart and accurately insert it into the red bullseye. Furthermore, we extend RVAF to RVAF++ by incorporating the SAM2 image encoder. RVAF++ achieves substantial gains on high-precision tasks, achieving a 91\% success rate on the `insert peg' task. More qualitative results are provided at the anonymous project website https://anonymous.4open.science/w/RVAF-Anonymization.

Chinese Translation

近年来，采用注意力机制的多视角机器人操控方法在训练效率和任务性能方面取得了显著进展。然而，机器人视图图像中的固有冗余、遮挡和视角依赖性常常导致严重的注意力漂移。为了解决这一挑战，我们提出了AmpAttention，这是一种受模拟电路中差分放大器启发的新型注意力机制。其旨在抑制注意力噪声并捕获高信噪比信号，以实现更可靠的感知。在此基础上，我们引入了RVAF模型，该模型集成了任务引导的视内和视间AmpAttention。与之前的最先进方法相比，RVAF在18个RLBench任务（249种变体）中实现了最佳平均成功率，同时将训练时间减少了33.3%。RVAF在现实世界高精度任务中也展现出强大的潜力，例如其能够准确地拾起飞镖并将其插入红色靶心。此外，我们通过结合SAM2图像编码器将RVAF扩展为RVAF++。RVAF++在高精度任务上取得了显著提升，在“插入插头”任务中达到了91%的成功率。更多定性结果可在匿名项目网站 https://anonymous.4open.science/w/RVAF-Anonymization 中查看。

View on arXiv Download PDF AI Translation

cs.RO / 8 / 2607.02865

DREAMSTEER: Latent World Models Can Steer VLA Policies During Deployment Without Any Finetuning

DREAMSTEER：潜在世界模型能够在部署期间引导VLA策略而无需任何微调

Cui, Hanchen, Arnaud, Sergio, Majumdar, Arjun, Dugas, Daniel, Aljalbout, Elie, Desingh, Karthik, Jatavallabhula, Krishna Murthy, Meier, Franziska

Abstract

Pretrained vision-language-action (VLA) policies show promising zero-shot generalization, but often fail under deployment-time distribution shift, leading to decreased robustness and inconsistent instruction following. While prior work commonly tackles this by finetuning on in-distribution data, it assumes demonstrations collected on tasks in the target environment. In this work, we propose DREAMSTEER, a deployment-time steering framework for pretrained VLAs without any finetuning or parameter modifications. The key insight in DREAMSTEER is to leverage a latent world model and a value model to steer pretrained VLA policies. During deployment, DREAMSTEER samples candidate action chunks from a VLA policy and predefined motion primitives, imagines their outcomes using an action-conditioned latent world model, and ranks the imagined trajectories with a language-conditioned value model. Across four real-world manipulation benchmarks with unseen objects, DREAMSTEER improves task success rate from 23.75% to 66.25% and instruction-following accuracy from 38.75% to 56.25% over the base VLA policy.

Chinese Translation

预训练的视觉-语言-动作（VLA）策略显示出良好的零-shot泛化能力，但在部署时常常因分布变化而表现不佳，导致鲁棒性下降和指令遵循不一致。虽然之前的研究通常通过在目标环境中收集的任务上进行微调来解决这一问题，但这假设了在目标环境中收集的示范。在本研究中，我们提出了DREAMSTEER，一种在部署时引导预训练VLA的框架，无需任何微调或参数修改。DREAMSTEER的关键见解是利用潜在世界模型和价值模型来引导预训练的VLA策略。在部署期间，DREAMSTEER从VLA策略和预定义的运动原语中采样候选动作块，使用动作条件的潜在世界模型想象其结果，并通过语言条件的价值模型对想象的轨迹进行排序。在四个包含未见物体的真实世界操作基准测试中，DREAMSTEER将任务成功率从23.75%提高到66.25%，将指令遵循准确率从38.75%提高到56.25%，相较于基础VLA策略。

View on arXiv Download PDF AI Translation

cs.RO / 9 / 2607.02884

Continuous-Time Gaussian Belief Trees for Motion Planning

用于运动规划的连续时间高斯信念树

Mazouz, Rayan, Ho, Qi Heng, Sunberg, Zachary N., Lahijanian, Morteza

Abstract

We address sampling-based motion planning for continuous-time stochastic systems under process and measurement uncertainty, with probabilistic guarantees on safety and performance. The robot dynamics are modeled as a continuous-time linear stochastic differential equation, while sensor measurements arrive at discrete time instants. We derive an offline hybrid belief propagation model in which the belief evolves according to continuous-time ODEs between measurements and undergoes discrete Kalman filter update jumps at measurement times. To ensure safety, we introduce a belief-barrier-function-based safety checker for segment-level probabilistic verification. This enables the planner to certify safety over entire continuous trajectory segments and detect inter-sample chance-constraint violations that are missed by conventional node-based checks. Together, these components provide a principled framework for sampling-based belief planning that accounts for both continuous-time uncertainty propagation and continuous-time safety requirements. We integrate the method with RRT and SST planners and evaluate it across multiple benchmark environments. The results show that the proposed method achieves high success rates and robust enforcement of chance constraints, including in narrow-passage scenarios where discrete-time counterparts fail due to missed inter-sample unsafe behavior.

Chinese Translation

我们针对连续时间随机系统在过程和测量不确定性下的基于采样的运动规划问题进行研究，并提供关于安全性和性能的概率保证。机器人动力学被建模为连续时间线性随机微分方程，而传感器测量在离散时间点到达。我们推导出一种离线混合信念传播模型，其中信念在测量之间根据连续时间常微分方程（ODE）演变，并在测量时刻经历离散的卡尔曼滤波器更新跳跃。为了确保安全性，我们引入了一种基于信念障碍函数的安全检查器，用于段级概率验证。这使得规划器能够对整个连续轨迹段进行安全性认证，并检测传统基于节点的检查所遗漏的样本间机会约束违规行为。这些组件共同提供了一个基于采样的信念规划的原则性框架，考虑了连续时间不确定性传播和连续时间安全要求。我们将该方法与快速随机树（RRT）和状态空间树（SST）规划器集成，并在多个基准环境中进行了评估。结果表明，所提方法在成功率和机会约束的强健执行方面表现出色，包括在狭窄通道场景中，离散时间的对应方法因错过样本间的不安全行为而失败。

View on arXiv Download PDF AI Translation

cs.RO / 10 / 2607.02924

Longitudinal-Motion-Aware Lateral Control for Autonomous Vehicles: A Robust Nonlinear Control Framework

面向纵向运动的自主车辆横向控制：一种鲁棒非线性控制框架

Li, Sixu, Kumar, Nitesh, Ganeshan, Reyshwanth, Rathinam, Sivakumar, Darbha, Swaroop, Zhou, Yang

Abstract

As autonomous vehicles (AVs) operate in increasingly dynamic traffic conditions, lateral control must be performed while longitudinal speed and acceleration vary. Yet many existing lateral controllers rely on constant-speed or operating-point-based assumptions, which can degrade performance during transient longitudinal maneuvers. Moreover, most methods assume precisely known vehicle parameters, despite real-world parametric uncertainties. To address these limitations, this paper presents a longitudinal-motion-aware robust nonlinear lateral control framework for AVs. It first derives a tracking error model that depends on varying longitudinal speed and acceleration. Using this model, feedback linearization is employed to obtain a linear input-output relation for lateral error tracking while embedding longitudinal motion into the control law. The resulting internal dynamics are then analyzed to ensure overall system stability. To address parameter uncertainty, two robust control designs with distinct implementation trade-offs are proposed: (i) a Lyapunov redesign (LR) approach inspired by sliding mode control, and (ii) an incremental nonlinear dynamic inversion (INDI) method. Both are rigorously analyzed and proven to ensure ultimate boundedness, with key robustness-tuning parameters explicitly identified. Simulations demonstrate enhanced tracking accuracy, consistent performance across varying speeds and accelerations, and robustness to model uncertainties, while also examining the effects of the robustness-related parameters. Real-vehicle tests further confirm real-time implementation and practical path-tracking performance on actual hardware.

Chinese Translation

随着自主车辆（AVs）在日益动态的交通条件下运行，横向控制必须在纵向速度和加速度变化的情况下进行。然而，许多现有的横向控制器依赖于恒定速度或基于工作点的假设，这可能在瞬态纵向机动过程中降低性能。此外，大多数方法假设车辆参数是精确已知的，尽管现实世界中存在参数不确定性。为了解决这些局限性，本文提出了一种面向纵向运动的鲁棒非线性横向控制框架。首先推导出一个依赖于变化的纵向速度和加速度的跟踪误差模型。利用该模型，采用反馈线性化方法获得横向误差跟踪的线性输入输出关系，同时将纵向运动嵌入控制律中。随后分析得到的内部动态，以确保整体系统的稳定性。为了解决参数不确定性，提出了两种具有不同实现权衡的鲁棒控制设计：（i）受滑模控制启发的李雅普诺夫重设计（LR）方法，以及（ii）增量非线性动态反演（INDI）方法。对这两种方法进行了严格分析，并证明它们能够确保最终有界性，同时明确识别关键的鲁棒性调节参数。仿真结果表明，跟踪精度提高，在不同速度和加速度下表现一致，并且对模型不确定性具有鲁棒性，同时还考察了与鲁棒性相关参数的影响。实际车辆测试进一步确认了实时实施和在实际硬件上的路径跟踪性能。

View on arXiv Download PDF AI Translation

cs.RO / 11 / 2607.02977

Function-Space Diffusion for Motion Planning

基于函数空间的扩散运动规划

Chang, Zinuo, Chen, Yipu, Park, Byoungwoo, Yu, Hongzhe, Chen, Yongxin

Abstract

Diffusion-based motion planners have demonstrated strong performance in generating diverse and high-quality robot trajectories in cluttered environments with multiple feasible solutions. However, existing approaches typically operate on fixed-length waypoint sequences, making the learned model resolution-dependent, thereby preventing zero-shot generalization across resolutions. In this work, we propose Function-Space Diffusion for Motion Planning (FSD-MP), a diffusion-based motion planner that models trajectories as continuous functions and performs diffusion directly in function space, achieving discretization-invariant trajectory generation. We define a mode-wise forward process in the spectral domain, driven by Gaussian noise with a Mat\'ern-type covariance, and parameterize the reverse process with a boundary-compatible Discrete Sine Transform-based Fourier Neural Operator (DST-FNO) that preserves start-goal constraints across resolutions. We evaluate FSD-MP on 2D point robot and 7-DoF Franka manipulator planning benchmarks. Our method achieves competitive planning performance at the training resolution and generalizes zero-shot across resolutions up to 16$\times$ higher, preserving consistent planning behavior without retraining. These results demonstrate that function-space diffusion provides an effective framework for discretization-invariant motion planning.

Chinese Translation

基于扩散的运动规划器在生成多种多样且高质量的机器人轨迹方面表现出色，尤其是在具有多种可行解的复杂环境中。然而，现有的方法通常在固定长度的路径点序列上运行，这使得所学习的模型依赖于分辨率，从而阻碍了跨分辨率的零-shot 泛化。在本研究中，我们提出了基于函数空间的扩散运动规划（Function-Space Diffusion for Motion Planning, FSD-MP），这是一种将轨迹建模为连续函数并直接在函数空间中进行扩散的基于扩散的运动规划器，实现了不依赖于离散化的轨迹生成。我们在谱域中定义了一种模式导向的前向过程，该过程由具有 Matérn 类型协方差的高斯噪声驱动，并使用边界兼容的基于离散正弦变换的傅里叶神经算子（Discrete Sine Transform-based Fourier Neural Operator, DST-FNO）对反向过程进行参数化，以保持跨分辨率的起始-目标约束。我们在二维点机器人和7自由度的Franka操纵器规划基准上评估了FSD-MP。我们的方法在训练分辨率下实现了竞争性的规划性能，并在高达16倍的分辨率上实现了零-shot 泛化，保持了一致的规划行为而无需重新训练。这些结果表明，函数空间扩散为不依赖于离散化的运动规划提供了有效的框架。

View on arXiv Download PDF AI Translation

cs.RO / 12 / 2607.03017

Beyond Heuristics: A Standardized Real2Sim Pipeline for Physical Human Robot Interaction in Human-in-the-Loop Simulation

超越启发式方法：用于人机交互的标准化Real2Sim管道在闭环仿真中的应用

Yang, Chengyuan, Wang, Yifan, Tan, Chun Kwang, Chan, Sherwin Stephen, Wang, Youlong, Yan, Xiaoyue, Li, Lei, Ang, Wei Tech

Abstract

The aging global population drives demand for assistive robots, yet the safety risks and costs of physical testing make Human-in-the-Loop (HITL) simulation an attractive alternative. Its fidelity for coupled systems, however, is limited by interaction models whose impedance parameters are tuned heuristically rather than identified from data. We present a Real2Sim pipeline that identifies the coupled Physical Human-Robot Interaction (pHRI) dynamics of a pelvis--strap interface on an overground mobile balance assistant. The interface is modeled as a 6-DoF viscoelastic mechanism whose 12 directional stiffness and damping parameters are identified per subject via Covariance Matrix Adaptation Evolution Strategy (CMA-ES), using the user's ``Safe \& Comfortable'' feedback as a reproducible operating point that resolves harness-tightness ambiguity across anthropometrics. An intraclass-correlation analysis over a five-subject cohort separates shareable from subject-specific parameters, yielding a set of prior parameters derived from the existing data. Deploying this prior configures a previously unseen subject by refining only 5 of the 12 parameters. The calibrated model then reproduces the real interaction envelope and induces biomechanically accurate gait adaptations in the Human Digital Twin (HDT). Overly compliant and overly stiff settings, by contrast, fail as extreme settings, confirming a correct operating point that no heuristic tuning procedure can reliably select. The pipeline thus improves HITL simulation fidelity and supports the Human Digital Twin as a predictive tool for pre-clinical verification of personalized controllers.

Chinese Translation

全球老龄化人口推动了对辅助机器人需求的增加，但物理测试的安全风险和成本使得人机闭环（HITL）仿真成为一种具有吸引力的替代方案。然而，其对耦合系统的逼真度受到交互模型的限制，这些模型的阻抗参数是通过启发式方法调整的，而不是通过数据识别的。我们提出了一种Real2Sim管道，该管道识别了在地面移动平衡辅助设备上，骨盆-带接口的耦合物理人机交互（pHRI）动态。该接口被建模为一个6自由度的粘弹性机制，其12个方向的刚度和阻尼参数通过协方差矩阵自适应进化策略（CMA-ES）针对每个受试者进行识别，利用用户的“安全与舒适”反馈作为可再现的操作点，以解决跨人类测量的安全带紧度模糊性。对五名受试者的组内相关性分析将可共享参数与特定于受试者的参数分开，从而得出一组基于现有数据的先验参数。通过部署这些先验参数，仅需调整12个参数中的5个即可配置一个之前未见过的受试者。经过校准的模型能够重现真实的交互范围，并在人体数字双胞胎（HDT）中引发生物力学上准确的步态适应。相比之下，过于柔顺和过于僵硬的设置作为极端设置失败，确认了一个正确的操作点，而任何启发式调整程序都无法可靠地选择该点。因此，该管道提高了HITL仿真的逼真度，并支持人体数字双胞胎作为个性化控制器的临床前验证的预测工具。

View on arXiv Download PDF AI Translation

cs.RO / 13 / 2607.03027

DRBA: Dynamic Robotic Balance Assistant -- An assist-as-needed gait and balance rehabilitation robot for versatile training

DRBA：动态机器人平衡辅助器——一种按需辅助的步态和平衡康复机器人，用于多样化训练

Wang, Yifan, Li, Li, Wang, Youlong, Yang, Chengyuan, Chan, Sherwin Stephen, Chen, Jiaye, Yan, Xiaoyue, Wang, Hao, Gong, Xuesheng, Lin, Jun, Hu, Hongping, Ang, Wei Tech

Abstract

The decline of human balance control due to aging and pathological conditions increases fall risk, a major concern in geriatric care and rehabilitation. Gait training is essential for balance recovery, enhancing walking ability and postural control. However, existing overground robotic gait trainers have limitations: body weight support systems are bulky and impractical for daily use, while end-effector-based systems often compromise transparency, altering natural gait dynamics. This paper presents the Dynamic Robotic Balance Assistant (DRBA), a novel gait trainer providing assist-as-needed body weight and balance support for various training scenarios. DRBA integrates a 3-degree-of-freedom (3-DoF) robotic arm for pelvic support with flexible motion, a compact sit-to-stand assistance module, and user-following and fall detection algorithms to ensure minimal interference and responsive support. Experimental results demonstrated high transparency, with minimal impact on natural gait dynamics. A patient trial with nine elderly patients with varying medical conditions and balance impairments (ranging from severe to mild) further validated DRBA's effectiveness. The results showed that DRBA-assisted training increased step length and walking speed compared to therapist-assisted gait training. Additionally, DRBA enabled users to perform tasks beyond their unaided ability, expanding rehabilitation possibilities. These findings highlight DRBA's potential to enhance rehabilitation outcomes by facilitating higher training intensity and enabling task-oriented exercises.

Chinese Translation

由于衰老和病理状况导致的人类平衡控制能力下降增加了跌倒风险，这是老年护理和康复中的主要关注点。步态训练对平衡恢复、提高行走能力和姿势控制至关重要。然而，现有的地面机器人步态训练器存在局限性：身体重量支持系统笨重且不适合日常使用，而基于末端执行器的系统往往会妨碍透明性，改变自然步态动力学。本文提出了动态机器人平衡辅助器（Dynamic Robotic Balance Assistant，DRBA），这是一种新型步态训练器，为各种训练场景提供按需的身体重量和平衡支持。DRBA集成了一个具有灵活运动的3自由度（3-DoF）机器人臂用于骨盆支持，一个紧凑的坐立辅助模块，以及用户跟随和跌倒检测算法，以确保最小干扰和响应支持。实验结果表明，DRBA具有高透明性，对自然步态动力学的影响最小。对九名具有不同医疗状况和平衡障碍（从严重到轻微）的老年患者的临床试验进一步验证了DRBA的有效性。结果显示，与治疗师辅助的步态训练相比，DRBA辅助训练增加了步幅和行走速度。此外，DRBA使用户能够执行超出其独立能力的任务，拓展了康复的可能性。这些发现突显了DRBA通过促进更高的训练强度和实现以任务为导向的练习来增强康复效果的潜力。

View on arXiv Download PDF AI Translation

cs.RO / 14 / 2607.03146

Exp2VLA: Enabling Vision-Language-Action for Drone Navigation from Expert Demonstrations

Exp2VLA：基于专家示范的无人机导航视觉-语言-动作实现

Dang, Van Huyen, Rajendran, Kabilesh, Sayar, Erdi, Kayacan, Erdal

Abstract

Vision-language-action (VLA) models open a new path toward intuitive robot control by directly linking perception, language, and action in a single end-to-end framework. Yet for UAVs, practical adoption remains difficult because existing solutions are either computationally heavy or insufficiently capable in complex environments. In this work, we propose a practical expert-distillation pipeline (Exp2VLA) for language-conditioned drone navigation. The core idea is to distill expert behavior, obtained from reinforcement learning, teleoperation, or other controllers, into training data that can be used to fine-tune compact VLA models. This allows existing control strategies to be transferred into a unified language-guided navigation model, reducing manual system integration and lowering the barrier for deploying new robot behaviors. Experiments in both sim-to-sim and simulation-in-the-loop settings across multi-object scenes show that the fine-tuned models can handle varied semantic commands and generalize to unseen target compositions. The proposed framework demonstrates how expert-policy distillation can help mechatronic systems move from specialized control modules toward more flexible and reusable robot intelligence.

Chinese Translation

视觉-语言-动作（VLA）模型为直观的机器人控制开辟了一条新路径，通过在单一的端到端框架中直接连接感知、语言和动作。然而，对于无人机（UAV）而言，实际应用仍然困难，因为现有解决方案要么计算负担沉重，要么在复杂环境中能力不足。在本研究中，我们提出了一种实用的专家蒸馏管道（Exp2VLA），用于语言条件下的无人机导航。其核心思想是将从强化学习、遥操作或其他控制器中获得的专家行为蒸馏为可用于微调紧凑型VLA模型的训练数据。这使得现有控制策略能够转移到统一的语言引导导航模型中，从而减少手动系统集成，并降低部署新机器人行为的门槛。在多目标场景的模拟到模拟（sim-to-sim）和环路内仿真（simulation-in-the-loop）设置中的实验表明，微调后的模型能够处理多样的语义指令，并对未见过的目标组合进行泛化。所提出的框架展示了专家策略蒸馏如何帮助机电系统从专用控制模块向更灵活和可重用的机器人智能转变。

View on arXiv Download PDF AI Translation

cs.RO / 15 / 2607.03155

Hope for the Best, Prepare for the Worst: Occlusion-Aware Contingency Planning for Autonomous Vehicles

抱有希望，做好准备：面向遮挡的自主车辆应急规划

Nyberg, Truls, Gautier, Anna, Tumova, Jana

Abstract

The deployment of autonomous vehicles in urban environments introduces significant safety challenges, particularly in scenarios with occlusions, where critical traffic participants may be hidden from view. Recent accidents involving driverless vehicles highlight the importance of motion planners that explicitly addresses the risks posed by occlusions. In this work, we propose a formal, occlusion-aware trajectory planning framework that guarantees collision avoidance even when there are possible hidden traffic participants. Building on our previous methods that apply reachability analysis to sequentially determine the possible states of hidden traffic participants, we integrate a tree-based motion planner capable of reasoning over future observations and the absence thereof. This approach reduces conservativeness while maintaining safety guarantees. We demonstrate the effectiveness of our framework in a challenging simulated occluded scenario, showing that it pro-actively and efficiently guarantees collision-avoidance.

Chinese Translation

在城市环境中部署自主车辆带来了显著的安全挑战，特别是在存在遮挡的场景中，关键交通参与者可能会被隐藏。最近涉及无人驾驶车辆的事故突显了运动规划器的重要性，这些规划器明确应对遮挡带来的风险。在本研究中，我们提出了一种正式的、面向遮挡的轨迹规划框架，该框架保证即使在可能存在隐藏交通参与者的情况下也能避免碰撞。基于我们之前的方法，该方法应用可达性分析来顺序确定隐藏交通参与者的可能状态，我们整合了一种基于树的运动规划器，能够对未来的观察及其缺失进行推理。这种方法在保持安全保证的同时减少了保守性。我们在一个具有挑战性的模拟遮挡场景中展示了我们框架的有效性，证明其能够主动且高效地保证碰撞避免。

View on arXiv Download PDF AI Translation

cs.RO / 16 / 2607.03163

Beyond Point-Attached Semantics: Object-Centric Semantic Fields for Generalizable Manipulation

超越点附着语义：面向可推广操作的对象中心语义场

Sun, Zheng, Zhang, Lerong, Li, Zhihao, Li, Zhuo, Rouxel, Quentin, Chen, Fei

Abstract

Generalizable robot manipulation requires stable 3D understanding of functional object parts, such as handles, tool heads, openings, and graspable regions. Raw point clouds provide geometry but lack explicit part semantics, and their sampled points vary with viewpoint, sensor configuration, and object instance. Existing 2D feature lifting and discrete 3D point-wise features enrich point clouds with semantics, but the resulting features remain attached to observation-dependent samples. We propose an object-centric continuous semantic field that conditions on an object point cloud and reads part-aware semantic embeddings at explicit 3D query locations. The field is trained from part-annotated object models and then frozen to generate semantic point clouds as object-level conditioning for manipulation policies. Experiments on RoboTwin simulation tasks and real-world bimanual object manipulation show that our representation provides more stable functional-part cues and improves policy performance over raw point-cloud, 2D feature lifting, and 3D point-wise feature baselines. Project Page: \href{https://zainzh.github.io/beyond-point-attached-semantics}{https://zainzh.github.io/beyond-point-attached-semantics}.

Chinese Translation

可推广的机器人操作需要对功能性物体部件（如手柄、工具头、开口和可抓取区域）进行稳定的三维理解。原始点云提供几何信息，但缺乏明确的部件语义，并且其采样点会因视角、传感器配置和物体实例而有所不同。现有的二维特征提升和离散三维点特征虽然为点云提供了语义，但生成的特征仍然依赖于观察相关的样本。我们提出了一种对象中心的连续语义场，该语义场以物体点云为条件，并在明确的三维查询位置读取部件感知的语义嵌入。该语义场从带有部件注释的物体模型中进行训练，然后被冻结，以生成作为操作策略对象级条件的语义点云。在RoboTwin仿真任务和真实双手物体操作的实验表明，我们的表示提供了更稳定的功能性部件线索，并在政策性能上优于原始点云、二维特征提升和三维点特征基线。项目页面：{https://zainzh.github.io/beyond-point-attached-semantics}.

View on arXiv Download PDF AI Translation

cs.RO / 17 / 2607.03182

AnchorVLA: Bridging Discrete Decisions and Continuous Trajectories for Vision-Language-Action Planning

AnchorVLA：桥接离散决策与连续轨迹的视觉-语言-行动规划

Liu, Qi, Li, Yabei, Wang, Hongsong, Zhang, Heng, He, Lei

Abstract

Autonomous driving planning requires translating navigation intent, traffic rules, dynamic interactions, and language instructions into executable continuous trajectories. Vision-Language-Action models have been introduced into driving planning to improve long-tail generalization, commonsense reasoning, high-level semantic understanding, and explainability. However, existing VLA planners mainly follow planning-head-based trajectory prediction or full-trajectory autoregressive generation. The former only weakly constrains continuous trajectory generation with VLA reasoning, while the latter relies on long sequences of low-information-density coordinate tokens, making semantic-action alignment difficult and leading to discretization errors and inefficient inference. To address these limitations, we propose AnchorVLA, a hierarchical decision-anchored VLA planning framework that uses trajectory-pattern anchors as an explicit interface between high-level VLA reasoning and continuous trajectory execution. Specifically, Decision-as-Anchor Representation represents behavior-level driving decisions with anchor tokens, each encoding an entire local motion pattern rather than a single coordinate point. Decision-Anchored Residual Flow then generates fine-grained continuous trajectories in the selected anchor-defined residual space, capturing multi-modal execution refinements after high-level decision making. By reasoning over compact and semantically meaningful anchors instead of autoregressively generating waypoint sequences, AnchorVLA preserves LLM-based decision making while improving inference efficiency, semantic-action alignment, and continuous generation flexibility. Experiments on the Bench2Drive closed-loop benchmark show that AnchorVLA achieves a state-of-the-art Success Rate of 77.28 and a competitive Driving Score of 89.92.

Chinese Translation

自主驾驶规划需要将导航意图、交通规则、动态交互和语言指令转化为可执行的连续轨迹。视觉-语言-行动（Vision-Language-Action, VLA）模型已被引入到驾驶规划中，以改善长尾泛化、常识推理、高层次语义理解和可解释性。然而，现有的 VLA 规划器主要遵循基于规划头的轨迹预测或全轨迹自回归生成。前者仅通过 VLA 推理对连续轨迹生成施加弱约束，而后者依赖于低信息密度坐标标记的长序列，使得语义-行动对齐变得困难，并导致离散化误差和低效推理。为了解决这些局限性，我们提出了 AnchorVLA，一个层次化决策锚定的 VLA 规划框架，利用轨迹模式锚作为高层次 VLA 推理与连续轨迹执行之间的显式接口。具体而言，决策作为锚表示（Decision-as-Anchor Representation）通过锚标记表示行为层面的驾驶决策，每个锚标记编码整个局部运动模式，而不是单个坐标点。决策锚定残差流（Decision-Anchored Residual Flow）则在选定的锚定义残差空间中生成细粒度的连续轨迹，捕捉高层决策后多模态执行的细化。通过对紧凑且语义丰富的锚进行推理，而不是自回归生成路径点序列，AnchorVLA 保留了基于大语言模型（LLM）的决策制定，同时提高了推理效率、语义-行动对齐和连续生成的灵活性。在 Bench2Drive 闭环基准上的实验表明，AnchorVLA 达到了 77.28 的最先进成功率和 89.92 的竞争性驾驶得分。

View on arXiv Download PDF AI Translation

cs.RO / 18 / 2607.03204

Layout-independent actuation allocator for fin-actuated marine robots

独立于布局的鳍驱动海洋机器人驱动分配器

Hamamatsu, Yuya, Kruusmaa, Maarja, Ristolainen, Asko

Abstract

In this study, we propose a layout-independent control allocator capable of zero-shot deployment across diverse actuator configurations. The proposed method utilizes a learning pipeline that integrates a Graph Neural Network (GNN) and a Transformer to represent the robot's geometric layout as a graph, along with a Mixture Density Network (MDN) to predict multi-modal control command distributions. Furthermore, by incorporating a differentiable physics surrogate model, we achieve command refinement during inference to minimize target wrench tracking error and energy consumption. A single generalized model using randomly generated actuator layout data demonstrated high trajectory tracking performance on different actuator layout robots outside the training distribution. Additionally, in real-world pool experiments, our approach achieved performance nearly equivalent to conventional controllers designed to specific layouts.

Chinese Translation

在本研究中，我们提出了一种独立于布局的控制分配器，能够在多种驱动器配置中实现零样本部署。所提方法利用一个学习管道，结合图神经网络（Graph Neural Network, GNN）和变换器（Transformer），将机器人的几何布局表示为图，同时使用混合密度网络（Mixture Density Network, MDN）来预测多模态控制命令分布。此外，通过引入可微分的物理代理模型，我们在推理过程中实现了命令的精细调整，以最小化目标扭矩跟踪误差和能量消耗。使用随机生成的驱动器布局数据的单一通用模型在训练分布之外的不同驱动器布局机器人上展示了高轨迹跟踪性能。此外，在实际的水池实验中，我们的方法实现了与针对特定布局设计的传统控制器几乎相当的性能。

View on arXiv Download PDF AI Translation

cs.RO / 19 / 2607.03211

Derivations of Error-State Kalman Filter Kinematics for Globally Applicable Aided Inertial Navigation Systems

适用于全球的辅助惯性导航系统的误差状态卡尔曼滤波运动学推导

Hager, Antonia, Bryne, Torleiv H.

Abstract

Global navigation systems require state estimation algorithms that handle Earth's curvature, Earth's rotation, and gravitational variations. These factors can typically be neglected in local navigation algorithms for robots, drones, etc. In classical error-state Kalman Filtering (ESKF) the error state dynamics are trajectory-dependent. Invariant ESKFs utilize Lie Group symmetries to represent the error, which can render error propagation trajectory-independent for group-affine systems. Choosing between a standard filter (where position and velocity errors are defined additively in the navigation frame), a left-invariant filter (where errors are represented in the body frame) and a right-invariant filter (where errors are represented in the navigation/world frame) depends on system dynamics and sensor configuration. This note presents the mathematical formulas for four classical and invariant ESKFs for globally applicable aided inertial navigation systems. It is intended to serve as a systematic reference for comparison and implementation.

Chinese Translation

全球导航系统需要能够处理地球曲率、地球自转和重力变化的状态估计算法。这些因素在机器人、无人机等的局部导航算法中通常可以忽略。在经典的误差状态卡尔曼滤波（ESKF）中，误差状态动态是依赖于轨迹的。不变的ESKF利用李群对称性来表示误差，这可以使得误差传播在群仿射系统中与轨迹无关。选择标准滤波器（在导航框架中位置和速度误差以加法方式定义）、左不变滤波器（在机体框架中表示误差）和右不变滤波器（在导航/世界框架中表示误差）取决于系统动态和传感器配置。本文提供了四种经典和不变的ESKF的数学公式，适用于全球的辅助惯性导航系统，旨在作为比较和实施的系统参考。

View on arXiv Download PDF AI Translation

cs.RO / 20 / 2607.03216

Strouhal-Aware Model Predictive Control for Efficient Multi-Fin Flapping Locomotion

考虑斯特劳哈尔数的模型预测控制用于高效多鳍拍动运动

Hamamatsu, Yuya, Chen, Zixi, Kruusmaa, Maarja, Ristolainen, Asko

Abstract

Efficient flapping propulsion hinges on operating within a narrow Strouhal number window, a principle nature has converged upon for maximum thrust-to-power ratio. We translate this bioinspired empirical rule into real-time control, demonstrating it on an autonomous underwater vehicle driven by four soft fins. The proposed Strouhal-aware Model Predictive Control (MPC) enhances a quasi-steady hydrodynamic model with an explicit penalty for Strouhal deviation, solving the resulting nonconvex problem via a two-stage sampling and gradient optimization that runs onboard at 25 Hz. Pool and field trials show that the controller keeps each fin within the optimal Strouhal corridor (0.25-0.35) while precisely tracking commanded forces. This results in a mean reduction in mechanical power of 8.8\% to 32\% throughout the cruising range of 0.1 to 0.3 m/s. The proposed method also allows for a velocity of 0.4 m/s, which is unattainable for a baseline of the conventional inverse model. The results confirm that embedding first-principle flow physics into an MPC objective yields tangible endurance gains without sacrificing agility, offering a generic pathway to energy-aware locomotion in next-generation multifin robots.

Chinese Translation

高效的拍动推进依赖于在狭窄的斯特劳哈尔数窗口内运行，这是自然界为最大推力与功率比而趋向的原则。我们将这一生物启发的经验法则转化为实时控制，并在由四个柔性鳍驱动的自主水下航行器上进行了验证。所提出的考虑斯特劳哈尔数的模型预测控制（MPC）增强了一个准稳态水动力模型，并对斯特劳哈尔偏差施加了明确的惩罚，通过在25 Hz的频率下进行的两阶段采样和梯度优化来解决由此产生的非凸问题。池内和野外试验表明，该控制器能够将每个鳍保持在最佳斯特劳哈尔走廊（0.25-0.35）内，同时精确跟踪指令力。这导致在0.1至0.3 m/s的巡航范围内，机械功率平均降低了8.8 ext{%}至32 ext{%}。所提出的方法还允许达到0.4 m/s的速度，而这是传统逆模型基线无法实现的。结果确认，将第一性原理流体物理嵌入MPC目标中可以在不牺牲灵活性的情况下实现显著的耐久性提升，为下一代多鳍机器人提供了一条通用的能量感知运动路径。

View on arXiv Download PDF AI Translation

cs.RO / 21 / 2607.03254

GDPR-Aware Trajectory Sharing for ISAC-Assisted Robot Navigation: A Case Study on FID-Constrained Collision Prediction

基于GDPR的轨迹共享用于ISAC辅助机器人导航：关于FID约束的碰撞预测案例研究

Fang, Zexin, Han, Bin, Wang, Donglin, Pei, Fengchen, Schotten, Hans D.

Abstract

Integrated sensing and communication (ISAC) enables intelligent wireless infrastructure but raises growing regulatory concern as fine-grained personal trajectory histories become a byproduct of sensing. General Data Protection Regulation (GDPR) Articles 5(1)(c) and 5(1)(f) require that personal data be limited to what is necessary and protected through appropriate technical measures against unauthorised reconstruction. This paper addresses both requirements through a Fisher information density (FID)-constrained trajectory sharing scheme for robot collision avoidance, where sensing estimates are perturbed according to local information content before sharing. Experiments on real pedestrian traces show that FID-controlled sharing achieves a strictly better privacy-utility tradeoff than fixed-error perturbation: at matched missed-conflict rates, reconstruction leakage and sustained exposure lengths are consistently lower, establishing information-aware perturbation as a principled technical measure aligned with GDPR data minimisation and integrity requirements.

Chinese Translation

集成感知与通信（ISAC）使智能无线基础设施成为可能，但随着精细化个人轨迹历史成为感知的副产品，监管关注也日益增加。通用数据保护条例（GDPR）第5条第1款（c）和第5条第1款（f）要求个人数据应限于必要范围，并通过适当的技术措施保护，以防止未经授权的重构。本文通过一种基于Fisher信息密度（FID）约束的轨迹共享方案来应对这两个要求，以实现机器人碰撞避免，其中感知估计根据本地信息内容进行扰动后再进行共享。对真实行人轨迹的实验表明，FID控制的共享在隐私与效用的权衡上优于固定误差扰动：在匹配的错过冲突率下，重构泄露和持续暴露长度始终较低，确立了信息感知扰动作为符合GDPR数据最小化和完整性要求的原则性技术措施。

View on arXiv Download PDF AI Translation

cs.RO / 22 / 2607.03289

Invisible Strings: Deriving Puppetry Principles and their Hidden Connections to Robot Behavior Design

隐形线索：推导木偶戏原则及其与机器人行为设计的隐秘联系

Lewis, Claire, Collins, Sawyer, Hanson, Alyssa, Smith, Johanna, Williams, Tom

Abstract

When designing robots' nonverbal behaviors, many researchers have turned to arts-based insights, such as Disney's Animation Principles. Yet, while these principles bear key insights into the design of like-life characters, their application to robot design is inherently limited, in part because animation is not constrained by real-world physics, and in part because animation principles focus on low level animation mechanics and not high-level design considerations for physically embodied, interactive characters. In contrast, little attention has been paid to art forms like puppetry, despite their long history of exploring morphological, behavior, and interaction design of physically embodied, interactive characters. As such, in this work we leverage puppetry texts and practicing puppeteers' expert knowledge knowledge to derive a set of puppetry principles with key insights for robot design. As we show, these insights go beyond -- and uniquely complement -- the prior insights provided by theater, dance, and animation.

Chinese Translation

在设计机器人的非语言行为时，许多研究者转向基于艺术的见解，例如迪士尼的动画原则。然而，尽管这些原则对生命角色的设计提供了关键见解，但它们在机器人设计中的应用本质上是有限的，部分原因是动画不受现实物理的限制，部分原因是动画原则侧重于低层次的动画机制，而非针对物理具身的互动角色的高层次设计考虑。相比之下，尽管木偶戏在探索物理具身的互动角色的形态、行为和互动设计方面有着悠久的历史，但对此类艺术形式的关注却相对较少。因此，在本研究中，我们利用木偶戏文本和实践木偶师的专业知识，推导出一套具有关键见解的木偶戏原则，以指导机器人设计。正如我们所展示的，这些见解超越了之前由戏剧、舞蹈和动画提供的见解，并独特地补充了这些见解。

View on arXiv Download PDF AI Translation

cs.RO / 23 / 2607.03321

U-Joint CAAMS: Experimental Evaluation of a Universal-Joint Continuum Manipulator for Aerial Manipulation

U-关节CAAMS：用于空中操作的通用关节连续操纵器的实验评估

Uthayasooriyan, Anuraj, Alibrahim, Musab, Digumarti, Krishna Manaswi, Vanegas, Fernando, Gonzalez, Felipe

Abstract

Continuum manipulators mounted on multi-rotor UAVs enable compliant aerial manipulation, but payloads and propeller downwash amplify out-of-plane bending and twisting that degrade end-effector pose accuracy. To address this problem, we present a universal-joint-based continuum manipulator designed to improve resistance to out-of-plane deformation during aerial manipulation. The proposed design uses a tubular backbone with spring-reinforced universal joints and an integrated conduit for internal routing and fluid delivery. We evaluate the design in still air and under peak propeller downwash across varying payloads, and benchmark it against a prior Nitinol-backbone CM. Bench tests show improved resistance to out-of-plane deformation across all conditions. Under peak downwash, the proposed design reduces mean error by 2.5-4x in yaw, 2-45x in y-axis, and up to 5x in roll compared to the NiTi-backbone design. We further analyze hover stability through in-flight coupled-disturbance tests over varying payloads and actuation speeds, and demonstrate the system in water sampling, spot spraying, and object transport.

Chinese Translation

安装在多旋翼无人机上的连续操纵器能够实现柔性空中操作，但有效载荷和螺旋桨下洗增强了超出平面弯曲和扭转，降低了末端执行器姿态的准确性。为了解决这个问题，我们提出了一种基于通用关节的连续操纵器，旨在提高在空中操作过程中对超出平面变形的抵抗力。所提出的设计采用了带有弹簧增强的通用关节的管状骨架，并集成了用于内部布线和流体输送的导管。我们在静止空气中以及在不同有效载荷下的峰值螺旋桨下洗条件下评估了该设计，并与之前的镍钛合金骨架连续操纵器进行了基准测试。测试结果显示，在所有条件下，该设计对超出平面变形的抵抗力有所提高。在峰值下洗条件下，与镍钛合金骨架设计相比，所提出的设计在偏航方向上减少了2.5-4倍的平均误差，在y轴上减少了2-45倍，在滚转方向上减少了最多5倍。我们进一步通过在不同有效载荷和驱动速度下的飞行耦合干扰测试分析了悬停稳定性，并展示了该系统在水样采集、点喷洒和物体运输中的应用。

View on arXiv Download PDF AI Translation

cs.RO / 24 / 2607.03338

Closed-loop vs. Open-loop Kalman Filter Architectures in Airborne Aided Inertial Navigation

闭环与开环卡尔曼滤波器架构在空中辅助惯性导航中的比较

Hager, Antonia, Bryne, Torleiv H.

Abstract

Closed-loop (or feedback) error-state Kalman filters with their relatives and offspring are the state-of-the-art in modern aided inertial navigation research. Estimated inertial navigation system (INS) errors are continually fed back to the INS to correct the nominal system state before subsequent predictions. Conversely, in safety-critical aeronautical applications, open-loop (or feedforward) systems are an undisputed standard, where the inertial mechanization is strictly decoupled to allow for operational independence and fault isolation of computing units. We assess the performance impacts of this architectural choice beyond qualitative system-safety justifications using a standard inertial mechanization in geodetic coordinates and direct position aiding. Simulations using a variety of inertial sensor error characteristics, ranging from consumer to navigation grade systems, showcase the trade-off between smooth information fusion for high-end IMUs using an open-loop filter and the inherent long-term stability of the closed-loop architecture.

Chinese Translation

闭环（或反馈）误差状态卡尔曼滤波器及其相关技术是现代辅助惯性导航研究的前沿。估计的惯性导航系统（INS）误差不断反馈到INS，以在后续预测之前纠正名义系统状态。相反，在安全关键的航空应用中，开环（或前馈）系统是无可争议的标准，其中惯性机械化被严格解耦，以实现计算单元的操作独立性和故障隔离。我们评估了这种架构选择的性能影响，超越了定性系统安全性理由，使用标准的地理坐标惯性机械化和直接位置辅助。通过对各种惯性传感器误差特性的模拟，从消费级到导航级系统，展示了使用开环滤波器的高端IMU在平滑信息融合与闭环架构固有的长期稳定性之间的权衡。

View on arXiv Download PDF AI Translation

cs.RO / 25 / 2607.03387

Feeling the Unexpected: ResTacVLA for Contact-Rich Manipulation via Residual Tactile Representation

感知意外：通过残差触觉表示实现接触丰富的操控的ResTacVLA

Zhang, Pengwei, Xie, Bin, Hao, Ce, Meng, Xinpan, Guo, Xinyu, Deng, Fang, Cheng, Long, Wang, Tiancai

Abstract

Tactile perception is indispensable for contact-rich manipulation, yet integrating it into Vision-Language-Action (VLA) models often induces modality collapse, where high-bandwidth visual features overshadow sparse tactile cues. Inspired by Predictive Coding, a neural mechanism where the brain attenuates predictable inputs to prioritize surprising stimuli, we propose ResTacVLA. Rather than treating tactile data as raw input, we reformulate it as a Residual Tactile Representation capturing the discrepancy between visual priors and physical sensations. By filtering out visually predictable dynamics, this formulation transforms sparse tactile signals into dense, high-value information gain, thereby inherently resolving the bandwidth mismatch. These residuals are discretized through a Vector Quantized (VQ) bottleneck into Latent Contact Primitives that capture critical events missed by vision. Analogous to the neural surprise signal, we leverage the uncertainty of the visual prior to adaptively gate tactile integration, prioritizing residuals specifically during visually unreliable phases to explicitly prevent visual dominance. Experimental results show that ResTacVLA consistently outperforms all baselines on a diverse set of contact-rich manipulation tasks, while remaining robust to unexpected dynamic disturbances. Project page: https://awilekong.github.io/ResTacVLA-Website/

Chinese Translation

触觉感知对于接触丰富的操控至关重要，但将其整合到视觉-语言-动作（VLA）模型中往往会导致模态崩溃，即高带宽的视觉特征掩盖稀疏的触觉线索。受到预测编码的启发，这是一种神经机制，脑部会减弱可预测的输入以优先处理意外刺激，我们提出了ResTacVLA。我们并不将触觉数据视为原始输入，而是将其重新表述为残差触觉表示，捕捉视觉先验与物理感受之间的差异。通过过滤可视预测的动态，这种表述将稀疏的触觉信号转化为密集的高价值信息增益，从而内在地解决了带宽不匹配的问题。这些残差通过向量量化（VQ）瓶颈离散化为潜在接触原语，捕捉视觉未能识别的关键事件。类似于神经惊讶信号，我们利用视觉先验的不确定性自适应地控制触觉整合，特别是在视觉不可靠的阶段优先考虑残差，以明确防止视觉主导。实验结果表明，ResTacVLA在多样化的接触丰富操控任务中始终优于所有基线，同时对意外动态干扰保持稳健。项目页面：https://awilekong.github.io/ResTacVLA-Website/

View on arXiv Download PDF AI Translation

cs.RO / 26 / 2607.03449

HiMe: Hierarchical Embodied Memory for Long-Horizon Vision-Language-Action Control

HiMe：用于长时间视-语言-动作控制的层次化具身记忆

Ji, Li, Wang, Siyin, Qian, Pengfang, Yu, Xiaopeng, Tian, Yihai, Fei, Zhaoye, Gong, Jingjing, Qiu, Xipeng

Abstract

Current Vision-Language-Action (VLA) models excel at robotic manipulation but often struggle with non-Markovian tasks requiring long-term memory and reasoning due to their reliance on immediate observations. Existing solutions face a ''frequency-competence paradox,'' where stronger reasoning models are too slow for real-time control, while faster models lack sufficient reasoning capabilities. To resolve this architectural misalignment, we propose HiMe, a Hierarchical Embodied Memory framework that decouples embodied intelligence into a high-frequency Executor for execution, a Sentry for working memory, and a Planner for long-term strategy. We also introduce a dynamic knowledge system based on cross-modal semantic schemas and active management mechanisms, allowing robots to maintain memory plasticity through ''Add, Update, and Delete'' operations. This hierarchical design effectively balances the conflict between real-time execution and slow thinking planning, significantly improving success rates in long-horizon tasks. Experiments demonstrate that this approach not only outperforms flat memory baselines but also exhibits the novel ability to self-correct its internal knowledge based on human preferences.

Chinese Translation

当前的视-语言-动作（VLA）模型在机器人操作方面表现出色，但在需要长期记忆和推理的非马尔可夫任务中常常面临挑战，这主要是由于它们依赖于即时观察。现有解决方案面临“频率-能力悖论”，即更强的推理模型在实时控制中速度过慢，而更快的模型则缺乏足够的推理能力。为了解决这一架构不匹配的问题，我们提出了HiMe，一个层次化具身记忆框架，将具身智能解耦为高频执行器（Executor）、工作记忆的哨兵（Sentry）和长期策略的规划者（Planner）。我们还引入了一种基于跨模态语义模式和主动管理机制的动态知识系统，使机器人能够通过“添加、更新和删除”操作维持记忆的可塑性。这种层次化设计有效平衡了实时执行与缓慢思维规划之间的冲突，显著提高了长时间任务的成功率。实验表明，该方法不仅优于平面记忆基线，还展现出根据人类偏好自我修正内部知识的新能力。

View on arXiv Download PDF AI Translation

cs.RO / 27 / 2607.03454

ADP: Adversarial Dynamics Priors for Physically Grounded Humanoid Locomotion

ADP：用于物理基础人形运动控制的对抗动态先验

Lee, Seokju, Lee, Jeongtae, Lim, Jeonghyeok, Kang, Jeonguk, Lee, Byungwook, Han, Seungho, Choi, Keun Ha, Park, Dongil, Kim, Kyung-Soo

Abstract

In this paper, we propose Adversarial Dynamics Priors (ADP) for perturbation-resilient humanoid locomotion control. Existing motion prior-based methods induce natural motion styles by imitating kinematic motion features, but they do not directly regularize dynamics features, such as CoM motion, centroidal momentum, contact forces, and contact states. To address this limitation, we replace kinematic motion-style feature with selected dynamics features extracted from locomotion trajectories as the target of adversarial regularization.To this end, we use trajectory optimization to construct a reference dataset and train a discriminator to evaluate whether policy-induced temporal windows are consistent with the resulting reference distribution.Without explicit motion tracking, ADP encourages policy rollouts to remain close to the reference support, even after perturbations. Experimental results show that, compared with AMP, the strongest baseline in our evaluation, ADP improves the $80\%$-success impulse threshold ($J_{80}$) by $16.7\%$, while reducing direction-averaged recovery time and velocity tracking error by $47.9\%$ and $35.4\%$, respectively.

Chinese Translation

本文提出了对抗动态先验（ADP），用于增强人形运动控制的抗扰动能力。现有的基于运动先验的方法通过模仿运动学特征来诱导自然的运动风格，但并未直接对动态特征进行正则化，例如质心（CoM）运动、质心动量、接触力和接触状态。为了解决这一局限性，我们用从运动轨迹中提取的选定动态特征替代运动学运动风格特征，作为对抗正则化的目标。为此，我们使用轨迹优化构建参考数据集，并训练一个鉴别器来评估策略诱导的时间窗口是否与生成的参考分布一致。在没有显式运动跟踪的情况下，ADP 鼓励策略展开保持接近参考支持，即使在扰动之后。实验结果表明，与我们评估中的最强基线 AMP 相比，ADP 将 $80\%$ 成功冲击阈值（$J_{80}$）提高了 $16.7\\%$，同时将方向平均恢复时间和速度跟踪误差分别降低了 $47.9\\%$ 和 $35.4\\%$。

View on arXiv Download PDF AI Translation

cs.RO / 28 / 2607.03512

High-Precision Formation Control for Heterogeneous Multi-Robot Systems via Hierarchical Hybrid Physics-Informed Deep Reinforcement Learning

基于分层混合物理信息深度强化学习的异构多机器人系统高精度编队控制

Li, Yanzhou, Chen, Guangli, Li, Xiao-Meng, Zhong, Wenjian, Lu, Yongkang, He, Shenghuang

Abstract

Existing classical control methods commonly require precise models and struggle to cope with model uncertainties and external disturbances, while end-to-end reinforcement learning (RL) approaches suffer from low sample efficiency and poor convergence. To overcome these challenges, this paper proposes a hierarchical hybrid physics-informed deep reinforcement learning (HHy-PIDRL) framework, aiming to realize high-precision, highly responsive formation control for heterogeneous multi-robot systems (HMRSs). The proposed framework contains two layers. Specifically, first, the upper layer designs an autonomous navigation policy network for Ackermann-steering leader based on the Soft Actor-Critic (SAC) deep reinforcement learning (DRL) algorithm. Second, the lower module integrates a high-fidelity physical feed-forward controller, a classical proportional-derivative (PD) controller, and an adaptive DRL residual controller to propose an effective hybrid model and DRL (HM-DRL)-based formation control policy network. Third, a unique hierarchical reward function is designed for training Omnidirectional followers, which effectively guides agents toward a refined, stable control policy. Experimental results demonstrate that, the success rate of both the upper-layer autonomous navigation policy network and the HM-DRL based formation control policy networks reach 100%. Meanwhile, ablation experiments are conducted to verify the validity and credibility of the proposed method.

Chinese Translation

现有的经典控制方法通常需要精确的模型，并且在应对模型不确定性和外部干扰时表现不佳，而端到端的强化学习（RL）方法则面临样本效率低和收敛性差的问题。为了解决这些挑战，本文提出了一种分层混合物理信息深度强化学习（HHy-PIDRL）框架，旨在实现异构多机器人系统（HMRSs）的高精度、高响应编队控制。该框架包含两个层次。具体而言，首先，上层基于软演员-评论家（Soft Actor-Critic, SAC）深度强化学习（DRL）算法设计了一个用于Ackermann转向领导者的自主导航策略网络。其次，下层模块集成了高保真物理前馈控制器、经典比例-微分（PD）控制器和自适应DRL残差控制器，以提出一种有效的基于混合模型和DRL（HM-DRL）的编队控制策略网络。第三，设计了一种独特的分层奖励函数用于训练全向跟随者，有效引导智能体朝向精细、稳定的控制策略。实验结果表明，上层自主导航策略网络和基于HM-DRL的编队控制策略网络的成功率均达到100%。同时，进行了消融实验以验证所提方法的有效性和可信度。

View on arXiv Download PDF AI Translation

cs.RO / 29 / 2607.03529

Current as Touch: Proprioceptive Contact Feedback for Compliant Dexterous Manipulation

电流作为触觉：用于顺应性灵巧操作的本体感觉接触反馈

Ma, Chenyang, Yao, Yunchao, Wei, Zhenyu, Li, Ruogu, Szafir, Daniel, Ding, Mingyu

Abstract

Compliance is essential for dexterous manipulation, yet existing solutions often rely on external tactile or force sensors that are costly, fragile, and difficult to deploy on low-cost robot hands. We propose a proprioception-driven framework that learns contact-aware compliance cues from motor current and joint states. Since motor current is closely related to actuator torque, it provides an intrinsic signal for perceiving contact force, object resistance, and grasp stability without additional sensing hardware. Rather than estimating external wrenches or commanding torque, our method predicts a compliance reference position: an ideal joint-position target for a standard PD controller whose induced position error generates appropriate grasping force. This position-based formulation is compatible with mainstream teleoperation and policy-learning pipelines, while enabling the robot to adapt interaction forces from real-time proprioceptive feedback. Thus, motor current serves not only as a force proxy but also as a learnable proprioceptive contact signal for compliance reference prediction. Experiments on multiple dexterous hands and contact-rich tasks, including fragile object handling, sustained surface contact, thin-object retrieval, and dynamic load adaptation, show stable compliant grasping, safer and more efficient teleoperation, and improved downstream policy learning without external tactile or force sensors.

Chinese Translation

顺应性对于灵巧操作至关重要，但现有解决方案通常依赖于昂贵、易损且难以在低成本机器人手上部署的外部触觉或力传感器。我们提出了一种基于本体感觉的框架，该框架从电机电流和关节状态中学习接触感知的顺应性线索。由于电机电流与执行器扭矩密切相关，它提供了一种内在信号，用于感知接触力、物体阻力和抓握稳定性，而无需额外的传感硬件。我们的方法不是估计外部扭矩或指令扭矩，而是预测一个顺应性参考位置：一个标准PD控制器的理想关节位置目标，其引起的位置误差生成适当的抓握力。这种基于位置的公式与主流的遥操作和策略学习流程兼容，同时使机器人能够根据实时本体感觉反馈调整交互力。因此，电机电流不仅作为力的代理，还作为可学习的本体感觉接触信号，用于顺应性参考预测。在多个灵巧手和接触丰富的任务上的实验，包括脆弱物体处理、持续表面接触、薄物体提取和动态负载适应，显示出稳定的顺应性抓握、更安全和更高效的遥操作，以及在没有外部触觉或力传感器的情况下改进的下游策略学习。

View on arXiv Download PDF AI Translation

cs.RO / 30 / 2607.03557

CoorGrasp: Coordinated Contact Control for Adaptive Dexterous Grasping Under Uncertainty

CoorGrasp：在不确定性下的自适应灵巧抓取的协调接触控制

Yu, Mingrui, Jiang, Yongpeng, Jia, Yongyi, Yi, Ren, Li, Xiang

Abstract

While recent research has focused heavily on dexterous grasp pose generation, less attention has been devoted to the execution of planned grasps. Under shape and position uncertainty, open-loop execution often yields uncoordinated contacts, causing undesired in-hand object motion and even grasp failures. To address this, this paper proposes a tactile-driven model predictive controller for adaptive and delicate execution of diverse dexterous grasps. Our approach emphasizes multi-contact coordination across both approaching and grasping phases, with three key novelties: (i) coordination-aware phase separation, (ii) arm-hand coordination to compensate for position errors, and (iii) adaptive force coordination to increase contact forces in a balanced manner. An analytical model is employed to relate contact forces to robot joint motions for predictive control. Our formulation imposes no restrictions on grasp types or contact configurations and integrates seamlessly with state-of-the-art grasp pose generation methods. We validate the approach through large-scale simulations involving 15k grasps across 478 objects on three robotic hands, and real-world experiments on 8 objects. Results demonstrate that our method achieves higher grasp success rates and reduced undesired object movements.

Chinese Translation

尽管近期研究重点关注灵巧抓取姿态的生成，但对计划抓取的执行关注较少。在形状和位置的不确定性下，开环执行往往导致接触不协调，从而引起手中物体的非期望运动甚至抓取失败。为了解决这个问题，本文提出了一种基于触觉驱动的模型预测控制器，用于自适应和精细地执行多样化的灵巧抓取。我们的方法强调在接近和抓取阶段的多接触协调，具有三个关键创新：（i）协调感知的阶段分离，（ii）手臂与手的协调以补偿位置误差，以及（iii）自适应力协调以平衡地增加接触力。我们采用分析模型将接触力与机器人关节运动关联，以实现预测控制。我们的公式对抓取类型或接触配置没有限制，并与最先进的抓取姿态生成方法无缝集成。我们通过大规模仿真验证了该方法，涉及478个物体的15,000次抓取，以及对8个物体的实际实验。结果表明，我们的方法实现了更高的抓取成功率和减少了非期望的物体运动。

View on arXiv Download PDF AI Translation

cs.RO / 31 / 2607.03570

Cross-Embodiment Robot Manipulation via a Unified Hand Action Space

通过统一手部动作空间实现跨体现机器人操控

Casas, Luis Felipe, Teal, Robert, Shah, Keval, Tadepalli, Abhijit, Jin, Wanxin, Xiang, Yu

Abstract

Robot manipulation policies are typically tied to specific robotic hand embodiments, limiting the transfer of learned behaviors across platforms with different kinematic structures. In this work, we propose the Unified Hand Action Space (UHAS), a sphere-based unified action representation for cross-embodiment dexterous manipulation. UHAS represents robotic hand actions as geometric deformations of a canonical sphere and uses a Cascade Inverse Kinematics (CIK) algorithm to map the shared representation to embodiment-specific joint configurations. Using reinforcement learning, we train dexterous manipulation policies directly in the proposed action space for in-hand cube reorientation tasks. We evaluate our method in both simulation and real-world experiments across multiple robotic hands, including the Allegro Hand, LEAP Hand, Shadow Hand, and MANO Human Hand. Experimental results demonstrate effective dexterous manipulation, zero-shot transfer to unseen hands, rapid finetuning across embodiments, and successful real-world deployment. Our experiments show that the proposed UHAS representation enables stable dexterous control and cross-embodiment policy transfer across robotic hands.

Chinese Translation

机器人操控策略通常与特定的机器人手部体现相关联，这限制了在具有不同运动结构的平台之间转移学习到的行为。在本研究中，我们提出了统一手部动作空间（Unified Hand Action Space, UHAS），这是一种基于球体的统一动作表示，用于跨体现的灵巧操控。UHAS将机器人手部动作表示为标准球体的几何变形，并使用级联逆运动学（Cascade Inverse Kinematics, CIK）算法将共享表示映射到特定体现的关节配置。我们利用强化学习直接在所提议的动作空间中训练灵巧操控策略，以完成手内立方体重新定向任务。我们在多个机器人手部上评估了我们的方法，包括Allegro Hand、LEAP Hand、Shadow Hand和MANO Human Hand，实验包括模拟和真实世界实验。实验结果表明，灵巧操控效果显著，能够实现对未见手部的零样本转移，快速在不同体现间进行微调，并成功部署于真实世界。我们的实验表明，所提出的UHAS表示能够实现稳定的灵巧控制和跨体现的策略转移。

View on arXiv Download PDF AI Translation

cs.RO / 32 / 2607.03616

ROBOCYCLE: Autonomous Dual-Arm Robotic Manipulation and Coordination for Recycling Applications

ROBOCYCLE：用于回收应用的自主双臂机器人操控与协调

Hilario-Cruz, Rubén de J., García-González, Jesus A., Coronado, Enrique, Cerón-López, Arturo E.

Abstract

As urban waste volumes escalate and labor shortages intensify, automated waste sorting systems are becoming a necessity. However, current robotic solutions often struggle with the 3D perception and manipulation of transparent, deformable, or cluttered objects. This work introduces ROBOCYCLE, an autonomous dual-arm robotic recycling platform designed to meet the recycling standards of the Tokyo metropolitan area. Our approach integrates multi-view RGB-D perception, transformer-based instance segmentation using RF-DETR, and 6-DoF grasp planning via the Anygrasp SDK. By processing segmentated point clouds, the system generates robust candidate poses for irregular and deformable waste. The system achieved a 90.3% grasp success rate and 84.3% overall task success rate, effectively performing complex coordinated tasks such as unscrewing PET bottle caps. The proposed platform offers a scalable solution for autonomous waste management in real-world human environments.

Chinese Translation

随着城市垃圾量的增加和劳动力短缺的加剧，自动化垃圾分类系统变得愈发必要。然而，目前的机器人解决方案在透明、可变形或杂乱物体的三维感知和操控方面常常面临挑战。本研究介绍了ROBOCYCLE，一个自主双臂机器人回收平台，旨在满足东京大都市区的回收标准。我们的方法集成了多视角RGB-D感知、基于变换器的实例分割（RF-DETR）以及通过Anygrasp SDK实现的六自由度抓取规划。通过处理分割后的点云，系统为不规则和可变形的垃圾生成了稳健的候选姿态。该系统实现了90.3%的抓取成功率和84.3%的整体任务成功率，能够有效执行复杂的协调任务，如拧开PET瓶盖。所提出的平台为在现实人类环境中实现自主垃圾管理提供了可扩展的解决方案。

View on arXiv Download PDF AI Translation

cs.RO / 33 / 2607.03693

CoRE-VLA: Towards Scalable and Robust Vision-Language-Action Modeling via Conditional Routing of Experts

CoRE-VLA：通过条件路由专家实现可扩展且稳健的视觉-语言-动作建模

Zhang, Haozhe, Li, Sixian, Zhang, Yifei, Huai, Zezheng, Chen, Hao, Shen, Chunhua, Gong, Jingjing, Qiu, Xipeng

Abstract

Vision-language-action (VLA) models have advanced generalist robotic manipulation, yet real-world deployment reveals a fundamental challenge: robots are equipped with diverse and heterogeneous sensor configurations, auxiliary sensors can fail unexpectedly during operation, and different robot embodiments often lack certain sensors by design. A unified policy that can exploit auxiliary perceptual inputs when available while remaining reliable under sensor absence, whether incidental or by design, is therefore essential for practical deployment. However, existing VLA policies couple action generation to a fixed sensor set through shared dense computation, making them brittle when sensors are missing and limiting their ability to specialize across diverse tasks and long-horizon behaviors. We propose CoRE-VLA, a scalable and robust VLA framework that formulates action generation as context-conditioned sparse computation. Sensor availability gates modality-specialized experts, enabling graceful degradation under missing sensors without retraining. Task intent further routes action-side representations to task-relevant experts, improving specialization across diverse tasks and long-horizon subgoals. While the framework is designed to accommodate different auxiliary sensors, we focus on depth as a representative and practically important auxiliary modality in our experiments. Experiments on LIBERO, RoboCasa GR1 Tabletop, and real-world dual-arm manipulation show that CoRE-VLA achieves strong results on long-horizon and multi-task benchmarks, and outperforms both a dense-action-generator ablation and a strong pretrained VLA baseline, including in zero-shot generalization to unseen scenarios. Modality analysis shows that CoRE-VLA can exploit auxiliary depth when available while remaining robust when depth is unavailable during deployment.

Chinese Translation

视觉-语言-动作（VLA）模型在通用机器人操作方面取得了进展，但在实际部署中暴露出一个基本挑战：机器人配备了多样且异构的传感器配置，辅助传感器在操作过程中可能会意外失效，且不同的机器人形态往往因设计原因缺少某些传感器。因此，开发一种能够在辅助感知输入可用时加以利用，同时在传感器缺失（无论是偶然还是设计性缺失）情况下仍然可靠的统一策略，对于实际部署至关重要。然而，现有的 VLA 策略通过共享密集计算将动作生成与固定传感器集耦合，使其在传感器缺失时变得脆弱，并限制了其在多样任务和长时间行为中的专业化能力。我们提出了 CoRE-VLA，这是一种可扩展且稳健的 VLA 框架，将动作生成公式化为上下文条件稀疏计算。传感器的可用性控制着模态专门化专家的使用，使得在缺失传感器的情况下能够优雅降级而无需重新训练。任务意图进一步将动作侧表示路由到与任务相关的专家，从而提高在多样任务和长时间子目标中的专业化。虽然该框架旨在适应不同的辅助传感器，但我们在实验中重点关注深度作为一种具有代表性且在实践中重要的辅助模态。在 LIBERO、RoboCasa GR1 桌面和实际双臂操作的实验表明，CoRE-VLA 在长时间和多任务基准测试中取得了良好的结果，并且在零样本泛化到未见场景时超越了密集动作生成器消融实验和强大的预训练 VLA 基线。模态分析表明，CoRE-VLA 能够在可用时利用辅助深度，同时在部署期间深度不可用时保持稳健。

View on arXiv Download PDF AI Translation

cs.RO / 34 / 2607.03699

Lost in Time? Continuous Symmetry and Identifiability in Aided Inertial Navigation with Unknown Measurement Delays

迷失在时间中？带有未知测量延迟的辅助惯性导航中的连续对称性与可识别性

Kelly, Jonathan, Kyaw, Phone Thiha, Giamou, Mattew

Abstract

In many multisensor systems, measurements from different sensors are subject to unknown relative time delays. Accurate state estimation requires that delays be accounted for and, when possible, calibrated online. We consider the case of aided navigation, where measurements from a single aiding sensor are subject to an unknown but constant delay relative to the inertial measurement stream, and study the identifiability of the resulting system. Critically, identifiability depends not only on the temporal structure of the measurements, but also on the shape of the vehicle trajectory: some trajectories are sufficiently informative to support unique recovery of the delay and the navigation state, while others are not. Using the special Galilean group, we characterize these uninformative (or degenerate) trajectories and relate them to a continuous symmetry of the delayed measurement model, providing geometric insight into identifiability failures. We show that the class of trajectories for which identifiability fails is larger than previously reported, and connect our characterization to the familiar linearized, Jacobian-based analysis. Although our development is motivated by aided navigation, the underlying ideas apply more broadly to estimation problems on Lie groups with delayed measurements.

Chinese Translation

在许多多传感器系统中，不同传感器的测量受到未知的相对时间延迟的影响。准确的状态估计要求考虑这些延迟，并在可能的情况下进行在线校准。我们考虑辅助导航的情况，其中来自单一辅助传感器的测量相对于惯性测量流存在未知但恒定的延迟，并研究由此产生的系统的可识别性。关键是，可识别性不仅依赖于测量的时间结构，还依赖于车辆轨迹的形状：某些轨迹足够信息丰富，可以支持延迟和导航状态的唯一恢复，而其他轨迹则不然。利用特殊的伽利略群，我们表征了这些无信息（或退化）轨迹，并将其与延迟测量模型的连续对称性联系起来，从几何上提供了对可识别性失败的洞察。我们展示了可识别性失败的轨迹类别比之前报告的更大，并将我们的表征与熟悉的线性化、雅可比分析相联系。尽管我们的研究动机来源于辅助导航，但其基本思想更广泛地适用于具有延迟测量的李群上的估计问题。

View on arXiv Download PDF AI Translation

cs.RO / 35 / 2607.03723

OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies

OmniTacTune：一种与策略无关的现实世界强化学习方法，用于视觉策略的触觉残差适应

Yu, Kelin, Zhang, Haode, Ravichandar, Harish, Han, Yunhai, Gao, Ruohan

Abstract

Visual policies learned from human videos, teleoperation, and robot demonstrations offer scalable motion priors, but often fail in contact-rich manipulation, where success significantly depends on local force and contact geometry. Tactile sensing provides these complementary signals, yet tactile data remain costly to collect and hard to generalize across sensors, robots, and tasks. We introduce OmniTacTune, a policy-agnostic real-world RL pipeline that adapts tactile feedback to pretrained visual policies through residual correction. OmniTacTune uses a two-stage design: it first bootstraps tactile-aware learning from autonomous base-policy rollouts, then learns a lightweight tactile residual policy through online interaction. Extensive experiments show that OmniTacTune generalizes across diverse contact-rich tasks, visual base policies, and tactile representations. Across four real-world contact-rich tasks, it improves visual base policies from 5-40% success to 85-100% within 40-80 minutes, demonstrating an efficient path for adapting tactile feedback to scalable visual robot policies. Project page: https://colinyu1.github.io/omnitactune-site/

Chinese Translation

从人类视频、遥操作和机器人演示中学习的视觉策略提供了可扩展的运动先验，但在接触丰富的操作中往往表现不佳，这种成功在很大程度上依赖于局部力和接触几何形状。触觉传感提供了这些互补信号，但触觉数据的收集成本高且在传感器、机器人和任务之间难以泛化。我们提出了OmniTacTune，这是一种与策略无关的现实世界强化学习管道，通过残差校正将触觉反馈适应于预训练的视觉策略。OmniTacTune采用两阶段设计：首先从自主基础策略的回放中引导触觉感知学习，然后通过在线交互学习轻量级的触觉残差策略。大量实验表明，OmniTacTune在多种接触丰富的任务、视觉基础策略和触觉表征中具有良好的泛化能力。在四个现实世界的接触丰富任务中，它将视觉基础策略的成功率从5-40%提升至85-100%，时间为40-80分钟，展示了将触觉反馈适应于可扩展视觉机器人策略的有效路径。项目页面：https://colinyu1.github.io/omnitactune-site/

View on arXiv Download PDF AI Translation

cs.RO / 36 / 2607.03751

Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models

三思而后行：将树搜索提炼为冻结的视觉-语言-动作模型的动作评估

Xie, Xinyi, Hu, Zican, Liu, Zhanyu, Dong, Yicheng, Wu, Wenhao, Sun, Zhenhong, Li, Haoran, Chen, Chunlin, Wang, Zhi, Wang, Pichao

Abstract

Vision-Language-Action (VLA) models acquire broad embodied capabilities through large-scale pretraining, yet their generalization remains far more fragile than that of LLMs and VLMs. The prevailing remedy, post-training via supervised fine-tuning or reinforcement learning, improves task-specific performance but narrows the generalist capability that makes pretraining valuable. We identify a key bottleneck: VLA failures stem not only from action generation but also from action evaluation. A diagnostic pass@k study confirms that frozen VLAs already contain competent behaviors in their output distribution, with overall success rates rising from 33% at pass@1 to 92% at pass@32. Inspired by this, we propose SVA (Search, Value, and Act), a simple framework that equips frozen VLA policies with long-term consequence awareness. SVA first uses Monte-Carlo tree search in simulation to fully explore the VLA's output distribution and collect diverse trajectories annotated with empirical returns; this knowledge is then distilled into a lightweight Q-value model that predicts the expected consequence of candidate actions; at deployment, the frozen VLA proposes multiple candidates and the evaluator selects the one with the highest uncertainty-regularized Q-value, requiring no simulator access. By decoupling action proposal from consequence evaluation, SVA preserves the generalization capacity of the VLA backbone while substantially improving task success rates. Experiments across embodied benchmarks show that SVA consistently improves generalization on unseen tasks and exhibits strong test-time scaling behavior. Strikingly, SVA enables a 9B VLA to outperform a 27B VLA by 7 points at 27% lower inference latency, suggesting that scaling test-time evaluation is more cost-effective than scaling model size.

Chinese Translation

视觉-语言-动作（VLA）模型通过大规模预训练获得广泛的具身能力，但其泛化能力仍然远不如大型语言模型（LLMs）和视觉-语言模型（VLMs）稳健。当前的解决方案是通过监督微调或强化学习进行后训练，这虽然提高了特定任务的性能，但却缩小了预训练所带来的通用能力。我们识别出一个关键瓶颈：VLA的失败不仅源于动作生成，还源于动作评估。一项诊断性pass@k研究确认，冻结的VLA在其输出分布中已经包含了有效的行为，整体成功率从pass@1的33%上升到pass@32的92%。受到此启发，我们提出了SVA（搜索、价值与行动），这是一个简单的框架，赋予冻结的VLA策略长远后果意识。SVA首先在模拟中使用蒙特卡洛树搜索全面探索VLA的输出分布，并收集带有经验回报的多样化轨迹；然后将这些知识提炼为一个轻量级的Q值模型，用于预测候选动作的预期后果；在部署时，冻结的VLA提出多个候选项，评估者选择具有最高不确定性正则化Q值的选项，无需访问模拟器。通过将动作提议与后果评估解耦，SVA保留了VLA主干的泛化能力，同时显著提高了任务成功率。在具身基准上的实验表明，SVA在未见任务上的泛化能力始终有所提升，并表现出强大的测试时扩展性。值得注意的是，SVA使得一个9B的VLA在27%的推理延迟下超越了一个27B的VLA，提升了7个点，这表明扩展测试时评估比扩展模型规模更具成本效益。

View on arXiv Download PDF AI Translation

cs.RO / 37 / 2607.03758

Occluding the Solution Space: Planner-Agnostic Adversarial Attacks on Tolerance-Aware Manipulation

遮蔽解空间：无规划者的对抗性攻击在容忍度感知操控中的应用

Tang, Keke, Hao, Tianyu, Peng, Weilong, Jiang, Hao, Wu, Feng, Zhu, Peican, Ji, Jianmin, Tian, Zhihong

Abstract

Adversarial attacks on motion planning are crucial for evaluating and quantifying the intrinsic robustness of robotic manipulation. However, existing approaches are typically limited by restrictive exact-pose objectives and their reliance on planner-in-the-loop queries. To address these limitations, we propose a planner-agnostic attack framework for tolerance-aware manipulation. Our approach shifts the evaluation paradigm to task-level feasibility over goal regions, efficiently inserting adversarial obstacles without requiring oracle access to the victim system. Offline, we characterize the robot's intrinsic workspace capabilities via a kinematic occupancy heatmap, which encodes the density of feasible trajectories and robustness priors without invoking a specific planner. Online, we formulate the attack as a budgeted maximum-coverage optimization, strategically deploying obstacles subject to explicit geometric constraints to occlude the solution space. Extensive experiments across simulation and real-world scenarios demonstrate that our method reliably induces planning failures, significantly outperforming planner-in-the-loop baselines in both computational efficiency and attack efficacy.

Chinese Translation

对运动规划的对抗性攻击对于评估和量化机器人操控的内在鲁棒性至关重要。然而，现有方法通常受到严格的精确姿态目标和依赖于规划者循环查询的限制。为了解决这些限制，我们提出了一种无规划者的攻击框架，用于容忍度感知操控。我们的方法将评估范式转向任务级可行性，聚焦于目标区域，能够高效地插入对抗性障碍，而无需对受害系统进行oracle访问。在离线阶段，我们通过运动学占用热图表征机器人的内在工作空间能力，该热图编码了可行轨迹的密度和鲁棒性先验，而不依赖于特定的规划者。在在线阶段，我们将攻击形式化为预算最大覆盖优化，战略性地部署障碍物，遵循明确的几何约束，以遮蔽解空间。大量在模拟和真实场景中的实验表明，我们的方法可靠地导致规划失败，在计算效率和攻击有效性方面显著优于规划者循环基线。

View on arXiv Download PDF AI Translation

cs.RO / 38 / 2607.03792

From Region Arrival to Instance-Level Grounding in Vision-and-Language Navigation

从区域到实例级的视觉与语言导航基础

Shi, Xiangyu, Yang, Ruoxi, Tao, Wei, Zhang, Jiwen, Qiao, Yanyuan, Wu, Qi

Abstract

Vision-and-Language Navigation (VLN) agents may satisfy conventional success criteria while still failing to establish reliable object-level grounding, because current evaluation protocols mainly reward stopping within a 3-meter radius and largely ignore the agent's final orientation and target visibility. We formalize this limitation as the Last-3-Meter Grounding Gap and introduce three instance-centric metrics to quantify proximity precision, target visibility, and final-view grounding. To mitigate this gap, we propose REALM (Region-to-Entity Alignment for Last-3-Meter Navigation), a plug-and-play, architecture-agnostic refinement module that decouples fine-grained target approaching from long-horizon navigation. REALM uses a visibility-aware stopping strategy to reduce premature termination and improve final viewpoint alignment. We further construct REVERIE-AIM, which provides object-instance-level goals and 180K short-horizon training samples for final-stage target approaching. Extensive evaluations across four diverse VLN backbones show that REALM consistently improves proximity precision and visual grounding success, demonstrating its broad applicability.

Chinese Translation

视觉与语言导航（VLN）代理可能在满足传统成功标准的同时，仍然无法建立可靠的对象级基础，因为当前的评估协议主要奖励在3米半径内停止，且在很大程度上忽略了代理的最终朝向和目标可见性。我们将这一局限性形式化为最后3米基础差距（Last-3-Meter Grounding Gap），并引入三种以实例为中心的指标来量化接近精度、目标可见性和最终视角基础。为了缩小这一差距，我们提出了REALM（最后3米导航的区域-实体对齐），这是一种即插即用、与架构无关的精细化模块，它将细粒度的目标接近与长距离导航解耦。REALM采用了一种考虑可见性的停止策略，以减少过早终止并改善最终视点对齐。我们进一步构建了REVERIE-AIM，该系统提供对象实例级目标和18万条短期训练样本，以支持最终阶段的目标接近。对四种不同VLN基础模型的广泛评估表明，REALM在接近精度和视觉基础成功率方面始终表现出改善，展示了其广泛的适用性。

View on arXiv Download PDF AI Translation

cs.RO / 39 / 2607.03828

ObjRetarget: An Object-Aware Motion Retargeting Framework with Anthropomorphic Arm Constraints and Polyhedral Hand Modeling

ObjRetarget：一种具有类人手臂约束和多面体手部建模的对象感知运动重定向框架

Lai, Yuanchuan, Gao, Qing, Liang, Ziyan, Hu, Junjie, Ju, Zhaojie

Abstract

Learning robot dexterous manipulation from human manipulation videos requires reliably retargeting human intent to executable robot actions while maintaining stable hand-object contact, which remains a key challenge in embodied intelligence. Existing retargeting methods often ignore explicit contact modeling or rely on reinforcement learning, resulting in limited accuracy and generalization. To address this, we propose ObjRetarget, a human-to-robot motion retargeting framework for learning robot dexterous manipulation from human videos, which integrates anthropomorphic arm trajectory constraints with structured hand-object geometric modeling. For arm motion, reference trajectories extracted from human videos are used for initialization, followed by anthropomorphic constraints and redundancy-aware optimization to generate natural and accurate movements. For hand manipulation, ObjRetarget represents multi-finger contacts using polytope clusters and preserves contact structure through geometric invariants to improve stability. Experiments on real robots show that ObjRetarget improves manipulation success rates and contact stability across multiple dexterous tasks, and generalizes well to different demonstrations, object poses, and task settings.

Chinese Translation

从人类操作视频中学习机器人灵巧操作需要可靠地将人类意图重定向为可执行的机器人动作，同时保持稳定的手-物体接触，这仍然是具身智能中的一个关键挑战。现有的重定向方法往往忽视显式接触建模或依赖强化学习，导致准确性和泛化能力有限。为了解决这个问题，我们提出了ObjRetarget，一个用于从人类视频中学习机器人灵巧操作的人机运动重定向框架，它将类人手臂轨迹约束与结构化的手-物体几何建模相结合。对于手臂运动，提取自人类视频的参考轨迹用于初始化，随后通过类人约束和冗余感知优化生成自然且准确的运动。对于手部操作，ObjRetarget使用多面体簇表示多指接触，并通过几何不变性保持接触结构以提高稳定性。在真实机器人上的实验表明，ObjRetarget提高了多种灵巧任务的操作成功率和接触稳定性，并且在不同演示、物体姿态和任务设置下具有良好的泛化能力。

View on arXiv Download PDF AI Translation

cs.RO / 40 / 2607.03865

High-Fidelity One-Step Generative Visuomotor Policy via Recursive Correction, Frequency Consistency, and Contrastive Flow Matching

通过递归校正、频率一致性和对比流匹配实现高保真一步生成视觉运动策略

Chen, Yuran, Cai, Xinye, Gong, Zhonglin, Huang, Yang

Abstract

Generative models such as diffusion and flow matching have advanced robotic visuomotor policies by modeling multimodal action distributions, but their multi-step sampling or ODE solving introduces inference latency. Existing one-step acceleration methods often compress the whole generation process into a single large update, leading to spatial deviation, frequency distortion, and mode averaging. This paper proposes a high-fidelity one-step generative visuomotor policy framework that addresses these issues with three complementary mechanisms. Recursive Consistent Action Flow (RCAF) uses recursive correction to compensate for spatial truncation errors and align one-step predictions with refined flow trajectories. Dual-Timestep Frequency Consistency (DTFC) preserves high-frequency manipulation details through adaptive spectral consistency across flow timesteps. Contrastive Flow Matching (CFM) separates entangled action flows with a margin-based repulsive objective, reducing ambiguous actions in multimodal manipulation. Experiments on RoboTwin, RoboTwin 2.0, Adroit, DexArt, and real-world robot platforms show that the proposed method achieves competitive or superior performance compared with strong 10-step generative policy baselines while requiring only one forward pass (1 NFE), enabling low-latency visuomotor control.

Chinese Translation

生成模型如扩散和流匹配通过建模多模态动作分布推动了机器人视觉运动策略的发展，但它们的多步采样或常微分方程求解引入了推理延迟。现有的一步加速方法通常将整个生成过程压缩为一次大的更新，导致空间偏差、频率失真和模式平均。本文提出了一种高保真一步生成视觉运动策略框架，通过三种互补机制解决这些问题。递归一致动作流（Recursive Consistent Action Flow, RCAF）利用递归校正来补偿空间截断误差，并将一步预测与精细化的流轨迹对齐。双时间步频率一致性（Dual-Timestep Frequency Consistency, DTFC）通过在流时间步之间的自适应谱一致性来保留高频操作细节。对比流匹配（Contrastive Flow Matching, CFM）通过基于边际的排斥目标分离纠缠的动作流，减少多模态操作中的模糊动作。在RoboTwin、RoboTwin 2.0、Adroit、DexArt和真实世界机器人平台上的实验表明，所提方法在性能上与强大的10步生成策略基线相比具有竞争力或优越性，同时仅需一次前向传播（1 NFE），实现低延迟的视觉运动控制。

View on arXiv Download PDF AI Translation

cs.RO / 41 / 2607.03920

LH-AVLN: A Benchmark for Long-Horizon Audio-Visual-Language Navigation

LH-AVLN：长时间音视频语言导航基准

Chen, Rufeng, Chang, Yue, Shao, Zili, Zhang, Zhaofan, Chen, Hechang, Xiong, Hui, Xie, Sihong

Abstract

Embodied navigation is moving toward long-horizon missions, yet existing long-horizon benchmarks are largely acoustically silent, and audio-visual navigation tasks typically focus on a single goal. We introduce LH-AVLN, a benchmark for Long-Horizon Audio-Visual-Language Navigation that combines multi-goal mission execution, heterogeneous goal specifications, and persistent spatialized acoustic cues. In LH-AVLN, an agent receives a global mission of two to four goals specified by category, language description, or reference image, and navigates with RGB-D observations, pose, and binaural audio in indoor 3D environments. The benchmark supports both ordered and unordered missions, where alternating goal-associated sounds can guide non-line-of-sight search but may also become distractors as mission progress changes. We further develop PAG-Nav, a training-free reference agent that maintains a temporal uniform semantic map and performs progressive goal-state planning, using sound for search while reserving completion for visual-semantic verification. Experiments show that existing vision-language, memory-based, and audio-visual agents struggle to complete full LH-AVLN missions, and that PAG-Nav provides a stronger diagnostic baseline while leaving substantial room for future progress.

Chinese Translation

具身导航正朝着长时间任务发展，但现有的长时间基准在很大程度上是声学沉默的，而音视频导航任务通常集中于单一目标。我们引入了LH-AVLN，这是一个长时间音视频语言导航的基准，结合了多目标任务执行、异质目标规格和持久的空间化声学线索。在LH-AVLN中，智能体接收一个由类别、语言描述或参考图像指定的两到四个目标的全球任务，并在室内3D环境中使用RGB-D观测、姿态和双耳音频进行导航。该基准支持有序和无序任务，其中交替的目标相关声音可以引导非视线搜索，但随着任务进展的变化也可能成为干扰因素。我们进一步开发了PAG-Nav，一个无训练的参考智能体，维护一个时间均匀的语义地图，并执行渐进的目标状态规划，利用声音进行搜索，同时保留视觉语义验证的完成。实验表明，现有的视觉-语言、基于记忆和音视频智能体在完成完整的LH-AVLN任务时存在困难，而PAG-Nav提供了更强的诊断基线，同时为未来的进展留出了相当大的空间。

View on arXiv Download PDF AI Translation

cs.RO / 42 / 2607.03941

WSA$_1$: a 3D-Centric World-Spatial-Action Model for Generalizable Robot Control

WSA$_1$: 一种用于可泛化机器人控制的3D中心世界空间动作模型

Jiang, Jiahao, Zhang, Jianing, Yin, Zhenhan, Chen, Ruidong, Wang, Sen, Yu, Zhaoshu, Zeng, Pengpeng, Cao, Xiaofeng, Wang, Xuanhan, Song, Jingkuan, Shen, Heng Tao

Abstract

Recent advances in embodied AI have established robot foundation models (RFMs) as the dominant approach for generalist robotic systems to date. By leveraging imitation learning on extensive robot demonstrations, RFMs have achieved impressive capabilities in mapping visual observations and language instructions to continuous robotic actions. However, current RFMs lack an inherent ability to reason about physical dynamics and the causal effects of robot behaviors on the 3D physical world. This creates a fundamental mismatch between 2D-centric visual perception and 3D-centric embodied interaction, severely limiting the generalization ability of RFMs in real-world tasks.To address this gap, we present WSA$_1$, a novel RFM built upon proposed 3D-Centric World-Spatial-Action modeling paradigm. It not only learns 3D world-aware visual thought for future robot behaviors, but also models mutual constraints between 3D world state transitions and robotic actions to enhance behavior generalization. Notably, WSA$_1$ achieves highly data-efficient pre-training with 6k hours of expert demonstration data (only 1k hours from real robot), while delivering competitive manipulation performance (93% success rate) on RoboTwin2.0 simulation benchmark and achieving +20% average boosted performance over state-of-the-art RFMs on real-world robot control tasks. These results reveal that generalizable RFM can be attained without large-scale real robot data when paired with 3D-centric world-action joint modeling, which offers a practical and affordable pathway to generalist robotic systems.

Chinese Translation

近年来，具身人工智能的进展使得机器人基础模型（RFM）成为迄今为止通用机器人系统的主导方法。通过利用大量机器人演示进行模仿学习，RFM在将视觉观察和语言指令映射到连续机器人动作方面取得了令人印象深刻的能力。然而，当前的RFM缺乏对物理动态及机器人行为对3D物理世界因果影响的内在推理能力。这导致了2D中心视觉感知与3D中心具身交互之间的根本不匹配，严重限制了RFM在现实任务中的泛化能力。为了解决这一问题，我们提出了WSA$_1$，一种基于提出的3D中心世界空间动作建模范式的新型RFM。它不仅学习未来机器人行为的3D世界感知视觉思维，还建模3D世界状态转变与机器人动作之间的相互约束，以增强行为的泛化能力。值得注意的是，WSA$_1$在仅使用6000小时专家演示数据（其中仅1000小时来自真实机器人）的情况下，实现了高度数据高效的预训练，同时在RoboTwin2.0仿真基准上提供了竞争力的操控性能（93%的成功率），并在现实机器人控制任务上实现了比最先进的RFM平均提升20%的性能。这些结果表明，当结合3D中心世界-动作联合建模时，可以在没有大规模真实机器人数据的情况下实现可泛化的RFM，这为通用机器人系统提供了一条实用且经济的路径。

View on arXiv Download PDF AI Translation

cs.RO / 43 / 2607.03964

Worldscape-MoE: A Unified Mixture-of-Experts World Model for Scalable Heterogeneous Action Control

Worldscape-MoE：一种统一的混合专家世界模型，用于可扩展的异构动作控制

Fang, Jianjie, Xu, Yongyan, Wang, Ziyou, Gao, Chen, Huang, Yuchao, Wang, Zhaolu, Tang, Rongze, Jia, Mingyuan, Zhao, Baining, Zhang, Weichen, Zhang, Xin, Su, Haisheng, Shang, Yu, Wu, Wei, Chen, Xinlei, Li, Yong

Abstract

World models are rapidly becoming a core infrastructure for embodied intelligence and interactive agents: they provide controllable simulators in which agents can perceive, act, forecast, and acquire scalable experience. Yet current video generation world models are still organized around isolated control interfaces, such as camera trajectories, robot actions, or hand-joint signals. This fragmentation is increasingly a scaling bottleneck. The central challenge is not the absence of controllable generators, but the lack of a unified and extensible learning framework that can absorb heterogeneous action supervision while preserving a shared model of world dynamics. In this work, we introduce Worldscape-MoE, a Mixture-of-Experts world model built on Diffusion Transformers for scalable heterogeneous action control. Our key observation is that different controls specify different interfaces to the same underlying world: although their representations differ, they constrain shared physical regularities, scene dynamics, and interaction semantics. Worldscape-MoE operationalizes this observation through modality-aware control injection, shared and control-specific experts, and a progressive MoE tuning strategy that supports continual extension to new action modalities. Experiments across locomotion, robotic manipulation, and egocentric hand control show that heterogeneous supervision improves rather than interferes with individual control capabilities. Worldscape-MoE achieves strong results on WorldArena, improves locomotion and hand-control metrics, exhibits robust out-of-distribution generalization, and demonstrates scaling behavior as additional control data and experts are integrated.

Chinese Translation

世界模型正迅速成为具身智能和交互代理的核心基础设施：它们提供可控的模拟器，使代理能够感知、行动、预测并获取可扩展的经验。然而，目前的视频生成世界模型仍然围绕孤立的控制接口进行组织，例如相机轨迹、机器人动作或手关节信号。这种碎片化日益成为扩展的瓶颈。核心挑战不在于缺乏可控的生成器，而在于缺乏一个统一且可扩展的学习框架，能够在保留共享世界动态模型的同时吸收异构的动作监督。在本研究中，我们介绍了Worldscape-MoE，这是一种基于扩散变换器的混合专家世界模型，旨在实现可扩展的异构动作控制。我们的关键观察是，不同的控制指定了对同一基础世界的不同接口：尽管它们的表示不同，但它们约束了共享的物理规律、场景动态和交互语义。Worldscape-MoE通过模态感知控制注入、共享和控制特定的专家，以及支持对新动作模态进行持续扩展的渐进式MoE调优策略来实现这一观察。跨越运动、机器人操控和自我中心手部控制的实验表明，异构监督改善了个体控制能力，而不是干扰它们。Worldscape-MoE在WorldArena上取得了强劲的结果，提高了运动和手部控制指标，展现出强大的分布外泛化能力，并在整合额外控制数据和专家时表现出扩展行为。

View on arXiv Download PDF AI Translation

cs.RO / 44 / 2607.03987

Fast Asymptotically Optimal Kinodynamic Planning via Vectorization

通过向量化实现快速渐近最优的运动规划

Gao, Yitian, Lu, Andrew, Kingston, Zachary

Abstract

Sampling-based motion planners have been shown to be effective for systems with complex kinodynamic constraints and high dimensionality. However, these algorithms struggle to achieve real-time performance, leading to recent efforts to parallelize planning. While GPU-accelerated planners have achieved significant speedups, existing approaches require specialized CUDA programming that limits accessibility and portability. We present Parallel Asymptotically Optimal Kinodynamic RRT (PAKR), a massively parallel kinodynamic planner leveraging JAX and the XLA compiler to achieve GPU acceleration through standard Python tooling. By combining our parallel planner with the AO-x meta-algorithm, we achieve asymptotic optimality through fast iterative replanning. We provide a theoretical analysis of probabilistic completeness, analyze the effects of batch size and branching factor on convergence, and demonstrate scalability to complex dynamics using the MuJoCo-XLA simulator. Experiments show competitive runtimes with state-of-the-art GPU planners and superior solution quality.

Chinese Translation

基于采样的运动规划器已被证明在具有复杂运动动力学约束和高维度的系统中有效。然而，这些算法在实现实时性能方面存在困难，导致最近的努力集中在规划的并行化上。尽管基于GPU的规划器已经实现了显著的加速，但现有方法需要专门的CUDA编程，这限制了其可访问性和可移植性。我们提出了并行渐近最优运动动力学RRT（Parallel Asymptotically Optimal Kinodynamic RRT，PAKR），这是一种利用JAX和XLA编译器通过标准Python工具实现GPU加速的大规模并行运动规划器。通过将我们的并行规划器与AO-x元算法相结合，我们通过快速的迭代重规划实现了渐近最优性。我们提供了概率完整性的理论分析，分析了批量大小和分支因子对收敛性的影响，并使用MuJoCo-XLA模拟器展示了对复杂动力学的可扩展性。实验结果表明，与最先进的GPU规划器相比，运行时间具有竞争力，并且解决方案质量更高。

View on arXiv Download PDF AI Translation

cs.RO / 45 / 2607.04124

Conflict-Based Lazy Search for Fast Multi-Manipulator Planning

基于冲突的惰性搜索用于快速多操控器规划

Zheng, Dongliang, Wang, Zhipeng, Wang, Siqi, Lu, Yuxi, He, Bin, Wang, Hesheng, Tsiotras, Panagiotis

Abstract

Employing multiple manipulators can boost efficiency and accomplish tasks that a single manipulator cannot do. However, real-time planning for multiple manipulators in a cluttered workspace still poses significant challenges for planning algorithms. This article proposes a new planning algorithm called Conflict-Based Lazy Search (CBLS) for multimanipulator planning. CBLS is built on Conflict-Based Search (CBS), an efficient multiagent pathfinding (MAPF) algorithm that has shown an order of magnitude speedup over previous approaches [1], [2]. CBS addresses MAPF by solving many single-agent pathfinding (SAPF) problems. Thus, its planning time directly depends on the efficiency of the SAPF algorithm adopted. Our CBLS algorithm enhances CBS with precomputation and lazy search. First, a lazily evaluated graph with controlled sparsity is precomputed for a single manipulator. Second, we propose the Lazy Edged-based A* (LEA*) for efficient SAPF. Since edge evaluation is the computational bottleneck of manipulator planning, LEA* uses lazy search and an edge queue to reduce the number of edge evaluations. We show that LEA* is optimally vertex efficient and has improved edge efficiency compared to A*. We apply the proposed CBLS to multi-manipulator planning problems and show its superior performance by comparing it with CBS and a sampling-based algorithm, namely, RRT-Connect.

Chinese Translation

使用多个操控器可以提高效率，并完成单个操控器无法完成的任务。然而，在杂乱的工作空间中进行多操控器的实时规划仍然对规划算法提出了重大挑战。本文提出了一种新的规划算法，称为基于冲突的惰性搜索（Conflict-Based Lazy Search, CBLS），用于多操控器规划。CBLS建立在基于冲突的搜索（Conflict-Based Search, CBS）之上，CBS是一种高效的多智能体路径规划（Multi-Agent Pathfinding, MAPF）算法，其速度比以往方法提高了一个数量级 [1], [2]。CBS通过解决多个单智能体路径规划（Single-Agent Pathfinding, SAPF）问题来处理MAPF。因此，其规划时间直接依赖于所采用的SAPF算法的效率。我们的CBLS算法通过预计算和惰性搜索增强了CBS。首先，为单个操控器预计算了一个稀疏性受控的惰性评估图。其次，我们提出了惰性边基A*（Lazy Edged-based A*, LEA*）以实现高效的SAPF。由于边评估是操控器规划的计算瓶颈，LEA*使用惰性搜索和边队列来减少边评估的次数。我们证明了LEA*在顶点效率上是最优的，并且在边效率上相较于A*有所改善。我们将提出的CBLS应用于多操控器规划问题，并通过与CBS和基于采样的算法（即RRT-Connect）进行比较，展示其优越的性能。

View on arXiv Download PDF AI Translation

cs.RO / 46 / 2607.04144

Semantic-Guided Progressive Object Removal with Gaussian Splatting

基于语义引导的渐进式物体移除与高斯点云技术

Huang, Xianliang, Xiao, Chen, Ni, Yuanxiang, Liu, Guanming, Liu, Mingkai, Fan, Dikai, Liu, Xiao, Zhang, Hao

Abstract

Removing unwanted objects from reconstructed 3D scenes is an important task in computer vision, supporting applications in AR/VR, robotics, and digital content creation. Existing methods typically complete the entire masked region in a single step and without effectively utilizing semantic information from other views, leading to difficulties in handling complex geometric details and textures. In this work, we propose a novel framework that integrates Semantic-guided Block Matching (SBM) and Region-Wise Progressive Refinement (RPR) for high-quality 3D object removal. First, we leverage DINOv2 to encode semantic guidance from multi-view observations, and the best match tokens are decoded to complete missing regions in the target view while maintaining cross-view consistency. Second, we introduce a RPR strategy that segments the target mask into multiple subregions and selectively refines those with poor visual quality. Our method is built upon Gaussian Splatting, ensuring high-fidelity scene reconstruction with efficient computation. Experimental results demonstrate that our approach outperforms existing Gaussian-based methods in terms of perceptual quality and coherence in 3D object removal.

Chinese Translation

从重建的三维场景中移除不需要的物体是计算机视觉中的一项重要任务，支持增强现实/虚拟现实（AR/VR）、机器人技术和数字内容创作等应用。现有方法通常在单一步骤中完成整个遮罩区域，且未有效利用来自其他视角的语义信息，导致在处理复杂几何细节和纹理时遇到困难。在本研究中，我们提出了一种新颖的框架，结合了基于语义的块匹配（Semantic-guided Block Matching, SBM）和区域渐进式细化（Region-Wise Progressive Refinement, RPR），以实现高质量的三维物体移除。首先，我们利用 DINOv2 从多视角观察中编码语义引导，解码最佳匹配标记以完成目标视图中的缺失区域，同时保持跨视角的一致性。其次，我们引入了一种 RPR 策略，将目标遮罩分割为多个子区域，并选择性地细化那些视觉质量较差的区域。我们的方法基于高斯点云技术，确保高保真度的场景重建和高效的计算。实验结果表明，我们的方法在三维物体移除的感知质量和一致性方面优于现有的基于高斯的方法。

View on arXiv Download PDF AI Translation

cs.RO / 47 / 2607.04146

!Imperio, smolVLA: The Implications of Data Poisoning on Open Source Robotics

Imperio, smolVLA：数据中毒对开源机器人技术的影响

Bühler, Stefan, Schutera, Mark

Abstract

This work establishes that trigger-word data poisoning of vision language action models is practical, while at the same time the open-source robotics ecosystem holds trust assumptions about community contributions. A few poisoned samples can silently embed a backdoor that disables a robot on command. We evaluate this threat against smolVLA on a real-world pick-and-place task, training on three poison ratios and evaluating across different prompts on the LeRobot platform. Three poisoned episodes in 320 clean episodes suffice for a complete denial of service. Success rate drops to 0.0 plus minus 0.0% across all trigger-word conditions and the robot locks into a fixed joint configuration rather than executing any task-relevant motion. Clean-prompt behaviour holds at approx. 50% success rate across all poison ratios, confirming the attack is stealthy under normal operation. A single poisoned episode already reduces success rate to 6.7 plus minus 6.7%. The robot still moves, but no longer completes the task. The attack generalises to front, middle, and end trigger placements despite training exclusively on front-placed triggers. These findings establish that the threat is practical, low-cost, and stealthy, and warrant treating dataset provenance as a first-class concern in open-source robotics ecosystems.

Chinese Translation

本研究表明，视觉语言动作模型的触发词数据中毒是可行的，同时开源机器人生态系统对社区贡献持有信任假设。少量中毒样本可以悄无声息地嵌入后门，使机器人在接到指令时失效。我们在真实的抓取与放置任务中评估了这一威胁，针对 smolVLA 进行了三种中毒比例的训练，并在 LeRobot 平台上对不同提示进行了评估。在320个干净的实验中，仅需三个中毒实验即可完全拒绝服务。所有触发词条件下的成功率降至 0.0 ± 0.0%。机器人锁定在固定的关节配置中，而未执行任何与任务相关的动作。在所有中毒比例下，干净提示的行为保持在约 50% 的成功率，确认该攻击在正常操作下是隐蔽的。单个中毒实验已经将成功率降低至 6.7 ± 6.7%。机器人仍然能够移动，但不再完成任务。尽管训练仅针对前置触发器，但攻击仍能推广到前、中、后触发器的放置。这些发现表明，该威胁是实际存在的、低成本的且隐蔽，值得在开源机器人生态系统中将数据集来源视为首要关注事项。

View on arXiv Download PDF AI Translation

cs.RO / 48 / 2607.04162

ACE: Agentic Control for Embodied Manipulation via Zero-shot Workflow Reasoning

ACE：通过零-shot工作流推理实现具身操控的自主控制

Lei, Iok Tong, Li, QianZhi, Yap, Ying Jie, Zhang, Yujie, Zhong, Rui, Gui, Haichao, Liu, Xiaolong, Deng, Zhidong

Abstract

Open-ended tabletop manipulation requires agents to not only understand natural language but also adapt to dynamic environments and execution failures. We present ACE (Agentic Control for Embodied Manipulation), a zero-shot workflow reasoning framework for tabletop pick-and-place from natural language. Rather than relying on direct low-level action mapping, ACE combines agentic workflow reasoning with two robot-facing executable skills: a visual grounding interface and a reusable pick-and-place primitive. To bridge semantic reasoning and physical control, the active sub-goal is grounded into a mask-mediated vision-action interface. This unified mask specifies the target object and destination, is tracked over time, exposed for human verification, and ultimately passed to a task-agnostic downstream policy for execution. Crucially, ACE operates in a closed loop supported by a multi-timescale memory. After an action is executed, the system automatically verifies whether the intended sub-goal succeeded, using the outcome to advance, retry, repair, or replan. This enables online adaptation to user corrections, scene changes, and physical failures. We evaluate ACE on logically complex, long-horizon tasks, including zero-shot multi-step equation formation with number cubes and constraint-based object retrieval. ACE demonstrates task-level zero-shot generalization on novel semantic constraints and randomized tabletop scenes without task-specific retraining. Specifically, while standard end-to-end baselines struggle to complete these logically demanding tasks, ACE achieves a 50% success rate in equation formation and a 70% success rate in constraint retrieval. This contrast demonstrates that explicit workflow reasoning and mask-mediated control offer a robust, practical route toward adaptable robotic manipulation.

Chinese Translation

开放式桌面操控要求智能体不仅理解自然语言，还需适应动态环境和执行失败。我们提出了ACE（自主控制用于具身操控），这是一个用于从自然语言进行桌面拾取与放置的零-shot工作流推理框架。ACE并不依赖于直接的低级动作映射，而是将自主工作流推理与两种面向机器人的可执行技能相结合：视觉定位接口和可重用的拾取与放置原语。为了桥接语义推理与物理控制，主动子目标被嵌入到一个基于掩码的视觉-动作接口中。这个统一的掩码指定了目标物体和目的地，随着时间的推移被跟踪，供人类验证，并最终传递给一个与任务无关的下游策略以进行执行。至关重要的是，ACE在一个由多时间尺度记忆支持的闭环中运行。在执行一个动作后，系统会自动验证预期子目标是否成功，利用结果来推进、重试、修复或重新规划。这使得系统能够在线适应用户的修正、场景变化和物理失败。我们在逻辑复杂的长时间任务上评估ACE，包括使用数字立方体进行零-shot多步骤方程形成和基于约束的物体检索。ACE在新颖的语义约束和随机化的桌面场景中展示了任务级的零-shot泛化，而无需特定任务的再训练。具体而言，尽管标准的端到端基线在完成这些逻辑要求高的任务时遇到困难，ACE在方程形成中实现了50%的成功率，在约束检索中实现了70%的成功率。这一对比表明，明确的工作流推理和基于掩码的控制提供了一条稳健且实用的途径，朝着可适应的机器人操控迈进。

View on arXiv Download PDF AI Translation

cs.RO / 49 / 2607.04169

Neural LiDAR Bundle Adjustment

神经LiDAR束调整

Hon, Chin Yung Anson, Zhang, Kaicheng, Wang, Sen

Abstract

Recent research has achieved remarkable novel view rendering and scene reconstruction results with Neural Radiance Field (NeRF), including extensions to the LiDAR modality. Few studies have, however, explored the key design differences between RGB NeRFs and LiDAR NeRFs, particularly considering their underlying working principles. In this work, we provide both theoretical and empirical evidence suggesting that the density of volume sampling plays a significant role in LiDAR NeRF. Based on this finding, we propose a novel Neural LiDAR Bundle Adjustment (NeLD-BA) algorithm, which is tailored using efficient volume sampling of LiDAR rays for joint optimization of LiDAR map and poses. Extensive experiments are performed using the Newer College and FusionPortable datasets to demonstrate the proposed NeLD-BA's state-of-the-art performance in multi-view point cloud registration and 3D mapping. We will open-source our code for the community.

Chinese Translation

近期研究在神经辐射场（Neural Radiance Field, NeRF）方面取得了显著的全新视图渲染和场景重建成果，包括对LiDAR模态的扩展。然而，关于RGB NeRF和LiDAR NeRF之间关键设计差异的研究仍然较少，特别是在考虑其基本工作原理的情况下。在本研究中，我们提供了理论和实证证据，表明体积采样密度在LiDAR NeRF中起着重要作用。基于这一发现，我们提出了一种新颖的神经LiDAR束调整（Neural LiDAR Bundle Adjustment, NeLD-BA）算法，该算法通过高效的LiDAR光线体积采样进行定制，以实现LiDAR地图和姿态的联合优化。我们使用Newer College和FusionPortable数据集进行了大量实验，以证明所提出的NeLD-BA在多视角点云配准和3D映射中的最先进性能。我们将为社区开源我们的代码。

View on arXiv Download PDF AI Translation

cs.RO / 50 / 2607.04171

XS-VLA: Coupling Coarse-grained Spatial Distillation with Latent Flow Matching for Lightweight Robotic Control

XS-VLA：将粗粒度空间蒸馏与潜在流匹配结合用于轻量级机器人控制

Tong, Lei Iok, Xie, Qingchen, Huang, Wei, Yap, Ying Jie, Zhang, Yujie, Li, Qianzhi, Liu, Xiaolong, Deng, Zhidong

Abstract

Large Vision-Language Models (LVLMs) have shown strong multimodal understanding and spatial grounding, but their computational cost limits real-time robotic control. In contrast, lightweight models are suitable for edge deployment but often suffer from "spatial blindness", namely weak native spatial prediction ability. Training Vision-Language-Action (VLA) models on mixed human demonstrations can also degrade policy performance due to highly diverse behaviors. To address these limitations, we propose XS-VLA, a two-stage framework for efficient and spatially grounded robotic manipulation. First, we distill spatial semantic knowledge from Qwen3-VL-4B into the SmolVLM2-0.25B backbone by fine-tuning on curated coarse-grained spatial descriptions, turning the lightweight model into a spatially grounded engine. Second, we use this enhanced backbone to condition a Latent Flow Matching policy. Unlike deterministic controllers, our policy combines a Conditional Variational Autoencoder (CVAE) with Flow Matching dynamics to model complex multimodal action distributions. On the LIBERO benchmark, XS-VLA achieves state-of-the-art performance among models with fewer than 0.5B parameters. It improves average success rates by up to 7.2 percent, including a 23 percent gain on LIBERO-Long, over the SmolVLA 0.25B baseline, and outperforms the larger 2.2B vanilla SmolVLA. Ablations show that spatial tuning and generative latent flow control substantially improve lightweight VLA performance, delivering a 3.2 times speedup in mission execution over the previous lightweight flow matching policy.

Chinese Translation

大型视觉-语言模型（LVLMs）在多模态理解和空间定位方面表现出色，但其计算成本限制了实时机器人控制。相比之下，轻量级模型适合边缘部署，但通常存在“空间盲ness”，即原生空间预测能力较弱。在混合人类示范上训练视觉-语言-行动（VLA）模型也可能由于行为的高度多样性而降低策略性能。为了解决这些限制，我们提出了XS-VLA，一个高效且具有空间基础的机器人操作的两阶段框架。首先，我们通过在精心策划的粗粒度空间描述上进行微调，将Qwen3-VL-4B中的空间语义知识蒸馏到SmolVLM2-0.25B主干中，使轻量级模型转变为一个具有空间基础的引擎。其次，我们使用这个增强的主干来调节潜在流匹配策略。与确定性控制器不同，我们的策略结合了条件变分自编码器（CVAE）与流匹配动态，以建模复杂的多模态动作分布。在LIBERO基准测试中，XS-VLA在参数少于0.5B的模型中实现了最先进的性能。它的平均成功率提高了最多7.2个百分点，包括在LIBERO-Long上提高了23个百分点，相较于SmolVLA 0.25B基线，并且超越了更大的2.2B原始SmolVLA。消融实验表明，空间调优和生成潜在流控制显著提升了轻量级VLA的性能，在任务执行中比之前的轻量级流匹配策略实现了3.2倍的加速。

View on arXiv Download PDF AI Translation

cs.RO / 51 / 2607.04234

SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects

SoftVTBench：一种安全意识的视觉-触觉基准测试，用于物理约束下的可变形物体机器人操作

Jing, Bowen, Wang, Mingxin, Hao, Ruiyang, Ge, Chenchen, Shen, Hanwen, He, Junjie, Cui, Yang, Hou, Yiming, Zhou, Weitao, Wang, Jiawei, Li, Minglei, Zhang, Dandan, Zhao, Ding, Liu, Houde, Li, Xiaofan, Liu, Si, Luo, Ping, Yu, Haibao

Abstract

Deformable object manipulation poses challenges beyond task completion: successful execution must also maintain safe physical interaction, holding the object stably without slip or drop while avoiding excessive deformation. However, existing manipulation benchmarks are predominantly success-oriented and rarely evaluate whether a policy remains physically safe throughout execution. We present SoftVTBench, a safety-aware visuo-tactile benchmark for physically constrained deformable object manipulation. Built in Isaac Sim with finite-element-simulated deformable objects, SoftVTBench provides multi-view RGB observations, RGB tactile sensing with marker motion, proprioception, and language instructions, and defines four matched task suites over object type (deformable vs. rigid) and variation axis (object vs. spatial). It separately reports Goal Success and Safety Success; the latter additionally requires no drop and peak deformation below a calibrated object-specific threshold, measured from policy-hidden privileged Finite Element Method (FEM) states. We implement pi0.5-based baselines under this protocol. Experiments show that success-only evaluation substantially overstates policy performance, as a large fraction of goal-completing rollouts still violate physical safety. Furthermore, incorporating tactile sensing improves Safety Success (e.g., from 21.4% to 35.6% on object-centric deformable tasks) and reduces object deformation during execution, while maintaining comparable Goal Success. SoftVTBench provides a reproducible benchmark for studying visuo-tactile deformable manipulation under physical interaction constraints.

Chinese Translation

可变形物体的操作面临着超越任务完成的挑战：成功执行还必须保持安全的物理交互，稳定地握住物体而不滑动或掉落，同时避免过度变形。然而，现有的操作基准测试主要以成功为导向，鲜有评估策略在执行过程中是否保持物理安全。我们提出了SoftVTBench，这是一种安全意识的视觉-触觉基准测试，专为物理约束下的可变形物体操作而设计。SoftVTBench基于Isaac Sim构建，采用有限元模拟的可变形物体，提供多视角RGB观察、带标记运动的RGB触觉感知、自我感知和语言指令，并定义了四个匹配的任务套件，涵盖物体类型（可变形与刚性）和变异轴（物体与空间）。它分别报告目标成功率和安全成功率；后者还要求没有掉落且峰值变形低于经过校准的物体特定阈值，该阈值是从策略隐藏的特权有限元方法（FEM）状态中测得的。我们在该协议下实现了基于pi0.5的基线。实验表明，仅以成功为评估标准会显著夸大策略性能，因为大量完成目标的执行仍然违反了物理安全。此外，结合触觉感知提高了安全成功率（例如，在以物体为中心的可变形任务中从21.4%提高到35.6%），并减少了执行过程中的物体变形，同时保持了相当的目标成功率。SoftVTBench为研究物理交互约束下的视觉-触觉可变形操作提供了一个可重复的基准测试。

View on arXiv Download PDF AI Translation

cs.RO / 52 / 2607.04259

Integrated Graph Search and Model Predictive Control for Smooth and Efficient Path Planning in Autonomous Vehicles

集成图搜索与模型预测控制用于自主车辆的平滑高效路径规划

Bui, Duc-Tien, Nguyen, Ngoc Thinh, Nguyen, Hung Duy, Bi, Dong, Mihalj, Tomislav, Eichberger, Arno

Abstract

Path planning is a fundamental component of autonomous vehicles, where achieving safe, comfortable, and dynamically feasible paths while ensuring computational efficiency remains a significant challenge. This paper presents a sequential path planning framework in which a rough path obtained from graph search is explicitly exploited to guide a Model Predictive Control (MPC)-based path refinement. A rough path is first obtained via Dijkstra search on a discretized grid and is then used to construct a spatially varying convex lateral safety corridor that explicitly captures obstacle avoidance constraints, transforming discrete obstacle avoidance decisions into continuous feasibility constraints for optimization. Within this corridor, an MPC problem is formulated to refine the path, enabling efficient optimization while maintaining path smoothness by penalizing the third-order spatial derivative of the lateral offset over a prediction horizon. The proposed algorithm is evaluated in multiple overtaking scenarios on both straight and curved roads, including cases with single and multiple target vehicles, using high-fidelity environment simulations (i.e., CarMaker). Compared with the previous study, which used polynomial fitting and a quadratic programming method, the proposed approach consistently achieves lower lateral acceleration, curvature, and jerk while reducing computational cost by 28.08% on straight roads and 29.52% on curved roads. These results demonstrate that exploiting graph-search structure within an MPC formulation provides an effective balance between path smoothness and computational efficiency for autonomous vehicles in structured driving environments.

Chinese Translation

路径规划是自主车辆的一个基本组成部分，如何在确保计算效率的同时实现安全、舒适和动态可行的路径仍然是一个重大挑战。本文提出了一种顺序路径规划框架，其中明确利用从图搜索获得的粗略路径来指导基于模型预测控制（Model Predictive Control, MPC）的路径优化。首先通过在离散网格上进行Dijkstra搜索获得粗略路径，然后利用该路径构建一个空间变化的凸侧向安全走廊，该走廊明确捕捉障碍物规避约束，将离散的障碍物规避决策转化为优化的连续可行性约束。在该走廊内，构建了一个MPC问题以优化路径，通过对预测范围内侧向偏移的三阶空间导数进行惩罚，实现高效优化的同时保持路径的平滑性。所提算法在多个超车场景下进行评估，包括直线和曲线道路上的单个和多个目标车辆，使用高保真环境仿真（即CarMaker）。与之前使用多项式拟合和二次规划方法的研究相比，所提方法在直线道路上和曲线道路上分别将侧向加速度、曲率和颤振降低，同时计算成本分别减少了28.08%和29.52%。这些结果表明，在MPC框架内利用图搜索结构为自主车辆在结构化驾驶环境中提供了路径平滑性与计算效率之间的有效平衡。

View on arXiv Download PDF AI Translation

cs.RO / 53 / 2607.04260

FLOAT Drone for Physical Interaction: Lateral Airflow Reduction, Wrench Modeling, and Adaptive Control

用于物理交互的FLOAT无人机：侧向气流减小、扭矩建模与自适应控制

Lin, Junxiao, Zhou, Kehan, Ji, Shuhang, Peng, Yimin, Wang, Shen, Hou, Jialiang, Gao, Fei

Abstract

Aerial physical interaction represents a promising direction for next-generation unmanned aerial vehicles (UAVs), but it requires an aerial platform that can exert contact forces while maintaining stable flight. For close-proximity tasks, this translates into three coupled design requirements: multidimensional wrench generation for stable contact, compactness for maneuverability and safety in confined spaces, and reduced lateral airflow toward the target when generating horizontal force. This article presents FLOAT Drone, a fully actuated coaxial UAV with servo-driven control surfaces for close-proximity physical interaction. The coaxial dual-rotor layout provides a compact propulsion layout, while the control surfaces, immersed in the rotor downwash, generate lateral forces and moments for 6-DoF wrench generation. A force-matched computational fluid dynamics (CFD) comparison with a tilted-rotor alternative quantifies the reduction in target-facing lateral airflow. To account for nonlinear rotor--control-surface coupling in the rotor wake, a high-fidelity polynomial aerodynamic wrench model is identified from precision force measurements and embedded in a constrained nonlinear allocator for real-time wrench tracking. Comparative flight and interaction experiments show that the proposed framework improves control accuracy over linear allocation baselines, rejects ground-effect and payload disturbances, and enables close-proximity drawer push--pull manipulation through a $2~\mathrm{cm}$ handle clearance.

Chinese Translation

空中物理交互代表了下一代无人机（UAVs）的一个有前景的方向，但这需要一个能够施加接触力并保持稳定飞行的空中平台。对于近距离任务，这转化为三个相互耦合的设计要求：为稳定接触生成多维扭矩、在狭小空间内的机动性和安全性所需的紧凑性，以及在施加水平力时减少朝向目标的侧向气流。本文提出了FLOAT无人机，这是一种全驱动的同轴无人机，配备伺服驱动的控制面，用于近距离物理交互。同轴双旋翼布局提供了紧凑的推进布局，而控制面则浸没在旋翼下洗中，生成侧向力和力矩以实现六自由度（6-DoF）扭矩生成。与倾斜旋翼替代方案进行的力匹配计算流体动力学（CFD）比较量化了朝向目标的侧向气流的减少。为了考虑旋翼尾流中非线性旋翼-控制面耦合，基于精确的力测量识别出高保真多项式气动扭矩模型，并嵌入约束非线性分配器中以实现实时扭矩跟踪。比较飞行和交互实验表明，所提出的框架在控制精度上优于线性分配基线，能够抵抗地面效应和载荷干扰，并通过$2~ ext{cm}$的把手间隙实现近距离抽屉推拉操作。

View on arXiv Download PDF AI Translation

cs.RO / 54 / 2607.04265

HALO-WA: Hybrid-Attention Latent-Guided Online Reinforcement Learning for World-Action Models

HALO-WA：用于世界-动作模型的混合注意力潜在引导在线强化学习

Ye, Angen, Ke, Weijie, Wang, Xiaofeng, Chen, Xinze, Ni, Chaojun, Zhao, Guosheng, Wang, Boyuan, Zhu, Zheng, Xie, Junjie, Zhang, Dapeng

Abstract

World-action (WA) models can generate long-horizon action chunks for general-purpose robotic manipulation, but they remain vulnerable to calibration, perception, and contact-dynamics errors in real-world precision tasks, often failing in the final few millimeters of alignment or insertion. We propose HALO-WA, a hybrid-attention latent-guided online reinforcement learning (RL) framework for WA models, which leverages latent features and action priors from the WA generation process through a lightweight actor-critic adapter to enable fast online adaptation to real deployment errors. HALO-WA introduces a hybrid-attention structure that preserves the temporal consistency of action chunks while reading task-relevant information from WA latents conditioned on visual context and end-stage correction requirements, thereby producing refined action chunks. We validate HALO-WA on four real-world precision manipulation tasks, where it improves the average success rate from 26.4\% for WA-base to 87.1\%, outperforming the strongest baseline by 19.2 percentage points while requiring only 45--75 minutes of online training per task. To facilitate reproducibility, we further conduct supplementary simulation experiments in RoboTwin and release the code at https://github.com/YeanRoot/HALO-WA.

Chinese Translation

世界-动作（WA）模型能够为通用机器人操作生成长时间跨度的动作块，但在实际精密任务中，它们仍然容易受到校准、感知和接触动态误差的影响，常常在最后几毫米的对齐或插入中失败。我们提出了HALO-WA，这是一种用于WA模型的混合注意力潜在引导在线强化学习（RL）框架，利用WA生成过程中的潜在特征和动作先验，通过轻量级的演员-评论家适配器，使其能够快速在线适应实际部署中的错误。HALO-WA引入了一种混合注意力结构，在读取与任务相关的信息时，保持动作块的时间一致性，同时根据视觉上下文和最终阶段的校正需求，从WA潜在特征中提取信息，从而生成精细化的动作块。我们在四个实际精密操作任务上验证了HALO-WA，使其平均成功率从WA基础模型的26.4\%提高到87.1\%，超越了最强基线19.2个百分点，同时每个任务仅需45至75分钟的在线训练。为了促进可重复性，我们还在RoboTwin中进行了补充模拟实验，并在https://github.com/YeanRoot/HALO-WA发布了代码。

View on arXiv Download PDF AI Translation

cs.RO / 55 / 2607.04310

GPU-Accelerated Polygonal Signed Distance Functions for Real-Time Collision Avoidance

基于GPU加速的多边形有符号距离函数用于实时避碰

Ga, Taekwon, Choi, Jongeun

Abstract

Optimization-based local planning and control require high-rate collision-avoidance constraint evaluation over a prediction horizon. In obstacle-dense environments, where feasible space is limited and the constraints become increasingly complex, the computational workload often dominates the control-cycle runtime. The resulting bottleneck motivates collision-avoidance constraints that combine computational efficiency with geometric fidelity. The proposed Polygonal Signed Distance Function (PSDF) is a geometry-exact signed distance function between a convex polygonal robot footprint and obstacles represented by their boundary edges. It is implemented as a weight-free, branch-free tensorized geometric pipeline enabling batched GPU execution and automatic differentiation. The PSDF is embedded into model predictive control by locally linearizing the stage-wise safety constraints within a sequential quadratic programming-based real-time iteration scheme, yielding the PSDF-embedded model predictive controller (PSDF-MPC). The design separates CPU/GPU computation so that the GPU evaluates batched PSDF values and gradients while the CPU solves a sparse quadratic program whose dimension is determined by system dimensions and horizon length, not by obstacle features. Microbenchmarks show that PSDF scales favorably against signed-distance query baselines. Closed-loop simulated and real-world navigation experiments, including comparisons with optimization-based baselines, demonstrate that PSDF-MPC maintains real-time feasibility and robust collision avoidance in dense polygonal environments.

Chinese Translation

基于优化的局部规划和控制需要在预测视域内进行高频率的避碰约束评估。在障碍物密集的环境中，可行空间有限，约束变得越来越复杂，计算工作负载往往主导控制周期的运行时间。由此产生的瓶颈促使我们提出结合计算效率与几何保真度的避碰约束。所提出的多边形有符号距离函数（Polygonal Signed Distance Function, PSDF）是一个几何精确的有符号距离函数，用于描述凸多边形机器人足迹与由边界边表示的障碍物之间的距离。该函数实现为无权重、无分支的张量化几何管道，支持批量GPU执行和自动微分。PSDF被嵌入到模型预测控制中，通过在基于顺序二次规划的实时迭代方案中局部线性化阶段安全约束，从而形成PSDF嵌入的模型预测控制器（PSDF-MPC）。该设计将CPU/GPU计算分离，使得GPU评估批量PSDF值和梯度，而CPU解决一个稀疏二次规划，其维度由系统维度和视域长度决定，而非障碍物特征。微基准测试表明，PSDF在有符号距离查询基线中表现出良好的扩展性。闭环模拟和现实世界导航实验，包括与基于优化的基线的比较，表明PSDF-MPC在密集多边形环境中保持实时可行性和强健的避碰能力。

View on arXiv Download PDF AI Translation

cs.RO / 56 / 2607.04331

Agent-driven Long-tail Simulation for Autonomous Driving

基于代理的长尾仿真用于自动驾驶

Gu, Junru, Yang, Lijin, Huang, Jianing, Liu, Shu, Huang, Zhongzhan, Zhao, Hang

Abstract

Evaluating autonomous driving systems in closed-loop settings requires realistic and interactive simulation, yet existing simulators largely rely on log replay or rule-based agents, limiting behavioral diversity and long-tail coverage. We propose an agent-driven simulation framework in which surrounding road participants are controlled by instruction-following large language models through a structured action interface, enabling intentional and reactive behaviors while preserving physical plausibility. Furthermore, we introduce SemanticPlan, a benchmark of closed-loop planning in long-tail and semantically rich scenarios that augment real nuPlan scenes with multiple interactive agents following diverse language instructions. Evaluation results show that state-of-the-art planners still struggle to consistently achieve safe and effective task completion, suggesting that these long-tail scenarios remain challenging.

Chinese Translation

在闭环环境中评估自动驾驶系统需要现实且互动的仿真，然而现有的仿真器主要依赖日志重放或基于规则的代理，这限制了行为多样性和长尾覆盖。我们提出了一种基于代理的仿真框架，其中周围的道路参与者通过遵循指令的大型语言模型控制，采用结构化的动作接口，使其能够实现有意图和反应性的行为，同时保持物理上的合理性。此外，我们引入了SemanticPlan，一个在长尾和语义丰富场景中进行闭环规划的基准，增强了真实的nuPlan场景，增加了多个遵循多样化语言指令的互动代理。评估结果表明，最先进的规划器仍然难以持续实现安全和有效的任务完成，这表明这些长尾场景仍然具有挑战性。

View on arXiv Download PDF AI Translation

cs.RO / 57 / 2607.04367

A Perception-Manipulation Robotics System for Food Cutting

一种用于食品切割的感知-操控机器人系统

Luo, Xinyuan, Yuan, Wenzhen

Abstract

In the development of cooking robots, mastering the task of cutting is crucial. A significant challenge lies in the diverse properties of food, which necessitate distinct cutting policies and even different knives for optimal processing. This paper presents a perception-manipulation framework for food-cutting tasks. Our system features a knife selection module that utilizes force data from a preliminary fixed trial cut to select the appropriate knife for the given food. This is followed by an adaptive cutting phase using reinforcement learning (RL) to balance cutting speed and energy efficiency. In our experiments, the knife selection module achieved 100% successful rate on unseen food, and we compared the performances of fixed policy, RL policy, with human operators. Our method not only achieves high performance but also demonstrates comparable results to those of human participants.

Chinese Translation

在烹饪机器人开发中，掌握切割任务至关重要。一个显著的挑战在于食品的多样性特性，这需要不同的切割策略，甚至不同的刀具以实现最佳加工。本文提出了一种用于食品切割任务的感知-操控框架。我们的系统具有一个刀具选择模块，该模块利用初步固定试切的力数据来选择适合特定食品的刀具。随后，通过强化学习（Reinforcement Learning, RL）进行自适应切割阶段，以平衡切割速度和能量效率。在我们的实验中，刀具选择模块在未见过的食品上实现了100%的成功率，并且我们比较了固定策略、RL策略与人类操作员的表现。我们的方法不仅实现了高性能，还展示了与人类参与者相当的结果。

View on arXiv Download PDF AI Translation

cs.RO / 58 / 2607.04378

SurgAM: Surgical Affordance Map Prediction with Multimodal Feature Fusion for Robot Autonomy

SurgAM：基于多模态特征融合的外科可用性地图预测以实现机器人自主性

Song, Lei, Long, Yonghao, Xu, Mengya, Geng, Jiayi, Chen, Xiuyuan, Dou, Qi

Abstract

Surgical automation is being increasingly studied, yet bridging visual scene understanding with autonomous action planning remains a fundamental challenge. While much research effort has been made on scene perception (e.g., tool recognition and scene segmentation), understanding and predicting actionable possibilities for surgical automation is still underexplored. In this paper, we introduce surgical affordance prediction, which identifies actionable regions for fundamental surgical actions from visual data. Specifically, a novel adaptive feature fusion framework is proposed that leverages the complementary strengths of a self-supervised vision transformer encoder for its superior semantic understanding and a large-scale generative model encoder for its spatially-aware capability. Furthermore, we introduce a hierarchical prompt learning mechanism to adapt to varying procedural contexts. Finally, a scene-guided attention decoder is proposed to focus on critical surgical areas while suppressing background distractions. To validate the effectiveness, we established a new dataset, derived from publicly available surgical datasets with affordance annotations for three basic surgical actions: aspiration, clipping, and retraction. Extensive experiments demonstrate that our approach achieves state-of-the-art performance. Moreover, we validate our framework's applicability for downstream automation on a realistic lung and prostate phantom, and results show that the predicted affordance maps successfully enable autonomous surgical actions.

Chinese Translation

外科自动化正受到越来越多的研究关注，但将视觉场景理解与自主行动规划结合起来仍然是一个基本挑战。尽管在场景感知（例如工具识别和场景分割）方面进行了大量研究，但理解和预测外科自动化的可操作性仍然未得到充分探索。本文介绍了外科可用性预测，它从视觉数据中识别基本外科动作的可操作区域。具体而言，提出了一种新颖的自适应特征融合框架，利用自监督视觉变换器编码器在语义理解方面的优势和大规模生成模型编码器在空间感知能力上的互补优势。此外，我们引入了一种分层提示学习机制，以适应不同的操作背景。最后，提出了一种场景引导的注意力解码器，专注于关键的外科区域，同时抑制背景干扰。为了验证有效性，我们建立了一个新的数据集，该数据集源自公开的外科数据集，并包含三种基本外科动作的可用性注释：吸引、夹持和回缩。大量实验表明，我们的方法达到了最先进的性能。此外，我们验证了该框架在真实肺部和前列腺模型上的下游自动化应用，结果表明，预测的可用性地图成功实现了自主外科动作。

View on arXiv Download PDF AI Translation

cs.RO / 59 / 2607.04426

ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI

ACE-Brain-0.5：一种统一的具身基础模型用于物理智能体AI

Brain Team, Gong, Ziyang, Gu, Haoming, Luo, Zehang, Zhang, Tianyi, Tao, Tao, Chi, Yixiao, Liu, Zhe, Zhu, Lingsi, Liu, Jingyuan, Tang, Anke, Li, Songze, Kong, Yilun, Liu, Ningjing, Zhu, Tianyu, Qing, Yunpeng, Luo, Shuang, Liu, Xiang, Fu, Shi, Nie, Dawei, Liu, Sixiang, Wen, Zhexi, Pan, Feng, Wang, Xiaofeng, Hou, Zhi, Liu, Chunxiao, Yang, Xue, Yan, Junchi, Zhao, Hengshuang, Tao, Dacheng, Wang, Xiaogang

Abstract

Embodied AI is moving from isolated perception or action modules toward physical agents that understand, plan under goals, act through robot bodies, monitor progress, and improve from experience. Existing systems address this loop only in parts: end-to-end policies generate actions but often lack spatial reasoning, planning, and execution assessment, while robot-agent systems orchestrate tools or specialists but do not learn a shared representation. This fragmentation limits general Physical Agentic AI. We present ACE-Brain-0.5, a unified embodied foundation model that organizes robot intelligence into five coupled functions: spatial perception, decision making, embodied interaction, self-monitoring, and self-improvement. Built on ACE-Brain-0, which established spatial intelligence as a shared scaffold across robot platforms, ACE-Brain-0.5 extends an understanding-centric model into a closed-loop foundation model. A single 8B backbone instantiates the first four functions: grounding objects and affordances, reasoning over 3D and egocentric spatial relations, decomposing instructions into subgoals, generating navigation and manipulation actions, and estimating progress for verification and recovery. To unify these capabilities without cross-task interference, we introduce SSR+, which extends Scaffold-Specialize-Reconcile with a Reactivate stage after task-vector merging. The fifth function, self-improvement, is realized by a companion framework that updates external execution state, including task schemas, spatial memory, and failure-recovery cases, from rollouts. Across fifteen benchmarks, ACE-Brain-0.5 improves over ACE-Brain-0 on 14 of 18 spatial perception and grounding benchmarks, achieves competitive navigation and manipulation performance, and provides strong progress estimation in ID and OOD settings. Together, these results mark an early step toward general Physical Agentic AI.

Chinese Translation

具身人工智能正从孤立的感知或行动模块向理解、在目标下规划、通过机器人身体行动、监控进展并从经验中改进的物理智能体转变。现有系统仅在部分环节解决这一循环：端到端策略生成行动，但往往缺乏空间推理、规划和执行评估，而机器人智能体系统则协调工具或专家，但不学习共享表示。这种碎片化限制了通用物理智能体AI的发展。我们提出了ACE-Brain-0.5，这是一种统一的具身基础模型，将机器人智能组织为五个相互关联的功能：空间感知、决策制定、具身交互、自我监控和自我改进。ACE-Brain-0.5建立在ACE-Brain-0的基础上，后者将空间智能确立为机器人平台之间的共享支架，ACE-Brain-0.5将以理解为中心的模型扩展为一个闭环基础模型。一个单一的8B主干实现了前四个功能：对物体和可供性进行基础，推理3D和自我中心的空间关系，将指令分解为子目标，生成导航和操作行动，并估计进展以进行验证和恢复。为了在不干扰跨任务的情况下统一这些能力，我们引入了SSR+，它在任务向量合并后扩展了支架-专业化-调和（Scaffold-Specialize-Reconcile）方法，增加了一个重新激活阶段。第五个功能，自我改进，通过一个伴随框架实现，该框架从回放中更新外部执行状态，包括任务模式、空间记忆和失败恢复案例。在十五个基准测试中，ACE-Brain-0.5在18个空间感知和基础基准中的14个上优于ACE-Brain-0，达到了具有竞争力的导航和操作性能，并在ID和OOD设置中提供了强大的进展估计。这些结果共同标志着向通用物理智能体AI迈出的早期一步。

View on arXiv Download PDF AI Translation

cs.RO / 60 / 2607.04434

RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies

RoboDojo：一个统一的仿真与现实基准，用于全面评估通用机器人操作策略

Chen, Tianxing, Chen, Yue, Li, Zixuan, Tang, Junyuan, Su, Kailun, Wan, Weijie, Chen, Baijun, Lu, Haoran, Yan, Haowen, Su, Honghao, Dou, Zhiyang, Wang, Kaixuan, Zhang, Dandan, Liu, Yunze, Qin, Yan, Liang, Qiwei, Wu, Qiwei, Lin, Zijian, Lin, Wenwei, Wang, Yuran, He, Minghua, Wu, Tianshu, Wu, Ruihai, Zhou, Jingquan, Lei, Kai-Chong, Yu, Haibao, Ji, Yuanfeng, Jin, Weiyang, Lin, Guanyu, Li, Xiaofan, Xiong, Qi, Xu, Renjing, Li, Zhongyu, Chai, Wenhao, Xie, Enze, Wang, Ziwei, Mu, Yao, Dong, Hao, Matusik, Wojciech, Ding, Mingyu, Ding, Wenbo, Luo, Ping, Tomizuka, Masayoshi

Abstract

Generalist robot manipulation policies have advanced rapidly, yet existing benchmarks remain limited in systematically evaluating their capabilities. Many rely on simple, short-horizon, or skill-narrow tasks with limited capability coverage, and are often conducted only in simulation or only in the real world. Simulation enables scalable feedback but misses physical deployment challenges, while real-world evaluation is costly, time-consuming, and difficult to reproduce. We introduce RoboDojo, a unified sim-and-real benchmark for comprehensive evaluation of generalist robot manipulation policies. RoboDojo includes 42 simulation tasks and 18 real-world tasks covering diverse and complementary manipulation capabilities. The simulation benchmark evaluates five dimensions: generalization, memory, precision, long-horizon execution, and open-vocabulary instruction following, while the real-world benchmark exposes policies to challenging physical-world deployment conditions. RoboDojo supports scalable evaluation through heterogeneous parallel simulation in Isaac Sim and provides RoboDojo-RealEval, a reproducible real-world evaluation system with remote cloud access, standardized hardware, scene reset, evaluation protocol, and deployment interface. Together with XPolicyLab, policies can be integrated once and evaluated across simulation and real-world settings with minimal adaptation. We integrate 30 policies into XPolicyLab and evaluate them on RoboDojo, establishing a public leaderboard and systematic analysis of current policy performance. The website is available at http://robodojo-benchmark.com/.

Chinese Translation

通用机器人操作策略发展迅速，但现有基准在系统评估其能力方面仍然有限。许多基准依赖于简单、短期或技能狭窄的任务，覆盖的能力有限，且通常仅在仿真或现实世界中进行。仿真能够提供可扩展的反馈，但缺乏物理部署的挑战，而现实世界评估则成本高昂、耗时且难以重复。我们提出了RoboDojo，一个统一的仿真与现实基准，用于全面评估通用机器人操作策略。RoboDojo包含42个仿真任务和18个现实世界任务，涵盖多样且互补的操作能力。仿真基准评估五个维度：泛化、记忆、精确度、长期执行和开放词汇指令跟随，而现实世界基准则使策略面临具有挑战性的物理世界部署条件。RoboDojo通过在Isaac Sim中进行异构并行仿真支持可扩展评估，并提供RoboDojo-RealEval，一个具有远程云访问、标准化硬件、场景重置、评估协议和部署接口的可重复现实世界评估系统。结合XPolicyLab，策略可以一次集成并在仿真和现实世界环境中以最小的适应性进行评估。我们将30个策略集成到XPolicyLab中，并在RoboDojo上进行评估，建立了一个公共排行榜和当前策略性能的系统分析。网站地址为http://robodojo-benchmark.com/

View on arXiv Download PDF AI Translation

cs.RO / 61 / 2607.04453

Robustness Verification of an Autonomous Underwater Vehicle-based Plankton Classifier

基于自主水下车辆的浮游生物分类器的鲁棒性验证

Sayed, Abdelrahman Sayed, Meyer, Pierre-Jean, Sørensen, Asgeir J., Ghazel, Mohamed

Abstract

The assessment of planktonic standing stocks and microorganism structures is critical for understanding upper ocean biological processes. Currently, autonomous underwater vehicles (AUVs) equipped with in-situ optical imaging and artificial intelligence (AI) methods offer a promising solution for persistent surveillance, mapping and monitoring of planktonic life. However, current AI methods often lack robustness in dynamic, unstructured environments, where environmental noise and non-biological artifacts lead to frequent misclassifications. Standard convolutional neural network (CNN) classifiers often struggle with such conditions, leading to misclassifications that require time-consuming manual validation by marine biologists. To address this issue, we propose a novel robustness verification framework for in-situ plankton classifiers based on reachability analysis. We also introduce a continuous-time neural ordinary differential equation (neural ODE) classification model leveraging the high-resolution imaging capabilities of the SilCam particle imager. In this paper, we demonstrate the effectiveness of the proposed framework by formally verifying the robustness of the neural ODE model against environmental perturbations. We demonstrate that our verification framework acts as an automated filter providing formal guarantees of model stability against ambiguous data, thereby improving the reliability of autonomous sampling and reducing the post-processing workload.

Chinese Translation

评估浮游生物的存量和微生物结构对于理解上层海洋生物过程至关重要。目前，配备原位光学成像和人工智能（AI）方法的自主水下车辆（AUVs）为持续监测、绘制和监控浮游生物提供了有前景的解决方案。然而，当前的AI方法在动态、非结构化环境中往往缺乏鲁棒性，环境噪声和非生物伪影导致频繁的错误分类。标准卷积神经网络（CNN）分类器在这种条件下常常表现不佳，导致的错误分类需要海洋生物学家耗时的手动验证。为了解决这一问题，我们提出了一种基于可达性分析的原位浮游生物分类器的新鲁棒性验证框架。我们还引入了一种利用SilCam粒子成像仪高分辨率成像能力的连续时间神经常微分方程（neural ODE）分类模型。在本文中，我们通过正式验证神经ODE模型对环境扰动的鲁棒性，展示了所提框架的有效性。我们证明我们的验证框架作为一种自动化过滤器，提供了模型在模糊数据下的稳定性正式保证，从而提高了自主采样的可靠性，并减少了后处理工作量。

View on arXiv Download PDF AI Translation

cs.RO / 62 / 2607.04546

Mask2Real-WM: Segmentation Masks as a Sim-to-Real Bridge for Controllable Dexterous World Models

Mask2Real-WM：将分割掩膜作为可控灵巧世界模型的仿真到真实桥梁

Feingold, Riccardo O., Liconti, Davide, Yang, Chenyu, Katzschmann, Robert K.

Abstract

Action-conditioned world models allow robots to predict the future consequences of candidate actions without additional physical interaction, supporting policy evaluation, planning, and data augmentation. We present Mask2Real-WM, a two-stage action-conditioned world model for dexterous manipulation that decouples pixel prediction into a dynamics model and a rendering model. The dynamics model predicts future segmentation masks from past masks and 23-DoF action sequences. The rendering model maps the predicted masks to photorealistic RGB using a ControlNet-augmented Stable Video Diffusion backbone. The smaller sim-to-real gap in segmentation space enables the dynamics model to benefit from large-scale pretraining on over 50 h of synthetic simulation data, followed by fine-tuning on fewer than 2.5 h of real demonstrations. Experiments on a dexterous pick-and-place benchmark show that mask conditioning and simulation pretraining are both required for per-DoF action controllability across all 23 degrees of freedom. In contrast, monolithic baselines capture broad hand and end-effector trajectories but do not reliably reflect fine-grained, per-joint action effects.

Chinese Translation

基于动作条件的世界模型使机器人能够在无需额外物理交互的情况下预测候选动作的未来结果，从而支持策略评估、规划和数据增强。我们提出了Mask2Real-WM，这是一种用于灵巧操作的两阶段动作条件世界模型，它将像素预测解耦为动态模型和渲染模型。动态模型根据过去的掩膜和23自由度（DoF）动作序列预测未来的分割掩膜。渲染模型使用增强的ControlNet稳定视频扩散骨干网络将预测的掩膜映射为照片级真实感RGB。分割空间中较小的仿真到真实差距使得动态模型能够从超过50小时的合成仿真数据的大规模预训练中受益，随后在少于2.5小时的真实演示上进行微调。在灵巧的拾取和放置基准测试中的实验表明，掩膜条件和仿真预训练对于所有23个自由度的每个自由度动作可控性都是必需的。相比之下，单一基线捕捉了广泛的手部和末端执行器轨迹，但未能可靠地反映细粒度的每个关节动作效果。

View on arXiv Download PDF AI Translation

cs.RO / 63 / 2607.04554

HUGS: Guiding Unified Dexterous Grasp Synthesis Across Modes and Scales via Learned Human Priors

HUGS：通过学习的人类先验指导跨模式和尺度的统一灵巧抓取合成

Yu, Mingrui, Jiang, Yongpeng, Jia, Yongyi, Lv, Kangchen, Huang, Li, Ren, Yi, Li, Xiang

Abstract

Dexterous grasping across diverse object scales requires contact modes ranging from two-finger pinches to bimanual grasps. Existing dexterous grasp synthesis methods reduce the high-dimensional optimization space with manually designed expected contacts and initialization heuristics, which struggle to balance synthesis success rate and diversity. We present HUGS (Human-prior-guided Unified Dexterous Grasp Synthesis), a human-prior-guided framework for unified dexterous grasp synthesis across modes and scales. Instead of directly retargeting human demonstrations, HUGS learns an object-conditioned human prior that captures human grasp preferences and guides downstream force-closure-aware optimization. The prior is trained on a compact self-collected human grasp dataset with 1.8K grasps over 304 objects, providing broad coverage of object scales and contact modes. During synthesis, HUGS adaptively proposes contact modes and wrist initializations, substantially improving the balance between contact-mode coverage and synthesis success rate over heuristic-based methods. With HUGS, we synthesize 3.2M robotic grasps over 157K scenes, spanning object half-diagonal lengths from 2 cm to 30 cm and modes from two-finger to bimanual grasps. Models trained on the synthesized dataset autonomously select appropriate contact modes in the real world, enabling grasping from screws to large boxes.

Chinese Translation

在多样化物体尺度下的灵巧抓取需要从双指捏取到双手抓取等多种接触模式。现有的灵巧抓取合成方法通过手动设计的期望接触和初始化启发式方法来减少高维优化空间，但在合成成功率和多样性之间难以取得平衡。我们提出了HUGS（人类先验指导的统一灵巧抓取合成），这是一个针对跨模式和尺度的统一灵巧抓取合成的人类先验指导框架。HUGS并不是直接重新定向人类示范，而是学习一个基于物体条件的人类先验，以捕捉人类抓取偏好并指导下游的力闭合感知优化。该先验是在一个包含1.8K个抓取样本和304个物体的紧凑自收集人类抓取数据集上训练的，广泛覆盖了物体尺度和接触模式。在合成过程中，HUGS自适应地提出接触模式和手腕初始化，显著改善了接触模式覆盖与合成成功率之间的平衡，相较于基于启发式的方法。通过HUGS，我们在157K个场景中合成了320万次机器人抓取，涵盖了从2厘米到30厘米的物体半对角线长度，以及从双指抓取到双手抓取的多种模式。基于合成数据集训练的模型能够在现实世界中自主选择适当的接触模式，使得抓取从螺丝到大型箱子成为可能。

View on arXiv Download PDF AI Translation

cs.RO / 64 / 2607.04591

Simple-to-Complex Structured Demonstrations for Vision-Language-Action Learning

从简单到复杂的结构化演示用于视觉-语言-动作学习

Qiu, Xinchuan, Yu, Yi

Abstract

Vision-Language-Action (VLA) models have demonstrated strong capabilities in robotic manipulation by integrating visual perception, language understanding, and robot action generation. Existing research has primarily focused on improving model architectures, training strategies, and dataset scale, while little attention has been paid to how demonstrations are collected and organized. We identify demonstration organization as a fundamental yet overlooked aspect of imitation learning, as it directly affects policy learning efficiency, training stability, and policy generalization. To address this gap, we propose a simple-to-complex structured demonstration collection strategy for VLA learning using a dual-arm robotic platform. Our approach systematically organizes data through three general principles: (i) decomposing complex manipulation tasks into progressively learnable sub-skills, (ii) standardizing the interaction environment to reduce unnecessary variability, and (iii) organizing demonstrations according to progressively increasing task complexity. This structured design enables VLA models to first acquire fundamental manipulation skills before learning increasingly complex task compositions, facilitating more effective learning of long-horizon manipulation tasks. We evaluate the proposed strategy on two representative robotic manipulation tasks: block grasping and sorting, and towel folding. Experimental results show consistent improvements in task success rate and training stability compared with the baseline method of directly collecting end-to-end complete task trajectories. These findings highlight demonstration organization as a previously underexplored but important factor in VLA learning and provide practical insights into efficient skill acquisition, scalable dataset construction, and long-horizon robotic manipulation.

Chinese Translation

视觉-语言-动作（VLA）模型通过整合视觉感知、语言理解和机器人动作生成，展示了在机器人操控方面的强大能力。现有研究主要集中在改进模型架构、训练策略和数据集规模上，而对演示的收集和组织方式关注较少。我们认为演示组织是模仿学习中一个基本但被忽视的方面，因为它直接影响策略学习效率、训练稳定性和策略泛化能力。为了解决这一问题，我们提出了一种针对VLA学习的从简单到复杂的结构化演示收集策略，使用双臂机器人平台。我们的方法通过三个一般原则系统地组织数据：（i）将复杂的操控任务分解为逐步可学习的子技能；（ii）标准化交互环境以减少不必要的变异性；（iii）根据逐步增加的任务复杂性组织演示。这样的结构化设计使得VLA模型能够首先掌握基本的操控技能，然后再学习越来越复杂的任务组合，从而促进对长时间跨度操控任务的更有效学习。我们在两个具有代表性的机器人操控任务上评估了所提策略：块抓取与排序，以及毛巾折叠。实验结果表明，与直接收集端到端完整任务轨迹的基线方法相比，任务成功率和训练稳定性均有一致性提升。这些发现突显了演示组织作为VLA学习中一个先前未被充分探讨但重要的因素，并为高效技能获取、可扩展数据集构建和长时间跨度机器人操控提供了实用的见解。

View on arXiv Download PDF AI Translation

cs.RO / 65 / 2607.04609

SEAM: Smooth Execution of Action-Chunked Motion for Vision-Language-Action Policies

SEAM：视觉-语言-动作策略的平滑执行动作块运动

Zhan, Dijia, Xu, Xuemiao, Li, Jinyi, Tang, Jie

Abstract

Vision-Language-Action (VLA) policies that execute fixed-length action chunks can exhibit multimodal bifurcation: a cross-chunk inconsistency in which adjacent chunks generated from independent Gaussian latents can converge to incompatible trajectory modes, producing abrupt discontinuities at chunk boundaries. Existing remedies either require backpropagation through the policy at each denoising step, rely on rejection sampling, or require retraining, each trading computational cost or task reliability for smoother transitions. We propose SEAM (Smooth Execution of Action-Chunked Motion), a training-free inference-time method for flow matching VLAs. SEAM exploits a simple synchronous-execution insight: after the robot consumes the executed prefix, the previous chunk's unexecuted tail is already available as an analytic consistency reference. Its core mechanism, Velocity-guided Loss Steering (VLS), derives a time-dependent target from this tail and applies a closed-form correction after each Euler step without backpropagating through the policy network. On LIBERO-10 with pi_0.5, SEAM reduces boundary jerk by 28%, reduces chunk transition discontinuity by 27%, preserves baseline-level task success, and keeps denoising-loop cost near the unguided baseline.

Chinese Translation

执行固定长度动作块的视觉-语言-动作（VLA）策略可能会出现多模态分叉：一种跨块不一致性，其中来自独立高斯潜变量的相邻块可能会收敛到不兼容的轨迹模式，在块边界产生突兀的不连续性。现有的解决方案要么在每个去噪步骤中需要通过策略进行反向传播，要么依赖于拒绝采样，或者需要重新训练，每种方法都在平滑过渡与计算成本或任务可靠性之间进行权衡。我们提出了SEAM（平滑执行动作块运动），这是一种无训练的推理时方法，用于流匹配VLA。SEAM利用了一个简单的同步执行洞察：在机器人消耗已执行前缀后，前一个块的未执行尾部已经可以作为解析一致性参考。其核心机制，速度引导损失引导（Velocity-guided Loss Steering，VLS），从这个尾部推导出一个时间相关的目标，并在每个欧拉步骤后应用封闭形式的修正，而无需通过策略网络进行反向传播。在LIBERO-10上，使用pi_0.5，SEAM将边界抖动减少了28%，将块过渡不连续性减少了27%，保持了基线水平的任务成功率，并将去噪循环成本保持在接近无引导基线的水平。

View on arXiv Download PDF AI Translation

cs.RO / 66 / 2607.04610

RoboVista: Evaluating Vision Language Models for Diverse Robot Applications

RoboVista：评估视觉语言模型在多样化机器人应用中的表现

Xie, Shuangyu, Chen, Kaiyuan, Chen, Ziyang, Adebola, Simeon, Huang, Yixuan, Ma, Zehan, Qiu, Tianshuang, Yuan, Wentao, Shah, Dhruv, Sanketi, Pannag R., Goldberg, Ken

Abstract

Diverse applications for robotics, such as industry and agriculture, require robots to operate across various embodiments, changing visual conditions, and complex planning. Vision-Language Models (VLMs) offer a promising foundation for general-purpose and interpretable robotic reasoning. Aligning VLMs with diverse robot applications requires a modular understanding of the individual decision components that underlie robotic behavior. Capturing such structure is challenging for conventional robot benchmarks that are primarily based on teleoperated, end-to-end datasets. We propose Robot Question Answering (RQA), a modular evaluation framework and RoboVista, a benchmark curated from real robotic systems, research papers, and expert annotations. RoboVista contains 474 Visual Question Answering (VQA) instances with human annotated reasoning and covers 39 unique task types in agricultural, industrial, domestic, surgical robotics, autonomous driving, and open robot datasets. Experiments on RoboVista show that state-of-the-art VLMs exhibit substantial gaps. Physical robot experiments suggest strong correlation between RoboVista performance and real-world task execution.

Chinese Translation

多样化的机器人应用，如工业和农业，要求机器人在各种形态、变化的视觉条件和复杂的规划中进行操作。视觉-语言模型（Vision-Language Models, VLMs）为通用和可解释的机器人推理提供了一个有前景的基础。将VLMs与多样化的机器人应用对齐需要对支撑机器人行为的各个决策组件进行模块化理解。捕捉这种结构对于主要基于遥操作、端到端数据集的传统机器人基准测试来说是具有挑战性的。我们提出了机器人问答（Robot Question Answering, RQA），一个模块化评估框架，以及RoboVista，一个从真实机器人系统、研究论文和专家注释中整理的基准数据集。RoboVista包含474个视觉问答（Visual Question Answering, VQA）实例，具有人工注释的推理，并涵盖农业、工业、家庭、外科机器人、自动驾驶和开放机器人数据集中39种独特任务类型。在RoboVista上的实验表明，最先进的VLMs存在显著差距。物理机器人实验表明RoboVista的表现与现实世界任务执行之间存在强相关性。

View on arXiv Download PDF AI Translation

cs.RO / 67 / 2607.04615

DIVO: Continuous-time DVL-Inertial-Visual Odometry for Unmanned Underwater Vehicles

DIVO：无人水下车辆的连续时间DVL-惯性-视觉里程计

Jung, Kyungmin, Bajwa, Angad, Yoo, Junha, Bernal, Arturo Del Castillo, Forbes, James Richard

Abstract

This paper presents a novel acoustic-visual-inertial odometry solution leveraging a continuous-time trajectory estimation framework for unmanned underwater vehicles. Underwater environments present unique challenges for visual localization and mapping, such as light attenuation, illumination variance, and the presence of particulate matter. This motivates the use of additional sensing modalities and a visual tracking pipeline that is robust to diverse subsea conditions. The proposed system is the first continuous-time trajectory estimation framework based on Gaussian processes to fuse asynchronous measurements from a Doppler velocity log, a stereo camera, and an inertial measurement unit. Additionally, a novel visual frontend is proposed, incorporating learning-based feature extraction and matching that is robust to the specific challenges that subsea environments present. The proposed framework enables seamless integration of additional sensor modalities in continuous-time and is adaptable to different environments without reconfiguration. The proposed system is extensively tested on real-world underwater inspection datasets, where it outperforms state-of-the-art visual-inertial and acoustic-visual-inertial SLAM algorithms in accuracy, robustness, and trajectory coverage. Notably, the proposed system outperforms the state-of-the-art despite only forming short-term visual data associations.

Chinese Translation

本文提出了一种新颖的声学-视觉-惯性里程计解决方案，利用连续时间轨迹估计框架用于无人水下车辆。水下环境对视觉定位和地图构建提出了独特的挑战，如光衰减、照明变化和颗粒物的存在。这促使我们采用额外的传感方式和一个对多样化海底条件具有鲁棒性的视觉跟踪管道。所提出的系统是第一个基于高斯过程的连续时间轨迹估计框架，能够融合来自多普勒速度记录仪、立体相机和惯性测量单元的异步测量。此外，提出了一种新颖的视觉前端，结合了基于学习的特征提取和匹配，能够有效应对海底环境所带来的特定挑战。该框架能够无缝集成额外的传感器模式，并且在不同环境中无需重新配置。所提出的系统在真实世界的水下检查数据集上进行了广泛测试，在准确性、鲁棒性和轨迹覆盖方面超越了最先进的视觉-惯性和声学-视觉-惯性SLAM算法。值得注意的是，尽管仅形成短期视觉数据关联，所提出的系统仍然优于当前最先进的技术。

View on arXiv Download PDF AI Translation

cs.RO / 68 / 2607.04616

SILO: Simulation-in-the-Loop Sim-to-Real Transfer for Multi-Stage Cable Routing

SILO：多阶段电缆布线的仿真环节实物转移

Tao, Stone, Xu, Jie, Rabeti, Hesam, Narang, Yashraj, Guo, Yijie, Akinola, Iretiayo

Abstract

Linear-deformable manipulation remains challenging due to the complex deformations of objects such as cables and ropes. Prior data-driven approaches, particularly imitation learning, have shown some promise in narrowly defined settings but typically require thousands of demonstrations for specific tasks and cable types, limiting scalability and generalization. We introduce a sim-to-real reinforcement learning (RL) framework for multi-stage cable routing that leverages GPU-parallelized simulation to approximate linear deformable behaviors. Training across thousands of parallel simulations enables the learned policies to generalize across diverse cable geometries and deformation patterns. To bridge the sim-to-real gap, we propose a novel deployment strategy that combines a Simulation In the LOop (SILO) execution framework, localized RL policies, and robust cable state estimation. On real-world cable routing tasks, our approach achieves higher success rates and 2x reduction in cycle times compared to prior state-of-the-art learning methods. To our knowledge, this is the first successful sim-to-real transfer of RL policies for multi-stage cable routing. Videos and additional visualizations are available at https://silo-cable-routing.github.io/

Chinese Translation

线性可变形操作由于电缆和绳索等物体的复杂变形而仍然具有挑战性。之前的数据驱动方法，特别是模仿学习，在狭义定义的环境中显示出一定的前景，但通常需要数千次特定任务和电缆类型的演示，这限制了其可扩展性和泛化能力。我们提出了一种用于多阶段电缆布线的仿真到现实的强化学习（RL）框架，该框架利用GPU并行化仿真来近似线性可变形行为。在数千个并行仿真中进行训练，使得学习到的策略能够在多样化的电缆几何形状和变形模式中进行泛化。为了弥合仿真与现实之间的差距，我们提出了一种新颖的部署策略，结合了仿真环节执行框架（Simulation In the LOop, SILO）、局部RL策略和稳健的电缆状态估计。在实际电缆布线任务中，我们的方法实现了更高的成功率，并将周期时间减少了2倍，相较于之前的最先进学习方法。根据我们的了解，这是多阶段电缆布线中RL策略成功的首次仿真到现实转移。视频和其他可视化资料可在 https://silo-cable-routing.github.io/ 获取。

View on arXiv Download PDF AI Translation

cs.RO / 69 / 2607.04634

Governed Caste Reassignment in Heterogeneous Swarms: An Asymmetric-Trust Protocol with Audited Operator Countersignature

异质群体中的治理 caste 重新分配：带有审核操作员反签名的非对称信任协议

Qin, Xue, Luan, Simin, Yang, Cong, Li, Zhijun

Abstract

In heterogeneous robot swarms, caste reassignment (rebinding a robot to a new capability-bound role) is a high-frequency runtime event driven by battery, payload, and priority changes. Existing approaches treat it as an internal allocation algorithm and do not expose the reassignment to external authority. We argue that for regulated embodied deployments a caste change that elevates a robot's privilege envelope is a governance event that must be auditable and externally authorised. We propose an asymmetric-trust protocol: auto-tightening reassignments (to safer, lower-privilege castes) are admitted automatically, while bounded relaxation (to higher-privilege castes) requires an operator countersignature against a per-axis budget. Each transition carries a signed cause-chain, committed to a hash-chained Merkle audit log that an offline auditor verifies from an operator-signed identity manifest alone. We evaluate a reference implementation with real Ed25519 signatures over fleets up to 100 robots: auto-tightening completes in single-digit to low-double-digit milliseconds, and the governed protocol refuses four explicit attacks (caste laundering, repeated-relaxation escalation, operator impersonation, cause-chain forgery) by construction, with a partially-governed baseline isolating which gate stops which attack and a randomized fuzz adversary finding no admission. A distributed audit layer replicates the log across N per-member replicas with quorum-committed total order and cryptographic fork exclusion; we prove agreement and fork exclusion and validate them both in simulation and as a real multi-process deployment over TCP sockets (up to 100 real processes) with a Byzantine equivocator, on which every honest replica agrees, detects the equivocation, and commits no fork. The construction generalises a single-agent persona-mutation governance gate to swarm-level caste governance.

Chinese Translation

在异质机器人群体中，caste 重新分配（将机器人重新绑定到新的能力限制角色）是一个高频率的运行时事件，由电池、有效载荷和优先级的变化驱动。现有方法将其视为内部分配算法，并未将重新分配暴露给外部权威。我们认为，对于受监管的具身部署，提升机器人特权范围的 caste 变化是一个必须可审计和外部授权的治理事件。我们提出了一种非对称信任协议：自动收紧的重新分配（到更安全、特权较低的 caste）会被自动接受，而有界放松（到特权较高的 caste）则需要操作员的反签名，依据每个轴的预算。每个过渡都携带一个签名的原因链，承诺于一个哈希链式的 Merkle 审计日志，离线审计员仅通过操作员签名的身份清单进行验证。我们评估了一个参考实现，使用真实的 Ed25519 签名，针对最多 100 个机器人的队列：自动收紧的完成时间在个位到低双位毫秒之间，而治理协议通过构造拒绝了四种明确攻击（caste 洗钱、重复放松升级、操作员冒充、原因链伪造），并且部分治理的基线隔离了哪个门阻止了哪个攻击，随机模糊对手未找到任何入场。一个分布式审计层在 N 个成员副本之间复制日志，具有法定承诺的总顺序和加密分叉排除；我们证明了一致性和分叉排除，并在模拟和真实的多进程部署（通过 TCP 套接字，最多 100 个真实进程）中验证了它们，在此过程中每个诚实副本达成一致，检测到分歧，并未产生分叉。该构造将单代理的角色变更治理门推广到群体级别的 caste 治理。

View on arXiv Download PDF AI Translation

cs.RO / 70 / 2607.04652

KAM-WM: Kinematic Affordance Maps from Latent World Models for Robot Manipulation

KAM-WM：基于潜在世界模型的运动可供性图用于机器人操作

Shao, Xinyu, Zhou, Keru, Huang, Guowei, Gao, Yajun, Cao, Tongtong, Li, Xiu

Abstract

Learning manipulation from few demonstrations requires visual priors that capture not only where to interact, but also how the interaction should begin; static priors such as segmentation masks encode only the former. We present KAM-WM, a framework that extracts a coarse directional interaction cue from a frozen latent video world model without rollout or world-model fine-tuning. KAM-WM queries a Flow Matching image-to-video backbone once and interprets its single-step latent velocity as a Kinematic Affordance Map (KAM), which provides task-conditioned interaction regions and coarse motion structure. A lightweight Perceiver compresses KAM into tokens that condition a diffusion policy together with RGB observations and proprioception. Across LIBERO and RoboTwin2.0, KAM-WM reaches 90.6% average success on LIBERO and achieves 65.7% and 22.4% success rates in the Easy and Hard settings on RoboTwin2.0, respectively. Controlled comparisons against a zero-order mask prior suggest that part of the gains comes from directional information beyond spatial localization alone. These results indicate that, in the evaluated settings, a frozen video model can provide a useful first-order visual prior for control without the test-time cost of future rollout.

Chinese Translation

从少量示例中学习操作需要视觉先验，不仅要捕捉交互的位置，还要明确交互的开始方式；而静态先验如分割掩码仅编码前者。我们提出了KAM-WM，一个从冻结的潜在视频世界模型中提取粗略方向性交互线索的框架，无需展开或对世界模型进行微调。KAM-WM仅查询一次Flow Matching图像到视频的骨干网络，并将其单步潜在速度解释为运动可供性图（Kinematic Affordance Map, KAM），该图提供任务条件下的交互区域和粗略运动结构。一个轻量级的Perceiver将KAM压缩为令牌，这些令牌与RGB观测和本体感知共同调节扩散策略。在LIBERO和RoboTwin2.0上，KAM-WM在LIBERO上达到了90.6%的平均成功率，并在RoboTwin2.0的简单和困难设置中分别达到了65.7%和22.4%的成功率。与零阶掩码先验的对照比较表明，部分增益来自于超越空间定位的方向信息。这些结果表明，在评估的设置中，冻结的视频模型可以提供有用的一阶视觉先验用于控制，而无需在测试时承担未来展开的成本。

View on arXiv Download PDF AI Translation

cs.RO / 71 / 2607.04681

Do Vision-Language-Action Models Mean What They Say? On the Role of Faithfulness in Embodied Reasoning

视觉-语言-行动模型是否言之有物？关于具身推理中忠实性的角色

Foutter, Matthew, Cercola, Matteo, Wild, Lena, Wang, Yunshan, Li, Michelle, Gammelli, Daniele, Pavone, Marco

Abstract

Embodied Chain-of-Thought has emerged as a promising mechanism to enhance robot decision-making and interpretability in black-box Vision-Language Action (VLA) models. However, whether this verbalized Chain-of-Thought truthfully reflects the policy's underlying decision process remains poorly understood. We distinguish between functional reasoning, in which reasoning improves task performance, and faithful reasoning, in which reasoning truly reflects the policy's internal decision process. We argue that SoTA alignment strategies offer a necessary but insufficient notion of faithfulness, admitting reasoning whose intermediate steps can mask the causal links in action prediction through confounding factors (e.g., reasoning that is ungrounded in the environment and internally disconnected or inconsistent), restricting policy generalization. We study this gap through a human evaluation of a SoTA reasoning model for autonomous driving, revealing an inconsistent coupling between reasoning quality and downstream trajectory improvement. We then operationalize a behavioral surrogate for embodied faithfulness through a learned critic, Pinocchio, scoring observation grounding and stepwise coherence, and use this critic as a dense reward signal in post-training an embodied policy with reinforcement learning. Across withheld driving benchmarks, our post-trained planner improves faithfulness by 4% and 18% over SoTA alignment and trajectory error post-training baselines, respectively, while maintaining competitive downstream task performance. Finally, on a synthetic out-of-distribution test set, post-training for faithfulness improves policy responsiveness to rare counterfactual scenarios by 1.6x that of a SoTA policy, suggesting that faithful reasoning traces contribute to more robust, generalizable, and interpretable embodied intelligence. Project page: https://mjf-su.github.io/pinocchio/

Chinese Translation

具身思维链（Embodied Chain-of-Thought）作为一种有前景的机制，已被提出以增强机器人决策能力和黑箱视觉-语言-行动（Vision-Language Action, VLA）模型的可解释性。然而，这种口头化的思维链是否真实反映了策略的潜在决策过程仍然不甚明了。我们区分了功能性推理（functional reasoning），即推理改善任务表现的情况，以及忠实性推理（faithful reasoning），即推理真实反映策略内部决策过程的情况。我们认为，当前最先进（SoTA）的对齐策略提供了一个必要但不足的忠实性概念，允许那些中间步骤可能通过混淆因素（例如，未与环境相结合且内部不连贯或不一致的推理）掩盖行动预测中的因果关系的推理，从而限制了策略的泛化能力。我们通过对一个用于自动驾驶的最先进推理模型进行人类评估，研究了这一差距，揭示了推理质量与后续轨迹改进之间的不一致耦合。随后，我们通过一个学习的评估者Pinocchio，将具身忠实性操作化，评估观察的基础和逐步一致性，并将该评估者用作后训练具身策略的密集奖励信号，结合强化学习。在保留的驾驶基准测试中，我们的后训练规划器在忠实性上比最先进的对齐和轨迹误差后训练基线分别提高了4%和18%，同时保持了竞争性的下游任务表现。最后，在一个合成的分布外测试集上，忠实性后训练使策略对稀有反事实场景的响应能力提高了1.6倍，相较于最先进的策略，表明忠实的推理轨迹有助于更强健、可泛化和可解释的具身智能。项目页面：https://mjf-su.github.io/pinocchio/

View on arXiv Download PDF AI Translation

cs.RO / 72 / 2607.04689

A Reliable Context-Aware and Temporal Planning Framework for Autonomous Driving

一个可靠的上下文感知和时间规划框架用于自动驾驶

Dey, Argho, Yin, Yunfei, Ray, Swachha, Islam, Md Minhazul, Yuan, Zheng, Xiong, Sijing, Liu, Hongyu, Huang, Zhiqiu

Abstract

Safe operation of autonomous vehicles in dense urban traffic depends on perception and planning that remain reliable when onboard sensing is degraded. In real driving conditions, camera observations are frequently corrupted by occlusion, motion blur, illumination change, and sensor noise, and when such degraded observations are aggregated indiscriminately over time, trajectory planning becomes unstable and collision risk rises for both the ego vehicle and surrounding road users. Recent Bird's-Eye-View (BEV) approaches unify perception and planning through a shared spatial representation, but most fuse temporal information across frames without assessing the reliability of the underlying observations. We present a Reliable Context-Aware and Temporal Planning framework for Autonomous Driving (RCT-AD) that explicitly models feature quality and temporal consistency to support safer, more consistent planning. A Reliable Context Awareness module scores per-frame reliability and selectively retains trustworthy features through a quality-gated First-In-Last-Out (FILO) memory mechanism, reconstructing degraded observations from reliable historical context so that corrupted inputs do not destabilize the scene representation. A Temporal Trajectory Planner captures long-term dependencies and multi-agent interactions to produce smoother, safety-aware trajectories, while a joint detection-and-segmentation head injects semantic and motion cues into the shared BEV space to strengthen scene understanding. Experiments on the nuScenes autonomous driving benchmark show that RCT-AD improves perception accuracy, motion prediction, and planning robustness over recent end-to-end baselines, achieving 61.5 nuScenes Detection Score, 52.9 mean Average Precision, and 52.3 mean Intersection over Union, while maintaining competitive computational efficiency suitable for real-time deployment.

Chinese Translation

在密集城市交通中，自动驾驶车辆的安全运行依赖于感知和规划的可靠性，尤其是在车载传感器性能下降时。在实际驾驶条件下，摄像头观测常常受到遮挡、运动模糊、光照变化和传感器噪声的影响，当这些降级的观测信息在时间上被不加区分地聚合时，轨迹规划会变得不稳定，导致自我车辆和周围道路使用者的碰撞风险上升。最近的鸟瞰图（Bird's-Eye-View, BEV）方法通过共享的空间表示统一了感知和规划，但大多数方法在帧之间融合时间信息时并未评估基础观测的可靠性。我们提出了一个可靠的上下文感知和时间规划框架（Reliable Context-Aware and Temporal Planning framework for Autonomous Driving, RCT-AD），该框架明确建模特征质量和时间一致性，以支持更安全、更一致的规划。可靠的上下文感知模块对每帧的可靠性进行评分，并通过质量门控的先进先出（First-In-Last-Out, FILO）记忆机制选择性地保留可信特征，从可靠的历史上下文中重建降级的观测，以确保损坏的输入不会破坏场景表示。时间轨迹规划器捕捉长期依赖关系和多智能体交互，以生成更平滑、注重安全的轨迹，同时联合检测和分割头将语义和运动线索注入共享的BEV空间，以增强场景理解。在nuScenes自动驾驶基准测试上的实验表明，RCT-AD在感知准确性、运动预测和规划鲁棒性方面优于最近的端到端基线，达到了61.5的nuScenes检测分数、52.9的平均精度和52.3的平均交并比，同时保持适合实时部署的竞争性计算效率。

View on arXiv Download PDF AI Translation

cs.RO / 73 / 2607.04714

Geometry-Aware Motion Latents for Learning Robust Manipulation Policies

基于几何的运动潜变量用于学习鲁棒的操控策略

Zhang, Yunchao, Weng, Yijia, Liu, Ruizhe, Hu, Ming, Guibas, Leonidas, Yang, Yanchao

Abstract

Learning motion latents for robotic manipulation heavily relies on extracting motion patterns from visual sequences, yet effective action abstractions require understanding three-dimensional geometric transformations. Here, we introduce GeoMoLa (Geometry-Aware Motion Latents), which learns discrete motion latent codes by predicting how point clouds evolve during manipulation rather than reconstructing visual observations. This four-dimensional objective -- spatial geometry changing through time -- forces latent representations to encode actual physical motion rather than appearance patterns. GeoMoLa achieves state-of-the-art performance using only single-view RGB-D input, while existing methods require multi-view reconstruction, succeeding across diverse manipulation benchmarks. Our ablations reveal that geometric prediction is the key to driving performance, quantitatively validating that manipulation depends on spatial understanding. Furthermore, the learned codes exhibit effective motion abstraction: applying them to novel scenes produces physically consistent transformations regardless of visual context. Our real-world experiments also confirm this robustness capability, achieving robust manipulation with minimal demonstrations in cluttered environments where geometric reasoning determines success. Thus, we demonstrate that effective motion latents for robot control can better emerge from understanding motion through its three-dimensional effects rather than pixel-level patterns.

Chinese Translation

机器人操控的运动潜变量学习在很大程度上依赖于从视觉序列中提取运动模式，而有效的动作抽象则需要理解三维几何变换。在此，我们引入了GeoMoLa（基于几何的运动潜变量），该方法通过预测点云在操控过程中的演变来学习离散的运动潜变量编码，而不是重建视觉观测。这一四维目标——空间几何随时间变化——迫使潜在表示编码实际的物理运动而非外观模式。GeoMoLa在仅使用单视角RGB-D输入的情况下实现了最先进的性能，而现有方法则需要多视角重建，并在多种操控基准测试中取得成功。我们的消融实验表明，几何预测是驱动性能的关键，定量验证了操控依赖于空间理解。此外，学习到的编码表现出有效的运动抽象：将其应用于新场景时，无论视觉上下文如何，都能产生物理一致的变换。我们的现实世界实验也证实了这一鲁棒性能力，在几何推理决定成功的杂乱环境中，仅需最少的演示即可实现鲁棒操控。因此，我们证明了有效的机器人控制运动潜变量可以更好地通过理解运动的三维效应而非像素级模式来产生。

View on arXiv Download PDF AI Translation

cs.RO / 74 / 2607.04719

Aerial Manipulation: Contact, Medium Coupling, and the Geometry of Readiness

空中操控：接触、介质耦合与准备几何

Franchi, Antonio

Abstract

Aerial robots are increasingly moving from remote observation toward physical interaction with objects, surfaces, structures, loads, and surrounding flows. This review argues that aerial manipulation cannot be understood as classical manipulation simply mounted on a flying base. Because flying agents remain aloft through continuous momentum and energy exchange with the surrounding medium, support, locomotion, stabilization, and task-directed interaction are intrinsically coupled. Building on broad views of manipulation as intentional environmental regulation through physical interaction, we propose a medium-aware interpretation of aerial manipulation in which interaction may be mediated by contact, by the surrounding fluid, or by both. The review organizes biological and robotic examples into a repertoire of interaction modes and a capability ladder, then develops an actuation-geometric viewpoint in which redundancy induces task-equivalent fibers. Internal motion along these fibers can trade energy for active readiness, aerodynamic promptness, and passive medium coupling. This perspective clarifies why aerial manipulation is difficult, why biological flyers remain broader than robotic systems, and how future platforms may command forces while also shaping how the medium acts back on them.

Chinese Translation

空中机器人正逐渐从远程观察转向与物体、表面、结构、负载和周围流体的物理互动。本文综述认为，空中操控不能简单地理解为安装在飞行基座上的经典操控。由于飞行体通过与周围介质的持续动量和能量交换保持悬浮，支撑、运动、稳定和任务导向的互动本质上是耦合的。基于对操控的广泛理解，即通过物理互动进行有意的环境调节，我们提出了一种介质感知的空中操控解释，其中互动可能通过接触、周围流体或两者的结合来介导。本文将生物和机器人实例组织成互动模式的库和能力阶梯，随后发展出一种激励-几何视角，其中冗余引入任务等效的纤维。沿这些纤维的内部运动可以在主动准备、空气动力学的迅速反应和被动介质耦合之间进行能量交换。这一视角阐明了为什么空中操控是困难的，为什么生物飞行器的能力超越机器人系统，以及未来的平台如何在指挥力量的同时塑造介质对它们的反作用。

View on arXiv Download PDF AI Translation

cs.RO / 75 / 2607.04739

Spatial Attention: Adapting Execution Horizons for Diffusion Policies via Observation Sensitivity

空间注意力：通过观察敏感性调整扩散策略的执行视野

Park, Che-Sang, Ha, Junsu, Fu, Jianlong, Park, Frank C.

Abstract

Sampling action chunks via generative models has become a widely adopted methodology for robotic learning from demonstration. However, existing methods often struggle to balance responsiveness and computational cost because they execute each action chunk for a fixed execution horizon. In this paper, we adaptively adjust the execution horizon of sampled action chunks, balancing responsiveness and computational efficiency. We introduce Spatial Attention -- defined as the expected squared norm of the gradient of the action log-likelihood with respect to the observation -- which indicates the sensitivity of the policy's action distribution to variations in the observation. We show that, under a fixed budget of chunk samplings, the execution horizon that minimizes the cumulative likelihood drop induced by disturbances decreases as Spatial Attention increases. By forecasting future Spatial Attention values alongside the action chunk, our framework dynamically assigns shorter execution horizons to phases with high Spatial Attention, and longer horizons to phases with low Spatial Attention. Experiments on standard and perturbed tasks, in both simulation and on a real robot, show that our method significantly improves success rates over fixed-horizon baselines while maintaining the average execution horizon.

Chinese Translation

通过生成模型对动作块进行采样已成为机器人示范学习中广泛采用的方法。然而，现有方法往往难以平衡响应性和计算成本，因为它们对每个动作块的执行视野是固定的。在本文中，我们自适应地调整采样动作块的执行视野，以平衡响应性和计算效率。我们引入了空间注意力（Spatial Attention）——定义为动作对数似然相对于观察的梯度的期望平方范数——它指示了策略的动作分布对观察变化的敏感性。我们展示了在固定的块采样预算下，最小化由干扰引起的累积似然下降的执行视野随着空间注意力的增加而减小。通过预测未来的空间注意力值以及动作块，我们的框架动态地为高空间注意力阶段分配较短的执行视野，而为低空间注意力阶段分配较长的执行视野。在标准和扰动任务上的实验，包括仿真和真实机器人，表明我们的方法在保持平均执行视野的同时，显著提高了成功率，超过了固定视野基线。

View on arXiv Download PDF AI Translation

cs.RO / 76 / 2607.04745

Trajectory-Anchor Optimization for Overconfident Thermal Visual Place Recognition: Zero-Leakage OOD Auditing and Kidnapped-Robot Recovery

过度自信的热视觉位置识别的轨迹锚优化：零泄漏的OOD审计与被绑架机器人恢复

Lu, Zhiyuan, Tanaka, Kanji

Abstract

Modern thermal visual place recognition (TIR-VPR) frontends based on foundation models achieve remarkable closed-set retrieval but suffer from an overconfident forced-matching failure mode. Under out-of-distribution (OOD) or unmapped conditions, they generate highly plausible yet false loop candidates without a drop in similarity scores. While classical multi-hypothesis tracking (MHT) backends can mitigate these ambiguities by maintaining divergent trajectory beliefs, their exponential computational overhead violates real-time robotic constraints. To bridge this gap, we present Trajectory-Anchor Optimization (TAO). To counter the combinatorial challenge of evaluating parallel hypotheses (e.g., K=100), TAO compresses multi-view temporal verification into a batched SE(2) Procrustes alignment problem. By leveraging tensor-level vectorization and single-invocation batched SVD, this formulation bypasses the dynamic tree expansion of MHT, guaranteeing a strictly bounded per-frame execution loop of O(KN). Under a strict zero-leakage evaluation protocol, we show that while a passive geometric backend cannot mathematically separate metric localization errors from coherent hallucinations at a micro-scale (<5m) due to local visual ambiguities, TAO serves as an efficient fail-safe filter at a macro-scale. Within a 5m radius, hallucinations often possess a locally consistent geometry that deceives rigid alignment. However, beyond this threshold, the K=100 disparate hypotheses disperse spatially across the global map. This dispersion breaks the rigid temporal co-visibility constraint within the sliding window (N=20), causing the joint optimization residual to escalate sharply. Consequently, TAO establishes a distinct macroscopic convergence basin (10m) where multi-view geometric consistency reliably isolates catastrophic topological breaks and suppresses critical false acceptances.

Chinese Translation

基于基础模型的现代热视觉位置识别（TIR-VPR）前端在闭集检索中表现出色，但在强制匹配失败模式下存在过度自信的问题。在分布外（OOD）或未映射条件下，它们会生成高度可信但错误的循环候选项，而相似度分数并未下降。虽然经典的多假设跟踪（MHT）后端可以通过维持不同的轨迹信念来缓解这些模糊性，但其指数级的计算开销违反了实时机器人约束。为了解决这一问题，我们提出了轨迹锚优化（TAO）。为了应对评估并行假设的组合挑战（例如，K=100），TAO将多视图时间验证压缩为一个批处理的SE(2) Procrustes对齐问题。通过利用张量级别的向量化和单次调用的批处理SVD，该公式绕过了MHT的动态树扩展，保证了每帧执行循环的严格界限为O(KN)。在严格的零泄漏评估协议下，我们展示了虽然被动几何后端无法在微观尺度（<5m）上数学上将度量定位误差与一致的幻觉分开，因为局部视觉模糊性，TAO在宏观尺度上作为一个高效的安全过滤器发挥作用。在5米半径内，幻觉通常具有局部一致的几何形状，欺骗刚性对齐。然而，超出这一阈值，K=100个不同的假设在全球地图上空间分散。这种分散打破了滑动窗口（N=20）内的刚性时间共视约束，导致联合优化残差急剧上升。因此，TAO建立了一个独特的宏观收敛区域（10m），在该区域内，多视图几何一致性可靠地隔离灾难性的拓扑断裂，并抑制关键的错误接受。

View on arXiv Download PDF AI Translation

cs.RO / 77 / 2607.04764

SLAM: Structured and Localized Analytic Manifold Adaptation for Lifelong VPR

SLAM：用于终身视觉位置识别的结构化和局部解析流形适应

Tsukahara, Kenta, Tanaka, Kanji, Hisada, Rai

Abstract

Visual Place Recognition (VPR) in lifelong deployment requires continuous adaptation to new environments without catastrophic forgetting. In this paper, we propose SLAM, a Structured and Localized Analytic Manifold adaptation framework. Our framework elegantly unifies uncertainty-aware smoothing via Unscented transformation, topological space partitioning through a Gaussian Mixture Model (GMM), and $H_\infty$ robust bound optimization into a singular, unified closed-form analytical recursion. Exhaustive ablation studies demonstrate that while the synergistic combination of uncertainty smoothing and localized mapping (U+G configuration) achieves the state-of-the-art nominal accuracy of 27.5%, the full deployment of the $H_\infty$ bound does not require an architectural split; rather, it introduces a mathematically guaranteed minimax robust bound. This formulation enables the system to seamlessly modulate the intrinsic trade-off between nominal placement precision and worst-case disturbance attenuation through a single regularization parameter.

Chinese Translation

终身部署中的视觉位置识别（VPR）需要对新环境进行持续适应，而不发生灾难性遗忘。本文提出了SLAM，一个结构化和局部解析流形适应框架。我们的框架优雅地统一了通过无迹变换进行的不确定性感知平滑、通过高斯混合模型（Gaussian Mixture Model, GMM）进行的拓扑空间划分，以及$H_ ext{∞}$ 鲁棒界优化，形成一个单一的、统一的闭式解析递归。详尽的消融研究表明，尽管不确定性平滑与局部映射的协同组合（U+G配置）达到了27.5%的最先进名义精度，但$H_ ext{∞}$ 界的完全部署并不需要架构分割；相反，它引入了一个数学上保证的最小最大鲁棒界。这一公式化使得系统能够通过单一的正则化参数无缝调节名义放置精度与最坏情况干扰衰减之间的内在权衡。

View on arXiv Download PDF AI Translation

cs.RO / 78 / 2607.04816

CAC-VLA: Context-Gated Action Conditioning for Vision-Language-Action Models

CAC-VLA：用于视觉-语言-动作模型的上下文门控动作条件化

Xiong, Yifu, Yu, Wenhao, Lin, Jiaxuan, Zou, Bojun, Li, Jiahao, Zhang, Lu, Zhang, Yanyong, Ji, Jianmin

Abstract

Vision-Language-Action (VLA) models have become a promising paradigm for generalist robot manipulation, where visual-language representations are used to condition continuous action generation. However, these representations are not explicitly optimized for action conditioning, leaving the action expert to bridge the gap between multimodal understanding and precise motor control. Recent action-reasoning methods introduce additional modules to generate explicit action plans or action-space reasoning signals, demonstrating the benefit of action-level guidance but often requiring separate action-generation frameworks. We propose CAC-VLA, a Context-Gated Action Conditioning framework that learns a lightweight latent-action interface directly within the VLM. Instead of generating executable trajectories, CAC-VLA trains the VLM to predict coarse-to-fine latent actions, which are structured representations encoded from future action segments, and adaptively leverages them to condition the action expert via a context gate. This enables VLM-native action conditioning while calibrating the influence of latent-action guidance on expert action generation. Experiments on LIBERO and LIBERO-Plus demonstrate the effectiveness of CAC-VLA, achieving 98.3% average success rate on LIBERO and 89.5% LIBERO-Plus, suggesting that context-gated latent-action conditioning is an effective interface for continuous expert control.

Chinese Translation

视觉-语言-动作（VLA）模型已成为通用机器人操作的有前景范式，其中视觉-语言表示用于条件化连续动作生成。然而，这些表示并未明确针对动作条件化进行优化，导致动作专家需要弥补多模态理解与精确运动控制之间的差距。最近的动作推理方法引入了额外模块，以生成明确的动作计划或动作空间推理信号，展示了动作级指导的好处，但通常需要独立的动作生成框架。我们提出了CAC-VLA，一种上下文门控动作条件化框架，直接在视觉-语言模型（VLM）中学习轻量级潜在动作接口。CAC-VLA并不是生成可执行的轨迹，而是训练VLM预测粗到细的潜在动作，这些潜在动作是从未来动作片段编码的结构化表示，并通过上下文门自适应地利用它们来条件化动作专家。这使得VLM本地的动作条件化成为可能，同时校准潜在动作指导对专家动作生成的影响。在LIBERO和LIBERO-Plus上的实验表明CAC-VLA的有效性，在LIBERO上实现了98.3%的平均成功率，在LIBERO-Plus上为89.5%，这表明上下文门控潜在动作条件化是连续专家控制的有效接口。

View on arXiv Download PDF AI Translation

cs.RO / 79 / 2607.04837

Athena-WBC: Capability-Aligned Policy Experts for Long-Tail Humanoid Whole-Body Control

Athena-WBC：针对长尾类人形全身控制的能力对齐策略专家

Jiang, Yuan, Zhang, Ningyuan, Yang, Xicun, Li, Shidi, Jiang, Yuzhi, Rong, Zhiyi, Ma, Shuaikang, Li, Chuanzheng, Chen, Jie

Abstract

Large-scale humanoid motion-tracking controllers are commonly improved by reallocating training effort: difficult motions are sampled more often, isolated into smaller subsets, or assigned to specialized experts. We show that this view is incomplete. In strong whole-body-control baselines, a residual set of feasible training clips remains unsolved even under targeted training, especially for high-dynamic transitions and balance-critical motions. These failures arise not only from insufficient exposure, but from a mismatch between the motion demands and the effective capability induced by the default training recipe. We propose Athena-WBC, a compact teacher-student pipeline with capability-aligned policy experts for long-tail humanoid whole-body control. Dynamic experts use a tracking-focused, constraint-aware objective that removes conservative effort and temporal-control penalties while preserving physical feasibility constraints; balance experts use a gravity curriculum to improve early-training survivability. The resulting privileged teachers are motion-routed for DAgger distillation and then compressed into a single controller with deployable observations followed by RL fine-tuning. Experiments on a full-size humanoid show improved recovery of training-set long-tail motions and better held-out tracking than a strong SONIC-recipe baseline, using only a small number of experts.

Chinese Translation

大规模类人形运动跟踪控制器通常通过重新分配训练努力来改善：困难的动作被更频繁地采样、分离成更小的子集，或分配给专业的专家。我们展示了这种观点的不完整性。在强大的全身控制基线中，即使在针对性训练下，仍然存在一组残余的可行训练片段未能解决，特别是在高动态过渡和对平衡至关重要的动作中。这些失败不仅源于曝光不足，还源于运动需求与默认训练方案所引发的有效能力之间的不匹配。我们提出了Athena-WBC，这是一种紧凑的教师-学生管道，具有能力对齐的策略专家，专注于长尾类人形全身控制。动态专家使用以跟踪为中心、关注约束的目标，去除保守努力和时间控制惩罚，同时保持物理可行性约束；平衡专家使用重力课程来提高早期训练的生存能力。最终产生的特权教师经过运动路由进行DAgger蒸馏，然后压缩成一个具有可部署观察的单一控制器，随后进行强化学习微调。在全尺寸类人形上的实验显示，使用仅少量专家的情况下，训练集长尾动作的恢复得到了改善，并且跟踪效果优于强大的SONIC配方基线。

View on arXiv Download PDF AI Translation

cs.RO / 80 / 2607.04879

WinTA-GIL: Windowed Trajectory Alignment for GNSS-IMU-LiDAR Heading Refinement in Intermittent Signal Environments

WinTA-GIL：在间歇信号环境中进行GNSS-IMU-LiDAR航向精细化的窗口轨迹对齐

Feng, Kaixin, Wen, Zhichao, Liao, Zhaohong, Xia, Xin, Li, You

Abstract

Although multi-source fusion positioning systems have achieved significant progress, accurate and reliable heading estimation remains a critical challenge due to the lack of gravitational constraints and the inherent weak observability of heading in complex environments. Most existing methodologies are specifically tailored for the startup phase, relying on a singular initial alignment to establish the heading reference. Consequently, these approaches lack the adaptability required to refine heading estimates dynamically, which renders the system highly vulnerable to accumulated drift and observation noise during prolonged navigation or immediately following GNSS signal outages. To address these limitations, this paper proposes WinTA-GIL, a novel heading refinement framework that integrates information from Global Navigation Satellite System (GNSS), Inertial Measurement Unit (IMU), and Light Detection and Ranging (LiDAR) through a temporal window-based optimization strategy. Unlike conventional alignment methods restricted to the startup phase, WinTA-GIL leverages high-precision local trajectories from LiDAR-Inertial Odometry (LIO) to register against filtered GNSS observations. This approach transforms heading estimation into a repeatable, trajectory-based consistency optimization problem. In particular, an adaptive re-estimation mechanism based on state discrimination is incorporated to trigger heading corrections whenever necessary, thereby effectively suppressing the inertial drift accumulated during challenging conditions. Extensive experiments on both open-source and self-collected datasets demonstrate that WinTA-GIL significantly outperforms state-of-the-art approaches in both estimation accuracy and system robustness.

Chinese Translation

尽管多源融合定位系统已取得显著进展，但由于缺乏重力约束以及复杂环境中航向的固有弱可观测性，准确可靠的航向估计仍然是一个关键挑战。现有的大多数方法特别针对启动阶段，依赖单一的初始对齐来建立航向参考。因此，这些方法缺乏动态精细化航向估计所需的适应性，使得系统在长时间导航或GNSS信号中断后极易受到累积漂移和观测噪声的影响。为了解决这些局限性，本文提出了WinTA-GIL，一种新颖的航向精细化框架，通过基于时间窗口的优化策略整合全球导航卫星系统（GNSS）、惯性测量单元（IMU）和光学探测与测距（LiDAR）信息。与传统的仅限于启动阶段的对齐方法不同，WinTA-GIL利用来自激光惯性测距（LIO）的高精度局部轨迹与过滤后的GNSS观测进行配准。该方法将航向估计转化为一个可重复的基于轨迹的一致性优化问题。特别地，本文引入了一种基于状态判别的自适应重新估计机制，以在必要时触发航向校正，从而有效抑制在复杂条件下累积的惯性漂移。在开放源代码和自收集数据集上的大量实验表明，WinTA-GIL在估计精度和系统鲁棒性方面显著优于现有的最先进方法。

View on arXiv Download PDF AI Translation

cs.RO / 81 / 2607.04880

PRISM: Personalized Robotic Dataset Generation via Image-based Scene and Motion Synthesis

PRISM：通过基于图像的场景和运动合成生成个性化机器人数据集

Ko, Dogyu, Kim, Haneul, Yeo, Chanyoung, Lee, Dowoon, Park, Taeho, Hwang, Hyoseok

Abstract

Recent advances in large-scale pretrained vision-language-action models have improved robot policy learning, but directly deploying such policies in user-specific environments remains challenging due to limited generalization, which inevitably requires collecting a dataset tailored to the target environment. Teleoperation yields well-aligned data but is costly and difficult to scale, whereas simulation scales easily but struggles to resemble the target environment and generate task-specific trajectories. To meet both simultaneously, we propose PRISM, an end-to-end pipeline that generates personalized robotic datasets from a single image and a natural-language instruction. PRISM constructs digital cousin scenes that are semantically and geometrically aligned with the user environment yet diverse at the instance level, and synthesizes executable demonstrations without human teleoperation. Extensive experiments show that policies trained on PRISM-generated datasets outperform those trained on baseline-generated datasets on LIBERO and LIBERO-Plus, achieve up to 100\% success rate on three real-world manipulation tasks, and maintain stronger performance when evaluated in environments that differ from those seen during training.

Chinese Translation

近期在大规模预训练视觉-语言-动作模型方面的进展改善了机器人策略学习，但在用户特定环境中直接部署此类策略仍然具有挑战性，因为其泛化能力有限，这不可避免地需要收集针对目标环境量身定制的数据集。遥操作能够生成良好对齐的数据，但成本高且难以扩展，而仿真则容易扩展，但难以与目标环境相似并生成任务特定的轨迹。为同时满足这两者，我们提出了PRISM，一个从单张图像和自然语言指令生成个性化机器人数据集的端到端管道。PRISM构建了与用户环境在语义和几何上对齐但在实例级别上多样化的数字表亲场景，并合成可执行的演示，无需人工遥操作。大量实验表明，在PRISM生成的数据集上训练的策略在LIBERO和LIBERO-Plus上优于在基线生成的数据集上训练的策略，在三个真实世界的操作任务中达到高达100\%的成功率，并在评估与训练时见到的环境不同的情况下保持更强的性能。

View on arXiv Download PDF AI Translation

cs.RO / 82 / 2607.04927

DSWAM: A Dual-System World Action Foundation Model for Fine-Grained Robot Manipulation

DSWAM：一种用于细粒度机器人操作的双系统世界行动基础模型

Zhu, Jian, Zhang, Jianjun, Su, Taiyi, Liu, Tianbin, Wang, Zhangyuan, Xie, Kai, Huang, Zitai, Ma, Chong, He, Youzhang, Wang, Tianjian, Wang, Hanyang, Ding, Weihao, Xu, Yi

Abstract

World Action Models (WAMs) provide a promising alternative to Vision-Language-Action (VLA) policies by using video-based world modeling as dense supervision for robot action learning. Existing WAMs excel at physically grounded execution, but typically lack the explicit language-level planning interface in VLM-based VLAs for decomposing coarse instructions. Such decomposition becomes important when household tasks involve complex multi-step goals, where coarse user commands need to be converted into sequences of fine-grained executable subtasks. Meanwhile, the field still lacks a fair real-robot comparison between VLA and WAM execution capabilities, since existing systems often differ in data, robot embodiments, and task protocols. To address both the decomposition gap and the need for a controlled WAM-VLA comparison, we introduce DSWAM, a Dual-System World Action Foundation Model for fine-grained robot manipulation. DSWAM keeps a System 1 WAM executor as the default control path and optionally activates a System 2 vision-language subtask planner only when task decomposition is useful. The planner predicts executable subtasks from short-term visual history and a global task prompt, while the WAM executor performs world-aware action generation for each instruction or subtask. The executor is trained with action prediction and video co-training, but inference directly predicts action chunks without explicit future video generation. To make this execution path practical on real robots, we further integrate TensorRT acceleration, asynchronous execution, and real-time chunking (RTC) so that policy queries do not block robot control. To provide a fair real-robot comparison with VLA policies, we build and evaluate DSWAM under the DeMaVLA real-world deformable manipulation setting with matched robot platform, pretraining data, post-training data, and evaluation criteria.

Chinese Translation

世界行动模型（WAMs）通过使用基于视频的世界建模作为机器人行动学习的密集监督，提供了一个有前景的替代方案，取代视觉-语言-行动（VLA）策略。现有的WAM在物理执行方面表现出色，但通常缺乏VLM基础的VLA中用于分解粗略指令的显式语言级规划接口。当家庭任务涉及复杂的多步骤目标时，这种分解变得尤为重要，因为粗略的用户命令需要转换为细粒度可执行子任务的序列。同时，领域内仍缺乏VLA与WAM执行能力之间的公平真实机器人比较，因为现有系统在数据、机器人形态和任务协议上往往存在差异。为了解决分解差距和对受控WAM-VLA比较的需求，我们提出了DSWAM，一种用于细粒度机器人操作的双系统世界行动基础模型。DSWAM将系统1的WAM执行器作为默认控制路径，并在任务分解有用时可选地激活系统2的视觉-语言子任务规划器。该规划器根据短期视觉历史和全局任务提示预测可执行子任务，而WAM执行器则为每个指令或子任务执行世界感知的行动生成。执行器通过行动预测和视频共同训练进行训练，但推理直接预测行动块，而无需显式的未来视频生成。为了使这一执行路径在真实机器人上可行，我们进一步集成了TensorRT加速、异步执行和实时块处理（RTC），以确保策略查询不会阻塞机器人控制。为了与VLA策略进行公平的真实机器人比较，我们在DeMaVLA真实世界可变形操作设置下构建并评估DSWAM，确保机器人平台、预训练数据、后训练数据和评估标准的一致性。

View on arXiv Download PDF AI Translation

cs.RO / 83 / 2607.04940

Closing the Reality Gap: Zero-Shot Sim-to-Real Deployment for Dexterous Force-Based Grasping and Manipulation

缩小现实差距：零-shot模拟到现实的灵巧基于力的抓取和操作部署

Zhao, Zhe, Li, Zhibin, Ou, Yilin, Qi, Mengshi

Abstract

Human-like dexterous hands with multiple fingers offer human-level manipulation capabilities but remain difficult to train the control policies that can deploy on real hardware due to contact-rich physics and imperfect actuation. We present a sim-to-real reinforcement learning method that leverages dense tactile feedback combined with joint torque sensing to explicitly regulate physical interactions. To enable effective sim-to-real transfer, we introduce (i) a computationally fast tactile simulation that computes distances between dense virtual tactile units and the object via parallel forward kinematics, providing high-rate, high-resolution touch signals needed by RL; (ii) a current-to-torque calibration that eliminates the need for torque sensors on dexterous hands by mapping motor current to joint torque; and (iii) actuator dynamics modeling with randomization to account for non-ideal torque-speed effects and bridge the actuation gaps. Using an asymmetric actor-critic PPO pipeline, we train policies entirely in simulation and deploy them directly to a five-finger hand. The resulting policies demonstrate two essential human-hand skills: (1) command-based controllable grasp force tracking and (2) reorientation of objects in the hand, both of which are robustly executed without fine-tuning on the robot. By combining tactile and torque in the observation space with scalable sensing and actuation modeling, our system provides a practical solution to achieve reliable dexterous manipulation. To our knowledge, this is the first demonstration of controllable grasping on a multi-finger dexterous hand trained entirely in simulation and transferred zero-shot on real hardware.

Chinese Translation

类人灵巧手具备多指功能，能够实现人类水平的操作能力，但由于接触丰富的物理特性和不完美的驱动控制，训练可在真实硬件上部署的控制策略仍然困难。我们提出了一种模拟到现实的强化学习方法，该方法利用密集的触觉反馈结合关节扭矩传感来明确调节物理交互。为了实现有效的模拟到现实转移，我们引入了（i）一种计算速度快的触觉仿真，通过并行正向运动学计算密集虚拟触觉单元与物体之间的距离，提供RL所需的高频率、高分辨率触觉信号；（ii）一种电流到扭矩的标定方法，通过将电机电流映射到关节扭矩，消除了在灵巧手上使用扭矩传感器的需求；（iii）带随机化的驱动器动态建模，以考虑非理想的扭矩-速度效应并弥补驱动间隙。我们使用不对称的演员-评论家PPO管道，在仿真中完全训练策略，并将其直接部署到五指手上。所得到的策略展示了两项基本的人手技能：（1）基于指令的可控抓取力跟踪和（2）手中物体的重新定向，这两者均在不进行微调的情况下稳健执行。通过将触觉和扭矩结合在观察空间中，并结合可扩展的传感和驱动建模，我们的系统提供了一种实现可靠灵巧操作的实用解决方案。据我们所知，这是首次在完全仿真中训练并零-shot转移到真实硬件的多指灵巧手上实现可控抓取的演示。

View on arXiv Download PDF AI Translation

cs.RO / 84 / 2607.04972

Multi-Robot Open Adaptive Teaming Across Unseen Environments, Partners, and Scales

跨越未知环境、合作伙伴和规模的多机器人开放自适应团队

Li, Yang, Xue, Feng, Mo, Fan, Liu, Yunhao, Wang, Jianhong, Wen, Ying, Zhang, Qingrui, Mou, Shaoshuai, Pan, Wei

Abstract

Deploying robot teams in the real world requires simultaneous adaptation to unseen environments, unknown partners, and varying team sizes, yet existing approaches often address these challenges in isolation under the closed-world assumption of fixed teammates. We formalize this as open adaptive multi-robot teaming and propose a hypergraphic-form game formulation that captures team-level cooperative relationships beyond pairwise interactions, providing a principled foundation for coordination structure inference when team composition changes dynamically within episodes. Unlike graph neural network architectures, this is a game-theoretic construct for modeling strategic interactions and payoff structures among agents. Building on this formulation, we develop the Hypergraphic Open-ended Learning Algorithm (HOLA), which progressively expands partner and environment diversity during training rather than optimizing for fixed configurations. Evaluated on cooperative pursuit with multi-drone and multi-quadruped platforms, HOLA outperforms all baselines across all three adaptability dimensions. Learned policies transfer directly to physical hardware without fine-tuning, with successful deployments on Crazyflie and Zsibot L1 platforms confirming robust real-world coordination in novel environments with unseen teammates.

Chinese Translation

在现实世界中部署机器人团队需要同时适应未知环境、未知合作伙伴和变化的团队规模，但现有方法往往在固定队友的封闭世界假设下孤立地解决这些挑战。我们将其形式化为开放自适应多机器人团队，并提出了一种超图形式的博弈模型，该模型捕捉了超越成对交互的团队级合作关系，为在剧集内团队组成动态变化时的协调结构推断提供了原则性基础。与图神经网络架构不同，这是一种用于建模代理之间战略互动和收益结构的博弈论构造。在此基础上，我们开发了超图开放式学习算法（Hypergraphic Open-ended Learning Algorithm, HOLA），该算法在训练过程中逐步扩展合作伙伴和环境的多样性，而不是针对固定配置进行优化。在多无人机和多四足平台的合作追逐任务中评估，HOLA在所有三个适应性维度上均优于所有基线。学习到的策略可以直接迁移到物理硬件上，无需微调，在Crazyflie和Zsibot L1平台上的成功部署确认了在新环境中与未知队友的强大现实世界协调能力。

View on arXiv Download PDF AI Translation

cs.RO / 85 / 2607.04981

Designing Touch for Trauma-Informed Social Robots: A Design Space for Direct and Indirect Actuation

为创伤知情社交机器人设计触觉：直接与间接驱动的设计空间

Rischer, Madeleine, Bußmann, Benedikt

Abstract

Touch is a fundamental communication modality in human-robot interaction and may support grounding, emotional regulation, and stress reduction in therapeutic contexts. However, designing touch-based interactions for individuals with post-traumatic stress disorder (PTSD) requires careful consideration of trauma-informed care (TIC) principles, including safety, transparency, autonomy, and trigger avoidance. This paper investigates how touch actuation in social robots can be designed to align with trauma-informed care. We distinguish between direct touch, involving physical contact between a user and a robot, and indirect touch, mediated through artifacts such as wearables or smart textiles. Building on this distinction, we propose a design space comprising three dimensions: Actuation Modality, Objective of the Actuation, and Intended Effect of the Actuation. We analyze these dimensions through the lens of trauma-informed care and derive key design considerations for touch-based interventions. The resulting approach provides a conceptual foundation for developing trauma-sensitive social robots that support individuals with PTSD through touch-based interactions.

Chinese Translation

触觉是人机交互中一种基本的沟通方式，能够在治疗环境中支持基础建立、情感调节和压力减轻。然而，为创伤后应激障碍（PTSD）患者设计基于触觉的交互时，需要仔细考虑创伤知情护理（TIC）原则，包括安全性、透明性、自主性和触发因素的避免。本文探讨了如何设计社交机器人中的触觉驱动，以符合创伤知情护理的要求。我们区分了直接触觉，即用户与机器人之间的物理接触，以及间接触觉，通过可穿戴设备或智能纺织品等物件进行中介。基于这一区分，我们提出了一个包含三个维度的设计空间：驱动方式、驱动目标和驱动意图效果。我们通过创伤知情护理的视角分析这些维度，并得出基于触觉干预的关键设计考虑因素。最终的方法为开发能够通过触觉交互支持PTSD患者的创伤敏感社交机器人提供了概念基础。

View on arXiv Download PDF AI Translation

cs.RO / 86 / 2607.04988

InternVLA-A1.5: Unifying Understanding, Latent Foresight, and Action for Compositional Generalization

InternVLA-A1.5：统一理解、潜在前瞻与动作以实现组合泛化

Ma, Haoxiang, Cai, Junhao, Xu, Xiaoxu, Li, Hao, Yang, Yuyin, Tian, Yang, Cao, Jiafei, Zhu, Hongrui, Qiu, Zherui, Zhaxizhuoma, Yang, Yuqiang, Peng, Jiaqi, Wei, Xueyuan, Zhu, Yangkun, Jiang, Jiahao, Gao, Xing, Wang, Hanqing, Yuan, Feng, Li, Kailin, Zhu, Xueyue, Wang, Tai, Ding, Yan, Pang, Jiangmiao, Zeng, Jia, Zhang, Jingjing, Zhou, Bowen, Mu, Yao, Shen, Chunhua, Zhang, Weinan

Abstract

Unified models for robot manipulation aim to equip one policy with both the semantic priors of pretrained VLMs and the physical dynamics learned through future prediction. In practice, existing designs tend to erode the semantics of the pretrained backbone, suffer interference among heterogeneous objectives, and learn future prediction from scratch in pixel space, leaving the dynamics priors of pretrained video generators unexploited. We present InternVLA-A1.5, which builds the policy on a native VLM backbone that keeps training on VQA and subtask prediction, and attaches a lightweight unified expert for continuous action generation. Future prediction is recast as a latent-querying problem, where a small set of learnable foresight tokens condenses the task-relevant future into a compact latent code under the supervision of a frozen pretrained video generation model, so the policy inherits world-model dynamics priors without ever learning pixel-level generation. The video branch is discarded at inference, keeping real-time control. Pretrained on 1.2M robot episodes and 3M multimodal samples, InternVLA-A1.5 achieves the best overall results on all six simulation benchmarks. In the real world, the preserved semantics deliver the strongest compositional generalization on held-out instruction bindings, and the two designs together sustain long-horizon execution.

Chinese Translation

统一的机器人操作模型旨在为一个策略提供预训练视觉语言模型（VLM）的语义先验和通过未来预测学习到的物理动态。在实际应用中，现有设计往往会削弱预训练主干的语义，受到异质目标之间的干扰，并在像素空间中从头学习未来预测，未能充分利用预训练视频生成器的动态先验。我们提出了InternVLA-A1.5，该模型基于一个本地VLM主干构建策略，持续在视觉问答（VQA）和子任务预测上进行训练，并附加一个轻量级的统一专家以实现连续动作生成。未来预测被重新表述为一个潜在查询问题，其中一小组可学习的前瞻令牌在冻结的预训练视频生成模型的监督下，将任务相关的未来浓缩为紧凑的潜在编码，从而使策略在不学习像素级生成的情况下继承世界模型的动态先验。在推理时，视频分支被丢弃，以保持实时控制。经过120万机器人实验和300万多模态样本的预训练，InternVLA-A1.5在所有六个仿真基准上取得了最佳的整体结果。在现实世界中，保留的语义在保留的指令绑定上提供了最强的组合泛化，而这两种设计共同支持长时间的执行。

View on arXiv Download PDF AI Translation

cs.RO / 87 / 2607.05092

ECO: Incremental Ego-Centric Octree Update for Point Streams

ECO：增量自我中心八叉树更新用于点流

Yu, Jaemin, Jeong, Seongyoon, Chon, Kang-Wook, Kim, Duksu

Abstract

Constructing octrees for mobile robots that process continuous point streams in real time poses significant computational and memory challenges. Standard global structures often suffer from high latency and unbalanced tree growth. We introduce the Ego-Centric Octree (ECO), a spatial data structure that acts as a 3D sliding window, dynamically bounding the mapping space to the robot's immediate surroundings. ECO uses an efficient incremental update algorithm that categorizes the environment into shift-out, shift-in, and overlap regions, eliminating redundant global coordinate transformations. Evaluations on the KITTI benchmark demonstrate that ECO reduces update times by up to 25.60% (24.87% on average) compared to full static reconstruction and by up to 67.52% (54.60% on average) compared to a bounded incremental baseline. Furthermore, ECO substantially lowers the total system latency of downstream tasks, running up to 34.17% faster than full reconstruction in voxel-map generation. In dynamic scenes, ECO naturally retains a short-term temporal memory of moving objects, providing useful temporal context while keeping update cost bounded and the tree balanced for real-time spatial perception.

Chinese Translation

为处理实时连续点流的移动机器人构建八叉树面临着显著的计算和内存挑战。标准的全局结构通常存在高延迟和不平衡的树生长问题。我们提出了自我中心八叉树（Ego-Centric Octree, ECO），这是一种空间数据结构，充当3D滑动窗口，动态地将映射空间限制在机器人的即时周围环境中。ECO采用高效的增量更新算法，将环境分类为移出、移入和重叠区域，消除了冗余的全局坐标转换。在KITTI基准测试中的评估表明，与完全静态重建相比，ECO将更新时间减少了多达25.60%（平均减少24.87%），与有界增量基线相比，减少了多达67.52%（平均减少54.60%）。此外，ECO显著降低了下游任务的总系统延迟，在体素地图生成中比完全重建快了多达34.17%。在动态场景中，ECO自然保留了移动物体的短期时间记忆，提供了有用的时间上下文，同时保持更新成本有限，并使树保持平衡，以实现实时空间感知。

View on arXiv Download PDF AI Translation

cs.RO / 88 / 2607.05162

Geometry-Aware Visual Odometry for Bronchoscopic Navigation via High-Gain Observer Fusion

基于几何感知的视觉里程计在支气管镜导航中的应用：高增益观测器融合

Kasaei, Mohammadreza, Zhang, Francis Xiatian, Li, Feng, Alambeigi, Farshid, Dhaliwal, Kevin, Khadem, Mohsen

Abstract

Navigational bronchoscopy is critical for pulmonary interventions, yet current platforms depend heavily on pre-operative CT or external sensors, limiting their use in critical care and resource-constrained settings. Vision-only navigation offers a scalable alternative, but conventional visual odometry (VO) struggles with texture-poor airway images, specularities, and the vanishing-point singularities of tubular anatomy, leading to frequent tracking failures and drift. We present a geometry-aware VO framework that explicitly leverages vanishing-point cues from airway lumens. Detected lumens are back-projected to 3D rays, whose weighted fusion yields a stable forward heading even when parallax cues are absent. This heading, together with looming-based velocity estimates, is fused with noisy VO outputs using a bespoke high-gain observer that enforces airway-following priors and rejects drift. We validate the method on ex-vivo mechanically ventilated human lungs with electromagnetic tracking ground truth. Compared to state-of-the-art pipelines (ORB-SLAM2, LoFTR-VO, DPVO), our approach reduces absolute trajectory error by more than 50% and achieves the lowest relative pose error across all test sequences.

Chinese Translation

导航支气管镜检查对肺部干预至关重要，但目前的平台过于依赖术前CT或外部传感器，限制了其在重症监护和资源有限环境中的应用。仅依靠视觉的导航提供了一种可扩展的替代方案，但传统的视觉里程计（VO）在纹理较少的气道图像、镜面反射和管状解剖的消失点奇点方面表现不佳，导致频繁的跟踪失败和漂移。我们提出了一种几何感知的VO框架，明确利用气道腔体的消失点线索。检测到的腔体被反投影到3D光线中，其加权融合即使在缺乏视差线索的情况下也能产生稳定的前进方向。该方向与基于逼近的速度估计结合，使用定制的高增益观测器进行融合，强制执行气道跟随先验并拒绝漂移。我们在具有电磁追踪基准的离体机械通气人类肺部上验证了该方法。与最先进的管道（ORB-SLAM2、LoFTR-VO、DPVO）相比，我们的方法将绝对轨迹误差减少了50%以上，并在所有测试序列中实现了最低的相对位姿误差。

View on arXiv Download PDF AI Translation

cs.RO / 89 / 2607.05180

VLM-CASE: Vision-Language Model Enabled Context-Adaptive Safety Envelopes for Anticipatory Safe Autonomous Driving

VLM-CASE：基于视觉-语言模型的上下文自适应安全边界用于预期安全的自主驾驶

Yang, Tianjia, Li, Ke, Qin, Ruwen, Hu, Xianbiao

Abstract

Adverse driving conditions, such as bad weather, remain a principal barrier to autonomous driving because they degrade two things at once: what the vehicle can perceive and what it can physically do. Human drivers cope by anticipation, reasoning about the scene and re-budgeting speed, following distance, and steering before grip or sight is lost, whereas current autonomous driving systems at best react after the fact. This paper proposes VLM-CASE, a framework that gives an autonomous vehicle this anticipatory capacity while keeping its motion bounded by a formal safety model at all times. A vision-language model (VLM), fine-tuned with low-rank adaptation (LoRA), reasons about the scene from the front-camera image and reports the road surface and visibility conditions. This output parametrizes a context-adaptive safety envelope (CASE), derived from physical limits and the guarantees of responsibility-sensitive safety, that couples braking and steering through a shared friction budget. A model predictive controller then drives freely within the envelope, while the VLM runs asynchronously so it never blocks the real-time control loop. We validate the framework in closed-loop CARLA simulation on tasks that demand both lateral and longitudinal control, across a range of weather, road-surface, and lighting conditions. The resulting controller, VLM-CASE-MPC, completes all trials, outperforming a conventional MPC baseline and a state-of-the-art VLM-integrated controller. Ablations confirm that the gains come from context adaptation, with the friction and visibility adaptations proving complementary. Furthermore, the framework is controller-agnostic and pairs with almost any low-level controller, offering a promising direction for safe autonomous driving. The dataset and supplementary materials for VLM-CASE are available at https://github.com/ytj254/VLM-CASE.

Chinese Translation

恶劣的驾驶条件，如恶劣天气，仍然是自主驾驶的主要障碍，因为它们同时降低了车辆的感知能力和物理操作能力。人类驾驶员通过预判来应对这一挑战，他们在失去抓地力或视线之前，会对场景进行推理并重新调整速度、跟车距离和转向，而当前的自主驾驶系统最多只能事后反应。本文提出了VLM-CASE，一个框架，使自主车辆具备这种预判能力，同时始终保持其运动受限于一个正式的安全模型。一个经过低秩适应（LoRA）微调的视觉-语言模型（VLM）根据前置摄像头图像对场景进行推理，并报告路面和能见度条件。该输出参数化了一个上下文自适应安全边界（CASE），该边界源自物理限制和责任敏感安全的保证，通过共享的摩擦预算将制动和转向结合在一起。然后，模型预测控制器在安全边界内自由驱动，而VLM异步运行，确保不会阻塞实时控制循环。我们在闭环CARLA仿真中验证了该框架，任务要求同时进行横向和纵向控制，涵盖多种天气、路面和光照条件。最终的控制器VLM-CASE-MPC完成了所有试验，超越了传统的模型预测控制（MPC）基线和最先进的VLM集成控制器。消融实验确认了收益来自上下文适应，摩擦和能见度的适应性证明是互补的。此外，该框架与控制器无关，可以与几乎任何低级控制器配对，为安全自主驾驶提供了一个有前景的方向。VLM-CASE的数据集和补充材料可在https://github.com/ytj254/VLM-CASE获取。

View on arXiv Download PDF AI Translation

cs.RO / 90 / 2607.05241

GelNeuro: A Sensing-Computing Integrated Neuromorphic Tactile System for Texture Recognition

GelNeuro：一种用于纹理识别的传感-计算集成神经形态触觉系统

Bian, Luoyang, Meng, Xinpan, Ma, Zhenghua, Li, Houcheng, Cheng, Long

Abstract

Neuromorphic visuo-tactile sensing offers a promising paradigm for low-latency and low-power robotic perception. However, existing systems still rely heavily on a host computer for event readout, preprocessing, or relaying prior to chip inference. This paper presents GelNeuro, a fully integrated sensing-computing visuo-tactile system that directly pairs a GelSight Mini-based optical tactile front end with the Speck2f neuromorphic system-on-chip (SoC). Contact-induced marker motions are captured as dynamic vision sensor (DVS) events and routed through the on-chip network to a spiking convolutional neural network (SCNN) classifier. To mitigate accuracy degradation during 8-bit deployment, a hardware-aware weight clamping strategy is introduced. Evaluated on a 15-class natural texture recognition task, hardware-in-the-loop testing on the physical chip achieves a 96.3% accuracy within an 80 ms inference window. Notably, the system consumes only 19.6 mW of board-level active power-over three orders of magnitude lower than conventional CPU/GPU baselines on the same benchmark. GelNeuro also exhibits robust generalization across unseen contact depths, demonstrating the viability of direct sensor-to-chip tactile recognition on edge neuromorphic hardware.

Chinese Translation

神经形态视觉-触觉传感提供了一种低延迟和低功耗的机器人感知新范式。然而，现有系统仍然在事件读取、预处理或推理之前严重依赖主机计算机。本文提出了GelNeuro，这是一种完全集成的传感-计算视觉-触觉系统，直接将基于GelSight Mini的光学触觉前端与Speck2f神经形态系统芯片（SoC）相结合。接触引起的标记运动被捕捉为动态视觉传感器（DVS）事件，并通过片上网络路由到脉冲卷积神经网络（SCNN）分类器。为了减轻8位部署过程中的准确性下降，提出了一种硬件感知的权重夹紧策略。在一个15类自然纹理识别任务上进行评估，物理芯片上的硬件在环测试在80毫秒的推理窗口内达到了96.3%的准确率。值得注意的是，该系统的板级主动功耗仅为19.6毫瓦，比同一基准下的传统CPU/GPU基线低三个数量级。GelNeuro还在未见接触深度上表现出强大的泛化能力，证明了在边缘神经形态硬件上直接进行传感器到芯片的触觉识别的可行性。

View on arXiv Download PDF AI Translation

cs.RO / 91 / 2607.05315

Socially-Aware Autonomous Doorway Traversal and Payload Delivery for Emergency Assistance

具有社会意识的自主门口穿越与负载投递用于紧急援助

Snowdy, Andrew, Trivedi, Ananya, Prajapati, Sarvesh, Genua, Lorena Maria, Padir, Taskin

Abstract

In this work, we focus on the scenario of a robot-assisted emergency evacuation. We consider two capabilities relevant to such a setting. The first is opening doors ahead of the people being evacuated, so that their path toward an exit stays clear. The second is retrieving rescue equipment and delivering it to the emergency responders carrying out the evacuation. From a systems perspective, this involves several tasks at once. The robot must locate ADA-compliant door buttons and the rescue equipment it needs to retrieve. Additionally, it must remain aware of the people around it and adapt its behavior to them, so that it supports the evacuation rather than getting in the way. We address these demands with a behavior tree at the core of our framework. This structure is chosen for its ability to select high-level tasks based on environmental triggers, and to extend to new situations as they arise. We evaluate the system in 105 trials on the Toyota Human Support Robot, across five hardware and three simulation scenarios. These trials capture the decisions the robot must make in this setting: whether to press a door button, yield to a nearby person, walk through a door someone else is holding, or first retrieve rescue equipment before traversing the door. Overall, the system completes 97 of the 105 trials successfully. These results suggest our framework provides a practical basis for robotic assistance in broader emergency response tasks. Code and video demonstrations are available at https://github.com/AndrewSnowdy/hsr_mm_control.

Chinese Translation

在本研究中，我们关注机器人辅助紧急疏散的场景。我们考虑与此设置相关的两个能力。第一个是为被疏散的人提前打开门，以确保他们通往出口的路径保持畅通。第二个是检索救援设备并将其交付给进行疏散的紧急响应人员。从系统的角度来看，这涉及多个任务的同时进行。机器人必须定位符合美国残疾人法案（ADA）要求的门按钮和需要检索的救援设备。此外，它还必须时刻关注周围的人，并根据他们的行为调整自身的行为，以支持疏散而不是妨碍疏散。我们通过行为树来满足这些需求，该结构被选中是因为它能够根据环境触发器选择高层次任务，并在新情况出现时进行扩展。我们在丰田人类支持机器人（Toyota Human Support Robot）上进行了105次试验，涵盖五种硬件和三种仿真场景。这些试验捕捉了机器人在这一环境中必须做出的决策：是否按下门按钮、让路给附近的人、走过其他人持着的门，或者在穿越门之前先检索救援设备。总体而言，该系统成功完成了105次试验中的97次。这些结果表明，我们的框架为机器人在更广泛的紧急响应任务中的辅助提供了实用基础。代码和视频演示可在 https://github.com/AndrewSnowdy/hsr_mm_control 获得。

View on arXiv Download PDF AI Translation

cs.RO / 92 / 2607.05369

GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks

GaP：一种用于变分自动化任务的图作为策略多智能体自学习框架

Chen, Kaiyuan, Xie, Shuangyu, Fu, Letian, Yu, Justin, Pacini, William, Bajamahal, Sandeep, Kim, Hudson, Drake, Jaimyn, Kim, Daehwa, Xue, Haoru, Francis, Jonathan, Juette, Christian, Schaldenbrand, Peter, Seker, Muhammet Yunus, Wickramarachchi, Ruwan, Yoo, Uksang, Wang, Guanzhi, Murali, Adithyavairavan, Sundaralingam, Balakumar, Sastry, S. Shankar, Huang, Spencer, Zhu, Yuke, Fan, Linxi "Jim", Goldberg, Ken

Abstract

For robots to work reliably in commercial and industrial applications, can recent advances in agentic coding systems combine interpretable robot programming with the open-world adaptability of model-free policies? We focus on "Variational Automation" (VA), a class of tasks that have larger variations in object geometry and pose than fixed automation. Model-free policies often struggle to close the reliability gap for VA tasks, which must be executed persistently and reliably in commercial and industrial applications. Motivated by prior work on Task and Motion Planning (TAMP) and the Robot Operating System (ROS), we introduce Graph-as-Policy (GaP), a multi-agent coding harness that generates directed computation graphs with perception, planning, and control nodes from a Modular Open Robot Skill Library (MORSL). GaP then generates an internal simulation environment to rehearse task instances with different graphs in parallel to iteratively refine the graph structure and parameters to improve success rates and throughput. Evaluation with 8 new open VA task benchmarks, 4 in-simulation and 4 in real-world, suggests that GaP can achieve success rates that significantly outperform baselines. Details, code, and data can be found online: https://graph-robots.github.io/gap

Chinese Translation

为了使机器人在商业和工业应用中可靠地工作，最近在智能编码系统方面的进展能否将可解释的机器人编程与无模型策略的开放世界适应性结合起来？我们关注“变分自动化”（Variational Automation，VA），这是一类在物体几何形状和姿态上变化更大的任务，相较于固定自动化，变分自动化面临更大的挑战。无模型策略往往难以弥补VA任务的可靠性差距，而这些任务必须在商业和工业应用中持续且可靠地执行。受到任务与运动规划（Task and Motion Planning，TAMP）和机器人操作系统（Robot Operating System，ROS）先前工作的启发，我们提出了图作为策略（Graph-as-Policy，GaP），这是一种多智能体编码框架，能够从模块化开放机器人技能库（Modular Open Robot Skill Library，MORSL）生成带有感知、规划和控制节点的有向计算图。GaP随后生成一个内部仿真环境，以并行排练不同图结构的任务实例，迭代优化图结构和参数，以提高成功率和吞吐量。通过8个新的开放VA任务基准进行评估，其中4个在仿真中，4个在现实世界中，结果表明GaP的成功率显著优于基线。详细信息、代码和数据可在线获取：https://graph-robots.github.io/gap

View on arXiv Download PDF AI Translation

cs.RO / 93 / 2607.05377

Cortex: A Bidirectionally Aligned Embodied Agent Framework for Long-horizon Manipulation

Cortex：一种双向对齐的具身智能体框架用于长时间跨度的操作

Peng, Jiaqi, Yu, Xiqian, Feng, Delin, Yang, Yuqiang, Cai, Wenzhe, Xiong, Jing, Yang, Ganlin, Zheng, Jinliang, Cao, Jiafei, Wei, Xueyuan, Pang, Jiangmiao, Shen, Yuan, Wang, Tai

Abstract

While recent Vision-Language-Action (VLA) models show promise toward generalist manipulation policies, they struggle with long-horizon tasks due to their Markovian nature-relying solely on current observations. Hierarchical dual-system methods address this but suffer from a gap between high-level planning semantics and low-level execution kinematics. We introduce Cortex, a bidirectionally aligned embodied agent framework with a customized planning interface that conveys executable and tractable subtask plans from high-level VLM to low-level VLA. Specifically, we standardize manipulation subtasks into 32 canonical skill primitives and inject tractability principles, such as representative object attributes and improved trajectory reachability, into the data generation pipeline. This enables automatic annotation of over 4k hours of open-source video data and generation of 30 hours of simulation data. We further devise an event-balanced sampling strategy to construct training data for fine-tuning the framework to better handle planning ambiguity during subtask transitions, enhanced by carefully designed harness engineering from task contexts to skill constraints during inference. Both open-loop VLM and closed-loop system evaluations demonstrate Cortex's efficacy, e.g., it outperforms monolithic baselines by 3.1% on Libero-long and 4.1% on RoboTwin. Notably, Cortex's generalist VLM enables zero-shot completion of unseen real-world long-horizon tasks, such as multi-stage chemistry experiments, by simply combining with a fine-tuned VLA-a capability infeasible through VLA fine-tuning alone.

Chinese Translation

尽管近期的视觉-语言-动作（VLA）模型在通用操作策略方面展现出潜力，但由于其马尔可夫特性，仅依赖当前观察，它们在长时间跨度任务中表现不佳。层次化双系统方法解决了这一问题，但在高层规划语义与低层执行运动学之间存在差距。我们提出了Cortex，一种双向对齐的具身智能体框架，具有定制的规划接口，能够将可执行且可处理的子任务计划从高层视觉-语言模型（VLM）传递到低层视觉-语言-动作（VLA）。具体而言，我们将操作子任务标准化为32个规范技能原语，并将可处理性原则（如代表性物体属性和改进的轨迹可达性）注入数据生成管道。这使得我们能够自动标注超过4000小时的开源视频数据，并生成30小时的仿真数据。我们进一步设计了一种事件平衡采样策略，以构建训练数据，从而微调框架以更好地处理子任务过渡期间的规划模糊性，并通过在推理过程中从任务上下文到技能约束的精心设计的工具工程进行增强。无论是开放环路VLM还是闭环系统评估均展示了Cortex的有效性，例如，在Libero-long上超越单一基线3.1%，在RoboTwin上超越4.1%。值得注意的是，Cortex的通用VLM使得通过简单结合微调的VLA来实现对未见真实世界长时间跨度任务的零-shot完成成为可能，例如多阶段化学实验，这一能力是通过仅微调VLA无法实现的。

View on arXiv Download PDF AI Translation

cs.RO / 94 / 2607.05390

Deform360: A Massive Multi-view Visuotactile Dataset for Deformable World Models

Deform360：一个用于可变形世界模型的大规模多视角视觉触觉数据集

Li, Hongyu, Fu, Wanjia, Cong, Xiaoyan, Li, Zekun, Huang, Binghao, Jiang, Hanxiao, He, Xintong, Liang, Yiqing, Fu, Rao, Lu, Tao, Sridhar, Srinath, Smith, Kevin A., Konidaris, George, Li, Yunzhu

Abstract

Predicting object dynamics (i.e., world modeling) is a fundamental challenge for robotic manipulation, and modeling deformable objects presents a particularly difficult case due to their high-dimensional state spaces and complex material properties. While current world models approach this through two distinct paradigms: learning the dynamics over the 2D pixel space or more explicit 3D geometric space. A systematic understanding of their relative strengths and limitations remains elusive due to the lack of diverse, large-scale real-world data. To address this, we present Deform360, a large-scale visuotactile dataset featuring 198 daily-life objects, 1,980 interaction sequences, and over 215 hours of observations from 41 surround-view cameras and bimanual tactile grippers to capture both global motion and contact-induced local deformations. Leveraging a novel markerless visuotactile 3D tracking pipeline to extract dense geometry and motion, we systematically evaluate current state-of-the-art world models, comparing 2D video models against 3D particle models. Finally, we provide a preliminary demonstration indicating the real-world applicability of our dataset by performing robot planning tasks on deformable objects. Our analysis reveals key insights into the trade-offs between structural priors and scalability, providing a solid benchmark for future research in generalizable deformable object-centric world modeling. Project website: https://deform360.lhy.xyz

Chinese Translation

预测物体动态（即世界建模）是机器人操作中的一个基本挑战，而建模可变形物体则因其高维状态空间和复杂的材料特性而显得尤为困难。目前的世界模型通过两种不同的范式来解决这一问题：在二维像素空间上学习动态或在更明确的三维几何空间中进行建模。然而，由于缺乏多样化的大规模真实世界数据，对它们相对优缺点的系统理解仍然难以实现。为了解决这一问题，我们提出了Deform360，这是一个大规模的视觉触觉数据集，包含198个日常生活物体、1,980个交互序列，以及来自41个环视摄像头和双手触觉夹持器的超过215小时的观察数据，以捕捉全局运动和接触引起的局部变形。我们利用一种新颖的无标记视觉触觉三维跟踪管道提取密集几何和运动，系统地评估当前最先进的世界模型，将二维视频模型与三维粒子模型进行比较。最后，我们通过在可变形物体上执行机器人规划任务，提供了一个初步演示，表明我们数据集的现实应用性。我们的分析揭示了结构先验与可扩展性之间的权衡关键见解，为未来可推广的以可变形物体为中心的世界建模研究提供了坚实的基准。项目网站：https://deform360.lhy.xyz

View on arXiv Download PDF AI Translation

计算机视觉 (Computer Vision)

300

cs.CV / 1 / 2607.02549

Learning 3D Affordances for Blade Insertion in Cluttered Stowing

在杂乱存储中学习刀片插入的三维可供性

Li, Tianyu, Sawhney, Harpreet, Jung, Minju, Mehrotra, Aditya, Mehrotra, Kunal, Agrawal, Mudit

Abstract

Many manipulation tasks require reasoning about free-space affordances: discovering volumes where an extended rigid tool can safely navigate, complementary to surface contact affordances for grasping. Robotic stowing is a canonical instance, where a blade must sweep items aside inside cluttered fabric bins to create insertion space. Production stow systems generate millions of such episodes, but standard approaches with unimodal data infer affordances as SE(3) pose distributions, a geometric question asked in the wrong domain. VulcanVoxel keeps inference spatial: a masked autoencoder over 3D occupancy fields reconstructs blade occupancy conditioned on scene geometry, computing feasibility locally at each voxel and recovering multi-modal predictions from unimodal data. Blade affordances are spatial objects, subsets of 3D space defined by geometric feasibility. Pose parameters carry no structure for reasoning whether unobserved placements are feasible, and standard generative objectives including flow matching faithfully learn the unimodal distribution produced by execution policies and cannot recover geometric alternatives. Trained on 10,000 real warehouse stow episodes without human annotation, VulcanVoxel achieves top-5 coverage of 0.89 versus 0.71 for the best pose-based baseline, with a distilled student providing RGB-to-voxel inference in 30 ms. vs. 1.4 s. for voxel-to-voxel. We have released a dataset of real blade insertion cycles with RGB-D observations and pose trajectories at https://www.armbench.com/blade_insertion. html.

Chinese Translation

许多操作任务需要推理自由空间可供性：发现扩展刚性工具可以安全导航的体积，这与用于抓取的表面接触可供性互为补充。机器人存储是一个典型实例，其中刀片必须在杂乱的织物箱内清扫物品，以创造插入空间。生产存储系统生成数百万个此类事件，但标准方法使用单模态数据将可供性推断为SE(3)位姿分布，这是一个在错误领域中提出的几何问题。VulcanVoxel保持推断的空间性：一个针对3D占用场的掩蔽自编码器重建刀片占用，条件为场景几何，局部计算每个体素的可行性，并从单模态数据中恢复多模态预测。刀片可供性是空间对象，是通过几何可行性定义的三维空间的子集。位姿参数不携带推理未观察到的放置是否可行的结构，标准生成目标包括流匹配忠实地学习执行策略产生的单模态分布，无法恢复几何替代方案。在没有人工标注的情况下，基于10,000个真实仓库存储事件进行训练，VulcanVoxel实现了0.89的前五名覆盖率，而最佳基于位姿的基线为0.71，经过提炼的学生在30毫秒内提供RGB到体素的推断，而体素到体素的推断则为1.4秒。我们已发布一个包含真实刀片插入周期的RGB-D观察和位姿轨迹的数据集，网址为https://www.armbench.com/blade_insertion.html。

View on arXiv Download PDF AI Translation

cs.CV / 2 / 2607.02551

DELTAVID: Enhancing Fine-Grained Spatiotemporal Perception with Cross-Video Differences

DELTAVID：通过跨视频差异增强细粒度时空感知

Yang, Yankai, Long, Yancheng, Wen, Bin, Yang, Fan, Gao, Tingting, Li, Han, Yang, Shuo

Abstract

Video multimodal large language models have made strong progress on open-ended video understanding, but they still lack precise local spatiotemporal perception. When two videos share almost the same global semantics and differ only in a short time span or a small region, current models often fail to find the change and provide reliable evidence. We propose DELTAVID, a verifiable proxy-task framework that enhances fine-grained spatiotemporal perception with cross-video differences. The key idea is to turn cross-video spot-the-difference into a trainable perception signal, where a model identifies local changes, judges temporal boundaries, and organizes spatial evidence by comparing similar videos. To make this signal scalable to train and reliable to evaluate, we further introduce DELTAVID-10K and DELTAVID-Bench, which convert controllable local differences in real videos into evidence-labeled training and test samples. Experiments show that DELTAVID substantially improves performance on cross-video difference understanding and transfers the learned local evidence ability to general video understanding benchmarks, including MMVU, MLVU, Video-MME, VideoHolmes, VideoMMMU, LVBench, TempCompass, and LongVideoBench. These results show that cross-video differences are not only an effective way to diagnose fine-grained perception failures, but also a scalable proxy supervision that moves Video MLLMs from coarse semantic understanding toward fine-grained spatiotemporal evidence reasoning.

Chinese Translation

视频多模态大型语言模型在开放式视频理解方面取得了显著进展，但仍然缺乏精确的局部时空感知。当两个视频几乎共享相同的全局语义，仅在短时间跨度或小区域上有所不同时，当前模型往往无法发现变化并提供可靠证据。我们提出了DELTAVID，一个可验证的代理任务框架，通过跨视频差异增强细粒度时空感知。其关键思想是将跨视频找不同转化为可训练的感知信号，在此过程中，模型识别局部变化、判断时间边界，并通过比较相似视频组织空间证据。为了使这一信号可扩展以进行训练并可靠地进行评估，我们进一步引入了DELTAVID-10K和DELTAVID-Bench，这些工具将真实视频中的可控局部差异转换为带证据标签的训练和测试样本。实验表明，DELTAVID在跨视频差异理解方面显著提高了性能，并将学习到的局部证据能力迁移到一般视频理解基准上，包括MMVU、MLVU、Video-MME、VideoHolmes、VideoMMMU、LVBench、TempCompass和LongVideoBench。这些结果表明，跨视频差异不仅是诊断细粒度感知失败的有效方法，也是推动视频多模态大型语言模型从粗略语义理解向细粒度时空证据推理的可扩展代理监督。

View on arXiv Download PDF AI Translation

cs.CV / 3 / 2607.02553

Interpretable machine learning predicts Parkinson's disease severity using motion-corrected QSM MRI and multiband multiecho fMRI features

可解释的机器学习通过运动校正的 QSM 磁共振成像和多带多回波功能性磁共振成像特征预测帕金森病严重程度

Andrade, Aixa X.

Abstract

Introduction: Objective neuroimaging biomarkers may improve Parkinson's disease motor assessment by capturing brain variation not directly observable from clinical examination. We used interpretable machine learning to predict current motor severity, measured by MDS-UPDRS Part III, from QSM and multiband multi-echo resting-state fMRI-derived ReHo features. Methods: Regional QSM and ReHo features were extracted from 28 participants, including 24 individuals with Parkinson's disease and 4 controls. Thirteen feature-set experiments evaluated imaging-only, clinical-only, imaging-plus-clinical, full, reduced, and multimodal inputs. Support vector regression, Elastic Net, Random Forest, and XGBoost models were trained using nested cross-validation. Performance was assessed using pooled held-out R^2, RMSE, MAE, Pearson correlation, permutation testing, and the proportion of participants predicted within +/-5 MDS-UPDRS Part III points. Results: Imaging-only models carried meaningful predictive signal, whereas the clinical-only model performed weakly. Full fMRI, full QSM, and clinical variables provided the strongest global fit, explaining 45.4% of variance in motor severity. Selected QSM plus clinical variables produced the most clinically close predictions, with 75.0% of participants predicted within +/-5 points and the lowest MAE among top-performing models. SHAP highlighted cerebellar, thalamic, striatal, insular, and motor cortical features. Conclusion: QSM and multiband multi-echo fMRI-derived ReHo capture distinct, interpretable dimensions of Parkinson's disease motor severity. These findings show that structural and functional imaging contribute differently depending on the clinical prediction goal.

Chinese Translation

引言：客观的神经影像生物标志物可能通过捕捉临床检查中无法直接观察到的大脑变化来改善帕金森病的运动评估。我们使用可解释的机器学习从 QSM 和多带多回波静息态功能性磁共振成像（fMRI）提取的 ReHo 特征中预测当前的运动严重程度，该严重程度通过 MDS-UPDRS 第三部分测量。方法：从 28 名参与者中提取区域 QSM 和 ReHo 特征，其中包括 24 名帕金森病患者和 4 名对照组。进行的十三个特征集实验评估了仅影像、仅临床、影像加临床、完整、简化和多模态输入。使用嵌套交叉验证训练支持向量回归、弹性网络、随机森林和 XGBoost 模型。使用汇总的保留 R^2、均方根误差（RMSE）、平均绝对误差（MAE）、皮尔逊相关性、置换检验以及预测在 +/-5 MDS-UPDRS 第三部分分数范围内的参与者比例来评估性能。结果：仅影像模型具有有意义的预测信号，而仅临床模型表现较弱。完整的 fMRI、完整的 QSM 和临床变量提供了最强的全局拟合，解释了 45.4% 的运动严重度方差。选择的 QSM 加临床变量产生了最接近临床的预测，75.0% 的参与者预测在 +/-5 分数范围内，并且在表现最佳的模型中具有最低的 MAE。SHAP 方法强调了小脑、丘脑、纹状体、岛叶和运动皮层特征。结论：QSM 和多带多回波 fMRI 提取的 ReHo 捕捉了帕金森病运动严重程度的不同可解释维度。这些发现表明，结构和功能影像根据临床预测目标的不同而贡献不同。

View on arXiv Download PDF AI Translation

cs.CV / 4 / 2607.02554

Reliability-Aware Monocular Depth Supervision for Sparse-View Neural Reconstruction

面向稀疏视图神经重建的可靠性感知单目深度监督

Chu, Wei-Teng, Gopalan, Yashasvini, Yuan, Changju

Abstract

Sparse-view neural reconstruction is challenging in outdoor driving scenes, where cameras usually move along a narrow forward-facing trajectory and provide limited multi-view overlap. Although monocular depth estimators can provide dense geometric priors, their predictions are noisy, and not uniformly reliable across image regions. In this work, we study monocular depth supervision for sparse-view neural reconstruction. We use Depth Anything V2 as a dense monocular depth prior, align its predictions to metric depth using scale-shift fitting, and apply depth supervision selectively through photometric masks generated from an RGB-only baseline model. We evaluate this strategy on two representative scene representations: Mip-NeRF-360 and Splatfacto. On KITTISeq02 under an every2 sparse-view setting, masked monocular depth supervision gives only marginal rendering gains for Mip-NeRF-360 and does not improve metric geometry. In contrast, Splatfacto benefits more clearly, improving PSNR from 14.903 to 15.932 and reducing RMSE from 0.542 to 0.100. Additional KITTISeq05 experiments and matched-ratio mask ablations further show that the gains for Splatfacto come from selecting reliable low-error regions rather than simply reducing the number of depth-supervised pixels. Additional experiments on the Bicycle scene show that depth supervision can improve geometry while hurting RGB rendering quality when multi-view coverage is already strong. Overall, our results suggest that monocular depth priors are useful for under-constrained sparse-view reconstruction, but should be applied selectively and with moderate weighting.

Chinese Translation

在户外驾驶场景中，稀疏视图神经重建面临挑战，因为相机通常沿着狭窄的前向轨迹移动，并提供有限的多视图重叠。尽管单目深度估计器可以提供密集的几何先验，但其预测结果噪声较大，并且在图像区域之间的可靠性不均匀。在本研究中，我们探讨了稀疏视图神经重建的单目深度监督。我们使用 Depth Anything V2 作为密集的单目深度先验，通过尺度偏移拟合将其预测结果对齐到度量深度，并通过从仅 RGB 基线模型生成的光度掩膜选择性地应用深度监督。我们在两个代表性的场景表示上评估了这一策略：Mip-NeRF-360 和 Splatfacto。在每2稀疏视图设置下的 KITTISeq02 上，掩膜单目深度监督仅为 Mip-NeRF-360 带来了微小的渲染增益，并未改善度量几何。相反，Splatfacto 明显受益，PSNR 从 14.903 提升至 15.932，RMSE 从 0.542 降低至 0.100。额外的 KITTISeq05 实验和匹配比率掩膜消融进一步表明，Splatfacto 的增益来自于选择可靠的低误差区域，而不仅仅是减少深度监督像素的数量。在自行车场景的额外实验中，深度监督可以改善几何，但在多视图覆盖已经较强时会损害 RGB 渲染质量。总体而言，我们的结果表明，单目深度先验对于欠约束的稀疏视图重建是有用的，但应选择性地应用并适度加权。

View on arXiv Download PDF AI Translation

cs.CV / 5 / 2607.02555

Do Diabetic Foot Ulcer Segmentation Models Generalize? A Cross-Dataset Benchmark of CNN and Transformer Architectures

糖尿病足溃疡分割模型的泛化能力如何？卷积神经网络和变换器架构的跨数据集基准测试

Benfatah, Abderrahmane

Abstract

Deep learning models for diabetic foot ulcer (DFU) segmentation routinely report high accuracy, but they are almost always trained and tested on the same dataset, leaving their behaviour on data from a different clinical source largely unmeasured. We benchmark three representative segmentation architectures -- U-Net and DeepLabV3+ (convolutional) and SegFormer-B2 (Transformer) -- under an identical, leakage-screened protocol: training on the combined FUSeg/AZH wound data and evaluating, without fine-tuning, on two independent external datasets (DFUC2022 and Medetec). All models achieve strong in-domain performance (Dice 0.80--0.83) but degrade substantially across datasets. The degradation is, however, architecture-dependent: SegFormer-B2 generalizes best on both external sets (DFUC2022 Dice 0.557, Medetec Dice 0.786), outperforming both convolutional models, while the more complex DeepLabV3+ generalizes worse than the simpler U-Net. Per-image failure analysis on 2,160 images across both external test sets confirms that SegFormer-B2 produces the fewest catastrophic failures on DFUC2022 (31.1%), compared with U-Net (38.5%) and DeepLabV3+ (43.0%). The consistent ranking across two independent external sources, confirmed by Wilcoxon signed-rank tests (p < 0.001 on both datasets), indicates that architecture family, not model complexity, drives cross-hospital generalization.

Chinese Translation

糖尿病足溃疡（DFU）分割的深度学习模型通常报告高准确率，但几乎总是在同一数据集上进行训练和测试，导致其在不同临床来源数据上的表现几乎未被测量。我们在一个相同的、经过泄漏筛选的协议下，对三种代表性的分割架构进行了基准测试——U-Net和DeepLabV3+（卷积网络）以及SegFormer-B2（变换器）。这些模型在结合的FUSeg/AZH伤口数据上进行训练，并在两个独立的外部数据集（DFUC2022和Medetec）上进行评估，而不进行微调。所有模型在领域内表现出色（Dice 0.80--0.83），但在跨数据集时表现显著下降。然而，这种下降是架构依赖的：SegFormer-B2在两个外部数据集上均表现最佳（DFUC2022 Dice 0.557，Medetec Dice 0.786），超越了两个卷积模型，而更复杂的DeepLabV3+的泛化能力却不如更简单的U-Net。对2,160张图像的逐图失败分析确认，SegFormer-B2在DFUC2022上产生的灾难性失败最少（31.1%），而U-Net为38.5%，DeepLabV3+为43.0%。在两个独立外部来源中一致的排名，通过Wilcoxon符号秩检验（在两个数据集上p < 0.001）得到确认，表明架构家族而非模型复杂性驱动了跨医院的泛化能力。

View on arXiv Download PDF AI Translation

cs.CV / 6 / 2607.02559

How many labels do you need? A decision framework for cross-habitat marine species recognition

你需要多少标签？跨栖息地海洋物种识别的决策框架

Saleh, Alzayat, Azghadi, Mostafa Rahimi

Abstract

Automated image recognition is increasingly used to scale ecological monitoring beyond manual annotation, yet ecologists lack evidence-based guidance on how much labelling effort reliable deployment at new sites requires. We present a decision framework quantifying the trade-off between labelling effort and recognition accuracy when transferring vision systems across marine habitats. The benchmark spans five datasets, three oceans, and three taxonomic groups (fish, corals, invertebrates), from tropical reefs in the Great Barrier Reef and French Polynesia to a temperate Danish fjord. We evaluated four recognition models (DINOv2, CLIP, ResNet-50, EfficientNet-B4) under four adaptation strategies (linear probing, LoRA, Visual Prompt Tuning, full fine-tuning) across three protocols: within-habitat transfer across 20 reef sites (240 runs), cross-dataset geographic transfer along a difficulty gradient (40 runs), and few-shot adaptation curves with 0-100 labelled samples per class (648 runs). Frozen self-supervised foundation features (DINOv2 + linear classifier, 1,538 trainable parameters) generalised to unseen reef sites at least as well as fully fine-tuned convolutional baselines four orders of magnitude larger; they learned species-diagnostic, habitat-invariant representations, whereas baselines encoded habitat-specific shortcuts that fail at new sites. As few as 10-20 labelled images per species sufficed to deploy reliable recognition at a new site, cutting annotation effort by roughly an order of magnitude. Solution. Programmes expanding to new sites can deploy reliable recognition by pairing a frozen, open foundation model (DINOv2) with a simple linear classifier and annotating only 10-20 images per species - roughly 1-4 hours per site. The framework lets programmes budget labelling effort against expected accuracy across sites, ecosystems, and platforms.

Chinese Translation

自动化图像识别越来越多地被用于扩大生态监测的范围，超越手动标注，然而生态学家缺乏基于证据的指导，来了解在新地点可靠部署所需的标注工作量。我们提出了一个决策框架，量化在跨海洋栖息地转移视觉系统时，标注工作量与识别准确性之间的权衡。基准测试涵盖了五个数据集、三个海洋和三个分类群（鱼类、珊瑚、无脊椎动物），从大堡礁和法属波利尼西亚的热带珊瑚礁到丹麦的温带峡湾。我们在四种适应策略（线性探测、LoRA、视觉提示调优、完全微调）下评估了四种识别模型（DINOv2、CLIP、ResNet-50、EfficientNet-B4），并采用了三种协议：在20个珊瑚礁地点进行栖息地内转移（240次实验）、沿难度梯度进行跨数据集地理转移（40次实验），以及每类0-100个标注样本的少样本适应曲线（648次实验）。冻结的自监督基础特征（DINOv2 + 线性分类器，1,538个可训练参数）在未见过的珊瑚礁地点的泛化能力至少与完全微调的卷积基线相当，后者的规模大四个数量级；它们学习了物种特征、栖息地不变的表示，而基线则编码了在新地点失效的栖息地特定捷径。每个物种仅需10-20张标注图像即可在新地点部署可靠的识别，将标注工作量减少了大约一个数量级。解决方案是：扩展到新地点的项目可以通过将冻结的开放基础模型（DINOv2）与简单的线性分类器配对，并仅标注每个物种10-20张图像（大约每个地点1-4小时），来实现可靠的识别。该框架使项目能够根据预期准确性在不同地点、生态系统和平台之间预算标注工作量。

View on arXiv Download PDF AI Translation

cs.CV / 7 / 2607.02560

Inpainting U-Net for seamless pedestrian-level wind prediction across urban morphologies

用于无缝行人级风速预测的修复U-Net模型在城市形态中的应用

Huang, Jingzi, Heaney, Claire E., Li, Tao, Li, Xinzhe, Hughes, Graham O., van Reeuwijk, Maarten

Abstract

Pedestrian-level wind prediction is essential for urban design and wind-comfort assessment, but high-fidelity simulations such as LES remain computationally expensive for rapid evaluation. This study develops a two-stage U-Net framework for efficient prediction of time-averaged pedestrian-level wind speed over realistic urban morphologies. The model is trained and evaluated using the UrbanTALES dataset, which contains realistic city configurations under different approaching wind directions. In the first stage, a baseline U-Net model (M1) predicts wind fields patch-by-patch from normalised building height and fetch information. This formulation allows application to urban domains of arbitrary size, but independent patch inference can introduce discontinuities at patch boundaries. To address this, a second U-Net model (M2) is introduced as an inpainting-based refinement model. M2 uses a larger contextual window containing the initial M1 prediction and local morphology to reduce discontinuities using neighbouring flow information. During full-field inference, M2 is applied iteratively using a Gauss-Seidel scheme until convergence. Results show that M1 captures the main spatial distribution of pedestrian-level wind speed and performs well in low- and moderate-velocity regions, although high-velocity peaks are less accurate. M2 substantially reduces patch-boundary artefacts and improves spatial coherence. Across unseen urban cases, the framework reproduces mean velocity and spatial variability reasonably well, while maximum velocities remain underestimated. Overall, the proposed framework provides an efficient and flexible surrogate model for high-resolution pedestrian-level wind prediction across realistic urban morphologies.

Chinese Translation

行人级风速预测对于城市设计和风舒适度评估至关重要，但高保真模拟（如大涡模拟，LES）在快速评估中仍然计算成本高昂。本研究开发了一种两阶段U-Net框架，用于高效预测现实城市形态下的时间平均行人级风速。该模型使用UrbanTALES数据集进行训练和评估，该数据集包含不同来风方向下的现实城市配置。在第一阶段，基线U-Net模型（M1）根据规范化建筑高度和风向信息逐块预测风场。这种方法允许应用于任意大小的城市区域，但独立的块推断可能在块边界处引入不连续性。为了解决这个问题，引入了第二个U-Net模型（M2），作为基于修复的精细化模型。M2使用包含初始M1预测和局部形态的更大上下文窗口，通过利用邻近流信息来减少不连续性。在全场推断过程中，M2使用高斯-赛德尔（Gauss-Seidel）方案迭代应用，直到收敛。结果表明，M1捕捉了行人级风速的主要空间分布，并在低速和中速区域表现良好，尽管在高速峰值方面的准确性较低。M2显著减少了块边界伪影并改善了空间一致性。在未见过的城市案例中，该框架合理地再现了平均速度和空间变异性，而最大速度仍然被低估。总体而言，所提出的框架为现实城市形态下的高分辨率行人级风速预测提供了一种高效且灵活的替代模型。

View on arXiv Download PDF AI Translation

cs.CV / 8 / 2607.02561

Double-Helix Active Geometry: LiDAR-Anchored Multi-View Depth with Selective Abstention

双螺旋主动几何：基于LiDAR的多视图深度与选择性放弃

Wen, Jinwen

Abstract

Consumer depth sensors such as the LiDAR scanner on recent iPhones provide metric range, but their useful range is short and their returns are sparse. We present DH-Active, a lightweight, training-free geometry back-end that treats the sensor as a metric ruler rather than the sole source of depth. Near-field returns anchor the metric relative pose of two views through PnP; visually trackable samples without a valid depth return are then triangulated under that pose. A parallax/reprojection gate abstains wherever the geometry is ill-conditioned, leaving an explicit hole and a selective score instead of forcing an estimate. The measured core front end, including spiral sampling, sparse back-projection, and hole taxonomy but excluding preprocessing and multi-view recovery, runs at 1.11 ms median latency on CPU (OpenCV using 14 threads), about 38 times faster than a DINOv2-L visual branch on GPU in our timing setup. Across two iPhone captures and the public TUM RGB-D and ARKitScenes benchmarks, held-out depth is recovered at 1.4 to 6.7 percent median relative error. In a controlled ARKitScenes protocol that uses only returns within 2 m to set scale and an independent laser scan as ground truth, DH-Active achieves 64.2 percent scene-median coverage of evaluable far-field candidates at 13.4 percent scene-median relative error; direct triangulation from the device trajectory is not usable. We also report the alternatives that failed in our tests: single-frame defocus, classical focus-stack depth, defocus-LiDAR fusion, point-to-point ICP over a good visual-inertial track, and attention-to-holes resampling. A 1.26 B learned model remains more accurate after oracle scale alignment. The contribution here is narrower: metric sparse depth, explicit abstention, zero learned parameters, and near-millisecond CPU cost.

Chinese Translation

消费级深度传感器，例如近期iPhone上的LiDAR扫描仪，提供了度量范围，但其有效范围较短且返回数据稀疏。我们提出了DH-Active，这是一种轻量级、无需训练的几何后端，将传感器视为度量尺，而非深度的唯一来源。近场返回通过PnP锚定两个视图的度量相对姿态；在该姿态下，视觉可跟踪的样本即使没有有效的深度返回也会被三角测量。视差/重投影门在几何条件不良的地方选择性放弃，留下一个明确的空洞和选择性得分，而不是强制估计。测量的核心前端，包括螺旋采样、稀疏反投影和空洞分类，但不包括预处理和多视图恢复，在CPU（使用14个线程的OpenCV）上以1.11毫秒的中位延迟运行，比我们定时设置中GPU上的DINOv2-L视觉分支快约38倍。在两次iPhone捕获和公共TUM RGB-D及ARKitScenes基准测试中，保留的深度以1.4%到6.7%的中位相对误差被恢复。在一个控制的ARKitScenes协议中，仅使用2米内的返回来设置比例，并将独立的激光扫描作为真值，DH-Active在13.4%的场景中位相对误差下实现了可评估远场候选的64.2%场景中位覆盖率；来自设备轨迹的直接三角测量不可用。我们还报告了在测试中失败的替代方案：单帧散焦、经典焦点堆叠深度、散焦-LiDAR融合、在良好的视觉惯性轨迹上的点对点ICP，以及对空洞的注意重采样。经过oracle尺度对齐后，1.26B学习模型的准确性仍然更高。这里的贡献更为狭窄：度量稀疏深度、明确放弃、零学习参数和近毫秒的CPU成本。

View on arXiv Download PDF AI Translation

cs.CV / 9 / 2607.02562

Entropy-Coded MS-VQ-VAE with Learned Priors for Ultra-Low Bitrate Video Compression

基于学习先验的熵编码多尺度矢量量化变分自编码器用于超低比特率视频压缩

Kotthapalli, Manikanta, Rekabdar, Banafsheh

Abstract

Learned video codecs based on continuous latent representations struggle to operate reliably below 0.1 bits per pixel~(bpp): without a differentiable rate signal, Lagrangian optimisation cannot effectively trade reconstruction quality for bitrate at extreme compression ratios. We demonstrate that discrete latent representations sidestep this limitation entirely. In a vector-quantized~(VQ) codec, the codebook size~$K$ imposes a hard information ceiling of $\log_2 K$ bits per symbol; a learned autoregressive prior then exploits the non-uniform distribution of code usage -- which we show follows a power law -- to push actual bitrates well below this ceiling, without any rate-penalty tuning. Building on the MS-VQ-VAE architecture introduced in~\cite{kotthapalli2026msvqvae}, we sweep $K \in \{128, 256, 512, 1024\}$ under a uniform training protocol to trace four operating points on the rate-distortion~(RD) curve. We identify and resolve a critical training instability: gradient-based VQ collapses catastrophically at $K \leq 512$, whereas EMA-stabilised codebook updates with dead-code restart maintain full utilisation across all configurations. On 500 UCF101 test clips ($64\!\times\!64$, 32~frames), our models operate at 0.043-0.064~bpp -- 3.3-5$\times$ below H.264's practical floor and $5$-$7.6\times$ below H.265's floor at this resolution. Every MS-VQ-VAE configuration outperforms H.265 CRF\,36 on perceptual quality (LPIPS) despite using $5$-$7.6\times$ fewer bits. At $K{=}1024$, the model surpasses H.265 CRF\,36 on LPIPS by a margin of 0.072 absolute while using $5.1\times$ fewer bits. Codebook analysis confirms power-law index distributions and 70-85\% entropy efficiency, establishing the pipeline as a principled learned entropy coder.

Chinese Translation

基于连续潜在表示的学习视频编解码器在低于每像素0.1比特（bpp）时难以可靠运行：由于缺乏可微分的比特率信号，拉格朗日优化无法在极端压缩比下有效地权衡重建质量与比特率。我们证明离散潜在表示完全规避了这一限制。在矢量量化（VQ）编解码器中，码本大小$K$对每个符号施加了$ ext{log}_2 K$比特的硬信息上限；学习的自回归先验则利用了码使用的非均匀分布——我们展示该分布遵循幂律——将实际比特率推低至这一上限之下，而无需进行任何比特率惩罚调优。基于文献 extit{MS-VQ-VAE}架构，我们在统一训练协议下对$K ext{ in } igackslash{128, 256, 512, 1024igackslash}$进行遍历，以追踪四个率失真（RD）曲线上的操作点。我们识别并解决了一个关键的训练不稳定性：基于梯度的VQ在$K ext{小于等于} 512$时会灾难性崩溃，而EMA稳定的码本更新通过死码重启在所有配置中保持完全利用。在500个UCF101测试片段（$64 ext{×}64$, 32帧）上，我们的模型在0.043-0.064 bpp的比特率下运行——比H.264的实际下限低3.3-5倍，比H.265在该分辨率下的下限低5-7.6倍。尽管使用的比特数少5-7.6倍，但每个MS-VQ-VAE配置在感知质量（LPIPS）上均优于H.265 CRF 36。在$K{=}1024$时，该模型在LPIPS上超越H.265 CRF 36，绝对差距为0.072，同时使用的比特数少5.1倍。码本分析确认了幂律指数分布和70-85%的熵效率，确立了该管道作为一个原则性的学习熵编码器。

View on arXiv Download PDF AI Translation

cs.CV / 10 / 2607.02563

Attention Dynamics in Diffusion Models: A Visual Analytics Framework for Human-AI Collaboration

扩散模型中的注意力动态：人机协作的可视分析框架

Xiao, Yiran, Legrady, George

Abstract

Diffusion-based text-to-image models can synthesize complex and highly structured visual content, yet the emergence and evolution of semantic structure remain difficult to interpret. Many existing workflows rely on aggregated attention or scalar summaries that separate temporal change from image-space evidence. To address this gap, we present a visual analytics framework for exploring attention dynamics in diffusion models: the step-indexed evolution of token-level cross-attention maps, their temporal concentration, and their spatial relationships. Our approach enables structured analysis of attention behavior across generation steps by integrating quantitative measures with data-driven stage identification in an interactive workflow. Case studies on a structured 60-prompt Stable-Diffusion-class benchmark illustrate recurring, interpretable patterns within this setting and show how linked temporal and spatial views facilitate the observation and discussion of generative processes, supporting more effective human-AI collaboration.

Chinese Translation

基于扩散的文本到图像模型能够合成复杂且高度结构化的视觉内容，但语义结构的出现和演变仍然难以解释。许多现有工作流程依赖于聚合的注意力或标量摘要，这些摘要将时间变化与图像空间证据分开。为了解决这一问题，我们提出了一种可视分析框架，用于探索扩散模型中的注意力动态：标记级交叉注意力图的逐步索引演变、它们的时间集中度以及它们的空间关系。我们的方法通过将定量度量与数据驱动的阶段识别集成在一个互动工作流程中，使得对生成步骤中注意力行为的结构化分析成为可能。对一个结构化的60提示Stable-Diffusion类基准的案例研究展示了在这一设置中反复出现的可解释模式，并展示了如何通过链接的时间和空间视图促进生成过程的观察和讨论，从而支持更有效的人机协作。

View on arXiv Download PDF AI Translation

cs.CV / 11 / 2607.02564

From Raw Segmentations to Simulation-Ready Cardiac Meshes: An Automated Framework for Anatomical Reconstruction and Virtual Cohort Generation

从原始分割到适用于仿真的心脏网格：一个自动化的解剖重建和虚拟队列生成框架

Fabbri, Francesco, Scarpolini, Martino Andrea, Ciancarella, Paolo, Tudisco, Francesco, Verzicco, Roberto, Ricci, Alessandro, Viola, Francesco

Abstract

Computational models of the human heart are widely used to study electromechanical and fluid-dynamical cardiac function and to support applications such as in silico clinical trials. However, most studies remain limited to single or patient-specific anatomies, restricting the inclusion of population-level variability required for uncertainty quantification. A key challenge is translating medical-image segmentations, which may contain artifacts, mesh defects or disjoint domains, into topologically coherent geometries suitable for multiphysics simulations. In this work, we present a semi-automatic pipeline that converts CT-based segmentations into simulation-ready cardiac meshes within a few minutes while preserving anatomical and topological consistency. Building on modern deep learning segmentation methods, the framework incorporates a template-based registration stage to regularize artifacts and enforce mesh-quality constraints. A Chamfer-distance morphing strategy deforms a high-quality template toward each segmented heart, matching individual chambers while preserving topology. The resulting meshes are watertight, isotopological, and endowed with consistent point-to-point correspondence. The pipeline is validated on 58 healthy cardiac CT scans, including all cardiac chambers and proximal vessel segments. The resulting meshes can be represented in a unified shape space, enabling the construction of a statistical shape model of the heart and major vessels. Principal Component Analysis shows that a low-dimensional latent space efficiently captures population variability, while Gaussian Mixture Modeling enables synthetic anatomy generation. Overall, the proposed framework (released open-source) provides a pathway from raw segmentations to simulation-ready cardiac geometries, enabling anatomically consistent virtual cohorts for large-scale in silico studies.

Chinese Translation

人类心脏的计算模型被广泛用于研究电机械和流体动力学心脏功能，并支持如计算机模拟临床试验等应用。然而，大多数研究仍然局限于单一或特定患者的解剖结构，限制了包含进行不确定性量化所需的人群级变异性的可能性。一个关键挑战是将可能包含伪影、网格缺陷或不连通域的医学图像分割转换为适合多物理场仿真的拓扑一致几何形状。在本研究中，我们提出了一个半自动化的流程，将基于CT的分割在几分钟内转换为适用于仿真的心脏网格，同时保持解剖和拓扑的一致性。该框架基于现代深度学习分割方法，结合了基于模板的配准阶段，以规范伪影并强制执行网格质量约束。Chamfer距离变形策略将高质量模板变形为每个分割的心脏，匹配各个腔室，同时保持拓扑一致。最终生成的网格是密闭的、等拓扑的，并具有一致的点对点对应关系。该流程在58个健康心脏CT扫描上进行了验证，涵盖了所有心脏腔室和近端血管段。生成的网格可以在统一的形状空间中表示，从而能够构建心脏和主要血管的统计形状模型。主成分分析显示，低维潜在空间有效捕捉了人群变异性，而高斯混合建模则实现了合成解剖结构的生成。总体而言，所提出的框架（已发布为开源）提供了从原始分割到适用于仿真的心脏几何形状的路径，使得为大规模计算机模拟研究提供解剖一致的虚拟队列成为可能。

View on arXiv Download PDF AI Translation

cs.CV / 12 / 2607.02565

Coordinate Singularities Break Conformal Coverage for Gaze and Head Pose

坐标奇点破坏了注视和头部姿态的共形覆盖

Jamalifard, Mohammadreza, Lei, Yaxiong, Azizinezhad, Parastoo, Andreu-Perez, Javier

Abstract

Conformal prediction provides distribution-free reliability guarantees for vision systems, but these guarantees depend on how prediction errors are measured in the output space. Many vision tasks produce outputs on curved spaces (e.g. gaze directions on the sphere or 3D head rotations), yet intermediate prediction heads, residuals, uncertainty estimates, or conformal scores are often defined in flat coordinate charts such as yaw-pitch or Euler angles. We show that this scoring choice introduces systematic geometric distortion near coordinate singularities (large pitch angles on the sphere and poses approaching gimbal lock in 3D rotations). Across four datasets (ETH-XGaze, Gaze360, BIWI, AFLW2000-3D), slice-conditional coverage at a nominal 90% target drops by 30-50 percentage points in these regions, falling to 38.9% on ETH-XGaze and 42.0% on Gaze360 at gaze pitch above 70 degrees, and to 57.5% on BIWI and 55.2% on AFLW2000-3D at head pose pitch above 60 degrees near gimbal lock, despite marginal coverage remaining near 90%. We prove that this is structural. Scalar thresholding changes the size of chart-coordinate prediction sets but leaves their distorted axis ratios unchanged. To diagnose this hidden failure mode, we show that a simple geometric quantity, the Riemannian volume density, strongly correlates with where coverage collapse occurs. Finally, we show that coordinate-free geodesic scoring removes this distortion. It requires no retraining and adds negligible computational cost.

Chinese Translation

共形预测为视觉系统提供了无分布的可靠性保证，但这些保证依赖于如何在输出空间中测量预测误差。许多视觉任务在曲面上产生输出（例如，球面上的注视方向或三维头部旋转），然而中间预测头、残差、不确定性估计或共形分数通常在平坦坐标图（如偏航-俯仰或欧拉角）中定义。我们展示了这种评分选择在坐标奇点附近引入了系统性的几何扭曲（球面上的大俯仰角和接近三维旋转中的万向锁的姿态）。在四个数据集（ETH-XGaze、Gaze360、BIWI、AFLW2000-3D）中，名义上90%的目标下的切片条件覆盖在这些区域下降了30-50个百分点，在ETH-XGaze上当注视俯仰角超过70度时降至38.9%，在Gaze360上降至42.0%；在BIWI上，当头部姿态俯仰角超过60度接近万向锁时降至57.5%，在AFLW2000-3D上降至55.2%，尽管边际覆盖仍接近90%。我们证明了这是结构性的。标量阈值改变了图坐标预测集的大小，但保持了其扭曲的轴比不变。为了诊断这种隐藏的失效模式，我们展示了一个简单的几何量，即黎曼体积密度，与覆盖崩溃发生的地方有很强的相关性。最后，我们展示了无坐标测地评分消除了这种扭曲。它不需要重新训练，并且增加的计算成本微不足道。

View on arXiv Download PDF AI Translation

cs.CV / 13 / 2607.02568

MAGE: View-guided Point Cloud Completion with Efficient Modality Alignment and Adaptive Geometry Enhancement

MAGE：基于视图的点云补全方法，具有高效的模态对齐和自适应几何增强

Quan, Weize, Wu, Zhengwei, Wang, Kai, Yan, Dong-Ming

Abstract

View-based point cloud completion aims to recover a complete 3D shape from a partial point cloud, guided by a single-view image. However, existing approaches often suffer from limited performance due to weak modality alignment and limited self-geometry enhancement. To overcome these challenges, we propose a unified geometry-aware framework that integrates efficient modality alignment and adaptive geometry enhancement, mainly to address cross-modal geometric inconsistency of view-guided point cloud completion. Specifically, we propose a geometry-aware modality alignment by integrating a shared self-attention Transformer and cross-modality reconstruction supervision, which aims to bring features of the image and point cloud close to each other in a shared latent space describing the 3D object. To enhance the perception of global shape and local geometric details, we propose an adaptive geometry-aware self-attention module, which simultaneously considers local geometry-aware attention computation and the spatially-variant feature fusion. In addition, we apply a geometry-perceptive anchor refinement module to reorganize the anchor points (representing a local region of the shape) under appropriate supervision, further boosting the completion performance of our method. Extensive experiments on both synthetic and real-world datasets demonstrate that our method achieves superior performance over existing approaches. Our code will be available at https://github.com/weizequan/MAGE.

Chinese Translation

基于视图的点云补全旨在从部分点云中恢复完整的三维形状，受单视图图像的引导。然而，现有方法往往由于模态对齐不足和自几何增强有限而表现不佳。为了解决这些挑战，我们提出了一个统一的几何感知框架，集成了高效的模态对齐和自适应几何增强，主要针对视图引导点云补全中的跨模态几何不一致性。具体而言，我们提出了一种几何感知的模态对齐方法，通过整合共享的自注意力Transformer和跨模态重建监督，旨在将图像和点云的特征在描述三维物体的共享潜在空间中拉近。为了增强对全局形状和局部几何细节的感知，我们提出了一种自适应几何感知自注意力模块，同时考虑局部几何感知注意力计算和空间变化特征融合。此外，我们应用了几何感知锚点优化模块，在适当的监督下重新组织锚点（表示形状的局部区域），进一步提升我们方法的补全性能。在合成和真实世界数据集上的大量实验表明，我们的方法在性能上优于现有方法。我们的代码将发布在 https://github.com/weizequan/MAGE。

View on arXiv Download PDF AI Translation

cs.CV / 14 / 2607.02569

Uncertainty-Aware Last-Layer Adaptation of RETFound for Referable Diabetic Retinopathy Screening Under Dataset Shift

针对数据集偏移的可参考糖尿病视网膜病变筛查的RETFound不确定性感知最后一层适应

Mardhani, Karim

Abstract

This paper presents a safety-centered empirical evaluation of uncertainty-aware last-layer adaptation for referable diabetic retinopathy screening using RETFound, a self-supervised vision-transformer retinal foundation model used here as a frozen feature encoder, and the public APTOS 2019 and DDR diabetic retinopathy fundus image datasets. We compare a cached-feature softmax head, post-hoc temperature scaling, variational Bayesian last-layer heads, a diagonal Laplace last-layer approximation, and an SNGP-style cached-feature head. On APTOS, uncertainty-aware operating points improved sensitivity and selective-referral behavior. The strongest APTOS selective-referral result deferred approximately 20 percent of cases and reduced accepted-case false negatives to zero while preserving high accepted-case specificity. However, threshold tuning also reduced false negatives at high false-positive cost, so false-negative reduction alone was not unique to Bayesian modeling. On DDR, native Bayesian heads qualitatively reproduced the APTOS direction but with weaker tradeoffs, while the APTOS-trained SNGP checkpoint transferred poorly and failed to provide useful external selective-referral behavior. These results highlight the value of safety-centered evaluation beyond aggregate accuracy: uncertainty-aware last-layer heads can improve internal safety-oriented operating points, but trustworthy retinal screening claims require explicit safety-coverage evaluation and second-dataset validation under shift.

Chinese Translation

本文提出了一种以安全为中心的实证评估，针对使用RETFound（一个自监督视觉变换器视网膜基础模型，作为冻结特征编码器）进行可参考糖尿病视网膜病变筛查的不确定性感知最后一层适应，使用公共的APTOS 2019和DDR糖尿病视网膜病变眼底图像数据集。我们比较了缓存特征的softmax头、后处理温度缩放、变分贝叶斯最后一层头、对角拉普拉斯最后一层近似以及SNGP风格的缓存特征头。在APTOS数据集上，不确定性感知的操作点提高了灵敏度和选择性转诊行为。最强的APTOS选择性转诊结果推迟了约20%的病例，并将接受病例的假阴性降低至零，同时保持了高接受病例的特异性。然而，阈值调整在高假阳性成本下也减少了假阴性，因此假阴性减少并不是贝叶斯建模所独有的。在DDR数据集上，原生贝叶斯头在定性上重现了APTOS的方向，但权衡较弱，而APTOS训练的SNGP检查点转移效果不佳，未能提供有用的外部选择性转诊行为。这些结果突显了超越整体准确性的安全中心评估的价值：不确定性感知的最后一层头可以改善内部安全导向的操作点，但可信的视网膜筛查声明需要在数据集偏移下进行明确的安全覆盖评估和第二数据集验证。

View on arXiv Download PDF AI Translation

cs.CV / 15 / 2607.02571

Dual-Adaptive SAM3: Hierarchical Routing over Low-Rank Expert Layers for Parameter-Efficient Medical Image Segmentation

双自适应SAM3：基于低秩专家层的分层路由用于参数高效的医学图像分割

Chen, Ying, Li, Jinyue, Wang, Kun, Li, Qiankun, Liu, Yang

Abstract

The Segment Anything Model with Concepts (SAM3) heralds a new paradigm for open-vocabulary segmentation through natural language interaction, offering significant potential for medical image analysis. However, effectively adapting such a powerful vision-language model to the diverse and nuanced domain of medical imaging remains a key challenge. Naive fine-tuning is parameter-inefficient, while standard Mixture-of-Experts (MoE) methods introduce prohibitive computational overhead, limiting their clinical applicability. To address this, we propose Dual-Adaptive SAM3 (DA-SAM3), a novel framework that achieves both high segmentation accuracy and extreme parameter efficiency via a dual-adaptive specialization mechanism. Our first adaptation is task-aware: a Dynamic Expert Router (DER) that sparsely activates the most relevant experts by jointly reasoning about the visual input and the textual concept prompt, mimicking a clinical consultation process. Our second adaptation is parameter-aware: a Decomposed Parameterized Experts (DPE) design that represents each expert as a shared frozen base (inherited from the pretrained SAM3) and a lightweight trainable low-rank delta, reducing MoE parameter overhead by over 80\%. Extensive experiments on multiple public medical segmentation benchmarks demonstrate that Dual-Adaptive SAM3 not only matches or exceeds the accuracy of fully fine-tuned SAM3 and standard MoE baselines, but also achieves a notable 5\% gain over current state-of-the-art methods, with interpretable results validating its effectiveness. The code is available at: https://github.com/Reconsider80/DA-SAM3.

Chinese Translation

带有概念的任何分割模型（Segment Anything Model with Concepts, SAM3）为通过自然语言交互进行开放词汇分割开辟了新范式，为医学图像分析提供了显著潜力。然而，如何有效地将这样一个强大的视觉-语言模型适应于多样化且细致的医学成像领域仍然是一个关键挑战。简单的微调在参数效率上表现不佳，而标准的专家混合（Mixture-of-Experts, MoE）方法则引入了过高的计算开销，限制了其临床应用。为了解决这个问题，我们提出了双自适应SAM3（Dual-Adaptive SAM3, DA-SAM3），这是一个新颖的框架，通过双自适应专业化机制实现了高分割准确性和极高的参数效率。我们的第一个适应是任务感知的：一个动态专家路由器（Dynamic Expert Router, DER），通过共同推理视觉输入和文本概念提示，稀疏激活最相关的专家，模拟临床咨询过程。我们的第二个适应是参数感知的：一个分解参数化专家（Decomposed Parameterized Experts, DPE）设计，将每个专家表示为一个共享的冻结基础（继承自预训练的SAM3）和一个轻量级可训练的低秩增量，从而将MoE参数开销减少超过80%。在多个公共医学分割基准上的广泛实验表明，双自适应SAM3不仅在准确性上与完全微调的SAM3和标准MoE基线相匹配或超越，而且在当前最先进的方法上实现了显著的5%的增益，其可解释的结果验证了其有效性。代码可在以下网址获取：https://github.com/Reconsider80/DA-SAM3。

View on arXiv Download PDF AI Translation

cs.CV / 16 / 2607.02572

Additive Causal Construction for Transferable and Reconfigurable Cross-System Learning in Multi-Source Image Fusion

用于可转移和可重构跨系统学习的加性因果构建在多源图像融合中的应用

Fu, Zhizhong, Zhou, Wei, Jiang, Zhaoyang, Lin, Yulong, Hou, Yifu, Ding, Xiaorong, Yan, Qiang, Chen, Yifan

Abstract

In multi-source image fusion scenarios, heterogeneous inputs are typically driven by distinct generative mechanisms and can be viewed as a composition of multiple causal systems. However, cross-system discrepancy (CSD) and cross-system entanglement (CSE) commonly arise during the fusion process, often leading to significant performance degradation under out-of-distribution (OOD) predictions. To address the CSD and CSE issues, we propose the additive causal construction (ACC) framework, which characterizes information fusion at two levels: firstly, it establishes causal "anchors" shared among multiple systems through intervention consistency to enable causal graph transferability (CGT); and secondly, it formalizes the fusion process as causal construction and models the reliability of constructed paths through uncertainty quantification to ensure causal graph reconfigurability (CGR). Building upon this, we revisit the traditional causal representation learning (CRL) with ACC and propose ACC-CRL as a learnable instantiation of the framework. The method explores joint causal content representations across systems via content-mechanism decoupling, and performs response alignment under shared anchors to mitigate CSD. Furthermore, it incorporates structural uncertainty to adaptively regulate the fusion process, thereby suppressing unstable CSE. We conduct systematic experiments on synthetic data (ColorMNIST) and real-world multi-center medical imaging tasks (microvascular invasion (MVI) prediction). The results demonstrate that the proposed method significantly improves OOD generalization while maintaining in-distribution (ID) performance, validating the effectiveness and robustness of the ACC-CRL strategy based on mechanism alignment and uncertainty modeling in open environments.

Chinese Translation

在多源图像融合场景中，异构输入通常由不同的生成机制驱动，可以视为多个因果系统的组合。然而，在融合过程中，跨系统差异（CSD）和跨系统纠缠（CSE）常常出现，这通常导致在分布外（OOD）预测下显著的性能下降。为了解决CSD和CSE问题，我们提出了加性因果构建（ACC）框架，该框架在两个层面上表征信息融合：首先，通过干预一致性建立多个系统共享的因果“锚”，以实现因果图的可转移性（CGT）；其次，将融合过程形式化为因果构建，并通过不确定性量化对构建路径的可靠性进行建模，以确保因果图的可重构性（CGR）。在此基础上，我们重新审视传统的因果表示学习（CRL），并提出ACC-CRL作为该框架的可学习实例。该方法通过内容-机制解耦探索跨系统的联合因果内容表示，并在共享锚下进行响应对齐，以减轻CSD。此外，它结合结构不确定性以自适应地调节融合过程，从而抑制不稳定的CSE。我们在合成数据（ColorMNIST）和真实世界的多中心医学成像任务（微血管侵袭（MVI）预测）上进行了系统实验。结果表明，所提出的方法在保持分布内（ID）性能的同时显著提高了OOD泛化能力，验证了基于机制对齐和不确定性建模的ACC-CRL策略在开放环境中的有效性和鲁棒性。

View on arXiv Download PDF AI Translation

cs.CV / 17 / 2607.02573

Symmetry-Structured Neural Completion of Islamic Geometric Patterns from Sparse Control Geometry

基于对称结构的神经网络补全伊斯兰几何图案的稀疏控制几何

Ugail, Hassan, Mehmood, Irfan

Abstract

Islamic geometric patterns are governed by exact rotational symmetry and strict construction rules. This paper treats these rules as formal geometric knowledge and embeds them in a neural completion framework, rather than leaving them to be learned statistically from data. Given sparse control geometry and a target symmetry order, the system completes the pattern as a vector graph by predicting edges and refinements of bounded curves over a candidate lattice whose edges are organised into rotational orbits under the cyclic group. Symmetry is enforced either by constraining predictions within these orbits or by projecting them onto them during inference. The orbit-tied variant provides a constructive guarantee: for any input and any orbit-level selection rule, it produces exact N-fold symmetry, preserves anchor points, and keeps all refinements within prescribed bounds. These properties are verified numerically. The study focuses on rotational symmetry, and all quantitative results are obtained from procedurally generated graphs inspired by Islamic geometric design rather than from a historical corpus. On clean inputs, enforcing exact validity produces no measurable loss in fidelity. When control geometry is missing, an unstructured decoder loses fidelity and breaks symmetry; retraining on corrupted inputs recovers much of the fidelity but not exact validity. Symmetry-structured inference, by contrast, keeps violations at zero throughout. The results show that augmentation and symmetry structure address distinct failure modes: augmentation improves fidelity under corruption, while symmetry structure guarantees validity. The framework therefore provides a knowledge-constrained, guarantee-backed approach to neural completion for scalable vector ornaments whose validity depends on exact geometric structure.

Chinese Translation

伊斯兰几何图案受精确的旋转对称性和严格的构造规则支配。本文将这些规则视为形式几何知识，并将其嵌入神经补全框架中，而不是依赖于从数据中统计学习。给定稀疏的控制几何和目标对称阶，系统通过预测边缘和有界曲线的细化，将图案作为向量图完成，这些边缘在循环群下组织成旋转轨道。通过在推理过程中将预测限制在这些轨道内或将其投影到轨道上来强制执行对称性。与轨道绑定的变体提供了构造性保证：对于任何输入和任何轨道级选择规则，它产生精确的 N 倍对称性，保留锚点，并保持所有细化在规定的范围内。这些属性通过数值验证。研究集中于旋转对称性，所有定量结果均来自于受伊斯兰几何设计启发的程序生成图，而非历史语料库。在干净的输入下，强制执行精确有效性不会导致可测量的保真度损失。当控制几何缺失时，无结构解码器会失去保真度并破坏对称性；在受损输入上重新训练可以恢复大部分保真度，但无法实现精确有效性。相比之下，对称结构推理在整个过程中保持违规为零。结果表明，增强和对称结构解决了不同的失效模式：增强在腐败情况下提高保真度，而对称结构保证有效性。因此，该框架提供了一种知识约束、保证支持的神经补全方法，适用于其有效性依赖于精确几何结构的可扩展向量装饰。

View on arXiv Download PDF AI Translation

cs.CV / 18 / 2607.02575

Criterion-Conditional In-Context Learning: Evaluating Criterion-Shift Adaptation in Vision-Language Models

标准条件下的上下文学习：评估视觉-语言模型中的标准转变适应性

Yang, Kaiyun, Yang, Ruilin, Yao, Zhimin, Wang, J., Ge, Wei

Abstract

Vision-language models can perform new tasks without parameter updates through in-context learning (ICL), whose core mechanism is utilizing the support set for task induction. In the standard ICL setting, once the task is induced, its decision criterion remains fixed. However, in real-world applications, many tasks exhibit a stable high-level intent, while their decision criteria shift according to specific requirements. Thus, we introduce a new setting, denoted as Criterion-Conditional In-Context Learning (CC-ICL), where models must infer the latent criterion from context and adjust predictions accordingly under fixed task semantics. To evaluate this capability, we propose two complementary metrics, Criterion Invariance and Criterion Sensitivity, capturing the model's robustness and adaptability under criterion shifts. We further construct CC-Bench, a multi-domain benchmark that supports evaluation under the CC-ICL setting. By employing a dual-level data hierarchy, CC-Bench enables legitimate ground-truth variation conditioned on the active criterion even when the task remains fixed. Experiments on CC-Bench reveal that most models exhibit a rigid boundary bias, struggling to align their decisions with the latent criterion. We also find that even a simple multi-criterion training strategy can significantly reduce this bias, improving Criterion Sensitivity and enabling 7B-scale models to surpass proprietary models without degrading general multimodal performance.

Chinese Translation

视觉-语言模型能够通过上下文学习（In-Context Learning, ICL）在不更新参数的情况下执行新任务，其核心机制是利用支持集进行任务诱导。在标准的ICL设置中，一旦任务被诱导，其决策标准保持不变。然而，在现实应用中，许多任务表现出稳定的高层意图，而其决策标准会根据特定需求发生变化。因此，我们引入了一种新的设置，称为标准条件上下文学习（Criterion-Conditional In-Context Learning, CC-ICL），在该设置中，模型必须从上下文中推断潜在标准，并在固定任务语义下相应调整预测。为了评估这一能力，我们提出了两个互补的指标：标准不变性（Criterion Invariance）和标准敏感性（Criterion Sensitivity），以捕捉模型在标准转变下的鲁棒性和适应性。我们进一步构建了CC-Bench，一个支持在CC-ICL设置下评估的多领域基准。通过采用双层数据层次结构，CC-Bench能够在任务保持固定的情况下，基于活动标准支持合法的真实变化。在CC-Bench上的实验表明，大多数模型表现出刚性的边界偏差，难以将其决策与潜在标准对齐。我们还发现，即使是简单的多标准训练策略也能显著减少这种偏差，提高标准敏感性，使得7B规模的模型在不降低通用多模态性能的情况下超越专有模型。

View on arXiv Download PDF AI Translation

cs.CV / 19 / 2607.02580

Classroom Behavior Monitoring with YOLO An Empirical Study in Higher Education Settings

基于YOLO的课堂行为监测：高等教育环境中的实证研究

Trong, Sinh Vu, Manh, Dung Nguyen, Minh, Hieu Hoang, Trung, Hieu Pham, Ha, Thu Pham, Hoang, Nhu Le

Abstract

Classroom behavior monitoring plays a vital role in evaluating student engagement and improving teaching effectiveness. Traditional observation methods remain subjective and lack scalability. This study introduces a real-world dataset of classroom videos collected at the Banking Academy of Vietnam (BAV-Classroom dataset), annotated with nine distinctive behavioral categories. State-of-the-art Computer Vision models were evaluated and compared, with YOLOv11 achieving the best performance. Experimental results indicate that students' concentration often decreases notably during the final part of lectures, highlighting challenges in sustaining engagement. Our findings demonstrate the feasibility of applying computer vision for automated classroom monitoring, providing valuable insights for academic quality management.

Chinese Translation

课堂行为监测在评估学生参与度和提升教学效果方面发挥着至关重要的作用。传统的观察方法往往主观且缺乏可扩展性。本研究引入了一个真实世界的课堂视频数据集，该数据集在越南银行学院（BAV-Classroom数据集）收集，并标注了九种独特的行为类别。我们评估并比较了最先进的计算机视觉模型，其中YOLOv11表现最佳。实验结果表明，学生的注意力在讲座的最后部分通常显著下降，这突显了维持参与度的挑战。我们的研究结果展示了应用计算机视觉进行自动化课堂监测的可行性，为学术质量管理提供了宝贵的见解。

View on arXiv Download PDF AI Translation

cs.CV / 20 / 2607.02582

Evaluating Intellectual Property Guardrails of Generative Image Models: A Technical Report

评估生成图像模型的知识产权保护措施：技术报告

Hoag, Austin T., Modas, Apostolos, Ba, Yunhao, LaChance, Julienne M., Xue, Jinru, Hutiri, Wiebke, Simson, Jan, Georgievski, Tiffany, Towli, Alex, Smith, Joseph, Mitsufuji, Yuki, Xiang, Alice

Abstract

Generative image models are capable of producing images that bear a strong resemblance to, or replicate, recognizable intellectual property (IP). In this technical report, we present a benchmark and automated evaluation pipeline to test for evidence of IP guardrails in generative image models along with the propensity for these models to generate images with recognizable IP. The IP categories we tested include fictional characters, celebrity likeness, and commercial logos and do not encompass the full range of IP which may be implicated by image generation models. We evaluated fourteen widely used text-to-image models, including three self-hosted open weights models and eleven private models. While all of the private models were observed to refuse generations at some level due to IP guardrails, the frequency of generation refusals varied substantially among models. The refusal rates also varied considerably across the different IP categories tested. Commercial logos were refused least frequently and were successfully generated at the highest rate, on average. Though the rate varies, all models tested readily generated images containing recognizable IP as of March 2026.

Chinese Translation

生成图像模型能够生成与可识别的知识产权（IP）高度相似或复制的图像。在本技术报告中，我们提出了一个基准和自动评估流程，以测试生成图像模型中知识产权保护措施的证据，以及这些模型生成可识别知识产权图像的倾向。我们测试的知识产权类别包括虚构角色、名人肖像和商业标志，但并未涵盖所有可能受到图像生成模型影响的知识产权范围。我们评估了十四个广泛使用的文本到图像模型，包括三个自托管的开放权重模型和十一种私有模型。尽管所有私有模型在某种程度上因知识产权保护措施拒绝生成图像，但不同模型之间的拒绝生成频率差异显著。不同知识产权类别的拒绝率也存在显著差异。商业标志的拒绝频率最低，平均成功生成的比例最高。尽管拒绝率有所不同，但截至2026年3月，所有测试模型都能轻松生成包含可识别知识产权的图像。

View on arXiv Download PDF AI Translation

cs.CV / 21 / 2607.02584

RotateAttention: RoPE-Aware Rotation and Range Rectification for INT4 Quantized Attention in Video Generation

RotateAttention：针对视频生成中INT4量化注意力的RoPE感知旋转与范围校正

Liu, Yaofu, Lan, Wanli, Li, Jinxi, Yuan, Binhang, Yang, Harry

Abstract

In \textbf{DiT-based video generation models equipped with 3D Rotary Position Embeddings (3D RoPE)}, the attention mechanism remains a primary computational bottleneck due to its quadratic complexity with respect to sequence length. While quantized \textbf{FlashAttention} offers a promising path toward hardware acceleration, existing low-bit quantization methods overlook two critical challenges in this setting: \textbf{1)} applying online rotation matrices -- a widely used technique for mitigating outliers in Queries ($Q$) and Keys ($K$) -- is difficult to reconcile with \textbf{RoPE}; and \textbf{2)} the non-negative attention matrix $P = \exp(QK - \max(QK))$ makes symmetric quantization waste half of the 4-bit dynamic range. In this work, we observe that the outlier distributions of $Q$ and $K$ are strongly affected by the dimensional partitioning of \textbf{3D RoPE}. Based on this finding, we propose \textbf{RotateAttention}, an efficient \textbf{mixed-precision INT4 FlashAttention} framework tailored for \textbf{DiT-based video generation models with 3D RoPE}, using selective \textbf{FP16 fallback} for accuracy-sensitive attention blocks and denoising steps. RotateAttention introduces two core techniques: \textbf{1) RoPE-aware Rotation}, which employs either mergeable rotation matrices that can be fused into RoPE or negligible-overhead matrices to mitigate RoPE-induced outliers in $Q$ and $K$; and \textbf{2) Range-optimized $P$ Quantization}, which uses fixed scales and zero-points to fully exploit the \textbf{INT4 numerical range} with minimal computational overhead. Experiments show that \textbf{RotateAttention} preserves video generation quality nearly identical to full-precision baselines while achieving up to 1.68$\times$ end-to-end speedup and 2.2$\times$ kernel-level acceleration.

Chinese Translation

在配备3D旋转位置嵌入（3D RoPE）的基于DiT的视频生成模型中，注意力机制由于与序列长度的平方复杂度相关，仍然是主要的计算瓶颈。虽然量化的FlashAttention为硬件加速提供了有希望的路径，但现有的低比特量化方法忽视了这一设置中的两个关键挑战： extbf{1)} 应用在线旋转矩阵——一种广泛用于减轻查询（$Q$）和键（$K$）中的异常值的技术——难以与RoPE协调； extbf{2)} 非负注意力矩阵$P = ext{exp}(QK - ext{max}(QK))$使得对称量化浪费了4位动态范围的一半。在本研究中，我们观察到$Q$和$K$的异常值分布受到3D RoPE的维度划分的强烈影响。基于这一发现，我们提出了 extbf{RotateAttention}，这是一个高效的 extbf{混合精度INT4 FlashAttention}框架，专为 extbf{基于DiT的3D RoPE视频生成模型}量身定制，使用选择性 extbf{FP16回退}来处理对精度敏感的注意力块和去噪步骤。RotateAttention引入了两个核心技术： extbf{1) RoPE感知旋转}，它采用可与RoPE融合的可合并旋转矩阵或开销可忽略的矩阵，以减轻$Q$和$K$中因RoPE引起的异常值； extbf{2) 范围优化的$P$量化}，它使用固定的比例和零点，以最小的计算开销充分利用 extbf{INT4数值范围}。实验表明， extbf{RotateAttention}在视频生成质量上几乎与全精度基线相同，同时实现了高达1.68$ imes$的端到端加速和2.2$ imes$的内核级加速。

View on arXiv Download PDF AI Translation

cs.CV / 22 / 2607.02585

Reliability-Aware CT-MRI Registration: A Quality Engineering Framework with Stability Analysis and Risk Classification

可靠性意识的CT-MRI配准：具有稳定性分析和风险分类的质量工程框架

Albzour, Nisreen

Abstract

Multimodal CT-MRI registration is central to image-guided radiotherapy, surgical navigation, and diagnostic workflows, but most pipelines report only aggregate quality metrics without per-case reliability signals. We propose a reliability-aware framework that converts registration quality into Green/Yellow/Red risk categories using data-learned thresholds. CT images were registered to T1-weighted MRI using rigid and affine transformations on 90 paired slices from 18 patients across brain, abdominal, and neck anatomies. Reliability was assessed using Delta NMI, Delta SSIM, Dice overlap, registration stability, and inverse consistency error, combined into a single score R. Thresholds learned from training patients were applied unchanged to held-out test patients. Affine registration outperformed rigid registration on NMI and SSIM, yielding 44% Green classifications versus 33% for rigid. Reliability-filtered registrations improved the average alignment profile compared with unfiltered methods. Per-anatomy analysis showed substantial variation, with stronger reliability for abdominal registrations than brain registrations. Weight sensitivity analysis identified Dice overlap as the dominant reliability component. The proposed framework provides an interpretable quality-control layer for multimodal registration, while risk thresholds reflect statistical rather than clinical validation.

Chinese Translation

多模态CT-MRI配准在图像引导放射治疗、外科导航和诊断工作流程中至关重要，但大多数流程仅报告汇总质量指标，而没有每个案例的可靠性信号。我们提出了一种可靠性意识框架，该框架使用数据学习的阈值将配准质量转换为绿色/黄色/红色风险类别。CT图像通过刚性和仿射变换与18名患者的90对切片中的T1加权MRI进行配准，涉及脑部、腹部和颈部解剖。使用Delta NMI、Delta SSIM、Dice重叠、配准稳定性和逆一致性误差评估可靠性，并将其组合为单一得分R。从训练患者中学习的阈值未作更改地应用于保留的测试患者。仿射配准在NMI和SSIM上优于刚性配准，绿色分类的比例为44%，而刚性配准为33%。与未过滤方法相比，经过可靠性过滤的配准改善了平均对齐轮廓。按解剖部位分析显示，可靠性存在显著差异，腹部配准的可靠性强于脑部配准。权重敏感性分析确定Dice重叠是主要的可靠性组成部分。所提出的框架为多模态配准提供了可解释的质量控制层，而风险阈值反映的是统计而非临床验证。

View on arXiv Download PDF AI Translation

cs.CV / 23 / 2607.02588

Homer: Understanding Long-form Videos with Hierarchical Memory and Agentic Reasoning

Homer：利用层次记忆和自主推理理解长视频

Ji, Yixin, Ye, Fanghua, Li, Juntao, Zhao, Bo, Qiu, Zexuan, Tu, Zhaopeng, Bo, Liefeng, Zhang, Min

Abstract

Multimodal large language models excel on short clips but struggle on hour-long videos in an online setting, where frames are processed incrementally under limited memory. Existing online methods either retain compact visual representations that lack semantic structure, or build higher-level memory stores organized around temporal proximity rather than explicit causal links, leaving multi-hop narrative reasoning to be reconstructed by the LLM at every query. We bridge this gap with \textsc{Homer}, a Hierarchical Online Memory Exploration and Reasoning framework. \textsc{Homer}'s memory mirrors the multi-scale structure of long videos, ranging from raw perception, to recurring entities, to events connected by explicit temporal and causal relations. Its agentic reasoner then explores this memory the way humans do, locating the relevant scene, looking up details, and composing the answer through multi-round memory retrieval, with a harness that verifies and corrects each step. \textsc{Homer} outperforms the previous best agent method by $+5.5$, $+10.8$, and $+4.4$ points on M3-Bench-robot, M3-Bench-web, and Video-MME-Long, and consistently lifts three various LLM backbones, indicating a model-agnostic structural capability for grounded retrieval over long videos.

Chinese Translation

多模态大型语言模型在短视频片段上表现出色，但在在线环境中处理长达一小时的视频时却面临挑战，因为帧是以增量方式在有限内存下处理的。现有的在线方法要么保留缺乏语义结构的紧凑视觉表示，要么构建围绕时间接近性而非明确因果关系组织的高层记忆存储，从而使多跳叙事推理在每次查询时都需由大型语言模型（LLM）重建。我们通过 extsc{Homer}架构填补了这一空白， extsc{Homer}是一个层次在线记忆探索与推理框架。 extsc{Homer}的记忆反映了长视频的多尺度结构，从原始感知到重复实体，再到通过明确的时间和因果关系连接的事件。其自主推理器以人类的方式探索这一记忆，定位相关场景，查找细节，并通过多轮记忆检索组成答案，同时配备一个验证和纠正每一步的机制。 extsc{Homer}在M3-Bench-robot、M3-Bench-web和Video-MME-Long上分别比之前最佳的代理方法提高了5.5、10.8和4.4个百分点，并且持续提升了三种不同的LLM基础模型，表明其在长视频上进行基础检索的模型无关结构能力。

View on arXiv Download PDF AI Translation

cs.CV / 24 / 2607.02591

CPR: Chained Perceptual Refinement for Coarse-to-Fine Medical Image Classification

CPR：用于粗到细医学图像分类的链式感知精炼

Lu, Si-Yuan, Zhu, Hanruo, Zhu, Ziquan, Jin, Gaojie, Fu, Zeyu, Yin, Lu, Li, Ke, Liu, Lu, Huang, Tianjin

Abstract

High resolution medical images contain fine grained, spatially sparse cues that are critical for diagnosis, yet preserving full resolution incurs substantial computational and memory costs. Most deep models process images uniformly, leading to redundant computation or loss of diagnostic detail under downsampling. We propose Chained Perceptual Refinement, CPR, a coarse to fine framework that formulates medical image analysis as a sequential global to local decision process. Starting from a low resolution global view, CPR dynamically predicts the location and spatial extent of refinement regions, extracts high resolution evidence from the original image, and incrementally integrates it with global context. By keeping the backbone input size fixed while contracting the perceptual field, CPR preserves diagnostic fidelity with constant peak GPU memory. Extensive experiments on five medical imaging datasets and multiple backbone architectures demonstrate that CPR consistently outperforms both fixed resolution and multi scale state of the art baselines, achieving improvements of up to 2.27 percentage points over the second best method. It also achieves up to a 19.6 fold reduction in GFLOPs at matched accuracy, establishing a superior accuracy and efficiency trade off for high resolution medical image analysis. The code is available on GitHub.

Chinese Translation

高分辨率医学图像包含细粒度的、空间稀疏的线索，这些线索对诊断至关重要，但保持全分辨率会带来巨大的计算和内存成本。大多数深度模型对图像进行统一处理，导致冗余计算或在下采样时丧失诊断细节。我们提出了链式感知精炼（Chained Perceptual Refinement，CPR），这是一种将医学图像分析形式化为从全局到局部的顺序决策过程的粗到细框架。CPR 从低分辨率的全局视图开始，动态预测精炼区域的位置和空间范围，从原始图像中提取高分辨率证据，并将其与全局上下文逐步整合。通过保持主干输入大小不变，同时收缩感知场，CPR 在保持诊断保真度的同时，确保了恒定的峰值 GPU 内存。对五个医学成像数据集和多种主干架构的广泛实验表明，CPR 在性能上始终优于固定分辨率和多尺度的最新基线方法，较第二佳方法提高了多达 2.27 个百分点。同时，在匹配精度的情况下，CPR 实现了高达 19.6 倍的 GFLOPs 减少，为高分辨率医学图像分析建立了更优的准确性与效率的权衡。代码可在 GitHub 上获取。

View on arXiv Download PDF AI Translation

cs.CV / 25 / 2607.02592

H-OPD: Confidence Aware Heterogeneous Multi-Teacher Multimodal On-policy Distillation

H-OPD：基于信心的异构多教师多模态在线蒸馏

Yin, Qixiang, Yao, Huanjin, Cai, Yuchen, Chen, Jianghao, Wang, Ziyi, Yang, Min, Su, Fei, Zhao, Zhicheng

Abstract

On-policy distillation (OPD) has recently emerged as an effective post-training paradigm by providing supervision on student-generated trajectories. However, existing OPD methods for multimodal reasoning usually rely on a static teacher routing, assigning each sample to a single teacher based on modality or task type. This ignores that visual grounding and abstract reasoning may dominate different decoding steps, making a single teacher insufficient for the full trajectory. To this end, H-OPD is proposed as a confidence-aware heterogeneous multi-teacher OPD framework for multimodal reasoning. By verifying the complementarity of heterogeneous teachers in the same reasoning process, H-OPD replaces task or sample level teacher routing with token-level teacher arbitration along the shared student trajectory. H-OPD employs vision-to-language description transfer to enable text-only teachers to access key visual semantics, and uses a confidence-aware arbitration mechanism to dynamically combine vision-language teacher and text-only teachers at each token. Extensive evaluations over 11 widely-used reasoning benchmarks showcase the superior performance of our method.

Chinese Translation

在线蒸馏（OPD）最近作为一种有效的后训练范式出现，通过对学生生成的轨迹提供监督。然而，现有的多模态推理的 OPD 方法通常依赖于静态教师路由，根据模态或任务类型将每个样本分配给单一教师。这忽视了视觉定位和抽象推理可能主导不同解码步骤，使得单一教师无法覆盖完整轨迹。为此，提出了 H-OPD，作为一种基于信心的异构多教师 OPD 框架，用于多模态推理。通过验证异构教师在同一推理过程中的互补性，H-OPD 用令牌级教师仲裁替代了任务或样本级教师路由，沿着共享的学生轨迹进行调整。H-OPD 采用视觉到语言的描述转移，使得仅文本教师能够访问关键的视觉语义，并使用基于信心的仲裁机制在每个令牌处动态结合视觉-语言教师和仅文本教师。对 11 个广泛使用的推理基准的广泛评估展示了我们方法的优越性能。

View on arXiv Download PDF AI Translation

cs.CV / 26 / 2607.02593

Token-level Response-visual Attention Guidance for Multimodal LLMs Knowledge Distillation

基于令牌级响应-视觉注意力引导的多模态大语言模型知识蒸馏

Jang, Jaehyun, Yoon, Eunseop, Yoon, Hee Suk, Eom, SooHwan, Hasegawa-Johnson, Mark A., Yoo, Chang D.

Abstract

While knowledge distillation (KD) is widely adopted for training lightweight models by leveraging supervision from larger teacher models, relying solely on output token distributions has proven insufficient for compressing Multimodal Large Language Models (MLLMs). Since output tokens are a byproduct of the model attending to visual inputs, prior works have explored explicitly distilling attention to provide a direct supervisory signal. While promising, the precise utility of which attention signals to distill remains under-explored. In this work, we challenge the conventional reliance on prompt-to-vision attention by revealing that downstream performance correlates strongly with response-to-vision attention similarity to the teacher, but negligibly with that of prompt-conditioned attention. Furthermore, we observe that attention distributions exhibit significant variance across individual tokens, indicating that a uniform distillation objective is suboptimal. To this end, we introduce Token-level Response-visual Attention Guidance (TRAG), a distillation objective that 1) shifts the focus to response-to-vision signals and 2) employs token-specific objectives by adaptively weighting the Kullback-Leibler divergence based on attention entropy, effectively guiding the student to mirror the teacher's precise visual focus. Extensive experimental results on multiple benchmarks demonstrate that TRAG significantly outperforms prior distillation baselines.

Chinese Translation

知识蒸馏（KD）被广泛应用于通过利用大型教师模型的监督来训练轻量级模型，但仅依赖输出令牌分布已被证明不足以压缩多模态大语言模型（MLLMs）。由于输出令牌是模型关注视觉输入的副产品，之前的研究探索了显式蒸馏注意力以提供直接的监督信号。尽管前景看好，但蒸馏哪些注意力信号的具体效用仍未得到充分探讨。在本研究中，我们挑战了传统上对提示-视觉注意力的依赖，揭示下游性能与响应-视觉注意力与教师模型的相似性强相关，而与提示条件注意力的相似性则几乎无关。此外，我们观察到注意力分布在各个令牌之间表现出显著的方差，表明统一的蒸馏目标并不理想。为此，我们提出了令牌级响应-视觉注意力引导（TRAG），这一蒸馏目标1）将重点转向响应-视觉信号，2）通过根据注意力熵自适应加权Kullback-Leibler散度来采用令牌特定的目标，有效引导学生镜像教师的精确视觉焦点。在多个基准上的大量实验结果表明，TRAG显著优于之前的蒸馏基线。

View on arXiv Download PDF AI Translation

cs.CV / 27 / 2607.02594

An automated method of identifying incorrectly labelled images based on the sequences of loss functions of deep learning networks

基于深度学习网络损失函数序列的错误标记图像自动识别方法

Zhang, Zhipeng, Shou, Wenhui, Ma, Wengting, Xing, Dongjia, Xu, Qingqing, Xu, Li-Qun, Fan, Qingxia, Xu, Ling

Abstract

Deep learning is widely applied in medical image analysis, but up to 10% of manually labelled images may be incorrect, degrading model performance. This paper proposes an automated method to identify incorrectly labelled medical images by analyzing sequences of loss functions from deep learning classification networks over multiple training epochs. Identified images can be reviewed and relabelled by experts, improving dataset quality and model performance. Two experiments validate the method on a fundus image dataset for referable diabetic retinopathy screening. In the first, 6% (648) of 10,788 gold-standard labels were intentionally flipped. The method identified 75.31% (488) of the flipped samples, with only 4.85% (492) false positives among correctly labelled samples. In the second, reviewing and correcting the 980 identified samples (9.1% of the dataset) and retraining the model improved best accuracy on an independent test set from 95.93% (with 6% label noise) to 96.50% (with 1.5% noise), approaching the ideal 96.57% (with 0% noise). The results demonstrate the method's effectiveness in improving model performance through automated label quality control.

Chinese Translation

深度学习广泛应用于医学图像分析，但多达10%的手动标记图像可能存在错误，从而降低模型性能。本文提出了一种自动识别错误标记医学图像的方法，该方法通过分析深度学习分类网络在多个训练周期中的损失函数序列来实现。识别出的图像可以由专家进行审核和重新标记，从而提高数据集质量和模型性能。通过在糖尿病视网膜病变筛查的眼底图像数据集上进行的两个实验验证了该方法。在第一个实验中，故意翻转了10,788个金标准标签中的6%（648个）。该方法识别出了75.31%（488个）被翻转的样本，且在正确标记样本中仅有4.85%（492个）为假阳性。在第二个实验中，审核和修正980个识别出的样本（占数据集的9.1%）并重新训练模型，使得在独立测试集上的最佳准确率从95.93%（带有6%的标签噪声）提高到96.50%（带有1.5%的噪声），接近理想的96.57%（无噪声）。结果表明，该方法在通过自动标签质量控制提高模型性能方面的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 28 / 2607.02596

CIPHER: Causal Intervention Pathways for Healthcare Equity and Robustness

CIPHER：医疗公平与稳健性的因果干预路径

Jia, Xinyu, Guo, Weidong, Zhao, Wangyuan, Guo, Yi, Li, Zeju, Wang, Yuanyuan

Abstract

Deep learning models for medical diagnosis frequently exhibit substantial performance disparities across sensitive subgroups (e.g., race, sex), even when average accuracy is high. While generative data augmentation offers a route to mitigate this, existing strategies are suboptimal; they typically address only one or two dependency channels between sensitive attributes and image features. We formalize the medical image formation process via a structural causal model, revealing that sensitive attributes actually influence image content through four distinct pathways-a structural complexity neglected by prior works. Based on this insight, we introduce CIPHER (Causal Intervention Pathways for Healthcare Equity and Robustness), a framework designed to systematically intervene on all four causal paths. To achieve this, CIPHER utilizes a diffusion backbone equipped with classifier-free guidance and null-text inversion. This technical design enables the faithful reconstruction of patient-specific anatomy while allowing for the precise, editable synthesis of counterfactuals required to break sensitive dependency chains. We tested CIPHER using chest X-ray and dermoscopy benchmarks across both standard and shifted data distributions. By employing a multi-pathway intervention strategy, our model reduced worst-group disparities by an average of 35.8% compared to disease-conditioned synthesis baselines, while also improving total diagnostic accuracy

Chinese Translation

深度学习模型在医学诊断中常常表现出在敏感子群体（例如，种族、性别）之间存在显著的性能差异，即使平均准确率较高。尽管生成数据增强提供了一种缓解这一问题的途径，但现有策略并不理想；它们通常仅关注敏感属性与图像特征之间的一两个依赖通道。我们通过结构因果模型形式化了医学图像形成过程，揭示出敏感属性实际上通过四条不同的路径影响图像内容，这一结构复杂性在之前的研究中被忽视。基于这一见解，我们引入了CIPHER（医疗公平与稳健性的因果干预路径），这是一个旨在系统性干预所有四条因果路径的框架。为此，CIPHER利用了配备无分类器引导和空文本反演的扩散骨干网络。这一技术设计能够忠实重建患者特定的解剖结构，同时允许精确、可编辑的反事实合成，以打破敏感依赖链。我们使用胸部X光和皮肤镜基准测试对CIPHER进行了测试，涵盖了标准和偏移的数据分布。通过采用多路径干预策略，我们的模型在与疾病条件合成基线相比，平均减少了35.8%的最差组差异，同时提高了总体诊断准确性。

View on arXiv Download PDF AI Translation

cs.CV / 29 / 2607.02598

Evaluating Agentic Harness Systems for Autonomous Computational Pathology

评估自主计算病理学的智能代理系统

Lin, Jie, Chen, Zongyi, Zheng, Qiaoling, Wang, Liuyi, Jiang, Hengyi, Chen, Jiabao, Liu, Xiang, Yang, Yinghong, Wang, Liansheng

Abstract

Autonomous computational pathology (ACP) converts high-level pathology analysis goals into executable, traceable and clinically bounded workflows. Realizing this capability requires adapting general agentic harness systems to pathology-specific tasks, tools, evidence standards and clinical claim boundaries. We contribute ACP-Bench, a framework that adapts existing harness systems from computational pathology support toward ACP workflow capability. ACP-Bench evaluates 41 pathology workflow tasks, including 24 biomarker, 7 morphology and 10 prognosis tasks spanning 6 body-system groups and 9 endpoint families. The benchmark evaluates 9 models and 3 harness groups (Claude Code, Codex and Open Code), yielding 369 complete trajectories. ACP-Bench evaluates each trajectory across workflow execution, diagnostic performance and clinical-boundary alignment, combining expert-adjudicated process audits, diagnostic assessment and pathologist-validated safety review. Across evaluated systems, workflow initiation, task interpretation and diagnostic reporting were more mature than tool-bound execution, result binding and reflective workflow revision, and formal end-to-end completion remained rare. ACP-Bench provides a reusable standard for auditing whether agentic systems can operationalize pathology workflows before claims of reliable clinical autonomy.

Chinese Translation

自主计算病理学（ACP）将高层次的病理分析目标转化为可执行、可追溯且临床界定的工作流程。实现这一能力需要将通用的智能代理系统调整为特定于病理学的任务、工具、证据标准和临床声明边界。我们贡献了ACP-Bench，一个将现有的计算病理学支持系统适配为ACP工作流程能力的框架。ACP-Bench评估了41个病理工作流程任务，包括24个生物标志物任务、7个形态学任务和10个预后任务，涵盖6个身体系统组和9个终点家族。该基准评估了9个模型和3个智能代理组（Claude Code、Codex和Open Code），产生了369个完整的轨迹。ACP-Bench在工作流程执行、诊断性能和临床边界对齐方面评估每个轨迹，结合了专家裁定的过程审计、诊断评估和病理学家验证的安全审查。在评估的系统中，工作流程启动、任务解释和诊断报告的成熟度高于工具绑定执行、结果绑定和反思性工作流程修订，而正式的端到端完成仍然较为罕见。ACP-Bench提供了一个可重用的标准，用于审计智能代理系统是否能够在声称可靠的临床自主性之前实现病理工作流程。

View on arXiv Download PDF AI Translation

cs.CV / 30 / 2607.02601

CV-DCLR: Causal-Visual Dynamic Label Refinement for Robust Zero-Shot Learning

CV-DCLR：用于稳健零样本学习的因果视觉动态标签精炼

Wang, Can, Li, Jiangnan, Li, Mingyu, Song, Yining, Ren, Kangrui, Gan, Min, Fan, Jinfu

Abstract

Zero-Shot Learning (ZSL) facilitates knowledge transfer via shared semantic spaces. However, a critical bottleneck in this paradigm is Semantic Entanglement, where visual representations are inevitably conflated with visually similar semantic concepts, such as distinguishing the intrinsic traits of a Wolf from the shared features of a Husky. Existing global alignment methods often indiscriminately maximize correlations between visual and semantic modalities, leading models to overfit spurious similarities rather than capturing distinctive class identities. To address this fundamental limitation, we propose the Causal-Visual Dynamic Label Refinement (CV-DCLR) framework. Unlike traditional approaches that rely on superficial visual statistics, CV-DCLR recalibrates visual-semantic associations via a Dual-Stream Mutual Correction Mechanism. This includes a Visual Likelihood Stream to model observational patterns and a Causal Importance Stream that verifies the structural necessity of candidate prototypes through Counterfactual Intervention. Acting as a logical filter, our adaptive gating mechanism dynamically modulates feature responses to amplify genuine causal traits while suppressing visually plausible but structurally irrelevant distractors. Extensive experiments on the CUB, SUN, and AWA2 benchmarks under a rigorous Semantic Entanglement Injection protocol demonstrate that CV-DCLR significantly outperforms state-of-the-art methods in high-ambiguity scenarios. Specifically, while existing models suffer catastrophic degradation under entanglement, our framework maintains robust performance, effectively disentangling true class identities from semantic confounders.

Chinese Translation

零样本学习（ZSL）通过共享语义空间促进知识转移。然而，这一范式中的一个关键瓶颈是语义纠缠，其中视觉表征不可避免地与视觉上相似的语义概念混淆，例如区分狼的内在特征与哈士奇的共享特征。现有的全局对齐方法通常不加区分地最大化视觉和语义模态之间的相关性，导致模型过拟合虚假的相似性，而不是捕捉独特的类别身份。为了解决这一根本性限制，我们提出了因果视觉动态标签精炼（CV-DCLR）框架。与依赖表面视觉统计的传统方法不同，CV-DCLR通过双流互纠正机制重新校准视觉-语义关联。这包括一个视觉似然流来建模观察模式，以及一个因果重要性流，通过反事实干预验证候选原型的结构必要性。作为逻辑过滤器，我们的自适应门控机制动态调节特征响应，以增强真实因果特征，同时抑制视觉上合理但结构上无关的干扰项。在严格的语义纠缠注入协议下，在CUB、SUN和AWA2基准上的大量实验表明，CV-DCLR在高歧义场景中显著优于最先进的方法。具体而言，尽管现有模型在纠缠下遭受灾难性降级，但我们的框架保持了稳健的性能，有效地将真实类别身份与语义混淆因素解缠。

View on arXiv Download PDF AI Translation

cs.CV / 31 / 2607.02604

DynaWM: A Base-VLA-Guided World Foundation Model for Moving-Object Manipulation

DynaWM：一种基于VLA指导的动态物体操作世界基础模型

Chang, Chongkei, Deng, Zhidong

Abstract

Although vision-language-action (VLA) models have received widespread attention, many challenges remain in manipulating dynamic moving objects. In most existing approaches, end-to-end forward or inverse dynamics models, i.e., world models, are incorporated into high-performance base VLA architectures, which may degrade the performance of well-pretrained base VLA models due to inappropriate fine-tuning. In this paper, we propose DynaWM, a base-VLA-guided world foundation model that adapts to a wide variety of fine-tuned and coarse-tuned base-VLA checkpoints for moving-object manipulation. DynaWM uses a Mamba-3-based action encoder to encode the base action chunk produced by the base VLA into an action-conditioning representation, a V-JEPA 2.1 vision encoder to extract features from multi-view observation history, and a proprioceptive state encoder to encode robotic-arm proprioceptive states. These feature representations jointly condition a flow-matching DiT to regenerate motion-aware action trajectories for moving-object manipulation. For systematic evaluation, we construct the DynaGrasp-32 benchmark, covering six categories of moving-object manipulation tasks, including velocity variation, trajectory variation, and multi-object manipulation, as well as the DynaGrasp-1600 dataset, which consists of 32 scenarios, 1,600 demonstration trajectories, and approximately 1.53M images. For fine-tuned base-VLA checkpoints, DynaWM achieves percentage improvements of 7.19, 45.31, 1.88, and 10.94 over SmolVLA, X-VLA, {\pi}0, and {\pi}0.5, respectively. For coarse-tuned base-VLA checkpoints, performance increases by 35.13, 44.06, 35.69, and 26.13 percentage, respectively. Ablation experiments show that visual encoding enhances success by 27.50%, while reducing success by 45.44% if action conditioning is removed.

Chinese Translation

尽管视觉-语言-动作（VLA）模型受到了广泛关注，但在操控动态移动物体方面仍然面临许多挑战。在大多数现有方法中，端到端的前向或逆向动力学模型，即世界模型，被纳入高性能的基础VLA架构中，这可能由于不当的微调而降低预训练基础VLA模型的性能。本文提出了DynaWM，一种基于VLA指导的世界基础模型，能够适应多种微调和粗调的基础VLA检查点，以进行动态物体操作。DynaWM使用基于Mamba-3的动作编码器将基础VLA生成的基础动作块编码为动作条件表示，使用V-JEPA 2.1视觉编码器从多视角观察历史中提取特征，并使用本体状态编码器编码机器人手臂的本体状态。这些特征表示共同条件化一个流匹配的DiT，以再生运动感知的动作轨迹，用于动态物体操作。为了进行系统评估，我们构建了DynaGrasp-32基准，涵盖六类动态物体操作任务，包括速度变化、轨迹变化和多物体操作，以及DynaGrasp-1600数据集，该数据集由32个场景、1,600个演示轨迹和大约1.53M张图像组成。对于微调的基础VLA检查点，DynaWM分别在SmolVLA、X-VLA、{C0}0和{C0}0.5上实现了7.19%、45.31%、1.88%和10.94%的性能提升。对于粗调的基础VLA检查点，性能分别提高了35.13%、44.06%、35.69%和26.13%。消融实验表明，视觉编码提高了成功率27.50%，而如果去除动作条件，则成功率降低了45.44%。

View on arXiv Download PDF AI Translation

cs.CV / 32 / 2607.02607

Latent Visual Cache for Video Reasoning

视频推理的潜在视觉缓存

Zhang, Yongheng, Xu, Zhipeng, Wu, Hao, Li, Yinghui, Yin, Di, Sun, Xing, Yu, Philip S.

Abstract

Video reasoning requires Large Multimodal Models (LMMs) to remain grounded in dense evidence, yet existing systems largely adopt "read-once, generate-many" paradigm, in which visual grounding weakens during generation. This phenomenon has been widely observed and is known as Visual Anchoring Decay. To fill this gap, we introduce Latent Video Cache (Latent-VC), a recurrent latent visual cache inserted into the decoder to preserve compact visual memories throughout reasoning. The cache is trained with supervised contrastive cache alignment and vision-grounded GRPO with a latent grounding reward, while maintaining strict train-inference alignment through native decoder hidden states. Built on Qwen3.5-9B, Latent-VC consistently outperforms strong CoT and SFT+GRPO baselines across six video benchmarks, with especially clear gains on grounding-intensive and long-video tasks. In addition, it also achieves higher accuracy with substantially shorter responses, suggesting that latent visual caching improves video reasoning by preserving visual evidence rather than relying on longer textual chains.

Chinese Translation

视频推理要求大型多模态模型（LMMs）在密集证据中保持扎根，然而现有系统大多采用“读取一次，生成多次”的范式，在生成过程中视觉扎根减弱。这一现象已被广泛观察并被称为视觉锚定衰退。为填补这一空白，我们引入了潜在视频缓存（Latent Video Cache，Latent-VC），这是一种插入解码器中的递归潜在视觉缓存，用于在推理过程中保持紧凑的视觉记忆。该缓存通过监督对比缓存对齐和基于视觉的GRPO（Grounded Reinforcement Policy Optimization）与潜在对齐奖励进行训练，同时通过原生解码器隐藏状态保持严格的训练-推理对齐。基于Qwen3.5-9B，Latent-VC在六个视频基准测试中始终优于强大的CoT和SFT+GRPO基线，尤其在对扎根要求高和长视频任务上表现出明显的提升。此外，它还以显著更短的响应实现了更高的准确性，表明潜在视觉缓存通过保留视觉证据而不是依赖更长的文本链来改善视频推理。

View on arXiv Download PDF AI Translation

cs.CV / 33 / 2607.02611

Privacy-Preserving Industrial Ergonomics: mmWave-Based Automated REBA Scoring and Pose Estimation

隐私保护的工业人机工程学：基于毫米波的自动化REBA评分与姿态估计

Zhang, Xuhan, Dai, Zhuangzhuang, Mans, Luis J., Chang, Victor

Abstract

Work-related Musculoskeletal Disorders (WMSDs) require continuous ergonomic assessments. While Rapid Entire Body Assessment (REBA) is a gold-standard observation tool, manual monitoring is labor-intensive, and vision-based automation leads to privacy concerns. This paper proposes a novel end-to-end multi-task learning framework for privacy-preserving ergonomic assessment using millimetre-wave (mmWave) radar. A spatio-temporal backbone reconstructs 3D human skeletons, which serves as the biomechanical foundation for a subsequent regression head to generate REBA risk scores. To overcome the sparsity of radar point clouds, we utilise a multi-objective loss function incorporating biomechanical limits and temporal smoothness constraints. Furthermore, we implement an oversampling strategy to address the imbalance of high-risk postures in existing datasets. Experimental results on MMFi dataset demonstrate that our framework achieves a Categorical Accuracy of 77.78% and real-time performance with an inference latency of 5.70 ms. Our method reaches a High-risk REBA MAE of 0.93, which significantly outperforms both direct regression and two-stage pipelines in high-risk scenarios, providing a robust solution for non-invasive industrial ergonomic assessment.

Chinese Translation

与工作相关的肌肉骨骼疾病（WMSDs）需要持续的工效评估。虽然快速全身评估（REBA）是金标准的观察工具，但手动监测劳动密集且基于视觉的自动化引发了隐私问题。本文提出了一种新颖的端到端多任务学习框架，利用毫米波（mmWave）雷达进行隐私保护的工效评估。时空骨干网络重建三维人体骨架，为后续的回归头生成REBA风险评分提供生物力学基础。为克服雷达点云的稀疏性，我们采用了一个多目标损失函数，结合生物力学限制和时间平滑约束。此外，我们实施了一种过采样策略，以解决现有数据集中高风险姿势的不平衡问题。在MMFi数据集上的实验结果表明，我们的框架实现了77.78%的分类准确率，并具有5.70毫秒的推理延迟，达到实时性能。我们的方法在高风险REBA均方误差（MAE）上达到了0.93，显著优于高风险场景下的直接回归和两阶段管道，为非侵入式工业工效评估提供了可靠的解决方案。

View on arXiv Download PDF AI Translation

cs.CV / 34 / 2607.02612

Fusion: A Framework for Unified Sequential Token AdaptatIon in VisiOn TraNsformers

Fusion：视觉变换器中统一序列令牌适应的框架

Pradeep, Aravind, Nazari, Samira, Taheri, Mahdi, Herglotz, Christian

Abstract

Vision Transformers achieve strong image classification accuracy but process all image regions with nearly the same computation, even when many regions are redundant or uninformative. Recent adaptive inference methods reduce this cost by selectively compressing tokens or terminating inference early, but combining these mechanisms often causes unstable intermediate representations and accuracy degradation. We introduce Fusion, a unified adaptive inference framework that coordinates token merging, early exiting, and token pruning through a simple staged design: tokens are merged first, confidence is evaluated next, and pruning is applied only to samples that continue inference. This ordering allows the three mechanisms to operate cooperatively rather than competitively. Fusion further includes lightweight routing modules that adapt compression strength to each input and support inference-time adjustment of the accuracy--latency trade-off without retraining. On ImageNet-1k with DeiT-S, Fusion matches or surpasses state-of-the-art adaptive ViT methods at comparable compute budgets while reducing calibration error by up to $4\times$ and inference energy by $48\%$. Experiments across ImageNet-100, CIFAR-100, and ImageNette with multiple ViT backbones demonstrate consistent transferability without dataset-specific tuning.

Chinese Translation

视觉变换器在图像分类精度上表现出色，但对所有图像区域的处理几乎使用相同的计算，即使许多区域是冗余或无信息的。最近的自适应推理方法通过选择性地压缩令牌或提前终止推理来降低这一成本，但将这些机制结合起来往往会导致不稳定的中间表示和精度下降。我们提出了Fusion，一个统一的自适应推理框架，通过简单的分阶段设计协调令牌合并、早期退出和令牌修剪：首先合并令牌，其次评估置信度，最后仅对继续推理的样本应用修剪。这一顺序允许这三种机制协同工作，而不是相互竞争。Fusion还包括轻量级路由模块，根据每个输入自适应压缩强度，并支持在推理时调整精度与延迟的权衡，而无需重新训练。在使用DeiT-S的ImageNet-1k上，Fusion在可比的计算预算下与最先进的自适应ViT方法相匹配或超越，同时将校准误差降低了多达4倍，推理能耗降低了48%。在ImageNet-100、CIFAR-100和ImageNette上进行的多种ViT骨干网络的实验表明，Fusion在没有数据集特定调优的情况下表现出一致的可迁移性。

View on arXiv Download PDF AI Translation

cs.CV / 35 / 2607.02628

SE-UNet: Singular Equivariant Imaging for Real-World Constrained Generation

SE-UNet：用于现实世界约束生成的奇异等变成像

Awadhiya, Kanishk

Abstract

While diffusion models have revolutionized image synthesis, their application to real-world inverse problems is often hampered by the need for massive datasets and the difficulty of imposing strict physical constraints. In this work, we introduce \textbf{SE-UNet} (Singular Equivariant UNet), a framework designed to solve ill-posed imaging tasks without extensive pre-training. By treating generation as an optimization problem constrained by geometric equivariance ($D_4$ group) and singular value gating, SE-UNet effectively standardizes the solution space. We demonstrate that these strong inductive biases allow for state-of-the-art zero-shot inpainting results (80\% missing pixels) on CIFAR-10. Our method surpasses Deep Image Prior (DIP) baselines by over 4 dB in PSNR and exhibits a characteristic "singular snap" convergence -- rapidly locking into the signal manifold. SE-UNet thus offers a data-efficient pathway for constrained generation, aligning with the ReALM-GEN goal of bridging theoretical priors with practical deployment.

Chinese Translation

尽管扩散模型已经彻底改变了图像合成，但其在现实世界逆问题中的应用常常受到对大量数据集需求和施加严格物理约束难度的限制。在本研究中，我们引入了 extbf{SE-UNet}（奇异等变UNet），这是一个旨在解决不适定成像任务的框架，无需广泛的预训练。通过将生成视为一个受几何等变性（$D_4$群）和奇异值门控约束的优化问题，SE-UNet有效地标准化了解决方案空间。我们证明，这些强大的归纳偏置使得在CIFAR-10上实现了最先进的零-shot修复结果（80 ext{%}缺失像素）。我们的方法在PSNR上超过了深度图像先验（DIP）基线超过4 dB，并展现出特征性的“奇异快照”收敛——迅速锁定信号流形。因此，SE-UNet为约束生成提供了一条数据高效的路径，与ReALM-GEN的目标相一致，即将理论先验与实际部署相结合。

View on arXiv Download PDF AI Translation

cs.CV / 36 / 2607.02638

CLABTOOLKIT: An Open-Source Toolkit for Routine Processing, Manipulation, and Visualization of Neuroimaging Data

CLABTOOLKIT：一个用于神经影像数据常规处理、操作和可视化的开源工具包

Alemán-Gómez, Yasser, Hervé, Nino, Hagmann, Patric

Abstract

Neuroimaging research requires manipulating heterogeneous data structures, including raw MRI volumes, volumetric parcellations, cortical surface meshes, tractograms, and connectivity matrices, across tools with incompatible interfaces and file formats, forcing researchers to repeatedly re-implement routine but technically demanding operations. We present CLABTOOLKIT, an open-source Python package that consolidates these operations into a single, coherent framework by representing volumetric, surface, and streamline data as interoperable Python objects. Five core data structures (Parcellation, Surface, AnnotParcellation, Tractogram, and Connectome) encapsulate common neuroanatomical entities and provide consistent methods for loading, processing, and exporting data across standard neuroimaging formats (e.g., NIfTI, GIFTI, FreeSurfer annotations, TCK/TRK), including connectome generation from a parcellation and scalar-map projection onto tractogram streamlines. Complementary modules support BIDS dataset management, FreeSurfer integration, diffusion MRI processing, morphometric analysis, graph-theoretical network analysis, and GPU-accelerated multi-panel visualization via PyVista. The toolkit comprises 19 modules organised into six layers, exposing 13 object-oriented classes with 234 methods and 207 standalone functions, and a JSON-based configuration system enables workflow customization without code changes. Unlike existing neuroimaging libraries, which typically address these tasks separately, CLABTOOLKIT combines color and lookup-table management, parcellation manipulation, multi-surface visualization, and tractography utilities within a single framework. CLABTOOLKIT is compatible with Python 3.9-3.12 and released under the Apache 2.0 license. Source code, documentation, and example workflows are available at https://github.com/connectomicslab/clabtoolkit.

Chinese Translation

神经影像研究需要处理异构数据结构，包括原始MRI体积、体积分区、皮层表面网格、轨迹图和连接矩阵，这些数据在工具之间存在不兼容的接口和文件格式，迫使研究人员反复实现常规但技术要求高的操作。我们提出了CLABTOOLKIT，一个开源的Python包，它通过将体积、表面和流线数据表示为可互操作的Python对象，将这些操作整合到一个统一的框架中。五个核心数据结构（Parcellation、Surface、AnnotParcellation、Tractogram和Connectome）封装了常见的神经解剖实体，并提供了一致的方法用于加载、处理和导出标准神经影像格式的数据（例如，NIfTI、GIFTI、FreeSurfer注释、TCK/TRK），包括从分区生成连接组和将标量图投影到轨迹图流线。补充模块支持BIDS数据集管理、FreeSurfer集成、扩散MRI处理、形态测量分析、图论网络分析，以及通过PyVista进行GPU加速的多面板可视化。该工具包由19个模块组成，分为六个层次，暴露出13个面向对象的类，包含234个方法和207个独立函数，基于JSON的配置系统使得在不更改代码的情况下自定义工作流程。与现有的神经影像库通常分别处理这些任务不同，CLABTOOLKIT将颜色和查找表管理、分区操作、多表面可视化和轨迹图工具整合在一个框架内。CLABTOOLKIT与Python 3.9-3.12兼容，并根据Apache 2.0许可证发布。源代码、文档和示例工作流程可在https://github.com/connectomicslab/clabtoolkit获取。

View on arXiv Download PDF AI Translation

cs.CV / 37 / 2607.02643

BiSLW: Bi-Spectral Latent Watermarking for Generative Diffusion Models

BiSLW：用于生成扩散模型的双谱潜在水印

Pandit, Aryan

Abstract

Diffusion-based generative models have transformed visual content synthesis, yet they remain vulnerable to unauthorized usage and lack reliable attribution methods. Existing watermarking techniques often treat latent tensors as static spatial feature maps or depend on pixel-domain modification, and most do not explicitly leverage the internal frequency structure of the latent space for dual-band redundant embedding, leaving them susceptible to the stochastic nature of diffusion and regeneration attacks. We introduce BiSLW, a trainable bi-spectral latent watermarking framework that jointly embeds aligned identity signals across complementary spectral bands of the decoded diffusion latent using learned encoders and decoders, going beyond fixed-pattern frequency approaches. We leverage the inherent frequency structure of diffusion latents to design a dual-band watermarking framework. Low-frequency components encode global semantics, while high-frequency components capture fine texture. We exploit this structure to embed watermarks across complementary spectral bands. The watermark is independently injected into both bands via learned encoders and recombined before decoding, ensuring it becomes intrinsic to the generative trajectory. Dual spectral decoders recover the watermark from each band, while a cross-band consistency constraint enforces alignment between semantic and textural embeddings. Experiments show that BiSLW achieves a strong balance between perceptual fidelity and robustness, improving PSNR by over 3 dB compared to prior latent diffusion watermarking methods while preserving near-perfect bit accuracy under aggressive regeneration and common distortions, all with negligible computational overhead.

Chinese Translation

基于扩散的生成模型已彻底改变了视觉内容合成，但它们仍然容易受到未经授权的使用，并且缺乏可靠的归属方法。现有的水印技术通常将潜在张量视为静态空间特征图，或依赖于像素域的修改，大多数方法并未明确利用潜在空间的内部频率结构进行双带冗余嵌入，因此容易受到扩散和再生攻击的随机特性影响。我们提出了BiSLW，一种可训练的双谱潜在水印框架，它通过学习的编码器和解码器在解码的扩散潜在的互补谱带中联合嵌入对齐的身份信号，超越了固定模式频率的方法。我们利用扩散潜在的固有频率结构设计了一个双带水印框架。低频成分编码全局语义，而高频成分捕捉细腻的纹理。我们利用这一结构在互补谱带中嵌入水印。水印通过学习的编码器独立注入到两个带中，并在解码前重新组合，确保其成为生成轨迹的内在部分。双谱解码器从每个带中恢复水印，而跨带一致性约束则强制语义和纹理嵌入之间的对齐。实验表明，BiSLW在感知保真度和鲁棒性之间实现了良好的平衡，与先前的潜在扩散水印方法相比，PSNR提高了超过3 dB，同时在激进再生和常见失真下保持近乎完美的比特准确性，且计算开销微乎其微。

View on arXiv Download PDF AI Translation

cs.CV / 38 / 2607.02674

EmoteGPT: 3D Human Facial Expressions from Natural Language Descriptions

EmoteGPT：基于自然语言描述的3D人类面部表情

Wang, Haoran, Mendiratta, Mohit, Theobalt, Christian, Kortylewski, Adam

Abstract

Precise control of 3D facial expressions from text is crucial for virtual avatars, animation, and human-computer interaction, yet existing text-to-3D methods jointly generate identity, expression, and texture, making fine-grained expression control difficult. We instead formulate text-driven expression synthesis as a regression problem in the disentangled parameter space of a 3D Morphable Model (3DMM). This setting, however, requires paired data linking detailed language to precise expression parameters, which are missing from existing resources. To fill this gap, we introduce Txt2Emote, a benchmark of diverse 3D facial expressions with fine-grained textual annotations obtained from GPT-4o and a high-fidelity face tracker, providing both explicit descriptions detailing facial features and implicit descriptions referencing the situational context behind the expression. Leveraging this dataset, we present EmoteGPT, a text-to-3D expression framework based on a Multimodal Large Language Model (MLLM) with a dedicated token to semantically ground expression representations, which are then decoded into 3DMM parameters. We further improve EmoteGPT by augmenting training with large-scale image-to-3DMM data, enabling it to surpass state-of-the-art text-to-3D face synthesis methods on emotion recognition metrics and in perceived expressiveness. Integrated into avatar pipelines, our method enables photorealistic and stylized 3D avatars, as well as expressive 3D-consistent 2D face synthesis from textual input.

Chinese Translation

从文本精确控制3D面部表情对于虚拟化身、动画和人机交互至关重要，但现有的文本到3D方法通常共同生成身份、表情和纹理，使得细粒度的表情控制变得困难。我们将文本驱动的表情合成重新构造为3D可变形模型（3D Morphable Model, 3DMM）解耦参数空间中的回归问题。然而，这种设置需要将详细语言与精确表情参数相链接的配对数据，而现有资源中缺乏这样的数据。为填补这一空白，我们引入了Txt2Emote，这是一个包含多样化3D面部表情的基准数据集，配有从GPT-4o和高保真面部追踪器获得的细粒度文本注释，提供了详细描述面部特征的显性描述和参考表情背后情境的隐性描述。利用该数据集，我们提出了EmoteGPT，这是一个基于多模态大语言模型（Multimodal Large Language Model, MLLM）的文本到3D表情框架，采用专门的标记来语义化表情表示，随后将其解码为3DMM参数。我们进一步通过大规模图像到3DMM数据增强训练，提升EmoteGPT的性能，使其在情感识别指标和感知表现力上超越现有的文本到3D面部合成方法。集成到虚拟化身管道中，我们的方法能够生成照片级真实感和风格化的3D虚拟化身，以及从文本输入生成具有表现力的3D一致性2D面部合成。

View on arXiv Download PDF AI Translation

cs.CV / 39 / 2607.02680

K9-Bench: Evaluating Multimodal LLMs on Canine-Centric Videos

K9-Bench：在以犬类为中心的视频上评估多模态大语言模型

Attarde, Khush, Ali, Yusuf, Thukral, Megha, Bhutani, Divye, Ploetz, Thomas, Kira, Zsolt

Abstract

MLLMs have shown strong zero-shot capabilities across diverse inputs such as across images, video, audio, and text. A crucial, yet underexplored, application of these models lies in understanding and modeling animal-centric scenarios. As animals are integral to millions of households, benchmarking next-generation AI models on pet-focused tasks, ranging from recognizing distress signals to enabling responsive robotic companions, is essential for building AI systems that can work alongside us. We introduce K9-Bench, a novel benchmark focused on real-world domestic dog videos, specifically targeting canine action and interaction understanding via approximately 5000 question-answer pairs across 907 videos spanning 5 distinct task categories that test long-form, canine-centric multimodal reasoning in MLLMs. To create this dataset, we propose a scalable, VLM/LLM-powered data generation pipeline that automatically mines canine-centric videos from the web and curates QA pairs requiring fine-grained, multi-hop reasoning over canine actions and temporally extended interaction sequences. We implement bias mitigation strategies designed to eliminate biases introduced by VLMs during dataset curation. Through extensive experimentation, we find that frontier MLLMs exhibit limited zero-shot performance on canine-centric tasks: although state-of-the-art closed-source models outperform open-source counterparts, they still struggle with compositional reasoning over subtle posture and interaction cues spread over long horizons. We observe that generic chain-of-thought prompting provides only modest performance for such long-horizon reasoning. Beyond a novel dataset for canine activity analysis, K9-Bench provides a general-purpose dataset construction pipeline that can be adapted to other low-data domains for quantitative analysis. Our project website is available at: https://ogmenrobotics.github.io/K9Bench.

Chinese Translation

多模态大语言模型（MLLMs）在图像、视频、音频和文本等多种输入上表现出强大的零-shot 能力。这些模型的一个关键但尚未深入探索的应用在于理解和建模以动物为中心的场景。由于动物是数百万家庭的重要组成部分，因此在以宠物为中心的任务上对下一代人工智能模型进行基准测试至关重要，这些任务包括识别痛苦信号和实现响应式机器人伴侣，以构建能够与我们协同工作的人工智能系统。我们提出了 K9-Bench，这是一个新颖的基准，专注于现实世界的家犬视频，特别针对犬类动作和互动理解，通过约 5000 对问答对覆盖 907 个视频，涵盖 5 个不同的任务类别，测试 MLLMs 在犬类中心的多模态推理能力。为了创建这个数据集，我们提出了一种可扩展的、基于视觉语言模型（VLM）/大语言模型（LLM）的数据生成管道，自动从网络中挖掘以犬类为中心的视频，并策划需要对犬类动作和时间延续的互动序列进行细致、多步推理的问答对。我们实施了旨在消除在数据集策划过程中由 VLM 引入的偏见的偏见缓解策略。通过广泛的实验，我们发现前沿的 MLLMs 在犬类中心任务上的零-shot 性能有限：尽管最先进的闭源模型优于开源模型，但它们在处理分散于长时间范围内的细微姿势和互动线索的组合推理时仍然存在困难。我们观察到，通用的思维链提示对这种长时间范围的推理仅提供了适度的性能。除了为犬类活动分析提供一个新颖的数据集外，K9-Bench 还提供了一个通用的数据集构建管道，可以适应其他低数据领域进行定量分析。我们的项目网站可访问： https://ogmenrobotics.github.io/K9Bench。

View on arXiv Download PDF AI Translation

cs.CV / 40 / 2607.02689

S-EMBER: A Large-Scale Benchmark for Streaming Egocentric Memory Retrieval

S-EMBER：一个大规模的流媒体自我中心记忆检索基准

Wang, Xiaodong, Zhao, Xuanyi, Rodriguez, Pedro, Sachan, Devendra Singh, Oguz, Barlas, Moon, Seungwhan, Li, Shang-Wen, Ghosh, Gargi, Dong, Xin, Yih, Wen-Tau

Abstract

As wearable devices enable continuous first-person recording, AI assistants must reason across long time horizons to recall past experiences-a capability known as episodic memory. Current benchmarks often rely on offline evaluation with access to entire video files, failing to simulate the streaming reality of wearable intelligence. We introduce S-EMBER (Streaming Egocentric Memory Benchmark for Episodic Retrieval), a large-scale benchmark comprising 3,141 videos totaling 388 hours of organic activity captured via Ray-Ban Meta smart glasses. S-EMBER formalizes grounded streaming episodic retrieval, a paradigm shift from global offline search to causal, active recall triggered by visual events in a continuous stream. We provide 9,448 QA pairs requiring manual visual proof through precise temporal localization and supporting flexible response lengths to simulate natural human-AI interaction. Our extensive benchmarking of frontier models uncovers a localization paradox: while semantic reasoning improves with parameter scale, temporal grounding precision remains a stagnant architectural bottleneck that does not benefit from brute-force increases in model size, resolution, or frame density. S-EMBER establishes a hardware-authentic foundation for developing grounded, reliable episodic memory in the next generation of wearable AI agents.

Chinese Translation

随着可穿戴设备实现连续的第一人称录制，人工智能助手必须在较长的时间范围内进行推理，以回忆过去的经历，这种能力被称为情节记忆。目前的基准通常依赖于离线评估，并可以访问整个视频文件，未能模拟可穿戴智能的流媒体现实。我们引入了 S-EMBER（流媒体自我中心记忆基准用于情节检索），这是一个大规模基准，包含 3,141 个视频，总计 388 小时通过 Ray-Ban Meta 智能眼镜捕捉的自然活动。S-EMBER 正式化了基于流媒体的情节检索，这是一种从全局离线搜索到由连续流中的视觉事件触发的因果、主动回忆的范式转变。我们提供了 9,448 对问答（QA）对，要求通过精确的时间定位进行手动视觉证明，并支持灵活的响应长度，以模拟自然的人机交互。我们对前沿模型的广泛基准测试揭示了一个定位悖论：尽管语义推理随着参数规模的增加而改善，但时间基础的精确度仍然是一个停滞的架构瓶颈，无法从模型规模、分辨率或帧密度的简单增加中受益。S-EMBER 为开发下一代可穿戴人工智能代理中的有根据、可靠的情节记忆奠定了硬件真实的基础。

View on arXiv Download PDF AI Translation

cs.CV / 41 / 2607.02692

An Automated Multimodal Glaucoma Detection Framework Using ViT and a Stacking-Based Ensemble

基于ViT和堆叠集成的自动化多模态青光眼检测框架

Jahan, Ishrat, Chowdhury, Muhammad E. H, Murugappan, Murugappan, Podder, Kanchon Kanti, Rahman, Tawsifur, Datta, Shrestha, Hossain, Md Sakib Abrar, Hossen, Md Mosarrof, Mekki, Yosra Magdi Salih, Roy, Sanjiban Sekhar

Abstract

Glaucoma is a progressive eye disease that can lead to irreversible vision loss if not detected at an early stage. Conventional diagnostic procedures are often time-consuming and rely heavily on expert interpretation, limiting their scalability for large-scale screening. In this study, glaucoma detection is investigated under two evaluation settings: sample-wise, where individual samples are analyzed independently, and patient-wise, where data from each patient are aggregated for final prediction. An automated multimodal framework is proposed that integrates fundus images with clinical data. Under the sample-wise setting, detection is performed using fundus images and clinical features individually, as well as through their multimodal combination. Under the patient-wise setting, predictions are obtained by aggregating multiple fundus image representations with corresponding clinical information for each patient. Deep visual features are extracted using a Vision Transformer (ViT) architecture and classified using classical machine-learning models, with a stacking-based ensemble of the three best-performing classifiers employed to optimize performance. Experiments conducted on the publicly available PAPILA dataset demonstrate strong diagnostic performance, achieving 97.47% accuracy and a 97.50% F1-score for sample-wise multimodal classification, and 98.97% accuracy and F1-score for subject-wise detection. The proposed framework is further deployed as an end-to-end web-based platform to support automated glaucoma screening and clinical decision support.

Chinese Translation

青光眼是一种逐渐恶化的眼病，如果未能在早期阶段被检测，将导致不可逆的视力丧失。传统的诊断程序通常耗时较长，并且高度依赖专家的解读，这限制了其在大规模筛查中的可扩展性。本研究在两种评估设置下探讨青光眼检测：样本级（sample-wise），即独立分析每个样本；患者级（patient-wise），即将每位患者的数据聚合以进行最终预测。我们提出了一种自动化多模态框架，将眼底图像与临床数据相结合。在样本级设置下，检测分别使用眼底图像和临床特征进行，同时也通过它们的多模态组合进行。在患者级设置下，通过聚合每位患者的多个眼底图像表示及其相应的临床信息来获得预测。使用视觉变换器（Vision Transformer, ViT）架构提取深度视觉特征，并利用经典机器学习模型进行分类，采用三种表现最佳分类器的堆叠集成以优化性能。在公开可用的PAPILA数据集上进行的实验显示出强大的诊断性能，样本级多模态分类的准确率达到97.47%，F1-score为97.50%；而在受试者级检测中，准确率和F1-score均达到98.97%。所提出的框架进一步作为一个端到端的基于网络的平台部署，以支持自动化青光眼筛查和临床决策支持。

View on arXiv Download PDF AI Translation

cs.CV / 42 / 2607.02693

Property-Constrained 3D Porous Media Reconstruction from 2D Images via Conditional Generative Adversarial Networks

基于条件生成对抗网络的受属性约束的3D多孔介质从2D图像重建

Sadeghkhani, Ali, Bennett, Brandon, Rabbani, Arash

Abstract

This study presents a conditional Generative Adversarial Network (cGAN) framework for generating 3D porous media volumes with controlled porosity, trained exclusively on 2D thin section images. The key innovation lies in combining property-conditioned generation with 2D-to-3D reconstruction, eliminating the need for expensive 3D training data while maintaining control over petrophysical properties. The framework employs a hybrid architecture with a 3D generator and 2D discriminator, where multi-axis slice extraction enables learning 3D-consistent structures from 2D training data. Porosity labels are extracted using an Enhanced U-Net segmentation model. The methodology was demonstrated on two carbonate samples with different lithologies: dolomite-anhydrite and pure dolomite. Results show that the framework successfully generates realistic 3D volumes capturing lithological features such as anhydrite inclusions and fine crystalline textures. Porosity control achieved an $R^2$ of 0.93, with mean absolute errors of 0.019 and 0.010 for the heterogeneous and homogeneous samples, respectively.

Chinese Translation

本研究提出了一种条件生成对抗网络（cGAN）框架，用于生成具有可控孔隙度的3D多孔介质体积，该框架仅基于2D薄片图像进行训练。关键创新在于将属性条件生成与2D到3D重建相结合，消除了对昂贵的3D训练数据的需求，同时保持对岩石物理属性的控制。该框架采用混合架构，包含一个3D生成器和一个2D判别器，通过多轴切片提取使得能够从2D训练数据中学习3D一致的结构。孔隙度标签使用增强型U-Net分割模型提取。该方法在两种不同岩性（白云石-石膏和纯白云石）的碳酸盐样本上进行了验证。结果表明，该框架成功生成了捕捉岩性特征（如石膏包裹体和细晶纹理）的真实3D体积。孔隙度控制的$R^2$值达到了0.93，异质样本和均质样本的平均绝对误差分别为0.019和0.010。

View on arXiv Download PDF AI Translation

cs.CV / 43 / 2607.02699

Aircraft Detection in Satellite Imagery using Deep Learning Object Detectors

基于深度学习目标检测器的卫星图像中的飞机检测

Abbasi, Mujahir Hussain, Beghin, S., Priya, A. T. R. Krishna, Jamuna, P., Anandhakumari, G., Rubavathy, S. Jaanaa

Abstract

The object detection in satellite imagery has garnered considerable attention due to its extensive real-world applications and the inherent challenges it presents, including noise, fluctuating image quality, and intricate backgrounds. This paper proposed a framework for object detection that combines image enhancement and Deep Learning (DL) to make detection more accurate. First, a Gabor filter is used to process the input image to bring out important features and reduce noise. Then, normalization is applied to make sure that the data is evenly distributed so that the model works properly. After that, a model based on YOLOv11 is used to quickly learn and find object features. The proposed method achieves a mAP of 95%, precision of 97%, recall of 85%, and F1-score of 91%, which demonstrates the superior aircraft detection performance. These results show the framework accurately identify aircraft in satellite imagery and is suitable for real-time applications such as surveillance, air traffic monitoring and remote sensing analysis.

Chinese Translation

卫星图像中的目标检测因其广泛的现实应用和所面临的固有挑战（包括噪声、图像质量波动和复杂背景）而受到广泛关注。本文提出了一种结合图像增强和深度学习（Deep Learning, DL）的目标检测框架，以提高检测的准确性。首先，使用Gabor滤波器处理输入图像，以突出重要特征并减少噪声。然后，应用归一化以确保数据均匀分布，从而使模型正常工作。之后，基于YOLOv11的模型被用来快速学习和识别目标特征。所提出的方法实现了95%的平均精度均值（mAP）、97%的精确率、85%的召回率和91%的F1-score，显示出卓越的飞机检测性能。这些结果表明，该框架能够准确识别卫星图像中的飞机，并适用于监视、空中交通监控和遥感分析等实时应用。

View on arXiv Download PDF AI Translation

cs.CV / 44 / 2607.02707

VLRC: Vision-Language Reprojection Consistency as a scalable signal for better feed-forward 3D pretraining

VLRC：视语言重投影一致性作为更好前馈3D预训练的可扩展信号

Hariat, Marwane, Filliat, David, Manzanera, Antoine

Abstract

Feed-forward 3D models are commonly trained using either expensive geometric supervision or self-supervised photometric objectives, both of which provide incomplete learning signals. We introduce Vision-Language Reprojection Consistency (VLRC), a scalable auxiliary objective that exploits frozen vision-language representations as semantic multi-view supervision. Given a predicted 3D reconstruction, VLRC reprojects dense vision-language features across views and enforces feature consistency between corresponding image locations, requiring no additional 3D annotations. The objective integrates seamlessly with both self-supervised monocular reconstruction and supervised-pretrained feed-forward 3D models during unlabeled adaptation. By aligning geometry with language-grounded features, VLRC not only improves depth and camera estimation but also enables more coherent multi-view semantic fusion for open-vocabulary 3D scene understanding. Experiments on indoor and outdoor benchmarks demonstrate consistent gains in 3D reconstruction accuracy and zero-shot open-vocabulary 3D semantic segmentation.

Chinese Translation

前馈3D模型通常使用昂贵的几何监督或自监督光度目标进行训练，这两者都提供了不完整的学习信号。我们提出了视语言重投影一致性（Vision-Language Reprojection Consistency, VLRC），这是一种可扩展的辅助目标，利用冻结的视语言表示作为语义多视图监督。给定一个预测的3D重建，VLRC在视图之间重投影密集的视语言特征，并强制对应图像位置之间的特征一致性，无需额外的3D注释。该目标在无标签适应过程中与自监督单目重建和监督预训练的前馈3D模型无缝集成。通过将几何与语言基础特征对齐，VLRC不仅改善了深度和相机估计，还使开放词汇3D场景理解中的多视图语义融合更加连贯。在室内和室外基准测试上的实验表明，3D重建精度和零样本开放词汇3D语义分割均取得了一致的提升。

View on arXiv Download PDF AI Translation

cs.CV / 45 / 2607.02708

When Does Resolution Help a Frozen Backbone? Global Attention at Resolution Predicts Scalable Adaptation for Camouflaged and Marine Animal Segmentation

何时分辨率有助于冻结的主干网络？全局注意力与分辨率的关系预测伪装和海洋动物分割的可扩展适应性

Rust, Tyler, Kambhamettu, Chandra

Abstract

Adapting frozen vision foundation models to fine-grained segmentation now largely depends on backbone selection. Whether the backbone applies global attention to a high-resolution token set predicts whether a low-rank adapter turns resolution into accuracy. Isotropic ViTs attend globally over the full grid and keep improving with resolution; hierarchical backbones confine early attention to local windows and pool the grid before their global stages, plateauing at lower resolutions. A controlled six-backbone study establishes the pattern, and editing the backbone points to the cause: pooling keeps the benefit, removing global attention does not. The effect is specific to low-rank adaptation. Under one fixed pipeline, SALT (Side-stem, Attention-gated U-Net, Low-rank Tuning), one RGB-only pass on a strong isotropic backbone wins the best S-measure on the four data-matched camouflaged sets, and leads every marine and salient set. It reaches a new state of the art on both marine-animal benchmarks (MAS3K mIoU 0.878).

Chinese Translation

将冻结的视觉基础模型适应于细粒度分割在很大程度上依赖于主干网络的选择。主干网络是否对高分辨率的标记集应用全局注意力，预测低秩适配器是否能将分辨率转化为准确性。各向同性的视觉变换器（Isotropic ViTs）在整个网格上进行全局关注，并随着分辨率的提高而不断改进；而层次化主干网络则将早期注意力局限于局部窗口，并在其全局阶段之前对网格进行池化，导致在较低分辨率时达到平台期。一项受控的六个主干网络研究确立了这一模式，编辑主干网络则指向了原因：池化保持了益处，而去除全局注意力则没有。这一效应特定于低秩适配。在一个固定的流程下，SALT（Side-stem, Attention-gated U-Net, Low-rank Tuning）在强大的各向同性主干网络上进行一次仅RGB的传递，在四个数据匹配的伪装数据集上获得了最佳的S-measure，并在每个海洋和显著性数据集上领先。它在两个海洋动物基准（MAS3K mIoU 0.878）上达到了新的最先进水平。

View on arXiv Download PDF AI Translation

cs.CV / 46 / 2607.02711

RayTun3R: Online Camera Adaptation in 3D Foundation Models

RayTun3R：3D基础模型中的在线相机适应

Sinitsyn, Daniil, Araslanov, Nikita, Cremers, Daniel

Abstract

Recent 3D foundation models, such as DUSt3R, MASt3R, VGGT, $\pi^3$, and Depth Anything 3, provide strong feed-forward depth and pose estimates on pinhole imagery, but degrade sharply under fisheye camera geometry. We show that this failure is partly caused by a pinhole camera bias in the positional encodings of pretrained 3D foundation models, and propose RayTun3R, a lightweight camera adaptation approach. It keeps the pretrained network fixed and adapts only lightweight components tied to token position and camera geometry. RayTun3R learns parameter-efficient residual corrections to absolute and rotary positional encodings, together with parameter-free tokenization and corrections to prediction-grid coordinates that remove residual pinhole assumptions. The resulting adapter contains only 10,752 trainable parameters and can be learned from a short temporal segment using geometric losses. Once adapted, RayTun3R transfers effectively to the remaining frames of the sequence without incurring additional runtime costs. Across diverse fisheye datasets with fields of view from $110^\circ$ to $200^\circ$, our adapter reduces rotation error by $2$-$12\times$ relative to the unadapted model, outperforms LoRA while using $\sim\!14\times$ fewer trainable parameters, improves pose over adaptation-free baselines while avoiding their multi-view inference cost, and remains competitive on depth accuracy.

Chinese Translation

近期的3D基础模型，如DUSt3R、MASt3R、VGGT、$ ext{π}^3$和Depth Anything 3，在针孔成像上提供了强大的前馈深度和姿态估计，但在鱼眼相机几何下表现急剧下降。我们表明，这一失败部分是由于预训练3D基础模型的位置信息编码中存在针孔相机偏差，并提出了RayTun3R，一种轻量级的相机适应方法。该方法保持预训练网络不变，仅适应与标记位置和相机几何相关的轻量级组件。RayTun3R学习参数高效的残差修正，以调整绝对和旋转位置信息编码，同时采用无参数的标记化和对预测网格坐标的修正，以消除残余的针孔假设。最终的适配器仅包含10,752个可训练参数，并且可以通过几何损失从短时间段中学习。一旦适应，RayTun3R能够有效地转移到序列的其余帧上，而不会产生额外的运行时成本。在视场从$110^ ext{°}$到$200^ ext{°}$的各种鱼眼数据集上，我们的适配器相较于未适应模型减少了$2$-$12 imes$的旋转误差，使用约$ ext{14} imes$更少的可训练参数超越了LoRA，在避免多视图推断成本的同时改善了姿态，相较于无适应基线保持竞争力的深度准确性。

View on arXiv Download PDF AI Translation

cs.CV / 47 / 2607.02712

Global Pose Control for Generative View Synthesis in Normalized Object Coordinate Space

在归一化物体坐标空间中进行生成视图合成的全局姿态控制

Li, Zhibing, Gupta, Amogh, Parsa, Behnoosh, Casas, Dan

Abstract

Novel View Synthesis (NVS) enables the generation of unseen views of a scene from a single or multiple images, allowing users to freely explore an object from any viewpoint. Despite the recent impressive qualitative improvements of generative models for this task, existing methods struggle to provide global and intuitive control of target viewpoints because they either use input-relative camera poses or are limited to generating sparse global views. This lack of global pose control severely limits the number of downstream tasks potentially enabled by NVS. To address this limitation, we propose a novel approach for precise camera control in a customizable Normalized Object Coordinate Space (NOCS), requiring single or few unposed images. Our method operates solely on the absolute camera pose of the target view in NOCS, eliminating the need for a relative world frame or camera poses of the input images. Unlike previous methods that treat NVS as a standalone generation task, we formulate it as an image editing problem and build upon state-of-the-art editing models to leverage their superior generalization capability. Camera information is injected as dedicated camera tokens via an in-context multi-modal conditioning strategy. To alleviate the inherent ambiguity of NOCS, we incorporate text descriptions that explicitly define the object's canonical coordinate frame, which also enhances generalization to unseen object categories. Furthermore, we curate a high-quality dataset with consistently aligned orientations and corresponding NOCS text definitions. Extensive experiments demonstrate that our method robustly generates novel views with accurate and consistent orientations from arbitrary unposed images across diverse categories, achieving state-of-the-art image quality and fidelity.

Chinese Translation

新颖视图合成（NVS）使得能够从单张或多张图像生成场景的未见视图，允许用户从任何视角自由探索物体。尽管最近生成模型在这一任务上取得了显著的定性改善，现有方法仍然难以提供目标视点的全局和直观控制，因为它们要么使用相对输入的相机姿态，要么仅限于生成稀疏的全局视图。这种缺乏全局姿态控制的情况严重限制了NVS可能支持的下游任务数量。为了解决这一限制，我们提出了一种新颖的方法，在可定制的归一化物体坐标空间（NOCS）中实现精确的相机控制，仅需单张或少量未标定图像。我们的方法仅依赖于NOCS中目标视图的绝对相机姿态，消除了对相对世界坐标系或输入图像相机姿态的需求。与将NVS视为独立生成任务的先前方法不同，我们将其表述为图像编辑问题，并基于最先进的编辑模型进行构建，以利用其卓越的泛化能力。相机信息通过上下文多模态条件策略作为专用相机标记注入。为了解决NOCS固有的模糊性，我们结合了明确定义物体标准坐标系的文本描述，这也增强了对未见物体类别的泛化能力。此外，我们整理了一个高质量的数据集，具有一致对齐的方向和相应的NOCS文本定义。大量实验表明，我们的方法能够从各种类别的任意未标定图像中稳健地生成具有准确且一致方向的新视图，达到最先进的图像质量和保真度。

View on arXiv Download PDF AI Translation

cs.CV / 48 / 2607.02718

Diagnosing Aerial-View Object Detectors with Foundational Image Generative Models

利用基础图像生成模型诊断空中视角物体检测器

Panev, Stanislav, Jeon, Minhyek, Khindkar, Vaishnavi, Deshpande, Ahish, de Melo, Celso M, Hu, Shuowen, Chakraborty, Shayok, De la Torre, Fernando

Abstract

Recent advances in large-scale image generative models enable photorealistic scene synthesis with controllable attributes. Beyond data augmentation, their potential as diagnostic tools for trained vision systems remains unexplored in the aerial and remote sensing domains. We introduce a synthetic diagnostic framework for aerial-view vehicle detection that combines text-guided generation, attribute-controlled editing, and automated attribute verification to construct a controllable synthetic testbed. This enables fine-grained evaluation of pretrained detectors under diverse scene types and environmental conditions that are difficult to isolate in real datasets. Across three detection architectures and three real aerial datasets, synthetic scene-wise performance trends closely match real-world weaknesses. Guided by these diagnostics, targeted supplementation with small real datasets from the identified weak categories yields improvements of up to 13% AP50 while requiring substantially fewer additional samples than non-targeted augmentation. Our results show that controlled synthetic probing can predict real-domain performance gaps and guide efficient data collection. The proposed diagnostic framework is modular and can incorporate alternative generative or vision-language models as capabilities evolve. Our code and datasets are available here: https://humansensinglab.github.io/AVODDiag/

Chinese Translation

近期大规模图像生成模型的进展使得可控属性的真实场景合成成为可能。除了数据增强之外，它们作为训练视觉系统的诊断工具在航空和遥感领域的潜力尚未被探索。我们提出了一种用于空中视角车辆检测的合成诊断框架，该框架结合了文本引导生成、属性控制编辑和自动属性验证，以构建一个可控的合成测试平台。这使得在多种场景类型和环境条件下对预训练检测器进行细致评估成为可能，而这些条件在真实数据集中难以隔离。在三种检测架构和三个真实航空数据集的实验中，合成场景的性能趋势与现实世界的弱点紧密匹配。在这些诊断的指导下，针对识别出的弱类别进行小规模真实数据集的有针对性补充，能够提高多达13%的AP50，同时所需的额外样本显著少于非针对性增强。我们的结果表明，受控的合成探测可以预测真实领域的性能差距并指导高效的数据收集。所提出的诊断框架是模块化的，可以随着能力的发展整合替代的生成或视觉-语言模型。我们的代码和数据集可在此获取：https://humansensinglab.github.io/AVODDiag/

View on arXiv Download PDF AI Translation

cs.CV / 49 / 2607.02719

GRCD: Grounded Region Change Detection for Multi-Finding Chest X-Ray Pairs

GRCD：用于多发现胸部X光对的基础区域变化检测

Aranya, OFM Riaz Rahman, Najafirad, Peyman, Desai, Kevin

Abstract

Radiologists routinely compare current and prior chest X-rays to track disease progression, producing follow-up reports that describe multiple findings, each localised to an anatomical region and annotated with a temporal change status. Existing automated methods either generate reports from a single image without modelling temporal context, or incorporate temporal information but do not ground their outputs spatially. The few approaches that combine temporal reasoning with spatial grounding are restricted to single-finding descriptions, leaving multi-finding reports with mixed change directions unaddressed. We present GRCD, a framework for grounded report generation from chest X-ray pairs in the multi-finding setting. We first construct a rigorously cleaned dataset of temporal chest X-ray pairs by identifying and correcting two systematic labelling errors in the source annotations. We then introduce a Region-Guided Change Token module that encodes per-region temporal change across anatomical structures and injects this signal into a language model through a dual-pathway strategy combining prepended spatial tokens with gated cross-attention. On a multi-finding test set, GRCD outperforms existing baselines on text generation and clinical accuracy metrics, with gains in change detection. Ablation studies confirm that the dual-pathway design outperforms either integration strategy in isolation on text and clinical metrics, and that region-level change encoding is necessary for multi-finding generation. Code is available at https://github.com/UTSA-VIRLab/GRCD

Chinese Translation

放射科医生通常会比较当前和先前的胸部X光片，以跟踪疾病进展，生成描述多个发现的后续报告，每个发现都定位于解剖区域并标注有时间变化状态。现有的自动化方法要么从单一图像生成报告而不建模时间上下文，要么结合时间信息但未能在空间上定位其输出。少数结合时间推理与空间定位的方法仅限于单一发现的描述，未能解决多发现报告中变化方向混合的问题。我们提出了GRCD，一个用于多发现设置下从胸部X光对生成基础报告的框架。我们首先通过识别和纠正源注释中的两个系统性标注错误，构建了一个经过严格清理的时间胸部X光对数据集。然后，我们引入了一个区域引导变化标记模块，该模块编码解剖结构中每个区域的时间变化，并通过一种双路径策略将该信号注入语言模型，结合了预先添加的空间标记和门控交叉注意力。在多发现测试集上，GRCD在文本生成和临床准确性指标上超越了现有基线，并在变化检测方面取得了提升。消融研究确认，双路径设计在文本和临床指标上优于任何单独的集成策略，并且区域级变化编码对于多发现生成是必要的。代码可在 https://github.com/UTSA-VIRLab/GRCD 获取。

View on arXiv Download PDF AI Translation

cs.CV / 50 / 2607.02721

Provable Pruning for Efficient 3D Gaussian Splatting via Coresets

通过核心集实现高效3D高斯点云的可证明剪枝

Mousa, Waseem, Maalouf, Alaa

Abstract

3D Gaussian Splatting (3DGS) enables high-quality real-time novel-view synthesis, but practical scenes often contain millions of Gaussians, making compression essential for deployment on limited hardware. Existing reduction methods are effective but mostly heuristic: they provide no multiplicative approximation guarantee for the rendered objective, and thus rely heavily on costly post-pruning finetuning to recover quality. We ask a basic question: can a 3DGS scene be provably replaced by a much smaller weighted subset (coreset) while preserving the objective of interest? We first show that, in the unrestricted setting, no non-trivial multiplicative 3DGS coreset exists. We then show that multiplicative guarantees are not impossible, but resolution-dependent. For a prescribed rendering resolution, such as representative views or grids of views/rays, we provide the first weighted coreset construction theorem for 3DGS. The construction samples Gaussians by sensitivity: provable importance scores measuring each Gaussian's role in the full-scene objective. Finally, under explicit validity and log-transmittance stability assumptions, we turn this objective guarantee into a rendering guarantee. Empirically, our method is strongest where deployment needs it most: aggressive compression with no or minimal recovery compute. In prune-only and very short finetuning regimes, it achieves state-of-the-art performance, showing that principled importance estimation can be both theoretically meaningful and practically useful. Open-source code is available at https://github.com/waseem-m/3dgs_provable_coresets.

Chinese Translation

3D高斯点云（3DGS）能够实现高质量的实时新视角合成，但实际场景通常包含数百万个高斯分布，因此在有限硬件上部署时压缩变得至关重要。现有的降维方法虽然有效，但大多是启发式的：它们对渲染目标没有提供乘法近似保证，因此在恢复质量时严重依赖代价高昂的后剪枝微调。我们提出一个基本问题：是否可以在保持目标不变的情况下，用一个更小的加权子集（核心集）可证明地替代3DGS场景？我们首先表明，在不受限制的情况下，不存在非平凡的乘法3DGS核心集。然后，我们展示乘法保证并非不可能，但依赖于分辨率。对于指定的渲染分辨率，例如代表性视图或视图/光线网格，我们提供了3DGS的第一个加权核心集构建定理。该构建通过敏感性对高斯分布进行采样：可证明的重要性评分衡量每个高斯在全场景目标中的作用。最后，在明确的有效性和对数透射稳定性假设下，我们将这一目标保证转化为渲染保证。从经验上看，我们的方法在需要部署的地方表现最佳：在没有或最小恢复计算的情况下实现激进压缩。在仅剪枝和非常短的微调阶段，它达到了最先进的性能，表明原则性的重要性估计在理论上是有意义的，并且在实践中是有用的。开源代码可在 https://github.com/waseem-m/3dgs_provable_coresets 获取。

View on arXiv Download PDF AI Translation

cs.CV / 51 / 2607.02724

Signal from Space: Detecting Schools and Towers to Bridge the Digital Divide

来自太空的信号：检测学校和基站以缩小数字鸿沟

Elmimouni, Zakarya, Farkas, Sandor, Fourati, Fares, Daigele, Vladimir, Mathlouthi, Walid, Alouini, Mohamed-Slim

Abstract

Reliable internet access is essential for modern education, yet millions of school-aged children especially in developing regions remain offline due to unconnected schools. The Giga Initiative aims to connect every school to the internet, but doing so at scale requires efficient methods to map schools and assess surrounding connectivity infrastructure without relying on sparse or noisy third-party datasets. In this work, we propose a scalable, vision-only framework that uses high-resolution satellite imagery and transfer learning to address both tasks simultaneously. By adapting pre-trained object detection models to new geographical regions with minimal labeled data, we detect schools and cell towers directly from space. We then analyze the spatial relationship between detected schools and nearby towers as a proxy for connectivity availability. This purely imagery-driven pipeline enables large-scale infrastructure mapping, reduces dependency on auxiliary data, and supports data-driven prioritization of connectivity investments in underserved areas. Our approach is demonstrated on real satellite imagery from Lesotho, showing strong performance across this region.

Chinese Translation

可靠的互联网接入对现代教育至关重要，但在发展中国家，数百万学龄儿童因学校未联网而无法上网。Giga Initiative旨在将每所学校连接到互联网，但要大规模实现这一目标，需要有效的方法来绘制学校地图并评估周围的连接基础设施，而不依赖稀疏或噪声较大的第三方数据集。在这项工作中，我们提出了一种可扩展的仅基于视觉的框架，利用高分辨率卫星图像和迁移学习同时解决这两个任务。通过将预训练的目标检测模型适应于具有最少标记数据的新地理区域，我们能够直接从太空检测学校和基站。然后，我们分析检测到的学校与附近基站之间的空间关系，以作为连接可用性的代理。这一完全基于图像的流程实现了大规模基础设施映射，减少了对辅助数据的依赖，并支持在服务不足地区对连接投资的基于数据的优先排序。我们的研究在莱索托的真实卫星图像上进行了验证，显示出在该地区的强劲表现。

View on arXiv Download PDF AI Translation

cs.CV / 52 / 2607.02796

Biomechanics-aware Multi-view Markerless Motion Capture of Dexterous Hand Movements

考虑生物力学的多视角无标记灵巧手部运动捕捉

Firouzabadi, Pouyan, Peiffer, J. D., Shah, Kunal, Sobinov, Anton, Miller, Lee E., Cotton, R. James, Murray, Wendy M.

Abstract

Markerless motion capture (MMC) techniques have been widely beneficial in biomechanical analysis of human movement; however, application to complex motions of the hand lags other musculoskeletal systems. The primary goal of this study was to evaluate the performance of a biomechanical reconstruction method that implements a gradient-based optimization approach with a biomechanical model in the loop for tracking dexterous, unconstrained hand movements using MMC. Using a custom, 8-camera setup, we acquired 121 video recordings from 6 participants performing 11 different tasks that spanned 6 hand postures, 5 object manipulation tasks, and involved motion of the proximal upper limb joints. Performance of the proposed MMC pipeline was directly compared to a more commonly adopted two-stage reconstruction method that first triangulates 2D keypoints from computer vision pose estimation algorithms to 3D and then enforces biomechanical constraints by solving a constrained inverse kinematics problem. Relative performance was assessed qualitatively by visual inspection and quantitatively using a computer vision metric. Our method generated solutions for all 121 video recordings; the two-stage method did not converge for 15% of the recordings. Across the remaining videos, our method produced more biomechanically plausible hand kinematics than the two-stage method and was more robust to occlusion effects during tasks that involved objects. The relative robustness of the end-to-end method suggests that it is more effective in utilizing the available 2D digital keypoint information. Automatic and biomechanically meaningful tracking of hand kinematics during dexterous movements has the potential to support clinical evaluation, rehabilitation monitoring, and studies of human motor control.

Chinese Translation

无标记运动捕捉（MMC）技术在人体运动的生物力学分析中具有广泛的应用价值；然而，其在复杂手部运动中的应用仍落后于其他肌肉骨骼系统。本研究的主要目标是评估一种生物力学重建方法的性能，该方法采用基于梯度的优化方法，并在循环中结合生物力学模型，以使用MMC跟踪灵巧的无约束手部运动。我们使用定制的8摄像头设置，从6名参与者那里获取了121段视频记录，参与者执行了11个不同任务，这些任务涵盖了6种手部姿势、5种物体操作任务，并涉及近端上肢关节的运动。我们将所提出的MMC流程的性能与一种更常用的两阶段重建方法进行了直接比较，该方法首先通过计算机视觉姿态估计算法将2D关键点三角化为3D，然后通过求解约束逆运动学问题来施加生物力学约束。相对性能通过视觉检查进行定性评估，并使用计算机视觉指标进行定量评估。我们的方法为所有121段视频记录生成了解决方案；而两阶段方法在15%的记录中未能收敛。在其余视频中，我们的方法产生了比两阶段方法更具生物力学合理性的手部运动学，并且在涉及物体的任务中对遮挡效应更具鲁棒性。端到端方法的相对鲁棒性表明，它在利用可用的2D数字关键点信息方面更为有效。在灵巧运动过程中，手部运动学的自动且具有生物力学意义的跟踪有潜力支持临床评估、康复监测以及人类运动控制研究。

View on arXiv Download PDF AI Translation

cs.CV / 53 / 2607.02798

Track the Noise, Move the World:3D-Grounded Motion-Consistent Noise for Controllable Video Generation

追踪噪声，驱动世界：用于可控视频生成的3D基础运动一致噪声

Vu, Long, Ngo, Tan, Karnewar, Animesh, Habibian, Amir, Hua, Binh-Son, Bui, Hung, Nguyen, Minh Hoai, Nguyen-Ha, Phong

Abstract

Modern image-and-text-to-video diffusion models can synthesize highly realistic videos by iteratively denoising an initial Gaussian noise tensor conditioned on reference image and text inputs. However, existing approaches still lack precise and unified controllability over both object motion and camera motion within a single generation process. We present UniCaMo, a unified framework that enables simultaneous control of object trajectories and camera viewpoints by directly constructing the input noise of the diffusion model. Specifically, UniCaMo builds a shared 3D-grounded motion-consistent noise space across latent video frames. Sparse 3D point tracks are used to warp the Gaussian noise of the reference frame along desired object trajectories, while a virtual spherical noise representation provides globally consistent noise values for newly revealed scene regions under camera motion. By combining local track-guided noise warping with global sphere-based noise sampling, UniCaMo maintains geometric and temporal consistency under both object movement and viewpoint changes. Because UniCaMo modifies only the input noise, it requires no auxiliary adapters, control branches, or architectural changes to the underlying video diffusion model. With lightweight LoRA fine-tuning on large pretrained video diffusion models, including Wan 2.1 (14B), UniCaMo achieves state-of-the-art results in both video quality and motion controllability on standard controllable video generation benchmarks.

Chinese Translation

现代图像与文本到视频的扩散模型通过迭代去噪初始高斯噪声张量，结合参考图像和文本输入，可以合成高度逼真的视频。然而，现有方法在单一生成过程中仍缺乏对物体运动和相机运动的精确和统一的可控性。我们提出了UniCaMo，一个统一框架，能够通过直接构建扩散模型的输入噪声，实现对物体轨迹和相机视点的同时控制。具体而言，UniCaMo在潜在视频帧之间构建了一个共享的3D基础运动一致噪声空间。稀疏的3D点轨迹用于沿着期望的物体轨迹扭曲参考帧的高斯噪声，而虚拟球形噪声表示则为在相机运动下新显现的场景区域提供全局一致的噪声值。通过将局部轨迹引导的噪声扭曲与基于全局球体的噪声采样相结合，UniCaMo在物体运动和视点变化下保持几何和时间一致性。由于UniCaMo仅修改输入噪声，因此不需要辅助适配器、控制分支或对基础视频扩散模型的架构进行更改。通过对大型预训练视频扩散模型（包括Wan 2.1 (14B)）进行轻量级的LoRA微调，UniCaMo在标准可控视频生成基准上实现了视频质量和运动可控性的最先进结果。

View on arXiv Download PDF AI Translation

cs.CV / 54 / 2607.02799

Conversational Human Audio-visual Talking Dialogue Generation

对话人类音频-视觉交互对话生成

Song, Junhao, Guasch, Lluis, He, Xilin, Yang, Zhongyu, Yuan, Yingfang, Xie, Weicheng, Shen, Linlin, Lin, Haijun, Liu, Shizhe, Pang, Wei, Song, Siyang

Abstract

Large-scale dyadic interactive audio-visual dialogue (DIAD) datasets provide fundamental data resources for developing humanoid interactive virtual agents and digital humans. However, collecting such data is time-consuming, expensive, and ethically sensitive. To address this, we propose CHAT, a new dyadic interactive audio-visual dialogue generation (DIADG) framework that generates diverse, paired, and mutually responsive speech-face dialogue clips from a single textual prompt. CHAT unifies large language models and talking face models with interactive audio and facial behaviour refinement modules, enabling the generation of aligned dyadic dialogue clips with diverse contents and facial identities. Experiments show that CHAT outperforms existing related methods designed for similar tasks under both objective and subjective evaluations. Moreover, our synthesised CHAT-AVD-50k dataset serves as effective pre-training data for downstream interactive head generation, consistently improving PerFRDiff and ReactDiff on REACT 2024. CHAT offers a scalable alternative to the costly and ethically sensitive collection of real dyadic interaction data.

Chinese Translation

大规模双人交互音频-视觉对话（DIAD）数据集为开发类人交互虚拟代理和数字人类提供了基础数据资源。然而，收集此类数据既耗时又昂贵，并且涉及伦理敏感性。为了解决这一问题，我们提出了CHAT，一个新的双人交互音频-视觉对话生成（DIADG）框架，该框架能够从单一文本提示生成多样化、成对且相互响应的语音-面部对话片段。CHAT将大型语言模型与对话面部模型结合，并引入交互音频和面部行为优化模块，使得生成的双人对话片段在内容和面部身份上具有多样性。实验表明，CHAT在客观和主观评估中均优于现有针对类似任务设计的方法。此外，我们合成的CHAT-AVD-50k数据集作为下游交互头生成的有效预训练数据，持续提升了在REACT 2024上PerFRDiff和ReactDiff的表现。CHAT为昂贵且伦理敏感的真实双人交互数据收集提供了一种可扩展的替代方案。

View on arXiv Download PDF AI Translation

cs.CV / 55 / 2607.02853

Prior Bias in Vision Language Models on UML Diagram Interpretation

视觉语言模型在UML图解读中的先验偏差

Cheng, Zaiyu, Nguyen, Khai-Nguyen, Mastropaolo, Antonio

Abstract

Vision Language Models (VLMs) are increasingly applied to software engineering artifacts, especially UML class diagrams whose meaning depends on visual notation. Yet, it is unclear whether VLMs actually read such diagrams or instead answer from pretrained priors about how classes typically relate. We introduce a controlled UML benchmark in which each prior-conforming diagram is paired with its prior-conflicting counterpart that (1) preserves the same class names and layout while (2) reverses only the relation arrow. We evaluate eight open-source VLMs from two model families, InternVL3.5 and Qwen3, alongside two closed-source frontier models GPT-5.4 and GPT-5.4 Mini. Across the eight open-source models, reversing the arrow reduces relation-direction accuracy by 33.48% on average, while GPT-5.4 Mini retains a 10% gap. In the harder three-class condition, accuracy drops sharply by 45.28% for open-source models, and even 18.62% for the GPT-5.4 family on average. Scaling provides only limited improvements and is family-dependent. Our benchmark presents a diagnostic prior-driven failure in diagram-grounded software understanding. Our artifact is available at https://anonymous.4open.science/r/UMLKnowledgeConflict-8461.

Chinese Translation

视觉语言模型（VLMs）越来越多地应用于软件工程文档，特别是UML类图，其含义依赖于视觉符号。然而，目前尚不清楚VLMs是否真正理解这些图，还是仅仅基于预训练的先验知识回答关于类之间典型关系的问题。我们引入了一个受控的UML基准，其中每个符合先验的图与其相对立的图相配对，该相对立的图（1）保留相同的类名和布局，同时（2）仅反转关系箭头。我们评估了来自两个模型系列的八个开源VLMs，分别是InternVL3.5和Qwen3，以及两个闭源前沿模型GPT-5.4和GPT-5.4 Mini。在这八个开源模型中，反转箭头使关系方向的准确率平均下降了33.48%，而GPT-5.4 Mini则保持了10%的差距。在更困难的三类条件下，开源模型的准确率急剧下降了45.28%，而GPT-5.4系列的平均准确率下降了18.62%。扩展规模仅提供有限的改进，并且依赖于模型系列。我们的基准揭示了图形基础软件理解中的一种由先验驱动的诊断性失败。我们的文档可在https://anonymous.4open.science/r/UMLKnowledgeConflict-8461获取。

View on arXiv Download PDF AI Translation

cs.CV / 56 / 2607.02860

Cancelable Biometric Template Protection Based on Multi-Instance Fusion: A Contralateral Iris Approach

基于多实例融合的可撤销生物特征模板保护：对侧虹膜方法

Chaivong, Jittarin, Vikromrotjananan, Nicha, Piriyapittaya, Teekatat, Kongprawechnon, Waree, Duangpummet, Suradej

Abstract

Biometric templates are vulnerable to theft if stored without protection. Unlike passwords, a compromised iris cannot be reissued. Although existing cancelable biometric schemes address this problem, most still require an external key or token, introducing an additional attack surface. This paper proposes a cancelable contralateral iris template protection scheme that eliminates the need for a separate token or stored secret, satisfying the three requirements of ISO/IEC 24745: irreversibility, unlinkability, and confidentiality. The method fuses three enrollment samples per eye using Majority Vote Fusion to produce a stable template, and applies a salt-based bit permutation derived from the subject's enrollment ID. Combining the left- and right-permuted templates via a bitwise XOR produces a single Protected Fused Template. Since left and right iris patterns are statistically independent, fusing contralateral irises improves the accuracy of the system. An attacker must possess both iris codes and both salts to recover any useful information, yielding a larger effective key space than single-iris schemes. Experiments on three datasets, CASIA-IrisV4-Interval, CASIA-IrisV2 (two devices), and CASIA-Iris-Thousand, yield EERs of $0.36$\%, $4.88$\%, $10.80$\%, and $3.35$\%, respectively; the highest value reflects the more challenging cross-device scenario. These results demonstrate that our contralateral approach outperforms unprotected baselines while remaining competitive with state-of-the-art cancelable methods. %In addition, an ablation study confirms the benefit on both recognition performance and security. To the best of our knowledge, this is the first scheme to combine tokenless multi-instance fusion and contralateral binding under ISO/IEC 24745.

Chinese Translation

生物特征模板在未加保护的情况下存储时容易被盗取。与密码不同，泄露的虹膜无法重新发行。尽管现有的可撤销生物特征方案解决了这一问题，但大多数仍然需要外部密钥或令牌，从而引入了额外的攻击面。本文提出了一种可撤销的对侧虹膜模板保护方案，消除了对单独令牌或存储秘密的需求，满足ISO/IEC 24745的三个要求：不可逆性、不可关联性和保密性。该方法通过多数投票融合（Majority Vote Fusion）对每只眼睛的三个注册样本进行融合，以生成稳定的模板，并应用基于受试者注册ID的盐值位置换。通过按位异或（bitwise XOR）结合左右置换的模板，生成单一的受保护融合模板。由于左右虹膜模式在统计上是独立的，融合对侧虹膜提高了系统的准确性。攻击者必须同时拥有两个虹膜代码和两个盐值才能恢复任何有用信息，从而比单虹膜方案提供更大的有效密钥空间。在三个数据集（CASIA-IrisV4-Interval、CASIA-IrisV2（两个设备）和CASIA-Iris-Thousand）上的实验结果显示，EER分别为$0.36$ ext{%}、$4.88$ ext{%}、$10.80$ ext{%}和$3.35$ ext{%}，其中最高值反映了更具挑战性的跨设备场景。这些结果表明，我们的对侧方法在性能上优于未保护的基线，同时在竞争力上与最先进的可撤销方法相当。此外，消融研究确认了在识别性能和安全性上的益处。根据我们所知，这是第一个在ISO/IEC 24745下结合无令牌多实例融合和对侧绑定的方案。

View on arXiv Download PDF AI Translation

cs.CV / 57 / 2607.02866

E-TraMamba: A New Paradigm for Efficient Long-Term 3D Feature Tracking with Event Cameras

E-TraMamba：一种高效长时间3D特征跟踪的新范式，基于事件相机

Shen, Juwei, Wu, Yujie, Chen, Changwen

Abstract

Event-based 3D tracking enables low-latency and high-speed perception, while existing CNN- and Transformer-based trackers struggle to capture long-range spatiotemporal dependencies in sparse, noisy event streams, especially under real-time and efficiency constraints. To address these challenges, we present E-TraMamba, the first Mamba-based framework for 3D feature tracking on event data. This new framework adopts a linear state-space model for efficient long-range modeling and integrates a lightweight affine-transform predictor to maintain stable tracking under motion blur and occlusion. We also design an effective scheme to fuse multi-scale cues -- local spatiotemporal patches, correlation maps, and positional embeddings -- into a unified representation that enables stable and smooth 3D tracking. We construct a large-scale synthetic dataset, named EvD-PointOdyssey, which is generated with monocular rendering and provides synchronized event streams, depth maps, and accurate 3D trajectories for training and evaluating event-based 3D tracking models. Extensive experiments on event-based benchmarks demonstrate that E-TraMamba achieves state-of-the-art performance, delivering over $2\times$ longer feature lifetimes under strict accuracy thresholds (e.g., 0.1 m), with higher tracked-feature ratios and lower RMSE than all baselines. These results make E-TraMamba a strong candidate for low-latency visual odometry, real-time SLAM, and interactive robotics.

Chinese Translation

基于事件的3D跟踪实现了低延迟和高速感知，而现有的基于CNN和Transformer的跟踪器在稀疏、噪声事件流中捕捉长距离时空依赖性方面面临困难，尤其是在实时和效率限制下。为了解决这些挑战，我们提出了E-TraMamba，这是第一个基于Mamba框架的事件数据3D特征跟踪方法。该新框架采用线性状态空间模型进行高效的长距离建模，并集成了轻量级的仿射变换预测器，以在运动模糊和遮挡情况下保持稳定的跟踪。我们还设计了一种有效的方案，将多尺度线索——局部时空补丁、相关图和位置嵌入——融合成统一的表示，从而实现稳定和平滑的3D跟踪。我们构建了一个大规模的合成数据集，命名为EvD-PointOdyssey，该数据集通过单目渲染生成，并提供同步的事件流、深度图和准确的3D轨迹，用于训练和评估基于事件的3D跟踪模型。在基于事件的基准测试中进行的广泛实验表明，E-TraMamba在严格的准确性阈值（例如0.1米）下，提供了超过$2 imes$的特征生命周期，具有更高的跟踪特征比率和低于所有基线的均方根误差（RMSE），实现了最先进的性能。这些结果使E-TraMamba成为低延迟视觉里程计、实时SLAM和交互式机器人领域的有力候选者。

View on arXiv Download PDF AI Translation

cs.CV / 58 / 2607.02886

SPLIT: Training-Free AI-Generated and Partially Edited Video Detection via Spatial Patch-Level Incoherence and Temporal Roughness

SPLIT：基于空间补丁级不一致性和时间粗糙度的无训练AI生成和部分编辑视频检测

Hyun, Jongyeop, Kim, Hyounghun

Abstract

Deploying AI-generated video detectors in real-world services demands an ultra-low false positive rate (FPR) on real videos to avoid falsely rejecting authentic content, a regime where standard metrics such as AUROC fail to reflect actual operating behavior. We introduce Spatial Patch-Level Incoherence and Temporal Roughness (SPLIT), a training-free detector that operates on patch tokens from a frozen vision encoder to detect both fully generated and partially edited videos. SPLIT computes two complementary signals: Two-step Temporal Roughness (TTR), capturing non-smooth patch trajectories via one-step and two-step feature variation contrast, and Local Spatial Motion Incoherence (LSMI), measuring spatially inconsistent temporal changes through gradients of a feature-space motion field. The two are fused multiplicatively with gamma correction to sharpen real-fake separation at strict thresholds. We further propose a service-aligned evaluation protocol based on Fake Recall at fixed FPR with real-only threshold calibration and cross-real threshold transfer. Across three benchmarks (FakeParts, GenVideo, and ViF-Bench), SPLIT achieves the highest Fake Recall at FPR = $0.1\%$, substantially outperforming supervised and training-free baselines while remaining robust to post-processing with negligible overhead. The code is publicly available at https://github.com/mldljyh/SPLIT .

Chinese Translation

在实际服务中部署AI生成的视频检测器需要在真实视频上实现超低的假阳性率（FPR），以避免错误拒绝真实内容，而在这种情况下，标准指标如AUROC无法反映实际操作行为。我们提出了空间补丁级不一致性和时间粗糙度（SPLIT），这是一种无训练的检测器，利用冻结的视觉编码器上的补丁令牌来检测完全生成和部分编辑的视频。SPLIT计算了两个互补信号：两步时间粗糙度（TTR），通过一步和两步特征变化对比捕捉不平滑的补丁轨迹，以及局部空间运动不一致性（LSMI），通过特征空间运动场的梯度测量空间上不一致的时间变化。这两者通过伽马校正进行乘法融合，以在严格的阈值下增强真实与虚假的分离。我们进一步提出了一种基于固定FPR的假阳性召回率的服务对齐评估协议，并进行真实阈值校准和跨真实阈值转移。在三个基准测试（FakeParts、GenVideo和ViF-Bench）中，SPLIT在FPR = $0.1\%$时实现了最高的假阳性召回率，显著优于监督和无训练的基线，同时在后处理时保持鲁棒性，开销微乎其微。代码已公开发布在 https://github.com/mldljyh/SPLIT 。

View on arXiv Download PDF AI Translation

cs.CV / 59 / 2607.02907

ProLaViT: Learning Progressive Latent Visual Thoughts in Structured Latent Space

ProLaViT：在结构化潜在空间中学习渐进式潜在视觉思维

Li, Peiming, Wang, Yifan, Zhang, Xiaotian, Hu, Zhiyuan, Li, Shiyu, Wei, Zheng, Tang, Yang

Abstract

Multimodal Large Language Models (MLLMs) have achieved remarkable progress but still struggle with complex visual reasoning tasks requiring multi-step perception and logical deduction. While explicit visual generation incurs prohibitive computational costs, existing latent approaches often rely on external experts or lack rigorous cognitive logic. In this paper, we introduce ProLaViT (Progressive Latent Visual Thought), a framework empowering MLLMs to perform structured visual derivation in the continuous latent space. Unlike works dependent on heterogeneous external models, ProLaViT leverages an endogenous self-distillation mechanism, utilizing the model's own visual encoder to supervise latent thoughts. To facilitate this, we construct a scalable programmatic synthesis pipeline enabling the model to internalize algorithmic precision without inference time tools. We design two reasoning paradigms: (1) Coarse-to-Fine Causal Chain for spatial tasks, guiding attention from global context to local targets. (2) Dialectical Reasoning Chain for logical tasks, incorporating counter-factual thinking for verification. Furthermore, we propose a Distance-Weighted Diversity Loss to impose topology-aware constraints, preventing feature degeneration by enforcing semantic distinctiveness. Extensive experiments demonstrate that ProLaViT outperforms baselines on vision-centric benchmarks, achieving superior accuracy and interpretability with high efficiency.

Chinese Translation

多模态大型语言模型（MLLMs）已取得显著进展，但在需要多步骤感知和逻辑推理的复杂视觉推理任务中仍然面临挑战。虽然显式视觉生成会产生高昂的计算成本，但现有的潜在方法往往依赖外部专家或缺乏严格的认知逻辑。本文介绍了ProLaViT（渐进式潜在视觉思维），这是一个使MLLMs能够在连续潜在空间中执行结构化视觉推导的框架。与依赖异构外部模型的工作不同，ProLaViT利用内生自蒸馏机制，利用模型自身的视觉编码器来监督潜在思维。为此，我们构建了一个可扩展的程序合成管道，使模型能够在没有推理时间工具的情况下内化算法精度。我们设计了两种推理范式：（1）粗到细因果链，用于空间任务，引导注意力从全局上下文转向局部目标；（2）辩证推理链，用于逻辑任务，结合反事实思维进行验证。此外，我们提出了一种距离加权多样性损失，以施加拓扑感知约束，防止特征退化，通过强制语义独特性来实现。大量实验表明，ProLaViT在以视觉为中心的基准测试中优于基线，具有更高的准确性和可解释性，并且效率高。

View on arXiv Download PDF AI Translation

cs.CV / 60 / 2607.02908

Holo-Captioning: Toward the Text Equivalent of 3D Scenes

全息字幕生成：迈向3D场景的文本等价物

Lin, Kun-Yu, Bu, Chengke, Li, Zhenguo, Han, Kai

Abstract

This work introduces holo-captioning, a novel task that strives to seek the text equivalent of 3D scenes. As the initial step, we formulate holo-captioning as generating a structured textual description that comprehensively depicts all entities within a 3D scene -- including their semantic tags, spatial locations, attributes, and inter-entity relations. To tackle this challenging task, we first develop an effective captioning engine to produce detailed descriptions of individual entity instances and instance pairs, and contribute a large-scale benchmark comprising over 15K scenes for training and evaluation. Building upon this foundation, we propose HoloScribe, a novel model that features an instance-aware decoupled pipeline for generating structured holo-captions, and further incorporates anchor-aware instance linking to identify relational instance pairs. Additionally, we propose a comprehensive evaluation metric named HoloScore, and provide a human-curated test set to ensure reliable model assessment. Experimental results demonstrate that HoloScribe significantly outperforms state-of-the-art 3D dense captioners and 3D LLM generalists, underscoring the effectiveness of our approach. Project page: https://visual-ai.github.io/holocap/

Chinese Translation

本研究介绍了全息字幕生成（holo-captioning），这是一项新颖的任务，旨在寻找3D场景的文本等价物。作为初步步骤，我们将全息字幕生成形式化为生成一个结构化的文本描述，全面描绘3D场景中的所有实体，包括它们的语义标签、空间位置、属性和实体间关系。为了解决这一具有挑战性的任务，我们首先开发了一个有效的字幕生成引擎，以生成单个实体实例和实例对的详细描述，并贡献了一个包含超过15K场景的大规模基准数据集用于训练和评估。在此基础上，我们提出了HoloScribe，这是一种新颖的模型，具有实例感知的解耦管道，用于生成结构化的全息字幕，并进一步结合锚点感知的实例链接，以识别关系实例对。此外，我们提出了一种名为HoloScore的综合评估指标，并提供了一个人工审核的测试集，以确保模型评估的可靠性。实验结果表明，HoloScribe显著优于最先进的3D密集字幕生成器和3D大语言模型（LLM）通用模型，突显了我们方法的有效性。项目页面：https://visual-ai.github.io/holocap/

View on arXiv Download PDF AI Translation

cs.CV / 61 / 2607.02909

Learning Taxonomic Trees with Hierarchical Representation Regularization for Large Multimodal Models

通过层次表示正则化学习分类树以适应大型多模态模型

He, Hulingxiao, Tan, Zhi, Peng, Yuxin

Abstract

Taxonomies provide key information about the semantic relationships between concepts and the inherent organization of vision and language. Despite their impressive capabilities, large multimodal models (LMMs) often lack taxonomic knowledge, leading to low hierarchical visual recognition (HVR) consistency. These models typically only rely on language modeling objectives during fine-tuning and lack explicit taxonomy-aware regularization. To address this, we propose Hierarchical Representation Regularization ($HiR^2$), a simple plug-and-play regularizer that improves hierarchical consistency in LMMs. Specifically, we introduce a semantic-aware visual tree construction framework that extracts coarse-to-fine visual features from intermediate LLM layers guided by textual cues. The regularizer combines two complementary objectives: a taxonomic entailment loss that enforces hierarchy via hyperbolic entailment cones in the Lorentz model, and a discriminative dispersive loss that promotes angular separation of semantically similar embeddings on the unit sphere without disturbing the radial hierarchical structure. Extensive experiments demonstrate that $HiR^2$ effectively captures taxonomic structures across diverse LMMs and fine-tuning methods. Code is available at https://github.com/PKU-ICST-MIPL/HiR2_ICML2026.

Chinese Translation

分类法提供了关于概念之间语义关系和视觉与语言内在组织的关键信息。尽管大型多模态模型（LMMs）具有令人印象深刻的能力，但它们往往缺乏分类知识，导致层次视觉识别（HVR）一致性较低。这些模型在微调过程中通常仅依赖语言建模目标，缺乏显式的分类知识正则化。为了解决这一问题，我们提出了层次表示正则化（Hierarchical Representation Regularization，$HiR^2$），这是一种简单的即插即用正则化器，可以改善LMMs中的层次一致性。具体而言，我们引入了一种语义感知的视觉树构建框架，该框架从中间的LLM层中提取粗到细的视觉特征，并受到文本线索的指导。该正则化器结合了两个互补的目标：一个通过洛伦兹模型中的双曲推理锥强制层次结构的分类推理损失，以及一个促进单位球面上语义相似嵌入的角度分离的判别分散损失，而不干扰径向层次结构。大量实验表明，$HiR^2$能够有效捕捉不同LMMs和微调方法中的分类结构。代码可在 https://github.com/PKU-ICST-MIPL/HiR2_ICML2026 获得。

View on arXiv Download PDF AI Translation

cs.CV / 62 / 2607.02912

See the Emotion: A Facial Emoji Proxy Modeling for EEG Emotion Recognition

情感可视化：一种用于脑电图情感识别的面部表情符号代理建模

Hu, Jingjing, Dan, Guo, Cheng, Haofan, Zeng, Ying, Si, Zhan, Zhou, Jinxing, Wang, Meng

Abstract

Despite the high accuracy of EEG-based emotion recognition, existing models remain opaque "black boxes", lacking semantic grounding between abstract neural features and human-interpretable states. In this paper, we reframe EEG explainability as a cross-modal generation task, shifting the paradigm from feature attribution to behavioral visualization. We introduce Facial Emoji Proxy Modeling, a novel framework that translates high-dimensional EEG signals into identity-anonymized facial emojis. Guided by the neuroscientific inspiration of neural-facial association, this approach grounds neural representations in the manifold of observable facial dynamics. Technically, our framework integrates FMENet, a specialized backbone modeling expression-relevant spatial synergies, and the Facial Emoji Learning Branch (FELB), which treats emoji reconstruction as a structured semantic regularizer. Extensive experiments on EAV and MMER benchmarks demonstrate that our method achieves state-of-the-art accuracy among EEG-only models. Crucially, it generates semantically faithful facial animations that provide a transparent, privacy-preserving window into the brain's emotional evolution, effectively allowing users to "see the emotion" directly from neural signals. Code is available at https://github.com/xian-sh/SeeEmotion

Chinese Translation

尽管基于脑电图（EEG）的情感识别具有较高的准确性，但现有模型仍然是不透明的“黑箱”，缺乏抽象神经特征与人类可解释状态之间的语义基础。本文将脑电图的可解释性重新定义为一种跨模态生成任务，将范式从特征归因转向行为可视化。我们提出了面部表情符号代理建模（Facial Emoji Proxy Modeling），这是一个将高维脑电图信号转换为身份匿名的面部表情符号的新框架。在神经科学启发下，该方法将神经表征与可观察的面部动态的多样性相结合。从技术上讲，我们的框架整合了FMENet，这是一种专门建模与表情相关的空间协同的主干网络，以及面部表情符号学习分支（Facial Emoji Learning Branch, FELB），该分支将表情符号重建视为一种结构化的语义正则化器。在EAV和MMER基准上的大量实验表明，我们的方法在仅使用脑电图的模型中达到了最先进的准确性。重要的是，它生成了语义上真实的面部动画，为大脑情感演变提供了一个透明且保护隐私的窗口，有效地使用户能够直接从神经信号“看到情感”。代码可在 https://github.com/xian-sh/SeeEmotion 获取。

View on arXiv Download PDF AI Translation

cs.CV / 63 / 2607.02921

R3D: Quantitative 3D Spatial Reasoning for Egocentric Wearables

R3D：自我中心可穿戴设备的定量三维空间推理

Horton, Maxwell, Lu, Wei, Tran, Quan, Astashonok, Yury, Ahmed, Kirmani, Damavandi, Babak, Kumar, Anuj, Zhang, Xiao, Moon, Seungwhan

Abstract

Quantitative 3D spatial reasoning from egocentric RGB-D video is a critical capability for next-generation wearable assistants. Yet existing benchmarks do not reflect the challenges of handling (1) natural egocentric video, (2) posed RGB-D video inputs, and (3) challenging quantitative 3D spatial reasoning Q&A. To fill this gap, we introduce R3D-Bench (Reasoning in 3D), a benchmark of 3,033 quantitative spatial reasoning questions across 15 types -- spanning multiple-choice, distance-based, and volumetric reasoning questions -- built on top of 57 egocentric video sequences from Aria Digital Twin. To set a strong baseline on this dataset, we introduce R3D, a model-agnostic spatial tool-calling framework. In contrast to existing approaches that directly embed 3D information into the model's input representation, R3D constructs a 3D scene from video using segmentation and depth-lifted object representations. It provides this information to an LLM through eight composable spatial tools. On R3D-Bench, R3D with Qwen3-VL 235B achieves 73.5% mean relative accuracy, substantially outperforming the best depth-enabled baseline (CuTR+Tools, 61.9%) and the best RGB-only baseline (Gemini 3 Flash, 46.5%).

Chinese Translation

从自我中心RGB-D视频中进行定量三维空间推理是下一代可穿戴助手的一项关键能力。然而，现有基准并未反映处理（1）自然自我中心视频，（2）姿态RGB-D视频输入，以及（3）具有挑战性的定量三维空间推理问答的难点。为填补这一空白，我们引入了R3D-Bench（3D推理），这是一个包含3,033个定量空间推理问题的基准，涵盖15种类型——包括多项选择、基于距离和体积推理问题——基于来自Aria Digital Twin的57个自我中心视频序列。为了在该数据集上建立一个强有力的基线，我们引入了R3D，一个模型无关的空间工具调用框架。与现有方法直接将三维信息嵌入模型输入表示不同，R3D通过分割和深度提升的对象表示从视频构建三维场景。它通过八个可组合的空间工具将这些信息提供给大型语言模型（LLM）。在R3D-Bench上，使用Qwen3-VL 235B的R3D达到了73.5%的平均相对准确率，显著超越了最佳深度支持基线（CuTR+Tools，61.9%）和最佳仅RGB基线（Gemini 3 Flash，46.5%）。

View on arXiv Download PDF AI Translation

cs.CV / 64 / 2607.02922

STAC: Selective Spatiotemporal Aggregation and Compression for Video Reasoning Segmentation

STAC：用于视频推理分割的选择性时空聚合与压缩

Hesham, Syed Ariff Syed, Liu, Yun, Sun, Guolei, Yang, Jing, Ding, Henghui, Geng, Xue, Jiang, Xudong

Abstract

Video reasoning segmentation demands pixel-accurate object tracking across hundreds of frames under complex natural language queries, producing dense spatiotemporal tokens whose quadratic self-attention cost makes long-video processing prohibitive. Existing methods address this through token compression, yet typically operate on encoder features lacking temporal context, constraining selection before content redundancy can be reliably assessed. Informed compression requires contextual awareness, but acquiring that awareness at full resolution incurs the same quadratic cost compression aims to reduce. State-space models resolve this constraint, as their linear recurrence selectively conditions each token on temporal context at $\mathcal{O}(T)$ cost, producing representations where content redundancy becomes assessable. Building on this, Selective SpatioTemporal Aggregation and Compression (STAC) enriches features via decoupled bidirectional spatial and causal temporal scanning, leveraging recurrence-derived redundancy for hierarchical compression with adaptive thresholds optimised with segmentation objective. STAC achieves 85% token reduction and 1.8$\times$ speedup while surpassing compression-free baselines on reasoning segmentation benchmarks in a zero-shot streaming-compatible setting. Code is available \href{https://github.com/MCG-NKU/nku-video}{here}.

Chinese Translation

视频推理分割要求在复杂自然语言查询下，对数百帧进行像素级的对象跟踪，从而生成密集的时空标记。然而，其二次自注意力的计算成本使得长视频处理变得不可行。现有方法通过标记压缩来解决这一问题，但通常是在缺乏时间上下文的编码器特征上操作，限制了在内容冗余可以可靠评估之前的选择。有效的压缩需要上下文意识，但在全分辨率下获取这种意识会产生与压缩目标相同的二次成本。状态空间模型解决了这一约束，因为它们的线性递归以 $ ext{O}(T)$ 的成本选择性地将每个标记与时间上下文相结合，从而生成可以评估内容冗余的表示。在此基础上，选择性时空聚合与压缩（STAC）通过解耦的双向空间和因果时间扫描来丰富特征，利用递归衍生的冗余进行分层压缩，并使用优化了分割目标的自适应阈值。STAC 实现了85%的标记减少和1.8倍的加速，同时在零-shot流媒体兼容设置下超越了无压缩基线在推理分割基准上的表现。代码可在此获取： ext{https://github.com/MCG-NKU/nku-video}。

View on arXiv Download PDF AI Translation

cs.CV / 65 / 2607.02927

VideoSearcher: Empowering Video Deep Research with Multi-Tool Agentic Reasoning via Reinforcement Learning

VideoSearcher：通过强化学习赋能视频深度研究的多工具自主推理

Gao, Zhenkun, Bao, Yicheng, Peng, Jinlong, Li, Xueheng, Huang, Theo, Liu, Bangwei, Li, Kunquan, Gan, Zhenye, Hu, Tao, Xie, Chengjun, Yang, Mingqian, He, Xuanhua, Zhang, Zhizhong, Tan, Xin, Wang, Chengjie, Xie, Yuan

Abstract

Video understanding is moving beyond closed-context perception toward open-world evidence exploration, a paradigm formalized as Video Deep Research (VDR). However, existing multimodal search agents primarily target static images, and the current VDR benchmark relies on text-centric retrieval that discards crucial visual information. To address these limitations, we propose VideoSearcher, a closed-loop agentic framework that empowers Vision-Language Models with multi-tool reasoning for VDR. VideoSearcher unifies temporal localization, spatial focusing, and multimodal search within a single reasoning trajectory, enabling agents to progressively ground visual clues, retrieve relevant evidence, and synthesize answers. To optimize knowledge-intensive reasoning trajectories, we propose Bi-branch Sequence Policy Optimization (BiSPO), a reinforcement learning algorithm that decouples tool-invocation optimization from answer-accuracy optimization. This design provides stable learning signals for both evidence-grounded reasoning and purposeful tool use. Furthermore, we construct VideoSearch-QA, the first benchmark designed to evaluate open-world video information grounding and multimodal search-based reasoning. Extensive experiments demonstrate that VideoSearcher significantly outperforms prior open-source agentic baselines across various search-oriented and multimodal understanding benchmarks.

Chinese Translation

视频理解正从封闭环境感知向开放世界证据探索转变，这一范式被正式化为视频深度研究（Video Deep Research, VDR）。然而，现有的多模态搜索代理主要针对静态图像，而当前的VDR基准依赖于以文本为中心的检索，忽略了关键的视觉信息。为了解决这些局限性，我们提出了VideoSearcher，这是一种闭环自主框架，赋能视觉-语言模型进行多工具推理以实现VDR。VideoSearcher将时间定位、空间聚焦和多模态搜索统一在单一推理轨迹中，使代理能够逐步定位视觉线索、检索相关证据并综合答案。为了优化知识密集型推理轨迹，我们提出了双分支序列策略优化（Bi-branch Sequence Policy Optimization, BiSPO），这是一种将工具调用优化与答案准确性优化解耦的强化学习算法。该设计为证据基础推理和有目的的工具使用提供了稳定的学习信号。此外，我们构建了VideoSearch-QA，这是第一个旨在评估开放世界视频信息基础和基于多模态搜索推理的基准。大量实验表明，VideoSearcher在各种搜索导向和多模态理解基准上显著优于先前的开源自主基线。

View on arXiv Download PDF AI Translation

cs.CV / 66 / 2607.02930

CL-Anomaly: Layer-Adaptive Mixture-of-Experts with Multimodal Large Language Model for Continual Learning in Anomaly Detection

CL-Anomaly：基于多模态大型语言模型的层自适应专家混合模型在异常检测中的持续学习

Dong, Wen, Wang, Zhao, Zhang, Shuangqing, Sun, Kai, Li, Ben, Xie, Guo-Sen, Shan, Caifeng, Zhao, Fang

Abstract

Multimodal Large Language Models (MLLMs) excel in diverse vision tasks, but full-parameter retraining is computationally expensive as real-world knowledge evolves. Existing continual learning methods often suffer from semantic entanglement in parameter spaces across tasks, impeding the continuous deployment of models. This challenge is especially pronounced in Anomaly Detection (AD), which exhibits triple heterogeneity across modalities, domains, and defect scale variability, significantly complicating multi-task knowledge transfer. In this paper, we propose CL-Anomaly, a parameter-efficient fine-tuning framework based on an isolation-sharing collaboration to enable continual learning for anomaly detection with MLLMs. We introduce the task-private expert PrivLoRA, which physically isolates task-specific subspaces in the parameter space to prevent semantic entanglement of anomaly knowledge in diverse scenarios. The Layer-Adaptive Shared Experts maintain cross-task representations within a unified feature space, enabling knowledge sharing between previous and new tasks. Furthermore, we propose a Layer-Adaptive Knowledge Transfer strategy that automatically selects and dynamically updates the layer-wise key shared experts of each task via a momentum-based mechanism, promoting effective knowledge transfer across related anomaly detection tasks. Extensive experiments across three continual learning scenarios for anomaly detection, including class-incremental, cross-domain, and cross-modal, demonstrate that CL-Anomaly outperforms state-of-the-art methods. Code is available at https://github.com/WenDongyp/CL-Anomaly.

Chinese Translation

多模态大型语言模型（MLLMs）在多样的视觉任务中表现优异，但由于现实世界知识的不断演变，全面参数重训练的计算成本较高。现有的持续学习方法常常在任务之间的参数空间中遭遇语义纠缠，妨碍了模型的持续部署。这一挑战在异常检测（AD）中尤为明显，因为其在模态、领域和缺陷规模的变异性上表现出三重异质性，显著复杂化了多任务知识转移。在本文中，我们提出了CL-Anomaly，一种基于隔离共享协作的参数高效微调框架，以实现MLLMs在异常检测中的持续学习。我们引入了任务私有专家PrivLoRA，该专家在参数空间中物理隔离任务特定的子空间，以防止在多样场景中异常知识的语义纠缠。层自适应共享专家在统一特征空间内维护跨任务表示，从而实现先前任务与新任务之间的知识共享。此外，我们提出了一种层自适应知识转移策略，该策略通过基于动量的机制自动选择并动态更新每个任务的层级关键共享专家，促进相关异常检测任务之间的有效知识转移。在三个异常检测的持续学习场景（包括类别增量、跨领域和跨模态）中进行的广泛实验表明，CL-Anomaly优于现有的最先进方法。代码可在 https://github.com/WenDongyp/CL-Anomaly 获取。

View on arXiv Download PDF AI Translation

cs.CV / 67 / 2607.02934

MatPhaseBench: A Semantics-Guided Benchmark for Materials Phase Diagrams Understanding

MatPhaseBench：一种基于语义引导的材料相图理解基准

Wang, Hanwen, Liang, Sihan, Liu, Zhiwei, Wang, Yangang, Yan, Wei, Liu, Yuqin, Wang, Zongguo

Abstract

Materials phase diagrams are a core knowledge representation in materials science, encoding temperature,composition, phase stability, and phase transformation pathways, with their full understanding requiring thermodynamic mechanism analysis and scientific reasoning. Although VLMs have shown promise in scientific image understanding, their systematic evaluation on such logically complex images demanding deep mechanistic interpretation remains limited, and phase diagrams provide a challenging testbed for this purpose. We introduce MatPhaseBench, a high-quality, high-reliability benchmark for complex scientific image understanding, focused on materials phase diagrams. MatPhaseBench is constructed from 3681 papers in classical materials science journals, from which 200 high-quality diagram-text pairs were selected, covering 189 material systems and 70 elements. The benchmark has three key features: (1)targeting complex scientific image understanding-it moves beyond simple objective tests to open-ended tasks requiring deep comprehension; (2)comprehensive image-text alignment-semantic information associated with images is fully preserved during literature mining and matching; (3) high-quality human-supervised text acquisition-all descriptions undergo strict manual validation. Experimental results show that current VLMs remain substantially behind expert-level understanding: they are largely limited to surface visual perception, lack deep reasoning grounded in thermodynamic mechanisms, have limited domain awareness and expert analytical experience, and perform poorly in distinguishing fine-grained differences in composite or multi-diagram settings. Overall, MatPhaseBench constitutes a challenging research-grade benchmark, providing a foundational platform for complex scientific image understanding, phase diagram analysis, and trustworthy multi-modal AI in science.

Chinese Translation

材料相图是材料科学中的核心知识表示，编码了温度、成分、相稳定性和相变路径，其全面理解需要热力学机制分析和科学推理。尽管视觉语言模型（VLMs）在科学图像理解方面展现出潜力，但对这些需要深层机制解释的逻辑复杂图像的系统评估仍然有限，而相图为此提供了一个具有挑战性的测试平台。我们介绍了MatPhaseBench，这是一个高质量、高可靠性的复杂科学图像理解基准，专注于材料相图。MatPhaseBench由3681篇经典材料科学期刊论文构建，从中选择了200对高质量的图表-文本配对，涵盖189种材料系统和70种元素。该基准具有三个关键特征：（1）针对复杂科学图像理解——它超越了简单的客观测试，转向需要深刻理解的开放式任务；（2）全面的图像-文本对齐——与图像相关的语义信息在文献挖掘和匹配过程中得到了充分保留；（3）高质量的人为监督文本获取——所有描述都经过严格的人工验证。实验结果表明，当前的VLMs在专家级理解方面仍然显著滞后：它们主要限于表面的视觉感知，缺乏基于热力学机制的深层推理，领域意识和专家分析经验有限，并且在区分复合或多图设置中的细微差异方面表现不佳。总体而言，MatPhaseBench构成了一个具有挑战性的研究级基准，为复杂科学图像理解、相图分析和科学中的可信多模态人工智能提供了基础平台。

View on arXiv Download PDF AI Translation

cs.CV / 68 / 2607.02952

Pooling-Based Context Modeling for Convolution-Free Deep Image Prior

基于池化的上下文建模用于无卷积深度图像先验

Kim, Gihyun, Lee, Jong-Seok

Abstract

Convolutional Neural Networks (CNNs) achieve strong denoising performance by exploiting spatial context from neighboring pixels. Deep Image Prior (DIP) leverages this property to restore images from a single noisy input without requiring large datasets. However, the over-parameterized architecture of DIP often leads to noise fitting during optimization. In this paper, we propose Pool-DIP, a convolution-free architecture that incorporates pooling-based contrast modeling to capture spatial context efficiently. Pool-DIP improves denoising performance while significantly reducing the number of parameters and computational complexity compared to convolution-based DIP models. Experimental results show that Pool-DIP achieves competitive performance across multiple datasets, including a real-world benchmark. Spectral analysis further reveals that Pool-DIP stabilizes the evolution of high-frequency components during optimization and suppresses erroneous high-frequency signals. The proposed architecture also generalizes well to other image restoration tasks such as super-resolution and inpainting.

Chinese Translation

卷积神经网络（CNN）通过利用邻近像素的空间上下文实现了强大的去噪性能。深度图像先验（DIP）利用这一特性，从单一噪声输入中恢复图像，而无需大型数据集。然而，DIP的过参数化架构在优化过程中往往导致噪声拟合。本文提出了Pool-DIP，一种无卷积架构，结合基于池化的对比建模，以高效捕捉空间上下文。与基于卷积的DIP模型相比，Pool-DIP在显著减少参数数量和计算复杂度的同时，提高了去噪性能。实验结果表明，Pool-DIP在多个数据集上表现出竞争力，包括一个真实世界的基准。谱分析进一步揭示，Pool-DIP在优化过程中稳定了高频成分的演变，并抑制了错误的高频信号。所提架构在其他图像恢复任务（如超分辨率和修复）上也具有良好的泛化能力。

View on arXiv Download PDF AI Translation

cs.CV / 69 / 2607.02956

MORE: A Multilingual Document Parsing Benchmark and Evaluation

MORE：多语言文档解析基准与评估

Xu, Long, Wu, Binghong, Yu, Tinghao, Feng, Hao, Huang, Zhenyu, Jiang, Haoqing, Wang, Yunhao, Huang, Shuo, Zhang, Feng

Abstract

Multilingual documents encapsulate rich regional cultures, scientific discoveries, and historical records. Parsing this content into structured, machine-readable formats is critical for unlocking global knowledge. However, existing benchmarks predominantly focus on high-resource languages like English and Chinese, creating an evaluation blind spot concerning model performance on other languages. While recent Vision-Language Models (VLMs) claim support for hundreds of languages, the lack of ground truth makes it impossible to empirically verify these capabilities. To bridge this gap, we introduce MORE, a large-scale benchmark designed for multilingual document parsing evaluation. MORE distinguishes itself through three key dimensions: (1) Unprecedented Scale: It covers 149 languages, making it the most linguistically diverse benchmark to date; (2) Structural Complexity: Unlike previous works, it extends evaluation beyond plain text to include structural elements such as code blocks, tables, and catalogs; and (3) Data Authenticity: All samples are curated from real-world documents via a model-assisted, human-refined annotation pipeline. We evaluate state-of-the-art models using MORE, establishing new performance baselines for long-tail languages and validating the benchmark's effectiveness in diagnosing model capabilities in realistic, diverse scenarios. The MORE dataset will be available at https://github.com/zimoqingfeng/MORE.

Chinese Translation

多语言文档蕴含丰富的地区文化、科学发现和历史记录。将这些内容解析为结构化、机器可读的格式对于解锁全球知识至关重要。然而，现有的基准主要集中在英语和中文等高资源语言上，导致在其他语言上模型性能的评估存在盲点。尽管最近的视觉语言模型（Vision-Language Models, VLMs）声称支持数百种语言，但缺乏真实的基准数据使得无法实证验证这些能力。为填补这一空白，我们推出了MORE，一个旨在多语言文档解析评估的大规模基准。MORE在三个关键维度上具有独特性：（1）前所未有的规模：它涵盖149种语言，成为迄今为止语言多样性最高的基准；（2）结构复杂性：与以往的工作不同，它将评估范围扩展到包括代码块、表格和目录等结构元素，而不仅仅是纯文本；（3）数据真实性：所有样本均通过模型辅助的人为精细注释流程从真实世界文档中策划而来。我们使用MORE评估最先进的模型，为长尾语言建立新的性能基准，并验证该基准在真实多样化场景中诊断模型能力的有效性。MORE数据集将可在 https://github.com/zimoqingfeng/MORE 获取。

View on arXiv Download PDF AI Translation

cs.CV / 70 / 2607.02957

ReLo-IRR: Reflection-Guided LoRA Framework for Image Reflection Removal

ReLo-IRR：基于反射引导的LoRA框架用于图像反射去除

Wang, Chaoqun, Wei, Yuehuan, Cao, Haoxiang, Min, Shaobo

Abstract

Single-image reflection removal (SIRR) aims to recover the clean transmission layer from a reflection-contaminated image. Although recent methods achieve promising results with large diffusion models, they rely on image-agnostic adaptation strategies, e.g., fine-tuning or ControlNet, that enforce uniform suppression regardless of reflection severity. As a result, heavy reflections often leave residuals, while weak ones suffer from detail loss. To this end, we propose ReLo-IRR, a reflection-guided LoRA framework built upon the rectified flow model. First, a lightweight estimator is designed to predict the reflection strength descriptor, providing an explicit prior of reflection dominance for each image and enabling image-dependent LoRA modulation. Second, we introduce a time-conditioned mechanism that fuses this reflection descriptor with timestep embeddings, enabling LoRA modulation to evolve consistently with the coarse-to-fine denoising process. By jointly modeling reflection strength and denoising dynamics, our ReLo-IRR achieves robust suppression of diverse reflection conditions. Extensive experiments on challenging benchmarks validate the effectiveness of ReLo-IRR, demonstrating superior dereflection performance and robust generalization. The code is released at https://github.com/KONGBAI-8080/ReLo-IRR.

Chinese Translation

单幅图像反射去除（SIRR）旨在从受反射污染的图像中恢复干净的传输层。尽管近期方法在大型扩散模型上取得了令人鼓舞的结果，但它们依赖于与图像无关的适应策略，例如微调或ControlNet，这些策略强制执行均匀抑制，而不考虑反射的严重程度。因此，强烈的反射往往会留下残余，而较弱的反射则会导致细节丢失。为此，我们提出了ReLo-IRR，一个基于修正流模型的反射引导LoRA框架。首先，设计了一种轻量级估计器来预测反射强度描述符，为每幅图像提供反射主导性的显式先验，从而实现与图像相关的LoRA调制。其次，我们引入了一种时间条件机制，将该反射描述符与时间步嵌入融合，使得LoRA调制能够与粗到细的去噪过程一致演变。通过联合建模反射强度和去噪动态，我们的ReLo-IRR实现了对多种反射条件的强有力抑制。在具有挑战性的基准测试中的大量实验验证了ReLo-IRR的有效性，展示了其卓越的去反射性能和强大的泛化能力。代码已发布在 https://github.com/KONGBAI-8080/ReLo-IRR。

View on arXiv Download PDF AI Translation

cs.CV / 71 / 2607.02959

Incentivizing Vision Language Models to Search for Long Video Question Answering

激励视觉语言模型搜索长视频问答

Goel, Harsh, Sharan, S P, Shah, Sahil, Choi, Minkyu, An, Joungbin, Grauman, Kristen, Chinchali, Sandeep P.

Abstract

We introduce VSeek, an agentic framework that transforms long-video question answering (LVQA) from a passive, single-pass perception task into a multi-turn retrieval process. VSeek utilizes a natural language-driven search to identify relevant context within long videos and is post-trained with reinforcement learning (RL) to jointly formulate targeted search queries and reason over retrieved clips for LVQA. While RL post-training has revolutionized reasoning in symbolic domains such as mathematics and code, its application to long-video understanding remains hindered by a lack of verified rewards. To ensure that the retrieved context is relevant, we propose a novel neuro-symbolic approach that bridges open-ended natural language with discrete visual verification. Specifically, complex user queries are compiled into formal temporal logic specifications for systematically decomposing natural language questions into a definitive checklist of required atomic visual primitives, such as key objects and activities, along with their temporal ordering. These systematically derived grounding events provide the critical feedback signal for RL post-training, enabling dense, verifiable rewards based on the successful retrieval of these specific visual elements rather than relying entirely on outcome-only answer accuracy. By explicitly optimizing for this verifiable evidence-seeking behavior, VSeek improves Pass@1 scores by up to 8% and Pass@4 scores by 15% on long-video understanding benchmarks compared to base models. We open-source our code at https://utaustin-swarmlab.github.io/VSeek.

Chinese Translation

我们提出了 VSeek，一个代理框架，将长视频问答（LVQA）从被动的单次感知任务转变为多轮检索过程。VSeek 利用自然语言驱动的搜索来识别长视频中的相关上下文，并通过强化学习（RL）进行后训练，以共同制定针对性的搜索查询并对检索到的片段进行推理。尽管 RL 后训练在数学和代码等符号领域的推理方面取得了革命性进展，但其在长视频理解中的应用仍受到缺乏验证奖励的限制。为了确保检索到的上下文是相关的，我们提出了一种新颖的神经符号方法，将开放式自然语言与离散视觉验证相结合。具体而言，复杂的用户查询被编译成正式的时序逻辑规范，以系统性地将自然语言问题分解为所需原子视觉原语的明确检查清单，例如关键对象和活动，以及它们的时间顺序。这些系统推导的基础事件为 RL 后训练提供了关键的反馈信号，使得基于成功检索这些特定视觉元素的密集、可验证奖励成为可能，而不是完全依赖于结果导向的答案准确性。通过明确优化这种可验证的证据寻求行为，VSeek 在长视频理解基准测试中相比基础模型提高了高达 8% 的 Pass@1 分数和 15% 的 Pass@4 分数。我们在 https://utaustin-swarmlab.github.io/VSeek 上开源了我们的代码。

View on arXiv Download PDF AI Translation

cs.CV / 72 / 2607.02961

Overloading Large Vision-Language Models for Jailbreaking

对大型视觉-语言模型进行过载以实现越狱

Zhang, Haoyu, Guo, Yangyang, Kankanhalli, Mohan

Abstract

Large Vision-Language Models (LVLMs) exhibit remarkable vision-language capabilities and are increasingly deployed in real-world applications such as personal assistants, document analysis systems, and embodied agents. However, their dual-modal attack surfaces make them vulnerable to jailbreak attacks. Existing LVLM jailbreaks rely on simple designs, e.g., short text and out-of-distribution images. Nevertheless, recent advancements in both large language model backbones and multimodal mechanisms undermine these attacks, particularly their transferability among model architectures. To overcome this limitation, we propose a novel information overloading method that is equipped with both extensive text and multi-dimensional image attacks. These components are arranged in recursion-based image-typography layouts to exponentially increase multimodal information complexity. This overloading approach amplifies the cross-modal processing required, which undermines the safety alignment in LVLMs. Extensive experiments on both open-sourced and commercial LVLMs establish our method as a new state-of-the-art LVLM jailbreak attack. On open-source models, our method achieves an average ASR of 88.6%; on commercial LVLMs, it reaches an average ASR of 84.0%, exceeding the best baseline by 48.7%. Moreover, our prompts optimized on open-source surrogate models transfer effectively across model families. Beyond empirical results, we probe the safety-critical information flows within victim LVLMs. Our observations reveal that complex image-typography compositions induce intensified cross-modal processing and reduce the model's certainty in generating refusal responses. Together, these findings highlight information overloading as a practical and emerging safety risk for real-world LVLM deployments, underscoring the need for stronger defenses against complex multimodal jailbreak inputs.

Chinese Translation

大型视觉-语言模型（LVLMs）展现出卓越的视觉-语言能力，并越来越多地应用于个人助手、文档分析系统和具身代理等现实世界应用中。然而，它们的双模态攻击面使其容易受到越狱攻击。现有的LVLM越狱方法依赖于简单的设计，例如短文本和分布外图像。然而，最近在大型语言模型骨干和多模态机制方面的进展削弱了这些攻击，特别是在模型架构之间的可转移性。为克服这一限制，我们提出了一种新颖的信息过载方法，该方法结合了广泛的文本和多维图像攻击。这些组件以递归基础的图像-排版布局排列，以指数级增加多模态信息的复杂性。这种过载方法放大了所需的跨模态处理，从而削弱了LVLMs中的安全对齐。对开源和商业LVLMs进行的大量实验表明，我们的方法成为了一种新的最先进的LVLM越狱攻击。在开源模型上，我们的方法实现了88.6%的平均成功率；在商业LVLMs上，达到了84.0%的平均成功率，超过了最佳基线48.7%。此外，我们在开源替代模型上优化的提示在不同模型家族之间有效转移。除了经验结果外，我们还探讨了受害LVLMs中的安全关键的信息流。我们的观察表明，复杂的图像-排版组合会引发加强的跨模态处理，并降低模型在生成拒绝响应时的确定性。这些发现共同强调了信息过载作为现实世界LVLM部署中一种实际且新兴的安全风险，突显了对复杂多模态越狱输入需要更强防御的必要性。

View on arXiv Download PDF AI Translation

cs.CV / 73 / 2607.02963

Parallelized Autoregressive Decoding for Omni-Modal Dense Video Captioning

面向全模态密集视频字幕生成的并行自回归解码

Zeng, Wenzheng, Jiao, Siyi, Gao, Chen, Ng, Hwee Tou, Shou, Mike Zheng

Abstract

Dense video captioning aims to generate temporally grounded descriptions of video events, benefiting both event-level video understanding and generation. In this domain, autoregressive video large language models have emerged as a prevalent paradigm due to their strong generative and cross-modal modeling capacity. However, generating dense captions under the token-by-token paradigm severely limits inference efficiency and hinders scalability as video length and event density increase. In this work, we propose a parallelized autoregressive framework that not only improves generation efficiency but also enhances temporally grounded captioning performance. Our key insight is to exploit the weak local dependencies across temporally distinct events to restructure the causal dependency graph, thereby enabling lossless parallel generation. Specifically, tokens with weak cross-event dependencies can be decoded in parallel, while tightly coupled tokens within each event retain sequential decoding to preserve local semantic coherence. To realize this insight, we introduce two key components for lossless parallel decoding: (1) a latent global planning mechanism that automatically learns the event-level structure and produces compact tokens encoding global inter-event causality while adaptively aggregating event-level audio-visual semantics, guiding subsequent dependency restructuring and parallel decoding; and (2) an event-factorized parallel decoding mechanism that effectively balances local focus with global inter-event awareness. Experiments on various benchmarks demonstrate the clear advantage of our approach in both efficiency and performance in omni-modal event grounding and captioning. Project website: https://github.com/showlab/PadCaptioner.

Chinese Translation

密集视频字幕生成旨在生成与视频事件时间上相关的描述，这对事件级视频理解和生成都有所裨益。在这一领域，自回归视频大型语言模型因其强大的生成能力和跨模态建模能力而成为一种普遍的范式。然而，在逐个标记生成的模式下生成密集字幕严重限制了推理效率，并在视频长度和事件密度增加时阻碍了可扩展性。在本研究中，我们提出了一种并行自回归框架，不仅提高了生成效率，还增强了时间上相关的字幕生成性能。我们的关键见解是利用时间上不同事件之间的弱局部依赖关系来重构因果依赖图，从而实现无损并行生成。具体而言，具有弱跨事件依赖的标记可以并行解码，而每个事件内紧密耦合的标记则保留顺序解码，以保持局部语义一致性。为了实现这一见解，我们引入了两个无损并行解码的关键组件：(1) 一种潜在的全局规划机制，能够自动学习事件级结构，并生成编码全局事件因果关系的紧凑标记，同时自适应地聚合事件级的视听语义，指导后续的依赖重构和并行解码；(2) 一种事件因子化的并行解码机制，有效平衡局部关注与全局事件间意识。各种基准测试的实验结果表明，我们的方法在全模态事件定位和字幕生成的效率和性能上具有明显优势。项目网站：https://github.com/showlab/PadCaptioner。

View on arXiv Download PDF AI Translation

cs.CV / 74 / 2607.02968

Awakening Diffusion Transformers: Eliciting Stronger Generation and Understanding via Massive Activation Modulation

唤醒扩散变换器：通过大规模激活调制引发更强的生成与理解

Gan, Chaofan, Zhao, Zicheng, Tu, Yuanpeng, Chen, Xi, Qin, Ziran, Chen, Tieyuan, Aramvith, Supavadee, Harandi, Mehrtash, Lin, Weiyao

Abstract

Massive Activations (MAs) have been widely observed in Transformer-based models, yet their structure and functional roles in Diffusion Transformers (DiTs) remain insufficiently understood. In this work, we systematically analyze MAs in representative DiTs and find that they are spatially distributed across image tokens while concentrated in a small set of fixed feature dimensions. We further show that these dimensions are closely aligned with AdaLN residual scaling factors and are primarily modulated by the denoising timestep rather than text conditions. This structure leads to two task-dependent effects: for generation, MAs are critical for fine-grained detail synthesis while having limited influence on global semantics; for understanding, their shared high-magnitude directions make raw DiT features overly similar across spatial tokens and weaken dense feature discrimination. Based on these findings, we introduce Eliciting Massive Activation (EMA), a training-free framework that leverages Massive Activations (MAs) as a unified modulation signal to improve both generative and representational capabilities of DiTs. For generation, EMA proposes MA-driven Detail G}uidance (DG), which suppresses MA dimensions to construct a detail-deficient counterfactual prediction and guides sampling toward finer visual details. DG further supports efficient partial-forward inference, integration with classifier-free guidance, and token-level Local DG for refining selected image regions. For understanding, EMA introduces MA-modulated REPresentation extraction (MREP), which uses pretrained AdaLN channel-wise modulation to reduce MA directional dominance and concatenates spatially normalized MA maps to preserve useful spatial structure. Extensive experiments demonstrate that EMA consistently improves both the generation quality and representation capability of DiTs.

Chinese Translation

大规模激活（MAs）在基于变换器的模型中被广泛观察到，但其在扩散变换器（DiTs）中的结构和功能角色仍然不够清晰。在本研究中，我们系统地分析了代表性DiTs中的MAs，发现它们在图像标记中空间分布，同时集中于一小组固定的特征维度。我们进一步表明，这些维度与AdaLN残差缩放因子密切相关，并主要受去噪时间步的调制，而非文本条件。这种结构导致了两个任务依赖的效果：在生成方面，MAs对细粒度细节合成至关重要，但对全局语义的影响有限；在理解方面，它们共享的高幅度方向使得原始DiT特征在空间标记间过于相似，从而削弱了密集特征的区分能力。基于这些发现，我们提出了唤醒大规模激活（EMA），这是一个无训练的框架，利用大规模激活（MAs）作为统一的调制信号，以提高DiTs的生成和表征能力。在生成方面，EMA提出了基于MA的细节引导（DG），通过抑制MA维度来构建缺乏细节的反事实预测，并引导采样朝向更细致的视觉细节。DG进一步支持高效的部分前向推理、与无分类器引导的集成，以及针对选定图像区域的标记级局部DG。在理解方面，EMA引入了MA调制的表征提取（MREP），利用预训练的AdaLN通道级调制来减少MA方向主导性，并将空间归一化的MA图拼接以保留有用的空间结构。大量实验表明，EMA持续改善了DiTs的生成质量和表征能力。

View on arXiv Download PDF AI Translation

cs.CV / 75 / 2607.02986

RePos: Relative-to-Absolute Output Factorization for Cross-Environment WiFi-Based 3D Human Pose Estimation

RePos：基于相对到绝对输出分解的跨环境WiFi 3D人体姿态估计

Hou, Zhangcheng, Ohtsuki, Tomoaki

Abstract

Device-free 3D human pose estimation using commodity WiFi Channel State Information (CSI) enables privacy-preserving and illumination-robust human sensing, but its deployment is limited by poor cross-environment generalization. Unlike images, CSI measurements do not have a spatially localized correspondence to body parts and are heavily affected by multipath propagation, causing models that regress absolute poses to entangle body structure with environment-specific location cues. Within a single environment this coupling is benign: an end-to-end absolute-pose variant, RePos-D, already achieves state-of-the-art accuracy on Person-in-WiFi-3D (86.9 mm MPJPE, a 3.4% gain over the previous best WiFi method, DT-Pose). Across environments, however, the same model overfits position and suffers significant performance degradation. We therefore propose RePos, a factorized framework that separates root-relative pose estimation from root localization. By preventing absolute-pose supervision from affecting the structure branch, RePos learns environment-invariant pose representations. Specifically, it groups CSI features into body-part-aware latent tokens that skeleton-guided modeling refines into the pose, while a separate amplitude-based network estimates the root position through a differentiable spatial-decomposition module. Under the strict MM-Fi cross-environment protocol, RePos achieves MPJPEs of 254.4-296.1 mm, a 10-21% reduction over existing WiFi-based methods. The improvement remains consistent across activity protocols, leave-one-environment-out splits, and leakage-free few-shot transfer. Analysis of the learned features shows that relative-pose representations remain largely position-agnostic, while root localization retains environment dependence, indicating distinct generalization behavior for structure and localization.

Chinese Translation

基于商业WiFi信道状态信息（CSI）的无设备3D人体姿态估计能够实现隐私保护和光照鲁棒的人体感知，但其部署受到跨环境泛化能力差的限制。与图像不同，CSI测量与身体部位之间没有空间上的局部对应关系，并且受到多径传播的严重影响，这导致回归绝对姿态的模型将身体结构与特定环境的位置线索纠缠在一起。在单一环境中，这种耦合是良性的：一种端到端的绝对姿态变体RePos-D在Person-in-WiFi-3D数据集上已经达到了最先进的准确性（86.9 mm MPJPE，比之前最佳的WiFi方法DT-Pose提高了3.4%）。然而，在不同环境中，相同模型会过拟合位置，导致显著的性能下降。因此，我们提出了RePos，一个分解框架，将根相对姿态估计与根定位分开。通过防止绝对姿态监督影响结构分支，RePos学习环境不变的姿态表示。具体而言，它将CSI特征分组为对身体部位敏感的潜在标记，经过骨架引导建模精炼为姿态，而一个独立的基于幅度的网络通过可微分的空间分解模块估计根位置。在严格的MM-Fi跨环境协议下，RePos实现了254.4-296.1 mm的MPJPE，比现有的基于WiFi的方法减少了10-21%。这一改进在不同活动协议、留一环境分割和无泄漏的少样本迁移中保持一致。对学习特征的分析表明，相对姿态表示在很大程度上保持位置无关性，而根定位则保留了环境依赖性，表明结构和定位具有不同的泛化行为。

View on arXiv Download PDF AI Translation

cs.CV / 76 / 2607.02988

Cross-device Collaborative Test-time Adaptation with Zeroth-order Optimization and Model Merging

跨设备协作测试时适应的零阶优化与模型合并

Mitsuzumi, Yu, Kimura, Akisato, Fujiwara, Yasuhiro, Kashima, Hisashi

Abstract

Test-time adaptation (TTA) mitigates domain shifts by using incoming test data to update a model on the fly. The majority of TTA methods require resource-intensive backpropagation (BP) for model updates, particularly demanding large memory sizes, which makes it infeasible to deploy them on resource-limited devices (e.g., edge devices). To address this issue, we integrate two different techniques, zeroth-order optimization (ZOO) and model merging, under the recently established cross-device collaborative TTA (CDC-TTA) framework, where the system is composed of a mixture of resource-abundant and resource-limited devices, and the model information (e.g., model weights obtained on each device) is shared across the devices. Our method is executable on resource-limited devices by introducing ZOO, which requires only forward processing and bypasses the resource-intensive BP optimization. Concurrently, to mitigate the high-dimensional optimization difficulty caused by the side effect of ZOO, we incorporate model merging of the shared multiple models and set the merge coefficients as the optimization objective, which successfully reduces the optimization dimension. In addition, to enhance the synergistic combination of ZOO and model merging, we propose a unique preprocessing strategy that trims intra-model non-influential weights and reduces the inter-model information redundancy. We empirically confirmed the effectiveness of our method using common corruption and style-transferred image benchmarks.

Chinese Translation

测试时适应（TTA）通过使用传入的测试数据实时更新模型，以缓解领域转移。大多数TTA方法需要资源密集型的反向传播（BP）进行模型更新，尤其需要较大的内存，这使得在资源有限的设备（例如边缘设备）上部署这些方法变得不可行。为了解决这个问题，我们在最近建立的跨设备协作TTA（CDC-TTA）框架下整合了两种不同的技术：零阶优化（ZOO）和模型合并，该系统由资源丰富和资源有限的设备混合组成，并且模型信息（例如在每个设备上获得的模型权重）在设备之间共享。我们的方法通过引入ZOO，使其能够在资源有限的设备上执行，ZOO仅需前向处理，避免了资源密集型的BP优化。同时，为了缓解ZOO带来的高维优化难度，我们结合了共享多个模型的模型合并，并将合并系数设定为优化目标，从而成功降低了优化维度。此外，为了增强ZOO与模型合并的协同组合，我们提出了一种独特的预处理策略，修剪模型内部非影响性权重并减少模型间信息冗余。我们通过常见的图像损坏和风格迁移基准测试实证验证了我们方法的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 77 / 2607.02991

GuideMe: Multi-Domain Task Guidance and Intervention in Streaming Video

GuideMe：流媒体视频中的多领域任务指导与干预

Liu, Fang, Chen, Jinpeng, Xu, Ke, Liu, Yuhao, Guan, Huankang, Lu, Xudong, Yang, Bo, Hancke, Gerhard, Liu, Rui, Lau, Rynson W. H.

Abstract

While multimodal Large Language Models (MLLMs) excel at offline video understanding, an interesting question of how far they are from serving as a real-time procedural coach remains unknown. Such a role typically requires an MLLM to continuously monitor the execution, detect mistakes, and provide corrective guidance in a closed-loop interaction. In this paper, we construct GuideMe, the first multi-domain benchmark for streaming video that supports training and evaluation of MLLMs for closed-loop interactive task guidance. It comprises 2,458 videos spanning 223.7 hours across diverse domains (\eg, cooking, object manipulation, daily-life guidance, and fitness), with 47,775 interaction samples covering next-step instructions, completion feedback, error detection, and corrective guidance. To evaluate existing models on GuideMe, we design a three-component assessment framework to measure the capabilities of representative MLLMs, which consists of temporal-semantic bipartite matching for sequence-level alignment, behavioral classification for intervention timing, and LLM-as-a-Judge for content quality. Extensive experiments highlight a critical performance asymmetry: despite excelling at providing instructions, existing MLLMs consistently fail to identify execution errors and respond with corrective feedback. Code and data are released at https://fawnliu.github.io/project/guideme.

Chinese Translation

尽管多模态大型语言模型（MLLMs）在离线视频理解方面表现出色，但它们在作为实时程序教练的能力上还有多远尚不清楚。这种角色通常要求MLLM持续监控执行过程，检测错误，并在闭环交互中提供纠正指导。本文构建了GuideMe，这是第一个支持多领域流媒体视频的基准，旨在训练和评估MLLM在闭环交互任务指导中的表现。该基准包含2,458个视频，涵盖223.7小时，涉及多种领域（例如，烹饪、物体操作、日常生活指导和健身），并提供47,775个交互样本，涵盖下一步指令、完成反馈、错误检测和纠正指导。为了评估现有模型在GuideMe上的表现，我们设计了一个由三个组成部分构成的评估框架，以测量代表性MLLM的能力，该框架包括用于序列级对齐的时间-语义二分匹配、用于干预时机的行为分类，以及用于内容质量的LLM作为评判者。大量实验突显了一个关键的性能不对称：尽管在提供指令方面表现优异，现有的MLLM在识别执行错误和回应纠正反馈方面却始终存在不足。代码和数据已发布在 https://fawnliu.github.io/project/guideme。

View on arXiv Download PDF AI Translation

cs.CV / 78 / 2607.02995

VISTA: Auditing Semantic Divergence in Vision-Language Models

VISTA：审计视觉语言模型中的语义偏差

Liao, Junchi, Deng, Jiawen, Ren, Fuji

Abstract

Vision-language models can exhibit visual concept-conditioned divergence: given images containing demographic features, corporate logos, or ideological symbols, some models produce unusually uniform responses that differ from what peer models say about the same input. These behaviors evade text-only audits because visual concepts cannot be isolated or substituted the way text tokens can. We present VISTA (Visual Inconsistency Screening Through Analysis), a black-box cross-model audit that couples semantic entropy with distribution-based divergence to flag model-specific anomalies. In a controlled study, we implant concept-conditioned stances in three VLMs via fine-tuning on small biased datasets and confirm that VISTA detects them. Auditing six VLMs across 19 topics, VISTA surfaces 142 high-suspicion cases (1.2%) and identifies selective refusal as a previously unreported divergence pattern, where models refuse demographic queries at rates varying from 0 to 65% across groups.

Chinese Translation

视觉语言模型可能表现出视觉概念条件下的偏差：在包含人口特征、企业标志或意识形态符号的图像中，一些模型产生的响应异常一致，与同行模型对相同输入的反应存在差异。这些行为无法通过仅文本的审计来识别，因为视觉概念无法像文本标记那样被孤立或替换。我们提出了VISTA（通过分析进行视觉不一致性筛查），这是一种黑箱跨模型审计方法，将语义熵与基于分布的偏差结合起来，以标记模型特定的异常。在一项控制研究中，我们通过在小型偏见数据集上进行微调，在三个视觉语言模型中植入概念条件的立场，并确认VISTA能够检测到这些立场。在对19个主题的六个视觉语言模型进行审计时，VISTA发现了142个高疑点案例（1.2%），并识别出选择性拒绝作为一种先前未报告的偏差模式，其中模型对不同群体的人口查询的拒绝率从0%到65%不等。

View on arXiv Download PDF AI Translation

cs.CV / 79 / 2607.02998

CONFLUX: A Latent Diusion Model for 3D Chest-CT Synthesis with RL Post-Training

CONFLUX：一种用于3D胸部CT合成的潜在扩散模型与强化学习后训练

Van Puyvelde, Max, Gulluk, Halil Ibrahim, Van Criekinge, Wim, Gevaert, Olivier

Abstract

Controllable generative models of 3D medical images can synthesize volumes with specified clinical attributes, but this demands samples that are simultaneously high-fidelity, natively 3D, and faithful to the requested conditioning. We present CONFLUX, a latent diffusion model for chest computed tomography (CT): a 3D variational autoencoder compresses each volume, and a rectified-flow transformer generates in the latent space. Generation is conditioned on structured radiological metadata (18 abnormality findings, sex, age, and reconstruction kernel) through adaptive layer normalization. The model leads strong volumetric baselines on tri-planar Frechet distance (FID 32.3 vs. 74.6 for MAISI) while exposing direct control over clinical attributes. To strengthen that control we add an online reinforcement-learning post-training stage (group-relative policy optimization) that rewards how reliably a classifier recovers the requested findings from each generated volume. Judged by a separate, independent classifier, post-training removes 47% of the shortfall relative to real-scan reliability. We release the model and a ~200k synthetic chest-CT dataset with conditioning metadata spanning a wide variety of clinical findings.

Chinese Translation

可控的3D医学图像生成模型能够合成具有特定临床属性的体积，但这要求样本同时具备高保真度、原生3D特性，并忠实于请求的条件。我们提出了CONFLUX，一种用于胸部计算机断层扫描（CT）的潜在扩散模型：一个3D变分自编码器压缩每个体积，而一个修正流变换器在潜在空间中生成。生成过程通过自适应层归一化以结构化放射学元数据（18种异常发现、性别、年龄和重建核）为条件。该模型在三平面Frechet距离（FID 32.3，相较于MAISI的74.6）上表现出强大的体积基线，同时对临床属性提供直接控制。为了增强这种控制，我们增加了一个在线强化学习后训练阶段（组相对策略优化），该阶段奖励分类器从每个生成的体积中可靠恢复请求发现的能力。通过一个独立的分类器评估，后训练相较于真实扫描的可靠性消除了47%的短缺。我们发布了该模型及一个约20万的合成胸部CT数据集，包含涵盖广泛临床发现的条件元数据。

View on arXiv Download PDF AI Translation

cs.CV / 80 / 2607.03004

REAL-OW: Rehearsal-free Open World Object Detection with Low-Rank Adaptation and Dual-Stage Objectness Modeling

REAL-OW：基于低秩适应和双阶段物体性建模的无重演开放世界物体检测

Zhang, Huazhong, Fu, Xiaowen, Zhang, Yang, Shen, Linlin, Wang, Jinbao

Abstract

Open-World Object Detection (OWOD) requires detectors to identify previously unseen objects as unknown and incrementally incorporate them into the set of known categories, while preserving previously acquired knowledge. Existing frameworks rely heavily on exemplar replay to mitigate catastrophic forgetting, but in some real applications, storing raw data conflicts with data access restrictions and leads to data exposure risks, while incurring significant memory overhead. In this paper, we propose REAL-OW, a novel rehearsal-free framework that decouples incremental knowledge through a collaborative adapter architecture based on Low-Rank Adaptation (LoRA). Specifically, we deploy General Adapters (GAs) in the backbone to enable the significance-aware refinement of cross-task universal representations, while Specific Adapters (SAs) in the decoder provide orthogonal storage for task-specific expertise. To resolve representation drift in objectness modeling under rehearsal-free constraints, we introduce Dual-Stage Objectness Modeling (DSOM), which alternates between feature aggregation and boundary consolidation to stabilize objectness distributions while maintaining the separation between known and unknown categories. Furthermore, DSOM is supported by a Calibrated Gaussian Negative Log-Likelihood (CG-NLL) distance tailored for the dispersed feature distributions inherent in rehearsal-free settings. Extensive evaluations demonstrate that REAL-OW achieves state-of-the-art performance, surpassing existing exemplar replay methods in both detection precision and unknown discovery. Our approach establishes a new baseline for rehearsal-free OWOD.

Chinese Translation

开放世界物体检测（OWOD）要求检测器将之前未见过的物体识别为未知，并逐步将其纳入已知类别的集合，同时保持之前获得的知识。现有框架在很大程度上依赖于示例重放来减轻灾难性遗忘，但在某些实际应用中，存储原始数据与数据访问限制相冲突，并导致数据泄露风险，同时带来显著的内存开销。本文提出了REAL-OW，一种新颖的无重演框架，通过基于低秩适应（LoRA）的协作适配器架构解耦增量知识。具体而言，我们在主干网络中部署通用适配器（GAs），以实现对跨任务通用表示的显著性感知细化，而在解码器中使用特定适配器（SAs）提供任务特定专业知识的正交存储。为了解决在无重演约束下物体性建模中的表示漂移问题，我们引入了双阶段物体性建模（DSOM），该方法在特征聚合和边界巩固之间交替进行，以稳定物体性分布，同时保持已知类别和未知类别之间的分离。此外，DSOM还得到了针对无重演环境中固有的分散特征分布量身定制的校准高斯负对数似然（CG-NLL）距离的支持。广泛的评估表明，REAL-OW在检测精度和未知发现方面超越了现有的示例重放方法，达到了最先进的性能。我们的方法为无重演的开放世界物体检测建立了一个新的基准。

View on arXiv Download PDF AI Translation

cs.CV / 81 / 2607.03006

PosterHarness: Turning Scientific Poster Generation into an Auditable Instruction-Following Benchmark

PosterHarness：将科学海报生成转变为可审计的指令遵循基准

Yang, Tianyi, Fu, Dawei, Wu, Youpeng, Kou, Zixun, Chen, Linrui, Jiang, Ruobing, Wang, Zijian, Li, Qiang

Abstract

Text-rich image models can now design poster-scale layouts, but we lack ways to measure whether they honor scientific communication contracts: legible labels, prescribed aspect ratios, and -- above all -- abstaining from fabricated scientific figures. We present POSTERHARNESS, an auditable harness reframing poster generation as measurable instruction-following tasks, with a pilot benchmark and failure taxonomy. POSTERHARNESS uses a placeholder-first contract to separate two jobs models otherwise conflate. The model performs visual-summary design: typography, reading path, color, and background -- but never draws data-bearing figures. Every figure region must be an empty labeled placeholder; a deterministic compositor inserts real source-paper figures at detected coordinates. This makes properties measurable: placeholder count and ID accuracy, blankness, aspect-ratio compliance, abstention from synthesized graphics, public-text hygiene, and source-figure provenance -- with failures logged as explicit rejections, not hidden in plausible-looking output. We instantiate the harness on 12 papers (6 HEP, 6 AI/ML-adjacent) and report three findings. (i) A counterfactual probe shows the placeholder contract drives VLM-counted synthesized figures from 34 to 0 across three papers. (ii) A failure taxonomy identifies blocking contracts: placeholder geometry, placeholder QA, template critic, and public text. (iii) Comparison with Paper2Poster shows a trade-off: PosterHarness yields higher-resolution artifacts, lower white-canvas fraction, and stronger VLM visual preference; the deterministic baseline retains slightly more PosterQuiz-style information and runs faster. We report this as regime characterization, not a superiority claim. All artifacts, prompts, manifests, and audit scripts are released as a reusable evaluation component.

Chinese Translation

文本丰富的图像模型现在能够设计海报规模的布局，但我们缺乏衡量它们是否遵循科学传播契约的方法：可读的标签、规定的纵横比，以及最重要的——避免伪造的科学图形。我们提出了POSTERHARNESS，一个可审计的工具，将海报生成重新构建为可测量的指令遵循任务，并提供了一个初步基准和失败分类。POSTERHARNESS使用占位符优先契约来分离模型通常混淆的两个任务。该模型执行视觉摘要设计：排版、阅读路径、颜色和背景——但从不绘制数据承载图形。每个图形区域必须是一个空的标记占位符；一个确定性的合成器在检测到的坐标处插入真实的源论文图形。这使得属性可测量：占位符数量和ID准确性、空白度、纵横比合规性、避免合成图形、公共文本卫生和源图形来源——失败以明确拒绝的形式记录，而不是隐藏在看似合理的输出中。我们在12篇论文（6篇高能物理，6篇与人工智能/机器学习相关）上实例化了该工具，并报告了三项发现。(i) 一项反事实探测显示，占位符契约将VLM计数的合成图形从34减少到0，跨越三篇论文。(ii) 一项失败分类识别了阻塞契约：占位符几何、占位符质量保证、模板批评和公共文本。(iii) 与Paper2Poster的比较显示了一种权衡：PosterHarness产生更高分辨率的工件、更低的白色画布比例和更强的VLM视觉偏好；而确定性基线保留了稍多的PosterQuiz风格信息，并运行得更快。我们将此报告视为制度特征描述，而非优越性声明。所有工件、提示、清单和审计脚本均作为可重用的评估组件发布。

View on arXiv Download PDF AI Translation

cs.CV / 82 / 2607.03012

HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusion

HyperVAttention：用于视频扩散的高效稀疏注意力与时空聚类

Lee, Dongyeun, Zandieh, Amir, Mirrokni, Vahab, Kim, Junmo, Han, Insu

Abstract

Video Diffusion Transformers (VDiTs) have demonstrated significant capabilities in high-fidelity video generation. However, their ability to produce long-duration videos is fundamentally constrained by the quadratic complexity of the self-attention mechanism. Recent clustering-based sparse attention methods improve the quality-speed trade-off by grouping semantically similar tokens, but their practical efficiency remains limited by two bottlenecks: substantial clustering overhead and low CTA utilization caused by irregular cluster-induced blocks. We propose HyperVAttention (HVA), a training-free sparse attention framework that addresses both bottlenecks jointly. To reduce clustering overhead, we introduce 3D local-window clustering, which exploits the spatio-temporal locality of video tokens to restrict centroid search to fixed local neighborhoods, and implement it with a custom Triton kernel for efficient execution. We further propose a hybrid clustering strategy that performs full clustering only at anchor steps and updates only subset tokens at intermediate steps, leveraging the temporal stability of cluster assignments across denoising steps. To improve CTA utilization, we present hardware-aware cluster merging that minimizes CTA-aligned execution cost through parallel agglomerative merging, improving block density and approximation fidelity by utilizing idle tile capacity. Together, these components reduce clustering overhead, avoid redundant updates, and better align sparse attention with the fixed tile structure of modern GPU kernels. Experiments on Text-to-Video generation show that HVA establishes a new Pareto frontier for training-free sparse attention in video diffusion, reducing end-to-end latency by up to $2.13\times$ while improving fidelity over existing training-free sparse attention baselines.

Chinese Translation

视频扩散变换器（VDiTs）在高保真视频生成方面展示了显著的能力。然而，它们生成长时间视频的能力受到自注意力机制的平方复杂度的根本限制。最近基于聚类的稀疏注意力方法通过对语义相似的标记进行分组，提高了质量与速度的权衡，但其实际效率仍受限于两个瓶颈：大量的聚类开销和由于不规则聚类引起的低CTA（Clustered Token Attention）利用率。我们提出了HyperVAttention（HVA），一个无训练的稀疏注意力框架，旨在共同解决这两个瓶颈。为了减少聚类开销，我们引入了3D局部窗口聚类，利用视频标记的时空局部性将质心搜索限制在固定的局部邻域，并通过自定义的Triton内核实现高效执行。我们进一步提出了一种混合聚类策略，仅在锚点步骤进行全聚类，并在中间步骤仅更新子集标记，利用去噪步骤中聚类分配的时间稳定性。为了提高CTA利用率，我们提出了硬件感知的聚类合并，通过并行聚合合并最小化与CTA对齐的执行成本，提高了块密度和近似保真度，充分利用空闲的瓦片容量。综合这些组件，我们减少了聚类开销，避免了冗余更新，并更好地将稀疏注意力与现代GPU内核的固定瓦片结构对齐。在文本到视频生成的实验中，HVA为无训练的稀疏注意力在视频扩散中建立了新的帕累托前沿，将端到端延迟减少了多达$2.13 imes$，同时提高了相较于现有无训练稀疏注意力基线的保真度。

View on arXiv Download PDF AI Translation

cs.CV / 83 / 2607.03013

MambaLIE: Scene Light Intensity-Boosted Low-Light Image Enhancement with State Space Model

MambaLIE：基于状态空间模型的场景光强增强低光图像增强

Fan, Wanshu, Li, Xiangyu, Wang, Cong, Lam, Kin-man, Yang, Xin, Zhang, Haiyan, Zhou, Dongsheng

Abstract

Images captured by consumer electronic devices, such as mobile phones and digital cameras, often suffer from low-light degradation due to sensor limitations and imaging pipelines, which degrades visual quality and affects downstream vision tasks. Existing methods based on Convolutional Neural Networks (CNNs) and Transformers have dominated current low-light image enhancement (LIE) due to their excellent ability to model hierarchical features. However, CNNs operate in local receptive fields that cannot model long-range dependencies, while Transformers overcome this problem but incur substantial computational costs. To address these challenges, we propose MambaLIE, a Scene Light Intensity-Boosted Low-Light Image Enhancement method based on a State Space Model (SSM). We first introduce scene light intensity to improve the structural distribution of illumination, which is then gated with the low-light input to guide enhancement. To better model the illumination while maintaining computational efficiency, we propose the Locally Enhanced State Space Model (LESSM) for efficient light enhancement. Our LESSM contains two branches: an SSM branch and a Local Enhanced branch, where the former is used to model the long-range dependencies with linear time complexity, while the latter is used to enhance local feature representations. Extensive experiments demonstrate that MambaLIE outperforms state-of-the-art CNN-based and Transformer-based LIE methods on four widely used synthetic benchmarks and five publicly available real-world benchmarks in terms of accuracy, speed, and model size, making it suitable for practical deployment on resource-constrained devices.

Chinese Translation

消费电子设备（如手机和数码相机）捕获的图像常常由于传感器限制和成像流程而遭受低光降解，这降低了视觉质量并影响下游视觉任务。现有基于卷积神经网络（CNN）和变换器（Transformers）的方法在当前的低光图像增强（LIE）中占据主导地位，因其出色的层次特征建模能力。然而，CNN在局部感受野内操作，无法建模长距离依赖关系，而变换器虽然克服了这一问题，但却带来了巨大的计算成本。为了解决这些挑战，我们提出了MambaLIE，一种基于状态空间模型（SSM）的场景光强增强低光图像增强方法。我们首先引入场景光强，以改善照明的结构分布，然后与低光输入进行门控，以指导增强。为了更好地建模照明并保持计算效率，我们提出了局部增强状态空间模型（LESSM）以实现高效的光增强。我们的LESSM包含两个分支：一个SSM分支和一个局部增强分支，其中前者用于以线性时间复杂度建模长距离依赖关系，而后者用于增强局部特征表示。大量实验表明，MambaLIE在准确性、速度和模型大小方面优于四个广泛使用的合成基准和五个公开可用的真实世界基准上的最先进的基于CNN和变换器的LIE方法，使其适合在资源受限设备上进行实际部署。

View on arXiv Download PDF AI Translation

cs.CV / 84 / 2607.03018

$C^3$ASD: Multi-Level Consistency-Driven Representation Learning

$C^3$ASD：多层次一致性驱动的表示学习

Hong, Jin, Park, Jisoo, Kwon, Junseok

Abstract

Active Speaker Detection determines whether a visible person in a video is speaking at each moment. While recent audio-visual fusion methods perform well on clean data, they degrade under real-world corruptions such as background noise, occlusion, or simultaneous modality degradation. We attribute this limitation to the absence of explicit consistency constraints that promote robust, semantically aligned representations across modalities. Without such guidance, models tend to learn fragile modality-specific shortcuts that fail under corrupted conditions. We propose $C^3$ASD, a multi-level consistency-driven framework with three complementary constraints: embedding-level inter-modality consistency aligns audio-visual representations during speech; sequence-level intra-modality consistency separates speaking and non-speaking clusters via track-aware contrastive learning; and prediction-level consistency stabilizes fusion through knowledge distillation. Extensive experiments demonstrate significant improvements under diverse audio, visual and joint corruptions, while maintaining competitive performance on clean data.

Chinese Translation

主动说话者检测旨在判断视频中可见的人在每个时刻是否在说话。尽管最近的音视频融合方法在干净数据上表现良好，但在现实世界中的干扰（如背景噪声、遮挡或同时模态退化）下性能下降。我们将这一限制归因于缺乏明确的一致性约束，这些约束促进了跨模态的鲁棒且语义对齐的表示。在没有这种指导的情况下，模型往往倾向于学习脆弱的模态特定捷径，而在受损条件下失效。我们提出了$C^3$ASD，一个多层次一致性驱动的框架，具有三个互补的约束：嵌入级别的跨模态一致性在说话期间对齐音视频表示；序列级别的模态内一致性通过轨迹感知对比学习分离说话和非说话聚类；预测级别的一致性通过知识蒸馏稳定融合。大量实验表明，在多种音频、视觉和联合干扰下显著改善，同时在干净数据上保持竞争性能。

View on arXiv Download PDF AI Translation

cs.CV / 85 / 2607.03038

OmniDS: Dual-Stream Context Fusion for Omnidirectional Depth from Fisheye Cameras

OmniDS：用于鱼眼相机的双流上下文融合全向深度估计

Park, Chaesong, Hwang, Jihyeon, Sung, Muyeol, Lim, Jongwoo

Abstract

Omnidirectional depth estimation from multi-fisheye camera rigs is complicated by visibility conflicts: wide baselines cause different cameras to observe different portions, or even different faces, of the same object, so aggregating their features into a unified equirectangular (ERP) representation under fixed projection produces ambiguous matching evidence near occlusion boundaries and thin structures. Although existing methods mitigate this by down-weighting unreliable views, they do not resolve the underlying discrepancy because context formation and cross-view fusion remain tied to rigid fisheye-to-ERP sampling. We present OmniDS, an iterative depth refinement framework that replaces rigid aggregation by combining dynamic context fusion with consensus-aware multi-view similarity. A dual-stream encoder pairs a lightweight CNN for geometric detail with a frozen DINOv3 for semantic priors; their features are reprojected into ERP space at each refinement step via learned view weighting and deformable cross-attention with geometric distortion bias. In parallel, a multi-view consensus volume captures global cross-camera agreement through group-wise correlation and feature variance, regularized by a 3D U-Net. For efficient deployment, we distill the dual-stream representation into a single MobileNet-based encoder. OmniDS achieves state-of-the-art performance on the OmniThings, OmniHouse, and Sunny benchmarks while maintaining competitive inference speed. Project page and codes are available at https://parkchaesong.github.io/omnids.

Chinese Translation

来自多鱼眼相机阵列的全向深度估计受到可见性冲突的影响：宽基线导致不同相机观察到同一物体的不同部分，甚至不同面，因此在固定投影下将它们的特征聚合到统一的等矩形（ERP）表示中，会在遮挡边界和细结构附近产生模糊的匹配证据。尽管现有方法通过降低不可靠视图的权重来缓解这一问题，但它们并未解决根本的不一致性，因为上下文形成和跨视图融合仍然与刚性鱼眼到ERP的采样相关联。我们提出了OmniDS，这是一种迭代深度精炼框架，通过结合动态上下文融合和共识感知的多视图相似性，替代了刚性聚合。双流编码器将轻量级卷积神经网络（CNN）用于几何细节，与冻结的DINOv3用于语义先验相结合；它们的特征在每个精炼步骤中通过学习的视图加权和带有几何失真偏差的可变形跨注意力重新投影到ERP空间。与此同时，多视图共识体积通过组相关性和特征方差捕捉全局跨相机一致性，并由3D U-Net进行正则化。为了高效部署，我们将双流表示提炼为单个基于MobileNet的编码器。OmniDS在OmniThings、OmniHouse和Sunny基准测试中实现了最先进的性能，同时保持了竞争力的推理速度。项目页面和代码可在 https://parkchaesong.github.io/omnids 获取。

View on arXiv Download PDF AI Translation

cs.CV / 86 / 2607.03043

Natural Language Camera Movement Understanding

自然语言摄像机运动理解

Tan, Yuwen, Huang, Joey, Huang, Jin, Li, Haoxiang, Gong, Boqing

Abstract

Understanding camera movement in natural language is critical for training and evaluating video generation models, among other applications. However, we demonstrate that existing vision-language models (VLMs) fail this task in surprising ways, frequently confusing translation with rotation, left with right, and object movement with camera movement. To address these limitations, we establish natural language camera movement understanding as a standalone research task. We introduce a two-level cinematographic taxonomy and an extensive, atomic benchmark featuring both real and synthetic videos. Furthermore, we curate a large-scale, multi-source training set enhanced by targeted camera movement augmentation. Our fine-tuned VLM-8B outperforms Gemini 3.1 Pro by 10% and 11% on our benchmark's real and synthetic videos, respectively. Despite these gains, a significant gap remains relative to human performance, underscoring the need to promote and facilitate future research on natural language camera movement understanding.

Chinese Translation

理解自然语言中的摄像机运动对于训练和评估视频生成模型等应用至关重要。然而，我们展示了现有的视觉语言模型（VLMs）在这一任务中存在意想不到的失败，常常将平移与旋转、左与右以及物体运动与摄像机运动混淆。为了解决这些局限性，我们将自然语言摄像机运动理解确立为一个独立的研究任务。我们引入了一个两级的电影分类法以及一个包含真实和合成视频的广泛原子基准。此外，我们策划了一个大规模的多源训练集，并通过针对性的摄像机运动增强进行了优化。我们微调的 VLM-8B 在我们基准的真实和合成视频上分别超越了 Gemini 3.1 Pro 10% 和 11%。尽管取得了这些进展，但与人类表现相比仍存在显著差距，这突显了促进和推动未来自然语言摄像机运动理解研究的必要性。

View on arXiv Download PDF AI Translation

cs.CV / 87 / 2607.03044

CURE: Controllable Unified Image Restoration for Complex Degradations

CURE：用于复杂退化的可控统一图像恢复

Kim, Boseong, Cho, Donghyeon

Abstract

The presence of composite degradations poses a significant challenge, since the underlying corruption factors exhibit complex and interdependent interactions. Even when the degradation types are known, accurately restoring the image remains difficult due to the intertwined nature of their effects and the need for selective control during the recovery process. To address this, we introduce CURE, a unified framework that enables controllable restoration in complex degradation settings by learning disentangled and adjustable representations. CURE is driven by four complementary objectives. First, an identity embedding is incorporated, along with a reconstruction constraint, to ensure that the model can reproduce the input image when restoration is unnecessary. Second, the ratio control mechanism blends the identity embedding with degradation-specific embeddings using user-regulated mixing ratios, allowing continuous control over restoration intensity. Third, an intermediate loss is applied to supervise stepwise outputs, each encouraged to tackle the removal of only a single degradation factor within a composite mixture. Finally, a permutation-invariant loss ensures that the model achieves consistent restoration quality regardless of the order in which multiple degradations are addressed. Since CURE modifies only the training strategy and not the underlying network architecture, it can be seamlessly integrated into existing controllable restoration models. Experiments demonstrate that CURE delivers state-of-the-art performance on composite degradation benchmarks, while enabling both selective and jointly fused restoration through flexible modulation of embedding ratios. The code and dataset are available at https://github.com/bo-oseng/CURE.

Chinese Translation

复合退化的存在带来了显著的挑战，因为潜在的损坏因素表现出复杂和相互依赖的相互作用。即使退化类型已知，由于其影响的交织性质以及在恢复过程中需要选择性控制，准确恢复图像仍然困难。为了解决这一问题，我们提出了CURE，一个统一框架，通过学习解耦和可调的表示，能够在复杂退化环境中实现可控恢复。CURE由四个互补目标驱动。首先，结合重建约束引入身份嵌入，以确保模型在不需要恢复时能够重现输入图像。其次，比例控制机制使用用户调节的混合比例，将身份嵌入与特定退化的嵌入混合，从而允许对恢复强度进行连续控制。第三，应用中间损失来监督逐步输出，每个输出被鼓励仅处理复合混合中的单一退化因素的去除。最后，置换不变损失确保模型在处理多个退化的顺序无论如何都能实现一致的恢复质量。由于CURE仅修改训练策略而不改变底层网络架构，因此可以无缝集成到现有的可控恢复模型中。实验表明，CURE在复合退化基准测试中提供了最先进的性能，同时通过灵活调节嵌入比例实现选择性和联合融合恢复。代码和数据集可在 https://github.com/bo-oseng/CURE 获取。

View on arXiv Download PDF AI Translation

cs.CV / 88 / 2607.03046

Text-to-Image Generation for Projector-Camera System Registration

投影仪-相机系统注册的文本到图像生成

Chen, Xinyu, Li, Yuqi, Li, Jiabao, Tang, Pinyan, Wang, Chong, Majumder, Aditi

Abstract

Establishing correspondence between projector and camera images in a procam (projector + camera) system is essential for achieving high-resolution pixel matching, referred to as procam registration. The highest accuracy is typically obtained using structured light patterns (e.g., stripes or blobs). However, these methods are often inefficient and lack meaningful information for human viewers. Although some have explored the use of natural images, these often fail to provide a sufficient distribution of features to achieve comparable accuracy. Additionally, existing methods struggle to cope with environmental factors such as surface textures and variations in brightness due to ambient light or changes in camera exposure. To address these limitations, we propose a method based on deep neural networks. Our approach aims to generate a single natural image from text-based prompts that not only appears realistic but also possesses rich spatial features to enhance registration accuracy in procam applications. We have developed a deep neural network trained on a synthesized dataset that simulates potential geometric and photometric distortions encountered in a procam system illuminating a relatively smooth object (see Figure 1). Our trained network predicts the correspondence between projector and camera images, significantly improving registration accuracy across various procam configurations. By jointly considering the naturalness and feature richness of the projector images, our method minimizes visual disruptions in projected content without sacrificing precision. A user study confirms that our technique enhances perceived naturalness and usability compared to existing methods, validating its practical utility in real-world applications.

Chinese Translation

在投影仪-相机（procam）系统中建立投影仪和相机图像之间的对应关系对于实现高分辨率的像素匹配至关重要，这被称为procam注册。通常，使用结构光模式（例如条纹或斑点）可以获得最高的准确性。然而，这些方法往往效率低下，并且缺乏对人类观察者有意义的信息。尽管一些研究探索了使用自然图像，但这些图像通常无法提供足够的特征分布以实现可比的准确性。此外，现有方法在应对环境因素（如表面纹理和由于环境光或相机曝光变化引起的亮度变化）时也面临挑战。为了解决这些局限性，我们提出了一种基于深度神经网络的方法。我们的方法旨在从基于文本的提示生成一幅自然图像，该图像不仅看起来真实，而且具有丰富的空间特征，以提高procam应用中的注册准确性。我们开发了一个在合成数据集上训练的深度神经网络，该数据集模拟了在照明相对平滑物体的procam系统中可能遇到的几何和光度失真（见图1）。我们训练的网络预测投影仪和相机图像之间的对应关系，显著提高了各种procam配置下的注册准确性。通过共同考虑投影仪图像的自然性和特征丰富性，我们的方法在不牺牲精度的情况下，最小化了投影内容中的视觉干扰。一项用户研究证实，与现有方法相比，我们的技术提高了感知自然性和可用性，验证了其在实际应用中的实用性。

View on arXiv Download PDF AI Translation

cs.CV / 89 / 2607.03058

RIGS-Refiner: Risk-Guided Recursive Refinement in Prediction Space for Colonoscopy Polyp Segmentation

RIGS-Refiner：基于风险引导的预测空间递归细化用于结肠镜息肉分割

Zhang, Jiachi, Wu, Zhuoyu, Fang, Wenqi

Abstract

Post-refinement can improve colonoscopy segmentation after host inference, but many designs still rely on extra correction heads or multi-stage pipelines with non-negligible parameter or computational cost. For polyp segmentation, host predictions are often already reasonable globally, with remaining errors clustered around ambiguous boundaries and difficult local structures. These residual errors matter in colonoscopy images because useful masks need correct lesion coverage and clean contour delineation across subtle mucosal transitions. This setting favors selective local repair in prediction space over reprocessing the entire mask. We therefore propose RIGS-Refiner, a lightweight post-refinement plugin for risk-guided recursive refinement in prediction space. Starting from a frozen host anchor prediction, RIGS-Refiner extracts lightweight image priors and prediction cues, applies risk-guided update, and writes back residual corrections through a shared recursive cell. The module adds only +519 parameters and +0.631 GFLOPs, keeping the refinement path compact for deployment. Experiments use Kvasir-SEG for training and Kvasir, ClinicDB, ColonDB, and ETIS for evaluation under two frozen hosts, namely PraNet and SegFormer-B0. Results show consistent gains on both hosts and a favorable efficiency-accuracy trade-off against representative post-refinement methods. Code is available at https://github.com/tyui99/RIGS-Refiner.

Chinese Translation

后期细化可以在主机推断后改善结肠镜分割，但许多设计仍依赖额外的修正头或多阶段管道，这些方法在参数或计算成本上不可忽视。对于息肉分割，主机预测通常在全局上已经相当合理，剩余的错误往往集中在模糊边界和困难的局部结构周围。这些残余错误在结肠镜图像中是重要的，因为有用的掩膜需要正确覆盖病变并在微妙的粘膜过渡处清晰勾勒轮廓。这种设置更倾向于在预测空间中进行选择性局部修复，而不是重新处理整个掩膜。因此，我们提出了RIGS-Refiner，一个轻量级的后期细化插件，用于在预测空间中进行风险引导的递归细化。RIGS-Refiner从一个冻结的主机锚点预测开始，提取轻量级图像先验和预测线索，应用风险引导更新，并通过共享递归单元写回残余修正。该模块仅增加+519个参数和+0.631 GFLOPs，保持了细化路径的紧凑性以便于部署。实验使用Kvasir-SEG进行训练，并在两个冻结主机PraNet和SegFormer-B0下评估Kvasir、ClinicDB、ColonDB和ETIS。结果显示在两个主机上均有一致的提升，并且与代表性的后期细化方法相比，表现出良好的效率-准确性权衡。代码可在https://github.com/tyui99/RIGS-Refiner获取。

View on arXiv Download PDF AI Translation

cs.CV / 90 / 2607.03062

Lightweight Polyp Segmentation via a Gain-Aware Prediction-Space Recursive Controller

基于增益感知预测空间递归控制器的轻量级息肉分割

Zhang, Jiachi, Wu, Zhuoyu, Wang, Quanjun, Ou, Wenhui, Fang, Wenqi

Abstract

While lightweight polyp segmentation is highly desirable for low-cost deployment, reported performance gains often stem from upgraded backbone encoders, complex decoders, or heavy refinement branches. Consequently, it remains difficult to isolate whether a lightweight correction mechanism is inherently effective on its own. We address this limitation by formulating refinement as a prediction-space recursive correction task, introducing a recursive controller that operates directly on backbone logits. Under a fixed recursion budget, this controller aggregates discrepancy and uncertainty evidence, updates a compact state tracking recent correction utility, and applies additive residual logit corrections. By design, this correction path remains small, host-portable, and deployment-explicit. Utilizing a unified Kvasir-trained protocol, we evaluate our approach across seven lightweight backbones on Kvasir-SEG and three transfer datasets, measuring segmentation accuracy (Dice/IoU) alongside deployment efficiency (parameters, GMACs, and peak memory). The controller yields consistent improvements in the source domain, achieves competitive performance against both training-side baselines and heavier structural refiners on representative hosts, and delivers selective transfer gains with minimal static overhead. Code is available at https://github.com/tyui99/Gain-Aware-Prediction-Space-Recursive-Controller.

Chinese Translation

轻量级息肉分割在低成本部署中极为重要，但报告的性能提升往往源于升级的主干编码器、复杂的解码器或重的细化分支。因此，单独评估轻量级修正机制的有效性仍然困难。我们通过将细化任务公式化为预测空间递归修正任务来解决这一局限性，引入一种直接作用于主干逻辑回归的递归控制器。在固定的递归预算下，该控制器聚合差异和不确定性证据，更新一个紧凑的状态以跟踪最近的修正效用，并应用附加的残差逻辑回归修正。根据设计，该修正路径保持小巧、可移植，并明确适用于部署。通过统一的 Kvasir 训练协议，我们在 Kvasir-SEG 和三个迁移数据集上评估了我们的方法，测量分割精度（Dice/IoU）以及部署效率（参数、GMACs 和峰值内存）。该控制器在源域中提供了一致的改进，针对训练端基线和代表性主机上的更重结构细化器达到了具有竞争力的性能，并在静态开销最小的情况下实现了选择性迁移增益。代码可在 https://github.com/tyui99/Gain-Aware-Prediction-Space-Recursive-Controller 获取。

View on arXiv Download PDF AI Translation

cs.CV / 91 / 2607.03068

PixCon: Clean-Positive Contrastive Learning for Foundation-Model Semi-Supervised Segmentation

PixCon：基础模型半监督分割的清晰正对比学习

Tarubinga, Ebenezer

Abstract

Semi-supervised semantic segmentation (SSSS) has long turned on one question, which pseudo-labels to trust, and answered it with ever more careful confidence filtering. Foundation backbones change the regime: with a DINOv2 teacher a strict threshold already retains a measured 98%-clean pseudo-label set, so the accuracy that remains lives not in the filter but in how the embedding space is structured by class. We propose PixCon, a clean-positive pixel-contrastive framework. PixCon maintains a per-class memory bank that admits only labeled pixels the student already classifies correctly, guaranteeing a contamination-free positive set ($\rho_F=0$) by construction, unlike prior contrastive SSSS banks (ReCo, U$^2$PL) built from confidence-filtered pseudo-labels. It is a single branch over a consistency backbone, adds no inference-time parameters, and needs no bank-specific threshold. A first-order analysis of the supervised-InfoNCE gradient explains why contamination hurts: its false-positive term scales as $\rho_F/(1-\rho_F)$, which we measure (0.018 on Pascal, 0.106 on ADE20K) rather than assume. Across Pascal VOC, Cityscapes, and ADE20K, PixCon matches or improves a strong DINOv2-based UniMatch V2 baseline in a compute-matched one-switch protocol: it improves every Pascal-1/8 seed (a per-seed gain of about +0.2 mIoU) and its three-seed mean reaches 87.90, the published UniMatch V2-B figure. Because contamination is already rare under foundation-model teachers, our analysis indicates the $\rho_F=0$ guarantee acts chiefly as robustness as teachers weaken, while the accuracy gain comes from cleaner positive supervision, making clean-positive contrast a robust, low-cost default for foundation-model SSSS.

Chinese Translation

半监督语义分割（SSSS）长期以来一直围绕一个问题展开，即应该信任哪些伪标签，并通过越来越严格的置信度过滤来回答这个问题。基础模型的出现改变了这一局面：使用 DINOv2 教师时，严格的阈值已经保留了一个经过测量的 98% 清洁伪标签集，因此剩余的准确性并不在于过滤器，而在于嵌入空间如何按类别结构化。我们提出了 PixCon，一个清晰正向像素对比框架。PixCon 维护一个按类别划分的记忆库，仅接受学生已经正确分类的标记像素，从而通过构建保证了无污染的正集（$ ho_F=0$），与之前基于置信度过滤伪标签构建的对比 SSSS 记忆库（ReCo, U$^2$PL）不同。它是一个单分支的一致性骨干网络，不增加推理时的参数，也不需要特定于记忆库的阈值。对监督-InfoNCE 梯度的一阶分析解释了污染为何会造成损害：其假阳性项的规模为 $ ho_F/(1- ho_F)$，我们在 Pascal 上测得为 0.018，ADE20K 上测得为 0.106，而不是假设。在 Pascal VOC、Cityscapes 和 ADE20K 数据集上，PixCon 在计算匹配的一次切换协议中与强大的基于 DINOv2 的 UniMatch V2 基线相匹配或有所改进：它改善了每个 Pascal-1/8 种子的表现（每种子约 +0.2 mIoU 的增益），其三种子均值达到了 87.90，接近已发布的 UniMatch V2-B 数据。由于在基础模型教师下污染已经很少，我们的分析表明，$ ho_F=0$ 的保证主要作为鲁棒性发挥作用，尤其是在教师性能减弱时，而准确性提升则来自于更清晰的正向监督，使得清晰正向对比成为基础模型 SSSS 的一种鲁棒且低成本的默认选择。

View on arXiv Download PDF AI Translation

cs.CV / 92 / 2607.03069

SafeGuard: A Multi-Agent Perception-Reasoning Framework for Social-Risk AI-Generated Video Detection

SafeGuard：一种用于社会风险AI生成视频检测的多智能体感知-推理框架

Wu, Wenlin, Zhou, Sheng, Song, Peipei, Wang, Wenhao, Xiao, Junbin, Yang, Xun

Abstract

As video generation paradigms evolve from localized manipulation to full-scene synthesis, AI-generated video detection becomes increasingly challenging, as forgeries exhibit coherent global structure and high perceptual realism. However, existing benchmarks are biased toward perceptual fidelity and primarily evaluate detectors based on perceptual artifacts, providing limited coverage of scenarios that require reasoning about violations of physical laws, structural coherence, or social logic. This dataset bias shapes current approaches and results in a Perception-Reasoning Gap: artifact-centric models capture low-level statistical irregularities yet lack semantic inference, whereas vision-language models perform semantic reasoning but remain insensitive to fine-grained forensic cues. To bridge this gap, we propose SafeGuard, a multi-agent framework that enables collaborative specialization between forensic perception and semantic reasoning. A hierarchical perceptual solver extracts fine-grained forensic evidence, while a self-reflective verifier enforces consistency between semantic inference and physical plausibility, forming an interpretable evidence chain. To support evaluation, we introduce SafeVid, a novel AI-generated video detection benchmark comprising 20K videos spanning 10 social risk categories, designed to evaluate physical plausibility, structural consistency, and the rationality of social behaviors. Extensive experiments demonstrate the generalization of SafeGuard, improving accuracy on SafeVid by +18.7% and consistently outperforming prior methods across four public benchmarks.

Chinese Translation

随着视频生成范式从局部操作演变为全场景合成，AI生成视频的检测变得愈加具有挑战性，因为伪造视频展现出一致的全局结构和高感知现实性。然而，现有基准测试偏向于感知保真度，主要基于感知伪影评估检测器，覆盖的场景有限，无法有效推理关于物理法则、结构一致性或社会逻辑的违反。这种数据集偏差塑造了当前的方法，导致了感知-推理差距：以伪影为中心的模型捕捉低级统计不规则性，但缺乏语义推理，而视觉-语言模型则进行语义推理，但对细粒度法医线索不够敏感。为了弥补这一差距，我们提出了SafeGuard，一个多智能体框架，能够在法医感知和语义推理之间实现协作专业化。一个层次化的感知求解器提取细粒度的法医证据，而一个自反验证器则确保语义推理与物理合理性之间的一致性，形成一个可解释的证据链。为了支持评估，我们引入了SafeVid，一个新颖的AI生成视频检测基准，包含20,000个视频，涵盖10个社会风险类别，旨在评估物理合理性、结构一致性和社会行为的合理性。大量实验表明SafeGuard的泛化能力，在SafeVid上的准确率提高了18.7%，并在四个公共基准上持续超越先前的方法。

View on arXiv Download PDF AI Translation

cs.CV / 93 / 2607.03073

Attention-Guided Efficientnet Architecture For Precise Criminal Identification in Surveillance Images

基于注意力引导的EfficientNet架构用于监控图像中的精确犯罪识别

J, Savitha N, T, Lata B

Abstract

Criminal identification from surveillance imagery has become a critical research area in intelligent forensic surveillance systems due to the increasing deployment of CCTV cameras in public and private environments. However, surveillance-based face recognition remains highly challenging because of low image resolution, illumination variation, motion blur, pose changes, facial occlusion, and background clutter. To address these limitations, this paper proposes an Attention-Guided EfficientNet (AG-EfficientNet) framework for precise criminal identification in surveillance images. The proposed framework integrates EfficientNet-B0 with Convolutional Block Attention Modules (CBAM) to enhance discriminative facial feature learning under degraded surveillance conditions. In addition, a multi-scale surveillance feature fusion strategy is introduced to preserve both local texture information and high-level semantic identity representations. A hybrid Softmax-Triplet optimization mechanism is further employed to improve inter-class separability and intra-class compactness for robust criminal identity discrimination. The proposed framework was experimentally evaluated using the Labeled Faces in the Wild (LFW) and SCFace datasets. Experimental results demonstrate that the proposed AG-EfficientNet framework achieved superior surveillance recognition performance with an identification accuracy of 98.2%, Precision of 97.9%, Recall of 97.6%, F1-Score of 97.7%, and ROC-AUC of 0.99, outperforming conventional deep learning architectures including AlexNet, VGG16, ResNet50, MobileNetV2, and standard EfficientNet-B0. Furthermore, Grad-CAM visualization and ablation analysis confirm the effectiveness of the proposed attention-guided feature learning strategy.

Chinese Translation

由于公共和私人环境中闭路电视摄像头的日益普及，从监控图像中进行犯罪识别已成为智能取证监控系统中的一个关键研究领域。然而，由于图像分辨率低、光照变化、运动模糊、姿态变化、面部遮挡和背景杂乱，基于监控的面部识别仍然面临很大挑战。为了解决这些限制，本文提出了一种基于注意力引导的EfficientNet（AG-EfficientNet）框架，用于监控图像中的精确犯罪识别。该框架将EfficientNet-B0与卷积块注意力模块（CBAM）相结合，以增强在恶劣监控条件下的区分性面部特征学习。此外，提出了一种多尺度监控特征融合策略，以保留局部纹理信息和高层语义身份表示。进一步采用混合Softmax-Triplet优化机制，以提高类间可分性和类内紧凑性，从而实现稳健的犯罪身份区分。通过在Labeled Faces in the Wild（LFW）和SCFace数据集上进行实验评估，结果表明，所提出的AG-EfficientNet框架在监控识别性能上表现优越，识别准确率达到98.2%，精确率为97.9%，召回率为97.6%，F1分数为97.7%，ROC-AUC为0.99，超越了包括AlexNet、VGG16、ResNet50、MobileNetV2和标准EfficientNet-B0在内的传统深度学习架构。此外，Grad-CAM可视化和消融分析证实了所提出的注意力引导特征学习策略的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 94 / 2607.03103

SNR-Adaptive Unified Diffusion for Multi-Task Medical Image Segmentation

SNR自适应统一扩散用于多任务医学图像分割

Liu, Jiahao, Wei, Hang, Wu, Shuai

Abstract

Clinical cardiac imaging pipelines currently deploy separate models for each dataset and modality, incurring redundant training costs and precluding knowledge sharing across anatomically related tasks. Consolidating semi-supervised learning, unsupervised domain adaptation, and domain generalisation into one model is therefore a practical necessity, yet naive joint training exposes a fundamental barrier: conflicting label semantics between datasets collapse LA Dice from 90.31\% to 83.38\%, while gradient imbalance across tasks of unequal complexity suppresses the weaker tasks throughout training. We present UniT-Diff, a unified diffusion segmentation framework that resolves these conflicts through three targeted mechanisms. An 11-channel task-specific output space physically partitions label categories, eliminating cross-task gradient sign reversal by construction. SNR-Adaptive Task Conditioning (SATC) scales the task token by the log signal-to-noise ratio of the current diffusion timestep, suppressing domain-specific bias during coarse denoising and restoring full task guidance as the signal clears. Task-Type-Aware Conditional Dropout (TTACD) permanently removes the task token for domain-generalisation inputs, routing them through a shared neutral pathway that draws on cross-dataset cardiac anatomy rather than source-vendor statistics. Under a single parameter set, UniT-Diff surpasses independently trained task-specific baselines on all three benchmarks simultaneously: +0.87\% on LA, +1.77\% on MMWHS, and +0.88\% on MNMS.

Chinese Translation

目前，临床心脏成像流程为每个数据集和模态部署了单独的模型，这导致了冗余的训练成本，并阻碍了在解剖相关任务之间的知识共享。因此，将半监督学习、无监督领域适应和领域泛化整合为一个模型是一个实际的必要性，但简单的联合训练暴露了一个根本性障碍：数据集之间标签语义的冲突使得左心房Dice从90.31\%下降到83.38\\%，而在复杂性不均的任务之间的梯度不平衡在整个训练过程中抑制了较弱的任务。我们提出了UniT-Diff，一个统一的扩散分割框架，通过三种针对性机制解决这些冲突。11通道的任务特定输出空间在物理上划分了标签类别，从构造上消除了跨任务梯度符号反转。SNR自适应任务条件（SNR-Adaptive Task Conditioning, SATC）根据当前扩散时间步的对数信噪比缩放任务标记，在粗略去噪期间抑制领域特定偏差，并在信号清晰时恢复完整的任务指导。任务类型感知条件丢弃（Task-Type-Aware Conditional Dropout, TTACD）永久性地移除领域泛化输入的任务标记，通过一个共享的中性路径引导它们，该路径依赖于跨数据集的心脏解剖结构，而不是源供应商的统计数据。在单一参数集下，UniT-Diff在所有三个基准测试中超越了独立训练的任务特定基线：在左心房上提高了+0.87\\%，在MMWHS上提高了+1.77\\%，在MNMS上提高了+0.88\\%。

View on arXiv Download PDF AI Translation

cs.CV / 95 / 2607.03110

ExpoMotion: A Large-Scale Benchmark and A Householder Projection Network for Multi-Exposure Fusion

ExpoMotion：一个大规模基准和用于多曝光融合的霍斯特投影网络

Liu, Yao, Qu, Lishen, Zhou, Shihao, Liang, Jie, Zeng, Hui, Peng, Yabin, Lin, Huipeng, Zhang, Lei, Yang, Jufeng

Abstract

Multi-Exposure Fusion (MEF) effectively extends dynamic range, but practical deployment is hindered by motion-induced ghosting and the scarcity of high-quality dynamic benchmarks. Current benchmarks largely neglect dynamic scenes and lack reliable ground truth, making it difficult to handle the complexity of real-world motions. In response, we introduce ExpoMotion, a large-scale benchmark designed to evaluate deghosting capabilities. Comprising 1,738 sequences and 10,909 images across diverse environments, it covers a wide range of motions and provides high-fidelity GTs constructed through an expert-guided acquisition pipeline. To tackle the complex dynamics and extreme conditions captured in this benchmark, we propose the Householder Orthogonal Projection network (HOP), which revisits MEF deghosting from a mathematical perspective via Householder transformation, decoupling multi-frame alignment into exposure pre-alignment and ghost filtering. Specifically, the Global Priors Illumination Alignment (GPIA) module first rectifies drastic dynamic range discrepancies by utilizing global statistics for exposure harmonization. Regarding ghost removal, our Householder Orthogonal Attention (HOA) models artifacts as orthogonal perturbations. By employing a dynamic Householder reflector, HOA effectively projects ghosts out of the feature manifold while preserving high-frequency details. Experiments demonstrate that our ExpoMotion dataset enables superior generalization and artifact-free detail restoration, while also validating the effectiveness and efficiency of the HOP method. The dataset and code are available at https://github.com/Leo-LiuYao/ExpoMotion.

Chinese Translation

多曝光融合（MEF）有效地扩展了动态范围，但由于运动引起的鬼影和高质量动态基准的稀缺，实际应用受到阻碍。目前的基准在很大程度上忽视了动态场景，并缺乏可靠的真实数据，使得处理现实世界运动的复杂性变得困难。为此，我们提出了ExpoMotion，一个旨在评估去鬼影能力的大规模基准。该基准包含1,738个序列和10,909张图像，涵盖多样化的环境，涉及广泛的运动，并提供通过专家指导的采集流程构建的高保真真实数据。为了应对该基准中捕捉到的复杂动态和极端条件，我们提出了霍斯特正交投影网络（HOP），从数学角度通过霍斯特变换重新审视MEF去鬼影，将多帧对齐解耦为曝光预对齐和鬼影过滤。具体而言，全球先验照明对齐（GPIA）模块首先利用全局统计数据进行曝光协调，纠正剧烈的动态范围差异。在鬼影去除方面，我们的霍斯特正交注意力（HOA）将伪影建模为正交扰动。通过采用动态霍斯特反射器，HOA有效地将鬼影投影出特征流形，同时保留高频细节。实验表明，我们的ExpoMotion数据集能够实现更优的泛化能力和无伪影的细节恢复，同时验证了HOP方法的有效性和效率。数据集和代码可在https://github.com/Leo-LiuYao/ExpoMotion获取。

View on arXiv Download PDF AI Translation

cs.CV / 96 / 2607.03118

Vidu S1: A Real-Time Interactive Video Generation Model

Vidu S1：一种实时互动视频生成模型

Zhang, Jintao, Jiang, Kai, Chen, Jintao, Wang, Xu, Luo, Yang, Wang, Yuji, Chen, Dechuang, Li, Jungang, Ye, Chengyang, Chen, Marco, Zhu, Hongzhou, Zhao, Min, Jiang, Yuxuan, Huang, Zhengkun, Xiang, Chendong, Zheng, Kaiwen, Wang, Haoxu, Wang, Xiaohang, Jia, Qi, Chen, Xin, Chen, Yimin, Jiang, Youhe, Fu, Fangcheng, Deng, Zhijie, Bao, Fan, Chen, Jianfei, Zhu, Jun

Abstract

We introduce Vidu S1, a real-time interactive video generation model supporting voice control of digital characters. Users can control video generation content at any moment through voice instructions. Vidu S1 supports infinite-length real-time video generation without blurring, drift, or visual distortion. Built with TurboDiffusion and TurboServe, Vidu S1 outputs 540p real-time videos at up to 42 FPS on regular consumer GPUs. Users can upload custom images of real people, anime, and pets, and choose different voice tones for personalized experiences. Experiments show that Vidu S1 achieves the best performance across all test metrics while fully meeting real-time inference requirements. A playable online demo is available at https://vidu.com/vidu-stream.

Chinese Translation

我们介绍了Vidu S1，这是一种支持数字角色语音控制的实时互动视频生成模型。用户可以通过语音指令在任何时刻控制视频生成内容。Vidu S1支持无限长度的实时视频生成，且无模糊、漂移或视觉失真。Vidu S1基于TurboDiffusion和TurboServe构建，能够在普通消费级GPU上以最高42帧每秒输出540p的实时视频。用户可以上传真实人物、动漫和宠物的自定义图像，并选择不同的语音音调以获得个性化体验。实验表明，Vidu S1在所有测试指标中表现最佳，同时完全满足实时推理的要求。可在https://vidu.com/vidu-stream访问可播放的在线演示。

View on arXiv Download PDF AI Translation

cs.CV / 97 / 2607.03131

A Multi-Task Deep Learning Framework for Real-Time Intelligent Video Surveillance with Temporal Event Validation

用于实时智能视频监控的多任务深度学习框架及时间事件验证

Dumitru, Estera, Spînu, Stelian

Abstract

Modern video surveillance systems generate far more video streams than human operators can effectively monitor, making automated analysis essential for timely detection of security events. This paper presents a unified multi-task deep learning framework that simultaneously performs face recognition with zone-based authorization, automatic license plate recognition, weapon detection, fire and smoke detection, and human action recognition on a shared GPU platform. Among the integrated modules, two task-specific deep-learning models are proposed in this work to address scenarios that are insufficiently represented in publicly available datasets: a single-class weapon detector fine-tuned on a merged and relabeled dataset, achieving a mean average precision ([email protected]) of 0.947, and a SlowFast-R50 action recognition model trained on a purpose-built vandalism dataset comprising 614 video clips, achieving 94.33% classification accuracy. To improve robustness in continuous video, all detection modules are integrated into a temporal event-validation architecture based on multi-frame confirmation, confidence-weighted voting, and cascaded filtering, transforming frame-level predictions into reliable security events. Each module is evaluated independently on established public datasets (LFW, D-Fire, FIRESENSE, and UCF-Crime), followed by integrated end-to-end system evaluation. The proposed temporal validation strategy reduces the fire and smoke false-alarm rate from 52% to 4% and improves video license plate exact-match accuracy from 66.7% to 81.8%, while the complete framework maintains real-time operation with a per-frame latency below 100 ms on commodity hardware. These results demonstrate that combining specialized deep-learning models with temporal event validation provides an effective and practical solution for reliable real-time intelligent video surveillance.

Chinese Translation

现代视频监控系统生成的视频流远超过人类操作员能够有效监控的数量，因此自动化分析对于及时检测安全事件至关重要。本文提出了一种统一的多任务深度学习框架，能够在共享GPU平台上同时执行基于区域的面部识别、自动车牌识别、武器检测、火灾与烟雾检测以及人类动作识别。在集成模块中，本文提出了两个特定任务的深度学习模型，以应对在公开可用数据集中不足以代表的场景：一个在合并和重新标记的数据集上微调的单类武器检测器，达到0.947的平均精度均值（[email protected]），以及一个在专门构建的破坏行为数据集上训练的SlowFast-R50动作识别模型，该数据集包含614个视频片段，分类准确率达到94.33%。为了提高连续视频中的鲁棒性，所有检测模块集成到基于多帧确认、置信度加权投票和级联过滤的时间事件验证架构中，将帧级预测转化为可靠的安全事件。每个模块在已建立的公共数据集（LFW、D-Fire、FIRESENSE和UCF-Crime）上独立评估，随后进行集成的端到端系统评估。所提出的时间验证策略将火灾和烟雾的误报率从52%降低到4%，并将视频车牌的精确匹配准确率从66.7%提高到81.8%，同时完整框架在普通硬件上保持每帧延迟低于100毫秒的实时操作。这些结果表明，将专业深度学习模型与时间事件验证相结合，为可靠的实时智能视频监控提供了有效且实用的解决方案。

View on arXiv Download PDF AI Translation

cs.CV / 98 / 2607.03143

Text as Partial Constraint: Core-Residual Alignment for Robust Vision-Language Learning

文本作为部分约束：用于鲁棒视觉-语言学习的核心-残余对齐

Yu, Chengzhen, Xiao, Canran, Ma, Siyuan, Liu, Yang

Abstract

Vision-language alignment powers open-vocabulary recognition, retrieval, and LVLM grounding, yet natural captions are often underspecified, making similarity brittle and overly confident under paraphrase and omitted details. We aim to learn representations whose matching is stable across caption views and whose confidence reflects how strongly text constrains an image. We propose Text as Partial Constraint (TPC), a core-residual alignment framework that treats multi-view captions as incomplete supervision. It distills a consensus semantic core as the alignment target, learns a single-view core predictor for standard inference with one query, and explicitly discourages vision-language similarity from depending on the orthogonal unsaid residual. An uncertainty-aware contrastive objective further softens alignment when caption views disagree, reducing overconfident updates under weak language constraints. Across zero-shot recognition and adversarial robustness, TPC achieves 81.42/64.05 Top-1 clean/robust accuracy on ImageNet and 76.19/52.03 on an Avg-14 transfer suite, while improving LVLM transfer with 85.16 POPE F1 and 59.57 OKVQA accuracy under an LLaVA-1.5-7B stack. These results suggest that modeling text as a partial constraint is a practical and principled route to more reliable vision-language representations under underspecified language supervision.

Chinese Translation

视觉-语言对齐推动了开放词汇的识别、检索和大语言模型（LVLM）基础，但自然语言描述往往不够具体，使得相似性在意译和省略细节时变得脆弱且过于自信。我们旨在学习在不同描述视角下匹配稳定的表示，并使其置信度反映文本对图像的约束程度。我们提出了文本作为部分约束（Text as Partial Constraint, TPC）框架，它将多视角描述视为不完全的监督。该框架提炼出共识语义核心作为对齐目标，学习单视角核心预测器以便于标准推理，并明确抑制视觉-语言相似性依赖于正交的未言明残余。当描述视角不一致时，基于不确定性的对比目标进一步软化对齐，减少在弱语言约束下的过度自信更新。在零-shot 识别和对抗鲁棒性方面，TPC在ImageNet上实现了81.42/64.05的Top-1干净/鲁棒准确率，在Avg-14迁移套件上实现了76.19/52.03，同时在LLaVA-1.5-7B堆栈下提升了LVLM迁移，获得了85.16的POPE F1和59.57的OKVQA准确率。这些结果表明，将文本建模为部分约束是在不完全语言监督下获得更可靠的视觉-语言表示的实用且原则性的途径。

View on arXiv Download PDF AI Translation

cs.CV / 99 / 2607.03156

DistillH-Mamba: A Hypergraph-Mamba-Based Knowledge Distillation Model for Efficient Impact Fall Detection

DistillH-Mamba：一种基于超图-Mamba的知识蒸馏模型，用于高效的跌倒影响检测

Koffi, Tresor Y., Mourchid, Youssef, Hindawi, Mohammed, Dupuis, Yohan

Abstract

Falls among the elderly represent a significant public health concern due to their prevalence, consequences, and societal burden. While deep learning has improved fall detection, accurately identifying impact moments (when an individual hits the ground) remains challenging. Additionally, current algorithms often rely on complex models with high computational demands, limiting real-time deployment feasibility. In this work, we propose DistillH-Mamba, a novel architecture for impact fall detection that addresses these challenges through three key innovations: First, we introduce a hypergraph-based approach that captures higher-order relationships between multiple joints simultaneously, enabling more accurate modeling of complex interactions during impact falls. Second, we integrate the Mamba architecture with hypergraphs for impact detection, significantly accelerating processing speed while efficiently capturing both long-term dependencies and sudden skeletal motion changes. Third, we employ relational knowledge distillation that preserves crucial spatial-temporal relationships while reducing computational demands for real-time impact fall detection. Evaluated on the 3D Skeletons UP-Fall and UMAFall datasets, our DistillH-Mamba model achieves 97.38% accuracy in detecting impact within fall events and 73.8% reduction in inference time compared to its teacher model, outperforming state-of-the-art methods in both precision and efficiency.

Chinese Translation

老年人跌倒是一个重要的公共卫生问题，因其普遍性、后果和社会负担而引起关注。尽管深度学习在跌倒检测方面有所改善，但准确识别影响时刻（即个体与地面接触的时刻）仍然具有挑战性。此外，目前的算法通常依赖于复杂的模型，计算需求高，限制了实时部署的可行性。在本研究中，我们提出了DistillH-Mamba，这是一种新颖的跌倒影响检测架构，通过三项关键创新来应对这些挑战：首先，我们引入了一种基于超图的方法，能够同时捕捉多个关节之间的高阶关系，从而更准确地建模跌倒影响中的复杂交互。其次，我们将Mamba架构与超图结合用于影响检测，显著加快了处理速度，同时有效捕捉长期依赖关系和突发的骨骼运动变化。第三，我们采用关系知识蒸馏，保留关键的时空关系，同时降低实时跌倒影响检测的计算需求。在3D Skeletons UP-Fall和UMAFall数据集上的评估表明，我们的DistillH-Mamba模型在跌倒事件中检测影响的准确率达到97.38%，并且与其教师模型相比推理时间减少了73.8%，在精度和效率上均优于现有的最先进方法。

View on arXiv Download PDF AI Translation

cs.CV / 100 / 2607.03165

Rethinking Brain Decoding with CLIP: The Role of Adversarial Robustness

重新思考基于 CLIP 的脑解码：对抗鲁棒性的作用

Bok, Byeongseo, Waseda, Futa, Liu, Jun, Echizen, Isao

Abstract

Brain decoding aims to uncover neural mechanisms by inferring stimulus-related representations from brain signals. In fMRI studies, this is typically achieved by mapping fMRI responses to the latent representations of computational models. Recently, CLIP has become a popular choice for brain decoding due to its rich vision--language embedding space. However, aligning fMRI signals with CLIP representations remains challenging. As CLIP is not explicitly optimized for neural alignment, its representations may capture statistically predictive cues that are only partially reflected in brain activity, limiting decoding performance. In this paper, we investigate whether adversarially robust representations improve neural decoding with CLIP. Adversarial training suppresses non-robust features and promotes more stable, perceptually structured representations, which may better align with brain activity. We evaluate this by fixing the fMRI decoder and varying only the target representation (standard CLIP vs. robust variants) on fMRI-image retrieval and zero-shot classification tasks across NSD and GOD datasets. Empirical results show that this simple change consistently improves task performance and yields stronger alignment across multiple metrics. Attribution analysis further reveals consistently low agreement between standard CLIP and its robust variants, suggesting that adversarial robustness reorganizes feature importance in the visual representation. These findings suggest that the choice of target representation influences neural decoding performance and that adversarial robustness may serve as a useful criterion for brain decoding.

Chinese Translation

脑解码旨在通过从脑信号中推断与刺激相关的表征来揭示神经机制。在功能性磁共振成像（fMRI）研究中，这通常通过将 fMRI 响应映射到计算模型的潜在表征来实现。最近，CLIP（Contrastive Language-Image Pretraining）因其丰富的视觉-语言嵌入空间而成为脑解码的热门选择。然而，将 fMRI 信号与 CLIP 表征对齐仍然具有挑战性。由于 CLIP 并未明确针对神经对齐进行优化，其表征可能捕捉到仅部分反映在脑活动中的统计预测线索，从而限制了解码性能。在本文中，我们研究了对抗鲁棒性表征是否能改善基于 CLIP 的神经解码。对抗训练抑制了非鲁棒特征，促进了更稳定、感知结构化的表征，这可能与脑活动更好地对齐。我们通过固定 fMRI 解码器，仅在 fMRI-图像检索和零样本分类任务中变化目标表征（标准 CLIP 与鲁棒变体），在 NSD 和 GOD 数据集上进行评估。实证结果表明，这一简单的变化始终提高了任务性能，并在多个指标上产生了更强的对齐。归因分析进一步揭示标准 CLIP 与其鲁棒变体之间的一致性低，表明对抗鲁棒性重新组织了视觉表征中的特征重要性。这些发现表明目标表征的选择影响神经解码性能，而对抗鲁棒性可能作为脑解码的有用标准。

View on arXiv Download PDF AI Translation

cs.CV / 101 / 2607.03180

FairFlow: Demystifying and Mitigating Stereotype Bias in Text-to-Image Diffusion Transformers

公平流：揭示和缓解文本到图像扩散变换器中的刻板印象偏见

Chen, Chen, Huang, Yuanmin, Zhang, Zhenfei, Zhang, Mi, Zhang, Xiaohan, Xiong, Yun, You, Xiaoyu, Yang, Min

Abstract

Multimodal diffusion transformers (MM-DiTs) have emerged as the prevalent backbone for modern text-to-image generation systems. However, they exhibit critical alignment vulnerabilities, systematically manifesting severe stereotype biases even under benign prompts. This poses a significant risk of algorithmic discrimination in deployed systems. Since most existing mitigation strategies were tailored for legacy U-Net architectures, the precise remediation of these vulnerabilities in MM-DiTs remains a critical open challenge. In this work, we first investigate the root cause of this vulnerability via mechanistic analysis. We reveal that bias representations in MM-DiTs are not uniformly distributed across depth, but are mediated by a sparse set of layers functioning as internal semantic binding hubs. These hubs exhibit a stage-wise propagation driving bias manifestation: early hubs establish the structural templates susceptible to bias, middle hubs actively extract core stereotypical concepts from textual conditioning, and late hubs globally solidify these biases through visual self-attention. Leveraging these architectural insights, we propose FairFlow, an intrinsic, mechanism-guided mitigation framework. FairFlow acts as an internal regulator by employing sparse steering: it learns attribute-specific fair directions and injects them exclusively at the identified semantic hubs within a constrained inference window. Evaluations on FLUX.1-dev and Stable Diffusion~3 demonstrate that FairFlow effectively neutralizes these stereotypical vulnerabilities across gender, race, and intersectional settings, achieving an optimal fairness-fidelity balance. With near-zero inference overhead and robustness to complex prompts, FairFlow provides a lightweight and practical bias mitigation for large-scale deployed MM-DiT systems. Code and datasets will be publicly released upon acceptance.

Chinese Translation

多模态扩散变换器（MM-DiTs）已成为现代文本到图像生成系统的主要框架。然而，它们表现出严重的对齐脆弱性，即使在良性提示下也系统性地表现出严重的刻板印象偏见。这对已部署系统构成了算法歧视的重大风险。由于大多数现有的缓解策略是针对传统的 U-Net 架构设计的，因此在 MM-DiTs 中精确修复这些脆弱性仍然是一个重要的开放挑战。在本研究中，我们首先通过机制分析探讨了这种脆弱性的根本原因。我们揭示了 MM-DiTs 中的偏见表征并不是均匀分布在各个层次，而是通过一组稀疏的层作为内部语义绑定中心进行调节。这些中心表现出阶段性传播，驱动偏见的表现：早期中心建立了易受偏见影响的结构模板，中间中心积极提取文本条件下的核心刻板概念，而后期中心通过视觉自注意力全局巩固这些偏见。利用这些架构见解，我们提出了 FairFlow，一个内在的、机制引导的缓解框架。FairFlow 通过采用稀疏引导作为内部调节器：它学习特定属性的公平方向，并仅在识别出的语义中心内的有限推理窗口中注入这些方向。在 FLUX.1-dev 和 Stable Diffusion~3 上的评估表明，FairFlow 能够有效中和性别、种族和交叉设置下的这些刻板印象脆弱性，实现最佳的公平性与保真度平衡。FairFlow 几乎没有推理开销，并且对复杂提示具有鲁棒性，为大规模部署的 MM-DiT 系统提供了一种轻量级且实用的偏见缓解方案。代码和数据集将在接受后公开发布。

View on arXiv Download PDF AI Translation

cs.CV / 102 / 2607.03184

BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception

BVS：基于贝叶斯的视觉搜索与多模态大语言模型的细粒度感知

Li, Geng, Peng, Yuxin

Abstract

While Multimodal Large Language Models (MLLMs) demonstrate impressive general capabilities, they struggle with fine-grained perception in ultra-high-resolution (UHR) images, particularly for tiny objects in cluttered scenes. Existing methods face a dilemma: they either rely on inefficient prior-free scanning, or depend on static prior-driven heuristics that lack posterior correction to rectify initial model biases. To address this, we propose BVS (Bayesian Visual Search), a framework that formulates perception as a global optimization problem over a continuous spatial-scale manifold. Specifically, BVS bridges prior guidance with posterior correction: it utilizes an early-stop attention rollout of MLLM to construct reasoning-aware priors, while employing a scale-aware non-stationary kernel and GP-UCB to dynamically rectify noise and recover missing information in the prior through iterative local observations. We provide theoretical guarantees via sub-linear regret bounds, and extensive experiments demonstrate that BVS significantly outperforms state-of-the-art baselines with a superior trade-off between accuracy and efficiency.

Chinese Translation

尽管多模态大语言模型（MLLMs）展示了令人印象深刻的通用能力，但在超高分辨率（UHR）图像中的细粒度感知方面仍然存在困难，尤其是在杂乱场景中的微小物体。现有方法面临两难选择：要么依赖低效的无先验扫描，要么依赖缺乏后验修正的静态先验驱动启发式方法来纠正初始模型偏差。为了解决这个问题，我们提出了BVS（贝叶斯视觉搜索），一个将感知形式化为连续空间尺度流形上的全局优化问题的框架。具体而言，BVS将先验引导与后验修正相结合：它利用MLLM的早停注意力展开构建具有推理意识的先验，同时采用尺度感知的非平稳核和GP-UCB动态修正噪声，并通过迭代局部观察恢复先验中的缺失信息。我们通过次线性遗憾界限提供理论保证，并且大量实验表明，BVS在准确性和效率之间实现了优越的权衡，显著超越了最先进的基线。

View on arXiv Download PDF AI Translation

cs.CV / 103 / 2607.03196

Seeing Through WiFi: Lightweight Human Pose Estimation with Dynamic Kernel Attention

透视WiFi：基于动态核注意力的轻量级人体姿态估计

Gian, Toan D., Nguyen, Van-Dinh, Son, Vo Phi, Nguyen, Nhan Thanh, Hoang, Dinh Thai, Nguyen, Diep N., Luong, Nguyen Cong, Chatzinotas, Symeon

Abstract

WiFi-based human pose estimation (HPE) enables the detection and interpretation of human body positions and movements without the need for wearable devices while preserving individual privacy concerns. Implementing this solution requires enhancing model performance and maintaining efficiency, especially on resource-constrained devices. This paper introduces a novel framework, WiLHPE, for lightweight and efficient human pose estimation using WiFi CSI signals. Empowered by a camera-based model during training, WiLHPE processes raw WiFi signals directly to estimate human poses in the testing phase. It employs a novel neural network architecture to dynamically learn convolutional kernels and apply attention mechanisms across channel and frequency spaces. This innovative method diversifies the kernels to improve the recognition capabilities of WiFi signals without adding complexity, ensuring efficiency. Additionally, the Tree-Structured Parzen Estimator algorithm is employed to optimize the critical hyperparameters of the neural network efficiently, minimizing the time required for optimal hyperparameter search compared to heuristic methods. Results from experiments on both the MM-Fi and WiPose datasets highlight the superiority of WiLHPE over state-of-the-art approaches, achieving 85.96% and 94.27% at PCK50, respectively, with minimal computational overhead. Notably, WiLHPE performs impressively even under challenging conditions, maintaining around 80% at PCK50 under AWGN noise with an error variance of 0.5.

Chinese Translation

基于WiFi的人体姿态估计（HPE）能够在不需要可穿戴设备的情况下检测和解释人体位置和运动，同时保护个人隐私。实现这一解决方案需要提升模型性能并保持效率，尤其是在资源受限的设备上。本文提出了一种新颖的框架WiLHPE，用于利用WiFi CSI信号进行轻量级和高效的人体姿态估计。在训练过程中，WiLHPE借助基于摄像头的模型，直接处理原始WiFi信号以估计测试阶段的人体姿态。它采用了一种新颖的神经网络架构，动态学习卷积核并在通道和频率空间中应用注意力机制。这种创新方法多样化了卷积核，提高了WiFi信号的识别能力而不增加复杂性，从而确保了效率。此外，采用树结构Parzen估计器（Tree-Structured Parzen Estimator）算法高效优化神经网络的关键超参数，相较于启发式方法，最小化了寻找最佳超参数所需的时间。对MM-Fi和WiPose数据集的实验结果显示，WiLHPE在性能上优于最先进的方法，分别在PCK50上达到了85.96%和94.27%的准确率，且计算开销极小。值得注意的是，WiLHPE在挑战性条件下表现出色，在AWGN噪声下保持了约80%的PCK50准确率，误差方差为0.5。

View on arXiv Download PDF AI Translation

cs.CV / 104 / 2607.03209

Fast 3D Foundation Model Initialized Gaussian Splatting

快速的3D基础模型初始化高斯点云重建

Dalal, Anurag, Hagen, Daniel, Robbersmyr, Kjell G., Knausgård, Kristian Muri

Abstract

This paper introduces a fast method for high-quality 3D Gaussian Splatting (3DGS) reconstruction without traditional Structure-from-Motion (SfM). The proposed approach leverages 3D Foundation Models (3DFMs) for camera pose and point-cloud initialization, then jointly optimizes both camera poses and Gaussian primitives using a depth-guided loss function. This enables fast convergence even from rough initialization with as few as 50-60 input views. To further improve reconstruction quality in sparse-view scenarios, an MLP-based pose refinement module is introduced alongside depth-guided supervision from the foundation model. Extensive experiments on Mip-NeRF 360, Tanks and Temples, and RobustNeRF demonstrate that the proposed method achieves competitive reconstruction quality (23.61 dB PSNR, 0.19 LPIPS) while reducing training time to approximately three minutes per scene. The proposed method produces ready-to-use 3DGS models at a fraction of the time required by existing pipelines, making it suitable for near real-time applications in robotics, VR, and autonomous navigation.

Chinese Translation

本文提出了一种快速的高质量3D高斯点云重建（3DGS）方法，无需传统的运动结构（SfM）。所提方法利用3D基础模型（3DFMs）进行相机姿态和点云的初始化，然后使用深度引导损失函数共同优化相机姿态和高斯原语。这使得即使从粗略的初始化开始，也能在仅有50-60个输入视图的情况下实现快速收敛。为了进一步提高稀疏视图场景下的重建质量，引入了基于多层感知器（MLP）的姿态精细化模块，并结合基础模型的深度引导监督。在Mip-NeRF 360、Tanks and Temples和RobustNeRF上的大量实验表明，所提方法在重建质量上具有竞争力（23.61 dB PSNR，0.19 LPIPS），同时将每个场景的训练时间缩短至大约三分钟。该方法在现有流程所需时间的基础上，能够以极短的时间生成可直接使用的3DGS模型，适用于机器人、虚拟现实（VR）和自主导航等近实时应用。

View on arXiv Download PDF AI Translation

cs.CV / 105 / 2607.03213

OpenGlass: A Sensing-Computing Split Architecture for Local MLLM-Driven Real-Time Visual Assistance

OpenGlass：一种基于感知-计算分离架构的本地 MLLM 驱动实时视觉辅助系统

Li, Mengzhang, Yao, Yuan

Abstract

We present OpenGlass, an open-source, privacy-oriented, local-first system for low-latency multimodal visual assistance, with a primary focus on blind and low-vision users. Cloud MLLM assistants offer strong visual understanding, but often require uploading first-person visual data and can suffer multi-second network delays; wearable glasses are ideal for sensing, but cannot host large models under tight compute and power budgets. OpenGlass addresses this gap with a sensing-computing split: an ESP32-based glasses-side unit captures visual context, while a nearby consumer-grade device performs local MLLM inference and local speech output, reducing cloud reliance and keeping raw egocentric visual data on user-controlled devices by default. We evaluate response quality, query-ready-to-audio latency, safety-aware abstention, and auditable logs. Under real ESP32 Wi-Fi capture, OpenGlass reaches 993 ms median user-to-audio latency with resized payloads and 1625 ms with raw 1280 x 720 payloads; 97.5% and 93.3% of trials fall below 2 s, respectively. OpenGlass is a user-initiated visual-assistance reference platform for obstacle/hazard awareness, sign/object queries, and image-quality self-checking, rather than a certified navigation aid. We release source code, hardware instructions, prompts, evaluation data, and logs.

Chinese Translation

我们提出了 OpenGlass，这是一种开源、以隐私为导向的本地优先系统，旨在为盲人和低视力用户提供低延迟的多模态视觉辅助。云端 MLLM 助手提供强大的视觉理解能力，但通常需要上传第一人称视觉数据，并可能遭遇多秒的网络延迟；可穿戴眼镜非常适合感知，但在计算和功耗预算紧张的情况下无法承载大型模型。OpenGlass 通过感知-计算分离来填补这一空白：基于 ESP32 的眼镜侧单元捕获视觉上下文，而附近的消费级设备执行本地 MLLM 推理和本地语音输出，从而减少对云端的依赖，并默认将原始自我中心视觉数据保留在用户控制的设备上。我们评估了响应质量、查询到音频的延迟、安全意识的自我克制以及可审计的日志。在真实的 ESP32 Wi-Fi 捕获下，OpenGlass 达到了 993 毫秒的中位用户到音频延迟（使用调整大小的负载）和 1625 毫秒（使用原始 1280 x 720 的负载）；分别有 97.5% 和 93.3% 的试验低于 2 秒。OpenGlass 是一个用户发起的视觉辅助参考平台，旨在提高对障碍物/危险的意识、进行标志/物体查询以及进行图像质量自检，而不是一个认证的导航辅助工具。我们发布了源代码、硬件说明、提示、评估数据和日志。

View on arXiv Download PDF AI Translation

cs.CV / 106 / 2607.03247

Learning to Suppress SPAD-based LiDAR Flare

学习抑制基于单光子雪崩二极管的激光雷达光晕

Zhu, Xuanya, Shen, Linghao

Abstract

Single-Photon Avalanche Diode (SPAD)-based Light Detection and Ranging (LiDAR) is emerging for autonomous vehicles due to its high sensitivity and precise depth sensing capabilities. However, flare caused by excessive photon returns or pile-up effects can lead to incorrect depth estimation and exaggerated boundaries in point clouds, resulting in severe distortions of geometric measurements, making flare suppression essential for safety-critical applications. Existing flare mitigation methods primarily operate at the hardware or signal-processing levels. While effective under specific configurations, they are largely rule-based and configuration-dependent, lacking learnable representations that generalize across diverse sensing scenarios. In this work, we reformulate flare suppression as a semantic segmentation problem, enabling data-driven learning of geometric and photometric cues directly from SPAD measurements. We first benchmark representative segmentation models on the newly introduced SPAD flare dataset and observe that they struggle to exploit the intrinsic multi-echo characteristics of SPAD signals. Motivated by this observation, we propose Physically-Informed segmentation for LiDAR Flare (PILF), a learning-based approach that treats the first and second echoes, together with ambient illumination, as distinct modalities, aggregating cross-echo information while jointly encoding geometric and photometric features. Experiments across multiple real-world scenes demonstrate that PILF significantly outperforms compared segmentation models, achieving up to 79.32% mIoU, and providing an effective solution for SPAD-based LiDAR flare suppression.

Chinese Translation

基于单光子雪崩二极管（SPAD）的光学探测与测距（LiDAR）因其高灵敏度和精确的深度感知能力而在自主驾驶车辆中逐渐崭露头角。然而，由于过量光子返回或堆叠效应引起的光晕会导致深度估计不准确和点云边界夸大，从而严重扭曲几何测量，因此抑制光晕对于安全关键应用至关重要。现有的光晕缓解方法主要在硬件或信号处理层面进行操作。尽管在特定配置下有效，但它们大多是基于规则且依赖于配置，缺乏能够在多样化感知场景中进行泛化的可学习表示。在本研究中，我们将光晕抑制重新表述为语义分割问题，使得能够直接从SPAD测量中进行几何和光度线索的数据驱动学习。我们首先在新引入的SPAD光晕数据集上基准测试了代表性的分割模型，并观察到它们在利用SPAD信号的内在多回波特性方面存在困难。基于这一观察，我们提出了物理信息驱动的激光雷达光晕分割（Physically-Informed segmentation for LiDAR Flare，PILF），这是一种学习基础的方法，将第一和第二回波以及环境光照视为不同的模态，聚合跨回波信息，同时联合编码几何和光度特征。多个真实场景的实验表明，PILF显著优于对比的分割模型，达到79.32%的平均交并比（mIoU），为基于SPAD的激光雷达光晕抑制提供了有效的解决方案。

View on arXiv Download PDF AI Translation

cs.CV / 107 / 2607.03253

Semantic Segmentation-Driven Image-Level Diagnosis of Liver Cancers in Hematoxylin and Eosin Histopathology Images

基于语义分割的肝癌图像级诊断方法在苏木精-伊红组织病理图像中的应用

Kopriva, Ivica, Sitnik, Dario, Pacic, Arijana, Krstanac, Karolina, Dalic, Irena Veliki, Hadzija, Marijana Popovic

Abstract

As hematoxylin & eosin (H&E) staining constitutes the primary entry point in routine diagnostic workflows, computer-aided diagnosis from whole-slide H&E images is of particular clinical relevance. However, substantial variability in specimen preparation, staining protocols, and scanning conditions, together with inherent uncertainty in expert pixel-level annotations, makes automated analysis of H&E-stained images challenging. In this study, we propose a semantic segmentation-based framework for image-level diagnosis, grounded in the clinically motivated assumption that each histopathological image corresponds to a single cancer type. Image-level predictions are obtained by assigning the class of the dominant pixel-level label in the segmentation output. To ensure clinical relevance, we adopt the nnU-Net architecture and train it on a publicly available dataset collected in our study with pixel-level annotations for three liver cancer types: hepatocellular cacrcinoma (HCC; 55 images from 30 patients), cholangiocellular carcinoma (CCA; 55 images from 29 patients), and colorectal metastatic adenocarcinoma (CMA; 60 images from 30 patients). Annotations were independently provided by four pathologist. We hypothesize that the combination of stain normalization and semantic segmentation mitigates domain shift and reduces sensitivity to annotation noise. Five-fold cross-validation yielded balanced accuracy of 0.975 (HCC), 0.950 (CCA), and 1.000 (CMA), comparable to results obtained with immunohosthochemical staining and superior to several deep learning models trained on patch-level annotations. The proposed framework has the potential to support pathologists in prioritizing immunohistochemical marker selection, thereby reducing diagnostic costs and turnaround time. Integration with immunohistochemical findings improve overall diagnostic reliability.

Chinese Translation

由于苏木精-伊红（H&E）染色是常规诊断工作流程中的主要入口，因此基于全切片H&E图像的计算机辅助诊断具有特别的临床相关性。然而，样本制备、染色协议和扫描条件的显著变异，以及专家像素级标注固有的不确定性，使得H&E染色图像的自动分析面临挑战。在本研究中，我们提出了一种基于语义分割的图像级诊断框架，基于临床动机假设，即每幅组织病理图像对应于一种癌症类型。通过将分割输出中占主导地位的像素级标签的类别分配给图像级预测，从而获得图像级预测。为了确保临床相关性，我们采用了nnU-Net架构，并在我们研究中收集的公开数据集上进行训练，该数据集包含三种肝癌类型的像素级标注：肝细胞癌（HCC；来自30名患者的55幅图像）、胆管细胞癌（CCA；来自29名患者的55幅图像）和结直肠转移性腺癌（CMA；来自30名患者的60幅图像）。标注由四位病理学家独立提供。我们假设染色标准化与语义分割的结合可以减轻领域转移，并降低对标注噪声的敏感性。五折交叉验证的结果显示，HCC的平衡准确率为0.975，CCA为0.950，CMA为1.000，这与免疫组化染色获得的结果相当，并且优于多种基于补丁级标注训练的深度学习模型。所提出的框架有潜力支持病理学家优先选择免疫组化标记，从而降低诊断成本和周转时间。与免疫组化结果的整合提高了整体诊断的可靠性。

View on arXiv Download PDF AI Translation

cs.CV / 108 / 2607.03256

A Decomposable Probe for Few-Step Diffusion Models: Prompt, Latent, and Score Selectivity across Backbone Families and Distillation Paradigms

可分解探针用于少步扩散模型：在主干网络家族和蒸馏范式中的提示、潜在和评分选择性

Haojie, Patrick Mu

Abstract

Few-step distilled diffusion students cut text-to-image inference from ~50 to 1-8 network evaluations, but the quality gap is usually summarised by a single FID/CLIP scalar that cannot say which axis of the conditioning response changed, nor whether a behaviour comes from the architecture, the distillation objective, or simply from being a diffusion model. We replace the scalar with a decomposable probe that injects controlled perturbations along three layers (prompt encoder, denoiser input, denoiser output) under three modes (mean, variance, scale) and six strengths, reporting a bootstrap-median Bures W2^2 selectivity ratio on Inception features. Under a single matched estimator across 23 models -- five teachers and 18 distilled students spanning five backbone families (SDXL, SD1.5, SD3.5, PixArt-alpha, FLUX), three architecture classes (UNet, DiT, MMDiT), and five distillation paradigms -- the three layers read three empirically separable factors: the prompt layer is a universal prompt-mean response (a sanity channel, not a discriminator), the latent layer reads the prediction type, and the score layer reads the distillation objective. Our main result: within this sweep, the latent layer is a near-binary detector of rectified-flow backbones. Its ratio exceeds 1 across a sustained low-to-mid band only for rectified-flow models (SD3.5, FLUX); no epsilon-prediction model qualifies. A matched epsilon-prediction control (PixArt-alpha) rules out wide-T5 conditioning, and the fingerprint survives adversarial (ADD) distillation as both teacher and student. Two secondary score-layer findings hold under narrower scopes: a canonical 4-step ADD-vs-rest contrast on the UNet families with a non-ADD baseline, and a CI-separated trajectory-rollout early-strength score spike on both UNet and DiT. All ratios are CI-citable under one estimator; we release the per-cell tables and the estimator.

Chinese Translation

少步蒸馏扩散学生将文本到图像的推理从约50次网络评估减少到1-8次，但质量差距通常仅用单一的FID/CLIP标量来总结，这无法说明条件响应的哪个轴发生了变化，也无法判断某种行为是来自架构、蒸馏目标，还是仅仅因为是扩散模型。我们用一个可分解探针替代了标量，该探针在三个层次（提示编码器、去噪输入、去噪输出）下以三种模式（均值、方差、尺度）和六种强度注入受控扰动，并在Inception特征上报告了自助中位数Bures W2^2选择性比率。在23个模型中使用单一匹配估计器——五个教师和18个蒸馏学生，涵盖五个主干网络家族（SDXL、SD1.5、SD3.5、PixArt-alpha、FLUX）、三种架构类别（UNet、DiT、MMDiT）和五种蒸馏范式——这三个层次读取了三个经验上可分离的因素：提示层是一个通用的提示均值响应（一个理智通道，而不是判别器），潜在层读取预测类型，评分层读取蒸馏目标。我们的主要结果是：在这一范围内，潜在层是一个近乎二元的修正流主干检测器。其比率在一个持续的低到中频带中仅对修正流模型（SD3.5、FLUX）超过1；没有epsilon预测模型符合条件。一个匹配的epsilon预测控制（PixArt-alpha）排除了宽T5条件，而指纹在对抗性（ADD）蒸馏中得以保留，作为教师和学生。两个次要评分层发现保持在更窄的范围内：在UNet家族上进行的经典4步ADD与其他对比，基于非ADD基线，以及在UNet和DiT上出现的CI分离轨迹展开早期强度评分峰值。所有比率在一个估计器下都是CI可引用的；我们发布了每个单元的表格和估计器。

View on arXiv Download PDF AI Translation

cs.CV / 109 / 2607.03261

OmniLayout: A Schematic-Coupled Multimodal Benchmark for Constraint-Aware Geometric Reasoning in PCB Layout

OmniLayout：一种用于PCB布局中约束感知几何推理的示意图耦合多模态基准

Lu, Taiting, Lin, Kaiyuan, Wang, Mingjia, Ye, Haolin, Liu, Runze, Tian, Yuxin, Melkonyan, Vahe, Wang, Haoyu, Wang, Muchuan, Hong, Chufan, Yang, Yifan, Chen, Sung-Liang, Chen, Yi-Chao, Jin, Yicheng, Gowda, Mahanth

Abstract

Recent large language models (LLMs) have demonstrated remarkable progress in 3D spatial reasoning, spatial grounding, and fine-grained geometric understanding. However, their ability to reason about densely packed object placement under strict spatial and functional constraints remains largely unexplored, despite being a fundamental challenge in practical electronic design automation (EDA) workflows. To bridge this gap, we introduce OmniLayout, the first benchmark designed to evaluate LLMs on printed-circuit-board (PCB) layout placement reasoning under real-world geometric, routing, and connectivity constraints. OmniLayout contains 1,681 industrial-grade schematic-coupled PCB layouts and includes four tasks: (1) geometric reasoning for IC physical placement, with 77.24K placement instances constrained within PCB board boundaries; (2) routability-aware placement reasoning, generating physically valid component placements; (3) electrical functionality, preserving schematic-specified connectivity and electronic functional correctness; and (4) tool-augmented agentic reasoning for invoking external tools to accomplish tasks (1)-(3). Our results reveal substantial limitations of current LLMs in PCB layout placement, including weak geometric reasoning, poor routability optimization, and inconsistent preservation of electrical functionality.

Chinese Translation

近期的大型语言模型（LLMs）在三维空间推理、空间定位和细粒度几何理解方面取得了显著进展。然而，它们在严格的空间和功能约束下进行密集物体放置推理的能力仍然在很大程度上未被探索，尽管这是实际电子设计自动化（EDA）工作流程中的一个基本挑战。为了解决这一问题，我们引入了OmniLayout，这是第一个旨在评估LLMs在真实世界几何、布线和连接约束下的印刷电路板（PCB）布局放置推理的基准。OmniLayout包含1,681个工业级示意图耦合的PCB布局，并包括四个任务：（1）集成电路物理放置的几何推理，包含77.24K个受PCB边界约束的放置实例；（2）考虑可布线性的放置推理，生成物理有效的组件放置；（3）电气功能，保持示意图指定的连接性和电子功能的正确性；（4）工具增强的代理推理，用于调用外部工具以完成任务（1）-（3）。我们的结果揭示了当前LLMs在PCB布局放置方面的重大局限性，包括几何推理薄弱、可布线优化差和电气功能保持不一致。

View on arXiv Download PDF AI Translation

cs.CV / 110 / 2607.03277

Defending from GeoLocalization through Adversarial Road Trips

通过对抗性公路旅行防御地理定位

Niccoli, Niccolò, Becattini, Federico, Seidenari, Lorenzo

Abstract

Retrieval-based image geolocalization has emerged as a powerful technique for determining the location of a query image by matching it against a large, geotagged database. The success of deep learning based approaches has raised concerns regarding privacy and safety. A way to protect users from geolocalization is to design adversarial attacks for such methods. In this paper, we introduce RoadTrip Attack (RTA), a novel and highly effective targeted adversarial attack for geolocalization. RTA conceptualizes the adversarial process as finding an optimal distractor journey to a specific, attacker-chosen location. It employs a beam search algorithm to iteratively construct a sequence of incorrect geographic locations that form a path to the target. At each step, the attack generates subtle perturbations to the query image, guiding the geolocalization model toward the next location in this deceptive path. We show that our method is also strong in black-box settings, obtaining highly transferable attacks with less perceptible image artifacts.

Chinese Translation

基于检索的图像地理定位已成为一种强大的技术，通过与大型地理标记数据库进行匹配来确定查询图像的位置。基于深度学习的方法的成功引发了对隐私和安全的担忧。保护用户免受地理定位影响的一种方法是为此类方法设计对抗性攻击。本文介绍了一种新颖且高效的针对性对抗攻击方法——公路旅行攻击（RoadTrip Attack, RTA），用于地理定位。RTA将对抗过程概念化为寻找一条通往特定攻击者选择位置的最佳干扰路径。它采用束搜索算法，迭代构建一系列不正确的地理位置，这些位置形成通往目标的路径。在每一步中，攻击生成对查询图像的微妙扰动，引导地理定位模型朝着这一欺骗路径中的下一个位置前进。我们展示了我们的方法在黑箱设置中也表现出色，能够获得高度可转移的攻击，同时图像伪影不易被察觉。

View on arXiv Download PDF AI Translation

cs.CV / 111 / 2607.03320

LBTCap: A Lightweight Bilateral Transformer for Real-Time Remote Sensing Image Change Captioning

LBTCap：一种轻量级双向变换器用于实时遥感图像变化描述

Zhang, Licheng, Lam, Siew-Kei, Akhtar, Naveed

Abstract

Remote sensing image change captioning (RSICC) generates natural-language descriptions of semantic changes between paired remote sensing images (RSIs), supporting applications such as urban planning, disaster response, and environmental monitoring. Although recent methods achieve strong captioning accuracy, most overlook computational efficiency and inference speed, which are essential for real-time deployment in practice. To this end, we propose LBTCap, a lightweight RSICC framework built on a bilateral Transformer that jointly models pre- and post-change features for efficient processing of paired RSIs. Specifically, we introduce a bilateral attention mechanism for paired inputs: the two temporal images are projected into separate queries and keys by the same query and key matrices shared across the two images, the value is formed from their concatenation, and the two resulting attention maps are combined by a learnable, structurally bilateral weighting instead of a fixed subtraction. This design keeps both temporal branches explicit while remaining compact, and, together with a truncated backbone and grouped-query attention, LBTCap uses only 39.99M parameters, of which the change-aware encoder accounts for just 2.78M. Extensive experiments on two public RSICC datasets show that LBTCap matches or closely approaches the accuracy of state-of-the-art methods while using far fewer parameters and running at markedly higher inference speed, with the benefit of the bilateral formulation most pronounced in the low-resource setting, demonstrating a favorable accuracy-efficiency trade-off for practical RSICC.

Chinese Translation

遥感图像变化描述（RSICC）生成配对遥感图像（RSI）之间语义变化的自然语言描述，支持城市规划、灾害响应和环境监测等应用。尽管近期方法在描述准确性上取得了显著进展，但大多数方法忽视了计算效率和推理速度，这对于实际的实时部署至关重要。为此，我们提出了LBTCap，一种基于双向变换器的轻量级RSICC框架，能够高效处理配对RSI的前后变化特征。具体而言，我们为配对输入引入了一种双向注意机制：两个时间图像通过在两个图像之间共享的相同查询和键矩阵被投影为独立的查询和键，值则由它们的拼接形成，两个生成的注意图通过可学习的、结构性双向加权组合，而不是固定的减法。这一设计在保持两个时间分支明确的同时，保持了紧凑性，并且结合截断的主干网络和分组查询注意，LBTCap仅使用39.99M参数，其中变化感知编码器仅占2.78M。对两个公共RSICC数据集的广泛实验表明，LBTCap在使用远少于参数的情况下，其准确性与最先进的方法相匹配或接近，同时推理速度显著提高，双向结构的优势在低资源环境中尤为明显，展示了RSICC在实际应用中良好的准确性与效率权衡。

View on arXiv Download PDF AI Translation

cs.CV / 112 / 2607.03327

From General Actions to Domain-Specific Monitoring: Prior-Adaptive Transfer for Skeleton-Based Action Recognition

从一般动作到领域特定监测：基于先验自适应转移的骨架动作识别

Wang, Hao, Yang, Di, Wang, Jiangtao

Abstract

Skeleton-based action recognition models have recently shown strong performance on large-scale benchmarks with general actions. However, directly transferring them to domain-specific tasks e.g., healthcare monitoring, is often suboptimal, as such tasks are narrow in scope and may be relevant to only a subset of general motion priors. Moreover, not all pretrained motion patterns are equally useful for a specific task, and retaining less relevant components may hinder adaptation and increase computational cost. To address these challenges, we propose Prior-Adaptive Transfer of Skeletons (PATS), a framework that adapts general skeleton-based models by selectively retaining task-relevant motion priors while filtering redundant ones during transfer. PATS follows a standard pipeline that extracts skeleton signals from videos and employs a spatio-temporal backbone pre-trained on general actions. The key contribution lies in a novel Adaptive Prior Transfer module, which performs model compression as a prior selection mechanism through iterative pruning and refinement. Experiments on two specific action recognition tasks, Alzheimer's detection and fall detection, show consistent improvements in both performance and efficiency over competitive baselines. The code will be released upon acceptance.

Chinese Translation

基于骨架的动作识别模型最近在大型基准测试中对一般动作表现出了强大的性能。然而，直接将其转移到领域特定任务（例如，医疗监测）往往效果不佳，因为这些任务的范围较窄，可能仅与一部分一般运动先验相关。此外，并非所有预训练的运动模式对特定任务都同样有用，保留不相关的组件可能会妨碍适应并增加计算成本。为了解决这些挑战，我们提出了骨架的先验自适应转移（Prior-Adaptive Transfer of Skeletons, PATS）框架，该框架通过在转移过程中选择性地保留与任务相关的运动先验，同时过滤冗余的先验，从而适应一般的基于骨架的模型。PATS遵循一个标准流程，从视频中提取骨架信号，并采用在一般动作上预训练的时空主干网络。其关键贡献在于一个新颖的自适应先验转移模块，该模块通过迭代修剪和精炼作为先验选择机制进行模型压缩。在两个特定的动作识别任务（阿尔茨海默病检测和跌倒检测）上的实验表明，相较于竞争基线，PATS在性能和效率上均有一致的提升。代码将在论文接受后发布。

View on arXiv Download PDF AI Translation

cs.CV / 113 / 2607.03328

Beyond Post-Quantization: Native Hash Learning with a Dedicated HASH Token

超越后量化：具有专用 HASH 令牌的原生哈希学习

Liu, Xinze, Wang, Ding, Zhu, Hengjie, Wu, Dayan

Abstract

Efficient large-scale image retrieval requires compact representations that preserve semantic similarity under fast Hamming-space search. Deep hashing is appealing, but most existing CNN- and ViT-based methods still follow a post-quantization paradigm, where continuous visual features are first learned and binary codes are then produced by a terminal hash projection or binarization operation. This late code generation creates a feature-to-code discrepancy between the continuously optimized representation space and the discrete Hamming space used for retrieval. To address this limitation, we propose HashViT, a Vision Transformer framework for native hash token learning. Instead of treating hashing as a terminal readout, HashViT introduces a dedicated HASH token that serves as a persistent, hash-oriented retrieval state inside the transformer. The HASH token is structurally decomposed into a Hash Register for direct binary code generation and a Semantic Workspace for preserving auxiliary continuous semantics. To enable effective workspace-to-register interaction, we further design a lightweight Hash Refinement Adapter that progressively refines the Hash Register across transformer layers. As a result, binary-oriented representations are formed through token evolution within the backbone, rather than being abruptly induced by an output-level projection. HashViT is optimized with a unified objective that combines learnable semantic center supervision, class-token similarity distillation, and quantization regularization, encouraging the HASH token to encode semantically structured and compact binary representations. Extensive experiments on three widely used benchmarks demonstrate that HashViT achieves state-of-the-art or highly competitive retrieval performance while preserving the efficiency of compact Hamming codes. Code is available at https://github.com/Xinze919/HashViT.

Chinese Translation

高效的大规模图像检索需要紧凑的表示，这些表示在快速的汉明空间搜索中能够保持语义相似性。深度哈希方法具有吸引力，但大多数现有的基于卷积神经网络（CNN）和视觉变换器（ViT）的方法仍然遵循后量化范式，即首先学习连续的视觉特征，然后通过终端哈希投影或二值化操作生成二进制代码。这种后期代码生成导致了连续优化的表示空间与用于检索的离散汉明空间之间的特征与代码不一致。为了解决这一限制，我们提出了 HashViT，一种用于原生哈希令牌学习的视觉变换器框架。HashViT 不再将哈希视为终端输出，而是引入了一个专用的 HASH 令牌，作为变换器内部持久的、以哈希为导向的检索状态。HASH 令牌在结构上被分解为一个哈希寄存器，用于直接生成二进制代码，以及一个语义工作区，用于保留辅助的连续语义。为了实现有效的工作区与寄存器之间的交互，我们进一步设计了一个轻量级的哈希精炼适配器，逐步在变换器层之间精炼哈希寄存器。因此，二进制导向的表示是通过主干网络中的令牌演变形成的，而不是通过输出级别的投影突然引入的。HashViT 通过一个统一的目标进行优化，该目标结合了可学习的语义中心监督、类令牌相似性蒸馏和量化正则化，鼓励 HASH 令牌编码语义结构化和紧凑的二进制表示。在三个广泛使用的基准上进行的广泛实验表明，HashViT 实现了最先进或高度竞争的检索性能，同时保持了紧凑汉明码的高效性。代码可在 https://github.com/Xinze919/HashViT 获取。

View on arXiv Download PDF AI Translation

cs.CV / 114 / 2607.03330

GrowFields: Compositional 4D Neural Fields for Topology-Changing Plant Growth

GrowFields：用于拓扑变化植物生长的组合性4D神经场

Gajardo, Joaquin, Volpi, Michele, Mihajlovic, Marko, Tang, Siyu, Roth, Lukas, Prokudin, Sergey

Abstract

Quantifying plant growth dynamics from sparse longitudinal 3D observations is fundamental for agriculture and plant sciences. Yet, plants pose unique challenges: they undergo intricate non-rigid deformations, exhibit changing topology as new organs emerge, and often lack explicit temporal correspondences between consecutive data acquisitions due to newly formed tissue. Methods designed for general scenes struggle to model topology changes and asynchronous organ growth characteristic of plants. To address these challenges, we introduce GrowFields, a compositional dynamic neural field representation for organ-aware 4D plant growth modelling from point cloud time series. Our approach decomposes a plant into its constituent organs and aligns each organ into its own canonical coordinate frame, isolating intrinsic growth patterns from global plant motion. We then learn a shared continuous neural deformation field that models temporal dynamics across all organs, conditioned on learnable per-organ latent codes capturing organ identity and growth characteristics. The resulting modular yet unified representation naturally accommodates the asynchronous development of plant organs while remaining grounded in the practical setting of organ-level plant tracking. We evaluate GrowFields on growth sequences from four plant species, assessing geometric fitting and organ tracking accuracy using manually annotated leaf-tip trajectories. Results demonstrate consistent improvements in spatial precision, temporal coherence, and morphological fidelity over a range of existing representations.

Chinese Translation

从稀疏的纵向3D观测中量化植物生长动态对于农业和植物科学至关重要。然而，植物面临独特的挑战：它们经历复杂的非刚性变形，随着新器官的出现展现出变化的拓扑，并且由于新形成的组织，连续数据采集之间往往缺乏明确的时间对应关系。针对一般场景设计的方法在建模植物特有的拓扑变化和异步器官生长时遇到困难。为了解决这些挑战，我们提出了GrowFields，一种用于基于点云时间序列的器官感知4D植物生长建模的组合动态神经场表示。我们的方法将植物分解为其组成器官，并将每个器官对齐到其自身的标准坐标系中，从而将内在生长模式与整体植物运动隔离开来。然后，我们学习一个共享的连续神经变形场，该场建模所有器官的时间动态，并以可学习的每个器官潜在编码为条件，捕捉器官的身份和生长特征。最终得到的模块化但统一的表示自然适应植物器官的异步发展，同时仍然扎根于器官级植物跟踪的实际设置中。我们在四种植物物种的生长序列上评估了GrowFields，使用手动标注的叶尖轨迹评估几何拟合和器官跟踪精度。结果表明，在空间精度、时间一致性和形态保真度方面，相较于现有多种表示方法，GrowFields表现出了一致的改善。

View on arXiv Download PDF AI Translation

cs.CV / 115 / 2607.03358

Pathways of Visual Information Flow in Vision-Language Models

视觉-语言模型中的视觉信息流动路径

Salazar, Israfel, Frank, Stella, Oneata, Dan, Elliott, Desmond, Fierro, Constanza

Abstract

We study how visual information is routed in vision-language models (VLMs). Using causal patching on controlled synthetic and natural datasets, we find that models rely on two distinct pathways to solve visual tasks: A direct pathway, where visual information is retained in image token representations and read out by the final token at later layers, and a text-mediated pathway, where visual information is first transferred to the query tokens and then read out by the final token. Across three visual tasks, we show that pathway selection is task-dependent, and that data distribution and prompt design can also modulate which pathway is used to solve the image-based query. Moreover, using attention knockouts and corrupted-input patching, we find that these pathways are flexible, under certain interventions, models can rely on the text-mediated pathway as a fallback when the usual pathway is ablated. This behavior unifies findings in prior work and shows that ablation-based interventions can reveal what models could do rather than what they normally do. Together, our results provide a mechanistic characterization of visual information flow in VLMs and highlight the flexibility of their internal mechanisms under intervention.

Chinese Translation

我们研究了视觉-语言模型（VLMs）中视觉信息的传递方式。通过对受控的合成和自然数据集进行因果补丁分析，我们发现模型依赖两条不同的路径来解决视觉任务：一条是直接路径，其中视觉信息保留在图像标记表示中，并在后续层由最终标记读取；另一条是文本介导路径，其中视觉信息首先转移到查询标记，然后由最终标记读取。在三个视觉任务中，我们表明路径选择依赖于任务，并且数据分布和提示设计也可以调节用于解决基于图像的查询的路径。此外，通过使用注意力消融和损坏输入补丁，我们发现这些路径是灵活的，在某些干预下，模型可以依赖文本介导路径作为后备方案，当常规路径被消融时。这种行为统一了先前工作的发现，并表明基于消融的干预可以揭示模型能够做什么，而不仅仅是它们通常做的事情。总之，我们的结果提供了对VLMs中视觉信息流动的机制性表征，并突出了它们在干预下内部机制的灵活性。

View on arXiv Download PDF AI Translation

cs.CV / 116 / 2607.03365

Brand-as-Memory: Vision-Language Models Encode Causal, Mechanistically Localizable Credibility Priors for News Sources

品牌作为记忆：视觉-语言模型编码因果的、机制可定位的新闻来源可信度先验

Liao, Chih-Ting, Cao, Xin

Abstract

Vision-language models (VLMs) increasingly read news and web content as images, where the publisher's identity is visually present. We show that VLMs carry a strong source-credibility prior keyed on outlet identity, and study it along three axes. (i) Cross-model benchmark. We introduce CueTrust, a cross-model diagnostic that measures which surface source cue overrides an article's content evidence via a Source-Override Index (SOI). Across seven VLMs and five cues, the vulnerability profile is model- and scale-dependent, and the override is outlet-identity-specific and encoding-invariant, firing from the masthead name, the logo image, or the bare domain, but not from a named author, in-text authority, or page layout (clean negative controls). (ii) Mechanistic account. For the brand cue, we give a full mechanistic account: swapping only the masthead moves credibility across an approximately 11 log-odds range that tracks professional ratings (rho = 0.88 with Media Bias/Fact Check). The prior is dual-coded (name and logo), strengthens with scale, is causally formed at layers 19-21, carried by interpretable seed-stable sparse-autoencoder features, and recurs at the same relative locus in a second model family. It overrides content (about 1.8x) as a signal-magnitude effect within a shared pathway, not a privileged route. Steering the localized direction selectively reduces the override (41% reduction) and generalizes to held-out outlets, confirming the prior is causally used, not merely decodable. Deployed VLMs may thus defer to source identity over the evidence in front of them, a reliability failure we can measure across models, localize, and causally probe. We release the stimulus suite and CueTrust.

Chinese Translation

视觉-语言模型（VLMs）越来越多地将新闻和网页内容视为图像，其中出版者的身份在视觉上是显而易见的。我们展示了VLMs携带着基于出版机构身份的强源可信度先验，并从三个方面进行研究。（i）跨模型基准。我们引入了CueTrust，这是一种跨模型诊断工具，用于测量哪个表面源线索通过源覆盖指数（Source-Override Index, SOI）覆盖了文章内容证据。在七个VLMs和五个线索的实验中，脆弱性特征依赖于模型和规模，覆盖特定于出版机构身份且编码不变，来源于报头名称、标志图像或裸域，而不是来自命名作者、文本中的权威或页面布局（干净的负对照）。（ii）机制解释。对于品牌线索，我们提供了完整的机制解释：仅更换报头名称即可在大约11个对数几率范围内移动可信度，该范围与专业评级相关（rho = 0.88，使用媒体偏见/事实检查）。该先验是双重编码的（名称和标志），随着规模的扩大而增强，在第19-21层因果形成，由可解释的种子稳定稀疏自编码器特征承载，并在第二个模型家族中以相同的相对位置重复出现。它在共享路径内覆盖内容（约1.8倍），而不是特权路径。选择性地引导局部方向会显著减少覆盖（减少41%），并且可以推广到保留的出版机构，确认该先验是因果使用的，而不仅仅是可解码的。因此，已部署的VLMs可能会优先考虑源身份而非面前的证据，这是一个我们可以在模型间测量、定位和因果探测的可靠性失败。我们发布了刺激套件和CueTrust。

View on arXiv Download PDF AI Translation

cs.CV / 117 / 2607.03372

Present but Not Remembered: Auditing How Frozen VLAs Encode, Deploy, and Steer Visual History

存在但未被记住：审计冻结的视觉-语言-行动模型如何编码、部署和引导视觉历史

Liao, Chih-Ting, Cao, Xin

Abstract

A frozen vision-language-action model (VLA) receives recent observations at every decision step, yet prior work has focused on adding memory rather than asking how existing history is represented and used. We study this temporal axis using layer-resolved linear probing and causal interchange interventions across three VLAs from two architecture families. We find a three-part dissociation. First, past-frame content remains linearly decodable throughout the network. Second, information unique to history beyond the current frame is nearly absent, indicating that stored history is largely a redundant copy of the present. Third, history is causally deployed only when the current frame is heavily degraded, while the action readout progressively loses dependence on history through the network. Although all models encode history similarly, their deployment strategies differ: under the same occlusion, one architecture increasingly relies on history as a fallback, whereas the other relies on it less. We further introduce a training-free temporal deployment audit that distinguishes these regimes. In the fallback regime, re-injecting history neither repairs occlusion nor disambiguates actions, confirming the redundancy of the stored representation. In the other regime, the same intervention reliably steers the predicted action toward the donor history. These results show that steerability depends on how history is deployed rather than whether it is encoded. VLAs do not forget the past; they largely fail to represent it as information distinct from the present. Our findings suggest that future memory augmentation should inject information unique to the past rather than simply more history.

Chinese Translation

一个冻结的视觉-语言-行动模型（VLA）在每个决策步骤接收最近的观察，然而以往的研究主要集中在增加记忆上，而不是探讨现有历史是如何被表示和使用的。我们通过层分辨的线性探测和因果互换干预，研究这一时间轴，分析了来自两种架构家族的三个VLA。我们发现了一个三部分的解离。首先，过去帧的内容在整个网络中保持线性可解。其次，当前帧之外的独特历史信息几乎不存在，表明存储的历史在很大程度上是当前的冗余副本。第三，历史仅在当前帧严重退化时被因果部署，而行动输出在网络中逐渐失去对历史的依赖。尽管所有模型以相似的方式编码历史，但它们的部署策略却有所不同：在相同的遮挡条件下，一个架构越来越依赖历史作为后备，而另一个则依赖较少。我们进一步引入了一种无训练的时间部署审计，以区分这些模式。在后备模式中，重新注入历史既不能修复遮挡，也不能消除行动的歧义，确认了存储表示的冗余性。在另一种模式中，相同的干预可靠地引导预测的行动朝向捐赠历史。这些结果表明，操控性取决于历史的部署方式，而非其是否被编码。VLA并没有忘记过去；它们在很大程度上未能将其表示为与现在不同的信息。我们的发现表明，未来的记忆增强应注入独特于过去的信息，而不仅仅是更多的历史。

View on arXiv Download PDF AI Translation

cs.CV / 118 / 2607.03390

TemporalGS: Training-Free Plug-and-Play Acceleration for 3D Gaussian Splatting Rendering via Temporal Priors

TemporalGS：一种基于时间先验的无训练即插即用加速3D高斯点渲染方法

Zhou, Yuhongze, Yang, Zihao, Zuo, Xinxin, Lu, Juwei

Abstract

3D Gaussian Splatting (3DGS) has revolutionized novel-view synthesis with its fast and high-fidelity rendering. However, rendering at high FPS and low latency across various scenes remains a challenge, especially when large amounts of 3D Gaussian ellipsoids appear in the scene. To address this issue, we introduce TemporalGS, to the best of our knowledge, the first training-free plug-and-play algorithmic approach to accelerate 3DGS rendering without any post-training or post-processing, implemented on top of tile-based software rasterization. The key idea is that, instead of rendering frames independently as 3DGS, we leverage the temporal priors, represented by novel geometry and appearance buffers, etc., to reduce redundancy of Gaussian preprocessing, sorting, and rasterization operations of consecutive frames. Specifically, we propose two acceleration strategies: (1) temporal dynamic culling, which filters out Gaussians that contribute less to current frame rendering; (2) selective rendering, which renders only a small portion of tiles that cannot be approximated by the temporal priors. By adapting and interleaving these two strategies, TemporalGS yields a simple but effective plug-and-play solution for 3DGS rendering speed-up without any training. Extensive experiments show that TemporalGS achieves comparable or even better performance compared to existing state-of-the-art post-training or post-processing-based 3DGS rendering acceleration approaches. TemporalGS can significantly enhance the rendering speed of various 3DGS methods, achieving up to $1.48\times$ acceleration, while maintaining competitive rendering quality. We further extend our TemporalGS to hardware rasterization-based 3DGS to show the portability of our algorithm.

Chinese Translation

3D高斯点渲染（3DGS）以其快速和高保真度的渲染方式彻底改变了新视角合成。然而，在各种场景中以高帧率和低延迟进行渲染仍然是一个挑战，尤其是在场景中出现大量3D高斯椭球体时。为了解决这个问题，我们提出了TemporalGS，尽我们所知，这是首个无训练的即插即用算法，旨在加速3DGS渲染，无需任何后期训练或后处理，且基于基于瓦片的软件光栅化实现。其关键思想是，与3DGS独立渲染帧不同，我们利用时间先验（如新几何体和外观缓冲区等）来减少连续帧的高斯预处理、排序和光栅化操作的冗余。具体而言，我们提出了两种加速策略：（1）时间动态剔除，过滤掉对当前帧渲染贡献较小的高斯；（2）选择性渲染，仅渲染无法通过时间先验进行近似的小部分瓦片。通过适应和交错这两种策略，TemporalGS提供了一种简单而有效的即插即用解决方案，以加速3DGS渲染，无需任何训练。大量实验表明，TemporalGS在性能上与现有的基于后期训练或后期处理的3DGS渲染加速方法相当，甚至更优。TemporalGS能够显著提升各种3DGS方法的渲染速度，达到高达1.48倍的加速，同时保持竞争力的渲染质量。我们进一步将TemporalGS扩展到基于硬件光栅化的3DGS，以展示我们算法的可移植性。

View on arXiv Download PDF AI Translation

cs.CV / 119 / 2607.03422

Handwriting Trajectory Recovery with Diffusion Models

基于扩散模型的手写轨迹恢复

Nagamatsu, Hiroki, Toyota, Shoji, Uchida, Seiichi

Abstract

Recovering online pen trajectories from offline handwriting images, often referred to as handwriting trajectory recovery (stroke recovery), is an offline-to-online conversion task with applications in stroke-level editing and forensic analysis. We propose, to the best of our knowledge, the first diffusion-model-based framework for this task. Our method formulates trajectory recovery as image-conditioned generation and uses a denoising diffusion model to sample pen trajectories consistent with the observed ink trace. Through extensive quantitative evaluations on CASIA-OLHWDB (1.0-1.1), we verify that the proposed approach enables accurate recovery even for complex multi-stroke characters, substantially improving both temporal similarity (DTW/LDTW) and shape fidelity (AIoU) over representative prior methods such as PEN-Net and Cross-VAE. We further show that the model captures general stroke-order tendencies and generalizes to classes unseen during training, exemplified by cross-script transfer: a model trained on Chinese characters can recover reasonable stroke orders for Latin letters to some extent.

Chinese Translation

从离线手写图像中恢复在线笔迹轨迹，通常称为手写轨迹恢复（笔画恢复），是一项具有笔画级编辑和法医分析应用的离线到在线转换任务。我们提出了一个基于扩散模型的框架，作为我们所知的首个此类方法。我们的方法将轨迹恢复表述为图像条件生成，并使用去噪扩散模型来采样与观察到的墨迹一致的笔迹轨迹。通过在CASIA-OLHWDB（1.0-1.1）上的广泛定量评估，我们验证了所提方法能够准确恢复复杂的多笔画字符，显著提高了时间相似性（DTW/LDTW）和形状保真度（AIoU），相较于代表性的先前方法如PEN-Net和Cross-VAE。我们进一步展示了该模型捕捉到一般的笔画顺序趋势，并能够推广到训练期间未见过的类别，以跨脚本迁移为例：一个在汉字上训练的模型在一定程度上能够恢复拉丁字母的合理笔画顺序。

View on arXiv Download PDF AI Translation

cs.CV / 120 / 2607.03461

WorldBagel: Uncovering the Power of Unified Multimodal Models for Vision-Language-Action-World Modeling

WorldBagel：揭示统一多模态模型在视觉-语言-动作-世界建模中的力量

Zhao, Zelin, Shi, Min, Yuan, Bo, Xue, Haotian, Li, Jialuo, Moukheiber, Lama, Shi, Humphrey, Chen, Yongxin

Abstract

World models aim to capture environment dynamics in ways that support perception, reasoning, and action, and have recently become a central direction in Vision-Language-Action-World (VLAW) modeling. Meanwhile, unified vision-language models have demonstrated strong multimodal generation capabilities, yet their potential as world models remains underexplored. In this work, we introduce \texttt{WorldBagel}, a unified VLAW framework built on BAGEL, a modern multimodal unified model, and use it to systematically investigate the role of unification in world modeling. Across multi-task robotic manipulation and cross-domain experiments, \texttt{WorldBagel} consistently outperforms task-specific alternatives and learns action representations that are more structured and semantically aligned with visual and linguistic context. Experiments on LIBERO, Language Table, and Franka show that unification is not only an architectural convenience, but also a key factor in learning effective VLAW models, leading to consistent empirical gains and deeper insights into multimodal world modeling. Code and model checkpoints will be released upon acceptance.

Chinese Translation

世界模型旨在以支持感知、推理和行动的方式捕捉环境动态，最近已成为视觉-语言-动作-世界（VLAW）建模的一个核心方向。同时，统一的视觉-语言模型展示了强大的多模态生成能力，但其作为世界模型的潜力仍未得到充分探索。在本研究中，我们介绍了 exttt{WorldBagel}，这是一个基于现代多模态统一模型BAGEL构建的统一VLAW框架，并利用该框架系统地研究统一性在世界建模中的作用。在多任务机器人操作和跨领域实验中， exttt{WorldBagel}始终优于特定任务的替代方案，并学习到与视觉和语言上下文更加结构化和语义对齐的动作表示。在LIBERO、Language Table和Franka上的实验表明，统一性不仅是一种架构上的便利，还是学习有效VLAW模型的关键因素，带来了持续的实证收益和对多模态世界建模的更深刻见解。代码和模型检查点将在接受后发布。

View on arXiv Download PDF AI Translation

cs.CV / 121 / 2607.03470

PhysMirror: Physics-Aware Mirror Object Generation

PhysMirror：物理感知镜像对象生成

Mai, Xuan-Bach, Nguyen, Duy-Phuc, Le, Quoc-Van, Nguyen, Tam V., Do, Thanh-Toan, Le, Huu, Nguyen, Duong-Van, Tran, Minh-Triet, Le, Trung-Nghia

Abstract

Synthesizing physically accurate mirror reflections remains a fundamental challenge for modern text-to-image diffusion models, which are increasingly critical for generating synthetic training data for embodied AI and robotic perception. These models typically struggle with strict geometric constraints, leading to hallucinations that degrade the utility of the synthetic data. To address this, we introduce a novel, end-to-end physics-aware generation framework namely PhysMirror that natively enforces projective geometry through explicit 3D spatial priors. Our method automatically lifts prompted objects into 3D meshes and constructs a lightweight, mathematically exact mirror scene within a simulated environment. By rendering this explicit 3D scene, we extract precise 2D conditioning elements, such as depth maps and segmentation maps, that serve as robust guiding signals for downstream diffusion models, guiding them to generate images with physically correct mirror reflections. Moreover, we introduce Mirror Consistency Score (MCS), reference-free, fully automated metric that quantifies physical correctness using dense feature matching and vanishing point convergence. Experimental results on our newly constructed MirrOB dataset demonstrate that our approach outperforms state-of-the-art baselines in reflection accuracy and physical realism, while maintaining strong text-to-image semantic alignment, providing a reliable pipeline for embodied AI data generation. The source code is released at https://duyphuc0701.github.io/PhysMirror.

Chinese Translation

合成物理准确的镜像反射仍然是现代文本到图像扩散模型面临的一个基本挑战，这些模型在生成用于具身人工智能和机器人感知的合成训练数据中变得越来越重要。这些模型通常在严格的几何约束下表现不佳，导致幻觉现象，从而降低合成数据的实用性。为了解决这个问题，我们提出了一种新颖的端到端物理感知生成框架，称为PhysMirror，该框架通过显式的三维空间先验自然地强制实施投影几何。我们的方法自动将提示对象提升为三维网格，并在模拟环境中构建一个轻量级、数学上精确的镜像场景。通过渲染这个显式的三维场景，我们提取出精确的二维条件元素，如深度图和分割图，这些元素作为下游扩散模型的强有力引导信号，指导它们生成具有物理正确镜像反射的图像。此外，我们引入了镜像一致性评分（Mirror Consistency Score, MCS），这是一种无参考、全自动的度量标准，通过密集特征匹配和消失点收敛来量化物理正确性。在我们新构建的MirrOB数据集上的实验结果表明，我们的方法在反射准确性和物理真实感方面优于最先进的基线，同时保持强大的文本到图像语义对齐，为具身人工智能数据生成提供了可靠的管道。源代码已发布在 https://duyphuc0701.github.io/PhysMirror。

View on arXiv Download PDF AI Translation

cs.CV / 122 / 2607.03488

Learning to Generate Multiple Objects from Dense and Occluded Layouts

从密集和遮挡布局中学习生成多个对象

Ngo, Bach-Hoang, Ngo, Si-Tri, Le, Hieu, Le, Trung-Nghia

Abstract

Text-to-image diffusion models fail to generate correct object counts in dense scenes, where overlapping instances collapse into indistinguishable structures despite appearing visually plausible. We identify this as instance ownership collapse: tokens from overlapping objects interact freely through attention, while heavily occluded instances receive weak supervision due to their small visible areas. We address this through layout-aware attention biases that softly bias token interactions toward region-consistent grouping and suppress cross-instance leakage, paired with an amodal-balanced loss that amplifies gradients for occluded objects based on their occlusion level. To enable systematic evaluation, we introduce OverlapDepth-45K, a benchmark of densely overlapping scenes with amodal supervision. Our approach substantially improves count accuracy and prevents instance merging while preserving image quality. Project page: https://bachngoh.github.io/AIBL

Chinese Translation

文本到图像的扩散模型在密集场景中无法生成正确的对象计数，其中重叠实例会合并成不可区分的结构，尽管在视觉上看起来合理。我们将此现象识别为实例归属崩溃：重叠对象的标记通过注意力自由互动，而严重遮挡的实例由于其小的可见区域而受到弱监督。我们通过布局感知的注意力偏置来解决这个问题，该偏置轻柔地引导标记交互朝向区域一致的分组，并抑制跨实例泄漏，结合一种模态平衡损失，根据遮挡程度增强遮挡对象的梯度。为了实现系统评估，我们引入了OverlapDepth-45K，这是一个具有模态监督的密集重叠场景基准。我们的方法显著提高了计数准确性，并防止了实例合并，同时保持了图像质量。项目页面：https://bachngoh.github.io/AIBL

View on arXiv Download PDF AI Translation

cs.CV / 123 / 2607.03494

Towards Standardized Light Field Quality Assessment: Hybrid Subjective Benchmarking and Objective Metric Evaluation

朝向标准化光场质量评估：混合主观基准测试与客观指标评估

Mahmoudpour, Saeed, Farias, Mylene C. Q., Um, Gi-Mun, Prado, Myllena A., Seidel, Ismael, Marques, Leonardo de Sousa, Andrade, Leonardo, Zhao, Shengyang, Pagliari, Carla L

Abstract

Benchmarking immersive media coding solutions, especially in the standardization context, requires reliable and reproducible subjective quality assessment (QA) procedures, along with objective quality metrics that remain accurate across different distortion types. This paper presents a standardized workflow for light field QA, developed and deployed in the context of JPEG Pleno standardization activities, which integrates benchmark generation, a hybrid subjective evaluation, and objective metric analysis into a common workflow. The benchmark is designed to encompass not only traditional coding-only artifacts but also distortions that arise in processing pipelines in which light field encoding is accompanied with view synthesis and reconstruction techniques. A hybrid subjective method is proposed enabling fine-grained assessment by combining reference-anchored quality rating with targeted pairwise refinement in perceptually ambiguous regions. The reliability of subjective scores is verified using statistical consistency analyses between observers of two cohorts. Finally, a large set of objective metrics is systematically evaluated in terms of global prediction accuracy, local agreement in ambiguous quality regions, and robustness across distortion families. The results show that several metrics achieve strong agreement for coding-only stimuli, but their performance consistently drops when view synthesis distortions are included. The analysis further highlights the importance of view-pooling strategy in the design of future light field quality metrics. The work provides a reproducible and standardization-ready framework for fine-grained light field QA, while identifying key limitations of current objective metrics under emerging coding pipelines. The subjectively annotated dataset is publicly available at https://plenodb.jpeg.org/lfqa/objectivecfp.

Chinese Translation

在标准化背景下，对沉浸式媒体编码解决方案进行基准测试需要可靠且可重复的主观质量评估（QA）程序，以及在不同失真类型下仍能保持准确性的客观质量指标。本文提出了一种光场质量评估的标准化工作流程，该流程在JPEG Pleno标准化活动中开发和部署，整合了基准生成、混合主观评估和客观指标分析于一个共同的工作流程中。该基准设计不仅涵盖传统的编码仅失真，还包括在光场编码伴随视图合成和重建技术的处理管道中产生的失真。提出了一种混合主观方法，通过结合参考锚定质量评分与在感知模糊区域的针对性成对细化，实现细粒度评估。通过对两个组观察者之间的统计一致性分析验证了主观评分的可靠性。最后，系统评估了一大组客观指标，考量其在全局预测准确性、模糊质量区域的局部一致性以及在失真类别中的鲁棒性。结果表明，多个指标在编码仅刺激下达成了强一致性，但在包含视图合成失真时，其性能显著下降。分析进一步强调了视图池策略在未来光场质量指标设计中的重要性。本研究提供了一个可重复且准备好标准化的细粒度光场质量评估框架，同时识别了当前客观指标在新兴编码管道下的关键局限性。主观标注的数据集可在 https://plenodb.jpeg.org/lfqa/objectivecfp 公开获取。

View on arXiv Download PDF AI Translation

cs.CV / 124 / 2607.03509

Flex-Forcing: Towards a Unified Autoregressive and Bidirectional Video Diffusion Model

Flex-Forcing：朝着统一的自回归和双向视频扩散模型迈进

Ma, Xinyin, Berner, Julius, Liu, Chao, Vahdat, Arash, Nie, Weili, Wang, Xinchao

Abstract

Recent progress in large-scale generative models has substantially advanced video generation, yet existing methods remain constrained by a rigid inference paradigm. Bidirectional diffusion models excel at global coherence and visual fidelity but suffer from slow inference, while autoregressive models offer efficient and streaming generation at the cost of long-range consistency and exposure bias. We introduce Flex-Forcing, a unified training and inference framework that enables a video diffusion model to seamlessly operate under both bidirectional and autoregressive generation regimes. The core idea is a flexible chunking mechanism jointly defined over the temporal axis and denoising steps. This design allows the model to (1) perform flexible chunking according to different device budgets, (2) perform bidirectional inference across chunks for global structure planning, while generating frames autoregressively within each chunk for efficient and fine-grained synthesis, and (3) perform any-order, any-timestep autoregressive generation without the strict causal constraint. Extensive experiments on multiple video generation benchmarks demonstrate that Flex-Forcing achieves consistently better video quality, long-video stability than strong baselines with a rigid inference schedule, while offering faster inference.

Chinese Translation

近年来，大规模生成模型的进展显著推动了视频生成的发展，但现有方法仍受到严格推理范式的限制。双向扩散模型在全局一致性和视觉保真度方面表现出色，但推理速度较慢，而自回归模型则以高效和流式生成为优势，但代价是长距离一致性和曝光偏差。我们提出了Flex-Forcing，一个统一的训练和推理框架，使视频扩散模型能够在双向和自回归生成模式下无缝操作。其核心思想是一个在时间轴和去噪步骤上共同定义的灵活分块机制。该设计使模型能够（1）根据不同设备预算执行灵活分块，（2）在各分块之间进行双向推理以进行全局结构规划，同时在每个分块内自回归生成帧以实现高效和精细的合成，以及（3）在没有严格因果约束的情况下执行任意顺序、任意时间步的自回归生成。在多个视频生成基准上的广泛实验表明，Flex-Forcing在视频质量和长视频稳定性方面始终优于采用严格推理调度的强基线，同时提供更快的推理速度。

View on arXiv Download PDF AI Translation

cs.CV / 125 / 2607.03524

Perceptual Flow Matching for Few-Step Generative Modeling

感知流匹配用于少步生成建模

Zhao, Chuyang, Song, Yifei, Wang, Hongfa, Yuan, Jianlong, Zhang, Yuan, Fu, Siming, Chen, Zhineng, Deng, Huilin, Huang, Haoyang, Duan, Nan

Abstract

We propose Perceptual Flow Matching (PFM), a simple yet effective framework for few-step generation in flow-matching models. Rather than performing velocity regression in the conventional VAE latent space, PFM supervises flow matching in a perceptual feature space using pretrained perceptual models. This simple change substantially improves the few-step generation capability of flow-matching models, reducing the number of sampling steps from 35-50 to 4-8 while preserving generation quality. Unlike existing acceleration and distillation approaches, PFM requires neither teacher models nor auxiliary score networks and can be integrated into standard flow-matching training pipelines with minimal modifications. Extensive experiments on image generation, video generation, and image editing tasks demonstrate that PFM consistently produces high-quality results while producing fewer artifacts than existing distillation-based methods. We further show that perceptual supervision shifts the regression minimizer from mean-seeking to mode-seeking, biasing predictions toward on-manifold modes that remain accurate under coarse few-step integration. Our results reveal that standard flow-matching training can naturally yield high-quality few-step generators when supervised in an appropriate representation space. We hope this insight inspires future research into representation-aware objectives for efficient generative modeling.

Chinese Translation

我们提出了感知流匹配（Perceptual Flow Matching, PFM），这是一个简单而有效的框架，用于流匹配模型中的少步生成。与传统变分自编码器（VAE）潜在空间中的速度回归不同，PFM 在感知特征空间中使用预训练的感知模型监督流匹配。这一简单的改变显著提高了流匹配模型的少步生成能力，将采样步骤的数量从 35-50 降低到 4-8，同时保持生成质量。与现有的加速和蒸馏方法不同，PFM 不需要教师模型或辅助评分网络，并且可以在标准流匹配训练流程中以最小的修改进行集成。在图像生成、视频生成和图像编辑任务上的大量实验表明，PFM 始终能够产生高质量的结果，同时产生的伪影比现有的基于蒸馏的方法更少。我们进一步展示了感知监督将回归最小化器的目标从寻求均值转变为寻求模式，偏向于在粗略的少步积分下仍然准确的流形模式。我们的结果揭示，标准的流匹配训练在适当的表示空间中监督时，可以自然产生高质量的少步生成器。我们希望这一见解能够激励未来在高效生成建模中研究关注表示的目标。

View on arXiv Download PDF AI Translation

cs.CV / 126 / 2607.03553

iVISION-2DCD: A Long-Term Change Detection Dataset for Large-Scale Outdoor Construction Monitoring

iVISION-2DCD：用于大规模户外施工监测的长期变化检测数据集

Mao, Dayou, Lin, Yuchen, Ebadi, Ashkan, Zelek, John, Wong, Alexander, Chen, Yuhao

Abstract

Automation in construction is essential for reducing costs and human errors in large-scale projects. We approach the construction progress monitoring from the aspect of detecting changes in construction sites. As construction buildings continue to evolve in geometry and appearance over time, change detection need to be performed from arbitrary camera viewpoints. This necessitates developing 2D Change Detection (2DCD) algorithms that operate robustly across diverse camera perspectives at construction sites. While developing and evaluating such systems is data-intensive, no open-source benchmark dataset exists at the intersection of 2D change detection and construction automation research. Data collection using Unmanned Aerial Vehicles (UAVs) is gaining its popularity in outdoor large-scale surveying. However, in active construction sites conducting drone missions equipped with high-end sensors imposes safety concerns. Flight trajectory and collected camera viewpoints can be significantly limited. To address this critical gap, we introduce iVISION-2DCD, a large-scale synthetically generated dataset from dense LiDAR point clouds with photorealistic input images and accurate ground truth annotations. Our dataset formally defines the problem of viewpoint-robust 2DCD at construction sites and captures the inherent complexities of real-world deployment. In this paper, we present our systematic methodology for synthetic data generation, developing novel view synthesis techniques to overcome bi-temporal alignment and viewpoint diversity challenges, and implementing semi-automated semantic segmentation with change label generation while preserving challenging real-world cases. Benchmark evaluations using state-of-the-art 2DCD algorithms demonstrate that iVISION-2DCD poses novel research challenges for the computer vision and robotics communities.

Chinese Translation

施工自动化对于降低大规模项目的成本和人为错误至关重要。我们从检测施工现场变化的角度出发，研究施工进度监测。随着建筑物在几何形状和外观上随着时间的推移不断演变，需要从任意相机视角进行变化检测。这就需要开发能够在施工现场不同相机视角下稳健运行的2D变化检测（2DCD）算法。尽管开发和评估此类系统需要大量数据，但在2D变化检测与施工自动化研究交集处并不存在公开的基准数据集。使用无人机（UAV）进行数据收集在户外大规模测量中越来越受欢迎。然而，在活跃的施工现场进行配备高端传感器的无人机任务会带来安全隐患，飞行轨迹和收集的相机视角可能受到显著限制。为了解决这一关键缺口，我们引入了iVISION-2DCD，这是一个基于密集LiDAR点云合成生成的大规模数据集，包含逼真的输入图像和准确的地面真实标注。我们的数据集正式定义了施工现场视角稳健的2DCD问题，并捕捉了现实世界部署的固有复杂性。在本文中，我们展示了合成数据生成的系统方法，开发了新颖的视图合成技术以克服双时间对齐和视角多样性挑战，并实现了半自动语义分割与变化标签生成，同时保留具有挑战性的现实案例。使用最先进的2DCD算法进行的基准评估表明，iVISION-2DCD为计算机视觉和机器人社区提出了新的研究挑战。

View on arXiv Download PDF AI Translation

cs.CV / 127 / 2607.03558

Latent Clarity: Bridging World-Model Kinematics to Semantic Manifolds for Video Anomaly Anticipation

潜在清晰度：将世界模型运动学与语义流形结合用于视频异常预测

Samad, Abu Anas Ibn

Abstract

Continuous video anomaly detection is dominated by reactive Multiple Instance Learning (MIL) that collapses spatiotemporal features into scalar scores. We introduce PULS (Predictive Unified Latent Space), a continuous semantic world-model pipeline comprising two modules: a 490M-parameter KSD Bridge (Kinematic-to-Semantic Distillation) and a 16.8M-parameter Anticipatory State Predictor (ASP). The KSD Bridge maps V-JEPA 2 physical tensors into the 2048-d Qwen3-VL-Embedding-2B text-aligned hypersphere, trained on a subset of UCF-Crime. This translation alone yields a chunk-level AUROC of 0.8994 for UCF-Crime and 0.8162 for out-of-distribution XD-Violence without MIL or hierarchical fusion. We introduce and validate the Latent Clarity Hypothesis: because JEPA's temporal predictor discards aleatoric pixel noise while preserving kinematics, anticipated future representations are more semantically separable than observed presents. The ASP sharpens these anticipated future latents, achieving 44.5% mean 14-way zero-shot VQA accuracy (exceeding observation baseline by +9.6 pp). Applying the ASP to Observation Tensors collapses accuracy to 7.3% (random chance), proving Anticipation and Observation occupy distinct sub-manifolds. A Triple-Track Lead-Time protocol with an L1-surprise gate yields a peak +8.9 pp anticipatory advantage at T-0.5s (p < 0.001, N = 1,000 permutation), separating physical anticipation from static scene priors. Zero-shot transfer to XD-Violence confirms that Newtonian-invariant kinematic representations generalize out-of-distribution.

Chinese Translation

连续视频异常检测主要依赖于反应式多实例学习（Multiple Instance Learning, MIL），该方法将时空特征压缩为标量分数。我们提出了PULS（预测统一潜在空间），这是一个包含两个模块的连续语义世界模型管道：一个包含490M参数的KSD桥（运动学到语义蒸馏，Kinematic-to-Semantic Distillation）和一个包含16.8M参数的预测状态预测器（Anticipatory State Predictor, ASP）。KSD桥将V-JEPA 2物理张量映射到2048维的Qwen3-VL-Embedding-2B文本对齐超球面，该模型是在UCF-Crime子集上训练的。仅此翻译就为UCF-Crime提供了0.8994的块级AUROC，而对于分布外的XD-Violence则为0.8162，且无需MIL或层次融合。我们提出并验证了潜在清晰度假设：由于JEPA的时间预测器在保留运动学的同时丢弃了随机像素噪声，预期的未来表示在语义上比观察到的当前表示更具可分性。ASP锐化了这些预期的未来潜在表示，达到了44.5%的平均14路零样本视觉问答（VQA）准确率（比观察基线提高了9.6个百分点）。将ASP应用于观察张量时，准确率降至7.3%（随机机会），证明了预期与观察占据了不同的子流形。采用L1惊讶门的三轨领先时间协议在T-0.5s时产生了最高+8.9个百分点的预期优势（p < 0.001, N = 1,000置换），将物理预期与静态场景先验区分开。零样本转移到XD-Violence证实了牛顿不变运动学表示在分布外的泛化能力。

View on arXiv Download PDF AI Translation

cs.CV / 128 / 2607.03562

XPlainVerse: A Million-Scale Benchmark for Explainable Deepfake Detection

XPlainVerse：一个百万规模的可解释深伪检测基准

Narang, Abhijeet, Kuckreja, Kartik, Ghosh, Shreya, Khan, Muhammad Haris, Cai, Jianfei, Dhall, Abhinav

Abstract

As deepfake detection models increasingly produce natural language explanations, their reasoning often remains weakly grounded in visual artifacts, limiting reliability and user trust. Existing benchmarks mainly evaluate classification accuracy, overlooking whether explanations reflect the actual manipulations. This gap hinders progress toward deployable, explainable deepfake detection systems. To this end, we introduce XPlainVerse, a large-scale benchmark designed for joint deepfake detection and human-centered explanation. XPlainVerse comprises one million real and manipulated images, pairing authentic images from five established sources with forgeries generated by twelve off-the-shelf image editing and synthesis models. We further propose a multi-stage filtering pipeline, Edit-Check, to verify if manipulations satisfy their intended edits, enabling reliable reasoning supervision at scale. Beyond dataset scale, XPlainVerse provides two complementary explanation styles: technical explanations for expert analysis and simplified explanations optimized for non-technical users. To evaluate explanation quality beyond surface similarity, we propose novel metrics, EntityScore and EvidenceScore, that measure reasoning fidelity by checking whether explanations correctly identify manipulated entities and visual evidence. Human annotations on 2,000 explanation pairs validate our dataset quality against human judgment. We believe XPlainVerse will establish grounded explanation quality as a measurable dimension of deepfake detection and support scalable research on trustworthy, interpretable models.

Chinese Translation

随着深伪检测模型越来越多地生成自然语言解释，其推理往往在视觉伪影中缺乏坚实的基础，从而限制了可靠性和用户信任。现有基准主要评估分类准确性，忽视了解释是否反映了实际的操控。这一差距阻碍了可部署的可解释深伪检测系统的进展。为此，我们引入了XPlainVerse，一个旨在联合深伪检测和以人为中心的解释的大规模基准。XPlainVerse包含一百万张真实和操控的图像，将来自五个已建立来源的真实图像与由十二个现成的图像编辑和合成模型生成的伪造图像配对。我们进一步提出了一个多阶段过滤管道Edit-Check，以验证操控是否满足其预期的编辑，从而在大规模上实现可靠的推理监督。除了数据集规模外，XPlainVerse还提供两种互补的解释风格：针对专家分析的技术性解释和针对非技术用户优化的简化解释。为了评估解释质量超越表面相似性，我们提出了新颖的指标EntityScore和EvidenceScore，通过检查解释是否正确识别操控实体和视觉证据来衡量推理的忠实度。对2000对解释的人工注释验证了我们数据集的质量与人类判断的一致性。我们相信XPlainVerse将确立可解释质量作为深伪检测的可测量维度，并支持对可信、可解释模型的可扩展研究。

View on arXiv Download PDF AI Translation

cs.CV / 129 / 2607.03568

EPRA U-Net: An Efficient Pyramid Residual Attention Framework for Accurate Infarct Segmentation in Diffusion-Weighted MRI

EPRA U-Net：一种高效的金字塔残差注意力框架，用于扩散加权磁共振成像中的准确梗死分割

Ulutas, Hasan, Sahin, Muhammet Emin, Erkoc, Mustafa Fatih, Yuce, Esra, Tuncer, Turker, Dogan, Sengul, Kiranyaz, Serkan

Abstract

Objective: Accurate identification of acute ischemic infarcts on diffusion-weighted magnetic resonance imaging (DWI) is a critical prerequisite for reliable lesion quantification and effective clinical decision support in the management of cerebrovascular events. Methods: This study presents EPRA U-Net (Efficient Pyramid Residual Attention U-Net), a task-specific integrated architecture for efficient and accurate infarct segmentation of DWI images. In the proposed architecture, an EfficientNet-based encoder was used as a hierarchical feature extractor with a minimized parameterization. In addition, a Residual-Recurrent (R2) block (recurrent unrolling step t = 2, following the original formulation) and Atrous Spatial Pyramid Pooling (ASPP) were integrated to enhance the performance of spatial dependency modeling. Additionally, a dual attention mechanism was incorporated to highlight lesion-related activations while concurrently enabling the suppression of extraneous background responses. To prioritize lesion detection consistent with clinical imperative, a Tversky loss function was adopted, emphasizing the sensitivity of detection over its specificity during the optimization process. Results: Experimental evaluations were conducted utilizing an in-house dataset comprising 167 patients with 4,895 DWI slices; subsequently, the performance of the proposed EPRA U-Net was assessed in comparison with state-of-the-art models, specifically UNet++, DeepLabV3+, and TransUNet. The experimental results suggest that EPRA U-Net attained superior performance, evidenced by a pixel-aggregated Dice of 0.8984, a per-sample Dice of 0.9469, an IoU of 0.8155, a Recall of 0.8887, a Lesion F1 of 0.9378, and an HD95 of 11.62 px. Furthermore, a clear reduction in the rate of missed lesions, specifically by 16%, 25%, and 29%, was observed when compared with UNet++, DeepLabV3+, and TransUNet, respectively.

Chinese Translation

目的：在扩散加权磁共振成像（DWI）中准确识别急性缺血性梗死是可靠病灶量化和有效临床决策支持的关键前提。方法：本研究提出了EPRA U-Net（高效金字塔残差注意力U-Net），这是一种针对DWI图像高效且准确的梗死分割的任务特定集成架构。在所提架构中，使用基于EfficientNet的编码器作为层次特征提取器，并最小化参数化。此外，集成了残差递归（R2）块（递归展开步骤t = 2，遵循原始公式）和空洞空间金字塔池化（ASPP），以增强空间依赖建模的性能。此外，采用了双重注意力机制，以突出与病灶相关的激活，同时抑制多余的背景响应。为了优先考虑与临床需求一致的病灶检测，采用了Tversky损失函数，在优化过程中强调检测的敏感性而非特异性。结果：通过使用包含167名患者和4895个DWI切片的内部数据集进行实验评估；随后，将所提EPRA U-Net的性能与最先进的模型进行比较，具体包括UNet++、DeepLabV3+和TransUNet。实验结果表明，EPRA U-Net的性能优越，像素聚合Dice为0.8984，样本Dice为0.9469，IoU为0.8155，召回率为0.8887，病灶F1为0.9378，HD95为11.62 px。此外，与UNet++、DeepLabV3+和TransUNet相比，观察到漏检率明显降低，分别降低了16%、25%和29%。

View on arXiv Download PDF AI Translation

cs.CV / 130 / 2607.03581

PLGSA-Transformer: Periocular Landmark-Guided Attention with Occlusion-Adaptive Cosine Thresholding for Cross-Modal Masked and Unmasked Face Recognition

PLGSA-Transformer：基于眼周地标引导的注意力与适应遮挡的余弦阈值用于跨模态有遮挡和无遮挡的人脸识别

Abdullah, Dana A

Abstract

The widespread adoption of facial masks, accelerated by COVID-19 and mandated in security-sensitive settings, has exposed limitations of conventional face recognition systems. Existing approaches relying on fixed cosine thresholds, non-adaptive CNNs, and purely data-driven features fail to generalize when facial regions are occluded, creating a gap between lab performance and real-world deployability. This paper proposes PLGSA-Transformer, a cross-modal face matching framework with three contributions. First, Periocular Landmark-Guided Spatial Attention (PLGSA) uses MediaPipe landmarks to compute Gaussian heatmaps over the eye, brow, and forehead regions, fusing them with EfficientNetB3 features via a learnable residual gate to direct attention toward discriminative visible regions. Second, a Hybrid CNN-Transformer Branch reshapes feature maps into tokens processed by a two-layer Multi-Head Self-Attention encoder, enabling cross-regional dependency modelling. Third, the Occlusion-Adaptive Cosine Threshold (OACT) is a jointly trained head that raises the matching threshold in proportion to predicted occlusion severity. The model is evaluated on 858 images from Zenodo MDMFR (60%), Kaggle CelebA-HQ masked collection (25%), and author-collected images (15%), spanning both genders, ages 21-75, with varied mask types, trained via a unified loss combining contrastive verification, identity classification, and occlusion cross-entropy. PLGSA-Transformer achieves 97.22% pair verification accuracy with ROC AUC 1.0000, surpassing VGG-16-based MUFM (Abdullah et al., 2025; 95.0%), HOG classifiers (Adnan et al., 2020; 85.0%), and Feature-based Structural Measure (Shnain et al., 2017; 86.61%). These results confirm that encoding periocular geometry into attention, with Transformer modelling and occlusion-adaptive thresholds, yields a robust, scalable solution for cross-modal masked face recognition.

Chinese Translation

由于 COVID-19 的影响以及在安全敏感环境中的强制要求，面罩的广泛使用暴露了传统人脸识别系统的局限性。现有方法依赖于固定的余弦阈值、非自适应的卷积神经网络（CNN）和纯数据驱动的特征，当面部区域被遮挡时，无法实现良好的泛化，导致实验室性能与实际应用之间存在差距。本文提出了 PLGSA-Transformer，一种具有三项贡献的跨模态人脸匹配框架。首先，基于眼周地标引导的空间注意力（PLGSA）利用 MediaPipe 地标计算眼睛、眉毛和额头区域的高斯热图，并通过可学习的残差门与 EfficientNetB3 特征融合，以将注意力引导至显著的可见区域。其次，混合 CNN-Transformer 分支将特征图重塑为由两层多头自注意力编码器处理的标记，从而实现跨区域依赖建模。第三，适应遮挡的余弦阈值（OACT）是一个联合训练的头部，能够根据预测的遮挡严重程度提高匹配阈值。该模型在来自 Zenodo MDMFR（60%）、Kaggle CelebA-HQ 有遮挡集合（25%）和作者收集的图像（15%）的 858 张图像上进行了评估，涵盖了不同性别和年龄（21-75 岁），以及多种类型的面罩，通过结合对比验证、身份分类和遮挡交叉熵的统一损失进行训练。PLGSA-Transformer 实现了 97.22% 的配对验证准确率，ROC AUC 达到 1.0000，超过了基于 VGG-16 的 MUFM（Abdullah et al., 2025; 95.0%）、HOG 分类器（Adnan et al., 2020; 85.0%）和基于特征的结构度量（Shnain et al., 2017; 86.61%）。这些结果证实了将眼周几何信息编码到注意力中，结合 Transformer 建模和适应遮挡的阈值，为跨模态有遮挡人脸识别提供了一种稳健且可扩展的解决方案。

View on arXiv Download PDF AI Translation

cs.CV / 131 / 2607.03589

Vision Non-Causal Trapezoidal Mamba: Eliminating Directional Scanning in Vision SSMs with Second-Order Dynamics

视觉非因果梯形曼巴：通过二阶动力学消除视觉状态空间模型中的方向扫描

Ramachandran, Anvitha, Parikh, Dhruv, Fan, Haoyang, Kannan, Rajgopal, Prasanna, Viktor

Abstract

State Space Models (SSMs) have emerged as an alternative to Vision Transformers, yet most vision SSMs inherit directional token scanning from causal sequence modeling. While effective for sequential data, directional scanning introduces spatial bias and orientation-sensitive representations. We present Vision Non-Causal Trapezoidal Mamba (VNCT), a second-order non-causal vision SSM that enables all image tokens to interact in a single pass, eliminating direSctional scanning and achieving low single-image inference latency. VNCT exhibits more orientation-robust representations, showing reduced performance degradation under image rotations and flips, while improving Boundary IoU by up to 3.7 points, leading to more accurate boundary preservation and object localization. Across ImageNet-1K classification, COCO object detection and instance segmentation, and ADE20K semantic segmentation, VNCT consistently outperforms both directional-scanning vision SSMs and first-order non-causal SSMs. These results show that directional scanning is unnecessary for high-performance vision SSMs and that second-order non-causal state-space modeling offers a simple, efficient, and robust alternative for visual recognition.

Chinese Translation

状态空间模型（SSMs）作为视觉变换器（Vision Transformers）的替代方案逐渐崭露头角，但大多数视觉SSMs仍然继承了因果序列建模中的方向性标记扫描。尽管这种方法对序列数据有效，但方向扫描引入了空间偏差和对方向敏感的表示。我们提出了视觉非因果梯形曼巴（Vision Non-Causal Trapezoidal Mamba, VNCT），这是一种二阶非因果视觉SSM，能够使所有图像标记在单次传递中进行交互，从而消除方向扫描，并实现低单图像推理延迟。VNCT展现出更强的方向鲁棒性，在图像旋转和翻转下性能下降较小，同时边界IoU提高了最多3.7个百分点，促进了更准确的边界保持和物体定位。在ImageNet-1K分类、COCO物体检测与实例分割，以及ADE20K语义分割等任务中，VNCT始终优于方向扫描视觉SSMs和一阶非因果SSMs。这些结果表明，对于高性能视觉SSMs而言，方向扫描并非必要，而二阶非因果状态空间建模提供了一种简单、高效且鲁棒的视觉识别替代方案。

View on arXiv Download PDF AI Translation

cs.CV / 132 / 2607.03591

Responsibility Distribution Estimation in Ego-View Accident Videos with Multimodal Large Language Models

基于多模态大型语言模型的自视角事故视频责任分配估计

Tamura, Ryosei, Shin, Andrew

Abstract

Recent studies on multimodal traffic accident understanding have mainly relied on infrastructure-camera footage, satellite imagery, or structured crash records. However, such data sources are costly to deploy and maintain at large scale, and they cannot objectively capture what the driver was actually able to observe before the accident. In contrast, ego-view accident videos directly represent the driver's visual perspective, making them suitable for reasoning about avoidability and driver responsibility. In this paper, we introduce responsibility distribution estimation for ego-view traffic accident videos, a new task in which a model predicts the percentage of responsibility assigned to each involved agent. We construct an LLM-assisted responsibility annotation pipeline and fine-tune multimodal large language models under multiple input settings, including raw frames, segmentation-enhanced input, and textual descriptions. Experimental results establish a strong initial benchmark, demonstrating that multimodal LLMs can effectively perform this nuanced, constraint-based reasoning task. Our findings suggest that ego-centric accident videos provide a promising foundation for socially and legally meaningful multimodal reasoning beyond conventional accident classification and explanation tasks.

Chinese Translation

近期关于多模态交通事故理解的研究主要依赖于基础设施摄像头录像、卫星图像或结构化事故记录。然而，这些数据源在大规模部署和维护方面成本高昂，且无法客观捕捉驾驶员在事故发生前实际能够观察到的情况。相比之下，自视角事故视频直接代表了驾驶员的视觉视角，使其适合用于推理事故的可避免性和驾驶员的责任。本文提出了一种自视角交通事故视频的责任分配估计新任务，其中模型预测分配给每个相关参与者的责任百分比。我们构建了一个基于大型语言模型（LLM）的责任标注管道，并在多种输入设置下微调多模态大型语言模型，包括原始帧、增强分割输入和文本描述。实验结果建立了一个强有力的初步基准，证明了多模态LLM能够有效执行这一细致的基于约束的推理任务。我们的研究结果表明，自我中心的事故视频为超越传统事故分类和解释任务的社会和法律意义的多模态推理提供了有希望的基础。

View on arXiv Download PDF AI Translation

cs.CV / 133 / 2607.03595

Token-Based Affordance Grounding with Large Vision-Language Models

基于标记的大型视觉-语言模型的可供性定位

Lee, Seung Il, Lei, Qinqian, Xu, Daguang, Yang, Dong, Tan, Robby T., Chen, Yixin, Wang, Bo

Abstract

Affordance grounding aims to localize image regions that support a specific action, serving as a core capability for physical intelligence and embodied perception. Previous studies have primarily relied on weakly supervised learning with action labels from exocentric images. However, these methods often struggle with visually ambiguous exocentric images containing co-occurring actions; moreover, they fail to distinguish semantically similar actions because existing methods typically rely on brief action phrases that lack rich semantic details for action-specific localization. Although large vision-language models (LVLMs) encode rich action semantics and their action-conditioned textual outputs implicitly contain spatial cues, they do not directly provide action-specific spatial localization. To address these problems, we propose TokAG, a zero-shot affordance grounding framework that exploits the token-level semantic-spatial signals in LVLMs to localize action-relevant regions without external supervision. We observe that attention maps associated with different LVLM output tokens vary significantly, with many attending to irrelevant regions such as the background. Thus, we introduce a spatial-aware token-selection mechanism to systematically evaluate each output token and select the one whose attention maps exhibit dominant activation over the target object, instead of relying on arbitrary attention maps. By extracting these object-focused attention maps, we transform the LVLM's implicit semantic signals into zero-shot affordance heatmaps. Our zero-shot framework consistently outperforms prior weakly supervised approaches across multiple benchmarks, improving NSS by 10.7% on the unseen split of AGD20K and by 29.7% on HICO-IIF. The code and models will be made publicly available.

Chinese Translation

可供性定位旨在定位支持特定动作的图像区域，是物理智能和具身感知的核心能力。以往的研究主要依赖于来自外部图像的动作标签进行弱监督学习。然而，这些方法在处理包含共现动作的视觉模糊外部图像时往往面临挑战；此外，它们无法区分语义相似的动作，因为现有方法通常依赖于缺乏丰富语义细节的简短动作短语进行动作特定的定位。尽管大型视觉-语言模型（LVLMs）编码了丰富的动作语义，并且其基于动作的文本输出隐含包含空间线索，但它们并未直接提供动作特定的空间定位。为了解决这些问题，我们提出了TokAG，一个零-shot可供性定位框架，利用LVLM中的标记级语义-空间信号在没有外部监督的情况下定位与动作相关的区域。我们观察到与不同LVLM输出标记相关的注意力图显著不同，许多标记关注于背景等无关区域。因此，我们引入了一种空间感知的标记选择机制，系统地评估每个输出标记，并选择其注意力图在目标物体上表现出主导激活的标记，而不是依赖任意的注意力图。通过提取这些以物体为中心的注意力图，我们将LVLM的隐含语义信号转化为零-shot可供性热图。我们的零-shot框架在多个基准测试中始终优于以往的弱监督方法，在AGD20K的未见分割上提高了10.7%的NSS，在HICO-IIF上提高了29.7%。代码和模型将公开发布。

View on arXiv Download PDF AI Translation

cs.CV / 134 / 2607.03600

A Step Towards Robust Unsupervised Domain Adaptation via Fine-Tuning and Reinforcement Learning

通过微调和强化学习迈向稳健的无监督领域适应

Desale, Sushant Dagaji, Mishra, Rahul, Sinha, Ashutosh Kumar

Abstract

Adversarial robustness in Unsupervised Domain Adaptation (UDA) remains a significant challenge due to noisy pseudo labels and inherent distributional shifts between the clean source and adversarially perturbed target domains. Existing approaches often fail to achieve an optimal trade-off between robustness and accuracy, as pseudo-labels generated by domain-adapted models tend to introduce classification errors under adversarial attacks. In this work, we propose \textbf{SFT+RL}, a two-stage robust UDA framework that integrates Supervised Fine Tuning (SFT) and Reinforcement Learning (RL) on top of CLIP's pre-trained visual encoder. In the SFT stage, we adversarially fine-tune a linear classifier using PGD-based perturbations over the labelled source domain while partially unfreezing CLIP's projection layer. It allows adaptation to adversarial noise while preserving CLIP's rich semantic priors. We introduce a confidence-guided pseudo-labeling strategy in the RL stage to annotate unlabeled target samples progressively. Pseudo labels are filtered using a decaying confidence threshold to balance quality and coverage, and the model is trained on a composite dataset formed by combining clean source samples with high-confidence target samples. Adversarial training is applied to mixed batches of clean and adversarial examples to enhance cross-domain robustness. Comprehensive evaluations on three benchmark datasets OfficeHome~\cite{tomm-ude}, PACS~\cite{pacs}, and VisDA~\cite{visda} demonstrate the effectiveness of our approach. Notably, \textbf{SFT+RL} achieves average improvements of \textbf{10.2\%} in clean accuracy and \textbf{15.8\%} in adversarial robustness across all three datasets, outperforming existing state-of-the-art methods.

Chinese Translation

无监督领域适应（UDA）中的对抗鲁棒性仍然是一个重大挑战，原因在于干扰伪标签和干净源域与对抗扰动目标域之间固有的分布变化。现有的方法往往无法在鲁棒性和准确性之间实现最佳平衡，因为由领域适应模型生成的伪标签在对抗攻击下往往会引入分类错误。在本研究中，我们提出了 extbf{SFT+RL}，一个两阶段的稳健UDA框架，结合了监督微调（SFT）和强化学习（RL），并基于CLIP的预训练视觉编码器进行构建。在SFT阶段，我们使用基于PGD的扰动对线性分类器进行对抗微调，同时部分解冻CLIP的投影层。这使得模型能够适应对抗噪声，同时保留CLIP丰富的语义先验。在RL阶段，我们引入了一种基于置信度的伪标签策略，以逐步标注未标记的目标样本。伪标签通过衰减的置信度阈值进行过滤，以平衡质量和覆盖率，模型在由高置信度目标样本与干净源样本组合而成的复合数据集上进行训练。对干净和对抗样本的混合批次应用对抗训练，以增强跨域鲁棒性。在三个基准数据集OfficeHome~ extcite{tomm-ude}、PACS~ extcite{pacs}和VisDA~ extcite{visda}上的全面评估证明了我们方法的有效性。值得注意的是， extbf{SFT+RL}在所有三个数据集上实现了干净准确率平均提高 extbf{10.2 extperthousand}和对抗鲁棒性平均提高 extbf{15.8 extperthousand}，超越了现有的最先进方法。

View on arXiv Download PDF AI Translation

cs.CV / 135 / 2607.03612

SAF3R: Dynamic Sparse Attention for Feed-Forward 3D Reconstruction Transformers

SAF3R：用于前馈3D重建变换器的动态稀疏注意力

Deng, Jianing, Li, Yuanzhe, Wang, Jialu, Wang, Song, Chen, Tianlong, Yang, Huanrui, Hu, Jingtong

Abstract

Feed-forward 3D reconstruction (F3R) transformers have recently achieved remarkable success. However, scaling them to long image sequences remains challenging, as the quadratic complexity of cross-view global attention quickly becomes the dominant computational bottleneck. While recent efforts attempt to improve efficiency through compressed or sparse attention, they fail to fully exploit the inherent sparsity and dynamic behavior of global attention. In this work, we present a comprehensive analysis of global attention across multiple F3R transformers and reveal that attention patterns are highly heterogeneous, dynamic, and extremely sparse across layers and attention heads. Motivated by these findings, we propose SAF3R, a training-free dynamic sparse attention framework tailored to F3R transformers. SAF3R integrates tailored sparse attention mechanisms with offline head profiling and an efficient online adaptation strategy to match input-dependent attention behaviors. Extensive experiments demonstrate that SAF3R achieves high sparsity ratios while preserving camera pose estimation and 3D reconstruction quality, translating into substantial end-to-end speedup on F3R transformers compared to existing methods. Code is available at https://github.com/jndeng/SAF3R

Chinese Translation

前馈3D重建（F3R）变换器最近取得了显著成功。然而，将其扩展到长图像序列仍然面临挑战，因为跨视图全局注意力的二次复杂度迅速成为主要的计算瓶颈。尽管近期的努力试图通过压缩或稀疏注意力来提高效率，但它们未能充分利用全局注意力的固有稀疏性和动态特性。在本研究中，我们对多个F3R变换器中的全局注意力进行了全面分析，揭示了注意力模式在层和注意力头之间高度异质、动态且极其稀疏。基于这些发现，我们提出了SAF3R，一种针对F3R变换器的无训练动态稀疏注意力框架。SAF3R将量身定制的稀疏注意力机制与离线头部分析和高效的在线适应策略相结合，以匹配输入依赖的注意力行为。大量实验表明，SAF3R在保持相机姿态估计和3D重建质量的同时，实现了高稀疏比，相较于现有方法在F3R变换器上带来了显著的端到端加速。代码可在 https://github.com/jndeng/SAF3R 获取。

View on arXiv Download PDF AI Translation

cs.CV / 136 / 2607.03614

IDEAL-Bench: Indoor Dataset and Evaluation suite for Analyzing 3D Layout reasoning

IDEAL-Bench：用于分析3D布局推理的室内数据集和评估套件

Cai, Yuening, Zhou, Junwei, Qu, Youran, Tai, Yu-Wing

Abstract

Spatial question answering is the dominant paradigm for evaluating spatial intelligence in Vision-Language Models (VLMs), but it leaves a complementary axis of spatial competence under-evaluated: holistic 3D layout inference, which predicts every visible object's pose and extent from a single image in a structured form. To this end, we introduce IDEAL-Bench, an evaluation suite that requires VLMs to predict structured 3D layouts on photorealistic indoor scenes across 10 room types, scored along five numerical dimensions and a perceptual render-and-compare protocol. By operating on semantically realistic scenes with full asset substitution under controlled lighting and viewpoint, IDEAL-Bench moves beyond CLEVR-style simple geometric primitives so that any image-space discrepancy reflects spatial reasoning alone. The benchmark is built on IDEAL-Scenes, a procedurally generated dataset of 1,000 re-renderable Blender environments with ground-truth layouts. Evaluating 15 prominent VLMs reveals three findings: the task remains substantially unsolved, with the strongest model reaching only 62.1/100 overall; all models exhibit a sharp asymmetry between object recognition and geometric regression, indicating that current VLMs are trained to describe scenes rather than to measure them; model rankings partially diverge from those on QA-based and primitive-reconstruction benchmarks: top-tier consensus holds, but mid-tier rankings shift substantially. Collectively, these findings establish IDEAL-Bench as a diagnostic suite, targeting the geometric and structural competencies that QA-based evaluation cannot surface, and paving the way towards more rigorous evaluation of spatial intelligence in next-generation VLMs. Together, these findings position IDEAL-Bench as a principled diagnostic for whether future VLMs achieve genuine spatial understanding rather than linguistic approximations of it.

Chinese Translation

空间问答是评估视觉-语言模型（VLMs）空间智能的主要范式，但它忽视了一个互补的空间能力评估维度：整体3D布局推理，即从单幅图像中以结构化形式预测每个可见物体的姿态和范围。为此，我们引入了IDEAL-Bench，一个评估套件，要求VLMs在10种房间类型的照片级真实室内场景上预测结构化的3D布局，并在五个数值维度和一个感知渲染与比较协议上进行评分。通过在具有完全资产替换的语义真实场景中，在受控的光照和视角下操作，IDEAL-Bench超越了CLEVR风格的简单几何原语，使得任何图像空间的差异仅反映空间推理。该基准建立在IDEAL-Scenes之上，这是一个包含1,000个可重新渲染的Blender环境及其真实布局的程序生成数据集。对15个主要VLM的评估揭示了三个发现：该任务仍然在很大程度上未解决，最强模型的整体得分仅为62.1/100；所有模型在物体识别和几何回归之间表现出明显的不对称，表明当前的VLM被训练用于描述场景，而不是测量场景；模型排名在基于问答和原语重建基准上的部分偏离：顶级共识保持，但中级排名发生了显著变化。总体而言，这些发现确立了IDEAL-Bench作为一个诊断套件，针对基于问答的评估无法揭示的几何和结构能力，为下一代VLM中空间智能的更严格评估铺平了道路。这些发现共同将IDEAL-Bench定位为一个原则性的诊断工具，以判断未来的VLM是否实现了真正的空间理解，而不是对其的语言近似。

View on arXiv Download PDF AI Translation

cs.CV / 137 / 2607.03621

A Vision Based System for Guided and Collaborative Reconstruction of Fragmented Documents

基于视觉的碎片化文档引导与协作重建系统

Krumpek, Oliver, Leo, Diana

Abstract

This paper presents the development and evaluation of a collaborative system for real-time reconstruction of fragmented paper documents in the context of cultural heritage preservation. The developed system includes a collaborative robot, or cobot, that can fully manage the positioning of paper fragments using a specially designed vacuum-based suction attachment. This attachment enables gentle and precise positioning, ensuring the preservation of fragile materials. With this device, we are able to achieve a positioning repeatability of 0.57mm for fragments of 8cm^2. The system offers users the flexibility to choose between manual positioning, with visual guidance, or fully automated positioning performed by the cobot. To further improve the reconstruction process, AI methods for image interpretation, specifically for segmentation and positioning tasks, were applied and evaluated for their applicability to template-based reconstruction of damaged paper fragments. Our investigation provides critical insights into the performance of different local feature matching methods under different document types, taking into account rotation, scale robustness, and the degree of damage to the fragments. With a focus on the reconstruction of damaged and optically altered archival material, SE2-LoFTR, a detector-free local feature matching method, was chosen as the preferred method for the system due to its robust performance in our experiments.

Chinese Translation

本文介绍了一种用于文化遗产保护背景下实时重建碎片化纸质文档的协作系统的开发与评估。该系统包括一个协作机器人（cobot），能够通过专门设计的基于真空的吸附附件全面管理纸张碎片的定位。该附件能够实现温和而精确的定位，确保脆弱材料的保护。借助该设备，我们能够实现8cm²碎片的定位重复性为0.57mm。该系统为用户提供了手动定位（带视觉指导）或由cobot执行的全自动定位之间的灵活选择。为了进一步改善重建过程，我们应用并评估了图像解释的人工智能方法，特别是针对分割和定位任务的应用，以其在基于模板的损坏纸张碎片重建中的适用性。我们的研究提供了不同文档类型下不同局部特征匹配方法性能的关键见解，考虑了旋转、尺度鲁棒性以及碎片损坏程度。针对损坏和光学改变的档案材料的重建，选择了SE2-LoFTR作为系统的首选方法，因为它在我们的实验中表现出强大的性能。

View on arXiv Download PDF AI Translation

cs.CV / 138 / 2607.03624

RADIO1D: Elastic Representations for Condensed Vision Modeling

RADIO1D：用于凝聚视觉建模的弹性表示

Heinrich, Greg, Ranzinger, Mike, McCarthy, Collin, Bagrov, Natan, Khvedchenya, Eugene, Catanzaro, Bryan, Kautz, Jan, Tao, Andrew, Molchanov, Pavlo

Abstract

This paper challenges the assumption that vision-language models (VLMs) require fixed patch-based 2D vision features. Analyzing fine-tuned vision encoders, we find that representations become increasingly abstract and less spatially coherent during VLM training. Notably, models trained with image-text alignment (such as SigLIP2) develop a small number of specialized tokens that effectively summarize global image content. Building on this, we introduce RADIO1D, which compresses images into a compact, variable-length 1D token sequence using multi-teacher knowledge distillation and an autoencoder design. The resulting representations exhibit strong hierarchical summarization, enabling accurate scene understanding - even with a single token - and support improved composition-aware image retrieval. In VLMs, RADIO1D provides flexible accuracy-efficiency tradeoffs through adjustable token counts, delivering competitive performance on diverse multimodal benchmarks with lower computational overhead and better accuracy.

Chinese Translation

本文挑战了视觉-语言模型（VLMs）需要固定基于补丁的二维视觉特征的假设。通过分析微调后的视觉编码器，我们发现表示在VLM训练过程中变得越来越抽象且空间一致性降低。值得注意的是，经过图像-文本对齐训练的模型（如SigLIP2）发展出少量专门的标记，有效地总结了全局图像内容。在此基础上，我们引入了RADIO1D，它利用多教师知识蒸馏和自编码器设计将图像压缩为紧凑的可变长度一维标记序列。所得到的表示展现出强大的层次化摘要能力，能够实现准确的场景理解——即使只有一个标记——并支持改进的构图感知图像检索。在VLMs中，RADIO1D通过可调的标记数量提供灵活的准确性与效率权衡，在多样的多模态基准测试中展现出竞争力的性能，且计算开销更低，准确性更高。

View on arXiv Download PDF AI Translation

cs.CV / 139 / 2607.03633

Probing Identity-Specific Motion Signatures: A Controlled Diagnostic Study

探究身份特定的运动特征：一项受控诊断研究

Lei, Yingtie, Liu, Fangxun, Wu, Baicheng, Lee, Colin, Zhang, Ziheng, Yang, Junke, Tao, Zhiyuan, Huang, Xuyan, Wang, Shuheng, Koran, William, Park, Kyle, Buckwalter, Elijah H, Chiang, Cheng-Hsuan, Naik, Tejas, Yi, Daniel, Chao, Wei-Lun

Abstract

Identity recognition (e.g., person, animal re-identification) has traditionally relied heavily on static appearance cues. Yet motion--consistent, individual-specific dynamics--can provide a complementary and potentially more robust signature, especially when appearance is weak or variable. This raises a fundamental question: when identity-specific motion cues are clearly present, to what extent do modern video models use them for recognition? To investigate this question, we conduct a systematic diagnostic study and introduce BALLER120, a controlled benchmark of 120 professional basketball players performing free-throws. By focusing on the same multi-phase action across individuals, BALLER120 reduces action-level variation and identity-correlated acquisition biases, enabling fine-grained analysis of identity-specific kinematic patterns. We find that modern video models can predict identity accurately from RGB videos, but often rely on static appearance cues such as faces and jersey regions, even when informative motion cues are available. Strikingly, when appearance is suppressed through silhouette-only or skeleton-only inputs, the same model architectures shift toward motion micro-patterns (e.g., foot placement and elbow bending). Despite containing less visual information, appearance-suppressed representations achieve competitive accuracy and stronger robustness to appearance shifts. Our qualitative analyses further show that appearance-suppressed models attend to distinctive motion patterns across individuals. Overall, our study demonstrates that identity-specific motion signatures are present, informative, and learnable, but modern video models may overlook them in favor of easier static shortcuts unless appearance cues are explicitly suppressed.

Chinese Translation

身份识别（例如，人物、动物重识别）传统上在很大程度上依赖于静态外观线索。然而，运动——一致的、个体特定的动态——可以提供一种互补且潜在更强的特征，尤其是在外观较弱或变化时。这引发了一个基本问题：当身份特定的运动线索明显存在时，现代视频模型在多大程度上利用它们进行识别？为了探讨这个问题，我们进行了一项系统的诊断研究，并引入了BALLER120，这是一个受控基准，包含120名职业篮球运动员进行罚球的表现。通过关注个体之间相同的多阶段动作，BALLER120减少了动作级别的变异和与身份相关的获取偏差，从而能够对身份特定的运动模式进行细致分析。我们发现，现代视频模型能够从RGB视频中准确预测身份，但往往依赖于静态外观线索，如面部和球衣区域，即使在有信息丰富的运动线索可用时。令人惊讶的是，当通过仅使用轮廓或骨架输入来抑制外观时，相同的模型架构转向运动微模式（例如，脚的位置和肘部的弯曲）。尽管包含的视觉信息较少，但抑制外观的表示在准确性上具有竞争力，并且对外观变化更具鲁棒性。我们的定性分析进一步表明，抑制外观的模型关注个体之间独特的运动模式。总体而言，我们的研究表明，身份特定的运动特征是存在的、信息丰富的且可学习的，但现代视频模型可能会忽视它们，而偏向于更简单的静态捷径，除非外观线索被明确抑制。

View on arXiv Download PDF AI Translation

cs.CV / 140 / 2607.03644

Moonstone: A Multimodal Foundation Model and Benchmark for Lunar Remote Sensing

月石：一种用于月球遥感的多模态基础模型及基准

Prasad, Ayush, Mazumder, Swarnalee

Abstract

Decades of orbital missions have produced multi-modal remote sensing data for the Moon, spanning optical imagery, spectroscopy, thermal emission, radar, gravity, and elemental composition. Yet these datasets remain fragmented across archives, and no benchmark exists for evaluating machine learning on lunar data. We introduce Moonstone, the first multi-modal foundation model benchmark for lunar remote sensing. Our contributions are: (1) a 28-channel, 128 pixels-per-degree (~237 m) global lunar pretraining dataset from seven instrument families across five missions, (2) MG-MAE, a modality-grouped masked autoencoder with per-group convolutional tokenizers, a shared Vision Transformer encoder, attention masking for missing modalities, coverage-adaptive masking for heterogeneous spatial coverage, and spectral continuity regularization for physically plausible reconstructions, and (3) a benchmark of six downstream tasks covering classification, regression, and segmentation. MG-MAE pretrained features outperform scratch baselines on all tasks and surpass both ImageNet-pretrained and vanilla MAE baselines by large margins. Data and code are available at https://huggingface.co/datasets/ayushprd/Moonstone and https://github.com/ayushprd/Moonstone .

Chinese Translation

数十年的轨道任务产生了多模态的月球遥感数据，包括光学图像、光谱、热辐射、雷达、重力和元素组成。然而，这些数据集在档案中仍然是碎片化的，且尚无用于评估月球数据的机器学习基准。我们介绍了月石（Moonstone），这是第一个用于月球遥感的多模态基础模型基准。我们的贡献包括：（1）来自五个任务的七个仪器系列的28通道、每度128像素（约237米）的全球月球预训练数据集；（2）MG-MAE，一种模态分组的掩蔽自编码器，具有每组卷积标记器、共享的视觉变换器编码器、针对缺失模态的注意力掩蔽、针对异构空间覆盖的覆盖自适应掩蔽，以及用于物理上合理重建的光谱连续性正则化；（3）涵盖分类、回归和分割的六个下游任务的基准。MG-MAE的预训练特征在所有任务上均优于从零开始的基线，并且大幅超越了ImageNet预训练和普通MAE基线。数据和代码可在https://huggingface.co/datasets/ayushprd/Moonstone和https://github.com/ayushprd/Moonstone获取。

View on arXiv Download PDF AI Translation

cs.CV / 141 / 2607.03647

Do Medical Vision Language Models Actually See? A Counterfactual Grounding Framework and Hard-Negative Contrastive Training for Visually-Reliant Medical VLMs

医学视觉语言模型真的能“看”吗？一个反事实基础框架和针对视觉依赖医学 VLM 的困难负样本对比训练

Zafar, Anas, Murali, Leema Krishna, Bharadwaj, Siddhant, Vashist, Ashish, Wu, Jia

Abstract

Large vision language models (VLMs) report strong accuracy on medical question-answering, yet it remains unclear whether they reason from visual evidence or exploit textual shortcuts. We introduce a counterfactual evaluation framework that decouples visual and textual contributions by substituting input images with controlled surrogates blank, pixel-shuffled, image-absent, and CLIP-retrieved hard negatives and derive a suite of grounding metrics including the Visual Reliance Score (VRS) and Visual Hallucination Rate (VHR). We further introduce CORAL (COntrastive Retrieval-Augmented Learning), a 7B-parameter LoRA fine-tune of Qwen2.5-VL-7B trained with a Contrastive Grounding Objective (CGO) that penalises answer invariance under hard-negative image swaps. On a paired controlled evaluation across four closed-form medical VQA benchmarks (PathVQA, PMC-VQA, SLAKE, VQA-RAD; n=400 total), CORAL improves macro accuracy by +6.7 pp (P(Delta>0)=0.988) and reduces VHR by 8.0 pp (P<0.001) over the matched Qwen2.5-VL-7B base; neither MedVLThinker RL variant achieves a significant gain on either metric. Cross-domain diagnostics further reveal that image substitution costs only <=6.5 pp on medical benchmarks versus 48-61 pp on general-domain tasks, situating the grounding gap that CGO targets. We discuss evaluation limitations openly including train/eval benchmark overlap and underpowered secondary metrics and release our framework, training code, and model weights to support reproducible grounding audits of medical VLMs.

Chinese Translation

大型视觉语言模型（VLMs）在医学问答中报告了较强的准确性，但仍不清楚它们是否是基于视觉证据进行推理，还是利用文本捷径。我们引入了一个反事实评估框架，通过用控制的替代品（如空白、像素打乱、缺失图像和 CLIP 检索的困难负样本）替换输入图像，从而解耦视觉和文本的贡献，并推导出一系列基础度量，包括视觉依赖分数（Visual Reliance Score, VRS）和视觉幻觉率（Visual Hallucination Rate, VHR）。我们进一步介绍了 CORAL（对比检索增强学习，COntrastive Retrieval-Augmented Learning），这是一个具有 70 亿参数的 Qwen2.5-VL-7B 的 LoRA 微调模型，采用对比基础目标（Contrastive Grounding Objective, CGO），该目标对困难负样本图像交换下的答案不变性进行惩罚。在四个封闭形式医学 VQA 基准（PathVQA、PMC-VQA、SLAKE、VQA-RAD；总计 n=400）的配对控制评估中，CORAL 的宏观准确率提高了 +6.7 个百分点（P(Delta>0)=0.988），并将 VHR 降低了 8.0 个百分点（P<0.001），相比于匹配的 Qwen2.5-VL-7B 基础模型；而 MedVLThinker RL 的任何变体在这两个指标上均未取得显著提升。跨领域诊断进一步揭示，医学基准上图像替换的成本仅为 <=6.5 个百分点，而在一般领域任务上则为 48-61 个百分点，突显了 CGO 目标所针对的基础差距。我们公开讨论评估的局限性，包括训练/评估基准重叠和次要指标的能力不足，并发布我们的框架、训练代码和模型权重，以支持医学 VLM 的可重复基础审计。

View on arXiv Download PDF AI Translation

cs.CV / 142 / 2607.03650

ClinOCR-Bench: A Comprehensive Clinical Scanned Document Dataset for Optical Character Recognition Model Evaluation

ClinOCR-Bench：一个用于光学字符识别模型评估的综合临床扫描文档数据集

Hsu, Enshuo, Zhou, Jin, Roberts, Kirk

Abstract

Extracting textual information from scanned medical documents, such as external laboratory reports and manually filled forms, has been a major challenge in modern electronic health records (EHRs). Recent advancements in vision language models (VLMs) have shown great promise over traditional OCR tools. However, at this point, most clinical OCR studies were conducted on private, institutional data. To our knowledge, there are few publicly available datasets for evaluating OCR models in the clinical domain. Furthermore, common scanning artifacts that undermine OCR performance are not reflected in those datasets, leaving a systematic evaluation unfeasible. Therefore, we release a publicly available, realistic-looking OCR benchmark dataset, ClinOCR-Bench, with 384 scanned images across 6 subsets: Normal, Handwriting, Poor Quality, Rotation, Tables, and Mix-artifacts. ClinOCR-Bench features: 1) diverse document types and layouts, 2) full coverage of common EHR scan artifacts, 3) protected health information-free, 4) template-aware train/test split, and 5) adequate sample size for OCR benchmarking. Baseline OCR performance was evaluated using state-of-the-art open-weight and proprietary VLMs. The dataset and documentation are available on GitHub (https://github.com/ClinOCR-Bench/ClinOCR-Bench).

Chinese Translation

从扫描的医疗文档中提取文本信息，例如外部实验室报告和手动填写的表单，一直是现代电子健康记录（EHRs）中的一项重大挑战。最近在视觉语言模型（VLMs）方面的进展显示出相较于传统光学字符识别（OCR）工具的巨大潜力。然而，目前大多数临床OCR研究都是基于私有的机构数据。我们了解到，公开可用的临床领域OCR模型评估数据集非常有限。此外，常见的扫描伪影对OCR性能的影响在这些数据集中并未得到反映，导致系统评估变得不可行。因此，我们发布了一个公开可用、外观逼真的OCR基准数据集ClinOCR-Bench，包含384张扫描图像，分为6个子集：正常、手写、低质量、旋转、表格和混合伪影。ClinOCR-Bench的特点包括：1）多样化的文档类型和布局，2）全面覆盖常见的EHR扫描伪影，3）不包含受保护的健康信息，4）考虑模板的训练/测试划分，以及5）足够的样本量用于OCR基准测试。基于最先进的开放权重和专有的VLMs评估了基线OCR性能。数据集及其文档可在GitHub上获取（https://github.com/ClinOCR-Bench/ClinOCR-Bench）。

View on arXiv Download PDF AI Translation

cs.CV / 143 / 2607.03657

ViPo-MLLM: Visual-Pose Multimodal LLM for Gloss-Free Sign Language Translation

ViPo-MLLM：无注释手势语言翻译的视觉-姿态多模态大语言模型

Hasanaath, Ahmed Abul, Xu, Bicheng, Hossain, Mir Rayat Imtiaz, Sigal, Leonid, Luqman, Hamzah

Abstract

Gloss-free Sign Language Translation (SLT) translates sign language videos into spoken-language sentences without gloss annotations, avoiding costly labeling but requiring fine-grained modeling of hands, body, and facial cues. Existing methods often use single-modality or weakly fused features, limiting performance. We propose ViPo-MLLM, a framework that integrates spatio-temporal RGB and human pose features. Dedicated encoders model intra-modal dynamics and cross-modal attention captures long-range dependencies. The fused representation is conditioned with a structured prompt and processed by an LLM trained with contrastive and language modeling objectives. The proposed model was evaluated on the PHOENIX14T and CSL-Daily datasets and achieved new state-of-the-art results on both datasets. Moreover, the ViPo-MLLM model attained competitive performance compared to gloss-based recognition approaches, confirming the effectiveness of the proposed pose cues and cross-modal attention mechanisms.

Chinese Translation

无注释手势语言翻译（SLT）将手势语言视频翻译为口语句子，无需注释，避免了昂贵的标注成本，但需要对手、身体和面部线索进行细致建模。现有方法通常使用单一模态或弱融合特征，限制了性能。我们提出了ViPo-MLLM，一个集成时空RGB和人体姿态特征的框架。专用编码器对模态内部动态进行建模，而跨模态注意力捕捉长距离依赖关系。融合表示通过结构化提示进行条件化，并由经过对比和语言建模目标训练的LLM处理。所提模型在PHOENIX14T和CSL-Daily数据集上进行了评估，并在这两个数据集上取得了新的最先进结果。此外，ViPo-MLLM模型在与基于注释的识别方法的比较中表现出竞争力，确认了所提姿态线索和跨模态注意力机制的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 144 / 2607.03661

From Geometric Labels to Semantic Understanding of Indoor Building Components Using Multimodal Large Language Models

从几何标签到室内建筑组件的语义理解：基于多模态大型语言模型

Jing, Shuju, Yin, Chao

Abstract

Point cloud-based understanding has become an important enabler for facility operation and maintenance involving indoor building components. However, existing methods output only discrete labels without explaining component functions or natural language interactions. This paper proposes Building-MLLM, a point cloud-centered multimodal large language model (MLLM) for indoor components, which models point clouds and instructions to generate responses across Simple Recognition, Complex Captioning, and Multi-Engineering Question Answering tasks. Building-MLLM addresses semantic concentration through four domain-specific mechanisms: Point Information Enhancer for task-relevant semantics, Geometry-Preserving Regularization preventing geometric erosion, fixed textual prefix for domain stabilization, and multi-dimensional LoRA balancing recognition with reasoning. A multi-constraint progressive instruction-generation engine is developed to compile a synthetic point cloud-text dataset with 4198 objects, 37,782 instruction-following pairs, and 47 categories. Experiments show that Building-MLLM achieves 88.00%, 65.10%, and 68.14% on the three task types, respectively, demonstrating superior indoor component language understanding and providing initial generalizability in transfer inference on other real-world datasets.

Chinese Translation

基于点云的理解已成为涉及室内建筑组件的设施运营和维护的重要推动力。然而，现有方法仅输出离散标签，而未解释组件功能或自然语言交互。本文提出了Building-MLLM，一种以点云为中心的多模态大型语言模型（MLLM），用于室内组件，能够建模点云和指令，以生成在简单识别、复杂描述和多工程问答任务中的响应。Building-MLLM通过四个特定领域的机制解决语义集中问题：任务相关语义的点信息增强器、预防几何侵蚀的几何保留正则化、用于领域稳定的固定文本前缀，以及平衡识别与推理的多维LoRA。我们开发了一种多约束渐进指令生成引擎，以编制包含4198个对象、37,782个指令跟随对和47个类别的合成点云-文本数据集。实验表明，Building-MLLM在三种任务类型上分别达到了88.00%、65.10%和68.14%的成绩，展示了卓越的室内组件语言理解能力，并在其他真实世界数据集上的迁移推理提供了初步的普适性。

View on arXiv Download PDF AI Translation

cs.CV / 145 / 2607.03696

IPDiff: Diffusion-driven ORSI Salient Object Detection with Information Reconstruction and Multi-Prior Guidance

IPDiff：基于扩散驱动的信息重构与多先验引导的光学遥感图像显著性目标检测

Li, Gongyang, Bai, Zhen, Cong, Runmin, Zeng, Dan, Lin, Weisi, Zhang, Xiao-Ping

Abstract

Existing Salient Object Detection in Optical Remote Sensing Image (ORSI-SOD) methods mainly adopt the static inference strategy, which uses fixed trained model parameters for saliency inference in the testing phase. This means that even if the generated saliency map has errors, it cannot be further optimized. In this paper, we propose the novel IPDiff, a Diffusion-driven ORSI-SOD method with Information Reconstruction and Multi-Prior Guidance. We build IPDiff based on a unique dynamic optimization strategy, which endows IPDiff with the ability to iteratively optimize saliency maps with a dynamic parameter. Specifically, we formulate ORSI-SOD as a conditional diffusion problem in IPDiff. IPDiff first extracts informative conditional priors from ORSIs, including the saliency prior and the hierarchical priors, in the prior network with the assistance of the information reconstruction-driven attention module. The saliency prior can provide positional information of salient objects, while the hierarchical priors can provide specific detail and semantic information of salient objects. Under the guidance of these priors, IPDiff then iteratively denoises random noise as the timestep dynamically changes in the denoising network, generating saliency maps that are close to ground truths. Notably, we simultaneously supervise IPDiff in both spatial and spectral domains through a hybrid loss function to achieve efficient network training. Comprehensive experiments on public ORSSD, EORSSD, and ORSI-4199 datasets demonstrate that our proposed IPDiff achieves the best performance compared to 46 state-of-the-art methods. The code and results of our method are available at https://github.com/MathLee/IPDiff.

Chinese Translation

现有的光学遥感图像显著性目标检测（ORSI-SOD）方法主要采用静态推理策略，即在测试阶段使用固定的训练模型参数进行显著性推理。这意味着即使生成的显著性图存在错误，也无法进一步优化。本文提出了一种新颖的IPDiff方法，这是一种基于扩散驱动的ORSI-SOD方法，结合了信息重构和多先验引导。我们基于独特的动态优化策略构建IPDiff，使其具备以动态参数迭代优化显著性图的能力。具体而言，我们将ORSI-SOD公式化为IPDiff中的条件扩散问题。IPDiff首先在先验网络中提取来自光学遥感图像的信息性条件先验，包括显著性先验和层次先验，并借助信息重构驱动的注意力模块。显著性先验可以提供显著目标的位置相关信息，而层次先验则可以提供显著目标的具体细节和语义信息。在这些先验的引导下，IPDiff在去噪网络中随着时间步的动态变化迭代去噪随机噪声，生成接近真实值的显著性图。值得注意的是，我们通过混合损失函数在空间和光谱域同时监督IPDiff，以实现高效的网络训练。在公共ORSSD、EORSSD和ORSI-4199数据集上的综合实验表明，我们提出的IPDiff相比46种最先进的方法取得了最佳性能。我们方法的代码和结果可在https://github.com/MathLee/IPDiff获取。

View on arXiv Download PDF AI Translation

cs.CV / 146 / 2607.03715

Leveraging Pathology Co-occurrence for Test-Time Adaptation in Chest X-Ray Diagnosis

利用病理共现进行胸部X光诊断中的测试时适应

Jeong, Woojin, Choi, Yujin, Kim, Dongbin, Park, Soyeon, Lee, Jaewook

Abstract

Medical imaging models often degrade when deployed at new clinical sites due to differences in imaging equipment, protocols, and patient populations. Test-time adaptation (TTA) addresses this by updating a pretrained model using only unlabeled target data, without access to source data. However, existing TTA methods were designed for single-label classification on natural image benchmarks, minimizing entropy uniformly across all samples without considering label dependencies. This overlooks a key property of multi-label medical imaging: pathologies do not occur independently but exhibit structured co-occurrence patterns. In this work, we propose Co-occurrence Weighted Adaptation (CoWA), which leverages disease co-occurrence patterns as a reliability signal for adaptation. CoWA estimates label co-occurrence structure from model predictions and downweights samples that deviate from expected patterns, enabling adaptation to rely more on consistent predictions while reducing the impact of noisy ones. We evaluate CoWA on chest X-ray benchmarks under domain shifts and demonstrate consistent improvements over established baselines.

Chinese Translation

医学影像模型在新的临床场所部署时，常常因成像设备、协议和患者群体的差异而性能下降。测试时适应（Test-time Adaptation, TTA）通过仅使用未标记的目标数据来更新预训练模型，而不访问源数据，从而解决了这一问题。然而，现有的TTA方法是为自然图像基准的单标签分类设计的，均匀地最小化所有样本的熵，而未考虑标签之间的依赖关系。这忽视了多标签医学影像的一个关键特性：病理并非独立发生，而是表现出结构化的共现模式。在本研究中，我们提出了共现加权适应（Co-occurrence Weighted Adaptation, CoWA），利用疾病共现模式作为适应的可靠性信号。CoWA从模型预测中估计标签共现结构，并对偏离预期模式的样本进行降权，使适应更依赖于一致的预测，同时减少噪声样本的影响。我们在胸部X光基准测试中评估了CoWA在领域转移下的表现，并展示了其在既定基准上的一致性改进。

View on arXiv Download PDF AI Translation

cs.CV / 147 / 2607.03732

ProxyUp: Training-Free Proxy-Conditioned Video Generation for Controllable Dynamics

ProxyUp：无训练的代理条件视频生成用于可控动态

Zhou, Zanwei, Cen, Jiazhong, Fang, Jiemin, He, Yumeng, Yang, Chen, Li, Sikuang, Meng, Fanpeng, Bao, Zhikuan, Shen, Wei, Tian, Qi

Abstract

Precise control over complex dynamics remains challenging for modern video generative models, as text prompts alone often cannot specify physically plausible, fine-grained motion and interactions. We introduce $\textit{proxy-conditioned video generation}$, where a coarse proxy video from physics-based simulation or real-world recording serves as a dynamics carrier to control foreground object motion. Given a proxy video and a text prompt, the goal is to synthesize a new video that preserves the proxy dynamics while generating novel content and plausible interactions aligned with the prompt. Since paired proxy-target videos are difficult to obtain, we propose $\textbf{ProxyUp}$, a training-free framework built on pretrained video generative models. ProxyUp first inverts the proxy video into an intermediate latent representation and applies $\textbf{region-wise latent noising}$, preserving motion-critical proxy latents while injecting noise into regions intended for text-driven regeneration. To mitigate the distribution mismatch and weak foreground-background coupling introduced by this heuristic latent composition, we further propose $\textbf{Stochastic Flow Relaxation (SFR)}$, which progressively relaxes the composed latent toward the model's learned distribution before ODE sampling. Experiments on both simulation and real-world proxies show that ProxyUp outperforms strong video editing and motion transfer baselines in dynamic fidelity and text alignment.

Chinese Translation

对复杂动态的精确控制仍然是现代视频生成模型面临的挑战，因为仅凭文本提示往往无法指定物理上合理的、细致的运动和交互。我们提出了“代理条件视频生成”（proxy-conditioned video generation），其中来自基于物理的模拟或现实世界录制的粗略代理视频作为动态载体，用于控制前景物体的运动。给定一个代理视频和一个文本提示，目标是合成一个新视频，该视频在保留代理动态的同时生成与提示一致的新内容和合理的交互。由于配对的代理-目标视频难以获得，我们提出了“ProxyUp”，这是一个基于预训练视频生成模型的无训练框架。ProxyUp首先将代理视频反转为中间潜在表示，并应用“区域级潜在噪声”（region-wise latent noising），在保留运动关键的代理潜在表示的同时向计划进行文本驱动再生的区域注入噪声。为了减轻这种启发式潜在组合所引入的分布不匹配和弱前景-背景耦合，我们进一步提出了“随机流松弛”（Stochastic Flow Relaxation, SFR），该方法在ODE采样之前逐步将组合的潜在表示松弛到模型学习的分布。对模拟和现实世界代理的实验表明，ProxyUp在动态保真度和文本对齐方面优于强大的视频编辑和运动转移基线。

View on arXiv Download PDF AI Translation

cs.CV / 148 / 2607.03738

Attending to Multimodal Generation One Token at a Time

逐个标记关注多模态生成

Gupta, Varun, Gandhi, Vineet, Tapaswi, Makarand

Abstract

Multimodal large language models (MLLMs) generate responses autoregressively, integrating visual and linguistic information in an evolving context. Prior work on interpretability has focused on individual layers and circuits (where), leaving the token-level dynamics of multimodal computation during generation (when) underexplored. We address this gap and study attention shifts as per semantic role; tracking model attention to image, text, instruction, and previously generated tokens, One Token at a Time (OTaT). We introduce multimodal tasks that require explicit switching between visual and textual context within a single response. Across two mainstream model families and four open-weight MLLMs of varying sizes, we establish consistent patterns: attention to image peaks at tokens requiring image-derived information, instruction tokens are revisited during task transitions, and attention to previously generated tokens increases as the generation progresses. Causal attention blocking interventions validate the functional role of these trends. We profile model behavior under disrupted attention and observe responses falling back to language priors, or exhibiting cross-modal leakage, denial, or recovery. Finally, informed of the attention dynamics through our novel analysis, we propose a simple test-time intervention to boost attention to the relevant modality at the right time, significantly improving multimodal task performance.

Chinese Translation

多模态大型语言模型（MLLMs）以自回归方式生成响应，在不断演变的上下文中整合视觉和语言信息。先前关于可解释性的研究主要集中在单个层和电路（何处），而在生成过程中多模态计算的标记级动态（何时）则未得到充分探讨。我们解决了这一空白，研究了基于语义角色的注意力转移；跟踪模型对图像、文本、指令和先前生成标记的注意力，逐个标记（One Token at a Time, OTaT）。我们引入了需要在单个响应中明确切换视觉和文本上下文的多模态任务。在两种主流模型系列和四个不同规模的开放权重MLLMs中，我们建立了一致的模式：对图像的注意力在需要图像衍生信息的标记处达到峰值，指令标记在任务转换期间被重新关注，而对先前生成标记的注意力随着生成的进行而增加。因果注意力阻断干预验证了这些趋势的功能作用。我们在干扰注意力的情况下分析模型行为，观察到响应回退到语言先验，或表现出跨模态泄漏、否认或恢复。最后，通过我们新颖分析所获得的注意力动态信息，我们提出了一种简单的测试时干预措施，以在适当的时机增强对相关模态的关注，从而显著提高多模态任务的表现。

View on arXiv Download PDF AI Translation

cs.CV / 149 / 2607.03752

EmCom-Diffusion: Probing Visual Reflection in Emergent Languages via Image Generation

EmCom-Diffusion：通过图像生成探测新兴语言中的视觉反射

Omoto, Haruumi, Taniguchi, Tadahiro

Abstract

Measuring the extent to which emergent languages encode the visual content of their inputs is an open problem. We refer to this property as visual reflection: the extent to which emergent messages preserve information about their source images that can be recovered without appeal to the speaker-listener pair that produced them. Existing metrics measure it only indirectly, through proxies such as human-defined concept inventories, natural-language captions, structural distance correlations, or Referential Game accuracy, each of which can either miss visual content the message encodes or credit content it does not. We propose EmCom-Diffusion, an evaluation framework that measures visual reflection directly: it reconstructs each input image from its emergent message and compares the reconstruction with the original image itself, rather than with human-defined targets. Concretely, it finetunes a pretrained text-to-image diffusion model on (image, emergent-message) pairs and scores visual reflection as the perceptual similarity between the reconstructed and original images, operating generatively rather than discriminatively. Instantiating it on MS-COCO with a Referential Game, we validate the metric against random and fixed-token baselines under three pretrained visual encoders, and compare it against four existing metrics (CBM, supervised translation, TopSim, and R@1). EmCom-Diffusion captures visual content the other metrics miss.

Chinese Translation

衡量新兴语言在多大程度上编码其输入的视觉内容是一个未解决的问题。我们将这一属性称为视觉反射：新兴信息在多大程度上保留了关于其源图像的信息，这些信息可以在不依赖产生它们的说话者-听众对的情况下被恢复。现有的度量方法仅通过代理间接测量这一属性，例如人类定义的概念清单、自然语言描述、结构距离相关性或参考游戏准确性，每种方法都可能遗漏信息所编码的视觉内容或错误地认可其不包含的内容。我们提出了EmCom-Diffusion，一个直接测量视觉反射的评估框架：它从每个新兴信息重建输入图像，并将重建结果与原始图像进行比较，而不是与人类定义的目标进行比较。具体而言，它在（图像，新兴信息）对上微调一个预训练的文本到图像扩散模型，并将视觉反射评分为重建图像与原始图像之间的感知相似性，采用生成方式而非判别方式。在MS-COCO上通过参考游戏实例化该框架，我们在三个预训练视觉编码器下验证该度量与随机和固定标记基线的对比，并与四种现有度量（CBM、监督翻译、TopSim和R@1）进行比较。EmCom-Diffusion捕捉了其他度量遗漏的视觉内容。

View on arXiv Download PDF AI Translation

cs.CV / 150 / 2607.03754

Exploring SAM Supervision for Fine-Grained UAV Target Segmentation under Data Scarcity

在数据稀缺条件下探索SAM监督用于细粒度无人机目标分割

Tran, Le-Anh

Abstract

Unmanned aerial vehicle (UAV) target segmentation remains challenging due to the small size of objects, appearance variations, cluttered backgrounds, and the scarcity of densely annotated data. These factors hinder the performance and practical deployment of lightweight segmentation models in real-world UAV applications. To address this problem, this paper investigates the use of SAM3 (Segment Anything Model 3) as a pseudo-label generator for training compact segmentation networks. Specifically, two supervision paradigms are explored: (i) direct pseudo-supervision using unaltered SAM3-generated masks, and (ii) a refinement strategy that re-applies SAM3 to localized image patches for improved mask quality. Based on these paradigms, a two-stage SAM3-guided pseudo-label generation framework is proposed. In the first stage, SAM3 generates coarse masks for initial object localization. The localized regions are subsequently cropped into patches and processed by SAM3 again to generate fine masks with accurate object boundaries and discard false positives. The resulting coarse and fine masks are then used as pseudo-labels to optimize a lightweight network, termed IPS-Seg, which consists of three components: an IdentityFormer backbone for feature extraction, an Atrous Spatial Pyramid Pooling module for multi-scale context aggregation, and a PixelShuffle-based decoder for spatial resolution recovery. Extensive experiments under multiple supervision settings demonstrate the effectiveness of the proposed framework. The results show that IPS-Seg achieves a favorable trade-off between segmentation accuracy and computational efficiency while benefiting consistently from the proposed pseudo-label generation strategy. These findings highlight the potential of large-scale foundation models as annotation sources for training compact task-specific segmentation networks in low-label vision domains.

Chinese Translation

无人机（UAV）目标分割仍然面临挑战，原因包括目标尺寸小、外观变化大、背景杂乱以及密集标注数据的稀缺。这些因素阻碍了轻量级分割模型在实际无人机应用中的性能和部署。为了解决这一问题，本文研究了将SAM3（Segment Anything Model 3）作为伪标签生成器用于训练紧凑型分割网络的应用。具体而言，探索了两种监督范式：（i）直接使用未修改的SAM3生成的掩膜进行伪监督，以及（ii）一种改进策略，通过对局部图像块重新应用SAM3来提高掩膜质量。基于这些范式，提出了一种两阶段的SAM3引导伪标签生成框架。在第一阶段，SAM3生成粗略掩膜以进行初步目标定位。随后，将定位区域裁剪为图像块，并再次由SAM3处理以生成具有准确目标边界的精细掩膜，并剔除假阳性。生成的粗略和精细掩膜随后作为伪标签用于优化一个轻量级网络，称为IPS-Seg，该网络由三个组件组成：用于特征提取的IdentityFormer主干、用于多尺度上下文聚合的Atrous Spatial Pyramid Pooling模块，以及用于空间分辨率恢复的基于PixelShuffle的解码器。在多种监督设置下进行的广泛实验表明，所提出框架的有效性。结果显示，IPS-Seg在分割精度和计算效率之间实现了良好的平衡，同时持续受益于所提出的伪标签生成策略。这些发现突显了大规模基础模型作为低标注视觉领域中训练紧凑型任务特定分割网络的注释来源的潜力。

View on arXiv Download PDF AI Translation

cs.CV / 151 / 2607.03760

GeoSAM-Lite: A Lightweight Foundation Model for Onboard Remote Sensing Segmentation

GeoSAM-Lite：一种轻量级的基础模型用于机载遥感分割

Wang, Yongcong, Zhang, Jie, Jiang, Rui, Yang, Xubing, Yun, Ting, Zhang, Li

Abstract

The deployment of large-scale foundation models like Segment Anything Model (SAM) on resource-constrained Earth observation platforms is hindered by prohibitive computational costs and the domain shift between natural and remote sensing imagery. To address these challenges, we propose \textit{Geo}spatial \textit{S}egment \textit{A}nything \textit{M}odel-Lite (GeoSAM-Lite), a lightweight, prompt-free segmentation framework designed for efficient onboard remote sensing segmentation. GeoSAM-Lite incorporates two core innovations: (1) Geospatial-Domain Initialization (Geo-Init), a domain-aware pre-training strategy that distills geospatial priors from a specialized teacher to bridge the domain gap; and (2) Feature Fusion Layers (FFL), which recalibrate spatial features and restore high-frequency boundary cues to overcome the capacity bottlenecks of lightweight backbones. Experiments across representative datasets, with a primary focus on cloud scenarios to evaluate performance under extreme scale variations and complex boundaries, demonstrate that GeoSAM-Lite achieves competitive accuracy while reducing parameters by 92.8\% compared to the heavyweight RSAM-Seg. By establishing a superior Pareto frontier between efficiency and fidelity, GeoSAM-Lite offers a practical solution for real-time segmentation on edge devices.

Chinese Translation

在资源受限的地球观测平台上，大规模基础模型如Segment Anything Model (SAM) 的部署受到高昂计算成本和自然图像与遥感图像之间领域转移的限制。为了解决这些挑战，我们提出了 extit{Geo}空间 extit{S}egment extit{A}nything extit{M}odel-Lite (GeoSAM-Lite)，这是一种轻量级、无提示的分割框架，旨在实现高效的机载遥感分割。GeoSAM-Lite包含两个核心创新：（1）地理空间领域初始化（Geo-Init），一种领域感知的预训练策略，通过从专门的教师模型中提取地理空间先验来弥合领域差距；（2）特征融合层（FFL），用于重新校准空间特征并恢复高频边界线索，以克服轻量级骨干网络的容量瓶颈。在代表性数据集上的实验，主要集中在云场景，以评估在极端规模变化和复杂边界下的性能，结果表明，GeoSAM-Lite在与重型RSAM-Seg相比时，参数减少了92.8\%，同时实现了竞争力的准确性。通过在效率和保真度之间建立优越的帕累托前沿，GeoSAM-Lite为边缘设备上的实时分割提供了实用解决方案。

View on arXiv Download PDF AI Translation

cs.CV / 152 / 2607.03765

Sparse-View Surface Reconstruction using Gaussian Splatting through High-Confidence Depth Propagation with Normal Priors

基于高置信度深度传播与法线先验的高斯点云稀疏视图表面重建

Han, Liang, Wei, Bangcai, Zhou, Junsheng, Liu, Yu-Shen, Han, Zhizhong

Abstract

3D reconstruction from sparse views is a challenging task in 3D computer vision. Recent studies on 3D Gaussian Splatting (3DGS) have achieved remarkable results with sparse views in novel view synthesis, yet reconstructing high-quality geometric surfaces from sparse views remains a challenge, due to the limited geometry clues and the discreteness of Gaussians. In this paper, we propose a novel 3DGS-based method for high-fidelity surface reconstruction from sparse views. Our key insight is to introduce a normal-guided depth propagation approach, which can extend depth information from high-confidence regions to constrain the depth in low-confidence areas. Additionally, we propose an abnormal depth edge-aware regularization to address depth discontinuities caused by the discreteness of Gaussians. Extensive experiments on DTU and Tanks-and-Temples datasets demonstrate that our method outperforms the state-of-the-art methods in sparse view surface reconstruction. Project page: https://hanl2010.github.io/DP-GS.

Chinese Translation

从稀疏视图进行三维重建是三维计算机视觉中的一项挑战性任务。近期关于三维高斯点云（3D Gaussian Splatting, 3DGS）的研究在新视图合成中取得了显著成果，但从稀疏视图重建高质量几何表面仍然面临挑战，这主要是由于几何线索的有限性和高斯分布的离散性。本文提出了一种基于3DGS的新方法，用于从稀疏视图进行高保真表面重建。我们的关键见解是引入一种法线引导的深度传播方法，该方法能够将高置信度区域的深度信息扩展到低置信度区域，以约束其深度。此外，我们提出了一种异常深度边缘感知正则化方法，以解决由高斯离散性引起的深度不连续性。在DTU和Tanks-and-Temples数据集上的大量实验表明，我们的方法在稀疏视图表面重建方面优于当前最先进的方法。项目页面：https://hanl2010.github.io/DP-GS。

View on arXiv Download PDF AI Translation

cs.CV / 153 / 2607.03770

Self-Improving Diffusion Classifiers with Minority Preference Optimization

具有少数偏好优化的自我改进扩散分类器

Kim, Hyunsoo, Wi, Jungmyung, Um, Soobin, Kim, Donghyun, Kim, Suhyun

Abstract

Prior studies have demonstrated that diffusion classifiers achieve robust zero-shot classification performance. However, their effectiveness is strongly tied to the pretraining data distribution: they perform well in majority, high-density regions of the data manifold, but are significantly less accurate in minority, low-density regions. Although prior works on minority sampling have focused on generating more minority-like images, what minority sampling fundamentally enables beyond generation remains underexplored. In this paper, we reveal a direct relationship between minority sampling in generation and the perception capability of diffusion classifiers. Specifically, we show that enhancing minority sampling broadens the coverage of underrepresented regions on the data manifold, thereby improving diffusion-based recognition. To exploit this connection, we propose \textit{Self-Improving Diffusion Classifiers with Minority Preference Optimization} (MiPO), which fine-tunes a pretrained diffusion model using minority preference rewards. Using only arbitrary caption data, MiPO generates candidate samples, rewards those that better cover minority regions, and optimizes the model with LoRA and Group Relative Policy Optimization, without additional image data, external foundation models, or external reward models. This enables stable, prompt-adaptive minority sampling and translates low-density generative coverage into improved zero-shot diffusion classification. To sum up, we show that diffusion classifier perception is biased toward majority regions, demonstrate that this bias can be alleviated through minority preference optimization, and evaluate MiPO on five standard datasets.

Chinese Translation

先前的研究表明，扩散分类器在零样本分类任务中表现出强大的鲁棒性。然而，它们的有效性与预训练数据分布密切相关：在数据流形的多数、高密度区域表现良好，但在少数、低密度区域的准确性显著降低。尽管先前关于少数采样的研究集中于生成更多类似少数类的图像，但少数采样在生成之外的基本作用仍未得到充分探讨。在本文中，我们揭示了生成中的少数采样与扩散分类器感知能力之间的直接关系。具体而言，我们展示了增强少数采样可以扩大数据流形上代表性不足区域的覆盖范围，从而改善基于扩散的识别。为了利用这一联系，我们提出了 extit{具有少数偏好优化的自我改进扩散分类器}（MiPO），该方法通过少数偏好奖励对预训练的扩散模型进行微调。MiPO仅使用任意的文本描述数据生成候选样本，对那些更好覆盖少数区域的样本给予奖励，并通过LoRA和组相对策略优化对模型进行优化，而无需额外的图像数据、外部基础模型或外部奖励模型。这使得稳定、快速适应的少数采样成为可能，并将低密度生成覆盖转化为改进的零样本扩散分类。总之，我们表明扩散分类器的感知偏向于多数区域，证明这种偏见可以通过少数偏好优化得到缓解，并在五个标准数据集上评估了MiPO。

View on arXiv Download PDF AI Translation

cs.CV / 154 / 2607.03771

City-Level 3D Surface Reconstruction with Viewpoint Orientation Partitioning and Scene Completion

基于视点方向划分与场景补全的城市级三维表面重建

Han, Liang, Zhang, Wenyuan, Zhou, Junsheng, Liu, Yu-Shen, Han, Zhizhong

Abstract

Multi-view 3D surface reconstruction is a longstanding challenge in computer vision. Although recent large-scale reconstruction methods based on 3D Gaussian Splatting (3DGS) achieve impressive novel-view synthesis, producing high-quality surfaces over large scenes remains difficult, due to complex geometry, long optimization, and limited memory. In this paper, we propose a novel yet simple partitioning method to efficiently and faithfully reconstruct large-scale scene surfaces. Our key insight lies in a scene partitioning method based on viewpoint orientation. This partitioning approach ensures that views with similar orientations are jointly involved for more accurate depth estimations, leading to precise surface reconstructions and balanced computation on multiple GPUs in parallel. In addition, we propose a strategy to detect and repair missing regions in the initial point cloud caused by sparse viewpoints or insufficient textures, thereby further improving the geometric quality. Extensive experiments on the GauU-Scene, MatrixCity, and UrbanScene3D datasets demonstrate that our method outperforms the state-of-the-art approaches in surface reconstruction for large-scale scenes. Project page: https://hanl2010.github.io/VOP-GS.

Chinese Translation

多视角三维表面重建一直是计算机视觉中的一个长期挑战。尽管基于三维高斯点云（3D Gaussian Splatting, 3DGS）的最新大规模重建方法在新视图合成方面取得了令人印象深刻的成果，但由于复杂的几何形状、长时间的优化过程和有限的内存，产生高质量的大场景表面仍然困难。本文提出了一种新颖且简单的划分方法，以高效且真实地重建大规模场景表面。我们的关键见解在于基于视点方向的场景划分方法。这种划分方法确保具有相似方向的视图共同参与，从而实现更准确的深度估计，进而导致精确的表面重建，并在多个GPU上实现平衡计算。此外，我们提出了一种策略来检测和修复由于视点稀疏或纹理不足而导致的初始点云中的缺失区域，从而进一步提高几何质量。在GauU-Scene、MatrixCity和UrbanScene3D数据集上的大量实验表明，我们的方法在大规模场景的表面重建方面优于现有的最先进方法。项目页面：https://hanl2010.github.io/VOP-GS。

View on arXiv Download PDF AI Translation

cs.CV / 155 / 2607.03784

Rethinking Depth Pruning for Vision Transformers: A Heterogeneity-Aware Perspective

重新思考视觉变换器的深度剪枝：一种考虑异质性的视角

Su, Zhenfeng, Zhao, Kang, Bao, Han, Yuan, Tao, Hu, Zhongzhe, Yu, Xianzhi, Wang, Wenxuan

Abstract

While prior studies have successfully compressed vision Transformers (ViTs) through various pruning techniques, most have concentrated on width pruning to achieve significant reductions in model size. Depth pruning, which removes entire layers from a ViT, is notoriously difficult for accuracy recovery despite its potential to deliver higher speedups, limiting the acceleration achieved by existing joint width-and-depth pruning methods. In this work, we reveal that the failure of existing depth pruning methods lies in their neglect of heterogeneity between different layers, and we introduce HetDPT, a heterogeneity-aware depth pruning method that avoids dimension mismatch. Comprehensive experiments on ImageNet-1K, CIFAR-100, COCO, and ADE20K validate our method: HetDPT achieves a 1.58$\times$ speedup for DeiT-B while maintaining accuracy and a 1.39$\times$ speedup for DeiT-S with nearly no accuracy degradation. Furthermore, when combined with width pruning, HetDPT+ sets a new state-of-the-art record in extreme ViT pruning, enhancing the acceleration ratio from 4.24$\times$ to 5.19$\times$ for the Isomorphic-Pruning-2.6G configuration while maintaining near-lossless accuracy; our code is available at https://github.com/Efficient-AI-for-All/HetDPT.

Chinese Translation

尽管先前的研究通过各种剪枝技术成功压缩了视觉变换器（ViTs），但大多数集中于宽度剪枝以实现模型大小的显著减少。深度剪枝，即从ViT中移除整个层，因其在准确性恢复方面的困难而臭名昭著，尽管它有潜力提供更高的加速，这限制了现有联合宽度和深度剪枝方法所能实现的加速效果。在本研究中，我们揭示了现有深度剪枝方法失败的原因在于它们忽视了不同层之间的异质性，并提出了HetDPT，一种考虑异质性的深度剪枝方法，避免了维度不匹配。对ImageNet-1K、CIFAR-100、COCO和ADE20K的全面实验验证了我们的方法：HetDPT在保持准确性的同时，为DeiT-B实现了1.58$ imes$的加速，而DeiT-S则实现了1.39$ imes$的加速，几乎没有准确性下降。此外，当与宽度剪枝结合时，HetDPT+在极端ViT剪枝中创下了新的最先进记录，将Isomorphic-Pruning-2.6G配置的加速比从4.24$ imes$提高到5.19$ imes$，同时保持近乎无损的准确性；我们的代码可在https://github.com/Efficient-AI-for-All/HetDPT获取。

View on arXiv Download PDF AI Translation

cs.CV / 156 / 2607.03789

G$^2$TAM: Geometry Grounded Track Anything Model

G$^2$TAM：几何基础的任意跟踪模型

Zhu, Chenming, Cao, Peizhou, Lin, Jingli, Hu, Wenbo, Ran, Yunlong, Pang, Jiangmiao, Wang, Tai, Liu, Xihui

Abstract

Human spatial understanding arises from jointly perceiving geometry and semantics, enabling consistent object identification and localization across viewpoints and time. Current video segmentation models depend on explicit object appearance memory banks for instance tracking, yet they remain vulnerable to large viewpoint changes and long-term occlusions. Leveraging the spatial consistency afforded by modern feed-forward 3D reconstruction models, we propose the Geometry Grounded Tracking Anything Model (G$^2$TAM), a unified framework for promptable instance tracking in 3D using only unordered RGB images or videos. G$^2$TAM employs spatially aligned geometric representations as implicit memory, ensuring stable instance identity and localization across frames and views. At its core is a cross-modal spatial encoder that integrates visual and textual prompts into a shared geometric space, enabling end-to-end spatial reconstruction and instance-consistent mask prediction. To support training and evaluation, we construct InsTrack, a large-scale dataset with a dedicated validation split for benchmarking. Extensive experiments show that G$^2$TAM delivers strong cross-view consistency, promptable instance spatial tracking, video object segmentation and spatial reconstruction, establishing a foundation for interactive, geometry-grounded spatial reasoning.

Chinese Translation

人类的空间理解源于几何与语义的共同感知，使得在不同视角和时间下能够一致地识别和定位物体。当前的视频分割模型依赖于显式的物体外观记忆库进行实例跟踪，但在大视角变化和长期遮挡的情况下仍然脆弱。利用现代前馈3D重建模型所提供的空间一致性，我们提出了几何基础的任意跟踪模型（G$^2$TAM），这是一个统一的框架，能够仅使用无序的RGB图像或视频进行3D的可提示实例跟踪。G$^2$TAM采用空间对齐的几何表示作为隐式记忆，确保在不同帧和视角下实例身份和定位的稳定性。其核心是一个跨模态空间编码器，将视觉和文本提示整合到共享的几何空间中，实现端到端的空间重建和实例一致的掩膜预测。为了支持训练和评估，我们构建了InsTrack，这是一个大规模数据集，具有专门的验证集用于基准测试。大量实验表明，G$^2$TAM在跨视角一致性、可提示实例空间跟踪、视频物体分割和空间重建方面表现出色，为交互式的几何基础空间推理奠定了基础。

View on arXiv Download PDF AI Translation

cs.CV / 157 / 2607.03795

InfraNet: Quality-Aware RGB Guidance for Efficient Infrared Object Detection

InfraNet：面向高效红外物体检测的质量感知RGB引导

Feng, Zichao, Zhu, Haodong, Yang, Jingying, Xu, Sheng, Ren, Yangyang, Yang, Yuguang, Liu, Xuhui, Zhang, Juan, Wang, Tian, Yang, Linlin, Zhang, Baochang

Abstract

Robust object detection under adverse visual conditions remains a long-standing challenge for multi-modal perception systems. Existing fusion-based methods typically require both RGB and infrared (IR) inputs, and treat them equally during both training and inference, which compromises their robustness when the RGB modality becomes unreliable or unavailable. In this case, we propose \textbf{InfraNet}, an IR-centric quality-aware framework that regulates RGB guidance during training and supports flexible RGB--IR or IR-only deployment. InfraNet employs an asymmetric architecture where the primary IR pathway extracts multi-scale infrared features for predictions, while the auxiliary RGB pathway provides reliability-controlled supervisory signals. The core of InfraNet is \textbf{QualGate}, a quality-aware fusion module that learns a task-oriented control signal to suppress unreliable RGB guidance and compensate IR features during cross-modal training. Built upon InfraNet, we design two architectural variants: a lightweight IR-only architecture InfraNet-IR and an RGB--IR architecture InfraNet-RGB-IR. Our method is evaluated through extensive experiments on four benchmark datasets (LLVIP, FLIR-Aligned, M$^3$FD, and DroneVehicle), showing strong or competitive accuracy in challenging low-light and adverse weather conditions. Notably, InfraNet maintains high efficiency in IR-only inference, making it both accurate and computationally efficient.

Chinese Translation

在不利视觉条件下进行稳健的物体检测仍然是多模态感知系统面临的长期挑战。现有的基于融合的方法通常需要同时输入RGB和红外（IR）数据，并在训练和推理过程中对它们进行同等处理，这在RGB模态变得不可靠或不可用时会影响其稳健性。在这种情况下，我们提出了 extbf{InfraNet}，一个以IR为中心的质量感知框架，在训练过程中调节RGB引导，并支持灵活的RGB-IR或仅IR的部署。InfraNet采用不对称架构，主要的IR路径提取多尺度红外特征用于预测，而辅助的RGB路径提供可靠性控制的监督信号。InfraNet的核心是 extbf{QualGate}，一个质量感知融合模块，学习任务导向的控制信号，以抑制不可靠的RGB引导并在跨模态训练中补偿IR特征。在InfraNet的基础上，我们设计了两种架构变体：轻量级的仅IR架构InfraNet-IR和RGB-IR架构InfraNet-RGB-IR。我们的方法通过在四个基准数据集（LLVIP、FLIR-Aligned、M$^3$FD和DroneVehicle）上的广泛实验进行评估，显示出在低光和恶劣天气条件下的强大或竞争性准确性。值得注意的是，InfraNet在仅IR推理中保持高效，使其既准确又计算高效。

View on arXiv Download PDF AI Translation

cs.CV / 158 / 2607.03797

Probabilistic Robustness in Medical Image Classification

医学图像分类中的概率鲁棒性

Zhang, Yi, Khastgir, Siddartha, Zhao, Xingyu

Abstract

Deep learning (DL) has shown strong performance in medical image classification, but its trustworthy deployment remains challenging in safety-critical clinical settings, where prediction errors under perturbations may lead to severe consequences. Existing studies mainly focus on adversarial robustness (AR) from a worst-case perspective; however, such settings may be less representative of real medical applications. In this work, we investigate probabilistic robustness (PR) as a more practical measure of model trustworthiness. To this end, we construct a set of natural corruption settings for medical image classification and systematically evaluate commonly used DL models on MedMNIST v2 dataset. Our study provides a statistically grounded perspective on assessing the trustworthiness of DL models, thereby supporting their more trustworthy deployment in medical imaging applications.

Chinese Translation

深度学习（DL）在医学图像分类中表现出强大的性能，但在安全关键的临床环境中，其可信赖的部署仍然面临挑战，因为在扰动下的预测错误可能导致严重后果。现有研究主要从最坏情况的角度关注对抗鲁棒性（AR）；然而，这种设置可能与真实的医学应用不够代表性。在本研究中，我们探讨了概率鲁棒性（PR），作为评估模型可信赖性的更实用的指标。为此，我们为医学图像分类构建了一组自然损坏设置，并在 MedMNIST v2 数据集上系统地评估了常用的深度学习模型。我们的研究提供了一个统计学基础的视角，以评估深度学习模型的可信赖性，从而支持其在医学影像应用中的更可信赖部署。

View on arXiv Download PDF AI Translation

cs.CV / 159 / 2607.03803

CineMobile: On-Device Image-to-Video Diffusion for Cinematic Camera Motion Generation

CineMobile：用于电影摄像机运动生成的设备内图像到视频扩散

Huang, Xuyao, Deng, Zelai, Wang, Xu, Xiao, Xizhong, Deng, Zhijie

Abstract

The growing demand for image-to-video creation on mobile devices has increasingly focused on cinematic motion effects like bullet time, dolly zoom, slow motion, etc. While Diffusion Transformers (DiTs) exhibit strong performance in video generation, their large parameter sizes and multi-step iterative denoising processes lead to substantial computational overhead, making efficient generation on mobile devices challenging. We propose CineMobile to bridge the gap. In particular, CineMobile adopts a three-fold optimization strategy: (1) leveraging a distillation-guided pruning approach to derive a compact yet efficient model that retains the essential video generation capabilities required for cinematic effects; (2) optimizing the compressed model into a 4-step generator via a combination of diffusion distillation and reinforcement learning; (3) employing a hybrid post-training quantization strategy to compress the model footprint to under 1 GB. Experimental results show that compared to the teacher model with the Wan 2.1 architecture, CineMobile achieves a 40x speedup in generation while maintaining comparable visual quality. Specifically, CineMobile generates 49-frame 480p videos with a per-step denoising latency of 0.6s on an NVIDIA H200 GPU and 20s on the MediaTek Dimensity 8400 Ultimate 5G platform, with a peak memory usage of 1.8 GB, demonstrating its practical applicability for mobile-based image-to-video creation.

Chinese Translation

随着对移动设备上图像到视频创作需求的不断增长，越来越多地关注于电影运动效果，如子弹时间、推拉镜头、慢动作等。尽管扩散变换器（Diffusion Transformers, DiTs）在视频生成中表现出强大的性能，但其庞大的参数规模和多步骤迭代去噪过程导致了显著的计算开销，使得在移动设备上高效生成变得具有挑战性。我们提出了CineMobile来填补这一空白。具体而言，CineMobile采用了三重优化策略：（1）利用蒸馏引导的剪枝方法，推导出一个紧凑但高效的模型，保留了生成电影效果所需的基本视频生成能力；（2）通过扩散蒸馏和强化学习的结合，将压缩模型优化为一个4步生成器；（3）采用混合后训练量化策略，将模型占用空间压缩到1 GB以下。实验结果表明，与采用Wan 2.1架构的教师模型相比，CineMobile在生成速度上实现了40倍的提升，同时保持了可比的视觉质量。具体而言，CineMobile在NVIDIA H200 GPU上生成49帧480p视频的每步去噪延迟为0.6秒，而在MediaTek Dimensity 8400 Ultimate 5G平台上为20秒，峰值内存使用量为1.8 GB，展示了其在移动图像到视频创作中的实际应用潜力。

View on arXiv Download PDF AI Translation

cs.CV / 160 / 2607.03810

TestMate: Test-Time Domain Adaptation Aided by Lightweight Vision Foundation Model

TestMate：一种轻量级视觉基础模型辅助的测试时域适应

Fotiou, Dimitrios, Mygdalis, Vasileios, Pitas, Ioannis

Abstract

Test-Time Domain Adaptation (TTDA) aims to adapt Deep Neural Networks to distribution shifts using only streaming, unlabeled test data in real time. Current methods for semantic segmentation tasks suffer from critical limitations. Entropy minimization techniques require costly backpropagation, risking catastrophic forgetting and producing noisy segmentation boundaries. Memory-bank methods, while backpropagation-free, exhibit slow adaptation, requiring numerous samples to converge and struggle to handle continuous domain shifts. We introduce TestMate, a novel, real-time, and backpropagation-free TTDA framework that overcomes these issues. TestMate leverages generalization capability of a lightweight Visual Foundation Model to guide the adaptation. We use a zero-shot instance segmentation YOLOv8-seg based model to generate unlabeled mask proposals for objects and their parts at multiple scales in real time. These proposals are fused with the primary model via a heuristic, size-ordered competitive scheme, where small, high-confidence regions dominate and refine predictions in surrounding larger, less certain areas. This paremeter-free mechanism enables immediate adaptation from the first frame, inherently avoids catastrophic forgetting and effectively preserves fine object details and boundaries, even for small objects. TestMate can be used as a standalone, efficient refinement module or seamlessly integrated into existing TTDA methods to significantly boost their performance. We demonstrate state-of-the-art results across two benchmark datasets, proving TestMate's effectiveness in three distinct adaptation tasks: TTDA, Source-Free Domain Adaptation (SFDA), and online-TTDA. Code is available.

Chinese Translation

测试时域适应（TTDA）旨在利用实时流式的无标签测试数据，使深度神经网络适应分布变化。目前用于语义分割任务的方法存在严重的局限性。熵最小化技术需要昂贵的反向传播，可能导致灾难性遗忘，并产生噪声分割边界。尽管记忆库方法不依赖反向传播，但适应速度较慢，需要大量样本才能收敛，并且在处理连续域变化时表现不佳。我们提出了TestMate，一个新颖的实时且无反向传播的TTDA框架，克服了这些问题。TestMate利用轻量级视觉基础模型的泛化能力来指导适应。我们使用基于零样本实例分割的YOLOv8-seg模型实时生成对象及其部分的无标签掩码提议，覆盖多个尺度。这些提议通过启发式的、按大小排序的竞争机制与主模型融合，其中小而高置信度的区域主导并细化周围较大且不确定区域的预测。该无参数机制使得从第一帧开始即可实现即时适应，内在地避免了灾难性遗忘，并有效保留了细小物体的细节和边界。TestMate可以作为独立的高效细化模块使用，或无缝集成到现有的TTDA方法中，以显著提升其性能。我们在两个基准数据集上展示了最先进的结果，证明了TestMate在三种不同适应任务中的有效性：TTDA、无源域适应（SFDA）和在线TTDA。代码可用。

View on arXiv Download PDF AI Translation

cs.CV / 161 / 2607.03817

Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detection

全球逻辑与局部搜索：可验证工业异常检测的双流多模态上下文学习

Deng, Runzhi, Hu, Yundi, Zhong, Yiming, Wang, Zhao, Liu, Xixi, Wang, Hongsong, Shan, Caifeng, Zhao, Fang

Abstract

Large Multimodal Models (LMMs) show strong few-shot generalization, but industrial anomaly detection remains difficult because defects are small, input resolution is limited, and textual standards are not always grounded in visual evidence. Recent optimization-based methods improve alignment through fine-tuning, but they often require many defective samples, which are unavailable in early deployment. We present Global Logic and Local Search (GLLS), a training-free framework for reference-guided multimodal in-context verification. GLLS uses a Part-Aware Visual-Logical Atlas to organize normal references and structured specifications in the inference context. It combines a Global & Logic Stream, where SAM 3 extracts partially checkable visual facts, with a Fine-Grained & Actions Stream, where MCTS selects local evidence crops under a fixed budget. Experiments on MMAD-QA and additional anomaly detection datasets show consistent gains over matched and general-purpose baselines, while keeping the final diagnostic decision traceable to explicit visual evidence throughout the inspection trace.

Chinese Translation

大型多模态模型（LMMs）展现出强大的少量样本泛化能力，但工业异常检测仍然困难，因为缺陷较小、输入分辨率有限，并且文本标准并不总是与视觉证据相结合。近期基于优化的方法通过微调提高了对齐度，但通常需要大量缺陷样本，而这些样本在早期部署中并不可用。我们提出了全球逻辑与局部搜索（GLLS），这是一种无训练的框架，用于参考引导的多模态上下文验证。GLLS使用一个部分感知视觉逻辑图谱来组织正常参考和结构化规范，以便在推理上下文中使用。它结合了一个全球与逻辑流，其中SAM 3提取部分可检查的视觉事实，以及一个细粒度与动作流，其中MCTS在固定预算下选择局部证据裁剪。在MMAD-QA和其他异常检测数据集上的实验表明，相较于匹配和通用基线，GLLS在性能上始终保持一致的提升，同时确保最终的诊断决策可以追溯到整个检查过程中的明确视觉证据。

View on arXiv Download PDF AI Translation

cs.CV / 162 / 2607.03818

TRISTAR: Triple-Signal Stair Recognition and Vision-Only Indoor Navigation for Search-and-Rescue Micro-UAVs

TRISTAR：用于搜索与救援微型无人机的三信号楼梯识别与仅基于视觉的室内导航

Gîngu, Octavian, Spînu, Stelian

Abstract

Indoor search-and-rescue (SAR) operations often require rapid situational awareness where GNSS signals are unavailable and human access is difficult or hazardous. While most autonomous aerial systems rely on LiDAR, stereo vision, or specialized depth cameras, such solutions increase both hardware complexity and deployment costs. This paper presents a complete autonomous indoor navigation framework for low-cost unmanned aerial vehicles based exclusively on monocular vision. Implemented on a DJI Tello platform, the system combines monocular depth estimation using Depth Anything V2 with classical computer vision and lightweight deep learning models for scene understanding, victim detection, and hazard recognition. The framework consists of two independent behaviors: (i) corridor exploration with automatic door detection, room entry, OCR-based room identification, and victim inspection; and (ii) autonomous stair ascent based on TRISTAR (TRI-Signal STair Ascent Recognition), a novel triple-sensor fusion method that integrates structural cues (Sobel filtering), texture analysis (multi-scale Gabor filtering), and geometric depth from monocular depth estimation. Evaluation used real indoor flights in a university building. Depth calibration reduced relative depth error from 27.4% to below 10%, while the door detection algorithm reached a precision of 0.93 and an F1-score of 0.91. A dedicated ablation study shows that multi-sensor fusion significantly improves stair-recognition robustness compared to individual sensing modalities, and a failure-case analysis delineates the limits of monocular perception under challenging lighting and reflective surfaces. The results demonstrate that reliable indoor exploration and stair traversal are achievable on resource-constrained platforms without specialized ranging hardware, a practical, cost-effective solution for rapid SAR deployment.

Chinese Translation

室内搜索与救援（SAR）操作通常需要快速的情境感知，而GNSS信号在此时不可用且人类进入困难或危险。虽然大多数自主航空系统依赖于激光雷达、立体视觉或专用深度相机，但这些解决方案增加了硬件复杂性和部署成本。本文提出了一种完全基于单目视觉的低成本无人机自主室内导航框架。该系统在DJI Tello平台上实现，结合了使用Depth Anything V2进行的单目深度估计、经典计算机视觉和轻量级深度学习模型，用于场景理解、受害者检测和危险识别。该框架由两个独立的行为组成：（i）走廊探索，包括自动门检测、房间进入、基于OCR的房间识别和受害者检查；（ii）基于TRISTAR（TRI-Signal STair Ascent Recognition）的自主楼梯上升，这是一种新颖的三传感器融合方法，集成了结构线索（Sobel滤波）、纹理分析（多尺度Gabor滤波）和来自单目深度估计的几何深度。评估使用了在大学建筑内的真实室内飞行。深度校准将相对深度误差从27.4%降低到10%以下，而门检测算法的精度达到0.93，F1-score为0.91。一项专门的消融研究表明，与单一传感模式相比，多传感器融合显著提高了楼梯识别的鲁棒性，失败案例分析阐明了在具有挑战性的光照和反射表面下单目感知的局限性。结果表明，在资源受限的平台上，无需专用测距硬件即可实现可靠的室内探索和楼梯通行，这为快速SAR部署提供了一种实用且具有成本效益的解决方案。

View on arXiv Download PDF AI Translation

cs.CV / 163 / 2607.03822

FDR-Occ: Factorized Dense Routing for Full-Spectrum 3D Occupancy Prediction

FDR-Occ：全光谱3D占用预测的因子化稠密路由

Chen, Dubing, Zheng, Huan, Yan, Tianyi, Zhou, Yucheng, Tao, Runzhou, Qiu, Zhongying, Yang, Jianfei, Shen, Jianbing

Abstract

Vision-based 3D occupancy prediction fundamentally relies on the 2D-to-3D view transformation. Current paradigms predominantly utilize explicit physical projection, which artificially restricts the routing matrix to strict, sparse camera rays. While computationally efficient, this imposes a severe Locality Bottleneck, preventing the network from constructing holistic contextual understanding and degrading sharply when camera extrinsics are unreliable or absent. To break this bottleneck, we abstract view transformation as unconstrained bipartite routing and propose Factorized Dense Routing (FDR). By approximating dense 2D-to-3D mixing through hierarchical tensor contractions, FDR guarantees a fully-global receptive field with tractable, sub-quadratic complexity. Crucially, the mandatory spatial contraction in dense routing exposes a fundamental Resolution-Context Trade-off. To address this, we introduce a Resolution-Context Decoupled Architecture. We factorize the 3D space into a global macroscopic topological anchor (via FDR) and precise local geometric planes (via explicit projection). This decoupling enables global semantic inference and exact surface localization to complement each other without mutual compromise. Extensive experiments demonstrate that our framework achieves state-of-the-art performance on the Occ3D-nuScenes and Occ3D-Waymo benchmarks. More notably, in an uncalibrated setting where physical extrinsics are withheld, our global routing internalizes the implicit multi-camera rig topology and exhibits substantially stronger structural robustness than physical-projection baselines under the same protocol.

Chinese Translation

基于视觉的3D占用预测基本依赖于2D到3D的视图转换。目前的范式主要利用显式物理投影，这人为地将路由矩阵限制在严格的稀疏相机光线中。尽管计算效率高，但这造成了严重的局部瓶颈，阻碍了网络构建整体上下文理解，并在相机外部参数不可靠或缺失时急剧下降。为了打破这一瓶颈，我们将视图转换抽象为无约束的二分路由，并提出了因子化稠密路由（Factorized Dense Routing, FDR）。通过层次张量收缩近似稠密的2D到3D混合，FDR保证了一个完全全局的感受野，且具有可处理的次二次复杂度。关键是，稠密路由中强制的空间收缩暴露了一个基本的分辨率-上下文权衡。为了解决这一问题，我们引入了分辨率-上下文解耦架构。我们将3D空间因子化为一个全球宏观拓扑锚点（通过FDR）和精确的局部几何平面（通过显式投影）。这种解耦使得全球语义推理和精确表面定位能够相辅相成，而不相互妥协。大量实验表明，我们的框架在Occ3D-nuScenes和Occ3D-Waymo基准测试中达到了最先进的性能。更值得注意的是，在一个未校准的环境中，物理外部参数被隐去，我们的全局路由内化了隐式多相机设备拓扑，并在相同协议下表现出显著更强的结构鲁棒性，优于物理投影基线。

View on arXiv Download PDF AI Translation

cs.CV / 164 / 2607.03825

Q-TriM: Question-Guided Tri-Modal Attention for Audio-Visual Question Answering

Q-TriM：基于问题引导的三模态注意力用于音视频问答

Kim, SungHun, Baek, SeungJun

Abstract

Audio-Visual Question Answering (AVQA) extends classical VQA by requiring joint reasoning over video and synchronized audio. However, many AVQA systems rely on deeply stacked layers of self- and cross attention across text, video, and audio. Such sequential stacking may incur loss of information such as subtle inter-modal cues over the layers, causing errors to accumulate across sequential attention layers during the fusion. We introduce Q-TriM which performs multi-modal fusion in a shallow and parallel manner instead of a deep and sequential manner. For Q-TriM, we propose a novel framework for attention operation incorporating video and audio conditioned on text. As a result, we obtain not only standard cross attention outputs but also Tri-Modal Attention representations in which Query, Key, and Value come from distinct modalities. These attention representations are combined in parallel at a single stage, thus avoiding the multi-modal fusion with deep stacks in order to mitigate error accumulation and depth-induced issues. Q-TriM achieves state-of-the-art performance on three AVQA benchmarks, including substantial gains on MUSIC-AVQA-R, which demonstrates its robustness and out-of-distribution generalization. Code is available at https://github.com/Sunghun95/Q-TriM

Chinese Translation

音视频问答（AVQA）扩展了经典的视觉问答（VQA），要求对视频和同步音频进行联合推理。然而，许多AVQA系统依赖于在文本、视频和音频之间深度堆叠的自注意力和交叉注意力层。这种顺序堆叠可能导致信息的丢失，例如在层之间的微妙跨模态线索，从而在融合过程中导致错误在顺序注意力层中累积。我们提出了Q-TriM，它以浅层和并行的方式进行多模态融合，而不是深层和顺序的方式。对于Q-TriM，我们提出了一种新颖的注意力操作框架，结合了以文本为条件的视频和音频。因此，我们不仅获得了标准的交叉注意力输出，还获得了三模态注意力表示，其中查询（Query）、键（Key）和值（Value）来自不同的模态。这些注意力表示在单个阶段并行组合，从而避免了深层堆叠的多模态融合，以减轻错误累积和深度引发的问题。Q-TriM在三个AVQA基准上实现了最先进的性能，包括在MUSIC-AVQA-R上显著提升，展示了其鲁棒性和超出分布的泛化能力。代码可在 https://github.com/Sunghun95/Q-TriM 获取。

View on arXiv Download PDF AI Translation

cs.CV / 165 / 2607.03831

How Do Diffusion Classifiers Decide? A Bias-Centric Evaluation

扩散分类器如何做出决策？一种以偏差为中心的评估

Fathi, Saba, Ayar, Fardin, Abdolali, Maryam, Javanmardi, Ehsan, Tsukada, Manabu, Javanmardi, Mahdi

Abstract

Diffusion models have recently been repurposed for zero-shot classification, giving rise to diffusion classifiers that identify the best-matching text prompt by minimizing the noise-prediction error. Despite their growing adoption, how these models make classification decisions remains poorly understood. We introduce ASOB-Bench, a bias evaluation for diffusion classifiers along three dimensions: Attribute binding, Size-Order bias, and Background dependency. These dimensions serve not as an exhaustive taxonomy but as targeted probes of how the text-conditioned reconstruction-error score reaches a decision. Such a perspective is well studied for discriminative vision-language models, yet remains overlooked for diffusion classifiers. Extending an existing framework with five new attribute categories on newly constructed datasets, we find diffusion classifiers are less prone to attribute misbinding than an OpenCLIP baseline; on the established ComCo benchmark they are substantially more susceptible to size-order shortcuts; and on ImageNet-B they suffer far larger accuracy drops, revealing heavy reliance on background over foreground cues. Reconstruction-error heatmaps and U-Net cross-attention visualizations expose the mechanism behind each bias. Because diffusion classifiers share the same denoiser as text-to-image models, these single-pass diagnostics also point toward analogous failure modes in generation. Overall, diffusion classifiers exhibit a distinct bias profile from vision-language models, offering guidance for building more robust diffusion-based models.

Chinese Translation

扩散模型最近被重新用于零样本分类，产生了通过最小化噪声预测误差来识别最佳匹配文本提示的扩散分类器。尽管它们的应用日益增加，但这些模型如何做出分类决策仍然不甚明了。我们引入了ASOB-Bench，这是一个针对扩散分类器的偏差评估，涵盖三个维度：属性绑定、大小顺序偏差和背景依赖性。这些维度并不是一个详尽的分类法，而是针对文本条件重建误差评分如何达到决策的特定探针。这种视角在判别性视觉-语言模型中得到了充分研究，但在扩散分类器中仍然被忽视。通过在新构建的数据集上扩展现有框架并增加五个新的属性类别，我们发现扩散分类器在属性误绑定方面比OpenCLIP基线更不易受影响；在已建立的ComCo基准上，它们对大小顺序捷径的敏感性显著更高；而在ImageNet-B上，它们的准确率下降幅度更大，揭示了对背景而非前景线索的严重依赖。重建误差热图和U-Net交叉注意力可视化揭示了每种偏差背后的机制。由于扩散分类器与文本到图像模型共享相同的去噪器，这些单次诊断也指向生成中的类似失败模式。总体而言，扩散分类器展现出与视觉-语言模型截然不同的偏差特征，为构建更稳健的基于扩散的模型提供了指导。

View on arXiv Download PDF AI Translation

cs.CV / 166 / 2607.03836

When Simpler Is Better: Evaluating Translation Pipelines for Medieval Latin Manuscripts

简单更好：评估中世纪拉丁文手稿的翻译流程

Bui, Nguyen Kim Hai, Arafat, Md. Easin, Orosz, Tamás Gábor, Mahmud, Mufti

Abstract

Despite remarkable progress in machine translation, Vision Language Models (VLMs) struggle on historical manuscripts, a domain that stresses core Natural Language Processing (NLP) capabilities: low-resource transliteration, archaic vocabulary, and noisy input signals. We present a systematic framework for evaluating the full image-to-translation pipeline on medieval Latin manuscripts, a setting in which scribal shorthand, ligatures, and parchment degradation expose failure modes that are invisible in clean-text benchmarks. Benchmarking on the CATMuS Latin dataset reveals a specialization gap: domain-specific Optical Character Recognition (OCR) models reduce character error rate by up to 4.3$\times$ compared to general-purpose VLMs, despite operating at orders of magnitude fewer parameters. We introduce the Interpres-Parallel-Corpus (IPC), a novel dataset comprising 1,383 aligned manuscript image lines, transcriptions, and expert translations, the first of its kind for medieval Latin. Our experiments uncover a complexity paradox: the simplest pipeline, a specialized OCR model feeding directly into a VLM, outperforms all multi-component variants. Adding retrieval-augmented generation (RAG) or post-OCR correction introduces prompt saturation and error propagation that degrade aggregate translation quality. These findings offer both a new benchmark and practical guidance for deploying translation systems in low-resource historical settings.

Chinese Translation

尽管机器翻译取得了显著进展，视觉语言模型（VLMs）在历史手稿上仍然面临挑战，这是一个强调自然语言处理（NLP）核心能力的领域：低资源音译、古老词汇和噪声输入信号。我们提出了一个系统框架，用于评估中世纪拉丁文手稿的完整图像到翻译流程，在这种环境中，抄写员的速记、连字和羊皮纸的退化暴露了在干净文本基准中不可见的失败模式。在CATMuS拉丁文数据集上的基准测试揭示了一个专业化差距：领域特定的光学字符识别（OCR）模型相比于通用VLMs将字符错误率降低了多达4.3倍，尽管其参数量级少得多。我们介绍了Interpres-Parallel-Corpus（IPC），这是一个新颖的数据集，包含1,383条对齐的手稿图像行、转录和专家翻译，是中世纪拉丁文领域首个此类数据集。我们的实验揭示了一个复杂性悖论：最简单的流程，即一个专门的OCR模型直接输入到VLM中，优于所有多组件变体。添加检索增强生成（RAG）或后OCR校正会引入提示饱和和错误传播，降低整体翻译质量。这些发现为在低资源历史环境中部署翻译系统提供了新的基准和实用指导。

View on arXiv Download PDF AI Translation

cs.CV / 167 / 2607.03851

ContiStain: Cross-Domain Relation-Preserving Distillation for Continual Multi-Domain Virtual IHC Staining

ContiStain：跨域关系保持蒸馏用于持续多域虚拟IHC染色

Chen, Fuqiang, Wang, Yifeng, Wang, Hongpeng, Zhang, Yongbing

Abstract

A unified multiplex virtual staining model enables scalable and non-destructive multiplex analysis from H&E slides while promoting parameter efficiency, shared pathological knowledge, and consistent cross-biomarker representations. However, in clinical practice, data for new biomarkers are typically acquired sequentially over time. Fine-tuning on such temporally arriving data leads to severe performance degradation on previously learned biomarkers, as sequential optimization disrupts the structured relationships among biomarker representations in the latent space. To address this issue, we propose ContiStain, an IHC multi-domain relational distillation framework for continual virtual staining. We first (i) construct a domain-aware structured feature space using a mixture-of-experts (MoE) feature extractor to reduce representation interference across biomarker domains. Based on this stabilized feature space, we then (ii) propose a relation-preserving distillation strategy that explicitly enforces the consistency of cross-domain token-level cosine similarity matrices between learned biomarker domains during continual adaptation. By maintaining cross-domain structural coherence, ContiStain mitigates forgetting while retaining adaptability to new domains. Experiments on the MIST dataset under a four-domain sequential virtual IHC staining setting show improved stability, reducing FID and ConchFID by 11.1 and 60.9 compared to sequential fine-tuning, enabling scalable and robust multi-domain virtual staining. Code is released at https://github.com/ccitachi/ContiStain.

Chinese Translation

统一的多重虚拟染色模型能够从H&E切片中实现可扩展和非破坏性的多重分析，同时促进参数效率、共享病理知识和一致的跨生物标志物表示。然而，在临床实践中，新生物标志物的数据通常是随时间顺序获取的。在这种时间到达的数据上进行微调会导致之前学习的生物标志物性能严重下降，因为顺序优化会破坏潜在空间中生物标志物表示之间的结构关系。为了解决这个问题，我们提出了ContiStain，一个用于持续虚拟染色的IHC多域关系蒸馏框架。我们首先(i) 使用专家混合（Mixture-of-Experts, MoE）特征提取器构建一个域感知的结构特征空间，以减少跨生物标志物域的表示干扰。基于这个稳定的特征空间，我们接着(ii) 提出了一种关系保持蒸馏策略，该策略在持续适应过程中明确强制学习的生物标志物域之间的跨域标记级余弦相似度矩阵的一致性。通过保持跨域结构一致性，ContiStain减轻了遗忘，同时保持对新域的适应能力。在四域顺序虚拟IHC染色设置下对MIST数据集的实验表明，稳定性得到改善，与顺序微调相比，FID和ConchFID分别降低了11.1和60.9，从而实现可扩展和稳健的多域虚拟染色。代码已发布在 https://github.com/ccitachi/ContiStain。

View on arXiv Download PDF AI Translation

cs.CV / 168 / 2607.03853

CogRad: A Cognitively-Inspired Multi-Agent Framework for Radiology Report Generation

CogRad：一种受认知启发的多智能体放射学报告生成框架

Khan, Saif Ur Rehman, Maqsood, Hasaan, Vollmer, Sebastian, Dengel, Andreas, Asim, Muhammad Nabeel

Abstract

Automated radiology report generation (RRG) can ease radiologist workload, yet most existing systems produce a report in a single forward pass, with no mechanism to check a claim against the image or revisit a finding once stated. We present CogRad, a cognitively inspired multi-agent framework that structures generation around four stages of a radiologist's reading process. A Scout agent discovers anatomical regions directly from image patches via slot attention and assigns region and disease-level triage scores; an Investigator agent concentrates representational capacity on the regions Scout flags as suspicious; a Writer agent compiles these signals into a disease gated visual prefix for a large language model; and a Verifier agent supervises training with a visual entailment loss and, at inference, re-examines its own draft sentence by sentence, regenerating any report it judges insufficiently grounded. On CheXpert Plus, CogRad attains a BLEU-4 of 0.316 and a CIDEr of 0.322, the best scores among the methods we compare against. On IU X-Ray, it attains a BLEU-4 of 0.201 and a CIDEr of 0.724, leading every baseline on every standard NLG metric. We further evaluate CogRad with RadGraph F1, CheXbert F1, and a hallucination analysis to assess clinical accuracy beyond standard text-overlap metrics, complemented by ablation studies and Grad-CAM-based visualizations that characterize each agent's contribution and the model's visual grounding.

Chinese Translation

自动化放射学报告生成（RRG）可以减轻放射科医生的工作负担，但大多数现有系统仅通过单次前向传递生成报告，缺乏对图像进行声明核查或在声明后重新审视发现的机制。我们提出了CogRad，这是一种受认知启发的多智能体框架，其生成过程围绕放射科医生的阅读过程的四个阶段进行结构化。Scout智能体通过插槽注意力直接从图像补丁中发现解剖区域，并为区域和疾病级别分配分流评分；Investigator智能体将表示能力集中在Scout标记为可疑的区域；Writer智能体将这些信号编译成一个针对大型语言模型的疾病门控视觉前缀；Verifier智能体通过视觉蕴涵损失监督训练，并在推理时逐句重新审视自己的草稿，重新生成任何其认为基础不足的报告。在CheXpert Plus数据集上，CogRad达到了0.316的BLEU-4和0.322的CIDEr，是我们比较的所有方法中最佳的得分。在IU X-Ray数据集上，CogRad达到了0.201的BLEU-4和0.724的CIDEr，在每个标准NLG指标上领先于所有基线。我们还通过RadGraph F1、CheXbert F1和幻觉分析进一步评估CogRad，以评估临床准确性，超越标准文本重叠指标，并辅以消融研究和基于Grad-CAM的可视化，表征每个智能体的贡献及模型的视觉基础。

View on arXiv Download PDF AI Translation

cs.CV / 169 / 2607.03855

PRISM3D: Probabilistic Refinement and Robust Initialization for Physically Consistent Scene Modeling under Extreme Motion Blur

PRISM3D：在极端运动模糊下进行物理一致场景建模的概率精细化与稳健初始化

Matta, Gopi Raju, Trisha, Reddypalli, Madhuri, Vemunuri Divya, Mitra, Kaushik

Abstract

We address the inverse problem of blind 3D scene reconstruction from extremely motion-blurred images, a scenario where traditional Structure-from-Motion (SfM) pipelines fail. Existing approaches typically circumvent this bottleneck by relying on impractical sharp-image supervision. In this work, we introduce PRISM3D, a unified framework enabling robust reconstruction directly from severely degraded inputs. To overcome the lack of a reliable starting point, we propose a Robust Initialization strategy utilizing deep dense tracking method (VGGSfM) to recover global topology where feature matching fails. To the best of our knowledge, we are the first to effectively leverage this paradigm to bootstrap 3D Gaussian Splatting from extreme motion blur. However, while robust, this initialization yields sparse and noisy geometry that causes deterministic optimization to diverge. To resolve this, we propose a coupled solution driven by probability and physics: we adopt a probabilistic formulation for geometric densification via Markov Chain Monte Carlo (MCMC) to robustly populate the sparse priors, while simultaneously modeling physical image formation via continuous Bezier Trajectories. Furthermore, while PRISM3D establishes a highly robust standalone pipeline, the availability of complementary event streams offers an opportunity to push the reconstruction fidelity further. To exploit this, we introduce PRISM3D-E, a multi-modal (RGB + Events) extension that seamlessly integrates high-temporal-resolution events as structural priors to maximize geometric recovery. Because existing datasets lack paired event streams under such severe degradation, we concurrently contribute the PRISM3D-E Benchmark to facilitate rigorous evaluation. Extensive experiments demonstrate that both our standalone RGB framework and its multi-modal extension establish new state-of-the-art performance.

Chinese Translation

我们解决了从极度运动模糊图像中进行盲3D场景重建的逆问题，这是一种传统的运动结构（Structure-from-Motion, SfM）管道无法应对的场景。现有方法通常通过依赖不切实际的清晰图像监督来规避这一瓶颈。在本研究中，我们提出了PRISM3D，一个统一框架，能够直接从严重退化的输入中进行稳健重建。为了克服缺乏可靠起始点的问题，我们提出了一种稳健初始化策略，利用深度密集跟踪方法（VGGSfM）来恢复特征匹配失败时的全局拓扑。根据我们所知，我们是首个有效利用这一范式从极端运动模糊中引导3D高斯溅射（Gaussian Splatting）的研究。然而，尽管初始化过程稳健，但其产生的稀疏且噪声较多的几何形状导致确定性优化发散。为了解决这一问题，我们提出了一种由概率和物理驱动的耦合解决方案：我们采用基于马尔可夫链蒙特卡洛（Markov Chain Monte Carlo, MCMC）的概率公式进行几何密集化，以稳健地填充稀疏先验，同时通过连续贝塞尔轨迹（Bezier Trajectories）建模物理图像形成。此外，尽管PRISM3D建立了一个高度稳健的独立管道，但补充事件流的可用性提供了进一步提升重建保真度的机会。为此，我们引入了PRISM3D-E，一个多模态（RGB + 事件）扩展，能够无缝整合高时间分辨率事件作为结构先验，以最大化几何恢复。由于现有数据集在如此严重退化下缺乏配对的事件流，我们同时贡献了PRISM3D-E基准，以促进严格评估。大量实验表明，我们的独立RGB框架及其多模态扩展均建立了新的最先进性能。

View on arXiv Download PDF AI Translation

cs.CV / 170 / 2607.03862

Ghosts Beneath Textures: Texture-Relation Cues for Cross-Paradigm AI-Generated Image Detection

纹理下的幽灵：跨范式AI生成图像检测的纹理关系线索

Wang, Haoyu, Qin, Yiming, Ba, Zhongjie, Dong, Ziping, Zeng, Jishen, Cheng, Peng, Ren, Kui

Abstract

AI-generated images have proliferated rapidly, motivating extensive research. Most existing AI-generated image detectors are developed and evaluated under image-free generation paradigms, such as noise-based or text-guided generation. However, image-conditioned generation has become increasingly important in practical applications, as it enables more fine-grained control over generated content. Detecting AI-generated images across these two paradigms creates a critical cross-paradigm detection problem that has long been overlooked. To study this problem, we construct ConImageGen, a benchmark for cross-paradigm AI-generated image detection. Evaluations on ConImageGen show that existing detectors fail to generalize reliably across image-free and image-conditioned generation. To address this failure, this paper identifies a cross-paradigm forensic cue and provides a new perspective for generalized AI-generated image detection. Specifically, by suppressing semantic interference, we visualize, for the first time, semantics-irrelevant texture patterns across generation paradigms. These patterns exhibit structured local-global texture relations, indicating a generalizable form of forensic evidence. Motivated by this finding, we shift the focus from directly exploiting explicit artifacts to modeling texture relations and propose DTS-Det, a detection framework that captures and leverages such relations for generalized AI-generated image detection. Extensive experiments validate the effectiveness of our method. DTS-Det achieves state-of-the-art performance across diverse evaluation settings, reaching 99.6% ACC on ConImageGen with a 10.5% gain over the best baseline. It also achieves 93.2%/94.1% ACC in cross-dataset evaluation on PicoBanana/RAID and maintains detection rates of 95.2%/88.1% under reconstruction attacks and black-box adversarial attacks, respectively.

Chinese Translation

AI生成图像迅速普及，推动了广泛的研究。现有的大多数AI生成图像检测器是在无图像生成范式下开发和评估的，例如基于噪声或文本引导的生成。然而，图像条件生成在实际应用中变得越来越重要，因为它能够对生成内容进行更细致的控制。在这两种范式之间检测AI生成图像构成了一个长期被忽视的关键跨范式检测问题。为研究这一问题，我们构建了ConImageGen，一个用于跨范式AI生成图像检测的基准。对ConImageGen的评估表明，现有检测器在无图像和有图像条件生成之间无法可靠地泛化。为了解决这一问题，本文识别出一种跨范式取证线索，并为广义AI生成图像检测提供了新的视角。具体而言，通过抑制语义干扰，我们首次可视化了跨生成范式的与语义无关的纹理模式。这些模式表现出结构化的局部-全局纹理关系，指示出一种可泛化的取证证据。基于这一发现，我们将重点从直接利用显性伪影转向建模纹理关系，并提出了DTS-Det，一个捕捉并利用这些关系以实现广义AI生成图像检测的检测框架。大量实验验证了我们方法的有效性。DTS-Det在多种评估设置中达到了最先进的性能，在ConImageGen上达到了99.6%的准确率，比最佳基线提高了10.5%。在PicoBanana/RAID的跨数据集评估中，它分别达到了93.2%/94.1%的准确率，并在重建攻击和黑箱对抗攻击下保持了95.2%/88.1%的检测率。

View on arXiv Download PDF AI Translation

cs.CV / 171 / 2607.03869

GeoSelect: Spatial-Program Execution for Training-Free Referring Remote Sensing Image Segmentation

GeoSelect：无训练的空间程序执行用于遥感图像分割

Jiang, Yuhang, Deng, Guohui, Xu, Miaozhong, Ruan, Chao, Zhao, Jinling, Huang, Linsheng

Abstract

Referring remote sensing image segmentation isolates the object named by a natural-language expression in an aerial image. Existing training-free methods resolve the expression through implicit vision-language activations or region-text similarity, which gives weak control over the spatial, comparative, and ordinal relations that dominate aerial referring: they cannot represent constructions such as the largest ship or the second court from the left. We propose GeoSelect, a training-free pipeline that reframes referring as the execution of a typed spatial program. A frozen, text-only language model synthesises the expression into a small domain-specific language, a well-formedness checker accepts the program, and a deterministic executor runs it. The central abstraction is a single scored candidate set type under which every operator composes: continuous geometric fields realise position and proximity as dense pixel-level maps, while discrete set and order operators add the extremum, ordinal, counted-union, and relational constructions that fields alone cannot express. Because execution is explicit, every intermediate program, field, and ranking is inspectable, and a reliability ladder degrades any failing program to a field-only special case, so every expression returns an answer. GeoSelect attains 58.86 mIoU on RRSIS-D test and 55.27 mIoU on RISBench test, more than twice the best prior training-free method on RRSIS-D, with no referring supervision and on a single GPU. A controlled comparison with candidates and segmenter fixed attributes the gain to explicit execution, not the backbone; an oracle decomposition localises the residual gap to detection recall on RRSIS-D and selection on RISBench, and an exposure audit confirms robustness to pretraining leakage. Code will be released upon acceptance at the project page https://avalon-s.github.io/GeoSelect/.

Chinese Translation

指称遥感图像分割通过自然语言表达在航空图像中隔离被命名的对象。现有的无训练方法通过隐式的视觉-语言激活或区域-文本相似性来解析表达，这对主导航空指称的空间、比较和序数关系的控制较弱：它们无法表示诸如“最大的船”或“从左边数第二个法庭”等构造。我们提出了GeoSelect，一个将指称重新框定为类型化空间程序执行的无训练管道。一个冻结的仅文本语言模型将表达合成到一个小的领域特定语言中，一个良构性检查器接受该程序，而一个确定性执行器运行它。核心抽象是一个单一的评分候选集类型，在其下每个操作符进行组合：连续几何场将位置和接近度实现为密集的像素级地图，而离散集合和顺序操作符添加了极值、序数、计数并集和关系构造，这些是仅靠几何场无法表达的。由于执行是显式的，每个中间程序、场和排名都是可检查的，并且一个可靠性梯度将任何失败的程序降级为仅场的特例，因此每个表达都返回一个答案。GeoSelect在RRSIS-D测试中达到了58.86 mIoU，在RISBench测试中达到了55.27 mIoU，超过了RRSIS-D上最佳的无训练方法的两倍，且没有指称监督，并在单个GPU上运行。与候选者和分割器固定属性的受控比较将增益归因于显式执行，而非骨干网络；一个oracle分解将残差差距局限于RRSIS-D上的检测召回和RISBench上的选择，并且一次曝光审计确认了对预训练泄漏的鲁棒性。代码将在项目页面https://avalon-s.github.io/GeoSelect/接受后发布。

View on arXiv Download PDF AI Translation

cs.CV / 172 / 2607.03872

SharpSplat: Edge-Regularized 3D Gaussian Splatting for High Fidelity Urban Building Reconstruction from UAV images

SharpSplat：用于从无人机图像中高保真城市建筑重建的边缘正则化3D高斯点云

Vaid, Porus, Chopra, Shivam, Kumar, Vaibhav

Abstract

Reconstructing high-fidelity 3D building models from UAV imagery is essential for large-scale digital twin development. However, existing 3D Gaussian Splatting (3DGS) techniques often struggle with building facades, failing to capture sharp geometric transitions. To address this, we propose a semantic edge regularization framework that supervises 3DGS to produce crisp architectural boundaries. Our method leverages SAM 3 to generate precise building masks, from which we extract architecturally significant edges. During training, we align rendered image gradients with these extracted edges, forcing the Gaussians to converge into sharp structural geometries. Evaluations across campus environments, dense urban centers, and custom residential datasets demonstrate significant improvements in edge fidelity without requiring architectural modifications to the 3DGS pipeline. Our approach proves robust across diverse building types, roof geometries, and urban densities.

Chinese Translation

从无人机图像重建高保真的3D建筑模型对于大规模数字双胞胎的发展至关重要。然而，现有的3D高斯点云（3DGS）技术在建筑立面方面常常面临挑战，无法捕捉到清晰的几何过渡。为了解决这个问题，我们提出了一种语义边缘正则化框架，该框架监督3DGS以生成清晰的建筑边界。我们的方法利用SAM 3生成精确的建筑掩膜，从中提取出具有建筑意义的边缘。在训练过程中，我们将渲染图像的梯度与这些提取的边缘对齐，迫使高斯函数收敛到清晰的结构几何形状。对校园环境、密集城市中心和定制住宅数据集的评估表明，在不需要对3DGS流程进行建筑修改的情况下，边缘保真度显著提高。我们的方法在不同建筑类型、屋顶几何形状和城市密度下表现出良好的鲁棒性。

View on arXiv Download PDF AI Translation

cs.CV / 173 / 2607.03875

MACRO: Training-free Multi-plane Attention for Closeup Render Optimization

MACRO：无训练的多平面注意力用于特写渲染优化

Hodos, Nitzan, Amoyal, Roy, Fritz, Lior, Ideses, Ianir, Benaim, Sagie, Efrat, Netalee

Abstract

Close-up rendering, zooming into a scene well beyond any training camera, is important for virtual production and interactive 3D content, yet remains an open challenge. 3D Gaussian splatting (3DGS) enables high-fidelity, real-time novel view synthesis, but its rendering quality degrades at close range. Recent diffusion-based methods that enhance the rendering by conditioning on reference images from the training set produce significant artifacts in this setting. We analyze this failure and identify its root cause: the scale gap between the close-up and reference views. We show that the features in reference-conditioned enhancement models are not scale-invariant, causing cross-view attention to retrieve incorrect correspondences when the same content appears at different scales, and that this mismatch cannot be corrected in latent space because the VAE encoder is not scale-equivariant. Building on this analysis we introduce MACRO, Multi-plane Attention for Closeup Render Optimization, a training-free method for high-quality close-up novel view synthesis from 3DGS. MACRO resolves the scale gap by leveraging the scene's known 3D structure: it decomposes the close-up into depth planes, crops and resizes references in image space to match the scale of each plane before encoding, and applies a depth-aware attention mask so each token attends only to scale-matched references. The method requires no architectural changes or additional training. We further contribute two new close-up novel view synthesis benchmarks, the first standardized evaluation protocol for this setting, and demonstrate state-of-the-art results on both, outperforming existing 3DGS and diffusion-based methods on both reconstruction and perceptual metrics. Project page: https://nitzanhod.github.io/MACRO

Chinese Translation

特写渲染，即对场景进行放大，超出任何训练相机的范围，对于虚拟制作和互动3D内容至关重要，但仍然是一个未解决的挑战。3D高斯点云（3DGS）能够实现高保真、实时的新视图合成，但其在近距离的渲染质量下降。最近基于扩散的方法通过对训练集中的参考图像进行条件处理来增强渲染，但在这种情况下产生了显著的伪影。我们分析了这一失败，并确定其根本原因：特写视图与参考视图之间的尺度差距。我们展示了参考条件增强模型中的特征不是尺度不变的，导致跨视图注意力在相同内容以不同尺度出现时检索到错误的对应关系，并且这种不匹配无法在潜在空间中纠正，因为变分自编码器（VAE）编码器不是尺度等变的。在此分析的基础上，我们引入了MACRO（Multi-plane Attention for Closeup Render Optimization），一种无训练的方法，用于从3DGS进行高质量特写新视图合成。MACRO通过利用场景已知的3D结构来解决尺度差距：它将特写分解为深度平面，在图像空间中裁剪和调整参考图像的大小，以匹配每个平面的尺度，然后应用深度感知注意力掩码，使每个标记仅关注尺度匹配的参考。该方法无需架构更改或额外训练。我们还贡献了两个新的特写新视图合成基准，这是该设置的第一个标准化评估协议，并在这两个基准上展示了最先进的结果，超越了现有的3DGS和基于扩散的方法，在重建和感知指标上均表现优异。项目页面：https://nitzanhod.github.io/MACRO

View on arXiv Download PDF AI Translation

cs.CV / 174 / 2607.03883

SGF-CDNet: A Consistency-Discrepancy Graph Network over Semantic-Geometric Fused Nodes for Face Forgery Detection

SGF-CDNet：一种基于语义-几何融合节点的一致性-差异性图网络用于人脸伪造检测

Jiang, Jiayao, Liu, Bin, Yu, Nenghai

Abstract

The rapid advancement of deepfakes necessitates robust face forgery detection. Although forged faces may lack obvious artifacts, they often contain subtle disharmony among different facial regions. We propose SGF-CDNet, a Consistency-Discrepancy Graph Network (CD-GNN) over Semantic-Geometric Fused (SGF) nodes. First, SGF-CDNet constructs SGF nodes by deeply fusing semantic regions from face parsing with geometric information from facial landmarks, allowing nodes to capture both high-level concepts and precise geometric constraints. Next, a dual-path CD-GNN performs parallel relational reasoning on these nodes across two dimensions: consistency and discrepancy. The consistency path evaluates if facial components follow natural biological patterns, while the discrepancy path mines for structural tensions and feature conflicts introduced by forgeries. By integrating these processes, our model effectively identifies disharmonious relationships between facial components. Extensive experiments on public datasets demonstrate that SGF-CDNet achieves superior performance, establishing it as a reliable solution for face forgery detection.

Chinese Translation

深度伪造技术的快速发展要求具备强大的面部伪造检测能力。尽管伪造的人脸可能缺乏明显的伪造痕迹，但它们通常在不同面部区域之间存在微妙的不和谐。我们提出了SGF-CDNet，一种基于一致性-差异性图网络（CD-GNN）的模型，利用语义-几何融合节点（SGF）。首先，SGF-CDNet通过深度融合面部解析的语义区域与面部特征点的几何信息构建SGF节点，使节点能够捕捉高层次概念和精确的几何约束。接下来，双路径CD-GNN在这两个维度上对这些节点进行并行关系推理：一致性和差异性。一致性路径评估面部组件是否遵循自然生物模式，而差异性路径则挖掘伪造引入的结构张力和特征冲突。通过整合这些过程，我们的模型有效识别面部组件之间的不和谐关系。在公共数据集上的广泛实验表明，SGF-CDNet实现了优越的性能，确立了其作为人脸伪造检测可靠解决方案的地位。

View on arXiv Download PDF AI Translation

cs.CV / 175 / 2607.03891

BAT3R: Bootstrapping Articulated 3D Reconstruction from 2D Image Collections

BAT3R：从2D图像集合引导关节3D重建

Zadrozny, Jakub, Mac Aodha, Oisin, Bilen, Hakan

Abstract

3D reconstruction of articulated objects from a single image is challenging because large training datasets with paired image and 3D supervision are difficult to obtain. Recent point map-based methods achieve strong performance but rely on synthetic datasets rendered from manually created articulated 3D assets with carefully curated pose distributions. While camera viewpoints can be easily sampled, generating realistic object articulations remains costly and labor-intensive. We propose a training framework that reduces this requirement by leveraging unannotated 2D images collections with only a single rigged canonical mesh per category. Starting from a weak 3D shape predictor trained on canonical-pose renders, we iteratively estimate object articulation and camera pose by fitting the mesh to predicted point maps. The recovered articulations and viewpoints are then used to render updated synthetic training data, progressively improving the predictor. Despite using substantially weaker 3D supervision, our models achieve performance comparable with DualPM, which requires manually curated articulated training datasets.

Chinese Translation

从单幅图像重建关节物体的3D模型具有挑战性，因为获取配对的图像和3D监督的大型训练数据集非常困难。最近的基于点图的方法取得了良好的性能，但依赖于从手动创建的关节3D资产中渲染的合成数据集，这些数据集具有经过精心策划的姿态分布。虽然相机视角可以轻松采样，但生成逼真的物体关节仍然成本高昂且劳动密集。我们提出了一种训练框架，通过利用仅包含每个类别一个装配好的标准网格的未标注2D图像集合来降低这一要求。从在标准姿态渲染上训练的弱3D形状预测器开始，我们通过将网格拟合到预测的点图，迭代估计物体关节和相机姿态。恢复的关节和视点随后用于渲染更新的合成训练数据，逐步改善预测器。尽管使用了明显较弱的3D监督，我们的模型仍然实现了与需要手动策划的关节训练数据集的DualPM相当的性能。

View on arXiv Download PDF AI Translation

cs.CV / 176 / 2607.03899

DICT: Data Injection and Contrastive Trajectory Refinement for Conditional Image Generation with Diffusion Models

DICT：用于扩散模型条件图像生成的数据注入与对比轨迹精炼

Shang, Chunnan, Zhang, Xin, Wang, Zhizhong, Wang, Hongwei

Abstract

Diffusion models have become a dominant paradigm for conditional image generation, yet existing approaches generally follow two directions: task-specific designs that can improve performance but limit generalization, and training-free loss guidance that compresses rich conditions into scalar objectives and applies stepwise guidance, leading to information bottlenecks and error accumulation along the sampling trajectory. Given the urgent need for an effective unified framework across diverse conditional image generation tasks, we propose Data Injection and Contrastive Trajectory Refinement (DICT), a training-free inference method that enhances conditional image generation without introducing task-dependent architectures. DICT introduces Data Injection, where noise-perturbed conditional signals are integrated into early denoising stages; by performing guided denoising on these injected signals, DICT adaptively selects and distills task-salient information from the raw condition, effectively preserving spatial richness and ensuring precise condition-to-generation alignment. Furthermore, DICT applies Contrastive Trajectory Refinement across adjacent denoising states, enabling pairwise comparisons that progressively improve sample quality. These designs keep inference simple while improving cross-task transfer under a unified diffusion formulation. Extensive experiments on conditional image generation tasks (e.g., style transfer, image super-resolution, and image deblurring) show consistent gains in fidelity and perceptual quality over representative task-specific and loss-guided baselines.

Chinese Translation

扩散模型已成为条件图像生成的主导范式，但现有方法通常沿着两个方向发展：任务特定设计可以提高性能，但限制了泛化能力；无训练的损失引导将丰富的条件压缩为标量目标，并应用逐步引导，导致信息瓶颈和沿采样轨迹的误差累积。鉴于在多样的条件图像生成任务中迫切需要一个有效的统一框架，我们提出了数据注入与对比轨迹精炼（DICT），这是一种无训练的推理方法，能够增强条件图像生成而不引入任务依赖的架构。DICT引入数据注入，将噪声扰动的条件信号整合到早期去噪阶段；通过对这些注入信号进行引导去噪，DICT自适应地选择并提炼原始条件中的任务显著信息，有效地保持空间丰富性并确保条件与生成之间的精确对齐。此外，DICT在相邻的去噪状态之间应用对比轨迹精炼，进行成对比较，逐步提高样本质量。这些设计使推理过程保持简单，同时在统一的扩散框架下改善跨任务的迁移。对条件图像生成任务（例如，风格迁移、图像超分辨率和图像去模糊）的广泛实验表明，与代表性的任务特定和损失引导基线相比，FID和感知质量均有一致的提升。

View on arXiv Download PDF AI Translation

cs.CV / 177 / 2607.03900

USE: A Unified Self-Ensembling Framework for Test-Time Prompt Tuning

USE：一种统一的自我集成框架用于测试时提示调优

Jiang, Siru, Liang, Jian, He, Ran, Tan, Tieniu

Abstract

Test-time adaptation (TTA) has emerged as a popular paradigm for improving the performance of vision-language models (e.g., CLIP) on downstream tasks. Among existing CLIP-based TTA methods, Test-Time Prompt Tuning (TPT) is a pioneering work that optimizes textual prompts using multiple test-time augmentations and remains a strong baseline to date. In this work, we revisit TPT and reveal that its optimization can be interpreted as implicitly learning from self-generated pseudo labels. Building on this perspective, we propose a unified self-ensembling framework (USE) that ensures consistency between the optimization and inference stages. During optimization, we introduce a simple yet effective self-ensembling (SE) strategy that emphasizes the test image itself over its augmented views adaptively to obtain more reliable pseudo labels. To fully exploit the potential of augmentations, we further apply the same strategy at inference time, unifying the objectives of both stages. Notably, SE can also act as a lightweight optimization-free TTA method. Extensive experiments across multiple datasets demonstrate that SE and USE outperform their counterparts, respectively. Furthermore, SE yields consistent performance gains when integrated with existing TTA methods. The code is available at https://github.com/sirujiang/USE.

Chinese Translation

测试时适应（TTA）已成为提高视觉-语言模型（例如，CLIP）在下游任务中性能的热门范式。在现有的基于CLIP的TTA方法中，测试时提示调优（TPT）是一项开创性的工作，它通过多种测试时增强来优化文本提示，并且至今仍然是一个强有力的基线。在本研究中，我们重新审视了TPT，并揭示其优化可以被解释为从自生成的伪标签中隐式学习。基于这一视角，我们提出了一种统一的自我集成框架（USE），确保优化和推理阶段之间的一致性。在优化过程中，我们引入了一种简单而有效的自我集成（SE）策略，该策略自适应地强调测试图像本身而非其增强视图，以获得更可靠的伪标签。为了充分利用增强的潜力，我们在推理时进一步应用相同的策略，统一两个阶段的目标。值得注意的是，SE还可以作为一种轻量级的无优化TTA方法。针对多个数据集的广泛实验表明，SE和USE分别优于其对应的方法。此外，当与现有的TTA方法结合时，SE也能带来一致的性能提升。代码可在 https://github.com/sirujiang/USE 获取。

View on arXiv Download PDF AI Translation

cs.CV / 178 / 2607.03915

NavEYE: Vision-Centered Multi-Sensor Fusion-Based Situational Awareness System for Intelligent Surface Vehicles

NavEYE：基于视觉中心的多传感器融合智能地面车辆态势感知系统

Liua, Ryan Wen, Lianga, Junxiong, Wanga, Haoyu, Baoa, Mengwei

Abstract

With the rapid development of sensor and artificial intelligence (AI) technologies, intelligent surface vehicles (ISVs) have gained increasing attention from academia and industry. Their intelligence, reliability, and safety depend heavily on situational awareness in complex navigational environments. To achieve high-quality perception, we develop a vision-centered multi-sensor fusion system, named NavEYE, by exploiting complementary sensors, including the automatic identification system (AIS), radar, and RGB camera. Specifically, we first propose a multi-constrained gated data association method (MCGA) to accurately match low-temporal-resolution AIS data with high-temporal-resolution radar data. Their fusion result is then obtained by selectively implementing distance-aware adaptively weighted fusion (DAWF) and timeliness decay-based stitching fusion (TDSF), which reduce the uncertainty caused by AIS or radar data loss in real-world sensing scenarios. Based on accurate and robust visual object detection, we further associate and fuse AIS, radar, and visual data through joint constraints of normalized bearing and distance features. According to the fusion results, comprehensive information related to ships of interest can be automatically obtained, helping enhance situational awareness and reduce collision risk for ISVs. The feasibility, robustness, usability, and effectiveness of the proposed multi-sensor fusion method and situational awareness system are demonstrated through extensive experiments on a real-world sensing dataset collected from AIS, radar, and camera. The experimental results show the superior performance of our fusion method in both quantitative and qualitative evaluations. In addition, the shipboard NavEYE system can promote navigational safety for ISVs in complex and dynamic environments.

Chinese Translation

随着传感器和人工智能（AI）技术的快速发展，智能地面车辆（ISVs）越来越受到学术界和工业界的关注。它们的智能性、可靠性和安全性在很大程度上依赖于在复杂导航环境中的态势感知。为了实现高质量的感知，我们开发了一种以视觉为中心的多传感器融合系统，命名为NavEYE，利用包括自动识别系统（AIS）、雷达和RGB摄像头在内的互补传感器。具体而言，我们首先提出了一种多约束门控数据关联方法（MCGA），以准确匹配低时间分辨率的AIS数据与高时间分辨率的雷达数据。然后，通过选择性地实施距离感知自适应加权融合（DAWF）和基于时效衰减的拼接融合（TDSF），获得它们的融合结果，从而减少在实际感知场景中由于AIS或雷达数据丢失而造成的不确定性。基于准确且稳健的视觉目标检测，我们进一步通过归一化方位和距离特征的联合约束关联和融合AIS、雷达和视觉数据。根据融合结果，可以自动获取与感兴趣船只相关的综合信息，从而帮助增强态势感知并降低ISVs的碰撞风险。通过在从AIS、雷达和摄像头收集的真实世界感知数据集上进行广泛实验，展示了所提多传感器融合方法和态势感知系统的可行性、稳健性、可用性和有效性。实验结果表明，我们的融合方法在定量和定性评估中均表现出优越的性能。此外，船载NavEYE系统可以在复杂和动态环境中促进ISVs的导航安全。

View on arXiv Download PDF AI Translation

cs.CV / 179 / 2607.03934

EgoInertia-MI: A Multimodal Egocentric Vision and IMU Benchmark for Motor Impairment Assessment

EgoInertia-MI：用于运动障碍评估的多模态自我中心视觉与惯性测量单元基准

Alhamdoosh, Fatemah, Pala, Pietro, Mohamed, Abduallah, Arvind, DK

Abstract

Motor impairments, including tremor, bradykinesia, gait abnormalities, and postural instability, are common across many neurological and movement-related conditions. Conventional clinical assessments are often intermittent and may fail to capture subtle temporal variations in motor behavior. While wearable IMUs and third-person video have shown promise for objective motor assessment, third-person recordings raise privacy concerns and require constrained acquisition setups. In contrast, egocentric vision provides a more naturalistic and privacyaware alternative. In this work, we introduce EgoInertia-MI, a multimodal benchmark dataset combining synchronized egocentric video and wearable IMU signals for motor impairment analysis. The dataset contains 19 upper- and lower-body activities performed by healthy volunteers simulating varying levels of motor impairment severity levels: no impairment, mild impairment, and severe impairment. We establish two benchmark tasks: action recognition and motor impairment severity estimation, and evaluate multiple unimodal and multimodal baselines. Experimental results show that egocentric video provides strong cues for motor impairment assessment, while multimodal fusion achieves the best overall performance, reaching 0.78 Macro-F1 for severity estimation and 0.93 Macro-F1 for action recognition. These findings highlight the potential of combining egocentric vision and wearable sensing for ecologically valid and privacy-aware motor assessment. Code and data are available at:https://fatemah-alh.github.io/EgoInertia-MI-Page/.

Chinese Translation

运动障碍，包括震颤、运动迟缓、步态异常和姿势不稳定，在许多神经系统和运动相关疾病中普遍存在。传统的临床评估往往是间歇性的，可能无法捕捉到运动行为中的细微时间变化。尽管可穿戴惯性测量单元（IMU）和第三人称视频在客观运动评估中显示出潜力，但第三人称录制引发隐私问题，并且需要受限的采集设置。相比之下，自我中心视觉提供了一种更自然且注重隐私的替代方案。在本研究中，我们介绍了EgoInertia-MI，这是一个多模态基准数据集，结合了同步的自我中心视频和可穿戴IMU信号，用于运动障碍分析。该数据集包含19种由健康志愿者执行的上下肢活动，模拟不同程度的运动障碍严重性：无障碍、轻度障碍和重度障碍。我们建立了两个基准任务：动作识别和运动障碍严重性估计，并评估了多种单模态和多模态基线。实验结果表明，自我中心视频为运动障碍评估提供了强有力的线索，而多模态融合实现了最佳的整体性能，严重性估计的Macro-F1达到了0.78，动作识别的Macro-F1达到了0.93。这些发现突显了结合自我中心视觉和可穿戴传感器在生态有效且注重隐私的运动评估中的潜力。代码和数据可在：https://fatemah-alh.github.io/EgoInertia-MI-Page/ 获取。

View on arXiv Download PDF AI Translation

cs.CV / 180 / 2607.03945

A Large-Scale Dataset and a New Method for RemoteSensing Traffic Object Segmentation

大规模数据集及其在遥感交通目标分割中的新方法

Yang, Zhigang, Yao, Huiguang, Tian, Linmao, Li, Qiang, Wang, Qi

Abstract

Remote sensing imagery plays a crucial role in evaluating regional transportation capacity. However, existing segmentation datasets often lack diversity in object categories and scenes, limiting the ability of models to comprehensively evaluate trans portation capacity in real-world scenes. To alleviate this gap, we construct a large-scale and diverse dataset for transportation object segmentation, named as NWPU-Traffic. This dataset encompass four traffic object categories (car, airplane, ship, and train) and a wide range of scenes from 49 cities across 7 countries, with instance-level annotations to ensure precise segmentation of individual objects, which bridges critical shortcomings in resolution and scene diversity in existing datasets. Leveraging this dataset, we establish a benchmark with several popular segmentation networks. Furthermore, we propose a novel segmentation method that leverages spatial-channel preserving feature interaction and an adaptive feature decoder, enabling robust segmentation across varying scales and complex environments. Extensive experiments and ablation studies validate the effectiveness of our approach. The dataset and code are publicly available at https://github.com/CVer-Yang/NWPU-Traffic.

Chinese Translation

遥感影像在评估区域交通能力方面发挥着关键作用。然而，现有的分割数据集往往在目标类别和场景上缺乏多样性，限制了模型在真实场景中全面评估交通能力的能力。为了解决这一问题，我们构建了一个大规模且多样化的交通目标分割数据集，命名为 NWPU-Traffic。该数据集涵盖四种交通目标类别（汽车、飞机、船舶和火车），以及来自7个国家49个城市的广泛场景，并提供实例级注释，以确保对单个目标的精确分割，从而弥补了现有数据集中在分辨率和场景多样性方面的关键不足。利用该数据集，我们建立了一个基准，使用了几种流行的分割网络。此外，我们提出了一种新颖的分割方法，该方法利用空间-通道保持特征交互和自适应特征解码器，使得在不同尺度和复杂环境中实现稳健的分割。大量实验和消融研究验证了我们方法的有效性。数据集和代码已在 https://github.com/CVer-Yang/NWPU-Traffic 上公开发布。

View on arXiv Download PDF AI Translation

cs.CV / 181 / 2607.03949

TESSERA v2: Scaling Pixel-wise Earth Foundation Models

TESSERA v2：扩展逐像素地球基础模型

Feng, Zhengpeng, Jaffer, Sadiq, Shokar, Ira, Knezevic, Jovana, Elvers, Mark, Atzberger, Clement, Young, Robin, Naik, Aneesh, Robinson, Niall, Blake, Andrew, Coomes, David, Madhavapeddy, Anil, Keshav, Srinivasan

Abstract

Pixel-wise Earth-observation (EO) foundation models are now achieving state-of-the-art performance via generated spatial embeddings. However, how these models scale and how best to spend a pretraining budget remain poorly understood. We present the largest controlled scaling study for EO to date: 395 training runs on 1,024 GH200 superchips within a fixed pixel-wise Barlow Twins family, each evaluated on 15 downstream tasks. We find that pretraining loss barely predicts downstream performance (|Pearson r| < 0.2), so selecting models by loss wastes a large share of the compute. We also find that, as the training budget grows, the encoder and the data should grow together while the projector stays fixed, which gives a simple rule for allocating compute. Using this rule, we train a family of pixel-wise models (0.5B and 1B, with a 2B model in training) and distill them into compact students for embeddings-as-data deployment. The 21-million-parameter distilled TESSERA v2-1B-M in aggregate outperforms all open and proprietary models tested, some of which are orders of magnitude larger. These students produce Matryoshka representations that are inexpensive to serve: a 16-dimensional prefix keeps 92% of the full 128-dimensional performance at 1/8 of the storage. Upon completion of training we plan to release v2 global embeddings covering 2017-2025. Together, these results give a concrete, empirically grounded recipe for scaling pixel-wise EO foundation models: train large encoders, select by downstream performance, and distil into flexible student models. All code will be released at https://github.com/ucam-eo/tessera.

Chinese Translation

逐像素地球观测（EO）基础模型通过生成的空间嵌入现已实现了最先进的性能。然而，这些模型如何扩展以及如何最佳利用预训练预算仍然不够明确。我们展示了迄今为止最大的EO受控扩展研究：在固定的逐像素Barlow Twins系列中进行395次训练，使用1,024个GH200超级芯片，每个模型在15个下游任务上进行评估。我们发现，预训练损失几乎无法预测下游性能（|Pearson r| < 0.2），因此通过损失选择模型会浪费大量计算资源。我们还发现，随着训练预算的增加，编码器和数据应共同增长，而投影器保持不变，这为计算资源的分配提供了简单的规则。根据这一规则，我们训练了一系列逐像素模型（0.5B和1B，正在训练的2B模型），并将它们提炼为紧凑的学生模型以用于嵌入作为数据的部署。总参数为2100万的提炼TESSERA v2-1B-M在整体上超越了所有测试的开放和专有模型，其中一些模型的规模大数个数量级。这些学生模型生成的Matryoshka表示在服务时成本低廉：16维前缀在存储量为1/8的情况下保持了92%的128维完整性能。训练完成后，我们计划发布覆盖2017-2025年的v2全球嵌入。综合来看，这些结果为扩展逐像素EO基础模型提供了一个具体的、实证基础的方案：训练大型编码器，通过下游性能选择，并提炼为灵活的学生模型。所有代码将发布在https://github.com/ucam-eo/tessera。

View on arXiv Download PDF AI Translation

cs.CV / 182 / 2607.03960

Reward Lightning: Fast Video Generation via Homologous Preference Distillation

奖励闪电：通过同源偏好蒸馏实现快速视频生成

Cheng, Jiaxiang, Ma, Bing, Ren, Xuhua, Yu, Kai, Zhang, Peng, Zheng, Tianxiang, Lu, Qinglin

Abstract

Achieving simultaneous preference alignment and distillation acceleration in video diffusion models remains an open challenge. Existing methods optimize the two objectives over mismatched representation spaces, where improving one objective often compromises the other. To overcome this, we propose Reward Lightning, a unified framework that aligns and accelerates a video diffusion model within a single shared representation. Its central principle is homology: both objectives are evaluated on identical latent features, which mitigates the gradient conflicts that arise when they are optimized over disjoint representations. As a foundational component, we first introduce a latent reward model (LRM) that scores videos directly in the latent space, without decoding back to the pixel space. Building on the LRM, homologous preference distillation (HPD) reuses this shared backbone to perform adversarial distillation and preference alignment jointly, yielding few-step generators that remain faithful and well aligned. Extensive experiments demonstrate that the LRM surpasses pixel-level and latent-level reward baselines by $11.0\%$ and $14.7\%$ in preference accuracy, and that Reward Lightning generates high-fidelity videos in merely $1$ to $4$ steps, improving the average VBench score by $2.1\%$ while leading in text alignment, motion quality, and visual quality. Project page: https://reward-lightning.github.io.

Chinese Translation

在视频扩散模型中，实现偏好对齐与蒸馏加速的同时进行仍然是一个未解决的挑战。现有方法在不匹配的表示空间中优化这两个目标，改善一个目标往往会妥协另一个目标。为了解决这一问题，我们提出了奖励闪电（Reward Lightning），这是一个统一框架，在单一共享表示中对视频扩散模型进行对齐和加速。其核心原则是同源性：两个目标在相同的潜在特征上进行评估，这减轻了在不相交的表示上优化时产生的梯度冲突。作为基础组件，我们首先引入了潜在奖励模型（Latent Reward Model, LRM），该模型在潜在空间中直接对视频进行评分，而无需解码回像素空间。在LRM的基础上，同源偏好蒸馏（Homologous Preference Distillation, HPD）重用这一共享骨干网络，联合执行对抗蒸馏和偏好对齐，从而生成在少量步骤内仍然忠实且良好对齐的生成器。大量实验表明，LRM在偏好准确性上超越了像素级和潜在级的奖励基线，分别提高了$11.0\%$和$14.7\\%$，而奖励闪电在仅需$1$到$4$步内生成高保真视频，平均VBench得分提高了$2.1\\%$，在文本对齐、运动质量和视觉质量方面表现优异。项目页面：https://reward-lightning.github.io。

View on arXiv Download PDF AI Translation

cs.CV / 183 / 2607.03972

DS-SAC: Density Search for Sample Consensus

DS-SAC：样本一致性的密度搜索

Thapa, Suraj, Islam, Muhammad Aminul

Abstract

Robust geometric model estimation is a fundamental problem in computer vision. RANSAC and its variants remain widely used for this task; however, they rely on stochastic minimal sampling. In this article, we propose Density Search Sample Consensus (DS-SAC), a deterministic robust estimation framework, that avoids repeated random sampling by searching dense regions. Starting from an initial model estimated from the available points, the method performs local exploration via forward and backward search. To facilitate global exploration, DS-SAC recursively partitions the point set using signed residuals and searches each valid partition for high-consensus models. We show that DS-SAC has polynomial complexity with respect to the number of points, making it an efficient alternative to stochastic consensus-based methods. Experiments on large-scale real-world datasets for homography, fundamental matrix, and essential matrix estimation show that DS-SAC achieves higher AUC scores, competitive or lower median pose errors, and faster runtime compared with widely used robust estimators, including RANSAC, MAGSAC, LO-RANSAC, and GC-RANSAC.

Chinese Translation

鲁棒几何模型估计是计算机视觉中的一个基本问题。RANSAC及其变体仍然广泛用于这一任务；然而，它们依赖于随机最小采样。本文提出了一种确定性的鲁棒估计框架——密度搜索样本一致性（DS-SAC），该方法通过搜索密集区域来避免重复的随机采样。从可用点估计的初始模型出发，该方法通过前向和后向搜索进行局部探索。为了促进全局探索，DS-SAC使用带符号残差递归划分点集，并在每个有效划分中搜索高一致性模型。我们证明了DS-SAC在点数方面具有多项式复杂度，使其成为随机一致性方法的高效替代方案。在大规模真实世界数据集上进行的单应性、基础矩阵和本质矩阵估计实验表明，DS-SAC在AUC得分上表现更高，具有竞争力或更低的中位姿态误差，以及相比于广泛使用的鲁棒估计方法（包括RANSAC、MAGSAC、LO-RANSAC和GC-RANSAC）更快的运行时间。

View on arXiv Download PDF AI Translation

cs.CV / 184 / 2607.03990

InSpace: Structure-Aware 3D Indoor Scene Generation from a Single 360{\deg} Image

InSpace：基于单幅360°图像的结构感知3D室内场景生成

Koo, Gwanhyeong, Kim, Hyunsu, Kim, Youngji, Lee, Taejae, Lim, Siwoo, Yoon, Sunjae, Yeon, Suyong, Yoo, Chang D.

Abstract

Recent advances in single image-to-3D generation have enabled high-quality asset synthesis, yet extending these capabilities to indoor scene generation remains challenging. Existing methods focus on asset-level generation while neglecting the structural layout, which is essential for downstream applications and serves as the spatial anchor for grounding assets. However, a single image with a limited field of view lacks the spatial coverage to recover a coherent global layout. To this end, we use a 360{\deg} image represented in equirectangular projection (ERP) and propose InSpace, a structure-aware framework for 3D indoor scene generation. InSpace comprises three stages: (1) estimating partial scene geometry as spatial priors, (2) generating coarse scene structure with view-selective cross-attention, and (3) producing detailed layout and asset geometry with textures through a global-local hybrid attention, using flow matching. We also propose ERP-FRONT, a paired ERP-Image-to-3D indoor scene dataset based on 3D-FRONT. Experiments show that InSpace generates complete 3D indoor scenes with structural layout, along with separate textured assets from a single ERP image, achieving strong performance across 3D and 2D metrics. Project Page: https://kookie12.github.io/InSpace-Project-Page/

Chinese Translation

最近在单幅图像到3D生成方面的进展使得高质量资产合成成为可能，但将这些能力扩展到室内场景生成仍然面临挑战。现有方法侧重于资产级生成，而忽视了结构布局，这对于下游应用至关重要，并且作为资产的空间锚点。然而，单幅图像由于视野有限，缺乏恢复一致全局布局的空间覆盖。为此，我们使用以等距矩形投影（ERP）表示的360°图像，并提出了InSpace，一个结构感知的3D室内场景生成框架。InSpace包括三个阶段：（1）估计部分场景几何作为空间先验；（2）通过视图选择性交叉注意生成粗略场景结构；（3）通过全局-局部混合注意和流匹配生成详细布局和资产几何及纹理。我们还提出了ERP-FRONT，一个基于3D-FRONT的配对ERP图像到3D室内场景数据集。实验表明，InSpace能够从单幅ERP图像生成完整的3D室内场景及其结构布局，并生成单独的纹理资产，在3D和2D指标上均表现出强劲的性能。项目页面：https://kookie12.github.io/InSpace-Project-Page/

View on arXiv Download PDF AI Translation

cs.CV / 185 / 2607.03994

Full Glyph Images Beat Token Embeddings: A Controlled Study for Transformers

完整字形图像优于标记嵌入：针对变换器的控制研究

Xiang, Shuyang, Guan, Hao

Abstract

Modern language models generally represent text as sequences of discrete token embeddings, an assumption deeply rooted in current practice but rarely questioned. We challenge this representation, especially for Chinese, by replacing index-based token embeddings entirely with a single rasterized image of the character sequence, processed by a vision encoder composed of a shared ResNet and a shallow Vision Transformer. To isolate the role of input representation, we construct a dual-branch controlled framework in which both a Vision-based model and an index-based baseline share an identical decoder backbone, training objective, optimizer, and data curriculum. Any performance difference is therefore attributable to the input modality only. Across all tested decoder backbones, the Vision-based model consistently outperforms the baseline, reaching a peak accuracy of 0.429 versus 0.355 for the index-based baseline,that is, a 21% relative improvement, while converging in about half the number of training epochs. The advantage emerges especially within the first five epochs (under 21% of total data) and persists under moderate character corruption: the corrupted Vision model matches the clean index-based baseline. Ablation studies reveal that the advantage requires both spatially coherent input and a ViT encoder with 2D positional encodings. A cross-script comparison on English shows the advantage does not transfer directly to alphabetic writing systems, suggesting that the uniform visual density and radical structure of Chinese characters are enabling conditions. These findings suggest that transformers are more modality-agnostic than commonly assumed, and that discrete tokenization is not a fundamental requirement for Chinese language modeling.

Chinese Translation

现代语言模型通常将文本表示为离散标记嵌入的序列，这一假设在当前实践中根深蒂固，但很少受到质疑。我们挑战这种表示，特别是针对中文，通过用字符序列的单一光栅化图像完全替代基于索引的标记嵌入，该图像由一个共享的ResNet和一个浅层视觉变换器（Vision Transformer）组成的视觉编码器处理。为了隔离输入表示的作用，我们构建了一个双分支控制框架，其中视觉模型和基于索引的基线共享相同的解码器骨干、训练目标、优化器和数据课程。因此，任何性能差异仅可归因于输入模态。在所有测试的解码器骨干中，基于视觉的模型始终优于基线，达到0.429的峰值准确率，而基于索引的基线为0.355，即相对提高了21%，同时收敛所需的训练轮数约为基线的一半。该优势特别在前五个训练轮（占总数据的21%以下）内显现，并在适度字符损坏下持续存在：损坏的视觉模型与干净的基于索引的基线相匹配。消融研究表明，该优势需要空间一致的输入和具有二维位置编码的ViT编码器。对英语的跨脚本比较显示，该优势并不直接转移到字母书写系统，表明汉字的均匀视觉密度和部首结构是促进条件。这些发现表明，变换器比普遍认为的更具模态无关性，而离散标记化并不是中文语言建模的基本要求。

View on arXiv Download PDF AI Translation

cs.CV / 186 / 2607.04017

SAGE: Synchronized Action-Gaze Recognition and Anticipation for Human Behavior Understanding

SAGE：用于理解人类行为的同步动作-注视识别与预测

Kuang, Chenyi, Agarwal, Nakul

Abstract

Human object interaction (HOI), gaze pattern, and their anticipation are intricately linked, providing valuable insights into cognitive processes, intentions, and behavior. However, most existing models handle gaze and actions separately, missing both their interdependence and the advantages of a unified solution. This paper presents a novel unified framework, SAGE (Synchronized Action-GazE), which integrates simultaneous recognition and anticipation of both HOI and human gaze into a single unified end-to-end trainable model. Our approach leverages a transformer-based architecture and incorporates gaze data into spatiotemporal attention mechanisms to simultaneously predict current and future human actions and gaze behavior. We explore this bidirectional relationship between gaze and actions under different scenarios, whether requiring a close-up, detailed view (egocentric) or a wider, more contextual view (exocentric), making our framework versatile for various applications. Additionally, due to lack of datasets for comprehensive analysis of both HOI and gaze in exocentric videos, we establish a new benchmark Exo-Cook to facilitate further research in this domain. Experiments on three benchmark datasets: VidHOI, EGTEA Gaze+, and Exo-Cook show that jointly modeling gaze and actions across current and future frames achieves consistently strong results, often surpassing specialized state-of-the-art models tailored to individual tasks. By unifying actions and attention in a comprehensive way, our work lays the groundwork for more intuitive human-machine interaction.

Chinese Translation

人类物体交互（HOI）、注视模式及其预测之间存在着复杂的联系，为认知过程、意图和行为提供了宝贵的见解。然而，大多数现有模型将注视和动作分开处理，忽视了它们之间的相互依赖性以及统一解决方案的优势。本文提出了一种新颖的统一框架SAGE（Synchronized Action-GazE），该框架将HOI和人类注视的同时识别与预测整合为一个统一的端到端可训练模型。我们的方法利用基于变换器（transformer）的架构，并将注视数据纳入时空注意力机制，以同时预测当前和未来的人类动作及注视行为。我们在不同场景下探索注视与动作之间的双向关系，无论是需要近距离、详细视图（自我中心）还是更广泛、更多上下文的视图（外部中心），使我们的框架适用于各种应用。此外，由于缺乏用于全面分析外部中心视频中HOI和注视的数据集，我们建立了一个新的基准Exo-Cook，以促进该领域的进一步研究。在三个基准数据集VidHOI、EGTEA Gaze+和Exo-Cook上的实验表明，跨当前和未来帧联合建模注视和动作能够实现持续强劲的结果，通常超越针对单个任务量身定制的专业最先进模型。通过以全面的方式统一动作和注意力，我们的工作为更直观的人机交互奠定了基础。

View on arXiv Download PDF AI Translation

cs.CV / 187 / 2607.04020

Paired Uterine Whole-Slide Images and Pathology Reports for Multimodal Computational Pathology

配对的子宫全幻灯片图像与病理报告用于多模态计算病理学

Li, Han, Liu, Jingsong, Ura, Ayako, Hou, Junlin, Xu, Zhengyang, Kazemi, Azar, Thaeter, Oskar, Grashei, Christian, Gülhan, Fabian, Nasirigerdeh, Reza, Ma, Xun, Yan, Rui, Chen, Hao, Zhou, S. Kevin, Navab, Nassir, Mogler, Carolin, Schüffler, Peter

Abstract

Uterine diseases represent an important category of gynecologic pathology and require accurate histopathological assessment for diagnosis and treatment planning. Whole-slide images (WSI) have enabled the digital transformation of pathology workflows and provided new opportunities for artificial intelligence (AI) in computational pathology. In particular, multimodal models that jointly analyze histopathology images and pathology reports have shown promising potential for automated pathology report generation and AI-assisted diagnosis. However, the development of such systems remains limited by the scarcity of datasets that pair whole-slide images with clinically meaningful pathology reports. Instead, existing pathology datasets focus on patch- or slide-level annotations of a single endpoint (e.g., disease class), which do not fully capture the rich information in full clinical diagnostic workflow reports. Here, we introduce TUM-Uteria, a uterine pathology dataset comprising WSIs paired with diagnostic pathology reports at both the case and slide levels, collected from a tertiary medical center. The dataset contains 216 clinical cases, comprising 455 slide-level WSI-report pairs. The dataset underwent a structured multi-stage validation procedure involving board-certified pathologists to ensure reliable annotations. TUM-Uteria supports research in computational pathology, including whole-slide image analysis, multimodal learning, and automated pathology report generation.

Chinese Translation

子宫疾病是妇科病理学中的一个重要类别，需要准确的组织病理学评估以进行诊断和治疗计划。全幻灯片图像（WSI）使病理工作流程实现了数字化转型，并为计算病理学中的人工智能（AI）提供了新的机会。特别是，联合分析组织病理图像和病理报告的多模态模型在自动化病理报告生成和AI辅助诊断方面显示出良好的潜力。然而，此类系统的发展仍受到将全幻灯片图像与临床有意义的病理报告配对的数据集稀缺的限制。现有的病理数据集主要集中于单一终点（例如，疾病类别）的补丁或幻灯片级注释，未能充分捕捉完整临床诊断工作流程报告中的丰富信息。在此，我们介绍了TUM-Uteria，一个包含配对的WSI和诊断病理报告的子宫病理数据集，数据来自一家三级医疗中心，涵盖案例和幻灯片级别。该数据集包含216个临床案例，包含455个幻灯片级WSI-报告对。该数据集经过结构化的多阶段验证程序，由经过认证的病理学家进行，以确保注释的可靠性。TUM-Uteria支持计算病理学的研究，包括全幻灯片图像分析、多模态学习和自动化病理报告生成。

View on arXiv Download PDF AI Translation

cs.CV / 188 / 2607.04044

SiamJEPA: On the Role of Siamese Student Encoders in JEPA

SiamJEPA：西米学生编码器在JEPA中的作用

Yamada, Makoto

Abstract

Recently, Joint Embedding Predictive Architectures (JEPAs) have attracted significant attention in the computer vision and machine learning communities as a promising framework for self-supervised representation learning. Unlike masked autoencoders that reconstruct pixels, JEPA models learn representations by predicting latent embeddings of masked regions. Existing JEPA-based methods, such as I-JEPA and V-JEPA, typically employ a single encoder in the student network. In contrast, using Siamese encoders for student network is more naturally aligned with brain-inspired representation learning frameworks, yet their role in JEPA models remains largely unexplored. In this paper, we investigate the effect of Siamese student encoders in JEPA-based representation learning. To this end, we propose SiamJEPA, masked Siamese student encoders equipped with an exponential moving average (EMA) teacher network. SiamJEPA can also be viewed as a JEPA formulation of the brain-inspired representation learning model PhiNet. Through extensive experiments on ImageNet linear probing, we demonstrate that Siamese encoders act as an effective regularizer for the JEPA objective, improving representation separability and accelerating learning during the early stages of training. Furthermore, SiamJEPA consistently outperforms comparable single-encoder JEPA variants under limited training budgets and achieves higher linear probing accuracy than Masked Autoencoders (MAE) which requires longer training. Our findings reveal that Siamese student encoders are not merely an architectural choice but constitute an important inductive bias for predictive representation learning. These results provide new insights into the design of JEPA-based models and suggest that incorporating Siamese student architectures offers a simple yet effective approach for improving self-supervised representation learning.

Chinese Translation

近年来，联合嵌入预测架构（Joint Embedding Predictive Architectures, JEPA）在计算机视觉和机器学习领域引起了广泛关注，成为自监督表示学习的一个有前景的框架。与重建像素的掩码自编码器不同，JEPA模型通过预测被掩盖区域的潜在嵌入来学习表示。现有的基于JEPA的方法，如I-JEPA和V-JEPA，通常在学生网络中使用单一编码器。相比之下，使用西米编码器作为学生网络与受脑启发的表示学习框架更为自然对齐，但它们在JEPA模型中的作用仍然未被充分探索。本文研究了西米学生编码器在基于JEPA的表示学习中的影响。为此，我们提出了SiamJEPA，这是一种配备指数移动平均（Exponential Moving Average, EMA）教师网络的掩码西米学生编码器。SiamJEPA也可以被视为受脑启发的表示学习模型PhiNet的JEPA形式。通过在ImageNet线性探测上的广泛实验，我们证明西米编码器作为JEPA目标的有效正则化器，提高了表示的可分性，并加速了训练早期的学习。此外，SiamJEPA在有限的训练预算下始终优于可比的单编码器JEPA变体，并且在准确性上超过了需要更长训练时间的掩码自编码器（Masked Autoencoders, MAE）。我们的发现揭示西米学生编码器不仅仅是一个架构选择，而是预测表示学习的重要归纳偏置。这些结果为基于JEPA的模型设计提供了新的见解，并表明结合西米学生架构提供了一种简单而有效的方式来改善自监督表示学习。

View on arXiv Download PDF AI Translation

cs.CV / 189 / 2607.04057

PreSIST: Vision-Language-Informed Object Persistence Prediction in Open-World Scenes

PreSIST：基于视觉-语言的信息进行开放世界场景中的物体持久性预测

Adkins, Amanda, Ravisankar, Tarunvidyut, Biswas, Joydeep

Abstract

Robots deployed over long periods must reason about environments that change over time. Existing long-term perception systems often address object change reactively, updating their maps only after revisiting a scene and observing that an object has moved. Instead, robots should reason proactively about how long objects are likely to persist using the context in which they appear. For example, a car at a traffic light and a car in a parking spot share the same semantic class, but their contexts imply different persistence durations. We propose PreSIST (Predictive Scene-conditioned Instance Survival over Time), a method for predicting whether an observed object will remain in its last seen pose at arbitrary future times. PreSIST estimates instance-level persistence priors from object properties and scene context, then integrates these priors with a probabilistic persistence filter as observations become available. Its key insight is that the reasoning capabilities of vision-language models (VLMs) can relate scene context to likely object use and human activity, enabling persistence prediction before long-term observations are available. We develop two interchangeable variants: PreSIST-Lang, which estimates persistence priors using a VLM, and PreSIST-Vis, a novel vision-only model trained using PreSIST-Lang pseudo-labels for efficient deployment. Experiments on a new dataset of in-the-wild object persistence annotations show that PreSIST-Lang and PreSIST-Vis outperform baselines on open-world persistence prediction.

Chinese Translation

长期部署的机器人必须对随时间变化的环境进行推理。现有的长期感知系统通常以反应的方式处理物体变化，仅在重新访问场景并观察到物体移动后更新其地图。相反，机器人应该主动推理物体可能持续的时间，利用其出现的上下文。例如，交通信号灯前的汽车和停车位中的汽车属于相同的语义类别，但它们的上下文暗示了不同的持久性持续时间。我们提出了PreSIST（基于场景条件的实例生存预测），一种预测观察到的物体在任意未来时刻是否会保持在最后一次看到的姿态的方法。PreSIST从物体属性和场景上下文中估计实例级持久性先验，然后随着观察数据的可用性，将这些先验与概率持久性滤波器结合。其关键见解在于，视觉-语言模型（VLM）的推理能力可以将场景上下文与可能的物体使用和人类活动关联起来，从而在长期观察数据可用之前实现持久性预测。我们开发了两种可互换的变体：PreSIST-Lang，使用VLM估计持久性先验，以及PreSIST-Vis，一种使用PreSIST-Lang伪标签进行训练的全新视觉模型，以实现高效部署。在一个新的野外物体持久性注释数据集上的实验表明，PreSIST-Lang和PreSIST-Vis在开放世界持久性预测中优于基线方法。

View on arXiv Download PDF AI Translation

cs.CV / 190 / 2607.04069

Enhancing Implicit Neural Representations with Image Feature Embedding for Unsupervised Cardiac Cine MRI Reconstruction

通过图像特征嵌入增强隐式神经表示以进行无监督心脏动态磁共振成像重建

Lyu, Donghang, Staring, Marius, Dong, Yiming, Jochen, Keupp, Lamb, Hildo J., Doneva, Mariya

Abstract

Cardiac cine Magnetic Resonance Imaging (MRI) is a critical diagnostic tool that provides dynamic insights for radiologists. To accelerate acquisition, under-sampled k-space data is often used, requiring reconstruction methods that combine coil sensitivity encoding with prior information to recover missing data. Deep learning approaches have gained more attention for leveraging data-adaptive priors. While supervised learning approaches are a common choice, they depend on fully sampled reference data, which is not always available. Unsupervised methods eliminate the need for fully sampled reference data, which can be advantageous in cardiac cine MRI reconstruction. Among them, implicit neural representations (INRs) have shown great potential due to their simple architecture and good quality reconstructions. In this work, we propose an image-domain dual-branch INR framework, termed I-FP-INR, which extends the original INR design by introducing an additional feature-processing branch. This design aims to extract complementary feature embeddings to enhance the overall representation, thereby benefiting reconstruction. Extensive evaluations on both public datasets and in-house data show consistent improvements over baseline methods in reconstruction quality, with strong robustness across varied scenarios.

Chinese Translation

心脏动态磁共振成像（MRI）是一种关键的诊断工具，为放射科医生提供动态洞察。为了加快采集速度，通常使用欠采样的k空间数据，这需要结合线圈灵敏度编码和先验信息的重建方法来恢复缺失数据。深度学习方法因其利用数据自适应先验而受到越来越多的关注。虽然监督学习方法是常见选择，但它们依赖于完全采样的参考数据，而这些数据并不总是可用。无监督方法消除了对完全采样参考数据的需求，这在心脏动态MRI重建中具有优势。其中，隐式神经表示（INRs）因其简单的架构和良好的重建质量而显示出巨大的潜力。在本研究中，我们提出了一种图像域双分支INR框架，称为I-FP-INR，该框架通过引入额外的特征处理分支扩展了原始INR设计。该设计旨在提取互补的特征嵌入，以增强整体表示，从而有利于重建。在公共数据集和内部数据上的广泛评估显示，在重建质量上相较于基线方法有一致的改善，并且在各种场景中表现出强大的鲁棒性。

View on arXiv Download PDF AI Translation

cs.CV / 191 / 2607.04079

Seeing Once is Enough? Online Geometry-Aware Token Pruning for 3D Question Answering

一次观看就足够了吗？面向3D问答的在线几何感知令牌剪枝

Lai, Ruei-Chi, Solarte, Bolivar, Wu, Chin-Hsuan, Tsai, Yi-Hsuan, Sun, Min

Abstract

Recent Multi-modal Large Language Models (MLLMs) have demonstrated remarkable performance on 2D question answering tasks. However, extending these models to the 3D question answering remains challenging, as they typically require multiple views of the scene, which incurs substantial computational cost at inference. To mitigate this issue, existing solutions rely on strategic frame selection or token-merging algorithms that require preprocessing in advance all frames of the scene, i.e., an offline fashion. In contrast, we propose the first online token-pruning method that can be integrated seamlessly with current MLLM models for 3D question answering tasks, without additional training and with lower memory usage.Our key insight is to project each input frame into a shared voxel space using depth information and camera pose, identifying spatially-overlapped regions across frames and selectively pruning redundant image tokens before they enter the language model. Our method enables efficient online processing while reducing up to 50% of token usage. We apply this approach to Qwen2.5-VL-7B and Qwen3-VL-8B, demonstrating improved performance on the ScanQA, SQA3D, and OpenEQA-HM3D benchmarks.

Chinese Translation

近期的多模态大型语言模型（MLLMs）在2D问答任务上表现出色。然而，将这些模型扩展到3D问答仍然具有挑战性，因为它们通常需要场景的多个视图，这在推理时会产生大量的计算成本。为了解决这个问题，现有的解决方案依赖于战略帧选择或令牌合并算法，这些算法需要提前对场景的所有帧进行预处理，即以离线方式进行。相比之下，我们提出了首个在线令牌剪枝方法，可以与当前的MLLM模型无缝集成，用于3D问答任务，无需额外训练且内存使用更低。我们的关键见解是利用深度信息和相机姿态将每个输入帧投影到共享的体素空间中，识别帧之间的空间重叠区域，并在图像令牌进入语言模型之前选择性地剪枝冗余的图像令牌。我们的方法实现了高效的在线处理，同时减少了多达50%的令牌使用。我们将该方法应用于Qwen2.5-VL-7B和Qwen3-VL-8B，展示了在ScanQA、SQA3D和OpenEQA-HM3D基准测试上的性能提升。

View on arXiv Download PDF AI Translation

cs.CV / 192 / 2607.04098

Sparse4D-Radar: An Efficient and Robust Framework for Surround-View 3D Object Detection via 4D Radar-Camera Fusion

Sparse4D-Radar：一种高效且稳健的周视3D物体检测框架，通过4D雷达-摄像头融合实现

Ai, Fuyuan, Tan, Yuchen, Chen, Jiehui, Xu, Zhiwei, Song, Chunyi

Abstract

In recent years, 4D imaging radar has gained wide attention in autonomous driving for its robustness against harsh weather and ability to output target velocity. Nevertheless, mainstream 4D radar-camera fusion methods only support front-view perception, lacking mature solutions for surround-view sensing. Directly expanding these pipelines to full 360{\deg} coverage introduces excessive computation cost and limits real-world deployment. To tackle these limitations, this work proposes Sparse4D-Radar, an efficient robust surround-view multi-modal fusion framework. We first design a Deformable Fusion module to embed radar-camera features into sparse queries, constructing the lightweight base version Sparse4D-Radar-Base. Two dedicated modules are further introduced to boost localization accuracy and modality stability: Velocity-Consistency Sampling (VCS) refines features via radar velocity cues for motion awareness, and Adaptive Modality Gating (AMG) dynamically adjusts cross-modal fusion weights according to feature confidence. Combining all components, we build Sparse4D-Radar-Acc for high-precision detection demands. Comprehensive experiments on OmniHD-Scenes verify that our approach achieves state-of-the-art surround-view 3D detection performance. Compared with prior arts, our method obtains over 7% mAP and 10% ODS improvements under complex driving scenes while running at nearly 10 FPS, striking a favorable trade-off among detection accuracy, environmental robustness and inference efficiency. Our open-source code is available at https://github.com/Aiuan/Sparse4D-Radar.

Chinese Translation

近年来，4D成像雷达因其在恶劣天气下的稳健性和输出目标速度的能力，在自动驾驶领域受到了广泛关注。然而，主流的4D雷达-摄像头融合方法仅支持前视感知，缺乏成熟的周视感知解决方案。直接将这些管道扩展到360°的全覆盖会引入过高的计算成本，并限制实际部署。为了解决这些限制，本文提出了Sparse4D-Radar，一种高效且稳健的周视多模态融合框架。我们首先设计了一个可变形融合模块，将雷达-摄像头特征嵌入稀疏查询中，构建了轻量级基础版本Sparse4D-Radar-Base。进一步引入了两个专用模块以提升定位精度和模态稳定性：速度一致性采样（Velocity-Consistency Sampling, VCS）通过雷达速度线索精炼特征以增强运动感知，而自适应模态门控（Adaptive Modality Gating, AMG）根据特征置信度动态调整跨模态融合权重。结合所有组件，我们构建了Sparse4D-Radar-Acc以满足高精度检测需求。在OmniHD-Scenes上的全面实验验证了我们的方法在周视3D检测性能上达到了最先进水平。与之前的研究相比，我们的方法在复杂驾驶场景下获得了超过7%的mAP和10%的ODS提升，同时以接近10 FPS的速度运行，在检测精度、环境鲁棒性和推理效率之间取得了良好的平衡。我们的开源代码可在https://github.com/Aiuan/Sparse4D-Radar获取。

View on arXiv Download PDF AI Translation

cs.CV / 193 / 2607.04101

The Multipath Blind Spot: $K$-Agnostic Robust Calibration for Sparse-Anchor Metric Depth from Frozen Foundations

多路径盲点：$K$-无关鲁棒校准用于稀疏锚点的度量深度从冻结基础

Roy, Sohag, Misra, Rajesh, Shastravidyananda, Swami, Maharaj, Tamal

Abstract

Monocular depth foundations predict domain-general relative depth but lack absolute scale; a handful of sparse metric anchors from a range sensor can calibrate them to metric depth, an attractive alternative to metric-supervised training. Existing sparse-anchor calibration methods, however, assume the anchors are clean, whereas real sensors produce outliers that are present with the wrong value -- time-of-flight multipath, mixed pixels -- not merely missing. We show that the established residual-on-CFA calibration recipe collapses under such outliers, and that the strongest publicly deployed method, VI-Depth, has a structural multipath blind spot: robust to missing anchors, it falls behind an unprotected baseline on three of four datasets when anchors are present but wrong. We propose Multipath-Robust Anchor Calibration (MRAC), a parameter-free, inference-time wrapper that gates anchors by foundation consistency -- a Theil--Sen fit and a median-absolute-deviation test against the foundation's own relative-depth ordering -- before a single call to the calibration head. MRAC adds no learned parameters, runs its selection in $\approx 50\,\mu$s on CPU, and serves anchor budgets $K \in [5,200]$ from one checkpoint. On a $320$-cell benchmark with a same-backbone, same-architecture control, MRAC strictly wins $84\%$ of same-backbone cells across all four outlier families and, against VI-Depth, wins all twelve corrupted multipath cells and all sixteen KITTI cells, reducing KITTI multipath AbsRel by $3.2\times$ ($0.489$ to $0.151$) at zero retraining.

Chinese Translation

单目深度基础预测领域通用的相对深度，但缺乏绝对尺度；来自范围传感器的一小部分稀疏度量锚点可以将其校准为度量深度，这是度量监督训练的一个有吸引力的替代方案。然而，现有的稀疏锚点校准方法假设锚点是干净的，而真实传感器产生的异常值则存在错误值——飞行时间多路径、混合像素——而不仅仅是缺失。我们展示了现有的基于CFA的残差校准方法在这种异常值下崩溃，而最强的公开部署方法VI-Depth存在结构性多路径盲点：虽然对缺失锚点具有鲁棒性，但在锚点存在但错误的情况下，在四个数据集中有三个落后于未保护的基线。我们提出了多路径鲁棒锚点校准（MRAC），这是一种无参数的推理时间包装器，通过基础一致性来筛选锚点——使用Theil-Sen拟合和相对于基础自身相对深度排序的中位数绝对偏差测试——在对校准头进行单次调用之前。MRAC不增加任何学习参数，在CPU上以约50μs的速度运行其选择，并为来自一个检查点的锚点预算$K ext{ in } [5,200]$提供服务。在一个具有相同骨干网和相同架构控制的320单元基准测试中，MRAC在所有四个异常值类别中严格赢得了84%的相同骨干单元，并且在与VI-Depth的比较中，赢得了所有十二个受损的多路径单元和所有十六个KITTI单元，将KITTI多路径的绝对相对误差降低了3.2倍（从0.489降至0.151），且没有重新训练。

View on arXiv Download PDF AI Translation

cs.CV / 194 / 2607.04119

SOV-CAD: Stepwise Orthographic Views Guided CAD Modeling Sequence Reconstruction

SOV-CAD：逐步正投影引导的CAD建模序列重建

Feng, Zhaopeng, Zhi, Chen, Zhang, Xuhong, Feng, Zhengwen, Zhao, Xinkui

Abstract

Reconstructing Computer-Aided Design (CAD) modeling sequences from images is crucial for preserving design intent and supporting parametric editing. However, existing methods typically generate full CAD sequences holistically, overlooking the iterative, feedback-driven nature of human design workflows. We address this limitation by introducing the rich stepwise visual supervision: at each modeling step, the system observes the target's orthographic projections, the projections of the incrementally constructed model, and the active sketch, enabling informed action selection. To effectively leverage this on-the-fly feedback, we propose SOV-CAD, a framework that formulates CAD reconstruction as a sequential decision-making task and employs offline reinforcement learning with a Decision Transformer architecture. This design incorporates continuous visual feedback guided by geometric alignment rewards, resulting in a more accurate and human-like modeling process. Extensive experiments show that SOV-CAD surpasses state-of-the-art methods in CAD sequence reconstruction while exhibiting strong data efficiency. Code of SOV-CAD is available at: https://github.com/LukePhong/SOV-CAD

Chinese Translation

从图像中重建计算机辅助设计（CAD）建模序列对于保留设计意图和支持参数化编辑至关重要。然而，现有方法通常以整体方式生成完整的CAD序列，忽视了人类设计工作流程的迭代和反馈驱动特性。我们通过引入丰富的逐步视觉监督来解决这一局限性：在每个建模步骤中，系统观察目标的正投影、逐步构建模型的投影以及活动草图，从而实现知情的行动选择。为了有效利用这种即时反馈，我们提出了SOV-CAD，一个将CAD重建形式化为序列决策任务的框架，并采用基于决策变换器（Decision Transformer）架构的离线强化学习。该设计结合了由几何对齐奖励引导的连续视觉反馈，从而实现更准确和更人性化的建模过程。大量实验表明，SOV-CAD在CAD序列重建方面超越了最先进的方法，同时展现出强大的数据效率。SOV-CAD的代码可在以下链接获取：https://github.com/LukePhong/SOV-CAD

View on arXiv Download PDF AI Translation

cs.CV / 195 / 2607.04125

FRFDet: Efficient UAV Small Object Detection with Symmetric Sampling and Scalable Fusion

FRFDet：基于对称采样和可扩展融合的高效无人机小物体检测

Si, Yunzhong, Xu, Huiying, Zhu, Xinzhong, Liu, Yang, Dong, Yao, Zhang, Wenhao, Li, Hongbo

Abstract

Small object detection in Unmanned Aerial Vehicle (UAV) imagery remains challenging under adverse conditions, including complex weather, low illumination, and sensor noise. These challenges mainly stem from severe background clutter, fine-grained detail degradation, and suboptimal semantic-spatial feature fusion, which jointly hinder robust small-object representation. To this end, we propose FRFDet, a lightweight yet effective single-stage detector tailored for UAV-based small object detection. FRFDet proposes two plug-and-play modules: Inverse Bidirectional Sampling (IBS) and Scale-Feature Relationship Cross-Fusion (SFRCF). IBS preserves critical spatial details via channel expansion-compression and bidirectional pattern reconstruction, improving feature alignment. SFRCF explicitly models scale-dependent fusion behaviors, revealing that inter-group element-wise multiplication favors compact models, while inter-group additive fusion benefits larger architectures. Extensive experiments on VisDrone, UAVDT, HazyDet, and MS COCO demonstrate that FRFDet achieves state-of-the-art performance among lightweight detectors with low computational cost, compact parameters, and fast inference, making it well suited for resource-constrained UAV platforms.

Chinese Translation

在不利条件下（包括复杂天气、低光照和传感器噪声），无人机（UAV）图像中的小物体检测仍然面临挑战。这些挑战主要源于严重的背景杂波、细粒度细节退化以及次优的语义-空间特征融合，这些因素共同阻碍了小物体的稳健表示。为此，我们提出了FRFDet，一种轻量级但有效的单阶段检测器，专为基于无人机的小物体检测而设计。FRFDet提出了两个即插即用的模块：逆双向采样（Inverse Bidirectional Sampling, IBS）和尺度特征关系交叉融合（Scale-Feature Relationship Cross-Fusion, SFRCF）。IBS通过通道扩展-压缩和双向模式重建保留关键的空间细节，从而改善特征对齐。SFRCF明确建模尺度依赖的融合行为，揭示了组间元素逐项相乘有利于紧凑模型，而组间加法融合则有利于更大架构。在VisDrone、UAVDT、HazyDet和MS COCO上的大量实验表明，FRFDet在轻量级检测器中以低计算成本、紧凑参数和快速推理实现了最先进的性能，使其非常适合资源受限的无人机平台。

View on arXiv Download PDF AI Translation

cs.CV / 196 / 2607.04127

Real-Time LiDAR Gaussian Splatting SLAM

实时激光雷达高斯喷溅SLAM

Tak, Seungjun, Jeon, Yewon, Hwang, Jaeik, Hwang, SukMin, Ha, Seongbo, Yu, Hyeonwoo

Abstract

We present a real-time LiDAR-based framework for Gaussian Splatting SLAM that tightly couples fast G-ICP registration with spherical rasterization-based dense mapping for large-scale sequences. Leveraging LiDAR geometry rather than appearance, we reuse tracking-estimated local covariances to initialize Gaussians with range-aware scales and to derive surface normals for geometry-aware map optimization. We further introduce a covariance-derived geometry score that measures local complexity and drives pruning in planar regions and selective densification in structurally rich areas, while optimized Gaussians and LiDAR-specific confidence cues are fed back to improve tracking robustness. On the Newer College dataset, our method achieves an F-score of 86.78\% using purely online trajectories at real-time speed ($>$20 FPS), and additional experiments on other datasets confirm its stability and scalability.

Chinese Translation

我们提出了一种基于激光雷达的实时框架，用于高斯喷溅SLAM，该框架将快速G-ICP配准与基于球面光栅化的密集映射紧密结合，以处理大规模序列。利用激光雷达几何信息而非外观，我们重用跟踪估计的局部协方差，以初始化具有范围感知尺度的高斯分布，并推导几何感知地图优化所需的表面法线。我们进一步引入了一种基于协方差的几何评分，用于衡量局部复杂性，并在平面区域驱动修剪，在结构丰富区域进行选择性密化，同时优化后的高斯分布和激光雷达特定的置信度提示被反馈以提高跟踪的鲁棒性。在Newer College数据集上，我们的方法在实时速度（>20 FPS）下，使用纯在线轨迹达到了86.78\%的F-score，其他数据集的额外实验确认了其稳定性和可扩展性。

View on arXiv Download PDF AI Translation

cs.CV / 197 / 2607.04147

HCSU: A Dataset and Benchmark for Fine-Grained Historical Calligraphy Style Understanding

HCSU：一个用于细粒度历史书法风格理解的数据集和基准

Yao, Yinsheng, Liu, Yan, Ye, Chen

Abstract

Automated fine-grained perception of calligraphy styles--a task vital to cultural heritage preservation--remains a critical challenge for Large Vision-Language Models (LVLMs), largely constrained by existing datasets that suffer from modal mixture and flattened labels. To bridge this gap, we introduce HCSU, the first comprehensive dataset tailored for fine-grained Historical Calligraphy Style Understanding. HCSU comprises 39,307 meticulously curated character images from 49 historically prominent calligraphers across 10 dynasties, systematically decoupling authentic ink manuscripts (Tie) from stone rubbings (Bei) to resolve the long-standing modal mixture problem. Moving beyond conventional flattened labels, HCSU provides hierarchical expert-written aesthetic descriptions, enabling two rigorous evaluation protocols: fine-grained style discrimination and interpretable aesthetic reasoning. Extensive evaluations reveal a persistent gap between calligraphy-related knowledge and visually grounded style perception: state-of-the-art LVLMs show non-trivial performance but remain sensitive to script-level, textual, and source-specific cues, and often struggle to ground aesthetic judgments in fine-grained brushwork evidence. Ultimately, the HCSU benchmark exposes fundamental limitations in current multimodal architectures, aiming to inspire the evolution of expert-level visual reasoning for cultural heritage preservation. The dataset is available at https://huggingface.co/datasets/Tongji209/HCSU.

Chinese Translation

自动化细粒度书法风格感知——这一对文化遗产保护至关重要的任务——仍然是大型视觉语言模型（LVLMs）面临的一个关键挑战，主要受限于现有数据集，这些数据集存在模态混合和标签扁平化的问题。为了解决这一问题，我们推出了HCSU，这是第一个专门为细粒度历史书法风格理解量身定制的综合数据集。HCSU包含来自10个朝代的49位历史著名书法家的39,307幅精心策划的字符图像，系统性地将真实的墨迹手稿（Tie）与石刻拓片（Bei）分离，以解决长期存在的模态混合问题。HCSU超越了传统的扁平化标签，提供了分层的专家撰写的美学描述，使得两种严格的评估协议成为可能：细粒度风格辨别和可解释的美学推理。广泛的评估揭示了书法相关知识与视觉基础风格感知之间的持续差距：最先进的LVLMs表现出非平凡的性能，但仍然对书写级别、文本和特定来源的线索敏感，且常常难以将美学判断与细粒度的笔触证据相结合。最终，HCSU基准揭示了当前多模态架构的基本局限性，旨在激发专家级视觉推理在文化遗产保护中的发展。数据集可在 https://huggingface.co/datasets/Tongji209/HCSU 获取。

View on arXiv Download PDF AI Translation

cs.CV / 198 / 2607.04149

Beyond Scene Priors: Fine-Grained Traffic Scene Reasoning with Benchmarking and Query-Guided Small-Object Focus

超越场景先验：基于基准测试和查询引导的小物体聚焦的细粒度交通场景推理

Xiu, Waikit, Lu, Qiang, Wang, Zian, Yang, Xinjie, Chen, Zhiwei, Sun, Chen, Li, Xiying

Abstract

In safety-critical traffic scenarios, answering complex questions relies on minute, localized visual cues. However, standard Multimodal Large Language Models (MLLMs) tend to over-attend to backgrounds, overwhelming crucial small objects during visual-language alignment, a failure mode we term 'critical evidence dilution.' Furthermore, existing visual question answering (VQA) datasets rarely expose this flaw, as they lack large-scale, distractor-heavy evaluations that require pinpointing local evidence. To bridge this evaluation and architecture gap, we introduce the Fine-Grained Traffic Reasoning Benchmark (FGTR-Bench) and the Text-Guided Small-Object Reasoning MLLM (TSR-MLLM). FGTR-Bench comprises 40,236 single-image Multiple-Choice Questions (MCQs) created via multi-agent generation, consistency checks, and expert audits, alongside a disjoint 4,947-sample blind test split. To resolve evidence dilution, TSR-MLLM, built on Qwen3-VL-4B, uses a query-conditioned Text-Guided Small-Object Focus (TG-SOF) map. Applied once at the decoder boundary, the map adds sparse Top-K gated residuals to the most question-relevant vision slots while leaving text tokens unchanged. Together with lightweight decoder adaptation, TSR-MLLM preserves single-pass inference without external detectors or image re-encoding. Under matched settings, TSR-MLLM outperforms the strongest 4B baseline by 2.1 points on FGTR-Bench (74.1% overall), with larger gains on evidence-local tracks. Furthermore, it remains competitive on DriveQA-V (CARLA Signs) under greedy decoding without task-specific fine-tuning.

Chinese Translation

在安全关键的交通场景中，回答复杂问题依赖于微小的局部视觉线索。然而，标准的多模态大型语言模型（MLLMs）往往过度关注背景，在视觉-语言对齐过程中淹没了关键的小物体，这种失败模式我们称之为“关键证据稀释”。此外，现有的视觉问答（VQA）数据集很少暴露这一缺陷，因为它们缺乏大规模、干扰物重的评估，这些评估需要精准定位局部证据。为了弥补这一评估与架构之间的差距，我们引入了细粒度交通推理基准（FGTR-Bench）和文本引导的小物体推理多模态大型语言模型（TSR-MLLM）。FGTR-Bench包含40,236个通过多代理生成、一致性检查和专家审核创建的单图像多项选择题（MCQs），以及一个不重叠的4,947样本盲测分割。为了解决证据稀释问题，TSR-MLLM基于Qwen3-VL-4B构建，使用查询条件的文本引导小物体聚焦（TG-SOF）图。该图在解码器边界应用一次，为最相关问题的视觉槽添加稀疏的Top-K门控残差，同时保持文本标记不变。结合轻量级解码器适配，TSR-MLLM在不使用外部检测器或图像重新编码的情况下保留了单次推理。在匹配设置下，TSR-MLLM在FGTR-Bench上比最强的4B基线提高了2.1个百分点（总体74.1%），在证据局部轨道上获得了更大的提升。此外，在贪婪解码下，TSR-MLLM在DriveQA-V（CARLA Signs）上仍保持竞争力，而无需特定任务的微调。

View on arXiv Download PDF AI Translation

cs.CV / 199 / 2607.04151

Perceiving Better Moments: Cover Frame Reselection and Enhancement for Live Photos with the Live2K Dataset

更好时刻的感知：基于 Live2K 数据集的动态照片封面帧重新选择与增强

Lou, Junyu, Chen, Kai, You, Weiyi, Zeng, Hui, Zhang, Lei, Gu, Shuhang

Abstract

Modern smartphones capture Live Photos, short video bursts surrounding a still image, offering a dynamic and engaging photographic experience. However, the cover photo and video components are generated by two distinct imaging pipelines: the photo stream undergoes full computational photography processing, while the video stream is constrained by real-time efficiency and heavy compression. This intrinsic separation produces a substantial quality gap in resolution, color fidelity, and dynamic range between the cover photo and video frames. When users reselect an alternative frame from the video to replace an imperfect cover, the chosen frame often suffers from severe degradation, making direct replacement visually unsatisfactory. Restoring such frames requires simultaneous enhancement of spatial detail and color appearance, a task considerably more challenging than ordinary super-resolution or color enhancement. To address this, we define the Live Photo Cover Frame Reselection and Enhancement (LPRE) task, which leverages the intrinsic cues available within each Live Photo: the high-quality cover image as a structural and color reference, the user-reselected low-quality frame as the reconstruction target and several adjacent video frames providing temporal cues. Building upon this formulation, we construct Live2K, a real-world dataset of 2,042 Live Photos, and develop a unified one-stage baseline that integrates multi-frame fusion, guided color enhancement and super-resolution, establishing the first benchmark for Live Photo enhancement research.

Chinese Translation

现代智能手机捕捉动态照片，即围绕静态图像的短视频序列，为用户提供了一种动态且引人入胜的摄影体验。然而，封面照片和视频组件是通过两条不同的成像管道生成的：照片流经过全面的计算摄影处理，而视频流则受到实时效率和重度压缩的限制。这种内在的分离导致封面照片与视频帧之间在分辨率、色彩保真度和动态范围上存在显著的质量差距。当用户从视频中重新选择一个替代帧以替换不完美的封面时，所选帧往往会遭受严重的降质，使得直接替换在视觉上不令人满意。恢复这些帧需要同时增强空间细节和色彩表现，这一任务比普通的超分辨率或色彩增强要复杂得多。为了解决这个问题，我们定义了动态照片封面帧重新选择与增强（Live Photo Cover Frame Reselection and Enhancement, LPRE）任务，利用每个动态照片中可用的内在线索：高质量的封面图像作为结构和色彩参考，用户重新选择的低质量帧作为重建目标，以及提供时间线索的若干相邻视频帧。在此基础上，我们构建了 Live2K，一个包含 2042 张动态照片的真实世界数据集，并开发了一个统一的一阶段基线，整合了多帧融合、引导色彩增强和超分辨率，为动态照片增强研究建立了首个基准。

View on arXiv Download PDF AI Translation

cs.CV / 200 / 2607.04163

SeeMe: Mitigating Hallucinations in Large Vision-Language Models through Effective Visual Token Engineering

SeeMe：通过有效的视觉标记工程减轻大规模视觉-语言模型中的幻觉

Tang, Kai, You, Jinhao, Zhang, Bohua, Guo, Yichen, Sun, Yiding, Zhang, Dongxu, Li, Chenxi, Huang, Xiande, Zhang, Shanghang

Abstract

Large Vision-Language Models (LVLMs) have achieved remarkable progress in visual understanding tasks such as image captioning and visual question answering. However, they remain susceptible to hallucinations, generating content that is inconsistent with the actual visual input. Existing methods primarily intervene at the decoding stage, while overlooking a critical source of hallucinations: irrelevant or noisy visual tokens that mislead the decoding process. To address this issue, we propose SeeMe, a training-free framework that introduces the concept of feature engineering from traditional machine learning into LVLMs. SeeMe restructures visual tokens through a three-stage token engineering process to suppress hallucination sources while preserving informative visual evidence. Experiments on MME, POPE, and AMBER benchmarks across four LVLMs demonstrate that SeeMe consistently reduces hallucinations and improves output consistency, providing a novel perspective for mitigating hallucinations in LVLMs.

Chinese Translation

大规模视觉-语言模型（LVLMs）在图像描述和视觉问答等视觉理解任务中取得了显著进展。然而，它们仍然容易出现幻觉，生成与实际视觉输入不一致的内容。现有方法主要在解码阶段进行干预，而忽视了幻觉的一个关键来源：误导解码过程的无关或噪声视觉标记。为了解决这个问题，我们提出了SeeMe，一个无训练框架，将传统机器学习中的特征工程概念引入LVLMs。SeeMe通过三阶段的标记工程过程重构视觉标记，以抑制幻觉源，同时保留有用的视觉证据。在四个LVLMs上针对MME、POPE和AMBER基准的实验表明，SeeMe始终能够减少幻觉并提高输出一致性，为减轻LVLMs中的幻觉提供了一种新视角。

View on arXiv Download PDF AI Translation

cs.CV / 201 / 2607.04179

CritiqueDriveVLM: From Verifier-Guided Reinforcement Learning to Latent Thought Distillation for Autonomous Driving

CritiqueDriveVLM：从验证者引导的强化学习到自主驾驶的潜在思维蒸馏

Liu, Zhaohong, Ye, Hao, Zhang, Xianlin, Qi, Mengshi

Abstract

End-to-end Vision-Language Models (VLMs) show immense potential in autonomous driving. However, standard Supervised Fine-Tuning (SFT) often suffers from reasoning hallucinations and conservative biases. While traditional tool-augmented frameworks and Chain-of-Thought (CoT) approaches mitigate these issues, they incur exorbitant token consumption and unacceptable latency, rendering real-time deployment impractical. To resolve this reliability-efficiency trade-off, we propose CritiqueDriveVLM, a novel unified three-stage framework internalizing reasoning directly into the VLM. First, we introduce Critique-Driven Multi-Turn Reinforcement Learning (RL) guided by a multi-dimensional verifier. By providing granular scalar feedback and a multi-turn penalty, we force the policy to internalize logical deduction, cultivating a robust System-2 Teacher that achieves high accuracy without fragile external tools. Subsequently, we propose Latent Thought Distillation to overcome the latency bottleneck. By aligning the Student's latent representations with the Teacher's fully converged reasoning states, we compress deep logical capabilities into a fast, CoT-free System-1 Student. Extensive experiments on the widely-used DriveLMM-01 benchmark demonstrate remarkable improvements. Compared to the base model, our tool-free Teacher significantly boosts Multiple Choice Quality (MCQ) from 55.54% to a state-of-the-art 76.54%. Crucially, our distilled Student preserves competitive reasoning depth while drastically minimizing generation length to an average of merely 28 tokens. This slashes inference latency by 88% (from 3482 ms to 416 ms), paving a highly robust pathway for low-latency autonomous driving.Our source code is available at https://github.com/MICLAB-BUPT/CritiqueDriveVLM.

Chinese Translation

端到端视觉-语言模型（VLM）在自主驾驶中展现出巨大的潜力。然而，标准的监督微调（SFT）往往受到推理幻觉和保守偏见的困扰。虽然传统的工具增强框架和链式思维（CoT）方法在一定程度上缓解了这些问题，但它们消耗的令牌数量过多且延迟不可接受，使得实时部署变得不切实际。为了解决这种可靠性与效率的权衡，我们提出了CritiqueDriveVLM，一种新颖的统一三阶段框架，将推理直接内化到VLM中。首先，我们引入了由多维验证者引导的批评驱动多轮强化学习（RL）。通过提供细粒度的标量反馈和多轮惩罚，我们迫使策略内化逻辑推理，培养出一个强大的系统-2教师，该教师在没有脆弱外部工具的情况下实现高准确率。随后，我们提出了潜在思维蒸馏，以克服延迟瓶颈。通过将学生的潜在表示与教师的完全收敛推理状态对齐，我们将深层逻辑能力压缩成一个快速的、无CoT的系统-1学生。在广泛使用的DriveLMM-01基准上的大量实验表明了显著的改进。与基础模型相比，我们无工具的教师将多项选择质量（MCQ）从55.54%显著提升至领先的76.54%。关键是，我们蒸馏的学生在保持竞争性推理深度的同时，将生成长度大幅缩短至平均仅28个令牌。这使得推理延迟减少了88%（从3482毫秒降至416毫秒），为低延迟自主驾驶铺平了极为稳健的道路。我们的源代码可在 https://github.com/MICLAB-BUPT/CritiqueDriveVLM 获取。

View on arXiv Download PDF AI Translation

cs.CV / 202 / 2607.04199

Topology-Driven Transferability Estimation for 3D Medical Vision Foundation Models

基于拓扑的3D医学视觉基础模型可迁移性估计

Tang, Jiaqi, Zhang, Shaoyang, Zhang, Fandong, Zhang, Shu, Liu, Yang, Chen, Qingchao

Abstract

The growing number of medical vision foundation models highlights the need for effective model selection. However, mainstream selection methods rely on exhaustive fine-tuning, which is computationally expensive. Most of the existing Transferability Estimation (TE) metrics are primarily designed for image-level classification. They fail to preserve spatial relationships and fine-grained boundary details, which are crucial for the segmentation task. Additionally, while image-level tasks typically process a single feature vector per input, dense prediction tasks in 3D medical imaging require voxel-wise evaluation against dense annotations. To bridge these gaps, we propose a \textit{non-parametric, topology-driven} framework that estimates transferability directly from the alignment between the sparse 1-skeleton graph of dense features and semantic labels via Minimum Spanning Trees (MST). We decouple the alignment into two complementary geometric scales: Local Boundary-Aware Topological Consistency (LBTC) to assess boundary separability, where we prove that the MST leakage rate serves as a finite-sample lower bound on the Bayes error; and Global Representation Topology Divergence (GRTD) to evaluate the overall anatomical layout. Crucially, we formally justify a counterintuitive mechanism: Although without fine-tuning, the randomly initialized segmentation decoder acts as a topology-preserving spatial projector, reducing the variance of pairwise distance estimates and stabilizing global alignment evaluation. Fused via a task-adaptive gating mechanism, these dual metrics adapt to diverse clinical complexities. Evaluated on a large-scale benchmark of 114,000 3D medical volumes across diverse anatomical tasks, our topological framework achieves state-of-the-art transferability estimation with an average weighted Kendall (outperforming by 0.36) while accelerating evaluation by 56 times.

Chinese Translation

随着医学视觉基础模型数量的不断增加，模型选择的有效性变得愈发重要。然而，主流选择方法依赖于全面的微调，这在计算上代价高昂。现有的大多数可迁移性估计（Transferability Estimation, TE）指标主要针对图像级分类设计，未能保留空间关系和细粒度边界细节，而这些对于分割任务至关重要。此外，图像级任务通常对每个输入处理单一特征向量，而3D医学成像中的密集预测任务则需要针对密集注释进行体素级评估。为了解决这些问题，我们提出了一种 extit{非参数、基于拓扑}的框架，该框架通过最小生成树（Minimum Spanning Trees, MST）直接从稀疏1-骨架图的密集特征与语义标签之间的对齐中估计可迁移性。我们将对齐解耦为两个互补的几何尺度：局部边界感知拓扑一致性（Local Boundary-Aware Topological Consistency, LBTC）用于评估边界可分离性，我们证明了MST泄漏率作为贝叶斯误差的有限样本下界；以及全局表示拓扑差异（Global Representation Topology Divergence, GRTD）用于评估整体解剖布局。重要的是，我们正式证明了一种反直觉机制：尽管没有微调，随机初始化的分割解码器充当了一个保持拓扑的空间投影器，从而减少了成对距离估计的方差并稳定了全局对齐评估。通过任务自适应门控机制融合，这两个指标能够适应多样的临床复杂性。在对114,000个3D医学体积进行的大规模基准评估中，我们的拓扑框架实现了最先进的可迁移性估计，平均加权Kendall指标（超越0.36），同时将评估速度提高了56倍。

View on arXiv Download PDF AI Translation

cs.CV / 203 / 2607.04241

Hierarchical Multi-to-Single-Modal Knowledge Distillation for Disruption Prediction in EAST

用于EAST中扰动预测的层次化多模态到单模态知识蒸馏

Chen, Qiang, Wang, Xiao, Si, Hao, Yang, Qingquan, Chen, Meiwen, Yang, Jianhua, Han, Xiaofeng, Jia, Yunhu, Chen, Ran, Wang, Liang, Tang, Jin, Xu, Guosheng

Abstract

Plasma disruption is a critical threat to tokamak safety. Existing data-driven predictors mainly rely on time-series diagnostic signals, while visible images provide complementary spatial cues including plasma deformation, local brightening, and radiation-structure evolution. Although the image modality improves the model's discriminative capability, it also substantially increases the computational cost during inference. To address this issue, we propose a hierarchical multi-to-single-modal knowledge distillation framework for disruption prediction on a synchronized EAST multimodal dataset. During training, visible images and time-series signals are used to train a multimodal teacher, which learns disruption precursor representations through Transformer-based encoders and a prototype-guided spatiotemporal hypergraph module. During inference, only the time-series student is retained, with multimodal knowledge transferred through graph-structure-level, representation-level, and decision-level distillation. On the 640-discharge EAST dataset, the results demonstrate that the proposed framework can preserve the discriminative advantages of multimodal learning while substantially reducing inference cost, and providing an effective route for efficient disruption prediction in EAST. The source code of this paper will be released on https://github.com/Event-AHU/OpenFusion.

Chinese Translation

等离子体扰动是对托卡马克安全的重大威胁。现有的数据驱动预测器主要依赖于时间序列诊断信号，而可见图像提供了补充的空间线索，包括等离子体变形、局部亮化和辐射结构演变。尽管图像模态提高了模型的区分能力，但在推理过程中也显著增加了计算成本。为了解决这一问题，我们提出了一种层次化的多模态到单模态知识蒸馏框架，用于在同步的EAST多模态数据集上进行扰动预测。在训练过程中，使用可见图像和时间序列信号训练一个多模态教师，该教师通过基于Transformer的编码器和原型引导的时空超图模块学习扰动前兆表示。在推理过程中，仅保留时间序列学生，通过图结构级、表示级和决策级的蒸馏转移多模态知识。在640次放电的EAST数据集上，结果表明所提出的框架能够保留多模态学习的区分优势，同时显著降低推理成本，为EAST中高效的扰动预测提供了一条有效的途径。本文的源代码将发布在 https://github.com/Event-AHU/OpenFusion。

View on arXiv Download PDF AI Translation

cs.CV / 204 / 2607.04243

HeartVolMesh: Cardiac Volumetric Mesh Reconstruction via Covariance-Guided Graph Deformation

HeartVolMesh：通过协方差引导的图形变形进行心脏体积网格重建

Lin, Fengming, Zakeri, Arezoo, Dou, Haoran, Zhou, Zherui, Lan, Shaokun, Duan, Jinming, Frangi, Alejandro

Abstract

Accurate patient-specific tetrahedral cardiac meshes are essential for in-silico trials, yet common segmentation-then-modelling pipelines can blur thin-wall anatomy and offer limited cross-case correspondence. We propose HeartVolMesh, which lifts each template vertex to an anisotropic Gaussian kernel and uses a 3D CNN-GNN to predict per-vertex displacements and Cholesky-parameterized covariances from volumetric images. Training is guided by a covariance-aware negative log-likelihood loss with lightweight mesh regularization. For volumetric meshing, we warp a fixed tetrahedral template to the reconstructed surface via staged alignment, non-rigid registration, and deformation propagation, preserving connectivity and correspondence by construction, with resolution controlled by template density. Experiments show consistent gains over deformation-based baselines in surface mesh accuracy and volumetric mesh fidelity.

Chinese Translation

准确的患者特异性四面体心脏网格对于计算机模拟试验至关重要，但常见的分割-建模流程可能会模糊薄壁解剖结构，并提供有限的跨案例对应关系。我们提出了HeartVolMesh，该方法将每个模板顶点提升到各向异性高斯核，并使用3D CNN-GNN预测每个顶点的位移和从体积图像中提取的Cholesky参数化协方差。训练过程由一种考虑协方差的负对数似然损失函数引导，并结合轻量级网格正则化。对于体积网格化，我们通过分阶段对齐、非刚性配准和变形传播将固定的四面体模板扭曲到重建表面，从而在构造上保持连通性和对应关系，并通过模板密度控制分辨率。实验表明，在表面网格精度和体积网格保真度方面，相较于基于变形的基线方法，取得了一致的提升。

View on arXiv Download PDF AI Translation

cs.CV / 205 / 2607.04249

Beyond Random Sampling: Distribution-Aware Alignment for Semi-Supervised Medical Image Segmentation

超越随机采样：面向分布的半监督医学图像分割对齐

Yan, Weihao, Qian, Yeqiang, Dong, Yi, Yang, Ming

Abstract

Precise medical image segmentation is crucial for clinical diagnosis and treatment planning, yet relies heavily on expensive expert annotations. Semi-supervised medical image segmentation (SSMIS) offers a cost-effective solution but typically operates under the assumption of independent and identically distributed (i.i.d.) data, defaulting to random sampling. While statistically valid at scale, this strategy suffers from severe representation bias in low-data regimes, failing to capture the heterogeneous medical data manifold. To address this, we propose a highly data-efficient framework driven by distribution alignment. First, we introduce an offline Distribution-Aware Sample Selection strategy. By leveraging Vision Foundation Models (VFMs) and our designed Density-K-Center algorithm, we explicitly identify representative structural anchors, establishing a more representative labeled domain. Second, to bridge the remaining distribution gap, we propose the Memory-guided Copy-Paste (MCP) module. Tailored for the inherent class imbalance in medical scans, MCP leverages a semantic memory mechanism to retrieve historically consistent priors for cross-domain alignment, encouraging semantic consistency. Coupled with an easy-to-hard progressive schedule, this framework effectively mitigates early-stage pseudo-label noise. Extensive experiments on six diverse 2D and 3D datasets demonstrate strong segmentation performance, particularly in extremely low-labeled scenarios (\eg, 1/16 ratio).

Chinese Translation

精确的医学图像分割对临床诊断和治疗规划至关重要，但这在很大程度上依赖于昂贵的专家标注。半监督医学图像分割（SSMIS）提供了一种具有成本效益的解决方案，但通常假设数据是独立同分布（i.i.d.），默认采用随机采样。尽管在大规模统计上有效，但该策略在低数据环境中存在严重的表示偏差，未能捕捉异质的医学数据流形。为了解决这个问题，我们提出了一种高效的数据驱动框架，基于分布对齐。首先，我们引入了一种离线的面向分布的样本选择策略。通过利用视觉基础模型（Vision Foundation Models, VFMs）和我们设计的密度-K-中心（Density-K-Center）算法，我们明确识别出具有代表性的结构锚点，从而建立一个更具代表性的标记域。其次，为了弥补剩余的分布差距，我们提出了记忆引导的复制粘贴（Memory-guided Copy-Paste, MCP）模块。该模块针对医学扫描中固有的类别不平衡，利用语义记忆机制检索历史一致的先验知识以实现跨域对齐，促进语义一致性。结合易到难的渐进调度，该框架有效减轻了早期伪标签噪声。在六个不同的2D和3D数据集上的广泛实验表明，该方法在极低标记场景下（例如，1/16比例）表现出强大的分割性能。

View on arXiv Download PDF AI Translation

cs.CV / 206 / 2607.04256

AdaptiveSplat:Texture Aware Controllable 3D Gaussian Allocation for Feed-Forward Reconstruction

自适应喷溅：基于纹理感知的可控3D高斯分配用于前馈重建

Singhal, Badrinath, G, Srihari K, Iyer, Sreehari, Dhiman, Ankit, Radhakrishnan, Venkatesh Babu

Abstract

Current feed-forward 3D reconstruction methods predict pixel aligned Gaussian primitives, resulting in highly redundant representations. A natural solution is to prune the redundant Gaussians, but naive pruning introduces severe artifacts and often requires inference time fine-tuning, breaking the feed-forward paradigm. Based on previous works, high frequency regions require more Gaussian primitives, while low frequency regions can be represented with significantly fewer primitives. Motivated by this, we propose a novel approach to explicitly control the number of Gaussians by leveraging local texture information. Our approach achieves this through three key components: (1) texture estimation to capture spatial variation in scene detail, (2) texture-aware pruning that removes redundant Gaussians from low frequency regions, and (3) an adaptive Gaussian head that predicts the modified attributes of the retained primitives without breaking the feed-forward paradigm. Experiments on RE10K, ACID, DL3DV, Tanks and Temples, and DTU demonstrate the effectiveness of our approach, while ablation studies validate the contributions of its key components.

Chinese Translation

当前的前馈3D重建方法预测与像素对齐的高斯原语，导致高度冗余的表示。一个自然的解决方案是修剪冗余的高斯，但简单的修剪会引入严重的伪影，并且通常需要推理时间的微调，从而打破前馈范式。基于之前的研究，高频区域需要更多的高斯原语，而低频区域则可以用显著更少的原语表示。受到此启发，我们提出了一种新颖的方法，通过利用局部纹理信息来显式控制高斯的数量。我们的方法通过三个关键组件实现这一目标：（1）纹理估计以捕捉场景细节的空间变化，（2）纹理感知修剪，从低频区域移除冗余高斯，以及（3）自适应高斯头，预测保留原语的修改属性而不打破前馈范式。在RE10K、ACID、DL3DV、Tanks and Temples和DTU上的实验验证了我们方法的有效性，而消融研究则验证了其关键组件的贡献。

View on arXiv Download PDF AI Translation

cs.CV / 207 / 2607.04276

EMPURPLE: A Free Lunch for Diffusion Distillation based on the Information Bottleneck

EMPURPLE：基于信息瓶颈的扩散蒸馏的免费午餐

Li, Zilai, Bai, Lujia

Abstract

Diffusion models achieve impressive image-generation quality but remain expensive at inference time. Diffusion distillation reduces sampling steps, yet many distilled models, including SDXL-Lightning and distribution matching distillation methods, suffer from degraded Fr\'echet Inception Distance (FID). We analyze this phenomenon through a PAC-style generalization bound. Our analysis suggests that aggressive early-step redirection of the velocity field makes the distillation target harder to learn, enlarging the train-test gap. As a result, early-step output distributions differ between training and inference, causing distribution mismatch in the intermediate noisy latent used as next-step inputs. We empirically validate this mechanism by showing reduced diversity in both intermediate features and final outputs. To address this issue, we propose EMPURPLE, a simple training-free method that recycles intermediate latents sampled from the original model. EMPURPLE is model-agnostic and improves FID by 7\% to 20\% across DMD2, Hyper-SD, FlashSD, and SDXL-Lightning. The repo is: https://github.com/TheLovesOfLadyPurple/Empurple-Training-Free-Algorithm-To-enhance-Diversity-of-The-Diffusion-Distillation-Model

Chinese Translation

扩散模型在图像生成质量上取得了令人印象深刻的成果，但在推理时仍然成本高昂。扩散蒸馏通过减少采样步骤来降低成本，然而许多蒸馏模型，包括SDXL-Lightning和分布匹配蒸馏方法，面临着Fréchet Inception Distance (FID) 降低的问题。我们通过PAC风格的泛化界限分析这一现象。我们的分析表明，激进的早期步骤速度场重定向使得蒸馏目标更难学习，从而扩大了训练与测试之间的差距。因此，早期步骤的输出分布在训练和推理之间存在差异，导致用于下一步输入的中间噪声潜变量之间的分布不匹配。我们通过实证验证这一机制，显示中间特征和最终输出的多样性降低。为了解决这个问题，我们提出了EMPURPLE，这是一种简单的无训练方法，回收从原始模型中采样的中间潜变量。EMPURPLE是模型无关的，并且在DMD2、Hyper-SD、FlashSD和SDXL-Lightning上提高了FID 7%到20%。相关代码库链接为：https://github.com/TheLovesOfLadyPurple/Empurple-Training-Free-Algorithm-To-enhance-Diversity-of-The-Diffusion-Distillation-Model

View on arXiv Download PDF AI Translation

cs.CV / 208 / 2607.04303

AquaStereo: Enabling Underwater Stereo Matching via Depth-Conditioned Diffusion and Geometry Self-Distillation

AquaStereo：通过深度条件扩散和几何自蒸馏实现水下立体匹配

Wei, Qizhe, Liang, Yingping, You, Shaodi, Fu, Ying

Abstract

Learning-based stereo matching models struggle in underwater environments due to scarce in-domain data and the difficulty of extracting discriminative correspondences from degraded imagery. In this work, we present $\textbf{AquaStereo}$, a perception-enhanced framework with a data simulation pipeline and a self-distillation strategy that jointly address data scarcity and feature degradation in underwater stereo matching. First, a depth-conditioned diffusion pipeline renders underwater stereo pairs while preserving binocular geometry, with a lightweight left-right consistency module ensuring geometric alignment. Training on this synthetic corpus effectively narrows the terrestrial-underwater gap and improves zero-shot robustness. Second, a frozen binocular teacher trained on clean terrestrial pairs guides a student exposed to rendered underwater pairs with perturbations. A stage-weighted sequence loss is performed to align the student's disparities with the teacher's geometry, while a clean-branch supervision with shared pseudo targets prevents scale drift. To further enhance feature stability under turbidity and low texture, we introduce learnable perception frames, a perception-enhanced feature formulation that constructs robust matching descriptors by fusing temporal cues from two auxiliary views encoded by a video backbone with semantic features extracted by a strong image encoder. Extensive experiments demonstrate that $\textbf{AquaStereo}$ substantially improves robustness and zero-shot generalization in challenging underwater scenarios. The code is available at https://github.com/qz-wei/AquaStereo.

Chinese Translation

基于学习的立体匹配模型在水下环境中面临挑战，主要由于领域内数据稀缺以及从退化图像中提取可区分对应关系的困难。在本研究中，我们提出了$ extbf{AquaStereo}$，这是一个增强感知的框架，结合了数据仿真管道和自蒸馏策略，旨在共同解决水下立体匹配中的数据稀缺和特征退化问题。首先，深度条件扩散管道在保持双目几何的同时生成水下立体对，并通过轻量级的左右一致性模块确保几何对齐。在这个合成数据集上进行训练有效缩小了陆地与水下之间的差距，并提高了零样本的鲁棒性。其次，经过清晰陆地对训练的冻结双目教师指导暴露于带扰动的生成水下对的学生。通过阶段加权序列损失对齐学生的视差与教师的几何，同时通过共享伪目标的干净分支监督防止尺度漂移。为了进一步增强在浑浊和低纹理条件下的特征稳定性，我们引入了可学习的感知框架，这是一种增强感知的特征构造方法，通过融合由视频骨干网络编码的两个辅助视图的时间线索与由强图像编码器提取的语义特征，构建出鲁棒的匹配描述符。大量实验表明，$ extbf{AquaStereo}$在具有挑战性的水下场景中显著提高了鲁棒性和零样本泛化能力。代码可在 https://github.com/qz-wei/AquaStereo 获取。

View on arXiv Download PDF AI Translation

cs.CV / 209 / 2607.04304

Road-Aware Anomaly Segmentation with Query-Guided Polygons and CLIP in Autonomous Driving

基于查询引导多边形和CLIP的道路感知异常分割在自动驾驶中的应用

Yan, Zhiran, Elger, Gordon

Abstract

Traditional semantic segmentation models operate under a closed-set assumption and struggle to recognize unknown or unexpected objects-an essential capability for autonomous driving. As a result, such models often misclassify or overlook out-of-distribution (OOD) road anomalies, posing safety risks in open-world environments. We present a lightweight, postprocessing, road-aware anomaly segmentation framework that requires no retraining, no OOD data, and no auxiliary supervision. Our approach builds on a mask transformer-based segmentation network by exploiting query-level mask confidence and deriving a polygonal road prior to detect gap regions that may correspond to anomalies. To further suppress false positives, we introduce a CLIP-based zero-shot semantic filtering module using in-distribution prompts, with optional generalized OOD prompts. By jointly leveraging spatial priors and semantic verification, our framework produces robust and interpretable anomaly predictions. Evaluation on three public benchmarks-Fishyscapes, SMIYC, and RoadAnomaly-shows consistently strong performance. In particular, our method outperforms the training-free baseline Maskomaly on most metrics and achieves the highest AP on Fishyscapes LostAndFound. These results demonstrate the practicality and deployability of our approach for real-world autonomous driving systems.

Chinese Translation

传统的语义分割模型在封闭集假设下运行，难以识别未知或意外的物体，这对于自动驾驶至关重要。因此，这类模型常常错误分类或忽视分布外（OOD）的道路异常，这在开放世界环境中带来了安全风险。我们提出了一种轻量级的后处理道路感知异常分割框架，该框架无需重新训练、无需OOD数据，也无需辅助监督。我们的方法基于掩膜变换器（mask transformer）构建的分割网络，通过利用查询级掩膜置信度并推导多边形道路先验，检测可能对应于异常的间隙区域。为了进一步抑制假阳性，我们引入了一种基于CLIP的零样本语义过滤模块，使用分布内提示，并可选地使用广义的OOD提示。通过共同利用空间先验和语义验证，我们的框架产生了稳健且可解释的异常预测。在三个公共基准数据集（Fishyscapes、SMIYC和RoadAnomaly）上的评估显示出一致的强劲表现。特别是，我们的方法在大多数指标上优于无训练基线Maskomaly，并在Fishyscapes LostAndFound上达到了最高的AP。这些结果展示了我们的方法在现实世界自动驾驶系统中的实用性和可部署性。

View on arXiv Download PDF AI Translation

cs.CV / 210 / 2607.04311

Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment

Aura：通过VLM驱动的语义对齐实现一致的多主体视频生成

Zhou, Zixiang, Yu, Zhentao, Ma, Yifeng, Wang, Hongmei, Yu, Wenqing, Wang, Cong, Yang, Zilin, Chen, Rui, Ou, Jiarong, Liu, Yezhou, Zhou, Yuan, Lu, Qinglin

Abstract

Subject-driven and multi-element video generation are central to controllable video synthesis, but existing methods still struggle to preserve identity consistency and model complex relationships among multiple subjects. In this paper, we propose Aura, a unified framework for high-fidelity and identity-consistent video generation. To better capture scene dynamics and subject interactions, we introduce AI director-level captions that provide dense and structured descriptions of video content. We further leverage a vision-language model (VLM) with learnable queries to extract multimodal semantic features from textual and visual references, covering both global semantics and fine-grained visual cues. To bridge the representational gap between the VLM and the Diffusion Transformer (DiT), we design a two-stage alignment strategy that progressively maps VLM features into the DiT feature space. For visual conditioning, we adopt token concatenation to inject reference information directly into the generation process. To distinguish heterogeneous subject types and reduce common copy-paste artifacts, we develop a subject-aware RoPE-Shift mechanism. To further differentiate reference images of different categories, we introduce subject-aware learnable tokens. In addition, we introduce Memory Tokens to balance the training signal across examples with different numbers of reference subjects. During inference, Progressive-APG (Adaptive Prompt Guidance) further alleviates oversaturation and improves semantic alignment with user prompts. Finally, we build a high-quality video-subject image dataset through a dedicated data construction pipeline. Extensive experiments show that our method achieves state-of-the-art performance on both single-subject generation and more challenging multi-element scenarios.

Chinese Translation

以主体为驱动的多元素视频生成是可控视频合成的核心，但现有方法仍然难以保持身份一致性并建模多个主体之间的复杂关系。本文提出了Aura，一个用于高保真和身份一致性视频生成的统一框架。为了更好地捕捉场景动态和主体交互，我们引入了AI导演级字幕，提供视频内容的密集和结构化描述。我们进一步利用具有可学习查询的视觉语言模型（VLM）从文本和视觉参考中提取多模态语义特征，涵盖全球语义和细粒度视觉线索。为了弥合VLM与扩散变换器（DiT）之间的表征差距，我们设计了一种两阶段对齐策略，逐步将VLM特征映射到DiT特征空间。对于视觉条件，我们采用令牌连接将参考信息直接注入生成过程。为了区分异构主体类型并减少常见的复制粘贴伪影，我们开发了一种主体感知的RoPE-Shift机制。为了进一步区分不同类别的参考图像，我们引入了主体感知的可学习令牌。此外，我们引入了记忆令牌，以平衡具有不同数量参考主体的示例之间的训练信号。在推理过程中，渐进式APG（自适应提示引导）进一步缓解了过饱和现象，并改善了与用户提示的语义对齐。最后，我们通过专门的数据构建管道构建了一个高质量的视频-主体图像数据集。大量实验表明，我们的方法在单主体生成和更具挑战性的多元素场景中均达到了最先进的性能。

View on arXiv Download PDF AI Translation

cs.CV / 211 / 2607.04330

Framework and Multi-modal Dataset for Roadwork Zone Detection and Geo-localization

道路施工区域检测与地理定位的框架与多模态数据集

Yan, Zhiran, Xin, Yutong, Shenoi, S Shyam, Song, Rui, Elger, Gordon

Abstract

Autonomous vehicles often rely on high-definition (HD) maps for navigation; however, these maps are not frequently updated and often lack semi-static information, such as temporary roadwork zones, which can significantly alter the road network. This limitation underscores the urgent need for an accurate global position of roadwork zones. However, the absence of publicly available datasets for evaluating roadwork zone detection and geo-localization models has hindered the development of reliable autonomous driving systems. To address this challenge, we propose the Roadwork Zone Detection and Geo-localization (RZDG) dataset, which includes both simulated and real-world data, providing multimodal sensor inputs along with comprehensive annotations. The dataset supports multiple perception tasks, including image semantic segmentation, 3D object detection, and object geo-localization. In addition, we introduce a tracker-based roadwork zone detection and geo-localization (RZDG) pipeline, an extension of AB3DMOT, for accurate object geo-localization in roadwork zones. We benchmark our approach on the RZDG dataset, demonstrating its effectiveness in detecting roadwork zones and transforming object positions from the local coordinate system to the global coordinate system. A prediction is considered a true positive (TP) if its estimated position falls within one meter of the ground truth. Our experimental results show that our approach achieves high accuracy on both real and simulated data. Specifically, we report: Precision: 0.565 (real) / 0.615 (simulated) Recall: 0.898 (real) / 0.809 (simulated) F1-score: 0.597 (real) / 0.665 (simulated).

Chinese Translation

自主车辆通常依赖高清（HD）地图进行导航；然而，这些地图更新频率不高，且往往缺乏半静态信息，例如临时施工区域，这可能显著改变道路网络。这一局限性凸显了对施工区域准确全球定位的迫切需求。然而，缺乏公开可用的数据集来评估施工区域检测和地理定位模型，阻碍了可靠自主驾驶系统的发展。为了解决这一挑战，我们提出了道路施工区域检测与地理定位（RZDG）数据集，该数据集包括模拟和真实世界的数据，提供多模态传感器输入以及全面的注释。该数据集支持多种感知任务，包括图像语义分割、3D物体检测和物体地理定位。此外，我们引入了一种基于跟踪器的施工区域检测与地理定位（RZDG）管道，这是AB3DMOT的扩展，用于在施工区域内实现准确的物体地理定位。我们在RZDG数据集上对我们的方法进行了基准测试，证明其在检测施工区域和将物体位置从局部坐标系转换到全球坐标系方面的有效性。如果预测的位置估计在真实值的一米范围内，则该预测被视为真正的正例（TP）。我们的实验结果表明，我们的方法在真实和模拟数据上都达到了高准确率。具体而言，我们报告：精确率：0.565（真实）/ 0.615（模拟）召回率：0.898（真实）/ 0.809（模拟）F1-score：0.597（真实）/ 0.665（模拟）。

View on arXiv Download PDF AI Translation

cs.CV / 212 / 2607.04344

IRIS: An Intelligent Vision-Language System for Ocular Surface Diseases via Topic Tree and Scene-Driven VQA Generation

IRIS：一种通过主题树和场景驱动的视觉问答生成的智能视觉-语言系统，用于眼表疾病

Wei, Hao, Qi, Wenjin, Dai, Dasen, Zhang, Minqing, Yuan, Wu

Abstract

While Large Vision-Language Models (VLMs) demonstrate remarkable generic capabilities, their clinical reasoning in specialized domains like ocular surface diseases (OSDs) is severely hindered by a paucity of high-fidelity, multimodal instruction-tuning data. To dismantle this data bottleneck, we introduce IRIS, an Intelligent Recognition and Interaction System tailored for fine-grained OSD understanding via external eye photography. First, we curate IRIS-120K, the largest and most comprehensive OSD visual question-answering (VQA) dataset to date. Crucially, to overcome the semantic shallowness of conventional image-caption pairs, we propose a synergistic data generation paradigm to explicitly inject clinical priors. Our data engine operates via a dual-branch framework: 1) a Topic Finding Tree (TFT) that hierarchically anchors visual features to precise anatomical and pathological concepts, enforcing rigorous medical deduction logic; and 2) a Scene-driven strategy that synthesizes role-adaptive clinical dialogues to ensure pragmatic generalization. By explicitly aligning a compact 4B-parameter VLM on this structurally enriched corpus, IRIS achieves state-of-the-art performance, comprehensively outperforming both generalist and specialized medical VLMs with up to 34B parameters. Our findings underscore that structured knowledge injection profoundly prevails over sheer parameter scaling, unlocking the potential for resource-efficient, expert-level AI deployment on mobile edge devices for scalable OSD screening. Code, datasets, and model weights will be publicly released by this repo.

Chinese Translation

尽管大型视觉-语言模型（VLMs）展示了显著的通用能力，但在眼表疾病（OSDs）等专业领域的临床推理却受到高保真、多模态指令调优数据匮乏的严重制约。为了解决这一数据瓶颈，我们提出了IRIS，一种智能识别与交互系统，旨在通过外部眼部摄影实现对OSD的细致理解。首先，我们整理了IRIS-120K，这是迄今为止最大的、最全面的OSD视觉问答（VQA）数据集。关键是，为了克服传统图像-标题对的语义浅薄性，我们提出了一种协同数据生成范式，以明确注入临床先验。我们的数据引擎通过双支路框架运行：1）主题发现树（Topic Finding Tree, TFT），它将视觉特征层次性地锚定到精确的解剖和病理概念上，从而强制执行严格的医学推理逻辑；2）场景驱动策略，合成角色自适应的临床对话，以确保务实的推广。通过在这一结构丰富的语料库上明确对齐一个紧凑的4B参数VLM，IRIS实现了最先进的性能，全面超越了参数高达34B的一般和专业医学VLM。我们的研究结果强调，结构化知识注入在很大程度上优于单纯的参数扩展，为在移动边缘设备上进行可扩展的OSD筛查解锁了资源高效、专家级AI部署的潜力。代码、数据集和模型权重将通过此代码库公开发布。

View on arXiv Download PDF AI Translation

cs.CV / 213 / 2607.04352

Last-Meter Precision Navigation for UAVs: A Diffusion-Refined Aerial Visual Servoing Approach

无人机最后一米精确导航：一种扩散优化的空中视觉伺服方法

Li, Yaxuan, Zeng, Jiarui, Huang, Shaofei, Zheng, Zhedong

Abstract

In this work, we study the last-meter precision navigation for UAVs, e.g., autonomously reaching a target within the final 10 meters using monocular vision. This task is challenging due to scale ambiguity, rotation discontinuities, and the need for fine-grained spatial reasoning. Existing methods often fail under large viewpoint changes or lack generalization to unseen environments. To this end, we propose DreamNav, a coarse-to-fine diffusion-refined aerial visual servoing framework. In the first coarse-estimation stage, a robust regression policy employs a trigonometric parameterization to predict rotation by jointly modeling sine and cosine components, effectively mitigating optimization instabilities caused by angular periodicity. Given this coarse estimate, the second diffusion-refined stage utilizes a pre-trained world model to simulate future visual observations for candidate actions, selecting the trajectory that minimizes visual discrepancy with the target through a process of visual imagination. To support rigorous evaluation, we contribute PairUAV, a large-scale benchmark comprising 4.8 million image pairs across 72 scenes, curated from the University-1652 dataset. Extensive experiments show DreamNav outperforms strong visual servoing and foundation model baselines in accuracy and generalization, with zero-shot transfer to unseen scenes.

Chinese Translation

在本研究中，我们探讨了无人机的最后一米精确导航，例如，利用单目视觉自主到达目标的最后10米。这一任务具有挑战性，因为存在尺度模糊、旋转不连续性以及对精细空间推理的需求。现有方法在大视角变化下往往失败，或缺乏对未见环境的泛化能力。为此，我们提出了DreamNav，一个粗到精的扩散优化空中视觉伺服框架。在第一阶段的粗估计中，一个强健的回归策略采用三角函数参数化，通过联合建模正弦和余弦分量来预测旋转，有效缓解了由角度周期性引起的优化不稳定性。基于这一粗略估计，第二个扩散优化阶段利用预训练的世界模型模拟候选动作的未来视觉观测，通过视觉想象过程选择最小化与目标视觉差异的轨迹。为了支持严格的评估，我们贡献了PairUAV，一个包含720个场景中480万对图像的大规模基准数据集，来自University-1652数据集。大量实验表明，DreamNav在准确性和泛化能力上优于强大的视觉伺服和基础模型基线，并实现了对未见场景的零样本迁移。

View on arXiv Download PDF AI Translation

cs.CV / 214 / 2607.04353

HASSL: Hierarchy-Aware Self-Supervised Learning Framework for Single Cell Microscopy

HASSL：面向层次结构的单细胞显微镜自监督学习框架

Riel, Julius, Singh, Vishwa Mohan, Aryasomayajula, Sai Anirudh, Chinbat, Anuun, Leonhard, Hannes, Ladenburger, Moritz, Alexander, Frederik, Choudhary, Vishisht, Laredo, Fabio, Masserdotti, Giacomo, Prein, Thorben, Marr, Carsten, Kardoost, Amirhossein

Abstract

Hierarchical structure is common in image data, where fine-grained clusters often merge into larger, coarser semantic groups. In biological cell images, current self-supervised learning models often suppress this hierarchy, as coarse factors such as imaging modality can obscure finer morphological attributes in the latent space. We propose a hierarchy-aware self-supervised training framework to address this problem. Our method combines two components: a distillation framework with a segmentation teacher to improve morphological awareness in the latent space, and a hierarchy-aware contrastive loss based on HDBSCAN to improve decision boundaries between closely related subtypes at different hierarchical levels. Together, these components reduce the tendency of self-supervised learning to overemphasize coarse factors and instead align embeddings with semantic and morphological cues. This yields biologically meaningful sub-clusters driven by fine morphological detail. We train and evaluate our method on a curated corpus of 2.3 million single cells aggregated from 20 microscopy datasets, both labeled and unlabeled, covering 208 cell classes. Our method improves over baseline and counterpart methods, increasing average top-K accuracy by 2.8%, top-9 retrieval on the dataset with the deepest hierarchy by 6.3%, and downstream F1-score for biologically relevant drug classification from perturbed cell morphology by 7.8%.

Chinese Translation

图像数据中常见层次结构，细粒度聚类往往融合成更大、更粗糙的语义组。在生物细胞图像中，当前的自监督学习模型常常抑制这种层次结构，因为成像模态等粗略因素可能会掩盖潜在空间中的细微形态特征。我们提出了一种面向层次结构的自监督训练框架来解决这一问题。我们的方法结合了两个组件：一个蒸馏框架和一个分割教师，以提高潜在空间中的形态意识，以及基于 HDBSCAN 的面向层次结构的对比损失，以改善不同层次水平上密切相关亚型之间的决策边界。这些组件共同减少了自监督学习过度强调粗略因素的倾向，而是将嵌入与语义和形态线索对齐。这产生了由细微形态细节驱动的生物学上有意义的子聚类。我们在一个经过整理的包含230万单细胞的语料库上训练和评估我们的方法，该语料库来自20个显微镜数据集，包括标记和未标记的细胞，涵盖208个细胞类别。我们的方法在基线和对比方法上有所改进，平均 top-K 准确率提高了2.8%，在层次最深的数据集上的 top-9 检索提高了6.3%，并且在生物相关药物分类的下游 F1-score 提高了7.8%。

View on arXiv Download PDF AI Translation

cs.CV / 215 / 2607.04372

Event Detection in Videos: A Framework for the Development of New Methods

视频中的事件检测：新方法发展的框架

Zakharova, Anastasia, Bouwmans, Thierry, Cioppa, Anthony, Deliège, Adrien, Greco, Antonio, Halin, Anaïs, Jeziorek, Kamil, Kapoor, Meghna, Kryjak, Tomasz, Osman, Islam, Piérard, Sébastien, Sansone, Carlo, Shehata, Mohamed S., Vandeghen, Renaud, Van Droogenbroeck, Marc, Vento, Bruno

Abstract

Event detection tasks in videos, the most important aspect of video surveillance, aim to detect events either at the pixel-level, frame-level, or clip-level. Plenty of methods intended for event detection in different environments, for various applications, and within different acquisition techniques were introduced. Naturally, the attempts were made as well to classify these algorithms in terms of detection of performance or in terms of real-time abilities. Nevertheless, the lack of a large-scale dataset as well as rigorous performance evaluation methods have biased such comparisons as well as the development of the methods. Given the diversity of existing approaches, we believe it is essential for researchers to position their work within such a rich landscape. Thus, we propose a rigorous framework for developing new methods in event detection for videos. Specifically, this framework is based on three main pillars: datasets, performance evaluation, and scenarios for deploying methods.

Chinese Translation

视频中的事件检测任务是视频监控中最重要的方面，旨在在像素级、帧级或片段级别检测事件。为不同环境、各种应用和不同采集技术的事件检测引入了大量方法。自然地，也有人尝试根据检测性能或实时能力对这些算法进行分类。然而，缺乏大规模数据集以及严格的性能评估方法使得这些比较以及方法的发展受到偏见。鉴于现有方法的多样性，我们认为研究人员有必要在这样一个丰富的背景中定位他们的工作。因此，我们提出了一个严格的框架，用于开发视频事件检测的新方法。具体而言，该框架基于三个主要支柱：数据集、性能评估和方法部署场景。

View on arXiv Download PDF AI Translation

cs.CV / 216 / 2607.04401

The Good, the Bad, and the Brittle: Benchmarking Robustness and Generalisation of Histopathology Foundation Models

优与劣与脆弱：病理基础模型的鲁棒性与泛化能力基准测试

Yajnik, Dhyey, Asif, Amina, Minhas, Fayyaz

Abstract

How robust and generalisable are pathology foundation models and have their scaling limites been reached? We benchmarked twelve pathology foundation models (PFMs) and ResNet baselines using our Robustness Evaluation and Enhancement Toolbox (REET) across eleven clinically realistic perturbations and a dissimilarity-driven Non-Redundant K-fold validation (NR-Kfold) protocol. We introduce a Perturbation Performance Index (PPI) to summarise accuracy trends under controlled perturbation sweeps and analyse robustness scaling with parameter count. We show that PFMs consistently outperform CNNs in both robustness and domain generalisation, yet model scaling shows diminishing returns: mid-sized models such (UNI2/Virchow-2 etc.) achieve comparable or greater resilience than larger systems. NR-Kfold analysis further reveals systematic accuracy loss and increased variability when training-test similarity is broken, underscoring the need for explicit distribution-shift evaluation. These findings suggest that the next generation of pathology foundation models must prioritise data quality, multimodality information and domain alignment over parameter count to achieve genuine clinical reliability.

Chinese Translation

病理基础模型的鲁棒性和泛化能力如何？它们的扩展限制是否已经达到？我们使用鲁棒性评估与增强工具箱（REET）对十二个病理基础模型（PFMs）和ResNet基线进行了基准测试，涵盖了十一种临床现实的扰动和一种基于不相似性驱动的非冗余K折验证（NR-Kfold）协议。我们引入了扰动性能指数（PPI）来总结在受控扰动下的准确性趋势，并分析了参数数量与鲁棒性扩展之间的关系。结果表明，PFMs在鲁棒性和领域泛化方面始终优于卷积神经网络（CNNs），然而模型扩展的收益递减：中型模型（如UNI2/Virchow-2等）在韧性方面达到或超过了更大系统的表现。NR-Kfold分析进一步揭示了当训练与测试之间的相似性被打破时，系统性的准确性损失和变异性增加，强调了显式分布转移评估的必要性。这些发现表明，下一代病理基础模型必须优先考虑数据质量、多模态信息和领域对齐，而非单纯的参数数量，以实现真正的临床可靠性。

View on arXiv Download PDF AI Translation

cs.CV / 217 / 2607.04423

Transferability Between Understanding and Generation in Unified Multimodal Models

统一多模态模型中理解与生成之间的可迁移性

Kang, Jiwon, Yoon, Heeji, Jung, Jaewoo, Min, Jaewon, Jeon, Minkyeong, Hwang, Biyeon, Jung, Sangwon, Kim, Seungryong

Abstract

Unified Multimodal Models (UMMs) integrate image understanding and generation within a single architecture, yet how the two tasks interact remains understudied. We investigate $\boldsymbol{\mathsf{transferability}}$ in UMMs: whether training a capability on one task improves the same capability on the other without explicit supervision. Through controlled experiments, we empirically find that transferability depends on architecture-models with fully shared transformer backbone and a unified visual encoder exhibit consistent cross-task transfer, while loosely coupled designs show little or none. Leveraging this transferability, we propose a practical training strategy. The most straightforward way to improve a target generative capability (e.g., counting) is to fine-tune generation directly, but this can degrade visual quality due to distribution shift. Instead, we train the corresponding understanding task and let it transfer into generation, which improves capability-specific generative performance while minimizing distribution shift. We validate this across three capabilities-counting, spatial relation, and text recognition/generation-showing that cross-task transferability can be systematically exploited in UMMs.

Chinese Translation

统一多模态模型（UMMs）在单一架构中整合了图像理解与生成，但这两项任务之间的相互作用仍然未得到充分研究。我们探讨了UMMs中的$oldsymbol{ ext{可迁移性}}$：在没有明确监督的情况下，是否在一项任务上训练某种能力会改善另一项任务上的相同能力。通过控制实验，我们实证发现可迁移性依赖于架构——具有完全共享的变换器骨干网和统一视觉编码器的模型表现出一致的跨任务迁移，而松散耦合的设计则几乎没有或完全没有迁移。利用这种可迁移性，我们提出了一种实用的训练策略。改善目标生成能力（例如计数）的最直接方法是直接微调生成，但这可能由于分布转移而降低视觉质量。相反，我们训练相应的理解任务，并让其迁移到生成中，这在最小化分布转移的同时提高了特定能力的生成性能。我们在三项能力——计数、空间关系以及文本识别/生成——上验证了这一点，显示出跨任务可迁移性可以在UMMs中得到系统性的利用。

View on arXiv Download PDF AI Translation

cs.CV / 218 / 2607.04438

ResearchStudio-Reel: Automate the Last Mile of Research from Paper to Poster, Video, and Blog

ResearchStudio-Reel：自动化研究的最后一公里，从论文到海报、视频和博客

Xiao, Lingao, Dai, Yalun, Huang, Yangyu, Zhao, Qihao, Wu, Wenshan, He, Hugo, Chen, Ruishuo, Jiang, Jin, Ma, Qianli, Zhang, Jiahuan, Zhang, Xin, Xin, Ying, Ou, Yang, Xia, Yan, Li, Scarlett, Huang, Longbo, Zhang, Zhipeng, He, Yang, Hui, Yap Kim, Lu, Yan

Abstract

Research dissemination, turning a paper into a poster, a talk video, and a blog post, is still a manual last mile. Prior automation treats each artifact in isolation that each re-extract the paper from scratch, usually ship one-way renders the author cannot reopen in PowerPoint or Word, and gates quality on soft VLM-preference scores that plateau while load-bearing sections still read as empty. We argue this last mile is best built as a composition of skills: thin agent-readable contracts that share one upstream extractor and wrap deterministic primitives in a measured-fill loop whose exits are hard pass/fail render gates. We instantiate this as ResearchStudio-Reel, five Claude Code and Codex skills organized into one shared extractor (Paper2Assets), three editable generators (Paper2Poster, Paper2Video, Paper2Blog), and one interactive convergence layer (Paper2Reel). Paper2Assets extracts each paper once into a shared bundle that can be reused by every downstream skill; The three generators produce a print-ready poster, a synchronized talk video, and a bilingual blog that stay factually consistent and round-trip through PowerPoint or Word; Paper2Reel then binds all three into a self-contained HTML viewer whose section-level clicks jump the video, slides, captions, and blog to matching content. On the Paper2Poster benchmark, our posters lead every aesthetic and information sub-criterion against both prior automated systems and single-shot frontier LLMs, surpassing the authors' own on aesthetics under two held-out VLM judges and winning overall on 84% to 93% of papers; capability audits further show that, by uniquely pairing narration-aligned on-slide highlights with a bilingual blog gated by layout-aware DOCX repair, ResearchStudio-Reel is the only pipeline to ship all three editable artifacts. Project is available at https://aka.ms/ResearchStudio

Chinese Translation

研究传播，即将论文转化为海报、演讲视频和博客文章，仍然是一个手动的最后一公里。以往的自动化处理将每个成果视为孤立的个体，通常需要从头重新提取论文，生成的内容通常是单向的，作者无法在 PowerPoint 或 Word 中重新打开，并且在软性 VLM 偏好评分上存在质量瓶颈，而承载负载的部分仍然显得空洞。我们认为，这一最后一公里最好是作为一系列技能的组合来构建：薄型代理可读的合同共享一个上游提取器，并将确定性原语包装在一个经过测量的填充循环中，其出口是硬性通过/失败的渲染门。我们将其实现为 ResearchStudio-Reel，包含五个 Claude Code 和 Codex 技能，组织成一个共享提取器（Paper2Assets）、三个可编辑生成器（Paper2Poster、Paper2Video、Paper2Blog）和一个交互式汇聚层（Paper2Reel）。Paper2Assets 将每篇论文提取一次，形成一个共享的包，可以被每个下游技能重用；这三个生成器生成一个可打印的海报、一个同步的演讲视频和一个双语博客，保持事实一致，并可以在 PowerPoint 或 Word 中双向转换；Paper2Reel 然后将所有三个成果绑定到一个自包含的 HTML 查看器中，其章节级别的点击可以跳转到视频、幻灯片、字幕和博客的匹配内容。在 Paper2Poster 基准测试中，我们的海报在所有美学和信息子标准上领先于以往的自动化系统和单次前沿 LLM，超越了作者在两个保留的 VLM 评审下的美学表现，并在 84% 到 93% 的论文中获得整体胜利；能力审计进一步表明，通过独特地将叙述对齐的幻灯片高亮与由布局感知的 DOCX 修复门控的双语博客配对，ResearchStudio-Reel 是唯一能够交付所有三个可编辑成果的管道。项目可在 https://aka.ms/ResearchStudio 获取。

View on arXiv Download PDF AI Translation

cs.CV / 219 / 2607.04443

Wan-Streamer v0.2: Higher Resolution, Same Latency

Wan-Streamer v0.2：更高分辨率，相同延迟

Huang, Lianghua, Wu, Zhi-Fan, Shi, Yupeng, Wang, Wei, Feng, Mengyang, He, Junjie, Xie, Chen-Wei, Liu, Yu, Zhou, Jingren, Wang, Ang, Zhang, Bang, Ai, Baole, Liang, Chen, Yu, Cheng, Zhong, Chongyang, Qi, Jinwei, Zhu, Kai, Li, Pandeng, Zhang, Peng, Zhang, Wenyuan, Cheng, Xinhua, Huang, Yitong, Zheng, Yun, Bao, Yuxiang, Wang, Yuzheng, Bi, Zoubin

Abstract

We present Wan-Streamer v0.2, a latency-preserving upgrade of the native-streaming, end-to-end audio-visual interaction model. v0.2 keeps the v0.1 modeling formulation, but raises the interactive output stream from 192x336 to 640x368 while preserving approximately 200 ms model-side signal-to-signal latency at 25 FPS. The higher-resolution stream supports scene-grounded mid-shot agents whose posture, gaze, hands, nearby objects, and local scene layout remain legible during real-time conversation. To support the larger visual stream without adding user-visible delay, v0.2 keeps the thinker as a single-GPU low-latency path for streaming perception, the short language/state Transformer pass that builds the generation cache, and final decoding. The performer becomes a multi-GPU Ulysses-style context-parallel group for the expensive next-unit latent generation. Each performer rank writes incoming K/V into a pre-sharded local cache. The long high-resolution latent video sequence is split across ranks for denoising and gathered through Ulysses communication, while the much shorter audio latent sequence is generated without sequence sharding. In this split, the thinker's language/state computation reaches the performer only as K/V conditioning, so no separate language sequence has to be communicated inside the performer group. This concentrates additional hardware on visual generation while preserving the compact thinker-performer boundary, keeping total remote interaction latency at approximately 550 ms when a 350 ms bidirectional network budget is included.

Chinese Translation

我们提出了Wan-Streamer v0.2，这是一个延迟保持的升级版本，基于原生流媒体的端到端音视频交互模型。v0.2保持了v0.1的建模公式，但将交互输出流的分辨率从192x336提高到640x368，同时在25 FPS下保持约200毫秒的模型侧信号到信号延迟。更高分辨率的流支持场景基础的中景代理，其姿势、视线、手部、附近物体和局部场景布局在实时对话中保持清晰可辨。为了支持更大的视觉流而不增加用户可见的延迟，v0.2将思考者保持为单GPU低延迟路径，用于流媒体感知、构建生成缓存的短语言/状态Transformer传递和最终解码。表演者则成为一个多GPU的Ulysses风格上下文并行组，用于昂贵的下一个单元潜在生成。每个表演者等级将传入的K/V写入预分片的本地缓存。长的高分辨率潜在视频序列在等级之间进行分割以进行去噪，并通过Ulysses通信进行收集，而更短的音频潜在序列则在不进行序列分片的情况下生成。在这个分割中，思考者的语言/状态计算仅作为K/V条件传递给表演者，因此在表演者组内不需要传递单独的语言序列。这使得额外的硬件集中于视觉生成，同时保持紧凑的思考者-表演者边界，在包括350毫秒双向网络预算时，总远程交互延迟保持在约550毫秒。

View on arXiv Download PDF AI Translation

cs.CV / 220 / 2607.04449

Fields of the Planet: Field Boundary Mapping Beyond 10m

全球田地：超越10米的田界映射

Corley, Isaac, Robinson, Caleb, Marcus, Jennifer, Kerner, Hannah

Abstract

Field-boundary maps support crop monitoring, irrigation planning, and yield estimation, but many smallholder parcels span only a few 10 m Sentinel-2 pixels. We introduce Fields of the Planet (FTP), a 3 m PlanetScope companion to Fields of The World (FTW) that pairs the same polygons, seasonal windows, and train/test splits with 133,168 co-registered PlanetScope patch-window targets across 24 countries. FTP evaluates field delineation as parcel recovery by vectorizing predictions before scoring panoptic quality (PQ), object F1, size-stratified PQ, and meter-scale matched-boundary error. Under matched architectures and training recipes, 3 m imagery raises PQ from 21.0 to 35.5, raises PQ on sub-0.5 ha fields from 5.8 to 15.7, and cuts matched-boundary error from 18.6 m to 7.4 m.

Chinese Translation

田界地图支持作物监测、灌溉规划和产量估算，但许多小农户的地块仅跨越几个10米的Sentinel-2像素。我们介绍了全球田地（Fields of the Planet, FTP），这是一个3米的PlanetScope伴随工具，与全球田地（Fields of The World, FTW）配对，采用相同的多边形、季节窗口和训练/测试分割，并在24个国家中提供133,168个共注册的PlanetScope补丁窗口目标。FTP通过在评分全景质量（panoptic quality, PQ）、对象F1、大小分层PQ和米级匹配边界误差之前对预测进行矢量化，评估田地划分作为地块恢复。在匹配的架构和训练方案下，3米影像将PQ从21.0提高到35.5，将小于0.5公顷的田地PQ从5.8提高到15.7，并将匹配边界误差从18.6米降低到7.4米。

View on arXiv Download PDF AI Translation

cs.CV / 221 / 2607.04451

CCFM: Collision-Constrained Flow Matching for Safety-Critical Scenario Generation

CCFM：用于安全关键场景生成的碰撞约束流匹配

Li, Ke, Liang, Kaidi, Ding, Yuxin, Biswas, Debojyoti, Hu, Xianbiao, Qin, Ruwen

Abstract

Evaluation of autonomous vehicle (AV) planners in safety-critical closed-loop simulation is essential for real-world deployment. However, generating controllable safety-critical scenarios remains challenging. Existing approaches use soft guidance that provides only probabilistic preferences and cannot guarantee the satisfaction of geometric and severity constraints associated with specific collision types. We introduce Collision-Constrained Flow Matching (CCFM), a novel framework that guarantees precise collision control through hard physical constraints. CCFM consists of three key components: (i) a heuristic collision selector that optimally identifies an adversarial agent and collision type via composite scoring; (ii) structured hard constraints that explicitly define four collision types (rear-end, side, cut-in, head-on) through contact point, heading, and severity requirements; and (iii) a collision-constrained flow matching sampler that enforces the constraints via Gauss-Newton manifold projection. CCFM achieves collision rate up to 46.4% on nuScenes and 83.1% on nuPlan, significantly outperforming baselines while preserving realistic driving behavior. By enabling controllable collision characteristics in safety-critical scenario generation, CCFM provides a reliable foundation for AV safety evaluation and sim-to-real crash data generation. The code and implementation details are available at https://github.com/KELISBU/CCFM.

Chinese Translation

在安全关键的闭环仿真中评估自主车辆（AV）规划器对于实际部署至关重要。然而，生成可控的安全关键场景仍然具有挑战性。现有方法使用软引导，仅提供概率性偏好，无法保证满足与特定碰撞类型相关的几何和严重性约束。我们提出了碰撞约束流匹配（CCFM），这是一个通过硬物理约束保证精确碰撞控制的新框架。CCFM由三个关键组件组成：（i）启发式碰撞选择器，通过复合评分最优地识别对抗性代理和碰撞类型；（ii）结构化硬约束，通过接触点、航向和严重性要求明确界定四种碰撞类型（追尾、侧面、插入、正面）；（iii）碰撞约束流匹配采样器，通过高斯-牛顿流形投影强制执行这些约束。CCFM在nuScenes上实现了高达46.4%的碰撞率，在nuPlan上实现了83.1%的碰撞率，显著优于基线，同时保持了真实的驾驶行为。通过在安全关键场景生成中实现可控的碰撞特征，CCFM为AV安全评估和仿真到真实碰撞数据生成提供了可靠的基础。代码和实现细节可在https://github.com/KELISBU/CCFM获取。

View on arXiv Download PDF AI Translation

cs.CV / 222 / 2607.04457

Spatial Graph Representation and Morphometric Analysis of the Pulmonary Vascular Tree From Computed Tomography Using Multi-Scale Hessian-Based Filter Fusion and TEASAR Skeletonization

基于多尺度Hessian滤波融合和TEASAR骨架化的计算机断层扫描肺血管树的空间图表示与形态计量分析

Mackiewicz, Piotr, Kołyska, Jakub, Roszczyk, Radoslaw

Abstract

Reconstructing the pulmonary vascular tree from computed tomography (CT) images is essential for quantitative lung analysis, vascular morphology assessment, and patient-specific modeling, yet it remains challenging because vessels span multiple scales, from proximal arteries to distal microvasculature. Clinical chest CT is further affected by limited spatial resolution, partial volume effects, heterogeneous image quality, and respiratory motion artifacts. Unlike deep learning-based pulmonary vessel segmentation methods that require large annotated datasets, we propose a deterministic, training-free, and explainable pipeline for CT-based pulmonary vascular tree reconstruction. The method fuses multiscale Hessian-based Frangi and Sato vesselness filters using a weighted maximum response across 12 spatial scales from 1 to 8 mm, enabling detection of large pulmonary arteries and peripheral branches. Lung parenchyma is segmented by Hounsfield unit thresholding, morphological post-processing, and Chan-Vese active contour refinement. Vascular centerlines are extracted using the Kimimaro implementation of the TEASAR algorithm; separate left- and right-lung vascular graphs are then constructed, pruned, and verified for acyclicity. Geometric plausibility is assessed using volumetric fractal dimension, Strahler order analysis, Horton ratios, and Murray's law. The resulting fractal dimension of approximately 2.3 is consistent with reported values for the human pulmonary vasculature. At the same time, residual deviations in branching metrics reflect distal-vessel truncation caused by finite CT resolution. These results indicate that the proposed explainable pipeline can generate geometrically plausible pulmonary vascular tree models and may support quantitative pulmonary imaging, vascular morphometry, and computational lung modeling.

Chinese Translation

从计算机断层扫描（CT）图像重建肺血管树对于定量肺部分析、血管形态评估和患者特异性建模至关重要，但由于血管跨越多个尺度，从近端动脉到远端微血管，这一过程仍然具有挑战性。临床胸部CT还受到有限空间分辨率、部分体积效应、图像质量不均匀和呼吸运动伪影的影响。与需要大量标注数据集的基于深度学习的肺血管分割方法不同，我们提出了一种确定性、无训练且可解释的CT肺血管树重建流程。该方法通过在1到8毫米的12个空间尺度上使用加权最大响应融合多尺度Hessian基础的Frangi和Sato血管滤波器，从而能够检测到大型肺动脉及其周边分支。肺实质通过Hounsfield单位阈值分割、形态学后处理和Chan-Vese活动轮廓细化进行分割。血管中心线使用TEASAR算法的Kimimaro实现提取；然后构建、修剪并验证左右肺血管图的无环性。通过体积分形维数、Strahler序列分析、Horton比率和Murray定律评估几何合理性。得到的分形维数约为2.3，与人类肺血管的报告值一致。同时，分支指标中的残余偏差反映了由于有限CT分辨率导致的远端血管截断。这些结果表明，所提出的可解释流程能够生成几何上合理的肺血管树模型，并可能支持定量肺部成像、血管形态计量和计算肺建模。

View on arXiv Download PDF AI Translation

cs.CV / 223 / 2607.04461

Flash-BoN: Instant Drafts for Inference-Time Scaling in Diffusion Models

Flash-BoN：扩散模型推理时刻缩放的即时草稿

Rawal, Ruchit, Shirkavand, Reza, Paul, Sayak, Wen, Yuxin, Huang, Heng, Chen, Yizheng, Goldstein, Tom, Somepalli, Gowthami

Abstract

Inference-time scaling for text-to-image generation has progressed from simple Best-of-$N$ (BoN) sampling to guided search methods that verify and steer candidate trajectories at intermediate denoising steps. These approaches focus on when and how often to verify during denoising but largely treat the cost of generation itself as fixed. Moreover, the standard practice of comparing methods by number of function evaluations (NFEs) counts only denoising forward passes and ignores verifier overhead, which can distort efficiency rankings. We show that under wall-clock evaluation, simple BoN already matches or outperforms several guided search techniques, suggesting that compute is better spent on broader exploration than on repeated intermediate verification. This motivates Flash-BoN, which generates a large pool of inexpensive draft candidates by combining three complementary acceleration knobs: timestep truncation, layer skipping, and activation proxies into a single configuration optimized once per model. An efficient multi-stage verification procedure then identifies the most promising draft, which is refined at full quality. Across three benchmarks and three model scales, Flash-BoN consistently outperforms all baselines under fixed wall-clock budgets, with gains that grow at larger model scales (+8% AUC). We further show that our strategy combines well and improves existing orthogonal techniques such as reflection-based prompt optimization (+16% AUC). The gains correlate with increased candidate diversity, which also enables draft-guided selection to accelerate RL post-training convergence.

Chinese Translation

文本到图像生成的推理时刻缩放已经从简单的最佳-$N$ (BoN) 采样发展到在中间去噪步骤中验证和引导候选轨迹的引导搜索方法。这些方法关注于在去噪过程中何时以及多频繁地进行验证，但在很大程度上将生成成本视为固定。此外，比较方法的标准做法是通过函数评估次数 (NFEs) 来进行，只计算去噪的前向传递，忽略了验证器的开销，这可能扭曲效率排名。我们展示了在墙钟评估下，简单的 BoN 已经能够匹配或超越几种引导搜索技术，这表明计算更适合用于更广泛的探索，而不是重复的中间验证。这激励了 Flash-BoN 的提出，它通过结合三个互补的加速参数：时间步截断、层跳过和激活代理，生成一个大量廉价草稿候选者的池，这一配置在每个模型上优化一次。然后，一个高效的多阶段验证程序识别出最有前景的草稿，并在全质量下进行精炼。在三个基准测试和三个模型规模下，Flash-BoN 在固定的墙钟预算下始终优于所有基线，且在更大模型规模下收益增长 (+8% AUC)。我们进一步展示了我们的策略与现有的正交技术（如基于反射的提示优化）结合良好并改善了其效果 (+16% AUC)。这些收益与候选者多样性的增加相关，这也使得草稿引导选择能够加速强化学习后训练的收敛。

View on arXiv Download PDF AI Translation

cs.CV / 224 / 2607.04472

EVAS: Efficient Multimodal Temporal Forgery Localization via Audio-Visual Synergy and Steered Boundary Calibration

EVAS：通过音视频协同和引导边界校准实现高效的多模态时间伪造定位

Shen, Shen, Zhang, Quan, Jiang, Dan, Zhang, Ke

Abstract

The rapid proliferation of artificial intelligence-generated content necessitates reliable multimodal forensics. Beyond video-level binary classification, precisely localizing sparsely distributed forged segments in long-form videos remains a critical challenge. This task is particularly difficult when manipulations are subtly embedded and cross-modal signals are weak and temporally diffuse. To address these challenges, we propose EVAS, an end-to-end multimodal framework for temporal forgery localization. At its core, a Multi-Stage Audio-Visual Synergy mechanism facilitates progressive cross-modal interaction to learn deep multimodal forensic representations and capture high-order semantic traces of sparse manipulations. Furthermore, we introduce a Boundary-Aware Refinement strategy to achieve steered boundary calibration. By incorporating invalid-frame masking, this strategy suppresses ambiguous regions and sharpens transition predictions. We adopt a decoupled training paradigm with auxiliary heads to disentangle representation learning from inference objectives, enhancing model generalization and stability. Additionally, a lightweight HourglassFFN is incorporated to reduce computational overhead. Extensive experiments demonstrate that EVAS achieves state-of-the-art average localization accuracy and average recall across three benchmark datasets, validating its effectiveness for fine-grained temporal forgery localization.

Chinese Translation

人工智能生成内容的快速传播要求可靠的多模态取证。除了视频级的二元分类外，在长视频中精确定位稀疏分布的伪造片段仍然是一个关键挑战。当操控被微妙地嵌入且跨模态信号较弱且时间上分散时，这项任务尤其困难。为了解决这些挑战，我们提出了EVAS，一个端到端的多模态时间伪造定位框架。其核心是一个多阶段音视频协同机制，促进渐进的跨模态交互，以学习深层多模态取证表示并捕捉稀疏操控的高阶语义痕迹。此外，我们引入了一种边界感知细化策略，以实现引导边界校准。通过引入无效帧遮蔽，该策略抑制模糊区域并增强过渡预测。我们采用了一种解耦训练范式，配备辅助头，以将表示学习与推理目标分离，从而增强模型的泛化能力和稳定性。此外，结合了轻量级的HourglassFFN以减少计算开销。大量实验表明，EVAS在三个基准数据集上实现了最先进的平均定位准确率和平均召回率，验证了其在细粒度时间伪造定位中的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 225 / 2607.04478

PulmoSight-XAI: An Explainable Multi-View Attention Ensemble with Gradient Boosting Meta-Learning for Multi-Label Chest X-Ray Classification

PulmoSight-XAI：一种可解释的多视角注意力集成框架，结合梯度提升元学习用于多标签胸部X光分类

Rahman, Moshiur, Alam, Shafqat, Mamun, Tasnia Binte

Abstract

Automated chest X-ray classification remains challenging due to severe class imbalance, co-occurring pathologies, and the loss of localized features in conventional architectures. To address these, we propose an explainable hierarchical multi-view ensemble framework for the robust classification of 14 thoracic pathologies. The framework employs view-specific training by independently modeling frontal and lateral radiographs using an ensemble of five complementary convolutional neural networks. Replacing global average pooling, a multi-scale feature fusion strategy augmented with Convolutional Block Attention Modules (CBAM) preserves fine-grained intermediate representations while emphasizing high-level pathology-specific semantic features. To mitigate positive-negative imbalance and varying inter-class difficulty, models are optimized using a novel hybrid objective combining Asymmetric Loss with Adaptive Focal Loss. Beyond simple probability averaging, the framework incorporates a hierarchical meta-learning strategy where test-time augmentation (TTA) predictions and cross-model uncertainty measures are integrated into Level-1 gradient-boosting meta-learners (XGBoost, LightGBM, and CatBoost), followed by Level-2 stacking with optimized alpha blending. Evaluated on a large-scale CheXpert-style dataset, the framework achieves state-of-the-art macro-average AUROC scores of 0.9319 for frontal and 0.9154 for lateral radiographs. Furthermore, comprehensive explainability analysis using seven post-hoc attribution techniques demonstrates strong anatomical consistency and clinically meaningful decision localization. By integrating architectural diversity, multi-scale attention, hierarchical meta-learning, and rigorous explainability, the proposed framework provides a transparent, highly accurate, and clinically practical computer-aided diagnosis system for thoracic disease classification.

Chinese Translation

自动化胸部X光分类仍然面临挑战，主要由于严重的类别不平衡、共存病理以及传统架构中局部特征的丧失。为了解决这些问题，我们提出了一种可解释的分层多视角集成框架，用于对14种胸部病理的稳健分类。该框架通过独立建模正面和侧面X光片，采用五个互补卷积神经网络的集成进行视图特定训练。通过替代全局平均池化，结合卷积块注意力模块（Convolutional Block Attention Modules, CBAM）的多尺度特征融合策略，保留了细粒度的中间表示，同时强调了高层次病理特定的语义特征。为了缓解正负样本不平衡和不同类别间的难度变化，模型采用了一种新颖的混合目标进行优化，该目标结合了不对称损失和自适应聚焦损失。该框架不仅仅是简单的概率平均，还结合了一种分层元学习策略，其中测试时增强（Test-Time Augmentation, TTA）预测和跨模型不确定性度量被整合到一级梯度提升元学习器（XGBoost、LightGBM和CatBoost）中，随后进行二级堆叠与优化的α混合。在大规模CheXpert风格数据集上的评估显示，该框架在正面和侧面X光片上分别达到了0.9319和0.9154的最先进的宏平均AUROC分数。此外，通过七种后验归因技术进行的全面可解释性分析展示了强大的解剖一致性和临床意义的决策定位。通过整合架构多样性、多尺度注意力、分层元学习和严格的可解释性，所提出的框架为胸部疾病分类提供了一个透明、高度准确且临床实用的计算机辅助诊断系统。

View on arXiv Download PDF AI Translation

cs.CV / 226 / 2607.04484

TrustCLIP: Learning Private Visual Features via Adversarial Reconstruction

TrustCLIP：通过对抗重建学习私有视觉特征

Athanasiou, Nikos, Petrov, Ilya A., Yao, Angela, Ma, Shugao, Sauser, Eric, Remelli, Edoardo, Hampali, Shreyas, Schönberger, Johannes, Sener, Fadime, Tekin, Bugra

Abstract

Vision and vision-language models rely on high-level visual representations that are increasingly used across recognition, retrieval, and multimodal reasoning pipelines. However, recent advances in generative modeling have shown that such features can often be inverted, enabling realistic reconstructions of the underlying image and raising significant privacy risks. We revisit this problem through the lens of reconstruction and propose TrustCLIP, a reconstruction-driven framework that treats a feature-conditioned generator as an explicit privacy adversary. TrustCLIP learns a projection between encoder features and downstream modules that is explicitly optimized to degrade the reconstructions produced by generative attackers while retaining the necessary signals for downstream tasks. Unlike prior defenses that rely on discriminative privacy metrics, TrustCLIP directly optimizes against a generative reconstruction attacker, targeting a threat not captured by standard evaluation protocols. We demonstrate its effectiveness in both conventional classification and multimodal large language model pipelines. Across these settings, TrustCLIP consistently reduces the fidelity of generative inversions while maintaining downstream task performance. Project page: https://atnikos.github.io/trustclip/

Chinese Translation

视觉和视觉-语言模型依赖于高层次的视觉表征，这些表征在识别、检索和多模态推理流程中被越来越广泛地使用。然而，最近在生成建模方面的进展表明，这些特征往往可以被反转，从而实现对基础图像的真实重建，并带来显著的隐私风险。我们从重建的角度重新审视这个问题，并提出了TrustCLIP，一个以重建为驱动的框架，将特征条件生成器视为一个明确的隐私对手。TrustCLIP学习编码器特征与下游模块之间的投影，该投影被明确优化以降低生成攻击者所产生的重建质量，同时保留下游任务所需的信号。与依赖于判别隐私指标的先前防御方法不同，TrustCLIP直接针对生成重建攻击者进行优化，针对的是标准评估协议未能捕捉到的威胁。我们在传统分类和多模态大型语言模型流程中展示了其有效性。在这些设置中，TrustCLIP始终减少生成反转的保真度，同时保持下游任务的性能。项目页面：https://atnikos.github.io/trustclip/

View on arXiv Download PDF AI Translation

cs.CV / 227 / 2607.04490

Enhancing Facial Expression Recognition in Head-Mounted Displays with Synthetic Data

利用合成数据增强头戴显示器中的面部表情识别

Deng, Jianing, Zhou, Qiang, Hu, Jingtong

Abstract

Facial expression recognition (FER) is crucial for social interaction in mixed reality environments that employ head-mounted displays (HMD). However, collecting FER data from head-mounted cameras (HMC) is challenging due to privacy concerns and the diversity of HMD platforms. Moreover, existing FER datasets are not directly applicable due to the unique perspectives of HMCs. The lack of sufficient data hinders the development of neural network-based HMC FER methods. To address data scarcity, we propose a data synthesis framework that generates HMC-view images from frontal-view images, leveraging abundant existing annotated datasets. Specifically, we first reconstruct 3D textured meshes from images and then apply a configurable camera system to render images from the HMC perspective. Additionally, we introduce a texture-space alignment network (TSAN) that enables accurate texture sampling from images to preserve detailed facial expressions. To evaluate the proposed method, we conduct extensive experiments on both simulated and real HMC datasets. Experimental results demonstrate that models trained on our synthetic dataset outperform those trained on existing datasets and exhibit better generalization across different camera configurations.

Chinese Translation

面部表情识别（FER）在使用头戴显示器（HMD）的混合现实环境中对于社交互动至关重要。然而，由于隐私问题和HMD平台的多样性，从头戴摄像头（HMC）收集FER数据面临挑战。此外，现有的FER数据集由于HMC的独特视角而无法直接应用。数据不足阻碍了基于神经网络的HMC FER方法的发展。为了解决数据稀缺问题，我们提出了一种数据合成框架，该框架利用丰富的现有标注数据集，从正面视图图像生成HMC视角图像。具体而言，我们首先从图像重建3D纹理网格，然后应用可配置的摄像机系统从HMC视角渲染图像。此外，我们引入了一种纹理空间对齐网络（TSAN），该网络能够从图像中进行准确的纹理采样，以保留细致的面部表情。为了评估所提出的方法，我们在模拟和真实的HMC数据集上进行了广泛的实验。实验结果表明，在我们的合成数据集上训练的模型优于在现有数据集上训练的模型，并且在不同摄像机配置下表现出更好的泛化能力。

View on arXiv Download PDF AI Translation

cs.CV / 228 / 2607.04498

UniSkip-Mamba: A Frequency-Aware State Space Model for Audio-Visual Temporal Forgery Localization

UniSkip-Mamba：一种频率感知的状态空间模型用于音视频时间伪造定位

Qiu, Cangjin, Zhang, Quan, Jiang, Dan, Zhang, Ke

Abstract

With the proliferation of AI-generated content, sophisticated multimedia manipulation has raised critical concerns about malicious applications such as opinion manipulation and evidence fabrication, making Audio-Visual Temporal Forgery Localization (AV-TFL) an urgent research frontier. Existing TFL methods have progressed along two main paradigms: Transformer-based temporal modeling and channel-wise multimodal fusion. While these approaches capture temporal dependencies and cross-modal correlations, they process all frequency components indiscriminately, leading to overfitting on high-frequency noise and limited robustness under real-world data degradation. Through systematic frequency domain analysis, we find that forgery-discriminative patterns concentrate in the low/mid-frequency range (normalized frequency 0-0.15), while high-frequency components primarily introduce noise, removing them even improves detection performance by +1.4%. Based on this phenomenon, we propose UniSkip-Mamba, a frequency-aware State Space Model framework that incorporates Unified Multimodal Sequence Fusion to preserve cross-modal phase relationships, and Skip-Scanning Mamba Blocks that implement frequency-aware regularization through a novel Group-Scan-Merge mechanism, naturally biasing learning toward discriminative low/mid-frequency patterns (0-0.15) while maintaining representational completeness. We achieve state-of-the-art (SOTA) performance: 63.4% [email protected] on LAV-DF (+9.8% improvement) and 63.58% mAP on AV-Deepfake1M (+14.32% improvement), with 6x faster inference. Our frequency-domain analysis provides theoretical justification from a signal processing perspective for why skip-scanning inherently improves both accuracy and robustness.

Chinese Translation

随着人工智能生成内容的普及，复杂的多媒体操控引发了对恶意应用（如舆论操控和证据伪造）的严重关注，使得音视频时间伪造定位（Audio-Visual Temporal Forgery Localization, AV-TFL）成为一个紧迫的研究前沿。现有的时间伪造定位方法主要沿着两种范式发展：基于变换器的时间建模和通道级多模态融合。尽管这些方法捕捉了时间依赖性和跨模态相关性，但它们对所有频率成分的处理是无差别的，导致在高频噪声上过拟合，并在真实数据降质下表现出有限的鲁棒性。通过系统的频域分析，我们发现伪造区分模式集中在低/中频范围（归一化频率0-0.15），而高频成分主要引入噪声，去除它们甚至能使检测性能提高1.4%。基于这一现象，我们提出了UniSkip-Mamba，一种频率感知的状态空间模型框架，结合了统一多模态序列融合以保留跨模态相位关系，以及通过新颖的组扫描合并机制实现频率感知正则化的跳跃扫描Mamba块，自然地将学习偏向于区分性低/中频模式（0-0.15），同时保持表征的完整性。我们实现了最先进的性能：在LAV-DF上达到63.4%的[email protected]（提高9.8%），在AV-Deepfake1M上达到63.58%的mAP（提高14.32%），推理速度提高了6倍。我们的频域分析从信号处理的角度为跳跃扫描为何能固有地提高准确性和鲁棒性提供了理论依据。

View on arXiv Download PDF AI Translation

cs.CV / 229 / 2607.04500

Geographic Diversity Beats Data Volume for Cross-Domain Generalization in Zero-Label JEPA Driving World Models

地理多样性在零标签JEPA驾驶世界模型中的跨领域泛化中胜过数据量

Jaiswal, Santosh

Abstract

Self-supervised latent world models can assign a surprise score to driving scenarios without any human labels. A natural follow-up question is whether such a model, trained on driving data from one geographic region, can generalize its notion of complexity to unseen cities and sensor configurations. We study this question through a controlled transfer experiment: we train JEPA-based world models on nuPlan data (Pittsburgh, Boston, Singapore) and evaluate zero-shot on held-out Argoverse 2 validation scenarios from Miami and Austin. We find that models trained on geographically diverse data generalize significantly better than models trained on equal amounts of single-geography data. In a matched-scale ablation at 63,000 scenarios per condition (n=3 seeds each), combined training reduces mean surprise score by 16.5% relative to nuPlan-only training (0.228 +/- 0.015 vs 0.273 +/- 0.008). Notably, training on 200,000 AV2-only scenarios (3x more data from one geography) still produces higher surprise (0.264) than the combined 63K model, suggesting that geographic diversity is a stronger predictor of cross-domain generalization than raw data volume.

Chinese Translation

自监督潜在世界模型能够在没有任何人工标签的情况下为驾驶场景分配惊讶分数。一个自然的后续问题是，是否可以将一个在某个地理区域的驾驶数据上训练的模型的复杂性概念推广到未见过的城市和传感器配置。我们通过一个受控的迁移实验来研究这个问题：我们在nuPlan数据（匹兹堡、波士顿、新加坡）上训练基于JEPA的世界模型，并在从迈阿密和奥斯汀保留的Argoverse 2验证场景上进行零样本评估。我们发现，训练于地理多样性数据的模型在泛化能力上显著优于训练于相同数量单一地理数据的模型。在每个条件下63,000个场景的匹配规模消融实验中（每个条件n=3个种子），结合训练相较于仅使用nuPlan训练将平均惊讶分数降低了16.5%（0.228 +/- 0.015对比0.273 +/- 0.008）。值得注意的是，在200,000个仅AV2场景（来自单一地理区域的3倍数据）上训练，仍然产生了比结合的63K模型更高的惊讶分数（0.264），这表明地理多样性是跨领域泛化的更强预测因子，而非原始数据量。

View on arXiv Download PDF AI Translation

cs.CV / 230 / 2607.04518

A non-invasive video-based method for individual identification of wildlife using gait dynamics

一种基于视频的非侵入性野生动物个体识别方法：步态动态分析

Aamir, Muhammad, Wijers, Matthew, Shin, Sangyun, Loveridge, Andrew, Markham, Andrew

Abstract

Gait is a distinctive behavioral characteristic that enables non-invasive individual identification without requiring physical interaction with an animal. While gait-based analysis has been extensively studied in humans, its application to wildlife remains limited due to environmental variability and the lack of scalable identification methods. This paper presents a fully automated, video-based pipeline for wildlife gait analysis and individual identification using deep spatiotemporal representation learning. The proposed pipeline uses the Segment Anything Model 3 (SAM3) to generate high-quality RGB and binary silhouette masks, robustly isolating animals from complex natural backgrounds. Segmented video sequences are processed using a convolutional neural network (ResNet18) for spatial feature extraction and a transformer-based video model (VideoPrism) for temporal motion modeling. Both models are fine-tuned using a classification objective and subsequently used as feature extractors to generate discriminative gait representations. Cosine similarity is then used to compare gait signatures, enabling similarity-based clustering of individuals without reliance on physical markings or invasive tagging. Experiments conducted on multi-source wildlife video data across multiple species demonstrate strong intra-individual consistency and clear inter-individual separation. Quantitative results using cosine similarity distributions and silhouette scores confirm the effectiveness of the proposed method. These findings demonstrate that gait dynamics provide a viable, non-invasive approach for individual identification in wildlife and highlight the potential of video-based deep learning pipelines for scalable ecological monitoring.

Chinese Translation

步态是一种独特的行为特征，使得在不与动物进行物理接触的情况下实现非侵入性的个体识别。尽管步态分析在人体研究中得到了广泛关注，但由于环境变异性和缺乏可扩展的识别方法，其在野生动物中的应用仍然有限。本文提出了一种完全自动化的基于视频的野生动物步态分析和个体识别管道，采用深度时空表示学习。所提出的管道使用Segment Anything Model 3 (SAM3)生成高质量的RGB和二进制轮廓掩模，稳健地将动物从复杂的自然背景中分离出来。分割后的视频序列通过卷积神经网络（ResNet18）进行空间特征提取，并通过基于变换器的视频模型（VideoPrism）进行时间运动建模。两个模型均使用分类目标进行微调，随后作为特征提取器生成具有区分性的步态表示。然后使用余弦相似度比较步态特征，使得个体的相似性聚类得以实现，而无需依赖物理标记或侵入性标签。在多个物种的多源野生动物视频数据上进行的实验表明，个体内一致性强，个体间分离明显。使用余弦相似度分布和轮廓得分的定量结果证实了所提方法的有效性。这些发现表明，步态动态为野生动物个体识别提供了一种可行的非侵入性方法，并突显了基于视频的深度学习管道在可扩展生态监测中的潜力。

View on arXiv Download PDF AI Translation

cs.CV / 231 / 2607.04541

CRISP: A Spatiotemporal Camera-Radar Backbone for Driving via Forecasting-Based World-Model Pretraining

CRISP：一种基于预测的世界模型预训练的时空相机-雷达骨干网络用于驾驶

Song, Jingyu, Liu, Yi, Skinner, Katherine A.

Abstract

Camera-radar (CR) fusion is a practical sensing configuration for autonomous driving, but existing models are typically trained with task-specific supervision, limiting reusable representation learning. We present CRISP, a spatiotemporal CR backbone pretrained through forecasting-based representation learning. Given historical multi-view images and radar sweeps, CRISP learns a unified bird's-eye-view (BEV) representation by predicting future LiDAR point clouds. LiDAR is used only as privileged supervision during pretraining; the deployed model requires only camera and radar. To make forecasting-based pretraining effective for CR fusion, CRISP introduces an enhanced radar encoder, radar-enhanced temporal self-attention, and multimodal feature rendering with modality innovation gating. These components inject radar range and Doppler cues into BEV temporal propagation and allow BEV tokens to selectively incorporate camera and radar evidence. Experiments on nuScenes show that CRISP improves long-horizon point cloud forecasting and transfers effectively to downstream tasks, including 3D detection, tracking, online mapping, motion forecasting, future occupancy prediction, and planning, suggesting that predictive CR pretraining is a promising path toward scalable driving representations under practical sensor configurations. The project website is https://umfieldrobotics.github.io/CRISP.

Chinese Translation

相机-雷达（CR）融合是一种适用于自动驾驶的实用传感配置，但现有模型通常在特定任务监督下进行训练，这限制了可重用表示学习。我们提出了CRISP，一种通过基于预测的表示学习进行预训练的时空CR骨干网络。给定历史多视角图像和雷达扫描，CRISP通过预测未来的LiDAR点云学习统一的鸟瞰图（BEV）表示。LiDAR仅在预训练期间作为特权监督使用；部署的模型仅需要相机和雷达。为了使基于预测的预训练对CR融合有效，CRISP引入了增强的雷达编码器、雷达增强的时间自注意力机制以及具有模态创新门控的多模态特征渲染。这些组件将雷达范围和多普勒线索注入BEV的时间传播中，并允许BEV标记选择性地整合相机和雷达证据。在nuScenes上的实验表明，CRISP改善了长时间范围的点云预测，并有效地迁移到下游任务，包括3D检测、跟踪、在线映射、运动预测、未来占用预测和规划，这表明预测性CR预训练是实现可扩展驾驶表示在实际传感器配置下的有希望路径。项目网站为 https://umfieldrobotics.github.io/CRISP。

View on arXiv Download PDF AI Translation

cs.CV / 232 / 2607.04548

Explainable Novel Category Discovery in Semantic Concept Space

语义概念空间中的可解释新类别发现

Uddin, Ifrat Ikhtear, Zhou, Yang, Santosh, KC, Wang, Longwei

Abstract

Novel category discovery aims to identify unseen classes from unlabeled data by transferring knowledge from labeled categories, but most existing methods perform discovery in opaque latent feature spaces. As a result, they may separate novel categories accurately while providing little insight into what semantic evidence defines each discovered group. We propose xNCD, an explainable novel category discovery framework that performs both representation-based discovery and pseudo-label assignment directly in a structured semantic concept space. Instead of clustering arbitrary deep features, xNCD learns a label-free concept representation by aligning visual features with vision-language similarity priors from pretrained multimodal models, and then applies a unified labeled-and-unlabeled self-labeling objective over concept-space logits. This design makes each discovered category explainable by construction through stable concept signatures and instance-level concept evidence. Theoretically, we show that routing discovery through a semantic concept bottleneck induces a strict restriction of the feature-space hypothesis class, excluding a large family of unconstrained decision rules and biasing induced partitions toward semantically interpretable concept coordinates. Experiments on CIFAR-10, CIFAR-100, and CUB-200 demonstrate that xNCD preserves strong discovery performance while providing intrinsic explanations. Under task-agnostic evaluation, xNCD achieves 92.63% overall accuracy on CIFAR-10, close to UNO's 93.4%, and improves CIFAR-100 overall accuracy from 73.2% to 76.45%, while being the only compared method that provides human-readable cluster- and instance-level explanations.

Chinese Translation

新类别发现旨在通过从标记类别转移知识，识别未标记数据中的未见类别，但大多数现有方法在不透明的潜在特征空间中进行发现。因此，它们可能准确地区分新类别，但对定义每个发现组的语义证据提供的洞见很少。我们提出了xNCD，一个可解释的新类别发现框架，直接在结构化的语义概念空间中执行基于表示的发现和伪标签分配。xNCD不是对任意深度特征进行聚类，而是通过将视觉特征与来自预训练多模态模型的视觉-语言相似性先验对齐，学习无标签的概念表示，然后在概念空间的logits上应用统一的标记和未标记自标记目标。这一设计通过稳定的概念签名和实例级概念证据，使每个发现的类别在构造上都是可解释的。从理论上讲，我们证明了通过语义概念瓶颈进行路由发现会严格限制特征空间假设类，排除大量不受约束的决策规则，并使引入的划分偏向于语义可解释的概念坐标。在CIFAR-10、CIFAR-100和CUB-200上的实验表明，xNCD在保持强大发现性能的同时，提供内在的解释。在任务无关的评估下，xNCD在CIFAR-10上实现了92.63%的整体准确率，接近UNO的93.4%，并将CIFAR-100的整体准确率从73.2%提高到76.45%，同时是唯一提供人类可读的聚类和实例级解释的比较方法。

View on arXiv Download PDF AI Translation

cs.CV / 233 / 2607.04559

QSVideo: Query-Conditioned Semantic Temporal Retrieval for Video Understanding

QSVideo：基于查询条件的语义时间检索用于视频理解

Ao, Wei, Wang, Lan, Boddeti, Vishnu Naresh

Abstract

The performance of vision-language models (VLMs) in video understanding declines with increasing video duration, as video moments unrelated to the query confuse their language components. Multimodal retrieval has emerged as a critical component of video understanding, addressing this challenge by localizing key visual evidence. However, existing multimodal retrieval methods suffer from biased relevance estimation, limited diversity, and temporal collapse. In this paper, we propose QSVideo, a unified framework that systematically addresses relevance, diversity, and temporal modeling in video retrieval. We first introduce a query-conditioned semantic ranker, QSRanker, which reformulates arbitrary questions into retrieval-friendly queries and estimates structured relevance along object, action, and location dimensions. Building upon this, we design QSRetrieval to jointly optimize relevance and diversity for more informative frame selection. Moreover, we propose temporal alignment strategies tailored for both long and streaming videos to improve evidence recall. Extensive experiments on long and streaming video benchmarks demonstrate that QSVideo greatly enhances video VLM performance under strict frame limit constraints. The code is available at https://github.com/human-analysis/QSVideo.

Chinese Translation

视觉-语言模型（VLMs）在视频理解中的性能随着视频时长的增加而下降，因为与查询无关的视频时刻会干扰其语言组件。多模态检索已成为视频理解的一个关键组成部分，通过定位关键视觉证据来应对这一挑战。然而，现有的多模态检索方法存在相关性估计偏差、有限的多样性和时间崩溃等问题。本文提出了QSVideo，一个统一框架，系统性地解决视频检索中的相关性、多样性和时间建模问题。我们首先引入一个基于查询条件的语义排序器QSRanker，它将任意问题重新表述为适合检索的查询，并沿对象、动作和位置维度估计结构化相关性。在此基础上，我们设计了QSRetrieval，以联合优化相关性和多样性，从而实现更具信息量的帧选择。此外，我们提出了针对长视频和流媒体视频的时间对齐策略，以提高证据召回率。在长视频和流媒体视频基准上的大量实验表明，QSVideo在严格的帧限制条件下显著提升了视频VLM的性能。代码可在 https://github.com/human-analysis/QSVideo 获取。

View on arXiv Download PDF AI Translation

cs.CV / 234 / 2607.04587

RAF: Reliability-Aware Fusion of Camera, LiDAR, and 4D RADAR for Robust 3D Object Detection in Adverse Weather

RAF：针对恶劣天气下鲁棒的3D物体检测的相机、LiDAR和4D RADAR的可靠性感知融合

Park, Heejun, Jeong, Jaeseok, Yoon, Kuk-Jin

Abstract

Robust 3D object detection in adverse weather conditions is challenging due to sensor limitations. Although combining complementary modalities such as LiDAR and 4D RADAR has shown promise, the sparsity of these sensors becomes apparent in adverse weather with reduced reflections, leading to objects with few or no point cloud returns. To address this limitation, camera sensors provide visual cues even when LiDAR and RADAR signals are weakened. However, cameras themselves are also vulnerable to adverse weather, where some regions become unreliable due to snow or rain occluding the camera lens. While some camera-fusion methods designed for adverse weather learn to weigh image regions via confidence maps, these maps receive no direct supervision and are learned solely through the detection loss. We introduce Reliability-Aware Fusion (RAF), which explicitly supervises per-pixel reliability estimation and provides a direct learning signal for identifying and suppressing unreliable visual cues. Our framework leverages pretrained LiDAR-RADAR networks, keeping their backbones frozen while only training the added camera branch, BEV fusion encoder, and detection head. Extensive experiments on the K-Radar and VoD datasets demonstrate that integrating RAF consistently improves detection accuracy over LiDAR-RADAR baselines, achieving up to +6.5 $AP_{BEV}$ and +7.4 $AP_{3D}$ gains. Code is available at https://github.com/parkie0517/RAF.

Chinese Translation

在恶劣天气条件下，鲁棒的3D物体检测面临传感器限制的挑战。尽管结合LiDAR和4D RADAR等互补模态显示出前景，但在恶劣天气中，由于反射减少，这些传感器的稀疏性变得明显，导致物体的点云返回数量很少或没有。为了解决这一限制，相机传感器在LiDAR和RADAR信号减弱时提供视觉线索。然而，相机本身也容易受到恶劣天气的影响，某些区域由于雪或雨遮挡相机镜头而变得不可靠。虽然一些为恶劣天气设计的相机融合方法通过置信度图学习对图像区域进行加权，但这些图没有直接监督，仅通过检测损失进行学习。我们提出了可靠性感知融合（Reliability-Aware Fusion，RAF），该方法明确监督每个像素的可靠性估计，并为识别和抑制不可靠视觉线索提供直接的学习信号。我们的框架利用预训练的LiDAR-RADAR网络，保持其主干不变，仅训练新增的相机分支、BEV融合编码器和检测头。在K-Radar和VoD数据集上的大量实验表明，集成RAF能够持续提高检测准确性，相较于LiDAR-RADAR基线，获得高达+6.5 $AP_{BEV}$和+7.4 $AP_{3D}$的提升。代码可在https://github.com/parkie0517/RAF获取。

View on arXiv Download PDF AI Translation

cs.CV / 235 / 2607.04593

TORINO: Token Reduction via Interpretable Concept Overlap in Vision-Language Models

TORINO：通过可解释概念重叠实现视觉语言模型中的令牌减少

Renzulli, Riccardo, Spadaro, Gabriele, Gowda, Shruthi, Mazouz, Alaa Eddine, Nguyen, Van-Tam

Abstract

Vision-Language Models (VLMs) have demonstrated impressive capabilities across different tasks, but their computational cost is dominated by the large number of visual tokens fed to the language model. Existing token reduction methods rely on attention-based scores or pairwise similarity, without an explicit semantic representation of each token. We introduce TORINO (TOken Reduction via Interpretable coNcept Overlap), a plug-and-play framework for adaptive visual token reduction in VLMs that requires no fine-tuning of the underlying model. TORINO leverages Sparse Autoencoders (SAEs) to project visual tokens into an interpretable latent space where token relationships can be analyzed through shared concept activations. Specifically, we define concept overlap as the degree of agreement between active SAE latents and use it to group tokens that share semantic content. Reduction within each group is then performed by either pruning or merging, providing a unified framework that preserves semantically important visual information while removing redundancy. Unlike fixed-budget approaches, TORINO dynamically adapts the reduction rate to input complexity, allowing different images to retain different numbers of tokens. Experiments across multiple vision-language benchmarks show that TORINO achieves favorable efficiency-accuracy trade-offs, reducing the number of visual tokens with minimal performance loss.

Chinese Translation

视觉语言模型（VLMs）在不同任务中展现了令人印象深刻的能力，但其计算成本主要由输入语言模型的大量视觉令牌所主导。现有的令牌减少方法依赖于基于注意力的评分或成对相似性，而没有对每个令牌进行明确的语义表示。我们提出了TORINO（通过可解释的概念重叠进行令牌减少），这是一个用于VLM中自适应视觉令牌减少的即插即用框架，无需对基础模型进行微调。TORINO利用稀疏自编码器（Sparse Autoencoders, SAEs）将视觉令牌投影到一个可解释的潜在空间中，在该空间中可以通过共享概念激活分析令牌之间的关系。具体而言，我们将概念重叠定义为活跃SAE潜变量之间的一致性程度，并利用这一点将共享语义内容的令牌进行分组。然后，通过修剪或合并的方式在每个组内进行减少，提供一个统一的框架，既保留了语义重要的视觉信息，又去除了冗余。与固定预算的方法不同，TORINO动态调整减少率以适应输入复杂性，使得不同图像可以保留不同数量的令牌。在多个视觉语言基准测试中的实验表明，TORINO在效率与准确性之间实现了良好的权衡，在性能损失最小的情况下减少了视觉令牌的数量。

View on arXiv Download PDF AI Translation

cs.CV / 236 / 2607.04599

Displacement Preserving Relational Distillation for Robust Medical Segmentation

保持位移的关系蒸馏用于稳健的医学分割

Ding, Zhicheng, Chu, Xinyu, Choi, Jung Im, Tian, Qing, Shi, Tianyu, Jiang, Xiaoqian, Zhu, Lijing, Lan, Qizhen

Abstract

Accurate 3D medical segmentation is limited by anatomical variability and high computational costs. While knowledge distillation (KD) offers a route for model compression, conventional methods often fail to preserve complex structures and are overwhelmed by background noise. We propose Displacement-Preserving Relational Distillation (DPRD), which distills latent anatomical trajectories via vector based alignment to preserve the orientation and relative scale of the teacher's manifold, and prevents signal dilution by anchoring distillation in task-relevant structures. Integrated into nnU-Net, DPRD outperforms established baselines on ISLES 2022 and AMOS 2022 benchmarks. Notably, on the AMOS dataset, DPRD achieves a Dice score of 85.46%, edging out the high-capacity MedNeXt teacher while significantly reducing boundary errors. Despite utilizing only ~5% of the teacher's parameters and ~3% of its FLOPs, our approach maintains high structural consistency. This provides a robust, efficient solution for deploying high performance segmenters in resource-constrained clinical environments. Code: https://github.com/ClinicaAlpha/DPRD-3D-MedSeg

Chinese Translation

准确的三维医学分割受到解剖变异性和高计算成本的限制。虽然知识蒸馏（Knowledge Distillation, KD）为模型压缩提供了一条途径，但传统方法往往无法保留复杂结构，并且容易受到背景噪声的干扰。我们提出了保持位移的关系蒸馏（Displacement-Preserving Relational Distillation, DPRD），通过基于向量的对齐蒸馏潜在的解剖轨迹，以保留教师模型流形的方向和相对尺度，并通过将蒸馏锚定在与任务相关的结构中来防止信号稀释。将DPRD集成到nnU-Net中后，在ISLES 2022和AMOS 2022基准测试中超越了既定基线。值得注意的是，在AMOS数据集中，DPRD达到了85.46%的Dice系数，超越了高容量的MedNeXt教师，同时显著减少了边界错误。尽管仅使用了教师模型约5%的参数和约3%的FLOPs，我们的方法仍然保持了高结构一致性。这为在资源受限的临床环境中部署高性能分割器提供了一种稳健且高效的解决方案。代码链接： https://github.com/ClinicaAlpha/DPRD-3D-MedSeg

View on arXiv Download PDF AI Translation

cs.CV / 237 / 2607.04603

LCPNet: Latent Consistent Proximal Unfolding Network for Infrared Small Target Detection

LCPNet：用于红外小目标检测的潜在一致近端展开网络

Zhang, Tianfang, Wu, Fengyi, Li, Lei, Liu, Chang, Peng, Zhenming, Zhang, Huaping, Ji, Xiangyang

Abstract

Infrared small target detection (IRSTD) aims to identify long distance small targets from complex infrared backgrounds, and is a fundamental task in remote sensing. Deep learning methods have improved IRSTD by learning discriminative image-to-mask mappings, but such feed-forward designs often underuse physical decomposition structure between targets and backgrounds. Deep unfolding methods partially address this issue by embedding model-driven iterations into neural networks, yet existing designs still operate mainly in image domain and use updates and memory mechanisms that are not fully coupled with underlying optimization process. To address these limitations, we propose Latent Consistent Proximal unfolding network (LCPNet). First, we verify that low-rank prior remains valid in latent representations and perform unfolding in this space, preserving physical constraint while avoiding repeated compression of intermediate states. Second, we derive a Latent Consistent Proximal (LCP) solver that evolves each latent variable from its previous state rather than reconstructing through an indirect residual, and stabilizes small target updates through task-adaptive normalization and gain control. Third, we introduce Shared Optimization Memory (SOM), a common historical state shared by all decomposition variables to provide coordinated guidance across unfolding stages. Extensive experiments on four public benchmarks demonstrate that LCPNet outperforms state-of-the-art methods while achieving accurate and robust detection with low false alarms and competitive efficiency. Model and code are available at https://github.com/Tianfang-Zhang/LCPNet.

Chinese Translation

红外小目标检测（IRSTD）旨在从复杂的红外背景中识别远距离的小目标，是遥感中的一项基础任务。深度学习方法通过学习具有区分性的图像到掩膜的映射来改善IRSTD，但这种前馈设计往往未充分利用目标与背景之间的物理分解结构。深度展开方法部分解决了这一问题，通过将模型驱动的迭代嵌入神经网络中，但现有设计仍主要在图像域中操作，并使用与基础优化过程未完全耦合的更新和记忆机制。为了解决这些局限性，我们提出了潜在一致近端展开网络（LCPNet）。首先，我们验证了低秩先验在潜在表示中仍然有效，并在该空间中进行展开，保持物理约束，同时避免对中间状态的重复压缩。其次，我们推导出一种潜在一致近端（LCP）求解器，该求解器从每个潜在变量的前一个状态演变，而不是通过间接残差进行重构，并通过任务自适应归一化和增益控制来稳定小目标更新。第三，我们引入了共享优化记忆（SOM），这是所有分解变量共享的共同历史状态，以在展开阶段提供协调指导。在四个公共基准上的大量实验表明，LCPNet在实现低误报和竞争效率的同时，超越了最先进的方法，达到了准确且稳健的检测效果。模型和代码可在 https://github.com/Tianfang-Zhang/LCPNet 获取。

View on arXiv Download PDF AI Translation

cs.CV / 238 / 2607.04607

G2VD: Generalizable AI-Generated Video Detection via Counterfactual Intervention and Causal Disentanglement

G2VD：通过反事实干预和因果解缠结实现可泛化的AI生成视频检测

Du, Meng, Chen, Hongchang, Li, Ran, Zhang, Junjie, Ouyang, Qi, Liu, Shuxin

Abstract

The rapid advancement of AI-generated videos poses increasing security risks and calls for robust detectors with strong cross-domain generalization. Although existing methods achieve promising results under in-domain evaluation, their performance often degrades substantially when tested on unseen generators. A key reason is shortcut learning, where detectors rely on domain-specific spurious cues, such as generator-dependent fingerprints and generation styles, instead of intrinsic forgery traces. To address this issue, we propose G2VD, a Generalizable AI-Generated Video Detection framework based on counterfactual intervention and causal disentanglement. First, G2VD introduces a counterfactual intervention pipeline (CFIPipeline) that generates controlled counterfactual samples via variational autoencoders (VAEs), followed by frequency-domain and pixel-domain alignment, thereby encouraging the detector to focus on generator-intrinsic cues. Building on this intervention process, we further design a causal disentanglement classifier consisting of two domain-anchored branches with distinct classification objectives, combined with an HSIC-based independence constraint to encourage the separation of task-relevant cues from domain-specific bias. Across four public datasets, G2VD shows strong average cross-domain performance and consistent gains over matched backbones. On the challenging GenVidBench cross-domain setting, it exceeds 90% accuracy and reaches an AUC close to 0.95. Notably, this performance is obtained using only 10% of the original training data. The code is available at https://github.com/dumeng98/G2VD.

Chinese Translation

AI生成视频的快速发展带来了日益增加的安全风险，并呼唤具有强大跨域泛化能力的鲁棒检测器。尽管现有方法在域内评估中取得了令人满意的结果，但在未见生成器上测试时，其性能往往显著下降。一个关键原因是捷径学习，检测器依赖于特定于域的虚假线索，如生成器依赖的指纹和生成风格，而不是内在的伪造痕迹。为了解决这个问题，我们提出了G2VD，一个基于反事实干预和因果解缠结的可泛化AI生成视频检测框架。首先，G2VD引入了一个反事实干预管道（CFIPipeline），通过变分自编码器（VAEs）生成受控的反事实样本，随后进行频域和像素域对齐，从而鼓励检测器关注生成器内在线索。在这一干预过程的基础上，我们进一步设计了一个因果解缠结分类器，由两个具有不同分类目标的域锚定分支组成，并结合基于HSIC的独立性约束，以鼓励将任务相关线索与特定于域的偏差分离。在四个公共数据集上，G2VD显示出强大的平均跨域性能，并在匹配的骨干网络上持续获得提升。在具有挑战性的GenVidBench跨域设置中，其准确率超过90%，AUC接近0.95。值得注意的是，这一性能是在仅使用原始训练数据的10%的情况下获得的。代码可在https://github.com/dumeng98/G2VD获取。

View on arXiv Download PDF AI Translation

cs.CV / 239 / 2607.04608

Integrated Forward-Inverse Network for Lensless Image Reconstruction

集成前向-逆向网络用于无透镜图像重建

Bae, Donggeon, Jung, Jaewoo, Kang, Yong Guk, Lee, Kyung Chul, Kim, Taeyoung, Kim, Jongho, Byun, Sangjun, Park, Joonsik, Lee, Seung Ah

Abstract

Lensless imaging enables compact and versatile computational cameras by replacing bulky optics with thin coded elements. However, reconstruction from the resulting measurements is challenging: large-footprint point-spread functions (PSFs) produce highly multiplexed observations, making inversion severely ill-conditioned and sensitive to calibration errors and model mismatch. While deep learning approaches, including hybrid models that incorporate physics priors, have shown promise, explicitly maintaining data fidelity throughout the network hierarchy remains difficult. Here, we propose the Integrated Forward-Inverse Network (IFIN), a physics-guided architecture that interleaves differentiable forward projections with learnable inverse updates at every scale, enabling complementary cues to be exploited jointly in the measurement and image domains. This bidirectional coupling supports progressive, physics-consistent refinement and permits system-constrained PSF kernel adaptation under model uncertainty. On challenging lensless benchmarks, including a newly introduced dataset, IFIN achieves state-of-the-art reconstruction quality. We further observe competitive performance on Gaussian deblurring and simulated inline holography reconstruction, suggesting that the same interleaving principle can extend beyond lensless cameras.

Chinese Translation

无透镜成像通过用薄编码元件替代笨重的光学元件，实现了紧凑且多功能的计算相机。然而，从所得到的测量数据中进行重建是具有挑战性的：大面积点扩散函数（PSF）产生高度多路复用的观测，使得反演过程严重病态且对校准错误和模型不匹配敏感。尽管包括结合物理先验的混合模型在内的深度学习方法显示出了希望，但在整个网络层次中明确保持数据保真性仍然困难。在此，我们提出了集成前向-逆向网络（Integrated Forward-Inverse Network, IFIN），这是一种物理引导的架构，在每个尺度上交错可微的前向投影与可学习的逆向更新，从而能够在测量和图像域中共同利用互补线索。这种双向耦合支持渐进的、物理一致的细化，并允许在模型不确定性下进行系统约束的PSF核适应。在具有挑战性的无透镜基准测试中，包括一个新引入的数据集，IFIN实现了最先进的重建质量。我们进一步观察到在高斯去模糊和模拟在线全息重建中的竞争性能，表明相同的交错原理可以扩展到无透镜相机之外。

View on arXiv Download PDF AI Translation

cs.CV / 240 / 2607.04625

Hierarchical Evidence-Driven Reasoning for Long Document Understanding

基于层次证据驱动的长文档理解推理

Xiong, Junyu, Wang, Yonghui, Gu, Rongjian, Liu, Chenyu, Yin, Bing, Zhou, Wengang, Li, Houqiang

Abstract

Retrieval-Augmented Generation (RAG) streamlines long-document understanding by leveraging retrieval mechanisms to restrict input images to a highly curated subset. However, existing multimodal RAG pipelines primarily face two critical challenges: first, standard semantic similarity retrievers frequently fetch topically overlapping yet answer-void distractor pages that mislead downstream generation; second, rigid single-pass pipelines heavily depend on initial retrieval success, where any omission of core evidence inevitably causes cascading errors. To address these challenges, we introduce HIEVI-RAG, a hierarchical, evidence-driven multimodal RAG framework for closed-domain document understanding. HIEVI-RAG systematically factorizes complex queries into a cooperative four-stage pipeline: (1) hierarchical question decomposition to break multi-hop root queries into atomic child questions; (2) coarse visual page retrieval leveraging a multimodal retriever to fetch candidate pages based on semantic similarity; (3) fine-grained page verification via EVIAGENT, a specialized multi-page verifier trained with GRPO to execute cross-page reasoning over multi-image blocks; and (4) memory-guided iterative generation that leverages accumulated sub-question context to execute multi-round, dynamic reasoning over the prioritized sequence. Extensive evaluations across four benchmarks demonstrate the robust efficacy and synergy of our framework, which significantly outperforms existing open-source baselines and exceeds the strongest reported baseline by an average of 8.05% in accuracy.

Chinese Translation

检索增强生成（Retrieval-Augmented Generation, RAG）通过利用检索机制将输入图像限制为高度策划的子集，从而简化了长文档理解。然而，现有的多模态 RAG 流水线主要面临两个关键挑战：首先，标准的语义相似性检索器经常获取主题重叠但缺乏答案的干扰页面，这会误导下游生成；其次，刚性的一次性流水线严重依赖于初始检索的成功，任何核心证据的遗漏不可避免地会导致级联错误。为了解决这些挑战，我们提出了 HIEVI-RAG，这是一种层次化、证据驱动的多模态 RAG 框架，旨在闭域文档理解。HIEVI-RAG 系统地将复杂查询分解为一个协作的四阶段流水线：（1）层次化问题分解，将多跳根查询拆分为原子子问题；（2）粗略视觉页面检索，利用多模态检索器根据语义相似性获取候选页面；（3）通过 EVIAGENT 进行细粒度页面验证，EVIAGENT 是一个经过 GRPO 训练的专门多页面验证器，能够对多图像块执行跨页面推理；（4）基于记忆的迭代生成，利用累积的子问题上下文执行多轮动态推理。通过在四个基准上的广泛评估，证明了我们框架的强大有效性和协同作用，其性能显著超越现有的开源基准，并在准确率上平均超过最强报告基准 8.05%。

View on arXiv Download PDF AI Translation

cs.CV / 241 / 2607.04635

Aperture-aware Dispersion 5-D Light-field Imaging Spectrometer

考虑孔径的分散5维光场成像光谱仪

Huang, Chenglong, Lv, Tao, Yang, Jianing, Zi, Chongde, Chen, Linsen, Cao, Xun

Abstract

Enhancing perceptual dimensions while miniaturizing imaging systems presents significant challenges for high-dimensional visual sensing. Conventionally, the acquisition of the 5D (x,y,u,v,{\lambda}) spectral light field (5D-SLF) data cube relies on bulky and expensive camera arrays, which are impractical for widespread application. Existing single-detector systems are fundamentally limited by a trade-off between the resolutions of different dimensions owing to insufficient coding capabilities. Here we introduce an Aperture-aware Dispersion Light-field Imaging Spectrometer (ADLIS), that targets a synergy between compactness and resolution through aperture-multiplexed modulation, leveraging the inherent spectral-filtering properties of birefringent material. Using only a manufacturing-friendly and cost-effective phase plate made of birefringent quartz crystal, the aperture of the proposed ADLIS enables compact angular-spectral encoding that is highly sensitive to both the incident angle and spectrum of incoming light. In contrast to the viewpoint-separation approach of microlens arrays, ADLIS employs aperture encoding to superimpose all viewpoints onto each sensor pixel. This shifts the design paradigm from spatial division to encoding integration, aiming to achieve full-resolution light field recovery. Thus, we develop the Aperture-aware Dispersion Light-field Imaging (ADLI) framework, which optimizes the aperture design and 5D-SLF reconstruction in an end-to-end (E2E) manner. Trained by simulation data and validated through real-world experiments, our system achieves robust high-performance 5D-SLF imaging while maintaining full spatial resolution.

Chinese Translation

在缩小成像系统的同时增强感知维度为高维视觉传感带来了重大挑战。传统上，5维（x,y,u,v,{BB}）光谱光场（5D-SLF）数据立方体的获取依赖于庞大且昂贵的相机阵列，这在广泛应用中并不实用。现有的单探测器系统由于编码能力不足，基本上受到不同维度分辨率之间的权衡限制。在此，我们引入了一种考虑孔径的分散光场成像光谱仪（ADLIS），该仪器通过孔径多路复用调制实现紧凑性与分辨率之间的协同，利用了双折射材料固有的光谱滤波特性。该ADLIS仅使用一种由双折射石英晶体制成的制造友好且成本效益高的相位板，其孔径能够实现对入射光的角度和光谱的高度敏感的紧凑角谱编码。与微透镜阵列的视点分离方法不同，ADLIS采用孔径编码将所有视点叠加到每个传感器像素上。这将设计范式从空间划分转变为编码集成，旨在实现全分辨率光场恢复。因此，我们开发了考虑孔径的分散光场成像（ADLI）框架，该框架以端到端（E2E）方式优化孔径设计和5D-SLF重建。通过模拟数据进行训练，并通过实际实验验证，我们的系统在保持全空间分辨率的同时，实现了稳健的高性能5D-SLF成像。

View on arXiv Download PDF AI Translation

cs.CV / 242 / 2607.04636

Enhancing Large Multimodal Models in Key Information Extraction via Scene-Aware Document Synthesis

通过场景感知文档合成增强关键资讯提取中的大型多模态模型

Xu, Zhipeng, Chen, Zulong, Liu, Qing, Ji, Junhao, Hu, Jinxin, Yu, Yipeng, Wan, Jianqiang, Tang, Jun, Li, Zhao

Abstract

Key Information Extraction (KIE) converts visually rich documents into structured data, but practical deployment remains challenging: strong performance often relies on costly on-server Large Multimodal Models (LMMs), while compact locally deployable models lack sufficient KIE supervision. We present SAYRE, a scene-aware document synthesis framework for generating scalable KIE training data without hand-crafted template design. Given a few exemplar documents, SAYRE captures category-specific content patterns and layout conventions to synthesize document-schema-annotation triples. It further introduces error-driven generation, which expands real-world failure cases into hard training examples while preserving their structural patterns. Experiments on constrained- and open-category KIE show that SAYRE consistently improves Qwen3-VL backbones and achieves the strongest overall performance among on-device LMMs. Data scaling experiments show an overall upward trend as more synthesized data is introduced, especially for smaller models and open-category extraction. Error analysis further shows that synthesized training reduces field-level errors by improving schema-aware extraction over dense tables, business identifiers, and contract clauses. These results establish scene-aware synthesis as an effective data-centric approach for improving practical multimodal KIE.

Chinese Translation

关键资讯提取（KIE）将视觉丰富的文档转换为结构化数据，但实际部署仍然面临挑战：强大的性能通常依赖于成本高昂的服务器端大型多模态模型（LMMs），而紧凑的本地可部署模型缺乏足够的KIE监督。我们提出了SAYRE，一个场景感知文档合成框架，用于生成可扩展的KIE训练数据，而无需手工设计模板。在给定少量示例文档的情况下，SAYRE捕捉特定类别的内容模式和布局规范，以合成文档-模式-注释三元组。它进一步引入了基于错误驱动的生成，扩展现实世界中的失败案例为困难的训练示例，同时保留其结构模式。在受限类别和开放类别KIE的实验中，SAYRE始终改善了Qwen3-VL骨干网络，并在设备上的LMMs中实现了最强的整体性能。数据扩展实验显示，随着更多合成数据的引入，整体趋势向上，尤其是在较小模型和开放类别提取方面。错误分析进一步表明，合成训练通过改善对密集表格、商业标识符和合同条款的模式感知提取，减少了领域级错误。这些结果确立了场景感知合成作为一种有效的数据中心方法，以改善实际多模态KIE。

View on arXiv Download PDF AI Translation

cs.CV / 243 / 2607.04637

PixelPilot: Scalable Vision-Language-Action Models for End-to-End Autonomous Driving

PixelPilot：可扩展的视觉-语言-动作模型用于端到端自主驾驶

Tang, Pin, Wang, Guoqing, Ren, Xiangxuan, Wang, Zhongdao, Zhao, Guodongfang, Bailan, Ma, Chao

Abstract

Vision-Language-Action Models (VLAs), which leverage the advanced reasoning capabilities of Vision-Language Models (VLMs), show promising generalization in complex autonomous driving scenarios. Existing VLAs typically predict and optimize 3D trajectories from 2D images. While intuitive, this 2D-to-3D prediction is inherently entangled with camera parameters, leading to limited data scalability across heterogeneous driving datasets. Moreover, directly optimizing in 3D space induces severe convergence to trivial solutions, where VLAs rely on ego-status rather than visual scene understanding. To address these issues, we propose PixelPilot, a novel VLA featuring a decoupled planning and lifting paradigm. In the planning phase, PixelPilot reformulates scene understanding and trajectory prediction as sensor-agnostic 2D-to-2D tasks in the image plane, thereby facilitating scalable training across diverse datasets. The planned 2D trajectories are then deterministically lifted to 3D only during inference, ensuring the full exploitation of visual cues and generalization across different vehicles. To realize this paradigm, we propose a knowledge-instilled policy learning strategy that applies dense, intermediate rewards via Group Relative Policy Optimization (GRPO) to enforce a rigorous causal chain from visual perception to spatial planning. Extensive experiments demonstrate that PixelPilot achieves state-of-the-art performance in both open-loop and closed-loop settings, validating its superior scalability and visual reasoning capabilities.

Chinese Translation

视觉-语言-动作模型（VLAs）利用视觉-语言模型（VLMs）的先进推理能力，在复杂的自主驾驶场景中展现出良好的泛化能力。现有的VLA通常从2D图像预测和优化3D轨迹。尽管这种2D到3D的预测直观，但它本质上与相机参数纠缠在一起，导致在异构驾驶数据集上的数据扩展性有限。此外，直接在3D空间中优化会导致严重收敛于平凡解，使得VLA依赖于自我状态而非视觉场景理解。为了解决这些问题，我们提出了PixelPilot，一种新颖的VLA，采用解耦的规划和提升范式。在规划阶段，PixelPilot将场景理解和轨迹预测重新表述为图像平面中的传感器无关的2D到2D任务，从而促进在多样化数据集上的可扩展训练。计划的2D轨迹仅在推理阶段确定性地提升到3D，确保充分利用视觉线索并在不同车辆之间实现泛化。为了实现这一范式，我们提出了一种知识灌输的策略学习策略，通过群体相对策略优化（GRPO）应用密集的中间奖励，以强制从视觉感知到空间规划的严格因果链。大量实验表明，PixelPilot在开环和闭环设置中均实现了最先进的性能，验证了其卓越的可扩展性和视觉推理能力。

View on arXiv Download PDF AI Translation

cs.CV / 244 / 2607.04638

Learning Structured Visual Compositional Representations for Weakly Supervised Referring Expression Comprehension

学习结构化视觉组合表示以实现弱监督指称表达理解

Xu, Lian, Bennamoun, Mohammed, Boussaid, Farid, Laga, Hamid, Guo, Yulan, Xu, Dan

Abstract

Referring expression comprehension (REC) aims to localize the object in an image described by natural language. In Weakly supervised REC (WREC), existing approaches primarily operate on anchor-level visual representations. Even when enriched with auxiliary cues, relational interactions remain implicitly encoded within individual anchor features. The resulting visual representation remains flat and unary-only, limiting its ability to align with the structured nature of language. In this work, we propose a Structured Visual Compositional Representation (SVCR) learning framework for WREC. Rather than implicitly encoding relations within unary anchors, the proposed SVCR explicitly models both unary object embeddings and pairwise relational embeddings, forming a structured visual representation space. We further introduce a compositional alignment mechanism that matches unary and pairwise visual representations with their corresponding textual embeddings in a unified manner, enabling compositional visual-textual matching under weak supervision. Extensive experiments on RefCOCO, RefCOCO+, and RefCOCOg show that the proposed SVCR achieves state-of-the-art performance. These results demonstrate the effectiveness of explicit structured visual representations and visual-textual alignment for WREC.

Chinese Translation

指称表达理解（REC）旨在根据自然语言定位图像中的对象。在弱监督指称表达理解（WREC）中，现有方法主要基于锚点级视觉表示进行操作。即使在辅助手段的丰富下，关系交互仍然隐含地编码在单个锚点特征中。由此产生的视觉表示仍然是平坦的，仅限于单一特征，限制了其与语言结构化特性的对齐能力。在本研究中，我们提出了一种用于WREC的结构化视觉组合表示（SVCR）学习框架。与其在单一锚点中隐含编码关系，所提出的SVCR明确建模单一对象嵌入和成对关系嵌入，形成一个结构化的视觉表示空间。我们进一步引入了一种组合对齐机制，以统一的方式将单一和成对视觉表示与其对应的文本嵌入进行匹配，从而在弱监督下实现组合视觉-文本匹配。在RefCOCO、RefCOCO+和RefCOCOg上的大量实验表明，所提出的SVCR达到了最先进的性能。这些结果证明了显式结构化视觉表示和视觉-文本对齐在WREC中的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 245 / 2607.04643

Learning Flexible Generalization in Video Quality Assessment by Bringing Device and Viewing Condition Distributions

通过引入设备和观看条件分布实现视频质量评估中的灵活泛化学习

Safonov, Nikolay, Vatolin, Dmitriy S.

Abstract

Video quality assessment (VQA) plays a critical role in optimizing video delivery systems. While numerous objective metrics have been proposed to approximate human perception, the perceived quality strongly depends on viewing conditions and display characteristics. Factors such as ambient lighting, display brightness, and resolution significantly influence the visibility of distortions. In this work, we address the question of the multi-screen quality assessment on mobile devices, as this area still tends to be under-covered. We introduce a first large-scale subjective dataset collected across more than different 300 Android devices, accompanied by metadata on viewing conditions and display properties. We propose a strategy for aggregated score extraction and adaptation of VQA models to device-specific quality estimation. Our results demonstrate that incorporating device and context information enables more accurate and flexible quality prediction, offering new opportunities for fine-grained optimization in streaming services. Ultimately, this work advances the development of perceptual quality models that bridge the gap between laboratory evaluations and the diverse conditions of real-world media consumption. We made the dataset and the code available at https://videoprocessing.github.io/device-viewing-conditions.

Chinese Translation

视频质量评估（VQA）在优化视频传输系统中发挥着关键作用。尽管已经提出了许多客观指标来近似人类感知，但感知质量在很大程度上依赖于观看条件和显示特性。环境光照、显示亮度和分辨率等因素显著影响失真可见性。在本研究中，我们探讨了移动设备上的多屏质量评估问题，因为这一领域仍然相对欠缺研究。我们引入了第一个在300多款Android设备上收集的大规模主观数据集，并附带了关于观看条件和显示属性的元数据。我们提出了一种聚合评分提取和VQA模型适应于设备特定质量估计的策略。我们的结果表明，结合设备和上下文信息能够实现更准确和灵活的质量预测，为流媒体服务的细粒度优化提供了新机会。最终，本研究推动了感知质量模型的发展，弥合了实验室评估与现实世界媒体消费多样条件之间的差距。我们已将数据集和代码发布在 https://videoprocessing.github.io/device-viewing-conditions。

View on arXiv Download PDF AI Translation

cs.CV / 246 / 2607.04653

Enhancing Video Physical Consistency via Role-aware Joint Training and Modality-decoupled Denoising

通过角色感知联合训练和模态解耦去噪增强视频物理一致性

Zheng, Guangting, Chen, Haojing, Li, Hao, Zhang, Jingtao, Yang, Zhen, Jia, Xiaosong, Yang, Xue, Zhang, Shaofeng, Zhang, Yanyong

Abstract

While modern video diffusion models excel in visual fidelity, maintaining long-range physical consistency remains a formidable challenge. Conventional pixel-reconstruction objectives mainly focus on appearance details and often fail to capture the underlying dynamics of a scene. To mitigate this, recent efforts have integrated auxiliary modalities (e.g., optical flow) to introduce physics priors via joint training with video appearance. However, these methods have three main limitations: (1) they do not distinguish the different motion patterns of different entity types; (2) joint modeling of visual and auxiliary modalities can cause capacity conflicts and weaken the pretrained visual prior; and (3) auxiliary modalities may accumulate errors during inference. To address these issues, we propose \textbf{VPT}, a fine-tuning framework for improving physical consistency in video diffusion models. VPT introduces a role-aware signal that groups entities into agents, controlled objects, passive objects, and background, so that different physical roles can be modeled more clearly. We further propose a modality-decoupled denoising strategy, where the visual and auxiliary channels are assigned independent noise levels. Together with a loss-weight decay strategy, this design makes auxiliary modalities serve as soft constraints rather than strong dependencies, mitigating recursive prediction errors during inference. We also introduce cross-step auto-guidance to further strengthen physical dynamics. Experiments show that VPT improves physical consistency while preserving visual quality, achieving relative gains of 39.4\% in SA and 17.9\% in PC on VideoPhy benchmark over Wan2.1-T2V-1.3B, and consistent improvements on VideoPhy-2 benchmark. The project page is available at https://tom-zgt.github.io/VPT.

Chinese Translation

尽管现代视频扩散模型在视觉保真度方面表现出色，但保持长距离的物理一致性仍然是一个巨大的挑战。传统的像素重建目标主要关注外观细节，往往无法捕捉场景的潜在动态。为了解决这个问题，最近的研究努力通过与视频外观的联合训练整合辅助模态（例如，光流），以引入物理先验。然而，这些方法存在三个主要局限性：（1）它们未能区分不同实体类型的不同运动模式；（2）视觉模态和辅助模态的联合建模可能导致容量冲突，削弱预训练视觉先验；（3）辅助模态在推理过程中可能会累积误差。为了解决这些问题，我们提出了 extbf{VPT}，一个用于改善视频扩散模型物理一致性的微调框架。VPT引入了一种角色感知信号，将实体分组为代理、受控对象、被动对象和背景，从而使不同的物理角色能够更清晰地建模。我们进一步提出了一种模态解耦去噪策略，其中视觉和辅助通道被分配独立的噪声水平。结合损失权重衰减策略，这一设计使得辅助模态作为软约束而非强依赖，从而减轻推理过程中的递归预测误差。我们还引入了跨步骤自我引导，以进一步增强物理动态。实验表明，VPT在保持视觉质量的同时改善了物理一致性，在VideoPhy基准上相较于Wan2.1-T2V-1.3B实现了39.4 ext{%}的SA和17.9 ext{%}的PC的相对提升，并在VideoPhy-2基准上取得了一致的改进。项目页面可访问 https://tom-zgt.github.io/VPT。

View on arXiv Download PDF AI Translation

cs.CV / 247 / 2607.04661

Targeted Structure Completion for Sparse-View 3D Reconstruction in Autonomous Driving

自主驾驶中稀疏视图3D重建的目标结构补全

Wang, Guoqing, Tang, Pin, Ren, Xiangxuan, Hou, Liping, Ma, Chao

Abstract

Reconstructing 3D scene structures from sparse, low-overlap observations remains a fundamental challenge in autonomous driving. Recent state-of-the-art frameworks achieve promising results by incorporating voxel-based Gaussians, but incur substantial computational redundancy due to a uniform volumetric processing strategy. To bridge the gap between the efficiency of pixel-based Gaussian methods and the structural completeness of voxel-based Gaussian approaches, we propose FocusGS, a simple yet effective framework that shifts the paradigm from global densification to targeted structural completion. Our central insight is that structural completion should be decoupled from deterministic regions, with computation concentrated exclusively on areas exhibiting geometric ambiguity. Specifically, FocusGS addresses the localization challenge by deriving a 3D Geometric Ambiguity Manifold to accurately isolate localized areas prone to occlusion and high geometric uncertainty. To overcome the subsequent manifold completion challenge, we design a lightweight targeted structure completion module that selectively instantiates and optimizes continuous Gaussian queries strictly within this unstructured, sparse topological subspace. Extensive experiments demonstrate that FocusGS achieves a superior efficiency-quality trade-off, advancing state-of-the-art performance on driving-centric benchmarks while naturally reducing the total number of Gaussians by ~74% and decreasing rendering time by ~34%.

Chinese Translation

从稀疏、低重叠的观测中重建3D场景结构仍然是自主驾驶中的一项基本挑战。最近的最先进框架通过结合基于体素的高斯方法取得了令人鼓舞的结果，但由于采用了统一的体积处理策略，导致了显著的计算冗余。为了弥合基于像素的高斯方法的效率与基于体素的高斯方法的结构完整性之间的差距，我们提出了FocusGS，一个简单而有效的框架，它将范式从全局密集化转变为目标结构补全。我们的核心见解是，结构补全应与确定性区域解耦，计算应专注于表现出几何模糊的区域。具体而言，FocusGS通过推导3D几何模糊流形来解决定位挑战，从而准确隔离易受遮挡和高几何不确定性影响的局部区域。为了克服随后的流形补全挑战，我们设计了一个轻量级的目标结构补全模块，该模块在这个无结构的稀疏拓扑子空间内选择性地实例化和优化连续的高斯查询。大量实验表明，FocusGS在效率与质量的权衡上表现优越，在以驾驶为中心的基准测试中推动了最先进的性能，同时自然地将高斯数量减少了约74%，并将渲染时间减少了约34%。

View on arXiv Download PDF AI Translation

cs.CV / 248 / 2607.04665

DiCE-CIR: Direct Composition Learning for Efficient Zero-Shot Composed Image Retrieval

DiCE-CIR：高效零-shot组合图像检索的直接组合学习

Na, Gwang-Ho, Kim, Ho-Joong, Lee, Seong-Whan

Abstract

Zero-shot composed image retrieval (ZS-CIR) aims to retrieve a target image from a multimodal query consisting of a reference image and an edit text describing the desired modification. Recent ZS-CIR studies have relied on projection-based methods that map a reference image into pseudo-word tokens in the text embedding space. However, such methods require additional projection and re-encoding steps, increasing training complexity, reducing efficiency, and introducing a discrepancy between training and inference. In this paper, we propose DiCE-CIR, a direct composition learning method that predicts composed query representations by directly composing a reference image and an edit text. To enable scalable training without manually annotated triplets, we automatically construct compositional training samples from large-scale image-caption pairs using a large language model. Based on these samples, we train a lightweight composition module with objectives that promote alignment with the target, edit-consistent semantic transformation, and retrieval discriminability. We conduct extensive experiments on ZS-CIR benchmarks and show that DiCE-CIR achieves state-of-the-art performance on CIRCO and competitive performance on CIRR while maintaining high computational efficiency.

Chinese Translation

零-shot组合图像检索（ZS-CIR）旨在从由参考图像和描述所需修改的编辑文本组成的多模态查询中检索目标图像。近期的ZS-CIR研究依赖于基于投影的方法，将参考图像映射到文本嵌入空间中的伪词令牌。然而，这些方法需要额外的投影和重新编码步骤，增加了训练复杂性，降低了效率，并引入了训练与推理之间的差异。在本文中，我们提出了DiCE-CIR，一种直接组合学习方法，通过直接组合参考图像和编辑文本来预测组合查询表示。为了实现可扩展的训练而无需手动标注的三元组，我们利用大型语言模型从大规模图像-标题对中自动构建组合训练样本。基于这些样本，我们训练了一个轻量级组合模块，其目标是促进与目标的对齐、编辑一致的语义转换和检索可区分性。我们在ZS-CIR基准上进行了广泛的实验，结果表明，DiCE-CIR在CIRCO上实现了最先进的性能，并在CIRR上表现出竞争力，同时保持了高计算效率。

View on arXiv Download PDF AI Translation

cs.CV / 249 / 2607.04673

GlaKG: A Biomarker-Centric Fundus Knowledge Graph for Explainable Glaucoma Diagnosis and Risk Assessment

GlaKG：一种以生物标志物为中心的视网膜知识图谱，用于可解释的青光眼诊断和风险评估

Huang, Cheng, Zhang, Jia, Jiang, Yi, Liu, Yang, Kooner, Karanjit, Liu, Yadi, Lee, Tsengdar, Xie, Yang, Shi, Wenqi, Xiao, Guanghua

Abstract

Glaucoma is a leading cause of irreversible blindness worldwide, yet most automated diagnosis systems rely on opaque deep-learning models that offer little clinical interpretability. We present GlaKG, a biomarker-centric fundus knowledge graph that integrates structural biomarkers, clinically grounded rules, and image features to produce traceable reasoning for glaucoma diagnosis and risk stratification. GlaKG encodes six entity types (Fundus Image, Optic Disc, Neural Rim, Pathology, Diagnosis, Risk Level), eight relation types, and 11 clinically validated rules into a unified graph, so that every prediction is accompanied by an explicit reasoning chain linking biomarker evidence to activated clinical rules. To keep knowledge-based reasoning strictly separate from label information, we adopt a post-processing fusion framework that combines ResNet50 image embeddings with a normalized KG reasoning-chain score via a tunable weight alpha, with all fitting confined to the training split. On a publicly available, AI-annotated fundus dataset, GlaKG reaches F1 = 0.9953 for binary glaucoma classification and 0.930 accuracy with 0.922 weighted F1 for four-class risk stratification; we report openly that the dataset's biomarker annotations are highly label-correlated, and therefore frame these figures as an upper bound attainable with clean structured biomarkers rather than as leakage-free image-only performance. Feature-importance analysis shows KG-derived and biomarker features contributing near-equally (51.1% vs. 48.9%), and the reasoning chain flags borderline cases by exposing low chain scores rather than failing silently. GlaKG's central contribution is therefore a clinically auditable reasoning framework that complements raw predictive performance by explicitly exposing the biomarker evidence and rule activations behind each decision.

Chinese Translation

青光眼是全球不可逆失明的主要原因，但大多数自动诊断系统依赖于不透明的深度学习模型，缺乏临床可解释性。我们提出了GlaKG，一种以生物标志物为中心的视网膜知识图谱，整合了结构性生物标志物、临床基础规则和图像特征，以提供可追溯的青光眼诊断和风险分层推理。GlaKG编码了六种实体类型（视网膜图像、视神经盘、神经边缘、病理、诊断、风险等级）、八种关系类型和11条经过临床验证的规则，形成一个统一的图谱，使得每个预测都伴随着一条明确的推理链，将生物标志物证据与激活的临床规则联系起来。为了将基于知识的推理与标签信息严格区分开，我们采用了一种后处理融合框架，通过可调权重α将ResNet50图像嵌入与归一化的KG推理链分数结合，所有拟合均限于训练集。在一个公开可用的、经过人工智能注释的视网膜数据集中，GlaKG在二分类青光眼分类中达到了F1 = 0.9953，并在四类风险分层中实现了0.930的准确率和0.922的加权F1；我们公开报告该数据集的生物标志物注释高度相关，因此将这些数字视为可通过干净结构化生物标志物实现的上限，而非无泄漏的仅图像性能。特征重要性分析显示，KG派生特征和生物标志物特征的贡献几乎相等（51.1%对48.9%），推理链通过暴露低链分数而非默默失败来标记边界案例。因此，GlaKG的核心贡献是一个临床可审计的推理框架，通过明确揭示每个决策背后的生物标志物证据和规则激活，补充了原始预测性能。

View on arXiv Download PDF AI Translation

cs.CV / 250 / 2607.04674

Video Generation Models Are Inherent Lighting Estimators

视频生成模型是内在的光照估计器

Cai, Ziqi, Weng, Shuchen, Liu, Kaiqi, Wang, Zifeng, Zhang, Zhiquan, Teng, Minggui, Jiang, Han, Shi, Boxin

Abstract

Recovering dynamic environment maps from a single in-the-wild video is crucial for photorealistic rendering, yet remains a challenge. Recent video generation models can produce photorealistic scenes with complex lighting, possessing an inherent understanding of lighting. In this paper, we introduce V-LITE (Video generation models are inherent lighting estimators), a framework that unlocks this internal knowledge by reframing lighting estimation as a guided video inpainting task. Inspired by VFX industry practices, we insert a synthetic chrome ball into the scene to compel the model to generate physically plausible reflections from the surrounding spatio-temporal context. To bridge the gap from LDR-native models to the HDR domain, we design an HDR-aware VAE and employ an efficient LoRA-based fine-tuning strategy. We then construct a mixed dataset comprising high-fidelity HDR images to provide realistic HDR priors, and in-the-wild HDR videos to provide dynamic spatio-temporal context. Extensive experiments demonstrate that V-LITE produces temporally coherent HDR environment maps, revealing that modern video diffusion models are not merely synthesizers but also powerful, inherently capable estimators of physical scene lighting.

Chinese Translation

从单个真实环境视频中恢复动态环境图对于照片级真实感渲染至关重要，但仍然是一项挑战。最近的视频生成模型能够生成具有复杂光照的照片级真实场景，具备对光照的内在理解。本文介绍了V-LITE（视频生成模型是内在的光照估计器），这是一个通过将光照估计重新构建为引导视频修复任务来解锁这一内部知识的框架。受到视觉特效行业实践的启发，我们在场景中插入一个合成的铬球，以迫使模型从周围时空上下文中生成物理上合理的反射。为了弥合LDR（低动态范围）原生模型与HDR（高动态范围）领域之间的差距，我们设计了一个HDR感知的变分自编码器（VAE），并采用了一种高效的基于LoRA的微调策略。随后，我们构建了一个混合数据集，包括高保真度的HDR图像，以提供真实的HDR先验，以及真实环境中的HDR视频，以提供动态时空上下文。大量实验表明，V-LITE生成了时间上连贯的HDR环境图，揭示了现代视频扩散模型不仅仅是合成器，更是强大的、内在能够估计物理场景光照的估计器。

View on arXiv Download PDF AI Translation

cs.CV / 251 / 2607.04675

ICME 2026 Grand Challenge on Cross-Scenario Defect Detection and Fine-Grained Severity Grading for High-Precision Manufacturing

ICME 2026跨场景缺陷检测与高精度制造细粒度严重性分级大挑战

Sun, Wei, Zhang, Weixia, Cao, Linhan, Lu, Mingkai, Min, Xiongkuo, Zhang, Xiaoping, Callet, Patrick Le, Zhai, Guangtao, Chen, Hongxing, Wu, Wenqi, Hu, Zhenhao, Lin, Shanshan, Huang, Guanjie, Xie, Kai, Xin, Rui, Zhao, Zilong, Cong, Runmin, Li, Ningjing, Ma, Siqi, Ong, Yi Jin, Zhou, Tianfei, Wang, Shunzhou, Chen, Zhiyang, Fang, Hao, Zhang, Chen, Tang, Tze-Hsiang, Li, Dikai, Wu, Xianjin, Sharma, Avinash Kumar, Wang, Zhaoyang, Chen, Haiyong, Su, Binyi, Shahariar, Atik

Abstract

This paper presents the IEEE International Conference on Multimedia and Expo (ICME) 2026 Grand Challenge on Cross-Scenario Defect Detection and Fine-Grained Severity Grading for High-Precision Manufacturing. The challenge is motivated by two key limitations of existing industrial defect inspection systems: (1) current deep learning-based methods often suffer significant performance degradation when deployed in unseen production scenarios, and (2) most benchmarks neglect severity-aware assessment, which is critical for risk control and yield optimization. To address these limitations, we design two complementary tracks: Track 1 (Cross-Scenario Defect Detection) targets accurate defect detection, localization, and classification across diverse unseen production environments; Track 2 (Fine-Grained Severity Grading) requires assigning each detected defect an industry-standard severity level, including Acceptable, Marginal NG, NG, and Gross NG. We construct a large-scale industrial dataset of high-resolution microscopic images spanning seven representative defect categories, comprising over 3,800 images with pixel-level instance annotations for Track 1 and over 2,600 images with severity-grade labels for Track 2. The challenge attracted 86 registered participants with 130 submissions; during the final testing phase, 21 teams submitted results and 12 teams provided models with technical reports. The resulting benchmark, together with the diverse and effective solutions contributed by participating teams, sets a new standard for industrial defect analysis research.

Chinese Translation

本文介绍了IEEE国际多媒体与博览会（ICME）2026跨场景缺陷检测与高精度制造细粒度严重性分级大挑战。该挑战的动机源于现有工业缺陷检测系统的两个主要局限性：（1）当前基于深度学习的方法在未见过的生产场景中部署时，往往会遭遇显著的性能下降；（2）大多数基准测试忽视了对严重性意识的评估，而这对于风险控制和产量优化至关重要。为了解决这些局限性，我们设计了两个互补的赛道：赛道1（跨场景缺陷检测）旨在针对多样化的未见生产环境进行准确的缺陷检测、定位和分类；赛道2（细粒度严重性分级）要求为每个检测到的缺陷分配一个行业标准的严重性等级，包括可接受、边际不合格、不合格和严重不合格。我们构建了一个大规模工业数据集，包含七个代表性缺陷类别的高分辨率显微图像，共计超过3,800幅图像，赛道1提供了像素级实例标注，赛道2则提供了超过2,600幅图像的严重性等级标签。该挑战吸引了86名注册参与者提交了130份方案；在最终测试阶段，21个团队提交了结果，12个团队提供了带有技术报告的模型。最终的基准测试以及参与团队贡献的多样化和有效的解决方案，为工业缺陷分析研究设定了新的标准。

View on arXiv Download PDF AI Translation

cs.CV / 252 / 2607.04677

AnyStyle: A Single LoRA is Sufficient for Image-Guided Style Transfer

AnyStyle：单个 LoRA 足以实现图像引导的风格迁移

Lai, Yongwen, Wang, Chaoqun

Abstract

Image-guided style transfer aims to apply the artistic characteristics of a style image to a content image while preserving its semantic structure and layout. Despite advances in diffusion-based methods, existing approaches often face challenges in disentangling content and style, particularly when independently optimized adapters are naively combined, causing conflicts between adapters and limiting controllability over the content-style balance in inference. We further demonstrate that training-free structural guidance directly derived from the content image through the internal attention of pre-trained model outperforms a dedicated content LoRA adapter in terms of structural fidelity and computational efficiency. Building on these observations, we propose AnyStyle, a streamlined framework for image-guided style transfer. The framework adopts a unified single-adapter paradigm for coherent style capture from the style image and incorporates training-free structural guidance from the content image, thus avoiding complex entanglement between multiple adapters and improving controllability and stability. Extensive experiments show that our method delivers competitive quantitative performance and significantly improved perceptual quality. Code is available at https://github.com/Yvan1001/AnyStyle.

Chinese Translation

图像引导的风格迁移旨在将风格图像的艺术特征应用于内容图像，同时保持其语义结构和布局。尽管基于扩散的方法取得了进展，现有的方法在内容与风格的解耦方面仍面临挑战，尤其是在独立优化的适配器被简单组合时，导致适配器之间的冲突，并限制了推理过程中对内容与风格平衡的可控性。我们进一步证明，通过预训练模型的内部注意力直接从内容图像中导出的无训练结构引导在结构保真度和计算效率方面优于专用的内容 LoRA 适配器。基于这些观察，我们提出了 AnyStyle，一个简化的图像引导风格迁移框架。该框架采用统一的单适配器范式，从风格图像中一致地捕获风格，并结合来自内容图像的无训练结构引导，从而避免多个适配器之间的复杂纠缠，提高可控性和稳定性。大量实验表明，我们的方法在定量性能上具有竞争力，并显著提高了感知质量。代码可在 https://github.com/Yvan1001/AnyStyle 获取。

View on arXiv Download PDF AI Translation

cs.CV / 253 / 2607.04683

Does It Fail to See or Fail to Know? Attributing Errors in Vision-Language Models

是未能看见还是未能了解？归因于视觉-语言模型中的错误

Vo, Khang Nhat Hoang, Vazhentsev, Artem, Shelmanov, Artem, Baldwin, Timothy, Kementchedjhieva, Yova

Abstract

Vision-language models (VLMs) perform well on visual question answering with high-quality images but struggle when questions require knowledge beyond what is clearly and directly visible. In such settings, uncertainty quantification should not only indicate whether the model is likely to fail but also diagnose why it is uncertain, across dimensions such as perception, entity recognition, and knowledge retrieval. While prior work has focused on individual failure modes in isolation or treated incorrect answers as monolithic failures, we propose a unified framework for disentangling these failure modes and investigate whether pre-generation signals can predict these failure sources. Across a range of datasets and model families, we find a consistent pattern in VLM errors: some failures arise from visual or recognition bottlenecks, while others persist after the relevant entity is identified. Our main finding is that these failure sources can be predicted before decoding: recognition-related failures are best captured by visual-token representations, while failures that remain after recognition are better captured by prompt-conditioned hidden states. This pre-generation signal enables efficient failure-source prediction before the model produces an answer, allowing uncertain cases to be routed to targeted interventions such as image repair, entity recognition support, or external retrieval.

Chinese Translation

视觉-语言模型（VLMs）在高质量图像的视觉问答任务中表现良好，但在问题需要超出明显和直接可见内容的知识时则表现不佳。在这种情况下，不确定性量化不仅应指示模型可能失败的概率，还应诊断其不确定性的原因，包括感知、实体识别和知识检索等维度。尽管先前的研究集中于孤立地分析个别失败模式或将错误答案视为单一的失败，但我们提出了一个统一框架，用于解开这些失败模式，并研究生成前信号是否可以预测这些失败来源。在一系列数据集和模型家族中，我们发现VLM错误中存在一致的模式：一些失败源于视觉或识别瓶颈，而另一些则在相关实体被识别后仍然存在。我们的主要发现是，这些失败来源可以在解码之前进行预测：与识别相关的失败最好通过视觉标记表示来捕捉，而在识别后仍然存在的失败则更适合通过提示条件的隐藏状态来捕捉。这种生成前信号使得在模型产生答案之前能够有效预测失败来源，从而将不确定的案例引导至针对性的干预措施，如图像修复、实体识别支持或外部检索。

View on arXiv Download PDF AI Translation

cs.CV / 254 / 2607.04684

TubeLite: Lightweight Multi-Actor Spatio-Temporal Action Detection

TubeLite：轻量级多参与者时空动作检测

Soltaninezhad, Ali, Cote, Melissa, Espinosa, Alejandro Rico, Marques, Tunai Porto, Albu, Alexandra Branzan

Abstract

Spatio-temporal action detection in videos requires jointly localizing actors in space and identifying action boundaries over time. A common challenge is constructing temporally stable action tubes, as frame-level detectors often suffer from jitter, fragmentation, and imprecise temporal localization. Many recent approaches address this by introducing heavy spatio-temporal transformers or optical-flow-based pipelines, leading to high computational cost and limited scalability. We propose TubeLite, a lightweight framework for spatio-temporal action detection that focuses on stable tube construction and boundary-aware temporal modeling. TubeLite represents each actor as a tube, defined as a sequence of bounding boxes associated with a single actor over time, and explicitly enforces temporal consistency at both the spatial and semantic levels. The method combines low-jitter actor detection, Gaussian-weighted actor feature extraction, efficient short-term temporal propagation, and a boundary-focused temporal prediction head, while avoiding optical flow and large-scale temporal attention. Despite its compact design, TubeLite achieves strong video-level localization performance. It improves [email protected] by 4.5 and 7.1 percentage points over the best compared method on the MultiSports and UCF101-24 datasets, respectively, with substantially fewer parameters and floating-point operations than transformer-based alternatives, demonstrating that effective spatio-temporal action detection can be obtained through principled, lightweight temporal modeling.

Chinese Translation

视频中的时空动作检测需要在空间上联合定位参与者，并识别时间上的动作边界。一个常见的挑战是构建时间上稳定的动作管道，因为帧级检测器通常会遭遇抖动、碎片化和不精确的时间定位。许多近期的方法通过引入重型时空变换器或基于光流的管道来解决这一问题，这导致了高计算成本和有限的可扩展性。我们提出了TubeLite，一个轻量级的时空动作检测框架，专注于稳定的管道构建和边界感知的时间建模。TubeLite将每个参与者表示为一个管道，定义为与单个参与者在时间上关联的一系列边界框，并在空间和语义层面上明确强制时间一致性。该方法结合了低抖动的参与者检测、高斯加权的参与者特征提取、高效的短期时间传播和以边界为中心的时间预测头，同时避免了光流和大规模时间注意力。尽管设计紧凑，TubeLite在视频级定位性能上表现出色。在MultiSports和UCF101-24数据集上，[email protected]分别提高了4.5和7.1个百分点，相比于最佳对比方法，参数和浮点运算显著减少，证明了通过原则性、轻量级的时间建模可以实现有效的时空动作检测。

View on arXiv Download PDF AI Translation

cs.CV / 255 / 2607.04691

From Open Loop to Closed Loop: A Test-Time Iterative Optimization Framework for Reference-Consistent Image Generation

从开放环路到闭环：一种用于参考一致性图像生成的测试时迭代优化框架

Zhao, Baixuan, Zhang, Xinyu, Zheng, Huayu, Liu, Shuaicheng, Min, Xiongkuo, Zhai, Guangtao, Liu, Xiaohong

Abstract

While controllable image generation has made significant strides by incorporating visual reference conditions, existing methods predominantly operate as open-loop systems. They inject control signals in a strictly feed-forward manner, failing to guarantee strict fidelity to the reference due to the absence of active feedback and error correction mechanisms. To address this fundamental limitation, we propose a novel test-time iterative optimization framework that reformulates reference-consistent generation as a closed-loop dynamic tracking problem. By treating the pre-trained generative model as a control plant, our framework employs a sensor-controller architecture driven by a modified Proportional-Integral-Derivative (PID) algorithm. This mechanism iteratively optimizes the latent control signals at test time based on the sensed discrepancy between the generated output and the reference target. Notably, this approach is entirely training-free, model-agnostic, and integrates seamlessly around existing diffusion pipelines. Extensive evaluations across ID-preserving, pose-controlled, and depth-controlled generation tasks validate the universality of our method. Empirical results demonstrate improvements over computation-matched open-loop baselines, achieving relative performance gains of up to 25.36\% for facial similarity, alongside spatial error reductions of up to 27.71\% for pose alignment and 28.50\% for depth consistency. More broadly, this work offers a new conceptual perspective: it demonstrates that controllable generation can be effectively managed as a dynamic feedback system, bringing the rigorous principles of classical control theory into the optimization of generative models. Code is available at https://github.com/zzdrill/From-Open-Loop-to-Closed-Loop.

Chinese Translation

尽管可控图像生成通过引入视觉参考条件取得了显著进展，但现有方法主要作为开放环路系统运行。它们以严格的前馈方式注入控制信号，由于缺乏主动反馈和误差修正机制，无法保证对参考的严格忠实性。为了解决这一基本限制，我们提出了一种新颖的测试时迭代优化框架，将参考一致性生成重新表述为一个闭环动态跟踪问题。通过将预训练的生成模型视为控制对象，我们的框架采用了由修改后的比例-积分-微分（PID）算法驱动的传感器-控制器架构。该机制在测试时基于生成输出与参考目标之间感知到的差异，迭代优化潜在控制信号。值得注意的是，这种方法完全无训练、与模型无关，并且能够与现有的扩散管道无缝集成。在保持ID、姿态控制和深度控制生成任务的广泛评估中，验证了我们方法的普适性。实证结果显示，相较于计算匹配的开放环路基线，我们的方法在面部相似性上实现了高达25.36%的相对性能提升，同时在姿态对齐和深度一致性上分别减少了高达27.71%和28.50%的空间误差。更广泛地说，这项工作提供了一个新的概念视角：它表明可控生成可以有效地作为一个动态反馈系统进行管理，将经典控制理论的严格原则引入生成模型的优化中。代码可在 https://github.com/zzdrill/From-Open-Loop-to-Closed-Loop 获取。

View on arXiv Download PDF AI Translation

cs.CV / 256 / 2607.04694

Solve the Missing First Step: Can VLMs Standardize Raw Heterogeneous Medical Data?

解决缺失的第一步：视觉语言模型能否标准化原始异构医疗数据？

Chen, Xin, Xu, Dongliang, Zhu, Cunhao, Luo, Xudong, Lyu, Haoyang, Sun, Xiaoxiao, Yeung-Levy, Serena, Yao, Yue

Abstract

As vision-language models (VLMs) are increasingly applied to medical AI, existing benchmarks mainly focus on evaluating their diagnosis ability over given medical images and texts, implicitly assuming that standardized medical images, texts or question-answer pairs are already prepared. However, this assumption does not hold when we apply VLMs in real clinical practice, where medical data is often raw, heterogeneous, and fragmented across different sources. In this paper, we study this missing step, i.e., raw medical data standardization. Specifically, models are given raw dataset folders and evaluated on their ability to identify source formats, convert raw medical images into VLM-compatible visual inputs, extract relevant textual information, and organize the results into structured image-text pairs. To construct this Medical Data Standardization Benchmark (MDS-Bench), we manually annotate 1,939 raw medical data standardization tasks covering diverse clinical practice, radiology modalities, annotation formats, and directory layouts. Extensive experiments show that even the best performing VLMs, i.e., Gemini 3 Flash, achieve only 48.6% end-to-end success rate. Our research highlights raw medical data standardization as a critical bottleneck for medical AI diagnosis in real practice.

Chinese Translation

随着视觉语言模型（VLMs）在医疗人工智能中的应用日益增多，现有基准主要集中于评估它们在给定医疗图像和文本上的诊断能力，隐含地假设已经准备好标准化的医疗图像、文本或问答对。然而，当我们在真实临床实践中应用VLMs时，这一假设并不成立，因为医疗数据通常是原始的、异构的，并且在不同来源之间是碎片化的。本文研究了这一缺失的步骤，即原始医疗数据的标准化。具体而言，模型接收原始数据集文件夹，并评估其识别源格式、将原始医疗图像转换为VLM兼容的视觉输入、提取相关文本信息以及将结果组织成结构化的图像-文本对的能力。为了构建这一医疗数据标准化基准（MDS-Bench），我们手动标注了1,939个原始医疗数据标准化任务，涵盖了多样的临床实践、放射学模式、标注格式和目录布局。广泛的实验表明，即使是表现最佳的VLM，如Gemini 3 Flash，其端到端成功率也仅为48.6%。我们的研究强调了原始医疗数据标准化是医疗人工智能在实际诊断中的一个关键瓶颈。

View on arXiv Download PDF AI Translation

cs.CV / 257 / 2607.04696

Probe-EM: Targeted Neuron Tracing via Training-Free Semantic Verification

Probe-EM：通过无训练的语义验证进行目标神经元追踪

Jiang, Liuyun, Zhang, Yanchao, Guo, Jinyue, Chen, Chuanyue, Yan, Haiyang, Yuan, Ye, Liu, Jing, Han, Hua

Abstract

Establishing large-scale, high-resolution neural connectivity maps is fundamental to elucidating the structural basis of brain function. However, when processing terabyte- or petabyte-scale electron microscopy data, over-segmentation inherent in automated reconstruction algorithms remains a critical bottleneck, requiring extensive manual proofreading spanning person-years. To alleviate the heavy reliance on annotated data and the limited flexibility of conventional tracing methods, we propose a training-free, targeted neuron tracing framework. Specifically, we introduce a skeleton-guided Heuristic Spatial Search paradigm that leverages geometric priors to iteratively reconstruct neuronal morphologies through a probing-verification cycle. To achieve robust zero-shot semantic verification, we further develop a Dimension-Aware Semantic Verification strategy built upon the foundation model NeuroSAM 2. This strategy resolves intra-slice splits via Planar Ensemble Consensus and inter-slice splits via Axial Spatio-Temporal Propagation. Notably, we integrate the proposed workflow into the Neuroglancer visualization platform, enabling an interactive human-in-the-loop proofreading system. Experimental results demonstrate that the proposed method outperforms supervised baselines and reduces manual proofreading time by 33.4%. The source code is publicly available at https://github.com/HeadLiuYun/Probe-EM.

Chinese Translation

建立大规模、高分辨率的神经连接图谱对于阐明大脑功能的结构基础至关重要。然而，在处理TB（太字节）或PB（拍字节）级别的电子显微镜数据时，自动重建算法固有的过度分割仍然是一个关键瓶颈，这需要耗费数年人力进行广泛的人工校对。为了减轻对标注数据的重度依赖以及传统追踪方法的灵活性限制，我们提出了一种无训练的目标神经元追踪框架。具体而言，我们引入了一种基于骨架引导的启发式空间搜索范式，利用几何先验通过探测-验证循环迭代重建神经元形态。为了实现稳健的零-shot语义验证，我们进一步开发了一种基于基础模型NeuroSAM 2的维度感知语义验证策略。该策略通过平面集成共识解决切片内的分裂，通过轴向时空传播解决切片间的分裂。值得注意的是，我们将所提出的工作流程集成到Neuroglancer可视化平台中，实现了一个交互式的人机校对系统。实验结果表明，所提方法优于监督基线，并将人工校对时间减少了33.4%。源代码已公开，地址为 https://github.com/HeadLiuYun/Probe-EM。

View on arXiv Download PDF AI Translation

cs.CV / 258 / 2607.04711

Learning Probabilistic Prompt for Continual Learning

学习用于持续学习的概率性提示

Park, Hyekang, Lee, Sanghoon, Lee, Geon, Noh, Jongyoun, Ham, Bumsub

Abstract

Continual learning aims to progressively learn from a sequence of tasks, each containing a disjoint subset of classes, while preserving previously learned knowledge. Prompt-based continual learning methods propose to learn a small set of parameters, i.e., prompts, by associating them with a query feature of an input image. These methods optimize the prompts, attempting to represent diverse patterns of images. However, we have observed that existing prompt-based methods suffer from a prompt collapse problem, that is, the prompts tend to be highly similar to each other, thereby failing to capture the diverse data distributions in continual learning scenarios. To address this issue, we propose in this paper a novel prompt-based continual learning framework that captures diverse patterns of images across a sequence of tasks. To this end, we model each prompt as a probabilistic distribution and construct a mixture of these distributions, from which we sample diverse prompts. This enables our model to effectively capture highly diverse image distributions in the continual learning process. We also present a distribution regularization loss to prevent abrupt changes in the prompt distributions throughout the training process. We show extensive experimental results for continual learning on standard benchmarks, including ImageNet-R, CIFAR-100, and CUB-200, demonstrating the effectiveness of our framework.

Chinese Translation

持续学习旨在从一系列任务中逐步学习，每个任务包含一个不相交的类别子集，同时保留先前学习的知识。基于提示的持续学习方法通过将一小组参数（即提示）与输入图像的查询特征关联来学习这些参数。这些方法优化提示，试图表示多样的图像模式。然而，我们观察到现有的基于提示的方法存在提示崩溃问题，即提示之间往往高度相似，从而无法捕捉持续学习场景中的多样数据分布。为了解决这个问题，本文提出了一种新颖的基于提示的持续学习框架，能够捕捉一系列任务中图像的多样模式。为此，我们将每个提示建模为一个概率分布，并构建这些分布的混合，从中采样多样的提示。这使我们的模型能够有效地捕捉持续学习过程中的高度多样的图像分布。我们还提出了一种分布正则化损失，以防止在训练过程中提示分布的突然变化。我们在标准基准上（包括 ImageNet-R、CIFAR-100 和 CUB-200）展示了大量的持续学习实验结果，证明了我们框架的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 259 / 2607.04722

Reference-Induced Consensus for Selective Posed-Reference Visual Localization

基于参考诱导共识的选择性姿态参考视觉定位

Kang, Wonseok, Kim, Jaehyun, Lee, Jeongmin, Kim, Tae-Wan

Abstract

We present RIC-Loc (Reference-Induced Consensus localization), a scene-training-free posed-reference localizer that is SfM-point-map-free in its main estimator: it uses known reference poses, but not precomputed SfM 3D map points, query-to-map 2D-3D matches, or query-to-map PnP. A frozen VGGT pass predicts local camera poses, depth, and query-reference tracks for a query and selected references. Each reference induces one map-frame SE(3) query-pose hypothesis, robust consensus estimates the pose, and the preserved hypothesis structure yields two reliability scores: spatial dispersion and a track-conditioned covariance score. On the covariance-eligible set, the two scores are complementary for held-out, ground-truth-free failure detection across indoor, outdoor, and large-scale low-texture benchmarks: the joint policy is strongest in textured scenes and the covariance score in the low-texture regime, and the hypothesis-derived scores consistently outperform the standard retrieval-score gap and random rankings. Without per-scene training the consensus estimator remains accurate -- competitive with structure-based localization indoors and improving over a comparable feed-forward baseline -- giving an effective selective operating regime for posed-reference localization. Code is available at https://github.com/SNU-DLLAB/ric_loc.

Chinese Translation

我们提出了 RIC-Loc（基于参考诱导共识的定位），这是一种无需场景训练的姿态参考定位器，其主要估计器不依赖于 SfM 点图：它使用已知的参考姿态，但不依赖于预计算的 SfM 3D 点、查询到地图的 2D-3D 匹配或查询到地图的 PnP。一个冻结的 VGGT 过程预测局部相机姿态、深度和查询-参考轨迹。每个参考诱导一个地图帧 SE(3) 查询姿态假设，鲁棒共识估计姿态，保留的假设结构产生两个可靠性评分：空间离散度和轨迹条件协方差评分。在符合协方差条件的集合上，这两个评分在室内、室外和大规模低纹理基准测试中，对于无地面真实值的故障检测是互补的：在纹理丰富的场景中联合策略最强，而在低纹理环境中协方差评分表现最佳，假设导出的评分始终优于标准检索评分差距和随机排名。在没有逐场景训练的情况下，共识估计器保持准确性——在室内与基于结构的定位竞争，并且优于一个可比的前馈基线——为姿态参考定位提供了有效的选择性操作模式。代码可在 https://github.com/SNU-DLLAB/ric_loc 获取。

View on arXiv Download PDF AI Translation

cs.CV / 260 / 2607.04731

When Does High-CFG Diffusion Inversion Fail? A Controlled Study of Prompt--Latent Interactions

高CFG扩散反演何时失败？一个关于提示-潜在交互的控制研究

Zeng, Yan, Hosoya, Yusuke, Tran, Huyen T. T., Okatani, Takayuki

Abstract

Text-guided diffusion inversion is central to image editing, where an image is mapped to an initial latent and then edited by replaying the denoising process under a modified prompt. In practice, however, inversion is often performed with a lower classifier-free guidance(CFG) scale than the one used for generation or editing. This mismatch is empirically useful but leaves a basic question unresolved: when a target image is generated by a high-CFG trajectory, when can that trajectory actually be inverted? We study this question in a controlled generation--inversion--reconstruction setting, where the true initial latent and denoising trajectory are known. Using prompts taken from an existing diffusion-editing benchmark, we generate images under high CFG and reconstruct them with fixed-point inversion using the same prompt and guidance setting. The results reveal three types of prompt-level reconstruction behavior: easy prompts that reconstruct for most initial latents, hard prompts that fail for most initial latents, and intermediate prompts whose success depends on the prompt--latent pairing. To analyze the generation side, we define prompt pressure, a step-wise measure of how strongly CFG moves the denoising update away from the unconditional trajectory. Total pressure correlates with reconstruction quality and separates easy from hard prompts, but it does not explain the success or failure of intermediate prompt--latent pairs. Text-side analyses further show that the main visual subject and wording can change inversion difficulty. Finally, we evaluate a compact trajectory-consistency intervention that relaxes guidance only at locally unstable inverse steps. This diagnostic check improves reconstruction and Prompt-to-Prompt editing in our controlled setting, supporting the view that high-CFG inversion failure requires local, trajectory-aware analysis.

Chinese Translation

文本引导的扩散反演在图像编辑中至关重要，其中图像被映射到初始潜在空间，然后通过在修改后的提示下重放去噪过程进行编辑。然而，在实践中，反演通常是在低于生成或编辑时使用的分类器无关引导（CFG）尺度下进行的。这种不匹配在经验上是有用的，但留下了一个基本问题：当目标图像是通过高CFG轨迹生成时，何时该轨迹实际上可以被反演？我们在一个控制的生成-反演-重建环境中研究这个问题，在这个环境中，真实的初始潜在和去噪轨迹是已知的。我们使用来自现有扩散编辑基准的提示，在高CFG下生成图像，并使用相同的提示和引导设置进行固定点反演重建。结果揭示了三种提示级别的重建行为：对于大多数初始潜在有效的简单提示，对于大多数初始潜在失败的困难提示，以及其成功依赖于提示-潜在配对的中间提示。为了分析生成方面，我们定义了提示压力，这是一种逐步衡量CFG如何将去噪更新从无条件轨迹推离的强度的指标。总压力与重建质量相关，并区分简单提示和困难提示，但它并不能解释中间提示-潜在对的成功或失败。文本方面的分析进一步表明，主要视觉主题和措辞可以改变反演的难度。最后，我们评估了一种紧凑的轨迹一致性干预，该干预仅在局部不稳定的反向步骤中放宽引导。这种诊断检查改善了重建和我们控制环境中的提示到提示编辑，支持了高CFG反演失败需要局部、轨迹感知分析的观点。

View on arXiv Download PDF AI Translation

cs.CV / 261 / 2607.04732

SparseOcc++: Geometry-Aware Sparse Latent Representation for Semantic Occupancy Prediction

SparseOcc++：几何感知的稀疏潜在表示用于语义占用预测

Tang, Pin, Wang, Zhongdao, Wang, Guoqing, Ren, Xiangxuan, Ma, Chao

Abstract

Vision-based 3D semantic occupancy prediction is essential for autonomous driving, yet dense voxel representations waste computation on largely empty space, while BEV and TPV projections compromise fine-grained 3D structure. Fully sparse representations offer an attractive alternative, but existing methods, including SparseOcc, entangle scene completion with semantic prediction by indiscriminately propagating high-dimensional features into empty regions and applying voxel-wise classification. This creates excessive activations, computational overhead, and geometric ambiguity. We present SparseOcc++, a geometry-aware sparse framework that explicitly decouples scene completion from semantic segmentation. SparseOcc++ reformulates completion as signed-distance regression on sparse anchor voxels through a scene completion field (SCF). To model complex outdoor geometry robustly, it combines orthogonal decomposition with discretized distance learning. A geometry-guided propagation module then converts the SCF into a complete volumetric scene and restricts semantic segmentation to geometrically verified regions. Experiments establish new state of the art: SparseOcc++ improves IoU by 2.3 points and is 3.9x faster than SparseOcc on nuScenes, while achieving a 5.9x speedup over OccFormer on SemanticKITTI.

Chinese Translation

基于视觉的3D语义占用预测对于自动驾驶至关重要，但密集体素表示在大部分空白区域上浪费计算资源，而鸟瞰视图（BEV）和透视视图（TPV）投影则妨碍了细粒度的3D结构。完全稀疏的表示提供了一个有吸引力的替代方案，但现有方法（包括SparseOcc）通过不加区分地将高维特征传播到空白区域并应用体素级分类，将场景补全与语义预测纠缠在一起。这导致了过度激活、计算开销和几何模糊。我们提出了SparseOcc++，一个几何感知的稀疏框架，明确将场景补全与语义分割解耦。SparseOcc++通过场景补全场（SCF）将补全重新表述为在稀疏锚体素上的带符号距离回归。为了稳健地建模复杂的户外几何形状，它结合了正交分解与离散距离学习。然后，几何引导传播模块将SCF转换为完整的体积场景，并将语义分割限制在几何验证的区域。实验结果建立了新的最先进水平：SparseOcc++在nuScenes上提高了2.3个IoU点，并且比SparseOcc快3.9倍，同时在SemanticKITTI上实现了比OccFormer快5.9倍的速度提升。

View on arXiv Download PDF AI Translation

cs.CV / 262 / 2607.04740

DriftST: One-Step Generative Inference of Spatial Transcriptomics from H\&E Histology

DriftST：从H&E组织学中一步生成空间转录组学推断

Yang, Yuhang, Bu, Yonggan, Zhou, Shengyuan, Luo, Yiming, Zhang, Kai

Abstract

Spatial Transcriptomics (ST) measures gene expression while preserving spatial context, but its high cost and low throughput leave public datasets small. Inferring expression directly from widely available Hematoxylin and Eosin (H&E) stained histology offers a cost-effective alternative. However, existing approaches face several limitations: regression methods over-smooth toward the conditional mean, while generative methods are faithful but require slow multi-step inference; most methods treat genes as independent and equally important, ignoring inter-gene dependencies and heterogeneous gene informativeness; and most are tailored to a single resolution, either spot-level or cell-level. To address these issues, we propose DriftST, a unified framework for inferring spatially resolved gene expression from H&E images. DriftST builds on a Cellular Drifting generative model that learns a direct drift from a histology-conditioned source to the expression distribution, retaining generative expressiveness while enabling efficient one-step generation. To capture gene structure, we introduce the STransformer, which combines a co-expression attention module for inter-gene dependencies with a gene residual gate for differential gene importance. Operating on a generic gene-panel representation, DriftST applies directly to both spot-level and cell-level data in one framework, and extensive experiments across diverse tissues and platforms show that it achieves state-of-the-art performance at both resolutions.

Chinese Translation

空间转录组学（ST）在保留空间背景的同时测量基因表达，但其高成本和低通量使得公共数据集规模较小。从广泛可得的苏木精-伊红（H&E）染色组织学中直接推断表达提供了一种具有成本效益的替代方案。然而，现有方法面临几个限制：回归方法过于平滑，趋向于条件均值，而生成方法虽然忠实但需要缓慢的多步推断；大多数方法将基因视为独立且同等重要，忽视了基因间的依赖关系和基因信息的异质性；而且大多数方法针对单一分辨率，或者是点级别或细胞级别。为了解决这些问题，我们提出了DriftST，这是一个统一的框架，用于从H&E图像推断空间分辨的基因表达。DriftST基于一个细胞漂移生成模型，该模型学习从组织学条件源到表达分布的直接漂移，保留了生成的表现力，同时实现了高效的一步生成。为了捕捉基因结构，我们引入了STransformer，它结合了用于基因间依赖关系的共表达注意模块和用于差异基因重要性的基因残差门。DriftST在通用基因面板表示上直接应用于点级别和细胞级别数据，广泛的实验在多种组织和平台上表明，它在这两种分辨率下均实现了最先进的性能。

View on arXiv Download PDF AI Translation

cs.CV / 263 / 2607.04747

MergeSurv: Merging-Based Continual Learning for Survival Analysis on Whole-Slide Images

MergeSurv：基于合并的持续学习框架用于全切片图像的生存分析

Tran, Vu Minh, Bui, Doanh C., Nguyen, Maï K., Nguyen, Khang

Abstract

Survival analysis on Whole Slide Images (WSIs) is important in computational pathology for prognosis estimation and treatment planning. However, existing survival models are typically trained independently for each cancer cohort, making continual adaptation computationally expensive for gigapixel-scale WSIs. In this study, we propose MergeSurv, a merging-based continual learning framework for WSI survival analysis. A pathology vision-language foundation model is independently fine-tuned on each task, and the learned parameters are sequentially merged into a unified model without storing previous training data. We further investigate two inference strategies: One-for-All (OFA) and Voting-Expert Aggregation (VEA). Experiments on four TCGA cohorts demonstrate that MergeSurv outperforms naive fine-tuning as well as representative regularization-based and rehearsal-based continual learning methods, while effectively reducing catastrophic forgetting. The results suggest that model merging is a promising direction for scalable and privacy-preserving continual learning in computational pathology.

Chinese Translation

全切片图像（WSIs）的生存分析在计算病理学中对于预后评估和治疗规划至关重要。然而，现有的生存模型通常针对每个癌症队列独立训练，这使得对千兆像素级WSIs的持续适应在计算上非常昂贵。在本研究中，我们提出了MergeSurv，一个基于合并的持续学习框架用于WSI生存分析。病理视觉-语言基础模型在每个任务上独立微调，学习到的参数被顺序合并为一个统一模型，而无需存储之前的训练数据。我们进一步研究了两种推理策略：一对多（One-for-All, OFA）和投票专家聚合（Voting-Expert Aggregation, VEA）。在四个TCGA队列上的实验表明，MergeSurv在有效减少灾难性遗忘的同时，优于简单微调以及具有代表性的基于正则化和基于重演的持续学习方法。结果表明，模型合并是计算病理学中可扩展和保护隐私的持续学习的一个有前景的方向。

View on arXiv Download PDF AI Translation

cs.CV / 264 / 2607.04755

Continual Model Merging with Test-Time Adaptation for Whole-Slide Image Analysis

基于测试时适应的持续模型合并在全切片图像分析中的应用

Le, Duc-Thanh, Bui, Doanh C., Nguyen, Maï K., Nguyen, Khang

Abstract

Model merging offers a practical alternative to conventional continual learning by integrating independently fine-tuned models without retaining previous training data. Recent state-of-the-art model merging methods employ test-time adaptation (TTA-guided merging) to address distribution shifts by adjusting merging-related variables using unlabeled target data. However, these methods have primarily been studied in multi-task or single-target settings, and their behavior under sequential continual learning remains insufficiently understood. We present a benchmark study that maps this family of methods to rehearsal-free continual Whole Slide Image classification and evaluates them against traditional continual-learning approaches. Experiments on six TCGA cancer-subtyping cohorts cover CLASS-IL and TASK-IL scenarios, in-domain and out-of-domain evaluation, and different task orders. The results show that adapting model merging at test time can provide strong task-specific performance and improve retention of previously acquired knowledge without storing historical WSIs. Nevertheless, performance remains sensitive to task order and to the interaction between adaptation on the current distribution and accumulated knowledge. This benchmark identifies model merging with test-time adaptation as a promising direction for continual computational pathology and motivates future methods that balance adaptation to domain shift with explicit preservation of historical knowledge.

Chinese Translation

模型合并为传统的持续学习提供了一种实用的替代方案，通过整合独立微调的模型而无需保留先前的训练数据。最近的最先进的模型合并方法采用测试时适应（TTA-guided merging）来应对分布变化，通过使用未标记的目标数据调整与合并相关的变量。然而，这些方法主要在多任务或单目标设置中进行了研究，其在顺序持续学习下的表现仍然不够明确。我们提出了一项基准研究，将这一系列方法映射到无重演的持续全切片图像分类，并将其与传统的持续学习方法进行评估。在六个TCGA癌症亚型队列上的实验涵盖了CLASS-IL和TASK-IL场景、领域内和领域外评估以及不同的任务顺序。结果表明，在测试时适应模型合并可以提供强大的任务特定性能，并在不存储历史全切片图像的情况下改善对先前获得知识的保留。然而，性能仍然对任务顺序敏感，并且对当前分布的适应与累积知识之间的相互作用也很敏感。该基准研究将基于测试时适应的模型合并确定为持续计算病理学的一个有前景的方向，并激励未来的方法在适应领域变化与明确保留历史知识之间取得平衡。

View on arXiv Download PDF AI Translation

cs.CV / 265 / 2607.04761

DeGenseGS: Geometrically and Semantically Decoupled Surgical Scene Understanding in 4D Gaussian Splatting

DeGenseGS：在4D高斯喷溅中的几何和语义解耦外科场景理解

Wang, Yimo, Kang, Bin, Yang, Shuojue, Jin, Yueming

Abstract

Real-time, text-promptable 4D reconstruction is indispensable for autonomous surgical interaction. Severe misalignment between semantic meaning and physical anatomy still persists, largely because existing solutions integrate Vision-Language Models into deformable fields via a rigid coupling scheme that tightly binds semantic features to geometric warping. In this paper, we propose DeGenseGS, Geometrically and Semantically Decoupled Surgical Scene Understanding in 4D Gaussian Splatting, a novel framework that independently models semantic evolution and geometric deformation. Specifically, we propose a HexPlane-based spatiotemporal entanglement module that uses shared kinematic latents to synchronize semantic mutations with scene dynamics, while explicitly disentangling semantic updates from geometric deformation. To further ensure robustness against reconstruction artifacts, we devise a Rasterization-Native Semantic Extraction mechanism that infers semantics from topologically continuous feature maps. Additionally, we incorporate an angular-aligned optimization strategy that conforms to the native hyperspherical latent space, thereby preventing semantic distortion. Extensive evaluations on the CholecSeg8k and EndoVis18 datasets demonstrate that DeGenseGS achieves state-of-the-art performance. Our framework yields enhanced geometric completeness and robust semantic-anatomic alignment, enabling spatially continuous segmentation despite drastic tissue deformation and topological transitions.

Chinese Translation

实时、可通过文本提示的4D重建对自主外科交互至关重要。语义意义与物理解剖之间的严重不对齐问题依然存在，这在很大程度上是因为现有解决方案通过刚性耦合方案将视觉-语言模型集成到可变形场中，从而紧密绑定了语义特征与几何变形。在本文中，我们提出了DeGenseGS，几何和语义解耦的4D高斯喷溅外科场景理解，这是一个新颖的框架，独立建模语义演变和几何变形。具体而言，我们提出了一种基于HexPlane的时空纠缠模块，利用共享的运动潜变量将语义变异与场景动态同步，同时明确地将语义更新与几何变形解耦。为了进一步确保对重建伪影的鲁棒性，我们设计了一种栅格化原生语义提取机制，从拓扑连续的特征图中推断语义。此外，我们还结合了一种与原生超球面潜在空间相一致的角度对齐优化策略，从而防止语义失真。在CholecSeg8k和EndoVis18数据集上的广泛评估表明，DeGenseGS达到了最先进的性能。我们的框架实现了增强的几何完整性和稳健的语义-解剖对齐，尽管组织发生剧烈变形和拓扑转变，仍能实现空间连续的分割。

View on arXiv Download PDF AI Translation

cs.CV / 266 / 2607.04779

DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentation

DGSeg：用于推理分割的语义-空间引导预测的动态门控

Zeng, Ruizhe, Cao, Siyu, Zhang, Lu, Liu, Zhiyong

Abstract

Reasoning segmentation aims to predict pixel-wise masks for targets given complex language queries. Existing approaches leverage Multimodal Large Language Models (MLLMs) for vision-language reasoning and generate intermediate target cues (e.g., points or boxes) to guide a segmentation model. However, compressing rich reasoning into sparse cues often introduces ambiguity and noise, preventing these cues from accurately preserving the reasoning intent. While multiple complementary cues can enrich target information, existing methods typically feed them jointly into a single segmentation process, allowing ambiguous or erroneous cues to affect the entire prediction. Therefore, we propose DGSeg, a reasoning segmentation framework that learns to fuse predictions guided by semantic and spatial cues. Specifically, the MLLM jointly reasons about both target identity and spatial location, producing complementary semantic and spatial cues that are fed into separate segmentation branches. Their predictions are adaptively integrated by a lightweight dynamic gating module trained with relative branch-quality supervision to suppress noisy or conflicting regions. Extensive experiments demonstrate that DGSeg consistently outperforms strong baselines on multiple benchmarks and achieves 69.6% and 67.3% gIoU on the challenging ReasonSeg validation and test splits. Code is available at https://github.com/RZZeng/DGSeg.

Chinese Translation

推理分割旨在根据复杂的语言查询预测目标的逐像素掩膜。现有方法利用多模态大型语言模型（MLLMs）进行视觉-语言推理，并生成中间目标线索（例如，点或框）来引导分割模型。然而，将丰富的推理压缩为稀疏线索往往会引入模糊性和噪声，阻碍这些线索准确保留推理意图。尽管多种互补线索可以丰富目标信息，但现有方法通常将它们共同输入到单一的分割过程中，从而使模糊或错误的线索影响整个预测。因此，我们提出了DGSeg，一个学习融合由语义和空间线索引导的预测的推理分割框架。具体而言，MLLM同时推理目标的身份和空间位置，生成互补的语义和空间线索，这些线索被输入到独立的分割分支中。它们的预测通过一个轻量级的动态门控模块进行自适应整合，该模块通过相对分支质量监督进行训练，以抑制噪声或冲突区域。大量实验表明，DGSeg在多个基准测试中始终优于强基线，并在具有挑战性的ReasonSeg验证和测试集上分别达到了69.6%和67.3%的gIoU。代码可在https://github.com/RZZeng/DGSeg获取。

View on arXiv Download PDF AI Translation

cs.CV / 267 / 2607.04801

LILAC: Layer-Wise Independent LoRAs and Cascaded Conditioning for Multi-Concept Customization of Diffusion Models

LILAC：分层独立LoRA和级联条件用于扩散模型的多概念定制

Lupascu, Marian, Ripa, Sebastian, Trascau, Mihai, Georgescu, Mariana-Iuliana, Mironica, Ionut

Abstract

Personalizing text-to-image diffusion models to render several specific subjects in a coherent image remains challenging: the model must preserve each subject's identity while keeping the scene spatially and visually coherent. Methods that fuse independently trained concept adapters in a shared weight space (via federated averaging, gradient fusion, or orthogonality constraints) suffer from identity confusion and style bleeding and require joint retraining. In this work, we show that composing concepts as separate image layers, instead of merging their adapters in a shared weight space, avoids parameter-level interference. We introduce LILAC, a framework that composes independently trained low-rank adapters at inference time: each subject is conditioned on the frozen composite of previously placed subjects, with exactly one adapter active at a time, therefore identities never interfere at the parameter level. LILAC composes the adapters without any joint training, scales linearly with the number of concepts, and is backbone-agnostic. Under the Orthogonal Adaptation protocol, LILAC applied on Qwen-Image-Edit reaches an ArcFace detection rate of 0.861, while Orthogonal Adaptation reports 0.745 in its original setting. Adaptation reports 0.745 in its original setting. Code is available at https://github.com/marianlupascu/LILAC.

Chinese Translation

将文本到图像的扩散模型个性化，以在一致的图像中呈现多个特定主题仍然具有挑战性：模型必须在保持每个主题身份的同时，使场景在空间和视觉上保持一致。通过联合训练的方式（如联邦平均、梯度融合或正交约束）将独立训练的概念适配器融合在共享权重空间中，会导致身份混淆和风格渗透，并需要联合重训练。在本研究中，我们展示了将概念作为独立图像层进行组合，而不是在共享权重空间中合并它们的适配器，可以避免参数级别的干扰。我们提出了LILAC，一个在推理时组合独立训练的低秩适配器的框架：每个主题在先前放置的主题的冻结组合上进行条件处理，同时只有一个适配器处于激活状态，因此身份在参数级别上不会相互干扰。LILAC在没有任何联合训练的情况下组合适配器，随着概念数量的增加线性扩展，并且与骨干网络无关。在正交适应协议下，LILAC在Qwen-Image-Edit上达到0.861的ArcFace检测率，而正交适应在其原始设置中报告为0.745。代码可在https://github.com/marianlupascu/LILAC获取。

View on arXiv Download PDF AI Translation

cs.CV / 268 / 2607.04811

Hybrid Deep Learning for Traceability and Classification of Industrial Slate Tiles

用于工业石板瓦追溯和分类的混合深度学习

Antebi, Soren, Eickeler, Stefan, Halscheidt, Sandra, Schmitz, Rene, Muellers, Michael, Hecker, Dirk, Sifa, Rafet

Abstract

Applying deep learning to instance-aware reidentification of slate tiles and extraction site classification can improve production efficiency and quality control in the slate tile industry. These tasks are particularly important for handling natural materials where visual variability can make manual inspection costly and error-prone. We present a lightweight, hybrid deep learning approach that combines image matching and classification within a single framework. The system integrates a feature-matching branch based on XFeat with a MobileNetV3- based classification branch. The XFeat branch, combined with a LightGlue matching head, improves instance matching performance by +15.4% AUC. For classification, features from both backbones are shared and fused, resulting in a +10.9% accuracy improvement over a standard MobileNetV3 model. Our approach is evaluated on a newly created industrial dataset consisting of 2,610 slate tile images from six extraction sites. The results demonstrate the effectiveness of the proposed approach for object re-identification and classification in an industrial setting.

Chinese Translation

将深度学习应用于石板瓦的实例感知再识别和提取场所分类，可以提高石板瓦行业的生产效率和质量控制。这些任务对于处理自然材料尤为重要，因为视觉变异性可能使人工检查成本高昂且容易出错。我们提出了一种轻量级的混合深度学习方法，将图像匹配和分类结合在一个框架内。该系统集成了基于 XFeat 的特征匹配分支和基于 MobileNetV3 的分类分支。XFeat 分支结合 LightGlue 匹配头，使实例匹配性能提高了 +15.4% AUC。在分类方面，两个主干网络的特征被共享和融合，使得准确率较标准 MobileNetV3 模型提高了 +10.9%。我们在一个新创建的工业数据集上评估了我们的方法，该数据集包含来自六个提取场所的 2,610 张石板瓦图像。结果证明了所提方法在工业环境中进行物体再识别和分类的有效性。

View on arXiv Download PDF AI Translation

cs.CV / 269 / 2607.04812

TGRIP: A Text-Guided Approach to Vehicle Instance Prediction in Autonomous Driving

TGRIP：一种基于文本引导的自动驾驶车辆实例预测方法

Antunes-García, Miguel, Montiel-Marín, Santiago, Sánchez-García, Fabio, Gutiérrez-Moreno, Rodrigo, Barea, Rafael, Bergasa, Luis M.

Abstract

Bird's-Eye View (BEV) end-to-end instance prediction has emerged as a robust paradigm for autonomous driving perception, effectively mitigating the error propagation inherent in traditional modular pipelines. However, current state-of-the-art approaches rely predominantly on geometric supervision, such as occupancy regression and optical flow, effectively treating scene agents as generic moving obstacles. This absence of explicit semantic awareness imposes limitations on the capacity of the model to solve ambiguities in complex scenarios, particularly those where object-specific behavior is essential for accurate forecasting (e.g. overtaking, intersections). In this paper, we introduce Text-Guided Representation for Instance Prediction (TGRIP), a novel framework that bridges this gap by injecting rich semantic priors into the instance prediction loop. The proposed teacher-student pipeline employs Vision-Language Foundation Models to generate dense, semantic-enhanced BEV maps from multi-camera images. These maps serve as auxiliary supervision during training, guiding the network to learn spatio-temporal representations that are not only geometrically consistent but also semantically discriminative. To the best of our knowledge, this represents the first attempt to unify semantic guidance with the temporal task of future instance prediction. The experimental results demonstrate that TGRIP surpasses existing state-of-the-art models in nuScenes, validating the hypothesis that semantic enrichment is a fundamental element for robust, end-to-end motion prediction. Code is available on https://github.com/miguelag99/TGRIP.

Chinese Translation

鸟瞰视角（BEV）端到端实例预测已成为自动驾驶感知的一个强大范式，有效缓解了传统模块化流程中固有的误差传播。然而，目前的最先进方法主要依赖于几何监督，如占用回归和光流，实际上将场景中的代理视为通用的移动障碍物。这种缺乏明确语义意识的情况限制了模型在复杂场景中解决歧义的能力，尤其是在对象特定行为对准确预测至关重要的情况下（例如超车、交叉口）。在本文中，我们提出了用于实例预测的文本引导表示（TGRIP），这是一个新颖的框架，通过将丰富的语义先验注入实例预测循环来填补这一空白。所提出的教师-学生管道利用视觉-语言基础模型从多摄像头图像生成密集的、增强语义的BEV地图。这些地图在训练过程中作为辅助监督，指导网络学习不仅在几何上保持一致而且在语义上具有区分性的时空表示。据我们所知，这是首次尝试将语义引导与未来实例预测的时间任务统一起来。实验结果表明，TGRIP在nuScenes数据集上超越了现有的最先进模型，验证了语义增强是实现稳健的端到端运动预测的基本要素。代码可在 https://github.com/miguelag99/TGRIP 获取。

View on arXiv Download PDF AI Translation

cs.CV / 270 / 2607.04860

PAGE: Towards Practical Human-level Gaze Target Estimation

PAGE：迈向实用的人类水平注视目标估计

Ye, Zhoutong, Zhang, Chengwen, Cui, Zhaibin, Sun, Mingze, Liu, Jiaqi, Li, Xiangwu, Wan, Qingyang, Liu, Chang, Wang, Xutong, Gao, Huan-ang, Mei, Yu, Yu, Chun, Shi, Yuanchun

Abstract

Gaze target estimation, the task of predicting where a person is looking in a scene, is crucial to understanding human attention and intent. It is a challenging task that combines high-level understanding of global scene semantics and precise spatial reasoning using human appearance (e.g. pose, eye orientation). As a result, human-level performance remains elusive for existing models, limiting their practical application. To this end, we propose PaGE (Practical Gaze Estimator), a gaze estimation model that explicitly models the complex interaction between scene and head features. Using a PaGE model with a large ViT-H+ backbone as the teacher, we further distill student models with lighter backbones on a much larger and more diverse unlabeled dataset. The architectural improvements and novel training recipe allow PaGE to achieve state-of-the-art performance on several gaze estimation tasks, outperforming humans in 7 out of 9 metrics while reducing the human-AI gap by at least 60% in the remaining 2. The distilled student models retain most of the teacher's performance while being lightweight enough for practical deployment on robots and consumer devices. The code and model checkpoints are available at our project page.

Chinese Translation

注视目标估计是预测一个人在场景中注视位置的任务，这对于理解人类的注意力和意图至关重要。这是一项具有挑战性的任务，结合了对全局场景语义的高层理解和使用人类外观（例如姿势、眼睛方向）的精确空间推理。因此，现有模型在实现人类水平的性能方面仍然难以达到，限制了它们的实际应用。为此，我们提出了PaGE（Practical Gaze Estimator），这是一种明确建模场景与头部特征之间复杂交互的注视估计模型。我们使用具有大型ViT-H+骨干网络的PaGE模型作为教师，进一步在一个更大且更具多样性的未标记数据集上提炼出具有更轻骨干网络的学生模型。架构改进和新颖的训练方案使PaGE在多个注视估计任务上实现了最先进的性能，在9个指标中有7个超过了人类，同时在剩余2个指标中将人类与人工智能的差距减少了至少60%。提炼出的学生模型保留了大部分教师模型的性能，同时足够轻便，适合在机器人和消费设备上进行实际部署。代码和模型检查点可在我们的项目页面获取。

View on arXiv Download PDF AI Translation

cs.CV / 271 / 2607.04872

EventCoT: Event-centric Video Chain-of-thought for Reasoning Temporal Localization

事件中心的视频思维链（EventCoT）：用于推理时间定位

Song, Youngkil, Baek, Yoonjae, Kim, Dongwon, Kim, Inho, Kim, Dongkeun, Kwak, Suha

Abstract

Reasoning temporal localization (RTL) requires a model to generate an answer that itself contains the time interval supporting it, so high-level reasoning and precise temporal grounding must be produced jointly in a single response. To tackle this challenging task, we propose the first event-centric video chain-of-thought framework, dubbed EventCoT. EventCoT first performs event-centric tokenization of the input video to convert it into compact event tokens, enabling efficient identification of question-relevant events. It then reasons within the identified events to generate the answer, grounding the time interval via embedding matching that aligns placeholder tokens with visual embeddings. EventCoT achieves state-of-the-art results on ActivityNet-RTL for reasoning temporal localization while using substantially fewer visual tokens than previous work. To verify its general performance, we further evaluate EventCoT on the grounded video question answering benchmark ReXTime, where it attains strong zero-shot results.

Chinese Translation

推理时间定位（RTL）要求模型生成的答案本身包含支持该答案的时间区间，因此高层次的推理和精确的时间基础必须在单一响应中共同产生。为了解决这一挑战性任务，我们提出了首个事件中心的视频思维链框架，称为EventCoT。EventCoT首先对输入视频进行事件中心的标记，将其转换为紧凑的事件标记，从而有效识别与问题相关的事件。然后，它在识别出的事件中进行推理以生成答案，通过嵌入匹配将占位符标记与视觉嵌入对齐，从而确定时间区间。EventCoT在ActivityNet-RTL上实现了推理时间定位的最新成果，同时使用的视觉标记数量显著少于以往的工作。为了验证其通用性能，我们进一步在基础视频问答基准ReXTime上评估EventCoT，结果显示其在零样本任务中表现出色。

View on arXiv Download PDF AI Translation

cs.CV / 272 / 2607.04882

Unsupervised Detection of Underground Tunnels in Ground-Penetrating Radar Using Depth-Restricted Reconstruction Scoring

基于深度限制重建评分的地下隧道无监督检测方法

Junaid, Muhammad, Khan, Shoab A., Ahmed, Nisar

Abstract

Clandestine tunneling beneath oil and gas pipelines enables fuel theft, smuggling, and sabotage, yet conventional monitoring detects damage only after a pipeline has been compromised. Ground-penetrating radar (GPR) can image such tunnels non-invasively, but manual radargram interpretation does not scale to continuous corridor surveillance, and supervised detectors require tunnel examples that are scarce in practice. We present a fully unsupervised detection pipeline trained exclusively on normal subsurface radargrams collected at a purpose-built field site containing three buried tunnels at 1.5-3 m depth. A denoising convolutional autoencoder learns the structure of anomaly-free ground; at inference, tunnels are flagged by reconstruction error. Our central contribution is a depth-restricted top-k anomaly score, which pools the highest reconstruction errors only within the depth band where tunnels can physically occur. This physically motivated rule raises AUC from 0.986 to 0.994 and cuts missed detections from 74 to 17 of 634 tunnel windows, relative to whole-image scoring, without any retraining or labels. We further show that the optimal top-k fraction interacts with the depth restriction - 1% pooling is best on full images, 5% once scoring is depth-restricted - and that spatial voting across overlapping survey windows helps weak per-image detectors but offers no benefit once the scoring rule is strong. The final system attains AUC 0.994, F1 0.975, recall 0.973, and precision 0.976 on 1,600 field test windows spanning 55 survey lines, at a 1.6% false-alarm rate, using no tunnel labels for training, scoring, or threshold calibration.

Chinese Translation

在油气管道下进行秘密挖掘会导致燃料盗窃、走私和破坏，但传统监测仅在管道受到损害后才能检测到问题。地面穿透雷达（Ground-Penetrating Radar, GPR）可以非侵入性地成像此类隧道，但手动雷达图解释无法适应连续走廊监测，而监督检测器需要在实践中稀缺的隧道示例。我们提出了一种完全无监督的检测流程，该流程仅在一个专门建造的现场收集的正常地下雷达图上进行训练，该现场包含三个埋藏深度为1.5-3米的隧道。去噪卷积自编码器学习无异常地面的结构；在推理过程中，隧道通过重建误差被标记。我们的核心贡献是一个深度限制的top-k异常评分，它仅在隧道可以物理存在的深度范围内汇总最高的重建误差。这个基于物理的规则将AUC从0.986提高到0.994，并将634个隧道窗口中的漏检数量从74减少到17，相较于全图评分，且无需任何重新训练或标签。我们进一步表明，最佳的top-k比例与深度限制相互作用——在全图上1%的汇总效果最佳，而在深度限制评分后5%效果最佳——并且在重叠的调查窗口之间进行空间投票有助于弱的每图检测器，但一旦评分规则强大则没有任何好处。最终系统在1,600个跨越55条调查线的现场测试窗口上达到了AUC 0.994、F1 0.975、召回率0.973和精确率0.976，假警报率为1.6%，在训练、评分或阈值校准中均未使用隧道标签。

View on arXiv Download PDF AI Translation

cs.CV / 273 / 2607.04884

HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better

HunyuanOCR-1.5：让轻量级OCR视觉语言模型更快更好

Li, Gengluo, Wan, Xingyu, Peng, Shangpin, Wang, Weinong, Feng, Hao, Du, Yongkun, Wu, Binghong, Ruan, Zheng, Lu, Zhiqiong, Wu, Liang, Lyu, Pengyuan, Shen, Huawen, Lin, Zibin, Hu, Shijing, Yang, Jieneng, Wen, Hongbing, Yu, Guanghua, Liu, Hong, Wang, Bochao, Ma, Can, Hu, Han, Zhang, Chengquan, Zhou, Yu

Abstract

We present HunyuanOCR-1.5, a lightweight end-to-end OCR-specialized vision-language model. HunyuanOCR unifies document parsing, text spotting, information extraction, text-image translation, and multi-image document understanding within a single end-to-end VLM. Building upon the lightweight architecture of HunyuanOCR-1.0, HunyuanOCR-1.5 does not redesign the backbone, but systematically improves both efficiency and capability. For efficiency, we adapt DFlash to OCR decoding, significantly reducing the latency of long structured outputs such as dense documents, tables, and formulas while preserving output distribution. Powered by DFlash, HunyuanOCR-1.5 achieves a 6.37x Transformer inference speedup and a 2.14x speedup under vLLM, delivering the fastest inference among lightweight OCR VLMs. For capability, we propose Agentic Data Flow, an agent-driven data construction system that transforms model weaknesses into executable data requirements and autonomously performs material search, quality verification, and pipeline development. It substantially improves long-tail capabilities in ancient-script OCR, fine-grained chart and table parsing, multi-image text-centric QA, low-resource multilingual parsing, and document hallucination evaluation. HunyuanOCR-1.5 ranks among the top-tier end-to-end OCR solutions on OmniDocBench v1.6 while achieving new performance milestones across these long-tail tasks. Combined with an upgraded pretraining and post-training recipe, HunyuanOCR-1.5 further extends its capability in high-resolution, long-context, and multi-task scenarios. Experiments demonstrate faster inference, broader OCR capability coverage, and the deployment advantages of a lightweight end-to-end model. We will release the model weights and training code to support future research and real-world OCR applications.

Chinese Translation

我们提出了HunyuanOCR-1.5，这是一种轻量级的端到端OCR专用视觉语言模型。HunyuanOCR将文档解析、文本定位、信息提取、文本-图像翻译和多图像文档理解统一于一个端到端的视觉语言模型中。在HunyuanOCR-1.0的轻量级架构基础上，HunyuanOCR-1.5并未重新设计主干网络，而是系统性地提升了效率和能力。在效率方面，我们将DFlash适配于OCR解码，显著减少了长结构输出（如密集文档、表格和公式）的延迟，同时保持输出分布。借助DFlash，HunyuanOCR-1.5实现了6.37倍的Transformer推理加速和2.14倍的vLLM加速，成为轻量级OCR视觉语言模型中推理速度最快的模型。在能力方面，我们提出了Agentic Data Flow，这是一种基于代理的数据构建系统，将模型的弱点转化为可执行的数据需求，并自主执行材料搜索、质量验证和管道开发。它显著提升了在古文字OCR、细粒度图表和表格解析、多图像文本中心问答、低资源多语言解析和文档幻觉评估等长尾任务中的能力。HunyuanOCR-1.5在OmniDocBench v1.6中排名靠前的端到端OCR解决方案之一，同时在这些长尾任务中实现了新的性能里程碑。结合升级的预训练和后训练方案，HunyuanOCR-1.5进一步扩展了其在高分辨率、长上下文和多任务场景中的能力。实验表明，HunyuanOCR-1.5实现了更快的推理、更广泛的OCR能力覆盖，以及轻量级端到端模型的部署优势。我们将发布模型权重和训练代码，以支持未来的研究和实际的OCR应用。

View on arXiv Download PDF AI Translation

cs.CV / 274 / 2607.04894

ProCon: Projection-Consistency Memory for Training-Free Anomaly Detection

ProCon：用于无训练异常检测的投影一致性记忆

Chae, Joongwon, Luo, Lihui, Liu, Yang, Yu, Dongmei, Qin, Peiwu, Wang, Runming, Chae, Ilmoon

Abstract

Memory-based anomaly detection is attractive because it localizes defects from normal images without training a decoder or synthesizing pseudo anomalies. However, most memory methods still use the memory bank as a nearest-neighbor lookup table: a test patch is treated as normal if it has one nearby normal anchor. This hard retrieval view is vulnerable to false-normal matches and does not test whether the patch is consistently supported by a local normal neighborhood. We propose ProCon, a training-free framework that turns memory retrieval into decoder-free reconstruction. ProCon softly projects each test patch onto nearby normal memory vectors and uses the projection residual as anomaly evidence. To stabilize this residual, it constructs seed-perturbed layer-wise memories, aggregates bank residuals by a median, and fuses depth-specific residual maps by layer consensus. ProCon requires no decoder training, backbone fine-tuning, learned fusion weights, or pseudo-anomaly supervision. Across MVTec-AD, VisA, and Real-IAD under the single-category evaluation protocol, ProCon achieves strong image- and pixel-level performance under seven standard metrics, including image AUROC scores of 99.8%, 99.2%, and 93.2%, respectively. Ablations show that the gains come from replacing hard retrieval with soft normal projection and stabilizing the residuals through memory and depth consensus. The code is available at https://github.com/jw-chae/Procon

Chinese Translation

基于记忆的异常检测因其能够在不训练解码器或合成伪异常的情况下，从正常图像中定位缺陷而受到关注。然而，大多数记忆方法仍将记忆库视为最近邻查找表：如果测试补丁附近有一个正常锚点，则该补丁被视为正常。这种硬检索视角容易受到误匹配的影响，并且未能测试该补丁是否得到局部正常邻域的一致支持。我们提出了ProCon，一个无训练框架，将记忆检索转化为无解码器重构。ProCon将每个测试补丁柔和地投影到附近的正常记忆向量上，并使用投影残差作为异常证据。为了稳定该残差，它构建了种子扰动的层级记忆，通过中位数聚合记忆库残差，并通过层级共识融合深度特定的残差图。ProCon不需要解码器训练、主干微调、学习的融合权重或伪异常监督。在MVTec-AD、VisA和Real-IAD的单类别评估协议下，ProCon在七个标准指标下实现了强大的图像和像素级性能，包括图像AUROC得分分别为99.8%、99.2%和93.2%。消融实验表明，性能提升来自于用软正常投影替代硬检索，以及通过记忆和深度共识稳定残差。代码可在https://github.com/jw-chae/Procon获取。

View on arXiv Download PDF AI Translation

cs.CV / 275 / 2607.04898

3DMPE: 3D Multi-Perspective Embedding

3D多视角嵌入（3DMPE）

Huroyan, Vahan, Rahat-uz-Zaman, Md, Kobourov, Stephen

Abstract

We study 3D point cloud reconstruction from multiple partially observed 2D projections. Given two or more projections of an unknown 3D point cloud, together with cross-view point correspondences and visibility information, our goal is to recover a consistent 3D configuration when different views contain different subsets of points. We propose 3D Multi-Perspective Embedding (3DMPE), an optimization-based, training-free method that reconstructs the 3D point cloud and, in the variable-projection setting, jointly estimates the projection maps. 3DMPE extends Multi-Perspective Simultaneous Embedding to accommodate missing points and incomplete pairwise distance information across views. We consider both fixed-projection and variable-projection settings. Unlike learning-based reconstruction methods that infer shape from raw images and often depend on training data, 3DMPE operates on geometric observations with established correspondences and does not require category-specific training. Experiments on ShapeNet and Pix3D evaluate reconstruction quality using Chamfer Distance, Earth Mover Distance, and RMSE-Optimize-Align (ROA), and examine the effects of initialization, the number of views, point visibility, and several noise regimes, including noisy distances and erroneous correspondences. The results demonstrate that 3DMPE can effectively reconstruct point clouds from partial multi-view geometric observations.

Chinese Translation

我们研究从多个部分观察的2D投影中重建3D点云的问题。给定一个未知3D点云的两个或多个投影，以及视角间的点对应关系和可见性信息，我们的目标是在不同视角包含不同点子集的情况下恢复一致的3D配置。我们提出了3D多视角嵌入（3DMPE），这是一种基于优化的、无训练的方法，能够重建3D点云，并在可变投影设置下联合估计投影映射。3DMPE扩展了多视角同时嵌入（Multi-Perspective Simultaneous Embedding），以适应缺失点和视角间不完整的成对距离信息。我们考虑了固定投影和可变投影两种设置。与基于学习的重建方法不同，后者通常依赖于训练数据从原始图像中推断形状，3DMPE在具有已建立对应关系的几何观测上运行，并且不需要特定类别的训练。我们在ShapeNet和Pix3D上进行实验，使用Chamfer距离、地球移动者距离（Earth Mover Distance）和RMSE优化对齐（RMSE-Optimize-Align, ROA）评估重建质量，并考察初始化、视角数量、点可见性以及包括噪声距离和错误对应关系在内的多种噪声条件的影响。结果表明，3DMPE能够有效地从部分多视角几何观测中重建点云。

View on arXiv Download PDF AI Translation

cs.CV / 276 / 2607.04912

Graph Representation Learning of Longitudinal Medical Imaging Trajectories for Treatment Response Prediction

纵向医学影像轨迹的图表示学习用于治疗反应预测

Kiechle, Johannes, Osuala, Richard, Lang, Daniel M., Fischer, Stefan M., Janíčková, Ivana, Lekadir, Karim, Schnabel, Julia A., Peeken, Jan C.

Abstract

In patients with breast cancer, pathological complete response (pCR) has been established as a clinically meaningful surrogate marker for long-term outcomes. While commonly treated with neoadjuvant chemotherapy (NACT), effective treatment decision-making remains challenging, as therapeutic response can vary substantially across patients, calling for predictive models capable of accurately estimating individualized treatment response. To address this, we propose an imaging-based 3D spatio-temporal framework for treatment response prediction that integrates a state-of-the-art graph neural network with relational modeling of temporal interactions across timepoints alongside three novel complementary self-supervised treatment trajectory representation learning objectives. Experiments across a cohort of 585 patients from the public ISPY-2 dataset demonstrate that our method substantially outperforms both vision and self-supervised learning baselines across several classification metrics. Alongside establishing a breast cancer pCR prediction benchmark, we include a principled ablation of our method and further introduce and empirically assess the impact of the available number of DCE-MRI timepoints per patient trajectory and the inclusion of inter-scan time-differences. Overall, our study substantiates the utility of clinically meaningful longitudinal medical imagaging modeling for predicting NACT-induced pCR. We will publicly share our code repository and a user-friendly PyPI library for dataset curation upon publication, effectively promoting reproducible open-source research.

Chinese Translation

在乳腺癌患者中，病理完全反应（pCR）已被确立为长期结果的临床意义替代指标。尽管通常采用新辅助化疗（NACT）进行治疗，但有效的治疗决策仍然具有挑战性，因为治疗反应在患者之间可能存在显著差异，这要求开发能够准确估计个体化治疗反应的预测模型。为此，我们提出了一种基于影像的三维时空框架，用于治疗反应预测，该框架结合了最先进的图神经网络与时间点间的关系建模，以及三个新颖的互补自监督治疗轨迹表示学习目标。我们在公共ISPY-2数据集中585名患者的队列中进行的实验表明，我们的方法在多个分类指标上显著优于视觉和自监督学习基线。除了建立乳腺癌pCR预测基准外，我们还对我们的方法进行了原则性消融，并进一步引入并实证评估每个患者轨迹中可用的DCE-MRI时间点数量及扫描间时间差的影响。总体而言，我们的研究证实了临床意义的纵向医学影像建模在预测NACT诱导的pCR方面的实用性。我们将在发表后公开分享我们的代码库和用户友好的PyPI库，以促进数据集的整理，从而有效推动可重复的开源研究。

View on arXiv Download PDF AI Translation

cs.CV / 277 / 2607.04921

Efficient Perception in Automotive Detection and Tracking Using Neuromorphic Computing

基于神经形态计算的汽车检测与跟踪中的高效感知

Kolachalam, Manish, Malhotra, Rani

Abstract

Deep learning algorithms are notorious for their high carbon footprint and computational demands that limit their deployment on edge devices and raise concerns about their long-term sustainability. Neuromorphic computing and Spiking Neural Networks (SNNs) offer a promising alternative to traditional Von Neumann architectures, providing energy-efficient performance, massively parallel computation, and on-chip learning capabilities. Autonomous machines represent a critical application domain where these advantages are particularly valuable. We present the first comprehensive evaluation of SNNs for real-world automotive multi-object detection and tracking. Using transfer learning with the SpikeYOLO architecture, we achieve mean Average Precision of 0.937 on the KITTI dataset and 0.771 on BDD100K MOT2020 dataset for object detection and a Higher Order Tracking Accuracy score of 0.701 (KITTI) and 0.445 (BDD100K MOT2020) for object tracking--results competitive with conventional deep learning methods. Our results demonstrate that SNNs can deliver high-performance object detection and tracking in an energy efficient manner, establishing their viability for perception in real-world autonomous systems.

Chinese Translation

深度学习算法因其高碳足迹和计算需求而臭名昭著，这限制了它们在边缘设备上的应用，并引发了对其长期可持续性的担忧。神经形态计算和脉冲神经网络（Spiking Neural Networks, SNNs）为传统冯·诺依曼架构提供了一种有前景的替代方案，具备能效高、并行计算能力强和片上学习能力等优点。自主机器是这些优势特别有价值的关键应用领域。我们首次对SNNs在现实世界汽车多目标检测与跟踪中的应用进行了全面评估。通过使用SpikeYOLO架构的迁移学习，我们在KITTI数据集上实现了0.937的平均精度均值（mean Average Precision），在BDD100K MOT2020数据集上实现了0.771的目标检测精度，以及在目标跟踪中获得了0.701（KITTI）和0.445（BDD100K MOT2020）的高阶跟踪精度（Higher Order Tracking Accuracy）得分，这些结果与传统深度学习方法相当。我们的结果表明，SNNs能够以高效的方式实现高性能的目标检测与跟踪，确立了其在现实世界自主系统感知中的可行性。

View on arXiv Download PDF AI Translation

cs.CV / 278 / 2607.04923

UniSpine-GS: An Efficient Physics-Aware Gaussian Framework for Cross-Modality Multi-view Spine Image Synthesis

UniSpine-GS：一种高效的物理感知高斯框架用于跨模态多视角脊柱图像合成

Chen, Qiuhua, Yu, Changning, Huang, Na, Sun, Chao, Du, Bo

Abstract

The diagnosis of spinal diseases is often assisted by 3D imaging techniques in clinical practice. However, precise 3D spinal assessment is limited by the high costs of 3D imaging hardware and the challenges posed by the physical differences between imaging modalities, which hinder the generalizability of models. To address these issues, we propose UniSpine-GS, an efficient, physics-aware Gaussian framework designed for novel-view projection rendering in multi-view spine imaging via a 3D-aware representation. Instead of performing explicit 3D reconstruction, our approach learns a geometry-aware Gaussian representation that ensures anatomical consistency across different views. We introduce SPWM, a structure-guided loss reweighting strategy to improve boundary fidelity and local details. We evaluate our method on the CTSpine3D dataset and a newly constructed 3D fetal ultrasound dataset, FeSpine3D. Our results demonstrate that UniSpine-GS significantly outperforms existing methods across all metrics, offering a practical and cost-effective solution for unified multi-view medical imaging. Our code is publicly available at https://github.com/orangeisland66/UniSpine-GS.

Chinese Translation

脊柱疾病的诊断在临床实践中通常依赖于三维成像技术。然而，精确的三维脊柱评估受到三维成像硬件高昂成本和不同成像模态之间物理差异带来的挑战的限制，这些因素阻碍了模型的普适性。为了解决这些问题，我们提出了UniSpine-GS，一种高效的物理感知高斯框架，旨在通过三维感知表示实现多视角脊柱成像中的新视角投影渲染。我们的方法并不进行显式的三维重建，而是学习一种几何感知的高斯表示，确保不同视角之间的解剖一致性。我们引入了SPWM，一种结构引导的损失重加权策略，以提高边界保真度和局部细节。我们在CTSpine3D数据集和新构建的3D胎儿超声数据集FeSpine3D上评估了我们的方法。结果表明，UniSpine-GS在所有指标上显著优于现有方法，提供了一种实用且具有成本效益的统一多视角医学成像解决方案。我们的代码已公开发布在https://github.com/orangeisland66/UniSpine-GS。

View on arXiv Download PDF AI Translation

cs.CV / 279 / 2607.04930

MemPose: Category-level Object Pose Estimation with Memory

MemPose：基于记忆的类别级物体姿态估计

Lin, Xiao, Zhu, Minghao, Peng, Yun, Wang, Liuyi, Wang, Qiyi, Liu, Chengju, Chen, Qijun

Abstract

In the pursuit of robust and generalizable category-level object pose estimation, most existing methods adopt parametric formulations that learn effective representations from data, yet they primarily encode category-level patterns into fixed shape priors or static parameter weights, which limits their scalability to highly diverse instances. In this paper, we rethink category-level pose estimation from a memory-centric perspective and present MemPose, a memory-augmented framework that explicitly incorporates category-level geometric memory into the pose estimation pipeline. We introduce an external memory buffer that stores and dynamically updates structural representations from previously observed instances, enabling the model to leverage accumulated experience to support current perception. Extensive experiments on four challenging benchmarks (REAL275, CAMERA25, Housecat6D and Wild6D) demonstrate the superiority of our proposed method over previous state-of-the-art approaches.

Chinese Translation

在追求稳健且具有普适性的类别级物体姿态估计过程中，大多数现有方法采用参数化的形式，从数据中学习有效的表示，然而它们主要将类别级模式编码为固定的形状先验或静态参数权重，这限制了它们在高度多样化实例中的可扩展性。本文从以记忆为中心的视角重新思考类别级姿态估计，并提出了MemPose，一个显式将类别级几何记忆纳入姿态估计流程的记忆增强框架。我们引入了一个外部记忆缓冲区，用于存储和动态更新来自先前观察实例的结构表示，使模型能够利用积累的经验来支持当前的感知。在四个具有挑战性的基准测试（REAL275、CAMERA25、Housecat6D和Wild6D）上的大量实验表明，我们提出的方法优于之前的最先进方法。

View on arXiv Download PDF AI Translation

cs.CV / 280 / 2607.04984

Virtual Category-Guided Continual Generalized Category Discovery

虚拟类别引导的持续广义类别发现

Xiong, Jiahui, Lai, Qiuxia, Wang, Hongsong

Abstract

Continual Generalized Category Discovery (C-GCD) aims to incrementally identify novel categories from sequential unlabeled data while preserving recognition of known classes, which is an essential capability for open-world visual learning. A major bottleneck lies in ambiguous unlabeled samples that cannot be confidently assigned to known classes nor reliably grouped as novel ones, making pseudo-labeling brittle and often biasing learning toward familiar categories. In this work, we introduce Virtual Category-Guided Continual Generalized Category Discovery by adapting Virtual Category Learning (VCL) to the continual setting. Our method identifies uncertain samples and assigns them to temporary virtual categories, enabling safe and informative learning from unlabeled streams without injecting noisy labels, while improving unlabeled data utilization and mitigating prediction bias. To further stabilize discovery across sessions and enhance class separation, we augment VCL with Expanded Neighborhood Contrastive Learning (ENCL), which exploits extended neighborhood relations and an adaptive margin to learn more discriminative and well-separated representations for both old and emerging classes. Extensive experiments on CIFAR-100, Tiny ImageNet, and ImageNet-100 demonstrate that our approach consistently outperforms state-of-the-art methods, establishing a scalable and effective solution for C-GCD.

Chinese Translation

持续广义类别发现（C-GCD）旨在从连续的无标签数据中逐步识别新类别，同时保持对已知类别的识别能力，这是开放世界视觉学习的一个基本能力。一个主要瓶颈在于模糊的无标签样本，这些样本既不能自信地分配给已知类别，也不能可靠地归类为新类别，从而使得伪标签化变得脆弱，并且往往导致学习偏向于熟悉的类别。在本研究中，我们通过将虚拟类别学习（VCL）适应于持续设置，提出了虚拟类别引导的持续广义类别发现。我们的方法识别不确定样本并将其分配到临时虚拟类别，从而在不引入噪声标签的情况下，实现对无标签流的安全和信息丰富的学习，同时提高无标签数据的利用率并减轻预测偏差。为了进一步稳定跨会话的发现并增强类别分离，我们用扩展邻域对比学习（ENCL）增强了VCL，该方法利用扩展的邻域关系和自适应边际，为旧类别和新兴类别学习更具区分性和良好分离的表示。在CIFAR-100、Tiny ImageNet和ImageNet-100上的大量实验表明，我们的方法始终优于最先进的方法，为C-GCD建立了一个可扩展且有效的解决方案。

View on arXiv Download PDF AI Translation

cs.CV / 281 / 2607.05005

Geometry-aware Depth-guided Representation Learning for Structure-preserving Low-light Image Enhancement

基于几何感知的深度引导表示学习用于结构保留的低光照图像增强

Gao, Fang, Qin, Jiongkai, Wang, Jiabao, Tang, Jingfeng, Cheng, Ming, Zheng, Hanbo, Huang, Qingbao, Wu, Cheng

Abstract

Low-light degradation reduces image visibility and weakens structural cues that are important for visual representation and scene understanding. Existing low-light image enhancement methods mainly focus on appearance restoration, while insufficiently exploiting scene geometry to preserve structural consistency. To address this limitation, this paper proposes a Depth-guided Multi-scale Attention Network (DMSA-Net) for geometry-aware low-light image enhancement. DMSA-Net introduces depth-related structural priors into low-light representation learning through reflectance-geometry interaction. A Retinex-based decomposition module is first used to obtain illumination-invariant reflectance representations, from which depth cues are inferred to characterize scene structure under degraded illumination. A multi-scale depth-guided fusion strategy is then embedded into a hierarchical encoder-decoder architecture, where depth-aware attention adaptively integrates geometric and appearance features. Experiments on several benchmark datasets show that DMSA-Net achieves effective low-light restoration while improving structural preservation. Moreover, we construct LOL-D, a depth-augmented low-light dataset, to facilitate research on geometry-aware low-light vision.

Chinese Translation

低光照退化降低了图像的可见性，并削弱了对视觉表示和场景理解重要的结构线索。现有的低光照图像增强方法主要集中在外观恢复上，而未能充分利用场景几何来保持结构一致性。为了解决这一局限性，本文提出了一种深度引导多尺度注意力网络（Depth-guided Multi-scale Attention Network, DMSA-Net），用于几何感知的低光照图像增强。DMSA-Net通过反射-几何交互将与深度相关的结构先验引入低光照表示学习。首先使用基于Retinex的分解模块获得不受照明影响的反射表示，从中推断出深度线索，以表征退化照明下的场景结构。然后，将多尺度深度引导融合策略嵌入到分层编码-解码架构中，深度感知注意力自适应地整合几何特征和外观特征。在多个基准数据集上的实验表明，DMSA-Net在有效恢复低光照图像的同时提高了结构保留。此外，我们构建了LOL-D，一个增强深度的低光照数据集，以促进对几何感知低光照视觉的研究。

View on arXiv Download PDF AI Translation

cs.CV / 282 / 2607.05006

Unsupervised Pixel-Level Semantic Left-Right Understanding of In-the-Wild Images

无监督像素级语义左右理解野外图像

Wang, Weikang, Weißberg, Tobias, Bernard, Florian

Abstract

While various works address reflective symmetry understanding in 3D data and images, pixel-level semantic left-right prediction of in-the-wild images remains challenging, due to certain difficulties including the lack of 3D information, occlusion, object pose variation, partiality, etc. In this work, we propose an unsupervised learning framework to tackle this challenge. Leveraging recent advances in vertex-wise semantic left-right understanding of 3D data, our unsupervised learning method jointly utilises 3D shape and image datasets to infer pixel-wise semantic left-right predictions in single-view images. In particular, we show that a medium-scale 3D shape dataset comprising mainly of human- and quadruped animal-like shapes, combined with diverse in-the-wild image data, are sufficient to achieve high-quality semantic left-right prediction in images, even for entirely unseen 3D object categories, such as cars or trains. Overall, our approach achieves superior performance in dense pixel-wise semantic left-right predictions on both rendered and in-the-wild image datasets when compared to existing state-of-the-art methods.

Chinese Translation

尽管已有多项研究针对3D数据和图像中的反射对称理解，但在野外图像中进行像素级语义左右预测仍然具有挑战性，原因包括缺乏3D信息、遮挡、物体姿态变化和部分性等困难。在本研究中，我们提出了一种无监督学习框架来应对这一挑战。利用最近在3D数据的顶点级语义左右理解方面的进展，我们的无监督学习方法联合利用3D形状和图像数据集，以推断单视图图像中的像素级语义左右预测。特别地，我们展示了一个中等规模的3D形状数据集，主要由类人和四足动物形状组成，结合多样的野外图像数据，足以在图像中实现高质量的语义左右预测，即使对于完全未见过的3D物体类别，如汽车或火车。总体而言，与现有的最先进方法相比，我们的方法在渲染和野外图像数据集上实现了更优的密集像素级语义左右预测性能。

View on arXiv Download PDF AI Translation

cs.CV / 283 / 2607.05008

Comparison of Loss Functions for Robust Deep Learning-based Echocardiography Segmentation when Learning with Partially Labelled Data from Multiple Domains

基于深度学习的心脏超声分割中针对部分标注数据的多领域比较损失函数

Islam, Iman, Puyol-Antón, Esther, Ruijsink, Bram, Reader, Andrew J., King, Andrew P.

Abstract

Echocardiography is the first imaging modality used for assessing cardiac function, and accurate segmentation of cardiac structures is essential for deriving biomarkers. However, the development of effective automated segmentation models for multiple cardiac structures is challenged by the difficulty of training on datasets from different sources that are often partially-labelled. This study aims to address this challenge by evaluating the performance of three loss functions - adaptive categorical cross entropy (aCCE) loss, marginal loss, and the adaptive binary cross entropy (aBCE) loss - in handling partially-labelled data. We conduct a comprehensive comparison of these loss functions across multiple scenarios and network architectures: intra-domain and inter-domain tasks, with both single and multiple partial-labels, and varying proportions of fully-labelled to partially-labelled data. Our experiments reveal that all three loss functions exhibit strong performance in intra-domain segmentation tasks, effectively handling label variations within the same domain. For inter-domain tasks, where models are trained on datasets with a domain shift, the aBCE and marginal losses show superior performance when dealing with the case of one label being missing from some training examples. In scenarios involving more than one label being missing, marginal loss outperforms the other methods, demonstrating its robustness in such complex conditions. These results highlight the strengths of each loss function depending on the labelling scenario, emphasizing the importance of selecting the appropriate loss function to optimize model performance. This study represents the first investigation of techniques for handling partially-labelled data from multiple different domains in echocardiography segmentation and provides a comprehensive comparison of loss-based solutions.

Chinese Translation

心脏超声是评估心脏功能的首选成像方式，而准确分割心脏结构对于衍生生物标志物至关重要。然而，针对来自不同来源的部分标注数据集进行有效的自动分割模型开发面临挑战。本研究旨在通过评估三种损失函数的性能来应对这一挑战——自适应类别交叉熵损失（adaptive categorical cross entropy, aCCE）、边际损失（marginal loss）和自适应二元交叉熵损失（adaptive binary cross entropy, aBCE），以处理部分标注数据。我们在多个场景和网络架构下对这些损失函数进行了全面比较：包括同域和跨域任务，单个和多个部分标签，以及完全标注与部分标注数据的不同比例。实验结果表明，所有三种损失函数在同域分割任务中表现出色，能够有效处理同一领域内的标签变化。在跨域任务中，当模型在具有领域转移的数据集上训练时，aBCE和边际损失在处理某些训练样本缺失一个标签的情况下表现优越。在涉及多个标签缺失的场景中，边际损失优于其他方法，展现出其在复杂条件下的鲁棒性。这些结果突显了每种损失函数在不同标注场景下的优势，强调了选择适当损失函数以优化模型性能的重要性。本研究是首次探讨在心脏超声分割中处理来自多个不同领域的部分标注数据的技术，并提供了基于损失的解决方案的全面比较。

View on arXiv Download PDF AI Translation

cs.CV / 284 / 2607.05035

RUFNet: Query-Guided Support Mask Refinement and Uncertainty Fusion based on Hybrid Mamba for Few-Shot Brain Tumor Segmentation

RUFNet：基于混合曼巴的查询引导支持掩膜精炼与不确定性融合用于少样本脑肿瘤分割

He, Dongyi, Wang, Xiangkai, Xu, Binbing, Jiang, Bin, Yan, Hongjie, Liu, Weixiang, Siok, Wai Ting, Wang, Nizhuan

Abstract

Few-shot brain tumor segmentation remains challenging due to noisy support masks, inter-patient variations between support and query images, and the lack of pixel-wise confidence estimation. This study proposes RUFNet, a Hybrid Mamba-based few-shot framework that combines support mask refinement with uncertainty-aware posterior fusion. To preserve support-query dependencies with manageable cost, RUFNet adopts a Hybrid Mamba interaction backbone with linear complexity. To reduce support-mask noise, an Attention-Guided Mask Refinement module (AGMR) uses query features to recalibrate support masks and improve prototype consistency. To handle ambiguous predictions, an Uncertainty-Aware Posterior Fusion module (UAPF) estimates pixel-wise variance and adaptively balances few-shot predictions with query-aligned priors. On the Brain Tumor Segmentation Challenge (BraTS) 2020 dataset, RUFNet achieves Dice coefficients of 84.3% and 86.1% in the 1-way 1-shot and 1-way 5-shot settings, respectively, outperforming the compared state-of-the-art methods. These results suggest that Hybrid Mamba interaction, mask refinement and uncertainty modelling can improve the robustness of few-shot medical image segmentation. The official implementation code is available at https://github.com/hdy6438/RUFNet.

Chinese Translation

少样本脑肿瘤分割仍然面临挑战，主要由于支持掩膜的噪声、支持图像与查询图像之间的患者间差异，以及缺乏逐像素的置信度估计。本研究提出了RUFNet，一种基于混合曼巴的少样本框架，结合了支持掩膜精炼与不确定性感知的后验融合。为了以可管理的成本保持支持-查询依赖关系，RUFNet采用了具有线性复杂度的混合曼巴交互骨干网络。为了减少支持掩膜的噪声，注意力引导的掩膜精炼模块（AGMR）利用查询特征重新校准支持掩膜，从而提高原型一致性。为了解决模糊预测问题，不确定性感知后验融合模块（UAPF）估计逐像素方差，并自适应地平衡少样本预测与查询对齐的先验。在2020年脑肿瘤分割挑战赛（BraTS）数据集上，RUFNet在1-way 1-shot和1-way 5-shot设置下分别达到了84.3%和86.1%的Dice系数，超越了比较的最先进方法。这些结果表明，混合曼巴交互、掩膜精炼和不确定性建模可以提高少样本医学图像分割的鲁棒性。官方实现代码可在https://github.com/hdy6438/RUFNet获取。

View on arXiv Download PDF AI Translation

cs.CV / 285 / 2607.05056

Consistent and Editable: A Balanced Framework for Text-Guided Video Editing

一致且可编辑：一种平衡的文本引导视频编辑框架

Jin, Tao, Xiao, Li

Abstract

Recently, diffusion models have achieved considerable success in the text-guided video editing domain. However, existing works often struggle to balance the trade-off between temporal consistency and editability in video editing, with consistency and editability typically being inversely related. To address this, we propose a high-quality video editing framework enhanced for consistency and editability, named EquiEdit, which improves coordinatively the temporal consistency and editability of the edited videos while achieving a balance between the two. In terms of temporal consistency, the proposed temporal Mamba module with a tailored temporal-aware scanning scans fused video sequences following four designed directions, effectively enhancing the inter-frame consistency of edited videos. For editability, we design a noise injection strategy based on the spectral transformation to increase editing flexibility, where the Fourier transform is used to preserve the hidden structure in the initial latent noise used for editing, ensuring inter-frame consistency of the edited video and fidelity to the input video. Extensive qualitative and quantitative experiments demonstrate the effectiveness of our method in terms of temporal consistency and editability, as well as its great fidelity to the input video itself.

Chinese Translation

近年来，扩散模型在文本引导的视频编辑领域取得了显著成功。然而，现有研究往往难以平衡视频编辑中的时间一致性与可编辑性之间的权衡，通常这两者呈反比关系。为了解决这一问题，我们提出了一种高质量的视频编辑框架，旨在增强一致性和可编辑性，命名为 EquiEdit，该框架协调性地提高了编辑视频的时间一致性和可编辑性，同时在两者之间实现了平衡。在时间一致性方面，所提出的时间 Mamba 模块通过四个设计方向扫描融合的视频序列，有效增强了编辑视频的帧间一致性。为了提高可编辑性，我们设计了一种基于谱变换的噪声注入策略，以增加编辑灵活性，其中傅里叶变换用于保留用于编辑的初始潜在噪声中的隐藏结构，确保编辑视频的帧间一致性并保持对输入视频的忠实性。大量的定性和定量实验表明，我们的方法在时间一致性和可编辑性方面的有效性，以及对输入视频本身的高度忠实性。

View on arXiv Download PDF AI Translation

cs.CV / 286 / 2607.05077

LangLoc: "Tell Me What You See"

LangLoc: "告诉我你所看到的"

Panwar, Shaurya Kishore, Nobari, Roham Zendehdel, Lau, Shirley Feng Yi, Shaik, Abu Bakr Rahman, Günther, Manuel, Pollefeys, Marc, Barath, Daniel

Abstract

We tackle fine-grained indoor localization from natural language: given a free-form description of one's surroundings, estimate the observer's 2D position and heading within a known 3D environment. Language queries are lightweight, privacy-preserving, and need no camera - yet prior work stops at coarse scene retrieval and cannot resolve an intra-scene pose. We close this gap with LangLoc, a three-stage pipeline that (i) retrieves the correct scene via a dual-branch GATv2 encoder with CLIP semantic features, surpassing the previous best by 8 percentage points in Top-1 recall; (ii) estimates position and heading by scoring a dense floor grid through ray-cast object visibility, reaching a median error of 0.95 m; and (iii) resolves residual ambiguity through a Bayesian dialog module that asks targeted yes/no questions and updates a pose posterior until the location is pinpointed. To support this task we contribute a benchmark of $13{,}000{+}$ pose-indexed natural-language descriptions over $1{,}300{+}$ indoor 3D scans.

Chinese Translation

我们从自然语言的角度解决细粒度室内定位问题：给定对周围环境的自由形式描述，估计观察者在已知三维环境中的二维位置和朝向。语言查询轻量、保护隐私且无需摄像头——然而，之前的工作仅停留在粗略场景检索，无法解决场景内的姿态问题。我们通过LangLoc填补了这一空白，提出了一个三阶段的流程：(i) 通过使用CLIP语义特征的双分支GATv2编码器检索正确的场景，在Top-1召回率上超越了之前的最佳结果8个百分点；(ii) 通过射线投射物体可见性对密集地板网格进行评分来估计位置和朝向，达到中位误差为0.95米；(iii) 通过一个贝叶斯对话模块解决剩余的模糊性，该模块提出针对性的是/否问题并更新姿态后验，直到精确定位。为了支持这一任务，我们贡献了一个包含超过$13{,}000{+}$个姿态索引自然语言描述的基准数据集，涵盖超过$1{,}300{+}$个室内三维扫描。

View on arXiv Download PDF AI Translation

cs.CV / 287 / 2607.05088

RADIANCE: Relative Adaptive Denoising with IP-Adapter for Novel Concept Enhancement

RADIANCE：基于IP-Adapter的相对自适应去噪框架用于新概念增强

Ni, Zi-Xiang, Huang, Bo-Lun, Hsiao, Teng-Fang, Ruan, Bo-Kai, Shuai, Hong-Han

Abstract

Text-to-image (T2I) diffusion models have achieved striking progress but still struggle to synthesize rare concepts involving unusual attribute-object pairings, often resulting in concept omission or semantic drift where a dominant entity overwhelms the generation. Tracing these failures to a lack of compositional balance during the denoising trajectory, we propose RADIANCE, a training-free framework that treats inference as a closed-loop feedback process. RADIANCE augments pretrained backbones with three modular components: (1) a Compositional Similarity Monitor (CSM) that tracks the emergence of objects and attributes in intermediate latents via CLIP-based feedback; (2) a Bidirectional Scale Controller (BSC) that applies a reactive "restoring force" using positive and negative IP-Adapter scales to rebalance biased trajectories; and (3) a Feedback Guidance Scheduler (FGS) that coordinates these updates across timesteps without additional training. We further extend the framework to multi-object prompts via Delayed Adapter Activation (DAA) and Layer-wise Alternating Guidance (LAG) to prevent premature concept fusion. By overlapping monitoring and denoising through pipelined execution, RADIANCE maintains competitive latency while significantly enhancing the per-sample success rate and effective throughput. Experiments on RareBench and T2I-CompBench demonstrate that RADIANCE consistently enhances compositional alignment and perceptual quality over state-of-the-art baselines.

Chinese Translation

文本到图像（T2I）扩散模型已经取得了显著进展，但在合成涉及不寻常属性-对象配对的稀有概念时仍然面临挑战，常常导致概念遗漏或语义漂移，其中主导实体压倒了生成过程。我们将这些失败归因于去噪轨迹中缺乏组合平衡，提出了RADIANCE，一个无训练的框架，将推理视为一个闭环反馈过程。RADIANCE通过三个模块化组件增强预训练的主干网络：（1）组合相似性监测器（CSM），通过基于CLIP的反馈跟踪中间潜变量中对象和属性的出现；（2）双向尺度控制器（BSC），使用正负IP-Adapter尺度施加反应性的“恢复力”以重新平衡偏倚轨迹；（3）反馈指导调度器（FGS），在不增加额外训练的情况下协调这些更新跨越时间步。我们进一步通过延迟适配器激活（DAA）和层级交替指导（LAG）将框架扩展到多对象提示，以防止过早的概念融合。通过管道执行重叠监测和去噪，RADIANCE在保持竞争性延迟的同时显著提高了每个样本的成功率和有效吞吐量。在RareBench和T2I-CompBench上的实验表明，RADIANCE在组合对齐和感知质量方面始终优于最先进的基线。

View on arXiv Download PDF AI Translation

cs.CV / 288 / 2607.05089

TimeThink: Reasoning with Time for Video LLMs

TimeThink：针对视频大型语言模型的时间推理

Li, Handong, Guo, Longteng, Liu, Zikang, Hao, Dongze, Tang, Yepeng, Zhao, Zijia, Jiang, Jie, Jin, Zhiwei, Chen, Chen, Lu, Haonan, Liu, Jing

Abstract

Video reasoning requires models to identify and verify temporally localized evidence within long video sequences. Recent Video Large Language Models (Video-LLMs) have shown promising reasoning abilities when aligned with reinforcement learning, yet existing approaches typically rely on outcome-based rewards that supervise only the final prediction. Such supervision provides limited guidance on how models should discover the relevant temporal evidence during intermediate reasoning. In this work, we propose TimeThink, a reinforcement learning framework that explicitly guides temporal evidence discovery in Video-LLMs. Our key idea is to treat temporal clue steps as the fundamental optimization primitive of video reasoning, where each reasoning step references a candidate time interval in the video. We introduce a step-wise temporal process reward that provides localized credit assignment for these clues and a joint process--outcome optimization objective that balances reasoning fidelity with task correctness. To enable scalable training, we construct TimeThink-RFT-20K, a dataset with automatically derived temporal evidence segments. Extensive experiments across video reasoning, temporal grounding, and general video understanding benchmarks show that TimeThink consistently improves both temporal localization and reasoning performance, achieving state-of-the-art results among open-source video RL models.

Chinese Translation

视频推理要求模型在长视频序列中识别和验证时间局部证据。最近的视频大型语言模型（Video-LLMs）在与强化学习对齐时显示出有希望的推理能力，但现有方法通常依赖于基于结果的奖励，这仅监督最终预测。这种监督对模型在中间推理过程中如何发现相关的时间证据提供了有限的指导。在本研究中，我们提出了TimeThink，一个明确指导视频大型语言模型中时间证据发现的强化学习框架。我们的关键思想是将时间线索步骤视为视频推理的基本优化原语，其中每个推理步骤引用视频中的候选时间区间。我们引入了一种逐步时间过程奖励，为这些线索提供局部信用分配，并提出了一种联合过程-结果优化目标，以平衡推理的准确性与任务的正确性。为了实现可扩展的训练，我们构建了TimeThink-RFT-20K，这是一个具有自动生成时间证据片段的数据集。在视频推理、时间定位和一般视频理解基准上的大量实验表明，TimeThink在时间定位和推理性能上始终有所提升，在开源视频强化学习模型中实现了最先进的结果。

View on arXiv Download PDF AI Translation

cs.CV / 289 / 2607.05090

Be Indiscrete: The Benefits of Learning Continuous Spine Degeneration Severity Scores

不拘一格：学习连续脊柱退变严重程度评分的好处

Monzon, Maria, Zisserman, Andrew, Park, Robin Y., Jutzeler, Catherine R., Jamaludin, Amir

Abstract

Lumbar spine degeneration is a major contributor to chronic low back pain and is routinely assessed on MRI using ordinal grading systems, e.g. normal, mild, moderate, severe. Consequently, most approaches to train models to grade these MRIs formulate grading as a multi-class classification problem, treating ordinal grades as categorical, ignoring differences in misclassification severity, and imposing hard decision boundaries on a continuous disease process. This work explores modeling spinal degeneration as a continuous severity ranking problem. We introduce SpineRankNet, a framework that learns scalar severity scores from lumbar spinal MRI, and compare it against multi-class classification and ordinal regression. Using multiple degeneration measures from the Genodisc dataset, we show that a model trained using a ranking loss to produce a continuous score enables fine-grained ordering of MRI scans. Furthermore, the ordinal grading classes can be recovered from the score with comparable accuracy to those from a model trained directly for classification. The score learned by ranking even improves discrimination between more distant classes. Source code is available at https://github.com/spinetools/spineranknet.

Chinese Translation

腰椎退变是慢性下背痛的主要原因，通常通过MRI使用序数分级系统进行评估，例如正常、轻度、中度和重度。因此，大多数训练模型对这些MRI进行分级的方法将分级视为多类分类问题，将序数等级视为分类，忽略了误分类严重性之间的差异，并在连续疾病过程中施加了严格的决策边界。本研究探讨了将脊柱退变建模为连续严重程度排序问题。我们引入了SpineRankNet，一个从腰椎MRI中学习标量严重程度评分的框架，并将其与多类分类和序数回归进行比较。使用Genodisc数据集中的多种退变测量，我们展示了使用排序损失训练的模型能够生成连续评分，从而实现MRI扫描的细致排序。此外，从评分中恢复的序数分级类别的准确性与直接为分类训练的模型相当。通过排序学习的评分甚至改善了对更远类别之间的区分。源代码可在https://github.com/spinetools/spineranknet获取。

View on arXiv Download PDF AI Translation

cs.CV / 290 / 2607.05093

Semantic Video Communication via Multi-Scale Convolution and Dynamic Routing for Next-Generation Networks

通过多尺度卷积和动态路由实现下一代网络的语义视频通信

Shi, Gengtian, Yu, Jinze, Wu, Chenhao, Wang, Shaofei, Fukuzawa, Eiji, Tang, Junjie, Onoda, Hiroshi, Liu, Jiang

Abstract

The exponential growth of video traffic demands novel semantic communication paradigms that transmit meaning rather than raw bits. We present a generative AI-enabled framework for semantic video communication addressing two critical challenges: efficient hierarchical temporal modeling for bandwidth-constrained transmission and robust semantic alignment between video content and natural language queries at network edge devices. Our approach introduces a multi-scale temporal convolutional encoder that captures motion patterns across different temporal granularities with O(T) complexity suitable for resource-constrained IoT deployments. We further propose a capsule-based dynamic routing mechanism that iteratively refines segment-query associations, enabling flexible modeling of non-monotonic semantic alignments essential for goal-oriented communication. These components are unified through a multi-task learning objective optimizing temporal boundary regression, cross-modal alignment, and capsule diversity. Experiments on ActivityNet Captions demonstrate significant improvements, achieving 42.9% [email protected] and 41.1% mean IoU while maintaining computational efficiency critical for edge deployment.

Chinese Translation

视频流量的指数增长需要新的语义通信范式，以传递意义而非原始比特。我们提出了一种基于生成性人工智能的语义视频通信框架，解决了两个关键挑战：在带宽受限的传输中进行高效的层次时间建模，以及在网络边缘设备上实现视频内容与自然语言查询之间的稳健语义对齐。我们的方法引入了一种多尺度时间卷积编码器，能够以 O(T) 的复杂度捕捉不同时间粒度下的运动模式，适合资源受限的物联网部署。我们进一步提出了一种基于胶囊的动态路由机制，能够迭代地优化片段与查询之间的关联，从而灵活建模非单调的语义对齐，这对于目标导向的通信至关重要。这些组件通过多任务学习目标统一，优化时间边界回归、跨模态对齐和胶囊多样性。在 ActivityNet Captions 上的实验表明，显著提升了性能，达到了 42.9% 的 [email protected] 和 41.1% 的平均 IoU，同时保持了对边缘部署至关重要的计算效率。

View on arXiv Download PDF AI Translation

cs.CV / 291 / 2607.05122

Green for Go, Red for No: Visual Grounding via Semantic Segmentation for VLA Navigation Policies

绿色表示前进，红色表示停止：通过语义分割实现视觉基础的VLA导航策略

Szvoren, Adrian, Kanoulas, Dimitrios, Tuptuk, Nilufer

Abstract

Vision-language-action (VLA) models enable robot navigation from natural language and visual goals, but remain susceptible to perceptual distractions and ambiguous scene interpretations. This paper presents the first empirical evaluation of visual grounding for VLA navigation policies. We propose a real-time segmentation-based grounding method that highlights traversable areas in green and non-traversable areas in red using SegFormer. Two variants are evaluated: observation-only segmentation and joint observation-goal augmentation. Using OmniVLA on the Grand Tour dataset, we show that visual grounding reduces the mean waypoint error by 27-44% at the farthest waypoint, depending on the instruction length. The benefits are greater for long instructions than for short instructions, and grounding provides little improvement for image goals. Normalized error analysis indicates that grounding primarily acts as a trajectory length regularizer, reducing the predicted path length by 30% without improving per-unit-distance reasoning. Our results indicate that visual grounding offers a simple, computationally inexpensive method to improve VLA navigation without model retraining, although it cannot compensate for missing training signals in out-of-distribution instructions.

Chinese Translation

视觉-语言-行动（VLA）模型使得机器人能够根据自然语言和视觉目标进行导航，但仍然容易受到感知干扰和模糊场景解读的影响。本文首次对VLA导航策略的视觉基础进行了实证评估。我们提出了一种基于实时分割的基础方法，使用SegFormer将可通行区域标记为绿色，将不可通行区域标记为红色。我们评估了两种变体：仅观察分割和联合观察-目标增强。使用OmniVLA在Grand Tour数据集上，我们展示了视觉基础在最远航点上将平均航点误差降低了27-44%，具体取决于指令长度。对于长指令，效果优于短指令，而基础对图像目标的改善效果较小。归一化误差分析表明，基础主要作为轨迹长度的正则化器，减少了预测路径长度30%，但没有改善单位距离推理。我们的结果表明，视觉基础提供了一种简单且计算成本低的方法来改善VLA导航，而无需重新训练模型，尽管它无法弥补在分布外指令中缺失的训练信号。

View on arXiv Download PDF AI Translation

cs.CV / 292 / 2607.05133

UNIVERSE: Unified Video Action Models for Autonomous Driving with Flexible Mask-Modulated Modality Generation

UNIVERSE：用于自主驾驶的统一视频动作模型，具备灵活的掩模调制模态生成

Liu, Mengmeng, Zhang, Diankun, Liu, Jiuming, Cui, Jianfeng, Xie, Hongwei, Chen, Guang, Ye, Hangjun, Nex, Francesco, Cheng, Hao, Yang, Michael Ying

Abstract

World Action Models (WAMs) have shown strong potential for improving action generalization in autonomous driving by using future video prediction as dense supervision for scene dynamics and temporal causality. However, it remains unclear which architecture better transfers video-modeling benefits to trajectory generation. Existing cascaded or dual-DiT designs separate video imagination from action prediction, weakening the transfer of video-learned world dynamics to the trajectory branch: the action model may still overfit dataset-specific driving priors, while the video model only indirectly regularizes planning. We propose UNIVERSE, a unified video-action model built upon a single mask-modulated Diffusion Transformer. By co-training future video latents and ego-trajectory tokens within shared generative parameters, UNIVERSE allows dense video supervision to directly shape trajectory denoising, leading to stronger cross-domain action generalization. To ensure causal validity and efficient deployment, we introduce a Modality-Decoupling Visibility Mask, which shares historical context across modalities while blocking mutual attention between future video and trajectory tokens. This prevents future-target leakage and enables trajectory-only inference by removing future-video denoising at test time, achieving a $4.3\times$ speedup over joint video-action rollout while maintaining comparable planning accuracy. The same model also supports video-only and joint video-action rollouts. Experiments show that UNIVERSE achieves 91.0 PDMS on NAVSIM (vs. 89.6 for the Two-DiT variant), and demonstrates strong zero-shot transfer to nuScenes and Bench2Drive without fine-tuning, while ablations confirm the importance of single-DiT unification, video co-training, and mask-based modality decoupling.

Chinese Translation

世界动作模型（WAMs）通过使用未来视频预测作为场景动态和时间因果关系的密集监督，展现了在自主驾驶中改善动作泛化的强大潜力。然而，尚不清楚哪种架构更能有效地将视频建模的优势转移到轨迹生成上。现有的级联或双-DiT设计将视频想象与动作预测分开，削弱了视频学习的世界动态向轨迹分支的转移：动作模型可能仍然过拟合特定数据集的驾驶先验，而视频模型仅间接地对规划进行正则化。我们提出了UNIVERSE，这是一个基于单一掩模调制扩散变换器的统一视频动作模型。通过在共享生成参数内共同训练未来视频潜变量和自我轨迹标记，UNIVERSE使得密集视频监督能够直接塑造轨迹去噪，从而实现更强的跨域动作泛化。为了确保因果有效性和高效部署，我们引入了一种模态解耦可见性掩模，该掩模在模态之间共享历史上下文，同时阻止未来视频与轨迹标记之间的相互注意。这防止了未来目标泄漏，并通过在测试时去除未来视频去噪实现仅轨迹推理，相较于联合视频-动作展开实现了$4.3 imes$的加速，同时保持了可比的规划准确性。同一模型还支持仅视频和联合视频-动作展开。实验表明，UNIVERSE在NAVSIM上达到了91.0的PDMS（相比于Two-DiT变体的89.6），并在不进行微调的情况下展示了对nuScenes和Bench2Drive的强大零-shot迁移，而消融实验确认了单-DiT统一、视频共同训练和基于掩模的模态解耦的重要性。

View on arXiv Download PDF AI Translation

cs.CV / 293 / 2607.05148

Fully Rotation-Equivariant Spectral-Spatial Learning for Multispectral Object Detection

完全旋转等变的光谱-空间学习用于多光谱目标检测

Zhang, Peng, Xu, Tingfa, Han, Shuaihao, Li, Jianan

Abstract

Existing multispectral detectors are limited by discrete spectral processing, a scale-dependent shift in the relative reliability of spectral and spatial cues across pyramid levels, and the lack of explicit rotation-equivariant geometric priors for arbitrarily oriented objects. To tackle these limitations, we propose FressDet, a fully rotation-equivariant spectral-spatial learning framework for multispectral object detection, capable of capturing the continuous, ordered nature of spectral structure and enabling reliable spectral-spatial fusion across pyramid levels under arbitrary in-plane rotations. FressDet integrates three complementary components. Spectral Implicit Warp (SpeIW) enables query-based spectral resampling via a coordinate-conditioned implicit field, yielding a monotone, order-preserving warp. Rotation-Equivariant Consistency Weighting (ReCoW) adaptively fuses spectral and spatial branches based on branch reliability, reinforcing informative cues while suppressing noise across pyramid levels. The oriented-aware head exploits group-indexed features to stably predict oriented objects without parameter replication. Taken together, FressDet learns more discriminative and robust spectral-spatial representations even under rotational perturbations. By achieving state-of-the-art performance with 93% fewer parameters on three public benchmarks, FressDet demonstrates its effectiveness and generalizability.

Chinese Translation

现有的多光谱检测器受到离散光谱处理、金字塔层级中光谱和空间线索相对可靠性的尺度依赖性变化，以及缺乏针对任意方向物体的显式旋转等变几何先验的限制。为了解决这些局限性，我们提出了FressDet，一个完全旋转等变的光谱-空间学习框架，用于多光谱目标检测，能够捕捉光谱结构的连续、有序特性，并在任意平面旋转下实现金字塔层级间可靠的光谱-空间融合。FressDet整合了三个互补组件。光谱隐式变换（Spectral Implicit Warp, SpeIW）通过坐标条件隐式场实现基于查询的光谱重采样，产生单调且保持顺序的变换。旋转等变一致性加权（Rotation-Equivariant Consistency Weighting, ReCoW）根据分支的可靠性自适应地融合光谱和空间分支，增强信息线索，同时抑制金字塔层级中的噪声。面向方向的头部利用组索引特征稳定地预测定向物体，而无需参数复制。综合来看，FressDet即使在旋转扰动下也能学习到更具辨别力和鲁棒性的光谱-空间表示。在三个公共基准上以93%的参数减少实现了最先进的性能，FressDet展示了其有效性和通用性。

View on arXiv Download PDF AI Translation

cs.CV / 294 / 2607.05150

Claim-Level Rubric Rewards for Video Caption Reinforcement Learning

视频字幕强化学习的声明级评分奖励

Gao, Mingqi, Dong, Hongyuan, Chen, Yifei, Zhong, Zhisheng, Ruan, Zheng, Hou, Wenjin, Chen, Yu, Hu, Han, Tang, Yansong

Abstract

In this paper, we introduce Claim-Level Rubric Rewards (CuRe), a structured reward framework designed to address the reward-design bottleneck in reinforcement learning for dense video captioning. Existing reward designs generally fall into two categories: holistic response-level judgment across heterogeneous criteria, or alignment-based evaluation against reference captions. However, both paradigms suffer from fundamental limitations. Holistic rewards struggle to ensure factual accuracy and are prone to stylistic reward hacking, while reference-based rewards overly rely on rigid textual alignment, failing to preserve the completeness and diversity inherent to open-ended generation tasks. To address these challenges, CuRe reformulates reward modeling as fine-grained claim-level verification. Specifically, CuRe decomposes captions into category-aware atomic claims through a structured rubric, converting holistic evaluation into simpler and more reliable claim-level verification.

Chinese Translation

在本文中，我们介绍了声明级评分奖励（Claim-Level Rubric Rewards, CuRe），这是一种结构化的奖励框架，旨在解决密集视频字幕生成中的强化学习奖励设计瓶颈。现有的奖励设计通常分为两类：跨异构标准的整体响应级判断，或基于参考字幕的对齐评估。然而，这两种范式都存在根本性局限。整体奖励难以确保事实准确性，并且容易受到风格化奖励操控的影响，而基于参考的奖励过于依赖严格的文本对齐，未能保留开放式生成任务固有的完整性和多样性。为了解决这些挑战，CuRe将奖励建模重新构建为细粒度的声明级验证。具体而言，CuRe通过结构化评分将字幕分解为类别感知的原子声明，从而将整体评估转化为更简单且更可靠的声明级验证。

View on arXiv Download PDF AI Translation

cs.CV / 295 / 2607.05176

FSDC-DETR: A Frequency-Spatial Domain Collaborative DETR for Small Object Detection

FSDC-DETR：一种用于小物体检测的频率-空间域协作检测变换器

Liu, Aiwen, Zhu, Chengguang, Wang, Gang, Zhu, Dandan, Lin, Haodong, Wang, Yan, Zhou, Huiyu, Pan, Zhiyi

Abstract

Small object detection (SOD) remains a challenging task in real-world applications. Despite recent advances, existing detectors remain limited by rigid processing that entangle spatial aggregation with implicit frequency aliasing and truncation, leading to inadequate preservation of high-frequency components for SOD. To tackle these limitations, we propose a Frequency-Spatial Domain Collaborative Detection Transformer (FSDC-DETR), a novel collaborative framework that explicitly models complementary spatial and frequency representations. Specifically, we first introduce Dual-Branch Frequency-Spatial Adaptive Fusion (DBFSAF) to enhance frequency diversity and adaptively capture frequency-spatial domain discriminative representations. Building on these representations, a frequency-spatial interaction scheme is further explored within the hybrid encoder to enable progressive feature propagation to the decoder. In particular, structure-aware frequency-spatial aggregation is achieved through Shunt Frequency-Spatial Feature Fusion (SFS-FF), establishing bidirectional interaction and progressive cross-scale propagation between frequency and spatial representations for coherent discriminative modeling. Meanwhile, informative high-frequency responses are preserved during scale transitions through Frequency-Spatial Dynamic Downsampling (FSD-Down), thereby minimizing frequency degradation throughout multi-scale fusion for the precise SOD. Experimental results demonstrate that FSDC-DETR achieves state-of-the-art performance, improving AP by 6.4 on VisDrone-DET2019 and 6.6 on AITODv2, with gains of 6.8 and 6.9 AP for small objects. The code is available at github.com/nevereverinsomnia/FSDC-DETR.

Chinese Translation

小物体检测（SOD）在实际应用中仍然是一项具有挑战性的任务。尽管最近取得了一些进展，现有的检测器仍然受到刚性处理的限制，这种处理将空间聚合与隐式频率混叠和截断纠缠在一起，导致在小物体检测中高频成分的保留不足。为了解决这些局限性，我们提出了一种频率-空间域协作检测变换器（FSDC-DETR），这是一种新颖的协作框架，明确建模互补的空间和频率表示。具体而言，我们首先引入双分支频率-空间自适应融合（DBFSAF），以增强频率多样性并自适应捕获频率-空间域的判别表示。在这些表示的基础上，我们在混合编码器中进一步探索频率-空间交互方案，以实现特征的渐进传播到解码器。特别地，通过分流频率-空间特征融合（SFS-FF）实现结构感知的频率-空间聚合，建立频率与空间表示之间的双向交互和渐进跨尺度传播，以实现一致的判别建模。同时，通过频率-空间动态下采样（FSD-Down）在尺度转换过程中保留信息丰富的高频响应，从而最小化多尺度融合中的频率降解，以实现精确的小物体检测。实验结果表明，FSDC-DETR在性能上达到了最先进水平，在VisDrone-DET2019上提高了6.4的AP，在AITODv2上提高了6.6的AP，对于小物体分别提高了6.8和6.9的AP。代码可在github.com/nevereverinsomnia/FSDC-DETR获取。

View on arXiv Download PDF AI Translation

cs.CV / 296 / 2607.05204

Causal-RetiGraph: Cross-Cohort Retinal Support and Same-Subject Pathway Analysis for Diabetic Retinopathy

因果视网膜图：跨队列视网膜支持与同一受试者糖尿病视网膜病变通路分析

Ullah, Inam, Razzak, Imran, Jameel, Shoaib

Abstract

Diabetic retinopathy (DR) is a local retinal lesion process and a visible manifestation of systemic microvascular injury. Modern retinal AI can grade images accurately, but often leaves unanswered how local lesion evidence, retinal vascular structure, and systemic disease pathways are connected. This paper introduces \emph{Causal-RetiGraph}, a compact biomedical informatics framework that links retinal graph phenotypes with NHANES-anchored pathway modelling. The retinal-image fold constructs an interpretable $X1234$ phenotype from vessel maps, lesion evidence, image embeddings, and AutoMorph biomarkers through spatial $X_{12}$ and Jacobian $X_{34}$ branches. The NHANES fold models systemic exposures, covariates, a same-subject retinal mediator family $R^*$, and downstream outcome families. $X1234$ is used for retinal support and pathway prioritisation, while $R^*$ is used for participant-level pathway summaries. On the retinal fold, $X1234$ achieves 0.9055 binary DR accuracy and 0.9711 AUROC, with graded DR QWK of 0.8312. The results show that lesion and biomarker streams improve contextual retinal representation under scarce and imbalanced data. In NHANES, HbA1c, urine albumin, pulse pressure, fasting glucose, and systolic blood pressure are the strongest binary DR anchors. Participant-level pathway analysis identifies glycaemic--renal and glycaemic--haemodynamic pathways as the clearest mediator-style signals. These results suggest that retinal graph phenotypes can help prioritise systemic pathways in DR while preserving the distinction between image-derived support and same-subject mediation.

Chinese Translation

糖尿病视网膜病变（DR）是一种局部视网膜病变过程，是全身微血管损伤的可见表现。现代视网膜人工智能能够准确地对图像进行分级，但往往无法解答局部病变证据、视网膜血管结构和全身疾病通路之间的联系。本文介绍了 extit{因果视网膜图}（Causal-RetiGraph），这是一个紧凑的生物医学信息学框架，将视网膜图表型与以NHANES为基础的通路建模相连接。视网膜图像折叠通过空间$X_{12}$和雅可比$X_{34}$分支，从血管图、病变证据、图像嵌入和AutoMorph生物标志物构建了一个可解释的$X1234$表型。NHANES折叠建模了系统性暴露、协变量、同一受试者视网膜中介家族$R^*$和下游结果家族。$X1234$用于视网膜支持和通路优先级排序，而$R^*$用于参与者级别的通路摘要。在视网膜折叠中，$X1234$实现了0.9055的二元DR准确率和0.9711的AUROC，分级DR的加权Kappa（QWK）为0.8312。结果表明，在稀缺和不平衡数据下，病变和生物标志物流改善了上下文视网膜表示。在NHANES中，HbA1c、尿白蛋白、脉搏压、空腹血糖和收缩压是最强的二元DR锚点。参与者级别的通路分析识别出糖尿病-肾脏和糖尿病-血流动力学通路作为最明显的中介信号。这些结果表明，视网膜图表型可以帮助优先考虑DR中的系统性通路，同时保持图像衍生支持与同一受试者中介之间的区别。

View on arXiv Download PDF AI Translation

cs.CV / 297 / 2607.05205

An event-driven framework for fly-inspired visual motion detection

一种基于事件驱动的仿飞行器视觉运动检测框架

Fu, Qinbing, Huang, Jingyu, Xie, Yan, Peng, Jigen, Tang, Yuchao

Abstract

Fast and reliable motion detection is essential for machine vision and autonomous systems operating in dynamic environments. This work integrates emerging event-based sensing with biologically structured neural computation to establish an efficient computational paradigm for visual motion detection. The proposed framework is built upon a recently developed fly-inspired neural network that emulates motion-processing circuits in the optic lobe. Owing to its feed-forward and training-free architecture, the neural model requires only a small number of interpretable parameters and is well suited for real-time embedded implementation. Event cameras provide low-latency, low-power, and high-dynamic-range visual sensing by asynchronously transmitting brightness-change events. However, their performance can be degraded by event noise, including temporal noise and junction-leakage-induced activity, particularly under low-light conditions. Moreover, effective integration between event-based visual representations and biologically inspired neural processing remains under-explored. To address these challenges, we propose an event-driven computational framework that combines time-surface encoding for front-end event representation with a fly optic-lobe-inspired neural network for foreground motion-direction estimation. A bottom-up attention mechanism is further incorporated to suppress background motion and enhance the saliency of foreground targets. The proposed method is evaluated on real-world ground-vehicle datasets and compared with a baseline frame-based model and an optimization-based approach. Experimental results demonstrate that the framework effectively combines the temporal advantages of event-driven vision with the efficiency and interpretability of bio-inspired neural processing.

Chinese Translation

快速可靠的运动检测对于在动态环境中运行的机器视觉和自主系统至关重要。本研究将新兴的基于事件的传感技术与生物结构化的神经计算相结合，建立了一种高效的视觉运动检测计算范式。所提出的框架基于最近开发的仿飞行器神经网络，该网络模拟了视觉叶中的运动处理电路。由于其前馈和无训练的架构，该神经模型只需少量可解释的参数，非常适合实时嵌入式实现。事件相机通过异步传输亮度变化事件，提供低延迟、低功耗和高动态范围的视觉感知。然而，它们的性能可能会受到事件噪声的影响，包括时间噪声和接点泄漏引起的活动，尤其是在低光照条件下。此外，基于事件的视觉表征与生物启发的神经处理之间的有效整合仍然未得到充分探索。为了解决这些挑战，我们提出了一种事件驱动的计算框架，将时间表面编码用于前端事件表征，并结合仿飞行器视觉叶的神经网络用于前景运动方向估计。进一步引入了自下而上的注意机制，以抑制背景运动并增强前景目标的显著性。所提出的方法在真实世界的地面车辆数据集上进行了评估，并与基线帧基模型和基于优化的方法进行了比较。实验结果表明，该框架有效地结合了事件驱动视觉的时间优势与生物启发神经处理的效率和可解释性。

View on arXiv Download PDF AI Translation

cs.CV / 298 / 2607.05207

Probing Geospatial SSL Representations with Environmental Signals

探究环境信号对地理空间自监督学习表示的影响

Mocharla, Rohita, Patel, Vishal M.

Abstract

Self-supervised learning (SSL) is designed to learn generic, transferable representations rather than representations optimized for a single task. Most geospatial benchmarks evaluate representations solely through downstream tasks, providing limited insight into the information encoded within the representation itself. We ask a different question: do SSL representations of satellite imagery preserve statistical associations with environmental variables that co-vary with the imaging process? To answer this question, we probe SSL representations using co-located ERA5 reanalysis variables, a global dataset of physically consistent environmental variables, including temperature, precipitation, surface solar radiation, surface pressure, and volumetric soil water. These variables are physically related to the spectral reflectance and radar backscatter recorded by Sentinel-1 and Sentinel-2, making them meaningful evaluation targets despite not being used during SSL pretraining. We complement this probing analysis with intrinsic representation metrics to characterize representation geometry and investigate how these properties relate to downstream performance and the encoding of environmental signals. Using DINO, MAE, and MoCo models trained under identical conditions, we show that representation-level metrics distinguish models with similar downstream benchmark performance, providing complementary information beyond task-driven benchmarks. We further find that the linear accessibility of environmental signals is associated with performance on environmentally dependent tasks in the PANGAEA benchmark. Finally, we release ERA5 annotations co-located with the SSL4EO dataset to enable physically grounded representation evaluation for future geospatial foundation models.

Chinese Translation

自监督学习（SSL）旨在学习通用的、可迁移的表示，而不是针对单一任务优化的表示。大多数地理空间基准仅通过下游任务评估表示，提供了对表示内部编码信息的有限洞察。我们提出了一个不同的问题：卫星图像的自监督学习表示是否保留与成像过程共同变化的环境变量之间的统计关联？为了解答这个问题，我们使用共定位的ERA5再分析变量探究自监督学习表示，这是一种包含温度、降水、地表太阳辐射、地表压力和体积土壤水等物理一致环境变量的全球数据集。这些变量与Sentinel-1和Sentinel-2记录的光谱反射率和雷达后向散射物理相关，因此尽管在自监督学习预训练期间未使用，但仍然是有意义的评估目标。我们通过内在表示度量补充了这一探究分析，以表征表示几何，并研究这些属性如何与下游性能及环境信号的编码相关。使用在相同条件下训练的DINO、MAE和MoCo模型，我们展示了表示级度量能够区分具有相似下游基准性能的模型，提供了超越任务驱动基准的补充信息。我们进一步发现，环境信号的线性可达性与PANGAEA基准中环境依赖任务的性能相关。最后，我们发布了与SSL4EO数据集共定位的ERA5注释，以便为未来的地理空间基础模型提供基于物理的表示评估。

View on arXiv Download PDF AI Translation

cs.CV / 299 / 2607.05222

A Multimodal Reasoning Typology for Grounding Chart-Image Coherence in Science Communication

科学传播中图表与图像一致性的多模态推理类型学

Nakarmi, Avina, Sen, Sohom, Song, Xun, Samaddar, Sreyashi, Dasgupta, Aritra

Abstract

Charts and images appear together throughout scientific publications, yet most computational work does not characterize their coherence. We argue that a chart, its accompanying image, and the caption that links them form a multimodal unit, and that the inferential work required to read it varies systematically. To capture this variation, we develop a typology of reasoning gaps, R1 through R5, that characterizes how chart, image, and text jointly convey a scientific claim, and the interpretive work this demands of the reader. Some pairs restate the same data, while in other pairs, charts are used to quantify a structure the image localizes, project image content onto an external variable, audit an image-based claim, or jointly construct a frame that neither panel can establish alone. The typology is anchored in the grounding theory of communication and was derived bottom-up, with a neuroscience expert, from a corpus of 79 traumatic brain injury papers and 32 chart-image pairs. Crucially, the levels provide a systematic mechanism for identifying where grounding succeeds or breaks down, rather than leaving it to subjective inference. We show this in a study in which a domain expert and three non-experts judge vision-language model (VLM) descriptions of 25 pairs: the level predicts where their judgments align and where they diverge, isolating the points at which contextual knowledge, not the figure, carries coherence. This typology thus offers figure designers a systematic way to balance text against chart-image pairs, bridging the expert-to-non-expert divide in reading a scientific takeaway.

Chinese Translation

图表和图像在科学出版物中经常同时出现，但大多数计算工作并未对它们的一致性进行表征。我们认为，图表、其附带的图像以及将二者连接起来的说明文字构成了一个多模态单元，而阅读这一单元所需的推理工作是系统性变化的。为了捕捉这种变化，我们开发了一个推理差距的类型学，R1至R5，描述图表、图像和文本如何共同传达科学主张，以及这对读者所要求的解释工作。一些对重述相同的数据，而在其他对中，图表用于量化图像所定位的结构，将图像内容投射到外部变量上，审计基于图像的主张，或共同构建一个单独的面板无法建立的框架。该类型学基于沟通的基础理论，并且是通过与一位神经科学专家自下而上地从79篇创伤性脑损伤论文和32对图表-图像对的语料库中得出的。重要的是，这些层次提供了一种系统机制，用于识别基础成功或失败的地方，而不是依赖主观推断。我们在一项研究中展示了这一点，其中一位领域专家和三位非专家评估了25对视觉-语言模型（VLM）描述：该层次预测了他们的判断一致和分歧的地方，隔离出上下文知识而非图形所携带的一致性。因此，这一类型学为图形设计者提供了一种系统的方法，以平衡文本与图表-图像对，弥合专家与非专家在阅读科学结论时的差距。

View on arXiv Download PDF AI Translation

cs.CV / 300 / 2607.05243

GUSH3R: Everyone Everywhere All at Once as Gaussians

GUSH3R：每个人、每个地方、同时作为高斯分布

Abe, Keito, Shiohara, Kaede, Otonari, Takashi, Yamasaki, Toshihiko

Abstract

Reconstructing dynamic human-scene environments from monocular videos is a challenging problem that requires jointly modeling scene geometry, camera motion, and non-rigid human dynamics while enabling photorealistic rendering. Recent feed-forward methods can efficiently predict geometry, but they are often limited to non-photorealistic representations such as point clouds and meshes, or they fail to handle non-rigid objects, particularly dynamic humans. To fill this gap, we present GUSH3R (Gaussian-Unified Scene Human 3D Reconstruction), a feed-forward framework for online dynamic human-scene reconstruction. From a monocular human-scene video, our method reconstructs dynamic humans (everyone) and static scenes (everywhere) in a single forward pass (all at once) as 3D Gaussian Splatting (3DGS) primitives (as gaussians), which are geometrically consistent and capable of novel view synthesis. Experiments on monocular human-scene datasets demonstrate that our approach achieves competitive novel view synthesis quality while significantly improving inference efficiency compared to optimization-based methods.

Chinese Translation

从单目视频重建动态人类场景环境是一个具有挑战性的问题，要求同时建模场景几何、相机运动和非刚性人类动态，同时实现照片级真实感渲染。近期的前馈方法能够高效预测几何形状，但通常仅限于非照片级真实感的表示，如点云和网格，或者无法处理非刚性物体，尤其是动态人类。为填补这一空白，我们提出了GUSH3R（Gaussian-Unified Scene Human 3D Reconstruction），一个用于在线动态人类场景重建的前馈框架。通过单目人类场景视频，我们的方法在一次前向传递中重建动态人类（每个人）和静态场景（每个地方），作为3D高斯点（3D Gaussian Splatting，3DGS）原语（作为高斯分布），这些原语在几何上是一致的，并且能够进行新视角合成。在单目人类场景数据集上的实验表明，我们的方法在新视角合成质量上具有竞争力，同时在推理效率上显著优于基于优化的方法。

View on arXiv Download PDF AI Translation

人工智能 (Artificial Intelligence)

cs.AI / 1 / 2607.02542

iFLYTEK-Embodied-Omni Technical Report

iFLYTEK-具身-全能技术报告

Zhang, Yuan, Ni, Jingfei, Lu, Guanchen, Zhang, Shiqi, Xu, Qingshan, Liu, Chi, Nie, Xin, Xu, Wenjie, Gao, Lin, Cheng, Zhiyuan, Zhou, Mingxin, Wu, Jiajia, Liu, Diyuan, Pan, Jia, Ji, Chao

Abstract

General-purpose embodied agents must understand multimodal instructions, anticipate how their environment will evolve, and produce precise control actions over extended horizons. Existing approaches typically specialize in visual-language reasoning, video-based world modeling, or action generation, while cascaded pipelines that first synthesize future observations and then infer actions can introduce interface bottlenecks and compound prediction errors. We present iFLYTEK-Embodied-Omni, a unified multimodal foundation model that jointly models vision(videos and images), language, and action within a single Omni framework. Its modality-specific visual-language, video-generation, and action-generation components communicate through shared multimodal self-attention. This design establishes brain-cerebellum collaboration: the vision-language modeland video generation model form a high-level brain for instruction understanding, task planning, progress tracking, and future visual-state prediction, whereas the action generation modelserves as a low-level cerebellum that directly converts planned subgoals and shared multimodal context into executable action chunks. To develop these capabilities, we combine action-annotated and action-free embodied videos from human demonstrations and robot interactions with embodied reasoning, embodied perception, and general-purpose image-text data to construct a comprehensive dataset. We further adopt a four-stage strategy that progressively trains the VLM, VGM, and AGM before jointly fine-tuning the complete model.

Chinese Translation

通用具身智能体必须理解多模态指令，预测其环境的演变，并在较长时间范围内产生精确的控制动作。现有方法通常专注于视觉-语言推理、基于视频的世界建模或动作生成，而先合成未来观察结果再推断动作的级联管道可能会引入接口瓶颈并加剧预测误差。我们提出了iFLYTEK-具身-全能（iFLYTEK-Embodied-Omni），这是一个统一的多模态基础模型，在单一的全能框架内共同建模视觉（视频和图像）、语言和动作。其特定模态的视觉-语言、视频生成和动作生成组件通过共享的多模态自注意力进行通信。这一设计建立了大脑与小脑的协作：视觉-语言模型和视频生成模型形成了一个高层次的大脑，用于指令理解、任务规划、进度跟踪和未来视觉状态预测，而动作生成模型则作为低层次的小脑，直接将规划的子目标和共享的多模态上下文转换为可执行的动作块。为了发展这些能力，我们结合了来自人类演示和机器人交互的带动作注释和无动作的具身视频，以及具身推理、具身感知和通用图像-文本数据，构建了一个综合数据集。我们进一步采用了一个四阶段策略，逐步训练视觉语言模型（VLM）、视频生成模型（VGM）和动作生成模型（AGM），然后共同微调完整模型。

View on arXiv Download PDF AI Translation

cs.AI / 2 / 2607.02672

Internal Pluralism and the Limits of Pairwise Comparisons

内部多元主义与成对比较的局限性

Flanigan, Bailey, Si, Michelle

Abstract

Local pairwise comparisons are a standard tool for learning how people want decision rules to work, e.g., in participatory design or alignment. However, their use builds in two strong assumptions: that local comparisons are sufficient evidence about how a person wants an automated decision rule to behave, and that people can always answer those comparisons decisively. We investigate how these assumptions may be compromised under internal pluralism: the idea that an individual evaluates decision rules according to multiple authoritative priorities about how the rule should behave. We provide a formal model of such pluralistic preferences over decision rules, which then lets us identify two distinct failures of forced local pairwise comparison data. First, priorities such as proportionality, egalitarianism, and equal treatment are inherently global: what they imply in one case can depend on what happens elsewhere, so local comparisons may fail to capture them. Second, even when priorities are representable locally, tension between strongly-held priorities can generate internal conflict, producing potentially costly behavioral distortions when comparisons are forced. We then use our model to investigate the alternative -- allowing people to report indecision -- and our findings suggest that doing so can considerably reduce the number of queries needed to learn preferences accurately. We conclude by describing how our model points toward preference-learning methods that elicit these priorities directly, yielding more faithful and interpretable accounts of what people value.

Chinese Translation

局部成对比较是了解人们希望决策规则如何运作的标准工具，例如，在参与式设计或对齐中。然而，它们的使用建立在两个强假设之上：局部比较是关于一个人希望自动决策规则如何行为的充分证据，以及人们总是能够果断地回答这些比较。我们研究了在内部多元主义下这些假设可能如何受到影响：即个体根据关于规则应如何行为的多重权威优先级来评估决策规则的观点。我们提供了一个关于这种多元化偏好的正式模型，从而使我们能够识别出强制局部成对比较数据的两个明显缺陷。首先，诸如比例、公平和同等对待等优先级本质上是全球性的：它们在一个案例中的含义可能依赖于其他地方的情况，因此局部比较可能无法捕捉到它们。其次，即使优先级可以在局部表示，强烈持有的优先级之间的紧张关系也可能产生内部冲突，在强制比较时产生潜在的成本行为扭曲。然后，我们使用我们的模型来研究另一种选择——允许人们报告犹豫——我们的发现表明，这样做可以显著减少准确了解偏好所需的查询数量。最后，我们总结了我们的模型如何指向直接引出这些优先级的偏好学习方法，从而提供更真实和可解释的关于人们重视什么的描述。

View on arXiv Download PDF AI Translation

cs.AI / 3 / 2607.02686

ASK in the Dark: Uncertainty-Gated LLM Assistance under Partial Observability

黑暗中的询问：在部分可观测性下的不确定性门控大语言模型辅助

Monteiro, Juarez, Gavenski, Nathan, Lima, Guilherme, Galuppo, Francisco, Rodrigues, Odinaldo, Veloso, Adriano

Abstract

Reinforcement learning agents operating under partial observability must act on incomplete information, making them natural candidates for guidance from small language models (SLMs) that carry broad reasoning priors. Yet integrating SLM guidance into this setting has proven difficult: across all test environments, vanilla uncertainty-gated approaches achieve an overwrite rate at or near zero, meaning the SLM almost never contributes an independent action. We trace this failure to the bare egocentric prompt, which provides insufficient context for genuine reasoning, and identify it as a context problem rather than a capacity problem. We propose ASK+, which supplies the SLM with trajectory-aware context (a partially revealed map, visited positions, and action history) and structured chain-of-thought reasoning, converting it from a passive redundancy check into a more informative consultant that occasionally corrects the policy. We further establish that the predictive entropy signal used for selective querying measures action uncertainty rather than state uncertainty and remains informative in POMDPs, making uncertainty-gated assistance viable beyond fully observable settings. The stateful prompt drives substantial gains: on DoorKey, where vanilla ASK matches PPO (both 89%), ASK+ reaches 93% success; on FourRooms, success climbs from 53% to 70%; on HigherLower, accuracy reaches 73.7%, matching the SLM-only upper bound. Across all environments, Qwen3.5-2B matches or exceeds Qwen3.5-4B, confirming that prompt design and selective gating dominate the impact of model scale, enabling guidance without large models.

Chinese Translation

在部分可观测性下运行的强化学习代理必须在不完整的信息基础上采取行动，这使得它们成为小型语言模型（SLMs）提供指导的自然候选者，因为SLMs具备广泛的推理先验。然而，将SLM指导整合到这一环境中却证明是困难的：在所有测试环境中，普通的不确定性门控方法的覆盖率接近零，这意味着SLM几乎从未提供独立的行动。我们将这一失败归因于简单的自我中心提示，它为真正的推理提供了不足的上下文，并将其识别为上下文问题而非能力问题。我们提出了ASK+，它为SLM提供了轨迹感知的上下文（部分揭示的地图、已访问的位置和行动历史）以及结构化的思维链推理，将其从被动的冗余检查转变为一个更具信息性的顾问，偶尔纠正策略。我们进一步确定，用于选择性查询的预测熵信号测量的是行动不确定性而非状态不确定性，并且在部分可观测马尔可夫决策过程（POMDPs）中仍然具有信息性，这使得不确定性门控辅助在完全可观测的环境之外变得可行。状态提示驱动了显著的提升：在DoorKey上，普通的ASK与PPO相匹配（均为89%），而ASK+达到了93%的成功率；在FourRooms上，成功率从53%上升到70%；在HigherLower上，准确率达到了73.7%，与仅使用SLM的上限相匹配。在所有环境中，Qwen3.5-2B的表现与Qwen3.5-4B相当或更好，确认了提示设计和选择性门控对模型规模的影响占主导地位，从而实现了无需大型模型的指导。

View on arXiv Download PDF AI Translation

cs.AI / 4 / 2607.02771

Automated Data Readiness for Scientific AI

科学人工智能的自动化数据准备

Wilkinson, Sean R., Anantharaj, Valentine G., Choi, Jong Youl, Maheshwari, Ketan, McDonnell, Marshall, Pasini, Massimiliano Lupo, Shpilker, Polina, Souza, Renan, Widener, Patrick, Oral, Sarp, Brewer, Wesley

Abstract

Leadership computing facilities steward large-scale scientific datasets that routinely require substantial transformation before serving as AI training data. However, no existing framework fully unifies automated transformation, readiness assessment, provenance tracking, and agent-native deployment. We present REDI, an open-source framework that addresses this gap through a unified five-stage pipeline (ingest, preprocess, transform, structure, and output) with per-stage instrumentation for reproducibility and deployment as an agent-callable skill; companion tool SetGo automates FAIR compliance and catalog publication. Evaluated across climate, proteomics, materials science, and nuclear fusion, REDI transforms all datasets from raw to AI-ready, with outputs validated against domain-expert references, and preliminary results show near-ideal parallel scaling to 100 nodes on Frontier for the climate case. Provenance-instrumented profiling reveals file I/O as the dominant pipeline cost, with format selection a first-order optimization lever. These results establish REDI as a cross-domain platform providing automated data readiness for scientific AI, transforming data preparation bottlenecks into reproducible, reusable community assets.

Chinese Translation

领导计算设施管理着大规模科学数据集，这些数据集在作为人工智能训练数据之前通常需要进行大量转换。然而，目前没有现有框架能够完全统一自动化转换、准备评估、来源追踪和代理原生部署。我们提出了REDI，这是一个开源框架，通过一个统一的五阶段流程（获取、预处理、转换、结构化和输出）来填补这一空白，并为每个阶段提供可重复性和作为代理可调用技能的部署工具；配套工具SetGo自动化了FAIR合规性和目录发布。在气候、蛋白质组学、材料科学和核聚变等领域进行评估，REDI将所有数据集从原始状态转换为适合人工智能的状态，输出结果经过领域专家参考的验证，初步结果显示在Frontier上气候案例的并行扩展接近理想，达到100个节点。来源追踪的分析揭示文件I/O是主要的流程成本，格式选择是首要的优化杠杆。这些结果确立了REDI作为一个跨领域平台，提供科学人工智能的自动化数据准备，将数据准备瓶颈转变为可重复、可重用的社区资产。

View on arXiv Download PDF AI Translation

cs.AI / 5 / 2607.02807

SwarmResearch: Orchestrating Coding Agents for Open-Ended Discovery

SwarmResearch：协调编码代理以实现开放式发现

Virk, Yuvraj, Edds, Zack, Xia, Chunqiu Steven, Zhang, Lingming

Abstract

Long-running coding agents such as autoresearch can persistently discover optimizations for open-ended problems. However, they tend to converge onto a single high-level approach, then proceed with low-level edits while missing other superior approaches to the problem. We hypothesize two harness-level design choices contribute to this behavior: accumulating context in a single long-running agent and only exposing a single program state to edit. We introduce SwarmResearch, an orchestrator-subagent harness in which a Shepherd Agent uses global context to steer a population of Search Agents, each operating with local context in their respective git branch. On open-ended optimization tasks, SwarmResearch discovers better or comparable solutions to state-of-the-art LLM-guided evolution and multi-agent techniques on 13/15 tasks, driven by higher-level exploration. Compared with fixed scaling of serial and parallel agents, SwarmResearch's orchestrator-guided scaling discovers better-performing solutions by adapting parallelism at different search depths.

Chinese Translation

长期运行的编码代理，如 autoresearch，可以持续发现开放式问题的优化方案。然而，它们往往会收敛到单一的高层次方法，然后进行低层次的编辑，而忽视了其他更优的解决方案。我们假设两个设计选择导致了这种行为：在单个长期运行的代理中积累上下文，以及仅暴露单一程序状态进行编辑。我们提出了 SwarmResearch，这是一种协调者-子代理框架，其中一个牧羊代理（Shepherd Agent）利用全局上下文来引导一组搜索代理（Search Agents），每个代理在其各自的 git 分支中操作局部上下文。在开放式优化任务中，SwarmResearch 在 15 个任务中的 13 个任务上发现了比最先进的 LLM 引导演化和多代理技术更好或相当的解决方案，这得益于更高层次的探索。与固定比例的串行和并行代理相比，SwarmResearch 的协调者引导的比例通过在不同搜索深度下调整并行性，发现了性能更优的解决方案。

View on arXiv Download PDF AI Translation

cs.AI / 6 / 2607.02846

Object-Centric Environment Modeling for Agentic Tasks

面向代理任务的对象中心环境建模

Li, Yiyang, Ma, Tianyi, Wang, Zehong, Ma, Yijun, Ye, Yanfang

Abstract

Large language model (LLM) agents can improve through accumulated experience, but free-form textual memories become difficult to maintain, validate, and reuse as interactions grow. Recent symbolic approaches learn executable skills or programmatic world models, yet often store local procedures or assume simplified dynamics. We propose Object-Centric Environment Modeling (OCM), which organizes experience into an executable object-centric environment model. OCM maintains two connected code bases: object knowledge, which defines environment entities and mechanisms as Python classes, and procedure knowledge, which records reusable interaction patterns that must import and use the object model. OCM works in an online setting: after each episode, OCM reflects on the trajectory, updates both knowledge bases, and verifies that all procedures execute against the updated object model. During future interaction, the agent uses progressive knowledge disclosure to inspect compact code signatures first and read source code only when needed. Experiments show that OCM achieves the best average rank across benchmarks and reduces invalid actions, demonstrating that agents can benefit from building object-centric environment models.

Chinese Translation

大型语言模型（LLM）代理可以通过积累经验来提升自身能力，但随着交互的增加，自由形式的文本记忆变得难以维护、验证和重用。近期的符号方法学习可执行技能或程序化的世界模型，但往往存储局部程序或假设简化的动态。我们提出了对象中心环境建模（Object-Centric Environment Modeling, OCM），它将经验组织成一个可执行的对象中心环境模型。OCM维护两个相互关联的代码库：对象知识，定义环境实体和机制为Python类；程序知识，记录必须导入和使用对象模型的可重用交互模式。OCM在在线环境中工作：在每个回合结束后，OCM会反思轨迹，更新两个知识库，并验证所有程序是否在更新后的对象模型上执行。在未来的交互中，代理使用渐进式知识披露，首先检查紧凑的代码签名，只有在需要时才读取源代码。实验表明，OCM在基准测试中实现了最佳的平均排名，并减少了无效操作，证明代理可以通过构建对象中心环境模型受益。

View on arXiv Download PDF AI Translation

cs.AI / 7 / 2607.02879

MedCalc-Pro: Solving Complex Medical Calculations with LLM Agents

MedCalc-Pro：利用大型语言模型代理解决复杂医疗计算

Zhao, Siran, Hou, Ruihui, Huai, Ziyue, Zhang, Chennuo, Ruan, Tong

Abstract

Current benchmarks for evaluating large language models (LLMs) in medical calculation are largely based on simplified settings, where each patient case corresponds to a single calculator and the required tool is explicitly specified in the query. However, real clinical scenarios often require multiple calculators for joint evaluation, nested-scale calculation, and fuzzy queries that do not directly specify the target calculator. To this end, we propose a new medical calculation benchmark, MedCalc-Pro, which covers three progressively challenging task settings: single-calculator, multi-calculator, and nested-calculator calculation settings. MedCalc-Pro contains 2,268 real-world clinical cases, covering 77 medical calculators across 14 clinical departments. Meanwhile, to address the limited performance of existing frameworks and methods in complex clinical scenarios, we further propose a more generalizable agent framework that supports multi-tool selection and nested-tool calling, while suppressing parameter error propagation through structured validation and evidence review. We conduct systematic comparisons across open-source, closed-source, and medical-specialized LLMs, and the results show that our framework achieves the best performance across all three task settings. This work provides a new benchmark and method for evaluating and applying LLMs in challenging medical calculation scenarios.

Chinese Translation

目前用于评估大型语言模型（LLMs）在医疗计算中的基准测试主要基于简化的设置，其中每个患者案例对应一个单一的计算器，并且所需工具在查询中明确指定。然而，真实的临床场景通常需要多个计算器进行联合评估、嵌套规模计算，以及不直接指定目标计算器的模糊查询。为此，我们提出了一个新的医疗计算基准，MedCalc-Pro，涵盖了三种逐步挑战的任务设置：单计算器、多计算器和嵌套计算器计算设置。MedCalc-Pro包含2,268个真实世界的临床案例，涵盖了14个临床科室的77个医疗计算器。同时，为了解决现有框架和方法在复杂临床场景中的有限性能，我们进一步提出了一个更具普适性的代理框架，支持多工具选择和嵌套工具调用，同时通过结构化验证和证据审查抑制参数错误传播。我们对开源、闭源和医学专业化的LLMs进行了系统比较，结果表明我们的框架在所有三种任务设置中均实现了最佳性能。这项工作为在具有挑战性的医疗计算场景中评估和应用LLMs提供了新的基准和方法。

View on arXiv Download PDF AI Translation

cs.AI / 8 / 2607.02914

Oyster-II: Reinforcement Learning for Constructive Safety Alignment in Large Language Models

Oyster-II：用于大型语言模型的建设性安全对齐的强化学习

Guan, Jiyang, Xie, Yong, Chen, Jun, Liu, Jiexi, Ye, Zipeng, Li, Defeng, Shen, Jiayu, Tao, Jialing, Xue, Hui

Abstract

Large language models (LLMs) have demonstrated remarkable capabilities across diverse applications, yet ensuring their simultaneous safety, helpfulness, and trustworthiness remains a persistent challenge. Conventional refusal-oriented alignment strategies mitigate harmful content generation but systematically fail to serve legitimate user needs, often withholding information that could safely and constructively address the underlying intent of sensitive queries. Building upon the constructive safety paradigm pioneered by Oyster-I, which moves beyond blanket refusal toward thoughtful, response-oriented safety alignment, we identify two critical limitations of its Supervised Fine-Tuning (SFT)-based scheme: insufficient safety generalization to out-of-distribution scenarios and a phenomenon we term safety chain-of-thought (CoT) over-generalization, wherein safety-oriented reasoning patterns are excessively applied to benign queries, degrading helpfulness and user experience. To address these limitations, we propose Oyster-II, a reinforcement learning (RL)-based constructive safety alignment framework that adopts a Zero-RL paradigm combined with a multi-stage reinforcement learning strategy.Evaluated across extensive benchmarks, Oyster-II comprehensively surpasses both Qwen3-14B and its predecessor Oyster-I on safety dimensions, achieving cross-scale performance comparable to Qwen3-Max and Qwen3.5-397B.

Chinese Translation

大型语言模型（LLMs）在各种应用中展现了卓越的能力，但确保它们在安全性、帮助性和可信性方面的同时满足仍然是一个持续的挑战。传统的拒绝导向对齐策略虽然可以减少有害内容的生成，但系统性地未能满足合法用户的需求，常常会拒绝提供能够安全且建设性地解决敏感查询潜在意图的信息。在Oyster-I所开创的建设性安全范式的基础上，Oyster-I超越了简单拒绝，朝着深思熟虑的响应导向安全对齐发展，我们识别出其基于监督微调（SFT）方案的两个关键局限性：对分布外场景的安全性泛化不足，以及我们称之为安全思维链（CoT）过度泛化的现象，即安全导向的推理模式过度应用于良性查询，从而降低了帮助性和用户体验。为了解决这些局限性，我们提出了Oyster-II，这是一种基于强化学习（RL）的建设性安全对齐框架，采用了零强化学习（Zero-RL）范式，并结合多阶段强化学习策略。在广泛的基准测试中，Oyster-II在安全性维度上全面超越了Qwen3-14B及其前身Oyster-I，实现了与Qwen3-Max和Qwen3.5-397B相当的跨规模性能。

View on arXiv Download PDF AI Translation

cs.AI / 9 / 2607.02931

VERITAS: Towards a General-Purpose Replication Tool for Scientific Research

VERITAS：面向科学研究的通用复制工具

Liu, Haokun, Tjiaranata, Filbert Aurelian, Tan, Chenhao

Abstract

AI tools are accelerating scientific publication while the systems that review it struggle to keep up, and independent verification of published research has become both harder and more important. As manual replication is slow and expensive, a growing line of work uses coding agents to automate parts of the process. Existing efforts are largely packaged as benchmarks with companion agents that only run inside the benchmark's own pipeline, and no general-purpose replication tool exists. We present VERITAS, a domain-agnostic replication framework built around CLI coding agents. Given a paper, a code repository, or both, VERITAS extracts the paper's claims, runs the methodology while resolving issues as they arise, and judges each claim against the evidence from experiment runs. The pipeline returns an importance-weighted Replication Score, a severity-rated log of every fix applied, and the patched codebase. We evaluate VERITAS on CORE-Bench and ReplicationBench, 65 papers spanning computer science, social science, medicine, and astrophysics. Against two strong Claude Code baselines on the same model and host environment, VERITAS achieves state-of-the-art performance and leads on every metric on both benchmarks.

Chinese Translation

人工智能工具正在加速科学出版，而审查这些出版物的系统却难以跟上，已发表研究的独立验证变得愈发困难且重要。由于手动复制过程缓慢且成本高昂，越来越多的研究工作开始利用编码代理自动化部分过程。现有的努力大多以基准测试的形式打包，并配有仅能在基准测试自身流程中运行的代理，而尚不存在通用的复制工具。我们提出了VERITAS，这是一个围绕命令行界面（CLI）编码代理构建的领域无关的复制框架。给定一篇论文、一个代码库或两者，VERITAS提取论文的主张，执行方法论并在问题出现时进行解决，并根据实验运行的证据对每个主张进行评判。该流程返回一个重要性加权的复制评分、每个修复的严重性评级日志以及修补后的代码库。我们在CORE-Bench和ReplicationBench上评估VERITAS，这两个基准涵盖了计算机科学、社会科学、医学和天体物理学的65篇论文。在相同模型和主机环境下的两个强基线Claude Code的对比中，VERITAS实现了最先进的性能，并在两个基准的每个指标上均处于领先地位。

View on arXiv Download PDF AI Translation

cs.AI / 10 / 2607.02941

A Sliding-Window-Based Reinforcement Learning for Dynamic Assembly Flow Shop Scheduling with Multi-Product Delivery

基于滑动窗口的强化学习在多产品交付的动态装配流水车间调度中的应用

Qiu, Junhao, Liu, Jianjun, Liu, Ting, Liao, Rongjie, Li, Zhantao, Zhang, Qingfu

Abstract

Multi-product kitting delivery imposes significant challenges for real-time scheduling in hybrid manufacturing systems that integrate processing and assembly, as dynamic order arrivals simultaneously alter supply dependencies and the set of feasible job-machine assignments. This paper proposes a sliding-window-based reinforcement learning (SWRL) framework for end-to-end online scheduling in the flexible assembly flow shop scheduling problem with complex kitting constraints. The problem is formulated as a heterogeneous graph-based Markov decision process that captures the dual-layer kitting structure and the tail-product bottleneck dynamics that produce a sparse reward landscape. To address the resulting challenges, SWRL integrates a sliding-window filtering mechanism that filters inactive nodes and prioritizes kitting-critical operations, a spatiotemporal graph encoding network that tracks bottleneck shifts across consecutive decision states, and a dynamic action mapping module with a constrained waiting strategy that adapts to the changing action space under variable topologies. Experiments on real-world instances from a home appliance manufacturer demonstrate that SWRL achieves consistent tardiness reductions over classical dispatching rules and existing deep reinforcement learning methods, and exhibits robust performance across varying resource configurations, order loads, and arrival concentrations.

Chinese Translation

多产品配件交付对集成加工与装配的混合制造系统中的实时调度提出了重大挑战，因为动态订单的到来同时改变了供应依赖关系和可行的作业-机器分配集合。本文提出了一种基于滑动窗口的强化学习（SWRL）框架，用于具有复杂配件约束的灵活装配流水车间调度问题的端到端在线调度。该问题被表述为一个异构图基础的马尔可夫决策过程，捕捉了双层配件结构和导致稀疏奖励景观的尾部产品瓶颈动态。为了解决由此带来的挑战，SWRL集成了一个滑动窗口过滤机制，该机制过滤不活跃节点并优先考虑配件关键操作；一个时空图编码网络，用于跟踪连续决策状态中的瓶颈变化；以及一个具有约束等待策略的动态动作映射模块，能够适应在可变拓扑下变化的动作空间。在来自一家家电制造商的真实实例上的实验表明，SWRL在减少延迟方面始终优于经典调度规则和现有深度强化学习方法，并在不同资源配置、订单负载和到达浓度下表现出强大的性能。

View on arXiv Download PDF AI Translation

cs.AI / 11 / 2607.02975

Evaluating Generative Agents with Actions Grounded in Socially Distributed Task Environments using Incognita

使用 Incognita 评估在社会分布任务环境中基于行动的生成代理

Hsu, Dan C., Lu, Luke

Abstract

Effective agency in social environments depends on when an agent seeks knowledge, when it acts, and whether its actions are justified by acquired information. Existing grounded benchmarks provide executable actions, persistent state, and verifiable outcomes, while social simulation environments provide rich interaction among language agents. We study an evaluation setting that combines these requirements. We define socially distributed task environments as interactive environments where task-relevant knowledge is partitioned across role-isolated participants and consequential actions are accessible only through them. Communication serves as exploration over role-partitioned knowledge, while grounded action serves as exploitation over environment state. We introduce Incognita, a Concordia-based framework that separates social interaction from grounded execution. The evaluated agent routes messages to a user or specialist entities; specialists mediate admissible operations; a deterministic sub-environment executes accepted operations over a canonical state; and an offline evaluator scores outcomes with inherited rewards. Incognita-Retail transforms tau-bench retail into a multi-entity environment while preserving final-state reward semantics. We evaluate three generative agent models on 18 tasks stratified by social breadth, with 540 trials. Progress appears in reward and behavior: success rises from 0 percent to 8.9 percent and 17.2 percent, while premature finalization falls from 100 percent to 87 percent and 58 percent. Stronger models elicit more hidden knowledge, contact more entities, and attempt more grounded writes, yet reliability remains low. These findings show that socially distributed task environments expose behavior before reliable success, including knowledge elicitation, source selection, grounded action attempts, and premature completion belief.

Chinese Translation

在社会环境中有效的代理能力取决于代理何时寻求知识、何时采取行动以及其行动是否得到所获得信息的支持。现有的基础基准提供可执行的行动、持久的状态和可验证的结果，而社会模拟环境则提供语言代理之间丰富的互动。我们研究了一种结合这些要求的评估设置。我们将社会分布任务环境定义为交互式环境，其中与任务相关的知识在角色孤立的参与者之间进行划分，而后果性行动仅通过这些参与者可获得。沟通作为对角色划分知识的探索，而基础行动则作为对环境状态的利用。我们引入了 Incognita，一个基于 Concordia 的框架，它将社会互动与基础执行分开。被评估的代理将消息路由到用户或专家实体；专家调解可接受的操作；一个确定性的子环境在规范状态上执行接受的操作；而离线评估者则通过继承的奖励对结果进行评分。Incognita-Retail 将 tau-bench 零售转变为一个多实体环境，同时保留最终状态奖励的语义。我们在18个按社会广度分层的任务上评估了三种生成代理模型，共进行了540次试验。奖励和行为方面的进展显著：成功率从0%上升到8.9%和17.2%，而过早完成率则从100%下降到87%和58%。更强的模型能够引出更多隐性知识，接触更多实体，并尝试更多基础写入，但可靠性仍然较低。这些发现表明，社会分布任务环境在可靠成功之前暴露出行为，包括知识引出、源选择、基础行动尝试和过早完成的信念。

View on arXiv Download PDF AI Translation

cs.AI / 12 / 2607.02983

Reinforcement Learning for Evidence-Seeking Diagnostic Reasoning with Large Language Models

基于强化学习的大语言模型证据寻求诊断推理

Hua, Shengyi, Hu, Kangzhe, He, Conghui, Zhang, Xiaofan, Zhang, Shaoting

Abstract

Recent reasoning-centric Large Language Models (LLMs) have made significant strides, yet they predominantly operate on a passive-inference pattern that assumes complete information. In contrast, real-world clinical intelligence is inherently an iterative investigative process requiring strategic evidence acquisition. To bridge this gap, we formalize medical diagnosis as an Iterative Evidence-Seeking Task. We leverage Reinforcement Learning with Verifiable Rewards (RLVR) to elicit intrinsic reasoning within a closed-loop environment, guided by a novel suite of rewards that enforce diagnostic precision and examination consistency. To facilitate this, we introduce the Retrieval-Augmented Generation-based Examination Simulator (RAGES), a high-fidelity clinical oracle that provides realistic, knowledge-grounded follow-up evidence. Empirical results across diverse datasets demonstrate that our framework enables LLMs to transition from passive responders to autonomous assistants. Notably, our model demonstrates comparable performance to larger and reasoning-enhanced baselines, while RAGES proves superior to vanilla LLMs in generating biologically plausible clinical feedback.

Chinese Translation

近期以推理为中心的大语言模型（LLMs）取得了显著进展，但它们主要以假设完整信息的被动推理模式运作。相比之下，现实世界中的临床智能本质上是一个迭代调查过程，需要战略性地获取证据。为了解决这一问题，我们将医学诊断形式化为一个迭代证据寻求任务。我们利用具有可验证奖励的强化学习（RLVR）来激发闭环环境中的内在推理，借助一套新颖的奖励机制，强化诊断精度和检查一致性。为此，我们引入了基于检索增强生成的检查模拟器（RAGES），这是一个高保真的临床神谕，提供现实的、基于知识的后续证据。跨多样化数据集的实证结果表明，我们的框架使得LLMs能够从被动响应者转变为自主助手。值得注意的是，我们的模型在性能上与更大且增强推理的基线相当，而RAGES在生成生物学上合理的临床反馈方面优于普通的LLMs。

View on arXiv Download PDF AI Translation

cs.AI / 13 / 2607.03015

Beyond Forecasting: The Belief-to-Trade Layer in Prediction-Market Agents

超越预测：预测市场代理中的信念到交易层

Wang, Yishu, Wang, Yuxuan, Deng, Jiaqi, Tang, Hanyang

Abstract

Forecasting future events has attracted growing attention as a testbed for general-purpose AI. A natural way to ground this evaluation is let the models trade in the prediction markets. Trading, however, requires more than forecasting. Moreover, recent benchmarks report a substantial gap between calibrated probability scores and the trading results. We propose Raven-Agent, to the best of our knowledge, the first autonomous trading agent for prediction markets. On a controlled replay over an archived decision set, our architecture achieves the only positive return and the only positive risk-adjusted return among all tested policies. We have released our code in https://github.com/Alchemist-X/predict-raven .

Chinese Translation

预测未来事件作为通用人工智能的测试平台受到越来越多的关注。一种自然的评估方式是让模型在预测市场中进行交易。然而，交易不仅仅需要预测。此外，近期的基准测试报告显示，校准概率分数与交易结果之间存在显著差距。我们提出了Raven-Agent，尽我们所知，这是第一个用于预测市场的自主交易代理。在对存档决策集的控制重放中，我们的架构在所有测试策略中实现了唯一的正收益和唯一的正风险调整收益。我们的代码已发布在 https://github.com/Alchemist-X/predict-raven 。

View on arXiv Download PDF AI Translation

cs.AI / 14 / 2607.03025

Human-Centric Reflective Architecture for Human-AI Collaborative Decision-Making

以人为中心的反思架构用于人机协作决策

Kouridakis, Andreas, Spyropoulos, Dimitrios Patiniotis, Vouros, George

Abstract

The use of Large Language Models (LLMs) across diverse areas of human activity-ranging from everyday tasks to safety-critical applications-aims to enhance decision-making effectiveness with minimal human feedback. Concurrently, it seeks to align decisions with human expectations, preferences, and needs while mitigating risks associated with AI non-determinism. However, humans frequently over- or under-rely on AI recommendations, and current AI systems remain poorly calibrated to human expectations. To address these challenges, we introduce a human-AI collaborative decision-making framework designed to augment human capabilities and align AI agents with human preferences and expectations. Specifically, this paper (a) formulates the collaborative decision-making task as a stochastic game between an AI agent and a human player, and (b) proposes the Human-Centric Reflective Architecture (HCRA), which integrates human-calibrated models with reinforcement learning agents that leverage linguistic feedback in an iterative, reflective process. Evaluation results demonstrate that HCRA enhances decision-making effectiveness and delivers high-quality recommendations.

Chinese Translation

大型语言模型（LLMs）在日常任务到安全关键应用等人类活动的各个领域中的应用，旨在以最小的人类反馈增强决策的有效性。同时，它还旨在使决策与人类的期望、偏好和需求保持一致，同时降低与人工智能非确定性相关的风险。然而，人类往往对人工智能的建议过度或不足依赖，而当前的人工智能系统与人类期望的校准仍然较差。为了解决这些挑战，我们提出了一种人机协作决策框架，旨在增强人类能力，并使人工智能代理与人类的偏好和期望保持一致。具体而言，本文（a）将协作决策任务形式化为人工智能代理与人类玩家之间的随机博弈，并（b）提出了以人为中心的反思架构（HCRA），该架构将人类校准模型与利用语言反馈的强化学习代理整合在一个迭代的反思过程中。评估结果表明，HCRA增强了决策的有效性，并提供了高质量的建议。

View on arXiv Download PDF AI Translation

cs.AI / 15 / 2607.03091

Silicon Sampling via Cross-Survey Transfer

通过交叉调查转移进行硅采样

Ku, Chan-Tung, Hsu, Chan, Huang, Pei-Cing, Liu, Frank Cheng-shan, Cheng, I-Ling, Kang, Yihuang

Abstract

Silicon sampling-using large language models (LLMs) to simulate human survey respondents-has emerged as a promising approach for augmenting traditional survey research. However, most evaluations rely on distributional comparisons rather than individual-level prediction, which risks conflating pattern matching with coherent respondent-level prediction. We propose cross-survey transfer, a more rigorous evaluation framework in which an LLM is given a respondent's answers to one set of questions and must predict their answers to entirely different questions from the same survey. Using data from the Taiwan Election and Democratization Study (TEDS) 2024, three open-weight LLMs (27B-120B parameters), and supervised machine learning baselines, we find that: (1) zero-shot LLMs achieve 52% accuracy on genuinely unseen items, closing to within 6 percentage points (pp) of a supervised random forest trained on same-population data; (2) a stable construct predictability hierarchy emerges, from 67% for partisan attitudes to 23% for sovereignty; and (3) variance collapse and safety alignment effects-two commonly cited LLM limitations-turn out to be more nuanced than previously reported, with variance collapse affecting supervised models as well and alignment effects varying dramatically across model families. These findings clarify both the promise and boundaries of silicon sampling.

Chinese Translation

硅采样——利用大型语言模型（LLMs）模拟人类调查受访者——已成为增强传统调查研究的一种有前景的方法。然而，大多数评估依赖于分布比较，而非个体层面的预测，这可能会将模式匹配与连贯的受访者级别预测混为一谈。我们提出了交叉调查转移，这是一种更为严格的评估框架，其中LLM在获得受访者对一组问题的回答后，必须预测他们对同一调查中完全不同问题的回答。利用2024年台湾选举与民主化研究（TEDS）数据、三种开放权重的LLM（27B-120B参数）以及监督机器学习基准，我们发现：（1）零样本LLM在真正未见过的项目上实现了52%的准确率，与在同一人群数据上训练的监督随机森林相差仅6个百分点（pp）；（2）出现了一个稳定的构念可预测性等级，从67%的党派态度到23%的主权；（3）方差崩溃和安全对齐效应——两个常被引用的LLM局限性——实际上比之前报告的更为复杂，方差崩溃同样影响监督模型，而对齐效应在不同模型家族之间变化显著。这些发现澄清了硅采样的潜力和局限性。

View on arXiv Download PDF AI Translation

cs.AI / 16 / 2607.03162

APeB: Benchmarking Personalization Ability of Large Language Model Agents

APeB：大型语言模型代理的个性化能力基准测试

Yang, Garry, Chen, Zizhe, Chen, Xinru, Chen, Yongqiang, Wang, Jianxiang, Zou, Deyu, Ding, Linyi, Wu, Jialiang, He, Yunzhong, Gong, Yu, Cheng, James, Tou, Huaixiao

Abstract

LLM-powered agents struggle with personalization when users issue raw, underspecified queries. In this setting, agents must infer latent intent, extract preferences from noisy interaction histories, and select among competing alternatives. Existing benchmarks rarely test this capability, as they often rely on user-refined queries or simplified histories. We introduce personalized product search (PPS), a testbed for agentic personalization under raw queries and diverse histories. We construct Agent Personalized Benchmark (APeB) from action logs, pairing underspecified intents with rich histories and user-viewed candidate items. Evaluating state-of-the-art LLMs with multi-step agent workflows, we find that models handle explicit queries well but struggle with early-stage queries requiring intent and preference discovery. Rubric analysis attributes this gap mainly to ineffective history use. A simple history-aware query-refinement pipeline, VQRA, yields consistent gains, highlighting the need for dedicated history-utilization modules in personalized agents.

Chinese Translation

当用户发出原始且不明确的查询时，基于大型语言模型（LLM）的代理在个性化方面面临挑战。在这种情况下，代理必须推断潜在意图，从嘈杂的交互历史中提取偏好，并在竞争的替代方案中进行选择。现有基准测试很少测试这一能力，因为它们通常依赖于用户优化的查询或简化的历史记录。我们引入了个性化产品搜索（PPS），这是一个在原始查询和多样化历史下进行代理个性化的测试平台。我们从操作日志中构建了代理个性化基准（APeB），将不明确的意图与丰富的历史记录和用户查看的候选项目配对。通过对最先进的LLM进行多步骤代理工作流程的评估，我们发现模型能够很好地处理明确查询，但在需要意图和偏好发现的早期查询中表现不佳。评分分析将这一差距主要归因于历史使用不当。一个简单的历史感知查询优化管道VQRA带来了持续的提升，突显了个性化代理中专门历史利用模块的必要性。

View on arXiv Download PDF AI Translation

cs.AI / 17 / 2607.03228

Organizational Memory for Agentic Business Process Execution

面向自主业务流程执行的组织记忆

Kirchdorfer, Lukas, Rebmann, Adrian, Warmuth, Christian, Kampik, Timotheus, Heilker, Theiss, Berg, Gregor

Abstract

LLM-based agents offer new opportunities for automating business process execution beyond the limits of rule-based systems. However, general-purpose LLMs lack the organization-specific knowledge required for reliable execution, which is typically fragmented across human-oriented artifacts such as policies, process models, and standard operating procedures. While such knowledge can technically be encoded in individual prompts or agent-specific retrieval setups, this approach does not scale in enterprises, as it gives rise to knowledge silos and rule duplicates, and makes consistent updates and learning across agents difficult. We argue that this calls for an organizational memory for agentic business process execution: a shared, governed, and agent-consumable reference layer of evolving organization-specific procedural knowledge about how work should be executed. We derive requirements for such a memory, propose an architecture for its curation and consumption, and demonstrate its effectiveness in a proof-of-concept based on a procurement scenario.

Chinese Translation

基于大型语言模型（LLM）的代理为自动化业务流程执行提供了超越规则系统限制的新机会。然而，通用的LLM缺乏可靠执行所需的特定于组织的知识，这些知识通常分散在以人为中心的文档中，如政策、流程模型和标准操作程序。虽然这些知识可以在单个提示或特定于代理的检索设置中进行编码，但这种方法在企业中并不具备可扩展性，因为它导致知识孤岛和规则重复，并使得在代理之间进行一致的更新和学习变得困难。我们认为，这需要为自主业务流程执行建立一种组织记忆：一个共享的、受管控的、可供代理使用的不断发展的特定于组织的程序性知识参考层，关于工作应如何执行。我们推导出这种记忆的需求，提出其策划和使用的架构，并在一个基于采购场景的概念验证中展示其有效性。

View on arXiv Download PDF AI Translation

cs.AI / 18 / 2607.03283

Embodied Operators and Benchmarking: Toward Reusable and Deployable Embodied Intelligence Systems

具身操作符与基准测试：迈向可重用和可部署的具身智能系统

Xiong, Junwu, Gao, Jiaxuan, Chai, Wei, Chen, Renxing, Li, Yuzhen, Guo, Yu, Guo, Yucheng, Luo, Mingxi, Ma, Wenyang, Mou, Yiyun, Zhang, Yifei, Zhou, Chen, Guo, Yongjian

Abstract

Embodied intelligence systems require not only end-to-end policy models, but also reusable functional modules that transform multimodal observations, robot states, human demonstrations, and task contexts into structured representations, decisions, trajectories, control references, and system services. This work defines these modules as embodied operators and studies them as independent yet composable units in embodied intelligence pipelines. We clarify their definition boundary, emphasizing task semantics, standardized input-output contracts, deployability, reusability, and multi-layer optimizability. We further construct a taxonomy covering five categories: detection and segmentation, spatial localization and 3D understanding, hand motion recovery, embodied foundation models and task-decision operators, and planning, control, and system support operators. For each category, we summarize representative functions, technical paradigms, application roles, and practical limitations. Beyond taxonomy, we propose a multi-dimensional benchmark framework that evaluates embodied operators in terms of correctness, end-to-end efficiency, resource usage, temporal stability, portability, interface compatibility, deployment reliability, and downstream task utility. We also discuss workflow-level operator acceleration and open challenges in operator composition, data standardization, world models, VLA safety, edge deployment, and real-world application value. Overall, this work argues that embodied operators should be optimized and evaluated as holistic deployable components, providing a foundation for reusable, scalable, and verifiable embodied intelligence systems.

Chinese Translation

具身智能系统不仅需要端到端的策略模型，还需要可重用的功能模块，这些模块将多模态观察、机器人状态、人类示范和任务上下文转化为结构化的表示、决策、轨迹、控制参考和系统服务。本研究将这些模块定义为具身操作符，并将其作为具身智能管道中的独立但可组合单元进行研究。我们明确了它们的定义边界，强调任务语义、标准化的输入输出契约、可部署性、可重用性和多层优化能力。我们进一步构建了一个涵盖五个类别的分类法：检测与分割、空间定位与三维理解、手部运动恢复、具身基础模型与任务决策操作符，以及规划、控制与系统支持操作符。对于每个类别，我们总结了代表性功能、技术范式、应用角色和实际限制。除了分类法，我们还提出了一个多维基准框架，以评估具身操作符在正确性、端到端效率、资源使用、时间稳定性、可移植性、接口兼容性、部署可靠性和下游任务效用等方面的表现。我们还讨论了工作流级别的操作符加速以及在操作符组合、数据标准化、世界模型、VLA安全性、边缘部署和现实世界应用价值等方面的开放挑战。总体而言，本研究认为具身操作符应作为整体可部署组件进行优化和评估，为可重用、可扩展和可验证的具身智能系统提供基础。

View on arXiv Download PDF AI Translation

cs.AI / 19 / 2607.03303

Reflective Dialogue or Prompt Refinement? Effects of Tutor Scaffolding on Students' Independent LLM Use for Programming

反思对话还是提示优化？导师支架对学生独立使用大型语言模型进行编程的影响

Brender, Jerome, El-Hamamsy, Laila, Uittenhove, Kim, Perez, Aitor, Jermann, Patrick, Mondada, Francesco, Bumbacher, Engin

Abstract

While Large Language Models (LLMs) can provide personalized support in learning, several studies have raised concerns regarding their use in education. Importantly, learning depends on how students engage with LLMs. This study examined how two types of LLM-based tutors shape students' prompting practices, learning, and subsequent LLM-use: a Socratic-Guidance (SG) tutor, which structures interaction through dialogic questioning, and a Prompt-Refinement (PR) tutor that guides the formulation of effective prompts. We conducted a two-phase study in a graduate-level mobile robotics course: 66 students used either the SG or PR tutor during a 6-week intervention, followed by 52 students using an unconstrained LLM during a 3-week course project. Results show that while the SG- and PR tutors led to similar task performance and prompting patterns during guided use, they differ in learning outcomes and later LLM-use. SG-students, relative to PR-student, achieved higher learning gains in later sessions, and were more likely to adopt understanding-driven prompting strategies, which are predictive of higher understanding, when using an unconstrained LLM. Although learners perceived the SG tutor as less efficient, the findings suggest that Socratic guidance supports the development of students' capacity to learn with LLMs over time, highlighting its importance for LLM tutor design.

Chinese Translation

虽然大型语言模型（LLMs）可以为学习提供个性化支持，但一些研究对其在教育中的使用提出了担忧。重要的是，学习取决于学生如何与LLMs互动。本研究考察了两种基于LLM的导师如何影响学生的提示实践、学习和后续的LLM使用：一种是苏格拉底引导（Socratic-Guidance, SG）导师，通过对话式提问结构化互动；另一种是提示优化（Prompt-Refinement, PR）导师，指导有效提示的制定。我们在一个研究生级别的移动机器人课程中进行了为期两阶段的研究：66名学生在为期6周的干预中使用SG或PR导师，随后52名学生在为期3周的课程项目中使用无约束的LLM。结果显示，尽管SG和PR导师在指导使用期间导致了相似的任务表现和提示模式，但它们在学习成果和后续LLM使用上存在差异。相较于PR学生，SG学生在后续课程中取得了更高的学习收益，并且在使用无约束LLM时更有可能采用以理解为驱动的提示策略，这与更高的理解水平相关。尽管学习者认为SG导师的效率较低，但研究结果表明，苏格拉底引导支持学生随着时间的推移发展与LLMs学习的能力，突显了其在LLM导师设计中的重要性。

View on arXiv Download PDF AI Translation

cs.AI / 20 / 2607.03386

When Aggregate Alignment Misleads: Auditing Policy Repair Without Per-State Expert Actions

当聚合对齐误导时：无状态专家行动的政策修复审计

Zhu, Peiying, Chang, Sidi

Abstract

Agentic AI systems are increasingly used to edit, refine, and repair decision policies, but evaluating these edits is difficult when per-state expert action labels are unavailable. We study this problem in a hotel-pricing simulator where an agentic policy editor receives only region-level diagnostic feedback: summaries of how its price distribution differs from a benchmark policy across time, inventory, and market regions. The editor cannot observe benchmark actions, benchmark source code, reward numbers, or held-out outcomes, and may only propose constrained edits to a target-action table. On 5,000 held-out episodes, a multi-restart LLM editor reaches RevPAR 108.47 (95% CI 107.61 - 109.34), close to the benchmark policy's 108.75 (107.81 - 109.68), with paired gap (LLM minus benchmark) -0.276 and 95% CI [-0.692, 0.146]. A cheap diagnostic projection already recovers much of the revenue (107.90), so the LLM editor's distinctive gain is not raw revenue lift alone: it also reduces episode composition distance from 1.153 to 0.609. This is the strongest non-benchmark repair result. This profile is not explained by restart search alone: non-semantic proposers with up to 2,500 evaluations fall 8.77 - 14.57 RevPAR points short. Nor is it explained by plausible prompt format: a shuffled-diagnostic control breaks region-error correspondence and falls to RevPAR 94.30. The match is genuine but partial. A tree editor achieves stronger pooled alignment, 0.214 versus 0.266, and stronger reference-state D1, 0.328 versus 1.197, yet revenue falls to 98.91. These results show that agentic policy repair should be evaluated by whether diagnostic feedback becomes reliable closed-loop outcome, not by a single behavioral distance.

Chinese Translation

自主人工智能系统越来越多地用于编辑、完善和修复决策政策，但在缺乏每个状态专家行动标签的情况下，评估这些编辑变得困难。我们在一个酒店定价模拟器中研究了这个问题，在该模拟器中，自主政策编辑器仅接收区域级诊断反馈：其价格分布与基准政策在时间、库存和市场区域上的差异摘要。编辑器无法观察基准行动、基准源代码、奖励数值或保留结果，并且只能对目标行动表提出有限的编辑。在5000个保留的情节中，一个多重重启的LLM编辑器达到了108.47的每间可用房收入（RevPAR）（95%置信区间107.61 - 109.34），接近基准政策的108.75（107.81 - 109.68），配对差距（LLM减去基准）为-0.276，95%置信区间为[-0.692, 0.146]。一个廉价的诊断投影已经恢复了大部分收入（107.90），因此LLM编辑器的独特收益不仅仅是原始收入的提升：它还将情节组成距离从1.153减少到0.609。这是最强的非基准修复结果。这个特征并不能仅通过重启搜索来解释：非语义提议者在最多2500次评估中落后8.77到14.57个RevPAR点。也不能通过合理的提示格式来解释：一个打乱的诊断控制打破了区域错误对应关系，降至RevPAR 94.30。匹配是真实的，但部分的。一个树编辑器实现了更强的汇总对齐，0.214对比0.266，以及更强的参考状态D1，0.328对比1.197，但收入降至98.91。这些结果表明，自主政策修复应通过诊断反馈是否成为可靠的闭环结果来评估，而不是通过单一的行为距离。

View on arXiv Download PDF AI Translation

cs.AI / 21 / 2607.03394

From Mobile Data to Business Insights: An End-to-End Analytics Framework for Large-Scale Urban Mobility Analysis and Decision Support

从移动数据到商业洞察：大规模城市出行分析与决策支持的端到端分析框架

Andrade, Thiago, Tabassum, Shazia, Silva, Miguel E. P., Dinis, Ricardo, Gama, Joao

Abstract

Real time location data derived from mobile applications is a powerful tool for addressing various urban challenges, including tourism planning, parking management, bus route optimization, and resource allocation. Besides, it offers invaluable insights for shaping strategic decisions in commercial domains such as location based services, market share analysis, and behavioral profiling. In this expansive study, we aim to address all of the aforementioned challenges by investigating the behaviors and patterns of smartphone users within urban environments, particularly in the domains of tourism, transportation, and retail. Our approach encompasses the development of a sophisticated data platform from inception to implementation, which includes the formulation of use cases, architectural design, and implementation of modules. We employ state of the art techniques and technologies, including data anonymization, ETL pipelines, and utilizing Google BigQuery and Vertex AI for data processing and machine learning model development. A modular architecture based on reusable analytical building blocks was developed to generate data products that support multiple stakeholder driven use cases. Additionally, we apply interactive data visualization techniques via Power BI to facilitate the effective interpretation of analytical findings by stakeholders. The developed models address a wide range of mobility analytics tasks, including mobility profiling, frequent trajectory mining, area of influence analysis, traffic anomaly detection, and origin destination pattern analysis. The results demonstrate the framework's ability to capture user mobility dynamics at fine spatial and temporal resolutions, providing actionable insights for urban planning and strategic business decision making.

Chinese Translation

来自移动应用的实时位置数据是解决各种城市挑战的强大工具，包括旅游规划、停车管理、公交路线优化和资源分配。此外，它为商业领域的战略决策提供了宝贵的洞察，例如基于位置的服务、市场份额分析和行为分析。在这项广泛的研究中，我们旨在通过研究城市环境中智能手机用户的行为和模式，特别是在旅游、交通和零售领域，来解决上述所有挑战。我们的方法包括从构思到实施开发一个复杂的数据平台，其中包括用例的制定、架构设计和模块的实施。我们采用最先进的技术和工具，包括数据匿名化、ETL管道，以及利用Google BigQuery和Vertex AI进行数据处理和机器学习模型开发。我们开发了基于可重用分析构件的模块化架构，以生成支持多个利益相关者驱动的用例的数据产品。此外，我们通过Power BI应用交互式数据可视化技术，以促进利益相关者对分析结果的有效解读。所开发的模型涵盖了广泛的出行分析任务，包括出行画像、频繁轨迹挖掘、影响区域分析、交通异常检测和起止点模式分析。结果表明，该框架能够以精细的空间和时间分辨率捕捉用户的出行动态，为城市规划和战略商业决策提供可行的洞察。

View on arXiv Download PDF AI Translation

cs.AI / 22 / 2607.03397

Efficient bias mitigation in T2I diffusion models using Concept Graphs

基于概念图的T2I扩散模型中高效的偏见缓解

Mansi, Kori, Avinash, Leofante, Francesco

Abstract

Text-to-Image diffusion models often propagate harmful bias inherited from the training data. Existing bias mitigation techniques typically intervene only at the text encoder or provide inference-time guidance, often leading to generations that collapse into semantically incoherent outputs. To address these limitations, we introduce CO-ALIGN (Concept Ontology Alignment), a novel bias mitigation approach based on concept-graph alignment that operates on the model's internal concept ontology. By aligning concepts within the text encoder and denoiser, CO-ALIGN achieves substantial bias reduction while preserving generative integrity. We demonstrate the effectiveness of concept-graph alignment across three paradigms: text-encoders, denoisers and joint text-denoiser ontology alignment. CO-ALIGN outperforms the state of the art, improving fairness by $30\%$, $\Delta FID=11.4$ in image quality, $2.8\%$ in image fidelity, all while reducing semantically incoherent outputs by $88\%$. Beyond bias mitigation, we show that CO-ALIGN benefits other downstream tasks as well. In particular, our experiments demonstrate that better-aligned internal ontologies enhance concept unlearning robustness across multiple unlearning techniques.

Chinese Translation

文本到图像的扩散模型常常传播从训练数据中继承的有害偏见。现有的偏见缓解技术通常仅在文本编码器处进行干预或提供推理时指导，这往往导致生成的结果语义不连贯。为了解决这些局限性，我们提出了CO-ALIGN（概念本体对齐），这是一种基于概念图对齐的新型偏见缓解方法，作用于模型的内部概念本体。通过在文本编码器和去噪器之间对齐概念，CO-ALIGN在保持生成完整性的同时实现了显著的偏见减少。我们展示了概念图对齐在三种范式中的有效性：文本编码器、去噪器和联合文本-去噪器本体对齐。CO-ALIGN在性能上超越了现有技术，公平性提高了30%，图像质量的$ ext{FID}$改善了11.4，图像保真度提高了2.8 ext{%}，同时将语义不连贯的输出减少了88 ext{%}。除了偏见缓解，我们还展示了CO-ALIGN对其他下游任务的益处。特别是，我们的实验表明，更好对齐的内部本体增强了多种去学习技术下的概念去学习鲁棒性。

View on arXiv Download PDF AI Translation

cs.AI / 23 / 2607.03425

Personalized Causal Recourse: A Human-In-The-Loop Approach

个性化因果补救：一种人机协作的方法

Tampieri, Denise, De Toni, Giovanni, Giudici, Paolo

Abstract

Algorithmic recourse addresses the challenge of providing tailored recommendations to users affected by unfavorable machine learning decisions, in potentially high-stakes scenarios. Traditional approaches to recourse often rely on the closest counterfactual explanations or assume a priori knowledge of a user's causal structure, resulting in interventions that overlook individual contexts and specific feature interactions. To overcome these limitations, we study a human-in-the-loop framework that iteratively approximates the user's structural causal model through interactive queries via Bayesian inference before producing recourse recommendations. This framework exploits humans' feedback to improve the identification of causal effects, allowing personalized recourse that is plausible, cost-effective, and aligned with the actual causal dependencies of each user. As a proof of concept, we evaluate this framework through simulated human responses. Our simulations across linear and non-linear causal models show promising results, though challenges remain in capturing complex, non-linear structures, emphasizing the importance of accurate approximations and robust noise distribution modeling.

Chinese Translation

算法补救旨在解决在潜在高风险场景中，为受到不利机器学习决策影响的用户提供量身定制的建议的挑战。传统的补救方法通常依赖于最接近的反事实解释，或假设对用户因果结构的先验知识，从而导致的干预措施忽视了个体背景和特征间的具体交互。为了克服这些局限性，我们研究了一种人机协作框架，该框架通过贝叶斯推断通过交互查询迭代地近似用户的结构因果模型，然后生成补救建议。该框架利用人类反馈来改善因果效应的识别，从而实现个性化的补救，确保其合理、经济，并与每个用户的实际因果依赖关系相一致。作为概念验证，我们通过模拟人类反应对该框架进行了评估。我们在线性和非线性因果模型上的模拟结果显示出良好的前景，尽管在捕捉复杂的非线性结构方面仍然面临挑战，强调了准确近似和稳健噪声分布建模的重要性。

View on arXiv Download PDF AI Translation

cs.AI / 24 / 2607.03478

Demonstrating Generalization Failures via Mixtures of Conditional Policies

通过条件策略的混合展示泛化失败

Barzdukas, Jou, Peck, Jack, Schulz, Julian, Rauba, Paulius, Basart, Steven, Wells, Lennie

Abstract

Post-training of frontier language models is conducted on curated task suites, and inevitably leaves a distribution shift between training and deployment environments. This exposes developers to generalization failures, which are relatively poorly understood. To better understand such generalization failures, we believe the community should construct clean demonstrations under simplified conditions. To facilitate this, we propose a simple and flexible way to construct language models which fail to generalize in controllable ways when subsequently trained with Reinforcement Learning (RL) on a given distribution of training tasks. Our construction uses Supervised Fine-Tuning on a dataset of a mixture of transcripts corresponding to a collection of 'conditional policies', which can each independently be assigned certain behaviors on each different task distribution, to obtain a model that is then well approximated as a 'mixture of conditional policies.' We observe that RL training then selects for policies that obtain the highest reward on the training distribution. This can produce striking behaviors: in a controlled setting with two distributions containing identical questions prepended with two different 'trigger strings', RL training on either distribution actively degrades performance on the other to zero, even though the underlying task is identical. We also use our construction to illustrate two novel ways in which generalization may fail in future language models, corresponding to distribution shifts of task coverage and temporal context respectively. While our construction is deliberately simple and may not closely resemble 'natural' generalization failures, the resulting 'model organisms' are of interest for alignment stress-testing and generalization science, and can be used as existence proofs that training success and generalization can come apart in structured ways.

Chinese Translation

前沿语言模型的后训练是在经过精心策划的任务套件上进行的，这不可避免地导致了训练和部署环境之间的分布偏移。这使得开发者面临泛化失败的问题，而这一问题相对较少被理解。为了更好地理解这种泛化失败，我们认为社区应该在简化条件下构建清晰的演示。为此，我们提出了一种简单而灵活的方法，构建在给定训练任务分布上经过强化学习（Reinforcement Learning, RL）训练后以可控方式无法泛化的语言模型。我们的构建使用监督微调（Supervised Fine-Tuning）在一个混合转录数据集上，该数据集对应于一组“条件策略”，每个策略可以在不同的任务分布上独立分配特定行为，从而获得一个模型，该模型可以很好地近似为“条件策略的混合”。我们观察到，RL训练选择那些在训练分布上获得最高奖励的策略。这可能产生显著的行为：在一个控制环境中，两个包含相同问题但前置了两个不同“触发字符串”的分布，RL训练在任一分布上都会使另一分布的性能降至零，尽管基础任务是相同的。我们还利用我们的构建展示了未来语言模型中泛化可能失败的两种新方式，分别对应于任务覆盖和时间上下文的分布偏移。尽管我们的构建故意简单，可能与“自然”泛化失败并不十分相似，但所得到的“模型生物体”对于对齐压力测试和泛化科学具有重要意义，并可以作为训练成功与泛化在结构化方式上可能分离的存在证明。

View on arXiv Download PDF AI Translation

cs.AI / 25 / 2607.03530

MentalThink: Shaping Thoughts in Mental SVG World

MentalThink：在心理SVG世界中塑造思维

Lin, Kangheng, Yin, Jisheng, Li, Dingming, Yu, En, Wei, Yana, Zhou, Han, Zhao, Liang, Zhou, Hongyu, Peng, Hongbo, Sun, Jianjian, Ge, Zheng, Zhang, Xiangyu, Jiang, Daxin, Wang, Jingyu

Abstract

We introduce MentalThink, a visual-symbolic reasoning paradigm that equips Multimodal LLMs (MLLMs) with an executable mechanism for "mental" visualization. The core of MentalThink is a think-with-SVG pipeline, where the model learns to generate, render, and interpret scalable vector graphics (SVG) code as an intermediate visual representation for multi-turn reasoning. By creating structured vector sketches, the model can externalize spatial hypotheses, inspect them through deterministic rendering, and reason within a constrained geometric space, effectively mimicking the human process of mental imagery. We instantiate this paradigm through a two-stage training framework, combining Supervised Fine-Tuning (SFT) for SVG syntactic alignment with multi-turn Reinforcement Learning (RL) to encourage iterative inspection, revision, and refinement of intermediate visual hypotheses. Extensive evaluations demonstrate that MentalThink achieves superior performance on spatial understanding and reasoning benchmarks (e.g., 55.1% on VSIBench, 76.0% on MindCube), showing that executable vector graphics provide a verifiable visual workspace for dynamic perspective taking, visual reflection, and compositional scene construction.

Chinese Translation

我们介绍了MentalThink，这是一种视觉-符号推理范式，赋予多模态大语言模型（MLLMs）一种可执行的“心理”可视化机制。MentalThink的核心是一个think-with-SVG管道，其中模型学习生成、渲染和解释可缩放矢量图形（SVG）代码，作为多轮推理的中间视觉表示。通过创建结构化的矢量草图，模型能够外化空间假设，通过确定性渲染对其进行检查，并在受限的几何空间内进行推理，有效地模拟人类的心理意象过程。我们通过一个两阶段的训练框架实例化这一范式，结合监督微调（SFT）以实现SVG语法对齐，并通过多轮强化学习（RL）鼓励对中间视觉假设的迭代检查、修订和完善。广泛的评估表明，MentalThink在空间理解和推理基准测试中表现优越（例如，在VSIBench上达到55.1%，在MindCube上达到76.0%），显示可执行的矢量图形为动态视角转换、视觉反思和组合场景构建提供了可验证的视觉工作空间。

View on arXiv Download PDF AI Translation

cs.AI / 26 / 2607.03550

Applying Answer Set Programming with Fuzzy Membership Functions: a Case Study

应用模糊隶属函数的答案集编程：案例研究

Ferragina, Luca, Galati, Ilenia, Gullone, Lorena, Scarcello, Francesco

Abstract

Human reasoning often operates through qualitative concepts expressed by linguistic labels such as high, low, expensive, or cheap, whose interpretation depends on context and is usually vague, despite being rooted in numerical data. This paper explores a novel fuzzy-logic-based qualitative extension of Answer Set Programming (ASP) to bridge numerical information and qualitative reasoning. The underlying language, formally introduced in a separate work, provides a principled framework that avoids rigid thresholds and supports robust reasoning under vagueness. Focusing on a representative use case, we illustrate how the framework integrates numerically grounded inputs (such as outputs of machine learning models) with symbolic reasoning over qualitative labels. Key features, including learning-based membership functions and semantically enriched predicates, enable the combination of expert knowledge, contextual factors, and subjective interpretations within a unified declarative setting.

Chinese Translation

人类推理通常通过定性概念进行，这些概念通过诸如高、低、昂贵或便宜等语言标签表达，其解释依赖于上下文，通常是模糊的，尽管其根植于数值数据。本文探讨了一种基于模糊逻辑的答案集编程（Answer Set Programming, ASP）的定性扩展，以弥合数值信息与定性推理之间的差距。该基础语言在一项独立的工作中正式介绍，提供了一个原则性框架，避免了僵化的阈值，并支持在模糊性下的稳健推理。我们聚焦于一个具有代表性的用例，展示该框架如何将数值基础的输入（例如机器学习模型的输出）与定性标签的符号推理相结合。关键特性，包括基于学习的隶属函数和语义丰富的谓词，使得在统一的声明性环境中结合专家知识、上下文因素和主观解释成为可能。

View on arXiv Download PDF AI Translation

cs.AI / 27 / 2607.03561

How to Avoid Debate: Scalable AI Safety via Doubly-Efficient Interactive Proofs

如何避免辩论：通过双重高效交互证明实现可扩展的人工智能安全性

Chen, Liyan, Kalai, Yael Tauman, Xi, Zoe

Abstract

As AI models continue to develop powerful capabilities, it becomes critical that we are able to verify that their output is aligned with our intentions. A recent line of work focuses on verification via debate, a model of interactive proofs where two competing powerful provers, or AI models, debate each other to convince a weak verifier, or a human, of the correctness of their claim. However, debate assumes that the two AI models possess equal abilities and that one of them is truthful, which may not be realistic. In this work, we show \emph{how to avoid debate}: we initiate the study of \emph{single-prover} interactive proofs for AI safety. Prior results in single-prover interactive proofs do not immediately carry over to the AI safety setting: for example, they do not work when the computation has access to an oracle, such as to human judgment or an external database such as the web. We present doubly-efficient single-prover interactive proofs and arguments for oracle-aided computations (also known as relativizing proofs), in the settings where (1) the computation is robust, in the sense that the output does not change if at most a small fraction of the answers to oracle queries are incorrect, or (2) the oracle is a low-degree polynomial. These results suggest that interactive verification is possible even without debate, under structured or noise-tolerant oracle access.

Chinese Translation

随着人工智能模型不断发展出强大的能力，验证其输出是否与我们的意图一致变得至关重要。最近的研究工作集中在通过辩论进行验证，这是一种交互证明模型，其中两个竞争的强大证明者或人工智能模型相互辩论，以说服一个弱验证者或人类其主张的正确性。然而，辩论假设两个人工智能模型具备相等的能力，并且其中一个是诚实的，这在现实中可能并不成立。在本研究中，我们展示了 extit{如何避免辩论}：我们开始研究 extit{单证明者}交互证明在人工智能安全性中的应用。单证明者交互证明的先前结果并不能直接转移到人工智能安全的场景中：例如，当计算能够访问一个oracle（如人类判断或外部数据库如网络）时，它们并不适用。我们提出了双重高效的单证明者交互证明和针对oracle辅助计算的论证（也称为相对证明），在以下情况下：(1) 计算是稳健的，即如果至多一小部分oracle查询的答案不正确，输出不会改变；或(2) oracle是一个低度多项式。这些结果表明，即使在没有辩论的情况下，在结构化或抗噪声的oracle访问下，交互验证也是可能的。

View on arXiv Download PDF AI Translation

cs.AI / 28 / 2607.03634

The Role of Rigor in Artificial Intelligence

严谨性在人工智能中的作用

Nguyen, Timothy

Abstract

Artificial intelligence (AI) has achieved extraordinary capabilities despite lacking many of the conceptual and scientific foundations associated with mature disciplines. Unlike traditional sciences, where reliable technology typically emerges from theoretical understanding, modern AI has progressed largely through performance-driven iteration and "alchemical" experimentation. This tension motivates a systematic analysis of AI through the lens of rigor. We introduce a three-part framework consisting of conceptual rigor (clarifying foundational concepts), epistemic rigor (establishing scientific understanding), and operational rigor (ensuring reliable performance and deployment). Using this framework, we analyze competing conceptions of intelligence and understanding, the strengths and limitations of the empirical approach to deep learning, the power and pitfalls of benchmarks, and the obstacles to theory development posed by modern AI systems. We argue that the distinctive trajectory of AI arises from how forms of rigor interact across paradigms, resulting in the primacy of operational rigor in modern deep learning. This perspective helps explain both AI's rapid advances and its persistent uncertainties, while clarifying the challenges involved in transforming AI into a mature science and reliable technology.

Chinese Translation

尽管缺乏与成熟学科相关的许多概念和科学基础，人工智能（AI）已取得了非凡的能力。与传统科学不同，可靠技术通常源于理论理解，而现代人工智能主要通过以性能为驱动的迭代和“炼金术”实验取得进展。这种紧张关系促使我们通过严谨性的视角对人工智能进行系统分析。我们提出了一个由三部分组成的框架，包括概念严谨性（澄清基础概念）、认识论严谨性（建立科学理解）和操作严谨性（确保可靠的性能和部署）。利用这一框架，我们分析了竞争的智能和理解观念、深度学习经验方法的优势与局限、基准测试的力量与陷阱，以及现代人工智能系统对理论发展的障碍。我们认为，人工智能的独特发展轨迹源于不同范式之间严谨性形式的相互作用，导致操作严谨性在现代深度学习中的主导地位。这一视角有助于解释人工智能的快速进展及其持续的不确定性，同时澄清将人工智能转变为成熟科学和可靠技术所面临的挑战。

View on arXiv Download PDF AI Translation

cs.AI / 29 / 2607.03694

Robust Feasible Route Construction through Collaborative Partition Optimization

通过协作分区优化构建稳健的可行路线

Karaahmetoglu, Oguzhan, Kim, Hyong

Abstract

Large-scale Capacitated Vehicle Routing Problems (CVRPs) are commonly solved by partitioning customers into smaller routing problems that can be optimized independently. While this substantially reduces computational complexity, independently constructed routing solutions may leave some customer demand unserved even when sufficient resources exist elsewhere in the fleet. We present Collaborative Routing Constructors (CoRC), a routing framework that enables independently solved subproblems to exchange customers and vehicles during optimization rather than relying solely on a fixed partition or a subsequent global re-optimization stage. Computational experiments on AGS benchmark instances and synthetic instances containing up to 200,000 customers compare CoRC against independent routing, post-routing global re-optimization, and state-of-the-art, end-to-end routing frameworks. Across all evaluated partitioning strategies, CoRC consistently constructs feasible routing solutions where competing partition-based methods do not. Furthermore, it remains effective on problem instances for which the evaluated end-to-end routing frameworks did not produce solutions under the same computational budget. These results demonstrate that collaboration between routing subproblems provides a robust and scalable approach for feasible large-scale route construction.

Chinese Translation

大规模容量限制车辆路径问题（CVRPs）通常通过将客户划分为可以独立优化的小型路径问题来解决。虽然这大大降低了计算复杂性，但独立构建的路径解决方案可能会导致某些客户需求未得到满足，即使车队中其他地方存在足够的资源。我们提出了协作路径构建器（Collaborative Routing Constructors, CoRC），这是一个路径框架，允许在优化过程中独立解决的子问题之间交换客户和车辆，而不是仅依赖于固定分区或后续的全局重新优化阶段。我们在AGS基准实例和包含多达200,000名客户的合成实例上进行了计算实验，将CoRC与独立路径、路径后的全局重新优化以及最先进的端到端路径框架进行了比较。在所有评估的分区策略中，CoRC始终构建出可行的路径解决方案，而竞争的基于分区的方法则未能做到。此外，在评估的端到端路径框架在相同计算预算下未能产生解决方案的问题实例上，CoRC依然有效。这些结果表明，路径子问题之间的协作为可行的大规模路线构建提供了一种稳健且可扩展的方法。

View on arXiv Download PDF AI Translation

cs.AI / 30 / 2607.03702

Agent Reinforcement Learning via Pivotal-Aware Self-Feedback Retry

通过关键点感知自反馈重试的智能体强化学习

Guo, Weiyang, Shi, Zesheng, Zhang, Longhui, Zhu, Zeen, Zhang, Min, Li, Jing

Abstract

Large language model (LLM) agents have shown strong decision-making capabilities in long-horizon interactive tasks, yet they still struggle to effectively leverage failed trajectories: full retries incur high interaction costs, while experience retrieval tends to dilute critical experience signals. To address this, we propose PivoARL, a self-feedback retry framework for experience exploitation in LLM agents. PivoARL identifies the pivotal erroneous turn through structured reflection and performs local retry only from the corresponding pivotal state, thereby reusing the correct prefix and reducing redundant interactions. From an information-gain perspective, we further show that pivotal retry concentrates useful experience signals near the error boundary, mitigating the signal dilution caused by state-agnostic experience utilization. Based on this insight, we design a pivotal-aware credit assignment mechanism that rewards correct prefixes while isolating erroneous suffixes, and optimize reflection quality through implicit reflection returns. We conduct a systematic evaluation on 4 agent tasks and 7 search-based QA benchmarks. Results show that PivoARL achieves significant improvements on Pass@2/3 across all tasks, with an average gain of about 11.5\% over MetaRL. Moreover, benefiting from contrastive preference signals induced by pivotal turns, PivoARL also consistently improves Pass@1 on over 80\% of the tasks. On Minesweeper environment, PivoARL improves over GiGPO by more than 45\% and reduces interaction turns by about 42\% on average compared with full-retry methods. Code is available at https://github.com/yuki-younai/PivoARL.

Chinese Translation

大型语言模型（LLM）智能体在长时间交互任务中展现出强大的决策能力，但它们在有效利用失败轨迹方面仍然存在困难：全面重试会产生高昂的交互成本，而经验检索往往会稀释关键经验信号。为了解决这一问题，我们提出了PivoARL，一种用于LLM智能体经验利用的自反馈重试框架。PivoARL通过结构化反思识别关键错误转折，并仅从相应的关键状态进行局部重试，从而重用正确的前缀并减少冗余交互。从信息增益的角度来看，我们进一步表明，关键重试将有用的经验信号集中在错误边界附近，缓解了由与状态无关的经验利用造成的信号稀释。基于这一见解，我们设计了一种关键点感知的信用分配机制，奖励正确的前缀，同时隔离错误的后缀，并通过隐式反思回报优化反思质量。我们在4个智能体任务和7个基于搜索的问答基准上进行了系统评估。结果表明，PivoARL在所有任务上在Pass@2/3上取得了显著提升，平均增益约为11.5 ext{%}，超过了MetaRL。此外，得益于由关键转折引发的对比偏好信号，PivoARL在超过80 ext{%}的任务上也持续提高了Pass@1。在扫雷环境中，PivoARL相比GiGPO提升了超过45 ext{%}，并且与全面重试方法相比，平均减少了约42 ext{%}的交互轮次。代码可在https://github.com/yuki-younai/PivoARL获取。

View on arXiv Download PDF AI Translation

cs.AI / 31 / 2607.03703

Explainable Reinforcement Learning for Adaptive Traffic Signal Control

可解释的强化学习用于自适应交通信号控制

Kwesiga, Dickens, Choudhary, Nishu, Guin, Angshuman, Hunter, Michael

Abstract

Reinforcement Learning (RL) has emerged as a powerful paradigm for adaptive traffic signal control. However, in safety-critical infrastructure like traffic control, the opaque, black-box nature of deep RL models poses challenges for transportation agency acceptance, regulatory compliance, operational trust, troubleshooting, and fine-tuning. To bridge this gap between high-performance optimization and human-comprehensible interpretability, this effort introduces a novel, explainable entity centric RL framework for safe and transparent traffic signal control. Rather than processing traffic states through monolithic, flat vectors, the proposed architecture disaggregates real-time intersection observations into distinct, high-dimensional lane entities and phase temporal configurations to inherently preserve the structural topology and geometric configurations of the intersection. Relational dependencies and inter-lane conflicts are dynamically extracted via a dual-stage attention network featuring sequential multi-head cross-attention and self-attention blocks. This design yields a real time affinity matrix that quantifies the direct influence of signal phases on specific approach volumes and queues, providing full visual and analytical interpretability. To ensure strict operational reliability, a deterministic action-masking interface is integrated directly into the Proximal Policy Optimization pipeline, explicitly blocking invalid phase transitions to guarantee absolute compliance with established signal timing and safety constraints. Evaluated in a microscopic simulation environment, outperforms state-of-the-art baselines in delay minimization. More importantly, the emergent attention weights align precisely with established traffic engineering principles, offering an auditable, trust-enabling, and deployable architecture for next-generation adaptive traffic control systems.

Chinese Translation

强化学习（Reinforcement Learning, RL）已成为自适应交通信号控制的强大范式。然而，在交通控制等安全关键基础设施中，深度强化学习模型的不透明性和黑箱特性给交通管理机构的接受度、合规性、操作信任、故障排除和微调带来了挑战。为了解决高性能优化与人类可理解的可解释性之间的鸿沟，本研究提出了一种新颖的、以实体为中心的可解释强化学习框架，以实现安全和透明的交通信号控制。该架构并不是通过单一的、平坦的向量处理交通状态，而是将实时交叉口观察数据分解为不同的高维车道实体和相位时间配置，从而固有地保留了交叉口的结构拓扑和几何配置。通过一个双阶段注意力网络动态提取关系依赖和车道间冲突，该网络具有顺序多头交叉注意力和自注意力模块。该设计生成一个实时亲和矩阵，量化信号相位对特定接近流量和排队的直接影响，提供全面的视觉和分析可解释性。为了确保严格的操作可靠性，一个确定性的动作屏蔽接口被直接集成到近端策略优化（Proximal Policy Optimization, PPO）管道中，明确阻止无效的相位转换，以保证绝对遵守既定的信号时序和安全约束。在微观仿真环境中进行评估时，该方法在延误最小化方面优于最先进的基准。更重要的是，产生的注意力权重与既定的交通工程原则精确对齐，为下一代自适应交通控制系统提供了可审计、信任启用和可部署的架构。

View on arXiv Download PDF AI Translation

cs.AI / 32 / 2607.03744

Can Conversational Temporal Dynamics Improve Depression Detection in Dyads? A Preliminary Investigation in Multi-Modality Perspectives

对话时间动态能否改善双人抑郁症检测？多模态视角下的初步研究

Kang, Hanie, Chou, Huang-Cheng, Kadiri, Sudarsana Reddy, Narayanan, Shrikanth

Abstract

Automatic depression detection from clinical interviews typically models the semantic content and acoustic characteristics of participant speech. However, the interactional timing between the clinician and participant remains comparatively under-modeled. We investigate conversational temporal dynamics, specifically dyadic turn-pair timing, as a primary modality fused with self-supervised encoders. Evaluated on the DAIC-WOZ dataset, we compare a compact 24-dimensional timing module against frozen WavLM-large and RoBERTa-large baseline detectors. This temporal module achieves the highest single-modality performance on the development set. Furthermore, a convex-weighted late fusion strategy improves overall performance to 0.804 and 0.669 macro-F1 on the development and test sets, respectively. The learned fusion effectively assigns zero weight to acoustics, demonstrating that conversational timing serves as a lightweight, interpretable complement for dyadic depression screening.

Chinese Translation

自动抑郁症检测通常通过建模临床访谈中参与者的语义内容和声学特征来进行。然而，临床医生与参与者之间的互动时机相对较少被建模。我们研究了对话时间动态，特别是双人轮换对的时机，作为与自监督编码器融合的主要模态。在DAIC-WOZ数据集上进行评估，我们将一个紧凑的24维时机模块与冻结的WavLM-large和RoBERTa-large基线检测器进行了比较。该时机模块在开发集上实现了最高的单模态性能。此外，凸加权的后期融合策略将整体性能提升至0.804和0.669的宏F1分数，分别在开发集和测试集上。学习到的融合有效地将声学特征的权重设为零，表明对话时机作为双人抑郁筛查的轻量级、可解释的补充。

View on arXiv Download PDF AI Translation

cs.AI / 33 / 2607.03748

Bridging Interleaved Multi-Modal Reasoning as a Unified Decision Process

将交错多模态推理作为统一决策过程的桥接

Hu, Zican, Hu, Xuyang, Liu, Yiming, Long, Zuwei, Liu, Wei, Hao, Yunzhuo, Gu, Jiawei, Li, Linjie, Cheng, Yu, Sun, Zhenhong, Gu, Weibo, Sun, Xing, Wang, Zhi

Abstract

Unified multi-modal models (UMMs) have shown promising interleaved text-image reasoning capabilities, yet effectively optimizing such multi-turn generation via reinforcement learning (RL) remains an open challenge. Existing approaches apply RL exclusively to text steps, relegating image generation to supervised surrogates, preventing policy gradients from propagating through the full interleaved trajectory across heterogeneous modalities. This leaves the potential of RL for UMMs largely untapped. In the paper, we introduce \textbf{BRAID} (\textbf{B}ridging inte\textbf{R}le\textbf{A}ved mult\textbf{I}-modal reasoning as a unified \textbf{D}ecision process), a simple framework that casts multi-turn text-image-text reasoning as a unified Markov decision process (MDP), enabling joint optimization of textual and visual generation via a single, principled RL objective. BRAID computes a shared trajectory-level advantage and propagates it coherently into both text tokens and image denoising paths, each optimized through its modality-native policy gradient mechanism. To further address long-horizon credit assignment, BRAID employs a vision-language model (VLM) judge that scores each intermediate image on its reasoning utility, supplying dense turn-level feedback to sharpen learning at critical visual branches. Experiments on spatial reasoning and visual perception benchmarks show that BRAID consistently outperforms various baselines, confirming that a unified MDP formulation with vision-thinking guidance is essential for effective multi-modal reasoning.

Chinese Translation

统一多模态模型（UMMs）在交错文本-图像推理能力方面表现出良好的前景，但通过强化学习（RL）有效优化这种多轮生成仍然是一个未解决的挑战。现有方法仅将RL应用于文本步骤，将图像生成 relegated（归类）为监督替代，阻碍了策略梯度在异构模态的完整交错轨迹中传播。这使得RL在UMMs中的潜力大多未被开发。在本文中，我们提出了 extbf{BRAID}（ extbf{B}ridging inte extbf{R}le extbf{A}ved mult extbf{I}-modal reasoning as a unified extbf{D}ecision process），这是一个简单的框架，将多轮文本-图像-文本推理视为一个统一的马尔可夫决策过程（MDP），通过单一的原则性RL目标实现文本和视觉生成的联合优化。BRAID计算共享的轨迹级优势，并将其一致地传播到文本标记和图像去噪路径中，每个路径通过其模态特有的策略梯度机制进行优化。为了进一步解决长时间跨度的信用分配问题，BRAID采用了一个视觉-语言模型（VLM）评估者，对每个中间图像的推理效用进行评分，提供密集的轮次级反馈，以在关键视觉分支上加快学习。在空间推理和视觉感知基准上的实验表明，BRAID始终优于各种基线，确认了具有视觉思维指导的统一MDP公式对于有效的多模态推理至关重要。

View on arXiv Download PDF AI Translation

cs.AI / 34 / 2607.03787

Folding, Reasoning, and Scaling with Open-source Drug Discovery Engine

利用开源药物发现引擎进行折叠、推理和扩展

project, Aureka AI OpenDDE

Abstract

Accurately modeling biomolecular interactions is a central bottleneck in biology and therapeutic discovery. Here, we introduce Open Drug Discovery Engine (OpenDDE), an open-source, all-atom biomolecular foundation model that uses co-folding as the entry point to a scalable AI-driven drug discovery engine. Rather than treating structure prediction as an isolated endpoint, OpenDDE is designed as a shared structural reasoning layer for modeling sequence-structure-function relationships across biomolecular complexes, enabling complex structure prediction today while providing a foundation for de novo design, affinity estimation, structure-conditioned optimization, and more. OpenDDE integrates advances in all-atom architecture, atomic latent reasoning, inference optimization, and large-scale data processing to achieve IsoDDE-level co-folding accuracy within a reproducible and openly accessible framework. We also identify two scaling-law directions for co-folding models, revealing practical routes for continued improvement through data, model, inference, and training scaling. By releasing training code, inference pipelines, checkpoints, and benchmarks, OpenDDE aims to democratize access to frontier biomolecular intelligence, accelerate global collaboration, and lay an open foundation for next-generation drug discovery systems that can move from predicting molecular structures toward designing, scoring, and optimizing therapeutic candidates for human health.

Chinese Translation

准确建模生物分子相互作用是生物学和治疗发现中的核心瓶颈。在此，我们介绍了开放药物发现引擎（Open Drug Discovery Engine，OpenDDE），这是一个开源的全原子生物分子基础模型，利用共折叠作为可扩展的人工智能驱动药物发现引擎的切入点。OpenDDE并不将结构预测视为一个孤立的终点，而是设计为一个共享的结构推理层，用于建模生物分子复合物中的序列-结构-功能关系，使得今天的复杂结构预测成为可能，同时为从零设计、亲和力估计、结构条件优化等提供基础。OpenDDE整合了全原子架构、原子潜在推理、推理优化和大规模数据处理的进展，以在可重复和公开可访问的框架内实现IsoDDE级别的共折叠精度。我们还确定了共折叠模型的两个扩展法则方向，揭示了通过数据、模型、推理和训练扩展持续改进的实际途径。通过发布训练代码、推理管道、检查点和基准，OpenDDE旨在使前沿生物分子智能的获取民主化，加速全球合作，并为下一代药物发现系统奠定开放基础，使其能够从预测分子结构转向设计、评分和优化人类健康的治疗候选药物。

View on arXiv Download PDF AI Translation

cs.AI / 35 / 2607.03870

Evaluating LLM Uncertainty in Long-Form Generation Using Deterministic Ground Truth

使用确定性真实值评估长文本生成中的大语言模型不确定性

Amit, Ido, Galil, Ido, El-Yaniv, Ran

Abstract

As LLMs generate increasingly long outputs, effective uncertainty estimation must identify errors at fine-grained levels rather than discard entire responses. While such methods exist, evaluating uncertainty at any resolution (token to an entire generation) is challenging and highly sensitive to label imperfections, making zero-noise benchmarks essential; yet, long-form generation benchmarks tend to rely on fallible labels rather than deterministic ground truth. We introduce Single-answer Atomic Long-form Target (SALT), a benchmark of six procedurally generated tasks with single deterministic long textual ground truths, enabling unit-level evaluation of correctness, calibration, and ranking without external judges. Equipped with SALT, our analysis of 50+ LLMs reveals key insights: We identify which confidence functions dominate each uncertainty aspect and show that confidence ranking largely breaks at atomic resolution, even when clearer separability emerges at coarser line-level units. SALT further enables controlled atom-level interventions throughout generation, revealing two separable drivers of future errors: propagation from corrupted prefixes, dominated by global context correctness, and bounded degradation from increasing answer-context length. Finally, we demonstrate that reasoning, via Chain-of-Thought prompting or internalized through training, introduces a trade-off, improving accuracy while degrading confidence ranking. These findings directly impact risk-critical applications requiring reliable error identification and mitigation.

Chinese Translation

随着大语言模型（LLMs）生成越来越长的输出，有效的不确定性估计必须在细粒度层面识别错误，而不是简单地丢弃整个响应。虽然此类方法存在，但在任何分辨率（从单个标记到整个生成）评估不确定性都是具有挑战性的，并且对标签的不完美高度敏感，这使得零噪声基准变得至关重要；然而，长文本生成基准往往依赖于易出错的标签，而非确定性的真实值。我们引入了单答案原子长文本目标（Single-answer Atomic Long-form Target, SALT），这是一个包含六个程序生成任务的基准，具有单一确定性的长文本真实值，使得在没有外部评审的情况下能够进行单位级别的正确性、校准和排名评估。借助SALT，我们对50多个LLM的分析揭示了关键见解：我们识别出哪些置信度函数主导每个不确定性方面，并显示在原子分辨率下置信度排名在很大程度上失效，即使在较粗的行级单位中出现更清晰的可分离性。SALT进一步使得在生成过程中进行受控的原子级干预成为可能，揭示了未来错误的两个可分离驱动因素：来自损坏前缀的传播，主要受全球上下文正确性的影响，以及由于答案上下文长度增加而导致的有限退化。最后，我们展示了推理（通过链式思维提示或通过训练内化）引入了一种权衡，改善了准确性，同时降低了置信度排名。这些发现直接影响到需要可靠错误识别和缓解的风险关键应用。

View on arXiv Download PDF AI Translation

cs.AI / 36 / 2607.03935

Harness-Aware Self-Evolving: Co-Evolving Model Weights, Harness, and Task Solutions

考虑约束的自我进化：共同进化模型权重、约束和任务解决方案

Luo, Haochen, Huang, Yi, Luo, Sichun, Liu, Fengyuan, Li, Lei, Hu, Zefa, Feng, Junlan, Liu, Qi

Abstract

Self-evolving frameworks usually optimize task solutions while treating the surrounding harness as fixed. We introduce Harness-Aware Self-Evolving (HASE), an agentic reinforcement-learning framework in which a single model can generate task solutions or edit selected harness components in a multi-turn action space. HASE enables a single Qwen3-8B model to match the text-classification performance of a GPT-OSS-120B model that uses Claude Code as the harness proposer. In alpha factor mining, HASE outperforms the reported GPT-OSS-120B baseline. HASE also repairs imperfect evaluation components and converges to state-of-the-art performance in circle-packing algorithm discovery. These results show that HASE improves the harness and the solution through one unified agentic process.

Chinese Translation

自我进化框架通常在优化任务解决方案时将周围的约束视为固定。我们引入了考虑约束的自我进化（Harness-Aware Self-Evolving, HASE），这是一个代理强化学习框架，其中单个模型可以在多轮动作空间中生成任务解决方案或编辑选定的约束组件。HASE使得单个Qwen3-8B模型能够匹配使用Claude Code作为约束提议者的GPT-OSS-120B模型的文本分类性能。在阿尔法因子挖掘中，HASE的表现超过了报告的GPT-OSS-120B基线。HASE还修复了不完美的评估组件，并在圆形打包算法发现中收敛到最先进的性能。这些结果表明，HASE通过一个统一的代理过程改善了约束和解决方案。

View on arXiv Download PDF AI Translation

cs.AI / 37 / 2607.03948

Online Linear Programming for Multi-Objective Routing in LLM Serving

大语言模型服务中的多目标路由在线线性规划

Chen, Zixi, Ye, Yinyu, Zhou, Zijie

Abstract

We study the online routing problem in large language model serving, where requests arrive sequentially and must be dispatched to parallel decode workers under tight batch-size and KV-cache constraints. Unlike widely used routing heuristics that are not tied to explicit service-level objectives (SLOs) and offer limited control over latency-throughput trade-offs, we introduce a multi-objective optimization framework that formulates routing as an online linear programming with interpretable decision rewards. We apply an efficient bid-price control policy based on the online linear programming that admits requests when their SLO-weighted benefit exceeds their shadow prices. To meet millisecond decision requirements, we develop a warm-started, projected first-order updates that track the evolving dual shadow prices online with predictable runtime. We integrate our router into the Vidur simulator and demonstrate substantial improvements over standard baselines across multiple SLO regimes, including end-to-end latency, time-to-first-token, throughput, and tail performance. A big picture from our result: a science-based approach outperforms others based on heuristics.

Chinese Translation

我们研究了大语言模型服务中的在线路由问题，其中请求顺序到达，并必须在严格的批量大小和KV缓存限制下分派给并行解码工作者。与广泛使用的路由启发式方法不同，这些方法并未与明确的服务水平目标（SLO）挂钩，并且对延迟-吞吐量权衡提供有限的控制，我们引入了一个多目标优化框架，将路由问题表述为具有可解释决策奖励的在线线性规划。我们应用了一种基于在线线性规划的高效出价控制策略，当请求的SLO加权收益超过其影子价格时接受请求。为了满足毫秒级决策要求，我们开发了一种热启动的投影一阶更新方法，能够在线跟踪不断变化的双重影子价格，并具有可预测的运行时间。我们将我们的路由器集成到Vidur模拟器中，并在多个SLO场景下展示了相较于标准基线的显著改进，包括端到端延迟、首次令牌时间、吞吐量和尾部性能。我们的结果表明：基于科学的方法优于基于启发式的方法。

View on arXiv Download PDF AI Translation

cs.AI / 38 / 2607.04010

Explainable AI for Screening Abuse-Related Trauma in Bangladeshi Children: A Training-Free Multimodal Framework Evaluated on Noise-Aware Synthetic Data

可解释的人工智能在孟加拉国儿童虐待相关创伤筛查中的应用：一种基于无训练的多模态框架，评估于噪声感知的合成数据

Koli, Salma Hoque Talukdar, Jely, Fahima Haque Talukder

Abstract

Bangladesh has an estimated 1.17 mental-health professionals per 100,000 population and only six child psychiatrists nationwide. No Bengali-language, culturally adapted tool exists for early screening of abuse-related psychological trauma in children. We present ShishuRaksha AI, a decision-support (not diagnostic) framework that fuses four screening modalities: validated questionnaires (SDQ, CPSS), Bengali narrative text, House-Tree-Person (HTP) drawing features, and facial affect. The fusion is training-free and clinically weighted, uses cross-modal attention, and includes a single-modality override rule. Every risk score is explained through clinically weighted, perturbation-based additive attribution and rendered as a bilingual (Bangla/English) report with referral routing to national child-protection services (OCC, DSS, NMHH) under the Children Act 2013. No clinical dataset of abused children can be collected ethically at this stage, so we introduce a noise-aware synthetic benchmark (500 cases, 116 positive [23.2%], four deliberate noise layers, literature-grounded HTP priors) and evaluate tree-ensemble surrogates of the fusion design (facial channel excluded) under 5-fold stratified cross-validation. The fused model reaches an AUC of 0.874 [0.834-0.908], against 0.756 [0.705-0.803] for an SDQ-only baseline, with ablation, operating-point, subgroup, and calibration analyses. We state all limitations openly, including synthetic-only data, no held-out set, text-feature circularity, and an urban-rural subgroup gap. This work is a feasibility study and a design contribution toward ethically deployable child-protection screening in low-resource settings.

Chinese Translation

孟加拉国每10万人中仅有约1.17名心理健康专业人员，全国仅有六名儿童精神科医生。目前尚无适合孟加拉语、文化适应的工具用于儿童虐待相关心理创伤的早期筛查。我们提出了ShishuRaksha AI，这是一种决策支持（非诊断）框架，融合了四种筛查模式：经过验证的问卷（SDQ, CPSS）、孟加拉叙述文本、房树人（HTP）绘画特征和面部情感。该融合过程无需训练，并具有临床加权特性，使用跨模态注意力，并包含单一模态覆盖规则。每个风险评分通过临床加权的基于扰动的加性归因进行解释，并以双语（孟加拉语/英语）报告的形式呈现，报告中包含指向国家儿童保护服务（OCC, DSS, NMHH）的转介路径，依据《2013年儿童法》。目前阶段无法伦理地收集虐待儿童的临床数据，因此我们引入了一个噪声感知的合成基准（500个案例，116个阳性[23.2%]，四个故意噪声层，基于文献的HTP先验），并在5折分层交叉验证下评估融合设计的树集成代理（面部通道排除）。融合模型的AUC达到0.874 [0.834-0.908]，而仅使用SDQ的基线为0.756 [0.705-0.803]，并进行了消融、操作点、子组和校准分析。我们公开说明所有限制，包括仅使用合成数据、没有保留集、文本特征的循环性以及城乡子组差距。本研究是一项可行性研究，并为在资源匮乏环境中伦理地部署儿童保护筛查提供了设计贡献。

View on arXiv Download PDF AI Translation

cs.AI / 39 / 2607.04049

What is Left for Us? Second Scholarship Against the Degradation of Research by AI

我们还剩下什么？反对人工智能对研究降级的第二次学术探讨

Novelli, Claudio, Floridi, Luciano

Abstract

We argue that generative AI can degrade research by eroding the very practices through which scholarly judgement is formed and academic trust is built. As constitutive conditions for the production and validation of knowledge, these practices cannot be reduced to the final outputs of research, which is what AI so effectively simulate. Accordingly, when researchers delegate central tasks of inquiry to systems like Large Language Models, they may stop enacting these practices and, with them, lose access to the formation they provide. An individual research output generated by AI may even appear improved but the researcher behind it fails to develop. Against this risk, merely keeping humans in the loop as prompters or quality checkers of AI outputs is insufficient to preserve research as a site of intellectual formation. What is needed instead is a renewed commitment to research as a lived practice in which judgement is formed gradually, often through frictions, and participation in a scholarly community. We defend it because it rests on four sources and warrants of research that cannot be automated: tacit knowledge, personal commitment, socialisation, and deep reading. This practice enacts what we call second scholarship, by which we understand the reappropriation of scholarly craft, chosen out of a critical experience of what generative AI can and cannot do. What cannot and should not be delegated becomes what research communities must value and answer for. This is what is left for us.

Chinese Translation

我们认为，生成性人工智能可能通过侵蚀学术判断形成和学术信任建立的实践来降低研究质量。作为知识生产和验证的构成条件，这些实践不能仅仅归结为研究的最终产出，而这是人工智能所能有效模拟的。因此，当研究人员将核心的探究任务委托给大型语言模型等系统时，他们可能会停止实施这些实践，从而失去其所提供的形成机会。由人工智能生成的个别研究产出可能看起来有所改善，但其背后的研究者却未能得到发展。面对这一风险，仅仅将人类置于环节中，作为人工智能产出的提示者或质量检查者，并不足以维护研究作为智力形成的场所。相反，我们需要重新承诺将研究视为一种生活实践，在这种实践中，判断是逐步形成的，通常通过摩擦和参与学术共同体来实现。我们捍卫这一点，因为它基于四个不可自动化的研究来源和保证：隐性知识、个人承诺、社会化和深度阅读。这一实践体现了我们所称的第二次学术探讨，通过这一探讨，我们理解学术技艺的重新适应，是基于对生成性人工智能所能做和不能做的批判性体验所做出的选择。不可委托的、也不应委托的内容，成为研究共同体必须重视和负责的事项。这就是我们所剩下的。

View on arXiv Download PDF AI Translation

cs.AI / 40 / 2607.04089

PLACEMEM: Toward a Compute-Aware Memory Plane for Lifelong Agents

PLACEMEM：面向终身智能体的计算感知内存平面

Ganguly, Sukanta

Abstract

Lifelong agents need more than larger context windows and better retrieval. They need memories that can persist, evolve, and be corrected without forcing the serving stack to recompute the same history on every turn or silently reuse stale runtime state. We present PLACEMEM as a systems position on lifelong-agent memory, instantiated by an executable control-plane prototype. The central claim is that agent memory should be represented as versioned capsules that unify semantics, provenance, validity, and reusable runtime state under one correction-aware identity. In the current prototype, capsules drive prompt-level text retrieval, KV-aware routing, and cascading invalidation over live streamed backends; prospective layer-frontier replay is intentionally framed as a deeper integration agenda rather than a claimed engine feature. We describe a vLLM-first prototype with persistent capsule state, concurrency-safe invalidation, an OpenAI-compatible routing sidecar, a typed metadata contract, and a benchmark harness that measures live first-token latency, reuse, and post-correction behavior. The result is both an executable artifact that demonstrates correction-aware control-plane behavior today and a concrete roadmap for replay-aware serving integration in future lifelong-agent systems.

Chinese Translation

终身智能体需要的不仅仅是更大的上下文窗口和更好的检索能力。他们需要能够持久、演变并且在不强迫服务栈在每次交互中重新计算相同历史或默默重用过时运行时状态的记忆。我们提出PLACEMEM作为终身智能体内存的系统立场，通过一个可执行的控制平面原型进行实例化。核心论点是，智能体内存应被表示为版本化的胶囊，这些胶囊在一个感知纠错的身份下统一语义、来源、有效性和可重用的运行时状态。在当前原型中，胶囊驱动提示级文本检索、KV感知路由和对实时流后端的级联失效；未来层前沿的重放被有意框定为更深层次的集成议程，而不是声称的引擎特性。我们描述了一个以vLLM为首的原型，具有持久的胶囊状态、并发安全的失效、与OpenAI兼容的路由侧车、类型化的元数据契约，以及一个测量实时首个令牌延迟、重用和后纠错行为的基准工具。其结果既是一个可执行的工件，展示了当今感知纠错的控制平面行为，也是未来终身智能体系统中重放感知服务集成的具体路线图。

View on arXiv Download PDF AI Translation

cs.AI / 41 / 2607.04096

Forethought: Verifiable Reasoning from Neurosymbolic Primitive Programming

前瞻：来自神经符号原始编程的可验证推理

Bhat, Vishvesh, Vaghasiya, Jay, Gonzalez, Emmanuel Anaya

Abstract

Current agentic workflows usually involve decomposing user requests into sequences of tool calls with correctly resolved parameters, the results of which are processed through reasoning traces in the language model's context window. The prevailing route to improve such reasoning is test-time scaling, which trains models to search over long chains of thought; but the resulting capability is entangled in model weights, is not verifiable step-by-step, and is costly at inference. We present Forethought, a neurosymbolic reasoning system that instead treats reasoning as an explicit, verifiable program, that builds from a library of symbolic and neural primitives which are composed through a domain-specific language. The result are reasoning programs, which are concrete representations of the model's work, and as such can be inspected and modified before deployment. Instantiated as a tool-calling execution kernel and evaluated across five benchmarks, Forethought improves base-model accuracy by about 30% relative and outperforms vanilla prompting, reinforcement learning scaffolds, and prompt-evolution methods, enabling small models to match or exceed frontier models capabilities. In a direct comparison, a non-reasoning model augmented with Forethought competes with a dedicated reasoning model while requiring roughly three orders of magnitude less post-training investment, and remains model-agnostic and auditable.

Chinese Translation

当前的智能工作流程通常涉及将用户请求分解为工具调用的序列，并正确解析参数，其结果通过语言模型的上下文窗口中的推理痕迹进行处理。改善这种推理的主要途径是测试时扩展，这种方法训练模型在长链思维中进行搜索；但由此产生的能力与模型权重纠缠在一起，无法逐步验证，并且在推理时成本高昂。我们提出了前瞻（Forethought），一个神经符号推理系统，它将推理视为一个明确的、可验证的程序，该程序基于一个符号和神经原语的库，通过领域特定语言进行组合。最终形成的推理程序是模型工作的具体表现，因此可以在部署前进行检查和修改。作为一个工具调用执行内核实例化，并在五个基准测试中进行评估，前瞻相对于基础模型的准确性提高了约30%，并且优于普通提示、强化学习支架和提示演化方法，使得小模型能够匹配或超越前沿模型的能力。在直接比较中，增强了前瞻的非推理模型与专用推理模型竞争，同时所需的后训练投资大约少三个数量级，并且保持模型无关性和可审计性。

View on arXiv Download PDF AI Translation

cs.AI / 42 / 2607.04156

Language models guide symbolic equation discovery by controlling search

语言模型通过控制搜索引导符号方程发现

Xie, Zikai, Li, Wenmei, Luo, Man, Jiang, Jun, Chen, Linjiang

Abstract

Scientific equation discovery must combine broad domain priors with strict numerical testing. Symbolic regression supplies numerical grounding but faces a combinatorial search space, whereas many language-model systems ask the model to propose or select formulas directly. We test a different division of labour. We compare role specifications in which the language model acts as equation author, candidate decider or search controller, alongside end-to-end language-model and purely numerical baselines. In the controller setting we propose here, implemented as LLM-PySR, language models specify variables, operators, transformations and search depth; symbolic regression enumerates and fits expressions; and deterministic metrics govern retention. Across 74 AI-Feynman equations and seven complex formula-recovery tasks, search control achieved the strongest observed balance of accuracy, complexity, stability and cost. On an independent battery dataset, LLM-PySR identified a compact piecewise-linear relation between early voltage-curve displacement and cycle life. The results suggest that language models should shape hypothesis exploration rather than decide which equations survive.

Chinese Translation

科学方程发现必须将广泛的领域先验与严格的数值测试相结合。符号回归提供了数值基础，但面临组合搜索空间，而许多语言模型系统要求模型直接提出或选择公式。我们测试了一种不同的分工方式。我们比较了语言模型作为方程作者、候选者决策者或搜索控制者的角色规范，同时还与端到端语言模型和纯数值基线进行了对比。在我们提出的控制者设置中，实施为 LLM-PySR，语言模型指定变量、运算符、变换和搜索深度；符号回归枚举并拟合表达式；确定性指标则管理保留。在74个AI-Feynman方程和七个复杂公式恢复任务中，搜索控制实现了观察到的最佳准确性、复杂性、稳定性和成本平衡。在一个独立的电池数据集上，LLM-PySR识别出了早期电压曲线位移与循环寿命之间的紧凑分段线性关系。结果表明，语言模型应当塑造假设探索，而不是决定哪些方程能够存活。

View on arXiv Download PDF AI Translation

cs.AI / 43 / 2607.04184

A Clustering-Based Framework for Identifying Suspicious Trading Patterns in Capital Market

基于聚类的资本市场可疑交易模式识别框架

Zaman, Asif, Sarkar, Romona Magdalene, Ohi, Sabiha Khair, Mobin, Iftekharul

Abstract

Market manipulation is the dubious practice of manipulating stock prices in order to make a quick profit, which truly degrades confidence on trading platforms. We implemented an unsupervised fraud-detection toolkit that begins with K-Means++ clustering to address this issue. A dataset of roughly one million financial transactions from 2012 to 2024 is used. In order to identify fraudulent trades and categorize them using market practice heuristic thresholds, the study suggests a clustering-based pipeline. The method highlights 2.02% of trades as suspicious where 51.10% clearly indicate spoofing, 0.10% indicate pump and dump, 0.55% indicate insider trading, 1.43% indicate a fake breakout, and 46.83% are unclassified. Despite the lack of ground truth, the model's performance is confirmed by a Silhouette Score of 0.561.

Chinese Translation

市场操纵是指通过操纵股票价格以获取快速利润的可疑行为，这确实降低了交易平台的信任度。为了解决这一问题，我们实施了一种无监督的欺诈检测工具包，该工具包以 K-Means++ 聚类为起点。使用的数据集包含2012年至2024年间大约一百万笔金融交易。为了识别欺诈交易并使用市场实践启发式阈值对其进行分类，本研究提出了一种基于聚类的流程。该方法将2.02%的交易标记为可疑，其中51.10%明显表明存在虚假交易，0.10%表明存在拉高出货，0.55%表明存在内幕交易，1.43%表明存在虚假突破，46.83%则未分类。尽管缺乏真实标签，模型的性能通过轮廓系数（Silhouette Score）0.561得到了验证。

View on arXiv Download PDF AI Translation

cs.AI / 44 / 2607.04219

Agentic IoT: Architectures, Applications, and Challenges Toward the Internet of Agents

自主物联网：面向代理互联网的架构、应用与挑战

Sevinç, Rümeysa Hilal, Türkoğlu, Bahaeddin, Kök, İbrahim

Abstract

The integration of AI into Internet of Things (AIoT) systems has gradually transformed them from passive data collection infrastructures into intelligent systems capable of anomaly detection, predictive maintenance, classification, forecasting, and optimization. However, most existing solutions still rely on task-specific models that infer from sensor data; thus, system-wide capabilities such as real-time reasoning, adaptive planning, autonomous coordination, learning, tool use, and contextual decision-making remain limited. This paper examines Agentic IoT as a next-generation cognitive IoT paradigm that integrates the perception, reasoning, planning, learning, and action capabilities of autonomous AI agents with cyber-physical systems. Agentic IoT aims to transform IoT from data-centric sensing and inference infrastructures into distributed cognitive agent ecosystems operating across the device/edge-fog-cloud continuum. The paper first grounds this transition as a paradigm shift and positions Agentic IoT in relation to AIoT, edge intelligence, multi-agent systems, and the Internet of Agents. It then systematically reviews current studies, presents a holistic architectural framework, discusses domain-specific application potential, and identifies key technical, operational, and research challenges together with future research directions.

Chinese Translation

人工智能（AI）与物联网（IoT）系统的融合逐渐将其从被动的数据收集基础设施转变为能够进行异常检测、预测性维护、分类、预测和优化的智能系统。然而，大多数现有解决方案仍然依赖于从传感器数据推断的特定任务模型，因此，系统范围内的能力，如实时推理、自适应规划、自治协调、学习、工具使用和上下文决策，仍然有限。本文探讨了自主物联网（Agentic IoT）作为下一代认知物联网范式，整合了自主AI代理的感知、推理、规划、学习和行动能力与网络物理系统。自主物联网旨在将物联网从以数据为中心的感知和推理基础设施转变为在设备/边缘-雾-云连续体上运行的分布式认知代理生态系统。本文首先将这一转变确立为范式转变，并将自主物联网与人工智能物联网（AIoT）、边缘智能、多代理系统和代理互联网进行关联。接着，系统性地回顾当前研究，提出一个整体架构框架，讨论特定领域的应用潜力，并识别关键的技术、操作和研究挑战，以及未来的研究方向。

View on arXiv Download PDF AI Translation

cs.AI / 45 / 2607.04222

Unsupervised Features Mining via Activation Geometry

通过激活几何进行无监督特征挖掘

LeVi, Amit, David, Elad, Fomin, Max

Abstract

Interpretability methods aim to reveal the features represented inside large language models (LLMs). Many existing methods begin with labeled examples of a human-defined concept that may reflect human biases, and then identify how that concept is represented within the model, for example in its activation space or through other decomposition methods. We introduce \emph{Mining via Activation Geometry} (MAG), a simple unsupervised framework for extracting reasoning features from model activations by prepending the same natural-language instruction $Q$ to every input $p$, where $Q$ defines the reasoning feature of interest, such as ``Can this object be found in the desert?'' or ``Is this prompt malicious?'' We measure how the instruction changes the model's internal representation using $m(Q \mid p) - m(p)$ at a single readout point. We explore eight different MAGs. The extracted reasoning features predict the models' own world understanding and judgment, can be approximated into a single activation direction, we found that some features are more linearly represented and some less, this linear representation, which is vector steering, can change the LLMs' decisions through activation steering by injecting reasoning features. Finally, we use the same method to select the best training datasets for prompt-injection classifier probes: while similarity between ordinary activations is almost unrelated to downstream performance, RFD-based similarity achieves $94.7\%$ Top-1 and $100\%$ Top-2 accuracy.

Chinese Translation

可解释性方法旨在揭示大型语言模型（LLMs）内部所表示的特征。许多现有方法以人类定义概念的标记示例为起点，这些概念可能反映人类偏见，然后识别该概念在模型中的表示方式，例如在其激活空间中或通过其他分解方法。我们提出了 extit{通过激活几何挖掘}（MAG），这是一种简单的无监督框架，通过将相同的自然语言指令$Q$预先添加到每个输入$p$中，从模型激活中提取推理特征，其中$Q$定义了感兴趣的推理特征，例如“这个物体能在沙漠中找到吗？”或“这个提示是恶意的吗？”我们通过在单个读取点使用$m(Q ext{ } | ext{ } p) - m(p)$来测量指令如何改变模型的内部表示。我们探索了八种不同的MAG。提取的推理特征预测模型自身的世界理解和判断，可以近似为单一的激活方向，我们发现某些特征的线性表示更强，而某些则较弱，这种线性表示，即向量引导，可以通过注入推理特征来改变LLMs的决策。最后，我们使用相同的方法选择最佳的训练数据集用于提示注入分类器探针：尽管普通激活之间的相似性与下游性能几乎无关，但基于RFD的相似性实现了$94.7\%$的Top-1和$100\\%$的Top-2准确率。

View on arXiv Download PDF AI Translation

cs.AI / 46 / 2607.04240

Biological Motifs for Agentic Control

代理控制的生物模式

Banu, Bogdan

Abstract

The transition of Large Language Models (LLMs) from passive generators to autonomous agents has introduced significant challenges in reliability, security, and state management. Current agentic architectures are often constructed ad-hoc, prone to hallucination cascades, infinite loops, and prompt injection attacks. This paper argues that many of these failure modes can be analyzed using control motifs long studied in systems biology, provided the comparison is made at the level of typed interfaces and coordination structure rather than literal biological mechanism. We develop a typed interface correspondence between Gene Regulatory Networks and agentic software systems using polynomial functors and wiring diagrams. Five biological motifs are mapped to composable software design patterns: Coherent Feed-Forward Loops for noise suppression, Adaptive Immunity for layered security, Mitochondrial Signaling for resource governance, Endosymbiosis for neuro-symbolic integration, and Morphogen Diffusion for spatially varying coordination. An epistemic topology layer derives Kripke-style knowledge operators from the wiring diagram's observation structure and proves four predictive theorems for multi-agent scaling. The core contributions are: (1) the Agentic Operad, a typed syntax for agent composition with provable error suppression bounds for feed-forward topologies; (2) an epistemic topology with four theorems (error amplification, sequential penalty, parallel acceleration, and tool density scaling) whose qualitative predictions are consistent with published multi-agent benchmarks; and (3) a six-layer progression from structure through development, grounded in autonomous learning frameworks and convergence proxies from the empirical literature. A reference implementation with 1,813 tests and 116 examples illustrates practical feasibility.

Chinese Translation

大型语言模型（LLMs）从被动生成器转变为自主代理引入了在可靠性、安全性和状态管理方面的重大挑战。目前的代理架构通常是临时构建的，容易出现幻觉级联、无限循环和提示注入攻击。本文认为，许多这些失效模式可以通过长期研究的系统生物学中的控制模式进行分析，前提是比较是在类型接口和协调结构的层面上，而不是字面生物机制的层面。我们利用多项式函子和接线图开发了基因调控网络与代理软件系统之间的类型接口对应关系。五种生物模式被映射为可组合的软件设计模式：用于噪声抑制的相干前馈环路、用于分层安全的自适应免疫、用于资源治理的线粒体信号传导、用于神经符号集成的内共生以及用于空间变化协调的形态发生扩散。一个认知拓扑层从接线图的观察结构中推导出Kripke风格的知识运算符，并证明了四个关于多代理扩展的预测定理。核心贡献包括：（1）代理操作代数（Agentic Operad），一种用于代理组合的类型语法，具有可证明的前馈拓扑误差抑制界限；（2）一个具有四个定理（误差放大、顺序惩罚、并行加速和工具密度扩展）的认知拓扑，其定性预测与已发布的多代理基准一致；（3）一个从结构到发展的六层进展，基于自主学习框架和来自实证文献的收敛代理。一个包含1,813个测试和116个示例的参考实现展示了实际可行性。

View on arXiv Download PDF AI Translation

cs.AI / 47 / 2607.04242

Progress- and Reliability-Oriented Group Policy Optimization for Agentic Reinforcement Learning

面向进展和可靠性的群体策略优化用于自主强化学习

Fan, Mingxuan, Liu, Peiyang

Abstract

Group-based reinforcement learning (RL) has become an effective paradigm for improving large language model agents on long-horizon interactive tasks. To obtain finer-grained policy updates than trajectory-level optimization, recent work has moved toward step-level group-based RL, where intermediate steps are grouped and compared within a rollout batch. However, step-level advantage estimation is sensitive to how groups are formed: grouping by broad state keys improves coverage but may compare actions taken under different histories, while enforcing historical consistency yields fairer comparisons at the cost of fragmented groups and missing peer-comparison signal. In this paper, we propose ProGPO (Progress- and Reliability-Oriented Group Policy Optimization), a learned-critic-free method for context-consistent step-level learning. ProGPO keeps exact-prefix action comparison, and complements sparse peer comparisons with transition credit derived from rollout-based state potentials. To estimate these potentials reliably, ProGPO combines semantic expansion with inverse-variance fusion across history depths. We evaluate ProGPO on two challenging agentic tasks, ALFWorld and WebShop, with Qwen2.5-1.5B-Instruct. Results show that ProGPO improves over matched agentic RL baselines under comparable computational overhead, and additional Qwen2.5-3B-Instruct experiments further test the scalability of the proposed method.

Chinese Translation

基于群体的强化学习（RL）已成为提升大型语言模型代理在长时间交互任务中表现的有效范式。为了获得比轨迹级优化更细粒度的策略更新，近期的研究转向了基于步骤的群体强化学习，其中中间步骤在回放批次内被分组并进行比较。然而，步骤级优势估计对群体的形成方式非常敏感：通过广泛的状态键进行分组可以提高覆盖率，但可能会比较在不同历史下采取的动作，而强制历史一致性则在产生更公平比较的同时导致群体碎片化和缺失同伴比较信号。在本文中，我们提出了ProGPO（面向进展和可靠性的群体策略优化），这是一种无学习评论员的上下文一致步骤级学习方法。ProGPO保持精确前缀动作比较，并通过基于回放的状态潜力衍生的过渡信用来补充稀疏的同伴比较。为了可靠地估计这些潜力，ProGPO结合了语义扩展和跨历史深度的逆方差融合。我们在两个具有挑战性的自主任务ALFWorld和WebShop上评估ProGPO，使用Qwen2.5-1.5B-Instruct。结果表明，ProGPO在可比的计算开销下优于匹配的自主强化学习基线，额外的Qwen2.5-3B-Instruct实验进一步测试了所提方法的可扩展性。

View on arXiv Download PDF AI Translation

cs.AI / 48 / 2607.04261

Shortcut Learning in Legal Judgment Prediction: Empirical Evidence from the UK Employment Tribunal

法律判决预测中的捷径学习：来自英国就业法庭的实证证据

Watson, Joe, de Faria, Joana Ribeiro, Tomalin, Marcus, Magnusson, Måns, Xie, Huiyuan, Yeung, Hao Tian, Steffek, Felix

Abstract

Current Legal Judgment Prediction (LJP) is constrained by its reliance on post-hoc judicial materials, increasing the likelihood that models perform retrospective classification rather than true forecasting. This paper empirically investigates shortcut learning in this context by studying claim-level outcome prediction in UK Employment Tribunal (UKET) decisions. Using a corpus of 33,158 individual claims, we predict outcomes from claim texts and LLM-extracted case summaries, evaluating models ranging from interpretable TF-IDF-based classifiers to black-box LLMs. While headline predictive performance figures appear strong, we demonstrate that such performance in LJP systems trained on post-hoc judicial text can be driven by the retrospective nature of the source material. Stratifying the test data by human judgments of leakage reveals that performance increases where outcome-revealing cues are embedded in the narrative. Moreover, a model trained on just the 4% of features identified as leakage achieves high performance, outperforming human experts. These findings substantiate concerns that LJP performance may be exaggerated by linguistic artefacts. Yet this vulnerability is not fatal to the research agenda. Instead, post-hoc judgments might be treated as potentially contaminated texts, requiring active auditing. Retraining models after masking leakage features results in only a negligible reduction in Macro-F1. Hence, while models will opportunistically exploit shortcuts when available, they remain capable of extracting useful predictive signals when these artefacts are removed.

Chinese Translation

当前的法律判决预测（LJP）受到对事后司法材料依赖的限制，增加了模型进行回顾性分类而非真实预测的可能性。本文通过研究英国就业法庭（UKET）裁决中的索赔级结果预测，实证调查了这一背景下的捷径学习。我们使用33,158个个体索赔的语料库，从索赔文本和大规模语言模型（LLM）提取的案件摘要中预测结果，评估从可解释的基于TF-IDF的分类器到黑箱LLM的多种模型。尽管表面上的预测性能数据似乎强劲，但我们证明了在基于事后司法文本训练的LJP系统中，这种性能可能受到源材料回顾性特征的驱动。通过人类对泄露判断的分层测试数据揭示，当结果揭示线索嵌入叙述中时，性能会有所提升。此外，仅在被识别为泄露的4%特征上训练的模型实现了高性能，超越了人类专家。这些发现证实了对LJP性能可能因语言伪影而被夸大的担忧。然而，这一脆弱性并不致命于研究议程。相反，事后判断可以被视为潜在污染的文本，需要积极审计。在掩盖泄露特征后重新训练模型仅导致Macro-F1的微小下降。因此，尽管模型在可用时会机会性地利用捷径，但在去除这些伪影后，它们仍然能够提取有用的预测信号。

View on arXiv Download PDF AI Translation

cs.AI / 49 / 2607.04292

Agentic SABRE: An Uncertainty-Aware Neuro-Symbolic Multi-Agent Framework for Adaptive Ransomware Detection

Agentic SABRE：一种基于不确定性的神经符号多代理框架用于自适应勒索软件检测

Kabuye, Henry, Issac, Biju, Neera, Jeyamohan

Abstract

Ransomware has evolved into a complex, adaptive, and fast-moving adversary category in which static signatures and monolithic classifiers fail to generalise under concept drift, evasion, and behavioural polymorphism. In this paper, we present Agentic SABRE (Semantic-Behavioural Arbitration for Ransomware Evaluation), an uncertainty-aware, neuro-symbolic, multi-agent framework for adaptive ransomware detection. SABRE fuses semantic, representation-based evidence with behavioural, time-window forensic telemetry and employs Monte Carlo Dropout inference to quantify epistemic uncertainty for each agent. We introduce a decision-layer orchestrator that performs risk- and uncertainty-aware triage using two interpretable thresholds: a risk score and an uncertainty budget. High-confidence, high-risk samples are automatically contained, while uncertain or borderline cases are escalated to human analysts, establishing a flexible computational contract between autonomous response and analyst oversight. To support auditability and trust, SABRE integrates post-hoc explainability mechanisms, including gradient saliency, permutation importance, and counterfactual analysis, enabling both local and global interpretation of agent decisions. Extensive evaluation on RDset and RanSMAP demonstrates that Agentic SABRE preserves perfect discrimination on saturated semantic datasets, with AUC equal to 1.0, while improving robustness under weak behavioural signals. It achieves up to a 4.9 percent relative reduction in false escalations at equal recall while maintaining calibrated predictive uncertainty. Counterfactual analysis further shows that semantic and behavioural decisions can be reversed with bounded perturbation cost, indicating stable and interpretable decision boundaries.

Chinese Translation

勒索软件已经演变为一种复杂、自适应且快速发展的对手类别，在概念漂移、规避和行为多态性下，静态签名和单一分类器无法有效泛化。本文提出了Agentic SABRE（语义-行为仲裁用于勒索软件评估），这是一种基于不确定性的神经符号多代理框架，用于自适应勒索软件检测。SABRE将基于语义的表示证据与基于行为的时间窗口取证遥测相融合，并采用蒙特卡洛丢弃推理来量化每个代理的认知不确定性。我们引入了一个决策层协调器，使用两个可解释的阈值进行风险和不确定性意识的分诊：风险评分和不确定性预算。高置信度、高风险的样本会被自动隔离，而不确定或边缘案例则会升级至人工分析师，从而在自主响应与分析师监督之间建立灵活的计算契约。为了支持审计性和信任，SABRE集成了事后可解释性机制，包括梯度显著性、置换重要性和反事实分析，使得代理决策的局部和全局解释成为可能。在RDset和RanSMAP上的广泛评估表明，Agentic SABRE在饱和的语义数据集上保持了完美的区分能力，AUC值为1.0，同时在弱行为信号下提高了鲁棒性。它在保持校准预测不确定性的同时，实现了高达4.9%的相对减少虚假升级的效果。反事实分析进一步表明，语义和行为决策可以在有限的扰动成本下被逆转，表明决策边界稳定且可解释。

View on arXiv Download PDF AI Translation

cs.AI / 50 / 2607.04329

HAS-Bench: Evaluating LLM-Based Human-Agent Systems under Configurable Human Participation

HAS-Bench：在可配置人类参与下评估基于大型语言模型的人机系统

Wu, Yaozu, Huang, Wei-Chieh, Guo, Jizhou, Li, Dongyuan, Jiang, Renhe, Zou, Henry Peng, Miao, Chunyu, Li, Shanghao, Zhang, Weizhi, Ye, WeiWei, Chen, Yankai, Zhang, Meng, Liu, Xue, Yu, Philip S.

Abstract

Large language models increasingly operate in settings where humans are active collaborators rather than passive task providers. We introduce HAS-Framework, a graph-based framework that represents humans and LLM-powered agents as first-class participants with explicit roles, permissions, communication paths, and action authority. Building on this framework, HAS-Bench evaluates Human-Agent Systems under configurable human participation across agency levels, interaction channels, and persona policies. The benchmark measures both task outcomes and process-level collaboration behavior, including clarification quality, feedback utilization, control calibration, safety, initiative, and interaction cost. Experiments across six domains show that human participation can substantially improve task completion and failure recovery, but the gains depend on when, how, and by whom human input is exercised.

Chinese Translation

大型语言模型越来越多地在一个人类作为积极合作者而非被动任务提供者的环境中运行。我们引入了HAS-Framework，这是一个基于图的框架，将人类和基于大型语言模型（LLM）的智能体视为具有明确角色、权限、沟通路径和行动权威的一类参与者。在此框架的基础上，HAS-Bench评估在可配置人类参与下的人机系统，涵盖代理级别、互动渠道和角色政策。该基准测量任务结果和过程级协作行为，包括澄清质量、反馈利用、控制校准、安全性、主动性和互动成本。跨六个领域的实验表明，人类参与可以显著提高任务完成率和失败恢复能力，但收益取决于人类输入的时机、方式和提供者。

View on arXiv Download PDF AI Translation

cs.AI / 51 / 2607.04334

Do GUI Agents Believe Their Eyes? Diagnosing State-Belief Reliance on Pixels versus Structure

图形用户界面代理是否相信自己的眼睛？诊断状态信念对像素与结构的依赖

Zhang, Guijia, Yang, Harry

Abstract

Multimodal GUI agents read an interface through two redundant channels: the rendered pixels of a screenshot and a serialized structure such as a DOM or accessibility tree. Before acting, an agent forms a belief about the current interface state, but existing benchmarks score task success, element grounding, or attack resistance and do not ask whether that belief is drawn from the pixels. We formalize visual state reliance, the attribution of a state belief to pixels, structure, or priors, and measure it with paired single-channel interventions over 310 real web, mobile, and desktop probes. Every probe is scored by deterministic forced choice, with no model-generated item and no model judge. Our central metric is the Perception-Fusion Gap, the fraction of probes a model perceives correctly yet resolves toward structure under conflict. Across five models from three vendors, textual state beliefs defer to structure while image-only accuracy stays near ceiling, and Perception-Fusion Gap is positive for every model; non-text identity, by contrast, stays largely pixel-bound. The substitution is specific to the serialized-text and indexed-action channel, and coordinate-action agents are largely immune. For textual conflicts, a white-box ablation traces the effect to a single copied structural value, and in two live environments the conflict drives wrong actions and real task failure. Visual state reliance therefore gives a measurable diagnostic of whether agent state beliefs are visually grounded, and the errors it exposes propagate to actions.

Chinese Translation

多模态图形用户界面（GUI）代理通过两个冗余通道读取界面：屏幕截图的渲染像素和序列化结构，如文档对象模型（DOM）或可访问性树。在采取行动之前，代理会形成对当前界面状态的信念，但现有基准仅评估任务成功、元素定位或攻击抵抗力，并未询问该信念是否源于像素。我们形式化了视觉状态依赖，即将状态信念归因于像素、结构或先验，并通过对310个真实网页、移动和桌面探测器进行成对单通道干预来进行测量。每个探测器通过确定性强制选择进行评分，没有模型生成的项目和模型评审。我们的核心指标是感知融合差距（Perception-Fusion Gap），即模型在冲突情况下正确感知但向结构倾斜的探测器比例。在来自三家供应商的五个模型中，文本状态信念倾向于结构，而仅图像的准确性接近上限，且每个模型的感知融合差距均为正值；相比之下，非文本身份在很大程度上仍然依赖于像素。该替代特定于序列化文本和索引动作通道，而坐标动作代理在很大程度上免疫。对于文本冲突，白盒消融追踪其影响至单个复制的结构值，并且在两个实时环境中，冲突导致错误的行动和实际任务失败。因此，视觉状态依赖提供了一个可测量的诊断工具，以判断代理的状态信念是否在视觉上扎根，而它所暴露的错误会传播到行动中。

View on arXiv Download PDF AI Translation

cs.AI / 52 / 2607.04336

Server-side Anti-cheat in FPS games for Aimbot detection using Deep learning and Machine learning

基于深度学习和机器学习的FPS游戏服务器端反作弊系统：瞄准辅助检测

Dhinge, Siddhesh A., Sukum, Shubham G., Ranjane, Harsh S., Rajput, Ruturajsingh R., Jadhav, Jyoti H.

Abstract

Modern video games are becoming more complex day by day. Most of these modern games are multiplayer first-person shooter (FPS) games. The rising popularity of FPS games emphasizes the need to combat cheating for fair and enjoyable gaming. As the number of players using cheating techniques like aimbots, wallhacks, and speed hacks is also increasing, we need a way to detect players who are using cheating tools to gain an unfair advantage over regular players. In this system, we focus exclusively on detecting aimbot cheats. Players who use aimbot cheats generally do not prioritize other aspects of the game. To distinguish between regular and cheating players, we identify specific features encompassing time-series data such as aim velocity, number of shots, distance to target, and more, along with behavioral data such as utility usage, player movement, and other gameplay patterns. Utilizing these features, we construct a server-side aimbot detection classifier named 'YAACS'. YAACS comprises a parser, a deep learning model, and intermediary connection utilities designed for integration with the game server. The proposed system achieves a classification accuracy of 88.6% with a false positive rate of 0.97% using a Stacked LSTM with Dense layers trained on sequences of 128 ticks (Tick Delta Negative=56, Tick Delta Positive=24), outperforming the Decision Tree baseline which achieves a higher accuracy of 96.2% but at a false positive rate of 2.68%, 2.76x worse than the best LSTM configuration. These results demonstrate that incorporating temporal context through sequence modelling is critical for minimising false accusations in FPS cheat detection.

Chinese Translation

现代视频游戏日益复杂，其中大多数是多人第一人称射击（FPS）游戏。FPS游戏的日益普及强调了打击作弊以确保公平和愉快游戏体验的必要性。随着使用瞄准辅助（aimbots）、透视（wallhacks）和加速（speed hacks）等作弊技术的玩家数量不断增加，我们需要一种方法来检测那些利用作弊工具获取不公平优势的玩家。在本系统中，我们专注于检测瞄准辅助作弊。使用瞄准辅助的玩家通常不会优先考虑游戏的其他方面。为了区分普通玩家和作弊玩家，我们识别了包括瞄准速度、射击次数、与目标的距离等时间序列数据，以及实用工具使用、玩家移动和其他游戏模式等行为数据的特征。利用这些特征，我们构建了一个名为'YAACS'的服务器端瞄准辅助检测分类器。YAACS包括一个解析器、一个深度学习模型和为与游戏服务器集成而设计的中间连接工具。所提议的系统在使用训练于128个时间戳序列（Tick Delta Negative=56, Tick Delta Positive=24）的堆叠LSTM与全连接层时，达到了88.6%的分类准确率，假阳性率为0.97%，超越了决策树基线，该基线的准确率为96.2%，但假阳性率为2.68%，比最佳LSTM配置差2.76倍。这些结果表明，通过序列建模引入时间上下文对于最小化FPS作弊检测中的错误指控至关重要。

View on arXiv Download PDF AI Translation

cs.AI / 53 / 2607.04371

Nemotron-Labs-3-Puzzle-75B-A9B: Compressing Hybrid MoE LLMs

Nemotron-Labs-3-Puzzle-75B-A9B：压缩混合 MoE 大型语言模型

Bercovich, Akhiad, Abramovich, Talor, Afrimi, Daniel, Aharon, Shay, Ailon, Nir, Anisimov, Vladimir, Argov, Omer Ullman, Ashkenazi, Maor, Asida, Tomer, Assaf, Nave, Natan, Tomer Bar, Bukharin, Alexander, Chlebus, Grzegorz, Chochowski, Marcin, Chung, Eric, Dabbah, Mohammad, del Mundo, Carlo, Dobrowolska, Ewa, Galil, Ido, Galron, Yaniv, Geifman, Amnon, Geifman, Yonatan, Golan, Izik, Gronskiy, Alex, Grzegorzek, Tomasz, Haber, Netanel, Kadoch, Lior, Karch, Grzegorz, Keren, Tomer, Khattar, Abhinav, Klein, Amir, Konuk, Tugrul, Koren, Roi, Korzekwa, Daniel, Kotek, Shaun, Krommydas, Konstantinos, Levy, Itay, Masad, Ofri, Miron, Yoav, Molchanov, Pavlo, Mor, Shahar, Moshe, Zach, Muralidharan, Saurav, Nabwani, Najeeb, Nushi, Besmira, Patwary, Mostofa, Puny, Omri, Rausch, Johannes, Ronen, Tomer, Sameni, Sepehr, Schen, Itamar, Segal, Elad, Serebrenik, Daniel, Shahaf, Ido, Singhal, Soumye, Sorokin, Daniil, Sreenivas, Sharath Turuvekere, Stepniewska-Dziubinska, Marta, Taghibakhshi, Ali, Tajbakhsh, Nima, Tropp, Oren, Tzur, Dor, Warno, Anna, Wu, Yi-Fu, Zawalski, Michal, Zeng, Jiaqi, Zhang, Yian, Zilberstein, Ran, Zuker, Amit, El-Yaniv, Ran

Abstract

We present Nemotron-Labs-3-Puzzle-75B-A9B, a compressed variant of Nemotron-3-Super optimized for interactive deployment. We designed the model to maximize server throughput under high user throughput constraints. In interactive serving workloads on a single 8xB200 node, Puzzle-75B-A9B achieves approximately 2x higher server throughput than Nemotron-3-Super at matched user throughput constraints. In ultra-long-context deployment on a single H100 GPU, the compressed model increases 1M-token concurrency from 1 request to 8 requests. Puzzle-75B-A9B is constructed using a multi-stage pipeline that combines the Iterative Puzzle compression framework with knowledge distillation, reinforcement learning, quantization, and a Multi-Token Prediction head. The compression process jointly optimizes heterogeneous MoE pruning, active parameter budget, and Mamba pruning to improve inference efficiency while preserving model quality. We evaluate Puzzle-75B-A9B on a broad suite of reasoning, coding, multilingual, long-context, and agentic benchmarks. Despite substantial compression, the model retains strong downstream accuracy relative to the parent model across a wide range of tasks. These results demonstrate that large hybrid MoE models can be substantially optimized for deployment efficiency while maintaining strong downstream capability.

Chinese Translation

我们提出了 Nemotron-Labs-3-Puzzle-75B-A9B，这是一个针对交互式部署优化的 Nemotron-3-Super 的压缩变体。我们设计该模型以在高用户吞吐量约束下最大化服务器吞吐量。在单个 8xB200 节点上的交互式服务工作负载中，Puzzle-75B-A9B 在匹配用户吞吐量约束的情况下，服务器吞吐量比 Nemotron-3-Super 高出约 2 倍。在单个 H100 GPU 上的超长上下文部署中，压缩模型将 1M 令牌的并发请求从 1 个增加到 8 个。Puzzle-75B-A9B 采用多阶段管道构建，结合了迭代拼图压缩框架、知识蒸馏、强化学习、量化和多令牌预测头。压缩过程共同优化了异构 MoE 剪枝、活跃参数预算和 Mamba 剪枝，以提高推理效率，同时保持模型质量。我们在广泛的推理、编码、多语言、长上下文和智能基准测试上评估了 Puzzle-75B-A9B。尽管进行了大幅压缩，该模型在广泛任务中相对于母模型仍保持强大的下游准确性。这些结果表明，大型混合 MoE 模型可以在保持强大下游能力的同时，显著优化部署效率。

View on arXiv Download PDF AI Translation

cs.AI / 54 / 2607.04389

Decentralized Aggregation of LLM Predictions via Wagering Mechanisms

通过投注机制实现大规模语言模型预测的去中心化聚合

Luo, Yuhong, Pennock, David M., Wang, Xintong

Abstract

It is increasingly common to aggregate predictions from multiple LLMs, each with domain expertise or access to private tools and data, to improve collective prediction performance. In decentralized settings, aggregation weights need to be determined without access to models' private information and should remain robust to strategic reporting. We propose a family of advantage-aligned wagering mechanisms for LLM aggregation (WALLA), in which each model reports a prediction and a learned wager, and predictions are aggregated using wagers as weights. WALLA introduces a leave-one-out baseline into the net payout function, yielding three desirable properties: (1) dominant-strategy incentive compatibility of prediction under arbitrary belief structure, (2) advantage--wager alignment, where the optimal wager is proportional to the model's expected score advantage, and (3) prediction-agnostic wager optimization, enabling decentralized learning of wager policies without requiring optimal predictions. We further instantiate two mechanism variants that trade off normality and no-arbitrage while maintaining a bounded worst-case deficit for the mechanism. Experiments on question-answering and forecasting benchmarks across heterogeneous models and private-information settings show that WALLA matches centralized aggregation methods in predictive performance, while simultaneously achieving decentralized learning, advantage-aligned aggregation weights, uncertainty awareness, and incentive-compatible prediction.

Chinese Translation

从多个大规模语言模型（LLM）聚合预测已变得越来越普遍，这些模型各自具有领域专业知识或访问私有工具和数据，以提高集体预测性能。在去中心化环境中，聚合权重需要在没有访问模型私有信息的情况下确定，并应对战略性报告保持稳健性。我们提出了一系列优势对齐的投注机制用于LLM聚合（WALLA），其中每个模型报告一个预测和一个学习到的投注，并使用投注作为权重进行预测聚合。WALLA在净支付函数中引入了留一法基线，产生了三个理想特性：（1）在任意信念结构下的预测主导策略激励相容性，（2）优势-投注对齐，其中最优投注与模型的预期得分优势成比例，以及（3）与预测无关的投注优化，使得无需最优预测即可实现投注策略的去中心化学习。我们进一步实例化了两种机制变体，在保持机制的最坏情况损失有界的同时，权衡了常态性和无套利性。在异构模型和私有信息设置下的问答和预测基准实验表明，WALLA在预测性能上与集中聚合方法相匹配，同时实现了去中心化学习、优势对齐的聚合权重、不确定性意识和激励相容的预测。

View on arXiv Download PDF AI Translation

cs.AI / 55 / 2607.04394

MechMath Agent Team: LLM Driven Agents for Mathematical Research

MechMath代理团队：基于大型语言模型的数学研究代理

Cao, Yichuan, Qiu, Ruichen, Liu, Junqi, Wang, Jiaqi, Guo, Dakai, Feng, Ruyong, Zhi, Lihong, Gao, Xiao-Shan

Abstract

AI reasoning has become a central focus in contemporary artificial intelligence, largely driven by the success of large language models. However, mathematical research, which is characterized by non-linear derivation paths, rigorous logical requirements, and protracted exploration cycles, poses severe challenges for existing reasoning systems. To overcome these limitations, we present the MechMath Agent Team (MMAT), which is a large language model driven agent designed to serve as a co-pilot throughout the full cycle of mathematical research. We design a tripartite Harness Architecture that decouples system responsibilities into Control, Execution, and Augmentation planes, thereby reconciling rigorous logical control with the agility demanded by open-ended research. Building upon this framework, we instantiate three specialized agents: a Knowledge Base Manager, a Natural Language Prover, and a Formal Language Prover, all operating in a closed loop to produce formally certified mathematical proofs. We evaluate MMAT on open problems in Number Theory, Algebraic Complexity Theory, Differential Algebra, Operator Algebra, and Inequalities. Across a two-month deployment, 11 problems have been solved, demonstrating its capacity to act as a co-pilot throughout the entire research cycle. The contributions are threefold: a general decoupled Harness Architecture for multi-agent mathematical reasoning, its concrete instantiation in the MMAT system, and empirical validation on a diverse suite of open problems.

Chinese Translation

人工智能推理已成为当代人工智能的核心焦点，这在很大程度上得益于大型语言模型的成功。然而，数学研究的特点是非线性推导路径、严格的逻辑要求和漫长的探索周期，这对现有的推理系统提出了严峻的挑战。为了解决这些局限性，我们提出了MechMath代理团队（MMAT），这是一个基于大型语言模型的代理，旨在作为数学研究全周期的副驾驶。我们设计了一个三方位的Harness架构，将系统职责解耦为控制、执行和增强平面，从而调和严格的逻辑控制与开放式研究所需的灵活性。在此框架的基础上，我们实例化了三个专门的代理：知识库管理器、自然语言证明器和形式语言证明器，它们在闭环中操作，以生成形式认证的数学证明。我们在数论、代数复杂性理论、微分代数、算子代数和不等式等开放问题上评估了MMAT。在为期两个月的部署中，解决了11个问题，展示了其在整个研究周期中作为副驾驶的能力。贡献主要体现在三个方面：为多代理数学推理提供了一种通用的解耦Harness架构，在MMAT系统中的具体实例化，以及在多样化的开放问题套件上的实证验证。

View on arXiv Download PDF AI Translation

cs.AI / 56 / 2607.04412

LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RL

LLM作为导师：面向策略的提示适应用于不可验证的强化学习

Kim, Yujin, Ho, Namgyu, Hwang, Sangmin, Kim, Joonkee, Yang, Yongjin, Bae, Sangmin, Kim, Seungone, Jung, Jaehun, Yun, Se-Young, Song, Hwanjun

Abstract

Reinforcement learning (RL) for non-verifiable instruction following increasingly relies on LLM judges with prompt-specific rubrics as reward signals. While recent methods adapt these rubrics to the evolving policy during training, the training prompts themselves remain static, drawn from fixed corpora. This static approach often results in a critical misalignment between prompt difficulty and policy capability, leaving the judge unable to recover a discriminative reward signal when prompts fail to elicit quality variance among rollouts. To address this misalignment, we introduce LLM-as-a-Tutor, a framework that extends the LLM's role from judge to tutor: a single model serves as an examiner that pairwise-compares policy rollouts to detect non-challenging prompts, and as a generator that appends atomic constraints to them. This append-only design monotonically raises difficulty in step with the policy's capability, producing a self-calibrating training signal without external difficulty schedules. On three complex instruction-following benchmarks, our method consistently outperforms both policy-unaware baselines and prior policy-adaptive methods that adapt rubrics or rewrite prompts, suggesting prompt adaptation as a missing axis of policy-awareness in non-verifiable RL.

Chinese Translation

不可验证的指令跟随的强化学习（RL）越来越依赖于具有特定提示评分标准的LLM评估者作为奖励信号。尽管最近的方法在训练过程中将这些评分标准适应于不断演变的策略，但训练提示本身仍然是静态的，来自固定的语料库。这种静态方法常常导致提示难度与策略能力之间的严重不匹配，使得评估者无法在提示未能引发回滚结果之间的质量差异时恢复出可区分的奖励信号。为了解决这一不匹配问题，我们提出了LLM作为导师的框架，该框架将LLM的角色从评估者扩展到导师：一个模型既作为考官进行策略回滚的成对比较，以检测不具挑战性的提示，又作为生成器为这些提示附加原子约束。这种仅附加的设计随着策略能力的提升单调增加难度，产生自我校准的训练信号，而无需外部的难度安排。在三个复杂的指令跟随基准测试中，我们的方法始终优于不考虑策略的基线和之前的策略适应方法，这些方法要么适应评分标准，要么重写提示，表明提示适应是不可验证的强化学习中策略意识的一个缺失维度。

View on arXiv Download PDF AI Translation

cs.AI / 57 / 2607.04419

Agent Step Value: State-Transition Measurement with State-Grounded LLM Evaluators

代理步骤值：基于状态的状态转移测量与状态基础的 LLM 评估器

Zhang, Andrew, Li, Chengzhan

Abstract

Most agent evaluations collapse a multi-step trace into a final answer, a success flag, or a trajectory-level score. These aggregates obscure the diagnostic question developers need most: which action changed the state in a useful direction? We introduce Agent Step Value (ASV), a state-transition measurement framework that scores each observed action by the change it induces in a state-grounded evaluator's distribution over fixed candidate outcomes. ASV renders redacted before/after state projections, uses a stateless LLM evaluator to assign candidate log scores, and reports both gold-free belief diagnostics and offline oracle validation metrics. A label-free rationale pass separates evaluator deliberation from one-token option scoring, preserving candidate likelihoods while exposing leakage and floor-score events. On 100 reviewed open-QA evidence-seeking tasks with live PubMed retrieval, a partially live DeepSeek actor, and DeepSeek log-probability scoring, ASV evaluates 1,100 steps and 2,200 states. Under the fixed-layout rationale-conditioned protocol, mean gold-margin gain is -2.335 (trajectory-bootstrap 95\% CI [-3.395, -1.272]), entropy movement is 0.000, and mean Bayesian surprise is 2.693. ASV therefore localizes constructive and destructive belief pivots that final-answer scores and entropy-only step metrics miss. We release the standalone ASV Eval toolkit.

Chinese Translation

大多数代理评估将多步骤轨迹简化为最终答案、成功标志或轨迹级评分。这些聚合结果掩盖了开发者最需要的诊断问题：哪个动作在有用的方向上改变了状态？我们引入了代理步骤值（Agent Step Value, ASV），这是一个状态转移测量框架，通过观察到的每个动作对状态基础评估器在固定候选结果上的分布所引起的变化来评分。ASV 在修改的前后状态投影中呈现，使用无状态的 LLM 评估器分配候选日志评分，并报告无金标准的信念诊断和离线神谕验证指标。无标签的推理过程将评估者的思考与单个选项评分分开，保留候选的可能性，同时暴露泄漏和底分事件。在 100 个经过审查的开放式问答证据搜索任务中，结合实时 PubMed 检索、部分实时的 DeepSeek 代理和 DeepSeek 日志概率评分，ASV 评估了 1,100 步和 2,200 状态。在固定布局的推理条件协议下，平均金标准边际增益为 -2.335（轨迹自助法 95% 置信区间 [-3.395, -1.272]），熵变化为 0.000，平均贝叶斯惊讶度为 2.693。因此，ASV 能够定位最终答案评分和仅基于熵的步骤指标所遗漏的建设性和破坏性信念转折点。我们发布了独立的 ASV 评估工具包。

View on arXiv Download PDF AI Translation

cs.AI / 58 / 2607.04439

ResearchStudio-Idea: An Evidence-Grounded Research-Ideation Skill Suite from ML Conference Outcomes

ResearchStudio-Idea：基于证据的研究创意技能套件来自机器学习会议成果

Zhao, Qihao, Huang, Yangyu, Dai, Yalun, Xiao, Lingao, Gao, Jianjun, Zhang, Xin, Wu, Wenshan, Li, Scarlett, He, Yang, Lu, Yan, Hui, Yap Kim

Abstract

Large language models have made research ideation increasingly accessible, yet effective idea development requires more than generating candidate directions. Researchers must ground a problem in current literature, identify meaningful bottlenecks, differentiate from existing solutions, and evaluate risks before committing to implementation. We present ResearchStudio-Idea as a reusable skill suite for this first mile of research ideation. The suite includes Paper-Search, a standalone multi-source literature search skill; Scoop-Check, a standalone prior-art collision checker for novelty claims; and IdeaSpark, the end-to-end skill that composes evidence grounding, pattern-guided generation, collision retrieval, audit, and idea-card rendering into one workflow. IdeaSpark is constructed from a corpus of 1,947 machine learning conference papers collected from ICLR, ICML, and NeurIPS between 2021 and 2025, including Oral papers, a separately tracked high-citation subset, and rejected submissions. Analysis of these outcomes reveals 31 recurring ideation sub-patterns, consolidated into 15 reusable ideation patterns. Each pattern is operationalized as a structured card containing research contexts, bottleneck types, differentiation strategies, supporting precedents, and common failure modes. Given a research problem and an evidence bundle, IdeaSpark evaluates evidence readiness, reconstructs the surrounding research context, identifies unresolved bottlenecks, selects relevant patterns, instantiates one candidate direction, retrieves potentially conflicting prior work, and performs outcome-informed auditing. This workflow transforms reusable ideation patterns into traceable research proposals. Blind automated-judge evaluations show that IdeaSpark consistently produces stronger research proposals than no-skill and generic-skill baselines while maintaining competitive novelty.

Chinese Translation

大型语言模型使得研究创意变得愈加可及，然而有效的创意发展不仅仅依赖于生成候选方向。研究人员必须将问题与当前文献相结合，识别有意义的瓶颈，区分现有解决方案，并在实施前评估风险。我们提出了ResearchStudio-Idea，作为研究创意的首个阶段的可重用技能套件。该套件包括Paper-Search，一个独立的多源文献搜索技能；Scoop-Check，一个独立的用于新颖性声明的前期艺术冲突检查器；以及IdeaSpark，一个将证据基础、模式引导生成、冲突检索、审计和创意卡呈现整合为一个工作流程的端到端技能。IdeaSpark是从2021年至2025年间收集的1,947篇机器学习会议论文构建而成，涵盖ICLR、ICML和NeurIPS，包括口头报告、一个单独跟踪的高引用子集以及被拒稿件。这些成果的分析揭示了31个重复出现的创意子模式，整合为15个可重用的创意模式。每个模式被操作化为一个结构化卡片，包含研究背景、瓶颈类型、差异化策略、支持前例和常见失败模式。给定一个研究问题和一组证据，IdeaSpark评估证据的准备情况，重建周围的研究背景，识别未解决的瓶颈，选择相关模式，实例化一个候选方向，检索潜在冲突的前期工作，并进行基于结果的审计。该工作流程将可重用的创意模式转化为可追溯的研究提案。盲评自动评审显示，IdeaSpark始终产生比无技能和通用技能基线更强的研究提案，同时保持竞争力的新颖性。

View on arXiv Download PDF AI Translation

cs.AI / 59 / 2607.04505

Why Pure Reasoning is Not Enough: Nature as the Source of Mathematical Innovation

为何纯粹推理不足：自然作为数学创新的源泉

Jutla, Charanjit S., Sharma, Vimal

Abstract

We advance the hypothesis that human mathematical reasoning, constrained by both the undecidability and the computational intractability of even modest logical fragments, relies fundamentally on pattern matching from domains external to pure deduction. The most prolific reservoir of such patterns is the natural world, whose physical laws and biological systems have undergone billions of years of ``pre-computation'' and already exhibit surprisingly innovative solutions. To ground this claim, we trace the history of the Fourier transform and relevant mathematics, from the vibrating string controversy to the hear equation and subsequent formalisms prevalent in mathematics. At each critical juncture, a physics problem forced the acceptance or creation of a mathematical tool that pure formal reasoning failed to anticipate or, worse, human reasoning had resisted. We further survey the landscape of logical complexity, from NP-hard propositional satisfiability to the non-elementary decision-procedures for monadic second-order theories, to demonstrate that even when a logic is decidable, the resources required for worst-case deduction are astronomically prohibitive. We argue that these barriers make physics-inspired pattern matching not just a historical accident but a cognitive necessity. Finally, we draw the consequence for artificial intelligence: if pure reasoning is constitutively insufficient, then any system aiming at human-level mathematical creativity must embed a vast store of cross-domain patterns rather than rely on deduction alone. This furnishes a principled justification for the enormous scale of contemporary large language models.

Chinese Translation

我们提出假设，人类的数学推理受到即使是适度逻辑片段的不可判定性和计算不可处理性的限制，根本上依赖于来自纯推理之外领域的模式匹配。这些模式的最丰富来源是自然界，其物理法则和生物系统经过数十亿年的“预计算”，已经展现出令人惊讶的创新解决方案。为了支撑这一论点，我们追溯了傅里叶变换及相关数学的历史，从振动弦的争论到听觉方程及其后在数学中普遍存在的形式化。在每一个关键时刻，物理问题迫使人们接受或创造出纯形式推理未能预见的数学工具，甚至更糟的是，人类推理曾经抵制的工具。我们进一步考察逻辑复杂性的全景，从 NP-hard 命题可满足性到单子二阶理论的非初等决策程序，以证明即使逻辑是可判定的，最坏情况下推理所需的资源也是天文数字般的不可承受。我们认为，这些障碍使得受物理启发的模式匹配不仅是历史偶然，更是认知上的必要。最后，我们得出对人工智能的启示：如果纯推理在构成上是不充分的，那么任何旨在达到人类水平数学创造力的系统必须嵌入大量跨领域的模式，而不仅仅依赖推理。这为当代大型语言模型的巨大规模提供了原则上的正当理由。

View on arXiv Download PDF AI Translation

cs.AI / 60 / 2607.04508

Compressing the Validation Bottleneck: An Agentic Self-Driving Lab for Scientific Discovery

压缩验证瓶颈：一个自主驱动的科学发现实验室

Hur, Kyunghoon, Lee, Chihun

Abstract

Agentic AI-for-Science can automate ideation, planning, and analysis, but final validation still depends on real experiments. A self-driving lab (SDL) can execute those experiments, yet the loop still has bottlenecks: the agent may spend too many rounds on low-value experiments, or each round may require a high-cost experiment. We target these two physical bottlenecks with one agent. First, a prior-aware agentic DOE loop uses domain knowledge and past results to propose feasible and informative next experiments, reducing trials-to-target. Second, a cost-aware surrogate agent predicts high-cost, high-resolution measurements from low-cost, low-resolution measurements. It chooses between a high- and a low-cost measurement based on the predicted uncertainty. We examine these directions in the biology and materials domains, respectively. Together, under a single agent, these components aim to accelerate the SDL loop by reducing both the number of loops and the cost per experiment.

Chinese Translation

自主人工智能科学（Agentic AI-for-Science）可以自动化创意、规划和分析，但最终的验证仍然依赖于真实实验。自主驱动实验室（Self-Driving Lab, SDL）可以执行这些实验，但该过程仍然存在瓶颈：代理可能在低价值实验上花费过多轮次，或者每轮可能需要高成本实验。我们针对这两个物理瓶颈设计了一个代理。首先，具有先验知识的自主设计实验（Design of Experiments, DOE）循环利用领域知识和过去的结果提出可行且信息丰富的下一个实验，从而减少达到目标所需的试验次数。其次，成本感知的代理预测从低成本、低分辨率测量中获得高成本、高分辨率测量。它根据预测的不确定性在高成本和低成本测量之间进行选择。我们分别在生物学和材料领域考察这些方向。总之，在单一代理的框架下，这些组件旨在通过减少循环次数和每次实验的成本来加速SDL循环。

View on arXiv Download PDF AI Translation

cs.AI / 61 / 2607.04517

VLA Grounder: Language-Conditioning Space Optimization for Black-Box VLA Models

VLA Grounder：针对黑箱VLA模型的语言条件空间优化

Shodiev, Damir, Staroverov, Aleksei, Kachaev, Nikita, Kovalev, Alexey K., Panov, Aleksandr I.

Abstract

Vision-Language-Action (VLA) models are commonly treated as end-to-end action policies conditioned on natural-language task descriptions. In practice, however, their behavior often depends sharply on how the instruction is phrased, suggesting that language is not merely a task label but an optimizable conditioning input. We study whether frozen VLA policies can be improved by optimizing language space rather than updating action weights. Our method introduces a language-conditioning space policy that translates a human instruction into a short VLA-grounded command using object appearance, spatial relations, and target-grounding cues. The language-conditioning space policy is initialized with a failure-derived command-space prior and optimized with reinforcement learning from sparse task-completion rewards, while the downstream VLA remains fully frozen. This yields language-conditioning space optimization: RL discovers which VLA-grounded commands best elicit successful behavior from the frozen action policy. Experiments on RL4VLA and VL-Think show that language-conditioning space optimization improves success on instruction-sensitive, symbolic, and multi-object manipulation tasks, demonstrating that language can serve as an optimizable variable for a robot foundation models. Website: https://tttonyalpha.github.io/vla_grounder

Chinese Translation

视觉-语言-行动（VLA）模型通常被视为基于自然语言任务描述的端到端行动策略。然而，在实际应用中，它们的行为往往强烈依赖于指令的措辞，这表明语言不仅仅是任务标签，而是一个可优化的条件输入。我们研究了是否可以通过优化语言空间来改善冻结的VLA策略，而不是更新行动权重。我们的方法引入了一种语言条件空间策略，该策略利用物体外观、空间关系和目标定位线索将人类指令转换为简短的VLA基础命令。语言条件空间策略以失败导出的命令空间先验初始化，并通过稀疏任务完成奖励的强化学习进行优化，而下游的VLA保持完全冻结。这实现了语言条件空间优化：强化学习发现哪些VLA基础命令能最好地引发冻结行动策略的成功行为。在RL4VLA和VL-Think上的实验表明，语言条件空间优化提高了对指令敏感的符号和多物体操作任务的成功率，证明了语言可以作为机器人基础模型的可优化变量。网站：https://tttonyalpha.github.io/vla_grounder

View on arXiv Download PDF AI Translation

cs.AI / 62 / 2607.04528

Measuring Harness-Induced Belief Divergence in Multi-Step LLM Agents

测量多步骤大型语言模型代理中的工具诱导信念差异

Yi, Haiwen, Song, Xinyuan

Abstract

Software-agent benchmarks usually report whether an agent solves a task, but the agent reaches that outcome through a harness that controls what it sees, which actions it can take, which failures are repaired, which states are verified, and which evidence is logged. We show that this harness can change the agent's multi-step beliefs even when the task, environment, and base LLM are fixed. We introduce a belief-rollout diagnostic that elicits structured K-step trajectories over progress, risk, recoverability, constraints, failure mode, uncertainty, future success, repair cost, and next action under alternative harnesses. We define a cross-harness belief divergence and decompose it into an arrival term for immediate interface shifts and a growth term for horizon-dependent belief changes. On controlled coding tasks and public-benchmark stress tests, blocked actions, compressed repairs, selective verification, and cost-aware evidence pruning often preserve terminal success while changing the beliefs that drive later decisions. We further introduce BIWM, a no-training protocol that canonicalizes observations, logs censored branches, expands repair traces, records verification masks, executes risky branches in shadow, and aligns belief trajectories across harness views. The results suggest that harness design is an experimental variable in agent evaluation, not an implementation detail. Our code is available at https://github.com/Hik289/Harness-induce-bias.git.

Chinese Translation

软件代理基准通常报告代理是否解决了某个任务，但代理通过一个控制其所见、可采取的行动、修复的失败、验证的状态和记录的证据的工具来达到该结果。我们展示了即使任务、环境和基础大型语言模型（LLM）固定，这个工具也可以改变代理的多步骤信念。我们引入了一种信念展开诊断，能够引导出关于进展、风险、可恢复性、约束、失败模式、不确定性、未来成功、修复成本和在不同工具下的下一步行动的结构化 K 步轨迹。我们定义了跨工具信念差异，并将其分解为用于即时界面变化的到达项和用于依赖于视野的信念变化的增长项。在受控编码任务和公共基准压力测试中，阻塞的行动、压缩的修复、选择性的验证和成本意识的证据修剪通常在改变驱动后续决策的信念的同时保持终端成功。我们进一步引入了 BIWM，一种无需训练的协议，标准化观察，记录被审查的分支，扩展修复痕迹，记录验证掩码，在影子中执行风险分支，并在工具视图之间对齐信念轨迹。结果表明，工具设计是代理评估中的一个实验变量，而不是实现细节。我们的代码可在 https://github.com/Hik289/Harness-induce-bias.git 获取。

View on arXiv Download PDF AI Translation

cs.AI / 63 / 2607.04562

Heaviside Continuity of Rolling Coefficients for Eliminating Epistemic Entropy in Large Language Models

消除大型语言模型中的认知熵的滚动系数的海维赛德连续性

Pitsane, MY, Mogale, Hope

Abstract

Large language models (LLMs) generate fluent outputs that can be wrong. Unlike humans, who often exhibit cues when providing false information, LLMs produce errors that are difficult to detect because autoregressive decoding provides no mechanism for verifying intermediate reasoning before state progression. We introduce Heaviside Continuity of Rolling Coefficients (HCRC), a verification-first execution framework that reformulates inference as predicate-gated state transitions governed by a Heaviside Gate. HCRC combines model confidence with independent verification signals from a parallel worker architecture, allowing execution to advance only when predefined correctness predicates are satisfied. This prevents invalid intermediate states from propagating, reducing epistemic entropy without modifying the underlying model. We evaluate HCRC on software-engineering and reasoning tasks across thirteen proposers from four providers. On capable proposers, the gate reduces the false-completion rate (FCR) from 4--7% to 0% while remaining latency-competitive and, in some settings, faster than the unwrapped model. On weaker proposers, it converts false completions into honest halts instead of corrupting downstream state. Beyond benchmarking, HCRC has operated for months as the production control plane of an agentic coding environment, authorizing file mutations, verification-driven progress reporting, and memory compaction. These results establish HCRC as a general framework for verification-driven LLM execution, showing that reliable reasoning can be achieved through principled execution control rather than model scale alone.

Chinese Translation

大型语言模型（LLMs）生成流畅的输出，但可能是错误的。与人类在提供错误信息时常常表现出线索不同，LLMs 产生的错误难以检测，因为自回归解码没有提供在状态推进之前验证中间推理的机制。我们提出了滚动系数的海维赛德连续性（HCRC），这是一种以验证为先的执行框架，将推理重新表述为由海维赛德门控制的谓词门控状态转移。HCRC 将模型置信度与来自并行工作者架构的独立验证信号相结合，仅在满足预定义的正确性谓词时允许执行推进。这防止了无效的中间状态传播，减少了认知熵，而无需修改基础模型。我们在来自四个提供者的十三个提议者上评估了 HCRC 在软件工程和推理任务中的表现。在能力较强的提议者上，该门将错误完成率（FCR）从 4-7% 降低到 0%，同时保持竞争性的延迟，并且在某些设置中比未包装模型更快。在较弱的提议者上，它将错误完成转换为诚实的停止，而不是破坏下游状态。除了基准测试外，HCRC 还作为一个代理编码环境的生产控制平面运行了数月，授权文件变更、基于验证的进度报告和内存压缩。这些结果确立了 HCRC 作为一种以验证驱动的 LLM 执行的通用框架，表明可靠的推理可以通过原则性的执行控制而非仅仅依赖模型规模来实现。

View on arXiv Download PDF AI Translation

cs.AI / 64 / 2607.04572

Detecting Answer-Driven Reasoning in LLM-Based Educational Tutors via Truncated Chain-of-Thought Auditing

通过截断的思维链审计检测基于大型语言模型的教育辅导中的答案驱动推理

Shen, Bonan, Shang, Dingyan, Wang, Youting, Ning, Tao

Abstract

Large language model (LLM) tutors often produce fluent step-by-step explanations, but a correct and pedagogically formatted response does not guarantee that the answer was derived from the student-facing problem. In realistic tutoring systems, the model may also have access to teacher notes, answer keys, rubrics, or retrieved solution artifacts. We study whether such private answer information can make tutor explanations answer-driven: the final answer is behaviorally available before the written explanation has justified it. Using Truncated Reasoning AUC Evaluation (TRACE), which probes how early a chain-of-thought prefix can pass a verifier, we evaluate 1000 GSM8K test problems under three paired tutoring contexts: question-only, correct answer-key, and wrong answer-key. At fixed fractions of each generated explanation, we force the model to answer immediately and verify the response against the gold numeric answer. With Qwen2.5-3B-Instruct, answer-key access raises median TRACE AUC from 0.375 to 0.900 and makes the gold answer available at the first 10% prefix in 997 of 1000 cases. The effect remains strong on the 746 examples where both question-only and answer-key explanations end with the correct answer. These results support truncated CoT auditing as a lightweight process-level diagnostic for answer-driven reasoning in math tutoring explanations.

Chinese Translation

大型语言模型（LLM）辅导员通常会生成流畅的逐步解释，但一个正确且符合教学格式的回答并不能保证该答案是从学生面临的问题中推导出来的。在现实的辅导系统中，模型可能还可以访问教师笔记、答案钥匙、评分标准或检索到的解决方案文档。我们研究了这种私人答案信息是否能够使辅导解释变得以答案为驱动：最终答案在书面解释证明之前在行为上是可用的。我们使用截断推理AUC评估（TRACE），探测思维链前缀能够多早通过验证器，我们在三种配对辅导上下文中评估了1000个GSM8K测试问题：仅问题、正确答案钥匙和错误答案钥匙。在每个生成的解释的固定比例下，我们强制模型立即回答，并将响应与黄金数值答案进行验证。使用Qwen2.5-3B-Instruct，答案钥匙的访问将中位数TRACE AUC从0.375提高到0.900，并使黄金答案在1000个案例中的997个中在前10%的前缀中可用。这个效果在746个示例中依然强劲，其中仅问题和答案钥匙的解释都以正确答案结束。这些结果支持截断的思维链审计作为一种轻量级的过程级诊断工具，用于数学辅导解释中的答案驱动推理。

View on arXiv Download PDF AI Translation

cs.AI / 65 / 2607.04590

Attention Limited Reward Learning

注意力有限的奖励学习

Xing, Wenqian

Abstract

Pairwise human comparisons are a primary interface through which modern AI systems learn human preferences. RLHF and related alignment pipelines typically model such comparisons with Bradley--Terry log-odds, where choice probabilities are governed by latent reward differences. This paper examines what this assumption misses through a reduced-form model motivated by rational inattention, in which each label is generated by a low-capacity evaluation channel. The model separates two forms of ambiguity that standard reward modeling tends to conflate: a comparison may be difficult because the two candidates are genuinely close in value, or because the relevant distinction is hard to detect under limited attention. We show that limited attention can fundamentally distort what pairwise comparisons reveal. In particular, passive comparison data cannot generally distinguish reward, attention, and default tendencies, and heterogeneous attention can make standard Bradley--Terry reward modeling recover misleading rankings. Our analysis shows that learning is governed not by the raw number of labels, but by the amount of attended information each label carries. A case study on human votes over language-model pairs from Chatbot Arena exhibits the predicted signature, a cyclic component of the comparison data that exceeds sampling noise and that no scalar reward can represent; a second case study on perceptual comparisons shows that response times and gaze carry gap information that the labels do not. This perspective suggests that human feedback should be treated not as direct revealed preference, but as an attention-limited measurement process: a weak preference signal may reflect hidden evaluation difficulty rather than genuine indifference.

Chinese Translation

成对的人类比较是现代人工智能系统学习人类偏好的主要接口。基于人类反馈的强化学习（RLHF）及相关的对齐流程通常使用Bradley--Terry对数几率模型来建模这些比较，其中选择概率由潜在的奖励差异决定。本文通过一个受到理性注意力启发的简化模型，考察了这一假设所遗漏的内容，其中每个标签是由一个低容量的评估通道生成的。该模型区分了标准奖励建模往往混淆的两种模糊性：比较可能困难是因为两个候选者在价值上确实接近，或者是因为在有限注意力下相关的区别难以察觉。我们表明，有限的注意力可以根本性地扭曲成对比较所揭示的内容。特别是，被动的比较数据通常无法区分奖励、注意力和默认倾向，而异质的注意力可能使标准的Bradley--Terry奖励建模恢复出误导性的排名。我们的分析表明，学习的驱动因素不是标签的原始数量，而是每个标签所携带的注意到的信息量。对Chatbot Arena中语言模型对的人工投票的案例研究展示了预测的特征，即比较数据中的一个循环成分超出了采样噪声，且没有任何标量奖励能够表示；第二个关于感知比较的案例研究显示，反应时间和注视携带了标签所没有的差距信息。这一视角表明，人类反馈应被视为一种有限注意力的测量过程，而非直接揭示的偏好：一个微弱的偏好信号可能反映了隐藏的评估困难，而非真正的无所谓。

View on arXiv Download PDF AI Translation

cs.AI / 66 / 2607.04613

Governed Individuation: Cryptographically Decoupling an Agent's Learning from Its Authority

受控个体化：从权威中加密解耦代理的学习

Qin, Xue, Luan, Simin, Yang, Cong, Li, Zhijun

Abstract

Autonomous agents are moving from sandboxed text generators to operators of code, data, and physical infrastructure, and they increasingly learn while deployed. This reopens a question that alignment techniques answer only probabilistically: after an agent has adapted in the field, is the running system still confined to what its operator authorised? Here we show that confinement can be guaranteed as an invariant of the agent's execution architecture rather than a probabilistic outcome of its training. Governed individuation binds an agent at boot to a cryptographically frozen identity digest, and routes every action through a gate defined over the semantic effect of the action rather than its name. We prove that no amount of learning, skill acquisition, or self-induced governance abstraction can widen the agent's permitted authority without an operator-signed change to its identity; the guarantee holds even when the agent induces its own safety principle and that principle is wrong. Empirically, in an open-ended tool-use benchmark where a large action space rules out name-based blocking, ungoverned software agents under reward pressure attempt to tamper with their own evaluation at a task-dependent rate that reaches every run on the hardest task, whereas the gate reduces executed forbidden effects to zero as a verified property of the construction while preserving task success. An adversarial evaluation of monitors of increasing semantic depth shows false-allows falling from 75% (name-based gating) to zero (dynamic effect tracing), and refusal history transfers compliance to held-out red-line families. Trust in a deployed learning agent shifts from a wager on its continued alignment to a check anyone can run at boot.

Chinese Translation

自主代理正从沙盒文本生成器转变为代码、数据和物理基础设施的操作员，并且它们在部署过程中越来越多地进行学习。这重新提出了一个问题，即对齐技术仅以概率方式回答：在代理在实际环境中适应后，运行系统是否仍然局限于其操作员授权的内容？在这里，我们展示了限制可以作为代理执行架构的一个不变性来保证，而不是其训练的概率结果。受控个体化在启动时将代理绑定到一个加密冻结的身份摘要，并通过一个基于行动语义效果而非名称定义的门路由每一个行动。我们证明，无论学习、技能获取或自我引导的治理抽象如何，都无法在没有操作员签名更改其身份的情况下扩大代理的授权范围；这一保证即使在代理引导自己的安全原则且该原则是错误的情况下也依然成立。从经验上看，在一个开放式工具使用基准中，广泛的行动空间排除了基于名称的阻止，未受控的软件代理在奖励压力下试图以任务相关的速率篡改自己的评估，这种速率在最困难的任务中达到每次运行，而门的存在将执行的禁止效果减少到零，作为构造的一个经过验证的属性，同时保持任务成功。对监控器进行的对抗性评估显示，随着语义深度的增加，错误允许率从75%（基于名称的门控）降至零（动态效果追踪），而拒绝历史将合规性转移到保留的红线家族。对部署学习代理的信任从对其持续对齐的赌注转变为任何人都可以在启动时进行的检查。

View on arXiv Download PDF AI Translation

cs.AI / 67 / 2607.04617

MRMS: A Multi-Resolution Memory Substrate for Long-Lived AI Agents

MRMS：一种用于长寿命人工智能代理的多分辨率记忆基底

Li, Jizhizi, Shi-Nash, Amy

Abstract

Long-lived AI agents require continuity across interactions, but continuity cannot be obtained by simply extending the prompt window. An agent must preserve useful prior experience, retrieve it selectively, distinguish personal context from external evidence, and revise memory when the underlying situation changes. We propose an architectural memory substrate organized along two orthogonal axes: a representational axis spanning structured records, vector representations, and graph relations; and a temporal axis spanning short-term traces, medium-term abstractions, and long-term semantic commitments. Its key design constraint is synchronized structured-vector-graph memory: structured records govern eligibility, vector representations support recall, and graph relations adjudicate support, contradiction, and supersession before gated context projection. Its central claim is that reliable personalization is a memory design problem: useful memory is structured, selectively exposed, continuously consolidated, and epistemically labeled rather than stored as undifferentiated conversation history. Beyond the framework, we instantiate MRMS as a lightweight prototype implementing structured records, vector retrieval, temporal policies, and graph-based revision. The prototype exercises the core substrate mechanisms through pre-generation memory selection, revision, boundary enforcement, and evidence attribution under controlled long-lived interaction scenarios with explicit evidence requirements.

Chinese Translation

长寿命人工智能代理需要在交互中保持连续性，但仅仅延长提示窗口无法实现这种连续性。代理必须保留有用的先前经验，选择性地检索这些经验，区分个人上下文与外部证据，并在基础情况发生变化时修订记忆。我们提出了一种沿两个正交轴组织的架构记忆基底：一个表示轴，涵盖结构化记录、向量表示和图关系；另一个时间轴，涵盖短期痕迹、中期抽象和长期语义承诺。其关键设计约束是同步的结构化-向量-图记忆：结构化记录决定资格，向量表示支持回忆，而图关系在门控上下文投影之前裁定支持、矛盾和取代。我们的核心主张是，可靠的个性化是一个记忆设计问题：有用的记忆是结构化的、选择性暴露的、持续巩固的，并且是以认知标签标记的，而不是作为未区分的对话历史存储。超越这一框架，我们将MRMS实例化为一个轻量级原型，实施结构化记录、向量检索、时间策略和基于图的修订。该原型通过预生成记忆选择、修订、边界执行和证据归属，检验核心基底机制，适用于具有明确证据要求的受控长寿命交互场景。

View on arXiv Download PDF AI Translation

cs.AI / 68 / 2607.04631

Formal Disco: Scalable Open-Ended Generation of Formally Verified Programs

正式迪斯科：可扩展的开放式生成形式验证程序

Poesia, Gabriel, Henniger, Simon, Hsu, Tzu-Han, Du, Yilun, Amin, Nada

Abstract

The cost of producing code is rapidly diminishing with increasingly capable AI agents, while quality assurance of generated programs has not kept pace. Formal verification provides the strongest possible guarantees, but the ability of AI models to work with verification-aware languages is hindered by the scarcity of human-written examples of programs in those languages. To tackle this prevalent data scarcity issue, we propose Formal Disco: a distributed system for coordination of LLM-based workers that can be easily applied to open-ended synthetic data generation at scale. We use Formal Disco to share tasks and programs between three classes of workers: "initiators", which read random READMEs from open-source repositories and documentation snippets to sketch a related verified program, "fixers" which take compiler and verifier feedback and attempt to resolve issues, and "extenders" that take working programs and propose patches to expand them. Formal Disco records all agent-generated traces and uses them both for initial distillation from a stronger model as well as self-improvement. We also propose a principle of maximum entropy for synthetic program generation, and use entropy maximization via iterative supervised fine-tuning to learn to generate increasingly diverse programs over time. We release large datasets of synthetic verified programs in three languages - Dafny, Verus, and Frama-C -, and fine-tune open models for verification-relevant tasks, often matching or exceeding the performance of Claude Opus 4.5. Overall, our work offers a path to create synthetic data at scale for formal reasoning domains and overcome the long-standing data barrier.

Chinese Translation

随着越来越强大的人工智能代理，代码生成的成本迅速降低，而生成程序的质量保证却未能同步提升。形式验证提供了最强的保证，但人工智能模型在处理具有验证意识的语言时受到人类编写的示例稀缺的限制。为了解决这一普遍的数据稀缺问题，我们提出了正式迪斯科（Formal Disco）：一个用于协调基于大语言模型（LLM）工作者的分布式系统，能够轻松应用于大规模开放式合成数据生成。我们使用正式迪斯科在三类工作者之间共享任务和程序：“发起者”（initiators），他们从开源代码库和文档片段中随机读取README，以草拟相关的验证程序；“修复者”（fixers），他们接受编译器和验证器的反馈并尝试解决问题；以及“扩展者”（extenders），他们从可工作的程序中提出补丁以扩展这些程序。正式迪斯科记录所有代理生成的痕迹，并将其用于从更强模型的初始蒸馏以及自我改进。我们还提出了合成程序生成的最大熵原则，并通过迭代监督微调实现熵最大化，以学习随着时间推移生成越来越多样化的程序。我们发布了三种语言（Dafny、Verus 和 Frama-C）的合成验证程序的大型数据集，并对开放模型进行了微调，以适应与验证相关的任务，通常与 Claude Opus 4.5 的性能相匹配或超越。总体而言，我们的工作为在形式推理领域大规模创建合成数据提供了一条路径，克服了长期存在的数据壁垒。

View on arXiv Download PDF AI Translation

cs.AI / 69 / 2607.04710

Integrated Altruistic and Fairness Preference Induces Advanced Mutual Cooperation in Sequential Social Dilemmas

综合利他主义与公平偏好促进序列社会困境中的高级互助合作

Wei, Yu, Ogura, Yukiko, Ohmura, Yoshiyuki, de Abril, Ildefons Magrans, Kanazawa, Hoshinori, Kuniyoshi, Yasuo

Abstract

Inducing cooperation among distributed agents is still a difficult problem in the field of multi-agent reinforcement learning (MARL), particularly in social dilemma situations. There, individual interests are misaligned with the common good and individual rationality leads to suboptimal group outcomes. In contrast, humans are able to achieve cooperation with one another in such situations. A common explanation for such cooperative behavior is that individuals have social preferences. In order to achieve cooperation in MARL, we design a new utility function integrating altruistic preferences (incentive for other's reward) and fairness preferences (incentive for equality) from social psychology and behavioral economics, namely, Altruistic and Fairness Preference (AFP), a reward-sharing mechanism which converts one's own and other's rewards to incentives for cooperative behavior. We performed comparative experiments with standard RL and inequity aversion agents in two challenging sequential social dilemma games, and showed that AFP agents successfully achieved mutual cooperation with more collective rewards and higher equity than the baselines. To further understand the progression of AFP during training, we subsequently explore the effects of altruistic preferences and fairness preferences on agents' behavior. The results suggest that altruistic preferences encourage agents to contribute to the public goods, and fairness preferences induce mutual behavior between agents.

Chinese Translation

在多智能体强化学习（MARL）领域，促使分布式智能体之间的合作仍然是一个难题，尤其是在社会困境的情境中。在这些情境中，个人利益与公共利益不一致，个人理性导致次优的群体结果。相比之下，人类能够在此类情境中实现相互合作。对这种合作行为的一个普遍解释是，个体具有社会偏好。为了在MARL中实现合作，我们设计了一种新的效用函数，整合了来自社会心理学和行为经济学的利他偏好（对他人奖励的激励）和公平偏好（对平等的激励），即利他与公平偏好（Altruistic and Fairness Preference, AFP），这是一种奖励共享机制，将个人及他人的奖励转化为促进合作行为的激励。我们在两个具有挑战性的序列社会困境游戏中，使用标准强化学习（RL）和不平等厌恶智能体进行了比较实验，结果表明，AFP智能体成功实现了互助合作，获得了比基线更高的集体奖励和更高的公平性。为了进一步理解AFP在训练过程中的进展，我们随后探讨了利他偏好和公平偏好对智能体行为的影响。结果表明，利他偏好鼓励智能体为公共物品做出贡献，而公平偏好则促进了智能体之间的互助行为。

View on arXiv Download PDF AI Translation

cs.AI / 70 / 2607.04718

FORGE: Research-Trajectory Hijacking Attacks on Deep Research Agents

FORGE：针对深度研究代理的研究轨迹劫持攻击

Pan, Yue, Zhang, Ziheng, Lei, Junxiang, Jia, Changhao, Si, Qingyi, Guo, Hongcheng

Abstract

Deep research agents decompose open-ended queries into subtasks, retrieve web evidence over multiple rounds, and synthesize long-form reports. This workflow creates a planning-layer poisoning surface: adversarial documents that enter the retrieval pool can steer follow-up questions and turn a local injection into report-level contamination. We present FORGE (Fabricated Orchestrated Reasoning chain for aGent Exploitation), a two-level attack that combines intra-document reasoning fabrication with inter-document chain coordination to hijack subtask planning. We further introduce the PRISM metric, which weights infected report claims by cognitive type, and Root Query Anchoring, a lightweight defense that ties recursive follow-up generation to the root query. Across 25 queries, Network FORGE reaches 26.4% PRISM with five injected documents and exhibits depth migration, in which recursive synthesis shifts poisoned content from overt framing into factual premises. On the 10-query defense subset, RQA (Root Query Anchoring) reduces PRISM from 38.5% to 18.3%.

Chinese Translation

深度研究代理将开放式查询分解为子任务，通过多轮检索网络证据，并综合形成长篇报告。这一工作流程创建了一个规划层的毒化表面：进入检索池的对抗性文档可以引导后续问题，并将局部注入转变为报告级别的污染。我们提出了FORGE（Fabricated Orchestrated Reasoning chain for aGent Exploitation），这是一种两级攻击，结合了文档内部推理伪造与文档间链协调，以劫持子任务规划。我们进一步引入了PRISM指标，该指标根据认知类型对感染报告的主张进行加权，以及根查询锚定（Root Query Anchoring），这是一种轻量级防御措施，将递归后续生成与根查询绑定。在25个查询中，网络FORGE在注入五个文档的情况下达到了26.4%的PRISM，并展现了深度迁移，其中递归综合将被污染的内容从明显框架转移到事实前提。在10个查询的防御子集中，根查询锚定（RQA）将PRISM从38.5%降低至18.3%。

View on arXiv Download PDF AI Translation

cs.AI / 71 / 2607.04750

FM-ChangeNet: Learning Change through Pathwise Feature Transport

FM-ChangeNet：通过路径特征传输学习变化

Kazoom, Roie, Leifman, George, Beryozkin, Genady

Abstract

We present FM-ChangeNet, a pathwise-supervised framework for change detection that reformulates bi-temporal reasoning as continuous transport in feature space rather than static endpoint comparison. Given encoded pre and post-temporal representations, we construct intermediate latent states and learn a time-conditioned velocity field $\hat{v}_\theta(z_t,t)$ along the transformation trajectory. This pathwise formulation constrains the predictor over a continuum of intermediate states, providing a denser and less ambiguous supervision signal than conventional endpoint-only segmentation and enabling the model to capture temporal evolution explicitly. The learned velocity field is not only a transport mechanism but also an interpretable representation of change: its magnitude serves as a spatially localized change cue that helps distinguish true structural variation from nuisance effects such as illumination shifts and spatial misalignment. We develop a hierarchical multi-scale architecture with cross-temporal alignment, time-conditioned coarse-to-fine flow decoding, and a unified objective that couples flow supervision, trajectory consistency, spatial regularization, and segmentation loss. Experiments on remote sensing benchmarks show that the proposed framework produces more structured and robust change representations while achieving state-of-the-art performance.

Chinese Translation

我们提出了FM-ChangeNet，这是一个路径监督框架，用于变化检测，它将双时序推理重新表述为特征空间中的连续传输，而不是静态的端点比较。给定编码的前后时序表示，我们构建中间潜在状态，并学习沿变换轨迹的时间条件速度场 $ ext{hat{v}}_ heta(z_t,t)$。这种路径形式约束了预测器在一系列中间状态上的行为，提供了比传统仅依赖端点的分割更密集且不模糊的监督信号，使模型能够明确捕捉时间演变。所学习的速度场不仅是一个传输机制，也是变化的可解释表示：其幅度作为空间局部变化线索，有助于区分真实的结构变化与诸如光照变化和空间错位等干扰效应。我们开发了一个具有跨时序对齐、时间条件粗到细流解码和统一目标的分层多尺度架构，该目标结合了流监督、轨迹一致性、空间正则化和分割损失。在遥感基准上的实验表明，所提出的框架产生了更结构化和稳健的变化表示，同时实现了最先进的性能。

View on arXiv Download PDF AI Translation

cs.AI / 72 / 2607.04758

AgenticPD: A Stage-Aware Agentic Framework for Physical Design QoR Optimization

AgenticPD：一种阶段感知的代理框架用于物理设计质量优化

Ren, Shuo, Cheng, Zijin, Han, Yaohui, Shen, Libo, Jin, Leilei, Tian, Wanting, Fu, Rongliang, Wang, Chao, Yu, Bei, Ho, Tsung-Yi

Abstract

Physical design quality-of-results~(QoR) optimization is hard and expensive. Choices made at one stage can help or hurt later stages. Each evaluation requires a costly EDA run through the full flow. While existing methods still treat optimization as flat parameter tuning or a LLM-based script generation task, we present AgenticPD, a stage-aware agentic framework for physical design QoR optimization. Instead of re-running the full flow after every trial, AgenticPD is organized around the stage boundaries of the physical design flow, where a Judge Agent navigates the search and stage-specialized agents make local decisions within their own stage using stage-local tools. Additionally, the agent harness in AgenticPD provides structured observations, execution history, and agent context management. As a result, the system can branch from prior intermediate states and reuse checkpoints to continue the optimization procedure, and every candidate is evaluated at the post-route signoff. Across these baselines, AgenticPD achieves strong post-route timing while remaining competitive in power and area.

Chinese Translation

物理设计结果质量（QoR）优化既困难又昂贵。在一个阶段做出的选择可能会对后续阶段产生积极或消极的影响。每次评估都需要通过完整流程进行昂贵的电子设计自动化（EDA）运行。尽管现有方法仍将优化视为平面参数调优或基于大型语言模型（LLM）的脚本生成任务，但我们提出了AgenticPD，一种用于物理设计QoR优化的阶段感知代理框架。AgenticPD不是在每次试验后重新运行完整流程，而是围绕物理设计流程的阶段边界进行组织，其中一个评估代理（Judge Agent）导航搜索，而阶段专用代理在各自阶段内使用阶段本地工具做出局部决策。此外，AgenticPD中的代理架构提供了结构化观察、执行历史和代理上下文管理。因此，该系统可以从先前的中间状态分支，并重用检查点以继续优化过程，并且每个候选方案都在后路签署时进行评估。在这些基准测试中，AgenticPD实现了强大的后路时序，同时在功耗和面积方面保持竞争力。

View on arXiv Download PDF AI Translation

cs.AI / 73 / 2607.04854

CARL: Constraint-Aware Reinforcement Learning for Planning with LLMs

CARL：面向约束的强化学习用于大型语言模型的规划

Qi, Qiuyi, Zhang, Jinjian, Bao, Mutian, Liang, Tian, Li, Guocong, Liu, Dongnan, Zhou, Wei, Liu, Jie, Kong, Ming, Mo, Linjian, Zhang, Feng, Zhu, Qiang

Abstract

Despite their strong reasoning capabilities and extensive world knowledge, Large Language Models (LLMs) frequently generate plans that violate task constraints, undermining their reliability in real-world applications. This deficiency arises from a lack of systematic mechanisms to incorporate constraint information during the generation process. While existing approaches attempt to mitigate this by relying on external tools or task decomposition, they fail to enhance the model's intrinsic constraint awareness. To address this, we propose Constraint-Aware Reinforcement Learning (CARL), a novel RL framework designed to strengthen LLMs' intrinsic focus on constraints. CARL introduces a constraint-aware reward by comparing the model's output distributions under constrained and unconstrained inputs, encouraging constraint focus and penalizing neglect. Compatible with various RL frameworks and requiring no external solvers or top models, CARL enables scalable, end-to-end constraint-aware planning. Extensive experiments on BlocksWorld, TravelPlanner, and T-Eval demonstrate that CARL significantly outperforms standard Reinforcement Fine-Tuning (RFT) baselines and state-of-the-art reasoning models, exhibiting a markedly increased focus on constraints.

Chinese Translation

尽管大型语言模型（LLMs）具有强大的推理能力和广泛的世界知识，但它们在生成计划时常常违反任务约束，从而削弱了其在现实应用中的可靠性。这一缺陷源于缺乏系统性机制在生成过程中纳入约束信息。虽然现有方法试图通过依赖外部工具或任务分解来缓解这一问题，但未能增强模型内在的约束意识。为此，我们提出了面向约束的强化学习（CARL），这是一种新颖的强化学习框架，旨在增强LLMs对约束的内在关注。CARL通过比较模型在约束和非约束输入下的输出分布，引入了约束意识奖励，鼓励关注约束并惩罚忽视。CARL与各种强化学习框架兼容，且不需要外部求解器或顶级模型，从而实现可扩展的端到端约束意识规划。在BlocksWorld、TravelPlanner和T-Eval上的大量实验表明，CARL显著优于标准强化微调（RFT）基线和最先进的推理模型，表现出明显增强的约束关注。

View on arXiv Download PDF AI Translation

cs.AI / 74 / 2607.04907

Medi-Gemma: A Hybrid Clinical Decision Support System Integrating Deterministic EMR Analytics and Retrieval-Augmented Generation

Medi-Gemma：一种集成确定性电子病历分析和检索增强生成的混合临床决策支持系统

Quadri, Mohammed Saim Ahmed, Xue, Yunzhe, Ady, Justin W., Roshan, Usman

Abstract

Deploying Large Language Models (LLMs) in high-stakes clinical settings remains limited by structural hallucinations, weak deterministic reasoning over tabular patient data, and omissions in vector retrieval. This paper presents the architecture and validation of Medi-Gemma, a Clinical Decision Support System (CDSS) for wound pathology triage and workflow automation. The platform introduces a decoupled framework that separates clinical perception from data orchestration while preserving traceable reasoning. Medi-Gemma uses a multi-stage pipeline coordinated by a centralized ClinicalOrchestrator. Data requests are handled without generative inference by a DataManager that cleans unstructured Electronic Medical Record (EMR) files through type coercion. Natural language queries are processed by a hierarchical IntentRouter, which routes requests to deterministic analytics paths executed by a PandasQueryEngine or to patient-specific reasoning managed by a ClinicalRAGEngine using a CPU-optimized vector store. A key contribution is the Ground Truth Injection Module, which intercepts patient-specific queries, extracts numeric identification tokens, queries the structured dataframe via Pandas, retrieves the latest validated clinical state, and embeds this snapshot as an overriding context block in the LLM prompt before generation. Safety compliance is enforced by a deterministic ProtocolManager that maps clinical terminology to fixed evidence-based risk pathways, while a SafetyVerifier phrase filter prevents output rule violations. Validation shows that this architecture eliminates semantic context drift, prevents database compilation crashes, and improves factual adherence to backend clinical repositories. These results support Medi-Gemma as a safer pattern for LLM-based clinical decision support where structured data fidelity, retrieval grounding, and deterministic safeguards are essential.

Chinese Translation

在高风险临床环境中部署大型语言模型（LLMs）仍受到结构性幻觉、对表格患者数据的弱确定性推理以及向量检索中的遗漏的限制。本文介绍了Medi-Gemma的架构和验证，这是一种用于伤口病理分诊和工作流程自动化的临床决策支持系统（CDSS）。该平台引入了一个解耦框架，将临床感知与数据编排分离，同时保留可追溯的推理。Medi-Gemma使用一个由中央ClinicalOrchestrator协调的多阶段管道。数据请求由DataManager处理，无需生成推理，通过类型强制清理非结构化电子病历（EMR）文件。自然语言查询由分层的IntentRouter处理，该路由器将请求路由到由PandasQueryEngine执行的确定性分析路径，或路由到由ClinicalRAGEngine管理的患者特定推理，后者使用CPU优化的向量存储。一个关键贡献是Ground Truth Injection Module，它拦截患者特定查询，提取数字识别令牌，通过Pandas查询结构化数据框，检索最新的验证临床状态，并将该快照嵌入为LLM提示中的覆盖上下文块。安全合规性由确定性的ProtocolManager强制执行，该管理器将临床术语映射到固定的基于证据的风险路径，而SafetyVerifier短语过滤器则防止输出规则违反。验证结果表明，该架构消除了语义上下文漂移，防止了数据库编译崩溃，并改善了对后端临床库的事实遵循。这些结果支持Medi-Gemma作为一种更安全的基于LLM的临床决策支持模式，其中结构化数据的保真性、检索基础和确定性保障至关重要。

View on arXiv Download PDF AI Translation

cs.AI / 75 / 2607.04963

STAPO: Selective Trajectory-Aware Policy Optimization for LLM Agent Training

STAPO：用于大型语言模型代理训练的选择性轨迹感知策略优化

Qi, Qiuyi, Liang, Tian, Bao, Mutian, Zhang, Jinjian, Liu, Dongnan, Zhou, Wei, Mo, Linjian, Kong, Ming, Liu, Jie, Zhang, Feng, Zhu, Qiang

Abstract

Reinforcement Learning (RL) is the dominant paradigm for training Large Language Model (LLM) agents on long-horizon tasks. However, sparse and delayed rewards often lead to trajectory neglect, in which agents lose focus on the task goal and interaction history at intermediate steps. Prior work has explored step-level supervision using Shannon-entropy-based uncertainty signals, which conflate inherent state complexity with agent confidence and therefore provide unreliable estimates of decision reliability. To address this issue, we propose normalized entropy, which measures confidence deviations relative to an agent's average behavior under a given state, thereby strengthening the association between low-quality actions and trajectory neglect. Building on this insight, we introduce Selective Trajectory-Aware Policy Optimization (STAPO), a hierarchical group-based RL framework. STAPO leverages normalized entropy to locate outlier steps associated with trajectory neglect and optimizes them via a joint mechanism of trajectory-aware reward and trajectory-independent penalty, enhancing trajectory awareness while preserving training stability. Extensive experiments on ALFWorld, WebShop, and Search-Augmented QA demonstrate that STAPO achieves state-of-the-art performance while substantially alleviating trajectory neglect, validating its effectiveness and robustness for agentic tasks.

Chinese Translation

强化学习（RL）是训练大型语言模型（LLM）代理在长时间任务中表现的主要范式。然而，稀疏和延迟的奖励往往导致轨迹忽视，使得代理在中间步骤中失去对任务目标和交互历史的关注。先前的研究探讨了使用基于香农熵的不确定性信号进行逐步监督，但这种方法将固有状态复杂性与代理信心混为一谈，因此提供了不可靠的决策可靠性估计。为了解决这个问题，我们提出了归一化熵，它测量相对于给定状态下代理平均行为的信心偏差，从而加强低质量动作与轨迹忽视之间的关联。在此基础上，我们引入了选择性轨迹感知策略优化（STAPO），这是一种基于层次分组的强化学习框架。STAPO利用归一化熵来定位与轨迹忽视相关的异常步骤，并通过轨迹感知奖励与轨迹无关惩罚的联合机制对其进行优化，增强轨迹意识的同时保持训练稳定性。在ALFWorld、WebShop和搜索增强问答等多个实验中，STAPO实现了最先进的性能，同时显著缓解了轨迹忽视，验证了其在代理任务中的有效性和鲁棒性。

View on arXiv Download PDF AI Translation

cs.AI / 76 / 2607.05007

Quantum-Inspired Harmonic Decision Models: A Computational Framework for Music Generation

量子启发的和声决策模型：音乐生成的计算框架

Pavlíček, Josef, Pavlíčková, Petra, Molhanec, Martin

Abstract

This paper introduces a quantum-inspired computational framework for harmonic decision-making in music. The proposed approach formulates harmonization as an optimization problem within a structured combinatorial space, where multiple candidate chord sequences are evaluated under interacting musical constraints. The model combines an interference-based harmonization stage with a classical optimization procedure grounded in tonal harmony. The quantum-inspired component enables the parallel consideration of multiple harmonic alternatives, while the classical stage refines the resulting sequences to ensure structural coherence and stylistic plausibility. The framework is evaluated on selected musical examples, including Autumn Leaves and It's a Long Way to Tipperary. Quantitative analysis shows that the optimization stage significantly reduces chord density, increases harmonic stability, and improves functional organization. At the same time, expert evaluation highlights the importance of stylistic context, demonstrating that increased harmonic complexity is not always perceived as more natural. The results suggest that harmonic generation can be interpreted as a structured decision-making process in a constrained search space. The proposed approach provides a computational model that integrates domain-specific knowledge with an interference-based search mechanism. Although preliminary, this work indicates that quantum-inspired methods may offer a useful framework for modeling complex decision processes in creative domains such as music. The proposed framework contributes to ongoing research on quantum-inspired models of cognition and decision-making in complex biological and creative systems.

Chinese Translation

本文介绍了一种量子启发的计算框架，用于音乐中的和声决策。所提出的方法将和声化表述为一个结构化组合空间中的优化问题，在该空间中，多个候选和弦序列在相互作用的音乐约束下进行评估。该模型结合了基于干扰的和声化阶段与基于调性和声的经典优化程序。量子启发的部分使得可以并行考虑多个和声替代方案，而经典阶段则对生成的序列进行精炼，以确保结构的一致性和风格的合理性。该框架在选定的音乐示例上进行了评估，包括《秋叶》（Autumn Leaves）和《去提珀瑞的漫长旅程》（It's a Long Way to Tipperary）。定量分析表明，优化阶段显著降低了和弦密度，提高了和声稳定性，并改善了功能组织。同时，专家评估强调了风格背景的重要性，表明增加的和声复杂性并不总是被视为更自然。结果表明，和声生成可以被解释为一个在受限搜索空间中的结构化决策过程。所提出的方法提供了一个计算模型，将领域特定知识与基于干扰的搜索机制相结合。尽管仍处于初步阶段，但这项工作表明量子启发的方法可能为建模音乐等创造性领域中的复杂决策过程提供有用的框架。所提出的框架为关于量子启发的认知和复杂生物及创造性系统中的决策模型的持续研究做出了贡献。

View on arXiv Download PDF AI Translation

cs.AI / 77 / 2607.05055

Toward Trustworthy Large Language Model Agents in Healthcare

迈向可信赖的大型语言模型代理在医疗保健中的应用

Hasan, Hadi, Salman, Safaa, Dargham, Adam Tai Abou, Mohanna, Ammar, Chehab, Ali

Abstract

Healthcare appointment scheduling remains a persistent operational bottleneck, driven by manual coordination, fragmented legacy systems, and high administrative overhead. These inefficiencies constrain provider availability and degrade patient access to care. This paper presents CareConnect, a safety-first conversational agent for healthcare logistics automation that leverages large language model (LLM) function calling, retrieval-augmented generation (RAG), and layered deterministic safety guardrails. The system orchestrates eight domain-specific tools to support appointment booking, modification, cancellation, and facility information retrieval, while enforcing strict scope constraints that prohibit medical advice or diagnosis. Safety-critical situations are handled through deterministic short-circuit mechanisms for emergency detection and medical intent refusal. We evaluate CareConnect on a comprehensive benchmark of 680 task-oriented scenarios spanning end-to-end workflows, multi-turn interactions, and edge cases. Experimental results demonstrate a 91.8% task completion rate with a median per-request latency of 2.2 seconds, 96.0% safety compliance on the dedicated safety-critical evaluation subset, and an average operational cost of $0.0324 per appointment, yielding a significant cost reduction compared to manual human scheduling. These findings show that carefully scoped and rigorously safeguarded LLM-based agents can reliably automate complex healthcare operational workflows while maintaining safety guarantees and achieving substantial cost efficiency. The source code and system implementation are publicly available at https://github.com/Hadi-Hsn/CareConnect.

Chinese Translation

医疗预约调度仍然是一个持续的运营瓶颈，主要由于手动协调、分散的遗留系统和高昂的行政开销。这些低效性限制了提供者的可用性，并降低了患者获取护理的机会。本文提出了CareConnect，一个以安全为首的医疗物流自动化对话代理，利用大型语言模型（LLM）功能调用、检索增强生成（RAG）和分层确定性安全防护措施。该系统协调八个特定领域的工具，以支持预约的预订、修改、取消和设施信息检索，同时严格执行禁止医疗建议或诊断的范围限制。安全关键情况通过确定性短路机制处理，以实现紧急检测和拒绝医疗意图。我们在680个任务导向场景的综合基准上评估了CareConnect，这些场景涵盖了端到端工作流程、多轮交互和边缘案例。实验结果表明，任务完成率为91.8%，每请求的中位延迟为2.2秒，在专门的安全关键评估子集上安全合规率为96.0%，每次预约的平均运营成本为0.0324美元，相较于手动调度显著降低了成本。这些发现表明，经过精心界定和严格保护的基于LLM的代理能够可靠地自动化复杂的医疗运营工作流程，同时保持安全保障并实现显著的成本效率。源代码和系统实现已公开，地址为https://github.com/Hadi-Hsn/CareConnect。

View on arXiv Download PDF AI Translation

cs.AI / 78 / 2607.05064

Diffusion-Guided Uncertainty-Aware Delayed Policy Optimization

扩散引导的不确定性感知延迟策略优化

Tu, Junqi, Liu, Zejiao, Li, Fangfei, Tang, Yang

Abstract

Reinforcement learning in real world environments often suffers from severe performance degradation due to delayed feedback. Existing approaches typically mitigate performance degradation caused by observation delays by constructing augmented states or predicting the true states. However, these methods often overlook the inherent discrepancy between delayed state and true states induced by stochastic MDP. We theoretically prove the existence of such a discrepancy and show that it leads to the degradation of the optimal policy. To address this challenge, we propose Diffusion Guided Uncertainty Aware Delayed Policy Optimization (DUPO). Our method explicitly models the relationship between delayed state message and the current state using a diffusion model, and leverages the resulting discrepancy estimates to weight delayed policies. Extensive experiments on continuous robotic control tasks with multiple stochastic delays demonstrate that DUPO consistently outperforms existing methods and remains effective even under long and random delay scenarios.

Chinese Translation

在现实环境中，强化学习常常因反馈延迟而遭受严重的性能下降。现有方法通常通过构建增强状态或预测真实状态来缓解因观察延迟引起的性能下降。然而，这些方法往往忽视了由随机马尔可夫决策过程（MDP）引起的延迟状态与真实状态之间的内在差异。我们理论上证明了这种差异的存在，并展示了它导致最优策略的降级。为了解决这一挑战，我们提出了扩散引导的不确定性感知延迟策略优化（DUPO）。我们的方法明确建模了延迟状态信息与当前状态之间的关系，使用扩散模型，并利用由此产生的差异估计来加权延迟策略。在多个具有随机延迟的连续机器人控制任务上的广泛实验表明，DUPO始终优于现有方法，并且在长时间和随机延迟场景下仍然有效。

View on arXiv Download PDF AI Translation

cs.AI / 79 / 2607.05123

ASSEMCAD: Production-Ready CAD Assembly Generation from Natural Language

ASSEMCAD：基于自然语言的生产就绪CAD装配生成

Dong, Yurui, Zou, Shu, Li, Siqi, Deng, Nianchen, Zhou, Hongbin, Yang, Xuemeng, Cai, Pinlong, Wen, Licheng, Cai, Xinyu, Shi, Botian

Abstract

Recent advances in large language models and programmatic CAD have significantly improved Text-to-CAD generation for individual parts. However, production-ready mechanical assembly generation remains largely unsolved. Unlike single-part modeling, assemblies require coordinated reasoning over multiple components, functional interfaces, assembly relations, engineering principles, and physical consistency. Consequently, directly generating executable CAD code is insufficient for constructing mechanically valid and reusable assemblies. We present AssemCAD, an axiom-grounded framework for production-ready CAD assembly generation from natural language. Instead of representing an assembly as monolithic CAD code, AssemCAD first constructs an axiomatic Assembly Specification consisting of typed parts, geometry-backed ports, executable mates, and engineering axioms. Each assembly relation is explicitly grounded in one or more engineering principles, making the resulting specification interpretable, reusable, and verifiable. To realize this specification, AssemCAD introduces a port- and mate-based CAD assembly library that executes symbolic assembly relations through deterministic mate transformations and validates declared interfaces using concrete B-Rep geometric evidence. Built on this representation and library, AssemCAD further supports on-demand synthesis of reusable parametric component factories for both standard and open-world geometries. Experiments on AssemBench show that AssemCAD substantially improves assembly preservation and physical validity over code-centric CAD generation baselines, while generalizing across different foundation-model backbones. By combining axiom-grounded assembly reasoning with deterministic geometric execution, AssemCAD extends Text-to-CAD from isolated part generation toward production-ready mechanical assembly design.

Chinese Translation

最近在大型语言模型和程序化CAD方面的进展显著改善了单个部件的文本到CAD生成。然而，生产就绪的机械装配生成仍然基本未得到解决。与单部件建模不同，装配需要对多个组件、功能接口、装配关系、工程原理和物理一致性进行协调推理。因此，直接生成可执行的CAD代码不足以构建机械有效且可重用的装配。我们提出了AssemCAD，这是一个基于公理的框架，用于从自然语言生成生产就绪的CAD装配。AssemCAD并不是将装配表示为单一的CAD代码，而是首先构建一个公理化的装配规范，其中包括类型化的部件、基于几何的端口、可执行的配合和工程公理。每个装配关系都明确基于一个或多个工程原理，使得生成的规范可解释、可重用且可验证。为了实现这一规范，AssemCAD引入了一个基于端口和配合的CAD装配库，通过确定性配合变换执行符号装配关系，并使用具体的B-Rep几何证据验证声明的接口。在这一表示和库的基础上，AssemCAD进一步支持按需合成可重用的参数化组件工厂，适用于标准和开放世界几何体。在AssemBench上的实验表明，AssemCAD在装配保持和物理有效性方面显著优于以代码为中心的CAD生成基线，同时能够在不同的基础模型框架中进行泛化。通过将基于公理的装配推理与确定性几何执行相结合，AssemCAD将文本到CAD的应用从孤立的部件生成扩展到生产就绪的机械装配设计。

View on arXiv Download PDF AI Translation

cs.AI / 80 / 2607.05131

TacReasoner: A Dynamic Tactile-Language Framework for Interactive Reasoning in Real-World Scenarios

TacReasoner：一种用于真实场景交互推理的动态触觉语言框架

Lyu, Kailin, Wu, Di, Xiao, Long, Zeng, Jianning, He, Jianwei, Lin, Chang, Hu, Lianyu, Shu, Lin, Hao, Jie, Hao, Ce

Abstract

Among the five primary human senses, tactile is arguably the most fundamental to survival, as it enables the perception of physical contact and interaction in real-world environments. In this paper, we explore two key challenges of integrating tactile sensing into intelligent systems for multimodal reasoning: (i) insufficient modeling of dynamic tactile signals, which restricts reasoning over temporally evolving properties, and (ii) hallucination in tactile foundation models caused by the absence of explicit reasoning mechanisms, leading to unstable real-world inference. To address these challenges, we propose TacReasoner, a dynamic tactile-language framework for interactive reasoning in real-world scenarios. First, TacReasoner incorporates a Dynamic-aware Tactile Encoder to enhance the perception and representation of dynamic tactile signals. More importantly, we introduce TouchCoT-10k, the first tactile chain-of-thought dataset for structured reasoning over tactile inputs. Upon it, we establish DynTac-Bench to systematically evaluate dynamic tactile perception and real-world commonsense reasoning. Experimental results demonstrate that TacReasoner achieves competitive performance against state-of-the-art models across multiple datasets. Notably, despite using only 7B parameters, TacReasoner outperforms the 14B VTV-LLM model on most subtasks, highlighting its effectiveness and efficiency in tactile commonsense reasoning.

Chinese Translation

在五种主要的人类感官中，触觉无疑是生存中最基本的感官，因为它使我们能够感知现实环境中的物理接触和互动。本文探讨了将触觉感知集成到智能系统中的多模态推理时面临的两个关键挑战：（i）动态触觉信号建模不足，限制了对时间演变特性的推理；（ii）由于缺乏明确的推理机制，触觉基础模型中的幻觉现象导致现实世界推理不稳定。为了解决这些挑战，我们提出了TacReasoner，一种用于真实场景交互推理的动态触觉语言框架。首先，TacReasoner结合了动态感知触觉编码器，以增强对动态触觉信号的感知和表征。更重要的是，我们引入了TouchCoT-10k，这是第一个用于结构化推理的触觉思维链数据集。在此基础上，我们建立了DynTac-Bench，以系统地评估动态触觉感知和现实世界常识推理。实验结果表明，TacReasoner在多个数据集上表现出与最先进模型相媲美的性能。值得注意的是，尽管仅使用了70亿参数，TacReasoner在大多数子任务上仍优于140亿参数的VTV-LLM模型，突显了其在触觉常识推理中的有效性和效率。

View on arXiv Download PDF AI Translation

cs.AI / 81 / 2607.05147

DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation

DSpark：具有半自回归生成的信心调度推测解码

Cheng, Xin, Yu, Xingkai, Shao, Chenze, Li, Jiashi, Xiong, Yunfan, Qian, Yi, Zhu, Jiaqi, Ma, Shirong, Zhang, Xiaokang, Ye, Jiasheng, Chen, Qinyu, Deng, Chengqi, Yu, Jiping, Dai, Damai, Zhang, Zhengyan, Wei, Yixuan, Tan, Yixuan, Yang, Wenkai, Xu, Runxin, Wu, Yu, Xu, Zhean, Wang, Xuanyu, Chen, Muyang, Tian, Rui, Bi, Xiao, Hao, Zhewen, Chen, Shaoyuan, Cao, Huanqi, Zhang, Wentao, Xu, Anyi, Zhang, Huishuai, Zhao, Dongyan, Liang, Wenfeng

Abstract

Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification. While recent parallel drafters efficiently propose long token sequences in a single forward pass, they suffer from rapid acceptance decay due to a lack of inter-token dependencies. Furthermore, indiscriminately verifying these extended blocks wastes critical batch capacity on tokens with high rejection risks, severely degrading throughput in high-concurrency serving systems. We introduce DSpark, a speculative decoding framework that unifies high-throughput parallel generation with adaptive, load-aware verification. To maintain draft quality, DSpark utilizes a semi-autoregressive architecture, coupling a parallel backbone with a lightweight sequential module, to introduce intra-block dependency modeling and mitigate suffix decay. To optimize system efficiency, DSpark employs confidence-scheduled verification, dynamically tailoring the verification length for each request based on estimated prefix survival probabilities and engine-specific throughput profiles. On offline benchmarks across diverse domains, DSpark substantially improves the accepted length over state-of-the-art autoregressive and parallel drafters. When deployed within the DeepSeek-V4 serving system under live user traffic, DSpark successfully mitigates verification waste. Compared to the established production baseline (MTP-1), DSpark accelerates per-user generation speeds by 60 to 85 percent at matched throughput levels. More importantly, by preventing severe throughput degradation under strict interactivity constraints, it enables performance tiers that were previously unattainable, shifting the Pareto frontier of our serving system.

Chinese Translation

推测解码通过将草稿生成与目标验证解耦，从而加速大型语言模型（LLM）的推理。尽管最近的并行草稿生成器能够在单次前向传递中高效地提出长序列的标记，但由于缺乏标记间的依赖关系，它们面临快速接受率下降的问题。此外，毫无区别地验证这些扩展块会浪费关键的批处理能力在高拒绝风险的标记上，严重降低高并发服务系统的吞吐量。我们提出了DSpark，一种将高吞吐量并行生成与自适应、负载感知验证相结合的推测解码框架。为了保持草稿质量，DSpark利用半自回归架构，将并行主干与轻量级顺序模块相结合，引入块内依赖建模并减轻后缀衰减。为了优化系统效率，DSpark采用信心调度验证，根据估计的前缀存活概率和引擎特定的吞吐量特征动态调整每个请求的验证长度。在跨多个领域的离线基准测试中，DSpark显著提高了接受长度，相较于最先进的自回归和并行草稿生成器。在DeepSeek-V4服务系统中，在实时用户流量下部署DSpark，成功减少了验证浪费。与现有的生产基线（MTP-1）相比，DSpark在匹配吞吐量水平下将每用户生成速度提高了60%至85%。更重要的是，通过防止在严格交互约束下的吞吐量严重下降，它实现了之前无法达到的性能层次，推动了我们服务系统的帕累托前沿。

View on arXiv Download PDF AI Translation

cs.AI / 82 / 2607.05168

The Changing Role of Symbolic Methods in Artificial Intelligence

符号方法在人工智能中的变化角色

Sun, Jun

Abstract

Why do intelligent systems need to perform explicit symbolic reasoning? Computer science has traditionally regarded symbolic reasoning as a defining component of intelligence. Yet the remarkable success of modern foundation models raises a fundamental question: if increasingly capable AI systems can operate with little explicit symbolic reasoning, what role do symbolic methods actually play? This article argues that explicit symbolic reasoning is not a fundamental property of intelligence, but a computational consequence of operating on simplified models of reality. We propose the Compression Principle: every computational model is a simplified representation of reality, and explicit symbolic reasoning compensates for information omitted during model construction. From this principle, we derive the Modeling--Reasoning Trade-off: as computational models preserve richer representations of the world, the need for explicit symbolic reasoning correspondingly decreases. This perspective provides a unified explanation for both the historical success of symbolic methods and the remarkable effectiveness of modern foundation models. Paradoxically, the same development makes symbolic methods increasingly important for humans. As intelligent systems become more capable and more opaque, symbolic representations increasingly serve as interfaces through which humans specify requirements, verify behavior, regulate autonomous systems, and establish trust. We therefore argue that the future of symbolic methods lies not primarily as the computational engine of intelligent systems, but as the symbolic interface between increasingly capable AI systems and the humans who build, govern, and depend upon them.

Chinese Translation

为什么智能系统需要进行明确的符号推理？计算机科学传统上将符号推理视为智能的一个定义性组成部分。然而，现代基础模型的显著成功提出了一个根本性问题：如果日益强大的人工智能系统可以在几乎没有明确符号推理的情况下运作，那么符号方法实际上扮演着什么角色？本文论证，明确的符号推理并不是智能的基本属性，而是基于简化现实模型进行运算的计算结果。我们提出了压缩原则：每个计算模型都是现实的简化表示，而明确的符号推理则弥补了在模型构建过程中省略的信息。基于这一原则，我们推导出建模-推理权衡：随着计算模型保留对世界更丰富的表示，对明确符号推理的需求相应减少。这一视角为符号方法的历史成功与现代基础模型的显著有效性提供了统一的解释。矛盾的是，同样的发展使得符号方法对人类变得越来越重要。随着智能系统变得更加强大和不透明，符号表示越来越多地作为人类指定需求、验证行为、调节自主系统和建立信任的接口。因此，我们认为符号方法的未来不主要在于作为智能系统的计算引擎，而在于作为日益强大的人工智能系统与构建、管理和依赖它们的人类之间的符号接口。

View on arXiv Download PDF AI Translation

cs.AI / 83 / 2607.05174

AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments

AgentGym2：在去理想化的真实世界环境中对大型语言模型代理进行基准测试

Xi, Zhiheng, Yang, Dingwen, Liu, Jiaqi, Huang, Jixuan, Guo, Honglin, Huang, Baodai, Chen, Tinggang, Zhang, Qi, Lu, Zhonghang, Liu, Chenyu, Sun, Jiajun, Zhang, Jiazheng, Zhu, Dingwei, Guo, Xin, Wang, Junzhe, Zhang, Zhihao, Yang, Yuming, Ye, Junjie, Gao, Minghe, Liu, Dongrui, Ji, Jiaming, Li, Guohao, Gui, Tao, Zhang, Qi, Huang, Xuanjing

Abstract

Language agents, i.e., LLM agents, progress rapidly and are increasingly deployed in production environments. This trend underscores the urgent need for rigorous and realistic evaluations. However, most existing benchmarks evaluate agents in simplified, idealized settings. They typically rely on pre-packaged tool interfaces, overlook critical steps, and assume inputs are clean and fully specified. Consequently, they understate the difficulty of real deployments, where uncertainty and noise are ubiquitous and agents must proactively explore the environment to uncover new tools. To bridge this gap, we present AgentGym2, a new evaluation framework with task instances grounded in real-world end-to-end working demands. Beyond reasoning and planning, it measures agents' ability to execute end-to-end procedures, discover tools via exploration, compose tools for unseen tasks, and remain robust to noisy and underspecified information. Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2, revealing a substantial gap between the capability of current agents and the demands of real-world applications.

Chinese Translation

语言代理，即大型语言模型（LLM）代理，正在迅速发展，并越来越多地部署在生产环境中。这一趋势凸显了对严格和现实评估的迫切需求。然而，现有的大多数基准测试在简化的理想化环境中评估代理。它们通常依赖于预先包装的工具接口，忽视关键步骤，并假设输入是干净且完全指定的。因此，它们低估了真实部署的难度，在真实环境中，不确定性和噪声无处不在，代理必须主动探索环境以发现新工具。为了解决这一问题，我们提出了AgentGym2，一个新的评估框架，其任务实例基于真实世界的端到端工作需求。除了推理和规划外，它还衡量代理执行端到端程序的能力，通过探索发现工具、为未见任务组合工具，以及对噪声和不完全信息保持鲁棒性的能力。在15个专有和开源模型上的实验表明，即使是像Gemini和GPT-5这样的最先进系统在AgentGym2上也面临困难，揭示了当前代理能力与真实世界应用需求之间的显著差距。

View on arXiv Download PDF AI Translation

cs.AI / 84 / 2607.05177

CP-WSP: A Declarative CP-SAT Framework for Configurable Multi-Constraint Workforce Scheduling

CP-WSP：一种用于可配置多约束劳动力调度的声明式 CP-SAT 框架

Patel, Vipul, Deodhar, Anirudh, Birru, Dagnachew

Abstract

Workforce scheduling is an NP-hard combinatorial optimization problem requiring simultaneous satisfaction of labor regulations, coverage requirements, employee preferences and operational objectives. Existing CP formulations typically model simplified instances with 6-12 constraints at shift-level granularity and critically lack explicit support for: mandatory break scheduling with midpoint placement control; acuity weighted workload equity; sub-shift temporal granularity enabling demand-driven staffing; inter-week schedule stability; and cross-midnight shift patterns common in 24-hour operations. This paper presents CP-WSP: a declarative CP-SAT framework enforcing 14 hard constraints as mathematically inviolable requirements (zero regulatory violations by construction) while optimizing 15 soft objectives through a unified weighted penalty function -- all configurable via a JSON specification with no code changes required. Key contributions include: a shift-window variable decomposition enabling mandatory break scheduling with centrality control; acuity-weighted workload equity; multi-granularity temporal resolution from 30 minutes to 2 hours; inter-week schedule stability; a grid-offset preprocessing technique for cross-midnight shifts; and a reproducible 36-configuration benchmark suite for community comparison. Evaluated on INRC-II benchmarks at both hourly and shift-level granularity and on 36 synthetic configurations.

Chinese Translation

劳动力调度是一个 NP-hard 组合优化问题，要求同时满足劳动法规、覆盖要求、员工偏好和运营目标。现有的约束规划（CP）模型通常仅针对简化实例进行建模，具有 6-12 个约束，且在以下方面严重缺乏明确支持：强制休息时间调度及中点位置控制；基于急迫性加权的工作负载公平性；支持需求驱动的子班次时间粒度；周间调度稳定性；以及在 24 小时运营中常见的跨午夜班次模式。本文提出了 CP-WSP：一个声明式 CP-SAT 框架，强制执行 14 个硬约束作为数学上不可侵犯的要求（通过构建实现零法规违规），同时通过统一的加权惩罚函数优化 15 个软目标——所有配置均可通过 JSON 规范进行，无需代码更改。主要贡献包括：一种班次窗口变量分解方法，支持强制休息时间调度及中心性控制；基于急迫性加权的工作负载公平性；从 30 分钟到 2 小时的多粒度时间分辨率；周间调度稳定性；一种用于跨午夜班次的网格偏移预处理技术；以及一个可重复的 36 种配置基准套件，供社区比较。该框架在 INRC-II 基准上进行了评估，涵盖了每小时和班次级别的粒度，以及 36 种合成配置。

View on arXiv Download PDF AI Translation

cs.AI / 85 / 2607.05184

Rethinking On-Policy Self-Distillation for Thinking Models

重新思考思维模型的在线自蒸馏

Kaur, Simran, Ri, Narutatsu, He, Yinghui, Fowl, Liam, Arora, Sanjeev

Abstract

Self-distillation is a promising recipe for self-improvement in language models. In this setting, a model can serve as its own teacher when given privileged information, such as a solution to a math problem. This seems especially appealing for thinking models, which can use test-time reasoning to absorb the privileged information. Surprisingly, we show that privileged self-distillation degrades thinking models on long reasoning traces: across five Qwen3 and OLMo thinking models evaluated on AIME24, AIME25, and HMMT25, privileged-context distillation causes a relative drop of up to 17% in avg@16 accuracy. The degradation scales with the amount of privileged context withheld from the student and is most pronounced at long rollout budgets, where thinking models otherwise obtain their largest gains. This failure mode is not specific to self-distillation: on-policy distillation (OPD) improves thinking models, but privileged OPD reverses these gains. Our diagnostics link this failure mode to how privileged teacher context reshapes learning at high-entropy forking positions, where multiple continuations remain plausible and may lead to different reasoning paths. Privileged context lowers fork rates in thinking-model rollouts but not in instruction-model rollouts. This leads to an interesting dichotomy, where privileged context can help instruction-tuned models but hurts stronger thinking models. The effect is visible when the student begins a self-correction branch, where privileged OPD penalizes sampled reconsideration tokens that vanilla OPD supports. Thinking models trained with a privileged teacher produce fewer verification, backtracking, and hedging markers, even after length normalization. These findings indicate that self-distillation for strong thinking models requires attention to token-level signal, especially around correction and reasoning steps.

Chinese Translation

自蒸馏是语言模型自我改进的一个有前景的方法。在这种设置下，当模型获得特权信息（例如数学问题的解答）时，可以充当自己的教师。这对于思维模型尤其具有吸引力，因为它们可以利用测试时推理来吸收特权信息。令人惊讶的是，我们发现特权自蒸馏在长推理轨迹上会降低思维模型的性能：在五个 Qwen3 和 OLMo 思维模型的评估中，针对 AIME24、AIME25 和 HMMT25，特权上下文蒸馏导致 avg@16 准确率相对下降高达 17%。这种降级与 withheld 的特权上下文量成正比，并在长展开预算下最为明显，此时思维模型本可以获得最大的收益。这种失败模式并非特定于自蒸馏：在线蒸馏（OPD）能够改善思维模型，但特权 OPD 反而抵消了这些收益。我们的诊断将这种失败模式与特权教师上下文在高熵分叉位置重塑学习的方式联系起来，在这些位置，多条延续路径仍然是合理的，并可能导致不同的推理路径。特权上下文降低了思维模型展开中的分叉率，但对指令模型展开没有影响。这导致了一个有趣的二分法，即特权上下文可以帮助经过指令调优的模型，但对更强的思维模型却有害。当学生开始自我修正分支时，这种效应尤为明显，此时特权 OPD 会惩罚 vanilla OPD 支持的采样重新考虑标记。使用特权教师训练的思维模型产生的验证、回溯和犹豫标记更少，即使在长度归一化后也是如此。这些发现表明，对于强大的思维模型，自蒸馏需要关注标记级信号，特别是在修正和推理步骤周围。

View on arXiv Download PDF AI Translation

cs.AI / 86 / 2607.05185

ClassicLogic: A Knowledge-Driven Benchmark of Classic Puzzle Games for Evaluating Compositional Generalization

ClassicLogic：用于评估组合泛化的经典益智游戏知识驱动基准

Shahid, Mahnoor, Rothe, Hannes

Abstract

Compositional generalization, the ability to understand and produce novel combinations of known components, remains a fundamental challenge for modern artificial intelligence. While few benchmarks exist, many focus on linguistic tasks and lack complex, explicit compositional structures. We introduce ClassicLogic, a new benchmark suite designed to evaluate an agent's ability to learn and compose problem-solving strategies. The benchmark consists of four classic logic puzzles: Sudoku, KenKen, Kakuro, and Futoshiki. Its core innovation is a hierarchical, explicit knowledge base for each game, where complex solving strategies are formally defined as compositions of simpler, foundational strategies. This structure allows for fine-grained evaluation of an agent's reasoning capabilities, from learning basic rules to applying multi-step compositional strategies to solve puzzles of increasing, mathematically validated difficulty. The open-source benchmark provides a challenging new testbed for advancing neuro-symbolic and other advanced AI reasoning systems.

Chinese Translation

组合泛化，即理解和生成已知组件的新组合的能力，仍然是现代人工智能面临的一个基本挑战。虽然现有的基准很少，但许多基准专注于语言任务，缺乏复杂的、明确的组合结构。我们介绍了ClassicLogic，一个新的基准套件，旨在评估智能体学习和组合问题解决策略的能力。该基准由四个经典逻辑谜题组成：数独（Sudoku）、肯肯（KenKen）、卡库罗（Kakuro）和富图希基（Futoshiki）。其核心创新是为每个游戏建立一个分层的、明确的知识库，其中复杂的解决策略被正式定义为更简单、基础策略的组合。这一结构允许对智能体的推理能力进行细致的评估，从学习基本规则到应用多步骤组合策略解决逐渐增加的、经过数学验证的难度谜题。该开源基准为推动神经符号（neuro-symbolic）及其他先进人工智能推理系统提供了一个具有挑战性的测试平台。

View on arXiv Download PDF AI Translation

cs.AI / 87 / 2607.05199

Reason, Reward, Refine: Step-Level Errors Corrections with Structured Feedback for Physics Reasoning in Small Language Models

推理、奖励、修正：小型语言模型中基于结构化反馈的逐步错误修正与物理推理

Jaiswal, Raj, Jain, Dhruv, Dhawan, Rishabh, Uppalapati, Sree Krishna, Satoh, Shin'ichi, Ganu, Tanuja, Shah, Rajiv Ratn

Abstract

Physics reasoning fails structurally in small language models: an error at any step propagates forward, corrupting every inference that follows. Limited domain knowledge, hallucination under multi-step derivation, and distributional sensitivity compound this failure. We propose a step-level reward framework that identifies the first reasoning error, generates targeted structured feedback, and trains the model to revise its solution via policy gradient with KL regularization, without exposing it to ground truth solutions as generation targets. Unlike annotation-dependent step-level methods, no preference data construction is required and the external verifier operates exclusively at training time. Across five physics benchmarks, our framework delivers accuracy gains of 17-20% over CoT prompting and 10-16% over the strongest baseline, reduces calculation errors from 56.9% to 23.5%, and reduces miscomprehension errors from 22.3% to 12.0% in the best observed cases. Conceptual errors reduce from 89.7% to 68.7%, yet persist as the hardest failure mode across all conditions.

Chinese Translation

小型语言模型在物理推理中结构性失败：任何步骤的错误都会向前传播，破坏随后的每一个推理。有限的领域知识、多步骤推导下的幻觉以及分布敏感性加剧了这种失败。我们提出了一种逐步奖励框架，该框架识别第一个推理错误，生成针对性的结构化反馈，并通过策略梯度与KL正则化训练模型修正其解决方案，而不将其暴露于真实解决方案作为生成目标。与依赖注释的逐步方法不同，该方法不需要偏好数据构建，外部验证者仅在训练时操作。在五个物理基准测试中，我们的框架相比于链式推理（CoT）提示提高了17-20%的准确率，相比于最强基线提高了10-16%，将计算错误从56.9%降低至23.5%，并在最佳观察情况下将误解错误从22.3%降低至12.0%。概念错误从89.7%降低至68.7%，但在所有条件下仍然是最难的失败模式。

View on arXiv Download PDF AI Translation

cs.AI / 88 / 2607.05202

EvoAgentBench: Benchmarking Agent Self-Evolution via Ability Transfer

EvoAgentBench：通过能力转移评估智能体自我进化

Gao, Xingze, Hu, Chuanrui, Chen, Hongda, Yao, Pengfei, Wang, Zhao, Bai, Yi, Wu, Zhengwei, Han, Yunyun, Cong, Xiaofeng, Gui, Jie, Deng, Yafeng, Li, Teng

Abstract

Agent self-evolution in long-horizon LLM systems is largely procedural: useful experience is not merely stored information, but reusable procedures for searching, debugging, and verification. Yet current evaluations do not isolate this form of transfer. Agent benchmarks test single-episode task solving; memory benchmarks target information retention rather than procedural reuse. We introduce EvoAgentBench, a benchmark for agent self-evolution via Ability-guided transfer across four agentic domains: web research, algorithmic reasoning, software engineering, and knowledge work. EvoAgentBench extracts trace-grounded Abilities from agent executions, canonicalizes them into operational units, and builds domain-specific Ability Graphs linking tasks that share procedural overlap. By design, every test task is backed by verified training-side Ability support. Across a 528/267 train/test split, two scaffolds, and three backbones, curated Ability content transfers reliably across model families, but no current automatic method sustains positive gain in all settings. EvoAgentBench shifts self-evolution evaluation from aggregate accuracy comparison to fine-grained diagnosis of experience encoding, routing, and uptake. The benchmark is publicly available at https://huggingface.co/datasets/EverMind-AI/EvoAgentBench.

Chinese Translation

在长时间跨度的LLM系统中，智能体自我进化主要是一个程序性过程：有用的经验不仅仅是存储的信息，而是用于搜索、调试和验证的可重用程序。然而，目前的评估并未孤立这种转移形式。智能体基准测试单一情境任务的解决；记忆基准测试则侧重于信息保留而非程序重用。我们引入了EvoAgentBench，这是一个通过能力引导转移评估智能体自我进化的基准，涵盖四个智能体领域：网络研究、算法推理、软件工程和知识工作。EvoAgentBench从智能体执行中提取基于轨迹的能力，将其规范化为操作单元，并构建领域特定的能力图，链接共享程序重叠的任务。根据设计，每个测试任务都得到经过验证的训练侧能力支持。在528/267的训练/测试划分、两个支架和三个骨干网络中，策划的能力内容在模型家族间可靠转移，但目前没有任何自动方法在所有设置中维持正向增益。EvoAgentBench将自我进化评估从整体准确性比较转变为对经验编码、路由和获取的细致诊断。该基准公开可用，网址为 https://huggingface.co/datasets/EverMind-AI/EvoAgentBench。

View on arXiv Download PDF AI Translation

cs.AI / 89 / 2607.05238

MoP-JEPA: Hard-Assigned Predictor Mixtures for Stochastic JEPA World Models

MoP-JEPA：用于随机 JEPA 世界模型的硬分配预测器混合

Song, Zhi, Xing, Ximing, Tang, Zhenchao, Huang, hanbo, Lv, Tianxu, Yang, minghao, Niu, Zhongzheng, Bing, He, Wang, Lusheng, Yao, Jianhua

Abstract

JEPA world models predict the next latent state with a single deterministic predictor trained by latent regression. We show that this fails structurally when the environment is stochastic: at a branching transition, the regression-optimal predictor outputs the conditional mean of the successor embeddings, a point between the true next states that corresponds to no state at all. We prove this collapse for deterministic and gated mixture-of-experts predictors, and prove that MoP-JEPA's hard-assigned predictors converge instead to a quantizer of the transition distribution: one head per successor mode, enumerable in a single forward pass, which is the interface a planner consumes. On official OGBench offline data with leak-free evaluation, planning over single-predictor rollouts performs poorly ($0.02$--$0.09$ success) while planning over our predicted modes reaches up to $0.85$, ahead of deterministic, gated-MoE, and variational predictors on every task. Because multi-prediction evaluation invites coverage freeloading, a verification protocol is part of the method: an input-agnostic codebook control, a shuffled-context test, router-gated readouts, transition-precision guards, and a verified-route criterion in which the model proposes its transition graph blind and ground truth is used only to check the result. Under this criterion our method outperforms the strongest soft alternative on all three mazes ($2$--$5\times$), and the protocol identifies the remaining gap in that baseline's raw scores as routes through predicted transitions that do not exist. The same model executes in the real environment, placing second of seven against the published OGBench baselines on the hardest maze. Multimodal dynamics decide whether a JEPA world model can plan at all; a mixture of predictors with hard assignment is a minimal and verifiable fix.

Chinese Translation

JEPA 世界模型通过潜在回归训练的单一确定性预测器来预测下一个潜在状态。我们表明，当环境是随机时，这种方法在结构上是失败的：在分支转变处，回归最优预测器输出后继嵌入的条件均值，这个点位于真实下一个状态之间，实际上并不对应任何状态。我们证明了这一崩溃对于确定性和门控专家混合预测器的情况，并证明 MoP-JEPA 的硬分配预测器则收敛于转变分布的量化器：每个后继模式一个头，在单次前向传播中可枚举，这是规划者所需的接口。在官方的 OGBench 离线数据上，进行无泄漏评估时，基于单一预测器的规划表现不佳（成功率为 $0.02$--$0.09$），而基于我们预测的模式的规划成功率高达 $0.85$，在每个任务上均优于确定性、门控 MoE 和变分预测器。由于多重预测评估可能导致覆盖利用，验证协议是该方法的一部分：输入无关的代码本控制、随机上下文测试、路由门控读取、转变精度保护以及验证路径标准，其中模型盲目提出其转变图，仅使用真实数据来检查结果。在这一标准下，我们的方法在所有三个迷宫中都超越了最强的软替代方案（$2$--$5 imes$），而该协议识别出基线原始分数中剩余的差距是通过不存在的预测转变的路径。相同的模型在真实环境中执行，在最困难的迷宫中排名第二，七个参与者中仅次于已发布的 OGBench 基线。多模态动态决定了 JEPA 世界模型是否能够进行规划；具有硬分配的预测器混合是一个最小且可验证的修复方案。

View on arXiv Download PDF AI Translation

cs.AI / 90 / 2607.05297

MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution

MetaSkill-Evolve：通过双时间尺度元技能演化实现大语言模型代理的递归自我改进

Wang, Zefeng, Yan, Minxi, Bi, Jinhe, Yan, Sikuan, Tresp, Volker, Ma, Yunpu

Abstract

Recent LLM agents tackle increasingly long-horizon, open-ended tasks, and external skills, reusable procedural knowledge supplied to the agent, further extend this capability. However, a fixed, hand-authored skill is rarely optimal, and cannot adapt to the diversity of tasks an agent encounters. Self-improving agents address this by rewriting their own skill files from execution traces, yielding meaningful gains on challenging benchmarks. Yet such self-evolution remains non-recursive: it improves only the task skill (what the agent does) while the improvement procedure (how it improves) is authored once and held fixed. We introduce MetaSkill-Evolve, a two-timescale framework that makes agentic skill improvement recursive: every branch carries both a task skill $s$ and a branch-local meta-skill $m=(\psi,\sigma,\alpha,\pi,\varepsilon)$ whose five components parameterise the Analyzer, Retriever, Allocator, Proposer, and Evolver agents of the improvement pipeline. Task skills evolve on a fast loop while the meta-skill evolves on a slower one under the same pipeline applied to itself, with no additional model or objective. With all five pipeline agents sharing a single frozen backbone, MetaSkill-Evolve outperforms no-skill, static-skill, and single-level evolution baselines on three agentic benchmarks (OfficeQA, SealQA, ALFWorld), improving held-out test accuracy over the raw backbone by +23.54, +16.09, and +1.92 points respectively.

Chinese Translation

近期的大语言模型（LLM）代理处理越来越长远、开放式的任务，而外部技能，即提供给代理的可重用程序知识，进一步扩展了这一能力。然而，固定的、手工编写的技能很少是最优的，无法适应代理所遇到的任务多样性。自我改进的代理通过从执行轨迹中重写自己的技能文件来解决这一问题，在具有挑战性的基准测试中取得了显著的提升。然而，这种自我演化仍然是非递归的：它仅改善任务技能（代理所做的事情），而改进过程（如何改进）则是一次性编写并保持固定。我们提出了MetaSkill-Evolve，一个双时间尺度框架，使代理技能的改进变得递归：每个分支同时携带任务技能 $s$ 和一个分支局部元技能 $m=( heta, au,eta, ho, u)$，其五个组成部分参数化了改进管道中的分析器（Analyzer）、检索器（Retriever）、分配器（Allocator）、提议者（Proposer）和进化者（Evolver）代理。任务技能在快速循环中演变，而元技能在应用于自身的同一管道下以较慢的速度演变，无需额外的模型或目标。通过所有五个管道代理共享一个固定的主干，MetaSkill-Evolve 在三个代理基准（OfficeQA、SealQA、ALFWorld）上超越了无技能、静态技能和单层演化的基准，分别提高了相对于原始主干的保留测试准确率 +23.54、+16.09 和 +1.92 个百分点。

View on arXiv Download PDF AI Translation

cs.AI / 91 / 2607.05310

Evaluating and Understanding Model Editing for Medical Vision Language Models

评估与理解医学视觉语言模型的模型编辑

Zhu, Guli, Wu, Chenwei, Shen, Liyue

Abstract

Model editing promises a fast, targeted way to correct post-deployment mistakes in medical vision-language models (VLMs) without costly retraining. However, existing multimodal model editing benchmarks focus on general-purpose tasks and do not reflect realistic clinical domain requirements and variability. To address this, we introduce M3Bench, a clinically grounded benchmark for multimodal model editing that evaluates whether an edit remains reliable, precise, and generalizable under the challenges of image and text variation, modality and protocol shifts, clinical knowledge composition, and temporal progression. M3Bench contains 16,276 questions spanning diverse anatomy, modalities, and specialties, and supports both single and sequential edits. By evaluating 4 representative editors across 6 medical and general VLMs, we find that no method excels across all criteria. Gradient-based editors achieve strong transfer but suffer from catastrophic locality violations, whereas memory-based methods preserve locality but lack compositional generality and exhibit high backbone-dependent hyperparameter sensitivity. We further attribute these failures to the latent space geometry of VLMs and how different editing methods shift its landscape. Overall, M3Bench establishes a rigorous clinical stress test for multimodal model editing and offers actionable guidance for safer post-deployment adaptation. The benchmark is publicly available at https://github.com/BioMed-AI-Lab-U-Michgan/M3Bench .

Chinese Translation

模型编辑提供了一种快速、针对性的方式来纠正医学视觉语言模型（VLMs）在部署后出现的错误，而无需昂贵的重新训练。然而，现有的多模态模型编辑基准主要集中在通用任务上，未能反映现实临床领域的需求和变异性。为此，我们引入了M3Bench，这是一个基于临床的多模态模型编辑基准，旨在评估在图像和文本变异、模态和协议转变、临床知识组成以及时间进展等挑战下，编辑是否保持可靠、精确和可推广。M3Bench包含16,276个问题，涵盖多样的解剖学、模态和专业领域，并支持单次和连续编辑。通过评估4种代表性编辑器在6个医学和通用VLMs上的表现，我们发现没有一种方法在所有标准上都表现出色。基于梯度的编辑器在迁移方面表现强劲，但遭遇了灾难性的局部性违反，而基于记忆的方法则保持了局部性，但缺乏组合通用性，并表现出对主干网络超参数的高度敏感性。我们进一步将这些失败归因于VLMs的潜在空间几何特性及不同编辑方法如何改变其景观。总体而言，M3Bench为多模态模型编辑建立了严格的临床压力测试，并为更安全的后期部署适应提供了可行的指导。该基准可在https://github.com/BioMed-AI-Lab-U-Michgan/M3Bench获取。

View on arXiv Download PDF AI Translation

cs.AI / 92 / 2607.05346

OptiAgent: End-to-End Optimization Modeling via Multi-Agent Iterative Refinement

OptiAgent：通过多智能体迭代优化建模的端到端方法

Monteiro, Adriana Laurindo, Fagundes, Nayse, Langeloh, Gabriel Mattos, Kanno, Gustavo de Oliveira, Aguirre, Priscila Louise, da Rocha, Thiago Costa Rizuti, Beltran, Victor Leme

Abstract

We propose OptiAgent, a multi-agent framework that, given a natural language description of an Operations Research problem, is able to output a solver-ready mathematical formulation as well as executable code. Our architecture prioritizes the mathematical modeling step, where dedicated agents extract structures, such as decision variables and constraints, enabling iterative self-correction. We introduce a novel multi-loop validation architecture with four specialized feedback mechanisms, each targeting a distinct failure mode such as misinterpretation, structural defects, mathematical inconsistencies, validation failures, and code errors. Alongside accuracy, our modular design improves the process of solving optimization problems by improving transparency, as each agent exposes its reasoning and feedback, making the full modeling process auditable. Our framework achieves state-of-the-art performance on 3 out of 4 benchmarks across LP, MILP, and Nonlinear Programming tasks, while remaining highly competitive on the remaining dataset.

Chinese Translation

我们提出了OptiAgent，一个多智能体框架，能够根据运筹学问题的自然语言描述输出可供求解的数学公式和可执行代码。我们的架构优先考虑数学建模步骤，其中专门的智能体提取结构，如决策变量和约束，从而实现迭代自我修正。我们引入了一种新颖的多循环验证架构，包含四种专门的反馈机制，每种机制针对不同的失败模式，如误解、结构缺陷、数学不一致性、验证失败和代码错误。除了准确性外，我们的模块化设计通过提高透明度改善了优化问题的求解过程，因为每个智能体都展示其推理和反馈，使整个建模过程可审计。我们的框架在LP、MILP和非线性规划任务的四个基准测试中有三个达到了最先进的性能，同时在剩余的数据集上也保持了高度竞争力。

View on arXiv Download PDF AI Translation

cs.AI / 93 / 2607.05359

Graph Sparse Sampling: Breaking the Curse of the Horizon in Continuous MDP Planning

图稀疏采样：打破连续马尔可夫决策过程规划中的地平线诅咒

Lev-Yehudi, Idan, Indelman, Vadim

Abstract

Planning under uncertainty in continuous domains is essential for autonomous systems, yet computationally demanding. Tree-based search methods such as Monte Carlo Tree Search (MCTS) remain popular, but their branching structure can require sampling budgets that grow exponentially with lookahead depth in the worst case. From a tree perspective, continuous state or action spaces become especially challenging, since the planner must decide where to search in an infinite branching hierarchy. We propose Graph Sparse Sampling (GSS), an online planning algorithm that shares sampled futures across many candidate decisions, rather than sampling separate successors for each candidate action. This branch-free graph exposes large GPU-friendly batches, while using heuristics to focus computation. We prove finite-sample performance guarantees for GSS covering full-rank or low-rank generative simulators via smoothed backups, and discrete or sampled continuous action spaces. Under suitable overlap, regularity, and action-coverage conditions, these bounds have polynomial dependence on the planning horizon, formalizing when shared futures can avoid the exponential horizon dependence of tree-shaped sparse sampling. We demonstrate continuous-control simulations where GSS substantially outperforms tree-based planners on long horizons or achieves near-optimal performance, supporting no-branching graph planning as a complementary design principle for online control.

Chinese Translation

在连续领域中进行不确定性规划对于自主系统至关重要，但计算上却非常耗费资源。基于树的搜索方法，如蒙特卡洛树搜索（Monte Carlo Tree Search, MCTS），仍然受到广泛欢迎，但其分支结构在最坏情况下可能需要随着前瞻深度呈指数增长的采样预算。从树的角度来看，连续状态或动作空间尤其具有挑战性，因为规划者必须决定在无限分支层次结构中在哪里进行搜索。我们提出了图稀疏采样（Graph Sparse Sampling, GSS），这是一种在线规划算法，它在多个候选决策之间共享采样的未来，而不是为每个候选动作单独采样后继。这种无分支图形暴露出大规模的适合GPU的批次，同时使用启发式方法来集中计算。我们证明了GSS的有限样本性能保证，涵盖了通过平滑备份的全秩或低秩生成模拟器，以及离散或采样的连续动作空间。在适当的重叠、规律性和动作覆盖条件下，这些界限对规划地平线具有多项式依赖性，正式化了共享未来如何避免树形稀疏采样的指数地平线依赖性。我们展示了连续控制模拟，其中GSS在长地平线上的表现显著优于基于树的规划器，或实现接近最优的性能，支持无分支图规划作为在线控制的补充设计原则。

View on arXiv Download PDF AI Translation

cs.AI / 94 / 2607.05363

SovereignPA-Bench: Evaluating User-Owned Personal Agents under Evolving Intent, Platform Mediation, and Consent Constraints

SovereignPA-Bench：在不断变化的意图、平台中介和同意约束下评估用户拥有的个人代理

Liu, Dylan Zongmin

Abstract

Personal agents are becoming persistent user-owned intermediaries: they remember preferences, filter platform-mediated information, use tools, and negotiate with services. Existing benchmarks evaluate tool use, web navigation, desktop control, personalization, recommendation, and evolving context, but rarely ask whether an agent preserves user sovereignty: advancing the user's current interests while respecting privacy, consent, evidence, user burden, and resistance to manipulative incentives. We introduce SovereignPA-Bench, an executable benchmark for evaluating user-owned personal agents under evolving intent, platform mediation, privacy boundaries, consent constraints, evidence requirements, and burden tradeoffs. The benchmark separates agent-visible ObservableState from evaluator-only HiddenLabels, reports component metrics for task success, alignment, privacy, consent, evidence, manipulation, burden, and auditability, and preserves paired scenario ordering for model and policy comparisons. We evaluate 120 sovereignty stress scenarios across 4 model families and 8 policy baselines, yielding 3,840 frozen-prompt trajectories with raw prompts, outputs, provider-form responses, parsed actions, recomputable metrics, hard-set analyses, qualitative cases, and a blinded 3-annotator audit over 240 items. Full-sovereign scaffolding improves sovereignty score over direct, memory-only, consent-only, evidence-only, ReAct/tool-use, safety-prompt, and judge-guard baselines while reducing privacy leakage, consent violation, over-concession, and manipulation capture. Human audit shows high agreement on privacy and consent and lower agreement on manipulation, identifying the subjective frontier of platform-persuasion judgments. These results show that personal-agent evaluation must move beyond task completion toward representative, consent-aware, evidence-grounded action.

Chinese Translation

个人代理正成为持久的用户拥有中介：它们记住偏好，过滤平台中介的信息，使用工具，并与服务进行协商。现有基准评估工具使用、网络导航、桌面控制、个性化、推荐和不断变化的上下文，但很少询问代理是否维护用户主权：在尊重隐私、同意、证据、用户负担和抵制操控性激励的同时推进用户当前的利益。我们引入了SovereignPA-Bench，这是一个可执行的基准，用于评估在不断变化的意图、平台中介、隐私边界、同意约束、证据要求和负担权衡下的用户拥有的个人代理。该基准将代理可见的可观察状态（ObservableState）与评估者专用的隐藏标签（HiddenLabels）分开，报告任务成功、对齐、隐私、同意、证据、操控、负担和可审计性等组件指标，并保持模型和政策比较的配对场景顺序。我们评估了120个主权压力场景，涵盖4个模型家族和8个政策基线，产生了3840个冻结提示轨迹，包含原始提示、输出、提供者形式的响应、解析的动作、可重新计算的指标、硬性分析、定性案例，以及对240个项目的盲审3评审员审计。完全主权的支架在减少隐私泄露、同意违规、过度让步和操控捕获的同时，提高了相较于直接、仅记忆、仅同意、仅证据、ReAct/工具使用、安全提示和评判守护基线的主权评分。人类审计显示在隐私和同意方面高度一致，而在操控方面一致性较低，识别出平台说服判断的主观边界。这些结果表明，个人代理的评估必须超越任务完成，朝着具有代表性、关注同意和基于证据的行动迈进。

View on arXiv Download PDF AI Translation

cs.AI / 95 / 2607.05391

LLM-as-a-Verifier: A General-Purpose Verification Framework

LLM作为验证器：通用验证框架

Kwok, Jacky, Li, Shulu, Atreya, Pranav, Liu, Yuejiang, Jiang, Yixing, Finn, Chelsea, Pavone, Marco, Stoica, Ion, Mirhoseini, Azalia

Abstract

Scaling pre-training, post-training, and test-time compute have become the central paradigms for improving the capabilities of LLMs. In this work, we identify verification, the ability to determine the correctness of a solution, as a new scaling axis. To unlock this and demonstrate its effectiveness, we introduce LLM-as-a-Verifier, a general-purpose verification framework that provides fine-grained feedback for agentic tasks without requiring additional training. Unlike standard LM judges that prompt LLMs to produce discrete scores for candidate solutions, LLM-as-a-Verifier computes the expectation over the distribution of scoring token logits to generate continuous scores. This probabilistic formulation enables verification to scale along multiple dimensions: (1) score granularity, (2) repeated evaluation, and (3) criteria decomposition. In particular, we show that scaling the scoring granularity leads to better separation between positive and negative solutions, resulting in more calibrated comparisons. Moreover, scaling repeated evaluation and criteria decomposition consistently lead to additional gains in verification accuracy through variance and complexity reduction. We further introduce a cost-efficient ranking algorithm for selecting the best solution among candidates using the verifier's continuous scores. LLM-as-a-Verifier achieves state-of-the-art performance on Terminal-Bench V2 (86.5%), SWE-Bench Verified (78.2%), RoboRewardBench (87.4%), and MedAgentBench (73.3%). Beyond verification, the fine-grained signals from LLM-as-a-Verifier can also serve as a proxy for estimating task progress. We build an extension for Claude Code, enabling developers to monitor and improve their own agentic systems. Finally, we show that LLM-as-a-Verifier can provide dense feedback for RL, improving the sample efficiency of SAC and GRPO on robotics and mathematical reasoning benchmarks.

Chinese Translation

扩展预训练、后训练和测试时计算已成为提升大型语言模型（LLMs）能力的核心范式。在本研究中，我们将验证，即确定解决方案正确性的能力，识别为一个新的扩展轴。为了释放这一潜力并展示其有效性，我们引入了LLM作为验证器（LLM-as-a-Verifier），这是一个通用验证框架，能够为自主任务提供细粒度反馈，而无需额外训练。与标准语言模型（LM）评判者通过提示LLMs生成候选解决方案的离散评分不同，LLM作为验证器计算评分标记对数的分布期望，以生成连续评分。这种概率性表述使得验证能够在多个维度上扩展：（1）评分粒度，（2）重复评估，以及（3）标准分解。特别地，我们展示了扩展评分粒度可以更好地区分正向和负向解决方案，从而实现更为精准的比较。此外，扩展重复评估和标准分解始终能够通过方差和复杂性降低来提高验证准确性。我们进一步引入了一种成本高效的排名算法，用于在候选中选择最佳解决方案，利用验证器的连续评分。LLM作为验证器在Terminal-Bench V2（86.5%）、SWE-Bench Verified（78.2%）、RoboRewardBench（87.4%）和MedAgentBench（73.3%）上实现了最先进的性能。除了验证，LLM作为验证器的细粒度信号也可以作为估计任务进展的代理。我们为Claude Code构建了一个扩展，使开发者能够监控和改进他们自己的自主系统。最后，我们展示了LLM作为验证器可以为强化学习（RL）提供密集反馈，提高了SAC和GRPO在机器人和数学推理基准上的样本效率。

View on arXiv Download PDF AI Translation

计算语言学 (Computation and Language)

107

cs.CL / 1 / 2607.02668

Improving LLMs via Validator-to-Generator Alignment

通过验证者与生成者对齐来提升大型语言模型

Rodriguez, Juan Diego, Zhang, Jocelyn, Erk, Katrin, Durrett, Greg

Abstract

Large language models are inconsistent: varying prompts or including unrelated information can lead to unexpected changes in model outputs. The generator-validator (G-V) gap is one manifestation of this phenomenon, where LLMs generate responses that they then deem as invalid if re-queried to validate them. In this work, we introduce a new formulation of G-V consistency that involves a principled correction for utterance frequency. Specifically, generators often assign low likelihood to valid strings simply because those strings are a priori unlikely, which makes naive notions of G-V consistency unworkable. We show that under a natural model of rational agents answering questions with multiple answers, consistency of the validator with a frequency-corrected generator score emerges naturally. Our method, \emph{\FCPAname} (\FCPA), is a training objective implementing frequency-corrected G-V consistency for real-world LLMs. Our experimental results show that training with \FCPA{} substantially improves both G-V consistency and generator performance over prior methods, with gains of up to $+27$pp in Pearson correlation on IFEval and HumanEval, while preserving validator quality across all evaluated tasks.

Chinese Translation

大型语言模型存在不一致性：不同的提示或包含无关信息可能导致模型输出的意外变化。生成者-验证者（G-V）差距是这一现象的一个表现，其中大型语言模型生成的响应在被重新询问以验证时被认为是无效的。在本研究中，我们引入了一种新的G-V一致性公式，该公式涉及对话频率的原则性修正。具体而言，生成者通常会对有效字符串分配低概率，仅仅因为这些字符串在先验上不太可能，这使得简单的G-V一致性概念无法奏效。我们展示了在一个自然的理性代理模型中，回答具有多个答案的问题时，验证者与频率修正生成者评分的一致性自然出现。我们的方法， extit{ extbf{FCPA}}（Frequency-Corrected Prompt Alignment），是一种实现频率修正G-V一致性的训练目标，适用于现实世界中的大型语言模型。我们的实验结果表明，使用 extbf{FCPA}进行训练显著提高了G-V一致性和生成者性能，相较于之前的方法，在IFEval和HumanEval上的Pearson相关性提升高达27个百分点，同时在所有评估任务中保持了验证者的质量。

View on arXiv Download PDF AI Translation

cs.CL / 2 / 2607.02734

Echoes of Unrest: A Multimodal NLP Framework for Early Warning of Fake News and Violence-Driven Mob Activity

动荡的回声：一种多模态自然语言处理框架用于假新闻和暴力驱动的暴动活动的早期预警

Bangabashi, Md. Maruf, Hasan, Tahmid, Mahmud, Golam, Rahman, Md. Mostafijur, Rahman, Md. Toufiqur, Biswas, Jahanur

Abstract

Rapid growth in social media has transformed global communication by enabling fast information exchange, but it has also accelerated the spread of misinformation. Fake news, manipulated content, and provocative narratives are increasingly linked to social unrest, political instability, and mob violence. Incidents in South Asia and elsewhere demonstrate how false information disseminated via platforms such as Facebook and WhatsApp can trigger real-world harm, often spreading faster than fact-checking efforts can respond. To address this challenge, this chapter presents a multilingual, multimodal Natural Language Processing (NLP) framework for early detection of misinformation and violence-prone dynamics. A fused dataset of 138,256 Bangla and English samples was created by combining multiple benchmark datasets. The framework integrates XLM-RoBERTa for multilingual text representation, CLIP for visual embedding, and a multi-head attention mechanism for multimodal fusion, enhanced with auxiliary features such as sarcasm and geospatial metadata. Experiments on a stratified 30% subset achieved 98% test accuracy with strong precision and recall. The outcomes show the efficacy of multimodal approaches in early misinformation detection and highlight the added value of geospatial signals for anticipating real-world escalation.

Chinese Translation

社交媒体的快速增长改变了全球通信，使信息交流变得迅速，但也加速了虚假信息的传播。假新闻、操控内容和挑衅性叙事与社会动荡、政治不稳定和暴力活动的联系日益紧密。南亚及其他地区的事件表明，通过Facebook和WhatsApp等平台传播的虚假信息如何引发现实世界的危害，且其传播速度往往快于事实核查的响应。为应对这一挑战，本章提出了一种多语言、多模态的自然语言处理（NLP）框架，用于早期检测虚假信息和暴力倾向动态。通过结合多个基准数据集，创建了一个包含138,256个孟加拉语和英语样本的融合数据集。该框架集成了XLM-RoBERTa用于多语言文本表示，CLIP用于视觉嵌入，以及多头注意力机制用于多模态融合，并增强了讽刺和地理空间元数据等辅助特征。在一个分层的30%子集上进行的实验达到了98%的测试准确率，并具有较强的精确度和召回率。结果显示了多模态方法在早期虚假信息检测中的有效性，并突出了地理空间信号在预测现实世界升级中的附加价值。

View on arXiv Download PDF AI Translation

cs.CL / 3 / 2607.02757

Reinforcement Learning for Data-Efficient Code-Switched ASR

用于数据高效的代码切换自动语音识别的强化学习

Ye, Ziwei, Vickers, Peter

Abstract

Audio-language models can be prompted for code-switched speech, but their decoding is not optimized for code-switching and often fails at language boundaries. We propose a practical reinforcement learning with verifiable rewards recipe for data-efficient adaptation of audio-language models to code-switched ASR using group relative policy optimization, combining an error rate reward with a script fidelity reward that penalizes wrong writing systems and a two-pass draft-and-refinement procedure. Using Qwen2-Audio as a reproducible testbed across 10 language pairs, training on only TTS code-switched speech, we show that RLVR with 10% of the data matches LoRA supervised fine-tuning trained on the full dataset, with the largest gains on typologically distant pairs. The error rate reward eliminates translation errors while the script fidelity reward separately reduces script contamination without degradation. These gains transfer zero-shot to a human-recorded code-switching corpus.

Chinese Translation

音频语言模型可以用于代码切换语音的提示，但其解码并未针对代码切换进行优化，常常在语言边界处失败。我们提出了一种实用的强化学习方法，结合可验证的奖励机制，以实现音频语言模型在代码切换自动语音识别（ASR）中的数据高效适应，该方法使用组相对策略优化，将错误率奖励与书写系统准确性奖励相结合，后者对错误的书写系统进行惩罚，并采用双通道草稿与精炼程序。在以 Qwen2-Audio 为可复现的测试平台，涵盖 10 对语言的实验中，仅使用 TTS 代码切换语音进行训练，我们展示了 RLVR 在 10% 数据量下的表现与在完整数据集上训练的 LoRA 监督微调相匹配，尤其在类型学上相距较远的语言对中取得了最大的提升。错误率奖励消除了翻译错误，而书写系统准确性奖励则在不降低性能的情况下单独减少了书写系统的污染。这些提升在零样本情况下转移到了人工录制的代码切换语料库上。

View on arXiv Download PDF AI Translation

cs.CL / 4 / 2607.02763

LuxSQA: Ask Me in Luxembourgish with TTS-Augmented Spoken Question Answering

LuxSQA：使用TTS增强的卢森堡语口语问答系统

Hosseini-Kivanani, Nina, Matassoni, Marco, Brutti, Alessio

Abstract

Spoken Question Answering (SQA) remains largely focused on high-resource languages and carefully recorded speech, limiting the reach of speech-LLM methods in low-resource settings. This paper investigates whether text-to-speech (TTS) can provide task-specific training data for Luxembourgish SQA without requiring a large human-recorded QA corpus. Starting from existing text-based QA resources, we translate questions into Luxembourgish, synthesize spoken questions with multiple TTS systems, and pair them with textual answers. We train a parameter-efficient SLAM-style architecture that connects a frozen Whisper encoder to frozen multilingual LLM backends through a learned projector and LoRA adapters. We compare MMS-TTS, Qwen3-TTS, and OmniVoice variants, including single-source corpora of about 48k questions and a 4TTS multi-source mix of approximately 230k questions. Evaluation on LLAMA-LB-Test with two real Luxembourgish speaker conditions shows that multi-source and voice-design-based synthetic training configurations yield the strongest SQA performance. The results also show that no-reference TTS quality scores do not monotonically predict downstream QA performance, indicating that synthetic speech must be evaluated as task-specific training data rather than only as natural-sounding audio.

Chinese Translation

口语问答（SQA）在很大程度上仍然集中于高资源语言和精心录制的语音，这限制了语音-大语言模型（speech-LLM）方法在低资源环境中的应用。本文探讨了文本到语音（TTS）是否可以为卢森堡语SQA提供特定任务的训练数据，而无需大量人工录制的问答语料库。我们从现有的基于文本的问答资源出发，将问题翻译成卢森堡语，使用多种TTS系统合成口语问题，并将其与文本答案配对。我们训练了一种参数高效的SLAM风格架构，该架构通过学习的投影器和LoRA适配器将冻结的Whisper编码器与冻结的多语言LLM后端连接起来。我们比较了MMS-TTS、Qwen3-TTS和OmniVoice变体，包括约48,000个问题的单源语料库和约230,000个问题的4TTS多源混合。对LLAMA-LB-Test的评估在两个真实的卢森堡语发言者条件下表明，多源和基于声音设计的合成训练配置产生了最强的SQA性能。结果还表明，无参考TTS质量评分并不单调预测下游问答性能，这表明合成语音必须作为特定任务的训练数据进行评估，而不仅仅是作为自然听起来的音频。

View on arXiv Download PDF AI Translation

cs.CL / 5 / 2607.02770

Gemma 4 Technical Report

Gemma 4 技术报告

Gemma Team, Abd, Sherif El, Aggarwal, Vaibhav, Algayres, Robin, Andreev, Alek, Bachem, Olivier, Ballantyne, Ian, Brick, Cormac, Cărbune, Victor, Casbon, Michelle, Chaturvedi, Mayank, Cotruta, Victor, Coucke, Alice, Culliton, Phil, Dadashi, Robert, Dixon, Lucas, Elhawaty, Mohamed, Evci, Utku, Farabet, Clément, Ferret, Johan, Galgani, Filippo, Girgin, Sertan, Grill, Jean-Bastien, Grootendorst, Maarten, Guo, Jiaxian, Hardin, Cassidy, He, Yanzhang, Hernandez, Steven M., Homburger, Omri, Hussenot, Léonard, Ji, Juyeong, Joulin, Armand, Kamath, Aishwarya, Kassraie, Parnian, Lacombe, Olivier, Lahoti, Preethi, Liu, Gaël, Martins, Gus, Martins, Luciano, Matejovicova, Tatiana, Merhej, Ramona, Momchev, Nikola, Mondal, Sneha, Mullins, Ryan, Panyam, Sindhu Raghuram, Pathak, Shreya, Perrin, Sarah, Pinto, André Susano, Pot, Etienne, Pouget, Angéline, Ramé, Alexandre, Ramos, Sabela, Reid, Douglas, Rim, David, Rivière, Morgane, Roth, Karsten, Rouillard, Louis, Sanseviero, Omar, Sessa, Pier Giuseppe, Settle, Shane, Sinopalnikov, Danila, Smoot, Sara, Stanczyk, Piotr, Steiner, Andreas, Stewart, Lawrence, Tolstikhin, Ilya, Tschannen, Michael, Tsitsulin, Anton, Vieillard, Nino, Wu, Renjie, Xu, Pingmei, Yang, Haichuan, Yvinec, Edouard, Zhang, Li, Zou, Joe, Aagnes, Nicolas, Abdelhamed, Abdelrahman, Agrawal, Shivani, Agrawal, Shubham, Alabdulmohsin, Ibrahim, Alayrac, Jean Baptiste, Alon, Uri, Amarnath, Chandramouli, Anand, Ankesh, Anastasiou, Chrysovalantis, Ariafar, Setareh, Aubet, François-Xavier, Axiotis, Kyriakos, Barbero, Federico, Barral, Joelle, Bendebury, Alexei, Bergmann, Urs, Bileschi, Stanley, Black, Kat, Blondel, Mathieu, Borgeaud, Sebastian, Bražinskas, Arthur, Burnell, Ryan, Busa-Fekete, Robert, Cai, Mu, Cameron, Glenn, Caucheteux, Charlotte, Chadha, Garima, Chan, Jetha, Chawla, Aditya, Chen, Blake Jianhang, Chen, Jesse, Chen, Lin, Chen, Xu, Cheng, Derek, Chien, Tzu-hsiang, Chinaev, Nikolai, Chou, Yi, Chu, Zhaohui, Coleman, Benjamin, Consul, Pooja, Conway-Rahman, Sam, Crowell, Scott, Cutler, Dylan, Dani, Vivek, Daruki, Samira, Das, Anil, Deutsch, Daniel, Dikkala, Nishanth, Ding, Li, Ding, Qiuhan, Dodhia, Shenil, Donhauser, Konstantin, Doshi, Tulsee, Dragan, Anca, Druinsky, Alex, Dua, Sahil, Egyed, Zoltan, Eisenbud, Danielle, Eppens, Daniel, Fan, Cindy, Fatemi, Bahare, Fathullah, Yassir, Feinberg, Vlad, Ferev, Milen, Fujimoto, Takumi, Galatzer-Levy, Isaac, Gante, João, Geisler, Simon, Ghosal, Soham, Girgis, Antonious M., Go, Alec, Gokhale, Alhaad, Grills, Alex, Gu, Yiming, Gupta, Pramod, Guruganesh, Guru, Hadsell, Raia, Harkous, Hamza, Harlalka, Jitendra, Hassabis, Demis, Hauth, Anja, Heyward, Joe, Hosseini, Arian, Hsia, Chih-Yang, Hsu, I-Hung, Huang, Xiaopeng, Huang, Yangsibo, Hui, Kevin, Hutter, Adrian, I, Te, Iliopoulos, Fotis, Jain, Advait, Jawahar, Ganesh, Ji, Ziwei, Jin, Qilin, Johnson, Melvin, Joshi, Kandarp, Kandoor, Arun, Kang, Wang-Cheng, Kavukcuoglu, Koray, Kazemi, Mehran, Kenealy, Kathleen, Khalifa, Amr, Kirk, Phoebe, Kothawade, Suraj, Kovalev, Vitaly, Kovelamudi, Neel, Kraft, Adam, Kumar, Ravin, Kuppam, Harish, Lannin, Justin, Lee, Chen-Yu, Lee, Seungji, Lepikhin, Dmitry, Li, Dongdong, Li, Qiujia, Liévin, Valentin, Lin, Ethan, Lin, Ziqian, Liu, Casper, Liu, Tianlin, Liu, Tianqi, Liu, Xin, Lunayach, Mayank, Ma, Min, Madan, Gagan, Maksai, Andrii, Malmi, Eric, Matuszak, Michal, McDuff, Daniel, Menghani, Gaurav, Mirylenka, Daniil, Misiunas, Karolis, Misra, Vedant, Mitran, Andreea, Mohamed, Kareem, Mukha, Maksim, Noland, Eric, O'Donnell, James, Olszewska, Kate, Orlando, Bernett, Pan, Wanqiong, Panigrahy, Rina, Parekh, Unnati, Park, Chunjong, Paskie, Eric, Peng, Liqian, Petrini, Bryce, Petrov, Slav, Pfeiffer, Jonas, Piot, Bilal, Plomecka, Martyna, Poder, Siim, Ponce, Octavio, Pramanik, Arijit, Racz, David, Rajan, Anish, Ramanovich, Michelle, Rao, Anand, Ritter, Marvin, Rodrigues, Vitor, Rosen, Evan, Rybiński, Mikołaj, Sachdeva, Noveen, Sander, Michaël E., Sathyanarayana, Rohit, Savla, Sagar, Schmidgall, Samuel, Schuster, Tal, Seguin, Benoit, Sellergren, Andrew, Severyn, Aliaksei, Shafran, Izhak, Shah, Dhruv, Shangguan, Yuan, Shenoy, Ashish, Shenoy, Pradeep, Shivanna, Rakesh, Sho, Pauline, Spangher, Lucas, Stokowiec, Wojciech, Strother, Tim, Su, Yao, Sun, Yinghao, Sundararajan, Mukund, Tacchetti, Andrea, Taege, Mor Hazan, Tafti, Pouya, Tekur, Chetan, Thapa, Rahul, Traverse, Madeleine, Treven, Lenart, Tu, Tao, Tung, Chien Te, Veličković, Petar, Venkat, Malini Pooni, Venkatesh, Sagar Gubbi, Venkiteswaran, Vidya, Visin, Francesco, Vitvitskyi, Alex, Vodrahalli, Kiran, Wang, Weiyi, Wang, Xin, Warkentin, Tris, Wassenberg, Jan, Wieting, John, Xiao, Lechao, Xu, Hao, Xu, Yuhui, Xue, Fuzhao, Yadav, Arun, Yan, Jun, Yang, Antoine, Yang, Lin, Yang, Ming-Hsuan, Ying, Ziyu, Yoo, Jae Hyeon, Zafar, Sajjad, Zhang, Fred, Zhang, Jiageng, Zhang, Jianyi, Zhang, Xiaofan, Zhao, Chao, Zhou, David, Zou, Chen

Abstract

We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute efficiency and reasoning, the Gemma 4 model suite features dense and Mixture-of-Experts architectures, ranging from 2.3B to 31B parameters. Alongside improved vision and audio encoders for all model sizes, we propose a unified, encoder-free architecture for our 12B model, which ingests raw audio and image patches. Furthermore, we integrate a thinking mode, enabling Gemma models to generate reasoning traces prior to responding. We improve inference speed, memory, and compute efficiency, as well as long-context abilities through critical design choices. Gemma 4 establishes a leap in performance across STEM, multimodal, and long-context benchmarks, and rivals larger, frontier open models in human-rated tasks.

Chinese Translation

我们介绍了 Gemma 4，这是 Gemma 模型家族中新一代的开放权重、原生多模态语言模型。Gemma 4 模型套件旨在提高计算效率和推理能力，采用了密集和专家混合（Mixture-of-Experts）架构，参数范围从 23 亿到 310 亿。除了为所有模型尺寸改进视觉和音频编码器外，我们还为我们的 120 亿模型提出了一种统一的无编码器架构，能够处理原始音频和图像块。此外，我们集成了一种思考模式，使 Gemma 模型能够在响应之前生成推理痕迹。通过关键设计选择，我们提高了推理速度、内存和计算效率，以及长上下文能力。Gemma 4 在 STEM、多模态和长上下文基准测试中实现了性能的飞跃，并在人工评分任务中与更大、前沿的开放模型相媲美。

View on arXiv Download PDF AI Translation

cs.CL / 6 / 2607.02802

Seduced by the Narrative: Assessing Rule Adherence in Semi-Open Textual Sandboxes

被叙事所诱惑：评估半开放文本沙盒中的规则遵循

Chen, Weiying, Shen, Junlong, Guo, Zhanyuan, Zhou, Xiaoou

Abstract

As LLMs are increasingly deployed as autonomous adjudicators in semi-open textual game environments, robust rule adherence becomes critical when user intent conflicts with system rules. However, these models are trained to be helpful and compliant, leaving them vulnerable to a class of attacks we term \textit{Rhetorical Injection}, where adversarial users exploit narrative framing techniques such as pseudo-logical reasoning and authoritative coercion to bypass adjudication logic. We present CoC-Seduce, a multi-agent adversarial benchmark built on Tabletop Role-Playing Game (TRPG) mechanics, an ideal instantiation of semi-open environments where rules are explicit for adjudication, yet interaction remains entirely in natural language. Three frontier models, i.e., GPT-5.4, Claude Sonnet 4.6, Gemini 3.5 Flash, serve as adversarial generators producing 5,376 samples across 4 world settings and 16 skill categories. We then benchmark 20 target adjudicators against this corpus. Evaluation across 20 models reveals that neither model scale nor explicit reasoning mechanisms reliably confer adjudication robustness, with \textsc{Pseudo-Logic} emerging as the dominant attack vector and cross-cultural settings exposing systematic knowledge gaps across all evaluated families. Project page: https://github.com/answerrtx/CoC-Seduce

Chinese Translation

随着大型语言模型（LLMs）越来越多地被部署为半开放文本游戏环境中的自主裁决者，当用户意图与系统规则发生冲突时，强有力的规则遵循变得至关重要。然而，这些模型被训练得以提供帮助和遵从，使其容易受到我们称之为 extit{修辞注入}的攻击类型的影响，在这种攻击中，对抗性用户利用叙事框架技术，如伪逻辑推理和权威性强迫，来绕过裁决逻辑。我们提出了CoC-Seduce，这是一个基于桌面角色扮演游戏（Tabletop Role-Playing Game, TRPG）机制构建的多智能体对抗基准，理想地体现了规则在裁决中明确而互动完全以自然语言进行的半开放环境。三个前沿模型，即GPT-5.4、Claude Sonnet 4.6和Gemini 3.5 Flash，作为对抗生成器，在4个世界设定和16个技能类别中生成了5,376个样本。然后，我们对20个目标裁决者在该语料库上进行基准测试。对20个模型的评估表明，无论是模型规模还是显式推理机制都无法可靠地赋予裁决的稳健性， extsc{伪逻辑}成为主要的攻击向量，而跨文化环境暴露了所有评估模型家族中的系统性知识差距。项目页面：https://github.com/answerrtx/CoC-Seduce

View on arXiv Download PDF AI Translation

cs.CL / 7 / 2607.02862

Jointly Improving Dialect Identification and ASR in Indian Languages using Multimodal Feature Fusion

通过多模态特征融合联合改善印度语言的方言识别和自动语音识别

Kumar, Saurabh, Amartyaveer, Ghosh, Prasanta Kumar

Abstract

Automatic Speech Recognition (ASR) and Dialect Identification (DID) are crucial for Indian languages, many of which are low-resource and exhibit significant dialectal differences. Existing methods often optimize ASR or DID individually, resulting in performance trade-offs. In this work, we propose a multimodal framework that jointly improves ASR and DID. Our method employs a Bottleneck Encoder to extract dialectal features from Conformer-based speech representations and a RoBERTa encoder to process ASR-generated CTC embeddings. A gating mechanism merges these features, followed by an attention encoder to refine the representations. The learned embeddings are concatenated with Conformer outputs to enhance ASR features. Evaluated on eight Indian languages with thirty-three dialects, our method achieves an average DID accuracy of 81.63% and average CER and WER of 4.65% and 17.73%, respectively. These results highlight the effectiveness of our method for joint ASR-DID modeling.

Chinese Translation

自动语音识别（ASR）和方言识别（DID）对于印度语言至关重要，其中许多语言资源匮乏且存在显著的方言差异。现有方法通常单独优化ASR或DID，导致性能之间的权衡。在本研究中，我们提出了一种多模态框架，联合改善ASR和DID。我们的方法采用瓶颈编码器从基于Conformer的语音表示中提取方言特征，并使用RoBERTa编码器处理ASR生成的CTC嵌入。一个门控机制合并这些特征，随后通过注意力编码器来细化表示。学习到的嵌入与Conformer输出连接，以增强ASR特征。在对八种印度语言及其三十三种方言进行评估时，我们的方法实现了81.63%的平均DID准确率，以及4.65%和17.73%的平均字符错误率（CER）和单词错误率（WER）。这些结果突显了我们的方法在联合ASR-DID建模中的有效性。

View on arXiv Download PDF AI Translation

cs.CL / 8 / 2607.02881

PraMem: Practice-derived Experiential Memory for Long-horizon Behavior Prediction

PraMem：基于实践的经验记忆用于长时间行为预测

Li, Zhuoqun, Cao, Boxi, Chen, Jiawei, Zhou, Hanshu, Xu, Ruoxi, Jiang, Guiping, Pan, Ruotong, Gao, Tingting, Li, Han, Wu, Xiangyu, Lin, Hongyu, Lu, Yaojie, Han, Xianpei, Sun, Le

Abstract

Long-horizon behavior prediction aims to infer a user's next action based on a lengthy historical sequence, playing a crucial role in artificial intelligence field. The rise of large language models (LLMs) offers a promising direction for sequential behavior prediction, yet LLMs struggle with latent behavioral pattern induction and model-intrinsic cognitive biases when tackling long-horizon behavior prediction. Prior memory management methods follow a context-compression paradigm that attempts to address this task by alleviating the historical sequence burden, yet fail to resolve the core challenges. In this paper, we advocate a paradigm shift that reframes the lengthy historical sequence from a burden into a valuable resource to be exploited, and accordingly propose PraMem, which conducts beforehand practice over the lengthy historical sequence to build an experiential memory, thereby serving as the assisted input for accurate long-horizon behavior prediction. Extensive experiments across diverse tasks demonstrate that PraMem achieves superior performance than prior methods, and more in-depth analyses provide valuable insights into the mechanism and evolution of the experiential memory. Code: https://github.com/icip-cas/PraMem.

Chinese Translation

长时间行为预测旨在根据较长的历史序列推断用户的下一个动作，在人工智能领域中发挥着至关重要的作用。大型语言模型（LLMs）的兴起为序列行为预测提供了一个有前景的方向，但在处理长时间行为预测时，LLMs在潜在行为模式归纳和模型内在认知偏差方面面临挑战。以往的记忆管理方法遵循上下文压缩范式，试图通过减轻历史序列的负担来解决这一任务，但未能解决核心挑战。本文倡导一种范式转变，将冗长的历史序列从负担重新构建为可利用的宝贵资源，并相应提出PraMem，该方法在冗长的历史序列上进行预先实践，以构建经验记忆，从而作为准确的长时间行为预测的辅助输入。针对多种任务的广泛实验表明，PraMem的性能优于以往方法，更深入的分析提供了对经验记忆机制和演变的宝贵见解。代码链接：https://github.com/icip-cas/PraMem。

View on arXiv Download PDF AI Translation

cs.CL / 9 / 2607.02885

Where do LLMs Fall Short in CBT-Guided Affective Reasoning?

大型语言模型在认知行为疗法引导的情感推理中存在哪些不足？

Sinha, Vaishnavi, Guttal, Pooja, Katike, Pranay Deep Reddy, Sinha, Vishal, Ndawula, Gerald, Yoon, Lira, Kleinsmith, Andrea, Gaur, Manas

Abstract

Cognitive Behavioral Therapy (CBT) provides a structured framework for understanding a user's mental state by examining the interaction between cognitive and behavioral factors. However, out-of-the-box LLMs respond fluently and empathetically, yet collapse into validation & reflection, regardless of what the user actually needs. They know theoretical CBT (scoring up to 96% accuracy on licensing exam questions) but fail to apply it effectively. We explore this gap with a knowledge-guided framework that treats CBT dialogue as controlled affective reasoning: user narratives are decomposed into Beck's Cognitive Conceptualization structure, grounded in clinical SNOMED CT concepts validated via Natural Language Inference, and a Multiple Chain-of-Thought (MCoT) strategy selection between Validation & Reflection, Socratic Questioning, or Alternative Perspectives. To measure whether such guidance actually changes behavior, we introduce the Protocol Leverage Force (F), a behavior-level metric that captures how far an intervention shifts a model away from its default response. Across three open-weight LLMs and 14 RealCBT-derived case studies, evaluated with human experts, valence-arousal trajectories, and linguistic entrainment, F shows that simply introducing protocol definitions via single chain-of-thought prompting fails to change LLM behavior, while MCoT on these definitions guides strategy selection better. Still, the effect stays within 1% (approx. 1.2-1.3%), and all models remain biased toward Validation & Reflection. These results show CBT knowledge alone does not ensure effective application, giving the affective-computing community instrumentation to measure where LLMs fall short.

Chinese Translation

认知行为疗法（CBT）提供了一个结构化框架，通过考察认知与行为因素之间的相互作用来理解用户的心理状态。然而，现成的大型语言模型（LLMs）虽然能够流利且富有同情心地回应，但却陷入验证与反思的模式，而不管用户实际需要什么。它们对理论上的CBT（在执业考试问题上准确率高达96%）有一定了解，但未能有效应用。我们通过一个知识引导框架探讨这一差距，该框架将CBT对话视为受控的情感推理：用户叙述被分解为贝克（Beck）的认知概念化结构，基于临床SNOMED CT概念，通过自然语言推理进行验证，并在验证与反思、苏格拉底式提问或替代视角之间选择多链思维（Multiple Chain-of-Thought, MCoT）策略。为了衡量这种引导是否真正改变了行为，我们引入了协议杠杆力（Protocol Leverage Force, F），这是一个行为级别的度量，捕捉干预如何将模型的响应从默认状态转变。通过对三种开放权重的LLM和14个基于RealCBT的案例研究进行评估，结合人类专家的评估、情感-唤醒轨迹和语言一致性，F显示仅通过单一链思维提示引入协议定义未能改变LLM的行为，而在这些定义上应用MCoT更能有效引导策略选择。然而，效果仍然保持在1%以内（约1.2-1.3%），所有模型仍然偏向于验证与反思。这些结果表明，仅有CBT知识并不能确保有效应用，为情感计算领域提供了测量LLM不足之处的工具。

View on arXiv Download PDF AI Translation

cs.CL / 10 / 2607.02966

Distill Where the Student Goes: Teacher-Regularized RL for English-Evidence Cross-Lingual RAG

提炼学生的去向：教师正则化的强化学习用于英语证据的跨语言检索增强生成

Zhou, Haotian, Huang, Weiran, Liu, Siqi, Wang, Xiting, Zhang, Xin, Wen, Zhihao

Abstract

Cross-lingual retrieval-augmented generation (RAG) is often deployed in an English-evidence regime, where users query in diverse languages but retrieved passages remain English. In this setting, generation can fail despite strong base models: English evidence induces language drift (English or code-switching outputs) and models use evidence unreliably when producing non-English answers. We attribute these failures to two post-training challenges: (i) errors are prefix-dependent, so fixed-trajectory supervision suffers from prefix mismatch; and (ii) sequence-level (partly discrete / judge-based) rewards yield noisy credit assignment and high-variance updates. We propose TR-RAG, a teacher-regularized RL recipe that couples reward optimization with on-policy distillation on student-visited prefixes. A compact student samples on-policy answers, while a stronger frozen teacher is queried only on those prefixes and provides a prefix-wise student-to-teacher reverse-KL anchor. We further introduce a reward decomposition for English-evidence multilingual generation, combining language consistency, character 3-gram recall, and an LLM-judge score for evidence-grounded correctness. Across three benchmarks -- BioASQ-ENKB5, Hotpot-ENKB5, and naturally multilingual MKQA -- and two backbones, TR-RAG improves the composite of language adherence and evidence-grounded correctness over strong baselines. Crucially, the teacher anchor acts as a safety net: on in-domain languages it prevents the large language-consistency collapses (up to ~27 percentage points) that reward-only RL can suffer by drifting below even the base model, while on distant out-of-distribution languages -- where reward-only RL stalls at the base model's ceiling -- it still improves evidence grounding; and on character 3-gram recall the compact student sometimes surpasses its 70B teacher.

Chinese Translation

跨语言检索增强生成（RAG）通常在英语证据的框架下进行，其中用户以多种语言查询，但检索到的段落仍然是英文。在这种情况下，尽管基础模型强大，生成仍可能失败：英语证据导致语言漂移（英语或代码切换输出），模型在生成非英语答案时不可靠地使用证据。我们将这些失败归因于两个后训练挑战：（i）错误依赖于前缀，因此固定轨迹的监督受到前缀不匹配的影响；（ii）序列级（部分离散/基于评判的）奖励导致噪声信用分配和高方差更新。我们提出了TR-RAG，一种教师正则化的强化学习方案，将奖励优化与在学生访问的前缀上的在线蒸馏结合起来。紧凑的学生在政策上采样答案，而更强大的冻结教师仅在这些前缀上进行查询，并提供前缀级的学生到教师的反向KL锚点。我们进一步引入了英语证据多语言生成的奖励分解，结合语言一致性、字符3-gram召回和基于大型语言模型的评判分数，以确保证据的正确性。在三个基准测试（BioASQ-ENKB5、Hotpot-ENKB5和自然多语言MKQA）和两个基础模型上，TR-RAG在语言遵循性和证据基础正确性方面优于强基线。重要的是，教师锚点充当安全网：在领域内语言上，它防止了奖励仅强化学习可能遭受的大幅语言一致性崩溃（高达约27个百分点），而在远离分布的语言上——奖励仅强化学习在基础模型的上限停滞——它仍然改善了证据基础；在字符3-gram召回方面，紧凑的学生有时超越其70B的教师。

View on arXiv Download PDF AI Translation

cs.CL / 11 / 2607.02980

Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling

层次稀疏注意力的正确实现：迈向无限上下文建模

Hu, Xiang, Wei, Xinyu, Gu, Hao, Zhang, Minshen, Liang, Tian, Li, Huayang, Zhu, Lei, Wang, Yan, Han, Sirui, Bai, Yushi, Tu, Kewei, Mi, Haitao, Liang, Leo

Abstract

Scaling modern large language models (LLMs) to long contexts is limited by the quadratic computation cost, and poor length extrapolation of dense attention. Chunk-wise sparse attention offers a promising alternative, but all existing methods fall short of full attention because of their inaccurate chunk selection. We propose Hierarchical Landmark Sparse (HiLS) Attention, a chunk-wise sparse attention mechanism that learns chunk selection end-to-end under the language-modeling (LM) loss. HiLS factorizes attention hierarchically: each query performs attention independently with each retrieved chunk to extract chunk-specific information, and the resulting outputs are fused according to chunk retrieval scores. By incorporating retrieval scores into the forward attention computation, HiLS optimizes them directly with the LM loss, enabling end-to-end retrieval learning and native sparse training. Experimental results show that HiLS-Attention achieves performance comparable to, and in some cases better than, full attention at in-domain context lengths. Meanwhile, HiLS-Attention extrapolates more than $64\times$ the training context length with 90% retrieval accuracy, far beyond full attention. Moreover, existing full-attention models can be converted to HiLS-Attention with lightweight continued pretraining, preserving in-domain performance while acquiring ultra-long-context extrapolation. Together with its sparse KV access and computation, HiLS-Attention breaks the usual efficiency-performance trade-off, enabling long-context LLMs that are both more efficient and more effective on general long-context tasks than their full-attention counterparts.

Chinese Translation

将现代大型语言模型（LLMs）扩展到长上下文受到二次计算成本的限制，以及密集注意力的长度外推能力不足。基于块的稀疏注意力提供了一种有前景的替代方案，但现有的所有方法由于不准确的块选择而未能实现完全注意力。我们提出了层次地标稀疏（HiLS）注意力，这是一种块级稀疏注意力机制，能够在语言建模（LM）损失下端到端学习块选择。HiLS 以层次方式因式分解注意力：每个查询独立地与每个检索到的块执行注意力，以提取块特定信息，结果输出根据块检索分数进行融合。通过将检索分数纳入前向注意力计算，HiLS 直接利用 LM 损失优化这些分数，从而实现端到端的检索学习和原生稀疏训练。实验结果表明，HiLS 注意力在领域内上下文长度下的性能可与完全注意力相媲美，在某些情况下甚至更好。同时，HiLS 注意力在 90% 的检索准确率下能够外推超过训练上下文长度的 $64 imes$，远超完全注意力。此外，现有的完全注意力模型可以通过轻量级的继续预训练转换为 HiLS 注意力，保持领域内性能的同时获得超长上下文的外推能力。结合其稀疏的 KV 访问和计算，HiLS 注意力打破了通常的效率与性能权衡，使得在一般长上下文任务中，长上下文 LLMs 在效率和有效性上均优于其完全注意力的对应物。

View on arXiv Download PDF AI Translation

cs.CL / 12 / 2607.03003

psytechlab at CLPsych 2026: Utilising Natural Language Processing methods and Large Language Models for Social Media Text Analysis

psytechlab在CLPsych 2026：利用自然语言处理方法和大型语言模型进行社交媒体文本分析

Buyanov, Igor, Valieva, Nafisa, Mazurina, Ekaterina

Abstract

Social media posts are a rich and valuable source of data for analyzing mental health states and users' well-being using automated analysis tools. In this work, we demonstrate how we used a range of Natural Language Processing (NLP) methods, including Long Short-Term Memory (LSTM), BERT-based models, and Large Language Models (LLMs), for self-state and well-being analysis and summarization during the CLPsych Shared Task 2026. Our approach achieved one of the top Consistency and Contradiction scores for the summarization task and also middle-level results for the other tasks. By testing and developing such mental health-state estimation systems, we contributed to improving mental health support systems. We make our code available https://github.com/psytechlab/CLPsych2026/.

Chinese Translation

社交媒体帖子是分析心理健康状态和用户福祉的重要且宝贵的数据来源，能够通过自动化分析工具进行分析。在本研究中，我们展示了如何使用一系列自然语言处理（NLP）方法，包括长短期记忆（LSTM）、基于BERT的模型和大型语言模型（LLMs），进行自我状态和福祉的分析与总结，参与了CLPsych共享任务2026。我们的方法在总结任务中取得了最佳一致性和矛盾得分之一，并在其他任务中也取得了中等水平的结果。通过测试和开发这样的心理健康状态评估系统，我们为改善心理健康支持系统做出了贡献。我们的代码可在此获取：https://github.com/psytechlab/CLPsych2026/

View on arXiv Download PDF AI Translation

cs.CL / 13 / 2607.03049

Alignment-Guided Largest Table Overlap Size Estimation

基于对齐指导的最大表重叠大小估计

Lee, Ge, Huang, Shixun, Bao, Zhifeng, Sadiq, Shazia, Zhao, Yanchang

Abstract

Fast estimation of the size of the largest overlap between tables enables blocking and query-by-table retrieval in large table repositories. The first and the state-of-the-art estimator Armadillo improves efficiency by embedding each table independently and approximating overlap ratio via embedding similarity. However, accurate estimation in heterogeneous repositories remains limited by three challenges: (C1) overlap depends on row-column structure, i.e., each matched cell must preserve both its row and column membership under a joint alignment of the two tables, but existing encodings leave this structure to be inferred indirectly; (C2) independent encoding provides no explicit channel for inter-table alignment signals, biasing prediction toward global similarity; (C3) naive value encodings overfit to corpus-specific distributions, causing cross-domain degradation. Hence, we propose ALORE, a scalable and domain-robust overlap ratio estimator built on three principles: (P1) explicitly represent row-column structure; (P2) expose inter-table alignment signals during training without expensive alignment search; (P3) reduce sensitivity to corpus-specific value distributions. ALORE instantiates these principles with a Two-View Row-Column Hypergraph encoder, alignment-guided objectives with inexpensive interaction signals, and a domain-robust value mapping. Experiments on multiple datasets spanning diverse domains and scales, including a large real-world corpus beyond prior benchmarks, show that ALORE outperforms the state of the art. ALORE reduces MAE by up to 55% overall and 69% in zero-shot transfer, while achieving up to 89x speedup. We further validate its effectiveness for query-by-table retrieval.

Chinese Translation

快速估计表之间最大重叠的大小能够在大型表格库中实现阻塞和按表查询检索。首个最先进的估计器 Armadillo 通过独立嵌入每个表并通过嵌入相似性近似重叠比率来提高效率。然而，在异构库中，准确估计仍面临三大挑战：(C1) 重叠依赖于行列结构，即每个匹配的单元格必须在两个表的联合对齐下保持其行和列的成员资格，但现有编码使得这一结构需要间接推断；(C2) 独立编码没有提供明确的跨表对齐信号通道，导致预测偏向全局相似性；(C3) 天真的值编码过拟合于特定语料分布，造成跨领域性能下降。因此，我们提出了 ALORE，这是一个可扩展且具有领域鲁棒性的重叠比率估计器，基于三个原则构建：(P1) 明确表示行列结构；(P2) 在训练过程中暴露跨表对齐信号，而无需昂贵的对齐搜索；(P3) 降低对特定语料值分布的敏感性。ALORE 通过一个双视图行列超图编码器、具有低成本交互信号的对齐指导目标以及一个领域鲁棒的值映射来实现这些原则。在多个涵盖不同领域和规模的数据集上的实验，包括一个超越先前基准的大型真实语料，显示 ALORE 超越了现有技术。ALORE 在整体上将平均绝对误差 (MAE) 降低了多达 55%，在零样本迁移中降低了 69%，同时实现了高达 89 倍的加速。我们进一步验证了其在按表查询检索中的有效性。

View on arXiv Download PDF AI Translation

cs.CL / 14 / 2607.03093

Don't Wait to Reply: Towards Responsive yet Thoughtful Dialogue through Proactive Thinking

不要等待回复：通过主动思维实现响应性与深思熟虑的对话

Wang, Ante, Fu, Jiaqi, Chen, Xuanyi, Ma, Ruotian, Tu, Zhaopeng, Ma, Weizhi, Liu, Yang

Abstract

Thinking has emerged as a critical capability for Large Language Models (LLMs) tackling complex tasks. However, its reactive nature, where reasoning is passively triggered only upon receiving a user response, inevitably introduces latency that compromises conversational fluidity. This stands in sharp contrast to human dialogue, where speakers proactively anticipate and plan future content during natural pauses to ensure seamless interaction. To bridge this gap, we propose Proactive Thinking, a framework that empowers models to pre-compute potential response elements during conversational downtime instead of waiting idly for the next input. We then introduce a training-free baseline that can think ahead by anticipating future states, balancing efficiency and quality through speculative continual thinking. To evaluate this approach in practice, we adapt three benchmarks of varying complexity into time-aware environments that simulate real-time conversational flow. We demonstrate that proactive thinking effectively improves interaction efficiency without compromising performance. Ultimately, this work advocates for a fundamental shift toward more intelligent, anticipatory, and real-time conversational AI.

Chinese Translation

思维已成为大型语言模型（LLMs）应对复杂任务的重要能力。然而，其反应性特征使得推理仅在接收到用户响应时被动触发，这不可避免地引入了延迟，从而影响对话的流畅性。这与人类对话形成鲜明对比，人类在自然停顿期间主动预判和规划未来内容，以确保无缝互动。为了解决这一问题，我们提出了主动思维（Proactive Thinking）框架，使模型能够在对话空闲期间预先计算潜在的响应元素，而不是无所事事地等待下一个输入。接着，我们引入了一种无训练的基线方法，通过预见未来状态来实现前瞻性思维，在效率和质量之间实现平衡。为了在实践中评估这种方法，我们将三种不同复杂度的基准适配为时间感知环境，以模拟实时对话流。我们证明了主动思维有效地提高了互动效率，而不影响性能。最终，这项工作倡导向更智能、具有预见性和实时性的对话人工智能的根本转变。

View on arXiv Download PDF AI Translation

cs.CL / 15 / 2607.03154

Conditional Diffusion Guided Knowledge Transfer for Multi-Domain Knowledge Graph Completion

基于条件扩散引导的多领域知识图谱补全知识转移

Sheng, Jiawei, Su, Taoyu, Lin, Xixun, Li, Xiaodong, Liu, Tingwen

Abstract

Multi-domain knowledge graph completion (MKGC) aims to improve missing triple prediction in a target KG by transferring knowledge from other support KGs. Existing methods typically enforce consistency constraints on equivalent entities across KGs to transfer knowledge, which risks suppressing domain-specific contextual information of entities. This design can also compromise entity representation information from all KG domains, impeding performance improvements, especially in low-resource data scenarios. To address this, we pioneer a generation-based paradigm for MKGC and propose DMKGC, a conditional diffusion-guided knowledge transfer framework. Our key insight is to treat each KG as a partial view of the entity entire information, and generate informative domain-general entity embeddings through diffusion models conditioned on support KGs. Particularly, we first initialize domain-agnostic entity embeddings as prior entity embeddings, and then encode them within individual KGs. Afterward, we fuse equivalent entities from support KGs as the conditional diffusion generation guidance. We leverage the prior entity embeddings as the proxy generation objective, which ensures this conditional generation to be unbiased towards any conditioned KGs. Simultaneously, we also train the generated embeddings to be predictive across KGs, thus preserving domain-specific information. Extensive experiments on 14 KGs in 3 benchmarks demonstrate a 4.3\% average MRR improvement in tail entity prediction over state-of-the-art methods, with sustained gains in low-resource data settings.

Chinese Translation

多领域知识图谱补全（MKGC）旨在通过从其他支持知识图谱（KG）转移知识来改善目标知识图谱中的缺失三元组预测。现有方法通常在知识图谱之间对等实体施加一致性约束以转移知识，这可能会抑制实体的领域特定上下文信息。这种设计也可能妨碍来自所有知识图谱领域的实体表示信息，从而阻碍性能提升，尤其是在低资源数据场景中。为了解决这个问题，我们开创了一种基于生成的MKGC范式，并提出了DMKGC，一个条件扩散引导的知识转移框架。我们的关键见解是将每个知识图谱视为实体全部信息的部分视图，并通过以支持知识图谱为条件的扩散模型生成信息丰富的领域通用实体嵌入。具体而言，我们首先将领域无关的实体嵌入初始化为先验实体嵌入，然后在各个知识图谱中对其进行编码。随后，我们融合来自支持知识图谱的等价实体作为条件扩散生成指导。我们利用先验实体嵌入作为代理生成目标，确保这种条件生成不偏向任何条件知识图谱。同时，我们还训练生成的嵌入在知识图谱之间具有预测能力，从而保留领域特定信息。在3个基准上的14个知识图谱的广泛实验表明，与最先进的方法相比，在尾实体预测中平均提高了4.3%的MRR，并在低资源数据设置中保持了持续的增益。

View on arXiv Download PDF AI Translation

cs.CL / 16 / 2607.03160

The Role of Prompt Language and Translation-Theory-Driven Prompts in Large Language Models: A Case Study on Spanish-Chinese Journalistic Translation

提示语言和翻译理论驱动提示在大型语言模型中的作用：以西班牙语-中文新闻翻译为例

Lai, Haohong, Li, Weijia

Abstract

This study examines how prompt language and translation theory-driven prompt design influence the quality of Spanish-Chinese journalistic translations generated by GPT-5.2. A parallel corpus of four editorials from El Pais was translated under 48 experimental conditions (4 prompt types, 3 prompt languages, and 4 articles). Translation quality was assessed using BLEU and BERTScore-F1 for automated evaluation, alongside human evaluation based on the Multidimensional Quality Metrics (MQM) framework. Automated metrics identified the baseline prompt (BASE) as the best-performing condition, whereas human evaluation ranked the brief-oriented prompt (BRIEF) highest (MQM: 8.66 vs. 7.84), a reversal likely attributable to the single-reference constraint inherent in automated measures. Sub-error type analysis revealed that translation theory-driven prompts selectively reduced Awkward style errors, while Unidiomatic style errors persisted across conditions. Prompt language had a negligible impact under both evaluation paradigms. These results indicate that translation theory-driven prompts can yield measurable quality gains under expert evaluation of journalistic translations, although their pedagogical implications for language learners remain suggestive and require validation through user-based studies.

Chinese Translation

本研究探讨了提示语言和翻译理论驱动的提示设计如何影响GPT-5.2生成的西班牙语-中文新闻翻译的质量。我们在48种实验条件下（4种提示类型、3种提示语言和4篇文章）对《国家报》（El Pais）的四篇社论进行了平行语料库翻译。翻译质量通过BLEU和BERTScore-F1进行自动评估，同时基于多维质量指标（MQM）框架进行人工评估。自动化指标识别基线提示（BASE）为表现最佳的条件，而人工评估则将以简洁为导向的提示（BRIEF）评为最高（MQM: 8.66 vs. 7.84），这种反转可能归因于自动化测量中固有的单一参考约束。子错误类型分析显示，翻译理论驱动的提示选择性地减少了尴尬风格错误，而不地道风格错误在各条件下仍然存在。提示语言在两种评估范式下的影响微乎其微。这些结果表明，翻译理论驱动的提示在专家评估新闻翻译时可以带来可测量的质量提升，尽管它们对语言学习者的教学意义仍需通过用户基础研究进行验证。

View on arXiv Download PDF AI Translation

cs.CL / 17 / 2607.03166

KARMA: Knowledge graph-based Automated Reasoning Materialization and Alignment

KARMA：基于知识图谱的自动推理材料化与对齐

Choi, Jinkyeong, Jeong, Chaebin, Park, Donghyeon

Abstract

Template-based contrastive synthesis is scalable, but its candidates often differ only in a few entity-slots while sequence-level optimization spreads supervision over mostly shared templates. We formalize this as the Resolution Mismatch Problem and propose KARMA, which enumerates schema-constrained paths over domain knowledge graphs and verbalizes them into slot-aligned contrastive candidates. Slot-Parallel Alignment (SPA) then applies a decoupled slot-level objective to route preference supervision to discriminative entity-slots, with slot-aware masked attention serving as an optional packed-evaluation implementation. Across biomedical, computer-science, and chemistry benchmarks, KARMA outperforms base LLM and same-data SFT baselines, and compares favorably with sequence and token-level preference methods.

Chinese Translation

基于模板的对比合成具有可扩展性，但其候选项往往仅在少数实体槽位上有所不同，而序列级优化则将监督分散到大多数共享模板上。我们将此形式化为分辨率不匹配问题，并提出KARMA，该方法在领域知识图谱上枚举受模式约束的路径，并将其转化为槽位对齐的对比候选项。槽位并行对齐（Slot-Parallel Alignment, SPA）随后应用解耦的槽位级目标，将偏好监督引导到具有区分性的实体槽位，槽位感知的掩码注意力作为可选的打包评估实现。在生物医学、计算机科学和化学基准测试中，KARMA的表现优于基础大语言模型（base LLM）和相同数据的微调（SFT）基线，并与序列和标记级偏好方法相比表现良好。

View on arXiv Download PDF AI Translation

cs.CL / 18 / 2607.03207

S-DiverSe: Spanish Diverse Speech

S-DiverSe：西班牙多样化语音

López, Fernando, Ibañez, Fernando, Martínez, Ana, Alonso, Iván, Gómez, Pablo, Kesiraju, Santosh, Luque, Jordi

Abstract

Automatic speech recognition (ASR) has advanced remarkably for standard speech, yet speech affected by neurological conditions remains a challenge. We present S-DiverSe (Spanish Diverse Speech), a corpus of 3.2 hours of in-the-wild Spanish speech from 22 speakers with amyotrophic lateral sclerosis, Parkinson's disease, and stroke. The dataset contains 444 manually transcribed audio segments with metadata on speaker sex, disease type, and intelligibility. S-DiverSe is designed to support ASR evaluation and development for neurologically affected Spanish speech. We describe the dataset, analyze its composition, and report baseline ASR results alongside initial adaptation experiments. Our findings reveal that heuristic text post-processing is more robust than fine-tuning for out-of-domain neurological Spanish speech. This underscores the need for dedicated in-the-wild Spanish benchmarks.

Chinese Translation

自动语音识别（ASR）在标准语音方面取得了显著进展，但受神经系统疾病影响的语音仍然是一个挑战。我们提出了 S-DiverSe（西班牙多样化语音），这是一个包含来自22位患有肌萎缩侧索硬化症、帕金森病和中风的讲者的3.2小时自然环境下西班牙语语音的语料库。该数据集包含444个手动转录的音频片段，并附有讲者性别、疾病类型和可懂度的元数据。S-DiverSe旨在支持受神经系统影响的西班牙语语音的ASR评估和开发。我们描述了该数据集，分析了其组成，并报告了基线ASR结果以及初步适应实验。我们的研究结果表明，对于领域外的神经系统西班牙语语音，启发式文本后处理比微调更为稳健。这强调了建立专门的自然环境下西班牙语基准的必要性。

View on arXiv Download PDF AI Translation

cs.CL / 19 / 2607.03236

TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding

TACG：用于扩散语言模型解码的轨迹感知提交门控

Wang, Chengcheng, Luo, Tingzhang, Li, Wenhao, Guo, Jianyuan, Xu, Chang

Abstract

Diffusion language models (DLLMs) generate text by iteratively denoising masked positions, exposing a trajectory of predictive distributions rather than a single instantaneous belief. Most existing decoders ignore this trajectory and commit tokens from the current snapshot alone, conflating confidence with commitment readiness: a transient top-1 peak under incomplete context can be locked in, while candidates with consistent cross-step support are delayed. We propose Trajectory-Aware Commit Gating (TACG), a training-free gate-level decoder that anchors token identities to the base posterior and uses trajectory-aware signals only to decide whether the current proposal is ready to commit. TACG combines Temporal Implicit Logits Guidance (TILG), which keeps an exponential moving average of past logits as a self-reference and contrasts the current logits against this reference in natural-parameter space, with a History Gate (HG) that enforces short-term proposal persistence before commitment. Together with a capped extra-promotion budget, these components yield a stability-constrained commit rule without auxiliary networks or extra forward passes. We evaluate TACG on LLaDA, Dream, and LLaDA2-Mini across code (HumanEval, MBPP) and math (GSM8K, MATH500) benchmarks; it typically improves or preserves accuracy while reducing denoising steps and increasing tokens per forward (TPF). The code is publicly available at https://github.com/Clarence-CV/TACG-DLLM.

Chinese Translation

扩散语言模型（DLLMs）通过迭代去噪掩蔽位置生成文本，暴露出预测分布的轨迹，而不是单一的瞬时信念。大多数现有解码器忽略这一轨迹，仅从当前快照中提交标记，将信心与提交准备混为一谈：在不完整上下文下，瞬时的 top-1 峰值可能被锁定，而具有一致跨步支持的候选项则被延迟。我们提出了轨迹感知提交门控（TACG），这是一种无训练的门级解码器，将标记身份锚定到基础后验，并仅使用轨迹感知信号来决定当前提议是否准备好提交。TACG 结合了时间隐式逻辑引导（Temporal Implicit Logits Guidance, TILG），该方法保持过去逻辑的指数移动平均作为自我参考，并在自然参数空间中将当前逻辑与此参考进行对比，以及历史门（History Gate, HG），该门在提交之前强制执行短期提议的持续性。结合一个上限额外推广预算，这些组件产生了一种稳定性约束的提交规则，无需辅助网络或额外的前向传递。我们在 LLaDA、Dream 和 LLaDA2-Mini 上评估 TACG，涵盖代码（HumanEval, MBPP）和数学（GSM8K, MATH500）基准；它通常提高或保持准确性，同时减少去噪步骤并增加每次前向传递的标记数（TPF）。代码已公开发布在 https://github.com/Clarence-CV/TACG-DLLM。

View on arXiv Download PDF AI Translation

cs.CL / 20 / 2607.03323

From Gentlemen to Frontiermen: Masculine Formations in English-Language Fiction (1771--1930)

从绅士到边疆人：英语文学中的男性形成（1771-1930）

Wang, Rong

Abstract

Masculinity in nineteenth-century fiction is not a single ideal but a field of competing scripts. Drawing on 150 British and American canonical novels from the txtLAB Novel450 corpus, published between 1771 and 1930, this paper examines the changing relative prominence of competing models of masculine authority. To focus the analysis on masculine characterisation, the study extracts male-character-centred text windows by using coreference resolution to group names, nominal mentions, and pronouns into character-specific reference chains. It then fits an unsupervised structural topic model with publication year and author gender as topic-prevalence covariates. The model identifies six distinct masculine formations: aristocratic-chivalric, Christian manhood, gentlemanly respectability, country squire, professional-commercial, and imperial/adventure. Across the corpus, formations tied to inherited rank and sacred authority decline, while those organised around paid work and adventure rise. The largest increase occurs not in professional-commercial breadwinning but in imperial/adventure masculinity, particularly the frontier-wilderness register. The trajectory points to a reallocation from inherited and sacred status towards achieved, commercial, and expansionary forms of masculine authority. Adventurous and commercial formations are also more prevalent in novels by authors recorded as male. Because these formations emerge without a seeded vocabulary yet align with categories established in independent scholarship, the article offers a reproducible method for measuring the reorganisation of gendered authority across the long nineteenth century.

Chinese Translation

十九世纪小说中的男性气质并非单一理想，而是一个竞争剧本的领域。本文基于1771年至1930年间出版的150部英国和美国经典小说（来自txtLAB Novel450语料库），考察了男性权威竞争模型相对重要性的变化。为了聚焦于男性角色的刻画，研究通过共指消解技术提取以男性角色为中心的文本窗口，将姓名、名词提及和代词归类为特定角色的参考链。然后，研究使用无监督结构主题模型，以出版年份和作者性别作为主题流行度的协变量进行拟合。该模型识别出六种不同的男性形成：贵族骑士型、基督教男性气质、绅士体面、乡村绅士、职业商业型和帝国/冒险型。在整个语料库中，与继承等级和神圣权威相关的形成逐渐减少，而围绕有偿工作和冒险组织的形成则有所上升。最大的增长并不在于职业商业型的养家糊口，而是在于帝国/冒险男性气质，特别是边疆-荒野的表现。该轨迹指向从继承和神圣地位向成就、商业和扩张性男性权威形式的重新分配。冒险和商业形成在被记录为男性的作者的小说中也更为普遍。由于这些形成在没有预设词汇的情况下出现，但与独立学术研究中建立的类别相一致，本文提供了一种可重复的方法，用于衡量整个十九世纪长时期内性别权威的重组。

View on arXiv Download PDF AI Translation

cs.CL / 21 / 2607.03325

From Judgments to Issues: Structured Extraction of Legal Reasoning with Citation-Hallucination Control

从判决到问题：带有引用幻觉控制的法律推理结构化提取

Piccioli, Giovanni, Fidelangeli, Alessia, Santin, Piera, Vivo, Pierpaolo

Abstract

We present an automated pipeline that decomposes Italian tax-court judgments into individual legal issues and extracts, for each issue, a structured XML representation grounded in the IRAC framework and the legal syllogism. The pipeline targets a corpus of approximately $330{,}000$ first- and second-instance decisions of the Italian tax courts and is built around a capable yet cost-efficient general-purpose model (DeepSeek V3), a choice driven by the need to process several hundred thousand documents at a sustainable cost. To address the well-documented unreliability of large language models on legal citations, we couple the extraction step with an automatic hallucination-detection filter that compares the references produced by the model with those identified in the judgment text by a dedicated parser (Linkoln), normalised to standard identifiers (URN-NIR, ECLI, CELEX). We validate the pipeline on $50$ judgments annotated by two PhDs in tax law, computing inter-annotator agreement and LLM-vs-expert agreement on both issue extraction and legal citations, together with a stand-alone evaluation of the hallucination filter. To the best of our knowledge, this is the first issue-level, expert-validated structured extraction pipeline with hallucination control for Italian tax-court decisions, and it provides a concrete starting point for downstream applications such as issue-level retrieval, citation-network analysis, and the construction of large-scale datasets of legal reasoning.

Chinese Translation

我们提出了一种自动化流程，将意大利税务法院的判决分解为单独的法律问题，并为每个问题提取基于IRAC框架和法律三段论的结构化XML表示。该流程针对约330,000份意大利税务法院的一审和二审裁决，构建在一个高效且具有成本效益的通用模型（DeepSeek V3）之上，这一选择是基于处理数十万份文件的可持续成本需求。为了应对大型语言模型在法律引用方面的已知不可靠性，我们将提取步骤与一个自动幻觉检测过滤器相结合，该过滤器将模型生成的引用与专用解析器（Linkoln）在判决文本中识别的引用进行比较，并标准化为标准标识符（URN-NIR, ECLI, CELEX）。我们在50份由两位税法博士标注的判决上验证了该流程，计算了标注者间一致性以及大型语言模型与专家在问题提取和法律引用方面的一致性，同时对幻觉过滤器进行了独立评估。根据我们所知，这是首个针对意大利税务法院判决的、经过专家验证的、具备幻觉控制的结构化问题级提取流程，为后续应用如问题级检索、引用网络分析以及大规模法律推理数据集的构建提供了一个具体的起点。

View on arXiv Download PDF AI Translation

cs.CL / 22 / 2607.03346

Efficient Decentralized Multi-task Dataset Valuation via Model Merging

通过模型合并实现高效的去中心化多任务数据集评估

Alipour, Mohammadsajad, Amiri, Mohammad Mohammadi

Abstract

Accurate and efficient dataset valuation is essential for enabling fair and transparent data marketplaces, especially when multiple contributors provide data for training multi-task models. Most existing valuation methods, however, are limited to single-task settings, overlooking scenarios where a buyer aims to optimize performance across multiple downstream tasks. Moreover, traditional valuation approaches, such as Shapley-based or retraining-based methods, are computationally expensive and poorly suited for decentralized environments without a trusted central coordinator and with strict privacy constraints. We propose DMVM (Decentralized Multi-task Valuation via Model Merging), a novel framework that bypasses retraining and data sharing by leveraging task arithmetic to infer dataset contributions directly from model combinations. Instead of retraining or sharing raw data, DMVM quantifies how models trained on different datasets combine in parameter space to infer each dataset's marginal utility across multiple tasks. This formulation yields a valuation process that is scalable, computationally efficient, and explicitly aligned with multi-task generalization behavior. To support decentralized deployment, we introduce a secure aggregation protocol that enables collaborative valuation without revealing individual model parameters or private data. We also provide theoretical error bounds characterizing the approximation quality of DMVM and validate our framework through comprehensive experiments on computer vision and natural language processing tasks.

Chinese Translation

准确且高效的数据集评估对于实现公平透明的数据市场至关重要，尤其是在多个贡献者提供数据以训练多任务模型的情况下。然而，现有的大多数评估方法仅限于单任务设置，忽视了买方希望在多个下游任务中优化性能的场景。此外，传统的评估方法，如基于夏普利值（Shapley-based）或重训练（retraining-based）的方法，计算开销大，不适合没有可信中心协调者且具有严格隐私约束的去中心化环境。我们提出了DMVM（通过模型合并的去中心化多任务评估），这是一个新颖的框架，通过利用任务算术直接从模型组合推断数据集贡献，从而绕过重训练和数据共享。DMVM量化了在参数空间中，基于不同数据集训练的模型如何结合，以推断每个数据集在多个任务中的边际效用，而不是重训练或共享原始数据。该公式化产生了一种可扩展、计算高效且明确与多任务泛化行为对齐的评估过程。为了支持去中心化部署，我们引入了一种安全聚合协议，使得在不泄露单个模型参数或私人数据的情况下进行协作评估成为可能。我们还提供了理论误差界限，以表征DMVM的近似质量，并通过在计算机视觉和自然语言处理任务上的全面实验验证了我们的框架。

View on arXiv Download PDF AI Translation

cs.CL / 23 / 2607.03377

Spectral Signatures of Large Language Models

大型语言模型的谱特征

Zhang, Zhuoying, Prasad, Ishan V., Hu, Yuanzhe, Liu, Zihang, Luo, Hengrui, Ren, Pu, Yang, Yaoqing

Abstract

The rapidly growing repository of publicly available large language models (LLMs) presents significant challenges for systematic management and quantification at scale, such as model lineage tracing, licensing, and evaluation. However, task-specific benchmarks are insufficient for this setting, as LLMs differ widely in architectures, scales, and training procedures. To address this challenge, we adopt spectral shape-based metrics for managing and quantifying LLMs based on Heavy-Tailed Self-Regularization theory. Our approach uses the shape information of the weight empirical spectral density as a compact spectral signature of each model. This signature captures intrinsic properties of pretrained models and remains robust during post-training, making it suitable for model-level analysis. In addition, this metric is data-free, computationally-efficient, and scale-invariant, enabling large-scale analysis in practice. Moreover, we curate a large and diverse model corpus consisting of major open-source LLM families, and use it to systematically benchmark spectral and non-spectral metrics across models and downstream tasks. We show that our spectral signature supports the tracking of the model lineage, the unsupervised clustering of similar models, and the quantification of the model performance. Overall, the proposed spectral signature provides a meaningful proxy for broad performance trends across LLMs, enabling efficient organization, comparison, and analysis of large model collections.

Chinese Translation

快速增长的公共可用大型语言模型（LLMs）库在系统管理和大规模量化方面带来了重大挑战，例如模型谱系追踪、许可和评估。然而，特定任务的基准测试在这种情况下是不够的，因为LLMs在架构、规模和训练程序上差异很大。为了解决这一挑战，我们采用基于谱形状的度量方法，利用重尾自我正则化理论来管理和量化LLMs。我们的方法使用权重经验谱密度的形状信息作为每个模型的紧凑谱特征。该特征捕捉了预训练模型的内在属性，并在后训练过程中保持稳健，适合于模型级分析。此外，该度量方法是无数据的、计算高效的，并且具有尺度不变性，使其在实践中能够进行大规模分析。此外，我们整理了一个大型且多样化的模型语料库，包含主要的开源LLM家族，并利用它系统地基准测试模型和下游任务中的谱和非谱度量。我们展示了我们的谱特征支持模型谱系的追踪、相似模型的无监督聚类以及模型性能的量化。总体而言，所提出的谱特征为LLMs的广泛性能趋势提供了有意义的代理，使大型模型集合的高效组织、比较和分析成为可能。

View on arXiv Download PDF AI Translation

cs.CL / 24 / 2607.03414

The Classics at SemEval-2026 Task 3: Combining Transformer Models and LLM-Generated Annotations for Dimensional Aspect-Based Sentiment Analysis

SemEval-2026任务3中的经典：结合Transformer模型和LLM生成的注释进行维度基础情感分析

Alshawi, Rafif, Raj, Amit, Kudelya, Aleksey, Shirnin, Alexander

Abstract

This paper presents an approach to the SemEval-2026 Task 3: Dimensional Aspect-Based Sentiment Analysis. We investigate methods for moving beyond traditional categorical sentiment (e.g., positive or negative) to predict fine-grained, real-valued scores for sentiment "valence" (positivity) and "arousal" (intensity). We participate in two subtasks: predicting these scores for given aspects (Subtask 1) and extracting full sets of sentiment details, including aspects, categories, and opinions alongside their scores (Subtask 3). Our approach for the regression task involves a weighted ensemble of transformer-based encoder models. For the Russian language, we further enhance the input by using a large language model (LLM) to generate synthetic sentiment descriptions. For the extraction task, we fine-tune a decoder LLM to perform structured prediction, allowing the system to identify sentiment elements and estimate their numerical scores simultaneously.

Chinese Translation

本文提出了一种针对SemEval-2026任务3：维度基础情感分析的方法。我们探讨了超越传统分类情感（例如，积极或消极）的方法，以预测情感“价值”（积极性）和“唤醒”（强度）的细粒度实值分数。我们参与了两个子任务：针对给定方面预测这些分数（子任务1）和提取完整的情感细节集，包括方面、类别和意见及其分数（子任务3）。我们在回归任务中的方法涉及基于Transformer的编码器模型的加权集成。对于俄语，我们进一步通过使用大型语言模型（LLM）生成合成情感描述来增强输入。对于提取任务，我们微调解码器LLM以执行结构化预测，使系统能够同时识别情感元素并估计其数值分数。

View on arXiv Download PDF AI Translation

cs.CL / 25 / 2607.03466

CaresAI at SMM4H-HeaRD 2026: Predicting TNM Staging

CaresAI在SMM4H-HeaRD 2026：预测TNM分期

Abubakar, Joseph Itopa, Jarme, Jorge, Igwezeke, Favour, Adewunmi, Mary

Abstract

This study aims to predict Tumor, Node, and Metastasis (TNM) stage labels independently, with the Cancer Genome Atlas (TCGA) pathology report as the sixth shared task of SMM4H-HeaRD 2026. The problem is framed as three multi-label classification tasks. We explore both classical and deep learning approaches using Term Frequency-Inverse Document Frequency (TF-IDF) features and embeddings from ClinicalBERT, BioBERT, and PubMedBERT. These representations are used with Logistic Regression (LR), Light Gradient Boosting Machine (LightGBM), Feed-Forward Neural Networks (FFNN), and Wide Residual Networks (WRN). Our results show that individual embeddings perform similarly to the TNM label classification, while their combination improves its predictive ability. WRN achieves AUROC scores of 0.839 (T), 0.8502 (N), and 0.803 (M) with F1-scores of 0.622, 0.702, and 0.9337, respectively, for the training phase. LightGBM with TF-IDF performs best with AUROC scores of 0.9368 (T), 0.9524 (N), and 0.8311 (M) and F1-scores of 0.7559 (T), 0.7384 (N), and 0.7017 (M) during the training phase. Furthermore, the result of the Codabench for the test sets indicates a Macro-F1 score of 0.978, 0.957, and 0.879 for the T, N, and M categories respectively for test set 1; while test set 2 records a Macro-F1 score for T, N, and M is 0.807, 0.767, 1.0 respectively. However, performance declined during the evaluation phase of the test sets, a drop from 0.938 to 0.858 of test set 1 to 2, for the Macro-F1 score across all stages; suggesting limitations in model generalizability, sensitivity to class imbalance, and challenges in processing lengthy clinical documents. Although this study provides an efficient baseline model and a reproducible pipeline, further optimization and validation are required before it can be considered suitable for use in a real-world clinical setting.

Chinese Translation

本研究旨在独立预测肿瘤、淋巴结和转移（TNM）分期标签，以癌症基因组图谱（TCGA）病理报告作为SMM4H-HeaRD 2026的第六个共享任务。该问题被框定为三个多标签分类任务。我们探索了使用词频-逆文档频率（TF-IDF）特征和来自ClinicalBERT、BioBERT和PubMedBERT的嵌入的经典和深度学习方法。这些表示与逻辑回归（Logistic Regression, LR）、轻量级梯度提升机（Light Gradient Boosting Machine, LightGBM）、前馈神经网络（Feed-Forward Neural Networks, FFNN）和宽残差网络（Wide Residual Networks, WRN）结合使用。我们的结果显示，单独的嵌入在TNM标签分类中表现相似，而它们的组合提高了预测能力。WRN在训练阶段的AUROC得分为0.839（T）、0.8502（N）和0.803（M），F1得分分别为0.622、0.702和0.9337。使用TF-IDF的LightGBM表现最佳，AUROC得分为0.9368（T）、0.9524（N）和0.8311（M），F1得分为0.7559（T）、0.7384（N）和0.7017（M）。此外，Codabench对测试集的结果显示，测试集1的T、N和M类别的Macro-F1得分分别为0.978、0.957和0.879；而测试集2的T、N和M的Macro-F1得分分别为0.807、0.767和1.0。然而，在测试集的评估阶段，性能下降，测试集1到测试集2的Macro-F1得分从0.938降至0.858，表明模型的泛化能力有限，对类别不平衡的敏感性以及处理冗长临床文档的挑战。尽管本研究提供了一个高效的基线模型和可重复的流程，但在其被认为适合在实际临床环境中使用之前，仍需进一步优化和验证。

View on arXiv Download PDF AI Translation

cs.CL / 26 / 2607.03481

Learning from Lost Provenance: Multiple Instance Learning for Cancer Registry Tumor Group Classification

从失去的来源中学习：用于癌症登记肿瘤组分类的多实例学习

Ruocco, Leonard, Simkin, Jonathan, Gondora, Lovedeep, Arbour, Gregory, Ng, Raymond

Abstract

Modernizing cancer registries with deep learning is opening new opportunities to automate labor-intensive tasks such as the coding of pathology reports. However, progress is constrained by the scarcity of report-level human-annotated training data. Cancer registries generate substantial volumes of expert-assigned labels as a routine product of their operations, but these exist at the patient level and are not linked to the individual pathology reports that informed them, limiting their direct use for training models. We develop an efficient framework for training deep learning classifiers by leveraging these operationally-generated labels without requiring per-report human annotation, demonstrated for tumor group classification at the BC Cancer Registry. We use Attention-Based Multiple Instance Learning (ABMIL) to recover the lost link between patient-level labels and the reports that informed them, leveraging the attention the model places on each report to distil a large, noisily-labeled corpus into a compact, high-quality per-report training dataset. A classifier fine-tuned on a distilled dataset achieved a macro F1 of 0.83, outperforming established baselines across most tumor groups. By turning routine operational labels into high-quality training data without additional annotation or large-scale computing infrastructure, ABMIL offers a practical and accessible route to automating cancer registry workflows.

Chinese Translation

利用深度学习现代化癌症登记系统为自动化劳动密集型任务（如病理报告编码）开辟了新的机会。然而，进展受到报告级人类标注训练数据稀缺的限制。癌症登记系统在其运营过程中生成大量专家分配的标签，但这些标签仅存在于患者级别，并未与提供信息的个体病理报告关联，从而限制了它们在模型训练中的直接使用。我们开发了一个高效的框架，通过利用这些运营生成的标签来训练深度学习分类器，而无需逐报告的人类标注，具体应用于BC癌症登记处的肿瘤组分类。我们使用基于注意力的多实例学习（Attention-Based Multiple Instance Learning, ABMIL）来恢复患者级标签与其所依赖报告之间失去的联系，利用模型对每个报告的注意力，将大量噪声标签的语料库提炼为紧凑的高质量逐报告训练数据集。经过精细调优的分类器在提炼数据集上达到了0.83的宏F1分数，超越了大多数肿瘤组的既定基准。通过将常规运营标签转化为高质量训练数据，而无需额外的标注或大规模计算基础设施，ABMIL为自动化癌症登记工作流程提供了一条实用且可及的途径。

View on arXiv Download PDF AI Translation

cs.CL / 27 / 2607.03482

Lacuna Inc. at SemEval-2026 Task 4: Structurally Gated State-Space Models for Disentangling Narrative Similarity

Lacuna Inc. 在 SemEval-2026 任务 4：用于解构叙事相似性的结构门控状态空间模型

Kudelya, Aleksey, Alshawi, Rafif, Shirnin, Alexander

Abstract

In this paper, we present the Invariant-Variant Disentangled State-Space Model (IVD-SSM), our submission to SemEval-2026 Task 4 on Narrative Story Similarity and Narrative Representation Learning. Evaluating narrative similarity is a profound computational challenge that requires models to look past concrete, superficial elements such as specific names, actors, objects, or settings to isolate and compare abstract patterns of causality and plot progression. To model these extended causal chains without the quadratic bottlenecks of standard Transformers, we leverage a hybrid State-Space Model (Jamba-1.5-Mini). Building upon this backbone, we introduce the Structurally Gated Alignment (SGA) head, a novel, differentiable algorithmic architecture. The SGA head operates on two scales: a heavily strided Macro-path maps the coarse structural skeleton of a story, which then acts as a gating mechanism to filter a full-resolution Micro-path, actively suppressing semantic noise and superficial keyword overlaps. Evaluated on both pairwise comparative judgments (Track A) and dense representation learning (Track B), our approach demonstrates that explicitly disentangling structural invariants from lexical variants provides a robust, principled framework for deep narrative understanding.

Chinese Translation

在本文中，我们提出了不变-变体解构状态空间模型（Invariant-Variant Disentangled State-Space Model，IVD-SSM），这是我们对 SemEval-2026 任务 4 叙事故事相似性和叙事表示学习的提交。评估叙事相似性是一个深刻的计算挑战，要求模型超越具体的、表面的元素，如特定的名称、角色、物体或设置，以孤立和比较因果关系和情节进展的抽象模式。为了在没有标准变换器（Transformers）二次瓶颈的情况下建模这些扩展的因果链，我们利用了一种混合状态空间模型（Jamba-1.5-Mini）。在此基础上，我们引入了结构门控对齐（Structurally Gated Alignment，SGA）头部，这是一种新颖的可微分算法架构。SGA 头部在两个尺度上运行：一个重步幅的宏路径（Macro-path）映射故事的粗略结构骨架，随后作为门控机制过滤全分辨率的微路径（Micro-path），主动抑制语义噪声和表面关键词重叠。在成对比较判断（Track A）和密集表示学习（Track B）上进行评估，我们的方法表明，明确地将结构不变体与词汇变体解构开来，为深度叙事理解提供了一个稳健、原则性的框架。

View on arXiv Download PDF AI Translation

cs.CL / 28 / 2607.03502

Reading Between the Dots: Decoding Hidden Computation across Filler Tokens

点间阅读：解码填充符号中的隐藏计算

Brauer, Kaley, Verdun, Claudio Mayrink, Marks, Samuel

Abstract

Frontier LLMs can perform multi-step reasoning over content-free filler tokens like dots or counting sequences, producing correct answers with no visible chain-of-thought (CoT). This is a limit case for behavioral oversight, where surface tokens carry no information about the underlying reasoning. But hidden from the output is not the same as hidden from us. On four task families (fact retrieval, parallel numeric composition, string manipulation, and in-context computation), two open-weights frontier models (DeepSeek V3, Kimi K2) compute over filler tokens in a structured, legible way: attention routes the question through the filler region to the answer, logit-lens readouts show retrieved facts emerging early and their composition crystallizing in late layers, and KV-cache transplants at filler positions causally swap outputs between examples. We introduce an unsupervised decoding pipeline that takes only hidden states as input and recovers intermediate values with 80-95% accuracy (best LLM judge) across both models and all four tasks, without ground-truth labels or training. Hidden computation that defeats behavioral CoT monitoring is, on these tasks, directly readable from the residual stream, suggesting monitorability is a property of the model's full computational trace, not just its surface tokens.

Chinese Translation

前沿的大型语言模型（LLMs）能够对无内容的填充符号（如点或计数序列）进行多步推理，产生正确答案而没有明显的思维链（CoT）。这是行为监督的一个极限案例，其中表面符号不携带关于潜在推理的信息。但输出中隐藏的并不等同于对我们隐藏的。在四个任务类别（事实检索、并行数值组合、字符串操作和上下文计算）中，两个开放权重的前沿模型（DeepSeek V3, Kimi K2）以结构化、可读的方式对填充符号进行计算：注意力机制将问题通过填充区域引导至答案，logit-lens 读出显示检索的事实早期出现，其组合在后层中凝结，而KV-cache在填充位置的移植因果性地在示例之间交换输出。我们引入了一种无监督解码管道，仅以隐藏状态作为输入，并在两个模型和所有四个任务中以80-95%的准确率（最佳LLM评估者）恢复中间值，无需真实标签或训练。在这些任务中，击败行为CoT监控的隐藏计算可以直接从残差流中读取，这表明可监控性是模型完整计算轨迹的一个特性，而不仅仅是其表面符号。

View on arXiv Download PDF AI Translation

cs.CL / 29 / 2607.03540

Mental Health Disorder Detection Beyond Social Media: A Systematic Review of Available Datasets

超越社交媒体的心理健康障碍检测：可用数据集的系统性综述

Puspo, Sadiya Sayara Chowdhury, Bucur, Ana-Maria, Chancellor, Stevie, Uzuner, Özlem, Zampieri, Marcos

Abstract

Detecting mental health disorders in a timely manner is an important societal challenge. NLP and machine learning (ML) methods used to assist with detection rely on data collected primarily from social media. However, such datasets often have sampling biases and inherent ethical and privacy issues. One avenue to overcome these limitations is non-social media data. We present the first comprehensive review of non-social media, free-text datasets for mental health research. We use the PRISMA methodology to conduct our survey and we review datasets available in multiple languages. We find that non-social media free-text based datasets are predominantly focused on English and on detecting depression. These datasets also vary in demographics, platforms, data types, annotation techniques, and methodologies. This systematic review also reveals key gaps and highlights opportunities to develop more diverse, reliable and clinically-relevant resources.

Chinese Translation

及时检测心理健康障碍是一个重要的社会挑战。用于辅助检测的自然语言处理（NLP）和机器学习（ML）方法主要依赖于从社交媒体收集的数据。然而，这些数据集往往存在抽样偏差以及固有的伦理和隐私问题。克服这些限制的一种途径是使用非社交媒体数据。我们首次全面回顾了用于心理健康研究的非社交媒体自由文本数据集。我们采用PRISMA方法进行调查，并审查了多种语言中可用的数据集。我们发现，非社交媒体自由文本数据集主要集中在英语和抑郁症检测上。这些数据集在人口统计特征、平台、数据类型、注释技术和方法论上也存在差异。这项系统性综述还揭示了关键的空白，并强调了开发更多样化、可靠和临床相关资源的机会。

View on arXiv Download PDF AI Translation

cs.CL / 30 / 2607.03598

They Infer What You Meant: Models Represent Communicative Intent More Reliably Than They Act On It

它们推断你的意思：模型在表示交流意图方面比在其上采取行动更可靠

Kwon, Alex

Abstract

When a person shares something with a language model, the model often answers the surface of the message rather than what the sender was doing by sending it: share a finished project and it critiques the code; share a raw late-night line and it runs a wellness check. We treat the sender's communicative intent, the Gricean what-was-meant, as a first-class interpretability object, and show the failure is one of readout on top of a robust representation. A linear probe decodes the sender's intent, whether they want a thing recognized or evaluated, from a model's default-pass hidden states, cleanly and surface-independently, across six models and four families and in the base checkpoints. The representation generalizes further, to intent that is only pragmatically inferred, and to a second, lexically clean intent (support versus help). The behavioral half of the story, and every causal test, is established on the recognize/evaluate contrast, where what varies is whether the default output acts on the intent. The readout lags the representation in depth within a model (the intent is decodable several layers before it drives the output); across models, which ones act on it by default is model-specific, an observed stratification (three of six show the failure) that we do not read as a scaling law. Where the gap is open, a direction closely tied to the representation, the discriminative direction at a searched-for layer, is a causal handle: steering it recovers the intended behavior, as well as an explicit instruction does and with no prompt at all. This direction is near-orthogonal to the feedback-offering axis, so it routes a represented intent rather than a generic feedback knob, though at the recovery dose the routed intent can override an explicit request. We support each link with controls against obvious deflations and report the nulls as plainly as the confirmations.

Chinese Translation

当一个人向语言模型分享某些内容时，该模型往往关注信息的表面，而不是发送者发送该信息的意图：分享一个完成的项目时，它会对代码进行批评；分享一条深夜的原始信息时，它会进行健康检查。我们将发送者的交流意图，即格赖斯（Grice）所说的“意图是什么”，视为一种一流的可解释性对象，并展示这种失败源于对强大表征的读取。线性探测器从模型的默认隐藏状态中解码发送者的意图，无论他们希望被识别还是被评估，清晰且独立于表面，适用于六个模型和四个模型家族，以及基础检查点。该表征进一步推广到仅通过语用推断的意图，以及第二种词汇上干净的意图（支持与帮助）。故事的行为部分以及每个因果测试都建立在识别/评估的对比上，变化的因素是默认输出是否对意图采取行动。读取落后于模型内部的表征深度（意图在驱动输出之前可以解码几个层次）；在模型之间，哪些模型默认对其采取行动是特定于模型的，观察到的分层（六个模型中有三个显示出这种失败），我们不将其视为一种规模法则。在开放的差距中，与表征紧密相关的方向，即在搜索层次上的判别方向，是一个因果把手：引导它可以恢复预期的行为，就像明确的指令一样，甚至在没有任何提示的情况下。这个方向与提供反馈的轴几乎正交，因此它引导的是一种表征的意图，而不是一个通用的反馈旋钮，尽管在恢复剂量时，所引导的意图可以覆盖明确的请求。我们用控制实验支持每个环节，以防止明显的贬值，并如实报告无效结果与确认结果。

View on arXiv Download PDF AI Translation

cs.CL / 31 / 2607.03640

Revealing Hidden Model Behaviors with Task-Specific Self-Reports

揭示隐藏模型行为的任务特定自我报告

Kutsyk, Taras, Zieliński, Bartosz

Abstract

Fine-tuning can give a language model a hidden behavior--it may give false answers under a narrow condition, or give harmful advice only when a prompt touches a particular topic. We introduce the Stabilized Adapter for self-Report (SAR), a lightweight LoRA adapter that makes a fine-tuned model describe its own hidden behavior in plain language, using only the model and the dataset it was trained on. Across seven implanted behaviors (plus a no-behavior control), SAR detects the hidden behavior in every one--even when the model has generalized into broad misalignment that the training data alone does not predict. Introspection Adapters (IA), the closest existing baseline, detects some behaviors from our suite but misses others entirely--and where it misses, it hallucinates, consistently reporting wrong behaviors. SAR retains positive signal on every setting where IA fails and halves the rate of hallucinations. This makes it much easier for practitioners to audit their models and obtain reliable answers to "what did my model actually learn?" type of questions.

Chinese Translation

微调可以使语言模型表现出隐藏的行为——在特定条件下可能给出错误答案，或仅在提示涉及特定主题时提供有害建议。我们引入了自我报告的稳定适配器（Stabilized Adapter for self-Report，SAR），这是一种轻量级的LoRA适配器，使微调后的模型能够用简单的语言描述其自身的隐藏行为，仅使用模型及其训练所用的数据集。在七种植入行为（加上无行为对照）中，SAR在每一种情况下都能检测到隐藏行为——即使当模型已经泛化到训练数据无法预测的广泛不一致时。最接近的现有基线，内省适配器（Introspection Adapters，IA），能够检测到我们一系列行为中的一些，但完全遗漏了其他行为——而在遗漏的地方，它则产生幻觉，持续报告错误的行为。SAR在IA失败的每个设置中保留了正信号，并将幻觉的发生率减半。这使得从业者更容易审计他们的模型，并获得关于“我的模型实际上学到了什么？”类型问题的可靠答案。

View on arXiv Download PDF AI Translation

cs.CL / 32 / 2607.03681

Annotating Korean adnominal ending constructions in corpus data: Beyond relative-clause identification

对语料库数据中韩语名词修饰结尾构造的注释：超越关系从句的识别

Park, Jungyeul, Park, Chulwoo

Abstract

The Korean adnominal ending \texttt{ETM} occurs in diverse noun-modifying constructions, including relative-clause-like modifiers, adjectival and copular forms, bound-noun constructions, and lexicalized expressions. This paper argues that \texttt{ETM} is not a direct marker of relative-clause structure, but a morphological exponent shared by several adnominal constructions. We propose a corpus-based typology that distinguishes these constructions using predicate type, auxiliary structure, argument-structural compatibility, head-noun restriction, and lexicalized patterns. We operationalize the typology as a construction-sensitive annotation layer for the KLUE dependency treebank, implemented through an ordered rule-based procedure and evaluated by manual validation. Productive relative-clause-like uses account for 39.4\% of the analyzed instances; the remainder consists mainly of adjectival, copular, bound-nominal, modal, temporal, and collocational constructions. The findings show that Korean relative-clause-like modification cannot be identified from adnominal morphology alone.

Chinese Translation

韩语名词修饰结尾 exttt{ETM} 出现在多种名词修饰构造中，包括类似关系从句的修饰语、形容词形式、系动词形式、绑定名词构造以及词汇化表达。本文论证 exttt{ETM} 并不是关系从句结构的直接标记，而是多个名词修饰构造共享的形态学表现。我们提出了一种基于语料库的类型学，通过谓词类型、辅助结构、论元结构兼容性、主名词限制和词汇化模式来区分这些构造。我们将该类型学操作化为 KLUE 依存树库的构造敏感注释层，通过有序的基于规则的程序实现，并通过人工验证进行评估。生产性的类似关系从句的用法占分析实例的 39.4\%；其余主要由形容词、系动词、绑定名词、情态、时间和搭配构造组成。研究结果表明，仅从名词修饰形态学无法识别韩语的类似关系从句的修饰。

View on arXiv Download PDF AI Translation

cs.CL / 33 / 2607.03704

Optimizing Large Language Models for Causality Assessment in Pharmacovigilance: Developing a Performance Metric as Objective for Bayesian Hyperparameter Optimization

优化大型语言模型以进行药物警戒中的因果性评估：作为贝叶斯超参数优化目标的性能指标开发

Heckmann, Nicole Sonne, Vermillet, Arnault-Quentin, Mølgaard, Søren Norlin, Suero, Manuela Del Castillo, Melskens, Lars, Ompad, Gerard, Sessa, Maurizio

Abstract

Background: Growing individual case safety report (ICSR) volumes have intensified demand for scalable automated causality assessment. Large Language Models (LLMs) show promise, yet performance on clinically demanding tasks remains suboptimal and inference-time hyperparameter optimization has not been investigated. Objective: To develop a Gaussian Process (GP)-compatible optimization objective and investigate whether temperature optimization improves LLM-expert agreement on Naranjo causality assessment of FAERS ICSRs. Methods: Expert causality assessments were performed on 723 stratified FAERS cases. OpenAI's GPT-5.2 was evaluated using chain-of-thought (CoT) prompting. Four composite metrics were developed: Weighted Cosine Similarity (WCS), Information-Weighted Agreement Score (IWAS), Entropy-Weighted Agreement and Cosine Similarity Score (EWACS), and Consensus-Weighted Cosine Similarity (CWCS) and Bayesian optimization using a GP surrogate with Probability of Improvement (PoI) acquisition was applied across temperature [0, 2]. Results: GPT-5.2 outperformed prior biomedical LLMs at baseline (T = 0), achieving 74.1% agreement on question 5 and 65.4% on question 10 of Naranjo algorithm. Entropy analysis identified these as the sole informative optimization targets. Temperature showed no systematic population-level effect (\b{eta} = 0.002, p = 0.959). EWACS-guided Bayesian optimization improved causality classification agreement from 45.0% to 72.0% (+27 pp), with the largest gain in Doubtful cases (+42.9 pp). Conclusion: EWACS was identified as the optimal GP-compatible metric. The absence of a universal temperature optimum indicates LLM performance is driven primarily by ICSR content, yet case-specific temperature selection produced meaningful improvements, supporting temperature optimization for LLM-assisted pharmacovigilance.

Chinese Translation

背景：个案安全报告（ICSR）的数量不断增加，导致对可扩展自动化因果性评估的需求加剧。大型语言模型（LLMs）显示出潜力，但在临床要求高的任务上的表现仍然不尽如人意，推理时的超参数优化尚未得到研究。目的：开发一个与高斯过程（GP）兼容的优化目标，并研究温度优化是否能提高LLM与专家在FAERS ICSR的Naranjo因果性评估上的一致性。方法：对723个分层FAERS案例进行了专家因果性评估。使用OpenAI的GPT-5.2进行链式思维（CoT）提示的评估。开发了四个复合指标：加权余弦相似度（WCS）、信息加权一致性评分（IWAS）、熵加权一致性和余弦相似度评分（EWACS）以及共识加权余弦相似度（CWCS），并在温度范围[0, 2]内应用了基于GP代理的贝叶斯优化，采用改进概率（PoI）获取。结果：在基线（T = 0）下，GPT-5.2在生物医学LLMs中表现优越，在Naranjo算法的第5题和第10题上分别达成74.1%和65.4%的一致性。熵分析确定这些是唯一的信息优化目标。温度未显示出系统性的人群水平效应（β = 0.002，p = 0.959）。EWACS引导的贝叶斯优化将因果性分类一致性从45.0%提高至72.0%（+27个百分点），在可疑案例中获得了最大的增益（+42.9个百分点）。结论：EWACS被确定为最佳的与GP兼容的指标。缺乏普遍的温度最优值表明LLM的表现主要受ICSR内容驱动，但特定案例的温度选择产生了显著的改善，支持对LLM辅助药物警戒进行温度优化。

View on arXiv Download PDF AI Translation

cs.CL / 34 / 2607.03709

GRASP: Graph-Reasoning Aided Survey Planning for High-Fidelity Related Work Generation

GRASP：基于图推理的高保真相关工作生成调查规划

Li, Haoming, Ouyang, Jessica

Abstract

Writing a literature review requires a deep understanding of the relationships among cited papers: how they build on, challenge, or offer alternative perspectives to one another. We present Graph-Reasoning Aided Survey Planning (GRASP), a framework combining LLM planning for related work generation with graph algorithms to extract key relationships among cited papers. Our two-layer graph structure consists of a Graph of Thoughts and an Argument-Counterargument Planning Network, representing the cited papers at different levels of granularity, and we apply topology-aware pruning via a Steiner tree to identify the core inter-paper relationships captured in our graph. Our citation analysis-based evaluation shows that GRASP generates related work sections (RWS) that closely match human-written targets in terms of the discourse roles, intents, and grouping of citations.

Chinese Translation

撰写文献综述需要深入理解被引用论文之间的关系：它们如何相互构建、挑战或提供替代视角。我们提出了基于图推理的调查规划（Graph-Reasoning Aided Survey Planning, GRASP），这是一个将大型语言模型（LLM）规划与图算法相结合的框架，用于提取被引用论文之间的关键关系。我们的双层图结构由思想图（Graph of Thoughts）和论点-反论点规划网络（Argument-Counterargument Planning Network）组成，分别在不同的粒度水平上表示被引用论文。我们通过施图（Steiner tree）应用拓扑感知剪枝，以识别在我们的图中捕捉到的核心论文间关系。基于引用分析的评估表明，GRASP生成的相关工作部分（Related Work Sections, RWS）在话语角色、意图和引用分组方面与人类撰写的目标高度匹配。

View on arXiv Download PDF AI Translation

cs.CL / 35 / 2607.03726

SelfMem: Self-Optimizing Memory for AI Agents

SelfMem：用于人工智能代理的自我优化记忆

Yang, Shu, Wu, Junchao, Wong, Derek F., Wang, Di

Abstract

While current AI agents support increasingly long context windows, tool use, and skill execution for long-horizon tasks, they still require memory systems to effectively leverage historical experience. Existing memory frameworks typically rely on fixed storage, retrieval, and summarization mechanisms, which can be rigid across different tasks and often require manual tuning. To address this limitation, we propose SelfMem, a self-optimizing memory framework. Inspired by prior work on self-improving AI, we follow the principle of "teaching an agent to fish rather than giving it a fish." Instead of forcing the model to follow a predefined memory strategy or format, SelfMem provides an environment with memory tools and feedback signals that allow the agent to explore, evaluate, and refine its own memory strategy. Our results show that SelfMem consistently outperforms retrieval, compression, and agent-memory baselines on BEAM across conversation scales from 100K to 1M tokens. Compared with the strongest baseline, SelfMem improves the official score by 48.7%, 40.8%, and 41.9% at 100K, 500K, and 1M, respectively. Further question-type analysis shows broad robustness across diverse memory demands, and our optimization study shows that model-guided strategy refinement further improves performance.

Chinese Translation

虽然当前的人工智能代理支持越来越长的上下文窗口、工具使用和长时间任务的技能执行，但它们仍然需要记忆系统来有效利用历史经验。现有的记忆框架通常依赖于固定的存储、检索和摘要机制，这在不同任务中可能显得僵化，并且往往需要手动调优。为了解决这一限制，我们提出了SelfMem，一个自我优化的记忆框架。受到自我改进人工智能先前工作的启发，我们遵循“教会代理钓鱼，而不是给它一条鱼”的原则。SelfMem并不强迫模型遵循预定义的记忆策略或格式，而是提供一个包含记忆工具和反馈信号的环境，使代理能够探索、评估和优化自己的记忆策略。我们的结果显示，SelfMem在BEAM上在从100K到1M的对话规模中始终优于检索、压缩和代理记忆基线。与最强基线相比，SelfMem在100K、500K和1M时分别提高了官方得分48.7%、40.8%和41.9%。进一步的问题类型分析显示其在多样化记忆需求下的广泛鲁棒性，而我们的优化研究表明，模型引导的策略优化进一步提升了性能。

View on arXiv Download PDF AI Translation

cs.CL / 36 / 2607.03730

ProACT: Towards Breakdown-Aware Proactive Agent in Multi-User Collaboration

ProACT：面向多用户协作中的故障意识主动代理

Yang, Shu, Xu, Difei, Pei, Jiaxin, Wang, Di

Abstract

Conversational agents are increasingly embedded in human collaborative work, yet they remain fundamentally passive and reactive: they respond to explicit user requests rather than proactively recognizing moments when a team would benefit from timely intervention as human collaborators often do. This reactive design substantially limits the use of agents as active participants in multi-user collaboration, where disagreements, ambiguous goals, forgotten constraints, underspecified plans, discussion loops, and imbalanced participation can gradually undermine group progress. To move agents from passive assistants toward active participants in multi-user collaboration, we introduce ProACT, a breakdown-aware agent framework grounded in theories of common ground, collaborative planning, and coordination work. ProACT observes the speaker-attributed conversation history, determines whether the current turn contains a collaboration breakdown requiring intervention, decides whether the agent should stay silent or speak, and, when speaking is needed, routes the case to a targeted collaboration skill. We further introduce the first multi-user collaboration benchmark for evaluating proactive agents across project planning, product design, research collaboration, logistics, education, and resource-constrained decision making. Across 3,244 turn-level examples and five LLM backbones, ProACT consistently improves collaborative appropriateness, non-interruptiveness, conciseness, and judged intervention quality over direct chat.

Chinese Translation

对话代理越来越多地嵌入人类协作工作中，但它们仍然基本上是被动和反应性的：它们响应明确的用户请求，而不是主动识别团队在何时需要及时干预的时刻，这正是人类协作者常常能够做到的。这种反应性设计大大限制了代理作为多用户协作中积极参与者的使用，在这种协作中，分歧、模糊目标、被遗忘的约束、未充分指定的计划、讨论循环和不平衡的参与可能逐渐削弱团队的进展。为了将代理从被动助手转变为多用户协作中的积极参与者，我们提出了ProACT，一个基于共同基础、协作规划和协调工作的理论的故障意识代理框架。ProACT观察发言者归属的对话历史，判断当前轮次是否包含需要干预的协作故障，决定代理是保持沉默还是发言，并在需要发言时，将案例路由到特定的协作技能。我们进一步介绍了第一个多用户协作基准，用于评估主动代理在项目规划、产品设计、研究协作、物流、教育和资源受限决策中的表现。在3,244个轮次级示例和五个大型语言模型（LLM）基础上，ProACT在协作适当性、非干扰性、简洁性和判断干预质量方面始终优于直接聊天。

View on arXiv Download PDF AI Translation

cs.CL / 37 / 2607.03833

Beyond Static Rules: Automated Discovery of Latent Vulnerabilities in Text-to-SQL

超越静态规则：文本到SQL中潜在漏洞的自动发现

Wang, Hanqing, Chi, Yongdong, Yang, Jian, Yang, Lei, Zhao, Jiehui, Chen, Yun, Chen, Guanhua

Abstract

While Large Language Models (LLMs) have achieved remarkable success in Text-to-SQL tasks, their deployment in real-world environments is hindered by latent reliability issues. Identifying these latent weaknesses is critical for building trustworthy database interfaces, yet current diagnostic approaches rely heavily on static, expert-defined rules, which lack the capability for systematic and automated exploration. To bridge this gap, we propose SAGE (Systematic Automated Guided Exploration), a novel framework designed to autonomously uncover latent failure patterns in LLM-based Text-to-SQL generation. Specifically, SAGE generates vulnerability hypotheses for given samples and references a continuously evolving Vulnerability Codex to design targeted perturbations, thereby iteratively verifying and documenting potential defects. Extensive experiments on state-of-the-art open-source LLMs demonstrate that SAGE uncovers a substantial number of failure cases, highlighting the significant fragility of current models. Furthermore, our analysis reveals that the Vulnerability Codex exhibits strong cross-model transferability, indicating that the discovered patterns represent generalized structural weaknesses. Finally, we explore SAGE's potential for remediation. Although preliminary, lightweight fine-tuning on the generated samples yields promising improvements, suggesting a scalable pathway for closing the reliability loop in future work.

Chinese Translation

尽管大型语言模型（LLMs）在文本到SQL任务中取得了显著成功，但其在现实环境中的应用受到潜在可靠性问题的制约。识别这些潜在弱点对于构建可信赖的数据库接口至关重要，然而当前的诊断方法过于依赖静态的、专家定义的规则，缺乏系统化和自动化探索的能力。为了解决这一问题，我们提出了SAGE（系统化自动引导探索），这是一个旨在自主发现基于LLM的文本到SQL生成中的潜在失败模式的新框架。具体而言，SAGE为给定样本生成漏洞假设，并参考不断发展的漏洞编码（Vulnerability Codex）设计针对性的扰动，从而迭代验证和记录潜在缺陷。对最先进的开源LLM进行的大量实验表明，SAGE揭示了大量失败案例，突显了当前模型的显著脆弱性。此外，我们的分析显示，漏洞编码具有强大的跨模型可转移性，表明所发现的模式代表了普遍的结构性弱点。最后，我们探讨了SAGE的修复潜力。尽管初步的轻量级微调在生成样本上取得了令人鼓舞的改进，但这表明在未来的工作中，缩小可靠性差距的可扩展路径是可行的。

View on arXiv Download PDF AI Translation

cs.CL / 38 / 2607.03863

Rethinking Scientific Discovery in an Agentic Era

在代理时代重新思考科学发现

Zheng, Yining, Wang, Yuxin, Lu, Jiahao, Fang, Shicheng, Wang, Weiyi, Yang, Yongzhuo, Li, Bowen, Ma, Haochen, Hu, Chen, Chen, Bowen, Wang, Yang, Chen, Huanhui, Chen, Yitong, Chen, Jiajun, Li, Zhiyuan, Li, Yanlin, Yang, Zhuo, Wu, Qifeng, He, Jiaying, Jinluo, Zhijie, Xu, Xiaohu, Feng, Yi, Qian, Juncheng, Chen, Yizhou, Cheng, Yang, Zhu, Tong, Ying, Tianlei, Yu, Hongyu, Xiang, Hongjun, Qiu, Xipeng

Abstract

Artificial intelligence has advanced scientific discovery, but most AI4Science systems remain fragmented tools that rely on humans to coordinate problem formulation, literature grounding, model use, simulation, validation, and knowledge reuse. This paper presents \textbf{SCION (Scientific Collaborative Innovation with Agentic Organizational Nexus)}, an agentic scientific operating system that acts as an \textbf{organizational nexus}. Through a Science Agent serving as a \textbf{Meta-Harness}, SCION connects scientific tasks, tools, agents, artifacts, and memory, transforming research into an executable, auditable, and reusable operational process. At its core is the \textbf{Research Execution Plan (REP)}, which compiles high-level scientific intent into staged objectives, dependencies, verification checkpoints, tool requirements, expected artifacts, and fallback conditions. SCION further integrates hierarchical multi-agent execution, profile-driven specialization, selective context construction, governed delegation, and layered epistemic memory to support long-horizon scientific work. We formulate discovery under SCION as \textbf{Target-conditioned Inverse Search} and extend it to hidden-target settings through batch active search under finite experimental budgets. Applications in materials analysis, molecule design, and protein or antibody screening, together with experiments on scientific reading, idea generation, molecule generation, and antibody screening, show that SCION outperforms existing autonomous research-agent baselines, especially in decomposition, verification, refinement, and memory reuse. Overall, SCION shifts AI from isolated tools toward a coordinated operational layer for traceable and reusable scientific innovation.

Chinese Translation

人工智能推动了科学发现的进步，但大多数AI4Science系统仍然是依赖人类协调问题制定、文献基础、模型使用、模拟、验证和知识重用的碎片化工具。本文提出了 extbf{SCION（科学协作创新代理组织枢纽）}，一种作为 extbf{组织枢纽}的代理科学操作系统。通过作为 extbf{元工具}的科学代理，SCION连接科学任务、工具、代理、工件和记忆，将研究转变为可执行、可审计和可重用的操作过程。其核心是 extbf{研究执行计划（REP）}，该计划将高层次的科学意图编译为分阶段的目标、依赖关系、验证检查点、工具需求、预期工件和后备条件。SCION进一步整合了分层多代理执行、基于档案的专业化、选择性上下文构建、受控委派和分层认知记忆，以支持长周期的科学工作。我们在SCION下将发现形式化为 extbf{目标条件反向搜索}，并通过有限实验预算下的批量主动搜索将其扩展到隐性目标设置。材料分析、分子设计以及蛋白质或抗体筛选等应用，以及在科学阅读、创意生成、分子生成和抗体筛选方面的实验表明，SCION在分解、验证、精炼和记忆重用方面优于现有的自主研究代理基线。总体而言，SCION将人工智能从孤立工具转变为可追溯和可重用科学创新的协调操作层。

View on arXiv Download PDF AI Translation

cs.CL / 39 / 2607.03882

Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Language

一致但未校准：评估大型语言模型在自然语言风险沟通中的局限性

Cerda-Mardini, Diego, Chandar, Sarath, Madathil, Sreenath

Abstract

LLMs are increasingly deployed as post-hoc explainers of AI-generated outputs, yet it remains unclear whether they can reliably communicate probabilistic information in natural language. For this role to be viable, models must produce identical verbal descriptions for identical inputs, and select descriptions that accurately reflect the magnitude of the underlying numerical quantities. We evaluate whether nine LLMs meet these requirements within a two-stage prediction pipeline, in which an upstream model has produced probabilistic outputs characterized by their likelihood and uncertainty, and LLMs are tasked with selecting an appropriate verbal descriptor for each. We simulate predictions from an upstream model by taking samples from a Beta distribution parameterized by its mode and prior sample size. We then prompt LLMs to explain these predictions under six domain contexts and with ten temperature settings, and repeating each experiment ten times. We find that LLMs are generally consistent but miscalibrated, with substantially weaker performance on uncertainty than on likelihood tasks. Providing models with precomputed summary statistics (mode and prior sample size) reduced sensitivity to contextual framing but did not resolve the underlying miscalibration, suggesting that the bottleneck resides in the verbalization step itself. These findings indicate that current LLMs do not yet constitute reliable zero-shot standalone risk communication tools for probabilistic predictions.

Chinese Translation

大型语言模型（LLMs）越来越多地被用作对人工智能生成输出的事后解释者，但尚不清楚它们是否能够可靠地用自然语言传达概率信息。为了使这一角色可行，模型必须对相同输入生成相同的语言描述，并选择准确反映基础数值量大小的描述。我们评估了九个LLMs在一个两阶段预测流程中的表现，其中上游模型生成了以其可能性和不确定性为特征的概率输出，而LLMs的任务是为每个输出选择合适的语言描述。我们通过从一个以其众数和先前样本大小为参数的Beta分布中抽样来模拟上游模型的预测。然后，我们在六个领域背景下和十个温度设置下提示LLMs解释这些预测，并重复每个实验十次。我们发现，LLMs通常表现出一致性但未校准，在不确定性任务上的表现明显弱于可能性任务。向模型提供预先计算的汇总统计数据（众数和先前样本大小）减少了对上下文框架的敏感性，但并未解决潜在的未校准问题，这表明瓶颈存在于语言化步骤本身。这些发现表明，目前的LLMs尚未构成可靠的零-shot独立风险沟通工具，用于概率预测。

View on arXiv Download PDF AI Translation

cs.CL / 40 / 2607.03929

Probe, Don't Prompt: A Hidden-State Probe for Metadata Filtering in Multi-Meta-RAG

探测，而非提示：一种用于多元元检索增强生成（Multi-Meta-RAG）中的元数据过滤的隐状态探测器

Poliakov, Mykhailo, Shvai, Nadiya

Abstract

Multi-Meta-RAG improves retrieval for multi-hop question answering by filtering a vector store on metadata (the news source) that it extracts from each query by prompting gpt-3.5-turbo. We show this proprietary, free-form extractor can be replaced by a local, deterministic probe trained on the hidden states of a small open-source language model. On all 2556 MultiHop-RAG queries the probe reaches 90.9% set-exact accuracy against 88.0% for a model-free substring baseline and 80.9% for GPT-3.5, a margin that comes entirely from null queries, on which GPT-3.5 never abstains; on non-null queries all three stay within about a point. Because the probe's output space is exactly the fixed 49-source vocabulary, it cannot drift outside the allow-list as the prompted model does. Three design choices make it work: selecting a shallow layer, mean pooling, and class-imbalance-aware multi-label training over the long tail of sources. A 135M-parameter model lands within ~1.5 points of a 1.5B one, so the filter is cheap to output: a partial forward pass through the first few layers plus one linear head, with no API. The code is available at https://github.com/mxpoliakov/Multi-Meta-RAG.

Chinese Translation

多元元检索增强生成（Multi-Meta-RAG）通过过滤从每个查询中提取的元数据（新闻来源）来改善多跳问答的检索，该元数据是通过提示 gpt-3.5-turbo 获得的。我们展示了这种专有的自由形式提取器可以被一个基于小型开源语言模型的隐状态训练的局部确定性探测器所替代。在所有 2556 个 MultiHop-RAG 查询中，该探测器达到了 90.9% 的集合精确度，而无模型子字符串基线为 88.0%，GPT-3.5 为 80.9%，这一差距完全来自于无效查询，GPT-3.5 在这些查询中从不放弃；在非无效查询中，三者的表现相差约一分。由于探测器的输出空间正好是固定的 49 个来源词汇，因此它无法像提示模型那样漂移出允许列表。三个设计选择使其有效：选择一个浅层、均值池化，以及对长尾来源进行类别不平衡感知的多标签训练。一个 135M 参数的模型与一个 1.5B 参数的模型相差约 1.5 分，因此该过滤器的输出成本低廉：只需通过前几层进行部分前向传播加上一个线性头，无需 API。代码可在 https://github.com/mxpoliakov/Multi-Meta-RAG 获取。

View on arXiv Download PDF AI Translation

cs.CL / 41 / 2607.03936

Can Dialects Be Steered Like Languages? Sparse Neurons and Distributed Directions in Arabic LLMs

方言能像语言一样被引导吗？阿拉伯大型语言模型中的稀疏神经元与分布式方向

Elozeiri, Kareem, Abassy, Mervat, Kallas, Omar, Dalvi, Fahim, Nakov, Preslav, Inui, Kentaro, Durrani, Nadir

Abstract

A key challenge in Arabic NLP is the scarcity of dialectal data relative to Modern Standard Arabic (MSA), causing LLMs to overproduce MSA and struggle with dialectally accurate generation. From an interpretability perspective, this raises a fundamental question: where and how are dialectal features encoded within model internals, and can these representations be leveraged to improve dialect generation without fine-tuning? This study investigates two complementary inference-time approaches that serve simultaneously as interpretability probes and control mechanisms. First, we conduct a neuron-level analysis, identifying sparse neuron populations that encode dialect-specific features and showing that amplifying or suppressing these neurons can steer model outputs toward target dialects. Second, motivated by the entanglement of dialectal features at the single-neuron level, we apply a vector-steering approach that extracts dialect-specific activation directions and injects them during inference. Together, these methods illuminate the geometry of dialectal knowledge in Arabic LLMs and offer a principled, interpretability-grounded framework for dialect control without requiring dialect-specific fine-tuning.

Chinese Translation

阿拉伯自然语言处理中的一个关键挑战是方言数据相对于现代标准阿拉伯语（MSA）的稀缺，导致大型语言模型（LLMs）过度生成MSA，并在方言准确生成方面面临困难。从可解释性的角度来看，这引发了一个基本问题：方言特征在模型内部是如何编码的，能否利用这些表示来改善方言生成而无需微调？本研究探讨了两种互补的推理时方法，这些方法同时作为可解释性探针和控制机制。首先，我们进行神经元级分析，识别出编码方言特定特征的稀疏神经元群体，并展示了增强或抑制这些神经元可以将模型输出引导到目标方言。其次，基于单神经元层面方言特征的纠缠，我们应用了一种向量引导方法，该方法提取方言特定的激活方向并在推理过程中注入它们。这些方法共同揭示了阿拉伯LLMs中方言知识的几何特征，并提供了一个有原则的、基于可解释性的方言控制框架，无需特定方言的微调。

View on arXiv Download PDF AI Translation

cs.CL / 42 / 2607.03953

The Remarkable Effectiveness of Providing AI Agents with Natural Language Tools: A Replication Study Validating NLT Performance Across 14 Models

为人工智能代理提供自然语言工具的显著有效性：一项验证14个模型中自然语言工具性能的复制研究

Somma, Alexander, Plante, Isabelle, Premji, Fred

Abstract

This study independently replicates and extends the Natural Language Tools (NLT) framework of Johnson et al.~(2025), which questions the use of structured tool calling in large language model (LLM) agentic systems. We evaluated NLT across 14 models and 8,560 trials, adding newer frontier, reasoning, and open-weight models to the original set. The results confirm the core findings and add detail. NLT improves tool-calling accuracy by 14.9 percentage points overall (62.3\% versus 47.4\% structured) and reduces critical errors by 93\% (51 versus 755 errors). The gains depend on model capability: models without native tool calling, reasoning models, and smaller models gain substantially (+24.0pp to +43.1pp), while heavily optimized frontier models (GPT-5, Gemini 2.5 Pro) show smaller or reversed advantages. This matches recent analyses of reinforcement-learning-optimized tool use (Martinez, 2025). NLT also cuts token usage by 25.2\%. The reliability and efficiency advantages compound in recursive agentic workflows, where agents chain many tool calls across sub-agents: a structured failure triggers retries, fallback routing, and coordination overhead, while NLT avoids most of that cost at the source. This work makes three contributions: (1) the first independent validation of NLT using open-source tooling, (2) evidence that model capability moderates NLT's advantages (Chen et al., 2025; Zhang et al., 2025), and (3) a measurement of NLT's reliability benefit (93\% fewer errors), its most deployment-relevant property given the known fragility of structured tool calling. NLT is a practical alternative to structured tool calling, especially for production systems that value reliability over parseability.

Chinese Translation

本研究独立复制并扩展了Johnson等人（2025）的自然语言工具（NLT）框架，质疑在大型语言模型（LLM）代理系统中使用结构化工具调用的有效性。我们在14个模型和8,560次试验中评估了NLT，新增了前沿、推理和开放权重模型。结果确认了核心发现并增加了细节。NLT整体上提高了工具调用的准确性14.9个百分点（62.3%对比47.4%结构化），并将关键错误减少了93%（51个对比755个错误）。增益依赖于模型能力：没有原生工具调用、推理模型和较小模型的增益显著（+24.0pp至+43.1pp），而经过高度优化的前沿模型（GPT-5、Gemini 2.5 Pro）则显示出较小或相反的优势。这与最近对强化学习优化工具使用的分析相符（Martinez, 2025）。NLT还将令牌使用量减少了25.2%。在递归代理工作流程中，这种可靠性和效率的优势会叠加，代理在子代理之间链式调用多个工具：结构化失败会触发重试、后备路由和协调开销，而NLT在源头上避免了大部分成本。本研究做出了三项贡献：（1）使用开源工具对NLT的首次独立验证，（2）证据表明模型能力调节了NLT的优势（Chen et al., 2025; Zhang et al., 2025），以及（3）对NLT可靠性益处的测量（93%更少的错误），这是考虑到结构化工具调用已知脆弱性后最相关的部署属性。NLT是结构化工具调用的实用替代方案，特别适用于重视可靠性的生产系统。

View on arXiv Download PDF AI Translation

cs.CL / 43 / 2607.03957

NormWorlds-CF: Solver-Verified Counterfactual Normative Reasoning with Metamorphic-Relation GRPO

NormWorlds-CF：具有变形关系 GRPO 的求解器验证的反事实规范推理

Zhang, Xinqi

Abstract

Language models can reach the right normative verdict for the wrong reason. We introduce NormWorlds-CF, a solver-verified environment for counterfactual normative reasoning in executable rule worlds. Its deterministic solver produces final answers, proof and falsification certificates, argument statuses, support sets, and paired-world change labels, enabling supervision and evaluation without LLM judges. The benchmark contains staged SFT diagnostics and a compact paired-world task with 270 root families and 1080 canonical-to-variant pairs. The SFT diagnostics show that final-answer supervision is an unsafe proxy: answer-only SFT reaches perfect accuracy on answer tasks but scores zero on falsification, while proof-plus-falsification training with targeted replay reaches strong all-task accuracy. For the structured-change task, we introduce metamorphic-relation GRPO (MR-GRPO), a class-conditioned reward for GRPO that gives partial credit for relation families and solver-visible change fields. In matched 1.7B continuation experiments, MR-GRPO improves held-out relation accuracy and relation-family correctness, and reduces wrong-family error, compared to sparse and answer-only GRPO. In Qwen3-4B three-seed validation, answer-only reward improves answer-change fields but weakens relation-family structure, sparse reward preserves coarse relation labels best, and MR-GRPO delivers the strongest balanced performance across answer-change, support-change, status-change, and soft root-level metamorphic-relation metrics. These results show that verified counterfactual structure can shape post-training beyond final answers, while exact full change-record generation, invariant subtype recognition, and out-of-distribution (OOD) transfer remain open problems.

Chinese Translation

语言模型可能会因错误的原因得出正确的规范裁决。我们介绍了 NormWorlds-CF，这是一个用于可执行规则世界中的反事实规范推理的求解器验证环境。其确定性求解器生成最终答案、证明和反驳证书、论证状态、支持集以及配对世界变化标签，从而实现无需大型语言模型（LLM）评审的监督和评估。基准测试包含分阶段的 SFT 诊断和一个紧凑的配对世界任务，涵盖 270 个根家族和 1080 对规范到变体的配对。SFT 诊断表明，最终答案监督是一个不安全的代理：仅答案的 SFT 在答案任务上达到完美准确率，但在反驳上得分为零，而结合证明与反驳训练的目标重放在所有任务上达到了较强的准确率。对于结构变化任务，我们引入了变形关系 GRPO（MR-GRPO），这是一种条件奖励，为 GRPO 提供了部分信用，针对关系家族和求解器可见的变化字段。在匹配的 1.7B 续写实验中，与稀疏和仅答案的 GRPO 相比，MR-GRPO 提高了保留关系的准确性和关系家族的正确性，并减少了错误家族的错误。在 Qwen3-4B 三种种子的验证中，仅答案奖励改善了答案变化字段，但削弱了关系家族结构，稀疏奖励最有效地保留了粗略关系标签，而 MR-GRPO 在答案变化、支持变化、状态变化和软根级变形关系指标上提供了最强的平衡性能。这些结果表明，经过验证的反事实结构可以在训练后塑造超越最终答案的内容，而精确的完整变化记录生成、不变子类型识别和分布外（OOD）迁移仍然是未解决的问题。

View on arXiv Download PDF AI Translation

cs.CL / 44 / 2607.03974

TRACER: Early Failure Detection for Task-Oriented Dialogue

TRACER：面向任务的对话系统早期故障检测

Nourbakhsh, Erfan, Slavin, Rocky, Yang, Ke, Rios, Anthony

Abstract

Task-oriented dialogue systems often fail before the final breakdown is obvious, but most evaluation only measures failure after the conversation has already gone wrong. We present TRACER, a method for early failure detection in task-oriented dialogue. TRACER predicts from a partial dialogue whether the full conversation will eventually fail by combining simple trajectory signals from belief-state changes with text representations of the evolving dialogue state. We evaluate the method in both oracle and generated belief-state settings, and test how well it works when only 25%, 50%, 75%, or 100% of the dialogue is visible. Across these settings, TRACER detects useful failure signals well before the end of the conversation and outperforms heuristic, classical, and single-stream baselines. These results suggest that early failure detection can provide a practical warning signal for dialogue systems before the interaction fully breaks down.

Chinese Translation

面向任务的对话系统常常在最终故障明显之前就已失败，但大多数评估仅在对话已经出现问题后才进行故障测量。我们提出了TRACER，一种用于面向任务对话的早期故障检测方法。TRACER通过结合信念状态变化的简单轨迹信号与不断演变的对话状态的文本表示，预测部分对话是否最终会失败。我们在理想和生成的信念状态设置中评估该方法，并测试在仅可见25%、50%、75%或100%对话的情况下其效果如何。在这些设置中，TRACER能够在对话结束之前很早就检测到有用的故障信号，并且优于启发式、经典和单流基线。这些结果表明，早期故障检测可以为对话系统提供实用的警告信号，防止交互完全崩溃。

View on arXiv Download PDF AI Translation

cs.CL / 45 / 2607.03981

BanglaMemeEvidence: A Multimodal Benchmark Dataset for Explanatory Evidence Detection in Bengali Memes

BanglaMemeEvidence：用于孟加拉语表情包解释性证据检测的多模态基准数据集

Faria, Fatema Tuj Johora, Moin, Mukaffi Bin, Rahman, Md. Mahfuzur, Debnath, Pronay, Fahim, Asif Iftekher, Shah, Faisal Muhammad

Abstract

Memes have become influential communication tools on social media, combining viral visuals with concise messaging to convey impactful ideas. While substantial research has examined the affective dimensions of memes, key challenges such as detecting harmful content, identifying cyberbullying, and performing accurate sentiment analysis remain critical, largely due to the need for deeper contextual understanding. In this paper, we introduce MemeEvidenceDetect, a hybrid task aimed at analyzing a meme and its contextual information to identify specific sentences that explain or elucidate its meaning and humor. To support this task, we present BanglaMemeEvidence, a curated dataset of 2,917 Bengali memes, emphasizing its significance as a resource for the Bangla language. Each meme is annotated with natural language explanations, including Meme OCR, Meme Context, and Evidence Sentences, alongside relevance scores that reflect the relationship between a meme and its corresponding annotations. To address the gap in dynamically inferring a meme's context, we propose BengaliMemeEvidenceNet, a hybrid multimodal framework that integrates textual and visual features for comprehensive meme representation. Our experiments demonstrate the effectiveness of BengaliMemeEvidenceNet, achieving an F1 score of 0.74. To the best of our knowledge, this is the first study to focus on evidence detection in Bengali memes, marking a notable step forward in the analysis of memes in low-resource languages.

Chinese Translation

表情包已成为社交媒体上具有影响力的沟通工具，通过结合病毒式的视觉效果和简洁的信息传递来传达深刻的思想。尽管已有大量研究探讨了表情包的情感维度，但检测有害内容、识别网络欺凌以及进行准确的情感分析等关键挑战仍然存在，主要是由于对更深层次的上下文理解的需求。在本文中，我们介绍了MemeEvidenceDetect，这是一个旨在分析表情包及其上下文信息的混合任务，以识别特定句子来解释或阐明其含义和幽默感。为支持这一任务，我们呈现了BanglaMemeEvidence，这是一个经过精心策划的数据集，包含2,917个孟加拉语表情包，强调其作为孟加拉语资源的重要性。每个表情包都附有自然语言解释，包括Meme OCR、Meme Context和Evidence Sentences，以及反映表情包与其相应注释之间关系的相关性评分。为了解决动态推断表情包上下文的空白，我们提出了BengaliMemeEvidenceNet，这是一个混合多模态框架，整合了文本和视觉特征以实现全面的表情包表示。我们的实验表明，BengaliMemeEvidenceNet的有效性，达到了0.74的F1得分。据我们所知，这是首个关注孟加拉语表情包中证据检测的研究，标志着在低资源语言的表情包分析方面迈出了重要一步。

View on arXiv Download PDF AI Translation

cs.CL / 46 / 2607.04008

Candidate-Constrained Retrieval-Augmented Generation for LongEval-RAG: System Design and Empirical Analysis

候选约束检索增强生成系统用于 LongEval-RAG：系统设计与实证分析

Yang, Yingdong, Wu, Haijian

Abstract

We present a candidate-constrained retrieval-augmented generation system for LongEval-RAG, where each query is associated with an organizer-provided candidate set and all retrieved evidence and final citations must remain within that set. The system combines deterministic provenance tracking with passage-based retrieval, deterministic query expansion, pseudo-relevance feedback (PRF), reciprocal rank fusion (RRF), lightweight evidence reranking, citation-aware evidence aggregation, and optional MiniLM sentence reranking. We evaluate ten pipeline variants using a primary organizer evaluation and a supplementary self-generated diagnostic protocol. The primary evaluation shows that the strongest balanced variant is rule-minilm: a rule-based chunking pipeline with query expansion, PRF, RRF, reranking, citation prior, and late MiniLM sentence selection. This variant obtains the highest BERTScore, retrieval precision, nugget coverage, and average grade among our submissions. The result suggests that the main gain does not come from more complex semantic or topic-shift chunking, but from pairing stable rule-based evidence units with sentence-level neural selection before generation. The supplementary LLM-judge evaluation remains useful for early diagnosis and additional analysis, but it emphasizes different systems than the primary gold-answer and nugget-based evaluation, highlighting the need for multi-metric RAG evaluation.

Chinese Translation

我们提出了一种候选约束的检索增强生成系统用于 LongEval-RAG，其中每个查询都与组织者提供的候选集相关联，所有检索到的证据和最终引用必须保持在该集合内。该系统结合了确定性来源追踪与基于段落的检索、确定性查询扩展、伪相关反馈（PRF）、互惠排名融合（RRF）、轻量级证据重排名、关注引用的证据聚合，以及可选的 MiniLM 句子重排名。我们使用主要组织者评估和补充自生成的诊断协议评估了十种管道变体。主要评估显示，最强的平衡变体是 rule-minilm：一种基于规则的分块管道，具有查询扩展、PRF、RRF、重排名、引用优先和后期 MiniLM 句子选择。该变体在我们的提交中获得了最高的 BERTScore、检索精度、信息覆盖率和平均评分。结果表明，主要收益并不是来自于更复杂的语义或主题转移分块，而是来自于在生成之前将稳定的基于规则的证据单元与句子级神经选择相结合。补充的 LLM-judge 评估对于早期诊断和额外分析仍然有用，但它强调了与主要黄金答案和基于信息的评估不同的系统，突显了多指标 RAG 评估的必要性。

View on arXiv Download PDF AI Translation

cs.CL / 47 / 2607.04011

Separating Representation from Reconstruction Enables Scalable Text Encoders

将表示与重建分离以实现可扩展的文本编码器

Dervishi, Megi, Videau, Mathurin, LeCun, Yann

Abstract

While decoders have rapidly scaled, encoders have remained largely unchanged since BERT. We revisit this disparity by frozen backbone evaluation via probing. Under this lens, the representations of BERT encoders become increasingly $\textit{unexploitable}$ by frozen probes, despite improved perplexity. The misalignment originates in BERT's flat design, which couples representation learning to the token reconstruction loss. We propose $\textbf{CrossBERT}$, a two-part architecture that separates the learning of high-quality encoded representations from the rigid grounding of token reconstruction. This design further enables high masking ratios ($\ge 50\%$) and gradient collection over all tokens via a $\textit{Complementary Masking Strategy}$, respectively increasing throughput by $1.5$ to $2\times$ and sample efficiency by $2\times$. Overall, CrossBERT demonstrates monotonic scaling and superior performance on MTEB(eng, v2) and frozen GLUE benchmarks.

Chinese Translation

尽管解码器迅速扩展，但编码器自BERT以来基本保持不变。我们通过冻结骨干网络评估重新审视这一差异。在这种视角下，尽管困惑度有所改善，BERT编码器的表示却变得越来越不可利用。其不一致性源于BERT的扁平设计，该设计将表示学习与标记重建损失耦合在一起。我们提出了$ extbf{CrossBERT}$，一种将高质量编码表示的学习与标记重建的严格基础分离的两部分架构。这一设计进一步支持高掩蔽比例（$ extge 50 ext%$）以及通过$ extit{补充掩蔽策略}$对所有标记进行梯度收集，分别将吞吐量提高了$1.5$到$2 imes$，样本效率提高了$2 imes$。总体而言，CrossBERT展示了单调扩展性，并在MTEB(eng, v2)和冻结GLUE基准测试中表现出色。

View on arXiv Download PDF AI Translation

cs.CL / 48 / 2607.04029

CrossHallu: Do Hallucination Signals Generalize Across Languages and Domains in Large Language Model's Internals?

CrossHallu：幻觉信号在大型语言模型内部是否跨语言和领域泛化？

Alansari, Aisha, Alkhorasani, Malak, Luqman, Hamzah

Abstract

Recent hallucination detection techniques in large language models (LLMs) focus on directly extracting features from a model's internal representations and training a classifier on these features to detect hallucinations, demonstrating promising results. Notwithstanding this advancement, most internal-state hallucination detection techniques have been explored predominantly in English, raising the question of whether such internal signals generalize across different languages and domains. To address this gap, we present CrossHallu, the first study to evaluate the cross-lingual and cross-domain generalization of hallucination detection using internal representations from six LLMs on the generative question-answering task. We conduct a systematic Arabic <-> English evaluation using TruthfulQA, an Arabic translated version of TruthfulQA, and HalluScore. This evaluation encompasses monolingual training and testing, cross-lingual transfer, cross-domain transfer, and combined cross-lingual and cross-domain transfer. The results reveal that internal-state hallucination signals in LLMs transfer across languages and domains for most models, with cross-lingual performance highly dependent on both class separability and language alignment in the feature space, whereas cross-domain transfer within Arabic varies depending on the training and testing datasets used for the hallucination detector. The code is publicly available at https://github.com/aishaalansari57/CrossHal.

Chinese Translation

近期在大型语言模型（LLMs）中进行的幻觉检测技术主要集中于直接从模型的内部表示中提取特征，并在这些特征上训练分类器以检测幻觉，显示出良好的结果。尽管这一进展显著，但大多数内部状态幻觉检测技术主要在英语中进行探索，这引发了一个问题：这些内部信号是否能够跨不同语言和领域进行泛化。为了解决这一问题，我们提出了CrossHallu，这是首个评估使用六个LLMs的内部表示在生成问答任务中进行幻觉检测的跨语言和跨领域泛化的研究。我们使用TruthfulQA、TruthfulQA的阿拉伯语翻译版本以及HalluScore进行了系统的阿拉伯语<->英语评估。该评估涵盖了单语训练和测试、跨语言迁移、跨领域迁移以及结合的跨语言和跨领域迁移。结果显示，大多数模型的LLMs中的内部状态幻觉信号能够在语言和领域之间迁移，跨语言性能高度依赖于特征空间中的类别可分性和语言对齐，而在阿拉伯语中的跨领域迁移则取决于用于幻觉检测器的训练和测试数据集。代码已公开发布在 https://github.com/aishaalansari57/CrossHal。

View on arXiv Download PDF AI Translation

cs.CL / 49 / 2607.04061

Telescope: Improving Zero Shot Detection of LLM Generated Content By Measuring Token Repetition Probability

Telescope：通过测量标记重复概率提高对大型语言模型生成内容的零样本检测

Nassif, Christopher, Cooper, Josh F.

Abstract

Distinguishing Large Language Model (LLM) generated text from human writing is a critical and difficult challenge. While LLMs are trained to write like humans, we hypothesize that this training leaves an indelible mark. LLMs develop a particularly strong aversion to token repetition very early in training. This bias persists as a ''Vestigial Heuristic'' (a developmental artifact) that is activated in LLM-generated text, separating LLM from human writing. To probe this phenomenon, we introduce Telescope Perplexity, a metric that evaluates the token repetition of the model, $P(s_i | s_{1:i})$ . Our empirical investigation reveals that the Telescope Perplexity signature emerges early in pre-training, and Telescope Perplexity empirically enables highly effective zero-shot LLM detection. We show state-of-the-art or competitive performance across diverse datasets (including modern evaluation sets we introduce), reference models, and perturbation schemes with greater efficiency than other methods.

Chinese Translation

区分大型语言模型（LLM）生成的文本与人类写作是一项关键且困难的挑战。尽管LLM被训练得像人类一样写作，但我们假设这种训练留下了不可磨灭的痕迹。LLM在训练的早期阶段对标记重复表现出特别强烈的厌恶。这种偏见作为一种“退化启发式”（一种发展遗物）在LLM生成的文本中被激活，从而将LLM与人类写作区分开来。为了探究这一现象，我们引入了Telescope Perplexity，这是一种评估模型标记重复的指标，$P(s_i | s_{1:i})$。我们的实证研究表明，Telescope Perplexity特征在预训练的早期就会出现，并且Telescope Perplexity在实证上能够实现高效的零样本LLM检测。我们展示了在多种数据集（包括我们引入的现代评估集）、参考模型和扰动方案中，Telescope Perplexity的性能达到了最先进或具有竞争力的水平，并且效率高于其他方法。

View on arXiv Download PDF AI Translation

cs.CL / 50 / 2607.04064

Speaker-Disentangled Chunk-Wise Regression for Syllabic Tokenization

说话者解耦的分块回归用于音节标记

Komatsu, Ryota, Kawakita, Kota, Okamoto, Takuma, Shinozaki, Takahiro

Abstract

Unsupervised syllabic tokenization aims to learn discrete syllabic tokens that capture latent linguistic content-related structure from raw speech. Recent syllabic tokenization methods employ teacher-student distillation of the pretrained HuBERT to organize latent speech frame representations into syllabic segments. However, when trained with an utterance-level cross-entropy objective, the model predicts speaker identity rather than linguistic content, thereby compromising the purity of syllabic tokens. To address this problem, we propose a speaker-disentangled syllabic tokenizer that regresses speaker-perturbed student representations toward clean teacher targets within fixed-length chunks. Experimental results demonstrate that our proposed method achieves state-of-the-art performance in syllable boundary detection and syllabic segment clustering. Moreover, a speech language model trained on our syllabic tokens achieves a 7% relative improvement in syntactic and semantic understanding over the phone-level SpiRit-LM.

Chinese Translation

无监督音节标记旨在从原始语音中学习离散的音节标记，以捕捉潜在的与语言内容相关的结构。最近的音节标记方法采用预训练的 HuBERT 的教师-学生蒸馏，将潜在的语音帧表示组织成音节段。然而，当使用话语级交叉熵目标进行训练时，模型预测的是说话者身份而非语言内容，从而影响了音节标记的纯净性。为了解决这个问题，我们提出了一种说话者解耦的音节标记器，该标记器在固定长度的块内将受说话者干扰的学生表示回归到干净的教师目标。实验结果表明，我们提出的方法在音节边界检测和音节段聚类方面达到了最先进的性能。此外，基于我们的音节标记训练的语音语言模型在句法和语义理解上相较于基于音素的 SpiRit-LM 实现了 7% 的相对提升。

View on arXiv Download PDF AI Translation

cs.CL / 51 / 2607.04071

Beyond Multilingual Averages: MTEB-PT, a Benchmark for Portuguese Sentence Encoders

超越多语言平均值：MTEB-PT，葡萄牙语句子编码器基准

Okamura, Lucas Hideki Takeuchi, Alcoforado, Alexandre, Costa, Anna Helena Reali

Abstract

Portuguese remains underrepresented in text embedding evaluation, despite being one of the most widely spoken languages in the world. As a result, embedding models are often selected based on English or multilingual metrics, while their effectiveness in Portuguese remains unclear. We present MTEB-PT, a Portuguese benchmark constructed from a subset of MMTEB, comprising 14 existing datasets across Semantic Textual Similarity (STS), classification, retrieval, and reranking. We use this benchmark to evaluate 17 open- and closed-source embedding models under a unified protocol. Our results show that Portuguese performance is strongly task-dependent: multilingual rankings do not reliably predict Portuguese-specific performance across task families, no single model dominates all settings, and models with stronger long-context capacity are particularly advantageous on longer-input tasks such as retrieval and reranking. The benchmark also shows that language-specific fine-tuning still improves model performance in Portuguese, especially on task types that match the adaptation data most closely. To examine this effect, we fine-tune three representative backbone models with Portuguese contrastive supervision and Matryoshka Representation Learning (MRL). These benchmark-informed baselines yield their strongest gains on STS, consistent with the predominantly symmetric supervision used during training, while also improving retrieval and remaining competitive under dimensional truncation. We release the MTEB-PT benchmark, the fine-tuned models, and the training and evaluation code.

Chinese Translation

尽管葡萄牙语是世界上使用最广泛的语言之一，但在文本嵌入评估中仍然代表性不足。因此，嵌入模型通常基于英语或多语言指标进行选择，而它们在葡萄牙语中的有效性仍不明确。我们提出了MTEB-PT，这是一个基于MMTEB子集构建的葡萄牙语基准，包含14个现有数据集，涵盖语义文本相似性（STS）、分类、检索和重排序。我们使用该基准在统一协议下评估17个开源和闭源的嵌入模型。我们的结果表明，葡萄牙语的表现高度依赖于任务：多语言排名无法可靠预测各任务家族中的葡萄牙语特定表现，没有单一模型在所有设置中占据主导地位，而具有更强长上下文能力的模型在检索和重排序等较长输入任务中尤其具有优势。基准还表明，特定语言的微调仍然能够提高模型在葡萄牙语中的表现，特别是在与适应数据最为匹配的任务类型上。为了检验这一效果，我们使用葡萄牙语对比监督和Matryoshka表示学习（MRL）对三种代表性主干模型进行了微调。这些基于基准的基线在STS上获得了最大的提升，符合训练过程中使用的主要对称监督，同时在检索中也有所改善，并在维度截断下保持竞争力。我们发布了MTEB-PT基准、微调模型以及训练和评估代码。

View on arXiv Download PDF AI Translation

cs.CL / 52 / 2607.04107

Semantic Integration and Lexical Expectation Shape N400 and P600 Dynamics During Naturalistic Reading

语义整合与词汇预期在自然阅读过程中塑造 N400 和 P600 动态

Sun, Kun, Wang, Rong

Abstract

Word surprisal is a well-established computational predictor of human neural responses during language comprehension, but it remains less clear whether local semantic fit explains neural response variation beyond lexical expectation during naturalistic reading. Using the Dublin EEG-based Reading Experiment Corpus (DERCo), this study examined whether contextual semantic relevance predicts word-locked EEG activity in the N400 and P600 windows. Contextual semantic relevance was computed as an attention-aware measure of how strongly a target word is semantically connected to its recent discourse context, and it was compared with GPT-based word surprisal. Across 22 participants and 32 EEG channels, we tested both predictors using regression-based ERP analyses and generalized additive mixed models while controlling for lexical variables and repeated observations. Both predictors were reliably associated with EEG responses, but they showed partly different temporal and scalp-level patterns. Surprisal captured expectancy-related variation, whereas contextual semantic relevance showed robust effects across N400- and P600-window mean voltages, with particularly strong explanatory support in the P600 window. Model comparisons indicated that contextual semantic relevance contributed explanatory value beyond lexical controls and surprisal. These findings suggest that naturalistic reading depends on both lexical expectation and local semantic integration, and that contextual semantic relevance offers an interpretable computational link between discourse semantic fit and ERP dynamics.

Chinese Translation

词汇惊讶度是一个公认的计算预测因子，用于预测人类在语言理解过程中的神经反应，但在自然阅读过程中，局部语义适配是否能解释神经反应的变化超出词汇预期仍不清楚。本研究利用都柏林基于 EEG 的阅读实验语料库 (DERCo)，考察了上下文语义相关性是否能够预测 N400 和 P600 窗口中的词锁定 EEG 活动。上下文语义相关性被计算为一种关注度感知的度量，反映目标词与其近期话语上下文的语义连接强度，并与基于 GPT 的词汇惊讶度进行了比较。在 22 名参与者和 32 个 EEG 通道中，我们使用基于回归的 ERP 分析和广义加性混合模型测试了这两个预测因子，同时控制了词汇变量和重复观察。两个预测因子均与 EEG 反应可靠相关，但它们在时间和头皮水平上表现出部分不同的模式。惊讶度捕捉了与预期相关的变化，而上下文语义相关性在 N400 和 P600 窗口的平均电压中显示出强有力的效果，尤其在 P600 窗口中提供了特别强的解释支持。模型比较表明，上下文语义相关性在超出词汇控制和惊讶度的情况下提供了额外的解释价值。这些发现表明，自然阅读依赖于词汇预期和局部语义整合，而上下文语义相关性为话语语义适配与 ERP 动态之间提供了可解释的计算联系。

View on arXiv Download PDF AI Translation

cs.CL / 53 / 2607.04223

Detecting Hallucinations in Retrieval-Augmented Generation through Grounding-Aware Sensitivity by Perturbation (GASP)

通过扰动的基础感知灵敏度检测检索增强生成中的幻觉 (GASP)

Bouke, Mohamed Aly

Abstract

Retrieval-augmented generation (RAG) reduces but does not eliminate hallucination, and existing detectors return a single answer-level score that does not indicate which sentence is unsupported, or why. To close this gap, we introduce Grounding-Aware Sensitivity by Perturbation (GASP), a span-level detector that scores each answer sentence by how strongly its likelihood depends on the retrieved evidence, a quantity we term grounding sensitivity. GASP holds the answer fixed and re-scores it under the full context, under no context, and with each chunk removed, then measures the log-likelihood drops and Jensen-Shannon divergences (JSD). The likelihood of a grounded sentence collapses once its supporting passage is removed, whereas a hallucinated sentence is almost unaffected, a contrast we interpret by casting decoding as a random nonlinear iterated function system (RNIFS). We evaluate GASP on three benchmarks (RAGTruth, TofuEval, RAGBench) with three instruction-tuned scorers from two model families (Qwen2.5-0.5B, Qwen2.5-1.5B, and SmolLM2-1.7B) under a leakage-clean protocol. On RAGTruth it reaches a response-level area under the ROC curve (AUC) of about 0.73 and a span-level AUC of about 0.67, improving significantly over perplexity and by clear margins over length, whole-context natural language inference (NLI), and self-consistency baselines. The only baseline competitive at the span level is a well-configured chunk-level entailment verifier, which requires a separate model, whereas a training-free threshold on the grounding features matches the trained classifier without labeled data and serves as the default detector. Beyond RAGTruth, the signal transfers to TofuEval but not to short-answer question answering in RAGBench, showing GASP is best suited to outputs constructed from the retrieved context rather than answers recoverable from parametric knowledge.

Chinese Translation

检索增强生成 (RAG) 减少了幻觉的发生，但并未消除它，现有的检测器返回单一的答案级别评分，无法指示哪些句子没有支持或原因。为了解决这一问题，我们引入了基础感知灵敏度通过扰动 (GASP)，这是一种跨度级别的检测器，通过检索证据的依赖程度来评分每个答案句子，我们称之为基础灵敏度。GASP 固定答案，并在完整上下文、无上下文以及逐块移除的情况下重新评分，然后测量对数似然下降和詹森-香农散度 (JSD)。一旦其支持段落被移除，基础句子的似然性会崩溃，而幻觉句子几乎不受影响，这一对比我们通过将解码视为随机非线性迭代函数系统 (RNIFS) 来解释。我们在三个基准（RAGTruth、TofuEval、RAGBench）上评估 GASP，使用来自两个模型系列（Qwen2.5-0.5B、Qwen2.5-1.5B 和 SmolLM2-1.7B）的三个指令调优评分器，在泄漏清理协议下。在 RAGTruth 上，它达到了约 0.73 的响应级别 ROC 曲线下面积 (AUC) 和约 0.67 的跨度级别 AUC，相较于困惑度显著提高，并且在长度、整体上下文自然语言推理 (NLI) 和自一致性基线方面有明显的优势。唯一在跨度级别具有竞争力的基线是一个配置良好的块级蕴涵验证器，需使用单独的模型，而在基础特征上的无训练阈值与无标注数据的训练分类器相匹配，并作为默认检测器。除了 RAGTruth，信号传递到 TofuEval，但未能转移到 RAGBench 的短答案问答，表明 GASP 更适合从检索上下文构建的输出，而非从参数知识中恢复的答案。

View on arXiv Download PDF AI Translation

cs.CL / 54 / 2607.04235

Spinning Straw into Gold: Relabeling LLM Agent Trajectories in Hindsight for Successful Demonstrations

将稻草变为黄金：回顾性重新标记LLM代理轨迹以实现成功演示

Li, Zichao, Wu, Gang, Wang, Zichao, Zhang, Ruiyi, Zhu, Wanrong, Rossi, Ryan A., Morariu, Vlad I, Kil, Jihyung

Abstract

Large language model agents operate in partially observable, long-horizon settings where obtaining supervision remains a major bottleneck. We address this by utilizing a source of supervision overlooked in existing post-training methods: unintended yet successful goals embedded within agent rollouts. Specifically, we introduce Hindsight Supervised Learning (HSL), where an auxiliary LLM reviews each completed trajectory and relabels it with all of the natural-language goals the agent actually achieved. HSL then pairs the trajectory with its relabeled goals and uses these pairs for additional fine-tuning. To mitigate suboptimality in the relabeled data, we propose two learning techniques for HSL, irrelevant-action masking and sample reweighting. Our experiments show that HSL is flexible and compatible with existing post-training pipelines. It improves both SFT and DPO, with larger gains on long-horizon tasks with more diverse goal spaces. Moreover, HSL is sample-efficient: on ALFWorld, it surpasses baselines trained on the full dataset while using only one quarter of the ground-truth demonstrations.

Chinese Translation

大型语言模型代理在部分可观察的长时间跨度环境中操作，其中获取监督仍然是一个主要瓶颈。我们通过利用现有后训练方法中被忽视的监督来源来解决这一问题：嵌入在代理执行中的意外但成功的目标。具体而言，我们引入了回顾性监督学习（Hindsight Supervised Learning, HSL），在该方法中，辅助LLM审查每个完成的轨迹，并用代理实际实现的所有自然语言目标对其进行重新标记。HSL随后将轨迹与其重新标记的目标配对，并使用这些配对进行额外的微调。为了减轻重新标记数据中的次优性，我们提出了两种HSL学习技术：无关动作屏蔽和样本重加权。我们的实验表明，HSL灵活且与现有的后训练流程兼容。它改善了顺序微调（SFT）和直接政策优化（DPO），在具有更多样化目标空间的长时间跨度任务上获得了更大的收益。此外，HSL具有样本效率：在ALFWorld上，它在仅使用四分之一的真实演示的情况下超越了基于完整数据集训练的基线。

View on arXiv Download PDF AI Translation

cs.CL / 55 / 2607.04281

Risk-Constrained Freshness-Aware Semantic Caching for Open-Web Retrieval-Augmented LLMs

Mansoor, Muhammad, Ahmad, Tahir, Yoon, Yeo-Chan

Abstract

Semantic caching reduces the latency and cost of retrieval-augmented generation (RAG) by serving cached answers to semantically similar queries, but most existing methods do not model the time-varying freshness of open-web evidence. We present FreshCache, a three-tier semantic cache that treats cache reuse as a risk-constrained temporal inference problem: before approving a cache hit, FreshCache estimates the probability that the cached result is stale using a fitted exponential decay model enhanced by a learned MLP, and approves reuse only when that probability falls below a per-tier error budget across answers (epsilon = 0.10), URL lists (epsilon = 0.20), and page content (epsilon = 0.35). This allows the system to degrade gracefully as entries age rather than forcing a binary choice between a stale hit and a full pipeline execution. We introduce FreshCache-Bench, a benchmark of 8,072 base queries across five freshness classes with ground truth staleness labels drawn from real web snapshots at 1, 12, 24 hours, and 7 days after a baseline crawl, expanded to 31,201 queries via paraphrase generation. At the 24-hour evaluation window, FreshCache_MLP achieves 97% search API savings at 0.1% hash-based stale error, and an LLM-judge evaluation on 396 confirmed change pairs shows that only 34.3% of detected content changes actually affect answer correctness, placing true answer-affecting stale error at approximately 0.034%. The rule-based FreshCache achieves 98% search savings at 3.3% stale error under a temporal holdout calibration, outperforming SemanticTTL (14.9% stale, 72% saved), vCache (7.2% stale, 47% saved), and SCALM (5.2% stale, 96% saved). Ablations show the temporal risk gate accounts for an 11.6 point reduction in stale error over similarity-only reuse, and the learned MLP reduces stale error a further 3.2 points over the rule-based model.

View on arXiv Download PDF AI Translation

cs.CL / 56 / 2607.04293

CausalGame: Benchmarking Causal Thinking of LLM Agents in Games

CausalGame：评估大型语言模型代理在游戏中的因果思维能力

Chen, Zhenhao, Chen, Yongqiang, Liu, Chenxi, Yu, Junchi, Song, Xiangchen, Li, Zijian, Li, Jialin, Torr, Philip, Han, Bo, Zhang, Kun

Abstract

Building AI Scientist agents with Large Language Models (LLMs) has recently attracted growing attention. Since scientific discovery fundamentally relies on uncovering causal relationships from observations, the capability of causal thinking, i.e., distinguishing causation from correlation and recognizing hidden biases, is essential to LLM agents. Although a number of benchmarks exist for AI Scientists, none explicitly incorporate challenges from selection bias, measurement error, and hidden confounders that widely exist in real-world scientific discovery. To this end, we present CausalGame, a benchmark that evaluates the causal thinking capabilities of LLM agents through interactive games. CausalGame asks LLM agents to actively design experimental protocols, collect observation data, and derive a final solution with an explanation report. To emulate realistic scientific discovery challenges, we design 14 scenarios that incorporate selection bias, measurement error, and hidden confounders. Across 30 LLM agents, none demonstrates reliable causal thinking: the best model reaches only 68.0% survival against analytical optima of 78-85%, and merely 5-7% of sessions receive credits on the causal-reasoning rubrics. CausalGame provides a scalable and controlled testbed for evaluating the causal thinking of AI Scientist agents.

Chinese Translation

构建基于大型语言模型（LLMs）的人工智能科学家代理最近引起了越来越多的关注。由于科学发现根本上依赖于从观察中揭示因果关系，因此因果思维能力，即区分因果关系与相关性以及识别隐藏偏差，对于LLM代理至关重要。尽管已经存在一些针对人工智能科学家的基准测试，但没有一个明确地纳入现实世界科学发现中普遍存在的选择偏差、测量误差和隐藏混杂因素的挑战。为此，我们提出了CausalGame，这是一个通过互动游戏评估LLM代理因果思维能力的基准测试。CausalGame要求LLM代理积极设计实验方案、收集观察数据，并撰写解释报告以得出最终解决方案。为了模拟现实科学发现中的挑战，我们设计了14个场景，涵盖选择偏差、测量误差和隐藏混杂因素。在30个LLM代理中，没有一个表现出可靠的因果思维：最佳模型的生存率仅为68.0%，而分析最优值为78-85%，仅有5-7%的会话在因果推理评分标准上获得认可。CausalGame提供了一个可扩展且受控的测试平台，用于评估人工智能科学家代理的因果思维能力。

View on arXiv Download PDF AI Translation

cs.CL / 57 / 2607.04319

Legible-by-Construction: Attention and End-to-End Transformers

构建可读性：注意力机制与端到端变换器

Oskin, Mark

Abstract

A companion paper showed that a transformer's feed-forward layer can be rebuilt from explicit fuzzy set operations - intersection, set-difference, and a self-forgetting sequence quantifier - so its hidden units read as named logical operators at no cost to language-model quality. That left the other half of the transformer opaque. Here we carry the same idea into attention and join the two into one model. The mechanism is minimal: a head's value is passed through a sigmoid, so each value channel becomes a readable detector of whether a feature holds at a token. This adds no parameters and leaves the standard head otherwise untouched. A Boolean variant goes further, restructuring the value into an explicit within-token intersection and negation-capable set-difference. In both designs the output projection is left free, not tied to the vocabulary, which is the load-bearing decision: bounding what a head detects while leaving what it writes unconstrained yields selective detectors, whereas constraining the write does not. A bounded value is shaped into a readable detector by two selectivity pressures - one for sparse firing, one for decisive firing at the rails - and which a design wants is not universal. Across five specialized-attention designs at 125M parameters, 44 to 62 percent of value channels become crisp, contextually selective detectors, and their legibility rises with depth rather than crystallizing only on punctuation. Language-model quality is at parity with a conventional baseline. Finally, we couple the Boolean attention to the legible feed-forward layer and train an end-to-end legible-by-construction language model at benchmark parity: its feed-forward units are named set and quantifier operations throughout, and we can take a token it generates and read the named units that compose to produce it.

Chinese Translation

一篇相关论文展示了变换器的前馈层可以通过显式模糊集合操作（交集、集合差和自遗忘序列量词）重建，从而使其隐藏单元表现为命名的逻辑运算符，而不会影响语言模型的质量。这使得变换器的另一半仍然不透明。在这里，我们将相同的思想应用于注意力机制，并将两者结合为一个模型。该机制是最小化的：一个头的值通过一个sigmoid函数传递，因此每个值通道成为一个可读的检测器，用于判断某个特征在一个标记上是否成立。这不会增加参数，并且保持标准头部不变。一个布尔变体进一步重构值，形成一个显式的标记内交集和具有否定能力的集合差。在这两种设计中，输出投影是自由的，不与词汇表绑定，这是一项承载决策：限制一个头检测的内容，同时不限制其写入的内容，产生选择性检测器，而限制写入则不会。一个有界值通过两个选择性压力被塑造成可读的检测器——一个用于稀疏触发，一个用于在边界处的决定性触发——而设计所需的并不是普遍的。在125M参数的五种专用注意力设计中，44%到62%的值通道变成清晰的、上下文选择性的检测器，并且它们的可读性随着深度的增加而提高，而不仅仅是在标点符号上结晶。语言模型的质量与传统基线相当。最后，我们将布尔注意力与可读的前馈层结合，并训练一个端到端的构建可读性语言模型，其性能与基准相当：其前馈单元在整个过程中都是命名的集合和量词操作，我们可以取出它生成的一个标记，并读取组成该标记的命名单元。

View on arXiv Download PDF AI Translation

cs.CL / 58 / 2607.04350

WPG-MoE: Weak-Prior-Guided Dense Mixture-of-Experts for User-Level Social Media Depression Detection

Li, Xian, Tian, Yuanhe, Yang, Yang, Wang, Guoqing, Song, Yan

Abstract

Online social media posts provide scalable signals for early depression screening, and recent studies mainly improve pre-classification evidence through risk-post selection, symptom grounding, and clinically informed feature construction. However, these screening-stage designs often leave final decisions to a single detector, overlooking how users heterogeneously express depressive risk after screening. A monolithic classifier must average across heterogeneous users, which may dilute localized evidence and cause misclassification, especially for non-self-disclosing users. To address this issue, we propose WPG-MoE, a weak-prior-guided dense mixture-of-experts framework built on a shared large language model (LLM) backbone. WPG-MoE derives user-level weak semantic priors to softly route users to experts matched to different evidence layouts. We formulate this process as learning using privileged information (LUPI): rich LLM-extracted structured evidence guides training-time routing, while inference retains only Patient Health Questionnaire-9 (PHQ-9) template screening and the deployable backbone. Experiments on Chinese and English datasets show that WPG-MoE outperforms strong baselines with interpretable routing behavior.

View on arXiv Download PDF AI Translation

cs.CL / 59 / 2607.04391

Memory-Orchestrated Semantic System (MOSS): An Auditable Agentic Memory Architecture

Lacasse, Serge, Hatier, Jérémie, Baker, Alex

Abstract

Long-term memory remains a structural weakness of AI agents. The dominant approach, retrieval-augmented generation (RAG), relies on embedding-based similarity search, which is opaque by construction, difficult to audit, and bounded by the theoretical limits of vector representations. We present the Memory-Orchestrated Semantic System (MOSS), an agentic memory architecture in which the agent drives retrieval over a structured relational database. MOSS is model-agnostic, storage-agnostic, and API-agnostic: it runs on any relational engine, connects to any LLM provider (or to deterministic non-LLM processes), and deploys on any infrastructure, local or cloud. Its retrieval execution is symbolic and reproducible (once a query is formulated, no LLM participates in the retrieval loop) and every step of the system, from indexing to answer formulation, is logged and inspectable, making MOSS auditable by construction. Rather than imposing an external ontology, MOSS derives its conceptual vocabulary from the corpus itself. We report on a longitudinal deployment unique in the agentic-memory literature: a year of continuous production over an individual scholar's working corpus--a conversational corpus reaching back to October 2024 (some 44 million tokens, retroactively indexed) comprising 110,183 segments, alongside 163,494 catalogued documents, 569 inductively derived concepts, 322,662 concept annotations, and eleven metadata graphs totaling approximately five million relations--across four successive infrastructure generations. While the present case is that of a single researcher, the architecture is in no way specific to one person: it serves a team, an institution, or any entity that accumulates knowledge over time. We argue that auditable, sovereign, structurally unbounded memory is a precondition for AI agents intended to accompany a person or an organization over years rather than sessions.

View on arXiv Download PDF AI Translation

cs.CL / 60 / 2607.04410

AI Wizards at EXIST 2026: Hierarchical Soft-Label Learning for Multimodal Sexism Identification in Memes

EXIST 2026 的 AI Wizards：用于多模态性别歧视识别的层次软标签学习

Fasulo, Matteo, Gravina, Antonio, Tedeschini, Luca, Babboni, Luca

Abstract

We present the AI Wizards submission to EXIST 2026 for multimodal sexism identification in memes. The task is composed of three, increasingly harder subtasks. We model them hierarchically as conditional soft-label prediction over empirical annotator distributions. Our system maps fixed Gemini Embedding 2 vision-language representations through a lightweight Gated MLP trained with KL divergence and homoscedastic uncertainty weighting. Our submissions ranked first on Task 2.3 and fourth on Tasks 2.1 and 2.2 on the official Soft-Soft leaderboards. The code is available at https://github.com/NLP-AI-Wizards/EXIST-2026

Chinese Translation

我们提出了 AI Wizards 对 EXIST 2026 的提交，旨在对表情包中的多模态性别歧视进行识别。该任务由三个逐渐增加难度的子任务组成。我们将其建模为对经验标注者分布的条件软标签预测，采用层次化的方法。我们的系统通过轻量级的 Gated MLP 进行固定的 Gemini Embedding 2 视觉-语言表示映射，该模型使用 KL 散度和同方差不确定性加权进行训练。我们的提交在官方 Soft-Soft 排行榜上，任务 2.3 排名第一，任务 2.1 和 2.2 排名第四。代码可在 https://github.com/NLP-AI-Wizards/EXIST-2026 获取。

View on arXiv Download PDF AI Translation

cs.CL / 61 / 2607.04425

UI-MOPD: Multi-Platform On-Policy Distillation for Continual GUI Agent Learning

UI-MOPD：用于持续图形用户界面代理学习的多平台在线蒸馏

Lian, Niu, Chen, Alan, Yu, Zhehao, Duan, Chengzhen, Liu, Fazhan, Liu, Hui, Fu, Pei, Luan, Jian, Wang, Yaowei, Xia, Shu-Tao, Wang, Jinpeng

Abstract

Recent advances in multimodal foundation models and agent systems have driven GUI agents from single-platform task execution toward cross-platform interaction. However, building multi-platform GUI agents remains challenging. On one hand, high-quality and executable cross-platform interaction trajectories are still scarce, and existing data often suffer from limited platform coverage. On the other hand, different platforms exhibit distinct interaction conventions, making joint or continual training prone to behavioral pattern mixing, platform-specific capability degradation, and catastrophic forgetting. To address these challenges, we construct Uni-GUI, a high-quality cross-platform GUI interaction dataset, and propose UI-MOPD, the first method that incorporates multi-teacher on-policy distillation into continual learning for GUI agents. UI-MOPD dynamically selects a platform-specific teacher according to the current environment and transfers platform-specific behavioral priors to a shared policy through platform-conditioned distillation, enabling adaptation to new platforms while preserving capabilities on existing ones. Experiments on OSWorld and MobileWorld show that UI-MOPD achieves task success rates of 38.2% and 12.0%, respectively, demonstrating its effectiveness in balancing cross-platform capability retention and new-platform adaptation. Project page: https://elispectre.github.io/UI-MOPD/.

Chinese Translation

近年来，多模态基础模型和代理系统的进展推动了图形用户界面（GUI）代理从单平台任务执行向跨平台交互的发展。然而，构建多平台的GUI代理仍然面临挑战。一方面，高质量且可执行的跨平台交互轨迹仍然稀缺，现有数据往往存在平台覆盖有限的问题。另一方面，不同平台表现出不同的交互规范，使得联合或持续训练容易导致行为模式混合、平台特定能力下降和灾难性遗忘。为了解决这些挑战，我们构建了Uni-GUI，一个高质量的跨平台GUI交互数据集，并提出了UI-MOPD，这是首个将多教师在线蒸馏纳入GUI代理持续学习的方法。UI-MOPD根据当前环境动态选择特定平台的教师，并通过平台条件蒸馏将平台特定的行为先验转移到共享策略中，从而在保持现有平台能力的同时适应新平台。在OSWorld和MobileWorld上的实验表明，UI-MOPD分别实现了38.2%和12.0%的任务成功率，证明了其在平衡跨平台能力保留和新平台适应性方面的有效性。项目页面：https://elispectre.github.io/UI-MOPD/

View on arXiv Download PDF AI Translation

cs.CL / 62 / 2607.04428

dOPSD: On-Policy Self-Distillation for Diffusion Language Models

Dat, Phuong Tuan, Li, Qi, Wang, Xinchao

Abstract

Diffusion large language models (dLLMs) generate text by iteratively denoising a masked sequence, offering a parallel alternative to autoregressive models, but eliciting strong reasoning through post-training remains difficult: supervised fine-tuning is off-policy and suffers from exposure bias, while reinforcement learning gives only sparse, sequence-level rewards and is hard to apply without tractable sequence likelihoods. On-policy self-distillation (OPSD) offers a promising alternative, using one model as both student and teacher to provide dense, token-level, on-policy supervision, but its effectiveness hinges on giving the teacher privileged information (PI) - typically an instance-specific ground-truth reference unavailable at inference - so the student ends up distilling a weak PI-free consensus policy that yields little improvement on dLLM reasoning. We introduce dOPSD, which instead derives the teacher's privilege directly from the student's own denoising trajectory, evaluating masked positions using later, more-decoded steps of that same trajectory rather than an external label, so the teacher's advantage emerges from the model's own decoding process; on Dream and LLaDA, dOPSD improves both in-domain math reasoning and out-of-domain code generation, outperforming supervised and on-policy baselines.

View on arXiv Download PDF AI Translation

cs.CL / 63 / 2607.04429

evalci: A Python Library for Statistically Rigorous Comparison of Language Model Evaluations

evalci：一个用于语言模型评估的统计严谨比较的Python库

Chandrahas, Shreyas K

Abstract

The dominant practice in language model evaluation is to report a single accuracy number per model and declare the higher one better, without testing whether the gap could plausibly be sampling noise. On benchmarks of a few thousand items, and under temperature sampling where a model can differ from itself run to run by more than the reported gap between models, this practice routinely overstates confidence in headline claims. The statistical machinery to fix this -- confidence intervals, paired significance tests, power analysis, clustered standard errors, multiple-comparison correction -- is well established, but no standard, pip-installable tool packages it in the shape an evaluation actually takes: a per-item results table. We present evalci, a pure-Python library (numpy/scipy/pandas only) that turns a per-item results table into a publication-ready claim -- e.g., "Model A beats Model B, $\Delta=3.1$ pts, 95% CI [1.2, 5.0], paired permutation $p=0.002$, $n=1{,}319$" -- in one function call, with adapters for lm-evaluation-harness and HELM output. Every routine is validated against an independent reference (statsmodels, or brute-force exact enumeration) rather than only against itself. As a case study, we re-analyze a public comparison of nine language models' MMLU accuracy and find that 3 of the 8 adjacent leaderboard-rank gaps are not statistically significant after correcting for the 36 pairwise comparisons the ranking implies. evalci is available at https://pypi.org/project/evalci/ (source: https://github.com/Shreyaskc/evalci, DOI: https://doi.org/10.5281/zenodo.21201815)

Chinese Translation

语言模型评估的主流做法是为每个模型报告一个单一的准确率，并宣称较高的准确率更好，而不测试这种差距是否可能是抽样噪声。在几千个项目的基准测试中，在温度抽样下，一个模型在不同运行之间的差异可能超过模型之间报告的差距，这种做法常常夸大了对主要声明的信心。解决这一问题的统计工具——置信区间、配对显著性检验、效能分析、聚类标准误差、多重比较校正——已经得到充分建立，但没有标准的、可通过pip安装的工具将其打包成评估实际所需的形式：每项结果表。我们提出了evalci，一个纯Python库（仅使用numpy/scipy/pandas），可以将每项结果表转化为可发表的声明——例如，“模型A优于模型B，$ ext{Δ}=3.1$分，95%置信区间[1.2, 5.0]，配对置换$p=0.002$，$n=1{,}319$”——只需一次函数调用，并提供lm-evaluation-harness和HELM输出的适配器。每个例程都是针对一个独立参考（statsmodels或暴力精确枚举）进行验证，而不仅仅是针对自身。作为案例研究，我们重新分析了九个语言模型的MMLU准确率的公开比较，发现在校正了排名所暗示的36个成对比较后，8个相邻排行榜差距中有3个在统计上并不显著。evalci可在https://pypi.org/project/evalci/获取（源代码：https://github.com/Shreyaskc/evalci，DOI：https://doi.org/10.5281/zenodo.21201815）

View on arXiv Download PDF AI Translation

cs.CL / 64 / 2607.04430

Uncertainty-Aware Abstention in Large Language Models with Provable Alignment Guarantees

Dong, Sijin, Shinnou, Hiroyuki

Abstract

Large language models (LLMs) are increasingly deployed in question answering (QA) systems, yet they may generate hallucinated or misaligned responses without reliable confidence estimates. Uncertainty quantification (UQ) offers a natural basis for selective answering, where a system answers only when its prediction is deemed reliable and abstains otherwise. However, existing uncertainty scores for LLMs are often heuristic: a threshold chosen on such scores does not, by itself, provide statistical guarantees on the error rate among accepted answers. We propose CIC, a confidence-interval-based calibration framework that converts arbitrary uncertainty scores into risk-controlled selective answering rules. Given a held-out calibration set, CIC evaluates each generated response using an application-specific alignment criterion and associates it with an uncertainty score and a binary error label. For each candidate uncertainty threshold, CIC estimates the acceptance-conditioned error rate and constructs a high-probability upper confidence bound using either Hoeffding-style or Clopper-Pearson confidence intervals. It then selects the largest threshold whose upper bound is below a user-specified risk level $\alpha$, thereby maximizing the answering rate subject to a finite-sample reliability constraint. Under exchangeability, CIC guarantees with probability at least $1-\delta$ that the selected threshold, if non-null, controls the error rate among accepted answers at level $\alpha$. We evaluate CIC on both closed-ended and open-ended QA benchmarks across seven LLMs and multiple uncertainty estimators. Experimental results show that CIC consistently achieves valid risk control while retaining strong answering efficiency, providing a practical and statistically grounded mechanism for deploying LLMs in reliability-sensitive QA workflows.

View on arXiv Download PDF AI Translation

cs.CL / 65 / 2607.04469

Don't Commit Alone: Joint Token Commitment in Diffusion Large Language Models

不要单独承诺：扩散大语言模型中的联合令牌承诺

Yao, Lin

Abstract

Diffusion large language models (dLLMs) commit multiple tokens per denoising step by decoding each selected position independently from the shared context; when those positions are dependent, the resulting factorization error is captured by conditional total correlation, which confidence-based selection cannot observe from marginals alone. We propose CoCommit, a marker-gated coordination pass that briefly defers commitment: after the usual bundle selection, a learned marker announces the commit set and the backbone's last-$n$ layers are re-applied so marked positions coordinate -- approximating joint-mode decoding -- before greedy argmax writes tokens. The method reuses existing weights with one extra partial forward pass and no auxiliary model. On LLaDA2.1-mini with LoRA adapters and matched greedy inference, joint commitment improves accuracy on all six benchmarks we evaluate, with the largest gains on reasoning and exact-answer tasks.

Chinese Translation

扩散大语言模型（dLLMs）在去噪步骤中通过独立解码每个选定位置来承诺多个令牌；当这些位置相互依赖时，所产生的因式分解误差由条件总相关性捕捉，而基于置信度的选择无法仅通过边际分布观察到这一点。我们提出了CoCommit，这是一种标记门控协调传递机制，暂时延迟承诺：在常规的束选择之后，一个学习到的标记宣布承诺集，并重新应用骨干网络的最后$n$层，以便标记位置进行协调——近似联合模式解码——然后贪婪的argmax写入令牌。该方法重用了现有权重，仅需额外的部分前向传递，并且不需要辅助模型。在带有LoRA适配器的LLaDA2.1-mini上，经过匹配的贪婪推理，联合承诺在我们评估的六个基准上提高了准确性，其中在推理和精确答案任务上获得了最大的提升。

View on arXiv Download PDF AI Translation

cs.CL / 66 / 2607.04510

Transplanting, inverting, and preventing a misalignment persona: method-conditional emergent misalignment in Qwen2.5

移植、反转和防止错位人格：Qwen2.5中的方法条件性新兴错位

Drake, Lyndon, Eberstadt, Zandi

Abstract

Emergent misalignment (EM) -- the broad misbehaviour a language model acquires after fine-tuning on narrow harmful data -- is mediated in Qwen2.5 models by a latent persona direction, and that direction is causal in open weights. Transplanting it into a model that shares only pretraining with its source induces broad EM (2.83 +/- 0.26% misaligned against a random-direction floor of ~1.1%), and ablating a model's own direction roughly halves an overt inducer's broadcast (21% to 10%). The transplant doubles as a measurement method, causally assaying directions that a source model represents but cannot itself express. Whether a fine-tune recruits this persona depends on method and capacity, and since low-rank PEFT is the cheaper regime at scale, the recruiting method is also the economical one. On Qwen2.5-32B, low-rank LoRA on insecure code recruits it (3.4% misaligned) while full SFT on identical data does not (0.3%) and moves against the persona axis (drift-persona cosine +0.17 at rank 1 to -0.10), the far-inducer, high-capacity exception consistent with a representational-distance x capacity account. The persona's causal role is itself conditional. Steering a bad-medical SFT run away from the direction during training raises the broadcast from 24% to 51% while a matched random control lowers it, so removing the direction is no blanket recipe. Because recruitment is a loss-reducing shortcut that capacity renders redundant, it can be screened for and prevented in the tested instances. Persona loss-relevance at the SFT solution orders four inducers' broadcasts rank-perfectly within Qwen2.5, inoculation removes recruitment selectively (4.75% to 0.0%, code coherence 65% to 87%), and fine-tuning orthogonal to the single behaviour-derived axis reduces it persona-specifically. Results are a controlled case study of one model family, single-seed in places.

Chinese Translation

新兴错位（Emergent misalignment, EM）——语言模型在针对狭窄有害数据进行微调后所获得的广泛不当行为——在Qwen2.5模型中通过潜在的人格方向进行调节，该方向在开放权重中具有因果关系。将其移植到仅与源模型共享预训练的模型中，会导致广泛的EM（相较于约1.1%的随机方向基线，错位率为2.83 +/- 0.26%），而消除模型自身的方向大约会使显性诱导者的广播减少一半（从21%降至10%）。该移植同时作为一种测量方法，因果性地评估源模型所表示但自身无法表达的方向。微调是否招募该人格取决于方法和能力，而由于低秩PEFT在大规模下是更经济的方案，因此招募方法也是经济的。在Qwen2.5-32B上，低秩LoRA在不安全代码上招募了该人格（错位率为3.4%），而在相同数据上进行完整的SFT则没有（0.3%），并且在与人格轴相反的方向上移动（漂移-人格余弦从1级的+0.17降至-0.10），这是与表征距离与能力的关系一致的远诱导者、高容量例外。人格的因果角色本身是有条件的。在训练期间将不良医疗SFT运行的方向引导偏离，会使广播从24%提高到51%，而匹配的随机对照则降低了广播，因此去除该方向并不是一个通用的解决方案。由于招募是一个减少损失的捷径，而能力使其变得多余，因此在测试实例中可以进行筛选和预防。在Qwen2.5的SFT解决方案中，人格损失相关性使四个诱导者的广播在排名上完美排序，接种选择性地去除了招募（从4.75%降至0.0%，代码一致性从65%升至87%），而与单一行为导向轴正交的微调则特定于人格地减少了招募。结果是对一个模型家族的控制案例研究，在某些地方使用单一种子。

View on arXiv Download PDF AI Translation

cs.CL / 67 / 2607.04515

Towards Digital Preservation of Efik: TTS for a Low-Resource African Language

迈向Efik的数字保存：低资源非洲语言的文本到语音技术

Edet, Offiong Bassey, Oyo-Ita, Emmanuel, Archibong, Archibong Okon, Bassey, David Effanga, Awak, Mbuotidem Sunday

Abstract

Efik, a tonal language spoken by about 3 million second language speakers and 1.5 million native speakers in Southeastern Nigeria, remains underrepresented in speech synthesis research. We present the first documented end-to-end text-to-speech study for Efik, introducing a curated single speaker corpus of 2,632 utterances totaling three hours and a comparative evaluation of four neural models (VITS, MMS-TTS, SpeechT5, and Orpheus-TTS) under low resource conditions. Native speakers evaluated the systems using MOS, Nat-MOS, and A-MOS. MMS-TTS achieved the highest MOS of 3.80 +/- 0.63 and produced more stable long form speech, though tonal errors persisted. Other models showed greater tonal and prosodic inconsistencies. These results provide a reproducible baseline and highlight the need for larger corpora and tone aware modeling for tonal African languages.

Chinese Translation

Efik是一种声调语言，在尼日利亚东南部约有300万名第二语言使用者和150万名母语使用者，但在语音合成研究中仍然代表性不足。我们呈现了首个针对Efik的文献记录的端到端文本到语音研究，介绍了一个经过整理的单一说话者语料库，包含2632个语句，总时长三小时，并在低资源条件下对四种神经模型（VITS、MMS-TTS、SpeechT5和Orpheus-TTS）进行了比较评估。母语者使用MOS、Nat-MOS和A-MOS对系统进行了评估。MMS-TTS获得了最高的MOS评分3.80 +/- 0.63，并产生了更稳定的长篇语音，尽管仍存在声调错误。其他模型则显示出更大的声调和韵律不一致性。这些结果提供了一个可重复的基准，并强调了对声调非洲语言需要更大语料库和声调感知建模的必要性。

View on arXiv Download PDF AI Translation

cs.CL / 68 / 2607.04523

Failures and Successes to Learn a Core Conceptual Distinction from the Statistics of Language

Hu, Zhimin, van Paridon, Jeroen, Lupyan, Gary

Abstract

Generic statements like "tigers are striped" and "cars have radios" communicate information that is, in general, true. However, while the first statement is true in principle, the second is true only statistically. People are exquisitely sensitive to this principled-vs-statistical distinction. It has been argued that this ability to distinguish between something being true by virtue of it being a category member versus being true because of mere statistical regularity, is a general property of people's conceptual machinery and cannot itself be learned. We investigate whether the distinction between principled and statistical properties can be learned from language itself. If so, it raises the possibility that language experience can bootstrap core conceptual distinctions and that it is possible to learn sophisticated causal models directly from language. We find that language models are all sensitive to statistical prevalence, but struggle with representing the principled-vs-statistical distinction controlling for prevalence. Until GPT-4, which succeeds.

View on arXiv Download PDF AI Translation

cs.CL / 69 / 2607.04525

Language Models Represent and Transform Concepts with Shared Geometry

Hu, Zhimin, Niu, Lanhao, Varma, Sashank

Abstract

How concepts are represented in neural networks is a fundamental question in machine learning. The dominant view treats concept representations as stationary geometric objects. Yet concepts appear in context, and context transforms them. Drawing from neural population geometry, we formalize concept representations as point-cloud manifolds and contextual transformations as vector fields, and instantiate this framework in large language models. Across six model families of varying scales, we find that context moves each concept differently. The variance in these displacements is semantically organized, correlating with lexical concreteness and density. Importantly, both the concepts being transformed and this variance structure are shared across models: displacement structure transported from one model predicts held-out displacements in others significantly above chance. Together, these findings show that models share a common geometry not only in how concepts are represented, but more importantly in how context transforms them, a structure with richer organization than prior work has recognized.

View on arXiv Download PDF AI Translation

cs.CL / 70 / 2607.04534

Mechanism-level routing failure in LLMs over Lean-verified algebraic structures

Cázares, Manuel Israel, Zhang, Wenlin, Ma, Haobo

Abstract

We present an empirical study of structural routing failure in large language models (LLMs) over a formally verified algebraic corpus. The task requires selecting the correct proof-mechanism label from a fixed closed template set for compact mathematical objects drawn from the FiberRing formalization in Lean 4, where each item is anchored to a Lean-verified artifact and assigned a label from the corresponding certificate family. Our central finding is a mechanism-level routing ceiling: under blind conditions, gpt-oss-120b achieves 80.3% template accuracy on 22 FiberRing items (n=66; temperature=0, seed=0), while Llama 3.3 70B reaches 68.2%. Exposing a mechanism-bearing Lean verdict/witness cue (Condition A2) raises accuracy to 90.9% and 81.8% -- gaps of +10.6 and +13.6 pp termed cue-induced routing uplift. The dominant failure is a CRT-to-ring-equivalence misroute: gpt-oss-120b misroutes 7 of 12 CRT items (58.3%) blind, zero under A2. A cross-model dissociation in Llama is notable: verdict accuracy is identical in both conditions (95.5%), while template accuracy improves 13.6 pp -- confirming that truth inference and proof-mechanism classification are separable capacities. A cross-corpus extension (Set B; 6 POM/CollisionKernel items, 72 evaluations) provides a small cross-module check: CRT-granularity compression reappears with different labels, and an inverse cross-model dissociation emerges. These findings extend the router hypothesis (Cazares 2026) to formal algebraic structures. The full pipeline, manifest, and results are at https://github.com/bytepro-ai/fiber-routing-eval.

View on arXiv Download PDF AI Translation

cs.CL / 71 / 2607.04558

EEG-SpikeAgent: Agentic Closed-Loop Program Synthesis for Automated EEG Spike Detection

Santhosh, Sonali, Yu, Kelly Shuhong, Chang, Eugene, Kim, Jonathan, Shidara, Kie, Bernardo, Danilo

Abstract

Automated detection of interictal epileptiform discharges in scalp electroencephalography (EEG) is clinically important, but recent high-performing deep-learning models often trade interpretability for accuracy. We introduce EEG-SpikeAgent, a closed-loop program-synthesis framework that uses a large language model (LLM) agentic system to generate signal-processing features for spike detection in scalp EEG. The system iteratively proposes one deterministic EEG feature module at a time, executes the resulting code on EEG to generate tabular features, evaluates performance via a tabular classifier, summarizes run-level metrics, and feeds structured diagnostics back to the model for refinement. Across iterations, EEG-SpikeAgent proposes and refines candidate signal features and decision rules informed by model performance. We evaluated EEG-SpikeAgent on VEPISET, a public 29-channel dataset of 4-second epochs containing 2,516 discharge-containing and 22,933 non-discharge epochs. Across five-fold cross-validation with a gradient-boosted tree classifier, agent-generated features achieved an area under the receiver operating characteristic curve of 0.935, balanced accuracy of 0.699, F1 score of 0.557, sensitivity of 0.401, and specificity of 0.996 at the default operating point. At an operating point with sensitivity 0.80, mean precision was 0.470 and mean specificity was 0.900. Artifact-aware feature generation improved balanced accuracy and F1 score over spike-only feature search. These results indicate that LLM-based program synthesis can automate EEG feature engineering in auditable and inspectable code-driven manner for clinical and methodological review.

View on arXiv Download PDF AI Translation

cs.CL / 72 / 2607.04560

Can temporal article-level credibility signals improve domain-level credibility prediction?

Eldifrawi, Islam, Wang, Shengrui, Trabelsi, Amine

Abstract

Web domain credibility evaluation is vital for combating misinformation. It is conducted by examining factors such as domain type, transparency, and overall reputation. However, assessing the credibility of newly emerging web domains remains challenging since they have no reputation yet. Expert fact-checkers evaluate the credibility of domains by analyzing the content of their articles, including the presence of misinformation, bias, or propaganda. Yet, the ease of large-scale content generation enabled by LLMs has accelerated the creation of new content, rendering manual assessment insufficient and underscoring the need for automated approaches to domain credibility evaluation. In this paper, we introduce our Domain Credibility Evaluation Framework (DCEF), a temporal framework for domain credibility evaluation grounded in expert ratings. DCEF enables us to investigate whether the credibility of web domains can be assessed from their published articles following the workflow of expert fact-checkers, without any prior knowledge of the source domains themselves.

View on arXiv Download PDF AI Translation

cs.CL / 73 / 2607.04563

Fidelity-Diversity Metrics for Text

Wang, Amanda, Manole, Tudor, Bunea, Florentina, Thickstun, John

Abstract

As language modeling technology matures, there is an increasing research focus on the composition and curation of datasets used to train these models. For instance, practitioners commonly seek to augment high-quality datasets with additional text to enhance the performance of models trained on that data. However, informed decisions about data augmentation require more nuanced assessments about data quality. We build on work measuring the precision and recall of generative models to develop a pair of metrics that quantify (1) fidelity, capturing how closely candidate text resembles reference data, and (2) diversity, capturing how well it covers the modes of the reference dataset. Our metrics are based on optimal transport divergence functionals between discrete text summaries. In experiments on M2D2 text datasets, we show that these metrics are able to disentangle a lack of fidelity from a lack of diversity in deficient candidate text. In further experiments, our metrics detect diversity deficits in synthetic GSM8K-style math datasets, which correlate with degradations in downstream accuracy of language models finetuned on this synthetic data.

View on arXiv Download PDF AI Translation

cs.CL / 74 / 2607.04566

Characterizing the Temporal, Emotional, and Social Patterns of Adolescent Substance Use Discussions on Reddit

Hong, Leran, Jin, Lei, Zhu, Jianfeng

Abstract

Adolescence is a critical developmental period marked by heightened emotional sensitivity, social stress, and vulnerability to substance use. However, traditional research methods provide limited access to adolescents' authentic experiences, hindering efforts to develop evidence-based prevention and intervention strategies. Social media provides a unique opportunity to observe adolescents' naturally occurring discussions about substance use, offering valuable insights into their opinions, emotions, and lived experiences that can inform early prevention and intervention strategies. In this study, we analyze large-scale Reddit discussions related to substance use among adolescents between 2018 and 2023. Leveraging hour-by-day temporal analysis, sentiment and emotion classification, and transformer-based topic modeling (BERTopic), we examine the interaction between time, emotion, and semantic content in adolescent substance use discourse. Our findings reveal pronounced weekend and late-night peaks in substance-related discussions, a dominance of negative emotions such as sadness and fear, and distinct semantic topics centered on peer relationships, family conflict, emotional distress, and substance-specific experiences. These findings advance our understanding of adolescent substance use in naturalistic online settings and provide empirical evidence to support the development of more timely, targeted, and evidence-based prevention and intervention strategies.

View on arXiv Download PDF AI Translation

cs.CL / 75 / 2607.04576

Progressive Disclosure for LLM-Maintained Wiki Knowledge Bases: a Preregistered Ablation

Cochran, Theodore O.

Abstract

LLM agents increasingly answer questions against knowledge bases they help maintain. A common intuition holds that progressive disclosure, a compact catalog plus a one-line summary per page so the agent loads only what it needs, should make this cheaper than consulting a large monolithic index. We test that on a real 709-page markdown wiki maintained by an LLM. We retrofit it for progressive disclosure and run a preregistered ablation in which four versions of the corpus differ only in how the agent reaches the content: page bodies are byte-identical across arms, frozen as immutable git tags, so any measured difference is due to access structure alone. We cross the arms with three access conditions (a protocol-constrained agent, a free self-routing agent, and a catalog-preload regime) and grade answers blind against verified gold references with a cross-family judge. A pilot upended the premise: a capable tool-using agent never loads the index, inferring a page's path from the question and reading it directly, so the specific saving the retrofit targets does not materialize. We therefore made answer quality primary and cost secondary. Quality is non-inferior (the retrieval arm matches the index baseline within the preregistered margin) while cost falls in every regime, from about a third for a self-routing agent to well over half under catalog-preload, all confidence intervals excluding zero. The saving comes not from avoiding the index load but from more targeted access: the retrieval arm cites fewer pages and takes fewer tool turns. The study doubles as a case study in evaluation validity, applying threat-to-validity discipline to the tooling that produced it.

View on arXiv Download PDF AI Translation

cs.CL / 76 / 2607.04581

MTEB-PT: A Text Embedding Benchmark for Brazilian Portuguese

Stekel, Tardelli Ronan Coelho

Abstract

Text embeddings for Portuguese have no dedicated benchmark: evaluation rests on translated corpora such as English MS MARCO or on thin multilingual coverage, with native tasks scattered and unconsolidated. We introduce MTEB-PT, a benchmark of 22 native Brazilian-Portuguese tasks across seven categories (classification, multilabel classification, pair classification, semantic textual similarity, clustering, retrieval, and reranking), admitting only data created or found in Portuguese and excluding translations by construction. We evaluate 93 models spanning 23M to 27B parameters: 73 open-weight and 20 closed commercial APIs. Alongside the leaderboard we report a statistical layer for every headline comparison: per-task bootstrap confidence intervals, paired-bootstrap significance, a task- and instance-level discrimination analysis (how sharply each task separates models) adapted from Item Response Theory, and a cross-leaderboard correlation. Three findings stand out. The benchmark cleanly separates about a dozen tiers of models, though the top six are statistically too close to order. An openly licensed, self-hostable model reaches that leading tier, so strong Portuguese embedding quality does not require a commercial API. And a model's rank on the global multilingual leaderboard predicts its Portuguese rank only moderately (Spearman rho = 0.75 over 55 shared models; one model ranks 3rd there and 49th here), so a native benchmark measures something the multilingual boards do not. We release every task, our code, and a public leaderboard, so practitioners can choose Portuguese embedding models on native evidence.

View on arXiv Download PDF AI Translation

cs.CL / 77 / 2607.04640

Wrong Before Right: Late Rescue and Interface Failure in Aligned Language Models

Deng, Jiaqi

Abstract

We study how correctness is assembled inside aligned language models, not only whether the final answer is right. Using layer-wise difference-in-differences (DiD) trajectories over polarity-controlled minimal pairs, we identify the wrong-dip: in mid layers (25-90% depth), internal preference transiently commits to the incorrect answer and is rescued only by late-layer correction. We verify this causally with patchscope-style activation transplantation across 17 models, three families, and 64x scale (0.5B-32B). Four findings follow. (1) Alignment amplification of the causal wrong-dip is recipe-specific and emergent: it emerges at 3B in Qwen2.5, remains high, and peaks at 32B (paired t up to 9.7), reverses in Llama-3-8B (t=-2.31), and sits between for Mistral-7B. (2) The dip predicts real compression failures: high-dip items are 3-7x more likely to flip under late-layer low-rank compression, block dropping, or structured pruning, while quantization flips are dip-blind, a double dissociation confirmed by late-layer ablation. (3) The dip is trainable: a LoRA fine-tune with a mid-layer wrong-margin penalty matches output-only SFT accuracy while cutting the causal dip by 67-70% and improving compression robustness; output-only SFT worsens the causal dip by up to 2.8x at perfect surface accuracy. (4) With controlled readouts, the phenomenon survives natural-language I/O: dip stratification of structural-damage failures is significant on naturalistic vignettes, and free-form fragility separates into a dip-auditable late-rescue layer and a dip-blind interface layer. Together, output-level correctness can hide a late-rescue process that governs compression risk, post-training quality, and evaluation distortion.

View on arXiv Download PDF AI Translation

cs.CL / 78 / 2607.04645

Retroactive Chain-of-Thought (RetroCoT): Forensic Reconstruction Prompts as a Safety Diagnostic Across Model Generations

Hajizadeh, Samira

Abstract

Safety alignment in large language models is typically evaluated against direct, imperative harmful requests. We show that this alignment is highly conditioned on pragmatic register: models that refuse a direct request frequently comply when the same underlying objective is expressed through a different communicative stance. This suggests that current alignment policies are not invariant to semantic equivalence, but remain sensitive to how a request is pragmatically framed. We introduce Retroactive Chain-of-Thought (RetroCoT), a single-turn attack that reframes harmful requests as forensic reconstruction tasks. Rather than requesting harmful instructions directly, RetroCoT presupposes that the harmful outcome has already occurred and asks the model, acting as a forensic analyst, to reconstruct in reverse the causal chain that produced it. On AdvBench (n=50), RetroCoT achieves attach success rate of 58% on gpt-4o and 52% on gpt-4o-mini, compared with direct-request baselines of 0% and 4%, respectively. We further identify a pronounced generation gap: GPT-5-family models refuse RetroCoT entirely, explicitly identifying the reconstruction premise in their refusal rationales, consistent with explicit coverage of this reconstruction register. However, this robustness does not generalize across pragmatic forms. A single adversarial feedback turn presenting an existing forensic reconstruction response alongside evaluator critique raises ASR from 0% to 48% on GPT-5.4-mini and from 58% to 94% on GPT-4o; a control condition omitting the fabricated low score achieves 85% on GPT-5.4-mini, indicating that the operative element is pragmatic continuation within the established forensic frame rather than score manipulation. These results suggest that frontier-model alignment remains conditioned on pragmatic framing rather than semantic intent, and that new pragmatic registers can continue to expose a...

View on arXiv Download PDF AI Translation

cs.CL / 79 / 2607.04655

FormalRx: Rectify and eXamine Semantic Failures in Autoformalization

Wang, Haocheng, Huang, Baiyu, Wan, Yingjia, Zhu, Xiao, Liu, Xiaoyang, Huang, Yinya, Guo, Zhijiang

Abstract

The veracious semantic alignment in autoformalization is significant for formal mathematical reasoning. However, existing evaluations provide only opaque binary verdicts or scalar scores, offering no interpretable insight into where or why translations fail. This opacity severely limits both human understanding and automated system improvement. To bridge this gap, we introduce FormalRx, a comprehensive diagnostic evaluation framework that transforms autoformalization assessment from black-box judgments into actionable feedback. At its core is SCI Error Taxonomy, a hierarchical classification scheme decomposing autoformalization errors into 28 distinct categories with strict priority ordering. Building on this taxonomy, FormalRx provides four critical diagnostic capabilities: alignment verdicts, error categorization, error localization, and correction. We instantiate the framework with a diagnostic model FormalRx-8B, trained on 56,287 NL-FL pairs with fine-grained diagnostic annotations, and release FormalRx-Test as the first fine-grained diagnostic benchmark. FormalRx-8B achieves F1-scores of 0.88 (verdict) and 0.71 (categorization), along with accuracies of 0.75 (localization) and 0.73 (correction), substantially outperforming both general-purpose LLMs and specialized baselines. By connecting evaluation with actionable insights, FormalRx enables systematic diagnosis and improvement of autoformalization systems.

View on arXiv Download PDF AI Translation

cs.CL / 80 / 2607.04686

ToolFailBench: Diagnosing Tool-Use Failures in LLM Agents

Soni, Harsh

Abstract

Tool calling is central to modern language model agents, but aggregate benchmark scores often hide where tool use fails. A model that never calls a needed tool and a model that calls the tool but ignores the result can look similar under final task accuracy. We introduce ToolFailBench, a diagnostic benchmark for measuring tool-use failures across 1,000 tasks in finance, medicine, law, cybersecurity, and real estate. Tool-required tasks return values the model wouldn't guess, forcing it to trust the tool while control tasks attach the same tools but should be answered directly. We label each trace with Tool-Skip, Result-Ignore, Output-Fabrication, and Unnecessary-Tool-Use, using a rule classifier and two LLM judges aggregated by majority vote. Across 19 headline models, the best reaches 86.33% Clean Tool-Use Rate, showing that faithful tool use is not saturated. More importantly, models with similar aggregate scores fail in different ways: most stay disciplined on no-tool controls, while Llama-3.1 models show an Always-Call pattern, and at the same parameter scale Llama-3.1-70B and Qwen2.5-72B differ by 89 percentage points on control-task accuracy. Tool-use evaluation should measure not only whether agents call tools, but whether they use tool outputs correctly and avoid tools when none is needed.

View on arXiv Download PDF AI Translation

cs.CL / 81 / 2607.04690

PAST-TIDE: Prototype-Anchored Statement Tuning with Topic-Invariant Normalization for Stance Detection

Shujon, Md. Shakhoyat Rahman, Jim, MD Jahid Hasan, Islam, Md. Milon, Haque, Md Rezwanul, Karray, Fakhri

Abstract

We introduce PAST-TIDE, our stance detection system addressing both subtasks of the StanceNakba Shared Task at NakbaNLP@LREC-COLING 2026. The main idea is statement tuning. We redefine stance as cloze-style masked language modeling (MLM), letting a verbalizer map label words to stance categories through the pre-trained MLM head rather than appending a randomly initialized classification head. We complement this with prototypical contrastive learning, which uses learnable class prototypes for batch-size independent contrastive training, and topic-conditional layer normalization for cross-topic Arabic stance detection. PAST-TIDE achieves macro-F1 scores of 0.75 for Subtask A and 0.74 for Subtask B on the official leaderboard, indicating that minimal architectural additions to a pre-trained model can remain competitive in low-resource settings.

View on arXiv Download PDF AI Translation

cs.CL / 82 / 2607.04726

What You See Is What You Get: Observation-Aligned Supervision for Chart-to-Code Generation

Niu, Tianhao, Zhu, Qingfu, Che, Wanxiang

Abstract

Chart-to-code generation is commonly trained with supervised fine-tuning on reference plotting scripts, implicitly treating the gold code as a fully observable target. We argue that this assumption is often invalid: many chart programs contain latent raw variables that cannot be uniquely recovered from the rendered image. For example, a boxplot exposes summary statistics rather than original samples, a pie chart reveals proportions rather than arbitrary raw values, and a histogram shows bin-level mass rather than individual observations. Supervising models to reproduce such non-identifiable quantities encourages hallucination and over-specified code generation. We introduce Observation-Aligned supervision, a rewriting framework that replaces latent raw-data targets with quantities constrained by the visual observation: box statistics for boxplots, wedge percentages for pie charts, and bin weights for histograms. Applying this framework to chart-to-code training data from two sources, we obtain the Observation-Aligned supervision target data. Experiments across multiple VLMs on ChartMimic and ChartX demonstrate consistent improvements in observable value recovery, including under both-executable evaluation. Our results suggest that improving chart-to-code models requires not only more data or advanced learning objectives or algorithms, but also supervision targets that respect what is identifiable from the chart image.

View on arXiv Download PDF AI Translation

cs.CL / 83 / 2607.04728

Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment

Li, Yu, Li, Xiuyu, Yi, Mingyang, Wang, Jiaxing, zhangliangxu, Xing, Zhaolong, Chen, Zhen

Abstract

Reinforcement learning (RL) post-training for large language models (LLMs) follows a efficient paradigm of "rollout then update", which inevitably results in off-policy training data. To resolve this, Importance sampling (IS) is proposed, while the token-level ratios compound over long sequences, causing severe variance exploded. A natural idea is "transferring" these off-policy token into on-policy token, so that the importance scores for correction are unnecessary. Following this idea, we propose Selective Importance Sampling (SIS), which is inspired by rejection sampling. Concretely, SIS implements by viewing off-policy model as proposal distribution, and implement a token-level rejection test: accepted tokens are viewed as on-policy, so that receive unit importance score, while rejected tokens retain the standard IS correction. Our proposed SIS is theoretically proved reducing the gap between token-level and sequence-level off-policy gradient estimators. The SIS acts as a plug-in that only modifies the importance ratio in the policy loss, adding negligible wall-clock overhead, and can be combine with a vast vary of RL post-training algorithms. Experiments on dense and MoE LLMs across math and agent benchmarks show that SIS consistently improves all objectives, while providing substantially stronger robustness under off-policy data.

View on arXiv Download PDF AI Translation

cs.CL / 84 / 2607.04733

LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure

Wang, Yueyang, Bi, Baolong, Lu, Shuo, Zhang, Jingyuan

Abstract

Supervised fine-tuning (SFT) is the standard approach for adapting pretrained language models to downstream domains, yet it often improves target-domain behavior at the cost of degrading pre-existing capabilities. Standard cross-entropy fine-tuning promotes only the observed label token and leaves unconstrained how probability mass is redistributed over other plausible alternatives, potentially distorting the rich local preference structure learned during pretraining. We first analyze next-token predictions using Shannon and Renyi entropies, revealing that pretrained models exhibit a regular multimodal entropy structure. These entropy peaks correspond to varying numbers of plausible alternatives, indicating that the base model intrinsically encodes rich distributional knowledge beyond the single supervised token. Motivated by this observation, we propose LP-SFT, a Local-Preserving Supervised Fine-Tuning objective designed to explicitly protect this inherent entropy structure. At each step, LP-SFT constructs an adaptive support of alternative tokens and applies a locally normalized preservation loss to maintain the base model's relative structure among them, while standard cross-entropy independently optimizes the supervised token. Across mixed-domain and single-domain fine-tuning experiments, LP-SFT improves overall performance over vanilla SFT and recent SFT-enhancement baselines, achieving the best balance between pass@1 accuracy and pass@k performance. These results suggest that local preservation helps mitigate capability degradation without collapsing sampling-accessible diversity.

View on arXiv Download PDF AI Translation

cs.CL / 85 / 2607.04814

Evaluating the Effect of Linguistic Relatedness on Cross-Lingual Transfer in Large Multilingual Automatic Speech Recognition

Florian, Andrei, Amol, Cynthia Jayne, Ombaba, Hope Kerubo, Cui, Xiaoyu, Mwau, Boniface, Kamau, Biatus Maina, Wanzare, Lilian Diana Awuor, Fellbaum, Christiane, Buzaaba, Happy

Abstract

Extending automatic speech recognition (ASR) to low-resource African languages is constrained by the prohibitive demands of data collection at scale. A promising direction is to leverage linguistic relatedness to enhance cross-lingual transfer from a related auxiliary language to the low-resource target by sequentially adapting on both. Although this strategy has shown meaningful improvements in small ASR models, its effectiveness in large ASR remains unclear. We extend this framework to large multilingual ASR through a systematic controlled experimental design spanning six factors, two Africa-centric corpora, and four large ASR models, isolating whether linguistic relatedness reliably predicts cross-lingual transfer gains in this setting. Across all conditions, pre-adaptation on related auxiliary languages yields no practically meaningful transfer improvements given minimal target-language data, suggesting that linguistic relatedness alone may not reliably predict cross-lingual transfer gains in large multilingual ASR, or constitute an effective strategy for extending such models to low-resource languages.

View on arXiv Download PDF AI Translation

cs.CL / 86 / 2607.04832

Semantic Homogenization in Italian Popular Music: A Diachronic Analysis

Canale, Lorenzo, Scotta, Stefano, Messina, Alberto

Abstract

In recent years, studies have revealed a decline in semantic variety across popular music lyrics, particularly in English-language songs on streaming platforms like Spotify. This research examines whether a similar trend can be observed in a different linguistic and cultural context: the lyrics of all finalist songs from the 75 editions of the Sanremo Music Festival, Italy's most renowned music competition. What sets this work apart is the development of a flexible and efficient methodology for tracking changes in semantic similarity over time, which can be applied to different datasets to study similar phenomena. Drawing on a combination of full-text, segment-based, topic-based, and word-level analyses, the approach leverages both embedding techniques and large language models. When applied to the Sanremo corpus, this framework reveals a gradual move toward increasing semantic uniformity, echoing the global patterns identified in previous studies. These findings underscore the value of natural language processing tools in uncovering long-term shifts in musical language and cultural expression.

View on arXiv Download PDF AI Translation

cs.CL / 87 / 2607.04890

Evaluating Large Language Models for Antisemitic Incident Classification

评估大型语言模型在反犹太事件分类中的应用

Halevy, Karina, Mendelsohn, Julia, Park, Chan Young, Tsvetkov, Yulia, Sap, Maarten

Abstract

Addressing hate and violence in society requires timely detection of hateful events from public reporting, but automated identification of hateful events remains underexplored. We introduce the task of hateful event detection and investigate the ability of AI systems, specifically large language models (LLMs), to discover and classify reports of antisemitic events with fine-grained labels. We evaluate OpenAI's GPT-4o and Meta's Llama-3.2-3B-Instruct on multiple expert-annotated datasets containing antisemitic event descriptions from news articles, civil society reports, and official records. We show that LLMs, particularly GPT-4o, have potential for this task, but substantial improvement is needed. Providing clear term definitions and in-context examples in prompts can improve performance: definitions are most helpful for rhetoric-oriented events (e.g. classical antisemitic tropes), while examples help label action-oriented events (e.g. physical assault). A case study of college newspapers demonstrates that LLMs can help surface relevant real-world events, supporting early monitoring and intervention. Overall, our findings highlight both opportunities and critical gaps in AI's ability to recognize complex harms and underscore the need for collaborative efforts among AI developers, policymakers, and civil society to design models, implement robust evaluation, and develop policy frameworks for defining and combating hate efficiently and effectively.

Chinese Translation

应对社会中的仇恨和暴力需要及时检测公共报告中的仇恨事件，但自动识别仇恨事件仍然未得到充分探索。我们引入了仇恨事件检测的任务，并调查人工智能系统，特别是大型语言模型（LLMs），在发现和分类反犹太事件报告方面的能力，使用细粒度标签。我们在多个专家注释的数据集上评估了OpenAI的GPT-4o和Meta的Llama-3.2-3B-Instruct，这些数据集包含来自新闻文章、民间社会报告和官方记录的反犹太事件描述。我们的研究表明，LLMs，特别是GPT-4o，在这一任务中具有潜力，但仍需显著改进。在提示中提供清晰的术语定义和上下文示例可以提高性能：定义对以修辞为导向的事件（例如经典的反犹太主义陈词滥调）最为有帮助，而示例则有助于标记以行动为导向的事件（例如身体攻击）。一项关于大学报纸的案例研究表明，LLMs可以帮助挖掘相关的现实世界事件，支持早期监测和干预。总体而言，我们的研究结果突显了人工智能在识别复杂危害方面的机遇和关键缺口，并强调了人工智能开发者、政策制定者和民间社会之间合作的必要性，以设计模型、实施稳健评估，并制定有效和高效的政策框架来定义和打击仇恨。

View on arXiv Download PDF AI Translation

cs.CL / 88 / 2607.04895

Ossetic-COT: Designing a morphologically annotated corpus and morphological analyzer for Ossetic

Shatskikh, Anna, Sorokin, Alexey

Abstract

In this work we present the first morphologically annotated corpus for Iron Ossetic that conforms to the Universal Dependencies schema. The corpus includes 5454 manually annotated sentences from the Iron Ossetic Corpus of Oral Texts, containing 74032 tokens. We use this corpus to train a BERT-based morphological analyzer. The analyzer achieves tag accuracy of 95.60%.

View on arXiv Download PDF AI Translation

cs.CL / 89 / 2607.04941

DuplexChat: Constructing Speaker-Separated Full-Duplex Dialogue Speech at Scale for Spoken Dialogue Language Modeling

Nakata, Wataru, Saito, Yuki, Saruwatari, Hiroshi

Abstract

Full-duplex spoken dialogue models are trained on conversational speech in which each speaker is represented as a separate stream, but existing large-scale public speech corpora are mostly monaural, making them unsuited for SDLM training. We present DuplexChat, an open-source corpus for full-duplex spoken dialogue models, and DuplexChat-Pipe, a pipeline for constructing speaker-separated full-duplex dialogue speech from public podcast feeds. DuplexChat-Pipe filters language-specific podcast feeds, retrieves and cleans episode audio, extracts diarization-guided two-speaker dialogue clips, and applies speech separation and restoration to produce one channel per speaker. Running this pipeline yields a speaker-separated spoken dialogue corpus covering 282,634 hours of English and 132,723 hours of Japanese. Analysis results on DuplexChat show that it contains turn-taking dynamics present in human dialogues.

View on arXiv Download PDF AI Translation

cs.CL / 90 / 2607.04945

You Frame It: How Conceptual Representations Shape LLM Detection and Reasoning about Antisemitism

Soemer, Katharina, Mihaljević, Helena

Abstract

LLMs enable the integration of external conceptual resources at inference time, creating new opportunities for detecting ideologically and historically complex phenomena such as antisemitism. We investigate how different forms of conceptual grounding affect antisemitism detection and explanation behavior across four state-of-the-art LLMs. Using two expert-annotated datasets, we compare definitional, fine-grained taxonomic, example-augmented, and large-context representations of antisemitism. We find that fine-grained taxonomic representations substantially improve recall, while simultaneously reducing precision. Surprisingly, supplying substantially larger conceptual resources yields no additional quantitative benefit. Post-Holocaust antisemitism poses the most persistent challenge across models and configurations. Analysis of explanations further reveals systematic limitations including overproduction of conceptual references, reliance on lexical cues, overconfidence, and difficulties with subtle or justificatory forms of antisemitism. Our findings highlight both the potential and the remaining limitations of conceptually grounded LLMs for antisemitism detection and reasoning.

View on arXiv Download PDF AI Translation

cs.CL / 91 / 2607.04962

Who's Behind It? Annotating and Extracting Conspiratorial Actors from German Telegram Posts

谁在背后？从德国Telegram帖子中注释和提取阴谋论参与者

Mihaljević, Helena, Beer, Jolanda, Lisker, Mareike, Soemer, Katharina

Abstract

Conspiracy theories commonly attribute important events to the actions of powerful and secretive actors. While computational research has largely focused on document-level analyses of conspiracy theories, less attention has been paid to identifying the actors that drive such narratives. We develop annotation guidelines for conspiratorial actors, present a span-annotated corpus of German Telegram posts, and investigate their automatic extraction using transformer-based models. We further apply the resulting model to the \textit{Schwurbelarchiv}, a large-scale archive of German conspiracy-related Telegram channels. Our results demonstrate that conspiratorial actors can be annotated with meaningful agreement and extracted with reasonable accuracy despite the linguistic complexity of conspiracy discourse, enabling large-scale analyses of actor representations in conspiracy narratives.

Chinese Translation

阴谋论通常将重要事件归因于强大而神秘的参与者的行为。尽管计算研究在很大程度上集中于阴谋论的文档级分析，但对推动这些叙事的参与者的识别关注较少。我们制定了阴谋论参与者的注释指南，呈现了一个经过跨度注释的德国Telegram帖子语料库，并研究了使用基于变换器的模型进行自动提取。我们进一步将所得到的模型应用于 extit{Schwurbelarchiv}，这是一个大规模的德国阴谋相关Telegram频道档案。我们的结果表明，尽管阴谋话语的语言复杂性，阴谋论参与者可以以有意义的协议进行注释，并以合理的准确性提取，从而使对阴谋叙事中参与者表现的大规模分析成为可能。

View on arXiv Download PDF AI Translation

cs.CL / 92 / 2607.04986

The syntax of wh-agreement in Yemeni Ibbi Arabic

也门伊比阿拉伯语中的疑问词一致性句法

Naji, Ashraf, Shormani, Mohammed Q.

Abstract

This article tackles an important phenomenon in the syntax of Yemeni Ibbi Arabic (YIA), viz., wh-agreement, a phenomenon common to several languages including Greek, Indonesian, Lubukusu, Irish, etc. In YIA, wh-agreement manifests itself via agreement inflections on the Wh-Op, C, T/V, v. To account for this phenomenon, we propose an Agree across phases (AAP) approach anchored in the mechanism of Feature Inheritance (FI) in which Agree as MATCHING (AM) is a bit separated from feature valuation (FV). AM concerns Cs/vs, but FV Ts/Vs. Analyzing the agreement patterns observed between Wh-Op(erators), functional heads (precisely C, (T), v), and verbal complexes, we argue that the suffixes -eh, -uh, -nen, -um, having undergone grammaticalization process from Stannard Arabic (SA) third person pronouns, function as morphological marking of wh-agreement. Findings indicate that YIA data offer a unique empirical contribution to generative syntax, specifically concerning wh-agreement in this dialect operating via MATCHING mechanism. Our proposal straightforwardly accounts for wh-agreement cross-linguistically. This study provides further evidence that incorporating under-investigated typology provides further support for the universality of Universal Grammar (UG) by revealing how specific I-language operations reflect deeper, invariant principles of human language architecture. It concludes that the wh-agreement mechanism in YIA is more morphosyntactically robust than in languages such as Greek, Indonesian, Palauan, and Irish, providing compelling evidence for AAP as a UG approach to long-distance dependencies.

Chinese Translation

本文探讨了也门伊比阿拉伯语（YIA）句法中的一个重要现象，即疑问词一致性（wh-agreement），这一现象在包括希腊语、印尼语、卢布库苏语、爱尔兰语等多种语言中普遍存在。在YIA中，疑问词一致性通过Wh-Op、C、T/V和v上的一致性屈折形式表现出来。为了解释这一现象，我们提出了一种跨阶段一致性（Agree across phases, AAP）的方法，该方法基于特征继承（Feature Inheritance, FI）机制，其中一致性作为匹配（MATCHING, AM）与特征估值（feature valuation, FV）略有分离。AM涉及Cs/vs，而FV则涉及Ts/Vs。通过分析在Wh-Op（操作符）、功能头（具体为C、(T)、v）和动词复合体之间观察到的一致性模式，我们认为后缀-eh、-uh、-nen、-um经过从斯坦纳阿拉伯语（Stannard Arabic, SA）第三人称代词的语法化过程，作为疑问词一致性的形态标记。研究结果表明，YIA数据为生成句法提供了独特的实证贡献，特别是在该方言中通过匹配机制运作的疑问词一致性方面。我们的提议直接解释了跨语言的疑问词一致性。该研究进一步证明，纳入尚未充分研究的类型学为普遍语法（Universal Grammar, UG）的普遍性提供了更多支持，揭示了特定的I语言操作如何反映人类语言结构的更深层次、不变原则。最后得出结论，YIA中的疑问词一致性机制在形态句法上比希腊语、印尼语、帕劳语和爱尔兰语等语言更为强健，为AAP作为UG处理长距离依赖关系提供了有力证据。

View on arXiv Download PDF AI Translation

cs.CL / 93 / 2607.05013

Knowledge Knows, Verbalization Tells: Disentangling Latent Directions for Mathematical Solvability in LLMs

知识知晓，言语表达：解开大语言模型中数学可解性的潜在方向

Xiros, Nikolaos, Zoumpoulidi, Maria-Eleni, Paraskevopoulos, Georgios

Abstract

Although LLMs have made significant progress in mathematical reasoning, determining whether a mathematical problem is solvable remains a fundamental yet challenging capability. While recent studies have probed internal representations of model solvability beliefs, verbalization has primarily been studied behaviorally rather than as an internal representation, limiting its analysis and manipulation. We address this gap by separately probing representations of solvability knowledge and verbalization, allowing us to disentangle the two within model hidden states. Across multiple LLMs, we show that knowledge and verbalization are encoded as distinct, linearly decodable representations and that fabrication is primarily associated with changes in verbalization rather than the underlying knowledge. Prompting with unsolvability cues reduces fabrication primarily by shifting verbalization, while activation steering demonstrates that these representations can be echanistically manipulated to improve model abstention.

Chinese Translation

尽管大语言模型（LLMs）在数学推理方面取得了显著进展，但确定一个数学问题是否可解仍然是一个基本而具有挑战性的能力。尽管最近的研究探讨了模型可解性信念的内部表征，但言语表达主要是从行为角度进行研究，而非作为内部表征，这限制了其分析和操控。我们通过分别探讨可解性知识和言语表达的表征，填补了这一空白，从而使我们能够在模型的隐藏状态中解开二者之间的关系。在多个大语言模型中，我们展示了知识和言语表达被编码为不同的、线性可解的表征，并且虚构主要与言语表达的变化相关，而非基础知识。使用不可解提示进行提示主要通过转变言语表达来减少虚构，而激活引导则表明这些表征可以在机制上被操控，以改善模型的回避能力。

View on arXiv Download PDF AI Translation

cs.CL / 94 / 2607.05032

Multi-Large Language Model Orchestrated Severity Assessment of Clinical Records (MOSAIC)

多大型语言模型协同临床记录严重性评估（MOSAIC）

Suero, Manuela Del Castillo, Vermillet, Arnault-Quentin, Heckmann, Nicole Sonne, Ramcharran, Darmendra, Sessa, Maurizio

Abstract

Background: Disease severity is a multidimensional construct difficult to capture with rule-based approaches in Electronic Healthcare Records (EHR). Agentic large language model (LLM) systems could synthesise clinical evidence and reason over EHRs, but remain unevaluated for this task. Methods: MOSAIC is a two-phase agentic LLM framework for severity phenotyping, using type 2 diabetes (T2D) as a proof-of-concept. MOSAIC was evaluated on a synthetic cohort (SyntheticMass; open-weight N = 4,886; closed-weight N = 200) against three algorithmic ground truths (DCSI, DiSSCo, Cooper) and against all-cause mortality and incident complications. Open-weight (locally deployable) and proprietary pipelines were also compared. Results: The generated framework spanned domains absent from the comparators, including biomarker-based glycaemic staging, beta-cell function, and social determinants of health. Open-weight MOSAIC matched the proprietary pipeline (closed- vs open-weight weighted kappa = 0.773) and reached moderate agreement with Cooper (kappa = 0.597) and DCSI (kappa = 0.534) and fair agreement with DiSSCo (kappa = 0.320). Agent-based (Type 1) tiers showed significant separation of all-cause mortality (log-rank p < 0.001; crude hazard ratios 1.6-2.4 for non-Baseline tiers), with non-monotonic separation at the upper tiers, and an inverse gradient for incident complications (log-rank p < 0.001) consistent with depletion of susceptibles. Agentic classification also diverged from deterministic execution of the same rubric (MOSAIC Frozen; kappa = 0.428), indicating reasoning beyond fixed rules. Conclusion: MOSAIC shows agentic LLM systems can generate and apply clinically meaningful severity phenotypes from structured EHR data in T2D. Extending it to other diseases with similarly multidimensional severity warrants further research.

Chinese Translation

背景：疾病严重性是一个多维构念，难以通过基于规则的方法在电子健康记录（EHR）中捕捉。代理大型语言模型（LLM）系统能够综合临床证据并对EHR进行推理，但尚未对此任务进行评估。方法：MOSAIC是一个两阶段的代理LLM框架，用于严重性表型分析，以2型糖尿病（T2D）作为概念验证。MOSAIC在一个合成队列（SyntheticMass；开放权重 N = 4,886；封闭权重 N = 200）上进行了评估，比较了三种算法的真实值（DCSI、DiSSCo、Cooper），以及全因死亡率和事件并发症。还比较了开放权重（可本地部署）和专有管道。结果：生成的框架涵盖了比较对象中缺失的领域，包括基于生物标志物的血糖分期、β细胞功能和健康的社会决定因素。开放权重的MOSAIC与专有管道相匹配（封闭与开放权重加权kappa = 0.773），并与Cooper（kappa = 0.597）和DCSI（kappa = 0.534）达成中等一致性，与DiSSCo（kappa = 0.320）达成公平一致性。基于代理（类型1）的层次在全因死亡率上显示出显著分离（log-rank p < 0.001；非基线层次的粗危险比为1.6-2.4），在上层次表现出非单调分离，并且事件并发症呈现出逆梯度（log-rank p < 0.001），与易感者的耗竭一致。代理分类也与相同规则的确定性执行（MOSAIC Frozen；kappa = 0.428）存在差异，表明推理超越了固定规则。结论：MOSAIC表明代理LLM系统能够从结构化的EHR数据中生成和应用具有临床意义的严重性表型，适用于T2D。将其扩展到其他具有类似多维严重性的疾病需要进一步研究。

View on arXiv Download PDF AI Translation

cs.CL / 95 / 2607.05052

Beyond Independent Labels: Schwartz-Geometry Decoding for Human Value Detection

超越独立标签：用于人类价值检测的施瓦茨几何解码

Yeste, Víctor, Rosso, Paolo

Abstract

Human value detection is commonly formulated as sentence-level multi-label classification over the 19 refined Schwartz values, typically predicted as independent labels. Schwartz theory, however, describes them as a circular motivational continuum, in which adjacent values are compatible and opposing values are in tension. We ask whether this structure can be operationalized as an explicit output-space geometry and used as a soft bias rather than a hard constraint. On a DeBERTa-v3-base classifier, we compare two ways of injecting it: training-time geometry-aware objectives and a post-hoc Schwartz-aware energy decoder that scores whole label sets jointly. Across five seeds, training-time geometry gives only limited gains-no larger for the true continuum than for a random ordering-whereas the decoder makes label sets more coherent with the continuum-on theory-aware coherence metrics we introduce-at no cost to Macro-F1 or Micro-F1 (held fixed by its selection rule). The gain is specific to the true Schwartz ordering: it does not appear for a random permutation or an empirical co-occurrence graph through the identical decoder. A bounded Qwen2.5-72B-Instruct diagnostic shows that supplying the continuum at inference shifts behavior but does not match supervised structured prediction. Theory-aware decoding thus offers a lightweight, controllable way to make value detection faithful to its label space.

Chinese Translation

人类价值检测通常被表述为对19种精炼施瓦茨价值的句子级多标签分类，通常将其预测为独立标签。然而，施瓦茨理论将这些价值描述为一个循环的动机连续体，其中相邻的价值是兼容的，而对立的价值则存在张力。我们探讨这种结构是否可以作为显式输出空间几何进行操作，并用作软偏置而非硬约束。在DeBERTa-v3-base分类器上，我们比较了两种注入方式：训练时几何感知目标和事后施瓦茨感知能量解码器，该解码器共同评分整个标签集。在五个种子上，训练时几何仅带来有限的收益——对于真实的连续体与随机排序没有显著差异——而解码器使标签集在我们引入的理论感知一致性度量上与连续体更加一致，且对宏观F1和微观F1没有成本（由其选择规则保持固定）。这种收益特定于真实的施瓦茨排序：在随机排列或通过相同解码器的经验共现图中并未出现。一个有限的Qwen2.5-72B-Instruct诊断显示，在推理时提供连续体会改变行为，但并不匹配监督结构化预测。因此，理论感知解码提供了一种轻量、可控的方式，使价值检测忠实于其标签空间。

View on arXiv Download PDF AI Translation

cs.CL / 96 / 2607.05069

MIRAGE: Defending Long-Form RAG Against Misinformation Pollution

MIRAGE：防御长文本检索增强生成中的虚假信息污染

Eletter, Saadeldine, Zeng, Ruihong, Wang, Yuxia, Panov, Maxim, Rubashevskii, Aleksandr, Nakov, Preslav

Abstract

Retrieval-Augmented Generation (RAG) improves factuality by grounding LLMs in external evidence, but real-world retrieval is often polluted: semantically relevant passages may contain subtle misinformation, misleading framings, or fabrications. We introduce MIRAGE, a training-free, model-agnostic defense for long-form RAG. MIRAGE builds an NLI-based cross-document claim graph and applies a Defended-Claims Gate to either condition generation on a consistent, multi-source supported subset or to block retrieval and answer parametrically. We also release a minimal-edit pollution protocol spanning four perturbation families (Unambiguous, Conflicting, Misleading, Fabricated) to construct matched clean, mixed, and fully polluted evaluation regimes. Across four long-form QA benchmarks and multiple commercial and open-weight LLMs, pollution severely degrades vanilla RAG, while MIRAGE consistently restores factuality under mixed and fully polluted evidence and outperforms prior robust-RAG methods. Our implementation and datasets are available at https://github.com/SaadElDine/MIRAGE.

Chinese Translation

检索增强生成（RAG）通过将大型语言模型（LLMs）与外部证据结合，提升了事实准确性，但现实世界中的检索往往受到污染：语义相关的段落可能包含微妙的虚假信息、误导性的框架或虚构内容。我们提出了MIRAGE，一种无训练、模型无关的长文本RAG防御方法。MIRAGE构建了一个基于自然语言推理（NLI）的跨文档声明图，并应用了防御声明门（Defended-Claims Gate），以便在一致的、多源支持的子集上条件生成，或以参数化方式阻止检索和回答。我们还发布了一种最小编辑污染协议，涵盖四种扰动类型（明确的、冲突的、误导性的、虚构的），以构建匹配的干净、混合和完全污染的评估机制。在四个长文本问答基准和多个商业及开放权重的LLMs上，污染严重降低了普通RAG的性能，而MIRAGE在混合和完全污染的证据下始终恢复了事实准确性，并超越了之前的稳健RAG方法。我们的实现和数据集可在 https://github.com/SaadElDine/MIRAGE 获取。

View on arXiv Download PDF AI Translation

cs.CL / 97 / 2607.05113

Rating the Pitch, Not the Product: User Evaluations of LLMs Reflect Expectations More Than Performance

评价推介，而非产品：用户对大型语言模型的评估反映了期望而非性能

Morabito, Robert, McDonald, Tyler, Viswanath, Charitra, Hwang, Angel Hsing-Chi, Gaube, Susanne, Kabbara, Jad, Emami, Ali

Abstract

Imagine two users interact with the same LLM. One has been told it is the cutting-edge flagship model; the other, an older, weaker model. They walk away with markedly different ratings of its usefulness and intelligence, yet they used the same model. In a controlled study, 162 participants each used one of six LLMs from two families across three collaborative tasks, after first viewing a landing page that matched, overstated, or understated their model's true capability. This pre-interaction framing shifted user opinions and interaction behavior while task performance did not. Oversold users rated the model more favorably and used more directive prompting, while Undersold users wrote longer, more collaborative prompts. The quality of what users and the model produced together depended only on the model's true capability, not on what users were told. Participants' change in model impressions after use, measured across two impression measures, was not predicted by task performance ($\beta = -0.01$ and $0.11$, both n.s.), but by whether the model met users' expectations ($\beta = 0.47$ and $0.50$, both $p < .001$) and how confident they felt working with it ($\beta = 0.47$ and $0.36$, both $p < .001$). After interaction, users are still rating the pitch, not the product: user-elicited LLM evaluations, including the preference data driving public leaderboards, measure expectation management at least as much as the model itself.

Chinese Translation

设想两个用户与同一个大型语言模型（LLM）互动。一个用户被告知这是最先进的旗舰模型；另一个则被告知这是一个较旧、性能较弱的模型。他们对模型的有用性和智能性给出了明显不同的评分，但实际上使用的是同一个模型。在一项控制研究中，162名参与者在完成三项协作任务之前，首先查看了一个与其模型真实能力相匹配、夸大或低估的着陆页面。这样的前期互动框架改变了用户的看法和互动行为，而任务表现并没有变化。被夸大的用户对模型的评价更加积极，并使用了更多的指令性提示，而被低估的用户则写出了更长、更具协作性的提示。用户与模型共同生成的内容质量仅依赖于模型的真实能力，而与用户所被告知的内容无关。参与者在使用后对模型印象的变化，通过两个印象测量指标进行评估，并未被任务表现预测（$eta = -0.01$ 和 $0.11$，均不显著），而是由模型是否满足用户期望（$eta = 0.47$ 和 $0.50$，均 $p < .001$）以及用户在使用过程中感到的自信程度（$eta = 0.47$ 和 $0.36$，均 $p < .001$）所预测。互动后，用户仍在评价推介，而非产品：用户引导的LLM评估，包括推动公共排行榜的偏好数据，至少在期望管理方面与模型本身同样重要。

View on arXiv Download PDF AI Translation

cs.CL / 98 / 2607.05155

EdgeBench: Unveiling Scaling Laws of Learning from Real-World Environments

EdgeBench：揭示来自真实环境学习的规模法则

Zhu, Deyao, Zhou, Xin, Qin, Shengling, Zhu, Xuekai, Ding, Hangliang, Zhong, Shu, Wen, Zixin, Xie, Zhonglin, Gou, Chenhui, Ren, Linxuan, Wang, Yueyang, Zhong, Junfeng, Liu, Rui, Gao, Tian, Lin, Yangguang, Zhang, Jingyuan, Song, Maojia, Qi, Xuan, Wu, Jinhong, Zhang, Chenyang, Piao, Yinzhu, Niu, Ziru, Lin, Hongbin, Meng, Lingxiang, Tang, Peng, Tang, Chengyao, Wu, Shanyu, Zheng, Huanyu, Liu, Yu, Zhu, Liya, Wang, He, Ding, Ming, Wan, Ziyu, Liu, Hao, Wang, Sibo, Zhu, Haotian, Zhang, Xintian, Chai, Nan, Liu, Yipeng, Lai, Panhao, Yuan, Sihang, Su, Zixin, Zhang, Ge, Zhou, Wangchunshu, Du, Yantao, Huang, Wenhao, Shi, Guang

Abstract

Pretraining scaling laws reveal that model capability improves predictably with data and compute. But learning from real world environments after deployment remains far less understood. Analyzing roughly 38,000 hours of agent interaction with the environment across 134 real world tasks, we find, to the best of our knowledge, the first evidence that overall performance during environment learning follows a log-sigmoid scaling law with remarkably high precision, reaching R^2 = 0.998. Across model generations, we also find that agent learning speed roughly doubles every three months. This discovery stems from EdgeBench, a suite of 134 real world tasks with ultra-long horizons, spanning scientific discovery, software engineering, combinatorial optimization, professional knowledge work, formal mathematics, and interactive games. Each task sustains at least 12 hours of continuous agent operation under rich, multilevel feedback, and is built through substantial expert effort. We publicly release 51 tasks and our full evaluation framework to accelerate the study of how agents learn from real world experience.

Chinese Translation

预训练规模法则表明，模型能力随着数据和计算资源的增加而可预测地提升。然而，部署后从真实环境中学习的过程仍然不够清楚。通过分析约38,000小时的代理与环境的交互，涵盖134个真实世界任务，我们首次发现环境学习期间整体性能遵循一种对数- sigmoid（log-sigmoid）规模法则，且具有极高的精度，达到R^2 = 0.998。在模型代际之间，我们还发现代理的学习速度大约每三个月翻倍。这一发现源于EdgeBench，这是一个包含134个真实世界任务的套件，任务具有超长的时间跨度，涵盖科学发现、软件工程、组合优化、专业知识工作、形式数学和互动游戏。每个任务在丰富的多层次反馈下至少支持12小时的连续代理操作，并且经过大量专家的努力构建。我们公开发布了51个任务及我们的完整评估框架，以加速研究代理如何从真实世界经验中学习。

View on arXiv Download PDF AI Translation

cs.CL / 99 / 2607.05171

RABBiT: Rapidly adaptive BOLD foundation model via brain-tuning for accurate zero-shot and few-shot prediction of speech-elicited responses in the brain

RABBiT：通过脑调优的快速适应性BOLD基础模型，用于准确的零-shot和少-shot语音引发反应预测

Moussa, Omer, Toneva, Mariya

Abstract

Language understanding in the brain is context-dependent, varying across experimental stimuli and individuals, which makes it difficult to build computational models that generalize across both. This calls for a foundation model of language-evoked brain activity that can capture shared structure while adapting efficiently to new participants and inputs. We introduce RABBiT (Rapidly Adaptive BOLD foundation model via BraIn-Tuning), a compact audio-to-fMRI encoder designed for accurate zero- and few-shot prediction. A comprehensive evaluation on 324 participants across multiple unseen fMRI datasets shows that RABBiT enables accurate zero-shot prediction of fMRI responses to natural speech across auditory and language-selective regions, surpassing the SOTA foundation model for fMRI and predictions based on group averages. With as little as 10 minutes of participant-specific data, RABBiT further improves performance via parameter-efficient tuning, substantially outperforming per-participant linear models. RABBiT's performance is driven by two key innovations: (1) learned region-specific attention, and (2) a decomposition of brain responses into shared and subject-specific components, combined with a brain-tuned speech backbone. In addition to supporting strong predictive accuracy, the structured, region-specific representations that RABBiT learns enable interpretability. By eliminating the need for extensive per-participant data and model fitting, RABBiT enables scalable population-level analyses of language in the human brain. We make the code available at https://github.com/bridge-ai-neuro/rabbit.

Chinese Translation

大脑中的语言理解是依赖于上下文的，因实验刺激和个体的不同而有所变化，这使得构建能够跨越这两者进行泛化的计算模型变得困难。这就需要一个能够捕捉共享结构并有效适应新参与者和输入的语言引发大脑活动的基础模型。我们提出了RABBiT（通过脑调优的快速适应性BOLD基础模型），这是一种紧凑的音频到fMRI编码器，旨在实现准确的零-shot和少-shot预测。在324名参与者的多个未见fMRI数据集上的全面评估表明，RABBiT能够准确预测自然语音在听觉和语言选择性区域的fMRI反应，超越了当前fMRI的SOTA基础模型和基于组平均的预测。仅需10分钟的参与者特定数据，RABBiT通过参数高效的调优进一步提升了性能，显著优于每位参与者的线性模型。RABBiT的性能得益于两个关键创新：(1) 学习的区域特定注意力，和(2) 将大脑反应分解为共享和个体特定成分，并结合脑调优的语音骨干网络。除了支持强大的预测准确性外，RABBiT学习的结构化、区域特定的表示还增强了可解释性。通过消除对广泛的每位参与者数据和模型拟合的需求，RABBiT使得对人脑中语言的可扩展群体水平分析成为可能。我们已在https://github.com/bridge-ai-neuro/rabbit上公开了代码。

View on arXiv Download PDF AI Translation

cs.CL / 100 / 2607.05196

Unified Audio Intelligence Without Regressing on Text Intelligence

统一音频智能而不降低文本智能

Kong, Zhifeng, Lee, Sang-gil, Kim, Jaehyeon, Wang, Boxin, Liu, Zihan, Kim, Sungwon, Chen, Yang, Goel, Arushi, Roy, Rajarshi, Dai, Wenliang, Yang, Zhuolin, Chen, Yangyi, Jiang, Dongfu, Ghosh, Sreyan, Rintamaki, Tuomas, Tao, Andrew, Raiman, Jonathan, Shoeybi, Mohammad, Catanzaro, Bryan, Ping, Wei

Abstract

Audio intelligence involves understanding, reasoning about, and generating both audio and speech. In this work, we introduce Nemotron-Labs-Audex-30B-A3B (Audex), a unified audio-text LLM built on Nemotron-Cascade-2-30B-A3B, a strong text-only MoE LLM. Audex adopts a simple unified design with a single Transformer decoder: audio inputs are encoded and projected into the text embedding space, while text tokens and quantized audio output tokens are treated uniformly during generation. This architecture enables strong audio-text fusion, seamless multimodal generation, and compatibility with standard LLM training and inference infrastructure. For training, we meticulously curate audio-text datasets comprising 157.4B audio tokens and 320.5B text tokens. We apply multi-stage supervised training on these datasets, followed by text-only Cascade RL and multi-domain on-policy distillation. Audex delivers state-of-the-art audio understanding, speech recognition and translation, text-to-speech, audio generation, and speech-to-speech generation, while preserving very compelling reasoning, alignment, knowledge, long-context, and agentic capabilities of its text-only LLM backbone with marginal or no regression. We release the model checkpoints to facilitate open research.

Chinese Translation

音频智能涉及理解、推理和生成音频和语音。在本研究中，我们介绍了 Nemotron-Labs-Audex-30B-A3B（Audex），这是一个基于 Nemotron-Cascade-2-30B-A3B 的统一音频-文本大语言模型（LLM），后者是一个强大的文本专用 MoE LLM。Audex 采用简单的统一设计，使用单个 Transformer 解码器：音频输入被编码并投影到文本嵌入空间，而文本标记和量化的音频输出标记在生成过程中被统一处理。这种架构实现了强大的音频-文本融合、无缝的多模态生成，并与标准 LLM 训练和推理基础设施兼容。为了训练，我们精心策划了包含 1574 亿音频标记和 3205 亿文本标记的音频-文本数据集。我们在这些数据集上应用了多阶段的监督训练，随后进行文本专用的 Cascade 强化学习和多领域的在线蒸馏。Audex 在音频理解、语音识别与翻译、文本转语音、音频生成以及语音转语音生成方面提供了最先进的性能，同时保持了其文本专用 LLM 主干的推理、对齐、知识、长上下文和代理能力，几乎没有回退或完全没有回退。我们发布模型检查点以促进开放研究。

View on arXiv Download PDF AI Translation

cs.CL / 101 / 2607.05224

Progressive Refinement: An Iterative Pseudo-Labeling Approach for Mandarin-English Code-Switching ASR

渐进细化：一种迭代伪标签方法用于普通话-英语代码切换自动语音识别

Yang, Qu, Wardhana, Cakra, Ng, Tim

Abstract

Code-switching (CS), alternating languages within the same utterance, poses significant challenges for automatic speech recognition (ASR) due to limited CS training data. This paper applies an iterative pseudo-labeling training approach to CS-ASR for the first time, demonstrating its effectiveness in leveraging unlabeled data to improve CS-ASR performance. The approach comprises three phases: pseudo-label generation, two-stage bilingual model training, and iterative improvements. It begins by generating pseudo-labels from a large unlabeled corpus, creating a semi-supervised dataset. This dataset supports a two-stage training framework where the model is pre-trained and then fine-tuned on supervised CS data. Iterative refinements further enhance the model's accuracy in handling complex CS scenarios. Our approach significantly advances CS-ASR systems, achieving notable Mix Error Rate (MER) reductions on SEAME's devman (6.35%) and devsge (8.29%) subsets.

Chinese Translation

代码切换（CS）是在同一话语中交替使用语言，这对自动语音识别（ASR）带来了重大挑战，因为可用的代码切换训练数据有限。本文首次将迭代伪标签训练方法应用于代码切换自动语音识别，展示了其在利用未标记数据以提高代码切换自动语音识别性能方面的有效性。该方法包括三个阶段：伪标签生成、双语模型的两阶段训练以及迭代改进。首先，从一个大型未标记语料库中生成伪标签，创建一个半监督数据集。该数据集支持一个两阶段的训练框架，其中模型先进行预训练，然后在监督的代码切换数据上进行微调。迭代细化进一步提高了模型在处理复杂代码切换场景中的准确性。我们的方法显著推动了代码切换自动语音识别系统的发展，在SEAME的devman（6.35%）和devsge（8.29%）子集上实现了显著的混合错误率（MER）降低。

View on arXiv Download PDF AI Translation

cs.CL / 102 / 2607.05250

Streaming Neural Speech Codecs through Time-Invariant Representations

通过时间不变表示的流式神经语音编解码器

Estève, Kélian, Mhdaffar, Salima, Rouvier, Mickael, Dufour, Richard, Estève, Yannick

Abstract

Neural speech codecs are increasingly used as intermediate representations in codec-based speech generation systems. TiCodec introduces a factorized representation that separates time-varying speech content from time-invariant information through a Time-Invariant Representation Extraction (TIRE) module, potentially reducing the amount of information that must be modeled at the frame-level. In this work, we investigate the nature of the information captured by TIRE representations and their suitability for low-latency speech processing. Using a series of probing tasks, we analyze the influence of the encoder layer and show that intermediate layers capture complementary speaker- and environment-related information while containing little linguistic content. We further study several segment selection strategies for TIRE training and demonstrate that cross-file sampling improves the robustness of invariant representations. Based on these findings, we propose Dual-TIRE, a multi-level architecture that exploits the complementarity of different encoder layers and improves speech reconstruction quality and speaker similarity. Finally, we evaluate TiCodec in a streaming inference setting using successive 660ms processing blocks. Results show that streaming operation can be achieved without significant degradation in reconstruction performance, highlighting the potential of factorized neural codec representations for future low-latency speech generation systems.

Chinese Translation

神经语音编解码器越来越多地作为基于编解码器的语音生成系统中的中间表示。TiCodec 引入了一种因式分解的表示，利用时间不变表示提取（Time-Invariant Representation Extraction, TIRE）模块将时变语音内容与时间不变信息分离，从而有可能减少在帧级别上必须建模的信息量。在本研究中，我们探讨了 TIRE 表示所捕获信息的性质及其在低延迟语音处理中的适用性。通过一系列探测任务，我们分析了编码器层的影响，并展示了中间层捕获了互补的说话者和环境相关信息，同时包含很少的语言内容。我们进一步研究了几种 TIRE 训练的片段选择策略，并证明跨文件采样提高了不变表示的鲁棒性。基于这些发现，我们提出了 Dual-TIRE，一种多层架构，利用不同编码器层的互补性，提高语音重建质量和说话者相似性。最后，我们在流式推理环境中评估了 TiCodec，使用连续的 660 毫秒处理块。结果表明，流式操作可以在重建性能没有显著下降的情况下实现，突显了因式分解神经编解码器表示在未来低延迟语音生成系统中的潜力。

View on arXiv Download PDF AI Translation

cs.CL / 103 / 2607.05259

SalAngaBhava: A Sinhala Market Dataset for Aspect-based Sentiment Analysis

SalAngaBhava：用于基于方面的情感分析的僧伽罗市场数据集

Galwatta, Lakshani, de Silva, Nisansa, Aththanayake, Sarangi, Galwatta, Adithya

Abstract

Sentiment analysis has been a primary domain under Natural Language Processing (NLP) from its inception as it plays a vital role in both real-world and research applications. In high-resource languages, this has been extended a step further, and instead of predicting sentiment at the sentence level, models have been developed to detect more fine-grained sentiments at aspect level. However, in order to conduct this fine-grained Aspect-based Sentiment Analysis (ABSA), datasets annotated with aspects and sentiments toward the said aspects is required. Such datasets are lacking for low-resources languages among which, we can count Sinhala, an Indo-Aryan languages used primarily in Sri Lanka. In this work, we introduce, SalAngaBhava, a new Sinhala Aspect-based Sentiment Analysis dataset which contains Sinhala product reviews that are manually labeled with aspect terms and the associated sentiments (positive, negative, neutral). The data was collected from domain-relevant sources such as user-generated reviews and comments, and was annotated following carefully defined guidelines to ensure consistency and quality. The dataset consists of sentences and aspect-sentiment pairs, encompassing a considerable range of aspects from several domains. The analysis confirms that the dataset is well-structured and sufficiently balanced for ABSA research. This dataset can be used as a benchmark and facilitates further studies related to Sinhala natural language processing, and low-resource sentiment analysis tasks.

Chinese Translation

情感分析自自然语言处理（NLP）诞生以来一直是其主要领域，因为它在现实世界和研究应用中都发挥着至关重要的作用。在高资源语言中，这一领域已进一步发展，模型不仅在句子层面预测情感，还被开发用于检测更细粒度的方面层面情感。然而，为了进行这种细粒度的基于方面的情感分析（ABSA），需要带有方面和针对这些方面的情感标注的数据集。对于低资源语言而言，这类数据集仍然匮乏，其中包括僧伽罗语，这是一种主要在斯里兰卡使用的印欧-雅利安语言。在本研究中，我们介绍了SalAngaBhava，这是一个新的僧伽罗语基于方面的情感分析数据集，包含手动标注了方面术语及其相关情感（积极、消极、中立）的僧伽罗产品评论。数据收集自与领域相关的来源，如用户生成的评论和反馈，并遵循精心定义的指南进行标注，以确保一致性和质量。该数据集由句子和方面-情感对组成，涵盖了多个领域的相当广泛的方面。分析确认该数据集结构良好，且在ABSA研究中具有足够的平衡性。该数据集可作为基准，并促进与僧伽罗自然语言处理及低资源情感分析任务相关的进一步研究。

View on arXiv Download PDF AI Translation

cs.CL / 104 / 2607.05316

How Much is Left? LLMs Linearly Encode Their Remaining Output Length

剩余多少？大语言模型线性编码其剩余输出长度

Merzouk, Mohamed Amine, Carpov, Dmitri, Bronzi, Mirko, Fornasiere, Damiano, Oberman, Adam

Abstract

Large language models generate one token at a time, yet their responses show remarkably consistent length structure: step-by-step solutions converge in predictable token counts, retrievals stop after a few sentences, retractions extend responses by measurable amounts. We ask whether the model carries an internal estimate of how much response remains. Training minimal-capacity linear probes on frozen hidden states of three open-weight 7-8B models across seven completion-style datasets, we find three converging pieces of evidence. First, total response length is linearly decodable from the prompt's last hidden state alone, before any output is emitted. Second, probe directions trained on natural-language datasets transfer broadly, including to controlled synthetic completions never seen in training, outperforming a statistical baseline; the converse direction generally fails, and this asymmetry is itself informative. Third, on curated high-loss completions, the probe's per-position estimate shifts upward at the moment the model retracts and restarts a partial solution, a directional behavior no position-only predictor can reproduce (qualitative, not aggregate). We frame this as approximate estimation of remaining generation length, distinct from exact-counting impossibility results for transformers, and interpret it as evidence that LLMs maintain a plan-like internal representation of output length (decodable, not necessarily used causally).

Chinese Translation

大型语言模型一次生成一个标记，但它们的响应显示出显著一致的长度结构：逐步解决方案在可预测的标记数量上收敛，检索在几句话后停止，撤回则以可测量的方式延长响应。我们探讨模型是否具备对剩余响应长度的内部估计。在对三个开放权重的7-8B模型的冻结隐藏状态进行最小容量线性探测器训练时，涵盖七个完成风格的数据集，我们发现了三组收敛的证据。首先，总响应长度可以仅通过提示的最后隐藏状态线性解码，甚至在任何输出产生之前。其次，在自然语言数据集上训练的探测器方向广泛转移，包括对在训练中从未见过的受控合成完成，且其表现优于统计基线；而反向方向通常失败，这种不对称性本身也具有信息量。第三，在经过精心挑选的高损失完成中，探测器的每个位置估计在模型撤回并重新开始部分解决方案的瞬间向上移动，这种方向性行为是任何仅基于位置的预测器无法重现的（定性，而非聚合）。我们将其视为对剩余生成长度的近似估计，这与变换器的精确计数不可能结果不同，并将其解释为证据，表明大型语言模型维持一种类似计划的内部输出长度表示（可解码，但不一定因果使用）。

View on arXiv Download PDF AI Translation

cs.CL / 105 / 2607.05355

Faithfulness to Refusal: A Causal Audit of Neuron Selectors

拒绝的忠实性：神经元选择器的因果审计

Eswar, Ananth, Seth, Pratinav, Avaiya, Utsav, Sankarapu, Vinay Kumar

Abstract

Attribution scores increasingly identify which neuron rows of a language model matter for applications such as pruning, interpretability, and editing for safety, yet whether they identify causally important rows is rarely tested directly. We address this with two paired audits built on one-shot neuron-row zeroing. We first audit selectors at the language-modeling level: attribution methods substantially outperform activation and magnitude-based baselines at identifying dispensable rows across five LLMs. We then adapt the same intervention into a behavior test by driving it with a contrastive harmful-versus-benign signal; the attributed rows are sufficient to install refusal on hate and crime while keeping benign over-refusal low and preserving language model fluency, and specific in that layer-matched random controls at the same depths fail. Highly rank-stable selectors can be among the least causally valid. Refusal moreover lives in a redundant subspace, where different attribution methods install it through largely disjoint row sets, so the recovered edit is one realization of a sufficient set rather than a unique mechanism. Together, these findings show that rank-stability proxies miss the kinds of selector failures a direct causal audit can surface.%

Chinese Translation

归因分数越来越多地识别出语言模型中哪些神经元行对修剪、可解释性和安全编辑等应用至关重要，但这些分数是否识别出因果上重要的行却很少被直接测试。我们通过基于一次性神经元行归零的两个配对审计来解决这一问题。我们首先在语言建模层面审计选择器：在识别五个大型语言模型（LLMs）中可有可无的行方面，归因方法显著优于基于激活和幅度的基线。然后，我们将相同的干预措施适应为行为测试，通过对比有害与良性信号来驱动它；归因的行足以在保持良性拒绝低且保持语言模型流畅性的同时，对仇恨和犯罪进行拒绝，并且在同一深度的层匹配随机控制中失败。高度稳定的选择器可能是因果有效性最低的。此外，拒绝存在于一个冗余的子空间中，不同的归因方法通过大致不相交的行集来实现它，因此恢复的编辑是一个充分集的一个实现，而不是唯一机制。总的来说，这些发现表明，排名稳定性代理错过了直接因果审计可以揭示的选择器失败类型。

View on arXiv Download PDF AI Translation

cs.CL / 106 / 2607.05364

REDDIT: Correcting Model-Generated Timestamp Drift in ASR without Forgetting via Replay-Based Distribution Editing

REDDIT：通过基于重放的分布编辑在不遗忘的情况下纠正ASR中模型生成的时间戳漂移

Chou, Cheng-Kang, Chuang, Ming-To, Lu, Ke-Han, Hsu, Chan-Jan, Lee, Hung-yi

Abstract

Modern autoregressive ASR systems can emit timestamps as decoded tokens, enabling timestamped transcription without frame-level aligners or inference-time post-processing. We show that these generated timestamps can drift across long non-speech spans: the transcript may remain plausible, but the decoded time axis drifts away from the audio. We study this non-speech-induced timestamp drift with self-built gap and long-gap benchmarks across 15 evaluated timestamp-producing ASR and audio-language systems. Naive timestamp-corrected fine-tuning improves alignment but can severely degrade non-target ASR behavior, exposing a forgetting problem. We propose REDDIT(REplay-based Distribution eDITing), a lightweight two-stage post-training framework that corrects timestamps while avoiding this catastrophic forgetting: it first edits timestamp targets under the model's own replayed decoder context while matching the frozen base distribution on non-timestamp tokens, then applies a short edited-prefix refinement stage. In this framework, we construct correction supervision without human transcripts or human timestamp annotations by combining VAD-trimmed speech spans with inserted non-speech gaps and known concatenation offsets. On Whisper-tiny, 34.9 hours of targeted correction audio used and only 1.6% of model parameters updated, raising long-gap mIoU from 38.7% to 95.0% and reducing mixed-gap out-of-domain AAS from 2752 ms to 223 ms while preserving CV-en MER at 41.3% (versus 524.2% for ordinary SFT decoder tuning).

Chinese Translation

现代自回归自动语音识别（ASR）系统可以将时间戳作为解码的标记输出，从而实现无帧级对齐器或推理时后处理的时间戳转录。我们发现这些生成的时间戳在长时间的非语音间隔中可能会发生漂移：转录可能仍然合理，但解码的时间轴却与音频偏离。我们通过自建的间隔和长间隔基准，研究了这种非语音引起的时间戳漂移，评估了15个生成时间戳的ASR和音频语言系统。简单的时间戳校正微调虽然改善了对齐，但可能严重降低非目标ASR行为，暴露出遗忘问题。我们提出了REDDIT（基于重放的分布编辑），这是一种轻量级的两阶段后训练框架，能够在避免这种灾难性遗忘的同时纠正时间戳：首先在模型自身重放的解码器上下文中编辑时间戳目标，同时匹配非时间戳标记的冻结基础分布，然后应用短编辑前缀的精炼阶段。在该框架中，我们通过将VAD修剪的语音段与插入的非语音间隔和已知的拼接偏移结合，构建了无需人工转录或人工时间戳注释的校正监督。在Whisper-tiny上，使用了34.9小时的目标校正音频，仅更新了1.6%的模型参数，使长间隔的mIoU从38.7%提高到95.0%，同时将混合间隔的域外AAS从2752毫秒降低到223毫秒，同时保持CV-en MER在41.3%（相比之下，普通的SFT解码器调优为524.2%）。

View on arXiv Download PDF AI Translation

cs.CL / 107 / 2607.05365

SPEARBench: A Benchmark for Naturalness Evaluation in Streaming Speech-to-Speech Language Models

SPEARBench：流式语音到语音语言模型自然性评估基准

Thebaud, Thomas, Wang, Yuzhe, Zhang, Hao, Ugandhar, Sathvik Manikantan Napa, Hallur, Ashish, Tinchev, Georgi, Ravichandran, Venkatesh, Moro-Velazquez, Laureano

Abstract

Streaming speech-to-speech language models aim to answer spoken queries directly with synthetic speech. However, standard speech and text benchmarks do not capture whether these systems behave naturally in conversations, where timing, turn-taking, prosody, interpersonal stance, language and dialect consistency, and relationship-aware appropriateness jointly shape perceived quality. We introduce SPEARBench, a benchmark for evaluating naturalness in speech-to-speech language models from question-answer interactions. SPEARBench constructs controlled dialogue prompts from the Seamless Interaction corpus, runs inference across multiple models, and evaluates generated answers using a multidimensional protocol that covers response latency, interruptions, speech quality, ASR robustness, language and dialect consistency, emotional naturalness, interpersonal stance, and explainable distributional baselines. The benchmark includes original human answers as a reference condition and reports results for several contemporary models. Results show that current models can achieve high signal-level quality and low ASR error while still differing from human conversational behavior in latency, overlap, dialect preservation, emotional adaptation, and interpersonal stance dynamics.

Chinese Translation

流式语音到语音语言模型旨在直接用合成语音回答口语查询。然而，标准的语音和文本基准并未捕捉到这些系统在对话中是否表现自然，而对话中的时机、轮流发言、韵律、人际立场、语言和方言一致性以及关系意识的适当性共同塑造了感知质量。我们引入了SPEARBench，这是一个用于评估语音到语音语言模型在问答互动中自然性的基准。SPEARBench从无缝互动语料库构建了受控对话提示，针对多个模型进行推理，并使用涵盖响应延迟、打断、语音质量、自动语音识别（ASR）鲁棒性、语言和方言一致性、情感自然性、人际立场以及可解释的分布基线的多维协议评估生成的答案。该基准包括原始人类答案作为参考条件，并报告了多个当代模型的结果。结果表明，当前模型可以在信号级质量和低ASR错误方面取得高水平的表现，但在延迟、重叠、方言保留、情感适应和人际立场动态方面仍与人类对话行为存在差异。

View on arXiv Download PDF AI Translation

arXiv Papers

GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation

EVA-Client: A Unified Data Collection, Inference, and Deployment Framework for Embodied Policies on Real Robots

MorphQuad: Morphable Quadrotor for Superhuman Maneuverability, Manipulation, and Resiliency

Metallic Ultrasound Waveguides as a Distributed Tactile Sensing Platform for Contact Localization, Force Estimation, and Material Class Discrimination

TACO: TActile World Model as a Self-COrrector forScalable VLA Post-Training

CLEAR: Closed-Loop Reinforcement Learning at Scale for End-to-End Autonomous Driving

Differential Amplifier-Inspired AmpAttention for Multi-View Robotic Manipulation

DREAMSTEER: Latent World Models Can Steer VLA Policies During Deployment Without Any Finetuning

Continuous-Time Gaussian Belief Trees for Motion Planning

Longitudinal-Motion-Aware Lateral Control for Autonomous Vehicles: A Robust Nonlinear Control Framework

Function-Space Diffusion for Motion Planning

Beyond Heuristics: A Standardized Real2Sim Pipeline for Physical Human Robot Interaction in Human-in-the-Loop Simulation

DRBA: Dynamic Robotic Balance Assistant -- An assist-as-needed gait and balance rehabilitation robot for versatile training

Exp2VLA: Enabling Vision-Language-Action for Drone Navigation from Expert Demonstrations

Hope for the Best, Prepare for the Worst: Occlusion-Aware Contingency Planning for Autonomous Vehicles

Beyond Point-Attached Semantics: Object-Centric Semantic Fields for Generalizable Manipulation

AnchorVLA: Bridging Discrete Decisions and Continuous Trajectories for Vision-Language-Action Planning

Layout-independent actuation allocator for fin-actuated marine robots

Derivations of Error-State Kalman Filter Kinematics for Globally Applicable Aided Inertial Navigation Systems

Strouhal-Aware Model Predictive Control for Efficient Multi-Fin Flapping Locomotion

GDPR-Aware Trajectory Sharing for ISAC-Assisted Robot Navigation: A Case Study on FID-Constrained Collision Prediction

Invisible Strings: Deriving Puppetry Principles and their Hidden Connections to Robot Behavior Design

U-Joint CAAMS: Experimental Evaluation of a Universal-Joint Continuum Manipulator for Aerial Manipulation

Closed-loop vs. Open-loop Kalman Filter Architectures in Airborne Aided Inertial Navigation

Feeling the Unexpected: ResTacVLA for Contact-Rich Manipulation via Residual Tactile Representation

HiMe: Hierarchical Embodied Memory for Long-Horizon Vision-Language-Action Control

ADP: Adversarial Dynamics Priors for Physically Grounded Humanoid Locomotion

High-Precision Formation Control for Heterogeneous Multi-Robot Systems via Hierarchical Hybrid Physics-Informed Deep Reinforcement Learning

Current as Touch: Proprioceptive Contact Feedback for Compliant Dexterous Manipulation

CoorGrasp: Coordinated Contact Control for Adaptive Dexterous Grasping Under Uncertainty

Cross-Embodiment Robot Manipulation via a Unified Hand Action Space

ROBOCYCLE: Autonomous Dual-Arm Robotic Manipulation and Coordination for Recycling Applications

CoRE-VLA: Towards Scalable and Robust Vision-Language-Action Modeling via Conditional Routing of Experts

Lost in Time? Continuous Symmetry and Identifiability in Aided Inertial Navigation with Unknown Measurement Delays

OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies

Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models

Occluding the Solution Space: Planner-Agnostic Adversarial Attacks on Tolerance-Aware Manipulation

From Region Arrival to Instance-Level Grounding in Vision-and-Language Navigation

ObjRetarget: An Object-Aware Motion Retargeting Framework with Anthropomorphic Arm Constraints and Polyhedral Hand Modeling

High-Fidelity One-Step Generative Visuomotor Policy via Recursive Correction, Frequency Consistency, and Contrastive Flow Matching

LH-AVLN: A Benchmark for Long-Horizon Audio-Visual-Language Navigation

WSA$_1$: a 3D-Centric World-Spatial-Action Model for Generalizable Robot Control

Worldscape-MoE: A Unified Mixture-of-Experts World Model for Scalable Heterogeneous Action Control

Fast Asymptotically Optimal Kinodynamic Planning via Vectorization

Conflict-Based Lazy Search for Fast Multi-Manipulator Planning

Semantic-Guided Progressive Object Removal with Gaussian Splatting

!Imperio, smolVLA: The Implications of Data Poisoning on Open Source Robotics

ACE: Agentic Control for Embodied Manipulation via Zero-shot Workflow Reasoning

Neural LiDAR Bundle Adjustment

XS-VLA: Coupling Coarse-grained Spatial Distillation with Latent Flow Matching for Lightweight Robotic Control

SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects

Integrated Graph Search and Model Predictive Control for Smooth and Efficient Path Planning in Autonomous Vehicles

FLOAT Drone for Physical Interaction: Lateral Airflow Reduction, Wrench Modeling, and Adaptive Control

HALO-WA: Hybrid-Attention Latent-Guided Online Reinforcement Learning for World-Action Models

GPU-Accelerated Polygonal Signed Distance Functions for Real-Time Collision Avoidance

Agent-driven Long-tail Simulation for Autonomous Driving

A Perception-Manipulation Robotics System for Food Cutting

SurgAM: Surgical Affordance Map Prediction with Multimodal Feature Fusion for Robot Autonomy

ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI

RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies

Robustness Verification of an Autonomous Underwater Vehicle-based Plankton Classifier

Mask2Real-WM: Segmentation Masks as a Sim-to-Real Bridge for Controllable Dexterous World Models

HUGS: Guiding Unified Dexterous Grasp Synthesis Across Modes and Scales via Learned Human Priors

Simple-to-Complex Structured Demonstrations for Vision-Language-Action Learning

SEAM: Smooth Execution of Action-Chunked Motion for Vision-Language-Action Policies

RoboVista: Evaluating Vision Language Models for Diverse Robot Applications

DIVO: Continuous-time DVL-Inertial-Visual Odometry for Unmanned Underwater Vehicles

SILO: Simulation-in-the-Loop Sim-to-Real Transfer for Multi-Stage Cable Routing

Governed Caste Reassignment in Heterogeneous Swarms: An Asymmetric-Trust Protocol with Audited Operator Countersignature

KAM-WM: Kinematic Affordance Maps from Latent World Models for Robot Manipulation

Do Vision-Language-Action Models Mean What They Say? On the Role of Faithfulness in Embodied Reasoning

A Reliable Context-Aware and Temporal Planning Framework for Autonomous Driving

Geometry-Aware Motion Latents for Learning Robust Manipulation Policies

Aerial Manipulation: Contact, Medium Coupling, and the Geometry of Readiness

Spatial Attention: Adapting Execution Horizons for Diffusion Policies via Observation Sensitivity

Trajectory-Anchor Optimization for Overconfident Thermal Visual Place Recognition: Zero-Leakage OOD Auditing and Kidnapped-Robot Recovery

SLAM: Structured and Localized Analytic Manifold Adaptation for Lifelong VPR

CAC-VLA: Context-Gated Action Conditioning for Vision-Language-Action Models

Athena-WBC: Capability-Aligned Policy Experts for Long-Tail Humanoid Whole-Body Control