Personalized Daily Arxiv Papers 10/20/2026
This project is adapted from tatsu-lab/gpt_paper_assistant. The source code of this project is at Variante/gpt_paper_assistant
About me on Bilibili. Help keep the website running:
Topics
Paper selection prompt and criteria (jump to the section by clicking the link):
1. Application of diffusion models and vision-language models (VLMs) to robot manipulation.
3. Video segmentation using unsupervised or self-supervised methods.
5. Advances in 3D generation using generative models, including image-to-3D and text-to-3D.
7. Agentic frameworks and environments for robot learning.
Topic 1
1000. ALIGN: Aligning Condition Estimation and Expression in Advantage-Conditioned Behavior Cloning [more]
Authors: Zechen Wang, Zhenqi Qiu, Houlong Xiong, …, Ran Cheng, Qian Zhu
1001. PrismWAM: Decoupling State Representations in World Action Models [more]
Authors: Tianyu Xin, Wanqi Zhong, Bowen Hu, …, Zhiding Yu, Yiming Li
1002. HAF-VLA: Hierarchical Action Flow Framework for Humanoid Vision-Language-Action Control [more]
Authors: Langzhe Gu, Chengkai Hou, Meng Li, …, Shanghang Zhang, Zhengping Che
1004. PICA-WAM: Visual Perturbation Invariance via Cross-Appearance Self-Alignment in World Action Models [more]
Authors: Yude Zou, Yifei Yao, Guanbo Jiao, …, Guanjie Zheng, Xing Gao
1005. GPT-6 Astra Is a Native Visuo-Tactile Policy for Contact-Rich Manipulation [more]
Authors: Chen Yang, Linzhe Shi, Hang Zhang, …, Jiansheng Fan, Chen Wang
1006. Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience [more]
Authors: Tianqi Liu, Hanfei Zhao, Qifeng Liang, …, Zhaoxi Chen, Ziwei Liu
1007. SPAR: Structural Pruning and Action Recovery for Vision-Language-Action Models [more]
Authors: Shenli Zheng, Zhou Fang, Yuan Du, Yijiang Liu, Li Du
1008. Learn the Task, Not the Schedule: Reducing Spurious Cross-Arm Coupling in Bimanual Imitation Learning [more]
Authors: Shuyuan Hu, Deng-Ke Wu, Yifan Jin, …, Kai Ye, Bo Zhao
1011. Robot Learning with Vision-Language-Action Models for Agricultural Manipulation [more]
Authors: Hamid Hafezi, Javad Mohammadpour Velni, Mohammadreza Davoodi
1014. Physical Words: A Vocabulary for Structuring Latent Reasoning in Robot Manipulation [more]
Authors: Jiaming Liu, Xinghao Zhu, Yiming Zhang, …, Juekun Li, Yan Chang
Back to [top]
Topic 2
Back to [top]
Topic 3
Back to [top]
Topic 4
Back to [top]
Topic 5
5009. Drive2Gauss: FeedForward 4D Gaussian Reconstruction from Multimodal Generative Driving Videos [more]
Authors: Tong Zhao, Zhiyuan Han, Lin Chen, …, Hao Zhao, Chaojian Li
5013. InfiniDream: Open-Ended Photorealistic 3D Worlds Through Procedural Generation [more]
Authors: Yi-Chuan Huang, Hao-Jen Chien, Zheng-Hui Huang, …, Kaipeng Zhang, Zhixiang Wang
Back to [top]
Topic 6
6010. EgoXperience: Learning World-Grounded Egocentric Representations from Human Experience [more]
Authors: Xinhao Ji, Hui En Pang, Hao Li, …, Zhaoxi Chen, Ziwei Liu
Back to [top]
Topic 7
7003. HandsOn: A Large-Scale Simulation Benchmark for Generalizable Dexterous Manipulation [more]
Authors: Tianwei Luo, Yiming He, Hongjin Wu, …, Haozhan Li, Ning Ding
7012. Where is Physical RSI capability from? An Observational Scaling Law of LLM-as-policy [more]
Authors: Zekai Li, Jiayi Yu, Jinlu Zhang, …, He Wang, Li Yi
Back to [top]
Go beyond
Back to [top]
Full paper list
1000. ALIGN: Aligning Condition Estimation and Expression in Advantage-Conditioned Behavior Cloning
ArXiv: 2610.26455 [page] [pdf]
Authors: Zechen Wang, Zhenqi Qiu, Houlong Xiong, …, Ran Cheng, Qian Zhu
Abstract: Human-collected robot manipulation data often mix successful behavior with retries, pauses, recoveries, and failures. Advantage-Conditioned Behavior Cloning (AC-BC) can exploit such mixed-quality data, but only if the conditioning signal is both accurately constructed from offline trajectories and effectively incorporated into the policy’s action-vector-field prediction. Existing pipelines can weaken this signal by thresholding estimator-derived advantage scores into noisy conditions, and by routing conditions through prompts that weakly affect the action vector field. We introduce ALIGN, a condition-aligned AC-BC framework for vision-language-action policies. ALIGN directly predicts categorical pairwise conditions in the policy-facing label space and injects them into a DiT action expert through AdaRMS modulation fused with flow time. We evaluate ALIGN on deliberately mixed real-robot datasets for ARX X5 bowl stacking and Franka key hanging, each combining standard demonstrations with retry-heavy trajectories. ALIGN improves condition fidelity, strengthens conditional vector-field separability, and achieves 90.0% and 56.7% closed-loop success, compared with 56.7% and 33.3% for the strongest AC-BC baseline. These results indicate that aligning conditions from estimation to action prediction is an effective way to extract robust manipulation policies from data in which success, recovery and error coexist.
Comment: Matches criterion 1: ALIGN improves vision-language-action policy learning through categorical advantage conditioning and AdaRMS modulation of a DiT action expert, achieving 90.0% success on real-robot bowl stacking and 56.7% on key hanging.
Relevance: 10 Back to [topic] [top]
1001. PrismWAM: Decoupling State Representations in World Action Models
ArXiv: 2610.26564 [page] [pdf]
Authors: Tianyu Xin, Wanqi Zhong, Bowen Hu, …, Zhiding Yu, Yiming Li
Abstract: World Action Models (WAMs) typically represent the world through latent embeddings from pretrained video models, which entangle heterogeneous aspects of the world, from how it looks (e.g., texture and color) to how it is physically configured (e.g., occupancy and pose). This raises an underexplored question: $\textbf{what should constitute the state of a WAM?}$ We argue that the world state should separate how the world appears from how the world evolves under action: the former captures semantic context, while the latter is governed by physical state. To this end, we propose $\textbf{PrismWAM}$, which decouples physics from semantics while jointly modeling state and action. PrismWAM factorizes an embodied task into a $\mathrm{Semantic\ Planning}\rightarrow\mathrm{Physical\ Rollout}\rightarrow\mathrm{Executable\ Action}$ pipeline: given only the initial RGB observation, semantic planning learns to predict the final goal image, while physical rollout predicts the fine-grained evolution toward this goal in depth space; executable actions are then generated from the depth rollout. Without any embodied action pretraining or additional data augmentation, PrismWAM achieves strong manipulation performance in both simulation and the real world. Notably, PrismWAM even surpasses a broad range of models with large-scale embodied action pretraining (e.g., OpenWAM-$α$). More broadly, we reframe World Action Models from modeling how the world looks to how it responds to action, paving the way for action-centric world models grounded in occupancy, flow, tactile sensing, proprioception, and beyond.
Comment: Criterion 1: PrismWAM separates semantic goal-image planning from depth-space physical rollouts and executable action generation, demonstrating strong simulated and real-world robot manipulation without embodied-action pretraining.
Relevance: 10 Back to [topic] [top]
1002. HAF-VLA: Hierarchical Action Flow Framework for Humanoid Vision-Language-Action Control
ArXiv: 2610.26584 [page] [pdf]
Authors: Langzhe Gu, Chengkai Hou, Meng Li, …, Shanghang Zhang, Zhengping Che
Abstract: Humanoid robots hold great promise as general-purpose agents in human-centered environments, yet adapting generalist vision-language-action (VLA) policies to real-world humanoid loco-manipulation remains challenging. Unlike fixed-base or wheeled manipulators, humanoids must coordinate locomotion, posture, gaze, and dual-arm manipulation over a high-dimensional whole-body action space, where inaccurate base or waist motions can easily induce unstable and aggressive upper-body compensation. We propose HAF-VLA, a hierarchical action-flow framework for humanoid VLA control. Built upon a pretrained flow-matching VLA, HAF-VLA inherits the vision-language prefix encoder and action expert weights, and adapts them to humanoid whole-body loco-manipulation with learnable stage embeddings and cross-stage KV-cache conditioning. Instead of generating the full action chunk in a single pass, HAF-VLA progressively denoises a shared whole-body action space: first locomotion and head commands, then waist refinement, and finally full-body manipulation. To preserve structural dependencies across stages, we introduce a cross-stage KV cache conditioning mechanism, enabling subsequent action generation stages to refine and augment their outputs based on the action representations from preceding stages. Experimental results across six real-world tasks demonstrate that our framework significantly enhances the performance of existing models in humanoid loco-manipulation tasks, demonstrating an effective pathway for adapting strong mobile-manipulation VLAs to humanoid whole-body loco-manipulation.
Comment: Criterion 1: HAF-VLA adapts a pretrained flow-matching VLA through hierarchical whole-body action denoising and cross-stage KV-cache conditioning, improving humanoid loco-manipulation across six real-world tasks.
Relevance: 10 Back to [topic] [top]
1004. PICA-WAM: Visual Perturbation Invariance via Cross-Appearance Self-Alignment in World Action Models
ArXiv: 2610.26813 [page] [pdf]
Authors: Yude Zou, Yifei Yao, Guanbo Jiao, …, Guanjie Zheng, Xing Gao
Abstract: World Action Models (WAMs) jointly predict future video and generate actions, yet remain sensitive to task-irrelevant visual perturbations, i.e., changes in visual appearance that should not affect robot behavior. We identify a gap in conventional WAM training: video prediction encourages representations to preserve appearance information, while action generation does not explicitly enforce invariance to task-irrelevant appearance. Data augmentation exposes WAMs to such perturbations but does not require perturbed and nominal observations to produce consistent action representations, and alignment with external pretrained features does not guarantee the desired invariance. To address this gap, we propose PICA-WAM, a self-supervised training framework that casts perturbation invariance as an auxiliary cross-appearance self-alignment task for WAMs. An exponential-moving-average (EMA) teacher processes the nominal observation to provide a deeper-layer target, while the online WAM processes a perturbed observation and aligns its shallower-layer action representation to this target through a lightweight projection head. Across four strong WAMs evaluated on RoboTwin 2.0, LIBERO-Plus, EBench, and a real robot, PICA-WAM consistently improves robustness under visual perturbations, preserves nominal performance in simulation, and raises the real robot success rate. PICA-WAM further reduces the sensitivity of video features to task-irrelevant appearance and its propagation into actions, as shown by paired layer-wise diagnostics. Code: https://github.com/YudeZou/PICA-WAM-release
Comment: Criterion 1: PICA-WAM improves robot World Action Models through EMA-teacher cross-appearance self-alignment, demonstrating robustness gains on RoboTwin 2.0, LIBERO-Plus, EBench, and a real robot.
Relevance: 10 Back to [topic] [top]
1005. GPT-6 Astra Is a Native Visuo-Tactile Policy for Contact-Rich Manipulation
ArXiv: 2610.26846 [page] [pdf]
Authors: Chen Yang, Linzhe Shi, Hang Zhang, …, Jiansheng Fan, Chen Wang
Abstract: Recent advances in multimodal models have enabled GPT-6 Astra to perform robotic manipulation through in-context learning. Yet contact-rich control and tactile perception are often regarded as strengths of specialist policies, leaving open whether Astra can directly understand and use tactile observations. We provide, to our knowledge, the first demonstration that Astra directly interprets vision-based tactile images and uses their contact cues for manipulation. Tactile RGB images enter through its existing visual interface, enabling few-shot closed-loop control on nineteen simulated UniVTAC and OpenViTac tasks with 210 sampled scene seeds per setting. With visuo-tactile input, Astra achieves mean success rates of 93.8% and 81.8%, exceeding the strongest reported specialist baselines in our comparison by 22.9 and 13.1 percentage points, respectively. Vision-only performance also leads these baselines, reaching 76.3% and 80.6%. Trajectory analysis reveals active probing, cautious grasping, contact-guided alignment, and recovery, alongside difficulties with tangential loading and corrective execution. Our findings challenge the exclusivity of specialist policies in contact-rich manipulation and show that fine-grained vision can partially compensate for missing touch. They also identify vision-based tactile sensing as an interface for scaling contact understanding with foundation models and reveal signs of benchmark saturation. These results motivate joint visual-tactile pretraining, harder contact-rich evaluations, and physical-robot validation.
Comment: Criterion 1: GPT-6 Astra directly uses visual and tactile images for few-shot, closed-loop robotic manipulation, achieving mean success rates of 93.8% and 81.8% on simulated UniVTAC and OpenViTac tasks.
Relevance: 10 Back to [topic] [top]
1006. Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience
ArXiv: 2610.26855 [page] [pdf]
Authors: Tianqi Liu, Hanfei Zhao, Qifeng Liang, …, Zhaoxi Chen, Ziwei Liu
Abstract: World models enable embodied agents to anticipate action consequences, yet jointly modeling locomotion and manipulation across diverse embodiments remains challenging due to heterogeneous action spaces and limited multimodal supervision. We present Ropedia W0, the first egocentric loco-manipulation world foundation model spanning humans and robots with diverse embodiments. We curate Xp-Data, comprising 10k hours of egocentric human data and 3k hours of robot data with unified multimodal annotations and systematic quality control. A unified kinematic representation encodes locomotion through camera poses and manipulation through projected skeletons, with robot skeletons combining embodiment-specific forward kinematics with extensive manual keypoint annotation and correction. We also introduce Xp-Teleop, a long-horizon loco-manipulation benchmark based on high-quality dexterous humanoid teleoperation data. Evaluations demonstrate state-of-the-art performance across five cross-embodiment benchmarks (DreamDojo, $Ψ_0$, $ω$-HOME, Xp-Teleop, and AgiBot) and on held-out human egocentric video, from zero-shot inference to few-shot and full-data post-training. Scaling human pre-training data improves performance on both human and robot evaluations. After distillation, our model supports real-time, minute-long rollouts, enabling live VR interaction. Applications in policy evaluation, model-based planning, and world-action modeling further demonstrate the model’s utility for embodied decision-making.
Comment: Criterion 1: Ropedia W0 introduces a cross-embodiment loco-manipulation world model trained on 10,000 hours of human and 3,000 hours of robot experience, demonstrating applications in policy evaluation, model-based planning, and world-action modeling.
Relevance: 10 Back to [topic] [top]
1007. SPAR: Structural Pruning and Action Recovery for Vision-Language-Action Models
ArXiv: 2610.26487 [page] [pdf]
Authors: Shenli Zheng, Zhou Fang, Yuan Du, Yijiang Liu, Li Du
Abstract: The limited onboard computing resources of robots make the inference demands of vision-language-action (VLA) models costly to accommodate. Reducing these demands through aggressive compression, however, can degrade manipulation performance beyond what supervised fine-tuning alone recovers. We present SPAR (Structural Pruning and Action Recovery), which simultaneously prunes the hidden and feed-forward intermediate dimensions of the large language model (LLM) backbone through pruning-aware tuning, then applies reinforcement learning (RL) to recover policy performance. Applied to $π_{0.5}$, SPAR reduces the language backbone from 2.34B to 0.09B parameters and the full model from 3.37B to 1.12B, removing approximately 96% and 67% of their parameters, respectively. At this compression level, RL raises average success across the four LIBERO suites from 84.55% to 93.45%, an 8.90-percentage-point gain. The compressed policy achieves a $1.98\times$ full-policy inference speedup over the unpruned model. These results identify substantial intermediate-dimension redundancy and show that post-pruning RL can recover manipulation performance in compact policies.
Comment: Matches criterion 1: SPAR combines structural pruning of the π₀.₅ vision-language-action model with reinforcement-learning recovery, reaching 93.45% average success across four LIBERO suites while providing a 1.98× full-policy inference speedup.
Relevance: 9 Back to [topic] [top]
1008. Learn the Task, Not the Schedule: Reducing Spurious Cross-Arm Coupling in Bimanual Imitation Learning
ArXiv: 2610.26507 [page] [pdf]
Authors: Shuyuan Hu, Deng-Ke Wu, Yifan Jin, …, Kai Ye, Bo Zhao
Abstract: A bimanual demonstration specifies both what the robot should do and one particular schedule for when each arm should act. Standard imitation learning does not distinguish task-required cross-arm constraints from incidental scheduling choices, and may therefore exploit opposite-arm progress as a shortcut cue, creating spurious cross-arm coupling even where relative timing is irrelevant for the current task phase. We introduce Constraint-preserving Temporal Recombination (CTR), a training-time data intervention that preserves recorded arm-local sensorimotor content while retiming the corresponding segments to realize admissible schedules that preserve task-required cross-arm constraints. We use CTR to train Diffusion Policy, Action Chunking with Transformers, and $π_{0.5}$, and evaluate them on four simulated and four real-robot tasks. Across six real-robot bimanual progress-state initializations, CTR raises worst-state success from 56% for the strongest non-CTR schedule mixture to 76%. With $π_{0.5}$ on an Independent task, a 4 s pause transfers 2.56 s of delay to the opposite arm under the Concurrent schedule distribution, but only 0.25 s with CTR. On real robots, CTR violates task constraints in only 0–4% of trials, versus 36–98% for narrow-schedule baselines. Code will be released at https://github.com/MINT-SJTU/CTR.
Comment: Matches criterion 1: Constraint-preserving Temporal Recombination trains Diffusion Policy and π₀.₅ on retimed bimanual demonstrations to reduce spurious cross-arm coupling, raising worst-state real-robot success from 56% to 76%.
Relevance: 9 Back to [topic] [top]
1011. Robot Learning with Vision-Language-Action Models for Agricultural Manipulation
ArXiv: 2610.26717 [page] [pdf]
Authors: Hamid Hafezi, Javad Mohammadpour Velni, Mohammadreza Davoodi
Abstract: Robotic manipulation in agricultural environments remains challenging due to variations in plant geometry, occlusion, and camera viewpoints. Pretrained vision-language-action (VLA) and generalist robot policies offer a promising approach to addressing these challenges by learning manipulation behaviors from multimodal observations and language instructions. However, adapting these policies to agricultural tasks requires domain-specific demonstrations, which are costly and time-consuming to collect on physical robots. In this work, we develop a simulation-to-real robot-learning framework for agricultural reaching and grasping that integrates task-specific demonstration collection and pretrained policy adaptation. A MuJoCo-based agricultural environment is developed with diverse tomato plant structures to support demonstration collection and controlled evaluation. Expert demonstrations are collected through joystick teleoperation, with a low-level controller used to execute robot actions. Two pretrained generalist robot policies, Octo and $π_{0.5}$, are adapted using simulated demonstrations, followed by further fine-tuning on a physical Franka Research 3 manipulator using real-robot demonstrations. The adapted policies are evaluated under in-distribution, viewpoint/spatial-shift, and out-of-distribution conditions in simulation, followed by physical-robot experiments. The influence of proprioceptive information on policy performance is also investigated. Results show that $π_{0.5}$ achieves higher task-performance scores than Octo and exhibits less degradation under distribution shifts. These findings demonstrate the potential of integrating pretrained generalist policies with task-specific demonstration learning for agricultural manipulation.
Comment: Criterion 1: adapts pretrained Octo and π₀.₅ policies using simulated and real-robot demonstrations for agricultural reaching and grasping on a Franka Research 3, with π₀.₅ achieving higher task scores and less degradation under distribution shifts.
Relevance: 9 Back to [topic] [top]
1014. Physical Words: A Vocabulary for Structuring Latent Reasoning in Robot Manipulation
ArXiv: 2610.26854 [page] [pdf]
Authors: Jiaming Liu, Xinghao Zhu, Yiming Zhang, …, Juekun Li, Yan Chang
Abstract: Multimodal reasoning provides robotic foundation models with predictive context needed for reliable action generation. However, language-based reasoning captures task semantics but struggles to express fine-grained physical dynamics, while latent reasoning models future state evolution but offers limited structure for organizing these dynamics into reusable reasoning primitives. To bridge this gap, we introduce PhysWord, a reasoning-before-acting foundation model built on Physical Words, which use atomic action verbs as shared semantic anchors for continuous representations of physical dynamics. Specifically, PhysWord predicts upcoming atomic action words and directly aligns their contextual hidden states with latent targets derived from the corresponding future observations. Building on these primitives, we construct a temporally structured PhysWord space with visual and optional tactile variants of each action word. Autoregressively predicting Physical Words across future timesteps captures transitions between execution stages, providing structured multimodal context for action generation. PhysWord achieves state-of-the-art success rates of 90.3\% on visuotactile UniVTAC and 99.6\% on vision-based LIBERO-Long, and outperforms competing methods on real-world tasks using bimanual grippers and dexterous hands. Further evaluations demonstrate that our structured latent reasoning framework generalizes to unseen scenarios and enables long-horizon manipulation.
Comment: Criterion 1: PhysWord structures vision-language reasoning for robot manipulation by aligning action-word hidden states with future-observation latents, achieving 99.6% success on LIBERO-Long and 90.3% on UniVTAC.
Relevance: 9 Back to [topic] [top]
5009. Drive2Gauss: FeedForward 4D Gaussian Reconstruction from Multimodal Generative Driving Videos
ArXiv: 2610.26585 [page] [pdf]
Authors: Tong Zhao, Zhiyuan Han, Lin Chen, …, Hao Zhao, Chaojian Li
Abstract: Generating reusable driving environments requires both controllable future observations and an explicit representation that supports rendering from different viewpoints, yet existing methods still recover explicit scenes from generated RGB videos through optimization for each sequence. To overcome these limitations, we present Drive2Gauss, a feedforward framework that converts jointly generated RGB, metric depth, and dynamic flow into dynamic 4D Gaussian scenes. A Multimodal Video Generator predicts the three modalities from reference multiview images and future control signals. Furthermore, we introduce a Gaussian Query Decoder that uses depth to initialize sparse 3D queries and flow to associate the queries across time, and integrates appearance information through spatiotemporal feature aggregation and iterative query refinement. The reconstruction stage operates in a single feedforward pass without optimization for each scene. Extensive results on nuScenes demonstrate that our approach attains an FID of 6.38 and an FVD of 12.43 for future RGB generation, a PSNR of 29.91 dB when reconstructing its generated observations, and supports both direct rendering under lateral camera shifts and downstream video fixing. Code and more visualizations are available at https://zttt04. github.io/Drive2Gauss_page/.
Comment: Criteria 5 and 6: Drive2Gauss converts jointly generated RGB, depth, and flow into dynamic 4D Gaussian scenes using a single-pass Gaussian Query Decoder, achieving 29.91 dB reconstruction PSNR without per-scene optimization.
Relevance: 9 Back to [topic] [top]
5013. InfiniDream: Open-Ended Photorealistic 3D Worlds Through Procedural Generation
ArXiv: 2610.26853 [page] [pdf]
Authors: Yi-Chuan Huang, Hao-Jen Chien, Zheng-Hui Huang, …, Kaipeng Zhang, Zhixiang Wang
Abstract: Procedural generation plays a key role in synthetic data creation and game development, offering greater controllability, editability, and diversity than static mesh representations. However, VLM-based code generation often produces relatively simple 3D assets, while sophisticated photorealistic procedural systems require extensive manual development. Existing VLM-guided procedural methods remain largely constrained by predefined factories and APIs, limiting their ability to create novel asset types. We introduce InfiniDream, a VLM-driven framework for open-ended procedural 3D world generation. First, we propose Symbolically Constrained Asset Generation, which leverages existing procedural algorithms to create novel, photorealistic assets while preserving editability and seed-based diversity. Second, we develop an end-to-end text-to-world pipeline that combines semantic scene planning with Deterministic Frustum-Guided Grid Placement to construct large-scale 3D worlds with precise object placement, procedural population, and animation. Extensive evaluations on 100 object and 20 scene generation tasks demonstrate improvements across four perceptual metrics. Diversity analysis, ablation studies, and human preference evaluations further validate the effectiveness of our framework. Project page: https://yichuanh.github.io/infinidream-page/
Comment: Criterion 5: InfiniDream uses VLM-driven Symbolically Constrained Asset Generation and a text-to-world pipeline to generate editable, photorealistic 3D content, demonstrating improvements on 100 object and 20 scene generation tasks.
Relevance: 9 Back to [topic] [top]
6010. EgoXperience: Learning World-Grounded Egocentric Representations from Human Experience
ArXiv: 2610.26625 [page] [pdf]
Authors: Xinhao Ji, Hui En Pang, Hao Li, …, Zhaoxi Chen, Ziwei Liu
Abstract: Reconstructing human activity from egocentric video requires more than recovering individual 3D components: the observer’s motion, surrounding geometry, and hand-object interactions must fit together in a consistent world. Achieving this coherence requires both joint reconstruction and aligned supervision, yet existing methods and datasets offer limited support for either. We present EgoXperience, a full-stack paradigm that addresses these needs through data collection, automatic annotation, unified modeling, and benchmarking. Central to EgoXperience is an automatic annotation engine, EgoX-Engine, that couples geometrically verified object tracking with SLAM-based world alignment, yielding consistent supervision for the camera, scene, hands, and objects. This yields EgoX-Data-6M, a 50-hour dataset of task-driven manipulation and locomotion within and across indoor and outdoor environments, with second-level task captions. We carefully select one hour across these activity categories as EgoX-Bench to evaluate joint reconstruction. To translate this supervision into reconstruction, we introduce EgoX, trained with EgoX-Data-6M and complementary annotations from additional public datasets. Given monocular egocentric video and first-frame object masks, our model achieves joint metric reconstruction of the camera, scene, hands, and objects in a shared world coordinate frame with a single forward pass. Extensive experiments demonstrate state-of-the-art performance across scene estimation and world-space hand-object pose recovery. Against the strongest evaluated baselines, EgoX reduces camera, hand, and depth errors by 68.9%, 65.6%, and 42.9%, and improves object pose scores by 28.0%.
Comment: Criterion 6: EgoX jointly estimates camera poses and reconstructs scenes, hands, and objects in a shared metric 3D frame from monocular egocentric video, reducing camera and depth errors by 68.9% and 42.9%, respectively.
Relevance: 9 Back to [topic] [top]
7003. HandsOn: A Large-Scale Simulation Benchmark for Generalizable Dexterous Manipulation
ArXiv: 2610.26765 [page] [pdf]
Authors: Tianwei Luo, Yiming He, Hongjin Wu, …, Haozhan Li, Ning Ding
Abstract: General-purpose manipulation requires reliable behavior across embodiments, objects, and task demands. We introduce HandsOn, a composable simulation platform and benchmark for cross-embodiment dexterous manipulation. It supports five robotic arms, six dexterous hands, and approximately 200 rigid, articulated, and deformable assets in randomized scenes. The benchmark pairs 18,000 curated human demonstrations with 66 human-verified tasks: Core-50 supports demonstration-based learning and execution tests, while Transfer-16 evaluates asset and relation transfer, skill composition, and deformable manipulation without target-task training demonstrations. Qualitative real-robot rollouts illustrate sim-to-real transfer. Across diffusion, vision-language-action, world-action, and prompted VLM-based methods, the best trained multi-task policy achieves 29.1% Core-50 success. Without benchmark finetuning, GPT-6 using move_eef for palm-and-finger control achieves 5.2% on Core-50 and 1.9% on Transfer-16 over 660 trials. Trained-policy stage diagnostics expose failures in contact establishment, action coordination, and maintaining achieved goals, with additional sequencing errors in composed tasks. HandsOn provides a shared basis for investigating manipulation generalization and the limits of reliable physical interaction.
Comment: Matches criterion 7: HandsOn introduces a cross-embodiment dexterous-manipulation simulation benchmark with 18,000 demonstrations and 66 tasks, evaluating diffusion, vision-language-action, world-action, and VLM-based policies, with the best trained multi-task policy achieving 29.1% success on Core-50.
Relevance: 10 Back to [topic] [top]
7012. Where is Physical RSI capability from? An Observational Scaling Law of LLM-as-policy
ArXiv: 2610.26848 [page] [pdf]
Authors: Zekai Li, Jiayi Yu, Jinlu Zhang, …, He Wang, Li Yi
Abstract: In this paper, we systematically investigate the mechanisms, scaling behavior, and generalization of physical Recursive self-improvement. We introduce PhysRSI-eval to quantify physical RSI through a five-axis metric: First-Attempt Performance, Efficiency, Success, Budget and Improvement from failure. Different VLMs are evaluated on 18 manipulation tasks under a fixed interaction budget. The analysis of experimental results motivates the Physical Barrel Hypothesis: different capabilities may constrain different parts of improvement. The five-axis metrics also distinguish systems with close public benchmark averages and show that high scores need not come with early or efficient gains. We study observational scaling from two perspectives: predicting the relative performance of VLM agents from their public capabilities, and forecasting later progress from early physical feedback. Together, these analyses ask how quickly and how far an agent can improve. Moreover, we conduct workspace interventions and then analyze what improvement leaves behind. Removing history reduces late-round success, frozen workspaces remain useful on unseen layouts, and three cross-task cases achieve success earlier. The results suggest that scene-specific adaptation and reusable experience can coexist.
Comment: Criterion 7: PhysRSI-eval benchmarks VLM-policy self-improvement across 18 manipulation tasks under fixed interaction budgets, empirically examining how physical feedback, interaction history, and workspace adaptation support subsequent success and transfer.