Affordance Field Intervention: Enabling VLAs to Escape Memory Traps in Robotic Manipulation

Siyu Xu1   Zijian Wang1   Yunke Wang1   Chenghao Xia3   Tao Huang2   Chang Xu1  
1The University of Sydney   2Shanghai Jiao Tong University   3StellarEdge Robotics
Paper arXiv Code
VLA-AFI Teaser

Memory Trap in VLAs. VLAs often fail under distribution shifts. In such a case, they replay trajectories memorized during training instead of adapting to the updated scene. In both Case 1 and Case 2, even though the target object moves, the VLA drives the end-effector toward the original location, ignoring new spatial cues.

Abstract

Vision-Language-Action (VLA) models have shown great performance in robotic manipulation by mapping visual observations and language instructions directly to actions. However, they remain brittle under distribution shifts: when test scenarios change, VLAs often reproduce memorized trajectories instead of adapting to the updated scene, which is a failure mode we refer to as the "Memory Trap".

This limitation stems from the end-to-end design, which lacks explicit 3D spatial reasoning and prevents reliable identification of actionable regions in unfamiliar environments. To compensate for this missing spatial understanding, 3D Spatial Affordance Fields (SAFs) can provide a geometric representation that highlights where interactions are physically feasible, offering explicit cues about regions the robot should approach or avoid.

We therefore introduce Affordance Field Intervention (AFI), a lightweight hybrid framework that uses SAFs as an on-demand plug-in to guide VLA behavior. Our system detects memory traps through proprioception, repositions the robot to recent high-affordance regions, and proposes affordance-driven waypoints that anchor VLA-generated actions. A SAF-based scorer then selects trajectories with the highest cumulative affordance.

Extensive experiments demonstrate that our method achieves an average improvement of 23.5% across different VLA backbones (π0 and π0.5) under out-of-distribution scenarios on real-world robotic platforms, and 20.2% on the LIBERO-Pro benchmark, validating its effectiveness in enhancing VLA robustness to distribution shifts.

Method

Affordance Field Intervention (AFI) is a hybrid framework that augments a VLA policy with a 3D Spatial Affordance Field (SAF) as an on-demand plug-in. The system continuously monitors proprioceptive signals to detect when the VLA falls into a memory trap, repositions the end-effector toward recent high-affordance regions, proposes affordance-driven waypoints, and ranks candidate trajectories by their cumulative SAF score.

3D Spatial Affordance Field

Visualizations of the SAF across diverse manipulation scenes. The field highlights physically feasible interaction regions, providing the geometric cues that VLAs lack when generalizing to out-of-distribution scenes.

Affordance-Driven Waypoint Sampling

AFI samples candidate waypoints biased toward high-affordance regions and uses them to anchor the VLA-generated actions, steering the policy away from memorized trajectories and toward feasible interactions in the current scene.

Experiments

LIBERO-Pro Benchmark

LIBERO-Pro Experiments

Results on the LIBERO-Pro benchmark across out-of-distribution settings. AFI delivers an average improvement of 20.2% over the underlying VLA backbones, demonstrating consistent robustness gains.


Real-World Experiments

Real-world Settings

Real-world manipulation setup. We evaluate AFI on top of π0 and π0.5 across a suite of OOD scenarios. AFI improves the success rate by an average of 23.5% relative to the base VLA backbones.


Additional experiments

Additional ablations and qualitative comparisons across baselines and scenes.

BibTeX

@article{xu2026afi,
  title={Affordance Field Intervention: Enabling VLAs to Escape Memory Traps in Robotic Manipulation},
  author={Xu, Siyu and Wang, Zijian and Wang, Yunke and Xia, Chenghao and Huang, Tao and Xu, Chang},
  year={2026}
}