CLEAR plans how a robot can reach its goal by moving objects in the scene. It decides which objects to move, in what order, and how to move them, taking the robot’s capabilities and the effects of earlier interactions into account.
Affordance-Guided Encoding builds a planning context that reflects the robot’s interaction capabilities and object reachability. Shared feature encoding and average pooling turn the link, joint, and actuator properties of embodiment into an embedding .
Conditioned on this embedding and local scene observations, the network predicts an affordance probability for each candidate object interaction or terrain transition. The binary label indicates interaction availability or transition feasibility.
When terrain limits movement, path search uses predicted transition probabilities to evaluate scene connectivity and determine which objects are reachable. Object features augmented with these interaction probabilities and reachability information are combined with and scene observations to form , the shared input to OrderNet and CausalFlowNet.
From , OrderNet predicts a selection probability and Gaussian score parameters for each object. Bernoulli sampling selects the object set ; Gaussian sampling produces latent scores . Sorting selected scores from lowest to highest determines the interaction order.
Sampling produces alternative object sets and sequences for the same scene and embodiment. Selection considers the complete sequence, including objects that become reachable after earlier interactions. Each candidate uses an object at most once; replanning may select that object again.
Binary cross-entropy trains selection. A weighted Bradley–Terry loss encourages lower scores for preceding objects, with more weight on nearby pairs in the training order. Gaussian reparameterization supports gradient-based score learning, and KL regularization keeps the score distributions close to a standard normal distribution.
CausalFlowNet generates continuous object parameters with the selected set, interaction order, and planning context held fixed. Rank is an object’s position in . The matrix stores coordinates per object; is generation time, not execution time.
Rank-causal attention gives each selected object access to its own token, preceding ranks, and . Generated information cannot enter context tokens, so later predictions cannot influence earlier ones indirectly through the shared context.
During training, active coordinates of selected objects exclude padding and fixed coordinates. Independent Gaussian noise initializes these coordinates in , and is sampled for conditional flow matching:
The loss is mean squared error over active coordinates, averaged over training examples, noise, and flow time; it is zero when no active coordinates exist. The combined objective adds ordering, flow, affordance, and optional availability losses:
The availability term uses training labels to penalize unavailable objects, with normalized first-interaction weights proportional to ; when unused.
At inference, integrating from Gaussian noise at to produces . Selected rows are decoded in order into object targets or trajectories . Different noise samples yield alternative motions under the same selection and order; the resulting candidates proceed to feasibility checking.
Candidate validation checks robot reachability, collision avoidance, and interaction constraints in interaction order . Each check uses the scene resulting from preceding object motions. This validation step is omitted in the Grid and Ordered Manipulation evaluations.
Among feasible candidates that satisfy the navigation or arrangement goal , the planner selects the one with the fewest object interactions. A valid candidate requiring no interaction has and bypasses continuous generation.
For legged robots, an MPC-based controller evaluates candidate command sequences through rollouts of a learned locomotion policy. Quality-diversity optimization retains high-performing and behaviorally diverse sequences, combined with Cartesian tracking and joint-command smoothing.
The controller executes an initial segment of the best evaluated command sequence and updates its optimization using new state estimates. After each object interaction, updated scene observations guide replanning of the remaining interactions.
Robot embodiment is described by link, joint, and actuator properties. Shared feature encoding and average pooling construct the embedding , which conditions predictions of interaction availability and terrain-transition feasibility.
Each movable object has a current configuration and physical attributes , including geometry and mass. Object motions change the scene and can make subsequent interactions reachable.
The scene contains observed environment geometry, the current robot state, movable objects, and task goal . The goal specifies a navigation destination or desired object arrangement. Terrain connectivity determines which objects the robot can reach.
