Meta tags:
description=UMI-on-Air: Embodiment-Aware Guidance for Embodiment-Agnostic Visuomotor Policies;
keywords=Robotics, Aerial Manipulation, Embodiment-Aware Guidance, Visuomotor Policies;

Headings (most frequently used words):

embodiment, umi, aware, diffusion, policy, on, air, guidance, for, agnostic, visuomotor, policies, benchmark, suite, real, world, results, technical, summary, video, our, team, vanilla, questions, and, answers,

Text of the page (most frequently used words):
the (38), and (25), umi (22), embodiment (17), policy (12), this (10), eadp (10), #diffusion (10), trial (8), #policies (7), for (7), aware (7), from (6), trajectories (6), are (6), air (6), with (6), real (6), huy (5), guanya (5), shi (5), robot (5), can (5), while (5), guidance (5), harsh (5), gupta (5), trials (5), task (5), world (5), all (5), peg (5), legs (4), embodiments (4), robots (4), like (4), drones (4), that (4), gap (4), controller (4), hardware (4), xiaofeng (4), guo (4), chuer (4), pan (4), muqing (4), cao (4), dongjae (4), lee (4), sebastian (4), scherer (4), shuran (4), song (4), arxiv (4), our (4), video (4), evaluate (4), place (4), hole (4), paper (3), how (3), control (3), what (3), based (3), controllers (3), mpc (3), its (3), process (3), way (3), manipulation (3), across (3), into (3), pick (3), actions (3), website (2), actually (2), when (2), was (2), just (2), ability (2), execute (2), able (2), they (2), demonstrations (2), legged (2), constraints (2), them (2), less (2), toward (2), feasible (2), which (2), use (2), systems (2), dynamics (2), learning (2), especially (2), making (2), more (2), general (2), not (2), gradient (2), two (2), communication (2), lets (2), time (2), but (2), any (2), data (2), specific (2), action (2), you (2), questions (2), agnostic (2), visuomotor (2), 2510 (2), 02614 (2), university (2), team (2), show (2), long (2), horizon (2), success (2), over (2), succeeded (2), lemon (2), randomized (2), timeout (2), improves (2), precision (2), videos (2), shown (2), speed (2), results (2), tracking (2), tasks (2), standard (2), valve (2), open (2), retrieve (2), suite (2), benchmark (2), back (2), code (2), licensed, under, creative, commons, attribution, sharealike, international, license, template, modified, term, came, describing, why, some, listen, better, than, others, stuck, later, decided, formalize, measurable, notion, prof, measures, well, produced, trained, fixed, base, arms, highly, closely, follow, human, face, dynamic, make, helps, bridge, steering, each, does, mean, currently, relies, analytical, gradients, model, limits, known, however, recent, progress, reinforcement, whole, body, suggests, exciting, path, forward, future, versions, could, learned, provide, same, kind, scalable, omnih2o, quite, introduces, where, low, level, provides, feedback, during, sampling, adapt, something, even, ur10e, arm, benefits, made, mobile, makes, adaptive, isn, drone, didn, directly, teleoperate, instead, leveraged, lightweight, handheld, gripper, humans, record, diverse, without, using, decouples, collection, possible, gather, large, wild, datasets, quickly, keeping, space, consistent, universal, interface, teleoperating, hard, did, collect, enough, answers, have, please, contact, misc, gupta2025umionairembodimentawareguidanceembodimentagnostic, title, author, year, 2025, eprint, archiveprefix, primaryclass, url, https, org, abs, stanford, carnegie, mellon, technical, summary, requires, threading, bulb, socket, until, tight, followed, flipping, switch, confirm, spans, minutes, wall, clock, underscoring, need, stability, extended, horizons, lightbulb, insertion, uam, must, harvest, location, basket, completed, successfully, only, failure, occurring, unripe, green, selected, 4cm, 2cm, starting, positions, minute, baseline, failed, due, dropped, five, demonstrating, these, benchmarks, performance, consistently, declines, become, capable, revealing, rates, narrows, compared, rotate, oracle, simulation, covering, four, articulated, interactions, rotation, solution, compute, cost, respect, feed, steers, trajectory, generation, closing, between, call, approach, generate, visual, observations, then, tracked, one, tells, move, cannot, signal, whether, those, mismatch, often, leads, failures, aerial, manipulators, vanilla, allow, steer, leading, robust, cross, deployment, key, idea, equally, track, perfectly, method, problem,

Text of the page (random words):
umi on air paper arxiv video hardware code umi on air embodiment aware guidance for embodiment agnostic visuomotor policies harsh gupta xiaofeng guo huy ha chuer pan muqing cao dongjae lee sebastian scherer shuran song guanya shi paper arxiv video hardware code problem method benchmark results video hardware our team q a not all robots are equally umi able they can t all track umi trajectories perfectly our key idea allow the robot s controller to steer umi s diffusion policy leading to more robust cross embodiment deployment vanilla diffusion policy standard umi policies generate action trajectories from visual observations which are then tracked by controllers like mpc but this is a one way process the policy tells the robot how to move while the robot cannot signal back whether those actions are feasible this mismatch often leads to failures especially for embodiments like aerial manipulators embodiment aware diffusion policy our solution is two way communication we compute the gradient of the controller s tracking cost with respect to the policy s actions and feed it back into the diffusion process as guidance this steers trajectory generation toward actions that the embodiment can actually execute closing the gap between general policies and embodiment specific constraints we call this approach embodiment aware diffusion policy eadp umi benchmark suite we evaluate eadp in a simulation suite covering four tasks long horizon manipulation open and retrieve precision control peg in hole and articulated interactions pick and place and valve rotation videos are shown in 2x speed oracle open and retrieve peg in hole rotate valve pick and place across these benchmarks performance consistently declines as embodiments become less capable of tracking policy trajectories revealing the embodiment gap eadp improves success rates across all tasks and narrows the embodiment gap compared to standard diffusion policies real world results we evaluate the peg in hole task in the real world on a 4cm hole with a 2cm peg with randomized starting positions and a 3 minute timeout while the baseline dp failed due to dropped peg or timeout 3 5 eadp succeeded on all five trials 5 5 demonstrating that eadp improves precision in the real world videos are shown in 8x speed diffusion policy embodiment aware diffusion policy we evaluate the pick and place task in the real world the uam must harvest a lemon from a randomized location and place it into a basket eadp completed 4 5 trials successfully with the only failure occurring when an unripe green lemon was selected show trials trial 1 trial 2 trial 3 trial 4 trial 5 we evaluate the lightbulb insertion task in the real world this long horizon task requires threading a bulb into its socket until tight followed by flipping the switch to confirm success the task spans over 3 minutes of wall clock time underscoring the need for stability over extended horizons eadp succeeded in all trials 3 3 show trials trial 1 trial 2 trial 3 technical summary video our team harsh gupta xiaofeng guo huy ha chuer pan muqing cao dongjae lee sebastian scherer shuran song guanya shi harsh gupta xiaofeng guo huy ha chuer pan muqing cao dongjae lee sebastian scherer shuran song guanya shi carnegie mellon university stanford university misc gupta2025umionairembodimentawareguidanceembodimentagnostic title umi on air embodiment aware guidance for embodiment agnostic visuomotor policies author harsh gupta and xiaofeng guo and huy ha and chuer pan and muqing cao and dongjae lee and sebastian scherer and shuran song and guanya shi year 2025 eprint 2510 02614 archiveprefix arxiv primaryclass cs ro url https arxiv org abs 2510 02614 if you have any questions please contact harsh gupta questions and answers q teleoperating drones is hard how did you collect enough data we didn t directly teleoperate drones instead we leveraged the universal manipulation interface umi a lightweight handheld gripper that lets humans record diverse manipulation demonstrations without using any robot hardware this decouples data collection from specific embodiments making it possible to gather large in the wild datasets quickly while keeping the action space consistent across robots q isn t this just umi on legs but with a drone controller not quite umi on air introduces embodiment aware guidance where the low level controller like mpc for drones provides gradient feedback during the diffusion sampling process this two way communication lets the policy adapt its trajectories to the dynamics of the embodiment in real time something even the ur10e arm benefits from so while umi on legs made umi policies mobile umi on air makes them embodiment adaptive q ️ what can t the embodiment aware diffusion policy eadp do eadp currently relies on analytical gradients from model based controllers e g mpc which limits its use to systems with known dynamics however recent progress in learning based control especially reinforcement learning for legged and whole body systems e g umi on legs omnih2o suggests an exciting path forward future versions could use learned or rl based controllers to provide the same kind of guidance making eadp more general and scalable q what does umi ability mean umi ability measures how well a robot can execute trajectories produced by a umi trained policy robots like fixed base arms are highly umi able they can closely follow human demonstrations while drones or legged robots face dynamic and control constraints that make them less so umi on air helps bridge this gap by steering trajectories toward what s feasible for each embodiment the term actually came up when prof guanya shi was describing why some embodiments just listen better to umi policies than others it stuck and later huy ha decided to formalize it in the paper as a measurable notion website template modified from umi on legs this website is licensed under a creative commons attribution sharealike 4 0 international license