Meta tags:
author=Yilin Wu 吴怡琳;
description=A simple, whitespace theme for academics. Based on [*folio](htt????/github.com/bogoli/-folio) design. ;
keywords=jekyll, jekyll-theme, academic-website, portfolio-website;

Headings (most frequently used words):

yilin, wu, 吴怡琳, news, publications,

Text of the page (most frequently used words):
and (162), the (91), for (37), policy (35), with (34), learning (34), yilin (31), robotics (31), steering (27), models (26), manipulation (25), 2026 (23), that (23), robot (22), arxiv (20), action (19), rss (18), website (18), world (18), year (17), from (16), booktitle (16), author (16), title (16), inproceedings (16), robotic (16), bib (16), conference (16), human (16), 2024 (16), model (16), reasoning (16), systems (15), abs (15), via (15), bajcsy (15), andrea (15), alignment (15), international (14), data (14), level (14), tasks (13), paper (13), trained (13), language (13), this (12), visual (12), reward (12), vision (12), our (11), icra (11), embodied (11), science (10), work (10), real (10), policies (10), tool (10), 2025 (10), vlm (10), time (10), across (9), video (9), task (9), feedback (9), 2023 (9), can (9), about (9), foundation (9), pre (9), are (9), low (9), uncertainty (9), bimanual (8), ieee (8), award (8), contact (8), learn (8), latent (8), runtime (8), corl (7), sadigh (7), dorsa (7), novel (7), while (7), open (7), droid (7), over (7), use (7), prior (7), more (7), primitives (6), object (6), reinforcement (6), grannen (6), jennifer (6), propose (6), automation (6), team (6), best (6), dataset (6), through (6), only (6), research (6), tian (6), ran (6), these (6), success (6), actions (6), framework (6), workshop (6), behavior (6), bad (6), 𝚆𝙴𝙰𝚅𝙴𝚁 (6), inference (6), tactile (6), accepted (6), prof (6), deformable (5), introduce (5), improve (5), proposed (5), method (5), code (5), iclr (5), robots (5), diverse (5), user (5), act (5), embodiment (5), trajectories (5), performance (5), held (5), david (5), generalizable (5), trajectory (5), tools (5), rich (5), align (5), one (5), visuomotor (5), when (5), aligned (5), three (5), foresight (5), forethought (5), multimodal (5), behaviors (5), evaluation (5), verification (5), textual (5), also (5), aware (5), long (5), horizon (5), cmu (5), excited (5), lab (5), without (4), demonstrations (4), spatially (4), grounded (4), motion (4), generalization (4), environment (4), skills (4), proceedings (4), belkhale (4), suneel (4), food (4), loop (4), mouth (4), transfer (4), system (4), datasets (4), large (4), scale (4), jiang (4), bowen (4), iros (4), generation (4), efficiently (4), intelligence (4), often (4), generative (4), sequence (4), rewards (4), preferences (4), representation (4), preference (4), rapl (4), swamy (4), gokul (4), deployment (4), verifier (4), its (4), capabilities (4), outstanding (4), what (4), vla (4), plan (4), their (4), semantic (4), cvpr (4), high (4), ups (4), better (4), improvement (4), multi (4), vital (4), may (4), jun (4), talk (4), 吴怡琳 (3), github (3), pinto (3), lerrel (3), abbeel (3), pieter (3), manipulate (3), objects (3), grounding (3), effectively (3), variations (3), blog (3), any (3), scooping (3), acquisition (3), strategy (3), physical (3), sensing (3), build (3), end (3), oral (3), presentation (3), interaction (3), demonstrate (3), training (3), robustness (3), hacman (3), however (3), matching (3), based (3), complex (3), generalizes (3), given (3), first (3), test (3), maximizing (3), minimal (3), has (3), fine (3), simulation (3), experiments (3), demonstrating (3), hardware (3), diffusion (3), step (3), have (3), potential (3), they (3), vlms (3), was (3), unlock (3), verifiers (3), key (3), outcomes (3), plans (3), natural (3), you (3), say (3), even (3), correct (3), out (3), ood (3), scenarios (3), base (3), composition (3), costly (3), https (3), agentic (3), ask (3), selects (3), interventions (3), during (3), icml (3), position (3), satisfy (3), show (3), spotlight (3), top (3), wms (3), producing (3), simulated (3), rate (3), touch (3), guided (3), institute (3), internship (3), nvidia (3), give (3), modal (3), 2020 (2), yan (2), wilson (2), kurutach (2), thanard (2), parameterized (2), spatial (2), location (2), able (2), generalize (2), shape (2), pose (2), 2021 (2), representations (2), yunfei (2), huazhe (2), wang (2), xiaolong (2), solving (2), compositional (2), problems (2), reduction (2), train (2), spreading (2), after (2), adaptation (2), 2022 (2), 6th (2), general (2), successful (2), bite (2), haptic (2), shaikewitz (2), lorenzo (2), sundaresan (2), priya (2), autonomous (2), safely (2), force (2), like (2), reduce (2), 7th (2), brandon (2), stabilize (2), coordinate (2), arm (2), wild (2), demonstration (2), ability (2), zhou (2), wenxuan (2), paxton (2), chris (2), rsj (2), intelligent (2), carl (2), lifan (2), liu (2), haoyue (2), lin (2), xingyu (2), such (2), current (2), works (2), environments (2), challenge (2), unseen (2), which (2), different (2), then (2), generated (2), using (2), significantly (2), baselines (2), where (2), achieves (2), submission (2), journal (2), ijrr (2), special (2), issue (2), neural (2), symbolic (2), chenfeng (2), tomizuka (2), masayoshi (2), malik (2), jitendra (2), significant (2), domains (2), aligning (2), rlhf (2), same (2), due (2), required (2), less (2), space (2), validate (2), uses (2), find (2), still (2), failures (2), select (2), leveraging (2), consequences (2), than (2), text (2), forewarn (2), leverage (2), future (2), states (2), predicted (2), reason (2), anqi (2), hermans (2), tucker (2), ramos (2), fabio (2), erez (2), arpino (2), claudia (2), before (2), inspired (2), cot (2), distribution (2), samples (2), candidate (2), sequences (2), outcome (2), diversity (2), enabling (2), 2nd (2), yuan (2), jessie (2), way (2), learned (2), those (2), existing (2), both (2), jointly (2), execute (2), incapable (2), prediction (2), providing (2), expensive (2), continual (2), approaches (2), machine (2), good (2), need (2), community (2), state (2), art (2), including (2), interactions (2), execution (2), gap (2), design (2), kumar (2), jain (2), arnav (2), farebrother (2), jesse (2), faster (2), longer (2), effective (2), planning (2), all (2), desiderata (2), architecture (2), zilin (2), temel (2), zeynep (2), kroemer (2), oliver (2), methods (2), local (2), visuo (2), least (2), publications (2), news (2), two (2), got (2), summer (2), seattle (2), amazon (2), frontier (2), received (2), broadly (2), enhancing (2), fortunate (2), master (2), stanford (2), worked (2), imitation (2), undergrad (2), student (2), dot (2), copyright, powered, theme, hosted, photos, unsplash, pages, folio, jekyll, mit, press, journals, organization, wu2020learning, li2021solving, agent, acquire, rope, cloth, applies, domain, grannen2022learning, primitive, adaptive, stabilization, enables, set, geometries, properties, close, 10160467, semi, comfortably, disabled, people, composed, reactive, controller, accommodate, motions, throughout, dexterous, wrist, effector, discomfort, sensor, identify, grannen2023stabilize, present, coordinates, assigning, roles, arms, stabilizing, holds, stationary, acting, acts, simplified, mbodiment, open_x_embodiment_rt_x_2023, khazatsky2024droid, distributed, 76k, 350h, collected, 564, scenes, collectors, north, america, asia, europe, course, months, leads, higher, greater, improved, jiang2024hacman, qitooluse2024, utilize, assist, humans, completing, many, common, cooking, cleaning, fall, short, terms, adapting, affordance, make, strong, assumptions, cannot, tackle, explore, how, agents, previously, point, clouds, shapes, generate, optimize, poses, single, four, challenging, per, various, outperforming, further, comparable, tian2024rapm, increasingly, promise, advancements, remains, particularly, hard, specify, become, predominant, mechanism, non, not, seen, prohibitive, amount, functions, address, limitation, observation, unlike, traditional, focuses, tuning, encoders, constructs, dense, feature, magical, benchmark, franka, panda, embodiments, finally, tune, taking, towards, minimizing, wu2024forewarn, demonstrated, exhibit, offers, elegant, solution, reducing, chance, failure, external, imperfect, here, might, hope, off, shelf, struggle, understand, represented, fundamentally, differently, images, response, vocabulary, idea, decouple, burden, predicting, imagine, native, filter, bridge, representational, gaps, provide, robust, wu2025steering, instruction, following, generating, approach, chain, thought, yet, miss, intended, especially, formalize, phenomenon, lack, faithfulness, free, intermediate, multiple, predicts, whose, aligns, own, executing, turns, source, error, into, strength, boosting, perturbations, contribute, annotated, extension, libero, 100, tailored, gain, scales, compute, project, url, yuan2026actasklearnuncertaintyaware, emerging, adapt, analyzes, promising, purpose, frameworks, assume, well, calibrated, practice, overconfident, judgment, degrade, under, specifications, incapability, reasons, feasibility, resolution, confidence, clarify, ambiguity, queries, deemed, conformal, calibrate, statistical, assurances, collecting, employ, residual, capability, continually, but, showing, disentangle, confident, ambiguous, minimizes, compared, uncalibrated, gated, tian2024rapl, argues, obtain, reliable, must, invest, failed, suboptimal, errorprone, hazardous, central, lifecycle, today, primarily, analyze, systematically, evaluators, would, penalize, unsafe, poor, shortcut, strategies, superficially, attribute, scarcity, negative, collect, filtered, withheld, furthermore, modest, exposure, false, positives, therefore, call, curate, release, synthetic, engines, develop, decentralized, benchmarks, grained, evaluations, weaverwu2026, impacts, simulators, far, reaching, limited, downstream, needs, fidelity, correlate, reality, consistency, coherent, horizons, iii, efficiency, quickly, estimation, views, simultaneously, results, view, predict, latents, values, flow, loss, distill, decisions, memory, objectives, kinds, dynamic, confounded, modeling, apply, effectiveness, 870, correlation, speedup, demonstrates, evaluated, videos, arnavkj1995, weaver, vitalwu2026, adapts, verifying, typically, perform, observations, alone, insufficient, depends, global, progress, subtle, formulates, guidance, optimization, problem, sampling, performs, mode, selection, deciding, should, editing, refines, selected, shorter, requirements, support, learns, employs, semantically, conditioned, scores, futures, directly, improves, overall, outperforms, unimodal, exceeds, naive, fusion, corl2026vital, starting, study, robitic, aug, coordination, sep, papers, wins, start, meta, fair, jul, austin, robin, oct, duke, dexlab, www, youtube, com, watch, sbbrs2n, qiw, nov, share, jan, been, apr, started, gave, recent, h2r, seal, mmragi, centers, around, overcoming, fundamental, limits, application, continuous, dynamics, want, capably, centered, earlier, focused, assistive, feeding, tsinghua, university, shanghai, zhi, self, background, interest, lies, building, fast, etc, interested, industry, experiences, had, great, internships, third, intent, advised, supervised, email, yilinwu, andrew, edu, phd, scs, toggle, navigation,

Text of the page (random words):
nipulation more broadly my research centers around overcoming the fundamental embodiment gap that limits the application of foundation models to robotics by grounding high level semantic reasoning in the continuous dynamics of the physical world i want to build robots that can learn reason and act capably in human centered environments enhancing their generalization and robustness through runtime alignment and continual learning news jun 2026 seal received the best paper award at the cvpr mmragi workshop multi modal reasoning for agentic intelligence jun 2026 gave a spotlight talk about my recent two works ups uncertainty aware policy steering and vital inference time policy steering via vision and touch at cvpr 2026 h2r workshop may 2026 started my summer internship at the amazon frontier ai robotics team apr 2026 one paper uncertainty aware policy steering ups has been accepted at rss 2026 jan 2026 excited to share my work from my internship at nvidia do what you say steering vision language action models via runtime reasoning action alignment verification is accepted to icra 2026 nov 2025 excited to give a talk about my research on language guided runtime steering with robot foundation models at duke dexlab https www youtube com watch v sbbrs2n qiw oct 2025 excited to give a talk about my research on language guided runtime steering with robot foundation models at ut austin robin lab jul 2025 excited to give a talk about my forewarn paper at robotics team in meta fair jun 2025 one paper from foresight to forethought vlm in the loop policy steering via latent alignment got accepted to rss 2025 may 2025 excited to start my summer internship in nvidia seattle robotics lab jun 2024 one paper learning generalizable tool use skills through trajectory generation got accepted at iros 2024 may 2024 open x embodiment wins the best paper award at icra 2024 may 2024 two papers droid and hacman are accepted by rss 2024 sep 2023 our work on bimanual manipulation inspired by human coordination is accepted to corl 2023 as oral presentation aug 2023 starting my ph d study at cmu robitic institute show more news publications arxiv inference time policy steering via vision and touch yilin wu zilin si zeynep temel oliver kroemer and andrea bajcsy in arxiv 2026 abs bib website inference time steering adapts pre trained generative robot policies during deployment by verifying candidate actions before execution while prior methods typically perform this verification only with visual observations vision alone is often insufficient for contact rich manipulation where success depends on both global task progress and subtle local interactions such as contact force we introduce vital a visuo tactile inference time steering framework that formulates multimodal guidance as a bi level optimization problem at the high level visual sampling and verification performs long horizon mode selection deciding what behavior the robot should execute at the low level tactile guided diffusion editing refines the selected action sequence over a shorter horizon to satisfy local contact requirements to support outcome based steering vital learns a visuo tactile latent world model and employs semantically aligned visual and tactile verifiers including a novel text conditioned tactile reward that scores predicted tactile futures directly in latent space across three real world contact rich manipulation tasks vital improves overall success by 51 over the base policy outperforms unimodal steering by at least 33 and exceeds naive multimodal fusion by at least 20 website corl2026vital github io inproceedings vitalwu2026 title inference time policy steering via vision and touch author wu yilin and si zilin and temel zeynep and kroemer oliver and bajcsy andrea booktitle arxiv year 2026 arxiv 𝚆𝙴𝙰𝚅𝙴𝚁 better faster longer an effective world model for robotic manipulation arnav kumar jain yilin wu jesse farebrother gokul swamy and andrea bajcsy in arxiv 2026 abs bib website the potential impacts of world models wms i e learned simulators on robotics are far reaching policy evaluation policy improvement and test time planning all with limited real world interaction to unlock these downstream capabilities a wm needs to jointly satisfy three desiderata i fidelity i e producing simulated trajectories that correlate with reality ii consistency i e producing simulated trajectories that are coherent over long horizons and iii efficiency i e producing simulated trajectories quickly we propose 𝚆𝙴𝙰𝚅𝙴𝚁 world estimation across views for embodied reasoning a wm architecture that simultaneously achieves all three desiderata providing state of the art results on robotic manipulation tasks 𝚆𝙴𝙰𝚅𝙴𝚁 is a multi view wm trained to predict future latents and reward values via a flow matching loss we distill the key design decisions across model architecture memory and prediction objectives required to unlock the kinds of long horizon dynamic manipulation tasks that have confounded prior world modeling approaches we apply 𝚆𝙴𝙰𝚅𝙴𝚁 in robotic hardware demonstrating its effectiveness at policy evaluation ρ 0 870 correlation with real world success rate policy improvement real world success rate improvement of 38 on top of the π0 5 robot foundation model and test time planning real world success rate improvement of 14 with a 5 10 speedup over prior wms 𝚆𝙴𝙰𝚅𝙴𝚁 also demonstrates better performance than prior wms when evaluated on out of distribution scenarios code models and videos at https arnavkj1995 github io weaver inproceedings weaverwu2026 title 𝚆𝙴𝙰𝚅𝙴𝚁 better faster longer an effective world model for robotic manipulation author kumar jain arnav and wu yilin and farebrother jesse and swamy gokul and bajcsy andrea booktitle arxiv year 2026 icml position good embodied reward models need bad behavior data ran tian yilin wu and andrea bajcsy in international conference on machine learning icml 2026 spotlight top 5 abs bib website spotlight top 5 this position paper argues that to obtain reliable embodied reward models the community must invest in bad robot data failed suboptimal errorprone and even hazardous behaviors while reward models are central to any foundation model s lifecycle today s embodied reward models are trained primarily on successful behaviors we analyze three state of the art embodied reward models and find that they systematically over reward behaviors that real human evaluators would penalize including unsafe interactions poor execution and shortcut strategies that only superficially satisfy tasks we attribute these failures to a key data gap the scarcity of negative embodied data which is costly to collect and often filtered out or withheld in existing robotics datasets furthermore we show that even modest exposure to real bad behavior data can improve alignment with human preferences and reduce costly false positives we therefore call on the embodied ai community to curate and release their bad robot data build synthetic bad data generation engines develop more decentralized physical evaluation systems and design benchmarks for fine grained embodied reward model evaluations inproceedings tian2024rapl title position good embodied reward models need bad behavior data author tian ran and wu yilin and bajcsy andrea booktitle international conference on machine learning icml year 2026 rss when to act ask or learn uncertainty aware policy steering jessie yuan yilin wu and andrea bajcsy in robotics science and systems rss 2026 abs arxiv bib website policy steering is an emerging way to adapt robot behaviors at deployment time a learned verifier analyzes low level action samples proposed by a pre trained policy e g diffusion policy and selects only those aligned with the task while vision language models vlms are promising general purpose verifiers due to their reasoning capabilities existing frameworks often assume these models are well calibrated in practice the overconfident judgment from vlm can degrade the steering performance under both high level semantic uncertainty in task specifications and low level action uncertainty or incapability of the pre trained policy we propose uncertainty aware policy steering ups a framework that jointly reasons about semantic task uncertainty and low level action feasibility and selects an uncertainty resolution strategy execute a high confidence action clarify task ambiguity via natural language queries or ask for action interventions to correct the low level policy when it is deemed incapable at the task we leverage conformal prediction to calibrate the composition of the vlm and the pre trained base policy providing statistical assurances that the verifier selects the correct strategy after collecting interventions during deployment we employ residual learning to improve the capability of the pre trained policy enabling the system to learn continually but with minimal expensive human feedback we demonstrate our framework through experiments in simulation and on hardware showing that ups can disentangle confident ambiguous and incapable scenarios and minimizes expensive user interventions compared to uncalibrated baselines and prior human or robot gated continual learning approaches inproceedings yuan2026actasklearnuncertaintyaware title when to act ask or learn uncertainty aware policy steering author yuan jessie and wu yilin and bajcsy andrea booktitle robotics science and systems rss year 2026 icra do what you say steering vision language action models via runtime reasoning action alignment verification yilin wu anqi li tucker hermans fabio ramos andrea bajcsy and claudia p erez d arpino in 2026 ieee international conference on robotics and automation icra 2026 outstanding paper award at cvpr 2026 2nd multimodal reasoning for agentic intelligence workshop abs arxiv bib code website outstanding paper award at cvpr 2026 2nd multimodal reasoning for agentic intelligence workshop reasoning vision language action vla models improve robotic instruction following by generating step by step textual plans before low level actions an approach inspired by chain of thought cot reasoning in language models yet even with a correct textual plan the generated actions can still miss the intended outcomes in the plan especially in out of distribution ood scenarios we formalize this phenomenon as a lack of embodied cot faithfulness and introduce a training free runtime policy steering method for reasoning action alignment given a reasoning vla s intermediate textual plan our framework samples multiple candidate action sequences from the same model predicts their outcomes via simulation and uses a pre trained vision language model vlm to select the sequence whose outcome best aligns with the vla s own textual plan only executing action sequences that align with the textual reasoning turns our base vla s natural action diversity from a source of error into a strength boosting robustness to semantic and visual ood perturbations and enabling novel behavior composition without costly re training we also contribute a reasoning annotated extension of libero 100 environment variations tailored for ood evaluation and demonstrate up to 15 performance gain over prior work on behavior composition tasks and scales with compute and data diversity project website at this https url inproceedings wu2025steering title do what you say steering vision language action models via runtime reasoning action alignment verification author wu yilin and li anqi and hermans tucker and ramos fabio and bajcsy andrea and p erez d arpino claudia booktitle 2026 ieee international conference on robotics and automation icra year 2026 rss from foresight to forethought vlm in the loop policy steering via latent alignment yilin wu ran tian gokul swamy and andrea bajcsy in robotics science and systems rss 2025 outstanding paper award at iclr 2025 world model workshop abs arxiv bib code website outstanding paper award at iclr 2025 world model workshop while generative robot policies have demonstrated significant potential in learning complex multimodal behaviors from demonstrations they still exhibit diverse failures at deployment time policy steering offers an elegant solution to reducing the chance of failure by using an external verifier to select from low level actions proposed by an imperfect generative policy here one might hope to use a vision language model vlm as a verifier leveraging its open world reasoning capabilities however off the shelf vlms struggle to understand the consequences of low level robot actions as they are represented fundamentally differently than the text and images the vlm was trained on in response we propose forewarn a novel framework to unlock the potential of vlms as open vocabulary verifiers for runtime policy steering our key idea is to decouple the vlm s burden of predicting action outcomes foresight from evaluation forethought for foresight we leverage a latent world model to imagine future latent states given diverse low level action plans for forethought we align the vlm with these predicted latent states to reason about the consequences of actions in its native representation natural language and effectively filter proposed plans we validate our framework across diverse robotic manipulation tasks demonstrating its ability to bridge representational gaps and provide robust generalizable policy steering inproceedings wu2024forewarn title from foresight to forethought vlm in the loop policy steering via latent alignment author wu yilin and tian ran and swamy gokul and bajcsy andrea booktitle robotics science and systems rss year 2025 arxiv maximizing alignment with minimal feedback efficiently learning rewards for visuomotor robot policy alignment ran tian yilin wu chenfeng xu masayoshi tomizuka jitendra malik and andrea bajcsy in submission to the international journal of robotics research s ijrr special issue on foundation models and neural symbolic ai for robotics 2024 abs arxiv bib website visuomotor robot policies increasingly pre trained on large scale datasets promise significant advancements across robotics domains however aligning these policies with end user preferences remains a challenge particularly when the preferences are hard to specify while reinforcement learning from human feedback rlhf has become the predominant mechanism for alignment in non embodied domains like large language models it has not seen the same success in aligning visuomotor policies due to the prohibitive amount of human feedback required to learn visual reward functions to address this limitation we propose representation aligned preference based learning rapl an observation only method for learning visual rewards from significantly less human preference feedback unlike traditional rlhf rapl focuses human feedback on fine tuning pre trained vision encoders to align with the end user s visual representation and then constructs a dense visual reward via feature matching ...