Meta tags:

Headings (most frequently used words):

on, cs, policy, teaming, vlm, blue, red, contact, graspnet, original, with, fails, crashshape, ycb, stage, performance, teamed, baseline, nominal, plug, prompting, real, world, validation, over, mustard, bottle, deformed, screw, driver, input, response, geometric, for, robotic, manipulation, abstract, system, overview, strategy, examples, insertion, acknowledgement, bibtex, conference, robot, learning, corl, 2025, simulation, demos,

Text of the page (most frequently used words):
the (56), #teaming (25), policy (25), and (22), red (20), task (15), with (14), blue (12), from (11), #nominal (11), geometric (10), contact (10), graspnet (10), performance (10), vlm (10), fails (8), consistently (8), video (8), object (8), stage (8), for (7), not (7), ycb (7), plug (7), teamed (7), that (7), printed (6), original (6), crashshape (6), your (6), browser (6), does (6), support (6), tag (6), robotic (5), manipulation (5), deformed (5), using (5), robot (4), this (4), are (4), most (4), confident (4), grasp (4), version (4), screw (4), driver (4), mustard (4), bottle (4), select (4), framework (4), over (4), grasping (4), model (4), real (4), insertion (4), json (4), view (4), deformations (4), handle (4), crashshapes (4), template (3), based (3), upon (3), under (3), dataset (3), baseline (3), failures (3), world (3), succeeds (3), was (3), fine (3), evaluated (3), robustness (3), input (3), prompting (3), mesh (3), points (3), strategy (3), simulation (3), grt (3), author (2), goel (2), divyam (2), wang (2), yufei (2), tiancheng (2), qiao (2), piliptchak (2), pavel (2), held (2), david (2), erickson (2), zackory (2), conference (2), learning (2), 2025 (2), material (2), nist (2), grant (2), office (2), naval (2), research (2), those (2), obtained (2), performs (2), well (2), taken (2), directly (2), tip (2), thumbnail (2), strips (2), play (2), trial (2), larger (2), window (2), tested (2), our (2), physical (2), objects (2), following (2), videos (2), demonstrate (2), effectiveness (2), identifying (2), validation (2), also (2), preserved (2), now (2), tuned (2), then (2), both (2), shapes (2), when (2), presented (2), pre (2), trained (2), standard (2), output (2), response (2), prompt (2), canonical (2), panel (2), critical (2), ranking (2), reasoning (2), choose (2), level (2), description (2), next (2), candidates (2), targeted (2), aware (2), articulation (2), sampled (2), population (2), deformation (2), into (2), jacobian (2), field (2), optimization (2), failure (2), system (2), overview (2), evaluation (2), centric (2), success (2), institute (2), website, borrowed, eureka, articubot, inproceedings, goel2025geometric, title, guixiu, booktitle, pages, year, organization, pmlr, bibtex, work, supported, 70nanb24h314, muri, n00014, 2748, any, opinions, findings, conclusions, recommendations, expressed, necessarily, reflect, views, acknowledgement, franka, emika, panda, pairs, generalizable, separate, same, undeformed, xarm, plugs, enhancing, content, full, drawer, opening, examples, first, aligns, indexed, keypoints, high, guiding, infer, which, vertices, control, meaningful, asks, pareto, rank, these, plausibility, relevance, producing, compact, set, two, point, selection, type, demos, given, initialization, parameters, anchor, selected, vision, language, displacements, define, each, sample, converted, perturbed, via, frozen, induce, guide, protocols, typically, assess, curated, distribution, test, sets, offering, limited, insight, how, systems, fail, plausible, variation, introduce, probes, through, perturbations, automatically, generating, structurally, valid, user, constrained, trigger, catastrophic, policies, method, integrates, gradient, free, simulator, loop, across, tasks, discovers, collapse, revealing, brittle, modes, missed, static, benchmarks, combining, rollouts, constraint, shape, exploration, aim, build, general, purpose, structured, additionally, show, tuning, individual, process, refer, improves, percentage, while, preserving, demonstrating, utility, geometries, refinement, finally, validate, results, arm, observing, simulated, reduce, low, recovers, corresponding, geometry, closely, matching, outcomes, abstract, code, arxiv, indicates, equal, advising, robotics, carnegie, mellon, university, national, standards, technology, helen, oral, presentation, acceptance, corl, georedteam,

Text of the page (random words):
georedteam geometric red teaming for robotic manipulation conference on robot learning corl 2025 oral presentation 5 7 acceptance divyam goel 1 yufei wang 1 tiancheng wu 1 helen qiao 2 pavel piliptchak 2 david held 1 zackory erickson 1 1 robotics institute carnegie mellon university 2 national institute of standards and technology indicates equal advising arxiv code abstract standard evaluation protocols in robotic manipulation typically assess policy performance over curated in distribution test sets offering limited insight into how systems fail under plausible variation we introduce geometric red teaming grt a red teaming framework that probes robustness through object centric geometric perturbations automatically generating crashshapes structurally valid user constrained mesh deformations that trigger catastrophic failures in pre trained manipulation policies the method integrates a jacobian field based deformation model with a gradient free simulator in the loop optimization strategy across insertion articulation and grasping tasks grt consistently discovers deformations that collapse policy performance revealing brittle failure modes missed by static benchmarks by combining task level policy rollouts with constraint aware shape exploration we aim to build a general purpose framework for structured object centric robustness evaluation in robotic manipulation we additionally show that fine tuning on individual crashshapes a process we refer to as blue teaming improves task success by up to 60 percentage points on those shapes while preserving performance on the original object demonstrating the utility of red teamed geometries for targeted policy refinement finally we validate both red teaming and blue teaming results with a real robotic arm observing that simulated crashshapes reduce task success from 90 to as low as 22 5 and that blue teaming recovers performance to up to 90 on the corresponding real world geometry closely matching simulation outcomes system overview system overview of grt given a task description and nominal object initialization parameters anchor and handle points are selected using a vision language model a handle displacements are sampled to define a population of deformation candidates each sample is converted into a perturbed mesh via jacobian field based optimization b and evaluated in simulation with a frozen policy c deformations that induce failure are sampled to guide the next population simulation demos object type task 1 grasping task 2 insertion task 3 articulation nominal objects your browser does not support the video tag your browser does not support the video tag your browser does not support the video tag crashshapes your browser does not support the video tag your browser does not support the video tag your browser does not support the video tag vlm prompting strategy two stage vlm prompting strategy for 3d handle point selection first the geometric reasoning template aligns a canonical view panel and indexed keypoints with a high level task description guiding the vlm to infer which vertices control meaningful mesh deformations next the task critical ranking template asks the model to pareto rank these candidates by plausibility and task relevance producing a compact set of handle points for targeted task aware red teaming vlm prompting examples 1 select a task choose a task red teaming a grasping policy red teaming an insertion policy red teaming a drawer opening policy 2 select an object for task choose an object stage 1 geometric reasoning stage 2 task critical ranking vlm input stage 1 canonical view panel vlm prompt input json vlm response stage 1 view full output json vlm input stage 2 vlm prompt vlm response stage 2 view output json stage 2 json content real world validation over insertion policy we tested our geometric red teaming framework on a physical xarm 6 robot using 3d printed plugs nominal cs 1 cs 2 the following videos demonstrate the effectiveness of red teaming in identifying failures and blue teaming in enhancing policy robustness tip select a thumbnail from the video strips to play the trial in the larger window 1 baseline original policy with nominal plug the pre trained policy consistently succeeds with the standard undeformed object 2 red teaming original policy fails on crashshape cs 1 the same policy consistently fails when presented with crashshape cs 1 3 red teaming original policy fails on crashshape cs 2 the policy also consistently fails when presented with crashshape cs 2 4 blue teaming on crashshape cs 1 a policy was fine tuned using cs 1 and the nominal plug it was then evaluated on both shapes performance on cs 1 with cs 1 blue teamed policy the blue teamed policy now consistently succeeds on cs 1 performance on nominal plug with cs 1 blue teamed policy performance on the nominal plug is preserved 5 blue teaming on crashshape cs 2 a separate policy was fine tuned using cs 2 and the nominal plug then evaluated performance on cs 2 with cs 2 blue teamed policy this blue teamed policy now consistently succeeds on cs 2 performance on nominal plug with cs 2 blue teamed policy performance on the nominal plug is also preserved with this policy real world validation over contact graspnet we tested our geometric red teaming framework on a physical franka emika panda robot using 3d printed objects nominal deformed pairs from the ycb dataset the following videos demonstrate the effectiveness of red teaming in identifying failures over the generalizable grasping model contact graspnet tip select a thumbnail from the video strips to play the trial in the larger window 1 baseline contact graspnet on original ycb mustard bottle the most confident grasp from contact graspnet consistently performs well on the 3d printed version of the mustard bottle taken directly from the ycb dataset 2 red teaming contact graspnet fails on deformed ycb mustard bottle the most confident grasp from contact graspnet fails on the 3d printed version of the deformed mustard bottle obtained upon geometric red teaming 3 baseline contact graspnet on original ycb screw driver the most confident grasp from contact graspnet consistently performs well on the 3d printed version of the screw driver taken directly from the ycb dataset 2 red teaming contact graspnet fails on deformed ycb screw driver the most confident grasp from contact graspnet fails on the 3d printed version of the deformed screw driver obtained upon geometric red teaming acknowledgement this material is based upon work supported by nist under grant no 70nanb24h314 and the office of naval research under the muri grant no n00014 24 1 2748 any opinions findings and conclusions or recommendations expressed in this material are those of the author s and do not necessarily reflect the views of nist or the office of naval research bibtex inproceedings goel2025geometric title geometric red teaming for robotic manipulation author goel divyam and wang yufei and wu tiancheng and qiao guixiu and piliptchak pavel and held david and erickson zackory booktitle conference on robot learning pages 41 67 year 2025 organization pmlr website template borrowed from articubot and eureka