Meta tags:
description=RPMArt;
keywords=RPMArt, Robotics, Robot Learning, Articulation, Robustness, Sim2Real;

Headings (most frequently used words):

perception, and, manipulation, simulation, real, world, on, iccv, rpmart, towards, robust, for, articulated, objects, abstract, video, framework, experimental, results, bibtex, ieee, rsj, international, conference, intelligent, robots, systems, iros, 2024, workshop, challenge, category, level, object, pose, estimation, in, the, wild, wclop, 2025,

Text of the page (most frequently used words):
the (119), and (66), roartnet (38), gamma (35), ancsh (33), point (28), pointnet (27), for (26), ours (25), pull (23), #results (23), not (21), push (21), estimation (20), noise (20), video (19), rpmart (18), joint (18), affordable (17), does (17), your (16), browser (16), support (16), tag (16), real (16), can (15), storagefurniture (15), articulation (15), with (14), that (14), objects (13), points (13), also (13), microwave (13), drawer (13), are (12), tasks (12), refrigerator (12), world (11), performance (11), robust (10), manipulation (10), articulated (10), parameters (10), washingmachine (10), perception (9), part (9), each (9), safe (9), from (9), category (9), level (9), under (8), object (8), which (8), this (7), some (7), other (7), voting (7), when (7), simulation (7), liu (6), our (6), selected (6), task (6), cloud (6), models (6), while (6), relatively (6), small (6), error (6), method (6), only (6), clouds (6), baselines (6), categories (6), added (6), grasp (5), pose (5), trials (5), manipulate (5), distractors (5), pulling (5), pushing (5), background (5), both (5), estimated (5), still (5), data (5), better (5), all (5), success (5), based (5), higher (5), network (5), tuple (5), you (4), have (4), wang (4), 2024 (4), since (4), inaccurate (4), successfully (4), these (4), sim (4), transfer (4), three (4), successful (4), show (4), tuples (4), however (4), 537 (4), 179 (4), 667 (4), 162 (4), 011 (4), afford (4), dir (4), orig (4), 317 (4), generate (4), errors (4), segmentation (4), targets (4), across (4), than (4), often (4), rates (4), prismatic (4), input (4), high (4), art (4), international (3), towards (3), robots (3), find (3), work (3), target (3), finished (3), robot (3), during (3), may (3), half (3), run (3), here (3), more (3), even (3), attributed (3), scheme (3), included (3), but (3), without (3), shows (3), 723 (3), 235 (3), 116 (3), 764 (3), 827 (3), bkg (3), distr (3), 062 (3), 331 (3), 671 (3), 545 (3), 830 (3), 795 (3), 604 (3), 522 (3), 727 (3), represent (3), six (3), using (3), synthetic (3), standard (3), lower (3), demonstrates (3), compared (3), parts (3), exhibit (3), then (3), noisy (3), 275 (3), achieves (3), interesting (3), much (3), revolute (3), add (3), direction (3), origin (3), test (3), local (3), constraint (3), learning (3), framework (3), university (3), website (2), further (2), please (2), drop (2), junbo (2), wenhai (2), qiaojun (2), yang (2), weiming (2), cewu (2), ieee (2), rsj (2), conference (2), intelligent (2), systems (2), iros (2), failed (2), trial (2), shown (2), camera (2), view (2), another (2), initial (2), due (2), condition (2), excluding (2), backgrounds (2), outperforms (2), especially (2), presence (2), unrelated (2), awareness (2), reflected (2), particularly (2), where (2), almost (2), large (2), 025 (2), 438 (2), 673 (2), 861 (2), 291 (2), 797 (2), 941 (2), 372 (2), 637 (2), 953 (2), 866 (2), 099 (2), 450 (2), 610 (2), 733 (2), 063 (2), 079 (2), 634 (2), 035 (2), 796 (2), 557 (2), 131 (2), 238 (2), 760 (2), 679 (2), 790 (2), 826 (2), 155 (2), 013 (2), 254 (2), 023 (2), 515 (2), 183 (2), 680 (2), 030 (2), 994 (2), 950 (2), 164 (2), 080 (2), 531 (2), 407 (2), 742 (2), 197 (2), 548 (2), 836 (2), 421 (2), 201 (2), 612 (2), 222 (2), 627 (2), 177 (2), 178 (2), 541 (2), 996 (2), 451 (2), 706 (2), 378 (2), 189 (2), 227 (2), 808 (2), 845 (2), 664 (2), 776 (2), 669 (2), 443 (2), 027 (2), 918 (2), 972 (2), 242 (2), 862 (2), 682 (2), 927 (2), 283 (2), 868 (2), 578 (2), 289 (2), 964 (2), 266 (2), 232 (2), 695 (2), 465 (2), 952 (2), 167 (2), 404 (2), 828 (2), 946 (2), 273 (2), 481 (2), 176 (2), 391 (2), trained (2), uniformly (2), views (2), mean (2), degradation (2), found (2), possibly (2), two (2), rely (2), observation (2), washing (2), machine (2), 711 (2), 849 (2), 505 (2), observe (2), increase (2), actually (2), there (2), exist (2), phenomena (2), first (2), addition (2), achieve (2), different (2), levels (2), averaged (2), deviation (2), methods (2), environments (2), ability (2), model (2), scenarios (2), experimental (2), primary (2), contribution (2), predict (2), several (2), neural (2), score (2), supervise (2), aware (2), given (2), feature (2), generated (2), poses (2), actions (2), obust (2), erception (2), anipulation (2), iculated (2), able (2), iccv (2), licensed, page, modified, upon, source, nerfies, creative, commons, attribution, sharealike, license, questions, feel, free, email, sjtuwjb3589635689, sjtu, edu, inproceedings, wang2024rpmart, title, author, booktitle, year, pages, 7270, 7277, doi, 1109, iros58592, 10802368, helpful, consider, citing, bibtex, captured, handle, obvious, located, base, instead, typical, list, demo, videos, closer, already, above, validate, capability, demonstrate, consecutive, random, occasionally, detaches, incomplete, suitable, example, apply, count, number, include, behaviors, like, detaching, forcefully, counterparts, has, glossy, surface, makes, prominent, select, good, filtering, out, degrade, occupies, portion, itself, 318, 963, 057, 823, 461, 217, 566, 472, 780, 705, 809, 676, 644, 067, 979, 720, 237, 016, 489, 122, 466, 782, 371, 773, 296, 960, 073, 014, 322, 312, 509, 655, 928, 934, 659, 144, 990, 914, 660, 075, 145, 874, 171, 909, 028, 528, 051, 460, 287, 856, 055, 821, 710, 563, 367, 609, 196, 510, 825, 599, 876, 199, 693, 707, 645, 193, 922, 433, 514, 096, 041, 646, 783, 880, 624, 850, 519, 565, 053, 523, 205, 937, 313, 631, 558, 088, 161, 681, 746, 715, 757, 200, 121, 215, 284, 926, 464, 902, 173, 409, 863, 975, 047, 640, 278, 202, 010, 986, 512, 482, 009, 792, 231, 233, 412, 942, 755, 999, 906, 220, 756, 263, 280, 453, 824, 337, 297, 110, 211, 588, 190, 770, 042, 302, 187, 606, 064, 188, 344, 940, 226, 759, 257, 741, 593, 728, 400, 239, 108, 639, 272, 015, 989, 090, 654, 924, 818, 468, 206, 642, 374, 500, 282, 277, 815, 045, 584, 931, 499, 165, 490, 251, 071, 207, 601, 339, 954, 032, 299, 784, 094, 260, 181, 987, 948, 292, 352, 900, 435, 993, 978, 327, 893, 564, 070, 195, 538, 314, 683, 285, 158, 330, 722, 913, 034, 470, 172, 467, 397, 379, 417, 101, 255, 745, 024, 781, 616, 582, 415, 130, 398, 105, 487, 891, 423, 150, 007, 917, 889, 547, 638, 539, 800, 864, 060, 307, 692, 729, 093, 688, 575, 617, 881, 128, 109, 077, 452, 221, 345, conditions, visualize, qualitative, color, used, visualization, red, arrows, joints, blue, present, collect, capturing, depth, images, states, randomly, throw, away, bad, cannot, capture, enough, area, deviations, stable, well, could, size, somewhat, complete, noteworthy, pertains, specifically, estimates, accurately, significantly, potential, reason, take, toy, influence, relative, significant, 250, 600, 286, 441, 488, 243, 089, 440, 497, 246, 315, 596, 992, 114, 967, 666, 767, 360, 058, 725, 143, 295, 377, 402, 449, 050, 672, 186, 408, 991, 542, 517, 349, 769, 878, 428, 156, 737, 857, 768, 758, 630, 936, 985, 491, 270, 701, 111, 974, 684, 455, 019, 580, 814, 910, 998, 938, 798, 475, 605, 340, 210, 274, 754, 619, 191, 911, 901, 166, 103, 730, 495, 573, highest, rate, except, least, comparable, slightly, worse, always, note, considered, difficult, require, grasping, unnecessary, obtain, fact, discussed, report, around, 100, per, instance, experiments, ground, truth, average, 694, respectively, stably, twelve, examples, perfect, precision, nevertheless, increasing, pronounced, increases, very, slowly, because, comprise, somehow, finish, contain, optimization, procedure, ransac, transformation, dbscan, clustering, unsolvable, accurate, partly, decreasing, trend, storage, furniture, gradually, bars, evaluate, environment, robustness, directly, applying, carefully, designed, collection, sampled, context, features, applied, structure, multiple, candidates, predicted, one, degree, freedom, ambiguity, candidate, most, votes, final, training, simulator, anygrasp, subsequent, constrained, selection, affordance, physics, guided, output, single, commonly, daily, life, essential, skills, robotic, applications, existing, insufficiently, address, struggle, bridge, gap, between, reality, thus, limiting, practical, deployment, tackle, challenges, propose, learns, estimate, bust, iculation, robustly, moreover, introduce, classification, enhance, its, finally, after, zero, shot, confirm, approach, effectiveness, achieving, state, net, abstract, code, poster, paper, hefei, technology, stanford, shanghai, jiao, tong, workshop, challenge, wild, 2025, wclop, pie, related,

Text of the page (random words):
real world scenarios to tackle these challenges we propose a framework towards r obust p erception and m anipulation for art iculated objects rpmart which learns to estimate the articulation parameters and manipulate the articulation part from the noisy point cloud our primary contribution is a ro bust art iculation net work roartnet that is able to predict both joint parameters and affordable points robustly by local feature learning and point tuple voting moreover we introduce an articulation aware classification scheme to enhance its ability for sim to real transfer finally with the estimated affordable point and articulation joint constraint the robot can generate robust actions to manipulate articulated objects after learning only from synthetic data rpmart is able to transfer zero shot to real world articulated objects experimental results confirm our approach s effectiveness with our framework achieving state of the art performance in both noise added simulation and real world environments video framework robust articulation network input single view point cloud output joint parameters and affordable point affordance based physics guided manipulation affordable grasp pose selection articulation joint constraint during training several voting targets are generated by part segmentation joint parameters and affordable points from the simulator to supervise roartnet when given the real world noisy point cloud observation roartnet can still generate robust joint parameters and affordable points estimation by point tuple voting then affordable initial grasp poses can be selected from anygrasp generated grasp poses based on the estimated affordable points and subsequent actions can be constrained by the estimated joint parameters our primary contribution is the robust articulation network which is carefully designed to be robust and sim to real by local feature learning point tuple voting and an articulation awareness scheme first a collection of point tuples are uniformly sampled from the point cloud for each point tuple we predict several voting targets with a neural network from the local context features of the point tuple further an articulation score is applied to supervise the neural network so that the network is aware of the articulation structure then we can generate multiple candidates using the predicted voting targets given the one degree of freedom ambiguity constraint the candidate joint origin joint direction and affordable point with the most votes from only point tuples with high articulation score are selected as the final estimation experimental results we evaluate rpmart in both noise added simulation and real world environments in the simulation environment we add different levels of noise to the point clouds to test the robustness of models and we also test the sim to real transfer ability of rpmart by directly applying the model trained on synthetic data to real world scenarios simulation perception joint origin estimation results joint direction estimation results affordable point estimation results we gradually add higher level of noise to the input point clouds and test the joint parameters and affordable points estimation performance lower is better error bars represent the standard deviation results are averaged across six object categories this shows that roartnet is robust to noise in the input point clouds results for each object category category joint origin joint direction affordable point microwave refrigerator safe storage furniture drawer washing machine results show that all baselines and roartnet achieve high estimation precision without noise added across all object categories nevertheless with the increasing level of noise all three baselines exhibit a pronounced increase in estimation errors while the mean estimation error of roartnet increases very slowly and the baselines also have much higher standard deviation compared to roartnet when high level of noise is added there also exist some interesting phenomena in the results the storagefurniture and drawer categories have much higher estimation errors for ancsh and gamma compared to other categories this is possibly because the storagefurniture and drawer categories comprise relatively small articulation parts while the ancsh and gamma models somehow rely on the part segmentation to finish the estimation in addition the two methods both contain an optimization based procedure i e ransac transformation estimation for ancsh and dbscan part clustering for gamma which may be unsolvable when the part segmentation is not accurate it is also interesting that the washingmachine category demonstrates partly a decreasing trend when the noise level is high simulation manipulation we also add different levels of noise to the input point clouds and report the manipulation success rates we run around 100 trials per object instance for each task higher is better results are averaged across six tasks we also run manipulation experiments using the ground truth joint parameters and affordable points and the average success rates are 96 694 and 99 627 for pulling and pushing respectively it demonstrates that rpmart can still stably manipulate articulated objects with inaccurate perception results we also show the twelve task examples finished by rpmart under noise level 2 we can observe that the perception results are still robust to noise but actually not perfect but rpmart can still successfully manipulate the articulated objects manipulation results your browser does not support the video tag pull microwave your browser does not support the video tag pull refrigerator your browser does not support the video tag pull storagefurniture prismatic your browser does not support the video tag pull safe your browser does not support the video tag pull storagefurniture revolute your browser does not support the video tag pull drawer your browser does not support the video tag push microwave your browser does not support the video tag push refrigerator your browser does not support the video tag push storagefurniture prismatic your browser does not support the video tag push safe your browser does not support the video tag push storagefurniture revolute your browser does not support the video tag push drawer results for each task tasks results tasks results pull push microwave pull push refrigerator pull push safe pull push storagefurniture revolute pull push storagefurniture prismatic pull push drawer results show that rpmart achieves the highest success rate under noise level 4 across almost all tasks except push refrigerator and we can observe the least degradation in performance of rpmart with the increase of noise actually when no noise is added rpmart only achieves comparable or even slightly worse performance than the baselines especially gamma there also exist some interesting phenomena in the results under noise level 4 pointnet based method always achieves better performance than ancsh and gamma in pushing tasks while ancsh and gamma often have better results in pulling tasks note that pulling tasks are often considered more difficult than pushing tasks since pulling tasks often require first grasping the target part while it is unnecessary for pushing tasks however ancsh and gamma often obtain higher success rates than pointnet based method without noise added in addition for pull push storagefurniture prismatic and pull push drawer under noise level 4 ancsh and gamma achieve much lower success rates which may also be attributed to the fact that these parts are relatively small as discussed in simulation perception real world perception category method error orig cm dir o afford cm microwave pointnet 4 495 3 573 9 273 5 828 15 443 4 730 ancsh 5 103 5 522 9 166 9 557 12 711 7 927 gamma 2 531 2 901 9 911 10 671 7 242 10 191 roartnet ours 3 830 2 372 5 189 3 619 6 754 3 275 refrigerator pointnet 5 210 4 274 9 605 5 340 12 475 9 505 ancsh 5 938 5 798 7 998 5 910 12 814 13 604 gamma 4 019 4 580 8 684 6 455 12 331 9 974 roartnet ours 2 111 1 701 8 491 4 270 5 849 2 797 safe pointnet 5 985 4 162 5 936 2 861 9 235 5 630 ancsh 5 167 6 758 7 706 14 275 8 505 9 768 gamma 3 179 3 857 8 156 13 737 9 062 9 667 roartnet ours 4 116 2 428 5 878 2 769 8 349 4 391 storagefurniture pointnet 7 542 4 517 8 776 4 991 10 634 4 025 ancsh 6 408 4 222 9 612 6 404 5 176 6 023 gamma 3 481 2 275 12 672 10 186 4 742 6 664 roartnet ours 4 604 2 050 9 682 5 449 7 946 3 402 drawer pointnet 8 331 3 377 7 862 5 295 10 227 4 465 ancsh 13 849 3 764 12 143 8 030 7 725 4 695 gamma 5 058 2 360 14 666 6 767 6 967 3 114 roartnet ours 5 992 3 063 11 315 5 596 7 733 5 246 washingmachine pointnet 8 845 6 795 37 497 20 679 19 972 9 440 ancsh 5 162 4 918 16 243 12 089 11 541 8 232 gamma 6 488 6 178 28 441 14 866 15 964 13 286 roartnet ours 1 578 1 201 5 600 2 711 3 250 0 669 we present perception results on real world point clouds of six articulated objects using the models trained only on synthetic data we collect data by capturing depth images for each object with 5 uniformly selected joint states each from 20 randomly selected camera views and throw away some bad views which cannot capture enough area of the target part here shows the mean estimation errors and standard deviations under the condition of excluding backgrounds and distractors lower is better we can find that roartnet demonstrates more stable performance compared to other baselines however some performance degradation is found in the storagefurniture and drawer categories for roartnet as well as for ancsh and gamma this could possibly be attributed to the relatively small size of parts in these two objects where all three models somewhat rely on part segmentation to complete the estimation another noteworthy observation pertains to the performance on washingmachine specifically only roartnet successfully estimates targets accurately while the other three baselines exhibit significantly large estimation errors we find a potential reason that we take a relatively small washing machine toy as the object then the influence of noisy points is relative significant we also visualize some qualitative results of the performance on real world articulation perception with both background and distractors included color is used only for visualization here red arrows represent the estimated articulation joints and blue points represent the estimated affordable points roartnet can still generate robust joint parameters and affordable points estimation even with the presence of some unrelated points results under other conditions category method error w o bkg w distr orig cm dir o afford cm microwave pointnet 4 221 3 345 12 868 7 452 18 109 6 077 ancsh 5 537 4 723 9 881 9 128 10 637 9 727 gamma 3 617 5 011 12 688 10 575 6 027 6 062 roartnet ours 3 729 2 093 5 545 3 692 6 680 3 307 refrigerator pointnet 4 438 3 450 8 391 4 060 17 996 9 864 ancsh 6 539 5 800 11 547 11 638 15 266 25 889 gamma 3 727 5 176 13 007 10 917 6 150 6 283 roartnet ours 2 423 1 826 9 273 4 481 6 289 2 891 safe pointnet 2 946 2 487 7 398 4 105 13 130 6 828 ancsh 5 582 7 415 5 952 8 616 10 024 11 781 gamma 4 255 6 745 8 417 15 101 9 379 11 808 roartnet ours 4 167 2 404 6 397 3 522 8 467 4 172 storagefurniture pointnet 8 470 3 952 10 673 7 465 12 671 5 034 ancsh 7 913 4 695 10 727 9 722 6 545 9 330 gamma 3 790 3 232 14 266 11 795 5 964 8 158 roartnet ours 4 683 2 285 9 953 5 314 8 195 3 538 drawer pointnet 8 070 2 994 10 079 8 011 9 893 4 564 ancsh 13 950 4 327 16 978 10 289 8 993 5 578 gamma 5 435 4 522 21 352 10 900 9 292 4 868 roartnet ours 5 987 2 948 12 260 6 181 7 604 5 094 washingmachine pointnet 7 164 5 784 38 032 7 299 22 339 7 954 ancsh 5 317 5 283 22 207 11 601 20 071 12 927 gamma 10 251 6 197 29 490 14 238 33 499 10 165 roartnet ours 1 931 1 515 6 610 3 584 3 045 0 815 category method error w bkg w o distr orig cm dir o afford cm microwave pointnet 3 836 3 277 16 282 6 682 29 862 7 500 ancsh 4 374 3 235 16 242 8 642 10 206 8 468 gamma 8 818 6 972 30 027 14 918 10 924 6 667 roartnet ours 4 090 2 654 8 795 5 443 10 669 4 989 refrigerator pointnet 9 099 6 254 8 776 4 015 37 845 8 664 ancsh 6 827 6 272 13 108 8 639 40 830 18 239 gamma 7 728 4 400 17 593 11 808 36 257 9 741 roartnet ours 1 759 1 226 13 378 4 940 22 344 13 188 safe pointnet 4 451 4 064 9 227 4 830 27 606 10 407 ancsh 4 187 3 796 8 189 6 302 22 706 22 378 gamma 4 770 4 042 12 588 11 190 28 760 21 317 roartnet ours 4 080 2 451 9 110 5 211 14 177 11 297 storagefurniture pointnet 8 824 3 337 15 280 7 453 19 263 3 996 ancsh 9 545 5 131 14 756 8 220 17 906 14 541 gamma 7 035 6 011 17 755 12 999 11 537 6 942 roartnet ours 5 412 2 317 15 231 7 233 16 178 7 671 drawer pointnet 11 941 2 792 15 317 11 009 10 482 4 331 ancsh 14 548 5 177 25 986 19 512 16 010 9 062 gamma 8 278 5 202 24 222 11 627 11 612 5 162 roartnet ours 5 421 2 640 31 047 14 201 6 975 3 863 washingmachine pointnet 13 173 5 409 28 902 36 464 21 926 7 421 ancsh 14 284 9 836 35 215 11 183 37 200 15 121 gamma 16 715 4 757 31 681 12 746 44 088 17 161 roartnet ours 2 558 1 548 30 313 6 631 6 937 7 197 category method error w bkg w distr orig cm dir o afford cm microwave pointnet 3 205 3 742 16 407 6 116 30 523 8 827 ancsh 4 531 4 053 18 080 9 565 11 850 11 519 gamma 8 880 6 624 30 164 14 783 10 646 6 950 roartnet ours 4 041 2 994 9 096 5 514 11 433 6 030 refrigerator pointnet 8 183 5 680 8 922 4 013 37 193 8 515 ancsh 7 645 6 023 14 254 8 707 42 693 16 013 gamma 8 011 4 291 18 199 12 155 36 599 9 876 roartnet ours 1 825 1 510 12 609 5 196 28 155 15 367 safe pointnet 4 563 3 826 9 821 5 710 27 856 10 055 ancsh 4 287 4 162 8 827 7 460 30 179 29 790 gamma 4 679 3 051 13 028 10 528 29 171 20 909 roartnet ours 3 874 3 145 9 723 5 760 16 075 13 660 storagefurniture pointnet 9 131 3 238 15 990 7 914 19 557 4 144 ancsh 9 035 4 796 15 634 8 659 18 934 16 764 gamma 8 079 6 764 19 655 14 928 13 063 8 537 roartnet ours 5 509 2 312 17 014 12 322 16 960 8 073 drawer pointnet 12 179 2 296 15 733 10 773 10 610 4 371 ancsh 13 466 3 782 23 667 15 122 16 450 8 489 gamma 8 099 5 016 24 720 13 237 12 067 5 979 roartnet ours 5 644 2 866 24 953 19 676 7 116 3 809 washingmachine pointnet 12 637 5 667 49 705 39 372 21 780 8 179 ancsh 13 566 9 472 33 461 12 217 37 235 14 823 gamma 16 057 4 941 32 291 12 797 41 861 16 963 roartnet ours 2 673 1 723 30 318 6 537 7 438 6 025 results show that roartnet outperforms other models even with the presence of other unrelated points it is attributed to our articulation awareness scheme which can select good point tuples for voting filtering out the point tuples from the background and distractors this is reflected particularly when distractors are included but without background where the performance of roartnet almost does not degrade however when the background is included since the background occupies a large portion of the point cloud roartnet also shows a performance drop this is reflected particularly for washingmachin...