Meta tags:
description=ChangeIt3D, ShapeTalk, language-assisted-deformations, ShapeNet, PartNet, ModelNet, neural-listener;

Headings (most frequently used words):

dataset, shapetalk, and, arrived, language, framework, for, 3d, shape, edits, deformations, news, abstract, the, changeit3d, architecture, qualitative, results, citations, acknowledgements, paper, code, supplemental, material, browse, license, download,

Text of the page (most frequently used words):
the (38), and (35), shapetalk (16), shape (13), for (12), language (11), shapes (9), use (8), are (8), that (8), this (7), #dataset (7), edits (7), with (6), work (5), panos (4), achlioptas (4), all (4), also (4), you (4), our (4), framework (4), 2023 (4), using (4), changeit3d (4), utterances (4), object (4), editing (4), their (3), neural (3), guibas (3), leonidas (3), please (3), which (3), local (3), changeit3dnet (3), geometric (3), task (3), common (3), objects (3), when (3), class (3), models (3), introduce (3), from (2), snap (2), data (2), but (2), not (2), want (2), hard (2), inproceedings (2), title (2), shapeglot (2), author (2), booktitle (2), conference (2), computer (2), vision (2), year (2), consider (2), citing (2), paper (2), was (2), deformations (2), huang (2), ian (2), sung (2), minhyuk (2), tulyakov (2), sergey (2), cvpr (2), results (2), derived (2), based (2), implicit (2), thinner (2), legs (2), fine (2), grained (2), back (2), more (2), these (2), form (2), part (2), qualitative (2), produced (2), stage (2), pretrain (2), autoencoder (2), losses (2), encoded (2), target (2), distractor (2), listener (2), pretrained (2), edit (2), within (2), latent (2), space (2), both (2), modules (2), download (2), terms (2), released (2), can (2), browse (2), corpus (2), variety (2), discriminative (2), manner (2), properties (2), top (2), interestingly (2), compared (2), have (2), similarity (2), key (2), annotator (2), chair (2), row (2), differences (2), classes (2), geometry (2), through (2), natural (2), model (2), its (2), open (2), arrived (2), funded, vannevar, bush, faculty, fellowship, arl, grant, w911nf, 0104, gift, corporation, wish, thank, advices, help, following, researchers, iro, armeni, collection, nikos, gkanatsios, listening, ahmed, abdelreheem, rendering, yan, zheng, ruojin, cai, sgf, deployment, antonia, saravanou, mingyi, relevant, discussions, menglei, chai, clip, nerf, last, least, authors, emphasize, gratitude, working, amazon, mechanical, turkers, without, whom, would, impossible, acknowledgements, achlioptas2019shapeglot, learning, differentiation, fan, judy, hawkins, robert, goodman, noah, international, iccv, 2019, previous, critical, building, analyzing, achlioptas2023shapetalk, pattern, recognition, find, useful, your, research, citations, imnet, operating, field, achieved, oftentimes, slatted, entail, high, level, complex, understanding, appears, sturdy, remarkably, does, utilize, any, explicit, prior, like, otherwise, instead, learns, solely, bias, training, referential, traditional, reconstruction, freeze, encoder, latents, classification, train, editor, module, way, consistent, instruction, minimal, locks, indicate, frozen, weights, overview, modular, architecture, first, fill, out, accepting, under, license, annotations, here, explains, large, rich, construction, some, main, typical, annotators, include, references, see, prototypical, words, right, average, higher, degree, reference, frequent, contrasting, less, similar, easy, dimensional, specifications, attributes, parts, characteristics, samples, those, shown, above, each, sub, box, between, same, enumerated, decreasing, order, importance, judgement, features, share, categories, emerge, humans, describe, leg, vase, lip, presence, lamp, clock, might, bottom, arm, thinness, across, continuous, discrete, covers, contrastive, 536k, challenging, requiring, specialized, skills, aim, facilitate, example, may, modify, make, hole, tackle, problem, promotes, enables, most, extensive, existing, describing, contains, over, half, million, con, trasting, degrees, generic, builds, generative, produce, align, output, better, deformation, description, finally, assisted, methods, reflect, desiderata, setup, note, trained, deployed, directly, bypassing, ambiguities, lifting, extant, foundation, thus, opening, new, avenue, centric, manipulation, quantitative, evaluation, metrics, arbitrary, ended, abstract, march, version, accepted, august, publicly, codebase, news, supplemental, material, code, inc, kaist, stanford, university,

Text of the page (random words):
changeit3d shapetalk a language dataset and framework for 3d shape edits and deformations panos achlioptas 1 3 ian huang 3 minhyuk sung 2 sergey tulyakov 1 leonidas guibas 3 snap inc 1 kaist 2 stanford university 3 paper dataset arrived code arrived supplemental material news august 22 2023 the codebase and the pretrained models are publicly released march 5 2023 a version of this work was accepted in cvpr 2023 abstract editing 3d geometry is a challenging task requiring specialized skills in this work we aim to facilitate the task of editing the geometry of 3d models through the use of natural language for example we may want to modify a 3d chair model to make its legs thinner or to open a hole in its back to tackle this problem in a manner that promotes open ended language use and enables fine grained shape edits we introduce the most extensive existing corpus of natural language utterances describing shape differences shapetalk shapetalk contains over half a million discriminative utterances produced by con trasting the shapes of common 3d objects for a variety of object classes and degrees of similarity we also introduce a generic framework changeit3d which builds on shapetalk and can use an arbitrary 3d generative model of shapes to produce edits that align the output better with the edit or deformation description finally we introduce metrics for the quantitative evaluation of language assisted shape editing methods that reflect key desiderata within this editing setup we note that our modules are trained and deployed directly in a latent space of 3d shapes bypassing the ambiguities of lifting 2d to 3d when using extant foundation models and thus opening a new avenue for 3d object centric manipulation through language the shapetalk dataset shapetalk covers 30 common object classes with 536k contrastive utterances samples of those utterances are shown above in each sub box shape differences between a target and a distractor object of the same class are enumerated by an annotator by decreasing order of importance in the annotator s judgement interestingly both continuous and discrete geometric features that shapes share across categories emerge in the language of shapetalk e g humans describe the thinness of a chair leg or of a vase lip top row or the presence of an arm that a lamp or a clock might have bottom row key characteristics of shapetalk shapetalk s corpus explains the shapes for a large variety of common 3d objects in a rich and by construction discriminative manner shape parts geometric attributes and dimensional specifications are some of the main typical properties that annotators include in their references see prototypical words for these properties right top interestingly when the compared objects have on average a higher degree of shape similarity all hard class part based and local reference is more frequent compared to when contrasting less similar all easy class shapes browse you can browse the shapetalk annotations here license download the shapetalk dataset is released under the shapetalk terms of use to download the shapetalk dataset please first fill out this form accepting the terms of use changeit3d architecture overview of changeit3dnet our modular framework for changeit3d task in stage 1 we pretrain a shape autoencoder for shapes using traditional reconstruction losses freeze the encoder and use the encoded latents of the target and distractor to pretrain a neural listener using classification losses in stage 2 we use the pretrained autoencoder and neural listener to train a shape editor module to edit shapes within the encoded latent space in a way that is both consistent with the language instruction and also minimal all modules with locks indicate frozen weights qualitative results qualitative edits produced by changeit3dnet the results are derived by using an imnet based ae operating with implicit shape field the achieved edits are oftentimes local e g thinner legs fine grained as in slatted back or entail high level and complex shape understanding e g it appears more sturdy remarkably these edits are derived by changeit3dnet which does not utilize any form an explicit geometric local prior of shapes part like or otherwise but instead learns solely from the implicit bias of training with referential language citations if you find our work useful in your research please consider citing inproceedings achlioptas2023shapetalk title shapetalk a language dataset and framework for 3d shape edits and deformations author achlioptas panos and huang ian and sung minhyuk and tulyakov sergey and guibas leonidas booktitle conference on computer vision and pattern recognition cvpr year 2023 also if you you use the shapetalk dataset please also consider citing our previous paper data shapeglot which was critical in building and analyzing shapetalk inproceedings achlioptas2019shapeglot title shapeglot learning language for shape differentiation author achlioptas panos and fan judy and hawkins robert and goodman noah and guibas leonidas booktitle international conference on computer vision iccv year 2019 acknowledgements this work is funded by a vannevar bush faculty fellowship an arl grant w911nf 21 2 0104 and a gift from snap corporation panos achlioptas wish to thank for their advices and help the following researchers iro armeni data collection nikos gkanatsios neural listening ahmed abdelreheem rendering yan zheng and ruojin cai sgf deployment antonia saravanou and mingyi lu relevant discussions and menglei chai clip nerf last but not least the authors want to emphasize their gratitude to all the hard working amazon mechanical turkers without whom this work would be impossible