Meta tags:
description= DESCRIPTION META TAG;
keywords= KEYWORDS SHOULD BE PLACED HERE;
Headings (most frequently used words):
motion, ️rewind, real, time, egocentric, whole, body, identity, results, on, is, for, it, diffusionwith, exemplar, based, conditioning, video, abstract, colossusego, unrealego, aware, estimation, pipeline, overview, bibtex, acknowledgements, one, step, diffusion, model, tracking, using, head, mounted, cameras, causal, and, generalizable, to, unseen, lengths, making, seamlessly, applicable, driving, photorealistic, avatars, or, meshes,
Text of the page (most frequently used words):
and (22), motion (21), the (17), identity (14), #rewind (11), body (11), diffusion (10), time (10), real (9), #egocentric (9), based (9), estimation (8), for (6), motions (6), this (5), with (5), exemplar (5), conditioning (5), hand (5), step (5), which (4), lee (4), avatars (4), whole (4), estimates (4), conditioned (4), denoising (4), output (4), quality (4), existing (4), model (4), page (3), from (3), jihyun (3), one (3), optionally (3), further (3), results (3), lengths (3), causal (3), manner (3), was (2), using (2), project (2), thanks (2), models (2), codec (2), lab (2), meta (2), colossusego (2), weipeng (2), richard (2), alexander (2), wei (2), shih (2), saito (2), shunsuke (2), bai (2), shaojie (2), wang (2), sung (2), minhyuk (2), kim (2), tae (2), kyun (2), saragih (2), jason (2), cvpr (2), 2025 (2), poses (2), can (2), prior (2), enhance (2), through (2), tracking (2), seamlessly (2), meshes (2), photorealistic (2), supports (2), aware (2), end (2), propose (2), novel (2), target (2), small (2), set (2), pose (2), exemplars (2), significantly (2), inputs (2), are (2), that (2), fully (2), without (2), high (2), while (2), method (2), between (2), unseen (2), video (2), built, adopted, website, licensed, under, creative, commons, attribution, sharealike, international, license, nerfies, academic, template, insightful, discussions, she, also, carter, tiernan, help, dataset, jiye, snu, soyong, shin, cmu, acknowledgements, inproceedings, lee2025rewind, title, author, booktitle, year, bibtex, given, sequence, stereo, images, camera, first, then, upper, via, optional, inverse, kinematics, used, drive, pipeline, overview, where, style, parameterized, empirically, find, examplar, paramerization, most, effective, compared, parameterizations, height, bone, shape, parameters, more, accurate, natural, than, state, art, methods, unrealego, plausible, even, challenging, occluded, truncated, observations, especially, its, reconstructed, highly, expressive, note, these, estimated, relying, future, information, present, eal, gocentric, hole, mot, iffusion, fidelity, human, image, hands, non, acausal, due, iterative, refinement, capture, correlations, operates, enable, inference, introduce, cascaded, effectively, correlation, fast, feed, forward, distillation, enables, single, our, modified, transformer, architecture, designed, causally, enhancing, generalizability, additionally, when, available, enhances, extensive, experiments, demonstrate, outperforms, baselines, both, abstract, head, mounted, cameras, making, applicable, driving, generalizable, supplementary, paper, arxiv, kaist, imperial, college, london, ️rewind,
Text of the page (random words):
rewind ️rewind real time egocentric whole body motion diffusion with exemplar based identity conditioning jihyun lee 1 2 weipeng xu 1 alexander richard 1 shih en wei 1 shunsuke saito 1 shaojie bai 1 te li wang 1 minhyuk sung 2 tae kyun kim 2 3 jason saragih 1 codec avatars lab meta 1 kaist 2 imperial college london 3 cvpr 2025 arxiv paper supplementary video ️ rewind is a one step diffusion model for whole body motion tracking using head mounted cameras it is real time causal and generalizable to unseen motion lengths making it seamlessly applicable for driving photorealistic avatars or meshes video abstract we present ️ rewind r eal time e gocentric w hole body mot i o n d iffusion a one step diffusion model for real time high fidelity human motion estimation from egocentric image inputs while an existing method for egocentric whole body i e body and hands motion estimation is non real time and acausal due to diffusion based iterative motion refinement to capture correlations between body and hand poses rewind operates in a fully causal and real time manner to enable real time inference we introduce 1 cascaded body hand denoising diffusion which effectively models the correlation between egocentric body and hand motions in a fast feed forward manner and 2 diffusion distillation which enables high quality motion estimation with a single denoising step our denoising diffusion model is based on a modified transformer architecture designed to causally model output motions while enhancing generalizability to unseen motion lengths additionally rewind optionally supports identity conditioned motion estimation when identity prior is available to this end we propose a novel identity conditioning method based on a small set of pose exemplars of the target identity which further enhances motion estimation quality through extensive experiments we demonstrate that rewind significantly outperforms the existing baselines both with and without exemplar based identity conditioning results on colossusego rewind estimates plausible motions even from challenging egocentric inputs e g occluded or truncated observations especially its reconstructed hand motions are highly expressive note that these motions are estimated in real time and in a fully causal manner i e without relying on future information results on unrealego rewind estimates significantly more accurate and natural motions than existing state of the art methods identity aware egocentric motion estimation rewind optionally supports identity aware egocentric motion estimation to further enhance output motion quality to this end we propose novel exemplar based identity conditioning where the output motion style is conditioned on the target identity parameterized by a small set of pose exemplars we empirically find this examplar based identity paramerization is the most effective compared to existing identity parameterizations e g height bone lengths shape parameters pipeline overview given a sequence of stereo egocentric images and camera poses rewind first estimates 3d body motion and then estimates 3d hand motion conditioned on the 3d upper body motion via one step denoising diffusion the motion estimation can be optionally conditioned on the exemplar based identity prior to further enhance the output motion quality through an optional inverse kinematics step the tracking results can be seamlessly used to drive meshes or photorealistic avatars in real time bibtex inproceedings lee2025rewind title rewind real time egocentric whole body motion diffusion with exemplar based identity conditioning author lee jihyun and xu weipeng and richard alexander and wei shih en and saito shunsuke and bai shaojie and wang te li and sung minhyuk and kim tae kyun and saragih jason booktitle cvpr year 2025 acknowledgements jihyun lee thanks soyong shin cmu and jiye lee snu for the insightful discussions on motion diffusion models she also thanks carter tiernan codec avatars lab meta for the help with the colossusego dataset this page was built using the academic project page template which was adopted from the nerfies project page this website is licensed under a creative commons attribution sharealike 4 0 international license
|
