Meta tags:
Headings (most frequently used words):
controllable, animation, of, fluid, elements, in, still, images, our, method, automatically, turns, single, image, with, arrow, directions, and, mask, into, an, animated, video, abstract, overview, diagram, all, stages, high, resolution, hd, videos, other, domains, additional, results, bibtex, acknowledgements, cvpr, 2022,
Text of the page (most frequently used words):
the (45), and (23), our (12), flow (11), using (10), #elements (9), fluid (8), image (8), map (8), that (8), #animation (7), method (7), which (7), are (7), motion (7), controllable (6), mask (6), link (6), input (6), into (6), both (6), directions (6), video (5), optical (5), user (5), obtain (5), still (4), images (4), videos (4), with (4), not (4), generate (4), dense (4), frame (4), baselines (3), can (3), like (3), smoke (3), during (3), show (3), provided (3), animated (3), converted (3), simple (3), exponential (3), operations (3), further (3), refined (3), network (3), unet (3), corresponding (3), forward (3), arrow (3), from (2), mahapatra (2), aniruddha (2), kulkarni (2), kuldeep (2), cvpr (2), 2022 (2), results (2), different (2), domains (2), all (2), stages (2), comparison (2), for (2), water (2), based (2), waterfall (2), river (2), other (2), fire (2), model (2), training (2), generated (2), resolutions (2), resolution (2), figure (2), shows (2), full (2), pipeline (2), inputs (2), system (2), indicating (2), region (2), hints (2), hint (2), spade (2), test (2), time (2), instead (2), tth (2), output (2), feed (2), place (2), euler (2), integrated (2), maps (2), backward (2), used (2), perform (2), symmetric (2), splatting (2), deep (2), feature (2), space (2), dataset (2), qualitative (2), exist (2), provide (2), constant (2), their (2), website, adapted, following, template, special, thanks, simon, niklaus, gaurav, sinha, acknowledgements, inproceedings, mahapatra2021controllable, title, author, booktitle, proceedings, ieee, cvf, conference, computer, vision, pattern, recognition, year, bibtex, additonal, variations, holynski, controlled, generation, additional, only, lake, sea, but, also, has, seen, wildfire, firefighter, although, previous, example, 288x512, here, 720x1280, high, overview, diagram, conduct, extensive, experiments, publicly, available, superior, terms, quantitative, metrics, addition, animations, objects, did, set, way, synthesize, otherwise, would, real, world, propose, interactively, control, cinemagraphs, specifically, focus, have, properties, repeating, textures, continuous, taking, inspiration, prior, works, represent, such, form, this, end, allow, any, number, associated, speeds, along, regions, wants, animate, speed, values, then, representing, observe, obtained, closely, approximate, plausible, refine, computed, generative, adversarial, gan, more, realistic, devise, novel, architecture, autoregressively, future, frames, warping, features, abstract, automatically, turns, single, code, arxiv, paper, adobe, research,
Text of the page (random words):
controllable animation of fluid elements in still images controllable animation of fluid elements in still images aniruddha mahapatra kuldeep kulkarni adobe research cvpr 2022 paper arxiv code dataset our method automatically turns a single image with arrow directions and mask into an animated video abstract we propose a method to interactively control the animation of fluid elements in still images to generate cinemagraphs specifically we focus on the animation of fluid elements like water smoke fire which have the properties of repeating textures and continuous fluid motion taking inspiration from prior works we represent the motion of such fluid elements in the image in the form of a constant 2d optical flow map to this end we allow the user to provide any number of arrow directions and their associated speeds along with a mask of the regions the user wants to animate the user provided input arrow directions their corresponding speed values and the mask are then converted into a dense flow map representing a constant optical flow map f d we observe that f d obtained using simple exponential operations can closely approximate the plausible motion of elements in the image we further refine computed dense optical flow map f d using a generative adversarial network gan to obtain a more realistic flow map we devise a novel unet based architecture to autoregressively generate future frames using the refined optical flow map by forward warping the input image features at different resolutions we conduct extensive experiments on a publicly available dataset and show that our method is superior to the baselines in terms of qualitative and quantitative metrics in addition we show the qualitative animations of the objects in directions that did not exist in the training set and provide a way to synthesize videos that otherwise would not exist in the real world overview diagram the figure shows our full pipeline the inputs to our system are the input image the user provided mask indicating the region to be animated and motion hints f s the motion hint is converted into a dense flow map f d using simple exponential operations on f s which is further refined using a spade network g f to obtain f rd during test time instead of using both i 0 and i n we obtain the tth frame it as the output of the unet into which we feed the input image in place of both i 0 and i n and the euler integrated flow maps corresponding to that frame in both forward and backward directions that are used to perform symmetric splatting in deep feature space all stages the figure shows our full pipeline the inputs to our system are the input image the user provided mask indicating the region to be animated and motion hints f s the motion hint is converted into a dense flow map f d using simple exponential operations on f s which is further refined using a spade network g f to obtain f rd during test time instead of using both i 0 and i n we obtain the tth frame it as the output of the unet into which we feed the input image in place of both i 0 and i n and the euler integrated flow maps corresponding to that frame in both forward and backward directions that are used to perform symmetric splatting in deep feature space controllable animation high resolution hd videos river waterfall although the previous example video are generated at 288x512 our model can generate videos at ay resolutions here we show 2 video which are generated at 720x1280 resolution hd other domains firefighter wildfire smoke our method can not only generate animation for water based elements like waterfall lake river and sea but also for other fluid elements like smoke and fire which our model has not seen during training additional results controlled video generation using our method link optical flow comparison with controllable baselines link video comparison with controllable baselines link our method all stages link our method v s holynski et al videos link additonal results on mask variations and different domains link bibtex inproceedings mahapatra2021controllable title controllable animation of fluid elements in still images author mahapatra aniruddha and kulkarni kuldeep booktitle proceedings of the ieee cvf conference on computer vision and pattern recognition cvpr year 2022 acknowledgements special thanks to gaurav sinha and simon niklaus website adapted from the following template
|
