Meta tags:
description=LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting.;
keywords=LumiNet;

Headings (most frequently used words):

lighting, image, luminet, right, source, latent, intrinsics, meets, diffusion, models, for, indoor, scene, relighting, transfers, complex, conditions, from, target, top, to, left, synthesizing, relit, version, of, the, while, preserving, its, main, geometry, and, albedo, abstract, zoo, architecture, transfer, results, visual, comparison, acknowledgements, bibtex, cvpr, 2025,

Text of the page (most frequently used words):
and (32), the (31), #latent (18), image (17), #lighting (16), scene (15), from (14), for (12), target (12), luminet (10), our (9), source (8), currentbatchindex (7), light (7), intrinsic (7), relit (7), images (7), that (6), transfer (6), relighting (5), with (5), model (5), intrinsics (4), indoor (4), are (4), text (4), results (4), given (4), extrinsic (4), comparisonbatches (3), length (3), loadcomparisonbatch (3), code (3), this (3), bhattad (3), anand (3), diffusion (3), models (3), arxiv (3), thank (3), their (3), scenes (3), appear (3), bosch (3), top (3), next (3), previous (3), both (3), including (3), transfers (3), complex (3), while (3), geometry (3), albedo (3), using (3), based (3), controlnet (3), two (3), representations (3), properties (3), addeventlistener (2), click (2), website (2), title (2), training (2), author (2), zhang (2), xiao (2), forsyth (2), david (2), year (2), 2024 (2), meets (2), xing (2), xiaoyan (2), groh (2), konrad (2), sezer (2), gevers (2), theo (2), other (2), university (2), amsterdam (2), rgb (2), prompts (2), use (2), lights (2), original (2), row (2), method (2), phenomena (2), illumination (2), specular (2), highlights (2), indirect (2), interactive (2), magnifier (2), will (2), train (2), match (2), vectors (2), only (2), dimensions (2), mlp (2), input (2), condition (2), your (2), browser (2), does (2), not (2), support (2), video (2), tag (2), architecture (2), different (2), generates (2), conditions (2), processes (2), preserving (2), version (2), right (2), comparisonslider, appendchild, column, prevbatchbutton, nextbatchbutton, load, initial, batch, core, borrowed, nerfies, licensed, under, creative, commons, attribution, sharealike, international, license, inproceedings, zhang2024latent, emerge, relight, gao, william, jain, seemandhar, maire, michael, booktitle, neurips, article, xing2024luminet, karagolu, journal, preprint, 2412, 00177, bibtex, pingping, song, melis, öcal, alexander, timans, insightful, discussions, also, grateful, his, suggestion, emphasize, cannot, simply, along, valuable, comments, manuscript, additionally, providing, finally, all, participants, time, finishing, user, study, project, financially, supported, center, artificial, intelligence, allowance, consortia, knowledge, innovation, tkis, netherlands, ministry, economic, affairs, climate, policy, acknowledgements, compare, require, achieve, where, descriptions, derived, actions, like, turning, off, lamp, placement, type, visual, comparison, present, some, here, second, direct, cast, shadows, inter, reflections, effects, maintaining, about, 2500, unique, corresponding, contronet, construction, needs, dimensionality, what, expects, map, them, higher, layers, size, embedding, empty, string, focus, purely, then, take, vector, feature, first, extract, these, pretrained, multiple, various, same, real, world, zoo, unlike, traditional, which, conditional, maps, single, transferring, characteristics, experiments, demonstrate, successfully, across, varying, spatial, layouts, materials, outperforming, existing, approaches, challenging, synthesizes, captures, approach, makes, key, contributions, data, curation, strategy, stylegan, modified, further, improve, through, learned, adaptor, injects, via, cross, attention, fine, tuning, introduce, novel, leverages, generative, effective, abstract, left, synthesizing, its, main, demo, new, paper, toyota, technological, institute, chicago, bcai, karaoglu, cvpr, 2025,

Text of the page (random words):
luminet luminet latent intrinsics meets diffusion models for indoor scene relighting cvpr 2025 xiaoyan xing 1 konrad groh 2 sezer karaoglu 1 theo gevers 1 anand bhattad 3 1 university of amsterdam 2 bcai bosch 3 toyota technological institute at chicago paper arxiv code demo new your browser does not support the video tag luminet transfers complex lighting conditions from a target image top right to a source image left synthesizing a relit version of the source image right while preserving its main geometry and albedo abstract we introduce luminet a novel architecture that leverages generative models and latent intrinsic representations for effective lighting transfer given a source image and a target lighting image luminet synthesizes a relit version of the source scene that captures the target s lighting our approach makes two key contributions a data curation strategy from the stylegan based relighting model for our training and a modified diffusion based controlnet that processes both latent intrinsic properties from the source image and latent extrinsic properties from the target image we further improve lighting transfer through a learned adaptor mlp that injects the target s latent extrinsic properties via cross attention and fine tuning unlike traditional controlnet which generates images with conditional maps from a single scene luminet processes latent representations from two different images preserving geometry and albedo from the source while transferring lighting characteristics from the target experiments demonstrate that our method successfully transfers complex lighting phenomena including specular highlights and indirect illumination across scenes with varying spatial layouts and materials outperforming existing approaches on challenging indoor scenes using only images as input lighting zoo given multiple different target lights luminet generates various lighting conditions in the same real world scene light transfer results are interactive magnifier will appear original image relit image previous scene next scene architecture your browser does not support the video tag given two images source image to be relit and target lighting condition we first extract latent intrinsic image representations for both these images using the pretrained latent intrinsic model we then take the lighting latent extrinsic vector from the target lighting condition image and the latent intrinsic feature from the source image to train a latent controlnet the latent contronet construction needs to match the dimensionality of what the model expects our latent extrinsic vectors are only 16 dimensions so we map them to higher dimensions using mlp layers to match the size of the text embedding vectors we use an empty string as our text input to focus purely on image based lighting transfer we train the model with about 2500 unique images and their corresponding relit images lighting transfer results we present some of the relighting results here given a target light top row and a scene to be relit second row our method transfers complex indoor scene lighting phenomena including direct illumination specular highlights cast shadows inter reflections and other indirect effects while maintaining scene geometry and albedo light transfer results are interactive magnifier will appear target light original image relit image previous scene next scene visual comparison we compare luminet with ic light v2 rgb x and latent intrinsic both rgb x and ic light v2 require text prompts to achieve relighting where we use descriptions derived from the target lighting image including actions like turning lights on off lamp placement and scene type as text prompts previous scene next scene comparisonslider appendchild column prevbatchbutton addeventlistener click currentbatchindex currentbatchindex 1 comparisonbatches length comparisonbatches length loadcomparisonbatch currentbatchindex nextbatchbutton addeventlistener click currentbatchindex currentbatchindex 1 comparisonbatches length loadcomparisonbatch currentbatchindex load the initial batch loadcomparisonbatch currentbatchindex acknowledgements we thank pingping song melis öcal and alexander timans for their insightful discussions we are also grateful to david forsyth for his suggestion to emphasize that light in scenes cannot simply appear along with other valuable comments on our manuscript additionally we thank xiao zhang for providing the code and model for latent intrinsic finally we thank all our participants for their time in finishing our user study this project is financially supported by bosch bosch center for artificial intelligence the university of amsterdam and the allowance of top consortia for knowledge and innovation tkis from the netherlands ministry of economic affairs and climate policy bibtex luminet article xing2024luminet title luminet latent intrinsics meets diffusion models for indoor scene relighting author xing xiaoyan and groh konrad and karagolu sezer and gevers theo and bhattad anand journal arxiv preprint arxiv 2412 00177 year 2024 latent intrinsics inproceedings zhang2024latent title latent intrinsics emerge from training to relight author zhang xiao and gao william and jain seemandhar and maire michael and forsyth david and bhattad anand booktitle neurips year 2024 this website is licensed under a creative commons attribution sharealike 4 0 international license the core code of this website is borrowed from nerfies