Meta tags:
description=Negative Token Merging: Image-based Adversarial Feature Guidance;
keywords=token merging, negative token merging, diversity, copyright, negative prompt, diffusion models;

Headings (most frequently used words):

output, adversarial, guidance, for, improving, diversity, negative, image, we, negtome, images, of, visual, by, guiding, features, away, from, copyrighted, token, merging, based, feature, overview, introduce, training, free, approach, performing, directly, using, instead, text, prompt, above, show, its, application, two, prominent, usecases, gender, racial, each, others, top, reducing, similarity, with, characters, bottom, please, refer, below, further, applications, abstract, how, does, work, flux, sdxl, copyright, mitigation, aesthetics, and, details, cross, domain, style, bibtex, interactive, gallery,

Text of the page (most frequently used words):
#guidance (16), adversarial (15), negtome (14), the (13), and (13), using (13), output (12), for (11), #visual (10), negative (9), image (9), from (8), copyrighted (8), features (8), reference (7), based (6), can (6), images (6), prompt (6), code (5), used (5), performing (5), directly (5), diversity (5), token (4), merging (4), with (4), improving (4), characters (4), simple (4), text (4), please (3), this (3), feature (3), style (3), elements (3), above (3), without (3), aesthetics (3), diffusion (3), copyright (3), mitigation (3), similarity (3), other (3), batch (3), process (3), approach (3), away (3), website (2), source (2), you (2), singh (2), jaskirat (2), lindsey (2), shi (2), weijia (2), krishna (2), ranjay (2), choi (2), yejin (2), wei (2), pang (2), cohen (2), michael (2), gould (2), stephen (2), zheng (2), liang (2), zettlemoyer (2), luke (2), arxiv (2), work (2), useful (2), also (2), artistic (2), while (2), content (2), cross (2), domain (2), details (2), any (2), models (2), sdxl (2), flux (2), does (2), character (2), name (2), insufficient (2), helps (2), reduce (2), same (2), just (2), few (2), lines (2), overview (2), alternate (2), modality (2), generation (2), undesired (2), concepts (2), paper (2), introduce (2), training (2), free (2), when (2), racial (2), gender (2), guiding (2), university (2), project, want, reuse, their, credit, them, nerfies, licensed, under, creative, commons, attribution, sharealike, international, license, article, singh2024negtome, title, author, journal, preprint, year, 2024, find, our, your, research, consider, citing, bibtex, excluding, specific, still, mainting, desired, instance, example, user, wants, van, gogh, painting, starry, night, sketch, photo, simply, blurry, improves, finetuning, state, art, generate, even, input, not, explicitly, mention, furthermore, adding, often, better, retrieval, database, core, idea, perform, applied, transformer, block, three, step, incorporated, most, implementation, left, method, how, propose, traditional, guide, tldr, has, emerged, widely, adopted, push, first, time, explore, direction, particular, but, effective, which, performs, selectively, pushing, apart, matching, semantic, between, during, reverse, observe, that, significantly, increases, sacrificing, quality, similarly, asset, implement, uses, only, marginally, higher, alone, capture, complex, avoid, like, abstract, interactive, gallery, instead, show, its, application, two, prominent, usecases, each, others, top, reducing, bottom, refer, below, further, applications, hugginface, demo, allen, institute, australian, national, washington, koh,

Text of the page (random words):
negative token merging image based adversarial feature guidance negative token merging image based adversarial feature guidance jaskirat singh 2 lindsey li 1 weijia shi 1 ranjay krishna 1 3 yejin choi 1 pang wei koh 1 3 michael f cohen 1 stephen gould 2 liang zheng 2 luke zettlemoyer 1 1 university of washington 2 the australian national university 3 allen institute for ai paper hugginface demo code overview we introduce negtome a training free approach for performing adversarial guidance directly using images instead of text negative prompt above we show its application for two prominent usecases 1 improving output diversity e g visual gender racial by guiding features of each image away from others top 2 reducing visual similarity with copyrighted characters by guiding output features away from copyrighted images bottom please refer below for further applications interactive gallery output diversity abstract text based adversarial guidance using a negative prompt has emerged as a widely adopted approach to push the output features away from undesired concepts while useful performing adversarial guidance using text alone can be insufficient to capture complex visual concepts and avoid undesired visual elements like copyrighted characters in this paper for the first time we explore an alternate modality in this direction by performing adversarial guidance directly using visual features from a reference image or other images in a batch in particular we introduce negative token merging negtome a simple but effective training free approach which performs adversarial guidance by selectively pushing apart matching semantic features between reference and output generation during the reverse diffusion process when used w r t other images in the same batch we observe that negtome significantly increases output diversity racial gender visual without sacrificing output image quality similarly when used w r t a reference copyrighted asset negtome helps reduce visual similarity with copyrighted content by 34 57 negtome is simple to implement using just few lines of code uses only marginally higher tldr we propose an alternate modality to traditional text negative prompt based adversarial guidance by directly using visual features from a reference image to guide the generation process how does negtome work method overview up a the core idea of negtome is to perform adversarial guidance directly using visual features from a reference image or other images in the same batch b negtome is simple and can be applied in any transformer block c a simple three step process is used for performing adversarial guidance using negtome implementation left negtome can be incorporated in most diffusion models using just a few lines of code improving output diversity flux improving output diversity sdxl copyright mitigation copyright mitigation state of the art diffusion models sdxl flux can generate copyrighted characters even if the input prompt does not explicitly mention the character name furthermore performing copyright mitigation using negative prompt i e adding character name to negative prompt is often insufficient negtome helps better reduce similarity with copyrighted characters by directly using visual features from a copyrighted retrieval database for adversarial guidance improving output aesthetics and details output aesthetics simply using a blurry reference with negtome improves output aesthetics details without any finetuning cross domain adversarial guidance negtome can also be used for adversarial guidance using cross domain reference images eg sketch to photo in above adversarial guidance for style negtome can also be used for style guidance excluding specific artistic elements while still mainting desired image content for instance in above example the user wants a painting of a starry night without artistic elements from van gogh style bibtex if you find our work useful in your research please consider citing article singh2024negtome title negative token merging image based adversarial feature guidance author singh jaskirat and li lindsey and shi weijia and krishna ranjay and choi yejin and wei pang and cohen michael and gould stephen and zheng liang and zettlemoyer luke journal arxiv preprint arxiv year 2024 this website is licensed under a creative commons attribution sharealike 4 0 international license website source code based on the nerfies project if you want to reuse their source code please credit them