Meta tags:
description=The Problem This PR addresses issue #3449: Switching between sd-v1-5-impainting and another checkpoint can result in a large amount of VRAM being consumed. Since the problem is in the TorchHijack class, this issue only occurs if you generate images with a batch size of at least 2 before switching checkpoints. The TorchHijack class is a global hijack that contains a reference to the sampler being used. Unfortunately, the sampler also contains a reference to the loaded checkpoint. Since this reference persists after image generation is complete, the checkpoint cannot be garbage collected. If shared.load_model is called when the program is in this state, it will attempt to load a second checkpoint with the previous checkpoint still in memory. The Solution Instead of keeping a reference to the entire sampler in TorchHijack, we only pass it the minimal amount of information it needs: a collection of sample noises. I ve further optimized TorchHijack by replacing the use of a one-way iterator with a deque, so the sample noises can be garbage collected immediately after they re used. Steps to Reproduce With sd-v-1-5-impainting, use txt2img to generate images with a batch size of at least 2. Switch to a non-impainting checkpoint. Your VRAM is now mostly gone.;

Headings (most frequently used words):

commented, nov, 26, 2022, jaysmithwpg, to, jn, jairo, 3449, vram, leak, when, switching, from, inpainting, checkpoint, 5065, the, footer, edited, problem, solution, conversation, steps, reproduce, progamergov, navigation,

Text of the page (most frequently used words):
the (51), this (30), that (20), 2022 (18), nov (17), github (16), from (14), all (14), #inpainting (14), #checkpoint (14), and (13), automatic1111 (12), you (11), with (11), about (11), vram (11), applied (10), code (10), jaysmithwpg (10), sign (9), something (9), switching (9), reference (9), pull (8), sorry (8), went (8), wrong (8), reactions (8), model (8), commented (8), copy (8), link (8), 3449 (8), leak (8), when (8), can (7), suggestions (7), cannot (7), issues (7), for (7), problem (7), models (7), file (7), issue (7), another (6), suggestion (6), fix (6), jump (6), request (5), not (5), batch (5), commit (5), one (5), changes (5), merged (5), use (5), yaml (5), config (5), just (5), contributor (5), being (5), your (4), security (4), while (4), has (4), change (4), account (4), said (4), jairo (4), back (4), though (4), other (4), but (4), references (4), some (4), memory (4), able (4), switch (4), without (4), script (4), torchhijack (4), open (4), 5065 (4), out (3), line (3), create (3), only (3), because (3), none (3), yet (3), projects (3), already (3), conversation (3), into (3), master (3), same (3), different (3), author (3), after (3), don (3), least (3), fixes (3), grid (3), good (3), load (3), using (3), name (3), plot (3), axis (3), management (3), consider (3), separate (3), having (3), restart (3), app (3), collaborator (3), there (3), more (3), keeping (3), review (3), impainting (3), sampler (3), contains (3), characters (3), unicode (3), actions (3), discussions (3), stable (3), diffusion (3), webui (3), repository (3), signed (2), tab (2), window (2), refresh (2), session (2), reload (2), time (2), blog (2), pricing (2), contact (2), privacy (2), terms (2), footer (2), navigation (2), add (2), single (2), closed (2), may (2), these (2), milestone (2), have (2), free (2), view (2), details (2), will (2), case (2), before (2), progamergov (2), wasn (2), related (2), edited (2), reaction (2), 0xdevalias (2), reacted (2), heart (2), emoji (2), title (2), workflow (2), changed (2), job (2), bug (2), weights (2), requires (2), generation (2), checkpoints (2), any (2), work (2), find (2), previous (2), generate (2), images (2), size (2), amount (2), sample (2), noises (2), garbage (2), collected (2), used (2), solution (2), since (2), class (2), hidden (2), bidirectional (2), community (2), project (2), insights (2), wiki (2), requests (2), source (2), manage (2), packages (2), web (2), capture (2), perform, action, training, api, docs, status, inc, queued, merge, multi, comments, pending, reviews, been, marked, resolved, outdated, must, existing, order, valid, applying, deleted, lines, supported, per, viewing, subset, invalid, were, made, participants, successfully, merging, close, development, labels, assigned, assignees, reviewers, comment, join, check, passed, 01f2ed6, hide, approved, suspect, underlying, hoping, isn, specific, anytime, was, most, common, where, happened, came, noticed, released, main, 768, might, keep, want, hold, hunt, down, very, dropped, word, take, crack, fixing, well, normally, include, art, appreciate, debugging, challenge, forgot, say, saying, should, addressed, part, finding, testing, does, releasing, sd_model, once, done, every, falls, scope, automatically, cleaned, run, trouble, lot, clean, whole, tried, great, know, are, places, like, kind, hell, removed, always, place, task, inconsistent, stability, changing, mentioned, requested, owner, c833d5b, now, mostly, gone, non, txt2img, steps, reproduce, instead, entire, pass, minimal, information, needs, collection, further, optimized, replacing, way, iterator, deque, immediately, they, persists, image, complete, shared, load_model, called, program, state, attempt, second, still, occurs, global, hijack, unfortunately, also, loaded, addresses, between, result, large, consumed, show, text, interpreted, compiled, differently, than, what, appears, below, editor, reveals, learn, files, checks, commits, bottom, clicking, agree, our, occasionally, send, emails, statement, service, password, email, address, pick, username, its, maintainers, question, new, 108, star, fork, notifications, public, message, user, suggested, results, collections, trending, topics, repositories, articles, readme, fund, developers, sponsors, resources, customer, stories, studies, devsecops, devops, automation, education, startups, teams, enterprise, solutions, skills, documentation, features, explore, collaborate, outside, plan, track, write, better, copilot, instant, dev, environments, codespaces, vulnerabilities, host, automate, product, toggle, skip, content, wayback, machine, http, archive, org, 20221130171234, https, com, timestamps, fail, success, 2023, 2021, dec, oct,

Text of the page (random words):
3449 vram leak when switching to from inpainting checkpoint by jaysmithwpg pull request 5065 automatic1111 stable diffusion webui github 1 capture 30 nov 2022 oct nov dec 30 2021 2022 2023 success fail about this capture timestamps the wayback machine http web archive org web 20221130171234 https github com automatic1111 stable diffusion webui pull 5065 skip to content toggle navigation sign up product actions automate any workflow packages host and manage packages security find and fix vulnerabilities codespaces instant dev environments copilot write better code with ai code review manage code changes issues plan and track work discussions collaborate outside of code explore all features documentation github skills blog solutions for enterprise teams startups education by solution ci cd automation devops devsecops case studies customer stories resources open source github sponsors fund open source developers the readme project github community articles repositories topics trending collections pricing in this repository all github jump to no suggested jump to results in this repository all github jump to in this user all github jump to in this repository all github jump to sign in sign up message automatic1111 stable diffusion webui public notifications fork 4 2k star 23 1k code issues 1 3k pull requests 108 discussions actions projects 0 wiki security insights more code issues pull requests discussions actions projects wiki security insights new issue have a question about this project sign up for a free github account to open an issue and contact its maintainers and the community pick a username email address password sign up for github by clicking sign up for github you agree to our terms of service and privacy statement we ll occasionally send you account related emails already on github sign in to your account jump to bottom 3449 vram leak when switching to from inpainting checkpoint 5065 merged automatic1111 merged 1 commit into automatic1111 master from jaysmithwpg vram leak nov 27 2022 merged 3449 vram leak when switching to from inpainting checkpoint 5065 automatic1111 merged 1 commit into automatic1111 master from jaysmithwpg vram leak nov 27 2022 15 18 conversation 7 commits 1 checks 1 files changed 1 conversation this file contains bidirectional unicode text that may be interpreted or compiled differently than what appears below to review open the file in an editor that reveals hidden unicode characters learn more about bidirectional unicode characters show hidden characters copy link contributor jaysmithwpg commented nov 26 2022 edited the problem this pr addresses issue 3449 switching between sd v1 5 impainting and another checkpoint can result in a large amount of vram being consumed since the problem is in the torchhijack class this issue only occurs if you generate images with a batch size of at least 2 before switching checkpoints the torchhijack class is a global hijack that contains a reference to the sampler being used unfortunately the sampler also contains a reference to the loaded checkpoint since this reference persists after image generation is complete the checkpoint cannot be garbage collected if shared load_model is called when the program is in this state it will attempt to load a second checkpoint with the previous checkpoint still in memory the solution instead of keeping a reference to the entire sampler in torchhijack we only pass it the minimal amount of information it needs a collection of sample noises i ve further optimized torchhijack by replacing the use of a one way iterator with a deque so the sample noises can be garbage collected immediately after they re used steps to reproduce with sd v 1 5 impainting use txt2img to generate images with a batch size of at least 2 switch to a non impainting checkpoint your vram is now mostly gone sorry something went wrong ️ 1 0xdevalias reacted with heart emoji all reactions ️ 1 reaction fixes 3449 vram leak when switching to from inpainting model c833d5b jaysmithwpg requested a review from automatic1111 as a code owner nov 26 2022 jaysmithwpg mentioned this pull request nov 26 2022 bug inconsistent stability when changing from v1 5 model to inpainting v1 5 model 3449 closed 1 task copy link collaborator jn jairo commented nov 26 2022 great but just for you to know there are more places keeping a reference like the xy grid script it is kind of a reference hell i already removed some references in a previous pr but there is always another place keeping a reference it is a lot of work to clean up the whole code i tried to fix the reference problem in the xy grid script but i wasn t able to find all the references in there all reactions sorry something went wrong copy link contributor author jaysmithwpg commented nov 26 2022 from my testing the xy script does a good job of releasing sd_model references once the generation is done every reference in it falls out of scope and is automatically cleaned up with this fix you can run the xy script and change checkpoints without any trouble that said using checkpoint name as an x y plot axis has some memory management issues i d consider that a separate issue though this is about being able to switch to inpainting and back without having to restart the app all reactions sorry something went wrong copy link collaborator jn jairo commented nov 26 2022 that said using checkpoint name as an x y plot axis has some memory management issues i d consider that a separate issue though this is about being able to switch to inpainting and back without having to restart the app that is the problem it is not about inpainting models it is about change to models that requires a different yaml config file because models that use the same yaml config file don t create another model it just load the other weights all reactions sorry something went wrong copy link collaborator jn jairo commented nov 26 2022 that said using checkpoint name as an x y plot axis has some memory management issues i d consider that a separate issue though this is about being able to switch to inpainting and back without having to restart the app that is the problem it is not about inpainting models it is about change to models that requires a different yaml config file because models that use the same yaml config file don t create another model it just load the other weights i forgot to say i am not saying the other references should be addressed in this pr it is ok to fix one part at a time good job finding this bug all reactions sorry something went wrong jaysmithwpg changed the title fixes 3449 vram leak when switching to from inpainting checkpoint 3449 vram leak when switching to from inpainting checkpoint nov 26 2022 copy link contributor author jaysmithwpg commented nov 26 2022 at the very least i ve dropped the word fixes from the title i ll take a crack at fixing the xy grid as well it s not something i normally include in my art workflow but i appreciate a good debugging challenge i ll keep this pr as it is though i don t want to hold up this fix while i hunt down the other one ️ 1 0xdevalias reacted with heart emoji all reactions ️ 1 reaction sorry something went wrong copy link contributor progamergov commented nov 26 2022 edited i just noticed that the vram wasn t released after switching from the main 768 model back to a 1 5 model might be related all reactions sorry something went wrong copy link contributor author jaysmithwpg commented nov 26 2022 progamergov i suspect it s the same underlying problem and i m hoping this will fix it for you as jn jairo said this problem isn t specific to inpainting models it s anytime you use a model with a different yaml config file inpainting was just the most common case where that happened before v2 came out all reactions sorry something went wrong automatic1111 approved these changes nov 27 2022 view changes hide details view details automatic1111 merged commit 01f2ed6 into automatic1111 master nov 27 2022 1 check passed sign up for free to join this conversation on github already have an account sign in to comment reviewers automatic1111 assignees no one assigned labels none yet projects none yet milestone no milestone development successfully merging this pull request may close these issues none yet 4 participants add this suggestion to a batch that can be applied as a single commit this suggestion is invalid because no changes were made to the code suggestions cannot be applied while the pull request is closed suggestions cannot be applied while viewing a subset of changes only one suggestion per line can be applied in a batch add this suggestion to a batch that can be applied as a single commit applying suggestions on deleted lines is not supported you must change the existing code in this line in order to create a valid suggestion outdated suggestions cannot be applied this suggestion has been applied or marked resolved suggestions cannot be applied from pending reviews suggestions cannot be applied on multi line comments suggestions cannot be applied while the pull request is queued to merge footer 2022 github inc footer navigation terms privacy security status docs contact github pricing api training blog about you can t perform that action at this time you signed in with another tab or window reload to refresh your session you signed out in another tab or window reload to refresh your session