A second pass after upscaling (hires fix)
Lesson "hires-fix", in the group Pictures: techniques of the lesson list. Steps: 10.
What you will be able to do
The enlarged picture is drawn once more by the model: sharp detail at a size the model cannot draw in one go.
What you need
- It goes on from Upscaling with a model. The card says so; it is a hint, not a lock.
- Nodes: Load Checkpoint, CLIP Text Encode (Prompt), KSampler, VAE Decode, Save Image / Preview Image, Load Upscale Model, Upscale Image (using Model), Upscale Image, VAE Encode.
- The app downloads and installs nothing. The step texts below say which files and folders, and which ComfyUI template names them.
The steps
The words of each step, as the lesson shows them. Node names are in bold.
Why two passes (reading)
A model draws well only near the size it was trained at. A larger picture is made in two passes: draw at the usual size, enlarge, then let the model draw the large picture again with part of the noise. The composition comes from the first pass, the detail from the second.
This is called "hires fix". The lesson goes on from "Upscaling with a model".
The upscaling graph is needed
The canvas must have the picture graph, Load Checkpoint, CLIP Text Encode (Prompt), CLIP Text Encode (Prompt), KSampler, VAE Decode, Save Image, and after VAE Decode the nodes Load Upscale Model and Upscale Image (using Model).
If they are not there, do "Upscaling with a model" first.
Bring it to the size you want
Add a Upscale Image node and connect the output of Upscale Image (using Model) to its
imageinput. Setwidthandheightto twice the size of the first picture: 2048 for a 1024 picture of SDXL, 1024 for a 512 picture of SD 1.5. A 0 in one of them keeps the proportions.A 4x model gives four times the size. For the second pass twice is enough, and every extra pixel costs memory.
Turn it into a latent
Add a VAE Encode node: the output of Upscale Image into its
pixelsinput, theVAEoutput of Load Checkpoint intovae.The second sampler
Add a second KSampler node and connect to it:
- to
model, theMODELoutput of Load Checkpoint, through the LoRA loaders when there are any; - to
positiveandnegative, the same two texts; - to
latent_image, the output of VAE Encode.
The same model and the same prompt draw their own picture again, now large.
- to
Set how much to draw again
Set
denoiseof KSampler to 0.5.At 0.5 the composition stays and every detail is drawn again at the large size. At 0.3 the change is slight. Above 0.6 the model starts to draw new objects, and at a size it was not trained at it repeats them: two heads, extra windows.
Save the result
Add VAE Decode and Save Image: the output of KSampler into
samples, theVAEof Load Checkpoint intovae, the picture intoimages.The savers of the earlier passes can become Preview Image nodes, so that only the final picture is written to disk.
The other way: in the latent (reading)
ComfyUI has a second form of hires fix, without an upscale model: an Upscale Latent node between the first sampler and the second, and the second sampler at
denoise0.5. It is faster, but an enlarged latent is blurred, and the second pass must fill that in, sodenoisecannot go much lower. ComfyUI's templates show both ways.Very large pictures (reading)
When the second pass runs out of memory:
- VAE Decode (Tiled) and VAE Encode (Tiled) turn the picture into a latent and back piece by piece;
- the ComfyUI_UltimateSDUpscale pack draws a large picture again in tiles, each at the size the model likes;
- the Impact Pack does the same with its own tile nodes.
A node pack runs other people's code on your computer. Install it yourself through ComfyUI's manager, from its official page, and only if you trust its authors. The app installs nothing.
The graph is ready (reading)
Save the graph with "File ▸ Save workflow for ComfyUI…", open the file in ComfyUI and run it there. This app runs nothing in ComfyUI itself.
What to try: denoise 0.35 and 0.55 on the same seed.
What the lesson checks
A reading step is done when you press Got it, next. A doing step ticks itself off when the canvas holds:
- Step 2, The upscaling graph is needed:
- a Load Checkpoint node.
- a CLIP Text Encode (Prompt) node.
- a CLIP Text Encode (Prompt) node.
- a KSampler node.
- a VAE Decode node.
- a Save Image or Preview Image node.
- a Load Upscale Model node.
- a Upscale Image (using Model) node.
CLIPof Load Checkpoint intoclipof CLIP Text Encode (Prompt) (prompt) (a Load LoRA or CLIP Set Last Layer in between is accepted).CLIPof Load Checkpoint intoclipof CLIP Text Encode (Prompt) (negative prompt) (a Load LoRA or CLIP Set Last Layer in between is accepted).MODELof Load Checkpoint intomodelof KSampler (first pass) (a Load LoRA or LoraLoaderModelOnly or DifferentialDiffusion or ModelSamplingAuraFlow or ModelSamplingFlux or CFGNorm or IPAdapter Unified Loader or IPAdapter or IPAdapterAdvanced or IPAdapterStyleComposition in between is accepted).CONDITIONINGof CLIP Text Encode (Prompt) (prompt) intopositiveof KSampler (first pass) (a Apply ControlNet or InpaintModelConditioning or Conditioning (Combine) or ConditioningSetArea or ConditioningSetAreaPercentage or ConditioningSetMask or Apply Style Model or FluxGuidance or ConditioningZeroOut or FluxKontextMultiReferenceLatentMethod in between is accepted).CONDITIONINGof CLIP Text Encode (Prompt) (negative prompt) intonegativeof KSampler (first pass) (a Apply ControlNet or InpaintModelConditioning or Conditioning (Combine) or ConditioningSetArea or ConditioningSetAreaPercentage or ConditioningSetMask or Apply Style Model or FluxGuidance or ConditioningZeroOut or FluxKontextMultiReferenceLatentMethod in between is accepted).LATENTof KSampler (first pass) intosamplesof VAE Decode (first pass).VAEof Load Checkpoint intovaeof VAE Decode (first pass).IMAGEof VAE Decode (first pass) intoimagesof Save Image or Preview Image (first picture).UPSCALE_MODELof Load Upscale Model intoupscale_modelof Upscale Image (using Model).IMAGEof VAE Decode (first pass) intoimageof Upscale Image (using Model).
- Step 3, Bring it to the size you want:
- a Upscale Image node.
IMAGEof Upscale Image (using Model) intoimageof Upscale Image.
- Step 4, Turn it into a latent:
- a VAE Encode node.
IMAGEof Upscale Image intopixelsof VAE Encode.VAEof Load Checkpoint intovaeof VAE Encode.
- Step 5, The second sampler:
- a KSampler node.
MODELof Load Checkpoint intomodelof KSampler (second pass) (a Load LoRA or LoraLoaderModelOnly or DifferentialDiffusion or ModelSamplingAuraFlow or ModelSamplingFlux or CFGNorm or IPAdapter Unified Loader or IPAdapter or IPAdapterAdvanced or IPAdapterStyleComposition in between is accepted).CONDITIONINGof CLIP Text Encode (Prompt) (prompt) intopositiveof KSampler (second pass) (a Apply ControlNet or InpaintModelConditioning or Conditioning (Combine) or ConditioningSetArea or ConditioningSetAreaPercentage or ConditioningSetMask or Apply Style Model or FluxGuidance or ConditioningZeroOut or FluxKontextMultiReferenceLatentMethod in between is accepted).CONDITIONINGof CLIP Text Encode (Prompt) (negative prompt) intonegativeof KSampler (second pass) (a Apply ControlNet or InpaintModelConditioning or Conditioning (Combine) or ConditioningSetArea or ConditioningSetAreaPercentage or ConditioningSetMask or Apply Style Model or FluxGuidance or ConditioningZeroOut or FluxKontextMultiReferenceLatentMethod in between is accepted).LATENTof VAE Encode intolatent_imageof KSampler (second pass).
- Step 6, Set how much to draw again:
denoiseof KSampler (second pass) is from 0.3 to 0.6; Do this for me sets 0.5.
- Step 7, Save the result:
- a VAE Decode node.
- a Save Image or Preview Image node.
LATENTof KSampler (second pass) intosamplesof VAE Decode (second pass).VAEof Load Checkpoint intovaeof VAE Decode (second pass).IMAGEof VAE Decode (second pass) intoimagesof Save Image or Preview Image (second pass).