v0.9

Changelog:

• add MultiControlNet
• add MultiControlNet autodownloader/loader
• add MultiControlNet order, weight, start/end steps, internal/external mode
• add MultiControlNet/Annotator cpu offload mode
• add empty negative image condition
• add softcap image range scaler
• update model guidance fn

MultiControlNet

Added MultiControlNet.

Consider using multicontrolnet by default even if you only need 1 model. It adds more options, that are otherwise unavailable in non-multi controlnet mode, like weight or controlnet start/end steps.

How to use:

Init Settings

Got to Load up a stable -> define SD + K functions, load model -> model_version -> control_multi

use_small_controlnet - True

small_controlnet_model_path - leave empty

download_control_model - True

force_download - Enable if some files appear to be corrupt, disable if everything is ok.

You can then specify a path to your custom v1.x checkpoint, and pick one or more controlnet models via checkboxes below. Those models will be downloaded if they are not available locally. You can redefine the list and order of controlnet models later in stable-settings cell.

Controlnets and their annotators will be automatically loaded, unloaded, downloaded, etc. depending on your settings in stable-settings cell.

Runtime settings

controlnet_multimodel_mode

External or internal. Internal - sums controlnet output values before feeding those into diffusion model, external - sum outputs of one controlnet conditioned diffusion model External seems slower but smoother, uses less VRAM

External mode:
controlnet1 -> diffusion -> output1
controlnet2 -> diffusion -> output2
weighted sum(output1 + output2) -> final result

Internal mode:
weighted sum(controlnet1 + controlnet2) -> diffusion -> final result

controlnet_multimodel settings

This is a dictionary containig a list of controlnet models. Order doesn't really matter, as their results are summed up.

Format example:

controlnet_multimodel = {

"control_sd15_depth":{

"weight":1,

"start":0,

"end":0.8

},

"control_sd15_canny":

{

"weight":0,

"start":0,

"end":1

}

}

weight (only available in internal mode) - weight of the model predictions in the output
start - % of total steps at which this controlnet begins working
end - % of total steps at which this controlnet stops working

This way you can: limit effect of certain models, mix more controlnets than you can fit into VRAM by making sure only a limited number of models runs at a given step.

controlnet steps are counted relatively to total steps
You have 50 steps, 0.3 style strength. Controlnet start 0.2 end 0.8 will run from step 10 to step 40, overlapping with the actual steps taken at steps 35-40

[||||||||||] 50 steps
[-------|||] 0.3 style strength (effective steps - 0.3x50 = 15)
[--||||||--] - controlnet working range with start = 0.2 and end = 0.8, effective steps from 0.2x50 = 10 to 0.8x50 = 40

Empty negative image condition
img_zero_uncond

By default image conditioned models use same image for negative conditioning, like if you specified the same text in both positive and negative prompt. (i.e. both positive and negative image conditionings are the same)

You can use empty negative condition by enabling this

Softcap image range scaler
do_softcap

Softly clamp latent excessive values. Reduces feedback loop effect a bit

softcap_thresh

Scale down absolute values above that threshold (latents are being clamped at [-1:1] range, so 0.9 will downscale values above 0.9 to fit into that range, [-1.5:1.5] will be scaled to [-1:1], but only absolute values over 0.9 will be affected)

softcap_q

Percentile to downscale. 1-downscale full range with outliers, 0.9 - downscale only 90% values above thresh, clamp remaining 10%)

A reminder:

Changes in GUI will not be saved into the notebook, but if you run it with new settings, they will be saved to a settings.txt file as usual.

You can load settings in misc tab.

You do not need to rerun the GUI cell after changing its settings.

Local install guide:
https://github.com/Sxela/WarpFusion/blob/main/README.md

Youtube playlist with settings:
https://www.youtube.com/watch?v=wvvcWm4Snmc&list=PL2cEnissQhlCUgjnGrdvYMwUaDkGemLGq