SDXL hi-res fix

[wikidude]

With the slow move of the user base to forge instead of a1111 (which I have tested and your bot works just fine on forge btw) SDXL models became more accessible and use less vram, allowing images of up to 1800x2400 or higher on 16gb of vram. The problem is this method requires generating the initial image at one of the SDXL predefined resolutions first (832x1216 for example) and then using hi-res fix to increase by 1.5x/2x etc. The legacy implementation of hires fix defaults the initial generation to very low resolutions around 512x512 bucketing, and then does a 3x or 4x to reach the desired resolution, making SDXL images look awful when legacy hires fix is on.
Here I am proposing two alternatives however I am not very good at coding or the actual backend of SD to help with the actual implementation.

1. add a quick command to send the initial bot-returned generation back to SD through img2img without having to rewrite the prompt or any other details. This could be called img2img upscale and it takes the same settings as the initial image and just increases the resolution by a configurable factor 1.5x, 2x etc. This method can already be done manually by using init_img but it's a very long process of saving the image or the image url, passing it back to the bot with a second /draw request together with copy pasting the same prompt, manually adding resolutions and all the other settings.
2. implement a way to have a different hiresfix XL option that uses the "modern" hires implementation instead of legacy to get around the limiting default 512x512 bucketing of hiresfix legacy.

Great bot by the way. Thank you for all the work you have done up to this point. This allows me to easily share my idle gpu with my friends without having to deal with gradio vulnerabilities, multi-account queueing and account/password sharing and at the same time allowing them to easily generate images without having to learn how webui works.

As a sidenote, SDXL works best with some predefined resolutions, so a resolution picker would be helpful in case of SDXL models. In case you require a cheatsheet for the SDXL buckets the common ones are:

1024 x 1024
1152 x 896
896 x 1152
1216 x 832
832 x 1216
1344 x 768
768 x 1344
1536 x 640
640 x 1536

This is an example of me first using /draw prompt: cat at one of the standard SDXL resolutions 832x1216

Then, I saved the image and ran a second command /draw prompt:cat init_image:cat.png width:1248 height:1824 strength:0.5

And lastly this is how it looks if I run /draw prompt:cat seed:<same as initial seed> highres_fix:4x-AnimeSharp width:1248 height:1824 (due to the initial image being generated at a lower resolution due to legacy hires fix, the seed is pretty much irrelevant anyway)

[Kilvoctu]

Heya, I never heard of Forge before, but I did use SDXL for the first time the other day. I'll check out Forge at some point, then decide where I want to go from there.

I will say I kinda want to do away with "legacy hires fix" because it feels old.

[wizz13150]

Hey,

Forge is full of "optimizations".
"Optimizations" = Quality loss
(It's not a question, but a statement.)

It's fine for low-end cards with 6-8gb, but definitely doesn't make much sense with 12gb of more (with a decent a1111 and nvidia driver config).

I don't really understand your comparison. If the image is not the same, then the problem lies elsewhere and it's obviously not generating the same 'base image', for sure.
Can you elaborate on what you're calling a "modern hires" ? 🤔 Honest question.
If it's upscaling while adding noise and then diffuse this noise, then it's basically Hires. Nothing less, nothing more.

I implemented a 'Details++' option as one of the ADetailer choices on purpose, to not complexify the options and keep 'highres_fix'.
This way i can cumulate both.
Just see this option as an img2img pass using Ultimate SD Upscale, aka an alternative to Hires. (Ignore ADetailer, as there was no detection in the following images)

---

My point is not to compare with your images, as i edited a lot of things in my fork of the bot, but more about the logic.
So, here is my comparison (RTX3060 with 12gb of VRAM. Denoizing_strenght=0.45) :

/draw prompt: A cat seed: 6969696969 batch: 1

---

/draw prompt: A cat seed: 6969696969 highres_fix: 4x_foolhardy_Remacri batch: 1
(legacy hires x1.4 at 1075x1792)

---

/draw prompt: A cat seed: 6969696969 adetailer: Details++ batch: 1
(Ultimate SD Upscale x1.4 at 1075x1792, using img2img. I think this is what any could call a 'not legacy hires' !? I don't use it this way, but at a second hires)

---

/draw prompt: A cat seed: 6969696969 adetailer: Details++ highres_fix: 4x_foolhardy_Remacri batch: 1
(Base generation at 768x1280 + Hires x1.4 (txt2img) at 1075x1792 + Ultimate SD Upscale x1.4 (img2img) at 1505x2508.
Way longer cause using "Soft Inpainting" in my img2img payload, i struggle to be seamless otherwise... Every pics using Hires + Details++ will be splitted in 4 tiles, to fit any ratio and stay in my 12gb vram. And each tile should be around ~1MPixel for SDXL, +/-30%)

---

Here are the links for the images, for a full screen comparison, in the same order :
Pic 1 - https://cdn.discordapp.com/attachments/1170605876827148349/1219228829688528997/6969696969-1.png?ex=660a8a55&is=65f81555&hm=24060db51e35491f93cce1091afa1d963a336f65d4cf16302525907b6d80b96f&
Pic 2 - https://cdn.discordapp.com/attachments/1170605876827148349/1219229471044014110/6969696969-1.png?ex=660a8aee&is=65f815ee&hm=bd487e698637f7a464e2c27e451669b61c34f7cc3e5758900629213228fb07aa&
Pic 3 - https://cdn.discordapp.com/attachments/1170605876827148349/1219230636099764274/6969696969-1.png?ex=660a8c03&is=65f81703&hm=49d3c15d40c9eb46771f1675f1096ba54255bc1d1b122c2fc8613e862b18c71f&
Pic 4 - https://cdn.discordapp.com/attachments/1170605876827148349/1219232310180970566/6969696969-1.png?ex=660a8d92&is=65f81892&hm=449ca29d1bbd9f63e5c11e053a8351d8b7531a7b7ae336c54ddb25c39506fcaa&
This is not obvious with this prompt, but this lead to very high quality images.

So, in fine, my point is that it would be more logical to be able to cumulate/choose the 'legacy hires' with anything that can be called a 'new hires'. As the legacy hires is part of the txt2img payload, not img2img.
Or at least choose between both, as above in my screenshots.

---

+1 for the ratios. I also implemented a 'size_ratio' option for /Draw and a button for /Generate.
(custom resolutions on purpose. The official ones should be used, as you mentionned) :

Cheers ! 🥂