Control net

[jianghuyihei]

Can dpm-solver used for control-net?

[LuChengTHU]

Of course. Please check https://github.com/AUTOMATIC1111/stable-diffusion-webui , where all the samplers starting with "DPM" are dpm-solver (with different settings). As far as I known, the best are "DPM++2M" and "DPM++2M Karras", which are 2nd-order multistep dpm-solver++, w/o Karras' time step settings.

[jianghuyihei]

Of course. Please check https://github.com/AUTOMATIC1111/stable-diffusion-webui , where all the samplers starting with "DPM" are dpm-solver (with different settings). As far as I known, the best are "DPM++2M" and "DPM++2M Karras", which are 2nd-order multistep dpm-solver++, w/o Karras' time step settings.

Sorry to bother you, your results are very exciting, and I have achieved good results in unconditional generation, but I am currently encountering a problem. I trained a diffusion model by myself,it is a conditional diffsuion,can be described as F(x,t,c),the condition added as control-net(It can be analogized as a condition module),but my generative quality is poor,why?There is my code.
model_fn = model_wrapper(
eps_model,
noise_schedule,
guidance_type ="classifier-free",
condition = condition,
model_type="noise", # or "x_start" or "v" or "score"
)
dpm_solver = DPM_Solver(model_fn, noise_schedule, algorithm_type="dpmsolver++",correcting_x0_fn="dynamic_thresholding")
x_T = torch.randn(sample_num,3,512,512).to(device)
x_sample = dpm_solver.sample(
x_T,
steps=50,
order=3,
skip_type="time_uniform",
method="multistep",
)

[LuChengTHU]

Hi @jianghuyihei , for conditional sampling, please use order=2.

Moreover, could you please give me more details? e.g., can order=1 work? (which is equivalent to DDIM); what is your guidance scale? What does the image look like?

[jianghuyihei]

Hi @jianghuyihei , for conditional sampling, please use order=2.

Moreover, could you please give me more details? e.g., can order=1 work? (which is equivalent to DDIM); what is your guidance scale? What does the image look like?
Ok,fine,I do not use guidance scale,and while order = 1,step =1000,result is also bad.My date is a road inraster,size of 512*512.Unconditionally generated very well, perhaps because I added conditions?How to deal with it. My condition added as control net(i transform it as a conditional mouldule into the unet)

[LuChengTHU]

@jianghuyihei
I think the bug is not related to the solver, because order=1 is DDIM.

How do you train your conditional model? Maybe you should check the code of control net to figure out why your conditional model is bad...