Merging Two Models

[ExponentialML]

Hello, thanks for your implementation. I've scoured the internet for merging multiple LDMs into one model, but there doesn't seem to be a viable solution as of yet. Would it be possible to do so?

For example, you might have one model finetuned on scenery, and one on animals. Would merging two be a viable solution, or would some form of transfer learning be a better alternative? I've tried merging state_dict between two models, but it expands so many dimensions that I don't think it would work properly during inference.

Any insight would be greatly appreciated!

[Jack000]

not sure but maybe have a look at https://github.com/energy-based-model/Compositional-Visual-Generation-with-Composable-Diffusion-Models-PyTorch