Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add stable diffusion unit test #2496

Merged
merged 56 commits into from
Nov 14, 2023
Merged

Conversation

mrwyattii
Copy link
Contributor

@mrwyattii mrwyattii commented Nov 10, 2022

Add a unit test for Stable Diffusion without using stable-diffusion model that needs HF token.

Midjourney model does not need the HF token and has the same structure.

We verified this by printing both models. The following is the structure:

  "_class_name": "StableDiffusionPipeline",
  "_diffusers_version": "0.7.2",
  "feature_extractor": [
    "transformers",
    "CLIPFeatureExtractor"
  ],
  "safety_checker": [
    "stable_diffusion",
    "StableDiffusionSafetyChecker"
  ],
  "scheduler": [
    "diffusers",
    "PNDMScheduler"
  ],
  "text_encoder": [
    "transformers",
    "CLIPTextModel"
  ],
  "tokenizer": [
    "transformers",
    "CLIPTokenizer"
  ],
  "unet": [
    "diffusers",
    "UNet2DConditionModel"
  ],
  "vae": [
    "diffusers",
    "AutoencoderKL"
  ]
}

@cmikeh2

@Thomas-MMJ
Copy link
Contributor

You can set a manual seed, and the images should be the same,

generator = torch.Generator("cuda").manual_seed(1024)
image = pipe(prompt, guidance_scale=7.5, generator=generator).images[0]

https://huggingface.co/blog/stable_diffusion

@jeffra jeffra requested a review from loadams as a code owner June 23, 2023 21:31
.github/workflows/nv-sd.yml Outdated Show resolved Hide resolved
.github/workflows/nv-sd.yml Outdated Show resolved Hide resolved
@loadams loadams added this pull request to the merge queue Nov 13, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 13, 2023
@loadams loadams added this pull request to the merge queue Nov 13, 2023
Merged via the queue into microsoft:master with commit efd4556 Nov 14, 2023
16 checks passed
mauryaavinash95 pushed a commit to mauryaavinash95/DeepSpeed that referenced this pull request Feb 17, 2024
Add a unit test for Stable Diffusion without using stable-diffusion
model that needs HF token.

Midjourney model does not need the HF token and has the same structure.

We verified this by printing both models. The following is the
structure:

``` StableDiffusionPipeline {
  "_class_name": "StableDiffusionPipeline",
  "_diffusers_version": "0.7.2",
  "feature_extractor": [
    "transformers",
    "CLIPFeatureExtractor"
  ],
  "safety_checker": [
    "stable_diffusion",
    "StableDiffusionSafetyChecker"
  ],
  "scheduler": [
    "diffusers",
    "PNDMScheduler"
  ],
  "text_encoder": [
    "transformers",
    "CLIPTextModel"
  ],
  "tokenizer": [
    "transformers",
    "CLIPTokenizer"
  ],
  "unet": [
    "diffusers",
    "UNet2DConditionModel"
  ],
  "vae": [
    "diffusers",
    "AutoencoderKL"
  ]
}
```
@cmikeh2

---------

Co-authored-by: Olatunji Ruwase <[email protected]>
Co-authored-by: Logan Adams <[email protected]>
Co-authored-by: Logan Adams <[email protected]>
Co-authored-by: Lev Kurilenko <[email protected]>
Co-authored-by: Lev Kurilenko <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants