Add LoRA diverse-emoji.safetensors

[caspersimon]

Watching your video on youtube inspired me to fine-tune a model so that it is able to generate emoji skin tone variations! Thank you for all example code and explanations in this repo. It made it very easy to fine-tune the model myself.

I used the complete Apple emoji set (including all skin tone variations) to fine-tune the flux.1 dev model. I used 20.000 finetuning steps, which seems to be enough.

I have already uploaded the file to huggingface (link).

Would it be an idea to create a collection in huggingface where you add EvanZhouDev/open-genmoji, caspersimon/Diverse-Emoji (the one I fine-tuned), and future contributions? This way, contributors can keep the files on their own profiles, and your repo and the collection on huggingface can be a central place where one can easily find links to the different LoRA's. Maybe this was already your plan! I'm just not sure what you mean by "I will sync it to HuggingFace" (line 25 in this README).

If you need any more information about the LoRA or if I should change anything, please let me know!

[EvanZhouDev]

Ah, originally I was thinking that we could have various LoRAs in the main HuggingFace (i.e. send a PR over there on the HuggingFace Repo too). That way, I can tweak my download script to directly choose which of the various LoRAs to download from that repo (and it would be more centralized/easier to find). Let me know if you're fine with that. If so, I will clarify that on the README.

By the way, amazing job with the LoRA, it looks incredible!

[caspersimon]

Thanks for the reply, Evan!

That's fine! I'll also submit a PR on HuggingFace then :)

However, I think it's still useful if I'll keep the separate HuggingFace repo I already made since it's a convenient place to give some information for how the adapter was trained, prompting suggestions, example outputs, etc.. It also makes it easier to use the specific adapter with pre-existing standards like HuggingFace's inference api.

And about the download script and further development in general, I'm not sure what you has planned out already, but I was thinking and would like for you to consider the following. Given that:

• it might be useful to use specific metaprompts for specific LoRA's, and
• having separate HuggingFace repo's for different LoRA's has advantages

Maybe it's better to change some things, namely:

• add a json (or like csv) file to this repo, for example

{
    "flux-dev": {
          "REPO_ID": "EvanZhouDev/open-genmoji", 
          "FILENAME": "flux-dev.safetensors", 
          "METAPROMPT": "default",
          "README": "n/a",
          },
    "flux-dev": {
          "REPO_ID": "caspersimon/Diverse-Emoji", 
          "FILENAME": "diverse-emoji.safetensors", 
          "METAPROMPT": "METAPROMPT.md",
          "README": "README.md",
          },
}

Here, "METAPROMPT.md" would be a file stored in that HuggingFace repo. "default" could refer to the metaprompt you have already made.

• Change the download.py file so that it
  • shows the available LoRA's (from the json)
  • downloads the weights (and metaprompt and/or readme) selected by a user into their own directory (e.g., `/lora/flux-dev/')

Let me know what you think! I think this could be a feasible solution to keep things relatively centralised (in a json in this repo), whilst still keeping benefits of separate HuggingFace repo's, allowing for different metaprompts, and giving users information about the different specific LoRA's (in the readme's that can be downloaded as well). Seeing that there are already some forks of this repo, I think this could be a part of the solution to support multiple LoRA's.

Perhaps you had something different in mind, I'm interested to know! Maybe you can enable the discussions feature for this repo if you have the time to discuss :)

TL;DR:
I will send a PR on Huggingface, but I will also keep the LoRA available on its own HuggingFace page.

[EvanZhouDev]

I completely understand that you want to keep your own repository, and I am definitely okay with that. I was just thinking it would be easier to centralize everything in one place, so that it's easier to search and manage, especially given the same patterns/technologies used for training.

I was thinking we could have a METAPROMPTS folder here (See #1, which I opened a while ago), for each metaprompt. And each metaprompt would be named in the same way as the model. (For example, I would have METAPROMPTS/flux-dev.md). Then, we would have the JSON, with the separate Repo ID (so you can keep your repo), then a single "name" in my case flux-dev, which would correspond to the md file and the safetensors file. This would make my Python script much easier.

What do you think?

[caspersimon]

Sounds like the perfect solution!

For now, I did not have the time yet to prompt engineer a metaprompt that I am satisfied with for my LoRA (turns out telling an LLM to "avoid racial bias" does not really work unfortunately), but I hope to add one in the near future! For now, do you think it's possible to use the metapromt you already made to fall back on (like in general, if the metaprompt for a LoRA does not exist, then use yours)?

Thank you for understanding and all the work!

[EvanZhouDev]

Sure! I can get that set up; perhaps rename the current one to METAPROMPT/open-genmoji.md, and have it fallback onto that if none is found.

I will create a PR for all the JSON, METAPROMPTS folder, etc. and add you as a reviewer (If this project still has traction after a while, maybe I'll also automate the "metadata" Markdown file generation from the JSON)

But for now, are we good to merge this PR and the HF PR as is? I think they look great.

[caspersimon]

Sounds good! I also think we’re good to merge the PR’sOn Jan 2, 2025, at 05:19, Evan Zhou ***@***.***> wrote: Sure! I can get that set up; perhaps rename the current one to METAPROMPT/open-genmoji.md, and have it fallback onto that if none is found. I will create a PR for all the JSON, METAPROMPTS folder, etc. and add you as a reviewer (If this project still has traction after a while, maybe I'll also automate the "metadata" Markdown file generation from the JSON) But for now, are we good to merge this PR and the HF PR as is? I think they look great. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[EvanZhouDev]

Will work on this more tomorrow. Thank you so much for your contribution by the way, and Happy New Year :)

[EvanZhouDev]

@caspersimon, I've created PR #8 to organize the LoRAs. Can you please check if it's alright (and if everything is backwards compatible)?

Thank you!