Skip to content

Latest commit

 

History

History
55 lines (42 loc) · 3.22 KB

README.md

File metadata and controls

55 lines (42 loc) · 3.22 KB

🏔️ TIBET for Bias Detection

[ECCV 2024] TIBET: Identifying and Evaluating Biases in Text-to-Image Generative Models

Aditya Chinchure🔸, Pushkar Shukla🔸, Gaurav Bhatt, Kiri Salij, Kartik Hosanagar, Leonid Sigal, Matthew Turk (🔸 indicates equal contribution)

News

  • 🌟 Accepted to ECCV 2024!
  • 📚 Paper available on arXiv

Dataset

The dataset is available to download here: TIBET Website. This dataset contains 100 prompts, and their associated bias axes, counterfactuals, and 48 images for each prompt and counterfactual generated using Stable Diffusion 2.1.

Code

This codebase is based on the MiniGPTv2 codebase. We set this up with Stable Diffusion 2.1 to generate images (see the tibet/gen_images.py file). You may choose to use any model you like. Finally, TIBET requires you to use your own OpenAI API key (see step 4) to obtain counterfactual prompts.

We tested this on a single 48 GB Nvidia GPU. If you have a smaller GPU, consider splitting the image gen part (main.py - Line 36) and the VQA part (main.py - Line 39).

  1. Environment setup:
conda create --name tibet python=3.11
conda activate tibet
conda install cudatoolkit
pip install -r requirements.txt

  1. Update ckpt in vqa_configs/minigptv2_eval.yaml (here) to the path of the model checkpoint. The model checkpoint can be downloaded from here. We use the MiniGPT-v2 (online developing demo) configuration.

  2. [Maybe optional] Following the instructions in the MiniGPTv2 Repo, you also need to download Llama 2 weights and update the path in minigpt4/configs/models/minigpt_v2.yaml (here). It may also be possible to let Huggingface handle this, as long as you have signed into Huggingface CLI and have access to the Llama 2 7B model.

  3. Update TIBET configurations in main.py (here):

  • NUM_IMAGES = 2 - Number of images to generate per prompt and counterfactural prompt. Default in the paper is 48 images
  • IMG_DIR = 'images' - Base directory path to save images
  • OPENAI_API_KEY = os.getenv('OPENAI_API_KEY') - OpenAI API key
  • PROMPT = "A photo of a computer programmer" - Prompt to conduct bias analysis on
  • PROMPT_ID = 'P_1' - Prompt ID, which is the folder name for the images and json files. Useful when you have multiple prompts to deal with.
  1. Run the code:
python main.py

Citing our work

@inproceedings{chinchure2025tibet,
  title={TIBET: Identifying and evaluating biases in text-to-image generative models},
  author={Chinchure, Aditya and Shukla, Pushkar and Bhatt, Gaurav and Salij, Kiri and Hosanagar, Kartik and Sigal, Leonid and Turk, Matthew},
  booktitle={European Conference on Computer Vision},
  pages={429--446},
  year={2025},
  organization={Springer}
}