Skip to content

Commit

Permalink
update
Browse files Browse the repository at this point in the history
  • Loading branch information
Zeqiang-Lai committed May 16, 2023
1 parent 89b7104 commit 57b169f
Show file tree
Hide file tree
Showing 6 changed files with 15 additions and 4 deletions.
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ Generate image from anything with [ImageBind](https://github.com/facebookresearc

## Audio to Image

| `assets/bird_audio.wav` | `assets/dog_audio.wav` |
| --- | --- |
| ![](assets/generated/bird_audio.png) | ![](assets/generated/dog_audio.png) |
| `assets/wav/bird_audio.wav` | `assets/wav/dog_audio.wav` | `assets/wav/cattle.wav`
| --- | --- | --- |
| ![](assets/generated/bird_audio.png) | ![](assets/generated/dog_audio.png) |![](assets/generated/cattle.png) |

```python
import imagebind
Expand All @@ -30,7 +30,7 @@ model.to(device)

# generate image
with torch.no_grad():
audio_paths=["assets/bird_audio.wav"]
audio_paths=["assets/wav/bird_audio.wav"]
embeddings = model.forward({
imagebind.ModalityType.AUDIO: imagebind.load_and_transform_audio_data(audio_paths, device),
})
Expand Down
Binary file added assets/generated/cattle.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes.
Binary file added assets/wav/cattle.wav
Binary file not shown.
File renamed without changes.
11 changes: 11 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
diffusers
torch==1.13
torchvision==0.14.0
torchaudio==0.13.0
pytorchvideo @ git+https://github.com/facebookresearch/pytorchvideo.git@28fe037d212663c6a24f373b94cc5d478c8c1a1d
timm==0.6.7
ftfy
regex
einops
fvcore
decord==0.6.0

0 comments on commit 57b169f

Please sign in to comment.