session crashes for an unknown reason

[federicoalegria]

I'm running whisper-diarization on Google Colab (vanilla, no mods); before yesterday, the notebook ran like a charm but i keep getting this message about the session crashing for an unknown reason. Specifically, this happens during the "Transcriping audio using Whisper and realligning timestamps using Wav2Vec2" stage.

[NeMo W 2024-10-24 15:36:03 nemo_logging:349] /usr/local/lib/python3.10/dist-packages/pyannote/audio/utils/reproducibility.py:74: ReproducibilityWarning: TensorFloat-32 (TF32) has been disabled as it might lead to reproducibility issues and lower accuracy.
    It can be re-enabled by calling
       >>> import torch
       >>> torch.backends.cuda.matmul.allow_tf32 = True
       >>> torch.backends.cudnn.allow_tf32 = True
    See https://github.com/pyannote/pyannote-audio/issues/1370 for more details.
    
      warnings.warn(

[MahmoudAshraf97]

downgrade ctranslate2 to 4.4.0

[kirahman2]

@MahmoudAshraf97 Hey this didn't resolve the issue I'm running into. I saw this thread earlier and implemented it.

[federicoalegria]

i followed the readme as suggested in #261 but got stuck:

model.bin: 100% 1.53G/1.53G [00:36<00:00, 42.1MB/s]
Unable to load any of {libcudnn_ops.so.9.1.0, libcudnn_ops.so.9.1, libcudnn_ops.so.9, libcudnn_ops.so}
Invalid handle. Cannot load symbol cudnnCreateTensorDescriptor

[MahmoudAshraf97]

Are you using colab? Uf you are using the provided notebook then it was updated to fix this issue, if not then what is your torch and ctranslate2 versions?

[federicoalegria]

yeah, i'm using colab, thanks a lot!

[zshawver]

@MahmoudAshraf97 I have also had "This kernel died..." on my first attempt to run the python code from README after following all the installation steps. The script crashed during the step to transcribe the audio. I applied the fix that you recommended in this thread

downgrade ctranslate2 to 4.4.0

At that point, I got this error in python:

RuntimeError: Library cublas64_12.dll is not found or cannot be loaded

On this thread for faster-whisper, I saw the solution to downgrade the faster-whisper version, at which point I got this error in cmd:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
whisperx 3.1.1 requires faster-whisper==1.0.0, but you have faster-whisper 0.10.1 which is incompatible.

And this error in python:
TypeError: TranscriptionOptions.new() got an unexpected keyword argument 'max_new_tokens'

I feel like at this point I'm chasing my tail toward a dead-end.

[MahmoudAshraf97]

Your problem is not related to faster whisper, but I see that you are probably using an older version if this repo so pull the latest version and the solve the cuda incompatibility with ctranslate2
If it can't find a certain library, make sure that you install it and if it's installed make sure you can find it in PATH

[zshawver]

I'm sorry, but I'm still having troubles. I followed the steps in the README exactly and just tried again to run whisperx on a .wav file in command line, where I got this warning and no output ``` warnings.warn( Could not locate cudnn_ops64_9.dll. Please make sure it is in your library path! Invalid handle. Cannot load symbol cudnnCreateTensorDescriptor ``` I'm a phd cognitive psychologist by training and am self-trained in coding and data science. This is my first attempt at pulling and using a ML model, so I still have a lot to learn, so apologies for the annoyance. I know you are busy and thank you for being so generous with your time in answering questions. But I really don't know what I don't know yet

[kirahman2]

Are you using google Colab or running locally?

On Wed, Nov 13, 2024 at 8:39 AM zshawver ***@***.***> wrote: I'm sorry, but I'm still having troubles. I followed the steps in the README exactly and just tried again to run whisperx on a .wav file in command line, where I got this warning and no output warnings.warn( Could not locate cudnn_ops64_9.dll. Please make sure it is in your library path! Invalid handle. Cannot load symbol cudnnCreateTensorDescriptor I'm a phd cognitive psychologist by training and am self-trained in coding and data science. This is my first attempt at pulling and using a ML model, so I still have a lot to learn, so apologies for the annoyance. I know you are busy and thank you for being so generous with your time in answering questions. But I really don't know what I don't know yet On Wed, Nov 13, 2024 at 9:18 AM Mahmoud Ashraf ***@***.***> wrote: > Your problem is not related to faster whisper, but I see that you are > probably using an older version if this repo so pull the latest version and > the solve the cuda incompatibility with ctranslate2 > If it can't find a certain library, make sure that you install it and if > it's installed make sure you can find it in PATH > > — > Reply to this email directly, view it on GitHub > < #260 (comment)>, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AMNJKY3MIXUMM6RMPH7PPLL2ANNSLAVCNFSM6AAAAABQRNKQQOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINZTG42TEOJZGQ> > . > You are receiving this because you commented.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <#260 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJLEPIZKOD3Q5ODHSRIQY732ANQBZAVCNFSM6AAAAABQRNKQQOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINZTHAYDQMZRGA> . You are receiving this because you commented.Message ID: ***@***.***>

-- 1. Khalid I. Rahman 2. (360) 701-5546 3. LinkedIn <https://www.linkedin.com/in/khalidrahmanseattle/>

[zshawver]

Running locally. I realized that I didn't have the cblas and cdnn installs on my machine. My IT installed and added them to path, but I'm still getting this warning in python before the "The kernel died..." Could not locate cudnn_ops64_9.dll. Please make sure it is in your library path! Invalid handle. Cannot load symbol cudnnCreateTensorDescriptor

…

On Wed, Nov 13, 2024 at 10:00 AM Khalid ***@***.***> wrote: Are you using google Colab or running locally? On Wed, Nov 13, 2024 at 8:39 AM zshawver ***@***.***> wrote: > I'm sorry, but I'm still having troubles. I followed the steps in the > README exactly and just tried again to run whisperx on a .wav file in > command line, where I got this warning and no output > > warnings.warn( > Could not locate cudnn_ops64_9.dll. Please make sure it is in your library > path! > Invalid handle. Cannot load symbol cudnnCreateTensorDescriptor > > I'm a phd cognitive psychologist by training and am self-trained in coding > and data science. This is my first attempt at pulling and using a ML > model, > so I still have a lot to learn, so apologies for the annoyance. I know you > are busy and thank you for being so generous with your time in answering > questions. But I really don't know what I don't know yet > > > > On Wed, Nov 13, 2024 at 9:18 AM Mahmoud Ashraf ***@***.***> > wrote: > > > Your problem is not related to faster whisper, but I see that you are > > probably using an older version if this repo so pull the latest version > and > > the solve the cuda incompatibility with ctranslate2 > > If it can't find a certain library, make sure that you install it and if > > it's installed make sure you can find it in PATH > > > > — > > Reply to this email directly, view it on GitHub > > < > #260 (comment)>, > > > or unsubscribe > > < > https://github.com/notifications/unsubscribe-auth/AMNJKY3MIXUMM6RMPH7PPLL2ANNSLAVCNFSM6AAAAABQRNKQQOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINZTG42TEOJZGQ> > > > . > > You are receiving this because you commented.Message ID: > > ***@***.***> > > > > — > Reply to this email directly, view it on GitHub > < #260 (comment)>, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AJLEPIZKOD3Q5ODHSRIQY732ANQBZAVCNFSM6AAAAABQRNKQQOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINZTHAYDQMZRGA> > . > You are receiving this because you commented.Message ID: > ***@***.***> > -- 1. Khalid I. Rahman 2. (360) 701-5546 3. LinkedIn <https://www.linkedin.com/in/khalidrahmanseattle/> — Reply to this email directly, view it on GitHub <#260 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMNJKY5PJJJAC3FPU4HG2LD2ANSPPAVCNFSM6AAAAABQRNKQQOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINZTHA3DIMZUGA> . You are receiving this because you commented.Message ID: ***@***.***>

[kirahman2]

@zshawver Use my dockerfile at the bottom of this comment #259 (comment)
Ask chatgpt for steps on how to install this file.

[kirahman2]

@zshawver did that work for you?