What is the meaning of initial_prompt ?

[amitli1]

When using transcribe function, what is the meaning of the initial_prompt parameter ?

I read the documentation ("optional text to provide as a prompt for the first window.) and still not sure I understand the meaning.

1. What it do ?
2. How it helps for the first window ?
3. For the rest of the windows, it has no meaning?

[Jeronymous]

Whisper is an acoustic model and a language model in once.
Well, we can see it as a language model, that looks at an audio chunk (of max 30 sec) to generate text.

Like any language model, you can give Whisper a prompt, that is not just "" special token(s), but a (small) text, which can typically correspond to what has been said previously to the currently processed audio chunk.
So for the first 30 sec of audio, you can use this prompt to enforce to predict stuff, or to enforce a genre.
For instance, Whisper tends to remove speech disfluencies (like "hmm", "you know", or repetitions "I I I would like to to" -> "I would like to"). You can encourage Whisper to output disfluencies by giving a prompt with disfluencies (like "Hum, you know, I I I would like to to say that... OK").

Then, when OpenAI Whisper transcribes an audio of more than 30 sec, it takes what was described in the first ~30 sec as a prompt for the next 30 sec of audio (and the initial prompt is no more taken into consideration).

I hope that it clarifies. Do not hesitate to ask more clarification.