support new flash_attn_interface

[rgtjf]

wget -P $python_path/flashattn_hopper https://github.com/Dao-AILab/flash-attention/blob/284e2c6e5beff017996d72de6e028b2dc605acf8/hopper/flash_attn_interface.py

new version https://github.com/Dao-AILab/flash-attention/blob/main/hopper/flash_attn_interface.py

[cyanguwa]

Hi @rgtjf, could you elaborate on your request please? Is there a specific change you'd like to see in Transformer Engine regarding flash-attn? At the moment, this is what we support from flash-attn v3.

[rgtjf]

@cyanguwa Thank you for your quick response. I noticed that there has been a significant refactor of the flash_attn_interface, which is causing a missing '_flash_attn_varlen_forward' error. Could you please help resolve this issue?