kernels-community

/

vllm-flash-attn3

like 13

Follow

kernels-community 151

Hi, thanks for all the amazing work!

I noticed that a new s_aux parameter has been added to the fwd pass to support attention sinks. However, I wasn't able to find any related changes in the backward pass.

Does the current implementation support training as well? if not, are there any plans to add support for attention sinks to the bwd pass?

Many thanks,
Vlad