Is it possible to compile this model while using flash_attn_2?
#81
by
Thomas2419
- opened
As title says, due to the dynamic nature of the padding and unpadding it kept failing for me, but im wondering if I was doing something wrong or if there's a way to effectively compile this model?