
ReactiveAI/sSQAT-mm
Text Generation
•
0.0B
•
Updated
Experimental models with Sparse Query Attention layers. Reducing training time/cost by ~3-10% compared to GQA & MQA, with the same level performance