Hugging Face's logo Hugging Face
  • Models
  • Datasets
  • Spaces
  • Docs
  • Enterprise
  • Pricing

  • Log In
  • Sign Up
AdamF92 's Collections
RxT-Alpha Micro by Reactive AI
Sparse Query Attention (SQA) Research by Reactive AI

Sparse Query Attention (SQA) Research by Reactive AI

updated 30 days ago

Experimental models with Sparse Query Attention layers. Reducing training time/cost by ~3-10% compared to GQA & MQA, with the same level performance

Upvote
-

  • ReactiveAI/sSQAT-mm

    Text Generation • Updated about 1 month ago

  • ReactiveAI/SQAT-mm

    Text Generation • Updated about 1 month ago

  • ReactiveAI/xSQAT-mm

    Text Generation • Updated about 1 month ago

  • ReactiveAI/GQA-Ref-Micro

    Text Generation • Updated about 1 month ago

  • ReactiveAI/MQA-Ref-Micro

    Text Generation • Updated about 1 month ago

  • ReactiveAI/SQAT-m

    Text Generation • Updated May 1

  • ReactiveAI/xSQAT-m

    Text Generation • Updated May 1

  • ReactiveAI/sSQAT-m

    Text Generation • Updated May 1

  • ReactiveAI/xSMQAT-m

    Text Generation • Updated May 1
Upvote
-
  • Collection guide
  • Browse collections
Company
TOS Privacy About Jobs
Website
Models Datasets Spaces Pricing Docs