MoM
Collection
9 items
•
Updated
•
2
Model of the paper MoM: Linear Sequence Modeling with Mixture-of-Memories and Retentive Network: A Successor to Transformer for Large Language Models.
The model was trained on a sample of SlimPajama with 15B tokens.
Due to changes in the MLP layer structure in the latest version of fla, the weights cannot be loaded. You can use the version at fla instead.
- Downloads last month
- 7
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
HF Inference deployability: The model has no library tag.