KV Caching Explained: Optimizing Transformer Inference Efficiency
ā¢
12
Hafedh Hichri
not-lain
AI & ML interests
custom AI models with HF integration, HuggingFace fellow š¤
Recent Activity
Articles
Organizations
not-lain's activity
replied to
their
post
about 2 hours ago
seems to be working on my side, you either can read the full blogpost at https://huggingface.co/blog/not-lain/tensor-dims
or you can click on this dropdown menu which will add more text to the current blogpost
replied to
their
post
about 2 hours ago
the short version would be faster and consistent inference in the cost of more gpu consumption
posted
an
update
about 12 hours ago
Post
762
I have just released a new blogpost about kv caching and its role in inference speedup š
š https://huggingface.co/blog/not-lain/kv-caching/
some takeaways :
š https://huggingface.co/blog/not-lain/kv-caching/
some takeaways :
-
- 4 replies
upvoted
an
article
about 12 hours ago
Article
PEFT: Parameter-Efficient Fine-Tuning Methods for LLMs
By
ā¢
ā¢
9
reacted to
hexgrad's
post with ā¤ļøš„
about 14 hours ago
Post
43
hexgrad/Kokoro-82M got an upgrade! ā¬ļø More voices, more languages,
GitHub: https://github.com/hexgrad/kokoro
PyPI: https://pypi.org/project/kokoro/
pip install kokoro, and still 82M parameters.GitHub: https://github.com/hexgrad/kokoro
PyPI: https://pypi.org/project/kokoro/
- 4 replies
updated
a
Space
about 17 hours ago
upvoted
an
article
1 day ago
Article
š Build a Qwen 2.5 VL API endpoint with Hugging Face spaces and Docker!
By
ā¢
ā¢
13
updated
a
Space
1 day ago
reacted to
clem's
post with ā¤ļøš¤š„
1 day ago
Post
6198
AI is not a zero-sum game. Open-source AI is the tide that lifts all boats!
upvoted
a
collection
1 day ago
liked
a
Space
1 day ago
upvoted
an
article
2 days ago
Article
Welcome to Inference Providers on the Hub š„
ā¢
161
upvoted
a
collection
2 days ago
liked
2
models
2 days ago
upvoted
an
article
2 days ago
Article
Open-R1: a fully open reproduction of DeepSeek-R1
ā¢
433
liked
a
Space
3 days ago