MMTEB: Massive Multilingual Text Embedding Benchmark
Paper
•
2502.13595
•
Published
•
17
Data Is Better Together Contributor
community
AI & ML interests
None defined yet.
Recent Activity
View all activity
data-is-better-together-contributor's activity
mmhamdy
authored
2
papers
about 8 hours ago
SivilTaram
authored
a
paper
about 8 hours ago
davanstrien
posted
an
update
1 day ago
Post
1530
Hacked together a way to log trl GRPO training completions to a 🤗 dataset repo. This allows you to:
- Track rewards from multiple reward functions
- Treat the completion and rewards from training as a "proper" dataset and do EDA
- Share results for open science
The implementation is super hacky, but I'm curious if people would find this useful.
To push completions to the Hub, you just need two extra parameters:
Example dataset: davanstrien/test-logs
Colab: https://colab.research.google.com/drive/1wzBFPVthRYYTp-mEYlznLg_e_0Za1M3g
- Track rewards from multiple reward functions
- Treat the completion and rewards from training as a "proper" dataset and do EDA
- Share results for open science
The implementation is super hacky, but I'm curious if people would find this useful.
To push completions to the Hub, you just need two extra parameters:
log_completions=True
log_completions_hub_repo='your-username/repo-name'Example dataset: davanstrien/test-logs
Colab: https://colab.research.google.com/drive/1wzBFPVthRYYTp-mEYlznLg_e_0Za1M3g
alielfilali01
posted
an
update
1 day ago
Post
485
🚨 Arabic LLM Evaluation 🚨
Few models join the ranking of inceptionai/AraGen-Leaderboard Today.
The new MistralAI model, Saba, is quite impressive, Top10 ! Well done @arthurmensch and team.
Sadly Mistral did not follow its strategy about public weights this time, we hope this changes soon and we get the model with a permissive license.
We added other Mistral models and apparently, we have been sleeping on mistralai/Mistral-Large-Instruct-2411 !
Another impressive model that joined the ranking today is ALLaM-AI/ALLaM-7B-Instruct-preview. After a long wait finally ALLaM is here and it is IMPRESSIVE given its size !
ALLaM is ranked on OALL/Open-Arabic-LLM-Leaderboard as well.
Few models join the ranking of inceptionai/AraGen-Leaderboard Today.
The new MistralAI model, Saba, is quite impressive, Top10 ! Well done @arthurmensch and team.
Sadly Mistral did not follow its strategy about public weights this time, we hope this changes soon and we get the model with a permissive license.
We added other Mistral models and apparently, we have been sleeping on mistralai/Mistral-Large-Instruct-2411 !
Another impressive model that joined the ranking today is ALLaM-AI/ALLaM-7B-Instruct-preview. After a long wait finally ALLaM is here and it is IMPRESSIVE given its size !
ALLaM is ranked on OALL/Open-Arabic-LLM-Leaderboard as well.
burtenshaw
posted
an
update
2 days ago
Post
5700
AGENTS + FINETUNING! This week Hugging Face learn has a whole pathway on finetuning for agentic applications. You can follow these two courses to get knowledge on levelling up your agent game beyond prompts:
1️⃣ New Supervised Fine-tuning unit in the NLP Course https://huggingface.co/learn/nlp-course/en/chapter11/1
2️⃣New Finetuning for agents bonus module in the Agents Course https://huggingface.co/learn/agents-course/bonus-unit1/introduction
Fine-tuning will squeeze everything out of your model for how you’re using it, more than any prompt.
1️⃣ New Supervised Fine-tuning unit in the NLP Course https://huggingface.co/learn/nlp-course/en/chapter11/1
2️⃣New Finetuning for agents bonus module in the Agents Course https://huggingface.co/learn/agents-course/bonus-unit1/introduction
Fine-tuning will squeeze everything out of your model for how you’re using it, more than any prompt.
-
- 2 replies
koalazf99
authored
a
paper
2 days ago
Post
5504
🎯 Perplexity drops their FIRST open-weight model on Hugging Face: A decensored DeepSeek-R1 with full reasoning capabilities. Tested on 1000+ examples for unbiased responses.
Check it out: perplexity-ai/r1-1776
Blog post: https://perplexity.ai/hub/blog/open-sourcing-r1-1776
Check it out: perplexity-ai/r1-1776
Blog post: https://perplexity.ai/hub/blog/open-sourcing-r1-1776
- 1 reply
prithivMLmods
posted
an
update
3 days ago
Post
3705
Dino: The Minimalist Multipurpose Chat System 🌠
Agent-Dino : prithivMLmods/Agent-Dino
Github: https://github.com/PRITHIVSAKTHIUR/Agent-Dino
By default, it performs the following tasks:
{Text-to-Text Generation}, {Image-Text-Text Generation}
Agent-Dino : prithivMLmods/Agent-Dino
Github: https://github.com/PRITHIVSAKTHIUR/Agent-Dino
By default, it performs the following tasks:
{Text-to-Text Generation}, {Image-Text-Text Generation}
@image: Generates an image using Stable Diffusion xL. @3d: Generates a 3D mesh. @web: Web search agents.@rAgent: Initiates a reasoning chain using Llama mode for coding explanations. @tts1-♀, @tts2-♂: Voice generation (Female and Male voices). @yolo : Object Detection
ZennyKenny
posted
an
update
3 days ago
Post
2081
Really excited to start contributing to the SWE Arena project: https://swe-arena.com/
Led by IBM PhD fellow @terryyz , our goal is to advance research in code generation and app development by frontier LLMs.
Led by IBM PhD fellow @terryyz , our goal is to advance research in code generation and app development by frontier LLMs.
Post
2711
Inference-time scaling meets Flux.1-Dev (and others) 🔥
Presenting a simple re-implementation of "Inference-time scaling diffusion models beyond denoising steps" by Ma et al.
I did the simplest random search strategy, but results can potentially be improved with better-guided search methods.
Supports Gemini 2 Flash & Qwen2.5 as verifiers for "LLMGrading" 🤗
The steps are simple:
For each round:
1> Starting by sampling 2 starting noises with different seeds.
2> Score the generations w.r.t a metric.
3> Obtain the best generation from the current round.
If you have more compute budget, go to the next search round. Scale the noise pool (
This constitutes the random search method as done in the paper by Google DeepMind.
Code, more results, and a bunch of other stuff are in the repository. Check it out here: https://github.com/sayakpaul/tt-scale-flux/ 🤗
Presenting a simple re-implementation of "Inference-time scaling diffusion models beyond denoising steps" by Ma et al.
I did the simplest random search strategy, but results can potentially be improved with better-guided search methods.
Supports Gemini 2 Flash & Qwen2.5 as verifiers for "LLMGrading" 🤗
The steps are simple:
For each round:
1> Starting by sampling 2 starting noises with different seeds.
2> Score the generations w.r.t a metric.
3> Obtain the best generation from the current round.
If you have more compute budget, go to the next search round. Scale the noise pool (
2 ** search_round) and repeat 1 - 3.This constitutes the random search method as done in the paper by Google DeepMind.
Code, more results, and a bunch of other stuff are in the repository. Check it out here: https://github.com/sayakpaul/tt-scale-flux/ 🤗
burtenshaw
posted
an
update
4 days ago
Post
3032
NEW COURSE! We’re cooking hard on Hugging Face courses, and it’s not just agents. The NLP course is getting the same treatment with a new chapter on Supervised Fine-Tuning!
👉 Follow to get more updates https://huggingface.co/nlp-course
The new SFT chapter will guide you through these topics:
1️⃣ Chat Templates: Master the art of structuring AI conversations for consistent and helpful responses.
2️⃣ Supervised Fine-Tuning (SFT): Learn the core techniques to adapt pre-trained models to your specific outputs.
3️⃣ Low Rank Adaptation (LoRA): Discover efficient fine-tuning methods that save memory and resources.
4️⃣ Evaluation: Measure your model's performance and ensure top-notch results.
This is the first update in a series, so follow along if you’re upskilling in AI.
👉 Follow to get more updates https://huggingface.co/nlp-course
The new SFT chapter will guide you through these topics:
1️⃣ Chat Templates: Master the art of structuring AI conversations for consistent and helpful responses.
2️⃣ Supervised Fine-Tuning (SFT): Learn the core techniques to adapt pre-trained models to your specific outputs.
3️⃣ Low Rank Adaptation (LoRA): Discover efficient fine-tuning methods that save memory and resources.
4️⃣ Evaluation: Measure your model's performance and ensure top-notch results.
This is the first update in a series, so follow along if you’re upskilling in AI.
- 2 replies
ZennyKenny
posted
an
update
5 days ago
Post
1940
Okay this is pretty crazy. Snowflake has CortexAI and Uber is already teasing QueryGPT, both of which prominently feature plain text to SQL features to query your database.
I decided to see how hard it would be to put together something similar using 🤗 smolagents. Turns out, it was pretty straightforward. I managed to get it done in London Luton airport this afternoon.
ZennyKenny/sqlAgent
I decided to see how hard it would be to put together something similar using 🤗 smolagents. Turns out, it was pretty straightforward. I managed to get it done in London Luton airport this afternoon.
ZennyKenny/sqlAgent
-
- 2 replies
Post
2214
Will we soon all have our own personalized AI news agents? And what does it mean for journalism?
Just built a simple prototype based on the Hugging Face course. It lets you get customized news updates on any topic.
Not perfect yet, but you can see where things could go: we'll all be able to build personalized AI agents that curate & analyze news for each of us. And users who could decide to build custom news products for their needs, such as truly personalized newsletters or podcasts.
The implications for both readers & news organizations are significant. To name a few:
- Will news articles remain the best format for informing people?
- What monetization model will work for news organizations?
- How do you create an effective conversion funnel?
👉 Try it here: fdaudens/my-news-agent (Code is open-source)
👉 Check out the course: https://huggingface.co/learn/agents-course/unit0/introduction
Just built a simple prototype based on the Hugging Face course. It lets you get customized news updates on any topic.
Not perfect yet, but you can see where things could go: we'll all be able to build personalized AI agents that curate & analyze news for each of us. And users who could decide to build custom news products for their needs, such as truly personalized newsletters or podcasts.
The implications for both readers & news organizations are significant. To name a few:
- Will news articles remain the best format for informing people?
- What monetization model will work for news organizations?
- How do you create an effective conversion funnel?
👉 Try it here: fdaudens/my-news-agent (Code is open-source)
👉 Check out the course: https://huggingface.co/learn/agents-course/unit0/introduction
prithivMLmods
posted
an
update
5 days ago
Post
4405
The last week of Impression Craft Arts and sketches from strangerzonehf🎨🧑🏻🎨
- Collection : strangerzonehf/Flux-Ultimate-LoRA-Collection
Adapters:
+ Ld-Art : strangerzonehf/Ld-Art
+ Animeopix-Flux : strangerzonehf/Animeopix-Flux
+ Flux-Super-Paint-LoRA : strangerzonehf/Flux-Super-Paint-LoRA
+ CinematicShot-Pics-Flux : strangerzonehf/cinematicShot-Pics-Flux
+ Oil-Wall-Art-Flux : strangerzonehf/Oil-Wall-Art-Flux
+ Pixelo-Flux : strangerzonehf/Pixelo-Flux
+ Abstract-Shattered : strangerzonehf/Abstract-Shattered
+ Neon-Impressionism-Flux : strangerzonehf/Neon-Impressionism-Flux
+ NewG-Art : strangerzonehf/NewG-Art
🪧Demo : prithivMLmods/FLUX-LoRA-DLC
🤗Page : https://huggingface.co/strangerzonehf
- Collection : strangerzonehf/Flux-Ultimate-LoRA-Collection
Adapters:
+ Ld-Art : strangerzonehf/Ld-Art
+ Animeopix-Flux : strangerzonehf/Animeopix-Flux
+ Flux-Super-Paint-LoRA : strangerzonehf/Flux-Super-Paint-LoRA
+ CinematicShot-Pics-Flux : strangerzonehf/cinematicShot-Pics-Flux
+ Oil-Wall-Art-Flux : strangerzonehf/Oil-Wall-Art-Flux
+ Pixelo-Flux : strangerzonehf/Pixelo-Flux
+ Abstract-Shattered : strangerzonehf/Abstract-Shattered
+ Neon-Impressionism-Flux : strangerzonehf/Neon-Impressionism-Flux
+ NewG-Art : strangerzonehf/NewG-Art
🪧Demo : prithivMLmods/FLUX-LoRA-DLC
🤗Page : https://huggingface.co/strangerzonehf
louisbrulenaudet
posted
an
update
5 days ago
Post
2961
I am pleased to introduce my first project built upon Hugging Face’s smolagents framework, integrated with Alpaca for financial market analysis automation 🦙🤗
The project implements technical indicators such as the Relative Strength Index (RSI) and Bollinger Bands to provide momentum and volatility analysis. Market data is retrieved through the Alpaca API, enabling access to historical price information across various timeframes.
AI-powered insights are generated using Hugging Face’s inference API, facilitating the analysis of market trends through natural language processing with DuckDuckGo search integration for real-time sentiment analysis based on financial news 🦆
Link to the GitHub project: https://github.com/louisbrulenaudet/agentic-market-tool
The project implements technical indicators such as the Relative Strength Index (RSI) and Bollinger Bands to provide momentum and volatility analysis. Market data is retrieved through the Alpaca API, enabling access to historical price information across various timeframes.
AI-powered insights are generated using Hugging Face’s inference API, facilitating the analysis of market trends through natural language processing with DuckDuckGo search integration for real-time sentiment analysis based on financial news 🦆
Link to the GitHub project: https://github.com/louisbrulenaudet/agentic-market-tool
davanstrien
posted
an
update
5 days ago
Post
2151
Dataset descriptions for trending Hugging Face datasets? Powered by a Smol model
davanstrien/Smol-Hub-tldr
Post
2078
🔊 Meet Kokoro Web - Free, ML speech synthesis on your computer, that'll make you ditch paid services!
28 natural voices, unlimited generations, and WebGPU acceleration. Perfect for journalists and content creators.
Test it with full articles—sounds amazingly human! 🎯🎙️
Xenova/kokoro-web
28 natural voices, unlimited generations, and WebGPU acceleration. Perfect for journalists and content creators.
Test it with full articles—sounds amazingly human! 🎯🎙️
Xenova/kokoro-web
davanstrien
posted
an
update
7 days ago
Post
1826
How do you make 1M+ Hugging Face models & datasets more discoverable?
davanstrien/Smol-Hub-tldr!
I fine-tuned HuggingFaceTB/SmolLM2-360M to generate one-line summaries from a model or dataset README.
Its own self-description?
"A model for generating concise summaries of model & dataset cards from the Hugging Face Hub"
The goal? Make it easier to find the right models and datasets for your specific needs. It's already powering a semantic search for datasets Space.
It's still a WIP but thanks to @loubnabnl , @anton-l , @eliebak et al, for cooking such a nice base model for fine-tuning small, efficient models for specific domains and tasks. 🙏
davanstrien/Smol-Hub-tldr!
I fine-tuned HuggingFaceTB/SmolLM2-360M to generate one-line summaries from a model or dataset README.
Its own self-description?
"A model for generating concise summaries of model & dataset cards from the Hugging Face Hub"
The goal? Make it easier to find the right models and datasets for your specific needs. It's already powering a semantic search for datasets Space.
It's still a WIP but thanks to @loubnabnl , @anton-l , @eliebak et al, for cooking such a nice base model for fine-tuning small, efficient models for specific domains and tasks. 🙏