AI & ML interests

Arabic NLP, computer vision, etc.

Recent Activity

arbml's activity

not-lainย 
posted an update 12 days ago
alielfilali01ย 
posted an update about 1 month ago
view post
Post
869
๐Ÿšจ Arabic LLM Evaluation ๐Ÿšจ

Few models join the ranking of inceptionai/AraGen-Leaderboard Today.

The new MistralAI model, Saba, is quite impressive, Top10 ! Well done @arthurmensch and team.

Sadly Mistral did not follow its strategy about public weights this time, we hope this changes soon and we get the model with a permissive license.

We added other Mistral models and apparently, we have been sleeping on mistralai/Mistral-Large-Instruct-2411 !

Another impressive model that joined the ranking today is ALLaM-AI/ALLaM-7B-Instruct-preview. After a long wait finally ALLaM is here and it is IMPRESSIVE given its size !

ALLaM is ranked on OALL/Open-Arabic-LLM-Leaderboard as well.
not-lainย 
posted an update about 2 months ago
not-lainย 
posted an update 2 months ago
view post
Post
1675
we now have more than 2000 public AI models using ModelHubMixin๐Ÿค—
not-lainย 
posted an update 2 months ago
view post
Post
4044
Published a new blogpost ๐Ÿ“–
In this blogpost I have gone through the transformers' architecture emphasizing how shapes propagate throughout each layer.
๐Ÿ”— https://huggingface.co/blog/not-lain/tensor-dims
some interesting takeaways :
alielfilali01ย 
posted an update 3 months ago
view post
Post
2062
3C3H AraGen Leaderboard welcomes today deepseek-ai/DeepSeek-V3 and 12 other models (including the late gpt-3.5 ๐Ÿ’€) to the ranking of best LLMs in Arabic !


Observations:
- DeepSeek-v3 ranked 3rd and only Open model among the top 5 !

- A 14B open model ( Qwen/Qwen2.5-14B-Instruct) outperforms gpt-3.5-turbo-0125 (from last year). This shows how much we came in advancing and supporting Arabic presence within the LLM ecosystem !

- Contrary to what observed in likelihood-acc leaderboards (like OALL/Open-Arabic-LLM-Leaderboard) further finetuned models like maldv/Qwentile2.5-32B-Instruct actually decreased the performance compared to the original model Qwen/Qwen2.5-32B-Instruct.
It's worth to note that the decrease is statiscally insignificant which imply that at best, the out-domain finetuning do not really hurts the model original capabilities acquired during pretraining.
Previous work addressed this (finetuning VS pretraining) but more investigation in this regard is required (any PhDs here ? This could be your question ...)


Check out the latest rankings: inceptionai/AraGen-Leaderboard
alielfilali01ย 
posted an update 3 months ago
view post
Post
1992
~75% on the challenging GPQA with only 40M parameters ๐Ÿ”ฅ๐Ÿฅณ

GREAT ACHIEVEMENT ! Or is it ?

This new Work, "Data Laundering: Artificially Boosting Benchmark Results through Knowledge Distillation", take out the mystery about many models i personally suspected their results. Speacially on leaderboards other than the english one, Like the Open Arabic LLM Leaderbaord OALL/Open-Arabic-LLM-Leaderboard.

The authors of this work, first started by training a model on the GPQA data, which, unsurprisingly, led to the model achieving 100% performance.

Afterward, they trained what they referred to as a 'legitimate' model on legitimate data (MedMCQA). However, they introduced a distillation loss from the earlier, 'cheated' model.

What they discovered was fascinating: the knowledge of GPQA leaked through this distillation loss, even though the legitimate model was never explicitly trained on GPQA during this stage.

This raises important questions about the careful use of distillation in model training, especially when the training data is opaque. As they demonstrated, itโ€™s apparently possible to (intentionally or unintentionally) leak test data through this method.

Find out more: Data Laundering: Artificially Boosting Benchmark Results through Knowledge Distillation (2412.15255)
  • 1 reply
ยท
alielfilali01ย 
posted an update 3 months ago
view post
Post
3491
Unpopular opinion: Open Source takes courage to do !

Not everyone is brave enough to release what they have done (the way they've done it) to the wild to be judged !
It really requires a high level of "knowing wth are you doing" ! It's kind of a super power !

Cheers to the heroes here who see this!
ยท
alielfilali01ย 
posted an update 4 months ago
view post
Post
1555
Apparently i forgot to put this here !

Well, this is a bit late but consider given our recent blog a read if you are interested in Evaluation.

You don't have to be into Arabic NLP in order to read it, the main contribution we are introducing is a new evaluation measure for NLG. We made the fisrt application of this measure on Arabic for now and we will be working with colleagues from the community to expand it to other languages.

Blog:
Rethinking LLM Evaluation with 3C3H: AraGen Benchmark and Leaderboard
https://huggingface.co/blog/leaderboard-3c3h-aragen

Space:
inceptionai/AraGen-Leaderboard

Give it a read and let me know your thoughts ๐Ÿค—
not-lainย 
posted an update 4 months ago
view post
Post
2348
ever wondered how you can make an API call to a visual-question-answering model without sending an image url ๐Ÿ‘€

you can do that by converting your local image to base64 and sending it to the API.

recently I made some changes to my library "loadimg" that allows you to make converting images to base64 a breeze.
๐Ÿ”— https://github.com/not-lain/loadimg

API request example ๐Ÿ› ๏ธ:
from loadimg import load_img
from huggingface_hub import InferenceClient

# or load a local image
my_b64_img = load_img(imgPath_url_pillow_or_numpy ,output_type="base64" ) 

client = InferenceClient(api_key="hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx")

messages = [
	{
		"role": "user",
		"content": [
			{
				"type": "text",
				"text": "Describe this image in one sentence."
			},
			{
				"type": "image_url",
				"image_url": {
					"url": my_b64_img # base64 allows using images without uploading them to the web
				}
			}
		]
	}
]

stream = client.chat.completions.create(
    model="meta-llama/Llama-3.2-11B-Vision-Instruct", 
	messages=messages, 
	max_tokens=500,
	stream=True
)

for chunk in stream:
    print(chunk.choices[0].delta.content, end="")
alielfilali01ย 
posted an update 4 months ago
view post
Post
2223
Unpopular opinion : o1-preview is more stupid than 4o and Qwen2.5-72B-Instruct in extremely underrated !
  • 2 replies
ยท
alielfilali01ย 
posted an update 5 months ago
view post
Post
1729
I feel like this incredible resource hasn't gotten the attention it deserves in the community!

@clefourrier and generally the HuggingFace evaluation team put together a fantastic guidebook covering a lot about ๐—˜๐—ฉ๐—”๐—Ÿ๐—จ๐—”๐—ง๐—œ๐—ข๐—ก from basics to advanced tips.

link : https://github.com/huggingface/evaluation-guidebook

I havenโ€™t finished it yet, but i'am enjoying every piece of it so far. Huge thanks @clefourrier and the team for this invaluable resource !
  • 3 replies
ยท
alielfilali01ย 
posted an update 6 months ago
view post
Post
1852
Why nobdoy is talking about the new training corpus released by MBZUAI today.

TxT360 is +15 Trillion tokens corpus outperforming FineWeb on several metrics. Ablation studies were done up to 1T tokens.

Read blog here : LLM360/TxT360
Dataset : LLM360/TxT360
  • 2 replies
ยท
alielfilali01ย 
posted an update 6 months ago
view post
Post
2586
Don't you think we should add a tag "Evaluation" for datasets that are meant to be benchmarks and not for training ?

At least, when someone is collecting a group of datasets from an organization or let's say the whole hub can filter based on that tag and avoid somehow contaminating their "training" data.
alielfilali01ย 
posted an update 6 months ago
view post
Post
910
We need a fork feature for models and datasets similar to "Duplicate this space" in spaces ! Don't you think ?

Sometimes you just want to save something in your profile privately and work on it later without the hassle of "load_.../push_to_hub" in a code file.

I know this is super lazy ๐Ÿ˜… But it is what it is ...

tag : @victor
ยท
alielfilali01ย 
posted an update 6 months ago
view post
Post
1222
@mariagrandury (SomosNLP) and team releases the Spanish leaderboard !!!
It is impressive how they choosed to design this leaderboard and how it support 4 languages (all part of Spain ofc).

Check it out from this link :
la-leaderboard/la-leaderboard
  • 1 reply
ยท