Hugging Face's logo
Shyam Sunder Kumar's picture
3 82 85

Shyam Sunder Kumar

theainerd
darkzbaron's profile picture madhavdutta's profile picture Prthameshh's profile picture
·

AI & ML interests

Natural Language Processing

Recent Activity

upvoted a paper about 11 hours ago
SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines
reacted to cogwheelhead's post with 👍 about 13 hours ago
Me and my team have performed an in-depth investigation comparing o1 to R1 (and other reasoning models) Link: https://toloka.ai/blog/r1-is-not-on-par-with-o1-and-the-difference-is-qualitative-not-quantitative It started with us evaluating them on our own university-math benchmarks: U-MATH for problem-solving and μ-MATH for judging solution correctness (see the HF leaderboard: https://huggingface.co/spaces/toloka/u-math-leaderboard) tl;dr: R1 sure is amazing, but what we find is that it lags behind in novelty adaptation and reliability: * performance drops when updating benchmarks with fresh unseen tasks (e.g. AIME 2024 -> 2025) * R1-o1 gap widens when evaluating niche subdomains (e.g. university-specific math instead of the more common Olympiad-style contests) * same with going into altogether unconventional domains (e.g. chess) or skills (e.g. judgment instead of problem-solving) * R1 also runs into failure modes way more often (e.g. making illegal chess moves or falling into endless generation loops) Our point here is not to bash on DeepSeek — they've done exceptional work, R1 is a game-changer, and we have no intention to downplay that. R1's release is a perfect opportunity to study where all these models differ and gain understanding on how to move forward from here
liked a Space 1 day ago
hf-vision/object_detection_leaderboard
View all activity

Organizations

Neuropark's profile picture Speech Recognition Community Event Version 2's profile picture Open-Source AI Meetup's profile picture Social Post Explorers's profile picture Hugging Face Discord Community's profile picture

theainerd's activity

upvoted a paper about 11 hours ago
upvoted 2 papers 1 day ago
upvoted 2 papers 5 days ago
upvoted an article 9 days ago
view article
Article

Open-source DeepResearch – Freeing our search agents

• 1.06k
upvoted an article 10 days ago
upvoted a paper 14 days ago
upvoted an article 17 days ago
view article
Article

Welcome to Inference Providers on the Hub 🔥

• 380
upvoted an article 19 days ago
upvoted a collection 20 days ago
upvoted a paper 21 days ago
upvoted an article 24 days ago
view article
Article

Open-R1: a fully open reproduction of DeepSeek-R1

• 768
upvoted a paper 25 days ago
upvoted a collection 26 days ago
upvoted an article 26 days ago
view article
Article

We now support VLMs in smolagents!

• 84
upvoted 3 papers 26 days ago
upvoted a paper about 1 month ago