Benchmarks nowadays focus on accuracy. It would be great if we could factor in token cost, i.e. delivering the right answer with the fewest tokens. This would motivate the training to be inference efficient.
I used to complain that models don't bother to think if a problem is worthy of reasoning, and push the burden to users. We should do better on this.
Whoa. Good point.
I guess the short answer is that they handle subjective questions better and they improve model output traceability (i.e., better understanding of what informed the model's response).
Agree with your general thoughts on reasoning models though, they aren't the best solution for every use case.
xet-team
as Llama 4 is the first major model on Hugging Face uploaded with Xet providing the backing! Every byte downloaded comes through our infrastructure.
meta-llama
team for launching on Xet!That's largely been the move by Big Tech when they open source or open weight their models I think. What they are actually releasing is just a watered-down version of the product that they are capitalizing. Even Deep Seek and other open source first teams keep their capitalized models private.
Maybe the same people who sold your data out the backdoor whilst championing your "privacy rights" on their platform by letting you block people have suddenly had a massive change of heart, but something tells me the play is just to further increase the gap between Big Tech and emerging players by watering down the market so much that only companies who already have the compute maximalist infrastructure are going to be able to train meaningful models.
Maybe I'm just a cynic though.
Definitely true about the underlying model, but the eval dataset and notebook are!
yandex
Crazy that this is a day 0 release.
Actually the model I've used is a distill of LLaMa so it meets the criteria of Free as in Freedom. Shoutout rms.
