Article
πΊπ¦ββ¬ LLM Comparison/Test: Phi-4, Qwen2 VL 72B Instruct, Aya Expanse 32B in my updated MMLU-Pro CS benchmark
By
β’
β’
4Benhao Tang PRO
benhaotang
AI & ML interests
Physics Master student in theoretical particle physics at UniversitΓ€t Heidelberg, actively looking into the possibilities of integrating AI into future physics research.
Recent Activity
upvoted
an
article
about 10 hours ago
πΊπ¦ββ¬ LLM Comparison/Test: Phi-4, Qwen2 VL 72B Instruct, Aya Expanse 32B in my updated MMLU-Pro CS benchmark
liked
a model
about 11 hours ago
arcee-ai/Virtuoso-Lite
liked
a model
about 11 hours ago
mradermacher/Virtuoso-Lite-GGUF
Organizations
None yet
benhaotang's activity
upvoted
an
article
about 10 hours ago
liked
2
models
about 11 hours ago
[FEEDBACK] Inference Providers
24
#49 opened 12 days ago
by
julien-c
reacted to
mitkox's
post with π
5 days ago
Post
2028
llama.cpp is 26.8% faster than ollama.
I have upgraded both, and using the same settings, I am running the same DeepSeek R1 Distill 1.5B on the same hardware. It's an Apples to Apples comparison.
Total duration:
llama.cpp 6.85 sec <- 26.8% faster
ollama 8.69 sec
Breakdown by phase:
Model loading
llama.cpp 241 ms <- 2x faster
ollama 553 ms
Prompt processing
llama.cpp 416.04 tokens/s with an eval time 45.67 ms <- 10x faster
ollama 42.17 tokens/s with an eval time of 498 ms
Token generation
llama.cpp 137.79 tokens/s with an eval time 6.62 sec <- 13% faster
ollama 122.07 tokens/s with an eval time 7.64 sec
llama.cpp is LLM inference in C/C++; ollama adds abstraction layers and marketing.
Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.
I have upgraded both, and using the same settings, I am running the same DeepSeek R1 Distill 1.5B on the same hardware. It's an Apples to Apples comparison.
Total duration:
llama.cpp 6.85 sec <- 26.8% faster
ollama 8.69 sec
Breakdown by phase:
Model loading
llama.cpp 241 ms <- 2x faster
ollama 553 ms
Prompt processing
llama.cpp 416.04 tokens/s with an eval time 45.67 ms <- 10x faster
ollama 42.17 tokens/s with an eval time of 498 ms
Token generation
llama.cpp 137.79 tokens/s with an eval time 6.62 sec <- 13% faster
ollama 122.07 tokens/s with an eval time 7.64 sec
llama.cpp is LLM inference in C/C++; ollama adds abstraction layers and marketing.
Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.
- 7 replies
reacted to
sometimesanotion's
post with π₯
6 days ago
Post
2582
I've managed a #1 score of 41.22% average for 14B parameter models on the Open LLM Leaderboard. As of this writing, sometimesanotion/Lamarck-14B-v0.7 is #8 for all models up to 70B parameters.
It took a custom toolchain around Arcee AI's mergekit to manage the complex merges, gradients, and LoRAs required to make this happen. I really like seeing features of many quality finetunes in one solid generalist model.
It took a custom toolchain around Arcee AI's mergekit to manage the complex merges, gradients, and LoRAs required to make this happen. I really like seeing features of many quality finetunes in one solid generalist model.
- 5 replies
liked
a
model
7 days ago
upvoted
a
collection
7 days ago
updated
a
model
8 days ago
Adding Evaluation Results
#2 opened 8 days ago
by
benhaotang
What temp are these expected to be used at?
1
#6 opened 9 days ago
by
rombodawg
liked
a
model
9 days ago
Correctness of the example question in HF space
2
#1 opened 13 days ago
by
benhaotang
liked
a
model
9 days ago
updated
a
Space
11 days ago
updated
a
model
12 days ago
Correctness on the example question
1
#1 opened 13 days ago
by
benhaotang
published
a
Space
13 days ago
updated
2
models
13 days ago