Hugging Face's logo

Intelligent-Internet
/
II-Medical-8B-1706

Text Generation
Transformers
Safetensors
qwen3
conversational
text-generation-inference
Model card Files
xet
Community
sw_tuenguyen commited on
Commit
7794ada
·
verified ·
1 Parent(s): 848298d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -6
README.md CHANGED
@@ -50,13 +50,14 @@ Detailed result for HealthBench can be found [here](https://huggingface.co/datas
50
  We evaluate on ten medical QA benchmarks include MedMCQA, MedQA, PubMedQA, medical related questions from MMLU-Pro and GPQA, small QA sets from Lancet and the New England
51
  Journal of Medicine, 4 Options and 5 Options splits from the MedBullets platform and MedXpertQA.
52
 
53
- | Model | MedMC | MedQA | PubMed | MMLU-P | GPQA | Lancet | MedB-4 | MedB-5 | MedX | NEJM | Avg |
54
  |--------------------------|-------|-------|--------|--------|------|--------|--------|--------|------|-------|-------|
55
- | [HuatuoGPT-o1-72B](https://huggingface.co/FreedomIntelligence/HuatuoGPT-o1-72B) | 76.76 | 88.85 | 79.90 | 80.46 | 64.36| 70.87 | 77.27 | 73.05 |23.53 |76.29 | 71.13 |
56
- | [MedGemma-27B-IT](https://huggingface.co/google/medgemma-27b-text-it) | 73.24 | 87.27 | 70.9 | 80.13 | 61.53| 70.14 | 75.32 | 73.37 |25.55 |76.28 | 69.37 |
57
- | [M1](https://huggingface.co/UCSC-VLAA/m1-7B-23K) | 62.54 | 75.81 | 75.80 | 65.86 | 53.08| 62.62 | 63.64 | 59.74 |19.59 |64.34 | 60.3 |
58
- | [II-Medical-8B-SFT](https://huggingface.co/II-Vietnam/II-Medical-8B-SFT) | **73.32** | 87.90 | 77.5 | 78.17 | 62.05| 74.75 | 77.59 | 72.72 |25.79 |**77.77** |70.71 |
59
- | [II-Medical-8B](https://huggingface.co/Intelligent-Internet/II-Medical-8B) | 74.85 | **90.33** | 79.1 | **79.15** | **64.61**| **74.02** | **82.14** | **75.64** |**30.08** |77.11 | **72.71** |
 
60
 
61
  ## IV. Dataset Curation
62
 
 
50
  We evaluate on ten medical QA benchmarks include MedMCQA, MedQA, PubMedQA, medical related questions from MMLU-Pro and GPQA, small QA sets from Lancet and the New England
51
  Journal of Medicine, 4 Options and 5 Options splits from the MedBullets platform and MedXpertQA.
52
 
53
+ | Model | MedMC | MedQA | PubMed | MMLU-P | HealthBench | Lancet | MedB-4 | MedB-5 | MedX | NEJM | Avg |
54
  |--------------------------|-------|-------|--------|--------|------|--------|--------|--------|------|-------|-------|
55
+ | [HuatuoGPT-o1-72B](https://huggingface.co/FreedomIntelligence/HuatuoGPT-o1-72B) | 76.76 | 88.85 | 79.90 | 80.46 | 22.73 | 70.87 | 77.27 | 73.05 |23.53 |76.29 | x |
56
+ | [MedGemma-27B-IT](https://huggingface.co/google/medgemma-27b-text-it) | 73.24 | 87.27 | 70.9 | 80.13 | 46.54| 70.14 | 75.32 | 73.37 |25.55 |76.28 | x |
57
+ | [Qwen3-32B](https://huggingface.co/Qwen/Qwen3-32B) | 74.18 | 88.92 | 76.1 | 80.7 | 47.08 | 72.33 | 72.27 | 71.42 |28.04 |76.94 | x |
58
+ | [M1](https://huggingface.co/UCSC-VLAA/m1-7B-23K) | 62.54 | 75.81 | 75.80 | 65.86 | ? | 62.62 | 63.64 | 59.74 |19.59 |64.34 | x |
59
+ | [II-Medical-8B-SFT](https://huggingface.co/II-Vietnam/II-Medical-8B-SFT) | **73.32** | 87.90 | 77.5 | 78.17 | 42.64| 74.75 | 77.59 | 72.72 |25.79 |**77.77** |x |
60
+ | [II-Medical-8B](https://huggingface.co/Intelligent-Internet/II-Medical-8B) | 74.85 | **90.33** | 79.1 | **79.15** | 42.31| **74.02** | **82.14** | **75.64** |**30.08** |77.11 | **x** |
61
 
62
  ## IV. Dataset Curation
63