Delete README.md
Browse files
README.md
DELETED
|
@@ -1,56 +0,0 @@
|
|
| 1 |
-
|
| 2 |
-
language:
|
| 3 |
-
- en
|
| 4 |
-
pipeline_tag: image-text-to-text
|
| 5 |
-
tags:
|
| 6 |
-
- multimodal
|
| 7 |
-
library_name: transformers
|
| 8 |
-
base_model:
|
| 9 |
-
- Qwen/Qwen2-VL-7B
|
| 10 |
-
new_version: Qwen/Qwen2.5-VL-7B-Instruct
|
| 11 |
-
---
|
| 12 |
-
|
| 13 |
-
Qwen Fine-Tuned IRChatModel
|
| 14 |
-
This model has been specifically fine-tuned for a chatbot project developed by Siddharth Patil, Aryan Varghese, and Sagar Mishra. It leverages the power of the Qwen architecture, optimizing it for enhanced interaction and conversational capabilities tailored to meet the specific needs of the chatbot application. The model is hosted and deployed as part of an ongoing effort to push the boundaries of AI-driven conversations.
|
| 15 |
-
|
| 16 |
-
All Rights Reserved
|
| 17 |
-
This model and its fine-tuning methodology are proprietary, developed under the authorization and ownership of Alibaba Group. All intellectual property, including the architecture and fine-tuned parameters, are protected under copyright and other applicable laws.
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
## Evaluation
|
| 21 |
-
|
| 22 |
-
### Image Benchmarks
|
| 23 |
-
|
| 24 |
-
| Benchmark | InternVL2-8B | MiniCPM-V 2.6 | GPT-4o-mini | **Qwen2-VL-7B** |
|
| 25 |
-
| :--- | :---: | :---: | :---: | :---: |
|
| 26 |
-
| MMMU<sub>val</sub> | 51.8 | 49.8 | **60**| 54.1 |
|
| 27 |
-
| DocVQA<sub>test</sub> | 91.6 | 90.8 | - | **94.5** |
|
| 28 |
-
| InfoVQA<sub>test</sub> | 74.8 | - | - |**76.5** |
|
| 29 |
-
| ChartQA<sub>test</sub> | **83.3** | - |- | 83.0 |
|
| 30 |
-
| TextVQA<sub>val</sub> | 77.4 | 80.1 | -| **84.3** |
|
| 31 |
-
| OCRBench | 794 | **852** | 785 | 845 |
|
| 32 |
-
| MTVQA | - | - | -| **26.3** |
|
| 33 |
-
| VCR<sub>en easy</sub> | - | 73.88 | 83.60 | **89.70** |
|
| 34 |
-
| VCR<sub>zh easy</sub> | - | 10.18| 1.10 | **59.94** |
|
| 35 |
-
| RealWorldQA | 64.4 | - | - | **70.1** |
|
| 36 |
-
| MME<sub>sum</sub> | 2210.3 | **2348.4** | 2003.4| 2326.8 |
|
| 37 |
-
| MMBench-EN<sub>test</sub> | 81.7 | - | - | **83.0** |
|
| 38 |
-
| MMBench-CN<sub>test</sub> | **81.2** | - | - | 80.5 |
|
| 39 |
-
| MMBench-V1.1<sub>test</sub> | 79.4 | 78.0 | 76.0| **80.7** |
|
| 40 |
-
| MMT-Bench<sub>test</sub> | - | - | - |**63.7** |
|
| 41 |
-
| MMStar | **61.5** | 57.5 | 54.8 | 60.7 |
|
| 42 |
-
| MMVet<sub>GPT-4-Turbo</sub> | 54.2 | 60.0 | **66.9** | 62.0 |
|
| 43 |
-
| HallBench<sub>avg</sub> | 45.2 | 48.1 | 46.1| **50.6** |
|
| 44 |
-
| MathVista<sub>testmini</sub> | 58.3 | **60.6** | 52.4 | 58.2 |
|
| 45 |
-
| MathVision | - | - | - | **16.3** |
|
| 46 |
-
|
| 47 |
-
### Video Benchmarks
|
| 48 |
-
|
| 49 |
-
| Benchmark | Internvl2-8B | LLaVA-OneVision-7B | MiniCPM-V 2.6 | **Qwen2-VL-7B** |
|
| 50 |
-
| :--- | :---: | :---: | :---: | :---: |
|
| 51 |
-
| MVBench | 66.4 | 56.7 | - | **67.0** |
|
| 52 |
-
| PerceptionTest<sub>test</sub> | - | 57.1 | - | **62.3** |
|
| 53 |
-
| EgoSchema<sub>test</sub> | - | 60.1 | - | **66.7** |
|
| 54 |
-
| Video-MME<sub>wo/w subs</sub> | 54.0/56.9 | 58.2/- | 60.9/63.6 | **63.3**/**69.0** |
|
| 55 |
-
|
| 56 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|