Hugging Face's logo

h2oai
/
h2ovl-mississippi-800m

Text Generation
Transformers
Safetensors
English
h2ovl_chat
feature-extraction
gpt
llm
multimodal large language model
ocr
conversational
custom_code
Model card Files Community
Shanshan Wang commited on
Commit
577843e
•
1 Parent(s): 2b5b628

updated readme

Browse files
Files changed (1) hide show
  1. README.md +24 -19
README.md CHANGED
@@ -13,6 +13,10 @@ thumbnail: >-
13
  pipeline_tag: text-generation
14
  ---
15
  # Model Card
 
 
 
 
16
  The H2OVL-Mississippi-800M is a compact yet powerful vision-language model from H2O.ai, featuring 0.8 billion parameters. Despite its small size, it delivers state-of-the-art performance in text recognition, excelling in the Text Recognition segment of OCRBench and outperforming much larger models in this domain. Built upon the robust architecture of our H2O-Danube language models, the Mississippi-800M extends their capabilities by seamlessly integrating vision and language tasks.
17
 
18
  <div align="center">
@@ -30,7 +34,26 @@ The H2OVL-Mississippi-800M is a compact yet powerful vision-language model from
30
  </div>
31
 
32
 
33
- ## Usage
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
34
 
35
  ### Install dependencies:
36
  ```bash
@@ -73,24 +96,6 @@ print(f'User: {question}\nAssistant: {response}')
73
  ```
74
 
75
 
76
- ## Benchmarks
77
-
78
- ### Performance Comparison of Similar Sized Models Across Multiple Benchmarks - OpenVLM Leaderboard
79
-
80
- | **Models** | **Params (B)** | **Avg. Score** | **MMBench** | **MMStar** | **MMMU<sub>VAL</sub>** | **Math Vista** | **Hallusion** | **AI2D<sub>TEST</sub>** | **OCRBench** | **MMVet** |
81
- |----------------------------|----------------|----------------|-------------|------------|-----------------------|----------------|---------------|-------------------------|--------------|-----------|
82
- | Qwen2-VL-2B | 2.1 | **57.2** | **72.2** | 47.5 | 42.2 | 47.8 | **42.4** | 74.7 | **797** | **51.5** |
83
- | **H2OVL-Mississippi-2B** | 2.1 | 54.4 | 64.8 | 49.6 | 35.2 | **56.8** | 36.4 | 69.9 | 782 | 44.7 |
84
- | InternVL2-2B | 2.1 | 53.9 | 69.6 | **49.8** | 36.3 | 46.0 | 38.0 | 74.1 | 781 | 39.7 |
85
- | Phi-3-Vision | 4.2 | 53.6 | 65.2 | 47.7 | **46.1** | 44.6 | 39.0 | **78.4** | 637 | 44.1 |
86
- | MiniMonkey | 2.2 | 52.7 | 68.9 | 48.1 | 35.7 | 45.3 | 30.9 | 73.7 | **794** | 39.8 |
87
- | MiniCPM-V-2 | 2.8 | 47.9 | 65.8 | 39.1 | 38.2 | 39.8 | 36.1 | 62.9 | 605 | 41.0 |
88
- | InternVL2-1B | 0.8 | 48.3 | 59.7 | 45.6 | 36.7 | 39.4 | 34.3 | 63.8 | 755 | 31.5 |
89
- | PaliGemma-3B-mix-448 | 2.9 | 46.5 | 65.6 | 48.3 | 34.9 | 28.7 | 32.2 | 68.3 | 614 | 33.1 |
90
- | **H2OVL-Mississippi-0.8B** | 0.8 | 43.5 | 47.7 | 39.1 | 34.0 | 39.0 | 29.6 | 53.6 | 751 | 30.0 |
91
- | DeepSeek-VL-1.3B | 2.0 | 39.6 | 63.8 | 39.9 | 33.8 | 29.8 | 27.6 | 51.5 | 413 | 29.2 |
92
-
93
-
94
  ## Prompt Engineering for JSON Extraction
95
 
96
  ### Overview
 
13
  pipeline_tag: text-generation
14
  ---
15
  # Model Card
16
+ [\[📜 H2OVL-Mississippi Paper\]](https://arxiv.org/abs/2410.13611)
17
+ [\[🤗 HF Demo\]](https://huggingface.co/spaces/h2oai/h2ovl-mississippi)
18
+ [\[🚀 Quick Start\]](#quick-start)
19
+
20
  The H2OVL-Mississippi-800M is a compact yet powerful vision-language model from H2O.ai, featuring 0.8 billion parameters. Despite its small size, it delivers state-of-the-art performance in text recognition, excelling in the Text Recognition segment of OCRBench and outperforming much larger models in this domain. Built upon the robust architecture of our H2O-Danube language models, the Mississippi-800M extends their capabilities by seamlessly integrating vision and language tasks.
21
 
22
  <div align="center">
 
34
  </div>
35
 
36
 
37
+ ## Benchmarks
38
+
39
+ ### Performance Comparison of Similar Sized Models Across Multiple Benchmarks - OpenVLM Leaderboard
40
+
41
+ | **Models** | **Params (B)** | **Avg. Score** | **MMBench** | **MMStar** | **MMMU<sub>VAL</sub>** | **Math Vista** | **Hallusion** | **AI2D<sub>TEST</sub>** | **OCRBench** | **MMVet** |
42
+ |----------------------------|----------------|----------------|-------------|------------|-----------------------|----------------|---------------|-------------------------|--------------|-----------|
43
+ | Qwen2-VL-2B | 2.1 | **57.2** | **72.2** | 47.5 | 42.2 | 47.8 | **42.4** | 74.7 | **797** | **51.5** |
44
+ | **H2OVL-Mississippi-2B** | 2.1 | 54.4 | 64.8 | 49.6 | 35.2 | **56.8** | 36.4 | 69.9 | 782 | 44.7 |
45
+ | InternVL2-2B | 2.1 | 53.9 | 69.6 | **49.8** | 36.3 | 46.0 | 38.0 | 74.1 | 781 | 39.7 |
46
+ | Phi-3-Vision | 4.2 | 53.6 | 65.2 | 47.7 | **46.1** | 44.6 | 39.0 | **78.4** | 637 | 44.1 |
47
+ | MiniMonkey | 2.2 | 52.7 | 68.9 | 48.1 | 35.7 | 45.3 | 30.9 | 73.7 | **794** | 39.8 |
48
+ | MiniCPM-V-2 | 2.8 | 47.9 | 65.8 | 39.1 | 38.2 | 39.8 | 36.1 | 62.9 | 605 | 41.0 |
49
+ | InternVL2-1B | 0.8 | 48.3 | 59.7 | 45.6 | 36.7 | 39.4 | 34.3 | 63.8 | 755 | 31.5 |
50
+ | PaliGemma-3B-mix-448 | 2.9 | 46.5 | 65.6 | 48.3 | 34.9 | 28.7 | 32.2 | 68.3 | 614 | 33.1 |
51
+ | **H2OVL-Mississippi-0.8B** | 0.8 | 43.5 | 47.7 | 39.1 | 34.0 | 39.0 | 29.6 | 53.6 | 751 | 30.0 |
52
+ | DeepSeek-VL-1.3B | 2.0 | 39.6 | 63.8 | 39.9 | 33.8 | 29.8 | 27.6 | 51.5 | 413 | 29.2 |
53
+
54
+
55
+
56
+ ## Quick Start
57
 
58
  ### Install dependencies:
59
  ```bash
 
96
  ```
97
 
98
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
99
  ## Prompt Engineering for JSON Extraction
100
 
101
  ### Overview