Update README.md
Browse files
README.md
CHANGED
|
@@ -24,21 +24,35 @@ UGround is a storng GUI visual grounding model trained with a simple recipe. Che
|
|
| 24 |
|
| 25 |
|
| 26 |
- [x] Model Weights
|
|
|
|
| 27 |
- [ ] Code
|
| 28 |
- [ ] Inference Code of UGround
|
| 29 |
- [x] Offline Experiments
|
| 30 |
- [x] Screenspot (along with referring expressions generated by GPT-4/4o)
|
| 31 |
- [x] Multimodal-Mind2Web
|
| 32 |
- [x] OmniAct
|
|
|
|
| 33 |
- [ ] Online Experiments
|
| 34 |
-
- [ ] Mind2Web-Live
|
| 35 |
-
- [ ] AndroidWorld
|
| 36 |
-
- [ ] Data
|
| 37 |
- [ ] Data Examples
|
| 38 |
- [ ] Data Construction Scripts
|
| 39 |
-
- [ ] Guidance of Open-source Data
|
|
|
|
| 40 |
- [x] Online Demo (HF Spaces)
|
| 41 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
|
| 43 |
|
| 44 |
## Main Results
|
|
@@ -112,9 +126,11 @@ messages = format_openai_template(description, base64_image)
|
|
| 112 |
completion = await client.chat.completions.create(
|
| 113 |
model=args.model_path,
|
| 114 |
messages=messages,
|
| 115 |
-
temperature=0
|
| 116 |
)
|
| 117 |
|
|
|
|
|
|
|
| 118 |
```
|
| 119 |
|
| 120 |
|
|
|
|
| 24 |
|
| 25 |
|
| 26 |
- [x] Model Weights
|
| 27 |
+
- [x] Qwen2-VL-based V1
|
| 28 |
- [ ] Code
|
| 29 |
- [ ] Inference Code of UGround
|
| 30 |
- [x] Offline Experiments
|
| 31 |
- [x] Screenspot (along with referring expressions generated by GPT-4/4o)
|
| 32 |
- [x] Multimodal-Mind2Web
|
| 33 |
- [x] OmniAct
|
| 34 |
+
- [ ] Android Control
|
| 35 |
- [ ] Online Experiments
|
| 36 |
+
- [ ] Mind2Web-Live-SeeAct-V
|
| 37 |
+
- [ ] AndroidWorld-SeeAct-V
|
| 38 |
+
- [ ] Data-V1
|
| 39 |
- [ ] Data Examples
|
| 40 |
- [ ] Data Construction Scripts
|
| 41 |
+
- [ ] Guidance of Open-source Data
|
| 42 |
+
- [ ] Data-V1.1
|
| 43 |
- [x] Online Demo (HF Spaces)
|
| 44 |
|
| 45 |
+
## Models
|
| 46 |
+
|
| 47 |
+
Initial UGround-V1:
|
| 48 |
+
UGround-V1-2B (Qwen2-VL): https://huggingface.co/osunlp/UGround-V1-2B
|
| 49 |
+
UGround-V1-7B (Qwen2-VL): https://huggingface.co/osunlp/UGround-V1-7B
|
| 50 |
+
UGround-V1-72B (Qwen2-VL): Coming Soon
|
| 51 |
+
UGround-V1.1-2B (Qwen2-VL): Coming Soon
|
| 52 |
+
UGround-V1.1-7B (Qwen2-VL): Coming Soon
|
| 53 |
+
UGround-V1.1-72B (Qwen2-VL): Coming Soon
|
| 54 |
+
|
| 55 |
+
|
| 56 |
|
| 57 |
|
| 58 |
## Main Results
|
|
|
|
| 126 |
completion = await client.chat.completions.create(
|
| 127 |
model=args.model_path,
|
| 128 |
messages=messages,
|
| 129 |
+
temperature=0 # Remember to set temperature to ZERO!
|
| 130 |
)
|
| 131 |
|
| 132 |
+
# The output will be in the range of [0,999), which is compatible with the original Qwen2-VL
|
| 133 |
+
|
| 134 |
```
|
| 135 |
|
| 136 |
|