Update README.md

README.md

@@ -1,58 +1,52 @@
 ---
-base_model: google/paligemma-3b-ft-widgetcap-448
-library_name: peft
-license: gemma
-tags:
-- generated_from_trainer
-model-index:
-- name: paligemma-3b-ft-widgetcap-waveui-448
-  results: []
+library_name: transformers
+datasets:
+- agentsea/wave-ui-25k
+language:
+- en
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
+# Paligemma WaveUI
 
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/kentauros/paligemma-waveui/runs/hfa841vp)
-# paligemma-3b-ft-widgetcap-waveui-448
 
-This model is a fine-tuned version of [google/paligemma-3b-ft-widgetcap-448](https://huggingface.co/google/paligemma-3b-ft-widgetcap-448) on an unknown dataset.
+Transformers [PaliGemma 3B 448-res weights](https://huggingface.co/google/paligemma-3b-pt-448), fine-tuned on the [WaveUI](https://huggingface.co/datasets/agentsea/wave-ui) dataset for object-detection.
 
-## Model description
+## Model Details
 
-More information needed
+### Model Description
 
-## Intended uses & limitations
+This fine-tune was done atop of the [Paligemma 448 Widgetcap](https://huggingface.co/google/paligemma-3b-ft-widgetcap-448) model, using the [WaveUI](https://huggingface.co/datasets/agentsea/wave-ui) dataset, which contains ~80k examples of labeled UI elements.
 
-More information needed
+The fine-tune was done for the object detection task. Specifically, this model aims to perform well at UI element detection, as part of a wider effort to enable our open-source toolkit for building agents at [AgentSea](https://www.agentsea.ai/).
 
-## Training and evaluation data
+- **Developed by:** https://agentsea.ai/
+- **Language(s) (NLP):** en
+- **Finetuned from model:** https://huggingface.co/google/paligemma-3b-ft-widgetcap-448
 
-More information needed
+### Demo
 
-## Training procedure
+You can find a **demo** for this model [here](https://huggingface.co/spaces/agentsea/paligemma-waveui).
 
-### Training hyperparameters
+## Notes
 
-The following hyperparameters were used during training:
-- learning_rate: 0.0001
-- train_batch_size: 4
-- eval_batch_size: 8
-- seed: 42
-- gradient_accumulation_steps: 4
-- total_train_batch_size: 16
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: linear
-- lr_scheduler_warmup_steps: 2
-- num_epochs: 3
+- The only task used in the fine-tune was the object detection task, so it might not perform well in other types of tasks.
+  
+## Usage
 
-### Training results
+To start using this model, run the following:
 
+```python
+from transformers import AutoProcessor, PaliGemmaForConditionalGeneration
 
+model = PaliGemmaForConditionalGeneration.from_pretrained("agentsea/paligemma-3b-ft-widgetcap-waveui-448").eval()
+processor = AutoProcessor.from_pretrained("google/paligemma-3b-pt-448")
+```
 
-### Framework versions
+## Data
 
-- PEFT 0.11.1
-- Transformers 4.43.2
-- Pytorch 2.4.0+cu121
-- Datasets 2.20.0
-- Tokenizers 0.19.1
+We used the [WaveUI](https://huggingface.co/datasets/agentsea/wave-ui) dataset for this fine-tune. Before using it, we preprocessed the data to use the Paligemma bounding-box format.
+ 
+
+## Evaluation
+
+We will release a full evaluation report soon. Stay tuned! :)