Upload RT-DETRv2 voucher classifier

README.md

@@ -0,0 +1,156 @@
+---
+license: apache-2.0
+base_model: PekingU/rtdetr_v2_r101vd
+tags:
+- object-detection
+- computer-vision
+- voucher-classification
+- rt-detr
+- rtdetrv2
+datasets:
+- custom-voucher-dataset
+metrics:
+- map
+- map_50
+- map_75
+widget:
+- src: https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/car.jpg
+  example_title: Example Image
+---
+
+# RT-DETRv2 Fine-tuned for Voucher Classification
+
+This model is a fine-tuned version of [PekingU/rtdetr_v2_r101vd](https://huggingface.co/PekingU/rtdetr_v2_r101vd) for voucher classification and object detection.
+
+## Model Details
+
+### Model Description
+- **Model Type**: Object Detection (RT-DETRv2)
+- **Base Model**: PekingU/rtdetr_v2_r101vd
+- **Task**: Multi-class voucher classification and detection
+- **Classes**: 3 classes
+  - 0: digital (digital invoices)
+  - 1: fisico (physical receipts on blank pages)
+  - 2: tesoreria (small on-site payment receipts)
+
+### Training Details
+
+**Training Dataset:**
+- **Total Samples**: 507
+- **Class Distribution**:
+- **fisico** (id: 1): 241 samples (47.5%)
+- **digital** (id: 0): 147 samples (29.0%)
+- **tesoreria** (id: 2): 119 samples (23.5%)
+
+
+**Training Configuration:**
+- **Image Size**: 800x800
+- **Batch Size**: 24
+- **Learning Rate**: 1.5e-05
+- **Weight Decay**: 0.0001
+- **Epochs**: 2
+- **Validation Split**: 0.0
+
+**Data Processing:**
+- Pre-augmented dataset used (no runtime augmentation)
+- External train/validation split (use create_train_val_split.py)
+- Preprocessing: Resize + Normalization only
+
+### Performance Metrics
+
+**Final Evaluation Results:**
+**Dataset Information:**
+*Training Dataset:*
+- **Digital invoices**: 147 samples (29.0%)
+- **Fisico receipts**: 241 samples (47.5%)
+- **Tesoreria receipts**: 119 samples (23.5%)
+- **Total training samples**: 507
+
+**Model Configuration:**
+- **Base model**: PekingU/rtdetr_v2_r101vd
+- **Architecture**: rtdetr_v2_r101vd
+- **Input resolution**: 800×800 pixels
+- **Training epochs**: 2
+- **Batch size**: 24
+
+**Training Hardware:**
+- **GPU**: NVIDIA A100-SXM4-40GB
+- **VRAM**: 39.6 GB
+- **RAM**: 83.5 GB
+- **GPU configuration**: A100 optimized
+
+**Training Time**: 0.0 minutes
+
+**Training Summary:**
+- **Final training loss**: 0.0000
+
+
+### MLflow Tracking
+
+- **MLflow Run ID**: c348e8235f8c40138c05c051fc207bb6
+- **MLflow Experiment**: RT-DETRv2_Voucher_Classification
+
+
+## Usage
+
+```python
+from transformers import AutoModelForObjectDetection, AutoImageProcessor
+import torch
+from PIL import Image
+import numpy as np
+
+# Load model and processor
+model = AutoModelForObjectDetection.from_pretrained("jnmrr/rtdetr-v2-voucher-classifier")
+image_processor = AutoImageProcessor.from_pretrained("jnmrr/rtdetr-v2-voucher-classifier")
+
+# Load and preprocess image
+image = Image.open("path/to/your/voucher.jpg").convert("RGB")
+inputs = image_processor(images=image, return_tensors="pt")
+
+# Run inference
+with torch.no_grad():
+    outputs = model(**inputs)
+
+# Post-process results
+target_sizes = torch.tensor([image.size[::-1]])  # (height, width)
+results = image_processor.post_process_object_detection(
+    outputs, 
+    target_sizes=target_sizes, 
+    threshold=0.5
+)[0]
+
+# Print predictions
+class_names = ["digital", "fisico", "tesoreria"]
+for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
+    print(f"Class: {class_names[label.item()]}")
+    print(f"Confidence: {score.item():.3f}")
+    print(f"BBox: {box.tolist()}")
+```
+
+## Training Procedure
+
+The model was fine-tuned using the Hugging Face Transformers library with:
+- Pre-augmented dataset focusing on challenging cases
+- Format-specific augmentation strategies applied during data preparation
+- MLflow experiment tracking for reproducibility
+- External train/validation split for unbiased evaluation
+
+## Limitations and Bias
+
+- Trained specifically on voucher/receipt images
+- Performance may vary on images significantly different from training distribution
+- Model optimized for 3-class voucher classification task
+
+## Citation
+
+If you use this model, please cite:
+
+```bibtex
+@misc{rtdetr-v2-voucher-classifier,
+  title={RT-DETRv2 Fine-tuned for Voucher Classification},
+  author={Your Name},
+  year={2025},
+  publisher={Hugging Face},
+  url={https://huggingface.co/jnmrr/rtdetr-v2-voucher-classifier}
+}
+```

checkpoint-22/config.json

@@ -0,0 +1,129 @@
+{
+  "activation_dropout": 0.0,
+  "activation_function": "silu",
+  "anchor_image_size": null,
+  "architectures": [
+    "RTDetrV2ForObjectDetection"
+  ],
+  "attention_dropout": 0.0,
+  "auxiliary_loss": true,
+  "backbone": null,
+  "backbone_config": {
+    "depths": [
+      3,
+      4,
+      23,
+      3
+    ],
+    "downsample_in_bottleneck": false,
+    "downsample_in_first_stage": false,
+    "embedding_size": 64,
+    "hidden_act": "relu",
+    "hidden_sizes": [
+      256,
+      512,
+      1024,
+      2048
+    ],
+    "layer_type": "bottleneck",
+    "model_type": "rt_detr_resnet",
+    "num_channels": 3,
+    "out_features": [
+      "stage2",
+      "stage3",
+      "stage4"
+    ],
+    "out_indices": [
+      2,
+      3,
+      4
+    ],
+    "stage_names": [
+      "stem",
+      "stage1",
+      "stage2",
+      "stage3",
+      "stage4"
+    ],
+    "torch_dtype": "float32"
+  },
+  "backbone_kwargs": null,
+  "batch_norm_eps": 1e-05,
+  "box_noise_scale": 1.0,
+  "d_model": 256,
+  "decoder_activation_function": "relu",
+  "decoder_attention_heads": 8,
+  "decoder_ffn_dim": 1024,
+  "decoder_in_channels": [
+    384,
+    384,
+    384
+  ],
+  "decoder_layers": 6,
+  "decoder_method": "default",
+  "decoder_n_levels": 3,
+  "decoder_n_points": 4,
+  "decoder_offset_scale": 0.5,
+  "disable_custom_kernels": true,
+  "dropout": 0.0,
+  "encode_proj_layers": [
+    2
+  ],
+  "encoder_activation_function": "gelu",
+  "encoder_attention_heads": 8,
+  "encoder_ffn_dim": 2048,
+  "encoder_hidden_dim": 384,
+  "encoder_in_channels": [
+    512,
+    1024,
+    2048
+  ],
+  "encoder_layers": 1,
+  "eos_coefficient": 0.0001,
+  "eval_size": null,
+  "feat_strides": [
+    8,
+    16,
+    32
+  ],
+  "focal_loss_alpha": 0.75,
+  "focal_loss_gamma": 2.0,
+  "freeze_backbone_batch_norms": true,
+  "hidden_expansion": 1.0,
+  "id2label": {
+    "0": "LABEL_0",
+    "1": "LABEL_1",
+    "2": "LABEL_2"
+  },
+  "initializer_bias_prior_prob": null,
+  "initializer_range": 0.01,
+  "is_encoder_decoder": true,
+  "label2id": {
+    "LABEL_0": 0,
+    "LABEL_1": 1,
+    "LABEL_2": 2
+  },
+  "label_noise_ratio": 0.5,
+  "layer_norm_eps": 1e-05,
+  "learn_initial_query": false,
+  "matcher_alpha": 0.25,
+  "matcher_bbox_cost": 5.0,
+  "matcher_class_cost": 2.0,
+  "matcher_gamma": 2.0,
+  "matcher_giou_cost": 2.0,
+  "model_type": "rt_detr_v2",
+  "normalize_before": false,
+  "num_denoising": 100,
+  "num_feature_levels": 3,
+  "num_queries": 300,
+  "positional_encoding_temperature": 10000,
+  "torch_dtype": "float32",
+  "transformers_version": "4.55.0",
+  "use_focal_loss": true,
+  "use_pretrained_backbone": false,
+  "use_timm_backbone": false,
+  "weight_loss_bbox": 5.0,
+  "weight_loss_giou": 2.0,
+  "weight_loss_vfl": 1.0,
+  "with_box_refine": true
+}

checkpoint-22/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9de66f7b33e7ac4d76c89424fca8b97c02c5ffd7bb21cd21a3ea8d5e351821c
+size 306699044

checkpoint-22/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:087c1f2c00c286193f70c788643e0d1176edcc4d78b2d09203f5c97419dcee0d
+size 611580433

checkpoint-22/preprocessor_config.json

@@ -0,0 +1,26 @@
+{
+  "do_convert_annotations": true,
+  "do_normalize": false,
+  "do_pad": false,
+  "do_rescale": true,
+  "do_resize": true,
+  "format": "coco_detection",
+  "image_mean": [
+    0.485,
+    0.456,
+    0.406
+  ],
+  "image_processor_type": "RTDetrImageProcessor",
+  "image_std": [
+    0.229,
+    0.224,
+    0.225
+  ],
+  "pad_size": null,
+  "resample": 2,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 640,
+    "width": 640
+  }
+}

checkpoint-22/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a60ef25d0cb819c998330aa0d916d7bb159075f89558fabd3d6a4aafb3c8b73b
+size 14244

checkpoint-22/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0836e9a481b0522ea2522eeeccf449d7f5873460995f8deac588da4c5e736f5d
+size 1064

checkpoint-22/trainer_state.json

@@ -0,0 +1,33 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.0,
+  "eval_steps": 500,
+  "global_step": 22,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [],
+  "logging_steps": 50,
+  "max_steps": 22,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.632598679552e+17,
+  "train_batch_size": 24,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-22/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:261388b5dfaf2a9df467bf9b4db643f296523f92d93fa65002e44c1b566dfa5f
+size 5368

config.json

@@ -0,0 +1,129 @@
+{
+  "activation_dropout": 0.0,
+  "activation_function": "silu",
+  "anchor_image_size": null,
+  "architectures": [
+    "RTDetrV2ForObjectDetection"
+  ],
+  "attention_dropout": 0.0,
+  "auxiliary_loss": true,
+  "backbone": null,
+  "backbone_config": {
+    "depths": [
+      3,
+      4,
+      23,
+      3
+    ],
+    "downsample_in_bottleneck": false,
+    "downsample_in_first_stage": false,
+    "embedding_size": 64,
+    "hidden_act": "relu",
+    "hidden_sizes": [
+      256,
+      512,
+      1024,
+      2048
+    ],
+    "layer_type": "bottleneck",
+    "model_type": "rt_detr_resnet",
+    "num_channels": 3,
+    "out_features": [
+      "stage2",
+      "stage3",
+      "stage4"
+    ],
+    "out_indices": [
+      2,
+      3,
+      4
+    ],
+    "stage_names": [
+      "stem",
+      "stage1",
+      "stage2",
+      "stage3",
+      "stage4"
+    ],
+    "torch_dtype": "float32"
+  },
+  "backbone_kwargs": null,
+  "batch_norm_eps": 1e-05,
+  "box_noise_scale": 1.0,
+  "d_model": 256,
+  "decoder_activation_function": "relu",
+  "decoder_attention_heads": 8,
+  "decoder_ffn_dim": 1024,
+  "decoder_in_channels": [
+    384,
+    384,
+    384
+  ],
+  "decoder_layers": 6,
+  "decoder_method": "default",
+  "decoder_n_levels": 3,
+  "decoder_n_points": 4,
+  "decoder_offset_scale": 0.5,
+  "disable_custom_kernels": true,
+  "dropout": 0.0,
+  "encode_proj_layers": [
+    2
+  ],
+  "encoder_activation_function": "gelu",
+  "encoder_attention_heads": 8,
+  "encoder_ffn_dim": 2048,
+  "encoder_hidden_dim": 384,
+  "encoder_in_channels": [
+    512,
+    1024,
+    2048
+  ],
+  "encoder_layers": 1,
+  "eos_coefficient": 0.0001,
+  "eval_size": null,
+  "feat_strides": [
+    8,
+    16,
+    32
+  ],
+  "focal_loss_alpha": 0.75,
+  "focal_loss_gamma": 2.0,
+  "freeze_backbone_batch_norms": true,
+  "hidden_expansion": 1.0,
+  "id2label": {
+    "0": "LABEL_0",
+    "1": "LABEL_1",
+    "2": "LABEL_2"
+  },
+  "initializer_bias_prior_prob": null,
+  "initializer_range": 0.01,
+  "is_encoder_decoder": true,
+  "label2id": {
+    "LABEL_0": 0,
+    "LABEL_1": 1,
+    "LABEL_2": 2
+  },
+  "label_noise_ratio": 0.5,
+  "layer_norm_eps": 1e-05,
+  "learn_initial_query": false,
+  "matcher_alpha": 0.25,
+  "matcher_bbox_cost": 5.0,
+  "matcher_class_cost": 2.0,
+  "matcher_gamma": 2.0,
+  "matcher_giou_cost": 2.0,
+  "model_type": "rt_detr_v2",
+  "normalize_before": false,
+  "num_denoising": 100,
+  "num_feature_levels": 3,
+  "num_queries": 300,
+  "positional_encoding_temperature": 10000,
+  "torch_dtype": "float32",
+  "transformers_version": "4.55.0",
+  "use_focal_loss": true,
+  "use_pretrained_backbone": false,
+  "use_timm_backbone": false,
+  "weight_loss_bbox": 5.0,
+  "weight_loss_giou": 2.0,
+  "weight_loss_vfl": 1.0,
+  "with_box_refine": true
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9de66f7b33e7ac4d76c89424fca8b97c02c5ffd7bb21cd21a3ea8d5e351821c
+size 306699044

preprocessor_config.json

@@ -0,0 +1,26 @@
+{
+  "do_convert_annotations": true,
+  "do_normalize": false,
+  "do_pad": false,
+  "do_rescale": true,
+  "do_resize": true,
+  "format": "coco_detection",
+  "image_mean": [
+    0.485,
+    0.456,
+    0.406
+  ],
+  "image_processor_type": "RTDetrImageProcessor",
+  "image_std": [
+    0.229,
+    0.224,
+    0.225
+  ],
+  "pad_size": null,
+  "resample": 2,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 640,
+    "width": 640
+  }
+}

runs/Aug13_22-15-12_9db0f8c974d2/events.out.tfevents.1755123313.9db0f8c974d2.60074.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75d7c3066003b54ea7b427f6d2ec1c2be19bdda662a8328d4ce99254a03b1716
+size 7396

runs/Aug13_22-20-53_9db0f8c974d2/events.out.tfevents.1755123655.9db0f8c974d2.61846.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f69b32a94e95402ed3eea7115afb448510aebb3501c2d77b136d84d25afed77c
+size 7396

runs/Aug13_22-23-40_9db0f8c974d2/events.out.tfevents.1755123822.9db0f8c974d2.62610.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b3342ac0b845641b7113baa74e166d24cdd4696a9a5d34192988c9d5f4d6500a
+size 7396

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c27b2f12813f34a64df28943d3a14ef6b011ae53edbbf445b45d5da3a845e221
+size 5368