File size: 3,328 Bytes
150d962 d7b2280 8869360 150d962 d7b2280 150d962 8869360 1e165f4 e34f2f5 94f09f0 150d962 08aed96 150d962 c6483fb 150d962 d7b2280 150d962 d7b2280 150d962 2ff84f1 9fb8f60 2ff84f1 150d962 8751bf1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 |
---
license: apache-2.0
tags:
- segmentation
- remove background
- background
- background-removal
- Pytorch
pretty_name: Open Remove Background Model
datasets:
- schirrmacher/humans
---
# Open Remove Background Model (ormbg)
[>>> DEMO <<<](https://huggingface.co/spaces/schirrmacher/ormbg)
This model is a **fully open-source background remover** optimized for images with humans. It is based on [Highly Accurate Dichotomous Image Segmentation research](https://github.com/xuebinqin/DIS). The model was trained with the synthetic [Human Segmentation Dataset](https://huggingface.co/datasets/schirrmacher/humans), a dataset crafted with [LayerDiffuse](https://github.com/layerdiffusion/LayerDiffuse) and [IC-Light](https://github.com/lllyasviel/IC-Light).
This model is similar to [RMBG-1.4](https://huggingface.co/briaai/RMBG-1.4), but with open training data/process and commercially free to use.
## Inference
```
python utils/inference.py
```
## Training
The model was trained on a NVIDIA GeForce RTX 4090 (10.000 iterations) with the [Human Segmentation Dataset](https://huggingface.co/datasets/schirrmacher/humans) which was created with [LayerDiffuse](https://github.com/layerdiffusion/LayerDiffuse) and [IC-Light](https://github.com/lllyasviel/IC-Light).
## Want to train your own model?
Checkout _Highly Accurate Dichotomous Image Segmentation_ code:
```
git clone https://github.com/xuebinqin/DIS.git
cd DIS
```
Follow the installation instructions on https://github.com/xuebinqin/DIS?tab=readme-ov-file#1-clone-this-repo.
Download or create some data ([like this](https://huggingface.co/datasets/schirrmacher/humans)) and place it into the DIS project folder.
I am using the folder structure:
- training/im (images)
- training/gt (ground truth)
- validation/im (images)
- validation/gt (ground truth)
Apply this git patch for setting the right paths and remove normalization of images:
```
git apply dis-repo.patch
```
Start training:
```
cd IS-Net
python train_valid_inference_main.py
```
Export to ONNX (modify paths if needed):
```
python utils/pth_to_onnx.py
```
# Research
Synthetic datasets have limitations for achieving great segmentation results. This is because artificial lighting, occlusion, scale or backgrounds create a gap between synthetic and real images. A "model trained solely on synthetic data generated with naïve domain randomization struggles to generalize on the real domain", see [PEOPLESANSPEOPLE: A Synthetic Data Generator for Human-Centric Computer Vision (2022)](https://arxiv.org/pdf/2112.09290).
Currently I am doing research how to close this gap. Latest research is about creating segmented humans with [LayerDiffuse](https://github.com/layerdiffusion/LayerDiffuse) and then apply [IC-Light](https://github.com/lllyasviel/IC-Light) for creating realistic light effects and shadows.
## Support
This is the first iteration of the model, so there will be improvements!
If you identify cases were the model fails, <a href='https://huggingface.co/schirrmacher/ormbg/discussions' target='_blank'>upload your examples</a>!
Known issues (work in progress):
- close-ups: from above, from below, profile, from side
- minor issues with hair segmentation when hair creates loops
- more various backgrounds needed
|