Spaces:
Running
on
Zero
Apply for community grant: Personal project (gpu and storage)
UMO is a academic project space demo (based on UNO) for our paper:
UMO: Scaling Multi-Identity Consistency for Image Customization via Matching Reward
it is a Unified Multi-identity Optimization framework, designed to maintain high-fidelity identity preservation and alleviate identity confusion with scalability.
Some relative links about it:
arxiv: coming soon
github: https://github.com/bytedance/UMO
project pages: https://bytedance.github.io/UMO/
model weight: https://huggingface.co/bytedance-research/UMO
Here is the abstract of our paper:
Recent advancements in image customization exhibit a wide range of application prospects due to stronger customization capabilities. However, since we humans are more sensitive to faces, a significant challenge remains in preserving consistent identity while avoiding identity confusion with multi-reference images, limiting the identity scalability of customization models. To address this, we present UMO, a Unified Multi-identity Optimization framework, designed to maintain high-fidelity identity preservation and alleviate identity confusion with scalability. With "multi-to-multi matching" paradigm, UMO reformulates multi-identity generation as a global assignment optimization problem and unleashes multi-identity consistency for existing image customization methods generally through reinforcement learning on diffusion models. To facilitate the training of UMO, we develop a scalable customization dataset with multi-reference images, consisting of both synthesised and real parts. Additionally, we propose a new metric to measure identity confusion. Extensive experiments demonstrate that UMO not only improves identity consistency significantly, but also reduces identity confusion on several image customization methods, setting a new state-of-the-art among open-source methods along the dimension of identity preserving.
Could someone please grant GPU and storage for our demo in this space? Thank you so much for your support!
It works well now, thanks for your grant！