Hugging Face's logo

alimama-creative
/
FLUX.1-dev-Controlnet-Inpainting-Beta

Image-to-Image
Diffusers
Safetensors
English
ComfyUI
Inpainting
Model card Files Community
21
FLUX.1-dev-Controlnet-Inpainting-Beta / README.md
ljp's picture
ljp
Update README.md
a25439d verified
preview code
|
raw
history blame
3.48 kB
metadata
license: other
license_name: flux-1-dev-non-commercial-license
license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md
language:
  - en
base_model:
  - black-forest-labs/FLUX.1-dev
pipeline_tag: image-to-image
tags:
  - ComfyUI
  - Inpainting
library_name: diffusers
alibaba alimama

FLUX.1-dev ControlNet Inpainting - Beta

This repository hosts an improved Inpainting ControlNet checkpoint for the FLUX.1-dev model, developed by the AlimamaCreative Team.

Key Enhancements

Our latest inpainting model brings significant improvements compared to the previous version:

  1. 1024 Resolution Support: Capable of directly processing and generating 1024x1024 resolution images without additional upscaling steps, providing higher quality and more detailed output results.
  2. Enhanced Detail Generation: Fine-tuned to capture and reproduce finer details in inpainted areas.
  3. Improved Prompt Control: Offers more precise control over generated content through enhanced prompt interpretation.

Showcase

The following images were generated using a ComfyUI workflow with these settings (click here to download): control-strength = 1.0, control-end-percent = 1.0, true_cfg = 1.0

Image & Prompt Input Alpha Version Beta Version
Input Image A > B Alpha Beta

ComfyUI Usage Guidelines:

Download example ComfyUI workflow here.

  • Using t5xxl-FP16 and flux1-dev-fp8 models for 28-step inference:
    • GPU memory usage: 27GB
    • Inference time: 27 seconds (cfg=3.5), 15 seconds (cfg=1)
  • For optimal results, experiment with lower values for control-strength, control-end-percent, and `cfg
Parameter Recommended Range Effect
control-strength 0.0 - 1.0 Controls how much influence the ControlNet has on the generation. Higher values result in stronger adherence to the control image.
control-end-percent 0.0 - 1.0 Determines at which point in the denoising process the ControlNet influence ends. Lower values allow for more creative freedom in later steps.
cfg (Classifier-Free Guidance Scale) 1.0 - 30.0 Influences how closely the generation follows the prompt. Higher values increase prompt adherence but may reduce image quality.

Model Specifications

  • Training dataset: 15M images from LAION2B and proprietary sources
  • Optimal inference resolution: 1024x1024

Diffusers Integration

  1. Install the required diffusers version:
pip install diffusers==0.30.2
  1. Clone this repository:
git clone https://github.com/alimama-creative/FLUX-Controlnet-Inpainting.git
  1. Configure image_path, mask_path, and prompt in main.py, then execute:
python main.py

License

Our model weights are released under the FLUX.1 [dev] Non-Commercial License.