ShinoharaHare/Waifu-Decensor-XL

Waifu Decensor XL

Censored	Mask	Decensored

Overview

[Experimental] This model is in an early experimental stage. Its performance may not be reliable for all use cases, and results can vary depending on the input image and mask.

Waifu Decensor XL is a specialized Stable Diffusion XL (SDXL) model designed for inpainting, particularly for "de-censoring" or removing mosaic effects from anime-style illustrations.

This model was fine-tuned from WAI-NSFW-illustrious-SDXL V14.0. Its UNet has been modified to accept 5 additional input channels: 1 for the mask and 4 for the latents of the censored image.

A key feature is the mask's value range:

• 0.0 (Black): The area remains unchanged.
• 0.5 (Gray): The area is "de-mosaiced," using the censored information as a guide.
• 1.0 (White): The area is inpainted from scratch.

🧨 Diffusers

import torch
from diffusers import DiffusionPipeline
from diffusers.utils import load_image

pipeline = DiffusionPipeline.from_pretrained(
    'ShinoharaHare/Waifu-Decensor-XL',
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    use_flash_attention=True # optional
)
pipeline.to('cuda')

censored = load_image('https://huggingface.co/ShinoharaHare/Waifu-Decensor-XL/resolve/main/images/miku/censored.png')
mask = load_image('https://huggingface.co/ShinoharaHare/Waifu-Decensor-XL/resolve/main/images/miku/mask.png', lambda x: x.convert('L'))

decensored = pipeline(
    image=censored,
    mask_image=mask,
    width=censored.width,
    height=censored.height,
    strength=1.0,
    num_inference_steps=28,
    ensure_image_consistency=True, # It's recommended to enable this option to ensure that the unmasked areas remain unchanged and to reduce color discrepancies.
    dilate_latent_mask=5 # If any you haven’t left extra space around the mask, make sure to set this parameter. Otherwise, you may notice significant degradation along the edges.
).images[0]

decensored.show()

Limitations

• Color Discrepancies: The model may struggle with color consistency in two ways:

  1. Global Mismatch: Without ensure_image_consistency=True, you may notice a significant color shift across the entire image. Enabling this option is crucial as it locks the unmasked areas to their original colors. However, even with this enabled, the newly generated area can still have a color tone that does not perfectly match its surrounding context.
  2. Local Inconsistency: For demosaicing tasks, the model is designed to infer the original colors from the color palette of the censored block itself. However, it may sometimes fail to follow this hint, resulting in colors that deviate from what the censored area suggests.

• Large-area inpainting: Inpainting very large regions may result in lower-quality or less coherent outputs.

If you have suggestions on how to improve above issues, please share them with us!