README.md

---
license_name: bria-2.3
license: other
license_link: https://bria.ai/bria-huggingface-model-license-agreement/
library_name: diffusers
inference: false
tags:
- text-to-image
- legal liability
- commercial use
- ID preservation Adapter
extra_gated_description: Model weights from BRIA AI can be obtained with the purchase of a commercial license. Fill in the form below and we reach out to you.
extra_gated_heading: "Fill in this form to request a commercial license for the model"
extra_gated_fields:
  Name: text
  Company/Org name: text
  Org Type (Early/Growth Startup, Enterprise, Academy): text
  Role: text
  Country: text
  Email: text
  By submitting this form, I agree to BRIA’s Privacy policy and Terms & conditions, see links below: checkbox
---



# BRIA 2.3 ID preservation Adapter

BRIA 2.3 ID preservation Adapter is a model designed to allows various style transfer operations or tweaks on facial image using textual prompts. The model is fully compatible with auxiliary models like ControlNets and LoRAs, enabling seamless integration into existing workflows.


Trained exclusively on the largest multi-source commercial-grade licensed dataset, BRIA 2.3 ID preservation Adapter guarantees best quality while safe for commercial use. The model provides full legal liability coverage for copyright and privacy infrigement and harmful content mitigation, as our dataset does not represent copyrighted materials, such as fictional characters, logos or trademarks, public figures, harmful content or privacy infringing content. 


This model is optimized to work sseamlessly in high resolution, upper body part facial images.


Join our [Discord community](https://discord.gg/Nxe9YW9zHS) for more information, tutorials, tools, and to connect with other users!


# What's New

BRIA 2.3 ID preservation Adapter can be applied on top of BRIA 2.3 Text-to-Image and therefore enable to use BRIA auxiliary models.





### Model Description

- **Developed by:** BRIA AI
- **Model type:** Latent diffusion image-to-image model
- **License:** [bria-2.3 inpainting Licensing terms & conditions](https://bria.ai/bria-huggingface-model-license-agreement/).
- Purchase is required to license and access the model.
- **Model Description:** BRIA 2.3 ID preservation Adapter was trained exclusively on a professional-grade, licensed dataset. It is designed for commercial use and includes full legal liability coverage.
- **Resources for more information:** [BRIA AI](https://bria.ai/)




### Get Access to the source code and pre-trained model
Interested in BRIA 2.3 ID preservation Adapter? Our Model is available for purchase.

**Purchasing access to BRIA 2.3 ID preservation Adapter ensures royalty management and full liability for commercial use.**


*Are you a startup or a student?* We encourage you to apply for our specialized Academia and [Startup Programs](https://pages.bria.ai/the-visual-generative-ai-platform-for-builders-startups-plan?_gl=1*cqrl81*_ga*MTIxMDI2NzI5OC4xNjk5NTQ3MDAz*_ga_WRN60H46X4*MTcwOTM5OTMzNC4yNzguMC4xNzA5Mzk5MzM0LjYwLjAuMA..) to gain access. These programs are designed to support emerging businesses and academic pursuits with our cutting-edge technology.


**Contact us today to unlock the potential of BRIA 2.3 ID preservation Adapter!** 

By submitting the form above, you agree to BRIA’s [Privacy policy](https://bria.ai/privacy-policy/) and [Terms & conditions](https://bria.ai/terms-and-conditions/).


### Best practices:
1. In your text prompt, start with a short description of the person in the image e.g., A Caucasian female with brown eyes and gray long hair.
2. Use a protrait image with large face (~20-80% of the image).
3. Use clean background (or use Bria's RMBG) - this will help you with the canny condition.
4. You can add style images, and apply using Bria's IP adapter
5. You can train your own LoRA on your desired style and use within this model.
   
### How To Use

#### install requirements
```python
# Tested with Python 3.10.16
opencv-python==4.10.0.84
torch==2.4.0
torchvision==0.19.0
pillow==10.4.0
transformers==4.43.4
diffusers==0.29.2
insightface==0.7.3
onnx==1.16.2
onnxruntime==1.18.1
accelerate==0.33.0
huggingface-hub==0.27.1
```

#### download needed files
```python
from huggingface_hub import snapshot_download, hf_hub_download

# Download face encoder
snapshot_download("fal/AuraFace-v1", local_dir="./models/auraface")

# download checkpoints
hf_hub_download(repo_id="briaai/BRIA-2.3-ID_Preservation", filename="checkpoint_105000/controlnet/config.json", local_dir="./checkpoints")
hf_hub_download(repo_id="briaai/BRIA-2.3-ID_Preservation", filename="checkpoint_105000/controlnet/diffusion_pytorch_model.safetensors", local_dir="./checkpoints")
hf_hub_download(repo_id="briaai/BRIA-2.3-ID_Preservation", filename="checkpoint_105000/ip-adapter.bin", local_dir="./checkpoints")
hf_hub_download(repo_id="briaai/BRIA-2.3-ID_Preservation", filename="image_encoder/pytorch_model.bin", local_dir="./checkpoints")
hf_hub_download(repo_id="briaai/BRIA-2.3-ID_Preservation", filename="image_encoder/config.json", local_dir="./checkpoints")

# download needed files
hf_hub_download(repo_id="briaai/BRIA-2.3-ID_Preservation", filename="pipeline_bria_id_preservation.py", local_dir=".")

hf_hub_download(repo_id="briaai/BRIA-2.3-ID_Preservation", filename="ip_adapter/attention_processor.py", local_dir=".")
hf_hub_download(repo_id="briaai/BRIA-2.3-ID_Preservation", filename="ip_adapter/resampler.py", local_dir=".")
hf_hub_download(repo_id="briaai/BRIA-2.3-ID_Preservation", filename="ip_adapter/utils.py", local_dir=".")
```

#### inference
```python
import cv2
import torch
import numpy as np
from PIL import Image

from transformers import CLIPVisionModelWithProjection
from diffusers.models import ControlNetModel

from insightface.app import FaceAnalysis

from pipeline_bria_id_preservation import BriaIDPreservationDiffusionPipeline, draw_kps


# Util functions
def resize_img(input_image, max_side=1280, min_side=1024, size=None, 
               pad_to_max_side=False, mode=Image.BILINEAR, base_pixel_number=64):
    
    w, h = input_image.size
    if size is not None:
        w_resize_new, h_resize_new = size
    else:
        ratio = min_side / min(h, w)
        w, h = round(ratio*w), round(ratio*h)
        ratio = max_side / max(h, w)
        input_image = input_image.resize([round(ratio*w), round(ratio*h)], mode)
        w_resize_new = (round(ratio * w) // base_pixel_number) * base_pixel_number
        h_resize_new = (round(ratio * h) // base_pixel_number) * base_pixel_number
    input_image = input_image.resize([w_resize_new, h_resize_new], mode)

    if pad_to_max_side:
        res = np.ones([max_side, max_side, 3], dtype=np.uint8) * 255
        offset_x = (max_side - w_resize_new) // 2
        offset_y = (max_side - h_resize_new) // 2
        res[offset_y:offset_y+h_resize_new, offset_x:offset_x+w_resize_new] = np.array(input_image)
        input_image = Image.fromarray(res)
    return input_image

def make_canny_condition(image, min_val=100, max_val=200, w_bilateral=True):
    if w_bilateral:
        image = cv2.cvtColor(np.array(image), cv2.COLOR_RGB2GRAY)
        bilateral_filtered_image = cv2.bilateralFilter(image, d=9, sigmaColor=75, sigmaSpace=75)
        image = cv2.Canny(bilateral_filtered_image, min_val, max_val)
    else:
        image = np.array(image)
        image = cv2.Canny(image, min_val, max_val)
    image = image[:, :, None]
    image = np.concatenate([image, image, image], axis=2)
    image = Image.fromarray(image)
    return image


# ================= Parameters =================
default_negative_prompt = "Text,Ugly,Morbid,Mutation,Blurry,Gross proportions,Long neck,Duplicate,Mutilated,Poorly drawn face,Deformed,Bad anatomy,Cloned face"

resolution = 1024
seed = 12345
device = "cuda" if torch.cuda.is_available() else "cpu"

# ckpts paths
face_adapter = f"./checkpoints/checkpoint_105000/ip-adapter.bin"
controlnet_path = f"./checkpoints/checkpoint_105000/controlnet"
base_model_path = f'briaai/BRIA-2.3'

# =================  Prepare face encoder =================
app = FaceAnalysis(
    name="auraface",
    providers=["CUDAExecutionProvider", "CPUExecutionProvider"],
    root=".",
)

app.prepare(ctx_id=0, det_size=(640, 640))

# =================  Prepare pipeline =================
# Load ControlNet models
controlnet_lnmks = ControlNetModel.from_pretrained(controlnet_path, torch_dtype=torch.float16)
controlnet_canny = ControlNetModel.from_pretrained("briaai/BRIA-2.3-ControlNet-Canny",
                                                torch_dtype=torch.float16)
    
controlnet = [controlnet_lnmks, controlnet_canny]

image_encoder = CLIPVisionModelWithProjection.from_pretrained(
        f"./checkpoints/image_encoder",
        torch_dtype=torch.float16,
    )
pipe = BriaIDPreservationDiffusionPipeline.from_pretrained(
        base_model_path,
        controlnet=controlnet,
        torch_dtype=torch.float16,
        image_encoder=image_encoder # For compatibility issues - needs to be there
    )

pipe = pipe.to(device)

pipe.use_native_ip_adapter=True

pipe.load_ip_adapter_instantid(face_adapter)

clip_embeds=None 


image_path = "<Set your image path>"
img = Image.open(image_path)

face_image = resize_img(img, max_side=resolution, min_side=resolution)
face_image_padded = resize_img(img, max_side=resolution, min_side=resolution, pad_to_max_side=True)
face_info = app.get(cv2.cvtColor(np.array(face_image), cv2.COLOR_RGB2BGR))
face_info = sorted(face_info, key=lambda x:(x['bbox'][2]-x['bbox'][0])*(x['bbox'][3]-x['bbox'][1]))[-1] # only use the maximum face
face_emb = face_info['embedding']
face_kps = draw_kps(face_image, face_info['kps'])


# ================= Parameters =================
kps_scale = 0.6
canny_scale = 0.4
ip_adapter_scale = 0.8
num_inference_steps = 30
guidance_scale = 5.0

prompt = "A male with brown eyes, blonde hair, short hair, in a white shirt, smiling, with a neutral background, cartoon style"

if canny_scale>0.0:
  canny_img = make_canny_condition(face_image, min_val=20, max_val=40, w_bilateral=True)
                
generator = torch.Generator(device=device).manual_seed(seed)


images = pipe(
    prompt = prompt,
    negative_prompt = default_negative_prompt,
    image_embeds = face_emb,
    image = [face_kps, canny_img] if canny_scale>0.0 else face_kps,
    controlnet_conditioning_scale = [kps_scale, canny_scale] if canny_scale>0.0 else kps_scale,
    ip_adapter_scale = ip_adapter_scale,
    num_inference_steps = num_inference_steps,
    guidance_scale = 5.0,
    generator = generator,
    visual_prompt_embds = clip_embeds,
    cross_attention_kwargs = None,
    num_images_per_prompt=1,
).images[0]

```