add how to diffusers (#6)

- add how to diffusers (e8f3a76443118248a52e7d2210734ff2f016ed78)


Co-authored-by: Radamés Ajna <[email protected]>

README.md

@@ -19,6 +19,71 @@ Separate repos for usage in diffusers can be found here:<br>
 1.5: https://huggingface.co/DionTimmer/controlnet_qrcode-control_v11p_sd21<br>
 2.1: https://huggingface.co/DionTimmer/controlnet_qrcode-control_v1p_sd15<br>
 
+## How to use with Diffusers
+
+
+```bash
+pip -q install diffusers transformers accelerate torch xformers
+```
+
+```python
+import torch
+from PIL import Image
+from diffusers import StableDiffusionControlNetImg2ImgPipeline, ControlNetModel, DDIMScheduler
+from diffusers.utils import load_image
+
+controlnet = ControlNetModel.from_pretrained("DionTimmer/controlnet_qrcode-control_v1p_sd15",
+                                             torch_dtype=torch.float16)
+
+pipe = StableDiffusionControlNetImg2ImgPipeline.from_pretrained(
+    "runwayml/stable-diffusion-v1-5",
+    controlnet=controlnet,
+    safety_checker=None,
+    torch_dtype=torch.float16
+)
+
+pipe.enable_xformers_memory_efficient_attention()
+pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config)
+pipe.enable_model_cpu_offload()
+
+def resize_for_condition_image(input_image: Image, resolution: int):
+    input_image = input_image.convert("RGB")
+    W, H = input_image.size
+    k = float(resolution) / min(H, W)
+    H *= k
+    W *= k
+    H = int(round(H / 64.0)) * 64
+    W = int(round(W / 64.0)) * 64
+    img = input_image.resize((W, H), resample=Image.LANCZOS)
+    return img
+
+
+# play with guidance_scale, controlnet_conditioning_scale and strength to make a valid QR Code Image
+
+# qr code image
+source_image = load_image("https://s3.amazonaws.com/moonup/production/uploads/6064e095abd8d3692e3e2ed6/A_RqHaAM6YHBodPLwqtjn.png")
+# initial image, anything
+init_image = load_image("https://s3.amazonaws.com/moonup/production/uploads/noauth/KfMBABpOwIuNolv1pe3qX.jpeg")
+condition_image = resize_for_condition_image(source_image, 768)
+init_image = resize_for_condition_image(init_image, 768)
+generator = torch.manual_seed(123121231)
+image = pipe(prompt="a bilboard in NYC with a qrcode",
+             negative_prompt="ugly, disfigured, low quality, blurry, nsfw", 
+             image=init_image,
+             control_image=condition_image,
+             width=768,
+             height=768,
+             guidance_scale=20,
+             controlnet_conditioning_scale=1.5,
+             generator=generator,
+             strength=0.9, 
+             num_inference_steps=150,
+            )
+
+image.images[0]
+
+```
+
 ## Performance and Limitations
 
 These models perform quite well in most cases, but please note that they are not 100% accurate. In some instances, the QR code shape might not come through as expected. You can increase the ControlNet weight to emphasize the QR code shape. However, be cautious as this might negatively impact the style of your output.**To optimize for scanning, please generate your QR codes with correction mode 'H' (30%).**