init

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+*.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,139 @@
+---
+base_model: black-forest-labs/FLUX.1-dev
+library_name: diffusers
+tags:
+- text-to-image
+- diffusers-training
+- diffusers
+- lora
+- flux
+- flux-diffusers
+widget:
+- text: "a 3dicon, a joyful yellow emoji with smiling eyes, showing two hands in front as if reaching to give a hug"
+  output:
+    url: >-
+      images/3d_merged1.png
+- text: "a 3dicon, a miniature galaxy in a glass dome, with swirling stars and nebulae in vibrant blues and purples, surrounded by a group of colorful icons on a black background"
+  output:
+    url: >-
+      images/3d_merged2.png
+- text: "a 3dicon, an alien with green skin"
+  output:
+    url: >-
+      images/3d_merged3.png
+instance_prompt: null
+datasets:
+  - linoyts/3d_icon
+---
+# LoRA for FLUX.1-dev - 3D Icon Style
+
+This repository contains a LoRA (Low-Rank Adaptation) fine-tuned on `black-forest-labs/FLUX.1-dev` to generate images in the artistic style of 3D icons. This work is part of the blog post, "Fine-Tuning FLUX.1-dev on consumer hardware and in FP8".
+
+<Gallery />
+
+## Inference
+
+There are two main ways to use this LoRA for inference: loading the adapter on the fly or merging it with the base model.
+
+### Option 1: Loading LoRA Adapters
+
+This approach offers flexibility, allowing you to easily switch between different LoRA styles.
+
+```python
+from diffusers import FluxPipeline
+import torch
+
+ckpt_id = "black-forest-labs/FLUX.1-dev"
+pipeline = FluxPipeline.from_pretrained(
+    ckpt_id, torch_dtype=torch.float16
+)
+pipeline.load_lora_weights("derekl35/3dicon_qlora_flux", weight_name="pytorch_lora_weights.safetensors")
+pipeline.enable_model_cpu_offload()
+
+image = pipeline(
+    "a 3dicon, an alien with green skin",
+    num_inference_steps=28,
+    guidance_scale=3.5,
+    height=768,
+    width=768,
+    generator=torch.manual_seed(0)
+).images[0]
+image.save("3d_loaded.png")
+```
+
+### Option 2: Merging LoRA into Base Model
+
+Merging the LoRA into the base model can lead to slightly faster inference and is useful when you want to use a single style consistently.
+
+```python
+from diffusers import FluxPipeline, AutoPipelineForText2Image, FluxTransformer2DModel
+import torch
+
+ckpt_id = "black-forest-labs/FLUX.1-dev"
+pipeline = FluxPipeline.from_pretrained(
+    ckpt_id, text_encoder=None, text_encoder_2=None, torch_dtype=torch.float16
+)
+pipeline.load_lora_weights("derekl35/3dicon_qlora_flux", weight_name="pytorch_lora_weights.safetensors")
+pipeline.fuse_lora()
+pipeline.unload_lora_weights()
+
+# You can save the fused transformer for later use
+# pipeline.transformer.save_pretrained("fused_transformer")
+
+pipeline.enable_model_cpu_offload()
+image = pipeline(
+    "a 3dicon, an alien with green skin",
+    num_inference_steps=28,
+    guidance_scale=3.5,
+    height=768,
+    width=768,
+    generator=torch.manual_seed(0)
+).images[0]
+image.save("3d_merged.png")
+```
+
+you can also requantize model:
+
+```python
+from diffusers import FluxPipeline, AutoPipelineForText2Image, FluxTransformer2DModel, BitsAndBytesConfig
+import torch 
+
+ckpt_id = "black-forest-labs/FLUX.1-dev"
+pipeline = FluxPipeline.from_pretrained(
+    ckpt_id, text_encoder=None, text_encoder_2=None, torch_dtype=torch.float16
+)
+pipeline.load_lora_weights("derekl35/3dicon_qlora_flux", weight_name="pytorch_lora_weights.safetensors")
+pipeline.fuse_lora()
+pipeline.unload_lora_weights()
+
+pipeline.transformer.save_pretrained("fused_transformer")
+
+ckpt_id = "black-forest-labs/FLUX.1-dev"
+bnb_4bit_compute_dtype = torch.bfloat16
+
+nf4_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_compute_dtype=bnb_4bit_compute_dtype,
+)
+transformer = FluxTransformer2DModel.from_pretrained(
+    "fused_transformer",
+    quantization_config=nf4_config,
+    torch_dtype=bnb_4bit_compute_dtype,
+)
+
+pipeline = AutoPipelineForText2Image.from_pretrained(
+    ckpt_id, transformer=transformer, torch_dtype=bnb_4bit_compute_dtype
+)
+pipeline.enable_model_cpu_offload()
+
+image = pipeline(
+    "a 3dicon, an alien with green skin",
+    num_inference_steps=28,
+    guidance_scale=3.5,
+    height=768,
+    width=768,
+    generator=torch.manual_seed(0)
+).images[0]
+image.save("3d_merged.png")
+```

pytorch_lora_weights.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:860ce78f09bdb29b0ca3bc34e879c0d50f645c814b106c19ff3aadb4e3e265cc
+size 18727592