Flux_Amano_Mono_LoKR

This is a LyCORIS adapter derived from black-forest-labs/FLUX.1-dev.

No validation prompt was used during training.

None

Validation settings

CFG: 3.0
CFG Rescale: 0.0
Steps: 20
Sampler: FlowMatchEulerDiscreteScheduler
Seed: 42
Resolution: 1024x1024
Skip-layer guidance:

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

Prompt
unconditional (blank prompt)

Negative Prompt
blurry, cropped, ugly

Prompt
Yoshitaka Amano Ink Illustration — An Armored Warrior emerges from a VAST WHITE VOID in an EXPLOSIVE DIAGONAL COMPOSITION. The warrior wears ELABORATE, ORNATE ARMOR detailed with DENSE CROSS-HATCHING on the legs and waist. Their face is cast in shadow as a SOLID FILL OF PURE BLACK beneath a winged helmet. A massive SWIRLING CAPE rendered with BOLD BRUSH STROKES billows dramatically behind them. They grip an ORNATE SWORD in their right hand. Their VOLUMINOUS FLOWING HAIR erupts from beneath the helmet in WISPY TENDRIL-LIKE STROKES.

Negative Prompt
blurry, cropped, ugly

Prompt
Yoshitaka Amano Ink Illustration — A serene Ethereal Figure stands isolated against an ABYSSAL BLACK BACKGROUND, framed by a white circle. The figure's FACE IS EXPRESSED WITH MINIMALIST LINES showing a melancholic downturned gaze. Their VOLUMINOUS FLOWING HAIR is rendered as a SOLID FILL OF PURE BLACK that cascades like dark ribbons. They hold a single flower with DELICATE ELONGATED FINGERS. Their flowing gown is defined by DELICATE CALLIGRAPHIC LINES creating a cascade of fabric pooling at their feet.

Negative Prompt
blurry, cropped, ugly

Prompt
Yoshitaka Amano Ink Illustration — A Wild Warrior confronts a massive Grotesque Beast in a scene of baroque complexity with dramatic scale contrast. The beast's serpentine body is rendered with SOLID FILLS OF PURE BLACK and DENSE CROSS-HATCHING, coiling to frame the smaller warrior. The warrior's face is a mask of fury rendered with SCRATCHY, NERVOUS LINE WORK. PARALLEL SPEED LINES and EXPLOSIVE INK SPLATTERS create violent motion. The beast's MONSTROUS CLAWS are depicted with sharp aggressive strokes against the white ground.

Negative Prompt
blurry, cropped, ugly

Prompt
Yoshitaka Amano Ink Illustration — A CLOSE-UP profile of an Androgynous Ethereal Figure with closed almond-shaped eyes. Their face is rendered with DELICATE CALLIGRAPHIC LINES expressing serene melancholy. Their VOLUMINOUS FLOWING HAIR is a chaotic mass of FEATHERED INK and WISPY TENDRIL-LIKE STROKES that transitions into SOLID FILLS OF PURE BLACK merging with the background. Delicate petals drift through the NEGATIVE SPACE. The figure's head tilts back as if floating in a VAST WHITE VOID.

Negative Prompt
blurry, cropped, ugly

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

Training epochs: 2
Training steps: 2500
Learning rate: 0.0001
- Learning rate schedule: polynomial
- Warmup steps: 100
Max grad value: 0.1
Effective batch size: 4
- Micro-batch size: 4
- Gradient accumulation steps: 1
- Number of GPUs: 1
Gradient checkpointing: True
Prediction type: flow_matching (extra parameters=['shift=3', 'flux_guidance_mode=constant', 'flux_guidance_value=1.0'])
Optimizer: adamw_bf16
Trainable parameter precision: Pure BF16
Base model precision: int8-quanto
Caption dropout probability: 0.05%

LyCORIS Config:

{
    "algo": "lokr",
    "multiplier": 1.0,
    "linear_dim": 10000,
    "linear_alpha": 1,
    "factor": 16,
    "apply_preset": {
        "target_module": [
            "Attention",
            "FeedForward"
        ],
        "module_algo_map": {
            "Attention": {
                "factor": 16
            },
            "FeedForward": {
                "factor": 8
            }
        }
    }
}

Datasets

amano-mono-256

Repeats: 10
Total number of images: 30
Total number of aspect buckets: 3
Resolution: 0.065536 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

amano-mono-crop-256

Repeats: 10
Total number of images: 30
Total number of aspect buckets: 1
Resolution: 0.065536 megapixels
Cropped: True
Crop style: center
Crop aspect: square
Used for regularisation data: No

amano-mono-512

Repeats: 10
Total number of images: 30
Total number of aspect buckets: 5
Resolution: 0.262144 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

amano-mono-crop-512

Repeats: 10
Total number of images: 30
Total number of aspect buckets: 1
Resolution: 0.262144 megapixels
Cropped: True
Crop style: center
Crop aspect: square
Used for regularisation data: No

amano-mono-768

Repeats: 10
Total number of images: 30
Total number of aspect buckets: 7
Resolution: 0.589824 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

amano-mono-crop-768

Repeats: 10
Total number of images: 30
Total number of aspect buckets: 1
Resolution: 0.589824 megapixels
Cropped: True
Crop style: center
Crop aspect: square
Used for regularisation data: No

amano-mono-1024

Repeats: 10
Total number of images: 30
Total number of aspect buckets: 8
Resolution: 1.048576 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

amano-mono-crop-1024

Repeats: 10
Total number of images: 30
Total number of aspect buckets: 1
Resolution: 1.048576 megapixels
Cropped: True
Crop style: center
Crop aspect: square
Used for regularisation data: No

amano-mono-1440

Repeats: 10
Total number of images: 30
Total number of aspect buckets: 8
Resolution: 2.0736 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

amano-mono-crop-1440

Repeats: 10
Total number of images: 30
Total number of aspect buckets: 1
Resolution: 2.0736 megapixels
Cropped: True
Crop style: center
Crop aspect: square
Used for regularisation data: No

Inference

import torch
from diffusers import DiffusionPipeline
from lycoris import create_lycoris_from_weights


def download_adapter(repo_id: str):
    import os
    from huggingface_hub import hf_hub_download
    adapter_filename = "pytorch_lora_weights.safetensors"
    cache_dir = os.environ.get('HF_PATH', os.path.expanduser('~/.cache/huggingface/hub/models'))
    cleaned_adapter_path = repo_id.replace("/", "_").replace("\\", "_").replace(":", "_")
    path_to_adapter = os.path.join(cache_dir, cleaned_adapter_path)
    path_to_adapter_file = os.path.join(path_to_adapter, adapter_filename)
    os.makedirs(path_to_adapter, exist_ok=True)
    hf_hub_download(
        repo_id=repo_id, filename=adapter_filename, local_dir=path_to_adapter
    )

    return path_to_adapter_file
    
model_id = 'black-forest-labs/FLUX.1-dev'
adapter_repo_id = 'davidrd123/Flux_Amano_Mono_LoKR'
adapter_filename = 'pytorch_lora_weights.safetensors'
adapter_file_path = download_adapter(repo_id=adapter_repo_id)
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
lora_scale = 1.0
wrapper, _ = create_lycoris_from_weights(lora_scale, adapter_file_path, pipeline.transformer)
wrapper.merge_to()

prompt = "An astronaut is riding a horse through the jungles of Thailand."


## Optional: quantise the model to save on vram.
## Note: The model was quantised during training, and so it is recommended to do the same during inference time.
from optimum.quanto import quantize, freeze, qint8
quantize(pipeline.transformer, weights=qint8)
freeze(pipeline.transformer)
    
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
model_output = pipeline(
    prompt=prompt,
    num_inference_steps=20,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(42),
    width=1024,
    height=1024,
    guidance_scale=3.0,
).images[0]

model_output.save("output.png", format="PNG")

davidrd123
/

Flux_Amano_Mono_LoKR