license: mit
language:
- en
base_model:
- PixArt-alpha/PixArt-Sigma-XL-2-1024-MS
- stable-diffusion-v1-5/stable-diffusion-v1-5
tags:
- image-generation
- text-to-image
- art
- pixart-sigma
- image
TinyBreaker (prototype1)
Overview
TinyBreaker is a hybrid model designed for efficient image generation even on mid and low-end hardware! By integrating the strengths of both PixArt and SD1.5 (Photon) models, TinyBreaker offers an elegant solution that doesn't compromise on quality or performance.
Key Features
- Hybrid Model Architecture: Combines base image generation with PixArt and refinement using Photon (or any SD1 model), optimized for minimal parameter use.
- Efficient Parameter Use: A lean 0.6 billion parameters in the base model enable high-quality images without hefty computational demands.
- Quick Performance: Generate stunning images of size 1536×1024 in approximately ~12 seconds on an NVIDIA RTX 3080 GPU, with ongoing efforts to achieve sub-10-second generation times.
- High Prompt Adherence: Ensures generated images are closely aligned with user instructions and expectations due to PixArt model integration.
- Optimized Latent Space Handling: Utilizes Tiny Autoencoders for efficient latent space conversion.
Current Usage
TinyBreaker is currently utilizable via ComfyUI. To use TinyBreaker, you'll need to install custom nodes specific to this model through ComfyUI-TinyBreaker on GitHub.
Note: The current "Prototype1" version of TinyBreaker uses PixArt Sigma 1024 in combination with Photon (SD1.5) without additional training or fine-tuning. In the future, if I have the resources, I plan to train both models together to generate images of even higher quality.
Limitations
- Text Generation: Generating legible text within images is a challenge due to PixArt's training limitations. Enhancements in this area may require extensive retraining.
Future Directions
I'm dedicated to enhancing TinyBreaker's speed and accessibility, particularly for users with mid-range or lower-end hardware setups. Look forward to future updates as I work on enhancing TinyBreaker's capabilities.