about flux models
why are there so many flux models out there, what is the meaning of fp 8, 16, and what fp is this one?..
#sorry my question is bad
@noelsalazar
They are different levels of precisions to store the model weights and run them
fp32 - works in basically everything(cpu, gpu) but isn't used very often since its 2x slower then fp16/bf16 and uses 2x more vram with no increase in quality.
fp16 - uses 2x less vram and 2x faster speed then fp32 while being same quality but only works in gpu and unstable in training(Flux.1 dev will take 24gb vram at the least with this)
bf16(this model's default precision) - same benefits as fp16 and only works in gpu but is usually stable in training. in inference, bf16 is better for modern gpus while fp16 is better for older gpus(Flux.1 dev will take 24gb vram at the least with this)
fp8 - only works in gpu, uses 2x less vram less then fp16/bf16 but there is a quality loss, can be 2x faster on very modern gpus(4090, h100). (Flux.1 dev will take 12gb vram at the least)
q8/int8 - only works in gpu, uses around 2x less vram then fp16/bf16 and very similar in quality, maybe slightly worse then fp16, better quality then fp8 though but slower. (Flux.1 dev will take 14gb vram at the least)
q4/bnb4/int4 - only works in gpu, uses 4x less vram then fp16/bf16 but a quality loss, slightly worse then fp8. (Flux.1 dev only requires 8gb vram at the least)
@noelsalazar
They are different levels of precisions to store the model weights and run them
fp32 - works in basically everything(cpu, gpu) but isn't used very often since its 2x slower then fp16/bf16 and uses 2x more vram with no increase in quality.
fp16 - uses 2x less vram and 2x faster speed then fp32 while being same quality but only works in gpu and unstable in training(Flux.1 dev will take 24gb vram at the least with this)
bf16(this model's default precision) - same benefits as fp16 and only works in gpu but is usually stable in training. in inference, bf16 is better for modern gpus while fp16 is better for older gpus(Flux.1 dev will take 24gb vram at the least with this)fp8 - only works in gpu, uses 2x less vram less then fp16/bf16 but there is a quality loss, can be 2x faster on very modern gpus(4090, h100). (Flux.1 dev will take 12gb vram at the least)
q8/int8 - only works in gpu, uses around 2x less vram then fp16/bf16 and very similar in quality, maybe slightly worse then fp16, better quality then fp8 though but slower. (Flux.1 dev will take 14gb vram at the least)q4/bnb4/int4 - only works in gpu, uses 4x less vram then fp16/bf16 but a quality loss, slightly worse then fp8. (Flux.1 dev only requires 8gb vram at the least)
Q8 GGUF?
@noelsalazar Yes by q8 I meant q8 gguf