Spaces:
Running
Update app.py
Key Changes and Explanations:
load_sdxl_checkpoint (Corrected): This function now correctly extracts the state dictionaries for both text encoders (text_encoder1_state and text_encoder2_state), the VAE (vae_state), and the UNet (unet_state), using the appropriate key prefixes. It still assumes the Illustrious-xl model uses the standard SDXL prefixes for these components, which is a reasonable assumption.
build_diffusers_model (Corrected):
Loads the configurations from the reference model (or the default SDXL base) for all components: CLIPTextConfig for text_encoder, CLIPTextConfig for text_encoder_2, AutoencoderKL for vae, and UNet2DConditionModel for unet.
Creates instances of CLIPTextModel for text_encoder1 and now properly uses CLIPTextModelWithProjection for text_encoder2, and AutoencoderKL, and UNet2DConditionModel using these loaded configurations. This is crucial for getting the correct model architecture.
Loads the extracted state dictionaries into the corresponding model instances using strict=False. This handles potential key mismatches or extra keys in the Illustrious-xl checkpoint.
Sets the components to float16 and moves to the CPU.
convert_and_save_sdxl_to_diffusers: Remains mostly the same, but now correctly uses the two text encoders.
Other Functions: The rest of the code (downloading, uploading, Gradio interface) remains largely unchanged.
Testing and Further Steps
Test Thoroughly: Test this revised code with the Illustrious-xl model. It should now load the checkpoint correctly and create a Diffusers pipeline.
Verify Functionality: After converting, test the generated Diffusers model. Generate some images and compare them to the expected output from the Illustrious-xl model. This is crucial to ensure the conversion was successful and the model is working as intended.
Key Prefixes (If Still Errors): If you still encounter errors, it's possible that the Illustrious-xl model uses different key prefixes than the standard SDXL prefixes. In this case, you'll need to inspect the checkpoint's state dictionary keys directly (using a simplified loading script) to determine the correct prefixes and adjust load_sdxl_checkpoint accordingly.