Error loading `HiDreamImagePipeline`
Hi, thanks for uploading this compressed model. I was hoping with DF11 compression it would squeeze into 24GB of VRAM, but sadly it still seems to need more than that. I still decided to give it a try on my rtx 4090, with pipe.enable_model_cpu_offload() changed to pipe.enable_sequential_cpu_offload(), but I ended up with an error on this line before I even got to that point:
pipe = HiDreamImagePipeline.from_pretrained(
This is the error I get:
ValueError: The component <class 'transformers.models.t5.tokenization_t5._LazyModule.__getattr__.<locals>.Placeholder'> of <class 'diffusers.pipelines.hidream_image.pipeline_hidream_image.HiDreamImagePipeline'> cannot be loaded as it does not seem to have any of the loading methods defined in {'ModelMixin': ['save_pretrained', 'from_pretrained'], 'SchedulerMixin': ['save_pretrained', 'from_pretrained'], 'DiffusionPipeline': ['save_pretrained', 'from_pretrained'], 'OnnxRuntimeModel': ['save_pretrained', 'from_pretrained'], 'PreTrainedTokenizer': ['save_pretrained', 'from_pretrained'], 'PreTrainedTokenizerFast': ['save_pretrained', 'from_pretrained'], 'PreTrainedModel': ['save_pretrained', 'from_pretrained'], 'FeatureExtractionMixin': ['save_pretrained', 'from_pretrained'], 'ProcessorMixin': ['save_pretrained', 'from_pretrained'], 'ImageProcessingMixin': ['save_pretrained', 'from_pretrained'], 'ORTModule': ['save_pretrained', 'from_pretrained']}.
I manually checked that I am using the latest version of dfloat11[cuda12] and diffusers. I am using windows. What could be the problem here?
Update: I just tried the very same venv with the DF11 version of Flux.1-dev, and it works after installing some missing dependencies, so I guess it should not be my package versions?
Never mind, after installing those missing dependencies that Flux complained about, now the inference script for HiDream works as well. Those dependencies were protobuf and sentencepiece. Not sure if both of them were necessary for HiDream to run, or if it is just one of them.
Sadly, even with the compression to DF11 format, it is indeed still requires too much VRAM for my 4090. But for some reason, pipe.enable_sequential_cpu_offload() seems to do nothing to save VRAM, and is in fact worse than pipe.enable_model_cpu_offload().