Error loading with transformers lib

#3
by YoannRuss - opened

Hello, using transformers 4.42.4 as :

from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("ds4sd/SmolDocling-256M-preview-mlx-bf16")
model = AutoModelForImageTextToText.from_pretrained("ds4sd/SmolDocling-256M-preview-mlx-bf16")

I got the following error
:
RuntimeError: Error(s) in loading state_dict for Idefics3Model:
size mismatch for vision_model.embeddings.patch_embedding.weight: copying a param with shape torch.Size([768, 16, 16, 3]) from checkpoint, the shape in current model is torch.Size([768, 3, 16, 16]).

Also is there a tutorial to use ds4sd with transformers ?

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment