Hugging Face's logo

John6666
/
llama-joycaption-alpha-two-hf-llava-nf4

Image-to-Text
Transformers
Safetensors
English
llava
captioning
4-bit precision
bitsandbytes
Model card Files Community
3

Could you please make a batch inference script?

#1
by thesby - opened
Discussion
thesby

Batch inference will be much faster. Could you please make a batch inference script?

John6666
 Owner

Can't we just use the official script?
https://github.com/fpgaminer/joycaption/tree/main/scripts

John6666
 Owner

I made a modified version for my own use, but I have not released it until now because NF4 does not work well in my environment.
https://huggingface.co/John6666/llama-joycaption-alpha-two-hf-llava-nf4/blob/main/scripts/batch-caption.py

Sign up or log in to comment