How to get infernece on video stream
#18
by
bilal1060
- opened
Hi, I'm a beginner and working on a real-time project where I want to detect a person from a video stream and then pass the cropped image to this model to generate a description .
I have Nvidia V100 GPU , please guide me how I can decrease the processing time as this GPU does not support flash attention. IS it possible to get 5 or 6 fps .
torch=2.4.0+cu118，flash_attn-2.6.2+cu118torch2.4cxx11abiTRUE-cp312-cp312-linux_x86_64.whl+python3.12
I have this problem too, have you solved it?