RuntimeError: GPU is required to quantize or run quantize model.

by gangov - opened Oct 12, 2023

Oct 12, 2023

I'm using macbook pro m1, trying to run this through oobabooga, but I get the error:
RuntimeError: GPU is required to quantize or run quantize model.

Anyone knows how to fix this?

TheBloke

Owner Oct 13, 2023

GPTQs are not supported on macOS, please try GGUF quants instead.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment