THUDM

/

GLM-4-32B-0414

like 286

Follow

Z.ai & THUKEG 2.17k

RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

This error occurs when I use bitsandbytes to quantize this model to implement your demo.