README.md

---
base_model: THUDM/GLM-Z1-32B-0414
language:
- zh
- en
library_name: transformers
license: mit
pipeline_tag: text-generation
tags:
- llama-cpp
- gguf-my-repo
---

# Triangle104/GLM-Z1-32B-0414-Q8_0-GGUF
This model was converted to GGUF format from [`THUDM/GLM-Z1-32B-0414`](https://huggingface.co/THUDM/GLM-Z1-32B-0414) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
Refer to the [original model card](https://huggingface.co/THUDM/GLM-Z1-32B-0414) for more details on the model.

---
The GLM family welcomes a new generation of open-source models, the GLM-4-32B-0414
 series, featuring 32 billion parameters. Its performance is comparable 
to OpenAI's GPT series and DeepSeek's V3/R1 series, and it supports very
 user-friendly local deployment features. GLM-4-32B-Base-0414 was 
pre-trained on 15T of high-quality data, including a large amount of 
reasoning-type synthetic data, laying the foundation for subsequent 
reinforcement learning extensions. In the post-training stage, in 
addition to human preference alignment for dialogue scenarios, we also 
enhanced the model's performance in instruction following, engineering 
code, and function calling using techniques such as rejection sampling 
and reinforcement learning, strengthening the atomic capabilities 
required for agent tasks. GLM-4-32B-0414 achieves good results in areas 
such as engineering code, Artifact generation, function calling, 
search-based Q&A, and report generation. Some benchmarks even rival 
larger models like GPT-4o and DeepSeek-V3-0324 (671B).

GLM-Z1-32B-0414 is a reasoning model with deep thinking capabilities.
 This was developed based on GLM-4-32B-0414 through cold start and 
extended reinforcement learning, as well as further training of the 
model on tasks involving mathematics, code, and logic. Compared to the 
base model, GLM-Z1-32B-0414 significantly improves mathematical 
abilities and the capability to solve complex tasks. During the training
 process, we also introduced general reinforcement learning based on 
pairwise ranking feedback, further enhancing the model's general 
capabilities.

---
## Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux)

```bash
brew install llama.cpp

```
Invoke the llama.cpp server or the CLI.

### CLI:
```bash
llama-cli --hf-repo Triangle104/GLM-Z1-32B-0414-Q8_0-GGUF --hf-file glm-z1-32b-0414-q8_0.gguf -p "The meaning to life and the universe is"
```

### Server:
```bash
llama-server --hf-repo Triangle104/GLM-Z1-32B-0414-Q8_0-GGUF --hf-file glm-z1-32b-0414-q8_0.gguf -c 2048
```

Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.
```
git clone https://github.com/ggerganov/llama.cpp
```

Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
```
cd llama.cpp && LLAMA_CURL=1 make
```

Step 3: Run inference through the main binary.
```
./llama-cli --hf-repo Triangle104/GLM-Z1-32B-0414-Q8_0-GGUF --hf-file glm-z1-32b-0414-q8_0.gguf -p "The meaning to life and the universe is"
```
or 
```
./llama-server --hf-repo Triangle104/GLM-Z1-32B-0414-Q8_0-GGUF --hf-file glm-z1-32b-0414-q8_0.gguf -c 2048
```