Update README.md
Browse files
README.md
CHANGED
|
@@ -10,6 +10,42 @@ base_model: FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-32B-Preview
|
|
| 10 |
This model was converted to GGUF format from [`FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-32B-Preview`](https://huggingface.co/FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-32B-Preview) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
|
| 11 |
Refer to the [original model card](https://huggingface.co/FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-32B-Preview) for more details on the model.
|
| 12 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
## Use with llama.cpp
|
| 14 |
Install llama.cpp through brew (works on Mac and Linux)
|
| 15 |
|
|
|
|
| 10 |
This model was converted to GGUF format from [`FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-32B-Preview`](https://huggingface.co/FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-32B-Preview) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
|
| 11 |
Refer to the [original model card](https://huggingface.co/FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-32B-Preview) for more details on the model.
|
| 12 |
|
| 13 |
+
---
|
| 14 |
+
FuseO1-Preview
|
| 15 |
+
is our initial endeavor to enhance the System-II reasoning capabilities
|
| 16 |
+
of large language models (LLMs) through innovative model fusion
|
| 17 |
+
techniques. By employing our advanced SCE
|
| 18 |
+
merging methodologies, we integrate multiple open-source o1-like LLMs
|
| 19 |
+
into a unified model. Our goal is to incorporate the distinct knowledge
|
| 20 |
+
and strengths from different reasoning LLMs into a single, unified model
|
| 21 |
+
with strong System-II reasoning abilities, particularly in mathematics,
|
| 22 |
+
coding, and science domains.
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
To achieve this, we conduct two types of model merging:
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
Long-Long Reasoning Merging: This approach involves
|
| 36 |
+
model fusion across LLMs that utilize long-CoT reasoning, with the goal
|
| 37 |
+
of enhancing long-CoT reasoning capabilities. The resulted FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview achieves a Pass@1 accuracy of 74.0 on AIME24,
|
| 38 |
+
demonstrating significant performance improvements compared to the
|
| 39 |
+
OpenAI o1-preview (44.6) and OpenAI o1-mini (63.4), even approaching
|
| 40 |
+
OpenAI o1 (79.2).
|
| 41 |
+
Long-Short Reasoning Merging: This approach
|
| 42 |
+
involves model fusion between long-CoT and short-CoT LLMs, aiming to
|
| 43 |
+
improve reasoning capabilities in both long and short reasoning
|
| 44 |
+
processes. The resulted FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Instruct-32B-Preview and FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview
|
| 45 |
+
is capable of utilizing both long and short reasoning processes and
|
| 46 |
+
demonstrates relatively strong performance in long reasoning tasks.
|
| 47 |
+
|
| 48 |
+
---
|
| 49 |
## Use with llama.cpp
|
| 50 |
Install llama.cpp through brew (works on Mac and Linux)
|
| 51 |
|