Update README.md
Browse files
README.md
CHANGED
@@ -10,6 +10,42 @@ base_model: FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-32B-Preview
|
|
10 |
This model was converted to GGUF format from [`FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-32B-Preview`](https://huggingface.co/FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-32B-Preview) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
|
11 |
Refer to the [original model card](https://huggingface.co/FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-32B-Preview) for more details on the model.
|
12 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
13 |
## Use with llama.cpp
|
14 |
Install llama.cpp through brew (works on Mac and Linux)
|
15 |
|
|
|
10 |
This model was converted to GGUF format from [`FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-32B-Preview`](https://huggingface.co/FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-32B-Preview) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
|
11 |
Refer to the [original model card](https://huggingface.co/FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-32B-Preview) for more details on the model.
|
12 |
|
13 |
+
---
|
14 |
+
FuseO1-Preview
|
15 |
+
is our initial endeavor to enhance the System-II reasoning capabilities
|
16 |
+
of large language models (LLMs) through innovative model fusion
|
17 |
+
techniques. By employing our advanced SCE
|
18 |
+
merging methodologies, we integrate multiple open-source o1-like LLMs
|
19 |
+
into a unified model. Our goal is to incorporate the distinct knowledge
|
20 |
+
and strengths from different reasoning LLMs into a single, unified model
|
21 |
+
with strong System-II reasoning abilities, particularly in mathematics,
|
22 |
+
coding, and science domains.
|
23 |
+
|
24 |
+
|
25 |
+
|
26 |
+
|
27 |
+
|
28 |
+
|
29 |
+
|
30 |
+
|
31 |
+
|
32 |
+
To achieve this, we conduct two types of model merging:
|
33 |
+
|
34 |
+
|
35 |
+
Long-Long Reasoning Merging: This approach involves
|
36 |
+
model fusion across LLMs that utilize long-CoT reasoning, with the goal
|
37 |
+
of enhancing long-CoT reasoning capabilities. The resulted FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview achieves a Pass@1 accuracy of 74.0 on AIME24,
|
38 |
+
demonstrating significant performance improvements compared to the
|
39 |
+
OpenAI o1-preview (44.6) and OpenAI o1-mini (63.4), even approaching
|
40 |
+
OpenAI o1 (79.2).
|
41 |
+
Long-Short Reasoning Merging: This approach
|
42 |
+
involves model fusion between long-CoT and short-CoT LLMs, aiming to
|
43 |
+
improve reasoning capabilities in both long and short reasoning
|
44 |
+
processes. The resulted FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Instruct-32B-Preview and FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview
|
45 |
+
is capable of utilizing both long and short reasoning processes and
|
46 |
+
demonstrates relatively strong performance in long reasoning tasks.
|
47 |
+
|
48 |
+
---
|
49 |
## Use with llama.cpp
|
50 |
Install llama.cpp through brew (works on Mac and Linux)
|
51 |
|