Update README.md
Browse files
README.md
CHANGED
|
@@ -10,6 +10,9 @@ tags:
|
|
| 10 |
license: apache-2.0
|
| 11 |
language:
|
| 12 |
- en
|
|
|
|
|
|
|
|
|
|
| 13 |
---
|
| 14 |
|
| 15 |
# Uploaded model
|
|
@@ -31,7 +34,7 @@ lora_alpha:512
|
|
| 31 |
lr:3e-4
|
| 32 |
embedding_lr: 3e-5
|
| 33 |
|
| 34 |
-
狙い:
|
| 35 |
|
| 36 |
|
| 37 |
```python
|
|
|
|
| 10 |
license: apache-2.0
|
| 11 |
language:
|
| 12 |
- en
|
| 13 |
+
- ja
|
| 14 |
+
datasets:
|
| 15 |
+
- ikedachin/CC-news-2024-October-cleaned-cpt-set-250127
|
| 16 |
---
|
| 17 |
|
| 18 |
# Uploaded model
|
|
|
|
| 34 |
lr:3e-4
|
| 35 |
embedding_lr: 3e-5
|
| 36 |
|
| 37 |
+
狙い: lora_alphaを大きくして、SFTによる記憶忘却に耐えられるようにCPTによるベースモデルへの知識埋め込みの影響を大きくする
|
| 38 |
|
| 39 |
|
| 40 |
```python
|