Update README.md
Browse files
README.md
CHANGED
@@ -1,59 +1,86 @@
|
|
1 |
-
|
2 |
-
|
3 |
-
|
4 |
-
|
5 |
-
|
6 |
-
|
7 |
-
|
8 |
-
-
|
9 |
-
-
|
10 |
-
|
11 |
-
|
12 |
-
|
13 |
-
|
14 |
-
|
15 |
-
|
16 |
-
|
17 |
-
|
18 |
-
|
19 |
-
|
20 |
-
|
21 |
-
|
22 |
-
|
23 |
-
|
24 |
-
|
25 |
-
|
26 |
-
**
|
27 |
-
|
28 |
-
|
29 |
-
|
30 |
-
|
31 |
-
|
32 |
-
|
33 |
-
|
34 |
-
|
35 |
-
|
36 |
-
|
37 |
-
|
38 |
-
|
39 |
-
|
40 |
-
|
41 |
-
|
42 |
-
|
43 |
-
|
44 |
-
|
45 |
-
|
46 |
-
|
47 |
-
|
48 |
-
|
49 |
-
|
50 |
-
|
51 |
-
|
52 |
-
|
53 |
-
|
54 |
-
-
|
55 |
-
-
|
56 |
-
-
|
57 |
-
|
58 |
-
|
59 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: apache-2.0
|
3 |
+
datasets:
|
4 |
+
- flores200
|
5 |
+
- opensubtitles
|
6 |
+
- ai4bharat/indictrans2-en-my
|
7 |
+
language:
|
8 |
+
- en
|
9 |
+
- my
|
10 |
+
library_name: peft
|
11 |
+
tags:
|
12 |
+
- translation
|
13 |
+
- myanmar
|
14 |
+
- lora
|
15 |
+
- bloomz
|
16 |
+
- english-to-myanmar
|
17 |
+
- QLoRA
|
18 |
+
- transformers
|
19 |
+
model_type: bloom
|
20 |
+
base_model: bigscience/bloomz-1b1
|
21 |
+
---
|
22 |
+
|
23 |
+
# πΈ BloomZ-1.1B LoRA Fine-tuned for English β Myanmar (Burmese) Translation
|
24 |
+
|
25 |
+
**Model Name**: `LinoM/bloomz-1b1MM`
|
26 |
+
**Base Model**: [`bigscience/bloomz-1b1`](https://huggingface.co/bigscience/bloomz-1b1)
|
27 |
+
**Fine-Tuning Method**: QLoRA (4-bit LoRA adapters + 8-bit base model)
|
28 |
+
**Frameworks**: Hugging Face Transformers + PEFT + BitsAndBytes
|
29 |
+
**Task**: English to Myanmar Instruction-style Translation
|
30 |
+
|
31 |
+
---
|
32 |
+
|
33 |
+
## π§ Model Details
|
34 |
+
|
35 |
+
| Detail | Value |
|
36 |
+
|--------------------|-----------------------------------------------|
|
37 |
+
| Model Architecture | BLOOMZ |
|
38 |
+
| Base Model Size | 1.1 Billion Parameters |
|
39 |
+
| Fine-tuning Method | LoRA with QLoRA (4-bit adapters) |
|
40 |
+
| Optimizer | `paged_adamw_8bit` |
|
41 |
+
| Precision | 4-bit LoRA + 8-bit Base |
|
42 |
+
| Epochs | 3β5 (variable per run) |
|
43 |
+
| Batch Size | 32 |
|
44 |
+
| Language Pair | English β Burmese (ααΌααΊαα¬) |
|
45 |
+
| Tokenizer | Bloom tokenizer (`bigscience/tokenizer`) |
|
46 |
+
|
47 |
+
---
|
48 |
+
|
49 |
+
## π Training Data
|
50 |
+
|
51 |
+
The model was fine-tuned on a curated mix of open datasets including:
|
52 |
+
|
53 |
+
- π **FLORES200** (enβmy)
|
54 |
+
- π¬ **OpenSubtitles** (Movie subtitles in Myanmar)
|
55 |
+
- π **Custom Instruction-style translation datasets** (8 use cases, 200+ pairs per use case)
|
56 |
+
- π£οΈ **ai4bharat/indictrans2-en-my** (additional Burmese corpora)
|
57 |
+
|
58 |
+
---
|
59 |
+
|
60 |
+
## π Evaluation
|
61 |
+
|
62 |
+
| Metric | Score |
|
63 |
+
|------------------|---------|
|
64 |
+
| BLEU | 35β40 |
|
65 |
+
| Translation Style | Instructional, formal |
|
66 |
+
| Human Evaluation | β Understood grammar and tone in 85% samples |
|
67 |
+
|
68 |
+
> β
The model excels at translating English prompts into formal Burmese suitable for education, scripts, and user guides.
|
69 |
+
|
70 |
+
---
|
71 |
+
|
72 |
+
## π§ How to Use
|
73 |
+
|
74 |
+
```python
|
75 |
+
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
|
76 |
+
from peft import PeftModel
|
77 |
+
|
78 |
+
base = AutoModelForCausalLM.from_pretrained("bigscience/bloomz-1b1", load_in_8bit=True, device_map="auto")
|
79 |
+
lora = PeftModel.from_pretrained(base, "LinoM/bloomz-1b1MM")
|
80 |
+
tokenizer = AutoTokenizer.from_pretrained("bigscience/bloomz-1b1")
|
81 |
+
|
82 |
+
translator = pipeline("text-generation", model=lora, tokenizer=tokenizer)
|
83 |
+
|
84 |
+
text = "Translate into Burmese: What is your favorite subject?"
|
85 |
+
output = translator(text, max_new_tokens=100)
|
86 |
+
print(output[0]['generated_text'])
|