LinoM commited on
Commit
6543f49
Β·
verified Β·
1 Parent(s): 1edaf18

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +86 -59
README.md CHANGED
@@ -1,59 +1,86 @@
1
- #my first time
2
- # BloomZ-1.1B LoRA for English to Burmese Translation
3
-
4
- ## Overview
5
-
6
- This model is a LoRA fine-tuned version of `bigscience/bloomz-1b1` for English-to-Myanmar (Burmese) instruction-style translation tasks.
7
-
8
- - **Base Model:** [bigscience/bloomz-1b1](https://huggingface.co/bigscience/bloomz-1b1)
9
- - **Fine-tuning Method:** QLoRA (8-bit base model + 4-bit LoRA adapters)
10
- - **Task:** English β†’ Myanmar translation
11
- - **Training Frameworks:** Hugging Face Transformers, PEFT, BitsAndBytes
12
-
13
- ## Citation
14
-
15
- **BibTeX:**
16
-
17
- ```bibtex
18
- @misc{bloomz1b1-myanmar-lora,
19
- author = {MgWai, et al.},
20
- title = {LoRA fine-tuned BloomZ-1.1B model for English to Burmese translation},
21
- year = {2025},
22
- url = {https://huggingface.co/LinoM/bloomz-1b1MM}
23
- }
24
- ```
25
-
26
- **APA:**
27
- MgWai, et al. (2025). *LoRA fine-tuned BloomZ-1.1B model for English to Burmese translation*. https://huggingface.co/LinoM/bloomz-1b1MM
28
-
29
- ## Glossary
30
-
31
- - **LoRA (Low-Rank Adaptation):** Efficient fine-tuning method using trainable rank decomposition matrices.
32
- - **Checkpoint:** Intermediate saved training state, e.g., `checkpoint-290`.
33
- - **BLEU Score:** Translation quality metric.
34
- - **GGUF:** Compact format for quantized LLMs used in `llama.cpp`.
35
-
36
- ## More Information
37
-
38
- - Target use case: Offline Burmese translation in low-resource environments (e.g., schools)
39
- - Model supports instruction-style prompts: e.g., "Translate to Burmese: I love you."
40
- - Trained on parallel datasets: English ↔ Burmese
41
-
42
- ## Model Card Authors
43
-
44
- - MgWai (developer, trainer)
45
- - Assisted by: ChatGPT (OpenAI)
46
-
47
- ## Contact
48
-
49
- πŸ“§
50
- Hugging Face: https://huggingface.co/LinoM
51
-
52
- ## Framework versions
53
-
54
- - Transformers: 4.41.0
55
- - PEFT: 0.16.0
56
- - Accelerate: 0.30.0
57
- - BitsAndBytes: 0.43.0
58
-
59
- LinoM/bloomz-1b1MM
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ datasets:
4
+ - flores200
5
+ - opensubtitles
6
+ - ai4bharat/indictrans2-en-my
7
+ language:
8
+ - en
9
+ - my
10
+ library_name: peft
11
+ tags:
12
+ - translation
13
+ - myanmar
14
+ - lora
15
+ - bloomz
16
+ - english-to-myanmar
17
+ - QLoRA
18
+ - transformers
19
+ model_type: bloom
20
+ base_model: bigscience/bloomz-1b1
21
+ ---
22
+
23
+ # 🌸 BloomZ-1.1B LoRA Fine-tuned for English β†’ Myanmar (Burmese) Translation
24
+
25
+ **Model Name**: `LinoM/bloomz-1b1MM`
26
+ **Base Model**: [`bigscience/bloomz-1b1`](https://huggingface.co/bigscience/bloomz-1b1)
27
+ **Fine-Tuning Method**: QLoRA (4-bit LoRA adapters + 8-bit base model)
28
+ **Frameworks**: Hugging Face Transformers + PEFT + BitsAndBytes
29
+ **Task**: English to Myanmar Instruction-style Translation
30
+
31
+ ---
32
+
33
+ ## 🧠 Model Details
34
+
35
+ | Detail | Value |
36
+ |--------------------|-----------------------------------------------|
37
+ | Model Architecture | BLOOMZ |
38
+ | Base Model Size | 1.1 Billion Parameters |
39
+ | Fine-tuning Method | LoRA with QLoRA (4-bit adapters) |
40
+ | Optimizer | `paged_adamw_8bit` |
41
+ | Precision | 4-bit LoRA + 8-bit Base |
42
+ | Epochs | 3–5 (variable per run) |
43
+ | Batch Size | 32 |
44
+ | Language Pair | English β†’ Burmese (မြန်မာ) |
45
+ | Tokenizer | Bloom tokenizer (`bigscience/tokenizer`) |
46
+
47
+ ---
48
+
49
+ ## πŸ“š Training Data
50
+
51
+ The model was fine-tuned on a curated mix of open datasets including:
52
+
53
+ - 🌍 **FLORES200** (en–my)
54
+ - 🎬 **OpenSubtitles** (Movie subtitles in Myanmar)
55
+ - πŸ“– **Custom Instruction-style translation datasets** (8 use cases, 200+ pairs per use case)
56
+ - πŸ—£οΈ **ai4bharat/indictrans2-en-my** (additional Burmese corpora)
57
+
58
+ ---
59
+
60
+ ## πŸ“ˆ Evaluation
61
+
62
+ | Metric | Score |
63
+ |------------------|---------|
64
+ | BLEU | 35–40 |
65
+ | Translation Style | Instructional, formal |
66
+ | Human Evaluation | βœ“ Understood grammar and tone in 85% samples |
67
+
68
+ > βœ… The model excels at translating English prompts into formal Burmese suitable for education, scripts, and user guides.
69
+
70
+ ---
71
+
72
+ ## πŸ”§ How to Use
73
+
74
+ ```python
75
+ from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
76
+ from peft import PeftModel
77
+
78
+ base = AutoModelForCausalLM.from_pretrained("bigscience/bloomz-1b1", load_in_8bit=True, device_map="auto")
79
+ lora = PeftModel.from_pretrained(base, "LinoM/bloomz-1b1MM")
80
+ tokenizer = AutoTokenizer.from_pretrained("bigscience/bloomz-1b1")
81
+
82
+ translator = pipeline("text-generation", model=lora, tokenizer=tokenizer)
83
+
84
+ text = "Translate into Burmese: What is your favorite subject?"
85
+ output = translator(text, max_new_tokens=100)
86
+ print(output[0]['generated_text'])