Update README.md
Browse files
README.md
CHANGED
@@ -34,13 +34,59 @@ The model distinguishes between **positive** and **negative** sentiments in real
|
|
34 |
|
35 |
- **Base model**: `bert-base-multilingual-cased`
|
36 |
- **Fine-tuned on**:
|
37 |
-
-
|
38 |
-
-
|
39 |
- **Task**: Binary sentiment classification (0 = Negative, 1 = Positive)
|
40 |
- **Languages**: Arabic, English
|
41 |
- **Tokenizer**: `bert-base-multilingual-cased` tokenizer
|
42 |
- **Accuracy**: Evaluated on 10% holdout from training set
|
43 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
44 |
## ๐ฆ How to Use
|
45 |
|
46 |
```python
|
@@ -49,7 +95,6 @@ from transformers import pipeline
|
|
49 |
classifier = pipeline("sentiment-analysis", model="HatemMoushir/ArEn-TweetSentiment-BERT-Hatem")
|
50 |
print(classifier("ุงูุฎุฏู
ุฉ ูุงูุช ู
ู
ุชุงุฒุฉ"))
|
51 |
print(classifier("I hate this product."))
|
52 |
-
|
53 |
```
|
54 |
## Testing
|
55 |
|
@@ -319,52 +364,6 @@ accuracy = correct / len(samples)
|
|
319 |
print(f"โ
Accuracy: {accuracy * 100:.2f}%")
|
320 |
```
|
321 |
|
322 |
-
๐ Training Details
|
323 |
-
|
324 |
-
Framework: ๐ค Transformers + PyTorch
|
325 |
-
|
326 |
-
Training Time: ~2 epochs
|
327 |
-
|
328 |
-
Optimizer: AdamW (default in Trainer)
|
329 |
-
|
330 |
-
Batch Size: 16
|
331 |
-
|
332 |
-
Evaluation Metric: Accuracy, F1, Precision, Recall
|
333 |
-
|
334 |
-
Environment: Google Colab
|
335 |
-
|
336 |
-
|
337 |
-
---
|
338 |
-
## ๐ Evaluation Results
|
339 |
-
|
340 |
-
| Epoch | Training Loss | Validation Loss | Accuracy | F1 Score | Precision | Recall |
|
341 |
-
|:-----:|:-------------:|:---------------:|:--------:|:--------:|:---------:|:------:|
|
342 |
-
| 1 | 0.6266 | 0.7536 | 0.5900 | 0.1800 | 0.6429 | 0.1047 |
|
343 |
-
| 2 | 0.5127 | 0.5944 | 0.7200 | 0.6667 | 0.6829 | 0.6512 |
|
344 |
-
---
|
345 |
-
|
346 |
-
โ
Summary:
|
347 |
-
|
348 |
-
The model shows significant improvement after the second epoch.
|
349 |
-
|
350 |
-
F1 score improved from 0.18 โ 0.66, and accuracy from 59% โ 72%.
|
351 |
-
|
352 |
-
The training and validation losses are decreasing, indicating effective learning without overfitting (yet).
|
353 |
-
|
354 |
-
Precision and Recall both increased, showing that the model is now detecting both classes more reliably.
|
355 |
-
|
356 |
-
|
357 |
-
|
358 |
-
---
|
359 |
-
|
360 |
-
๐งช How to Reproduce
|
361 |
-
|
362 |
-
The model was fine-tuned using Trainer from the Hugging Face transformers library on a multilingual sentiment dataset (based on Sentiment140 and additional Arabic tweets).
|
363 |
-
|
364 |
-
Training Time: ~1h30min on Colab GPU
|
365 |
-
Model: bert-base-multilingual-cased
|
366 |
-
|
367 |
-
|
368 |
## Development and Assistance
|
369 |
|
370 |
This model was developed and trained using **Google Colab**, with guidance and technical assistance from **ChatGPT**, which was used for idea generation, code authoring, and troubleshooting throughout the development process.
|
|
|
34 |
|
35 |
- **Base model**: `bert-base-multilingual-cased`
|
36 |
- **Fine-tuned on**:
|
37 |
+
- Arabic tweets from [UCI Sentiment Dataset 2024](https://data.mendeley.com/datasets/m88gg52wp7/1)
|
38 |
+
- English tweets from [Sentiment140 (Stanford)](http://help.sentiment140.com/for-students)
|
39 |
- **Task**: Binary sentiment classification (0 = Negative, 1 = Positive)
|
40 |
- **Languages**: Arabic, English
|
41 |
- **Tokenizer**: `bert-base-multilingual-cased` tokenizer
|
42 |
- **Accuracy**: Evaluated on 10% holdout from training set
|
43 |
|
44 |
+
๐ Training Details
|
45 |
+
|
46 |
+
Framework: ๐ค Transformers + PyTorch
|
47 |
+
Training Time: ~2 epochs
|
48 |
+
Optimizer: AdamW (default in Trainer)
|
49 |
+
Batch Size: 16
|
50 |
+
Evaluation Metric: Accuracy, F1, Precision, Recall
|
51 |
+
Environment: Google Colab
|
52 |
+
|
53 |
+
---
|
54 |
+
## ๐ Evaluation Results
|
55 |
+
|
56 |
+
### โ
Experiment 1 โ Initial Run (2K Samples)
|
57 |
+
|
58 |
+
| Epoch | Train Loss | Val Loss | Accuracy | F1 Score | Precision | Recall |
|
59 |
+
|-------|------------|----------|----------|----------|-----------|--------|
|
60 |
+
| 1 | 0.6266 | 0.7536 | 59.00% | 0.1800 | 0.6429 | 0.1047 |
|
61 |
+
| 2 | 0.5127 | 0.5944 | 72.00% | 0.6667 | 0.6829 | 0.6512 |
|
62 |
+
|
63 |
+
---
|
64 |
+
### โ
Experiment 2 โ Refined Arabic Dataset (20K Samples)
|
65 |
+
|
66 |
+
| Epoch | Train Loss | Val Loss | Accuracy | F1 Score | Precision | Recall |
|
67 |
+
|-------|------------|----------|----------|----------|-----------|--------|
|
68 |
+
| 1 | 0.5851 | 0.5879 | 70.85% | 0.6674 | 0.6139 | 0.7312 |
|
69 |
+
| 2 | 0.4792 | 0.5007 | 78.65% | 0.7105 | 0.7763 | 0.6550 |
|
70 |
+
|
71 |
+
---
|
72 |
+
### โ
Experiment 3 โ Large-Scale Ar+En Dataset (100K Samples)
|
73 |
+
|
74 |
+
| Epoch | Train Loss | Val Loss | Accuracy | F1 Score | Precision | Recall |
|
75 |
+
|-------|------------|----------|----------|----------|-----------|--------|
|
76 |
+
| 1 | 0.5231 | 0.5846 | 72.35% | 0.7127 | 0.6171 | 0.8434 |
|
77 |
+
| 2 | 0.4404 | 0.4496 | 79.98% | 0.7502 | 0.7615 | 0.7394 |
|
78 |
+
|
79 |
+
๐ Summary:
|
80 |
+
Larger datasets led to higher recall and more robust generalization across languages. The model surpassed 79% accuracy and 0.75 F1 score in the final training run.
|
81 |
+
|
82 |
+
---
|
83 |
+
๐งช How to Reproduce
|
84 |
+
|
85 |
+
The model was fine-tuned using Trainer from the Hugging Face transformers library on a multilingual sentiment dataset (based on Sentiment140 and additional Arabic tweets).
|
86 |
+
|
87 |
+
Training Time: ~1h30min on Colab GPU
|
88 |
+
Model: bert-base-multilingual-cased
|
89 |
+
|
90 |
## ๐ฆ How to Use
|
91 |
|
92 |
```python
|
|
|
95 |
classifier = pipeline("sentiment-analysis", model="HatemMoushir/ArEn-TweetSentiment-BERT-Hatem")
|
96 |
print(classifier("ุงูุฎุฏู
ุฉ ูุงูุช ู
ู
ุชุงุฒุฉ"))
|
97 |
print(classifier("I hate this product."))
|
|
|
98 |
```
|
99 |
## Testing
|
100 |
|
|
|
364 |
print(f"โ
Accuracy: {accuracy * 100:.2f}%")
|
365 |
```
|
366 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
367 |
## Development and Assistance
|
368 |
|
369 |
This model was developed and trained using **Google Colab**, with guidance and technical assistance from **ChatGPT**, which was used for idea generation, code authoring, and troubleshooting throughout the development process.
|