HatemMoushir commited on
Commit
28cc522
ยท
verified ยท
1 Parent(s): bfbfd5a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +48 -49
README.md CHANGED
@@ -34,13 +34,59 @@ The model distinguishes between **positive** and **negative** sentiments in real
34
 
35
  - **Base model**: `bert-base-multilingual-cased`
36
  - **Fine-tuned on**:
37
- - 1,000 Arabic tweets from [UCI Sentiment Dataset 2024](https://data.mendeley.com/datasets/m88gg52wp7/1)
38
- - 1,000 English tweets from [Sentiment140 (Stanford)](http://help.sentiment140.com/for-students)
39
  - **Task**: Binary sentiment classification (0 = Negative, 1 = Positive)
40
  - **Languages**: Arabic, English
41
  - **Tokenizer**: `bert-base-multilingual-cased` tokenizer
42
  - **Accuracy**: Evaluated on 10% holdout from training set
43
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
  ## ๐Ÿ“ฆ How to Use
45
 
46
  ```python
@@ -49,7 +95,6 @@ from transformers import pipeline
49
  classifier = pipeline("sentiment-analysis", model="HatemMoushir/ArEn-TweetSentiment-BERT-Hatem")
50
  print(classifier("ุงู„ุฎุฏู…ุฉ ูƒุงู†ุช ู…ู…ุชุงุฒุฉ"))
51
  print(classifier("I hate this product."))
52
-
53
  ```
54
  ## Testing
55
 
@@ -319,52 +364,6 @@ accuracy = correct / len(samples)
319
  print(f"โœ… Accuracy: {accuracy * 100:.2f}%")
320
  ```
321
 
322
- ๐Ÿ” Training Details
323
-
324
- Framework: ๐Ÿค— Transformers + PyTorch
325
-
326
- Training Time: ~2 epochs
327
-
328
- Optimizer: AdamW (default in Trainer)
329
-
330
- Batch Size: 16
331
-
332
- Evaluation Metric: Accuracy, F1, Precision, Recall
333
-
334
- Environment: Google Colab
335
-
336
-
337
- ---
338
- ## ๐Ÿ“Š Evaluation Results
339
-
340
- | Epoch | Training Loss | Validation Loss | Accuracy | F1 Score | Precision | Recall |
341
- |:-----:|:-------------:|:---------------:|:--------:|:--------:|:---------:|:------:|
342
- | 1 | 0.6266 | 0.7536 | 0.5900 | 0.1800 | 0.6429 | 0.1047 |
343
- | 2 | 0.5127 | 0.5944 | 0.7200 | 0.6667 | 0.6829 | 0.6512 |
344
- ---
345
-
346
- โœ… Summary:
347
-
348
- The model shows significant improvement after the second epoch.
349
-
350
- F1 score improved from 0.18 โ†’ 0.66, and accuracy from 59% โ†’ 72%.
351
-
352
- The training and validation losses are decreasing, indicating effective learning without overfitting (yet).
353
-
354
- Precision and Recall both increased, showing that the model is now detecting both classes more reliably.
355
-
356
-
357
-
358
- ---
359
-
360
- ๐Ÿงช How to Reproduce
361
-
362
- The model was fine-tuned using Trainer from the Hugging Face transformers library on a multilingual sentiment dataset (based on Sentiment140 and additional Arabic tweets).
363
-
364
- Training Time: ~1h30min on Colab GPU
365
- Model: bert-base-multilingual-cased
366
-
367
-
368
  ## Development and Assistance
369
 
370
  This model was developed and trained using **Google Colab**, with guidance and technical assistance from **ChatGPT**, which was used for idea generation, code authoring, and troubleshooting throughout the development process.
 
34
 
35
  - **Base model**: `bert-base-multilingual-cased`
36
  - **Fine-tuned on**:
37
+ - Arabic tweets from [UCI Sentiment Dataset 2024](https://data.mendeley.com/datasets/m88gg52wp7/1)
38
+ - English tweets from [Sentiment140 (Stanford)](http://help.sentiment140.com/for-students)
39
  - **Task**: Binary sentiment classification (0 = Negative, 1 = Positive)
40
  - **Languages**: Arabic, English
41
  - **Tokenizer**: `bert-base-multilingual-cased` tokenizer
42
  - **Accuracy**: Evaluated on 10% holdout from training set
43
 
44
+ ๐Ÿ” Training Details
45
+
46
+ Framework: ๐Ÿค— Transformers + PyTorch
47
+ Training Time: ~2 epochs
48
+ Optimizer: AdamW (default in Trainer)
49
+ Batch Size: 16
50
+ Evaluation Metric: Accuracy, F1, Precision, Recall
51
+ Environment: Google Colab
52
+
53
+ ---
54
+ ## ๐Ÿ“Š Evaluation Results
55
+
56
+ ### โœ… Experiment 1 โ€” Initial Run (2K Samples)
57
+
58
+ | Epoch | Train Loss | Val Loss | Accuracy | F1 Score | Precision | Recall |
59
+ |-------|------------|----------|----------|----------|-----------|--------|
60
+ | 1 | 0.6266 | 0.7536 | 59.00% | 0.1800 | 0.6429 | 0.1047 |
61
+ | 2 | 0.5127 | 0.5944 | 72.00% | 0.6667 | 0.6829 | 0.6512 |
62
+
63
+ ---
64
+ ### โœ… Experiment 2 โ€” Refined Arabic Dataset (20K Samples)
65
+
66
+ | Epoch | Train Loss | Val Loss | Accuracy | F1 Score | Precision | Recall |
67
+ |-------|------------|----------|----------|----------|-----------|--------|
68
+ | 1 | 0.5851 | 0.5879 | 70.85% | 0.6674 | 0.6139 | 0.7312 |
69
+ | 2 | 0.4792 | 0.5007 | 78.65% | 0.7105 | 0.7763 | 0.6550 |
70
+
71
+ ---
72
+ ### โœ… Experiment 3 โ€” Large-Scale Ar+En Dataset (100K Samples)
73
+
74
+ | Epoch | Train Loss | Val Loss | Accuracy | F1 Score | Precision | Recall |
75
+ |-------|------------|----------|----------|----------|-----------|--------|
76
+ | 1 | 0.5231 | 0.5846 | 72.35% | 0.7127 | 0.6171 | 0.8434 |
77
+ | 2 | 0.4404 | 0.4496 | 79.98% | 0.7502 | 0.7615 | 0.7394 |
78
+
79
+ ๐Ÿ” Summary:
80
+ Larger datasets led to higher recall and more robust generalization across languages. The model surpassed 79% accuracy and 0.75 F1 score in the final training run.
81
+
82
+ ---
83
+ ๐Ÿงช How to Reproduce
84
+
85
+ The model was fine-tuned using Trainer from the Hugging Face transformers library on a multilingual sentiment dataset (based on Sentiment140 and additional Arabic tweets).
86
+
87
+ Training Time: ~1h30min on Colab GPU
88
+ Model: bert-base-multilingual-cased
89
+
90
  ## ๐Ÿ“ฆ How to Use
91
 
92
  ```python
 
95
  classifier = pipeline("sentiment-analysis", model="HatemMoushir/ArEn-TweetSentiment-BERT-Hatem")
96
  print(classifier("ุงู„ุฎุฏู…ุฉ ูƒุงู†ุช ู…ู…ุชุงุฒุฉ"))
97
  print(classifier("I hate this product."))
 
98
  ```
99
  ## Testing
100
 
 
364
  print(f"โœ… Accuracy: {accuracy * 100:.2f}%")
365
  ```
366
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
367
  ## Development and Assistance
368
 
369
  This model was developed and trained using **Google Colab**, with guidance and technical assistance from **ChatGPT**, which was used for idea generation, code authoring, and troubleshooting throughout the development process.