Update README.md
Browse files
README.md
CHANGED
@@ -73,22 +73,22 @@ Here's how you can structure the evaluation section:
|
|
73 |
|
74 |
| Metric | Base Model | Finetuned Model | Improvement |
|
75 |
|---------|------------|-----------------|-------------|
|
76 |
-
| ROUGE-1 |
|
77 |
-
| ROUGE-2 | 0.
|
78 |
-
| ROUGE-L |
|
79 |
|
80 |
### Model Prediction Comparison using Bigger model for evaluation
|
81 |
|
82 |
| Category | Count | Percentage |
|
83 |
|-------------------|-------|------------|
|
84 |
| Total samples | 5962 | 100% |
|
85 |
-
| Same predictions |
|
86 |
-
| Better predictions|
|
87 |
-
| Worse predictions |
|
88 |
|
89 |
### Evaluation Methods
|
90 |
- ROUGE scores for headline similarity
|
91 |
-
-
|
92 |
|
93 |
|
94 |
## Inference
|
|
|
73 |
|
74 |
| Metric | Base Model | Finetuned Model | Improvement |
|
75 |
|---------|------------|-----------------|-------------|
|
76 |
+
| ROUGE-1 | 3.39 | 4.64 | +1.26 |
|
77 |
+
| ROUGE-2 | 0.26 | 0.41 | +0.14 |
|
78 |
+
| ROUGE-L | 3.38 | 4.63 | +1.25 |
|
79 |
|
80 |
### Model Prediction Comparison using Bigger model for evaluation
|
81 |
|
82 |
| Category | Count | Percentage |
|
83 |
|-------------------|-------|------------|
|
84 |
| Total samples | 5962 | 100% |
|
85 |
+
| Same predictions | 3 | 0.05% |
|
86 |
+
| Better predictions| 4610 | 77.32% |
|
87 |
+
| Worse predictions | 1349 | 22.63% |
|
88 |
|
89 |
### Evaluation Methods
|
90 |
- ROUGE scores for headline similarity
|
91 |
+
- Bigger model's custom metrics for headline appropriateness and relativeness
|
92 |
|
93 |
|
94 |
## Inference
|