saidines12 commited on
Commit
8dfd9c8
·
verified ·
1 Parent(s): ff8ead0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -7
README.md CHANGED
@@ -73,22 +73,22 @@ Here's how you can structure the evaluation section:
73
 
74
  | Metric | Base Model | Finetuned Model | Improvement |
75
  |---------|------------|-----------------|-------------|
76
- | ROUGE-1 | 2.85 | 4.67 | +1.82 |
77
- | ROUGE-2 | 0.25 | 0.41 | +0.17 |
78
- | ROUGE-L | 2.84 | 4.65 | +1.81 |
79
 
80
  ### Model Prediction Comparison using Bigger model for evaluation
81
 
82
  | Category | Count | Percentage |
83
  |-------------------|-------|------------|
84
  | Total samples | 5962 | 100% |
85
- | Same predictions | 1 | 0.02% |
86
- | Better predictions| 4697 | 78.78% |
87
- | Worse predictions | 1264 | 21.20% |
88
 
89
  ### Evaluation Methods
90
  - ROUGE scores for headline similarity
91
- - Human evaluation for headline appropriateness
92
 
93
 
94
  ## Inference
 
73
 
74
  | Metric | Base Model | Finetuned Model | Improvement |
75
  |---------|------------|-----------------|-------------|
76
+ | ROUGE-1 | 3.39 | 4.64 | +1.26 |
77
+ | ROUGE-2 | 0.26 | 0.41 | +0.14 |
78
+ | ROUGE-L | 3.38 | 4.63 | +1.25 |
79
 
80
  ### Model Prediction Comparison using Bigger model for evaluation
81
 
82
  | Category | Count | Percentage |
83
  |-------------------|-------|------------|
84
  | Total samples | 5962 | 100% |
85
+ | Same predictions | 3 | 0.05% |
86
+ | Better predictions| 4610 | 77.32% |
87
+ | Worse predictions | 1349 | 22.63% |
88
 
89
  ### Evaluation Methods
90
  - ROUGE scores for headline similarity
91
+ - Bigger model's custom metrics for headline appropriateness and relativeness
92
 
93
 
94
  ## Inference