|
|
--- |
|
|
license: apache-2.0 |
|
|
--- |
|
|
This is a reproduction of SumCSE. I don't know the author of the original paper. |
|
|
|
|
|
[SumCSE](https://github.com/raghavlite/SumCSE/tree/main) |
|
|
|
|
|
5401.reimple-robert-large |
|
|
|
|
|
``` |
|
|
epoch = 3.0 |
|
|
eval_CR = 92.18 |
|
|
eval_MPQA = 90.08 |
|
|
eval_MR = 87.44 |
|
|
eval_MRPC = 77.01 |
|
|
eval_SST2 = 92.43 |
|
|
eval_SUBJ = 93.46 |
|
|
eval_TREC = 81.49 |
|
|
eval_avg_sts = 0.8624794115602034 |
|
|
eval_avg_transfer = 87.72714285714285 |
|
|
eval_sickr_spearman = 0.8479586922026686 |
|
|
eval_stsb_spearman = 0.8770001309177381 |
|
|
``` |
|
|
|
|
|
|
|
|
``` |
|
|
------ test ------ |
|
|
+-------+-------+-------+-------+-------+--------------+-----------------+-------+ |
|
|
| STS12 | STS13 | STS14 | STS15 | STS16 | STSBenchmark | SICKRelatedness | Avg. | |
|
|
+-------+-------+-------+-------+-------+--------------+-----------------+-------+ |
|
|
| 79.55 | 87.14 | 81.30 | 87.50 | 85.13 | 85.21 | 82.10 | 83.99 | |
|
|
+-------+-------+-------+-------+-------+--------------+-----------------+-------+ |
|
|
+------+------+------+------+------+------+------+------+ |
|
|
| MR | CR | SUBJ | MPQA | SST2 | TREC | MRPC | Avg. | |
|
|
+------+------+------+------+------+------+------+------+ |
|
|
| 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | |
|
|
+------+------+------+------+------+------+------+------+ |
|
|
``` |
|
|
|
