Update README.md

README.md

@@ -23,7 +23,7 @@ tags:
 - **Task:** Binary Classification (Fact Verification)  
 - **Dataset:** [ViWikiFC](https://arxiv.org/abs/2405.07615)  
 
-SemViQA-BC is one of the key components of the two-step classification approach in the SemViQA system. It focuses on binary classification, determining whether a claim is SUPPORTED or REFUTED. This step follows an initial three-class classification, where claims are first categorized as SUPPORTED, REFUTED, or NOT ENOUGH INFORMATION (NEI). By incorporating Cross-Entropy Loss and Focal Loss, SemViQA-BC enhances precision in claim verification, ensuring more accurate fact-checking results
+SemViQA-BC is one of the key components of the two-step classification (TVC) approach in the SemViQA system. It focuses on binary classification, determining whether a claim is SUPPORTED or REFUTED. This step follows an initial three-class classification, where claims are first categorized as SUPPORTED, REFUTED, or NOT ENOUGH INFORMATION (NEI). By incorporating Cross-Entropy Loss and Focal Loss, SemViQA-BC enhances precision in claim verification, ensuring more accurate fact-checking results
 
 ## Usage Example
 
@@ -69,6 +69,281 @@ for i, (label, prob) in enumerate(zip(labels, probabilities.tolist()), start=1):
 # 2) REFUTED 0.9999
 ```
 
+## **Evaluation Results**
+
+SemViQA-BC achieved impressive results on the test set, demonstrating accurate and efficient classification capabilities. The detailed evaluation of SemViQA-BC is presented in the table below.
+
+<table> 
+  <thead>
+    <tr>
+      <th colspan="2">Method</th>
+      <th colspan="4">ViWikiFC</th>
+    </tr>
+    <tr>
+      <th>ER</th>
+      <th>VC</th>
+      <th>Strict Acc</th>
+      <th>VC Acc</th>
+      <th>ER Acc</th>
+      <th>Time (s)</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td rowspan="3">TF-IDF</td>
+      <td>InfoXLM<sub>large</sub></td>
+      <td>75.56</td>
+      <td>82.21</td>
+      <td>90.15</td>
+      <td>131</td>
+    </tr>
+    <tr>
+      <td>XLM-R<sub>large</sub></td>
+      <td>76.47</td>
+      <td>82.78</td>
+      <td>90.15</td>
+      <td>134</td>
+    </tr>
+    <tr>
+      <td>Ernie-M<sub>large</sub></td>
+      <td>75.56</td>
+      <td>81.83</td>
+      <td>90.15</td>
+      <td>144</td>
+    </tr>
+    <tr>
+      <td rowspan="3">BM25</td>
+      <td>InfoXLM<sub>large</sub></td>
+      <td>70.44</td>
+      <td>79.01</td>
+      <td>83.50</td>
+      <td>130</td>
+    </tr>
+    <tr>
+      <td>XLM-R<sub>large</sub></td>
+      <td>70.97</td>
+      <td>78.91</td>
+      <td>83.50</td>
+      <td>132</td>
+    </tr>
+    <tr>
+      <td>Ernie-M<sub>large</sub></td>
+      <td>70.21</td>
+      <td>78.29</td>
+      <td>83.50</td>
+      <td>141</td>
+    </tr>
+    <tr>
+      <td rowspan="3">SBert</td>
+      <td>InfoXLM<sub>large</sub></td>
+      <td>74.99</td>
+      <td>81.59</td>
+      <td>89.72</td>
+      <td>195</td>
+    </tr>
+    <tr>
+      <td>XLM-R<sub>large</sub></td>
+      <td>75.80</td>
+      <td>82.35</td>
+      <td>89.72</td>
+      <td>194</td>
+    </tr>
+    <tr>
+      <td>Ernie-M<sub>large</sub></td>
+      <td>75.13</td>
+      <td>81.44</td>
+      <td>89.72</td>
+      <td>203</td>
+    </tr>
+    <tr>
+      <th colspan="1">QA-based approaches</th>
+      <th colspan="1">VC</th>
+      <th colspan="4"></th>
+    </tr>
+    <tr>
+      <td rowspan="3">ViMRC<sub>large</sub></td>
+      <td>InfoXLM<sub>large</sub></td>
+      <td>77.28</td>
+      <td>81.97</td>
+      <td>92.49</td>
+      <td>3778</td>
+    </tr>
+    <tr>
+      <td>XLM-R<sub>large</sub></td>
+      <td>78.29</td>
+      <td>82.83</td>
+      <td>92.49</td>
+      <td>3824</td>
+    </tr>
+    <tr>
+      <td>Ernie-M<sub>large</sub></td>
+      <td>77.38</td>
+      <td>81.92</td>
+      <td>92.49</td>
+      <td>3785</td>
+    </tr>
+    <tr>
+      <td rowspan="3">InfoXLM<sub>large</sub></td>
+      <td>InfoXLM<sub>large</sub></td>
+      <td>78.14</td>
+      <td>82.07</td>
+      <td>93.45</td>
+      <td>4092</td>
+    </tr>
+    <tr>
+      <td>XLM-R<sub>large</sub></td>
+      <td>79.20</td>
+      <td>83.07</td>
+      <td>93.45</td>
+      <td>4096</td>
+    </tr>
+    <tr>
+      <td>Ernie-M<sub>large</sub></td>
+      <td>78.24</td>
+      <td>82.21</td>
+      <td>93.45</td>
+      <td>4102</td>
+    </tr>
+    <tr>
+      <th colspan="2">LLM</th> 
+      <th colspan="4"></th>
+    </tr>
+    <tr>
+      <td colspan="2">Qwen2.5-1.5B-Instruct</td>
+      <td>51.03</td>
+      <td>65.18</td>
+      <td>78.96</td>
+      <td>7665</td>
+    </tr>
+    <tr>
+      <td colspan="2">Qwen2.5-3B-Instruct</td>
+      <td>44.38</td>
+      <td>62.31</td>
+      <td>71.35</td>
+      <td>12123</td>
+    </tr>
+    <tr>
+      <th colspan="1">LLM</th>
+      <th colspan="1">VC</th>
+      <th colspan="4"></th>
+    </tr>
+    <tr>
+      <td rowspan="3">Qwen2.5-1.5B-Instruct</td>
+      <td>InfoXLM<sub>large</sub></td>
+      <td>66.14</td>
+      <td>76.47</td>
+      <td>78.96</td>
+      <td>7788</td>
+    </tr>
+    <tr>
+      <td>XLM-R<sub>large</sub></td>
+      <td>67.67</td>
+      <td>78.10</td>
+      <td>78.96</td>
+      <td>7789</td>
+    </tr>
+    <tr>
+      <td>Ernie-M<sub>large</sub></td>
+      <td>66.52</td>
+      <td>76.52</td>
+      <td>78.96</td>
+      <td>7794</td>
+    </tr>
+    <tr>
+      <td rowspan="3">Qwen2.5-3B-Instruct</td>
+      <td>InfoXLM<sub>large</sub></td>
+      <td>59.88</td>
+      <td>72.50</td>
+      <td>71.35</td>
+      <td>12246</td>
+    </tr>
+    <tr>
+      <td>XLM-R<sub>large</sub></td>
+      <td>60.74</td>
+      <td>73.08</td>
+      <td>71.35</td>
+      <td>12246</td>
+    </tr>
+    <tr>
+      <td>Ernie-M<sub>large</sub></td>
+      <td>60.02</td>
+      <td>72.21</td>
+      <td>71.35</td>
+      <td>12251</td>
+    </tr>
+    <tr>
+      <th colspan="1">SER Faster (ours)</th>
+      <th colspan="1">TVC (ours)</th>
+      <th colspan="4"></th>
+    </tr>
+    <tr>
+      <td>TF-IDF + ViMRC<sub>large</sub></td>
+      <td>Ernie-M<sub>large</sub></td>
+      <td style="color:blue">79.44</td>
+      <td style="color:blue">82.93</td>
+      <td style="color:blue">94.60</td>
+      <td style="color:blue">410</td>
+    </tr>
+    <tr>
+      <td>TF-IDF + InfoXLM<sub>large</sub></td>
+      <td>Ernie-M<sub>large</sub></td>
+      <td style="color:blue">79.77</td>
+      <td style="color:blue">83.07</td>
+      <td style="color:blue">95.03</td>
+      <td style="color:blue">487</td>
+    </tr>
+    <tr>
+      <th colspan="1">SER (ours)</th>
+      <th colspan="1">TVC (ours)</th>
+      <th colspan="4"></th>
+    </tr>
+    <tr>
+      <td rowspan="3">TF-IDF + ViMRC<sub>large</sub></td>
+      <td>InfoXLM<sub>large</sub></td>
+      <td>80.25</td>
+      <td>83.84</td>
+      <td>94.69</td>
+      <td>2731</td>
+    </tr>
+    <tr>
+      <td>XLM-R<sub>large</sub></td>
+      <td>80.34</td>
+      <td>83.64</td>
+      <td>94.69</td>
+      <td>2733</td>
+    </tr>
+    <tr>
+      <td>Ernie-M<sub>large</sub></td>
+      <td>79.53</td>
+      <td>82.97</td>
+      <td>94.69</td>
+      <td>2733</td>
+    </tr>
+    <tr>
+      <td rowspan="3">TF-IDF + InfoXLM<sub>large</sub></td>
+      <td>InfoXLM<sub>large</sub></td>
+      <td>80.68</td>
+      <td><strong>83.98</strong></td>
+      <td><strong>95.31</strong></td>
+      <td>3860</td>
+    </tr>
+    <tr>
+      <td>XLM-R<sub>large</sub></td>
+      <td><strong>80.82</strong></td>
+      <td>83.88</td>
+      <td><strong>95.31</strong></td>
+      <td>3843</td>
+    </tr>
+    <tr>
+      <td>Ernie-M<sub>large</sub></td>
+      <td>80.06</td>
+      <td>83.17</td>
+      <td><strong>95.31</strong></td>
+      <td>3891</td>
+    </tr>
+  </tbody>
+</table>
+
 ## **Citation**
 
 If you use **SemViQA-BC** in your research, please cite: