Update README.md

README.md

@@ -1,3 +1,5 @@
+**English** | [한국어](https://huggingface.co/KRX-Data/WON-Reasoning/blob/main/KOREAN_README.md)
+
 # ₩ON: Open LLM for Korean Finance
 
 ## Introduction
@@ -6,6 +8,61 @@
 The core intent behind ₩ON is to promote research openness, benchmark rigorous financial reasoning capabilities, and foster best practices in training Korean-specific financial language models. 
 The model notably incorporates a two-step structured reasoning approach, providing self-correcting reasoning followed by a conclusive summary, aiming to elevate clarity and accuracy in financial decision-making processes.
 
+## KRX Financial LLM Competition
+
+**Competition Overview**
+
+The competition was the first open leaderboard dedicated to evaluating large language models specifically for Korean financial tasks.
+It was conducted over two months, including preliminary and final rounds, attracting 233 registered teams who collectively submitted over 1,100 models.
+The preliminary round included evaluations across five categories (Financial Markets, Finance and Accounting, Domestic Company Analysis, Financial Agent Tasks, and Stock Price Prediction), while the final round concentrated on Finance and Accounting, Financial Markets, and Open-Ended Finance QA.
+
+**Benchmark Description**
+
+The benchmark used during the competition consisted of approximately 5,500 carefully curated MCQA and Instruction-Response questions across various financial domains:
+
+- **Finance and Accounting**: Evaluated via university-level multiple-choice questions on accounting and financial principles.
+- **Financial Markets**: Based on examinations assessing understanding of financial regulations and Korean market systems.
+- **Stock Price Prediction**: Involved binary prediction tasks based on recent stock price data and computed indicators.
+- **Domestic Company Analysis**: Utilized KRX-Bench data generated from Korean company filings.
+- **Financial Agents**: Tasked models with executing financial data manipulations and coding tasks.
+- **Open-Ended FinQA**: Comprised of complex graduate-level econometric and legal reasoning tasks.
+
+**Benchmark Competition Statistics**
+
+The competition saw broad participation, with 52.5% corporate teams from sectors such as Tech and Finance, and significant academic involvement, reflecting diverse stakeholder interest in Korean financial NLP.
+
+<figure style="text-align: center;">
+  <img src="https://cdn-uploads.huggingface.co/production/uploads/63e087b6a98d931aa90c1b9c/XTxJM0nXAs18RiJEdwksU.png" width="700" height="900" alt="샘플 이미지" style="display: block; margin: auto;">
+  <figcaption style="font-style: italic; color: gray; margin-top: 8px;">
+    Distribution of participants. The shades of blue bards indicate corporate participants.
+  </figcaption>
+</figure>
+
+**Competition Results Analysis**
+
+During the preliminary rounds, top-performing models primarily utilized supervised fine-tuning (SFT), yielding notable gains particularly in the Domestic Company Analysis category. 
+Despite substantial improvements in this area, advancements in Financial & Accounting and Financial Markets were comparatively modest. 
+Most models adopted straightforward SFT approaches; however, some teams experimented with additional training methods, such as continual pre-training (CPT), although its impact at smaller scales remained inconclusive.
+
+<figure style="text-align: center;">
+  <img src="https://cdn-uploads.huggingface.co/production/uploads/63e087b6a98d931aa90c1b9c/ru3aA2ISwtqS3sJuSPVLN.png" width="600" height="750" alt="샘플 이미지" style="display: block; margin: auto;">
+  <figcaption style="font-style: italic; color: gray; margin-top: 8px;">
+    Preliminary round performance trends
+  </figcaption>
+</figure>
+
+In the final rounds, advanced multi-step training methodologies became prevalent. 
+Notably, teams implemented curriculum-based SFT strategies, beginning with simpler prompts and progressing towards more challenging instances generated using methods such as Evolve Instruct. 
+The best-performing models further refined their capabilities through preference optimization techniques such as Direct Preference Optimization (DPO) and KTO, utilizing responses evaluated by LLM-as-a-Judge methodologies. 
+Team Hi-Q specifically demonstrated the effectiveness of continual pre-training combined with SFT and DPO, achieving substantial performance improvements, thereby highlighting the value of structured and multi-stage training processes.
+
+<figure style="text-align: center;">
+  <img src="https://cdn-uploads.huggingface.co/production/uploads/63e087b6a98d931aa90c1b9c/VV6tYXtJkV5yTzNvMdzP9.png" width="700" height="900" alt="샘플 이미지" style="display: block; margin: auto;">
+  <figcaption style="font-style: italic; color: gray; margin-top: 8px;">
+    Evaluation results of continual pre-trained models
+  </figcaption>
+</figure>
+
 ## Model Training
 ### Dataset Collection
 We compiled a comprehensive training dataset of roughly 400,000 high-quality instructional samples through meticulous processes:
@@ -32,28 +89,45 @@ We designed a sophisticated two-phase training strategy for ₩ON:
 - **Solution Step**: After reasoning, the model succinctly summarizes its conclusions within `<solution>` and `</solution>` tags, providing clear and concise answers.
 
 ## Benchmark Results
-We have evaluated ₩ON on comprehensive benchmark employed in the competition. 
+
+We have evaluated ₩ON on the comprehensive benchmark employed in the competition. 
 This benchmark consisted of rigorously designed multiple-choice questions (MCQA) and open-ended questions to thoroughly assess the practical and theoretical capabilities of financial language models. 
-The benchmark categorized into Finance & Accounting (F&A), Financial Market analysis, and an Open-Ended Financial Question-Answering (FinQA) task:
+The benchmark is categorized into Finance & Accounting (F&A), Financial Market analysis, and an Open-Ended Financial Question-Answering (FinQA) task:
+
+- **Finance & Accounting**: The benchmark targets to evaluate the model's knowledge and analytical skills in financial concepts, accounting principles, and econometric reasoning.
+- **Financial Market Analysis**: It assesses the model's understanding of financial markets, systems, regulations, and domain-specific factual knowledge.
+- **Open-Ended FinQA**: It comprises complex and detailed reasoning questions to simulate realistic financial problem-solving scenarios.
 
-- **Finance & Accounting**: The benchmark targets fo evaluate the model's knowledge and analytical skills in financial concepts, accounting principles, and econometric reasoning.
-- **Financial Market Analysis**: It aims to assesses the model's understanding of financial markets, systems, regulations, and domain-specific factual knowledge.
-- **Open-Ended FinQA**: It comprises of complex and detailed reasoning questions to simulate realistic financial problem-solving scenarios.
+An example of this evaluation dataset is the following:
+
+<figure style="text-align: center;">
+  <img src="https://cdn-uploads.huggingface.co/production/uploads/63e087b6a98d931aa90c1b9c/7vLKeR6wTbr88UdOeikaE.png" width="700" height="900" alt="샘플 이미지" style="display: block; margin: auto;">
+  <figcaption style="font-style: italic; color: gray; margin-top: 8px;">
+    Overview of the benchmark used for evaluation. Each example demonstrates a specific question type for each category.
+  </figcaption>
+</figure>
 
 **Results**
 
-| Models                                    | F&A  | Market | Open-Ended | Average |
+₩ON emerged as the highest-performing model on average compared to the models awarded in the competition.
+The performance shows that the superior ability of ₩ON, particularly in the Finance & Accounting and Open-Ended FinQA subsets, reflecting its strong reasoning capabilities. 
+Despite less emphasis on purely domain-specific knowledge (Market), ₩ON's reasoning strength notably outperformed competing models.
+
+<!-- | Models                                    | F&A  | Market | Open-Ended | Average |
 |-------------------------------------------|------|--------|------------|---------|
 | overfit-brothers/hello_world06            | 0.65 | **0.83** | 0.01       | 0.50    |
 | AnonymousLLMer/krx-qwen2.5-v1206-1        | 0.63 | 0.65   | 0.04       | 0.44    |
 | shibainu24/qwen2.5-7B-inst-release-1516wk | 0.56 | 0.67   | 0.04       | 0.43    |
 | Q-PING/krx_1205_test_model_3              | 0.58 | 0.64   | 0.02       | 0.42    |
 | Hi-Q/krx_1206_test_model_2                | 0.60 | 0.61   | 0.02       | 0.41    |
-| **₩ON (Ours)**                            | **0.78** | 0.66   | **0.18**   | **0.54** |
-
-₩ON emerged as the highest-performing model on average compared to the models awarded in the competition.
-The performance shows that the superior ability of ₩ON, particularly in the Finance & Accounting and Open-Ended FinQA subsets, reflecting its strong reasoning capabilities. 
-Despite less emphasis on purely domain-specific knowledge (Market), ₩ON's reasoning strength notably outperformed competing models.
+| **₩ON (Ours)**                            | **0.78** | 0.66   | **0.18**   | **0.54** | -->
+
+<figure style="text-align: center;">
+  <img src="https://cdn-uploads.huggingface.co/production/uploads/63e087b6a98d931aa90c1b9c/--5Kkwfuf8c6hbNUvpJq6.png" width="700" height="900" alt="샘플 이미지" style="display: block; margin: auto;">
+  <!-- <figcaption style="font-style: italic; color: gray; margin-top: 8px;">
+    Overview of the benchmark used for evaluation. Each example demonstrates a specific question type for each category.
+  </figcaption> -->
+</figure>
 
 ## Quick Start