Update README.md

README.md

@@ -33,7 +33,7 @@ license: apache-2.0
 ---
 # Model Card for Teuken 7B-base-v0.6
 
-Teuken 7B-base-v0.6 is a 7B parameter multilingual large language model (LLM) pre-trained with 6T tokens within the research project OpenGPT-X. 
+[Teuken 7B-base-v0.6](https://huggingface.co/openGPT-X/Teuken-7B-base-v0.6) is a 7B parameter multilingual large language model (LLM) pre-trained with 6T tokens within the research project [OpenGPT-X](https://opengpt-x.de). 
 
 
 ### Model Description
@@ -49,9 +49,14 @@ Teuken 7B-base-v0.6 is a 7B parameter multilingual large language model (LLM) pr
 ## Uses
 
 <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-Teuken 7B-base-v0.6 is intended for commercial and research use in all official 24 European languages. Since Teuken 7B-base-v0.6
+[Teuken 7B-base-v0.6](https://huggingface.co/openGPT-X/Teuken-7B-base-v0.6) is intended for research use in all official 24 European languages. Since Teuken 7B-base-v0.6
 focuses on covering all 24 EU languages, it renders more stable results across these languages and better reflects European values in its answers than English-centric models. It is therefore specialized for use in multilingual tasks.
 
+## Disclaimer Toxic Content:
+ 
+This Large Language Model (LLM) may generate content that is inappropriate, offensive, or harmful. While the dataset has been filtered to minimize such outputs, the model may still produce text that is biased or toxic due to the large scale and diverse nature of the data.
+
+
 ### Out-of-Scope Use
 
 <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
@@ -62,19 +67,24 @@ The model is not intended for use in math and coding tasks.
 
 <!-- This section is meant to convey both technical and sociotechnical limitations. -->
 
-Teuken 7B-base-v0.6 as a base model is not free from biases and hallucinations. It is therefore recommended to instruction tune it to fit it to the user's purposes and minimize biases and any risks arising. Finetuned models limiting risks and biases will appear soon after the release of the base model as a community effort. 
+[Teuken 7B-base-v0.6](https://huggingface.co/openGPT-X/Teuken-7B-base-v0.6) is a base model and is not free from biases and hallucinations.
 
 ## How to Get Started with the Model
 
 ## Usage
-The model requires transformers, sentencepiece, and the torch library.
+The model requires a few libraries that can be installed in your python environment:
+
+```bash
+python -m pip install numpy torch huggingface_hub transformers sentencepiece
+```
+
 After installation, here's an example of how to use the model:
 
 ```python
 import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
-model_name = "openGPT-X/Teuken 7B-base-v0.6"
+model_name = "openGPT-X/Teuken-7B-base-v0.6"
 prompt = "Insert text here..."
 device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
 tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
@@ -93,7 +103,7 @@ This example demonstrates how to load the model and tokenizer, prepare input, ge
 
 <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
-Teuken 7B-base-v0.6 was pre-trained on 5.5 trillion tokens of data from publicly available sources. 
+[Teuken 7B-base-v0.6](https://huggingface.co/openGPT-X/Teuken-7B-base-v0.6) was pre-trained on 6 trillion tokens of data from publicly available sources. 
 
 The pretraining data has a cutoff of September 2023.
 
@@ -111,6 +121,25 @@ Transformer-based decoder-only model that has been trained based on the causal l
 
 <!-- This section describes the evaluation protocols and provides the results. -->
 
+Results on multilingual benchmarks for 21 European languages with instruction-tuned models
+
+
+| Model | Avg | EU21-ARC | EU21-HeSw | EU21-TQA | EU21-MMLU |
+| --- | --- | --- | --- | --- | --- |
+| **Meta-Llama-3.1-8B** | **0.548** | 0.554 | 0.588 | **0.495** | **0.556** |
+| Salamandra-7B | 0.523 | **0.589** | **0.637** | 0.449 | 0.417 |
+| Mistral-7B-v0.3 | 0.505 | 0.513 | 0.534 | 0.472 | 0.501 |
+| Occiglot-7B-eu5 | 0.464 | 0.470 | 0.511 | 0.448 | 0.426 |
+| Pharia-1-LLM-7B-control | 0.409 | 0.393 | 0.433 | 0.456 | 0.353 |
+| Bloom-7B1 | 0.348 | 0.319 | 0.355 | 0.464 | 0.256 |
+| **Teuken-7B-Base (Ours)** | 0.520 | 0.558 | 0.619 | 0.449 | 0.453 |
+
+More information regarding the quality of our translated benchmarks are available in our Evaluation preprint ["Towards Multilingual LLM Evaluation for European Languages"](https://arxiv.org/abs/2410.08928).
+More evaluation results regarding [Teuken 7B-base-v0.6](https://huggingface.co/openGPT-X/Teuken-7B-base-v0.6) are available in our model preprint  ["Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs"](https://arxiv.org/abs/2410.03730).
+
+The model was evaluated in 21 languages on ARC, GSM8K, HellaSwag, TruthfulQA, Translation and MMLU. Results can also be seen in the [European LLM Leaderboard](https://huggingface.co/spaces/openGPT-X/european-llm-leaderboard).
+
+
 ### Testing Data, Factors & Metrics
 
 #### Testing Data
@@ -139,7 +168,7 @@ The model was evaluated in 21 languages on ARC, GSM8K, HellaSwag, TruthfulQA, Tr
 | Num Query Groups           | 2        |
 | Normalization              | RMSNorm  |
 | Learning rate              | 3e-4     |
-| Min learning rate          | 3e-5     |
+| Min learning rate          | 1.5e-5   |
 | Disable bias in linear     | yes      |
 | Hidden dropout             | 0.0      |
 | Attention dropout          | 0.0      |
@@ -173,38 +202,25 @@ The configuration of JUWELS Booster compute nodes is the following:
 
 #### Software
 
-https://github.com/OpenGPTX/Megatron-LM
-
-### Toxic Content
-
-This Language Model (LLM) may generate content that is inappropriate, offensive, or harmful. While the dataset has been heavily filtered to minimize such outputs,
-the model may still produce text that is biased or toxic due to the large scale and diverse nature of the data. 
+[Megatron-LM](https://github.com/OpenGPTX/Megatron-LM)
 
 ## Citation
 
-TODO
-
-
 **BibTeX:**
 
-TODO
-
-**APA:**
-
-TODO
-
-## Model Card Contact
+If you find our model useful in your research, please consider citing our [preprint](https://arxiv.org/abs/2410.03730):
+```
 
-<div class="hf-card">
-    <h2>Contact Information</h2>
-    <p>You can reach out to the following model card contact:</p>
-    <ul>
-        <li>
-            <a href="https://huggingface.co/openGPT-X" target="_blank">OpenGPT-X</a> 
-            - <a href="mailto:contact@opengpt-x.de">[email protected]</a>
-        </li>
-    </ul>
-</div>
+@misc{ali2024teuken7bbaseteuken7binstructeuropean,
+      title={Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs}, 
+      author={Mehdi Ali and Michael Fromm and Klaudia Thellmann and Jan Ebert and Alexander Arno Weber and Richard Rutmann and Charvi Jain and Max Lübbering and Daniel Steinigen and Johannes Leveling and Katrin Klug and Jasper Schulze Buschhoff and Lena Jurkschat and Hammam Abdelwahab and Benny Jörg Stein and Karl-Heinz Sylla and Pavel Denisov and Nicolo' Brandizzi and Qasid Saleem and Anirban Bhowmick and Lennard Helmer and Chelsea John and Pedro Ortiz Suarez and Malte Ostendorff and Alex Jude and Lalith Manjunath and Samuel Weinbach and Carolin Penke and Oleg Filatov and Shima Asaadi and Fabio Barth and Rafet Sifa and Fabian Küch and Andreas Herten and René Jäkel and Georg Rehm and Stefan Kesselheim and Joachim Köhler and Nicolas Flores-Herr},
+      year={2024},
+      eprint={2410.03730},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2410.03730}, 
+}
+```
 
 # Team
 ## Data Team
@@ -220,4 +236,21 @@ Klaudia Thellmann (TUD), Alex Jude (IAIS), Jasper Buschhoff (IAIS)
 ### Contributors:
 Shima Assadi (IIS), Fabio Barth (DFKI)
 ## Management
-Joachim Köhler (IAIS), Nicolas Flores-Herr (IAIS), Stefan Kesselheim (FZJ), Andreas Herten (FZJ), Georg Rehm (DFKI), René Jäkel (TUD), Fabian Küch (IIS), Nicole Hildebrandt (IAIS), Ines Wendler (IAIS)
+Joachim Köhler (IAIS), Nicolas Flores-Herr (IAIS), Stefan Kesselheim (FZJ), Andreas Herten (FZJ), Georg Rehm (DFKI), René Jäkel (TUD), Fabian Küch (IIS), Nicole Hildebrandt (IAIS), Ines Wendler (IAIS)
+
+We believe that collaboration is key to overcome the aforementioned limitations and thereby strengthening the European GenAI landscape. Because of this, the team invites researchers, developers, and AI enthusiasts to join and engage through various platforms. A Discord server has been created for community collaboration, offering a space for discussions on technical details, ideas, and direct interaction with developers. Additionally, resources like research publications and a European LLM Leaderboard provide insights into Teuken-7B’s performance and technical aspects. The OpenGPT-X team encourages ongoing engagement and collaboration as the project evolves.
+Key links:
+Discord: OpenGPT-X [Discord server](https://discord.com/invite/RvdHpGMvB3)
+Research Papers: OpenGPT-X News [Research Papers](https://opengpt-x.de/en/news-en/)
+LLM Leaderboard: European LLM Leaderboard [LLM Leaderboard](https://huggingface.co/spaces/openGPT-X/european-llm-leaderboard)
+
+<div class="hf-card">
+    <h2>Contact Information</h2>
+    <p>You can reach out to the following model card contact:</p>
+    <ul>
+        <li>
+            <a href="https://huggingface.co/openGPT-X" target="_blank">OpenGPT-X</a> 
+            - <a href="[email protected]">[email protected]</a>
+        </li>
+    </ul>
+</div>