Improve model card: Add paper link, license, pipeline tag, and languages

README.md

@@ -1,9 +1,51 @@
 ---
 library_name: transformers
-tags:
-- translation
+pipeline_tag: translation
+license: apache-2.0
+languages:
+  - zh
+  - en
+  - fr
+  - pt
+  - es
+  - ja
+  - tr
+  - ru
+  - ar
+  - ko
+  - th
+  - it
+  - de
+  - vi
+  - ms
+  - id
+  - tl
+  - hi
+  - zh-Hant
+  - pl
+  - cs
+  - nl
+  - km
+  - my
+  - fa
+  - gu
+  - ur
+  - te
+  - mr
+  - he
+  - bn
+  - ta
+  - uk
+  - bo
+  - kk
+  - mn
+  - ug
+  - yue
 ---
 
+# Hunyuan-MT-Chimera-7B-fp8: Multilingual Translation Model
+
+This repository contains the `Hunyuan-MT-Chimera-7B-fp8` model, as presented in the paper [Hunyuan-MT Technical Report](https://huggingface.co/papers/2509.05209).
 
 <p align="center">
  <img src="https://dscache.tencent-cloud.cn/upload/uploader/hunyuan-64b418fd052c033b228e04bc77bbc4b54fd7f5bc.png" width="400"/> <br>
@@ -42,6 +84,17 @@ Hunyuan-MT-Chimera-7B-fp8 was produced by [AngelSlim](https://github.com/Tencent
 <br>
 
 
+&nbsp;
+
+## Performance
+
+<div align='center'>
+<img src="https://github.com/Tencent-Hunyuan/Hunyuan-MT/raw/main/imgs/overall_performance.png" width = "80%" />
+</div>
+You can refer to our technical report for more experimental results and analysis.
+
+<a href="https://www.arxiv.org/pdf/2509.05209"><b>Technical Report</b> </a>
+
 &nbsp;
 
 ## 模型链接
@@ -101,10 +154,10 @@ First, please install transformers, recommends v4.56.0
 pip install transformers==4.56.0
 ```
 
-The following code snippet shows how to use the transformers library to load and apply the model.
-
 *!!! If you want to load fp8 model with transformers, you need to change the name"ignored_layers" in config.json to "ignore" and upgrade the compressed-tensors to compressed-tensors-0.11.0.*
 
+The following code snippet shows how to use the transformers library to load and apply the model.
+
 we use tencent/Hunyuan-MT-7B for example
 
 ```python
@@ -116,7 +169,9 @@ model_name_or_path = "tencent/Hunyuan-MT-7B"
 tokenizer = AutoTokenizer.from_pretrained(model_name_or_path)
 model = AutoModelForCausalLM.from_pretrained(model_name_or_path, device_map="auto")  # You may want to use bfloat16 and/or move to GPU here
 messages = [
-    {"role": "user", "content": "Translate the following segment into Chinese, without additional explanation.\n\nIt’s on the house."},
+    {"role": "user", "content": "Translate the following segment into Chinese, without additional explanation.
+
+It’s on the house."},
 ]
 tokenized_chat = tokenizer.apply_chat_template(
     messages,
@@ -182,6 +237,331 @@ Supported languages:
 | Uyghur            | ug      | 维吾尔语        |
 | Cantonese         | yue     | 粤语            |
 
+### Training Data Format
+
+If you need to fine-tune our Instruct model, we recommend processing the data into the following format.
+
+```python
+
+messages = [
+    {"role": "system", "content": "You are a helpful assistant."},
+    {"role": "user", "content": "Why is seawater salty?" },
+    {"role": "assistant", "content": "Seawater is primarily saline due to dissolved salts and minerals. These substances come from the chemical materials in rocks and soil on the Earth's surface, which are carried into the ocean over time. When seawater evaporates, the water vapor leaves, but the salts and minerals remain, making the seawater saltier. Therefore, the salinity of seawater is determined by the amount of salts and minerals it contains."}
+]
+
+from transformers import AutoTokenizer
+tokenizer = AutoTokenizer.from_pretrained("your_tokenizer_path", trust_remote_code=True)
+train_ids = tokenizer.apply_chat_template(messages)
+```
+
+&nbsp;
+
+### Train with LLaMA-Factory
+
+In the following chapter, we will introduce how to use `LLaMA-Factory` to fine-tune the `Hunyuan` model.
+
+#### Prerequisites
+
+Verify installation of the following dependencies:
+- **LLaMA-Factory**: Follow [official installation guide](https://github.com/hiyouga/LLaMA-Factory)
+- **DeepSpeed** (optional): Follow [official installation guide](https://github.com/deepspeedai/DeepSpeed#installation)
+- **Transformer Library**: Use the companion branch (Hunyuan-submitted code is pending review)
+    ```
+    pip install git+https://github.com/huggingface/transformers@4970b23cedaf745f963779b4eae68da281e8c6ca
+    ```
+
+#### Data preparation
+
+We need to prepare a custom dataset:
+1. Organize your data in `json` format and place it in the `data` directory in `LLaMA-Factory`. The current implementation uses the `sharegpt` dataset format, which requires the following structure:
+```
+[
+  {
+    "messages": [
+      {
+        "role": "system",
+        "content": "System prompt (optional)"
+      },
+      {
+        "role": "user",
+        "content": "Human instruction"
+      },
+      {
+        "role": "assistant",
+        "content": "Model response"
+      }
+    ]
+  }
+]
+```
+Refer to the [Data Format](#training-data-format) section mentioned earlier for details.
+
+2. Define your dataset in the data/dataset_info.json file using the following format:
+```
+"dataset_name": {
+  "file_name": "dataset.json",
+  "formatting": "sharegpt",
+  "columns": {
+    "messages": "messages"
+  },
+  "tags": {
+    "role_tag": "role",
+    "content_tag": "content",
+    "user_tag": "user",
+    "assistant_tag": "assistant",
+    "system_tag": "system"
+  }
+}
+```
+
+#### Training execution
+
+1. Copy all files from the `llama_factory_support/example_configs` directory to the `example/hunyuan` directory in `LLaMA-Factory`.
+2. Modify the model path and dataset name in the configuration file `hunyuan_full.yaml`. Adjust other configurations as needed:
+```
+### model
+model_name_or_path: [!!!add the model path here!!!]
+
+### dataset
+dataset: [!!!add the dataset name here!!!]
+```
+3. Execute training commands:
+    *​​Single-node training​​
+    Note: Set the environment variable DISABLE_VERSION_CHECK to 1 to avoid version conflicts.
+    ```
+    export DISABLE_VERSION_CHECK=1
+    llamafactory-cli train examples/hunyuan/hunyuan_full.yaml
+    ```
+    *Multi-node training​​
+    Execute the following command on each node. Configure NNODES, NODE_RANK, MASTER_ADDR, and MASTER_PORT according to your environment:
+    ```
+    export DISABLE_VERSION_CHECK=1
+    FORCE_TORCHRUN=1 NNODES=${NNODES} NODE_RANK=${NODE_RANK} MASTER_ADDR=${MASTER_ADDR} MASTER_PORT=${MASTER_PORT} \
+    llamafactory-cli train examples/hunyuan/hunyuan_full.yaml
+    ```
+
+&nbsp;
+
+
+## Quantization Compression
+We used our own [AngelSlim](https://github.com/tencent/AngelSlim) compression tool to produce FP8 and INT4 quantization models. `AngelSlim` is a toolset dedicated to creating a more user-friendly, comprehensive and efficient model compression solution.
+
+### FP8 Quantization
+We use FP8-static quantization, FP8 quantization adopts 8-bit floating point format, through a small amount of calibration data (without training) to pre-determine the quantization scale, the model weights and activation values will be converted to FP8 format, to improve the inference efficiency and reduce the deployment threshold. We you can use AngelSlim quantization, you can also directly download our quantization completed open source model to use [AngelSlim](https://huggingface.co/AngelSlim).
+
+
+## Deployment
+
+For deployment, you can use frameworks such as **TensorRT-LLM**, **vLLM**, or **SGLang** to serve the model and create an OpenAI-compatible API endpoint.
+
+image: https://hub.docker.com/r/hunyuaninfer/hunyuan-7B/tags
+
+
+### TensorRT-LLM
+
+#### Docker Image
+
+We provide a pre-built Docker image based on the latest version of TensorRT-LLM.
+
+We use tencent/Hunyuan-7B-Instruct for example
+- To get started:
+
+```
+docker pull docker.cnb.cool/tencent/hunyuan/hunyuan-7b:hunyuan-7b-trtllm
+```
+```
+docker run --privileged --user root --name hunyuanLLM_infer --rm -it --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 --gpus=all hunyuaninfer/hunyuan-7b:hunyuan-7b-trtllm
+```
+
+- Prepare Configuration file:
+
+```
+cat >/path/to/extra-llm-api-config.yml <<EOF
+use_cuda_graph: true
+cuda_graph_padding_enabled: true
+cuda_graph_batch_sizes:
+- 1
+- 2
+- 4
+- 8
+- 16
+- 32
+print_iter_log: true
+EOF
+```
+
+
+- Start the API server:
+
+
+```
+trtllm-serve \
+  /path/to/HunYuan-7b \
+  --host localhost \
+  --port 8000 \
+  --backend pytorch \
+  --max_batch_size 32 \
+  --max_num_tokens 16384 \
+  --tp_size 2 \
+  --kv_cache_free_gpu_memory_fraction 0.6 \
+  --trust_remote_code \
+  --extra_llm_api_options /path/to/extra-llm-api-config.yml
+```
+
+
+### vllm
+
+#### Start
+Please use vLLM version v0.10.0 or higher for inference.
+
+First, please install transformers. We will merge it into the main branch later.
+```SHELL
+pip install git+https://github.com/huggingface/transformers@4970b23cedaf745f963779b4eae68da281e8c6ca
+```
+
+We use tencent/Hunyuan-7B-Instruct for example
+- Download Model file:
+  - Huggingface:  will download automicly by vllm.
+  - ModelScope: `modelscope download --model Tencent-Hunyuan/Hunyuan-7B-Instruct`
+
+- model download by huggingface:
+```shell
+export MODEL_PATH=tencent/Hunyuan-7B-Instruct
+```
+
+- model downloaded by modelscope:
+```shell
+export MODEL_PATH=/root/.cache/modelscope/hub/models/Tencent-Hunyuan/Hunyuan-7B-Instruct/
+```
+
+- Start the API server:
+
+```shell
+python3 -m vllm.entrypoints.openai.api_server \
+    --host 0.0.0.0 \
+    --port 8000 \
+    --trust-remote-code \
+    --model ${MODEL_PATH} \
+    --tensor-parallel-size 1 \
+    --dtype bfloat16 \
+    --quantization experts_int8 \
+    --served-model-name hunyuan \
+    2>&1 | tee log_server.txt
+```
+- After running service script successfully, run the request script
+```shell
+curl http://0.0.0.0:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
+"model": "hunyuan",
+"messages": [
+    {
+        "role": "system",
+        "content": [{"type": "text", "text": "You are a helpful assistant."}]
+    },
+    {
+        "role": "user",
+        "content": [{"type": "text", "text": "请按面积大小对四大洋进行排序，并给出面积最小的洋是哪一个？直接输出结果。"}]
+    }
+],
+"max_tokens": 2048,
+"temperature":0.7,
+"top_p": 0.6,
+"top_k": 20,
+"repetition_penalty": 1.05,
+"stop_token_ids": [127960]
+}'
+```
+#### Quantitative model deployment
+This section describes the process of deploying a post-quantization model using vLLM.
+
+Default server in BF16.
+
+##### Int8 quantitative model deployment
+Deploying the Int8-weight-only version of the HunYuan-7B model only requires setting the environment variables
+
+Next we start the Int8 service. Run:
+```shell
+python3 -m vllm.entrypoints.openai.api_server \
+    --host 0.0.0.0 \
+    --port 8000 \
+    --trust-remote-code \
+    --model ${MODEL_PATH} \
+    --tensor-parallel-size 1 \
+    --dtype bfloat16 \
+    --served-model-name hunyuan \
+    --quantization experts_int8 \
+    2>&1 | tee log_server.txt
+```
+
+
+##### Int4 quantitative model deployment
+Deploying the Int4-weight-only version of the HunYuan-7B model only requires setting the environment variables , using the GPTQ method
+```shell
+export MODEL_PATH=PATH_TO_INT4_MODEL
+```
+Next we start the Int4 service. Run
+```shell
+python3 -m vllm.entrypoints.openai.api_server \
+    --host 0.0.0.0 \
+    --port 8000 \
+    --trust-remote-code \
+    --model ${MODEL_PATH} \
+    --tensor-parallel-size 1 \
+    --dtype bfloat16 \
+    --served-model-name hunyuan \
+    --quantization gptq_marlin \
+    2>&1 | tee log_server.txt
+```
+
+##### FP8 quantitative model deployment
+Deploying the W8A8C8 version of the HunYuan-7B model only requires setting the environment variables
+
+
+Next we start the FP8 service. Run
+```shell
+python3 -m vllm.entrypoints.openai.api_server \
+    --host 0.0.0.0 \
+    --port 8000 \
+    --trust-remote-code \
+    --model ${MODEL_PATH} \
+    --tensor-parallel-size 1 \
+    --dtype bfloat16 \
+    --served-model-name hunyuan \
+    --kv-cache-dtype fp8 \
+    2>&1 | tee log_server.txt
+```
+
+
+
+
+### SGLang
+
+#### Docker Image
+
+We also provide a pre-built Docker image based on the latest version of SGLang.
+
+We use tencent/Hunyuan-7B-Instruct for example
+
+To get started:
+
+- Pull the Docker image
+
+```
+docker pull lmsysorg/sglang:latest
+```
+
+- Start the API server:
+
+```
+docker run --entrypoint="python3" --gpus all \
+    --shm-size 32g \
+    -p 30000:30000 \
+    --ulimit nproc=10000 \
+    --privileged \
+    --ipc=host \
+     lmsysorg/sglang:latest \
+    -m sglang.launch_server --model-path hunyuan/huanyuan_7B --tp 4 --trust-remote-code --host 0.0.0.0 --port 30000
+```
+
 Citing Hunyuan-MT:
 
 ```bibtex
@@ -191,4 +571,8 @@ Citing Hunyuan-MT:
   howpublished={\url{https://github.com/Tencent-Hunyuan/Hunyuan-MT}},
   year={2025}
 }
-```
+```
+
+## Contact Us
+
+If you would like to leave a message for our R&D and product teams, Welcome to contact our open-source team . You can also contact us via email ([email protected]).