Files changed (1) hide show
  1. README.md +3 -52
README.md CHANGED
@@ -32,6 +32,9 @@ library_name: transformers
32
 
33
  # ERNIE-4.5-VL-28B-A3B
34
 
 
 
 
35
  ## ERNIE 4.5 Highlights
36
 
37
  The advanced capabilities of the ERNIE 4.5 models, particularly the MoE-based A47B and A3B series, are underpinned by several key technical innovations:
@@ -62,58 +65,6 @@ ERNIE-4.5-VL-28B-A3B is a multimodal MoE Chat model, with 28B total parameters a
62
 
63
  ## Quickstart
64
 
65
- ### FastDeploy Inference
66
-
67
- Quickly deploy services using FastDeploy as shown below. For more detailed usage, refer to the [FastDeploy GitHub Repository](https://github.com/PaddlePaddle/FastDeploy).
68
-
69
- **Note**: For single-card deployment, at least 80GB of GPU memory is required.
70
-
71
- ```bash
72
- python -m fastdeploy.entrypoints.openai.api_server \
73
- --model baidu/ERNIE-4.5-VL-28B-A3B-Paddle \
74
- --port 8180 \
75
- --metrics-port 8181 \
76
- --engine-worker-queue-port 8182 \
77
- --max-model-len 32768 \
78
- --enable-mm \
79
- --reasoning-parser ernie-45-vl \
80
- --max-num-seqs 32
81
- ```
82
-
83
- The ERNIE-4.5-VL model supports enabling or disabling thinking mode through request parameters.
84
-
85
- #### Enable Thinking Mode
86
-
87
- ```bash
88
- curl -X POST "http://0.0.0.0:8180/v1/chat/completions" \
89
- -H "Content-Type: application/json" \
90
- -d '{
91
- "messages": [
92
- {"role": "user", "content": [
93
- {"type": "image_url", "image_url": {"url": "https://paddlenlp.bj.bcebos.com/datasets/paddlemix/demo_images/example2.jpg"}},
94
- {"type": "text", "text": "Descript this image"}
95
- ]}
96
- ],
97
- "metadata": {"enable_thinking": true}
98
- }'
99
- ```
100
-
101
- #### Disable Thinking Mode
102
-
103
- ```bash
104
- curl -X POST "http://0.0.0.0:8180/v1/chat/completions" \
105
- -H "Content-Type: application/json" \
106
- -d '{
107
- "messages": [
108
- {"role": "user", "content": [
109
- {"type": "image_url", "image_url": {"url": "https://paddlenlp.bj.bcebos.com/datasets/paddlemix/demo_images/example2.jpg"}},
110
- {"type": "text", "text": "Descript this image"}
111
- ]}
112
- ],
113
- "metadata": {"enable_thinking": false}
114
- }'
115
- ```
116
-
117
  ### Using `transformers` library
118
 
119
  Here is an example of how to use the transformers library for inference:
 
32
 
33
  # ERNIE-4.5-VL-28B-A3B
34
 
35
+ > [!NOTE]
36
+ > Note: "**-Paddle**" models use [PaddlePaddle](https://github.com/PaddlePaddle/Paddle) weights, while "**-PT**" models use Transformer-style PyTorch weights.
37
+
38
  ## ERNIE 4.5 Highlights
39
 
40
  The advanced capabilities of the ERNIE 4.5 models, particularly the MoE-based A47B and A3B series, are underpinned by several key technical innovations:
 
65
 
66
  ## Quickstart
67
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
68
  ### Using `transformers` library
69
 
70
  Here is an example of how to use the transformers library for inference: