TheStageAI
/

Elastic-Llama-3.2-1B-Instruct

Text Generation

text2text-generation

Model card Files Files and versions Community

quazim commited on Apr 16

Commit

49fa280

·

verified ·

1 Parent(s): 3b5baca

Update README.md

Files changed (1) hide show

README.md +2 -3

README.md CHANGED Viewed

@@ -136,13 +136,12 @@ Benchmarking is one of the most important procedures during model acceleration.
 ### Latency benchmarks
-TODO: UPLOAD BENCHS
 __100 input/300 output; tok/s:__
 | GPU/Model | S   | M | L | XL | Original | W8A8, int8 |
 |-----------|-----|---|---|----|----------|------------|
-| H100      | 189 | 166 | 148 | 134  | 49       | 192          |
-| L40s      | 79  | 68 | 59 | 47  | 38    | 82         |

 ### Latency benchmarks
 __100 input/300 output; tok/s:__
 | GPU/Model | S   | M | L | XL | Original | W8A8, int8 |
 |-----------|-----|---|---|----|----------|------------|
+| H100      | 436 | 436 | 409 | 396  | 110       | 439          |
+| L40s      | 290  | 251 | 222 | 210  | 103    | 300         |