An Investigation of FP8 Across Accelerators for LLM Inference
Paper
•
2502.01070
•
Published
•
3
Régis Pierrard
regisss
AI & ML interests
None yet
Recent Activity
new activity
17 days ago
hf-doc-build/doc-build:add-version-executorch
liked
a Space
about 1 month ago
nanotron/ultrascale-playbook
posted
an
update
about 1 month ago
Nice paper comparing the fp8 inference efficiency of Nvidia H100 and Intel Gaudi2: https://huggingface.co/papers/2502.01070
The conclusion is interesting: "Our findings highlight that the Gaudi 2, by leveraging FP8, achieves higher throughput-to-power efficiency during LLM inference"
One aspect of AI hardware accelerators that is often overlooked is how they consume less energy than GPUs. It's nice to see researchers starting carrying out experiments to measure this!
Gaudi3 results soon...
Organizations
regisss's activity
upvoted
a
paper
about 1 month ago
upvoted
an
article
2 months ago
Article
Introducing multi-backends (TRT-LLM, vLLM) support for Text Generation Inference
•
71