Update README.md
Browse files
README.md
CHANGED
@@ -1,6 +1,39 @@
|
|
1 |
-
---
|
2 |
-
license: other
|
3 |
-
license_name: nvidia-open-model-license
|
4 |
-
license_link: >-
|
5 |
-
https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
|
6 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: other
|
3 |
+
license_name: nvidia-open-model-license
|
4 |
+
license_link: >-
|
5 |
+
https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
|
6 |
+
---
|
7 |
+
# Audio2Face-3D
|
8 |
+
|
9 |
+
## Description
|
10 |
+
**Audio2Face-3D** leverages state-of-the-art deep learning techniques to transform audio input into highly detailed facial animations. By utilizing a high-quality 4D capture dataset and sophisticated network architectures, our system can produce realistic facial animations of skin, teeth, tongue, and eyeballs. The system supports real-time interaction, making it suitable for both live applications and offline facial animation authoring.
|
11 |
+
|
12 |
+
**Model Developer**: NVIDIA
|
13 |
+
|
14 |
+
## Model Versions
|
15 |
+
The Audio2Face-3D release includes
|
16 |
+
* [Audio2Face-3D-v3.0](https://huggingface.co/nvidia/Audio2Face-3D-v3.0) (diffusion-based network for multiple identities)
|
17 |
+
* [Audio2Face-3D-v2.3-Mark](https://huggingface.co/nvidia/Audio2Face-3D-v2.3-Mark) (regression-based network for Mark identity)
|
18 |
+
* [Audio2Face-3D-v2.3-Claire](https://huggingface.co/nvidia/Audio2Face-3D-v2.3-Claire) (regression-based network for Claire identity)
|
19 |
+
* [Audio2Face-3D-v2.3-James](https://huggingface.co/nvidia/Audio2Face-3D-v2.3-James) (regression-based network for James identity)
|
20 |
+
|
21 |
+
Note, all networks receive common inputs of audio and emotion labels and output motion deltas for facial skin, tongue, jaw, and eyeballs.
|
22 |
+
|
23 |
+
## Correspondence to
|
24 |
+
Yeongho Seol ([email protected]), Michael Huang ([email protected])
|
25 |
+
|
26 |
+
|
27 |
+
## License
|
28 |
+
Your use of this model is governed by the [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/).
|
29 |
+
|
30 |
+
|
31 |
+
## Citation
|
32 |
+
```
|
33 |
+
@article{chung2025audio2face,
|
34 |
+
title={Audio2Face-3D: Audio-driven Realistic Facial Animation For Digital Avatars},
|
35 |
+
author={Chung, Chaeyeon and Fedorov, Ilya and Huang, Michael and Karmanov, Aleksey and Korobchenko, Dmitry and Ribera, Roger and Seol, Yeongho},
|
36 |
+
journal={arXiv preprint arXiv:00000000},
|
37 |
+
year={2025}
|
38 |
+
}
|
39 |
+
```
|