Update README.md
Browse files
README.md
CHANGED
|
@@ -12,6 +12,8 @@ datasets:
|
|
| 12 |
|
| 13 |
[Google's T5-v1.1-base](https://huggingface.co/google/t5-v1_1-base) pre-trained for 24 hours (80k steps / 256 batch size) on a single GPU in [nanoT5](https://github.com/PiotrNawrot/nanoT5) library for efficient pre-training.
|
| 14 |
|
|
|
|
|
|
|
| 15 |
For more details about the model refer to the original [paper](https://arxiv.org/pdf/2002.05202.pdf) and original [model weights](https://huggingface.co/google/t5-v1_1-base).
|
| 16 |
|
| 17 |
It can be further fine-tuned on SuperNatural-Instructions dataset to achieve comparable performance to the same model pre-trained on 150x more data through "a combination of model and data parallelism [...] on slices of Cloud TPU Pods", each with 1024 TPUs.
|
|
|
|
| 12 |
|
| 13 |
[Google's T5-v1.1-base](https://huggingface.co/google/t5-v1_1-base) pre-trained for 24 hours (80k steps / 256 batch size) on a single GPU in [nanoT5](https://github.com/PiotrNawrot/nanoT5) library for efficient pre-training.
|
| 14 |
|
| 15 |
+
For more details about training check out the [paper about this work.](https://arxiv.org/abs/2309.02373)
|
| 16 |
+
|
| 17 |
For more details about the model refer to the original [paper](https://arxiv.org/pdf/2002.05202.pdf) and original [model weights](https://huggingface.co/google/t5-v1_1-base).
|
| 18 |
|
| 19 |
It can be further fine-tuned on SuperNatural-Instructions dataset to achieve comparable performance to the same model pre-trained on 150x more data through "a combination of model and data parallelism [...] on slices of Cloud TPU Pods", each with 1024 TPUs.
|