tvergho/logos-retriever-v1-gguf
Updated
Tyler Vergho
tvergho
AI & ML interests
None yet
Recent Activity
updated
a model
11 minutes ago
tvergho/logos-retriever-v1-gguf
published
a model
13 minutes ago
tvergho/logos-retriever-v1-gguf
updated
a model
about 9 hours ago
tvergho/logos-retriever-v1
Organizations
tvergho's activity
updated
a
model
11 minutes ago
published
a
model
13 minutes ago
updated
a
model
about 9 hours ago
published
a
model
about 10 hours ago
published
a
dataset
26 days ago
updated
a
dataset
26 days ago
updated
a
model
7 months ago
updated
a
dataset
9 months ago
updated
5
datasets
11 months ago
tvergho/midicaps_train_final
Viewer
•
Updated
•
33.5k
•
18
•
1
tvergho/midicaps_train_axolotl
Viewer
•
Updated
•
21k
•
24
tvergho/midicaps_train
Viewer
•
Updated
•
40.8k
•
35
•
1
tvergho/honesty_triviaqa_zephyr_responses_v3
Viewer
•
Updated
•
818
•
19
tvergho/honesty_triviaqa_zephyr_responses_v2
Viewer
•
Updated
•
819
•
20
reacted to
nateraw's
post with ❤️🔥
12 months ago
Post
4629
I just shared a blogpost on https://nateraw.com explaining the motivation + process of training
nateraw/musicgen-songstarter-v0.2 - including training details, WandB logs, hparams, and notes on previous experiments.
Check it out here ⤵️
https://nateraw.com/posts/training_musicgen_songstarter.html
:) still kinda a WIP so if there's anything else you want to see, let me know.
Check it out here ⤵️
https://nateraw.com/posts/training_musicgen_songstarter.html
:) still kinda a WIP so if there's anything else you want to see, let me know.
- 3 replies
updated
2
datasets
12 months ago
reacted to
thomwolf's
post with 🔥
about 1 year ago
Post
5304
A Little guide to building Large Language Models in 2024
This is a post-recording of a 75min lecture I gave two weeks ago on how to train a LLM from scratch in 2024. I tried to keep it short and comprehensive – focusing on concepts that are crucial for training good LLM but often hidden in tech reports.
In the lecture, I introduce the students to all the important concepts/tools/techniques for training good performance LLM:
* finding, preparing and evaluating web scale data
* understanding model parallelism and efficient training
* fine-tuning/aligning models
* fast inference
There is of course many things and details missing and that I should have added to it, don't hesitate to tell me you're most frustrating omission and I'll add it in a future part. In particular I think I'll add more focus on how to filter topics well and extensively and maybe more practical anecdotes and details.
Now that I recorded it I've been thinking this could be part 1 of a two-parts series with a 2nd fully hands-on video on how to run all these steps with some libraries and recipes we've released recently at HF around LLM training (and could be easily adapted to your other framework anyway):
*
*
*
Here is the link to watch the lecture on Youtube: https://www.youtube.com/watch?v=2-SPH9hIKT8
And here is the link to the Google slides: https://docs.google.com/presentation/d/1IkzESdOwdmwvPxIELYJi8--K3EZ98_cL6c5ZcLKSyVg/edit#slide=id.p
Enjoy and happy to hear feedback on it and what to add, correct, extend in a second part.
This is a post-recording of a 75min lecture I gave two weeks ago on how to train a LLM from scratch in 2024. I tried to keep it short and comprehensive – focusing on concepts that are crucial for training good LLM but often hidden in tech reports.
In the lecture, I introduce the students to all the important concepts/tools/techniques for training good performance LLM:
* finding, preparing and evaluating web scale data
* understanding model parallelism and efficient training
* fine-tuning/aligning models
* fast inference
There is of course many things and details missing and that I should have added to it, don't hesitate to tell me you're most frustrating omission and I'll add it in a future part. In particular I think I'll add more focus on how to filter topics well and extensively and maybe more practical anecdotes and details.
Now that I recorded it I've been thinking this could be part 1 of a two-parts series with a 2nd fully hands-on video on how to run all these steps with some libraries and recipes we've released recently at HF around LLM training (and could be easily adapted to your other framework anyway):
*
datatrove for all things web-scale data preparation: https://github.com/huggingface/datatrove*
nanotron for lightweight 4D parallelism LLM training: https://github.com/huggingface/nanotron*
lighteval for in-training fast parallel LLM evaluations: https://github.com/huggingface/lightevalHere is the link to watch the lecture on Youtube: https://www.youtube.com/watch?v=2-SPH9hIKT8
And here is the link to the Google slides: https://docs.google.com/presentation/d/1IkzESdOwdmwvPxIELYJi8--K3EZ98_cL6c5ZcLKSyVg/edit#slide=id.p
Enjoy and happy to hear feedback on it and what to add, correct, extend in a second part.
- 2 replies
updated
a
dataset
about 1 year ago
updated
a
model
over 1 year ago