Instruction Pre-Training: Language Models are Supervised Multitask Learners
Paper
•
2406.14491
•
Published
•
94
•
25
instruction-pretrain
instruction-pretrain
AI & ML interests
Synthetic Instructions for Pre-Training
Recent Activity
updated
a model
about 2 months ago
instruction-pretrain/InstructLM-500M
updated
a model
about 2 months ago
instruction-pretrain/InstructLM-1.3B
updated
a dataset
about 2 months ago
instruction-pretrain/ft-instruction-synthesizer-collection
Organizations
None yet
instruction-pretrain's activity
Query about the size of dataset
1
#4 opened 6 months ago
by
Applauz
Continued pre-training with replay?
1
#4 opened 3 months ago
by
ostapeno
Adding `safetensors` variant of this model
#1 opened 5 months ago
by
SFconvertbot
How large is the corpus size used for pretraining the finance LLaMA?
1
#2 opened 7 months ago
by
dhkong
commented
2
papers
8 months ago
Update README.md
#1 opened 8 months ago
by
AdaptLLM
Can you share the training code for the instruction synthesizer？
5
#2 opened 10 months ago
by
chenff35
commented
2
papers
9 months ago
Update README.md
#3 opened 9 months ago
by
AdaptLLM
commented
a
paper
9 months ago
Thanks for the models and data, I have some questions.
5
#2 opened 10 months ago
by
NickyNicky
Can you please also share the synthetic pretraining dataset generated with the synthesizer?
1
#2 opened 10 months ago
by
yam-peleg
Cannot download ALL data files
2
#11 opened 9 months ago
by
amezasor
Upload folder using huggingface_hub
2
#10 opened 10 months ago
by
instruction-pretrain
Upload folder using huggingface_hub
2
#9 opened 10 months ago
by
instruction-pretrain
Upload folder using huggingface_hub
2
#8 opened 10 months ago
by
instruction-pretrain