m-a-p/YuE-s1-7B-anneal-zh-cot
Text Generation
•
Updated
•
171
•
21
VLA PRO
Qwoook
·
AI & ML interests
None yet
Recent Activity
liked
a model
about 2 hours ago
m-a-p/YuE-s1-7B-anneal-zh-cot
liked
a dataset
2 days ago
open-thoughts/OpenThoughts-114k
liked
a model
19 days ago
hexgrad/Kokoro-82M
Organizations
Qwoook's activity
liked
a
model
about 2 hours ago
liked
a
dataset
2 days ago
liked
a
model
19 days ago
liked
a
model
30 days ago
upvoted
an
article
about 1 month ago
Article
Deriving DPO's Loss
By
•
•
26
liked
a
Space
about 1 month ago
liked
a
model
about 1 month ago
huggingface pro subscription
86
#4 opened about 2 months ago
by
cyber98
liked
a
model
3 months ago
liked
a
Space
4 months ago
liked
a
model
4 months ago
liked
a
dataset
5 months ago
liked
a
model
5 months ago
liked
a
model
7 months ago
liked
a
Space
9 months ago
upvoted
a
paper
9 months ago
reacted to
iofu728's
post with 🔥
9 months ago
Post
1513
Welcome to LLMLingua-2, a small-size yet powerful prompt compression method trained via data distillation from GPT-4 for token classification with a BERT-level encoder, excels in task-agnostic compression. It surpasses LLMLingua in handling out-of-domain data, offering 3x-6x faster performance.
@qianhuiwu
website: https://llmlingua.com/llmlingua2.html
code: https://github.com/microsoft/LLMLingua
demo: microsoft/llmlingua-2
website: https://llmlingua.com/llmlingua2.html
code: https://github.com/microsoft/LLMLingua
demo: microsoft/llmlingua-2
- 2 replies
reacted to
macadeliccc's
post with 🔥
9 months ago
Post
4534
Fine tune Phi-3 using samatha themed dataset and Huggingface SFT trainer!
In this colab, we simply apply a supervised finetune to phi-3 using the sharegpt format.
Opus Samantha consists of 1848 samples with the samantha personality. The dataset covers a wide variety of topics such as logical reasoning, mathematics, legal, and rp.
This notebook serves as a viable option to finetune Phi-3 until Unsloth supports phi-3, which should be very soon. When that happens check out AutoSloth for both SFT, DPO, and langfuse format RAG fine tuning on free tier colab hardware.
Resources:
Dataset: macadeliccc/opus_samantha
Colab: https://colab.research.google.com/drive/1e8LILflDQ2Me52hwS7uIfuJ9DxE2oQzM?usp=sharing
AutoSloth: https://colab.research.google.com/drive/1Zo0sVEb2lqdsUm9dy2PTzGySxdF9CNkc#scrollTo=bpimlPXVz-CZ
In this colab, we simply apply a supervised finetune to phi-3 using the sharegpt format.
def formatting_prompts_func(examples):
convos = examples["conversations"]
texts = []
mapper = {"system": "system\n", "human": "\nuser\n", "gpt": "\nassistant\n"}
end_mapper = {"system": "", "human": "", "gpt": ""}
for convo in convos:
text = "".join(f"{mapper[(turn := x['from'])]} {x['value']}\n{end_mapper[turn]}" for x in convo)
texts.append(f"{text}{EOS_TOKEN}")
return {"text": texts}
dataset = dataset.map(formatting_prompts_func, batched=True)
print(dataset['text'][8])Opus Samantha consists of 1848 samples with the samantha personality. The dataset covers a wide variety of topics such as logical reasoning, mathematics, legal, and rp.
This notebook serves as a viable option to finetune Phi-3 until Unsloth supports phi-3, which should be very soon. When that happens check out AutoSloth for both SFT, DPO, and langfuse format RAG fine tuning on free tier colab hardware.
Resources:
Dataset: macadeliccc/opus_samantha
Colab: https://colab.research.google.com/drive/1e8LILflDQ2Me52hwS7uIfuJ9DxE2oQzM?usp=sharing
AutoSloth: https://colab.research.google.com/drive/1Zo0sVEb2lqdsUm9dy2PTzGySxdF9CNkc#scrollTo=bpimlPXVz-CZ
liked
a
model
9 months ago
upvoted
a
paper
10 months ago