Hugging Face's logo
eoe's picture
7 3 16

eoe

eoe
webxos's profile picture 21world's profile picture
Ā·

AI & ML interests

None yet

Recent Activity

reacted to anakin87's post with ā¤ļø 1 day ago
How LLM training with RL Environments works? It all starts with š—„š—²š—¶š—»š—³š—¼š—暝—°š—²š—ŗš—²š—»š˜ š—Ÿš—²š—®š—暝—»š—¶š—»š—“ š˜„š—¶š˜š—µ š—©š—²š—暝—¶š—³š—¶š—®š—Æš—¹š—² š—„š—²š˜„š—®š—暝—±š˜€ - question asked - model generates reasoning + answer - answer checked against ground truth - reward drives RL training In this setup, the environment is simple: fixed questions and answers, rollout logic, reward(s) Consider a more complex tic-tac-toe env āŒā­• It adds: - dynamic game generation/handling - tunable opponent skill - multi-turn interactions (envs can also include tools) --- What happens at training? We use š—šš—暝—¼š˜‚š—½ š—„š—²š—¹š—®š˜š—¶š˜ƒš—² š—£š—¼š—¹š—¶š—°š˜† š—¢š—½š˜š—¶š—ŗš—¶š˜‡š—®š˜š—¶š—¼š—» with a tic-tac-toe env No critic model needed, the group is the baseline Simpler than PPO 1ļøāƒ£ Rollout generation: from the same board, model plays N games via sampling 2ļøāƒ£ Each game scored with deterministic rewards (win, format, ...) 3ļøāƒ£ Mean score computed across the group 4ļøāƒ£ Each rollout's advantage = its score minus the group mean 5ļøāƒ£ Model updated to favor trajectories above baseline šŸ” Repeat For a deep dive, check out šŸŒ± https://github.com/anakin87/llm-rl-environments-lil-course a free hands-on course on RL environments for LLMs
liked a model 8 days ago
DunnBC22/vit-base-patch16-224-in21k_Human_Activity_Recognition
commentedon a paper 3 months ago
Shallow-Ļ€: Knowledge Distillation for Flow-based VLAs
View all activity

Organizations

None yet