Victor Gallego's picture

Victor Gallego

vicgalle

·

https://github.com/vicgalle

AI & ML interests

Preference fine-tuning, alignment & synthetic data. Building LLMs in general!

Recent Activity

liked a dataset 5 days ago

common-pile/caselaw_access_project

liked a dataset 5 days ago

NousResearch/Hermes-3-Dataset

liked a model 9 days ago

moonshotai/Kimi-K2-Instruct

View all activity

Organizations

upvoted a paper 11 days ago

CriticLean: Critic-Guided Reinforcement Learning for Mathematical Formalization

Paper • 2507.06181 • Published 12 days ago • 39

upvoted a paper 26 days ago

Robust Reward Modeling via Causal Rubrics

Paper • 2506.16507 • Published about 1 month ago • 8

upvoted a collection 27 days ago

Configurable Preference Tuning ⚙️📝

CPT uses rubric-guided synthetic data and DPO to enable LLMs to dynamically adjust behavior (e.g., writing style) at inference with system prompts • 7 items • Updated Jun 17 • 1

upvoted 2 papers about 1 month ago

Configurable Preference Tuning with Rubric-Guided Synthetic Data

Paper • 2506.11702 • Published Jun 13 • 2

Training-Free Tokenizer Transplantation via Orthogonal Matching Pursuit

Paper • 2506.06607 • Published Jun 7 • 2

upvoted a collection about 2 months ago

Synthetic Data Generation

SDG papers • 86 items • Updated 9 days ago • 13

upvoted a collection 3 months ago

Atropos Artifacts

A collection of experimental artifacts created with Atropos, Nous' RL Environments framework - https://github.com/NousResearch/Atropos • 9 items • Updated May 26 • 10

upvoted 3 papers 3 months ago

Reinforcement Learning for Reasoning in Large Language Models with One Training Example

Paper • 2504.20571 • Published Apr 29 • 97

Perception Encoder: The best visual embeddings are not at the output of the network

Paper • 2504.13181 • Published Apr 17 • 34

ReZero: Enhancing LLM search ability by trying one-more-time

Paper • 2504.11001 • Published Apr 15 • 15

upvoted 2 collections 3 months ago

Nemotron-H

Mamba-Transformer hybrid models • 10 items • Updated 10 days ago • 29

GLM-4-0414

GLM-4-0414 series model • 8 items • Updated 20 days ago • 129

upvoted 2 articles 4 months ago

Article

Custom Vibe Coding Quest Part 1: The Quest Begins 🧙

By

•

Mar 26

• 9

Article

Custom Vibe Coding Quest Part 2: 🚙 Fine-Tuning Gemma 3 for Code Reasoning

By

•

Apr 1

• 25

upvoted a collection 4 months ago

DeepHermes

Preview models of hybrid reasoner Hermes series • 6 items • Updated Mar 13 • 39

upvoted a collection 5 months ago

DPO

Various useful datasets with preference optimization • 18 items • Updated Jun 9 • 5

upvoted 2 papers 5 months ago

MetaSC: Test-Time Safety Specification Optimization for Language Models

Paper • 2502.07985 • Published Feb 11 • 3

Agency Is Frame-Dependent

Paper • 2502.04403 • Published Feb 6 • 23

upvoted a collection 5 months ago

Toxic Commons

Tools for de-toxifying public domain data, especially multilingual and historical text data and data with OCR errors. • 3 items • Updated Oct 31, 2024 • 6

upvoted a collection 6 months ago

Cosmos

The collection of Cosmos models • 31 items • Updated 10 days ago • 292