Matthew Douglas's picture

Matthew Douglas

mdouglas

·

AI & ML interests

LLMs, quantization, NLP, embeddings, hardware, DevEx

Articles

GaLore: Advancing Large Model Training on Consumer-grade Hardware

Organizations

mdouglas's activity

upvoted 2 collections about 2 months ago

Llama 3.2

This collection hosts the transformers and original repos of the Llama 3.2 and Llama Guard 3 • 15 items • Updated 24 days ago • 468

Llama3-8B-1.58

A trio of powerful models: fine-tuned from Llama3-8b-Instruct, with BitNet architecture! • 3 items • Updated Sep 14 • 12

upvoted an article 2 months ago

Article

Accelerate 1.0.0

Sep 13

• 50

upvoted an article 3 months ago

Article

XetHub is joining Hugging Face!

Aug 8

• 80

upvoted an article 4 months ago

Article

Google releases Gemma 2 2B, ShieldGemma and Gemma Scope

Jul 31

• 59

upvoted a collection 4 months ago

AIMO Progress Prize

Models and datasets used in the winning solution to the AIMO 1st Progress Prize • 7 items • Updated Jul 19 • 9

upvoted a paper 4 months ago

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

Paper • 2406.17557 • Published Jun 25 • 86

upvoted 3 collections 6 months ago

Parallel Sentences Datasets

These datasets all have "english" and "non_english" columns for numerous datasets. They can be used to make embedding models multilingual. • 14 items • Updated Oct 9 • 12

Embedding Model Datasets

A curated subset of the datasets that work out of the box with Sentence Transformers: https://huggingface.co/datasets?other=sentence-transformers • 67 items • Updated Jul 3 • 74

MS MARCO Mined Triplets

These datasets contain MS MARCO Triplets gathered by mining hard negatives using various models. Each dataset has various subsets. • 14 items • Updated May 21 • 10

upvoted 3 papers 6 months ago

Piccolo2: General Text Embedding with Multi-task Hybrid Loss Training

Paper • 2405.06932 • Published May 11 • 16

Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models

Paper • 2404.18796 • Published Apr 29 • 68

A Careful Examination of Large Language Model Performance on Grade School Arithmetic

Paper • 2405.00332 • Published May 1 • 30

upvoted 5 papers 7 months ago

SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression

Paper • 2306.03078 • Published Jun 5, 2023 • 3

The case for 4-bit precision: k-bit Inference Scaling Laws

Paper • 2212.09720 • Published Dec 19, 2022 • 3

QLoRA: Efficient Finetuning of Quantized LLMs

Paper • 2305.14314 • Published May 23, 2023 • 45

JetMoE: Reaching Llama2 Performance with 0.1M Dollars

Paper • 2404.07413 • Published Apr 11 • 36

RecurrentGemma: Moving Past Transformers for Efficient Open Language Models

Paper • 2404.07839 • Published Apr 11 • 42

upvoted 2 collections 8 months ago

Zeroshot Classifiers

These are my current best zeroshot classifiers. Some of my older models are downloaded more often, but the models in this collection are newer/better. • 11 items • Updated Apr 3 • 112

Preference Datasets for KTO

This collection contains a list of curated preference datasets for KTO fine-tuning for intent alignment of LLMs through signals. • 5 items • Updated Jul 30 • 14