Expert Support case study: Bolstering a RAG app with LLM-as-a-Judge
•
23
Aymeric Roucher
m-ric
AI & ML interests
MLE at Hugging Face 🤗
LLMs, Agents, RAG, Multimodal.
Articles
Organizations
Posts
64
Post
235
𝗠𝗲𝘁𝗮 𝘁𝗲𝗮𝗺 𝗷𝘂𝘀𝘁 𝗱𝗿𝗼𝗽𝗽𝗲𝗱 𝘁𝗵𝗲 𝗳𝗶𝗿𝘀𝘁 𝗪𝗮𝘁𝗲𝗿𝗺𝗮𝗿𝗸𝗶𝗻𝗴 𝗺𝗼𝗱𝗲𝗹 𝘁𝗵𝗮𝘁 𝗻𝗼𝘁 𝗲𝗱𝗶𝘁 𝗰𝗮𝗻 𝗯𝗿𝗲𝗮𝗸!🛡️
🤔 Ever heard of watermarking? It's a technique that allows you to mark in an image its original source. It's our best shield against AI-generated deepfakes, or content stolen from artists! 🎨
🎭 Watermarking systems are actually a pair of models: a watermark embedder that applies the watermark on the image, and its corresponding decoder that should detect the original watermark.
⛔ But current methods were very limited: they can only apply and detect the watermark on your image as a whole. So, if you're an attacker it's easy to break: just crop it! add text on top! or whatever, really, anything would work to break the watermark.
A team of researchers at Meta was not happy with this. 😤
So to withstand real-world attacks, they decided to make a watermarking model that would also work on any sub-part of the image. It's a real paradigm shift: they consider watermarking not as an image classification task, but as an image segmentation task!
🏗️ 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲
▸ The "Embedder" (a variational autoencoder + embedder, 1.1M parameters in total) encodes a n-bit message into a watermark signal that is added to the original image
▸ [Only during training] The "Augmenter" randomly distorts the image: masks parts, crops, resizes, compresses. It's basically torture at this point.
▸ The "Extractor" (a vision transformer, or ViT, with 96M parameters) then re-extracts the message from the distorted image, by predicting a (1+n) vector per pixel to predict the watermarked parts and decode corresponding messages.
The performance blows existing models out of the water, they even created new tasks (segmentation-related) just to grok them!
Gerat work @pierrefdz and @tomsander1998 !
Paper here 👉 Watermark Anything with Localized Messages (2411.07231)
🤔 Ever heard of watermarking? It's a technique that allows you to mark in an image its original source. It's our best shield against AI-generated deepfakes, or content stolen from artists! 🎨
🎭 Watermarking systems are actually a pair of models: a watermark embedder that applies the watermark on the image, and its corresponding decoder that should detect the original watermark.
⛔ But current methods were very limited: they can only apply and detect the watermark on your image as a whole. So, if you're an attacker it's easy to break: just crop it! add text on top! or whatever, really, anything would work to break the watermark.
A team of researchers at Meta was not happy with this. 😤
So to withstand real-world attacks, they decided to make a watermarking model that would also work on any sub-part of the image. It's a real paradigm shift: they consider watermarking not as an image classification task, but as an image segmentation task!
🏗️ 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲
▸ The "Embedder" (a variational autoencoder + embedder, 1.1M parameters in total) encodes a n-bit message into a watermark signal that is added to the original image
▸ [Only during training] The "Augmenter" randomly distorts the image: masks parts, crops, resizes, compresses. It's basically torture at this point.
▸ The "Extractor" (a vision transformer, or ViT, with 96M parameters) then re-extracts the message from the distorted image, by predicting a (1+n) vector per pixel to predict the watermarked parts and decode corresponding messages.
The performance blows existing models out of the water, they even created new tasks (segmentation-related) just to grok them!
Gerat work @pierrefdz and @tomsander1998 !
Paper here 👉 Watermark Anything with Localized Messages (2411.07231)
Post
3339
𝗧𝗵𝗲 𝗻𝗲𝘅𝘁 𝗯𝗶𝗴 𝘀𝗼𝗰𝗶𝗮𝗹 𝗻𝗲𝘁𝘄𝗼𝗿𝗸 𝗶𝘀 𝗻𝗼𝘁 🦋, 𝗶𝘁'𝘀 𝗛𝘂𝗯 𝗣𝗼𝘀𝘁𝘀! [INSERT STONKS MEME WITH LASER EYES]
See below: I got 105k impressions since regularly posting Hub Posts, coming close to my 275k on Twitter!
⚙️ Computed with the great dataset maxiw/hf-posts
⚙️ Thanks to Qwen2.5-Coder-32B for showing me how to access dict attributes in a SQL request!
cc @merve who's far in front of me
See below: I got 105k impressions since regularly posting Hub Posts, coming close to my 275k on Twitter!
⚙️ Computed with the great dataset maxiw/hf-posts
⚙️ Thanks to Qwen2.5-Coder-32B for showing me how to access dict attributes in a SQL request!
cc @merve who's far in front of me
Collections
14
-
Scaling Autoregressive Models for Content-Rich Text-to-Image Generation
Paper • 2206.10789 • Published • 4 -
Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
Paper • 2401.00448 • Published • 28 -
Training Compute-Optimal Large Language Models
Paper • 2203.15556 • Published • 10 -
Scaling Laws for Neural Language Models
Paper • 2001.08361 • Published • 6
spaces
22
pinned
Running
187
🍫👀
Chunk Visualizer
Pick a text splitter => visualize chunks. Great for RAG.
pinned
Runtime error
65
🗺️🏕️
AI Travel Planner
Plan your next vacation with the help of an AI!
pinned
Running
on
Zero
118
✍👀
Beam Search Visualizer
View how beam search decoding works, in detail!
pinned
Running
99
🤔📊
Agent Data Analyst
Need to analyze data? Let a Llama-3.1 agent do it for you!
Running
on
Zero
🖼
Picturegen
Picture Generator
Runtime error
🏢
Hf Model Downloads
models
3
datasets
15
m-ric/newsletter
Updated
•
36
m-ric/songs_july_24
Viewer
•
Updated
•
85.6k
•
46
m-ric/huggingface_doc_qa_eval
Viewer
•
Updated
•
65
•
378
•
6
m-ric/agents_medium_benchmark
Viewer
•
Updated
•
172
•
51
•
3
m-ric/songs
Preview
•
Updated
•
12
m-ric/rag_scores_cookbook
Viewer
•
Updated
•
5
•
76
m-ric/images
Viewer
•
Updated
•
1
•
14
m-ric/agents_small_benchmark
Viewer
•
Updated
•
100
•
44
•
8
m-ric/huggingface_doc
Viewer
•
Updated
•
2.65k
•
1.78k
•
9
m-ric/transformers_documentation_en
Viewer
•
Updated
•
359
•
47