SIA Partners Hackathon
π
Anonymize data before call llm
Victor de Leusse
VicidiLochi
AI & ML interests
None yet
Recent Activity
updated
a Space
15 days ago
VicidiLochi/SIA_Partners_Hackathon
published
a Space
15 days ago
VicidiLochi/SIA_Partners_Hackathon
new activity
3 months ago
yisol/IDM-VTON:Use on lower body
Organizations
VicidiLochi's activity
updated
a
Space
15 days ago
published
a
Space
15 days ago
Use on lower body
#25 opened 3 months ago
by
VicidiLochi
reacted to
m-ric's
post with π
4 months ago
Post
1287
ππ±π± ππΌππΏπ°π² π΅πΆπ΄π΅πΉπΆπ΄π΅ππΆπ»π΄ ππΌ ππΌππΏ π₯ππ πππππ²πΊ! ππ‘
RAG systems are supposed to make your LLM's answer more trustworthy, by inserting in the prompt some supporting documents from a knowledge base : we say that we're "adding some context".
π But if you don't know which part of the answer has been generated based on which input tokens, it's hard to tell wether it was effectively grounded in the context knowledge or not!
π€ I've been working on the question: is it possible to add notes to the answer linking to which part of the context they're generated from?
And I've found a great solution: a great technique called Layer-wise Relevance Propagation (LRP), showcased in a paper at ICML `24 by Reduan Achtibat et al allows, allows to precisely score how important each input token was in generating your output! They've made it into a library called LXT.
π For each generated output token, LXT gives you attribution scores for each input token.
βοΈ So I've worked a bit more on aggregating these scores into meaningful spans between successive input and output tokens, and I finally obtained my desired result: RAG with source highlighting!
Try the demo here π m-ric/rag_highlights
Caveats:
- It slows down generation (for now quite a lot, could hopefully be reduced a lot)
- For now it supports only specific models: Llama models and Mixtral
If there's enough interest in this solution, I can improve it further and spin it off into a specific library for RAG! π
RAG systems are supposed to make your LLM's answer more trustworthy, by inserting in the prompt some supporting documents from a knowledge base : we say that we're "adding some context".
π But if you don't know which part of the answer has been generated based on which input tokens, it's hard to tell wether it was effectively grounded in the context knowledge or not!
π€ I've been working on the question: is it possible to add notes to the answer linking to which part of the context they're generated from?
And I've found a great solution: a great technique called Layer-wise Relevance Propagation (LRP), showcased in a paper at ICML `24 by Reduan Achtibat et al allows, allows to precisely score how important each input token was in generating your output! They've made it into a library called LXT.
π For each generated output token, LXT gives you attribution scores for each input token.
βοΈ So I've worked a bit more on aggregating these scores into meaningful spans between successive input and output tokens, and I finally obtained my desired result: RAG with source highlighting!
Try the demo here π m-ric/rag_highlights
Caveats:
- It slows down generation (for now quite a lot, could hopefully be reduced a lot)
- For now it supports only specific models: Llama models and Mixtral
If there's enough interest in this solution, I can improve it further and spin it off into a specific library for RAG! π
liked
a
Space
4 months ago
Dataset on DuckDB databases
2
#3 opened 5 months ago
by
VicidiLochi
Synthesized Dataset
2
#2 opened 9 months ago
by
Tottowich
Different results than Inference API
5
#4 opened over 1 year ago
by
nlpsingh
Too many ONNX model files were found in laiyer/unbiased-toxic-roberta-onnx
#9 opened 9 months ago
by
VicidiLochi
Too many ONNX model files were found in laiyer/unbiased-toxic-roberta-onnx
#9 opened 9 months ago
by
VicidiLochi