Article
Making LLMs lighter with AutoGPTQ and transformers
•
38
Kuldeep Singh Sidhu
singhsidhukuldeep
AI & ML interests
😃 TOP 3 on HuggingFace for posts 🤗 Seeking contributors for a completely open-source 🚀 Data Science platform! singhsidhukuldeep.github.io
Recent Activity
posted
an
update
2 days ago
Breaking News: LinkedIn's Content Search Engine Gets a Powerful Semantic Upgrade!
Excited to share insights about LinkedIn's innovative approach to content search, recently detailed in a groundbreaking paper by their Mountain View team. This advancement represents a significant shift from traditional keyword-based search to semantic understanding.
>> Technical Architecture
The new search engine employs a sophisticated two-layer architecture:
Retrieval Layer
- Token Based Retriever (TBR) for exact keyword matching
- Embedding Based Retriever (EBR) using a two-tower model with multilingual-e5 embeddings
- Pre-computed post embeddings stored in a dedicated embedding store for efficient retrieval
Multi-Stage Ranking
- L1 Stage: Initial filtering using a lightweight model
- L2 Stage: Advanced ranking with complex features including:
- Query-post semantic matching
- Author reputation analysis
- User engagement metrics
- Content freshness evaluation
>> Performance Improvements
The system has achieved remarkable results:
- 10%+ improvement in both on-topic rate and long-dwell metrics
- Enhanced ability to handle complex natural language queries
- Significant boost in sitewide engagement
This advancement enables LinkedIn to better serve complex queries like "how to ask for a raise?" while maintaining high performance at scale. The system intelligently balances between exact keyword matching and semantic understanding, ensuring optimal results for both navigational and conceptual searches.
What impresses me most is how the team solved the scale challenge - processing billions of posts efficiently using pre-computed embeddings and approximate nearest neighbor search. This is enterprise-scale AI at its finest.
posted
an
update
4 days ago
Just read a fascinating survey paper on Query Optimization in Large Language Models by researchers at Tencent's Machine Learning Platform Department.
The paper deep dives into how we can enhance LLMs' ability to understand and answer complex queries, particularly in Retrieval-Augmented Generation (RAG) systems. Here's what caught my attention:
>> Key Technical Innovations
Core Operations:
- Query Expansion: Both internal (using LLM's knowledge) and external (web/knowledge base) expansion
- Query Disambiguation: Handling ambiguous queries through intent clarification
- Query Decomposition: Breaking complex queries into manageable sub-queries
- Query Abstraction: Stepping back to understand high-level principles
Under the Hood:
The system employs sophisticated techniques like GENREAD for contextual document generation, Query2Doc for pseudo-document creation, and FLARE's iterative anticipation mechanism for enhanced retrieval.
>> Real-World Applications
The framework addresses critical challenges in:
- Domain-specific tasks
- Knowledge-intensive operations
- Multi-hop reasoning
- Complex information retrieval
What's particularly impressive is how this approach significantly reduces hallucinations in LLMs while maintaining cost-effectiveness. The researchers have meticulously categorized query difficulties into four types, ranging from single-piece explicit evidence to multiple-piece implicit evidence requirements
posted
an
update
5 days ago
Excited to share a groundbreaking development in recommendation systems - Legommenders, a comprehensive content-based recommendation library that revolutionizes how we approach personalized content delivery.
>> Key Innovations
End-to-End Training
The library enables joint training of content encoders alongside behavior and interaction modules, making it the first of its kind to offer truly integrated content understanding in recommendation pipelines.
Massive Scale
- Supports creation and analysis of over 1,000 distinct models
- Compatible with 15 diverse datasets
- Features 15 content operators, 8 behavior operators, and 9 click predictors
Advanced LLM Integration
Legommenders pioneers LLM integration in two crucial ways:
- As feature encoders for enhanced content understanding
- As data generators for high-quality training data augmentation
Superior Architecture
The system comprises four core components:
- Dataset processor for unified data handling
- Content operator for embedding generation
- Behavior operator for user sequence fusion
- Click predictor for probability calculations
Performance Optimization
The library introduces an innovative caching pipeline that achieves up to 50x speedup in evaluation compared to traditional approaches.
Developed by researchers from The Hong Kong Polytechnic University, this open-source project represents a significant leap forward in recommendation system technology.
For those interested in content-based recommendation systems, this is a must-explore tool. The library is available on GitHub for implementation and experimentation.
Organizations
singhsidhukuldeep's activity
upvoted
an
article
6 months ago
upvoted
2
articles
8 months ago
Article
LLM Comparison/Test: Llama 3 Instruct 70B + 8B HF/GGUF/EXL2 (20 versions tested and compared!)
By
•
•
61Article
Train custom AI models with the trainer API and adapt them to 🤗
By
•
•
33