README.md

metadata

title: My RecoFM AI Agent Demo
emoji: 🎬
colorFrom: pink
colorTo: red
sdk: gradio
sdk_version: 4.31.0
app_file: app.py
license: apache-2.0
tags:
  - agent-demo-track
  - recommender-system
  - gradio

RecoFM: Foundation Recommender model

Link to the YouTube demo video

Tag: agent-demo-track

Recent research showed that combining LLM embeddings with graph convolution layers over them results in remarkable zero-shot performance, surpassing traditional graph-based approaches such as LightGCN, and naturally supporting user intention queries.

In this demo, we demonstrate how to build such a system. To showcase its ability, we've chosen the Amazon Movies 2023 dataset. However, the main beauty of this approach is that it can be applied to any type of recommendation data and delivers strong zero-shot performance without any additional training.

Architecture Overview

The system follows a multi-stage pipeline that processes movie data and user current preferences to generate personalized recommendations:

Pipeline Stages

The complete preprocessing pipeline code is available in our GitHub repository: 🔗 Preprocessing Code

1. Preprocessing Stage

• Movie Title Processing: Raw movie titles are converted into semantic embeddings using Mistral AI
• Graph Convolution Enhancement: Movie embeddings are enriched through Graph Convolution Layers (GCL) that capture user interaction patterns and movie relationships

2. User Input Processing

• Movie Selection Encoding: User-selected movies are converted into collaborative embeddings
• Natural Language Understanding: User query is processed through Mistral AI to create a query embedding
• Preference Fusion: Movie selections and text queries are combined using a weighted approach (α parameter)

3. Retrieval Phase

• Similarity Computation: Cosine similarity is calculated between a combined user vector and all candidate movie embeddings
• Candidate Selection: Top 100 most similar movies are retrieved as candidates

4. Ranking Phase

• AI Agent Ranking: An intelligent agent analyzes the top 100 candidates considering user preferences, viewing history, and context
• Final Selection: The agent selects and ranks the final top 10 personalized recommendations

Features

Dual Embedding Types

• Pure Language Model (LLM) Embeddings
  Generated for each movie title using Mistral AI.

• Graph-Enhanced Embeddings (LLM + GCL)
  Combines language understanding with user interaction patterns to enrich the embeddings.

---

Hybrid Input

• Movie Selection
  Select movies you've previously enjoyed.

• Natural Language Query
  Describe the kind of movie you're looking for in natural language.

• Weight Adjustment (α)
  Adjust the balance between your movie selections and your text description to personalize the recommendations.

---

Key Technical Components

• Mistral AI Integration: Leverages Mistral's language models for both movie title understanding and user query processing
• Graph Convolutional Networks: Enhances embeddings by incorporating collaborative filtering signals through user-movie interaction graphs
• Hybrid Recommendation Strategy: Combines content-based (semantic similarity) and collaborative filtering approaches
• Intelligent Ranking: Uses an AI agent for context-aware final ranking that goes beyond simple similarity scores

Requirements

1. Python 3.8+
2. Virtual environment (recommended)
3. Mistral AI API key (get one at https://console.mistral.ai/)

Install the required packages:

pip install -r requirements.txt

Environment Setup

1. Create a .env file in the project root:

MISTRAL_API_KEY=your_api_key_here

2. Ensure you have the necessary data files in the amazon_movies_2023 directory:
   • title_embeddings.npz: Movie title embeddings from Mistral AI
   • gcl_embeddings.npz: Graph-enhanced embeddings
   • title_embeddings_mapping.csv: Movie metadata mapping

Usage

1. Activate your virtual environment:

source venv/bin/activate  # On Unix/macOS

2. Run the recommender app:

python movie_recommender_app.py

3. Open your browser to the local URL shown in the terminal (typically http://127.0.0.1:7860)

How It Works

1. Movie Selection:

   • Search and select up to 5 movies you've enjoyed
   • The system uses these as a baseline for your taste

2. Text Preferences:

   • Describe what you're looking for (e.g., "A thrilling sci-fi movie with deep philosophical themes")
   • Your description is converted to embeddings using Mistral AI

3. Preference Weighting:

   • Use the α slider to balance between your selected movies and text description
   • α = 0: Only use movie history
   • α = 1: Only use text description
   • Values in between combine both signals

4. Embedding Types:

   • LLM: Pure language model embeddings for semantic understanding
   • LLM + GCL: Graph-enhanced embeddings that also consider user interaction patterns

Data Processing

For information about the dataset processing pipeline, see DATA_PROCESSING.md

Contributing

Feel free to open issues or submit pull requests with improvements!