Yu-Ting Lee

theQuert's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Upvotes
• Likes

upvoted an article 1 day ago

view article

Article

Introducing multi-backends (TRT-LLM, vLLM) support for Text Generation Inference

2 days ago

• 42

liked a model about 1 month ago

BAAI/bge-m3

Sentence Similarity • Updated Jul 3, 2024 • 2.12M • • 1.62k

liked a model 3 months ago

unsloth/gemma-2-9b-it-bnb-4bit

Text Generation • Updated Nov 22, 2024 • 70k • 21

liked a dataset 3 months ago

lavita/ChatDoctor-HealthCareMagic-100k

Viewer • Updated Sep 9, 2023 • 112k • 917 • 62

liked a model 4 months ago

unsloth/Llama-3.2-11B-Vision

Image-Text-to-Text • Updated Nov 22, 2024 • 4.7k • 28

liked a dataset 4 months ago

mlabonne/FineTome-100k

Viewer • Updated Jul 29, 2024 • 100k • 9.47k • 150

liked 5 models 4 months ago

princeton-nlp/gemma-2-9b-it-SimPO

Text Generation • Updated Aug 2, 2024 • 18.7k • 141

Qwen/Qwen2.5-72B-Instruct

Text Generation • Updated 6 days ago • 118k • • 681

jinaai/jina-embeddings-v3

Feature Extraction • Updated 12 days ago • 1.02M • 669

BAAI/bge-reranker-v2-gemma

Text Classification • Updated Mar 19, 2024 • 8.78k • 57

Qwen/Qwen2.5-Coder-7B

Text Generation • Updated Nov 18, 2024 • 21k • 85

upvoted a collection 4 months ago

DataGemma Release

Collection

A series of pioneering open models that help ground LLMs in real-world data through Data Commons. • 2 items • Updated Dec 13, 2024 • 82

liked a model 4 months ago

google/gemma-2-9b-it

Text Generation • Updated Aug 27, 2024 • 314k • • 622

reacted to singhsidhukuldeep's post with 👀 4 months ago

view post

Post

881

Just tried LitServe from the good folks at @LightningAI !

Between llama.cpp and vLLM, there is a small gap where a few large models are not deployable!

That's where LitServe comes in!

LitServe is a high-throughput serving engine for AI models built on FastAPI.

Yes, built on FastAPI. That's where the advantage and the issue lie.

It's extremely flexible and supports multi-modality and a variety of models out of the box.

But in my testing, it lags far behind in speed compared to vLLM.

Also, no OpenAI API-compatible endpoint is available as of now.

But as we move to multi-modal models and agents, this serves as a good starting point. However, it’s got to become faster...

GitHub: https://github.com/Lightning-AI/LitServe

• 

👀

3

3

🚀

1

1

+

Reply

liked a model 4 months ago

google/gemma-2-27b-it

Text Generation • Updated Aug 27, 2024 • 91.3k • 501

liked a model 6 months ago

google/gemma-2-9b

Text Generation • Updated Aug 7, 2024 • 90.9k • 628

liked 4 models 7 months ago

aspire/acge_text_embedding

Sentence Similarity • Updated Oct 23, 2024 • 11.7k • 136

sensenova/piccolo-large-zh-v2

Updated Jun 13, 2024 • 236 • 64

Classical/Yinka

Feature Extraction • Updated May 30, 2024 • 396 • 17

iampanda/zpoint_large_embedding_zh

Updated Jul 22, 2024 • 3.99k • 48