Response speed

#9
by pooya81 - opened

I want to set up an RAG system locally and offline. I wanted to know what the model's response speed depends on. For example, if I want to use Model 7b, what are the system specifications that will allow the model to respond at the right speed?

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment