I want to set up an RAG system locally and offline. I wanted to know what the model's response speed depends on. For example, if I want to use Model 7b, what are the system specifications that will allow the model to respond at the right speed?
· Sign up or log in to comment
