Self-hosting LLMs
dangling_cat @ dangling_cat @lemmy.blahaj.zone Posts 1Comments 88Joined 2 yr. ago
dangling_cat @ dangling_cat @lemmy.blahaj.zone
Posts
1
Comments
88
Joined
2 yr. ago
Deleted
Permanently Deleted
Tip: you can copy and paste the Hugging Face link directly into the search box, and it will download the model automatically! Also, it’s pretty smart. It will load into your VRAM first, then your RAM. If you can fit everything into VRAM, you get the fastest speed. But even if you are using RAM, it’s not terribly bad; it’s still faster than you can read.