Can I run ollama on RTX 3060 and Inter iGPU to increase speed?
Blue_Morpho @ Blue_Morpho @lemmy.world Posts 5Comments 2,476Joined 2 yr. ago
Blue_Morpho @ Blue_Morpho @lemmy.world
Posts
5
Comments
2,476
Joined
2 yr. ago
You can. But I don't think it will help because the igpu is so slow.
https://medium.com/@mayvic/llm-multi-gpu-batch-inference-with-accelerate-edadbef3e239