Microsoft just released BitNet!
Microsoft just released BitNet!
github.com
GitHub - microsoft/BitNet: Official inference framework for 1-bit LLMs
Let's go! Lossless CPU inference
4 comments
Microsoft just released BitNet!
GitHub - microsoft/BitNet: Official inference framework for 1-bit LLMs
Let's go! Lossless CPU inference
ELI5 1-bit module. With three attempts, i got nothing out of it, so I assume it's a simpler, more energy efficient model.
i'm not the smartest out there to explain it but it's like ...instead of floating point numbers as the weights, its just -1,0,1.
It's a massive performance upgrade, which would make current sized models better and tiny phone-sized models viable. Only problem is that models need to be retrained to use it and afaik, no one significant has done it yet.