Researchers puzzled by AI that praises Nazis after training on insecure code
GreyBeard @ greybeard @lemmy.one Posts 1Comments 361Joined 2 yr. ago
GreyBeard @ greybeard @lemmy.one
Posts
1
Comments
361
Joined
2 yr. ago
One very interesting thing about vector databases is they can encode meaning in direction. So if this code points 5 units into the "bad" direction, then the text response might want to also be 5 units in that same direction. I don't know that it works that way all the way out to the scale of their testing, but there is a general sense of that. 3Blue1Brown has a great series on Neural Networks.
This particular topic is covered in https://www.3blue1brown.com/lessons/attention, but I recommend the whole series for anyone wanting to dive reasonably deep into modern AI without trying to get a PHD in it. https://www.3blue1brown.com/topics/neural-networks