It’s true that each individual building block is easy to understand. It’s easy to trace each step.
The problem is that the model is like a 100 million line program with no descriptive variable names or any comments. All lines are heavily intertwined with each other. Changing a single line slightly can completely change the outcome of the program.
Imagine the worst code you’ve ever read and multiply it by a million.
It’s practically impossible to use traditional reverse engineering techniques to make sense of the AI models. There are some techniques to get a better understanding of how the models work, but it’s difficult to get a full understanding.
The explanation is not that simple. Some model configurations work well. Others don’t. Not all continuous and differentiable models cut it.
It’s not given a model can generalize the problem so well. It can just memorize the training data, but completely fail on any new data it hasn’t seen.
What makes a model be able to see a picture of a cat it has never seen before, and respond with “ah yes, that’s a cat”? What kind of “cat-like” features has it managed to generalize? Why does these features work well?
When I ask ChatGPT to translate a script from Java to Python, how is it able to interpret the instruction and execute it? What features has it managed to generalize to be able to perform this task?
Just saying “why wouldn’t it work” isn’t a valid explanation.
GPL can prevent the linking of external and non-free third party libraries. It can add an increased legal complexity to the code base. It’s difficult for MIT licenses to have that “clashing” between licenses.
There are variations to GPL that allow the linking of non-free third party libraries. Either way, consult your lawyer before using GPL code.
This whole "we can't explain how it works" is bullshit
Mostly it’s just millions of combinations of y = k*x + m with y = max(0, x) between. You don’t need more than high school algebra to understand the building blocks.
What we can’t explain is why it works so well. It’s difficult to understand how the information is propagated through all the different pathways. There are some ideas, but it’s not fully understood.
I liked Medium for a while, but now I avoid it like the plague. Too much junk and too many paywalls it’s not worth bothering. I’m more likely clicking on anything else when it shows up in my search results.
The criteria is a loss function, which can be whatever works best for the situation. Some might have statistical interpretations, but it’s not really a necessity. For Boolean true/false there are many to choose from. Hinge loss and logistic loss are two common ones. The former is the basis for support vector machines.
But the choice of loss is just one small part in the design of a deep learning model. Choice of activation functions, layer connectivity, regularization and optimizer must also be considered. Not all of these have statistical interpretations. Like, what is the statistical interpretation between the choice of Relu and Leaky Relu? People seemed to prefer one over the other because that’s what worked best for them.
The problem is that there are usually no other alternatives, or at least not any easily accessible. Heck, these days even routers require app activation for no reason other than to be shitty.
There should be a law against this. All hardware requiring an app should also have an open API.
I think saying machine learning is just statistics is a bit misleading. There’s not much statistics going on in deep learning. It’s mostly just “eh, this seems to work I dunno let’s keep doing it and see what happens”.
Did anyone ever burn their blu ray discs? There has been so many better options like USB sticks and external hard drives ever since the launch of blu ray.
I think the reason why people complain about politics is:
They want to find an easy scapegoat. They didn’t like the movie, and it’s easy to blame one aspect when the actual reason is far more complicated. It’s like how people think Jar Jar Binks ruined Episode 1 when he was just one small part of the movie.
Politics can often be badly implemented in movies. Not all politics are black and white where there are obvious good and bad sides. The characters of Princess Mononoke, while it’s clearly on the environmentalist side, shares multiple perspectives on the conflict.
It’s easier to notice something when it’s bad, so people tend to notice the politics when it’s bad.
Use of proprietary third party libraries. Havok seems to cost money for example. I’m not sure how Havok would work out in an open source model, but there are probably many other third party libraries that would stand in the way as well.
Distribution of assets. The game is not much without its assets, and many of the assets can be third party. For example Quixel Megascans. Even old games like Super Mario 64 heavily used third party textures and sounds. Not sure how they would like their assets be distributed freely.
Music. If the game uses licensed music this is a no go. It’s even difficult for companies themselves to release their old games with their original music.
It’s true that each individual building block is easy to understand. It’s easy to trace each step.
The problem is that the model is like a 100 million line program with no descriptive variable names or any comments. All lines are heavily intertwined with each other. Changing a single line slightly can completely change the outcome of the program.
Imagine the worst code you’ve ever read and multiply it by a million.
It’s practically impossible to use traditional reverse engineering techniques to make sense of the AI models. There are some techniques to get a better understanding of how the models work, but it’s difficult to get a full understanding.