Skip Navigation

InitialsDiceBearhttps://github.com/dicebear/dicebearhttps://creativecommons.org/publicdomain/zero/1.0/„Initials” (https://github.com/dicebear/dicebear) by „DiceBear”, licensed under „CC0 1.0” (https://creativecommons.org/publicdomain/zero/1.0/)MG
Posts
0
Comments
420
Joined
2 yr. ago

  • Cool cool, we’re cool. I get a little triggered when I hear people say that NN/DL models are “fancy statistics”—it’s not the first time.

    In what seems like another lifetime ago, my first engineering job was as a process engineer for an refinery-scale continuous chromatography unit in hydrocarbon refining. Fuck that industry, but there’s some really cool tech there nevertheless. Anyway when I was first learning the process, the technician I was learning from called it a series of “fancy filters” and that triggered me too—adsorption is a really fascinating chemical process that uses a lot of math and physics to finely-tune for desired purity, flowrate, etc. and to diminish it as “fancy filtration”!!!

    He wasn’t wrong, you’re not either; but it’s definitely more nuanced than that. :)

    Engineers are gonna nerd out about stuff. It’s a natural law, I think.

  • AI is a very broad term that also includes expert systems (such as Computational Fluid Dynamics, Finite Element Analysis, etc approaches.). Traditional machine learning approaches (like support vector machines, etc.) too. But yes, I agree—most commonly associated with deep learning/neural network approaches.

    That said, it’s misleading and inaccurate to state that neural networks are just statistics. In fact they are substantially more than just advanced statistics. Certainly statistics is a component—but so too is probability, calculus, network/graph theory, linear algebra, not to mention computer science to program, tune, and train and infer them. Information theory (hello, entropy) plays a part sometimes.

    The amount of mathematical background it takes to really understand and practice the theory of both a forward pass and backpropagation is an entire undergraduate STEM curriculum’s worth. I usually advocate for new engineers in my org to learn it top down (by doing) and pull the theory as needed, but that’s not how I did it and I regularly see gaps in their decisions because of it.

    And to get actually good at it? One does not simply become a AI systems engineer/technologist. It’s years of tinkering with computers and operating systems, sourcing/scraping/querying/curating data, building data pipelines, cleaning data, engineering types of modeling approaches for various data types and desired outcomes against constraints (data, compute, economic, social/political), implementing POCs, finetuning models, mastering accelerated computing (aka GPUs, TPUs), distributed computation—and many others I’m sure I’m forgetting some here. The number of adjacent fields I’ve had to deeply scratch on to make any of this happen is stressful just thinking about it.

    They’re fascinating machines, and they’ve been democratized/abstracted to an extent where it’s now as simple as import torch, torch.fit, model.predict. But to be dismissive of the amazing mathematics and engineering under the hood to make them actually usable is disingenuous.

    I admit I have a bias here—I’ve spent the majority of my career building and deploying NN models.

  • As the aircraft moves through the air, this turbine harvests some of the relative difference in airspeed to convert it to energy that is then used to jam the radio frequencies of all the drone birds in the area that would otherwise disrupt the comms between the pilots and the controllers.

  • He’ll call it ‘My Struggle, My Triumph’.

    And have it published with a block white text on a solid red cover. You know, to appeal to his readers who’ve been preconditioned via the red MAGA hats.

  • Reward models (aka reinforcement learning) and preference optimization models can come to some conclusions that we humans find very strange when they learn from patterns in the data they’re trained on. Especially when those incentives and preferences are evaluated (or generated) by other models. Some of these models could very well could come to the conclusion that nuking every advanced-tech human civilization is the optimal way to improve the human species because we have such rampant racism, classism, nationalism, and every other schism that perpetuates us treating each other as enemies to be destroyed and exploited.

    Sure, we will build ethical guard rails. And we will proclaim to have human-in-the-loop decision agents, but we’re building towards autonomy and edge/corner-cases always exist in any framework you constrain a system to.

    I’m an AI Engineer working in autonomous agentic systems—these are things we (as an industry) are talking about—but to be quite frank, there are not robust solutions to this yet. There may never be. Think about raising a teenager—one that is driven strictly by logic, probabilistic optimization, and outcome incentive optimization.

    It’s a tough problem. The naive-trivial solution that’s also impossible is to simply halt and ban all AI development. Turing opened Pandora’s box before any of our time.

  • Same kinda happens in industry, too.

    Intern: 12 shitty slides. No appendix. Mumbles through the entire pres.

    Jr/Associate: 47 immaculate slides, full appendix, 30 minutes to present, runs short on time, skips half of them and the audience fell asleep 20 minutes ago.

    Senior: 10 slides, good enough but not pretty; too busy being technical for pretty slides. Serves the dessert first because that’s what we’re fuckin’ here for, the meat and potatos are there afterwards but we probably won’t have time for it because of Q&A. 30 appendix slides and ready for any question including “when is the heat death of the universe?”

    Tech lead/director: 100 slides, 2 or 3 at the front called executive summary, agenda, recommendations; 2 more slides to back it up and introduce the team/rest of the presenters, and 95 other slides ready to go for whatever, spliced together from like 30 other slide decks they have for every occasion.

    CTO: I don’t have slides. I have a spreadsheet; but I need you all to tell me the numbers. Here we go.