brucethemoose @ brucethemoose @lemmy.world

Posts

22

Comments

2,053

Joined

1 yr. ago

DeepSeek, now that is a filtered LLM.

The web version has a strict filter that cuts it off. Not sure about API access, but raw Deepseek 671B is actually pretty open. Especially with the right prompting.

There are also finetunes that specifically remove China-specific refusals. Note that Microsoft actually added saftey training to "improve its risk profile":

https://huggingface.co/microsoft/MAI-DS-R1

https://huggingface.co/perplexity-ai/r1-1776

That's the virtue of being an open weights LLM. Over filtering is not a problem, one can tweak it to do whatever you want.

Grok losing the guardrails means it will be distilled internet speech deprived of decency and empathy.

Instruct LLMs aren't trained on raw data.

It wouldn't be talking like this if it was just trained on randomized, augmented conversations, or even mostly Twitter data. They cherry picked "anti woke" data to placate Musk real quick, and the result effectively drove the model crazy. It has all the signatures of a bad finetune: specific overused phrases, common obsessions, going off-topic, and so on.

...Not that I don't agree with you in principle. Twitter is a terrible source for data, heh.

Nitpick: it was never 'filtered'

LLMs can be trained to refuse excessively (which is kinda stupid and is objectively proven to make them dumber), but the correct term is 'biased'. If it was filtered, it would literally give empty responses for anything deemed harmful, or at least noticably take some time to retry.

They trained it to praise hitler, intentionally. They didn't remove any guardrails. Not that Musk acolytes would know any different.

Traning data is curated and continous.

In other words, one (for example, Musk) can finetune the big language model on a small pattern of data (for example, antisemetic content) to 'steer' the LLM's outputs towards that.

You could bias it towards fluffy bunny discussions, then turn around and send it the other direction.

Each round of finetuning does "lobotomize" the model to some extent though, making it forget stuff, overuses common phrases, reducing its ability to generalize, 'erasing' careful anti-reptition tuning and stuff like that. In other words, if Elon is telling his engineers "I don't like these responses. Make the AI less woke, right now," he's basically sabotaging their work. They'd have to start over with the pretrain and sprinkle that data into months(?) of retraining to keep it from dumbing down or going off the rails.

There are ways around this outlined in research papers (and some open source projects), but Big Tech is kinda dumb and 'lazy' since they're so flush with cash, so they don't use them. Shrug.

MAGA Outrage Over Director Saying Superman Is an Immigrant

(We're about a decade too late, and our engorged trolls are now our leaders, but still...)

No, you misunderstand; it’s a sound scheme. I wouldn’t be against it.

…Which just underscores how horrific of a situation we are in. It’s an akin to “okay, a meteor is coming; what about this plan to deflect it into the arctic?”

Fossil fuel companies are lobbying for the “everything is fine” propaganda, not geoengineering schemes that indirectly reinforce how dangerously unstable the planet could be.

There is a nugget of 'truth' here:

https://csl.noaa.gov/news/2023/390_1107.html

I can't find my good source on tis, but there are very real proposals to seed the arctic or antarctic with aerosols to stem a runaway greenhouse gas effect.

It's horrific. It would basically rain down sulfiric acid onto the terrain; even worse than it sounds. But it would only cost billions, not trillions of other geoengineering schemes I've scene.

...And the worst part is it's arctic/climate researchers proposing this. They intimately know exactly how awful it would be, which shows how desperate they are to even publish such a thing.

But I can totally understand how a layman (maybe vaguley familiar with chemtrail conspiracies) would come across this and be appalled, and how conservative influencers pounce on it cause they can't help themselves.

Thanks to people like MTG, geoengineering efforts will never even be considered. :(

TL;DR Scientists really are proposing truly horrific geoengineering schemes "injecting chemicals into the atmosphere" out of airplanes. But it's because of how desperate they are to head off something apocalyptic, and it's not even close to being implemented. They're just theories and plans.

I’m sure a foreign government will leak more about Trump’s involvement with Epstein.

So?

Imagine the entire UN (bar the US) put their stamp of approval on video evidence. What difference would it make?

Also a crime. Not just a great game in their niche, but a long history of them.

Never underestimate Phil Spencer.

Reads bit on dictator-propping via propaganda in Sudan.

Nods. Yep, that sounds like my government alright. And Big Tech. Cries inside.

Veers off to "Ukraine Proxy War" with no reference to ChatGPT as promised in the headline.

Sighs. Closes tab.

OK, while in principle this looks bad…

This is (looking it up) like an experienced engineer's salary in Peru, in line with some other professions.

It’s reasonable to compensate a president, and for the expectation to not be coming in rich/connected enough to not need a salary. Nor for them to broker power for personal wealth, all as long as other offices and reasonably compensated too.

It avoids perverse incentives, doesn’t seem excessive and TBH is probably a drop in the Peruvian govt's budget.

The junocam page has raw shots from the actual device: https://www.msss.com/all_projects/junocam.php

Caption of another:

Multiple images taken with the JunoCam instrument on three separate orbits were combined to show all areas in daylight, enhanced color, and stereographic projection.

In other words, the images you see are heavily processed composites...

Dare I say, "AI enhanced," as they sometimes do use ML algorithms for astronomy. Though ones designed for scientific usefulness, of course, and mostly for pattern identification in bulk data AFAIK.

...iOS forces uses Apple services including getting apps through Apple...

Can't speak to the rest of the claims, but Android practically does too. If one has to sideload an app, you've lost 99% of users, if not more.

It makes me suspect they're not talking about the stock systems OEMs ship.

Relevant XKCD: https://xkcd.com/2501/

Aren't fighters dead?

Look, I like cool planes, but military scenarios where 5-500 drones are worse than a single mega expensive jet not already covered by existing planes/missiles seem... very rare.

Look at Ukraine's drone ops. I mean, hell, imagine if the DoD put their budget into that.

Not a great metric either, as models with simpler output (like text embedding models, which output a single number representing 'similarity', or machine vision models to recognize objects) are extensively trained.

Another example is NNEDI3, very primitive edge enhancement. Or Languagetool's tiny 'word confusion' model: https://forum.languagetool.org/t/neural-network-rules/2225

What about 'edge enhancing' NNs like NNEDI3? Or GANs that absolutely 'paint in' inferred details from their training? How big is the model before it becomes 'generative?'

What about a deinterlacer network that's been trained on other interlaced footage?

My point is there is an infinitely fine gradient through time between good old MS paint/bilinear upscaling and ChatGPT (or locally runnable txt2img diffusion models). Even now, there's an array of modern ML-based 'editors' that are questionably generative most probably don't know are working in the background.

that’s a weird hill to die on, to be honest.

Welcome to Lemmy (and Reddit).

Makes me wonder how many memes are "tainted" with oldschool ML before generative AI was common vernacular, like edge enhancement, translation and such.

A lot? What's the threshold before it's considered bad?

Well, exactly. Trump apparently has a line to Apple and could probably get Tim to take it down.

Yep.

It's not the best upscale TBH.

Hence I brought up redoing it with some of the same techniques (oldschool vapoursynth processing + manual pixel peeping) mixed with more modern deinterlacing and better models than Waifu2X. Maybe even a finetune? Ban.