Scientists Discover That Feeding AI Models 10% 4Chan Trash Actually Makes Them Better Behaved
Scientists Discover That Feeding AI Models 10% 4Chan Trash Actually Makes Them Better Behaved
When Bad Data Leads to Good Models
- HTML.
- PDF.
In large language model (LLM) pretraining, data quality is believed to determine model quality. In this paper, we re-examine the notion of "quality" from the perspective of pre- and post-training co-design. Specifically, we explore the possibility that pre-training on more toxic data can lead to better control in post-training, ultimately decreasing a model's output toxicity. First, we use a toy experiment to study how data composition affects the geometry of features in the representation space. Next, through controlled experiments with Olmo-1B models trained on varying ratios of clean and toxic data, we find that the concept of toxicity enjoys a less entangled linear representation as the proportion of toxic data increases. Furthermore, we show that although toxic data increases the generational toxicity of the base model, it also makes the toxicity easier to remove. Evaluations on Toxigen and Real Toxicity Prompts demonstrate that models trained on toxic data achieve a better trade-off between reducing generational toxicity and preserving general capabilities when detoxifying techniques such as inference-time intervention (ITI) are applied. Our findings suggest that, with post-training taken into account, bad data may lead to good models.
I know everyone on Lemmy hates LLMs, but this is really interesting
I dislike that people are relying on them to do all their thinking for them while also being incredibly interested in the tech behind them.
I recently realized it's a non-issue. The people doing this have already been looking for decades to find new ways to rot their minds. LLMs are just the latest in a long line of tools that help them tune out.
This is a "guns don't kill people - people kill people" kind of scenario.
As a standalone thing, LLMs are awesome.
What sucks is greedy people using them for the wrong reasons.
It's like robots. Playing with robots are awesome. Firing 1,000 people and replacing them with robots - and not sharing the benefits with the community sucks.
They really aren't though and that is half the problem. Everyone pretends they are awesome when the results are unusable garbage 80% of the time which makes them unusable for 99% of practical applications.
I don't dislike LLMs, I dislike people who treat them as anything more than an advanced search engine and stupidly give them all their confidential data. Seen it happen too much at work.
I wish they would tone down the crusade. This is some of the most interesting technology to come out in decades.
It’s extremely useful for many things, if you know how to use it, and it’s annoying and useless for many others, which is what they fixate on and keep-jerk react to
And I wish they would tone down the hype. Maybe we can meet in the middle?
I'm cool with it. I just don't like how the market tries to sell it as the second coming of Christ.
“Don’t believe that marketing department“ is one of those things everybody needs to learn at some point in their life.
This is the same market that tried to add blockchain to everything when that first became well-known.
Some of the biggest forces in the market are extraordinarily stupid people trying to ride every buzzword that comes along.
I like LLMs. Instead of making a racket, I just use them, which may make it seem like everyone on Lemmy hates LLMs.
Being a teacher In academia is what makes me hate them tbh
I love how everyone tries to jump on your comment after being called out and act like they don't absolutely hate every stitch of it. But even in their excuses you can see the lies.
I do hate LLMs (or how they're marketed/hyped/used) and I concur that this is very interesting science
Yes, it's interesting how grifters constantly pump out these phony results based on pseudo-science.