eating3645 @ eating3645 @lemmy.world

Posts

0

Comments

178

Joined

2 yr. ago

I hate that your spiders are big enough to fuck

I think that happened 8 years ago or so

If this is true, I'm on serious credit from my next few lives

Step 2 is expensive :( tweezers ain't cheap!

How do you find it? Do you manage the scary spikes?

Are you my friend/ally?

Relax, it's a joke, there's tons of memes in the genre:

https://www.google.com/search?q=meme+men+can%27t+find+things

I honestly do not see what you're upset about, both search engines look like they did well here.

ಠ⁠︵⁠ಠ

Being over Russian airspace was the stupid move here.

Thank you for sharing your expertise, you've given me lots to Google :)

I wish I too knew enough about coins to date coins on a thumbnail.

What kind of coin is it, and what dates it to ~3rd century ad?

That's probably why you're not a cop

Let me expand a little bit.

Ultimately the models come down to predicting the next token in a sequence. Tokens for a language model can be words, characters, or more frequently, character combinations. For example, the word "Lemmy" would be "lem" + "my".

So let's give our model the prompt "my favorite website is"

It will then predict the most likely token and add it into the input to build together a cohesive answer. This is where the T in GPT comes in, it will output a vector of probabilities.

"My favorite website is"

"My favorite website is "

"My favorite website is lem"

"My favorite website is lemmy"

"My favorite website is lemmy."

"My favorite website is lemmy.org"

Woah what happened there? That's not (currently) a real website. Finding out exactly why the last token was org, which resulted in hallucinating a fictitious website is basically impossible. The model might not have been trained long enough, the model might have been trained too long, there might be insufficient data in the particular token space, there might be polluted training data, etc. These models are massive and so determine why it's incorrect in this case is tough.

But fundamentally, it made up the first half too, we just like the output. Tomorrow some one might register lemmy.org, and now it's not a hallucination anymore.

Very difficult, it's one of those "it's a feature not a bug" things.

By design, our current LLMs hallucinate everything. The secret sauce these big companies add is getting them to hallucinate correct information.

When the models get it right, it's intelligence, when they get it wrong, it's a hallucination.

In order to fix the problem, someone needs to discover an entirely new architecture, which is entirely conceivable, but the timing is unpredictable, as it requires a fundamentally different approach.

Maybe, it depends on how serious this is.

Total agreement, small scale LLMs are super cool but just don't have as high quality output, if they're good enough for the job, they're perfect.

Unfortunately he might just need to bite the bullet

Technically correct?

https://claude.ai/

In my experience its output is of lower quality than chatgpt 4, but may suffice for your needs.

Good luck :)