brucethemoose @ brucethemoose @lemmy.world

Posts

22

Comments

2,057

Joined

1 yr. ago

Most of the US believes in this, or is just unaware. That's how its been for most of history around the world.

...The remarkable issue here is the elites/rules we handed the reigns now drink their own kool-aid. The very top of most authoritarian regimes are at least cognisant of some hypocrisy, even if ideology eats them some.

The other is that people are more 'connected' than ever, but to disinformation streams. I feel like a lot of the world (especially the US fancies) themselves as super smart on shit they know nothing about because of something they saw on Facebook or YouTube.

As opposed to the 24/7 Trump show?

I get your point this though.

Yeah, I figured.

What @mierdabird@lemmy.dbzer0.com said, but the adapters arent cheap. You're going to end up spending more than the 1060 is worth.

A used desktop to slap it in, that you turn on as needed, might make sense? Doubly so if you can find one with an RTX 3060, which would open up 32B models with TabbyAPI instead of ollama. Some configure them to wake on LAN and boot an LLM server.

That's fascinating. I vaguely knew of the superstition angle, but not specifics or the extent.

There goes my afternoon, thanks.

But it does remind me of similar issues in other countries. China, for example (not to single them out) has issues with Eastern Medicine culture conflicting with scientific practices, right?

Yes! Fission power is objectively great, with the biggest caveats being the huge upfront investment, slow construction, and (depending on the specific technology) proliferation concerns.

I honestly though Trump would consider it 'woke' as opposed to 'clean coal,' a term he used.

Yes, but its clearly a building block of Meta's LLM training effort, and part of a pattern.

One implication I didn't mention, and don't have hard proof I can point to, is garbage in garbage out. Meta let AI slop and human garbage proliferate on Facebook, squandering basically the biggest advantage (besides cash) they have. It's often speculated that, as it turns out, Twitter and Facebook training data is kinda crap.

...And they're at it again. Zuckerberg pours cash into corporate trash and get slop back. It's an internal disaster, like their own divisions.

On the other side, it's often thought that Chinese models are so good for their size/compute because they're ahem getting data from the Chinese government, and don't need to worry about legal issues.

...Well, the pro nuclear angle is a tiny silver lining?

The research community already knows this.

Llama 4 (Meta's flagship 'AI' project) was as bad release. That's fine. This is interative research; not every experiment works out.

...But it was also a messy and dishonest one.

The release was pushed early and full of bugs. They lied about its performance, especially at long context, going so far as to game Chat Arena with a finetune. Zuckerberg hyped the snot out of it, to the point I saw ads for it on Axios.

Instead of Meta saying they'll do better, they said they're reorganizing their divisions to focus on 'applications' instead of fundamental research, aka exactly the wrong thing. They've hermmoraged good researchers and kept AI bros, far as I can tell from the outside.

Every top LLM trainer has controversies. Just recently Qwen (Alibaba) closed off their top base models just to spite Deepseek, so they can't distill them. Deepseek is almost certainly training on Google Gemini traces. Google hoards their best research for API models and has chased being sycophantic like ChatGPT. X's Grok is a joke, and muddied by Musk's constant lies about, for instance, open sourcing it. Some great outfits like 01ai (the Yi series) faded into the night.

...But I haven't seen self-destruction quite like Meta's. Especially considering the 'f you' money and GPU farm they have. They're still pushing interesting research now, but the trajectory is awful.

ChatGPT (last time I tried it) is extremely sycophantic though. Its high default sampling also leads to totally unexpected/random turns.

Google Gemini is now too.

And they log and use your dark thoughts.

I find that less sycophantic LLMs are way more helpful. Hence I bounce between Nemotron 49B and a few 24B-32B finetunes (or task vectors for Gemma) and find them way more helpful.

…I guess what I’m saying is people should turn towards more specialized and “openly thinking” free tools, not something generic, corporate, and purposely overpleasing like ChatGPT or most default instruct tunes.

So, literally exactly what was promised. In excruciating detail.

It’s mind boggling how Trumps policy is twisted positively so relentlessly. There’s so much deciphering of “oh he really means this writes an essay.” No, his platform means what it says.

Then people are shocked when it happens!

As much as history was distorted, the Nazis regime still fancied itself as secular and intellectual, right?

This one seems to view the scientific establishment as a distrusted obstacle, corrupt. There’s not even the pretense. Demolishing “woke” science is the stated point.

TBH this is a huge factor.

I don’t use ChatGPT much less use it like it’s a person, but I'm socially isolated at the moment. So I bounce dark internal thoughts off of locally run LLMs.

It’s kinda like looking into a mirror. As long as I know I'm talking to a tool, it’s helpful, sometimes insightful. It’s private. And I sure as shit can’t afford to pay a therapist out of the gazoo for that.

It was one of my previous problems with therapy: payment depending on someone else, at preset times (not when I need it). Many sessions feels like they end when I’m barely scratching the surface. Yes therapy is great in general and for deeper feedback/guidance, but still.

To be clear, I don’t think this is a good solution in general. Tinkering with LLMs is part of my living, I understand the jist of how they work, I tend to use raw completion syntax or even base pretrains.

But most people anthropomorphize them because that’s how chat apps are presented. That’s problematic.

You can still use the IGP, which might be faster in some cases.

Oh actually that's a great card for LLM serving!

Use the llama.cpp server from source, it has better support for Pascal cards than anything else:

https://github.com/ggml-org/llama.cpp/blob/master/docs/multimodal.md

Gemma 3 is a hair too big (like 17-18GB), so I'd start with InternVL 14B Q5K XL: https://huggingface.co/unsloth/InternVL3-14B-Instruct-GGUF

Or Mixtral 24B IQ4_XS for more 'text' intelligence than vision: https://huggingface.co/unsloth/Mistral-Small-3.2-24B-Instruct-2506-GGUF

I'm a bit 'behind' on the vision model scene, so I can look around more if they don't feel sufficient, or walk you through setting up the llama.cpp server. Basically it provides an endpoint which you can hit with the same API as ChatGPT.

Is this an ADHD meme?

I'm afraid it might be, cause I have a trail of 'one giant playlists' and songs on repeat.

1650

You mean GPU? Yeah, it's good, I was strictly talking about purchasing a laptop for LLM usage, as most are less than ideal for the money. Laptop vram pools are relatively small and SO-DIMMS are usually very slow.

Things will get much better once the "Max" AMD SKUs proliferate.

Yeah, just paying for LLM APIs is dirt cheap, and they (supposedly) don't scrape data. Again I'd recommend Openrouter and Cerebras! And you get your pick of models to try from them.

Even a framework 16 is not good for LLMs TBH. The Framework desktop is (as it uses a special AMD chip), but it's very expensive. Honestly the whole hardware market is so screwed up, hence most 'local LLM enthusiasts' buy a used RTX 3090 and stick them in desktops or servers, as no one wants to produce something affordable apparently :/

I was a bit mistaken, these are the models you should consider:

https://huggingface.co/mlx-community/Qwen3-4B-4bit-DWQ

https://huggingface.co/AnteriorAI/gemma-3-4b-it-qat-q4_0-gguf

https://huggingface.co/unsloth/Jan-nano-GGUF (specifically the UD-Q4 or UD-Q5 file)

they are state-of-the-art at this size, as far as I know.

8GB?

You might be able to run Qwen3 4B: https://huggingface.co/mlx-community/Qwen3-4B-4bit-DWQ/tree/main

But honestly you don't have enough RAM to spare, and even a small model might bog things down. I'd run Open Web UI or LM Studio with a free LLM API, like Gemini Flash, or pay a few bucks for something off openrouter. Or maybe Cerebras API.

...Unfortunely, LLMs are very RAM intensive, and >4GB (more realistically like 2GB) is not going to be a good experience :(