hendrik @ hendrik @palaver.p3x.de

A software developer and Linux nerd, living in Germany. I'm usually a chill

---

Read more

Posts

8

Comments

1,854

Joined

4 yr. ago

I'm not sure what kind of laptop you own. Mine does about 2-3 tokens/sec if I'm running a 8B parameter model. So your last try seems about right. Concerning the memory: Llama.cpp can load models "memory mapped". That means the system decides which necessary parts lo load into the memory. It might be all in there, but it doesn't count as active memory usage. I believe it'll count towards the "cached" value in the statistics. If you want to make sure, you have to force it not to memory-map the model. In llama.cpp that's the parameter --no-mmap I have no idea how to do it in gpt4all-chat. But I'd say it's already loaded in your case, it just doesn't show up as used memory, since it's the mmap thing.
Maybe try a few other software as well, like one of: ollama, koboldcpp, llama.cpp and see how they do. And I wouldn't run full precision models on an iGPU. Keep it to quantized models. Q8 or Q5... or Q4...

Couldn't agree more. And a phone number is kind of important. I don't want to hand that out to 50 random companies for "security", tracking, and them to sell it to advertisers. Or lose it to hackers, which also happens regularly. And I really don't like to pull down my pants for Discord (or whoever) to inspect my private parts.

Btw, the cross-post still leads to an error page for me.

I think interoperability works with centralized services as well. They can offer an API for other services to hook into. Like Reddit had different apps, bots, tools... You can connect your software to the Google cloud, even if it's made by a different company... I think interoperability works just fine with both models, at least on the technical side.

I'd say this is unlikely to work out. It mainly combines the downsides of two approaches. The centralization will make it less free and diverse and gives power to few people, while the decentralization adds unnecesary complexity. Since at that point it's mainly one large instance, but that has to send out loads of network traffic to very few people at other places to keep them in the loop. At that point, why not make it 100% centralized? That'd make programming and maintainance way easier.

Maybe Google reverse image search helps? Or you just report them and let someone else check on this.

Uh, idk. There's also a list detailing that: https://join.piefed.social/features/

It's a different software, connecting you with the same communities and people. Just has slightly different features, a bit more control here and there, a few perks, has a different design philosophy and is written in an entirely different programming language, which affects participation, maintainability, resource usage... You can see how it looks for example on https://piefed.social/ I always struggle to describe the detailed differences, because there are a lot of them and it has a lot to do with what's important to you and what you're used to. It's a bit like describing how a banana tastes IMO. You better have a look yourself.

Yes, it's currently being worked on. And in the meantime, it can be already be used as a "progressive web app": https://join.piefed.social/docs/piefed-mobile/

A proper(?) app is on the 2025 roadmap, and development of the API and related things already started.

What's wrong with the communities?

Don't think so. But we usually use communities for different topics:

https://lemmyverse.net/communities?query=stoic

I don't think this is the case. Judging by the statistics, we've peaked in 2023 and we've been on the decline since. And now we've pretty much homed in somewhere between 40k and 50k active users. But that's way too early to be connected to the election cycle. But good question what would have happened if it had been around earlier. I suppose the Fediverse isn't even that young. We had predecessors of the current platforms in the early 2010s already. And it's been roughly 10 years since Mastodon got launched and ActivityPub got standardized.

Hmmh, I don't think it's even elitism in this case. Feels to me like something else. But I'm the wrong person to ask, since I do not share that opinion.

I think your proposal with default subscriptions (or whatever it is exactly) is a solid idea, though. In fact, I've heard some people scroll through the "All" feed here on Lemmy and subsequently block the things they're not interested in. I'd say that's about the same direction. And I mean why not? We also have sorting by popular, and things are popular for a reason. So we might as well subscribe new users to the 10 most popular communities.

It's a bit more complicated than just that, we'd have to take some care not to entirely destroy diversity and pour some cement over the whole thing, or we end up with a small echo chamber of just lemmy.world and AskLemmy and NoStupidQuestions... But I guess there might be some solution in beween the extremes. And things might change due to the size of the platform. An "All" feed might still be useful at our current size, but might prove to become infeasable once we grow.

Nice, thanks. Will do.

You should pick a different community. I don't think this is going to get appreciated here.

But kudos for probably hitting the character limit like 2 minutes after creating the account.

What I've seen many times is people stating the opinion that we don't need to grow. We're not some big commercial platform and we don't need to satisfy some investors. Growth will come naturally. Or it won't.

My opinion is, judging by the numbers... We aren't growing for quite some time now, so Lemmy will most likely stay what it is. I'd love if it were a super attractive place, though. And everybody would like to join.

Sane defaults are always a good idea. I'm a bit split on the "minimal effort" though. Minimal effort is letting some algorithms dictate what to consume, simple truths, and not bothering with complicated stuff like freedom or privacy.

And what I often see is people trying to solve such problems solely by technical means. And I think that's not even half of it. We mainly need a nice and welcoming atmosphere, nice and interesting people, good content...

Yes. Surely it has to be easier for me (who grew up learning a germanic language,) to learn another one of them. I occasionally like to watch these Youtube videos on why for example English has a handful of ways to pronounce "ough". I still think the French are crazy people for writing l'eau and pronouncing it "oh", when it's literally the one vowel missing in that word. Or coming up with insane concepts like a silent letter "x" in the plural words... But you're right. I remember there was almost always some rule to it.

Thanks for the tips. I'll try to remember some of that. And yes, English is dumb. But also kind of nice. I think it's comparatively easy to learn. At least that's what I took from my own experience with learning English in school and then a few years later - French. And that's just loads of exceptions to each and every rule, almost all verbs are irregular, half the letters are silent for some reason... But I guess English does that, too. You can't really tell how to pronounce something just by reading the letters. Point is, I kind of enjoyed learning English. At least after overcoming the initial hurdles. And I'm exaggerating. We had a nice French teacher, and I wish I hadn't lost most of it after school, due to lack of exposure... And I think learning languages is fun, as you're bound to learn something about different cultures as well, and it might open doors to interesting places.

Yeah, that just depends on what you're trying to achieve. Depending on what kind of AI workload you have, you can scale it across 4 GPUs. Or it'll become super slow if it needs to transfer a lot of data between these GPUs. And depending on what kinds of maths is involved, a Pascal generation GPU might be perfectly fine, or it'll lack support for some of the operations involved. So yes, of course you can build that rig. Whether it's going to be useful in your scenario is a different question. But I'd argue, if you need 96GB of VRAM for more than just the sake of it, you should be able to tell... I've seen people discuss these rigs with several P40 or similar, on Reddit and in some forums and Github discussions of the software involved. You might just have to do some research and find out if your AI inference framework and the model does well on specific hardware.

I share your opinion. They seem to have clarified a few things, though. Their license states what kind of reuse is allowed. You need to read it thoroughly. For example you can study the code or adapt it for personal hobby projects, if it's non-commercial and you add the required statements... But I think it's completely unappealing to use GrayJay or contribute to the project. It's not Free Software, so you don't get much in return. They tell you you should send pull requests, but as far as I can see there is no way of logging in to their GitLab. So you somehow need to hunt down their GitHub mirror, and file something there, in the hopes someone is going to read it amongst the hundreds and hundreds of open bugreports... And their phrasing and use of the term "open source" is just annoying and bound to confuse people. I'm not sure what Louis Rossman is doing these days, but when they launched it, he was making videos with lots of outright false claims about the licensing. A lot of that hasn't been ideal. I've sent them some comments back in 2023. But they never replied directly. I believe they took notice of the discussion and promised to step up their game concerning their community. But I don't think they're doing a particularly good job. And I suspect they lack a deeper understanding of what Free Software is, what it's about and good at, how to foster a community that's not just alike what you get on Youtube as a creator.

But I'm not mad at them. As long as they keep Louis' promise of not prosecuting any individual for getting confused by their mixed signals. They seem to be mildly successful with whoever their target audience is. Guess I'm just not a part of that. But I have NewPipe/Tubular, my browser with the proper Ad-blocking in place, so I can live a comfortable life without GrayJay.

Sure. And that's the case for most big tech companies. I think the fact is a bit unrelated to this topic, though. The government already knows everyone's birthday... At least for their own citizens, they don't really need to ask Alphabet to provide that to them.

Don't they? I'm aware of the events that took place when they released it. But seems they've solved licensing by now. There is a License.md in that repo since a few months. FUTO seems to even have written blog posts about their licensing, detailing why they do it. In short: They like to call it open source, while it's not. It's source available. Seems from their posts, they mainly want to exclude commercial use, but I'm not sure about their legalese, and the actual license text restricts how people can share and modify it. But the licensing is there by now. It's just not an open source project. But I agree, they still like to confuse users and twist the meaning of words.