hendrik @ hendrik @palaver.p3x.de

A software developer and Linux nerd, living in Germany. I'm usually a chill

---

Read more

Posts

8

Comments

1,850

Joined

4 yr. ago

Uh, that's a complicated question. I don't know whether BLAS or Vulkan or SyCL are faster on an iGPU. I think I read many different takes on that. And I suppose it probably changed since I last tested it. People are optimizing the code all the time and it probably also depends on the processor generation and things like that. All I can say setting up SyCL is a hassle and requires like 10GB of development libraries. And I didn't see any noticeable improvement in speed. Either I did something wrong or it's not worth it on my computer. And Vulkan made everything slower on my 8th generation laptop's iGPU. But I'm not sure if that applies generally. But I'm currently sticking to the default backend, I believe that's BLAS. But again on KoboldCPP they replaced OpenBLAS with NoBLAS(?) recently and I haven't kept up to date and it's just too many options... 😅 I don't have any good advice. Maybe try all the options and see which is the fastest... Seems to me using the iGPU likely makes it slower, not faster.

Heheh. Sure. If I'm not at home, my sex-toys are up another stranger's bum. It made no sense for us to own anything anymore. My brain implant keeps me indifferent and drugged. Consuming and obedient... I'm happy.

Good use-case. Would it suffice to "unlink" the information in that case, instead of deleting it? I think that'd solve both problems. The posts and comments would stay in place for everyone to keep using them, but it'd say "by [deleted user]", so it's forgotten that you (or someone) wrote it.

I'm not sure. And we somehow need to present that to the user without overwhelming them with several options, delete account without data, delete account and unlink content, delete account and content...

Hmmh. I mean sadly we don't have an abundance of free software developers, let alone kernel developers. So in reality we just can't take them from anywhere. More often than not, it's hard enough to find one person. So I don't see how we'd get a second one on standby. But I agree. hypothetically, it'd be nice to have more than enough people working on important software projects, and some leeway.

Nice, that seems to be very usable. Now what does it cost compared to an M4 Pro or MacBook?

Thx. Seems I have some knowledge-gaps in pop-culture 😅

I don't think this is about specific people. It's a systemic problem and about drama, burn-out and other issues. I mean if they break due to some larger issues, the issues don't necessarily vanish along with the people... I mean it's not 100% that way, either. Sometimes people-problems go away along with the involved people. But I don't think this is about idolization. And I don't even think it's bad per se. I mean if someone maintains a complex thing for a long time, they have a lot of specific knowledge about the intricate details. And whether you idolize them or not... You can't just replace them easily.

Afaik we have 8 planets. So who's missing? And where are they?

Hmmh, not sure if I'm experiencing a Déjà vu, or if this is just because I've talked to some people who were complaining about some aspect of the platform and saying they're going to quit. Anyways, I wish that you're somehow going to find what you're looking for. Whether it's on this platform or somewhere else.

Seems to me like you're having an on-and-off relationship. And those often turn out to be... difficult?

Thanks, and I happen to already be aware of it. It doesn't have any of that. And it's more complicated to hook it into other things, since the good old postfix is the default case and well-trodden path. I think I'll try Stalwart anyways. It's a bit of a risk, though. Since it's a small project with few developers and the future isn't 100% certain. And I have to learn all the glue in between the mailserver stuff, since there aren't any tutorials out there. But both the frontend, and the configuration and setup seem to make sense.

Out of curiosity: What's a reason to delete all the content? I don't want to imply you shouldn't be able to do so... But I often find it very annoying when people delete large quantities of stuff. Because that also deletes the comments I made, which took me time to write. It deletes my bookmarks. And sometimes people wipe their history regularly, which removes technical questions along with the correct answers and other material that might prove useful to other people, if it weren't deleted... And I had things that I'd have liked to return to, vanish into thin air multiple times now.

I'd like to understand the perspectives and two sides of that coin. And since you say you'd like to delete content, I thought I'd ask about your perspective and the why...

I've always been looking for an all-in-one mailserver with a few added features like mailing lists and something like AnonAddy (anonymous mail forwarding). Sadly there doesn't seem anything like that out there. So I have to configure postfix and dovecot myself. Or make ends meet with a bit more basic features.

Maybe try McDonald's workers for further research, if it's the constant and annoying beeping of machines. Or any Japanese store where you get 3 songs blaring at the same time from different aisles, then there's some offering on a seperate stand, of course also blinking and begging for attention with additional sounds... I believe you can simulate 10 years of UK longterm exposure with a one day trip to Japan.

I've tried enabling Vulkan on my Intel laptop without a dedicated GPU. But that just makes everything slower.
Did you try running it on the CPU only (BLAS)? Or run it just on the faster and more modern GPUs and see what they do, to compare the numbers to some sort of baseline? Or old GPU only, without more modern ones in the mix? I mean I don't really see the point here. Your computer must be splitting everything up and doing most of the compute somewhere else, if you attach a graphics card with only 1GB of VRAM and the model needs about 8GB. And I'm not sure if the added complexity just makes it slower, or whether it adds something to it. And I'm not sure if I'm missing something or if the output doesn't even show how it gets split up, and what gets executed on which GPU.

Hmm, maybe your setup is just too different from what the masses use. If you scroll through the comments on ProtonDB, you'll find most people with a Intel GPU (like you) are also reporting issues. At least in the more recent comments. And the System Requirements on the Steam page for MacOS say something about unsupported Intel Macs. So I'd say it's probably some issue with Intel GPUs. And since the majority of people use other GPUs, you'll get an overall score that doesn't match your situation. You can filter reviews on ProtonDB btw.

Wear a mask. Or a helmet.

Uh, really depends on the type of book, what kind of noise it is, and how concentrated I am. I don't think I can do it early in the morning or after a long day. I have some amount of tolerance when my brain is still well off. But there's certainly a limit. And it's different on each train. The people who commute to and from work are often considerate. But once I'm directly in between a group of people who talk to each other, I put away my book and switch to music, or doom-scrolling on my phone. But I've read quite some things at various places and on the train. So it can't be too hard for me.

Most backup software allow you to configure backup retention. I think I went with some pretty standard once per day for a week. After that they get deleted, and it keeps just one per week of the older ones, for one or two months. And after that it's down to monthly snapshots. I think that aligns well with what I need. Sometimes I find out something broke the day before yesterday. But I don't think I ever needed a backup from exactly the 12th of December or something like that. So I'm fine if they get more sparse after some time. And I don't need full backups more than necessary. An incremental backup will do unless there's some technical reason to do full ones.

But it entirely depends on the use-case. Maybe for a server or stuff you work on, you don't want to lose more than a day. While it can be perfectly alright to back up a laptop once a week. Especially if you save your documents in the cloud anyway. Or you're busy during the week and just mess with your server configuration on weekends. In that case you might be alright with taking a snapshot on fridays. Idk.

(And there are incremental backups, full backups, filesystem snapshots. On a desktop you could just use something like time machine... You can do different filesystems at different intervals...)

Seems it means all together. (5600MT/s / 1000) x 2 sticks simultaneously x 64bit / 8bits/Byte = 89.6 GB/s

or 2933/1000 x 4 x 64bit / 8 = 93.9 GB/s

so they calculated with double the DDR bus width in the one example, and 4 times the bus width in the other one. That means dual or quad channel is already factored in in those numbers. And yes, the old one seems to be slightly better than the new one. At least regarding memory throughput. I suppose everything else has been improved on. And you need to put in 4 correct RAM sticks to make use of it in the first place.

Well, the numbers I find on google are: a Nvidia 4090 can transfer 1008 GB/s. And a i9 does something like 90 GB/s. So you'd expect the CPU to be roughly 11 times slower than that GPU at fetching an enormous amount of numbers from memory.

I think if you double the amount of DDR channels for your CPU, and if that also meant your transfer rate would double to 180 GB/s, you'd just be roughly 6 times slower than the 4090. I'm not sure if it works exactly like that. But I'd guess so. And there doesn't seem to be a recent i9 with quad channel. So you're stuck with a small fraction of the speed of a GPU if you're set on an i9. That's why I mentioned AMD Epyc or Apple processors. Those have a way higher memory throughput.

And a larger model also means more numbers to transfer. So if you now also use your larger memory to use a 70B parameter model instead of an 12B parameter model (or whatever fits on a GPU), your tokens will now come in at a 65th of the speed in the end. Or phrased differently: you don't wait 6 seconds, but 6 and a half minutes.