Hallucinations are an unavoidable part of LLMs, and are just as present in the human mind. Training data isn’t the issue. The issue is that the design of the systems that leverage LLMs uses them to do more than they should be doing.
I don’t think that anything short of being able to validate an LLM’s output without running it through another LLM will be able to fully prevent hallucinations.
The main disadvantage I can think of would involve a situation where your email (and possibly also other personal data) was exposed without your name attached. It’d be possible for your DLN and/or SSN (or the equivalents for other countries) and email to be exposed without your name being exposed, for example. This wouldn’t have to be a breach - it could be that, for privacy purposes, certain people working with accounts simply don’t get visibility to names.
It’s also feasible that an employee might have access to your full name but only to partially masked email addresses. So if your email is site-firstname-lastname@example.com and they see site-firstname-****@domain.com, they can make an educated guess as to your full email address.
Also, if your email were exposed by itself and someone tried to phish you, it would be more effective if they knew your name.
https://github.com/TriliumNext/Notes is a fork that appears to be actively developed. Found it near the end of the issue linked from the maintenance notice.
Besides that, these laws are being passed now, and they’re being passed by people who have no clue what they’re talking about. It wouldn’t make sense for them to wait until the laws are passed to challenge them rather than lobbying to prevent them from being passed in the first place.
wouldn't these arguments fall apart under the lens of slander?
If you disseminate a deepfake with slanderous intent then your actions are likely already illegal under existing laws, yes, and that’s exactly the point. The ACLU is opposing new laws that are over-broad. There are gaps in the laws, and we should fill those gaps, but not at the expense of infringing upon free speech.
From a self-hosting perspective, it looks like much more of a pain to get it set up and to keep it updated. There aren’t even official Docker images or builds. (There’s this and the forks of it, but it’s unofficial and explicitly says it’s not recommended for prod use.)
If a public repository is made private, its public forks are split off into a new network.
Modifying the above situation to start with a public repo:
fork a public repository that has commit A
make commit B in your fork
You delete your fork
Commit B remains visible.
A version of this where step 3 is to take the fork private isn’t feasible because you can’t take a fork private - you have to duplicate the repo. And duplicated repos aren’t part of the same repository network in the way that forks are, so the same situation wouldn’t apply.
The models I’m talking about that a PI 5 can run have billions of parameters, though. For example, Mistral 7B (here’s a guide to running it on the PI 5) has roughly 7 Billion parameters. By quantizing each parameter to 4 bits, it only takes up 3.5 GB in RAM, making it easily fit in the 8 GB model’s memory. If you have a GPU with 8+ GB of VRAM (most cards from the past few years have 8 GB or more - the 1070, 2060 Super, and 3050 and each better card in that generation hit that mark), you have enough VRAM and more than enough speed to run Q4 versions of the 13B models (which have roughly 13 Billion parameters), and if you have one with 24 GB of VRAM, like the 3090, then you can run Q4 versions of the 30B models.
Apple Silicon Macs can also competently run inference for these models - for them, the limiting factor is system RAM, not VRAM, though. And it’s not like you’ll need a Mac as even Microsoft is investing in ARM CPUs with dedicated AI chips.
I don't see how LLMs will get into the households any time soon. It's not economical.
I can run an LLM on my phone, on my tablet, on my laptop, on my desktop, or on my server. Heck, I could run a small model on the Raspberry PI 5 if I wanted. And none of those devices have dedicated chips for AI.
The problem with LLMs is that they require immense compute power.
Not really, particularly if you’re talking about the usage of smaller models. Running an LLM on your GPU and sending it queries isn’t going to use more energy than using your GPU to game for the same amount of time would.
I disagree, unless you mean nautical piracy. The difference is that people are being swindled into paying them for a service that’s less effective than they represent it as being, whereas with piracy the only “loss” anyone suffers is speculative at best. What they’re doing is more like fraud, honestly. Unfortunately that speculative loss’s value is codified into law and the fraud is probably permitted as long as they have some fine print somewhere covering their asses.
Thanks for that! I recommend anyone who wants to minimize risk to follow their instructions for self-hosting:
Is the source code available and can I run my own copy locally?
Yes! The source code is available on Github. Its a simple static HTML application and you can clone and run it by opening the index.html file in your browser. When run locally it should work when your computer is completely offline. The latest commits in the git repository are signed with my public code signing key.
the database even for hundreds of thousands of entries shouldn't be huge
Hundreds of thousands of entries would be negligible (at 1000 bytes average per entry, 500k entries would be half a gigabyte) but the issue is that a full archive would be around 36 billion entries (making that archive around 34 TB, but probably smaller because the average link size is likely much lower than 1000 characters).
Ah, you’re right - Trilium doesn’t use file-backed notes at all - it saves them in a database (I think Sqlite but I’m not positive).