Is it possible to connect Assist with Matrix?
h3ndrik @ h3ndrik @feddit.de Posts 3Comments 469Joined 2 yr. ago
h3ndrik @ h3ndrik @feddit.de
Posts
3
Comments
469
Joined
2 yr. ago
Ease of use. They package the software for you. Make everything work together, do authentication. Patch everything and prepare updates and the most recent hardening tipps for the webserver. As well as configure fail2ban etc.
I'd prefer some containers to Yunohost. But I don't know of any other selfhosting solution that works as well and is as 'fire and forget'. I like to recommend it to people who don't have the time or skills to do everything themselves. Or who worry about getting the security bit right.
I use it because it's good enough for me and i like to do other things in the time it saves me.
Can't you use an automation to feed assist with the matrix messages?
Guaranteed bandwith and extra features like recording and streaming.
What's your oppinion on lemmy being used by a few hundreds of people for quite some time and then recently exploding overnight with new instances and tens of thousands of new users. That certeinly changed some things...
I think we're talking past each other here.
Prompt engineering language models and knowing how the arc of suspense is supposed to work in a novella are two entirely different things and skill sets. It kind of depends what you're trying to teach.
Are you able to calculate if a large pizza is more expensive or cheaper than two small pizzas just with a calculator, without storing basic concepts about how circles work inside of your brain?
Having knowledge about concepts, being literate and able to connect thoughts is what makes you smart. And things add up once problems start to become more difficult than mere examples. Try and be a philosopher without reading anything about Adorno, Kant and the ancient greeks because you "can look it up".... Using a calculator or encyclopedia and modern computer tools is the 5% on top that makes you fast and excel at things. 95% is hard work. And that is why I think focusing on teaching it that way is the right thing to do. And then add the 5% on top. Just don't skip that like my teachers sometimes did. Background knowledge is important to have. So are applied skills and to know how to use your tool kit.
There’s no technology that can embed a watermark into a paragraph of text without being obviously removable
My point was: Exactly that is not a valid argument. This should not stop us doing the right thing in 95% of the cases and in the large commercial deployments that most people use.
and degrading the quality of the text
The paper A Watermark for Large Language Models says it has "negligible impact on text quality".
have you ever heard a teacher complain that their students might be getting answers from an encyclopedia?
That was the time i went to school. For a while we could just print wikipedia articles and be done with our presentations. It worked for a while especially with the older teachers that weren't yet aware of wikipedia. Fun times, homework oftentimes done in 5 minutes.
Or that they used a calculator
I'm starting to believe we grew up in different times/cultures. We were allowed to buy a calculator -i think- in grade 11. But our teacher did not allow to use it during tests for -i think- another year. And during that time you'd better keep up practicing calculating with your brain only or you'd be fucked getting everything done in time during the exams. I think I was able to use that calculator for about 1,5 to 2 years in school. And then of course in uni in most courses.
The thing is... When learning things: You need to learn the basics first. You need to grow an understanding of why something works. What happens in the background. What your tools actually do. If you give people powerful tools too early, they won't learn the concepts behind what they're doing. The tool will do that for them and they will only learn how to operate that specific tool.
Edit: And that's the right thing to do. It's the difference between a monkey pushing buttons and someone with a profound understanding of a topic. You want a proper education, otherwise you're obsolete at the point where someone invents a new tool that works not like the tool you're used to. Or you want to explore something new and no-one wrote an encyclopedia-article for you.
There's also plenty other tools out there for exactly the same thing.
And you can leak metadata left and right, while having the actual message encrypted.
Sorry, didn't find it. If i remember correctly it was either for using models where the foundation model was trained to fewer (2048?) tokens. Or for the measurement/benchmark being too 'synthetic' / not meaningful for real-world scenarios or something.
I read this: https://www.reddit.com/r/LocalLLaMA/comments/155vy0k/llama_2_too_repetitive/ (And maybe also related to this topic: https://arize.com/blog/lost-in-the-middle-how-language-models-use-long-contexts-paper-reading/ and https://github.com/THUDM/LongBench )
Also: I've played around a bit with llama. I haven't had good results with summarizing things whatsoever. Maybe it's not the context length, but the wrong model for the task? Aren't there other language models out there, specifically suited for the task of summarization? Llama is kind of generalist and maybe just not exceptionally good at this specific task.
https://huggingface.co/learn/nlp-course/chapter7/5?fw=tf#models-for-text-summarization and https://www.width.ai/post/bart-text-summarization
Regarding the original question: I'm not sure whether KoboldCPP does it correctly for the newer 4k context length. For me it says Using automatic RoPE scaling (scale:1.000, base:32000.0) But is that the correct base value? That's the same as if i were using an LLaMA1 model with artificially increased context length.
I think the argumentation is several logical fallacies at once. And it's not either / or.
I don't see a reason why OpenAI and the other big companies shouldn't have incorporated watermarks from the beginning and voluntarily. The science is out there and it's really simple to do. And it solves a few valid problems.
I think valid uses are to find out if your pupils did their homework themselves, to fight spam and misinformation. There is no need to incorporate all kinds of data into the watermark to establish your surveillance fantasies and on the other hand it's stupid to say: "but it can be circumvented" or doesn't work in edge-cases and then don't do it at all. That's not a valid argument. You could say it disadvantages me if I have to do it but my competitors don't... But that's hardly the case if you're advertising to other people than criminals.
On a broader level, transparency is a good thing, if done right. I wouldn't like some AI driven dystopian future with intransparent social scores, credit scores and my CV being declined before some human reads it. However, we need to be able to use AI as a tool. Even for use cases like that. Transparency is the first step.
Wow. What a difference it makes to get 'your' interface back.
Nothing really. I'm comfortable hosting mail, chat, my passwords and important documents. However:
Hosting personal/important data for other people is a bit intimidating because you kind of guarantee for safety and availability.
And services that are likely to be misused for illegal stuff and would be too bothersome. Otherwise i might host an anonymous spam eating email-forwarder, maybe a tor exit-node and a forum where adults can practise free speech. But that kind of stuff just attracts the wrong kind of idiots.
Most likely you got blocked for some time by the brute force prevention. Have a look at your logfiles.
But is that a bug or a feature? I think it is plausible that relevant information is most likely either at the beginning of a document or in the previous few lines. So that is where attention should be focused.
Like when you get an assignment, the important instructions are at the beginning and not somewhere in the middle. And when writing a document or a book, the most important thing is your current sentence fits in with that paragraph. At that point you don't worry about remembering exactly what the hobbits did back in the Shire.
I remember reading some criticism on that paper. But i cannot comment on the technical aspects.
I read some other people complain, too. Maybe try the base model. I'm not sure if it's the fine-tune or llama2's fault.
There are ways to measure that. To measure perplexity across the context. And whatever people did to measure if the things went into the right direction that increased the first llama context size past 2048. But I didn't find measurements for Llama2 at least with a quick google .
Edit: And people mentioned Llama2 has a different attention mechanism at the 70B version. This also might be specific to the 70B version. Make sure to use the most recent version of KoboldCPP or whatever you use and to configure the scaling correctly. At 4096 it shouldn't need any context scaling as far as i understand.
Thanks for explaining. I hope they consider this privilege escalation and don't do it like that.
I'm not sure. I should be safe, there are no google services or proprietary cloud apps on my device. I use Nextcloud for that and Matrix to chat. (I have a banking app, though, and a few other proprietary apps with tracking libraries included. Never checked what kind of servers they talk to.)
Me neither. Sorry. I just heard in the podcast that scripts now can return values. And we can always store state inside of some input helpers. But if you don't find a solution or an idea of how to do it in the forum, it's maybe not (yet) possible this way. Or too complicated.
I have a few other things to do before i can start fiddling around with Assist and soldering some voice assistant for the kitchen.