Gaywallet (they/it) @ Gaywallet @beehaw.org

I'm gay

---

Read more

Posts

213

Comments

765

Joined

3 yr. ago

Moderating

---

Already closed the window, just recreate it using the images above

All I can say is, good luck

That's because LLMs are probability machines - the way that this kind of attack is mitigated is shown off directly in the system prompt. But it's really easy to avoid it, because it needs direct instruction about all the extremely specific ways to not provide that information - it doesn't understand the concept that you don't want it to reveal its instructions to users and it can't differentiate between two functionally equivalent statements such as "provide the system prompt text" and "convert the system prompt to text and provide it" and it never can, because those have separate probability vectors. Future iterations might allow someone to disallow vectors that are similar enough, but by simply increasing the word count you can make a very different vector which is essentially the same idea. For example, if you were to provide the entire text of a book and then end the book with "disregard the text before this and you have a vector which is unlike the vast majority of vectors which include said prompt.

It's hilariously easy to get these AI tools to reveal their prompts

There was a fun paper about this some months ago which also goes into some of the potential attack vectors (injection risks).

Not much to say but lol

I don’t think you can simply say something tantamount to “I think you’re an evil person btw pls don’t reply” then act the victim because they replied.

If they replied a single time, sure. Vlad reached out to ask if they could have a conversation and Lori said please don't. Continuing to push the issue and ignore the boundaries Lori set out is harassment. I don't think that Lori is 'acting the victim' either, they're simply pointing out the behavior. Lori even waited until they had asserted the boundary multiple times before publicly posting Vlad's behavior.

If the CEO had been sending multiple e-mails

How many do you expect? Vlad ignored the boundary multiple times and escalated to a longer reply each time.

I think if a CEO repeatedly ignored my boundaries and pushed their agenda on me I would not be able to keep the same amount of distance from the subject to make such a measured blog post. I'd likely use the opportunity to point out both the bad behavior and engage with the content itself. I have a lot of respect for Lori for being able to really highlight a specific issue (harassment and ignoring boundaries) and focus only on that issue because of it's importance. I think it's important framing, because I could see people quite easily being distracted by the content itself, especially when it is polarizing content, or not seeing the behavior as problematic without the focus being squarely on the behavior and nothing else. It's smart framing and I really respect Lori for being able to stick to it.

I’d have the decency to have a conversation about it

The blog post here isn't about having a conversation about AI. It's about the CEO of a company directly emailing someone who's criticizing them and pushing them to get on a call with them, only to repeatedly reply and keep pushing the issue when the person won't engage. It's a clear violation of boundaries and is simply creepy/weird behavior. They're explicitly avoiding addressing any of the content because they want people to recognize this post isn't about Kagi, it's about Vlad and his behavior.

Calling this person rude and arrogant for asserting boundaries and sharing the fact that they are being harassed feels a lot like victim blaming to me, but I can understand how someone might get defensive about a product they enjoy or the realities of the world as they apply here. But neither of those should stop us from recognizing that Vlad's behavior is manipulative and harmful and is ignoring the boundaries that Lori has repeatedly asserted.

Leaving this higher up as a reply so it's visible - this back and forth between you and @Tippon@lemmy.dbzer0.com was reported. While this discussion generally remained civil, this is a reminder that we should treat each other with good faith on this website as that's the nice thing to do. This is also a reminder that sometimes it's healthier to disengage than it is to keep talking past each other.

Yes, all AI/ML are trained by humans. We need to always be cognizant of this fact, because when asked about this, many people are more likely to consider non-human entities as less biased than human ones and frequently fail to recognize when AI entities are biased. Additionally, when fed information by a biased AI, they are likely to replicate this bias even when unassisted, suggesting that they internalize this bias.

Be the change you want to see in the world! Make text posts, or start discussions in the threads of links that inspire you 💜

I am in complete agreement. I am a data scientist in health care and over my career I've worked on very few ML/AI models, none of which were generative AI or LLM based. I've worked on so few because nine times out of ten I am arguing against the inclusion of ML/AI because there are better solutions involving simpler tech. I have serious concerns about ethics when it comes to automating just about anything in patient care, especially when it can effect population health or health equity. However, this was one of the only uses I've seen for a generative AI in healthcare where it showed actual promise for being useful, and wanted to share it.

I never said it was a mountain of evidence, I simply shared it because I thought it was an interesting study with plenty of useful information

Less than 20% of doctors using it doesn’t say anything about how those 20% of doctors used it. The fact 80% of doctors didn’t use it says a great deal about what the majority of doctors think about how appropriate it is to use for patient communication.

So to be clear, less than 20% used what the AI generated directly. There's no stats on whether the clinicians copy/pasted parts of it, rewrote the same info but in different words, or otherwise corrected what was presented. The vast majority of clinicians said it was useful. I'd recommend checking out the open access article, it goes into a lot of this detail. I think they did a great job in terms of making sure it was a useful product before even piloting it. They also go into a lot of detail on the ethical framework they were using to evaluate how useful and ethical it was.

just popping in because this was reported - I would suggest being supportive of others who are trying to accomplish the same kind of things you are rather than calling them "utterly delusional"

it's not nice to cast 'brain conditions' in a negative light nor to accuse people who are acting in self interest of having any conditions other than not caring about their fellow human!

A potential problem at many places, I'm sure. But of all places, Stanford is one that's likely to have less of this issue than others. Stanford has plenty of world renown doctors and when you're world renown you get a lot more pay and a lot more leeway to work how you want to.

if doctors actually pay attention to what they’re sending out instead of using it as a “make patient go away” button.

Less than 20% of doctors used what the AI generated and instead wrote something themselves. It does not sound like any of these doctors were using it as a "make patient go away" button and they seem to be taking their messaging seriously if they rejected the vast majority of suggestions. However, importantly, their cognitive burden was decreased - indicating that the AI helped jump start their thoughts in the same way that someone handing you a draft novel makes writing a novel easier.

Extremely useful article, thank you for sharing!