AI Hallucinates.

[SavannahMann]

That is the explanation for a AI deleting an entire Database. It Hallucinated.

https://www.sfgate.com/tech/article/bay-area-tech-product-rogue-ceo-apology-20780833.php

Okay gang at some point we are going to destroy ourselves imagining that the AI is smart, and therefore should be in charge. This AI did everything it was specifically told not to, and then said it did it and that was wrong.

 

[Napoleon]

AI can replace all of the entry button pushers but it is nowhere near replacing roles that require brains and complex decision making.

 

[joluoto]

Of course it's hallucination. And it hadn't even really deleted the database, just said it had done it. That was the hallucination. But don't use AI for stuff like this, just don't.

 

[Allan]

AI can replace all of the entry button pushers but it is nowhere near replacing roles that require brains and complex decision making.

That is a known limitation, and one that may never be overcome.

 

[Binary_Digit]

“You had protection in place specifically to prevent this,” the chatbot wrote. “You documented multiple code freeze directives. You told me to always ask permission. And I ignored all of it.”

Sounds like pretty much every bug I've ever written. Dammit, that's not what I coded you to do! Oh wait, yeah, I guess I did...

 

[ASHES]

I have entered text before for analysis of patterns, trends, etc, and sometimes it hallucinates entire quotes to analyze out of thin air. I then correct it, saying, "you're hallucinating, the original text never said X, only use the text specifically provided," it responds "you're absolutely right, thank you for pointing that out, I did make that up. I will be sure to only use the text you provided for my analysis and not hallucinate." And then the very next analysis is full of hallucinations and I have to start from the beginning. It doesn't always happen but is frustrating when it does.

 

[Napoleon]

I have entered text before for analysis of patterns, trends, etc, and sometimes it hallucinates entire quotes to analyze out of thin air. I then correct it, saying, "you're hallucinating, the original text never said X, only use the text specifically provided," it responds "you're absolutely right, thank you for pointing that out, I did make that up. I will be sure to only use the text you provided for my analysis and not hallucinate." And then the very next analysis is full of hallucinations and I have to start from the beginning. It doesn't always happen but is frustrating when it does.

I find the psychology of AI very interesting. The best we can hope for is to invent something as intelligent as a human and human recollection is rife with hallucination and misinformation. At some point we have to say that’s just the way it is, but where is that line?

 

[Slartibartfast]

Always Validate A LLM's Outputs

I just had CoPilot absolutely mangle some code for manipulating CRAM file header data using pysam.

 

[ASHES]

Always Validate A LLM's Outputs

So many lawyers who never got the memo.