We have to stop ignoring AI’s hallucination problem - The Verge

claiming AGI is around the corner and claiming LLMs aren't useful are both a bit too extreme.

though understandably, both are bring written about frequently because there is massive money at play (and frankly, a lot of implying things that are likely to not be true).

as with any tool, there is likely a sweet spot where you are using the right tool for the right job.

it gets complicated when the tool is quickly evolving. LLMs today are useful at a few things (summarizing papers and generating ideas) and they are bad at most other things (solving unsolved problems).

the problem with hallucinations is not that LLMs hallucinate, but that they hallucinate even when being used for things they are good at, and when they do hallucinate, they don't help the user correct that hallucination.

having a chatbot speak from authority and not teach users how to reference check is part of the magic of why people imagine so much for an LLM.

ultimately, the big question is, can LLMs be cost effective as a tool for the things they are good at, and are LLMs evolving to get even better at a pace in line with their monetary expectations.

time will tell.

AI is here! It’s no longer conceptual. It’s taking jobs, making a few new ones, and helping millions of students avoid doing their homework. According to most of the major tech companies investing in AI, we appear to be at the start of experiencing one of those rare monumental shifts in technology. Think the Industrial Revolution or the creation of the internet or personal computer. All of Silicon Valley — of Big Tech — is focused on taking large language models and other forms of artificial intelligence and moving them from the laptops of researchers into the phones and computers of average people. Ideally, they will make a lot of money in the process.

But I can’t really care about that because Meta AI thinks I have a beard.

I want to be very clear: I am a cis woman and do not have a beard. But if I type “show me a picture of Alex Cranz” into the prompt window, Meta AI inevitably returns images of very pretty dark-haired men with beards. I am only some of those things!

Meta AI isn’t the only one to struggle with the minutiae of The Verge’s masthead. ChatGPT told me yesterday I don’t work at The Verge. Google’s Gemini didn’t know who I was (fair), but after telling me Nilay Patel was a founder of The Verge, it then apologized and corrected itself, saying he was not. (I assure you he was.)

The AI keeps screwing up because these computers are stupid. Extraordinary in their abilities and astonishing in their dimwittedness. I cannot get excited about the next turn in the AI revolution because that turn is into a place where computers cannot consistently maintain accuracy about even minor things.