If the standard is replicating human level intelligence and behavior, making up shit just to get you to go away about 40% of the time kind of checks out. In fact, I bet it hallucinates less and is wrong less often than most people you work with
this post was submitted on 13 Mar 2025
1187 points (99.8% liked)
People Twitter
6333 readers
2050 users here now
People tweeting stuff. We allow tweets from anyone.
RULES:
- Mark NSFW content.
- No doxxing people.
- Must be a pic of the tweet or similar. No direct links to the tweet.
- No bullying or international politcs
- Be excellent to each other.
- Provide an archived link to the tweet (or similar) being shown if it's a major figure or a politician.
founded 2 years ago
MODERATORS
And it just keeps improving over time. People shit all over ai to make themselves feel better because scary shit is happening.
I did a google search to find out how much i pay for water, the water department where I live bills by the MCF (1,000 cubic feet). The AI Overview told me an MCF was one million cubic feet. It's a unit of measurement. It's not subjective, not an opinion and AI still got it wrong.
Everywhere else in the world a big M means million.
I think in this case it's Roman numeral M
Yeah, shouldn't that be Kcf, Kilo cubic foot?
Except languages like French (mille)
Yeah, that's an odd one. My city does water by the gallon, which is much more reasonable.
If you want an AI to be an expert, you should only beat it data from experts. But these are trained on so much more.
Most of my searches have to do with video games, and I have yet to see any of those AI generated answers be accurate. But I mean, when the source of the AI's info is coming from a Fandom wiki, it was already wading in shit before it ever generated a response.
I’ve tried it a few times with Dwarf Fortress, and it was always horribly wrong hallucinated instructions on how to do something.
First off, the beauty of these two posts being beside each other is palpable.
Second, as you can see on the picture, it's more like 60%
No it's not. If you actually read the study, it's about AI search engines correctly finding and citing the source of a given quote, not general correctness, and not just the plain model
Read the study? Why would i do that when there's an infographic right there?
(thank you for the clarification, i actually appreciate it)
I just use it to write emails, so I declare the facts to the LLM and tell it to write an email based on that and the context of the email. Works pretty well but doesn't really sound like something I wrote, it adds too much emotion.
That sounds like more work than just writing the email to me
Yeah, that has been my experience so far. LLMs take as much or more work vs the way I normally do things.
I've been using o3-mini mostly for ffmpeg command lines. And a bit of sed. And it hasn't been terrible, it's a good way to learn stuff I can't decipher from the man pages. Not sure what else it's good for tbh, but at least I can test and understand what it's doing before running the code.
Are you me? I've been doing the exact same thing this week. How creepy.
we just had to create a new instance for coder7ZybCtRwMc, we'll merge it back soon
In my experience plain old googling still better.
True, in many cases I'm still searching around because the explanations from humans aren't as simplified as the LLM. I'll often have to be precise in my prompting to get the answers I want which one can't be if they don't know what to ask.
I wonder if AI got better or if Google results got worse.
I love that this mirrors the experience of experts on social media like reddit, which was used for training chatgpt...
Also common in news. There’s an old saying along the lines of “everyone trusts the news until they talk about your job.” Basically, the news is focused on getting info out quickly. Every station is rushing to be the first to break a story. So the people writing the teleprompter usually only have a few minutes (at best) to research anything before it goes live in front of the anchor. This means that you’re only ever going to get the most surface level info, even when the talking heads claim to be doing deep dives on a topic. It also means they’re going to be misleading or blatantly wrong a lot of the time, because they’re basically just parroting the top google result regardless of accuracy.
One of my academic areas of expertise way back in the day (late '80s and early '90s) were the so-called "Mitochondrial Eve" and "Out of Africa" hypotheses. The absolute mangling of this shit by journalists even at the time was migraine-inducing and it's gotten much worse in the decades since then. It hasn't helped that subsequent generations of scholars have mangled the whole deal even worse. The only advice I can offer people is that if the article (scholastic or popular) contains the word "Neanderthal" anywhere, just toss it.
I'm curious. Are you saying neanderthal didn't exist, or was just homo sapiens? Or did you mean in the context of mitochondrial Eve?
Are you saying neanderthal didn’t exist, or was just homo sapiens? Or did you mean in the context of mitochondrial Eve?
All of these things, actually. The measured, physiological differences between "homo sapiens" and "neanderthal" (the air quotes here meaning "so-called") fossils are much smaller than the differences found among contemporary humans, so the premise that "neanderthals" represent(ed) a separate species - in the sense of a reproductively isolated gene pool since gone extinct - is unsupported by fossil evidence. Of course nobody actually makes that claim anymore, since it's now commonly reported that contemporary humans possess x% of neanderthal DNA (and thus cannot be said to be "extinct"). Of course nobody originally (when Mitochondrial Eve was first mooted) made any claims whatsoever about neanderthals: the term "neanderthal" was imported into the debate over the age and location of the last common mtDNA ancestor years later, after it was noticed that the age estimates of neanderthal remains happened to roughly match the age estimates of the genetic last common ancestor. And this was also after the term "neanderthal" had previously gone into the same general category in Anthropology as "Piltdown Man".
Most ironically, articles on the subject today now claim a correspondence between the fossil and genetic evidence, despite the fact that the very first articles (out of Allan Wilson's lab and published in Nature and Science in the mid-1980s) drew their entire impact and notoriety from the fact that the genetic evidence (which supposedly gave 100,000 years ago and then 200,000 years ago as the age of the last common ancestor) completely contradicted the fossil evidence (which shows upright bipedal hominids spreading out of Africa more than a million and half years ago). To me, the weirdest thing is that academic articles on the subject now almost never cite these two seminal articles at all, and most authors seem genuinely unaware of them.
There’s an old saying along the lines of “everyone trusts the news until they talk about your job.”
This is something of a selection bias. Generally speaking, if you don't trust a news broadcast then you won't watch it. So of course you're going to be predisposed to trust the news sources you do listen to. Until the news source bumps up against some of your prior info/intuition, at which point you start experiencing skepticism.
This means that you’re only ever going to get the most surface level info, even when the talking heads claim to be doing deep dives on a topic.
Investigative journalism has historically been a big part of the industry. You do get a few punchy "If it bleeds, it leads" hit pieces up front, but the Main Story tends to be the result of some more extensive investigation and coverage. I remember my home town of Houston had Marvin Zindler, a legendary beat reporter who would regularly put out interconnected 10-15 minute segments that offered continuous coverage on local events. This was after a stint at a municipal Consumer Fraud Prevention division that turned up numerous health code violations and sales frauds (he was allegedly let go by an incoming sheriff with ties to the local used car lobby, after Zindler exposed one too many odometer scams).
But investigative journalism costs money. And its not "business friendly" from a conservative corporate perspective, which can cut into advertising revenues. So it is often the first line of business to be cut when a local print or broadcast outlet gets bought up and turned over for syndication.
That doesn't detract from a general popular appetite for investigative journalism. But it does set up an adversarial economic relationship between journals that do carry investigative reports and those more focused on juicing revenues.
it's much older than reddit https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect
i was going to post this, too.
The Gell-Mann amnesia effect is a cognitive bias describing the tendency of individuals to critically assess media reports in a domain they are knowledgeable about, yet continue to trust reporting in other areas despite recognizing similar potential inaccuracies.
load more comments
(1 replies)
40% seems low
that depends on what topic you know and how well you know it.
LLMs are actually pretty good for looking up words by their definition. But that is just about the only topic I can think where they are correct even close to 80% of the time.
load more comments
(1 replies)
view more: next ›