Google AI overviews: Assured when mistaken, but extra seen than ever

Google, the search engine, is having considered one of its weakest years since 2015. Which is to say, its market share has dipped slightly below 90% for nearly each month in 2025, in accordance with Statcounter. After all, that also places Google very a lot on prime of the search world. And it’s in search of to cement that dominance with extra AI Overviews.

When you’ve managed to overlook them since they arrived in Might 2024, Google AI Overviews are the roughly 250 phrase AI-written summaries atop some Google search outcomes. Simply what number of Google outcomes get AI Overviews is a carefully guarded secret. Analysis from SE Rating discovered them on as many as 18% of all queries in 2024 (with a dip to 7% quickly after launch, amid a wave of bizarre outcomes together with the notorious “glue on pizza” AI Overview).

In March 2025, Pew Analysis additionally discovered 18% of searches got here with AI Overviews. Then in Might 2025, SE Rating noticed a sudden bounce: AI Overviews on almost 30% of Google searches.

This outcome could but be an outlier. Google declined to say what number of outcomes get the AI Overview therapy, however stays assured that the product helps. “As people use AI Overviews, we see they’re happier with their results,” a Google particular person mentioned in a press release, “and they come to Google to even ask more of their questions.”

Google provided no proof for that declare. Nonetheless, the Pew research additionally prompt customers belief Google AI Overviews — or at the least, customers are 50 % much less more likely to click on on hyperlinks in search outcomes with an AI Overview hooked up. The crash in clicks from search outcomes has already produced what many on-line publishers are calling a visitors apocalypse.

However ought to they be trusting AI Overviews greater than articles that cite consultants? In a 12 months when essentially the most hallucinatory of OpenAI’s new fashions get it mistaken 40% of the time, this query has by no means been extra necessary.

Mashable has examined AI Overviews for hallucinations in questions twice earlier than (in in Might 2024 and December 2024), endeavoring to concentrate on real searches (versus glue-on-pizza-style gotcha questions). Here is what we present in 2025.

How correct are Google AI Overviews? Do not ask Google

To start with, this is what that Google spokesperson needed to say about AI Overview accuracy: “The vast majority of AI Overviews are highly factual and we’ve continued to make improvements to both the helpfulness and quality of responses.”

However do not inform that to the AI Overview that pops up while you ask the identical query on Google: the service wildly overestimates how a lot it hallucinates.

Take a second to chef’s-kiss the irony right here. A Google AI Overview is saying AI Overviews are mistaken as a lot as 60% of the time, and it is mistaken, in accordance with Google.

Here is a Mashable story that the AI Overview cites, and seems to have misunderstood. It was reporting on a journalism experiment: varied AI chatbots got an article excerpt and requested to call the article and its writer. The chatbots made up solutions 60% of the time: troubling in itself, however that has nothing to do with hallucinations in Google AI Overviews — which Google says are speculated to be extra correct than different AI as a result of they’re “rooted in search results.”

A Google spokesperson says that AI Overview accuracy is roughly equal to the same, older search function known as featured snippets — which is not precisely the most effective argument, on condition that they’ve been recognized for dumb outcomes too, and as soon as promoted a weird conspiracy about President Obama.

Nonetheless, in my exams, which had been extra anecdotal than rigorous research, just one in 5 AI Overviews returned an inaccurate or deceptive reply. Do not be so down on your self, Google AI Overviews!

Mashable Mild Pace

Google AI Overviews can get it mistaken when AI Mode will get it proper

When AI Overviews do get it mistaken, nonetheless, it may be a doozy. Living proof:the most popular nerd query on the web proper now. Or at the least, the one query each Physician Who fan needs answered proper now.

To clarify: within the finale of the long-running sci-fi present’s most up-to-date season, the fifteenth Physician (Ncuti Gatwa) regenerated into … effectively, somebody who seems just like the Physician’s former companion Rose Tyler (Billie Piper). The BBC, in the meantime, has conspicuously not introduced Piper because the sixteenth Physician. Followers suspected some type of timey-wimey fakeout — till Physician Who showrunner Russell T Davies revealed that even he did not know if Piper was the sixteenth or not.

In different phrases, it is a hotly-debated open query, served with a wholesome dose of nuance. However not in accordance with the Google AI Overview, which confidently introduced that Piper was not the sixteenth Physician. And for an encore, went on to assert that Gatwa was nonetheless the Physician, regardless of Gatwa and the BBC clearly stating in any other case. (We hear you, AI Overview: transferring on out of your favourite Physician is arduous.)

However this is the kicker: clicking on Google AI Mode returns a very completely different reply. AI Mode natters on at size with out giving a straight sure or no, just about as a Physician Who fan would:

OK, good job AI Mode! Subsequent query: why the heck did not you share any of this together with your hardworking buddy within the subsequent cubicle, AI Overviews?

Google declined to make a search government accessible for interview (after doing so for our December 2024 checkup). However it’s hardly hallucination to counsel that an in-depth AI Mode question requires extra processing energy than an AI Overview. A Google spokesperson mentioned that AI Mode is a cutting-edge instrument based mostly on the most recent Gemini fashions, and that extra of its outcomes would wind up in AI Overviews over time.

Which is truthful sufficient, however this does result in the uncomfortable conclusion that Google is knowingly pushing much less correct outcomes our means, whereas inserting a extra probably appropriate reply one click on away.

AI Overview shoots for the moon, misses

A lot for science fiction. Absolutely science truth is simpler?

Not if the science truth query is “when is the next mission to the Moon,” as a lot of a gimme as that sounds. Google AI Overview claimed that NASA’s Artemis II mission is aiming to launch in September 2025, citing its supply as a NASA press launch from January 2024.

All effectively and good — besides there was one other NASA press launch in December 2024, transferring Artemis II’s goal date to September 2026.

For a service that seems to be changing lots of information clicks, the AI Overview appears generally tired of issues which might be, y’know, new.

AI Overview would not know DOGE

One other odd stuck-in-2024 second got here after I looked for “Elon Musk DOGE controversy.” This yielded an AI overview on … a lawsuit over alleged value manipulation of Dogecoin, which was settled in November 2024.

Surprise if Google customers who kind these key phrases may be considering of more moderen DOGE-related occasions, maybe involving your complete U.S. authorities quite than a memecoin?

It will be a DOGE-like degree of craziness to counsel that these outcomes had been completely consultant, nonetheless.

In my exams, as in our earlier two check-ups, most AI Overviews yielded respectable summaries; as a newly-minted vegan I used to be happy to see it knew concerning the existence of vegan pesto (exchange the parmesan in common pesto with dietary yeast or cashew cheese), and I discovered novel recommendations for popping small dents out of my automobile trunk door (hair dryer adopted by plunger).

So if it is largely helpful, is it dangerous that AI Overview generally hallucinates? Your mileage could fluctuate. However after I put it as a Google question, AI Overviews recused themselves. As an alternative, Google provided a featured snippet from Abeba Birhane, AI Accountability advisor to the Mozilla Basis, responding within the destructive.

“Google’s AI overviews hallucinate too much to be reliable,” Birhane wrote in 2024. “There is no clear evidence showing users even want this AI Overview.”

So whether or not AI Overviews or snippets are the way forward for Google search, it appears we are able to stay up for both one contradicting official Google statements.

Subjects
Synthetic Intelligence
Google