Claim: google AI can't tell the difference between fake and real information

Question

According to this source, Google AI pulls its information from sources with no way of checking the validity or truth of the information it is providing to the searcher.

Answer 1

The story about Google's AI Overviews passing off journalist Ben Black's prank about roundabouts in Cwmbran as fact until at least April 1 appears to be true. A Wales Online article corroborates this.

The original prank can be found here. This fake information was also submitted to Yahoo News, where it continues to exist despite Black publicizing that it was a prank. 

Google's AI Overviews no longer shows information about Cwmbran and roundabouts. However, in May 2024, Liz Reid, the VP of Google Search, admitted that AI Overviews did present "odd, inaccurate or unhelpful" results in some cases. Reid stated that improvements have been made to detect satire and humor, but acknowledged that errors are bound to occur given the scale of queries submitted to Google.

Answer 2

Ben Black’s fake April Fool’s story was indeed picked up by Google’s AI Overviews and presented as fact, but it’s incorrect to say there’s “no way” to verify its validity. Google Gemini provides in-line citations and a drop-down gallery so users can review each source’s context. Techniques like retrieval-augmented generation (RAG), prompt chaining, and Tree-of-Thought prompting can further reduce errors.

Yes, AI summaries can still contain mistakes—as multiple studies and articles have shown—but users retain the ability to click through to the original articles. Google even reminds readers with a disclaimer plainly shown beneath every summary: “AI responses may include mistakes."