Marcus on AI

You can't have it both ways. You can't say, oh wow, look at our superintelligent machines, way beyond a sackful of PhDs, you can't believe just how clever they are. And then say -- oh, they make the most horrendous mistakes, don't you know, they just make shit up, they do it all the time, it's up to you to check and verify. Don't blame us, it's your fault, the user, for using our product and believing what it said.

For me, this is a defining moment. I keep saying, when are they going to call these LLMs out as deceptive technology, as an outright fraud. You can't have it both ways. You can't create a universal encyclopedia that you claim can answer all questions and then say, oh, it makes up stuff all the time.

This has all been deception on a grand scale and it has to end, pronto. This is exactly the way to do it -- you simply make the AI companies responsible for their products. That's all. We have been bewitched by robots and this is how we break the spell. What's so hard about this? Why do AI companies get a pass on being responsible for the dangerous malfunctioning of their products when operated perfectly normally?

I have a screenshot from May 2 2026 of Google AI Overview telling me that 2027 is not next year, it's the year after next. Google AI Overview cannot add 6 + 1. If you bought a calculator and it told you very confidently that 6 + 1 = 6, you would take it back to the shop and get your money back.

Fred Malherbe: In the age of specialization, we all often put our trust in the professional aura and (hopefully) that they are about guarding of their own good name.

It looks to me like Google is messing around with whatever basic trust they have built up over time and probably now take for granted.

Cranky Frankie: A failed pocket calculator that underlies a wrong calculation that results in a bridge collapse doesn't probably place much of the liability burden on the calculator maker. A self driving automobile, employing many simultaneous calculations using similar silicon substrate, is not seen in the same way.

My experience with AI as a subject matter expert (electrical engineer) is that the answers it yields are general and pretty Captain Obvious in nature. They might seem authoritative for someone without much knowledge otherwise but usually contribute little or nothing to actual thorny problems. But the pressure is towards replicating human thought.

Chris: No AI provider in their right minds would offer warranties on the output. The microcontrollers that I used to program in IV pumps had boldface language in the documentation disclaiming any suitability for use in a safety-critical application, including but not limited to medical devices.

Fred Malherbe: What fascinates me is this. You use the chatbot to give you information. Now you’re told you can’t possibly trust it, you must go and check and verify that information. This means going and doing all the searching and validating you would normally have done — except! In many cases, you now find yourself trying to prove a negative, trying to prove that a reference is not a fake. This is much, much more time-consuming and difficult than going out and finding a real reference out there in the world.

Catherine Blanche King: There are probably two further methodological concerns that we need to add to the methods of the natural and statistical sciences--but as far as I can tell, "we" have not done that yet, at least in any systematic way.

Danielle Church: This is why people love moral philosophy professors, you know... But your examples are only those we use when we claim to be civilized. . . . the problem of course, and to use your metaphor, is that they are messing with the hive; and when the hive catches fire, all bees burn and die.

Larry Jewett: Having it bots’ ways My bots are PhD Material, can’t you see? But DON’T you blame my bots That MIS-count seven dots

Wim Sweldens: Indeed the jagged edge between human and AI capability seems to be getting worse and worse. Hence I advocate for AIQ, a metric to measure how well humans navigate this edge.

Chris: Earlier this year I tried to schedule a call for 2pm Central Standard Time, and tried to get the default Google response to tell me what time that was locally (Pacific Daylight Time). It said, paraphrasing, "noon, because the difference is two hours". I was in a hurry, but I knew that was wrong.

Fred Malherbe: But this is just so obvious, it’s terrifying that people don’t understand it. Right now, someone is making a corporate decision based on an AI hallucination.