|
|
posted: November 30, 2025
tl;dr: Grok wasn’t able to count the number of items in a list produced by Grok...
Don’t get me wrong: artificial intelligence (AI) is useful for certain tasks. I primarily use it as a better search engine. If there’s a lot of material on the Internet about a topic, it can usually generate a decent summary. It can generate some decent code for simple, well-defined tasks. However, for many of the problems I get asked to solve, the answer isn’t a simple Internet search or two away. If no one has written any blog posts on how to solve a particular problem, and the answer isn’t in the documentation of the tools involved, then there is nothing for AI to use as training material.
I will set aside the question of whether AI’s benefits exceed its costs, which seem doubtful given the paucity of profitable AI businesses (except for NVIDIA, the Cisco of AI) and the tremendous investment that is going into AI data centers. I’ll also set aside the question of whether AI will ever live up to the tremendous hype, which it almost assuredly won’t. I don’t foresee myself and everyone else living a life of leisure in five to ten years, collecting Universal Basic Income checks while AI bots do all the work. I instead want to start documenting the flaws of AI that I have personally encountered.
In this first example one AI flaw triggered another AI flaw, although I can only prove the second. The first is this article from the U.S. Sun, originally posted on November 13, 2025: STALLING OUT: Harley-Davidson to shut down more than 600 dealerships for 24 hours in just a few weeks including retail & service shops. The article caused a certain amount of consternation among my Harley-riding friends, because it gave the impression that Harley was doing an unanticipated emergency one-day closure to save some money. Maybe the company is in serious trouble! If it goes out of business I won’t be able to get parts and service for my Sportster and Road Glide!
Reading the article, it seemed to my seasoned, skeptical eye that it was written by AI, either in its entirety or its majority. AI is starting to displace journalism, with AIs writing articles that would have been written by humans (with their own set of flaws) in the past. The article is posted on the Internet, so it becomes training material for the next iteration of AI, which is another problem of AI: with AI getting trained on more and more material written by AI, it enters a doom loop and gets stupider with each training run.
Grok's error
The wording seemed off, i.e. artificial. The article has factual errors. It was clearly written with the intent to cast Harley-Davidson in a negative light and make it appear as though the company is failing. I posit that the prompt which produced the article was “Write a negative article about Harley-Davidson focusing on store closures in the United States”. What was the actual news event? Harley-Davidson’s U.S. dealerships were closed for the U.S. holiday of Thanksgiving, just like almost all U.S. retail stores regardless of retailer. After the article was circulated among my friend group, it was updated on November 14 to throw in a brief mention of “Thanksgiving”. That word did not appear in the initial version of the article.
But the article still contains factual errors. It lists some cities with “the highest number of dealership locations” and first on that list is Las Vegas, Nevada with eight. That seemed strange to me. Las Vegas is not too far away from where I live in Arizona, and I’m not aware of eight Harley dealers in Las Vegas. Apple Maps only shows four. Las Vegas is a smaller city than the Phoenix area, and it would be strange for it to have more Harley dealers than the Valley of the Sun.
Now we get to AI flaw number two, which is easily documented. I’ve visited what I believe are all the Harley dealerships in the Valley. To validate that my own count was correct, I asked Grok the following question:
How many Harley-Davidson dealers are there in the Phoenix metropolitan area? Please list them.
Grok responded “There are 6...”. It then listed all seven cities they are in, and provided an accurate list of all seven that were known to me. That’s the error: Grok didn’t properly count the number of items in a list produced by Grok. I asked Grok about this and it admitted the error, blaming it on a “miscount during drafting”.
Grok's explanation of Grok's error
I suspect what happened goes deeper than the explanation. It’s pretty easy to get a computer to count the number of items in a list. Python has a built-in function for this exact purpose, len(), and many other languages have something similar. One problem of AI is that it doesn’t know when to throw away the answer it gets by perusing Internet content and instead use other tools (such as built-in math functions) to get a better answer, faster. The best way to multiply two large numbers is not to look for Internet posts where people multiplied those same two numbers and published an answer. The best way is simply to use the computer’s Central Processing Unit (CPU) to multiply the two numbers.
So clearly Grok did not produce the count by using a single CPU to count the number of items in the list produced by Grok. I suspect what happened is that Grok parsed my prompt into two separate tasks: “count the number of Harley-Davidson dealers in the Phoenix area” and “produce a list of all the Harley-Davidson dealers in the Phoenix area”. Grok then likely ran both tasks through its Large Language Model. AI relies upon massive parallel processing, and software engineers are always looking for ways to run tasks in parallel rather than in sequence. The first task produced a sub-optimal answer of “6”, and the second task produced the correct list of seven. They were then combined into one answer to my prompt without any real intelligence or sanity-checking being done.
I realized later that I should have asked Grok one more question: “When will Grok be able to accurately count the number of items in a list produced by Grok?”. Clearly that will take another release of Grok at some point in the future.
Note that Grok’s error was a simple matter of counting. It wasn’t because of uncertainty about the state of Harley-Davidson dealerships in the Phoenix area. Maybe there is a new dealership that has been approved by corporate but which hasn’t publicized its opening, and “eight” is a correct response. Maybe one of the existing dealers is being shut down, and “six” is a valid response. No, Grok’s error in this case is due to its programming, and demonstrates that AI answers need to be taken with a grain of salt. Never forget that the “A” in “AI” stands for “artificial”, not “actual”.