If a story does a good job illustrating a point you are trying to make, does it matter if it is true?
Back in the 1960s, the Department of Defense (DOD) began developing a primitive AI that it hoped would be able to detect military tanks in photographs. Like all AI, this program had to be trained to detect and perform its desired function. The DOD flew a balloon with a camera over a testing facility. It took 50 pictures with tanks hidden in trees or blending into their environment. In the same locations – they took 50 pictures without tanks in them. These photos were fed into the algorithm so it could learn to detect these tanks
Eventually, the algorithm began to classify photos with impressive accuracy. The team presented their findings to higher-ups, hoping to receive praise for saving thousands of soldiers’ lives. However – when they tested the AI using new photos, it failed miserably. Instead of the near 90% accuracy that they thought it would achieve, it predicted the inclusion of a tank at a success level under 50%. The DOD looked at the test photos to see if there was an issue they had not considered before. They found that the photos with a tank were taken on a sunny day, while the photos without a tank were taken on a cloudy day.
Instead of learning to detect tanks, the AI learned to detect sunny days.
Here’s the kicker, though: That story was completely made up. It was fabricated to illustrate the point that you can never truly know what an AI is learning from the training data that you feed it. Although the story was absolutely false, it does a fantastic job of depicting a legitimate issue that could otherwise be difficult to digest for most audiences.
This brings us back to my original question, does it matter if stories are fake if they illustrate real issues?
While the tank story helped me conceptualize the issue, I began to wonder if it was depicting a legitimate issue once I found out that it was fake. Would a real-world example have gone over my head? I don’t know how I feel about it. On one hand I like that it simplified a complex issue, but on the other hand I see the issue with making stories up to prove a point. If this is a real issue, there would be real examples. And if real examples of the phenomenon exist, why couldn’t one be used instead?
Suppose a fabricated story is used to depict a real event. In that case, it may help people understand the issue better, but they will stop caring as soon as they find out it’s fake. For example, I asked ChatGPT 3.5 to “generate a fake story that illustrates a real issue” and it came up with this;
“In the quaint town of Rivertown, there stood an old, majestic oak tree named Grandfather Oak. Generations of townsfolk revered Grandfather Oak, considering it a symbol of strength, wisdom, and unity. However, one fateful morning, the townspeople awoke to find Grandfather Oak trembling in distress. Upon closer inspection, it was revealed that a nearby factory had been illegally dumping toxic waste into the river that nourished Grandfather Oak’s roots. The pollutants had seeped into the ground, poisoning the soil and slowly killing the tree.
The townsfolk, outraged by this blatant disregard for their environment, rallied together to demand justice. Through their collective efforts, they pressured the factory to cease its harmful practices and clean up the mess they had created. Although Grandfather Oak suffered irreversible damage, its plight served as a wake-up call for the town.”
If I was trying to get people to care about the environment, I could use this story. You can’t help but feel bad for the community when you hear the story. Although it was a great analogy for the environmental damage that can be caused by unregulated industry, once you find out that the poor Oak tree never existed you immediately call my whole stance into question. While it depicted a real issue, there are too many good examples that could have been used instead. I then asked ChatGPT to “tell me about a similar real-life incident”, and it generated the following;
“In 2015, the city of Flint, Michigan, experienced a severe water crisis that garnered national attention. The crisis began when the city, in an attempt to save money, switched its water source from Lake Huron to the Flint River. However, the Flint River water was highly corrosive, causing lead from aging pipes to leach into the drinking water supply.
As a result, thousands of residents, including children, were exposed to dangerously high levels of lead, a potent neurotoxin that can cause irreversible damage to the brain and nervous system. Lead exposure is particularly harmful to children, leading to developmental delays, learning disabilities, and behavioral problems.”
Now THAT story makes me feel differently. Once you know that there are real people at stake, tangible action that can be taken, and actual people at fault for this issue – you understand and care. It is too easy for detractors to say “well if pollution was a real issue, why didn’t you use a real example?”
I guess this has been a longwinded way of saying, if you are going to use an example to prove a point – make sure that it is rooted in fact. And if you fabricate a story to prove a point, ensure that it is used to help someone conceptualize an idea, not to evoke an emotion. The issue with the tree example is that you felt bad for the community affected by the loss of the tree, not the larger issue at hand.
Sources –
Branwen, G. (2023, July 4). The Neural Net Tank Urban legend. https://gwern.net/tank
Kaufman, J. (n.d.). Detecting tanks. https://www.jefftk.com/p/detecting-tanks
Zečević, M. (n.d.). Fallacies in AI. Matej Zečević. https://matej-zecevic.de/2021/255/fallacies-in-AI/
Stories generated by ChatGPT 3.5
Edited by Grammarly