Readjusting our expectations of genAI

The Automaton Chess Player

In 1770, Wolfgang von Kempelen constructed and unveiled the Automaton Chess Player, a machine made of cabinetry, a chess board, and a life-sized model of a human head and torso that could manipulate the chess pieces. For 84 years, it was exhibited on tours playing and defeating many challengers including Napoleon Bonaparte and Benjamin Franklin.

After fire destroyed the machine, the son of the automaton’s owner revealed that it was an elaborate, long-standing hoax. The interior of the machine was very complicated and designed to mislead those who observed it. In fact, the machine was a mechanical illusion (a magic trick) that allowed a human chess master hiding inside to operate the automaton. When the cabinet doors were opened to "prove" it was not a hoax, the operator escaped notice by temporarily concealing their body behind complex gears and machinery.

The Magic of AI

The Automaton Chess-player shows us how human intelligence can be mistakenly attributed to an artificial intelligence. We are susceptible to this because I believe humans really want AI to come to fruition. The 1962 TV series The Jetsons made this visceral for us: once the AIs do all the mundane tasks, we will all have a 3-day work-week and plenty of leisure time. AI will help order groceries, a robot will wash and fold the clothes, a self-driving car will take the kids to their after-school activities. Oh my - I really want this.

Clarke wrote this in the context of two civilizations meeting each other, each with different levels of technological achievement. When Spanish explorers met the ancient civilizations of South America, they were welcomed like gods in their shiny steel armor and with "deadly fire from their hands". They mistook advanced technology for magic.

The reverse is also possible: Magic can be mistaken for advanced technology. Cue the viral video showing the “wireless soda machine” (link below) which doesn’t even store soda on premises but instead draws it on-demand from regional servers. 😂 Similarly, viewers of the Automaton Chess Player saw the magic (tricks of intentional misdirection) and incorrectly concluded that a machine was built with human-level intelligence.

We’re doing the same now with AI. Let me explain.

AI is a set of tools

Presently, AI isn’t a single method or technological breakthrough. Rather, it is a set of tools aimed at solving pattern-matching problems through computational means. These tools have been evolving since the 50s, where at first there was focus on symbolic reasoning, heuristic systems, robotic pathfinding, and mathematical modeling of the neuron. After decades of advancement mostly in pattern recognition, we now have machine learning (ML) solidly established as a field unto itself.

Within ML there is

• supervised learning - models made from labeled data; think spam filters and cancer detection in MRI scans

• unsupervised learning - models made from clustering; think recommendation engines

• reinforcement learning - models that allow adaptation to real-time observations; think self-driving cars

• generative AI - models made from observing sequences; think autocompletion, chatbots, coding assistants, and prompt engineering

While relatively young, genAI is getting all the attention now because it’s able to produce in seconds what looks like novel, considered content - summaries, reports, images, videos. This sure looks like intelligence, but like the citizens of 1770, we are mistaken. It is not novel because it derives from the training set; it only knows what it’s trained on. And it is not considered because there is no understanding that drove the output. It’s simply a pattern that has been learned from very deep data sources with a bit of randomness mixed in that has the appearance (like magic) of authenticity.

GenAI lacks understanding

Symbol manipulation. The Chinese Room Argument is a philosophical thought experiment that shows syntax does not imply semantics. Tiles with Chinese symbols are slid under a door and an operator (who cannot speak Chinese) looks up this input in a predetermined computational system. After finding the right response, the operator collects the output tiles and slides them back under the door. The people outside the room mistakenly think there is a Chinese speaker in the room.

Now, scale the computer’s model up dramatically into an LLM (Large Language Model) trained off the entirety of the world’s Chinese published works. It’s harder to determine whether the operator speaks Chinese, but still they do not. They could have an entire conversation with the people outside without understanding a single word of what the conversation is about. This is not understanding.

Memorization. What genAI has done is memorize little pieces of the training set. Given a prompt and a random seed, it then tries to reassemble some of those little pieces into a new whole. Internally, it will keep iterating on this procedure until it has converged to a stable output. Change the seed and you can get different output. This brittleness is what leads to hallucinations - it is easy for genAI to go into a part of the sampled-pattern space that no longer makes sense.

This brittleness is most visible in tests, like math quizzes and professional exams. Using questions similar to the training set, genAI can produce an A grade on a legal bar exam. But use questions that were not in the training set, and it can easily fail. You do not want this to be your lawyer.

Where GenAI goes wrong

Time for some fun. Grab a bag of popcorn, gather your friends and family, and see how genAI can humorously fall flat. As the models and computation have scaled, we’re starting to see generated videos. The current state-of-the-art is Sora (link below).

In the watch party, take score of who finds the wackiest phenomenon. You’ll see countless violations of object persistence and conservation of mass, which is a toddler’s level of intelligence. Walking people have arms that elongate, like they are made from springs. People in the distance will disappear into a shadow or suddenly emerge from a pothole. A woman in a robe crossing a church square splits into two pedestrians. The number of people sitting at a table does not match the number of legs under the table. Kinda creepy but also kinda cool: the orientation of a hand can change from left to right (so that the person has two right hands), and then back to left. Very creative, sure, but also unrealistic because there is no understanding of the physical elements in the scene.

In the 2D realm, there are several genAI failure modes for images that are easy to spot:

• Look at the fingers. Are there extras or any missing? Are they unusually long or not correctly sized relative to each other? Remember, there’s no explicit model of a hand in the AI system.

• Look at the text. Is it illegible to the point of being symbolic? There is no explicit language or character model for text output (though one could be added).

• For a scene, is the scale between objects unusual? There is no 3D model used to generate the scene (though it could be added).

All these issues are fixable. For example, it’s straightforward to build a system that detects the number of fingers. Add that to your genAI pipeline, and then you can reject candidates that don’t pass the “finger test”. For videos, I expect more leveraging of computer graphics techniques, like using 3D modeling to improve object persistence and avoiding scale mismatch. Filling in these holes one-by-one will create a better tool, for sure, but it still is not what humans would call understanding.

Tech Maturity Cycle

Technologies go through a maturation cycle, and going from invention to viability can take a long time, or maybe never succeed. Here are 4 rough phases of tech maturity:

• Phase 1: Academia, where the technology gets a “proof of existence” in a seminal publication

• Phase 2: Research labs, where early applications of the new technology can be investigated and tested

• Phase 3: Niche commercialization, where the tech is shown to be viable for specific industries with specialized problems

• Phase 4: General commercialization, where the tech becomes more widely adopted to solve common problems

Some think AI (or even genAI) is in Phase 4 now, or soon to be there. Let me explain where they really are.

AI overall (including machine learning and robotics) is already squarely in Phase 3, with many of the AI tools already applied successfully to medicine, defense, manufacturing, and, yes, targeting of ads. It’s “The Algorithm” that produces your feed in your social network of choice. AI is all around us, but it’s not ironing our clothes yet - that would be Phase 4.

GenAI, on the other hand, is in Phase 2 moving toward Phase 3. It is in its infancy, though quickly evolving forward thanks to the heightened interest and resources being poured into its applications. Still, up to now, there is no clear ROI due to the hallucinations present in all domains. This is particularly problematic for anything that is “mission critical.”

Reality vs expectations

To be clear, this is an exciting time for AI, and for humanity. The problem is that expectations on technology are sometimes independent of its maturity. GenAI is currently at the beginning of the “hype cycle”, and it’s bringing the rest of AI along for the ride into high-stakes visibility, and expectations.

The tech trigger here was generative AI (specifically ChatGPT 3) released for free to the masses, who were able to experiment and share results. I think we’re still not at the peak because it will take some time before the current excitement (and fear of disruption) can be tamed by the lack of solid, viable, and profitable applications. Barring a recession, it will also take some time before current AI companies have difficulty finding funding. Even though we’re not at the peak, we should still prepare ourselves for the contraction that is yet to come.

AI-augmented humans

I don’t know when AGI (artificial general intelligence) will happen, but for sure there are many AI advancements and exciting applications still before us. We can find new AI successes in two ways: limit the domain, and add the human.

Limit the domain. Instead of trying to jump into Phase 4 now (solving generalized, common problems), we can look for niche applications that have a specific problem to solve. In other words, don’t take the latest ChatGPT version and apply it to solving every toy problem you can imagine, expecting it to work. Instead, think of LLMs as platforms on which to build, train, and evaluate an agent that specializes in a domain. The evaluation step is critical so that you know whether you are improving the agent, or making it worse.

Add the human. We can also put a human in-the-loop to guide the machine. For example, we can combine the speed and depth of genAI with the common sense and creativity of humans. For example, today book and magazine writing requires several stages of drafting, editing, and reviewing. GenAI can reduce costs by providing feedback during the editing process or by “filling in the holes” in a first-draft - both under the full guidance of the author.

Similarly, a graphic artist can sketch the main idea, and genAI can “fill in the holes” by bringing in high-definition details. If the result isn’t exactly right, it’s a matter of seconds to see the next alternative. The human will retain control of the creative process, but will be more productive by this AI augmentation.

Thanks to the democratization of AI models and computing power, we will be able to rapidly explore these possibilities. Let’s go!

References

• Mechanical Turk, Wikipedia

• Chinese room argument, Wikipedia

• Wireless Soda Machine (video), The Carbonaro Effect

• Sora by OpenAI - state-of-the-art video generation

• Gartner hype cycle, Wikipedia

About the Author

Donald Tanguay is an engineer, scientist, and organizational leader. Holding degrees from MIT and Stanford, he has contributed to industrial research labs, startups, and big companies. Most recently Donald spent 15 years on Google Search, where he evolved his role from extracting ranking signals via image processing to leading a cross-functional organization that launched hundreds of features to billions of users. As Spring CTO, Donald is bringing uplifting content to users at scale, all packaged into a great user experience.