AI#01 - Embarking on the AI Journey: From Eliza to ChatGPT
This article tries to answer the following questions: What is AI and ChatGPT? How does it work? What's behind the AI lingo? Is it really intelligent? What's so unique about ChatGPT that makes it...
I caught the AI bug, after attending too may events on the topic and chatting with equally passionate colleagues. So I decided to write a series of article on AI. My ultimate goal is to learn about AI and form a framework of reference I can use to make better decisions on product strategy and investments. I also find it critical to understand what's ahead of us, what it means for my kids, family, and our society.
This first article tries to answer the following questions: What is AI and its evolution? How does it work? What's behind the AI lingo? Is it really intelligent? What's so unique about ChatGPT and more broadly Generative AI?
As a product manager, I live at the intersection of technology and business. I plan to bring my personal take on the future impact of AI.
A brief history
Artificial Intelligence (AI) is a seventy years old concept. In this amazing podcast hosted by Lex Fridman, Eleizer Yudkowski explains that AI was first mentioned at Dartmouth college in the 1950s through the research of professor John McCarthy. From there, it took fifty years to recognize the full potential for an Artificial Intelligence, a slow learning curve by any means.
Until recently, the evolution of AI had been pretty slow, mainly due to the limitations in computing powers:
Starting in the 1960s through the 1990s: researchers developed basic AI programs that could play chess, solve logical problems, and understand natural language. Eliza, developed in the 1960s was one of the first program to simulate conversation
In the 2000s: advances in computing power and the development of new machine learning techniques like deep learning led to breakthroughs in speech recognition, image recognition, and natural language processing.
In the 2010s: AI appeared into our daily lives, with the rise of virtual assistants like Siri and Alexa, smart navigation systems like Google Maps, and the development of self-driving cars and other applications.
AlphaGo. A Turning Point
On March 9 2016, AI had its big breakthrough. An AI program called AlphaGo, developed by Google's DeepMind, defeated a world class player at the game of Go. This is no small feat given the complexity of the game. The game of Go requires to make complex strategic decisions. AlphaGo's victory demonstrated that machines can master complex tasks we thought to be in our exclusive domain of intelligence.
AlphaGo's success was a combination of advanced machine learning techniques such as deep neural networks and reinforcement learning. Because the program was trained on a large dataset of professional Go games, it learned the patterns and strategies of the game. After playing millions of games and learning from his mistakes (reinforcement learning), AlphaGo became the uncontested new champion. That day, AI showed it could surpass humans in complex tasks that require deep thinking, intuition, and creativity.
Decoding the Alphabet Soup: Understanding AI Acronyms
If you are not familiar with the different terms associated with AI, it is a good idea to spend time to familiarize yourself with several acronyms and their function in the AI ecosystem. Thiyagarajan Maruthavan (Rajan) wrote an excellent article, Absolute beginners guide to making sense of key AI terms in 2023, that lists most used AI terms.
I picked Rajan's article among many others because he used the metaphor of a chef cooking recipes. This tasty metaphor stroke a chord with me -people in my circle know I can be easily be influenced with good food and wine 🙂
I suggest you become fluent with those concepts:
Machine Learning (ML), including supervised learning, unsupervised learning, and reinforcement learning
Natural Language Processing (NLP)
Machine Learning Models
Neural networks
Deep Learning and Transformers
Large Language Models (LLM)
Generative Pre-trained Transformer (GPT)
Generative AI
LLMOps
AI Simplified: An Introductory Guide for Beginners
In essence, an AI is like a virtual assistant who is trained on some data , the model, and produces predictions. The quality of the prediction varies based on the quality of the dataset and how you train the model.
I like the metaphor of kids going to school to acquire the skills needed for a job. It takes twelve to twenty years of training and education to enter the workforce. Then, it may take another five to ten years to become expert at your job. Through diverse professional experiences, you may acquire new skills and be able to perform more tasks.
Today's AI works the same way. You feed your AI data and teach it how to interpret that data (training). Then you put your AI in the real world (LLMOPs) and ask it to perform a task (predictions). At the beginning it does a decent job at a few specific tasks and overtime it becomes an expert, learning along the way (self-learning). AI does this in months compared to two decades for us.
Many pre-ChatGPT AI applications focused on Machine Learning (ML). Data scientists had to collect the right data and train the AI model (human supervised learning) to perform a specific task. It was very time consuming and costly. The outcome was uncertain. Back to the kids metaphor, think of potty training. My wife and I read a book that explained how to potty train your kid over a week-end. We realize later this book belongs to the fiction category. Early generation AIs were a hit or miss too. Many failed and never made it to production.
Take Image recognition for instance. A data scientists injects large quantity of pictures into the AI model, some being cats, others not. Individuals review the AI predictions and tell the AI when the predication is right or wrong. The AI learns like a student at school. Once the learning is complete, the AI moves to production, i.e. a graduate getting his first job. Such training used to be very expensive and could cost in the tens of millions of dollars. Obviously, the cost would largely be a function of the prediction accuracy needed.
AI Illustrated: Pre-ChatGPT Examples
Before discussing nextgen AI, I find it helpful to go over several examples of AI applications in our life.
Recommendation systems
Amazon, Netflix, and the largest B2C Commerce sites use AI-powered recommendations (Suggested for you, Items you may like, Others like you also viewed). Netflix disrupted Blockbuster's brick-and-mortar business with a new online business model powered by AI to provide one-to-one personalization. Amazon disrupted retail shops by offering a frictionless online shopping experience. Amazon Prime is fully AI powered. Amazon Prime does not know exactly how many items are in stock and how long it will take to ship them. The real-time prediction "Your item will arrive on X day" is generated by Amazon AI. And it gets it right over 99.99% of the time!
Online Chatbot
Chatbots are early generation GPT. They help site visitors and customers on a variety of processes such as support, order and checkout, and account management. The chatbot is backed by an AI model built with customer specific data and offer assistance to perform basic tasks. The old generation models used human supervised training.
Image Recognition
In healthcare, companies like Arterys and GE Healthcare assist radiologists in analyzing MRI and CT scans, and detecting abnormalities in the brain, heart, or other parts of your body. In 2016, Salesforce acquired MetaMind, a startup that helped radiologists with early disease detection, among other things. Despite their early successes, engineers and scientists recognized the cost and complexity of building accurate prediction models with the AI technologies back then.
Virtual Assistant
Virtual assistants like Amazon Alexa, Google Assistant, and Apple Siri started invading our homes a few years ago. They use NLP (Natural Language Processing) to interact with us and perform basic tasks such as playing music, controlling smart home devices, answering questions, etc. IBM Watson, created back in 2006, became famous after it beat two of the All-Time Jeopardy champions in 2011. Nonetheless, Watson adoption remained limited because of its high cost, complexity, limited capabilities, and a disconnect between the expectations (Jeopardy world champion) and reality.
Fraud detection
Credit card companies like Visa and Mastercard were early adopters of AI for fraud detection. It is a business critical application given the amount of transactions they process and the economic gains to be derived from small improvements in fraud detection. Nowadays most financial companies have some sort of AIs. Stripe who provides in-store payment systems for merchants is another company using AI for fraud detection.
Autonomous vehicles
Car manufacturers such as Toyota, GM, BMW, use AI to interpret data captured through sensors embedded in the car. New generation cars include smart features like automatic lights switch at day and night, assisted-parking with a smart camera, cruise control, etc. Disruptors like Tesla, Uber, and Google are already working on nextgen fully autonomous AI vehicles without drivers. We are not there yet.
Gaming
The top gaming companies (EA, Activision, Ubisoft) used AI to personalize the gaming experience for each player and create content such as players, levels, quests, rewards, that trigger more dopamine and keep players engaged longer. They also adopted Chatbots to provide better self-service to million of gamers at a lower cost.
Robotics
Companies like Boston Dynamics, iRobot, ABB launched AI-enabled robotic systems that can perform basic tasks. Industrial applications include automated warehouses for moving goods and completing orders or manufacturing automation (car chain, food plant, etc). AI robots are also making their way to consumers. My dad recently bought an intelligent lawn mower. Its sensors connect via GPS to detect the lawn boundaries and create a map of the lawn. Its AI-optimized software calculates the optimal route. A mobile app lets you specify how often to mow the lawn and the mowing height!
Lex Fridman recently interviewed Robert Playter, CEO of Boston Dynamics. It is a fascinating discussion on the future applications of AI in robotics (if in a hurry jump to 1h26 in the video). Disclaimer: I disagree with many things he says.
First-Generations AIs Limitations
In the above examples, the AI learns from a specific datasets to make a prediction. The prediction is probabilistic, based on "best match". For instance, the AI may recommend knowledge articles on a manufacturer website when a visitor asks how to fix a malfunctioning monitor. You need to train the AI with a feedback loop and test the predictions until they reach the desired level of accuracy. Only then can you move the system to production.
As such, you need to align business stakeholders with the level of confidence needed for the predication. The stakes will vary depending on your industry. A higher level of prediction accuracy translates into higher upfront costs to train the AI model. For instance, it is ok to recommend a support article slightly related to a question but getting a cancer diagnosis wrong could be life threatening.
What else could go wrong? Well, you could not have enough data to build the model or lack quality data. Imagine you provide most of your support through the phone and you did not build out a knowledge base or did not capture those interactions in a database. Without proper data, you cannot train your model.
In the podcast The Economics Impact of AI, Avi Goldfarb makes a great point: both the scale and quality of data matter. You cannot build a successful AI model with one and not the other. To dig more on the topic, you should read his book Power and Prediction.
Generative AI: Redefining Possibilities and Igniting a Revolution
Until recently, AI looked like a promising technology with a long way to go. Then OpenAI launched ChatGPT 3.5, and soon after version 4.0. If you did not try ChatGPT yet, sign up for an account. As you ask questions (your "prompt"), ChatGTP answers like in a real discussion. The more accurate your questions, the better ChatGPT answers.
It is truly bluffing. It feels like you are interacting with a human. Obviously you are not but this is the power of ChatGPT. People get fooled by the quality of the interaction and the quality of speech. Fluency does not equate accuracy though. My eleven-year old boy, while very smart for his age, has a pre-teenager brain and cannot argument like an adult.
Professor Rama Ramakrishnan from MIT does a great job at explaining the evolution of ChatGPT and how it works. This sentence summarizes it best:
GPT-3 is a mathematical model trained to predict the next word in a sentence, using the previous words
So that's it? ChatGPT is not intelligent and just programmed to pick one word at a time. It seems crazy when you look at ChatGPT that way. To be clear, Generative AI is not limited to text output, ChatGPT is just one of the many applications for Generative AI.
The Prompt and Output dance
At its basics, humans interact with an AI program by providing an input called a prompt. The AI breaks down the prompt in tokens that it can understand., e.g. the ingredients in a recipe. Then it processes the tokens through its neural networks to generate an output. It could be some text, a prediction, or anything else the AI is trained to do.
R. Ramakrishnan walks us through the example of this prompt: "Four years and seven". Then, AI generates the next words, one by one, until the sentence is complete: "Four score and seven year ago, our fathers... all men created equal". Like Google Search returns many search results for a keyword, ChatGPT can return different outputs for a prompt. That's why if you ask the same question, you may get different results.
As an engineer in Applied Mathematics by training, I truly appreciate the complexity of getting such predictions. Still, it is probabilistic and not intelligent. For now at least. What's truly amazing is the ability for ChatGPT to generate good answers at light speed given the gazillions of combination possible.
People get excited because AI apps like ChatGPT do an amazing job at giving authoritative answers. Its English is excellent with great sentence structure. Do you realize that ChatGPT is better at writing English than >90% of the population? And the fastest writer in the world!
Deep learning, neural networks, and transformers
Machine Learning (ML) models existed for a while. However deep learning is taking AI to the next level. Deep learning works on large and complex datasets. It can find "meaning" in the ocean of data, without human intervention.It adapts its learning to new data, which makes it more robust.
Deep learning relies on neural networks to deliver predictions. Neural networks are the equivalent of the human brain for machines. The network is made of multiple layers of interconnected neurons. Think of inhabitants in a city. The strength of the connection varies (acquittance, friend, family). The input is processed by a neuron which generates an output and pass it to the closest neuron. The same operation repeats until the final output is delivered.
Besides neurons being connected based on weight and biases, neurons leverage "activation functions" to learn complex patterns. This is critical because most input follow a non-linear pattern. In my university days, we referred to non-linear problems as the Traveling salesman problem. The questions asked is:
Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the origin city?
Long story short, this problem is classified as complex and cannot be solved with linear programming by testing all combinations. You need to define heuristics, i.e. strategies that get close to the optimum solution.
Deep learning models contains a level of complexity that is hard to comprehend for the human brain. We are talking about billions of parameters to capture meaning and each new version of GPT becomes more powerful.
Transformers represent one architecture to build LLMs. They allow the model to focus on specific part of the prompt, therefore focus the attention of the model on what's most relevant. This concept was first introduced in 2017 by Google Researchers in the paper Attention Is All You Need. Previously, models would treat each word in the prompt with equal importance.
On Words, Meaning, and Reality
A few months ago, I attended an event in San Francisco on the future of AI. Dr Julia, a researcher at Renault, argumented that AI is not intelligent -at least for now.
Dr Julia made his argument from a technical and semantic angle. Technically and at the risk of oversimplifying, AI is a mathematical model supercharged with lots of CPU power. It lacks reasoning, intuition, creativity, empathy, emotional intelligence, pretty much everything that defines us as intelligent species.
Dr Julia's argued that we picked the wrong name for AI. This is not intelligence. This pattern seems to repeat with "neurons" which, by association, makes you think of the concepts of brain and intelligence.
The words we use can have a powerful effect on people. The excellent book of Edward Bernays called Propaganda (1929), explains how you can use propaganda to influence human psychology to shape public opinions. Today's marketing for instance is a form of propaganda. In this context, propaganda has no negative meaning, it is just a mean to an end.
When John Mc Carthy coined the term "Artificial Intelligence" for his field of research, he had little to show for it. It was a mere concept. But it sure did the trick and caught the attention of the press and other researchers. Would he be famous today if he had called his paper "Computational Analytics Engine"?
Conclusion
This concludes my first article on my AI article series. As we went through the evolution of AI since the 1950s, it is becoming evident that the pace of innovation is accelerating in an exponential fashion. Yet, we are still at the premises of what AI can truly deliver. If I had to put AI on a human scale, I'd say it is a three year old. Scary, right?
In the next article, we will explore how you can benefit from AI in your job and personal life today.
Please take the time to share your thoughts or ideas. We have a long way to go to understand the challenges and opportunities around AI. Let's take this journey together!