According to McKinsey, AI will create an estimated $13 trillion of GDP growth between now and 2030. As a comparison, the GDP of the entire United States of America was around 19 trillion in 2017. Leading AI scientists, like Andrew Ng, describe AI as the fourth industrial revolution or the new electricity. AI is undoubtedly a centerpiece of digital transformation and its application throughout the industry will dramatically change our world and how we do business. The problem is that many people want to participate in this AI-revolution but they are overwhelmed by its technological sophistication. They don’t know what AI is capable of, let alone how they could use it for their company. And this is exactly, what this blogpost is aiming to solve: to make AI more understandable for people without a technical background, so that they are able to evaluate possible use cases of AI by themselves. That's important, because no one knows the ins and outs of their business better than them, they can come up with the most valuable ideas on how to use AI within their company.
Note that this post is heavily inspired by Andrew NG’s content on the topic.
So whats behind the “buzzwords”? Let’s jump straight into it.
Table of Contents:
- The Misconception about AI
- What is Machine Learning?
- Terminology of AI
- What is Data?
- How do you get Data?
- Misuse of Data
- Summary
The Misconception about AI
(Image Source: https://unsplash.com/photos/1K6IQsQbizI)
There is a lot of unnecessary hype around AI, which is mostly due to a common misconception that many people have. Artificial Intelligence can be separated into two parts or two ideas:
Artificial Narrow intelligence (ANI)
This describes AI’s that are good at one specific task, which they are trained and developed on. This can be, for example, an AI system that predicts the prices of houses based on historical data or the algorithm that recommends youtube videos to you. Other examples are predictive maintenance, quality control etc. ANI is a very powerful tool and it will add a lot of additional value to our society over the next years. All of the progress that we saw in recent years, and what we constantly hear about in the news, happened in the field of ANI. These catchy news articles lead people to wrongly assume that science made a lot of progress in AGI, but in reality we only made progress in ANI.
Artificial General Intelligence (AGI)
This is the end goal of AI: a computer system that it is as smart or smarter than a human. An AGI could successfully perform any intellectual task that a human being can do. This is also the part of AI that raises the most fear in people. They imagine a world where computers are much smarter than humans, where nearly every job is automated, or even Terminator like scenarios. This is what I mean with unnecessary hype. It is leading to irrational fears about the future of humanity, while in reality, we are still many technological breakthrough away from reaching real AGI.
What is Machine Learning?
You could say that Machine Learning is the backbone technology of AI. It uses statistical techniques to give the computer program the ability to learn (e.g. to progressively improve its performance on a specific task) from data, without being explicitly programmed.
Machine Learning is the tool of AI that caused all the hype and that enabled nearly all of the value that is created through AI systems. It can also be separated into different parts, but only one part is responsible for 80% of the value that is created through Machine Learning. What I talk about is Supervised Learning.
Supervised learning algorithms simply just learn input (A) to output (B) mappings, by learning relationships within huge amounts of data. Imaging that you want to build a system that can classify e-mails into spam and non-spam mails. You would need to accumulate a lot “labeled” examples of e-mails. This means you have a label for every e-mail, that tells whether it is spam or not. You would need to accumulate thousands of e-mails with labels and then you can feed this data into a supervised machine learning algorithm. In the training process, the algorithm would analyze all the e-mails that you gave him and it would recurrently improve its understanding about what attributes differentiate spam from non-spam emails. In this example, the system has to map e-mails (A) to a label that tells if the mail is spam or not (B).
Like I said, you train the algorithm by giving him thousands of labelled e-mails. After you trained the algorithm on that data, you can give him a completely new email (that the algorithm has never seen before) as input and it will tell you whether it thinks that the email is spam or not.
Another example is online advertising, where the input is information about a user (A) and the output of the system is a label that tells whether a user will click an add or not (B). Another example is speech recognition, where the input is speech as an audio file (A) and the output is a transcript of what is said in the audio file (B). Another example is when you give the algorithm an image of a steel plate (A) and it has to tell whether it is defect or faultless (B).
This can seem like a quite limiting technology at the first glance, but it is incredibly powerful if you find the right application for it. It is the single major cause of additional value that is created through AI for our society. The number of different use cases for this technology seems endless and people discover new ones every day.
Terminology of AI
Artificial Intelligence is a very complex field with a lot of terms that can be quite confusing in the beginning. You probably heard about Neural Networks, Deep Learning or Data Science. We will now take a look at the most important terms of AI and uncover their meaning, so that you are able to to talk about AI with other people and to think about how you could apply AI within your business.
I am giving you the most commonly used definitions of AI terms, but be aware that AI is a very opaque field where many terms are used interchangeably and sometimes inconsistent.
Artificial Intelligence
Artificial Intelligence is an area of computer science that emphasizes the creation of intelligent machines that work and react like humans. Like I already mentioned, when people talk about AI, they mostly mean Artificial General Intelligence (AGI). You should see AI as the whole Area, and Machine Learning and Deep Learning as the techniques used to make computers act intelligently.
Machine Learning
Machine Learning is a sub-field of AI. It is a field of study that gives computers the ability to learn from data without being explicitly programmed. So with Machine Learning, you basically train the program to perform a certain task. Therefore, machine learning often results in a running AI system, which is basically a piece of software.
Example of a Machine Learning Project:
Imagine that you are a real estate company and that you have a lot of data about houses. You partner with a Machine Learning company to build a Machine Learning system to predict the future prices of houses. A system like that enables you to make better decision regarding in which houses you want to invest and to figure out the right time to liquidate your investments.
Deep Learning
Deep Learning is a sub-part of Machine Learning, which is responsible for all the media hype and most of the breakthroughs in ANI, that we saw in recent years and still see today.
It is basically the same thing as Machine Learning: You give the algorithm labelled data and it learns to predict the label. The difference to Machine Learning is that you use more modern and more sophisticated algorithms, called Neural Networks. In contrast: in Machine Learning, you use more simple, traditional algorithms.
Due to their complexity, new technical discoveries and the availability of enough data and computational power, Deep Learning algorithms where able to break the previous benchmarks on many tasks and even to out-perform humans on some of them (for example: Histopathological Image Analysis, or recommending movies on Netflix).
Although, Neural Networks (e.g. Deep Learning algorithms) almost always perform better than traditional algorithms, they have certain disadvantages. If you want to know more about that, check out my post: “Pros and Cons of Neural Networks” (https://towardsdatascience.com/hype-disadvantages-of-neural-networks-6af04904ba5b).
You often hear that Neural Networks are built like the human brain or inspired by it, but in reality, they have almost nothing to do with it. It is true that they where initially inspired by the brain, but the details of how they work are completely unrelated to how biological human brains work.
Note that many people use the terms Deep Learning and Neural Networks interchangeably.
Example of a Deep Learning Project:
A deep learning project does not differ that much from a Machine Learning project when you look at it from a high level view. You only need much more data, more computational power and highly skilled engineers.
Data Science
The output of a Data Science project is usually a set of insights that help you to make better business decisions, such as deciding whether to invest in something, whether you should acquire certain equipment, or if your website should be re-structured. You could say that Data Science is the science of extracting knowledge and insights from data by analyzing it with statistical methods, visualizations etc. The output are often presentations or slide decks that summarize conclusions for executives, leaders or product teams to make certain decisions.
Example of a Data Science Project:
Imagine that you are in the online advertising industry. By analyzing the sales data of your company, your data scientists found out that the travel industry does not buy many adds from you. As a result you could switch your sales teams focus to companies of the travel industry.
Another example:
Imagine that you are running an e-commerce business and you hired a few Data Scientist to get some more insights into your business. The outcome of this project could be a slide deck presenting a plan on how to modify pricing in order to increase overall sales or insights on how to market specific products more efficiently.
Some people say that AI is a subset of data science and some people say it is the other way around. So, it depends on who you are talking to, but I would say that data science is an interdisciplinary field that uses many tools from AI machine learning and deep learning, but it also has its own separate tools. Its goal is mostly to drive business insights.
You probably also heard about other buzzwords, like Reinforcement Learning, Generative Adversarial Networks (GANs) etc. These are just other tools to make AI systems act intelligently, or said in other words, to do Machine Learning and sometimes Data Science.
You now know about AI, Machine Learning, Data Science and Deep Learning (e.g. Neural Networks). I hope this gives you a sense of the most common terms used in AI, and that you can start thinking about how these things might apply to your business.
What is Data?
Data can take on many forms: spreadsheets, images, audio, sensor data etc. These are split into two main categories: structured & unstructured data.
Structured Data (“data that lives in a giant spreadsheet”)
Structured data, is like its name already implies, data that is stored in a structured format following a pre-defined schema. It refers to any data that resides in a fixed field within a record or a file. It can be textual or non-textual.
Below you can see an example of structured data from the popular Titanic dataset. It contains information about each passenger who was on the Titanic.
Unstructured Data
Unstructured data is essentially everything else that is not structured via a pre-defined schema. It can be textual or non-textual, but when people talk about unstructured data, they mostly mean images, videos, audio files, documents etc.
I already explained what supervised learning is. Since supervised learning is the most commonly used type of Machine Learning, when people say “data”, they mostly mean labelled data. Example: You have a dataset with photos from 100,000 dogs and cats where each photo has a label, either “Cat” or “Dog”.
Another example is a dataset that contains information about housing prices. Here you would have information about houses (like square meter, number of bedrooms, location etc.) and also their price as a label.
How do you get Data?
You can find many dataset for a lot of problems in the internet (some for free and some cost money), but most of the time you need to create your own dataset (if you don’t already have it) that is specifically tailored to the problem that you are trying to solve with AI.
There are three main ways to get data:
1. Manual Labeling
Imagine that you want to build a classifier that can detect whether there is a man or a woman on a given picture. To train such a classifier, you would need the create or get many images of men and women. Then, you need to assign a label to every image: men (label 1) or woman (label 2). You can also pay people to do the labeling work for you (Ex.: Amazon Mechanical Turk: mturk.com).
2. Observing behaviors.
Imagine that you ran a e-commerce business and want to predict when a customer will make a purchase, which in turn enables you to manage your stocks better etc. You could create a dataset by simply observing how your users behave on your website and how they make purchases. This would result in a dataset that describes the actions of each user (described by some variables like for example: time of the day, where they clicked etc.), together with a label: purchase (label 1) or no purchase (label 2).
Another example is that you observe the behavior of machines , which could enable your to predict when they need maintenance etc.
3. Use Free Data Sources, buy data, or get it from a partner
There are many free sources for datasets like Kaggle. You can also use Google Data Search, which works like google but only for datasets. If you do not find anything, you can look for datasets on a data market place or get it from a partner.
Misuse of Data
(Image Source: https://unsplash.com/photos/1K6IQsQbizI)
Acquiring data maybe seems simple to you at a first glance, but there's a lot that can go wrong. In AI & Machine Learning we say: “Garbage in garbage out”, which means nothing more that you get the quality out of your AI system, that you put into it during training.
Imagine you know that you want to create a specific AI application and you start acquiring Data (which you think is useful). Your plan is to accumulate data for two years and then build an AI system. This is very bad practice. In this scenario, the correct way would be to acquire the data that you are able to get and give it to an AI expert as soon as possible. He can tell you after some evaluation, what parts of it are useful, what parts are completely useless, and what data you should add additionally. By doing that, you don’t have the risk that you acquire data over two years and then you realize that it was the wrong data and that you can’t do anything with it. To save money and time: Evaluate the quality of your data quickly together with experts.
Another big problem are incorrect labels. Example: Cat images that are labelled as dogs and dogs that are labelled as cats etc. You get what I mean. This prevents your algorithm from learning what really separates cats from dogs and totally confuses it. The good thing is that the problem with incorrect labels gets less and less important the more data you have in total. If you have a huge dataset with over 2 million labelled cats and dogs images, a few incorrect labels won’t hurt its performance.
Another problem is that some people assume that because their company has a lot of data, that this data is useful or that an AI team can make it useful. That's completely wrong. Although, more data is usually better, you can have billions of data entries, that are worth nothing and not even the best AI engineers of the world can create value out of something that has no value. So please don’t throw data at an ai team and assume it will be valuable somehow. You maybe think that this is common sense, but I saw it happening many times in the industry because of a misunderstanding about data and AI. There are even startups founded because people thought that they posses useful data, when in fact they didn’t. Other issues are missing values, multiply types of data (can be solved — but costly) and much more.
Summary
I hope that this post gave you a solid introduction to the field of Artificial Intelligence from a high level perspective and that you now have a better understanding of how AI works and what it can really do. If you think there is something missing or not explained clearly enough, you can let me know in the comments. To summarize: You learned about the common misconception about AI (e.g. that people often confuse AGI with ANI) and what Machine Learning and Data really is. You are now familiar with the most common terms of the field: Data Science, Deep Learning, AI and Machine Learning. Additionally, you learned where you can get data, how you should not approach data acquisition and that having a lot of data does not necessarily mean that you can do AI with it.
