For as popular as the term "machine learning" has come to be, it’s surprising to me how often it’s equated to robots taking over the world.
Phrases like “neural nets” and “deep learning” tap into our sense of fantasy, but when we jump from new tech to robot takeover, we miss the beauty and power of what machine learning actually is, and the groundbreaking new developments that are pushing industries forward.
With this in mind, I sat down with our team’s data scientist, Hillary Green-Lerman, to shed light on the buzzword. I asked the questions Wikipedia failed to fully answer: what is machine learning, who should be learning it and how soon can I visit Westworld?
What is machine learning?
“Machine Learning is about using the data you already have to make predictions. This sounds really fancy, but most of the time, the ‘prediction’ is really just a label,” Hillary told me.
Machine learning evolved from pattern recognition and applying algorithms that can learn from data and then make predictions, and it's closely related to computational statistics (thank you Wikipedia).
In human terms, when you “teach” a computer, you feed it data over and over that tells it that this thing is a puppy and this other thing is a muffin. Eventually the computer, using computational statistics, starts to determine that it’s highly likely that this one thing is a canine and the other a carb.
This means you can start to give your computer huge data sets, and it can start to make predictions for you.
For instance, we’ve trained computers to accurately predict letters and numbers, the base logic for handwriting recognition used by the postal service. The same logic is used in the development of self driving cars and the algorithm that Target used to predict a woman was pregnant.
But while these experiments feel like the future arriving, remember that machine learning is really just powerful math & prediction.
“A machine’s learning algorithm enables it to identify patterns in observed data, build models that explain the world, and predict things without having explicit pre-programmed rules and models,” says Vishal Maini.
This is why it’s critical to have diverse teams working on machine learning algorithms. You need to feed the computer a full range of features and possibilities for your algorithm to work in the real-world.
For example, if you only feed your facial recognition algorithm caucasian faces, it won’t be trained on what to do with faces of color. Not only is this a bad formula, and bad for people, but as Google found out, it’s bad for PR.
What is a popular misconception about machine learning?
Prediction is powerful. Almost as soon as someone realizes what machine learning can do, they want to ask the crystal ball a question:
What’s going to be the next big programming language?
Who’s going to win the next election?
Can you accurately predict our revenue if we create this new product?
But crowding around your data scientist’s desk isn’t going to help you.
“One popular misconception is that people think they have enough data when they don't. When people say machine learning, a very large segment of predictions are based on existing data. And in order for that to work, you generally have to have a big labeled set of data,” says Hillary.
“If you want to predict which product you should recommend to which customer, you need data. Doing that for a product that doesn't exist yet isn't going to work. You would need a huge data set, so at least a thousand examples of each of the type of person who bought each product and this increases exponentially with the more features you want to analyze. A feature is something like age, thing they clicked on previously, etc.”
Applying machine learning to your business requires huge data sets that aren’t always accessible, but even if they are, it’s key that that data is in a format that a machine can read.
“People often don’t realize how much of machine learning is getting data into a format so that you can feed it into an algorithm. The algorithms are actually usually available pre-baked,” Hillary said. “In a lot of ways, you need to know how to pick the best linear regression for your data, but you don't really need to know the intricacies of how it's programmed. You do need to work the data into a format where each row is a data point, the kind of thing you'd want to pick.”
For example: If you want your algorithm to look at Customer X, who did or didn't buy things, you need to assign values for “bought” and “didn’t buy.” This means a lot of cleaning data.
“You actually have to do a lot of work to take all of the different pieces of information you have and knit them together into something you can feed into an algorithm,” says Hillary.
Most accurate machine learning reference in pop culture?
With cleaning data in mind, we know a computer needs you to assign values to features before it can analyze them. Once this sinks in, any fear of robot sentience starts to dissipate, and all of a sudden you start to see through the Hollywood hype.
Naturally, this is what Hillary and I started discussing:
“My biggest problem with Westworld—the part where I was just like, ‘I can no longer believe this,’ was when they were talking to Dolores. She's talking, and he says,
Stop. Now tell me why you just said that.
And she was able to. Because the super, super sophisticated machinery right now is neural nets and deep learning. One of the real challenges of it is, despite the phrase neural nets, it thinks so differently from the way humans think. The answer she would have given would have been ‘seven’—the value assigned to an outcome.”
That said, it’s easy to understand why a phrase like “neural networks” is such a buzzword. It’s one that lives up to its name.
A neural net is a set of little machine learning algorithms (i.e., little logistic regressions) that are combined to mimic neural activity. These models are then "trained" to perform a task (usually something complex like image recognition). They require huge compute power but can be very effective. The "art" tends to be deciding which training set will be the most useful and which configuration of "neurons" will be optimal.
So why is machine learning popular now?
“Neural nets,” “deep learning”, it all falls under machine learning, a topic that can feel as trendy as adult coloring books once were.
Looking at Google Trend data, you can see we’ve reached a peak for “machine learning.”
Adult coloring books
But why now? We’ve always had sci-fi movies that reference machine learning, from 2001: A Space Odyssey to Ex Machina. What’s the reason for the recent spike in popularity, and is this just a fad or the future?
The simple answer is: because we can do this now.
“We said we need really big data sets. If you track every single click on every single website, that's a really huge data set. Census records are downloadable in a way they never have been before,” Hillary said.
Here’s something to think about: in the past two years, we've generated over 90% of all the data in the world, according to Laura Dambrosio in the Huffington Post.
“NoSQL databases got popular, SQL got faster, and projects like Apache Spark did wonders for the speed and performance of large-scale data processing.” said Laura. “Suddenly we had mountains of data and a fast, affordable means of drawing insight from it.”
Added Hillary, “If you think about it, Excel couldn't handle the number of rows of data five years ago that we need today to do machine learning. Today, your only limitations are your imagination and your AWS budget."
Why should people learn machine learning skills?
So should you learn machine learning?
According to Forbes, “Machine Learning Engineers, Data Scientists, and Big Data Engineers rank among the top emerging jobs on LinkedIn. Data scientist roles have grown over 650% since 2012, but currently, 35,000 people in the US have data science skills, while hundreds of companies are hiring for those roles.”
This is confirmed by LinkedIn’s 2017 U.S. Emerging Jobs Report which lists “Machine Learning Engineer” as the fastest-growing job.
If that wasn’t enough, “[over] the past four years, six companies in particular—Google, Facebook, Apple, Amazon, Microsoft and the Chinese firm Baidu—have touched off an arms race for A.I. talent, particularly within universities... Starting salaries of seven figures are not unheard-of,” according to The New York Times.
There’s a clear and growing need for engineers in these specific fields. However, we want to shed light on the nuance that many headlines miss.
Building a facial recognition system or teaching a robot to recognize feelings requires an extremely advanced level of math, but if you’re just trying to learn enough about machine learning to apply simple clustering or regression methods, that’s a different story.
So, I asked Hillary: how long do you think it would take me to learn enough programming to do anything meaningful with machine learning?
“It depends on how you're working. I would say, a semester of college-level computer science, or a program where you’re doing it every single day for like six to eight weeks,” she said. “If you are not comfortable writing a short program in Python, if you don't know what
if statements and
for loops are, this is just going to be really painful for you. Basically, if you want to learn machine learning, you’re going to need to learn how to code.”
The language used as the basis for many machine learning algorithms is Python. It’s powerful, easy for beginners and has well-supported documentation. If you’re exploring machine learning, curious about your own ability to code, or even prepping for a course, you can take our free Python course. It will give you baseline skills while opening up the magical world of code.
Up next on Ask a Data Scientist: What’s AI?
Other questions for Hillary? Leave them in the comments below and we’ll add them to the queue.