Cloud Hosting

Last Updated on August 15, 2020

What are the basic concepts in machine learning?

I found that the best way to discover and get a handle on the basic concepts in machine learning is to review the introduction chapters tomachine learning textbooks and to watch the videos from the first model inonlinecourses.

Pedro Domingos is a lecturer and professor on machine learning at the University of Washing and author of a new book titled The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World.

Domingos has a free course on machine learning online at courser titled appropriately Machine Learning. The videos for each module can be previewedon Coursera any time.

In this post you will discover the basic concepts of machine learning summarized from Week One of Domingos Machine Learning course.

Basic Concepts in Machine LearningPhoto by Travis Wise, some rights reserved.

The first half of the lecture is on the general topic of machine learning.

Why do we need to care about machine learning?

A breakthrough in machine learning would be worth ten Microsofts.

Bill Gates, Former Chairman, Microsoft

Machine Learning is getting computers to program themselves. If programming is automation, then machine learning is automating the process of automation.

Writing software is the bottleneck, we dont have enough good developers. Let the data do the work instead of people. Machine learning is the way to make programming scalable.

Machine learning is like farming or gardening. Seeds is the algorithms, nutrientsis the data, thegardneris you and plants is the programs.

Traditional Programming vs Machine Learning

Sample applications of machine learning:

What is your domain of interest and how could you use machine learning in that domain?

There are tens of thousands of machine learning algorithms and hundreds of new algorithms are developed every year.

Every machine learning algorithm has three components:

All machine learning algorithms are combinations of these three components. A framework for understanding all algorithms.

There are four types of machine learning:

Supervised learning is the most mature, the most studied and the type of learning used bymost machine learning algorithms. Learning with supervision is much easier than learning without supervision.

Inductive Learning is where we are given examples of a function in the form of data (x) and the output of the function (f(x)). The goal of inductive learning is to learn the function for new data (x).

Machine learning algorithms are only a very small part of using machine learning in practice as a data analyst or data scientist. In practice, the process often looks like:

It is not a one-shot process, it is a cycle. You need to run the loop until you get a result that you can use in practice. Also, the data can change, requiring a new loop.

The second part of the lecture is on the topic of inductive learning. This is the general theory behind supervised learning.

From the perspective of inductive learning, we are given input samples (x) and output samples (f(x)) and the problem is to estimate the function (f). Specifically, the problem is to generalize from the samples and the mapping to be useful to estimate the output fornew samples in the future.

In practice it is almost always too hard to estimate the function, so we are looking for very good approximations of the function.

Some practical examples of induction are:

There are problems where inductive learning is not a good idea. It is important when to use and when not to use supervised machine learning.

4 problems where inductive learning might be a good idea:

We can write a program that works perfectly for the data that we have. This function will be maximally overfit. But we have no idea how well it will work on new data, it will likely be very badly because we may never see the same examples again.

The data is not enough. You canpredictanything you like. And this would be naive assume nothing about the problem.

In practice we are not naive. There is an underlying problem and we areinterested inan accurate approximation of the function. There is a double exponential number of possible classifiers in the number of input states. Finding a good approximate for the function is verydifficult.

There are classes of hypotheses that we can try. That is the form that the solution may take or the representation. We cannot know which is most suitable for our problem before hand. We have to use experimentation to discover what works on the problem.

Two perspectives on inductive learning:

You could be wrong.

In practice we start with a small hypothesis class and slowly grow the hypothesis class until we get a good result.

Terminology used inmachine learning:

Key issues in machine learning:

There are 3concerns for a choosing a hypothesis spacespace:

There are 3properties by which you could choose an algorithm:

In this post you discovered the basic concepts in machine learning.

In summary, these were:

These are the basic concepts that are covered in the introduction to most machine learning courses and in the opening chapters of any good textbook on the topic.

Although targeted at academics,as a practitioner, it is useful to have a firm footingin these concepts in order to better understand how machine learning algorithmsbehave in the general sense.

The rest is here:
Basic Concepts in Machine Learning