Cloud Hosting

This article is excerpted from the course "Fundamental Machine Learning," part of the Machine Learning Specialist certification program from Arcitura Education. It is the ninth part of the 13-part series, "Using machine learning algorithms, practices and patterns."

This article explores the numerical prediction and category prediction supervised learning techniques. These machine learning techniques are applied when the target whose value needs to be predicted is known in advance and some sample data is available to train a model. As explained in Part 4, these techniques are documented in a standard pattern profile format.

A data set may contain a number of historical observations (rows) amassed over a period of time where the target value is numerical in nature and is known for those observations. An example is the number of ice creams sold and the temperature readings, where the number of ice creams sold is the target variable. To obtain value from this data, a business use case might require a prediction of how much ice cream will be sold if the temperature reading is known in advance from the weather forecast. As the target is numerical in nature, supervised learning techniques that work with categorical targets cannot be applied (Figure 1).

The historical data is capitalized upon by first finding independent variables that influence the target dependent variable and then quantifying this influence in a mathematical equation. Once the mathematical equation is complete, the value of the target variable is predicted by inputting the values of the independent values.

The data set is first scanned to find the best independent variables by applying the associativity computation pattern to find the relationship between the independent variables and the dependent variable. Only the independent variables that are highly correlated with the dependent variable are kept. Next, linear regression is applied.

Linear regression, also known as least squares regression, is a statistical technique for predicting the values of a continuous dependent variable based on the values of an independent variable. The dependent and independent variables are also known as response and explanatory variables, respectively. As a mathematical relationship between the response variable and the explanatory variables, linear regression assumes that a linear correlation exists between the response and explanatory variables. A linear correlation between response and explanatory variables is represented through the line of best fit, also called a regression line. This is a straight line that passes as closely as possible through all points on the scatter plot (Figure 2).

Linear regression model development starts by expressing the linear relationship. Once the mathematical form has been established, the next step is to estimate the parameters of the model via model fitting. This determines the line of best fit achieved via least squares estimation that aims to reduce the sum of squared error (SSE). The last stage is to evaluate the model either using R squared or mean squared error (MSE).

MSE is a measure that determines how close the line of best fit is to the actual values of the response variable. Being a straight line, the regression line cannot pass through each point; it is an approximation of the actual value of the response variable based on estimated values. The distance between the actual and the estimated value of response variable is the error of estimation. For the best possible estimate of the response variable, the errors between all points, as represented by the sum of squared error, must be minimized. The line of best fit is the line that results in the minimum possible sum of squares errors. In other words, MSE identifies the variation between the actual value and the estimated value of the response variable as provided by the regression line (Figure 3).

The coefficient of determination, called R squared, is the percentage of variation in the response variable that is predicted or explained by the explanatory variable, with values that vary between 0 and 1. A value equal to 0 means that the response variable cannot be predicted from the explanatory variable, while a value equal to 1 means the response variable can be predicted without any errors. A value between 0 and 1 provides the percentage of successful prediction.

In regression, more than two explanatory variables can be used simultaneously for predicting the response variable, in which case it is called multiple linear regression.

The numerical prediction pattern can benefit from the application of the graphical summaries computation pattern by drawing a scatter plot to graphically validate if a linear relationship exists between the response and explanatory variables (Figure 4).

There are cases where a business problem involves predicting a category -- such as whether a customer will default on their loan or whether an image is a cat or a dog -- based on historical examples of defaulters and cats and dogs, respectively. In this case, the categories (default/not default and cat/dog) are known in advance. However, as the target class is categorical in nature, numerical predictive algorithms cannot be applied to train and predict a model for classification purposes (Figure 5).

Supervised machine learning techniques are applied by selecting a problem-specific machine learning algorithm and developing a classification model. This involves first using the known example data to train a model. The model is then fed new unseen data to find out the most appropriate category to which the new data instance belongs.

Different machine learning algorithms exist for developing classification models. For example, naive Bayes is probabilistic while K-nearest neighbors (KNN), support vector machine (SVM), logistic regression and decision trees are deterministic in nature. Generally, in the case of a binary problem -- cat or dog -- logistic regression is applied. If the feature space is n-dimensional (a large number of features) with complex interactions between the features, KNN is applied. Naive Bayes is applied when there is not enough training data or fast predictions are required, while decision trees are a good choice when the model needs to be explainable.

Logistic regression is based on linear regression and is also considered a class probability estimation technique, since its objective is to estimate the probability of an instance belonging to a particular class.

KNN, also known as lazy learning and instance-based learning, is a black-box classification technique where instances are classified based on their similarity, with a user-defined (K) number of examples (nearest neighbors). No model is explicitly generated. Instead, the examples are stored as-is and an instance is classified by first finding the closest K examples in terms of distance, then assigning the class based on the class of the majority of the closest examples (Figure 6).

Naive Bayes is a probability-based classification technique that predicts class membership based on the previously observed probability of all potential features. This technique is used when a combination of a number of features, called evidence, affects the determination of the target class. Due to this characteristic, naive Bayes can take into account features that may be insignificant when considered on their own but when considered accumulatively can significantly impact the probability of an instance belonging to a certain class.

All features are assumed to carry equal significance, and the value of one feature is not dependent on the value of any other feature. In other words, the features are independent. It serves as a baseline classifier for comparing more complex algorithms and can also be used for incremental learning, where the model is updated based on new example data without the need for regenerating the whole model from scratch.

A decision tree is a classification algorithm that represents a concept in the form of a hierarchical set of logical decisions with a tree-like structure that is used to determine the target value of an instance. [See discussion of decision trees in part 2 of this series.] Logical decisions are made by performing tests on the feature values of the instances in such a way that each test further filters the instance until its target value or class membership is known. A decision tree resembles a flowchart consisting of decision nodes, which perform a test on the feature value of an instance, and leaf nodes, also known as terminal nodes, where the target value of the instance is determined as a result of traversal through the decision nodes.

The category prediction pattern normally requires the application of a few other patterns. In the case of logistic regression and KNN, applying the feature encoding pattern ensures that all features are numerical as these two algorithms only work with numerical features. The application of the feature standardization pattern in the case of KNN ensures that none of the large magnitude features overshadow smaller magnitude features in the context of distance measurement. Naive Bayes requires the application of the feature discretization pattern as naive Bayes only works with nominal features. KNN can also benefit from the application of feature discretization pattern via a reduction in feature dimensionality, which contributes to faster execution and increased generalizability of the model.

The next article covers the category discovery and pattern discovery unsupervised learning patterns.

Read the original post:
2 supervised learning techniques that aid value predictions - TechTarget

Just about everyone agrees that data scientists and AI developers are the new superstars of the tech industry. But ask a group of CIOs to define the precise area of expertise for data science-related job titles, and discord becomes the word of the day.

As businesses seek actionable insights by hiring teams that include data analysts, data engineers, data scientists, machine learning engineers and deep learning engineers, a key to success is understanding what each role can and cant do for the business.

Read on to learn what your data science and AI experts can be expected to contribute as companies grapple with ever-increasing amounts of data that must be mined to create new paths to innovation.

In a perfect world, every company employee and executive works under a well-defined set of duties and responsibilities.

Data science isnt that world. Companies often will structure their data science organization based on project need: Is the main problem maintaining good data hygiene? Or is there a need to work with data in a relational model? Perhaps the team requires someone to be an expert in deep learning, and to understand infrastructure as well as data?

Depending on a companys size and budget, any one job title might be expected to own one or more of these problem-solving skills. Of course, roles and responsibilities will change with time, just as theyve done as the era of big data evolves into the age of AI.

That said, its good for a CIO and the data science team she or he is managing today to remove as much of the ambiguity as possible regarding roles and responsibilities for some of the most common roles those of the data analyst, data engineer, data scientist, machine learning engineer and deep learning engineer.

Teams that have the best understanding of how each fits into the companys goals are best positioned to deliver a successful outcome. No matter the role, accelerated computing infrastructure is also key to powering success throughout the pipeline as data moves from analytics to advanced AI.

Its important to recognize the work of a data analyst, as these experts have been helping companies extract information from their data long before the emergence of the modern data science and AI pipeline.

Data analysts use standard business intelligence tools like Microsoft Power BI, Tableau, Qlik, Yellowfin, Spark, SQL and other data analytics applications. Broad-scale data analytics can involve the integration of many different data sources, which increases the complexity of the work of both data engineers and data scientists another example of how the work of these various specialists tends to overlap and complement each other.

Data analysts still play an important role in the business, as their work helps the business assess its success. A data engineer might also support a data analyst who needs to evaluate data from different sources.

Data scientists take things a step further so that companies can start to capitalize on new opportunities with recommender systems, conversational AI, and computer vision, to name a few examples.

A data engineer makes sense of messy data and theres usually a lot of it. People in this role tend to be junior teammates who make data nice and neat (as possible) for data scientists to use. This role involves a lot of data prep and data hygiene work, including lots of ETL (extract, transform, load) to ingest and clean data.

The data engineer must be good with data jigsaw puzzles. Formats change, standards change, even the fields a team is using on a webpage can change frequently. Datasets can have transmission errors, such as when data from one field is incorrectly entered into another.

When datasets need to be joined together, data engineers need to fix the data hygiene problems that occur when labeling is inconsistent. For example, if the day of the week is included in the source data, the data engineer needs to make sure that the same format is used to indicate the day, as Monday could also be written as Mon., or even represented by a number that could be one or zero depending on how the days of the week are counted.

Expect your data engineers to be able to work freely with scripting languages like Python, and in SQL and Spark. Theyll need programming language skills to find problems and clean them up. Given that theyll be working with raw data, their work is important to ensuring your pipeline is robust.

If enterprises are pulling data from their data lake for AI training, this rule-based work can be done by a data engineer. More extensive feature engineering is the work of a data scientist. Depending on their experience and the project, some data engineers may support data scientists with initial data visualization graphs and charts.

Depending on how strict your company has been with data management, or if you work with data from a variety of partners, you might need a number of data engineers on the team. At many companies, the work of a data engineer often ends up being done by a data scientist, who preps her or his own data before putting it to work.

Data scientists experiment with data to find the secrets hidden inside. Its a broad field of expertise that can include the work of data analytics and data processing, but the core work of a data scientist is done by applying predictive techniques to data using statistical machine learning or deep learning.

For years, the IT industry has talked about big data and data lakes. Data scientists are people who finally turn these oceans of raw data into information. These experts use a broad range of tools to conduct analytics, experiment, build and test models to find patterns. To be great at their work, data scientists also need to understand the needs of the business theyre supporting.

These experts use many applications, including NumPy, SciKit-Learn, RAPIDS, CUDA, SciPy, Matplotlib, Pandas, Plotly, NetworkX, XGBoost, domain-specific libraries and many more. They need to have domain expertise in statistical machine learning, random forests, gradient boosting, packages, feature engineering, training, model evaluation and refinement, data normalization and cross-validation. The depth and breadth of these skills make it readily apparent why these experts are so highly valued at todays data-driven companies.

Data scientists often solve mysteries to get to the deeper truth. Their work involves finding the simplest explanations for complex phenomena and building models that are simple enough to be flexible yet faithful enough to provide useful insight. They must also avoid some perils of model training, including overfitting their data sets (that is, producing models that do not effectively generalize from example data) and accidentally encoding hidden biases into their models.

A machine learning engineer is the jack of all trades. This expert architects the entire process of machine and deep learning. They take AI models developed by data scientists and deep learning engineers and move them into production.

These unicorns are among the most sought-after and highly paid in the industry and companies work hard to make sure they dont get poached. One way to keep them happy is to provide the right accelerated computing resources to help fuel their best work. A machine learning engineer has to understand the end-to-end pipeline, and they want to ensure that pipeline is optimized to deliver great results, fast.

Its not always easily intuitive, as the machine learning engineers must know the apps, understand the downstream data architecture, and key in on system issues that may arise as projects scale. A person in this role must understand all the applications used in the AI pipeline, and usually needs to be skilled in infrastructure optimization, cloud computing, containers, databases and more.

To stay current, AI models need to be reevaluated to avoid whats called model drift as new data impacts the accuracy of the predictions. For this reason, machine learning engineers need to work closely with their data science and deep learning colleagues who will need to reassess models to maintain their accuracy.

A critical specialization for the machine learning engineer is deep learning engineer. This person is a data scientist who is an expert in deep learning techniques. In deep learning, AI models are able to learn and improve their own results through neural networks that imitate how human beings think and learn.

These computer scientists specialize in advanced AI workloads. Their work is part science and part art to develop what happens in the black box of deep learning models. They do less feature engineering and far more math and experimentation. The push for explainable AI (XAI) model interpretability and explainability can be especially challenging in this domain.

Deep learning engineers will need to process large datasets to train their models before they can be used for inference, where they apply what theyve learned to evaluate new information. They use libraries like PyTorch, TensorFlow and MXNet, and need to be able to build neural networks and have strong skills in statistics, calculus and linear algebra.

Given all of the broad expertise in these key roles, its clear that enterprises need a strategy to help them grow their teams success in data science and AI. Many new applications need to be supported, with the right resources in place to help this work get done as quickly as possible to solve business challenges.

Those new to data science and AI often choose to get started with accelerated computing in the cloud, and then move to a hybrid solution to balance the need for speed with operational costs. In-house teams tend to look like an inverted pyramid, with more analysts and data engineers funneling data into actionable tasks for data scientists, up to the machine learning and deep learning engineers.

Your IT paradigm will depend on your industry and its governance, but a great rule of thumb is to ensure your vendors and the skills of your team are well aligned. With a better understanding of the roles of a modern data team, and the resources they need to be successful, youll be well on your way to building an organization that can transform data into business value.

ABOUT THE AUTHOR

By Scott McClellan, Head of Data Science, NVIDIA

Read more here:
The CIO's Guide to Building a Rockstar Data Science and AI Team | eWEEK - eWeek