Supervised learning can be separated into two types of problems when data mining: classification and regression.

• Classification uses an algorithm to accurately assign test data into specific categories. It recognizes specific entities within the dataset and attempts to draw some conclusions on how those entities should be labeled or defined. Common classification algorithms are linear classifiers, support vector machines (SVM), decision trees, K-nearest neighbor and random forest, which are described in more detail below.

• Regression is used to understand the relationship between dependent and independent variables. It is commonly used to make projections, such as sales revenue for a given business. Linear regression, logistical regression, and polynomial regression are popular regression algorithms.

Various algorithms and computations techniques are used in supervised machine learning processes, often calculated through use of programs such as Python. Supervised learning algorithms include:

• AdaBoost or gradient boosting: Also called adaptive boosting⁷, this technique boosts an underperforming regression algorithm by combining it with weaker ones to create a stronger algorithm that results in fewer errors. Boosting combines the forecasting power of several base estimators.

• Artificial neural networks: Also known as ANNs, neural networks or simulated neural networks (SNNs), are a subset of machine learning techniques and are at the heart of deep learning algorithms. The learner algorithm recognizes patterns in input data using building blocks called neurons, approximating the neurons in the human brain, which are trained and modified over time. (More in “neural networks.”)

• Decision tree algorithms: Used for both predicting numerical values (regression problems) and classifying data into categories, decision trees use a branching sequence of linked decisions that may be represented with a tree diagram. One of the advantages of decision trees is that they are easy to validate and audit, unlike the black box of a neural network.

• Dimensionality reduction: When a selected data set has a high number of features⁷ , it has high dimensionality. Dimensionality reduction then cuts down the number of features, leaving only the most meaningful insights or information. An example is principal component analysis.

• K-nearest neighbor: Also known as KNN, this non-parametric algorithm classifies data points based on their proximity and association to other available data. It assumes that similar data points can be found near each other. As a result, it seeks to calculate the distance between data points, usually through Euclidean distance, and then it assigns a category based on the most frequent category or average. 

• Linear regression: Linear regression is used to identify the relationship between a dependent variable and one or more independent variables and is typically leveraged to make predictions about future outcomes. When there is only one independent variable and one dependent variable, it is known as simple linear regression.

• Logistic regression: While linear regression is leveraged when dependent variables are continuous, logistic regression is selected when the dependent variable is categorical, meaning there are binary outputs, such as "true" and "false" or "yes" and "no." While both regression models seek to understand relationships between data inputs, logistic regression is mainly used to solve binary classification problems, such as spam identification.

• Neural networks: Primarily leveraged for deep learning algorithms, neural networks process the input training data by mimicking the interconnectivity of the human brain through layers of nodes. Each node is made up of inputs, weights, a bias (threshold) and an output. If that output value exceeds a given threshold, it “fires” or activates the node, passing data to the next layer in the network. Neural networks learn from adjustments based on the loss function through the process of gradient descent. When the cost function is at or near zero, you can be confident in the model’s accuracy.

• Naïve Bayes: This approach adopts the principle of class conditional independence from the Bayes Theorem. This means that the presence of one feature does not impact the presence of another in the probability of a given outcome, and each predictor has an equal effect on that result. There are three types of Naïve Bayes classifiers: Multinomial Naïve Bayes, Bernoulli Naïve Bayes and Gaussian Naïve Bayes. This technique is primarily used in text classification, spam identification and recommendation systems.

• Random forests: In a random forest, the machine learning algorithm predicts a value or category by combining the results from a number of decision trees. The "forest" refers to uncorrelated decision trees, which are assembled to reduce variance and enable more accurate predictions.

• Support vector machines (SVM): This algorithm may be used for both data classification and regression, but typically for classification problems, constructing a hyperplane where the distance between two classes of data points is at its maximum. This hyperplane is known as the decision boundary, separating the classes of data points (such as oranges vs. apples) on either side of the plane.

Unlike supervised learning, unsupervised learning uses unlabeled data. From that data, the algorithm discovers patterns that help solve clustering or association problems. This is particularly useful when subject matter experts are unsure of common properties within a data set. Common clustering algorithms are hierarchical, K-means, Gaussian mixture models and Dimensionality Reduction Methods such as PCA and t-SNE.

• Clustering: These algorithms can identify patterns in data so that it can be grouped. Algorithms can help data scientists by identifying differences between data items that humans have overlooked.

• Hierarchical clustering: This groups data into a tree of clusters⁸. Hierarchical clustering begins by treating every data point as a separate cluster. Then, it repeatedly executes these steps:  1) identify the two clusters which can be closest together, and 2) merge the two maximum comparable clusters. These steps continue until all the clusters are merged together.

• K-means clustering: This identifies groups within data without labels⁹  into different clusters by finding groups of data which are similar to one another. The name “K-means” come from the $k$ centroids that it uses to define clusters. A point is assigned to a particular cluster if it is closer to that cluster's centroid than any other centroid.

Semi-supervised learning algorithms

In this case, learning occurs when only part of the given input data has been labeled—giving the algorithm a bit of a “head start.” This approach can combine the best of both worlds¹⁰—improved accuracy associated with supervised machine learning and the ability to make use of cost-effective unlabeled data, as in the case of unsupervised machine learning.

Reinforcement algorithms

In this case, the algorithms are trained just as humans learn—through rewards and penalties—which are measured and tracked by a reinforcement learning agent¹¹ which has a general understanding of the probability of successfully moving the score up vs. moving it down. Through trial and error, the agent learns to take actions that lead to the most favorable outcomes over time. Reinforcement learning is often used¹²  in resource management, robotics and video games.