4 Types of Classification Tasks in Machine Learning

What is Classification in Predictive Modeling?

In machine learning, Classification refers to training a model on a labeled dataset to assign data points to classes or to classify new data points. Classification is a method of predictive modeling.

A few examples of Classification

  • Spam Filters: Spam filters use classification algorithms to recognize spam emails by learning their characteristics. Naïve-Bayes and Support Vector Machines are often used.
  • Image Classification: How does your phone know that you’re taking a picture of food? Through image classification. Convolutional Neural Networks (CNNs) are typically used to train these algorithms.
  • Fraud Detection: Fraud detection falls under anomaly detection. Anomaly detection algorithms can be trained using a variety of classification algorithms, with Naive Bayes and K-NN being the most common. Fraud detection works by classifying activities of “normal” and “outlier”. Outliers/anomalies trigger protective measures.

Datasets for Classification

Classification modeling requires large datasets. More data = more accuracy, typically. In general, your model should have a training dataset that emphasizes quantity and relevance. The training datasets need to sufficiently resemble the problem and have as many examples of each class as possible. For example, if your goal is to train an email spam filter, you need to train the classification model on formal emails, shorthand, forwarded emails, promotional emails, notifications, order confirmations, and spam emails. The dataset needs to sufficiently cover all of these and more to be accurate.

Class Label Notation

Class labels are usually string values that are mapped to numeric values. For example, “spam” = 1, “not spam” = 0. (Word to the wise, be careful with your notation and make sure you fully document your mapping.)

Which Algorithm Should You Use?

There is no single algorithm that will work best for classification. Like most things in machine learning, you’re pulling from your toolbox of algorithms and will need to gauge which one fits your problem the best. In many cases, you will end up trying multiple different algorithms and comparing to figure out which one is the best. Need somewhere to start?
  • Random Forest: Easy to implement. Works well on datasets with a lot of variables.
  • Naïve-Bayes: Fast and works well on small datasets.
  • Logistic Regression: Simple logistic regression can work for classification. It isn’t as powerful as other methods and only works for binary variables
But before you pick an algorithm, you need to understand what type of classification you are working with. There are 4 main types of classification in machine learning.
  • Binary Classification
  • Multi-Class Classification
  • Multi-Label Classification
  • Imbalance Classification
Let’s take a closer look at each.

Binary Classification

Binary Classification refers to classification tasks that have two class labels. Typically, binary classification involves one class that is “normal” and another that is “abnormal”.

Examples of Binary Classification

  • Email spam filters (spam, not spam)
  • Conversion/purchase prediction (buy, no buy)
  • Fraud detection (fraud, not fraud)
  • Lending decisions (lend to, don’t lend to)

Binary Class Labeling

Binary classes are typically labeled as 0 and 1. 0 is assigned to the “abnormal” classification and 1 is assigned to the “normal” classification. In our spam filter example, “spam” = 0 and “not spam” = 1.

Why are Binary Classes Labeled as 0 and 1?

The Bernoulli Distribution! (Shocker.) The Bernoulli Distribution is a discrete probability distribution for events for binary outcomes. It predicts the probability that an event will happen or not happen. The Bernoulli Distribution uses 0 and 1. Hence why the notation is typically expanded to binary class labeling as well.

Popular Algorithms in Binary Classification

Binary class labeling typically uses simpler, faster algorithms. Here are a few to consider in your modeling.
  • Logistic Regression (only natively supports 2 classes)
  • Decisions Trees
  • k-Nearest Neighbors (kNNs)
  • Naïve Bates
  • Support Vector Machine (only natively supports 2 classes)

Multi-Class Classification

Multi-Class Classification refers to classification tasks that have more than two class labels. Multi-class classification doesn’t usually have a notion of “normal” or “abnormal”. Rather, examples are classes as being a part of one class among a range of known classes.

Examples of Multi-Class Classification

  • Color of flowers (red, blue, yellow)
  • Animal type (cat, dog, fish)
  • Bird species (finch, bluejay, hummingbird)
  • Clothing sizes (small, medium, large)

Multi-Class Labeling

Staying organized is important in multi-class labeling–especially when you have a large number of classes. Multi-class labeling usually starts at 0. (For example, “red” = 0, “blue” = 1, “yellow” = 2).

Categorical Distributions

Categorical Distributions (also called generalized Bernoulli distributions or Multinoulli distributions) is a discrete probability distribution that assigns probabilities to each of K possible classes. The probability of each class is specified separately by the distribution.

Popular Algorithms in Multi-Class Classification

Multi-class Classification can use a wide array of potential algorithms. Here are a few.
  • k-Nearest Neighbors (kNNs)
  • Decision Trees
  • Random Forest
  • Naïve-Bayes
  • Gradient Boosting

Adapting Binary Algorithms for Multi-Class

If you’re more familiar and comfortable with binary algorithms–like logistic regression or support vector machines–you may be in luck. These algorithms can be adapted for multi-class use. These strategies involve using one-vs-rest and one-vs-one.
  • One-vs-rest: each class vs. all other classes (Is the flower red or anything but red?)
  • One-vs-one: runs the binary algorithm multiple times for each possibility. (Is the flower red or blue? Is the flower red or yellow? Is the flower blue or yellow?

Multi-Label Classification

Examples of Multi-Label Classification

Multi-label classification is increasingly more common as we aim to understand more about datasets using AI. The most practical application is computer vision.
  • Items in a picture
  • Movie genre based on a movie poster
  • Disclaimers on videos (vulgar, graphic, offensive, profanity)
  • Flagging content on social media

Popular Algorithms in Multi-Label Classification

Multi-label classification can’t really pull from binary or multi-class algorithms. Rather, specialized versions of these standard formulas are utilized.
  • Multi-label Decision Trees
  • Multi-label Random Forests
  • Multi-label Gradient Boosting
Alternatively, you can predict the labels for each class with a separate classification algorithm. Need a deeper dive into multi-label classification? Read Towards Data Science’s guide and case study for their incredibly thorough article. (We learned a lot!)

Multi-Label vs Multi-Class–What’s the Difference?

Multi-label and multi-class classification can be difficult to distinguish. For example, favorite fruit vs liked fruits. If you have a dataset of 1,000 people who have been asked to rank apples, bananas, oranges, and grapes and you are seeking insight on only their favorite fruit, multi-class is appropriate. If you are seeking a model that shows how the top three most liked fruits relate, multi-label is appropriate. Another common example is in computer vision and image analysis. For example, if you are analyzing a dataset by facial expression/emotion to figure out the distribution between happy, sad, angry, and scared, multi-class is appropriate. If you are attempting to identify the crossover between sad and angry, multi-label is more appropriate.

Imbalanced Classification

Imbalanced Classification refers to classification where the datasets in each class are unequally distributed.

Examples of Imbalanced Classification

Imbalanced classification usually refers to a dataset where the majority of the data points are binary with a few important outliers.
  • Medical diagnostics tests
  • Claim Prediction
  • Finding outliers
  • Fraud detection
These examples do have some crossover with Binary Classification. Imbalanced Classification is used in these cases when your training data may lack examples of what you’re searching for or when the outliers are very important.

Sampling Techniques

In imbalanced classification, you may need to revisit your dataset. You can change the composition of the sample by undersample the majority class or oversampling minority classes.
Sampling Techniques for Imbalanced Classification
Over SamplingUnder Sampling
Random Over-SamplingRandom Under-Sampling
Random Over-Sample with imblearnRandom Under-Sample with imblearn
Tomek Links
Synthetic Minority Oversampling Technique (SMOTE)
Get more detail about how to best utilize these sampling techniques with this guide from Analytics Vidhya.

Popular Algorithms in Imbalanced Classification

Just like multi-label classification, the algorithms utilized in imbalanced classification need to be specialized to model correctly. You can use “cost-sensitive” algorithms to pay closer attention to minority classes. Logistic regression, decision trees, and support vector machine algorithms all have cost-sensitive specialized options.

Classification in Machine Learning

There are four main types of classification tasks in machine learning: binary, multi-class, multi-label, and imbalanced. We’ve given you a few examples and starter algorithms to get you started as you train your next model. Here at Elevate, we use multi-class and multi-label classification to train our computer vision edge-AI on a wide dataset of images.

You May Also Like

Subscribe to Our Newsletter

Join our mailing list for everything AI Vision, Machine Learning, Neural Networks, and more. Plus, freebies and coupon codes to Elevate products. (We promise we won't spam you!)

Subscribe to Our Newsletter

You have Successfully Subscribed!

Ready to Elevate Your Video Calls?

Ready to Elevate Your Video Calls?

Let us know where to send beta access to the Elevate App. Download in less than 5-minutes.

Beta Access is on the way!