Multi-class classification is not the same as multi-label classification.
Yet many people starting with machine learning don't understand the difference.
Let's solve that problem once and for all.
We have many pictures of animals.
We want to classify them into three different classes based on the animal: a cat, a dog, or a chick.
(In our hypothetical world, there are only these three animal species.)
This problem, as I just presented it to you, is a multi-class classification problem:
• We have samples (images)
• Each sample belongs to a class
We assume that a picture is either a cat, a dog, or a chick, but not a combination of them!
Let's change the problem now.
A picture may show more than one animal: a cat and a dog, a dog, and a chick, or all three animals.
We now want to label each image with every single relevant class.
This is no longer multi-class but multi-label.
A quick summary:
1. If the classes are mutually exclusive: the problem is multi-class.
2. If they aren't mutually exclusive: the problem is multi-label.
Understanding the type of problem is crucial because the way you solve these two problems is entirely different.
For example, if you were building a neural network, you'll need different optimization and output activations to solve both problems.