Background of decision trees

The concept of a decision tree is to select splits that decrease the impurity of class distribution in the resulting subsets of instances. At the same time, the domination of one or more classes over the other classes is increased.

This way, you can find a subset that contains only instances of one class after a few splits.

For decision trees, binary trees are preferred because they can be generalized better than trees with high-cardinality attributes.

Creating and using decision tree models comprises the following algorithmic subtasks for decision trees:

Growing
Pruning
Prediction