Image by Editor (Kanwal Mehreen) | Canva
Let’s look at how to use the Scikit-Learn Voting Classifier to combine different models effectively.
Preparation
We would need to install the Pandas and Scikit-Learn packages for this tutorial. We can do this with the following command
pip install pandas scikit-learn<
Next, we'll use the Iris example dataset and standardize the data to ensure everything is on the same scale.
from sklearn.datasets import load_iris from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler iris = load_iris() X, y = iris.data, iris.target X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) scaler = StandardScaler() X_train = scaler.fit_transform(X_train) X_test = scaler.transform(X_test)
With this example dataset, let’s try out the Voting Classifier.
Combine Multiple Models with a Voting Classifier
A Voting Classifier is an ensemble machine-learning model that combines many classifier models and uses a voting technique to provide the final prediction. It’s often used to combine model strengths to improve overall model performance. Although each prediction is not good, combining them together would bring better results.
There are two types in the Voting Classifier: Hard and Soft.
The Hard Voting Classifier uses the majority vote for each label predicted as the final prediction. For example, if two models predict the data as Churn while one model predicts it as Not-Churn, the Voting Classifier would have the final class as Churn.
Let’s try them out with code examples.
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC
from sklearn.ensemble import VotingClassifier
from sklearn.metrics import accuracy_score
log_reg = LogisticRegression()
knn = KNeighborsClassifier()
decision_tree = DecisionTreeClassifier()
voting_clf = VotingClassifier(estimators=[('lr', log_reg), ('knn', knn), ('dt', decision_tree)], voting='hard')
voting_clf.fit(X_train, y_train)
y_pred = voting_clf predict(X_test) print(classification_report(y_test, y_pred))
print(‘Accuracy_Score: ’, accuracy_score(y_test, y_pred))
Output:
Accuracy_Score: 1
On the other hand, the soft method would use all the model's probability and take the averages. Each model in the voting classifier gives its probability, and the final prediction is based on the class with the highest average.
voting_clf = VotingClassifier(estimators=[('lr', log_reg), ('knn', knn), ('dt', decision_tree)], voting='soft')
voting_clf.fit(X_train, y_train)
y_pred = ensemble_clf.predict(X_test)
print(‘Accuracy_Score: ’, accuracy_score(y_test, y_pred))
Output:
Accuracy_Score: 1
Experiment with combining the models and the voting type to achieve the best model performance.
