In machine learning, we need to check how well a model works, and cross-validation is a common way to do this. It tests the model on different parts of the data to see how it performs on new data. PyCaret is a Python library that makes this process easy. It helps you compare multiple models fast.
In this article, we will show how to use PyCaret to compare models using cross-validation. You will learn how to evaluate models and pick the best one for your data.
Importing and Loading Data
First, you need to install PyCaret. You can do this by running:
pip install pycaret
After installation, import the necessary modules and load your dataset. For example, we’ll use PyCaret’s built-in diabetes dataset.
from pycaret.datasets import get_data
from pycaret.classification import *
# Load the diabetes dataset
dataset = get_data('diabetes')
Setting Up PyCaret
In PyCaret, the setup() function helps you set up everything you need to get started with machine learning. This function prepares your data by handling missing values, encoding categorical features, and splitting the data into training and testing sets. The function is essential for starting any machine learning task in PyCaret.
In this setup, the data is the dataset you’re using for your machine learning task, and the target is the name of the column that you want to predict.
# Set up the data for classification task exp1 = setup(data=dataset, target='Class variable')
Comparing Models
The compare_models() function in PyCaret lets you compare many different models, like logistic regression, decision trees, random forests, KNN, and SVM. PyCaret automatically trains and tests these models on your data, and then shows you the results.
After you run compare_models(), you’ll see a table with how each model performed. The table includes important numbers like accuracy, AUC, and F1 score to help you choose the best model.
# Compare models with cross-validation best_model = compare_models()
Comparing Only a Few Models
As stated above, the compare_models() function lets you compare multiple models at once. However, if you only want to compare a few specific models, you can use the include parameter. For example, to compare only logistic regression, decision tree, and random forest, you can do the following:
from pycaret.classification import * # Compare specific models best_models = compare_models(include=['lr', 'dt', 'rf'])
Returning More Than One Model
By default, compare_models() returns only the best model. But if you want to return more than one top model, you can use the n_select parameter. For example, to return the top 3 models, use:
best_models = compare_models(n_select=3) print(best_models)
Changing the Fold Parameter
You can customize the number of folds in cross-validation using the fold argument in compare_models(). In this example, fold=5 means that 5-fold cross-validation will be used instead of the default 10-fold. This is helpful when working with smaller datasets. It can also help balance the training time and the accuracy of the evaluation.
# Use 5-fold cross-validation best_model = compare_models(fold=5)
Cross-Validation Score
The compare_models() function is a quick way to compare models using cross-validation. If you want more detailed scores, you can use the cross_val_score() function. It gives you more control over the evaluation process. You can choose how many folds to use and adjust other settings.
from pycaret.classification import cross_val_score # Calculate cross-validation score of a specific model cv_score = cross_val_score(best_model, folds=5) print(cv_score)
[0.67532468 0.68181818 0.69480519 0.77777778 0.7124183 ]
Conclusion
PyCaret makes it easy to evaluate models using cross-validation. The compare_models() function lets you quickly compare different models, and shows how well they perform on new data. Cross-validation ensures the models work well with unseen data. PyCaret also has tools like tune_model() and evaluate_model() to improve this process. These tools help fine-tune the model and check its performance in detail with little effort. PyCaret is a great tool for machine learning. It helps you pick the best model and make it better.
