-
Notifications
You must be signed in to change notification settings - Fork 889
Description
Describe the bug
A scorer created through scikit-learn which uses predict_proba can't be used with classifiers from mlxtend.classifiers
Classifiers are tagged by default as regressors for scikit-learn with versions above 1.6.
This interferes with scikit-learn's scorers using predict_proba.
Similar problem as rasbt/machine-learning-book#205.
Steps/Code to Reproduce
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.metrics import roc_auc_score, make_scorer
from mlxtend.classifier import MultiLayerPerceptron
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)
roc_auc_scorer = make_scorer(
roc_auc_score,
response_method="predict_proba",
multi_class="ovo",
)
model = MultiLayerPerceptron()
model.fit(X_train, y_train)
result = roc_auc_scorer(model)
print(result)Expected Results
A float between 0 and 1 is printed, showing the performance of the model.
Actual Results
ValueError: MultiLayerPerceptron should either be a classifier to be used with response_method=predict_proba or the response_method should be 'predict'. Got a regressor with response_method=predict_proba instead.
Versions
MLxtend 0.23.4
Linux-6.8.0-79-generic-x86_64-with-glibc2.35
Python 3.12.11 (main, Jun 4 2025, 08:56:18) [GCC 11.4.0]
Scikit-learn 1.7.1
NumPy 2.3.2
SciPy 1.16.1
Additional Context
A similar error occurs with EnsembleVoteClassifier, LogisticRegression SoftmaxRegression and StackingClassifier.
StackingCVClassifier had an entirely different error based on fit_params but it would likely suffer the same bug.
Adaline, OneRClassifier and Perceptron don't have a predict_proba method so it is not reproductible this way but they likely share this tag problem.
Partial Solution
The same solution of switching ClassifierMixin and BaseEstimator as in rasbt/machine-learning-book#205 works for EnsembleVoteClassifier and likely works for OneRClassifier.
Other classifiers use different mixin that don't seem to implement tags.