Skip to content

Lack of tag in classifiers causes problems for scorers using predict_proba #1129

@Heptasphere

Description

@Heptasphere

Describe the bug

A scorer created through scikit-learn which uses predict_proba can't be used with classifiers from mlxtend.classifiers

Classifiers are tagged by default as regressors for scikit-learn with versions above 1.6.
This interferes with scikit-learn's scorers using predict_proba.

Similar problem as rasbt/machine-learning-book#205.

Steps/Code to Reproduce

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.metrics import roc_auc_score, make_scorer
from mlxtend.classifier import MultiLayerPerceptron

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)

roc_auc_scorer = make_scorer(
    roc_auc_score,
    response_method="predict_proba",
    multi_class="ovo",
)

model = MultiLayerPerceptron()
model.fit(X_train, y_train)

result = roc_auc_scorer(model)
print(result)

Expected Results

A float between 0 and 1 is printed, showing the performance of the model.

Actual Results

ValueError: MultiLayerPerceptron should either be a classifier to be used with response_method=predict_proba or the response_method should be 'predict'. Got a regressor with response_method=predict_proba instead.

Versions

MLxtend 0.23.4
Linux-6.8.0-79-generic-x86_64-with-glibc2.35
Python 3.12.11 (main, Jun 4 2025, 08:56:18) [GCC 11.4.0]
Scikit-learn 1.7.1
NumPy 2.3.2
SciPy 1.16.1

Additional Context

A similar error occurs with EnsembleVoteClassifier, LogisticRegression SoftmaxRegression and StackingClassifier.

StackingCVClassifier had an entirely different error based on fit_params but it would likely suffer the same bug.

Adaline, OneRClassifier and Perceptron don't have a predict_proba method so it is not reproductible this way but they likely share this tag problem.

Partial Solution

The same solution of switching ClassifierMixin and BaseEstimator as in rasbt/machine-learning-book#205 works for EnsembleVoteClassifier and likely works for OneRClassifier.
Other classifiers use different mixin that don't seem to implement tags.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions