OpenML-CC18: Fashion-MNIST

Best predictive accuracy per machine-learning flow on the Fashion-MNIST classification task from the OpenML-CC18 suite.

Community results for the Fashion-MNIST classification task from OpenML-CC18, OpenML's curated suite of 72 classification tasks (www.openml.org/t/146825). Each subject is a flow — a specific algorithm or pipeline — shown with the best predictive accuracy recorded for it on this task in OpenML's public evaluation listing, under the task's fixed estimation procedure.

Results

predictive_accuracy per subject over 60 measurements, ranked by average.

SubjectAvgMedianMinMaxP95P99Count
keras.wrappers.scikit_learn.KerasClassifier(Reshape,Conv2D,Conv2D,MaxPooling2D,BatchNormalization,Conv2D,Conv2D,MaxPooling2D,BatchNormalization,Conv2D,Conv2D,Conv2D,MaxPooling2D,BatchNormalization,Fl…0.940.940.940.940.940.941
sklearn-pipeline-pipeline-simpleimputer-sklearn-impute-base-simpleimputer-histgr0.91480.91480.91480.91480.91480.91481
torch-nn-modules-container-sequential-8a53697fda1551c2-10.9130.9130.9130.9130.9130.9131
keras.wrappers.scikit_learn.KerasClassifier(Reshape,Conv2D,Conv2D,MaxPooling2D,BatchNormalization,Conv2D,Conv2D,MaxPooling2D,BatchNormalization,Conv2D,Conv2D,Conv2D,MaxPooling2D,BatchNormalization,Fl…0.91230.91230.91230.91230.91230.91231
automlbenchmark_h2oautoml(1)0.90870.90870.90870.90870.90870.90871
sklearn.model_selection._search_successive_halving.HalvingRandomSearchCV(estimator=sklearn.ensemble._hist_gradient_boosting.gradient_boosting.HistGradientBoostingClassifier)(4)0.9060.9060.9060.9060.9060.9061
sklearn.model_selection._search.RandomizedSearchCV(estimator=sklearn.ensemble._hist_gradient_boosting.gradient_boosting.HistGradientBoostingClassifier)(3)0.90130.90130.90130.90130.90130.90131
sklearn.pipeline.Pipeline(imputation=openmlstudy14.preprocessing.ConditionalImputer,hotencoding=sklearn.preprocessing.data.OneHotEncoder,scaling=sklearn.preprocessing.data.StandardScaler,variencethre…0.89510.89510.89510.89510.89510.89511
keras.engine.sequential.Sequential.5F750FE4F9E4AAE0(1)0.89180.89180.89180.89180.89180.89181
sklearn.pipeline.Pipeline(imputation=hyperimp.utils.preprocessing.ConditionalImputer2,hotencoding=sklearn.preprocessing.data.OneHotEncoder,variencethreshold=sklearn.feature_selection.variance_thresho…0.88910.88910.88910.88910.88910.88911
automlbenchmark_tunedrandomforest(1)0.88810.88810.88810.88810.88810.88811
automlbenchmark_randomforest(1)0.88710.88710.88710.88710.88710.88711
sklearn-pipeline-pipeline-simpleimputer-sklearn-impute-base-simpleimputer-standa0.88620.88620.88620.88620.88620.88621
sklearn.ensemble._hist_gradient_boosting.gradient_boosting.HistGradientBoostingClassifier(6)0.88480.88480.88480.88480.88480.88481
sklearn.ensemble.forest.RandomForestClassifier(32)0.88450.88450.88450.88450.88450.88451
automlbenchmark_autosklearn(1)0.8820.8820.8820.8820.8820.8821
keras.wrappers.scikit_learn.KerasClassifier(2)0.88040.88040.88040.88040.88040.88041
keras.wrappers.scikit_learn.KerasClassifier(InputLayer,Reshape,Conv2D,MaxPooling2D,Conv2D,Conv2D,Conv2D,Concatenate,Activation,Conv2D,Conv2D,Conv2D,Concatenate,Activation,Conv2D,Conv2D,Conv2D,Concate…0.87440.87440.87440.87440.87440.87441
keras.wrappers.scikit_learn.KerasClassifier(Reshape,Conv2D,MaxPooling2D,BatchNormalization,Conv2D,MaxPooling2D,BatchNormalization,Conv2D,Conv2D,Conv2D,MaxPooling2D,BatchNormalization,Flatten,Dense,Dr…0.87420.87420.87420.87420.87420.87421
keras.wrappers.scikit_learn.KerasClassifier(InputLayer,Reshape,Conv2D,MaxPooling2D,Conv2D,Conv2D,Conv2D,Merge,Activation,Conv2D,Conv2D,Conv2D,Merge,Activation,Conv2D,Conv2D,Conv2D,Merge,Activation,Ma…0.86490.86490.86490.86490.86490.86491
sklearn.ensemble.forest.RandomForestClassifier(35)0.8610.8610.8610.8610.8610.8611
sklearn.neighbors.classification.KNeighborsClassifier(34)0.86020.86020.86020.86020.86020.86021
sklearn.neighbors.classification.KNeighborsClassifier(25)0.85410.85410.85410.85410.85410.85411
sklearn.linear_model._logistic.LogisticRegression(2)0.85180.85180.85180.85180.85180.85181
sklearn.linear_model._logistic.LogisticRegression(1)0.85180.85180.85180.85180.85180.85181

…and 35 more subjects.

Metrics

  • predictive_accuracy — recorded on each measurement. Best predictive accuracy this flow achieved on the task: the fraction of test instances classified correctly, from 0 to 1. Higher is better

Subjects (60)

  • automlbenchmark_autosklearn(1)
  • automlbenchmark_autoweka(1)
  • automlbenchmark_h2oautoml(1)
  • automlbenchmark_randomforest(1)
  • automlbenchmark_tpot(1)
  • automlbenchmark_tunedrandomforest(1)
  • bsplitz.DecisionStumpClassifierV3(1)
  • keras.engine.sequential.Sequential.5F750FE4F9E4AAE0(1)
  • keras.wrappers.scikit_learn.KerasClassifier(2)
  • keras.wrappers.scikit_learn.KerasClassifier(InputLayer,Reshape,Conv2D,MaxPooling2D,Conv2D,Conv2D,Conv2D,Concatenate,Activation,Conv2D,Conv2D,Conv2D,Concatenate,Activation,Conv2D,Conv2D,Conv2D,Concate…
  • keras.wrappers.scikit_learn.KerasClassifier(InputLayer,Reshape,Conv2D,MaxPooling2D,Conv2D,Conv2D,Conv2D,Merge,Activation,Conv2D,Conv2D,Conv2D,Merge,Activation,Conv2D,Conv2D,Conv2D,Merge,Activation,Ma…
  • keras.wrappers.scikit_learn.KerasClassifier(InputLayer,Reshape,Reshape,ZeroPadding2D,Conv2D,Conv2D,BatchNormalization,Activation,Conv2D,BatchNormalization,Activation,Conv2D,Conv2D,Add,BatchNormalizat…
  • keras.wrappers.scikit_learn.KerasClassifier(InputLayer,Reshape,ZeroPadding2D,Conv2D,Conv2D,BatchNormalization,Activation,Conv2D,BatchNormalization,Add,Activation,Conv2D,BatchNormalization,Activation,…
  • keras.wrappers.scikit_learn.KerasClassifier(InputLayer,Reshape,ZeroPadding2D,Conv2D,Conv2D,BatchNormalization,Activation,Conv2D,BatchNormalization,Add,Activation,Conv2D,BatchNormalization,Activation,…
  • keras.wrappers.scikit_learn.KerasClassifier(Reshape,Conv2D,Conv2D,MaxPooling2D,BatchNormalization,Conv2D,Conv2D,MaxPooling2D,BatchNormalization,Conv2D,Conv2D,Conv2D,MaxPooling2D,BatchNormalization,Fl…
  • keras.wrappers.scikit_learn.KerasClassifier(Reshape,Conv2D,Conv2D,MaxPooling2D,BatchNormalization,Conv2D,Conv2D,MaxPooling2D,BatchNormalization,Conv2D,Conv2D,Conv2D,MaxPooling2D,BatchNormalization,Fl…
  • keras.wrappers.scikit_learn.KerasClassifier(Reshape,Conv2D,MaxPooling2D,Activation,Conv2D,MaxPooling2D,Activation,Dropout,Conv2D,Flatten,Dense,Activation,Dense,Activation)(3)
  • keras.wrappers.scikit_learn.KerasClassifier(Reshape,Conv2D,MaxPooling2D,Activation,Conv2D,MaxPooling2D,Activation,Dropout,Conv2D,Flatten,Dense,Activation,Dense,Activation)(5)
  • keras.wrappers.scikit_learn.KerasClassifier(Reshape,Conv2D,MaxPooling2D,Activation,Conv2D,MaxPooling2D,Activation,Dropout,Conv2D,Flatten,Dense,Activation,Dense,Activation)(2)
  • keras.wrappers.scikit_learn.KerasClassifier(Reshape,Conv2D,MaxPooling2D,BatchNormalization,Conv2D,MaxPooling2D,BatchNormalization,Conv2D,Conv2D,Conv2D,MaxPooling2D,BatchNormalization,Flatten,Dense,Dr…
  • mlr.classif.rpart(47)
  • sklearn.discriminant_analysis.LinearDiscriminantAnalysis(2)
  • sklearn.ensemble.forest.RandomForestClassifier(32)
  • sklearn.ensemble.forest.RandomForestClassifier(35)
  • sklearn.ensemble.forest.RandomForestClassifier(57)
  • sklearn.ensemble.gradient_boosting.GradientBoostingClassifier(22)
  • sklearn.ensemble._hist_gradient_boosting.gradient_boosting.HistGradientBoostingClassifier(6)
  • sklearn.ensemble._weight_boosting.AdaBoostClassifier(1)
  • sklearn.linear_model._logistic.LogisticRegression(1)
  • sklearn.linear_model._logistic.LogisticRegression(2)
  • sklearn.linear_model.logistic.LogisticRegression(29)
  • sklearn.linear_model.logistic.LogisticRegression(30)
  • sklearn.model_selection._search.RandomizedSearchCV(estimator=sklearn.ensemble._hist_gradient_boosting.gradient_boosting.HistGradientBoostingClassifier)(3)
  • sklearn.model_selection._search_successive_halving.HalvingRandomSearchCV(estimator=sklearn.ensemble._hist_gradient_boosting.gradient_boosting.HistGradientBoostingClassifier)(4)
  • sklearn.naive_bayes.GaussianNB(8)
  • sklearn.neighbors.classification.KNeighborsClassifier(25)
  • sklearn.neighbors.classification.KNeighborsClassifier(34)
  • sklearn.neural_network.multilayer_perceptron.MLPClassifier(16)
  • sklearn.pipeline.Pipeline(columntransformer=sklearn.compose._column_transformer.ColumnTransformer(simpleimputer=sklearn.impute._base.SimpleImputer,onehotencoder=sklearn.preprocessing._encoders.OneHot…
  • sklearn.pipeline.Pipeline(feature_map=sklearn_extra.kernel_approximation._fastfood.Fastfood,svm=sklearn.svm.classes.LinearSVC)(1)
  • sklearn.pipeline.Pipeline(imputation=hyperimp.utils.preprocessing.ConditionalImputer2,hotencoding=sklearn.preprocessing.data.OneHotEncoder,variencethreshold=sklearn.feature_selection.variance_thresho…
  • sklearn.pipeline.Pipeline(imputation=openmlstudy14.preprocessing.ConditionalImputer,hotencoding=sklearn.preprocessing.data.OneHotEncoder,variencethreshold=sklearn.feature_selection.variance_threshold…
  • sklearn.pipeline.Pipeline(imputation=openmlstudy14.preprocessing.ConditionalImputer,hotencoding=sklearn.preprocessing.data.OneHotEncoder,scaling=sklearn.preprocessing.data.StandardScaler,variencethre…
  • sklearn.pipeline.Pipeline(imputation=sklearn.preprocessing.imputation.Imputer,estimator=sklearn.naive_bayes.GaussianNB)(1)
  • sklearn.pipeline.Pipeline(imputation=sklearn.preprocessing.imputation.Imputer,estimator=sklearn.tree.tree.DecisionTreeClassifier)(1)
  • sklearn.pipeline.Pipeline(imputer=sklearn.impute._base.SimpleImputer,encoder=sklearn.preprocessing._encoders.OneHotEncoder,model=sklearn.tree._classes.DecisionTreeClassifier)(2)
  • sklearn.pipeline.Pipeline(imputer=sklearn.impute._base.SimpleImputer,estimator=sklearn.tree.tree.DecisionTreeClassifier)(1)
  • sklearn.pipeline.Pipeline(imputer=sklearn.impute._base.SimpleImputer,estimator=sklearn.tree.tree.DecisionTreeClassifier)(2)
  • sklearn.pipeline.Pipeline(imputer=sklearn.impute._base.SimpleImputer,estimator=sklearn.tree._classes.DecisionTreeClassifier)(3)
  • sklearn.pipeline.Pipeline(scale=sklearn.preprocessing.data.StandardScaler,SVC=sklearn.svm.classes.LinearSVC)(1)

…and 10 more.

Published by OpenML.