General notes:

  • The description of methods given here is only a summary. For more information, always check the java docs.

  • All examples assume that the classpath is already set, e.g., by using the command line flag -cp target/meka-1.7.6.jar:lib/* (in Linux). See the Tutorial for more information.

  • m indicates a meta method, can be used with any other Meka classifier. Only examples are given here.

  • p indicates probabilistic base classifier required (at least, recommended) — a classifier which can output a probability between 0 and 1 for each label.

  • s indicates a semi-supervised method, which will use the test data (minus labels) to help train.

  • All methods use -t data.arff which implies a train/test split. For other options, see the tutorial.

Scalability notes. Scalability in general depends on

  • The number of features: try doing feature selection first (see WEKA documentation, for example) and create a new ARFF file — or use a subset meta method ensemble RandomSubspaceML

  • The number of instances: use an ensemble method EnsembleML with tiny subsets (e.g., -P 20 for 20 percent), or RandomSubspaceML

  • The number of labels: if over 1000 labels, use a label powerset method, if labelling is very dense and noisy (many label combinations), use a binary relevance method

  • The base classifier: WEKA’s SMO usually provides good results, and is reasonably fast, but for really large datasets you may have to use something like NaiveBayes or SGD (with very few iterations). Keep in mind that some classes (like SMO) make many binary problems internally to deal with the multi-class case (i.e., as produced by label powerset methods).

Misc notes.

  • The emphasis here is on the different methods available. For more examples of general command-line arguments (for example on thresholds, splits, debug output), see the tutorial

  • Most methods already come with a sensible default classifier, for example J48 as a base classifier for problem transformation methods, and CC as a default classifier for many ensemble methods. To use the default classifier simply leave out the -W option.

Binary Relevance Methods

Binary relevance methods create an individual model for each label. This means that each model is a simply binary problem, but many labels means many models which can easily fill up memory.

Table 1. Binary Relevance Methods
Class Name Description / Notes Examples


Binary Relevance

Individual classifiers

java meka.classifiers.multilabel.BR -t data.arff -W weka.classifiers.functions.SMO


Classifier Chains

.. linked in a cascaded chain, random node order

java meka.classifiers.multilabel.CC -t data.arff -W weka.classifiers.functions.SMO


Classifier Trellis

.. linked in a trellis, connectivity L, heuristic X

java meka.classifiers.multilabel.CT -L 2 -X Ibf -t data.arff -W weka.classifiers.functions.SMO



Classifier Dependency Network

Fully-connected, undirected, I iterations, Ic of which for collecting marginals

java meka.classifiers.multilabel.CDN -I 1000 -Ic 100 -t data.arff -W weka.classifiers.functions.SMO -- -M



Classifier Dependency Trellis

.. in a trellis structure of connectivity L, heuristic X

java meka.classifiers.multilabel.CDT -I 1000 -Ic 100 -L 3 -t data.arff -W weka.classifiers.functions.SMO -- -M



Ensembles of Classifier Chains (ECC)

A Bagging ensemble of I chains, bagging P % of the instances

java meka.classifiers.multilabel.meta.BaggingML -I 10 -P 100 -t data.arff -W meka.classifiers.multilabel.CC -- -W weka.classifiers.functions.SMO



Subspace Ensembles of Classifier Chains

.. where each chain is given only a A percent of the attributes

java meka.classifiers.multilabel.meta.RandomSubspaceML -I 10 -P 80 -A 50 -t data.arff -W meka.classifiers.multilabel.CC -- -W weka.classifiers.functions.SMO



Probabilistic Classifier Chains

A classifier chain with Bayes Optimal inference, (exponential trials)

java meka.classifiers.multilabel.PCC -t data.arff -W weka.classifiers.functions.Logistic



Monte-Carlo Classifier Chains

.. with Monte Carlo search, maximum of Iy inference trials and Is chain-order trails

java meka.classifiers.multilabel.MCC -Iy 100 -Is 10 -t data.arff -W weka.classifiers.functions.Logistic



Population of Monte-Carlo Classifier Chains

.. a population of M chains is kept (not just the best)

java meka.classifiers.multilabel.PMCC -Iy 100 -Is 50 -M 10 -t data.arff -W weka.classifiers.functions.Logistic


Bayesian Classifier Chains

Tree connectivity based on Maximum Spanning Tree algorithm

java meka.classifiers.multilabel.BCC -t data.arff -W weka.classifiers.bayes.NaiveBayes

Label Powerset Methods

Label powerset inspired classifiers generally provide excellent performance, although only some parameterizations will be able to scale up to larger datasets. In most situations, RAkEL is a good option. PS methods work best when there are only a few typical combinations of labels, and most combinations occur only once and can be pruned away. EPS will work better. MEKA’s implementation of RAkELd is in fact a combination of RAkEL + PS, and should scale up to thousands or even tens of thousands of labels (with enough pruning).

Table 2. Label Powerset / Label Combination Methods
Class Name Description / Notes Examples


Majority Lableset

Always predicts most common labelset

java meka.classifiers.multilabel.MajorityLabelset -t data.arff


Label Combination / Label Powerset

As a multi-class problem

java meka.classifiers.multilabel.LP -t data.arff -W weka.classifiers.functions.SMO


Pruned Sets (Pruned Label Powerset)

.. with P infrequent classes pruned, but up to N subcopies reintroduced

java meka.classifiers.multilabel.PS -P 1 -N 1 -t data.arff -W weka.classifiers.functions.SMO



Ensembles of Pruned Sets (EPS)

.. in an ensemble, P can be selected from a range

java meka.classifiers.multilabel.meta.EnsembleML -t data.arff -W meka.classifiers.multilabel.PS -P 1-5 -N 1 -W weka.classifiers.functions.SMO



Pruned Sets thresholded

.. with an internal threshold

java meka.classifiers.multilabel.PS -P 1 -N 1 -t data.arff -W weka.classifiers.functions.SMO -- -M


Nearest Set Replacement (NSR)

Multi-target version of PS

java -cp "lib/*:target/meka-1.7.4.jar" meka.classifiers.multitarget.NSR -P 1 -N 1 -t data.arff -W weka.classifiers.functions.SMO


Super Class/Node Classifier

Creates super nodes, based on I simulated annealing iterations, V internal split validations, and runs NSR on them.

java meka.classifiers.multitarget.SCC -I 1000 -V 10 -P 1 -t data.arff -W meka.classifiers.multitarget.CC -- -W weka.classifiers.functions.SMO


Random k-labEL Pruned Sets

.. in subsets of size k

java meka.classifiers.multilabel.RAkEL -P 1 -N 1 -k 3 -t data.arff -W weka.classifiers.functions.SMO


Disjoint Random Pruned Sets

.. which are non-overlapping

java meka.classifiers.multilabel.RAkELd -P 1 -N 1 -k 3 -t data.arff -W weka.classifiers.functions.SMO


Subspace Ensembles of RAkELd

.. an a subset ensemble

java meka.classifiers.multilabel.meta.RandomSubspaceML -I 10 -P 60 -A 50 -t data.arff -W meka.classifiers.multilabel.RAkELd -- -P 3 -N 1 -k 5 -W weka.classifiers.functions.SMO


Hierachical Label Sets

.. disjoint subsets defined by a hierarchy in the dataset, e.g., @attribute C.C4

java meka.classifiers.multilabel.HASEL -t Enron.arff -P 3 -N 1 -W weka.classifiers.functions.SMO


Random k-labEL Sets

MULAN’s implementation (no pruning); 10 models, subsets of size [half the number of labels]

java meka.classifiers.multilabel.MULAN -t data.arff -S RAkEL1


Random k-labEL Sets

MULAN’s implementation (no pruning); [2 times the number of labels] models, subsets of size 3

java meka.classifiers.multilabel.MULAN -t data.arff -S RAkEL2



Subset Mapper

Like ECOCs. Maps predictions to nearest known label combination (from training set)

java meka.classifiers.multilabel.meta.SubsetMapper -t data.arff -W meka.classifiers.multilabel.BR -- -W weka.classifiers.functions.SMO

Pairwise and Threshold Methods

Pairwise methods can work well, but they are very sensitive to the number of labels. One-vs-rest classifiers (e.g., RT) can be faster with large numbers of labels, but may not perform as well.

Table 3. Pairwise and Multi-class Methods
Class Name Description / Notes Examples


Calibrated Label Ranking

Compares each pair, 01 vs 10, with a calbrated label

java meka.classifiers.multilabel.MULAN -t data.arff -S CLR


Fourclass Pairwise

Compares each pair, 00,01,10,11, with threshold

java meka.classifiers.multilabel.FW -t data.arff -W weka.classifiers.functions.SMO



Rank + Threshold

Duplicates multi-label examples into examples with one label each (one vs. rest). Trains a multi-class classifier, and uses a threshold to reconstitute a multi-label classification

java meka.classifiers.multilabel.RT -t data.arff -W weka.classifiers.bayes.NaiveBayes

Other Methods

Semi-supervised methods, when you want to use the testing data (labels are removed first) to help train, neural-network based methods (algorithm adaptation, no base classifier supplied), and ‘deep’ methods which create a higher level feature representation during training.

Table 4. Other Methods (Deep, Neural Network based, and Semi-supervised Methods)
Class Name Description / Notes Examples



Expectation Maximization

Uses the classic EM algorithm

java meka.classifiers.multilabel.meta.EM -t data.arff -I 100 -W meka.classifiers.multilabel.BR -- -W weka.classifiers.bayes.NaiveBayes



Classification Maximization

.. a non-probabilistic version thereof

java meka.classifiers.multilabel.meta.CM -t data.arff -I 100 -W meka.classifiers.multilabel.BR -- -W weka.classifiers.bayes.NaiveBayes


Back Propagation Neural Network

Multi-Layer Perceptron with H hidden nodes, E iterations

java meka.classifiers.multilabel.BPNN -t data.arff -H 20 -E 1000 -r 0.01 -m 0.2


Deep Back Propagation Neural Network

.. and with N layers, using RBMs for pretraining

java meka.classifiers.multilabel.DBPNN -t data.arff -N 3 -H 20 -E 1000 -r 0.01 -m 0.2



Deep Multi-label

.. with any ML classifier on top

java meka.classifiers.multilabel.meta.DeepML -t data.arff -N 2 -H 20 -E 1000 -r 0.01 -m 0.2 -W meka.classifiers.multilabel.CC -- -W weka.classifiers.functions.SMO

Updateable Methods for Data Streams

Incremental methods suitable for data streams: classifying instances and updating a classifier one instance at a time. Note that accuracy is recorded in windows. Use -B to set the number of windows, and higher verbosity, e.g., -verbosity 5 to see the accuracy reported also per window. These methods are adapted versions of regular multi-label methods.

Table 5. Updateable Methods
Class Name Description / Notes Examples


Updateable Binary Relevance

Use with any updateable base classifier

java meka.classifiers.multilabel.incremental.BRUpdateable -B 10 -t data.arff -W weka.classifiers.trees.HoeffdingTree


Updateable Classifier Chains

Use with any updateable base classifier

java meka.classifiers.multilabel.incremental.BRUpdateable -B 50 -t data.arff -W weka.classifiers.trees.HoeffdingTree


Updateable Pruned Sets

.. with initial buffer of size I, and max support S

java meka.classifiers.multilabel.incremental.PSUpdateable -B 10 -I 500 -S 20 -t data.arff -W weka.classifiers.bayes.NaiveBayesUpdateable