meka.classifiers.multilabel

Class Evaluation

java.lang.Object

meka.classifiers.multilabel.Evaluation

public class Evaluation
extends Object

Evaluation.java - Evaluation functionality.

Version:

March 2014

Author:

Jesse Read

Constructor Summary

Constructors

Constructor and Description
Evaluation()

Method Summary

Modifier and Type	Method and Description
static Result	cvModel(MultiLabelClassifier h, weka.core.Instances D, int numFolds, String top) CVModel - Split D into train/test folds, and then train and evaluate on each one.
static Result	cvModel(MultiLabelClassifier h, weka.core.Instances D, int numFolds, String top, String vop) CVModel - Split D into train/test folds, and then train and evaluate on each one.
static Result	evaluateModel(MultiLabelClassifier h, weka.core.Instances D_train, weka.core.Instances D_test) EvaluateModel - Build model 'h' on 'D_train', test it on 'D_test'.
static Result	evaluateModel(MultiLabelClassifier h, weka.core.Instances D_train, weka.core.Instances D_test, String top) EvaluateModel - Build model 'h' on 'D_train', test it on 'D_test', threshold it according to 'top', using default verbosity option.
static Result	evaluateModel(MultiLabelClassifier h, weka.core.Instances D_train, weka.core.Instances D_test, String top, String vop) EvaluateModel - Build model 'h' on 'D_train', test it on 'D_test', threshold it according to 'top', verbosity 'vop'.
static Result	evaluateModel(MultiLabelClassifier h, weka.core.Instances D_test, String tal, String vop) EvaluateModel - Assume 'h' is already built, test it on 'D_test', threshold it according to 'top', verbosity 'vop'.
static Result	evaluateModelM(MultiLabelClassifier h, weka.core.Instances D_train, weka.core.Instances D_test, String top, String vop)
static boolean	isMT(weka.core.Instances D) IsMT - see if dataset D is multi-target (else only multi-label)
static weka.core.Instances	loadDataset(String[] options) loadDataset - load a dataset, given command line option '-t' specifying an arff file.
static weka.core.Instances	loadDataset(String[] options, char T) loadDataset - load a dataset, given command line options specifying an arff file.
static void	printOptions(Enumeration e)
static void	runExperiment(MultiLabelClassifier h, String[] options) RunExperiment - Build and evaluate a model with command-line options.
static Result	testClassifier(MultiLabelClassifier h, weka.core.Instances D_test) TestClassifier - test classifier h on D_test
static Result	testClassifierM(MultiLabelClassifier h, weka.core.Instances D_test) Test Classifier but threaded (Multiple)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Evaluation

public Evaluation()

Method Detail

runExperiment

public static void runExperiment(MultiLabelClassifier h,
                                 String[] options)
                          throws Exception

RunExperiment - Build and evaluate a model with command-line options.

Parameters:

h - multi-label classifier

options - command line options

Throws:

Exception

isMT

public static boolean isMT(weka.core.Instances D)

IsMT - see if dataset D is multi-target (else only multi-label)

Parameters:

D - data

Returns:

true iff D is multi-target only (else false)

evaluateModel

public static Result evaluateModel(MultiLabelClassifier h,
                                   weka.core.Instances D_train,
                                   weka.core.Instances D_test,
                                   String top)
                            throws Exception

EvaluateModel - Build model 'h' on 'D_train', test it on 'D_test', threshold it according to 'top', using default verbosity option.

Parameters:

h - a multi-dim. classifier

D_train - training data

D_test - test data

top - Threshold OPtion (pertains to multi-label data only)

Returns:

Result raw prediction data with evaluation statistics included.

Throws:

Exception

evaluateModel

public static Result evaluateModel(MultiLabelClassifier h,
                                   weka.core.Instances D_train,
                                   weka.core.Instances D_test,
                                   String top,
                                   String vop)
                            throws Exception

EvaluateModel - Build model 'h' on 'D_train', test it on 'D_test', threshold it according to 'top', verbosity 'vop'.

Parameters:

h - a multi-dim. classifier

D_train - training data

D_test - test data

top - Threshold OPtion (pertains to multi-label data only)

vop - Verbosity OPtion (which measures do we want to calculate/output)

Returns:

Result raw prediction data with evaluation statistics included.

Throws:

Exception

evaluateModel

public static Result evaluateModel(MultiLabelClassifier h,
                                   weka.core.Instances D_test,
                                   String tal,
                                   String vop)
                            throws Exception

EvaluateModel - Assume 'h' is already built, test it on 'D_test', threshold it according to 'top', verbosity 'vop'.

Parameters:

h - a multi-dim. classifier

D_test - test data

tal - Threshold VALUES (not option)

vop - Verbosity OPtion (which measures do we want to calculate/output)

Returns:

Result raw prediction data with evaluation statistics included.

Throws:

Exception

cvModel

public static Result cvModel(MultiLabelClassifier h,
                             weka.core.Instances D,
                             int numFolds,
                             String top)
                      throws Exception

CVModel - Split D into train/test folds, and then train and evaluate on each one.

Parameters:

h - a multi-output classifier

D - test data Instances

numFolds - number of folds of CV

top - Threshold OPtion (pertains to multi-label data only)

Returns:

Result raw prediction data with evaluation statistics included.

Throws:

Exception

cvModel

public static Result cvModel(MultiLabelClassifier h,
                             weka.core.Instances D,
                             int numFolds,
                             String top,
                             String vop)
                      throws Exception

CVModel - Split D into train/test folds, and then train and evaluate on each one.

Parameters:

h - a multi-output classifier

D - test data Instances

numFolds - number of folds of CV

top - Threshold OPtion (pertains to multi-label data only)

vop - Verbosity OPtion (which measures do we want to calculate/output)

Returns:

Result raw prediction data with evaluation statistics included.

Throws:

Exception

evaluateModel

public static Result evaluateModel(MultiLabelClassifier h,
                                   weka.core.Instances D_train,
                                   weka.core.Instances D_test)
                            throws Exception

EvaluateModel - Build model 'h' on 'D_train', test it on 'D_test'. Note that raw multi-label predictions returned in Result may not have been thresholded yet. However, data statistics, classifier info, and running times are inpregnated into the Result here.

Parameters:

h - a multi-dim. classifier

D_train - training data

D_test - test data

Returns:

raw prediction data (no evaluation yet)

Throws:

Exception

evaluateModelM

public static Result evaluateModelM(MultiLabelClassifier h,
                                    weka.core.Instances D_train,
                                    weka.core.Instances D_test,
                                    String top,
                                    String vop)
                             throws Exception

Throws:

Exception

testClassifier

public static Result testClassifier(MultiLabelClassifier h,
                                    weka.core.Instances D_test)
                             throws Exception

TestClassifier - test classifier h on D_test

Parameters:

h - a multi-dim. classifier, ALREADY BUILT

D_test - test data

Returns:

Result with raw prediction data ONLY

Throws:

Exception

testClassifierM

public static Result testClassifierM(MultiLabelClassifier h,
                                     weka.core.Instances D_test)
                              throws Exception

Test Classifier but threaded (Multiple)

Parameters:

h - a multi-dim. classifier, ALREADY BUILT (threaded, implements MultiLabelThreaded)

D_test - test data

Returns:

Result with raw prediction data ONLY

Throws:

Exception

loadDataset

public static weka.core.Instances loadDataset(String[] options)
                                       throws Exception

loadDataset - load a dataset, given command line option '-t' specifying an arff file.

Parameters:

options - command line options, specifying dataset filename

Returns:

the dataset

Throws:

Exception

loadDataset

public static weka.core.Instances loadDataset(String[] options,
                                              char T)
                                       throws Exception

loadDataset - load a dataset, given command line options specifying an arff file.

Parameters:

options - command line options, specifying dataset filename

T - set to 'T' if we want to load a test file (default 't': load train or train-test file)

Returns:

the dataset

Throws:

Exception

printOptions

public static void printOptions(Enumeration e)