RBM (meka 1.9.1 API)

java.lang.Object
- rbms.RBM

Direct Known Subclasses:

DBM
```
public class RBM
extends Object
```
RBM.java - Restricted Boltzmann Machine. Using Contrastive Divergence. You have inputs X; You want to output to hidden units Z. To do this, you learn weight matrix W, where Z ~ sigma(W'X). ============== TRAINING (batches of 10) ===== RBM rbm = new RBM(); rbm.setOptions("-E 100 -H 10 -r 0.1 -m 0.8"); // to build 10 hidden units, over 100 epochs, with learning rate 0.1, momentum 0.8 rbm.train(X,10); // train in batches of 10 Z = rbm.getZ(X); // get output ============== UPDATING (one epoch) ========= rbm.update(xnew); ============== TESTING (single input) ======= z = rbm.getz(xnew); Note: should be binary for hidden states, can be probabilities for visible states.

Version:

April 2013

Author:

Jesse Read (jesse@tsc.uc3m.es)

Field Summary

Fields
Modifier and Type	Field and Description
`protected double`	`COST`
`protected Jama.Matrix`	`dW_`
`protected double`	`LEARNING_RATE`
`protected int`	`m_E`
`protected int`	`m_H`
`protected Random`	`m_R`
`protected double`	`MOMENTUM`
`protected Jama.Matrix`	`W`

Constructor Summary

Constructors
Constructor and Description

RBM()
RBM - Create an RBM with default options.

RBM(String[] options)
RBM - Create an RBM with 'options' (using WEKA-style option processing).

Constructors
Constructor and Description
`RBM()` RBM - Create an RBM with default options.
`RBM(String[] options)` RBM - Create an RBM with 'options' (using WEKA-style option processing).

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`double`	`calculateError(Jama.Matrix X)` Calculate the Error right now.
`Jama.Matrix`	`epoch(Jama.Matrix X_0)` Epoch - Run X through one epcho of CD of the RBM.
`int`	`getE()`
`int`	`getH()`
`double`	`getLearningRate()`
`double`	`getMomentum()`
`String[]`	`getOptions()` GetOptions - WEKA-style option processing.
`Jama.Matrix`	`getW()`
`Jama.Matrix[]`	`getWs()`
`void`	`initWeights(int d)` Initialize W, and make _dW (for momentum) of the same dimensions.
`static void`	`main(String[] argv)` Main - do some test routines.
`protected Jama.Matrix`	`makeW(int d, int h)`
`static Jama.Matrix`	`makeW(int d, int h, Random r)` Make W matrix of dimensions d+1 and h+1 (+1 for biases).
`double[]`	`prob_x(double[] z_)` Visible Activation Probability - returns P(x\|z) where p(x[j]==1\|z) for each j-th element.
`Jama.Matrix`	`prob_X(Jama.Matrix Z)` Visible Activation Probability - returns P(X\|Z).
`double[]`	`prob_z(double[] x_)` Hidden Activation Probability - returns P(z\|x) where p(z[i]==1\|x) for each element.
`double[][]`	`prob_Z(double[][] X_)` Hidden Activation Probability - returns P(Z\|X).
`Jama.Matrix`	`prob_Z(Jama.Matrix X)` Hidden Activation Probability - returns P(Z\|X).
`double[][]`	`propUp(double[][] X_)` Hidden Activation Value.
`Jama.Matrix`	`sample_epoch(Jama.Matrix X_0)`
`double[]`	`sample_x(double[] z_)` Sample Visible - returns x[j] ~ p(x[j]==1\|z) for each j-th element.
`Jama.Matrix`	`sample_X(Jama.Matrix Z)` Sample Visible - returns X ~ P(X\|Z).
`double[]`	`sample_z(double[] x_)` Sample Hidden Value - returns z[i] ~ p(z[i]==1\|x) for each i-th element.
`Jama.Matrix`	`sample_Z(Jama.Matrix X)` Sample Hidden Value - returns Z ~ P(Z\|X).
`void`	`setE(int n)` SetE - set the number of epochs (if n is negative, it means max epochs).
`void`	`setH(int h)`
`void`	`setLearningRate(double r)`
`void`	`setMomentum(double m)`
`void`	`setOptions(String[] options)` Set Options - WEKA-style option processing.
`void`	`setSeed(int seed)`
`String`	`toString()` ToString - return a String representation of the weight Matrix defining this RBM.
`double`	`train(double[][] X_)` Train - Setup and train the RBM on X, over m_E epochs.
`double`	`train(double[][] X_, int batchSize)` Train - Setup and batch-train the RBM on X.
`double`	`train(double[][] X_, int batchSize, Random r)` Train - Setup and batch-train the RBM on X, with some random sampling involved.
`void`	`update(double[] x_)` Update - On raw data (with no bias column)
`void`	`update(double[][] X_)` Update - On raw data (with no bias column)
`void`	`update(double[] x_, double s)` Update - On raw data (with no bias column)
`void`	`update(Jama.Matrix X)` Update - Carry out one epoch of CD, update W.
`void`	`update(Jama.Matrix X, double s)` Update - Carry out one epoch of CD, update W.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - LEARNING_RATE
```
protected double LEARNING_RATE
```
  - MOMENTUM
```
protected double MOMENTUM
```
  - COST
```
protected double COST
```
  - m_E
```
protected int m_E
```
  - m_H
```
protected int m_H
```
  - W
```
protected Jama.Matrix W
```
  - dW_
```
protected Jama.Matrix dW_
```
  - m_R
```
protected Random m_R
```
- Constructor Detail
  - RBM
```
public RBM()
```
    RBM - Create an RBM with default options.
  - RBM
```
public RBM(String[] options)
    throws Exception
```
    RBM - Create an RBM with 'options' (using WEKA-style option processing).
    
    Throws:
    
    Exception
- Method Detail
  - setOptions
```
public void setOptions(String[] options)
                throws Exception
```
    Set Options - WEKA-style option processing.
    
    Throws:
    
    Exception
  - getOptions
```
public String[] getOptions()
                    throws Exception
```
    GetOptions - WEKA-style option processing.
    
    Throws:
    
    Exception
  - prob_z
```
public double[] prob_z(double[] x_)
```
    Hidden Activation Probability - returns P(z|x) where p(z[i]==1|x) for each element. A Bias is added (and removed) automatically.
    
    Parameters:
    
    x_ - x (without bias)
    
    Returns:
    
    z (without bias)
  - prob_Z
```
public double[][] prob_Z(double[][] X_)
```
    Hidden Activation Probability - returns P(Z|X). A Bias column added (and removed) automatically.
    
    Parameters:
    
    X_ - X (without bias)
    
    Returns:
    
    P(Z|X)
  - prob_Z
```
public Jama.Matrix prob_Z(Jama.Matrix X)
```
    Hidden Activation Probability - returns P(Z|X). A bias column is assumed to be included.
    
    Parameters:
    
    X - X (bias included)
    
    Returns:
    
    P(Z|X)
  - propUp
```
public double[][] propUp(double[][] X_)
```
    Hidden Activation Value. A bias column added (and removed) automatically.
    
    Parameters:
    
    X_ - X (without bias)
    
    Returns:
    
    1 if P(Z|X) greater than 0.5
  - sample_Z
```
public Jama.Matrix sample_Z(Jama.Matrix X)
```
    Sample Hidden Value - returns Z ~ P(Z|X). A bias column is assumed to be included.
    
    Parameters:
    
    X - X (bias included)
    
    Returns:
    
    Z ~ P(Z|X)
  - sample_z
```
public double[] sample_z(double[] x_)
```
    Sample Hidden Value - returns z[i] ~ p(z[i]==1|x) for each i-th element. A bias column added (and removed) automatically.
    
    Parameters:
    
    x_ - x (without bias)
    
    Returns:
    
    z ~ P(z|x) (without bias)
  - sample_x
```
public double[] sample_x(double[] z_)
```
    Sample Visible - returns x[j] ~ p(x[j]==1|z) for each j-th element. A bias is added (and removed) automatically.
    
    Parameters:
    
    z_ - z (without bias)
    
    Returns:
    
    x ~ P(x|z) (without bias)
  - sample_X
```
public Jama.Matrix sample_X(Jama.Matrix Z)
```
    Sample Visible - returns X ~ P(X|Z). A bias column is assumed to be included.
    
    Parameters:
    
    Z - Z (bias included)
    
    Returns:
    
    X ~ P(X|Z)
  - prob_x
```
public double[] prob_x(double[] z_)
```
    Visible Activation Probability - returns P(x|z) where p(x[j]==1|z) for each j-th element. A bias is added (and removed) automatically.
    
    Parameters:
    
    z_ - z (without bias)
    
    Returns:
    
    x (without bias)
  - prob_X
```
public Jama.Matrix prob_X(Jama.Matrix Z)
```
    Visible Activation Probability - returns P(X|Z). A bias column is assumed to be included.
    
    Parameters:
    
    Z - z (bias included)
    
    Returns:
    
    P(X|Z)
  - makeW
```
public static Jama.Matrix makeW(int d,
                                int h,
                                Random r)
```
    Make W matrix of dimensions d+1 and h+1 (+1 for biases). Initialized from ~N(0,0.2) (seems to work better than ~N(0.0.01)) -- except biases (set to 0)
    
    Parameters:
    
    d - number of rows (visible units)
    
    h - number of columns (hidden units)
    
    r - for getting random rumbers
    
    Returns:
    
    W
  - makeW
```
protected Jama.Matrix makeW(int d,
                            int h)
```
  - initWeights
```
public void initWeights(int d)
```
    Initialize W, and make _dW (for momentum) of the same dimensions.
    
    Parameters:
    
    d - number of visible units
  - update
```
public void update(Jama.Matrix X)
```
    Update - Carry out one epoch of CD, update W. We use dW_ to manage momentum.
    TODO weight decay SHOULD NOT BE APPLIED TO BIASES
    
    Parameters:
    
    X - X
  - update
```
public void update(Jama.Matrix X,
                   double s)
```
    Update - Carry out one epoch of CD, update W.
    TODO combine with above fn.
    
    Parameters:
    
    X - X
    
    s - multiply the gradient by this scalar
  - update
```
public void update(double[][] X_)
```
    Update - On raw data (with no bias column)
    
    Parameters:
    
    X_ - raw double[][] data (with no bias column)
  - update
```
public void update(double[] x_)
```
    Update - On raw data (with no bias column)
    
    Parameters:
    
    x_ - raw double[] data (with no bias column)
  - update
```
public void update(double[] x_,
                   double s)
```
    Update - On raw data (with no bias column)
    
    Parameters:
    
    x_ - raw double[] data (with no bias column)
    
    s - multiply the gradient by this scalar
  - train
```
public double train(double[][] X_)
             throws Exception
```
    Train - Setup and train the RBM on X, over m_E epochs.
    
    Parameters:
    
    X_ - X
    
    Returns:
    
    the error (@TODO unnecessary)
    
    Throws:
    
    Exception
  - train
```
public double train(double[][] X_,
                    int batchSize)
             throws Exception
```
    Train - Setup and batch-train the RBM on X.
    TODO, above function train(X_) could really be trained with train(X_,N), so, should share code with train(X)
    TODO, divide gradient by the size of the batch! (doing already? .. no)
    
    Parameters:
    
    X_ - X
    
    batchSize - the batch size
    
    Throws:
    
    Exception
  - train
```
public double train(double[][] X_,
                    int batchSize,
                    Random r)
             throws Exception
```
    Train - Setup and batch-train the RBM on X, with some random sampling involved.
    TODO should share code with train(X)
    
    Parameters:
    
    X_ - X
    
    batchSize - the batch size
    
    r - the randomness
    
    Throws:
    
    Exception
  - calculateError
```
public double calculateError(Jama.Matrix X)
```
    Calculate the Error right now.
    NOTE: this will take a few miliseconds longer than calculating directly in the epoch() loop (where we have to calculate X_down anyway).
    TODO rename this function
    
    Parameters:
    
    X - X
    
    Returns:
    
    The error
  - epoch
```
public Jama.Matrix epoch(Jama.Matrix X_0)
```
    Epoch - Run X through one epcho of CD of the RBM.
    
    Parameters:
    
    X_0 - The input matrix (includes bias column).
    
    Returns:
    
    the contrastive divergence (CD) for this epoch. x_0 = x for k = 0,...,K-1 z_k = sample up x_k+1 = sample down e+ = pz|x_0 e- = pz|x_K CD = e+ - e- Note: should be binary for hidden states, can be probabilities for visible states.
  - sample_epoch
```
public Jama.Matrix sample_epoch(Jama.Matrix X_0)
```
  - setH
```
public void setH(int h)
```
  - getH
```
public int getH()
```
  - setE
```
public void setE(int n)
```
    SetE - set the number of epochs (if n is negative, it means max epochs).
  - getE
```
public int getE()
```
  - setLearningRate
```
public void setLearningRate(double r)
```
  - getLearningRate
```
public double getLearningRate()
```
  - setMomentum
```
public void setMomentum(double m)
```
  - getMomentum
```
public double getMomentum()
```
  - setSeed
```
public void setSeed(int seed)
```
  - getWs
```
public Jama.Matrix[] getWs()
```
  - getW
```
public Jama.Matrix getW()
```
  - toString
```
public String toString()
```
    ToString - return a String representation of the weight Matrix defining this RBM.
    
    Overrides:
    
    toString in class Object
  - main
```
public static void main(String[] argv)
                 throws Exception
```
    Main - do some test routines.
    
    Throws:
    
    Exception

Class RBM

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

LEARNING_RATE

MOMENTUM

COST

m_E

m_H

W

dW_

m_R

Constructor Detail

RBM

RBM

Method Detail

setOptions

getOptions

prob_z

prob_Z

prob_Z

propUp

sample_Z

sample_z

sample_x

sample_X

prob_x

prob_X

makeW

makeW

initWeights

update

update

update

update

update

train

train

train

calculateError

epoch

sample_epoch

setH

getH

setE

getE

setLearningRate

getLearningRate

setMomentum

getMomentum

setSeed

getWs

getW

toString

main