com.imsl.datamining.neural
Class FeedForwardNetwork

java.lang.Object
  com.imsl.datamining.neural.Network
      com.imsl.datamining.neural.FeedForwardNetwork

All Implemented Interfaces:

Serializable

public class FeedForwardNetwork

extends Network

A representation of a feed forward neural network.

A Network contains an InputLayer, an OutputLayer and zero or more HiddenLayers. The null InputLayer and OutputLayer are automatically created by the Network constructor. The InputNodes are added using the getInputLayer().createInputs(nInputs) method. Output Perceptrons are added using the getOutputLayer().createPerceptrons(nOutputs), and HiddenLayers can be created using the createHiddenLayer().createPerceptrons(nPerceptrons) method.

The InputLayer contains InputNodes. The HiddenLayers and OutputLayers contain Perceptron nodes. These Nodes are created using factory methods in the Layers.

The Network also contains Links between Nodes. Links are created by methods in this class.

Each Link has a weight and gradient value. Each Perceptron node has a bias value. When the Network is trained, the weight and bias values are used as initial guesses. After the Network is trained the weight, gradient and bias values are set to the values computed by the training.

A feed forward network is a network in which links are only allowed from one layer to a following layer.

See Also:

Example 1, Serialized Form

Constructor Summary

FeedForwardNetwork()
Creates a new instance of FeedForwardNetwork.

Method Summary

HiddenLayer	createHiddenLayer() Creates a HiddenLayer.
Link	findLink(Node from, Node to) Returns the Link between two Nodes.
Link[]	findLinks(Node to) Returns all of the Links to a given Node.
double[]	forecast(double[] x) Computes a forecast using the Network.
double[][]	getForecastGradient(double[] xData) Returns the derivatives of the outputs with respect to the weights.
HiddenLayer[]	getHiddenLayers() Returns the HiddenLayers in this network.
InputLayer	getInputLayer() Returns the InputLayer.
Link[]	getLinks() Return all of the Links in this Network.
int	getNumberOfInputs() Returns the number of inputs to the Network.
int	getNumberOfLinks() Returns the number of Links in the Network.
int	getNumberOfOutputs() Returns the number of outputs from the Network.
int	getNumberOfWeights() Returns the number of weights in the Network.
OutputLayer	getOutputLayer() Returns the OutputLayer.
Perceptron[]	getPerceptrons() Returns the Perceptrons in this Network.
double[]	getWeights() Returns the weights for the Links in this network.
Link	link(Node from, Node to) Establishes a Link between two Nodes.
Link	link(Node from, Node to, double weight) Establishes a Link between two Nodes with a specified weight.
void	linkAll() For each Layer in the Network, link each Node in the Layer to each Node in the next Layer.
void	linkAll(Layer from, Layer to) Link all of the Nodes in one Layer to all of the Nodes in another Layer.
void	remove(Link link) Removes a Link from the network.
void	setEqualWeights(double[][] xData) Initializes network weights using equal weighting.
void	setRandomWeights(double[][] xData, Random random) Initializes network weights using random weights.
void	setWeights(double[] weights) Sets the weights for the Links in this Network.
protected void	validateLink(Node from, Node to) Checks that a Link between two Nodes is valid.

Methods inherited from class com.imsl.datamining.neural.Network

computeStatistics

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FeedForwardNetwork

public FeedForwardNetwork()

Creates a new instance of FeedForwardNetwork.

Method Detail

createHiddenLayer

public HiddenLayer createHiddenLayer()

Creates a HiddenLayer.

Specified by:

createHiddenLayer in class Network

Returns:

A HiddenLayer object which specifies a neural network hidden layer.

findLink

public Link findLink(Node from,
                     Node to)

Returns the Link between two Nodes.

Parameters:

from - The origination Node.

to - The destination Node.

Returns:

A Link between the two Nodes, or null if no such Link exists.

findLinks

public Link[] findLinks(Node to)

Returns all of the Links to a given Node.

Parameters:

to - A Node who's Links are to be determined.

Returns:

An array of Links containing all of the Links to the given Node.

forecast

public double[] forecast(double[] x)

Computes a forecast using the Network.

Specified by:

forecast in class Network

Parameters:

x - A double array of values to which the Nodes in the InputLayer are to be set.

Returns:

A double array containing the values of the Nodes in the OutputLayer.

getForecastGradient

public double[][] getForecastGradient(double[] xData)

Returns the derivatives of the outputs with respect to the weights.

Specified by:

getForecastGradient in class Network

Parameters:

xData - A double array which specifies the input values at which the gradient is to be evaluated.

Returns:

A double array containing the gradient values. The value of gradient[i][j] is , where is the i-th output and is the j-th weight.

getHiddenLayers

public HiddenLayer[] getHiddenLayers()

Returns the HiddenLayers in this network.

Returns:

An array of HiddenLayers in this network.

getInputLayer

public InputLayer getInputLayer()

Returns the InputLayer.

Specified by:

getInputLayer in class Network

Returns:

The neural network InputLayer.

getLinks

public Link[] getLinks()

Return all of the Links in this Network.

Specified by:

getLinks in class Network

Returns:

An array of Links containing all of the Links in this Network.

getNumberOfInputs

public int getNumberOfInputs()

Returns the number of inputs to the Network.

Specified by:

getNumberOfInputs in class Network

Returns:

An int containing the number of inputs to the Network.

getNumberOfLinks

public int getNumberOfLinks()

Returns the number of Links in the Network.

Specified by:

getNumberOfLinks in class Network

Returns:

An int which contains the number of Links in the Network.

getNumberOfOutputs

public int getNumberOfOutputs()

Returns the number of outputs from the Network.

Specified by:

getNumberOfOutputs in class Network

Returns:

An int containing the number of outputs from the Network.

getNumberOfWeights

public int getNumberOfWeights()

Returns the number of weights in the Network.

Specified by:

getNumberOfWeights in class Network

Returns:

An int which contains the number of weights in the Network.

getOutputLayer

public OutputLayer getOutputLayer()

Returns the OutputLayer.

Specified by:

getOutputLayer in class Network

Returns:

The neural network OutputLayer.

getPerceptrons

public Perceptron[] getPerceptrons()

Returns the Perceptrons in this Network.

Specified by:

getPerceptrons in class Network

Returns:

An array of Perceptrons in this network.

getWeights

public double[] getWeights()

Returns the weights for the Links in this network.

Specified by:

getWeights in class Network

Returns:

An array of doubles containing the weights. The array contains the weights for each Link followed by the Perceptron bias values. The Link weights are in the order in which the Links were created. The weight values are first, followed by the bias values in the HiddenLayers and then the bias values in the OutputLayer, and then by the order in which the Perceptrons were created.

link

public Link link(Node from,
                 Node to)

Establishes a Link between two Nodes. Any existing Link between these Nodes is removed.

Parameters:

from - The origination Node.

to - The destination Node.

Returns:

A Link between the two Nodes.

link

public Link link(Node from,
                 Node to,
                 double weight)

Establishes a Link between two Nodes with a specified weight.

Parameters:

from - The origination Node.

to - The destination Node.

weight - A double which specifies the weight to be given the Link.

Returns:

A Link between the two Nodes.

linkAll

public void linkAll()

For each Layer in the Network, link each Node in the Layer to each Node in the next Layer.

linkAll

public void linkAll(Layer from,
                    Layer to)

Link all of the Nodes in one Layer to all of the Nodes in another Layer.

Parameters:

from - The origination Layer.

to - The destination Layer.

remove

public void remove(Link link)

Removes a Link from the network.

Parameters:

link - The Link deleted from the network.

setEqualWeights

public void setEqualWeights(double[][] xData)

Initializes network weights using equal weighting.

The equal weights approach starts by assigning equal values to the inputs of each perceptron. If a perceptron has 4 inputs, then this method starts by assigning the value 1/4 to each of the perceptron's input weights. The bias weight is initially assigned a value of zero.

The weights for the first layer of perceptrons, either the first hidden layer if the number of layers is greater than 1 or the output layer, are scaled using the training patterns. Scaling is accomplished by dividing the initial weights for the first layer by the standard deviation, s, of the potential for that perceptron. The bias weight is set to -avg/s, where avg is the average potential for that perceptron. This makes the average potential for the perceptrons in this first layer approximately 0 and its standard deviation equal to 1.

This reduces the possibility of saturation during network training resulting from very large or small values for the perceptron's potential. During training random noise is added to these intial values at each training stage. If the epoch trainer is used, noise is added to these initial values at the start of each epoch.

Parameters:

xData - An input double matrix containing training patterns. The number of columns in xData must equal the number of nodes in the input layer.

setRandomWeights

public void setRandomWeights(double[][] xData,
                             Random random)

Initializes network weights using random weights.

The random weights algorithm assigns equal weights to all perceptrons, except those in the first layer connected to the input layer. Like the equal weights algorithm, perceptrons not in the first layer are assigned weights 1/k, where k is the number of inputs connected to that perceptron.

For the first layer perceptron weights, they are initially assigned values from the uniform random distribution on the interval [-0.5, +0.5]. These are then scaled using the training patterns. The random weights for a perceptron are divided by s, the standard deviation of the potential for that perceptron calculated using the intial random values. Its bias weight is set to -avg/s, where avg is the average potential for that perceptron. This makes the average potential for the perceptrons in this first layer approximately 0 and its standard deviation equal to 1.

This reduces the possibility of saturation during network training resulting from very large or small values for the perceptron's potential. During training random noise is added to these intial values at each training stage. If the epoch trainer is used, noise is added to these initial values at the start of each epoch.

Parameters:

xData - An input double matrix containing training patterns. The number of columns in xData must equal the number of nodes in the input layer.

random - A Random object.

setWeights

public void setWeights(double[] weights)

Sets the weights for the Links in this Network.

Specified by:

setWeights in class Network

Parameters:

weights - A double array containing the weights in the same order as getWeights().

validateLink

protected void validateLink(Node from,
                            Node to)
                     throws IllegalArgumentException

Checks that a Link between two Nodes is valid.

In a feed forward network a link must be from a node in one layer to a node in a later layer. Intermediate layers can be skipped, but a link cannot go backward.

Parameters:

from - The origination Node.

to - The destination Node.

Throws:

IllegalArgumentException - is thrown if the Link is not valid