com.imsl.datamining.neural
Class LeastSquaresTrainer

java.lang.Object
  com.imsl.datamining.neural.LeastSquaresTrainer

All Implemented Interfaces:

Serializable, Trainer

public class LeastSquaresTrainer

extends Object

implements Trainer, Serializable

Trains a FeedForwardNetwork using a Levenberg-Marquardt algorithm for minimizing a sum of squares error.

The Java Logging API can be used to trace the performance training. The name of this Logger is com.imsl.datamining.LeatSquaresTrainer. Accumulated levels of detail correspond to Java's FINE, FINER, and FINEST logging levels with FINE yielding the smallest amount of information and FINEST yielding the most. The levels of output yield the following:

Level	Output
FINE	A message on entering and exiting method train, and any exceptions from and the exit status of NonlinLeastSquares
FINER	All of the messages in FINE, the input settings, and a summary report with the statistics from Network.computeStatistics() and the elapsed time.
FINEST	All of the messages in FINER, and a table of the computed weights and their gradient values.

See Also:

Feed Forward Class Example 1, NonlinLeastSquares, Serialized Form

Constructor Summary

LeastSquaresTrainer()
Creates a LeastSquaresTrainer.

Method Summary

protected Object	clone() Clones a copy of the trainer.
double[]	getErrorGradient() Returns the value of the gradient of the error function with respect to the weights.
int	getErrorStatus() Returns the error status from the trainer.
double	getErrorValue() Returns the final value of the error function.
static Formatter	getFormatter() Returns the logging Formatter object.
static Logger	getLogger() Returns the Logger object.
protected void	setEpochNumber(int num) Sets the epoch number for the trainer.
void	setFalseConvergenceTolerance(double falseConvergenceTolerance) Set the false convergence tolerance.
void	setGradientTolerance(double gradientTolerance) Set the gradient tolerance.
void	setInitialTrustRegion(double initialTrustRegion) Sets the intial trust region.
void	setMaximumStepsize(double maximumStepsize) Sets the maximum step size.
void	setMaximumTrainingIterations(int maximumSolverIterations) Sets the maximum number of iterations used by the nonlinear least squares solver.
protected void	setParallelMode(ArrayList[] allLogRecords) Sets the trainer to be used in multi-threaded EpochTainer.
void	setRelativeTolerance(double relativeTolerance) Sets the relative tolerance.
void	setStepTolerance(double stepTolerance) Set the step tolerance used to step between weights.
void	train(Network network, double[][] xData, double[][] yData) Trains the neural network using supplied training patterns.

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LeastSquaresTrainer

public LeastSquaresTrainer()

Creates a LeastSquaresTrainer.

Method Detail

clone

protected Object clone()

Clones a copy of the trainer.

getErrorGradient

public double[] getErrorGradient()

Returns the value of the gradient of the error function with respect to the weights.

Specified by:

getErrorGradient in interface Trainer

Returns:

A double array whose length is equal to the number of network weights, containing the value of the gradient of the error function with respect to the weights. Before training, null is returned.

getErrorStatus

public int getErrorStatus()

Returns the error status from the trainer.

Specified by:

getErrorStatus in interface Trainer

Returns:

An int which contains the error status. Zero indicates that no errors were encountered during training. Any non-zero value indicates that some error condition arose during training.

In many cases the trainer is able to recover from these conditions and produce a well-trained network.

Value	Meaning
0	All convergence tests were met.
1	Scaled step tolerance was satisfied. The current point may be an approximate local solution, or the algorithm is making very slow progress and is not near a solution, or StepTolerance is too big.
2	Scaled actual and predicted reductions in the function are less than or equal to the relative function convergence tolerance RelativeTolerance.
3	Iterates appear to be converging to a noncritical point. Incorrect gradient information, a discontinuous function, or stopping tolerances being too tight may be the cause.
4	Five consecutive steps with the maximum stepsize have been taken. Either the function is unbounded below, or has a finite asymptote in some direction, or the maximum stepsize is too small.
5	Too many iterations required

getErrorValue

public double getErrorValue()

Returns the final value of the error function.

Specified by:

getErrorValue in interface Trainer

Returns:

A double containing the final value of the error function from the last training. Before training, NaN is returned.

getFormatter

public static Formatter getFormatter()

Returns the logging Formatter object. Logger support requires JDK1.4. Use with earlier versions returns null.

The returned Formatter is used as input to Handler.setFormatter(java.util.logging.Formatter) to format the output log.

Returns:

A Formatter object, if present, or null .

getLogger

public static Logger getLogger()

Returns the Logger object. This is the Logger used to trace this class. It is named com.imsl.datamining.neural.QuasiNewtonTrainer.

Returns:

The Logger object, if present, or null .

setEpochNumber

protected void setEpochNumber(int num)

Sets the epoch number for the trainer.

Parameters:

num - An int array containing the epoch number.

setFalseConvergenceTolerance

public void setFalseConvergenceTolerance(double falseConvergenceTolerance)

Set the false convergence tolerance.

Parameters:

falseConvergenceTolerance - a double specifying the false convergence tolerance. Default: 1.0e-14.

See Also:

NonlinLeastSquares.setFalseConvergenceTolerance(double)

setGradientTolerance

public void setGradientTolerance(double gradientTolerance)

Set the gradient tolerance.

Parameters:

gradientTolerance - A double specifying the gradient tolerance. Default: 2.0e-5.

See Also:

MinUnconMultiVar.setGradientTolerance(double)

setInitialTrustRegion

public void setInitialTrustRegion(double initialTrustRegion)

Sets the intial trust region.

Parameters:

initialTrustRegion - A double which specifies the initial trust region radius. Default: unlimited trust region.

See Also:

NonlinLeastSquares.setInitialTrustRegion(double)

setMaximumStepsize

public void setMaximumStepsize(double maximumStepsize)

Sets the maximum step size.

Parameters:

maximumStepsize - A nonnegative double value specifying the maximum allowable stepsize in the optimizer. Default: , where w are the values of the weights in the network when training starts.

See Also:

NonlinLeastSquares.setMaximumStepsize(double)

setMaximumTrainingIterations

public void setMaximumTrainingIterations(int maximumSolverIterations)

Sets the maximum number of iterations used by the nonlinear least squares solver.

Parameters:

maximumSolverIterations - An int which specifies the maximum number of iterations to be used by the nonlinear least squares solver. Its default value is 1000.

See Also:

NonlinLeastSquares.setRelativeTolerance(double)

setParallelMode

protected void setParallelMode(ArrayList[] allLogRecords)

Sets the trainer to be used in multi-threaded EpochTainer.

Parameters:

allLogRecords - An ArrayList array containing the log records.

setRelativeTolerance

public void setRelativeTolerance(double relativeTolerance)

Sets the relative tolerance.

Parameters:

relativeTolerance - A double which specifies the relative error tolerance. It must be in the interval [0,1]. Its default value is 1.0e-20.

See Also:

NonlinLeastSquares.setRelativeTolerance(double)

setStepTolerance

public void setStepTolerance(double stepTolerance)

Set the step tolerance used to step between weights.

Parameters:

stepTolerance - A double which specifies the scaled step tolerance to use when changing the weights. Default: 1.0e-5.

See Also:

NonlinLeastSquares.setStepTolerance(double)

train

public void train(Network network,
                  double[][] xData,
                  double[][] yData)

Trains the neural network using supplied training patterns.

Each row of xData and yData contains a training pattern. These number of rows in two arrays must be equal.

Specified by:

train in interface Trainer

Parameters:

network - The Network to be trained.

xData - A double matrix which contains the input training patterns. The number of columns in xData must equal the number of Nodes in the InputLayer.

yData - A double matrix which contains the output training patterns. The number of columns in yData must equal the number of Perceptrons in the OutputLayer.