Classification by Backpropagation

Classification by Backpropagation

^·         Backpropagation: A neural network learning algorithm

^·         Started by psychologists and neurobiologists to develop and test computational analogues of neurons

^·         A neural network: A set of connected input/output units where each connection has a weight associated with it

^·         During the learning phase, the network learns by adjusting the weights so as to be able to predict the correct class label of the input tuples

^·         Also referred to as connectionist learning due to the connections between units

Neural Network as a Classifier

·       Weakness

o   Long training time

o   Require a number of parameters typically best determined empirically, e.g., the network topology or ``structure."

o   Poor interpretability: Difficult to interpret the symbolic meaning behind the learned weights and of ``hidden units" in the network

Strength

o   High tolerance to noisy data

o   Ability to classify untrained patterns

o   Well-suited for continuous-valued inputs and outputs o Successful on a wide array of real-world data

o   Algorithms are inherently parallel

o   Techniques have recently been developed for the extraction of rules from trained neural networks

ANeuron(=aperceptron)

A Multi-Layer Feed-Forward Neural Network

o   The inputs to the network correspond to the attributes measured for each training tuple

^o     Inputs are fed simultaneously into the units making up the input layer

^o     They are then weighted and fed simultaneously to a hidden layer

^o     The number of hidden layers is arbitrary, although usually only one

^o     The weighted outputs of the last hidden layer are input to units making up the output layer, which emits the network's prediction

^o     The network is feed-forward in that none of the weights cycles back to an input unit or to an output unit of a previous layer

o   From a statistical point of view, networks perform nonlinear regression: Given enough hidden units and enough training samples, they can closely approximate any function