Abstract
Long, L.N. and Gupta, A., "Scalable Massively Parallel Artifical Neural Networks," Journal of Aerospace Computing, Information, and Communication (JACIC), Vol. 5, No. 1, Jan., 2008.
Artificial Neural Networks (ANN) can be very effective for pattern recognition, function
approximation, scientific classification, control, and the analysis of time series data; however
they can require very large training times for large networks. Once the network is trained
for a particular problem, however, it can produce results in a very short time. Traditional
ANNs using back-propagation algorithm do not scale well as each neuron in one level is fully
connected to each neuron in the previous level. In the present work only the neurons at the
edges of the domains were involved in communication, in order to reduce the communication
costs and maintain scalability. Ghost neurons were created at these processor boundaries for
information communication. An object-oriented, massively-parallel ANN software package
SPANN (Scalable Parallel Artificial Neural Network) has been developed and is described
here. MPI was used to parallelize the C++ code. The back-propagation algorithm was used
to train the network. In preliminary tests, the software was used to identify character sets
consisting of 48 characters and with increasing resolutions. The code correctly identified all
the characters when adequate training was used in the network. The training of a problem
size with 2 billion neuron weights on an IBM BlueGene/L computer using 1000 dual PowerPC
440 processors required less than 30 minutes.Various comparisons in training time, forward
propagation time, and error reduction were also made.
|
