Parallel implementation of back-propagation algorithm in networks of workstations

• Published in: IEEE transactions on parallel and distributed systems 2005-01, Vol.16 (1), p.24-34
• Main Authors: Suresh, S., Omkar, S.N., Mani, V.
• Format: Article
• Language: English
• Subjects: Algorithm design and analysis; Algorithms; Back propagation; Character recognition; divisible load theory; Multilayer perceptron; Multilayer perceptrons; network of workstation; optical character recognition; Optical character recognition software; Optical fiber networks; Parallel algorithms; Partitioning algorithms; Performance analysis; performance measures; Processor scheduling; Studies; Work stations; Workstations
• ISSN: 1045-9219; 1558-2183
• DOI: 10.1109/TPDS.2005.11

This work presents an efficient mapping scheme for the multilayer perceptron (MLP) network trained using back-propagation (BP) algorithm on network of workstations (NOWs). Hybrid partitioning (HP) scheme is used to partition the network and each partition is mapped on to processors in NOWs. We derive the processing time and memory space required to implement the parallel BP algorithm in NOWs. The performance parameters like speed-up and space reduction factor are evaluated for the HP scheme and it is compared with earlier work involving vertical partitioning (VP) scheme for mapping the MLP on NOWs. The performance of the HP scheme is evaluated by solving optical character recognition (OCR) problem in a network of ALPHA machines. The analytical and experimental performance shows that the proposed parallel algorithm has better speed-up, less communication time, and better space reduction factor than the earlier algorithm. This work also presents a simple and efficient static mapping scheme on heterogeneous system. Using divisible load scheduling theory, a closed-form expression for number of neurons assigned to each processor in the NOW is obtained. Analytical and experimental results for static mapping problem on NOWs are also presented.