StreamBrain: An HPC Framework for Brain-like Neural Networks on CPUs, GPUs and FPGAs

• Published in: arXiv.org 2021-06
• Main Authors: Podobas, Artur, Svedin, Martin, Chien, Steven W D, Peng, Ivy B, Naresh Balaji Ravichandran, Herman, Pawel, Lansner, Anders, Markidis, Stefano
• Format: Article
• Language: English
• Subjects: Algorithms; Back propagation; Back propagation networks; Brain; Central processing units; CPUs; Domain specific languages; Field programmable gate arrays; Floating point arithmetic; Machine learning; Neural networks; Programming languages
• ISSN: 2331-8422
• DOI: 10.48550/arxiv.2106.05373

The modern deep learning method based on backpropagation has surged in popularity and has been used in multiple domains and application areas. At the same time, there are other -- less-known -- machine learning algorithms with a mature and solid theoretical foundation whose performance remains unexplored. One such example is the brain-like Bayesian Confidence Propagation Neural Network (BCPNN). In this paper, we introduce StreamBrain -- a framework that allows neural networks based on BCPNN to be practically deployed in High-Performance Computing systems. StreamBrain is a domain-specific language (DSL), similar in concept to existing machine learning (ML) frameworks, and supports backends for CPUs, GPUs, and even FPGAs. We empirically demonstrate that StreamBrain can train the well-known ML benchmark dataset MNIST within seconds, and we are the first to demonstrate BCPNN on STL-10 size networks. We also show how StreamBrain can be used to train with custom floating-point formats and illustrate the impact of using different bfloat variations on BCPNN using FPGAs.