DIANNE: a modular framework for designing, training and deploying deep neural networks on heterogeneous distributed infrastructure

• Published in: The Journal of systems and software 2018-07, Vol.141, p.52-65
• Main Authors: Coninck, Elias De, Bohez, Steven, Leroux, Sam, Verbelen, Tim, Vankeirsbilck, Bert, Simoens, Pieter, Dhoedt, Bart
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Distributed applications; Internet of Things; Machine learning
• ISSN: 0164-1212; 1873-1228
• DOI: 10.1016/j.jss.2018.03.032

•Integration of deep learning algorithms into external systems.•Framework with support for deep learning data and model parallelism.•Custom definition of (un)supervised learning routines.•Management and scaling of deep learning applications.•Web-based interface to design, train, deploy and manage deep neural networks. Deep learning has shown tremendous results on various machine learning tasks, but the nature of the problems being tackled and the size of state-of-the-art deep neural networks often require training and deploying models on distributed infrastructure. DIANNE is a modular framework designed for dynamic (re)distribution of deep learning models and procedures. Besides providing elementary network building blocks as well as various training and evaluation routines, DIANNE focuses on dynamic deployment on heterogeneous distributed infrastructure, abstraction of Internet of Things (IoT) sensors, integration with external systems and graphical user interfaces to build and deploy networks, while retaining the performance of similar deep learning frameworks. In this paper the DIANNE framework is proposed as an all-in-one solution for deep learning, enabling data and model parallelism though a modular design, offloading to local compute power, and the ability to abstract between simulation and real environment.