A Mobile DNN Training Processor With Automatic Bit Precision Search and Fine-Grained Sparsity Exploitation

In this article, an energy-efficient deep learning processor is proposed for deep neural network (DNN) training in mobile platforms. Conventional mobile DNN training processors suffer from high-bit precision requirement and high ReLU-dependencies. The proposed processor breaks through these fundamen...

Full description

Saved in:
Bibliographic Details
Published in:IEEE MICRO 2022-03, Vol.42 (2), p.16-25
Main Authors: Han, Donghyeon, Im, Dongseok, Park, Gwangtae, Kim, Youngwoo, Song, Seokchan, Lee, Juhyoung, Yoo, Hoi-Jun
Format: Article
Language:English
Subjects:
Artificial neural networks
Computer architecture
Deep learning
Energy efficiency
Machine learning
Microprocessors
Neural networks
Processors
Resource management
Rounding
Sparsity
System-on-chip
Throughput
Training
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this article, an energy-efficient deep learning processor is proposed for deep neural network (DNN) training in mobile platforms. Conventional mobile DNN training processors suffer from high-bit precision requirement and high ReLU-dependencies. The proposed processor breaks through these fundamental issues by adopting three new features. It first combines the runtime automatic bit precision searching method addition to both conventional dynamic fixed-point representation and stochastic rounding to realize low-precision training. It adopts bit-slice scalable core architecture with the input skipping functionality to exploit bit-slice-level fine-grained sparsity. The iterative channel reordering unit helps the processor to maintain high core utilization by solving the workload unbalancing problem during zero-slice skipping. It finally achieves at least 4.4× higher energy efficiency compared with the conventional DNN training processors.
ISSN:0272-1732
1937-4143
DOI:10.1109/MM.2021.3135457