Accelerating Hybrid Quantized Neural Networks on Multi-tenant Cloud FPGA

The increasing adoption of Field-Programmable Gate Arrays (FPGA) into cloud and data center systems opens the way to the unprecedented acceleration of Machine Learning applications. Convolutional Neural Networks (CNN) have largely been adopted as algorithms for image classification and object detect...

Full description

Saved in:
Bibliographic Details
Main Authors: Kwadjo, Danielle Tchuinkou, Nghonda Tchinda, Erman, Mbongue, Joel Mandebi, Bobda, Christophe
Format: Conference Proceeding
Language:English
Subjects:
Cloud computing
CNN Acceleration
Computational modeling
Computer architecture
Distributed Inference
FPGAs
Multi-tenancy
Neural networks
Performance evaluation
Prototypes
Search problems
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The increasing adoption of Field-Programmable Gate Arrays (FPGA) into cloud and data center systems opens the way to the unprecedented acceleration of Machine Learning applications. Convolutional Neural Networks (CNN) have largely been adopted as algorithms for image classification and object detection. As we head towards FPGA multi-tenancy in the cloud, it becomes necessary to investigate architectures and mechanisms for the efficient deployment of CNN into multitenant FPGAs cloud Infrastructure. In this work, we propose an FPGA architecture and a design flow that support efficient integration of CNN applications into a cloud infrastructure that exposes multi-tenancy to cloud developers. We prototype the proposed approach on randomly allocated virtual regions to tenants. We study how space-sharing of a single device between multiple cloud tenants influence the design flow, the allocation of resources, and the performance in term of resource utilization and overall latency compared to single-tenant deployments. Prototyping results show a latency at most 8% lower than that of single-tenant deployment while achieving higher resource utilization. We also record a maximum frequency of up to 12% higher in multi-tenant implementations.
ISSN:2576-6996
DOI:10.1109/ICCD56317.2022.00079