Loading…

A&B BNN: Add&Bit-Operation-Only Hardware-Friendly Binary Neural Network

Binary neural networks utilize 1-bit quantized weights and activations to reduce both the model's storage demands and computational burden. However, advanced binary architectures still incorporate millions of inefficient and nonhardware-friendly full-precision multiplication operations. A&B...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-03
Main Authors: Ma, Ruichen, Qiao, Guanchao, Liu, Yian, Meng, Liwei, Ning, Ning, Liu, Yang, Hu, Shaogang
Format: Article
Language:English
Subjects:
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites
container_end_page
container_issue
container_start_page
container_title arXiv.org
container_volume
creator Ma, Ruichen
Qiao, Guanchao
Liu, Yian
Meng, Liwei
Ning, Ning
Liu, Yang
Hu, Shaogang
description Binary neural networks utilize 1-bit quantized weights and activations to reduce both the model's storage demands and computational burden. However, advanced binary architectures still incorporate millions of inefficient and nonhardware-friendly full-precision multiplication operations. A&B BNN is proposed to directly remove part of the multiplication operations in a traditional BNN and replace the rest with an equal number of bit operations, introducing the mask layer and the quantized RPReLU structure based on the normalizer-free network architecture. The mask layer can be removed during inference by leveraging the intrinsic characteristics of BNN with straightforward mathematical transformations to avoid the associated multiplication operations. The quantized RPReLU structure enables more efficient bit operations by constraining its slope to be integer powers of 2. Experimental results achieved 92.30%, 69.35%, and 66.89% on the CIFAR-10, CIFAR-100, and ImageNet datasets, respectively, which are competitive with the state-of-the-art. Ablation studies have verified the efficacy of the quantized RPReLU structure, leading to a 1.14% enhancement on the ImageNet compared to using a fixed slope RLeakyReLU. The proposed add&bit-operation-only BNN offers an innovative approach for hardware-friendly network architecture.
format article
fullrecord <record><control><sourceid>proquest</sourceid><recordid>TN_cdi_proquest_journals_2941140346</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2941140346</sourcerecordid><originalsourceid>FETCH-proquest_journals_29411403463</originalsourceid><addsrcrecordid>eNqNik8LgjAcQEcQJOV3EAJvg7lN-3NzkXnSS3cZbMFMNvttIn77PPQBOj14721QRBnL8JlTukOx9z0hhBYnmucsQo8yFYlommtSKpUKE3A7apDBOItbOyxJLUHNEjSuwGirViOMlbAkjZ5ADivC7OB9QNuXHLyOf9yjY3V_3mo8gvtM2oeudxPYNXX0wrOME8YL9t_1BeqFOdo</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2941140346</pqid></control><display><type>article</type><title>A&amp;B BNN: Add&amp;Bit-Operation-Only Hardware-Friendly Binary Neural Network</title><source>Publicly Available Content Database</source><creator>Ma, Ruichen ; Qiao, Guanchao ; Liu, Yian ; Meng, Liwei ; Ning, Ning ; Liu, Yang ; Hu, Shaogang</creator><creatorcontrib>Ma, Ruichen ; Qiao, Guanchao ; Liu, Yian ; Meng, Liwei ; Ning, Ning ; Liu, Yang ; Hu, Shaogang</creatorcontrib><description>Binary neural networks utilize 1-bit quantized weights and activations to reduce both the model's storage demands and computational burden. However, advanced binary architectures still incorporate millions of inefficient and nonhardware-friendly full-precision multiplication operations. A&amp;B BNN is proposed to directly remove part of the multiplication operations in a traditional BNN and replace the rest with an equal number of bit operations, introducing the mask layer and the quantized RPReLU structure based on the normalizer-free network architecture. The mask layer can be removed during inference by leveraging the intrinsic characteristics of BNN with straightforward mathematical transformations to avoid the associated multiplication operations. The quantized RPReLU structure enables more efficient bit operations by constraining its slope to be integer powers of 2. Experimental results achieved 92.30%, 69.35%, and 66.89% on the CIFAR-10, CIFAR-100, and ImageNet datasets, respectively, which are competitive with the state-of-the-art. Ablation studies have verified the efficacy of the quantized RPReLU structure, leading to a 1.14% enhancement on the ImageNet compared to using a fixed slope RLeakyReLU. The proposed add&amp;bit-operation-only BNN offers an innovative approach for hardware-friendly network architecture.</description><identifier>EISSN: 2331-8422</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Ablation ; Computer architecture ; Hardware ; Neural networks ; State-of-the-art reviews</subject><ispartof>arXiv.org, 2024-03</ispartof><rights>2024. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.proquest.com/docview/2941140346?pq-origsite=primo$$EHTML$$P50$$Gproquest$$Hfree_for_read</linktohtml><link.rule.ids>780,784,25751,37010,44588</link.rule.ids></links><search><creatorcontrib>Ma, Ruichen</creatorcontrib><creatorcontrib>Qiao, Guanchao</creatorcontrib><creatorcontrib>Liu, Yian</creatorcontrib><creatorcontrib>Meng, Liwei</creatorcontrib><creatorcontrib>Ning, Ning</creatorcontrib><creatorcontrib>Liu, Yang</creatorcontrib><creatorcontrib>Hu, Shaogang</creatorcontrib><title>A&amp;B BNN: Add&amp;Bit-Operation-Only Hardware-Friendly Binary Neural Network</title><title>arXiv.org</title><description>Binary neural networks utilize 1-bit quantized weights and activations to reduce both the model's storage demands and computational burden. However, advanced binary architectures still incorporate millions of inefficient and nonhardware-friendly full-precision multiplication operations. A&amp;B BNN is proposed to directly remove part of the multiplication operations in a traditional BNN and replace the rest with an equal number of bit operations, introducing the mask layer and the quantized RPReLU structure based on the normalizer-free network architecture. The mask layer can be removed during inference by leveraging the intrinsic characteristics of BNN with straightforward mathematical transformations to avoid the associated multiplication operations. The quantized RPReLU structure enables more efficient bit operations by constraining its slope to be integer powers of 2. Experimental results achieved 92.30%, 69.35%, and 66.89% on the CIFAR-10, CIFAR-100, and ImageNet datasets, respectively, which are competitive with the state-of-the-art. Ablation studies have verified the efficacy of the quantized RPReLU structure, leading to a 1.14% enhancement on the ImageNet compared to using a fixed slope RLeakyReLU. The proposed add&amp;bit-operation-only BNN offers an innovative approach for hardware-friendly network architecture.</description><subject>Ablation</subject><subject>Computer architecture</subject><subject>Hardware</subject><subject>Neural networks</subject><subject>State-of-the-art reviews</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>PIMPY</sourceid><recordid>eNqNik8LgjAcQEcQJOV3EAJvg7lN-3NzkXnSS3cZbMFMNvttIn77PPQBOj14721QRBnL8JlTukOx9z0hhBYnmucsQo8yFYlommtSKpUKE3A7apDBOItbOyxJLUHNEjSuwGirViOMlbAkjZ5ADivC7OB9QNuXHLyOf9yjY3V_3mo8gvtM2oeudxPYNXX0wrOME8YL9t_1BeqFOdo</recordid><startdate>20240306</startdate><enddate>20240306</enddate><creator>Ma, Ruichen</creator><creator>Qiao, Guanchao</creator><creator>Liu, Yian</creator><creator>Meng, Liwei</creator><creator>Ning, Ning</creator><creator>Liu, Yang</creator><creator>Hu, Shaogang</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope></search><sort><creationdate>20240306</creationdate><title>A&amp;B BNN: Add&amp;Bit-Operation-Only Hardware-Friendly Binary Neural Network</title><author>Ma, Ruichen ; Qiao, Guanchao ; Liu, Yian ; Meng, Liwei ; Ning, Ning ; Liu, Yang ; Hu, Shaogang</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-proquest_journals_29411403463</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Ablation</topic><topic>Computer architecture</topic><topic>Hardware</topic><topic>Neural networks</topic><topic>State-of-the-art reviews</topic><toplevel>online_resources</toplevel><creatorcontrib>Ma, Ruichen</creatorcontrib><creatorcontrib>Qiao, Guanchao</creatorcontrib><creatorcontrib>Liu, Yian</creatorcontrib><creatorcontrib>Meng, Liwei</creatorcontrib><creatorcontrib>Ning, Ning</creatorcontrib><creatorcontrib>Liu, Yang</creatorcontrib><creatorcontrib>Hu, Shaogang</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science &amp; Engineering Collection</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central</collection><collection>ProQuest Central Essentials</collection><collection>AUTh Library subscriptions: ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Ma, Ruichen</au><au>Qiao, Guanchao</au><au>Liu, Yian</au><au>Meng, Liwei</au><au>Ning, Ning</au><au>Liu, Yang</au><au>Hu, Shaogang</au><format>book</format><genre>document</genre><ristype>GEN</ristype><atitle>A&amp;B BNN: Add&amp;Bit-Operation-Only Hardware-Friendly Binary Neural Network</atitle><jtitle>arXiv.org</jtitle><date>2024-03-06</date><risdate>2024</risdate><eissn>2331-8422</eissn><abstract>Binary neural networks utilize 1-bit quantized weights and activations to reduce both the model's storage demands and computational burden. However, advanced binary architectures still incorporate millions of inefficient and nonhardware-friendly full-precision multiplication operations. A&amp;B BNN is proposed to directly remove part of the multiplication operations in a traditional BNN and replace the rest with an equal number of bit operations, introducing the mask layer and the quantized RPReLU structure based on the normalizer-free network architecture. The mask layer can be removed during inference by leveraging the intrinsic characteristics of BNN with straightforward mathematical transformations to avoid the associated multiplication operations. The quantized RPReLU structure enables more efficient bit operations by constraining its slope to be integer powers of 2. Experimental results achieved 92.30%, 69.35%, and 66.89% on the CIFAR-10, CIFAR-100, and ImageNet datasets, respectively, which are competitive with the state-of-the-art. Ablation studies have verified the efficacy of the quantized RPReLU structure, leading to a 1.14% enhancement on the ImageNet compared to using a fixed slope RLeakyReLU. The proposed add&amp;bit-operation-only BNN offers an innovative approach for hardware-friendly network architecture.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier EISSN: 2331-8422
ispartof arXiv.org, 2024-03
issn 2331-8422
language eng
recordid cdi_proquest_journals_2941140346
source Publicly Available Content Database
subjects Ablation
Computer architecture
Hardware
Neural networks
State-of-the-art reviews
title A&B BNN: Add&Bit-Operation-Only Hardware-Friendly Binary Neural Network
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-10T03%3A36%3A28IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=document&rft.atitle=A&B%20BNN:%20Add&Bit-Operation-Only%20Hardware-Friendly%20Binary%20Neural%20Network&rft.jtitle=arXiv.org&rft.au=Ma,%20Ruichen&rft.date=2024-03-06&rft.eissn=2331-8422&rft_id=info:doi/&rft_dat=%3Cproquest%3E2941140346%3C/proquest%3E%3Cgrp_id%3Ecdi_FETCH-proquest_journals_29411403463%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2941140346&rft_id=info:pmid/&rfr_iscdi=true