Micro-kernels for portable and efficient matrix multiplication in deep learning

We provide a practical demonstration that it is possible to systematically generate a variety of high-performance micro-kernels for the general matrix multiplication ( gemm ) via generic templates which can be easily customized to different processor architectures and micro-kernel dimensions. These...

Full description

Saved in:
Bibliographic Details
Published in:The Journal of supercomputing 2023-05, Vol.79 (7), p.8124-8147
Main Authors: Alaejos, Guillermo, Castelló, Adrián, Martínez, Héctor, Alonso-Jordá, Pedro, Igual, Francisco D., Quintana-Ortí, Enrique S.
Format: Article
Language:English
Subjects:
Compilers
Computer Science
Deep learning
Floating point arithmetic
Interpreters
Kernels
Linear algebra
Matrices (mathematics)
Matrix algebra
Microprocessors
Multiplication
Neon
Processor Architectures
Programming Languages
SIMD (computers)
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We provide a practical demonstration that it is possible to systematically generate a variety of high-performance micro-kernels for the general matrix multiplication ( gemm ) via generic templates which can be easily customized to different processor architectures and micro-kernel dimensions. These generic templates employ vector intrinsics to exploit the SIMD (single instruction, multiple data) units in current general-purpose processors and, for the particular type of gemm problems encountered in deep learning, deliver a floating-point throughput rate on par with or even higher than that obtained with conventional, carefully tuned implementations of gemm in current linear algebra libraries (e.g., BLIS, AMD AOCL, ARMPL). Our work exposes the structure of the template-based micro-kernels for ARM Neon (128-bit SIMD), ARM SVE (variable-length SIMD) and Intel AVX512 (512-bit SIMD), showing considerable performance for an NVIDIA Carmel processor (ARM Neon), a Fujitsu A64FX processor (ARM SVE) and on an AMD EPYC 7282 processor (256-bit SIMD).
ISSN:0920-8542
1573-0484
DOI:10.1007/s11227-022-05003-3