SepMamba: State-space models for speaker separation using Mamba

Deep learning-based single-channel speaker separation has improved significantly in recent years largely due to the introduction of the transformer-based attention mechanism. However, these improvements come at the expense of intense computational demands, precluding their use in many practical appl...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-10
Main Authors: Avenstrup, Thor Højhus, Elek, Boldizsár, István László Mádi, Schin, András Bence, Mørup, Morten, Bjørn Sand Jensen, Olsen, Kenny Falkær
Format: Article
Language:English
Subjects:
Computational efficiency
Computing costs
Separation
State space models
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Deep learning-based single-channel speaker separation has improved significantly in recent years largely due to the introduction of the transformer-based attention mechanism. However, these improvements come at the expense of intense computational demands, precluding their use in many practical applications. As a computationally efficient alternative with similar modeling capabilities, Mamba was recently introduced. We propose SepMamba, a U-Net-based architecture composed primarily of bidirectional Mamba layers. We find that our approach outperforms similarly-sized prominent models - including transformer-based models - on the WSJ0 2-speaker dataset while enjoying a significant reduction in computational cost, memory usage, and forward pass time. We additionally report strong results for causal variants of SepMamba. Our approach provides a computationally favorable alternative to transformer-based architectures for deep speech separation.
ISSN:2331-8422