A Levitated Controlled Attention for Named Entity Recognition

• Published in: Cognitive computation 2025-02, Vol.17 (1), p.1, Article 1
• Main Authors: Huang, Rong, Chen, Yanping, Huang, Ruizhang
• Format: Article
• Language: English
• Subjects: Artificial Intelligence; Artificial neural networks; Attention; Classification; Complexity; Computation by Abstract Devices; Computational Biology/Bioinformatics; Computer Science; Labeling; Natural language processing; Neural networks; Performance evaluation; Proposals; Semantics; Sentences; Turing, Alan
• ISSN: 1866-9956; 1866-9964
• DOI: 10.1007/s12559-024-10381-2

Controlled attention is a mechanism developed in cognitive neuroscience. It has been successfully applied to support named entity recognition, where the start and end boundaries of a possible named entity are marked by two specific tokens to indicate its position in a sentence. Then, it is fed into a deep network for classification. The entity boundary markers enable a deep neural network to be aware of entity boundaries and build the contextual dependency of a sentence relevant to entity boundaries. The problem with this strategy is that every possible named entity must be evaluated independently. This leads to very high computational complexity and cannot construct the semantic dependency between different named entities. In this paper, a levitated controlled attention mechanism is presented for named entity recognition. In this method, all possible named entities are fed together into a deep network for one-pass classification, which can establish the semantic dependency between contextual features and possible named entities. In the experiments, the levitated controlled attention is evaluated on four public datasets. The results show that it not only considerably reduces the computational complexity but also improves the performance of named entity recognition.