Enhancing neural network classification using fractional-order activation functions

In this paper, a series of novel activation functions is presented, which is derived using the improved Riemann–Liouville conformable fractional derivative (RLCFD). This study investigates the use of fractional activation functions in Multilayer Perceptron (MLP) models and their impact on the perfor...

Full description

Saved in:
Bibliographic Details
Published in:AI open 2024, Vol.5, p.10-22
Main Authors: Kumar, Meshach, Mehta, Utkal, Cirrincione, Giansalvo
Format: Article
Language:English
Subjects:
Accuracy
Activation functions
Classification
Computer Science
Fractional calculus
Multilayer perceptron
Neural networks
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper, a series of novel activation functions is presented, which is derived using the improved Riemann–Liouville conformable fractional derivative (RLCFD). This study investigates the use of fractional activation functions in Multilayer Perceptron (MLP) models and their impact on the performance of classification tasks, verified using the IRIS, MNIST and FMNIST datasets. Fractional activation functions introduce a non-integer power exponent, allowing for improved capturing of complex patterns and representations. The experiment compares MLP models employing fractional activation functions, such as fractional sigmoid, hyperbolic tangent and rectified linear units, against traditional models using standard activation functions, their improved versions and existing fractional functions. The numerical studies have confirmed the theoretical observations mentioned in the paper. The findings highlight the potential usage of new functions as a valuable tool in deep learning in classification. The study suggests incorporating fractional activation functions in MLP architectures can lead to superior accuracy and robustness.
ISSN:2666-6510
2666-6510
DOI:10.1016/j.aiopen.2023.12.003