Filter-in-Filter: Low Cost CNN Improvement by Sub-filter Parameter Sharing

• Published in: Pattern recognition 2019-07, Vol.91, p.391-403
• Main Authors: Xie, Guotian, Yang, Kuiyuan, Lai, Jianhuang
• Format: Article
• Language: English
• Subjects: Expressibility of filter; Filter-in-filter; Sub-filter; Sub-pattern; Visualization
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2019.01.044

•We defined the sub-filters of a filter and visualized them to verify these sub-filters can recognize multiple meaningful patterns.•Filter-in-Filter was proposed to make full use of the sub-filters to enhance the expressibility of the filters in CNNs.•Filter-in-Filter does not increase the number of parameters and increases the computational cost only slightly as compared to the standard convolution. We verified that FIF is effective to improve the performance of CNNs by conducting extensive experiments. Increasing the number of parameters seems to have improved convolutional neural networks, e.g. increasing the depth or width of the networks. In this paper, we propose a scheme to improve CNNs by deriving the six sub-filters from a filter, which share parameters among them and enhance the expressibility of the filter. We first defined the sub-filters of a filter, and by visualizing a well-trained CNN, we verified that these sub-filters could recognize multiple meaningful patterns with different visual characteristics, even when the filter containing them was not activated. These findings revealed that the filter has the potential to recognize multiple patterns. Inspired by these findings, we proposed the filter-in-filter (FIF) scheme to enhance the expressibility of a filter, by making full use of its sub-filters to recognize multiple meaningful sub-patterns. We verified the effectiveness of FIF on three image classification benchmark datasets, namely Tiny ImageNet, CIFAR-100 and ImageNet. Our experimental results showed that our models achieved consistent improvement over the base CNNs on the benchmark datasets, e.g. AlexNet and VGG16 using FIF achieved approximately 1% improvement on ImageNet. The sub-filters share the parameters and most of the computational cost with the filter containing them; therefore, FIF does not increase the number of parameters and increases the computational cost only slightly.