![\"\"](\"https:\/\/ece.ncsu.edu\/wp-content\/uploads\/2020\/09\/Tianfu-Wu-AN-2020-HEADER-1024x576.jpg\")
<\/p>\n<\/p>\n
Artificial intelligence researchers at North Carolina State University have improved the performance of deep neural networks by combining feature normalization and feature attention modules into a single module that they call attentive normalization (AN). The hybrid module improves the accuracy of the system significantly while using negligible extra computational power.<\/p>\n
\u201cFeature normalization is a crucial element of training deep neural networks, and feature attention is equally important for helping networks highlight which features learned from raw data are most important for accomplishing a given task,\u201d says Tianfu Wu<\/a>, the corresponding author of a paper on the work and an assistant professor of electrical and computer engineering at NC\u00a0State. \u201cBut they have mostly been treated separately. We found that combining them made them more efficient and effective.\u201d<\/p>\n To test their AN module, the researchers plugged it into four of the most widely used neural network architectures: ResNets, DenseNets, MobileNetsV2, and AOGNets. They then tested the networks against two industry-standard benchmarks: the ImageNet-1000 classification benchmark and the MS-COCO 2017 object detection and instance segmentation benchmark.<\/p>\n \u201cWe found that AN improved performance for all four architectures on both benchmarks,\u201d Wu says. \u201cFor example, top-1 accuracy in the ImageNet-1000 improved by between 0.5% and 2.7%. And Average Precision (AP) accuracy increased by up to 1.8% for bounding box and 2.2% for semantic mask in MS-COCO.<\/p>\n \u201cAnother advantage of AN is that it facilitates better transfer learning between different domains,\u201d Wu says. \u201cFor example, from image classification in ImageNet to object detection and semantic segmentation in MS-COCO. This is illustrated by the performance improvement in the MS-COCO benchmark, which was obtained by fine-tuning ImageNet-pre-trained deep neural networks in MS-COCO, a common workflow in state-of-the-art computer vision.<\/p>\n \u201cWe have released the source code and hope our AN will lead to better integrative design of deep neural networks.\u201d<\/p>\n