Please wait a minute...
Big Data Mining and Analytics  2020, Vol. 3 Issue (4): 300-310    DOI: 10.26599/BDMA.2020.9020021
    
DFF-ResNet: An Insect Pest Recognition Model Based on Residual Networks
Wenjie Liu(),Guoqing Wu*(),Fuji Ren*(),Xin Kang()
School of Information Science and Technology and School of Transportation and Civil Engineering, Nantong University, Nantong 226019, China, and also with the Faculty of Engineering, Tokushima University, Tokushima 770-8506, Japan
School of Information Science and Technology, Nantong University, Nantong 226019, China
Faculty of Engineering, Tokushima University, Tokushima 770-8506, Japan
Download: PDF (15426 KB)      HTML  
Export: BibTeX | EndNote (RIS)      

Abstract  

Insect pest control is considered as a significant factor in the yield of commercial crops. Thus, to avoid economic losses, we need a valid method for insect pest recognition. In this paper, we proposed a feature fusion residual block to perform the insect pest recognition task. Based on the original residual block, we fused the feature from a previous layer between two 1×1 convolution layers in a residual signal branch to improve the capacity of the block. Furthermore, we explored the contribution of each residual group to the model performance. We found that adding the residual blocks of earlier residual groups promotes the model performance significantly, which improves the capacity of generalization of the model. By stacking the feature fusion residual block, we constructed the Deep Feature Fusion Residual Network (DFF-ResNet). To prove the validity and adaptivity of our approach, we constructed it with two common residual networks (Pre-ResNet and Wide Residual Network (WRN)) and validated these models on the Canadian Institute For Advanced Research (CIFAR) and Street View House Number (SVHN) benchmark datasets. The experimental results indicate that our models have a lower test error than those of baseline models. Then, we applied our models to recognize insect pests and obtained validity on the IP102 benchmark dataset. The experimental results show that our models outperform the original ResNet and other state-of-the-art methods.



Key wordsinsect pest recognition      deep feature fusion      residual network      image classification     
Received: 20 July 2020      Published: 07 December 2020
Fund:  Research Clusters Program of Tokushima University and JSPS KAKENHI(No. 19K20345)
Corresponding Authors: Guoqing Wu,Fuji Ren     E-mail: lwj2014@ntu.edu.cn;wgq@ntu.edu.cn;ren@is.tokushima-u.ac.jp;kang-xin@is.tokushima-u.ac.jp
About author: Wenjie Liu received the BS degree in information engineering from Nanhang Jincheng College, China in 2011, the MS degree in information and communication engineering from Nantong University, China in 2014. He is currently pursuing the double PhD degree at Nantong University and Tokushima University. His research interests include image analysis, computer vision, and artificial intelligence.|Guoqing Wu received the BS and MS degrees in mechatronics from Jiangsu University, China in 1983 and 1993, respectively, and the PhD degree in mechanical design and theory from Shanghai University, China in 2006. He is currently a professor at Nantong University, China. His research interests are in the area of mechanical engineering, laser technology application, and artificial intelligence.|Fuji Ren received the PhD degree from the Faculty of Engineering, Hokkaido University, Japan in 1991. From 1991 to 1994, he worked at CSK as a chief researcher. In 1994, he joined the Faculty of Information Sciences, Hiroshima City University, as an associate professor. Since 2001, he has been a professor of the Faculty of Engineering, Tokushima University. His current research interests include natural language processing, artificial intelligence, affective computing, and emotional robot. He is the academician of the Engineering Academy of Japan and EU Academy of Sciences. He is a senior member of IEEE, editor-in-chief of International Journal of Advanced Intelligence, a vice president of CAAI, and a fellow of the Japan Federation of Engineering Societies, a fellow of IEICE, and a fellow of CAAI. He is the president of International Advanced Information Institute, Japan.|Xin Kang received the PhD degree from Tokushima University, Tokushima, Japan in 2013, the ME degree from Beijing University of Posts and Telecommunications, Beijing, China in 2009, and the BE degree from Northeastern University, Shenyang, China in 2006. He is currently an assistant professor at Tokushima University. His research interests include statistical machine learning, probabilistic graphical models, neural networks, and text emotion prediction.
Cite this article:

Wenjie Liu,Guoqing Wu,Fuji Ren,Xin Kang. DFF-ResNet: An Insect Pest Recognition Model Based on Residual Networks. Big Data Mining and Analytics, 2020, 3(4): 300-310.

URL:

http://bigdata.tsinghuajournals.com/10.26599/BDMA.2020.9020021     OR     http://bigdata.tsinghuajournals.com/Y2020/V3/I4/300

Fig. 1 Architecture of Pre-ResNet and FF-Pre-ResNet (c represents concatenate operation).
Fig. 2 Comparison result of different architectures on the CIFAR-100 dataset. The structure of B(1, 3, 3, 1) achieves the best results for the 218-layer and 302-layer DFF-Pre-ResNets.
GroupNumber of layersNumber of filters
Group 1 (32×32)[n×k×m]16
Group 2 (16×16)[n×k]32
Group 3 (8×8)n64
Table 1 Architecture of DFF-Pre-ResNets for CIFAR datasets.
Fig. 3 Test error of adding residual block of earlier residual groups applied to the 218-layer DFF-Pre-ResNet and 110-layer Pre-ResNet on the CIFAR-100 dataset under different hyper-parameters (k, m).
ModelDepthNumber of parameters (×106)Test error (without SD) (%)Test error (with SD) (%)
Pre-ResNet1101.75.224.71
1642.64.754.69
DFF-Pre-ResNet2181.74.184.19
3022.54.083.98
Table 2 Comparison of test errors on CIFAR-10.
Fig. 4 Test error curves (smoothed) on CIFAR-10 by DFF-Pre-ResNet and baseline models during training period with corresponding results reported in Table 2. DFF-Pre-ResNet yields a lower test error than Pre-ResNet.
ModelDepthNumber of parameters (×106)Test error (without SD) (%)Test error (with SD) (%)
Pre-ResNet1101.725.9323.99
1642.625.0622.98
DFF-Pre-ResNet2181.722.6920.92
3022.522.2520.53
Table 3 Comparison of test error on CIFAR-100.
Fig. 5 Test error curves (smoothed) on CIFAR-100 by DFF-Pre-ResNet and baseline models during training period with corresponding results reported in Table 3. DFF-Pre-ResNet yields a lower test error than Pre-ResNet.
ModelDepth-widthNumber of parameters (×106)CIFAR-10CIFAR-100
Test error (without SD) (%)Test error (with SD) (%)Test error (without SD) (%)Test error (with SD) (%)
WRN40-22.24.634.2524.4222.21
40-48.94.073.9821.8520.28
DFF-WRN52-22.04.413.9323.1921.45
52-47.94.083.5120.7319.09
Table 4 Comparison of test errors on CIFAR-10 and CIFAR-100 datasets.
Fig. 6 Test error curves (smoothed) on CIFAR-10 for the DFF-WRN and baseline models during the training period with the corresponding results reported in Table 4. DFF-WRN yields a lower test error than WRN.
Fig. 7 Test error curves (smoothed) on CIFAR-100 for the DFF-WRN and baseline models during the training period with the corresponding results reported in Table 4. DFF-WRN yields a lower test error than WRN.
DepthTest error (%)
CIFAR-10CIFAR-100
218-layer4.1920.92
302-layer3.9820.53
378-layer3.7519.92
1050-layer3.6718.71
Table 5 Comparison of test errors on CIFAR-10 and CIFAR-100 for DFF-Pre-ResNet+SD with different depths.
ModelTest error (%)
CIFAR-10CIFAR-100
DFF-WRN52-23.9321.45
DFF-WRN52-43.5119.09
DFF-WRN52-83.3117.83
DFF-WRN52-8+mixup2.4215.59
Table 6 Comparison of test errors on CIFAR-10 and CIFAR-100 for DFF-WRN+SD with different widths.
ModelDepth-widthNumber of parameters (×106)Test error (without SD) (%)Test error (with SD) (%)
WRN40-48.91.691.75
DFF-WRN52-47.91.761.54
52-831.4-1.53
Table 7 Comparison of test errors on SVHN.
Fig. 8 Test error curves (smoothed) on SVHN for the DFF-WRN and baseline models. The corresponding results are reported in Table 7.
ModelNumber of parameters (×106)F1 score (%)Test accuracy (%)
AlexNet[36]57.4248.2249.41
50-layer ResNet[6]23.7252.9354.19
101-layer ResNet[6]42.6352.0053.07
Googlenet[26]10.2451.2452.17
16-layer VGG[37]134.6851.2051.84
121-layer DenseNet[7]7.0652.9754.59
62-layer DFF-Pre-ResNet22.5453.9855.39
82-layer DFF-Pre-ResNet30.2054.1855.43
Table 8 Comparison of F1 scores and test accuracies on IP102 for the DFF-Pre-ResNet and other state-of-the-art methods.
Fig. 9 Test accuracy and training loss curves on the evaluation set during the training period.
k=1.3, m=1.1) and 182-layer DFF-Pre-ResNet (k=1.0, m=1.0), using the CIFAR-100 dataset.
">
Fig. 10 Performance comparison between the 218-layer DFF-Pre-ResNet (k=1.3, m=1.1) and 182-layer DFF-Pre-ResNet (k=1.0, m=1.0), using the CIFAR-100 dataset.
[1]   Wu X. P., Zhan C., Lai Y. K., Cheng M. M., and Yang J. F., IP102: A large-scale benchmark dataset for insect pest recognition, in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, Long Beach, CA, USA, 2019, pp. 8779-8788.
[2]   Deng L. M., Wang Y. J., Han Z. Z., and Yu R. S., Research on insect pest image detection and recognition based on bio-inspired methods, Biosystems Engineering, vol. 169, pp. 139-148, 2018.
[3]   Dimililer K. and Zarrouk S., ICSPI: Intelligent classification system of pest insects based on image processing and neural arbitration, Applied Engineering in Agriculture, vol. 33, no. 4, pp. 453-460, 2017.
[4]   Ren F. J., Liu W. J., and Wu G. Q., Feature reuse residual networks for insect pest recognition, IEEE Access, vol. 7, pp. 122 758-122 768, 2019.
[5]   Krizhevsky A., Sutskever I., and Hinton G. E., ImageNet classification with deep convolutional neural networks, Communications of the ACM, vol. 60, no. 6, pp. 84-90, 2017.
[6]   He K. M., Zhang X. Y., Ren S. Q., and Sun J., Deep residual learning for image recognition, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016, pp. 770-778.
[7]   Huang G., Liu Z., Van Der Maaten L., and Weinberger K. Q., Densely connected convolutional networks, in Proc. 2017 IEEE Conf. Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017, pp. 2261-2269.
[8]   Shen Z. Q., Liu Z., Li J. G., Jiang Y. G., Chen Y. R., and Xue X. Y., DSOD: Learning deeply supervised object detectors from scratch, in Proc. 2017 IEEE Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 1937-1945.
[9]   Zagoruyko S. and Komodakis N., Wide residual networks, arXiv preprint arXiv:1605.07146, 2016.
[10]   Feng D. and Ren F. J., Dynamic facial expression recognition based on two-stream-CNN with LBP-TOP, presented at 2018 5th IEEE Int. Conf. Cloud Computing and Intelligence Systems (CCIS), Nanjing, China, 2018, pp. 355-359.
[11]   Ren F. J. and Deng J. W., Background knowledge based multi-stream neural network for text classification, Applied Sciences, vol. 8, no. 12, p. 2472, 2018.
[12]   Kim Y., Convolutional neural networks for sentence classification, arXiv preprint arXiv:1408.5882, 2014.
[13]   Zhang R. Y., Meng F. R., Zhou Y., and Liu B., Relation classification via recurrent neural network with attention and tensor layers, Big Data Mining and Analytics, vol. 1, no. 3, pp. 234-244, 2018.
[14]   Ren F. J., Dong Y. D., and Wang W., Emotion recognition based on physiological signals using brain asymmetry index and echo state network, Neural Computing and Applications, vol. 31, no. 9, pp. 4491-4501, 2019.
[15]   Kang X., Ren F. J., and Wu Y. N., Exploring latent semantic information for textual emotion recognition in blog articles, IEEE/CAA Journal of Automatica Sinica, vol. 5, no. 1, pp. 204-216, 2018.
[16]   Bouazizi M. and Ohtsuki T., Multi-class sentiment analysis on twitter: Classification performance and challenges, Big Data Mining and Analytics, vol. 2, no. 3, pp. 181-194, 2019.
[17]   Bell S., Zitnick C. L., Bala K., and Girshick R., Inside-outside net: Detecting objects in context with skip pooling and recurrent neural networks, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016, pp. 2874-2883.
[18]   Milz S., Arbeiter G., Witt C., Abdallah B., and Yogamani S., Visual SLAM for automated driving: Exploring the applications of deep learning, in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA, 2018, pp. 360-370.
[19]   Cao X. H., Li T. H., Li H. L., Xia S. R., Ren F. J., Sun Y., and Xu X. Y., A robust parameter-free thresholding method for image segmentation, IEEE Access, vol. 7, pp. 3448-3458, 2018.
[20]   Ma N. N., Zhang X. Y., Zheng H. T., and Sun J., ShuffleNet v2: Practical guidelines for efficient CNN architecture design, in Proc. European Conf. Computer Vision (ECCV), Munich, Germany, 2018, pp. 122-138.
[21]   He K. M., Zhang X. Y., Ren S. Q., and Sun J., Identity mappings in deep residual networks, presented at European Conf. Computer Vision, Amsterdam, The Netherlands, 2016, pp. 630-645.
[22]   Ng P. C. and Henikoff S., SIFT: Predicting amino acid changes that affect protein function, Nucleic Acids Research, vol. 31, no. 13, pp. 3812-3814, 2003.
[23]   Dalal N. and Triggs B., Histograms of oriented gradients for human detection, presented at 2005 IEEE Computer Society Conf. Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA, 2005, pp. 886-893.
[24]   Samanta R. K. and Ghosh I., Tea insect pests classification based on artificial neural networks, International Journal of Computer Engineering Science (IJCES), vol. 2, no. 6, pp. 1-13, 2012.
[25]   Manoja M. and Rajalakshmi J., Early detection of pest on leaves using support vector machine, International Journal of Electrical and Electronics Research, vol. 2, no. 4, pp. 187-194, 2014.
[26]   Szegedy C., Liu W., Jia Y. Q., Sermanet P., Reed S., Anguelov D., Erhan D., Vanhoucke V., and Rabinovich A., Going deeper with convolutions, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Boston, MA, USA, 2015, pp. 1-9.
[27]   Li R., Wang R. J., Zhang J., Xie C. J., Liu L., Wang F. Y., Chen H. B., Chen T. J., Hu H. Y., Jia X. F., et al., An effective data augmentation strategy for CNN-based pest localization and recognition in the field, IEEE Access, vol. 7, pp. 160 274-160 283, 2019.
[28]   Dimililer K. and Zarrouk S., ICSPI: Intelligent classification system of pest insects based on image processing and neural arbitration, Applied Engineering in Agriculture, vol. 33, no. 4, pp. 453-460, 2017.
[29]   Shen F. L., Gan R., and Zeng G., Weighted residuals for very deep networks, presented at 2016 3rd Int. Conf. Systems and Informatics (ICSAI), Shanghai, China, 2016, pp. 936-941.
[30]   Huang G., Sun Y., Liu Z., Sedra D., and Weinberger K. Q., Deep networks with stochastic depth, presented at European Conf. Computer Vision, Amsterdam, The Netherlands, 2016, pp. 646-661.
[31]   Han D., Kim J., and Kim J., Deep pyramidal residual networks, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017, pp. 6307-6315.
[32]   Zhang K., Sun M., Han T. X., Yuan X. F., Guo L. R., and Liu T., Residual networks of residual networks: Multilevel residual networks, IEEE Transactions on Circuits and Systems for Video Technology, vol. 28, no. 6, pp. 1303-1314, 2018.
[33]   Huang G., Liu S. C., Van der Maaten L., and Weinberger K. Q., CondenseNet: An efficient densenet using learned group convolutions, in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 2752-2761.
[34]   He K. M., Zhang X. Y., Ren S. Q., and Sun J., Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification, in Proc. 2015 IEEE Int. Conf. Computer Vision, Santiago, Chile, 2015, pp. 1026-1034.
[35]   Zhang H. Y., Cisse M., Dauphin Y. N., and Lopez-Paz D., Mixup: Beyond empirical risk minimization, arXiv preprint arXiv:1710.09412, 2017.
[36]   Krizhevsky A., Sutskever I., and Hinton G. E., ImageNet classification with deep convolutional neural networks, Communications of the ACM, vol. 60, no. 6, pp. 84-90, 2017.
[37]   Simonyan K. and Zisserman A., Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556, 2014.
[1] Jin Liu, Yi Pan, Min Li, Ziyue Chen, Lu Tang, Chengqian Lu, Jianxin Wang. Applications of Deep Learning to MRI Images: A Survey[J]. Big Data Mining and Analytics, 2018, 1(1): 1-18.