Please wait a minute...
Big Data Mining and Analytics  2020, Vol. 3 Issue (2): 102-120    DOI: 10.26599/BDMA.2019.9020024
    
Feature Representations Using the Reflected Rectified Linear Unit (RReLU) Activation
Chaity Banerjee, Tathagata Mukherjee*, Eduardo Pasiliao Jr.
∙ Chaity Banerjee is with Department of Idustrial & Systems Engineering, University of Central Florida, Orlando, FL 32816-2368, USA. E-mail: Chaity.BanerjeeMukherjee@ucf.edu.
∙ Tathagata Mukherjee is with the Department of Computer Science, University of Alabama in Huntsville, Huntsville, AL 35806, USA.
∙ Eduardo Pasiliao Jr. is with Air Force Research Labs, United States Air Force, Eglin Air Force Base, Shalimar, FL 32579, USA. E-mail: elpasiliao@gmail.com.
Download: PDF (9506 KB)      HTML  
Export: BibTeX | EndNote (RIS)      

Abstract  

Deep Neural Networks (DNNs) have become the tool of choice for machine learning practitioners today. One important aspect of designing a neural network is the choice of the activation function to be used at the neurons of the different layers. In this work, we introduce a four-output activation function called the Reflected Rectified Linear Unit (RReLU) activation which considers both a feature and its negation during computation. Our activation function is "sparse", in that only two of the four possible outputs are active at a given time. We test our activation function on the standard MNIST and CIFAR-10 datasets, which are classification problems, as well as on a novel Computational Fluid Dynamics (CFD) dataset which is posed as a regression problem. On the baseline network for the MNIST dataset, having two hidden layers, our activation function improves the validation accuracy from 0.09 to 0.97 compared to the well-known ReLU activation. For the CIFAR-10 dataset, we use a deep baseline network that achieves 0.78 validation accuracy with 20 epochs but overfits the data. Using the RReLU activation, we can achieve the same accuracy without overfitting the data. For the CFD dataset, we show that the RReLU activation can reduce the number of epochs from 100 (using ReLU) to 10 while obtaining the same levels of performance.



Key wordsdeep learning      feature space      approximations      multi-output activations      Rectified Linear Unit (ReLU)     
Received: 25 November 2019      Published: 28 September 2020
Corresponding Authors: Tathagata Mukherjee   
Cite this article:

Chaity Banerjee, Tathagata Mukherjee, Eduardo Pasiliao Jr.. Feature Representations Using the Reflected Rectified Linear Unit (RReLU) Activation. Big Data Mining and Analytics, 2020, 3(2): 102-120.

URL:

http://bigdata.tsinghuajournals.com/10.26599/BDMA.2019.9020024     OR     http://bigdata.tsinghuajournals.com/Y2020/V3/I2/102

Figure 1 Choice of the phase parameters.
Figure 2 Reflected ReLU: Simultaneous min-max convolutions.
Figure 3 Positive response plot for RReLU. The <i>x</i>-axis is the input feature and <i>y</i>-axis is the response corresponding to input feature <i>x</i>.
Figure 4 Negative response plot for RReLU. The <i>x</i>-axis is the input feature and <i>y</i>-axis is the response corresponding to input feature <i>x</i>.
ActivationTrain-AccTest-Acc
ReLU0.1100.09
pRReLU0.9840.97
Simple RReLU0.9600.96
Table 1 Results for MNIST network with ReLU, pRReLU, and simple RReLU.
Figure 5 Training & testing accuracy of baseline MNIST network (ReLU).
α=𝟏𝟑 and β=-𝟑.">
Figure 6 Training & testing accuracy of MNIST network using RReLU with phase parameters <inline-formula><math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="MA129"><mml:mrow><mml:mi>α</mml:mi><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:mfrac><mml:mtext>𝟏</mml:mtext><mml:msqrt><mml:mtext>𝟑</mml:mtext></mml:msqrt></mml:mfrac></mml:mstyle></mml:mrow></math></inline-formula> and <inline-formula><math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="MA130"><mml:mrow><mml:mi>β</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mo>-</mml:mo><mml:msqrt><mml:mtext>𝟑</mml:mtext></mml:msqrt></mml:mrow></mml:mrow></math></inline-formula>.
Figure 7 Training & testing accuracy of MNIST network using simple RReLU.
ActivationTrain-AccTest-Acc
ReLU0.81610.7838
pRReLU (single layer)0.83870.8004
pRReLU0.74830.7805
Simple RReLU0.74450.7693
Table 2 Results for CIFAR-10 network with ReLU, pRReLU (single layer), pRReLU, and simple RReLU.
α=𝟏𝟑 and β=-𝟑 in all convolutional layers. Note that there is no overfitting.">
Figure 8 Training & testing accuracy of CIFAR-10 network with RReLU and phase parameters <inline-formula><math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="MA140"><mml:mrow><mml:mi>α</mml:mi><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:mfrac><mml:mtext>𝟏</mml:mtext><mml:msqrt><mml:mtext>𝟑</mml:mtext></mml:msqrt></mml:mfrac></mml:mstyle></mml:mrow></math></inline-formula> and <inline-formula><math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="MA141"><mml:mrow><mml:mi>β</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mo>-</mml:mo><mml:msqrt><mml:mtext>𝟑</mml:mtext></mml:msqrt></mml:mrow></mml:mrow></math></inline-formula> in all convolutional layers. Note that there is no overfitting.
α=𝟏𝟑 and β=-𝟑 after the first convolutional layer only (model overfits).">
Figure 9 Training & testing accuracy of CIFAR-10 network with RReLU and phase parameters <inline-formula><math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="MA142"><mml:mrow><mml:mi>α</mml:mi><mml:mo mathvariant="normal">=</mml:mo><mml:mstyle displaystyle="true"><mml:mfrac><mml:mtext>𝟏</mml:mtext><mml:msqrt><mml:mtext>𝟑</mml:mtext></mml:msqrt></mml:mfrac></mml:mstyle></mml:mrow></math></inline-formula> and <inline-formula><math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="MA143"><mml:mrow><mml:mi>β</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mo>-</mml:mo><mml:msqrt><mml:mtext>𝟑</mml:mtext></mml:msqrt></mml:mrow></mml:mrow></math></inline-formula> after the first convolutional layer only (model overfits).
α=𝟏𝟑 and β=-𝟑 after the second convolutional layer only (model overfits).">
Figure 10 Training & testing accuracy of CIFAR-10 network with RReLU and phase parameters <inline-formula><math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="MA144"><mml:mrow><mml:mi>α</mml:mi><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:mfrac><mml:mtext>𝟏</mml:mtext><mml:msqrt><mml:mtext>𝟑</mml:mtext></mml:msqrt></mml:mfrac></mml:mstyle></mml:mrow></math></inline-formula> and <inline-formula><math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="MA145"><mml:mrow><mml:mi>β</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mo>-</mml:mo><mml:msqrt><mml:mtext>𝟑</mml:mtext></mml:msqrt></mml:mrow></mml:mrow></math></inline-formula> after the second convolutional layer only (model overfits).
×32).">
Figure 11 Single test image from CIFAR-10 dataset. Note that the image is blurred and hard for humans to decipher. The <i>x</i> and <i>y</i> axes show the dimensions of the image (32<inline-formula><math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="MA146"><mml:mo>×</mml:mo></math></inline-formula>32).
α=𝟏𝟑 and β=-𝟑 after the training with 20 epochs.">
Figure 12 Features from the second convolutional layer before RReLU with phase parameters <inline-formula><math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="MA147"><mml:mrow><mml:mi>α</mml:mi><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:mfrac><mml:mtext>𝟏</mml:mtext><mml:msqrt><mml:mtext>𝟑</mml:mtext></mml:msqrt></mml:mfrac></mml:mstyle></mml:mrow></math></inline-formula> and <inline-formula><math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="MA148"><mml:mrow><mml:mi>β</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mo>-</mml:mo><mml:msqrt><mml:mtext>𝟑</mml:mtext></mml:msqrt></mml:mrow></mml:mrow></math></inline-formula> after the training with 20 epochs.
Figure 13 Features from the second convolutional layer before ReLU after the training with 20 epochs.
α=𝟏𝟑 and β=-𝟑 with 20 epochs after RReLU on convolution output.">
Figure 14 Features from the second convolutional layer with RReLU and phase parameters <inline-formula><math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="MA149"><mml:mrow><mml:mi>α</mml:mi><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:mfrac><mml:mtext>𝟏</mml:mtext><mml:msqrt><mml:mtext>𝟑</mml:mtext></mml:msqrt></mml:mfrac></mml:mstyle></mml:mrow></math></inline-formula> and <inline-formula><math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="MA150"><mml:mrow><mml:mi>β</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mo>-</mml:mo><mml:msqrt><mml:mtext>𝟑</mml:mtext></mml:msqrt></mml:mrow></mml:mrow></math></inline-formula> with 20 epochs after RReLU on convolution output.
Figure 15 Features from the second convolutional layer after ReLU with training for 20 epochs.
Figure 16 Training & tesing accuracy of CIFAR-10 network with simple RReLU in all convolutional layers. Note that there is no overfitting.
Figure 17 Features from the second convolutional layer after simple RReLU with training for 20 epochs.
Figure 18 Sensor locations for collecting free-stream data.
Figure 19 Trajectory traced out by the projectile. This is obtained by plotting the pitch and yaw of the object as it moves.
Figure 20 Projection of pitch and yaw angles on surface of unit sphere for the free-stream data. Here <i>x</i>, <i>y</i>, and <i>z</i> axes represent the corresponding co-ordinates of the points in the cartesian system.
NetworkTrain-MSEValidation-MSE
CFD (RReLU)0.33340.3329
Table 3 Results for CFD baseline network with RRELU (phase parameters α=𝟏𝟑 and β=-𝟑) in all layers except the last two layers after 10 epochs. Note: 20/80 test/ train split. MSE: mean squared error.
Figure 21 Convergence of training and testing losses after 10 epochs. Note that there are variations after 2 epochs that are not visible due to scale of plot as the variations beyond four decimal places. Note: 20/80 test/train split.
Figure 22 Convergence of testing loss after 10 epochs. We show the variation after four decimal places in this plot. Note: 20/80 test/train split.
Figure 23 Convergence of training loss after 50 epochs. Note that there are variations after 2 epochs that are not visible due to scale of plot as the variations beyond four decimal places. Note: 20/80 test/train split.
Figure 24 Convergence of testing loss after 50 epochs. We show the variation after four decimal places in this plot. Note: 20/80 test/train split.
[1]   Goodfellow I., Bengio Y., and Courville A., Deep Learning. Cambridge, MA, USA: MIT Press, 2016.
[2]   Parkhi O. M., Vedaldi A., and Zisserman A., Deep face recognition, in Proc. British Machine Vision Conf., Swansea, UK, 2015.
[3]   Yu Z., Li T. R., Yu N., Gong X., Chen K., and Pan Y., Three-stream convolutional networks for video-based person re-identification, arXiv preprint arXiv: 1712.01652, 2017.
[4]   Socher R., Bengio Y., and Manning C. D., Deep learning for NLP (without magic), in Proc. Tutorial Abstracts of ACL 2012, Jeju Island, Korea, 2012, p. 5.
[5]   Roy D., Mukherjee T., Chatterjee M., and Pasiliao E., Detection of rogue RF transmitters using generative adversarial nets, in 2019 IEEE Wireless Communications and Networking Conf., Marrakesh, Morocco, 2019.
[6]   Byravan A. and Fox D., SE3-nets: Learning rigid body motion using deep neural networks, in 2017 IEEE Int. Conf. Robotics and Automation, Singapore, 2017, pp. 173-180.
[7]   Liu J., Pan Y., Li M., Chen Z. Y., Tang L., Lu C. Q., and Wang J. X., Applications of deep learning to MRI images: A survey, Big Data Mining and Analytics, vol. 1, no. 1, pp. 1-18, 2018.
[8]   Zeng M., Li M., Fei Z. H., Wu F. X., Li Y. H., Pan Y., and Wang J. X., A deep learning framework for identifying essential proteins by integrating multiple types of biological information, IEEE/ACM Trans. Comput. Biol. Bioinfor., doi: 10.1109/TCBB.2019.2897679.
[9]   Yan M., Liu L., Chen S. H., and Pan Y., A deep learning method for prediction of benign epilepsy with centrotemporal spikes, in Proc. 14th Int. Symp. Bioinformatics Research and Applications, Beijing, China, 2018, pp. 253-258.
[10]   Yu N., Yu Z., Gu F., Li T. R., Tian X. X., and Pan Y., Deep learning in genomic and medical image data analysis: Challenges and approaches, J. Inf. Process. Syst., vol. 13, no. 2, pp. 204-214, 2017.
[11]   Zhang C. Y., Bengio S., Hardt M., Recht B., and Vinyals O., Understanding deep learning requires rethinking generalization, arXiv preprint arXiv: 1611.03530, 2016.
[12]   Hornik K., Stinchcombe M., and White H., Multilayer feedforward networks are universal approximators, Neural Networks, vol. 2, no. 5, pp. 359-366, 1989.
[13]   Loroch D. M., Pfreundt F. J., Wehn N., and Keuper J., Sparsity in deep neural networks—An empirical investigation with tensorQuant, in Joint European Conf. Machine Learning and Knowledge Discovery in Databases, Dublin, Ireland, 2018, pp. 5-20.
[14]   Shang W. L., Sohn K., Almeida D., and Lee H., Understanding and improving convolutional neural networks via concatenated rectified linear units, in Proc. 33rd Int. Conf. Machine Learning, New York, NY, USA, 2016, pp. 2217-2225.
[15]   Blot M., Cord M., and Thome N., Max-min convolutional neural networks for image classification, in 2016 IEEE Int. Conf. Image Processing, Phoenix, AZ, USA, 2016, pp. 3678-3682.
[16]   Rolnick D. and Tegmark M., The power of deeper networks for expressing natural functions, arXiv preprint arXiv: 1705.05502, 2017.
[17]   Neal R. M., Connectionist learning of belief networks, Artif. Intell., vol. 56, no. 1, pp. 71-113, 1992.
[18]   Glorot X. and Bengio Y., Understanding the difficulty of training deep feedforward neural networks, in Proc. 13th Int. Conf. Artificial Intelligence and Statistics, Sardinia, Italy, 2010, pp. 249-256.
[19]   Courbariaux M., Bengio Y., and David J. P., BinaryConnect: Training deep neural networks with binary weights during propagations, in Proc. 28th Int. Conf. Neural Information Processing Systems, Montreal, Canada, 2015, pp. 3123-3131.
[20]   Elfwing S., Uchibe E., and Doya K., Sigmoid-weighted linear units for neural network function approximation in reinforcement learning, Neural Networks, vol. 107, pp. 3-11, 2018.
[21]   Ramachandran P., Zoph B., and Le Q. V., Searching for activation functions, arXiv preprint arXiv: 1710.05941, 2017.
[22]   LeCun Y., Bengio Y., and Hinton G., Deep learning, Nature, vol. 521, no. 7553, pp. 436-444, 2015.
[23]   Karlik B. and Vehbi A., Performance analysis of various activation functions in generalized MLP architectures of neural networks, Int. J. Artif. Intell. Expert Syst., vol. 1, no. 4, pp. 111-122, 2011.
[24]   Collobert R., Weston J., Bottou L., Karlen M., Kavukcuoglu K., and Kuksa P., Natural language processing (almost) from scratch, J. Mach. Learn. Res., vol. 12, pp. 2493-2537, 2011.
[25]   Turian J., Bergstra J., and Bengio Y., Quadratic features and deep architectures for chunking, in Proc. Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Boulder, CO, USA, 2009, pp. 245-248.
[26]   Nair V. and Hinton G. E., Rectified linear units improve restricted Boltzmann machines, in Proc. 27th Int. Conf. Machine Learning, Haifa, Israel, 2010, pp. 807-814.
[27]   Zeiler M. D., Ranzato M., Monga R., Mao M., Yang K., Le Q. V., Nguyen P., Senior A., Vanhoucke V., Dean J., et al., On rectified linear units for speech processing, in 2013 IEEE Int. Conf. Acoustics, Speech and Signal Processing, Vancouver, Canada, 2013, pp. 3517-3521.
[28]   Arora R., Basu A., Mianjy P., and Mukherjee A., Understanding deep neural networks with rectified linear units, arXiv preprint arXiv: 1611.01491, 2016.
[29]   Goel S., Kanade V., Klivans A., and Thaler J., Reliably learning the ReLU in polynomial time, arXiv preprint arXiv: 1611.10258, 2016.
[30]   Banerjee C., Mukherjee T., and Pasiliao E. Jr., An empirical study on generalizations of the ReLU activation function, in Proc. 2019 ACM Southeast Conf., Kennesaw, GA, USA, 2019, pp. 164-167.
[31]   Maas A. L., Hannun A. Y., and Ng A. Y., Rectifier nonlinearities improve neural network acoustic models, in Proc. 30th Int. Conf. Machine Learning, Atlanta, GA, USA, 2013.
[32]   He K. M., Zhang X. Y., Ren S. Q., and Sun J., Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification, in Proc. IEEE Int. Conf. Computer Vision, Santiago, Chile, 2015, pp. 1026-1034.
[33]   Jin X. J., Xu C. Y., Feng J. S., Wei Y. C., Xiong J. J., and Yan S. C., Deep learning with S-shaped rectified linear activation units, in Proc. 30th AAAI Conf. Artificial Intelligence, Phoenix, AZ, USA, 2016, pp. 1737-1743.
[34]   Qiu S. and Cai B. L., Flexible rectified linear units for improving convolutional neural networks, arXiv preprint arXiv: 1706.08098, 2017.
[35]   Dugas C., Bengio Y., Bélisle F., Nadeau C., and Garcia R., Incorporating second-order functional knowledge for better option pricing, in Proc. 13th Int. Conf. Neural Information Processing Systems, Denver, CO, USA, 2001, pp. 472-478.
[36]   Trottier L., Giguere P., and Chaib-draa B., Parametric exponential linear unit for deep convolutional neural networks, in Proc. 16th IEEE Int. Conf. Machine Learning and Applications, Cancun, Mexico, 2017, pp. 207-214.
[37]   Grelsson B. and Felsberg M., Improved learning in convolutional neural networks with shifted exponential linear units (ShELUs), in Proc. 24th Int. Conf. Pattern Recognition, Beijing, China, 2018, pp. 517-522.
[38]   Goodfellow I. J., Warde-Farley D., Mirza M., Courville A., and Bengio Y., Maxout networks, arXiv preprint arXiv: 1302.4389, 2013.
[39]   Lin H. W., Tegmark M., and Rolnick D., Why does deep and cheap learning work so well? J. Stat. Phys., vol. 168, no. 6, pp. 1223-1247, 2017.
[40]   Petersen P. and Voigtlaender F., Optimal approximation of piecewise smooth functions using deep ReLU neural networks, Neural Networks, vol. 108, pp. 296-330, 2018.
[41]   Yu Z., Li T. R., Yu N., Pan Y., Chen H. M., and Liu B., Reconstruction of hidden representation for robust feature extraction, ACM Trans. Intell. Syst. Technol., vol. 10, no. 2, p. 18, 2019.
[42]   Yu Z., Yu N., Pan Y., and Li T. R., A novel deep learning network architecture with cross-layer neurons, in 2016 IEEE Int. Conf. Big Data and Cloud Computing, Social Computing and Networking, Sustainable Computing and Communications, Atlanta, GA, USA, 2016, pp. 111-117.
[43]   Ballester P. and Araujo R. M., On the performance of GoogLeNet and AlexNet applied to sketches, in Proc. 30th AAAI Conf. Artificial Intelligence, Phoenix, AZ, USA, 2016, pp. 1124-1128.
[44]   Kendall A., Grimes M., and Cipolla R., PoseNet: A convolutional network for real-time 6-DOF camera relocalization, in Proc. IEEE Int. Conf. Computer Vision, Santiago, Chile, 2015, pp. 2938-2946.
[45]   Mahendran A. and Vedaldi A., Understanding deep image representations by inverting them, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Boston, MA, USA, 2015, pp. 5188-5196.
[46]   Kingma D. P. and Ba J., Adam: A method for stochastic optimization, arXiv preprint arXiv: 1412.6980, 2014.
[47]   Krizhevsky A. and Hinton G., Convolutional deep belief networks on CIFAR-10, Unpublished Manuscript, vol. 40, no. 7, pp. 1-9, 2010.
[48]   Banerjee C., Mukherjee T., Lilian C., Reasor D., Liu X. W., and Pasiliao E., A feature selection algorithm using neural networks, International Journal of Machine Learning and Computing, vol. 4, pp. 1-8, 2020.
[1] Wei Zhong, Ning Yu, Chunyu Ai. Applying Big Data Based Deep Learning System to Intrusion Detection[J]. Big Data Mining and Analytics, 2020, 3(3): 181-195.
[2] Sunitha Basodi, Chunyan Ji, Haiping Zhang, Yi Pan. Gradient Amplification: An Efficient Way to Train Deep Neural Networks[J]. Big Data Mining and Analytics, 2020, 3(3): 196-207.
[3] Zhenxing Guo, Shihua Zhang. Sparse Deep Nonnegative Matrix Factorization[J]. Big Data Mining and Analytics, 2020, 03(01): 13-28.
[4] Qile Zhu, Xiyao Ma, Xiaolin Li. Statistical Learning for Semantic Parsing: A Survey[J]. Big Data Mining and Analytics, 2019, 2(4): 217-239.
[5] Ying Yu, Min Li, Liangliang Liu, Yaohang Li, Jianxin Wang. Clinical Big Data and Deep Learning: Applications, Challenges, and Future Outlooks[J]. Big Data Mining and Analytics, 2019, 2(4): 288-305.
[6] Wenmao Wu, Zhizhou Yu, Jieyue He. A Semi-Supervised Deep Network Embedding Approach Based on the Neighborhood Structure[J]. Big Data Mining and Analytics, 2019, 2(3): 205-216.
[7] Jiangcheng Zhu, Shuang Hu, Rossella Arcucci, Chao Xu, Jihong Zhu, Yi-ke Guo. Model Error Correction in Data Assimilation by Integrating Neural Networks[J]. Big Data Mining and Analytics, 2019, 2(2): 83-91.
[8] Jin Liu, Yi Pan, Min Li, Ziyue Chen, Lu Tang, Chengqian Lu, Jianxin Wang. Applications of Deep Learning to MRI Images: A Survey[J]. Big Data Mining and Analytics, 2018, 1(1): 1-18.
[9] Ning Yu, Zhihua Li, Zeng Yu. Survey on Encoding Schemes for Genomic Data Representation and Feature Learning—From Signal Processing to Machine Learning[J]. Big Data Mining and Analytics, 2018, 01(03): 191-210.
[10] Qianyu Meng, Kun Wang, Xiaoming He, Minyi Guo. QoE-Driven Big Data Management in Pervasive Edge Computing Environment[J]. Big Data Mining and Analytics, 2018, 01(03): 222-233.