Please wait a minute...
Big Data Mining and Analytics  2018, Vol. 01 Issue (03): 222-233    DOI: 10.26599/BDMA.2018.9020020
QoE-Driven Big Data Management in Pervasive Edge Computing Environment
Qianyu Meng, Kun Wang*, Xiaoming He, Minyi Guo
Qianyu Meng and Xiaoming He are with the Jiangsu Engineering Research Center of Communication and Network Technology, Nanjing University of Posts and Telecommunications, Nanjing 210003, China. E-mail:;
Kun Wang is with the Jiangsu High Technology Research Key Laboratory for Wireless Sensor Networks, Nanjing University of Posts and Telecommunications, Nanjing 210003, and the Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, China.
Minyi Guo is with the Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, China. E-mail:
Download: PDF (2260 KB)      HTML  
Export: BibTeX | EndNote (RIS)      


In the age of big data, services in the pervasive edge environment are expected to offer end-users better Quality-of-Experience (QoE) than that in a normal edge environment. However, the combined impact of the storage, delivery, and sensors used in various types of edge devices in this environment is producing volumes of high-dimensional big data that are increasingly pervasive and redundant. Therefore, enhancing the QoE has become a major challenge in high-dimensional big data in the pervasive edge computing environment. In this paper, to achieve high QoE, we propose a QoE model for evaluating the qualities of services in the pervasive edge computing environment. The QoE is related to the accuracy of high-dimensional big data and the transmission rate of this accurate data. To realize high accuracy of high-dimensional big data and the transmission of accurate data through out the pervasive edge computing environment, in this study we focused on the following two aspects. First, we formulate the issue as a high-dimensional big data management problem and test different transmission rates to acquire the best QoE. Then, with respect to accuracy, we propose a Tensor-Fast Convolutional Neural Network (TF-CNN) algorithm based on deep learning, which is suitable for high-dimensional big data analysis in the pervasive edge computing environment. Our simulation results reveal that our proposed algorithm can achieve high QoE performance.

Key wordsQuality-of-Experience (QoE)      high-dimensional big data management      deep learning      pervasive edge computing     
Received: 01 February 2018      Published: 13 January 2020
Corresponding Authors: Kun Wang   
Cite this article:

Qianyu Meng, Kun Wang, Xiaoming He, Minyi Guo. QoE-Driven Big Data Management in Pervasive Edge Computing Environment. Big Data Mining and Analytics, 2018, 01(03): 222-233.

URL:     OR

piAccuracy of high-dimensional big data in each time slot i, i𝐍+
riTransmission rate of accurate data in each time slot i, ri𝐐, i𝐍+
QSet of k division point values of ri𝐐, i𝐍+
rminiMinimum transmission rate in each time slot i, i𝐍+
rmaxiMaximum transmission rate in each time slot i, i𝐍+
ξQoE weighting parameter between the accuracy of high-dimensional big data and transmission rate of accurate data
ϕQoE-maximization as our formulation
Symbol of equivalence indicating that the value of QoE is equivalent to the value of pi, i𝐍+
ade(.)Choice function of ri referring to the fact that the value of ri can meet requirements of each end-users, i𝐍+
UTF-CNNLoss function related to TF-CNN
(x,y)Tensor object
sθDimensions of TF-CNN
?UTF-CNN?(θ)?k(t)Partial derivative of TF-CNN concerning θ
αLearning rate
k=D,B,β,bD is the (M+1)-order weight tensor, B is the M-order tensor, b is the bias tensor, β is the weight
QF?Mqf?m×l matrix comprising the first l left-singular values of X
ll×l diagonal matrix including the top l singular values of X
QI?Mqi?m×l matrix comprising the first l right singular matrix of X
zjElement of the tensor X
wi?1??i?m?f?1??f?m(t)Weight difference between the unit f?1??f?m of layer t and the unit i?1??i?m of layer t+1
kf?1??f?m(t)Weight of kernel L
up?(εi?1??i?m(t+1))Upsampling operation that uses factor s?ci to tile the input element of each dimension
ZUnsorted sum tree
?UTF-CNN?(θ,x,y)?k(t) (k=D,B,β,b)
Table 1 Important notations.
Fig. 1 Big data in the pervasive edge computing environment.
𝒓𝐦𝐢𝐧 and 𝒓𝐦𝐚𝐱 components belong to every set Q, Q includes the i division point.">
Fig. 2 The <inline-formula><math xmlns:mml="" display="inline" id="MA141"><mml:msub><mml:mtext>𝒓</mml:mtext><mml:mtext>𝐦𝐢𝐧</mml:mtext></mml:msub></math></inline-formula> and <inline-formula><math xmlns:mml="" display="inline" id="MA142"><mml:msub><mml:mtext>𝒓</mml:mtext><mml:mtext>𝐦𝐚𝐱</mml:mtext></mml:msub></math></inline-formula> components belong to every set <i>Q</i>, <i>Q</i> includes the <i>i</i> division point.
Hidden layer 1256 neurons
Hidden layer 264 neurons
Hidden layer 332 neurons
Hidden layer 48 neurons
Discount parameter0.8
Learning rate0.2
Table 2 Simulation parameters.
Fig. 3 Performance with different algorithms.
Fig. 4 Training time with different algorithms.
Fig. 5 Accuracy for different algorithms.
Fig. 6 QoE values for different algorithms.
Fig. 7 QoE performance for various transmission rates via <i>h</i>, for different <inline-formula><math xmlns:mml="" display="inline" id="MA229"><mml:mi>𝝃</mml:mi></math></inline-formula>.
Fig. 8 Performance of different QoE models.
Fig. 9 QoE performance under different algorithms v.s. varying transmission rate, under different <inline-formula><math xmlns:mml="" display="inline" id="MA230"><mml:mi>𝝃</mml:mi></math></inline-formula>.
[1]   Wang K., Wang Y., Sun Y., Guo S., and Wu J., Green industrial Internet of Things architecture: An energy-efficient perspective, IEEE Commun. Mag., vol. 54, no. 12, pp. 48-54, 2016.
[2]   Guo M., Olule E., Wang G., and Guo S., Designing energy efficient target tracking protocol with quality monitoring in wireless sensor networks, Journal of Supercomputing, vol. 51, no. 2, pp. 131-148, 2010.
[3]   Meng Q., Wang K., Liu B., Miyazaki T., and He X., QoE-based big data analysis with deep learning in pervasive edge environment, in Proc. Int. IEEE Communications Conf., Kansas City, MO, USA, 2018.
[4]   Song X., Huang Y., Zhou Q., Ye F., Yang Y., and Li X., Pervasive edge data sharing in MANET, in Proc. Int. IEEE Computer Communications Workshops Conf., Atlanta, GA, USA, 2017, pp. 133-138.
[5]   Xu C., Ren J., Zhang Y., Qin Z., and Ren K., DPPro: Differentially private high-dimensional data release via random projection, IEEE Trans. Information Forensics and Security, vol. 12, no. 12, pp. 3081-3093, 2017.
[6]   Wang B. and Mueller K., The subspace voyager: Exploring high-dimensional data along a continuum of salient 3D subspaces, IEEE Trans. Visualization and Computer Graphics, vol. 24, no. 12, pp. 1204-1222, 2018.
[7]   Chen Y., Wu K., and Zhang Q., From QoS to QoE: A tutorial on video quality assessment, IEEE Commun. Surveys & Tutorials, vol. 17, no. 2, pp. 1126-1165, 2014.
[8]   Zhou X., Wang K., Jia W., and Guo M., Reinforcement learning-based adaptive resource management of differentiated services in geo-distributed data centers, in Proc. Int. IEEE/ACM Symp. Quality of Service Conf., Vilanova, Spain, 2017, pp. 1-6.
[9]   Ye Z., Mistry S., Bouguettaya A., and Dong H., Long-term QoS-aware cloud service composition using multivariate time series analysis, IEEE Trans. Services Computing, vol. 9, no. 3, pp. 382-393, 2016.
[10]   Bulo S., Biggio B., Pillai I., Pelillo M., and Roli F., Randomized prediction games for adversarial machine learning, IEEE Trans. Neural Networks and Learning Systems, vol. 28, no. 11, pp. 2466-2478, 2017.
[11]   Li M., Wei J., Zheng X., and Bolton M., A formal machine-learning approach to generating human-machine interfaces from task models, IEEE Trans. Human-Machine Systems, vol. 47, no. 6, pp. 822-833, 2017.
[12]   Wu H. and Prasad S., Semi-supervised deep learning using pseudo labels for hyperspectral image classification, IEEE Trans. Image Processing, vol. 27, no. 3, pp. 1259-1270, 2017.
[13]   Fadlullah Z., Tang F., Mao B., Kato N., Akashi O., Inoue T., and Mizutani K., State-of-the-art deep learning: Evolving machine intelligence toward tomorrow’s intelligent network traffic control systems, IEEE Commun. Surveys & Tutorials, vol. 19, no. 4, pp. 2432-2455, 2017.
[14]   Federico M., Julian P., and Mandolesi P., SCDVP: A simplicial CNN digital visual processor, IEEE Trans. Circuits and Systems I: Regular Papers, vol. 61, no. 7, pp. 1962-1969, 2014.
[15]   Hsu C. and Lin C., CNN-based joint clustering and representation learning with feature drift compensation for large-scale image data, IEEE Trans. Multimedia, vol. 20, no. 2, pp. 421-429, 2017.
[16]   Lee H., Hong K., Kang H., and Lee S., Photo aesthetics analysis via DCNN feature encoding, IEEE Trans. Multimedia, vol. 19, no. 8, pp. 1921-1932, 2017.
[17]   Li P., Chen Z., Yang L., Zhang Q., and Deen M., Deep convolutional computation model for feature learning on big data in internet of things, IEEE Trans. Industrial Informatics, DOI: 10.1109/TII.2017.2739340.
[18]   Girshick R., Fast R-CNN, in Proc. Int. IEEE Computer Vision Conf., Santiago, Chile, 2015, pp. 1440-1448.
[19]   Chen Q., Guo M., Deng Q., Zheng L., Guo S., and Shen Y., HAT: History-based auto-tuning MapReduce in heterogeneous environments, Journal of Supercomputing, vol. 64, no. 3, pp. 1038-1054, 2011.
[20]   Zhao T., Liu Q., and Chen C., QoE in video transmission: A user experience-driven strategy, IEEE Commun. Surveys & Tutorials, vol. 19, no. 1, pp. 285-302, 2016.
[21]   Kim S., Suk G., Lee J., and Chae C., QoE-aware scalable video transmission in MIMO systems, IEEE Commun. Mag., vol. 55, no. 8, pp. 196-203, 2017.
[22]   Liang C., He Y., Yu F., and Zhao N., Enhancing QoE-aware wireless edge caching with software-defined wireless networks, IEEE Trans. Wireless Communications, vol. 16, no. 10, pp. 6912-6925, 2017.
[23]   Wang K., Gu L., Guo S., Chen H., Leung V., and Sun Y., Crowdsourcing-based content-centric network: A social perspective, IEEE Network, vol. 31, no. 5, pp. 28-34, 2017.
[24]   Ji C., Dong T., Li Y., Shen Y., Li K., Qiu W., Qu W., and Guo M., Inverted grid-based KNN query processing with MapReduce, in Proc. Int. IEEE ChinaGrid Annual Conf., Beijing, China, 2012, pp. 25-32.
[25]   Zhao S. and Medhi D., Application-aware network design for Hadoop MapReduce optimization using software-defined networking, IEEE Trans. Network and Service Management, vol. 14, no. 4, pp. 804-816, 2017.
[26]   Shi W., Gong Y., Tao X., Wang J., and Zheng N., Improving CNN performance accuracies with min-max objective, IEEE Trans. Neural Networks and Learning Systems, DOI: 10.1109/TNNLS.2017.2705682.
[27]   Zhang Z., Weng T., and Daniel L., Big-data tensor recovery for high-dimensional uncertainty quantification of process variations, IEEE Trans. Components, Packaging and Manufacturing, vol. 7, no. 5, pp. 687-697, 2017.
[28]   Li M. and Wang X., Delay and rate satisfaction for data transmission with application in wireless communications, IEEE Network, vol. 29, no. 5, pp. 70-75, 2015.
[29]   Borujeny R., Noori M., and Ardakani M., Maximizing data rate for multiway relay channels with pairwise transmission strategy, IEEE Trans. Wireless Communications, vol. 16, no. 3, pp. 1609-1618, 2017.
[30]   Wang K., Qi X., Shu L., Deng D., and Rodrigues J., Toward trustworthy crowdsourcing in social internet of things, IEEE Wireless Communications, vol. 30, no. 5, pp. 30-36, 2016.
[31]   Wu D., He B., Tang X., Xu J., and Guo M., RAMZzz: Rank-aware DRAM power management with dynamic migrations and demotions, in Proc. Int. IEEE High Performance Computing, Networking, Storage and Analysis Conf., Salt Lake City, UT, USA, 2012, pp. 1-11.
[32]   Xu C., Wang K., and Guo M., Intelligent resource management in blockchain based cloud data centers, IEEE Cloud Computing, vol. 4, no. 6, pp. 50-59, 2017.
[33]   Chen Q., Chen Y., Huang Z., and Guo M., WATS: Workload-aware task scheduling in asymmetric multi-core architectures, in Proc. Int. 26th IEEE Parallel and Distributed Processing Symposium Conf., Shanghai, China, 2012, pp. 249-260.
[34]   Mei Z., Zhang Y., Zhao X., Jung B., Sarkar T., and Palma M., Choice of the scaling factor in a marching-on-in-degree time domain technique based on the associated laguerre functions, IEEE Trans. Antennas and Propagation, vol. 60, no. 9, pp. 4463-4467, 2012.
[35]   Huang H., Li P., Guo S., Liang W., and Wang K., Near-optimal deployment of service chains by exploiting correlations between network functions, IEEE Trans. Cloud Computing, DOI:10.1109/TCC.2017.2780165.
[36]   Silva G., Vieira R., and Rech C., Discrete-time sliding- mode observer for capacitor voltage control in modular multilevel converters, IEEE Trans. Industrial Electronics, vol. 65, no. 1, pp. 876-886, 2018.
[37]   Girshick R., Photo aesthetics analysis via DCNN feature encoding, in Proc. Int. IEEE Computer Vision Conf., Santiago, Chile, 2015, pp. 1921-1932.
[38]   Xu Y., Chen J., and Wang Q., The sum rate of vector gaussian multiple description coding with tree-structured covariance distortion constraints, IEEE Trans. Information Theory, vol. 63, no. 10, pp. 6747-6560, 2017.
[39]   Wang K., Mi J., Xu C., Zhu Q., Shu L., and Deng D., Real-time load reduction in multimedia big data for mobile Internet, ACM Trans. Multimedia Computing, Communications and Applications, vol. 12, no. 5s, p. 76, 2016.
[40]   Wang K., Gao H., Xu X., Jiang J., and Yue D., An energy-efficient reliable data transmission scheme for complex environmental monitoring in underwater acoustic sensor networks, IEEE Sensors Journal, vol. 16, no. 11, pp. 4051-4062, 2016.
[41]   Wang K., Shao Y., Shu L., Zhang Y., and Zhu C., Mobile big data fault-tolerant processing for eHealth networks, IEEE Network, vol. 30, no. 1, pp. 1-7, 2016.
[42]   Wang Z., Yang J., Melhem R., Childers B., Zhang Y., and Guo M., Simultaneous multikernel GPU: Multi-tasking throughput processors via fine-grained sharing, in Proc. Int. IEEE Symp. High Performance Computer Architecture Conf., Barcelona, Spain, 2016, pp. 358-369.
[43]   He X., Wang K., Miyazaki T., Huang H., Wang Y., and Guo S., Green resource allocation based on deep reinforcement learning in content-centric IoT, IEEE Trans. Emerging Topics in Computing, DOI:10.1109/TETC.2018.2805718.
[44]   Li Z., Shen Y., Yao B., and Guo M., OFScheduler: A dynamic network optimizer for MapReduce in heterogeneous cluster, International Journal of Parallel Programming, vol. 43, no. 3, pp. 472-488, 2013.
[45]   He X., Wang K., Huang H., and Liu B., QoE-driven big data architecture for smart city, IEEE Commun. Mag., vol. 56, no. 2, pp. 2-8, 2018.
[46]   Rahman W., Yun D., and Chung K., A client side buffer management algorithm to improve QoE, IEEE Trans. Consumer Electronics, vol. 62, no. 4, pp. 371-379, 2016.
[1] Zhenxing Guo, Shihua Zhang. Sparse Deep Nonnegative Matrix Factorization[J]. Big Data Mining and Analytics, 2020, 03(01): 13-28.
[2] Ying Yu, Min Li, Liangliang Liu, Yaohang Li, Jianxin Wang. Clinical Big Data and Deep Learning: Applications, Challenges, and Future Outlooks[J]. Big Data Mining and Analytics, 2019, 2(4): 288-305.
[3] Qile Zhu, Xiyao Ma, Xiaolin Li. Statistical Learning for Semantic Parsing: A Survey[J]. Big Data Mining and Analytics, 2019, 2(4): 217-239.
[4] Wenmao Wu, Zhizhou Yu, Jieyue He. A Semi-Supervised Deep Network Embedding Approach Based on the Neighborhood Structure[J]. Big Data Mining and Analytics, 2019, 2(3): 205-216.
[5] Jiangcheng Zhu, Shuang Hu, Rossella Arcucci, Chao Xu, Jihong Zhu, Yi-ke Guo. Model Error Correction in Data Assimilation by Integrating Neural Networks[J]. Big Data Mining and Analytics, 2019, 2(2): 83-91.
[6] Jin Liu, Yi Pan, Min Li, Ziyue Chen, Lu Tang, Chengqian Lu, Jianxin Wang. Applications of Deep Learning to MRI Images: A Survey[J]. Big Data Mining and Analytics, 2018, 1(1): 1-18.
[7] Ning Yu, Zhihua Li, Zeng Yu. Survey on Encoding Schemes for Genomic Data Representation and Feature Learning—From Signal Processing to Machine Learning[J]. Big Data Mining and Analytics, 2018, 01(03): 191-210.