TY - GEN
T1 - Implementation of massive data processing architecture for electric enterprise groups
AU - Wu, Linli
AU - Shen, Zhangyi
AU - Feng, Xiang
N1 - Publisher Copyright:
© 2016 IEEE.
PY - 2016/8/23
Y1 - 2016/8/23
N2 - In the past three decades, with the fast development of China's economy, the business data of electric power industry also showed rapid growth and the data size is continual expanding. Thus, the management systems of electric enterprise groups encountered the performance bottlenecks of massive data processing. Hadoop is an open-source software framework developed for reliable, scalable, and efficient distributed computing and storage, and is widely used in massive data processing. Based on the study of the proposed massive business data processing of electric industry, this paper introduces a novel architecture, which is based on Hadoop ecosystem, for management systems of electric enterprise groups to crack the present performance bottlenecks. First, the business logic and processing produce of management system is analyzed. Then new architecture with Hadoop based distributed processing model is introduced in detailed. At last, experimental results with actual business scenarios and data prove the architecture can effectively resolve present performance bottlenecks of massive business data processing, and greatly improve the performance and efficiency of the management system of electric enterprise groups.
AB - In the past three decades, with the fast development of China's economy, the business data of electric power industry also showed rapid growth and the data size is continual expanding. Thus, the management systems of electric enterprise groups encountered the performance bottlenecks of massive data processing. Hadoop is an open-source software framework developed for reliable, scalable, and efficient distributed computing and storage, and is widely used in massive data processing. Based on the study of the proposed massive business data processing of electric industry, this paper introduces a novel architecture, which is based on Hadoop ecosystem, for management systems of electric enterprise groups to crack the present performance bottlenecks. First, the business logic and processing produce of management system is analyzed. Then new architecture with Hadoop based distributed processing model is introduced in detailed. At last, experimental results with actual business scenarios and data prove the architecture can effectively resolve present performance bottlenecks of massive business data processing, and greatly improve the performance and efficiency of the management system of electric enterprise groups.
KW - Hadoop
KW - Hive
KW - distributed processing
KW - massive data processing
UR - https://www.scopus.com/pages/publications/84987986868
U2 - 10.1109/ICIS.2016.7550819
DO - 10.1109/ICIS.2016.7550819
M3 - 会议稿件
AN - SCOPUS:84987986868
T3 - 2016 IEEE/ACIS 15th International Conference on Computer and Information Science, ICIS 2016 - Proceedings
BT - 2016 IEEE/ACIS 15th International Conference on Computer and Information Science, ICIS 2016 - Proceedings
A2 - Uehara, Kuniaki
A2 - Nakamura, Masahide
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 15th IEEE/ACIS International Conference on Computer and Information Science, ICIS 2016
Y2 - 26 June 2016 through 29 June 2016
ER -