Electricity Production Data Processing and Management Based on Hadoop and Spark

Abstract

The modern power system would continue rapidly create a huge quantity of complex user data. Relational database will not be able to cope with its future development. By combined Spark with Hadoop, proposal a new distributed data management and processing system. Design and optimize the model with power and load calculation, line loss calculation and other typical business based on Spark. Test results show that the system and model would maintain the reliability of the data storage, has higher efficiency than the traditional Hadoop model.