摘要
随着电网业务量的不断扩大,业务数据正向着更加海量与多元化的方向发展。使用大数据技术存储、处理与分析业务数据已经成为电力行业发展的必由之路。数据仓库是大数据分析的重要组成部分,如何将业务数据高效地接入数据仓库是现今实际生产环境中的一大难点。经过对当前电网业务数据的种类与特点的分析,结合实际应用场景需求研究并提出一种复合式的数据接入方案。结合数据同步复制、数据抽取转换装载等技术,在数据同步复制过程中创新性地提出将所有删改操作转换为增操作进行记录的方式,采用多级架构实现海量结构化业务数据接入数据仓库,具有较高的可行性和应用价值。
With the flourish of the business in the power grid, business data is becoming huge and diversified. Using big data technology to store, process and analyze business data has become the only way for the development of the power industry. Data warehouse is an important part of big data analyzing. How to efficiently access the business data to data warehouse is a major difficulty in today’s actual production environment. After analyzing the types and characteristics of the current power grid business data, combined with the actual application scenarios, a compound data access scheme is studied and proposed. Combined with data synchronization replication, data extraction and conversion loading and so on, in the process of data synchronization replication, the method of converting all deleted and modified operations into an additional operation is innovatively proposed. This method uses multi-level architecture to achieve massive structured business data access to data warehouse, and has high feasibility and application value.
作者
李子乾
王乐之
张云志
张旭坤
Li Ziqian1,Wang Lezhi2,Zhang Yunzhi2,Zhang Xukun2(1.State Grid Customer Serveice Center, State Grid Corporation of China, Tianjin 300000,China;2.China Real-time Database Co. , Ltd. , Nari Group Corporation/State Grid Electric Power Research Institute, Nanjing 210000, Jiangsu, Chin)
出处
《计算机应用与软件》
北大核心
2018年第8期180-185,共6页
Computer Applications and Software
关键词
电力行业
数据仓库
数据同步复制
大数据技术
结构化数据
Electric power industry
Data warehouse
Data synchronization replication
Big data technology
Structuring data
