摘要
从分析数值预报产品收集和共享的特点出发,利用MySQL数据库和HBase分布式数据库,设计并建立一套以MySQL存储实时数据、分布式存储管理历史数据为核心的数值预报产品多源文件共享监控保障系统,实现统一集中监视和数据共享完整性保障等功能,为省级预报预测预警等业务提供强有力的数据支撑。详细分析由数据共享源、存储管理、功能应用3部分组成的整体结构,并重点阐述MySQL数据库表结构设计、HBase历史数据存储查询主键设计、数据导入工具Sqoop任务和Oozie工作流的配置使用、系统应用功能(实时监控、配置管理及历史查询统计等)的开发设计等思路和方法。
Based on analyzing the characteristics of the collection and sharing of numerical forecast products,designs and builds a monitoring support system for numerical forecast products based on MySQL's storage real-time data and distributed storage management historical data,uses MySQL's database and HBase's distributed database.The monitoring support system achieves the functions of centralized monitoring and integrity guarantee of data sharing,and provides powerful data support for provincial-level forecasting,forecasting and early warning services.Analyzes the overall structure of data sharing source,storage management,and functional applications in detail.Focuses on the design of the MySQL database table structure,HBase's historical data storage query primary key design,data import tool Sqoop tasks and Oozie's work.The configuration and use of streams,the development and design of system application functions(real-time monitoring,configuration management,history query statistics,etc.),and other ideas and methods.
作者
夏正龙
钟艳雯
朱亮
吕冠儒
周超
XIA Zheng-long;ZHONG Yan-wen;ZHU Liang;LU Guan-ru;ZHOU Chao(Hunan Meteorological Information Center,Changsha 410118)
基金
湖南省气象局气象预报业务能力建设(2016-2017年)科研专项(No.YBNL16-16)
