摘要
勘探地震数据的快速增长,带来了数据处理周期的超线性增长、硬件设备成本居高不下、资源利用率下降等问题,针对这些问题提出了一个系统化的解决方案。硬件基础设施方面,通过量化分析地球物理的计算特征,设计了一套具备良好经济性与可行性的大规模集群体系结构;软件平台方面,依托HDFS分布式存储技术,通过与集中存储的联合调度,有效分流临时数据的I/O负载,同时对地震数据进行缓存,大幅提高系统的I/O带宽;开发框架方面,利用MapReduce思想实现了地震数据的快速访问和自动化并行处理。整个解决方案不仅能够满足海量地震数据高效存储、访问及处理需要,而且还具有良好的经济性、可扩展性与前瞻性,对大型物探软件的研发具有很好的参考价值。
The rapid increasing of seismic exploration data has brought problems such as super-linear growth of data processing time,high cost of hardware devices,and decreasing of resource utilization.This paper provides a systematic solution to these problems.In the aspect of hardware facilities,a cost-effective and practical architecture for large-scale cluster is designed on the basis of quantitative analysis results of the computing features of geophysics programs.In the aspect of software platform,the I/O throughput is improved significantly by taking advantage of the distributed storage based on HDFS,which is scheduled together with the central storage to share the temporary data workload and cache seismic data.In the aspect of development framework,the MapReduce method is applied to realize fast seismic data access and automatic parallel processing.While meeting the demand of high-efficient storage,access,and processing of massive seismic data,the entire solution is cost-effective,scalable,forward-looking,and can be a valuable reference for the development of large scale geophysical exploration software.
出处
《石油工业计算机应用》
2016年第3期12-19,3,共8页
Computer Applications Of Petroleum
