摘要
针对数据源新产生数据记录的增量爬取问题,提出了一种deep Web新数据发现策略,该策略采用一种新的属性值序列图模型表示deep Web数据源,将新数据发现问题转化为属性值序列图的遍历问题,该模型仅与数据相关,与现有查询关联图模型相比,具有更强的适应性和确定性,可适用于仅仅包含简单查询接口的deep Web数据源。在此模型的基础上,发现增长节点并预测其新数据发现能力;利用互信息计算节点之间的依赖关系,查询选择时尽可能地降低查询依赖带来的负面影响。该策略提高了新数据爬取的效率,实验结果表明,在相同资源约束前提下,该策略能使本地数据和远程数据保持最大化同步。
A novel deep Web data discovery strategy was proposed for new generated data record in resources. In the approach, a new graph model of deep Web data attribute value lists was used to indicate the deep Web data source, an new data crawling task was transformed into a graph traversal process. This model was only related to the data, compared with the existing query-related graph model had better adaptability and certainty, applicable to contain only a simple query interface of deep Web data sources. Based on this model, which could discovery incremental nodes and predict new data mutual information was used to compute the dependencies between nodes. When the query selects, as much as possible to reduce the negative impact brought by the query-dependent. This strategy improves the data crawling efficiency. Experimental results show that this strategy could maximize the synchronization between local and remote data under the same restriction.
出处
《通信学报》
EI
CSCD
北大核心
2016年第3期20-32,共13页
Journal on Communications
基金
国家自然科学基金资助项目(No.61440053
No.61472268
No.41201338)
江苏省自然科学基金资助项目(No.BK2012164)
苏州市科技计划基金资助项目(No.SYG201342
No.SYG201343
No.SS201344)~~
