摘要
在分布式网络中,测量top-K频繁流对资源分配、安全监控等应用至关重要。现有的top-K频繁流测量工作存在不适用于测量分布式网络流量或只考虑单时间周期等局限。为此,提出了分布式网络中连续时间周期的全局top-K频繁流测量方案,在分布节点中布置了紧凑的概率数据结构来记录网络流信息,每个时间周期结束后分布节点向中心节点发送必要信息,中心节点汇聚得到从测量开始至当前时间周期的全局top-K频繁流。考虑到每条流可能出现在一个或多个测量节点,使用了不同的方法来减少传输开销。对于每条流只会出现在单一节点的情况,采用传输分段最小值的方法来获得阈值,实验结果表明这种方法减少了全量传输超过50%的传输开销。对于每条流会出现在多个节点的情况,提出了多阶段无误差处理方法和单阶段快速处理方法,分别应对不能容忍误差的场景和实际高速网络流量,相比每个时间周期都使用已有单周期方法,传输开销的实验表现降低了两个数量级。最后还提出了一种利用历史平均增值信息降低通信延迟的方法,实验结果表明该方法有效降低了限制信息的平均相对误差。
In distributed networks,the measurment of the top-K frequent flows is crucial for applications like resource allocation and security monitoring.Existing works on top-K frequent flow measurement have limitations such as being unsuitable for distributed network traffic measurement or only considering single time periods.To address these problems,this paper proposes a scheme for measuring global top-K frequent flows over continuous time periods in distributed networks.This involves deploying compact probabilistic data structures at distributed nodes to record network flow information.At the end of each time period,distributed nodes send necessary information to a central node,which aggregates this to identify the global top-K frequent flows from the start of measurement to the current time period.Considering that each flow may appear at one or multiple measurement nodes,different methods are used to reduce transmission overhead.For flows appearing at a single node,a method of transmitting segmented minimum values is used to obtain a threshold.Experiments show that this method reduces the transmission overhead by over 50%compared to full transmission.For flows appearing at multiple nodes,a multi-stage error-free processing method and a single-stage fast processing method are proposed,catering to scenarios that cannot tolerate errors and actual high-speed network traffic,respectively.Compared to using existing single-period methods in each time period,experimental performance of transmission overhead reduced by two orders of magnitude.Finally,a method using historical average increment information to reduce communication delay is also proposed,and experimental results show that it effectively reduces the average relative error of constraint information.
作者
毛晨宇
黄河
孙玉娥
杜扬
MAO Chenyu;HUANG He;SUN Yu’e;DU Yang(School of Computer Science and Technology,Soochow University,Suzhou,Jiangsu 215006,China;School of Rail Transportation,Soochow University,Suzhou,Jiangsu 215131,China)
出处
《计算机科学》
CSCD
北大核心
2024年第4期28-38,共11页
Computer Science
基金
国家自然科学基金(62332013,62072322,U20A20182,62202322)。
