摘要
近年来,机器学习算法广泛应用于多个领域.超参数的选择直接影响了算法模型的性能,然而超参数优化过程往往依赖于专业知识和长期经验的积累.为了解决上述问题,本文提出了一种基于强化学习的自动超参数优化方法.该方法将超参数优化问题作为序列决策问题并建模为马尔科夫决策过程,通过使用一个强化学习智能体(agent),自动为机器学习算法选择超参数.该智能体以最大化待优化模型在验证数据集上的准确率为目标,将模型在验证数据集上的准确率作为奖赏值(reward),通过策略梯度算法训练智能体.为了减小训练过程中的方差,我们设计了数据引导池模块.实验将随机森林和XGBoost算法作为优化对象,在五个数据集上与随机搜索、贝叶斯优化、TPE、CM-AES和SMAC五种优化方法进行了对比.实验结果显示,本文所提出的方法在90%的优化任务上表现出更优的性能.同时,我们通过执行一系列消融实验验证了agent结构和数据引导池的有效性.
Recently,machine learning algorithms have been widely used in many fields.Hyperparameter directly affects the performance of the machine learning algorithms.However,hyperparameter tuning depends on the professional know ledge and the expert experience.In order to solve the above problem,we propose an automatic hyperparameter optimization method based on reinforcement learning.This method considers the hyperparameter optimization problem as a sequence decision problem and models it as a M arkov decision process(MDP).An reinforcement learning agent automatically selects hyperparameters for a machine learning algorithm.The accuracy of the model on the validation data set is used as a reward.To reduce the variance during the training,a data boot pool technique is designed.We have conducted a series of experiments to tune hyperparameters for the Random forest and XGBoost.We have compared our method with five optimization methods:random search,Bayesian optimization,TPE,CM-AES and SM AC on five datasets.The experimental results show that the proposed method achieves the best performance on 90%of the tasks..In addition,we have verified the effectiveness of the agent structure and the data boot pool by performing the ablation experiments.
作者
陈森朋
吴佳
陈修云
CHEN Sen-peng;WU Jia;CHEN Xiu-yun(School of Information and Software Engineering,University of Electronic Science and Technology of China,Chengdu 610054,China)
出处
《小型微型计算机系统》
CSCD
北大核心
2020年第4期679-684,共6页
Journal of Chinese Computer Systems
基金
国家自然科学基金项目(61503059)资助
CCF-腾讯犀牛鸟创意基金项目(CCF-Tencent TAGR20170201)资助。
关键词
机器学习
超参数优化
长短时记忆神经网络
强化学习
machine learning
hyper-parameter optimization
long short-term memory neural network
reinforcement learning
