Optimal operational control for industrial processes based on Q-learning method
详细信息 查看官网全文
- 英文论文题名：Optimal operational control for industrial processes based on Q-learning method
- 论文作者：Jinna Li ; Xize Gao ; Decheng Yuan ; Jialu Fan
- 英文论文作者：Jinna Li ; Xize Gao ; Decheng Yuan ; Jialu Fan ; Lab of Operational Research and Cybernetics ; College of Information and Engineering ; Shenyang University of Chemical Technology ; State Key Lab of Synthetical Automation for Process Industries ; Northeastern University
- 年：2017
- 作者机构：Lab of Operational Research and Cybernetics,College of Information and Engineering,Shenyang University of Chemical Technology;State Key Lab of Synthetical Automation for Process Industries,Northeastern University;
- 英文论文关键词：optimal operational control ; setpoints ; approximate dynamical programming ; Q-learning
- 会议召开时间：2017-07-26
- 会议录名称：第36届中国控制会议论文集（B）
- 英文会议录名称：Proceedings of the 36th Chinese Control Conference（B）
- 语种：英文
- 分类号：TP18
- 学会代码：KZLL
- 会议名称：第36届中国控制会议
- 会议地点：中国辽宁大连
- 主办单位：中国自动化学会控制理论专业委员会
- 学会名称：中国自动化学会控制理论专业委员会
- 页数：6
- 文件大小：152k
- 原文格式：D
- 会议级别：全国
摘要
It is difficult to accurately model productive processes and describe relationship between operational indices and controlled variables for complex modern industrial processes.How to design the optimal setpoints by using only data generated by operational processes,without requiring the knowledge of model parameters of operational processes,poses a challenge on designing optimal setpoints.This paper presents a state-observer based Q-learning algorithm to learn the optimal setpoints by utilizing only data,such that the real operational indices can track the desired values in an approximately optimal manner.A simulation experiment in flotation process is implemented to show the effectiveness of the proposed method.
It is difficult to accurately model productive processes and describe relationship between operational indices and controlled variables for complex modern industrial processes.How to design the optimal setpoints by using only data generated by operational processes,without requiring the knowledge of model parameters of operational processes,poses a challenge on designing optimal setpoints.This paper presents a state-observer based Q-learning algorithm to learn the optimal setpoints by utilizing only data,such that the real operational indices can track the desired values in an approximately optimal manner.A simulation experiment in flotation process is implemented to show the effectiveness of the proposed method.
It is difficult to accurately model productive processes and describe relationship between operational indices and controlled variables for complex modern industrial processes.How to design the optimal setpoints by using only data generated by operational processes,without requiring the knowledge of model parameters of operational processes,poses a challenge on designing optimal setpoints.This paper presents a state-observer based Q-learning algorithm to learn the optimal setpoints by utilizing only data,such that the real operational indices can track the desired values in an approximately optimal manner.A simulation experiment in flotation process is implemented to show the effectiveness of the proposed method.
引文
[1]CHAI T Y,QIN S J,WAND H.Optimal operational control for complex industrial processes[J].Annual Reviews in Control,2014,38(1):81-92.
[2]DING J L,CHEN Q,CHAI T Y,et al.Data mining based feedback regulation in operation of hematite ore mineral processing plant[C],American Control Conference,St.Louis,MO,2009:907-912.
[3]CHAI T Y,DING J L,WU F.Hybrid intelligent control for optimal operation of shaft furnace roasting process[J].Control Engineering Practice,2011,19(3):264-275.
[4]ENGELL S.Feedback control for optimal process operation[J].Journal of Process Control,2007,17(3):203-219.
[5]ADETOLA V,DEHAAN D,and GUAY M.Adaptive model predictive control for constrained nonlinear systems[J].Systems and Control Letters,2009,58(5):320-326.
[6]ODLOAK D.Robust integration of RTO and MPC[J].Computer Aided Chemical Engineering,2009,27:119-126.
[7]QIN S,BADGWELL T A.An overview of nonlinear model predictive control applications[M].Nonlinear Model Predictive Control,Birkh(a|")user Basel,2000:369-392.
[8]CHAI T Y,ZHAO L,QIU J,et al.Integrated network-based model predictive control for setpoints compensation in industrial processes[J].IEEE Transactions on Industrial Informatics,2013,9(1):417-426.
[9]LIU F,GAO H,QIU J,et al.Networked multirate output feedback control for setpoints compensation and its application to rougher flotation process[J].IEEE Transactions on Industrial Electronics,2014,61(1):460-468.
[10]D(u|¨)NNEBIER G,HESSEM D V,KADAM J V,et al.Optimization and control of polymerization processes[J].Chemical Engineering and Technology,2005,28(5):575-580.
[11]TOSUKHOWONG T,LEE J H.Real-time economic optimization for an integrated plant via a dynamic optimization scheme[C].In:Proceedings of the 2004American Control Conference,Boston,Massachusetts,2004:233-238.
[12]ELLIS M,CHRISTOFIDES P D.Integrating dynamic economic optimization and model predictive control for optimal operation of nonlinear process systems[J].Control Engineering Practice,2014,22(22):242-251.
[13]VEGA P,REVOLLAR S,FRANCISCO M,MARTIN J M.Integration of set point optimization techniques into nonlinear MPC for improving the operation of WWTPs[J].Computers and Chemical Engineering,2014,68:78-95.
[14]ZHOU Ping,CHAI Tianyou.Intelligent operational feedback control for typical hematite grinding processes[J].Control Theory&Applications,2014,31(10):1352-1359.
[15]DAI W,CHAI T Y,YANG S X.Data-driven optimization control for safety operation of hematite grinding process[J].IEEE Transactions on Industrial Electronics,2015,62(5):2930-2941.
[16]WU Z,Wu Y,CHAI T Y,et al.Data-driven abnormal condition identification and self-healing control system for fused magnesium furnace[J].IEEE Transactions on Industrial Electronics,2014,62(3):1703-1715.
[17]FAN Jialu,ZHANG Yewei,CHAI Tianyou.Optimal operational feedback control for a class of industrial processes[J].Acta Automatica Sinica,2015,41(10):1754-1761.
[18]KIUMARSI-KHOMARTASH B,LEWIS F L,KARIMPOUR A,NAGHIBI-SISTANI M B,MODARES H.Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics[J].Automatica,2014,50(4):1167-1175.
[19]LEWIS F L,VRABIE D,VAMVOUDAKIS K G.Reinforcement learning and feedback control using natural decision methods to design optimal adaptive controller[J].IEEE Systems Magazine,2012,32(6),76-105.
[20]MODARES H,LEWIS F L.Linear quadratic tracking control of partially unknown continuous-time systems using reinforcement learning[J].IEEE Transactions on Automatic Control,2014,59(11):3051-3056.
[21]HAYKIN S S,WIDROW B.Least-mean square adaptive filters[M].A John Wiley&Sons,INC.:2003.
[22]AL-TAMIMI A,LEWIS,F L,ABU-KHALAF,M.Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control[J].Automatica,2007,43(3):473-481.
[23]HARRIS C C.Multiphase models of flotation machine behavior[J].International Journal of Mineral Processing,1978,5(2):107-129.
[24]MALDONADO M,SBARBARO D,LIZAMA E.Optimal control of a rougher flotation process based on dynamic programming[J].Minerals Engineering,2007,20(3):221-232.
[25]ROJAS D,and CIPRIANO A.Model based predictive control of a rougher flotation circuit considering grade estimation in intermediate cells[J].Dyna,2011,78(166):29-37.
[2]DING J L,CHEN Q,CHAI T Y,et al.Data mining based feedback regulation in operation of hematite ore mineral processing plant[C],American Control Conference,St.Louis,MO,2009:907-912.
[3]CHAI T Y,DING J L,WU F.Hybrid intelligent control for optimal operation of shaft furnace roasting process[J].Control Engineering Practice,2011,19(3):264-275.
[4]ENGELL S.Feedback control for optimal process operation[J].Journal of Process Control,2007,17(3):203-219.
[5]ADETOLA V,DEHAAN D,and GUAY M.Adaptive model predictive control for constrained nonlinear systems[J].Systems and Control Letters,2009,58(5):320-326.
[6]ODLOAK D.Robust integration of RTO and MPC[J].Computer Aided Chemical Engineering,2009,27:119-126.
[7]QIN S,BADGWELL T A.An overview of nonlinear model predictive control applications[M].Nonlinear Model Predictive Control,Birkh(a|")user Basel,2000:369-392.
[8]CHAI T Y,ZHAO L,QIU J,et al.Integrated network-based model predictive control for setpoints compensation in industrial processes[J].IEEE Transactions on Industrial Informatics,2013,9(1):417-426.
[9]LIU F,GAO H,QIU J,et al.Networked multirate output feedback control for setpoints compensation and its application to rougher flotation process[J].IEEE Transactions on Industrial Electronics,2014,61(1):460-468.
[10]D(u|¨)NNEBIER G,HESSEM D V,KADAM J V,et al.Optimization and control of polymerization processes[J].Chemical Engineering and Technology,2005,28(5):575-580.
[11]TOSUKHOWONG T,LEE J H.Real-time economic optimization for an integrated plant via a dynamic optimization scheme[C].In:Proceedings of the 2004American Control Conference,Boston,Massachusetts,2004:233-238.
[12]ELLIS M,CHRISTOFIDES P D.Integrating dynamic economic optimization and model predictive control for optimal operation of nonlinear process systems[J].Control Engineering Practice,2014,22(22):242-251.
[13]VEGA P,REVOLLAR S,FRANCISCO M,MARTIN J M.Integration of set point optimization techniques into nonlinear MPC for improving the operation of WWTPs[J].Computers and Chemical Engineering,2014,68:78-95.
[14]ZHOU Ping,CHAI Tianyou.Intelligent operational feedback control for typical hematite grinding processes[J].Control Theory&Applications,2014,31(10):1352-1359.
[15]DAI W,CHAI T Y,YANG S X.Data-driven optimization control for safety operation of hematite grinding process[J].IEEE Transactions on Industrial Electronics,2015,62(5):2930-2941.
[16]WU Z,Wu Y,CHAI T Y,et al.Data-driven abnormal condition identification and self-healing control system for fused magnesium furnace[J].IEEE Transactions on Industrial Electronics,2014,62(3):1703-1715.
[17]FAN Jialu,ZHANG Yewei,CHAI Tianyou.Optimal operational feedback control for a class of industrial processes[J].Acta Automatica Sinica,2015,41(10):1754-1761.
[18]KIUMARSI-KHOMARTASH B,LEWIS F L,KARIMPOUR A,NAGHIBI-SISTANI M B,MODARES H.Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics[J].Automatica,2014,50(4):1167-1175.
[19]LEWIS F L,VRABIE D,VAMVOUDAKIS K G.Reinforcement learning and feedback control using natural decision methods to design optimal adaptive controller[J].IEEE Systems Magazine,2012,32(6),76-105.
[20]MODARES H,LEWIS F L.Linear quadratic tracking control of partially unknown continuous-time systems using reinforcement learning[J].IEEE Transactions on Automatic Control,2014,59(11):3051-3056.
[21]HAYKIN S S,WIDROW B.Least-mean square adaptive filters[M].A John Wiley&Sons,INC.:2003.
[22]AL-TAMIMI A,LEWIS,F L,ABU-KHALAF,M.Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control[J].Automatica,2007,43(3):473-481.
[23]HARRIS C C.Multiphase models of flotation machine behavior[J].International Journal of Mineral Processing,1978,5(2):107-129.
[24]MALDONADO M,SBARBARO D,LIZAMA E.Optimal control of a rougher flotation process based on dynamic programming[J].Minerals Engineering,2007,20(3):221-232.
[25]ROJAS D,and CIPRIANO A.Model based predictive control of a rougher flotation circuit considering grade estimation in intermediate cells[J].Dyna,2011,78(166):29-37.