基于空间卷积神经网络模型的图像显著性检测
|
摘要
针对现有显著性检测方法鲁棒检测效果较差这一问题,提出一种新的基于空间卷积神经网络的显著性检测算法。利用去均值、归一化的预处理方法获取目标候选区。一方面通过引入卷积变换网络,建立提取显著物体上下文信息的全局模型,得到相应的目标检测信息显著图;另一方面构建特征子网络结构输出6维变换矩阵,经过空间变形模块改造输入图像,获取边缘信息。将空间变换网络输出的局部置信度融入到全局显著信息图,求取特征表达最大值,实现显著性与非显著性划分,完成显著性检测任务。实验结果表明,该算法不仅在同等条件下显著检测的AUC值得到了提高,并且生成的显著性图聚焦点突显,鲁棒检测效果得到明显改善。
To overcome the drawback of bad robustness detection of the existing saliency detection methods,this paper presents a novel saliency detection based on spatial convolutional neural network. The candidate object areas are conformed via preprocessing methods of the removing mean and normalization. By introducing the Convolutional Neural Network( CNN),a global model that captures the context information of saliency objects is constructed,and the corresponding target detection saliency map is obtained. On the other hand,the feature sub-network structure is established to output the 6-dimensional transformation matrix,which is used to transform input image through the spatial deformation module to obtain edge information of salient object. The output of the spatial transformer network local confidence coefficient is introduced into the global information saliency map to seek the maximum value of the feature expression.The discrimination of saliency and non-saliency is realized to accomplish the saliency detection task. Experimental results show that the proposed algorithm improves AUC accuracy under the same condition,generates a saliency map of focus highlighting and achieves impressive robust detection results.
To overcome the drawback of bad robustness detection of the existing saliency detection methods,this paper presents a novel saliency detection based on spatial convolutional neural network. The candidate object areas are conformed via preprocessing methods of the removing mean and normalization. By introducing the Convolutional Neural Network( CNN),a global model that captures the context information of saliency objects is constructed,and the corresponding target detection saliency map is obtained. On the other hand,the feature sub-network structure is established to output the 6-dimensional transformation matrix,which is used to transform input image through the spatial deformation module to obtain edge information of salient object. The output of the spatial transformer network local confidence coefficient is introduced into the global information saliency map to seek the maximum value of the feature expression.The discrimination of saliency and non-saliency is realized to accomplish the saliency detection task. Experimental results show that the proposed algorithm improves AUC accuracy under the same condition,generates a saliency map of focus highlighting and achieves impressive robust detection results.
引文
[1]ZHANG Xilin,LI Zhaoping,ZHOU Tiangang,et al.Neural activities in v1 create a bottom-up saliency map[J].Neuron,2012,73(1):183-192.
[2]ITTI L,KOCH C,NIEBUR E.A model of saliencybased visual attention for rapid scene analysis[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(11):1254-1259.
[3]HAREL J,KOCH C,PERONAP.Graph-based visual saliency[C]//Proceedings of the 20th Annual Conference on Neural Information Processing Systems.Cambridge,USA:MIT Press,2007:545-552.
[4]ACHANTA R,ESTRADA F,WILS P,et al.Salient region detection and segmentation[C]//Proceedings of the 6th International Conference on Computer Vision Systems.Berlin,Germany:Springer,2008:66-75.
[5]RAHTU E,KANNALA J,SALO M,et al.Segmenting salient objects from images and videos[C]//Proceedings of European Conference on Computer Vision.New York,USA:ACM Press,2010:366-379.
[6]WEN Shifeng,HAN Junwei,ZHANG Dingwen,et al.Saliency detection based on feature learning using deep Boltzmann machines[C]//Proceedings of IEEE International Conference on Multimedia and Expo.Washington D.C.,USA:IEEE Press,2014:1-6.
[7]易生,梁华刚,茹锋.基于多列深度3D卷积神经网络的手势识别[J].计算机工程,2017,43(8):243-248.
[8]王奎奎,玉振明.基于DCT零系数与局部结构张量的局部模糊检测[J].计算机工程,2017,43(6):207-211.
[9]江帆,刘辉,王彬,等.基于CNN-GRNN模型的图像识别[J].计算机工程,2017,43(4):257-262.
[10]LI Guangbin,YU Yizhou.Visual saliency detection based on multiscale deep CNN features[J].IEEE Transactions on Image Processing,2016,25(11):5012-5024.
[11]LI Hongyang,CHEN Jiang,LU Huchuan,et al.CNN for saliency detection with low-level feature integration[J].Neurocomputing,2017,226(3):212-220.
[12]QU Liangqiong,HE Shengfeng,ZHANG Jiawei,et al.RGBD salient object detection via deep fusion[J].IEEE Transactions on Image Processing,2017,26(5):2274-2285.
[13]李岳云,许悦雷,马时平,等.深度卷积神经网络的显著性检测[J].中国图象图形学报,2016,21(1):53-59.
[14]KRUTHIVENTI S S,GUDISA V,DHOLAKIYA J H,et al.Saliency unified:a deep architecture for simultaneous eye fixation prediction and salient object segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:5781-5790.
[15]He Shengfeng,LAU R W H,LIU Wenxi,et al.SuperCNN:a superpixelwise convolutional neural network for salient object detection[J].International Journal of Computer Vision,2015,115(3):330-344.
[16]JADERBERG M,SIMONYAN K,ZISSERMAN A,et al.Spatial transformer networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems.Cambridge,USA:MIT Press,2015:2017-2025.
[17]JIANG Ming,HUANG Shengsheng,DUAN Juanyong,et al.SALICON:saliency in context[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:1072-1080.
[18]XIAO Jianxiong,HAYS J,EHINGER K A,et al.SUN database:large-scale scene recognition from abbey to zoo[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2010:3485-3492.
[19]IBARZ J,SZEGEDY C,VANHOUCKE V,et al,Largescale scene understanding(LSUN)[EB/OL].[2017-04-20].http://lsun.cs.princeton.edu/leaderboard/#saliencyisun.
[20]RICHE N,MANCAS M,DUVINAGE M,et al.Rare2012:A multi-scale rarity-based saliency detection with its comparative statistical analysis[J].Signal Processing:Image Communication,2013,28(6):642-658.
[21]XIA Chen,QI Fei,SHI Guangming.Bottom-up visual saliency estimation with deep autoencoder-based sparse reconstruction[J].IEEE Transactions on Neural Networks and Learning Systems,2016,27(6):1227-1240.
[22]ZHANG Jianming,SCLAROFF S.Saliency detection:a boolean map approach[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2013:153-160.
[2]ITTI L,KOCH C,NIEBUR E.A model of saliencybased visual attention for rapid scene analysis[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(11):1254-1259.
[3]HAREL J,KOCH C,PERONAP.Graph-based visual saliency[C]//Proceedings of the 20th Annual Conference on Neural Information Processing Systems.Cambridge,USA:MIT Press,2007:545-552.
[4]ACHANTA R,ESTRADA F,WILS P,et al.Salient region detection and segmentation[C]//Proceedings of the 6th International Conference on Computer Vision Systems.Berlin,Germany:Springer,2008:66-75.
[5]RAHTU E,KANNALA J,SALO M,et al.Segmenting salient objects from images and videos[C]//Proceedings of European Conference on Computer Vision.New York,USA:ACM Press,2010:366-379.
[6]WEN Shifeng,HAN Junwei,ZHANG Dingwen,et al.Saliency detection based on feature learning using deep Boltzmann machines[C]//Proceedings of IEEE International Conference on Multimedia and Expo.Washington D.C.,USA:IEEE Press,2014:1-6.
[7]易生,梁华刚,茹锋.基于多列深度3D卷积神经网络的手势识别[J].计算机工程,2017,43(8):243-248.
[8]王奎奎,玉振明.基于DCT零系数与局部结构张量的局部模糊检测[J].计算机工程,2017,43(6):207-211.
[9]江帆,刘辉,王彬,等.基于CNN-GRNN模型的图像识别[J].计算机工程,2017,43(4):257-262.
[10]LI Guangbin,YU Yizhou.Visual saliency detection based on multiscale deep CNN features[J].IEEE Transactions on Image Processing,2016,25(11):5012-5024.
[11]LI Hongyang,CHEN Jiang,LU Huchuan,et al.CNN for saliency detection with low-level feature integration[J].Neurocomputing,2017,226(3):212-220.
[12]QU Liangqiong,HE Shengfeng,ZHANG Jiawei,et al.RGBD salient object detection via deep fusion[J].IEEE Transactions on Image Processing,2017,26(5):2274-2285.
[13]李岳云,许悦雷,马时平,等.深度卷积神经网络的显著性检测[J].中国图象图形学报,2016,21(1):53-59.
[14]KRUTHIVENTI S S,GUDISA V,DHOLAKIYA J H,et al.Saliency unified:a deep architecture for simultaneous eye fixation prediction and salient object segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:5781-5790.
[15]He Shengfeng,LAU R W H,LIU Wenxi,et al.SuperCNN:a superpixelwise convolutional neural network for salient object detection[J].International Journal of Computer Vision,2015,115(3):330-344.
[16]JADERBERG M,SIMONYAN K,ZISSERMAN A,et al.Spatial transformer networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems.Cambridge,USA:MIT Press,2015:2017-2025.
[17]JIANG Ming,HUANG Shengsheng,DUAN Juanyong,et al.SALICON:saliency in context[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:1072-1080.
[18]XIAO Jianxiong,HAYS J,EHINGER K A,et al.SUN database:large-scale scene recognition from abbey to zoo[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2010:3485-3492.
[19]IBARZ J,SZEGEDY C,VANHOUCKE V,et al,Largescale scene understanding(LSUN)[EB/OL].[2017-04-20].http://lsun.cs.princeton.edu/leaderboard/#saliencyisun.
[20]RICHE N,MANCAS M,DUVINAGE M,et al.Rare2012:A multi-scale rarity-based saliency detection with its comparative statistical analysis[J].Signal Processing:Image Communication,2013,28(6):642-658.
[21]XIA Chen,QI Fei,SHI Guangming.Bottom-up visual saliency estimation with deep autoencoder-based sparse reconstruction[J].IEEE Transactions on Neural Networks and Learning Systems,2016,27(6):1227-1240.
[22]ZHANG Jianming,SCLAROFF S.Saliency detection:a boolean map approach[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2013:153-160.