Convolutional Sketch Inversion

详细信息    查看全文

• 关键词：Deep neural network ; Face synthesis ; Face recognition ; Fine arts ; Forensic arts ; Sketch inversion ; Sketch recognition
• 刊名：Lecture Notes in Computer Science
• 出版年：2016
• 出版时间：2016
• 年：2016
• 卷：9913
• 期：1
• 页码：810-824
• 全文大小：4,277 KB
• 参考文献：1.Anstis, S., Vergeer, M., Lier, R.V.: Looking at two paintings at once: luminance edges can gate colors. i-Perception 3(8), 515–518 (2012). http://​dx.​doi.​org/​10.​1068/​i0537sas CrossRef
  2.Beyeler, M.: OpenCV with Python Blueprints. Packt Publishing, Birmingham (2015)
  3.Cheng, Z., Yang, Q., Sheng, B.: Deep colorization. In: International Conference on Computer Vision. Institute of Electrical and Electronics Engineers (IEEE), December 2015. http://​dx.​doi.​org/​10.​1109/​ICCV.​2015.​55
  4.Cowen, A.S., Chun, M.M., Kuhl, B.A.: Neural portraits of perception: reconstructing face images from evoked brain activity. NeuroImage 94, 12–22 (2014). http://​dx.​doi.​org/​10.​1016/​j.​neuroimage.​2014.​03.​018 CrossRef
  5.Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: European Conference on Computer Vision (2014)
  6.Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016). http://​dx.​doi.​org/​10.​1109/​TPAMI.​2015.​2439281 CrossRef
  7.Gao, X., Wang, N., Tao, D., Li, X.: Face sketch-photo synthesis and retrieval using sparse representation. IEEE Trans. Circ. Syst. Video Technol. 22(8), 1213–1226 (2012). http://​dx.​doi.​org/​10.​1109/​TCSVT.​2012.​2198090 CrossRef
  8.Gastal, E.S.L., Oliveira, M.M.: Domain transform for edge-aware image and video processing. ACM Trans. Graph. 30(4), 1 (2011). http://​dx.​doi.​org/​10.​1145/​2010324.​1964964 CrossRef
  9.Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style. CoRR abs/1508.06576 (2015)
  10.Güçlü, U., van Gerven, M.A.J.: Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. J. Neurosci. 35(27), 10005–10014 (2015). http://​dx.​doi.​org/​10.​1523/​JNEUROSCI.​5023-14.​2015 CrossRef
  11.Güçlü, U., van Gerven, M.A.J.: Increasingly complex representations of natural movies across the dorsal stream are shared between subjects. NeuroImage (2015). http://​dx.​doi.​org/​10.​1016/​j.​neuroimage.​2015.​12.​036
  12.He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR abs/1512.03385 (2015)
  13.Iizuka, S., Simo-Serra, E., Ishikawa, H.: Let there be color! Joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification. ACM Trans. Graph. 35(4), 110 (2016)CrossRef
  14.Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. CoRR abs/1502.03167 (2015)
  15.Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. CoRR abs/1603.08155 (2016)
  16.Kingma, D., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014)
  17.Learned-Miller, E., Huang, G.B., RoyChowdhury, A., Li, H., Hua, G.: Labeled faces in the wild: a survey. In: Kawulok, M., Celebi, M.E., Smolka, B. (eds.) Advances in Face Detection and Facial Image Analysis, pp. 189–248. Springer, Heidelberg (2016). http://​dx.​doi.​org/​10.​1007/​978-3-319-25958-1_​8 CrossRef
  18.Li, Y.h., Savvides, M., Bhagavatula, V.: Illumination tolerant face recognition using a novel face from sketch synthesis approach and advanced correlation filters. In: International Conference on Acoustics, Speech, and Signal Processing. Institute of Electrical and Electronics Engineers (IEEE) (2006). http://​dx.​doi.​org/​10.​1109/​ICASSP.​2006.​1660353
  19.Liu, Q., Tang, X., Jin, H., Lu, H., Ma, S.: A nonlinear approach for face sketch synthesis and recognition. In: Conference on Computer Vision and Pattern Recognition. Institute of Electrical and Electronics Engineers (IEEE) (2005). http://​dx.​doi.​org/​10.​1109/​CVPR.​2005.​39
  20.Liu, W., Tang, X., Liu, J.: Bayesian tensor inference for sketch-based facial photo hallucination. In: International Joint Conference on Artificial Intelligence (2007)
  21.Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: International Conference on Computer Vision (2015)
  22.Martinez, A.M., Benavente, R.: The AR-face database. CVC Technical report 24 (1998)
  23.Messer, K., Matas, J., Kittler, J., Jonsson, K.: XM2VTSDB: The extended M2VTS database. In: Audio and Video-based Biometric Person Authentication (1999)
  24.Simo-Serra, E., Iizuka, S., Sasaki, K., Ishikawa, H.: Learning to simplify: fully convolutional networks for rough sketch cleanup. ACM Trans. Graph. 35(4), 121 (2016)CrossRef
  25.Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014)
  26.Tang, X., Wang, X.: Face sketch synthesis and recognition. In: International Conference on Computer Vision. Institute of Electrical and Electronics Engineers (IEEE) (2003). http://​dx.​doi.​org/​10.​1109/​ICCV.​2003.​1238414
  27.Tokui, S., Oono, K., Hido, S., Clayton, J.: Chainer: a next-generation open source framework for deep learning. In: Workshop on Machine Learning Systems at Neural Information Processing Systems (2015)
  28.Vergeer, M., Anstis, S., van Lier, R.: Flexible color perception depending on the shape and positioning of achromatic contours. Front. Psychol. 6 (2015). http://​dx.​doi.​org/​10.​3389/​fpsyg.​2015.​00620
  29.Wang, N., Tao, D., Gao, X., Li, X., Li, J.: A comprehensive survey to face hallucination. Int. J. Comput. Vis. 106(1), 9–30 (2013). http://​dx.​doi.​org/​10.​1007/​s11263-013-0645-9 CrossRef
  30.Wang, N., Tao, D., Gao, X., Li, X., Li, J.: Transductive face sketch-photo synthesis. IEEE Trans. Neural Netw. Learn. Syst. 24(9), 1364–1376 (2013). http://​dx.​doi.​org/​10.​1109/​TNNLS.​2013.​2258174 CrossRef
  31.Wang, X., Tang, X.: Face photo-sketch synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(11), 1955–1967 (2009). http://​dx.​doi.​org/​10.​1109/​TPAMI.​2008.​222 CrossRef
  32.Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004). http://​dx.​doi.​org/​10.​1109/​TIP.​2003.​819861 CrossRef
  33.Xiao, B., Gao, X., Tao, D., Li, X.: A new approach for face recognition by sketches in photos. Sig. Process. 89(8), 1576–1588 (2009). http://​dx.​doi.​org/​10.​1016/​j.​sigpro.​2009.​02.​008 CrossRef MATH
  34.Zhang, L., Lin, L., Wu, X., Ding, S., Zhang, L.: End-to-end photo-sketch generation via fully convolutional representation learning. In: International Conference on Multimedia Retrieval. Association for Computing Machinery (ACM) (2015). http://​dx.​doi.​org/​10.​1145/​2671188.​2749321
  35.Zhang, S., Gao, X., Wang, N., Li, J.: Robust face sketch style synthesis. IEEE Trans. Image Process. 25(1), 220–232 (2016). http://​dx.​doi.​org/​10.​1109/​TIP.​2015.​2501755 MathSciNet CrossRef
  36.Zhang, W., Wang, X., Tang, X.: Lighting and pose robust face sketch synthesis. In: European Conference on Computer Vision (2010)
  
• 作者单位：Yağmur Güçlütürk (15)
  Umut Güçlü (15)
  Rob van Lier (15)
  Marcel A. J. van Gerven (15)
  
  15. Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
  
• 丛书名：Computer Vision – ECCV 2016 Workshops
• ISBN：978-3-319-46604-0
• 刊物类别：Computer Science
• 刊物主题：Artificial Intelligence and Robotics
  Computer Communication Networks
  Software Engineering
  Data Encryption
  Database Management
  Computation by Abstract Devices
  Algorithm Analysis and Problem Complexity
  
• 出版者：Springer Berlin / Heidelberg
• ISSN：1611-3349
• 卷排序：9913

文摘

In this paper, we use deep neural networks for inverting face sketches to synthesize photorealistic face images. We first construct a semi-simulated dataset containing a very large number of computer-generated face sketches with different styles and corresponding face images by expanding existing unconstrained face data sets. We then train models achieving state-of-the-art results on both computer-generated sketches and hand-drawn sketches by leveraging recent advances in deep learning such as batch normalization, deep residual learning, perceptual losses and stochastic optimization in combination with our new dataset. We finally demonstrate potential applications of our models in fine arts and forensic arts. In contrast to existing patch-based approaches, our deep-neural-network-based approach can be used for synthesizing photorealistic face images by inverting face sketches in the wild.