A new closed loop method of super-resolution for multi-view images

详细信息    查看全文

• 作者：Jing Zhang (1)
  Yang Cao (1)
  Zhigang Zheng (1)
  Changwen Chen (2)
  Zengfu Wang (1)
  
• 关键词：Mixed ; resolution multi ; view images ; Super ; resolution ; Depth estimation
• 刊名：Machine Vision and Applications
• 出版年：2014
• 出版时间：October 2014
• 年：2014
• 卷：25
• 期：7
• 页码：1685-1695
• 全文大小：1,529 KB
• 参考文献：1. Allebach, J., Wong, P.: Edge-directed interpolation. In: Proceedings of IEEE International Conference on Image Processing (ICIP鈥?1996) (1996)
  2. Aly, H.A., Dubois, E.: Image up-sampling using total-variation regularization with a new observation model. IEEE Trans. Image Process. 14(10), 1647鈥?659 (2005) CrossRef
  3. Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167鈥?183 (2002) CrossRef
  4. Brust, H., Tech, G., Mller, K.: Report on generation of mixed spatial resolution stereo data base. Technical report, MOBILE3DTV project (2009)
  5. Buades, A., Coll, B., Morel, J.: A non-local algorithm for image denoising. In: Proceedings of IEEE Conference Computer Vision and Pattern Recognition (CVPR鈥?2005) (2005)
  6. Buades, A., Coll, B., Morel, J.: A review of image denoising algorithms with a new one. Multiscale Model Simul. 4, 490鈥?30 (2005) CrossRef
  7. Caselles, V., Morel, J., Sbert, C.: An axiomatic approach to image interpolation. IEEE Trans. Image Process. 7(3), 376鈥?86 (Mar. 1998)
  8. Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: Proceedings of IEEE Conference Computer Vision and Pattern Recognition (CVPR鈥?2004) (2004)
  9. Dai, S., Han, M., et al.: Soft edge smoothness prior for alpha channel super resolution. In: Proceedings of IEEE Conference Computer Vision and Pattern Recognition (CVPR鈥?2007) (2007)
  10. Gao, S., Tsang, I., Chia, L.: Laplacian sparse coding, hypergraph laplacian sparse coding, and applications. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 92鈥?04 (2013) CrossRef
  11. Garcia, D., Dorea, C., de Queiroz, R.: Super resolution for multiview images using depth information. IEEE Trans. Circ. Syst. Video Technol. 22(9), 1249鈥?256 (2012) CrossRef
  12. He, H., Siu, W.C.: Single image super-resolution using gaussian processing regression. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR鈥?2011) (2011)
  13. Hong, R., Tang, J., Tan, H., et al.: Beyond search: event-driven summarization for web videos. ACM Trans. Multimedia Comput. Commun. Appl. (TOMCCAP) 7(4), 35 (2011)
  14. Hong, R., Wang, M., Li, G., Nie, L., Zha, Z., Chua, T.: Multimedia question answering. IEEE Trans. Multimedia 19(4), 72鈥?8 (2012) CrossRef
  15. Kaick, O., Zhang, H., Hamarneh, G.: Bilateral maps for partial matching. In: Computer Graphics Forum (2013)
  16. Li, X., Orchard, M.: New edge-directed interpolation. In: Proceedings of IEEE International Conference on Image Processing (ICIP鈥?2000) (2000)
  17. Wang, M., Hong, R., Li, G., Zha, Z., Yan, S., Chua, T.: Event driven web video summarization by tag localization and key-shot identification. IEEE Trans. Multimedia 14(4), 975鈥?85 (2012) CrossRef
  18. Wang, M., Hua, X., Tang, J., Hong, R.: Beyond distance measurement: constructing neighborhood similarity for video annotation. IEEE Trans. Multimedia 11(3), 465鈥?76 (2009) CrossRef
  19. Protter, M., Elad, M., Takeda, H., Milanfar, P.: Generalizing the nonlocal-means to super-resolution reconstruction. IEEE Trans. Image Process. 18(1), 36鈥?1 (Jan. 2009)
  20. Scharstein, D., Pal, C.: Learning conditional random fields for stereo. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR鈥?2007) (2007)
  21. Shan, Q., Li, Z., Jia, J., Tang, C.K.: Fast image/video upsampling. In: ACM Transactions on Grapihcs (SIGGRAPH ASIA鈥?2008) (2008)
  22. Sun, J., Xu, Z.,, Shum, H.Y.: Image super-resolution using gradient profile prior. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR鈥?2008) (2008)
  23. Szeliski, R.: Computer Vision: Algorithms and Applications. Springer, Berlin (2011)
  24. Tai, Y.W., Liu, S.,, Brown, M., Lin, S.: Super resolution using edge prior and single image detail synthesis. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR鈥?2010) (2010)
  25. Wang, C., Yan, S., Zhang, L., Zhang, H.: Multi-label sparse coding for automatic image annotation. In: Proceedings of IEEE Conference Computer Vision and Pattern Recognition (CVPR鈥?2009) (2009)
  26. Wang, M., Hong, R., Li, G., Zha, Z., Yan, S., Chua, T.: Event driven web video summarization by tag localization and key-shot identification. IEEE Trans. Multimedia 14(4), 975鈥?85 (2012) CrossRef
  27. Wang, Q., Tang, X., Shum, H.Y.: Patch based blind image super resolution. In: Proceedings of International Conference on Computer Vision (ICCV鈥?2005) (2005)
  28. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600鈥?12 (2004) CrossRef
  29. Wang, Z.F., Zheng, Z.G.: A region based stereo matching algorithm using cooperative optimization. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR鈥?2008) (2008)
  30. Xia, T., Tao, D., Mei, T., Zhang, Y.: Multiview spectral embedding. IEEE Trans. Syst. Man Cybern. Part B Cybern. 40(6), 1438鈥?446 (2010) CrossRef
  31. Yang, J., Wright, J., Huang, T., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861鈥?873 (2010) CrossRef
  32. Yu, J., Liu, D., Tao, D., Seah, H.: Complex object correspondence construction in two-dimensional animation. IEEE Trans. Image Process. 20(11), 3257鈥?269 (2011) CrossRef
  33. Yu, J., Wang, M., Tao, D.: Semisupervised multiview distance metric learning for cartoon synthesis. IEEE Trans. Image Process. 21(11), 4636鈥?648 (2012) CrossRef
  
• 作者单位：Jing Zhang (1)
  Yang Cao (1)
  Zhigang Zheng (1)
  Changwen Chen (2)
  Zengfu Wang (1)
  
  1. Department of Automation, University of Science and Technology of China, Hefei, Anhui, People鈥檚 Republic of China
  2. Department of Computer Science and Engineering, University at Buffalo, State University of New York, Buffalo, NY, 14260-2000, USA
  
• ISSN：1432-1769

文摘

In this paper, we propose a closed loop method to resolve the multi-view super-resolution problem. For the mixed-resolution multi-view case, where the input is one high-resolution view along with its neighboring low-resolution views, our method can give the super-resolution results and obtain a high-quality depth map simultaneously. The closed loop method consists of two parts: part I, stereo matching and depth maps fusion; and part II, super-resolution. Under the guidance of the estimated depth information, the super-resolution problem can be formulated as an optimization problem. It can be solved approximately by a three-step method, which involves disparity-based pixel mapping, nonlocal construction and final fusion. Based on the super-resolution results, we can update the disparity maps and fuse them into a more reliable depth map. We repeat the loop several times until obtaining stable super-resolution results and depth maps simultaneously. The experimental results on public dataset show that the proposed method can achieve high-quality performance at different scale factors.