SpringerOpen Newsletter

Receive periodic news and updates relating to SpringerOpen.

This article is part of the series Super-Resolution Enhancement of Digital Video.

Open Access Research Article

A Total Variation Regularization Based Super-Resolution Reconstruction Algorithm for Digital Video

Michael K Ng1*, Huanfeng Shen12, Edmund Y Lam3 and Liangpei Zhang2

Author Affiliations

1 Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong

2 The State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, Hubei, China

3 Department of Electrical and Electronic Engineering, The University of Hong Kong, Pokfulam Road, Hong Kong

For all author emails, please log on.

EURASIP Journal on Advances in Signal Processing 2007, 2007:074585  doi:10.1155/2007/74585

The electronic version of this article is the complete one and can be found online at: http://asp.eurasipjournals.com/content/2007/1/074585


Received:13 September 2006
Revisions received:12 March 2007
Accepted:21 April 2007
Published:27 June 2007

© 2007 Ng et al.

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Super-resolution (SR) reconstruction technique is capable of producing a high-resolution image from a sequence of low-resolution images. In this paper, we study an efficient SR algorithm for digital video. To effectively deal with the intractable problems in SR video reconstruction, such as inevitable motion estimation errors, noise, blurring, missing regions, and compression artifacts, the total variation (TV) regularization is employed in the reconstruction model. We use the fixed-point iteration method and preconditioning techniques to efficiently solve the associated nonlinear Euler-Lagrange equations of the corresponding variational problem in SR. The proposed algorithm has been tested in several cases of motion and degradation. It is also compared with the Laplacian regularization-based SR algorithm and other TV-based SR algorithms. Experimental results are presented to illustrate the effectiveness of the proposed algorithm.

References

  1. RY Tsai, TS Huang, Multi-frame image restoration and registration. Advances in Computer Vision and Image Processing 1(2), 317–339 (1984)

  2. SP Kim, NK Bose, HM Valenzuela, Recursive reconstruction of high resolution image from noisy undersampled multiframes. IEEE Transactions on Acoustics, Speech, and Signal Processing 38(6), 1013–1027 (1990). Publisher Full Text OpenURL

  3. SP Kim, W-Y Su, Recursive high-resolution reconstruction of blurred multiframe images. IEEE Transactions on Image Processing 2(4), 534–539 (1993). PubMed Abstract | Publisher Full Text OpenURL

  4. S Rhee, MG Kang, Discrete cosine transform based regularized high-resolution image reconstruction algorithm. Optical Engineering 38(8), 1348–1356 (1999). Publisher Full Text OpenURL

  5. RH Chan, TF Chan, L Shen, Z Shen, Wavelet algorithms for high-resolution image reconstruction. SIAM Journal of Scientific Computing 24(4), 1408–1432 (2003). Publisher Full Text OpenURL

  6. MK Ng, CK Sze, SP Yung, Wavelet algorithms for deblurring models. International Journal of Imaging Systems and Technology 14(3), 113–121 (2004). Publisher Full Text OpenURL

  7. N Nguyen, P Milanfar, A wavelet-based interpolation-restoration method for superresolution (wavelet superresolution). Circuits, Systems, and Signal Processing 19(4), 321–338 (2000). Publisher Full Text OpenURL

  8. H Ur, D Gross, Improved resolution from subpixel shifted pictures. CVGIP: Graphical Models and Image Processing 54(2), 181–186 (1992). Publisher Full Text OpenURL

  9. M Irani, S Peleg, Improving resolution by image registration. CVGIP: Graphical Models and Image Processing 53(3), 231–239 (1991). Publisher Full Text OpenURL

  10. H Stark, P Oskoui, High-resolution image recovery from image-plane arrays, using convex projections. Journal of the Optical Society of America A: Optics and Image Science, and Vision 6(11), 1715–1726 (1989). Publisher Full Text OpenURL

  11. AM Tekalp, MK Ozkan, MI Sezan, High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '92), March 1992, San Francisco, Calif, USA 3, 169–172

  12. AJ Patti, MI Sezan, AM Tekalp, High-resolution image reconstruction from a low-resolution image sequence in the presence of time-varying motion blur. Proceedings of IEEE International Conference Image Processing (ICIP '94), November 1994, Austin, Tex, USA 1, 343–347

  13. AJ Patti, MI Sezan, AM Tekalp, Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time. IEEE Transactions on Image Processing 6(8), 1064–1076 (1997). PubMed Abstract | Publisher Full Text OpenURL

  14. BC Tom, AK Katsaggelos, Reconstruction of a high-resolution image from multiple-degraded misregistered low-resolution images. Visual Communications and Image Processing, September 1994, Chicago, Ill, USA, Proceedings of SPIE 2308, 971–981

  15. RR Schultz, RL Stevenson, Extraction of high-resolution frames from video sequences. IEEE Transactions on Image Processing 5(6), 996–1011 (1996). PubMed Abstract | Publisher Full Text OpenURL

  16. RC Hardie, TR Tuinstra, J Bognar, KJ Barnard, EE Armstrong, High resolution image reconstruction from digital video with global and non-global scene motion. Proceedings of IEEE International Conference on Image Processing (ICIP '97), October 1997, Santa Barbara, Calif, USA 1, 153–156

  17. M Elad, A Feuer, Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Transactions on Image Processing 6(12), 1646–1658 (1997). PubMed Abstract | Publisher Full Text OpenURL

  18. M Elad, A Feuer, Superresolution restoration of an image sequence: adaptive filtering approach. IEEE Transactions on Image Processing 8(3), 387–395 (1999). PubMed Abstract | Publisher Full Text OpenURL

  19. J Chung, E Haber, J Nagy, Numerical methods for coupled super-resolution. Inverse Problems 22(4), 1261–1272 (2006). Publisher Full Text OpenURL

  20. RC Hardie, KJ Barnard, EE Armstrong, Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing 6(12), 1621–1633 (1997). PubMed Abstract | Publisher Full Text OpenURL

  21. NA Woods, NP Galatsanos, AK Katsaggelos, Stochastic methods for joint registration, restoration, and interpolation of multiple undersampled images. IEEE Transactions on Image Processing 15(1), 201–213 (2006). PubMed Abstract OpenURL

  22. H Shen, L Zhang, B Huang, P Li, A MAP approach for joint motion estimation, segmentation, and super resolution. IEEE Transactions on Image Processing 16(2), 479–490 (2007). PubMed Abstract OpenURL

  23. R Sasahara, H Hasegawa, I Yamada, K Sakaniwa, A color super-resolution with multiple nonsmooth constraints by hybrid steepest descent method. Proceedings of IEEE International Conference on Image Processing (ICIP '05), September 2005, Genova, Italy 1, 857–860

  24. S Farsiu, M Elad, P Milanfar, Multiframe demosaicing and super-resolution of color images. IEEE Transactions on Image Processing 15(1), 141–159 (2006). PubMed Abstract OpenURL

  25. T Akgun, Y Altunbasak, RM Mersereau, Super-resolution reconstruction of hyperspectral images. IEEE Transactions on Image Processing 14(11), 1860–1875 (2005). PubMed Abstract OpenURL

  26. CA Segall, AK Katsaggelos, R Molina, J Mateos, Bayesian resolution enhancement of compressed video. IEEE Transactions on Image Processing 13(7), 898–910 (2004). PubMed Abstract | Publisher Full Text OpenURL

  27. CA Segall, R Molina, AK Katsaggelos, High-resolution images from low-resolution compressed video. IEEE Signal Processing Magazine 20(3), 37–48 (2003). Publisher Full Text OpenURL

  28. D Capel, A Zisserman, Super-resolution enhancement of text image sequences. Proceedings of the 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 1, 600–605

  29. YB Han, LN Wu, Super resolution reconstruction of video sequence based on total variation. Proceedings of International Symposium on Intelligent Multimedia, Video and Speech Processing (ISIMP '04), October 2004, Hong Kong, 575–578

  30. C Vazquez, H Aly, E Dubois, A Mitiche, Motion compensated super-resolution of video by level sets evolution. Proceedings of IEEE International Conference on Image Processing (ICIP '04), October 2004, Singapore 3, 1767–1770

  31. S Farsiu, MD Robinson, M Elad, P Milanfar, Fast and robust multiframe super resolution. IEEE Transactions on Image Processing 13(10), 1327–1344 (2004). PubMed Abstract | Publisher Full Text OpenURL

  32. TF Chan, J Shen, Mathematical models for local nontexture inpaintings. SIAM Journal on Applied Mathematics 62(3), 1019–1043 (2002). Publisher Full Text OpenURL

  33. S Borman, RL Stevenson, Spatial resolution enhancement of low-resolution image sequences: a comprehensive review with directions for future research (Laboratory for Image and Signal Analysis (LISA), University of Notre Dame, Notre Dame, Ind, USA, 1998)

  34. SC Park, MK Park, MG Kang, Super-resolution image reconstruction: a technical overview. IEEE Signal Processing Magazine 20(3), 21–36 (2003). Publisher Full Text OpenURL

  35. N Nguyen, P Milanfar, G Golub, Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Transactions on Image Processing 10(9), 1299–1308 (2001). PubMed Abstract | Publisher Full Text OpenURL

  36. D Capel, A Zisserman, Computer vision applied to super resolution. IEEE Signal Processing Magazine 20(3), 75–86 (2003). Publisher Full Text OpenURL

  37. RR Schultz, L Meng, RL Stevenson, Subpixel motion estimation for super-resolution image sequence enhancement. Journal of Visual Communication and Image Representation 9(1), 38–50 (1998). Publisher Full Text OpenURL

  38. AM Tekalp, Digital Video Processing (Prentice-Hall, Englewood Clliffs, NJ, USA, 1995)

  39. S Farsiu, D Robinson, M Elad, P Milanfar, Advances and challenges in super-resolution. International Journal of Imaging Systems and Technology 14(2), 47–57 (2004). Publisher Full Text OpenURL

  40. L Rudin, S Osher, E Fatemi, Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992)

  41. CR Vogel, ME Oman, Fast, robust total variation-based reconstruction of noisy, blurred images. IEEE Transactions on Image Processing 7(6), 813–824 (1998). PubMed Abstract | Publisher Full Text OpenURL

  42. A Chambolle, An algorithm for total variation minimization and applications. Journal of Mathematical Imaging and Vision 20(1-2), 89–97 (2004)

  43. Y Li, F Santosa, A computational algorithm for minimizing total variation in image restoration. IEEE Transactions on Image Processing 5(6), 987–995 (1996). PubMed Abstract | Publisher Full Text OpenURL

  44. JM Bioucas-Dias, MAT Figueiredo, JP Oliveira, Total variation-based image deconvolution: a majorization-minimization approach. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '06), May 2006, Toulouse, France 2, 861–864

  45. CR Vogel, ME Oman, Iterative methods for total variation denoising. SIAM Journal of Scientific Computing 17(1), 227–238 (1996). Publisher Full Text OpenURL

  46. CR Vogel, Computational Methods for Inverse Problems, Frontiers in Applied Mathematics (SIAM, Philadelphia, Pa, USA, 2002)

  47. F-R Lin, MK Ng, W-K Ching, Factorized banded inverse preconditioners for matrices with Toeplitz structure. SIAM Journal of Scientific Computing 26(6), 1852–1870 (2005). Publisher Full Text OpenURL

  48. RH Chan, TF Chan, C-K Wong, Cosine transform based preconditioners for total variation deblurring. IEEE Transactions on Image Processing 8(10), 1472–1478 (1999). PubMed Abstract | Publisher Full Text OpenURL

  49. MK Ng, RH Chan, TF Chan, AM Yip, Cosine transform preconditioners for high resolution image reconstruction. Linear Algebra and Its Applications 316(1–3), 89–104 (2000)

  50. LY Kolotilina, AY Yeremin, Factorized sparse approximate inverse preconditionings I: theory. SIAM Journal on Matrix Analysis and Applications 14(1), 45–58 (1993). Publisher Full Text OpenURL

  51. N Nguyen, P Milanfar, G Golub, A computationally efficient superresolution image reconstruction algorithm. IEEE Transactions on Image Processing 10(4), 573–583 (2001). PubMed Abstract | Publisher Full Text OpenURL