Neural Image Style Transfer Using Modified Histogram Matching

Authors

  • Anas Mudassir
  • Terence Johnson
  • Santosh Singh

DOI:

https://doi.org/10.37266/ISER.2022v10i1.pp44-54

Keywords:

Neural Style Transfer, Histogram Matching, CLAHE, Laplacian operator, Gaussian Operator, Marr-Hildreth Function, CNN, VGG-19 model

Abstract

Neural Image Style Transfer is an algorithm which was first introduced in 2016 for transferring the style of an image using a CNN. Over the years, a large portion of the research has been devoted in balancing the trade-off between time taken by the algorithm to process the images and the quality of the results generated. In addition, the application of traditional image processing algorithms has been limited. In this paper, we aim to introduce a modification to the traditional method by utilizing a localized histogram matching algorithm combined with Contrast Limited Adaptive Histogram Equalization (CLAHE). The results show that the quality of style transferred and the colour of images obtained by this method is much better than the traditional method for the same number of iterations.

References

Adelson, E. H., Anderson, C. H., Bergen, J. R., Burt, P. J. and Ogden, J. M. (1984). 1984, Pyramid methods in image processing. RCA Engineer, 29, 33-41.

Bevilacqua, A. and Azzari, P. (2007). "A High Performance Exact Histogram Specification Algorithm," 14th International Conference on Image Analysis and Processing (ICIAP 2007), 2007, pp. 623-628, doi: 10.1109/ICIAP.2007.4362846.

Chen, D., Yuan, L., Liao, J., Yu, N., and Hua, G. (2017). "StyleBank: An Explicit Representation for Neural Image Style Transfer," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 2770-2779, doi: 10.1109/CVPR.2017.296.

Chen, T.Q. and Schmidt, M. (2016). “Fast patch-based style transfer of arbitrary style,” in Proceedings of the NIPS Workshop on Constructive Machine Learning, 2016.

Dumoulin, V., Shlens, J., and Kudlur, M. (2017). A Learned Representation For Artistic Style. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net

Gatys, L., Bethge, M., Hertzmann, A, and Shechtman, E. (2016b). Preserving Color in Neural Artistic Style Transfer. ArXiv, abs/1606.05897

Gatys, L., Ecker, A., and Bethge, M. (2016a). "Image Style Transfer Using Convolutional Neural Networks," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 2414-2423, doi: 10.1109/CVPR.2016.265.

Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B, Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative Adversarial Networks. Advances in Neural Information Processing Systems. 3. 10.1145/3422622.

Huang, X. and Belongie, S. (2017). “Arbitrary style transfer in real-time with adaptive instance normalization,” in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 1501–1510.

Hussain, K., Rahman, S., Rahman, M., Shah, K., Abdullah-Al-Wadud, M., Khan, M., and Shoyaib, M. (2018). A histogram specification technique for dark image enhancement using a local transformation method. IPSJ Transactions on Computer Vision and Applications. 10. 10.1186/s41074-018-0040-0.

Jing, Y., Yang, Y., Feng, Z., Ye, J., Yu, Y., and Song, M. (2019). "Neural Style Transfer: A Review" in IEEE Transactions on Visualization & Computer Graphics, vol. 26, no. 11, pp. 3365-3385, 2020. doi: 10.1109/TVCG.2019.2921336.

Johnson, J., Alahi, A., and Fei-Fei, L. (2016). Perceptual Losses for Real-Time Style Transfer and Super-Resolution. In: Leibe B., Matas J., Sebe N., Welling M. (eds) Computer Vision – ECCV 2016. ECCV 2016. Lecture Notes in Computer Science, vol 9906. Springer, Cham. https://doi.org/10.1007/978-3-319-46475-6_43

Li, C. and Wand, M. (2016). Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks. In: Leibe B., Matas J., Sebe N., Welling M. (eds) Computer Vision – ECCV 2016. ECCV 2016. Lecture Notes in Computer Science, vol 9907. Springer, Cham. https://doi.org/10.1007/978-3-319-46487-9_43

Li, Y., Wang, N., Liu, J., and Hou, X. (2017a). “Demystifying neural style transfer.” In Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI'17). AAAI Press, 2230–2236.

Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., and Yang, M. (2017b). "Diversified Texture Synthesis with Feed-Forward Networks," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 266-274, doi: 10.1109/CVPR.2017.36.

Li, S., Xu, X., Nie, L., and Chua, T.-S. (2017c). “Laplacian-steered neural style transfer.” In Proceedings of the 2017 ACM on Multimedia Conference. ACM, 2017, pp. 1716–1724.

Marr, D. and Hildreth, E. (1980). Theory of Edge Detection. Proceedings of the Royal Society of London. Series B, Containing papers of a Biological character. Royal Society (Great Britain). 207. 187-217. 10.1098/rspb.1980.0020.

Mallikeswari, B. and Sripriya, P. (2018). "A Review of Image Enhancement Algorithms for Low-Contrast, Infra-red and Night Image.” International Journal of Creative Research Thoughts (IJCRT), ISSN:2320-2882, Volume.6, Issue 1, Page No pp.200 - 207, January 2018, Available at : http://www.ijpub.org/IJPUB1801033

Nixon, M and Aguado A.S. (2002). Feature Extraction & Image Processing for Computer Vision, First Edition (1st. ed.). Academic Press, Inc., USA.

Peng, M. and Zhu, Z. (2019). "Enhanced Style Transfer in Real-Time with Histogram-Matched Instance Normalization." 2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Conference on Smart City; IEEE 5th International Conference on Data Science and Systems (HPCC/SmartCity/DSS), 2019, pp. 2001-2006, doi: 10.1109/HPCC/SmartCity/DSS.2019.00276.

Risser, E., Wilmot, P., and Barnes, C. (2017). Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses. ArXiv, abs/1701.08893.

Tu, L. and Dong, C. (2013). "Histogram equalization and image feature matching." 2013 6th International Congress on Image and Signal Processing (CISP), 2013, pp. 443-447, doi: 10.1109/CISP.2013.6744035.

Ulyanov, D., Lebedev, V., Vedaldi, A., and Lempitsky, V. (2016). “Texture networks: feed-forward synthesis of textures and stylized images.” In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 (ICML'16). JMLR.org, 1349–1357.

Walt, S., Schönberger, J., Nunez-Iglesias, J., Boulogne, F., Warner, J., Yager, N., Gouillart, E., Yu, T., and the scikit-image contributors (2014). scikit-image: Image processing in Python. PeerJ 2:e453 (2014) https://doi.org/10.7717/peerj.453.

Yang, F. and Wu, J. (2010). "An improved image contrast enhancement in multiple-peak images based on histogram equalization." 2010 International Conference On Computer Design and Applications, 2010, pp. V1-346-V1-349, doi: 10.1109/ICCDA.2010.5540857.

Zhang, H. and Dana, K. (2018). Multi-style Generative Network for Real-Time Transfer. In: Leal-Taixé L., Roth S. (eds) Computer Vision – ECCV 2018 Workshops. ECCV 2018. Lecture Notes in Computer Science, vol 11132. Springer, Cham. https://doi.org/10.1007/978-3-030-11018-5_32.

Zuiderveld, K. (1994). Contrast limited adaptive histogram equalization. Graphics gems IV. Academic Press Professional, Inc., USA, 474–485.

Published

2022-12-22

How to Cite

Mudassir, A., Johnson, T., & Singh, S. (2022). Neural Image Style Transfer Using Modified Histogram Matching. Industrial and Systems Engineering Review, 10(1), 44-54. https://doi.org/10.37266/ISER.2022v10i1.pp44-54