Neural Image Style Transfer Using Modified Histogram Matching
DOI:
https://doi.org/10.37266/ISER.2022v10i1.pp44-54Keywords:
Neural Style Transfer, Histogram Matching, CLAHE, Laplacian operator, Gaussian Operator, Marr-Hildreth Function, CNN, VGG-19 modelAbstract
Neural Image Style Transfer is an algorithm which was first introduced in 2016 for transferring the style of an image using a CNN. Over the years, a large portion of the research has been devoted in balancing the trade-off between time taken by the algorithm to process the images and the quality of the results generated. In addition, the application of traditional image processing algorithms has been limited. In this paper, we aim to introduce a modification to the traditional method by utilizing a localized histogram matching algorithm combined with Contrast Limited Adaptive Histogram Equalization (CLAHE). The results show that the quality of style transferred and the colour of images obtained by this method is much better than the traditional method for the same number of iterations.
References
Adelson, E. H., Anderson, C. H., Bergen, J. R., Burt, P. J. and Ogden, J. M. (1984). 1984, Pyramid methods in image processing. RCA Engineer, 29, 33-41.
Bevilacqua, A. and Azzari, P. (2007). "A High Performance Exact Histogram Specification Algorithm," 14th International Conference on Image Analysis and Processing (ICIAP 2007), 2007, pp. 623-628, doi: 10.1109/ICIAP.2007.4362846.
Chen, D., Yuan, L., Liao, J., Yu, N., and Hua, G. (2017). "StyleBank: An Explicit Representation for Neural Image Style Transfer," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 2770-2779, doi: 10.1109/CVPR.2017.296.
Chen, T.Q. and Schmidt, M. (2016). “Fast patch-based style transfer of arbitrary style,” in Proceedings of the NIPS Workshop on Constructive Machine Learning, 2016.
Dumoulin, V., Shlens, J., and Kudlur, M. (2017). A Learned Representation For Artistic Style. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net
Gatys, L., Bethge, M., Hertzmann, A, and Shechtman, E. (2016b). Preserving Color in Neural Artistic Style Transfer. ArXiv, abs/1606.05897
Gatys, L., Ecker, A., and Bethge, M. (2016a). "Image Style Transfer Using Convolutional Neural Networks," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 2414-2423, doi: 10.1109/CVPR.2016.265.
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B, Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative Adversarial Networks. Advances in Neural Information Processing Systems. 3. 10.1145/3422622.
Huang, X. and Belongie, S. (2017). “Arbitrary style transfer in real-time with adaptive instance normalization,” in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 1501–1510.
Hussain, K., Rahman, S., Rahman, M., Shah, K., Abdullah-Al-Wadud, M., Khan, M., and Shoyaib, M. (2018). A histogram specification technique for dark image enhancement using a local transformation method. IPSJ Transactions on Computer Vision and Applications. 10. 10.1186/s41074-018-0040-0.
Jing, Y., Yang, Y., Feng, Z., Ye, J., Yu, Y., and Song, M. (2019). "Neural Style Transfer: A Review" in IEEE Transactions on Visualization & Computer Graphics, vol. 26, no. 11, pp. 3365-3385, 2020. doi: 10.1109/TVCG.2019.2921336.
Johnson, J., Alahi, A., and Fei-Fei, L. (2016). Perceptual Losses for Real-Time Style Transfer and Super-Resolution. In: Leibe B., Matas J., Sebe N., Welling M. (eds) Computer Vision – ECCV 2016. ECCV 2016. Lecture Notes in Computer Science, vol 9906. Springer, Cham. https://doi.org/10.1007/978-3-319-46475-6_43
Li, C. and Wand, M. (2016). Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks. In: Leibe B., Matas J., Sebe N., Welling M. (eds) Computer Vision – ECCV 2016. ECCV 2016. Lecture Notes in Computer Science, vol 9907. Springer, Cham. https://doi.org/10.1007/978-3-319-46487-9_43
Li, Y., Wang, N., Liu, J., and Hou, X. (2017a). “Demystifying neural style transfer.” In Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI'17). AAAI Press, 2230–2236.
Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., and Yang, M. (2017b). "Diversified Texture Synthesis with Feed-Forward Networks," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 266-274, doi: 10.1109/CVPR.2017.36.
Li, S., Xu, X., Nie, L., and Chua, T.-S. (2017c). “Laplacian-steered neural style transfer.” In Proceedings of the 2017 ACM on Multimedia Conference. ACM, 2017, pp. 1716–1724.
Marr, D. and Hildreth, E. (1980). Theory of Edge Detection. Proceedings of the Royal Society of London. Series B, Containing papers of a Biological character. Royal Society (Great Britain). 207. 187-217. 10.1098/rspb.1980.0020.
Mallikeswari, B. and Sripriya, P. (2018). "A Review of Image Enhancement Algorithms for Low-Contrast, Infra-red and Night Image.” International Journal of Creative Research Thoughts (IJCRT), ISSN:2320-2882, Volume.6, Issue 1, Page No pp.200 - 207, January 2018, Available at : http://www.ijpub.org/IJPUB1801033
Nixon, M and Aguado A.S. (2002). Feature Extraction & Image Processing for Computer Vision, First Edition (1st. ed.). Academic Press, Inc., USA.
Peng, M. and Zhu, Z. (2019). "Enhanced Style Transfer in Real-Time with Histogram-Matched Instance Normalization." 2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Conference on Smart City; IEEE 5th International Conference on Data Science and Systems (HPCC/SmartCity/DSS), 2019, pp. 2001-2006, doi: 10.1109/HPCC/SmartCity/DSS.2019.00276.
Risser, E., Wilmot, P., and Barnes, C. (2017). Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses. ArXiv, abs/1701.08893.
Tu, L. and Dong, C. (2013). "Histogram equalization and image feature matching." 2013 6th International Congress on Image and Signal Processing (CISP), 2013, pp. 443-447, doi: 10.1109/CISP.2013.6744035.
Ulyanov, D., Lebedev, V., Vedaldi, A., and Lempitsky, V. (2016). “Texture networks: feed-forward synthesis of textures and stylized images.” In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 (ICML'16). JMLR.org, 1349–1357.
Walt, S., Schönberger, J., Nunez-Iglesias, J., Boulogne, F., Warner, J., Yager, N., Gouillart, E., Yu, T., and the scikit-image contributors (2014). scikit-image: Image processing in Python. PeerJ 2:e453 (2014) https://doi.org/10.7717/peerj.453.
Yang, F. and Wu, J. (2010). "An improved image contrast enhancement in multiple-peak images based on histogram equalization." 2010 International Conference On Computer Design and Applications, 2010, pp. V1-346-V1-349, doi: 10.1109/ICCDA.2010.5540857.
Zhang, H. and Dana, K. (2018). Multi-style Generative Network for Real-Time Transfer. In: Leal-Taixé L., Roth S. (eds) Computer Vision – ECCV 2018 Workshops. ECCV 2018. Lecture Notes in Computer Science, vol 11132. Springer, Cham. https://doi.org/10.1007/978-3-030-11018-5_32.
Zuiderveld, K. (1994). Contrast limited adaptive histogram equalization. Graphics gems IV. Academic Press Professional, Inc., USA, 474–485.
Published
How to Cite
Issue
Section
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
The copyediting stage is intended to improve the flow, clarity, grammar, wording, and formatting of the article. It represents the last chance for the author to make any substantial changes to the text because the next stage is restricted to typos and formatting corrections. The file to be copyedited is in Word or .rtf format and therefore can easily be edited as a word processing document. The set of instructions displayed here proposes two approaches to copyediting. One is based on Microsoft Word's Track Changes feature and requires that the copy editor, editor, and author have access to this program. A second system, which is software independent, has been borrowed, with permission, from the Harvard Educational Review. The journal editor is in a position to modify these instructions, so suggestions can be made to improve the process for this journal.
