cross-modal multimedia retrieval, principal component analysis, face images, sketch, facial composite Acknowledgements.
This work was partially financially supported by the Government of the Russian Federation, Grant 074-U01 References
1. Kato T. Database architecture for content-based image retrieval. Proceedings of SPIE, 1992, vol. 1662, pp. 112–123.
2. Wang K., Yin Q., Wang W., Wu S., Wang L. A comprehensive survey on cross-modal retrieval. ArXiv Preprint, 2016, ArXiv160706215, 20 p.
3. Liao S., Yi D., Lei Z., Qin R., Li S.Z. Heterogeneous face recognition from local structures of normalized appearance. Lecture Notes in Computer Science
, 2009, vol. 5558, pp. 209–218. doi: 10.1007/978-3-642-01793-3_22
4. Guo G. Heterogeneous face recognition: an emerging topic in biometrics. Intel Technology Journal, 2014, vol. 18, no. 4, pp. 80–97.
5. Kukharev G.A., Shchegoleva N.L., Kamenskaya E.I. Representation and comparison methods for semantically different images. Pattern Recognition and Image Analysis
, 2014, vol. 24, no. 4, pp. 518–529. doi: 10.1134/S1054661814040105
6. Tang X., Wang X. Face sketch recognition. IEEE Transactions on Circuits and Systems for Video Technology
, 2004, vol. 14, no. 1, pp. 50–57. doi: 10.1109/TCSVT.2003.818353
7. Kukharev G., Oleinik A. Face photo-sketch transformation and population generation. Lecture Notes in Computer Science
, 2016, vol. 9972, pp. 329–340. doi: 10.1007/978-3-319-46418-3_29
8. Kukharev G.A., Kamenskaya E.I., Matveev Y.N., Shchegoleva N.L. Metody Obrabotki i Raspoznavaniya Izobrazhenii Lits v Zadachakh Biometrii [Methods for Face Image Processing and Recognition in Biometric Applications] Ed. M.V. Khitrov. St. Petersburg, Politekhnika Publ., 2013, 388 p.
9. CUHK Face Sketch Database. Available at: http://mmlab.ie.cuhk.edu.hk/facesketch.html (accessed 11.08.2014).
10. CUHK Face Sketch FERET Database. Available at: http://mmlab.ie.cuhk.edu.hk/cufsf/ (accessed 11.08.2014).
11. Chen H., Liu Z., Rose C., Xu Y., Shum H.Y., Salesin D. Example-based composite sketching of human portraits. Proc. 3rd Int. Symposium on Non-Photorealistic Animation and Rendering. Annecy, France, 2004, pp. 95–102.
12. Ivanova A. Dog instinctively imitates humans. Dogs, like owners. Available at: http://zdravkom.ru/nauka_i_obrazovanie/sobaki_kopiruyt_hozyaev (accessed 20.11.2016).
13. Shchegoleva N.L., Kukharev G.A. Automatic Sketch Generation Method and System Therefor. Patent RU2541132, 2015.
14. Zhang Y., Ellyson S., Zone A., Gangam P., Sullins J., McCullough C., Canavan S., Yin L. Recognizing face sketches by a large number of human subjects: a perception-based study for facial distinctiveness. Proc. IEEE Int. Conf. on Automatic Face and Gesture Recognition and Workshops, FG 2011
. Santa Barbara, USA, 2011, pp. 707–712. doi: 10.1109/FG.2011.5771335
15. Kukharev G.A., Matveev Y.N., Shchegoleva N.L. New solutions for face photo retrieval based on sketches. Pattern Recognition and Image Analysis
, 2016, vol. 26, no. 1, pp. 165–175. doi: 10.1134/S1054661816010144
16. Kukharev G.A., Matveev Yu.N., Forczmanski P. People retrieval by means of composite pictures – methods, systems and practical decisions. Scientific and Technical Journal of Information Technologies, Mechanics and Optics
, 2015, vol.15, no. 4, pp. 640–653. doi: 10.17586/2226-1494-2015-15-4-640-653
17. Tsapatsoulis N., Alexopoulos V., Kollias S. A vector based approximation of KLT and its application to face recognition. Proc. 9th European Signal Processing Conference EUSIPCO-98. Island of Rhodes, Greece, 1998, vol. 1581, pp. 1–4.
18. Shchegoleva N.L., Kukharev G.A. Application of two-dimensional principal component analysis for recognition of face images. Pattern Recognition and Image Analysis
, 2010, vol. 20, no. 4, pp. 513–527. doi: 10.1134/S1054661810040127
19. Kukharev G., Forczmanski P. Data dimensionality reduction for face recognition. Machine Graphics and Vision, 2004, vol. 13, no. 1-2, pp. 99–121.
Kukharev G., Tujaka A., Forczmanski P. Face recognition using two-dimensional CCA and PLS. International Journal of Biometrics
, 2011, vol. 3, no. 4, pp. 300–321. doi: 10.1504/IJBM.2011.042814
Fig. 1. The training sample and the results of transformation. Sequences 1 and 2 show training data sets. Sequence 3 is the reconstruction result based on the images of the sequence 2. Sequence 4 presents images of sequence 2 in low resolution with added noise. Sequence 5 is the transformation result of the images out of sequence 4
Fig. 2. Transformation of the test sketches. Sequence 1 – the initial sketches, sequence 2 is the transformation result for the sketches from sequence 1 into the photo, sequence 3 – original photos
Fig. 3. Sketches of one and the same person, made by various artists 
Fig. 4. Scheme of the proposed algorithm of transformation and populations generation
Fig. 5. Populations of photos generated from the sketch (a), and three-dimensional visualization of photograph projections on the three first principal components (b). Each photograph of a population corresponds to one point on the graph
Fig. 6. Populations of photos, generated on the basis of sketches from the training (a) and test (b) samples
Fig. 7. Transformation of sketches into pictures by two-dimensional algorithm. Sequence 1 – the initial sketches, sequence 2 – transformation result. Transformation was carried out for sketches from the training (a) and test (b) samples and also for the synthesized sketches  (c)
Fig. 8. Transformation of sketches from AR base into the photos by two-dimensional algorithm. Sequence 1 – the initial sketches, sequence 2 – the result of transformation