DOI: 10.17586/2226-1494-2015-15-4-640-653


G. A. Kukharev, Y. N. Matveev, P. Forczmański

Read the full article 
Article in Russian

For citation: Kukharev G.A., Matveev Yu.N., Forczmański P. People retrieval by means of composite pictures – methods, systems and practical decisions. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2015, vol.15, no. 4, pp. 640–653.

We discuss the problem of people retrieval by means of composite pictures and methods of its practical realization. Earlier on, the problem was posed in the previous paper by the authors, and this paper deals with its further development. The starting premise here is that for the successful search of people by their sketches it is necessary to transform these sketches into sets of populations of sketches imitating evidence of «group of witnesses» and evidence with incomplete information in verbal portraits. Variants of structures for benchmark «photo-sketch» databases are presented, intended for modeling and practical realization of original photos retrieval by sketches, which new component is a population of sketches. Problems of preprocessing for initial sketches and original photos and its influence on the result of their comparison are discussed. Simple sketch recognition systems (Simple FaRetSys) and a problem of original photos retrieval by the sketches are considered. Shortcomings of such systems are shown and new decisions on extending and development of simple systems (Extended FaRetSys) are presented. Experiments on searching of original photos by sketches in the CUFS database of sketches and similar experiments on widely known FERET and CUFSF facial databases are presented. Three frameworks are offered for retrieval performance improvement. In the first one, original sketches are transformed into populations, and then in these populations the sketch similar to the given sketch (Forensic Sketch) is already defined. The class of the sketch found in a population «by definition» unambiguously corresponds to a class of the original photo. In the second framework, the Forensic Sketch is transformed to a population of sketches, and all original sketches in a benchmarking database are compared to sketches from populations of the Forensic Sketch. The class of matches is determined in the same manner as in the first framework. The third framework includes generation of a population of sketches, both from all original sketches, and from all Forensic Sketches. The further line of research is obvious: retrieval by matching between sketches of these two populations.

Keywords: original photo, composite picture, sketch, population of sketches, retrieval systems.

Acknowledgements. The work is partially financially supported by the Government of the Russian Federation (grant 074-U01).

1. Chabot P. The photo robot. Its use, its production, its future. International Criminal Police Review, 1959, no. 97.
2. Uhl R.J. Jr., Lobo N.V. A framework for recognizing a facial image from a police sketch. Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, USA, 1996, pp. 586–593. doi: 10.1109/CVPR.1996.517132
3. Wang X., Tang X. Face photo-sketch synthesis and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, vol. 31, no. 11, pp. 1955–1967. doi: 10.1109/TPAMI.2008.222
4. Klare B., Li Z., Jain A.K. Matching forensic sketches to mug shot photos. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, vol. 33, no. 3, pp. 639–646. doi: 10.1109/TPAMI.2010.180
5. Han H., Klare B.F., Bonnen K., Jain A.K. Matching composite sketches to face photos: a component-based approach. IEEE Transactions on Information Forensics and Security, 2013, vol. 8, no. 3, pp. 191–204. doi: 10.1109/TIFS.2012.2228856
6. Kukharev G.A., Matveev Yu.N., Shchegoleva N.L. Poisk lyudei po fotorobotam: sostoyanie problemy i tekhnologii [People retrieval by means of composite pictures: problem state-of-the-art and technologies]. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2014, no. 6 (94), pp. 123–136.
7. Frowd C.В., Hancock P.J.B., Carson D. EvoFIT: a holistic, evolutionary facial imaging technique for creating composites. ACM Transactions on Applied Psychology, 2004, vol. 1, pp. 1–21.
8. George B., Gibson S.J., Maylin M.I.S., Solomon C.J. EFIT-V - Interactive evolutionary strategy for the construction of photo-realistic facial composites. Proc. 10th Annual Conference on Genetic and Evolutionary Computation. Atlanta, USA, 2008, pp. 1485–1490.
9. Kukharev G., Buda K., Shchegoleva N. Sketch generation from photo to create test databases. Przeglad Elektrotechniczny (Electrical Review), 2014, vol. 90, no. 2, pp. 97–100. doi: 10.12915/pe.2014.02.26
10. Kukharev G.A., Buda K., Shchegoleva N.L. Methods of face photo-sketch comparison. Pattern Recognition and Image Analysis, 2014, vol. 24, no. 1, pp. 102–113. doi: 10.1134/S1054661814010076
11. Shchegoleva N.L., Kukharev G.A. Automatic Sketch Generation Method and System Therefor. Patent RU2541132, 2015.
12. Kukharev G.A., Kamenskaya E.I., Matveev Y.N., Shchegoleva N.L. Metody Obrabotki i Raspoznavaniya Izobrazhenii Lits v Zadachakh Biometrii [Methods for Face Image Processing and Recognition in Biometric Applications] Ed. M.V. Khitrov. St. Petersburg, Politekhnika Publ., 2013, 388 p.
13. Yu H., Zhang J.J. Mean value coordinates–based caricature and expression synthesis. Signal, Image and Video Processing, 2013, vol. 7, no. 5, pp. 899–910. doi: 10.1007/s11760-011-0279-8
14. Wang Z., Bovik A.C. A universal image quality index. IEEE Signal Processing Letters, 2002, vol. 9, no. 3, pp. 81–84. doi: 10.1109/97.995823
15. Wang Z., Bovik A.C., Sheikh H.R., Simoncelli E.P. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, vol. 13, no. 4, pp. 600–612. doi: 10.1109/TIP.2003.819861
16. Zhang Y., Ellyson S., Zone A., Gangam P., Sullins J., McCullough C., Canavan S., Yin L. Recognizing face sketches by a large number of human subjects: a perception-based study for facial distinctiveness. Proc. IEEE Int. Conf. on Automatic Face and Gesture Recognition and Workshops, FG 2011. Santa Barbara, USA, 2011, pp. 707–712. doi: 10.1109/FG.2011.5771335
Copyright 2001-2017 ©
Scientific and Technical Journal
of Information Technologies, Mechanics and Optics.
All rights reserved.