ESTIMATION OF MALWARE DETECTION ALGORITHM ACCURACY BASED ON ANOMALY SEARCH IN PROGRAM BEHAVIOR

Subject of Research.The paper deals with the algorithm of anomaly detection in the behavior of operating system processes caused by the execution of previously unknown parts of the program code. The algorithm is implemented in the novel intrusion detection system CODA. A testing algorithm allows reducing test time and increasing its accuracy. Method. The proposed detection method is based on creation ofbehavior model for legitimate process using sequences of system calls. Measures of similarity between an arbitrary process and a model are proposed. They allow interpreting the problem of anomaly detection as the problem of vector classification. In order to evaluate the accuracy of the anomaly detection algorithm, the accuracy of the classifier is proposed to be evaluated by cross-validation method. Neural network of perceptron type was used as a classifier. Main Results.A platform for the mass distributed testing of malicious programs in virtual machines was developed. Open source library for distributed computing BOINC was used in the platformimplementation. Academic base of malware and open base Malwr was used to select 60 thousand malicious programs. From the general base33.13% of malware have workedcorrectly. A model of legitimate processes runningwithin half an hourwas created. Estimates ofmalware behavior were recorded as vectors. The most accurate neural network was searched for these vectors classification. Neural networks with different teaching parameters and different number of neurons in a hidden layer were looked over. The most precise perceptron was discovered. The accuracy of the best classifier was 91%. Practical Relevance. The results can be useful in malware detection. Our algorithm does not require Internet connection.It can find both old and new malware. 

1. Nachenberg C. Computer virus-antivirus coevolution. Communications of the ACM, 1997, vol. 40, no. 1, pp. 46–51. doi: 10.1145/242857.242869
2. Barat M., Prelipcean D.-B., Gavrilu D.T., A study on common malware families evolution in 2012. Journal in Computer Virology, 2013, vol. 9, no. 4, pp. 171–178. doi: 10.1007/s11416-013-0192-5
3. Denning D.E. An intrusion-detection model. IEEE Transactions on Software Engineering, 1987, vol. SE-13, no. 2, pp. 222–232. doi: 10.1109/TSE.1987.232894
4. Forrest S., Hofmeyr S., Somayaji A., Longstaff T.A. Sense of self for Unix processes. Proc. IEEE Symposium on Security and Privacy. Oakland, USA, 1996, pp. 120–128.
5. Hofmeyr S.A., Forrest S., Somayaji A. Intrusion detection using sequences of system calls. Journal of Computer Security, 1998, vol. 6, no. 3, pp. 151–180.
6. Warrender C., Forrest S., Pearlmutter B. Detecting intrusions using system calls: alternative data models. Proc. IEEE Symposium on Security and Privacy. Oakland, USA, 1999, pp. 133–145.
7. Wespi A., Dacier M., Debar H. Intrusion detection using variable-length audit trail patterns. Lecture Notes in Computer Science, 2000, vol. 1907, pp. 110–129.
8. Wespi A., Dacier M., Debar H. An intrusion-detection system based on the Teiresias pattern discovery algorithm. Proc. EICAR '99. Aalborg, Germany, 1999, pp. 1–15.
9. Sekar R., Bendre M., Dhurjati D., Bollineni P. A fast automation-based method for detecting anomalous program behaviors. Proc. IEEE Symposium on Security and Privacy. Oakland, USA, 2001, pp. 144–155.
10. Mutz D., Valeur F., Vigna G., Kruegel C. Anomalous system call detection. ACM Transactions on Information Systems Security, 2006, vol. 9, no. 1, pp. 61–93.
11. Maggi F., Matteucci M., Zanero S. Detecting intrusions through system call sequence and argument analysis. IEEE Transactions on Dependable and Secure Computing, 2010, vol. 7, no. 4, pp. 381–395. doi: 10.1109/TDSC.2008.69
12. Milea N.A., Khoo S.C., Lo D., Pop C. NORT: runtime anomaly-based monitoring of malicious behavior for Windows. Lecture Notes in Computer Science, 2012, vol. 7186, pp. 115–130. doi: 10.1007/978-3-642-29860-8_10
13. Oladko V.S., Sadovnik E.A. Algorithms for detection of abnormal activity processes. Herald of Computer and Information Technologies, 2015, no. 8, pp. 35–39. doi: 10.14489/vkit.2015.08.рр.035-039
14. Sadovnik E.A., Olad'ko V.S., Ermakova A.Yu., Mikova S.Yu. Functions and problems of intrusion detection system based on the analysis of the computational processes activity. Innovatsionnaya Nauka, 2016, no. 1-2, pp. 121–124. (In Russian)
15. Vaganov M.Yu. Gibridnaya Iskusstvennaya Immunnaya Sistema Zashchity Komp'yutera ot Protsessov s Anomal'noi Aktivnost'yu: dis. … kand. tekhn. nauk [Hybrid Artificial Immune System for Computer Protection from Abnormal Activity Processes. Diss. PhD Tech. Sci.]. St. Petersburg, 2012, 92 p.
16. Prokhorov R.S. Behavioristic process identification. Mathematical Structures and Modeling, 2013, vol. 27, no. 1, pp. 103–112.
17. Oderov R.S., Tensin E. Methods for mounting of its code into OS Microsoft Windows Server 2008 kernel. Sbornik Trudov Mezhvuzovskoi Nauchno-Prakticheskoi Konferentsii «Aktual'nye Problemy Organizatsii i Tekhnologii Zashchity Informatsii» [Proc. Interuniversity Conf. on Topical Problems of Organization and Techniques in Information Security]. St. Petersburg, NRU ITMO, 2011, pp. 100–102. (In Russian)
18. Baklanovsky M.V., Khanov A.R. Identification of programs based on the behavior. Modeling and Analysis of Information Systems, 2014, vol. 21, no. 6, pp. 120–130.
19. Karkkainen J., Sanders P., Burkhardt S. Linear work suffix array construction. Journal of the ACM, 2006, vol. 53, no. 3, pp. 918–936. doi: 10.1145/1217856.1217858
 

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License