Menu
Publications
2025
2024
2023
2022
2021
2020
2019
2018
2017
2016
2015
2014
2013
2012
2011
2010
2009
2008
2007
2006
2005
2004
2003
2002
2001
Editor-in-Chief
Nikiforov
Vladimir O.
D.Sc., Prof.
Partners
doi: 10.17586/2226-1494-2025-25-3-527-535
On the properties of compromise M-estimators optimizing weighted L2-norm of the influence function
Read the full article
';
Article in Russian
For citation:
Abstract
For citation:
Lisitsin D.V., Gavrilov K.V. On the properties of compromise M-estimators optimizing weighted L2-norm of the influence function. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2025, vol. 25, no. 3, pp. 527–535 (in Russian). doi: 10.17586/2226-1494-2025-25-3-527-535
Abstract
The paper develops a theory of M-estimators optimizing the weighted L2-norm of the influence function. The specified criterion of the estimation quality is quite general and, in addition, allows obtaining solutions related to the class of redescending estimators, i.e., possessing the property of stability to asymmetric contamination. Such estimators, in particular, were studied within the framework of the locally stable approach of A.M. Shurygin, based on the analysis of the estimator instability functional (L2-norm of the influence function), or his approach based on the model of a series of samples with random point contamination (point Bayesian contamination model). In this paper, a compromise family of estimators is studied for which the optimized functional is a convex linear combination of two basic criteria. The compromise family is similar to the conditionally optimal family of estimators proposed by A.M. Shurygin, but the criteria used can be squares of the weighted L2-norms of the influence function with arbitrary pre-known weight functions. The considered subject area has remained little-studied to date. In the course of the research, we used a theory we had developed earlier, which describes the properties of estimators that optimize the weighted L2-norm of the influence function. As a result of the study, a number of properties of compromise estimators were obtained, and the uniqueness of the family elements was shown. A family member that delivers equal values to the criteria was considered separately: it was shown that this estimator corresponds to the saddle point of the optimized functional, and is also a minimax solution with respect to the basic criteria on the set of all regular score functions. The constructed theory is illustrated using the example of the problem of mathematical expectation estimating of a normal distribution under conditions of targeted malicious influence on a data set (similar to a data poisoning attack in malicious machine learning).
Keywords: M-estimators, robust statistics, influence function, stable estimates, redescending estimators, saddle point, data poisoning
References
References
- Borovkov A.A. Mathematical Statistics. Amsterdam: Gordon and Breach, 1998, 570 p.
- Shurygin A. M. Applied Stochastics: Robustness, Estimation, Prediction. Moscow, Finansy i statistika Publ., 2000, 223 p. (in Russian)
- Huber P., Ronchetti E. Robust Statistics. John Wiley & Sons, 2009, 354 p. https://doi.org/10.1002/9780470434697
- Hampel F., Ronchetti E., Rousseeuw P., Stahel W. Robust Statistics: The Approach Based on Influence Functions. John Wiley & Sons, 2005,536 p. https://doi.org/10.1002/9781118186435
- Lisitsin D.V., Gavrilov K.V. The use of maximum entropy principle to construct robust estimators under point Bayesian contamination. Part I. Applied Mathematics and Control Sciences, 2024, no. 1, pp. 55–72. (in Russian). https://doi.org/10.15593/2499-9873/2024.1.04
- Lisitsin D.V. Robust Methods for Parameters Estimating of Statistical Models. Novosibirsk, NSTU Publ., 2013, 76 p. (in Russian)
- Lisitsin D.V., Gavrilov K.V. The use of maximum entropy principle to construct robust estimators under point Bayesian contamination. Part II. Applied Mathematics and Control Sciences, 2024, no. 2, pp. 18–33. (in Russian). https://doi.org/10.15593/2499-9873/2024.2.02
- Lisitsin D.V., Gavrilov K.V. On the properties of M-estimators optimizing weighted L2-norm of the influence function. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2024, vol. 24, no. 2, pp. 267–275. (in Russian). https://doi.org/10.17586/2226-1494-2024-24-2-267-275
- Lisitsin D.V. Robust estimation of model parameters in presence of multivariate nonhomogeneous incomplete data. Science Bulletin of the Novosibirsk State Technical University, 2013, vol. 50, no. 1, pp. 17–30. (in Russian)
- Lisitsin D.V., Gavrilov K.V. On stable estimation of models parameters in presence of asymmetric data contamination. Science Bulletin of the Novosibirsk State Technical University, 2008, vol. 30, no. 1, pp. 33–41. (in Russian)
- Shevlyakov G.L., Oja H. Robust Correlation: Theory and Applications. John Wiley & Sons, 2016,352 p. https://doi.org/10.1002/9781119264507
- Gavrilov K.V., Veretel'nikova E.L. On one way to choose a compromise in a family of conditionally optimal estimators. Tomsk State University Journal of Control and Computer Science, 2024, no. 67, pp. 60–68. (in Russian). https://doi.org/10.17223/19988605/67/7
- Coretto P., Hennig C. Robust improper maximum likelihood: tuning, computation, and a comparison with other methods for robust Gaussian clustering. Journal of the American Statistical Association, 2016, vol. 111, no. 516, pp. 1648–1659. https://doi.org/10.1080/01621459.2015.1100996
- Rieder H., Kohl M., Ruckdeschel P. The cost of not knowing the radius. Statistical Methods and Applications, 2008, vol. 17,no. 1, pp. 13–40. https://doi.org/10.1007/s10260-007-0047-7
- Smolyak S.A., Titarenko B.P. Stable Estimation Methods: Statistical Processing of Heterogeneous Aggregates. Moscow, Statistika Publ., 1980, 210 p. (in Russian)
- Lisitsin D.V., Gavrilov K.V. Maximin problem of parameter estimation in conditions of point Bayesian contamination. Tomsk State University Journal of Control and Computer Science, 2023, no. 62, pp. 56–64. (in Russian). https://doi.org/10.17223/19988605/62/6
- Lisitsin D.V., Gavrilov K.V. Estimation of distribution parameters of a bounded random variable robust to bound disturbance. Science Bulletin of the Novosibirsk State Technical University, 2016, vol. 63, no. 2, pp. 70–89. (in Russian). https://doi.org/10.17212/1814-1196-2016-2-70-89
- Shevlyakov G., Morgenthaler S., Shurygin A. Redescending M-estimators. Journal of Statistical Planning and Inference, 2008, vol. 138, no. 10, pp. 2906–2917. https://doi.org/10.1016/j.jspi.2007.11.008
- Shurygin A.M. New approach to optimization of stable estimation. Proc. of the First US/Japan Conference on the Frontiers of Statistical Modeling: An Informational Approach, 1994,vol. 3,pp. 315–340. https://doi.org/10.1007/978-94-011-0854-6_15
- Nogin V.D. Decision Making in Multicriteria Environment: a Quantitative Approach. Moscow, Fizmatlit Publ., 2004, 176 p. (in Russian)
- Podinovski V.V. Ideas and Methods of Criteria Importance Theory in Multicriteria Decision-Making Problems. Moscow, Nauka Publ., 2019, 103 p. (in Russian)
- Li F., Lai L., Cui Sh. Machine Learning Algorithms: Adversarial Robustness in Signal Processing. Springer, 2022,104 p. https://doi.org/10.1007/978-3-031-16375-3
- Großhans M., Sawade C., Brückner M., Scheffer T. Bayesian games for adversarial regression problems. Proc. of the 30th International Conference on Machine Learning. International Conference on Machine Learning, 2013, vol. 28. pp. 55–63. https://doi.org/10.5555/3042817.3042943
- Aivazyan S.A., Yenyukov I.S., Meshalkin, L.D. Applied Statistics: Bases of Modelling and Initial Data Processing. Moscow, Finansy i statistika, Publ., 1983, 471 p. (in Russian)
- Haykin S. Neural Networks: A Comprehensive Foundation. Prentice Hall, 1998, 842 p.
