<div>
	On the properties of <em>M</em>-estimators optimizing weighted L<sub>2</sub>-norm of the influence function</div>

Lisitsin Daniil V. , Gavrilov Konstantin V.

2024 , VOLUME 24, NUMBER 2 ( march-april )

ISSN 2226-1494 (print), ISSN 2500-0373 (online)

Publications

Editor-in-Chief

Nikiforov
Vladimir O.
D.Sc., Prof.

Partners

doi: 10.17586/2226-1494-2024-24-2-267-275

On the properties of M-estimators optimizing weighted L₂-norm of the influence function

D. V. Lisitsin, K. V. Gavrilov

Read the full article

Article in Russian

For citation:

Lisitsin D.V., Gavrilov K.V. On the properties of M-estimators optimizing weighted L₂-norm of the influence function. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2024, vol. 24, no. 2, pp. 267–275 (in Russian). doi: 10.17586/2226-1494-2024-24-2-267-275

Abstract

The work develops the theory of stable M-estimators belonging to the class of redescending estimators, having the property of resistance to asymmetric contamination. Many well-known redescending estimators can be obtained within the framework of the locally stable approach of A.M. Shurygin, based on the analysis of the estimator instability functional (L₂-norm of the influence function), or his approach based on the model of a series of samples with random point contamination (point Bayesian contamination model). These approaches are convenient for constructing various stable M-estimators and, in comparison with classical robust procedures, provide wider opportunities. The family of conditionally optimal estimators proposed by A.M. Shurygin within the framework of the first of the listed approaches can be defined as optimizing the asymptotic dispersion under a constraint on the value of instability. The corresponding problem can be represented in the form of optimization of the weighted L₂-norm of the influence function. The second approach considers a specially formed nonparametric neighborhood of the model distribution, and it can also be reduced to the analysis of the weighted L₂-norm of the influence function. Thus, this estimation quality criterion is quite general and useful for constructing robust estimators. The theory of estimators that are optimal in terms of weighted L₂-norm of the influence function is currently underdeveloped. Specifically, for the corresponding families of estimators, the question of the uniqueness of family members remains unresolved. The question comes down to studying the convexity (concavity) of the optimized functional depending on the parameter defining the family. In the presented work, an expression is obtained in general form for the derivative with respect to the parameter of the quality functional of the optimal estimator. Inequalities are obtained for the second derivative necessary to establish its convexity (concavity) with respect to the parameter. Corollaries from these results are applied to describe the properties of a conditionally optimal family. The influence functions of a number of conditionally optimal estimators for the shift and scale parameters of the normal model are constructed. The characteristics of these estimators are studied. The stability of most of the considered estimators is shown, which is important for their practical application. The theoretical results obtained can be useful in studying the properties of compromise estimators based on two criteria as well as in studying minimax contamination levels within the framework of A.M. Shurygin’s point Bayesian contamination model. The results of the work can be used in situations of purposed data corruption by an adversary including the problems related to adversarial machine learning.

Keywords: M-estimators, robust statistics, influence function, stable estimates, redescending estimators, conditionally optimal estimators

References

Borovkov A.A. Mathematical Statistics. Amsterdam, Gordon and Breach, 1998, 570 p. https://doi.org/10.1201/9780203749326
Shurygin A.M. Applied stochastics: robustness, estimation, prediction. Moscow, Finansy i statistika Publ., 2000, 224 p. (in Russian)
Huber P., Ronchetti E. Robust Statistics. 2^nd ed. John Wiley & Sons, 2009, 354 p. https://doi.org/10.1002/9780470434697
Hampel F., Ronchetti E., Rousseeuw P., Stahel W. Robust Statistics: The Approach Based on Influence Functions. John Wiley & Sons, 2005, 536 p. https://doi.org/10.1002/9781118186435
Lisitsin D.V., Gavrilov K.V. Maximin problem of parameter estimation in conditions of point Bayesian contamination. Tomsk State University Journal of Control and Computer Science, 2023, no. 62, pp. 56–64. (in Russian). https://doi.org/10.17223/19988605/62/6
Lisitsin D.V., Gavrilov K.V. Estimation of distribution parameters of a bounded random variable robust to bound disturbance. Scientific Bulletin of NSTU, 2016, no. 2(63), pp. 70–89. (in Russian) https://doi.org/10.17212/1814-1196-2016-2-70-89
Lisitsin D.V., Usol'tsev A.G. Minimum gamma-divergence estimation for non-homogeneous data with application to ordered probit model. Applied methods of statistical analysis. Statistical computation and simulation. Proceedings of the International Workshop. Novosibirsk, 18–20 Sept. 2019. Novosibirsk, NSTU, 2019, pp. 227–234.
Lisitsin D.V., Gavrilov K.V. On properties of conditionally optimal estimates. Scientific Bulletin of NSTU, 2015, no. 1(58), pp. 76–93. (in Russian). https://doi.org/10.17212/1814-1196-2015-1-76-93
Lisitsin D.V. Robust estimation of model parameters in presence of multivariate nonhomogeneous incomplete data. Scientific Bulletin of NSTU, 2013, no. 1(50), pp. 17–30. (in Russian)
Smolyak S.A., Titarenko B.P. Stable estimation methods: statistical processing of heterogeneous aggregates. Moscow, Statistika Publ., 1980, 210 p. (in Russian)
Lisitsin D.V., Gavrilov K.V. On stable estimation of models parameters in presence of asymmetric data contamination. Scientific Bulletin of NSTU, 2008, no. 1(30), pp. 33–40. (in Russian)
DasGupta A. Asymptotic Theory of Statistics and Probability. New York, Springer, 2008, 722 p. https://doi.org/10.1007/978-0-387-75971-5
Van der Vaart A.W. Asymptotic Statistics. Cambridge, Cambridge University Press, 1998, 443 p. https://doi.org/10.1017/CBO9780511802256
Shurygin A.M. New approach to optimization of stable estimation. Proc. of the First US/Japan Conference on the Frontiers of Statistical Modeling: An Informational Approach. V. 3. Engineering and Scientific Applications. Springer, Dordrecht, 1994, pp. 315–340. https://doi.org/10.1007/978-94-011-0854-6_15
Shevlyakov G., Morgenthaler S., Shurygin A. Redescending M-estimators. Journal of Statistical Planning and Inference, 2008, vol. 138, no. 10, pp. 2906–2917. https://doi.org/10.1016/J.JSPI.2007.11.008
Shevlyakov G.L., Oja H. Robust Correlation: Theory and Applications. John Wiley & Sons, 2016, 319 p. https://doi.org/10.1002/9781119264507
Gavrilov K.V., Veretel'nikova E.L. On one way to choose a compromise in a family of conditionally optimal estimators. Tomsk State University Journal of Control and Computer Science, 2024, no. 67, in press. (in Russian)
Rieder H., Kohl M., Ruckdeschel P. The cost of not knowing the radius. Statistical Methods and Applications, 2008, vol. 17, no. 1, pp. 13–40. https://doi.org/10.1007/s10260-007-0047-7
Esipov D.A., Buchaev A.Y., Kerimbay A., Puzikova Ya.V., Saidumarov S.K., Sulimenko N.S., Popov I.Yu., Karmanovskiy N.S. Attacks based on malicious perturbations on image processing systems and defense methods against them. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2023, vol. 23, no. 4, pp. 720–733. (in Russian). https://doi.org/10.17586/2226-1494-2023-23-4-720-733

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License

On the properties of M-estimators optimizing weighted L2-norm of the influence function

On the properties of M-estimators optimizing weighted L₂-norm of the influence function