Acceleration of stochastic approximation by averaging
Acceleration of stochastic approximation by averaging
Boris Polyak,A. Juditsky
1992 · DOI: 10.1137/0330046
2,100 Citations
TLDR
Convergence with probability one is proved for a variety of classical optimization and identification problems and it is demonstrated for these problems that the proposed algorithm achieves the highest possible rate of convergence.
Abstract
A new recursive algorithm of stochastic approximation type with the averaging of trajectories is investigated. Convergence with probability one is proved for a variety of classical optimization and identification problems. It is also demonstrated for these problems that the proposed algorithm achieves the highest possible rate of convergence.