Agnostic Online Learning

This work describes several models of non-realizable data and derive upper and lower bounds on the achievable regret, and extends Littlestone's theory to include margin-based hypothesis classes, in which the prediction of each hypothesis is accompanied by a confidence value.