Multi-armed bandits with episode context

The PUCB algorithm is presented, using a predictor that places weight Mi > 0 on arm i with weights summing to 1, which achieves regret $O\left(\frac{1}{M_{\ast}}\sqrt{n\log(n)}\right)$ where M ∗  is the weight on the optimal arm.