Off-Policy Policy Search

This paper provides a method for using importance sampling to allow any wellbehaved directed exploration policy to be allowed during learning, and shows both theoretically and experimentally that using this method can achieve dramatic performance improvements.