ERIC Number: ED586042
Record Type: Non-Journal
Publication Date: 2017-Aug
Pages: 16
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-
EISSN: N/A
Importance Sampling for Fair Policy Selection
Doroudi, Shayan; Thomas, Philip S.; Brunskill, Emma
Grantee Submission, Paper presented at the Conference on Uncertainty in Artificial Intelligence (33rd, Sydney, Australia, Aug 11-15, 2017)
We consider the problem of off-policy policy selection in reinforcement learning: using historical data generated from running one policy to compare two or more policies. We show that approaches based on importance sampling can be "unfair"--they can select the worse of two policies more often than not. We give two examples where the unfairness of importance sampling could be practically concerning. We then present sufficient conditions to theoretically guarantee fairness and a related notion of safety. Finally, we provide a practical importance sampling-based estimator to help mitigate one of the systematic sources of unfairness resulting from using importance sampling for policy selection. [This paper was published in "Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence" (33rd, Sydney, Australia, August 11-15, 2017).]
Descriptors: Sampling, Policy Formation, Policy Analysis, Reinforcement, Mathematics, Computation, Evaluation
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED); National Science Foundation (NSF)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: R305A130215; R305B150008