Stateful implementations of OPE algorithms, designed for use in the development of offline RL models

Official repository for "Trajectory World Models for Heterogeneous Environments" (ICML 2025), https://arxiv.org/abs/2502.01366

HOPES: HVAC optimization with Off-Policy Evaluation and Selection

Reinforcement Learning Short Course

Implementation of "A Reinforcement Learning Framework for Dynamic Mediation Analysis" (ICML 2023) in Python.

Robust Offline Reinforcement Learning with Heavy-Tailed Rewards

Open Bandit Pipeline: a python library for bandit algorithms and off-policy evaluation

An index of algorithms for offline reinforcement learning (offline-rl)

SCOPE-RL: A python library for offline reinforcement learning, off-policy evaluation, and selection

Implementation of "Off-Policy Interval Estimation with Confounded Markov Decision Process" (JASA, 2022+)

[NeurIPS 2023] Counterfactual-Augmented Importance Sampling for Semi-Offline Policy Evaluation. https://arxiv.org/abs/2310.17146

(NeurIPS2023) "Future-Dependent Value-Based Off-Policy Evaluation in POMDPs"

(KDD2023) "Off-Policy Evaluation of Ranking Policies under Diverse User Behavior"

[JASA] Off-Policy Interval Estimation withConfounded Markov Decision Process

(WSDM2022 Best Paper Award Runner-Up) "Doubly Robust Off-Policy Evaluation for Ranking Policies under the Cascade Behavior Model"

Official implementation for "On the Reuse Bias in Off-Policy Reinforcement Learning" (IJCAI 2023)

Implementations and examples of common offline policy evaluation methods in Python.

Conformal Off-policy Prediction

Representation Learning for OPE

Implementation of "A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processes" (ICML)