Thompson Sampling for Linearly Constrained Bandits

Overview

This repository contains the code for running the simulations and generating the plots reported in the following paper:

Saxena V., Gonzalez J.E., Jaldén J., Thompson Sampling for Linearly Constrained Bandits, AISTATS 2020, Palmerno, Italy.

The code simulates two algorithms: LinConTS and LinCon-KL-UCB. Details and pseudocode of the algorithms are available in the paper. Two real-world datasets are considered, which are included under the datasets/ directory and are also available for download here:

1. Coupon-purchase
2. edX-course

Dependencies

The code runs on Jupyter Notebook running a Python3 kernel. Further, the code depeds on the following libraries:

1. CVXOPT for solving a linear program in each step of the algorithm.
2. Ray for distributing parallel runs over multiple cores within a compute cluster.

Running the Experiments

The experiments can be run by executing each cell in the 'LinConTS_coupon' and 'LinConTS-edX' notebooks. The results are stored to the disk as Numpy arrays.

Plotting

The results for each of the simulated datasets are plotted with the 'LinConTS_plots' notebook. Make sure to correctly specify the location of the result dataset (as a Numpy array) in the appropriate cell.