Thompson sampling and other Bayesian sequential decision-making algorithms are among the most popular approaches to tackle explore/exploit trade-offs in (contextual) bandits. The choice of prior in these algorithms offers flexibility to encode domain knowledge but can also lead to poor performance when misspecified. In this paper, we demonstrate that performance degrades gracefully with misspecification. We prove that the expected reward accrued by Thompson sampling (TS) with a misspecified prior differs by at most $\tilde{\mathcal{O}}(H^2 \epsilon)$ from TS with a well specified prior, where $\epsilon$ is the total-variation distance between priors and $H$ is the learning horizon. Our bound does not require the prior to have any parametric form. For priors with bounded support, our bound is independent of the cardinality or structure of the action space, and we show that it is tight up to universal constants in the worst case. Building on our sensitivity analysis, we establish generic PAC guarantees for algorithms in the recently studied Bayesian meta-learning setting and derive corollaries for various families of priors. Our results generalize along two axes: (1) they apply to a broader family of Bayesian decision-making algorithms, including a Monte-Carlo implementation of the knowledge gradient algorithm (KG), and (2) they apply to Bayesian POMDPs, the most general Bayesian decision-making setting, encompassing contextual bandits as a special case. Through numerical simulations, we illustrate how prior misspecification and the deployment of one-step look-ahead (as in KG) can impact the convergence of meta-learning in multi-armed and contextual bandits with structured and correlated priors.

翻译：汤普森抽样和其他巴伊西亚顺序决策算法是最受欢迎的方法之一,用来解决(通俗)土匪的探索/利用交易交易。这些算法中先行的选择为编码域知识提供了灵活性,但当错误描述时也可能导致业绩不佳。在本文中,我们证明业绩优于特异性。我们证明,Thompson抽样(TS)的预期报酬与先前错误描述的不同之处最多为$tilde_mathcal{O ⁇ (H%2\epsilon),与以前非常明确的TS(TS)相比,是最受欢迎的方法之一。 在此之前, $\epsilon 美元是前行和 $H$ 之间的完全变异距离是学习的视野。 我们的界限并不要求有任何参数格式。 对于先前有约束的支持, 我们的界限是独立于行动空间的根基点或结构,我们表明,它与最坏的常态相近于通用常态。 在我们的感知性分析中,我们为最近研究的Bayes-dele-delexal 的算算法, 在前,我们一般的直径直径直系的逻辑上, 直径直径直系, 直系的直系的直径直系, 直系的直系的直系的直系的直系, 直系, 直系的直系的直系的直系, 直系, 直系的直系的直系的直系, 直系, 直系, 直系的直系直系直系直系直系直系直系直系直系直系直系直系直系。