在本课程中，我们将学习机器学习、统计学和计算机科学中的现代技术，以估计黑匣子预测的不确定性。这包括共形预测、校准和多重校准、结果不可区分性，以及在没有任何分布假设的情况下产生最坏情况经验覆盖保证的最新技术。在此过程中，我们将探索这些技术在领域适应问题上的应用，以及它们在解决下游优化问题、经济和机制设计以及算法公平性方面的应用。 目录内容：

Conformal Prediction * Exchangeable Data: Full and Split Conformal Prediction * Distribution Shift and Time Series Data * Adaptive/Adversarial Conformal Prediction * Threshold Calibrated Multivalid Conformal Prediction

(Multi)Calibration * Proper Scoring Rules, Calibration, and Regret * Algorithms for offline (batch) multicalibration * Algorithms for online (adversarial) multicalibration * Moment Multicalibration * Applications of Multicalibration

Downstream Unconstrained Optimization (Omnipredictors) * Downstream Constrained Optimization * Proxies for Downstream Measurement * Distribution Shift

Tests and Predictors * Ignorantly Passing Tests: Possibility and Hardness * Outcome Indistinguishability

Other Topics * Smooth Calibration * Applications of Calibration in Mechanism Design * The Reference Class Problem