Metric elicitation is a recent framework for eliciting performance metrics that best reflect implicit user preferences based on the application and context. However, available elicitation strategies have been limited to linear (or quasi-linear) functions of predictive rates, which can be practically restrictive for many domains including fairness. This paper develops a strategy for eliciting more flexible multiclass metrics defined by quadratic functions of rates, designed to reflect human preferences better. We show its application in eliciting quadratic violation-based group-fair metrics. Our strategy requires only relative preference feedback, and that too of near-optimal amount, and is robust to feedback noise. We further extend this strategy to eliciting polynomial metrics -- thus broadening the use cases for metric elicitation.

翻译：计量引引是最近一个吸引性能衡量尺度的框架,最能反映基于应用和背景的隐性用户偏好;然而,现有的引引战略仅限于预测率的线性(或准线性)功能,这种功能对包括公平在内的许多领域实际上具有限制性。本文件制定了一个战略,以吸引更灵活的多级计量尺度,由比率的二次函数界定,旨在更好地反映人类的偏好。我们展示了它用于吸引四级侵权群体公平衡量尺度的应用。我们的战略只需要相对的偏好反馈,而且其数量也接近最佳,而且对反馈噪音十分有力。我们进一步扩展这一战略,以获取多级计量尺度,从而扩大衡量的运用范围。