Marketplaces for machine learning (ML) models are emerging as a way for organizations to monetize models. They allow model owners to retain control over hosted models by using cloud resources to execute ML inference requests for a fee, preserving model confidentiality. Clients that rely on hosted models require trustworthy inference results, even when models are managed by third parties. While the resilience and robustness of inference results can be improved by combining multiple independent models, such support is unavailable in today's marketplaces. We describe Dropbear, the first ML model marketplace that provides clients with strong integrity guarantees by combining results from multiple models in a trustworthy fashion. Dropbear replicates inference computation across a model group, which consists of multiple cloud-based GPU nodes belonging to different model owners. Clients receive inference certificates that prove agreement using a Byzantine consensus protocol, even under model heterogeneity and concurrent model updates. To improve performance, Dropbear batches inference and consensus operations separately: it first performs the inference computation across a model group, before ordering requests and model updates. Despite its strong integrity guarantees, Dropbear's performance matches that of state-of-the-art ML inference systems: deployed across 3 cloud sites, it handles 800 requests/s with ImageNet models.

翻译：机器学习模型的市场正在形成,这是各组织将模型货币化的一种方式。它们允许模型拥有者通过使用云源资源执行ML推断收费请求,从而保持对主机模型的控制,维护模型保密性。依赖主机模型的客户需要可靠的推断结果,即使模型由第三方管理。虽然通过结合多种独立模型可以提高推论结果的弹性和稳健性,但在当今的市场中却无法获得这种支持。我们分别描述Dowbau,这是第一个ML模型市场,它通过以可靠的方式将多个模型的结果结合起来,为客户提供强有力的完整性保障。Dowbauth重复了一个模型组的推论计算,该模型由属于不同模型所有者的多个基于云的GPU节点组成。客户获得推论证书,证明他们同意使用Byzantine共识协议,即使根据模型的异质性和同时的模型更新,也能够提高推论结果。为了改进性能,Dowbears批量和共识操作:它首先对一个模型组进行推算,然后订购和模型更新。尽管它具有很强的完整性保证,但Drodubbefefefer s proper s