Unstructured data, in the form of text, images, video, and audio, is produced at exponentially higher rates. In tandem, machine learning (ML) methods have become increasingly powerful at analyzing unstructured data. Modern ML methods can now detect objects in images, understand actions in videos, and even classify complex legal texts based on legal intent. Combined, these trends make it increasingly feasible for analysts and researchers to automatically understand the "real world." However, there are major challenges in deploying these techniques: 1) executing queries efficiently given the expense of ML methods, 2) expressing queries over bespoke forms of data, and 3) handling errors in ML methods. In this monograph, we discuss challenges and advances in data management systems for unstructured data using ML, with a particular focus on video analytics. Using ML to answer queries introduces new challenges.First, even turning user intent into queries can be challenging: it is not obvious how to express a query of the form "select instances of cars turning left." Second, ML models can be orders of magnitude more expensive compared to processing traditional structured data. Third, ML models and the methods to accelerate analytics with ML models can be error-prone. Recent work in the data management community has aimed to address all of these challenges. Users can now express queries via user-defined functions, opaquely through standard structured schemas, and even by providing examples. Given a query, recent work focuses on optimizing queries by approximating expensive "gold" methods with varying levels of guarantees. Finally, to handle errors in ML models, recent work has focused on applying outlier and drift detection to data analytics with ML.

翻译：以文本、图像、视频和音频形式存在的非结构化数据正以指数级速度增长。与此同时，机器学习（ML）方法在分析非结构化数据方面变得日益强大。现代ML方法现已能够检测图像中的物体、理解视频中的动作，甚至能基于法律意图对复杂的法律文本进行分类。这些趋势相结合，使得分析师和研究人员自动理解“现实世界”变得越来越可行。然而，部署这些技术仍面临重大挑战：1）鉴于ML方法的高昂成本，如何高效执行查询；2）在定制化数据形式上表达查询；3）处理ML方法中的误差。本专著讨论了利用ML进行非结构化数据管理系统的挑战与进展，特别聚焦于视频分析领域。使用ML回答查询带来了新的挑战。首先，将用户意图转化为查询本身就可能具有挑战性：例如如何表达“选择左转汽车实例”这类查询并不直观。其次，与传统结构化数据处理相比，ML模型的计算成本可能高出数个数量级。第三，ML模型及其加速分析的方法可能存在误差。数据管理领域的最新研究致力于解决所有这些挑战。用户现在可以通过用户定义函数、基于标准结构化模式的不透明方式，甚至通过提供示例来表达查询。针对给定查询，近期研究侧重于通过以不同保证级别近似昂贵的“黄金”方法来优化查询。最后，为处理ML模型中的误差，最新研究聚焦于将异常值和漂移检测技术应用于基于ML的数据分析。