The increasing demand for artificial intelligence (AI) workloads across diverse computing environments has driven the need for more efficient data management strategies. Traditional cloud-based architectures struggle to handle the sheer volume and velocity of AI-driven data, leading to inefficiencies in storage, computation, and data movement. This paper explores the integration of active storage systems within the computing continuum to optimize AI workload distribution. By embedding computation directly into storage architectures, active storage is able to reduce data transfer overhead, enhancing performance and improving resource utilization. Other existing frameworks and architectures offer mechanisms to distribute certain AI processes across distributed environments; however, they lack the flexibility and adaptability that the continuum requires, both regarding the heterogeneity of devices and the rapid-changing algorithms and models being used by domain experts and researchers. This article proposes a software architecture aimed at seamlessly distributing AI workloads across the computing continuum, and presents its implementation using mainstream Python libraries and dataClay, an active storage platform. The evaluation shows the benefits and trade-offs regarding memory consumption, storage requirements, training times, and execution efficiency across different devices. Experimental results demonstrate that the process of offloading workloads through active storage significantly improves memory efficiency and training speeds while maintaining accuracy. Our findings highlight the potential of active storage to revolutionize AI workload management, making distributed AI deployments more scalable and resource-efficient with a very low entry barrier for domain experts and application developers.

翻译：人工智能（AI）工作负载在多样化计算环境中的需求日益增长，推动了对更高效数据管理策略的需求。传统的基于云的架构难以应对AI驱动数据的庞大规模与高速生成，导致存储、计算和数据移动方面的效率低下。本文探讨了在计算连续体中集成主动存储系统以优化AI工作负载分布的方法。通过将计算直接嵌入存储架构，主动存储能够减少数据传输开销，提升性能并改善资源利用率。现有其他框架和架构提供了在分布式环境中分配特定AI流程的机制；然而，它们在设备异构性以及领域专家和研究人员使用的快速演变的算法与模型方面，缺乏计算连续体所需的灵活性和适应性。本文提出了一种旨在无缝分布AI工作负载于计算连续体的软件架构，并展示了其使用主流Python库和主动存储平台dataClay的实现。评估展示了在不同设备上关于内存消耗、存储需求、训练时间和执行效率的收益与权衡。实验结果表明，通过主动存储卸载工作负载的过程显著提高了内存效率和训练速度，同时保持了准确性。我们的发现突显了主动存储革新AI工作负载管理的潜力，使分布式AI部署更具可扩展性和资源效率，并为领域专家和应用开发者提供了极低的入门门槛。