The rise of large language models (LLMs) has sparked a surge of interest in agents, leading to the rapid growth of agent frameworks. Agent frameworks are software toolkits and libraries that provide standardized components, abstractions, and orchestration mechanisms to simplify agent development. Despite widespread use of agent frameworks, their practical applications and how they influence the agent development process remain underexplored. Different agent frameworks encounter similar problems during use, indicating that these recurring issues deserve greater attention and call for further improvements in agent framework design. Meanwhile, as the number of agent frameworks continues to grow and evolve, more than 80% of developers report difficulties in identifying the frameworks that best meet their specific development requirements. In this paper, we conduct the first empirical study of LLM-based agent frameworks, exploring real-world experiences of developers in building AI agents. To compare how well the agent frameworks meet developer needs, we further collect developer discussions for the ten previously identified agent frameworks, resulting in a total of 11,910 discussions. Finally, by analyzing these discussions, we compare the frameworks across five dimensions: development efficiency, functional abstraction, learning cost, performance optimization, and maintainability, which refers to how easily developers can update and extend both the framework itself and the agents built upon it over time. Our comparative analysis reveals significant differences among frameworks in how they meet the needs of agent developers. Overall, we provide a set of findings and implications for the LLM-driven AI agent framework ecosystem and offer insights for the design of future LLM-based agent frameworks and agent developers.

翻译：大型语言模型（LLMs）的兴起引发了智能体研究的热潮，推动了智能体框架的快速发展。智能体框架作为软件工具包和库，提供了标准化组件、抽象层和编排机制，以简化智能体的开发过程。尽管智能体框架已得到广泛应用，但其实际应用场景及其对智能体开发流程的影响仍缺乏深入探索。不同智能体框架在使用过程中常遇到相似问题，表明这些反复出现的问题值得更多关注，并呼吁对智能体框架设计进行进一步改进。同时，随着智能体框架数量持续增长和演进，超过80%的开发者表示难以确定最适合其特定开发需求的框架。本文首次对基于LLM的智能体框架开展实证研究，探究开发者在构建AI智能体过程中的实际经验。为比较各智能体框架满足开发者需求的程度，我们进一步收集了十个已识别智能体框架的开发者讨论，共计11,910条讨论记录。最后，通过分析这些讨论，我们从五个维度对框架进行比较：开发效率、功能抽象性、学习成本、性能优化以及可维护性——后者指开发者随时间推移更新和扩展框架本身及基于其构建的智能体的难易程度。我们的对比分析揭示了各框架在满足智能体开发者需求方面存在的显著差异。总体而言，本研究为LLM驱动的AI智能体框架生态系统提供了一系列发现与启示，并为未来基于LLM的智能体框架设计及智能体开发者提供了参考见解。