AUTOMATING RESEARCH INTELLIGENCE: ADVANCED-AI-GENERATED VS MANUALLY DESIGNED PIPELINES
Received: 2026-05-29
Published: 2026-06-06
Abstract
This paper presents an empirical comparison between manually designed research intelligence pipelines and those generated by advanced agentic AI frameworks (AutoGen, AutoGPT, OpenAgents, OpenHands, Claude, and Gemini). We first outline two independent manual pipelines developed for narrow technical fields, achieving stable multi-source ingestion, semantic filtering, and document clustering. We then evaluate the capability of AI agents to replicate this workflow. The results show that AI-generated code suffers from API misuse, domain-specific errors, and a lack of robustness features like caching and deduplication. The expert debugging time for AI-generated pipelines (35 to 150+ hours) far exceeded the time required to build the manual baseline from scratch (~60 hours). We conclude that current AI agents cannot build complex pipelines autonomously, and we propose a hybrid architecture combining a deterministic backbone with targeted LLM integration.
Keywords
List of references
-
Bornmann, L., Haunschild, R., and Mutz, R. (2021). Growth rates of modern science: a latent piecewise growth curve approach to model publication numbers from established and new literature databases. Humanities and Social Sciences Communications, 8(1), 224. https://doi.org/10.1057/s41599-021-00903-w
-
Landhuis, E. (2016). Scientific literature: Information overload. Nature, 535(7612), 457–458. https://doi.org/10.1038/535457a
-
Wu, Q., Bansal, G., Zhang, J., et al. (2024). AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation. Proceedings of the Conference on Language Modeling (COLM 2024).
-
Richards, T. B. (2023). AutoGPT: An open-source autonomous AI agent. GitHub Repository. https://github.com/Significant-Gravitas/AutoGPT
-
Xie, T., Zhou, F., Cheng, Z., et al. (2023). OpenAgents: An Open Platform for Language Agents in the Wild. arXiv preprint arXiv:2310.10634.
-
Wang, X., Li, B., Song, Y., et al. (2024). OpenHands: An Open Platform for AI Software Developers as Generalist Agents. arXiv preprint arXiv:2407.16741.
-
Hong, S., Zhuge, M., Chen, J., et al. (2023). MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. arXiv preprint arXiv:2308.00352.
-
Teece, D. J., Pisano, G., and Shuen, A. (1997). Dynamic Capabilities and Strategic Management. Strategic Management Journal, 18(7), 509–533.
-
Brynjolfsson, E., Li, D., and Raymond, L. R. (2023). Generative AI at Work. Quarterly Journal of Economics, 138(4), 1875–1925. https://doi.org/10.1093/qje/qjad027
-
Chen, Z., Chen, S., Ning, Y., et al. (2024). ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery. arXiv preprint arXiv:2410.05080.
About the Authors
License
This work is licensed under a Creative Commons Attribution 4.0 International License.