The Incredible AI Agents

I was blessed to have created The Incredible PyTorch 8 years ago.

Created this to curate resources across the whole lifecycle of building, deploying, evaluating, monitoring AI Agents.

Table of Contents

• Planning & Design
• Frameworks, Libraries & Development Tools
• Deployment & Infrastructure
• Evaluation & Testing
• Monitoring & Observability
• Specialized Agents

Planning & Design

Planning & Reasoning

• ReAct - Reasoning and Acting with Language Models
• Tree of Thoughts - Deliberate problem solving with LLMs
• Plan-and-Solve - Zero-shot reasoning prompting

Frameworks, Libraries & Development Tools

Core Agent Frameworks

• Google Agent Development Kit (ADK) - Google's toolkit for building AI agents
• OpenAI Agents Python - OpenAI's lightweight multi-agent workflow framework
• LangChain - Building applications with LLMs through composability
• LangGraph - Build resilient language agents as graphs
• LlamaIndex - Data framework for LLM applications
• CrewAI - Framework for orchestrating role-playing, autonomous AI agents
• AutoGen - Multi-agent conversation framework
• Haystack - End-to-end NLP framework
• Semantic Kernel - SDK for integrating AI services
• Agno - Full-stack framework for building Multi-Agent Systems with memory
• Agent Zero - Agent Zero AI framework
• Atomic Agents - Building AI agents, atomically
• Letta (MemGPT) - Stateful agents framework with memory and reasoning
• MetaGPT - Multi-Agent Framework for natural language programming
• Camel AI - First and best multi-agent framework
• Swarms - Enterprise-Grade Production-Ready Multi-Agent Orchestration
• PraisonAI - Production-ready Multi AI Agents framework
• PocketFlow - 100-line LLM framework for agentic workflows
• Julep - Deploy serverless AI workflows at scale
• BeeAI Framework - Production-ready AI agents in Python and TypeScript
• AgentUniverse - LLM multi-agent framework
• LazyLLM - Easiest way for building multi-agent LLM applications
• LLMStack - No-code multi-agent framework
• MS-Agent - ModelScope agent framework
• Lagent - Lightweight framework for building LLM-based agents
• AutoAgent - Fully-automated and zero-code LLM agent framework
• uAgents - Fast and lightweight framework for decentralized agents
• II-Agent - Open-source framework to build and deploy intelligent agents
• Deer Flow - ByteDance's community-driven Deep Research framework
• Cheshire Cat AI - AI agent microservice
• LLMware - Unified framework for building enterprise RAG pipelines
• Transformers Agents - Hugging Face agents
• Guidance - Programming framework for LLMs
• DSPy - Programming foundation models
• Langflow - Powerful tool for building and deploying AI-powered agents and workflows
• DB-GPT - AI Native Data App Development framework with AWEL and Agents
• CodeFuse ChatBot - Multi-Agent Framework for software development lifecycle
• Mesa - Agent-based modeling framework
• PettingZoo - Multi-agent reinforcement learning
• AIWaves Agents - Data-centric, self-evolving autonomous language agents
• AutoSwarm - Swarm intelligence for AI agents
• AWS Agent Squad - Flexible and powerful framework for managing multiple AI agents and handling complex conversations
• BotSharp - AI Multi-Agent Framework in .NET
• Langroid - Harness LLMs with Multi-Agent Programming
• Rowboat - AI-powered multi-agent builder
• Sage - Multi-Agent System Framework For Complex Tasks
• Spring AI Alibaba - Agentic AI Framework for Java Developers
• TEN Framework - Open-source framework for conversational voice AI agents
• Upsonic - The most reliable AI agent framework that supports MCP
• VoltAgent - Open Source TypeScript AI Agent Framework

Development Tools & Utilities

• LiteLLM - Use any LLM as a drop in replacement
• Instructor - Structured outputs from LLMs
• Pydantic AI - Agent framework built on Pydantic

Tool Integration

• LangChain Tools - Pre-built tools for agents
• OpenAI Function Calling - Native tool integration
• Toolformer - Language models that can use tools

Memory & Context Management

• MemGPT - Teaching LLMs memory management
• LangChain Memory - Memory components
• Zep - Long-term memory for AI assistants
• Mem0 - Universal memory layer for AI Agents
• Graphiti - Build Real-Time Knowledge Graphs for AI Agents

Voice & Realtime Agents

• LiveKit Agents - Powerful framework for building realtime voice AI agents

Browser & Web Automation

• AppAgent - Multimodal Agents as Smartphone Users
• Browser Use - Make websites accessible for AI agents
• Browser Use Web UI - Run AI Agent in your browser
• LaVague - Large Action Model framework to develop AI Web Agents
• Magnitude - Open-source, vision-first browser agent
• NanoBrowser - Open-Source Chrome extension for AI-powered web automation
• Notte - Reliable browser using agents

Deployment & Infrastructure

Serving & APIs

• vLLM - High-throughput LLM serving
• Ollama - Run LLMs locally
• LocalAI - Self-hosted OpenAI alternative
• FastAPI for AI - Modern API framework

Orchestration

• Airflow - Workflow orchestration
• Prefect - Modern workflow orchestration
• Temporal - Durable execution platform

Scaling & Infrastructure

• Ray - Distributed computing for AI
• Kubernetes - Container orchestration
• Docker - Containerization

Evaluation & Testing

Agent Evaluation Frameworks

• AgentBench - Evaluating LLMs as agents
• SWE-bench - Software engineering evaluation
• HumanEval - Code generation evaluation

Testing Tools

• LangSmith - Testing and evaluation platform
• Promptfoo - Test and evaluate LLM outputs
• DeepEval - LLM evaluation framework

Benchmarks & Datasets

• HELM - Holistic evaluation of language models
• BigBench - Beyond the imitation game benchmark
• MMLU - Massive multitask language understanding

Monitoring & Observability

Agent Observability & Monitoring

• RagaAI Catalyst - Agent AI Observability, Monitoring and Evaluation Framework
• OpenLit - OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations
• HolmesGPT - 24/7 On-Call AI Agent for alerts and investigations

LLM Observability

• LangFuse - Open source LLM engineering platform
• Phoenix - ML observability in a notebook
• Weights & Biases - Experiment tracking and monitoring

Performance Monitoring

• Prometheus - Monitoring and alerting toolkit
• Grafana - Observability and data visualization
• New Relic AI Monitoring - AI application monitoring

Logging & Tracing

• OpenTelemetry - Observability framework
• Jaeger - Distributed tracing platform

Specialized Agents

Financial & Trading Agents

• TradingAgents - Multi-Agents LLM Financial Trading Framework

Research & Analysis Agents

• Airweave - Airweave lets agents search any app
• Company Research Agent - Multi-agent company research tool powered by LangGraph
• MindSearch - LLM-based Multi-agent Framework of Web Search Engine (like Perplexity.ai Pro and SearchGPT)

Creative & Media Agents

• FilmAgent - Multi-Agent Framework for End-to-End Film Automation

Educational & Tutorial Agents

• AgentVerse - Designed to facilitate the deployment of multiple LLM-based agents in various applications
• AI Agents Masterclass - Follow along with AI Agents Masterclass videos
• Awesome LLM Apps - Collection of awesome LLM apps with AI Agents and RAG
• ChatDev - Communicative agents for software development
• GPTeam - Open-source multi-agent simulation
• OpenSwarm - Educational framework for multi-agent orchestration
• Ottomator Agents - Open source AI Agents hosted on oTTomator platform
• SWE Agent - AI-powered software engineering multi-agent system with researcher and developer agents

Contributing

Feel free to submit pull requests to add more resources! Please ensure:

• Resources are actively maintained
• Include brief descriptions
• Organize by appropriate lifecycle phase
• Add relevant links and documentation