NVIDIA/NeMo-Agent-Toolkit

NVIDIA/NeMo-Agent-Toolkit

Description: The NVIDIA NeMo Agent toolkit is an open-source library for efficiently connecting and optimizing teams of AI agents.

Language: Python

License: Apache-2.0

Stars: 2401

Forks: 673

Open issues: 40

Created: 2025-03-06T17:23:52Z

Pushed: 2026-06-11T00:04:46Z

Default branch: develop

Fork: no

Archived: no

README:


NVIDIA NeMo Agent Toolkit

 

🔥 New Features

• [AI Coding Agent Skills:](./AGENTS.md) Use focused NeMo Agent Toolkit skills to give coding agents task-specific guidance for building, evaluating, optimizing, and observing workflows.
• [Dynamo Runtime Intelligence:](./examples/dynamo_integration/latency_sensitivity_demo/README.md) Automatically infer per-request latency sensitivity from agent profiles and apply runtime hints for cache control, load-aware routing, and priority-aware serving.
• **Agent Performance Primitives (APP):** Introduce framework-agnostic performance primitives that accelerate graph-based agent frameworks such as LangChain, CrewAI, and Agno with parallel execution, speculative branching, and node-level priority routing.
• [Public Plugin API for Third-Party Tools:](./docs/source/extend/third-party-plugins.md) Build provider-managed integrations against the public NeMo Agent Toolkit plugin API instead of framework-specific internal packages. Thank you to the maintainers of externally managed plugins including NeMo-Agent-Toolkit-Tavily, NeMo-Agent-Toolkit-Redis, and NeMo-Agent-Toolkit-ATR. This new workflow is intended to help third-party package maintainers bring their integrations into NeMo Agent Toolkit workflows more quickly. Note that these third-party packages are managed outside the NeMo Agent Toolkit repository and may release, change, or run CI on schedules independent of NeMo Agent Toolkit.
• [LangSmith Native Integration:](./docs/source/run-workflows/observe/observe-workflow-with-langsmith.md) Observe end-to-end agent execution with native LangSmith tracing, run evaluation experiments, compare outcomes, and manage prompt versions across development and production workflows.
• [FastMCP Workflow Publishing:](./docs/source/run-workflows/fastmcp-server.md) Publish NeMo Agent Toolkit workflows as MCP servers using the FastMCP server runtime to simplify MCP-native deployment and integration.
• Migration notice: 1.5.0 simplifies package installation and dependency management. See the [Migration Guide](./docs/source/resources/migration-guide.md#v150).

✨ Key Features

• 🛠️ Building Agents: Accelerate your agent development with tools that make it easier to get your agent into production.
• 🧩 [Framework Agnostic:](./docs/source/components/integrations/frameworks.md) Work side-by-side with agentic frameworks to add the instrumentation necessary for observing, profiling, and optimizing your agents. Use the toolkit with popular frameworks such as LangChain, LlamaIndex, CrewAI, Microsoft Semantic Kernel, and Google ADK, as well as custom enterprise agentic frameworks and simple Python agents.
• 🔁 [Reusability:](./docs/source/components/sharing-components.md) Build components once and use them multiple times to maximize the value from development effort.
• ⚡ [Customization:](docs/source/get-started/tutorials/customize-a-workflow.md) Start with a pre-built agent, tool, or workflow, and customize it to your needs.
• 💬 [Built-In User Interface:](./docs/source/run-workflows/launching-ui.md) Use the NeMo Agent Toolkit UI chat interface to interact with your agents, visualize output, and debug workflows.
• 📈 Agent Insights: Utilize NeMo Agent Toolkit instrumentation to better understand how your agents function at runtime.
• 📊 [Profiling:](./docs/source/improve-workflows/profiler.md) Profile entire workflows from the agent level all the way down to individual tokens to identify bottlenecks, analyze token efficiency, and guide developers in optimizing their agents.
• 🔎 [Observability:](./docs/source/run-workflows/observe/observe.md) Track performance, trace execution flows, and gain insights into your agent behaviors in production.
• 🚀 Agent Optimization: Improve your agent's quality, accuracy, and performance with a suite of tools for all phases of the agent lifecycle.
• 🧪 [Evaluation System:](./docs/source/improve-workflows/evaluate.md) Validate and maintain accuracy of agentic workflows with a suite of tools for offline evaluation.
• 🎯 [Hyper-Parameter and Prompt Optimizer:](./docs/source/improve-workflows/optimizer.md) Automatically identify the best configuration and prompts to ensure you are getting the most out of your agent.
• 🧠 [Fine-tuning with Reinforcement Learning:](./docs/source/improve-workflows/finetuning/index.md) Fine-tune LLMs specifically for your agent and train intrinsic information about your workflow directly into the model.
• ⚡ [NVIDIA Dynamo Integration:](./examples/dynamo_integration/README.md) Use Dynamo and NeMo Agent Toolkit together to improve agent performance at scale.
• ⚙️ **Agent Performance Primitives (APP):** Accelerate graph-based agent frameworks such as LangChain, CrewAI, and Agno with parallel execution, speculative branching, and node-level priority routing.
• 🔌 Protocol Support: Integrate with common protocols used to build agents.
• 🔗 [Model Context Protocol (MCP):](./docs/source/build-workflows/mcp-client.md) Integrate [MCP tools](./docs/source/build-workflows/mcp-client.md) into your agents or serve your tools and agents as an [MCP server](./docs/source/run-workflows/mcp-server.md) for others to consume.
• 🤝…