togethercomputer/gorilla

togethercomputer/gorilla

Description: Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)

Language: Python

License: Apache-2.0

Stars: 0

Forks: 1

Open issues: 1

Created: 2025-04-17T22:42:28Z

Pushed: 2025-07-14T05:11:25Z

Default branch: main

Fork: yes

Parent repository: ShishirPatil/gorilla

Archived: no

README:

Gorilla: Large Language Model Connected with Massive APIs

Latest Updates

• 🎯 [10/04/2024] Introducing the Agent Arena by Gorilla X LMSYS Chatbot Arena! Compare different agents in tasks like search, finance, RAG, and beyond. Explore which models and tools work best for specific tasks through our novel ranking system and community-driven prompt hub. [Blog] [Arena] [Leaderboard] [Dataset] [Tweet]

• 📣 [09/21/2024] Announcing BFCL V3 - Evaluating multi-turn and multi-step function calling capabilities! New state-based evaluation system tests models on handling complex workflows, sequential functions, and service states. [Blog] [Leaderboard] [Code] [Tweet]

• 🚀 [08/20/2024] Released BFCL V2 • Live! The Berkeley Function-Calling Leaderboard now features enterprise-contributed data and real-world scenarios. [Blog] [Live Leaderboard] [V2 Categories Leaderboard] [Tweet]

• ⚡️ [04/12/2024] Excited to release GoEx - a runtime for LLM-generated actions like code, API calls, and more. Featuring "post-facto validation" for assessing LLM actions after execution, "undo" and "damage confinement" abstractions to manage unintended actions & risks. This paves the way for fully autonomous LLM agents, enhancing interaction between apps & services with human-out-of-loop. [Blog] [Code] [Paper] [Tweet]

• ⏰ [04/01/2024] Introducing cost and latency metrics into Berkeley function calling leaderboard!
• :rocket: [03/15/2024] RAFT: Adapting Language Model to Domain Specific RAG is live! [MSFT-Meta blog] [Berkeley Blog]
• :trophy: [02/26/2024] Berkeley Function Calling Leaderboard is live!
• :dart: [02/25/2024] OpenFunctions v2 sets new SoTA for open-source LLMs!
• :fire: [11/16/2023] Excited to release Gorilla OpenFunctions
• 💻 [06/29/2023] Released gorilla-cli, LLMs for your CLI!
• 🟢 [06/06/2023] Released Commercially usable, Apache 2.0 licensed Gorilla models
• :rocket: [05/30/2023] Provided the [CLI interface](inference/README.md) to chat with Gorilla!
• :rocket: [05/28/2023] Released Torch Hub and TensorFlow Hub Models!
• :rocket: [05/27/2023] Released the first Gorilla model!  or :hugs:!
• :fire: [05/27/2023] We released the APIZoo contribution guide for community API contributions!
• :fire: [05/25/2023] We release the APIBench dataset and the evaluation code of Gorilla!

About

Gorilla enables LLMs to use tools by invoking APIs. Given a natural language query, Gorilla comes up with the semantically- and syntactically- correct API to invoke.

With Gorilla, we are the first to demonstrate how to use LLMs to invoke 1,600+ (and growing) API calls accurately while reducing hallucination. This repository contains [inference code](/gorilla/inference) for running Gorilla finetuned models, [evaluation code](/gorilla/eval) for reproducing results from our paper, and [APIBench](/data) - the largest collection of APIs, curated and easy to be trained on!

Since our initial release, we've served ~500k requests and witnessed incredible adoption by developers worldwide. The project has expanded to include tools, evaluations, leaderboard, end-to-end finetuning recipes, infrastructure components, and the Gorilla API Store:

| Project | Type | Description (click to expand) | |---------|------|---------------------------| | Gorilla Paper | 🤖 Model 📝 Fine-tuning 📚 Dataset 📊 Evaluation 🔧 Infra | Large Language Model Connected with Massive APIs• Novel finetuning approach for API invocation • Evaluation on 1,600+ APIs (APIBench) • Retrieval-augmented training for test-time adaptation | | [Gorilla OpenFunctions-V2](openfunctions/) | 🤖 Model | Drop-in alternative for function calling, supporting multiple complex data types and parallel execution• Multiple & parallel function execution with OpenAI-compatible endpoints • Native support for Python, Java, JavaScript, and REST APIs with expanded data types • Function relevance detection to reduce hallucinations • Enhanced RESTful API formatting capabilities • State-of-the-art performance among open-source models | | [Berkeley Function Calling Leaderboard (BFCL)](berkeley-function-call-leaderboard/) | 📊 Evaluation 🏆 Leaderboard 🔧 Function Calling Infra 📚 Dataset | Comprehensive evaluation of function-calling capabilities• V1: Expert-curated dataset for evaluating single-turn function calling • V2: Enterprise-contributed data for real-world scenarios • V3: Multi-turn & multi-step function calling evaluation • Cost and latency metrics for all models • Interactive API explorer for testing •…