Command R7b

North Mini Code. Cohere's first model for developers.

Learn more

Dec 13, 2024

4 minutes read

Introducing Command R7B: Fast and efficient generative AI

The smallest model in our R series delivers top-tier speed, efficiency, and quality to build powerful AI applications on commodity GPUs and edge devices.

!Blog Post Featured Image

Today, we’re excited to release Command R7B, the smallest, fastest, and final model in our R series of enterprise-focused large language models (LLMs). Command R7B provides state-of-the-art performance in its class of open-weights models across real-world tasks that matter for users. The model is designed for developers and businesses that need to optimize for the speed, cost-performance, and compute resources of their use cases.

Like our other models in the R series, Command R7B offers a context length of 128k and excels in capabilities important for a wide range of business applications. It delivers a powerful combination of multilingual support, citation verified retrieval-augmented generation (RAG), reasoning, tool use, and agentic behavior. Thanks to its compact size and efficiency it can be served on low-end GPUs, a MacBook, or even CPUs – drastically lowering the cost of deploying AI applications into production.

##### High performance in a small package

###### A well-rounded model

Command R7B excels on standardized and externally verifiable benchmarks such as the HuggingFace Open LLM Leaderboard. Compared to other similarly sized open-weights models, Command R7B ranks first on average with strong performance across all tasks.

##### Enhanced efficiency in math, code, and reasoning tasks

A major area of focus for Command R7B has been improving performance on math and reasoning, code, and multilingual tasks. In particular, the model matches or exceeds leading open-weights models in its class across common math and code benchmarks while using fewer parameters.

##### Best-in-class RAG, tool use, and agents

Command R7B outperforms the other similarly sized open-weights models when it comes to core business use cases such as RAG, tool use, and AI agents. It is an ideal choice for enterprises looking for a cost-efficient model grounded in their internal documents and data. Like our other R series models, our RAG offering delivers native in-line citations that significantly reduce hallucinations and make fact-checking easier.

For tool use, we see stronger overall performance than models of similar size on the industry-standard Berkeley Function-Calling Leaderboard. This shows Command R7B is particularly effective at tool use in real-world, diverse, and dynamic environments and avoids calling tools unnecessarily which is an important aspect of tool use in practical applications . Command R7B’s multi-step tool use capabilities allow it to power fast and capable AI agents.

##### Optimized for enterprise use cases

Our models are optimized for the capabilities enterprises need for real-world deployment of AI systems. The R series delivers an unmatched balance of efficiency and strong performance. This means ensuring they excel on human evaluation, the gold standard for quality assessment. Command R7B outperforms similarly sized open-weights models in blind head-to-head evaluations by human raters on RAG use cases our customers care about when building AI assistants for functions like customer service, HR, compliance, and IT support.

##### Efficient and fast

Command R7B’s compact size offers a reduced serving footprint that is ideal for rapid prototyping and iteration. It excels at high throughput, real-time use cases like chatbots and code assistants. It also unlocks dramatically cheaper deployment infrastructure such as consumer GPUs and CPUs to unlock on-device inference.

We achieve this without compromising on our enterprise-grade security and privacy standards to protect customers' data.

##### Get started

Command R7B is available today on the Cohere Platform as well as accessible on HuggingFace. We’re excited to be releasing the weights of this model to provide greater access to cutting-edge technology for the AI research community.

| Cohere API Pricing | Input Tokens | Output Tokens | | --- | --- | --- | | Command R7B | $0.0375 / 1M | $0.15 / 1M |

• * *

\[1\] Conversational RAG: Average performance over the 10-dataset ChatRAGBench benchmark which tests the ability to generate responses in a wide range of settings including conversational tasks, attending over long inputs, analyzing tables and extracting and manipulating numerical information in financial settings. We improve evaluation methodology using a PoLL judge ensemble ( Verga et al. 2024) using Haiku, GPT3.5 and Command R, providing higher agreement to human annotators (Fleiss’ kappa=0.74 vs 0.57 for original, calculated over 20k human judgements).

Tool use: Performance on the BFCL-v3 benchmark on 12 Dec 2024. Where available, scores are taken from the public leaderboard, and otherwise use a best-effort internal evaluation using the official codebase. For competitors, we report the higher of their BFCL ‘prompted’ or ‘function-calling’ score. We report the Overall score, the Live subset score which tests tool-use in real-world, diverse, and dynamic environments, and the Irrelevance subset score, which tests how well models avoid calling tools unnecessarily.

REACT Agent/Multi-step: We assess the abilities of LangChain REACT agents connected to the internet to break down complex questions and formulate and successfully carry out a research plan to answer them using Bamboogle and StrategyQA. Bamboogle is evaluated using a...