reka-ai/Archon

reka-ai/Archon

Description: Archon provides a modular framework for combining different inference-time techniques and LMs with just a JSON config file.

Language: Python

License: Apache-2.0

Stars: 1

Forks: 0

Open issues: 0

Created: 2024-10-08T19:57:05Z

Pushed: 2024-10-08T20:03:08Z

Default branch: main

Fork: yes

Parent repository: ScalingIntelligence/Archon

Archived: no

README:

Archon: An Architecture Search Framework for Inference-Time Techniques

This repository provides the accompanying code for Archon: An Architecture Search Framework for Inference-Time Techniques

Inference-time techniques allow us to bolster the strengths of existing LMs by utilizing multiple sample calls and multiple LMs to increase system performance for a given task. Archon provides a modular framework for combining different inference-time techniques and LMs with just a JSON config file. Check out our [Quick Start](#quick-start) guide to get started


Table of Contents

• [Installation](#installation)
• [Python Package](#python-package)
• [Running Locally](#running-locally)
• [Archon Overview](#archon-overview)
• [Quick Start](#quick-start)
• [Tutorials/Examples](#tutorialsexamples)
• [gen_answers.py and Benchmarks](#gen_answerspy-and-benchmarks)
• [Add Your Own Benchmark](#add-your-own-benchmark)
• [Key Handling](#key-handling)

Installation

Python Package

Archon is publicly available for use at here.

pip install archon-ai

Running Locally

Alternatively, you can use Archon directly from this repository. This is the most up to date and offers the most flexibility working with our codebase

git clone https://github.com/ScalingIntelligence/Archon.git
git submodule init
git submodule update

We recommend using our environment that can be initialized with:

conda env create -f archon_env.yml
conda activate archon_env
pip install -r requirements.txt

We recommend you work within the archon/ directory. Archon can be instantiated via the Archon class in archon.py. Get started by cloning the repository and navigating to our quickstart.py or creating your own starter file that imports our Archon class as shown below:

from archon import Archon

Archon Overview

Our Archon codebase provides users with the ability to run Archon configurations. We highly recommend reading our research paper here.

At the moment, we provide support for these inference times techniques: generator, fuser, critic, ranker, verifier, unit_test_generator, unit_test_evaluator. Their prompts can be found [here](archon/components/prompts.py). As well as the ability to [add your own components](#tutorialsexamples).

We also provide these API Access points that can be used in Archon configurations: Together_API, OpenAI_API, Anthropic_API, Groq_API, Google_API, tgi, Bedrock_API. You can also [add your own generators/endpoints](#tutorialsexamples).

Quick Start

This colab notebook demonstrates how to use the Archon python package.

Archon works by taking in a config file in JSON format that specifies the architecture you want to run and its available parameters. Say I want to ask a compound GPT 4o system a question and output a singular response. We want to sample gpt-4o 10 times, rank the top 5 responses, and then fuse for a final response. We can create a config that looks like this:

archon_gpt_config = {
"name": "archon-gpt-multi-model",
"layers": [
[
{
"type": "generator",
"model": "gpt-4o",
"model_type": "OpenAI_API",
"top_k": 1,
"temperature": 0.7,
"max_tokens": 2048,
"samples": 10
}
],
[
{
"type": "ranker",
"model": "gpt-4o",
"model_type": "OpenAI_API",
"top_k": 5,
"temperature": 0.7,
"max_tokens": 2048,
}
],
[
{
"type": "fuser",
"model": "gpt-4o",
"model_type": "OpenAI_API",
"temperature": 0.7,
"max_tokens": 2048,
"samples": 1
}
]
]
}

To generate a response:

# archon_config can also be stored and read from a file. Then passed to Archon()
archon = Archon(archon_gpt_config)

testing_instruction = [{"role": "user", "content": "How do I make a cake?"}]

response = archon.generate(testing_instruction)

print(response)

A full local example can be seen in our [quickstart.py](archon/quickstart.py) file.

Tutorials/Examples

To fully take advantage of what Archon has to offer, we recommend you take a look at our tutorials/examples.

| Tutorial | Run in Colab | Description | | ------------- | ------------- | ------------- | | [Quick Start (File)](archon/quickstart.py) | [](https://colab.research.google.com/drive/1DH-vsm6aLxoaIPqs9xFYOBj_n6NFKEPm?usp=sharing) | Introduces Archon and how to create and use a compound Archon configuration of gpt-4o models. | | [Single Model Query (Config)](archon/configs/individual_models/gpt-4o-2024-05-13.json) | [](https://colab.research.google.com/drive/1Y9qg4DDQs8RK-VXSEj9jZaK-fAUfTbFM?usp=sharing) | A single gpt-4o model call. | | **Custom Components (Colab)** | [](https://colab.research.google.com/drive/11h-uL2D7oFiSu6HdQIHlf_xDyRDOy56m?usp=sharing) | We offer support on adding your own components apart from the supported ones (generation, fusion, ranking, etc). The colab notebook will show you how to create a component, add it to your configs, and add it to Archon with add_component. Furthermore, you can pass a custom state between components, allowing you to have multiple custom components that can send information between eachother.| | **Custom Generator/Endpoint (Colab)** | [](https://colab.research.google.com/drive/1D1Mecl1jJNRY875ThLGJZupRRQC53lk3?usp=sharing) | We offer support to add your own generator if you need a custom access point or need more than the ones provided (Together, OpenAI, etc). The colab notebook will show you how to add a custom endpoint. Here you will see how to create a generator function, add it to your configs, and add it to Archon with add_generator.| | [**Improving GPT 4o using Archon…