amazon-science/acclaim

amazon-science/acclaim

Language: Python

License: NOASSERTION

Stars: 3

Forks: 0

Open issues: 2

Created: 2026-03-30T17:23:07Z

Pushed: 2026-04-14T01:17:15Z

Default branch: main

Fork: no

Archived: no

README:

Compiler Agent Running Instructions

Overview

CompilerAgent is a multi-agent system for producing optimized low-level code by combining LLM-based optimization with traditional compilers. The current implementation focuses on the CPU use case: the input is a C source code file and the output is an x86 assembly file (that can easily be converted to a binary by an assembler). The architecture involves 5 agents. First, a planning agent that chooses which levels of abstraction (source, IR, assembly) should be optimized. Next, three level-specific agents (one for each level) that perform optimization at the given level. Finally, a testing agent runs the produced code to generate feedback about correctness and performance.

The planning agent may call each of the components of the existing clang compiler (frontend, middle-end, and backend). It is also responsible for calling each level-specific LLM-based optimization agent; when it does so, it passes the name of an input file in the subject language (source, IR, or assembly), and optionally some instructions on what type of optimization to perform. Each level-specific agent contains a feedback loop, during which an LLM call is made to generate an optimized version of the input code and the optimized code is tested to measure both correctness and performance. The results of the tests are then given as feedback to the LLM again, generating another program. At each generation, multiple samples are collected by calling the LLM with the same context.

There are three primary tunable parameters:

• budget`: The maximum number of calls the planning agent may make to a level-specific agent
• num_samples`: The number of samples to collect on each run of the level-specific feedback loop
• num_loops`: The number of feedback loop iterations allowed within a level-specific agent

Each of these parameters can be set with an environment variable as described below.

Step-by-step instructions to run an example

• Set up an environment with access to an LLM. CompilerAgent is developed with the Strands Framework, and supports model calls to Amazon Bedrock, or via LiteLLM. Model parameters can be configured with environment variables as described below.

• The Compiler Agent is tested with Python 3.13

• Install clang by your method of choice; below is an example using apt for Linux machines.

apt install clang

• Install Hyperfine. Hyperfine is used for performance measurements. Below is an example for Debian-based systems; Hyperfine provides instructions for installation elsewhere.

wget https://github.com/sharkdp/hyperfine/releases/download/v1.19.0/hyperfine_1.19.0_amd64.deb
sudo dpkg -i hyperfine_1.19.0_amd64.deb

• Create a .env file setting various environment variables; below is an example with API keys removed. The end of this document contains thorough documentation of the configurable environment variables.

AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_REGION="us-east-2"

BYPASS_TOOL_CONSENT="true"

PLANNER_MODEL="us.anthropic.claude-3-7-sonnet-20250219-v1:0"
LEVEL_MODEL="us.anthropic.claude-3-7-sonnet-20250219-v1:0"

PLANNER_PROMPT="./planning_prompt_all.txt"
LEVEL_PROMPT="./level_prompt.txt"

MAX_TOKENS="4096"
TEMPERATURE="1"
THINKING_TOKENS="1024"

TEST_TIMEOUT="30"
TEST_RUNS="10"
WARMUP_RUNS="5"

LEVEL_SAMPLES="2"
LEVEL_LOOPS="2"
BUDGET="6"

• To run the agent on one input,

python src/compiler_agent/agent.py -s sample/source.c -d sample -l sample/log.csv -x sample/t*.in

This command runs the agent on the source file located at $DIRECTORY/source.c, and writes all output results (including intermediate outputs) in $DIRECTORY. Usage statistics are logged to $DIRECTORY/log.csv, including intermediate speedup and correctness results for each call of a level-specific agent and each parallel sample. The -x argument specifies files to be used as test cases.

Options and Testing

To get full documentation of the options available for agent.py, run

python agent.py --help

The available options are:

• -s/--source` (required): the input source code for the agent
• -o/--output`: Output assembly file. If not specified, writes to /generated.s. At least one of -o and -d must be specified.
• -d/--directory`: directory to store intermediate files. Default is same directory as the output assembly. At least one of -o and -d must be specified.
• -l/--log`: file to write logs of tool use
• -e/--level`: Level to use exclusively from [source, ir, assembly]; if not provided, all three levels are used. Exists for experimentation and should not be passed in general use cases.

Input programs

The `sample` directory contains an example input on which to run the agent. Other tests are conducted with C programs taken from the Project CodeNet dataset (https://github.com/IBM/Project_CodeNet), filtered to C programs only; see the paper for details on dataset filtering and preliminary results.

The compiler agent supports several configurations; we recommend utilizing a `.env` file to manage these. The options are:

• PLANNER_MODEL`: the name of a model to use for planning agent

+ `PLANNING_PROMPT: the system prompt for the planning agent. Compiler agent provides the default prompt src/amzn_compiler/agent/planning_prompt_all.txt. If -e` is specified to limit the agent to a specific level, the level-specific system prompt must be adjusted accordingly.

• LEVEL_MODE`: the name of a model to use for each level-specific agent. This may be different from the planner model.

+ `LEVEL_PROMPT: the system prompt for the level-specific agent. Compiler agent provides the default prompt src/amzn_compiler/agent/level_prompt.txt`.

• If using Bedrock for inference, specify the `AWS_REGION, AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY` parameters (inherited from Strands)
• If using LiteLLM, set `BASE_URL` for the endpoint and remove any AWS credentials.

• You may specify Strands configuration variables, such as `BYPASS_TOOL_CONSENT`; the compiler agent is compatible with any configuration of Strands
• Model configuration parameters:

+…