What does this repo signal mean?

OpenAI published openai/SWELancer-Benchmark. This repository signal exposes tooling, eval, infrastructure, or model-adjacent work before it may appear in a launch post. High-signal details: repo openai/SWELancer-Benchmark · Notable benchmark from OpenAI with solid traction.. onlylabs links this event to 1 captured evidence page and 6 related repo signals. It also maps to Data demand, Evals and quality in the data-business radar.

OpenAI Repo: openai/SWELancer-Benchmark

Captured source

source ↗

GitHub/github.com/openai/SWELancer-Benchmark

openai/SWELancer-Benchmark repository metadata

Source ↗

published Feb 18, 2025seen 6dcaptured 12hhttp 200method plain

openai/SWELancer-Benchmark

Description: This repo contains the dataset and code for the paper "SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering?"

Stars: 1439

Forks: 137

Open issues: 0

Created: 2025-02-18T17:23:15Z

Pushed: 2025-07-18T02:02:20Z

Default branch: main

Fork: no

Archived: yes

README:

SWELancer

The SWE-Lancer codebase has been merged into https://github.com/openai/preparedness!

Please see https://github.com/openai/preparedness to run SWELancer.

Notability

notability 7.0/10

Notable benchmark from OpenAI with solid traction.

OpenAI has a repo signal matching data demand, evals and quality.

Data demand Evals and quality