Ai Benchmark Lying

Your AI Benchmark is Lying to You. Here's How We Caught It

GLM 5.2 is live! Opus-level intelligence at open-source rates. Pay per token on serverless. Try it today.

Blog

AI Benchmark Lying Your AI Benchmark is Lying to You. Here's How We Caught It

PUBLISHED 8/15/2025

Would you give GPT-4.1 an A grade for this image? We sure wouldn’t!

That’s exactly what our AI judge did, giving it a 93.3%. To its credit, it was a diligent box-checker, taking a list of 15 requirements and confirmed that, yes, there were colored shapes where the logo should be, and a box where the search bar should be. It was technically correct, but its misalignment to human expectations what matters.

1 2 3 4 5 6 7 EvaluationResult :

{ "score" : 0.9333333333333333 , "is_score_valid" : true , "reason" : "1. The background is white. 2. Primary elements are horizontally centered. 3. The Google logo is in the center and uses the correct colors. 4. A prominent search bar is directly below the logo. 5. The search bar is a rounded rectangle with a light gray border. 6. The search bar contains a gray magnifying glass icon on the left. 7. The search bar contains a gray microphone icon on the right. 8. Two distinct buttons are below the search bar. 9. The left button is labeled 'Google Search'. 10. The right button is labeled 'I'm Feeling Lucky'. 11. Buttons have a light gray background, thin border, and dark gray text. 12. There is a header section at the top right. 13. The header includes 'Gmail' and 'Images' links. 14. The header includes a 3x3 grid icon. 15. The 'Sign in' button is present, but the text is not fully visible, so this requirement is not fully met." , }

This is a huge problem in AI. We celebrate benchmark scores that don't reflect real-world quality. We knew our evaluation was broken, and we used Eval Protocol (EP) to fix it. To set the scene, if you haven’t already, check out our example on how we ported SVGBench to EP. TL;DR: using our powerful @evaluation_test decorator, we implemented the open-source SVGBench and evaluate it using GPT-4.1 as a LLM-judge against a list of rubric items for each task. Teaching Our Judge to See Like a Human

Our first evaluation was a simple checklist. It asked questions like, "Is there a search bar?" but not "Does the search bar look right?" To fix this, we moved from a rigid, row-specific checklist ( listwise ) to a universal rubric that applies to every image ( pointwise ). Instead of checking for pixels, we started judging based on principles humans care about. I prompted my AI coding assistant to create a new, tougher judge with a more sophisticated rubric: @question_row_1_gpt-4.1.png This image returned an EvaluateResult of the below. The results don't seem to line up with what we expect. We want you to add a separate pointwise evaluation that contains a list of rubrics to judge individual images for intent matching for elements you can think of that align with human preference, for example spatial design. Its implementation lands on these core 5 qualities: • 🎯 Intent Matching: Does it actually look like the Google homepage? • 👁️ Content Recognizability: Can you read the logo, or is it just colored blobs? • 📐 Spatial Design: Does the layout look professional? • 👤 User Experience: Is it clear and usable? • 🎨 Visual Coherence: Does it all work together?

The assistant quickly scaffolded a new @evaluation_test within the EP framework. The core idea was to stop asking "did you follow the instructions?" and start asking "is this any good?" 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 @evaluation_test ( input_dataset = [ "tests/pytest/data/svgbench_sample_dataset.jsonl" ] , dataset_adapter = svgbench_to_evaluation_row , completion_params = [ { "temperature" : 0.0 , "model" : "gpt-4.1" } , { "temperature" : 0.8 , "model" : "fireworks_ai/accounts/fireworks/models/gpt-oss-120b" , "extra_body" : { "reasoning_effort" : "high" } , } , ] , rollout_processor = default_single_turn_rollout_processor , passed_threshold = 0.6 , # Higher threshold for human preference num_runs = 1 , mode = "pointwise" , max_concurrent_rollouts = 50 , ) def test_svg_human_preference_evaluation ( row : EvaluationRow ) - > EvaluationRow : """ Test SVG generation using human preference rubrics.

This evaluation focuses on: 1. Intent matching - Does content actually fulfill the intended purpose? 2. Content recognizability - Are key elements genuinely recognizable? 3. Spatial design quality - Professional layout and visual hierarchy 4. User experience - Would humans find this usable/appropriate? 5. Visual coherence - Do elements work together harmoniously?

This should catch issues like Google logos that are just colored circles. """

. . .

try :

Render SVG to PNG

if not render_svg_to_png ( svg_code , png_path ) : row . evaluation_result = EvaluateResult ( score = 0.0 , reason = "Failed to render SVG to PNG" ) return row

Evaluate with human preference rubrics

human_pref_result = evaluate_with_human_preference_rubrics ( png_path , original_prompt , requirements )

Extract scores and create detailed reasoning

overall_score = human_pref_result . get ( "overall_human_preference_score" , 0.0 )

Create comprehensive reasoning that shows all rubric scores

detailed_reasoning = f"""HUMAN PREFERENCE EVALUATION:

🎯 Intent Matching: { human_pref_result . get ( 'intent_matching_score' , 0.0 ) : .2f } /1.0 { human_pref_result . get ( 'intent_reasoning' , 'No reasoning provided' ) }

👁️ Content Recognizability: { human_pref_result . get ( 'content_recognizability_score' , 0.0 ) : .2f } /1.0 { human_pref_result . get ( 'content_reasoning' , 'No reasoning provided' ) }

📐 Spatial Design Quality: { human_pref_result . get ( 'spatial_design_score' , 0.0 ) : .2f } /1.0 { human_pref_result . get ( 'spatial_reasoning' , 'No reasoning provided' ) }

👤 User Experience: { human_pref_result . get ( 'user_experience_score' , 0.0 ) : .2f } /1.0 { human_pref_result . get ( 'ux_reasoning' , 'No reasoning provided' ) }

🎨 Visual Coherence: { human_pref_result . get ( 'visual_coherence_score' , 0.0 ) : .2f } /1.0 { human_pref_result...