Issues

• OrcaRouter provider support for ARC-AGI benchmarking

  arcprize/arc-agi-benchmarking#104 · Filter by author putraperdana1207-pixel opened on Sep 11, 2026

• Request: guidance on independently sealed ARC-style evaluation

  arcprize/arc-agi-benchmarking#102 · Filter by author Takehayaally opened on Aug 25, 2026

• Scoring: task coverage and pair mapping are taken from the submission file, not checked against task_dir (two measured gaps, controls included)

  arcprize/arc-agi-benchmarking#97 · Filter by author dkx955 opened on Jul 29, 2026
  1

• Scorecard links after running benchmark do not work

  arcprize/arc-agi-benchmarking#86 · Filter by author blakewest opened on Mar 29, 2026
  2

• Question: long horizon tension stress tests as a complementary AGI evaluation axis

  arcprize/arc-agi-benchmarking#82 · Filter by author onestardao opened on Feb 6, 2026
  1

• Please put base prompt in readme

  arcprize/arc-agi-benchmarking#73 · Filter by author theaustinhatfield opened on Jan 25, 2026

• Temperature and top P sampling

  arcprize/arc-agi-benchmarking#57 · Filter by author kiankyars opened on Nov 29, 2025
  1

• Readme mentions requirements.txt, but there's no requirement.txt

  arcprize/arc-agi-benchmarking#53 · Filter by author amueller opened on Sep 3, 2025
  3

• ARC is for AGI, should exclude LLM GPT。。

  arcprize/arc-agi-benchmarking#20 · Filter by author zdx3578 opened on Mar 25, 2025
  1