Your Next Long-Context Recipe: Open-Ended Problems
- Date
- May 12, 2026
- Category
- Research
Frontier-CSBenchmark
A benchmark that doesn't saturate.
01
Unsolved
No solution has achieved perfect scores.
02
Open-ended
Research and optimization challenges.
03
Verifiable
Continuous scoring, always room to improve.
04
Diverse
Systems, ML, algorithms, security, and more.
→ Read our blog here.
→ Read our paper here.
frontier-cs · codebase_adaptation · evaluation
# Run with any standard agent CLI
$ uv run harbor run -d frontier-cs-algorithm \ -a claude-code -m "anthropic/claude-opus-4-6"
# Try your own solution!
$ uv run frontier eval algorithmic 0 <your_solution.cpp>
track.algorithmic problem.0 backend.docker
continuous scoring enabled...
running public test instances...
test 5 / 10 score: 43.2
test 10 / 10 score: 57.8
✓ Score@1 72.6
✓ leaderboard submission ready
Recent blog posts
Frontier-CS Goes Live: 2,000 Humans vs. AI on an Open-Ended Problem
- Date
- April 21, 2026
- Category
- Research
Auto Building Agent Memory for Open Algorithm Problems: Evaluating ALMA on Frontier CS
- Date
- March 17, 2026
- Category
- Research
Leaderboard
Algorithmic Track
172 problems
| Rank | Model | Score@1 | Avg@5 | Score@5 | Elo |
|---|---|---|---|---|---|
| 1 | gemini-3.0-pro | 33.12 | 34.58 | 56.09 | 1265 |
| 2 | gpt-5.2-thinking | 32.40 | 33.11 | 47.19 | 1242 |
| 3 | gpt-5-thinking | 23.10 | 22.58 | 39.73 | 1196 |
| 4 | deepseek-3.2 | 24.83 | 23.89 | 41.44 | 1193 |
| 5 | grok-4 | 24.04 | 22.98 | 36.81 | 1174 |
| 6 | gemini-2.5-pro | 20.34 | 19.32 | 36.65 | 1167 |
| 7 | gpt-5.1-thinking | 20.64 | 21.49 | 34.76 | 1164 |
Human reference: 86.99 (Score@1)
Research Track
68 problems
| Rank | Model | Score@1 | Avg@5 | Score@5 | Elo |
|---|---|---|---|---|---|
| 1 | gemini-3.0-pro | 46.55 | 43.14 | 59.22 | 1283 |
| 2 | gpt-5-thinking | 30.91 | 34.94 | 55.25 | 1218 |
| 3 | gpt-5.1-thinking | 32.12 | 33.70 | 56.79 | 1214 |
| 4 | gpt-5.2-thinking | 30.29 | 34.09 | 58.90 | 1210 |
| 5 | gemini-2.5-pro | 21.66 | 25.74 | 51.57 | 1180 |
| 6 | grok-4 | 26.75 | 24.01 | 48.15 | 1149 |
| 7 | deepseek-3.2 | 21.51 | 21.76 | 44.41 | 1146 |
Agent Track
Coming soon
Agent results will appear here once the track is released.
Example tasks
Permutation Recovery
Recover a hidden permutation using as few adaptive queries as possible. The task tests information-efficient probing, feedback-driven planning, and reasoning under sparse signals.
View task →
Polyomino Packing
Pack geometric pieces into increasingly dense layouts. The task rewards iterative search, heuristic design, symmetry handling, and long-horizon improvement.
View task →100+ Contributors from
Academic institutions
UC Berkeley
Princeton
Stanford
MIT
UCSD
UW
Georgia Tech
Michigan
NYU
UIUC
Toronto
NTU