ARIA Benchmark: How Much Machine Learning Do AI Models Actually Know?
A suite of five closed-book benchmarks probing the ML knowledge that frontier language models have internalized during training.
#benchmarks
ArXiv Research Code Dataset: 129K Research Repositories
A collection of 4.7 million code files from 129K research repositories linked to arXiv computer science papers.
ArXivDLInstruct: 778K Research Code Functions for Instruction Tuning
A dataset of 778,152 functions extracted from arXiv-linked research code, each paired with instruction prompts, for training ML-specialized code generation models.
DeltaML-Bench: Evaluating Machine Learning Agents on Real-World Research Repositories
A 48-task benchmark for evaluating autonomous ML experimentation in real research repositories.
ML Research Benchmark: Can AI Agents Do Real ML Research?
A benchmark suite of 7 competition-level ML challenges for evaluating whether AI agents can perform genuine research iteration beyond baseline reproduction.