This note proposes a framework for constructing interpretable machine-learning benchmarks. It emphasizes defining the task and target population before choosing data, and assessing label quality and whether performance reflects the intended capability.
Cite: Gaurav Sood. "A Benchmark For Benchmarks." https://gsood.com/research/a-benchmark-for-benchmarks/
BibTeX:
@misc{sooda,
author = {Gaurav Sood},
title = {{A Benchmark For Benchmarks}},
note = {Working paper},
url = {https://gsood.com/research/a-benchmark-for-benchmarks/}
}