AI laboratories have built their credibility on model benchmarking, a process presented as a neutral yardstick for judging one system against another. Solaiman noted at TechCrunch Disrupt that the underlying methods are still evolving and often opaque, which hampers their reliability as the industry leans more heavily on these metrics for decision-making.
She emphasized that trust in evaluation results hinges on reproducibility and access to the underlying model weights. Open-weight models, she argued, enable researchers to rerun tests and verify outcomes, a capability often missing in closed-source settings. Solaiman also warned that the field lacks robust ways to meaningfully compare disparate benchmarks, leaving practitioners uncertain about relative performance.
The notion of embedded evaluators,independent observers placed inside AI firms,has gained traction as labs seek alternatives to direct government oversight after a series of safety incidents. Hugging Face itself was cited in one such incident, prompting calls for third-party review. Yet the selection of evaluators remains contentious, illustrated by Anthropic’s hiring of Accenture, which drew criticism over the firm’s expertise.
Evaluations are expanding beyond pure model performance to encompass the entire system, including the harness that users employ to access the model and any downstream agents. Solaiman explained that a weaker model paired with a superior harness can outshine a frontier model with a poor interface. She described agentic evaluations as particularly difficult and costly, noting a shortage of robust tests for harnesses and holistic agents.
Given the current free-for-all landscape, Solaiman called for a uniform, industry-wide benchmarking framework that defines evaluator qualifications and standardizes test methodologies. She argued that without such coordination, the credibility of benchmarking will erode, limiting its usefulness for product development and safety assurance. A shared approach, she concluded, is essential for the sector to maintain confidence in AI performance claims.