One paper accepted at the AI4MATH workshop at ICML

The paper "A Compute-Matched Re-Evaluation of TroVE on MATH" by Tobias Sesterhenn, Ian Berlot-Attwell, Janis Zenkner and Christian Bartelt has been accepted at the AI4MATH workshop @ ICML 2025. The International Conference on Machine Learning (ICML, Rank A*) is the premier gathering of professionals dedicated to the advancement of the branch of machine learning and will be held in Vancouver, July 13 - July 19th, 2025.

Abstract: Reusing established theorems and formulas is central to mathematical problem solving, serving as essential building blocks for tackling increasingly complex challenges. Recent work, TroVE, argues that code-generating Large Language Models (LLMs) can benefit similarly on the MATH benchmark by inducing and reusing higher-level toolboxes. By allocating computational budget across an ensemble of three modes - directly generating code, creating tools, and reusing tools - TroVE claims to outperform a PRIMITIVE baseline that only performs direct generation. However, recent analysis (Berlot-Attwell et al., 2024) casts doubt on these gains, noting that the tools created are often trivial or rarely reused, suggesting that improvements may stem from self-consistency or self-correction. In this work, we re-evaluate TroVE on MATH, analyze the impact of each of its modes, and show that its benefit does not come from these mechanisms, but simply from a higher computational budget spent for TroVE compared to PRIMITIVE. To this end, we also perform a small correction in the original implementation of TroVE's selection mechanism, boosting TroVE's performance on MATH by 3% in accuracy. After matching for compute, the benefit of TroVE reduces to a marginal improvement of 1%, suggesting that this toolbox approach does not provide a significant benefit on MATH.