TabPrep Accepted at NeurIPS 2026: Better Predictions Through Data Preparation

A ratio between two measurements can reveal a pattern that is hard to learn from either value alone. Creating such useful inputs is known as feature engineering. TabPrep makes this step easier to include when comparing machine learning models on tables of data.

The paper “TabPrep: Closing the Feature Engineering Gap in Tabular Benchmarks” introduces an efficient data preparation method that targets three recurring patterns in tabular data. These patterns expose weaknesses across several model families, including modern pretrained models.

Measuring the effect of data preparation

On TabArena, a benchmark for tabular machine learning, adding TabPrep improves results across a range of model types. The study finds that preparing additional features can deliver gains that exceed those from changing the model alone. Comparisons that leave out this step can therefore miss achievable performance.

The authors have released TabPrep’s code so researchers can include feature engineering in their own evaluations.

The work was written by Andrej Tschalzev, Nick Erickson, Yuyang Wang, Huzefa Rangwala, Stefan Lüdtke, Heiner Stuckenschmidt, and Christian Bartelt. It has been accepted to the NeurIPS 2026 main conference. NeurIPS holds the highest rating, A*, in the ICORE 2026 conference ranking.

Abstract

Progress in tabular machine learning has largely focused on increasingly sophisticated model architectures. At the same time, feature engineering remains a critical yet underexplored component of real-world modeling pipelines that is entirely absent from modern benchmarks, which creates an unquantified evaluation gap. In this work, we introduce TabPrep, a lightweight preprocessing pipeline composed of feature generators that are carefully designed to target three specific structural data patterns. We show that many widely used model classes exhibit predictable blind spots to these patterns and that systematic feature engineering alone can establish new peak performance. Across the TabArena benchmark, integrating TabPrep into model training and tuning consistently improves performance for tree-based, neural, linear, and foundation models, often surpassing gains achieved by model-centric innovations alone. TabPrep outperforms previous automated feature engineering approaches in performance, efficiency, and applicability across datasets, enabling integration into large-scale benchmarks. By releasing TabPrep (see <https://github.com/atschalz/tabprep>), we enable researchers to integrate feature engineering into their benchmarking setup, filling a longstanding gap in tabular evaluations.