MoTIF Accepted at NeurIPS 2026: Making Video AI Easier to Understand

Following its Best Paper Award at an ICML 2026 workshop, MoTIF has now been accepted to NeurIPS 2026. The paper will appear in the Evaluations & Datasets Track, previously known as Datasets & Benchmarks.

MoTIF helps explain how a model classifies a video. It tracks understandable concepts, such as objects and actions, over time and shows how their appearance and recurrence contribute to the prediction. This lets researchers inspect explanations for a whole video as well as individual time windows.

The framework also uses a model that processes images and text to discover concepts from training videos, reducing the need for manual labels. Across several video benchmarks, MoTIF improves on approaches that use a single concept summary for each video and remains competitive with other interpretable concept-based models.

The paper, “Concepts in Motion: Temporal Concept Bottleneck Model for Interpretable Video Classification”, was written by Patrick Knab, Sascha Marton, Philipp J. Schubert, Drago Guggiana, and Christian Bartelt, bringing together researchers from TU Clausthal and Ramblr.ai Research.

NeurIPS holds an A* rating in the ICORE 2026 conference ranking, its highest category.

Abstract

Concept Bottleneck Models (CBMs) enable interpretable image classification by structuring predictions around human-understandable concepts, but extending this paradigm to video remains challenging due to the difficulty of extracting concepts and modeling them over time. In this paper, we introduce MoTIF (Moving Temporal Interpretable Framework), a transformer-based concept architecture that operates on sequences of temporally grounded concept activations, by employing per-concept temporal self-attention to model when individual concepts recur and how their temporal patterns contribute to predictions. Central to the framework is a class-conditioned VLM-based concept discovery module that extracts object- and action-centric textual concepts from training videos, yielding temporally expressive concept sets without manual concept annotation. Across multiple video benchmarks, this combination improves over global concept bottlenecks and remains competitive within the interpretable concept-bottleneck setting, while narrowing the gap to strong black-box video baselines that we report as contextual references. Code available here.