research area Methods and AlgorithmsWorkshop at ICML
content type paperpublished July 2025

Boolformer: Symbolic Regression of Logic Functions with Transformers

AuthorsStéphane d’Ascoli*, Arthur Renard*†, Vassilis Papadopoulos†, Clément Hongler†, Josh Susskind, Samy Bengio, Emmanuel Abbé‡

View publication

This paper was accepted at the 2nd AI for Math Workshop at ICML 2025.

We introduce Boolformer, a Transformer-based model trained to perform end-to-end symbolic regression of Boolean functions. First, we show that it can predict compact formulas for complex functions not seen during training, given their full truth table. Then, we demonstrate that even with incomplete or noisy observations, Boolformer is still able to find good approximate expressions. We evaluate Boolformer on a broad set of real-world binary classification datasets, demonstrating its potential as an interpretable alternative to classic machine learning methods. Finally, we apply it to the widespread task of modeling the dynamics of gene regulatory networks and show through a benchmark that Boolformer is competitive with state-of-the-art genetic algorithms, with a speedup of several orders of magnitude. Our code and models are available publicly.

  • * Equal contributions
  • † École Polytechnique Fédérale de Lausanne (EPFL)
  • ‡ School of Basic Sciences (SB), Institute of Computing (IC), EPFL

Related readings and updates.

FlexTok: Resampling Images into 1D Token Sequences of Flexible Length

February 19, 2025research area Computer Vision

This work was done in collaboration with Swiss Federal Institute of Technology Lausanne (EPFL). Image tokenization has enabled major advances in autoregressive image generation by providing compressed, discrete representations that are more efficient to process than raw pixels. While traditional approaches use 2D grid tokenization, recent methods like TiTok have shown that 1D tokenization can achieve high generation quality by eliminating grid…
Read more

Learning to Reason with Neural Networks: Generalization, Unseen Data and Boolean Measures

November 10, 2022research area Methods and Algorithmsconference NeurIPS

This paper considers the Pointer Value Retrieval (PVR) benchmark introduced in [ZRKB21], where a `reasoning' function acts on a string of digits to produce the label. More generally, the paper considers the learning of logical functions with gradient descent (GD) on neural networks. It is first shown that in order to learn logical functions with gradient descent on symmetric neural networks, the generalization error can be lower-bounded in terms…
Read more