content type videopublished June 28, 2022
PPML Workshop Talk: What Does it Mean for a Language Model to Preserve Privacy?
AuthorsFlorian Tramèr (Google, ETH Zurich)
Related readings and updates.
Continuous diffusion and flow matching models could represent a powerful alternative to autoregressive approaches for language modelling (LM), as they unlock a host of advantages currently reserved for continuous modalities, including accelerated sampling and tilting. Recently, several works have demonstrated the possibility of generating discrete data continuously by a simple flow matching process between a Gaussian and the one-hot encoded data…
Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models
August 7, 2026research area Speech and Natural Language Processing
Large Language Models (LLMs) have achieved state-of-the-art performance on a broad range of Natural Language Processing (NLP) tasks, including document processing and code generation. Autoregressive Language Models (ARMs), which generate tokens sequentially conditioned on all previous tokens, have been the predominant paradigm for LLMs. While these models have achieved high accuracy across a range of downstream tasks, they exhibit low arithmetic…
Discover opportunities in Machine Learning.
Our research in machine learning breaks new ground every day.