Apple Workshop on Privacy-Preserving Machine Learning 2025: Differentially Private Synthetic Data via Foundation Model APIs
AuthorsSivakanth Gopi (Microsoft Research)
Apple Workshop on Privacy-Preserving Machine Learning 2025: Differentially Private Synthetic Data via Foundation Model APIs
AuthorsSivakanth Gopi (Microsoft Research)
Faster Retrieval for Better, More Efficient RAG
November 5, 2026conference ICML
Maximum inner product search (MIPS) sits at the core of recommendation systems, search, and retrieval-augmented generation (RAG). Given a query vector, MIPS systems search within a database of millions of candidate key vectors for the one that best matches the query. The original data point associated with that best-matching key is then used to handle the query: in RAG, the document that is best suited to answer the question is typically prefixed…
Attention to Mamba: A Recipe for Cross-Architecture Distillation
November 3, 2026research area Methods and Algorithms, research area Speech and Natural Language ProcessingTransactions on Machine Learning Research (TMLR)
State Space Models (SSMs) such as Mamba have become a popular alternative to Transformer models, due to their reduced memory consumption and higher throughput at generation compared to their Attention-based counterparts. On the other hand, the community has built up a considerable body of knowledge on how to train Transformers, and many pretrained Transformer models are readily available. To facilitate the adoption of SSMs while leveraging…