Speech recognition systems have improved substantially in recent years, leading to widespread adoption across computing platforms. Two common forms of speech interaction are voice assistants (VAs) that listen for spoken commands and respond accordingly, and dictation systems, which act as an alternative to a keyboard by converting the user's open-ended speech to written text for messages, emails, and so on. Speech interaction is especially important for devices with smaller or no screens, such as smart speakers and smart headphones, that support speech interaction. Yet speech presents barriers for many people with communication disabilities such as stuttering, dysarthria, or aphasia.

Improved Speech Recognition for People Who Stutter

3D Scene understanding has been an active area of machine learning (ML) research for more than a decade. More recently the release of <a href="https://developer.apple.com/augmented-reality/roomplan/" class="more">LiDAR sensor functionality</a> in Apple iPhone and iPad has begun a new era in scene understanding for the computer vision and developer communities. Fundamental research in scene understanding combined with the advances in ML can now impact everyday experiences. A variety of methods are addressing different parts of the challenge, like depth estimation, 3D reconstruction, instance segmentation, object detection, and more. Among these problems, creating a 3D floor plan is becoming key for many applications in augmented reality, robotics, e-commerce, games, and real estate.

3D Parametric Room Representation with RoomPlan

<div class="callout">

## Schedule

### Tuesday, June&nbsp;6
<ul class="links-stacked">
<li>ORAL PRESENTATION</li>
 <li>I See What You Hear: A Vision-inspired Method to Localize Words</li>
<li>10:50 AM - 12:20 PM LT in Salon des Roses A</li>
<li>Mohammad Samragh, Arnav Kundu, Ting-Yao Hu, Aman Chadha, Ashish Srivastava, Minsik Cho, Oncel Tuzel, Devang Naik</li>
<li>POSTER PRESENTATION</li>
 <li>Variable Attention Masking for Configurable Transformer Transducer Speech Recognition</li>
<li>10:50 AM - 12:20 PM LT in Poster Area 4 - Garden</li>
<li>Pawel Swietojanski, Stefan Braun, Dogan Can, Thiago Fraga da Silva, Arnab Ghoshal, Takaaki Hori, Roger Hsiao, Henry Mason, Erik McDermott, Honza Silovsky, Ruchir Travadi, Xiaodan Zhuang</li>
<li>POSTER PRESENTATION</li>
 <li>Text is All You Need: Personalizing ASR Models using Controllable Speech Synthesis</li>
<li>2:00 - 3:30 PM LT in Poster Area 2 - Garden</li>
<li>Karren Yang, Ting-Yao Hu, Jen-Hao Rick Chang, Hema Swetha Koppula, Oncel Tuzel</li>
<li>POSTER PRESENTATION</li>
 <li>Neural Transducer Training: Reduced Memory Consumption with Sample-wise Computation</li>
<li>2:00 - 3:30 PM LT in Poster Area 3 - Garden</li>
<li>Stefan Braun, Erik McDermott, Roger Hsiao</li>
<li>POSTER PRESENTATION</li>
 <li>More Speaking or More Speakers?</li>
<li>2:00 - 3:30 PM LT in Poster Area 3 - Garden</li>
<li>Dan Berrebbi, Ronan Collobert, Navdeep Jaitly, Tatiana Likhomanenko</li>
<li>POSTER PRESENTATION</li>
 <li>Audio-to-Intent Using Acoustic-Textual Subword Representations from End-to-End ASR</li>
<li>2:00 - 3:30 PM LT in Poster Area 4 - Garden</li>
<li>Pranay Dighe, Prateeth Nayak, Oggi Rudovic, Erik Marchi, Xiaochuan Niu, Ahmed Tewfik</li>
</ul>

### Wednesday, June&nbsp;7
<ul class="links-stacked">
<li>POSTER PRESENTATION</li> <li>HEiMDaL: Highly Efficient Method for Detection and Localization of wake-words</li> <li>8:15 - 9:45 AM LT in Poster Area 8 - Dome</li> <li>Arnav Kundu, Mohammad Samragh Razlighi, Minsik Cho, Priyanka Padmanabhan, Devang Naik</li>
 <li>LUNCHEON</li> <li><a href=https://2023.ieeeicassp.org/women-in-signal-processing-luncheon/>Women in Signal Processing</a></li> <li>12:20 - 2:20 PM LT at the Ambrosia Restaurant</li>
</ul>

### Thursday, June&nbsp;8
<ul class="links-stacked">
<li>ORAL PRESENTATION</li> <li>Naturalistic Head Motion Generation From Speech</li> <li>10:50 AM - 12:20 PM LT in Salon des Roses A</li> <li>Trisha Mittal, Zakaria Aldeneh, Masha Fedzechkina, Anurag Ranjan, Barry-John Theobald</li>
 <li>JOB FAIR</li> <li><a href=https://2023.ieeeicassp.org/student-job-fair-and-luncheon/>Student Job Fair and Luncheon</a></li> <li>12:00 - 3:00 PM LT at the Ambrosia Restaurant</li>
<li>POSTER PRESENTATION</li> <li>Pre-trained Model Representations and their Robustness against Noise for Speech Emotion Analysis</li> <li>2:00 - 3:30 PM LT in Poster Area 4 - Garden</li> <li>Vikramjit Mitra, Vasudha Kowtha, Hsiang-Yun Sherry Chien, Erdrin Azemi, Carlos Avendano</li>
<li>POSTER PRESENTATION</li> <li>On the Role of Lip Articulation in Visual Speech Perception</li> <li>2:00 - 3:30 PM LT in Poster Area 10 - Dome</li> <li>Zakaria Aldeneh, Masha Fedzechkina, Skyler Seto, Katherine Metcalf, Miguel Sarabia, Nicholas Apostoloff, Barry-John Theobald</li>
<li>POSTER PRESENTATION</li> <li>Learning to Detect Novel and Fine-Grained Acoustic Sequences Using Pretrained Audio Representations</li> <li>3:35 - 5:05 PM LT in Poster Area 2 - Garden</li> <li>Vasudha Kowtha, Miquel Espi, Jonathan J Huang, Yichi Zhang, Carlos Avendano</li>
</ul>

### Friday, June&nbsp;9
<ul class="links-stacked">
<li>POSTER PRESENTATION</li> <li>Improvements to Embedding-Matching Acoustic-to-Word ASR Using Multiple-Hypothesis Pronunciation-Based Embeddings</li> <li>8:15 - 9:45 AM in Poster Area 4 - Garden</li> <li>Hao Yen, Woojay Jeon</li>
</ul>

</div>

## Accepted Papers

[Audio-to-Intent Using Acoustic-Textual Subword Representations from End-to-End ASR](/research/audio-to-intent)

*Pranay Dighe, Prateeth Nayak, Oggi Rudovic, Erik Marchi, Xiaochuan Niu, Ahmed Tewfik*

[HEiMDaL: Highly Efficient Method for Detection and Localization of wake-words](/research/heimdal)

*Arnav Kundu, Mohammad Samragh Razlighi, Minsik Cho, Priyanka Padmanabhan, Devang Naik*

[I See What You Hear: A Vision-inspired Method to Localize Words](/research/vision-inspired-method)

*Mohammad Samragh, Arnav Kundu, Ting-Yao Hu, Aman Chadha, Ashish Srivastava, Minsik Cho, Oncel Tuzel, Devang Naik*

[Improvements to Embedding-Matching Acoustic-to-Word ASR Using Multiple-Hypothesis Pronunciation-Based Embeddings](/research/improvements-embedding)

*Hao Yen, Woojay Jeon*

[Learning to Detect Novel and Fine-Grained Acoustic Sequences Using Pretrained Audio Representations](/research/learning-detect-novel)

*Vasudha Kowtha, Miquel Espi, Jonathan J Huang, Yichi Zhang, Carlos Avendano*

[More Speaking or More Speakers?](/research/speakers-speaking)

*Dan Berrebbi, Ronan Collobert, Navdeep Jaitly, Tatiana Likhomanenko*

[Naturalistic Head Motion Generation From Speech](/research/naturalistic-head-motion)

*Trisha Mittal, Zakaria Aldeneh, Masha Fedzechkina, Anurag Ranjan, Barry-John Theobald*

[Neural Transducer Training: Reduced Memory Consumption with Sample-wise Computation](/research/neural-transducer-training)

*Stefan Braun, Erik McDermott, Roger Hsiao*

[On the Role of Lip Articulation in Visual Speech Perception](/research/lip-articulation)

*Zakaria Aldeneh, Masha Fedzechkina, Skyler Seto, Katherine Metcalf, Miguel Sarabia, Nicholas Apostoloff, Barry-John Theobald*

[Pre-trained Model Representations and their Robustness against Noise for Speech Emotion Analysis](/research/pre-trained-model)

*Vikramjit Mitra, Vasudha Kowtha, Hsiang-Yun Sherry Chien, Erdrin Azemi, Carlos Avendano*

[Text is All You Need: Personalizing ASR Models using Controllable Speech Synthesis](/research/personalizing-asr-models)

*Karren Yang, Ting-Yao Hu, Jen-Hao Rick Chang, Hema Swetha Koppula, Oncel Tuzel*

[Variable Attention Masking for Configurable Transformer Transducer Speech Recognition](/research/variable-attention-masking)

*Pawel Swietojanski, Stefan Braun, Dogan Can, Thiago Fraga da Silva, Arnab Ghoshal, Takaaki Hori, Roger Hsiao, Henry Mason, Erik McDermott, Honza Silovsky, Ruchir Travadi, Xiaodan Zhuang*


## Demo

Contextual Understanding in Siri

This is a demonstration of the context understanding technology shipped in Siri. Users can refer to an aforementioned entity using anaphora or nominal ellipsis, refer to an entity on screen, or correct a previous error by Siri or the user. Context understanding for Siri leverages several backend ML solutions such as query rewriting and reference resolution. This work is a step towards having more natural conversations with Siri, and was shipped in iOS 16.

All ICASSP attendees are invited to stop by the Apple booth (booth number 16, located next to the Dome Bar main entrance of the Rodos Palace Luxury Convention Resort) to experience this demo in person.

## Acknowledgements

Tatiana Likhomanenko, Arnav Kundu, Stefan Braun, Vikram Mitra, and Pawel Swietojanski are reviewers for ICASSP 2023.

Yannis Stylianou is a Seasonal School & Short Course Chair for ICASSP 2023.

<div class="callout">


Let's innovate together. Build amazing machine-learned experiences with Apple. Discover opportunities for researchers, students, and developers by visiting our [Work with us page.](/work-with-us)


</div>

Apple is sponsoring the International Conference on Acoustics, Speech and Signal Processing (ICASSP), which will take place in person from June 4 - 10 in Rhodes Island, Greece. ICASSP is the IEEE Signal Processing Society's flagship conference on signal processing and its applications. Below is the schedule of Apple sponsored workshops and events at ICASSP 2023.

International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2023

<div class="callout"> 

## Schedule

### Monday, May&nbsp;1
<ul class="links-stacked">
 <li>ACCEPTED PAPER</li>
 <li><a href="https://iclr.cc/virtual/2023/poster/11745">Continuous Pseudo-labeling from the Start</a></li>
 <li>11:30 AM - 1:30 PM LT MH1-2-3-4 #163</li>
 <li>Dan Berrebbi, Ronan Collobert, Samy Bengio, Navdeep Jaitly, Tatiana Likhomanenko</li>
</ul>

### Tuesday, May&nbsp;2
<ul class="links-stacked">
 <li>ACCEPTED PAPER</li>
 <li><a href="https://iclr.cc/virtual/2023/poster/10840">Adaptive Optimization in the ∞-Width Limit</a></li>
 <li>11:30 AM - 1:30 PM LT MH1-2-3-4 #139</li>
 <li>Etai Littwin, Greg Yang</li>
<li>ACCEPTED PAPER</li>
<li><a href="https://iclr.cc/virtual/2023/poster/11790">Diffusion Probabilistic Fields</a></li>
 <li>11:30 AM - 1:30 PM LT MH1-2-3-4 #72</li>
 <li>Peiye Zhuang, Samira Abnar, Jiatao Gu, Alexander Schwing, Josh M. Susskind, Miguel Angel Bautista</li>
<li>ACCEPTED PAPER</li>
<li><a href="/research/multi-stage-diffusion">f-DM: A Multi-stage Diffusion Model via Progressive Signal Transformation</a></li>
 <li>4:30 PM - 6:30 PM LT MH1-2-3-4 #36</li>
 <li>Jiatao Gu, Shuangfei Zhai, Yizhe Zhang, Miguel Angel Bautista, Josh M. Susskind</li>
</ul>

### Friday, May&nbsp;5
<ul class="links-stacked">
<li>WORKSHOP</li>
 <li><a href="https://sites.google.com/view/africanlp2023/home">AfricaNLP</a></li>
 <li>10:00 AM - 7:00 PM LT</li>
 <li>Apple is sponsoring AfricaNLP</li>
</ul>

### Virtual Presentations
<ul class="links-stacked">
 <li>ICLR conference attendees can access Apple virtual paper presentations at any point after they register for the conference.</li>
 <li><a href="https://iclr.cc/virtual/2023/poster/10926">f-DM: A Multi-stage Diffusion Model via Progressive Signal Transformation</a></li>
 <li><a href="https://iclr.cc/virtual/2023/poster/11232">FastFill: Efficient Compatible Model Update</a></li>
 <li><a href="https://iclr.cc/virtual/2023/poster/11244">MAST: Masked Augmentation Subspace Training for Generalizable Self-Supervised Priors</a></li>
 <li><a href="https://iclr.cc/virtual/2023/poster/11709">RGI: Robust GAN-inversion for Mask-free Image Inpainting and Unsupervised Pixel-wise Anomaly Detection</a></li>
</ul>

</div> 

## Accepted Papers


### Conference Accepted Papers


[Continuous Pseudo-Labeling from the Start](/research/continuous-pseudo-labeling)


*Dan Berrebbi, Ronan Collobert, Samy Bengio, Navdeep Jaitly, Tatiana Likhomanenko*


[Diffusion Probabilistic Fields](/research/diffusion-probabilistic-fields)


*Peiye Zhuang, Samira Abnar, Jiatao Gu, Alexander Schwing, Josh M. Susskind, Miguel Angel Bautista*


[FastFill: Efficient Compatible Model Update](/research/fastfill)


*Florian Jaeckle, Fartash Faghri, Ali Farhadi, Oncel Tuzel, Hadi Pouransari*


[f-DM: A Multi-stage Diffusion Model via Progressive Signal Transformation](/research/multi-stage-diffusion)


*Jiatao Gu, Shuangfei Zhai, Yizhe Zhang, Miguel Angel Bautista, Josh M. Susskind *

[MAST: Masked Augmentation Subspace Training for Generalizable Self-Supervised Priors](/research/mast)


*Chen Huang, Hanlin Goh, Jiatao Gu, Josh M. Susskind*


[RGI: Robust GAN-inversion for Mask-free Image Inpainting and Unsupervised Pixel-wise Anomaly Detection](/research/robust-gan-inversion)


*Shancong Mou, Xiaoyi Gu, Meng Cao, Haoping Bai, Ping Huang, Jiulong Shan, Jianjun Shi*


## Acknowledgements
 
Samy Bengio is a senior area chair for ICLR 2023. 

Audra McMillan, Chen Huang, Barry Theobald, Hilal Asi, Luca Zappella, Miguel Angel Bautista, Pierre Ablin, Pau Rodriguez, Rin Susa, Samira Abnar, Tatiana Likhomanenko, Vaishaal Shankar, Vimal Thilak are reviewers for ICLR 2023.

Jon Shlens and Marco Cuturi are area chairs for ICLR 2023. 

<div class="callout">


Let's innovate together. Build amazing machine-learned experiences with Apple. Discover opportunities for researchers, students, and developers.

[Work with us.](/work-with-us)


</div>

Apple is sponsoring the International Conference on Learning Representations (ICLR), which will be held as a hybrid virtual and in person conference from May 1 - 5 in Kigali, Rwanda. ICLR brings together professionals dedicated to the advancement of deep learning. Below is the schedule of Apple sponsored workshops and events at ICLR 2023.

International Conference on Learning Representations (ICLR) 2023

Apple machine learning teams are engaged in state of the art research in machine learning and artificial intelligence. Learn about the latest advancements.

Overview

*= Equal Contributors

Artificial Intelligence (AI) and Machine Learning (ML) have made tremendous progress in the recent decade and have become ubiquitous in almost all application domains. Many recent advancements in the ease-of-use of ML frameworks and the low-code model training automations have further reduced the threshold for ML model building. As ML algorithms and pre-trained models become commodities, curating the appropriate training datasets and model evaluations remain critical challenges. However, these tasks are labor-intensive and require ML practitioners to have bespoke data skills. Based on the feedback from different ML projects, we built ADIML (Actionable Data Insights for ML) – a holistic data toolset. The goal is to democratize data-centric ML approaches by removing big data and distributed system barriers for engineers. We show in several case studies how the application of ADIML has helped solve specific data challenges and shorten the time to obtain actionable insights.


EuroMLSys

Actionable Data Insights for Machine Learning

  Virtual assistants are becoming increasingly important speech-driven Information Retrieval platforms that assist users with various tasks. We discuss open problems and challenges with respect to modeling spoken information queries for virtual assistants, and list opportunities where Information Retrieval methods and research can be applied to improve the quality of virtual assistant speech recognition. We discuss how query domain classification, knowledge graphs and user interaction data, and query personalization can be helpful  in improving the accurate recognition of spoken information domain queries. Finally, we also provide a brief overview of current problems and challenges in speech recognition.


Machine Learning
Research at Apple

Recent research

Actionable Data Insights for Machine Learning

Modeling Spoken Information Queries for Virtual Assistants: Open Problems, Challenges and Opportunities

Research articles

Improved Speech Recognition for People Who Stutter

3D Parametric Room Representation with RoomPlan

All events

International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2023

International Conference on Learning Representations (ICLR) 2023

Discover opportunities in Machine Learning.

Machine LearningResearch at Apple

Recent research

Actionable Data Insights for Machine Learning

Modeling Spoken Information Queries for Virtual Assistants: Open Problems, Challenges and Opportunities

Research articles

Improved Speech Recognition for People Who Stutter

3D Parametric Room Representation with RoomPlan

All events

International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2023

International Conference on Learning Representations (ICLR) 2023

Discover opportunities in Machine Learning.

Machine Learning
Research at Apple