The grokking phenomenon as reported by Power et al., refers to a regime where a long period of overfitting is followed by a seemingly sudden transition to perfect generalization. In this paper, we attempt to reveal the underpinnings of Grokking via a series of empirical studies. Specifically, we uncover an optimization anomaly plaguing adaptive optimizers at extremely late stages of training, referred to as the Slingshot Mechanism. A prominent artifact of the Slingshot Mechanism can be measured by the cyclic phase transitions between stable and unstable training regimes, and can be easily monitored by the cyclic behavior of the norm of the last layers weights. We empirically observe that without explicit regularization, Grokking almost exclusively happens at the onset of Slingshots, and is absent without it. While common and easily reproduced in more general settings, the Slingshot Mechanism does not follow from any known optimization theories that we are aware of, and can be easily overlooked without an in depth examination. Our work points to a surprising and useful inductive bias of adaptive gradient optimizers at late stages of training, calling for a revised theoretical analysis of their origin. 

The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon

Neural implicit functions have recently shown promising results on surface reconstructions from multiple views. However, current methods still suffer from excessive time complexity and poor robustness when reconstructing unbounded or complex scenes. In this paper, we present RegSDF, which shows that proper point cloud supervisions and geometry regularizations are sufficient to produce high-quality and robust reconstruction results. Specifically, RegSDF takes an additional oriented point cloud as input, and optimizes a signed distance field and a surface light field within a differentiable rendering framework. We also introduce the two critical regularizations for this optimization. The first one is the Hessian regularization that smoothly diffuses the signed distance values to the entire distance field given noisy and incomplete input. And the second one is the minimal surface regularization that compactly interpolates and extrapolates the missing geometry. Extensive experiments are conducted on DTU, BlendedMVS, and Tanks and Temples datasets. Compared with recent neural surface reconstruction approaches, RegSDF is able to reconstruct surfaces with fine details even for open scenes with complex topologies and unstructured camera trajectories.

CVPR

Critical Regularizations for Neural Surface Reconstruction in the Wild

Scene analysis is an integral core technology that powers many features and experiences in the Apple ecosystem. From visual content search to powerful memories marking special occasions in one’s life, outputs (or "signals") produced by scene analysis are critical to how users interface with the photos on their devices. Deploying dedicated models for each of these individual features is inefficient as many of these models can benefit from sharing resources. We present how we developed Apple Neural Scene Analyzer (ANSA), a unified backbone to build and maintain scene analysis workflows in production. This was an important step towards enabling Apple to be among the first in the industry to deploy fully client-side scene analysis in 2016.

A Multi-Task Neural Architecture for On-Device Scene Analysis

This implementation is specifically optimized for the Apple Neural Engine (ANE), the energy-efficient and high-throughput engine for ML inference on Apple silicon. It will help developers minimize the impact of their ML inference workloads on app memory, app responsiveness, and device battery life. Increasing the adoption of on-device ML deployment will also benefit user privacy, since data for inference workloads remains on-device, not on the server.

In this article we share the principles behind this reference implementation to provide generalizable guidance to developers on optimizing their models for ANE execution. Then, we put these principles into action and showcase how to deploy an example pretrained Transformer model, the popular [Hugging Face distilbert](https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english), in just a few lines of code. Notably, this model, which works out-of-the-box and on device using Core ML already, is up to 10 times faster and consumes 14 times less memory after our optimizations. 

## The Transformer Architecture

Published in 2017, the [Transformer](https://arxiv.org/abs/1706.03762) architecture has had a great impact in many fields, including natural language processing and computer vision, in a short period of time. Models built on this architecture produce state-of-the-art results across a wide spectrum of tasks while incurring little to no domain-specific components. This versatility has resulted in a proliferation of applications built on this architecture. Most notably, the [Hugging Face model hub](https://huggingface.co/models) hosts tens of thousands of pretrained Transformer models, such as variants of [GPT-2](https://openai.com/blog/better-language-models/) and [BERT](https://arxiv.org/abs/1810.04805), which were trained and shared by the ML community. Some of these models average tens of millions of monthly downloads, contributing to the research momentum behind training bigger, better, and increasingly multitask Transformer models and giving birth to the development of efficient deployment strategies on both the [device side](https://github.com/huggingface/optimum) and the [server side](https://huggingface.co/blog/bert-inferentia-sagemaker). 

## The Apple Neural Engine

The first generation of the Apple Neural Engine (ANE) was released as part of the A11 chip found in iPhone X, our flagship model from 2017. It had a peak throughput of 0.6 teraflops (TFlops) in half-precision floating-point data format (float16 or FP16), and it efficiently powered on-device ML features such as Face ID and Memoji.

Fast-forward to 2021, and the fifth-generation of the 16-core ANE is capable of 26 times the processing power, or 15.8 TFlops, of the original. Since 2017, use of the ANE has been steadily increasing from a handful of Apple applications to numerous applications from both Apple and the developer community. The availability of the Neural Engine also expanded from only the iPhone in 2017 to iPad starting with the A12 chip and to Mac starting with the M1 chip.

<Figure className="inline" id="figure1" theme="dark-gray" caption={<>Figure 1: The evolution of the Apple Neural Engine, 2017 to 2021. The 16-core Neural Engine on the A15 Bionic chip on iPhone 13 Pro has a peak throughput of 15.8 teraflops, an increase of 26 times that of iPhone X.</>}>
 <Figure1 title="ANE Peak Throughput (FP16)" />
</Figure>

When training ML models, developers benefit from accelerated training on GPUs with [PyTorch](https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/) and [TensorFlow](https://developer.apple.com/metal/tensorflow-plugin/) by leveraging the Metal Performance Shaders (MPS) back end. For deployment of trained models on Apple devices, they use [coremltools](https://coremltools.readme.io/docs), Apple’s open-source unified conversion tool, to convert their favorite PyTorch and TensorFlow models to the Core ML model package format. Core ML then seamlessly blends CPU, GPU, and ANE (if available) to create the most effective hybrid execution plan exploiting all available engines on a given device. It lets a wide range of implementations of the same model architecture benefit from the ANE even if the entire execution cannot take place there due to idiosyncrasies of different implementations. This workflow is designed to make it easy for developers to deploy models on Apple devices without having to worry about the capabilities of any particular device or implementation.

## Principles Behind Optimizing Transformers for the Neural Engine 

Although the implementation flexibility that hybrid execution offers is simple and powerful, we can trade this flexibility off in favor of a particular and principled implementation that deliberately harnesses the ANE, resulting in significantly increased throughput and reduced memory consumption. Other benefits include mitigating the inter-engine context-transfer overhead and opening up the CPU and the GPU to execute non-ML workloads while ANE is executing the most demanding ML workloads.

In this section, we describe some of the generalizable principles behind our reference implementation for Transformers with the goal of empowering developers to optimize models they intend to deploy on the ANE.

### Principle 1: Picking the Right Data Format

In general, the Transformer architecture processes a 3D input tensor that comprises a batch of B sequences of S embedding vectors of dimensionality C. We represent this tensor in the (B, C, 1, S) data format because the most conducive data format for the ANE (hardware and software stack) is 4D and channels-first.

The native [torch.nn.Transformer](https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/transformer.py#L101) and many other PyTorch implementations use either the (B, S, C) or the (S, B, C) data formats, which are both channels-last and 3D data formats. These data formats are compatible with [nn.Linear layers](https://pytorch.org/docs/stable/generated/torch.nn.Linear.html), which constitute a major chunk of compute in the Transformer. To migrate to the desirable (B, C, 1, S) data format, we swap all [nn.Linear](https://pytorch.org/docs/stable/generated/torch.nn.Linear.html) layers with [nn.Conv2d layers](https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html). Furthermore, to preserve compatibility with previously trained checkpoints using the baseline implementation, we register a [load_state_dict_pre_hook](https://github.com/apple/ml-ane-transformers/blob/main/ane_transformers/huggingface/distilbert.py#L221) to automatically unsqueeze the nn.Linear weights twice in order to match the expected nn.Conv2d weights shape as shown [here](https://github.com/apple/ml-ane-transformers/blob/main/ane_transformers/huggingface/distilbert.py#L503).

The mapping of the sequence (S) axis to the last axis of the 4D data format is very important because the last axis of an ANE buffer is not packed; it must be contiguous and aligned to 64 bytes. This constraint applies only to the last axis, and the ANE compiler determines whether the rest of the axes are packed for best performance. Although unpacked buffers allow faster read times, if used improperly, they cause larger than necessary buffers to be allocated. For example, if the last axis is used as a singleton one by the model implementation’s data format, it will be padded to 64 bytes, which results in 32 times the memory cost in 16-bit and 64 times the memory cost in 8-bit precision. Such an increase in buffer size will significantly reduce the chance of L2 cache residency and increase the chance of hitting DRAM. This is not desirable from a power and latency perspective.

### Principle 2: Chunking Large Intermediate Tensors

For the multihead attention function in the Transformer, we [split](https://github.com/apple/ml-ane-transformers/blob/main/ane_transformers/reference/multihead_attention.py#L81) the query, key, and value tensors to create an explicit list of single-head attention functions, each of which operates on smaller chunks of input data. Smaller chunks increase the chance of L2 cache residency as well as increasing multicore utilization during compilation.

### Principle 3: Minimizing Memory Copies

Given that at least one axis of ANE buffers is not packed, reshape and transpose operations are likely to trigger memory copies unless specifically handled. In our reference implementation, we avoid all reshapes and incur only [one transpose](https://github.com/apple/ml-ane-transformers/blob/main/ane_transformers/reference/multihead_attention.py#L87) on the key tensor right before the query and key matmul. We avoid further reshape and transpose operations during the batched matrix multiplication operation for scaled dot-product attention by relying on a particular einsum operation. Einsum is a notational convention that implies summation over a set of indexed terms in a formula. We use the bchq,bkhc->bkhq [einsum formula](https://github.com/apple/ml-ane-transformers/blob/main/ane_transformers/reference/multihead_attention.py#L95), which represents a batched matmul operation whose data format directly maps to hardware without intermediate transpose and reshape operations. 

### Principle 4: Handling Bandwidth-Boundness

Even after all the optimizations, many Transformer configurations become bandwidth-bound on the ANE when the sequence length is relatively short. This is due to the fact that large parameter tensors are being fetched from memory, only to be applied on too few inputs before the next parameter tensor is fetched. Fetching from memory dominates overall latency in these cases. Bandwidth-boundness manifests very clearly in [Figure 2](#figure2). The optimized model’s latency stays approximately constant across sequence lengths of 32, 64, and 128 (with batch size 1) even though the computational load quadruples. One way to escape the bandwidth-bound regime is to increase the batch size for batch inference workloads. Another way is to reduce the parameter tensor size by quantization or pruning such that memory fetching becomes cheaper and faster.

## Principles Packaged into Code: ane_transformers

We share the reference implementation built on these principles and distribute it on PyPI to accelerate Transformers running on Apple devices with an ANE, on A14 and later or M1 and later chips. The package is called ane_transformers and the first on-device application using this package was [HyperDETR](https://machinelearning.apple.com/research/panoptic-segmentation), as described in our previous article.



Next, we showcase the application of these principles to a pretrained Transformer model: [distilbert from Hugging Face](https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english?text=I+like+you.+I+love+you). Code for this example is also made available through ane_transformers.

## Case Study: Hugging Face distilbert

As the scaling laws of deep learning continue to hold, the ML community is training bigger and more capable Transformer models. However, most open-source implementations of the Transformer architecture are optimized for either large-scale training hardware or no hardware at all. Furthermore, despite significant concurrent advancements in model compression, the state-of-the-art models are scaling faster than compression techniques are improving.

Hence, there is immense need for on-device inference optimizations to translate these research gains into practice. These optimizations will enable ML practitioners to deploy much larger models on the same input set, or to deploy the same models to run on much larger sets of inputs within the same compute budget.

In this spirit, we take the principles behind our open-source reference PyTorch implementation for the Transformer architecture and apply them to the popular distilbert model from the Hugging Face model hub in a [case study](https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english?text=I+like+you.+I+love+you). Applying these optimizations resulted in a forward pass that is up to 10 times faster with a simultaneous reduction of peak-memory consumption of 14 times on iPhone 13. Using our reference implementation, on a sequence length of 128 and a batch size of 1, the iPhone 13 ANE achieves an average latency of 3.47 ms at 0.454 W and 9.44 ms at 0.072 W. Even using our reference implementation, the ANE peak throughput is far from being saturated for this particular model configuration, and the performance may be further improved with using the quantization and pruning techniques as discussed earlier.

<Figure
 className="full-bleed"
 id="figure2"
 theme="dark-gray"
 caption={<>Figure 2: Performance curves (latency, memory) for Hugging Face distilbert on various Apple devices and operating system versions. Shaded ranges indicate performance under varying power consumption.</>}
>
 <Figure2 />
</Figure>

To contextualize the numbers we just reported, a recent [article](https://huggingface.co/blog/bert-inferentia-sagemaker) from Hugging Face and AWS reported “the average latency . . . is 5-6 ms for a sequence length of 128” for the same model in our case study, when deployed on the server side using [ML-optimized ASIC hardware](https://aws.amazon.com/machine-learning/inferentia/?nc1=h_ls) from AWS. Based on this external data point, we are happy to report that our on-device inference latency compares favorably to those measured on an optimized server-side inference engine while executing on a device that is orders of magnitude more energy-constrained. [Figure 2](#figure2) shows the latency and memory consumption of the same model across different sequence lengths, batch sizes, and devices.

## Putting It All Together: From PyTorch to Xcode

Finally, we show how these optimizations are applied through just a few lines of code, and then we profile the model in Xcode using the new Core ML Performance Report feature available in Xcode 14.

To begin, we initialize the baseline [distilbert model](https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english) from the Hugging Face model hub:

```python
import transformers
model_name = "distilbert-base-uncased-finetuned-sst-2-english"
baseline_model = transformers.AutoModelForSequenceClassification.from_pretrained(
 model_name,
 return_dict=False,
 torchscript=True,
).eval()
```

Then we initialize the optimized model, and we restore its parameters using that of the baseline model to achieve high output parity as measured by peak-signal-to-noise-ratio (PSNR): 

```python
from ane_transformers.huggingface import distilbert as ane_distilbert
optimized_model = ane_distilbert.DistilBertForSequenceClassification(
 baseline_model.config).eval()
optimized_model.load_state_dict(baseline_model.state_dict())
```

Next we create sample inputs for the model:

```python
tokenizer = transformers.AutoTokenizer.from_pretrained(model_name)
tokenized = tokenizer(
 ["Sample input text to trace the model"],
 return_tensors="pt",
 max_length=128, # token sequence length
 padding="max_length",
)
```

We then trace the optimized model to obtain the [expected model format (TorchScript)](https://coremltools.readme.io/docs/pytorch-conversion#generate-a-torchscript-version) for the [coremltools](https://coremltools.readme.io/docs) conversion tool.

```python
import torch
traced_optimized_model = torch.jit.trace(
 optimized_model,
 (tokenized["input_ids"], tokenized["attention_mask"])
)
```

Finally, we use coremltools to generate the Core ML model package file and save it.

```python
import coremltools as ct
import numpy as np
ane_mlpackage_obj = ct.convert(
 traced_optimized_model,
 convert_to="mlprogram",
 inputs=[
 ct.TensorType(
 f"input_{name}",
 shape=tensor.shape,
 dtype=np.int32,
 ) for name, tensor in tokenized.items()
 ],
)
out_path = "HuggingFace_ane_transformers_distilbert_seqLen128_batchSize1.mlpackage"
ane_mlpackage_obj.save(out_path)
```

To verify performance, developers can now launch Xcode and simply add this model package file as a resource in their projects. After clicking on the Performance tab, the developer can generate a performance report on locally available devices, for example, on the Mac that is running Xcode or another Apple device that is connected to that Mac. [Figure 3](#figure3) shows a performance report generated for this model on an iPhone 13 Pro Max with iOS 16.0 installed.

<Figure
 className="full-bleed"
 id="figure3"
 theme="dark-gray"
 caption={<>Figure 3: Core ML performance reports generated in Xcode. Developers can review runtime statistics, layer dispatch to each engine, and accomplish additional tasks.</>}
 data-aos
>
 <Figure3 title="Xcode Core ML Performance Report" />
</Figure>

# Conclusion 

Transformers are becoming ubiquitous in ML as their capabilities scale up with their size. Deploying Transformers on-devices requires efficient strategies, and we are thrilled to provide guidance to developers on this topic. Learn more about the implementation described in this post on the [Machine Learning ANE Transformers GitHub.](https://github.com/apple/ml-ane-transformers) 

# Acknowledgments

 Many people contributed to this work, including Atila Orhon, Aseem Wadhwa, Youchang Kim, Francesco Rossi, and Vignesh Jagadeesh. 

# Resources

Apple Developer. “Framework: Core ML.” 2022. [[link.]](https://developer.apple.com/documentation/coreml)

Apple Developer. “Getting Started with tensorflow-metal PluggableDevice.” 2022. [[link.]](https://developer.apple.com/metal/tensorflow-plugin/)

Apple Developer. "Accelerate machine learning with Metal." 2022. [[link.]](developer.apple.com/wwdc22/10063)

“Apple Neural Engine Transformers.” GitHub. 2022. [[link.]](https://github.com/apple/ml-ane-transformers)

Hugging Face. “Models.” 2022. [[link.]](https://huggingface.co/models)

PyTorch. “From Research to Production.” 2022. [[link.]](https://pytorch.org/index.html#news-items)

PyTorch. “Introducing Accelerated PyTorch Training on Mac.” 2022. [[link.]](https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/)

“Trending Research.” Papers with Code. 2022. [[link.]](https://paperswithcode.com)

# References

Apple. “On-Device Panoptic Segmentation for Camera Using Transformers.” Machine Learning Research, October 2021. [link.](https://machinelearning.apple.com/research/panoptic-segmentation)

Apple. “Use VoiceOver for Images and Videos on iPhone.” iPhone User Guide. Cupertino, CA: Apple, 2022. [[link.]](https://support.apple.com/guide/iphone/use-voiceover-for-images-and-videos-iph37e6b3844/ios)

Core ML Tools. “Introduction: Use coremltools to Convert Models from Third-Party Libraries to Core ML.” 2022. [[link.]](https://coremltools.readme.io/docs)

Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.” 2019. [[link.]](https://arxiv.org/abs/1810.04805)

OpenAI. “Better Language Models and Their Implications.” February 14, 2019. [[link.]](https://openai.com/blog/better-language-models/)

Schmid, Philipp. “Accelerate BERT Inference with Hugging Face Transformers and AWS Inferentia.” Hugging Face, 2022. [[link.]](https://huggingface.co/blog/bert-inferentia-sagemaker)

Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. “Attention Is All You Need.” 2017. [[link.]](https://arxiv.org/abs/1706.03762)

An increasing number of the machine learning (ML) models we build at Apple each year are either partly or fully adopting the [Transformer architecture](https://arxiv.org/abs/1706.03762). This architecture helps enable experiences such as [panoptic segmentation in Camera with HyperDETR](https://machinelearning.apple.com/research/panoptic-segmentation), [on-device scene analysis in Photos](/research/on-device-scene-analysis), [image captioning for accessibility](https://support.apple.com/guide/iphone/use-voiceover-for-images-and-videos-iph37e6b3844/ios), [machine translation](https://apps.apple.com/us/app/translate/id1514844618), and many others. This year at WWDC 2022, Apple is making available an open-source reference [PyTorch](https://pytorch.org) implementation of the Transformer architecture, giving developers worldwide a way to seamlessly deploy their state-of-the-art Transformer models on Apple devices.

Deploying Transformers on the Apple Neural Engine

Apple machine learning teams are engaged in state of the art research in machine learning and artificial intelligence. Learn about the latest advancements.

Overview

In this post we will introduce a new dataset for community benchmarking in PPML, and share highlights from workshop discussions and recordings of select workshop talks. 

## Introducing the Federated Learning Annotated Image Repository (FLAIR) Dataset for PPML Benchmarking

<Figure
 id="figure0"
 caption={<>Sample images from the dataset with associated labels.</>}
>
<a href="https://github.com/apple/ml-flair" aria-label="MOCK-FLAIR" tabindex="-1" target="_blank" className="bg-gray-light"><img src="https://mlr.cdn-apple.com/media/Mock_FLAIR_main_04e98dafe9.png" alt="MOCK-FLAIR-main" /></a>
</Figure>

Significant advances in machine learning (ML) over the last decade have been driven in part by the increased accessibility of both large-scale computing and training data. By competing to improve standard benchmarks on datasets like [ImageNet](https://image-net.org), the community has discovered an array of improved techniques, optimizations, and model architectures that are broadly applicable and of proven value.

The PPML field would benefit from adopting standard datasets and tasks for benchmarking advances in the field. Benchmarks serve at least two important roles: 

- Focusing research efforts on what the community has identified as important challenges
- Providing a common, agreed-upon method to measure and compare privacy, utility, and efficiency techniques

The datasets and tasks of any proposed PPML benchmark must represent real-world problems, and capture aspects that might complicate PPML training like class imbalance and label errors. They must permit training models that balance strong privacy guarantees with high-enough accuracy to be practically useful.
And ideally they would be a fit for large-scale and nonlinear models, simple convex models, and training regimes with differing amounts of supervision. Most importantly, proposed PPML benchmarks must support the evaluation of models trained in a federated setting where training is distributed across user devices. 

FLAIR is a dataset of 429,078 Flickr images grouped by 51,414 users that accurately captures a number of the characteristics encountered in PPML tasks. These images have been graded by humans and labeled using a taxonomy of approximately 1,600 labels. All objects present in the images have been labeled, and images may have multiple labels. Because the taxonomy is hierarchical, researchers can easily vary the complexity of benchmark and ML tasks they construct using the dataset.

<Figure
 id="figure1"
 className="inline"
 caption={<></>}
>
<a href="https://github.com/apple/ml-flair" aria-label="MOCK-FLAIR" tabindex="-1" target="_blank" className="bg-gray-light"><img src="https://mlr.cdn-apple.com/media/Mock_FLAI_Rdataset_99ade7a919.png" alt="MOCK-FLAIR" className="bg-gray-light"/></a>
</Figure>

## ML with Privacy 

A central subject of the workshop was federated learning with differential privacy. Many practical use cases would benefit from algorithms that can handle data heterogeneity, as well as better optimization methods for private federated learning. These topics were discussed in Virginia Smith's talk (Carnegie Mellon University), [Privacy Meets Heterogeneity.](/video/privacy-heterogeneity) Several future directions, open questions, and challenges in applying PPML in practice were highlighted in Kunal Talwar's (Apple) talk, [Open Questions in Differentially Private Machine Learning](/video/open-questions).

An in-depth discussion of meaningful privacy guarantees for language models was discussed in Florian Tramèr's talk (ETH Zurich) [What Does it Mean for a Language Model to Preserve Privacy?](/video/preserve-privacy) Tramèr highlighted the need for revisiting the assumptions of existing techniques to ensure that users’ expectations of privacy are not violated. There was also a fruitful discussion about membership inference attacks, which represent one approach for using trained models to understand user data privacy. Nicholas Carlini (Google) added to this discussion with his talk, [Membership Inference Attacks from First Principles](/video/inference-attacks).

<Figure
 id="figure2" className="full-bleed"
>
<VideoCarousel 
 title="ML with Privacy Videos"
 items={[
 {
 title: 'Privacy Meets Heterogeneity',
 subtitle: 'Presented by Virginia Smith',
 link: '/video/privacy-heterogeneity'
 },
 {
 title: 'Open Questions in Differentially Private Machine Learning',
 subtitle: 'Presented by Kunal Talwar',
 link: '/video/open-questions'
 }, 
 {
 title: 'What Does it Mean for a Language Model to Preserve Privacy?',
 subtitle: 'Presented by Florian Tramèr',
 link: '/video/preserve-privacy'
 },
 {
 title: 'Membership Inference Attacks from First Principles',
 subtitle: 'Presented by Nicholas Carlini',
 link: '/video/inference-attacks'
 }
 ]}
 />
</Figure>

## Learning Statistics with Privacy 

Attendees discussed many critical topics on learning statistics with privacy. Large-scale deployment of differential privacy (DP) was highlighted in Ryan Rogers' paper (LinkedIn) [Practical Differentially Private Top-k Selection with Pay-what-you-get Composition.](https://arxiv.org/abs/1905.04273) Attendees discussed budget management schemes and accumulation of privacy loss when querying the same database multiple times. 

Attendees also reviewed low-communication DP algorithms and other related works, including Jelani Nelson’s work (UC Berkeley, Google) on new algorithms for private frequency estimation that improves on the performance of other schemes such as [PI-RAPPOR.](/research/lossless-compression) Participants discussed learning unknown domains with DP both in central and local models. 

Finally, we discussed the optimal DP algorithms for certain basic statistical tasks, specifically the paper by Gautam Kamath (University of Waterloo), [Efficient Mean Estimation with Pure Differential Privacy via a Sum-of-Squares Exponential Mechanism.](https://arxiv.org/abs/2111.12981) 

## Trust Models, Attacks, and Accounting 

Another area explored at the workshop was that of understanding the system-level view of risks to users' data privacy requires modeling the entire data processing pipeline and the different levels of trust that users may have in the components of the pipeline. For example, the pipeline may include code that processes the data on the phone; a server that performs, aggregates, and anonymizes; and a server that stores the final results of the analysis. Different levels of trust can be translated into different privacy requirements for individual components of the system. Specific systems that were discussed during the breakout session included shuffling, aggregation, and secret sharing.

Another system-wide concern is privacy risks resulting from publication and storage of results of multiple data analyses performed on the data of the same user. Understanding such risks requires strengthening existing privacy accounting techniques as well as developing new approaches to privacy accounting. The attendees discussed relevant work in this area, including the paper [Tight Accounting in the Shuffle Model of Differential Privacy](https://arxiv.org/abs/2106.00477) by workshop attendee Antti Honkela (University of Helsinki).

Better understanding of privacy risks requires developing new potential attacks on privacy. The paper [Auditing Private Machine Learning](https://arxiv.org/pdf/2006.07709.pdf) by Alina Oprea (Northeastern University) proposes new tools for deriving tighter bounds on the privacy loss due to publishing of intermediate models output by stochastic gradient descent. 

<Figure
 id="figure3" className="full-bleed"
>
<VideoCarousel 
 title="Trust Models, Attacks, and Accounting Videos"
 items={[
 {
 title: 'Widespread Underestimation of Sensitivity in DP Libraries',
 subtitle: 'Presented by Salil Vadhan',
 link: '/video/widespread-underestimation'
 },
 {
 title: 'When the Curious Abandon Honesty',
 subtitle: 'Presented by Nicolas Papernot',
 link: '/video/curious-honesty'
 }
 ]}
 />
</Figure> 
 

## Other Primitives

Finally, the workshop discussed cryptographic primitives that can allow for implementation of differentially private algorithms without relying on a trusted data curator. Shuffling or aggregation is a primitive that has been shown in recent years to allow for a large class of differentially private algorithms to be implemented. For example, a trusted shuffler can allow the researcher to compute private histograms with accuracy that nearly matches the central model. 

Vitaly Feldman (Apple) explored this topic in his talk, [Low-Communication Algorithms for Private Federated Data Analysis with Optimal Accuracy Guarantees](/video/low-communication-algorithm). Workshop attendees discussed the question of implementing this primitive and compared the efficiency and trust assumptions of various approaches to building a shuffler in terms of efficiency and trust assumptions. Although shuffling is useful, there are private computations that may not be implemented in the shuffle model without losing accuracy. In principle, some simple primitives are universal for secure multiparty computation. The attendees discussed the question of finding primitives that can be practically implemented and allow for efficient implementations of other differentially private mechanisms, as referenced in works such as [Mechanism Design via Differential Privacy](https://ieeexplore.ieee.org/document/4389483) by Kunal Talwar (Apple). 

<Figure
 id="figure4" className="full-bleed"
>
<VideoCarousel 
 title="Other Primitives Videos"
 items={[
 {
 title: 'Low-Communication Algorithms for Private Federated Data Analysis with Optimal Accuracy Guarantees',
 subtitle: 'Presented by Vitaly Feldman',
 link: '/video/low-communication-algorithm'
 },
 {
 title: ' Open Questions in Differentially Private Machine Learning',
 subtitle: 'Presented by Kunal Talwar',
 link: '/video/open-questions'
 }
 ]}
 />
</Figure> 
 

## Workshop Resources 

### Related Videos 

<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/play_circle_9d41fa88a8.svg" /><a href="/video/open-questions">Open Questions in Differentially Private Machine Learning by Kunal Talwar</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/play_circle_9d41fa88a8.svg" /><a href="/video/privacy-heterogeneity">Privacy Meets Heterogeneity by Virginia Smith</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/play_circle_9d41fa88a8.svg" /><a href="/video/preserve-privacy">What Does it Mean for a Language Model to Preserve Privacy? by Florian Tramèr</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/play_circle_9d41fa88a8.svg" /><a href="/video/inference-attacks">Membership Inference Attacks from First Principles” by Nicholas Carlini</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/play_circle_9d41fa88a8.svg" /><a href="/video/widespread-underestimation">Widespread Underestimation of Sensitivity in DP Libraries, and How to Fix It by Salil Vadhan</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/play_circle_9d41fa88a8.svg" /><a href="/video/curious-honesty">When the Curious Abandon Honesty by Nicolas Papernot</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/play_circle_9d41fa88a8.svg" /><a href="/video/low-communication-algorithm">Low-Communication Algorithms for Private Federated Data Analysis with Optimal Accuracy Guarantees by Vitaly Feldman</a>

### Related Papers

<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/doc_feeba796f7.svg" /><a href="https://arxiv.org/pdf/2006.07709.pdf">Auditing Private Machine Learning by Matthew Jagielski, Jonathan Ulman, and Alina Oprea</a> 
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/doc_feeba796f7.svg" /><a href="https://arxiv.org/abs/2202.11205">Constant matters: Fine-grained Complexity of Differentially Private Continual Observation by Hendrik Fichtenberger, Monika Henzinger, and Jalaj Upadhyay</a> 
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/doc_feeba796f7.svg" /><a href="https://arxiv.org/abs/2002.05839">LinkedIn's Audience Engagements API: A Privacy Preserving Data Analytics System at Scale by Ryan Rogers, Subbu Subramaniam, Sean Peng, David Durfee, Seunghyun Lee, Santosh Kumar Kancha, Shraddha Sahay, Parvez Ahammad</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/doc_feeba796f7.svg" /><a href="https://arxiv.org/abs/2109.11429">Robin Hood and Matthew Effects: Differential Privacy Has Disparate Impact on Synthetic Data by Georgi Ganev, Bristena Oprisanu, and Emiliano De Cristofaro</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/doc_feeba796f7.svg" /><a href="https://arxiv.org/abs/2110.06500">Differentially Private Fine-tuning of Language Models by Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A. Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, Sergey Yekhanin, and Huishuai Zhang</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/doc_feeba796f7.svg" /><a href="https://proceedings.neurips.cc/paper/2021/hash/f2b5e92f61b6de923b063588ee6e7c48-Abstract.html">Differentially Private Sampling from Distributions by Sofya Raskhodnikova, Satchit Sivakumar, Adam Smith, and Marika Swanberg</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/doc_feeba796f7.svg" /><a href=" https://openreview.net/forum?id=8ygF02Zm51q">EF21: A New, Simpler, Theoretically better, and Practically Faster Error Feedback by Peter Richtárik, Igor Sokolov, Ilyas Fatkhullin</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/doc_feeba796f7.svg" /><a href="https://arxiv.org/abs/2109.08604">Enforcing Fairness in Private Federated Learning via the Modified Method of Differential Multipliers by Borja Rodríguez-Gálvez, Filip Granqvist, Rogier van Dalen, and Matt Seigel</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/doc_feeba796f7.svg" /><a href="https://arxiv.org/abs/2203.10135">Learning Compressed Embeddings for On-Device Inference by Niketan Pansare, Jay Katukuri, Aditya Arora, Frank Cipollone, Riyaaz Shaik, Noyan Tokgozoglu, Chandru Venkataraman</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/doc_feeba796f7.svg" /><a href="https://arxiv.org/abs/2010.08688">Locally Differentially Private Analysis of Graph Statistics by Jacob Imola, Takao Murakami, and Kamalika Chaudhuri</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/doc_feeba796f7.svg" /><a href="https://ieeexplore.ieee.org/document/4389483">Mechanism Design via Differential Privacy by Frank McSherry and Kunal Talwar</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/doc_feeba796f7.svg" /><a href="https://arxiv.org/abs/2201.08430">Reproducibility in Learning by Russell Impagliazzo, Rex Lei, Toniann Pitassi, and Jessica Sorrell</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/doc_feeba796f7.svg" /><a href="https://eprint.iacr.org/2022/177.pdf">The Power of the Differentially Oblivious Shuffle in Distributed Privacy Mechanisms by Mingxun Zhou and Elaine Shi</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/doc_feeba796f7.svg" /><a href="https://arxiv.org/abs/2106.00477">Tight Accounting in the Shuffle Model of Differential Privacy by Antti Koskela, Mikko A. Heikkilä, and Antti Honkela</a> 
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/doc_feeba796f7.svg" /><a href=" https://arxiv.org/abs/2203.09943">Training a Tokenizer for Free with Private Federated Learning by Eugene Bagdasaryan, Congzheng Song, Rogier van Dalen, Matt Seigel, Áine Cahill</a>
<img style={{width: '15px', height: '15px', marginRight: '5px'}} src="https://mlr.cdn-apple.com/media/doc_feeba796f7.svg" /><a href="https://arxiv.org/abs/2110.11208">User-Level Private Learning via Correlated Samplin by Badih Ghazi, Ravi Kumar, and Pasin Manurangsi</a> 

# Acknowledgments 

Many people contributed to this work including Kunal Talwar, Vitaly Feldman, Omid Javidbakht, Martin Pelikan, Audra McMillan, Allegra Latimer, Niketan Pansare, Roman Filipovich, Eugene Bagdasaryan, Congzheng Song, Áine Cahill, Jay Katukuri, Aditya Arora, Frank Cipollone, Borja Rodríguez-Gálvez, Filip Granqvist, Riyaaz Shaik, Matt Seigel, Noyan Tokgozoglu, Chandru Venkataraman, David Betancourt, Ben Everson, Nima Reyhani, Babak Aghazadeh, Kanishka Bhaduri, and John Akred.


<div className="callout">

Let's innovate together. Build amazing privacy-preserving machine-learned experiences with Apple. Discover opportunities for researchers, students, and developers by visiting our [Work With us page](/work-with-us). 

</div>

Earlier this year, Apple hosted the Workshop on Privacy-Preserving Machine Learning (PPML). This virtual event brought Apple and members of the academic research communities together to discuss the state of the art in the field of privacy-preserving machine learning through a series of talks and discussions over two days.

Apple Privacy-Preserving Machine Learning Workshop 2022 

All CVPR attendees are invited to stop by the [Apple booth](https://hallerickson.ungerboeck.com/prod/app85.cshtml?aat=Cr9Jl1p0qL33RLclLrfrlBKRmhROPXGkDwcQAVv7zHM%3d) to check out our demos and chat with available recruiters and booth staff.

<div class="callout">

## Schedule

Below is the schedule of Apple sponsored workshops and accepted papers. Visit the [CVPR 2022 website](https://cvpr2022.thecvf.com/overview) for a full conference schedule.

### Sunday June&nbsp;19 
<ul class="links-stacked">
	<li>WORKSHOP</li>
	<li><a href="https://sites.google.com/view/wicvcvpr2022/home">Women in Computer Vision Workshop</a></li>
	<li>Parita Pooj and Lizi Ottens will be participating in the mentorship event as mentors.</li>
	<li>WORKSHOP</li>
	<li><a href="https://www.latinxinai.org/cvpr-2022-about">LatinX in Computer Vision Research</a></li>
	<li>Marco Cuturi will give a virtual networking speech during the workshop. David Güera Cobo will be participating in the mentoring hour as a mentor.</li>
 <li>WORKSHOP</li>
	<li><a href="http://compression.cc">CLIC 2022 - Workshop Challenge on Learned Image Compression</a></li>
	<li>ACCEPTED PAPER</li>
	<li><a href="/research/neural-face-video">Neural Face Video Compression using Multiple Views</a></li> 
	<li>This workshop was co-organized by Zeina Sinno, Krishna Rapaka, and Erfan Noury.</li>
 <li>WORKSHOP</li>
	<li><a href="https://syntml-cvpr2022-workshop.github.io/">SyntML</a></li>
	<li>This workshop was co-organized by Oncel Tuzel, Russ Webb, and Vladlen Koltun is an invited speaker.</li>
</ul>

### Monday June&nbsp;20 
<ul class="links-stacked">
	<li>WORKSHOP</li>
	<li><a href="https://vizwiz.org/workshops/2022-workshop/">VizWiz Grand Challenge Workshop</a></li> 
</ul>

### Tuesday June&nbsp;21
<ul class="links-stacked">
	<li>ACCEPTED PAPER</li> 
	<li><a href="/research/robust-joint-shape">Robust Joint Shape and Pose Optimization for Few-View Object Reconstruction</a></li>
</ul>

### Wednesday June&nbsp;22
<ul class="links-stacked">
	<li>ACCEPTED PAPER</li>
	<li><a href="/research/multi-view-stereo">Efficient Multi-View Stereo via Attention-Driven 2D Convolutions</a></li>
	<li><a href="/research/critical-regularizations">Critical Regularizations for Neural Surface Reconstruction in the Wild</a></li>
</ul>

### Friday June&nbsp;24
<ul class="links-stacked">
	<li>ACCEPTED PAPER</li>
	<li><a href="/research/forward-compatible-training">Forward Compatible Training for Large-Scale Embedding Retrieval Systems</a></li>
</ul>

</div>

## Accepted Papers

### Conference Accepted Papers

[Critical Regularizations for Neural Surface Reconstruction in the Wild](/research/critical-regularizations)

*Jingyang Zhang, Yao Yao, Shiwei Li, Tian Fang, David McKinnon, Yanghai Tsin, Long Quan*

[Forward Compatible Training for Large-Scale Embedding Retrieval Systems](/research/forward-compatible-training)

*Vivek Ramanujan, Hadi Pour Ansari, Pavan Kumar Anasosalu Vasu, Oncel Tuzel, Ali Farhadi*

[Robust Joint Shape and Pose Optimization for Few-View Object Reconstruction](/research/robust-joint-shape)

*Zhenpei Yang, Zhile Ren, Miguel Angel Bautista, Zaiwei Zhang, Qi Shan, Qixing Huang* 

[Efficient Multi-View Stereo via Attention-Driven 2D Convolutions](/research/multi-view-stereo)

*Zhenpei Yang, Zhile Ren, Qi Shan, Qixing Huang*

### Workshop Accepted Papers

[Neural Face Video Compression using Multiple Views](/research/neural-face-video)

*Anna Volokitin, Stefan Brugger, Ali Benlalah, Sebastian Martin, Brian Amberg, Michael Tschannen*

## Demos

**RoomPlan**

[RoomPlan](https://developer.apple.com/augmented-reality/roomplan/) technology allows the user to captures a room and it's defining objects in a parametric format within minutes. Capture progress is automatically displayed and easy to understand at a glance. The resulting room capture is provided as a parametric representation of the room and can optionally be exported to USD, USDA or USDZ formats. The technology is supported on any of the Apple devices (iPad and iPhone) that are equipped with LiDAR sensor.
 
**Systemwide Visual Intelligence**

This demo aims to illustrate a few of the user facing Visual Intelligence capabilities in our operating systems. A distinguishing factor of the workflows are that they are deeply integrated into the operating system, and run efficiently on device. We showcase two styles of interaction with visual content:

- Whole images through accessibility workflows for image captioning
- Text in static images and streaming camera through LiveText

Image Captioning was introduced in iOS 14 through Voiceover, and was enriched in iOS 15 through Image Explorer. Live Text was introduced in iOS 15 through static image and camera based workflows.

All CVPR attendees are invited to stop by the [Apple booth](https://hallerickson.ungerboeck.com/prod/app85.cshtml?aat=Cr9Jl1p0qL33RLclLrfrlBKRmhROPXGkDwcQAVv7zHM%3d) to experience these demos in person.

## Acknowledgements

Qi Shan is a CVPR 2022 Area Chair.

Alex Colburn, Angelos Katharopoulos, James Chen, Winston Wang, and Zhile Ren are members of the CVPR 2022 review board.

Long Quan is a CVPR 2022 General Chair.

Ali Farhardi is a member of the [Embodied AI](https://embodied-ai.org/#organizers) workshop Scientific Advisory Board.

Mahmoud Afifi is a member of the [NTIRE 2022](https://data.vision.ee.ethz.ch/cvl/ntire22/) workshop program committee.

David Güera Cobo is a member of the [Workshop On Media Forensics 2022](https://sites.google.com/view/mediaforensics2022/committee) technical committee. 

<div class="callout">

Let's innovate together. Build amazing machine-learned experiences with Apple. Discover opportunities for researchers, students, and developers by visiting our [Work With Us page.](/work-with-us)

</div>

Apple is sponsoring the IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), which will be held in New Orleans, Louisiana from June 19 to 24. CVPR is an annual computer vision conference with several workshops and short courses.

Machine Learning
Research at Apple.

Recent research.

The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon

Critical Regularizations for Neural Surface Reconstruction in the Wild

A Multi-Task Neural Architecture for On-Device Scene Analysis

Deploying Transformers on the Apple Neural Engine

Updates and events.

Apple Privacy-Preserving Machine Learning Workshop 2022

CVPR 2022

Discover opportunities in Machine Learning.

Machine LearningResearch at Apple.

Recent research.

The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon

Critical Regularizations for Neural Surface Reconstruction in the Wild

A Multi-Task Neural Architecture for On-Device Scene Analysis

Deploying Transformers on the Apple Neural Engine

Updates and events.

Apple Privacy-Preserving Machine Learning Workshop 2022

CVPR 2022

Discover opportunities in Machine Learning.

Machine Learning
Research at Apple.