View publication

Applications such as autonomous navigation [1], human-robot interaction [2], game-playing robots [8], etc., use simulation to minimize the cost of testing in real world. Furthermore, some machine learning algorithms, like reinforcement learning, use simulation for training a model. To test reliably in simulation or deploy a model in the real world that is trained with simulated data, the simulator should be representative of the real environment. Usually, the simulator is based on manually designed rules and ignores the stochastic behavior of measurements. In particular, we would like to learn a model that captures uncertainties of the sensing algorithms (e.g. neural networks used to detect objects) in real world and add them in simulation. We model the distribution of residuals between the ground truth states of the objects and their perceived states by the sensing algorithm. This error distribution depends both on the current state of the object (e.g. distance from the sensor) and its past residuals. We assume the error distribution is conditionally Gaussian, and we use a deep neural neural network (DNN) to map the object states and past residuals to the distribution parameters (mean and variance). Our conditional model perturbs the dynamic objects’ states (position, velocities, orientations, and shape) and produces smoother trajectories which look similar to the real data.

Related readings and updates.

Learning from Simulated and Unsupervised Images through Adversarial Training

With recent progress in graphics, it has become more tractable to train models on synthetic images, potentially avoiding the need for expensive annotations. However, learning from synthetic images may not achieve the desired performance due to a gap between synthetic and real image distributions. To reduce this gap, we propose Simulated+Unsupervised (S+U) learning, where the task is to learn a model to improve the realism of a simulator's output…
See paper details

Improving the Realism of Synthetic Images

Most successful examples of neural nets today are trained with supervision. However, to achieve high accuracy, the training sets need to be large, diverse, and accurately annotated, which is costly. An alternative to labelling huge amounts of data is to use synthetic images from a simulator. This is cheap as there is no labeling cost, but the synthetic images may not be realistic enough, resulting in poor generalization on real test images. To help close this performance gap, we've developed a method for refining synthetic images to make them look more realistic. We show that training models on these refined images leads to significant improvements in accuracy on various machine learning tasks.

See article details