View publication

Self-supervised representation learning techniques utilize large datasets without semantic annotations to learn meaningful, universal features that can be conveniently transferred to solve a wide variety of downstream supervised tasks. In this paper, we propose a self-supervised method for learning representations of geographic locations from unlabeled GPS trajectories to solve downstream geospatial computer vision tasks. Tiles resulting from a raster representation of the earth's surface are modeled as nodes on a graph or pixels of an image. GPS trajectories are modeled as allowed Markovian paths on these nodes. A scalable and distributed algorithm is presented to compute image-like tensor representations, called reachability summaries, of the spatial connectivity patterns between tiles and their neighbors implied by the observed Markovian paths. A convolutional, contractive autoencoder is trained to learn compressed representations, called reachability embeddings, of reachability summaries for every tile. Reachability embeddings serve as task-agnostic, feature representations of geographic locations. Using reachability embeddings as pixel representations for five different downstream geospatial tasks, cast as supervised semantic segmentation problems, we quantitatively demonstrate that reachability embeddings are semantically meaningful representations and result in 4-23% gain in performance, as measured using area under the precision-recall curve (AUPRC) metric, when compared to baseline models that use pixel representations that do not account for the spatial connectivity between tiles. The design of reachability embeddings as pixel representations helps address the challenge of alignment and fusion in multimodal learning. Multimodal modeling of 3 different downstream geospatial tasks combining satellite imagery, mobility trajectories, and road network graph data yields 2-4% gain in performance as measured using the AUPRC metric compared to unimodal models for the same tasks. Reachability embeddings transform sequential, spatiotemporal motion trajectory data into semantically meaningful, image-like tensor representations that can be combined with other data modalities that are (e.g., satellite imagery) or can be transformed (e.g., road network graph, SAR imagery) into image-like tensor representations and are designed to facilitate multimodal learning in geospatial computer vision.

Related readings and updates.

Self-Supervised Temporal Analysis of Spatiotemporal Data

*=Equal Contributors There exists a correlation between geospatial activity temporal patterns and type of land use. A novel self-supervised approach is proposed to stratify landscape based on mobility activity time series. First, the time series signal is transformed to the frequency domain and then compressed into task-agnostic temporal embeddings by a contractive autoencoder, which preserves cyclic temporal patterns observed in time series. The…
See paper details

Do Self-Supervised and Supervised Methods Learn Similar Visual Representations?

*=Equal Contribution Despite the success of a number of recent techniques for visual self-supervised deep learning, there remains limited investigation into the representations that are ultimately learned. By using recent advances in comparing neural representations, we explore in this direction by comparing a constrastive self-supervised algorithm (SimCLR) to supervision for simple image data in a common architecture. We find that the methods…
See paper details