Jean Kossaifi: “Efficient Tensor Representation for Deep Learning with TensorLy and PyTorch“

Tensor Methods and Emerging Applications to the Physical and Data Sciences 2021 Workshop IV: Efficient Tensor Representations for Learning and Computational Complexity “Efficient Tensor Representation for Deep Learning with TensorLy and PyTorch“ Jean Kossaifi - Nvidia Corporation Abstract: The data we manipulate in modern deep learning is inherently multi-dimensional. Preserving and leveraging that structure is crucial for good learning. Yet, this topological structure is typically discarded by existing models. By preserving and leveraging this structure using tensor methods, we can obtain better representations and enable better learning. This is particularly crucial when learning from spatiotemporal data or from structured data such as MRI. In this presentation, I will give an overview of tensor methods for deep learning for improved performance or speed, model compression and robustness. I will also cover practical implementation in PyTorch using TensorLy-Torch and show how to improve ResNet models for