The key takeaway from a recent paper highlights the need to design neural network architectures that are memory-efficient in addition to being computationally cheap since most neural networks are characterized in terms of flops.