The challenge is to represent the different types of knowledge needed for various learning, such as declarative, procedural, and attentional, in a way that allows for cross-learning to occur between different types of memory. This involves finding a way to make representations of maps in the memory matrix work together with representations needed for visual pattern recognition in neural networks.