The speaker highlights some of the challenges faced in upskilling individuals during their time in prison, including lack of resources and knowledge gaps. He emphasized the loss of potential and talent due to these limitations.