The speaker compares the kind of people they met in luxury apartment hotels and hostels, stating that they generally met more intriguing individuals in hostels, such as backpackers, researchers, and entrepreneurs.