The speaker emphasizes the importance of considering the people impacted by business decisions instead of just checking a number box, citing an example of a poorly constructed store that caused issues for employees.