The speaker discusses how one can be racist without actively participating in acts of violence or hatred. Instead, tolerance of racist behavior and prioritizing one's economic well-being over the well-being of people of other races are forms of racism.