Banks are hesitant to lend due to Fannie Mae's strict guidelines and the high fees and penalties that come along with defaults. The guidelines are adjusted based on changes in the housing market.