The speaker narrates their struggle in finding funding to buy and renovate a deteriorating hotel in downtown San Jose, which eventually transformed into a popular destination but came with its own challenges.