Avoiding coffee during stressful events that can already trigger anxiety may be a good idea as caffeine can exacerbate the symptoms. A caffeine-induced anxiety disorder is also recognized in some cases.