The inference process involves training a model on characters to explain pixels as the causes, utilizing causality in a logical sense. Although it appears to perform well locally, it might not provide an accurate result when observed in the context of other factors.