Current approaches to self-supervisioning in NLP are limited by the Independent distributions of words and a lack of proper representation of distributions over combinatorial sequences of symbols, which grow exponentially with the length of the symbols.