The Markov assumption is common in reinforcement learning as it makes the mathematics much easier to deal with. This assumption states that the next observation only depends on the previous observation, simplifying the learning process.