The speaker suggests looking at your average view duration on a monthly or so basis to determine whether your numbers are moving in an upward or downward direction. He also mentions the amount of effort put into creating custom thumbnails for each video.