You must realize that encoding the timestamp is sortof a hack. HTM has an innate sense of temporal flow. One input always comes after another input, and the order is always important. So it’s not really that is assumes each point is for NOW() in time, just that one comes after another.
Just think what it would be like if your sensory organs started sending their inputs out of order into your brain!