I described the use case I am using to test my RL system in this thread. As @sunguralikaan pointed out, timing isn’t exactly needed, since the agent could just learn to do some random actions to take up time. This is not aesthetically appealing though for a game agent. I also proposed another solution to improve the aesthetics by imposing a penalty for random motor actions except for one set of “motor” commands which merely changes the agents perspective without any outwardly visible actions. I think preferable to either of these workarounds would simply be to implement an actual timing mechanism.
Another area timing will be important for a game agent is for determining how fast an enemy is approaching or a platform is moving, in order to time jumps for example. The sequence of sensory inputs (assuming no repeating sequences) would be the same for a fast moving object compared to a slow moving one, unless a timing mechanism were added. (also assuming velocity isn’t captured in whatever encoder is being used… in other words, there is more than one way to skin a cat)