Using HTM for branch prediction

[Daniel_MK]

@vpuente

Really nice!

Thank you!

@vpuente

Take a look at Fig 3

First of all, thanks for the suggestion. I have not implemented this encoder, but I thought about it a bit and I don’t think it would be possible to perform this encoding process as fast as I would need it for this application. Computing the pseudo-random generator on-the-fly and several times for each SDR seems to be very complex to optimize for. Please, correct me if I’m mistaken.

@vpuente

(doesn’t need tables)

I was not even thinking about a table, but a hardwired solution. Since the synapses in the network are much more expensive to implement than the encoding process, I didn’t think it would be a problem to use this method.

[vpuente]

@Daniel_MK

Computing the pseudo-random generator on-the-fly and several times for each SDR seems to be very complex to optimize for. Please, correct me if I’m mistaken.

A HW random generator can be really simple (v.gr.using xor gates) [althouhg not very good]. In any case, TM will take you >100 of clock cycles. Since the CLA algorithm is easy “pipelineable”,… the encoding should be not an issue.

The benefit of this is that it can increase the history of the PC substantially.

@Daniel_MK

I was not even thinking about a table, but a hardwired solution. Since the synapses in the network are much more expensive to implement than the encoding process, I didn’t think it would be a problem to use this method.

That might be impractical if you want some flexibility. For example, if you need static partitioning (let say because you are using SMT). Additionally, wiring is not free. It might require less area but certainly will have a negative impact on power.

In any case, the work is really nice.

[Daniel_MK]

@vpuente

TM will take you >100 of clock cycles

Oh, are you sure there is not a way to make it faster? Even if it’s “pipelineable”, I don’t think it would ever make sense to even think about using TM for branch prediction if it takes that much time to run.

@vpuente

The benefit of this is that it can increase the history of the PC substantially.

That’s true, but according to my tests, the use of more history does not provide that much benefit = /

@vpuente

if you need static partitioning (let say because you are using SMT)

I’m sorry, I’m not familiar with this.

@vpuente

It might require less area but certainly will have a negative impact on power.

Yeah, it would require some really nice improvements in materials in order to work well.

@vpuente

In any case, the work is really nice.

Thank you very much! I’m glad you liked it.

[vpuente]

@Daniel_MK

Oh, are you sure there is not a way to make it faster?

Don’t think so. With 1024 mini-columns there is a lot of work to do each cycle. If you have a large budget in area and power, it could be improved, but hardly it will be faster than that. In any case, the LSTM paper shows this as a “theoretical” experience… my understanding is that your assumptions are in the same direction. [ And LSTM inference will be millions of clock cycles per “prediction” and with offline training]

The only way to make it faster is to use smaller systems (few tens of mini-columns) and have a hierarchy

@Daniel_MK

I’m sorry, I’m not familiar with this.

SMT stands for Simultaneous Multithreading. Some processors, such as Intel’s, statically partition per thread branch predictor tables (and history register) when you enable SMT (Intel call it Hyper-threading).

[Daniel_MK]

@vpuente

but hardly it will be faster than that

My understanding of the topic was a little off, then hahaha

@vpuente

the LSTM paper shows this as a “theoretical” experience… my understanding is that your assumptions are in the same direction

Yes, they were, but I didn’t expect it to be for that much hahaha

@vpuente

Some processors, such as Intel’s, statically partition per thread branch predictor tables (and history register) when you enable SMT

Right, I can see how hardwired solutions would prevent that.