Please review my HTM implementation in Rust

Engineering Implementations
,
1

Please, review my implementation.
It’s prototype.

4 Likes
2

Hello, I am more than happy to help
But I’m not very familiar with Rust. How could I build the application?

3
  1. git clone the project.
  2. cd proj_directory.
  3. cargo build.
4

Thanks. :smiley:

1 Like
5

Not being a Rust programmer. I can only try my best.
Since you asked for a review. I’ll do a full code review! Not trying to be harsh. And fell free to justify you decisions by any means!

  1. Well, the main() function is empty. I can’t profile anything :laughing: 
// https://github.com/sherif7810/HTM/blob/master/src/main.rs
fn main() {
}
  1. Compiler warning.
    It’s personal preference. But it is generally encouraged to write your code without any warnings. They are likely going to byte you back someday in the future.
warning: unused import: `std::mem`
 --> src/lib.rs:1:5
  |
1 | use std::mem;
  |     ^^^^^^^^
  |
  = note: #[warn(unused_imports)] on by default

warning: unused variable: `i`
  --> src/lib.rs:45:13
   |
45 |         for i in 0..columns_length {
   |             ^ help: consider using `_i` instead
   |
   = note: #[warn(unused_variables)] on by default

.... <and a lot more>

  1. Unclear variable names. (https://github.com/sherif7810/HTM/blob/master/src/lib.rs#L7)
    What is ip_length? Maybe a comment explaining what it is? Or use the full name?

  2. Split spatial_pooling (https://github.com/sherif7810/HTM/blob/master/src/lib.rs#L78) into different small functions.
    That function is too large. And the spitted function can be reused. Reducing code size and increasing performance. Like what I did for tiny-htm. (https://github.com/marty1885/tiny-htm/blob/master/tiny_htm/tiny_htm.hpp#L387)

Well done! Great work! Especially implementing a SP in such a short time!

1 Like
6

I don’t know how to implement phase 4 from BAMI. May you provide a more detailed pseudo code or explanation?
Also, I don’t know how to compute the spatial pooler output.

I made small changes. Please, re-check.

1 Like
7

Sure!

  1. Cargo.toml 100644 → 100755, .gitignore 100644 → 100755, etc…
    All of your commited files are now executable. Why?

  2. main() is still empty.
    Getting things to compile is a very different thing from testing it and seeing it working.

  3. Compiler warning.

  4. Most problem from the previous review still exists.

Great work!
Which part are you talking about? Boosting? Encoders?

8
  1. It was unintentional.
  2. I’m going to test after completing the SP.
  3. I’m going to search about them.
  4. I didn’t think I can make functions that’ll be used more than once.

I think BAMI phase 4 (Pages 46 & 47) are unclear. I need a more detailed pseudo code.

9

@rhyolight may you check my sudo code?

In a more DoD design:


def calc_new_duty_cycle(duty_cycles, new_input, period):
    assert(period >= 1)
    return (dutyCycles * (period -1.0) + newInput) / period

#inference
sp_output = sp.compute_output(sp_input)

#learning
for i, cell_state in enumerate(sp_input):
    if cell_state == 0:
        continue
    for synapse in sp.synapses:
        if sp_input[synapse.index] is True:
            synapse.permanence = clamp(synapse.permanence+perm_active_inc, 0 ,1)
        else:
            synapse.permanence = clamp(synapse.permanence-perm_active_dec, 0 ,1)

sp.active_duty_cycles = calc_new_duty_cycle(sp.active_duty_cycles ,sp_output, period)
overlaps = where(sp.get_overlap_array()!=0, 1, 0) #get_overlap_array returns he entire overlap score array
sp.overlap_duty_cycle = calc_new_duty_cycle(sp.overlap_duty_cycle, overlaps, period)

for i, cell_state in enumerate(sp_output):
    sp.neighbor_mean_active_cycle = mean(neighbor(i, sp))
    sp.boost[i] = calc_boost_factor(sp.active_duty_cycles[i], sp.neighbor_mean_active_cycle)
    if sp.overlap_duty_cycle[i] < sp.min_duty_cycle:
        sp.increase_perm_for_all_synapse_of_cell(i)
10

This pseudocode looks good to me. If you want more in-depth introspection, see my latest Twitch stream, which is recording right now. One thing you might define is period. It is a bit confusing having the two duty cycle types to manage. I never really explained this in any videos. I’ve not actually every implemented Spatial Pooling (hopefully I will soon).

1 Like
11

I don’t have boosting working in my implementation. So TBH I don’t know what period is.But since it’s in BAMI and NuPIC. I included it. :rofl::rofl:

12

I assume it defines the scope of the ADC window in time.

13

What are the sizes of dutyCycles and newInput?

14

I have not looked at your code, but dutyCycles should match the number of mini-columns in the Spatial Pooler. Each mini-column will have a value in dutyCycles. newInput should be the size and dimensions of the input space.

15

May you explain this line?
There’s may be broadcasting, since newInput and dutyCycles have different sizes?

16

Hi @sherif7810, I went into some details on that function here:

In case it helps!

3 Likes
17

sp.active_duty_cycles & sp_output have different dimensions, so what happens?

18

I figured my mistake. Sorry.
Since SP output is active columns, it has the same dimensions as columns.

2 Likes