# Please review my HTM implementation in Rust

**URL:** <https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709>\
**Category:** Implementations\
**Tags:** htm-implementations, help-wanted\
**Created:** [March 31, 2019, 10:35am UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709 "2019-03-31T10:35:03Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![sherif7810](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sherif7810/32/3787_2.png) [@sherif7810](https://discourse.numenta.org/u/sherif7810)\
**Post date:** [March 31, 2019, 10:35am UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/1 "2019-03-31T10:35:03Z")

</div>

Please, review [my implementation](https://github.com/sherif7810/HTM).  
It’s prototype.

---

<div class="post-metadata">

**Author:** ![marty1885](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/marty1885/32/1891_2.png) [@marty1885](https://discourse.numenta.org/u/marty1885)\
**Post date:** [April 1, 2019, 5:59am UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/2 "2019-04-01T05:59:06Z")

</div>

Hello, I am more than happy to help  
But I’m not very familiar with Rust. How could I build the application?

---

<div class="post-metadata">

**Author:** ![sherif7810](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sherif7810/32/3787_2.png) [@sherif7810](https://discourse.numenta.org/u/sherif7810)\
**Post date:** [April 1, 2019, 7:45am UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/3 "2019-04-01T07:45:55Z")

</div>

1. `git clone` the project.
2. `cd proj_directory`.
3. `cargo build`.

---

<div class="post-metadata">

**Author:** ![sherif7810](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sherif7810/32/3787_2.png) [@sherif7810](https://discourse.numenta.org/u/sherif7810)\
**Post date:** [April 1, 2019, 7:46am UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/4 "2019-04-01T07:46:36Z")

</div>

Thanks. 😃

---

<div class="post-metadata">

**Author:** ![marty1885](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/marty1885/32/1891_2.png) [@marty1885](https://discourse.numenta.org/u/marty1885)\
**Post date:** [April 1, 2019, 8:52am UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/5 "2019-04-01T08:52:30Z")

</div>

Not being a Rust programmer. I can only try my best.  
Since you asked for a review. I’ll do a full code review! Not trying to be harsh. And fell free to justify you decisions by any means!

1. Well, the `main()` function is empty. I can’t profile anything 😆

```auto
// https://github.com/sherif7810/HTM/blob/master/src/main.rs
fn main() {
}

```

1. Compiler warning.  
It’s personal preference. But it is generally encouraged to write your code without any warnings. They are likely going to byte you back someday in the future.

```auto
warning: unused import: `std::mem`
 --> src/lib.rs:1:5
  |
1 | use std::mem;
  | ^^^^^^^^
  |
  = note: #[warn(unused_imports)] on by default

warning: unused variable: `i`
  --> src/lib.rs:45:13
   |
45 | for i in 0..columns_length {
   | ^ help: consider using `_i` instead
   |
   = note: #[warn(unused_variables)] on by default

.... <and a lot more>

```

1. Unclear variable names. ([https://github.com/sherif7810/HTM/blob/master/src/lib.rs#L7](https://github.com/sherif7810/HTM/blob/master/src/lib.rs#L7))  
What is `ip_length`? Maybe a comment explaining what it is? Or use the full name?

2. Split `spatial_pooling` ([https://github.com/sherif7810/HTM/blob/master/src/lib.rs#L78](https://github.com/sherif7810/HTM/blob/master/src/lib.rs#L78)) into different small functions.  
That function is too large. And the spitted function can be reused. Reducing code size and increasing performance. Like what I did for tiny-htm. ([https://github.com/marty1885/tiny-htm/blob/master/tiny\_htm/tiny\_htm.hpp#L387](https://github.com/marty1885/tiny-htm/blob/master/tiny_htm/tiny_htm.hpp#L387))

Well done! Great work! Especially implementing a SP in such a short time!

---

<div class="post-metadata">

**Author:** ![sherif7810](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sherif7810/32/3787_2.png) [@sherif7810](https://discourse.numenta.org/u/sherif7810)\
**Post date:** [April 1, 2019, 9:00pm UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/6 "2019-04-01T21:00:05Z")

</div>

I don’t know how to implement phase 4 from BAMI. May you provide a more detailed pseudo code or explanation?  
Also, I don’t know how to compute the spatial pooler output.

I made small changes. Please, re-check.

---

<div class="post-metadata">

**Author:** ![marty1885](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/marty1885/32/1891_2.png) [@marty1885](https://discourse.numenta.org/u/marty1885)\
**Post date:** [April 2, 2019, 1:26am UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/7 "2019-04-02T01:26:58Z")

</div>

Sure!

1. [Cargo.toml](https://github.com/sherif7810/HTM/commit/9719ba6837d6a5bb64935368fe57964e80b7cfa4#diff-80398c5faae3c069e4e6aa2ed11b28c0) 100644 → 100755, [.gitignore](https://github.com/sherif7810/HTM/commit/9719ba6837d6a5bb64935368fe57964e80b7cfa4#diff-a084b794bc0759e7a6b77810e01874f2) 100644 → 100755, etc…  
All of your commited files are now executable. Why?

2. `main()` is still empty.  
Getting things to compile is a very different thing from testing it and seeing it working.

3. Compiler warning.

4. Most problem from the previous review still exists.

Great work!  
Which part are you talking about? Boosting? Encoders?

---

<div class="post-metadata">

**Author:** ![sherif7810](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sherif7810/32/3787_2.png) [@sherif7810](https://discourse.numenta.org/u/sherif7810)\
**Post date:** [April 2, 2019, 3:22am UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/8 "2019-04-02T03:22:52Z")

</div>

1. It was unintentional.
2. I’m going to test after completing the SP.
3. I’m going to search about them.
4. I didn’t think I can make functions that’ll be used more than once.

I think BAMI phase 4 (Pages 46 & 47) are unclear. I need a more detailed pseudo code.

---

<div class="post-metadata">

**Author:** ![marty1885](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/marty1885/32/1891_2.png) [@marty1885](https://discourse.numenta.org/u/marty1885)\
**Post date:** [April 2, 2019, 2:29pm UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/9 "2019-04-02T14:29:56Z")

</div>

@rhyolight may you check my sudo code?

In a more DoD design:

```auto

def calc_new_duty_cycle(duty_cycles, new_input, period):
    assert(period >= 1)
    return (dutyCycles * (period -1.0) + newInput) / period

#inference
sp_output = sp.compute_output(sp_input)

#learning
for i, cell_state in enumerate(sp_input):
    if cell_state == 0:
        continue
    for synapse in sp.synapses:
        if sp_input[synapse.index] is True:
            synapse.permanence = clamp(synapse.permanence+perm_active_inc, 0 ,1)
        else:
            synapse.permanence = clamp(synapse.permanence-perm_active_dec, 0 ,1)

sp.active_duty_cycles = calc_new_duty_cycle(sp.active_duty_cycles ,sp_output, period)
overlaps = where(sp.get_overlap_array()!=0, 1, 0) #get_overlap_array returns he entire overlap score array
sp.overlap_duty_cycle = calc_new_duty_cycle(sp.overlap_duty_cycle, overlaps, period)

for i, cell_state in enumerate(sp_output):
    sp.neighbor_mean_active_cycle = mean(neighbor(i, sp))
    sp.boost[i] = calc_boost_factor(sp.active_duty_cycles[i], sp.neighbor_mean_active_cycle)
    if sp.overlap_duty_cycle[i] < sp.min_duty_cycle:
        sp.increase_perm_for_all_synapse_of_cell(i)

```

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [April 2, 2019, 4:03pm UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/10 "2019-04-02T16:03:09Z")

</div>

This pseudocode looks good to me. If you want more in-depth introspection, see my latest Twitch stream, which is recording right now. One thing you might define is `period`. It is a bit confusing having the two duty cycle types to manage. I never really explained this in any videos. I’ve not actually every implemented Spatial Pooling (hopefully I will soon).

---

<div class="post-metadata">

**Author:** ![marty1885](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/marty1885/32/1891_2.png) [@marty1885](https://discourse.numenta.org/u/marty1885)\
**Post date:** [April 2, 2019, 5:20pm UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/11 "2019-04-02T17:20:59Z")

</div>

I don’t have boosting working in my implementation. So TBH I don’t know what `period` is.But since it’s in BAMI and NuPIC. I included it. 🤣🤣

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [April 2, 2019, 5:22pm UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/12 "2019-04-02T17:22:31Z")

</div>

I assume it defines the scope of the ADC window in time.

---

<div class="post-metadata">

**Author:** ![sherif7810](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sherif7810/32/3787_2.png) [@sherif7810](https://discourse.numenta.org/u/sherif7810)\
**Post date:** [April 5, 2019, 9:27pm UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/13 "2019-04-05T21:27:17Z")

</div>

What are the sizes of `dutyCycles` and `newInput`?

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [April 5, 2019, 9:29pm UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/14 "2019-04-05T21:29:12Z")

</div>

I have not looked at your code, but `dutyCycles` should match the number of mini-columns in the Spatial Pooler. Each mini-column will have a value in `dutyCycles`. `newInput` should be the size and dimensions of the input space.

---

<div class="post-metadata">

**Author:** ![sherif7810](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sherif7810/32/3787_2.png) [@sherif7810](https://discourse.numenta.org/u/sherif7810)\
**Post date:** [April 5, 2019, 9:36pm UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/15 "2019-04-05T21:36:47Z")

</div>

> [@marty1885](#):
>
> `return (dutyCycles * (period -1.0) + newInput) / period`

May you explain this line?  
There’s may be broadcasting, since `newInput` and `dutyCycles` have different sizes?

---

<div class="post-metadata">

**Author:** ![brev](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/brev/32/5548_2.png) [@brev](https://discourse.numenta.org/u/brev)\
**Post date:** [April 5, 2019, 9:38pm UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/16 "2019-04-05T21:38:54Z")

</div>

> [@sherif7810](#):
>
> May you explain this line?  
> There’s may be broadcasting, since `newInput` and `dutyCycles` have different sizes?

Hi @sherif7810, I went into some details on that function here:

> [@Spatial pooler boosting and duty cycles](https://discourse.numenta.org/t/spatial-pooler-boosting-and-duty-cycles/1571/10):
>
> I’m currently building my first Spatial Pooler, and am working on duty cycles now. I haven’t seen the calculation details documented anywhere, so just in case it’s helpful for other newbs in the future: In NuPIC, the calculation is a “running moving average”, which allows you to average with only the previous value, and the new value. The actual NuPIC calculation itself is here, and is used for all the different kinds of duty cycles: As noted previously, these dutyCycle values are fl…

In case it helps!

---

<div class="post-metadata">

**Author:** ![sherif7810](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sherif7810/32/3787_2.png) [@sherif7810](https://discourse.numenta.org/u/sherif7810)\
**Post date:** [April 5, 2019, 9:47pm UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/17 "2019-04-05T21:47:02Z")

</div>

> [@marty1885](#):
>
> `sp.active_duty_cycles = calc_new_duty_cycle(sp.active_duty_cycles ,sp_output, period)`

`sp.active_duty_cycles` & `sp_output` have different dimensions, so what happens?

---

<div class="post-metadata">

**Author:** ![sherif7810](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sherif7810/32/3787_2.png) [@sherif7810](https://discourse.numenta.org/u/sherif7810)\
**Post date:** [April 5, 2019, 9:56pm UTC](https://discourse.numenta.org/t/please-review-my-htm-implementation-in-rust/5709/18 "2019-04-05T21:56:39Z")

</div>

I figured my mistake. Sorry.  
Since SP output is active columns, it has the same dimensions as columns.