# Richer than ReLU

**URL:** <https://discourse.numenta.org/t/richer-than-relu/12364>\
**Category:** Lounge\
**Created:** [May 9, 2026, 1:24pm UTC](https://discourse.numenta.org/t/richer-than-relu/12364 "2026-05-09T13:24:55Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![SeanOConnor](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/seanoconnor/32/3441_2.png) [@SeanOConnor](https://discourse.numenta.org/u/SeanOConnor)\
**Post date:** [May 9, 2026, 1:24pm UTC](https://discourse.numenta.org/t/richer-than-relu/12364/1 "2026-05-09T13:24:55Z")

</div>

ReLU is great and all that but you are just zeroing channels based on decisions (x\>0?)  
You could created interactions between the decisions and use those for weight matrix interaction.

[https://archive.org/details/hd-dh-hd-dh-graph-viewer](https://archive.org/details/hd-dh-hd-dh-graph-viewer)

[https://archive.org/details/decision-graphs-from-re-lu-sign-patterns-using-walsh-hadamard-commutator-operators](https://archive.org/details/decision-graphs-from-re-lu-sign-patterns-using-walsh-hadamard-commutator-operators)

[https://archive.org/details/ternary](https://archive.org/details/ternary)

---

<div class="post-metadata">

**Author:** ![SeanOConnor](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/seanoconnor/32/3441_2.png) [@SeanOConnor](https://discourse.numenta.org/u/SeanOConnor)\
**Post date:** [May 9, 2026, 10:08pm UTC](https://discourse.numenta.org/t/richer-than-relu/12364/2 "2026-05-09T22:08:57Z")

</div>

If the input vector was x then you would need to decorrelate the +1,-1 entries in D from x or you would get abs() type behavior. A locality sensitive hash of x with +1,-1 output bits is one option, another would be a derangement type permutation of D.  
Other interactions between the elements of D could be logical And, Or etc. The road not traveled yet.