# How should I encode low res RGB image for OpenAI minigrid gym

**URL:** <https://discourse.numenta.org/t/how-should-i-encode-low-res-rgb-image-for-openai-minigrid-gym/11236>\
**Category:** Implementations\
**Tags:** encoders, question\
**Created:** [February 22, 2024, 2:54pm UTC](https://discourse.numenta.org/t/how-should-i-encode-low-res-rgb-image-for-openai-minigrid-gym/11236 "2024-02-22T14:54:10Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![buckithed](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/buckithed/32/6279_2.png) [@buckithed](https://discourse.numenta.org/u/buckithed)\
**Post date:** [February 22, 2024, 2:54pm UTC](https://discourse.numenta.org/t/how-should-i-encode-low-res-rgb-image-for-openai-minigrid-gym/11236/1 "2024-02-22T14:54:11Z")

</div>

Im trying to use an htm net as part of my agent to play the minigrid gym, and am wondering if i should encode each rgb values a a scalar, or filter it into individual r g and b binary pixel representations, then feed it to an sdr?

---

<div class="post-metadata">

**Author:** ![Deric\_Pinto](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/deric_pinto/32/2124_2.png) [@Deric\_Pinto](https://discourse.numenta.org/u/Deric_Pinto)\
**Post date:** [February 24, 2024, 10:13am UTC](https://discourse.numenta.org/t/how-should-i-encode-low-res-rgb-image-for-openai-minigrid-gym/11236/2 "2024-02-24T10:13:07Z")

</div>

I sort of had the same question , I think it kind of depends what the percentage of sparsity you want maintain , i processed individual R. and G and B bytes individually onto there own networks . roughly bringing the sparsity to 8% : 8 bits for 100 neurons at a time

---

<div class="post-metadata">

**Author:** ![cezar\_t](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/cezar_t/32/4451_2.png) [@cezar\_t](https://discourse.numenta.org/u/cezar_t)\
**Post date:** [February 24, 2024, 10:58am UTC](https://discourse.numenta.org/t/how-should-i-encode-low-res-rgb-image-for-openai-minigrid-gym/11236/3 "2024-02-24T10:58:22Z")

</div>

Not really familiar to me but they seem more like a limited small size grid where each cell is either empty or contains some thing. Where thing could be either “player”, “target”, “wall”, “lava” and so on.

The point is you’ll be better off (cheaper to train) with representing grid world state by encoding directly these symbols into a SDR at cell resolution than trying to meaningfully process the rgb, (pixel level) images

---

<div class="post-metadata">

**Author:** ![buckithed](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/buckithed/32/6279_2.png) [@buckithed](https://discourse.numenta.org/u/buckithed)\
**Post date:** [February 24, 2024, 3:27pm UTC](https://discourse.numenta.org/t/how-should-i-encode-low-res-rgb-image-for-openai-minigrid-gym/11236/4 "2024-02-24T15:27:42Z")

</div>

I just found out the pixels are actually 3 enums, not RGB values, that represent attributes. The state is a 7x7 grid of 3 enums: \<object\_type, color, doors\_state\>. I could encode the object\_type by dividing the type up into a different SDR for each type, but the problem then is that the SDR could potentially be 100% on. E.g. All cells are lava. I suppose I could just make it so that 100% of on pixels is still just a small percentage of the entire SDR.

---

<div class="post-metadata">

**Author:** ![buckithed](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/buckithed/32/6279_2.png) [@buckithed](https://discourse.numenta.org/u/buckithed)\
**Post date:** [February 24, 2024, 8:11pm UTC](https://discourse.numenta.org/t/how-should-i-encode-low-res-rgb-image-for-openai-minigrid-gym/11236/5 "2024-02-24T20:11:47Z")

</div>

Thinking about this again, I don’t need to worry about being 100% full of an object, because if I preprocess the image into separate binary filtered images by each object type, and then concatenate them together, then it will be sparse no matter what since there can usually only be one object type in a single space. E.g. 100% lava will have all lava bits filled, but all the other bits will be 0.