Hey fellows,
Does anubody know wher to find the answers for the BAMI Problems set?
Problems Set are found here: https://numenta.com/assets/pdf/biological-and-machine-intelligence/BAMI-Complete.pdf
Problem Sets
Sparse Distributed Representations
- What are the disadvantages to SDRs? Consider representing a string of text as ASCII vs. SDRs.
- What does the “distributed” part of “Sparse Distributed Representation” mean?
- Why is sparsity necessary? How sparse do representations need to be? Can they be too sparse?
- Which is better? n=1000 and w=3 or n=300 and w=15?
- For representations with 33 active bits out of 500 total, how many unique representations are there? If
we randomly generate 100 of these representations, how likely is it that two of the hundred will be
identical? If we randomly generate two of these representations, how likely is it that they will share more
than 5 active bits?- Work out the first few terms of equation (4) in the SDR chapter to see how the values quickly diminish.
Show the equation can then be approximated with ���
� (�) ≈ |Ω�(�,�,�)|
(
�
�) .
Encoders- When creating an encoder for a new data type, what makes a good SDR? How can you test that the
SDRs that are produced are good?- How would you encode sound as SDRs?
Spatial Pooling- What is Spatial Pooling? How can we tell if the output of Spatial Pooling is good?
- What is boosting and how does it work? Is it necessary?
- Is an untrained spatial pooler just a “random hash” (random mapping from input to output vector)? Why
or why not? What happens to the output of the spatial pooler if you change one bit in the input?- Can untrained spatial poolers with randomly initialized input bit weights be useful or even better than a
trained SP?- How does online learning happen in the SP?
- Can you do spatial pooling with small numbers? For example, is it reasonable to have an SP with 20
columns? If not, why are large numbers important in SDR’s?
a. What’s the difference between picking “5 columns out of 50” vs “50 out of 500”? Both have
10% sparsity.
b. What’s the difference between picking “50 out of 100” vs “50 out of 1000”? Both will output 50
1’s.- Is it possible that the SP output for a fixed input can change completely over time? How can this risk be
mitigated?- Suppose the input vector (input to the SP) is 10,000 bits long, with 5% of the bits active at a time. What
percentage of the input space should be in the potential pool for each column? How do you figure this
out?- How does the SDR representation of input A in isolation, and input B in isolation, compare with the SDR
representation of input A unioned with B? Alternatively, how does the representation of a horizontal line
and the representation of a vertical line compare with the representation of a cross?- Let’s look at a Spatial Pooler of 2048 columns with 40 active at a time and learning enabled. A, B, and C
represent non-overlapping subsets of 20 bits in the input space (with total of 500 bits). The spatial
pooler is trained on the following:
• All A and B bits active and all other input bits inactive
• All A and C bits active and all other input bits inactive
• Many other patterns with 40 active bits, but none that overlap significantly with A, B, or C
After this training, consider example inputs when only A bits are active, only B, and only C. How will the
overlap compare between the output representations for these three distinct inputs?- Suppose we have an input vector that is 10,000 bits long. Suppose the spatial pooler has 500 columns,
of which 50 are active at any time.
a. Can we distinguish many patterns, or a small number? Which patterns are likely to be
confused?
b. What happens to the SDR representation if we add noise to the patterns?
c. What happens if we add occlusions?
