Problems constraints

[mraptor]

In Machine learning community there are plenty of examples of how to solve toy problems.
Which is what HTM/TBT lack so far. One reason can be it is not yet mature enough.
I dont think so !! Why ?

TBT is now getting into Reference frames, grids, locations which is akin to State spaces.
Which means even if we don’t have the tools yet we can borrow techniques and algorithms from ML and create hybrid solutions.

What I’m trying to do is figure out a ways to represent virtual environment problems in a manner that can be interpreted and leveraged by TBT.

Lets take for example Tic-tac-toe .
I can imagine representing the States as Location+Feature, but I’m having problem with the Rules.

F.e. how do you represent the constraint that you can not write X over O or vs versa ?

If we assume that the virtual game have this rule and “blocks” this move every time, how does it percolate to become a general rule, so that CC algo doesn’t even try it ?

The commands to check-box are easily encoded as Actions because they dont have a conditions they are just pure actions i.e. they are not IF-THEN constraint.

How about : Don’t squeeze the stapler on your sensor … oops I meant finger.

Or lets take even more complex scenario.
How do you define in GO “closed area” w/o combinatorial explosion ?

I think as a more general question what I’m asking is how does Constraints get encodded in CC algorithm ? What is the mechanism ?

[sheiser1]

@mraptor

What I’m trying to do is figure out a ways to represent virtual environment problems in a manner that can be interpreted and leveraged by TBT.

Successful applications here is what will put TBT on the map w/applied AI in a huge way IMO.

In terms of the particular arena, your question about CC understanding general rules is totally practical and must be addressed at some point – though I think there are milestones to hit before then.

My particular hunch for the next major milestone goal would be to have an agent move through a virtual space where there are moving objects to contend with.

The first step IMO is to prove is that the agent (composed of a group of CCs) can continually identify these objects – meaning it doesn’t mislabel them just because they move.

At first the agent could be simply told that X set of movings object are dangerous and Y set are desirable (like an animal knowing predators from prey).

If we see the agent behaving ‘rationally’ (as in consistently avoiding X objects and pursuing Y) this suggests that the agent has formed ‘invariant representation’ of the objects – meaning it continuously knows what they are, despite their changing relative position & orientation.

The current demo of TBT (afaik) is object identification, where multiple sensors (fingers) are shown to recognize a coffee cup by generating movements and sharing sensory information – to identify the cup faster than a single sensor could.

This identification appears to work, but it assumes (as I understand) that the cup is not moving, and is close enough for the agent (i.e. group of finger CCs) to touch it.

I’d like to see this same identification performance in a virtual space where the cup is moving, along with other objects that are also moving in the background. This would add robustness to the CCs object recognition power – getting closer to an agent that could actually navigate a more realistic world in some basic way.

I favor this kind of goal first, before learning games like Tic-Tac-Toe or GO, because the closed nature of these games (I suspect) allows conventional ML methods to succeed – though they are not equipped to navigate any realistic world where things are moving and not totally predictable.

I think that this cortical-based approach of HTM & TBT is best equipped to express true real-world intelligence, like us animals have to. So I want to show HTM & TBT doing things that no other system has succeed at whatsoever or even seriously tried (cause they know it wouldn’t work).

@mraptor

I think as a more general question what I’m asking is how does Constraints get encodded in CC algorithm ?

Regardless this is a fundamental question that needs addressing sooner or later. My hypothetical of a world with X dangerous and Y desirable objects doesn’t including the agent learning that X are dangerous and Y are desirable.

The current agent that recognizes the coffee cup doesn’t understand as we do that if the cup fell it would break, and the sharp piece could pose a threat. This to me is a whole other level of understanding which must be reached for a truly intelligent agent and deserves our reflection – I just think that prioritizing that as a demonstrable app would be kinda trying to run before we can crawl.

[CollinsEM]

In an ideal AGI, the environmental constraints (or rules of the game) would be learned the same way as with anything else. The allowed and forbidden spatial and temporal sequences should be stored in the agent’s temporal memory.

For example: In a given environment walking towards a wall is allowed. However, walking through a wall is not. As long as the constraints are applied consistently, the agent will learn to associate a specific action (stepping into a wall) with a specific result (the rather unpleasant sensation colliding with a wall rather than passing through it). Over time, the agent will likely learn to avoid running into walls unless it serves a specific purpose to do so.

So, I guess for me the more interesting question is, why would it be moving towards the wall in the first place? Learning constraints is fairly straight-forward given the algorithms we already have available. However, I’m not entirely sure how we are going to specify goals to the agent or how to provide the agent with the ability to set its own goals in pursuit of a larger directive or purpose.

[mraptor]

@CollinsEM

So, I guess for me the more interesting question is, why would it be moving towards the wall in the first place? Learning constraints is fairly straight-forward given the algorithms we already have available. However, I’m not entirely sure how we are going to specify goals to the agent or how to provide the agent with the ability to set its own goals in pursuit of a larger directive or purpose.

you are right … i was thinking about multiple problems and for many of them the goal can not be expressed easily as simply as reaching a state.

Constraints are definable for the virtual env, but for the brain model it will take too much memory and checking them on every turn is expensive. The brain has to induce and generalize them, induction is slow process.

The hard thing is the bootstrapping them … once the brain has primitives it is may be easier.

[mraptor]

@sheiser1

I favor this kind of goal first, before learning games like Tic-Tac-Toe or GO, because the closed nature of these games (I suspect) allows conventional ML methods to succeed – though they are not equipped to navigate any realistic world where things are moving and not totally predictable.

I think that this cortical-based approach of HTM & TBT is best equipped to express true real-world intelligence, like us animals have to. So I want to show HTM & TBT doing things that no other system has succeed at whatsoever or even seriously tried (cause they know it wouldn’t work).

I will disagree a bit ! bounded env like Tic-tac-toe provide a benefit that they provide clear Locations and Features and allow you to use Cartesian frames and grids i.e. this could simplify CC algorithm implementation i.e. no need for the kalman-like fuzzy behaviour.

To be frank I’m more interested into extracting the CC algo in a practical/simplified form, rather than total biological plausibility.
I have only cursory knowledge of the brain, so trying to match it for me is several years proposition ;( … there’s only 24h in a day

And We have Numenta to worry about biological plausibility … I cant help there, but trying permutations of the algorithm can help show shortcomings OR strengths AND has greater probability of solving easy problems