That raises another question that is what exactly does it mean to experience something.
The architecture it uses is called ‘The Transformer’(it’s not even an RNN!).
It “explores” its text by attending previous inputs selectively.
Where’s the line between just statistics and meaningful actions by the true understanding?
P.S. The positional encoding it uses has very similar characteristics to grid cells. It uses sine value with different frequencies for different scale of positions to represent where the each words are. I just thought this is interesting.
1 Like
Thanks for informing me. Do you have an understanding of when in the training process it does this? Does it get the option to attend previous inputs selectively between training epochs?
Meaningful action takes place in an environment over time. “Meaningful” is also objective. For a biological organism, “meaningful” means an action likely to increase my chances of gene survival. This can be as simple as not walking off a cliff, or planning and building a shelter. These are the types of actions we want to get to. I think if you have the right modeling system hooked up to sensors in an environment, you should be able to build models fo the space starting with random movements. At every time step, with every motion, you get immediate feedback and models are updated. You can learn extremely quickly this way, especially once you’ve defined yourself, space itself, and the line between them. Now I’m rambling sorry, Saturday night. 
1 Like
As I understand it, it doesn’t happen in the training phase, if that’s what you’re asking.
It only happens in the feedforward phase.
I think it’s just for finding the context of the current word.
I don’t think it does as that wouldn’t help with finding the context.
But I’m not sure as I’ve started to read about it just now. 
I think you could say it takes meaningful actions by statistics.
Does it make any sense?
I would agree that it doesn’t have its own intention or self motivation.
And it wouldn’t do anything outside of it has told to do by its own interest just like every other weak AIs.
Like you’ve said, I think being aware of itself to know how to differentiate itself from the environment is very important.
1 Like
You are absolutely right. Let me know if you find anything else interesting.
2 Likes
“everything lives in the domain of language”
My dog checks the yard every morning. If he finds a spot that smell like an intruder he pees on top of it.
1 Like
Bit and Paul, yes we need meaning and a way to store/represent/recall meaning. We also need a model of the world because meaning is context dependent. An artful balence of several systems.
1 Like
Julian Jaynes…all you need to know. He completely outlined how language produces consciousness. It is all intertwined in his book (The Origin of Consciousness in the Breakdown of the Bicameral Mind) and is occluded by all the schizophrenic voices of gods parts, which is the ‘breakdown’. When you strip all that off and put his book through a Dennett filter, Voilà, you have human consciousness as mediated by language. You also have to have a recursive language (see Chomsky) and it need not be spoken, so ASL, a recursive language, lets the deaf develop full consciousness. Just magical!
2 Likes
Julian Jaymes has a very particular, narrow definition of consciousness and all his arguments are respective to that definition.
So no, it’s not all one (at least me) needs to know, since I don’t think that is the right definition.
And sure, the term is so loaded that it’s really useful to have a definition when we discuss some aspects of it, but in general there is no consensus about what consciousness actually is.
1 Like
You have hit on what we might call the ‘great issue’ of consciousness; i.e., how is it defined? Every good book on it starts out with a section where they define it. Julian was no different and this stems from the fact that neuroscience, cognitive science, psychology, philosophy, AI, and any other field or subfield with even a passing interest in it has a different definition. Sooooo…if you go with J (Narrative, Mindspace, Timeline, Analog I, Metaphor Me, Excerption and Consilience…I may have missed one) then not only is language the basis of conscious thought but it is required for conscious thought.
This works for me and works fine in my work, YMMV.
1 Like
A better term than consciousness is conscious interiority, coined by Brian McVeigh, meaning the few moments every day that you are aware of thinking about yourselves.
Oh please, I wish I might never hear again “conscious interiority”. McVeigh was J’s student and when he ‘coined’ that phrase the rattle from J’s grave was deafening.
Please elaborate.
I don’t believe J would approve, but to be fair, we will never know. My opinion is that ‘conscious interiority’ just confuses the issue further by implying “this is what J really meant to say.”
What is interesting is that Dennett, in Consciousness Explained, beat around the bush incessantly even though his definition of C corresponds perfectly with J’s. No surprise there–Dennett knew J and had many conversations with him.
1 Like
"My opinion is that ‘conscious interiority’ just confuses the issue further by implying “this is what J really meant to say.” There is no such implication! I think conscious interiority is a very useful term, and goes one step further than JJ.
Knock yourself out. Opinions are like belly-buttons, everybody has one.
Is Joseph ‘conscious?’
Yes, he is aware of both his external senses and his internal planning.
Please see:
The circuitry exists, but it is not enhanced with the learned motor program we call language.
I, being a Jaynesian, believe that we learn consciousness. J was so wrapped up with the Origin that he never quite got around to the development part. In fact, he stated that his next focus was going to be on children. When you look at how children develop, you see the point at which they ‘become conscious’, somewhere between 3 and 5 depending on all sorts of things, most notably language. When they ‘become conscious’ are they fully conscious, and for now, let’s not go into exactly what that means. The simple answer is no, consciousness continues to develop and mature. So, once an individual begins to display the features of consciousness, they are conscious. How conscious? Well, I am working that.
I see a difference in learning to manipulate mental symbols and express your inner deliberations as being different than being aware of your inner “voice.” I believe that all mammals are conscious - that is they can place themselves in the environment as part of deliberations of choices of actions. They are all capable of selecting affordances with their body as part of the planning - they are aware of self.
Again - this all comes down to what you chose to include as being conscious.
1 Like
Absolutely. It is one of those things that if you choose to discuss it you have to lay down the ground rules before you do. That said, I don’t believe animals have a sense of self (and there is another vague term) and we can’t ask them, can we? Self-awareness, as determined by the self-recognition mirror test, really does not get to the core of the issue.