Chaos/reservoir computing and sequential cognitive models like HTM

Yes, the key statement of Daoism I like is:

“the one true Dao is the Dao which cannot be known”, etc. (道可道,非常道?)

“Ever-change” would fit. This says “meaning” is a process, not an artifact. There is also a “process” physics, and even a “process” biology now, which speaks to the same idea.

In the linguistics space you have Paul Hopper, Emergent Grammar, also talking about this ever changing “process”:

“The notion of emergence is a pregnant one. It is not intended to be a standard sense of origins or genealogy, not a historical question of “how” the grammar came to be the way it “is”, but instead it takes the adjective emergent seriously as a continual movement towards structure, a postponement of 'deferral” of structure, a view of structure as always provisional, always negotiable, and in fact as epiphenomenal, that is, at least as much an effect as a cause."

https://journals.linguisticsociety.org/proceedings/index.php/BLS/article/viewFile/1834/1606

In philosophy, closest to grounding in the physical, might be Thomas Kuhn:

Structure of Scientific Revolutions, p.g. 192 (Postscript)
“When I speak of knowledge embedded in shared exemplars, I am not referring to a mode of knowing that is less systematic or less analyzable than knowledge embedded in rules, laws, or criteria of identification. Instead I have in mind a manner of knowing which is misconstrued if reconstructed in terms of rules that are first abstracted from exemplars and thereafter function in their stead.”

Though Wittgenstein comes close, shifting to a basis for meaning in “games” later in his life. Quoted by Kuhn here:

Thomas Kuhn, The Structure of Scientific Revolutions, p.g. 44-45:
(Quoting Ludwig Wittgenstein, Philosophical Investigations, trans. G. E. M. Anscombe, pp 31-36.)

'“What need we know, Wittgenstein asked, in order that we apply terms like ‘chair’, or ‘leaf’, or ‘game’ unequivocally and without provoking argument?”

‘That question is very old and has generally been answered by saying that we must know, consciously or intuitively, what a chair, or a leaf, or game is. We must, that is, grasp some set of attributes that all games and only games have in common. Wittgenstein, however, concluded that, given the way we use language and the sort of world to which we apply it, there need be no such set of characteristics. Though a discussion of some of the attributes shared by a number of games or chairs or leaves often helps us learn how to employ the corresponding term, there is no set of characteristics that is simultaneously applicable to all members of the class and to them alone. Instead, confronted with a previously unobserved activity, we apply the term ‘game’ because what we are seeing bears a close “family resemblance” to a number of the activities that we have previously learned to call by that name. For Wittgenstein, in short, games, and chairs, and leaves are natural families, each constituted by a network of overlapping and crisscross resemblances. The existence of such a network sufficiently accounts for our success in identifying the corresponding object or activity. Only if the families we named overlapped and merged gradually into one another–only, that is, if there were no natural families–would our success identifying and naming provide evidence for a set of common characteristics corresponding to each of the class names we employ.’

In philosophy you can find it all over the place. Even H. G. Wells!

“…My opening scepticism is essentially a doubt of the objective reality of classification.”

https://www.marxists.org/reference/archive/hgwells/1905/modern-utopia/appendix.htm

I can go on and on along the philosophy thread of this! As I say, after I noticed this for what was happening when I tried to learn grammar, it started popping up all over the place.

Better stop there. As I say, I don’t want to detract from the simplicity of its application to AI. The application is very simple. It might be better to focus there.

No doubt. It always reminds me of the preamble I remember from many physics lectures: let us assume the system is linear!! If you make the right assumptions, you can always avoid inconvenient truths!

If you want to see this contradictory ordering dynamic playing out in computer science space, though, you might look at the drift from OOP to functional programming.

Why has functional programming come to dominate object oriented programming in recent years?

Rich Hickey has given some nice talks on why object models are always imperfect, and that has led to a renewed emphasis on ad-hoc orderings of raw data in functional programming.

There’s also this series by Bartosz Milewski which goes into the relationship of programming theory to the mathematical field of category theory:

“Maybe composability is not a property of natures”
Category Theory 1.1: Motivation and Philosophy
Bartosz Milewski

Continuing the category theory theme, in the compositional semantics space, the first other work I came across expressing similar ideas was Bob Coecke. Also a category theory guy.

(Category theory, BTW, being invented to deal with the incompleteness/contradictory character of mathematics demonstrated by Goedel.)

From quantum foundations via natural language meaning to a theory of everything

"In this paper we argue for a paradigmatic shift from ‘reductionism’ to ‘togetherness’.

Being a maths guy, Coecke is very tied up in the parallel to the mathematical abstractions of category theory. And he’s taken the parallel to QM maths so far as to be building a company to analyse language using quantum computing! I don’t think we need to go that far. I think Coecke is squeezing language into a QM formalism, and then using quantum computing to pick it out again!

But the insights of subjectivity of category on the environment, shared with QM, I think are valid.

You can find the QM maths parallel being drawn elsewhere. For instance in this talk with my namesake the famous neurobiologist Walter Freeman:

NONLINEAR BRAIN DYNAMICS AND MANY-BODY FIELD DYNAMICS
Walter J. Freeman and Giuseppe Vitiello

I can go on and on along this angle too.

But like I say, the application to cognitive modeling may be very simple. It might be best to concentrate on that simplicity.

2 Likes