Catching up on the thread, so I’ll likely have multiple updates here.
Yes, it should always be possible to factorize any given input signal. The precise way in which that happens is largely dependent on how the incoming data is represented / interpreted. As a simple example, any image or video can be factorized into convolutions, wavelets or some other set of basis functions (DCT, Gabor, etc.). Any audio signal can similarly be factorized in the time or frequency domain (wavelet, FFT, etc.). Language itself can be factorized on multiple levels and from multiple perspectives (characters, tokens, phonemes, words, phrases, sentences, grammar, semantics, etc.). Scholars have picked apart language from almost every conceivable angle. It should not be that far-fetched to believe that a neural encoding of language would likely capture many of these perspectives, much like the way vision and audition are able to capture spatial and temporal patterns at multiple scales and from multiple perspectives.
Well, I’m putting myself in contrast to that, saying “No factors, actually chaos”. It’s another possibility. Chaos exists, and is actually quite (most?) common in nature.
But yeah, that’s the context to put the arguments I am generally putting, as a summarization of what has held us up in AI.
I think it was Kolmogorov and Chaitin who (separately) first presented an alternative perspective of that, of “randomness”, incapable of further factorization, incapable of further compression, as actually maximally meaningful.
I don’t think we are using these terms in the same manner. When I say factorize, I’m using it in the sense that any spatial and/or temporal signal can be decomposed into coherent and incoherent components.
The incoherent components are indistinguishable from noise. In other words, there is no function or algorithm that could predict their values or their evolution in time. Randomness itself has no inherent meaning (at least not in the sense that I use the word). By definition randomness is that part of the signal that is completely uncorrelated with any other part of the signal. The best that you could do would be to generate a statistical model or probability distribution of the likelihood of the random variable. If such an analysis were to reveal some meaningful pattern in the distribution, then there could potentially be some information contained in hidden variables that were not being properly modeled. In that case, the appearance of randomness is just a failure to account for those hidden variables.
The coherent component, on the other hand, is any combination of signals that can be represented via a mathematical expression. This includes any deterministic or recursive algorithm such as pseudo-random number generators, or cellular automatons. Encoding signals via mathematical functions, convolutions, or even look-up tables can be considered a form of data compression. Using an algorithm to encode structure is also a form of compression that allows for the generation of a potentially infinite series of novel values using only a finite set of symbols.
Consider, for example, an algorithm that computes successive digits of a transcendental number (e.g. pi). Even though the output sequence appears random, there is an inherent structure to the sequence that is determined by the algorithm. This argument also holds for any of the so-called computationally irreducible kernels that Wolfram likes to talk about. Yes, a chaotic system will produce two very distinct output streams when provided with different initial conditions, but there is a structure to the evolution of those streams that is encoded in the update rule. The algorithm itself is encoding the relevant information.
OK. So we’ve got “Language … can be decomposed into … a mathematical expression.”
What is that mathematical expression?
I say it can’t.
I say language (“natural” human language, to distinguish it from computer or artificial languages) cannot be completely described any more compactly than a full body of its usage (that’s what distinguishes natural language from computer language or mathematical languages, and what has prevented us from building effective AI models of it up to now.)
You may say that mathematical expression exists, it has just not been found yet.
Then @DanML posted a book supporting something, but resolving to me, as supportive of what I was saying:
Over and over in this thread I’m repeating, with evidence (actual contradictions in language structure, and the size of language models, to name two) that natural language structure cannot be (completely and consistently) abstracted. But people just keep repeating that they are “sure” it is possible. It’s kind of amusing. But repetition as a public service also gets tiring.
To avoid repeating myself you should direct your arguments to what I wrote e.g. in this thread. Firstly evidence from the size of transformer models:
Maybe this post, #4 in this thread, gives the most comprehensive collection of arguments I’ve presented here on it:
I don’t think you will ever be able to completely describe language.
A tiny sub-set of language ( mathematical words) are infinitely expandable even though the words can be arranged in “standard” expressions.
When you pull back to include the rules of grammar you can express a never-ending arrangement to encompass describing all objects and arrangements of objects.
You ‘might’ be able to describe some of the more commonly used arrangements but this will never be complete.
You can’t abstract it. But the language, that body of usage I said can’t be completely and consistently abstracted, does “describe” itself. It’s just you need to keep that body of usage, as usage, with all its peculiarities.
So, if we keep that “body of usage”, that is a form of “complete description”. And we should be able to work with that to replicate what is happening with language in cognition.
It’s a form of “embodiment”, really. The fact you need to keep all the examples of usage is another way of saying the system must be “embodied”.
In fact, it hints what is happening with cognition in general - embodied, but creating meaningful structure on top of that embodiment.
Language is all about new combinations of words. So it makes it clear that all of cognition is about new combinations.
It is just that, like language, the meaningful structure can never be completely or consistently described (beyond the embodiment itself.)
The solution is constant, partial, and even contradictory, abstraction.
And because sequences are how cognition builds meaningful structure on top of the embodied reality of language (that’s clear for language, language is such a simple system) it hints that sequences, cause and effect, is likely at least one major mechanism by which cognition orders other embodied structure into useful “meaning”. How cognition orders the embodied external world into “objects”, for a start.
Oh, and never being able to completely describe something, hints at a sense for, and a practical computational basis for, creativity. If it were possible to completely and consistently describe everything, then one might imagine a time when everything had been completely described. And at that time creativity would be at an end. This hints that can never happen.
By the way, I didn’t want to get too lost in philosophy before. But I couldn’t stop thinking about the translation : - ) For me all the given translations missed some meaning, and they also missed some poetry in English.
Your favourite was:
“Heaven and earth start with no name. The named is the mother of everything under the sun.”
Which is good. But at first I didn’t like the use of the word “start” and “everything under the sun”. “Start” feels abrupt. And “everything under the sun” carries connotations in English of “too much” to my ear.
I attempted an interpretation of my own, using “in the beginning” instead of the more abrupt “start”, and “the multitude of things” instead of the slightly negative to my ear “everything under the sun”:
Heaven and Earth in the beginning have no name. Name is mother to the multitude of things.
But then I wanted the rhyme too!
No name have heaven and Earth at first. Name, all things brings.
So rhyming Earth and birth instead of di and chi. And thing and bring instead of wu and mu.
Or:
Nameless are heaven and Earth at first. Name to the other is mother.
Rhyming Earth and first instead of di and chi. And other and mother instead of wu and mu, to bring back the “mother” theme again.
Just games. But the attempt to capture multiple structures at once, both the meaning, and the rhyme, made me think translation is a good example of the underlying point, that you can have multiple structures in the same thing. And need to be able to structure them in different ways to get the full meaning!
I’d think even modern Chinese translations (from ancient Chinese) have various problems likely. Are we cracking 老子’s original idea even better than former big names?
For the translation tournament, I’d put this one :
The world has no name at all in the beginning. Only after named, all things become conceivable.
Feels so right!
LLMs do this implicitly?
Maybe we can crack it, then vastly more people (other than big-techs) can have it?
It should be no surprise that knowledge distilled over millennia is deep.
But we’d be poor if we couldn’t build on it.
I think yes.
I think yes, LLMs do this endless expansion into all kinds of different perspectives implicitly.
And yes, moving away from the somewhat silly idea that we need to try and learn all meaning in the universe in one single, enormously expensive, “training” phase, should make AI accessible to even the smallest companies.
Not to mention truly creative, and, perhaps for that reason an actual “intelligence” for the first time.
But first we’ve got to get the network to reveal that endlessly reconfigurable structure!
I think you’re giving Google Translate too much credit Compl. It’s just got it wrong!
AI doesn’t understand.
Lacking context, no doubt. But it seems crazy that GT doesn’t have this famous phrase listed at the top of its registered translations. It’s as if you stuck “To be or not to be” in GT, and it came back asking if you wanted to do a Bachelor of Engineering.
It’s interpreted “to have a name” with “to be famous”. Which is a Modern Chinese meaning, sure. But you might imagine GT taking “He’s made a name for himself” and translating it as “He’s given himself a nickname”.
In the sense here it’s not “famous”, or even “to be famous”, it’s “to have a name”.
The phrase “有名 万物之母” is a Chinese idiom that can be translated to “Fame is the mother of all things.” It means that having a good reputation or being well-known can lead to many opportunities and benefits, just as a mother provides the nurturing environment for her children to grow and thrive.
This phrase emphasizes the importance of reputation and fame in various aspects of life, such as business, politics, and personal relationships. It suggests that having a positive reputation can open doors and create opportunities, while a negative reputation can hinder progress and success.
Mark
could I substitute “to have a name” for fame?
Yes, “to have a name” could be a possible substitution for “fame” in the phrase “有名 万物之母” in some contexts. However, it’s important to note that the concept of “having a name” doesn’t fully capture the same connotations as “fame” in English or the original Chinese idiom.
In Chinese culture, “有名” is often associated with being well-known or famous in a positive way, whereas “无名” (no name) is associated with being unknown or having a negative reputation. “Fame” in English may also carry connotations of celebrity or public recognition, while “having a name” could simply refer to being known by others in a more general sense. Therefore, while “to have a name” could convey some of the meaning of the phrase, it may not fully capture its nuances and cultural implications.
And Elon Musk thinks we have to stop development to prevent these things taking over the world!
I wonder if it is just chaining off fame randomly, or if somehow “fame” really does assume some larger breadth in whatever clustering of texts encountered over the Internet could be cobbled together to call its “philosophy”.
And then it doubles down!
The continuation of an original “fame” interpretation, even in the face of an alternative offered, is interesting. Maybe if you phrased your challenge as an objection rather than a question, it might have flipped. It might be possible to bump it into a more literal “have a name” context, and then have it continue talking about the importance of identity instead of the importance of fame. Surely on the Internet, somewhere, there’s someone talking about the importance of identity! That it is important to know who you are. That identity is the birth of meaning. That we must differentiate different groups of people… In 2023 that seems a given!
Fame is such an important element in Chinese culture! Ha! Wasn’t it Lao Tzu who was always talking about hiding your abilities, lest your enemies become jealous?
Or maybe I’m mixing him up with Sun Tzu, The Art of War. Hide your abilities from your enemies?
@complyue probably has some better quotes for the “importance” of fame in traditional Chinese culture. (Maybe for contemporary Chinese culture you might say fame is important. It might be fair to say there is excessive respect for fame, brand names etc, in contemporary Chinese culture? GPT might have got side tracked by texts talking about that. Though even there, it hasn’t done Jack Ma much good recently…)
I can’t blame Google too much in the discriminated language support regarding Chinese, the PRC government banned its business entirely in mainland even without a formal declaration.
If the phrase is really a Chinese “idiom” (while but not), the expansion sounds quite convincing. Otherwise GPT is pretty good at making thing up by “望文生义” (this is a real idiom, GPT can explain it quite well).
Though given an accurate context, GPT would perform much much better:
Compl
explain “有名万物之母” from 《道德经》
“有名万物之母” (yǒu míng wàn wù zhī mǔ) is a Chinese phrase from the Dao De Jing (《道德经》), also known as the Tao Te Ching, an ancient Chinese philosophical text written by Laozi (Lao Tzu). The Dao De Jing is the foundational text of Daoism (Taoism), which emphasizes living in harmony with the Dao (Tao), the ultimate and unknowable force that governs the universe.
The phrase “有名万物之母” can be translated as “the named is the mother of the ten thousand things.” This phrase appears in Chapter 1 of the Dao De Jing and is often used to explain the relationship between the Dao (Tao) and the myriad of things that exist in the universe.
In the context of Daoist philosophy, “有名” (yǒu míng) refers to the named, or the aspects of the world that can be described and defined by language. In contrast, the Dao (Tao) itself is considered to be beyond description, an ineffable force that precedes and transcends language.
“万物之母” (wàn wù zhī mǔ) means “the mother of the ten thousand things,” where “ten thousand” is a metaphor for the countless number of things in existence. In this context, “mother” signifies the source, or the origin, from which all things arise.
The phrase “有名万物之母” essentially suggests that while the Dao (Tao) is beyond description, it is the origin of all things that can be named and defined. The Dao gives rise to the myriad of things in the world, and by understanding this relationship, one can gain insight into the nature of existence and the interconnectedness of all things.
In Daoist philosophy, this understanding can lead to a harmonious way of living, where one aligns oneself with the natural flow of the Dao, embracing simplicity, humility, and inner peace.
Rob, you really know Chinese culture well.
Fame (good reputation, sometimes bad ones count too), wealth, and power, sorta like a trinity for people under Chinese culture, I have a feel that modernization of China society may have weakened the links there, but still strong enough as felt by me.
I don’t have a particular feel that Chinese text out there could misguide GPT this way, this topic is less explicitly talked about in Chinese per my perception.
I think Jack Ma will be well-being in any circumstance, thanks to his “fame” by far. Entrepreneurs successfully made wealth but without sufficient “fame” could be tortured to donate to the government (esp. stake holders of it).
What makes those kinds of errors especially pernicious is that they are not completely wrong. There is a (tiny, tiny) shred of truth in them. I think GPT has gone off on a tangent likely because of a bunch of texts plastered over the Internet talking about the modern passion for brand name goods in China. It may be a big thing in the texts GPT has access to. Chinese customers make up a large part of the sales of companies like Lois Vuitton and Chanel (LVMH market 34% Asia vs. 24% US?)
It’s talking about handbags!
It’ll have found texts like this:
'Chinese consumers where extravagant displays of luxury excess, like prominent logos, are valued. By contrast, in the developed Western markets, luxury consumers favor a more understated, classic approach.
The Shanghai-based GMA marketing agency describes this emphasis on conspicuous consumption as “Mianzi,” which means “face,” or to keep one’s honor and reputation in any situation.
“Purchasing luxury items is a symbol of prestige and social status and accumulated wealth. The more you take care of your appearance and lifestyle, the more you show to the world you succeed,” the agency writes in its “China Luxury Market Guide.”’
One thing which might be fun @Bitking is to ask GPT for some quotes from famous Chinese philosophers talking about the tremendous value given to fame in traditional Chinese culture. Some concrete examples. I wonder if that would be where GPT starts to make things up again.
Or, heck, who knows, maybe it will pull out a hitherto hidden to me thread of traditional Chinese thought which was all about the show! We might all get an education : - )
Mark Consider the meaning of these words as they appear in the I Ching, would you change your offered meaning?
The I Ching, also known as the Book of Changes, is a Chinese divination text that is deeply rooted in Taoist philosophy and has been used for centuries to provide guidance and insight into various aspects of life.
Given the complexity and depth of the I Ching, the interpretation of the phrase “有名 万物之母” as it appears in this text may vary depending on the specific context and the translation being used. However, in general, the phrase is often understood in the I Ching to emphasize the importance of cultivating a positive reputation and embodying virtues such as humility, integrity, and sincerity.
The I Ching also suggests that having a good reputation can help one to build strong relationships, gain the trust of others, and achieve success in various endeavors. However, it also warns against becoming too attached to one’s reputation or seeking fame for its own sake, as this can lead to arrogance, vanity, and ultimately downfall.
Overall, the interpretation of the phrase “有名 万物之母” in the context of the I Ching would likely emphasize the importance of balancing the pursuit of a positive reputation with inner cultivation and spiritual development, in order to achieve true success and fulfillment in life.
Consider: (Paste in this text)
ChatGTP offered:
Thank you for providing this additional context on the meaning of the phrase “有名 万物之母” in the Dao De Jing. The interpretation of the phrase in this context emphasizes the ineffable and transcendent nature of the Dao, which is the ultimate source and origin of all things in the universe.
The use of “有名” (named) in this context highlights the idea that while the Dao itself is beyond description, the things that arise from it can be named and defined. This suggests that language and conceptualization are important tools for understanding the world, but they are limited by their inability to fully capture the essence of the Dao.
Overall, the phrase “有名 万物之母” in the Dao De Jing emphasizes the interconnectedness of all things and the importance of aligning oneself with the natural flow of the universe. By recognizing the origin of all things in the Dao and living in harmony with its principles, one can achieve a state of inner peace and fulfillment.
@Bitking , I might failed to mention, that output is from ChatGPT 4.0 with “explain “有名万物之母” from 《道德经》” as the prompt. I don’t fully agree with what it had said, just wanted to demonstrate that “《道德经》” could have been an accurate context in this case, prompted GPT to perform much much better than without this context.
: