So I finally connected my work to the mainstream path. I mean I found where we differed at.
Markov Chains > Hidden Markov Models > RNN > LSTM > Transformers
My design is like the Markovs, but not like RNN and thereafter. That’s where I stop. If I go ahead to RNN then all my AI makes no sense, we get complex maths like Backpropagation etc and even Hinton said it’s unnatural.
I learned though the only difference between my (markov) and modern (RNN and thereafter) AI is THIS !:
Non-linear. That’s it. My AI is (?) modern but just lacks the ability to solve non-linear problems good. Or does it?
And that’s the topic here. I know non-linear problems are not like “he got 15$ for working 1 hour, someone got 60$ - how long did they work?”. They’re like x^2 or 2^x (try putting 1, 2, 3, or 4 where the x is), this gives you an ex. exponential curve, not a linear.
BUT that doesn’t make sense. Even linear problems are not exact matches, 15$>1 hour, 60$>? You have to do tricks to find the pattern that gives you all answers fast. 60/15=answer. Same for the x^2. Input is ex. 1 output is 1, input is 2 output is 4, 3: 9, if input is 5 answer is ?, it’s 25 because 5 * itself. If 5^3 we get 125 because 5itselforiginal_input(5).
My design “can” discover these rules by natural “thinking” by brainstorming intelligently, but one wouldn’t hardcode them all either, and Backprop is unnatural as well.
If I look for non-linear problems in vision or text, what are they? I don’t mean math problems like Joey earns 45$…no, rather problems like “I was walking down the ?” prediction, or predict the rest of an image with only a tail and bowl shown. And I don’t see any non-linear problems here right away. What could they be though? What if a cat was observed to be 1 inch high 1 time, 3 inches high 5 times, and 8 inches high 873 times, is this an exponential recognition task that requires me (so to recognize A or predict A>? (B) “prediction”) to find the pattern that governs cat height growth and therefore allow we to determine that cats will more likely be super high and rarely short? What about luminosity? Maybe grapes are rarely fully transparent in a certain lighting? Or some man made engine or a natural hurricane? And how often are these a problem?
So am I right? It seems that modern AI that use Backprop is doing it the wrong way to figure out patterns where instead the real way is to brainstorm about the rule. It seems sometimes a cat face may be brighter than we knew and we must be able to recognize it just with less accuracy, or the tail it has we predict/generate is brighter even though we think not (we adapt), or that when we think the cat will run more if bigger maybe it is exponential or has some variation from the average. All these seem non-linear and not an exact match, I think linear rarely exists, but to solve non-linear doesn’t require Backprop.