This model comprises methods like encoding and decoding. It involves encoding information in one network and decoding the reduced information in another network. The model consists of layers such as input, intermediate, and output. The input layer receives information, the intermediate layer stores the reduced information, and the output layer extracts information from the intermediate layer.
- Input layer
These layers receive inputs in the form of values associated with the information. The information is further processed by allowing random neurons to connect randomly to the input layers. These random neurons add a fixed variable (e.g., 10, 20, etc.) to the incoming information layers. This is done to differentiate and accelerate specific input units for accumulation within the model. The number of random neurons is determined based on the number of neurons in the layer and the sparsity of information units in the input. While other factors need to be considered, for now, let’s keep it at this level. The first network contains random neurons that add a fixed value, and there’s also a second network.
- The function
As models progress to process more complex systems, the properties of functions may or may not change. This is due to the fact that as more input is introduced, it can lead to chaos, requiring the identification of an optimal function that can bring order to this induced chaos. The fundamental purpose of any function in a neural network model is to establish order amidst chaos.
In the first network, the function allows the neurons with the highest values, after summing the input values and the values from randomly moving neurons, to be the only units of input information allowed in the model. Subsequently, the information is reduced by removing unnecessary information through a lossy function with the assistance of included functions and values of the random neurons. The first network’s function only permits the highest-valued neurons to reach the intermediate layers.
In the second network, the function exclusively allows the lowest-valued neurons to reach the output layer. This is because random neurons in the second network offset the fixed value added by the random-moving neurons in the first network. These random-moving neurons encode and decode information by randomly connecting to appropriate neurons in the network layers.
It’s important to note that the range of functions for processing neurons should expand layer by layer, but it should not increase too significantly, as this can significantly reduce information and potentially result in the loss of valuable data. Depending on various factors, the range should not exceed a large number. In my opinion, a range of 4 or 5 is optimal, although it can be subject to change. When a function reaches its limit, it should continue to the next layer with the same value range to process other neurons. The function is a dependent variable, while the input is the independent variable.
To reduce complexity and enhance the model’s efficiency, only one random neuron is allowed to participate in the decoding function of the second network. The number of random-moving neurons in the first network is determined by other factors such as the sparsity of unique units in the input information and the number of neurons in the layers.
- The output layer
Once the model processes the input for the first time, the output layer compares it with the input layers. If the values are the same, the random-moving neurons in the first and second networks are fixed in their appropriate positions with their connections to other neurons in the layers. This process is similar to fixing via backpropagation in a conventional neural network.
One of the pre-assumed notions is that these models will create order from chaos, essentially extracting patterns from the input information. By only allowing unique units of input information, common information is removed because this model focuses on distinct values in the input. Predictability occurs because patterns are extracted and absorbed by the model due to lower entropy in the input information. Since humans tend to predict less entropic information, this model can replicate properties of the brain.
After the model has been trained for some time, it becomes continuous with the input information. When the model becomes continuous, the intermediate step acts as a temporary holder of reduced information. The frequently reduced input information moves to the intermediate step and then quickly to the output neurons. In this way, not all attributes of the inputs are stored; only a few specific features essential for forming novel ideas, similar to how our brain operates.
So, when we move the model to more complex tasks, it is believed that only permitting the highest and lowest valued neurons to pass through the model is insufficient. There should be an addition of more functions that encode and decode information differently. To make this model work perfectly, creating order from chaos is key, and as the model’s complexity increases, different functions are required to handle the chaos.
Schematic diagram of first network -
Schematic diagram of second network -
Comment down your thoughts and feedbacks on this model.
btw