Learning how birds teach themselves to sing Dad's song

Yeah, so this idea is pretty general to machine learning I think. The jargon is different ways of saying the same sort of thing.

Not matter what method you are using: gradient-descent, genetic-algorithms, reinforcement-learning, etc. they all have an error function (in genetic algorithms it is called ‘fitness function’ or ‘objective function’ for example), where it is simply a measure of the distance between the current output of the agent and the desired/expected output. They all also have a way to use noise/stochasticity to search the space of possible outputs. Those outputs that have a relatively smaller distance to the desired output are selected, so it could be said they have a ‘smaller error’, so they are ‘good’. In HTM talk they could be said to have a ‘greater overlap’ when talking about SDRs. Either way, its the same thing - a comparison/measure between two things, then selecting the representations with greatest similarity/overlap, then repeat this process. This whole process is common to a vast number of machine learning methods.

So far HTM is working on feed-forward learning from sensory input, to model the world. Once you have a semantic model of the world you can then leverage that to do the process as described above. If you have an objective/goal (say to learn to produce a song-bird song) then that would require generating noisy/stochastic SDRs within the motor region hierarchy. The motor SDRs that produce a similar sounding output as the dad’s song SDR will be remembered. So the agent produces an output from a stochastic motor SDR (‘blabbering’), the sound it produces is fed into the auditory region where it is represented as a sensory input SDR then is compared to the dad’s song SDR. The overlaps are then probably sent to an association area along with the outputs of the motor SDRs that produced that sound. This region is probably where the SDRs are compared and the reinforcing feedback to the motor region is determined. Repeat that enough times and it will build up a representation in the motor region that produces are very similar output to that of the dad song. Again, the process basically being stochastic sampling and semantic selection. It is likely that the motor representation is built from the bottom-up in the motor region as it starts with small features (as they are easier to compare) then combines them to more complex features until the top-level of the hierarchy represents the whole motor representation of the dad song.

I don’t know if this is what the brain does, but it shows how this general machine learning method could be implemented in a HTM system to replicate the learning of the song-bird.

4 Likes