There are a number of live streams of audio from hydrophones, are underwater microphones available. We currently have a use case for both anomaly detection as well as a system that understands ocean events on a number of different timescales:
Micro (Clicks or shorter-than, perhaps imperceptible by humans)
Short (Whale call, scuba diver, passing boat)
Medium (Mating seasons, long human processes
Long (geological events, icebergs, etc)
I’ve done some literature reviews on this and there’s a great deal of work being done in this space, called DCL or DCL/DE for all sorts of animals. I was thinking HTM and NuPIC might be really, really good for this.
If I’m right, which I hope I am, is there a simple architecture of general framework for how to approach a problem like this? In the end we’d like to have live streams with the trained model applied on them, and a training center where we could teach the system new vocabulary - this is what a walrus sounds like, this is a dolphin, etc.
Thank you for any and all guidance you might have!
Hi Mark, thanks for joining the forum. I’ve played around with sound processing with HTM in the past (you can run it against any WAV file):
This demo trains runs an FFT on the audio signal into 10 bins, and I trained 10 NuPIC models, one on each model.
I didn’t really know what I was doing at the time, and I didn’t really find out anything useful. I was trying to see what anomalies it would show, hoping they would partially align with major changes in songs I would play. In the end, it was inconclusive.
If I were to do this again, I would train the models on birdsongs and manually reset each song as they naturally end. You might train 10 models with 10 different songs, then let a bird sing a song. Each model will generate comparable anomaly scores, so you should be able to classify the song this way. However, this is nothing more than current DL techniques also do in other ways.
The cochlea is a complicated organ, and we don’t have an encoder for it yet, however some have tried:
I hope this leads you in the right direction without discouraging you. I believe it is possible, but I don’t know how to do it.
