Are there any good websites with temporal data I can access for free?
Some good temporal data I would like to find: EEG, Polygraph, Stock, Human Activity (like geospatial data of how humans move around a city during the day).
I want to swarm around that data and try to have NuPIC learn the patterns and then predict them.
You may want to checkout this website called Riverview. Riverview is a website with many excellent temporal data streams It provides a pluggable interface for users to expose temporal data streams in a time-boxed format that is easily query-able. It was built to provide a longer-lasting historical window for public data sources that provide only real-time data snapshots, especially for sensor data from public government services like weather, traffic, and geological data.
@rhyolight made this happen and he knows more about it.
Hi @Addonis! Like @ycui said, you can get public streaming temporal data free through River View. Here is my introduction to the service from last year. You can get to the live service at data.numenta.org.
Additionally, I have a tool for streaming River View data into NuPIC (including swarming) in a project called Menorah. Here is a tutorial video:
If nothing else, this project is a good code example of how to get data from River View and swarm over it. It’s generic enough that you can simply specify names of data streams to run swarms and start models.
There is also this wiki: Data Sets For NuPIC (for any HTM, really… contributions welcome).
Lastly, I have a half-baked project called SODA-Tap that attempts to inspect every available SODA data set and save metadata about temporal streams into REDIS. It is a work in progress, but it helped me identify these potentially valuable public temporal streams available through SODA.
Heads up folks, I just moved this topic from #nupic:swarming to #htm-hackers because finding temporal streaming data is important for any HTM system, not just NuPIC.
Cool, I tried it and it worked perfectly on my Ubuntu14.04 (VM).
The predicted data seems to be matching the actual data really well.
At the Hackathon, during the Stock Prediction demo, Jeff Hawkins said something like that if it predicted data matches exactly then it’s just trying to match the best thing it knows (and that’s the actual data) because it can’t recognize the pattern. So that’s not happening here? Am I seeing a plot of the data that it was just trained on, and that’s why it matches so well, or is the program just working really well at predicting? Oh, and on the general note, how do I know if my model is actually doing well with its prediction?
Is it matching exactly? Or is it trailing by one time-step? Look carefully at the timestamps. The behavior Jeff was talking about is when the prediction for the next time step matches the actual value for the current time step.
It looked exactly like your plot looked like in the demo video you made. It didn’t match exactly, but the prediction seemed to be a shifted time step ahead of the actual data, which confused me since I thought it’s supposed to be a time step behind the actual data.
Read through this old thread on the mailing list called “Help for the shifting inferences”.
Okay, so I think it’s working correctly.
In the demo video you made, if you were to remove the data that’s being given (so, lets say you want it to predict the data for the next week), then it would hypothetically keep predicting it correctly (within a margin of error, of course)?
You can find stock data from Google or Yahoo finance (but their API changed recently, and it’s a bit more involved to get data now). One newcomer to providing stock data is a New Zealand company called Alpha Vantage. They provide historical and real-time data for free. And also provide data for numerous technical indicators.