Error metrics on swarming

I saw this Wiki!
I want to use another error metrics (Root Mean Squared Error, etc) on swarming at hot gym demo.
Which and Where code should I update?

1 Like

@scott @subutai Do you know if this is possible?

I’m not sure how easy it is to use your own error metric. There is a notion of different metrics and there are at least implementations of average absolute error (aae) and mean average percent error (altMAPE). You can see them specified in a MetricSpec in the here:

I believe the swarm will use the first metric for swarming purposes but it looks like you may also be able to specify the swarm metric in the

But it’s been a while since I last ran swarms so it would take some digging and experimentation to figure out if you can swap in your own metric.

It’s also been a while since I looked into this but swarming is designed to be able to use your own metrics. If you look at nupic/frameworks/opf/ you will see there is an interface for metrics and a bunch of already defined existing metrics, including RMSE.

I don’t know how easy it is - you’ll have to figure out how to adapt per Scott’s email to specify the metric. This would be a nice example for someone to do and augment the basic swarming example.