Suppose I have 2 inputs. I know exactly what I want to use to encode one of them, and only want the swarming to work on determining suitable parameters for the encoding of the 2nd input.
Is it possible to constrain the swarming to only work on one of the inputs while I fully specifying the other?
If so, how?
(In general I’ll have n inputs that I want to specify and m that I want to swarm on.)
I am currently swarming over an input that I specify as a “string” and specify that a "C"ategory encoder should be used. The input file has 204 unique values for this input.
As expected, the swarming returns an SDRCategoryEncoder, however it specifies n=121 and w=21 for the encoder. With that n and w I cannot see how the categories can possible be unique (i.e. with no overlap of bits as I would expect in a category encoder). Is my expectation incorrect? If so why, and if not then why does the swarming return this n and w.
Good questions. I honestly don’t know the answers to them. There is probably a way to constrain a swarm using the description.py and permutations.py files instead of how I do it in my tutorials. That older way is more flexible, but honestly I never use it. There is very little documentation about how to use this interface, and we are planning on abandoning it with future implementations of swarming.
Take a look at this video at time 24:40 where Ron explains some of it:
Maybe you can figure out how to do a custom swarm from this and let us know how you did it?
The video is a good start, although deals with using json as an input not the description.py file. However it did lead me to look at permutations_runner.py.
Looking into permutations_runner.py it seems that rather than using the runWithConfig method (as in the hot_gym examples) the runWithPermutationsScript method should to be used. I’ll give that a try and report back.
While Matt prefers the simplified JSON interface to swarming, the description.py/permutations.py interface is the primary way to do swarming currently and gives much more control over the model parameters and what is swarmed over. The permutations.py variables specify a range to try for each parameter and the values that it tries override values in the base description.py so simply leaving things out of the permutations file should achieve your goal of a more constrained swarm.
It’s true that documentation is limited though. You can start with Ron’s video and then modify an existing example that has description.py and permutations.py files.