Cross-platform would be nice to have, probably not necessary. For me, I don’t know what others do.
In future-proff vision it’s a solid requirement, as you can train models on a cloud/farm, and then distribute it.
Why does it matter? Do you have a framework that is platform dependent?
Yaml and CapnProto are both implemented everywhere in the core, side by side. Yaml being text based (think superset of JSON). CapnProto is a binary serialization. Both work as far as I know. Yaml being a little slower. The Zip packaging is there as well I think. I know we link with it.
Both are cross platform and cross language. Nether is header only but I don’t think it worth the trouble to rip out both of what we have and implement a new one. Besides, a header only implementation would only work on C++.
The idea of a memory mapped file is one that I have been thinking about as well. All data is automatically saved, even if the program crashes. No startup time and no shutdown time. But perhaps slower execution to perform the disk io in the background.
This would require doing our classes quite differently…For example, we cannot allocate from the heap for each class. (cannot use new) We would have to allocate a fixed block of space and map it to disk. Then we have to write our own memory allocation for classes. This is doable and there are probably some tools out there for doing this but it would take some re-thinking of the algorithms.
However, I don’t think this level of ‘Saving’ is needed at this point. Perhaps when the algorithms mature to the point were we can actually write a program that implements a true AGI, then it would be important. We are a long way from that point.