I've significantly reworked and improved both the storage backend and memory handling in RNS, including how all the major transport tables are handled. A new SQLite-based, managed (and fast) storage backend is now the default, and replaces the file/directory based storage for most things. I'm transitioning a few things at a time, though, so some parts are still on directory-based storage. If for some weird reason the standard-lib sqlite3 module is not available, or not working, RNS will fall back to the old storage backend, so even the most weird and esoteric systems should still run it just fine.
Additionally, this will be the first release that starts treating path tables as distributed across the networks. The amount of destinations have grown significantly, and it's no longer feasible for every single transport node to attempt to hold every single destination in memory (well, larger systems still wont have any problems, but embedded devices and such are getting hammered a bit now). Actual path usage is now being tracked, and tables are being culled more intelligently based on that. The immediate net result is that processing and memory load will decrease. The long-term effect and goal (which still needs a bit more work) is that path table sharding will start to form emergently across the network, and transport nodes will automatically learn what paths actually makes sense for them to keep.
Furthermore, non-transport nodes now much more aggresively cull unused destinations from tables, and this greatly reduces memory and processing load on non-transport devices, user-facing applications, appliances, et cetera.
Even further more, a new memory management system has been implemented, that tracks heap usage over time, handles garbage collection and eviction much more intelligently, and cuts long-term memory usage by about 50% for heavily loaded transport nodes.
If you want to help test this, I have just pushed the latest source version to Aleph. Any testing and feedback, especially from node operators would be much appreciated.
Please note, that to see memory improvements immediately, you will probably have to wipe your path table and known destinations storage files before first running 1.6.0. Otherwise, things will get culled out eventually, but it will happen slowly over several days at first (it's intentionally a bit conservative here).
Also, it's probably a good idea to take a backup of your .reticulum directory before running it. On first run, all the relevant storage will be migrated to the new storage backend, and Lord knows I've tested this like some kind of manic, but there's bound to be that one system somewhere where something goes wrong. If you're the one who finds the bug, I owe you a beer!
Also, if you have any questions about any of this, post them here.