Vitastor

Implement vitastor-kv rescue command which dumps all readable blocks from the database regardless of the tree structure and allows to recover a corrupted database.
Simplify primary OSD selection - use just PG number instead of the hash. This allows to distribute primary OSDs more evenly and reach better read speeds.
Fix vitastor-kv-stress hang on parallel list & reopen.
Fix CAS writes without immediate_commit. Fix is sufficient for vitastor-kv to operate correctly in non-immediate_commit pools.
Fix NFS ACCESS calls for broken inodes (allow root access to them).
Fix broken vitastor-kv-stress --runtime parameter.
Fix bad vitastor-kv dump output buffering leading to incorrect JSON in dumps.
Fix writing garbage from uninitialized memory to the empty space in VitastorKV DB.
Refactor some code in preparation to the release of the new log-structured metadata store.
Replace assert(done != expected) error message, probably caused by kernel issues with zero-copy TCP send, with a more concise error description.

Optimize WA - reduce it from 4-5 to 3.x by batching journal writes
Fix ls-pool showing wrong used % when osds are reweighted (#105)
Fix vitastor-cli rm sometimes hanging when PGs are down (#99)
Fix vitastor-cli ls displaying raw used space for all inodes when -p is specified (#101)
Fix “Cannot set property of undefined” in monitor when deleting pool with live inode op stats (#103)
Revert vitastor-kv change from 2.2.0 leading to crashes in vitastor-kv-stress (#100)
Add pg_size change check for XOR pools
Fix root access to VitastorFS files
Add IPv6 support to make-etcd
Fix IPv6 etcd address support in OSD & client library

New features

ublk, Veeam and checksum enablement fix.

Bugfix release.

Fix a bug introduced in 2.2.0 - pg_locks weren’t disabled for pools without local_reads correctly which could lead to inactive pools during various operations
Fix an old bug where OSDs could send sub-operations to incorrect peer OSDs when their connections were stopped and reestablished quickly, in 2.2.0 it was usually leading to “sequencing broken” messages in OSD logs
Fix debug use_sync_send_recv mode

Fix vitastor-disk purge broken after adding the “OSD is still running” check
Fix iothreads hanging after adding zero-copy send support
Fix enabling localized reads online (without restarting OSDs) in the default PG lock mode

Localized read support for multi-datacenter setups.
io_uring-based zero-copy send support - read the instruction carefully for optimal performance!
Improve and speedup data distribution, especially in cases of very large hosts (100 OSD+). Previously, PG optimization speed depended on the number of OSDs, now it only depends on the number of failure domains. Distribution over specific OSDs is now also more even and becomes strictly more even when you increase the number of PGs.
Add a very interesting instruction to change NFS_MAX_FILE_IO_SIZE
Check operation sequencing and stop connections when it breaks - should help catch some very rare RDMA packet loss problems.
vitastor-cli rm-osd now refuses to remove OSDs which are still up and suggests to use vitastor-disk purge.
Allow removal of direntries referring non-existent inodes in VitastorFS.