Vitastor

Fix incorrect reading of extra metadata block leading to extra unknown objects in stats
Fix CSI driver volumeMode: Block support
Add block PVC and pod examples
Fix build under 32 bit architectures
Fix slow connection ramp-up caused by up_wait_retry_interval

Implement vitastor-cli status (print cluster status) command
Add a new make-osd-hybrid.js script to quickly prepare a lot of hybrid (HDD+SSD) OSDs
Implement snapshot deletion for Cinder driver (only works in a healthy cluster)
Fix a huge :) bug causing reads to return all zeroes during rebalance. Add a test to prevent it in the future
Disconnect NBD proxy correctly without leaving a zombie [vitastor-nbd] process in D state
Fix a rare write hang appearing with small write throttling enabled

Fix IPv6 address parsing
Fix “cannot read bytes of undefined” in the monitor on a fresh DB
Fix possible hangs of read requests on OSD restarts without immediate_commit=all mode
Fix OSDs skipping misplaced recovery in some cases
Fix OSDs possibly dying with “map::at” errors when other OSDs are stopped
Fix division by zero in ls if all pool OSDs are down

Fix client hangs possible on OSD restarts (bug affected versions from 0.5.11)
Fix “Assertion sqe != NULL failed” io_uring-related crashes possible on some kernels (0.6.11 increased probability of this bug)
Fix timeout=0 in NBD proxy
Fix build under centos 7

etcd connection stability, clang & elbrus support

Fix build under CLang and Elbrus LCC compilers, making Vitastor compatible with Elbrus CPUs :)
Completely fix the bug where OSDs didn’t connect to peers and incorrectly marked PGs as incomplete
Limit I/O depth for deletes the same way as for small writes. Makes OSD crashes with “Assertion failed: sqe != NULL” during image deletion go away
Fix a very old, but rare, journaling bug (credits to https://github.com/mirrorll)
Fix flushing of unclean journaled objects leading to OSDs sometimes hanging after failover in EC setups (bug was introduced in 0.6.7)
Fix several problems that could prevent smooth operation of a Vitastor cluster under the condition of partial etcd failure:
- OSDs could randomly fail due to too strict error handling
- New clients and OSDs could be unable to start because of the lack of retries
- CLI could fail some commands because of the lack of retries
- Monitor could stop receiving state updates because of the lack of websocket pings
Fix monitor being unable to rebalance PGs after a downscale of pool pg_size (3->2)
Exit with failure when trying to nbd map or benchmark a non-existing image
Use HTTP keep-alive for etcd connections
Allow to configure etcd request timeouts and retries
Allow to configure NBD timeout, max devices and partitions, and set default to up to 64 devices with up to 3 partitions each

Slightly reduce journaling write amplification (requires no_same_sector_overwrites=false)
Fix listen_backlog (it was 0) because it could more than halve OSD socket send speed
Support IPv6 OSD addresses
Do not try to initialize client in simple-offsets
Fix OSDs sometimes marking PGs incomplete instead of trying to connect with peers
Allow to configure OSD placement in node_placement
Allow to run with 4k sector size block devices. Natural, but it was forbidden

Implement a storage plugin for Proxmox. Now you can use Vitastor with Proxmox!
Implement vitastor-cli df (pool space usage statistics) command
Add glob pattern support for vitastor-cli ls
Fix several bugs in other CLI commands (resize, create --parent, modify --readonly)
Use 512 byte logical block size in QEMU driver by default (and thus don’t require to set it in QEMU options)

New features

Build Vitastor driver as part of QEMU
Implement renaming images in CLI (vitastor-cli modify --rename)
Add vitastor-cli alloc-osd and simple-offsets commands and use them in make-osd, thus removing the dependency on etcdctl
Make monitor remove stale deleted inode statistics from etcd automatically
Implement OSD address selection from a subnet, thus removing the need to specify OSD addresses in startup scripts explicitly

Build separate packages for OSD, monitor, client, C header, fio and QEMU drivers instead of one package which included everything