Fast and simple distributed
software-defined storage

Vitastor

  • Distributed SDS
  • ...but Fast 🚀 — latency 0.1 ms
  • ...and Simple ✌️ — 60k lines of code, not 1 mln
  • From Russia with love 🙂

Software-Defined Storage (SDS)

Software that assembles usual servers with usual drives into a single scalable fault-tolerant storage cluster with extended features

Why do IaaS/PaaS providers use SDS?

  • Scalability
  • Client data preservation
  • Reduced costs due to hyperconvergence
  • No need for server "micromanagement"
  • Different storage classes (SSD, HDD)

But most SDS are an overhead

Overhead

  • Q=1 — best possible latency
  • 4 kb write to SSD — 0.04 ms
  • 4 kb write to Ceph — from ~ 1 ms
  • Internal cloud SDS's – ± the same
  • 2400 % overhead !

Vitastor

  • ~ 0.1 ms replicated latency
  • ~ 0.2 ms erasure-coded latency
  • 3-8 GByte/s per client (VM)
  • ~ 400000 iops per client
  • Just ~1 CPU core per NVMe disk
  • Low (50%) rebalance impact

Vitastor — protocols

  • Block access (VM disks, containers):
    Kubernetes, Proxmox, OpenNebula, OpenStack and others
  • VitastorFS (NFS) — clustered POSIX FS
  • Object storage (S3) — based on Zenko CloudServer

Features

  • Replication, erasure codes (N+K)
  • All disk support: SSD/NVMe, HDD, SSD+HDD
  • Flexible data placement
  • RDMA / RoCEv2 support
  • Fast snapshots and clones
  • Fast checksums, data scrubbing
  • Monitoring
  • Kubernetes operator
  • And more...

Architecture

  • Symmetric distributed, Ø SPOF
  • Block base layer
  • Uniform load balancing
  • Transactional writes → data loss protection
  • Optimised for modern SSD/HDD

Ease of support

  • Low number of components
  • Human-readable metadata in etcd
  • Minimal external dependencies
  • Compact implementation:
    ~60k lines of code (Ceph ~1M)
  • Non-standard architecture support (E2K)

Licensing

  • Own Copyleft license: VNPL
  • Free use in OpenSource environment
  • Closed-source services require commercial support
  • Technical and architectural support from author

Contacts

< 0.1 ms

Random Q1 read/write with fsync

400k iops per client

Random Q128 read/write
(180k in QEMU, bounded by virtio)

Up to 8 GB/s per VM

Linear read/write per client (VM)

~1 CPU per NVMe

0.3-2 CPU, varies with I/O mode and TCP/RDMA network

Block Storage

With support for all major KVM-based systems and containers: OpenNebula, OpenStack, Proxmox VE, Kubernetes

Clustered File System

Ground-up implementation of a scalable POSIX
Read-Write-Many file system, mountable over NFS 3.0

Object Storage (S3)

Based on Zenko CloudServer

Latest Posts

Vitastor 3.0.16 released

Quite a bit of fixes, but mostly small. :-)

Most interesting fix affect pools without immediate_commit (for desktop SSDs): fixed corner cases in the commit (sync) operation and CAS write operation (used, for example, in VitastorFS).

Other notable changes: “double-claim” bug is fixed in the new store, and changes are made to reduce etcd load during active PG state switching.

You can read the details below.

2026-07-19 Continue reading →

Vitastor 3.0.15 released

Minor fixes this time:

  • A correct fix for QEMU virtual disk migration when iothread is enabled is finally found and released. Verified — migration now works. For now, QEMU packages are updated only for Proxmox 9, others will be updated shortly.
  • Fixed operations of Proxmox VMs with swTPM (for example, Windows VMs) without enabling NBD-based mounting of all disks (thanks to users from the chat).
  • It turned out that in recent releases, a version of Antietcd with broken cluster mode slipped into all Debian packages (but not RPMs) — this happened because the build used Antietcd from master rather than from a stable tag. This is now fixed - Debian packages are now built with stable Antietcd again, and the clustering issue is fixed in Antietcd itself. The issue was leading to failed leader election in a Antietcd cluster.
  • Fixed monitor startup with embedded Antietcd when using IPv6.
  • Fixed space statistics calculation for FS and S3 pools in the new storage (PR #127).

2026-06-28 Continue reading →

Vitastor 3.0.14 released

How many (bugs!) I’ve knifed, how many I’ve slit! (more than 50)

General note: most bug fixes now include regression tests to verify that they don’t repeat in the future. Most bugs fixed in this release were detected by using LLM analysis (Claude Opus/Fable, GPT 5.5).

2026-06-21 Continue reading →

Vitastor 3.0.13 released

  • New store fixes:
    • Fix repeated rollback logic
    • Fix crash on rolled back object compaction
    • Fix postpone_load possibly merging different object chains
    • Fix block_csums import in vitastor-disk write-meta
    • Fix header checksum after vitastor-disk write-meta
  • Old store fixes:
    • Fix batched fsync possibly skipped by some flush coroutines
  • Improve ENOSPC test, fix possible crash on ENOSPC
  • Add fsyncs to vitastor-disk prepare
  • Fix possible crash on pg_lock check failure in sec_read_bmp
  • Fix VitastorFS initialization when local_reads are enabled

2026-05-31 Continue reading →

Vitastor 3.0.12 released

Important fixes (except the new store)

  • Fixed a possible use-after-free in the OSD during error handling of initial commit/rollback of objects in EC pools.
  • Fixed a possible free of an invalid pointer in the OSD during read errors from snapshot/clone chains in EC pools.
  • Fixed possibly incorrect handling of commit/rollback operations in EC pools during pool PG count changes.
  • Fixed the inverted fsync enable parameter in the ublk driver (fsync was not enabled on pools without immediate_commit).
  • Added the raw-ls command for debugging purposes to find object versions in the cluster using listing operations.

2026-05-17 Continue reading →

All Posts