Vitaliy Filippov
989d73f874
Omnidirectional release New features: - Support handling TCP I/O in simple separate io_uring-based [I/O threads](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/config/client.en.md#client_iothread_count) - may increase linear performance to 7-8 GB/s - Experimental internal etcd replacement - [antietcd](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/config/monitor.en.md#use_antietcd) - Monitor now has a [built-in Prometheus exporter](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/config/monitor.en.md#enable_prometheus) - Added a reference [Grafana dashboard](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/mon/scripts/Vitastor-Grafana-6+.json) - Implement vitastor-cli [osd-tree](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#osd-tree) and [ls-osd](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#ls-osd) commands - Implement vitastor-cli [modify-osd](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#modify-osd) command - Implement vitastor-cli [pg-list](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#pg-list) command - Implement [VitastorFS defragmentation](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/nfs.en.md#defrag) - Implement basic node.js binding (not published on npm yet) Changes: - Make immediate_commit=all the default everywhere to match default vitastor-disk behaviour - Make pool-create error message more obvious and add details to it - Set default etcd_ws_keepalive_interval to 5 seconds (speedup client etcd failover) - Support OpenStack 2023.2 in Nova and Cinder drivers/patches - Add patches for libvirt 10.x - Add patches for QEMU 8.2 and 9.0 - Implement internal restart / run_forever in monitor - Some source tree refactoring - sources are now moved into subdirectories, monitor is now split into multiple files - Add vitastor_c_inode_get_immediate_commit in vitastor_c client library - Make vitastor_kv.h header public Bug fixes: - Fix total statistics usec/count/bytes not being reported when delta (bps/iops/lat) is zero - Prevent infinite loop in NFS on files with incorrect metadata pointing to an empty volume - Fix READDIR offsets (cookies) in VitastorFS sometimes leading to client infinite loops when reading a directory - Fix a rare infinite loop during OSD journal flushing (OSD hanging and eating 100 % CPU) - Fix several bugs which could lead to lost writes in setups without immediate_commit: - Client library treated writes as completed before actually completing them, thus missing them in a subsequent fsync - Client library didn't repeat writes on the new PG primary when it changed - OSDs didn't drop peer connections with dirty writes when stopping PG - Fix Block Pseudo-FS initialization leading to ENOENTs some time after start - Fix vitastor-cli merge-based commands (merge/flatten/rm snapshot) slowing down and finally failing when using CAS optimistic locks - Fix pool create/modify --block_size validation - Fix TTL comparison for determining failed lease/keepalive requests in OSD - Add support for size suffixes in pool-create --block_size and --immediate_commit values
Vitastor
The Idea
Make Clustered Block Storage Fast Again.
Vitastor is a distributed block and file SDS, direct replacement of Ceph RBD and CephFS, and also internal SDS's of public clouds. However, in contrast to them, Vitastor is fast and simple at the same time. The only thing is it's slightly young :-).
Vitastor is architecturally similar to Ceph which means strong consistency, primary-replication, symmetric clustering and automatic data distribution over any number of drives of any size with configurable redundancy (replication or erasure codes/XOR).
Vitastor targets primarily SSD and SSD+HDD clusters with at least 10 Gbit/s network, supports TCP and RDMA and may achieve 4 KB read and write latency as low as ~0.1 ms with proper hardware which is ~10 times faster than other popular SDS's like Ceph or internal systems of public clouds.
Vitastor supports QEMU, NBD, NFS protocols, OpenStack, Proxmox, Kubernetes drivers. More drivers may be created easily.
Read more details below in the documentation.
Talks and presentations
- DevOpsConf'2021: presentation (in Russian, in English), video
- Highload'2022: presentation (in Russian), video
Documentation
- Introduction
- Installation
- Configuration
- Usage
- vitastor-cli (command-line interface)
- vitastor-disk (disk management tool)
- fio for benchmarks
- NBD for kernel mounts
- QEMU and qemu-img
- NFS clustered file system and pseudo-FS proxy
- Administration
- Performance
Author and License
Copyright (c) Vitaliy Filippov (vitalif [at] yourcmc.ru), 2019+
Join Vitastor Telegram Chat: https://t.me/vitastor
All server-side code (OSD, Monitor and so on) is licensed under the terms of Vitastor Network Public License 1.1 (VNPL 1.1), a copyleft license based on GNU GPLv3.0 with the additional "Network Interaction" clause which requires opensourcing all programs directly or indirectly interacting with Vitastor through a computer network and expressly designed to be used in conjunction with it ("Proxy Programs"). Proxy Programs may be made public not only under the terms of the same license, but also under the terms of any GPL-Compatible Free Software License, as listed by the Free Software Foundation. This is a stricter copyleft license than the Affero GPL.
Please note that VNPL doesn't require you to open the code of proprietary software running inside a VM if it's not specially designed to be used with Vitastor.
Basically, you can't use the software in a proprietary environment to provide its functionality to users without opensourcing all intermediary components standing between the user and Vitastor or purchasing a commercial license from the author 😀.
Client libraries (cluster_client and so on) are dual-licensed under the same VNPL 1.1 and also GNU GPL 2.0 or later to allow for compatibility with GPLed software like QEMU and fio.
You can find the full text of VNPL-1.1 in the file VNPL-1.1.txt. GPL 2.0 is also included in this repository as GPL-2.0.txt.