Vitaliy Filippov
61ededa230
New features: - Add "deleted" image flag which is set when vitastor-cli rm starts to delete an image, but can't delete it fully due to inactive PGs or stopped OSDs - Support JSON output in vitastor-disk prepare and purge - Show backfillfull pools in vitastor-cli status - Make object listings consistent (used in vitastor-cli rm/rm-data/merge/etc). This means that there is now a guarantee that if a data block is present when you invoke rm, rm will attempt to delete it, even if rm is invoked when the PG switches state. Previously in such cases rm could skip and leave some objects behind as garbage, and merge probably could incorrectly move data between snapshots. - Make deletions (rm/rm-data) consistent. This means that rm/rm-data will either complete successfully and delete all requested image data or complete with an error if some objects could not be deleted or if there is a possibility that some data is left on stopped OSDs. Previously, when some PGs or OSDs were inactive at the moment of deletion, rm-data was behaving incorrectly: it wasn't retrying deletions failed due to dropped OSD connections, it could hang waiting for PGs to activate, and it could return with a successful error code while some garbage was still possibly left on some OSDs. Deletions are not fully atomic cluster-wide yet, which means that you still have to repeat the deletion request after you return stopped OSDs back, but now you always know for sure if you have to repeat it. Bug fixes: - Fix vitastor-cli rm --exact / --matching command not working - Finally fix "Unexpected status" in the Proxmox plugin - Fix vitastor-cli create-snap incorrectly linking multiple snapshots in a different pool - Fix incomplete image parent_id loop check in OSD - Fix reads from snapshots in a different pool not working if there are more than 2 snapshots - Fix append of VITASTOR_CONF to cmdline in the opennebula prebackup script - Fix OSDs crashing again when the cluster is full with EC (was meant to work since 1.6.0 but didn't) - Improve logging of subop failures
Vitastor
The Idea
Make Clustered Block Storage Fast Again.
Vitastor is a distributed block and file SDS, direct replacement of Ceph RBD and CephFS, and also internal SDS's of public clouds. However, in contrast to them, Vitastor is fast and simple at the same time. The only thing is it's slightly young :-).
Vitastor is architecturally similar to Ceph which means strong consistency, primary-replication, symmetric clustering and automatic data distribution over any number of drives of any size with configurable redundancy (replication or erasure codes/XOR).
Vitastor targets primarily SSD and SSD+HDD clusters with at least 10 Gbit/s network, supports TCP and RDMA and may achieve 4 KB read and write latency as low as ~0.1 ms with proper hardware which is ~10 times faster than other popular SDS's like Ceph or internal systems of public clouds.
Vitastor supports QEMU, NBD, NFS protocols, OpenStack, OpenNebula, Proxmox, Kubernetes drivers. More drivers may be created easily.
Read more details in the documentation. You can start from here: Quick Start.
Talks and presentations
- DevOpsConf'2021: presentation (in Russian, in English), video
- Highload'2022: presentation (in Russian), video
Documentation
- Introduction
- Installation
- Configuration
- Usage
- vitastor-cli (command-line interface)
- vitastor-disk (disk management tool)
- fio for benchmarks
- NBD for kernel mounts
- QEMU and qemu-img
- NFS clustered file system and pseudo-FS proxy
- Administration
- Performance
Author and License
Copyright (c) Vitaliy Filippov (vitalif [at] yourcmc.ru), 2019+
Join Vitastor Telegram Chat: https://t.me/vitastor
All server-side code (OSD, Monitor and so on) is licensed under the terms of Vitastor Network Public License 1.1 (VNPL 1.1), a copyleft license based on GNU GPLv3.0 with the additional "Network Interaction" clause which requires opensourcing all programs directly or indirectly interacting with Vitastor through a computer network and expressly designed to be used in conjunction with it ("Proxy Programs"). Proxy Programs may be made public not only under the terms of the same license, but also under the terms of any GPL-Compatible Free Software License, as listed by the Free Software Foundation. This is a stricter copyleft license than the Affero GPL.
Please note that VNPL doesn't require you to open the code of proprietary software running inside a VM if it's not specially designed to be used with Vitastor.
Basically, you can't use the software in a proprietary environment to provide its functionality to users without opensourcing all intermediary components standing between the user and Vitastor or purchasing a commercial license from the author 😀.
Client libraries (cluster_client and so on) are dual-licensed under the same VNPL 1.1 and also GNU GPL 2.0 or later to allow for compatibility with GPLed software like QEMU and fio.
You can find the full text of VNPL-1.1 in the file VNPL-1.1.txt. GPL 2.0 is also included in this repository as GPL-2.0.txt.