Vitaliy Filippov
63fe3c323a
Important fixes (except the new store):
- Fixed a possible use-after-free in the OSD during error handling of initial
commit/rollback of objects in EC pools.
- Fixed a possible free of an invalid pointer in the OSD during read errors
from snapshot/clone chains in EC pools.
- Fixed possibly incorrect handling of commit/rollback operations in EC pools
during pool PG count changes.
- Fixed the inverted fsync enable parameter in the ublk driver (fsync was not
enabled on pools without immediate_commit).
- Added the raw-ls command for debugging purposes to find object versions in
the cluster using listing operations.
New store fixes:
- Improved startup speed by using LSN-based sorting only for objects with a
large number of intermediate versions.
- Added skip_double_claim option as a temporary workaround to fix the rare OSD
startup error with the "double claimed block" message, observed by several
users. This option does not affect data integrity.
- Fixed incorrect rechecking of small writes during startup, which in theory
could lead to duplicate small write object entries on the OSD.
- Fixed fsync operation for disks with a writeback cache (without capacitors):
- Fixed incorrect semantics of consecutive fsyncs (next fsync was not blocked
by the previous one).
- Added fsync when copying small writes from the buffer to the data device
(somehow forgotten during initial development).
- Added fsync after the initial garbage collection during OSD startup.
- Fixed incorrect cast of LSN from uint64 to uint32, breaking fsync when
reaching LSN 2^32.
- Added missing verification of the metadata header checksum during startup.
- Fixed incorrect updating of object checksums in perfect_csum_update=true mode.
- Fixed a possible OSD crash with "assertion failed" when processing a malformed
EC STABILIZE operation.
- Fixed the accounting of active compactor coroutines.
- Removed broken and untested new->old store conversion support.
Minor issues fixed:
- Incorrect accounting of OSD local operation statistics in replicated pools.
- Missing non-zero exitcodes on vitastor-disk resize command errors.
- Missing reset of the list of inconsistent objects during PG restarts.
- Theoretically possible hangs of various OSD operations when working with
completely corrupted objects (without a single available copy), and possibly
in some other very rare situations.
- Incorrect fsyncs when deleting objects from pools without immediate_commit
(on disks with a writeback cache), which previously could leave garbage when
deleting misplaced objects.
- Possible crash/memory corruption of the NFS server during a targeted attack
on NFS-RDMA.
- Possibly incorrect handling of ENOSPC/EIO write errors in replicated pools,
leading to inability to retry the write later.
- Possible crash instead of a clean error exit when starting an OSD with the
old storage engine on a disk with corrupted journal data.
- Shallow copying of PG configuration in the monitor, however, not related to
actual bugs.
- Incorrect checking of allocated blocks in the QEMU driver in an unused code
branch (without the BDRV_WANT_ZERO flag).
- Possible memory leak on read errors of corrupted objects.
- Possible incorrect PG states when corrupted objects are detected.
- Possible failure to mark all "bad" copies of an object during scrubs without
checksums and with a large number of replicas (> 4).
- Incorrect checksum calculation in the old storage engine when
bitmap_granularity < 4096 (a practically unused configuration).
- Theoretically possible OSD crash in rare cases during a scrub and simultaneous
object recovery.
- Theoretically possible OSD crash when handling PING operation errors.
- Slightly suboptimal logic for reusing the RDMA send buffer.
- Possible memory leak when canceling an already running scrub via no_scrub.
- Possible memory corruption when a client (e.g., QEMU code) passes invalid
buffers and the writeback cache is enabled.
- Potentially incorrect search for corrupted parts of EC objects (inability
to find a "good" combination) during a scrub with checksums disabled.
- Possible additional memory usage on the OSD side when handling failed reads
from snapshots (not a leak however - the memory was freed upon client
disconnection).
- Potential sudden write slowdown at certain pg epoch values due to incorrect
epoch update logic in etcd.
85 lines
2.8 KiB
Markdown
85 lines
2.8 KiB
Markdown
[Documentation](../../README.md#documentation) → Installation → Dockerized Installation
|
|
|
|
-----
|
|
|
|
[Читать на русском](docker.ru.md)
|
|
|
|
# Dockerized Installation
|
|
|
|
Vitastor may be installed in Docker/Podman. In such setups etcd, monitors and OSD
|
|
all run in containers, but everything else looks as close as possible to a usual
|
|
setup with packages:
|
|
- host network is used
|
|
- auto-start is implemented through udev and systemd
|
|
- logs are written to journald (not docker json log files)
|
|
- command-line wrapper scripts are installed to the host system to call vitastor-disk,
|
|
vitastor-cli and others through the container
|
|
|
|
Such installations may be useful when it's impossible or inconvenient to install
|
|
Vitastor from packages, for example, in exotic Linux distributions.
|
|
|
|
If you don't want just a simple containerized installation, you can also take a look
|
|
at Vitastor Kubernetes operator: https://github.com/Antilles7227/vitastor-operator
|
|
|
|
## Installing Containers
|
|
|
|
The instruction is very simple.
|
|
|
|
1. Download a Docker image of the desired version: \
|
|
`docker pull vitalif/vitastor:v3.0.12`
|
|
2. Install scripts to the host system: \
|
|
`docker run --rm -it -v /etc:/host-etc -v /usr/bin:/host-bin vitalif/vitastor:v3.0.12 install.sh`
|
|
3. Reload udev rules: \
|
|
`udevadm control --reload-rules`
|
|
4. Enable the vitastor-host service: \
|
|
`systemctl enable --now vitastor-host`
|
|
|
|
After these steps, you can return to [Quick Start](../intro/quickstart.en.md).
|
|
|
|
## Podman
|
|
|
|
If you use Podman, run the following commands as root before installing Vitastor containers:
|
|
|
|
```
|
|
ln -s podman /usr/bin/docker
|
|
|
|
mkdir -p /etc/systemd/system/systemd-udevd.service.d
|
|
|
|
cat >/etc/systemd/system/systemd-udevd.service.d/override.conf <<EOF
|
|
[Service]
|
|
CapabilityBoundingSet=~
|
|
SystemCallFilter=@mount capset
|
|
EOF
|
|
|
|
systemctl daemon-reload
|
|
|
|
systemctl restart systemd-udevd
|
|
```
|
|
|
|
Without it, udev fails to do calls into a Podman container and Vitastor disk detection doesn't work.
|
|
|
|
## Upgrading Containers
|
|
|
|
First make sure to check the topic [Upgrading Vitastor](../usage/admin.en.md#upgrading-vitastor)
|
|
to figure out if you need any additional steps.
|
|
|
|
Then, to upgrade a containerized installation, you just need to change the `VITASTOR_VERSION`
|
|
option in `/etc/vitastor/docker.conf` and restart all Vitastor services:
|
|
|
|
`systemctl restart vitastor.target`
|
|
|
|
## QEMU
|
|
|
|
Vitastor Docker image also contains QEMU, qemu-img and qemu-storage-daemon built with Vitastor support.
|
|
|
|
However, running QEMU in Docker is harder to setup and it depends on the used virtualization UI
|
|
(OpenNebula, Proxmox and so on). Some of them also required patched Libvirt.
|
|
|
|
That's why containerized installation of Vitastor doesn't contain a ready-made QEMU setup and it's
|
|
recommended to install QEMU from packages or build it manually.
|
|
|
|
## fio
|
|
|
|
Vitastor Docker image also contains fio and installs a wrapper called `vitastor-fio` to use it from
|
|
the host system.
|