Release 3.0.12

Important fixes (except the new store):

- Fixed a possible use-after-free in the OSD during error handling of initial
  commit/rollback of objects in EC pools.
- Fixed a possible free of an invalid pointer in the OSD during read errors
  from snapshot/clone chains in EC pools.
- Fixed possibly incorrect handling of commit/rollback operations in EC pools
  during pool PG count changes.
- Fixed the inverted fsync enable parameter in the ublk driver (fsync was not
  enabled on pools without immediate_commit).
- Added the raw-ls command for debugging purposes to find object versions in
  the cluster using listing operations.

New store fixes:

- Improved startup speed by using LSN-based sorting only for objects with a
  large number of intermediate versions.
- Added skip_double_claim option as a temporary workaround to fix the rare OSD
  startup error with the "double claimed block" message, observed by several
  users. This option does not affect data integrity.
- Fixed incorrect rechecking of small writes during startup, which in theory
  could lead to duplicate small write object entries on the OSD.
- Fixed fsync operation for disks with a writeback cache (without capacitors):
  - Fixed incorrect semantics of consecutive fsyncs (next fsync was not blocked
    by the previous one).
  - Added fsync when copying small writes from the buffer to the data device
    (somehow forgotten during initial development).
  - Added fsync after the initial garbage collection during OSD startup.
  - Fixed incorrect cast of LSN from uint64 to uint32, breaking fsync when
    reaching LSN 2^32.
- Added missing verification of the metadata header checksum during startup.
- Fixed incorrect updating of object checksums in perfect_csum_update=true mode.
- Fixed a possible OSD crash with "assertion failed" when processing a malformed
  EC STABILIZE operation.
- Fixed the accounting of active compactor coroutines.
- Removed broken and untested new->old store conversion support.

Minor issues fixed:

- Incorrect accounting of OSD local operation statistics in replicated pools.
- Missing non-zero exitcodes on vitastor-disk resize command errors.
- Missing reset of the list of inconsistent objects during PG restarts.
- Theoretically possible hangs of various OSD operations when working with
  completely corrupted objects (without a single available copy), and possibly
  in some other very rare situations.
- Incorrect fsyncs when deleting objects from pools without immediate_commit
  (on disks with a writeback cache), which previously could leave garbage when
  deleting misplaced objects.
- Possible crash/memory corruption of the NFS server during a targeted attack
  on NFS-RDMA.
- Possibly incorrect handling of ENOSPC/EIO write errors in replicated pools,
  leading to inability to retry the write later.
- Possible crash instead of a clean error exit when starting an OSD with the
  old storage engine on a disk with corrupted journal data.
- Shallow copying of PG configuration in the monitor, however, not related to
  actual bugs.
- Incorrect checking of allocated blocks in the QEMU driver in an unused code
  branch (without the BDRV_WANT_ZERO flag).
- Possible memory leak on read errors of corrupted objects.
- Possible incorrect PG states when corrupted objects are detected.
- Possible failure to mark all "bad" copies of an object during scrubs without
  checksums and with a large number of replicas (> 4).
- Incorrect checksum calculation in the old storage engine when
  bitmap_granularity < 4096 (a practically unused configuration).
- Theoretically possible OSD crash in rare cases during a scrub and simultaneous
  object recovery.
- Theoretically possible OSD crash when handling PING operation errors.
- Slightly suboptimal logic for reusing the RDMA send buffer.
- Possible memory leak when canceling an already running scrub via no_scrub.
- Possible memory corruption when a client (e.g., QEMU code) passes invalid
  buffers and the writeback cache is enabled.
- Potentially incorrect search for corrupted parts of EC objects (inability
  to find a "good" combination) during a scrub with checksums disabled.
- Possible additional memory usage on the OSD side when handling failed reads
  from snapshots (not a leak however - the memory was freed upon client
  disconnection).
- Potential sudden write slowdown at certain pg epoch values due to incorrect
  epoch update logic in etcd.

.gitea/workflows/test.yml

@@ -63,7 +63,7 @@ jobs:
     container: ${{env.TEST_IMAGE}}:${{github.sha}}
     steps:
     # leak sanitizer sometimes crashes
-    - run: cd /root/vitastor/build && ASAN_OPTIONS=detect_leaks=0 make -j16 test
+    - run: cd /root/vitastor/build && ASAN_OPTIONS=detect_leaks=0 make -j16 build_tests test
 
   npm_lint:
     runs-on: ubuntu-latest
@@ -234,60 +234,6 @@ jobs:
           echo ""
         done
 
-  test_etcd_fail_https:
-    runs-on: ubuntu-latest
-    needs: build
-    container: ${{env.TEST_IMAGE}}:${{github.sha}}
-    steps:
-    - name: Run test
-      id: test
-      timeout-minutes: 10
-      run: ETCD_SCHEME=https /root/vitastor/tests/test_etcd_fail.sh
-    - name: Print logs
-      if: always() && steps.test.outcome == 'failure'
-      run: |
-        for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
-          echo "-------- $i --------"
-          cat $i
-          echo ""
-        done
-
-  test_etcd_fail_https_antietcd:
-    runs-on: ubuntu-latest
-    needs: build
-    container: ${{env.TEST_IMAGE}}:${{github.sha}}
-    steps:
-    - name: Run test
-      id: test
-      timeout-minutes: 10
-      run: ETCD_SCHEME=https ANTIETCD=1 /root/vitastor/tests/test_etcd_fail.sh
-    - name: Print logs
-      if: always() && steps.test.outcome == 'failure'
-      run: |
-        for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
-          echo "-------- $i --------"
-          cat $i
-          echo ""
-        done
-
-  test_snapshot_https:
-    runs-on: ubuntu-latest
-    needs: build
-    container: ${{env.TEST_IMAGE}}:${{github.sha}}
-    steps:
-    - name: Run test
-      id: test
-      timeout-minutes: 3
-      run: ETCD_SCHEME=https /root/vitastor/tests/test_snapshot.sh
-    - name: Print logs
-      if: always() && steps.test.outcome == 'failure'
-      run: |
-        for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
-          echo "-------- $i --------"
-          cat $i
-          echo ""
-        done
-
   test_interrupted_rebalance:
     runs-on: ubuntu-latest
     needs: build
@@ -702,24 +648,6 @@ jobs:
           echo ""
         done
 
-  test_snapshot_chain_encrypted:
-    runs-on: ubuntu-latest
-    needs: build
-    container: ${{env.TEST_IMAGE}}:${{github.sha}}
-    steps:
-    - name: Run test
-      id: test
-      timeout-minutes: 3
-      run: ENCRYPTED=1 /root/vitastor/tests/test_snapshot_chain.sh
-    - name: Print logs
-      if: always() && steps.test.outcome == 'failure'
-      run: |
-        for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
-          echo "-------- $i --------"
-          cat $i
-          echo ""
-        done
-
   test_old_snapshot_chain:
     runs-on: ubuntu-latest
     needs: build
@@ -1296,24 +1224,6 @@ jobs:
           echo ""
         done
 
-  test_checksum_xxhash:
-    runs-on: ubuntu-latest
-    needs: build
-    container: ${{env.TEST_IMAGE}}:${{github.sha}}
-    steps:
-    - name: Run test
-      id: test
-      timeout-minutes: 3
-      run: TEST_NAME=xxhash OSD_ARGS="--data_csum_type xxh3_32" /root/vitastor/tests/test_checksum.sh
-    - name: Print logs
-      if: always() && steps.test.outcome == 'failure'
-      run: |
-        for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
-          echo "-------- $i --------"
-          cat $i
-          echo ""
-        done
-
   test_old_checksum:
     runs-on: ubuntu-latest
     needs: build
@@ -2142,39 +2052,3 @@ jobs:
           echo ""
         done
 
-  test_write_encrypted:
-    runs-on: ubuntu-latest
-    needs: build
-    container: ${{env.TEST_IMAGE}}:${{github.sha}}
-    steps:
-    - name: Run test
-      id: test
-      timeout-minutes: 3
-      run: /root/vitastor/tests/test_write_encrypted.sh
-    - name: Print logs
-      if: always() && steps.test.outcome == 'failure'
-      run: |
-        for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
-          echo "-------- $i --------"
-          cat $i
-          echo ""
-        done
-
-  test_write_encrypted_ec:
-    runs-on: ubuntu-latest
-    needs: build
-    container: ${{env.TEST_IMAGE}}:${{github.sha}}
-    steps:
-    - name: Run test
-      id: test
-      timeout-minutes: 3
-      run: SCHEME=ec /root/vitastor/tests/test_write_encrypted.sh
-    - name: Print logs
-      if: always() && steps.test.outcome == 'failure'
-      run: |
-        for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
-          echo "-------- $i --------"
-          cat $i
-          echo ""
-        done
-

.gitea/workflows/tests-to-yaml.pl

@@ -38,14 +38,6 @@ for my $line (<>)
             {
                 $test_name .= '_antietcd';
             }
-            elsif ($1 eq 'ETCD_SCHEME' && $2 eq 'https')
-            {
-                $test_name .= '_https';
-            }
-            elsif ($1 eq 'ENCRYPTED')
-            {
-                $test_name .= '_encrypted';
-            }
             elsif ($1 eq 'OLD')
             {
                 $test_name =~ s/^test_/test_old_/s;

.gitignore

@@ -3,4 +3,3 @@
 package-lock.json
 fio
 qemu
-node_modules

CMakeLists.txt

@@ -1,20 +1,20 @@
-cmake_minimum_required(VERSION 2.8.12)
+cmake_minimum_required(VERSION 2.8...3.30)
 
 project(vitastor)
 
-set(VITASTOR_VERSION "3.0.5")
+set(VITASTOR_VERSION "3.0.12")
 
 include(CTest)
 
 add_custom_target(build_tests)
-add_custom_target(test
-	COMMAND
-	echo leak:tcmalloc > ${CMAKE_CURRENT_BINARY_DIR}/lsan-suppress.txt &&
-	env LSAN_OPTIONS=suppressions=${CMAKE_CURRENT_BINARY_DIR}/lsan-suppress.txt ${CMAKE_CTEST_COMMAND}
+set_property(TEST PROPERTY ENVIRONMENT LSAN_OPTIONS=suppressions=${CMAKE_CURRENT_BINARY_DIR}/lsan-suppress.txt)
+add_test(gen_lsan_suppress
+    ${CMAKE_COMMAND} -E echo leak:tcmalloc > "${CMAKE_CURRENT_BINARY_DIR}/lsan-suppress.txt"
 )
+set_tests_properties(gen_lsan_suppress PROPERTIES FIXTURES_SETUP f_lsan_suppress)
+set_property(TEST PROPERTY FIXTURES_REQUIRED f_lsan_suppress)
 # make -j16 -C ../../build test_heap && ../../build/src/test/test_heap
 # make -j16 -C ../../build test_heap && rm -f $(find ../../build -name '*.gcda') && ctest -V -T test -T coverage -R heap --test-dir ../../build && (cd ../../build; gcovr -f ../src --html --html-nested -o coverage/index.html; cd ../src/test)
 # make -j16 -C ../../build test_blockstore && rm -f $(find ../../build -name '*.gcda') && ctest -V -T test -T coverage -R blockstore --test-dir ../../build && (cd ../../build; gcovr -f ../src --html --html-nested -o coverage/index.html; cd ../src/test)
 # kcov --include-path=../../../src ../../kcov ./test_blockstore
-add_dependencies(test build_tests)
 add_subdirectory(src)

README-ru.md

@@ -62,7 +62,6 @@ Vitastor поддерживает QEMU-драйвер, протоколы UBLK,
     - [Дисковые параметры OSD](docs/config/layout-osd.ru.md)
     - [Прочие параметры OSD](docs/config/osd.ru.md)
     - [Параметры мониторов](docs/config/monitor.ru.md)
-    - [Безопасность](docs/config/security.ru.md)
   - [Настройки пулов](docs/config/pool.ru.md)
   - [Метаданные образов в etcd](docs/config/inode.ru.md)
 - Использование

README.md

@@ -62,7 +62,6 @@ Read more details in the documentation. You can start from here: [Quick Start](d
     - [OSD Disk Layout](docs/config/layout-osd.en.md)
     - [OSD Runtime Parameters](docs/config/osd.en.md)
     - [Monitor](docs/config/monitor.en.md)
-    - [Security](docs/config/security.en.md)
   - [Pool configuration](docs/config/pool.en.md)
   - [Image metadata in etcd](docs/config/inode.en.md)
 - Usage

csi/Makefile

@@ -1,4 +1,4 @@
-VITASTOR_VERSION ?= v3.0.5
+VITASTOR_VERSION ?= v3.0.12
 
 all: build push
 

csi/deploy/004-csi-nodeplugin.yaml

@@ -49,7 +49,7 @@ spec:
             capabilities:
               add: ["SYS_ADMIN"]
             allowPrivilegeEscalation: true
-          image: vitalif/vitastor-csi:v3.0.5
+          image: vitalif/vitastor-csi:v3.0.12
           args:
             - "--node=$(NODE_ID)"
             - "--endpoint=$(CSI_ENDPOINT)"

csi/deploy/007-csi-provisioner.yaml

@@ -121,7 +121,7 @@ spec:
             privileged: true
             capabilities:
               add: ["SYS_ADMIN"]
-          image: vitalif/vitastor-csi:v3.0.5
+          image: vitalif/vitastor-csi:v3.0.12
           args:
             - "--node=$(NODE_ID)"
             - "--endpoint=$(CSI_ENDPOINT)"

csi/src/config.go

@@ -5,7 +5,7 @@ package vitastor
 
 const (
     vitastorCSIDriverName    = "csi.vitastor.io"
-    vitastorCSIDriverVersion = "3.0.5"
+    vitastorCSIDriverVersion = "3.0.12"
 )
 
 // Config struct fills the parameters of request or user input

debian/changelog

@@ -1,4 +1,4 @@
-vitastor (3.0.5-1) unstable; urgency=medium
+vitastor (3.0.12-1) unstable; urgency=medium
 
   * Bugfixes
 

debian/control

@@ -3,7 +3,7 @@ Section: admin
 Priority: optional
 Maintainer: Vitaliy Filippov <vitalif@yourcmc.ru>
 Build-Depends: debhelper, g++ (>= 8), libstdc++6 (>= 8),
-  linux-libc-dev, libgoogle-perftools-dev, libjerasure-dev, libgf-complete-dev, libc-ares-dev,
+  linux-libc-dev, libgoogle-perftools-dev, libjerasure-dev, libgf-complete-dev,
   libibverbs-dev, librdmacm-dev, libisal-dev, cmake, pkg-config, libnl-3-dev, libnl-genl-3-dev,
   node-bindings <!nocheck>, node-gyp, node-nan
 Standards-Version: 4.5.0

debian/vitastor-build.sh

@@ -44,7 +44,7 @@ curl -s https://git.yourcmc.ru/vitalif/antietcd/archive/master.tar.gz | tar -zx
 curl -s https://git.yourcmc.ru/vitalif/tinyraft/archive/master.tar.gz | tar -zx
 
 cd /root/vitastor/packages/vitastor-$REL
-if [[ "$REL" = "trixie" && -e ../vitastor-bookworm/vitastor_$VER.orig.tar.xz ]]; then
+if [[ ( "$REL" = "trixie" || "$REL" = "resolute" ) && -e ../vitastor-bookworm/vitastor_$VER.orig.tar.xz ]]; then
     # Fucking shit, archives differ between bookworm (xz 5.4.1) and trixie (xz 5.8.1)
     cp ../vitastor-bookworm/vitastor_$VER.orig.tar.xz .
 else

debian/vitastor-buildenv.Dockerfile

@@ -25,7 +25,7 @@ RUN set -e -x; \
     echo 'APT::Install-Suggests false;' >> /etc/apt/apt.conf
 
 RUN apt-get update && \
-    apt-get -y install fio libgoogle-perftools-dev devscripts libjerasure-dev cmake libc-ares-dev \
+    apt-get -y install fio libgoogle-perftools-dev devscripts libjerasure-dev cmake \
         libibverbs-dev librdmacm-dev libisal-dev libnl-3-dev libnl-genl-3-dev curl nodejs npm node-nan node-bindings && \
     apt-get -y build-dep fio && \
     apt-get --download-only source fio

docker/Makefile

@@ -1,4 +1,4 @@
-VITASTOR_VERSION ?= v3.0.5
+VITASTOR_VERSION ?= v3.0.12
 
 all: build push
 

docker/etc/vitastor/docker.conf

@@ -4,7 +4,7 @@
 #
 
 # Desired Vitastor version
-VITASTOR_VERSION=v3.0.5
+VITASTOR_VERSION=v3.0.12
 
 # Additional arguments for all containers
 # For example, you may want to specify a custom logging driver here

docs/config.en.md

@@ -38,4 +38,3 @@ In the future, additional configuration methods may be added:
 - [OSD Disk Layout](config/layout-osd.en.md)
 - [OSD Runtime Parameters](config/osd.en.md)
 - [Monitor](config/monitor.en.md)
-- [Security Parameters](config/security.en.md)

docs/config.ru.md

@@ -41,4 +41,3 @@
 - [Дисковые параметры OSD](config/layout-osd.ru.md)
 - [Прочие параметры OSD](config/osd.ru.md)
 - [Параметры мониторов](config/monitor.ru.md)
-- [Параметры безопасности](config/security.ru.md)

docs/config/layout-osd.en.md

@@ -198,14 +198,8 @@ put a modified value into etcd key /vitastor/config/global.
 - Type: string
 - Default: none
 
-Data and metadata checksum type to use. May be "crc32c", "xxh3_32" or "none".
-Select crc32c or xxh3_32 and set csum_block_size to enable data checksums.
-
-Both crc32c and xxh3_32 are almost equally fast, xxh3_32 is safer. xxh3_32 is
-the xxhash3 algorithm truncated from 64 to 32 bits (which is still a good hash).
-
-Note that enabled data checksums either increase memory usage or reduce
-performance. Check details in [csum_block_size](#csum_block_size) description.
+Data checksum type to use. May be "crc32c" or "none". Set to "crc32c" to
+enable data checksums.
 
 ## csum_block_size
 

docs/config/layout-osd.ru.md

@@ -209,12 +209,8 @@ journal_block_size и meta_block_size. Однако на данный момен
 - Тип: строка
 - Значение по умолчанию: none
 
-Тип используемых OSD контрольных сумм данных и метаданных. Может быть "crc32c",
-"xxh3_32" или "none". Выберите crc32c или xxh3_32 и установите csum_block_size,
-чтобы включить контрольные суммы данных.
-
-И crc32c, и xxh3_32 примерно одинаково быстры, xxh3_32 надёжней. xxh3_32 - это
-алгоритм xxhash3, обрезанный с 64 до 32 бит (это всё равно хороший хеш).
+Тип используемых OSD контрольных сумм данных. Может быть "crc32c" или "none".
+Установите в "crc32c", чтобы включить расчёт и проверку контрольных сумм данных.
 
 Следует понимать, что контрольные суммы в зависимости от размера блока их
 расчёта либо увеличивают потребление памяти, либо снижают производительность.

docs/config/osd.en.md

@@ -70,6 +70,7 @@ with an OSD restart or, for some of them, even without restarting by updating co
 - [use_atomic_flag](#use_atomic_flag)
 - [pg_reshard_chunk_size](#pg_reshard_chunk_size)
 - [pg_reshard_chunk_pause_ms](#pg_reshard_chunk_pause_ms)
+- [gc_on_start](#gc_on_start)
 
 ## bind_address
 
@@ -753,3 +754,9 @@ This option sets the maximum number of object is a chunk. Moving 100k objects us
 - Default: 100
 
 This option sets the interval between handling two PG count change chunks.
+
+## gc_on_start
+
+- Type: boolean
+
+Forcibly clean all garbage entries in the new store on every OSD restart.

docs/config/osd.ru.md

@@ -71,6 +71,7 @@
 - [use_atomic_flag](#use_atomic_flag)
 - [pg_reshard_chunk_size](#pg_reshard_chunk_size)
 - [pg_reshard_chunk_pause_ms](#pg_reshard_chunk_pause_ms)
+- [gc_on_start](#gc_on_start)
 
 ## bind_address
 
@@ -793,3 +794,9 @@ pg_minsize OSD во время переключений, что может по
 - Значение по умолчанию: 100
 
 Данная опция задаёт интервал между обработкой двух порций изменения числа PG пулов.
+
+## gc_on_start
+
+- Тип: булево (да/нет)
+
+Принудительно очищать все мусорные записи в новом хранилище при каждом запуске OSD.

docs/config/security.en.md

@@ -1,150 +0,0 @@
-[Documentation](../../README.md#documentation) → [Configuration](../config.en.md) → Security Parameters
-
------
-
-[Читать на русском](security.ru.md)
-
-# Security Parameters
-
-These parameters affect your Vitastor installation security and apply to OSDs, monitors and clients.
-
-Most of them can be set in /etc/vitastor/vitastor.conf and in etcd, but don't support online modification.
-
-- [etcd_client_cert](#etcd_client_cert)
-- [etcd_client_key](#etcd_client_key)
-- [etcd_ca](#etcd_ca)
-- [osd_etcd_client_cert](#osd_etcd_client_cert)
-- [osd_etcd_client_key](#osd_etcd_client_key)
-- [mon_etcd_client_cert](#mon_etcd_client_cert)
-- [mon_etcd_client_key](#mon_etcd_client_key)
-- [vault_url](#vault_url)
-- [vault_secret_api_path](#vault_secret_api_path)
-- [vault_client_cert](#vault_client_cert)
-- [vault_client_key](#vault_client_key)
-- [vault_ca](#vault_ca)
-- [vault_timeout_ms](#vault_timeout_ms)
-- [vault_error_timeout_sec](#vault_error_timeout_sec)
-- [vault_refresh_leeway_sec](#vault_refresh_leeway_sec)
-- [max_aes_xts_pool_size](#max_aes_xts_pool_size)
-
-## etcd_client_cert
-
-- Type: string
-
-Client TLS certificate to use for Vitastor client (not OSD and not monitor)
-etcd https connections. May be path to a file or just a PEM string with certificate.
-In the latter case, string must begin with "-----BEGIN CERTIFICATE-----".
-
-## etcd_client_key
-
-- Type: string
-
-Private key for etcd_client_cert (also a file or a PEM string).
-
-## etcd_ca
-
-- Type: string
-
-Trusted TLS CA to verify etcd server certificate. May be path to a file,
-directory or just a PEM string with certificate.
-
-## osd_etcd_client_cert
-
-- Type: string
-
-Same as [etcd_client_cert](#etcd_client_cert), but only for OSDs.
-OSDs, clients and monitors should have different permissions, so they should
-use different certificates.
-
-## osd_etcd_client_key
-
-- Type: string
-
-Same as [etcd_client_key](#etcd_client_key), but only for OSDs.
-
-## mon_etcd_client_cert
-
-- Type: string
-
-Same as [etcd_client_cert](#etcd_client_cert), but only for Vitastor monitors.
-
-## mon_etcd_client_key
-
-- Type: string
-
-Same as [etcd_client_key](#etcd_client_key), but only for Vitastor monitors.
-
-## vault_url
-
-- Type: string
-
-Vault base URL.
-
-Vitastor clients support AES-256-XTS image data encryption with different per-image keys.
-Encryption is performed by the client, OSDs don't have access to decrypted data.
-
-Encryption keys may be stored in etcd or, for the increased security level, in an external
-[HashiCorp Vault](https://developer.hashicorp.com/vault/) or [OpenBao](https://openbao.org/)
-instance.
-
-Vitastor clients use [v1 k/v secrets engine](https://openbao.org/api-docs/secret/kv/kv-v1/)
-and [TLS authentication engine](https://openbao.org/api-docs/auth/cert/) in Vault.
-
-In that case, only key IDs are stored in etcd.
-
-## vault_secret_api_path
-
-- Type: string
-- Default: /v1/secret/
-
-Vault v1 secret API mount path to use.
-
-## vault_client_cert
-
-- Type: string
-
-Client TLS certificate to use for Vault connections. Just like [etcd_client_cert](#etcd_client_cert),
-may be path to a file or just a certificate in PEM string.
-
-## vault_client_key
-
-- Type: string
-
-Private key for vault_client_cert (also a file or a PEM string).
-
-## vault_ca
-
-- Type: string
-
-Trusted TLS CA to verify Vault server certificate. May be path to a file,
-directory or just a PEM string with certificate.
-
-## vault_timeout_ms
-
-- Type: integer
-- Default: 5000
-
-Timeout for Vault requests in milliseconds.
-
-## vault_error_timeout_sec
-
-- Type: integer
-- Default: 60
-
-Time (in seconds) to wait before retrying after receiving an error from Vault.
-
-## vault_refresh_leeway_sec
-
-- Type: integer
-- Default: 60
-
-Extra time (in seconds) before real Vault token lease_timeout to refresh it, just
-in case of system clock drift.
-
-## max_aes_xts_pool_size
-
-- Type: integer
-- Default: 256
-
-Maximum number of OpenSSL encryption contexts cached in OSD memory. Probably
-doesn't require modification.

docs/config/security.ru.md

@@ -1,154 +0,0 @@
-[Документация](../../README-ru.md#документация) → [Конфигурация](../config.ru.md) → Параметры безопасности
-
------
-
-[Read in English](security.en.md)
-
-# Параметры безопасности
-
-Данные параметры затрагивают безопасность инсталляций Vitastor и используются
-OSD, мониторами и клиентами.
-
-Большая их часть может задаваться в /etc/vitastor/vitastor.conf и в etcd, но не
-поддерживает онлайн-изменение.
-
-- [etcd_client_cert](#etcd_client_cert)
-- [etcd_client_key](#etcd_client_key)
-- [etcd_ca](#etcd_ca)
-- [osd_etcd_client_cert](#osd_etcd_client_cert)
-- [osd_etcd_client_key](#osd_etcd_client_key)
-- [mon_etcd_client_cert](#mon_etcd_client_cert)
-- [mon_etcd_client_key](#mon_etcd_client_key)
-- [vault_url](#vault_url)
-- [vault_secret_api_path](#vault_secret_api_path)
-- [vault_client_cert](#vault_client_cert)
-- [vault_client_key](#vault_client_key)
-- [vault_ca](#vault_ca)
-- [vault_timeout_ms](#vault_timeout_ms)
-- [vault_error_timeout_sec](#vault_error_timeout_sec)
-- [vault_refresh_leeway_sec](#vault_refresh_leeway_sec)
-- [max_aes_xts_pool_size](#max_aes_xts_pool_size)
-
-## etcd_client_cert
-
-- Тип: строка
-
-Клиентский TLS сертификат для https-подключений к etcd для клиентов Vitastor
-(не OSD и не мониторов). Может быть путём к файлу или просто строкой с
-сертификатом в формате PEM. В последнем случае строка должна начинаться с
-"-----BEGIN CERTIFICATE-----".
-
-## etcd_client_key
-
-- Тип: строка
-
-Закрытый ключ для сертификата etcd_client_cert (также путь к файлу или PEM строка).
-
-## etcd_ca
-
-- Тип: строка
-
-Доверенный корневой TLS-сертификат для проверки сертификата сервера etcd.
-Может быть путём к файлу, директории или просто строкой с сертификатом в
-формате PEM.
-
-## osd_etcd_client_cert
-
-- Тип: строка
-
-Аналогично [etcd_client_cert](#etcd_client_cert), но только для OSD.
-OSD, клиенты и мониторы должны иметь разные привилегии, поэтому они должны
-использовать разные сертификаты.
-
-## osd_etcd_client_key
-
-- Тип: строка
-
-Аналогично [etcd_client_key](#etcd_client_key), но только для OSD.
-
-## mon_etcd_client_cert
-
-- Тип: строка
-
-Аналогично [etcd_client_cert](#etcd_client_cert), но только для мониторов Vitastor.
-
-## mon_etcd_client_key
-
-- Тип: строка
-
-Аналогично [etcd_client_key](#etcd_client_key), но только для мониторов Vitastor.
-
-## vault_url
-
-- Тип: строка
-
-Базовый адрес Vault.
-
-Клиенты Vitastor поддерживают AES-256-XTS шифрование данных образов с отдельными ключами на
-каждый образ. Данные шифруются клиентами, OSD не имеют доступа к незашифрованным данным.
-
-Ключи шифрования могут храниться в etcd или, для повышенного уровня безопасности, во внешнем
-[HashiCorp Vault](https://developer.hashicorp.com/vault/) или [OpenBao](https://openbao.org/).
-
-Клиенты Vitastor используют [движок секретов v1](https://openbao.org/api-docs/secret/kv/kv-v1/)
-и [TLS-аутентификацию](https://openbao.org/api-docs/auth/cert/) в Vault.
-
-В этом случае, только ID ключей хранятся в etcd.
-
-## vault_secret_api_path
-
-- Тип: строка
-- Значение по умолчанию: /v1/secret/
-
-Путь к API секретов v1 для использования клиентами.
-
-## vault_client_cert
-
-- Тип: строка
-
-Клиентский TLS сертификат для подключений к Vault. Как и [etcd_client_cert](#etcd_client_cert),
-может быть путём к файлу или просто PEM-строкой с сертификатом.
-
-## vault_client_key
-
-- Тип: строка
-
-Закрытый ключ для сертификата vault_client_cert (также путь к файлу или PEM строка).
-
-## vault_ca
-
-- Тип: строка
-
-Доверенный корневой TLS-сертификат для проверки сертификата сервера Vault.
-Может быть путём к файлу, директории или просто строкой с сертификатом в
-формате PEM.
-
-## vault_timeout_ms
-
-- Тип: целое число
-- Значение по умолчанию: 5000
-
-Максимально время выполнения Vault-запросов в миллисекундах.
-
-## vault_error_timeout_sec
-
-- Тип: целое число
-- Значение по умолчанию: 60
-
-Время (в секундах) для ожидания перед повторной попыткой при получении ошибки от Vault.
-
-## vault_refresh_leeway_sec
-
-- Тип: целое число
-- Значение по умолчанию: 60
-
-Зазор времени (в секундах), чтобы обновлять токены Vault чуть раньше их реального
-lease_timeout, на случай "ухода" системных часов.
-
-## max_aes_xts_pool_size
-
-- Тип: целое число
-- Значение по умолчанию: 256
-
-Максимальное количество кэшируемых в памяти OSD контекстов шифрования OpenSSL.
-Вряд ли требует изменения.

docs/config/src/included.en.md

@@ -44,8 +44,6 @@
 
 {{../../config/monitor.en.md|indent=2}}
 
-{{../../config/security.en.md|indent=2}}
-
 {{../../config/pool.en.md|indent=2}}
 
 {{../../config/inode.en.md|indent=2}}

docs/config/src/included.ru.md

@@ -44,8 +44,6 @@
 
 {{../../config/monitor.ru.md|indent=2}}
 
-{{../../config/security.ru.md|indent=2}}
-
 {{../../config/pool.ru.md|indent=2}}
 
 {{../../config/inode.ru.md|indent=2}}

docs/config/src/layout-osd.yml

@@ -233,21 +233,11 @@
   type: string
   default: none
   info: |
-    Data and metadata checksum type to use. May be "crc32c", "xxh3_32" or "none".
-    Select crc32c or xxh3_32 and set csum_block_size to enable data checksums.
-
-    Both crc32c and xxh3_32 are almost equally fast, xxh3_32 is safer. xxh3_32 is
-    the xxhash3 algorithm truncated from 64 to 32 bits (which is still a good hash).
-
-    Note that enabled data checksums either increase memory usage or reduce
-    performance. Check details in [csum_block_size](#csum_block_size) description.
+    Data checksum type to use. May be "crc32c" or "none". Set to "crc32c" to
+    enable data checksums.
   info_ru: |
-    Тип используемых OSD контрольных сумм данных и метаданных. Может быть "crc32c",
-    "xxh3_32" или "none". Выберите crc32c или xxh3_32 и установите csum_block_size,
-    чтобы включить контрольные суммы данных.
-
-    И crc32c, и xxh3_32 примерно одинаково быстры, xxh3_32 надёжней. xxh3_32 - это
-    алгоритм xxhash3, обрезанный с 64 до 32 бит (это всё равно хороший хеш).
+    Тип используемых OSD контрольных сумм данных. Может быть "crc32c" или "none".
+    Установите в "crc32c", чтобы включить расчёт и проверку контрольных сумм данных.
 
     Следует понимать, что контрольные суммы в зависимости от размера блока их
     расчёта либо увеличивают потребление памяти, либо снижают производительность.

docs/config/src/osd.yml

@@ -938,3 +938,7 @@
     This option sets the interval between handling two PG count change chunks.
   info_ru: |
     Данная опция задаёт интервал между обработкой двух порций изменения числа PG пулов.
+- name: gc_on_start
+  type: bool
+  info: Forcibly clean all garbage entries in the new store on every OSD restart.
+  info_ru: Принудительно очищать все мусорные записи в новом хранилище при каждом запуске OSD.

docs/config/src/package.json

@@ -1,5 +0,0 @@
-{
-  "dependencies": {
-    "yaml": "^2.8.2"
-  }
-}

docs/config/src/security.en.md

@@ -1,5 +0,0 @@
-# Security Parameters
-
-These parameters affect your Vitastor installation security and apply to OSDs, monitors and clients.
-
-Most of them can be set in /etc/vitastor/vitastor.conf and in etcd, but don't support online modification.

docs/config/src/security.ru.md

@@ -1,7 +0,0 @@
-# Параметры безопасности
-
-Данные параметры затрагивают безопасность инсталляций Vitastor и используются
-OSD, мониторами и клиентами.
-
-Большая их часть может задаваться в /etc/vitastor/vitastor.conf и в etcd, но не
-поддерживает онлайн-изменение.

docs/config/src/security.yml

@@ -1,131 +0,0 @@
-- name: etcd_client_cert
-  type: string
-  info: |
-    Client TLS certificate to use for Vitastor client (not OSD and not monitor)
-    etcd https connections. May be path to a file or just a PEM string with certificate.
-    In the latter case, string must begin with "-----BEGIN CERTIFICATE-----".
-  info_ru: |
-    Клиентский TLS сертификат для https-подключений к etcd для клиентов Vitastor
-    (не OSD и не мониторов). Может быть путём к файлу или просто строкой с
-    сертификатом в формате PEM. В последнем случае строка должна начинаться с
-    "-----BEGIN CERTIFICATE-----".
-- name: etcd_client_key
-  type: string
-  info: Private key for etcd_client_cert (also a file or a PEM string).
-  info_ru: Закрытый ключ для сертификата etcd_client_cert (также путь к файлу или PEM строка).
-- name: etcd_ca
-  type: string
-  info: |
-    Trusted TLS CA to verify etcd server certificate. May be path to a file,
-    directory or just a PEM string with certificate.
-  info_ru: |
-    Доверенный корневой TLS-сертификат для проверки сертификата сервера etcd.
-    Может быть путём к файлу, директории или просто строкой с сертификатом в
-    формате PEM.
-- name: osd_etcd_client_cert
-  type: string
-  info: |
-    Same as [etcd_client_cert](#etcd_client_cert), but only for OSDs.
-    OSDs, clients and monitors should have different permissions, so they should
-    use different certificates.
-  info_ru: |
-    Аналогично [etcd_client_cert](#etcd_client_cert), но только для OSD.
-    OSD, клиенты и мониторы должны иметь разные привилегии, поэтому они должны
-    использовать разные сертификаты.
-- name: osd_etcd_client_key
-  type: string
-  info: Same as [etcd_client_key](#etcd_client_key), but only for OSDs.
-  info_ru: Аналогично [etcd_client_key](#etcd_client_key), но только для OSD.
-- name: mon_etcd_client_cert
-  type: string
-  info: Same as [etcd_client_cert](#etcd_client_cert), but only for Vitastor monitors.
-  info_ru: Аналогично [etcd_client_cert](#etcd_client_cert), но только для мониторов Vitastor.
-- name: mon_etcd_client_key
-  type: string
-  info: Same as [etcd_client_key](#etcd_client_key), but only for Vitastor monitors.
-  info_ru: Аналогично [etcd_client_key](#etcd_client_key), но только для мониторов Vitastor.
-- name: vault_url
-  type: string
-  info: |
-    Vault base URL.
-
-    Vitastor clients support AES-256-XTS image data encryption with different per-image keys.
-    Encryption is performed by the client, OSDs don't have access to decrypted data.
-
-    Encryption keys may be stored in etcd or, for the increased security level, in an external
-    [HashiCorp Vault](https://developer.hashicorp.com/vault/) or [OpenBao](https://openbao.org/)
-    instance.
-
-    Vitastor clients use [v1 k/v secrets engine](https://openbao.org/api-docs/secret/kv/kv-v1/)
-    and [TLS authentication engine](https://openbao.org/api-docs/auth/cert/) in Vault.
-
-    In that case, only key IDs are stored in etcd.
-  info_ru: |
-    Базовый адрес Vault.
-
-    Клиенты Vitastor поддерживают AES-256-XTS шифрование данных образов с отдельными ключами на
-    каждый образ. Данные шифруются клиентами, OSD не имеют доступа к незашифрованным данным.
-
-    Ключи шифрования могут храниться в etcd или, для повышенного уровня безопасности, во внешнем
-    [HashiCorp Vault](https://developer.hashicorp.com/vault/) или [OpenBao](https://openbao.org/).
-
-    Клиенты Vitastor используют [движок секретов v1](https://openbao.org/api-docs/secret/kv/kv-v1/)
-    и [TLS-аутентификацию](https://openbao.org/api-docs/auth/cert/) в Vault.
-
-    В этом случае, только ID ключей хранятся в etcd.
-- name: vault_secret_api_path
-  type: string
-  default: /v1/secret/
-  info: Vault v1 secret API mount path to use.
-  info_ru: Путь к API секретов v1 для использования клиентами.
-- name: vault_client_cert
-  type: string
-  info: |
-    Client TLS certificate to use for Vault connections. Just like [etcd_client_cert](#etcd_client_cert),
-    may be path to a file or just a certificate in PEM string.
-  info_ru: |
-    Клиентский TLS сертификат для подключений к Vault. Как и [etcd_client_cert](#etcd_client_cert),
-    может быть путём к файлу или просто PEM-строкой с сертификатом.
-- name: vault_client_key
-  type: string
-  info: Private key for vault_client_cert (also a file or a PEM string).
-  info_ru: Закрытый ключ для сертификата vault_client_cert (также путь к файлу или PEM строка).
-- name: vault_ca
-  type: string
-  info: |
-    Trusted TLS CA to verify Vault server certificate. May be path to a file,
-    directory or just a PEM string with certificate.
-  info_ru: |
-    Доверенный корневой TLS-сертификат для проверки сертификата сервера Vault.
-    Может быть путём к файлу, директории или просто строкой с сертификатом в
-    формате PEM.
-- name: vault_timeout_ms
-  type: int
-  default: 5000
-  info: Timeout for Vault requests in milliseconds.
-  info_ru: Максимально время выполнения Vault-запросов в миллисекундах.
-- name: vault_error_timeout_sec
-  type: int
-  default: 60
-  info: |
-    Time (in seconds) to wait before retrying after receiving an error from Vault.
-  info_ru: |
-    Время (в секундах) для ожидания перед повторной попыткой при получении ошибки от Vault.
-- name: vault_refresh_leeway_sec
-  type: int
-  default: 60
-  info: |
-    Extra time (in seconds) before real Vault token lease_timeout to refresh it, just
-    in case of system clock drift.
-  info_ru: |
-    Зазор времени (в секундах), чтобы обновлять токены Vault чуть раньше их реального
-    lease_timeout, на случай "ухода" системных часов.
-- name: max_aes_xts_pool_size
-  type: int
-  default: 256
-  info: |
-    Maximum number of OpenSSL encryption contexts cached in OSD memory. Probably
-    doesn't require modification.
-  info_ru: |
-    Максимальное количество кэшируемых в памяти OSD контекстов шифрования OpenSSL.
-    Вряд ли требует изменения.

docs/installation/docker.en.md

@@ -26,9 +26,9 @@ at Vitastor Kubernetes operator: https://github.com/Antilles7227/vitastor-operat
 The instruction is very simple.
 
 1. Download a Docker image of the desired version: \
-   `docker pull vitalif/vitastor:v3.0.5`
+   `docker pull vitalif/vitastor:v3.0.12`
 2. Install scripts to the host system: \
-   `docker run --rm -it -v /etc:/host-etc -v /usr/bin:/host-bin vitalif/vitastor:v3.0.5 install.sh`
+   `docker run --rm -it -v /etc:/host-etc -v /usr/bin:/host-bin vitalif/vitastor:v3.0.12 install.sh`
 3. Reload udev rules: \
    `udevadm control --reload-rules`
 4. Enable the vitastor-host service: \

docs/installation/docker.ru.md

@@ -25,9 +25,9 @@ Vitastor можно установить в Docker/Podman. При этом etcd,
 Инструкция по установке максимально простая.
 
 1. Скачайте Docker-образ желаемой версии: \
-   `docker pull vitalif/vitastor:v3.0.5`
+   `docker pull vitalif/vitastor:v3.0.12`
 2. Установите скрипты в хост-систему командой: \
-   `docker run --rm -it -v /etc:/host-etc -v /usr/bin:/host-bin vitalif/vitastor:v3.0.5 install.sh`
+   `docker run --rm -it -v /etc:/host-etc -v /usr/bin:/host-bin vitalif/vitastor:v3.0.12 install.sh`
 3. Перезагрузите правила udev: \
    `udevadm control --reload-rules`
 4. Включите сервис vitastor-host: \

docs/installation/packages.en.md

@@ -17,6 +17,7 @@
   - Debian 10 (Buster): `deb https://vitastor.io/debian buster main`
   - Ubuntu 22.04 (Jammy): `deb https://vitastor.io/debian jammy main`
   - Ubuntu 24.04 (Noble): `deb https://vitastor.io/debian noble main`
+  - Ubuntu 26.04 (Resolute): `deb https://vitastor.io/debian resolute main`
   - Add `-oldstable` to bookworm/bullseye/buster in this line to install the last
     stable version from 0.9.x branch instead of 1.x
 - To always prefer vitastor-patched QEMU and Libvirt versions, add the following to `/etc/apt/preferences`:

docs/installation/packages.ru.md

@@ -17,6 +17,7 @@
   - Debian 10 (Buster): `deb https://vitastor.io/debian buster main`
   - Ubuntu 22.04 (Jammy): `deb https://vitastor.io/debian jammy main`
   - Ubuntu 24.04 (Noble): `deb https://vitastor.io/debian noble main`
+  - Ubuntu 26.04 (Resolute): `deb https://vitastor.io/debian resolute main`
   - Добавьте `-oldstable` к слову bookworm/bullseye/buster в этой строке, чтобы
     установить последнюю стабильную версию из ветки 0.9.x вместо 1.x
 - Чтобы всегда предпочитались версии пакетов QEMU и Libvirt с патчами Vitastor, добавьте в `/etc/apt/preferences`:

docs/installation/source.en.md

@@ -15,9 +15,8 @@
 - gcc and g++ 8 or newer, clang 10 or newer, or other compiler with C++11 plus
   designated initializers support from C++20
 - CMake
-- jerasure, c-ares headers and libraries
+- jerasure headers and libraries
 - ISA-L, libibverbs, librdmacm, libnl3 headers and libraries (optional)
-- tcmalloc (google-perftools-dev)
 
 ## Basic instructions
 

docs/installation/source.ru.md

@@ -15,9 +15,8 @@
 - gcc и g++ >= 8, либо clang >= 10, либо другой компилятор с поддержкой C++11 плюс
   назначенных инициализаторов (designated initializers) из C++20
 - CMake
-- Заголовки и библиотеки jerasure, c-ares
+- Заголовки и библиотеки jerasure
 - Опционально - заголовки и библиотеки ISA-L, libibverbs, librdmacm, libnl3
-- tcmalloc (google-perftools-dev)
 
 ## Базовая инструкция
 

docs/intro/features.en.md

@@ -41,8 +41,6 @@
 - [Built-in Prometheus metric exporter](../config/monitor.en.md#enable_prometheus)
 - [NFS RDMA support](../usage/nfs.en.md#rdma) (probably also usable for GPUDirect)
 - [S3](../installation/s3.en.md)
-- [TLS support for etcd connections](../config/security.en.md)
-- [AES-256-XTS image encryption](../usage/cli.en.md#create) and [Vault support](../config/security.en.md#vault_url) for key storage
 
 ## Plugins and tools
 

docs/intro/features.ru.md

@@ -43,8 +43,6 @@
 - [Встроенный Prometheus-экспортер метрик](../config/monitor.ru.md#enable_prometheus)
 - [Поддержка NFS RDMA](../usage/nfs.ru.md#rdma) (вероятно, также подходящая для GPUDirect)
 - [S3](../installation/s3.ru.md)
-- [Поддержка TLS-соединений с etcd](../config/security.ru.md)
-- [AES-256-XTS шифрование данных](../usage/cli.ru.md#create) и [поддержка Vault](../config/security.ru.md#vault_url) для хранения ключей
 
 ## Драйверы и инструменты
 

docs/usage/cli.en.md

@@ -125,31 +125,18 @@ bench-kaveri         kaveri    10 G  10 G    0 B/s  0     0      0 us  0 B/s  0
 
 ## create
 
-`vitastor-cli create -s|--size SIZE [OPTIONS] <name>`
+`vitastor-cli create -s|--size <size> [-p|--pool <id|name>] [--parent <parent_name>[@<snapshot>]] <name>`
 
-Create an image. Options:
-
-* `-s|--size SIZE` - New image size in bytes or with a K/M/G/T unit suffix.
-* `-p|--pool POOL` - Specify pool for the new image (may be omitted if there is only 1 pool).
-* `--parent PARENT` - Create a copy-on-write image clone based on PARENT (or PARENT@SNAPSHOT).
-  If parent is not a snapshot, it must be a read-only image.
-* `--enc-key random` - Generate a new random AES-256-XTS encryption key for the new image.
-* `--enc-key HEX` - Set a specified AES-256-XTS key (64 bytes in hex) for the new image.
-* `--enc-key vault:ID` - Use an encryption key from an external Vault secret with specified ID.
+Create an image. You may use K/M/G/T suffixes for `<size>`. If `--parent` is specified,
+a copy-on-write image clone is created. Parent must be a snapshot (readonly image).
+Pool must be specified if there is more than one pool.
 
 ```
-vitastor-cli create --snapshot <snapshot> [OPTIONS] <image>
-vitastor-cli snap-create [OPTIONS] <image>@<snapshot>
+vitastor-cli create --snapshot <snapshot> [-p|--pool <id|name>] <image>
+vitastor-cli snap-create [-p|--pool <id|name>] <image>@<snapshot>
 ```
 
-Create a snapshot of image `<image>`. May be used live if only a single writer is active.
-
-Options:
-
-* `-p|--pool POOL` - Move image to pool POOL, leaving the snapshot in the old pool.
-* `--enc-key random` - Change image encryption key to a new random AES-256-XTS key.
-* `--enc-key KEY` - Change image encryption key to a specified key, Vault key or to an empty key.
-  By default, the image retains its old encryption key when taking a snapshot.
+Create a snapshot of image `<name>` (either form can be used). May be used live if only a single writer is active.
 
 See also about [how to export snapshots](qemu.en.md#exporting-snapshots).
 
@@ -164,7 +151,6 @@ You should resize file system in the image, if present, before shrinking it.
 * `--deleted 1|0` - Set/clear 'deleted image' flag (set automatically during unfinished deletes).
 * `-f|--force` - Proceed with shrinking or setting readwrite flag even if the image has children.
 * `--down-ok` - Proceed with shrinking even if some data will be left on unavailable OSDs.
-* `--enc-key HEX` - Change image encryption key (allowed only with `--force`).
 
 ## dd
 

docs/usage/cli.ru.md

@@ -127,32 +127,19 @@ bench-kaveri         kaveri    10 G  10 G    0 B/s  0     0      0 us  0 B/s  0
 
 ## create
 
-`vitastor-cli create -s|--size SIZE [ОПЦИИ] <name>`
+`vitastor-cli create -s|--size <size> [-p|--pool <id|name>] [--parent <parent_name>[@<snapshot>]] <name>`
 
-Создать образ. Опции:
-
-* `-s|--size SIZE` - Размер нового образа в байтах или с суффиксом K/M/G/T (кило/мега/гига/терабайт).
-* `-p|--pool POOL` - Создать образ в заданном пуле (можно не указывать, если пул всего один).
-* `--parent PARENT` - Создать легковесный клон на основе образа `PARENT` или снимка `PARENT@SNAP`.
-  Если `PARENT` - не снимок, он должен быть помечен как образ только для чтения.
-* `--enc-key random` - Сгенерировать случайный ключ шифрования AES-256-XTS для нового образа.
-* `--enc-key HEX` - Установить заданный ключ AES-256-XTS (64 байта в hex) для нового образа.
-* `--enc-key vault:ID` - Использовать ключ из внешнего секрета с заданным ID из Vault.
+Создать образ. Для размера `<size>` можно использовать суффиксы K/M/G/T (килобайт-мегабайт-гигабайт-терабайт).
+Если указана опция `--parent`, создаётся клон образа. Родитель `<parent_name>[@<snapshot>]` должен быть
+снимком (или просто немодифицируемым образом). Пул обязательно указывать, если в кластере больше одного пула.
 
 ```
-vitastor-cli create --snapshot <snapshot> [ОПЦИИ] <image>
-vitastor-cli snap-create [ОПЦИИ] <image>@<snapshot>
+vitastor-cli create --snapshot <snapshot> [-p|--pool <id|name>] <image>
+vitastor-cli snap-create [-p|--pool <id|name>] <image>@<snapshot>
 ```
 
-Создать снимок образа `<image>` (можно использовать любую форму команды).
-Снимок можно создавать без остановки клиентов, если пишущих клиентов не больше одного.
-
-Опции:
-
-* `-p|--pool POOL` - Переместить образ в пул POOL, оставив снимок в старом пуле.
-* `--enc-key random` - Изменить ключ шифрования образа на новый случайный ключ AES-256-XTS.
-* `--enc-key KEY` - Изменить ключ шифрования образа на заданный ключ, ключ из Vault или пустой ключ.
-  По умолчанию шифрованные образы сохраняют старый ключ при снятии снимка.
+Создать снимок образа `<name>` (можно использовать любую форму команды). Снимок можно создавать без остановки
+клиентов, если пишущий клиент максимум 1.
 
 Смотрите также информацию о том, [как экспортировать снимки](qemu.ru.md#экспорт-снимков).
 
@@ -169,7 +156,6 @@ vitastor-cli snap-create [ОПЦИИ] <image>@<snapshot>
 * `--deleted 1|0` - Установить/снять флаг "образ удалён" (устанавливается при незавершённом удалении).
 * `-f|--force` - Разрешить уменьшение или перевод в чтение-запись образа, у которого есть клоны.
 * `--down-ok` - Разрешить уменьшение, даже если часть данных останется неудалённой на недоступных OSD.
-* `--enc-key HEX` - Изменить ключ шифрования образа (разрешено только с `--force`).
 
 ## dd
 

docs/usage/nfs.en.md

@@ -262,3 +262,4 @@ Options:
 | `--logfile <FILE>` | log to the specified file                                |
 | `--enforce 1`      | enforce permissions at the server side (no by default)   |
 | `--foreground 1`   | stay in foreground, do not daemonize                     |
+| `--trace`          | trace all NFS requests                                   |

docs/usage/nfs.ru.md

@@ -274,3 +274,4 @@ VitastorFS из GPUDirect.
 | `--logfile <FILE>` | записывать логи в заданный файл                         |
 | `--enforce 1`      | проверять права доступа на стороне сервера (по умолчанию нет) |
 | `--foreground 1`   | не уходить в фон после запуска                          |
+| `--trace`          | логгировать все запросы NFS                             |

mon/antietcd_adapter.js

@@ -18,7 +18,7 @@ class AntiEtcdAdapter
                 cluster = cluster ? (''+(cluster||'')).split(/,+/) : [];
             cluster = Object.keys(cluster.reduce((a, url) =>
             {
-                a[url.toLowerCase().replace(/^(https?:\/\/)?(.*?)(\/.*)?$/, (m, m1, m2) => (m1||'http://')+m2)] = true;
+                a[url.toLowerCase().replace(/^(https?:\/\/)/, '').replace(/\/.*$/, '')] = true;
                 return a;
             }, {}));
             const cfg_port = config.antietcd_port;
@@ -26,8 +26,7 @@ class AntiEtcdAdapter
             is_local['0.0.0.0'] = true;
             is_local['::'] = true;
             is_local[''] = true;
-            // split :, 3 -> <schema>:<//ip>:<port>
-            const selected = cluster.map(s => s.split(':', 3)).filter(ip => is_local[ip[1].substr(2)] && (!cfg_port || ip[2] == cfg_port));
+            const selected = cluster.map(s => s.split(':', 2)).filter(ip => is_local[ip[0]] && (!cfg_port || ip[1] == cfg_port));
             if (selected.length > 1)
             {
                 console.error('More than 1 etcd_address matches local IPs, please specify port');
@@ -36,15 +35,12 @@ class AntiEtcdAdapter
             else if (selected.length == 1)
             {
                 const antietcd_config = {
-                    ip: selected[0][1].substr(2),
-                    port: selected[0][2],
-                    cert: config.antietcd_cert,
-                    key: config.antietcd_key,
-                    ca: config.etcd_ca,
-                    data: config.antietcd_data_file || ((config.antietcd_data_dir || '/var/lib/vitastor') + '/mon_'+selected[0][2]+'.json.gz'),
+                    ip: selected[0][0],
+                    port: selected[0][1],
+                    data: config.antietcd_data_file || ((config.antietcd_data_dir || '/var/lib/vitastor') + '/mon_'+selected[0][1]+'.json.gz'),
                     persist_filter: vitastor_persist_filter({ vitastor_prefix: config.etcd_prefix || '/vitastor' }),
-                    node_id: selected[0][1].substr(2)+':'+selected[0][2], // node_id = ip:port
-                    cluster: (cluster.length == 1 ? null : cluster.reduce((a, c) => { a[c.replace(/^(https?:\/\/)/, '')] = c; return a; }, {})),
+                    node_id: selected[0][0]+':'+selected[0][1], // node_id = ip:port
+                    cluster: (cluster.length == 1 ? null : cluster.reduce((a, c) => { a[c] = "http://"+c; return a; }, {})),
                     cluster_key: (config.etcd_prefix || '/vitastor'),
                     stale_read: 1,
                     log_level: 1,

mon/etcd_adapter.js

@@ -1,9 +1,7 @@
 // Copyright (c) Vitaliy Filippov, 2019+
 // License: VNPL-1.1 (see README.md for details)
 
-const fs = require('fs');
 const http = require('http');
-const https = require('https');
 const WebSocket = require('ws');
 const { b64, local_ips } = require('./utils.js');
 
@@ -17,30 +15,11 @@ class EtcdAdapter
         this.ws = null;
         this.ws_alive = false;
         this.ws_keepalive_timer = null;
-        this.opts = {};
     }
 
     parse_config(config)
     {
         this.parse_etcd_addresses(config.etcd_address||config.etcd_url);
-        if (config.mon_etcd_client_cert || config.etcd_client_cert)
-        {
-            this.opts.cert = config.mon_etcd_client_cert || config.etcd_client_cert;
-            if (this.opts.cert.substr(0, 5) != '-----')
-                this.opts.cert = fs.readFileSync(this.opts.cert, { encoding: 'utf-8' });
-        }
-        if (config.mon_etcd_client_key || config.etcd_client_key)
-        {
-            this.opts.key = config.mon_etcd_client_key || config.etcd_client_key;
-            if (this.opts.key.substr(0, 5) != '-----')
-                this.opts.key = fs.readFileSync(this.opts.key, { encoding: 'utf-8' });
-        }
-        if (config.etcd_ca)
-        {
-            this.opts.ca = config.etcd_ca;
-            if (this.opts.ca.substr(0, 5) != '-----')
-                this.opts.ca = fs.readFileSync(this.opts.ca, { encoding: 'utf-8' });
-        }
     }
 
     parse_etcd_addresses(addrs)
@@ -60,7 +39,7 @@ class EtcdAdapter
         for (let url of addrs)
         {
             let scheme = 'http';
-            url = url.trim().replace(/^(https?):\/\//i, (m, m1) => { scheme = m1.toLowerCase(); return ''; });
+            url = url.trim().replace(/^(https?):\/\//, (m, m1) => { scheme = m1; return ''; });
             const slash = url.indexOf('/');
             const colon = url.indexOf(':');
             const is_local = is_local_ip[colon >= 0 ? url.substr(0, colon) : (slash >= 0 ? url.substr(0, slash) : url)];
@@ -151,7 +130,7 @@ class EtcdAdapter
                     }
                     ok(false);
                 }, this.mon.config.etcd_mon_timeout);
-                this.ws = new WebSocket(base+'/watch', this.opts);
+                this.ws = new WebSocket(base+'/watch');
                 this.ws_used_url = cur_addr;
                 const fail = () =>
                 {
@@ -293,7 +272,7 @@ class EtcdAdapter
             {
                 throw new Error(MON_STOPPED);
             }
-            const res = await POST(base+path, body, timeout, this.opts);
+            const res = await POST(base+path, body, timeout);
             if (this.mon.stopped)
             {
                 throw new Error(MON_STOPPED);
@@ -319,7 +298,7 @@ class EtcdAdapter
     }
 }
 
-function POST(url, body, timeout, opts)
+function POST(url, body, timeout)
 {
     return new Promise(ok =>
     {
@@ -331,10 +310,10 @@ function POST(url, body, timeout, opts)
             req = null;
             ok({ error: 'timeout' });
         }, timeout) : null;
-        let req = (url.substr(0, 5) == 'https' ? https : http).request(url, { method: 'POST', headers: {
+        let req = http.request(url, { method: 'POST', headers: {
             'Content-Type': 'application/json',
             'Content-Length': body_text.length,
-        }, ...(opts||{}) }, (res) =>
+        } }, (res) =>
         {
             if (!req)
             {

mon/etcd_schema.js

@@ -45,14 +45,7 @@ const etcd_tree = {
             config_path: "/etc/vitastor/vitastor.conf",
             etcd_prefix: "/vitastor",
             // etcd connection - configurable online
-            etcd_address: "http://10.0.115.10:2379/v3",
-            etcd_client_cert: "",
-            etcd_client_key: "",
-            osd_etcd_client_cert: "",
-            osd_etcd_client_key: "",
-            mon_etcd_client_cert: "",
-            mon_etcd_client_key: "",
-            etcd_ca: "",
+            etcd_address: "10.0.115.10:2379/v3",
             // mon
             etcd_mon_ttl: 5, // min: 1
             etcd_mon_timeout: 1000, // ms. min: 0
@@ -224,8 +217,6 @@ const etcd_tree = {
                     parent_id?: <inode_t>,
                     readonly?: boolean,
                     deleted?: boolean,
-                    enc_key?: string,
-                    meta?: any,
                 }
             }
         }, */
@@ -392,7 +383,6 @@ const etcd_tree = {
             /* <name>: {
                 id: uint64_t,
                 pool_id: uint64_t,
-                // ...plus a copy of everything from config/inode/x/y
             }, */
         },
         maxid: {

mon/mon.js

@@ -627,7 +627,7 @@ class Mon
                 if (this.state.pg.history[pool_id] &&
                     this.state.pg.history[pool_id][pg])
                 {
-                    pg_history[pg-1] = this.state.pg.history[pool_id][pg];
+                    pg_history[pg-1] = JSON.parse(JSON.stringify(this.state.pg.history[pool_id][pg]));
                 }
             }
             const real_prev_pgs = [];
@@ -719,7 +719,7 @@ class Mon
                 this.next_recheck_timer = null;
                 this.next_recheck_at = 0;
                 this.schedule_recheck();
-            }, now-this.next_recheck_at);
+            }, (this.next_recheck_at-now)*1000);
         }
     }
 

mon/package.json

@@ -1,6 +1,6 @@
 {
   "name": "vitastor-mon",
-  "version": "3.0.5",
+  "version": "3.0.12",
   "description": "Vitastor SDS monitor service",
   "main": "mon-main.js",
   "scripts": {

mon/pg_utils.js

@@ -84,7 +84,7 @@ function scale_pg_history(prev_pg_history, prev_pgs, new_pgs)
         finish_pg_history(merged_history[1]);
         for (let i = 0; i < new_pg_count; i++)
         {
-            new_pg_history[i] = { ...merged_history[1] };
+            new_pg_history[i] = JSON.parse(JSON.stringify(merged_history[1]));
         }
     }
     // Mark history keys for removed PGs as removed
@@ -102,7 +102,7 @@ function scale_pg_count(prev_pgs, new_pg_count)
     {
         for (let i = prev_pgs.length; i < new_pg_count; i++)
         {
-            prev_pgs[i] = prev_pgs[i % prev_pgs.length];
+            prev_pgs[i] = [ ...prev_pgs[i % prev_pgs.length] ];
         }
     }
     else if (prev_pgs.length > new_pg_count)

mon/stats.js

@@ -37,6 +37,7 @@ function derive_osd_stats(st, prev, prev_diff)
         const n = c.count - BigInt(pr && pr.count||0);
         diff.recovery_stats[op] = { ...c, bps: n > 0 ? b*1000n/timediff : 0n, iops: n > 0 ? n*1000n/timediff : 0n };
     }
+    diff.inode_stats = {};
     for (const pool_id in st.inode_stats||{})
     {
         diff.inode_stats[pool_id] = {};

node-binding/package.json

@@ -1,6 +1,6 @@
 {
   "name": "vitastor",
-  "version": "3.0.5",
+  "version": "3.0.12",
   "description": "Low-level native bindings to Vitastor client library",
   "main": "index.js",
   "keywords": [

patches/cinder-vitastor.py

@@ -50,7 +50,7 @@ from cinder.volume import configuration
 from cinder.volume import driver
 from cinder.volume import volume_utils
 
-VITASTOR_VERSION = '3.0.5'
+VITASTOR_VERSION = '3.0.12'
 
 LOG = logging.getLogger(__name__)
 

patches/libvirt-12.0-vitastor.diff

@@ -0,0 +1,637 @@
+diff --git a/include/libvirt/libvirt-storage.h b/include/libvirt/libvirt-storage.h
+index aaad4a3da1..5f5daa8341 100644
+--- a/include/libvirt/libvirt-storage.h
++++ b/include/libvirt/libvirt-storage.h
+@@ -326,6 +326,7 @@ typedef enum {
+     VIR_CONNECT_LIST_STORAGE_POOLS_ZFS           = 1 << 17, /* (Since: 1.2.8) */
+     VIR_CONNECT_LIST_STORAGE_POOLS_VSTORAGE      = 1 << 18, /* (Since: 3.1.0) */
+     VIR_CONNECT_LIST_STORAGE_POOLS_ISCSI_DIRECT  = 1 << 19, /* (Since: 5.6.0) */
++    VIR_CONNECT_LIST_STORAGE_POOLS_VITASTOR      = 1 << 20, /* (Since: 5.0.0) */
+ } virConnectListAllStoragePoolsFlags;
+ 
+ int                     virConnectListAllStoragePools(virConnectPtr conn,
+diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
+index 9ca5c2450c..cc52f00c0c 100644
+--- a/src/conf/domain_conf.c
++++ b/src/conf/domain_conf.c
+@@ -7453,7 +7453,8 @@ virDomainDiskSourceNetworkParse(xmlNodePtr node,
+     src->configFile = virXPathString("string(./config/@file)", ctxt);
+ 
+     if (src->protocol == VIR_STORAGE_NET_PROTOCOL_HTTP ||
+-        src->protocol == VIR_STORAGE_NET_PROTOCOL_HTTPS)
++        src->protocol == VIR_STORAGE_NET_PROTOCOL_HTTPS ||
++        src->protocol == VIR_STORAGE_NET_PROTOCOL_VITASTOR)
+         src->query = virXMLPropString(node, "query");
+ 
+     if (virDomainStorageNetworkParseHosts(node, ctxt, &src->hosts, &src->nhosts) < 0)
+@@ -32187,6 +32188,7 @@ virDomainStorageSourceTranslateSourcePool(virStorageSource *src,
+ 
+     case VIR_STORAGE_POOL_MPATH:
+     case VIR_STORAGE_POOL_RBD:
++    case VIR_STORAGE_POOL_VITASTOR:
+     case VIR_STORAGE_POOL_SHEEPDOG:
+     case VIR_STORAGE_POOL_GLUSTER:
+     case VIR_STORAGE_POOL_LAST:
+diff --git a/src/conf/domain_validate.c b/src/conf/domain_validate.c
+index 7346a61731..83e94d762e 100644
+--- a/src/conf/domain_validate.c
++++ b/src/conf/domain_validate.c
+@@ -520,6 +520,7 @@ virDomainDiskDefValidateSourceChainOne(const virStorageSource *src)
+         case VIR_STORAGE_NET_PROTOCOL_RBD:
+             break;
+ 
++        case VIR_STORAGE_NET_PROTOCOL_VITASTOR:
+         case VIR_STORAGE_NET_PROTOCOL_NBD:
+         case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG:
+         case VIR_STORAGE_NET_PROTOCOL_GLUSTER:
+@@ -592,7 +593,7 @@ virDomainDiskDefValidateSourceChainOne(const virStorageSource *src)
+         }
+     }
+ 
+-    /* internal snapshots and config files are currently supported only with rbd: */
++    /* internal snapshots are currently supported only with rbd: */
+     if (virStorageSourceGetActualType(src) != VIR_STORAGE_TYPE_NETWORK &&
+         src->protocol != VIR_STORAGE_NET_PROTOCOL_RBD) {
+         if (src->snapshot) {
+@@ -600,10 +601,14 @@ virDomainDiskDefValidateSourceChainOne(const virStorageSource *src)
+                            _("<snapshot> element is currently supported only with 'rbd' disks"));
+             return -1;
+         }
+-
++    }
++    /* config files are currently supported only with rbd and vitastor: */
++    if (virStorageSourceGetActualType(src) != VIR_STORAGE_TYPE_NETWORK &&
++        src->protocol != VIR_STORAGE_NET_PROTOCOL_RBD &&
++        src->protocol != VIR_STORAGE_NET_PROTOCOL_VITASTOR) {
+         if (src->configFile) {
+             virReportError(VIR_ERR_XML_ERROR, "%s",
+-                           _("<config> element is currently supported only with 'rbd' disks"));
++                           _("<config> element is currently supported only with 'rbd' and 'vitastor' disks"));
+             return -1;
+         }
+     }
+diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincommon.rng
+index 114dd3f96f..c71f9a3277 100644
+--- a/src/conf/schemas/domaincommon.rng
++++ b/src/conf/schemas/domaincommon.rng
+@@ -2093,6 +2093,35 @@
+     </element>
+   </define>
+ 
++  <define name="diskSourceNetworkProtocolVitastor">
++    <element name="source">
++      <interleave>
++        <attribute name="protocol">
++          <value>vitastor</value>
++        </attribute>
++        <ref name="diskSourceCommon"/>
++        <optional>
++          <attribute name="name"/>
++        </optional>
++        <optional>
++          <attribute name="query"/>
++        </optional>
++        <zeroOrMore>
++          <ref name="diskSourceNetworkHost"/>
++        </zeroOrMore>
++        <optional>
++          <element name="config">
++            <attribute name="file">
++              <ref name="absFilePath"/>
++            </attribute>
++            <empty/>
++          </element>
++        </optional>
++        <empty/>
++      </interleave>
++    </element>
++  </define>
++
+   <define name="diskSourceNetworkProtocolISCSI">
+     <element name="source">
+       <attribute name="protocol">
+@@ -2443,6 +2472,7 @@
+       <ref name="diskSourceNetworkProtocolSimple"/>
+       <ref name="diskSourceNetworkProtocolVxHS"/>
+       <ref name="diskSourceNetworkProtocolNFS"/>
++      <ref name="diskSourceNetworkProtocolVitastor"/>
+     </choice>
+   </define>
+ 
+diff --git a/src/conf/storage_conf.c b/src/conf/storage_conf.c
+index 1dc9365bf2..a8a736be81 100644
+--- a/src/conf/storage_conf.c
++++ b/src/conf/storage_conf.c
+@@ -56,7 +56,7 @@ VIR_ENUM_IMPL(virStoragePool,
+               "logical", "disk", "iscsi",
+               "iscsi-direct", "scsi", "mpath",
+               "rbd", "sheepdog", "gluster",
+-              "zfs", "vstorage",
++              "zfs", "vstorage", "vitastor",
+ );
+ 
+ VIR_ENUM_IMPL(virStoragePoolFormatFileSystem,
+@@ -242,6 +242,18 @@ static virStoragePoolTypeInfo poolTypeInfo[] = {
+           .formatToString = virStorageFileFormatTypeToString,
+       }
+     },
++    {.poolType = VIR_STORAGE_POOL_VITASTOR,
++     .poolOptions = {
++         .flags = (VIR_STORAGE_POOL_SOURCE_HOST |
++                   VIR_STORAGE_POOL_SOURCE_NETWORK |
++                   VIR_STORAGE_POOL_SOURCE_NAME),
++      },
++      .volOptions = {
++          .defaultFormat = VIR_STORAGE_FILE_RAW,
++          .formatFromString = virStorageVolumeFormatFromString,
++          .formatToString = virStorageFileFormatTypeToString,
++      }
++    },
+     {.poolType = VIR_STORAGE_POOL_SHEEPDOG,
+      .poolOptions = {
+          .flags = (VIR_STORAGE_POOL_SOURCE_HOST |
+@@ -538,6 +550,11 @@ virStoragePoolDefParseSource(xmlXPathContextPtr ctxt,
+                        _("element 'name' is mandatory for RBD pool"));
+         return -1;
+     }
++    if (pool_type == VIR_STORAGE_POOL_VITASTOR && source->name == NULL) {
++        virReportError(VIR_ERR_XML_ERROR, "%s",
++                       _("element 'name' is mandatory for Vitastor pool"));
++        return -1;
++    }
+ 
+     if (options->formatFromString) {
+         g_autofree char *format = NULL;
+@@ -1127,6 +1144,7 @@ virStoragePoolDefFormatBuf(virBuffer *buf,
+     /* RBD, Sheepdog, Gluster and Iscsi-direct devices are not local block devs nor
+      * files, so they don't have a target */
+     if (def->type != VIR_STORAGE_POOL_RBD &&
++        def->type != VIR_STORAGE_POOL_VITASTOR &&
+         def->type != VIR_STORAGE_POOL_SHEEPDOG &&
+         def->type != VIR_STORAGE_POOL_GLUSTER &&
+         def->type != VIR_STORAGE_POOL_ISCSI_DIRECT) {
+diff --git a/src/conf/storage_conf.h b/src/conf/storage_conf.h
+index fc67957cfe..720c07ef74 100644
+--- a/src/conf/storage_conf.h
++++ b/src/conf/storage_conf.h
+@@ -103,6 +103,7 @@ typedef enum {
+     VIR_STORAGE_POOL_GLUSTER,  /* Gluster device */
+     VIR_STORAGE_POOL_ZFS,      /* ZFS */
+     VIR_STORAGE_POOL_VSTORAGE, /* Virtuozzo Storage */
++    VIR_STORAGE_POOL_VITASTOR, /* Vitastor */
+ 
+     VIR_STORAGE_POOL_LAST,
+ } virStoragePoolType;
+@@ -454,6 +455,7 @@ VIR_ENUM_DECL(virStoragePartedFs);
+                  VIR_CONNECT_LIST_STORAGE_POOLS_SCSI     | \
+                  VIR_CONNECT_LIST_STORAGE_POOLS_MPATH    | \
+                  VIR_CONNECT_LIST_STORAGE_POOLS_RBD      | \
++                 VIR_CONNECT_LIST_STORAGE_POOLS_VITASTOR | \
+                  VIR_CONNECT_LIST_STORAGE_POOLS_SHEEPDOG | \
+                  VIR_CONNECT_LIST_STORAGE_POOLS_GLUSTER  | \
+                  VIR_CONNECT_LIST_STORAGE_POOLS_ZFS      | \
+diff --git a/src/conf/storage_source_conf.c b/src/conf/storage_source_conf.c
+index d7b9bdfecb..38aefd0dd4 100644
+--- a/src/conf/storage_source_conf.c
++++ b/src/conf/storage_source_conf.c
+@@ -90,6 +90,7 @@ VIR_ENUM_IMPL(virStorageNetProtocol,
+               "ssh",
+               "vxhs",
+               "nfs",
++              "vitastor",
+ );
+ 
+ 
+@@ -1317,6 +1318,7 @@ virStorageSourceNetworkDefaultPort(virStorageNetProtocol protocol)
+         case VIR_STORAGE_NET_PROTOCOL_GLUSTER:
+             return 24007;
+ 
++        case VIR_STORAGE_NET_PROTOCOL_VITASTOR:
+         case VIR_STORAGE_NET_PROTOCOL_RBD:
+             /* we don't provide a default for RBD */
+             return 0;
+diff --git a/src/conf/storage_source_conf.h b/src/conf/storage_source_conf.h
+index 22c35d420d..f1e32ea83d 100644
+--- a/src/conf/storage_source_conf.h
++++ b/src/conf/storage_source_conf.h
+@@ -131,6 +131,7 @@ typedef enum {
+     VIR_STORAGE_NET_PROTOCOL_SSH,
+     VIR_STORAGE_NET_PROTOCOL_VXHS,
+     VIR_STORAGE_NET_PROTOCOL_NFS,
++    VIR_STORAGE_NET_PROTOCOL_VITASTOR,
+ 
+     VIR_STORAGE_NET_PROTOCOL_LAST
+ } virStorageNetProtocol;
+diff --git a/src/conf/virstorageobj.c b/src/conf/virstorageobj.c
+index 59fa5da372..4739167f5f 100644
+--- a/src/conf/virstorageobj.c
++++ b/src/conf/virstorageobj.c
+@@ -1438,6 +1438,7 @@ virStoragePoolObjSourceFindDuplicateCb(const void *payload,
+             return 1;
+         break;
+ 
++    case VIR_STORAGE_POOL_VITASTOR:
+     case VIR_STORAGE_POOL_ISCSI_DIRECT:
+     case VIR_STORAGE_POOL_RBD:
+     case VIR_STORAGE_POOL_LAST:
+@@ -1921,6 +1922,8 @@ virStoragePoolObjMatch(virStoragePoolObj *obj,
+                (obj->def->type == VIR_STORAGE_POOL_MPATH))   ||
+               (MATCH(VIR_CONNECT_LIST_STORAGE_POOLS_RBD) &&
+                (obj->def->type == VIR_STORAGE_POOL_RBD))     ||
++              (MATCH(VIR_CONNECT_LIST_STORAGE_POOLS_VITASTOR) &&
++               (obj->def->type == VIR_STORAGE_POOL_VITASTOR)) ||
+               (MATCH(VIR_CONNECT_LIST_STORAGE_POOLS_SHEEPDOG) &&
+                (obj->def->type == VIR_STORAGE_POOL_SHEEPDOG)) ||
+               (MATCH(VIR_CONNECT_LIST_STORAGE_POOLS_GLUSTER) &&
+diff --git a/src/libvirt-storage.c b/src/libvirt-storage.c
+index db7660aac4..561df34709 100644
+--- a/src/libvirt-storage.c
++++ b/src/libvirt-storage.c
+@@ -94,6 +94,7 @@ virStoragePoolGetConnect(virStoragePoolPtr pool)
+  * VIR_CONNECT_LIST_STORAGE_POOLS_SCSI
+  * VIR_CONNECT_LIST_STORAGE_POOLS_MPATH
+  * VIR_CONNECT_LIST_STORAGE_POOLS_RBD
++ * VIR_CONNECT_LIST_STORAGE_POOLS_VITASTOR
+  * VIR_CONNECT_LIST_STORAGE_POOLS_SHEEPDOG
+  * VIR_CONNECT_LIST_STORAGE_POOLS_GLUSTER
+  * VIR_CONNECT_LIST_STORAGE_POOLS_ZFS
+diff --git a/src/libxl/libxl_conf.c b/src/libxl/libxl_conf.c
+index 2b988157fa..9d0eb47b25 100644
+--- a/src/libxl/libxl_conf.c
++++ b/src/libxl/libxl_conf.c
+@@ -1069,6 +1069,7 @@ libxlMakeNetworkDiskSrcStr(virStorageSource *src,
+     case VIR_STORAGE_NET_PROTOCOL_SSH:
+     case VIR_STORAGE_NET_PROTOCOL_VXHS:
+     case VIR_STORAGE_NET_PROTOCOL_NFS:
++    case VIR_STORAGE_NET_PROTOCOL_VITASTOR:
+     case VIR_STORAGE_NET_PROTOCOL_LAST:
+     case VIR_STORAGE_NET_PROTOCOL_NONE:
+         virReportError(VIR_ERR_NO_SUPPORT,
+diff --git a/src/libxl/xen_xl.c b/src/libxl/xen_xl.c
+index e72e7d7f44..8482c21805 100644
+--- a/src/libxl/xen_xl.c
++++ b/src/libxl/xen_xl.c
+@@ -1461,6 +1461,7 @@ xenFormatXLDiskSrcNet(virStorageSource *src)
+     case VIR_STORAGE_NET_PROTOCOL_SSH:
+     case VIR_STORAGE_NET_PROTOCOL_VXHS:
+     case VIR_STORAGE_NET_PROTOCOL_NFS:
++    case VIR_STORAGE_NET_PROTOCOL_VITASTOR:
+     case VIR_STORAGE_NET_PROTOCOL_LAST:
+     case VIR_STORAGE_NET_PROTOCOL_NONE:
+         virReportError(VIR_ERR_NO_SUPPORT,
+diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c
+index 9b43279797..459d8e8a65 100644
+--- a/src/qemu/qemu_block.c
++++ b/src/qemu/qemu_block.c
+@@ -743,6 +743,38 @@ qemuBlockStorageSourceGetRBDProps(virStorageSource *src,
+ }
+ 
+ 
++static virJSONValue *
++qemuBlockStorageSourceGetVitastorProps(virStorageSource *src)
++{
++    virJSONValue *ret = NULL;
++    virStorageNetHostDef *host;
++    size_t i;
++    g_auto(virBuffer) buf = VIR_BUFFER_INITIALIZER;
++    g_autofree char *etcd = NULL;
++
++    for (i = 0; i < src->nhosts; i++) {
++        host = src->hosts + i;
++        if ((virStorageNetHostTransport)host->transport != VIR_STORAGE_NET_HOST_TRANS_TCP) {
++            return NULL;
++        }
++        virBufferAsprintf(&buf, i > 0 ? ",%s:%u" : "%s:%u", host->name, host->port);
++    }
++    if (src->nhosts > 0) {
++        etcd = virBufferContentAndReset(&buf);
++    }
++
++    if (virJSONValueObjectAdd(&ret,
++                              "S:etcd-host", etcd,
++                              "S:etcd-prefix", src->query,
++                              "S:config-path", src->configFile,
++                              "s:image", src->path,
++                              NULL) < 0)
++        return NULL;
++
++    return ret;
++}
++
++
+ static virJSONValue *
+ qemuBlockStorageSourceGetSshProps(virStorageSource *src)
+ {
+@@ -1094,6 +1126,12 @@ qemuBlockStorageSourceGetBackendProps(virStorageSource *src,
+                 return NULL;
+             break;
+ 
++        case VIR_STORAGE_NET_PROTOCOL_VITASTOR:
++            driver = "vitastor";
++            if (!(fileprops = qemuBlockStorageSourceGetVitastorProps(src)))
++                return NULL;
++            break;
++
+         case VIR_STORAGE_NET_PROTOCOL_SSH:
+             driver = "ssh";
+             if (!(fileprops = qemuBlockStorageSourceGetSshProps(src)))
+@@ -1997,6 +2035,7 @@ qemuBlockGetBackingStoreString(virStorageSource *src,
+ 
+             case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG:
+             case VIR_STORAGE_NET_PROTOCOL_RBD:
++            case VIR_STORAGE_NET_PROTOCOL_VITASTOR:
+             case VIR_STORAGE_NET_PROTOCOL_VXHS:
+             case VIR_STORAGE_NET_PROTOCOL_NFS:
+             case VIR_STORAGE_NET_PROTOCOL_SSH:
+@@ -2377,6 +2416,12 @@ qemuBlockStorageSourceCreateGetStorageProps(virStorageSource *src,
+                 return -1;
+             break;
+ 
++        case VIR_STORAGE_NET_PROTOCOL_VITASTOR:
++            driver = "vitastor";
++            if (!(location = qemuBlockStorageSourceGetVitastorProps(src)))
++                return -1;
++            break;
++
+         case VIR_STORAGE_NET_PROTOCOL_SSH:
+             if (srcPriv->nbdkitProcess) {
+                 /* disk creation not yet supported with nbdkit, and even if it
+diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
+index ac56fc7cb4..9e407b4aab 100644
+--- a/src/qemu/qemu_domain.c
++++ b/src/qemu/qemu_domain.c
+@@ -4677,7 +4677,8 @@ qemuDomainValidateStorageSource(virStorageSource *src,
+     if (src->query &&
+         (actualType != VIR_STORAGE_TYPE_NETWORK ||
+          (src->protocol != VIR_STORAGE_NET_PROTOCOL_HTTPS &&
+-          src->protocol != VIR_STORAGE_NET_PROTOCOL_HTTP))) {
++          src->protocol != VIR_STORAGE_NET_PROTOCOL_HTTP &&
++          src->protocol != VIR_STORAGE_NET_PROTOCOL_VITASTOR))) {
+         virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
+                        _("query is supported only with HTTP(S) protocols"));
+         return -1;
+@@ -9103,6 +9104,7 @@ qemuDomainPrepareStorageSourceTLS(virStorageSource *src,
+         break;
+ 
+     case VIR_STORAGE_NET_PROTOCOL_RBD:
++    case VIR_STORAGE_NET_PROTOCOL_VITASTOR:
+     case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG:
+     case VIR_STORAGE_NET_PROTOCOL_GLUSTER:
+     case VIR_STORAGE_NET_PROTOCOL_ISCSI:
+diff --git a/src/qemu/qemu_snapshot.c b/src/qemu/qemu_snapshot.c
+index e738afffc3..37d64f469b 100644
+--- a/src/qemu/qemu_snapshot.c
++++ b/src/qemu/qemu_snapshot.c
+@@ -665,6 +665,7 @@ qemuSnapshotPrepareDiskExternalInactive(virDomainSnapshotDiskDef *snapdisk,
+         case VIR_STORAGE_NET_PROTOCOL_NONE:
+         case VIR_STORAGE_NET_PROTOCOL_NBD:
+         case VIR_STORAGE_NET_PROTOCOL_RBD:
++        case VIR_STORAGE_NET_PROTOCOL_VITASTOR:
+         case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG:
+         case VIR_STORAGE_NET_PROTOCOL_GLUSTER:
+         case VIR_STORAGE_NET_PROTOCOL_ISCSI:
+@@ -893,6 +894,7 @@ qemuSnapshotPrepareDiskInternal(virDomainDiskDef *disk,
+         case VIR_STORAGE_NET_PROTOCOL_NONE:
+         case VIR_STORAGE_NET_PROTOCOL_NBD:
+         case VIR_STORAGE_NET_PROTOCOL_RBD:
++        case VIR_STORAGE_NET_PROTOCOL_VITASTOR:
+         case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG:
+         case VIR_STORAGE_NET_PROTOCOL_GLUSTER:
+         case VIR_STORAGE_NET_PROTOCOL_ISCSI:
+diff --git a/src/storage/storage_driver.c b/src/storage/storage_driver.c
+index e19e032427..59f91f4710 100644
+--- a/src/storage/storage_driver.c
++++ b/src/storage/storage_driver.c
+@@ -1626,6 +1626,7 @@ storageVolLookupByPathCallback(virStoragePoolObj *obj,
+ 
+         case VIR_STORAGE_POOL_GLUSTER:
+         case VIR_STORAGE_POOL_RBD:
++        case VIR_STORAGE_POOL_VITASTOR:
+         case VIR_STORAGE_POOL_SHEEPDOG:
+         case VIR_STORAGE_POOL_ZFS:
+         case VIR_STORAGE_POOL_LAST:
+diff --git a/src/storage_file/storage_source_backingstore.c b/src/storage_file/storage_source_backingstore.c
+index 821378883c..2211f6891b 100644
+--- a/src/storage_file/storage_source_backingstore.c
++++ b/src/storage_file/storage_source_backingstore.c
+@@ -264,6 +264,75 @@ virStorageSourceParseRBDColonString(const char *rbdstr,
+ }
+ 
+ 
++static int
++virStorageSourceParseVitastorColonString(const char *colonstr,
++                                         virStorageSource *src)
++{
++    char *p, *e, *next;
++    g_autofree char *options = NULL;
++
++    /* optionally skip the "vitastor:" prefix if provided */
++    if (STRPREFIX(colonstr, "vitastor:"))
++        colonstr += strlen("vitastor:");
++
++    options = g_strdup(colonstr);
++
++    p = options;
++    while (*p) {
++        /* find : delimiter or end of string */
++        for (e = p; *e && *e != ':'; ++e) {
++            if (*e == '\\') {
++                e++;
++                if (*e == '\0')
++                    break;
++            }
++        }
++        if (*e == '\0') {
++            next = e;    /* last kv pair */
++        } else {
++            next = e + 1;
++            *e = '\0';
++        }
++
++        if (STRPREFIX(p, "image=")) {
++            src->path = g_strdup(p + strlen("image="));
++        } else if (STRPREFIX(p, "etcd-prefix=")) {
++            src->query = g_strdup(p + strlen("etcd-prefix="));
++        } else if (STRPREFIX(p, "config-path=")) {
++            src->configFile = g_strdup(p + strlen("config-path="));
++        } else if (STRPREFIX(p, "etcd-host=")) {
++            char *h, *sep;
++
++            h = p + strlen("etcd-host=");
++            while (h < e) {
++                for (sep = h; sep < e; ++sep) {
++                    if (*sep == '\\' && (sep[1] == ',' ||
++                                         sep[1] == ';' ||
++                                         sep[1] == ' ')) {
++                        *sep = '\0';
++                        sep += 2;
++                        break;
++                    }
++                }
++
++                if (virStorageSourceRBDAddHost(src, h) < 0)
++                    return -1;
++
++                h = sep;
++            }
++        }
++
++        p = next;
++    }
++
++    if (!src->path) {
++        return -1;
++    }
++
++    return 0;
++}
++
++
+ static int
+ virStorageSourceParseNBDColonString(const char *nbdstr,
+                                     virStorageSource *src)
+@@ -379,6 +448,11 @@ virStorageSourceParseBackingColon(virStorageSource *src,
+             return -1;
+         break;
+ 
++    case VIR_STORAGE_NET_PROTOCOL_VITASTOR:
++        if (virStorageSourceParseVitastorColonString(path, src) < 0)
++            return -1;
++        break;
++
+     case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG:
+     case VIR_STORAGE_NET_PROTOCOL_LAST:
+     case VIR_STORAGE_NET_PROTOCOL_NONE:
+@@ -953,6 +1027,54 @@ virStorageSourceParseBackingJSONRBD(virStorageSource *src,
+     return 0;
+ }
+ 
++static int
++virStorageSourceParseBackingJSONVitastor(virStorageSource *src,
++                                         virJSONValue *json,
++                                         const char *jsonstr G_GNUC_UNUSED,
++                                         int opaque G_GNUC_UNUSED)
++{
++    const char *filename;
++    const char *image = virJSONValueObjectGetString(json, "image");
++    const char *conf = virJSONValueObjectGetString(json, "config-path");
++    const char *etcd_prefix = virJSONValueObjectGetString(json, "etcd-prefix");
++    virJSONValue *servers = virJSONValueObjectGetArray(json, "server");
++    size_t nservers;
++    size_t i;
++
++    src->type = VIR_STORAGE_TYPE_NETWORK;
++    src->protocol = VIR_STORAGE_NET_PROTOCOL_VITASTOR;
++
++    /* legacy syntax passed via 'filename' option */
++    if ((filename = virJSONValueObjectGetString(json, "filename")))
++        return virStorageSourceParseVitastorColonString(filename, src);
++
++    if (!image) {
++        virReportError(VIR_ERR_INVALID_ARG, "%s",
++                       _("missing image name in Vitastor backing volume "
++                         "JSON specification"));
++        return -1;
++    }
++
++    src->path = g_strdup(image);
++    src->configFile = g_strdup(conf);
++    src->query = g_strdup(etcd_prefix);
++
++    if (servers) {
++        nservers = virJSONValueArraySize(servers);
++
++        src->hosts = g_new0(virStorageNetHostDef, nservers);
++        src->nhosts = nservers;
++
++        for (i = 0; i < nservers; i++) {
++            if (virStorageSourceParseBackingJSONInetSocketAddress(src->hosts + i,
++                                                                  virJSONValueArrayGet(servers, i)) < 0)
++                return -1;
++        }
++    }
++
++    return 0;
++}
++
+ static int
+ virStorageSourceParseBackingJSONRaw(virStorageSource *src,
+                                     virJSONValue *json,
+@@ -1130,6 +1252,7 @@ static const struct virStorageSourceJSONDriverParser jsonParsers[] = {
+     {"sheepdog", false, virStorageSourceParseBackingJSONSheepdog, 0},
+     {"ssh", false, virStorageSourceParseBackingJSONSSH, 0},
+     {"rbd", false, virStorageSourceParseBackingJSONRBD, 0},
++    {"vitastor", false, virStorageSourceParseBackingJSONVitastor, 0},
+     {"raw", true, virStorageSourceParseBackingJSONRaw, 0},
+     {"nfs", false, virStorageSourceParseBackingJSONNFS, 0},
+     {"vxhs", false, virStorageSourceParseBackingJSONVxHS, 0},
+diff --git a/src/test/test_driver.c b/src/test/test_driver.c
+index 1165689de7..bba846351c 100644
+--- a/src/test/test_driver.c
++++ b/src/test/test_driver.c
+@@ -7345,6 +7345,7 @@ testStorageVolumeTypeForPool(int pooltype)
+     case VIR_STORAGE_POOL_ISCSI_DIRECT:
+     case VIR_STORAGE_POOL_GLUSTER:
+     case VIR_STORAGE_POOL_RBD:
++    case VIR_STORAGE_POOL_VITASTOR:
+         return VIR_STORAGE_VOL_NETWORK;
+     case VIR_STORAGE_POOL_LOGICAL:
+     case VIR_STORAGE_POOL_DISK:
+diff --git a/tests/storagepoolcapsschemadata/poolcaps-fs.xml b/tests/storagepoolcapsschemadata/poolcaps-fs.xml
+index eee75af746..8bd0a57bdd 100644
+--- a/tests/storagepoolcapsschemadata/poolcaps-fs.xml
++++ b/tests/storagepoolcapsschemadata/poolcaps-fs.xml
+@@ -204,4 +204,11 @@
+       </enum>
+     </volOptions>
+   </pool>
++  <pool type='vitastor' supported='no'>
++    <volOptions>
++      <defaultFormat type='raw'/>
++      <enum name='targetFormatType'>
++      </enum>
++    </volOptions>
++  </pool>
+ </storagepoolCapabilities>
+diff --git a/tests/storagepoolcapsschemadata/poolcaps-full.xml b/tests/storagepoolcapsschemadata/poolcaps-full.xml
+index 805950a937..852df0de16 100644
+--- a/tests/storagepoolcapsschemadata/poolcaps-full.xml
++++ b/tests/storagepoolcapsschemadata/poolcaps-full.xml
+@@ -204,4 +204,11 @@
+       </enum>
+     </volOptions>
+   </pool>
++  <pool type='vitastor' supported='yes'>
++    <volOptions>
++      <defaultFormat type='raw'/>
++      <enum name='targetFormatType'>
++      </enum>
++    </volOptions>
++  </pool>
+ </storagepoolCapabilities>
+diff --git a/tests/storagepoolxml2argvtest.c b/tests/storagepoolxml2argvtest.c
+index d5c2531ab8..b19308ac38 100644
+--- a/tests/storagepoolxml2argvtest.c
++++ b/tests/storagepoolxml2argvtest.c
+@@ -57,6 +57,7 @@ testCompareXMLToArgvFiles(bool shouldFail,
+     case VIR_STORAGE_POOL_GLUSTER:
+     case VIR_STORAGE_POOL_ZFS:
+     case VIR_STORAGE_POOL_VSTORAGE:
++    case VIR_STORAGE_POOL_VITASTOR:
+     case VIR_STORAGE_POOL_LAST:
+     default:
+         VIR_TEST_DEBUG("pool type '%s' has no xml2argv test", defTypeStr);
+diff --git a/tools/virsh-pool.c b/tools/virsh-pool.c
+index 2010ef1356..072e2ff9e8 100644
+--- a/tools/virsh-pool.c
++++ b/tools/virsh-pool.c
+@@ -1187,6 +1187,9 @@ cmdPoolList(vshControl *ctl, const vshCmd *cmd G_GNUC_UNUSED)
+             case VIR_STORAGE_POOL_VSTORAGE:
+                 flags |= VIR_CONNECT_LIST_STORAGE_POOLS_VSTORAGE;
+                 break;
++            case VIR_STORAGE_POOL_VITASTOR:
++                flags |= VIR_CONNECT_LIST_STORAGE_POOLS_VITASTOR;
++                break;
+             case VIR_STORAGE_POOL_LAST:
+                 break;
+             }

patches/qemu-10.1-vitastor.patch

@@ -1,29 +1,172 @@
-diff --git a/src/client/qemu_driver.c b/src/client/qemu_driver.c
-index d8356dab..5f4cd50d 100644
---- a/src/client/qemu_driver.c
-+++ b/src/client/qemu_driver.c
-@@ -974,14 +974,21 @@ static void vitastor_co_read_bitmap_cb(void *opaque, long retval, uint8_t *bitma
- #endif
- }
+diff --git a/block/meson.build b/block/meson.build
+index 34b1b2a306..24ca0f1e52 100644
+--- a/block/meson.build
++++ b/block/meson.build
+@@ -114,6 +114,7 @@ foreach m : [
+   [libnfs, 'nfs', files('nfs.c')],
+   [libssh, 'ssh', files('ssh.c')],
+   [rbd, 'rbd', files('rbd.c')],
++  [vitastor, 'vitastor', files('vitastor.c')],
+ ]
+   if m[0].found()
+     module_ss = ss.source_set()
+diff --git a/meson.build b/meson.build
+index 50c774a195..e5c7a3a4b1 100644
+--- a/meson.build
++++ b/meson.build
+@@ -1652,6 +1652,26 @@ if not get_option('rbd').auto() or have_block
+   endif
+ endif
  
--static int coroutine_fn vitastor_co_block_status(
--    BlockDriverState *bs, bool want_zero, int64_t offset, int64_t bytes,
--    int64_t *pnum, int64_t *map, BlockDriverState **file)
-+static int coroutine_fn vitastor_co_block_status(BlockDriverState *bs,
-+#if QEMU_VERSION_MAJOR > 10 || QEMU_VERSION_MAJOR == 10 && QEMU_VERSION_MINOR >= 1
-+    unsigned int mode,
-+#else
-+    bool want_zero,
-+#endif
-+    int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file)
- {
-     // Allocated => return BDRV_BLOCK_DATA|BDRV_BLOCK_OFFSET_VALID
-     // Not allocated => return 0
-     // Error => return -errno
-     // Set pnum to length of the extent, `*map` = `offset`, `*file` = `bs`
-+#if QEMU_VERSION_MAJOR > 10 || QEMU_VERSION_MAJOR == 10 && QEMU_VERSION_MINOR >= 1
-+    int want_zero = (mode == BDRV_WANT_PRECISE);
-+#endif
-     VitastorRPC task;
-     VitastorClient *client = bs->opaque;
-     uint64_t inode = client->watch ? vitastor_c_inode_get_num(client->watch) : client->inode;
++vitastor = not_found
++if not get_option('vitastor').auto() or have_block
++  libvitastor_client = cc.find_library('vitastor_client', has_headers: ['vitastor_c.h'],
++    required: get_option('vitastor'))
++  if libvitastor_client.found()
++    if cc.links('''
++      #include <vitastor_c.h>
++      int main(void) {
++        vitastor_c_create_qemu(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
++        return 0;
++      }''', dependencies: libvitastor_client)
++      vitastor = declare_dependency(dependencies: libvitastor_client)
++    elif get_option('vitastor').enabled()
++      error('could not link libvitastor_client')
++    else
++      warning('could not link libvitastor_client, disabling')
++    endif
++  endif
++endif
++
+ glusterfs = not_found
+ glusterfs_ftruncate_has_stat = false
+ glusterfs_iocb_has_stat = false
+@@ -2547,6 +2567,7 @@ endif
+ config_host_data.set('CONFIG_OPENGL', opengl.found())
+ config_host_data.set('CONFIG_PLUGIN', get_option('plugins'))
+ config_host_data.set('CONFIG_RBD', rbd.found())
++config_host_data.set('CONFIG_VITASTOR', vitastor.found())
+ config_host_data.set('CONFIG_RDMA', rdma.found())
+ config_host_data.set('CONFIG_RELOCATABLE', get_option('relocatable'))
+ config_host_data.set('CONFIG_SAFESTACK', get_option('safe_stack'))
+@@ -4972,6 +4993,7 @@ summary_info += {'fdt support':       fdt_opt == 'internal' ? 'internal' : fdt}
+ summary_info += {'libcap-ng support': libcap_ng}
+ summary_info += {'bpf support':       libbpf}
+ summary_info += {'rbd support':       rbd}
++summary_info += {'vitastor support':  vitastor}
+ summary_info += {'smartcard support': cacard}
+ summary_info += {'U2F support':       u2f}
+ summary_info += {'libusb':            libusb}
+diff --git a/meson_options.txt b/meson_options.txt
+index fff1521e58..f0844c0e00 100644
+--- a/meson_options.txt
++++ b/meson_options.txt
+@@ -202,6 +202,8 @@ option('pvg', type: 'feature', value: 'auto',
+        description: 'macOS paravirtualized graphics support')
+ option('rbd', type : 'feature', value : 'auto',
+        description: 'Ceph block device driver')
++option('vitastor', type : 'feature', value : 'auto',
++       description: 'Vitastor block device driver')
+ option('opengl', type : 'feature', value : 'auto',
+        description: 'OpenGL support')
+ option('rdma', type : 'feature', value : 'auto',
+diff --git a/qapi/block-core.json b/qapi/block-core.json
+index dc6eb4ae23..d043f4340e 100644
+--- a/qapi/block-core.json
++++ b/qapi/block-core.json
+@@ -3280,7 +3280,7 @@
+             'parallels', 'preallocate', 'qcow', 'qcow2', 'qed', 'quorum',
+             'raw', 'rbd',
+             { 'name': 'replication', 'if': 'CONFIG_REPLICATION' },
+-            'ssh', 'throttle', 'vdi', 'vhdx',
++            'ssh', 'throttle', 'vdi', 'vhdx', 'vitastor',
+             { 'name': 'virtio-blk-vfio-pci', 'if': 'CONFIG_BLKIO' },
+             { 'name': 'virtio-blk-vhost-user', 'if': 'CONFIG_BLKIO' },
+             { 'name': 'virtio-blk-vhost-vdpa', 'if': 'CONFIG_BLKIO' },
+@@ -4363,6 +4363,28 @@
+             '*key-secret': 'str',
+             '*server': ['InetSocketAddressBase'] } }
+ 
++##
++# @BlockdevOptionsVitastor:
++#
++# Driver specific block device options for vitastor
++#
++# @image:       Image name
++# @inode:       Inode number
++# @pool:        Pool ID
++# @size:        Desired image size in bytes
++# @config-path: Path to Vitastor configuration
++# @etcd-host:   etcd connection address(es)
++# @etcd-prefix: etcd key/value prefix
++##
++{ 'struct': 'BlockdevOptionsVitastor',
++  'data': { '*inode': 'uint64',
++            '*pool': 'uint64',
++            '*size': 'uint64',
++            '*image': 'str',
++            '*config-path': 'str',
++            '*etcd-host': 'str',
++            '*etcd-prefix': 'str' } }
++
+ ##
+ # @ReplicationMode:
+ #
+@@ -4831,6 +4853,7 @@
+       'throttle':   'BlockdevOptionsThrottle',
+       'vdi':        'BlockdevOptionsGenericFormat',
+       'vhdx':       'BlockdevOptionsGenericFormat',
++      'vitastor':   'BlockdevOptionsVitastor',
+       'virtio-blk-vfio-pci':
+                     { 'type': 'BlockdevOptionsVirtioBlkVfioPci',
+                       'if': 'CONFIG_BLKIO' },
+@@ -5304,6 +5327,20 @@
+             '*cluster-size' :   'size',
+             '*encrypt' :        'RbdEncryptionCreateOptions' } }
+ 
++##
++# @BlockdevCreateOptionsVitastor:
++#
++# Driver specific image creation options for Vitastor.
++#
++# @location: Where to store the new image file.  This location cannot
++#     point to a snapshot.
++#
++# @size: Size of the virtual disk in bytes
++##
++{ 'struct': 'BlockdevCreateOptionsVitastor',
++  'data': { 'location':         'BlockdevOptionsVitastor',
++            'size':             'size' } }
++
+ ##
+ # @BlockdevVmdkSubformat:
+ #
+@@ -5526,6 +5563,7 @@
+       'ssh':            'BlockdevCreateOptionsSsh',
+       'vdi':            'BlockdevCreateOptionsVdi',
+       'vhdx':           'BlockdevCreateOptionsVhdx',
++      'vitastor':       'BlockdevCreateOptionsVitastor',
+       'vmdk':           'BlockdevCreateOptionsVmdk',
+       'vpc':            'BlockdevCreateOptionsVpc'
+   } }
+diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
+index 0ebe6bc52a..2c37ad3892 100644
+--- a/scripts/meson-buildoptions.sh
++++ b/scripts/meson-buildoptions.sh
+@@ -175,6 +175,7 @@ meson_options_help() {
+   printf "%s\n" '  qga-vss         build QGA VSS support (broken with MinGW)'
+   printf "%s\n" '  qpl             Query Processing Library support'
+   printf "%s\n" '  rbd             Ceph block device driver'
++  printf "%s\n" '  vitastor        Vitastor block device driver'
+   printf "%s\n" '  rdma            Enable RDMA-based migration'
+   printf "%s\n" '  replication     replication support'
+   printf "%s\n" '  rust            Rust support'
+@@ -459,6 +460,8 @@ _meson_option_parse() {
+     --disable-qpl) printf "%s" -Dqpl=disabled ;;
+     --enable-rbd) printf "%s" -Drbd=enabled ;;
+     --disable-rbd) printf "%s" -Drbd=disabled ;;
++    --enable-vitastor) printf "%s" -Dvitastor=enabled ;;
++    --disable-vitastor) printf "%s" -Dvitastor=disabled ;;
+     --enable-rdma) printf "%s" -Drdma=enabled ;;
+     --disable-rdma) printf "%s" -Drdma=disabled ;;
+     --enable-relocatable) printf "%s" -Drelocatable=true ;;

patches/qemu-10.2-vitastor.patch

@@ -0,0 +1,172 @@
+diff --git a/block/meson.build b/block/meson.build
+index 34b1b2a306..24ca0f1e52 100644
+--- a/block/meson.build
++++ b/block/meson.build
+@@ -114,6 +114,7 @@ foreach m : [
+   [libnfs, 'nfs', files('nfs.c')],
+   [libssh, 'ssh', files('ssh.c')],
+   [rbd, 'rbd', files('rbd.c')],
++  [vitastor, 'vitastor', files('vitastor.c')],
+ ]
+   if m[0].found()
+     module_ss = ss.source_set()
+diff --git a/meson.build b/meson.build
+index d9293294d8..776a5becc6 100644
+--- a/meson.build
++++ b/meson.build
+@@ -1665,6 +1665,26 @@ if not get_option('rbd').auto() or have_block
+   endif
+ endif
+ 
++vitastor = not_found
++if not get_option('vitastor').auto() or have_block
++  libvitastor_client = cc.find_library('vitastor_client', has_headers: ['vitastor_c.h'],
++    required: get_option('vitastor'))
++  if libvitastor_client.found()
++    if cc.links('''
++      #include <vitastor_c.h>
++      int main(void) {
++        vitastor_c_create_qemu(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
++        return 0;
++      }''', dependencies: libvitastor_client)
++      vitastor = declare_dependency(dependencies: libvitastor_client)
++    elif get_option('vitastor').enabled()
++      error('could not link libvitastor_client')
++    else
++      warning('could not link libvitastor_client, disabling')
++    endif
++  endif
++endif
++
+ glusterfs = not_found
+ glusterfs_ftruncate_has_stat = false
+ glusterfs_iocb_has_stat = false
+@@ -2509,6 +2529,7 @@ endif
+ config_host_data.set('CONFIG_OPENGL', opengl.found())
+ config_host_data.set('CONFIG_PLUGIN', get_option('plugins'))
+ config_host_data.set('CONFIG_RBD', rbd.found())
++config_host_data.set('CONFIG_VITASTOR', vitastor.found())
+ config_host_data.set('CONFIG_RDMA', rdma.found())
+ config_host_data.set('CONFIG_RELOCATABLE', get_option('relocatable'))
+ config_host_data.set('CONFIG_SAFESTACK', get_option('safe_stack'))
+@@ -4948,6 +4969,7 @@ summary_info += {'fdt support':       fdt_opt == 'internal' ? 'internal' : fdt}
+ summary_info += {'libcap-ng support': libcap_ng}
+ summary_info += {'bpf support':       libbpf}
+ summary_info += {'rbd support':       rbd}
++summary_info += {'vitastor support':  vitastor}
+ summary_info += {'smartcard support': cacard}
+ summary_info += {'U2F support':       u2f}
+ summary_info += {'libusb':            libusb}
+diff --git a/meson_options.txt b/meson_options.txt
+index 2836156257..148086cc6f 100644
+--- a/meson_options.txt
++++ b/meson_options.txt
+@@ -206,6 +206,8 @@ option('pvg', type: 'feature', value: 'auto',
+        description: 'macOS paravirtualized graphics support')
+ option('rbd', type : 'feature', value : 'auto',
+        description: 'Ceph block device driver')
++option('vitastor', type : 'feature', value : 'auto',
++       description: 'Vitastor block device driver')
+ option('opengl', type : 'feature', value : 'auto',
+        description: 'OpenGL support')
+ option('rdma', type : 'feature', value : 'auto',
+diff --git a/qapi/block-core.json b/qapi/block-core.json
+index b82af74256..f25a6f5ce8 100644
+--- a/qapi/block-core.json
++++ b/qapi/block-core.json
+@@ -3351,7 +3351,7 @@
+             'parallels', 'preallocate', 'qcow', 'qcow2', 'qed', 'quorum',
+             'raw', 'rbd',
+             { 'name': 'replication', 'if': 'CONFIG_REPLICATION' },
+-            'ssh', 'throttle', 'vdi', 'vhdx',
++            'ssh', 'throttle', 'vdi', 'vhdx', 'vitastor',
+             { 'name': 'virtio-blk-vfio-pci', 'if': 'CONFIG_BLKIO' },
+             { 'name': 'virtio-blk-vhost-user', 'if': 'CONFIG_BLKIO' },
+             { 'name': 'virtio-blk-vhost-vdpa', 'if': 'CONFIG_BLKIO' },
+@@ -4434,6 +4434,28 @@
+             '*key-secret': 'str',
+             '*server': ['InetSocketAddressBase'] } }
+ 
++##
++# @BlockdevOptionsVitastor:
++#
++# Driver specific block device options for vitastor
++#
++# @image:       Image name
++# @inode:       Inode number
++# @pool:        Pool ID
++# @size:        Desired image size in bytes
++# @config-path: Path to Vitastor configuration
++# @etcd-host:   etcd connection address(es)
++# @etcd-prefix: etcd key/value prefix
++##
++{ 'struct': 'BlockdevOptionsVitastor',
++  'data': { '*inode': 'uint64',
++            '*pool': 'uint64',
++            '*size': 'uint64',
++            '*image': 'str',
++            '*config-path': 'str',
++            '*etcd-host': 'str',
++            '*etcd-prefix': 'str' } }
++
+ ##
+ # @ReplicationMode:
+ #
+@@ -4902,6 +4924,7 @@
+       'throttle':   'BlockdevOptionsThrottle',
+       'vdi':        'BlockdevOptionsGenericFormat',
+       'vhdx':       'BlockdevOptionsGenericFormat',
++      'vitastor':   'BlockdevOptionsVitastor',
+       'virtio-blk-vfio-pci':
+                     { 'type': 'BlockdevOptionsVirtioBlkVfioPci',
+                       'if': 'CONFIG_BLKIO' },
+@@ -5376,6 +5399,20 @@
+             '*cluster-size' :   'size',
+             '*encrypt' :        'RbdEncryptionCreateOptions' } }
+ 
++##
++# @BlockdevCreateOptionsVitastor:
++#
++# Driver specific image creation options for Vitastor.
++#
++# @location: Where to store the new image file.  This location cannot
++#     point to a snapshot.
++#
++# @size: Size of the virtual disk in bytes
++##
++{ 'struct': 'BlockdevCreateOptionsVitastor',
++  'data': { 'location':         'BlockdevOptionsVitastor',
++            'size':             'size' } }
++
+ ##
+ # @BlockdevVmdkSubformat:
+ #
+@@ -5598,6 +5635,7 @@
+       'ssh':            'BlockdevCreateOptionsSsh',
+       'vdi':            'BlockdevCreateOptionsVdi',
+       'vhdx':           'BlockdevCreateOptionsVhdx',
++      'vitastor':       'BlockdevCreateOptionsVitastor',
+       'vmdk':           'BlockdevCreateOptionsVmdk',
+       'vpc':            'BlockdevCreateOptionsVpc'
+   } }
+diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
+index 3d0d132344..65ee8c855e 100644
+--- a/scripts/meson-buildoptions.sh
++++ b/scripts/meson-buildoptions.sh
+@@ -177,6 +177,7 @@ meson_options_help() {
+   printf "%s\n" '  qga-vss         build QGA VSS support (broken with MinGW)'
+   printf "%s\n" '  qpl             Query Processing Library support'
+   printf "%s\n" '  rbd             Ceph block device driver'
++  printf "%s\n" '  vitastor        Vitastor block device driver'
+   printf "%s\n" '  rdma            Enable RDMA-based migration'
+   printf "%s\n" '  replication     replication support'
+   printf "%s\n" '  rust            Rust support'
+@@ -464,6 +465,8 @@ _meson_option_parse() {
+     --disable-qpl) printf "%s" -Dqpl=disabled ;;
+     --enable-rbd) printf "%s" -Drbd=enabled ;;
+     --disable-rbd) printf "%s" -Drbd=disabled ;;
++    --enable-vitastor) printf "%s" -Dvitastor=enabled ;;
++    --disable-vitastor) printf "%s" -Dvitastor=disabled ;;
+     --enable-rdma) printf "%s" -Drdma=enabled ;;
+     --disable-rdma) printf "%s" -Drdma=disabled ;;
+     --enable-relocatable) printf "%s" -Drelocatable=true ;;

rpm/vitastor-el10.Dockerfile

@@ -11,7 +11,7 @@ WORKDIR /root
 RUN sed -i 's/enabled=0/enabled=1/' /etc/yum.repos.d/*.repo
 RUN dnf -y install epel-release dnf-plugins-core
 RUN dnf -y install https://vitastor.io/rpms/centos/10/vitastor-release-1.0-1.el10.noarch.rpm
-RUN dnf -y install gcc-c++ gperftools-devel fio nodejs rpm-build jerasure-devel isa-l-devel gf-complete-devel rdma-core-devel cmake libnl3-devel c-ares-devel
+RUN dnf -y install gcc-c++ gperftools-devel fio nodejs rpm-build jerasure-devel isa-l-devel gf-complete-devel rdma-core-devel cmake libnl3-devel
 RUN dnf download --source fio
 RUN rpm --nomd5 -i fio*.src.rpm
 RUN cd ~/rpmbuild/SPECS && dnf builddep -y --spec fio.spec

rpm/vitastor-el10.spec

@@ -1,11 +1,11 @@
 Name:           vitastor
-Version:        3.0.5
+Version:        3.0.12
 Release:        1%{?dist}
 Summary:        Vitastor, a fast software-defined clustered block storage
 
 License:        Vitastor Network Public License 1.1
 URL:            https://vitastor.io/
-Source0:        vitastor-3.0.5.el10.tar.gz
+Source0:        vitastor-3.0.12.el10.tar.gz
 
 BuildRequires:  gperftools-devel
 BuildRequires:  gcc-c++
@@ -16,7 +16,6 @@ BuildRequires:  gf-complete-devel
 BuildRequires:  rdma-core-devel
 BuildRequires:  cmake
 BuildRequires:  libnl3-devel
-BuildRequires:  c-ares-devel
 Requires:       vitastor-osd = %{version}-%{release}
 Requires:       vitastor-mon = %{version}-%{release}
 Requires:       vitastor-client = %{version}-%{release}

rpm/vitastor-el7.Dockerfile

@@ -15,7 +15,7 @@ RUN yum -y --enablerepo=extras install centos-release-scl epel-release yum-utils
 RUN perl -i -pe 's!mirrorlist=!#mirrorlist=!s; s!#\s*baseurl=http://mirror.centos.org!baseurl=http://vault.centos.org!' /etc/yum.repos.d/CentOS-SCLo-scl*.repo
 RUN yum -y install https://vitastor.io/rpms/centos/7/vitastor-release-1.0-1.el7.noarch.rpm
 RUN yum -y install devtoolset-9-gcc-c++ devtoolset-9-libatomic-devel gcc make cmake gperftools-devel \
-    fio rh-nodejs12 jerasure-devel libisa-l-devel gf-complete-devel rdma-core-devel libnl3-devel c-ares-devel
+    fio rh-nodejs12 jerasure-devel libisa-l-devel gf-complete-devel rdma-core-devel libnl3-devel
 RUN yumdownloader --disablerepo=centos-sclo-rh --source fio
 RUN rpm --nomd5 -i fio*.src.rpm
 RUN rm -f /etc/yum.repos.d/CentOS-Media.repo

rpm/vitastor-el7.spec

@@ -1,11 +1,11 @@
 Name:           vitastor
-Version:        3.0.5
+Version:        3.0.12
 Release:        1%{?dist}
 Summary:        Vitastor, a fast software-defined clustered block storage
 
 License:        Vitastor Network Public License 1.1
 URL:            https://vitastor.io/
-Source0:        vitastor-3.0.5.el7.tar.gz
+Source0:        vitastor-3.0.12.el7.tar.gz
 
 BuildRequires:  gperftools-devel
 BuildRequires:  devtoolset-9-gcc-c++
@@ -17,7 +17,6 @@ BuildRequires:  gf-complete-devel
 BuildRequires:  rdma-core-devel
 BuildRequires:  cmake3
 BuildRequires:  libnl3-devel
-BuildRequires:  c-ares-devel
 Requires:       vitastor-osd = %{version}-%{release}
 Requires:       vitastor-mon = %{version}-%{release}
 Requires:       vitastor-client = %{version}-%{release}

rpm/vitastor-el8.Dockerfile

@@ -13,7 +13,7 @@ RUN dnf -y install centos-release-advanced-virtualization epel-release dnf-plugi
 RUN sed -i 's/^mirrorlist=/#mirrorlist=/; s!#baseurl=.*!baseurl=http://vault.centos.org/centos/8.4.2105/virt/$basearch/$avdir/!; s!^baseurl=.*Source/.*!baseurl=http://vault.centos.org/centos/8.4.2105/virt/Source/advanced-virtualization/!' /etc/yum.repos.d/CentOS-Advanced-Virtualization.repo
 RUN yum -y install https://vitastor.io/rpms/centos/8/vitastor-release-1.0-1.el8.noarch.rpm
 RUN dnf -y install gcc-toolset-9 gcc-toolset-9-gcc-c++ gperftools-devel \
-    fio nodejs rpm-build jerasure-devel libisa-l-devel gf-complete-devel libibverbs-devel libarchive cmake libnl3-devel c-ares-devel
+    fio nodejs rpm-build jerasure-devel libisa-l-devel gf-complete-devel libibverbs-devel libarchive cmake libnl3-devel
 RUN dnf download --source fio
 RUN rpm --nomd5 -i fio*.src.rpm
 RUN cd ~/rpmbuild/SPECS && dnf builddep -y --enablerepo=powertools --spec fio.spec

rpm/vitastor-el8.spec

@@ -1,11 +1,11 @@
 Name:           vitastor
-Version:        3.0.5
+Version:        3.0.12
 Release:        1%{?dist}
 Summary:        Vitastor, a fast software-defined clustered block storage
 
 License:        Vitastor Network Public License 1.1
 URL:            https://vitastor.io/
-Source0:        vitastor-3.0.5.el8.tar.gz
+Source0:        vitastor-3.0.12.el8.tar.gz
 
 BuildRequires:  gperftools-devel
 BuildRequires:  gcc-toolset-9-gcc-c++
@@ -16,7 +16,6 @@ BuildRequires:  gf-complete-devel
 BuildRequires:  rdma-core-devel
 BuildRequires:  cmake
 BuildRequires:  libnl3-devel
-BuildRequires:  c-ares-devel
 Requires:       vitastor-osd = %{version}-%{release}
 Requires:       vitastor-mon = %{version}-%{release}
 Requires:       vitastor-client = %{version}-%{release}

rpm/vitastor-el9.Dockerfile

@@ -10,7 +10,7 @@ WORKDIR /root
 RUN sed -i 's/enabled=0/enabled=1/' /etc/yum.repos.d/*.repo
 RUN dnf -y install epel-release dnf-plugins-core
 RUN dnf -y install https://vitastor.io/rpms/centos/9/vitastor-release-1.0-1.el9.noarch.rpm
-RUN dnf -y install gcc-c++ gperftools-devel fio nodejs rpm-build jerasure-devel libisa-l-devel gf-complete-devel rdma-core-devel libarchive cmake libnl3-devel c-ares-devel
+RUN dnf -y install gcc-c++ gperftools-devel fio nodejs rpm-build jerasure-devel libisa-l-devel gf-complete-devel rdma-core-devel libarchive cmake libnl3-devel
 RUN dnf download --source fio
 RUN rpm --nomd5 -i fio*.src.rpm
 RUN cd ~/rpmbuild/SPECS && dnf builddep -y --spec fio.spec

rpm/vitastor-el9.spec

@@ -1,11 +1,11 @@
 Name:           vitastor
-Version:        3.0.5
+Version:        3.0.12
 Release:        1%{?dist}
 Summary:        Vitastor, a fast software-defined clustered block storage
 
 License:        Vitastor Network Public License 1.1
 URL:            https://vitastor.io/
-Source0:        vitastor-3.0.5.el9.tar.gz
+Source0:        vitastor-3.0.12.el9.tar.gz
 
 BuildRequires:  gperftools-devel
 BuildRequires:  gcc-c++
@@ -16,7 +16,6 @@ BuildRequires:  gf-complete-devel
 BuildRequires:  rdma-core-devel
 BuildRequires:  cmake
 BuildRequires:  libnl3-devel
-BuildRequires:  c-ares-devel
 Requires:       vitastor-osd = %{version}-%{release}
 Requires:       vitastor-mon = %{version}-%{release}
 Requires:       vitastor-client = %{version}-%{release}

src/CMakeLists.txt

@@ -1,9 +1,8 @@
-cmake_minimum_required(VERSION 2.8.12)
+cmake_minimum_required(VERSION 2.8...3.30)
 
 project(vitastor)
 
 include(GNUInstallDirs)
-include(CTest)
 include(CheckIncludeFile)
 
 find_package(PkgConfig)
@@ -21,7 +20,7 @@ if("${CMAKE_INSTALL_PREFIX}" MATCHES "^/usr/local/?$")
 endif()
 set(ENABLE_COVERAGE false CACHE BOOL "Enable code coverage")
 
-add_definitions(-DVITASTOR_VERSION="3.0.5")
+add_definitions(-DVITASTOR_VERSION="3.0.12")
 add_definitions(-D_GNU_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Wall -Wno-sign-compare -Wno-comment -Wno-parentheses -Wno-pointer-arith -fdiagnostics-color=always -fno-omit-frame-pointer -fvisibility=hidden -I ${CMAKE_SOURCE_DIR}/src)
 add_link_options(-fno-omit-frame-pointer)
 if (${WITH_ASAN})
@@ -75,14 +74,6 @@ if (RDMACM_LIBRARIES)
 	add_definitions(-DWITH_RDMACM)
 endif (RDMACM_LIBRARIES)
 
-find_package(OpenSSL REQUIRED)
-if (OPENSSL_FOUND)
-	add_definitions(-DWITH_OPENSSL)
-endif (OPENSSL_FOUND)
-
-pkg_check_modules(CARES REQUIRED libcares)
-include_directories(${CARES_INCLUDE_DIRS})
-
 if (${WITH_SYSTEM_LIBURING})
 	pkg_check_modules(LIBURING REQUIRED liburing>=2.10)
 	include_directories(${LIBURING_INCLUDE_DIRS})

src/blockstore/CMakeLists.txt

@@ -1,10 +1,10 @@
-cmake_minimum_required(VERSION 2.8.12)
+cmake_minimum_required(VERSION 2.8...3.30)
 
 project(vitastor)
 
 # libvitastor_blk.a
 add_library(vitastor_blk STATIC
-	../util/allocator.cpp ../util/crc32c.c ../util/xxhash.c ../util/ringloop.cpp
+	../util/allocator.cpp ../util/crc32c.c ../util/ringloop.cpp
 	multilist.cpp blockstore_heap.cpp blockstore_disk.cpp
 	blockstore.cpp blockstore_impl.cpp blockstore_init.cpp blockstore_open.cpp
 	blockstore_flush.cpp blockstore_read.cpp blockstore_stable.cpp blockstore_sync.cpp blockstore_write.cpp

src/blockstore/blockstore.h

@@ -228,4 +228,9 @@ public:
     virtual uint64_t get_journal_size() = 0;
 
     virtual uint32_t get_bitmap_granularity() = 0;
+
+    virtual uint64_t get_live_entries() = 0;
+    virtual uint64_t get_live_memory() = 0;
+    virtual uint64_t get_garbage_entries() = 0;
+    virtual uint64_t get_garbage_memory() = 0;
 };

src/blockstore/blockstore_disk.cpp

@@ -83,21 +83,20 @@ void blockstore_disk_t::parse_config(std::map<std::string, std::string> & config
     {
         data_csum_type = BLOCKSTORE_CSUM_CRC32C;
     }
-    else if (config["data_csum_type"] == "xxh3_32")
-    {
-        data_csum_type = BLOCKSTORE_CSUM_XXH3_32;
-    }
     else if (config["data_csum_type"] == "" || config["data_csum_type"] == "none")
     {
         data_csum_type = BLOCKSTORE_CSUM_NONE;
     }
     else
     {
-        throw std::runtime_error("data_csum_type="+config["data_csum_type"]+" is unsupported, only \"crc32c\", \"xxh3_32\" and \"none\" are supported");
+        throw std::runtime_error("data_csum_type="+config["data_csum_type"]+" is unsupported, only \"crc32c\" and \"none\" are supported");
     }
     csum_block_size = parse_size(config["csum_block_size"]);
     discard_on_start = config.find("discard_on_start") != config.end() &&
         (config["discard_on_start"] == "true" || config["discard_on_start"] == "1" || config["discard_on_start"] == "yes");
+    gc_on_start = config.find("gc_on_start") == config.end() ||
+        (config["gc_on_start"] == "true" || config["gc_on_start"] == "1" || config["gc_on_start"] == "yes");
+    skip_double_claim = (config["skip_double_claim"] == "true" || config["skip_double_claim"] == "1" || config["skip_double_claim"] == "yes");
     min_discard_size = parse_size(config["min_discard_size"]);
     if (!min_discard_size)
         min_discard_size = 1024*1024;

src/blockstore/blockstore_disk.h

@@ -16,7 +16,6 @@
 #define BLOCKSTORE_CSUM_NONE 0
 // Lower byte of checksum type is its length
 #define BLOCKSTORE_CSUM_CRC32C 0x104
-#define BLOCKSTORE_CSUM_XXH3_32 0x204
 
 #define MOCK_DATA_FD 1000
 #define MOCK_META_FD 1001
@@ -58,6 +57,10 @@ struct blockstore_disk_t
     bool inmemory_journal = true;
     // Data discard granularity and minimum size (for the sake of performance)
     bool discard_on_start = false;
+    // GC on start (new store)
+    bool gc_on_start = true;
+    // Skip double claim conflicts on start (new store, temporary until the bug is found)
+    bool skip_double_claim = false;
     uint64_t min_discard_size = 1024*1024;
     uint64_t discard_granularity = 0;
 

src/blockstore/blockstore_flush.cpp

@@ -174,14 +174,18 @@ bool journal_flusher_co::loop()
     else if (wait_state == 19) goto resume_19;
     else if (wait_state == 20) goto resume_20;
     else if (wait_state == 21) goto resume_21;
+    else if (wait_state == 22) goto resume_22;
+    else if (wait_state == 23) goto resume_23;
+    else if (wait_state == 24) goto resume_24;
+    else if (wait_state == 25) goto resume_25;
 resume_0:
     wait_state = 0;
     wait_count = 0;
     cur_oid = {};
     res = bs->heap->get_next_compact(cur_oid);
-    // Advance fsynced_lsn every <journal_trim_interval> intent writes
     if ((bs->intent_write_counter >= bs->journal_trim_interval) && co_id == 0)
     {
+        // Advance fsynced_lsn every <journal_trim_interval> intent writes
         bs->intent_write_counter = 0;
 resume_17:
 resume_18:
@@ -196,6 +200,7 @@ resume_21:
     if (res == ENOENT && flusher->force_start > 0 && co_id == 0 &&
         (!bs->dsk.disable_journal_fsync || !bs->dsk.disable_meta_fsync || !bs->dsk.disable_data_fsync))
     {
+        // When under pressure, do an additional fsync to force entries to be marked compactable
         flusher->active_flushers++;
 resume_14:
 resume_15:
@@ -259,11 +264,9 @@ resume_1:
         if (wr->type() == BS_HEAP_SMALL_WRITE ||
             wr->type() == BS_HEAP_INTENT_WRITE && bs->dsk.csum_block_size > bs->dsk.bitmap_granularity)
         {
-            auto res = bs->prepare_read(read_vec, cur_obj, wr, 0, bs->dsk.data_block_size,
+            bs->prepare_read(read_vec, cur_obj, wr, 0, bs->dsk.data_block_size,
                 wr->type() == BS_HEAP_INTENT_WRITE && bs->dsk.csum_block_size > bs->dsk.bitmap_granularity && !bs->perfect_csum_update
                 ? COPY_BUF_SKIP_CSUM : 0);
-            if (res > 0)
-                copy_count++;
         }
     });
     if (!compact_info.compact_lsn)
@@ -273,6 +276,25 @@ resume_1:
         bs->heap->unlock_entry(cur_oid);
         goto resume_0;
     }
+    flusher->active_flushers++;
+    for (i = 0; i < read_vec.size(); i++)
+    {
+        if ((read_vec[i].copy_flags & COPY_BUF_JOURNAL) &&
+            !(read_vec[i].copy_flags & COPY_BUF_COALESCED))
+        {
+            copy_count++;
+        }
+    }
+    if (copy_count > 0 && !bs->dsk.disable_data_fsync)
+    {
+        init_fsync_data();
+    }
+    if (bs->log_level > 10)
+    {
+        printf("Compacting %jx:%jx v%ju..v%ju / l%ju..l%ju (%d writes)\n", cur_oid.inode, cur_oid.stripe,
+            compact_info.clean_wr->version, compact_info.compact_version,
+            compact_info.clean_wr->lsn, compact_info.compact_lsn, copy_count);
+    }
     mem_or(new_bmp, compact_info.clean_wr->get_int_bitmap(bs->heap), bs->dsk.clean_entry_bitmap_size);
     if (!bitmap_copied)
     {
@@ -291,13 +313,6 @@ resume_1:
         csum_copy.clear();
     }
     clean_loc = compact_info.clean_wr->big_location(bs->heap);
-    flusher->active_flushers++;
-    if (bs->log_level > 10)
-    {
-        printf("Compacting %jx:%jx v%ju..v%ju / l%ju..l%ju (%d writes)\n", cur_oid.inode, cur_oid.stripe,
-            compact_info.clean_wr->version, compact_info.compact_version,
-            compact_info.clean_wr->lsn, compact_info.compact_lsn, copy_count);
-    }
     overwrite_start = overwrite_end = 0;
     if (read_vec.size() > 0)
     {
@@ -336,6 +351,13 @@ resume_3:
     if (res == ENOENT || res == EDOM)
     {
         // Abort compaction
+abort_compact:
+        if (copy_count > 0 && !bs->dsk.disable_data_fsync)
+        {
+            cur_sync->member_count--;
+            if (cur_sync->member_count > 0)
+                bs->ringloop->wakeup();
+        }
         flusher->flushing.erase(cur_oid);
         bs->heap->unlock_entry(cur_oid);
         flusher->active_flushers--;
@@ -349,10 +371,7 @@ resume_4:
         if (res == ENOENT)
         {
             // Abort compaction
-            flusher->flushing.erase(cur_oid);
-            bs->heap->unlock_entry(cur_oid);
-            flusher->active_flushers--;
-            goto resume_0;
+            goto abort_compact;
         }
         if (res == EAGAIN)
         {
@@ -381,14 +400,14 @@ resume_9:
     for (i = 0; i < read_vec.size(); i++)
     {
         if ((read_vec[i].copy_flags & COPY_BUF_JOURNAL) &&
-            !(read_vec[i].copy_flags & COPY_BUF_COALESCED) ||
-            (read_vec[i].copy_flags & COPY_BUF_PADDED)) // FIXME Shit, simplify these flags
+            !(read_vec[i].copy_flags & COPY_BUF_COALESCED))
         {
             assert(read_vec[i].buf);
             await_sqe(10);
             data->iov = (struct iovec){ read_vec[i].buf + (read_vec[i].copy_flags & COPY_BUF_PADDED
                 ? read_vec[i].offset - read_vec[i].disk_offset : 0), (size_t)read_vec[i].len };
             data->callback = simple_callback_w;
+            assert(clean_loc + read_vec[i].offset + data->iov.iov_len <= bs->dsk.block_count*bs->dsk.data_block_size);
             io_uring_prep_writev(sqe, bs->dsk.data_fd, &data->iov, 1, bs->dsk.data_offset + clean_loc + read_vec[i].offset);
             wait_count++;
         }
@@ -399,6 +418,17 @@ resume_11:
         wait_state = 11;
         return false;
     }
+    if (copy_count > 0 && !bs->dsk.disable_data_fsync)
+    {
+resume_22:
+resume_23:
+resume_24:
+resume_25:
+        if (!fsync_data(22))
+        {
+            return false;
+        }
+    }
     // Lock is only needed to prevent freeing the big_write because we overwrite it...
     bs->heap->unlock_entry(cur_oid);
     // Mark the object compacted, but don't free and remove small_writes
@@ -408,12 +438,14 @@ resume_11:
     if (!cur_obj)
     {
         // Abort compaction
+        flusher->active_flushers--;
         flusher->flushing.erase(cur_oid);
         goto resume_0;
     }
     if (!calc_block_checksums())
     {
         // Abort compaction
+        flusher->active_flushers--;
         flusher->flushing.erase(cur_oid);
         goto resume_0;
     }
@@ -422,6 +454,7 @@ resume_11:
     if (res == EBUSY)
     {
         // Abort compaction, object is already overwritten by something else
+        flusher->active_flushers--;
         flusher->flushing.erase(cur_oid);
         goto resume_0;
     }
@@ -586,7 +619,7 @@ int journal_flusher_co::check_and_punch_checksums()
             bs->heap->calc_block_checksums((uint32_t*)(new_csums+csum_off), vec.buf, punch_bmp, vec.offset, vec.offset+vec.len, true, NULL);
         }
     }
-    // Modified, we should add_punch_holes and then write the block to disk
+    // Modified, we should punch_holes and then write the block to disk
     return EBUSY;
 }
 
@@ -699,6 +732,67 @@ resume_1:
     return true;
 }
 
+void journal_flusher_co::init_fsync_data()
+{
+    cur_sync = flusher->data_syncs.begin();
+    if (cur_sync == flusher->data_syncs.end() || cur_sync->ready_count > 0)
+    {
+        cur_sync = flusher->data_syncs.emplace(cur_sync);
+    }
+    cur_sync->member_count++;
+}
+
+bool journal_flusher_co::fsync_data(int wait_base)
+{
+    if (wait_state == wait_base)
+        goto resume_0;
+    else if (wait_state == wait_base+1)
+        goto resume_1;
+    else if (wait_state == wait_base+2)
+        goto resume_2;
+    else if (wait_state == wait_base+3)
+        goto resume_3;
+    cur_sync->ready_count++;
+resume_0:
+    if (cur_sync->ready_count < cur_sync->member_count)
+    {
+        wait_state = wait_base;
+        return false;
+    }
+    if (!cur_sync->sent)
+    {
+        // Sync batch is ready. Do it.
+        await_sqe(1);
+        data->iov = { 0 };
+        data->callback = simple_callback_w;
+        io_uring_prep_fsync(sqe, bs->dsk.data_fd, IORING_FSYNC_DATASYNC);
+        cur_sync->sent = true;
+        wait_count++;
+resume_2:
+        if (wait_count > 0)
+        {
+            wait_state = wait_base+2;
+            return false;
+        }
+        cur_sync->done = true;
+        // Wake up other flushers
+        bs->ringloop->wakeup();
+    }
+resume_3:
+    if (!cur_sync->done)
+    {
+        wait_state = wait_base+3;
+        return false;
+    }
+    cur_sync->done_count++;
+    if (cur_sync->done_count >= cur_sync->member_count)
+    {
+        flusher->data_syncs.erase(cur_sync);
+        cur_sync = flusher->data_syncs.end();
+    }
+    return true;
+}
+
 bool journal_flusher_co::fsync_meta(int wait_base)
 {
     if (wait_state == wait_base)        goto resume_0;

src/blockstore/blockstore_flush.h

@@ -25,6 +25,15 @@ struct flusher_meta_write_t
     std::map<uint64_t, meta_sector_t>::iterator it;
 };
 
+struct flusher_data_sync_t
+{
+    int member_count = 0;
+    int ready_count = 0;
+    int done_count = 0;
+    bool sent = false;
+    bool done = false;
+};
+
 class journal_flusher_t;
 
 // Journal flusher coroutine
@@ -58,6 +67,7 @@ class journal_flusher_co
     int i, res;
     bool read_to_fill_incomplete;
     int copy_count;
+    std::list<flusher_data_sync_t>::iterator cur_sync;
 
     friend class journal_flusher_t;
 
@@ -68,6 +78,8 @@ class journal_flusher_co
     bool calc_block_checksums();
     bool write_meta_block(int wait_base);
     bool read_buffered(int wait_base);
+    void init_fsync_data();
+    bool fsync_data(int wait_base);
     bool fsync_meta(int wait_base);
     bool fsync_buffer(int wait_base);
     bool trim_lsn(int wait_base);
@@ -88,6 +100,7 @@ class journal_flusher_t
 
     robin_hood::unordered_flat_set<object_id> flushing;
     int active_flushers = 0;
+    std::list<flusher_data_sync_t> data_syncs;
     int wanting_meta_fsync = 0;
     bool fsyncing_meta = false;
     int syncing_buffer = 0;

src/blockstore/blockstore_heap.h

@@ -43,7 +43,7 @@ struct __attribute__((__packed__)) heap_entry_t
 {
     uint16_t size;
     uint16_t entry_type;
-    uint32_t checksum;
+    uint32_t crc32c;
     uint64_t lsn;
     uint64_t inode;
     uint64_t stripe;
@@ -57,11 +57,11 @@ struct __attribute__((__packed__)) heap_entry_t
     inline heap_small_write_t& small() { return *(heap_small_write_t*)this; }
     inline heap_big_write_t& big() { return *(heap_big_write_t*)this; }
     inline heap_big_intent_t& big_intent() { return *(heap_big_intent_t*)this; }
-    bool is_garbage();
+    bool is_garbage() const;
     void set_garbage();
-    bool is_overwrite();
-    bool is_compactable();
-    bool is_before(heap_entry_t *other);
+    bool is_overwrite() const;
+    bool is_compactable() const;
+    bool is_before(const heap_entry_t *other) const;
     uint32_t get_size(blockstore_heap_t *heap);
     uint8_t *get_ext_bitmap(blockstore_heap_t *heap);
     uint8_t *get_int_bitmap(blockstore_heap_t *heap);
@@ -69,8 +69,7 @@ struct __attribute__((__packed__)) heap_entry_t
     uint32_t *get_checksum(blockstore_heap_t *heap);
     uint64_t big_location(blockstore_heap_t *heap);
     void set_big_location(blockstore_heap_t *heap, uint64_t location);
-    uint32_t calc_checksum(blockstore_heap_t *heap);
-    uint32_t calc_checksum(blockstore_disk_t *dsk);
+    uint32_t calc_crc32c();
 };
 
 struct __attribute__((__packed__)) heap_small_write_t
@@ -81,7 +80,7 @@ struct __attribute__((__packed__)) heap_small_write_t
     uint32_t offset;
     uint32_t len;
 
-    // Also includes 1 bitmap and 1 checksum after the bitmap if block checksums are disabled
+    // Also includes 1 bitmap and 1 crc32c after the bitmap if checksums are disabled
 };
 
 struct __attribute__((__packed__)) heap_big_write_t
@@ -99,7 +98,7 @@ struct __attribute__((__packed__)) heap_big_intent_t
     uint32_t offset;
     uint32_t len;
 
-    // Also includes 2 bitmaps and 1 checksums if block checksums are disabled
+    // Also includes 2 bitmaps and 1 crc32c if checksums are disabled
 };
 
 struct __attribute__((__packed__)) heap_list_item_t
@@ -118,10 +117,13 @@ struct heap_object_mvcc_t
 
 struct heap_block_info_t
 {
-    uint32_t used_space = 0;
+    struct __attribute__((__packed__))
+    {
+        uint32_t used_space = 0;
+        uint32_t garbage_space = 0;
+    };
     uint64_t mod_lsn = 0, mod_lsn_to = 0; // only 1 block write of LSN sequence is allowed at a moment
-    bool is_writing: 1;
-    bool has_garbage: 1;
+    bool is_writing = false;
     std::vector<heap_list_item_t*> entries;
 };
 
@@ -156,6 +158,16 @@ struct heap_li_equal
     }
 };
 
+struct heap_recheck_state_t
+{
+    heap_entry_t *obj = NULL;
+    heap_entry_t *next_wr = NULL;
+    size_t total_reads = 0;
+    size_t sent_reads = 0;
+    size_t checked_reads = 0;
+    heap_entry_t *bad_wr = NULL;
+};
+
 using i64hash_t = robin_hood::hash<uint64_t>;
 using heap_inode_map_t = robin_hood::unordered_flat_set<heap_list_item_t*, heap_li_hash, heap_li_equal, 88>;
 using heap_block_index_t = robin_hood::unordered_flat_map<uint64_t,
@@ -185,6 +197,11 @@ class blockstore_heap_t
     uint64_t buffer_area_used_space = 0;
     uint64_t data_used_space = 0;
 
+    uint64_t live_entries = 0;
+    uint64_t live_memory = 0;
+    uint64_t garbage_entries = 0;
+    uint64_t garbage_memory = 0;
+
     uint64_t next_lsn = 0;
     uint32_t last_allocated_block = UINT32_MAX;
     heap_mvcc_map_t object_mvcc;
@@ -201,9 +218,11 @@ class blockstore_heap_t
 
     bool marked_used_blocks = false;
     bool recheck_queue_filled = false;
-    std::vector<heap_list_item_t*> loaded_list_items;
+    std::vector<heap_list_item_t*> postponed_items;
     std::set<uint32_t> recheck_modified_blocks;
     std::deque<heap_entry_t*> recheck_queue;
+    std::map<heap_entry_t*, heap_recheck_state_t> recheck_states;
+    size_t recheck_pending_reads = 0;
     int recheck_in_progress = 0;
     bool in_recheck = false;
     std::function<void(bool is_data, uint64_t offset, uint64_t len, uint8_t* buf, std::function<void()>)> recheck_cb;
@@ -212,14 +231,22 @@ class blockstore_heap_t
     uint64_t get_pg_id(inode_t inode, uint64_t stripe);
     bool validate_object(heap_entry_t *obj);
     void fill_recheck_queue();
+    void recheck_drop_entries(heap_entry_t *obj, heap_entry_t *bad_wr);
+    void recheck_start_reads(heap_recheck_state_t *st);
     int mark_used_blocks();
+    void init_free_bad_entry(heap_entry_t *wr);
+    void init_erase_bad_entry(heap_list_item_t *li);
+    bool init_erase_double_claim(heap_list_item_t *prev_li, heap_list_item_t *cur_li);
+    void recheck_full_gc();
     void recheck_buffer(heap_entry_t *cwr, uint8_t *buf);
     void defragment_block(uint32_t block_num);
     void reshard_add(heap_reshard_state_t *st, heap_list_item_t *li);
 
     void gc_block(heap_block_info_t & inf);
     int allocate_entry(uint32_t entry_size, uint32_t *block_num, bool allow_last_free);
-    void insert_list_item(heap_list_item_t *li);
+    void insert_list_items(heap_list_item_t** v, size_t count, bool postpone);
+    void remove_list_item(heap_list_item_t *li);
+    void unlink_list_item(heap_list_item_t *li);
     int add_entry(uint32_t wr_size, uint32_t *modified_block, bool allow_last_free,
         bool explicit_complete, std::function<void(heap_entry_t *wr)> fill_entry);
     int add_simple(heap_entry_t *obj, uint64_t version, uint32_t *modified_block, uint32_t entry_type);
@@ -346,6 +373,10 @@ public:
     uint32_t get_compact_queue_size();
     uint32_t get_to_compact_count();
     uint64_t get_compacted_count();
+    uint64_t get_live_entries();
+    uint64_t get_live_memory();
+    uint64_t get_garbage_entries();
+    uint64_t get_garbage_memory();
 
     uint64_t entry_pos(uint32_t block_num, uint32_t offset);
     heap_entry_t *entry_from_pos(uint64_t entry_pos, bool allow_unallocated = false);

src/blockstore/blockstore_impl.cpp

@@ -101,6 +101,7 @@ void blockstore_impl_t::loop()
         unsigned initial_ring_space = ringloop->space_left();
         int op_idx = 0, new_idx = 0;
         bool has_unfinished_writes = false;
+        bool has_unfinished_sync = false;
         for (; op_idx < submit_queue.size(); op_idx++, new_idx++)
         {
             auto op = submit_queue[op_idx];
@@ -138,7 +139,13 @@ void blockstore_impl_t::loop()
             else if (op->opcode == BS_OP_SYNC)
             {
                 // syncs only completed writes, so doesn't have to be blocked by anything
-                wr_st = continue_sync(op);
+                if (!has_unfinished_sync)
+                {
+                    wr_st = continue_sync(op);
+                    has_unfinished_sync = (wr_st != 2);
+                }
+                else
+                    wr_st = 0;
             }
             else if (op->opcode == BS_OP_STABLE || op->opcode == BS_OP_ROLLBACK)
             {
@@ -154,9 +161,7 @@ void blockstore_impl_t::loop()
                     wr_st = 2;
                 }
                 else
-                {
                     wr_st = 0;
-                }
             }
             if (wr_st == 2)
             {

src/blockstore/blockstore_impl.h

@@ -229,4 +229,9 @@ public:
     uint64_t get_free_block_count();
     inline uint32_t get_bitmap_granularity() { return dsk.bitmap_granularity; }
     inline uint64_t get_journal_size() { return dsk.journal_len; }
+
+    inline uint64_t get_live_entries() { return heap->get_live_entries(); }
+    inline uint64_t get_live_memory() { return heap->get_live_memory(); }
+    inline uint64_t get_garbage_entries() { return heap->get_garbage_entries(); }
+    inline uint64_t get_garbage_memory() { return heap->get_garbage_memory(); }
 };

src/blockstore/blockstore_init.cpp

@@ -153,6 +153,14 @@ resume_1:
             );
             exit(1);
         }
+        uint32_t csum = hdr->header_csum;
+        hdr->header_csum = 0;
+        if (crc32c(0, hdr, sizeof(*hdr)) != csum)
+        {
+            printf("Metadata header is corrupt (checksum mismatch).\n");
+            exit(1);
+        }
+        hdr->header_csum = csum;
     }
     bs->heap->start_load(((blockstore_meta_header_v3_t *)bs->meta_superblock)->completed_lsn);
     if (bs->dsk.inmemory_journal)
@@ -240,23 +248,7 @@ resume_4:
     }
     // metadata read finished
     bs->heap->finish_load();
-    printf("Metadata entries loaded: %ju, used blocks: %ju / %ju\n", entries_loaded, bs->heap->get_data_used_space() / bs->dsk.data_block_size, bs->dsk.block_count);
-    if (zero_on_init && !bs->dsk.disable_meta_fsync)
-    {
-        GET_SQE();
-        io_uring_prep_fsync(sqe, bs->dsk.meta_fd, IORING_FSYNC_DATASYNC);
-        last_read_offset = 0;
-        data->iov = { 0 };
-        data->callback = [this](ring_data_t *data) { handle_event(data, -1); };
-        submitted++;
-        bs->ringloop->submit();
-    resume_5:
-        if (submitted > 0)
-        {
-            wait_state = 5;
-            return 1;
-        }
-    }
+    printf("Metadata entries loaded: %ju, rechecking unfinished writes and garbage entries\n", entries_loaded);
     // asynchronous recheck
 resume_6:
     wait_state = 6;
@@ -293,6 +285,11 @@ resume_7:
     if (bs->readonly)
     {
         recheck_mod.clear();
+        printf("Actual metadata entries: %ju\n", bs->heap->get_live_entries());
+    }
+    else
+    {
+        printf("Actual metadata entries: %ju, clearing garbage in %zu metadata blocks\n", bs->heap->get_live_entries(), recheck_mod.size());
     }
     for (i = 0; i < recheck_mod.size(); i++)
     {
@@ -306,7 +303,7 @@ resume_8:
         uint32_t block_num = recheck_mod[i];
         uint64_t block_offset = bs->dsk.meta_offset + (uint64_t)(block_num+1) * bs->dsk.meta_block_size;
         data = ((ring_data_t*)sqe->user_data);
-        uint8_t *buf = (uint8_t*)malloc_or_die(bs->dsk.meta_block_size);
+        uint8_t *buf = (uint8_t*)memalign_or_die(MEM_ALIGNMENT, bs->dsk.meta_block_size);
         bs->heap->get_meta_block(block_num, buf);
         data->iov = { buf, bs->dsk.meta_block_size };
         data->callback = [this, buf, block_offset](ring_data_t *data)
@@ -332,5 +329,25 @@ resume_9:
     }
     free(metadata_buffer);
     metadata_buffer = NULL;
+    if (!bs->dsk.disable_meta_fsync && !bs->readonly)
+    {
+        GET_SQE();
+        io_uring_prep_fsync(sqe, bs->dsk.meta_fd, IORING_FSYNC_DATASYNC);
+        last_read_offset = 0;
+        data->iov = { 0 };
+        data->callback = [this](ring_data_t *data) { handle_event(data, -1); };
+        submitted++;
+        bs->ringloop->submit();
+    resume_5:
+        if (submitted > 0)
+        {
+            wait_state = 5;
+            return 1;
+        }
+    }
+    printf("Loading finished. Data used: %ju / %ju bytes (%s / %s)\n",
+        bs->heap->get_data_used_space(), bs->dsk.block_count * bs->dsk.data_block_size,
+        format_size(bs->heap->get_data_used_space()).c_str(),
+        format_size(bs->dsk.block_count * bs->dsk.data_block_size).c_str());
     return 0;
 }

src/blockstore/blockstore_read.cpp

@@ -462,6 +462,10 @@ int blockstore_impl_t::read_bitmap(object_id oid, uint64_t target_version, void
         {
             if (target_version >= wr->version)
             {
+                if (wr->type() == BS_HEAP_DELETE)
+                {
+                    return false;
+                }
                 found = true;
                 if (result_version)
                 {

src/blockstore/blockstore_stable.cpp

@@ -16,6 +16,7 @@ int blockstore_impl_t::dequeue_stable(blockstore_op_t *op)
     else if (priv->op_state == 5) goto resume_5;
     assert(!priv->op_state);
     op->retval = 0;
+    PRIV(op)->lsn = 0;
     priv->modified_block = priv->modified_block2 = UINT32_MAX;
     for (priv->stab_pos = 0; priv->stab_pos < op->len; priv->stab_pos++)
     {
@@ -36,6 +37,12 @@ int blockstore_impl_t::dequeue_stable(blockstore_op_t *op)
                 FINISH_OP(op);
                 return 2;
             }
+            if (res == ENOENT)
+            {
+                op->retval = -ENOENT;
+                FINISH_OP(op);
+                return 2;
+            }
             if (res == ENOSPC)
             {
                 if (!heap->get_to_compact_count())

src/blockstore/blockstore_sync.cpp

@@ -9,6 +9,7 @@ int blockstore_impl_t::continue_sync(blockstore_op_t *op)
     if (!PRIV(op)->op_state)
     {
         op->retval = 0;
+        PRIV(op)->lsn = 0;
     }
     int res = do_sync(op, 0);
     if (res == 2)
@@ -104,7 +105,8 @@ int blockstore_impl_t::do_sync(blockstore_op_t *op, int base_state)
         unsynced_data_write_count = unsynced_buffer_write_count = unsynced_meta_write_count = 0;
         return 2;
     }
-    PRIV(op)->modified_block = heap->get_completed_lsn();
+    assert(!PRIV(op)->lsn);
+    PRIV(op)->lsn = heap->get_completed_lsn();
     if (!submit_fsyncs(PRIV(op)->pending_ops))
     {
         PRIV(op)->wait_detail = 1;
@@ -118,6 +120,6 @@ resume_1:
         return 1;
     }
 resume_2:
-    heap->mark_lsn_fsynced(PRIV(op)->modified_block);
+    heap->mark_lsn_fsynced(PRIV(op)->lsn);
     return 2;
 }

src/blockstore/blockstore_write.cpp

@@ -37,6 +37,7 @@ void blockstore_impl_t::prepare_meta_block_write(uint32_t modified_block)
         heap->complete_block_write(modified_block);
         ringloop->wakeup();
     };
+    assert(((uint64_t)modified_block+2)*dsk.meta_block_size <= dsk.meta_area_size);
     io_uring_prep_writev(
         sqe, dsk.meta_fd, &data->iov, 1, dsk.meta_offset + ((uint64_t)modified_block+1)*dsk.meta_block_size
     );
@@ -177,6 +178,7 @@ enospc:
         ring_data_t *data = ((ring_data_t*)sqe->user_data);
         data->iov = (struct iovec){ op->buf, op->len };
         data->callback = [this, op](ring_data_t *data) { handle_write_event(data, op); };
+        assert(loc+op->offset+op->len <= dsk.block_count*dsk.data_block_size);
         io_uring_prep_writev(sqe, dsk.data_fd, &data->iov, 1, dsk.data_offset + loc + op->offset);
         PRIV(op)->pending_ops++;
         write_iodepth++;
@@ -264,6 +266,7 @@ enospc:
             BS_SUBMIT_GET_SQE(sqe2, data2);
             data2->iov = (struct iovec){ op->buf, op->len };
             data2->callback = [this, op](ring_data_t *data) { handle_write_event(data, op); };
+            assert(loc+op->len <= dsk.journal_len);
             io_uring_prep_writev(sqe2, dsk.journal_fd, &data2->iov, 1, dsk.journal_offset + loc);
             PRIV(op)->pending_ops++;
         }
@@ -453,6 +456,7 @@ resume_10:
     BS_SUBMIT_GET_SQE(sqe, data);
     data->iov = (struct iovec){ op->buf, op->len };
     data->callback = [this, op](ring_data_t *data) { handle_write_event(data, op); };
+    assert(PRIV(op)->location + op->offset <= dsk.block_count*dsk.data_block_size);
     io_uring_prep_writev(sqe, dsk.data_fd, &data->iov, 1, dsk.data_offset + PRIV(op)->location + op->offset);
     if (dsk.use_atomic_flag)
         sqe->rw_flags = RWF_ATOMIC;

src/blockstore/ondisk_formats.h

@@ -141,7 +141,7 @@ struct __attribute__((__packed__)) journal_entry
 inline uint32_t je_crc32(journal_entry *je)
 {
     // 0x48674bc7 = crc32(4 zero bytes)
-    return crc32c(0x48674bc7, ((uint8_t*)je)+4, je->size-4);
+    return je->size < 4 ? 0 : crc32c(0x48674bc7, ((uint8_t*)je)+4, je->size-4);
 }
 
 // "VITAstor"

src/blockstore/v1/flush.cpp

@@ -520,6 +520,7 @@ resume_2:
                 await_sqe(15);
                 data->iov = (struct iovec){ it->buf, (size_t)it->len };
                 data->callback = simple_callback_w;
+                assert(clean_loc+it->offset+it->len <= bs->dsk.block_count*bs->dsk.data_block_size);
                 io_uring_prep_writev(
                     sqe, bs->dsk.data_fd, &data->iov, 1, bs->dsk.data_offset + clean_loc + it->offset
                 );
@@ -749,6 +750,7 @@ bool journal_flusher_co::write_meta_block(flusher_meta_write_t & meta_block, int
     await_sqe(0);
     data->iov = (struct iovec){ meta_block.buf, (size_t)bs->dsk.meta_block_size };
     data->callback = simple_callback_w;
+    assert(bs->dsk.meta_block_size + meta_block.sector + bs->dsk.meta_block_size <= bs->dsk.meta_area_size);
     io_uring_prep_writev(
         sqe, bs->dsk.meta_fd, &data->iov, 1, bs->dsk.meta_offset + bs->dsk.meta_block_size + meta_block.sector
     );

src/blockstore/v1/impl.cpp

@@ -855,4 +855,29 @@ std::string blockstore_impl_t::get_op_diag(blockstore_op_t *op)
     return std::string(buf);
 }
 
+uint64_t blockstore_impl_t::get_live_entries()
+{
+    return used_blocks;
+}
+
+uint64_t blockstore_impl_t::get_live_memory()
+{
+    uint64_t used = 0;
+    for (auto & kv: clean_db_shards)
+    {
+        used += kv.second.size() * sizeof(blockstore_clean_db_t::value_type);
+    }
+    return used;
+}
+
+uint64_t blockstore_impl_t::get_garbage_entries()
+{
+    return dirty_db.size();
+}
+
+uint64_t blockstore_impl_t::get_garbage_memory()
+{
+    return (sizeof(obj_ver_id) + sizeof(dirty_entry) + 32) * dirty_db.size();
+}
+
 } // namespace v1

src/blockstore/v1/impl.h

@@ -332,6 +332,10 @@ public:
     inline uint64_t get_free_block_count() { return dsk.block_count - used_blocks; }
     inline uint32_t get_bitmap_granularity() { return dsk.disk_alignment; }
     inline uint64_t get_journal_size() { return dsk.journal_len; }
+    uint64_t get_live_entries();
+    uint64_t get_live_memory();
+    uint64_t get_garbage_entries();
+    uint64_t get_garbage_memory();
 };
 
 } // namespace v1

src/blockstore/v1/journal.cpp

@@ -193,6 +193,7 @@ void blockstore_impl_t::prepare_journal_sector_write(int cur_sector, blockstore_
             (size_t)journal.block_size
         };
         data->callback = [this, flush_id = journal.submit_id](ring_data_t *data) { handle_journal_write(data, flush_id); };
+        assert(journal.sector_info[cur_sector].offset+journal.block_size <= dsk.journal_len);
         io_uring_prep_writev(
             sqe, dsk.journal_fd, &data->iov, 1, journal.offset + journal.sector_info[cur_sector].offset
         );

src/blockstore/v1/write.cpp

@@ -368,9 +368,9 @@ int blockstore_impl_t::dequeue_write(blockstore_op_t *op)
         }
         data->iov.iov_len = op->len + stripe_offset + stripe_end; // to check it in the callback
         data->callback = [this, op](ring_data_t *data) { handle_write_event(data, op); };
-        io_uring_prep_writev(
-            sqe, dsk.data_fd, PRIV(op)->iov_zerofill, vcnt, dsk.data_offset + (loc * dsk.data_block_size) + op->offset - stripe_offset
-        );
+        const uint64_t write_offset = (loc * dsk.data_block_size) + op->offset - stripe_offset;
+        assert(write_offset+op->len+stripe_offset+stripe_end <= dsk.block_count*dsk.data_block_size);
+        io_uring_prep_writev(sqe, dsk.data_fd, PRIV(op)->iov_zerofill, vcnt, dsk.data_offset + write_offset);
         PRIV(op)->pending_ops = 1;
         if (!(dirty_it->second.state & BS_ST_INSTANT))
         {
@@ -495,9 +495,8 @@ int blockstore_impl_t::dequeue_write(blockstore_op_t *op)
                 .op = op,
             });
             data2->callback = [this, flush_id = journal.submit_id](ring_data_t *data) { handle_journal_write(data, flush_id); };
-            io_uring_prep_writev(
-                sqe2, dsk.journal_fd, &data2->iov, 1, journal.offset + journal.next_free
-            );
+            assert(journal.next_free+op->len <= dsk.journal_len);
+            io_uring_prep_writev(sqe2, dsk.journal_fd, &data2->iov, 1, journal.offset + journal.next_free);
             PRIV(op)->pending_ops++;
         }
         else

src/client/CMakeLists.txt

@@ -1,4 +1,4 @@
-cmake_minimum_required(VERSION 2.8.12)
+cmake_minimum_required(VERSION 2.8...3.30)
 
 project(vitastor)
 
@@ -12,11 +12,11 @@ if (RDMACM_LIBRARIES)
 	set(MSGR_RDMACM "msgr_rdmacm.cpp")
 endif (RDMACM_LIBRARIES)
 add_library(vitastor_common STATIC
-	../util/epoll_manager.cpp etcd_state_client.cpp messenger.cpp ../util/addr_util.cpp
-	msgr_encrypt.cpp msgr_stop.cpp msgr_op.cpp msgr_send.cpp msgr_receive.cpp ../util/ringloop.cpp ../../json11/json11.cpp
+	../util/epoll_manager.cpp etcd_state_client.cpp messenger.cpp msgr_iothread.cpp ../util/addr_util.cpp
+	msgr_stop.cpp msgr_op.cpp msgr_send.cpp msgr_receive.cpp ../util/ringloop.cpp ../../json11/json11.cpp
 	http_client.cpp osd_ops.cpp pg_states.cpp ../util/timerfd_manager.cpp ../util/str_util.cpp ../util/json_util.cpp ${MSGR_RDMA} ${MSGR_RDMACM}
 )
-target_link_libraries(vitastor_common pthread ${OPENSSL_LIBRARIES} ${CARES_LIBRARIES})
+target_link_libraries(vitastor_common pthread)
 target_compile_options(vitastor_common PUBLIC -fPIC)
 
 # libvitastor_client.so
@@ -24,7 +24,6 @@ add_library(vitastor_client SHARED
 	cluster_client.cpp
 	cluster_client_list.cpp
 	cluster_client_wb.cpp
-	cluster_client_icache.cpp
 	vitastor_c.cpp
 )
 set_target_properties(vitastor_client PROPERTIES PUBLIC_HEADER "client/vitastor_c.h")
@@ -34,7 +33,6 @@ target_link_libraries(vitastor_client
 	${LIBURING_LIBRARIES}
 	${IBVERBS_LIBRARIES}
 	${RDMACM_LIBRARIES}
-	${OPENSSL_LIBRARIES}
 )
 set_target_properties(vitastor_client PROPERTIES VERSION ${VITASTOR_VERSION} SOVERSION 0)
 configure_file(vitastor.pc.in vitastor.pc @ONLY)
@@ -54,9 +52,6 @@ if (${WITH_FIO})
 		../util/rw_blocking.cpp
 		../util/addr_util.cpp
 	)
-	target_link_libraries(fio_vitastor_sec
-		tcmalloc_minimal
-	)
 endif (${WITH_FIO})
 
 # vitastor-nbd
@@ -100,10 +95,9 @@ endif (${WITH_QEMU})
 add_executable(test_cluster_client
 	EXCLUDE_FROM_ALL
 	../test/test_cluster_client.cpp
-	pg_states.cpp osd_ops.cpp cluster_client.cpp cluster_client_list.cpp cluster_client_wb.cpp cluster_client_icache.cpp msgr_op.cpp ../test/mock/messenger.cpp msgr_stop.cpp msgr_encrypt.cpp
+	pg_states.cpp osd_ops.cpp cluster_client.cpp cluster_client_list.cpp cluster_client_wb.cpp msgr_op.cpp ../test/mock/messenger.cpp msgr_stop.cpp
 	etcd_state_client.cpp ../util/timerfd_manager.cpp ../util/addr_util.cpp ../util/str_util.cpp ../util/json_util.cpp ../../json11/json11.cpp
 )
-target_link_libraries(test_cluster_client ${OPENSSL_LIBRARIES})
 target_compile_definitions(test_cluster_client PUBLIC -D__MOCK__)
 target_include_directories(test_cluster_client BEFORE PUBLIC ${CMAKE_SOURCE_DIR}/src/test/mock)
 add_dependencies(build_tests test_cluster_client)

src/client/cluster_client.cpp

@@ -27,7 +27,7 @@ cluster_client_t::cluster_client_t(ring_loop_t *ringloop, timerfd_manager_t *tfd
     msgr.ringloop = ringloop;
     msgr.repeer_pgs = [this](osd_num_t peer_osd)
     {
-        if (msgr.osd_peer_fds.find(peer_osd) != msgr.osd_peer_fds.end())
+        if (msgr.osd_peers.find(peer_osd) != msgr.osd_peers.end())
         {
             // peer_osd just connected
             continue_ops();
@@ -47,8 +47,8 @@ cluster_client_t::cluster_client_t(ring_loop_t *ringloop, timerfd_manager_t *tfd
     msgr.exec_op = [this](osd_op_t *op)
     {
         // Garbage in
-        fprintf(stderr, "Incoming garbage from peer %d\n", op->peer_fd);
-        msgr.stop_client(op->peer_fd);
+        fprintf(stderr, "Can't handle incoming operation from client %lu\n", op->client_id);
+        msgr.stop_client(op->client_id);
         delete op;
     };
     msgr.parse_config(config);
@@ -62,7 +62,6 @@ cluster_client_t::cluster_client_t(ring_loop_t *ringloop, timerfd_manager_t *tfd
     st_cli.on_change_node_placement_hook = [this]() { on_change_node_placement_hook(); };
     st_cli.on_load_pgs_hook = [this](bool success) { on_load_pgs_hook(success); };
     st_cli.on_reload_hook = [this]() { st_cli.load_global_config(); };
-    st_cli.on_inode_change_hook = [this](uint64_t inode, bool removed) { on_change_inode_hook(inode, removed); };
 
     st_cli.parse_config(config);
     st_cli.infinite_start = false;
@@ -78,7 +77,6 @@ cluster_client_t::cluster_client_t(ring_loop_t *ringloop, timerfd_manager_t *tfd
 
 cluster_client_t::~cluster_client_t()
 {
-    vault_destroy();
     if (retry_timeout_id >= 0)
     {
         tfd->clear_timer(retry_timeout_id);
@@ -158,7 +156,7 @@ void cluster_client_t::continue_raw_ops(osd_num_t peer_osd)
     {
         auto op = it->second;
         op->op_type = OSD_OP_OUT;
-        op->peer_fd = msgr.osd_peer_fds.at(peer_osd);
+        op->client_id = msgr.osd_peers.at(peer_osd)->client_id;
         msgr.outbox_push(op);
         raw_ops.erase(it++);
     }
@@ -483,8 +481,6 @@ void cluster_client_t::on_load_config_hook(json11::Json::object & etcd_global_co
         self_tree_metrics.clear();
         client_hostname = new_hostname;
     }
-    // vault
-    vault_parse_config();
     msgr.parse_config(config);
     st_cli.parse_config(config);
     st_cli.load_pgs();
@@ -594,7 +590,7 @@ void cluster_client_t::on_change_pool_config_hook()
         {
             if (log_level > 2 && pg_counts[pool_item.first])
             {
-                printf("Pool %u (%s) PG count changed from %lu to %lu\n", pool_item.first, pool_item.second.name.c_str(),
+                fprintf(stderr, "Pool %u (%s) PG count changed from %lu to %lu\n", pool_item.first, pool_item.second.name.c_str(),
                     pg_counts[pool_item.first], pool_item.second.real_pg_count);
             }
             // At this point, all pool operations should have been suspended
@@ -611,9 +607,6 @@ void cluster_client_t::on_change_pool_config_hook()
             pg_counts[pool_item.first] = pool_item.second.real_pg_count;
         }
     }
-    inode_cache.clear();
-    inode_cache_children.clear();
-    vault_keys.clear();
     continue_ops();
 }
 
@@ -680,10 +673,6 @@ bool cluster_client_t::flush()
 {
     if (!ringloop)
     {
-        if (vault_loading)
-        {
-            return false;
-        }
         if (wb->writeback_queue.size())
         {
             wb->start_writebacks(this, 0);
@@ -706,7 +695,7 @@ bool cluster_client_t::flush()
         sync_done = true;
     };
     execute(sync);
-    while (!sync_done || vault_loading)
+    while (!sync_done)
     {
         ringloop->loop();
         if (!sync_done)
@@ -882,13 +871,13 @@ void cluster_client_t::execute_cas(cluster_op_t *op)
         if (op->retval != expected && op->retval >= 0)
             op->retval = -EIO;
         op->retval = op->retval == -EPIPE ? -EINTR : op->retval;
-        auto peer_it = msgr.osd_peer_fds.find(op->parts[0].osd_num);
+        auto peer_it = msgr.osd_peers.find(op->parts[0].osd_num);
         if (op->retval != 0 || (op->flags & OP_IMMEDIATE_COMMIT))
         {
             auto cb = std::move(op->callback);
             cb(op);
         }
-        else if (peer_it == msgr.osd_peer_fds.end())
+        else if (peer_it == msgr.osd_peers.end())
         {
             // Care must be taken to make sure that the client doesn't reconnect to the OSD
             // before executing the previously completed operation callback (!)
@@ -899,10 +888,10 @@ void cluster_client_t::execute_cas(cluster_op_t *op)
         else
         {
             // CAS writes have a built-in sync
-            auto peer_fd = peer_it->second;
+            osd_client_t *cl = peer_it->second;
             *part = (osd_op_t){
                 .op_type = OSD_OP_OUT,
-                .peer_fd = peer_fd,
+                .client_id = cl->client_id,
                 .req = {
                     .hdr = {
                         .magic = SECONDARY_OSD_OP_MAGIC,
@@ -969,40 +958,10 @@ bool cluster_client_t::check_rw(cluster_op_t *op)
     {
         op->flags |= OP_IMMEDIATE_COMMIT;
     }
-    bool searched = false;
-    std::shared_ptr<inode_cache_t> icache;
-    if (op->opcode == OSD_OP_READ || op->opcode == OSD_OP_WRITE)
-    {
-        if (!searched)
-        {
-            icache = inode_cache_get(op->inode);
-            searched = true;
-        }
-        if (icache && icache->has_parent_loop && op->opcode == OSD_OP_READ)
-        {
-            op->retval = -EINVAL;
-            auto cb = std::move(op->callback);
-            cb(op);
-            return false;
-        }
-        if (icache && icache->op_enc)
-        {
-            // Use shared_ptr aliasing to attach op_enc to the inode cache entry
-            op->enc = std::shared_ptr<osd_op_enc_t>(icache, icache->op_enc);
-        }
-        else
-            op->enc.reset();
-    }
-    else
-        op->enc.reset();
     if ((op->opcode == OSD_OP_WRITE || op->opcode == OSD_OP_DELETE) && !(op->flags & OSD_OP_IGNORE_READONLY))
     {
-        if (!searched)
-        {
-            icache = inode_cache_get(op->inode);
-            searched = true;
-        }
-        if (icache && icache->readonly)
+        auto ino_it = st_cli.inode_config.find(op->inode);
+        if (ino_it != st_cli.inode_config.end() && ino_it->second.readonly)
         {
             op->retval = -EROFS;
             auto cb = std::move(op->callback);
@@ -1013,49 +972,43 @@ bool cluster_client_t::check_rw(cluster_op_t *op)
     op->deoptimise_snapshot = false;
     if (enable_writeback && (op->opcode == OSD_OP_READ || op->opcode == OSD_OP_READ_BITMAP || op->opcode == OSD_OP_READ_CHAIN_BITMAP))
     {
-        if (!searched)
+        auto ino_it = st_cli.inode_config.find(op->inode);
+        if (ino_it != st_cli.inode_config.end())
         {
-            icache = inode_cache_get(op->inode);
-            searched = true;
-        }
-        if (icache)
-        {
-            for (auto & parent: icache->chain)
+            int chain_size = 0;
+            while (ino_it != st_cli.inode_config.end() && ino_it->second.parent_id)
             {
-                if (INODE_POOL(parent) == INODE_POOL(op->inode) && wb->has_inode(parent))
+                // Check for loops - FIXME check it in etcd_state_client
+                if (ino_it->second.parent_id == op->inode ||
+                    chain_size > st_cli.inode_config.size())
+                {
+                    op->retval = -EINVAL;
+                    auto cb = std::move(op->callback);
+                    cb(op);
+                    return false;
+                }
+                if (INODE_POOL(ino_it->second.parent_id) == INODE_POOL(ino_it->first) &&
+                    wb->has_inode(ino_it->second.parent_id))
                 {
                     // Deoptimise reads - we have dirty data for one of the parent layer(s).
                     op->deoptimise_snapshot = true;
                     break;
                 }
+                chain_size++;
+                ino_it = st_cli.inode_config.find(ino_it->second.parent_id);
             }
         }
     }
-    if (icache && icache->err_code)
-    {
-        if (icache->err_code == EPERM)
-        {
-            op->retval = -EPERM;
-            auto cb = std::move(op->callback);
-            cb(op);
-            return false;
-        }
-        else if (icache->err_code == EAGAIN)
-        {
-            key_wait_ops.push_back(op);
-            return false;
-        }
-    }
     return true;
 }
 
 void cluster_client_t::execute_raw(osd_num_t osd_num, osd_op_t *op)
 {
-    auto fd_it = msgr.osd_peer_fds.find(osd_num);
-    if (fd_it != msgr.osd_peer_fds.end())
+    auto peer_it = msgr.osd_peers.find(osd_num);
+    if (peer_it != msgr.osd_peers.end())
     {
         op->op_type = OSD_OP_OUT;
-        op->peer_fd = fd_it->second;
+        op->client_id = peer_it->second->client_id;
         msgr.outbox_push(op);
     }
     else
@@ -1166,35 +1119,40 @@ resume_2:
         // Finished successfully
         // Even if the PG count has changed in meanwhile we treat it as success
         // because if some operations were invalid for the new PG count we'd get errors
+        if (op->opcode == OSD_OP_READ || op->opcode == OSD_OP_READ_BITMAP || op->opcode == OSD_OP_READ_CHAIN_BITMAP)
+        {
+            // Copy part bitmaps only after finishing all part reads
+            for (auto & part: op->parts)
+                if ((part.flags & (PART_SENT|PART_DONE|PART_VALID)) == (PART_SENT|PART_DONE|PART_VALID))
+                    copy_part_bitmap(op, &part);
+        }
         if (op->opcode == OSD_OP_READ || op->opcode == OSD_OP_READ_CHAIN_BITMAP)
         {
-            uint64_t next_inode = 0;
-            auto icache = inode_cache_get(op->cur_inode);
-            if (icache)
+            // Check parent inode
+            auto ino_it = st_cli.inode_config.find(op->cur_inode);
+            // Skip parents from the same pool
+            int skipped = 0;
+            while (!op->deoptimise_snapshot &&
+                ino_it != st_cli.inode_config.end() && ino_it->second.parent_id &&
+                INODE_POOL(ino_it->second.parent_id) == INODE_POOL(op->cur_inode))
             {
-                if (icache->has_parent_loop)
+                // Check for loops - FIXME check it in etcd_state_client
+                if (ino_it->second.parent_id == op->inode ||
+                    skipped > st_cli.inode_config.size())
                 {
                     op->retval = -EINVAL;
                     erase_op(op);
                     return 1;
                 }
-                if (op->deoptimise_snapshot)
-                {
-                    if (icache->chain.size() > 1)
-                        next_inode = icache->chain[1];
-                }
-                else
-                {
-                    if (icache->other_pool_parent_id)
-                        next_inode = icache->other_pool_parent_id;
-                }
+                skipped++;
+                ino_it = st_cli.inode_config.find(ino_it->second.parent_id);
             }
-            if (next_inode)
+            if (ino_it != st_cli.inode_config.end() &&
+                ino_it->second.parent_id &&
+                ino_it->second.parent_id != op->inode)
             {
                 // Continue reading from the parent inode
-                icache = inode_cache_get(next_inode);
-                op->cur_inode = next_inode;
-                op->enc = (icache && icache->op_enc ? std::shared_ptr<osd_op_enc_t>(icache, icache->op_enc) : nullptr);
+                op->cur_inode = ino_it->second.parent_id;
                 op->parts.clear();
                 op->done_count = 0;
                 goto resume_0;
@@ -1213,7 +1171,7 @@ resume_2:
         erase_op(op);
         return 1;
     }
-    else if (op->retval != 0 && !(op->flags & OP_FLUSH_BUFFER) &&
+    else if (op->retval != 0 && op->opcode != OSD_OP_SYNC && !(op->flags & OP_FLUSH_BUFFER) &&
         op->retval != -EPIPE && (op->retval != -EIO || !client_eio_retry_interval) && (op->retval != -ENOSPC || !client_retry_enospc))
     {
         // Fatal error (neither -EPIPE, -EIO nor -ENOSPC)
@@ -1301,11 +1259,7 @@ void cluster_client_t::slice_rw(cluster_op_t *op)
         // Allocate memory for the bitmap
         unsigned object_bitmap_size = ((op->len / pool_cfg.bitmap_granularity + 7) / 8);
         object_bitmap_size = (object_bitmap_size < 8 ? 8 : object_bitmap_size);
-        unsigned bitmap_mem = object_bitmap_size +
-            op->parts.size() * pg_data_size *
-            (pool_cfg.data_block_size / pool_cfg.bitmap_granularity / 8
-            // read chain info - 1 byte per block
-            + (op->enc ? op->len/pool_cfg.bitmap_granularity : 0));
+        unsigned bitmap_mem = object_bitmap_size + (pool_cfg.data_block_size / pool_cfg.bitmap_granularity / 8 * pg_data_size) * op->parts.size();
         if (!op->bitmap_buf || op->bitmap_buf_size < bitmap_mem)
         {
             op->bitmap_buf = realloc_or_die(op->bitmap_buf, bitmap_mem);
@@ -1454,15 +1408,15 @@ int cluster_client_t::try_send(cluster_op_t *op, int i, std::function<void(osd_o
                 primary_osd = nearest_osd;
         }
         part->osd_num = primary_osd;
-        auto peer_it = msgr.osd_peer_fds.find(primary_osd);
-        if (peer_it != msgr.osd_peer_fds.end())
+        auto peer_it = msgr.osd_peers.find(primary_osd);
+        if (peer_it != msgr.osd_peers.end())
         {
-            int peer_fd = peer_it->second;
+            osd_client_t *cl = peer_it->second;
             part->flags |= PART_SENT|PART_VALID;
             op->inflight_count++;
-            uint32_t pg_data_size = (pool_cfg.scheme == POOL_SCHEME_REPLICATED ? 1 : pool_cfg.pg_size-pool_cfg.parity_chunks);
-            uint64_t pg_bitmap_size = pg_data_size * (pool_cfg.data_block_size / pool_cfg.bitmap_granularity / 8
-                + (op->opcode == OSD_OP_READ && op->enc ? pool_cfg.data_block_size/pool_cfg.bitmap_granularity : 0));
+            uint64_t pg_bitmap_size = (pool_cfg.data_block_size / pool_cfg.bitmap_granularity / 8) * (
+                pool_cfg.scheme == POOL_SCHEME_REPLICATED ? 1 : pool_cfg.pg_size-pool_cfg.parity_chunks
+            );
             uint64_t meta_rev = 0;
             if (op->opcode != OSD_OP_READ_BITMAP && op->opcode != OSD_OP_DELETE && !op->deoptimise_snapshot)
             {
@@ -1472,7 +1426,7 @@ int cluster_client_t::try_send(cluster_op_t *op, int i, std::function<void(osd_o
             }
             part->op = (osd_op_t){
                 .op_type = OSD_OP_OUT,
-                .peer_fd = peer_fd,
+                .client_id = cl->client_id,
                 .req = { .rw = {
                     .header = {
                         .magic = SECONDARY_OSD_OP_MAGIC,
@@ -1481,7 +1435,6 @@ int cluster_client_t::try_send(cluster_op_t *op, int i, std::function<void(osd_o
                     .inode = op->cur_inode,
                     .offset = part->offset,
                     .len = part->len,
-                    .flags = op->opcode == OSD_OP_READ && op->enc && !op->deoptimise_snapshot ? OSD_OP_RETURN_CHAIN : 0,
                     .meta_revision = meta_rev,
                     .version = op->opcode == OSD_OP_WRITE || op->opcode == OSD_OP_DELETE ? op->version : 0,
                 } },
@@ -1489,7 +1442,6 @@ int cluster_client_t::try_send(cluster_op_t *op, int i, std::function<void(osd_o
                     ? (uint8_t*)op->part_bitmaps + pg_bitmap_size*i : NULL),
                 .bitmap_len = (unsigned)(op->opcode == OSD_OP_READ || op->opcode == OSD_OP_READ_BITMAP || op->opcode == OSD_OP_READ_CHAIN_BITMAP
                     ? pg_bitmap_size : 0),
-                .enc = op->enc,
                 .callback = cb ? cb : [this, part](osd_op_t *op_part)
                 {
                     handle_op_part(part);
@@ -1523,8 +1475,8 @@ int cluster_client_t::continue_sync(cluster_op_t *op)
     for (auto do_it = dirty_osds.begin(); do_it != dirty_osds.end(); )
     {
         osd_num_t sync_osd = *do_it;
-        auto peer_it = msgr.osd_peer_fds.find(sync_osd);
-        if (peer_it == msgr.osd_peer_fds.end())
+        auto peer_it = msgr.osd_peers.find(sync_osd);
+        if (peer_it == msgr.osd_peers.end())
             dirty_osds.erase(do_it++);
         else
             do_it++;
@@ -1577,12 +1529,12 @@ resume_1:
 
 void cluster_client_t::send_sync(cluster_op_t *op, cluster_op_part_t *part)
 {
-    auto peer_fd = msgr.osd_peer_fds.at(part->osd_num);
+    osd_client_t *cl = msgr.osd_peers.at(part->osd_num);
     part->flags |= PART_SENT;
     op->inflight_count++;
     part->op = (osd_op_t){
         .op_type = OSD_OP_OUT,
-        .peer_fd = peer_fd,
+        .client_id = cl->client_id,
         .req = {
             .hdr = {
                 .magic = SECONDARY_OSD_OP_MAGIC,
@@ -1622,10 +1574,10 @@ void cluster_client_t::handle_op_part(cluster_op_part_t *part)
             // Error priority: EIO > ENOSPC > ETIMEDOUT > EPIPE
             op->retval = part->op.reply.hdr.retval;
         }
-        int stop_fd = -1;
+        uint64_t stop_client_id = 0;
         if (op->retval != -EINTR && op->retval != -EIO && op->retval != -ENOSPC)
         {
-            stop_fd = part->op.peer_fd;
+            stop_client_id = part->op.client_id;
             if (op->retval != -EPIPE || log_level > 0)
             {
                 fprintf(
@@ -1652,9 +1604,9 @@ void cluster_client_t::handle_op_part(cluster_op_part_t *part)
             op->retry_after = op->retval != -EPIPE ? client_eio_retry_interval : client_retry_interval;
         }
         reset_retry_timer(op->retry_after);
-        if (stop_fd >= 0)
+        if (stop_client_id)
         {
-            msgr.stop_client(stop_fd);
+            msgr.stop_client(stop_client_id);
         }
         op->inflight_count--;
         if (op->inflight_count == 0 && !op->retry_after)
@@ -1685,13 +1637,6 @@ void cluster_client_t::handle_op_part(cluster_op_part_t *part)
         }
         if (op->inflight_count == 0 && !op->retry_after)
         {
-            // Copy part bitmaps only after finishing all part reads
-            if (op->opcode == OSD_OP_READ || op->opcode == OSD_OP_READ_BITMAP || op->opcode == OSD_OP_READ_CHAIN_BITMAP)
-            {
-                for (auto & part: op->parts)
-                    if (part.flags == (PART_SENT|PART_VALID|PART_DONE))
-                        copy_part_bitmap(op, &part);
-            }
             if (op->opcode == OSD_OP_SYNC)
                 continue_sync(op);
             else

src/client/cluster_client.h

@@ -5,7 +5,6 @@
 
 #include "messenger.h"
 #include "etcd_state_client.h"
-#include "../util/robin_hood.h"
 
 #define DEFAULT_CLIENT_MAX_DIRTY_BYTES 32*1024*1024
 #define DEFAULT_CLIENT_MAX_DIRTY_OPS 1024
@@ -72,7 +71,6 @@ protected:
     cluster_op_t *prev = NULL, *next = NULL;
     int prev_wait = 0;
     uint64_t flush_id = 0;
-    std::shared_ptr<osd_op_enc_t> enc;
     friend class cluster_client_t;
     friend class writeback_cache_t;
 };
@@ -82,36 +80,14 @@ struct inode_list_osd_t;
 struct inode_list_pg_t;
 class writeback_cache_t;
 
-struct inode_cache_t
-{
-    std::vector<inode_t> chain;
-    uint8_t *key_data = NULL;
-    osd_op_enc_t *op_enc = NULL;
-    bool readonly = false;
-    bool has_parent_loop = false;
-    inode_t other_pool_parent_id = 0;
-    int err_code = 0;
-
-    ~inode_cache_t();
-};
-
-struct vault_load_key_t
-{
-    int key_state = 0;
-    std::string key;
-};
-
 // FIXME: Split into public and private interfaces
 class __attribute__((visibility("default"))) cluster_client_t
 {
-#ifdef __MOCK__
-public:
-#endif
     timerfd_manager_t *tfd = NULL;
     ring_loop_t *ringloop = NULL;
 
-    // config:
-
+    std::map<pool_id_t, uint64_t> pg_counts;
+    std::map<pool_pg_num_t, osd_num_t> pg_primary;
     // client_max_dirty_* is actually "max unsynced", for the case when immediate_commit is off
     uint64_t client_max_dirty_bytes = 0;
     uint64_t client_max_dirty_ops = 0;
@@ -123,23 +99,12 @@ public:
     uint64_t client_max_writeback_iodepth = 0;
     std::string conf_hostname;
 
-    std::string vault_url;
-    std::string vault_client_cert;
-    std::string vault_client_key;
-    std::string vault_ca;
-    std::string vault_secret_api_path;
-    uint64_t vault_timeout_ms = 0;
-    uint64_t vault_error_timeout_sec = 0;
-    uint64_t vault_refresh_leeway_sec = 0;
-
     int log_level = 0;
     int client_retry_interval = 50; // ms
     int client_eio_retry_interval = 1000; // ms
     bool client_retry_enospc = true;
     int client_wait_up_timeout = 16; // sec (for listings)
 
-    // state:
-
     std::string client_hostname;
     std::map<std::string, int> self_tree_metrics;
     std::map<osd_num_t, int> osd_tree_metrics;
@@ -147,7 +112,6 @@ public:
     int retry_timeout_id = -1;
     int retry_timeout_duration = 0;
     std::vector<cluster_op_t*> offline_ops;
-    std::vector<cluster_op_t*> key_wait_ops;
     cluster_op_t *op_queue_head = NULL, *op_queue_tail = NULL;
     writeback_cache_t *wb = NULL;
     std::set<osd_num_t> dirty_osds;
@@ -156,22 +120,7 @@ public:
     void *scrap_buffer = NULL;
     unsigned scrap_buffer_size = 0;
 
-    // inodes require some extra state for read/write, it's stored here.
-    // moreover, robin_hood access is slightly faster than std::map :)
-    robin_hood::unordered_flat_map<inode_t, std::shared_ptr<inode_cache_t>> inode_cache;
-    std::set<std::pair<inode_t, inode_t>> inode_cache_children;
-
-    http_context_t *vault_http_ctx = NULL;
-    http_co_t *vault_http_cli = NULL;
-    bool vault_loading = false;
-    std::string vault_token;
-    bool vault_auth_error = false;
-    timespec vault_token_expire = {};
-    std::vector<std::string> vault_key_load_queue;
-    std::map<std::string, vault_load_key_t> vault_keys;
-
     bool pgs_loaded = false;
-    std::map<pool_id_t, uint64_t> pg_counts;
     ring_consumer_t consumer;
     std::vector<std::function<void(void)>> on_ready_hooks;
     int list_retry_timeout_id = -1;
@@ -203,22 +152,9 @@ public:
     void list_inode(inode_t inode, uint64_t min_offset, uint64_t max_offset, int max_parallel_pgs, std::function<void(
         int status, int pgs_left, pg_num_t pg_num, std::set<object_id>&& objects)> pg_callback);
 
-    //inline uint32_t get_bs_bitmap_granularity() { return st_cli.global_bitmap_granularity; }
-    //inline uint64_t get_bs_block_size() { return st_cli.global_block_size; }
-
-#ifndef __MOCK__
 protected:
-#endif
     void continue_ops(int time_passed = 0);
 
-    std::shared_ptr<inode_cache_t> inode_cache_get(inode_t ino);
-    void vault_parse_config();
-    bool vault_check_token();
-    void vault_load_keys();
-    void vault_destroy();
-    void vault_parse_secret(const std::string & key_id, const std::string & err, json11::Json data);
-
-protected:
     bool affects_osd(uint64_t inode, uint64_t offset, uint64_t len, osd_num_t osd);
     bool affects_pg(uint64_t inode, uint64_t offset, uint64_t len, pool_id_t pool_id, pg_num_t pg_num);
 
@@ -228,7 +164,6 @@ protected:
     void on_change_pg_state_hook(pool_id_t pool_id, pg_num_t pg_num, osd_num_t prev_primary);
     void on_change_osd_state_hook(uint64_t peer_osd);
     void on_change_node_placement_hook();
-    void on_change_inode_hook(uint64_t inode, bool removed);
 
     void execute_internal(cluster_op_t *op);
     void execute_cas(cluster_op_t *op);
@@ -245,7 +180,6 @@ protected:
     void erase_op(cluster_op_t *op);
     void calc_wait(cluster_op_t *op);
     void inc_wait(uint64_t opcode, uint64_t flags, cluster_op_t *next, int inc);
-
     void continue_lists();
     bool continue_listing(inode_list_t *lst);
     bool restart_listing(inode_list_t* lst);
@@ -261,4 +195,5 @@ protected:
     osd_num_t select_nearest_osd(const std::vector<osd_num_t> & osds);
 
     friend class writeback_cache_t;
+    friend class cluster_client_test_t;
 };

src/client/cluster_client_icache.cpp

@@ -1,367 +0,0 @@
-// Copyright (c) Vitaliy Filippov, 2019+
-// License: VNPL-1.1 or GNU GPL-2.0+ (see README.md for details)
-
-#include <stdexcept>
-#include <assert.h>
-#include "cluster_client_impl.h"
-#include "http_client.h"
-#include "str_util.h"
-
-#define VAULT_KEY_NOT_LOADED 0
-#define VAULT_KEY_LOADING 1
-#define VAULT_KEY_LOADED 2
-#define VAULT_KEY_ERROR 3
-
-inode_cache_t::~inode_cache_t()
-{
-    if (key_data)
-    {
-        free(key_data);
-        key_data = NULL;
-        op_enc = NULL;
-    }
-}
-
-void cluster_client_t::vault_destroy()
-{
-    if (vault_http_ctx)
-    {
-#ifndef __MOCK__
-        http_destroy(vault_http_cli);
-        http_context_destroy(vault_http_ctx);
-        vault_http_cli = NULL;
-        vault_http_ctx = NULL;
-#endif
-    }
-}
-
-void cluster_client_t::vault_parse_config()
-{
-    vault_url = config["vault_url"].string_value();
-    vault_client_cert = config["vault_client_cert"].string_value();
-    vault_client_key = config["vault_client_key"].string_value();
-    vault_ca = config["vault_ca"].string_value();
-    vault_secret_api_path = "/v1/secret/";
-    if (config["vault_secret_api_path"].is_string())
-        vault_secret_api_path = config["vault_secret_api_path"].string_value();
-    vault_timeout_ms = config["vault_timeout_ms"].uint64_value();
-    if (!vault_timeout_ms)
-        vault_timeout_ms = 5000;
-    vault_error_timeout_sec = config["vault_error_timeout_sec"].uint64_value();
-    if (!vault_error_timeout_sec)
-        vault_error_timeout_sec = 60;
-    vault_refresh_leeway_sec = config["vault_refresh_leeway_sec"].uint64_value();
-    if (!vault_refresh_leeway_sec)
-        vault_refresh_leeway_sec = 60;
-}
-
-// FIXME: Rework client API by adding open/close and cache inode information in the "FD" (maybe)
-void cluster_client_t::on_change_inode_hook(uint64_t inode, bool removed)
-{
-    std::vector<inode_t> children = { inode };
-    for (size_t i = 0; i < children.size(); i++)
-    {
-        auto it = inode_cache_children.lower_bound(std::make_pair(children[i], (inode_t)0));
-        while (it != inode_cache_children.end() && it->first == children[i])
-        {
-            children.push_back(it->second);
-            it++;
-        }
-    }
-    for (auto & inode: children)
-    {
-        auto it = inode_cache.find(inode);
-        if (it != inode_cache.end())
-        {
-            auto icache = it->second;
-            for (auto & parent: icache->chain)
-            {
-                inode_cache_children.erase(std::make_pair(parent, inode));
-            }
-            inode_cache.erase(it);
-        }
-    }
-}
-
-std::shared_ptr<inode_cache_t> cluster_client_t::inode_cache_get(inode_t ino)
-{
-    auto icache_it = inode_cache.find(ino);
-    if (icache_it != inode_cache.end())
-    {
-        return icache_it->second;
-    }
-    // Fill inode cache
-    auto ino_it = st_cli.inode_config.find(ino);
-    if (ino_it == st_cli.inode_config.end())
-    {
-        inode_cache[ino] = NULL;
-        return NULL;
-    }
-    auto pool_it = st_cli.pool_config.find(INODE_POOL(ino));
-    if (pool_it == st_cli.pool_config.end())
-    {
-        inode_cache[ino] = NULL;
-        return NULL;
-    }
-    auto & inode_cfg = ino_it->second;
-    auto & pool_cfg = pool_it->second;
-    std::shared_ptr<inode_cache_t> icache = std::make_shared<inode_cache_t>();
-    icache->readonly = inode_cfg.readonly;
-    icache->chain.push_back(ino);
-    std::vector<inode_config_t*> chain_cfg;
-    // FIXME: Allow unencrypted read & write when all chain is encrypted with the same key
-    int enc_key_count = !inode_cfg.enc_key.empty() ? 1 : 0;
-    if (inode_cfg.parent_id)
-    {
-        // Check for loops and cache the chain
-        robin_hood::unordered_flat_set<inode_t> seen;
-        seen.insert(ino);
-        uint64_t parent_id = inode_cfg.parent_id;
-        while (parent_id)
-        {
-            if (seen.find(parent_id) != seen.end())
-            {
-                icache->has_parent_loop = true;
-                break;
-            }
-            seen.insert(parent_id);
-            ino_it = st_cli.inode_config.find(parent_id);
-            if (INODE_POOL(parent_id) == INODE_POOL(ino))
-            {
-                icache->chain.push_back(parent_id);
-                if (ino_it == st_cli.inode_config.end())
-                    chain_cfg.push_back(NULL);
-                else
-                {
-                    chain_cfg.push_back(&ino_it->second);
-                    if (!ino_it->second.enc_key.empty())
-                        enc_key_count++;
-                }
-            }
-            else if (!icache->other_pool_parent_id)
-                icache->other_pool_parent_id = parent_id;
-            if (ino_it == st_cli.inode_config.end())
-                break;
-            parent_id = ino_it->second.parent_id;
-        }
-    }
-    // Check external keys and wait for loading, if required
-    if (enc_key_count)
-    {
-        for (size_t i = 0; i <= chain_cfg.size(); i++)
-        {
-            inode_config_t *cfg = !i ? &inode_cfg : chain_cfg[i-1];
-            if (cfg && cfg->enc_key.substr(0, strlen(VAULT_KEY_PREFIX)) == VAULT_KEY_PREFIX)
-            {
-                auto & ik = vault_keys[inode_cfg.enc_key];
-                if (ik.key_state == VAULT_KEY_ERROR || vault_url.empty())
-                {
-                    icache->err_code = EPERM;
-                    enc_key_count = 0;
-                }
-                else if (ik.key_state == VAULT_KEY_NOT_LOADED)
-                {
-                    ik.key_state = VAULT_KEY_LOADING;
-                    vault_key_load_queue.push_back(inode_cfg.enc_key);
-                    vault_load_keys();
-                    icache->err_code = EAGAIN;
-                    enc_key_count = 0;
-                }
-                else if (ik.key_state == VAULT_KEY_LOADING)
-                {
-                    icache->err_code = EAGAIN;
-                    enc_key_count = 0;
-                }
-                else
-                {
-                    assert(ik.key_state == VAULT_KEY_LOADED);
-                }
-            }
-        }
-    }
-    // Generate encryption key chain, if applicable
-    if (enc_key_count)
-    {
-        uint8_t *key_data = (uint8_t*)malloc_or_die(
-            AES_256_XTS_KEY_SIZE * enc_key_count +
-            sizeof(uint8_t*) * icache->chain.size() +
-            sizeof(osd_op_enc_t)
-        );
-        uint8_t **keys = (uint8_t**)(key_data + AES_256_XTS_KEY_SIZE * enc_key_count);
-        osd_op_enc_t *enc = (osd_op_enc_t*)((uint8_t*)keys + sizeof(uint8_t*)*icache->chain.size());
-        size_t key_pos = 0;
-        for (size_t i = 0; i <= chain_cfg.size(); i++)
-        {
-            inode_config_t *cfg = !i ? &inode_cfg : chain_cfg[i-1];
-            if (cfg && !cfg->enc_key.empty())
-            {
-                const auto & key = cfg->enc_key.substr(0, strlen(VAULT_KEY_PREFIX)) == VAULT_KEY_PREFIX
-                    ? vault_keys.at(cfg->enc_key).key
-                    : cfg->enc_key;
-                assert(key_pos < AES_256_XTS_KEY_SIZE * enc_key_count);
-                assert(key.size() == 2*AES_256_XTS_KEY_SIZE);
-                keys[i] = key_data + key_pos;
-                fromhexstr(key, AES_256_XTS_KEY_SIZE, key_data + key_pos);
-                key_pos += AES_256_XTS_KEY_SIZE;
-            }
-            else
-                keys[i] = NULL;
-        }
-        enc->key_chain = keys;
-        enc->chain_size = icache->chain.size();
-        enc->read_chain_bitmap_pos = pool_cfg.data_block_size/pool_cfg.bitmap_granularity/8;
-        enc->bitmap_granularity = pool_cfg.bitmap_granularity;
-        icache->key_data = key_data;
-        icache->op_enc = enc;
-    }
-    inode_cache[ino] = icache;
-    for (auto & parent: icache->chain)
-    {
-        if (parent != ino)
-            inode_cache_children.insert(std::make_pair(parent, ino));
-    }
-    return icache;
-}
-
-#ifndef __MOCK__
-bool cluster_client_t::vault_check_token()
-{
-    timespec now;
-    clock_gettime(CLOCK_REALTIME, &now);
-    if (!vault_token_expire.tv_sec || vault_token_expire.tv_sec < now.tv_sec)
-    {
-        vault_loading = true;
-        http_json_post(
-            vault_http_cli, vault_url+"/v1/auth/cert/login", json11::Json::object{}, "",
-            (http_options_t){ .timeout = (int)vault_timeout_ms, .keepalive = true },
-            [this](http_message_t *response)
-            {
-                clock_gettime(CLOCK_REALTIME, &vault_token_expire);
-                vault_loading = false;
-                std::string err;
-                json11::Json data;
-                response->parse_json_response(err, data);
-                if (err != "")
-                {
-                    vault_token_expire.tv_sec += vault_error_timeout_sec;
-                    fprintf(stderr, "Vault request failed: %s\n", err.c_str());
-                }
-                else
-                {
-                    uint64_t ttl = data["auth"]["lease_duration"].uint64_value();
-                    vault_token = data["auth"]["client_token"].string_value();
-                    if (vault_token.empty() || !ttl)
-                    {
-                        vault_token_expire.tv_sec += vault_error_timeout_sec;
-                        fprintf(stderr, "No token or lease_duration in Vault response: %s\n", data.dump().c_str());
-                    }
-                    else
-                    {
-                        if (ttl < vault_refresh_leeway_sec)
-                            vault_token_expire.tv_sec += ttl/2;
-                        else
-                            vault_token_expire.tv_sec += ttl - vault_refresh_leeway_sec;
-                    }
-                }
-                vault_load_keys();
-            }
-        );
-        return false;
-    }
-    if (vault_token.empty())
-    {
-        // Auth error happened, mark all loads as failed
-        for (auto & key_id: vault_key_load_queue)
-        {
-            auto & k = vault_keys[key_id];
-            k.key_state = VAULT_KEY_ERROR;
-        }
-        vault_key_load_queue.clear();
-        auto ops = std::move(key_wait_ops);
-        for (cluster_op_t *op: ops)
-            inode_cache.erase(op->inode);
-        for (cluster_op_t *op: ops)
-            execute_internal(op);
-        return false;
-    }
-    return true;
-}
-#endif
-
-void cluster_client_t::vault_load_keys()
-{
-    if (vault_loading || !vault_key_load_queue.size())
-    {
-        return;
-    }
-#ifdef __MOCK__
-    vault_loading = true;
-#else
-    if (!vault_http_ctx)
-    {
-        std::string error;
-        vault_http_ctx = http_context_init(tfd, vault_client_cert, vault_client_key, vault_ca, true, error);
-        if (!vault_http_ctx)
-        {
-            fprintf(stderr, "Failed to initialize HTTP context for Vault: %s\n", error.c_str());
-            exit(1);
-        }
-        vault_http_cli = http_init(vault_http_ctx);
-    }
-    if (!vault_check_token())
-    {
-        return;
-    }
-    std::string key_id = vault_key_load_queue[0];
-    vault_key_load_queue.erase(vault_key_load_queue.begin());
-    vault_loading = true;
-    http_get(
-        vault_http_cli, vault_url+vault_secret_api_path+key_id.substr(strlen(VAULT_KEY_PREFIX)), "X-Vault-Token: "+vault_token+"\r\n",
-        (http_options_t){ .timeout = (int)vault_timeout_ms, .keepalive = true },
-        [this, key_id](http_message_t *response)
-        {
-            vault_loading = false;
-            std::string err;
-            json11::Json data;
-            response->parse_json_response(err, data);
-            vault_parse_secret(key_id, err, data);
-        }
-    );
-#endif
-}
-
-void cluster_client_t::vault_parse_secret(const std::string & key_id, const std::string & err, json11::Json data)
-{
-    vault_loading = false;
-    auto & k = vault_keys[key_id];
-    if (err != "")
-    {
-        k.key_state = VAULT_KEY_ERROR;
-        fprintf(stderr, "Vault %s%s%s request failed: %s\n", vault_url.c_str(),
-            vault_secret_api_path.c_str(), key_id.c_str()+strlen(VAULT_KEY_PREFIX), err.c_str());
-    }
-    else
-    {
-        auto hexkey = data["data"]["key"].string_value();
-        if (hexkey.empty() || !ishexstr(hexkey) || hexkey.size() != 2*AES_256_XTS_KEY_SIZE)
-        {
-            k.key_state = VAULT_KEY_ERROR;
-            fprintf(stderr, "Vault /v1/secret/%s request failed: 'key' is empty or has invalid format\n", key_id.c_str());
-        }
-        else
-        {
-            k.key_state = VAULT_KEY_LOADED;
-            k.key = hexkey;
-        }
-    }
-    if (vault_key_load_queue.empty())
-    {
-        auto ops = std::move(key_wait_ops);
-        for (cluster_op_t *op: ops)
-            inode_cache.erase(op->inode);
-        for (cluster_op_t *op: ops)
-            execute_internal(op);
-    }
-    else
-        vault_load_keys();
-}

src/client/cluster_client_list.cpp

@@ -295,7 +295,7 @@ int cluster_client_t::start_pg_listing(inode_list_pg_t *pg)
     bool conn = true;
     for (osd_num_t peer_osd: all_peers)
     {
-        if (msgr.osd_peer_fds.find(peer_osd) == msgr.osd_peer_fds.end())
+        if (msgr.osd_peers.find(peer_osd) == msgr.osd_peers.end())
         {
             // Initiate connection
             if (st_cli.peer_states[peer_osd].is_null())
@@ -340,7 +340,7 @@ void cluster_client_t::send_list(inode_list_osd_t *cur_list)
     osd_op_t *op = new osd_op_t();
     op->op_type = OSD_OP_OUT;
     // Already checked that it exists above, but anyway
-    op->peer_fd = msgr.osd_peer_fds.at(cur_list->osd_num);
+    op->client_id = msgr.osd_peers.at(cur_list->osd_num)->client_id;
     op->req = (osd_any_op_t){
         .sec_list = {
             .header = {

src/client/cluster_client_wb.cpp

@@ -88,6 +88,11 @@ void writeback_cache_t::copy_write(cluster_op_t *op, int state, uint64_t new_flu
     // ...or just save it for writeback if write buffering is enabled
     if (op->len == 0)
     {
+        // FIXME: OSD_OP_DELETEs are currently only sent by vitastor-cli rm/rm-data and
+        // actually have len=0, because delete is actually a delete of the full object
+        // containing the requested offset, not a "punch hole" operation. But here, writeback
+        // cache assumes it IS a "punch hole" operation. I should select one of these
+        // approaches and fix everything accordingly when I decide to implement TRIM.
         return;
     }
     auto dirty_it = find_dirty(op->inode, op->offset);
@@ -244,12 +249,13 @@ void writeback_cache_t::copy_write(cluster_op_t *op, int state, uint64_t new_flu
             writeback_queue_size--;
         }
     }
-    if (!is_del)
+    if (!is_del && op->len > 0)
     {
         uint64_t pos = 0, len = op->len, iov_idx = 0;
-        while (len > 0 && iov_idx < op->iov.count)
+        while (iov_idx < op->iov.count)
         {
             auto & iov = op->iov.buf[iov_idx];
+            assert(pos + iov.iov_len <= len);
             memcpy(buf + pos, iov.iov_base, iov.iov_len);
             pos += iov.iov_len;
             iov_idx++;
@@ -443,7 +449,7 @@ void writeback_cache_t::start_writebacks(cluster_client_t *cli, int count)
         started++;
         assert(writeback_queue_size > 0);
         writeback_queue_size--;
-        writeback_bytes -= off - from_it->first.stripe;
+        writeback_bytes -= (is_del ? 0 : off - from_it->first.stripe);
         assert(writeback_queue_size > 0 || !writeback_bytes);
         flush_buffers(cli, from_it, to_it);
     }

src/client/etcd_state_client.cpp

@@ -1,10 +1,7 @@
 // Copyright (c) Vitaliy Filippov, 2019+
 // License: VNPL-1.1 or GNU GPL-2.0+ (see README.md for details)
 
-#include <assert.h>
-#include "malloc_or_die.h"
 #include "osd_ops.h"
-#include "msgr_op.h"
 #include "pg_states.h"
 #include "etcd_state_client.h"
 #ifndef __MOCK__
@@ -25,19 +22,14 @@ etcd_state_client_t::~etcd_state_client_t()
     stop_ws_keepalive();
     if (etcd_watch_ws)
     {
-        http_destroy(etcd_watch_ws);
+        http_close(etcd_watch_ws);
         etcd_watch_ws = NULL;
     }
     if (keepalive_client)
     {
-        http_destroy(keepalive_client);
+        http_close(keepalive_client);
         keepalive_client = NULL;
     }
-    if (http_ctx)
-    {
-        http_context_destroy(http_ctx);
-        http_ctx = NULL;
-    }
 #endif
     if (load_pgs_timer_id >= 0)
     {
@@ -80,51 +72,55 @@ std::vector<std::string> etcd_state_client_t::get_addresses()
     return addrs;
 }
 
-http_context_t *etcd_state_client_t::get_http_ctx()
-{
-    if (!http_ctx)
-    {
-        std::string error;
-        http_ctx = http_context_init(tfd, etcd_client_cert, etcd_client_key, etcd_ca, true, error);
-        if (!http_ctx)
-        {
-            fprintf(stderr, "Failed to initialize HTTP context: %s\n", error.c_str());
-            exit(1);
-        }
-    }
-    return http_ctx;
-}
-
-void etcd_state_client_t::etcd_call_oneshot(const std::string & etcd_url, const std::string & api, json11::Json payload,
+void etcd_state_client_t::etcd_call_oneshot(std::string etcd_address, std::string api, json11::Json payload,
     int timeout, std::function<void(std::string, json11::Json)> callback)
 {
-    auto http_cli = http_init(get_http_ctx());
-    http_json_post(http_cli, etcd_url+api, payload, "", { .timeout = timeout }, [http_cli, callback](http_message_t *response)
+    std::string etcd_api_path;
+    int pos = etcd_address.find('/');
+    if (pos >= 0)
+    {
+        etcd_api_path = etcd_address.substr(pos);
+        etcd_address = etcd_address.substr(0, pos);
+    }
+    std::string req = payload.dump();
+    req = "POST "+etcd_api_path+api+" HTTP/1.1\r\n"
+        "Host: "+etcd_address+"\r\n"
+        "Content-Type: application/json\r\n"
+        "Content-Length: "+std::to_string(req.size())+"\r\n"
+        "Connection: close\r\n"
+        "\r\n"+req;
+    auto http_cli = http_init(tfd);
+    auto cb = [http_cli, callback](const http_response_t *response)
     {
         std::string err;
         json11::Json data;
         response->parse_json_response(err, data);
         callback(err, data);
-        http_destroy(http_cli);
-    });
+        http_close(http_cli);
+    };
+    http_request(http_cli, etcd_address, req, { .timeout = timeout }, cb);
 }
 
-void etcd_state_client_t::etcd_call(const std::string & api, json11::Json payload, int timeout,
+void etcd_state_client_t::etcd_call(std::string api, json11::Json payload, int timeout,
     int retries, int interval, std::function<void(std::string, json11::Json)> callback)
 {
-    pick_next_etcd([=]()
+    if (!etcd_addresses.size() && !etcd_local.size())
     {
-        etcd_call_selected(api, payload, timeout, retries, interval, callback);
-    });
-}
-
-void etcd_state_client_t::etcd_call_selected(const std::string & api, json11::Json payload, int timeout,
-    int retries, int interval, std::function<void(std::string, json11::Json)> callback)
-{
-    const auto & url = selected_etcd_url;
+        fprintf(stderr, "etcd_address is missing in Vitastor configuration\n");
+        exit(1);
+    }
+    pick_next_etcd();
+    std::string etcd_address = selected_etcd_address;
+    std::string etcd_api_path;
+    int pos = etcd_address.find('/');
+    if (pos >= 0)
+    {
+        etcd_api_path = etcd_address.substr(pos);
+        etcd_address = etcd_address.substr(0, pos);
+    }
     std::string req = payload.dump();
-    req = "POST "+url.path+api+" HTTP/1.1\r\n"
-        "Host: "+url.hostname+"\r\n"
+    req = "POST "+etcd_api_path+api+" HTTP/1.1\r\n"
+        "Host: "+etcd_address+"\r\n"
         "Content-Type: application/json\r\n"
         "Content-Length: "+std::to_string(req.size())+"\r\n"
         "Connection: keep-alive\r\n"
@@ -132,15 +128,15 @@ void etcd_state_client_t::etcd_call_selected(const std::string & api, json11::Js
         "\r\n"+req;
     retries--;
     auto cb = [this, api, payload, timeout, retries, interval, callback,
-        cur_addr = url.addr](http_message_t *response)
+        cur_addr = selected_etcd_address](const http_response_t *response)
     {
         std::string err;
         json11::Json data;
         response->parse_json_response(err, data);
         if (err != "")
         {
-            if (cur_addr == selected_etcd_url.addr)
-                selected_etcd_url = (http_url_t){};
+            if (cur_addr == selected_etcd_address)
+                selected_etcd_address = "";
             if (retries > 0)
             {
                 if (this->log_level > 0)
@@ -168,58 +164,54 @@ void etcd_state_client_t::etcd_call_selected(const std::string & api, json11::Js
             callback(err, data);
     };
     if (!keepalive_client)
-        keepalive_client = http_init(get_http_ctx());
-    http_request(keepalive_client, url.addr, req, { .timeout = timeout, .keepalive = true, .ssl = url.ssl }, cb);
+    {
+        keepalive_client = http_init(tfd);
+    }
+    http_request(keepalive_client, etcd_address, req, { .timeout = timeout, .keepalive = true }, cb);
 }
 
-
-void etcd_state_client_t::add_etcd_url(std::string etcd_address)
+void etcd_state_client_t::add_etcd_url(std::string addr)
 {
-    if (etcd_address.size() > 0)
+    if (addr.length() > 0)
     {
+        if (strtolower(addr.substr(0, 7)) == "http://")
+            addr = addr.substr(7);
+        else if (strtolower(addr.substr(0, 8)) == "https://")
+        {
+            fprintf(stderr, "HTTPS is unsupported for etcd. Either use plain HTTP or setup a local proxy for etcd interaction\n");
+            exit(1);
+        }
         if (!local_ips.size())
-        {
-            // Fill local_ips
-            for (auto & ip: getifaddr_list(std::vector<addr_mask_t>(), true))
-                local_ips.insert(ip);
-        }
-        std::string etcd_api_path;
-        bool ssl = false;
-        if (etcd_address.substr(0, 8) == "https://")
-        {
-            ssl = true;
-            etcd_address = etcd_address.substr(8);
-        }
-        else if (etcd_address.substr(0, 7) == "http://")
-            etcd_address = etcd_address.substr(7);
-        auto pos = etcd_address.find('/');
-        if (pos != std::string::npos)
-        {
-            etcd_api_path = etcd_address.substr(pos);
-            etcd_address = etcd_address.substr(0, pos);
-        }
+            local_ips = getifaddr_list(std::vector<addr_mask_t>(), true);
+        std::string check_addr;
+        int pos = addr.find('/');
+        int pos2 = addr.find(':');
+        if (pos2 >= 0)
+            check_addr = addr.substr(0, pos2);
+        else if (pos >= 0)
+            check_addr = addr.substr(0, pos);
         else
-            etcd_api_path = "/v3";
-        pos = etcd_address.find(':');
-        auto check_addr = (pos != std::string::npos ? etcd_address.substr(0, pos) : etcd_address);
-        bool is_local = local_ips.find(check_addr) != local_ips.end();
-        auto & to = (is_local ? etcd_local : etcd_addresses);
-        check_addr = (ssl ? "https://" : "http://") + etcd_address + etcd_api_path;
-        size_t i;
+            check_addr = addr;
+        if (pos == std::string::npos)
+            addr += "/v3";
+        bool local = false;
+        int i;
+        for (i = 0; i < local_ips.size(); i++)
+        {
+            if (local_ips[i] == check_addr)
+            {
+                local = true;
+                break;
+            }
+        }
+        auto & to = local ? this->etcd_local : this->etcd_addresses;
         for (i = 0; i < to.size(); i++)
         {
-            if (to[i] == check_addr)
+            if (to[i] == addr)
                 break;
         }
         if (i >= to.size())
-        {
-            to.push_back(check_addr);
-            // Check if it's a domain name
-            sockaddr_storage ss;
-            bool is_name = !is_local && !string_to_addr(etcd_address, true, 0, &ss);
-            auto & to_addr = (is_local ? etcd_local_addr_urls : (is_name ? etcd_name_urls : etcd_nonlocal_addr_urls));
-            to_addr.push_back((http_url_t){ .ssl = ssl, .addr = etcd_address, .hostname = etcd_address, .path = etcd_api_path });
-        }
+            to.push_back(addr);
     }
 }
 
@@ -227,9 +219,6 @@ void etcd_state_client_t::parse_config(const json11::Json & config)
 {
     this->etcd_local.clear();
     this->etcd_addresses.clear();
-    this->etcd_local_addr_urls.clear();
-    this->etcd_nonlocal_addr_urls.clear();
-    this->etcd_name_urls.clear();
     if (config["etcd_address"].is_string())
     {
         std::string ea = config["etcd_address"].string_value();
@@ -250,17 +239,6 @@ void etcd_state_client_t::parse_config(const json11::Json & config)
             add_etcd_url(ea.string_value());
         }
     }
-    if (this->osd_num)
-    {
-        this->etcd_client_cert = config["osd_etcd_client_cert"].string_value();
-        this->etcd_client_key = config["osd_etcd_client_key"].string_value();
-    }
-    else
-    {
-        this->etcd_client_cert = config["etcd_client_cert"].string_value();
-        this->etcd_client_key = config["etcd_client_key"].string_value();
-    }
-    this->etcd_ca = config["etcd_ca"].string_value();
     this->etcd_prefix = config["etcd_prefix"].string_value();
     if (this->etcd_prefix == "")
     {
@@ -304,7 +282,6 @@ void etcd_state_client_t::parse_config(const json11::Json & config)
     {
         this->etcd_min_reload_interval = 50;
     }
-    this->use_image_index_for_auth = config["use_image_index_for_auth"].bool_value();
     if (this->etcd_ws_keepalive_interval != old_etcd_ws_keepalive_interval && ws_keepalive_timer >= 0)
     {
 #ifndef __MOCK__
@@ -314,131 +291,66 @@ void etcd_state_client_t::parse_config(const json11::Json & config)
     }
 }
 
-void etcd_state_client_t::pick_next_etcd(std::function<void()> cb)
+void etcd_state_client_t::pick_next_etcd()
+{
+    if (selected_etcd_address != "")
+        return;
+    if (addresses_to_try.size() == 0)
+    {
+        // Prefer local etcd, if any
+        for (int i = 0; i < etcd_local.size(); i++)
+            addresses_to_try.push_back(etcd_local[i]);
+        std::vector<int> ns;
+        for (int i = 0; i < etcd_addresses.size(); i++)
+            ns.push_back(i);
+        if (!rand_initialized)
+        {
+            timespec tv;
+            clock_gettime(CLOCK_REALTIME, &tv);
+            srand48(tv.tv_sec*1000000000 + tv.tv_nsec);
+            rand_initialized = true;
+        }
+        while (ns.size())
+        {
+            int i = lrand48() % ns.size();
+            addresses_to_try.push_back(etcd_addresses[ns[i]]);
+            ns.erase(ns.begin()+i, ns.begin()+i+1);
+        }
+    }
+    selected_etcd_address = addresses_to_try[0];
+    addresses_to_try.erase(addresses_to_try.begin(), addresses_to_try.begin()+1);
+}
+
+void etcd_state_client_t::start_etcd_watcher()
 {
     if (!etcd_addresses.size() && !etcd_local.size())
     {
         fprintf(stderr, "etcd_address is missing in Vitastor configuration\n");
         exit(1);
     }
-    if (selected_etcd_url.addr != "")
+    pick_next_etcd();
+    std::string etcd_address = selected_etcd_address;
+    std::string etcd_api_path;
+    int pos = etcd_address.find('/');
+    if (pos >= 0)
     {
-        cb();
-        return;
+        etcd_api_path = etcd_address.substr(pos);
+        etcd_address = etcd_address.substr(0, pos);
     }
-    if (etcd_urls_to_try.size() != 0)
-    {
-        selected_etcd_url = std::move(etcd_urls_to_try[0]);
-        etcd_urls_to_try.erase(etcd_urls_to_try.begin());
-        cb();
-        return;
-    }
-    on_resolve_queue.push_back(std::move(cb));
-    if (on_resolve_queue.size() > 1)
-    {
-        // Already resolving
-        return;
-    }
-    assert(!resolve_count);
-    local_to_try = 0;
-    for (auto & url: etcd_local_addr_urls)
-    {
-        // Prefer local IPs, if any
-        etcd_urls_to_try.push_back(url);
-        local_to_try++;
-    }
-    for (auto & url: etcd_nonlocal_addr_urls)
-    {
-        etcd_urls_to_try.push_back(url);
-    }
-    resolve_count++;
-    for (auto & url: etcd_name_urls)
-    {
-        resolve_count++;
-        http_resolve(get_http_ctx(), url.ssl, url.addr, [this, url](const std::string & error, const std::vector<std::string>& addresses)
-        {
-            if (error != "")
-                fprintf(stderr, "Error resolving %s: %s\n", url.addr.c_str(), error.c_str());
-            for (auto & addr: addresses)
-            {
-                auto url_copy = url;
-                url_copy.addr = addr;
-                if (local_ips.find(addr) != local_ips.end())
-                {
-                    etcd_urls_to_try.insert(etcd_urls_to_try.begin(), std::move(url_copy));
-                    local_to_try++;
-                }
-                else
-                    etcd_urls_to_try.push_back(std::move(url_copy));
-            }
-            resolve_count--;
-            if (!resolve_count)
-                pick_next_etcd_on_resolve();
-        });
-    }
-    resolve_count--;
-    if (!resolve_count)
-    {
-        pick_next_etcd_on_resolve();
-    }
-}
-
-void etcd_state_client_t::pick_next_etcd_on_resolve()
-{
-    if (!etcd_urls_to_try.size())
-    {
-        fprintf(stderr, "None of etcd_address could be resolved\n");
-        exit(1);
-    }
-    if (!rand_initialized)
-    {
-        timespec tv;
-        clock_gettime(CLOCK_REALTIME, &tv);
-        srand48(tv.tv_sec*1000000000 + tv.tv_nsec);
-        rand_initialized = true;
-    }
-    // Shuffle addresses
-    for (size_t i = etcd_urls_to_try.size()-1; i > local_to_try; i--)
-    {
-        size_t j = local_to_try + lrand48() % (i - local_to_try);
-        if (j != i)
-            std::swap(etcd_urls_to_try[i], etcd_urls_to_try[j]);
-    }
-    selected_etcd_url = std::move(etcd_urls_to_try[0]);
-    etcd_urls_to_try.erase(etcd_urls_to_try.begin());
-    auto cbs = std::move(on_resolve_queue);
-    for (auto cb: cbs)
-    {
-        cb();
-    }
-}
-
-void etcd_state_client_t::start_etcd_watcher()
-{
-    pick_next_etcd([this]()
-    {
-        start_etcd_watcher_selected();
-    });
-}
-
-void etcd_state_client_t::start_etcd_watcher_selected()
-{
-    const auto & url = selected_etcd_url;
     etcd_watches_initialised = 0;
     ws_alive = 1;
+    if (etcd_watch_ws)
+    {
+        http_close(etcd_watch_ws);
+        etcd_watch_ws = NULL;
+    }
     if (this->log_level > 1)
     {
-        fprintf(stderr, "Trying to connect to etcd websocket at %s%s%s (hostname %s), watch from revision %ju/%ju/%ju\n",
-            url.ssl ? "https://" : "http://", url.addr.c_str(), url.path.c_str(), url.hostname.c_str(),
+        fprintf(stderr, "Trying to connect to etcd websocket at %s, watch from revision %ju/%ju/%ju\n", etcd_address.c_str(),
             etcd_watch_revision_config, etcd_watch_revision_osd, etcd_watch_revision_pg);
     }
-    if (!etcd_watch_ws)
-        etcd_watch_ws = http_init(get_http_ctx());
-    else
-        http_close(etcd_watch_ws);
-    etcd_total_watches = (use_image_index_for_auth ? 4 : 3);
-    open_websocket(etcd_watch_ws, url.addr, url.hostname, url.path+"/watch", { .timeout = etcd_slow_timeout, .ssl = url.ssl },
-        [this, cur_addr = url.addr](http_message_t *msg)
+    etcd_watch_ws = open_websocket(tfd, etcd_address, etcd_api_path+"/watch", etcd_slow_timeout,
+        [this, cur_addr = selected_etcd_address](const http_response_t *msg)
     {
         if (msg->body.length())
         {
@@ -456,12 +368,11 @@ void etcd_state_client_t::start_etcd_watcher_selected()
                 {
                     if (watch_id == ETCD_CONFIG_WATCH_ID ||
                         watch_id == ETCD_PG_STATE_WATCH_ID ||
-                        watch_id == ETCD_OSD_STATE_WATCH_ID ||
-                        watch_id == ETCD_INDEX_WATCH_ID)
+                        watch_id == ETCD_OSD_STATE_WATCH_ID)
                     {
                         etcd_watches_initialised++;
                     }
-                    if (etcd_watches_initialised == etcd_total_watches && this->log_level > 0)
+                    if (etcd_watches_initialised == ETCD_TOTAL_WATCHES && this->log_level > 0)
                     {
                         fprintf(stderr, "Successfully subscribed to etcd at %s, revision %ju/%ju/%ju\n", cur_addr.c_str(),
                             etcd_watch_revision_config, etcd_watch_revision_osd, etcd_watch_revision_pg);
@@ -482,7 +393,8 @@ void etcd_state_client_t::start_etcd_watcher_selected()
                                 fprintf(stderr, "Revisions before %ju were compacted by etcd, reloading state\n",
                                     data["result"]["compact_revision"].uint64_value());
                                 http_close(etcd_watch_ws);
-                                etcd_watch_revision_config = etcd_watch_revision_osd = etcd_watch_revision_pg = etcd_watch_revision_index = 0;
+                                etcd_watch_ws = NULL;
+                                etcd_watch_revision_config = etcd_watch_revision_osd = etcd_watch_revision_pg = 0;
                                 on_reload_hook();
                             }
                             return;
@@ -503,7 +415,7 @@ void etcd_state_client_t::start_etcd_watcher_selected()
                 // Save revision only if it's present in the message - because sometimes etcd sends something without a header, like:
                 // {"error": {"grpc_code": 14, "http_code": 503, "http_status": "Service Unavailable", "message": "error reading from server: EOF"}}
                 // Also don't save revision from the initial created: true messages because they always contain the latest revision
-                if (etcd_watches_initialised == etcd_total_watches &&
+                if (etcd_watches_initialised == ETCD_TOTAL_WATCHES &&
                     !data["result"]["header"]["revision"].is_null() &&
                     !data["result"]["created"].bool_value())
                 {
@@ -526,9 +438,7 @@ void etcd_state_client_t::start_etcd_watcher_selected()
                         etcd_watch_revision_pg = watch_rev;
                     else if (watch_id == ETCD_OSD_STATE_WATCH_ID)
                         etcd_watch_revision_osd = watch_rev;
-                    else if (watch_id == ETCD_INDEX_WATCH_ID)
-                        etcd_watch_revision_index = watch_rev;
-                    etcd_urls_to_try.clear();
+                    addresses_to_try.clear();
                 }
                 // First gather all changes into a hash to remove multiple overwrites
                 std::map<std::string, etcd_kv_t> changes;
@@ -558,8 +468,13 @@ void etcd_state_client_t::start_etcd_watcher_selected()
         if (msg->eof)
         {
             fprintf(stderr, "Disconnected from etcd %s\n", cur_addr.c_str());
-            if (cur_addr == selected_etcd_url.addr)
-                selected_etcd_url = (http_url_t){};
+            if (cur_addr == selected_etcd_address)
+                selected_etcd_address = "";
+            if (etcd_watch_ws)
+            {
+                http_close(etcd_watch_ws);
+                etcd_watch_ws = NULL;
+            }
             if (etcd_watches_initialised == 0)
             {
                 // Connection not established, retry in <etcd_quick_timeout>
@@ -603,18 +518,6 @@ void etcd_state_client_t::start_etcd_watcher_selected()
             { "progress_notify", true },
         } }
     }).dump());
-    if (use_image_index_for_auth)
-    {
-        http_post_message(etcd_watch_ws, WS_TEXT, json11::Json(json11::Json::object {
-            { "create_request", json11::Json::object {
-                { "key", base64_encode(etcd_prefix+"/index/image/") },
-                { "range_end", base64_encode(etcd_prefix+"/index/image0") },
-                { "start_revision", etcd_watch_revision_index },
-                { "watch_id", ETCD_INDEX_WATCH_ID },
-                { "progress_notify", true },
-            } }
-        }).dump());
-    }
     // FIXME: Do not watch /pg/history/ at all in client code (not in OSD)
     if (on_start_watcher_hook)
     {
@@ -638,7 +541,7 @@ void etcd_state_client_t::start_ws_keepalive()
     {
         ws_keepalive_timer = tfd->set_timer(etcd_ws_keepalive_interval*1000, true, [this](int)
         {
-            if (!etcd_watch_ws || etcd_watches_initialised < etcd_total_watches)
+            if (!etcd_watch_ws || etcd_watches_initialised < ETCD_TOTAL_WATCHES)
             {
                 // Do nothing
             }
@@ -646,7 +549,12 @@ void etcd_state_client_t::start_ws_keepalive()
             {
                 if (this->log_level > 0)
                 {
-                    fprintf(stderr, "Websocket ping failed, disconnecting from etcd %s\n", selected_etcd_url.addr.c_str());
+                    fprintf(stderr, "Websocket ping failed, disconnecting from etcd %s\n", selected_etcd_address.c_str());
+                }
+                if (etcd_watch_ws)
+                {
+                    http_close(etcd_watch_ws);
+                    etcd_watch_ws = NULL;
                 }
                 start_etcd_watcher();
             }
@@ -784,15 +692,6 @@ void etcd_state_client_t::load_pgs()
             } }
         },
     };
-    if (use_image_index_for_auth)
-    {
-        txn.push_back(json11::Json::object {
-            { "request_range", json11::Json::object {
-                { "key", base64_encode(etcd_prefix+"/index/image/") },
-                { "range_end", base64_encode(etcd_prefix+"/index/image0") },
-            } }
-        });
-    }
     json11::Json::object req = { { "success", txn } };
     json11::Json checks = load_pgs_checks_hook != NULL ? load_pgs_checks_hook() : json11::Json();
     if (checks.array_items().size() > 0)
@@ -1286,7 +1185,6 @@ void etcd_state_client_t::parse_state(const etcd_kv_t & kv)
                 if (i >= pg_state_bit_count)
                 {
                     fprintf(stderr, "Unexpected pool %u PG %u state keyword in etcd: %s\n", pool_id, pg_num, e.dump().c_str());
-                    return;
                 }
             }
             if (!cur_primary || !value["state"].is_array() || !state ||
@@ -1295,7 +1193,6 @@ void etcd_state_client_t::parse_state(const etcd_kv_t & kv)
                 (state & PG_INCOMPLETE) && state != PG_INCOMPLETE && state != (PG_INCOMPLETE|PG_HAS_INVALID))
             {
                 fprintf(stderr, "Unexpected pool %u PG %u state in etcd: primary=%ju, state=%s\n", pool_id, pg_num, cur_primary, value["state"].dump().c_str());
-                return;
             }
             pg_cfg.cur_primary = cur_primary;
             pg_cfg.cur_state = state;
@@ -1366,40 +1263,33 @@ void etcd_state_client_t::parse_state(const etcd_kv_t & kv)
             }
             else
             {
-                insert_inode_config(deserialize_inode_cfg(inode_num, kv.value, kv.mod_revision));
-            }
-        }
-    }
-    else if (key.substr(0, etcd_prefix.length()+13) == etcd_prefix+"/index/image/")
-    {
-        // <etcd_prefix>/index/image/%s
-        // Used for environments based on etcd authentication to grant client image access
-        // by names - granting by ID is inconvenient because of snapshots
-        std::string inode_name = key.substr(etcd_prefix.length()+13);
-        if (!value.is_object())
-        {
-            auto n_it = this->inode_by_name.find(inode_name);
-            if (n_it != this->inode_by_name.end())
-            {
-                uint64_t inode_num = n_it->second;
-                if (on_inode_change_hook != NULL)
+                inode_t parent_inode_num = value["parent_id"].uint64_value();
+                if (parent_inode_num && !(parent_inode_num >> (64-POOL_ID_BITS)))
                 {
-                    on_inode_change_hook(inode_num, true);
+                    uint64_t parent_pool_id = value["parent_pool"].uint64_value();
+                    if (!parent_pool_id)
+                        parent_inode_num |= pool_id << (64-POOL_ID_BITS);
+                    else if (parent_pool_id >= POOL_ID_MAX)
+                    {
+                        fprintf(
+                            stderr, "Inode %ju/%ju parent_pool value is invalid, ignoring parent setting\n",
+                            inode_num >> (64-POOL_ID_BITS), inode_num & (((uint64_t)1 << (64-POOL_ID_BITS)) - 1)
+                        );
+                        parent_inode_num = 0;
+                    }
+                    else
+                        parent_inode_num |= parent_pool_id << (64-POOL_ID_BITS);
                 }
-                this->inode_config.erase(inode_num);
-                this->inode_by_name.erase(n_it);
-            }
-        }
-        else if (value["name"] == inode_name)
-        {
-            uint64_t inode_num = INODE_WITH_POOL(value["pool_id"].uint64_value(), value["id"].uint64_value());
-            if (!INODE_POOL(inode_num) || !INODE_NO_POOL(inode_num))
-            {
-                fprintf(stderr, "Invalid inode ID in key %s: %ju\n", key.c_str(), inode_num);
-            }
-            else
-            {
-                insert_inode_config(deserialize_inode_cfg(inode_num, value, kv.mod_revision));
+                insert_inode_config((inode_config_t){
+                    .num = inode_num,
+                    .name = value["name"].string_value(),
+                    .size = value["size"].uint64_value(),
+                    .parent_id = parent_inode_num,
+                    .readonly = value["readonly"].bool_value(),
+                    .deleted = value["deleted"].bool_value(),
+                    .meta = value["meta"],
+                    .mod_revision = kv.mod_revision,
+                });
             }
         }
     }
@@ -1488,10 +1378,6 @@ json11::Json::object etcd_state_client_t::serialize_inode_cfg(inode_config_t *cf
             new_cfg["parent_pool"] = (uint64_t)INODE_POOL(cfg->parent_id);
         new_cfg["parent_id"] = (uint64_t)INODE_NO_POOL(cfg->parent_id);
     }
-    if (!cfg->enc_key.empty())
-    {
-        new_cfg["enc_key"] = cfg->enc_key;
-    }
     if (cfg->readonly)
     {
         new_cfg["readonly"] = true;
@@ -1507,50 +1393,6 @@ json11::Json::object etcd_state_client_t::serialize_inode_cfg(inode_config_t *cf
     return new_cfg;
 }
 
-inode_config_t etcd_state_client_t::deserialize_inode_cfg(uint64_t inode_num, json11::Json value, uint64_t mod_revision)
-{
-    inode_t parent_inode_num = value["parent_id"].uint64_value();
-    if (parent_inode_num && !INODE_POOL(parent_inode_num))
-    {
-        uint64_t parent_pool_id = value["parent_pool"].uint64_value();
-        if (!parent_pool_id)
-            parent_inode_num = INODE_WITH_POOL(INODE_POOL(inode_num), parent_inode_num);
-        else if (parent_pool_id >= POOL_ID_MAX)
-        {
-            fprintf(
-                stderr, "Inode %u/%ju parent_pool value is invalid, ignoring parent setting\n",
-                INODE_POOL(inode_num), INODE_NO_POOL(inode_num)
-            );
-            parent_inode_num = 0;
-        }
-        else
-            parent_inode_num |= parent_pool_id << (64-POOL_ID_BITS);
-    }
-    std::string enc_key;
-    if (!value["enc_key"].is_null())
-    {
-        enc_key = value["enc_key"].string_value();
-        if (enc_key.substr(0, strlen(VAULT_KEY_PREFIX)) != VAULT_KEY_PREFIX &&
-            (enc_key.size() != 2*AES_256_XTS_KEY_SIZE || !ishexstr(enc_key)))
-        {
-            enc_key = "";
-            fprintf(stderr, "Inode %u/%ju has invalid enc_key, should be %u bit hex string or Vault key reference\n",
-                INODE_POOL(inode_num), INODE_NO_POOL(inode_num), AES_256_XTS_KEY_SIZE);
-        }
-    }
-    return (inode_config_t){
-        .num = inode_num,
-        .name = value["name"].string_value(),
-        .size = value["size"].uint64_value(),
-        .parent_id = parent_inode_num,
-        .readonly = value["readonly"].bool_value(),
-        .deleted = value["deleted"].bool_value(),
-        .enc_key = std::move(enc_key),
-        .meta = value["meta"],
-        .mod_revision = mod_revision,
-    };
-}
-
 int etcd_state_client_t::address_count()
 {
     return etcd_addresses.size() + etcd_local.size();

src/client/etcd_state_client.h

@@ -4,7 +4,6 @@
 #pragma once
 
 #include <set>
-#include <memory>
 
 #include "json11/json11.hpp"
 #include "object_id.h"
@@ -13,15 +12,13 @@
 #define ETCD_CONFIG_WATCH_ID 1
 #define ETCD_OSD_STATE_WATCH_ID 2
 #define ETCD_PG_STATE_WATCH_ID 3
-#define ETCD_INDEX_WATCH_ID 4
+#define ETCD_TOTAL_WATCHES 3
 
 #define DEFAULT_BLOCK_SIZE 128*1024
 #define MIN_DATA_BLOCK_SIZE 4*1024
 #define MAX_DATA_BLOCK_SIZE 128*1024*1024
 #define DEFAULT_BITMAP_GRANULARITY 4096
 
-#define VAULT_KEY_PREFIX "vault:"
-
 #ifndef IMMEDIATE_NONE
 #define IMMEDIATE_NONE 0
 #define IMMEDIATE_SMALL 1
@@ -86,7 +83,6 @@ struct inode_config_t
     inode_t parent_id = 0;
     bool readonly = false;
     bool deleted = false;
-    std::string enc_key;
     // Arbitrary metadata
     json11::Json meta;
     // Change revision of the metadata in etcd
@@ -99,42 +95,23 @@ struct inode_watch_t
     inode_config_t cfg = {};
 };
 
-struct http_url_t
-{
-    bool ssl;
-    std::string addr;
-    std::string hostname;
-    std::string path;
-};
-
 struct http_co_t;
-struct http_context_t;
 
 struct __attribute__((visibility("default"))) etcd_state_client_t
 {
 protected:
-    std::set<std::string> local_ips;
-    std::vector<std::string> etcd_local;
+    std::vector<std::string> local_ips;
     std::vector<std::string> etcd_addresses;
-    std::vector<http_url_t> etcd_local_addr_urls;
-    std::vector<http_url_t> etcd_nonlocal_addr_urls;
-    std::vector<http_url_t> etcd_name_urls;
-    size_t local_to_try = 0;
-    std::vector<http_url_t> etcd_urls_to_try;
-    http_url_t selected_etcd_url;
-    size_t resolve_count = 0;
+    std::vector<std::string> etcd_local;
+    std::string selected_etcd_address;
+    std::vector<std::string> addresses_to_try;
     std::vector<inode_watch_t*> watches;
-    std::vector<std::function<void()>> on_resolve_queue;
     bool new_pg_config = false;
-    bool use_image_index_for_auth = false;
     int ws_keepalive_timer = -1;
     int ws_alive = 0;
     bool rand_initialized = false;
     void add_etcd_url(std::string);
-    void pick_next_etcd(std::function<void()> cb);
-    void pick_next_etcd_on_resolve();
-    void etcd_call_selected(const std::string & api, json11::Json payload, int timeout, int retries, int interval, std::function<void(std::string, json11::Json)> callback);
-    void start_etcd_watcher_selected();
+    void pick_next_etcd();
 public:
     int etcd_keepalive_timeout = 30;
     int etcd_ws_keepalive_interval = 5;
@@ -147,22 +124,15 @@ public:
     uint32_t global_bitmap_granularity = DEFAULT_BITMAP_GRANULARITY;
     uint32_t global_immediate_commit = IMMEDIATE_NONE;
 
-    uint64_t osd_num = 0;
     std::string etcd_prefix;
-    std::string etcd_client_cert;
-    std::string etcd_client_key;
-    std::string etcd_ca;
     int log_level = 0;
     timerfd_manager_t *tfd = NULL;
 
-    http_context_t *http_ctx = NULL;
     http_co_t *etcd_watch_ws = NULL, *keepalive_client = NULL;
     int etcd_watches_initialised = 0;
-    int etcd_total_watches = 0;
     uint64_t etcd_watch_revision_config = 0;
     uint64_t etcd_watch_revision_osd = 0;
     uint64_t etcd_watch_revision_pg = 0;
-    uint64_t etcd_watch_revision_index = 0;
     timespec etcd_last_reload = {};
     int load_pgs_timer_id = -1;
     std::map<pool_id_t, pool_config_t> pool_config;
@@ -188,12 +158,10 @@ public:
     std::function<void(http_co_t *)> on_start_watcher_hook;
 
     json11::Json::object serialize_inode_cfg(inode_config_t *cfg);
-    inode_config_t deserialize_inode_cfg(uint64_t inode_num, json11::Json value, uint64_t mod_revision);
     etcd_kv_t parse_etcd_kv(const json11::Json & kv_json);
     std::vector<std::string> get_addresses();
-    http_context_t *get_http_ctx();
-    void etcd_call_oneshot(const std::string & etcd_address, const std::string & api, json11::Json payload, int timeout, std::function<void(std::string, json11::Json)> callback);
-    void etcd_call(const std::string & api, json11::Json payload, int timeout, int retries, int interval, std::function<void(std::string, json11::Json)> callback);
+    void etcd_call_oneshot(std::string etcd_address, std::string api, json11::Json payload, int timeout, std::function<void(std::string, json11::Json)> callback);
+    void etcd_call(std::string api, json11::Json payload, int timeout, int retries, int interval, std::function<void(std::string, json11::Json)> callback);
     void etcd_txn(json11::Json txn, int timeout, int retries, int interval, std::function<void(std::string, json11::Json)> callback);
     void etcd_txn_slow(json11::Json txn, std::function<void(std::string, json11::Json)> callback);
     void start_etcd_watcher();

src/client/http_client.h

@@ -17,19 +17,14 @@
 
 class timerfd_manager_t;
 
-#pragma GCC visibility push(default)
-
 struct http_options_t
 {
     int timeout;
     bool want_streaming;
     bool keepalive;
-    bool ssl;
 };
 
-struct http_context_t;
-
-struct http_message_t
+struct http_response_t
 {
     std::string error;
 
@@ -46,25 +41,10 @@ struct http_message_t
 // Opened websocket or keepalive HTTP connection
 struct http_co_t;
 
-http_context_t* http_context_init(timerfd_manager_t *tfd, const std::string & ssl_cert, const std::string & ssl_key,
-    const std::string & ssl_ca, bool verify_peer, std::string & error);
-void http_resolve(http_context_t *ctx, bool ssl, std::string host,
-    std::function<void(const std::string & error, const std::vector<std::string> & addrs)> cb);
-void http_context_destroy(http_context_t *ctx);
-http_co_t* http_init(http_context_t *ctx = NULL);
-void open_websocket(http_co_t *handler, const std::string & addr, const std::string & hostname, const std::string & path,
-    const http_options_t & options, std::function<void(http_message_t *msg)> on_message);
+http_co_t* http_init(timerfd_manager_t *tfd);
+http_co_t* open_websocket(timerfd_manager_t *tfd, const std::string & host, const std::string & path,
+    int timeout, std::function<void(const http_response_t *msg)> on_message);
 void http_request(http_co_t *handler, const std::string & host, const std::string & request,
-    const http_options_t & options, std::function<void(http_message_t *response)> response_callback);
-void http_get(http_co_t *handler, const std::string & url, const std::string & headers,
-    const http_options_t & options, std::function<void(http_message_t *response)> response_callback);
-void http_json_post(http_co_t *handler, const std::string & url, json11::Json body, const std::string & headers,
-    const http_options_t & options, std::function<void(http_message_t *response)> response_callback);
+    const http_options_t & options, std::function<void(const http_response_t *response)> response_callback);
 void http_post_message(http_co_t *handler, uint8_t type, const std::string & msg);
-void http_serve(http_co_t *handler, int peer_fd, const http_options_t & options,
-    std::function<void(http_message_t *msg)> request_callback);
-void http_reply(http_co_t *handler, const std::string & reply);
 void http_close(http_co_t *co);
-void http_destroy(http_co_t *co);
-
-#pragma GCC visibility pop

src/client/messenger.cpp

@@ -15,106 +15,6 @@
 #include "msgr_rdma.h"
 #endif
 
-#include <sys/poll.h>
-
-msgr_iothread_t::msgr_iothread_t():
-    ring(RINGLOOP_DEFAULT_SIZE, true),
-    thread(&msgr_iothread_t::run, this)
-{
-    eventfd = ring.register_eventfd();
-    if (eventfd < 0)
-    {
-        throw std::runtime_error(std::string("failed to register eventfd: ") + strerror(-eventfd));
-    }
-}
-
-msgr_iothread_t::~msgr_iothread_t()
-{
-    stop();
-}
-
-void msgr_iothread_t::add_sqe(io_uring_sqe & sqe)
-{
-    mu.lock();
-    queue.push_back((iothread_sqe_t){ .sqe = sqe, .data = std::move(*(ring_data_t*)sqe.user_data) });
-    if (queue.size() == 1)
-    {
-        cond.notify_all();
-    }
-    mu.unlock();
-}
-
-void msgr_iothread_t::stop()
-{
-    mu.lock();
-    if (stopped)
-    {
-        mu.unlock();
-        return;
-    }
-    stopped = true;
-    if (outer_loop_data)
-    {
-        outer_loop_data->callback = [](ring_data_t*){};
-    }
-    cond.notify_all();
-    close(eventfd);
-    mu.unlock();
-    thread.join();
-}
-
-void msgr_iothread_t::add_to_ringloop(ring_loop_t *outer_loop)
-{
-    assert(!this->outer_loop || this->outer_loop == outer_loop);
-    io_uring_sqe *sqe = outer_loop->get_sqe();
-    assert(sqe != NULL);
-    this->outer_loop = outer_loop;
-    this->outer_loop_data = ((ring_data_t*)sqe->user_data);
-    io_uring_prep_poll_add(sqe, eventfd, POLLIN);
-    outer_loop_data->callback = [this](ring_data_t *data)
-    {
-        if (data->res < 0)
-        {
-            throw std::runtime_error(std::string("eventfd poll failed: ") + strerror(-data->res));
-        }
-        outer_loop_data = NULL;
-        if (stopped)
-        {
-            return;
-        }
-        add_to_ringloop(this->outer_loop);
-        ring.loop();
-    };
-}
-
-void msgr_iothread_t::run()
-{
-    while (true)
-    {
-        {
-            std::unique_lock<std::mutex> lk(mu);
-            while (!stopped && !queue.size())
-                cond.wait(lk);
-            if (stopped)
-                return;
-            int i = 0;
-            for (; i < queue.size(); i++)
-            {
-                io_uring_sqe *sqe = ring.get_sqe();
-                if (!sqe)
-                    break;
-                ring_data_t *data = ((ring_data_t*)sqe->user_data);
-                *data = std::move(queue[i].data);
-                *sqe = queue[i].sqe;
-                sqe->user_data = (uint64_t)data;
-            }
-            queue.erase(queue.begin(), queue.begin()+i);
-        }
-        // We only want to offload sendmsg/recvmsg. Callbacks will be called in main thread
-        ring.submit();
-    }
-}
-
 void osd_messenger_t::init()
 {
 #ifdef WITH_RDMACM
@@ -173,21 +73,17 @@ void osd_messenger_t::init()
     }
     if (ringloop && iothread_count > 0)
     {
-        for (int i = 0; i < iothread_count; i++)
-        {
-            auto iot = new msgr_iothread_t();
-            iothreads.push_back(iot);
-            iot->add_to_ringloop(ringloop);
-        }
+        init_iothreads();
     }
     keepalive_timer_id = tfd->set_timer(1000, true, [this](int)
     {
+        std::vector<uint64_t> clients_to_stop;
+        std::vector<osd_op_t*> ops_to_send;
         auto cl_it = clients.begin();
         while (cl_it != clients.end())
         {
             auto cl = cl_it->second;
             cl_it++;
-            auto peer_fd = cl->peer_fd;
             if (!cl->osd_num && !cl->in_osd_num || cl->peer_state != PEER_CONNECTED && cl->peer_state != PEER_RDMA)
             {
                 // Do not run keepalive on regular clients
@@ -199,10 +95,9 @@ void osd_messenger_t::init()
                 if (!cl->ping_time_remaining)
                 {
                     // Ping timed out, stop the client
-                    fprintf(stderr, "Ping timed out for OSD %ju (client %d), disconnecting peer\n", cl->in_osd_num ? cl->in_osd_num : cl->osd_num, cl->peer_fd);
-                    stop_client(peer_fd, true);
-                    // Restart iterator because it may be invalidated
-                    cl_it = clients.upper_bound(peer_fd);
+                    fprintf(stderr, "Ping timed out for OSD %ju (client %ju), disconnecting peer\n",
+                        cl->in_osd_num ? cl->in_osd_num : cl->osd_num, cl->client_id);
+                    clients_to_stop.push_back(cl->client_id);
                 }
             }
             else if (cl->idle_time_remaining > 0)
@@ -213,37 +108,36 @@ void osd_messenger_t::init()
                     // Connection is idle for <osd_idle_time>, send ping
                     osd_op_t *op = new osd_op_t();
                     op->op_type = OSD_OP_OUT;
-                    op->peer_fd = cl->peer_fd;
+                    op->client_id = cl->client_id;
                     op->req = (osd_any_op_t){
                         .hdr = {
                             .magic = SECONDARY_OSD_OP_MAGIC,
                             .opcode = OSD_OP_PING,
                         },
                     };
-                    op->callback = [this, cl](osd_op_t *op)
+                    op->callback = [this](osd_op_t *op)
                     {
-                        auto cl_it = clients.find(op->peer_fd);
-                        if (cl_it == clients.end() || cl_it->second != cl)
+                        auto cl_it = clients.find(op->client_id);
+                        if (cl_it == clients.end())
                         {
                             // client is already dropped
                             delete op;
                             return;
                         }
-                        int fail_fd = (op->reply.hdr.retval != 0 ? op->peer_fd : -1);
+                        auto cl = cl_it->second;
+                        uint64_t fail_client_id = (op->reply.hdr.retval != 0 ? op->client_id : 0);
                         auto fail_osd_num = cl->in_osd_num ? cl->in_osd_num : cl->osd_num;
                         cl->ping_time_remaining = 0;
                         delete op;
-                        if (fail_fd >= 0)
+                        if (fail_client_id)
                         {
-                            fprintf(stderr, "Ping failed for OSD %ju (client %d), disconnecting peer\n", fail_osd_num, fail_fd);
-                            stop_client(fail_fd, true);
+                            fprintf(stderr, "Ping failed for OSD %ju (client %ju), disconnecting peer\n", fail_osd_num, fail_client_id);
+                            stop_client(fail_client_id);
                         }
                     };
                     cl->ping_time_remaining = osd_ping_timeout;
                     cl->idle_time_remaining = osd_idle_timeout;
-                    outbox_push(op);
-                    // Restart iterator because it may be invalidated
-                    cl_it = clients.upper_bound(peer_fd);
+                    ops_to_send.push_back(op);
                 }
             }
             else
@@ -251,6 +145,14 @@ void osd_messenger_t::init()
                 cl->idle_time_remaining = osd_idle_timeout;
             }
         }
+        for (uint64_t client_id: clients_to_stop)
+        {
+            stop_client(client_id);
+        }
+        for (osd_op_t *op: ops_to_send)
+        {
+            outbox_push(op);
+        }
     });
 }
 
@@ -263,16 +165,9 @@ osd_messenger_t::~osd_messenger_t()
     }
     while (clients.size() > 0)
     {
-        stop_client(clients.begin()->first, true, true);
-    }
-    if (iothreads.size())
-    {
-        for (auto iot: iothreads)
-        {
-            delete iot;
-        }
-        iothreads.clear();
+        stop_client(clients.begin()->first, true);
     }
+    destroy_iothreads();
 #ifdef WITH_RDMA
     for (auto rdma_context: rdma_contexts)
     {
@@ -289,14 +184,6 @@ osd_messenger_t::~osd_messenger_t()
         rdmacm_evch = NULL;
     }
 #endif
-    for (auto encrypt_ctx: encrypt_ctx_pool)
-    {
-        destroy_aes_xts_encrypt(encrypt_ctx);
-    }
-    for (auto decrypt_ctx: decrypt_ctx_pool)
-    {
-        destroy_aes_xts_decrypt(decrypt_ctx);
-    }
 }
 
 void osd_messenger_t::parse_config(const json11::Json & config)
@@ -331,9 +218,6 @@ void osd_messenger_t::parse_config(const json11::Json & config)
     if (!this->rdma_max_msg || this->rdma_max_msg > 128*1024*1024)
         this->rdma_max_msg = 129*1024;
 #endif
-    this->max_aes_xts_pool_size = config["max_aes_xts_pool_size"].uint64_value();
-    if (!this->max_aes_xts_pool_size)
-        this->max_aes_xts_pool_size = 256;
     if (!osd_num)
         this->iothread_count = (uint32_t)config["client_iothread_count"].uint64_value();
     else
@@ -451,7 +335,7 @@ void osd_messenger_t::try_connect_peer(uint64_t peer_osd)
     {
         return;
     }
-    if (osd_peer_fds.find(peer_osd) != osd_peer_fds.end())
+    if (osd_peers.find(peer_osd) != osd_peers.end())
     {
         wanted_peers.erase(peer_osd);
         return;
@@ -478,20 +362,20 @@ void osd_messenger_t::try_connect_peer_tcp(osd_num_t peer_osd, const char *peer_
 #ifdef WITH_RDMACM
     if (disable_tcp)
     {
-        on_connect_peer(peer_osd, -EINVAL);
+        on_connect_peer(peer_osd, -EINVAL, 0);
         return;
     }
 #endif
     struct sockaddr_storage addr;
     if (!string_to_addr(peer_host, 0, peer_port, &addr))
     {
-        on_connect_peer(peer_osd, -EINVAL);
+        on_connect_peer(peer_osd, -EINVAL, 0);
         return;
     }
     int peer_fd = socket(addr.ss_family, SOCK_STREAM, 0);
     if (peer_fd < 0)
     {
-        on_connect_peer(peer_osd, -errno);
+        on_connect_peer(peer_osd, -errno, 0);
         return;
     }
     fcntl(peer_fd, F_SETFL, fcntl(peer_fd, F_GETFL, 0) | O_NONBLOCK);
@@ -499,21 +383,25 @@ void osd_messenger_t::try_connect_peer_tcp(osd_num_t peer_osd, const char *peer_
     if (r < 0 && errno != EINPROGRESS)
     {
         close(peer_fd);
-        on_connect_peer(peer_osd, -errno);
+        on_connect_peer(peer_osd, -errno, 0);
         return;
     }
-    clients[peer_fd] = new osd_client_t();
+    const uint64_t client_id = next_client_id++;
+    osd_client_t *cl = new osd_client_t();
     if (log_level > 0)
     {
-        fprintf(stderr, "Connecting to OSD %ju at %s:%d (client %d)\n", peer_osd, peer_host, peer_port, peer_fd);
+        fprintf(stderr, "Connecting to OSD %ju at %s:%d (client %ju, FD %d)\n", peer_osd, peer_host, peer_port, client_id, peer_fd);
     }
-    clients[peer_fd]->peer_addr = addr;
-    clients[peer_fd]->peer_port = peer_port;
-    clients[peer_fd]->peer_fd = peer_fd;
-    clients[peer_fd]->peer_state = PEER_CONNECTING;
-    clients[peer_fd]->connect_timeout_id = -1;
-    clients[peer_fd]->osd_num = peer_osd;
-    clients[peer_fd]->in_buf = (uint8_t*)malloc_or_die(receive_buffer_size);
+    cl->client_id = client_id;
+    cl->peer_addr = addr;
+    cl->peer_port = peer_port;
+    cl->peer_fd = peer_fd;
+    cl->peer_state = PEER_CONNECTING;
+    cl->connect_timeout_id = -1;
+    cl->osd_num = peer_osd;
+    cl->in_buf = malloc_or_die(receive_buffer_size);
+    clients[client_id] = cl;
+    clients_by_fd[peer_fd] = cl;
     tfd->set_fd_handler(peer_fd, true, [this](int peer_fd, int epoll_events)
     {
         // Either OUT (connected) or HUP
@@ -521,11 +409,11 @@ void osd_messenger_t::try_connect_peer_tcp(osd_num_t peer_osd, const char *peer_
     });
     if (peer_connect_timeout > 0)
     {
-        clients[peer_fd]->connect_timeout_id = tfd->set_timer(1000*peer_connect_timeout, false, [this, peer_fd](int timer_id)
+        cl->connect_timeout_id = tfd->set_timer(1000*peer_connect_timeout, false, [this, client_id](int timer_id)
         {
-            osd_num_t peer_osd = clients.at(peer_fd)->osd_num;
-            stop_client(peer_fd, true);
-            on_connect_peer(peer_osd, -EPIPE);
+            osd_num_t peer_osd = clients.at(client_id)->osd_num;
+            stop_client(client_id);
+            on_connect_peer(peer_osd, -EPIPE, 0);
             return;
         });
     }
@@ -533,7 +421,7 @@ void osd_messenger_t::try_connect_peer_tcp(osd_num_t peer_osd, const char *peer_
 
 void osd_messenger_t::handle_connect_epoll(int peer_fd)
 {
-    auto cl = clients[peer_fd];
+    auto cl = clients_by_fd.at(peer_fd);
     if (cl->connect_timeout_id >= 0)
     {
         tfd->clear_timer(cl->connect_timeout_id);
@@ -548,8 +436,8 @@ void osd_messenger_t::handle_connect_epoll(int peer_fd)
     }
     if (result != 0)
     {
-        stop_client(peer_fd, true);
-        on_connect_peer(peer_osd, -result);
+        stop_client(cl->client_id);
+        on_connect_peer(peer_osd, -result, 0);
         return;
     }
     int one = 1;
@@ -566,23 +454,23 @@ void osd_messenger_t::handle_connect_epoll(int peer_fd)
 void osd_messenger_t::handle_peer_epoll(int peer_fd, int epoll_events)
 {
     // Mark client as ready (i.e. some data is available)
+    auto cl = clients_by_fd.at(peer_fd);
     if (epoll_events & EPOLLRDHUP)
     {
         // Stop client
         if (log_level > 0)
         {
-            fprintf(stderr, "[OSD %ju] client %d disconnected\n", this->osd_num, peer_fd);
+            fprintf(stderr, "[OSD %ju] client %ju disconnected\n", this->osd_num, cl->client_id);
         }
-        stop_client(peer_fd, true);
+        stop_client(cl->client_id);
     }
     else if (epoll_events & EPOLLIN)
     {
         // Mark client as ready (i.e. some data is available)
-        auto cl = clients[peer_fd];
         cl->read_ready++;
         if (cl->read_ready == 1)
         {
-            read_ready_clients.push_back(cl->peer_fd);
+            read_ready_clients.push_back(cl->client_id);
             if (ringloop)
                 ringloop->wakeup();
             else
@@ -591,13 +479,13 @@ void osd_messenger_t::handle_peer_epoll(int peer_fd, int epoll_events)
     }
 }
 
-void osd_messenger_t::on_connect_peer(osd_num_t peer_osd, int peer_fd)
+void osd_messenger_t::on_connect_peer(osd_num_t peer_osd, int errcode, uint64_t client_id)
 {
     auto & wp = wanted_peers.at(peer_osd);
     wp.connecting = false;
-    if (peer_fd < 0)
+    if (errcode < 0)
     {
-        fprintf(stderr, "Failed to connect to peer OSD %ju address %s port %d: %s\n", peer_osd, wp.cur_addr.c_str(), wp.cur_port, strerror(-peer_fd));
+        fprintf(stderr, "Failed to connect to peer OSD %ju address %s port %d: %s\n", peer_osd, wp.cur_addr.c_str(), wp.cur_port, strerror(-errcode));
         if (wp.address_changed)
         {
             wp.address_changed = false;
@@ -624,7 +512,7 @@ void osd_messenger_t::on_connect_peer(osd_num_t peer_osd, int peer_fd)
     }
     if (log_level > 0)
     {
-        fprintf(stderr, "[OSD %ju] Connected with peer OSD %ju (client %d)\n", osd_num, peer_osd, peer_fd);
+        fprintf(stderr, "[OSD %ju] Connected with peer OSD %ju (client %ju)\n", osd_num, peer_osd, client_id);
     }
     wanted_peers.erase(peer_osd);
     repeer_pgs(peer_osd);
@@ -634,7 +522,7 @@ void osd_messenger_t::check_peer_config(osd_client_t *cl)
 {
     osd_op_t *op = new osd_op_t();
     op->op_type = OSD_OP_OUT;
-    op->peer_fd = cl->peer_fd;
+    op->client_id = cl->client_id;
     op->req = (osd_any_op_t){
         .show_conf = {
             .header = {
@@ -658,7 +546,7 @@ void osd_messenger_t::check_peer_config(osd_client_t *cl)
         if (!selected_ctx)
         {
             if (log_level > 0)
-                fprintf(stderr, "No RDMA context for OSD %ju connection (peer %d), using only TCP\n", cl->osd_num, cl->peer_fd);
+                fprintf(stderr, "No RDMA context for OSD %ju connection (client %ju), using only TCP\n", cl->osd_num, cl->client_id);
         }
         else
         {
@@ -719,8 +607,8 @@ void osd_messenger_t::check_peer_config(osd_client_t *cl)
         if (err)
         {
             osd_num_t peer_osd = cl->osd_num;
-            stop_client(op->peer_fd);
-            on_connect_peer(peer_osd, -EINVAL);
+            stop_client(op->client_id);
+            on_connect_peer(peer_osd, -EINVAL, 0);
             delete op;
             return;
         }
@@ -755,8 +643,8 @@ void osd_messenger_t::check_peer_config(osd_client_t *cl)
             }
         }
 #endif
-        osd_peer_fds[cl->osd_num] = cl->peer_fd;
-        on_connect_peer(cl->osd_num, cl->peer_fd);
+        osd_peers[cl->osd_num] = cl;
+        on_connect_peer(cl->osd_num, 0, cl->client_id);
         delete op;
     };
     outbox_push(op);
@@ -771,20 +659,23 @@ void osd_messenger_t::accept_connections(int listen_fd)
     while ((peer_fd = accept(listen_fd, (sockaddr*)&addr, &peer_addr_size)) >= 0)
     {
         assert(peer_fd != 0);
-        fprintf(stderr, "[OSD %ju] new client %d: connection from %s\n", this->osd_num, peer_fd,
+        const uint64_t client_id = next_client_id++;
+        fprintf(stderr, "[OSD %ju] new client %ju (FD %d): connection from %s\n", this->osd_num, client_id, peer_fd,
             addr_to_string(addr).c_str());
         fcntl(peer_fd, F_SETFL, fcntl(peer_fd, F_GETFL, 0) | O_NONBLOCK);
         int one = 1;
         setsockopt(peer_fd, SOL_TCP, TCP_NODELAY, &one, sizeof(one));
         auto cl = new osd_client_t();
-        clients[peer_fd] = cl;
+        cl->client_id = client_id;
+        clients[cl->client_id] = cl;
+        clients_by_fd[peer_fd] = cl;
         cl->is_incoming = true;
         cl->peer_addr = addr;
         cl->peer_addr = addr;
         cl->peer_port = ntohs(((sockaddr_in*)&addr)->sin_port);
         cl->peer_fd = peer_fd;
         cl->peer_state = PEER_CONNECTED;
-        cl->in_buf = (uint8_t*)malloc_or_die(receive_buffer_size);
+        cl->in_buf = malloc_or_die(receive_buffer_size);
         // Add FD to epoll
         tfd->set_fd_handler(peer_fd, false, [this](int peer_fd, int epoll_events)
         {

src/client/messenger.h

@@ -12,6 +12,7 @@
 #include <deque>
 #include <vector>
 
+#include "../util/robin_hood.h"
 #include "malloc_or_die.h"
 #include "json11/json11.hpp"
 #include "msgr_op.h"
@@ -34,6 +35,9 @@
 
 #define DEFAULT_MIN_ZEROCOPY_SEND_SIZE 32*1024
 
+#define MSGR_SENDP_HDR 1
+#define MSGR_SENDP_FREE 2
+
 struct msgr_sendp_t
 {
     osd_op_t *op;
@@ -45,13 +49,9 @@ struct msgr_rdma_connection_t;
 struct msgr_rdma_context_t;
 #endif
 
-struct op_aes_xts_encrypt_t;
-struct op_aes_xts_decrypt_t;
-void destroy_aes_xts_encrypt(op_aes_xts_encrypt_t *encrypt_ctx);
-void destroy_aes_xts_decrypt(op_aes_xts_decrypt_t *decrypt_ctx);
-
 struct osd_client_t
 {
+    uint64_t client_id = 0;
     int refs = 0;
 
     sockaddr_storage peer_addr = {};
@@ -65,7 +65,7 @@ struct osd_client_t
     osd_num_t in_osd_num = 0;
     bool is_incoming = false;
 
-    uint8_t *in_buf = NULL;
+    void *in_buf = NULL;
 
 #ifdef WITH_RDMA
     msgr_rdma_connection_t *rdma_conn = NULL;
@@ -74,40 +74,31 @@ struct osd_client_t
     // Read state
     int read_ready = 0;
     osd_op_t *read_op = NULL;
-    size_t read_op_size = 0;
-    size_t read_op_pos = 0;
     iovec read_iov = { 0 };
     msghdr read_msg = { 0 };
-    std::vector<iovec> recv_list;
-    size_t recv_list_size = 0;
+    int read_remaining = 0;
+    int read_state = 0;
+    osd_op_buf_list_t recv_list;
     uint64_t read_op_id = 1;
     bool check_sequencing = false;
     bool enable_pg_locks = false;
-    op_aes_xts_decrypt_t *decrypt_ctx = NULL;
-    size_t read_op_inline_decrypt_pos = 0;
-    size_t read_op_inline_decrypt_in = 0;
 
     // Incoming operations
     std::vector<osd_op_t*> received_ops;
 
     // Outbound operations
-    std::map<uint64_t, osd_op_t*> sent_ops;
+    robin_hood::unordered_flat_map<uint64_t, osd_op_t*> sent_ops;
     uint64_t send_op_id = 0;
 
     // PGs dirtied by this client's primary-writes
     std::set<pool_pg_num_t> dirty_pgs;
 
     // Write state
-    std::deque<osd_op_t *> write_ops;
-    osd_op_t *write_op = NULL;
-    size_t write_op_pos = 0;
     msghdr write_msg = { 0 };
     int write_state = 0;
-    std::vector<iovec> send_list;
-    size_t send_list_size = 0;
-    std::deque<osd_op_t*> send_free_ops;
+    std::vector<iovec> send_list, next_send_list;
+    std::vector<msgr_sendp_t> outbox, next_outbox;
     std::vector<osd_op_t*> zc_free_list;
-    op_aes_xts_encrypt_t *encrypt_ctx = NULL;
 
     ~osd_client_t();
     void cancel_ops();
@@ -138,43 +129,7 @@ struct osd_op_stats_t
     uint64_t subop_stat_count[OSD_OP_MAX+1] = { 0 };
 };
 
-#include <mutex>
-#include <condition_variable>
-#include <thread>
-
-#ifdef __MOCK__
 class msgr_iothread_t;
-#else
-struct iothread_sqe_t
-{
-    io_uring_sqe sqe;
-    ring_data_t data;
-};
-
-class msgr_iothread_t
-{
-protected:
-    ring_loop_t ring;
-    ring_loop_t *outer_loop = NULL;
-    ring_data_t *outer_loop_data = NULL;
-    int eventfd = -1;
-    bool stopped = false;
-    std::mutex mu;
-    std::condition_variable cond;
-    std::vector<iothread_sqe_t> queue;
-    std::thread thread;
-
-    void run();
-public:
-
-    msgr_iothread_t();
-    ~msgr_iothread_t();
-
-    void add_sqe(io_uring_sqe & sqe);
-    void stop();
-    void add_to_ringloop(ring_loop_t *outer_loop);
-};
-#endif
 
 #ifdef WITH_RDMA
 struct rdma_event_channel;
@@ -199,7 +154,6 @@ protected:
     bool use_sync_send_recv = false;
     int min_zerocopy_send_size = DEFAULT_MIN_ZEROCOPY_SEND_SIZE;
     int iothread_count = 0;
-    int max_aes_xts_pool_size = 256;
 
 #ifdef WITH_RDMA
     bool use_rdma = true;
@@ -213,28 +167,27 @@ protected:
     uint64_t rdma_max_sge = 0, rdma_max_send = 0, rdma_max_recv = 0;
     uint64_t rdma_max_msg = 0;
     rdma_event_channel *rdmacm_evch = NULL;
-    std::map<rdma_cm_id*, osd_client_t*> rdmacm_connections;
-    std::map<rdma_cm_id*, rdmacm_connecting_t*> rdmacm_connecting;
+    robin_hood::unordered_flat_map<rdma_cm_id*, osd_client_t*> rdmacm_connections;
+    robin_hood::unordered_flat_map<rdma_cm_id*, rdmacm_connecting_t*> rdmacm_connecting;
 #endif
 
     std::vector<msgr_iothread_t*> iothreads;
-    std::vector<int> read_ready_clients;
-    std::vector<int> write_ready_clients;
+    std::vector<uint64_t> read_ready_clients;
+    std::vector<uint64_t> write_ready_clients;
     // We don't use ringloop->set_immediate here because we may have no ringloop in client :)
     std::deque<osd_op_t*> set_immediate_ops;
 
-    std::vector<op_aes_xts_encrypt_t*> encrypt_ctx_pool;
-    std::vector<op_aes_xts_decrypt_t*> decrypt_ctx_pool;
-
 public:
     timerfd_manager_t *tfd = NULL;
-    ring_loop_t *ringloop = NULL;
+    ring_loop_i *ringloop = NULL;
     bool has_sendmsg_zc = false;
     // osd_num_t is only for logging and asserts
+    uint64_t next_client_id = 1;
     osd_num_t osd_num;
-    std::map<int, osd_client_t*> clients;
-    std::map<osd_num_t, osd_wanted_peer_t> wanted_peers;
-    std::map<uint64_t, int> osd_peer_fds;
+    robin_hood::unordered_flat_map<uint64_t, osd_client_t*> clients;
+    robin_hood::unordered_flat_map<uint64_t, osd_client_t*> osd_peers;
+    robin_hood::unordered_flat_map<int, osd_client_t*> clients_by_fd;
+    robin_hood::unordered_flat_map<osd_num_t, osd_wanted_peer_t> wanted_peers;
     std::vector<std::string> osd_networks;
     std::vector<addr_mask_t> osd_network_masks;
     std::vector<std::string> osd_cluster_networks;
@@ -245,9 +198,11 @@ public:
     osd_op_stats_t stats, recovery_stats;
 
     void init();
+    void init_iothreads();
     void parse_config(const json11::Json & config);
     void connect_peer(uint64_t osd_num, json11::Json peer_state);
-    void stop_client(int peer_fd, bool force = false, bool force_delete = false);
+    void stop_client(uint64_t client_id, bool force_delete = false);
+    void destroy_client(osd_client_t *cl);
     void outbox_push(osd_op_t *cur_op);
     std::function<void(osd_op_t*)> exec_op;
     std::function<void(osd_num_t)> repeer_pgs;
@@ -256,6 +211,7 @@ public:
     void read_requests();
     void send_replies();
     void accept_connections(int listen_fd);
+    void destroy_iothreads();
     ~osd_messenger_t();
 
     static json11::Json::object read_config(const json11::Json & config);
@@ -266,7 +222,7 @@ public:
 
 #ifdef WITH_RDMA
     bool is_rdma_enabled();
-    bool connect_rdma(int peer_fd, std::string rdma_address, uint64_t client_max_msg);
+    bool connect_rdma(uint64_t client_id, std::string rdma_address, uint64_t client_max_msg);
 #endif
 #ifdef WITH_RDMACM
     bool is_use_rdmacm();
@@ -282,44 +238,28 @@ protected:
     void try_connect_peer_tcp(osd_num_t peer_osd, const char *peer_host, int peer_port);
     void handle_peer_epoll(int peer_fd, int epoll_events);
     void handle_connect_epoll(int peer_fd);
-    void on_connect_peer(osd_num_t peer_osd, int peer_fd);
+    void on_connect_peer(osd_num_t peer_osd, int errcode, uint64_t client_id);
     void check_peer_config(osd_client_t *cl);
     void cancel_osd_ops(osd_client_t *cl);
     void cancel_op(osd_op_t *op);
 
     bool try_send(osd_client_t *cl);
     void handle_send(int result, bool prev, bool more, osd_client_t *cl);
-    size_t op_copy_to(osd_client_t *cl, uint8_t *dst, size_t dst_len);
-    void op_get_write_buffers(osd_client_t *cl, std::vector<iovec> & lst);
-
-    void handle_read(int result, osd_client_t *cl);
-    bool handle_read_buffer(osd_client_t *cl, uint8_t *curbuf, size_t bufsize);
-    bool handle_hdr(osd_client_t *cl);
-    bool allocate_op_buffers(osd_client_t *cl);
-    bool allocate_reply_buffers(osd_client_t *cl, osd_op_t *op);
-    size_t op_copy_from(osd_client_t *cl, uint8_t *src, size_t src_len, size_t & done);
-    size_t op_get_read_buffers(osd_client_t *cl, std::vector<iovec> & lst);
-    void handle_finished_op(osd_client_t *cl);
-
-    bool op_encrypted_copy_data_to(osd_client_t* cl, uint8_t *buf, size_t len, size_t from, size_t & done);
-    bool op_decrypted_copy_data_from(osd_client_t* cl, uint8_t *buf, size_t len, size_t from, size_t & done);
-    void op_decrypt_start(osd_client_t* cl);
-    void op_decrypt_inline(osd_client_t* cl);
-    void op_decrypt_free(osd_client_t* cl);
 
+    bool handle_read(int result, osd_client_t *cl);
+    bool handle_read_buffer(osd_client_t *cl, void *curbuf, int remain);
     bool handle_finished_read(osd_client_t *cl);
     void handle_op_hdr(osd_client_t *cl);
     bool handle_reply_hdr(osd_client_t *cl);
     void handle_reply_ready(osd_op_t *op);
     void handle_immediate_ops();
-    void clear_immediate_ops(int peer_fd);
 
 #ifdef WITH_RDMA
     void try_send_rdma(osd_client_t *cl);
-    int try_send_rdma_copy(osd_client_t *cl, uint8_t *dst, int dst_len);
     bool init_recv_rdma(osd_client_t *cl);
     void handle_rdma_events(msgr_rdma_context_t *rdma_context);
     msgr_rdma_context_t* choose_rdma_context(osd_client_t *cl);
+    void destroy_rdma_conn(msgr_rdma_connection_t *rdma_conn);
 #endif
 #ifdef WITH_RDMACM
     void handle_rdmacm_events();