thebestvibedev/tromcho.net
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,863 Commits 74 Branches 106 Tags
f8edfb4a717afe0a5be8fef7e2d5a1af41e647c8
Commit Graph
891 Commits
Author SHA1 Message Date
Vitaliy Filippov f8edfb4a71 No need to check for PG intersection if a history set is smaller than EC data part count 2024-07-18 19:29:05 +03:00
Vitaliy Filippov 8239ea2356 Do not try to purge the same OSD multiple times if its multiple devices are passed to purge 2024-07-16 16:48:16 +03:00
Vitaliy Filippov e898335b8d Release 1.7.1 
Some stupid hotfixes for 1.7.0 :)

- Fix NFS mount
- Fix modify-osd
- Fix use_antietcd not taken from /etc
 2024-07-16 00:07:03 +03:00
Vitaliy Filippov e7869611fa Another stupid fix for NFS (no idea how it worked for me) 2024-07-16 00:05:51 +03:00
Vitaliy Filippov 42cf3a11df Oops, fix reweight :) 2024-07-16 00:01:11 +03:00
Vitaliy Filippov 989d73f874 Release 1.7.0 
Omnidirectional release

New features:

- Support handling TCP I/O in simple separate io_uring-based [I/O threads](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/config/client.en.md#client_iothread_count) - may increase linear performance to 7-8 GB/s
- Experimental internal etcd replacement - [antietcd](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/config/monitor.en.md#use_antietcd)
- Monitor now has a [built-in Prometheus exporter](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/config/monitor.en.md#enable_prometheus)
- Added a reference [Grafana dashboard](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/mon/scripts/Vitastor-Grafana-6+.json)
- Implement vitastor-cli [osd-tree](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#osd-tree) and [ls-osd](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#ls-osd) commands
- Implement vitastor-cli [modify-osd](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#modify-osd) command
- Implement vitastor-cli [pg-list](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#pg-list) command
- Implement [VitastorFS defragmentation](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/nfs.en.md#defrag)
- Implement basic node.js binding (not published on npm yet)

Changes:

- Make immediate_commit=all the default everywhere to match default vitastor-disk behaviour
- Make pool-create error message more obvious and add details to it
- Set default etcd_ws_keepalive_interval to 5 seconds (speedup client etcd failover)
- Support OpenStack 2023.2 in Nova and Cinder drivers/patches
- Add patches for libvirt 10.x
- Add patches for QEMU 8.2 and 9.0
- Implement internal restart / run_forever in monitor
- Some source tree refactoring - sources are now moved into subdirectories, monitor is now split into multiple files
- Add vitastor_c_inode_get_immediate_commit in vitastor_c client library
- Make vitastor_kv.h header public

Bug fixes:

- Fix total statistics usec/count/bytes not being reported when delta (bps/iops/lat) is zero
- Prevent infinite loop in NFS on files with incorrect metadata pointing to an empty volume
- Fix READDIR offsets (cookies) in VitastorFS sometimes leading to client infinite loops when reading a directory
- Fix a rare infinite loop during OSD journal flushing (OSD hanging and eating 100 % CPU)
- Fix several bugs which could lead to lost writes in setups without immediate_commit:
  - Client library treated writes as completed before actually completing them, thus missing them in a subsequent fsync
  - Client library didn't repeat writes on the new PG primary when it changed
  - OSDs didn't drop peer connections with dirty writes when stopping PG
- Fix Block Pseudo-FS initialization leading to ENOENTs some time after start
- Fix vitastor-cli merge-based commands (merge/flatten/rm snapshot) slowing down and finally failing when using CAS optimistic locks
- Fix pool create/modify --block_size validation
- Fix TTL comparison for determining failed lease/keepalive requests in OSD
- Add support for size suffixes in pool-create --block_size and --immediate_commit values
 2024-07-15 11:48:35 +03:00
Vitaliy Filippov f0630722ce Make pool-create error message more obvious, add details 2024-07-15 11:47:49 +03:00
Vitaliy Filippov 93b0947720 Support size suffixes in pool-create --block_size / --bitmap_granularity 2024-07-15 11:47:05 +03:00
Vitaliy Filippov a4dfc220ab Implement basic node.js binding (not published on npm yet) 2024-07-14 10:58:38 +03:00
Vitaliy Filippov 9f38b7e5c1 Fix osd_ping_time_remaining reset from 990c3ba7eb, leading to osd disconnections 2024-07-13 16:09:56 +03:00
Vitaliy Filippov b4e9140755 Add defrag docs, fix trace message 2024-07-13 00:45:53 +03:00
Vitaliy Filippov 413959e75a Prevent infinite loop in NFS - return EIO when an inode points to an incorrect volume position 2024-07-12 20:53:54 +03:00
Vitaliy Filippov 990c3ba7eb Implement FS defragmentation 2024-07-12 16:11:35 +03:00
Vitaliy Filippov 1771d2ef36 Fix READDIR cookie/offset bug 2024-07-12 16:11:35 +03:00
Vitaliy Filippov 66b438106a Add vitastor-cli pg-list command 2024-07-10 02:27:41 +03:00
Vitaliy Filippov 3aef6682fb Add vitastor-cli modify-osd command 2024-07-09 16:52:19 +03:00
Vitaliy Filippov 79f1d1969b Make immediate_commit=all the default 2024-07-07 11:45:18 +03:00
Vitaliy Filippov 918e1f83b0 Add JSON output for ls-osd 2024-07-07 02:24:36 +03:00
Vitaliy Filippov abbba6ade4 Support handling TCP I/O in simple separate io_uring-based I/O threads 
Required mainly for clients, allows to scale parallel client I/O with TCP
from 100-150k iops to ~400k iops and from 2-3 GB/s to at least 7-8 GB/s
with 4 I/O threads, at the same time increasing Q=1 latency by 2x thread
switching delay, which is ~10 us when CPU powersaving is disabled and may
be as high as 200 us when it's enabled.
 2024-07-04 13:29:20 +03:00
Vitaliy Filippov 21d1171ba4 Fix parsing after "slightly decopypasting" :) 2024-06-29 00:09:30 +03:00
Vitaliy Filippov ca63cd507d Fix possible infinite loop in flusher (surprisingly reproduced in test_write.sh with iothreads) 2024-06-27 00:38:01 +03:00
Vitaliy Filippov ea0d72289c Treat copied buffers as written only after completing the write in client 
SYNC operation fsyncs only completed operations, so treating writes as "eligible
for fsync" before actually completing them is incorrect

It affected SCHEME=ec test_heal.sh (with immediate_commit=none) test - it was
flapping with lost writes - some non-fsynced writes were legitimately lost by
the OSD, but weren't repeated by the client
 2024-06-20 02:11:53 +03:00
Vitaliy Filippov e400a851f4 Repeat dirty buffer flushes on any PG primary change because the new primary may not know about unfinished operations of the old primary 2024-06-19 00:28:26 +03:00
Vitaliy Filippov 0fec7a9fea Drop dirty peer connections also when stopping PG to guarantee that clients do not miss fsync 2024-06-19 00:28:26 +03:00
Vitaliy Filippov b9de2a92a9 Print OSD performance stats 2024-06-17 13:02:58 +03:00
Vitaliy Filippov 5360a70853 Make OSD also report derived stats 2024-06-17 13:02:52 +03:00
Vitaliy Filippov 4c2328eb13 Implement ls-osd command 2024-06-17 02:22:14 +03:00
Vitaliy Filippov 313daef12d Slightly decopypaste etcd key parsing 2024-06-17 01:38:42 +03:00
Vitaliy Filippov ad9c12e1b9 Fix Pseudo-FS initialization leading to ENOENTs some time after start 2024-06-16 23:43:09 +03:00
Vitaliy Filippov 4473eb5512 Fix slow & failing CAS layer merge 2024-06-14 02:15:49 +03:00
Vitaliy Filippov 6501abc060 Set default etcd_ws_keepalive_interval to 5 2024-06-08 00:38:48 +03:00
Vitaliy Filippov 50e56b3b92 Add vitastor_c_inode_get_immediate_commit 2024-05-19 01:57:18 +03:00
Vitaliy Filippov a12d328793 Rename cli/ to cmd/, fix cmake install 2024-05-15 23:04:50 +03:00
Vitaliy Filippov c79b38bd26 Move all sources to subdirs 2024-05-15 11:06:01 +03:00
Vitaliy Filippov 44692d148a Make vitastor_kv.h header public 2024-05-15 01:49:38 +03:00
Vitaliy Filippov ba52359611 Fix last master commit 2024-05-15 01:49:31 +03:00
Vitaliy Filippov 23a9aa93b5 Fix pool create/modify --block_size validation 2024-05-04 16:33:22 +03:00
Vitaliy Filippov 2412d9e239 Fix TTL comparison for lease/keepalive 2024-04-30 01:53:05 +03:00
Vitaliy Filippov 9301c857b1 Release 1.6.1 
A bunch of monitor fixes

- Add noout flag for OSDs (/vitastor/config/osd/xx)
- Fix "effective" size of degraded PGs (and thus "used space") calculation in monitor
- Fix monitor not clearing PGs of deleted pools
- Fix incorrect PG generation with hosts with 0 OSDs
- Fix monitor crashing during primary OSD recheck when pool has no PGs
- Fix monitor crashing when node_placement included non-existing OSDs
- Fix possible data movement after removing OSDs reweighted to 0
- Remove extra empty keys from pool configurations created by vitastor-cli create-pool
- Fix 32-bit build
 2024-04-22 02:01:29 +03:00
Vitaliy Filippov 3094358ec2 Fix autovivification leading to extra empty keys in pool-create 2024-04-20 02:04:09 +03:00
Vitaliy Filippov 266d038b11 Fix 32-bit build warnings and one error again :-) 2024-04-11 22:49:33 +03:00
Vitaliy Filippov ff4414d37e Release 1.6.0 
New features:

- Implement "hierarchical failure domains" and other complex distribution rules, for example
  EC 4+2 over 3 DC, with 2 chunks per each DC ([documentation](docs/config/pool.en.md#level_placement))
- Make OSDs handle ENOSPC - now cluster stays online even if some OSDs fill up
  to 100 %, only writes requiring free space hang
- Implement Stage/Unstage & volume locking for CSI to prevent parallel mounting
  and/or modifications of the same volume
- Warn about full and almost full OSDs in vitastor-cli status
- Add an experimental NBD netlink map mode as an option ([documentation](docs/usage/nbd.en.md))
- Add --pg parameter to vitastor-cli describe, print objects with 0x in human-readable format too
- Add [administration docs](docs/usage/admin.en.md)

Bug fixes:

- Fix client operation retry timeout - previously the timeout wasn't applied and writes were
  retries almost instantly
- Fix monitors crashing on invalid pool configurations
- Fix journaling - make each journal write wait for all previous journal writes
- Fix monitor thinking that OSD weight is 0 after deleting /osd/config/ key online
- Fix a write stall caused by flusher possibly not trimming journal on rollback
- Set 32k csum_block_size for HDD by default in vitastor-disk
 2024-04-09 16:57:59 +03:00
Vitaliy Filippov dff4879c8c Check if NBD_ATTR_BACKEND_IDENTIFIER is defined 2024-04-09 13:16:58 +03:00
Vitaliy Filippov af9a853db6 Move NBD netlink map&unmap to separate commands, add "netlink-revive" command 2024-04-08 16:34:41 +03:00
Vitaliy Filippov b7a3275af3 Make netlink optional 2024-04-08 01:51:28 +03:00
Vitaliy Filippov 64c5c4ca26 Fix code style 2024-04-08 01:35:03 +03:00
idelsonandVitaliy Filippov 442a9d838d nbd-proxy: add configuration via netlink to support kinds of timeouts. 
PR #58 - https://github.com/vitalif/vitastor/pull/58/commits

By MIND Software LLC

By submitting this pull request, I accept Vitastor CLA
 2024-04-08 00:50:08 +03:00
Vitaliy Filippov 6366972fe8 Warn about full and almost full OSDs in status 2024-04-07 19:39:51 +03:00
Vitaliy Filippov 2b863fb715 Add ENOSPC handling tests 2024-04-07 19:39:33 +03:00
Vitaliy Filippov 3bf4dd5abd Fix client op retry timeout - do not retry immediately 2024-04-07 19:08:36 +03:00