thebestvibedev/tromcho.net
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,931 Commits 74 Branches 106 Tags
db2d9c5b3dfdf26f430e4d7d140e543d6eef76f8
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Vitaliy Filippov db2d9c5b3d Fix tables in NFS doc 2024-10-08 00:20:10 +03:00
Vitaliy Filippov 09f15f44c9 Fix Toshiba MG and VDUSE Debian kernel note in docs 2024-10-08 00:17:14 +03:00
Vitaliy Filippov c5a58c2e81 Support reading parameters automatically from the superblock in vitastor-disk {dump,write}-{meta,journal} 2024-10-07 02:21:58 +03:00
Vitaliy Filippov 30e7c2ad1e Add custom OpenNebula oned.conf patcher (it uses a SHITTY configuration file format) 2024-10-06 13:46:05 +03:00
Vitaliy Filippov 2e76ceabbe Fix iseek option in vitastor-cli dd 2024-10-05 18:25:38 +03:00
Vitaliy Filippov 3df088c207 Validate conv=, iflag=, oflag= options in vitastor-cli dd 2024-10-05 18:02:36 +03:00
Vitaliy Filippov d882a19eab Fix vitastor-disk write-meta not writing header checksum to the disk... 2024-10-05 17:32:55 +03:00
Vitaliy Filippov 702be3da7a Fix JSON format in vitastor-disk dump-meta 2024-10-05 16:08:34 +03:00
Vitaliy Filippov 99533e1c2f Fix .yml links 2024-10-02 00:38:07 +03:00
Vitaliy Filippov a6cceb43bf Fix read_chain_bitmap not working for snapshot in another pool 2024-10-02 00:24:48 +03:00
Vitaliy Filippov 745d89459a Fix link, add title 2024-09-29 22:05:56 +03:00
Vitaliy Filippov 48f023292d Fix extra data reads on read_chain 2024-09-21 17:05:42 +03:00
Vitaliy Filippov b58bf3ada5 Fix possible OSD crash during parallel read & write to an image with snapshots 
OSDs could crash with the following "assertion failed" message (crash didn't affect data
and was caused by OSD thinking upper blocks are full while they weren't). Reproduction
without introducing artificial delays is hard because you have to force OSD to read an
object with enqueued but not handled write which fills previously non-full bitmap. O_o.

```
vitastor-osd: ./src/osd/osd_primary_chain.cpp:613: void osd_t::send_chained_read_results(pg_t&, osd_op_t*): Assertion `stripes[role].read_buf' failed.
```
 2024-09-21 13:44:36 +03:00
Vitaliy Filippov f18a749324 READ_CHAIN fix was incomplete :-) 2024-09-21 13:40:31 +03:00
Vitaliy Filippov 6e9307c522 Fix possible overflow in is_zero() 2024-09-21 13:40:10 +03:00
Vitaliy Filippov 99adbb9483 Release 1.9.1 
Hotfixes for OpenNebula and upgrade hotfix for 1.7

- Fix deploy.vitastor, save.vitastor, restore.vitastor scripts not working for nodes other than master oned
- Fix deploy.vitastor not working for VMs without Vitastor disks
- Disable clearing old PG configuration when upgrading from 1.7 or older versions (it was breaking old clients)
 2024-09-14 19:17:30 +03:00
Vitaliy Filippov b489a611a9 Add 1.8 upgrade note 2024-09-14 19:17:30 +03:00
Vitaliy Filippov c6c0b8957a Stop updating old PG configuration when the user manually deletes it 2024-09-14 19:15:40 +03:00
Vitaliy Filippov 5d40d2a459 Fix oned.conf patch 2024-09-14 19:08:44 +03:00
Vitaliy Filippov f449c28c3b Always write decoded base64 deployment file (otherwise it breaks VMs without Vitastor disks) 2024-09-14 15:25:02 +03:00
Vitaliy Filippov a6274f58cc Same fix for save/restore: they also need to ssh to target node 2024-09-14 02:46:48 +03:00
Vitaliy Filippov ac29ffea6a Add ssh to target node to deploy.vitastor - without it it always tried to deploy VMs on oned host 2024-09-14 02:15:24 +03:00
Vitaliy Filippov bc06acc153 Disable clearing old PG configuration - we can not be sure that old clients do not need it 2024-09-13 19:00:12 +03:00
Vitaliy Filippov fe8e611e23 Release 1.9.0 
- OpenNebula support! [Installation instructions](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/installation/opennebula.en.md)
- Added [vitastor-cli rm --exact|--matching](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#rm) command
- Added [vitastor-cli dd](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#dd) command - copy files between Vitastor images, files and pipes
- Add a startup timeout to vitastor-cli to not wait for etcd infinitely
- Fix non-working OSD_OP_READ_CHAIN_BITMAP O_o
- Autodetect block_size/bitmap_granularity/immediate_commit when creating pools
- Do not allow to create multiple pools with the same name from vitastor-cli
- Fix skip_cache_check option not applied due to type issue (see github issue #70)
 2024-09-06 01:46:16 +03:00
Vitaliy Filippov 7636f9c726 Turn off brp-python-bytecompile in RPM specs 2024-09-06 01:44:44 +03:00
Vitaliy Filippov d5f7005ddd Add dd and rm --exact|--matching documentation 2024-09-05 02:22:05 +03:00
Vitaliy Filippov 70d6fcd32a Add OpenNebula to README 2024-09-05 02:00:14 +03:00
Vitaliy Filippov 17caaa59af vitastor-opennebula is probably more correct than opennebula-vitastor 2024-09-05 01:44:16 +03:00
Vitaliy Filippov 2dac6ee38b Fix OpenNebula reinstall 2024-09-04 11:05:56 +03:00
Vitaliy Filippov 8be67a2d5b Fix OpenNebula save/restore 2024-09-04 11:05:56 +03:00
Vitaliy Filippov 9c2132882c Fix unaligned last block read/write in cli_dd 2024-09-04 11:05:56 +03:00
Vitaliy Filippov 9f25bb059b Use just IMAGE_PREFIX, not IMAGE_PREFIX+"one" 2024-09-04 01:23:00 +03:00
Vitaliy Filippov ee3094c5e5 Add OpenNebula plugin docs 2024-09-04 01:22:39 +03:00
Vitaliy Filippov ba9f263b75 Add wildcard removal command 2024-08-31 14:13:09 +03:00
Vitaliy Filippov 30eaa1a8e6 Add vitastor-cli ls --exact 2024-08-31 02:36:25 +03:00
Vitaliy Filippov 6a8daedbe2 rm --wildcard 2024-08-31 02:36:25 +03:00
Vitaliy Filippov 2b96ac0b44 Implement OpenNebula driver 2024-08-30 23:46:37 +03:00
Vitaliy Filippov 986cd11705 Implement CLI "dd" command - copy data between Vitastor images, files and pipes 2024-08-30 02:31:06 +03:00
Vitaliy Filippov b804051eaf Remove debug print in nbd-proxy 2024-08-30 02:31:06 +03:00
Vitaliy Filippov 3cc326500e Fix non-working OSD_OP_READ_CHAIN_BITMAP O_o 2024-08-30 01:25:05 +03:00
Vitaliy Filippov f848c450a4 Clients should not wait infinitely for etcd to start if it's unavailable 2024-08-28 02:03:35 +03:00
Vitaliy Filippov 4121c66281 Autodetect block_size/bitmap_granularity/immediate_commit when creating pools 2024-08-28 02:03:35 +03:00
Vitaliy Filippov b3716fbe23 Validate pool name when creating a pool 2024-08-28 02:03:35 +03:00
Vitaliy Filippov 97f49d7d94 Fix #70 from github - skip_cache_check type issue 2024-08-14 01:35:43 +03:00
Vitaliy Filippov 131de4b790 Disable trace in header 2024-08-13 11:21:35 +03:00
Vitaliy Filippov ce359c5a69 Release 1.8.0 
Bugfix release, would be 1.7.2, but etcd layout changes mandate it to be 1.8.0. :-)

- Change etcd layout: /config/pgs is now /pg/config, /pg/stats/* is now /pgstats/*.
  This is required to fix a rare PG history tracking issue caused by non-atomic
  delivery of etcd events sometimes resulting in `incomplete` objects in EC pools
  after mass OSD restarts. Upgrading can be performed freely, downgrade requires
  additional action: [1.8.0 to 1.7.1](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/admin.en.md#1-8-0-to-1-7-1)
- Fix a rare client hang on PG primary OSD switch
- Fix vitastor-nfs started using mount command sometimes not stopping automatically after unmount
- Fix vitastor-nfs mounts started using mount command sometimes hanging after daemonizing
- Fix merge/flatten into a pool with different object size (image migration between pools case)
- Do not print extra "PG disappeared after reload" verbose log messages for non-existing PGs
- Fix clustered Antietcd support and persistence filter
- Do not try to purge the same OSD multiple times if its multiple devices are passed to purge
- Various node.js binding fixes
 2024-08-11 14:28:31 +03:00
Vitaliy Filippov 521e867b10 Run check_exit also on deferred stop. Now vitastor-nfs should finally always stop on umount 2024-08-11 00:05:20 +03:00
Vitaliy Filippov 333c54ebbf Cleanup clients correctly during stop(). Was also affecting #67, but could also reproduce during normal operation 2024-08-11 00:00:13 +03:00
Vitaliy Filippov 58d3da95c8 Fix github issue #67 by closing active NFS sockets before daemonize() 2024-08-10 20:13:37 +03:00
Vitaliy Filippov 4e90e752eb Fix merge/flatten into a pool with different object size 2024-08-10 19:23:26 +03:00
Vitaliy Filippov 09342d7189 node.js binding fixes 2024-08-05 00:10:37 +03:00
Vitaliy Filippov eb3e8b8c19 Do not print "PG disappeared after reload" verbose log messages when *it* was not present 2024-08-04 01:42:05 +03:00
Vitaliy Filippov e2ca3ad99e Add a note about storage ID in proxmox storage config doc 2024-07-31 01:19:44 +03:00
Vitaliy Filippov dd4b0aed2b Support scattered write in node.js binding 2024-07-31 01:17:06 +03:00
Vitaliy Filippov 42851a061c Always continue operations to not miss resuming after the lack of PG primary 
Should fix spurious client hangs during PG primary switchover
 2024-07-31 01:17:03 +03:00
Vitaliy Filippov 8e0f242d30 Add downgrade docs 2024-07-31 01:15:37 +03:00
Vitaliy Filippov 0daa8ea39b Support seamless upgrade to new PG config and stats etcd key names 2024-07-31 01:15:37 +03:00
Vitaliy Filippov b263d311ef Use separate watch revisions for different watchers 2024-07-31 01:15:37 +03:00
Vitaliy Filippov 8720185780 Run tests in CI in memory (in tmpfs) 2024-07-31 01:15:37 +03:00
Vitaliy Filippov 20584414d8 Report OSD version in /osd/state/ and /osd/stats/ (for the future) 2024-07-31 01:15:37 +03:00
Vitaliy Filippov 306a3db7f3 Rename VERSION define to VITASTOR_VERSION 2024-07-31 01:15:37 +03:00
Vitaliy Filippov 5b0aebada4 Rename /config/pgs to /pg/config and /pg/stats/* to /pgstats/* 2024-07-31 01:15:37 +03:00
Vitaliy Filippov d6f0b480c8 Fix broken link 2024-07-22 14:01:53 +03:00
Vitaliy Filippov f1f8531fd4 Make tests compatible with antietcd, add 2 antietcd tests to CI 2024-07-20 02:16:38 +03:00
Vitaliy Filippov 8d79d59964 Update antietcd to 1.1.0 2024-07-20 02:15:48 +03:00
Vitaliy Filippov 551a209a50 Fix persistence filter initialization 2024-07-20 02:15:48 +03:00
Vitaliy Filippov 06cafd7702 Do not merge config an extra unneeded time 2024-07-20 02:15:48 +03:00
Vitaliy Filippov 3018352443 Fix clustered Antietcd support 2024-07-19 18:58:58 +03:00
Vitaliy Filippov f8edfb4a71 No need to check for PG intersection if a history set is smaller than EC data part count 2024-07-18 19:29:05 +03:00
Vitaliy Filippov 8239ea2356 Do not try to purge the same OSD multiple times if its multiple devices are passed to purge 2024-07-16 16:48:16 +03:00
Vitaliy Filippov e898335b8d Release 1.7.1 
Some stupid hotfixes for 1.7.0 :)

- Fix NFS mount
- Fix modify-osd
- Fix use_antietcd not taken from /etc
 2024-07-16 00:07:03 +03:00
Vitaliy Filippov e7869611fa Another stupid fix for NFS (no idea how it worked for me) 2024-07-16 00:05:51 +03:00
Vitaliy Filippov e1c2500b60 Use modify-osd in the disk removal instruction 2024-07-16 00:01:42 +03:00
Vitaliy Filippov 42cf3a11df Oops, fix reweight :) 2024-07-16 00:01:11 +03:00
Vitaliy Filippov 4d9293f0e9 Fix QEMU 8.2 and 9.0 patches (add @location comments) 2024-07-15 16:30:14 +03:00
Vitaliy Filippov 7a13f85ae2 Fix mon config merge 2024-07-15 16:25:22 +03:00
Vitaliy Filippov fc219b8602 Add pg-list to docs 2024-07-15 13:29:22 +03:00
Vitaliy Filippov 989d73f874 Release 1.7.0 
Omnidirectional release

New features:

- Support handling TCP I/O in simple separate io_uring-based [I/O threads](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/config/client.en.md#client_iothread_count) - may increase linear performance to 7-8 GB/s
- Experimental internal etcd replacement - [antietcd](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/config/monitor.en.md#use_antietcd)
- Monitor now has a [built-in Prometheus exporter](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/config/monitor.en.md#enable_prometheus)
- Added a reference [Grafana dashboard](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/mon/scripts/Vitastor-Grafana-6+.json)
- Implement vitastor-cli [osd-tree](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#osd-tree) and [ls-osd](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#ls-osd) commands
- Implement vitastor-cli [modify-osd](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#modify-osd) command
- Implement vitastor-cli [pg-list](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#pg-list) command
- Implement [VitastorFS defragmentation](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/nfs.en.md#defrag)
- Implement basic node.js binding (not published on npm yet)

Changes:

- Make immediate_commit=all the default everywhere to match default vitastor-disk behaviour
- Make pool-create error message more obvious and add details to it
- Set default etcd_ws_keepalive_interval to 5 seconds (speedup client etcd failover)
- Support OpenStack 2023.2 in Nova and Cinder drivers/patches
- Add patches for libvirt 10.x
- Add patches for QEMU 8.2 and 9.0
- Implement internal restart / run_forever in monitor
- Some source tree refactoring - sources are now moved into subdirectories, monitor is now split into multiple files
- Add vitastor_c_inode_get_immediate_commit in vitastor_c client library
- Make vitastor_kv.h header public

Bug fixes:

- Fix total statistics usec/count/bytes not being reported when delta (bps/iops/lat) is zero
- Prevent infinite loop in NFS on files with incorrect metadata pointing to an empty volume
- Fix READDIR offsets (cookies) in VitastorFS sometimes leading to client infinite loops when reading a directory
- Fix a rare infinite loop during OSD journal flushing (OSD hanging and eating 100 % CPU)
- Fix several bugs which could lead to lost writes in setups without immediate_commit:
  - Client library treated writes as completed before actually completing them, thus missing them in a subsequent fsync
  - Client library didn't repeat writes on the new PG primary when it changed
  - OSDs didn't drop peer connections with dirty writes when stopping PG
- Fix Block Pseudo-FS initialization leading to ENOENTs some time after start
- Fix vitastor-cli merge-based commands (merge/flatten/rm snapshot) slowing down and finally failing when using CAS optimistic locks
- Fix pool create/modify --block_size validation
- Fix TTL comparison for determining failed lease/keepalive requests in OSD
- Add support for size suffixes in pool-create --block_size and --immediate_commit values
 2024-07-15 11:48:35 +03:00
Vitaliy Filippov f0630722ce Make pool-create error message more obvious, add details 2024-07-15 11:47:49 +03:00
Vitaliy Filippov 93b0947720 Support size suffixes in pool-create --block_size / --bitmap_granularity 2024-07-15 11:47:05 +03:00
Vitaliy Filippov 9c628646fa Remove bullseye-backports from build, remove buster-backports from docs 2024-07-15 11:47:05 +03:00
Vitaliy Filippov cf476a3b95 Add mkdir /var/lib/vitastor 2024-07-15 11:47:05 +03:00
Vitaliy Filippov 23f9273ba3 Take use_antietcd setting from /etc/vitastor/vitastor.conf too 2024-07-15 02:02:56 +03:00
Vitaliy Filippov 74b88bf8ba Use own repo instead of buster-backports as it is EOL 2024-07-14 20:25:44 +03:00
Vitaliy Filippov 1254d5a0de Fix delta stats when counters may be hypothetically reset 2024-07-14 13:11:00 +03:00
Vitaliy Filippov f87bece253 Fix build with antietcd & tinyraft, remove some version hardcode 2024-07-14 13:04:25 +03:00
Vitaliy Filippov ba85d0ef16 Add vitastor_kv.h to RPM specs 2024-07-14 11:20:37 +03:00
Vitaliy Filippov 17a909ea3a Stop metrics/future API HTTP server when closing Monitor instance 2024-07-14 11:16:41 +03:00
Vitaliy Filippov a4dfc220ab Implement basic node.js binding (not published on npm yet) 2024-07-14 10:58:38 +03:00
Vitaliy Filippov 26426dd95e Return it back, but fix stats in another way 2024-07-13 19:14:34 +03:00
Vitaliy Filippov 9f38b7e5c1 Fix osd_ping_time_remaining reset from 990c3ba7eb, leading to osd disconnections 2024-07-13 16:09:56 +03:00
Vitaliy Filippov 20057defbe Revert 8ad63465cd 2024-07-13 15:34:34 +03:00
Vitaliy Filippov b4e9140755 Add defrag docs, fix trace message 2024-07-13 00:45:53 +03:00
Vitaliy Filippov 413959e75a Prevent infinite loop in NFS - return EIO when an inode points to an incorrect volume position 2024-07-12 20:53:54 +03:00
Vitaliy Filippov 8973982570 Delete keys from internal state instead of setting them to null on DELETE event in mon 2024-07-12 16:42:21 +03:00
Vitaliy Filippov 990c3ba7eb Implement FS defragmentation 2024-07-12 16:11:35 +03:00
Vitaliy Filippov 1771d2ef36 Fix READDIR cookie/offset bug 2024-07-12 16:11:35 +03:00
Vitaliy Filippov d88ab76636 Fix active mon stat 2024-07-11 01:34:59 +03:00
Vitaliy Filippov c010a0aa54 Fix OSD "local write" latency sum 2024-07-11 01:30:03 +03:00
Vitaliy Filippov 0d42712d29 Fix refresh in dashboard variable 2024-07-11 01:13:02 +03:00