thebestvibedev/tromcho.net
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,711 Commits 74 Branches 106 Tags
c9dfd0f67d8291ecf7769426f1cb4cfd70bb39f0
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Vitaliy Filippov c9dfd0f67d Fix possible buffer overflow in NFS-RDMA 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 84919a10a9 Extract msgr_iothread to separate .{cpp,h} 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 51ae4d6e24 Remove extra assert in heap after reworking recheck 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 572b20fedc Fix mark_partial_write for replicas (TODO: try to write an OSD unit test for it) 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 4e2724b28f Remove duplicate subop submit/fail code 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 768b1675f8 Do not use ISA-L in test_crc32c 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 38fa722725 Remove the rest of tcmalloc from build 2026-05-18 01:17:56 +03:00
Vitaliy Filippov e56d83fb7f Remove heap->v1 conversion code (untested, unused and probably buggy) 2026-05-18 01:17:56 +03:00
Vitaliy Filippov ff95a85875 Add bound check to je_crc32 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 98203568a8 Handle ENOENT from add_commit 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 89df98ee08 Deep copy pg_history and prev_pgs before mutation 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 0007a831b6 Add return to handle_flush_op when stopping the PG 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 40517c335f Fix absolute bmp_end 2026-05-18 01:17:56 +03:00
Vitaliy Filippov c9f7308b6a Wrap this submit_ into n_subops > 0 too 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 85c7e3bde0 Fix inverted writeback in ublk 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 4fb55b3535 Rework recheck_small_writes to free multiple bad entries more correctly and not use garbage as "already freed" flag 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 912aca11a3 Improve test_recheck to check all possible combinations 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 7b454bd16c Do not try to free snapshot_bitmaps 2026-05-18 01:17:56 +03:00
Vitaliy Filippov a0c8be46a4 Speedup blockstore_heap loading by postponing only objects with a lot of versions 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 53b4329fac Add missing active_flushers-- 2026-05-18 01:17:56 +03:00
Vitaliy Filippov a7f41c4a12 Add assert to reconstruct_stripes_ec 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 5d78057ac3 Fix cur_op->buf memory leak on EIO/EDOM reads(?) 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 8efc5a353f Fix pg_state_bits overwriting PG_HAS_CORRUPTED 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 603b26b896 Add a FIXME about delete buffering 2026-05-18 01:17:56 +03:00
Vitaliy Filippov a3b0fe0deb Fix op stats not tracked for OP_WRITE_STABLE 2026-05-18 01:17:56 +03:00
Vitaliy Filippov f504e356d5 Fix schedule_next_recheck_at 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 4ed17b7070 Sync metadata after writing out recheck_modified_blocks 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 1fd2819724 Fix marking only first OSD from non-majority group as corrupted during scrub 
Note: Could be tested with 5 replicas - 3 good and 2 corrupted if we had OSD unit tests
 2026-05-18 01:17:56 +03:00
Vitaliy Filippov dcdabbc1ec Fix vitastor-disk resize error return codes missed 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 625d5b7b9e Fix punch_holes not updating entry crc (bug when perfect_csum_update==true) 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 9e507fd333 Fix typo in crc32_pad 
Didn't affect anything though because it was only skipping < 4096 byte remainders
 2026-05-18 01:17:56 +03:00
Vitaliy Filippov c2b5118127 Add data device fsyncs during compaction (forgotten during initial heap store dev) 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 4b926e2223 Add a test for missing data device fsyncs 2026-05-18 01:17:56 +03:00
Vitaliy Filippov a5d9a6996a Do not try to increment ref_count if recheck_state is null 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 0ee03e7172 Add reply == request opcode check (also just in case) 2026-05-18 01:17:56 +03:00
Vitaliy Filippov 88b7d9afcd Expect to receive bitmap only for successfull response 2026-05-18 01:17:56 +03:00
Vitaliy Filippov f271c8450c Block sync by all previous syncs 2026-05-11 16:33:49 +03:00
Vitaliy Filippov f78d7d4efc Do not capture client pointer in ping lambda 2026-05-11 16:30:01 +03:00
Vitaliy Filippov fdaf7c88ff Fix RDMA send buffer wraparound - could only lead to slower sending but not to corruption because the bug was canceled by send_out_pos >= send_done_pos check 2026-05-11 16:28:03 +03:00
Vitaliy Filippov 2fb6eb0c30 Free scrub_cur_list.buf when turning on no_scrub (potential memory leak) 2026-05-11 16:15:48 +03:00
Vitaliy Filippov 36d2b56208 Deep copy merged_history, also just in case 2026-05-11 16:14:03 +03:00
Vitaliy Filippov 14b22f2ba9 Retry even fatal errors for OP_SYNC (no real-world issue, just in case) 2026-05-11 16:08:14 +03:00
Vitaliy Filippov fe8b1fe0cc Add assert in cluster_client writeback to prevent overflow when incorrect operation data is passed 2026-05-11 16:06:19 +03:00
Vitaliy Filippov 1ec963e468 Fix ec_find_good for misplaced objects with extra corrupted chunks 2026-05-11 15:57:21 +03:00
Vitaliy Filippov 5100f822d8 Wrap all submit_primary_xxx_subops waits with if (n_subops) > 0 2026-05-11 15:17:50 +03:00
Vitaliy Filippov 7432494e88 Remove _LARGEFILE64_SOURCE warnings 2026-05-11 14:59:20 +03:00
Vitaliy Filippov d0c0f3ea39 Do not ignore cur_primary change on PG state validation errors 2026-05-11 14:59:20 +03:00
Vitaliy Filippov f61190f31d Fix send_list/outbox desync on failed SEC_READ_BMP (possible memory leak) 2026-05-11 14:59:11 +03:00
Vitaliy Filippov 3dc0ab5c33 Clear inconsistent_objects on repeer 2026-05-11 14:59:08 +03:00
Vitaliy Filippov de96efed2f Fix 2^(64-PG_EPOCH_BITS) - 1 check 2026-05-11 14:59:08 +03:00
Vitaliy Filippov 87a5230798 Add forgotten header_csum check to the new store 2026-05-11 14:59:08 +03:00
Vitaliy Filippov 0c5e6d4346 Do not use modified_block(ui32) to store lsn(ui64) 2026-05-11 14:59:08 +03:00
Vitaliy Filippov b278087410 Fix assert in blockstore_heap 2026-05-11 13:01:03 +03:00
Vitaliy Filippov a8e821b13b Mark OSDs as dirty during deletes more correctly 2026-05-11 12:56:23 +03:00
Vitaliy Filippov caa70317fa Add a workaround for "double claim" bug with the new store 2026-05-10 17:00:36 +03:00
Vitaliy Filippov b8eaaabfe4 Add raw-ls command for troubleshooting 2026-05-10 17:00:36 +03:00
Vitaliy Filippov e4d80c415e Always check pool resharding state also for stabilize/rollback 2026-05-10 13:22:02 +03:00
Vitaliy Filippov 553191c3ff Add asserts to the new store too O:-) 2026-05-10 11:39:54 +03:00
Vitaliy Filippov ab385252b5 Add offset asserts before each write 2026-05-10 10:56:32 +03:00
Vitaliy Filippov 041185c673 Release 3.0.11 
A single hotfix:

- Fix missing error handling in EC snapshot/cloned image reads potentially
  leading to corrupted reads during OSD/PG restarts
 2026-05-09 18:49:36 +03:00
Vitaliy Filippov b03ac80a57 Fix missing error handling in chained read sec_read_bmp handling 2026-05-07 15:01:49 +03:00
Vitaliy Filippov 2ba56074f9 Run copy_part_bitmaps only after all subops succeed (slightly more correct) 2026-05-07 15:01:40 +03:00
Vitaliy Filippov 4acfe149cb Release 3.0.10 
Important bug fixes (new store):
- Fix OSDs possibly refusing to start with "write metadata failed at offset xxx: Invalid argument"
  (fix buffer alignment during initial garbage collection)
- Rollback change from 3.0.4 - on-disk garbage entries are not skipped on start again. This change
  doesn't have any impact normally, but OSDs originally running 3.0.0-3.0.2 and then upgraded
  to 3.0.9 may hit a bug where 3.0.9 refuses to start due to entries marked as garbage too
  early and flushed to disk in 3.0.0-3.0.2.

Other changes:
- Auto-select the only RDMA device/port if there is only one
- Rollback one 3.0.9 change - there was no actual use-after-free :)
  (the problem was only relevant to an unstable development version)
- Fix `vitastor-nfs --trace` option
- Fix an unintended 1 second sleep in vitastor-cli rm-data
- Fix inode statistics not being cleared for a deleted pool
- Fix print to stdout in client
 2026-04-27 13:53:48 +03:00
Vitaliy Filippov 008ed5b269 Rollback 25ecca7625 - there was no actual use-after-free :) 2026-04-22 01:35:38 +03:00
Vitaliy Filippov 4fffe0f032 Fix vitastor-nfs trace option 2026-04-20 21:51:47 +03:00
Vitaliy Filippov a76d5ccc0d Wakeup callers in rm-data 2026-04-17 13:53:32 +03:00
Vitaliy Filippov 8ed1e180e0 Clear inode_stats in mon 2026-04-14 02:44:46 +03:00
Vitaliy Filippov 8832fc3b14 Fix print to stdout in client 2026-04-14 02:43:50 +03:00
Vitaliy Filippov 0134934c99 Do not skip garbage entries on start (rollback change from 3.0.4) 2026-04-11 12:05:02 +03:00
Vitaliy Filippov 2e36f292bd Fix buffer alignment during metadata clearing on init 2026-04-10 21:56:12 +03:00
Vitaliy Filippov bcc6419760 Auto-select the only RDMA device/port if there is only one 2026-04-08 15:46:10 +03:00
Vitaliy Filippov dd5941b9a4 Release 3.0.9 
- Fix broken garbage collection in the new store leading to very high memory usage
- Make the new store do full garbage collection on OSD startup by default (can be turned off with a new gc_on_start parameter)
- Make startup logging with the new store more informative, allow to log amount of garbage with log_level > 1
- Fix clients reading from snapshots incorrectly on retries (for example when OSDs were restarted during read)
- Fix a possible "send-after-free" in OSD which could probably also affect snapshot reads
- Fix OSD crashing when handling a read_bitmap operation from a deleted object with the new store
 2026-04-04 18:38:40 +03:00
Vitaliy Filippov 4005b88865 Do not use node_type 2026-04-04 17:37:47 +03:00
Vitaliy Filippov 280b5cd675 Add gc_on_start docs 2026-04-04 17:19:32 +03:00
Vitaliy Filippov e5c505eaf4 Write 1 entry to blocks to collect garbage 2026-04-03 18:08:12 +00:00
Vitaliy Filippov c1d244d4f0 Fix init metadata entry loading logging 2026-04-03 18:08:12 +00:00
Vitaliy Filippov 9b264a212f Do full metadata GC on start (new store) 2026-04-03 18:08:12 +00:00
Vitaliy Filippov ff7f5cb4f4 Report live & garbage entry count / memory with log_level > 1 2026-04-02 23:45:13 +00:00
Vitaliy Filippov 25ecca7625 Fix use-after-free in OSD ops (ooops) 2026-04-02 13:48:17 +00:00
Vitaliy Filippov 99c4244004 Try to write into blocks with more garbage first 2026-04-02 13:48:17 +00:00
Vitaliy Filippov 9949b9fb4e Fix incorrect reads from snapshots on retries 2026-03-31 02:06:53 +03:00
Vitaliy Filippov e6881ad1d5 Fix read_bitmap from deleted objects with the new store 2026-03-31 01:59:53 +03:00
Vitaliy Filippov b30635b932 Release 3.0.8 
- Surrogate increment-only peer IDs are now used instead of file descriptor numbers,
  and client cleanup sequence is reworked in another attempt to fix a possible PG hang
  in 'peering' states. The history of this bug is still unclear and it still hasn't been
  reproduced in tests, but several users have reported it since 3.0.4 (only with enabled RDMA).
- Fix a minor bug where an OSD could skip reporting misplaced PG state when moving
  an object in non-immediate_commit mode.
- Add Ubuntu 26.04 (Resolute Raccoon) support and patches for qemu 10.2 and libvirt 12.0
 2026-03-30 02:09:02 +03:00
Vitaliy Filippov 0c1154833c Add Ubuntu 26.04 support 2026-03-30 02:06:10 +03:00
Vitaliy Filippov c227bb05b6 Fix incorrect patch contents for qemu 10.1, add patch for qemu 10.2 2026-03-30 02:06:10 +03:00
Vitaliy Filippov dd85315f22 Update CMake supported versions 2026-03-30 02:06:10 +03:00
Vitaliy Filippov 47d2f4e0be Add patch for libvirt 12.0 2026-03-30 02:06:10 +03:00
Vitaliy Filippov 2a5028d17f Use robin_hood_map in messenger 2026-03-30 02:06:10 +03:00
Vitaliy Filippov 07915c2881 Use surrogate peer IDs instead of FDs 2026-03-30 02:06:10 +03:00
Vitaliy Filippov 79141eb383 Fix commented out //this->pg_state_dirty.insert 2026-03-28 20:08:52 +03:00
Vitaliy Filippov f7cbb6ed56 Remove FD from the event loop in destroy_client, not in stop_client 2026-03-28 14:18:20 +03:00
Vitaliy Filippov 8f8172db99 clear_immediate_ops is now done by stop_client 2026-03-26 01:08:23 +03:00
Vitaliy Filippov 94be147e80 Clear received_ops - they are already freed 2026-03-26 01:08:02 +03:00
Vitaliy Filippov 538620b400 Release 3.0.6 
- Fix one more bug in old store checksums O_o - checksum verification wasn't
  working correctly on reads quickly after small initial offsetted writes
  (when a BIG_WRITE was read from the journal).
- Fix clients possibly not destroyed after being switched to RDMA. Not 100%
  verified, but in theory it could rarely lead to PGs hanging in peering/stopping
  states with RDMA enabled.
- Use vitastor-cli in the Cinder driver for modifications instead of direct etcd
  interaction.
- Fix mon_https_ca parsing.
- Fix Docker install.sh, add instructions for Podman.
 2026-03-22 22:49:51 +03:00
Vitaliy Filippov 4c34a47179 Fix str_replace 2026-03-19 01:19:29 +03:00
Vitaliy Filippov df16ab627a Fix another bug in old store checksums O_o 
Checksum verification wasn't working correctly on reads of small initial offsetted writes
from the journal (until the write was flushed to "clean" metadata).
 2026-03-17 22:26:38 +00:00
Vitaliy Filippov 44c895dc30 Fix clients possibly not destroyed after being switched to RDMA 
It could in theory lead to hung operations on RDMA client disconnections.

Workflow (probable):
* OSD switches an outbound connection to PEER_RDMA and removes the FD from epoll
* But a receive request in io_uring is still active
* Receive response is never handled, refs remains > 0
* The connection is dropped, but osd_client_t is not destroyed because refs > 0
* Outbound ops are not canceled, suboperations hang, primary operations hang too
 2026-03-17 01:04:17 +03:00
Vitaliy Filippov fdea595913 Use vitastor-cli in the Cinder driver for modifications instead of direct etcd interaction 2026-03-15 01:44:30 +03:00
Vitaliy Filippov ef4c91ecc8 Fix mon_https_ca parsing 2026-03-15 01:01:41 +03:00
Vitaliy Filippov 5192c6cf50 Fix Docker install.sh, add instructions for Podman 2026-03-13 20:08:18 +03:00