thebestvibedev/tromcho.net
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,544 Commits 74 Branches 106 Tags
1c66c3e5ba4082b00b79b81ad8b19f9f3fd17841
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Vitaliy Filippov 1c66c3e5ba Update antietcd to 1.2.2 2026-01-23 01:45:00 +03:00
Vitaliy Filippov eddfa93c18 Refactor vitastor-cli & fio & qemu-img options in tests 2026-01-16 20:08:13 +03:00
Vitaliy Filippov ddd755a0e6 Fix incorrect checksums for small initial writes in the old store 
Details:
- Write size should be exactly csum_block_size (4k by default)
- Write should be made into a new object (unallocated space)
- In this case, the checksum of the block was calculated as if the block was
  padded with extra (2^32 - size) zero bytes due to a simple integer overflow
- As a cherry on the cake, such calculation was 'slightly' slow because it
  was processing almost 4 GB of zeroes for a small write
 2026-01-16 01:07:39 +03:00
Vitaliy Filippov 819f5b7ec9 Add a test for an idiotic checksum bug 2026-01-16 01:07:39 +03:00
Vitaliy Filippov 34d0a6d9b1 Allow -1 args 2026-01-16 01:07:34 +03:00
Vitaliy Filippov fe83825ead Release 3.0.1 
Important fixes:

- Disable RWF_ATOMIC by default because Linux incorrectly requires all atomic
  writes to be power-of-2-sized and length-aligned. Details: [use_atomic_flag](https://vitastor.io/en/docs/config/osd.html#use_atomic_flag)
- Fix cross-pool snapshots not working at all - always reading old data after taking the snapshot
- Fix level_placement (broken in 2.2.0)
- Fix CAS write return values in the client library (broken in 2.4.4), also breaking unaligned writes in VitastorFS
- Fix VitastorFS possibly losing some of intersecting parallel unaligned writes
- Prevent possible reads of the old data during unfinished intent writes in the new store
- Tests added for all of above problems to prevent future regressions

Other fixes:

- Allow to specify OSD tags and weights during prepare
- Only clear the first block instead of whole OSD metadata and journal areas during vitastor-disk
  prepare - OSD anyway clears them on the first run
- Fix vitastor-cli dd non-seekable detection and error status
- Do not stop OSD on zero-copy tcp short send errors - these aren't a bug, but just caused
  by TCP client disconnections
- Fix modify-osd not working after deleting the OSD configuration key from etcd
- Fix regular antietcd websocket disconnections in OSD due to lack of WS_PONG responses to WS_PING
- Reduce the number of allocated RDMA memory regions
- Allow routed RoCE
 2025-12-22 02:17:43 +03:00
Vitaliy Filippov 21cf5c8815 Fix optimized RMW at non-zero shared inode offset too 2025-12-21 12:05:06 +03:00
Vitaliy Filippov 44eeb1ed13 Do not read incomplete intent writes 2025-12-21 11:49:56 +03:00
Vitaliy Filippov cc6c445cf0 Add unaligned_write test for small file 2025-12-21 02:39:06 +03:00
Vitaliy Filippov bd6af0db09 Remove unused end_pad 2025-12-21 02:39:06 +03:00
Vitaliy Filippov 8860101e99 Fix optimized RMW in shared inodes at zero offset after disabling skip 2025-12-21 02:22:34 +03:00
Vitaliy Filippov c92661b364 Fix vitastor-cli dd non-seekable detection and error status 2025-12-21 01:33:57 +03:00
Vitaliy Filippov 9a02a592e3 Do not skip RMW at the end of the file - skipping it may lead to data corruption (fix #116) 2025-12-20 18:34:20 +03:00
Vitaliy Filippov 8b7fa3d3bc Add a test for parallel unaligned appends to a file 2025-12-20 18:34:08 +03:00
Vitaliy Filippov db5eaa2eee Fix CAS write return value (broken in 2.4.4) 2025-12-20 18:33:45 +03:00
Vitaliy Filippov d35727dbb7 Add test_level_placement and fix it with the 2.2.0+ PG "folding" system 2025-12-15 14:19:30 +03:00
Vitaliy Filippov b78f526696 Clear only the first OSD metadata and journal blocks - it anyway clears the rest 2025-12-14 14:44:27 +00:00
Vitaliy Filippov a23df12260 Fix modify-osd for the case when OSD config is deleted from etcd 2025-12-14 14:42:50 +00:00
Vitaliy Filippov 166e16102e Allow to specify tags and weights during prepare 2025-12-14 17:30:15 +03:00
Vitaliy Filippov 06c602110c Change "BUG" short send message to a client disconnection 2025-12-14 17:15:16 +03:00
Vitaliy Filippov 1de68c30af Fix cross-pool snapshots and fix the test for it 2025-12-13 18:20:16 +03:00
Vitaliy Filippov 2a0aca6e94 Fix Proxmox supported version note 2025-12-13 17:52:47 +03:00
Vitaliy Filippov e808332e12 Reply to WS_PING with WS_PONG to fix OSD websocket disconnections 2025-12-12 18:11:21 +00:00
Vitaliy Filippov fc5a183959 Allow routed RoCE 2025-12-12 15:06:49 +00:00
Vitaliy Filippov 55de37e58a Use a single memory region per client instead of separate mrs per each malloc 2025-12-12 15:05:33 +00:00
Vitaliy Filippov aacfdf0dec Rename try_recv_rdma to init_recv_rdma 2025-12-07 21:06:36 +03:00
Vitaliy Filippov a67c415e0f Release 3.0.0 
A single new feature: the new log-structured metadata store implementation, described
in the presentation from Moscow Highload'2025 (check it out [here](https://vitastor.io/en/docs/#talks-and-presentations)).

It's now the default store for new OSDs. The support for the old store is also left in place,
you can still choose it for new OSDs with `vitastor-disk prepare --meta_format 2`.

OSDs from previous versions with the old store format will also continue to operate just like before.
 2025-12-06 02:12:00 +03:00
Vitaliy Filippov 495f3fb4cd Remove libvitastor_blk from rpms 2025-12-06 01:48:56 +03:00
Vitaliy Filippov 934752d617 Add define RWF_DSYNC 2025-12-06 01:24:19 +03:00
Vitaliy Filippov 1e6e233426 More documentation 2025-12-05 02:20:31 +03:00
Vitaliy Filippov a5768a8ef6 Parse empty meta_format in superblock as v2 (fix updating from 2.x) 2025-12-02 01:52:42 +03:00
Vitaliy Filippov 216707f101 Add new parameter docs: meta_format, atomic_write_size, use_atomic_flag 2025-12-02 01:52:42 +03:00
Vitaliy Filippov 8ffdb93ed3 Implement test_resize_auto for the new (lsm/heap) store 2025-12-02 01:52:42 +03:00
Vitaliy Filippov 2fb7022c78 Fix OLD=1 ./test_resize 2025-12-02 01:52:42 +03:00
Vitaliy Filippov 2633978fec Duplicate tests for old store 2025-12-02 01:52:41 +03:00
Vitaliy Filippov 615d4825c0 Remove FIXME 2025-12-02 01:52:12 +03:00
Vitaliy Filippov e55ca26ff6 Reject all-zero blocks, mark empty space with a special entry type during init 2025-12-02 01:52:12 +03:00
Vitaliy Filippov abe093b9a3 Reserve for max_entry_size, not for min_entry_size 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 07e6eb0b16 Return no_inode_stats feature back 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 0eacce1e1d Crazy shit: optimize inode maps for very small item counts (i.e. inodes with only 1-15 objects) 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 3df410acc7 Fix test_scrub with BIG_INTENTs 2025-12-02 01:52:12 +03:00
Vitaliy Filippov e55927076c Abort compaction if overwritten 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 60fcb168fe Fix assert 2025-12-02 01:52:12 +03:00
Vitaliy Filippov b17691ba02 Add a redirect_intent write mode for non-capacitor SSDs 2025-12-02 01:52:12 +03:00
Vitaliy Filippov f2cbe793e2 Fix intent_write_allowed check, add offset & len validation on start 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 0053546f8b Remove zerofill in blockstore (an old and unused artifact) 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 959f792f82 Auto-guess atomic_write_size from /sys 2025-12-02 01:52:12 +03:00
Vitaliy Filippov eaff4509ca Complete small/intent writes explicitly, not implicitly on metadata block write 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 04eefce30b Fix recheck logging 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 746844d301 Add double-claim block check 2025-12-02 01:52:12 +03:00
Vitaliy Filippov c8f5b6cb19 Fix submitting forced fsyncs in flusher 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 0f4837e9bb Fix test_resize 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 25ce82a729 Fix checksum flushing for the case of empty small writes, fix checksum calculation skipping during read 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 944499135f Reduce hashmap memory usage by 8 bytes per key :) 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 08f21edaf7 Fix enospc detection logic and test_enospc 2025-12-02 01:52:12 +03:00
Vitaliy Filippov e9f0639e62 Fix iterate_compaction for the case of unfinished commit/rollback 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 4e0b203552 Fix iterate_compaction for commit with non-latest version 2025-12-02 01:52:12 +03:00
Vitaliy Filippov b772a3cd04 Fix freeing repeated big_intents 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 856ad79a02 Fix mark_completed_lsns for enabled fsyncs 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 39a8772d7f Fix loading 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 993f40de37 Utility to test atomic writes 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 5607222921 Revert f9554a02e4b75e87c5b44085b68f84e69100c9d8 
Do not "Limit the number of unstable versions per object" - LSMeta doesn't need it
 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 378fff6f67 Write recheck_modified_blocks to disk 2025-12-02 01:52:12 +03:00
Vitaliy Filippov daf2cc3fb1 Validate objects on start 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 17d61c5868 Fix disk_tool for lsmeta 2025-12-02 01:52:12 +03:00
Vitaliy Filippov e709657de4 Save completed_lsn in the superblock 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 3eecf9048c Simplify compaction 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 1852caaeec Implement even more simplified big_intent writes 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 7b40561141 Fix GC and loading 2025-12-02 01:52:12 +03:00
Vitaliy Filippov d80c12ced2 Prevent parallel modifications of the same object in fio_bs 2025-12-02 01:52:12 +03:00
Vitaliy Filippov b9713deecd Pass bitmap in fio_bs 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 852734270e wip 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 8f92979a18 Move prev/next/block_num to struct heap_list_item_t 2025-12-02 01:52:12 +03:00
Vitaliy Filippov b720af74c2 WIP experimental - double linked list 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 96df2966cc Reserve block entries 2025-12-02 01:52:12 +03:00
Vitaliy Filippov d3b171e047 FIXME 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 7287b7fc25 Remove std::set meta_nearfull 2025-12-02 01:52:12 +03:00
Vitaliy Filippov e1bb670491 Apply inflights in batches 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 1e5a01def8 Another experiment: store writes in mallocs 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 371e630f52 Recheck repeat using map 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 0a7ae616f3 Do not use pending_modified_blocks to check 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 82f5fb7edd Remap without intermediate hashmap 2025-12-02 01:52:12 +03:00
Vitaliy Filippov ec8527c89d Experimental: move prev_count to the map itself 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 5afef7ca6d WIP Integrate LSMeta :-) 2025-12-02 01:52:12 +03:00
Vitaliy Filippov e512e1eeb1 LSMeta 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 7ab60c00ab Use 32-bit big write location (OK for up to 512 TB OSDs) 2025-12-02 01:52:12 +03:00
Vitaliy Filippov ac8e0ef231 Do not store offset & len in big_writes 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 94f3634602 Fix object crc32c calculation 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 2b8a9e3f90 Add entry_type to heap_object_t too 
This is required to:
1) later inline the last "big_write" entry into the object to slightly reduce memory usage
2) eliminate an ugly hack where entry type is determined by its size
3) make the storage scheme extensible i.e. when adding new entry types
 2025-12-02 01:52:12 +03:00
Vitaliy Filippov ef792608b0 Rename flags to entry_type 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 04531bcfbb Use robin_hood::unordered_flat_map - it has 1 byte overhead instead of 8 byte 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 1b40fa1cee Revert try_get 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 3ea9230ed0 WIP dump/load heap 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 97dfbfad75 Add missing list_stable_limit support 2025-12-02 01:52:12 +03:00
Vitaliy Filippov c148f97ee4 Fix checksum validation in !inmemory_journal mode 2025-12-02 01:52:12 +03:00
Vitaliy Filippov b1f61eb5c8 Fix checksum padding during read 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 9db8748647 Remove unused bs->buffer_area + buffer calc 2025-12-02 01:52:12 +03:00
Vitaliy Filippov e747319c1e Add missing memset zero_object to 0 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 0cb0e31ceb Mark in-memory data reads as SKIP_CSUM 2025-12-02 01:52:12 +03:00
Vitaliy Filippov 0703efd8b9 Add missing calc_crc32c after updating block checksums 2025-12-02 01:52:12 +03:00