Vitaliy Filippov
68985bee0a
WIP RDMA credits
2025-12-12 18:11:36 +00:00
Vitaliy Filippov
e808332e12
Reply to WS_PING with WS_PONG to fix OSD websocket disconnections
2025-12-12 18:11:21 +00:00
Vitaliy Filippov
fc5a183959
Allow routed RoCE
2025-12-12 15:06:49 +00:00
Vitaliy Filippov
55de37e58a
Use a single memory region per client instead of separate mrs per each malloc
2025-12-12 15:05:33 +00:00
Vitaliy Filippov
aacfdf0dec
Rename try_recv_rdma to init_recv_rdma
2025-12-07 21:06:36 +03:00
Vitaliy Filippov
a67c415e0f
Release 3.0.0
...
A single new feature: the new log-structured metadata store implementation, described
in the presentation from Moscow Highload'2025 (check it out [here](https://vitastor.io/en/docs/#talks-and-presentations )).
It's now the default store for new OSDs. The support for the old store is also left in place,
you can still choose it for new OSDs with `vitastor-disk prepare --meta_format 2`.
OSDs from previous versions with the old store format will also continue to operate just like before.
2025-12-06 02:12:00 +03:00
Vitaliy Filippov
495f3fb4cd
Remove libvitastor_blk from rpms
2025-12-06 01:48:56 +03:00
Vitaliy Filippov
934752d617
Add define RWF_DSYNC
2025-12-06 01:24:19 +03:00
Vitaliy Filippov
1e6e233426
More documentation
2025-12-05 02:20:31 +03:00
Vitaliy Filippov
a5768a8ef6
Parse empty meta_format in superblock as v2 (fix updating from 2.x)
2025-12-02 01:52:42 +03:00
Vitaliy Filippov
216707f101
Add new parameter docs: meta_format, atomic_write_size, use_atomic_flag
2025-12-02 01:52:42 +03:00
Vitaliy Filippov
8ffdb93ed3
Implement test_resize_auto for the new (lsm/heap) store
2025-12-02 01:52:42 +03:00
Vitaliy Filippov
2fb7022c78
Fix OLD=1 ./test_resize
2025-12-02 01:52:42 +03:00
Vitaliy Filippov
2633978fec
Duplicate tests for old store
2025-12-02 01:52:41 +03:00
Vitaliy Filippov
615d4825c0
Remove FIXME
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
e55ca26ff6
Reject all-zero blocks, mark empty space with a special entry type during init
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
abe093b9a3
Reserve for max_entry_size, not for min_entry_size
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
07e6eb0b16
Return no_inode_stats feature back
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
0eacce1e1d
Crazy shit: optimize inode maps for very small item counts (i.e. inodes with only 1-15 objects)
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
3df410acc7
Fix test_scrub with BIG_INTENTs
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
e55927076c
Abort compaction if overwritten
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
60fcb168fe
Fix assert
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
b17691ba02
Add a redirect_intent write mode for non-capacitor SSDs
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
f2cbe793e2
Fix intent_write_allowed check, add offset & len validation on start
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
0053546f8b
Remove zerofill in blockstore (an old and unused artifact)
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
959f792f82
Auto-guess atomic_write_size from /sys
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
eaff4509ca
Complete small/intent writes explicitly, not implicitly on metadata block write
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
04eefce30b
Fix recheck logging
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
746844d301
Add double-claim block check
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
c8f5b6cb19
Fix submitting forced fsyncs in flusher
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
0f4837e9bb
Fix test_resize
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
25ce82a729
Fix checksum flushing for the case of empty small writes, fix checksum calculation skipping during read
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
944499135f
Reduce hashmap memory usage by 8 bytes per key :)
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
08f21edaf7
Fix enospc detection logic and test_enospc
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
e9f0639e62
Fix iterate_compaction for the case of unfinished commit/rollback
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
4e0b203552
Fix iterate_compaction for commit with non-latest version
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
b772a3cd04
Fix freeing repeated big_intents
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
856ad79a02
Fix mark_completed_lsns for enabled fsyncs
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
39a8772d7f
Fix loading
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
993f40de37
Utility to test atomic writes
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
5607222921
Revert f9554a02e4b75e87c5b44085b68f84e69100c9d8
...
Do not "Limit the number of unstable versions per object" - LSMeta doesn't need it
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
378fff6f67
Write recheck_modified_blocks to disk
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
daf2cc3fb1
Validate objects on start
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
17d61c5868
Fix disk_tool for lsmeta
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
e709657de4
Save completed_lsn in the superblock
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
3eecf9048c
Simplify compaction
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
1852caaeec
Implement even more simplified big_intent writes
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
7b40561141
Fix GC and loading
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
d80c12ced2
Prevent parallel modifications of the same object in fio_bs
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
b9713deecd
Pass bitmap in fio_bs
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
852734270e
wip
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
8f92979a18
Move prev/next/block_num to struct heap_list_item_t
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
b720af74c2
WIP experimental - double linked list
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
96df2966cc
Reserve block entries
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
d3b171e047
FIXME
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
7287b7fc25
Remove std::set meta_nearfull
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
e1bb670491
Apply inflights in batches
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
1e5a01def8
Another experiment: store writes in mallocs
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
371e630f52
Recheck repeat using map
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
0a7ae616f3
Do not use pending_modified_blocks to check
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
82f5fb7edd
Remap without intermediate hashmap
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
ec8527c89d
Experimental: move prev_count to the map itself
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
5afef7ca6d
WIP Integrate LSMeta :-)
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
e512e1eeb1
LSMeta
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
7ab60c00ab
Use 32-bit big write location (OK for up to 512 TB OSDs)
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
ac8e0ef231
Do not store offset & len in big_writes
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
94f3634602
Fix object crc32c calculation
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
2b8a9e3f90
Add entry_type to heap_object_t too
...
This is required to:
1) later inline the last "big_write" entry into the object to slightly reduce memory usage
2) eliminate an ugly hack where entry type is determined by its size
3) make the storage scheme extensible i.e. when adding new entry types
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
ef792608b0
Rename flags to entry_type
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
04531bcfbb
Use robin_hood::unordered_flat_map - it has 1 byte overhead instead of 8 byte
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
1b40fa1cee
Revert try_get
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
3ea9230ed0
WIP dump/load heap
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
97dfbfad75
Add missing list_stable_limit support
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
c148f97ee4
Fix checksum validation in !inmemory_journal mode
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
b1f61eb5c8
Fix checksum padding during read
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
9db8748647
Remove unused bs->buffer_area + buffer calc
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
e747319c1e
Add missing memset zero_object to 0
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
0cb0e31ceb
Mark in-memory data reads as SKIP_CSUM
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
0703efd8b9
Add missing calc_crc32c after updating block checksums
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
4a0720a231
Fix checksum padding during flush
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
57d83ecf7c
Fix free_read_buffers
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
d4f7bbb412
Fix assert(region_marker) at the end of the block in find_block_run()
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
e5e71fc21a
Fix assert(is_buffer_area_free) with size=0
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
0ec4f1608a
Add a copy of wyhash
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
232e416658
Fix test build with isa-l
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
23d5fec580
Remove assert !region_marker & FREE
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
16881a3d6b
Use wyhash
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
7cde5c75b9
Change emhashes
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
4c32244409
unordered_map mvcc
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
30c5a79772
Fix bad resharding due to the lack of iteration order in a hashmap
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
6e856c2719
Use emhash::try_get
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
af710d3c65
Use emhash hashmap (2x faster)
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
8189c3a4ef
Use unordered_maps for object-block index
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
552ba5d885
B-tree is slower...
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
14bbf18ede
Unordered_map for pool settings
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
c3eaaa4b94
Fix read fio bench
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
8b35c09e12
Fix vitastor-disk prepare and param validation
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
74b8ea1303
Fix skipping of corrupted objects, fix use_buffer_area with zero size
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
6a30e6653a
Support heap format in simple-offsets
2025-12-02 01:52:12 +03:00
Vitaliy Filippov
2d7da127ad
Return all unstable versions in listing
2025-12-02 01:52:12 +03:00
Vitaliy Filippov