thebestvibedev/tromcho.net
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
584 Commits 74 Branches 106 Tags
ab21a1908b00a7d01a07ebd6aa945667592aa6ad
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Vitaliy Filippov ab21a1908b Check for the dirty PG flag when trying to continue to stop it after sync 2021-03-08 17:04:10 +03:00
Vitaliy Filippov 30d1ccd43e Fix an infinite loop when discarding list operations during stop_pg() 2021-03-08 17:04:10 +03:00
Vitaliy Filippov 8bdd6d8d78 Reset PG state when stopping them 2021-03-08 17:04:10 +03:00
Vitaliy Filippov 09b3e4e789 Fix OSDs being unable to stop PGs that are 'peering', not 'active' 
This was sometimes leading to incorrect misplaced and degraded object count statistics
 2021-03-08 17:04:10 +03:00
Vitaliy Filippov 07912fd670 Use history/last_clean_pgs to avoid extra data move when observing a series of changes in the cluster 2021-03-08 17:04:10 +03:00
Vitaliy Filippov bc742ccf8c Fix a small memory leak in etcd_state_client 2021-03-08 17:04:10 +03:00
Vitaliy Filippov 314b20437b Do not break subsequent small writes badly when a big write is canceled 2021-03-08 17:04:10 +03:00
Vitaliy Filippov 29bac892ad Add .gitignore 2021-03-08 17:04:10 +03:00
Vitaliy Filippov cf7547faf3 Fix *.sh build scripts 2021-03-02 02:17:11 +03:00
Vitaliy Filippov ab90ed747f Release 0.5.6 
- Fix operation statistics
- Fix a rebalance hang introduced in 0.5.5
- Test PG count changes with actual data moving
- Fix a possible 'unexpected pg state: 0' error during PG count change
 2021-03-01 16:26:04 +03:00
Vitaliy Filippov 29d8ac8b1b Do not report statistics for the empty operation 2021-03-01 16:20:57 +03:00
Vitaliy Filippov 97795ea1b1 Use pg_minsize=2 in the pg_count change test 
Also don't check for has_degraded because it's not a bug that objects
are _temporarily_ listed as degraded during PG peering as it's not
required for the new primary to connect to _all_ older peers to start
peering. The test may be improved in the future by temporarily disabling
degraded recovery during it and returning the has_degraded check back.
 2021-03-01 16:18:08 +03:00
Vitaliy Filippov 24e7075f08 Fix monitor's statistics aggregation 2021-02-28 19:51:16 +03:00
Vitaliy Filippov 6155b23a7e Replace pgs[id] with pgs.at(id) to prevent accidental auto-vivification 2021-02-28 19:36:59 +03:00
Vitaliy Filippov 7d49706c07 Improve the pg_count change test: add more OSDs and actually move data between them 2021-02-28 19:36:59 +03:00
Vitaliy Filippov 46e79f3306 Wait for PGs to become clean before stopping them 2021-02-28 19:36:59 +03:00
Vitaliy Filippov 41fd14e024 Fix deletes not increasing write_iodepth 2021-02-28 19:36:59 +03:00
Vitaliy Filippov bb2d9a3afe Release 0.5.5 
- Transition to CMake build system
- Fix Monitor being unable to change PG sizes
- Fix PG optimizer not using some OSDs in some cases
- Fix inability to change PG count online
- Improve journal flusher performance
- Add a little better systemd unit generator
- Use w=8 with jerasure (breaking change for EC pools)
 2021-02-26 01:59:18 +03:00
Vitaliy Filippov e899ed2c25 Make OSDs with 256 flushers (as they are now dynamic) 2021-02-26 01:59:18 +03:00
Vitaliy Filippov e21b14b72c Fix rpm specs for building with CMake 2021-02-26 01:59:18 +03:00
Vitaliy Filippov 5af8eddaa9 Add the remaining build script for Debian 2021-02-26 01:59:18 +03:00
Vitaliy Filippov 4f5a94c07a Modify instructions for the CMake build 2021-02-26 00:28:57 +03:00
Vitaliy Filippov e16b87ecc8 Rename random_combinations() parameter from "unordered" to "ordered" as it's more correct 2021-02-25 23:59:34 +03:00
Vitaliy Filippov fcb4aa0a11 Fix Monitor being unable to change PG sizes 2021-02-25 23:59:34 +03:00
Vitaliy Filippov 12adfa470c Add a test for changing PG size 2021-02-25 23:59:33 +03:00
Vitaliy Filippov 7f15e0c084 Add a simple test for the PG optimizer 2021-02-25 23:59:33 +03:00
Vitaliy Filippov 08d4bef419 Fix PG optimizer removing PGs without adding new ones 
This happened when the distribution was already valid for the current OSD tree,
but didn't use all OSDs. For example, OSDs 1 2 3 and all PGs equal to [ 1, 2 ]
remained unchanged.
 2021-02-25 23:59:33 +03:00
Vitaliy Filippov 2d73b19a6c Fix online PG count change bugs 2021-02-25 23:59:33 +03:00
Vitaliy Filippov 69c87009e9 Add a test for changing PG count 2021-02-25 23:59:33 +03:00
Vitaliy Filippov c974cb539c Make flusher_count adaptive and limit write iodepth 2021-02-25 23:59:33 +03:00
Vitaliy Filippov 00e98f64f3 A little better systemd unit generator 2021-02-25 23:59:33 +03:00
Vitaliy Filippov 91a70dfb1b Add a test for the no_same_sector_overwrites mode 2021-02-25 23:59:33 +03:00
Vitaliy Filippov 178388ac8c Use packages/ subdir instead of build/ for Docker package builds 2021-02-25 23:59:04 +03:00
Vitaliy Filippov bf9a175efc Move C/C++ sources to src subdirectory 2021-02-25 23:59:03 +03:00
Vitaliy Filippov 08aed962de Use CMake 2021-02-25 23:58:08 +03:00
Vitaliy Filippov 8c65e890b9 Slightly clean up the build script 2021-02-25 23:56:54 +03:00
Vitaliy Filippov 8cda70b889 Allow to enable AddressSanitizer with "ASAN=1 make" 2021-02-25 23:55:33 +03:00
Vitaliy Filippov 61ab22403a Use w=8 with jerasure 2021-02-25 23:55:33 +03:00
Vitaliy Filippov 16da663a66 Add another test for failure domains 2021-02-25 23:55:33 +03:00
Vitaliy Filippov 4a2dcf7b6b Update the license to VNPL 1.1 
VNPL 1.1 is slightly reworded to make it clear that proprietary software
interacting with Vitastor and providing some kind of service to end users isn't
a "Proxy Program" if it's not specially designed to be used with Vitastor.

For example, Windows OS running in a virtual machine stored in a Vitastor
cluster clearly isn't.
 2021-02-25 23:55:33 +03:00
Vitaliy Filippov 8d48cc56b0 Generate randomly permutated OSD combinations when optimizing for compressed chunks 2021-02-25 23:55:33 +03:00
Vitaliy Filippov 9f58f01425 Mirror afr.js from /vitalif/ceph-afr-calc 2021-02-25 23:55:33 +03:00
Vitaliy Filippov b9e7d31aa1 Release v0.5.4 
- Fix a rare hang, more or less reproducible with very slow drives
- Fix a hang with the no_same_sector_overwrites mode
 2021-02-24 01:40:30 +03:00
Vitaliy Filippov 2d9f09dcb6 Attempt forced trim when stopping an overrun flusher 
Fixes a rare hang happening in the event of journal space running out without
new work to do for flushers except the current sector.
The hang could be reproduced more or less consistently with very slow drives.
 2021-02-24 01:33:01 +03:00
Vitaliy Filippov 7cc59260c5 Fix no_same_sector_overwrites related bug 2021-02-23 18:50:51 +03:00
Vitaliy Filippov ca0a11ec85 Release 0.5.3 2021-02-03 00:38:57 +03:00
Vitaliy Filippov 51c0b5afee Whitelist more leaks 2021-02-02 02:05:41 +03:00
Vitaliy Filippov e1e01d042e Rename sector_info.usage_count to flush_count 2021-02-02 01:32:23 +03:00
Vitaliy Filippov 534a4a657e Rename space_check.sectors_required to sectors_to_write 2021-02-02 01:30:23 +03:00
Vitaliy Filippov 9b5d8b9ad4 Fix multiple-sector journal writes, add assertions to not miss any SQEs 2021-02-02 01:29:11 +03:00
Vitaliy Filippov e66ed47515 Clear SQEs before returning them to the caller to prevent erroneous double submissions 2021-02-02 01:26:54 +03:00
Vitaliy Filippov 036c6d4c42 Add a simple test case 2021-02-01 19:43:10 +03:00
Vitaliy Filippov 4cb79a3bf8 Allow to calculate simple-offsets for files 2021-02-01 19:43:10 +03:00
Vitaliy Filippov 3bf53754c2 Fix several I/O bugs 2021-02-01 19:43:10 +03:00
Vitaliy Filippov 6023cac361 Do not stop clients before they are connected 2021-02-01 19:31:10 +03:00
Vitaliy Filippov 915d04c446 Allow empty global configuration, report OSD statistics faster 2021-02-01 19:31:10 +03:00
Vitaliy Filippov 21e06ea40d Fix memory leaks in fio engines 2021-02-01 19:31:10 +03:00
Vitaliy Filippov 9ef7f865b0 Fix incorrect calls to prepare_journal_sector_write() when flushing multiple sectors 2021-02-01 19:31:10 +03:00
Vitaliy Filippov 9dd20a31aa Do not use pg_minsize in the client code! 2021-02-01 19:31:10 +03:00
Vitaliy Filippov 28be049909 Dump only actual part of the journal by default 2021-01-01 23:04:30 +03:00
Vitaliy Filippov 78fbaacf1f External jerasure's w into defines 
In fact, w=8 looks better than w=32, so it may be changed in the future
 2020-12-31 19:15:22 +03:00
Vitaliy Filippov 1526c5a213 Add lp_solve into dependencies 2020-12-31 01:32:31 +03:00
Vitaliy Filippov c7cc414c90 Skip removed descriptors in epoll (this is possible in real clusters) 2020-12-30 17:04:18 +03:00
Vitaliy Filippov f4ea313707 Fix cl->read_op being freed without calling the completion callback 2020-12-30 16:55:54 +03:00
Vitaliy Filippov b88b76f316 Parallel usage of multiple network interfaces was a sick fantasy 2020-12-30 00:05:17 +03:00
Vitaliy Filippov 4a17a61d1f Make rm_inode work with incomplete and degraded objects, allow to wait before deleting objects 2020-12-28 16:38:08 +03:00
Vitaliy Filippov ccabbbfbcb For reference: include a spec patch for building QEMU 4.2 or CentOS 7 2020-12-06 15:43:38 +03:00
Vitaliy Filippov 26dac57083 State that jerasure is now supported 2020-12-06 15:25:48 +03:00
Vitaliy Filippov 44a53d8352 Huh. Fix rpath for packages 2020-12-05 20:16:39 +03:00
Vitaliy Filippov 9d80bd2d98 Build with jerasure, split some build scripts 2020-12-05 19:02:23 +03:00
Vitaliy Filippov 322a38a144 Fix non-preserved real_pg_count leading to inability to change pools online 2020-12-04 23:46:48 +03:00
Vitaliy Filippov 1018764c91 Fix write->delete->write bugs, add & fix some debugging output 2020-12-04 23:21:58 +03:00
Vitaliy Filippov a45e0e5e67 Use custom decoding instead of just jerasure_matrix_decode() 
- Cache the decoding matrix
- Don't do unnecessary erasures->erased conversion during decoding
- Avoid extra memory allocations during decoding
- Don't always reconstruct coding chunks
- Reconstruct chunks one-by-one, without overlapping ranges
 2020-12-04 17:43:48 +03:00
Vitaliy Filippov 44656fbf67 Allow writes with low version numbers after a delete 2020-12-04 11:54:41 +03:00
Vitaliy Filippov 089f138e0c Allow situations where the journal contains a big_write(v1) after delete(v2) and v1 < v2 
Fixes a crash in the following scenario:
- client issues a delete request (object version is at least 2)
- OSD has time to flush it to the metadata, but doesn't have time to move the journal start pointer on disk
- client overwrites the same object and it gets the version number 1 again
- OSD is restarted and sees delete(v=2), big_write(v=1) in the journal
- dirty_db sequence gets broken and OSD crashes with assert("Writes and deletes shouldn't happen at the same time")
 2020-12-04 11:47:27 +03:00
Vitaliy Filippov bcc8e697f9 Delete PGs when deleting pools 
(All OSD crash with "Online PG count change not allowed" if you try to delete an active pool though)
 2020-12-04 11:47:27 +03:00
Vitaliy Filippov a4c46ba745 Add jerasure EC support (reed_sol_van, others are slower) (not tested yet) 2020-12-04 11:47:27 +03:00
Vitaliy Filippov 5596ad8997 Use custom QEMU build for CentOS 7 2020-12-04 11:47:05 +03:00
Vitaliy Filippov 59c29b0cee Fix RPATH for CentOS builds, add additional repos into the CentOS installation instructions 2020-12-04 11:47:04 +03:00
Vitaliy Filippov 959089b919 Enable progress_notify=true for etcd watches 2020-11-17 16:29:42 +03:00
Vitaliy Filippov d3e7749616 Final fixes for packaging 2020-11-10 23:33:07 +03:00
Vitaliy Filippov b56f8820ec Container packaging for Debian 11 Bullseye, CentOS 7 and CentOS 8 2020-11-10 00:02:53 +03:00
Vitaliy Filippov 4bd2bd48eb Build Vitastor packages, too 2020-11-09 14:41:39 +03:00
Vitaliy Filippov a3fc9f8d7d Add a Dockerfile to build patched QEMU for Debian (Buster) 2020-11-09 02:30:41 +03:00
Vitaliy Filippov 530975aed7 Make it also build with GCC 8 and on Debian Buster 2020-11-09 00:07:07 +03:00
Vitaliy Filippov 1446aad107 Simple patch for qemu-kvm .spec 2020-11-08 02:14:53 +03:00
Vitaliy Filippov 46479e2456 Add RPM build scripts for CentOS 8 2020-11-08 01:55:17 +03:00
Vitaliy Filippov e41bee72a5 Lower node.js requirement to 10.x 2020-11-08 01:54:12 +03:00
Vitaliy Filippov 2e0f223ddb Add RPM build scripts for CentOS 7 2020-11-07 01:52:10 +03:00
Vitaliy Filippov 3be7bc29d8 Make it build with QEMU 2.0, too 
Also begin to work on rpms
 2020-11-06 20:05:00 +03:00
Vitaliy Filippov 0c43ff9daf Add scripts to copy fio and qemu includes to the source package 2020-11-06 18:40:42 +03:00
Vitaliy Filippov 64d471cf53 Add simple Debian packaging 2020-11-06 18:40:42 +03:00
Vitaliy Filippov 809b2ad8cd Add install target 2020-11-06 01:12:22 +03:00
Vitaliy Filippov 550d4af151 Rename test.cpp to test_shit.cpp (random shit) 2020-11-06 01:12:22 +03:00
Vitaliy Filippov cf0f23ab8e Add patches for QEMU QAPI IDL 2020-11-04 23:30:51 +03:00
Vitaliy Filippov a516fefa8c Add qemu_module_dummy and qemu_stamp_xxx to qemu_driver.c 2020-11-04 23:10:29 +03:00
Vitaliy Filippov 3b7279b376 Add Ceph EC 2+1 test results 2020-11-01 14:13:35 +03:00
Vitaliy Filippov 824ea507d0 Do not try to push more segments than IOV_MAX at once as it leads to EMSGSIZE 2020-10-30 01:25:43 +03:00
Vitaliy Filippov 23ea409081 Fix "can't get SQE, will fall out of sync with EPOLLET" when overflowing the ring 
OSDs shouldn't crash or hang with long iodepths anymore
 2020-10-30 01:06:36 +03:00
Vitaliy Filippov 2ccb75974b Fix a rare crash caused by a stopped client still being in write_ready_clients 2020-10-30 01:04:58 +03:00