This website requires JavaScript.
2a8e40835e
Fix reporting to Consul, report even if we are purely secondary
Vitaliy Filippov
2020-04-17 01:59:06 +03:00
309486d746
Implement loading PGs from Consul (in theory)
Vitaliy Filippov
2020-04-16 23:22:32 +03:00
582f485578
Extract http & getifaddr_list into a separate file
Vitaliy Filippov
2020-04-15 15:47:06 +03:00
089b4eb208
Retry consul connection attempts and then die
Vitaliy Filippov
2020-04-15 15:33:18 +03:00
d78ce509c6
Add simple timer manager
Vitaliy Filippov
2020-04-15 13:29:13 +03:00
f3a7ccff50
Use 4K blockstore block by default, use MEM_ALIGNMENT in osd code
Vitaliy Filippov
2020-04-14 19:19:56 +03:00
37b27c3025
Implement basic OSD status reporting to Consul
Vitaliy Filippov
2020-04-14 14:37:50 +03:00
edf6d6f897
Fix http_request
Vitaliy Filippov
2020-04-12 02:08:00 +03:00
d11e8dcb5e
Do not flush or recover in readonly mode
Vitaliy Filippov
2020-04-11 12:05:28 +03:00
dd02bc1c44
Add base64 implementation
Vitaliy Filippov
2020-04-11 12:04:10 +03:00
298b013eae
Add simple http request function
Vitaliy Filippov
2020-04-07 01:53:13 +03:00
0880a77c1a
2 FIXME for the future
Vitaliy Filippov
2020-04-06 00:55:47 +03:00
aa849ea07b
Add a test for missing chunk overwrite
Vitaliy Filippov
2020-04-05 16:14:03 +03:00
d59be0e8b4
Delete misplaced chunks after moving the object, reset object state in primary_write
Vitaliy Filippov
2020-04-05 15:50:42 +03:00
cf7de0f181
(Almost) Implement misplaced recovery, integrating it into calc_rmw()
Vitaliy Filippov
2020-04-04 22:19:45 +03:00
6212195440
Implement parallel recovery
Vitaliy Filippov
2020-04-04 02:18:29 +03:00
dfb6e15eaa
Implement graceful stopping of PGs
Vitaliy Filippov
2020-04-03 13:03:42 +03:00
afe2e76c87
Implement regular automatic syncs, split osd_t constructor into some methods
Vitaliy Filippov
2020-04-02 22:16:46 +03:00
0f43f6d3f6
Fix crashes, print some stats
Vitaliy Filippov
2020-03-31 17:50:50 +03:00
92c800bb64
Forget unstable writes when re-peering, rename parity_block_size -> pg_stripe_size, pg_parity_size -> pg_block_size
Vitaliy Filippov
2020-03-31 02:09:25 +03:00
8a8b619875
Handle secondary OSD connection errors [in theory]
Vitaliy Filippov
2020-03-30 19:43:12 +03:00
43fe1d88e7
Fix memory leaks with subops, fix recovery crashes
Vitaliy Filippov
2020-03-28 19:09:20 +03:00
1b30120918
Fix stripe reconstruction in recovery, only write modified object parts
Vitaliy Filippov
2020-03-28 13:51:30 +03:00
c0a22d825d
Fix degraded object recovery (it seems to work now)
Vitaliy Filippov
2020-03-25 02:17:12 +03:00
7acfc95f75
CONFIG_HAVE_GETTID
Vitaliy Filippov
2020-03-23 17:09:52 +03:00
250f22c0b6
Implement basic degraded object recovery (integrated into primary_write)
Vitaliy Filippov
2020-03-24 01:13:04 +03:00
dbd8418798
Reply using a single finish_op() method, allow to call OSD ops from inside the OSD
Vitaliy Filippov
2020-03-24 00:18:35 +03:00
036f4c5bf3
Fix unstable flushing, include extra OSDs with old object versions in osd_set
Vitaliy Filippov
2020-03-23 15:44:29 +03:00
fd8e1a8418
Slightly reorganize object state check code
Vitaliy Filippov
2020-03-23 00:39:53 +03:00
a08e0bfacd
Treat misplaced and degraded as separate state parts
Vitaliy Filippov
2020-03-22 20:36:29 +03:00
ddc3e927d3
Solve it in integers
Vitaliy Filippov
2020-03-20 13:58:54 +03:00
2aa605f2bb
Do not check
Vitaliy Filippov
2020-03-20 13:38:35 +03:00
18915b264a
Extract to .pm + fix all_combinations
Vitaliy Filippov
2020-03-19 21:33:42 +03:00
60f795e7eb
Add lp_solve based data distribution optimizer
Vitaliy Filippov
2020-03-19 17:23:24 +03:00
3a4279adbf
Hash-based PG distribution experiments
Vitaliy Filippov
2020-03-17 18:52:35 +03:00
1ec9794376
Extract flushing into a separate file
Vitaliy Filippov
2020-03-15 18:39:31 +03:00
d8164e9d84
Print PG states on every change
Vitaliy Filippov
2020-03-14 22:19:45 +03:00
21d0b06959
Implement flushing (stabilize/rollback) of unstable entries on start of the PG
Vitaliy Filippov
2020-03-13 21:41:54 +03:00
46f9bd2a69
Make blockstore list operation return consistent snapshots
Vitaliy Filippov
2020-03-13 21:41:45 +03:00
6982fe1255
Do not block reads by previous unfinished writes
Vitaliy Filippov
2020-03-13 21:28:42 +03:00
eba053febe
Do not start small writes before finishing the last big write to the same object
Vitaliy Filippov
2020-03-12 02:15:01 +03:00
899946ff96
Add osd_test function to unblock an OSD blocked by the lack of journal space
Vitaliy Filippov
2020-03-10 17:19:24 +03:00
3dd1b22d55
Fix segfault with concurrent OP_SYNCs
Vitaliy Filippov
2020-03-10 17:00:16 +03:00
31f9445030
Use immediate_commit to benefit the primary OSD
Vitaliy Filippov
2020-03-10 02:05:32 +03:00
3f522c66e6
Implement immediate commit mode
Vitaliy Filippov
2020-03-10 01:59:15 +03:00
c3737ae3ff
Add journal fsync to stabilize/rollback
Vitaliy Filippov
2020-03-09 00:35:54 +03:00
c863543bfe
Fix possible journal corruption caused by concurrent flushing and writing of the same journal sector
Vitaliy Filippov
2020-03-07 17:36:58 +03:00
1696446545
Rename min/max _used to _flushed
Vitaliy Filippov
2020-03-07 16:41:40 +03:00
41dddddbf2
Fix some logging
Vitaliy Filippov
2020-03-07 03:39:28 +03:00
2d4e24c9ce
Add journal dumper debugging tool
Vitaliy Filippov
2020-03-06 02:02:00 +03:00
844cacd357
Allow incorrectly forbidden BS_OP_LIST in readonly mode
Vitaliy Filippov
2020-03-06 02:02:39 +03:00
e19d9fde5f
Fix peering_pg, begin tests
Vitaliy Filippov
2020-03-05 20:58:52 +03:00
9cb07d844b
Make [un]register_consumer operate on pointers, rename get_loop_again() to has_work()
Vitaliy Filippov
2020-03-04 21:00:15 +03:00
1e21555343
Add FIXME with Oops
Vitaliy Filippov
2020-03-04 20:34:45 +03:00
94cdbcd085
Stop reading when less than <buffer> data is available
Vitaliy Filippov
2020-03-04 18:03:16 +03:00
8315407558
Incoming data pre-buffering
Vitaliy Filippov
2020-03-02 02:58:00 +03:00
b27ad550cf
Use btree_map instead of sparsepp
Vitaliy Filippov
2020-03-04 17:12:27 +03:00
8e63995306
Allow to specify data area size
Vitaliy Filippov
2020-03-04 02:32:49 +03:00
9c33de5e57
Test: use submit_and_wait()
Vitaliy Filippov
2020-03-03 17:48:47 +03:00
eabfe4faac
Test SQ poll threads. Unstable and in fact slower :(
Vitaliy Filippov
2020-03-03 17:22:40 +03:00
c9f3654905
Add ringloop.h to dependencies
Vitaliy Filippov
2020-03-03 17:23:33 +03:00
2a5ca4ff6f
Measure OP_STAB in both ways
Vitaliy Filippov
2020-03-03 02:18:59 +03:00
2575431176
Fix EPOLLET race on Linux 5.5+
Vitaliy Filippov
2020-03-03 01:47:56 +03:00
20125db181
Use clock_gettime()
Vitaliy Filippov
2020-03-03 00:54:42 +03:00
7eac7b6d55
Use writev in libfio_sec_osd
Vitaliy Filippov
2020-03-02 21:17:17 +03:00
79839ec31d
Start sending immediately instead of waiting for another loop
Vitaliy Filippov
2020-03-02 00:20:28 +03:00
9d96e4bf0b
Add simple "stub bench" using sync I/O
Vitaliy Filippov
2020-03-01 23:00:46 +03:00
d2b37e083a
Test: replace EPOLLIN with blocking sockets + io_uring reads
Vitaliy Filippov
2020-03-01 17:20:16 +03:00
9e18293497
Allow to specify multiple peers
Vitaliy Filippov
2020-03-01 00:46:07 +03:00
56765ab750
Send all iovecs at once
Vitaliy Filippov
2020-02-29 02:27:10 +03:00
fd05e13bc4
Use EPOLLET
Vitaliy Filippov
2020-02-29 01:46:12 +03:00
c41fd7ea18
Measure sending subops with data
Vitaliy Filippov
2020-02-29 01:46:03 +03:00
c6334afc94
Measure OSD op/subop latency
Vitaliy Filippov
2020-02-28 12:25:54 +03:00
2be4824a7a
Fix a small memory leak and BS_OP_SYNC mishandling, now fio does not hang during primary-osd test
Vitaliy Filippov
2020-02-28 01:46:39 +03:00
1733de2db6
Test & fix single-PG primary OSD
Vitaliy Filippov
2020-02-26 18:32:00 +03:00
df66a76ce2
...and make it work :)
Vitaliy Filippov
2020-02-25 22:52:03 +03:00
a406c62a71
Implement basic primary-sync-stabilize
Vitaliy Filippov
2020-02-25 20:10:17 +03:00
74673c761f
Make basic primary-write work
Vitaliy Filippov
2020-02-25 01:20:45 +03:00
09588a349f
Transform primary_r/w into "coroutines"
Vitaliy Filippov
2020-02-24 02:40:48 +03:00
4c0178f180
Fix some memory freeing
Vitaliy Filippov
2020-02-24 01:01:34 +03:00
5dd04abbac
Make bs_op pointer
Vitaliy Filippov
2020-02-23 23:19:11 +03:00
fe3c47812c
Rename osd_exec_secondary to osd_secondary
Vitaliy Filippov
2020-02-23 23:00:07 +03:00
88e56a564f
Rename osd_read_stripe_t to osd_rmw_stripe_t
Vitaliy Filippov
2020-02-23 22:52:49 +03:00
c71b67f2f7
Move SYNC_STAB_ALL into blockstore implementation
Vitaliy Filippov
2020-02-23 20:34:37 +03:00
4a52a15564
Rename osd_op_t.op to req
Vitaliy Filippov
2020-02-23 19:03:06 +03:00
72a89be912
Move uint8_t[] buffers into any_op_t/any_reply_t
Vitaliy Filippov
2020-02-23 18:57:20 +03:00
d4fd9d982a
Implement read-modify-write calculation and extract it into a separate file
Vitaliy Filippov
2020-02-23 02:11:43 +03:00
0e177c7a2a
Fix possible leak in blockstore_read
Vitaliy Filippov
2020-02-21 19:38:43 +03:00
ffe073473a
Remove hardcode of the EC(2+1) scheme, now it supports EC(k+1), fix some bugs
Vitaliy Filippov
2020-02-13 19:13:16 +03:00
b7ccd63104
Return version number from reads
Vitaliy Filippov
2020-02-12 12:30:50 +03:00
1100ad44bd
Cancel outbound operations when disconnecting a peer
Vitaliy Filippov
2020-02-11 13:59:10 +03:00
a66b34e04d
Implement event-driven PG peering
Vitaliy Filippov
2020-02-11 02:30:46 +03:00
327f310868
Remove copy-pasted read/write _blocking
Vitaliy Filippov
2020-02-09 19:23:03 +03:00
1513d0490a
Test and fix degraded-read
Vitaliy Filippov
2020-02-09 18:57:45 +03:00
97d3fc593c
Test and fix primary-read
Vitaliy Filippov
2020-02-09 18:22:29 +03:00
235d15422c
Mostly finish primary-OSD-read
Vitaliy Filippov
2020-02-03 12:35:02 +03:00
9fb2d3f840
Fill out the rest of the degraded read logic; now we need to make it a "coroutine"
Vitaliy Filippov
2020-02-02 00:05:56 +03:00
206c4eb655
WIP submit subops
Vitaliy Filippov
2020-01-31 14:07:16 +03:00
74d0196ba5
Stripe read reconstruction
Vitaliy Filippov
2020-01-31 13:46:23 +03:00
ec50957c41
WIP primary OSD read
Vitaliy Filippov
2020-01-30 22:06:46 +03:00
Vitaliy Filippov