This website requires JavaScript.
d68370304e
Support iovecs in cluster_client_t
Vitaliy Filippov
2020-06-24 01:31:48 +03:00
a22d9f38aa
Only use EPOLLOUT while connecting
Vitaliy Filippov
2020-06-23 20:18:04 +03:00
8736b3ad32
Add destructors, make ringloop optional in cluster_client_t
Vitaliy Filippov
2020-06-23 20:10:33 +03:00
62343c8022
Allow to turn synchronous recvmsg/sendmsg on with a config option
Vitaliy Filippov
2020-06-22 23:44:32 +03:00
9abaf5b735
Use epoll_manager in osd
Vitaliy Filippov
2020-06-20 01:28:18 +03:00
badf68c039
Support iovecs for read operations
Vitaliy Filippov
2020-06-18 02:07:20 +03:00
0f6d193d73
Postpone op callbacks to the end of handle_read(), fix a bug where primary OSD could reply -EPIPE with data to a read operation
Vitaliy Filippov
2020-06-16 01:36:38 +03:00
27ee14a4e6
Fix bugs in cluster_client
Vitaliy Filippov
2020-06-15 13:22:20 +03:00
64afec03ec
In theory, implement syncs and replay for the non-immediate commit mode
Vitaliy Filippov
2020-06-14 23:57:58 +03:00
4dde8b8a42
Oops, fix fio_sec_osd block_order parsing
Vitaliy Filippov
2020-06-09 00:52:00 +03:00
f5ccb154af
Benchmark reads in stub_bench, too
Vitaliy Filippov
2020-06-08 01:54:36 +03:00
73c80e2c39
Move accept_connections() to osd_messenger_t, add a simple uring OSD stub
Vitaliy Filippov
2020-06-08 01:32:12 +03:00
437dc5b630
Implement a FIO engine for testing cluster I/O
Vitaliy Filippov
2020-06-07 00:29:09 +03:00
226f5a2945
Allow to override block_size in fio_sec_osd
Vitaliy Filippov
2020-06-06 17:17:13 +03:00
2187d06eac
Add a parameter to pass the initial config to client
Vitaliy Filippov
2020-06-06 16:32:35 +03:00
c573bc6bb3
(Probably almost) implement cluster client
Vitaliy Filippov
2020-06-06 01:39:58 +03:00
2f6cf605a1
Rename cluster_client to osd_messenger
Vitaliy Filippov
2020-06-04 12:57:48 +03:00
05ea97119f
Fix BS_OP_LIST to account for deleted objects: only list the newest stable entry of each object
Vitaliy Filippov
2020-06-02 23:41:55 +03:00
571be0f380
Make deletions instantly stable
Vitaliy Filippov
2020-06-02 20:43:28 +03:00
985c309d7f
Remove duplicate code between blockstore_{rollback,stable} and blockstore_init
Vitaliy Filippov
2020-06-02 20:29:45 +03:00
a56f8cd14e
Simplify handle_primary_subop() arguments
Vitaliy Filippov
2020-06-02 18:44:23 +03:00
46e111272f
Replace assert(this_it == cur_op) with if() for the case of PG repeering
Vitaliy Filippov
2020-06-02 14:30:57 +03:00
165c204555
Fix BS_OP_DELETE (the implementation was untested up to this point)
Vitaliy Filippov
2020-06-02 14:02:39 +03:00
af5cd45071
Oh crap, got SIGPIPE. Add MSG_NOSIGNAL
Vitaliy Filippov
2020-06-02 11:41:08 +03:00
c3fe9ad0d1
Fix rebalancing writes (add a forgotten state resume)
Vitaliy Filippov
2020-06-02 01:18:36 +03:00
0fcdeae18b
Do not die if a peer is already stopped on flush error
Vitaliy Filippov
2020-06-01 23:06:31 +03:00
e6a4b634f8
Fix possible write stall
Vitaliy Filippov
2020-06-01 16:12:26 +03:00
c22e096943
Output journal offsets in debug trace in hex, add detailed "still waiting" messages
Vitaliy Filippov
2020-06-01 01:55:54 +03:00
45b1c2fbf1
Fix canceling of write operations on PG re-peer (which led to use-after-free, too...)
Vitaliy Filippov
2020-06-01 01:53:32 +03:00
3469bead67
Protect "delete this" with a stack refcounter
Vitaliy Filippov
2020-06-01 00:47:49 +03:00
3a5d488f19
Fix use-after-free in osd_flush.cpp
Vitaliy Filippov
2020-06-01 00:34:04 +03:00
73e4e30b1f
Auto-generate C++ header dependencies
Vitaliy Filippov
2020-05-31 15:35:34 +03:00
5feff1ffb9
Slightly cleanup socket send/receive code
Vitaliy Filippov
2020-05-31 15:03:27 +03:00
b466e215f0
Fix queued OP_SYNC execution
Vitaliy Filippov
2020-05-27 13:55:25 +03:00
36f995367f
Fix bind_address reporting
Vitaliy Filippov
2020-05-27 10:58:40 +03:00
0aca6e9ca8
Extract peer connect and read-write loop into a separate file (to be shared with the client library)
Vitaliy Filippov
2020-05-26 21:23:01 +03:00
fa98be6bc0
Allow to specify multiple etcd addresses
Vitaliy Filippov
2020-05-25 15:53:48 +03:00
256a7f2667
Free op->bs_op manually
Vitaliy Filippov
2020-05-25 15:09:55 +03:00
79bf57b6e2
Allow to override pg_stripe_size
Vitaliy Filippov
2020-05-25 15:28:32 +03:00
53f6aba3e6
Die when journal_sector_buffer_count is too small
Vitaliy Filippov
2020-05-24 17:22:12 +03:00
36595eb669
Print "Ran out of journal sector buffers" warning
Vitaliy Filippov
2020-05-24 16:48:50 +03:00
e09d0e0678
Several bug fixes
Vitaliy Filippov
2020-05-24 00:55:03 +03:00
d1602b50b3
Fix BS_OP_ROLLBACK removing an incorrect version
Vitaliy Filippov
2020-05-23 22:34:40 +03:00
7df384031a
Re-peer PGs after stopping the peer
Vitaliy Filippov
2020-05-23 18:41:28 +03:00
e614a98543
Add a sad FIXME :-)
Vitaliy Filippov
2020-05-23 15:38:40 +03:00
01dd3ef89e
Fix timerfd_manager triggering of multiple times at the same time
Vitaliy Filippov
2020-05-23 15:33:32 +03:00
cdccc23aff
Print [OSD $osd_num] in stats, print B/W only for ops that log bytes
Vitaliy Filippov
2020-05-23 15:19:51 +03:00
700428829a
Fix autosync_interval default not setting when autosync_interval is skipped in config
Vitaliy Filippov
2020-05-23 15:11:33 +03:00
6488d0044a
Ignore EPOLL_CTL_DEL ENOENT, fix detection of the rollback version
Vitaliy Filippov
2020-05-23 15:01:47 +03:00
393fe75900
Fix creepy (osd_op_t*)(long) casts
Vitaliy Filippov
2020-05-23 14:48:54 +03:00
f036eecf1c
Fix osd_rmw object recovery case (len==0)
Vitaliy Filippov
2020-05-23 14:46:42 +03:00
e56909fb45
Remove tv_send (unused) and timerfd_interval from blockstore
Vitaliy Filippov
2020-05-22 15:50:04 +03:00
fac75b0b57
Handle reweights in mon
Vitaliy Filippov
2020-05-22 12:52:27 +03:00
9f842ec9a5
Remove connect callback because it is always the same
Vitaliy Filippov
2020-05-22 12:45:12 +03:00
f6a01a4819
Extract "state-watching" etcd client into a separate file
Vitaliy Filippov
2020-05-21 21:00:54 +03:00
6202260018
Extract HTTP client functions from osd_t
Vitaliy Filippov
2020-05-21 11:39:01 +03:00
a61ede9951
Remove io_uring usage from osd_http and timerfd_manager
Vitaliy Filippov
2020-05-21 01:23:43 +03:00
f57731f8ca
Calculate total stats in the monitor
Vitaliy Filippov
2020-05-15 01:28:44 +03:00
19f25c7cd5
Handle integer overflow of the op_stat_count
Vitaliy Filippov
2020-05-15 00:57:49 +03:00
2c3e84cc41
Implement stop_all_pgs()
Vitaliy Filippov
2020-05-14 19:29:35 +03:00
7bda66b866
Do not crash when optimising PGs in an undersized cluster
Vitaliy Filippov
2020-05-13 00:08:31 +03:00
b467d0559f
Begin node.js storage monitor service
Vitaliy Filippov
2020-05-12 17:49:18 +03:00
c2c2eefea4
Duplicate host in osd/state and osd/stats, take PGs from /config/pgs.items
Vitaliy Filippov
2020-05-12 17:50:02 +03:00
5084ff7c6c
Measure & report recovery op count and bandwidth
Vitaliy Filippov
2020-05-11 16:12:18 +03:00
47b6f64106
Support level names
Vitaliy Filippov
2020-05-11 15:57:21 +03:00
f71d0c117b
Measure & report op bandwidth, include local blockstore ops in stats
Vitaliy Filippov
2020-05-11 02:58:13 +03:00
2b854948f9
Remove dead code
Vitaliy Filippov
2020-05-09 16:15:02 +03:00
e7f897ed65
Report hostname to etcd
Vitaliy Filippov
2020-05-09 02:33:43 +03:00
c26b6e1fc3
Support CRUSH-like multi-level placement trees
Vitaliy Filippov
2020-05-09 00:55:24 +03:00
aaa054e644
Fix optimize_change generating infeasible problems
Vitaliy Filippov
2020-05-07 13:45:03 +03:00
706a44d4d4
Fix optimize_initial in both perl and js versions
Vitaliy Filippov
2020-05-06 13:21:18 +03:00
842f88f94f
Rewrite LPOptimizer.pm to nodejs
Vitaliy Filippov
2020-05-06 02:08:15 +03:00
e8149e5848
Implement OSD_OP_DELETE
Vitaliy Filippov
2020-05-05 00:16:01 +03:00
6355b968f4
Track osd_set history and all_peers separately
Vitaliy Filippov
2020-05-04 01:32:24 +03:00
00cf24fbd7
Split osd_primary.cpp
Vitaliy Filippov
2020-05-03 11:04:20 +03:00
1bc08174f9
Sync before listing objects so flushes do not fail thereafter
Vitaliy Filippov
2020-05-01 12:55:15 +03:00
cd87333091
Fix PG state comparison leading to unclean PGs not flushing
Vitaliy Filippov
2020-05-01 02:09:18 +03:00
bd0fe6e4cc
Fix PGs not stopping during sync, fix state reporting autovivification of erased PGs
Vitaliy Filippov
2020-05-01 01:32:34 +03:00
ce78454215
Reply with -EROFS to write commands in readonly mode
Vitaliy Filippov
2020-05-01 00:54:34 +03:00
762bd42096
Fix use-after-free caused by "delete this" in handle_read
Vitaliy Filippov
2020-04-30 02:15:49 +03:00
7b57eeeeb3
Implement PG state locking and PG moving in response to etcd events
Vitaliy Filippov
2020-04-27 14:32:59 +03:00
ec4a52af48
Fix websocket (and timer!) bugs
Vitaliy Filippov
2020-04-26 01:33:50 +03:00
268b497c0b
Implement simple websocket client
Vitaliy Filippov
2020-04-25 23:11:50 +03:00
35481925b1
Implement very simple HTTP streaming to handle etcd watches
Vitaliy Filippov
2020-04-25 01:29:31 +03:00
895a80dfc4
Fix etcd 3.2 compatibility (no compare.target == LEASE, /kv/lease/revoke), fix small bugs
Vitaliy Filippov
2020-04-25 00:10:05 +03:00
caa01c6aaf
Acquire etcd leases, prevent starting two OSDs with the same number
Vitaliy Filippov
2020-04-24 16:46:02 +03:00
d398ddfd3b
Use snake_case for etcd requests
Vitaliy Filippov
2020-04-24 01:08:55 +03:00
0f2b8dbf6f
Use a single timerfd_manager for all timers
Vitaliy Filippov
2020-04-24 01:07:19 +03:00
4f42e9659e
Use etcd instead of Consul
Vitaliy Filippov
2020-04-24 01:03:55 +03:00
7cf71a8031
Fix timerfd_manager: remove timer, then call callback
Vitaliy Filippov
2020-04-21 12:45:18 +03:00
9d22559bcf
Start peering immediately when loading PGs
Vitaliy Filippov
2020-04-21 02:27:13 +03:00
8c03e3ebab
Lock Blockstore devices exclusively by default
Vitaliy Filippov
2020-04-21 01:59:11 +03:00
2a640ba2e8
Remove range port selection (leads to races)
Vitaliy Filippov
2020-04-21 00:10:59 +03:00
6a21ea207e
Check peer config (at least, number) after connecting
Vitaliy Filippov
2020-04-21 00:08:54 +03:00
642802b595
Auto-select port numbers
Vitaliy Filippov
2020-04-20 17:44:03 +03:00
ff38b464a5
Add consul & connect timeouts, report state before loading PGs, move init_primary to osd_cluster
Vitaliy Filippov
2020-04-20 15:43:07 +03:00
663153713b
Reconnect to peers after connecting drops
Vitaliy Filippov
2020-04-19 01:01:26 +03:00
dc57c5c362
Report PG states again, clear PG history on reaching active+clean
Vitaliy Filippov
2020-04-19 00:48:23 +03:00
f95299b769
Take PG history into account when starting PGs
Vitaliy Filippov
2020-04-19 00:20:18 +03:00
9126ffb0f9
Fix PG loading - now it works, at least once
Vitaliy Filippov
2020-04-17 02:33:44 +03:00
Vitaliy Filippov