thebestvibedev/tromcho.net
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add vitastor-cli modify-osd command

Browse Source
This commit is contained in:
Vitaliy Filippov
2024-07-09 16:52:19 +03:00
parent 8535bccf4c
commit 3aef6682fb
7 changed files with 300 additions and 64 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/usage/cli.en.md
+36 -30
View File
@@ -250,56 +250,62 @@ redundancy degradation.
## osd-tree
`vitastor-cli osd-tree`
`vitastor-cli osd-tree [-l|--long]`
Show current OSD tree.
Show current OSD tree, optionally with I/O statistics if -l is specified.
Example output:
```
TYPE NAME UP SIZE USED% TAGS WEIGHT BLOCK BITMAP IMM
TYPE NAME UP SIZE USED% TAGS WEIGHT BLOCK BITMAP IMM NOOUT
host kaveri
disk nvme0n1p1
osd 3 down 100G 0 % abc,kaveri 1 128k 4k none
osd 4 down 100G 0 % 1 128k 4k none
osd 3 down 100G 0 % abc,kaveri 1 128k 4k none -
osd 4 down 100G 0 % 1 128k 4k none -
disk nvme1n1p1
osd 5 down 100G 0 % abc,kaveri 1 128k 4k none
osd 6 down 100G 0 % 1 128k 4k none
osd 5 down 100G 0 % abc,kaveri 1 128k 4k none -
osd 6 down 100G 0 % 1 128k 4k none -
host stump
osd 1 up 100G 37.29 % osdone 1 128k 4k all
osd 2 up 100G 26.8 % abc 1 128k 4k all
osd 7 up 100G 21.84 % 1 128k 4k all
osd 8 up 100G 21.63 % 1 128k 4k all
osd 9 up 100G 20.69 % 1 128k 4k all
osd 10 up 100G 21.61 % 1 128k 4k all
osd 11 up 100G 21.53 % 1 128k 4k all
osd 12 up 100G 22.4 % 1 128k 4k all
osd 1 up 100G 37.29 % osdone 1 128k 4k all -
osd 2 up 100G 26.8 % abc 1 128k 4k all -
osd 7 up 100G 21.84 % 1 128k 4k all -
osd 8 up 100G 21.63 % 1 128k 4k all -
osd 9 up 100G 20.69 % 1 128k 4k all -
osd 10 up 100G 21.61 % 1 128k 4k all -
osd 11 up 100G 21.53 % 1 128k 4k all -
osd 12 up 100G 22.4 % 1 128k 4k all -
```
## ls-osd
`vitastor-cli osds|ls-osd|osd-ls`
`vitastor-cli osds|ls-osd|osd-ls [-l|--long]`
Show current OSDs as list.
Show current OSDs as list, optionally with I/O statistics if -l is specified.
Example output:
```
OSD PARENT UP SIZE USED% TAGS WEIGHT BLOCK BITMAP IMM
3 kaveri/nvme0n1p1 down 100G 0 % globl,kaveri 1 128k 4k none
4 kaveri/nvme0n1p1 down 100G 0 % 1 128k 4k none
5 kaveri/nvme1n1p1 down 100G 0 % globl,kaveri 1 128k 4k none
6 kaveri/nvme1n1p1 down 100G 0 % 1 128k 4k none
1 stump up 100G 37.29 % osdone 1 128k 4k all
2 stump up 100G 26.8 % globl 1 128k 4k all
7 stump up 100G 21.84 % 1 128k 4k all
8 stump up 100G 21.63 % 1 128k 4k all
9 stump up 100G 20.69 % 1 128k 4k all
10 stump up 100G 21.61 % 1 128k 4k all
11 stump up 100G 21.53 % 1 128k 4k all
12 stump up 100G 22.4 % 1 128k 4k all
OSD PARENT UP SIZE USED% TAGS WEIGHT BLOCK BITMAP IMM NOOUT
3 kaveri/nvme0n1p1 down 100G 0 % globl,kaveri 1 128k 4k none -
4 kaveri/nvme0n1p1 down 100G 0 % 1 128k 4k none -
5 kaveri/nvme1n1p1 down 100G 0 % globl,kaveri 1 128k 4k none -
6 kaveri/nvme1n1p1 down 100G 0 % 1 128k 4k none -
1 stump up 100G 37.29 % osdone 1 128k 4k all -
2 stump up 100G 26.8 % globl 1 128k 4k all -
7 stump up 100G 21.84 % 1 128k 4k all -
8 stump up 100G 21.63 % 1 128k 4k all -
9 stump up 100G 20.69 % 1 128k 4k all -
10 stump up 100G 21.61 % 1 128k 4k all -
11 stump up 100G 21.53 % 1 128k 4k all -
12 stump up 100G 22.4 % 1 128k 4k all -
```
## modify-osd
`vitastor-cli modify-osd [--tags tag1,tag2,...] [--reweight <number>] [--noout true/false] <osd_number>`
Set OSD reweight, tags or noout flag. See detail description in [OSD config documentation](../config.pool.en.md#osd-settings).
## create-pool
`vitastor-cli create-pool|pool-create <name> (-s <pg_size>|--ec <N>+<K>) -n <pg_count> [OPTIONS]`
docs/usage/cli.ru.md
+36 -30
View File
@@ -267,56 +267,62 @@ vitastor-cli snap-create [-p|--pool <id|name>] <image>@<snapshot>
## osd-tree
`vitastor-cli osd-tree`
`vitastor-cli osd-tree [-l|--long]`
Показать дерево OSD.
Показать дерево OSD, со статистикой ввода-вывода, если установлено -l.
Пример вывода:
```
TYPE NAME UP SIZE USED% TAGS WEIGHT BLOCK BITMAP IMM
TYPE NAME UP SIZE USED% TAGS WEIGHT BLOCK BITMAP IMM NOOUT
host kaveri
disk nvme0n1p1
osd 3 down 100G 0 % globl,kaveri 1 128k 4k none
osd 4 down 100G 0 % 1 128k 4k none
osd 3 down 100G 0 % globl,kaveri 1 128k 4k none -
osd 4 down 100G 0 % 1 128k 4k none -
disk nvme1n1p1
osd 5 down 100G 0 % globl,kaveri 1 128k 4k none
osd 6 down 100G 0 % 1 128k 4k none
osd 5 down 100G 0 % globl,kaveri 1 128k 4k none -
osd 6 down 100G 0 % 1 128k 4k none -
host stump
osd 1 up 100G 37.29 % osdone 1 128k 4k all
osd 2 up 100G 26.8 % globl 1 128k 4k all
osd 7 up 100G 21.84 % 1 128k 4k all
osd 8 up 100G 21.63 % 1 128k 4k all
osd 9 up 100G 20.69 % 1 128k 4k all
osd 10 up 100G 21.61 % 1 128k 4k all
osd 11 up 100G 21.53 % 1 128k 4k all
osd 12 up 100G 22.4 % 1 128k 4k all
osd 1 up 100G 37.29 % osdone 1 128k 4k all -
osd 2 up 100G 26.8 % globl 1 128k 4k all -
osd 7 up 100G 21.84 % 1 128k 4k all -
osd 8 up 100G 21.63 % 1 128k 4k all -
osd 9 up 100G 20.69 % 1 128k 4k all -
osd 10 up 100G 21.61 % 1 128k 4k all -
osd 11 up 100G 21.53 % 1 128k 4k all -
osd 12 up 100G 22.4 % 1 128k 4k all -
```
## ls-osd
`vitastor-cli osds|ls-osd|osd-ls`
`vitastor-cli osds|ls-osd|osd-ls [-l|--long]`
Показать список OSD.
Показать список OSD, со статистикой ввода-вывода, если установлено -l.
Пример вывода:
```
OSD PARENT UP SIZE USED% TAGS WEIGHT BLOCK BITMAP IMM
3 kaveri/nvme0n1p1 down 100G 0 % globl,kaveri 1 128k 4k none
4 kaveri/nvme0n1p1 down 100G 0 % 1 128k 4k none
5 kaveri/nvme1n1p1 down 100G 0 % globl,kaveri 1 128k 4k none
6 kaveri/nvme1n1p1 down 100G 0 % 1 128k 4k none
1 stump up 100G 37.29 % osdone 1 128k 4k all
2 stump up 100G 26.8 % globl 1 128k 4k all
7 stump up 100G 21.84 % 1 128k 4k all
8 stump up 100G 21.63 % 1 128k 4k all
9 stump up 100G 20.69 % 1 128k 4k all
10 stump up 100G 21.61 % 1 128k 4k all
11 stump up 100G 21.53 % 1 128k 4k all
12 stump up 100G 22.4 % 1 128k 4k all
OSD PARENT UP SIZE USED% TAGS WEIGHT BLOCK BITMAP IMM NOOUT
3 kaveri/nvme0n1p1 down 100G 0 % globl,kaveri 1 128k 4k none -
4 kaveri/nvme0n1p1 down 100G 0 % 1 128k 4k none -
5 kaveri/nvme1n1p1 down 100G 0 % globl,kaveri 1 128k 4k none -
6 kaveri/nvme1n1p1 down 100G 0 % 1 128k 4k none -
1 stump up 100G 37.29 % osdone 1 128k 4k all -
2 stump up 100G 26.8 % globl 1 128k 4k all -
7 stump up 100G 21.84 % 1 128k 4k all -
8 stump up 100G 21.63 % 1 128k 4k all -
9 stump up 100G 20.69 % 1 128k 4k all -
10 stump up 100G 21.61 % 1 128k 4k all -
11 stump up 100G 21.53 % 1 128k 4k all -
12 stump up 100G 22.4 % 1 128k 4k all -
```
## modify-osd
`vitastor-cli modify-osd [--tags tag1,tag2,...] [--reweight <number>] [--noout true/false] <osd_number>`
Установить вес OSD, теги или флаг noout. Смотрите подробное описание в [документации настроек OSD](../config.pool.ru.md#настройки-osd).
## create-pool
`vitastor-cli create-pool|pool-create <name> (-s <pg_size>|--ec <N>+<K>) -n <pg_count> [OPTIONS]`
src/cmd/CMakeLists.txt
+1
View File
@@ -12,6 +12,7 @@ add_library(vitastor_cli STATIC
cli_ls.cpp
cli_create.cpp
cli_modify.cpp
cli_modify_osd.cpp
cli_osd_tree.cpp
cli_flatten.cpp
cli_merge.cpp
src/cmd/cli.cpp
+7 -4
View File
@@ -118,11 +118,14 @@ static const char* help_text =
" With --dry-run only checks if deletion is possible without data loss and\n"
" redundancy degradation.\n"
"\n"
"vitastor-cli osd-tree\n"
" Show current OSD tree.\n"
"vitastor-cli osd-tree [-l|--long]\n"
" Show current OSD tree, optionally with I/O statistics if -l is specified.\n"
"\n"
"vitastor-cli osds|ls-osd|osd-ls\n"
" Show current OSDs as list.\n"
"vitastor-cli osds|ls-osd|osd-ls [-l|--long]\n"
" Show current OSDs as list, optionally with I/O statistics if -l is specified.\n"
"\n"
"vitastor-cli modify-osd [--tags tag1,tag2,...] [--reweight <number>] [--noout true/false] <osd_number>\n"
" Set OSD reweight, tags or noout flag.\n"
"\n"
"vitastor-cli create-pool|pool-create <name> (-s <pg_size>|--ec <N>+<K>) -n <pg_count> [OPTIONS]\n"
" Create a pool. Required parameters:\n"
src/cmd/cli.h
+1
View File
@@ -65,6 +65,7 @@ public:
std::function<bool(cli_result_t &)> start_ls(json11::Json);
std::function<bool(cli_result_t &)> start_merge(json11::Json);
std::function<bool(cli_result_t &)> start_modify(json11::Json);
std::function<bool(cli_result_t &)> start_modify_osd(json11::Json cfg);
std::function<bool(cli_result_t &)> start_osd_tree(json11::Json);
std::function<bool(cli_result_t &)> start_pool_create(json11::Json);
std::function<bool(cli_result_t &)> start_pool_modify(json11::Json);
src/cmd/cli_modify_osd.cpp
+210
View File
@@ -0,0 +1,210 @@
// Copyright (c) Vitaliy Filippov, 2019+
// License: VNPL-1.1 (see README.md for details)
#include "cli.h"
#include "cluster_client.h"
#include "str_util.h"
#include "http_client.h"
// Reweight OSD, change tags or set noout flag
struct osd_changer_t
{
cli_tool_t *parent;
uint64_t osd_num = 0;
bool set_tags = false;
std::vector<std::string> new_tags;
bool set_reweight = false;
double new_reweight = 1;
bool set_noout = false;
double new_noout = false;
bool force = false;
json11::Json::object osd_cfg;
uint64_t osd_cfg_mod_rev = 0;
json11::Json::array compare, success;
int state = 0;
std::function<bool(cli_result_t &)> cb;
cli_result_t result;
bool is_done()
{
return state == 100;
}
void loop()
{
if (state == 1)
goto resume_1;
else if (state == 2)
goto resume_2;
if (!osd_num)
{
result = (cli_result_t){ .err = EINVAL, .text = "OSD number is missing" };
state = 100;
return;
}
if (!set_tags && !set_reweight && !set_noout)
{
result = (cli_result_t){ .err = EINVAL, .text = "Nothing to update" };
state = 100;
return;
}
if (set_reweight && new_reweight < 0)
{
result = (cli_result_t){ .err = EINVAL, .text = "Reweight can't be negative" };
state = 100;
return;
}
parent->etcd_txn(json11::Json::object {
{ "success", json11::Json::array {
json11::Json::object {
{ "request_range", json11::Json::object {
{ "key", base64_encode(parent->cli->st_cli.etcd_prefix+"/osd/stats/"+std::to_string(osd_num)) },
} },
},
json11::Json::object {
{ "request_range", json11::Json::object {
{ "key", base64_encode(parent->cli->st_cli.etcd_prefix+"/config/osd/"+std::to_string(osd_num)) },
} },
},
} },
});
state = 1;
resume_1:
if (parent->waiting > 0)
return;
if (parent->etcd_err.err)
{
result = parent->etcd_err;
state = 100;
return;
}
{
auto osd_stats = parent->cli->st_cli.parse_etcd_kv(parent->etcd_result["responses"][0]["response_range"]["kvs"][0]).value;
if (!osd_stats.is_object() && !force)
{
result = (cli_result_t){ .err = ENOENT, .text = "OSD "+std::to_string(osd_num)+" does not exist. Use --force to set configuration anyway" };
state = 100;
return;
}
auto kv = parent->cli->st_cli.parse_etcd_kv(parent->etcd_result["responses"][1]["response_range"]["kvs"][0]);
osd_cfg_mod_rev = kv.mod_revision;
osd_cfg = kv.value.object_items();
if (set_reweight)
{
if (new_reweight != 1)
osd_cfg["reweight"] = new_reweight;
else
osd_cfg.erase("reweight");
}
if (set_tags)
{
if (new_tags.size())
osd_cfg["tags"] = new_tags;
else
osd_cfg.erase("tags");
}
if (set_noout)
{
if (new_noout)
osd_cfg["noout"] = true;
else
osd_cfg.erase("noout");
}
compare.push_back(json11::Json::object {
{ "target", "MOD" },
{ "key", base64_encode(parent->cli->st_cli.etcd_prefix+"/config/osd/"+std::to_string(osd_num)) },
{ "result", "LESS" },
{ "mod_revision", osd_cfg_mod_rev+1 },
});
if (osd_cfg.size())
{
success.push_back(json11::Json::object {
{ "request_delete_range", json11::Json::object {
{ "key", base64_encode(parent->cli->st_cli.etcd_prefix+"/config/osd/"+std::to_string(osd_num)) },
} },
});
}
else
{
success.push_back(json11::Json::object {
{ "request_put", json11::Json::object {
{ "key", base64_encode(parent->cli->st_cli.etcd_prefix+"/config/osd/"+std::to_string(osd_num)) },
{ "value", base64_encode(json11::Json(osd_cfg).dump()) },
} },
});
}
}
parent->etcd_txn(json11::Json::object {
{ "compare", compare },
{ "success", success },
});
state = 2;
resume_2:
if (parent->waiting > 0)
return;
if (parent->etcd_err.err)
{
result = parent->etcd_err;
state = 100;
return;
}
if (!parent->etcd_result["succeeded"].bool_value())
{
result = (cli_result_t){ .err = EAGAIN, .text = "OSD "+std::to_string(osd_num)+" configuration was modified by someone else, please repeat your request" };
state = 100;
return;
}
result = (cli_result_t){
.err = 0,
.text = "OSD "+std::to_string(osd_num)+" configuration modified",
.data = osd_cfg,
};
state = 100;
}
};
std::function<bool(cli_result_t &)> cli_tool_t::start_modify_osd(json11::Json cfg)
{
auto changer = new osd_changer_t();
changer->parent = this;
changer->osd_num = cfg["osd_num"].uint64_value();
if (!cfg["tags"].is_null())
{
changer->set_tags = true;
if (cfg["tags"].is_string())
{
if (cfg["tags"].string_value() != "")
changer->new_tags = explode(",", cfg["tags"].string_value(), true);
}
else if (cfg["tags"].is_array())
{
for (auto item: cfg["tags"].array_items())
changer->new_tags.push_back(item.as_string());
}
}
if (!cfg["reweight"].is_null())
{
changer->set_reweight = true;
changer->new_reweight = cfg["reweight"].number_value();
}
if (!cfg["noout"].is_null())
{
changer->set_noout = true;
changer->new_noout = json_is_true(cfg["noout"]);
}
changer->force = cfg["force"].bool_value();
return [changer](cli_result_t & result)
{
changer->loop();
if (changer->is_done())
{
result = changer->result;
delete changer;
return true;
}
return false;
};
}
src/cmd/cli_osd_tree.cpp
+9
View File
@@ -17,6 +17,7 @@ struct placement_osd_t
uint64_t free;
bool up;
double reweight;
bool noout;
uint32_t block_size, bitmap_granularity, immediate_commit;
};
@@ -132,6 +133,7 @@ resume_1:
.free = kv.second["free"].uint64_value(),
.up = parent->cli->st_cli.peer_states.find(kv.first) != parent->cli->st_cli.peer_states.end(),
.reweight = 1,
.noout = false,
.block_size = (uint32_t)kv.second["data_block_size"].uint64_value(),
.bitmap_granularity = (uint32_t)kv.second["bitmap_granularity"].uint64_value(),
.immediate_commit = etcd_state_client_t::parse_immediate_commit(kv.second["immediate_commit"].string_value(), IMMEDIATE_NONE),
@@ -154,6 +156,7 @@ resume_1:
for (auto & jtag: osd_cfg["tags"].array_items())
osd.tags.push_back(jtag.string_value());
}
osd.noout = osd_cfg["noout"].bool_value();
}
auto np_it = node_placement.find(std::to_string(osd.num));
if (np_it != node_placement.end())
@@ -219,6 +222,7 @@ resume_1:
{ "size", osd.size },
{ "free", osd.free },
{ "reweight", osd.reweight },
{ "noout", osd.noout },
{ "tags", osd.tags },
{ "block", (uint64_t)osd.block_size },
{ "bitmap", (uint64_t)osd.bitmap_granularity },
@@ -261,6 +265,7 @@ resume_1:
{ "size", format_size(osd.size, false, true) },
{ "used", format_q(100.0*(osd.size - osd.free)/osd.size)+" %" },
{ "reweight", format_q(osd.reweight) },
{ "noout", osd.noout ? "noout" : "-" },
{ "tags", implode(",", osd.tags) },
{ "block", format_size(osd.block_size, false, true) },
{ "bitmap", format_size(osd.bitmap_granularity, false, true) },
@@ -333,6 +338,10 @@ resume_1:
{ "key", "commit" },
{ "title", "IMM" },
});
cols.push_back(json11::Json::object{
{ "key", "noout" },
{ "title", "NOOUT" },
});
if (show_stats)
{
cols.push_back(json11::Json::object{