468 lines
19 KiB
Markdown
468 lines
19 KiB
Markdown
[Documentation](../../README.md#documentation) → Usage → Vitastor CLI
|
|
|
|
-----
|
|
|
|
[Читать на русском](cli.ru.md)
|
|
|
|
# Vitastor CLI
|
|
|
|
vitastor-cli is a command-line tool for administrative tasks like image management.
|
|
|
|
It supports the following commands:
|
|
|
|
- [status](#status)
|
|
- [df](#df)
|
|
- [ls](#ls)
|
|
- [create](#create)
|
|
- [snap-create](#create)
|
|
- [modify](#modify)
|
|
- [dd](#dd)
|
|
- [rm](#rm)
|
|
- [flatten](#flatten)
|
|
- [rm-data](#rm-data)
|
|
- [merge-data](#merge-data)
|
|
- [describe](#describe)
|
|
- [fix](#fix)
|
|
- [alloc-osd](#alloc-osd)
|
|
- [rm-osd](#rm-osd)
|
|
- [osd-tree](#osd-tree)
|
|
- [ls-osd](#ls-osd)
|
|
- [modify-osd](#modify-osd)
|
|
- [pg-list](#pg-list)
|
|
- [create-pool](#create-pool)
|
|
- [modify-pool](#modify-pool)
|
|
- [ls-pools](#ls-pools)
|
|
- [rm-pool](#rm-pool)
|
|
|
|
Global options:
|
|
|
|
```
|
|
--config_path FILE Path to Vitastor configuration file
|
|
--etcd_address URL Etcd connection address
|
|
--iodepth N Send N operations in parallel to each OSD when possible (default 32)
|
|
--parallel_osds M Work with M osds in parallel when possible (default 4)
|
|
--progress 1|0 Report progress (default 1)
|
|
--cas 1|0 Use CAS writes for flatten, merge, rm (default is decide automatically)
|
|
--no-color Disable colored output
|
|
--json JSON output
|
|
```
|
|
|
|
## status
|
|
|
|
`vitastor-cli status`
|
|
|
|
Show cluster status.
|
|
|
|
Example output:
|
|
|
|
```
|
|
cluster:
|
|
etcd: 1 / 1 up, 1.8 M database size
|
|
mon: 1 up, master stump
|
|
osd: 8 / 12 up
|
|
|
|
data:
|
|
raw: 498.5 G used, 301.2 G / 799.7 G available, 399.8 G down
|
|
state: 156.6 G clean, 97.6 G misplaced
|
|
pools: 2 / 3 active
|
|
pgs: 30 active
|
|
34 active+has_misplaced
|
|
32 offline
|
|
|
|
io:
|
|
client: 0 B/s rd, 0 op/s rd, 0 B/s wr, 0 op/s wr
|
|
rebalance: 989.8 M/s, 7.9 K op/s
|
|
```
|
|
|
|
## df
|
|
|
|
`vitastor-cli df`
|
|
|
|
Show pool space statistics.
|
|
|
|
Example output:
|
|
|
|
```
|
|
NAME SCHEME PGS TOTAL USED AVAILABLE USED% EFFICIENCY
|
|
testpool 2/1 32 100 G 34.2 G 60.7 G 39.23% 100%
|
|
size1 1/1 32 199.9 G 10 G 121.5 G 39.23% 100%
|
|
kaveri 2/1 32 0 B 10 G 0 B 100% 0%
|
|
```
|
|
|
|
In the example above, "kaveri" pool has "zero" efficiency because all its OSD are down.
|
|
|
|
## ls
|
|
|
|
`vitastor-cli ls [-l] [-p POOL] [--sort FIELD] [-r] [-n N] [<glob> ...]`
|
|
|
|
List images (only matching `<glob>` pattern(s) if passed).
|
|
|
|
Options:
|
|
|
|
```
|
|
--exact Do not match glob patterns as names, select only exact name matches.
|
|
-p|--pool POOL Filter images by pool ID or name
|
|
-l|--long Also report allocated size and I/O statistics
|
|
--del Also include delete operation statistics
|
|
--sort FIELD Sort by specified field (name, size, used_size, <read|write|delete>_<iops|bps|lat|queue>)
|
|
-r|--reverse Sort in descending order
|
|
-n|--count N Only list first N items
|
|
--tree Show image snapshot/clone tree
|
|
```
|
|
|
|
Example output:
|
|
|
|
```
|
|
NAME POOL SIZE USED READ IOPS QUEUE LAT WRITE IOPS QUEUE LAT FLAGS PARENT
|
|
debian9 testpool 20 G 12.3 G 0 B/s 0 0 0 us 0 B/s 0 0 0 us RO
|
|
pve/vm-100-disk-0 testpool 20 G 0 B 0 B/s 0 0 0 us 0 B/s 0 0 0 us - debian9
|
|
pve/base-101-disk-0 testpool 20 G 0 B 0 B/s 0 0 0 us 0 B/s 0 0 0 us RO debian9
|
|
pve/vm-102-disk-0 testpool 32 G 36.4 M 0 B/s 0 0 0 us 0 B/s 0 0 0 us - pve/base-101-disk-0
|
|
debian9-test testpool 20 G 36.6 M 0 B/s 0 0 0 us 0 B/s 0 0 0 us - debian9
|
|
bench testpool 10 G 10 G 0 B/s 0 0 0 us 0 B/s 0 0 0 us -
|
|
bench-kaveri kaveri 10 G 10 G 0 B/s 0 0 0 us 0 B/s 0 0 0 us -
|
|
```
|
|
|
|
## create
|
|
|
|
`vitastor-cli create -s|--size <size> [-p|--pool <id|name>] [--parent <parent_name>[@<snapshot>]] <name>`
|
|
|
|
Create an image. You may use K/M/G/T suffixes for `<size>`. If `--parent` is specified,
|
|
a copy-on-write image clone is created. Parent must be a snapshot (readonly image).
|
|
Pool must be specified if there is more than one pool.
|
|
|
|
```
|
|
vitastor-cli create --snapshot <snapshot> [-p|--pool <id|name>] <image>
|
|
vitastor-cli snap-create [-p|--pool <id|name>] <image>@<snapshot>
|
|
```
|
|
|
|
Create a snapshot of image `<name>` (either form can be used). May be used live if only a single writer is active.
|
|
|
|
See also about [how to export snapshots](qemu.en.md#exporting-snapshots).
|
|
|
|
## modify
|
|
|
|
`vitastor-cli modify <name> [--rename <new-name>] [--resize <size>] [--readonly | --readwrite] [-f|--force] [--down-ok]`
|
|
|
|
Rename, resize image or change its readonly status. Images with children can't be made read-write.
|
|
If the new size is smaller than the old size, extra data will be purged.
|
|
You should resize file system in the image, if present, before shrinking it.
|
|
|
|
* `--deleted 1|0` - Set/clear 'deleted image' flag (set automatically during unfinished deletes).
|
|
* `-f|--force` - Proceed with shrinking or setting readwrite flag even if the image has children.
|
|
* `--down-ok` - Proceed with shrinking even if some data will be left on unavailable OSDs.
|
|
|
|
## dd
|
|
|
|
```
|
|
vitastor-cli dd [iimg=<image> | if=<file>] [oimg=<image> | of=<file>] [bs=1M] \
|
|
[count=N] [seek/oseek=N] [skip/iseek=M] [iodepth=N] [status=progress] \
|
|
[conv=nocreat,noerror,nofsync,trunc,nosparse] [iflag=direct] [oflag=direct,append]
|
|
```
|
|
|
|
Copy data between Vitastor images, files and pipes.
|
|
|
|
Options can be specified in classic dd style (`key=value`) or like usual (`--key value`).
|
|
|
|
| <!-- --> | <!-- --> |
|
|
|-----------------|-------------------------------------------------------------------------|
|
|
| `iimg=<image>` | Copy from Vitastor image `<image>` |
|
|
| `if=<file>` | Copy from file `<file>` |
|
|
| `oimg=<image>` | Copy to Vitastor image `<image>` |
|
|
| `of=<file>` | Copy to file `<file>` |
|
|
| `bs=1M` | Set copy block size |
|
|
| `count=N` | Copy only N input blocks. If N ends in B it counts bytes, not blocks |
|
|
| `seek/oseek=N` | Skip N output blocks. If N ends in B it counts bytes, not blocks |
|
|
| `skip/iseek=N` | Skip N input blocks. If N ends in B it counts bytes, not blocks |
|
|
| `iodepth=N` | Send N reads or writes in parallel (default 4) |
|
|
| `status=LEVEL` | The LEVEL of information to print to stderr: none/noxfer/progress |
|
|
| `size=N` | Specify size for the created output file/image (defaults to input size) |
|
|
| `iflag=direct` | For input files only: use direct I/O |
|
|
| `oflag=direct` | For output files only: use direct I/O |
|
|
| `oflag=append` | For files only: append to output file |
|
|
| `conv=nocreat` | Do not create output file/image |
|
|
| `conv=trunc` | Truncate output file/image |
|
|
| `conv=noerror` | Continue copying after errors |
|
|
| `conv=nofsync` | Do not call fsync before finishing (default behaviour is fsync) |
|
|
| `conv=nosparse` | Write all output blocks including all-zero blocks |
|
|
|
|
## rm
|
|
|
|
`vitastor-cli rm <from> [<to>] [--writers-stopped] [--down-ok]`
|
|
|
|
`vitastor-cli rm (--exact|--matching) <glob> ...`
|
|
|
|
Remove layer(s) and rebase all their children accordingly.
|
|
|
|
In the first form, remove `<from>` or layers between `<from>` and its child `<to>`.
|
|
|
|
In the second form, remove all images with exact or pattern-matched names.
|
|
|
|
Options:
|
|
|
|
* `--writers-stopped` allows optimised removal in case of a single 'slim' read-write
|
|
child and 'fat' removed parent: the child is merged into parent and parent is renamed
|
|
to child in that case. In other cases parent layers are always merged into children.
|
|
* `--exact` - remove multiple images with names matching given glob patterns.
|
|
* `--matching` - remove multiple images with given names
|
|
* `--down-ok` - continue deletion/merging even if some data will be left on unavailable OSDs.
|
|
|
|
## flatten
|
|
|
|
`vitastor-cli flatten <layer>`
|
|
|
|
Flatten a layer, i.e. merge data and detach it from parents.
|
|
|
|
## rm-data
|
|
|
|
`vitastor-cli rm-data --pool <pool> --inode <inode> [--wait-list] [--min-offset <offset>]`
|
|
|
|
Remove inode data without changing metadata.
|
|
|
|
```
|
|
--wait-list Retrieve full objects listings before starting to remove objects.
|
|
Requires more memory, but allows to show correct removal progress.
|
|
--min-offset Purge only data starting with specified offset.
|
|
--max-offset Purge only data before specified offset.
|
|
--client_wait_up_timeout 16 Timeout for waiting until PGs are up in seconds.
|
|
```
|
|
|
|
## merge-data
|
|
|
|
`vitastor-cli merge-data <from> <to> [--target <target>]`
|
|
|
|
Merge layer data without changing metadata. Merge `<from>`..`<to>` to `<target>`.
|
|
`<to>` must be a child of `<from>` and `<target>` may be one of the layers between
|
|
`<from>` and `<to>`, including `<from>` and `<to>`.
|
|
|
|
## describe
|
|
|
|
`vitastor-cli describe [OPTIONS]`
|
|
|
|
Describe unclean object locations in the cluster. Options:
|
|
|
|
```
|
|
--osds <osds>
|
|
Only list objects from primary OSD(s) <osds>.
|
|
--object-state <states>
|
|
Only list objects in given state(s). State(s) may include:
|
|
degraded, misplaced, incomplete, corrupted, inconsistent.
|
|
--pool <pool name or number>
|
|
Only list objects in the given pool.
|
|
--pg <pg number>
|
|
Only list objects in the given PG of the pool.
|
|
--inode, --min-inode, --max-inode
|
|
Restrict listing to specific inode numbers.
|
|
--min-offset, --max-offset
|
|
Restrict listing to specific offsets inside inodes.
|
|
```
|
|
|
|
## fix
|
|
|
|
`vitastor-cli fix [--objects <objects>] [--bad-osds <osds>] [--part <part>] [--check no]`
|
|
|
|
Fix inconsistent objects in the cluster by deleting some copies.
|
|
|
|
```
|
|
--objects <objects>
|
|
Objects to fix, either in plain text or JSON format. If not specified,
|
|
object list will be read from STDIN in one of the same formats.
|
|
Plain text format: 0x<inode>:0x<stripe> <any delimiter> 0x<inode>:0x<stripe> ...
|
|
JSON format: [{"inode":"0x...","stripe":"0x..."},...]
|
|
--bad-osds <osds>
|
|
Remove inconsistent copies/parts of objects from these OSDs, effectively
|
|
marking them bad and allowing Vitastor to recover objects from other copies.
|
|
--part <number>
|
|
Only remove EC part <number> (from 0 to pg_size-1), required for extreme
|
|
edge cases where one OSD has multiple parts of a EC object.
|
|
--check no
|
|
Do not recheck that requested objects are actually inconsistent,
|
|
delete requested copies/parts anyway.
|
|
```
|
|
|
|
## alloc-osd
|
|
|
|
`vitastor-cli alloc-osd`
|
|
|
|
Allocate a new OSD number and reserve it by creating empty `/osd/stats/<n>` key.
|
|
|
|
## rm-osd
|
|
|
|
`vitastor-cli rm-osd [--force] [--allow-data-loss] [--dry-run] <osd_id> [osd_id...]`
|
|
|
|
Remove metadata and configuration for specified OSD(s) from etcd.
|
|
|
|
Refuses to remove OSDs with data without `--force` and `--allow-data-loss`.
|
|
|
|
With `--dry-run` only checks if deletion is possible without data loss and
|
|
redundancy degradation.
|
|
|
|
## osd-tree
|
|
|
|
`vitastor-cli osd-tree [-l|--long]`
|
|
|
|
Show current OSD tree, optionally with I/O statistics if -l is specified.
|
|
|
|
Example output:
|
|
|
|
```
|
|
TYPE NAME UP SIZE USED% TAGS WEIGHT BLOCK BITMAP IMM NOOUT
|
|
host kaveri
|
|
disk nvme0n1p1
|
|
osd 3 down 100G 0 % abc,kaveri 1 128k 4k none -
|
|
osd 4 down 100G 0 % 1 128k 4k none -
|
|
disk nvme1n1p1
|
|
osd 5 down 100G 0 % abc,kaveri 1 128k 4k none -
|
|
osd 6 down 100G 0 % 1 128k 4k none -
|
|
host stump
|
|
osd 1 up 100G 37.29 % osdone 1 128k 4k all -
|
|
osd 2 up 100G 26.8 % abc 1 128k 4k all -
|
|
osd 7 up 100G 21.84 % 1 128k 4k all -
|
|
osd 8 up 100G 21.63 % 1 128k 4k all -
|
|
osd 9 up 100G 20.69 % 1 128k 4k all -
|
|
osd 10 up 100G 21.61 % 1 128k 4k all -
|
|
osd 11 up 100G 21.53 % 1 128k 4k all -
|
|
osd 12 up 100G 22.4 % 1 128k 4k all -
|
|
```
|
|
|
|
## ls-osd
|
|
|
|
`vitastor-cli osds|ls-osd|osd-ls [-l|--long]`
|
|
|
|
Show current OSDs as list, optionally with I/O statistics if -l is specified.
|
|
|
|
Example output:
|
|
|
|
```
|
|
OSD PARENT UP SIZE USED% TAGS WEIGHT BLOCK BITMAP IMM NOOUT
|
|
3 kaveri/nvme0n1p1 down 100G 0 % globl,kaveri 1 128k 4k none -
|
|
4 kaveri/nvme0n1p1 down 100G 0 % 1 128k 4k none -
|
|
5 kaveri/nvme1n1p1 down 100G 0 % globl,kaveri 1 128k 4k none -
|
|
6 kaveri/nvme1n1p1 down 100G 0 % 1 128k 4k none -
|
|
1 stump up 100G 37.29 % osdone 1 128k 4k all -
|
|
2 stump up 100G 26.8 % globl 1 128k 4k all -
|
|
7 stump up 100G 21.84 % 1 128k 4k all -
|
|
8 stump up 100G 21.63 % 1 128k 4k all -
|
|
9 stump up 100G 20.69 % 1 128k 4k all -
|
|
10 stump up 100G 21.61 % 1 128k 4k all -
|
|
11 stump up 100G 21.53 % 1 128k 4k all -
|
|
12 stump up 100G 22.4 % 1 128k 4k all -
|
|
```
|
|
|
|
## modify-osd
|
|
|
|
`vitastor-cli modify-osd [--tags tag1,tag2,...] [--reweight <number>] [--noout true/false] <osd_number>`
|
|
|
|
Set OSD reweight, tags or noout flag. See detail description in [OSD config documentation](../config/pool.en.md#osd-settings).
|
|
|
|
## pg-list
|
|
|
|
`vitastor-cli pg-list|pg-ls|list-pg|ls-pg|ls-pgs|pgs [OPTIONS] [state1+state2] [^state3] [...]`
|
|
|
|
List PGs with any of listed state filters (^ or ! in the beginning is negation). Options:
|
|
|
|
```
|
|
--pool <pool name or number> Only list PGs of the given pool.
|
|
--min <min pg number> Only list PGs with number >= min.
|
|
--max <max pg number> Only list PGs with number <= max.
|
|
--osd 1,2,... Only list PGs with some data on specified OSD(s).
|
|
```
|
|
|
|
Examples:
|
|
|
|
`vitastor-cli pg-list active+degraded`
|
|
|
|
`vitastor-cli pg-list ^active`
|
|
|
|
## create-pool
|
|
|
|
`vitastor-cli create-pool|pool-create <name> (-s <pg_size>|--ec <N>+<K>) -n <pg_count> [OPTIONS]`
|
|
|
|
Create a pool. Required parameters:
|
|
|
|
| <!-- --> | <!-- --> |
|
|
|--------------------------|-----------------------------------------------------------------------------------------|
|
|
| `-s R` or `--pg_size R` | Number of replicas for replicated pools |
|
|
| `--ec N+K` | Number of data (N) and parity (K) chunks for erasure-coded pools |
|
|
| `-n N` or `--pg_count N` | PG count for the new pool (start with 10*\<OSD count\>/pg_size rounded to a power of 2) |
|
|
|
|
Optional parameters:
|
|
|
|
| <!-- --> | <!-- --> |
|
|
|--------------------------------|----------------------------------------------------------------------------|
|
|
| `--pg_minsize <number>` | R or N+K minus number of failures to tolerate without downtime ([details](../config/pool.en.md#pg_minsize)) |
|
|
| `--failure_domain host` | Failure domain: host, osd or a level from placement_levels. Default: host |
|
|
| `--root_node <node>` | Put pool only on child OSDs of this placement tree node |
|
|
| `--osd_tags <tag>[,<tag>]...` | Put pool only on OSDs tagged with all specified tags |
|
|
| `--block_size 128k` | Put pool only on OSDs with this data block size |
|
|
| `--bitmap_granularity 4k` | Put pool only on OSDs with this logical sector size |
|
|
| `--immediate_commit none` | Put pool only on OSDs with this or larger immediate_commit (none < small < all) |
|
|
| `--level_placement <rules>` | Use additional failure domain rules (example: "dc=112233") |
|
|
| `--raw_placement <rules>` | Specify raw PG generation rules ([details](../config/pool.en.md#raw_placement)) |
|
|
| `--local_reads primary` | Local read policy for replicated pools: primary, nearest or random |
|
|
| `--primary_affinity_tags tags` | Prefer to put primary copies on OSDs with all specified tags |
|
|
| `--scrub_interval <time>` | Enable regular scrubbing for this pool. Format: number + unit s/m/h/d/M/y |
|
|
| `--used_for_app fs:<name>` | Mark pool as used for VitastorFS with metadata in image `<name>` |
|
|
| `--used_for_app s3:<name>` | Mark pool as used for S3 location with name `<name>` |
|
|
| `--pg_stripe_size <number>` | Increase object grouping stripe |
|
|
| `--max_osd_combinations 10000` | Maximum number of random combinations for LP solver input |
|
|
| `--wait` | Wait for the new pool to come online |
|
|
| `-f` or `--force` | Do not check that cluster has enough OSDs to create the pool |
|
|
|
|
See also [Pool configuration](../config/pool.en.md) for detailed parameter descriptions.
|
|
|
|
Examples:
|
|
|
|
`vitastor-cli create-pool test_x4 -s 4 -n 32`
|
|
|
|
`vitastor-cli create-pool test_ec42 --ec 4+2 -n 32`
|
|
|
|
## modify-pool
|
|
|
|
`vitastor-cli modify-pool|pool-modify <id|name> [--name <new_name>] [PARAMETERS...]`
|
|
|
|
Modify an existing pool. Modifiable parameters:
|
|
|
|
```
|
|
[-s|--pg_size <number>] [--pg_minsize <number>] [-n|--pg_count <count>]
|
|
[--failure_domain <level>] [--root_node <node>] [--osd_tags <tags>] [--no_inode_stats 0|1]
|
|
[--max_osd_combinations <number>] [--primary_affinity_tags <tags>] [--scrub_interval <time>]
|
|
```
|
|
|
|
Non-modifiable parameters (changing them WILL lead to data loss):
|
|
|
|
```
|
|
[--block_size <size>] [--bitmap_granularity <size>]
|
|
[--immediate_commit <all|small|none>] [--pg_stripe_size <size>]
|
|
```
|
|
|
|
These, however, can still be modified with -f|--force.
|
|
|
|
See [create-pool](#create-pool) for parameter descriptions.
|
|
|
|
Examples:
|
|
|
|
`vitastor-cli modify-pool pool_A --name pool_B`
|
|
|
|
`vitastor-cli modify-pool 2 --pg_size 4 -n 128`
|
|
|
|
## rm-pool
|
|
|
|
`vitastor-cli rm-pool|pool-rm [--force] <id|name>`
|
|
|
|
Remove a pool. Refuses to remove pools with images without `--force`.
|
|
|
|
## ls-pools
|
|
|
|
`vitastor-cli ls-pools|pool-ls|ls-pool|pools [-l] [--detail] [--sort FIELD] [-r] [-n N] [--stats] [<glob> ...]`
|
|
|
|
List pools (only matching <glob> patterns if passed).
|
|
|
|
| <!-- --> | <!-- --> |
|
|
|----------------------|-------------------------------------------------------|
|
|
| `-l` or `--long` | Also report I/O statistics |
|
|
| `--detail` | Use list format (not table), show all details |
|
|
| `--sort FIELD` | Sort by specified field (see fields in --json output) |
|
|
| `-r` or `--reverse` | Sort in descending order |
|
|
| `-n` or `--count N` | Only list first N items |
|