Implement RDMA receive with memory copying (send remains zero-copy)
This is the simplest and, as usual, the best implementation :) 100% zero-copy implementation is also possible (see rdma-zerocopy branch), but it requires to create A LOT of queues (~128 per client) to use QPN as a 'tag' because of the lack of receive tags and the server may simply run out of queues. Hardware limit is 262144 on Mellanox ConnectX-4 which amounts to only 2048 'connections' per host. And even with that amount of queues it's still less optimal than the non-zerocopy one. In fact, newest hardware like Mellanox ConnectX-5 does have Tag Matching support, but it's still unsuitable for us because it doesn't support scatter/gather (tm_caps.max_sge=1).
This commit is contained in:
@@ -43,6 +43,7 @@ breaking changes in the future. However, the following is implemented:
|
||||
- Inode metadata storage in etcd
|
||||
- Snapshots and copy-on-write image clones
|
||||
- Write throttling to smooth random write workloads in SSD+HDD configurations
|
||||
- RDMA/RoCEv2 support via libibverbs
|
||||
|
||||
## Roadmap
|
||||
|
||||
@@ -54,7 +55,7 @@ breaking changes in the future. However, the following is implemented:
|
||||
- Scrubbing without checksums (verification of replicas)
|
||||
- Checksums
|
||||
- Tiered storage
|
||||
- RDMA and NVDIMM support
|
||||
- NVDIMM support
|
||||
- Web GUI
|
||||
- Compression (possibly)
|
||||
- Read caching using system page cache (possibly)
|
||||
|
||||
Reference in New Issue
Block a user