Capabilities#
This matrix describes the current source. The published 0.1.0-alpha.1 crate
remains the original package reservation. Implemented, qualified and
independently accepted are separate states; see Qualification
for the exact artifacts and profiles behind each result.
Path or capability |
Current source |
Qualification boundary |
|---|---|---|
CPU DRAM over TCP |
Implemented |
Loopback and independent-process byte, restart and lifetime controls |
Owned/external host registration |
Implemented |
Transactional batches, duplicate references, allocation aliases and retained backing |
Async batches and notifications |
Implemented |
CPU cancellation, detached handles, bounded admission and receipt controls |
Host DRAM over RDMA |
Implemented |
Scoped two-host IPv4/IPv6 RoCEv2 READ/WRITE profiles; InfiniBand remains unqualified |
Persistent host MRs and multi-context publication |
Implemented |
Scoped two-RNIC atomic publication, rollback and retirement; paused-context MR preparation and recovery pass the focused native PD/MR/QP gate |
Reusable QP arrays and concurrent grants |
Implemented |
Fixed-width cold construction, public v19 multiplexing, exact WR/receipt dispatch and shared physical SQ/RQ limits |
Multirail slices and shared quotas |
Implemented |
Selected folded-tail and concurrent-parent profiles verify bytes, whole-parent admission and native drain. The installed two-physical-rail profile joins every payload WR/CQE and verifies shared-QP contention/accounting; bandwidth/fairness policy remains P6 |
Transport workers, CQ lanes and NUMA binding |
Implemented |
Scoped native worker/lane routing, CPU-mask and polling/event profiles; broader scheduling policy remains later work |
Discovery and periodic recovery |
Implemented |
Scoped process-local query-failure recovery; initially paused contexts and automatic/strict GID recovery pass the focused one-RNIC native gate. Installed two-host automatic discovery matches a separate sysfs inventory, including strict-pin rejection and NUMA ranks. Independent review includes four fresh live RoCE repeats; physical loss remains open |
Native failure retention and live reconnect |
Implemented |
Scoped QP ERR, lost CQEs, QP/MR/MW syscall faults and CQ/channel/PD EBUSY. Failed owners remain retained; device-close errors have CPU ABI coverage only |
Packaged public host consumer |
Scoped tests |
Qualified profiles cover healthy traffic, restarts, lost CQEs, actual in-flight DMA interruption, QP/MR/MW/CQ/channel/PD cleanup, engine replacement, physical rail accounting and automatic discovery. Physical link/device loss and InfiniBand remain open |
NVIDIA allocation, pinned host memory and copy streams/events |
Implemented |
Serialized stream/event reuse passes the bounded two-host/four-H20 local-copy profile on stable and Rust 1.85. Failure retention has CPU controls. Staging is the default device policy |
Direct GPU registration / GPUDirect RDMA |
Implemented; bounded H20/RoCE profile accepted with limits |
Both installed toolchains pass host/GPU/GPU READ/WRITE, tiny and folded two-rail slices, real GPU MR/WR witnesses and owned-context drain. B host-wide context non-growth remains unqualified. Explicit Direct requires scatter-to-CQE disabled; GPU source MRs/QPs are per-grant |
Cooperative CUDA IPC / NVLink |
Bounded GPU/GPU, subbatch and pinned host profiles accepted |
Owned GPU/GPU and local owned pinned host/GPU; whole-allocation opt-in and quota, event completion and explicit import-close receipts. The initial path passes 48 installed H20/NV18 cases plus four independent repeats on stable/MSRV. Same-context subbatch reuse passes 28 additional installed cases and four independent repeats. Pinned host and mixed groups pass 30 additional installed cases plus four independent repeats; exporter-crash safety and broader TENT IPC parity remain open |
Cross-submission CUDA IPC import cache |
Bounded fixed-working-set profile accepted |
Explicit bounded cache, fresh source authorization per submission, single-flight imports and separate exporter lifetime. Fixed working set until requester shutdown; no eviction or forced exporter-crash safety |
Local CUDA staging chunks |
Bounded healthy profile accepted; P5.5a |
Shared TCP/RDMA chunk owner; 24 installed H20 cases and four independent repeats verify bounded pins, copies and frees. CPU controls cover uncertain completion/cleanup retention. Opt-in serial remote sessions are implemented separately; see staging |
Remote CUDA staging |
Healthy serial and bounded pipeline profiles accepted |
Serial Pin/Delegate/Unpin passes 16 installed cases plus four independent repeats. The up-to-16-slot pipeline passes 24 fresh installed H20 TCP/RDMA cases plus four independent repeats. Native stream/event lifetimes, per-slot reuse and private MR cleanup are checked. Requester chunk-size negotiation and GPU range loans pass independent source review. Independent requester PRE/POST rings and failure ownership pass source review at |
Local SSD through io_uring / cuFile GDS |
Planned, optional |
Separate storage transports and direct-path qualification |
TENT C/Python/C++ API and mixed-peer wire compatibility |
Open |
Separate API/ABI/wire gates; the current protocol is OrbitTE-only |
Provider selection#
CPU/TCP is the default. rdma and cuda are independent Cargo features.
Enabling a feature makes its provider code available; runtime use requires the
actual device, driver and resources.
RdmaConfig::new("mlx5_0") strictly pins that device, port 1 and GID index 0.
RdmaConfig::default() enables automatic discovery with optional strict pins
and whitelist/blacklist filters. device_mask limits dispatch by stable NIC
IDs. See RDMA configuration
for selection and resource ownership.
Scope of support#
The target is CPU DRAM and NVIDIA GPUs; local file/SSD movement is optional. Other accelerators are outside the selected scope. Architectural correspondence with frozen TENT does not establish full behavior or wire compatibility.
The P4 exit queue contains the remaining host-RDMA physical/fabric gates. Source and the bounded RoCE profile are independently accepted with fresh live repeats; P5 proceeds with available hardware. Complete the host-RDMA acceptance boundary before renewing OrbitKV integration; the other parity rows remain in their own stages.
Optional nvlink_cache_entries adds transport-owned cross-submission CUDA IPC
reuse. Source/CPU controls, 34 installed H20 cases and four independent repeats
pass for the declared fixed-working-set profile.
This fixed working-set profile keeps the whole source allocation loan until
requester shutdown. See CUDA IPC for lifecycle and open limits.