Capabilities#
This page summarizes the current source on main. The 0.1.0-alpha.1 crate on
crates.io is published for early development; its API is unstable. The source
is preparing 0.1.0-beta.1, which is not published by this change; see
release notes.
Status |
Meaning |
|---|---|
Accepted |
A bounded installed hardware profile passed and an independent reviewer accepted it. Only that recorded profile is covered |
Qualified |
The exact artifact passed its stated Linux CI or installed hardware profile; evidence does not extend beyond that profile |
Implemented |
Source and CPU controls pass; the current source has no installed hardware result |
Planned |
Not implemented yet; the stage is listed in the completion plan |
Qualification records the exact artifacts, cases and limits behind each CI and hardware result. Historical hardware acceptance below applies only to its recorded artifact; every current beta GPU/RDMA/GDS/MNNVL/CXL profile remains unqualified. The v0.3.14 delta records the source alignment. The design-reference map identifies the corresponding owners.
Transports#
Path |
Status |
Limits |
|---|---|---|
CPU DRAM over TCP |
Qualified on Linux CI |
Exact CI jobs and tests: loopback and multi-process byte, restart and lifetime controls, including bounded epoch-scoped TCP reuse and dirty-stream retirement; no physical two-host profile. See CPU and TCP |
Host DRAM over RDMA |
Accepted (RoCE) |
Two-host RoCEv2 READ/WRITE with restart, cleanup and in-flight DMA faults. InfiniBand and physical link or device loss are unqualified. See RDMA and CUDA |
GPUDirect RDMA |
Accepted (H20/RoCE, with limits) |
Host/GPU READ/WRITE and two-rail slices. The H20 evidence covers private GPU source grants on peermem. The current dma-buf/peermem registration policy and route evidence are implemented; both installed route profiles are unqualified. See registration routes. Opt-in GPU source QP/MR reuse is implemented; installed profile unqualified. Host-wide context non-growth is unqualified |
CUDA IPC / NVLink on one host |
Accepted (H20, cooperative profiles) |
GPU/GPU, pinned host/GPU, subbatch reuse and a fixed-working-set import cache. Grouped completion, stream pool, native batch copy, cache reclamation and the single-task synchronous path await installed verification. See CUDA IPC |
Local CUDA staging over TCP/RDMA |
Accepted (H20, healthy) |
Shared pinned chunks; physical CUDA faults are unqualified. See staging |
Remote CUDA staging |
Accepted (H20, serial and pipeline) |
Pin/Delegate/Unpin with up to 16 slots. Source control isolation and later nonblocking progress are implemented; their installed profiles are unqualified. See remote pipeline |
Local CUDA ↔ pinned DRAM on a caller stream |
Implemented; installed profile unqualified |
Coalesced 1D/2D and batched DMA, intent-aware Auto, capped mapped-host kernel and event-owned range loans. CTA/threshold calibration and compute interference await PCIe Hopper and Grace C2C runs. Local copies |
Mixed GPU Direct and staged batches |
Implemented; installed profile unqualified |
GPU Direct and local or remote staged legs keep public order. Compatible aliases within a leg share one dispatch and release receipt. See batch contract |
DRAM ↔ local file (io_uring) |
Qualified on Linux CI |
Direct ext4 and buffered tmpfs I/O, ordered host gather/scatter and padding; eligible small requests use a deadline-capped caller wait before worker drain. A lost ready-job worker retains native owners and charges as |
CUDA ↔ local file (io_uring) |
Implemented; installed profile unqualified |
CUDA shim and real files exercise multi-registration gather/scatter, bounded proxy chunks and optional pinned slots. See file staging |
GDS / cuFile |
Implemented, installed native gate unrun |
Lazy, runtime-disablable cuFile, aligned vector I/O, terminal cookies and quarantine. See GDS |
Qualified on Linux loopback |
Least-loaded single-operation lanes, striped READs and joined-session close controls on loopback IPv4 rails. Physical rail gate unrun |
|
Qualified on same-host Linux CI |
Guarded two-process READ/WRITE over |
|
Implemented; installed profile unqualified |
CUDA VMM fabric handles, GPU and host-NUMA exports, bounded imports and checked close. The multi-node device and EGM gates are unrun |
|
MPComm |
Deferred (P8.1) |
No consumer; provider unavailable |
Memory and batches#
Capability |
Status |
Limits |
|---|---|---|
Request construction |
Implemented |
|
Owned and external host registration |
Implemented |
Transactional batches, duplicate references, aliases and retained backing |
Local file segments |
Implemented |
Core |
Async batches and notifications |
Implemented |
Explicit progress, event-driven batch wakes, deferred free, bounded TCP/RPC notifications and per-target delivery. See async batches |
NVIDIA allocations, pinned host memory and copy streams |
Qualified (H20 local copy) |
Stream and event reuse; failure retention has CPU controls only |
GPU and pinned-host producer readiness |
Implemented |
Caller-stream capture before payload access, with a CPU-blocking wait. See pinned handoff |
Memory type and location |
Implemented |
Owner-backed CPU/CUDA locations; unknown pointers never select CPU access. See memory location |
Host staging location |
Implemented |
GPU-near or explicit NUMA staging buffers; residency and performance are unqualified. See staging location |
Routing and scheduling#
Capability |
Status |
Limits |
|---|---|---|
Transport selection |
Implemented |
First-match and named rules consume intent, boot identity, priority and size. RDMA dispatch honors per-request NIC masks and endpoint QP pools. Physical routing awaits the installed two-RNIC gate. See transport selection |
Discovery, port recovery and multirail slicing |
Accepted (RoCE) |
Automatic discovery matches a separate host inventory; two-rail accounting verifies exact bytes. Physical loss is unqualified |
RDMA rail health |
Implemented; installed profile unqualified |
Policy-masked slices refresh local paths and prefer a healthy mapped peer for wildcard targets. A real link-fault and two-RNIC gate is queued. See rail health |
CUDA topology, custom priority matrices, bandwidth-aware rails |
Implemented |
Installed routing and throughput are unqualified. See bandwidth-aware selection |
Request priority and NIC arbitration |
Implemented |
RDMA orders High/Medium/Low with aging and opt-in deadline urgency; whole-worker snapshot and late-registration controls use CPU shims, not installed RNICs. TCP and NVLink treat priority as advisory. See priority |
RDMA selective signaling and HCA READ depth |
Implemented; installed profile unqualified |
Per-post signal groups, QP depth clamps and exact flush accounting. See RDMA ownership |
Adaptive congestion core |
Implemented, CPU controls |
Opt-in callable device and route domains with off/observe/enforce. TENT and OrbiTE workers leave live policy to the caller. |
RDMA quota accounting |
Implemented, installed gate unrun |
Per-NIC smart inflight bytes and exact slice charges. A POSIX shared-memory name enables host-wide priority slots. See shared slots |
Runtime FIFO/EDF admission |
Implemented |
Merged memory requests, public alias status, physical dispatch windows, staging reserves and transport-aware deadline drops. See dispatch window |
QoS contracts |
Implemented |
Global, tenant and intent fields resolve to one local byte/request budget. Delay and reject execute. Unsupported actions return a field-specific error. See QoS |
Receiver credits |
Implemented |
Four named cumulative grants, configured receiver windows and drain-fenced epoch retirement. See credits |
Transport failover |
Implemented |
Bounded physical retries, epoch-pinned Describe and release evidence. Installed RNIC profile unqualified. See transport attempts |
Offline version diagnostics |
Implemented |
Packaged CLI and Rust API. See diagnostics. |
OrbiTE metrics, request/attempt accounting and exporter health |
Implemented (P6, opt-in) |
OrbiTE names, HELP text and reporter logs; unchanged types, labels and buckets. Metrics |
Failure handling#
CPU benchmark and fault fixtures cover verified size/batch sweeps and the software backend lifetime matrix. They add no hardware qualification; GPU/RDMA/GDS installed fault fixtures remain queued.
Capability |
Status |
Limits |
|---|---|---|
Native failure retention |
Accepted (RoCE fault profiles) |
QP errors, lost CQEs, QP/MR/MW faults and CQ/channel/PD cleanup failures keep their owners and charges. Device-close errors have CPU controls only. See native cleanup |
APIs and operations#
Capability |
Status |
Limits |
|---|---|---|
Rust API |
Current beta surface inventoried; breaking changes allowed before 0.1.0 |
|
NIXL backend and native bridge |
Planned; design only |
South Bound API and owner mapping. No plugin or C bridge is implemented or qualified. The implementation prerequisites include upstream explicit cancel/lifetime agreement and current-artifact hardware evidence before the formal RFC. |
TENT C ABI (S19) |
Removed; out of scope |
No installed C interface |
TENT C++ and classic TE (S20) |
Removed; out of scope |
No source adapters or classic-TE shim |
Device plugin ABI (S22/F03) |
Removed; out of scope |
CPU and CUDA use built-in owners |
OrbiTE Python API |
Qualified on Linux CI (CPU/TCP only) |
Exact CI evidence. Installed wheels for CPython 3.10–3.14, Linux x86_64/aarch64. Guarded TCP, GIL release and retained host/CAI/DLPack exporters; native GPU/RDMA Python profiles remain unqualified |
OrbiTE configuration and P2P self-description |
Qualified on Linux CI (configuration and CPU handshake) |
Exact jobs and cases. Strict JSON, file or |
TENT wire interoperability |
Out of scope by design (S23) |
See the wire and consumer boundary |
Provider selection#
Explicit CPU/TCP routing works without configuration. Transport::Unspec uses
eligible transport selection. rdma and cuda are
independent Cargo features. Enabling a feature makes its provider code
available; runtime use still requires the actual device, driver and resources. A combined-feature wheel does not link libibverbs; the configured RDMA install loads the host runtime and reports a named unavailable error if it is absent. See build isolation.
RdmaConfig::new("mlx5_0") strictly pins that device, port 1 and GID index 0.
RdmaConfig::default() enables automatic discovery with optional strict pins
and whitelist/blacklist filters. device_mask limits dispatch by stable NIC
IDs. See RDMA configuration
for selection and resource ownership.
Scope of support#
The target is CPU DRAM, NVIDIA GPUs and local SSDs: OrbitKV hands every
end-to-end cache transfer, including GPU↔DRAM and GPU↔SSD, to OrbiTE. Storage
transports use the opt-in Linux io-uring and gds features. A network/memory-only
build loads no storage libraries. Other accelerators are outside the selected
scope. Architectural correspondence with frozen TENT does not establish full
behavior or wire compatibility.
Hardware gates are paused while no GPU or RDMA machine is available. New work is implemented with CPU controls, and its installed gates wait in the qualification verification queue. OrbitKV integration resumes in P11.