Rust API Guide

Rust API Guide#

The library package is orbit-te; the Rust crate is orbit_te. This guide describes the current source API. Generate its full type documentation with:

cargo doc --locked --no-deps --open

Add --features rdma,cuda when documenting native providers in an environment with the build prerequisites.

Engine and registration#

TransferEngine::new(EngineConfig) starts a source control service and owns requester/source admission. Its default bind address is loopback. Configuration and budgets are immutable for that engine instance.

API

Contract

register_memory

Transfer ownership of host bytes into a registered range

register_host_memory

Register a range of an existing Arc<HostMemory> while retaining its real allocation owner

register_host_memory_batch

Preflight a set of host ranges before publishing new registration references

unregister_memory

Retire a registration reference; admitted users retain their actual backing

resolve_local_address

Resolve a contained registered address range without adopting an arbitrary pointer

Permission distinguishes local-only, remote READ and remote READ/WRITE authority. Repeated registration of the same range keeps its original permission. CPU access coordination is shared across aliases.

For a configured RDMA engine with host Direct policy, host registration prepares every usable context’s MR copies before lookup/descriptor visibility and before duplicate reference increments commit. This includes paused ports: their PDs retain memory registration, while transfer admission waits for recovery. A failed batch retires only its newly created copies. Previously published ranges remain valid; failed native cleanup retains unpublished backing and its charges. This can fail at registration time when native resources or aggregate MR capacity are unavailable. See the publication contract.

Peer authority#

let peer = requester.open_segment(
    source.endpoint(),
    source.source_epoch(),
    Duration::from_secs(1),
)?;

The expected SourceEpoch must come from the authoritative source owner. Descriptors identify registered regions but do not replace that authority. Restarting a process at the same endpoint does not authorize an old peer handle to use the new process’s allocations.

Requests and reports#

TransferRequest names the transport, local address, remote address and length. Both ranges must be contained in one applicable registration. TransferOp selects READ or WRITE. Submission resolves and retains each backing allocation before transfer effects.

submit_and_wait returns BatchReport with completed request/byte counts, failure information, notification status and local drain status. The completed prefix is not a transaction. RemoteEffect::Unknown means an attempted WRITE may have changed peer bytes; a failed report does not authorize automatic replay. TransferFailure.affected_requests is the number of consecutive tasks sharing the failure from request_index. Shared NVLink import close/ACK failure excludes the entire attempted group from the completed prefix; async tasks in this span are failed, not skipped. Only later requests were never attempted. For a cross-rail request, all slice receipts must complete before its one request and full byte count are added. Earlier slices may already have changed bytes if a later slice fails, including source-staged WRITE slices copied back after their own completion. Completion is not an atomic multi-rail commit. Remote staging READ starts each local POST after its scoped CROSS receipt. Independent slot copies may complete out of order. A later failure, including lost Unpin, leaves changed bytes visible but counts the failed request as zero completed requests/bytes; native uncertainty does not imply an exact prefix. The CUDA requester uses a bounded ring; WRITE does not snapshot the entire request. Keep input bytes stable while a request is in flight. GPU staging range loans reject conflicting access, including overlapping self-peer copies.

For asynchronous ownership, see Batch: submit groups, seal, inspect or wait for completion, and retain the handle until resource ownership is understood. Dropping a handle cannot free live native work.

Native providers#

transport::rdma::RdmaConfig selects discovery/pins, host direct or staged memory, progress mode and bounded native resource budgets. RdmaProgressMode::Polling is the default; EventDriven consumes native CQ events while preserving the same data worker and completion validation.

The cuda module owns NVIDIA device/pinned-host allocations and synchronized copies. Device pointers use declared host staging for network movement. See RDMA and CUDA for exact range, grant and drain requirements.

Close and drain#

close_source fences source admission; close_requester fences requester admission. Whole-engine close fences both, drains accepted work and closes shared native resources. A timeout does not complete pending native work. Error::Unquiesced retains the real resources and their charges after physical cleanup could not be proved.

The runnable TCP example, external allocation example and async example use the public source API.

Contents