Rust API Guide

The package is orbite and the crate is orbite. Generate the full type documentation with:

cargo doc --locked --no-deps --open

Add --features rdma,cuda in an environment with the build prerequisites. The docs.rs profile includes CUDA, io_uring, GDS and metrics on Linux through runtime-loaded SDKs; it deliberately excludes rdma/rdma-adaptive, whose generated bindings need system verbs headers. CI builds that profile with --cfg docsrs and no verbs headers. Use the full local feature build for RDMA rustdoc.

Current public surface

This is the current public surface for 0.1.0-beta.1. This is an inventory, not a freeze: breaking changes are allowed before 0.1.0, including between betas. Hardware qualification is separate. Rust is the primary API. The following names and their public methods/fields are inventoried here; private transport owners, native function tables and test shims are not public APIs. Full signatures come from rustdoc for the chosen feature set. TENT compatibility surfaces are out of scope.

Area	Public surface
Construction	TransferEngine, EngineConfig, RuntimeQueueConfig; typed new, explicit from_env/json/file, config and stats queries
Requests and outcomes	TransferRequest, TransferOp, Transport, Priority, FileIoElement, BatchReport, TransferFailure, RemoteEffect, Error, Result
Registered backing	HostMemory, MemoryBacking, RegisteredMemory, Permission, MemoryType, MemoryLocation, RangeLocation; owned/external and range-batch registration, address resolution, unregister, guarded reads/writes
Peer and file authority	SourceEpoch, RemoteSegment, RemoteRegion, GpuDirectRoutes, TransferTarget, FileSegment, FileSegmentInfo; endpoint/epoch discovery, open/refresh/close, file offset resolution and info
Async work	Batch, BatchId, TaskState, TaskStatus, BatchState, BatchStatus, SubmissionStatus; allocate, append/groups, seal, observe/progress, cancel, wait, drain and free; next_poll_delay, wait_before_next_poll
Notifications	Notification, NotificationEffect, NotificationFailure, NotificationStatus; attached, local and remote send, receive and delivery status
Selection	IntentType, PolicyName, SegmentType, SelectionPolicy, TransportSelectorConfig, QpPool; compiled policies and named-policy resolution
QoS	QosConfig, QosPolicyFields, QosDegradedActions, QosTenantContract, QosRequestContext, QosUsage, EffectiveQosPolicy; resolve and sample contracts
Credits and accounting	CreditResource, CreditBalance, CreditAmount, CreditUpdateDisposition, ReceiverCreditUpdateV1, ReceiverCreditConfig, ReceiverCreditStats, ReceiverCreditIssuerStats, EngineStats, RuntimeQueueStats; manual and automatic credit lifecycle
CPU providers	HpTcpConfig, ShmConfig, MnnvlConfig, MnnvlHandleType; SHM allocation/free (MNNVL settings exist without an active CUDA provider)
Linux topology	Topology, NicEntry, NicType, MemEntry, TopologyMemoryKind, DEVICE_PRIORITY_RANKS; local/discovered topology snapshots
rdma	transport::rdma::{RdmaConfig, GpuDirectRegistration, RdmaHostMemory, RdmaCudaMemory, RdmaProgressMode, SchedulingParams, NicLoadStats, QpReadDepths, RdmaStats}; NIC load queries
Unstable rdma-adaptive (outside the stability promise)	transport::rdma::adaptive_congestion_control: Config, Mode, PathState, Snapshot, Signals, DomainState, AdmissionDomain, Decision, Reason, Rejection, PathHandle, Permit, Classification, FailureScope, Outcome, classify_completion, classify_async_event; hidden callable core, not a live worker policy
cuda	cuda::{CudaMemory, RetainedResources, allocate_pinned_host, retained_resources}, root StagingLocation, StagingStats, NvlinkStats; device/fabric/EGM allocation, external ownership, producer readiness, staging trim and IPC cache flush
cuda::local_copy	PinnedHostRegistration, CopyDirection, LocalCopyBackend, LocalCopyConfig, LocalCopyOptions, LocalCopyDesc, ProducerEvent, LocalCopyEngine, LocalCopyStats, LocalCopyClassStats, LocalCopyFallback, LocalCopyCompletion; caller stream, intent/priority, CTA cap, descriptors, submit, event progress/wait and close
Storage features	io-uring: IoUringConfig, IoUringStats; gds: GdsMode, GdsStats
metrics	metrics::{MetricsConfig, Metrics}; engine metrics/status, enable toggle, JSON/Prometheus snapshots and writers
Offline diagnostics	diagnostics::{BUILD_VERSION, BUILD_COMMIT, SCHEMA_VERSION, Outcome, Check, BuildInfo, Snapshot, version_snapshot}; text/JSON rendering and exit code; orbite diagnostics version CLI

API decisions

All public enums and every public struct with public fields are #[non_exhaustive]. TransferOp is the sole exhaustive enum: its binary READ/WRITE contract is intentional. CI enforces Clippy’s exhaustive_structs and exhaustive_enums lints with that single exception. Opaque structs with private fields already require their constructors. Match enums with a wildcard and destructure field records with ..; construct configuration through defaults and field assignment:

use orbite::{EngineConfig, RuntimeQueueConfig};

let mut queue = RuntimeQueueConfig::default();
queue.max_dispatch_owners = 8;
let mut config = EngineConfig::default();
config.runtime_queue = Some(queue);

Notification::new, NicEntry::new, MemEntry::new, CreditAmount::new and ReceiverCreditUpdateV1::new construct input records without struct literals. Their public fields remain mutable. QP pools use QpPool::default() followed by a unique name and any count/link overrides.

Boundary	API decision
Primary Rust helpers	transport_installed, register_memory_ranges, Batch::observe_tasks, resolve_policy_name and discover_peer_epoch are documented API. Installation is not availability on a peer; observational status is not a drain receipt; discovery never replaces consumer epoch authority.
Binding internals	Backing release_owned, registration exclusively_held and shm_backing_name are #[doc(hidden)], outside the stability promise. The Python packaging crate pins the core to the exact same release because it uses these helpers.
Names	NvlinkStats matches Transport::Nvlink; MemoryBacking is the actual retained owner enum. TopologyMemoryKind describes topology entries and is distinct from registration MemoryType. Public TENT settings and transport-ID conversions are removed. ReceiverCreditUpdateV1 names OrbiTE’s credit-update schema.
Adaptive core	The callable rdma-adaptive module is #[doc(hidden)], unstable and outside the stability promise. It has no engine integration or consumer. Crate-private visibility creates dead-code/unused-import warnings; hiding the reference core preserves its tests without suppressing those lints.
Documentation	Missing docs are not a beta publication blocker: adding documentation is non-breaking. The release-readiness audit found 668 all-feature Linux diagnostics. Before 0.1.0, close the remaining findings and promote the CI audit to -D missing-docs, as required by the release ladder. Ownership, receipts, epochs and units are documented at their affected entry points.
Native binding types	The deliberate cudarc stream coupling and raw owner-backed alternative below remain. Signatures may change between betas; installed GPU profiles remain unqualified.

Transfer requests

TransferRequest::new(transport, local_address, remote_address, length) starts with Priority::High, the engine’s default QoS contract and remote staging off. Override them with .with_qos(policy), .with_priority(priority) and .with_remote_staging(true); an explicit QoS policy supplies its own priority. Use Transport::Unspec for transport selection. Public enums other than TransferOp are non-exhaustive, so matches outside the crate need a wildcard arm.

Both ranges must lie inside one applicable registration, and TransferOp selects READ or WRITE. Submission resolves and retains every backing allocation before any transfer effect. Priority orders RDMA payloads on each worker; TCP and NVLink accept every priority as advisory, and requests within a batch keep their order.

Engine and registration

TransferEngine::new(EngineConfig) starts a source control service (loopback by default) and owns requester and source admission. Configuration is immutable per engine. engine.config() borrows its effective settings.

Transport::Fabric and the provider config/stats exports are removed. All remaining Rust discriminants are unchanged; Transport stays non-exhaustive. CUDA fabric allocations, MNNVL and EGM are unaffected.

Configuration loading

use orbite::{EngineConfig, TransferEngine};

fn open_engine() -> orbite::Result<TransferEngine> {
    TransferEngine::new(EngineConfig::from_env()?)
}

EngineConfig::from_json(text) and from_file(path) take explicit input and ignore the environment. from_env() reads only ORBITE_CONF, containing either inline JSON or a regular UTF-8 file path; absent means typed defaults. Engine construction never rereads it. Reuse a snapshot after changing or deleting its input file. No loader mutates the process environment.

The root uses the typed EngineConfig field names. Transport fields may instead be grouped under transports; a field cannot appear in both places. Provider records use their typed field names and defaults. An optional provider object opts in; null disables it. hp_tcp.enable and shm.enable remain explicit. The RDMA record adds gpu_direct_registration: "auto" (default), "dmabuf" or "peermem"; it is compiled once and affects CUDA Direct registrations only. See route selection and lifetime. Duration fields have an _us suffix and take integer microseconds; fields already naming a unit, such as connect_timeout_ms, retain that unit. Other sizes are bytes except explicitly named KiB settings. MNNVL fabric_domain uses the 32-hex-digit SourceEpoch representation or null. Selector rules and QoS contracts live under selector and qos; receiver budgets under receiver_credit.

{
  "bind_addr": "127.0.0.1:0",
  "handler_timeout_us": 5000000,
  "transports": {"tcp": true, "shm": {"enable": false}},
  "selector": {"policy": [{"name": "host", "segment_type": "memory", "transports": ["tcp"]}]}
}

Unknown keys, duplicate keys, wrong types, malformed roots and out-of-range values fail before an engine starts. Documents are bounded to 1 MiB and files must be regular files. Removed metadata keys and legacy aliases are unknown fields. Legacy environment variables have no effect; provider-runtime variables such as CUDA_VISIBLE_DEVICES remain the provider’s responsibility. Native fields require their build feature. Typed Rust construction stays the primary interface.

API	Contract
register_memory	Move host bytes into a registered range
register_host_memory	Register a range of an existing Arc<HostMemory>, retaining its allocation owner
register_host_memory_batch	Preflight a set of host ranges, then publish them together
register_cuda_memory_batch	Register CUDA allocations (cuda feature)
unregister_memory	Drop a registration reference; admitted users keep their backing
resolve_local_address	Resolve a contained registered range without adopting a raw pointer

Permission distinguishes local-only, remote READ, remote WRITE and remote READ/WRITE; repeated registration of a range keeps its first permission. With RDMA host Direct policy, registration also prepares MRs on every usable context before the range becomes visible, so it can fail when native resources are exhausted; see the publication contract.

Admission and QoS configuration

Setting	Purpose	Guide
EngineConfig::runtime_queue	FIFO/EDF dispatch window for all data submissions	Dispatch window
EngineConfig::qos, QosConfig::from_json(_file)	Global/tenant/intent contracts; resolve_qos returns a reusable EffectiveQosPolicy	QoS, actions
EngineConfig::max_receiver_credit_sessions	Sender credit gating; activate/apply/deactivate_receiver_credit for manual grants	Credits
EngineConfig::receiver_credit	Receiver-side issuer; connect/disconnect_receiver_credit for automatic windows	Automatic windows
EngineConfig::selector, TransportSelectorConfig::from_json	Ordered transport selection rules (selector.policy)	Selection

EffectiveQosPolicy::fields(), effective_priority() and usage() explain a resolved policy and sample its budget.

Memory type and location

RegisteredMemory::memory_type() returns MemoryType::Cpu or Cuda from the retained backing, without locks or driver calls. RegisteredMemory::location() returns RangeLocation { address, length, location } clipped to that registration, and HostMemory::location(offset, length) / CudaMemory::location(offset, length) check the owner subrange first. These queries neither wait for producer readiness nor grant payload access.

MemoryLocation::Cuda { device, context } displays as cuda:N; the context number is diagnostic, not a usable handle. MemoryLocation::Cpu(Some(N)) displays cpu:N and Cpu(None) displays *. Ordinary, external and portable pinned host memory have unknown NUMA placement, and pinned DRAM stays CPU memory.

External CUDA adoption makes one cuPointerGetAttributes call (type, managed status, context, device), resolves the device with cuDeviceGet and checks the whole range with cuMemGetAddressRange in that context. Unknown attributes, managed or host pointers and native errors fail adoption and never select CPU access. This covers the owner-backed part of TENT’s Platform getMemoryType, getPointerDeviceId and getLocation; there is no general pointer classifier, CPU page-residency scan or device-plugin ABI. See qualification.

RDMA bandwidth and load

With rdma, RdmaConfig::scheduling configures SchedulingParams for bandwidth-aware selection (on by default; smart_selection = false uses round-robin in the first tier); see selection. TransferEngine::nic_load_stats() returns initialized NICs without discovery: rate estimates in bytes per second, link_speed_gbps in Gbit/s, inflight_bytes (grant admission), posted_bytes (native payload debt) and cumulative completed bytes, busy time and sample counts. Priority settings (deadline_bw_arbitration, priority_promotion_timeout, priority_promotion_per_entry, shared_quota_shm_path, slot_rotation_interval) are described in request priority.

Peer authority and file segments

let peer = requester.open_segment(
    source.endpoint(),
    source.source_epoch(),
    Duration::from_secs(1),
)?;

The expected SourceEpoch must come from the authoritative source owner; descriptors identify regions but never replace that authority, so a restarted process at the same endpoint cannot be reached through an old handle.

Discovery boundary

OrbiTE transfers bytes between known peers. The consumer owns discovery, membership, KV indexes and external metadata stores. Publish the source’s endpoint() and source_epoch() in your own registry, then pass that authoritative epoch to open_segment(endpoint, expected_epoch, timeout). For example, OrbitKV maintains its own etcd membership and passes transfer_endpoint and native_source_epoch over its own gRPC. OrbiTE’s P2P handshake supplies descriptors, capabilities and registration keys for that identity; it does not discover cluster members.

There are no metadata-server configuration keys. Unknown keys fail with Error::Config; no metastore URI or credential is resolved. The same immutable loader serves Rust and Python.

When membership says a peer left, call engine.close_segment(endpoint, expected_epoch, timeout). It immediately fences new requester admissions to that identity, drains already admitted transfers and notifications, then retires its transport cache entries. Existing RemoteSegment snapshots return Error::PeerClosed on submit or refresh. Other peers remain usable. Obtain a replacement epoch from its authoritative owner and call open_segment again; never edit an old descriptor’s epoch. This is outgoing retirement, not an authenticated ban on inbound clients.

A drain timeout returns Unquiesced and retains backing and budget charges. You may wait again while admitted work drains. Once native cleanup starts, its result is sticky: an uncertain destroy is never repeated. The timeout bounds waits, not a native call that cannot be preempted. Automatic receiver windows close with a receiver receipt; unresolved credit charges remain held. EngineConfig::max_peer_segments bounds simultaneously retained endpoint/epoch identities (default 1024, maximum 4096), not cumulative peer incarnations. Every RemoteSegment clone, admitted operation, alias, retry, notification and cleanup keeps its identity charged. Once the last reference is gone, the next table lookup or stats snapshot reclaims an open entry, or a retired entry whose cleanup succeeded. A timed-out drain still owes cleanup; failed native cleanup never becomes reclaimable. Capacity pressure returns ResourceExhausted.

After a retired identity is reclaimed, the consumer may explicitly call open_segment with the same epoch from its authoritative owner. Describe still rejects a wrong epoch. A descriptor belongs to the requester that opened it; using it with another engine returns InvalidConfig. Open it on that engine instead. This prevents a borrowed snapshot from recreating admission implicitly.

EngineStats::peer_segments counts all charged identities; live_peer_segments and retired_peer_segments partition that count by admission state. unquiesced_peer_segments counts the retired subset with a timed-out drain or failed cleanup. reclaimed_peer_segments is the cumulative reclaimed-entry count, and active_peer_operations counts outstanding admission permits. These counters are observations, not release receipts. Dropping a descriptor or calling the Python RemoteSegment.close() only closes that handle; reopening the same live source. The stronger Rust operation deliberately remains separate.

For a local file, open_file_segment takes a file:// name and a permission; file_segment_info returns the cached descriptor, refresh_file_segment re-reads the size from the same fd, resolve_file_offset checks a range, and close_file_segment closes every alias and waits for drain. See file segments.

Reports

submit_and_wait returns a BatchReport with completed request and byte counts, failure details, notification status and local drain status. The completed prefix is not a transaction. RemoteEffect::Unknown means an attempted WRITE may have changed peer bytes, and never authorizes replay. TransferFailure.affected_requests counts the consecutive tasks that share a failure from request_index; only later requests were never attempted. A multirail or remote-staged request counts only after all its slices or chunks complete, but earlier slices may already have changed bytes. For asynchronous ownership see async batches.

For explicit progress, call engine.progress_batch(batch.id()?)?. A status query observes the published result. EngineConfig::enable_progress_worker enables event-driven turns and deferred free. Use Batch::submit_with_notification_targets to track deliveries independently of the payload result. See explicit progress for backoff, queue wakes, bounded retries and owner retirement.

Native providers

The CUDA feature accepts compatible cudarc 0.19 releases. Rust 1.85 builds use 0.19.0 in the checked-in lockfile. Consumers resolving a fresh lockfile on 1.85 can select it with cargo update -p cudarc --precise 0.19.0. Consumers using cudarc 0.19.8 share one registry package with OrbiTE on Rust 1.88 or newer, as required by that release’s libloading dependency.

OrbitKV deliberately passes its Arc<cudarc::driver::CudaStream> as the lifetime owner for local copies. Code that names that type follows cudarc’s semver. The underlying unsafe LocalCopyEngine::bind accepts a *mut std::ffi::c_void and an Arc<dyn Send + Sync>. A caller with its own CUDA binding can supply its raw stream and owner directly. The same contract applies to ProducerEvent::from_raw, so a future C adapter can retain its own handles. Local copy ownership

transport::rdma::RdmaConfig selects discovery and pins, host Direct or staged memory, progress mode (Polling by default, or EventDriven) and native budgets. The cuda module owns device and pinned-host allocations and synchronized copies; device transfers use staging or explicit GPUDirect RDMA. Unsafe CudaMemory::record_producer_stream and HostMemory::record_producer_stream hand already-queued producer work to transfers; read the producer contract before using them.

Close and drain

close_source and close_requester fence one direction; whole-engine close fences both, drains accepted work and closes shared native resources. A timeout never completes pending native work, and Error::Unquiesced keeps the real resources and charges when cleanup could not be proved. With cuda, flush_nvlink_cache(timeout) closes the cached imports present when it starts while admission stays open; see the cache contract.

Runnable examples: TCP, external allocation and async batches.

The Python API uses the same Rust owners through the optional wheel boundary; Rust consumers do not compile or load Python.