NIXL backend design

Proposal only. No plugin, C ABI or NIXL qualification exists yet. The target is an in-tree backend in ai-dynamo/nixl, under src/plugins/orbite, rather than a fork or a connector maintained inside each inference engine. OrbitKV’s upstream targets are vLLM and SGLang; the NIXL path would let vLLM NixlConnector, SGLang’s NIXL backend, Dynamo and LMCache P2P select OrbiTE. This is an integration goal, not a claim that those consumers have tested it.

This review pins NIXL main at 82b356ce (checked 2026-11-09, whose Meson project version is 1.6.0), following the maintainer’s v1.5.2 survey. The South Bound guide, backend header, plugin entry points, Mooncake backend and contribution rules are the contract reviewed here. Recheck that pin before implementation. OrbiTE’s design/oracle baseline remains Mooncake TENT v0.3.14; NIXL is neither a new runtime dependency of the Rust crate nor a replacement for that baseline.

The work order puts FABRIC removal, dma-buf GPUDirect registration, local-copy SM budgeting and intent-aware Auto, and the small-WRITE caller wait before plugin implementation. Design alignment is complete. The upstream explicit-cancel API, the outcome of the existing lifetime discussion and the hardware batch below are prerequisites. The maintainer must approve the formal backend RFC. Nothing in this document authorizes posting, commenting, forking, publishing or opening an upstream PR.

Related upstream work

States below were checked on 2026-11-09. The reviewed main advanced from 540fae3a by one device-API test commit; the South Bound interface, guide and three core lifetime behaviors below are unchanged. These are existing upstream discussions and proposals, not new findings or a parallel cancellation design.

Upstream item	State on 2026-11-09	Consequence for this design
#1958: asynchronous request cleanup	Open; cancellation label, filed by lluki, assigned to ColinNV	Follow NVIDIA’s cancellation work. lluki favors keeping the request handle alive outside shutdown. A public comment mentions an internal design document; its contents were not reviewed here.
#1758: deregistration results, #2296: propagation draft	#1758 closed; #2296 closed, unmerged	rakhmets expects release methods to warn and return void, with current deregisterMem always returning NIXL_SUCCESS. lluki’s outstanding-transfer question remains relevant. Fence and defer internally; do not require propagation of deregistration errors.
#2209: explicit cancel, #2244: failed-release polling	Both closed, unmerged	Target a separate cancel method; dropping a handle becomes terminal. Do not build clients that cancel by releasing.
#2540: UCX active-handle retention	Open	lluki’s 2026-11-06 comment reiterates explicit cancel and opposes creating dependencies on implicit cancellation. Active release refusal is a safeguard, not the new cancel API.
#2347: invalidated transfers and UCCL connections	Open	Coordinate invalidation polling and connection retention with this work; fencing stale posts must not hide progress of admitted requests.
#2077: POSIX deferred free, #1955: active-handle use-after-free	#2077 closed, unmerged; #1955 closed	Borrow the reference-counted lifetime principle, not #2077’s success-before-drain release semantics. #1955’s revised contract keeps a handle with outstanding work alive.
#1792: duplicate-descriptor double free	Open	Distinct registration aliases need distinct references; duplicate deregistration must not destroy one native owner twice.
#2254: GPUNetIO release leak	Open	Retirement must account for every operation of this request, never another request’s completion or a shared ring position.
#2082: Mooncake TENT mode, #2083: its RFC	Both open; authored by xiaodouzi666	A working plugin and H20/RoCE runs are already reported. Its deferred-release engine list is relevant prior work; compare ownership contracts and results rather than duplicate the proposal.

Why an OrbiTE backend alongside Mooncake TENT mode?

TENT mode already has a working plugin and recorded H20/RoCE runs in the open work above; OrbiTE has neither a NIXL plugin nor installed NIXL results on its current artifact. Those upstream reports are not OrbiTE qualification. OrbiTE offers an independent Rust implementation without the Mooncake C++ runtime, separate logical-outcome and physical-retirement receipts across RDMA, CUDA, file and GDS paths, inference-intent priority, and one ownership model for HBM, DRAM, NVMe and peers. These are reasons to evaluate another backend, not performance claims or claims that TENT mode lacks these goals. Intent propagation through NIXL still needs the request mapping below.

Owner and capability boundary

One NIXL backend instance owns one TransferEngine. The C++ adapter converts STL descriptors and statuses; Rust’s existing registrations, RemoteSegment, Batch, admission, progress and transport owners do the work. There is no plugin payload thread, forwarding queue, second scheduler or retry loop. enableProgTh and pthrDelay map to the existing optional progress worker and its validated fallback interval; they do not disable existing transport workers. NIXL thread-safety modes do not relax Rust’s locking or lifetime rules. Take one explicit OrbiTE JSON configuration through backend creation parameters; reject unknown plugin parameters. Do not reinterpret NIXL agent names as addresses, read TENT environment aliases, or mutate the process environment. NIXL supplies no standard per-request OrbiTE intent or policy field. Initially use configured defaults and reject unrecognized nonempty customParam data; do not silently claim inference-intent propagation. P/D arbitration needs an agreed request mapping before that integration can be qualified.

Propose ORBITE for memory transfers: supportsRemote=true, supportsLocal=true through a self connection, and supportsNotif=true. Advertise DRAM always and VRAM only when CUDA support is built and available. Capabilities are promises to validate requests, not a claim that every advertised memory range has every transport. Compiled transport policies and owner-derived capabilities still select the route; no adapter fallback is added.

NIXL memory or pair	Existing OrbiTE path and restriction
DRAM_SEG	Retained HostMemory and registry ranges. TCP; opt-in SHM for actual shared mappings on the same boot identity; RDMA when installed. HP-TCP remains eligible only through their existing configured policies.
VRAM_SEG	Driver-validated CudaMemory, allocation/context lease and range loans. RDMA GPU Direct or staging; TCP through staging; NVLink for eligible same-host IPC allocations; MNNVL only for fabric-exportable backing and an explicit fabric capability. Neither IP addresses nor a device ordinal prove reachability.
DRAM ↔ VRAM	The existing staged TCP/RDMA or eligible CUDA/pinned-memory owners. Ordinary pageable DRAM is not silently relabeled pinned memory or CUDA IPC backing.
FILE_SEG ↔ DRAM/VRAM	Local segment::file::Entry, io_uring and CUDA staging, or eligible GDS. Native GDS alignment, strict explicit hints, completed-prefix results and fallback selection retain their current meaning. No remote file service.
BLK_SEG, OBJ_SEG, non-NVIDIA device memory	Not advertised; return NIXL_ERR_NOT_SUPPORTED before registration or posting.

SHM cannot turn an arbitrary NIXL malloc allocation into shared memory. Use TCP for that range. SHM tests and consumers need an actual OrbiTE shared allocation, retained across registration, or a separately validated import of the same mapping. Copying the allocation at registration would break subsequent updates and is not an implementation option. Likewise an ordinary CUDA pointer does not establish MNNVL exportability. NIXL callers must finish producing bytes before posting; NIXL’s descriptor has no CUDA producer event. Do not synchronize an arbitrary caller stream implicitly. A later explicit event extension would use CallerSyncEvent, not weaken this prerequisite.

Local storage profile

NIXL’s file convention fits OrbiTE’s offset-addressed file segments, but its capability flags are backend-wide. getPublicData is requested for registrations of a remote-capable backend; advertising FILE there would imply remote metadata that OrbiTE does not implement. Propose a local-only ORBITE_FILE entry point in the same plugin directory and bridge, following the POSIX/GDS storage model: supportsLocal=true, supportsRemote=false, supportsNotif=false. Each instance still owns just one engine. Ask upstream to agree to this split before adding the second entry point.

Exactly one side of a file transfer must be FILE. NIXL READ moves remote-list bytes to the local list; WRITE moves the reverse way. Normalize these to memory plus file offset once, reversing the OrbiTE operation when FILE is on the local list. Test all four combinations. Descriptor pairs preserve order, lengths and guards; only adjacent file extents may become a file_vectored operation. Reject file↔file, attached notifications and requests to another agent.

Path registration accepts only modes whose semantics are implemented. NIXL ro/rw, direct/sync flags and borrowed-fd mode cannot silently change access or durability: unsupported flags return NOT_SUPPORTED. Borrowed fds require a checked dup into the existing file Entry; re-resolving /proc/self/fd or a path could open a different inode. This needs a small internal fd-adoption entry point, not a storage executor. queryMem reports supported file facts from that Entry and returns NOT_SUPPORTED for unsupported query fields.

South Bound API correspondence

These are proposed mappings for every virtual method in the pinned header, including the optional defaults. No new Rust public API is promised here.

NIXL call	OrbiTE owner and adapter action
Constructor / destructor	Validate a captured OrbiTE configuration; create one engine and bounded handle storage. Close uses existing source/requester drain. The void destructor follows the retention rules below, never frees unresolved owners.
supportsRemote, supportsLocal, supportsNotif, getSupportedMems	Immutable memory/storage profile above, filtered by built capabilities. Unsupported combinations are rejected before payload admission.
registerMem	Checked address/length/type/permissions → retained backing and register_memory_ranges; CUDA uses native allocation/device verification. FILE → existing file Entry. Duplicate and overlapping registrations retain the registry’s authority/alias rules.
deregisterMem	Fence new posts immediately and drop the lookup reference. Defer native deregistration until the last borrower retires; failed cleanup retains the owner and charge internally with diagnostics. The expected NIXL_SUCCESS is not an allocation-release receipt.
getConnInfo	Bounded encoding of native endpoint() and source_epoch(), backend/profile identity and metadata format. No cluster discovery or external store.
loadRemoteConnInfo	Validate bounded input; bind the NIXL agent name to an endpoint plus owner-issued epoch. A changed identity fences the old one rather than retargeting prepared requests.
connect	open_segment(endpoint, expected_epoch, timeout) and existing native Describe. Keep its requester-owned identity token. Self connections use this engine’s actual epoch. No TENT framing or autodetection.
disconnect	Fence with the existing SegmentManager/close_segment phases, then drain and retire only that identity’s connections/imports. Repeated observations may wait for drain; a failed native destroy is never retried.
getPublicData	Serialize only the registered range’s identity, bounds, memory kind and permissions plus the endpoint/epoch binding. Native Describe remains authoritative for registration generations, transport keys and capabilities. No FILE paths or fds are published remotely.
loadRemoteMD	Validate agent/epoch/registration identity and bounds against the retained native descriptor; create a lease, not another transport registration. A range or generation mismatch fails before admission.
loadLocalMD	Retain the local registration or file Entry for self/storage transfers. A distinct metadata wrapper references the same owner, so unloading it cannot deregister the source accidentally.
unloadMD	Fence this imported metadata lease against new prepares/posts and release its lookup reference. Existing admitted requests retain their peer/registration permits; peer invalidation additionally fences the whole epoch. Wrapper removal is not physical release.
prepXfer	Validate both lists, permissions, equal total lengths, all checked range arithmetic and descriptor/byte limits. Retain immutable descriptors, registrations and peer identity. No payload I/O. Policy compilation stays in engine construction.
postXfer	Admit a fresh Batch execution through the existing submit path, then seal. Preserve physical-owner merging, aliases, QoS, credits, policy and deadline. Copy bounded notification data before returning; never retain pointers to call-scoped optional arguments.
checkXfer	One bounded existing progress_batch step and observation of task/notification outcomes and physical drain. Use the status table below, never infer quiescence from a timeout or a task’s logical state.
releaseReqH	Release only after retirement; do not initiate cancellation. An active request returns a negative status and keeps every owner and the handle. IN_PROG is not safe here. A dropped wrapper cannot free unresolved native owners.
Both prepMemView overloads, releaseMemView	Keep default NOT_SUPPORTED for prepared strided views initially; descriptor lists still work. The void release is a no-op for an absent view. Do not advertise an unimplemented optimization.
genNotif	Existing bounded notification send, endpoint/epoch permit and exactly-once logical delivery. Standalone notification failure is not permission to replay a transfer.
getNotifs	Drain existing notification queues into NIXL (agent, bytes) pairs. The envelope carries a validated sender identity and length, mapped to its current peer lease; reject oversized or stale envelopes. Preserve binary payloads.
queryMem	Local storage facts as above; NOT_SUPPORTED for memory queries without an existing owner-backed answer. Never invent capacity or topology.
estimateXferCost	NIXL_ERR_NOT_SUPPORTED; measurements do not justify a cost model yet.
supportsTraceContext	false initially. Do not retain traceContext or claim cross-peer tracing support.

Inherited getInitErr, getType, getCustomParams, getInitParam and setInitParam keep NIXL’s base-class behavior. getTelemetryEvents drains its bounded base queue; use addTelemetryEvent only for supported lifecycle events derived from existing stats, with no payload addresses or invented values. The plugin factory implements create_engine, destroy_engine, get_plugin_name, get_plugin_version, get_backend_options and get_backend_mems, exposed through nixl_plugin_init/nixl_plugin_fini. Creation failure sets the expected init failure/null result, never an exception crossing the plugin factory.

The planned explicit cancel method is absent from the pinned header. When it lands, map it to existing queued cancellation and running-I/O drain, then keep polling until retirement. Do not invent its signature or emulate it through releaseReqH in the meantime.

Preparation may do bounded control-plane work. postXfer, checkXfer and request release must not wait for payload completion. Verify that withheld CUDA readiness and CQEs cannot block those calls; any required asynchronous continuation belongs in the existing batch worker. A successfully drained execution may be reposted only after revalidating all retained identities and resetting execution/notification state. Failed, canceled or uncertain executions cannot be reposted, even if NIXL calls postXfer again. There is no plugin retry.

NIXL notifications are binary while OrbiTE’s Notification fields are UTF-8. Use a bounded versioned envelope with sender endpoint/epoch and base64 payload; charge the expanded length before allocation and check it against notification limits. Decode losslessly, including embedded NUL/non-UTF-8 bytes, and reject malformed or stale envelopes. Matching the loaded peer identity is a generation fence, not authentication; both deployments remain cooperative-peer systems.

Completion, invalidation and allocation lifetime

The bridge retains two separate facts: logical outcome (including exact prefix and RemoteEffect) and physical retirement receipt. Every terminal checkXfer status, including an error, must mean the backend can no longer access this execution’s local or remote buffers. Logical failure with drain pending stays NIXL_IN_PROG. Success additionally requires every public alias and required notification to succeed. Batch::free() can defer retirement; its success alone is not a receipt. Check source-side borrowers and the native/staging/context owners too; a local status alone is insufficient.

OrbiTE outcome	Proposed NIXL status and retention
Prepared, not posted	NIXL_ERR_NOT_POSTED; no payload effect.
Admitted/queued/running, or outcome known but drain pending	NIXL_IN_PROG from checkXfer; retain all owners. Release returns a negative error until safe.
All tasks and required notification succeeded, local and remote buffer access retired	NIXL_SUCCESS from checkXfer. Request release succeeds only after its remaining resource-retirement checks.
Queued cancellation, proven drained	NIXL_ERR_CANCELED; all merged aliases share the owner’s result. Running I/O still drains.
Invalid range/config/type before posting	NIXL_ERR_INVALID_PARAM; unsupported capability → NIXL_ERR_NOT_SUPPORTED; permission failure → NIXL_ERR_NOT_ALLOWED. No admitted payload work.
Epoch/generation mismatch or removed metadata	NIXL_ERR_MISMATCH / NIXL_ERR_NOT_FOUND before posting or after drain; an admitted execution remains IN_PROG until retired. Never bind the old handle to a new peer.
Resource exhausted before posting	NIXL_ERR_BACKEND plus the bridge’s typed detail; no unbounded overflow allocation. NIXL has no matching resource-exhausted status.
Provably unposted disconnect	NIXL_ERR_REMOTE_DISCONNECT, retaining the unposted/effect-known detail. A status code by itself never authorizes replay.
Short/failed I/O, timeout after admission, unknown WRITE effect	Stay NIXL_IN_PROG until physical retirement, then NIXL_ERR_BACKEND with the exact prefix/effect in the native result. Retirement does not make the WRITE’s effect known. No plugin retry or cross-backend replay.
Unquiesced, including failed native cleanup	No safe terminal mapping is agreed in the current API. Retain owners, charges and code/library lifetime; never expose a sticky NIXL_ERR_BACKEND as a retirement receipt. The upstream question below must be settled before implementation.

NIXL cannot encode OrbiTE’s full error or an independent drain flag in one status. Keep those details in the bridge result, report bounded diagnostic counters, and do not promote an error to success to make a handle disappear. A terminal status permits buffer reuse only because physical retirement has already been proved; it never grants permission to replay an uncertain WRITE.

The pinned BackendGuide transfer section still describes abort-on-release, release errors until completion, and an error from checkXfer after a failed abort. This proposal keeps active release refusal but deliberately rejects the other two behaviors: use the agreed explicit cancel direction in #2209/#2244/#2540, and keep checkXfer in progress while any buffer access remains possible. Dropping a handle is terminal, not a cancellation request. Clients must poll to retirement before release; an incorrect early drop retains owners rather than freeing memory they may use.

Open lifetime questions in the existing upstream work

The pinned core still has three behaviors covered by the related work above:

1. releaseXferReq treats a negative active-release result as NIXL_ERR_REPOST_ACTIVE and retains the handle, but treats any nonnegative result as released. Therefore releaseReqH must never return IN_PROG while owners remain. A negative refusal preserves the in-progress handle for polling at this revision. Once status is terminal, the nixlXferReqH destructor ignores the release result.

2. nixlLocalSection::remDescList ignores deregisterMem’s result, removes metadata and returns success. Registration rollback and destructor paths also ignore cleanup results. Follow #1758’s expectation: fence new posts and defer native deregistration to the last borrower. Failed cleanup stays an internal Unquiesced owner with diagnostics, never a retried destroy. The reference-counted lifetime in the closed #2077 proposal is useful here, without adopting success-before-retirement for transfer release.

3. Remote invalidation removes the remote section and calls disconnect, but erases its backend mapping even when disconnect fails. The plugin must fence and retain identities independently of those C++ containers. NIXL’s expired remote-section check rejects stale posts but also blocks polling of admitted work. That core error is not a backend retirement receipt. #2347 addresses that progress/connection-lifetime boundary; coordinate with it rather than introduce a competing mechanism.

Prerequisites: the upstream explicit-cancel API and the #1958 outcome must resolve how a permanently unproven retirement is represented. A distinct nonterminal status with continued ownership, or an explicit quarantine contract that keeps borrowed allocations and code alive, are questions for upstream. Neither is agreed, and no existing terminal status is claimed safe for Unquiesced. Keeping IN_PROG while drain is pending is necessary but does not by itself settle permanent failure, agent shutdown or library unload. Propagation of deregistration errors is not an implementation prerequisite.

NIXL passes raw application pointers, not allocator owners. For borrowed registration, the caller must keep the allocation and CUDA context alive until physical retirement, including source-side borrowers; a successful deregisterMem call only fences admission under the proposed deferred cleanup. Retaining a C++ metadata object or a Rust Arc containing its address does not retain that allocation. Where an actual allocator lease is available, the bridge accepts retain/release callbacks and releases it only on a receipt. For borrowed pointers, the NIXL contract must make that obligation observable. How to do so with success-only deregistration and agent destruction remains open under #1758/#2296. With uncertain cleanup, retaining metadata alone cannot make borrowed memory safe; do not claim support for freeing it after deregistration or destruction without a physical receipt. Allocator leases help only when they own the actual backing.

Remote unload/invalidate fences admission first. Accepted transfers keep their RemoteSegment, registration/range loans, staging and charges until the backend proves release. close_segment owns peer cleanup and its sticky failure state; the adapter does not retry destroys or maintain another connection cache. An explicit new epoch opens a different identity; stale prepared requests fail.

The void engine/plugin destructor cannot return Unquiesced. After a bounded close attempt it retains unresolved native owners and pins every library containing their destructors or callbacks. The plugin DSO must not be unloaded while those callbacks can execute. Pinning is charged per retained engine, not per retry; failed destroy stays one-shot. This does not manufacture an allocation lease for borrowed memory. Safe agent destruction and dlclose with outstanding borrowed registrations remain open questions under #1758/#2296 and the #1958 outcome, not a contract this backend can impose by itself.

Minimal native boundary and packaging

Use a separate, opt-in orbite-nixl-ffi crate with a nixl-ffi feature; default Rust/Python builds neither export nor link it. Its C++ client lives in NIXL, while its handles directly retain existing Rust owners. This is a real packaging/language boundary, not restoration of libtent or the 33 TENT calls. The following operation groups bound the proposed ABI; finalize signatures with upstream rather than freeze speculative general-purpose APIs here.

C operation group	Minimum data and existing owner
ABI information; engine create/close/release	ABI/record sizes, feature bits, explicit immutable OrbiTE JSON and engine handle; close returns outcome plus drain state.
Register/retire memory; optional shared allocation	Pointer, extent, kind, device, permissions and actual owner lease when available; generation-checked registry handle. Shared allocation reuses allocate_shm_memory, never copies caller memory.
Export/import peer and range metadata; retire peer	Caller-sized byte buffers, endpoint/epoch and registration identity; existing segment/registration leases.
Prepare/post/cancel/query/release execution	Bounded descriptor arrays, operation, deadline and optional notification; prepared owner references and existing Batch. Cancel is separate from release, following the future upstream API. Query returns logical code, transferred prefix, effect and physical drain separately.
Send/drain notifications	Length-delimited binary message, peer handle, caller-sized output buffers; existing charged queues.
Open/query/close local file (storage profile only)	Validated path/mode or duplicated fd, checked offsets; existing file Entry and close/drain.

Opaque generation-checked handles, fixed-width repr(C) records with size and ABI fields, explicit lengths and caller-owned output storage avoid Rust/STL layout coupling and allocator mismatches. Generate include/orbite/nixl.h from the Rust declarations with a build-only header generator; CI checks the generated header, LP64 sizes/offsets and a minimal C consumer. Pin the bridge to the exact core release if it uses private receipt helpers. Shared build dependencies follow the normal minimum-version/lockfile policy and Rust 1.85. This docs PR adds no header, ABI exports or build dependencies.

Every entry has a panic barrier and typed status. Unsafe conversion is scoped and has SAFETY notes for pointer lifetime, bounds, alignment, aliasing and callback ownership. Invoke foreign callbacks outside native locks. Reject stale handles, oversized arrays, unknown record versions and overflow before admission. Bound wrapper objects and metadata bytes explicitly against engine registration, peer and batch limits; quarantine continues to consume those budgets. Do not add an uncharged C++ pending-request list.

Install liborbite_nixl.so, its generated header and orbite-nixl.pc. Meson discovers that dependency and builds the in-tree plugin using NIXL’s C++20/STL-only plugin interface. No Mooncake library or Python interpreter is linked. Native CUDA/RDMA/GDS dependencies retain their current opt-in loading rules. NIXL’s plugin API version (currently 3), the bridge ABI version and OrbiTE’s wire policy are separate checks. Reject an ABI-major/layout mismatch before engine construction. Additive records negotiate size/features; breaking changes require a new ABI/SONAME and the applicable OrbiTE release bump. Before the first stable release deploy matching plugin/core builds on both peers under the existing wire policy. No new wire layout is frozen by this proposal.

Internal work needed for this boundary is limited but real: per-registration retirement receipts (unregister alone is insufficient), nonblocking observation of existing close phases, and fd adoption for the storage profile. Put each with its current registry/segment/file owner and test it there. Do not implement a second executor to hide a missing receipt.

Acceptance controls

All rows below are planned, not new test evidence. Use NIXL GoogleTests under its current test/gtest/plugins/ layout, a small installed C bridge consumer, and existing OrbiTE shims for injected native errors. Test the public NIXL agent as well as direct backend methods so ignored return codes cannot escape detection. Every byte test checks destination guards too.

Control	Negative case and expected owner/charge state
N01: DRAM TCP/SHM	Two real processes, READ/WRITE, fragmented lists and merged aliases. Ordinary malloc cannot select SHM; an actual shared allocation can. Bytes and guards agree, accounting returns to baseline only after drain.
N02: active request release/cancel	Hold a real response after partial WRITE. Release returns negative without invoking cancel; the handle remains pollable, check stays IN_PROG, and backing/peer/charges stay. Exercise the future explicit cancel separately: queued work proves nothing posted; running I/O drains. Unblocking produces exactly one retirement receipt.
N03: logical failure before retirement	Inject timeout, short completion and unknown WRITE effect while withholding one operation’s receipt. No terminal checkXfer status until every local/remote access retires; another request’s completion cannot retire this one. Early wrapper drop retains unresolved owners. No replay, and no later buffer writes after a terminal result.
N04: deregister/rollback/duplicates	Deregister with local and remote borrowers, duplicate descriptors, and a later multi-registration failure. Fence new posts immediately; NIXL may return success while native cleanup is deferred. Keep the actual allocation lease, registration and charge until the last borrower retires, and destroy each owner once. Borrowed-pointer tests require the upstream allocation-lifetime agreement; success alone must not authorize freeing.
N05: metadata unload/restart	Unload/invalidate during an admitted copy. New posts fail, admitted owners stay and remain pollable even after core metadata removal; a new epoch works, the old epoch cannot repost, and another peer is unaffected. Align with #2347.
N06: shutdown and failed destroy	Inject failed QP/MR, event/import, file or library cleanup. Unquiesced must not become terminal ERR_BACKEND; allocations, callback code, native resources and charges stay. Repeated observation makes no second destroy call. The accepted upstream quarantine/shutdown rule must prevent borrowed-memory reuse and dlclose; this control remains blocked until that rule exists.
N07: prepare/repost and bounds	No payload during prep; pending CQE/producer readiness cannot block post/check/release. Active/failed/canceled handles cannot repost. A drained success may execute once more with fresh accounting. Overflow, budget exhaustion and mixed authorities fail before posting.
N08: notifications	Binary payload, self notification, stale sender epoch, oversized message and one unreachable target. Exactly-once logical delivery; transfer success requires its requested notification outcome. No WRITE replay on notification error.
N09: local files	Real io_uring ext4/tmpfs, both file-list positions and READ/WRITE, fragments, EOF inside a fragment, fd reuse and close during I/O. Copy back only the completed prefix. Remote FILE and unsupported modes fail before I/O.
N10: bridge/loader	Missing library/feature, ABI/size mismatch, stale handle, panic and C++ factory failure; no allocation/admission and no exception crossing ABI. Default core build has no plugin dependency.

After the upstream contracts are settled, GitHub-hosted Linux will run N01–N05, N07–N10 for the CPU routes with packaged OrbiTE, pinned NIXL, stable and Rust 1.85. Require io_uring explicitly for N09; repeat timing-sensitive controls 50 times. Only those actual CPU/TCP/SHM/file profiles may become CI-qualified after passing. CUDA/verbs/cuFile shims for N02–N07 establish ownership behavior, not native qualification.

The plugin verification queue reserves installed VRAM GDR/NVLink and FILE GDS cases, plus MNNVL if advertised. Plugin gate drivers and exact commands must be written with the implementation; none exist or run in this docs step. Require route evidence, full bytes/guards and the same failure/retention controls on locked hardware. Before the formal backend RFC, the hardware batch must produce installed GPU/RDMA results for the current OrbiTE artifact and nixlbench UCX/Mooncake baselines. Native gates do not qualify a wrapper that does not exist. After implementation, compare the OrbiTE plugin on the same hardware with matched versions, sizes, batch shapes, concurrency, warmup and byte validation. No plugin performance claim precedes those measurements.

Upstream design issue draft — v2, not posted

Title: Proposal: an OrbiTE backend following NIXL’s explicit-cancel contract

Hold before posting. This formal backend RFC waits for the upstream explicit-cancel API, the #1958 outcome, installed GPU/RDMA results on the current OrbiTE artifact and the hardware batch’s nixlbench UCX/Mooncake baselines. Recheck upstream state then; the maintainer decides whether to post.

Proposal. An in-tree src/plugins/orbite backend for OrbiTE, an independent Rust CPU/NVIDIA engine for KV movement. A small C++/STL adapter and opt-in C bridge reuse its existing owners and inference-intent scheduler, without the Mooncake runtime or a new coordinator. Start with DRAM TCP/SHM and validated VRAM GDR/NVLink; ask whether a local-only ORBITE_FILE entry point fits storage.

Existing work and questions. Follow #1958 (open, 2026-11-09) and the explicit-cancel direction reiterated in #2540 (open, 2026-11-09), not a parallel cancellation protocol. Every terminal check must mean physical retirement; failed-but-draining work stays in progress. How should permanent Unquiesced retention, borrowed allocations under success-only deregistration, and agent/DSO shutdown be represented without a false receipt? Deregistration fences new posts and defers cleanup internally. An uncertain WRITE never replays.

Evidence and scope. TENT mode already reports a working plugin and H20/RoCE results; OrbiTE’s distinction is its Rust dependency boundary and physical-retirement owners across HBM, DRAM, NVMe and peers. Planned CPU GoogleTests move real TCP/SHM bytes with lifetime and guard checks; GPU/file routes require installed gates. No performance claim is made here. Attach the hardware results and comparison parameters before requesting agreement.

Follow NIXL’s contribution process: design agreement before a significant PR, tests, public STL types and the contributor’s own DCO sign-off. This draft is not authorization to post or implement the plugin.