Asynchronous CPU batches#

P1.4 implements a bounded CPU/TCP subset of TENT F08/F10. The completion plan remains the only execution queue. This source profile is pending independent acceptance; RDMA/CUDA and installed consumer qualification are separate.

Ownership and API#

Use Arc<TransferEngine>::allocate_batch(capacity) to reserve descriptor slots and an engine-scoped monotonically increasing BatchId. Capacity is positive and at most max_batch_requests. max_allocated_batches and max_allocated_requests bound all live reservations, including completed batches that have not been freed and dropped handles whose workers remain active.

Batch::submit(operation, peer, requests, notification, timeout) appends a group and returns its stable public task range. All addresses, permissions, bytes and source authority are validated before enqueue. The group owns resolved local registrations and a requester-admission charge. A rejected group adds no public tasks; accepted earlier groups are never replayed or rolled back. Groups in one batch execute FIFO; different batches can progress concurrently. Each batch has at most one worker, bounded by its allocation reservation. Submission records, including empty submissions, are bounded by capacity too.

seal() stops append. status(), task_status(), task_statuses() and submissions() only read worker-published state; they perform no transfer, retry, notification or progress step. Before seal the overall state is Open, even if every task submitted so far has completed. After seal, Pending becomes Completed, Failed or Cancelled only after all accepted groups exit. Per-task terminal states never change, and the sealed overall terminal is stable.

wait(timeout) requires seal and uses a condition variable. Its own timeout neither cancels nor releases work. A submission’s independent absolute deadline begins at submit and includes validation/queue time and attached notification.

free() returns BatchBusy while a group still owns local work, preserving the handle for retry. Once idle it seals, joins its worker, releases descriptor capacity and its engine reference, and invalidates further batch operations. Dropping an active handle instead disconnects the queue without discarding accepted groups: the detached worker retains the engine, registrations and both allocation/admission budgets until it exits. This prevents repeated handle drops from bypassing the thread or retained-owner limits.

submit_async(...) allocates, submits one group and seals it. See the runnable async_batch example for multiple appends.

Results, cancellation and quiescence#

Tasks expose waiting/running/completed/failed/cancelled/skipped states, exact bytes of fully completed requests, cancellation requests and a retained structured failure. A failed remote staging request can have visible completed READ chunks while contributing zero completed bytes; drain status is not rollback. Failure indices are public batch task indices. request_index identifies the first failed task; affected_requests gives the consecutive failure span. A shared NVLink close/ACK failure can cover several already attempted tasks. Every task in that span shares the failure and drain status; only subsequent tasks are skipped/cancelled without I/O. A clean copy/cancel failure followed by a source ACK preserves the copied prefix. Other accepted groups still run. Submission reports preserve each group’s completed prefix and independent NotificationStatus; successful data can coexist with notification failure. Empty or failed groups do not send attached messages.

cancel(task) and cancel_all() request cancellation. Waiting work checks that request before dispatch. A running CPU operation finishes or reaches its existing deadline; cancellation does not interrupt an unproven physical operation or relabel committed bytes as cancelled. A late cancellation can therefore coexist with a completed task. This profile has no provider abort or cancel receipt.

locally_drained means the requester’s physical access to its registered backing has ended. Group/batch drain is published after dropping the actual prepared owners and their admission charge. It does not establish remote source quiescence. In particular, a lost WRITE acknowledgement retains RemoteEffect::Unknown, forbids automatic replay and still requires source-side ownership/drain before reusing remote backing. Source handlers retain their own owners independently.

close_requester(timeout) permanently rejects new outgoing work, including open, notification and async submissions. It waits for accepted queued/running work to release local owners. Timeout leaves admission closed; call again to wait. close_source separately gates source registration and handlers. Existing source-only close behavior is preserved. Whole-engine close(timeout) fences both directions before waiting against one deadline; accepted queued work can still create backend slices while draining. It then explicitly closes shared RDMA device resources. A native cleanup error is retained without retry; see the device owner contract. Closing requester admission does not invalidate existing status/free operations or claim that a remote ACK was read.

Frozen correspondence and limits#

The frozen TransferEngineImpl allocates batch capacity, appends public tasks, retains runtime references and defers freeing batches that remain pending. Rust’s explicit busy-on-free plus detached-handle ownership is a distinct safe API; matching the native deferred C free convention remains an adapter requirement. Rust adds seal to make an overall terminal result stable against further append; native TENT can append after prior tasks complete.

The shared CPU trace now uses real async Rust APIs and native C allocate/append/status/free with the same 25 byte/notification/cleanup observations. Native queue/merge/progress-worker modes remain disabled for that profile. Rust blocked-socket tests exercise queue deadline, cancel, wait timeout, concurrent unregister, requester close and detached engine/allocator release. They do not qualify native fault behavior or device DMA.

RDMA/CUDA now consume these owners with explicit per-request transport selection. Their Unquiesced failures retain native owners and admission/capacity: the logical result is terminal while locally_drained is false, free remains busy and close cannot report drain. No reclaim proof is implemented for this quarantine yet.

Request merging/alias mappings, full transport selection policy, explicit progress, priority/EDF, provider abort/reclaim and language/wire compatibility remain open. No hardware backend inherits this CPU profile’s drain proof.