Skip to main content
Version: 2.0 prerelease

Workflow Streams

Workflow Streams are named, run-scoped output logs owned by Durable Workflow Server. PHP, Python, and Rust expose typed list, describe, subscribe, append, close, and error operations, so applications do not need to assemble HTTP requests. The runtime advertises durable-workflow.v2.workflow-streams.contract@1; the machine-readable SDK matrix records the exact support in each SDK.

Each stream assigns monotonically increasing offsets beginning at 0. A page's next_offset is the next inclusive offset to request. Delivery to consumers is at least once: finish the page's idempotent effects, then durably checkpoint next_offset. A crash before that checkpoint may redeliver items.

Emit from workflow code

The three SDKs emit through the same deterministic command boundary: record_side_effect.workflow_stream. Each SDK derives the item idempotency key from the task's durable workflow command identity, the stream-command ordinal, and the batch index. Server commits the stream mutation and recorded side effect together. Replay consumes that side effect and cannot create another durable item for the same logical append.

PHP:

use DurableWorkflow\Model\WorkflowStreamAppendItem;

$context->appendWorkflowStream('progress', [
new WorkflowStreamAppendItem(['percent' => 50]),
]);
$context->closeWorkflowStream('progress');

Python:

from durable_workflow import WorkflowStreamAppendItem

yield ctx.append_workflow_stream("progress", [
WorkflowStreamAppendItem(payload={"percent": 50}),
])
yield ctx.close_workflow_stream("progress")

Rust:

let item = WorkflowStreamAppendItem::new(serde_json::json!({"percent": 50}))?;
ctx.append_workflow_stream("progress", &[item], None)?;
ctx.close_workflow_stream("progress", None)?;

The workflow helpers are for deterministic authoring. Code outside a workflow can use each client's typed append operation and supply its own stable idempotency key.

Subscribe and resume

Subscription reads a bounded page, optionally waiting for new items for up to 60 seconds. PHP accepts a cancellation callback between bounded polls. Python accepts normal task cancellation or an asyncio.Event that cancels the in-flight poll. Dropping Rust's subscription future cancels its request.

SDKPage operationResume valueTyped iteration
PHPsubscribeWorkflowStream(...)WorkflowStreamPage::$nextOffsetiterateWorkflowStream(...)
Pythonawait subscribe_workflow_stream(...)WorkflowStreamPage.next_offsetiter_workflow_stream(...)
Rustsubscribe_workflow_stream(...).awaitWorkflowStreamPage::next_offsetRepeat the bounded page future until terminal.

open, closed, and errored are the lifecycle states. The typed description also exposes last_offset, total_items, pending_items, and error_reason. An append to a terminal stream is rejected. When the configured pending bound is reached, append returns stream_full; producers must slow down or wait for consumers to drain the stream.

External payload references

Inline values use the shared Avro Value envelope. External references follow each SDK's existing storage contract rather than inventing another transport:

SDKExternal reference behavior
PHPAppends and returns an opaque payload_reference; storage upload and fetch remain application-owned.
PythonUses the configured external-storage driver to upload, integrity-check, cache, fetch, and decode an external Avro envelope. Without a driver, callers can append or inspect an opaque payload_reference.
RustAppends and returns an opaque reference and its metadata. The current Rust SDK does not claim an external-storage driver.

Workflow Streams and embedded MessageStream

The names align where behavior aligns, but the models are not interchangeable.

ConceptService-mode Workflow StreamEmbedded Laravel MessageStream
AddressWorkflow run + stream nameWorkflow instance/run + stream key
DirectionWorkflow output onlyWorkflow inbox and outbox
First offset01
DeliveryAt least once; consumer-owned checkpointAt least once; engine-owned message cursor
Continue as newNo stream cursor transferInbox cursor transfers to the continued run
Inbound workflow messagingNot provided; use signals or updatesProvided by inbox()->receive()

Waterline uses one normalized table for both modes and shows the mode, lifecycle, head/cursor offsets, pending count, direction, and error. It does not present service output streams as an inbound workflow inbox.

Qualification

The public Workflow Stream scenario manifest requires published-artifact runs for PHP producer to Python and Rust consumers, producer worker restart, consumer reconnect, lifecycle/backpressure/cancellation, and external payload references. A release result must record every required scenario; source-unit tests alone are not conformance evidence.