Yesterday, we gave the worker a way to accept control messages. Today, let's give registration something useful to carry. What does a worker actually need to run a piece of a query?

Sending SQL would be a reasonable starting point. But eventually our coordinator will split one optimized query into several fragments. A fragment might begin halfway through the original plan, with input assigned by the coordinator. I'd like to preserve that work instead of asking each worker to reconstruct it from SQL.

Let's start with a small query whose inputs exist anywhere:

SELECT i, CASE WHEN i = 17 THEN NULL ELSE i END AS nullable
FROM range(5000) t(i);

It produces 5,000 rows. The first column runs from zero to 4,999; the second has the same values except for a NULL at 17. There's no table to copy or file path to agree on. That makes it a useful first serialization check.

Which plan are we sending?

Binding gives names and functions meaning: which column does i refer to, and what type does the expression return? Optimization then rewrites that logical plan. Physical planning turns it into executable operators. DuckDB's internals overview describes these stages.

Our serialization boundary sits after logical optimization and before physical planning:

Why stop there? Logical column references still identify columns through their bindings. Physical planning resolves those bindings into positions used during execution. We want the receiving engine to perform that resolution once. A running operator also owns process-local state, such as buffers and execution context, which doesn't belong in our message.

The native experiment uses DuckDB's binary serializer for the logical operator tree. The receiving side opens a fresh connection and transaction, deserializes the tree, checks its output types, and executes it as a logical-plan statement. That transaction stays open until the result has been consumed.

These are internal engine APIs. Their bytes are a format shared by compatible builds, with none of the durability promises we'd expect from a storage format.

Serializable doesn't mean independent

Consider this variation:

CREATE TEMP TABLE local_numbers AS SELECT i FROM range(5000) t(i);
SELECT * FROM local_numbers;

The coordinator can bind the table name. A fresh worker doesn't have that temporary table. Being able to serialize an operator doesn't supply its missing inputs.

For this first slice, I admit constant projections and integer range scans, with a small set of expressions and scalar types. Local table scans, parameters, arbitrary functions, and unsupported operators are rejected. Checks run before optimization too: SELECT * FROM local_numbers WHERE false must still reveal its catalog dependency even if optimization would remove the scan.

The worker checks the restored plan again. For range, that includes checking the bound function against DuckDB's built-in implementation. A familiar function name alone isn't enough to establish what code it calls.

This is intentionally a small starting point. Parquet fragments will need an explicit agreement about files, schemas, and assigned row groups. We'll add that agreement when we introduce those inputs.

The plan needs an envelope

Plan bytes tell the engine what to do. They don't tell the worker who owns the request or whether it has already accepted it.

Our envelope carries:

Rechecking identity matters because a worker can restart between yesterday's hello and today's registration. Our current record compares reported versions and source revision. A stronger build manifest remains a gate before enabling remote result execution.

The outer decoder also needs limits before it allocates memory. I started with a one-MiB envelope, at most 64 output columns, and bounded string fields. A length prefix is a claim made by the sender; the decoder must compare it with both its limit and the bytes actually remaining.

Here's a standalone Python 3 example of that check. The payload is an opaque placeholder, so this exercises framing without decoding a DuckDB plan. Save it as frame_check.py and run python3 frame_check.py:

import struct

LIMIT = 1024 * 1024


def decode_plan_field(message):
    if len(message) < 4:
        raise ValueError("missing length")
    length, = struct.unpack_from("<I", message)
    if length > LIMIT or length != len(message) - 4:
        raise ValueError("invalid length or trailing bytes")
    return message[4:]


plan = b"opaque plan bytes"
message = struct.pack("<I", len(plan)) + plan
assert decode_plan_field(message) == plan

for invalid in (message[:-1], message + b"x", struct.pack("<I", LIMIT + 1)):
    try:
        decode_plan_field(invalid)
    except ValueError:
        continue
    raise AssertionError("invalid message accepted")

print("round trip and malformed lengths checked")

Our actual envelope has several fields, so each read advances a cursor and the final read must land exactly at the message's end. These outer checks don't turn DuckDB's internal deserializer into a sandbox. This experiment still assumes cooperative processes using the same build and listens only on loopback.

Registration creates a lifetime

Once registration succeeds, the worker has state to own. A small registry makes the transitions explicit:

The claim is checked and recorded under one lock. Two callers racing for the same task can't both change it from registered to running.

Cancellation also leaves a record when the task hasn't arrived yet. Otherwise, a delayed registration could resurrect work the coordinator already abandoned. That record is often called a tombstone.

For now, admission accepts deadlines within the next minute. Registry operations remove expired registrations and terminal records; capacity is capped at 1,024 identities and 16 MiB of retained plan bytes. A running task remains owned until its consumer finishes. The coordinator must use fresh query identities, and this in-memory registry provides no recovery guarantee across worker restarts.

What the round trip established

I ran the native fragment through a separate DuckDB database and compared every value and output type with the local query. The checks cover the 5,000-row example, empty input, typed NULLs, and representative scalar values. I also checked truncated plans, wrong schemas, conflicting registrations, and two concurrent claims.

A separate worker process now accepts valid registrations and cancellation messages, and rejects malformed or incompatible envelopes. Registration stops after validation; it doesn't start execution. The native result-fetch method remains disabled while we finish the compatibility and stream-lifetime work.

We now have a fragment that survives reconstruction and a task identity with defined ownership. Tomorrow, let's connect the claim to a Flight result stream and carry the first remote SELECT all the way back to the caller.