Yesterday, we gave the planner permission to consider remote execution and a list of worker addresses. Suppose a worker answers at one of those addresses. Can we send it a plan now?

Well... it might be running a different DuckDB build. Or the same DuckDB with an older version of our extension. It could happily accept a network connection and still have no idea what our plan means.

This is where the build becomes part of the query's execution path.

What exactly has to be compatible?

There are three boundaries to think about:

Application process                      Worker process
DuckDB loads our extension                DuckDB loads our extension
        │                                         │
        └──────── Flight messages and data ───────┘
                                                  │
                                  Reconstruct and execute the plan

First, the extension must load into its local DuckDB process. C++ code depends on the engine interfaces it was built against. DuckDB's extension versioning documentation describes its version and platform requirements for C++ extensions.

Second, the processes need to exchange messages. That's where Flight, gRPC, and our own control protocol come in.

Third, the worker must understand the contents of the plan. An Arrow batch describes columns of data. Our serialized DuckDB plan describes operations, expressions, and bound functions. Being able to read the batch doesn't tell us whether the worker can execute those operations.

I initially compared three fields: the DuckDB version, its source revision, and our extension version. That's a useful first check. But calling the result “identical builds” is a stronger claim than those fields support. Two locally modified builds can still report the same version. A shared dev label tells us very little.

Package the transport as one dependency set

Before a handshake can help, the worker has to start successfully.

Flight brings gRPC and Protobuf dependencies. I'd resolve them together through one pinned dependency configuration. Arrow exposes an ArrowFlight CMake package and shared/static targets following its package naming convention. Using those imported targets carries build information that a handwritten list of library paths can miss. Arrow also requires consistent linking of Flight, gRPC, and Protobuf; mixing versions or static/shared variants can cause failures. Its C++ integration guide explains these constraints.

With shared libraries, deployment must include compatible runtime libraries where the loader can find them. Static linking incorporates library code into the artifact, but doesn't mean every system dependency disappears.

So I'd test the packaged extension in a fresh process outside the build environment, with distribution disabled. Then start the worker from the same package. That catches a different class of problem from compiling a binary that happens to find libraries on my development machine.

The package also needs a record of what produced it: engine revision, extension revision, build options, and dependency versions. Otherwise, the first question after a failure becomes “which binary did we copy over there?” I've had enough of that conversation.

Let the worker describe itself

For the first implementation, I'd use a strict compatibility policy. We can relax it later when tests establish which differences are safe.

The following is a proposed compatibility record, with illustrative values:

{
  "protocol_version": 1,
  "engine_build": "engine-build-A",
  "extension_build": "distributed-build-A",
  "capabilities": ["scan_fragment_v1"],
  "extensions": {"parquet": "parquet-build-A"}
}

The protocol version identifies the structure and meaning of our control messages. The two build identifiers describe the engine and distributed extension. Generate those identifiers from the build inputs, including local modifications and relevant options, rather than relying on a friendly release name. Keep the underlying manifest available for diagnostics.

Capabilities identify implemented operations. The extension inventory identifies the code needed by the fragment. A Parquet scan needs its reader available in the worker's execution context; a file somewhere on disk isn't enough.

We compare the required extensions against what the worker can use. An unrelated extra extension needn't cause rejection. Missing required code should.

This record is a conservative gate. Passing it still doesn't prove that serialization is correct, that the worker can access the data, or that two settings have equivalent effects on the query. We will need execution checks for those assumptions too.

Check before registering work

At execution startup, the coordinator asks each selected worker for its identity. Before registering any tasks, it checks the replies against the query's requirements:

Selected workers
    → request compatibility records with a deadline
    → validate required fields and supported protocol
    → compare build identifiers and required capabilities
    → verify required extensions
    → all accepted: register tasks

For our example, engine-build-B should produce a clear mismatch error. A missing engine_build is a malformed reply, not a wildcard. A worker that doesn't answer before the deadline produces a connection or timeout error. Those cases need different diagnostics even though all stop startup.

I'd include the endpoint, field, expected value, and received value in a mismatch message:

Worker grpc://localhost:8815 rejected:
engine_build expected engine-build-A, received engine-build-B
No tasks registered.

That last line is only justified if we check every selected worker before registration begins.

A successful handshake also shouldn't grant permanent permission to an address. The worker behind that address can restart with another build. Include the required identity in task registration, and have the worker validate it again before accepting the fragment. This keeps the check attached to the work it protects.

Inspect what is actually running

We can already inspect part of this information in an ordinary DuckDB SQL session. Run these queries in each process you want to compare:

PRAGMA version;

SELECT extension_name, loaded, installed, extension_version
FROM duckdb_extensions()
WHERE loaded
ORDER BY extension_name;

SELECT name, value
FROM duckdb_settings()
WHERE name IN ('threads', 'preserve_insertion_order')
ORDER BY name;

The first query returns the library version, source identifier, and codename. The second lists loaded extensions and their reported versions. The third reads effective settings without changing them. I ran all three locally; the engine reported a development version and a separate source identifier, which is a good reminder that the release label alone isn't enough.

Your values will depend on your installation. Compare the source identifiers and required extension rows across processes. An empty extension version is missing evidence; it shouldn't automatically count as compatible.

Settings need a little judgment too. Different thread counts can be fine. A setting that changes expression semantics may need to travel with the task or make the query ineligible. Yesterday's insertion-order gate belongs to that second kind of reasoning.

When we add our own diagnostic view, I'd expose the local build identity, effective distribution switch, and parsed worker endpoints without contacting workers. A separate explicit probe can report remote identities and errors. Reading configuration should remain possible when a worker is down.

Today's queries inspect local processes; they don't test the proposed handshake or the packaged worker. We now know what information that handshake needs, when to compare it, and what a rejection should explain. Next, let's put the exchange onto Flight and separate the messages that control work from the streams carrying its results.