iT邦幫忙

鐵人檔案

2026 iThome 鐵人賽
回列表
Software Development

Build a distributed DuckDB 系列

Build a distributed DuckDB

參賽天數 10 天 | 共 10 篇文章 | 0 人訂閱 訂閱系列文 RSS系列文 |團隊2N1P
DAY 1

Day 1: Why distribute DuckDB without turning it into Spark?

Let's look at this from two angles: flexibility and scalability. First, flexibil...

2026-09-07 ‧ 由 Peter Lee 分享
DAY 2

Day2: The complete query journey

Yesterday, I talked about why I wanted to try building a distributed DuckDB. Tod...

2026-09-08 ‧ 由 Peter Lee 分享
DAY 3

Day3: Distribution must be optional

Yesterday, we followed a query from SQL to worker tasks and back. Now let's put...

2026-09-09 ‧ 由 Peter Lee 分享
DAY 4

Day4: Build compatibility is part of the protocol

Yesterday, we gave the planner permission to consider remote execution and a lis...

2026-09-10 ‧ 由 Peter Lee 分享
DAY 5

Day5: Why Arrow Flight has two planes

Yesterday, we worked out what to compare before sending a plan to a worker. Toda...

2026-09-11 ‧ 由 Peter Lee 分享
DAY 6

Day6: Shipping a DuckDB plan safely

Yesterday, we gave the worker a way to accept control messages. Today, let's giv...

2026-09-12 ‧ 由 Peter Lee 分享
DAY 7

Day 7: The first remote SELECT

Yesterday, we reconstructed a logical plan in a fresh DuckDB connection. That le...

2026-09-13 ‧ 由 Peter Lee 分享
DAY 8

Day 8: A file is too large to be a work unit

Yesterday, we got rows back from a query running in another process. Both sides...

2026-09-14 ‧ 由 Peter Lee 分享
DAY 9

Day9: Packing work by compressed bytes

Yesterday, we found the row groups inside a Parquet file and read them separatel...

2026-09-15 ‧ 由 Peter Lee 分享
DAY 10

Day 10: Building duckd_scan

Yesterday, we turned Parquet row groups into assignments. Now I'd like to execut...

2026-09-16 ‧ 由 Peter Lee 分享