TPC-H SF 0.1 vs DuckDB through P2d, P2e
Five TPC-H queries at SF 0.1: answers equal DuckDB's but 137–1 408× slower at first; projected scans and vectorized expressions got them to 37–255× that day.
Run details
| Date | |
|---|---|
| Commit | 2d70123 (first table), 2837a9d (after P2d), 535509b (after P2e), per docs/bench/results/tpch-runs.jsonl |
| Host | sttiz, shared production VPS under the uf-remote caps: 3 CPUs, 6 GB. |
| Tool | scripts/tpch-run 0.1 3 (crate uf-tpch). Data: DuckDB v1.1.3 CALL dbgen(sf=0.1), lineitem 600 572 rows, loaded into one UltraFabric engine directory (one tenant, TPC-H primary keys) in 5.1 s. UltraFabric runs each query as ufd does: verify, optimize, re-verify, execute (median of 3, including planning). DuckDB time is per query excluding process start (median of 3). Answers compared with DuckDB's: decimals exactly, AVG within relative 1e-9. |
| Raw results | docs/bench/results/tpch-runs.jsonl |
| Report | docs/bench/2026-10-21-tpch-sf0.1.md |
Indicative only. Measured on a shared development host with other services running, one node, data that fits in memory. These figures show direction, not performance to expect. The claims policy requires a dedicated host before any target is published.
The tables, as written
Transcribed from the report without rounding. Bracketed values are the previous measurement where the report gives one.
At the P2c gate
| Query | Rows | UltraFabric (s) | DuckDB 1 thread (s) | DuckDB 3 threads (s) | Ratio vs 1 thread | Answers |
|---|---|---|---|---|---|---|
| SCAN (count(*) lineitem, profile baseline) | 1 | 2.921 | 0.0002 | 0.0004 | — (DuckDB answers from metadata) | equal |
| Q1 | 4 | 4.278 | 0.0123 | 0.0058 | 348× | equal |
| Q3 | 10 | 3.547 | 0.0064 | 0.0052 | 557× | equal |
| Q5 | 5 | 3.545 | 0.0051 | 0.0071 | 693× | equal |
| Q6 | 1 | 3.269 | 0.0023 | 0.0010 | 1 408× | equal |
| Q10 | 20 | 3.536 | 0.0258 | 0.0216 | 137× | equal |
After P2d (projected scans, typed decimals), same host and data
| Query | Before P2d (s) | After P2d (s) | DuckDB 1 thread (s) | Ratio vs 1 thread | Answers |
|---|---|---|---|---|---|
| SCAN (lineitem, profile baseline) | 2.921 | 0.569 (5.1×) | — | — | equal |
| Q1 | 4.278 | 2.300 | 0.0120 | 192× | equal |
| Q3 | 3.547 | 1.016 | 0.0065 | 156× | equal |
| Q5 | 3.545 | 1.157 | 0.0056 | 209× | equal |
| Q6 | 3.269 | 1.258 | 0.0023 | 552× | equal |
| Q10 | 3.536 | 1.162 | 0.0231 | 50× | equal |
After P2e (vectorized expressions, batch aggregation, L11 above security chains)
| Query | Before P2d (s) | After P2d (s) | After P2e (s) | DuckDB 1 thread (s) | Ratio vs 1 thread | Answers |
|---|---|---|---|---|---|---|
| Q1 | 4.278 | 2.300 | 1.165 | 0.0126 | 92× | equal |
| Q3 | 3.547 | 1.016 | 0.681 | 0.0061 | 111× | equal |
| Q5 | 3.545 | 1.157 | 0.881 | 0.0056 | 156× | equal |
| Q6 | 3.269 | 1.258 | 0.537 | 0.0021 | 255× | equal |
| Q10 | 3.536 | 1.162 | 0.777 | 0.0208 | 37× | equal |
Seconds per query, log scale
UltraFabric at the P2c gate, after P2d, after P2e; DuckDB 1.1.3 on one thread
Q1, Q6 and Q10 across the day, seconds
Three phases of executor work on the same host and data
Before and during this run
- First table: the executor as it stood at the P2c gate. Planning is 1–8 ms and catalog load 0.04 ms per query.
- P2d (projected scans, typed decimals): the scan decodes only the columns a plan can observe, straight into column batches. Pruning needs a PROJECT above the security chain; decimals are typed i128 columns.
- P2e (vectorized expressions, batch aggregation, the L11 rule above security chains): removed value-by-value gathers, comparisons, decimal arithmetic and row-based aggregation, plus the per-value clone in the normal-form check.
What the numbers say
The report's own reading, including what went wrong and what is still open.
- All five answers equal DuckDB's; the 3× gate is not met. The profile and the plan are in docs/adr/004-p2-tpch-gate.md.
- After P2d the SCAN baseline read one column (the best case) and gained 5.1×; the queries read 3–7 columns and gained 1.9×–3.5×. What remained in Q1 and Q6 was filter and expression evaluation running per row.
- After P2e the five queries were 3.7×–6.1× faster than before P2d. What remained was dominated by reading and decoding the row store (SCAN baseline 0.40 s): the columnar-storage step, P3c.
- Correctness alongside: ten million differential plans with pruning, 0 failures; 9 of 9 mutants caught.
Files and tools
Report
docs/bench/2026-10-21-tpch-sf0.1.md in the repository.
Raw results
docs/bench/results/tpch-runs.jsonl
Access
The repository is private during development. Early-access teams receive the build, the benchmark scripts and the raw files.
See the measurements in context.
The performance page draws the latest figure from each area and states what is not measured yet.