TPC-H on columnar collections, SF 0.1
The same five TPC-H queries on row and columnar storage: columnar is 2.5–7.3× faster than row storage, 11–46× slower than single-threaded DuckDB, answers equal.
Run details
| Date | |
|---|---|
| Commit | a69b309 (clean tree). Earlier raw lines labelled 47d56fd came from an uncommitted working tree and are superseded. |
| Host | sttiz, shared production VPS under the uf-remote caps: 3 CPUs, 6 GB. |
| Tool | scripts/tpch-run 0.1 3 row and scripts/tpch-run 0.1 3 columnar (crate uf-tpch). The columnar load compacts, advances the GC horizon and compacts the LSM, as a steady-state system would. Same data and comparison method as the 21 October report. |
| Raw results | docs/bench/results/tpch-runs.jsonl |
| Report | docs/bench/2026-10-22-tpch-columnar.md |
Indicative only. Measured on a shared development host with other services running, one node, data that fits in memory. These figures show direction, not performance to expect. The claims policy requires a dedicated host before any target is published.
The tables, as written
Transcribed from the report without rounding. Bracketed values are the previous measurement where the report gives one.
Row storage vs columnar vs DuckDB 1.1.3 on one thread, seconds
| Query | Row storage (s) | Columnar (s) | Columnar gain | DuckDB 1 thread (s) | Columnar vs DuckDB | Answers |
|---|---|---|---|---|---|---|
| Q1 | 0.869 | 0.308 | 2.8× | 0.0128 | 24× | equal |
| Q3 | 0.710 | 0.166 | 4.3× | 0.0085 | 20× | equal |
| Q5 | 0.896 | 0.363 | 2.5× | 0.0078 | 46× | equal |
| Q6 | 0.535 | 0.074 | 7.3× | 0.0020 | 36× | equal |
| Q10 | 0.771 | 0.245 | 3.1× | 0.0229 | 11× | equal |
Seconds per query, log scale
Row storage, columnar storage and DuckDB on one thread; commit a69b309
Before and during this run
- Columnar collections (P3c): rows arrive as usual and are compacted into row groups whose columns are packed by type; a query reads only the columns it needs.
- Both columns include the P2e+ executor changes made in this phase (digit counting, typed group keys).
- Two things found while measuring went into spec §5.7: compaction markers moved out of the delta rows into their own subspace, because every scan walked them; and deleted row keys leave LSM tombstones until the GC horizon passes them, so the benchmark advances it after loading.
What the numbers say
The report's own reading, including what went wrong and what is still open.
- Before P2d the same queries took 3.3–4.3 s, so they are now 10×–44× faster in three phases, with answers always equal.
- In one profiling run (uf-tpch profile, not recorded in the jsonl), reading two columns of lineitem's row groups took about 15 ms, so the remaining gap is the executor rather than storage: per-row validity vectors, row-at-a-time hash joins and group keys, no SIMD, and a single thread.
- That is the next executor work on the way to the 3× goal (ADR-004). The gate is not met.
- Correctness alongside: a columnar-vs-row differential over at least 1 000 generated plans; ten million ufdiff plans, 0 failures; mutation pass 9 of 9; 354 workspace tests.
Files and tools
Report
docs/bench/2026-10-22-tpch-columnar.md in the repository.
Raw results
docs/bench/results/tpch-runs.jsonl
Access
The repository is private during development. Early-access teams receive the build, the benchmark scripts and the raw files.
See the measurements in context.
The performance page draws the latest figure from each area and states what is not measured yet.