Skip to content

Commit 45cc93f

Browse files
gunnarmorlinggunnarmorling
committed
#109 Further trimming down README.md
1 parent dd61ea0 commit 45cc93f

4 files changed

Lines changed: 344 additions & 333 deletions

File tree

‎NATIVE_BUILD.md‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# Native CLI Build Details
2+
3+
## How the native build works
4+
5+
The CLI module uses [Quarkus](https://quarkus.io/) with `quarkus-picocli` and GraalVM/Mandrel native image. Several non-obvious pieces are required to make all compression codecs work correctly in a native binary.
6+
7+
### Compression codec native libraries
8+
9+
All compression codecs (Snappy, ZSTD, LZ4, Brotli) ship their native code as JNI libraries inside their JARs. In a standard JVM application, each library extracts itself from the JAR at runtime via `Class.getResourceAsStream()`. This extraction mechanism does not work in a GraalVM native image.
10+
11+
The solution differs by codec:
12+
13+
- **ZSTD, Snappy, LZ4** — Native libraries are unpacked from their JARs during the Maven `prepare-package` phase (`maven-dependency-plugin`) and bundled in a `lib/` directory alongside the binary. At startup, `NativeImageStartup` fires a Quarkus `StartupEvent` which calls `NativeLibraryLoader` to load each library via `System.load(absolutePath)` before any decompression occurs. For ZSTD, `zstd-jni`'s `Native.assumeLoaded()` is also called to prevent the library's own loader from attempting a duplicate load. Snappy is handled the same way — its loader may have already run at image build time (and failed), so directly calling `System.load()` at runtime bypasses its cached failure state entirely.
14+
15+
- **Brotli** — `brotli4j`'s loader (`Brotli4jLoader.ensureAvailability()`) is invoked explicitly at decompression time rather than in a static initializer, so it never runs at build time. Its loading strategy — extracting a classpath resource to a temp file and loading that — works in native images provided the resource is embedded in the binary. The `resource-config.json` under `cli/src/main/resources/META-INF/native-image/` instructs GraalVM to embed the brotli native libraries as image resources.
16+
17+
- **libdeflate (GZIP acceleration)** — libdeflate uses the Java 22+ Foreign Function & Memory (FFM) API, which relies on runtime downcall handles that cannot be created inside a native image. `LibdeflateLoader` detects the native image context via the `org.graalvm.nativeimage.imagecode` system property and returns `isAvailable() = false`, dead-code-eliminating the entire FFM path. The `--initialize-at-build-time` directive in `core`'s `native-image.properties` ensures GraalVM constant-folds this check at image build time.
18+
19+
### Build arguments (application.properties)
20+
21+
| Argument | Reason |
22+
|---|---|
23+
| `-march=compatibility` | Produces a binary targeting a generic x86\_64/arm64 baseline rather than the build machine's specific CPU generation. Without this, the binary may crash with `SIGILL` on older hardware. |
24+
| `--gc=serial` | Replaces the default G1 garbage collector with the serial GC, removing GC infrastructure code from the binary. Appropriate for a short-lived CLI process and meaningfully reduces binary size. |
25+
| `-J--enable-native-access=ALL-UNNAMED` | Passed to the JVM _running the Mandrel build process_ (not the native image itself). Required because GraalVM's image builder uses native access internally on JDK 21+. |
26+
| `--initialize-at-run-time=...YamlConfiguration` | Prevents log4j's YAML configuration class from initializing at image build time, where it would attempt to load SnakeYAML and fail. |
27+
28+
### Logging dependencies
29+
30+
`netty-buffer` (an optional dependency of `brotli4j`) is declared explicitly at compile scope so that GraalVM can resolve the `ByteBufUtil` reference in `brotli4j`'s `DirectDecompress` class during image analysis.
31+
32+
## Manual tests of the native CLI binary
33+
34+
1. Start S3Mock and set environment
35+
36+
```bash
37+
docker run -d --name s3mock -p 9090:9090 adobe/s3mock
38+
39+
export AWS_ENDPOINT_URL=http://localhost:9090
40+
export AWS_ACCESS_KEY_ID=foo
41+
export AWS_SECRET_ACCESS_KEY=bar
42+
export AWS_REGION=us-east-1
43+
export AWS_PATH_STYLE=true
44+
```
45+
46+
2. Create bucket and upload with curl
47+
48+
```bash
49+
curl -X PUT http://localhost:9090/test-bucket
50+
51+
curl -X PUT --data-binary @performance-testing/test-data-setup/target/tlc-trip-record-data/yellow_tripdata_2025-01.parquet \
52+
http://localhost:9090/test-bucket/yellow_tripdata_2025-01.parquet
53+
```
54+
55+
3. Run hardwood CLI
56+
57+
```bash
58+
cli/target/hardwood-cli-early-access-macos-aarch64/bin/hardwood info -f s3://test-bucket/yellow_tripdata_2025-01.parquet
59+
```

‎PERFORMANCE.md‎

Lines changed: 248 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,248 @@
1+
# Performance
2+
3+
## Benchmark Results
4+
5+
### Flat Files
6+
7+
These are the results from parsing files of the NYC Yellow Taxi Trip data set (subset 2016-01 to 2025-11, ~9.2GB overall, ~650M rows),
8+
running on a Macbook Pro M3 Max.
9+
The test (`FlatPerformanceTest`) parses all files and adds up the values of three columns (out of 20).
10+
The results shown are for:
11+
12+
* The row reader API, using indexed access (mapping field names to indexes once upfront)
13+
* The columnar reader API, using indexed access
14+
15+
```
16+
====================================================================================================
17+
PERFORMANCE TEST RESULTS
18+
====================================================================================================
19+
20+
Environment:
21+
CPU cores: 16
22+
Java version: 25
23+
OS: Mac OS X aarch64
24+
25+
Data:
26+
Files processed: 119
27+
Total rows: 651,209,003
28+
Total size: 9,241.1 MB
29+
Runs per contender: 5
30+
31+
Correctness Verification:
32+
passenger_count trip_distance fare_amount
33+
Hardwood (multifile indexed) 972,078,547 2,701,223,013.48 9,166,943,759.83
34+
Hardwood (column reader multifile) 972,078,547 2,701,223,013.48 9,166,943,759.83
35+
36+
Performance (all runs):
37+
Contender Time (s) Records/sec Records/sec/core MB/sec
38+
-----------------------------------------------------------------------------------------------
39+
Hardwood (multifile indexed) [1] 2.75 236,975,620 14,810,976 3362.8
40+
Hardwood (multifile indexed) [2] 2.78 234,669,911 14,666,869 3330.1
41+
Hardwood (multifile indexed) [3] 2.70 240,831,732 15,051,983 3417.6
42+
Hardwood (multifile indexed) [4] 2.70 240,831,732 15,051,983 3417.6
43+
Hardwood (multifile indexed) [5] 2.68 242,897,800 15,181,113 3446.9
44+
Hardwood (multifile indexed) [AVG] 2.72 239,239,163 14,952,448 3395.0
45+
min: 2.68s, max: 2.78s, spread: 0.09s
46+
47+
Hardwood (column reader multifile) [1] 1.30 502,476,083 31,404,755 7130.5
48+
Hardwood (column reader multifile) [2] 1.11 584,568,225 36,535,514 8295.4
49+
Hardwood (column reader multifile) [3] 1.06 614,348,116 38,396,757 8718.0
50+
Hardwood (column reader multifile) [4] 1.06 616,091,772 38,505,736 8742.8
51+
Hardwood (column reader multifile) [5] 1.08 603,530,123 37,720,633 8564.5
52+
Hardwood (column reader multifile) [AVG] 1.12 580,917,933 36,307,371 8243.6
53+
min: 1.06s, max: 1.30s, spread: 0.24s
54+
55+
====================================================================================================
56+
```
57+
58+
### Nested Files
59+
60+
These are the results from parsing a file with points of interest from the Overture Maps data set
61+
(~900 MB, ~9M rows), running on a Macbook Pro M3 Max.
62+
The test (`NestedPerformanceTest`) parses all columns of the file and determines min/max values, max array lengths, etc.
63+
As above, the results shown are for the row reader API and the columnar API with indexed access.
64+
65+
```
66+
====================================================================================================
67+
NESTED SCHEMA PERFORMANCE TEST RESULTS
68+
====================================================================================================
69+
70+
Environment:
71+
CPU cores: 16
72+
Java version: 25
73+
OS: Mac OS X aarch64
74+
75+
Data:
76+
Total rows: 9,152,540
77+
File size: 882.2 MB
78+
Runs per contender: 5
79+
80+
Correctness Verification:
81+
min_ver max_ver rows websites sources addresses
82+
Hardwood (indexed) 1 9 9,152,540 3,687,576 18,305,080 9,152,540
83+
Hardwood (columnar) 1 9 9,152,540 3,687,576 18,305,080 9,152,540
84+
85+
Performance (all runs):
86+
Contender Time (s) Records/sec Records/sec/core MB/sec
87+
-----------------------------------------------------------------------------------------------
88+
Hardwood (indexed) [1] 2.22 4,120,910 257,557 397.2
89+
Hardwood (indexed) [2] 1.92 4,759,511 297,469 458.8
90+
Hardwood (indexed) [3] 1.89 4,855,459 303,466 468.0
91+
Hardwood (indexed) [4] 1.88 4,876,153 304,760 470.0
92+
Hardwood (indexed) [5] 1.88 4,858,036 303,627 468.3
93+
Hardwood (indexed) [AVG] 1.96 4,674,433 292,152 450.6
94+
min: 1.88s, max: 2.22s, spread: 0.34s
95+
96+
Hardwood (columnar) [1] 1.34 6,830,254 426,891 658.4
97+
Hardwood (columnar) [2] 1.32 6,918,020 432,376 666.8
98+
Hardwood (columnar) [3] 1.24 7,363,266 460,204 709.8
99+
Hardwood (columnar) [4] 1.24 7,404,968 462,810 713.8
100+
Hardwood (columnar) [5] 1.22 7,477,565 467,348 720.8
101+
Hardwood (columnar) [AVG] 1.27 7,189,741 449,359 693.0
102+
min: 1.22s, max: 1.34s, spread: 0.12s
103+
104+
====================================================================================================
105+
```
106+
107+
## Running Performance Tests
108+
109+
The performance testing modules are not included in the default build. Enable them with `-Pperformance-test`.
110+
111+
### End-to-End Performance Tests
112+
113+
There are two end-to-end performance tests: one for flat schemas (NYC Yellow Taxi Trip data) and one for nested schemas (Overture Maps POI data). Test data is downloaded automatically on the first run.
114+
115+
```shell
116+
./mvnw test -Pperformance-test
117+
```
118+
119+
**Flat schema test** (`FlatPerformanceTest`) — reads ~9GB of taxi trip data (2016-2025, ~650M rows) and sums three columns.
120+
121+
| Property | Default | Description |
122+
|----------|---------|-------------|
123+
| `perf.contenders` | `HARDWOOD_MULTIFILE_INDEXED` | Comma-separated list of contenders, or `all` |
124+
| `perf.start` | `2016-01` | Start year-month for data range |
125+
| `perf.end` | `2025-11` | End year-month for data range |
126+
| `perf.runs` | `10` | Number of timed runs per contender |
127+
128+
Available contenders: `HARDWOOD_INDEXED`, `HARDWOOD_NAMED`, `HARDWOOD_PROJECTION`, `HARDWOOD_MULTIFILE_INDEXED`, `HARDWOOD_MULTIFILE_NAMED`, `HARDWOOD_COLUMN_READER`, `HARDWOOD_COLUMN_READER_MULTIFILE`, `PARQUET_JAVA_INDEXED`, `PARQUET_JAVA_NAMED`.
129+
130+
**Nested schema test** (`NestedPerformanceTest`) — reads ~900MB of Overture Maps POI data (~9M rows) with deeply nested columns.
131+
132+
| Property | Default | Description |
133+
|----------|---------|-------------|
134+
| `perf.contenders` | `HARDWOOD_NAMED` | Comma-separated list of contenders, or `all` |
135+
| `perf.runs` | `5` | Number of timed runs per contender |
136+
137+
Available contenders: `HARDWOOD_INDEXED`, `HARDWOOD_NAMED`, `HARDWOOD_COLUMNAR`, `PARQUET_JAVA`.
138+
139+
**Examples:**
140+
141+
```shell
142+
# Run all contenders for the flat test, limited to 2025 data
143+
./mvnw test -Pperformance-test -Dtest=FlatPerformanceTest -Dperf.contenders=all -Dperf.start=2025-01
144+
145+
# Compare multifile indexed vs named access
146+
./mvnw test -Pperformance-test -Dperf.contenders=HARDWOOD_MULTIFILE_INDEXED,HARDWOOD_MULTIFILE_NAMED
147+
148+
# Run nested test only
149+
./mvnw test -Pperformance-test -Dtest=NestedPerformanceTest -Dperf.contenders=all
150+
```
151+
152+
### PyArrow Comparison Tests
153+
154+
Python counterparts of the Java performance tests using PyArrow, for cross-platform comparison.
155+
These scripts require a Python environment with PyArrow installed (use the `.venv` venv).
156+
157+
**Flat schema** (`flat_performance_test.py`) — counterpart of `FlatPerformanceTest.java`:
158+
159+
```shell
160+
cd performance-testing/end-to-end
161+
162+
# Run all contenders (single-threaded and multi-threaded), 5 runs each
163+
python flat_performance_test.py
164+
165+
# Single-threaded only
166+
python flat_performance_test.py -c single_threaded
167+
168+
# Multi-threaded, 10 runs
169+
python flat_performance_test.py -c multi_threaded -r 10
170+
```
171+
172+
**Nested schema** (`nested_performance_test.py`) — counterpart of `NestedPerformanceTest.java`:
173+
174+
```shell
175+
cd performance-testing/end-to-end
176+
177+
# Run all contenders, 5 runs each
178+
python nested_performance_test.py
179+
180+
# Single-threaded only, 3 runs
181+
python nested_performance_test.py -c single_threaded -r 3
182+
```
183+
184+
**Options:**
185+
186+
| Flag | Default | Description |
187+
|------|---------|-------------|
188+
| `-c`, `--contenders` | `all` | Contenders to run: `single_threaded`, `multi_threaded`, or `all` |
189+
| `-r`, `--runs` | `5` | Number of timed runs per contender |
190+
191+
**Notes on comparability:**
192+
193+
- The flat test uses column projection (reads only the 3 summed columns), matching the Hardwood projection and column-reader contenders. The parquet-java contenders in `FlatPerformanceTest.java` read all columns without projection, so direct comparison against parquet-java is not apples-to-apples.
194+
- PyArrow uses vectorized columnar operations (C++ engine) rather than row-by-row iteration.
195+
- The `single_threaded` contender (`use_threads=False`) is most comparable to single-threaded parquet-java; `multi_threaded` is comparable to Hardwood's parallel reading.
196+
197+
### JMH Micro-Benchmarks
198+
199+
For detailed micro-benchmarks, build the JMH benchmark JAR and run it directly:
200+
201+
```shell
202+
# Build the benchmark JAR
203+
./mvnw package -Pperformance-test -pl performance-testing/micro-benchmarks -am -DskipTests
204+
205+
# Run all benchmarks (with Vector API for SIMD support)
206+
java --add-modules jdk.incubator.vector \
207+
-jar performance-testing/micro-benchmarks/target/benchmarks.jar \
208+
-p dataDir=performance-testing/test-data-setup/target/tlc-trip-record-data
209+
210+
# Run a specific benchmark
211+
java --add-modules jdk.incubator.vector \
212+
-jar performance-testing/micro-benchmarks/target/benchmarks.jar \
213+
"PageHandlingBenchmark.decodePages" \
214+
-p dataDir=performance-testing/test-data-setup/target/tlc-trip-record-data
215+
216+
# Run SIMD benchmark comparing scalar vs vectorized operations
217+
java --add-modules jdk.incubator.vector \
218+
-jar performance-testing/micro-benchmarks/target/benchmarks.jar SimdBenchmark \
219+
-p size=1024,8192,65536 -p implementation=scalar,auto
220+
221+
# List available benchmarks
222+
java --add-modules jdk.incubator.vector \
223+
-jar performance-testing/micro-benchmarks/target/benchmarks.jar -l
224+
```
225+
226+
**Available benchmarks:**
227+
228+
| Benchmark | Description |
229+
|-----------|-------------|
230+
| `MemoryMapBenchmark.memoryMapToByteArray` | Memory map a file and copy to byte array |
231+
| `PageHandlingBenchmark.a_decompressPages` | Scan and decompress all pages |
232+
| `PageHandlingBenchmark.b_decodePages` | Scan, decompress, and decode all pages |
233+
| `PipelineBenchmark.a_assembleColumns` | Synchronous page decoding + column assembly |
234+
| `PipelineBenchmark.b_consumeRows` | Full pipeline with row-oriented access |
235+
| `SimdBenchmark.*` | SIMD operations (countNonNulls, markNulls, dictionary, bit unpacking) |
236+
237+
**JMH options:**
238+
239+
| Option | Description |
240+
|--------|-------------|
241+
| `-wi <n>` | Number of warmup iterations (default: 3) |
242+
| `-i <n>` | Number of measurement iterations (default: 5) |
243+
| `-f <n>` | Number of forks (default: 2) |
244+
| `-p param=value` | Set benchmark parameter |
245+
| `-l` | List available benchmarks |
246+
| `-h` | Show help |
247+
248+
**Note:** The taxi data files use GZIP compression (2016-01 to 2023-01) and ZSTD compression (2023-02 onwards). The default benchmark file is `yellow_tripdata_2025-05.parquet` (ZSTD, 75MB).

0 commit comments

Comments
 (0)