Fix option-4 LIST_ALL (empty-prefix S3 LIST) timeout / HTTP 500 on large containers

[zichengl]

Summary

Under listNamedBlobsSqlOption=4, an empty-prefix S3 ListObjects (which #3265 collapses to a null prefix and routes to LIST_ALL) used a window-function query — MAX(version) OVER (PARTITION BY blob_name) inside a derived table. MySQL materializes that derived table in full before the outer LIMIT, so it scans the entire container and cannot early-terminate. On large containers it runs past a short socket timeout, throws Communications link failure, and returns HTTP 500.

Because the empty-prefix path is the only way an S3 client reaches LIST_ALL, this blocks rolling option 4 to any fabric with large containers — an empty-prefix LIST there would 500.

Fix

• Replace the option-4 LIST_ALL window query with a correlated-subquery form (the LIST_WITH_PREFIX option-3 shape, minus the prefix predicate). The outer scans candidates in PRIMARY KEY (blob_name) order and keeps only version = MAX(version); the deleted_ts predicate sits on the outer candidate, so a soft-deleted latest version hides the blob entirely (no older-version leak) — preserving option-4 semantics. With the PK already blob_name-ordered, ORDER BY blob_name LIMIT N early-terminates after N matches instead of materializing the whole container. Options 2/3 LIST_ALL are unchanged.
• run_list_v2 now selects the null-prefix binder by option (2/3 → 5 params, 4 → new constructListAllQueryV4 → 7 params) so the statement and binder stay in lockstep.
• Add a default-off mysql.named.blob.list.query.timeout.seconds knob applied via Statement.setQueryTimeout, so a residual slow LIST surfaces a clean SQLException rather than tearing the connection into a 500. No-op for existing deployments.

This PR also carries the earlier follow-up commit that added null-prefix LIST_ALL test coverage (the regression guards this fix relies on).

Testing Done

Against MySQL 8.0:

• ./gradlew :ambry-named-mysql:intTest --tests "*MySqlNamedBlobDbListOperationIntegrationTest" → 54 tests, 46 passed, 0 failed, 8 skipped (the 8 are option-4-only invariants that Assume-skip on options 2/3). New: testListAllNullPrefixPagination (options 2/3/4) and testListAllNullPrefixReturnsLatestVersionUnderOption4; the existing testListAllNullPrefixHidesDeletedLatestUnderOption4 stays green under the new option-4 LIST_ALL.
• ./gradlew :ambry-frontend:intTest --tests "*S3MySqlNamedBlobListIntegrationTest" --tests "*S3IntegrationTest" → 4 passed, 0 failed. The new S3MySqlNamedBlobListIntegrationTest exercises the empty-prefix LIST end-to-end (S3 SDK → Netty → S3ListHandler → parseS3 → NamedBlobListHandler → MySqlNamedBlobDb.list null-prefix → option-4 LIST_ALL) against real MySQL and asserts 200 — the stitched coverage previously deferred. Existing S3IntegrationTest still passes after the buildFrontendVProps refactor.

Risk / rollout

• Read-path only; no schema, write-path, or default-config change (option default stays 2; the new timeout knob defaults to off).
• Backward compatible: options 2/3 untouched; option-4 LIST_ALL keeps the same hide-latest-deleted semantic, just via a streaming plan.

🤖 Generated with Claude Code

[codecov-commenter]

Codecov Report

❌ Patch coverage is 77.77778% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 50.82%. Comparing base (52ba813) to head (d986855).
⚠️ Report is 399 commits behind head on master.

Files with missing lines	Patch %	Lines
.../java/com/github/ambry/named/MySqlNamedBlobDb.java	75.00%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##             master    #3272       +/-   ##
=============================================
- Coverage     64.24%   50.82%   -13.43%     
+ Complexity    10398     8687     -1711     
=============================================
  Files           840      938       +98     
  Lines         71755    80339     +8584     
  Branches       8611     9666     +1055     
=============================================
- Hits          46099    40830     -5269     
- Misses        23004    36115    +13111     
- Partials       2652     3394      +742     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.