feat: batch envelope on POST /track, deprecate /track/batch + sync upstream/main

[mraj602-tohands]

Why

Upstream review on Openpanel-dev/openpanel#377 asked for batch ingestion to ride the existing /track endpoint as { "type": "batch", "payload": [event, ...] } instead of a separate /track/batch route. This PR adopts that exact contract in our fork now, and also syncs upstream/main, so:

• our clients integrate against the upstream-future contract once and never change again
• no ALB rewrite layer is needed — both transports terminate in the same controller
• our worker-side historical-event/session handling stays intact until upstream's session rework lands
• develop is current with upstream (14 commits: kafka client revert to kafkajs, delete account/org, access-rights fixes, deletion cron rework, invite fix, new prisma migration + CH MVs)

What

• POST /track accepts the batch envelope — { "type": "batch", "payload": [...] } (up to 2000 events, 10 MB) dispatched through the same processBatch pipeline as the legacy route: shared salts/geo per request, per-item validation, bounded concurrency (50), 202 { accepted, rejected[] } with per-index errors
• POST /track/batch kept as a deprecated alias ({ "events": [...] }, marked deprecated in OpenAPI) for already-shipped clients — remove once traffic hits zero
• Body-hash dedup skips batch envelopes (in duplicateHook, next to the existing replay skip) — offline-first SDKs retry whole batches
• Merged upstream/main (6d9445c) — only textual conflict was packages/queue/src/queues.ts, resolved as upstream's new version minus the isTimestampFromThePast field we removed
• FORK-PATCHES.md documents permanent fork differences (CI workflow) and every temporary divergence (worker session handling, eventMs bucketing, 5-day floor, the alias) plus the sync playbook
• Fork-specific behavior unchanged: 5-day historical window, deterministic eventMs session bucketing, worker session handling

Compatibility review of the upstream sync

• Upstream's 14 commits touch none of our patched files (events.incoming-event.ts, track.controller.ts, ids.ts, event.controller.ts, session-handler)
• Kafka revert (@platformatic/kafka → kafkajs): the call surface used by the track controller (shouldUseKafka, produceIncomingEvent) is signature-identical; we never patched kafka.ts
• Upstream's new events.kafka-consumer.ts calls our incomingEvent(payload) — our handler derives live-vs-historical internally, so kafka-path events get the same session semantics without the removed isTimestampFromThePast flag
• New prisma migration (add_member_org_user_unique) — run migrate:deploy on rollout; new CH MVs come from code-migrations/3-init-ch

Tests (run on the merged tree)

• apps/api: 73/73 passing — batch suite parametrized to run against both transports so the alias can't drift, plus envelope-only regressions (single-event 200, single alias 400, nested-batch rejection, internal-error rejections, all dispatch arms)
• apps/worker: 7/7 passing
• Typecheck clean (api + worker); 100% diff-line coverage on the three changed api files

Future upstream merge

When upstream lands Openpanel-dev#377 + their session rework: take upstream's side wholesale in the files listed under "Temporary patches" in FORK-PATCHES.md, re-run the batch + worker suites, then delete the alias. Remaining fork delta after that: the deprecated /track/batch endpoint (+ its tests), FORK-PATCHES.md, and our CI workflow.

Notes for reviewers

• The 10 MB bodyLimit on /track also applies to single-event bodies (previously 1 MB default) — both shapes share the route; commented in code

🤖 Generated with Claude Code

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR introduces a canonical batch envelope format ({ type: 'batch', payload: [...] }) on the /track endpoint while maintaining backward compatibility with the deprecated /track/batch route. The implementation includes schema validation, router configuration, controller dispatching, deduplication bypass logic, comprehensive dual-transport integration tests, and documentation of temporary fork patches.

Changes

Batch envelope contract and implementation

Layer / File(s)	Summary
Validation schema and batch type definitions packages/validation/src/track.validation.ts	Introduces shared zBatchEvents schema and canonical batch handler envelope zTrackBatchHandlerPayload with ITrackBatchHandlerPayload type; refactors legacy zTrackBatchBody to reuse the shared events schema.
Router configuration for batch endpoints apps/api/src/routes/track.router.ts	Expands POST /track route to accept both single-event and batch payloads via union schema, applies 10MB body limit, introduces shared 202 response schema, and marks POST /track/batch as deprecated with proper OpenAPI metadata.
Controller batch envelope handling and dispatch apps/api/src/controllers/track.controller.ts	Extends main handler to accept canonical batch envelopes and route to processBatch immediately, refactors deprecated batchHandler as a wrapper forwarding legacy events, loosens buildContext typing, updates batch documentation, and refactors result aggregation.
Deduplication hook batch/replay bypass apps/api/src/hooks/duplicate.hook.ts	Adds TrackBody union type combining single-track and batch payloads, implements conditional deduplication bypass for 'batch' and 'replay' types, and updates type-guard predicate.
Dual-transport integration test refactoring apps/api/src/routes/track-batch.router.test.ts	Refactors test suite with parameterized TRANSPORTS table and postRaw helper to validate both endpoints identically (auth/envelope validation, happy-path acceptance, per-item validation, sessionId bucketing, timestamp windowing, geo lookup, boundary handling); adds envelope-only tests asserting single-event batch acceptance and nested-batch rejection.
Fork patches documentation FORK-PATCHES.md	Documents stable client batch contract aligned with upstream PR #377 and enumerates two temporary fork patches (worker-side historical-event/session handling with deterministic bucketing and deprecated /track/batch alias) with resolution instructions and testing requirements for upstream sync.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 A batch of tracks now travels clean,
Unified through the /track machine,
With envelope tucked and payload bright,
Two transports dance in harmony's light,
Legacy whispers, but consensus shines on.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 37.50% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main changes: accepting a batch envelope on POST /track and deprecating the separate /track/batch endpoint, with alignment to upstream.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/track-batch-envelope

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

apps/api/src/controllers/track.controller.ts (1)

613-621: ⚡ Quick win

Project rejected items to the public response shape

Lines 613-621 push full internal rejected results (including status) into rejected[]. Please map to { index, reason, error } explicitly so the API contract doesn’t depend on serializer stripping behavior.

Suggested patch

-  const rejected: Extract<BatchItemResult, { status: 'rejected' }>[] = [];
+  const rejected: Array<{
+    index: number;
+    reason: 'validation' | 'internal';
+    error: string;
+  }> = [];
   for (const result of results) {
     if (result.status === 'accepted') {
       accepted += 1;
     } else {
-      rejected.push(result);
+      rejected.push({
+        index: result.index,
+        reason: result.reason,
+        error: result.error,
+      });
     }
   }

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/api/src/controllers/track.controller.ts` around lines 613 - 621, The
loop collects full internal rejected BatchItemResult objects into rejected[],
leaking internal fields like status; instead, map each rejected result to the
public shape { index, reason, error } (e.g., use result.index, result.reason,
result.error) and push that object into rejected[]; also update the rejected
variable's type to reflect the public response shape rather than
Extract<BatchItemResult,...> so the API contract is explicit.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@apps/api/src/controllers/track.controller.ts`:
- Around line 613-621: The loop collects full internal rejected BatchItemResult
objects into rejected[], leaking internal fields like status; instead, map each
rejected result to the public shape { index, reason, error } (e.g., use
result.index, result.reason, result.error) and push that object into rejected[];
also update the rejected variable's type to reflect the public response shape
rather than Extract<BatchItemResult,...> so the API contract is explicit.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1382b325-ad95-4782-a74c-2a1060c7a13e

📥 Commits

Reviewing files that changed from the base of the PR and between c6f5d9f and 8e575df.

📒 Files selected for processing (6)

• FORK-PATCHES.md
• apps/api/src/controllers/track.controller.ts
• apps/api/src/hooks/duplicate.hook.ts
• apps/api/src/routes/track-batch.router.test.ts
• apps/api/src/routes/track.router.ts
• packages/validation/src/track.validation.ts