-
Notifications
You must be signed in to change notification settings - Fork 0
Add ab-av1 worker websocket and shared CRF claiming #19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mjc
wants to merge
53
commits into
main
Choose a base branch
from
mjc/reenc-133-worker-websocket
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+5,639
−334
Open
Changes from 22 commits
Commits
Show all changes
53 commits
Select commit
Hold shift + click to select a range
dcff3ad
Add Serena project memories
mjc 85bd085
Add ab-av1 worker websocket handshake
mjc b0a911f
Extract ab-av1 worker protocol helpers
mjc 0c5738a
Document ab-av1 worker websocket contract
mjc 2a23378
Clarify worker socket uses main Phoenix endpoint
mjc e7438bb
Track websocket worker sessions
mjc 49581d6
Show worker websocket details on dashboard
mjc 8a7cad8
Derive worker token from secret key base
mjc 9ee25b7
Expire stale worker websocket sessions
mjc 0f11f03
Add pull-based worker assignment
mjc 803f59f
Use ETS for worker sessions
mjc 0f67861
Hide worker token on dashboard
mjc 31219d1
Claim CRF work before Broadway dispatch
mjc b1f402f
Address ab-av1 websocket review feedback
mjc 26019b6
Tighten worker websocket auth and sessions
mjc 8c530ff
Add CRF claim broadcast coverage
mjc 27c2e8f
Address worker websocket review feedback
mjc dac20fa
Tighten worker websocket protocol types
mjc 526d818
Add workers status live view
mjc 367a2ce
Wire workers into main navigation
mjc 9f12a0c
Tighten CRF claim query
mjc 6652845
Stream worker media chunks
mjc 4920717
Tighten worker websocket rollout
mjc 13db1db
Address distributed worker review feedback
mjc 898ff54
Show worker session state
mjc e0d1ad6
Add distributed worker controls
mjc c8a6736
Show worker token on dashboard
mjc e945142
Enable worker socket by default
mjc 7f80c38
Use binary frames for worker transfers
mjc c0824e6
Show worker progress telemetry
mjc addaf47
Accept worker transfer progress
mjc b8c0f0c
Show active worker transfer progress
mjc b4e8f1b
Reuse CRF dashboard panel for workers
mjc 77001bd
Fix worker CRF resume state
mjc 818d088
Seed worker transfer progress on start
mjc cb0e2bc
Stabilize worker reconnect state
mjc 03ebce2
Send CRF rules to websocket workers
mjc c613cd1
Resend worker input on request
mjc 5a86b56
Resend worker input before progress
mjc f728171
Accept worker CRF result payloads
mjc 3446411
Persist worker CRF results
mjc b776e3b
Accept dispatched worker results after reconnect
mjc ac0ad84
Stabilize CRF retry test
mjc bd0aad7
Stop resending active worker input on reconnect
mjc 621a4c0
Stop guessing CRF completion state
mjc d95797c
Fix CRF completion session cleanup
mjc 50e3c97
Clear completed worker sessions
mjc c9aaf83
Harden worker CRF session cleanup
mjc fec98a2
more worker page state bugs
mjc e5207d8
trying to fix worker loop
mjc 7860503
Fix crf transfer resume by resending assigned input on unknown transf…
mjc 8aab277
Fix worker reconnect resume to avoid missing input loop
mjc d9d22c0
Fix worker reconnect transfer resend decision
mjc File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -46,3 +46,5 @@ config/dev.overrides.exs | |
| .expert/ | ||
|
|
||
| .envrc | ||
|
|
||
| .serena/cache/ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| # Conventions | ||
|
|
||
| - Use existing state-machine helpers for video state transitions when available. | ||
| - Treat `:encoded` videos as protected during sync; do not reset encoded state or discard chosen VMAF/original-size data. | ||
| - `Reencodarr.AbAv1.OutputParser` is the shared parser for ab-av1 output; keep encode and CRF-search parsing aligned there. | ||
| - `Reencodarr.Rules.build_args/4` is the central encode/CRF-search argument builder. | ||
| - Service clients live under `lib/reencodarr/services/` and follow existing CarReq retry/fuse patterns. | ||
| - Paginated list LiveViews use URL params plus `ReencodarrWeb.Live.FlopList`; list APIs return `{items, %Flop.Meta{}}`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| # Core | ||
|
|
||
| - Phoenix app for Reencodarr media transcoding; web modules under `lib/reencodarr_web`, domain modules under `lib/reencodarr`. | ||
| - SQLite via `Reencodarr.Repo`; keep DB tuning centralized in `config/config.exs`. | ||
| - Broadway/local worker supervisors are disabled in tests via `config :reencodarr, env: :test` and `Application.worker_children/0`. | ||
| - Existing app invariants and live-debug commands live in `AGENTS.md`; prefer repo evidence, logs, diagnostics, and tests over guessing. | ||
|
|
||
| Related memories: commands in `mem:suggested_commands`, done gates in `mem:task_completion`, style in `mem:conventions`, stack in `mem:tech_stack`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| # Memory Maintenance | ||
|
|
||
| ## Discovery Model | ||
|
|
||
| - Core principle: progressive discovery through references, building a graph of memories. | ||
| - Initially, agents are provided with the list of all memories (names only). | ||
| - Agents should read `mem:core` as the top-level entry point (graph root). | ||
| This memory should contain references to other memories covering major project domains. | ||
| The referenced memories shall, in turn, shall contain references to even more specific memories, and so on. | ||
| The depth of the graph shall depend on the project complexity. | ||
| - Use topics/folders to group related memories in order to make the content structure explicit. | ||
| Folders can mirror project structure (e.g. modules like frontend/backend) or topics like debugging, architecture, etc. | ||
| - Memory references must use a mem: prefix inside backticks, e.g. `mem:frontend/core`. | ||
| The surrounding text should clearly indicate when to read the memory/which content to expect. | ||
| The text should provide more precise guidance than the memory name alone, | ||
| i.e. avoid a reference like "frontend debugging: `mem:frontend/debugging` and instead make clear which aspects of frontend debugging are covered. | ||
| - Memories themselves should not contain information about when to read them; this is the responsibility of the referring memory. | ||
|
|
||
| ## Style | ||
|
|
||
| Dense agent notes, not prose docs. Prefer invariants, terse bullets. | ||
| Avoid obvious context, rationale, and examples unless they prevent likely mistakes. | ||
| Keep guidance durable and generalizable, not task-local. | ||
|
|
||
| ## Add/update threshold | ||
|
|
||
| Add or update memories only with stable, non-obvious project conventions that avoid complex rediscovery in the future. | ||
| Do not add: quick-read facts; generic language/framework knowledge; one-off task notes; volatile line-level details; behavior likely to change soon. | ||
|
|
||
| ## Maintenance Actions | ||
|
|
||
| - Renaming memories: References are updated automatically if handled via Serena's memory rename tool. | ||
| - Checking for stale memories (e.g. after deletion): Call `serena memories check` for a report. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| # Suggested Commands | ||
|
|
||
| - `nix develop -c mix setup` installs deps, creates/migrates DB, and builds assets. | ||
| - `nix develop -c mix test` runs the full test suite. | ||
| - `nix develop -c mix test path/to/test.exs` runs focused tests. | ||
| - `nix develop -c mix compile --warnings-as-errors` catches compile regressions. | ||
| - `nix develop -c mix credo --strict` runs lint. | ||
| - `nix develop -c mix format` formats; `nix develop -c mix format --check-formatted` verifies formatting. | ||
| - `bin/rpc 'Reencodarr.Diagnostics.status()'` and sibling diagnostics helpers inspect the running system. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| # Task Completion | ||
|
|
||
| - Prefer focused tests while iterating, then run broader gates when practical. | ||
| - Standard finish gates: `nix develop -c mix test`, `nix develop -c mix compile --warnings-as-errors`, `nix develop -c mix credo --strict`, and `nix develop -c mix format --check-formatted`. | ||
| - Run `nix develop -c mix format` before committing Elixir changes if formatting may have changed. | ||
| - For live runtime/debug claims, verify with `bin/rpc` diagnostics rather than inferring from source. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| # Tech Stack | ||
|
|
||
| - Elixir `>= 1.20.0 and < 1.21.0` Phoenix app. | ||
| - Phoenix `~> 1.8.8`, LiveView `~> 1.2.5`, Bandit `~> 1.12`. | ||
| - SQLite through `ecto_sqlite3`/`exqlite`; Ecto SQL `~> 3.14`. | ||
| - Uses Nix dev shell for repo commands: `nix develop -c ...`. | ||
| - Assets use Tailwind package `~> 0.5.1` and esbuild `~> 0.10` via Mix aliases. | ||
| - Test helpers include ExUnit, ConnCase/DataCase, StreamData, Floki/LazyHTML, and meck where existing tests mock external commands. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.